# Tidal Forces in Kottler Spacetimes

V.P. Vandeev∗, A.N. Semenova+

###### Abstract

The article considers tidal forces in the vicinity of the Kottler black hole.
We find a solution of the geodesic deviation equation for radially falling
bodies, which is determined by elliptic integrals. And also the asymptotic
behavior of all spatial geodesic deviation vector components were found. We
demonstrate that the radial component of the tidal force changes sign outside
the single event horizon for any negative values of the cosmological constant,
in contrast to the Schwarzschild black hole, where all the components of the
tidal force are sign-constant. We also find the similarity between the Kottler
black hole and the Reissner – Nordström black hole, because we indicate the
value of the cosmological constant, which ensures the existence of two
horizons of the black hole, between which the angular components of the tidal
force change sign. It was possible to detect non-analytical behavior of
geodesic deviation vector components in Anti-de Sitter spacetime and to
describe it locally.

∗BALTIC STATE TECHNICAL UNIVERSITY «VOENMEH» named after D.F. Ustinov, 1-Ya
Krasnoarmeyskaya Ulitsa, 1, St Petersburg, 190005, Russia

+Petersburg Nuclear Physics Institute of National Research Centre ‘‘Kurchatov
Institute’’, Gatchina, 188300, Russia

##  1 Introduction

The physical properties of black holes are still very interesting for the
scientific community, despite the fact that for the first time a black hole,
as a solution to Einstein’s equations, was discovered in GR more than a
hundred years ago. That was the Schwarzschild solution [1], which has a single
parameter - the mass of the black hole. Subsequently, many solutions of
Einstein’s equations appeared, which generalized Schwarzschild’s solution:
Kerr (rotating) black hole [2], Reissner – Nordström (electrically charged)
black hole [3], [4], Kerr – Newman (charged and rotating) black hole [5].
However, non-vacuum solutions are of particular interest. And the simplest
solution of this type is the Kottler metric [6], which solves the equations of
general relativity when the cosmological constant plays the role of matter. It
is also called the Schwarzschild – De Sitter metric, and its physical [7] and
geometric [8] properties are actively explored.

In this article, we focus on the studying of tidal forces in the Kottler
metric, like articles [9] and [10], which discuss the same issues only for
Reisnner – Nordström and Kiselev black holes, respectively. General spherical
symmetry spaces are considered in [11]. The tidal effect in spaces with
unusual regular black hole metrics and Schwarzschild metric with massive
graviton is considered in [12] and [13]. It is well known that in
Schwarzschild spacetime a test body falling towards the event horizon of BH
experiences stretching in the radial direction and compression in the both
angular directions [14]. However, the presence of matter, which is represented
by a cosmological constant, significantly changes the behavior of tidal forces
in comparison with empty space. The most important differences are that at
infinity all the components of the tidal forces are nonzero and also they
cease to be a constant sign. At certain points of Kottler metric, the tidal
forces in the radial or angular direction change their sign unlike in
Schwarzschild metric. The paper is organized as follows. In Sec. 2 we discuss
properties of Kottler spacetime. We analyze geodesics in Kottler metric in
Sec. 3. And study tidal forces of such black holes in Sec. 4. In Sec. 5 we
obtain the solutions of the geodesic deviation equations. Then in Sec. 6. we
provide an asymptotic analysis of solutions in the vicinity of a physical
singularity and infinitely far from it. And Sec. 7 includes an analysis of
geodesic deviation equations solutions in the Schwarzschild – Anti-de Sitter
spacetime, there we manage to show the presence of non-analytical behavior of
all deviation vector components at large distances from the black hole
horizon. We use the metric signature $(+,-,-,-)$ and set the speed of light
$c$ and Newtonian gravitational constant $G$ to $1$ throughout this paper.

##  2 Kottler Black Holes

In this paper, we investigate the properties of a static solution of the
Einstein equation in the presence of cosmological constant

$R_{\mu\nu}-\frac{R}{2}g_{\mu\nu}+{\Lambda}g_{\mu\nu}=0.$ (1)

The line element of a static black hole in the presence of constant energy
density is given by [6]

$ds^{2}=g_{\mu\nu}dx^{\mu}dx^{\nu}=f(r)dt^{2}-\frac{dr^{2}}{f(r)}-r^{2}\left(d\theta^{2}+\sin^{2}{\theta}d\varphi^{2}\right),$
(2)

with

$f(r)=1-\frac{2M}{r}-\frac{{\Lambda}r^{2}}{3},$ (3)

where $M$ is mass of a black hole and $\Lambda$ is cosmological constant. To
find the radius of the event horizon of a black hole, we find solutions of the
equation $f(r)=0$. Passing to the dimensionless parameters $\rho=\frac{r}{M}$
and $\lambda={\Lambda}M^{2}$ horizon equation takes the form

$f(\rho)=\frac{-{\lambda}\rho^{3}+3\rho-6}{3\rho}=0.$ (4)

There are four possible cases of the structure of the Kottler spacetime. They
are described in [15] and come down to:

  1. 1.

For ${\Lambda}M^{2}\leq 0$ we have single event horizon $R_{\Lambda}$, which
matches with the Schwarzschild horizon $R_{s}=2M$ at ${\Lambda=0}$.

  2. 2.

For $0<{\Lambda}M^{2}<\frac{1}{9}$ there are two horizons of the black hole,
which we call $R_{+}$ and $R_{-}$.

  3. 3.

For ${\Lambda}M^{2}=\frac{1}{9}$ there is only one critical horizon
$R_{c}=3M$.

  4. 4.

For ${\Lambda}M^{2}>\frac{1}{9}$ black hole does not have any horizons.

Using the Cardano formula [16] we obtain dimensionless positive roots of (4)

$\rho_{\Lambda}=\frac{R_{\Lambda}}{M}=\sqrt[3]{-\frac{3}{\lambda}+\frac{1}{\lambda}\sqrt{9-\frac{1}{\lambda}}}+\sqrt[3]{-\frac{3}{\lambda}-\frac{1}{\lambda}\sqrt{9-\frac{1}{\lambda}}},\\\
$ (5)
$\left[\begin{gathered}\rho_{-}=\frac{R_{-}}{M}=\frac{2}{\sqrt{\lambda}}\cos{\left(\frac{\arccos{\left(-3\sqrt{\lambda}\right)}+4\pi}{3}\right)},\\\
\rho_{+}=\frac{R_{+}}{M}=\frac{2}{\sqrt{\lambda}}\cos{\left(\frac{\arccos{\left(-3\sqrt{\lambda}\right)}}{3}\right)},\end{gathered}\right.$
(6)

$\rho_{c}=\frac{R_{c}}{M}=3.$ (7)

As a result, we have the opportunity to build a dependence of the radius of
the event horizon of a black hole on the magnitude of the cosmological
constant, the value of which is presented in Fig. 1. Where the square and
circle indicate $R_{s}$ and $R_{c}$ correspondingly.

Figure 1: Event horizon radius $\rho_{h}=\frac{R_{h}}{M}$ dependence on
cosmological constant $\lambda=\Lambda M^{2}$.

Further in the article, we will call an arbitrary black hole horizon $R_{h}$
(or its dimensionless analog $\rho_{h}=\frac{R_{h}}{M}$) if we are not
interested in the specific properties of the horizon or horizons.

##  3 Radial geodesics in Kottler Spacetime

In radial motion, there are two meaningful geodesic equations [17] for the
time and radial variables

$\frac{dt}{d\tau}=\frac{E}{f(r)},$ (8)
$\left(\frac{dr}{d\tau}\right)^{2}=E^{2}-f(r).$ (9)

where $d\tau$ is the proper time. And it should be noted that the dynamics in
angular variables is trivial because we study radial geodesics and therefore
we set the angular momentum to zero. For the radial infall of a test particle
from rest at position $b$, we obtain $E^{2}=f(b)$ from $\frac{dr}{d\tau}=0$
[18].

By defining the ’’Newtonian radial acceleration’’ [19] as
$A^{r}\equiv\ddot{r}$, we find from Eq. (9) that

$A^{r}=-\frac{f^{\prime}(r)}{2},$ (10)

where the prime denotes the differentiation with respect to the radial
coordinate $r$. For Kottler metric this becomes

$A^{r}=-\frac{M}{r^{2}}+\frac{{\Lambda}r}{3}.$ (11)

Where it can be seen that a negative value of $\Lambda$ gives us additional
attraction to a black hole, and for positive $\Lambda$, regions of attraction
and repulsion appear. These areas are separated by the surface of a sphere of
radius $r_{eq}$ which is the solution of equation $A^{r}(r_{eq})=0$

$r_{eq}=\sqrt[3]{\frac{3M}{\Lambda}}.$ (12)

It is interesting to note that the test particle falling freely from rest at
$r=b>R_{h}$ can turn back at $R^{stop}$. The radius $R^{stop}$ can easily be
found as a root of $E^{2}=f(r)$ as

$R^{stop}=-\frac{b}{2}+\sqrt{\frac{b^{2}}{4}+\frac{6M}{{\Lambda}b}},$ (13)

which is real and positive when ${\Lambda}>0$. It should be noted that for
$\Lambda=0$ (Schwarzschild metric) $R_{stop}$ is absent. Eq. (13) allows us to
compare points $R_{stop}$ and $b$:

$\left[\begin{aligned} &R^{stop}\geq
b\>\text{at}\>\Lambda\in\bigg{(}0,\frac{3M}{b^{3}}\bigg{]},\\\
&R^{stop}<b\>\text{at}\>\Lambda\in\bigg{(}\frac{3M}{b^{3}},+\infty\bigg{)}.\end{aligned}\right.$
(14)

We are interested in case where $R_{stop}<b$ because a turn of test particle
is possible in $R_{stop}$ point.

##  4 Tidal forces in Kottler Spacetime

###  4.1 Geodesic deviation equation

Now let us consider tidal forces in the Kottler metric. As is well known [14],
the equation for the spacelike components of the geodesic deviation vector
$\tilde{\xi}^{\mu}$ that describes the distance between two infinitesimally
close particles in free fall is given by

$\frac{D^{2}\tilde{\xi}^{\mu}}{d\tau^{2}}=R^{\mu}_{\nu\alpha\beta}u^{\nu}u^{\alpha}\tilde{\xi}^{\beta},$
(15)

where $u^{\nu}$ is the unit vector of 4-velocity tangent to the geodesic. The
tetrad basis for radial free-fall reference frames have form:

$e_{t}^{\mu}=\left(\frac{E}{f},\sqrt{E^{2}-f},0,0\right),$ (16a)
$e_{r}^{\mu}=\left(-\frac{\sqrt{E^{2}-f}}{f},-E,0,0\right),$ (16b)
$e_{\theta}^{\mu}=\left(0,0,\frac{1}{r},0\right),$ (16c)
$e_{\varphi}^{\mu}=\left(0,0,0,\frac{1}{r\sin{\theta}}\right).\\\ $ (16d)

where $e^{\mu}_{\alpha}$ satisfy normalization condition
$e^{\mu}_{\alpha}\>e^{\nu}_{\beta}\>g_{\mu\nu}=\eta_{\alpha\beta}$ and
$\eta_{\alpha\beta}$ is Minkowski metric. The geodesic deviation vector can be
submitted as

$\tilde{\xi}^{\mu}=e^{\mu}_{\nu}\>\xi^{\nu}.$ (17)

The nonzero components of the Riemann tensor are calculated by the Kottler
metric from expression (2) and have the form

$R^{0}_{.101}=-\frac{f^{\prime\prime}}{2f},\\\ $ (18a)
$R^{0}_{.202}=R^{1}_{.212}=-\frac{f^{\prime}r}{2},\\\ $ (18b)
$R^{0}_{.303}=R^{1}_{.313}=-\frac{f^{\prime}r\sin^{2}{\theta}}{2},\\\ $ (18c)
$R^{2}_{.323}=\left(1-f\right)\sin^{2}{\theta}.$ (18d)

Using these components in Eq. (15) and linear transformation from Eq. (17), we
find the equations for tidal forces in free-fall reference frames:

$\ddot{\xi}^{r}=-\frac{f^{\prime\prime}}{2}\xi^{r}=\left(\frac{2M}{r^{3}}+\frac{\Lambda}{3}\right)\xi^{r},$
(19)
$\ddot{\xi}^{\theta}=-\frac{f^{\prime}}{2r}\xi^{\theta}=\left(-\frac{M}{r^{3}}+\frac{\Lambda}{3}\right)\xi^{\theta},$
(20)
$\ddot{\xi}^{\varphi}=-\frac{f^{\prime}}{2r}\xi^{\varphi}=\left(-\frac{M}{r^{3}}+\frac{\Lambda}{3}\right)\xi^{\varphi}.$
(21)

This result for $\Lambda=0$ corresponds to the Schwarzschild case presented in
[14]. It is worth noting that Eq. (15) have four components but we have given
expressions only for three: $\xi^{r}$, $\xi^{\theta}$, $\xi^{\varphi}$. The
reason for this is that the spectrum of
$R^{\mu}_{\nu\alpha\beta}u^{\nu}u^{\alpha}$ has one zero eigenvalue, which
gives the insignificant equation $\ddot{\xi}^{t}=0$. We see that the tidal
forces in this spacetime depend on the mass of a black hole and value of
cosmological constant. We also see that components of tidal force may vanish,
in contrast to what happens in the Schwarzschild spacetime
$\left(\Lambda=0\right)$. Below we consider the Eqs. (19), (20), (21) in more
detail.

Eq. (19) gives us zero radial component of tidal force at $r=R^{r}_{0}$:

$\displaystyle R^{r}_{0}=-\sqrt[3]{\frac{6M}{\Lambda}},\text{or in terms of
($\lambda$, $\rho$)}:\rho_{0}^{r}=-\sqrt[3]{\frac{6}{\lambda}},$ (22)

which is positive when $\lambda<0$. This means that for a negative
cosmological constant, the test body inside the sphere with a radius
$R^{r}_{0}$ undergoes tidal stretch, and in the outside of the sphere with a
radius $R^{r}_{0}$ it experiences tidal compression along the radial
direction. For a positive cosmological constant, the test body is subjected
only to radial tidal stretch.

From Eq. (20) and (21) we get zero of angular components of tidal force at
$r=R^{a}_{0}$:

$R^{a}_{0}=\sqrt[3]{\frac{3M}{\Lambda}},\text{or in terms of ($\lambda$,
$\rho$)}:\rho_{0}^{a}=\sqrt[3]{\frac{3}{\lambda}},$ (23)

which is positive when $\lambda>0$. This means that for a positive
cosmological constant, the test body inside the sphere with a radius
$R^{a}_{0}$ undergoes tidal angular (transverse) compression, and outside the
sphere with a radius $R^{a}_{0}$ it experiences tidal stretch along the
transverse direction. For a negative cosmological constant, the test body is
subjected only to angular (transverse) tidal compression.

It is interesting to compare our results for Kottler metric with results for
Reissner – Nordström metric because both these spacetimes are the simplest
generalization of Schwarzschild metric and have spherical symmetry. The
presence of the Reissner – Nordström black hole charge causes all components
of the tidal forces to change sign as well in Kottler case. Unlike Reissner –
Nordström spacetime [9], all components of tidal force in Kottler metric have
no extremes and do not change their monotonicity.

Figure 2: Comparison of the equilibrium radii $\rho_{0}^{a}$, $\rho_{0}^{r}$
and horizons of the Kottler black hole.

In Fig. 2 we can see $R^{r}_{0}>R_{\lambda}$ for all negative values of the
cosmological constant, and for value range of cosmological constant where two
horizons exist: $R_{-}<R^{a}_{0}<R_{+}$.

###  4.2 Radial tidal force

In Fig. 3 we plot the radial tidal force given by Eq. (19) for Kottler black
holes for different values of the cosmological constant. The presence of a
nonzero $\Lambda$-term does not change the monotonicity of the dependence of
tidal force on $\rho=\frac{r}{M}$, it changes only the asymptotic behavior at
infinitely large distances. The radial component of the tidal force does not
vanish at infinity. At short distances, tidal forces increase as in the case
of the Schwarchild black hole ($\Lambda=0$). It illustrates that the radial
tidal force changes (for $\Lambda<0$) sign always outside the event horizon,
such $R^{r}_{0}>R_{h}$.

Figure 3: Radial tidal force for Kottler BH with different choices of
$\lambda$. We have chosen $b=10M$.

The scale of Fig. 3 does not allow us to see $R_{stop}$ points on it.
Therefore in Fig. 4 the same force is shown but on a different scale. And for
any positive value of $\lambda$ the radial component of the tidal force is
positive for any $r$. Also all points $R_{stop}$, where the radial component
of the velocity of a freely falling body is zero, are below the event horizon
$R_{stop}<R_{h}$.

Figure 4: Radial tidal force for Kottler BH with different choices of
$\lambda$ on a small scale. We have chosen $b=10M$.

But this scale makes all curves on Fig. 4 merge into one.

###  4.3 Angular tidal force

In Fig. 5 we plot the angular tidal force given by Eq. (20) or Eq. (21) (in a
spherically symmetric problem azimuthal and polar angles are equal) for
Kottler black holes for different values of the cosmological constant. In this
graph, we see that the angular component of the tidal force changes sign on
the event horizon at a critical value of the cosmological constant
$\lambda=\frac{1}{9}$. And for any $\lambda\in\left(0,\frac{1}{9}\right)$ the
tidal force changes sign between the two black hole horizons. But at any
negative values of $\lambda$, this component of the tidal force does not
change sign.

Figure 5: Angular tidal forces for Kottler BH with different choices of
$\lambda$. We have chosen $b=10M$.

In Fig. 6 we show the same force but on a different scale. These curves
demonstrate that the stopping points $R_{stop}$ are inside the event horizon,
as well as for the radial component. This is another common property of the
Kottler and Reissner – Nordström metrics. But this scale makes all curves on
Fig. 6 merge into one.

Figure 6: Angular tidal forces for Kottler BH with different choices of
$\lambda$ on a small scale. We have chosen $b=10M$.

##  5 Solutions of the Geodesic Deviation Equations in Kottler Spacetime

In this section we solve deviation equations (19), (20) and (21) and find the
geodesic deviation vectors for radially free-falling geodesics as functions of
$r$. We consider a test body infalling radially towards the Kottler black
hole. It is simple to convert Eqs. (19), (20) and (21) to differential
equations in $r$ by using Eq. (9). Thus, we find

$\left(E^{2}-f\right)\frac{d^{2}\xi^{r}}{dr^{2}}-\frac{f^{\prime}}{2}\frac{d\xi^{r}}{dr}+\frac{f^{\prime\prime}}{2}{\xi}^{r}=0,$
(24)
$\left(E^{2}-f\right)\frac{d^{2}\xi^{a}}{dr^{2}}-\frac{f^{\prime}}{2}\frac{d\xi^{a}}{dr}+\frac{f^{\prime\prime}}{2r}{\xi}^{a}=0,$
(25)

where $a=\theta,\varphi$ means angular variables, because Eqs. (20) and (21)
have the same structure due to spherical symmetry of the problem and prime
means the derivative with respect to $r$. In terms of dimensionless variables
$\rho$ and $\lambda$ Eqs. (24) and (25) look accordingly

$\left(E^{2}-1+\frac{2}{\rho}+\frac{\lambda}{3}\rho^{2}\right){\xi^{r}}^{\prime\prime}+\left(-\frac{1}{\rho^{2}}+\frac{\lambda}{3}\rho\right){\xi^{r}}^{\prime}-\left(\frac{2}{\rho^{3}}+\frac{\lambda}{3}\right){\xi^{r}}=0,$
(26)
$\left(E^{2}-1+\frac{2}{\rho}+\frac{\lambda}{3}\rho^{2}\right){\xi^{a}}^{\prime\prime}+\left(-\frac{1}{\rho^{2}}+\frac{\lambda}{3}\rho\right){\xi^{a}}^{\prime}-\left(-\frac{1}{\rho^{3}}+\frac{\lambda}{3}\right){\xi^{a}}=0,$
(27)

where prime means the derivative with respect to dimensionless $\rho$. The
analytical solutions of Eqs. (26) and (27) can be expressed as quadratures.
For the radial component we have

$\xi^{r}(\rho)=\sqrt{E^{2}-1+\frac{2}{\rho}+\frac{{\lambda}}{3}\rho^{2}}\left[A+B\int\frac{d\rho}{\left(E^{2}-1+\frac{2}{\rho}+\frac{{\lambda}}{3}\rho^{2}\right)^{\frac{3}{2}}}\right],$
(28)

and for the angular components we have

$\xi^{a}(\rho)=\rho\left[C+D\int\frac{d\rho}{\rho^{2}\sqrt{E^{2}-1+\frac{2}{\rho}+\frac{{\lambda}}{3}\rho^{2}}}\right],$
(29)

where $A$, $B$, $C$ and $D$ are constants of integration. However, the
integrals in Eqs. (28) and (29) are elliptic, which significantly complicates
the analysis of these functions.

Fig. 6 and Fig. 7 show the numerical solutions of differential equations (26)
and (27), respectively. They present the dependence of the component of the
geodesic deviation vector on the dimensionless radial variable $\rho$ for
different values of $\lambda$.

Figure 7: Radial components of the geodesic deviation vector for several
values of $\lambda$. We have chosen $b=10M$ and $E=10$, initial conditions:
$\xi^{r}(b)=1$, $\xi^{r}{}^{\prime}(b)=0$.

The behavior of the radial component of the deviation vector in the Kottler
spacetime in the vicinity of the physical singularity is similar to the
dependence of the radial component of the deviation vector in the
($\Lambda=0$). It aims at infinity for $\rho\to 0$. However, far from the
center of black hole, the influence of the cosmological constant begins to
dominate, therefore, infinitely far from the black hole, the radial geodesic
deviation in the Kottler metric differs significantly from asymptotically flat
Schwarzschild and Reissner – Nordström [9] metrics. For $\Lambda>0$ $\xi^{r}$
grows linearly at $\rho\to\infty$. Case of negative $\lambda$ requires a
separate study because the denominator of the integrand from (28) vanishes.

Figure 8: Angular components of the geodesic deviation vector for several
values of $\lambda$. We have chosen $b=10M$ and $E=10$, initial conditions:
$\xi^{a}(b)=1$, $\xi^{a}{}^{\prime}(b)=0$.

Angular components $\xi^{a}$, where $a=\theta,\varphi$, of deviation vector
behave like a radial component at spatial infinity. In the vicinity of the
center of the black hole they are finite for all values $\lambda$ in contrast
to the radial component that is similar to case of Reissner – Nordström
spacetime [9].

##  6 Behavior of solutions near singularity and infinity

In this section, we give a local description of solutions to the Eqs. (26) and
(27). These are linear homogeneous differential equations with variable
coefficients. Since we have elliptic integrals in Eqs. (28) and (29), so we
would like to describe the local behavior of solutions in the surrounding area
of the singularity of a black hole and infinitely far from it. We show how
expressions (28) and (29) behave for $\rho\to 0$ and $\rho\to\infty$.

###  6.1 Radial component

To determine the behavior of $\xi^{r}$ for small or large $\rho$ let us
consider Taylor series at $\rho\to 0$ and $\rho\to\infty$ (or
$\frac{1}{\rho}\to 0$) respectively. In Eq. (28), the root and the integrand
are expanded in a series

$\xi^{r}(\rho)=\left(\sqrt{\frac{2}{\rho}}+\frac{\sqrt{2}}{4}\left(E^{2}-1\right)\sqrt{\rho}+O\left(\rho^{\frac{3}{2}}\right)\right)\times\\\
\times\left[A+B\int\left\\{\frac{\sqrt{2}}{4}\rho^{\frac{3}{2}}-\frac{3\sqrt{2}}{16}\left(E^{2}-1\right)\rho^{\frac{5}{2}}+O\left(\rho^{\frac{7}{2}}\right)\right\\}d\rho\right]\;\text{for
$\rho\to 0$},$ (30a)
$\xi^{r}\left(\rho\right)=\left(\sqrt{\frac{\lambda}{3}}\rho+\sqrt{\frac{3}{\lambda}}\frac{E^{2}-1}{2\rho}+O\left(\frac{1}{\rho^{2}}\right)\right)\times\\\
\times\left[A+\left(\frac{3}{\lambda}\right)^{\frac{3}{2}}B\int\left\\{\frac{1}{\rho^{3}}+\frac{3}{\lambda}\frac{E^{2}-1}{2\rho^{5}}+O\left(\frac{1}{\rho^{6}}\right)\right\\}d\rho\right]\;\text{for
$\rho\to\infty$}.$ (30b)

After integrating the sum of power functions and multiplying brackets in both
expressions (30a) and (30b) we get

$\xi^{r}(\rho)=\frac{\sqrt{2}A}{\sqrt{\rho}}+\frac{\sqrt{2}A\left(E^{2}-1\right)}{4}\sqrt{\rho}+O\left(\rho^{\frac{3}{2}}\right)\;\text{for
$\rho\to 0$},$ (31a)
$\xi^{r}\left(\rho\right)=\sqrt{\frac{\lambda}{3}}A\rho+\left[\sqrt{\frac{3}{\lambda}}\frac{A\left(E^{2}-1\right)}{2}-\frac{3B}{2\lambda}\right]\frac{1}{\rho}+O\left(\frac{1}{\rho^{2}}\right)\;\text{for
$\rho\to\infty$}.$ (31b)

From Eq. (31a) we see that near the black hole singularity the radial
component of deviation vector tends to infinity as $\frac{1}{\sqrt{\rho}}$,
whereas according to Eq. (31b) at spatial infinity it grows linearly only for
positive $\lambda$. The asymptotic behavior of $\xi^{r}(\rho)$ from (31a) and
(31b) completely coincides with the numerical solution shown in Fig. 7, except
for the case when $\lambda<0$. Numerical solutions from Fig. 7 correspond to
(31a) and (31b) in the indicated areas of $\rho$.

###  6.2 Angular components

A similar analysis of Eq. (29) in the same way leads first to

$\xi^{a}(\rho)=\rho\left[C+D\int\left(\frac{\rho^{-\frac{3}{2}}}{\sqrt{2}}-\frac{E^{2}-1}{4\sqrt{2}}\rho^{-\frac{1}{2}}+O\left(\sqrt{\rho}\right)\right)d\rho\right]\;\text{for
$\rho\to 0$},$ (32a) $\xi^{a}(\rho)=\rho C+\rho
D\sqrt{\frac{3}{\lambda}}\int\left\\{\frac{1}{\rho^{3}}-\frac{3}{\lambda}\frac{E^{2}-1}{2\rho^{5}}+O\left(\frac{1}{\rho^{6}}\right)\right\\}d\rho\;\text{for
$\rho\to\infty$},$ (32b)

and after integration and multiplication to

$\xi^{a}(\rho)=-\sqrt{2}D\sqrt{\rho}+C\rho+O\left(\rho^{\frac{3}{2}}\right)\;\text{for
$\rho\to 0$},$ (33a)
$\xi^{a}(\rho)=C\rho-\sqrt{\frac{3}{\lambda}}\frac{D}{2\rho}+O\left(\frac{1}{\rho^{3}}\right)\;\text{for
$\rho\to\infty$}.$ (33b)

Eq. (33a) means the angular components of the deviation vector near the black
hole singularity grows as $\sqrt{\rho}$ (this is different from Eq. (31a)),
while the linear growth at infinity in Eq. (33b) is similar Eq. (31b), despite
the absence of a $\lambda$ leading order. The asymptotic behavior of
$\xi^{a}(\rho)$ from (33a) and (33b) completely coincides with the numerical
solution shown in Fig. 8, except for the case when $\lambda<0$. Numerical
solutions from Fig. 8 correspond to (33a) and (33b) in the indicated areas of
$\rho$.

##  7 Separately about Anti-de Sitter

Now let us consider the Anti-de Sitter spacetime where $\lambda<0$
($\lambda=-|\lambda|$). In this case we can see that the denominator of
expressions (28) and (29) can vanish.

$E^{2}-1+\frac{2}{\rho}-\frac{{|\lambda|}}{3}\rho^{2}=0.$ (34)

Eq. (34) reduces to the cubic equation

$\rho^{3}-\frac{3(E^{2}-1)}{|\lambda|}\rho-\frac{6}{|\lambda|}=0.$ (35)

Using the Cardano formula we get

$\rho_{0}=\sqrt[3]{\frac{3}{|\lambda|}+\frac{3}{|\lambda|}\sqrt{1-\frac{(E^{2}-1)^{3}}{9|\lambda|}}}+\sqrt[3]{\frac{3}{|\lambda|}-\frac{3}{|\lambda|}\sqrt{1-\frac{(E^{2}-1)^{3}}{9|\lambda|}}}.$
(36)

The nature of the roots of this equation depends on the sign of the expression
$1-\frac{(E^{2}-1)^{3}}{9|\lambda|}$:

  1. 1.

$1>\frac{(E^{2}-1)^{3}}{9|\lambda|}$ means that Eq. (35) has a single real
positive simple root $\rho_{0}$ described by the Eq. (36).

  2. 2.

$1=\frac{(E^{2}-1)^{3}}{9|\lambda|}$ means that Eq. (35) takes the form

$\rho^{3}-\frac{27}{(E^{2}-1)^{2}}\rho-\frac{54}{(E^{2}-1)^{3}}=\left(\rho-\frac{6}{E^{2}-1}\right)\left(\rho+\frac{3}{E^{2}-1}\right)^{2}=0,$
(37)

which has only one real simple positive root $\rho=\frac{6}{E^{2}-1}$, but the
second root $\rho=-\frac{3}{E^{2}-1}$ is negative what is pointless for us.

  3. 3.

$1<\frac{(E^{2}-1)^{3}}{9|\lambda|}$ means that Eq. (35) has three real root

$\rho=2\sqrt{\frac{E^{2}-1}{|\lambda|}}\cos\left(\frac{\arccos\left(\frac{3\sqrt{|\lambda|}}{(E^{2}-1)^{\frac{3}{2}}}\right)+2\pi
k}{3}\right),$ (38)

where $k=0,1,2$. And among them, only one root is positive. This is the root
for $k=0$.

Thus, all the cases described above show us that for any $\lambda<0$ there is
simple positive root of Eq. (34). Let us label this single simple root
$\rho_{0}$, so the left side of the Eq. (34) can be represented as

$E^{2}-1+\frac{2}{\rho}-\frac{{|\lambda|}}{3}\rho^{2}\equiv(\rho_{0}-\rho)Q(\rho).$
(39)

Where $Q(\rho)$ is rational function which has no positive roots. Expression
(39) allows us to explore (28) and (29) in the vicinity of $\rho_{0}$.
Representing square roots and integrands as a Taylor series, we can, after
integration, obtain the local behavior of the deviation vector components
$\xi^{r}(\rho)$ and $\xi^{a}(\rho)$

$\xi^{r}(\rho)=\frac{2B}{Q(\rho_{0})}+A\sqrt{(\rho_{0}-\rho)Q(\rho_{0})}+O(\rho_{0}-\rho)\;\text{for
$\rho\to\rho_{0}-0$},$ (40)
$\xi^{a}(\rho)=\rho_{0}C-\frac{2D\sqrt{\rho_{0}-\rho}}{\rho_{0}\sqrt{Q(\rho_{0})}}+O\left(\rho_{0}-\rho\right)\;\text{for
$\rho\to\rho_{0}-0$}.$ (41)

Thus, we found a root feature for all components of the deviation vector. If
we extend the numerical solution of Eqs. (26) and (27) shown in Fig. 7 and
Fig. 8 to longer distances we can demonstrate in Fig. 9 and Fig. 10 that for
negative $\lambda$ $\xi^{r}$ and $\xi^{a}$ behave in a root manner.

Figure 9: Radial components of the geodesic deviation vector for several
values of $\lambda$. We have chosen $b=10M$ and $E=10$, initial conditions:
$\xi^{r}(b)=1$, $\xi^{r}{}^{\prime}(b)=0$. Figure 10: Angular components of
the geodesic deviation vector for several values of $\lambda$. We have chosen
$b=10M$ and $E=10$, initial conditions: $\xi^{a}(b)=1$,
$\xi^{a}{}^{\prime}(b)=0$.

Thus Eqs. (40) and (41) accurately describe the behavior of the bottom bold
curves in Fig. 9 and Fig. 10 in the vicinity of the singular points
$\rho=\rho_{0}$. The reason for non-analytical behavior of the components of
the geodesic deviation vector is that geodesic equation (8) for $\lambda<0$
defined only for $\rho\leq\rho_{0}$. It means that negative energy density
leads to tidal compression along all spatial directions at large distances
from the black hole.

##  8 Conclusion

In this article we discuss the Kottler black hole which is solution of
Einstein’s equations when the cosmological constant plays the role of a
matter. We analyze properties of Kottler spacetime and build the dependence of
event horisons radius on cosmological constant value and describe properties
of radial geodesics in Kottler metric. We detected the presence of stopping
points in the process of free movement in the investigated space (13). Using
geodesic deviation equation we study tidal force in Kottler black hole, we
have shown that they can change sign (22) and (23) as in the Reissner –
Nordström spacetime in contrast to the Schwarzschild metric. It was shown that
unlike Schwarzschild spacetime ($\Lambda$ = 0), in the case of Kottler metric
both radial and angular components of tidal force may vanish. But tidal force
in the Kottler and Schwarzschild spaces also have common features different
from charged Reissner – Nordström black hole, they do not have an extrema and
therefore do not change their monotonicity. The nonzero $\Lambda$-term changes
the asymptotic behavior of tidal force at infinitely large distances (19),
(20) and (21): the radial and angular components do not vanish at infinity.
However, at short distances tidal forces increase as in the case of the
Schwarzschild black hole. The analytical solutions of the geodesic deviation
equations expressed as quadratures were founded (28) and (29). For these
analytical solutions, we managed to obtain expressions describing the
asymptotic behavior of the deviation vector components near the singularity of
the black hole (31a), (33a) and at large distances from it (31b), (33b). The
solutions of the geodesic deviation equations in the Schwarzschild – Anti-de
Sitter spacetime (Kottler with $\Lambda<0$) are considered separately. With a
negative value of the cosmological constant, it was possible to detect a root
peculiarity in the behavior of the deviation vector at large distances (40)
and (41) in comparison with the radius of the event horizon. This means that,
in Schwarzschild – Anti-de Sitter spacetime, there is a region of tidal
compression at a great distance from the black hole. Analytical calculations
are confirmed by the numerical solution of the equations for the deviation of
geodesics.

Development of tidal effects studying in general relativity can have several
directions. First of all, it is interesting to consider any combinations of
matter, for example, quintessence and cosmological constant together.
Secondary it can be investigated how tidal forces are affected by the presence
of non-zero angular momentum of the freely falling body. The third direction
in the study of tidal forces near black holes can be the study of the
properties of geodesic deviation equations in multidimensional spacetime. It
is also very interesting to observe how tidal forces behave in axially
symmetric spaces in the presence of matter. A separate completely unexplored
question is the nature of tidal forces in the metrics of black rings. As we
can see, despite the fact that the classical effects in the general theory of
relativity have been studied for more than a hundred years, this issue still
contains a large number of unsolved problems.

## References

  * [1] Schwarzschild K. Berliner Sitzungsbesichte (Phys. Math. Klasse), 189 – 196 3 Feb. (1916).
  * [2] Kerr R. P. Phys. Rev. Lett., 11, 237 – 238 September 1. (1963).
  * [3] Reissner H. Ann. Physik, 50, 106 – 120 (1916).
  * [4] Nordström G. Proc. Kon. Ned. Akad. Wet., 20, 1238 – 1245 (1918).
  * [5] Newman E. T., Couch E., Chindapared K. et al. J. Math. Phys., 6, 918 – 919 (1965).
  * [6] Kottler F. Ann. Physik, 56, 401 – 462 (1918).
  * [7] Pappas T., Kanti P., Phys. Lett. B 775 140 – 146 (2017).
  * [8] Podolsky J. Gen. Rel. Grav. 31 1703 – 1725 (1999).
  * [9] Crispino L.C.B., et al.: Eur. Phys. J. C 76, 168 (2016).
  * [10] Shahzad M. U., Jawad A., Eur. Phys. J. C 77, 372 (2017).
  * [11] Fush H. Ann. Physik, 495, 231 – 233 (1983).
  * [12] Sharif M., Sadiq S., JETP 153, 232 – 239 (2018).
  * [13] Soon-Tae Hong, et al. Phys. Lett. B 881 135967 (2020).
  * [14] Misner C. W., Thorne K. S., Wheeler J. A., Gravitation (San Francisco: W.H. Freeman, 1973).
  * [15] Gibbons G. W., Hawking S. W., Phys. Rev. D 15, 2739 – 2751 (1977).
  * [16] Faddeev D. K. Lektsii po Algebre ("Nauka", Moskva, 1984).
  * [17] S. Chandrasekhar, The Mathematical Theory of Black Holes (Clarendon Press, Oxford, 1983).
  * [18] K. Martel and E. Poisson, Phys. Rev. D 66, 084001 (2002).
  * [19] K. R. Symon, Mechanics (Addison-Wesley Publishing Company, Massachusetts, 1971)


(eccv) Package eccv Warning: Package ‘hyperref’ is loaded with option
‘pagebackref’, which is *not* recommended for camera-ready version

11institutetext: Indian Institute of Technology (IIT) Tirupati, India
11email<EMAIL_ADDRESS><EMAIL_ADDRESS>
# NSD-DIL: Null-Shot Deblurring Using Deep Identity Learning

Sree Rama Vamsidhar S 11 Rama Krishna Gorthi 11

###### Abstract

In this paper, we propose to reformulate the blind image deblurring task to
directly learn an inverse of the degradation model using a deep linear
network. We introduce “Deep Identity Learning (DIL)”, a novel learning
strategy that includes a dedicated regularization term based on the properties
of linear systems, to exploit the identity relation between the degradation
and inverse degradation models. The salient aspect of our proposed framework
is it neither relies on a deblurring dataset nor a single input blurred image
(like Polyblur, a self-supervised method). Since it is purely image-data-
independent, we term our model as “Null-Shot deblurring Using Deep Identity
Learning (NSD-DIL)". We also provide an explicit representation of the learned
deep linear network in a matrix form, called “Deep Restoration Kernel (DRK)"
for deblurring task. The proposed framework detours the typical degradation
kernel estimation step involved in most of the existing blind deblurring
solutions by the proposition of our Random Kernel Gallery (RKG) dataset. In
this work, we focus on the restoration of mild blur images, generated by small
out-of-focus, lens blur, or slight camera motion, which often occurs in real
images. Our experiments show that the proposed method outperforms both
traditional and deep learning based deblurring methods, with at least an order
of 100 lesser computational resources. The proposed NSD-DIL method can be
effortlessly extended to the Image Super-Resolution (ISR) task as well to
restore the low-resolution images with fine details. The NSD-DIL model and its
kernel form representation (DRK) are lightweight yet robust and restore the
mild blur input in a fraction of a second. Hence, more suitable for wide real-
time applications.

###### Keywords:

Convolution Deblurring Deconvolution Degradation kernel

##  1 Introduction

Image deblurring is a well-established low-level vision task whose objective
is to generate a sharp image from the given corresponding blurred input
observation(s). Blur can be caused by various factors like, but not limited
to, camera shake, observed relative motion between camera and target, camera
defocus, camera lens aberration, light diffraction due to finite lens
aperture, integration of light in the sensor [10]. Deblurring is a severely
ill-posed problem as multiple combinations of a sharp image and a blur kernel
can result in the same blurred image. However, real-world applications in
various prominent domains like medical imaging, satellite imaging, and
surveillance demand a sharp version of the scene of interest for its analysis.
Image deblurring solutions can be categorized into non-blind and blind
approaches. Non-blind methods aim to restore clear images from blurred inputs
when the degradation kernel is known. In contrast, blind deblurring methods
work with an unknown degradation kernel. The traditional non-blind and blind
deblurring methods include like Wiener filter [55], Richardson-Lucy algorithm
[43], sophisticated priors [54, 26, 33], Maximum a-Posterior (MAP) approaches
with a variety of data terms [42, 11] and prior terms [23, 48, 57, 65].
However, the optimization problems of these sophisticated model limit their
practical usage. The blind deblurring works either estimate both the blur
kernel and the sharp image simultaneously [33, 24, 4, 38] or the blur kernel
is estimated first and then non-blind deblurring is performed [14, 8, 35]. It
is demonstrated that estimating the blur kernel is easier than jointly
estimating the blur kernel and sharp image together [27]. With the advent and
continuous advancements of Deep Learning (DL) techniques, remarkable
improvement in the restored sharp image quality has been observed. The recent
deblurring solutions, including the pioneering works [50, 34], adopt
sophisticated techniques like Residual blocks [40, 51, 61], Generative
adversarial networks [25, 62], Recurrent Neural Networks [60, 37],
Transformers [30, 59], and specialized loss functions [46, 63].
The major limitation, in non-blind methods is the error-free assumption on the
blur kernel. In practice, the estimated blur kernel is erroneous to some
extent which could cause severe artifacts, and degradations in the generated
images [17, 29]. In the blind deblurring methods, the extensive dependency on
datasets for prior information could limit their generalization ability.
Besides these drawbacks, the present State-of-the-Art (SotA) image deblurring
methods employ DL techniques using a large amount of image data with heavy
computational and time complexities. Hence, not suitable for real-time
applications [10, 7].
In recent literature, it is observed that most of the works were leaning
towards addressing the strong motion and non-uniform blur [29, 41, 25, 18,
56]. However, very little attention was paid to the most common and practical
use case where the blur is small i.e. mild blur that is often present in real
images which degrades the image quality. Mild blur can be harmful to the
performance of other image processing tasks, such as super-resolution. These
tasks often rely on sharp images to produce accurate results. The works [10,
64], detached from the mainstream deblurring methods and were proposed
solutions towards addressing the mild blur. For instance, Polyblur [10] is a
blind and self-supervised approach that uses polynomial functions to restore
image details blurred by mild camera shake or motion blur supporting real-time
applications. It estimates the blur kernel of the image by analyzing the
distribution of the gradient in the image and then applies the kernel multiple
times, based on the severity of the blur.
In this paper, we redefine the task of learning the deblurring model from
image data to merely computing an inverse of the degradation model. We
characterized the degradation model with a wide set of anisotropic Gaussian
kernels perturbed with noise, called the Random Kernel Gallery (RKG) dataset.
Here, we propose a novel learning strategy called “Deep Identity Learning
(DIL)" to compute the inverse of the constructed degradation model i.e. RKG
dataset. DIL exploits the identity relation between the degradation model and
its inverse i.e. $K^{-1}$, deblur model. In this work, a custom lightweight
and computationally efficient CNN with no activation functions, referred to as
Linear-CNN (L-CNN), is trained on the RKG dataset with DIL as its objective.
DIL includes a regularization term based on the frequency and spatial domain
properties of the convolution operator to effectively ensure the identity
system properties of our inverse degradation model. Unlike existing blind
self-supervised deblur frameworks like [10], the proposed deblurring framework
is image-independent and doesn’t require blurred or pair of blurred and sharp
image data at any stage of the deblur task learning. Hence, we term the
proposed model as “Null-Shot image Deblurring using Deep Identity Learning
(NSD-DIL)". In this work, we provide a compact and effective representation of
the proposed L-CNN, in a matrix form, called ”Deep Restoration Kernel (DRK)".
To perform the deblur task, the L-CNN can be replaced with the DRK at the
inference stage. This step transforms the deblur task into a simple
convolution operation between the given blurred input image and the DRK.
Therefore, the deblur task with the DRK further reduces the computational
complexity in terms of time and memory by several orders. The proposed DRK is
the convolution inverse of the degradation model and performs deblur tasks
across a wide range of degradations more effectively. The highlights of our
proposition are as follows.

  1. 1.

To the best of our knowledge, NSD-DIL is the first DL-based image data-
independent deblurring framework.

  2. 2.

We propose a novel learning function i.e. DIL based on identity system
properties with a dedicated regularization term driven by the spatial and
frequency domain inferences to effectively learn an inverse degradation model.

  3. 3.

We provide an effective and explainable, explicit representation in matrix
form, of the learned inverse degradation model (L-CNN), i.e. DRK.

  4. 4.

The proposed NSD-DIL model and the DRK perform superior deblurring, in case of
mild blur, against the existing SotA deblur methods while being at least 100
times more computationally efficient.

  5. 5.

We do demonstrate that the proposed NSD-DIL framework can be readily extended
to the Image Super-Resolution (ISR) task.

The overall organization of the rest of the paper is as follows. Section 2
presents Related work. A detailed explanation of our proposed NSD-DIL method
is given in Section 3. In Section 4, a thorough experimental setup and results
are presented, followed by a Discussion in Section 5 and a Conclusion in
Section 6.

Figure 1: Qualitative results on real blurry inputs captured using a Samsung
J7 prime smartphone camera (13MP, f/1.9).

##  2 Related work

Besides the blind deblurring works mentioned in the Sec 1, the other related
works are as follows.
Image sharpening methods are simple and intuitive to generate sharp images.
These methods share a common framework, the input image is filtered and then a
proportion of the residual image is added back to the original input. This
procedure boosts high-frequencies and increases the contrast in the generated
image. The most well-established works include Unsharp Masking and its
adaptations. These are simple and effective but sensitive to noise and
generate artifacts [39, 45, 21]. Bilateral filter [53] and its variants [15,
3] were used to enhance the local contrast and high-frequency details in an
image. Based on local structure and, sharpness information [64] proposed an
adaptive sharpening method that performs simultaneous noise reduction and
sharpening. However, the deblurring performance of these methods is limited
since they do not explicitly estimate image blur. These methods are strictly
local operators.
Gaussian deblurring, as one of the causes of mild blur i.e. out-of-focus is
usually modeled as a Gaussian function [31]. Most of the Gaussian deblurring
methods [16, 5, 58], estimate the blur kernel (i.e. Gaussian) parameters like
variance, location of mean, and its orientation and then apply non-blind
techniques to perform the deblurring task. However, these methods are
sensitive to noise and require high inference time. Also, the error in the
estimates of the Gaussian kernel parameters leads to over-sharp or over-
smoothed results with artifacts [12].
A line of works [29, 41, 28, 6], achieves uniform and nonuniform deblurring
tasks from the given single test input blurry image through self-supervised
learning. In [41, 28], two neural networks were used to predict the sharp
image and the blur kernel using a total-variation (TV) based MAP estimator.
Bayesian inference implemented via Monte-Carlo sampling and [6] using deep
neural network-based ensemble learning technique respectively for uniform
deblurring. [29] addresses both uniform and nonuniform motion deblurring using
the Monte Carlo expectation maximization algorithm. These methods require
high-inference time and are computationally complex though.
Towards addressing the limitations of data dependency, time, and computational
complexities for the practical use case of real-time implementation, we
proposed a blind lightweight yet robust purely image data-independent model to
restore the distorted image by mild blur.

##  3 Proposed method

In this section, the introduced RKG dataset, the proposed NSD-DIL model,
dedicated Regularization, and explainable DRK are discussed in greater detail.

###  3.1 Problem formulation

In general, the End-to-End Deblurring (E2ED) framework can be observed as the
degradation model followed by its inverse degradation model. Both the input
and output of the E2ED model are sharp images. At first, the degradation model
outputs a degraded version of the sharp image i.e. the blurred image,
depicting the degradation process. Later, the inverse degradation model
restores the sharp image from the obtained blur image i.e. the sharp image,
through the restoration process. The degradation and inverse degradation
models are discussed below.

Figure 2: The training and inference methodology of the proposed NSD-DIL
model.

Degradation model: The degradation model typically represents the convolution
operation between the input sharp image and the degradation kernel ($K$),
which results in a degraded image i.e. blurred image. The mathematical
representation of the correspondence between sharp image $X\in{R}^{M\times N}$
and blur image $y\in{R}^{M\times N}$ is given in Eq. (1), assuming the spatial
invariant blur [27].

$y=X\ast K$ (1)

Here $\ast$ represents the convolution operation.
Inverse degradation model: The objective of the deblurring task is to achieve
the inverse of the degradation model i.e., to reconstruct the sharp image from
the given blurred image input. The mathematical representation of the inverse
degradation model is given in Eq. (2).

$y\ast K^{-1}=X$ (2)

Here $K^{-1}$ represents the inverse degradation kernel.
End-to-End Deblurring (E2ED) model: The unified degradation and inverse
degradation models form an E2ED problem. Here, it is important to observe that
the unified framework represents an “ _identity model_ ” with the same entity
as the input and also as the output i.e. sharp image (in the ideal case). Here
the degradation and inverse degradation models are characterized and
effectively represented by their respective kernels $K$ and $K^{-1}$.
Therefore, for an ideal deblurring, the convolution operation between $K$ and
$K^{-1}$ should result in an identity relation. In practice, the output of the
inverse model is the restored image with high-frequency information from the
blurry input image.
Based on the observed E2ED model, we propose a simple, computationally
efficient, and novel framework that is independent of images to learn the
deblurring task. Here, we propose to learn the inverse degradation model
forming an identity relation with the degradation kernel. The identity
relation is given in Eq. (3). The proposed formulation simplifies the task of
learning the deblurring problem from image datasets using deep architectures
with several millions of learnable parameters, to the task of  “identity”
learning between the degradation kernel $K$ and the inverse degradation kernel
$K^{-1}$ through a custom linear CNN (discussed in Sec.3.3). Hence, we term
the proposed identity learning using a deep model as “ _Deep Identity Learning
(DIL)_ ".

$K\ast K^{-1}=~{}\delta$ (3)

Where $\delta$ is a two-dimensional impulse function of dimension $P\times Q$,
and $P,Q$ values are based on the dimensions of $K$ & $K^{-1}$.

###  3.2 Random Kernel Gallery (RKG) Dataset

To realize the identity learning task, a set of degradation maps sampled from
the degradation kernel space is generated. In deblurring literature, the
degradation modeling is mainly carried out conventionally by using Gaussian
prior [10, 31]. Theoretically, $K$ is modeled as unimodal [32] & Gaussian
[44]. The existing well-known blind Deblurring works like [10] consider the
$K$ to be isotropic or anisotropic Gaussian models. Also, the work [27]
demonstrated that with an appropriate estimation algorithm, blind
deconvolution can be performed even with a weak Gaussian prior. Following
these widely adopted premises, we constructed a set of degradation models with
the anisotropic Gaussian kernels.
In this work, a set of anisotropic Gaussian kernels with dimensions $11\times
11$, having $\sigma_{1},\sigma_{2}\in U[0.175,3]$ and rotation angle
$\theta\in U[0,\pi]$ were randomly generated as representative samples of the
degradation kernel space. The number of $K$ samples required to effectively
learn the inverse degradation kernel $K^{-1}$ is determined empirically as
$2,400$. The reference output to learn the inverse degradation model is the
two-dimensional impulse function mentioned in Eq.3. We term this dataset as
the “Random Kernel Gallery (RKG)” dataset.
The proposed NSD-DIL method is not limited to the “RKG-dataset” with
anisotropic Gaussian kernels. NSD-DIL can be re-trained on any other kernel
distribution if proven to be a more accurate representation of the
degradations kernel space.

###  3.3 Linear Convolutional Neural Network (L-CNN)

Deblurring is an ill-posed inverse problem for which a unique inverse will not
exist. Computing a single-layer network to learn the inverse of the
degradation kernel cannot serve the deblurring task’s objective. This is
because a matrix/single layer accepts only one set of parameters/global minima
with convex loss. Also, the $K$ can usually be a low-rank matrix. Further, it
was empirically found that single-layer architecture does not converge to the
correct solution [9]. Whereas the multi-layered linear networks have many good
and equally valued local minima. This allows many valid optimal solutions to
the optimization objective in the form of different factorizations of the same
matrix [19], [47], [1]. Following these research results, we propose a multi-
layer Linear CNN (L-CNN) with no activations to learn the inverse degradation
kernel, with the degradation kernels ($K$) from the RKG dataset as its input.
The proposed L-CNN is a computationally efficient architecture having a depth
of five layers and a width of 32 with $3\times 3$ filters across the depth.
Here the L-CNN is chosen to maintain the dimensions of the input throughout
the network i.e. the same output dimension at every layer. Therefore, at the
inference stage, L-CNN operates on blur images to generate sharp images with
fine details.
The general limitations of the networks that maintain the dimensions of the
input at the output, like computational time, complexity, and memory are
because of operations in high dimensional space. It is important to note that
these limitations are not valid in this work as the input to the CNN is kernel
$K$, a matrix of very small dimension (i.e., $11\times 11$), compared to very
high dimensional or 3-D tensor input images, in practice. The learning and
inference methodology of the NSD-DIL model is depicted in Fig. 2.

###  3.4 Deep Identity Learning (DIL)

The L-CNN is trained on the RKG dataset with DIL objective given in Eq. (4).

$Loss(L)=~{}\lVert K\ast K^{-1}-~{}\delta\rVert^{2}_{2}+R$ (4)

Here, $\lVert K\ast K^{-1}-~{}\delta\rVert^{2}_{2}$ term ensures the identity
relation between the degradation and its inverse model and $R$ is the proposed
regularization, defined in Eq. (5).

$R=\lambda_{1}\times R_{1}+\lambda_{2}\times R_{2}+\lambda_{3}\times R_{3}$
(5)

where,

  1. 1.

$R_{1}=L_{ConvArea}=~{}\mid 1-\sum_{i,j}K^{-1}_{i,j}\mid$; This term ensures
the validity of the area property of the convolution operation. It states that
the product of the area under the two input signals is equal to the area under
their convolution output signal. Consider the identity relation given in Eq.
3, the input signals are degradation kernel and its inverse i.e. $K$ and
$K^{-1}$, while the output signal is a 2-D impulse function. The area property
of the convolution operation is discussed in detail below.
The 2-dimensional Fourier transform can be written as

$\mathcal{F}(K(m,n))=\hat{K}(p,q)=\sum_{m}\sum_{n}K(m,n)\exp^{-j({\frac{2\pi
pm}{N_{1}}}+{\frac{2\pi qn}{N_{2}}})}$ (6)

Here, $\mathcal{F}(.)$ represent the Fourier transform operation, (m,n) and
(p,q) represent the index locations of the degradation kernel matrix in the
Time-domain and Frequency-domain respectively, $\hat{K}(.)$ represent the
Fourier transform of $K$ and $N_{1},N_{2}$ represent the dimensions of
degradation kernel.
We apply the Fourier transform to Eq. 3.
$\therefore~{}\mathcal{F}(K*K^{-1})=\mathcal{F}(\delta)$;
Since Convolution operation in the Time domain is equivalent to Multiplication
in the Frequency domain (and vice-versa), the above expression can be written
as $\hat{K}(p,q).\hat{K^{-1}}(p,q)=1$ ($\because~{}\mathcal{F}(\delta)=1$).
Here ’.’ represents the element-wise multiplication operation.
Keeping the values of $(p,q)=(0,0)$ in the Eq. 6, we arrive at another
important property of the convolution operation that states the area under the
Time-domain signal is equal to the value at the origin in the Frequency-domain
form of that signal. Therefore, $\hat{K}(0,0)=\sum_{m}\sum_{n}K(m,n)$.

The $\sum_{m}\sum_{n}K(m,n)$ is set to be 1 in our RKG dataset, $K$ being a
blur kernel. To satisfy the relation $\hat{K}(0,0).\hat{K^{-1}}(0,0)=1$, the
area under the inverse degradation kernel is imposed to be 1 i.e.
$\sum_{m}\sum_{n}K^{-1}(m,n)=1$ as a regularizer in our proposed DIL
objective.

  2. 2.

$R_{2}=L_{zerophase}=~{}\mid
M_{0}-({\angle\mathcal{F}(K)+\angle\mathcal{F}(K^{-1}}))\mid$; enables the sum
of phase of degradation kernel $K$ and the inverse degradation kernel $K^{-1}$
to be equal to $0$. Here $M_{0}$ represents a matrix of zeros having
dimensions same as $\delta$ (in Eq.3). This term ensures no phase degradation
in the overall system. $\lambda_{2}$ is a hyper-parameter.

  3. 3.

$R_{3}=L_{onemag}=~{}\mid
M_{1}-(\mid\mathcal{F}(K)\mid.\mid\mathcal{F}(K^{-1})\mid)\mid$; enables the
product of magnitude of the degradation kernel $K$ and the inverse degradation
kernel $K^{-1}$ to be equal to 1. Here $M_{1}$ represents a matrix of ones
having dimensions the same as $\delta$ (in Eq.3). This term ensures no shape
degradation of the input image and also ensures the presence of all
frequencies in the output same as the input of the E2ED model i.e. sharp
image.

Ideally, the _identity model_ (i.e. $K*K^{-1}=\delta$) should behave like an
all-pass system with a unit magnitude and with zero-phase so that the input
and output of the _identity model_ will be identical. Hence the terms
$L_{zerophase}$ and $L_{onemag}$ were included in learning the deblurring
task. Here, $\lambda_{1}$, $\lambda_{2}$, $\lambda_{3}$ are hyper-parameters.
Note that the proposed regularization terms are not general regularization
terms just to facilitate the deep learning model to achieve the task. Each
term in Eq. 5 is driven by the linear systems point-of-view and dedicated to
delivering robust deblurring performance, besides constraining the solution
space of the inverse degradation kernel for being a valid inverse of the
degradation kernel space.
Note: It was observed that the reference label i.e. two-dimensional impulse
function, generated using the Python code has unit magnitude but non-zero
phase properties. Hence to adjust the phase to be equal to zero and having
unit magnitude, the $1$ located at the center is shifted to the very location
of the matrix.

###  3.5 Deep Restoration Kernel (DRK)

Figure 3: (a) The L-CNN and its effective representation, DRK. The mesh plots
of the DRK (b) spatial domain, (c) The top view of its magnitude spectrum.

We proposed L-CNN with solely convolution layers in the interest of extracting
the inverse degradation kernel $K^{-1}$, from the trained network. According
to the associative property of the Linear Time-Invariant (LTI) system, the
series interconnection of any arbitrary number of LTI systems is equivalent to
a single system. And, the impulse response of the ‘n’ cascaded LTI systems is
the convolution of their individual impulse responses. Therefore, an effective
representation of the L-CNN model can be obtained in a single matrix form by
convolving all the filters of L-CNN sequentially with stride $1$ with impulse
as its input. The extracted $K^{-1}$ from the L-CNN is referred to as “Deep
Restoration Kernel (DRK)”. It gives the explicit matrix representation of the
inverse degradation model for the deblurring problem. The DRK is equivalent to
the deep L-CNN model and can be presented as a filter as shown in Fig. 3 (a).
Explainability: Black box to White box model The explicit representation of
the inverse degradation kernel i.e. DRK, assists in the progress of
understanding the learning mechanism of the deep model (i.e. L-CNN) for the
deblurring task. The DRK, a matrix, and the L-CNN are the inverse instances of
the degradation space spanned by the RKG dataset. The mesh plots of the DRK
were presented in the spatial domain in Fig. 3 (b) and the top view of its
magnitude spectrum in Fig. 3 (c). It can be vividly observed that the
magnitude spectrum of the DRK has the characteristics of both low-pass and
high-pass filters. It has a magnitude equal to one around the center (i.e.
low-pass filter) and increasing magnitude away from the center (i.e. high-pass
filter). The spatial domain form of DRK has a Mexican hat-like structure.
Therefore, it can be noted that the extracted DRK can preserve the low
frequencies and enhance the high frequencies, acting as a high-frequency-
boosting filter or unsharp masking filter. Henceforth, it can be inferred that
the proposed deep model can learn the actual deconvolution task, unlike the
existing DL-based deblurring models which learn mapping from blurred data to
sharp data distribution.

In this way, we can have a deeper understanding of the weights that were
obtained from a trained deep model and explain the trained model. Therefore,
the proposed NSD-DIL model for the deblurring task is no longer a black box
deep learning model.
We also note that this DRK alone could also be employed for the Deblurring
task as per Eq. 2 (refer Fig. 3), and the results were presented in Table 1.
This makes the proposed method highly efficient for real-world applications as
a single matrix with parameters in the order of 100, serves the deblurring
task effectively. Also, the computational complexity was reduced from
processing the blurred image through a deep model to the convolution operation
between the DRK and the blurred image. The effective representation of the
L-CNN facilitates the application of the regularization constraints on it
easily. With DRK, it is a unique restoration process as it provides ample
scope for the development of explainable models for deblurring.

##  4 Experiments

In this section, the implementation details, deblurring results, the effect of
the proposed regularization term, and other related studies were discussed.

###  4.1 Training setup

We trained the proposed L-CNN (refer Sec.3.3 for architecture details of
L-CNN) on the RKG dataset (refer Sec. 3.2) with the learning objective given
in Eq. (4). The number of epochs was $40$ and the learning rate was $10e-4$.
The Adam [22] optimization was used, with $\beta=0.9$. The values of hyper-
parameters used in Eq. (5), set empirically, are as follows,
$\lambda_{1}=0.8,~{}\lambda_{2}=0.8,~{}\lambda_{3}=0.4$. The NSD-DIL code and
RKG dataset will be made publicly available soon for reproducibility.

###  4.2 Computational complexity:

The proposed L-CNN is an efficient and lightweight deblurring model. The
computational complexity in terms of parameters and inference time of the NSD-
DIL model was compared to SotA methods and is given in Table 1. Here, the
proposed NSD-DIL model requires very less computational resources, by order of
1000 in terms of parameters and inference time when compared to the recent
SotA work i.e. IFFS [56]. Besides NSD-DIL, the deblurring task with DRK
further reduces the required computational resources parameters and inference
time by an order of 10 to NSD-DIL and by an order of 10K to SotA methods. Thus
the proposed NSD-DIL model is more suitable for real-time deblurring tasks.

###  4.3 Results

The deblurring performance of our proposed method is compared with a wide
range of methods including standard and recent non-DL blind deblurring methods
NSM [24], PIBKE [36], Polyblur [10] and DL-based methods DeblurGAN [25], DUN
[61], XY [18], IFFS [56]. For the deblurring task, the blind deblurring
methods, NSM proposed a regularization term in the spatial domain based on
normalized sparsity measure. While, the PIBKE and IFFS operate on the
frequency domain features of the input blurred image for $K$ estimation based
on the phase information of the blurred image and frequency filtering
techniques respectively. The methods DeblurGAN and XY are supervised and DUN
is an unsupervised data-driven deblurring method. We note that the proposed
NSD-DIL is only the DL approach among the above-mentioned DL-based methods
that achieve the deblurring task without training on image data.

Table 1: Comparison of deblur results on simulated blur dataset in terms of
PSNR$\uparrow$/SSIM$\uparrow$ along with the number of parameters (in
Millions) and Inference time (in seconds) of various methods were given below.
Here, the red indicates the best score and the blue indicates the second-best
score.

| NSM | PIBKE | DeblurGAN | DUN | XY | Polyblur | IFFS | NSD-LCNN | NSD-DRK
---|---|---|---|---|---|---|---|---|---
| # parameters
---
N.A. | N.A. | 17.1 | 6.1 | 4.92 | N.A. | 19.5 | 0.0028 | 0.00001
| Inference
---
time
120 | 120 | 1 | 1 | 0.729 | 0.001 | 3.5 | 0.0002 | 0.0005
$K_{11\times 11}$ | 27.02/0.8053 | 24.19/0.7280 | 26.71/0.7925 | 24.32/0.7400 | 27.32/0.8097 | 27.21/0.7829 | 26.59/0.8276 | 27.26/0.8434 | 28.09/0.8405
$K_{15\times 15}$ | 27.63/0.8227 | 24.83/0.7312 | 26.65/0.7940 | 24.32/0.7390 | 27.12/0.8041 | 27.06/0.7771 | 26.50/0.8225 | 26.96/0.8350 | 27.99 /0.8384
$K_{19\times 19}$ | 27.66/0.8232 | 24.89/0.7439 | 26.68/0.7970 | 24.33/0.7428 | 27.30/0.8111 | 27.20/0.7874 | 26.51/0.8260 | 26.97/0.8328 | 27.94/0.8385
$K_{23\times 23}$ | 27.57/0.8162 | 25.44/0.7552 | 26.59/0.7911 | 24.31/0.7384 | 27.07/0.8027 | 26.96/0.7707 | 26.51/0.8215 | 26.95/0.8333 | 28.05/0.8372
$K_{27\times 27}$ | 27.67/0.8176 | 24.97/0.7341 | 26.62/0.7921 | 24.29/0.7366 | 27.10/0.8019 | 27.11/0.7789 | 26.53/0.8235 | 26.90/0.8315 | 28.02/0.8373
PSNR: | 27.51 $\pm$ 0.2474 | 24.86 $\pm$ 0.3997 | 26.65 $\pm$ 0.0424 | 24.31 $\pm$ 0.0135 | 27.18 $\pm$ 0.1059 | 27.10 $\pm$ 0.0928 | 26.52 $\pm$ 0.0324 | 27.01 $\pm$ 0.1167 | 28.02 $\pm$ 0.0511
SSIM: | 0.8170 $\pm$ 0.0064 | 0.7384 $\pm$ 0.0099 | 0.7933 $\pm$ 0.0020 | 0.7396 $\pm$ 0.0020 | 0.8059 $\pm$ 0.0037 | 0.7794 $\pm$ 0.0056 | 0.8242 $\pm$ 0.0022 | 0.8333 $\pm$ 0.0012 | 0.8383 $\pm$ 0.0011

Figure 4: Visual results of different deblur methods.

Comparison on simulated blur: We generated a synthetic mild blur dataset
following the [10]. Random Gaussian kernels of varying shapes, orientations )
(i.e. $\sigma_{1},\sigma_{2}\in U[0.175,3]$ and rotation angle $\theta\in
U[0,\pi]$) and sizes $11\times 11,15\times 15,19\times 19,23\times 23,27\times
27$ were generated. Each set of kernels, varying in size, was applied to the
set of 100 sharp images from the DIV2K validation dataset [52]. To have a
little deviation from the Gaussian distribution, a small amount of
multiplicative noise was introduced to the random Gaussian kernels. We
compared the NSD-DIL model to the above-mentioned SotA works using the
simulated blur dataset. To assess the ability to restore the finer details
during the deblurring process, we employ PSNR and SSIM for qualitative and
quantitative similarity with the corresponding sharp images. The results were
tabulated and presented in Table 1. The qualitative results were provided in
Fig. 4.
Despite being very lightweight, the proposed NSD-DIL demonstrated its
effectiveness significantly with standard metrics. The compared SotA methods
need supervised learning from images for $K$ estimation and/or deblurring
model learning. Thus requires more memory, and time resources at least by
order of 100 in parameters and inference time respectively compared to the
proposed NSD-DIL.
Evaluation on real data: Further, we provide the qualitative comparison of
deblurring performance on real captured blurred images provided in Fig. 1. The
qualitative results given in Fig. 4 and Fig. 1 illustrate that the proposed
model did not produce artifacts in the generated deblurred output images,
unlike the compeer, zero-shot non-DL SotA works like [24], [36]. Besides, the
proposed NSD-DIL method demonstrates its generalizability by generating sharp
images with good dynamic range, without any knowledge of the corresponding
blur kernel, has no strict limitations on the blur kernel estimation, and is
not restrictive to synthetic kernels in our training dataset.

###  4.4 NSD-DIL for Image Super-Resolution

Table 2: Comparison of ISR results on DIV2KRK dataset in terms of
SSIM$\uparrow$/PSNR$\uparrow$/NIMA$\uparrow$ along with the number of
parameters (in Millions) and Inference time (in seconds) of various methods
were given below. Here, the red indicates the best score and the blue
indicates the second-best score.

| -
---
Bicubic | EDSR | ZSSR |  | KernelGAN
---
\+ ZSSR
DBPI | DualSR |  | NSD-LCNN
---
(Ours)
| NSD-DRK
---
(Ours)
| # parameters
---
N.A. | 2.98 | 0.29 | 0.151+0.29 | 0.5 | 0.44 | 0.0028 | 0.00001
Inference time | N.A. | 1 |  $\>=$ 10 |  $>=$13 |  $>=$1 |  $>=$3 | 0.002 | 0.0005
x2 | 0.7846/27.24/4.127 |  0.8034/29.17/4.142 | 0.7925/27.51/4.111 | 0.8379/28.24/4.071 |  0.8684/30.77/4.049 |  0.8250/25.78/5.005 |  0.8980/27.53/4.976 |  0.9038/27.27/ 5.006
x4 | 0.6478/23.89/4.147 |  0.6615/25.63/4.181 | 0.6550/24.05/4.156 | 0.6799/24.76/4.089 |  0.7368/26.86/4.146 | -/-/- |  0.7331/23.63/4.904 |  0.7487/23.93/4.903

Figure 5: Visual results of different ISR methods.

Image Super Resolution (ISR) is a well-established low-level vision task whose
objective is to generate a High-Resolution (HR) image from the given
corresponding LR observation(s). We propose to extend the proposed model to
the ISR task. The NSD-DIL model can be directly employed at the inference
stage, as the trained L-CNN acts as a pre-upsampled network that operates on
traditional upsampled (Bicubic) input blur images of any desired scale factor
to generate sharp images with fine details. The same trained L-CNN for the
deblurring task was employed to perform the ISR task. We also note that this
DRK alone could also be employed for the ISR task in place of L-CNN on a pre-
upsampled image and operates the same as L-CNN. In these experiments, no
kernel estimation stage is required. The super-resolution ability of our
approach was evaluated on the benchmark dataset DIV2KRK [2] for sf 2, and sf 4
using PSNR, SSIM, and NIMA metrics. The ISR results were presented in Table 2.
For a fair comparison, we considered works like KernelGAN [2], ZSSR [49], DBPI
[20], and DualSR [13] which consider single input LR image only for modeling
the ISR task as in our experiments. From the quantitative results, it was
observed that the NSD-DIL model delivers competitive performance very close to
the SotA ISR works despite being absolutely independent of LR and HR image
space. The sample visual results for sf 2 were provided in Fig. 5. The
computational complexity in terms of parameters and super-resolution time
(inference time) in minutes, of the NSD-DIL model was compared to SotA methods
and is given in Table 2. The potential of the proposed NSD-DIL model is
observed from the given Table 2. Here, the NSD-DIL requires approximately 100
times fewer memory resources even when compared with the least among the SotA
methods i.e. ZSSR, where the DRK requires by order 10 times further lesser
than NSD-DIL. Besides, the inference time required for the NSD-DIL and the DRK
is reduced by order of 10 and 100 respectively compared to existent zero-shot
ISR methods.

###  4.5 Ablation study

In this section, the significance of the proposed regularization term ($R$)
(refer Eq. 5), and the deblurring performance of the NSD-DIL method on various
blur levels along with varied kernel sizes and various sizes of the RKG
dataset were discussed. Further, we present the comparison of the deblurring
performance of our proposed method with the traditional deconvolution filter
i.e. Wiener filter.

####  4.5.1 Regularization term (R)

The regularization term ($R$) (refer Eq. 5) is proposed to have a faithful
deconvolution model with reliable deblurring performance. The influence of
each entity in the proposed $R$ is quantified and presented in Table 3. It was
observed that the presence of $L_{ConvArea}$ in $R$ has shown a greater impact
on the performance of the proposed method. The terms
$L_{PhLoss}~{}\&~{}L_{MagLoss}$ influenced the perceptual quality of the
images. The term $L_{ConvArea}$ together with the
$L_{PhLoss}~{}\&~{}L_{MagLoss}$ achieve a reliable performance.

Table 3: The effect of proposed regularization terms in R on the NSD-DIL
model’s performance with the synthetic Gaussian blur dataset.

Loss (L) | $\lVert K\ast K^{-1}-~{}\delta\rVert^{2}_{2}$ |  |  $\lVert K\ast K^{-1}-~{}\delta\rVert^{2}_{2}$ +
---
$\lambda_{1}\times R_{1}$
|  $\lVert K\ast K^{-1}-~{}\delta\rVert^{2}_{2}$ +
---
$\lambda_{2}\times R_{2}$
|  $\lVert K\ast K^{-1}-~{}\delta\rVert^{2}_{2}$ +
---
$\lambda_{3}\times R_{3}$
|  $\lVert K\ast K^{-1}-~{}\delta\rVert^{2}_{2}$ +
---
$\lambda_{2}\times R_{2}+\lambda_{3}\times R_{3}$
|  $\lVert K\ast K^{-1}-~{}\delta\rVert^{2}_{2}$ +
---
$\lambda_{1}\times R_{1}+\lambda_{2}\times R_{2}$
|  $\lVert K\ast K^{-1}-~{}\delta\rVert^{2}_{2}$ +
---
$\lambda_{1}\times R_{1}+\lambda_{2}\times R_{2}+\lambda_{3}\times R_{3}$
PSNR$\uparrow$ | 6.50 | 21.07 | 8.12 | 17.69 | 17.21 | 25.04 | 27.26
SSIM$\uparrow$ | 0.0244 | 0.5455 | 0.2137 | 0.2030 | 0.5644 | 0.7419 | 0.8434

Note: The deblurring performance of the NSD-DIL method on various blur levels,
which are disjoint with our training dataset, was studied and observed only an
approximately $6\%$ decrease in terms of PSNR and SSIM metrics as the blur
level varied from $\sigma_{1},\sigma_{2}\in U[0.175,3]$ to $U[6,9]$. More
details on the study of the deblurring performance of the NSD-DIL method on
various blur levels along with varied kernel sizes and various sizes of the
RKG dataset details are provided in the supplementary material. In addition,
we observed that the conventional inverse degradation modeling by the Wiener
filter failed to hold the robustness across the wide range of degradations in
the generated simulated blur dataset, itself. The quantitative results are
provided in the supplementary material.

###  4.6 Discussion

In this work, it is important to note that our null-shot model detours the
degradation kernel estimation stage to achieve the deblur and ISR tasks,
unlike the existing well-established methods including the zero-shot self-
supervised deblur method i.e. Polyblur [10] and ISR method i.e. KernelGAN [2].
Here, although we didn’t consider the additive noise explicitly in our
degradation model represented by Eq. 1, the proposed model showcased its
ability to generate super-resolved images with superior SSIM scores (refer to
Table 1 and Table 2). However, in future work, we would extend our null-shot
method to address stronger and non-uniform blurs along with additive noise.
In the case of the ISR task, for higher scale factors, the proposed method can
deliver improved performance as the inverse degradation kernel size is made
large enough. It is planned to explore in our future work on the null-shot
approach for other image restoration tasks.

###  4.7 Conclusion

We have established a new problem formulation based on the DIL objective for
the deblurring task, which can be extended to the ISR task as well. The
proposed NSD-DIL is the first DL-based image data-independent deblurring model
that involves a restoration process rather than learning mapping from
distorted to sharp image distribution. The proposed method uses the
degradation kernel space effectively to restore the sharp images from blurred
images based on the proposed dedicated regularization term provided in the DIL
objective function. Our NSD-DIL method paves the path to have a deeper look
into the learning and understanding of the inverse degradation kernel since
the explicit representation i.e. DRK, in a matrix form is made possible.
Additionally, we have demonstrated the deblurring task of replacing the deep
model with a simple convolution operation with the computed DRK, which doesn’t
require high-performance computing resources. The proposed method’s
performance is superior to the SotA works, despite being a very lightweight
deblurring model. The experimental results and computational performance
comparisons with the SotA indicate that the proposed NSD-DIL and the DRK are
remarkably suitable for real-time embedded applications of deblurring and ISR
tasks.

## References

  * [1] Arora, S., Cohen, N., Hazan, E.: On the optimization of deep networks: Implicit acceleration by overparameterization. In: International Conference on Machine Learning. pp. 244–253. PMLR (2018)
  * [2] Bell-Kligler, S., Shocher, A., Irani, M.: Blind super-resolution kernel estimation using an internal-gan. Advances in Neural Information Processing Systems 32 (2019)
  * [3] Buades, A., Coll, B., Morel, J.M.: A non-local algorithm for image denoising. In: 2005 IEEE computer society conference on computer vision and pattern recognition (CVPR’05). vol. 2, pp. 60–65. Ieee (2005)
  * [4] Cai, J.F., Ji, H., Liu, C., Shen, Z.: Blind motion deblurring from a single image using sparse approximation. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition. pp. 104–111. IEEE (2009)
  * [5] Carasso, A.S.: The apex method in image sharpening and the use of low exponent lévy stable laws. SIAM Journal on Applied Mathematics 63(2), 593–618 (2003)
  * [6] Chen, M., Quan, Y., Xu, Y., Ji, H.: Self-supervised blind image deconvolution via deep generative ensemble learning. IEEE Transactions on circuits and systems for video technology 33(2), 634–647 (2022)
  * [7] Chiang, C.M., Tseng, Y., Xu, Y.S., Kuo, H.K., Tsai, Y.M., Chen, G.Y., Tan, K.S., Wang, W.T., Lin, Y.C., Tseng, S.Y.R., et al.: Deploying image deblurring across mobile devices: A perspective of quality and latency. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops. pp. 502–503 (2020)
  * [8] Cho, S., Lee, S.: Fast motion deblurring. In: ACM SIGGRAPH Asia 2009 papers, pp. 1–8 (2009)
  * [9] Choromanska, A., Henaff, M., Mathieu, M., Arous, G.B., LeCun, Y.: The loss surfaces of multilayer networks. In: Artificial intelligence and statistics. pp. 192–204. PMLR (2015)
  * [10] Delbracio, M., Garcia-Dorado, I., Choi, S., Kelly, D., Milanfar, P.: Polyblur: Removing mild blur by polynomial reblurring. IEEE Transactions on Computational Imaging 7, 837–848 (2021)
  * [11] Dong, J., Pan, J., Sun, D., Su, Z., Yang, M.H.: Learning data terms for non-blind deblurring. In: Proceedings of the European Conference on Computer Vision (ECCV). pp. 748–763 (2018)
  * [12] Efrat, N., Glasner, D., Apartsin, A., Nadler, B., Levin, A.: Accurate blur models vs. image priors in single image super-resolution. In: Proceedings of the IEEE International Conference on Computer Vision. pp. 2832–2839 (2013)
  * [13] Emad, M., Peemen, M., Corporaal, H.: Dualsr: Zero-shot dual learning for real-world super-resolution. In: Proceedings of the IEEE/CVF winter conference on applications of computer vision. pp. 1630–1639 (2021)
  * [14] Fergus, R., Singh, B., Hertzmann, A., Roweis, S.T., Freeman, W.T.: Removing camera shake from a single photograph. In: Acm Siggraph 2006 Papers, pp. 787–794 (2006)
  * [15] He, K., Sun, J., Tang, X.: Guided image filtering. IEEE transactions on pattern analysis and machine intelligence 35(6), 1397–1409 (2012)
  * [16] Honarvar, B., Paramesran, R., Lim, C.L.: Image reconstruction from a complete set of geometric and complex moments. Signal Processing 98, 224–232 (2014)
  * [17] Ji, H., Wang, K.: Robust image deblurring with an inaccurate blur kernel. IEEE Transactions on Image processing 21(4), 1624–1634 (2011)
  * [18] Ji, S.W., Lee, J., Kim, S.W., Hong, J.P., Baek, S.J., Jung, S.W., Ko, S.J.: Xydeblur: divide and conquer for single image deblurring. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 17421–17430 (2022)
  * [19] Kawaguchi, K.: Deep learning without poor local minima. Advances in neural information processing systems 29 (2016)
  * [20] Kim, J., Jung, C., Kim, C.: Dual back-projection-based internal learning for blind super-resolution. IEEE Signal Processing Letters 27, 1190–1194 (2020). https://doi.org/10.1109/LSP.2020.3005043
  * [21] Kim, S.H., Allebach, J.P.: Optimal unsharp mask for image sharpening and noise removal. Journal of Electronic Imaging 14(2), 023005–023005 (2005)
  * [22] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. In: Bengio, Y., LeCun, Y. (eds.) 3rd International Conference on Learning Representations, ICLR 2015, San Diego, CA, USA, May 7-9, 2015, Conference Track Proceedings (2015)
  * [23] Krishnan, D., Fergus, R.: Fast image deconvolution using hyper-laplacian priors. Advances in neural information processing systems 22 (2009)
  * [24] Krishnan, D., Tay, T., Fergus, R.: Blind deconvolution using a normalized sparsity measure. In: CVPR 2011. pp. 233–240. IEEE (2011)
  * [25] Kupyn, O., Budzan, V., Mykhailych, M., Mishkin, D., Matas, J.: Deblurgan: Blind motion deblurring using conditional adversarial networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition. pp. 8183–8192 (2018)
  * [26] Levin, A., Fergus, R., Durand, F., Freeman, W.T.: Image and depth from a conventional camera with a coded aperture. ACM transactions on graphics (TOG) 26(3), 70–es (2007)
  * [27] Levin, A., Weiss, Y., Durand, F., Freeman, W.T.: Understanding and evaluating blind deconvolution algorithms. In: 2009 IEEE conference on computer vision and pattern recognition. pp. 1964–1971. IEEE (2009)
  * [28] Li, J., Nan, Y., Ji, H.: Un-supervised learning for blind image deconvolution via monte-carlo sampling. Inverse Problems 38(3), 035012 (2022)
  * [29] Li, J., Wang, W., Nan, Y., Ji, H.: Self-supervised blind motion deblurring with deep expectation maximization. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 13986–13996 (2023)
  * [30] Liang, J., Cao, J., Sun, G., Zhang, K., Van Gool, L., Timofte, R.: Swinir: Image restoration using swin transformer. In: Proceedings of the IEEE/CVF international conference on computer vision. pp. 1833–1844 (2021)
  * [31] Liu, Y.Q., Du, X., Shen, H.L., Chen, S.J.: Estimating generalized gaussian blur kernels for out-of-focus image deblurring. IEEE Transactions on circuits and systems for video technology 31(3), 829–843 (2020)
  * [32] Michaeli, T., Irani, M.: Nonparametric blind super-resolution. In: 2013 IEEE International Conference on Computer Vision. pp. 945–952 (2013). https://doi.org/10.1109/ICCV.2013.121
  * [33] Michaeli, T., Irani, M.: Blind deblurring using internal patch recurrence. In: Computer Vision–ECCV 2014: 13th European Conference, Zurich, Switzerland, September 6-12, 2014, Proceedings, Part III 13. pp. 783–798. Springer (2014)
  * [34] Nah, S., Hyun Kim, T., Mu Lee, K.: Deep multi-scale convolutional neural network for dynamic scene deblurring. In: Proceedings of the IEEE conference on computer vision and pattern recognition. pp. 3883–3891 (2017)
  * [35] Pan, J., Sun, D., Pfister, H., Yang, M.H.: Deblurring images via dark channel prior. IEEE transactions on pattern analysis and machine intelligence 40(10), 2315–2328 (2017)
  * [36] Pan, L., Hartley, R., Liu, M., Dai, Y.: Phase-only image based kernel estimation for single image blind deblurring. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 6034–6043 (2019)
  * [37] Park, D., Kang, D.U., Kim, J., Chun, S.Y.: Multi-temporal recurrent neural networks for progressive non-uniform single image deblurring with incremental temporal training. In: European Conference on Computer Vision. pp. 327–343. Springer (2020)
  * [38] Perrone, D., Favaro, P.: Total variation blind deconvolution: The devil is in the details. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. pp. 2909–2916 (2014)
  * [39] Polesel, A., Ramponi, G., Mathews, V.J.: Image enhancement via adaptive unsharp masking. IEEE transactions on image processing 9(3), 505–510 (2000)
  * [40] Purohit, K., Rajagopalan, A.: Region-adaptive dense network for efficient motion deblurring. In: Proceedings of the AAAI Conference on Artificial Intelligence. vol. 34, pp. 11882–11889 (2020)
  * [41] Ren, D., Zhang, K., Wang, Q., Hu, Q., Zuo, W.: Neural blind deconvolution using deep priors. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 3341–3350 (2020)
  * [42] Ren, D., Zuo, W., Zhang, D., Zhang, L., Yang, M.H.: Simultaneous fidelity and regularization learning for image restoration. IEEE transactions on pattern analysis and machine intelligence 43(1), 284–299 (2019)
  * [43] Richardson, W.H.: Bayesian-based iterative method of image restoration. JoSA 62(1), 55–59 (1972)
  * [44] Riegler, G., Schulter, S., Rüther, M., Bischof, H.: Conditioned regression models for non-blind single image super-resolution. In: 2015 IEEE International Conference on Computer Vision (ICCV). pp. 522–530 (2015). https://doi.org/10.1109/ICCV.2015.67
  * [45] Russo, F.: An image-enhancement system based on noise estimation. IEEE Transactions on Instrumentation and measurement 56(4), 1435–1442 (2007)
  * [46] Sajjadi, M.S., Scholkopf, B., Hirsch, M.: Enhancenet: Single image super-resolution through automated texture synthesis. In: Proceedings of the IEEE international conference on computer vision. pp. 4491–4500 (2017)
  * [47] Saxe, A., McClelland, J., Ganguli, S.: Exact solutions to the nonlinear dynamics of learning in deep linear neural networks. International Conference on Learning Represenatations 2014 (2014)
  * [48] Schmidt, U., Roth, S.: Shrinkage fields for effective image restoration. In: Proceedings of the IEEE conference on computer vision and pattern recognition. pp. 2774–2781 (2014)
  * [49] Shocher, A., Cohen, N., Irani, M.: “zero-shot” super-resolution using deep internal learning. In: Proceedings of the IEEE conference on computer vision and pattern recognition. pp. 3118–3126 (2018)
  * [50] Sun, J., Cao, W., Xu, Z., Ponce, J.: Learning a convolutional neural network for non-uniform motion blur removal. In: Proceedings of the IEEE conference on computer vision and pattern recognition. pp. 769–777 (2015)
  * [51] Tao, X., Gao, H., Shen, X., Wang, J., Jia, J.: Scale-recurrent network for deep image deblurring. In: Proceedings of the IEEE conference on computer vision and pattern recognition. pp. 8174–8182 (2018)
  * [52] Timofte, R., Gu, S., et al.: Ntire 2018 challenge on single image super-resolution: Methods and results. In: The IEEE Conference on Computer Vision and Pattern Recognition (CVPR) Workshops (June 2018)
  * [53] Tomasi, C., Manduchi, R.: Bilateral filtering for gray and color images. In: Sixth international conference on computer vision (IEEE Cat. No. 98CH36271). pp. 839–846. IEEE (1998)
  * [54] Wang, Y., Yang, J., Yin, W., Zhang, Y.: A new alternating minimization algorithm for total variation image reconstruction. SIAM Journal on Imaging Sciences 1(3), 248–272 (2008)
  * [55] Wiener, N.: Extrapolation, Interpolation, and Smoothing of Stationary Time Series: With Engineering Applications. M.I.T. Press (1964)
  * [56] Xintian Mao, Y.L., et al.: Intriguing findings of frequency selection for image deblurring. In: Proceedings of the 37th AAAI Conference on Artificial Intelligence (2023)
  * [57] Xu, L., Jia, J.: Two-phase kernel estimation for robust motion deblurring. In: Computer Vision–ECCV 2010: 11th European Conference on Computer Vision, Heraklion, Crete, Greece, September 5-11, 2010, Proceedings, Part I 11. pp. 157–170. Springer (2010)
  * [58] Xue, F., Blu, T.: A novel sure-based criterion for parametric psf estimation. IEEE Transactions on Image Processing 24(2), 595–607 (2014)
  * [59] Zamir, S.W., Arora, A., Khan, S., Hayat, M., Khan, F.S., Yang, M.H.: Restormer: Efficient transformer for high-resolution image restoration. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. pp. 5728–5739 (2022)
  * [60] Zhang, J., Pan, J., Ren, J., Song, Y., Bao, L., Lau, R.W., Yang, M.H.: Dynamic scene deblurring using spatially variant recurrent neural networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition. pp. 2521–2529 (2018)
  * [61] Zhang, K., Van Gool, L., Timofte, R.: Deep unfolding network for image super-resolution. In: IEEE Conference on Computer Vision and Pattern Recognition. pp. 3217–3226 (2020)
  * [62] Zhang, K., Luo, W., Zhong, Y., Ma, L., Stenger, B., Liu, W., Li, H.: Deblurring by realistic blurring. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 2737–2746 (2020)
  * [63] Zhang, W., Liu, Y., Dong, C., Qiao, Y.: Ranksrgan: Generative adversarial networks with ranker for image super-resolution. In: Proceedings of the IEEE/CVF International Conference on Computer Vision. pp. 3096–3105 (2019)
  * [64] Zhu, X., Milanfar, P.: Restoration for weakly blurred and strongly noisy images. In: 2011 IEEE Workshop on Applications of Computer Vision (WACV). pp. 103–109. IEEE (2011)
  * [65] Zoran, D., Weiss, Y.: From learning models of natural image patches to whole image restoration. In: 2011 international conference on computer vision. pp. 479–486. IEEE (2011)