# Rubio de Francia Extrapolation Theorems
for Quasi-Monotone Functions

Arun Pal Singh1, Rahul Panchal2, Pankaj Jain3 and Monika Singh4

###### Abstract

We prove Rubio de Francia extrapolation results in Lebesgue and grand Lebesgue
spaces for quasi monotone functions with $QB_{\beta,p}$ weights. The
extrapolation in Lebesgue spaces with the weight class $QB_{\beta,\infty}$ has
also been investigated. As an application, we characterize the boundedness of
the Hardy averaging operator for quasi monotone functions in the grand
Lebesgue spaces.

2010 _AMS Subject Classification._ 26D10, 26D15, 46E35.
_Key words and Phrases._ Rubio de Francia extrapolation; grand Lebesgue space;
$QB_{\beta,p}$-weights; Hardy averaging operator, quasi-monotone functions.

_1 Department of Mathematics, Dyal Singh College (University of Delhi), Lodhi
Road,
Delhi - 110003, INDIA_ Email<EMAIL_ADDRESS>

_2 Department of Mathematics, University of Delhi, Delhi - 110007, INDIA_
Email<EMAIL_ADDRESS>

_3 Department of Mathematics, South Asian University, Akbar Bhawan, Chanakya
Puri,
New Delhi - 110021, INDIA_ Email<EMAIL_ADDRESS><EMAIL_ADDRESS>

_4 Department of Mathematics Lady Shri Ram College for Women (University of
Delhi),
Lajpat Nagar, Delhi - 110024, INDIA_ Email<EMAIL_ADDRESS>

##  1 Introduction

We shall denote by $\mathcal{M},$ the set of all measurable functions defined
and finite almost everywhere (a.e.) on $\mathbb{R}^{+}.$ Also,
$\mathcal{M}^{+}\subset\mathcal{M}$ and
$\mathcal{M}^{+}_{\downarrow}\subset\mathcal{M}^{+}$ will denote,
respectively, the cones of non-negative and non-negative non-increasing
($\downarrow$) functions in $\mathcal{M}.$ By a weight $w,$ we mean a function
in $\mathcal{M}^{+}$ which is locally integrable as well. For a weight $w$ and
$1\leq p<\infty,$ denote by $L_{w}^{p},$ the weighted Lebesgue space
consisting of all $f\in\mathcal{M}$ such that

$\|f\|_{L_{w}^{p}}:=\left(\int_{0}^{\infty}|f|^{p}w\right)^{1/p}<\infty.$

A weight $w$ is said to be in the Muckenhoupt class $A_{p},~{}1<p<\infty,$ if

$[w]_{A_{p}}:=\sup_{J}\left(\frac{1}{|J|}\int_{J}w\right)\left(\frac{1}{|J|}\int_{J}w^{-p^{\prime}/p}\right)^{p-1}<\infty,$

and in class $A_{1},$ if

$[w]_{A_{1}}:=\sup_{J}\text{ess}\sup_{x\in J}\frac{W(J)}{w(x)|J|}<\infty,$

where supremum is taken over all non-degenerate intervals
$J\subset\mathbb{R}^{+},~{}\frac{1}{p}+\frac{1}{p^{\prime}}=1$ and
$W(J):=\int_{J}w(x)dx.$ The weight class $A_{p}$ is found to be useful in so
many ways. It characterizes the boundedness of the maximal operator [24] and
Riesz potential [9] in Lebesgue spaces. Moreover, this class also
characterizes the boundedness of these operators in grand Lebesgue spaces [7],
[21]. Another beauty of the $A_{p}$-class of weights can be realized via the
celebrity extrapolation result of J.L. Rubio de Francia [25], which asserts
that if a sublinear operator $T$ is bounded on $L_{w}^{p_{0}}$ for every $w\in
A_{p_{0}}~{}(p_{0}\geq 1,$ fixed) with the constant of inequality depending
only on $[w]_{A_{p_{0}}},$ then for every $1<p<\infty,~{}T$ is bounded on
$L_{w}^{p}$ for every $w\in A_{p}.$ This extrapolation result was further re-
investigated and explored by many people (see [6] and the references therein)
and now it is known that the operator $T$ has no role to play. In fact, it is
known that if $(f,g)$ is a pair of non-negative measurable functions such that
for some $1\leq p_{0}<\infty,$ the inequality

$\int_{0}^{\infty}f^{p_{0}}(x)w(x)dx\leq C\int_{0}^{\infty}g^{p_{0}}(x)w(x)dx$

holds for every $w\in A_{p_{0}}$ with constant $C$ depending on
$[w]_{A_{p_{0}}},$ then for every $1<p<\infty,$ the inequality

$\int_{0}^{\infty}f^{p}(x)w(x)dx\leq C\int_{0}^{\infty}g^{p}(x)w(x)dx$

holds for every $w\in A_{p}$ with constant $C$ depending on $[w]_{A_{p}}.$

This theory has been generalized to $A_{\infty}$-weights also, see [5]. In
[4], Carro and Lorente established a parallel extrapolation theory for a pair
of functions from $\mathcal{M}^{+}_{\downarrow}$ in the framework of
$B_{p}$-class of weights: A weight $w$ is said to belong to the class
$B_{p}~{}(p>0)$ if there exists a constant $C>0$ such that the inequality

$\int_{r}^{\infty}\left(\frac{r}{x}\right)^{p}w(x)dx\leq C\int_{0}^{r}w(x)dx$

holds for every $r>0$. Like the $A_{p}$-class of weights, the weight class
$B_{p}$ is also an important class of weights. It characterizes the
boundedness of the Hardy averaging operator

$Hf(x):=\frac{1}{x}\int_{0}^{x}f(t)dt$

in $L_{w}^{p}$ spaces for $f\in\mathcal{M}^{+}_{\downarrow}$ (see [1, 26]) and
also in grand Lebesgue spaces (defined in Section 3) for
$f\in\mathcal{M}^{+}_{\downarrow}$ [13, 21]. These characterizations, in fact,
are equivalent to the boundedness of the maximal operator, respectively in,
Lorentz space $\Lambda^{p}(w)$ [2] and grand Lorentz space $\Lambda^{p)}(w)$
[13].

In this paper, we consider quasi non-increasing functions, the class of such
functions being denoted by $Q_{\beta}:$ A function $f\in\mathcal{M^{+}}$ is
said to belong to $Q_{\beta},~{}\beta\in\mathbb{R},$ if $x^{-\beta}f(x)$ is
non-increasing. Clearly $\mathcal{M}^{+}_{\downarrow}=Q_{0}.$ For the
functions $f\in Q_{\beta},$ Bergh, Burenkov and Persson [3] investigated
Hardy’s inequality with power type weights, while for general weights it has
been proved in [16] that the inequality

$\left(\int_{0}^{\infty}{\left(\frac{1}{x}\int_{0}^{x}f(t)dt\right)}^{p}w(x)dx\right)\leq
C\int_{0}^{\infty}f^{p}(x)w(x)dx,~{}1\leq p<\infty$

holds for all $f\in Q_{\beta}$ if and only if $w\in QB_{\beta,p},~{}\beta>-1,$
i.e.,

$\displaystyle\int_{r}^{\infty}\left(\frac{r}{x}\right)^{p}w(x)dx\leq
C\int_{0}^{r}{\left(\frac{x}{r}\right)}^{\beta p}w(x)dx,~{}r>0.$ (1.1)

Note that for $\beta=0,$ the weight class $QB_{\beta,p}$ reduces to the class
$B_{p}.$ In the present paper, we define a variant of the class
$QB_{\beta,p},$ to be denoted by $\widehat{Q}B_{\beta,p},$ and prove the
extrapolation results for this class of weights, as well as for the weight
class

$QB_{\beta,\infty}:=\bigcup_{p>0}QB_{\beta,p}~{}.$

Further, we prove the extrapolation result for quasi-monotone functions in the
frame of grand Lebesgue spaces. As an application, we prove the boundedness of
the Hardy averaging operator for quasi-monotone functions in the grand
Lebesgue spaces. Our results generalize the extrapolation results of Carro and
Lorente [4] and Meskhi [22]. Throughout, all the functions used in this paper
are assumed to be non-negative and measurable.

##  2 Extrapolation results in Lebesgue spaces

For $p>0,$ we say that a weight $w\in QB_{\beta,\psi,p}$ if

$\displaystyle\int_{r}^{\infty}\left(\frac{\Psi(r)}{\Psi(x)}\right)^{p}w(x)\,dx\leq
C\int_{0}^{r}\left(\frac{\Psi(x)}{\Psi(r)}\right)^{\beta p}w(x)~{}dx,r>0$
(2.1)

for some constant $C>0$ and $\Psi(x):=\int_{0}^{x}\psi(t)dt,$ where $\psi$ is
a non-negative, non-increasing locally integrable function, i.e., $\psi\in
L_{loc}^{1}.$ For $\psi\equiv 1,$ the weight class $QB_{\beta,\psi,p}$ reduces
to the class $QB_{\beta,p}.$

In [16], the class $QB_{\beta,\psi,p}$ was used to characterize the
boundedness of the operator

$S_{\psi}f(x):=\frac{1}{\Psi(x)}\int_{0}^{x}f(t)\psi(t)dt$

on the cone of functions $f\in Q_{\beta}.$ Precisely, the following was proved
:

Theorem A [16]. _Let $p\geq 1$ and $-1<\beta\leq 0$. Then the inequality_

$\int_{0}^{\infty}{\left(S_{\psi}f\right)}^{p}(x)w(x)dx\leq
C^{\prime}\int_{0}^{\infty}f^{p}(x)w(x)dx$

_holds for all $f\in Q_{\beta}$ if and only if $w\in QB_{\beta,\psi,p},$ where
$C^{\prime}=\frac{C+1}{{(\beta+1})^{p}}$ and $C$ is as in _( 2.1)_._

We define $QB_{\beta,\psi,p}$-constant for a weight $w\in QB_{\beta,\psi,p}$
as follows

$\displaystyle[w]_{QB_{\beta,\psi,p}}:=$
$\displaystyle\inf\left\\{D:\int_{r}^{\infty}\left(\frac{\Psi(r)}{\Psi(x)}\right)^{p}w(x)\,dx\leq(D-1)\int_{0}^{r}\left(\frac{\Psi(x)}{\Psi(r)}\right)^{\beta
p}w(x)dx,r>0\right\\}.$ (2.2)

######  Remark 2.1.

Note that

  1. 1.

$[w]_{QB_{\beta,\psi,p}}>1.$

  2. 2.

For $-1<\beta\leq 0$ and $p\leq q,$ we have $QB_{\beta,\psi,p}\subset
QB_{\beta,\psi,q}.$

We begin with the following :

######  Lemma 2.2.

Let the function $\varphi$ be non-decreasing $(\uparrow)$ defined on
$(0,\infty),~{}f,g\in Q_{\beta}~{}(\beta>-1),0<p_{0}<\infty,~{}\psi\in
L_{loc}^{1}$ be $\downarrow$ and
$\displaystyle\lim_{x\rightarrow\infty}\Psi(x)=\infty.$ Suppose that for each
$w\in QB_{\beta,\psi,p_{0}},$ the inequality

$\int_{0}^{\infty}f(x)w(x)dx\leq\varphi([w]_{QB_{\beta,\psi,p_{0}}})\int_{0}^{\infty}g(x)w(x)dx$

holds. Then for every $0<\varepsilon<p_{0}({\beta+1})$ and $t>0,$ the
following inequality holds:

$\int_{0}^{t}f(s)(\Psi(s))^{p_{0}-1-\varepsilon}\psi(s)ds\leq\varphi\left(\frac{p_{0}(\beta+1)}{\varepsilon}\right)\int_{0}^{t}g(s)(\Psi(s))^{p_{0}-1-\varepsilon}\psi(s)ds.$

###### Proof.

Let $v\in\mathcal{M^{+}_{\downarrow}}.$ Set
$w(x)=v(x)(\Psi(x))^{p_{0}-1-\varepsilon}\psi(x)$ so that $w\in L_{loc}^{1}$.
We claim that $w\in QB_{\beta,\psi,p_{0}}.$ Indeed, we have

$\displaystyle\left(\Psi(r)^{\beta+1}\right)^{p_{0}}\int_{r}^{\infty}\frac{w(x)}{\Psi(x)^{p_{0}}}dx$
$\displaystyle=\left(\Psi(r)^{\beta+1}\right)^{p_{0}}\int_{r}^{\infty}v(x)(\Psi(x))^{-1-\varepsilon}\psi(x)dx$
$\displaystyle\leq\frac{v(r)}{\varepsilon}(\Psi(r))^{p_{0}(\beta+1)-\varepsilon}$
$\displaystyle=\frac{(p_{0}(\beta+1)-\varepsilon)}{\varepsilon}v(r)\int_{0}^{r}(\Psi(x))^{p_{0}(\beta+1)-1-\varepsilon}\psi(x)dx$
$\displaystyle\leq\left(\frac{p_{0}(\beta+1)}{\varepsilon}-1\right)\int_{0}^{r}v(x)(\Psi(x))^{p_{0}(\beta+1)-1-\varepsilon}\psi(x)dx$
$\displaystyle\leq\frac{p_{0}(\beta+1)}{\varepsilon}\int_{0}^{r}(\Psi(x))^{\beta
p_{0}}v(x)(\Psi(x))^{p_{0}-1-\varepsilon}\psi(x)dx$
$\displaystyle=\frac{p_{0}(\beta+1)}{\varepsilon}\int_{0}^{r}(\Psi(x))^{\beta
p_{0}}w(x)dx.$

The assertion now follows on taking $v(x)=\chi_{(0,s]}(x)$ and using the fact
that $[w]_{QB_{\beta,\psi,p_{0}}}\leq\frac{p_{0}(\beta+1)}{\varepsilon}.$ ∎

######  Definition 2.3.

For a given $\beta>-1,$ a weight function $w$ is said to be in the class
$\widehat{Q}B_{\beta,p}$ if
(i) $w\in QB_{\beta,p}$; and
(ii) there exists $0<\varepsilon<p(\beta+1)$ such that $w\in
QB_{\beta,p-\varepsilon}.$

######  Remark 2.4.

The class $\widehat{Q}B_{\beta,p}$ in Definition2.3 is reasonably defined. In
view of Lemma 2.3 [16], it is clear that for $\beta\geq 0,$
$\widehat{Q}B_{\beta,p}=QB_{\beta,p}$. We prove below that for $-1<\beta<0,$
the power weights belong to the class $\widehat{Q}B_{\beta,p}$. It is of
interest if the same can be proved for general weights as well.

######  Lemma 2.5.

Let $1\leq p<\infty,~{}-1<\beta<0$ and $\alpha\in\mathbb{R}$. If
$x^{\alpha}\in QB_{\beta,p},$ then there exists $0<\varepsilon<p(\beta+1)$
such that $x^{\alpha}\in QB_{\beta,p-\varepsilon}.$

###### Proof.

Since $x^{\alpha}\in QB_{\beta,p},$ we have that

$\displaystyle\int_{r}^{\infty}\left(\frac{r}{x}\right)^{p}x^{\alpha}dx\leq
C\int_{0}^{r}\left(\frac{x}{r}\right)^{\beta p}x^{\alpha}dx,~{}r>0$ (2.3)

which holds if and only if

$\displaystyle-\beta p-1<\alpha<p-1.$ (2.4)

Choose $\varepsilon>0$ such that $0<\varepsilon<p-\alpha-1$. Clearly,
$0<\varepsilon<p(\beta+1).$ Now, using the estimates $(\ref{eqn 1})$ and
$(\ref{eqn 2})$ at appropriate places, we obtain

$\displaystyle\int_{r}^{\infty}\left(\frac{r}{x}\right)^{p-\varepsilon}x^{\alpha}dx$
$\displaystyle=\frac{r^{\alpha+1}}{p-\varepsilon-\alpha-1}$
$\displaystyle=\frac{p-\alpha-1}{p-\varepsilon-\alpha-1}\int_{r}^{\infty}\left(\frac{r}{x}\right)^{p}x^{\alpha}dx$
$\displaystyle\leq
C\left(\frac{p-\alpha-1}{p-\varepsilon-\alpha-1}\right)\int_{0}^{r}\left(\frac{x}{r}\right)^{\beta
p}x^{\alpha}dx$ $\displaystyle=\frac{K}{(\alpha+\beta p+1)r^{\beta p}}r^{\beta
p+\alpha+1}$
$\displaystyle=K\left(\frac{\beta(p-\varepsilon)+\alpha+1}{\alpha+\beta
p+1}\right)\int_{0}^{r}\left(\frac{x}{r}\right)^{\beta(p-\varepsilon)}x^{\alpha}dx$

i.e., $x^{\alpha}\in QB_{\beta,p-\varepsilon}$ with the constant

$C^{*}:=K\left(\frac{\beta(p-\varepsilon)+\alpha+1}{\alpha+\beta p+1}\right),$

where $K=C\left(\frac{p-\alpha-1}{p-\varepsilon-\alpha-1}\right)$ and $C$ is
as in (2.3). ∎

######  Remark 2.6.

For $-\beta p-1<\alpha<p-1,$ from Lemma 2.5 and (2.2) it follows that

$[x^{\alpha}]_{QB_{\beta,p-\varepsilon}}\leq
C^{*}+1=C\left(\frac{p-\alpha-1}{p-\varepsilon-\alpha-1}\right)\left(\frac{\beta(p-\varepsilon)+\alpha+1}{\alpha+\beta
p+1}\right)+1.$

We now prove the first main extrapolation theorem:

######  Theorem 2.7.

Let $\varphi\uparrow$ be defined on $(0,\infty),~{}(f,g)$ be a pair of
functions such that $f,g\in Q_{\beta},-1<\beta\leq 0$ and $1\leq
p_{0}<\infty.$ Suppose that for every $w\in{Q}B_{\beta,p_{0}},$ the inequality

$\int_{0}^{\infty}f^{p_{0}}(x)w(x)dx\leq\varphi([w]_{QB_{\beta,p_{0}}})\int_{0}^{\infty}g^{p_{0}}(x)w(x)dx$

holds. Then for all $p_{0}\leq p<\infty$ and all $w\in\widehat{Q}B_{\beta,p},$
the following holds:

$\int_{0}^{\infty}f^{p}(x)w(x)dx\leq C\int_{0}^{\infty}g^{p}(x)w(x)dx,$

where

$C=\displaystyle\inf_{0<\varepsilon<p_{0}(\beta+1)}[w]_{QB_{\beta,(p_{0}-\varepsilon)\frac{p}{p_{0}}}}\left[\frac{1}{\beta+1}\left(\frac{p_{0}(\beta+1)-\varepsilon}{p_{0}-\varepsilon}\right)\varphi\left(\frac{p_{0}(\beta+1)}{\varepsilon}\right)\right]^{p/p_{0}}.$

###### Proof.

The case $\beta=0$ is just Theorem 2.1 of [4]. So, we assume that
$-1<\beta<0.$

Let $p_{0}\leq p<\infty$,$w\in\widehat{Q}B_{\beta,p}$ and
$0<\varepsilon<p_{0}(\beta+1)$. Clearly the function $h(x):=x^{-\beta}f(x)$ is
$\downarrow.$ Note that

$\displaystyle\int_{0}^{\infty}f^{p}(x)w(x)dx=\int_{0}^{\infty}h^{p}(x)w(x)x^{\beta
p}dx.$ (2.5)

Since $h$ is decreasing, we have

$h^{p_{0}}(x)\leq\frac{p_{0}(\beta+1)-\varepsilon}{x^{p_{0}(\beta+1)-\varepsilon}}\int_{0}^{x}h^{p_{0}}(s)s^{p_{0}(\beta+1)-\varepsilon-1}ds$

which together with $(\ref{eqn 4})$ and Lemma 2.2 (for $\psi\equiv 1$) gives

$\displaystyle\int_{0}^{\infty}f^{p}(x)w(x)dx$
$\displaystyle\leq\left(\frac{p_{0}(\beta+1)-\varepsilon}{p_{0}-\varepsilon}\right)^{p/p_{0}}\int_{0}^{\infty}\left(\frac{p_{0}-\varepsilon}{x^{p_{0}-\varepsilon}}\int_{0}^{x}f^{p_{0}}(s)s^{p_{0}-1-\varepsilon}ds\right)^{p/p_{0}}w(x)dx$
$\displaystyle\leq\left(\frac{p_{0}(\beta+1)-\varepsilon}{p_{0}-\varepsilon}\right)^{p/p_{0}}\varphi\left(\frac{p_{0}(\beta+1)}{\varepsilon}\right)^{p/p_{0}}\int_{0}^{\infty}\left(\frac{p_{0}-\varepsilon}{x^{p_{0}-\varepsilon}}\int_{0}^{x}g^{p_{0}}(s)s^{p_{0}-1-\varepsilon}ds\right)^{p/p_{0}}w(x)dx$
$\displaystyle=\gamma\int_{0}^{\infty}\left(\frac{p_{0}-\varepsilon}{x^{p_{0}-\varepsilon}}\int_{0}^{x}g^{p_{0}}(s)s^{p_{0}-1-\varepsilon}ds\right)^{p/p_{0}}w(x)dx$
$\displaystyle=\gamma\int_{0}^{\infty}\left(S_{\psi}g^{p_{0}}(x)\right)^{p/p_{0}}w(x)dx,$
(2.6)

where $\psi(s)=s^{p_{0}-1-\varepsilon}$ and

$\gamma=\left(\frac{p_{0}(\beta+1)-\varepsilon}{p_{0}-\varepsilon}\right)^{p/p_{0}}\varphi\left(\frac{p_{0}(\beta+1)}{\varepsilon}\right)^{p/p_{0}}.$

Now, since $w\in\widehat{Q}B_{\beta,p},$ by definition, there exists
$\widetilde{\varepsilon}>0$ such that $w\in
QB_{\beta,p-\widetilde{\varepsilon}}.$ It is sufficient to take $\varepsilon$
so that $p-\widetilde{\varepsilon}=(p_{0}-\varepsilon)\frac{p}{p_{0}}$ or
$\varepsilon=\frac{p_{0}}{p}\widetilde{\varepsilon}.$ Then $w\in
QB_{\beta,(p_{0}-\varepsilon)\frac{p}{p_{0}}}$ which gives that for all $r>0$
the following inequality holds:

$\int_{r}^{\infty}\left(\frac{r}{x}\right)^{(p_{0}-\varepsilon)\frac{p}{p_{0}}}w(x)dx\leq(A-1)\int_{0}^{r}\left(\frac{x}{r}\right)^{\beta(p_{0}-\varepsilon)\frac{p}{p_{0}}}w(x)dx,$

or

$\int_{r}^{\infty}\left(\frac{\Psi(r)}{\Psi(x)}\right)^{p/p_{0}}w(x)dx\leq(A-1)\int_{0}^{r}\left(\frac{\Psi(x)}{\Psi(r)}\right)^{\beta(p/p_{0})}w(x)dx$

with $\psi(s)=s^{p_{0}-1-\varepsilon},$ which by Theorem A holds if and only
if

$\int_{0}^{\infty}\left(S_{\psi}g^{p_{0}}(x)\right)^{p/p_{0}}w(x)dx\leq\frac{A}{(\beta+1)^{p/p_{0}}}\int_{0}^{\infty}g^{p}(x)w(x)dx,$

where
$A=[w]_{QB_{\beta,(p_{0}-\varepsilon)\frac{p}{p_{0}}}}=[w]_{QB_{\beta,p-\widetilde{\varepsilon}}}$.

Consequently, $(\ref{ineqn z})$ gives

$\displaystyle\int_{0}^{\infty}f^{p}(x)w(x)dx$ $\displaystyle\leq\frac{\gamma
A}{(\beta+1)^{p/p_{0}}}\int_{0}^{\infty}g^{p}(x)w(x)dx$
$\displaystyle=K\int_{0}^{\infty}g^{p}(x)w(x)dx,$

where

$K=[w]_{QB_{\beta,(p_{0}-\varepsilon)\frac{p}{p_{0}}}}\left[\left(\frac{p_{0}(\beta+1)-\varepsilon}{(\beta+1)(p_{0}-\varepsilon)}\right)\varphi\left(\frac{p_{0}(\beta+1)}{\varepsilon}\right)\right]^{p/p_{0}}.$

Since $\varepsilon\in(0,p_{0}(\beta+1))$ is arbitrary, taking infimum over all
such $\varepsilon$, the assertion follows. ∎

In view of the Remark 2.1 (for $\psi\equiv 1$), following the definition of
the class $B_{\infty}$ [4], we define the class $QB_{\beta,\infty}$ as

$QB_{\beta,\infty}:=\bigcup_{p>0}QB_{\beta,p}$

and we also define

$[w]_{QB_{\beta,\infty}}:=\inf\left\\{[w]_{QB_{\beta,p}}:w\in
QB_{\beta,p},~{}p>0\right\\}.$

Similarly, we define

$QB_{\beta,\psi,\infty}:=\bigcup_{p>0}QB_{\beta,\psi,p}$

and

$1\leq[w]_{QB_{\beta,\psi,\infty}}:=\inf\left\\{[w]_{QB_{\beta,\psi,p}}:w\in
QB_{\beta,\psi,p},~{}p>0\right\\}.$

We prove the following:

######  Lemma 2.8.

Let $\psi:\mathbb{R}^{+}\to\mathbb{R}^{+}$ be $\uparrow,~{}v\in L^{1}_{loc}$
be $\downarrow,~{}-1<\beta\leq 0$ and $\alpha>-1.$ Then the function $w$
defined by

$w(x)=\Psi^{\alpha}(x)\psi(x)v(x)$

belongs to the class $QB_{\beta,\psi,\infty}.$

###### Proof.

Let $0<r<\infty$ be arbitrary and choose $p_{0}$ such that
$\alpha+1<p_{0}<-\frac{1}{\beta}(\alpha+1)$. Then we have

$\displaystyle\int_{r}^{\infty}\left(\frac{\Psi^{\beta+1}(r)}{\Psi(x)}\right)^{p_{0}}w(x)dx$
$\displaystyle=(\Psi(r))^{(\beta+1)p_{0}}\int_{r}^{\infty}(\Psi(x))^{\alpha-
p_{0}}\psi(x)v(x)dx$
$\displaystyle\leq\frac{1}{(p_{0}-\alpha-1)}(\Psi(r))^{\beta
p_{0}+\alpha+1}v(r)$ $\displaystyle\leq\left(\frac{\beta
p_{0}+\alpha+1}{p_{0}-\alpha-1}\right)\int_{0}^{r}(\Psi(x))^{\beta
p_{0}+\alpha}\psi(x)v(x)dx$ $\displaystyle=\left(\frac{\beta
p_{0}+\alpha+1}{p_{0}-\alpha-1}\right)\int_{0}^{r}(\Psi(x))^{\beta
p_{0}}w(x)dx$

and the assertion follows. Moreover,
$[w]_{QB_{\beta,\psi,\infty}}\leq\frac{\beta
p_{0}+\alpha+1}{p_{0}-\alpha-1}+1.$ ∎

Below, we prove an extrapolation result for $QB_{\beta,\infty}$\- class of
weights.

######  Theorem 2.9.

Let $\varphi$ be $\uparrow$ defined on $(0,\infty),-1<\beta\leq 0,(f,g)$ be a
pair of functions such that $f,g\in Q_{\beta}$ and $0<p_{0}<\infty$. Suppose
that for every weight $w\in QB_{\beta,\infty},$ the inequality

$\displaystyle\int_{0}^{\infty}f^{p_{0}}(t)w(t)dt\leq\varphi([w]_{QB_{\beta,\infty}})\int_{0}^{\infty}g^{p_{0}}(t)w(t)dt$
(2.7)

holds. Then for every $p_{0}\leq p<\infty$ and $w\in QB_{\beta,\infty}$ the
following holds

$\int_{0}^{\infty}f^{p}(t)w(t)dt\leq K\int_{0}^{\infty}g^{p}(t)w(t)dt,$

with

$K=\inf_{\alpha>-1}[w]_{QB_{\beta,\frac{(\alpha+1)p}{p_{0}}}}\left(\frac{\varphi(1)}{\beta+1}\right)^{p/p_{0}}.$

###### Proof.

For $s>0$ and $\alpha>-1,$ consider the following

$\widetilde{w}(t)=\chi_{(0,s)}(t)t^{\alpha}.$

Clearly by Lemma $\ref{lemma c},~{}\widetilde{w}\in QB_{\beta,\infty}.$ Then,
in view of Remark 2.1 and Lemma 2.8, we have

$1\leq[\widetilde{w}]_{QB_{\beta,\psi,\infty}}\leq\displaystyle\lim_{p_{0}\rightarrow\infty}\frac{\beta
p_{0}+\alpha+1}{p_{0}-\alpha-1}+1=\beta+1\leq 1$

and consequently, in view of $(\ref{eqn 5})$ the following holds

$\displaystyle\int_{0}^{s}f^{p_{0}}(t)t^{\alpha}dt\leq\varphi(1)\int_{0}^{s}g^{p_{0}}(t)t^{\alpha}dt.$
(2.8)

Since $t^{-\beta}f(t)$ is $\downarrow$, we find that

$\displaystyle f^{p_{0}}(t)$
$\displaystyle=\frac{\alpha+1}{t^{\alpha+1}}\int_{0}^{t}f^{p_{0}}(t)s^{\alpha}ds$
$\displaystyle=\frac{\alpha+1}{t^{\alpha+1}}\int_{0}^{t}(t^{-\beta}f(t))^{p_{0}}t^{\beta
p_{0}}s^{\alpha}ds$
$\displaystyle\leq\frac{\alpha+1}{t^{\alpha+1}}\int_{0}^{t}(s^{-\beta}f(s))^{p_{0}}t^{\beta
p_{0}}s^{\alpha}ds$
$\displaystyle=\frac{\alpha+1}{t^{\alpha+1}}\int_{0}^{t}f^{p_{0}}(s)\left(\frac{t}{s}\right)^{\beta
p_{0}}s^{\alpha}ds$
$\displaystyle\leq\frac{\alpha+1}{t^{\alpha+1}}\int_{0}^{t}f^{p_{0}}(s)s^{\alpha}ds$

which in view of $(\ref{eqn 6})$ gives

$\displaystyle\int_{0}^{\infty}f^{p}(t)w(t)dt$
$\displaystyle\leq\int_{0}^{\infty}\left(\frac{\alpha+1}{t^{\alpha+1}}\int_{0}^{t}f^{p_{0}}(s)s^{\alpha}ds\right)^{p/p_{0}}w(t)dt$
$\displaystyle\leq\varphi(1)^{p/p_{0}}\int_{0}^{\infty}\left(\frac{\alpha+1}{t^{\alpha+1}}\int_{0}^{t}g^{p_{0}}(s)s^{\alpha}ds\right)^{p/p_{0}}w(t)dt$
$\displaystyle=\varphi(1)^{p/p_{0}}\int_{0}^{\infty}\left(S_{\psi}g^{p_{0}}(t)\right)^{p/p_{0}}w(t)dt,$
(2.9)

with $\psi(s)=s^{\alpha}.$

Now, let $w\in QB_{\beta,\infty}.$ Then there exists $q>0$ such that $w\in
QB_{\beta,q}.$ We can choose $\alpha>-1$ such that
$q=(\alpha+1)\frac{p}{p_{0}}.$ Then $w\in
QB_{\beta,(\alpha+1)\frac{p}{p_{0}}},$ which in view of $(\ref{eqn aa})$
implies that the following holds for all $r>0:$

$\int_{r}^{\infty}\left(\frac{r}{t}\right)^{(\alpha+1)\frac{p}{p_{0}}}w(t)dt\leq(C-1)\int_{0}^{r}\left(\frac{t}{r}\right)^{\beta(\alpha+1)\frac{p}{p_{0}}}w(t)dt$

or, equivalently

$\int_{r}^{\infty}\left(\frac{\Psi(r)}{\Psi(t)}\right)^{p/p_{0}}w(t)dt\leq(C-1)\int_{0}^{r}\left(\frac{\Psi(t)}{\Psi(r)}\right)^{\beta
p/p_{0}}w(t)dt$

with $\psi(s)=s^{\alpha}$. But the last inequality, in view of Theorem A,
holds if and only if

$\displaystyle\int_{0}^{\infty}\left(S_{\psi}g^{p_{0}}(t)\right)^{p/p_{0}}w(t)dt\leq\frac{C}{(\beta+1)^{p/p_{0}}}\int_{0}^{\infty}g^{p}(t)w(t)dt,$
(2.10)

where $C=[w]_{QB_{\beta,(\alpha+1)\frac{p}{p_{0}}}}.$ Now $(\ref{ineq a1})$
and $(\ref{eqn 7})$ give that

$\int_{0}^{\infty}f^{p}(t)w(t)dt\leq[w]_{QB_{\beta,(\alpha+1)\frac{p}{p_{0}}}}\left(\frac{\varphi(1)}{\beta+1}\right)^{p/p_{0}}\int_{0}^{\infty}g^{p}(t)w(t)dt,$

so that on taking the infimum over all $\alpha>-1,$ the assertion follows. ∎

##  3 Extrapolation results in grand Lebesgue spaces

In this section, we shall prove a version of the extrapolation result (Theorem
2.7) in the framework of grand Lebesgue spaces defined on finite intervals,
which without any loss of generality is taken as $I=(0,1)$.

Let $0<p<\infty$ and $-1<\beta<\infty$. We say that a weight function $w$ on
$I$ belongs to the class$QB_{\beta,p}(I)$ if there exists a constant $C>0$
such that the inequality:

$\int_{r}^{1}\left(\frac{r}{t}\right)^{p}w(t)dt\leq
C\int_{0}^{r}\left(\frac{t}{r}\right)^{\beta p}w(t)dt$

holds for all $0<r\leq 1.$ Also, for $0<r\leq 1,$ we set

$[w]_{QB_{\beta,p}(I)}:=\inf\left\\{C>1:\int_{r}^{1}\left(\frac{r}{t}\right)^{p}w(t)dt\leq(C-1)\int_{0}^{r}\left(\frac{t}{r}\right)^{\beta
p}w(t)dt\right\\}.$

It can be seen that if $0<p<\infty$ and $w\in QB_{\beta,p}(I),$ then the
function $\widetilde{w}=w\chi_{I}\in QB_{\beta,p}$ and

$[w]_{QB_{\beta,p}(I)}=[\widetilde{w}]_{QB_{\beta,p}}.$

Following the arguments used in Lemma 2.5, we can prove:

######  Lemma 3.1.

Let $-1<\beta\leq 0$ and $1\leq p<\infty.$ If $x^{\alpha}\in QB_{\beta,p}(I),$
then there exists $0<\varepsilon<p(\beta+1)$ such that $x^{\alpha}\in
QB_{\beta,p-\varepsilon}(I).$

######  Definition 3.2.

For a given $-1<\beta<\infty,$ a weight function
$w\in\widehat{Q}B_{\beta,p}(I)$ if
(i) $w\in QB_{\beta,p}(I)$; and
(ii) there exists $0<\varepsilon<p(\beta+1)$ such that $w\in
QB_{\beta,p-\varepsilon}(I).$

######  Remark 3.3.

It can be checked that for $-1<\beta\leq 0,$ the power weights $x^{\alpha}\in
QB_{\beta,p}(I)$ if and only if $-\beta p-1<\alpha<p-1.$ Then, in view of
Lemma $\ref{prop 1},$ the class $\widehat{Q}B_{\beta,p}(I)$ is reasonably
defined.

It is seen that Theorem 2.7 can be modified for the interval $I.$ We state it
formally for later purpose.

######  Theorem 3.4.

Let $\varphi\uparrow$ be defined on $\mathbb{R}^{+}$ and $(f,g)$ be a pair of
functions such that $f,g\in Q_{\beta}(I),~{}-1<\beta\leq 0$. Let $1\leq
p_{0}<\infty$ and that for every weight function $w\in QB_{\beta,p_{0}}(I),$
the inequality

$\int_{0}^{1}f^{p_{0}}(x)w(x)dx\leq\varphi\left([w]_{QB_{\beta,p_{0}}(I)}\right)\int_{0}^{1}g^{p_{0}}(x)w(x)dx$

holds. Then for every $p_{0}\leq p<\infty$ and every
$w\in\widehat{Q}B_{\beta,p}(I),$ the following inequality holds:

$\int_{0}^{1}f^{p}(x)w(x)dx\leq K^{\prime}(p)\int_{0}^{1}g^{p}(x)w(x)dx,$

where

$K^{\prime}(p):=\displaystyle\inf_{0<\delta<p_{0}(\beta+1)}[w]_{QB_{\beta,(p_{0}-\delta)\frac{p}{p_{0}}}(I)}\left[\frac{1}{\beta+1}\left(\frac{p_{0}(\beta+1)-\delta}{p_{0}-\delta}\right)\varphi\left(\frac{p_{0}(\beta+1)}{\delta}\right)\right]^{p/p_{0}}.$

In this section, we shall prove Theorem 2.7 in the framework of grand Lebesgue
spaces $L^{p),\theta}(I)$ which consist of all measurable functions $f$ finite
a.e. on $I$ for which

$\|f\|_{L^{p),\theta}(I)}:=\sup_{0<\varepsilon<p-1}\left(\varepsilon^{\theta}\int_{0}^{1}|f(t)|^{p-\varepsilon}dt\right)^{1/(p-\varepsilon)}<\infty.$

These spaces without weight have been defined in [10], which in fact, were
initially defined for $\theta=1$ by Iwaniec and Sbordone [12] and later have
been generalized, studied and applied by several people in different
directions. We refer to [15] and the references therein. For some very recent
updates on grand Lebesgue spaces, we mention [8], [11], [14], [17], [18],
[19], [23].

We now prove the following:

######  Theorem 3.5.

Let $\theta>0,~{}\varphi$ be a non-negative $\uparrow$ function defined on
$(0,\infty),~{}-1<\beta\leq 0,1<p_{0}<\infty$ and $(f,g)$ be a pair of
functions such that $f,g\in Q_{\beta}(I)$. Suppose that for every $w\in
QB_{\beta,p_{0}}(I)$, the following inequality holds:

$\int_{0}^{1}f^{p_{0}}(x)w(x)dx\leq\varphi\left([w]_{QB_{\beta,p_{0}}(I)}\right)\int_{0}^{1}g^{p_{0}}(x)w(x)dx.$

Then for every $p:~{}p_{0}\leq p<\infty$ and every
$w\in\widehat{Q}B_{\beta,p}(I),$ the inequality

$\|f\|_{L_{w}^{p),\theta}(I)}\leq C^{*}\|g\|_{L_{w}^{p),\theta}(I)}$

holds with

$C^{*}=\displaystyle\inf_{0<\sigma<p-1}\left[\max\left\\{1,p^{\theta}\sigma^{-\frac{\theta}{p-\sigma}}\left(W(I)+1\right)^{\frac{p-1-\sigma}{p-\sigma}}\right\\}\sup_{0<\varepsilon\leq\sigma}{(K^{\prime}(p-\varepsilon))}^{\frac{1}{p-\varepsilon}}\right].$

###### Proof.

Let $w\in\widehat{Q}B_{\beta,p}(I),$ then by definition $w\in
QB_{\beta,p}(I),$ and there exists $0<\xi<p(\beta+1)$ such that $w\in
QB_{\beta,p-\xi}(I).$ Take $\sigma=\min\\{\xi,p-p_{0}\\}.$ Clearly
$0<\sigma<p-1,$ so, by Remark 2.1, $w\in QB_{\beta,p-\sigma}(I).$ Let
$\varepsilon\in(0,\sigma).$ Then, in view of the fact that
$\widehat{Q}B_{\beta,p}\subset\widehat{Q}B_{\beta,q}$ for $p<q,$ we have
$w\in\widehat{Q}B_{\beta,p-\varepsilon}(I).$ Therefore, by Theorem 3.4 we have

$\displaystyle\int_{0}^{1}f^{p-\varepsilon}(x)w(x)dx\leq
K^{\prime}(p-\varepsilon)\int_{0}^{1}g^{p-\varepsilon}(x)w(x)dx,$ (3.1)

where

$K^{\prime}(p-\varepsilon):=\displaystyle\inf_{0<\delta<p_{0}(\beta+1)}[w]_{QB_{\beta,(p_{0}-\delta)\frac{p-\varepsilon}{p_{0}}}(I)}\left[\frac{1}{\beta+1}\left(\frac{p_{0}(\beta+1)-\delta}{p_{0}-\delta}\right)\varphi\left(\frac{p_{0}(\beta+1)}{\delta}\right)\right]^{p-\varepsilon/p_{0}}.$

Now, for $\sigma<\varepsilon<p-1,$ using Hölder’s inequality with the indices
$\frac{p-\sigma}{p-\varepsilon}$ and $\frac{p-\sigma}{\varepsilon-\sigma},$ we
obtain

$\displaystyle\|f\|_{L_{w}^{p-\varepsilon}}$
$\displaystyle=\left(\int_{0}^{1}f^{p-\varepsilon}(x)w(x)dx\right)^{1/{p-\varepsilon}}$
$\displaystyle\leq\left(\int_{0}^{1}f^{p-\sigma}(x)w(x)dx\right)^{\frac{1}{p-\sigma}}\left(W(I)\right)^{\frac{\varepsilon-\sigma}{(p-\sigma)(p-\varepsilon)}}$
$\displaystyle\leq\left(\int_{0}^{1}f^{p-\sigma}(x)w(x)dx\right)^{\frac{1}{p-\sigma}}\left(W(I)+1\right)^{\frac{p-1-\sigma}{p-\sigma}}.$
(3.2)

Now in view of $(\ref{eqn31})$ and $(\ref{ineq z7}),$ we get

$\displaystyle\|f\|_{L_{w}^{p),\theta}(I)}$
$\displaystyle=\max\left\\{\sup_{0<\varepsilon\leq\sigma}\varepsilon^{\frac{\theta}{p-\varepsilon}}\|f\|_{L_{w}^{p-\varepsilon}},\sup_{\sigma<\varepsilon<p-1}\varepsilon^{\frac{\theta}{p-\varepsilon}}\|f\|_{L_{w}^{p-\varepsilon}}\right\\}$
$\displaystyle\leq\max\left\\{\sup_{0<\varepsilon\leq\sigma}\varepsilon^{\frac{\theta}{p-\varepsilon}}\|f\|_{L_{w}^{p-\varepsilon}},\sup_{\sigma<\varepsilon<p-1}\varepsilon^{\frac{\theta}{p-\varepsilon}}\|f\|_{L_{w}^{p-\sigma}}\left(W(I)+1\right)^{\frac{p-1-\sigma}{p-\sigma}}\right\\}$
$\displaystyle\leq\max\left\\{1,p^{\theta}\sigma^{-\frac{\theta}{p-\sigma}}\left(W(I)+1\right)^{\frac{p-1-\sigma}{p-\sigma}}\right\\}\sup_{0<\varepsilon\leq\sigma}\varepsilon^{\frac{\theta}{p-\varepsilon}}\|f\|_{L_{w}^{p-\varepsilon}}$
$\displaystyle\leq\max\left\\{1,p^{\theta}\sigma^{-\frac{\theta}{p-\sigma}}\left(W(I)+1\right)^{\frac{p-1-\sigma}{p-\sigma}}\right\\}\sup_{0<\varepsilon\leq\sigma}\varepsilon^{\frac{\theta}{p-\varepsilon}}(K^{\prime}(p-\varepsilon))^{\frac{1}{p-\varepsilon}}\|g\|_{L_{w}^{p-\varepsilon}}$
$\displaystyle\leq\max\left\\{1,p^{\theta}\sigma^{-\frac{\theta}{p-\sigma}}\left(W(I)+1\right)^{\frac{p-1-\sigma}{p-\sigma}}\right\\}\sup_{0<\varepsilon\leq\sigma}(K^{\prime}(p-\varepsilon))^{\frac{1}{p-\varepsilon}}\|g\|_{L_{w}^{p),\theta}(I)}$
$\displaystyle=C^{*}\|g\|_{L_{w}^{p),\theta}(I)},$

where
$C^{*}=c(p,\theta,\sigma)\displaystyle\sup_{0<\varepsilon\leq\sigma}(K^{\prime}(p-\varepsilon))^{\frac{1}{p-\varepsilon}}$
and

$c(p,\theta,\sigma)=\max\left\\{1,~{}p^{\theta}\sigma^{-\frac{\theta}{p-\sigma}}\left(W(I)+1\right)^{\frac{p-1-\sigma}{p-\sigma}}\right\\}.$

The proof is completed. ∎

##  4 Application

We provide an application of the extrapolation result proved in the previous
section to characterize the boundedness of the Hardy averaging operator $H$
between weighted grand Lebesgue spaces $L_{w}^{p),\theta}(I)$ for quasi-
monotone functions. We prove the following:

######  Theorem 4.1.

Let $1<p<\infty,-1<\beta\leq 0$ and $\theta>0.$ The inequality

$\displaystyle\|Hf\|_{L_{w}^{p),\theta}(I)}\leq C\|f\|_{L_{w}^{p),\theta}(I)}$
(4.1)

holds for all $f\in Q_{\beta}(I)$ if and only if
$w\in\widehat{Q}B_{\beta,p}(I).$

###### Proof.

Let us first assume that $w\in\widehat{Q}B_{\beta,p}(I).$ Note that if $f\in
Q_{\beta}(I),$ then for $0<t\leq s$ and $\alpha\in I,$ we have that

$t^{-\beta}f\left(\frac{\alpha t}{s}\right)\geq s^{-\beta}f(\alpha)$

using which we get that

$\displaystyle s^{-\beta}Hf(s)$
$\displaystyle\leq\frac{1}{s}\int_{0}^{s}t^{-\beta}f\left(\frac{\alpha
t}{s}\right)d\alpha$ $\displaystyle=t^{-\beta-1}\int_{0}^{t}f(z)dz$
$\displaystyle=t^{-\beta}Hf(t)$

i.e., $Hf\in Q_{\beta}(I).$

Further, on taking $\psi\equiv 1$ in a modified form of Theorem A, and
considering the functions $f$ defined on $I$ instead of $(0,\infty),$ we see
that the inequality

$\int_{0}^{1}\left(Hf(x)\right)^{p}w(x)dx\leq C\int_{0}^{1}f^{p}(x)w(x)dx$

holds. Now, in view of Theorem $\ref{thm 3},$ the inequality $(\ref{eqn z})$
holds.

Conversely, assume that the inequality $(\ref{eqn z})$ holds. Consider the
test function $f_{r}(x)=x^{\beta}\chi_{(0,r)}(x)$ for $0<r<1.$ Then

$\displaystyle\|f_{r}\|_{L_{w}^{p),\theta}(I)}$
$\displaystyle=\sup_{0<\varepsilon<p-1}\left(\varepsilon^{\theta}\int_{0}^{r}x^{\beta(p-\varepsilon)}w(x)dx\right)^{\frac{1}{p-\varepsilon}}$
$\displaystyle=\max\left\\{\sup_{0<\varepsilon\leq\sigma}\varepsilon^{\frac{\theta}{p-\varepsilon}}\|f_{r}\|_{L_{w}^{p-\varepsilon}},\sup_{\sigma<\varepsilon<p-1}\varepsilon^{\frac{\theta}{p-\varepsilon}}\|f_{r}\|_{L_{w}^{p-\varepsilon}}\right\\},$

where $\sigma$ is chosen such that
$0<\sigma<\displaystyle\min\\{(\beta+1)p,~{}p-1\\}.$ Now, for
$\sigma<\varepsilon<p-1,$ taking the conjugate indices
$\frac{p-\sigma}{p-\varepsilon}$ and $\frac{p-\sigma}{\varepsilon-\sigma},$ on
using Hölder’s inequality we obtain

$\displaystyle\|f_{r}\|_{L_{w}^{p-\varepsilon}}$
$\displaystyle\leq\left(\int_{0}^{r}x^{\beta(p-\sigma)}w(x)dx\right)^{\frac{1}{p-\sigma}}\left(W(I)\right)^{\frac{\varepsilon-\sigma}{(p-\sigma)(p-\varepsilon)}}$
$\displaystyle\leq\left(\int_{0}^{r}x^{\beta(p-\sigma)}w(x)dx\right)^{\frac{1}{p-\sigma}}\left(W(I)+1\right)^{\frac{p-1-\sigma}{p-\sigma}}.$
(4.2)

Thus, on using (4) and an argument from ([22], Theorem 3.1), we have

$\displaystyle\|f_{r}\|_{L_{w}^{p),\theta}(I)}$
$\displaystyle\leq\max\left\\{1,p^{\theta}\sigma^{-\frac{\theta}{p-\sigma}}\left(W(I)+1\right)^{\frac{p-1-\sigma}{p-\sigma}}\right\\}\sup_{0<\varepsilon\leq\sigma}\varepsilon^{\frac{\theta}{p-\varepsilon}}\|f_{r}\|_{L_{w}^{p-\varepsilon}}$
$\displaystyle=\max\left\\{1,p^{\theta}\sigma^{-\frac{\theta}{p-\sigma}}\left(W(I)+1\right)^{\frac{p-1-\sigma}{p-\sigma}}\right\\}\varepsilon_{r}^{\frac{\theta}{p-\varepsilon_{r}}}\|f_{r}\|_{L_{w}^{p-\varepsilon_{r}}}$
$\displaystyle=C_{1}\left(\varepsilon_{r}^{\theta}\int_{0}^{r}x^{\beta(p-\varepsilon_{r})}w(x)dx\right)^{\frac{1}{p-\varepsilon_{r}}}$
(4.3)

for some $0<\varepsilon_{r}\leq\sigma,$ where
$C_{1}:=\displaystyle\inf_{0<\sigma<(\beta+1)p}\max\left\\{1,p^{\theta}\sigma^{-\frac{\theta}{p-\sigma}}\left(W(I)+1\right)^{\frac{p-1-\sigma}{p-\sigma}}\right\\}.$
Further, note that

$\displaystyle\int_{0}^{1}\left(Hf_{r}(x)\right)^{p-\varepsilon}w(x)dx$
$\displaystyle\geq\int_{r}^{1}\left(Hf_{r}(x)\right)^{p-\varepsilon}w(x)dx$
$\displaystyle=\left(\frac{r^{\beta+1}}{\beta+1}\right)^{p-\varepsilon}\int_{r}^{1}\frac{w(x)}{x^{p-\varepsilon}}dx$

so that

$\displaystyle\|Hf_{r}\|_{L_{w}^{p),\theta}(I)}$
$\displaystyle\geq\frac{r^{\beta+1}}{\beta+1}\sup_{0<\varepsilon<p-1}\left(\varepsilon^{\theta}\int_{r}^{1}\frac{w(x)}{x^{p-\varepsilon}}dx\right)^{\frac{1}{p-\varepsilon}}$
$\displaystyle\geq\frac{r^{\beta+1}}{\beta+1}\left({\varepsilon_{r}}^{\theta}\int_{r}^{1}\frac{w(x)}{x^{p-\varepsilon_{r}}}dx\right)^{\frac{1}{p-\varepsilon_{r}}}.$

The above estimate together with (4), and the assumption that $(\ref{eqn z})$
holds, gives that

$\frac{r^{\beta+1}}{\beta+1}\left({\varepsilon_{r}}^{\theta}\int_{r}^{1}\frac{w(x)}{x^{p-\varepsilon_{r}}}dx\right)^{\frac{1}{p-\varepsilon_{r}}}\leq
CC_{1}\left({\varepsilon_{r}}^{\theta}\int_{0}^{r}x^{\beta(p-\varepsilon_{r})}w(x)dx\right)^{\frac{1}{p-\varepsilon_{r}}}.$

Therefore,

$\displaystyle\int_{r}^{1}\left(\frac{r}{x}\right)^{p-\varepsilon_{r}}w(x)dx$
$\displaystyle\leq\left(CC_{1}(\beta+1)\right)^{p-\varepsilon_{r}}\int_{0}^{r}\left(\frac{x}{r}\right)^{\beta(p-\varepsilon_{r})}w(x)dx$
$\displaystyle\leq\left(CC_{1}(\beta+1)+1\right)^{p}\int_{0}^{r}\left(\frac{x}{r}\right)^{\beta(p-\varepsilon_{r})}w(x)dx.$

Thus, $w\in QB_{\beta,p-\varepsilon_{r}}(I),$ where
$0<\varepsilon_{r}<(\beta+1)p.$ Consequently, $w\in QB_{\beta,p}(I)$ and hence
$w\in\widehat{Q}B_{\beta,p}(I).$ ∎

Acknowledgment. The first author acknowledges the MATRICS Research Grant No.
MTR/2019 /000783 of SERB, Department of Science and Technology (DST), India.
Also, the second author acknowledges the research fellowship award No.:
09/045(1716)/2019-EMR-I of Council of Scientific and Industrial Research
(CSIR), INDIA.

Conflict of interest statement. On behalf of all authors, the corresponding
author states that there is no conflict of interest.

## References

  * [1] K.F. Andersen, _Weighted generalized Hardy inequalities for non-increasing functions_ , Can. J. Math., 43 (1991), 1121–1135.
  * [2] M.A. Arino and B. Muckenhoupt, _Maximal functions on classical Lorentz spaces and Hardy’s inequality with weights for non-increasing functions_ , Trans. Amer. Math. Soc., 320 (1990), 727–735.
  * [3] J. Bergh, V. Burenkov and L.E. Persson, _Best constants in reversed Hardy’s inequalities for quasimonotone functions_ , Acta Sci. Math. (Szeged), 59 (1994), 221-239.
  * [4] M.J. Carro and M. Lorente, _Rubio de Francia’s extrapolation theorem for $B_{p}$ weights_, Proc. Amer. Math. Soc., 138 (2010), 629–640.
  * [5] D.V. Cruze-Uribe, J.M. Martell and C. Pérez, _Extrapolation from $A_{\infty}$ weights and applications_, J. Func. Anal., 213 (2004), 412–439.
  * [6] D.V. Cruze-Uribe, J.M. Martell and C. Pérez, _Weights, Extrapolation and the Theory of Rubio de Francia_ , Birkh$\ddot{a}$user, 2011.
  * [7] A. Fiorenza, B. Gupta and P. Jain, _The maximal theorem for weighted grand Lebesgue spaces_ , Studia Math., 188 (2008), 123–133.
  * [8] A. Fiorenza and V. Kokilashvili, _Nonlinear harmonic analysis of integral operators in weighted grand Lebesgue spaces and applications_ , Ann. Funct. Anal., 2017, 1-13. doi.org/10.1215/20088752-2017-0056
  * [9] J. Garcia-Cuerva and A.E. Gatto, _Boundedness properties of fractional integral operators associated to non-doubling measures_ , Studia Math., 162 (2004), 245–-261.
  * [10] L. Greco, T. Iwaniec and C. Sbordone, _Inverting the $p$-harmonic operator_, Manuscripta Math. 92 (1997), 249-258.
  * [11] T. Hagverdi, _On Stability of Bases Consisting of Perturbed Exponential Systems in Grand Lebesgue Spaces_ , J. Contemp. Appl. Math., 2 (2021), 81-92.
  * [12] T. Iwaniec, C. Sbordone, _On the integrability of the Jacobian under minimal hypotheses_ , Arch. Ration. Mech. Anal. 119 (1992), 129-143.
  * [13] P. Jain and S. Kumari, _On grand Lorentz spaces and the maximal operator_ , Georgian Math. J., 19 (2012), 235-246.
  * [14] P. Jain, A. Molchanova, M. Singh and S. Vodopyanov, _On grand Sobolev spaces and pointwise description of Banach function spaces_ , Nonlinear Analysis, 202 (2021), 1-17. doi.org/10.1016/j.na.2020.112100.
  * [15] P. Jain, M. Singh and A.P. Singh, _Recent trends in grand Lebesgue spaces_ in Function Spaces and Inequalities, Springer Proceedings in Mathematics and Statistics, Volume 206 (2015), 137-159, New Delhi, India.
  * [16] P. Jain, M. Singh and A.P. Singh, _Hardy-type integral inequalities for quasi-monotone functions_ , Georgian Math. J., 24 (2016), 523-533.
  * [17] V. M. Kokilashvili, _Weighted grand Lebesgue space with a mixed norm and integral operators_ , Doklady Mathematics, 100 (2019), 549–550.
  * [18] V. Kokilashvili, and A. Meskhi, _Extrapolation in Grand Lebesgue Spaces with $A_{\infty}$ Weights_, Mathematical Notes, 104 (2018), 518–529.
  * [19] V. Kokilashvili and A. Meskhi, _On integral operators in weighted grand Lebesgue spaces of Banach-valued functions_ , Math Meth Appl Sci., 2020, 1–17. doi.org/10.1002/mma.6779
  * [20] L. Maligranda, _Weighted inequalities for monotone functions_ , Collect. Math. 48 (1997), 687-700.
  * [21] A. Meskhi, _Criteria for the boundedness of potential operators in grand Lebesgue spaces_ , Proc. A. Razmadze Math. Inst., 169 (2015), 119–132.
  * [22] A. Meskhi, _Weighted criteria for the Hardy transform under the $B_{p}$ condition in grand Lebesgue spaces and some applications_, J. Math Scs., 178 (2011), 622-636.
  * [23] A. Molchanova, _A note on the continuity of minors in grand Lebesgue spaces_ , J. Fixed Point Theory Appl., 2019, pp.1-13. doi.org/10.1007/s11784-019-0686-y
  * [24] B. Muckenhoupt, _Weighted norm inequalities for the Hardy maximal function_ , Trans. Amer. Math. Soc., 165 (1972), 207-226.
  * [25] J.L. Rubio de Francia, _Factorization theory and $A_{p}$ weights_, Amer. J. Math., 106 (1984), 533–547.
  * [26] E. Sawyer, _Boundedness of classical operators on classical Lorentz spaces_ , Studia Math., 96 (1990), 145–158.


# LoReTrack: Efficient and Accurate Low-Resolution Transformer Tracking

Shaohua Dong1, Yunhe Feng1, Qing Yang1, Yuewei Lin2, Heng Fan1
1Department of Computer Science and Engineering, University of North Texas
2Brookhaven National Laboratory
<EMAIL_ADDRESS><EMAIL_ADDRESS>
###### Abstract

High-performance Transformer trackers have shown excellent results, yet they
often bear a heavy computational load. Observing that a smaller input can
_immediately_ and _conveniently_ reduce computations without changing the
model, an easy solution is to adopt the low-resolution input for efficient
Transformer tracking. Albeit faster, this hurts tracking accuracy much due to
information loss in low resolution tracking. In this paper, we aim to mitigate
such information loss to boost the performance of the low-resolution
Transformer tracking via dual knowledge distillation from a frozen high-
resolution (but _not_ a larger) Transformer tracker. The core lies in two
simple yet effective distillation modules, comprising query-key-value
knowledge distillation (QKV-KD) and discrimination knowledge distillation
(Disc-KD), across resolutions. The former, from the global view, allows the
low-resolution tracker to inherit the features and interactions from the high-
resolution tracker, while the later, from the target-aware view, enhances the
target-background distinguishing capacity via imitating discriminative regions
from its high-resolution counterpart. With the dual knowledge distillation,
our Low-Resolution Transformer Tracker (_LoReTrack_) enjoys not only _high
efficiency_ owing to reduced computation but also _enhanced accuracy_ by
distilling knowledge from the high-resolution tracker. In extensive
experiments, LoReTrack with a 2562 resolution consistently improves baseline
with the same resolution, and shows competitive or even better results
compared to 3842 high-resolution Transformer tracker, while running 52% faster
and saving 56% MACs. Moreover, LoReTrack is resolution-scalable. With a 1282
resolution, it runs 25 _fps_ on a CPU with 64.9%/46.4% SUC scores on
LaSOT/LaSOText, surpassing all other CPU real-time trackers. Code will be
released.

##  1 Introduction

Visual tracking aims to continuously locate the target of interest throughout
a video. It is a fundamental computer vision problem, and has received
extensive attention in the past decades for its crucial roles in many
applications including visual surveillance, robotics, and others. With the
introduction of Transformer vaswani2017attention ; dosovitskiy2020image in
recent years, considerable progress has been achieved in tracking community.
Many high-performance Transformer trackers (e.g., lin2022swintrack ;
ye2022joint ; cui2022mixformer ; chen2023seqtrack ; wei2023autoregressive )
have been proposed and exhibited previously unattainable accuracy. Despite
this, they usually suffer from heavy computational burden, which largely
limits their practical deployment to real applications.

In order to overcome this limitation, existing solutions mostly focus on
reducing the model complexity, such as leveraging a lighter Transformer
backbone (e.g., kang2023exploring ), compressing the Transformer backbone
(e.g., cui2024mixformerv2 ), and developing compact Transformer attentions
(e.g., blatter2023efficient ), to improve the running efficiency of
Transformer trackers. In addition to model complexity, another _crucial
factor_ affecting computation and tracking efficiency is input resolution
tan2019efficientnet . For example, when decreasing input resolution111For
simplicity, input resolution $k^{2}$ (i.e., $k\times k$) indicates the
resolution for the search region of a Transformer tracker, and the resolution
of its template is usually half that of search region unless otherwise
specified. of the popular one-stream Transformer tracker OSTrack ye2022joint
(e.g., 3842$\rightarrow$2562$\rightarrow$1282$\rightarrow$962), its
computation in Multiply–Accumulate Operations (MACs), as displayed in Fig. 1
(a), is _immediately_ and _greatly_ reduced (e.g.,
65.3$\rightarrow$29.0$\rightarrow$7.2$\rightarrow$4.1), without changing the
tracking model, and the GPU/CPU speeds are largely increased as revealed in
Fig. 1 (c). Such an observation is seen in many other Transformer trackers
(e.g., ye2022joint ; lin2022swintrack ; cui2022mixformer ), and hence they
usually devise a low-resolution speed-oriented variant for efficient visual
tracking.

Figure 1: Comparison of computation (a), accuracy (b), and GPU/CPU speed in
different resolutions for OSTrack ye2022joint and LoReTrack on LaSOT
fan2019lasot . Please note, candidate elimination is not used in both methods.
_Best viewed in color and by zooming in for all figures_.

Albeit faster, simply reducing the input resolution often hurts tracking
accuracy much (e.g., 71.0 $\rightarrow$68.7$\rightarrow$62.2$\rightarrow$59.0
in success score), as shown in Fig. 1 (b). The major reason resulting in such
a performance drop is _spatial information loss_ including two aspects. On one
hand, downsizing the input image to a smaller scale inevitably loses spatial
details in pixels, which may degrade feature extraction wang2016studying . On
the other hand, when lowering the resolution for the Transformer tracker,
e.g., ye2022joint , the tokens of the input become much coarser using the same
patch size (e.g.,16$\times$16) and decrease in quantity (e.g.,
242$\rightarrow$162$\rightarrow$82$\rightarrow$62). Despite largely saving the
computation of attention operations, these coarse-grained tokens are _not_
capable of capturing fine-grained information interaction within the image
that is crucial for discriminative feature learning for enhancing performance
in complicated scenarios wang2021not . One solution is to reduce patch size to
allow more fine-grained tokens for interaction as in a high-resolution model,
while this leads to increased computation. Considering that a trained high-
resolution Transformer tracker has already preserved crucial information by
learning from fine-grained token interaction in the image, we pose a question:
_Can we borrow an off-the-shelf high-resolution model to mitigate information
loss in low-resolution Transformer tracking to enhance its accuracy while
maintaining high efficiency?_

Our Solution. We show that the answer is _positive_ by introducing a simple
and novel _dual knowledge distillation_ framework, that allows a low-
resolution Transformer tracker to learn fine-grained knowledge from its high-
resolution counterpart for improving accuracy without sacrificing speed. The
crux of our framework lies in two meticulously devised knowledge distillation
modules, including _query-key-value knowledge distillation_ (QKV-KD) and
_discrimination knowledge distillation_ (Disc-KD), across resolutions.
Specifically, QKV-KD, from the holistic perspective, allows the low-resolution
tracker to implicitly inherit more discriminative features and fine-grained
interactions in the image from a high-resolution tracker. Compared with common
distillation on feature maps (generated by query/key/value) for Transformer,
our QKV-KD enables comprehensive knowledge learning from high-resolution
feature in multiple views, resulting in better low-resolution feature
representation. Besides, another benefit of QKV-KD is indirect knowledge
distillation on attention maps, because query and key are directly leveraged
for attentional relation computation, better facilitating knowledge learning
from the high-resolution Transformer tracker. To further boost the low-
resolution tracking, Disc-KD is applied to improve its discriminative ability.
Particularly, Disc-KD, from the target-aware view, works to imitate
discriminative region generated from the high-resolution model, and enhances
the target-background distinguishing capacity of the low-resolution tracker
for better performance.

With dual knowledge distillation by QKV-KD and Disc-KD, the Low-Resolution
Transformer tracker, coined as LoReTrack, enjoys not only _high efficiency_
owing to reduced computation using a smaller input, but also _enhanced
accuracy_ via distilling discriminative and fine-grained knowledge from the
high-resolution tracker. Please notice that, although knowledge distillation,
originated from hinton2015distilling , has been adopted for efficient tracking
shen2021distilled , this approach boosts performance of a lighter student
tracker with a stronger model. _Differently_ , LoReTrack improves the accuracy
of the low-resolution tracker using a high-resolution but _not_ a stronger
model, which avoids designing new student network in shen2021distilled and is
thus more compact. Compared with a recent work kou2024zoomtrack on applying
non-uniform resizing on smaller input for efficient Transformer tracking,
LoReTrack uniformly resizes the resolution and borrows a high-resolution model
for knowledge learning, which is simpler yet shows better results.

We implement LoReTrack on top of the popular one-stream Transformer tracking
architecture ye2022joint . In order to validate its effectiveness, we conduct
extensive experiments on five benchmarks, including LaSOT fan2019lasot ,
LaSOText fan2021lasot , GOT-10k huang2019got , TrackingNet
muller2018trackingnet , and UAV123 mueller2016benchmark . LoReTrack, with a
relatively low resolution of 2562, consistently improves the baseline
Transformer tracker with the same input resolution, and achieves competitive
or even better accuracy compared to the 3842 high-resolution tracker, while
running 52% faster and saving 56% MACs, as in Fig. 1. Moreover, LoReTrack is
highly resolution-scale. With a 1282 resolution, it run 25 _fps_ on a CPU with
64.9%/46.4% SUC scores on LaSOT/LaSOText, outperforming all other CPU real-
time trackers.

In summary, our contributions are three-fold: (i) We propose LoReTrack, an
efficient and accurate method that effectively boosts low-resolution
Transformer tracking to bridge the gap with its high-resolution counterpart;
(ii) We propose two novel knowledge distillation modules, including QKV-KD and
Disc-KD, allowing a low-resolution tracker to learn discriminative and fine-
grained information from the high-resolution model; and (iii) Extensive
experiments exhibit that, LoReTrack achieves consistent improvements over the
baseline and meanwhile runs fast, evidencing its efficacy.

##  2 Related Work

Transformer Tracking. Transformer dosovitskiy2020image ; vaswani2017attention
has recently greatly advanced tracking for its excellent capacity of
dependency modeling in images. Early methods (e.g., chen2021transformer ;
wang2021transformer ; yan2021learning ; gao2022aiatrack ) mainly leverage
Transformer to fuse the features from the search region and template for
enhancements. To further improve feature representation, fully Transformer
tracking (e.g., lin2022swintrack ; xie2022correlation ) has been introduced.
In these approaches, Transformer is employed for both feature extraction and
fusion, leading to better performance. Despite this, in these methods the
feature extraction backbones for search region and template are learned
separately, which neglects the demand for early interaction between search
region and template. To address this, the one-stream Transformer tracking
framework (e.g., cui2022mixformer ; chen2022backbone ; ye2022joint ) emerges
in recent years. It performs feature extraction and interaction for search
region and template in a joint way, exhibiting new state-of-the-arts. Owing to
its compact design and excellent performance, one-stream architecture has been
employed in many subsequent trackers (e.g., gao2023generalized ;
chen2023seqtrack ; wu2023dropmae ; wei2023autoregressive ).

Efficient Tracking. High efficiency is of importance for the real-world
deployment of visual tracking. In recent years, numerous efforts have been
made to reduce the computational load of object tracking. The approaches of
yan2021lighttrack ; borsuk2022fear ; kang2023exploring propose to search or
directly leverage a more lightweight network for efficient tracking. The work
of blatter2023efficient designs a more efficient attention operation to reduce
computational complexity of tracking. The method of cui2024mixformerv2
compresses the Transformer backbone for accelerating the tracking inference
speed. The approach of shen2021distilled proposes to distill a smaller but
faster tracking model from a larger one. The method in kou2024zoomtrack
leverages a smaller input with non-uniform resizing for decreasing the
computational load of Transformer tracking. Our work shares similar spirit
with shen2021distilled in using knowledge distillation for high efficiency,
but _difference_ is that we distill knowledge from a high-resolution model for
efficient one-stream Transformer tracking, yet not a larger model as in
shen2021distilled for convolutional network-based tracking. In addition,
_different_ from kou2024zoomtrack that uses a complicated non-uniform resizing
method for the smaller input, LoReTrack uniformly resizes the input and learns
knowledge from a high-resolution model for improvement, which is simpler yet
shows better results.

Knowledge Distillation. Knowledge distillation (KD) is first introduced in
hinton2015distilling for learning a more effective student model under the
supervision of the teacher model. Later, it has been leveraged for improving
model efficiency by distilling knowledge from a larger model to a relatively
smaller model, and demonstrated great potential in various tasks such as image
recognition lin2022knowledge , object detection chen2017learning , vision-
language learning wu2023tinyclip , semantic segmentation he2019knowledge ,
video recognition ma2022rethinking , etc. Our work is relevant to but
significantly different than ma2022rethinking . Specifically, we focus on
improving the accuracy of low-resolution Transformer tracking, while
ma2022rethinking for convolutional network-based video recognition.

##  3 The Proposed Methodology

Overview. In this work, we aim to boost the performance of low-resolution
Transformer tracking by distilling knowledge from its frozen high-resolution
counterpart while maintaining the efficiency. Specifically, built upon the
popular one-stream Transformer tracker ye2022joint ($\S$3.1), we introduce a
simple but effective dual knowledge distillation framework ($\S$3.2) that
consists of two knowledge distillation modules, including QKV-KD ($\S$3.2.1)
and Disc-KD ($\S$3.2.2), for fine-grained and discriminative feature learning.
Fig. 2 illustrates the overview pipeline of our approach.

Figure 2: Overview of the proposed method which enhances the accuracy of low-
resolution Transformer tracking via dual knowledge distillation from its
frozen high-resolution counterpart.

###  3.1 Base Tracking Architecture

Our LoReTrack builds upon a popular one-stream Transformer tracking framework
OSTrack ye2022joint for its compact architecture and excellent performance.
The core idea of OSTrack lies in exploiting the vision Transformer (ViT)
dosovitskiy2020image for joint feature extraction and interaction, which
enables earlier interaction between the search region and the template for
better performance. Specifically, given the template $T$ and search region
$S$, we first tokenize them into embeddings
$\textbf{E}_{t}=\mathtt{Tokenize}(T)$ and
$\textbf{E}_{s}=\mathtt{Tokenize}(S)$, where $\mathtt{Tokenize}(\cdot)$ is the
tokenization process consisting of patch partitioning and linear projection.
Then, two learnable position embeddings $\textbf{P}_{t}$ and $\textbf{P}_{s}$
are added to $\textbf{E}_{t}$ and $\textbf{E}_{s}$ to generate the final
template token embedding T and search region token embedding S as follows,

$\textbf{T}=\textbf{E}_{t}+\textbf{P}_{t}\;\;\;\;\;\;\textbf{S}=\textbf{E}_{s}+\textbf{P}_{s}$
(1)

Afterwards, T and S are first concatenated as the final token embedding
$\textbf{H}_{0}=\mathtt{Concat}(\mathbf{T},\mathbf{S})$, where
$\mathtt{Concat}(\cdot,\cdot)$ is the concatenation operation. Then,
$\textbf{H}_{0}$ is sent to the Transformer encoder dosovitskiy2020image with
M ($\text{M}=12$) layers. Each layer consists of the multi-head self-attention
(MSA) and a multi-layer perceptron (MLP) block for feature interaction and
learning. It receives the output from last layer as input (for the first
layer, input is $\textbf{H}_{0}$) to perform feature interaction and learning.
This process can be expressed as follows,

$\begin{split}\textbf{Q}_{m}&=\hat{\textbf{H}}_{m}\textbf{W}_{m,q}\;\;\;\textbf{K}_{m}=\hat{\textbf{H}}_{m}\textbf{W}_{m,k}\;\;\;\textbf{V}_{m}=\hat{\textbf{H}}_{m}\textbf{W}_{m,v}\;\;\;\hat{\textbf{H}}_{m}=\mathtt{LN}(\mathbf{H}_{m-1})\\\
\tilde{\textbf{H}}_{m}&=\mathtt{MSA}(\textbf{Q}_{m},\textbf{K}_{m},\textbf{V}_{m})+\mathbf{H}_{m-1}\\\
\textbf{H}_{m}&=\mathtt{MLP}(\mathtt{LN}(\tilde{\textbf{H}}_{m}))+\tilde{\textbf{H}}_{m}\end{split}$
(2)

where $\mathtt{MSA}(\textbf{q},\textbf{k},\textbf{v})$ represents the multi-
head attention with query q, key k, and value v, $\texttt{MLP}(\cdot)$ the MLP
block, $\mathtt{LN}(\cdot)$ the layer normalization, and
$m=1,2,\cdots,\text{M}$. The output $\textbf{H}_{\text{M}}$ from the
$\text{M}^{\text{th}}$ layer contains the final features of template and
search region. We detach the search region feature $\textbf{F}_{s}$ from
$\textbf{H}_{\text{M}}$ by $\textbf{F}_{t}$,
$\textbf{F}_{s}=\mathtt{DeConcat}(\textbf{H}_{\text{M}})$, where
$\mathtt{Deconcat}(\cdot)$ is the deconcatenation operation, and feed it the
prediction head that contains classification and regression branches for
target localization.

In this work, both the high-resolution and low-resolution trackers are
developed upon OSTrack ye2022joint , and use the same architectures except for
different input resolution. For simplicity, we use superscripts “$h$” and
“$l$” to distinguish high- and low-resolution in the following. Please note,
candidate elimination in ye2022joint is _not_ used as it may result in heavy
spatial misalignment in features of different resolutions.

###  3.2 Low-Resolution Transformer Tracking with Dual Knowledge Distillation

To boost the performance of low-resolution Transformer tracking while
maintaining its high efficiency, we introduce dual knowledge distillation,
which consists of two simple effective modules, including query-key-value
knowledge distillation (QKV-KD) and discrimination knowledge distillation
(Disc-KD), for learning fine-grained and discriminative features from the
high-resolution model. Please note, during knowledge distillation, the high-
resolution Transformer tracker is _frozen_ after training.

####  3.2.1 Query-Key-Value Knowledge Distillation

Figure 3: Illustration of QKV-KD.

In low-resolution Transformer tracking, coarse tokens are difficult for
learning discriminative features in the image, which causes accuracy drop. To
alleviate this, we devise the _query-key-value knowledge distillation_ (QKV-
KD), which, from the holistic perspective view, allows the low-resolution
Transformer tracker to directly learn discriminative features by fine-grained
tokens from the high-resolution model. Different from existing approaches for
Transformer that perform knowledge distillation on features (generated by
query, key and value) lin2022knowledge , QKV-KD is able to learn more
comprehensive knowledge from high-resolution features from _multiple views_
(i.e., the query, key, and value views). Besides, another advantage of QKV-KD
is that it _indirectly_ enables knowledge distillation of the attention maps
in Transformer, because query, key, and values are directly leveraged for
attention computation, which benefits deeper mimicry for the student low-
resolution Transformer tracker wang2020minilm .

Specifically, we use QKV-KD on the search region in the last
$\text{M}^{\text{th}}$ encoder layer containing the most semantic information,
as shown in Fig. 3. More concretely, let us denote
$\textbf{H}_{\text{M}-1}^{h}$ and $\textbf{H}_{\text{M}-1}^{l}$ as inputs to
the $\text{M}^{\text{th}}$ encoder layer for the high-resolution and low-
resolution Transformer trackers. Then, queries, keys, and values, before being
sent to MSA, for these two trackers can be generated as follows,

$\small\begin{split}\textbf{Q}_{\text{M}}^{h}&=\hat{\textbf{H}}^{h}_{\text{M}-1}\textbf{W}_{\text{M},q}^{h}\;\;\;\;\;\textbf{K}_{\text{M}}^{h}=\hat{\textbf{H}}^{h}_{\text{M}-1}\textbf{W}_{\text{M},k}^{h}\;\;\;\;\;\textbf{V}_{\text{M}}^{h}=\hat{\textbf{H}}^{h}_{\text{M}-1}\textbf{W}_{\text{M},v}^{h}\;\;\;\;\;\hat{\textbf{H}}^{h}_{\text{M}-1}=\mathtt{LN}(\textbf{H}^{h}_{\text{M}-1})\\\
\textbf{Q}_{\text{M}}^{l}&=\hat{\textbf{H}}^{l}_{\text{M}-1}\textbf{W}_{\text{M},q}^{l}\;\;\;\;\;\textbf{K}_{\text{M}}^{l}=\hat{\textbf{H}}^{l}_{\text{M}-1}\textbf{W}_{\text{M},k}^{l}\;\;\;\;\;\textbf{V}_{\text{M}}^{l}=\hat{\textbf{H}}^{l}_{\text{M}-1}\textbf{W}_{\text{M},v}^{l}\;\;\;\;\;\hat{\textbf{H}}^{l}_{\text{M}-1}=\mathtt{LN}(\textbf{H}^{l}_{\text{M}-1})\end{split}$
(3)

where
$\textbf{Q}_{\text{M}}^{h}$/$\textbf{K}_{\text{M}}^{h}$/$\textbf{V}_{\text{M}}^{h}$
and
$\textbf{Q}_{\text{M}}^{l}$/$\textbf{K}_{\text{M}}^{l}$/$\textbf{V}_{\text{M}}^{l}$
denote queries, keys, and values in the $\text{M}^{\text{th}}$ encoder layers
of the low-resolution and high-resolution trackers. Afterwards, we could
detach template and search region related queries, keys and values from them
via deconcatenation as follows,

$\small\begin{split}{\color[rgb]{.5,.5,.5}\textbf{Q}_{\text{M},t}^{h}},\textbf{Q}_{\text{M},s}^{h}&=\mathtt{DeConcat}(\textbf{Q}_{\text{M}}^{h})\;\;\;\;\;{\color[rgb]{.5,.5,.5}\textbf{K}_{\text{M},t}^{h}},\textbf{K}_{\text{M},s}^{h}=\mathtt{DeConcat}(\textbf{V}_{\text{M}}^{h})\;\;\;\;\;{\color[rgb]{.5,.5,.5}\textbf{V}_{\text{M},t}^{h}},\textbf{V}_{\text{M},s}^{h}=\mathtt{DeConcat}(\textbf{K}_{\text{M}}^{h})\\\
{\color[rgb]{.5,.5,.5}\textbf{Q}_{\text{M},t}^{l}},\textbf{Q}_{\text{M},s}^{l}&=\mathtt{DeConcat}(\textbf{Q}_{\text{M}}^{l})\;\;\;\;\;{\color[rgb]{.5,.5,.5}\textbf{K}_{\text{M},t}^{l}},\textbf{K}_{\text{M},s}^{l}=\mathtt{DeConcat}(\textbf{V}_{\text{M}}^{l})\;\;\;\;\;{\color[rgb]{.5,.5,.5}\textbf{V}_{\text{M},t}^{l}},\textbf{V}_{\text{M},s}^{l}=\mathtt{DeConcat}(\textbf{K}_{\text{M}}^{l})\end{split}$
(4)

where
$\textbf{Q}_{\text{M},s}^{h}$/$\textbf{K}_{\text{M},s}^{h}$/$\textbf{V}_{\text{M},s}^{h}$
and
$\textbf{Q}_{\text{M},s}^{l}$/$\textbf{K}_{\text{M},s}^{l}$/$\textbf{V}_{\text{M},s}^{l}$
are the queries, keys, and values for the search regions in the high-
resolution and low-resolution trackers.

With the above, QKV-KD for search region can be mathematically expressed as
follows,

$\mathcal{L}_{\text{QKV-
KD}}=\mathcal{L}_{\text{2}}(\textbf{Q}_{\text{M},s}^{h},\Phi(\textbf{Q}_{\text{M},s}^{l}))+\mathcal{L}_{\text{2}}(\textbf{K}_{\text{M},s}^{h},\Phi(\textbf{K}_{\text{M},s}^{l}))+\mathcal{L}_{\text{2}}(\textbf{V}_{\text{M},s}^{h},\Phi(\textbf{V}_{\text{M},s}^{l}))$
(5)

where $\Phi(\cdot)$ denotes the bilinear interpolation operation to align the
dimensions, and $\mathcal{L}_{2}(\cdot,\cdot)$ stands for the mean squared
loss. With $\mathcal{L}_{\text{QKV-KD}}$ in Eq. (5), the search region feature
of the low-resolution Transformer tracker are able to learn from the
discriminative and fine-grained search region feature of the high-resolution
Transformer tracker from multiple views, which enhances final performance.
Note that, QKV-KD is applied only on the search region. We study using QKV-KD
on both template and search region, yet observe no performance gains, as
described in experiments later.

####  3.2.2 Discrimination Knowledge Distillation

To further foster the discriminative capacity of low-resolution Transformer
tracking, we introduce the _discrimination knowledge discrimination_ (Disc-
KD), which, from the target-aware perspective, focuses on imitating
discriminative region generated from the high-resolution model for enhancing
the target-background distinguishing ability of the low-resolution tracker.

Figure 4: Illustration of Disc-KD.

Considering that the high-level features usually contain more semantic
information for discrimination, we apply Disc-KD on the search region feature
after the last $\text{M}^{\text{th}}$ encoder layer. Specifically, given the
output features $\textbf{H}_{\text{M}}^{h}$ and $\textbf{H}_{\text{M}}^{l}$
after the $\text{M}^{\text{th}}$ encoder layer from the high- and low-
resolution trackers, we first detach the search region features from them, as
follows,

$\begin{split}{\color[rgb]{.5,.5,.5}\textbf{F}_{t}^{h}},\textbf{F}_{s}^{h}&=\mathtt{DeConcat}(\textbf{H}_{\text{M}}^{h})\\\
{\color[rgb]{.5,.5,.5}\textbf{F}_{t}^{l}},\textbf{F}_{s}^{l}&=\mathtt{DeConcat}(\textbf{H}_{\text{M}}^{l})\end{split}$
(6)

Inspired by zagoruyko2016paying that the importance of hidden neuron
activation can be indicated utilizing its absolute values in feature maps, we
generate discriminative maps for the search region using $\textbf{F}_{s}^{h}$
from the high-resolution Transformer model, similar to ye2022joint . In
specific, we compute values for each token in $\textbf{F}_{s}^{h}$ along the
channel dimension via

$\textbf{D}_{s}^{h}(i)=\frac{1}{C^{h}}\sum_{c=1}^{C^{h}}\big{(}\textbf{F}_{s}^{h}(i,c)\big{)}^{2}\;\;\;i=1,2,\cdots,N^{h}$
(7)

where $\textbf{D}_{s}^{h}$ is the discriminative map of $\textbf{F}_{s}^{h}$,
$N^{h}$ number of tokens in $\textbf{F}_{s}^{h}$, and $C^{h}$ channel
dimension of the token. Note that in Eq. (7), we use square value to compute
discriminative map for slightly better performance.

Instead of directly performing knowledge distillation on $\textbf{F}_{s}^{h}$
and $\textbf{F}_{s}^{l}$, we propose to decouple discrimination and non-
discrimination regions from features for knowledge distillation, which allows
more focus on learning the ability to distinguish target in the low-resolution
tracker. More concretely, given $\textbf{D}_{s}^{h}$, we first extract the
discrimination mask $\mathcal{M}_{s}^{h}$ from it with a threshold $\tau$, as
follows,

$\mathcal{M}_{s}^{h}(i,j)=\left\\{\begin{aligned}
1&,&\textbf{D}_{s}^{h}(i,j)\geq\tau\\\
0&,&\text{otherwise}\end{aligned}\right.$ (8)

It is worth noticing that, here we employ the discriminative map from the
high-resolution tracker to produce the discrimination mask, because the high-
resolution feature is more robust than that of low-resolution trackers and the
discriminative region is thus more accurate. After this, the Disc-KD can be
formulated as follows,

$\mathcal{L}_{\text{Disc-
KD}}=\alpha_{1}\mathcal{L}_{2}\big{(}\mathcal{M}_{s}^{h}\otimes\textbf{F}_{s}^{h},\mathcal{M}_{s}^{h}\otimes\Phi(\textbf{F}_{s}^{l})\big{)}+\alpha_{2}\mathcal{L}_{2}\big{(}(1-\mathcal{M}_{s}^{h})\otimes\textbf{F}_{s}^{h},(1-\mathcal{M}_{s}^{h})\otimes\Phi(\textbf{F}_{s}^{l})\big{)}$
(9)

where $\alpha_{1}$ and $\alpha_{2}$ are weight parameters to balance the loss.
By setting $\alpha_{1}$ larger than $\alpha_{2}$, Disc-KD enables paying more
attention to distilling knowledge from discriminative target regions.

###  3.3 Loss and Inference for Low-Resolution Transformer Tracking

Similar to OSTrack, after the last $\text{M}^{\text{th}}$ encoder layer, the
search region feature $\textbf{F}_{s}^{l}$, which is detached from
$\textbf{H}_{\text{M}}^{l}$ by deconcatenation via
${\color[rgb]{.5,.5,.5}\textbf{F}_{t}^{l}},\textbf{F}_{s}^{l}=\mathtt{DeConcate}(\textbf{H}_{\text{M}}^{l})$,
is reshaped and fed to the prediction head to generate classification scores
and regression offsets. During training, we consider the loss terms for
classification and regression, as in OSTrack, as well as for knowledge
distillation including query-key-value knowledge distillation and
discrimination knowledge distillation. Eventually, the total loss
$\mathcal{L}_{\text{LoReTrack}}$ for training the low-resolution Transformer
tracker can be expressed as follows,

$\mathcal{L}_{\text{LoReTrack}}=\mathcal{L}_{\text{Cls}}+\mathcal{L}_{\text{Reg}}+\beta_{1}\mathcal{L}_{\text{QKV-
KD}}+\beta_{2}\mathcal{L}_{\text{Disc-KD}}$ (10)

where $\mathcal{L}_{\text{Cls}}$ and $\mathcal{L}_{\text{Reg}}$ are losses for
classification and regression, and $\beta 1$ and $\beta_{2}$ the weight
parameters. For details of these two, please kindly refer to OSTrack
ye2022joint . Please note that, the high-resolution Transformer tracking model
is frozen during the training of LoReTrack.

Once the training of LoReTrack is completed, we perform inference for tracking
target object as in the base tracker OSTrack. For further elaboration on this,
we kindly refer the readers to ye2022joint

Table 1: State-of-the-art comparison on five datasets, including LaSOT
fan2019lasot , LaSOT${}_{\text{ext}}$ fan2021lasot , GOT-10k huang2019got ,
TrackingNet muller2018trackingnet , and UAV123 mueller2016benchmark . The
speed is measured by _fps_. The best two results are highlighted in red and
blue. ‘$\P$’ indicates the usage of temporal contextual information for
improving tracking. Overall, our LoReTrack shows better performance with
faster running speed.

Tracker | Res. | LaSOT | LaSOT${}_{\text{ext}}$ | GOT-10k | TrackingNet | UAV123 |  GPU Speed
---|---|---|---|---|---|---|---
SUC | PRE | SUC | PRE | AO | SR0.5 | SR0.75 | SUC | PRE | SUC | PRE
LoReTrack-256 (ours) | 2562 | 70.3 | 76.2 | 51.3 | 58.7 | 73.5 | 84.0 | 70.4 | 82.9 | 81.4 | 70.6 | 92.1 | 130
OSTrack-256 ye2022joint | 2562 | 68.7 | 74.6 | 46.3 | 51.9 | 71.5 | 81.6 | 67.7 | 82.9 | 81.6 | 68.3 | 88.6 | 130
LoReTrack-192 (ours) | 1922 | 68.6 | 73.7 | 50.0 | 56.8 | 71.5 | 82.8 | 66.7 | 80.9 | 78.4 | 69.9 | 91.4 | 186
OSTrack-192 ye2022joint | 1922 | 66.2 | 70.2 | 47.1 | 53.0 | 67.6 | 78.8 | 62.1 | 80.7 | 77.8 | 67.8 | 88.5 | 186
OSTrack-Zoom kou2024zoomtrack | 2562 | 70.2 | 76.2 | 50.5 | 57.4 | 73.5 | 83.6 | 70.0 | 83.2 | 82.2 | 69.3 | 91.4 | 128
MixFormerV2 cui2024mixformerv2 ¶ | 2882 | 69.5 | 75.0 | - | - | - | - | - | 82.9 | 81.0 | 70.5 | 91.9 | 165
OSTrack-CE-256 ye2022joint | 2562 | 69.1 | 75.2 | 47.4 | 53.3 | 71.0 | 80.4 | 68.2 | 83.1 | 82.0 | 68.3 | 88.8 | 147
ROMTrack cai2023robust ¶ | 2562 | 69.3 | 75.6 | 48.9 | 55.0 | 72.9 | 82.9 | 70.2 | 83.6 | 82.7 | - | - | 62
SwinTrack-T lin2022swintrack ¶ | 2242 | 67.2 | 70.8 | 47.6 | 53.9 | 71.3 | 81.9 | 64.5 | 81.1 | 78.4 | 68.8 | - | 98
SimTrack-B chen2022backbone | 2242 | 69.3 | - | - | - | 68.6 | 78.9 | 62.4 | 82.3 | - | 69.8 | 89.6 | 40
TransInMo guo2022learning | 2552 | 65.7 | 70.7 | - | - | - | - | - | 81.7 | - | 69.0 | 89.0 | 34
TransT chen2021transformer | 2562 | 64.9 | 69.0 | - | - | 72.3 | 82.4 | 68.2 | 81.4 | 80.3 | 69.1 | - | 50.0
ToMP-101 mayer2022transforming ¶ | 2882 | 68.5 | 73.5 | 45.9 | - | - | - | - | 81.5 | 78.9 | 66.9 | - | 20
SeqTrack-B chen2023seqtrack ¶ | 3842 | 71.5 | 77.8 | 50.5 | 57.5 | 74.5 | 84.3 | 71.4 | 83.9 | 83.6 | 68.6 | - | 15
OSTrack-CE-384 ye2022joint | 3842 | 71.1 | 77.6 | 50.5 | 57.6 | 73.7 | 83.2 | 70.8 | 83.9 | 83.2 | 70.7 | - | 81
SwinTrack-B lin2022swintrack ¶ | 3842 | 71.3 | 76.5 | 49.1 | 55.6 | 72.4 | 80.5 | 67.8 | 84.0 | 82.8 | 70.5 | - | 45
MixFormer-22k cui2022mixformer ¶ | 3202 | 69.2 | 74.7 | - | - | 70.7 | 80.0 | 67.8 | 83.1 | 81.6 | 70.4 | 91.8 | 25
AiATrack gao2022aiatrack ¶ | 3202 | 69.0 | 73.8 | 46.2 | - | 69.6 | 80.0 | 63.2 | 82.7 | 80.4 | 70.6 | - | 38

##  4 Experiments

Implementation details. The proposed low-resolution Transformer tracker,
coined as LoReTrack, is built upon OSTrack ye2022joint and implemented using
PyTorch paszke2019pytorch . The parameter settings are kept the same as in the
base OSTrack except for the removal of candidate elimination, and please refer
to ye2022joint for details. Four datasets, including the training splits of
LaSOT fan2019lasot , TrackingNet muller2018trackingnet , COCO lin2014microsoft
, and GOT-10k huang2019got (1K forbidden sequences from GOT-10k training set
are removed). The threshold $\tau$ is empirically set to 0.2. $\alpha_{1}$ and
$\alpha_{2}$ are set to 0.6 and 0.4. $\beta_{1}$ and $\beta_{2}$ are set to
0.01 and 0.01, respectively. Our model is trained utilizing 4$\times$ NVIDIA
A6000 GPUs. The inference is performed on a single NVIDIA A5500 GPU with an
Intel i9-11900K CPU.

Resolution Setting. In this work, we use the base OSTrack ye2022joint with
3842 input as the high-resolution Transformer. The high-resolution tacker can
be directly initialized using off-the-shelf pre-trained model from ye2022joint
and used in a frozen way. LoReTrack denotes the base tracker with lower
resolutions $\\{256^{2},192^{2},128^{2},96^{2}\\}$. Among them, LoReTrack with
2562 and 1922, i.e., LoReTrack-256 and LoReTrack-192, are designed for low-
resolution tracking with GPU, while that with 1282 and 962, i.e.,
LoReTrack-128 and LoReTrack-96, for real-time low-resolution tracking on the
CPU.

###  4.1 State-of-the-art Comparison

We evaluate our LoReTrack-256/192 and compare them with other state-of-the-art
trackers on five challenging benchmarks, including LaSOT fan2019lasot ,
LaSOT${}_{\text{ext}}$ fan2021lasot , GOT-10k huang2019got , TrackingNet
muller2018trackingnet , and UAV123 mueller2016benchmark . The success (SUC)
and precision (PRE) scores are employed as evaluation metrics on LaSOT,
LaSOT${}_{\text{ext}}$, TrackingNet, and UAV123, and average overlap (AO) and
success rate (SR) for comparison on GOT-10k. Please note that, we mainly
compare with trackers with similar resolutions (e.g., smaller than 3002), and
trackers with higher-resolution (e.g., larger than 3002) or using much larger
backbones are displayed but excluded in comparison due to different aims. For
tracker OSTrack and OSTrack-Zoom, we use the same GPU for speed measurement.

LaSOT fan2019lasot is a large-scale dataset with 280 long-term sequences for
testing. As shown in Tab. 1, our LoReTrack with 2562 resolution achieves the
best SUC score of 70.3%, and improves OSTrack-256 with 68.7% SUC score by
1.6%. When using the 1922 resolution for tracking, LoReTrack-192 improves
OSTrack-192 from 66.2% to 6.6% with absolute gains of 2.4%. All these results
show the effectiveness of our dual knowledge distillation from the high-
resolution tracking model to enhance low-resolution Transformer tracking
performance.

LaSOT${}_{\text{ext}}$ fan2021lasot is an extension of LaSOT by adding 150 new
challenging sequences. As demonstrated in Tab. 1, our LoReTrack achieves the
51.3% SUC score, which improves the baseline OSTrack-256 by 5.0%, showing its
efficacy. In addition, LoReTrack-256 outperforms the recent OSTrack-Zoom with
50.5% SUC score by 0.8%, which displays the advantage of our simple but
effective approach. Moreover, LoReTrack outperforms many trackers with higher
3842 resolution such as SeqTrack-B, OSTrack-CE, and SwinTrack-B and meanwhile
runs faster at 130 _fps_ , validating its effectiveness.

GOT-10k huang2019got evaluates the zero-shot performance of trackers. As shown
in Tab. 1, LoReTrack-256 obtains the 73.5% AO score, which improves OSTrack
with the same 2562 resolution with 71.0% AO score by 2.5% and outperforms
other efficient trackers. Compared to OSTrack-Zoom, LoReTrack achieves the
same AO score but better SR0.5 and SR0.75 scores.

TrackingNet muller2018trackingnet provides 511 videos for evaluation.
LoReTrack reveals no improvements in SUC score over the base tracker on
TrackingNet, similar to the observation in OSTrack-Zoom. The possible reason
is that TrackingNet has a very different test split than other datasets
kou2024zoomtrack , and contains very few videos with scenarios that can
benefit from looking at fine-grained information.

UAV123 mueller2016benchmark is a challenging drone-based tracking dataset with
123 videos. LoReTrack-256 achieves the SUC score of 70.6%, which improves
OSTrack-256 with 68.3% by 2.3%. It surpasses other efficient trackers
including the recent OSTrack-Zoom with 69.3% and MixFormerV2 with 70.5% SUC
scores, and shows competitive performance compared to the high-resolution
OSTrack-CE-384.

Table 2: Comparison with CPU real-time tracking algorithms on five datasets,
including LaSOT fan2019lasot , LaSOT${}_{\text{ext}}$ fan2021lasot , GOT-10k
huang2019got , TrackingNet muller2018trackingnet , and UAV123
mueller2016benchmark . The speed is measured by _fps_. The best two results
are highlighted in red and blue.

Tracker | Res. | LaSOT | LaSOT${}_{\text{ext}}$ | GOT-10k | TrackingNet | UAV123 |  CPU Speed
---|---|---|---|---|---|---|---
SUC | PRE | SUC | PRE | AO | SR0.5 | SR0.75 | SUC | PRE | SUC | PRE
LoReTrack-128 (ours) | 1282 | 64.9 | 67.5 | 46.4 | 50.9 | 64.3 | 76.4 | 53.6 | 77.7 | 73.6 | 69.0 | 90.8 | 25
OSTrack-128 ye2022joint | 1282 | 62.2 | 63.6 | 43.4 | 46.8 | 62.9 | 75.2 | 51.7 | 76.6 | 71.5 | 66.0 | 87.1 | 25
LoReTrack-96 (ours) | 962 | 61.0 | 61.1 | 45.1 | 49.0 | 58.9 | 70.9 | 42.2 | 74.0 | 68.3 | 67.1 | 88.9 | 31
OSTrack-96 ye2022joint | 962 | 59.0 | 58.2 | 44.0 | 47.1 | 57.3 | 68.7 | 40.8 | 72.7 | 65.9 | 64.8 | 87.1 | 31
HiT-Small kang2023exploring | 1282 | 60.5 | 61.5 | 40.4 | - | 62.6 | 71.2 | 54.4 | 77.7 | 73.1 | 65.6 | 61.8 | 72
MixFormerV2-S cui2024mixformerv2 ¶ | 2242 | 60.6 | 60.4 | 43.6 | 46.2 | - | - | - | 75.8 | 70.4 | 65.8 | 86.8 | 30
FEAR-L borsuk2022fear ¶ | 2562 | 57.9 | 60.9 | - | - | 64.5 | - | - | - | - | - | - | -
HCAT chen2022efficient | 2562 | 59.3 | 61 |  |  | 65.1 | 76.5 | 56.7 | 76.6 | 72.9 | 62.7 | - | 45
E.T.Track blatter2023efficient | 2562 | 59.1 | - | - | - | - | - | - | 75.0 | 70.6 | 62.3 | - | 47
LightTrack yan2021lighttrack | 2562 | 53.8 | 53.7 | - | - | 61.1 | 71.0 | - | 72.5 | 69.5 | - | - | 44

Comparison with CPU real-time Trackers. LoReTrack is resolution-scalable, and
can be applied to various resolutions for tracking. When further reducing the
input resolution to 1282 or 962, LoReTrack achieves CPU real-time tracking
(i.e., tracking speed above 20 _fps_ following kristan2020eighth ). Tab. 2
shows the results of our CPU real-time LoReTrack-128 and LoReTrack-96 and
their comparison with other trackers on five benchmarks including LaSOT
fan2019lasot , LaSOT${}_{\text{ext}}$ fan2021lasot , GOT-10k huang2019got ,
TrackingNet muller2018trackingnet , and UAV123 mueller2016benchmark .
LoReTrack-128 achieves superior 64.9%, 46.4%, 77.7%, and 69.0% SUC scores on
LaSOT, LaSOT${}_{\text{ext}}$, TrackingNet, and UAV123, and shows competitive
performance on GOT-10k.

###  4.2 Ablation Study

In order to further analyze different design choices for LoReTrack, we conduct
various ablation studies using LoReTrack-256 on LaSOT fan2019lasot as follows.

Impact of QKV-KD and Disc-KD. The key of LoReTrack lies in two knowledge
distillation modules of QKV-KD and Disc-KD. To analyze their roles, we conduct
ablations in Tab. 4.2. From Tab. 4.2, we can see that, when removing QKV-KD,
the performance drops from 70.3% to 69.0% by 1.3%. When not applying Disc-KD,
the SUC score is decreased from 70.3% to 69.7% by 0.6%. These results reveal
that both QKV-KD and Disc-KD play a crucial for enhancing the performance of
LoReTrack.

Impact of the threshold $\tau$. In Disc-KD, an important parameter is the
threshold $\tau$ for discrimination mask generation. We empirically study
different values for $\tau$ in Tab. 4.2. As shown in Tab. 4.2, we can see that
when setting $\tau$ to 0.20, LoReTrack shows the best performance with 70.3%
SUC score.

Table 3: Ablations for QKV-KD and Disc-KD. Methods | SUC (%)
---|---
w/o QKV-KD | 69.0
w/o Disc-KD | 69.7
Ours | 70.3
Table 4: Ablations for the threshold $\tau$ in Disc-KD. Methods | SUC (%)
---|---
$\tau$ = 0.18 | 69.5
$\tau$ = 0.20 (ours) | 70.3
$\tau$ = 0.22 | 69.9
Table 5: Ablations for the impact of template in query-key-value knowledge distillation. Methods | SUC (%)
---|---
w/ template | 69.7
w/o template (ours) | 70.3

Table 6: Ablations for applying QKV-KD on more layers. Layer | SUC (%)
---|---
Layer $\\{11,12\\}$ | 69.4
Layer $\\{12\\}$ (ours) | 70.3
Table 7: Ablations for candidate elimination in LoReTrack. Methods | SUC (%)
---|---
w/ CE | 66.8
w/o CE (ours) | 70.3
Table 8: Ablations for different distillation strategies. Methods | SUC (%)
---|---
Feature distillation | 69.4
QKV-KD (ours) | 70.3

Impact of template on knowledge distillation. To explore the impact of
template on QKV-KD, we conduct an experiment in Tab. 4.2. Our result shows
that, when performing knowledge distillation on both template and search
region in QKV-KD, the accuracy is degraded from 70.3% to 69.7% by 0.6%. The
possible reason is that, the search region feature is more crucial for
tracking because target is localized in the search region. When performing
knowledge distillation for template, knowledge learning from high-resolution
model for the search region will be weakened, degrading performance.

Impact of encoder layers for QKV-KD. QKV-KD aims to distill knowledge from the
different views of query, key and value. In the experiment, we explore the
impact of encoder layers on QKV-KD in Tab 4.2. As shown in Tab 4.2, we can see
that, when applying QKV-KD on more layers, the performance is degraded from
70.3% to 69.4% by 0.9%. The reason is that, when applied on multiple, it may
be hard to QKV-KD to effectively learn the high-level knowledge from the high-
resolution model.

Impact of candidate elimination. To avoid heavy spatial misalignment in
features, we do not employ candidate elimination (CE). To validate our claim,
we conduct an experiment in Tab. 4.2. As shown in Tab. 4.2, we can clearly see
that, when applying CE in LoReTrack, the SUC score significantly drops from
70.3% to 66.8% by 3.5%. For this reason, we do not adopt CE in our method.

Impact on different distillation strategies for global features. To showcase
the role of QKV-KD for holistic knowledge learning, we compare it with another
version that directly utilizes the feature generated by query, key, and value
in Tab 4.2. As shown in Tab 4.2, we observe that distillation on query, key,
and value shows better performance, evidencing its effectiveness.

Table 9: Ablations for $\alpha_{1}$ and $\alpha_{2}$.

Methods | SUC (%)
---|---
$\alpha_{1}$ = 0.9, $\alpha_{2}$ = 0.1 | 70.0
$\alpha_{1}$ = 0.8, $\alpha_{2}$ = 0.2 | 69.7
$\alpha_{1}$ = 0.7, $\alpha_{2}$ = 0.3 | 69.7
$\alpha_{1}$ = 0.6, $\alpha_{2}$ = 0.4 (ours) | 70.3
$\alpha_{1}$ = 0.5, $\alpha_{2}$ = 0.5 | 70.0

Impact on different weight parameters $\alpha_{1}$ and $\alpha_{2}$. The
parameters $\alpha_{1}$ and $\alpha_{2}$ are used to control the weight of
knowledge distillation on discrimination and non-discrimination regions. When
$\alpha_{1}$ is way larger than $\alpha_{2}$, it indicates the distillation
focuses on the discrimination region, while less on the background, which may
cause weak ability in suppressing background. We conduct experiments to study
different values for $\alpha_{1}$ and $\alpha_{2}$. Empirically, when setting
$\alpha_{1}$ and $\alpha_{2}$ to be 0.6 and 0.4, we show the best performance.

##  5 Conclusion

In this work, we propose to enhance the low-resolution Transformer tracking
performance with dual knowledge distillation. The key lies in two specially
designed modules, including query-key-value knowledge distillation and
discrimination knowledge distillation. The former aims at learning fine-
grained features from the high-resolution model, while the later works to
mimic the discrimination regions from its high-resolution counterpart. With
dual knowledge distillation, the low-resolution Transformer tracker, named
LoReTrack, enjoys both high accuracy by distilling useful knowledge from the
high-resolution model and fast speed by reduced resolution. In our extensive
experiments, we show that LoReTrack effectively improves the baseline
Transformer tracker with the same resolution in most cases. Moreover,
LoReTrack is resolution-scalable. When further reducing the resolution,
LoReTrack achieves CPU real-time tracking while being accurate, outperforming
other methods.

## References

  * [1] Philippe Blatter, Menelaos Kanakis, Martin Danelljan, and Luc Van Gool.  Efficient visual tracking with exemplar transformers.  In WACV, 2023.
  * [2] Vasyl Borsuk, Roman Vei, Orest Kupyn, Tetiana Martyniuk, Igor Krashenyi, and Jiři Matas.  Fear: Fast, efficient, accurate and robust visual tracker.  In ECCV, 2022.
  * [3] Yidong Cai, Jie Liu, Jie Tang, and Gangshan Wu.  Robust object modeling for visual tracking.  In ICCV, 2023.
  * [4] Boyu Chen, Peixia Li, Lei Bai, Lei Qiao, Qiuhong Shen, Bo Li, Weihao Gan, Wei Wu, and Wanli Ouyang.  Backbone is all your need: A simplified architecture for visual object tracking.  In ECCV, 2022.
  * [5] Guobin Chen, Wongun Choi, Xiang Yu, Tony Han, and Manmohan Chandraker.  Learning efficient object detection models with knowledge distillation.  NIPS, 2017.
  * [6] Xin Chen, Ben Kang, Dong Wang, Dongdong Li, and Huchuan Lu.  Efficient visual tracking via hierarchical cross-attention transformer.  In ECCV, 2022.
  * [7] Xin Chen, Houwen Peng, Dong Wang, Huchuan Lu, and Han Hu.  Seqtrack: Sequence to sequence learning for visual object tracking.  In CVPR, 2023.
  * [8] Xin Chen, Bin Yan, Jiawen Zhu, Dong Wang, Xiaoyun Yang, and Huchuan Lu.  Transformer tracking.  In CVPR, 2021.
  * [9] Yutao Cui, Cheng Jiang, Limin Wang, and Gangshan Wu.  Mixformer: End-to-end tracking with iterative mixed attention.  In CVPR, 2022.
  * [10] Yutao Cui, Tianhui Song, Gangshan Wu, and Limin Wang.  Mixformerv2: Efficient fully transformer tracking.  In NeurIPS, 2023.
  * [11] Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, et al.  An image is worth 16x16 words: Transformers for image recognition at scale.  In ICLR, 2021.
  * [12] Heng Fan, Hexin Bai, Liting Lin, Fan Yang, Peng Chu, Ge Deng, Sijia Yu, Harshit, Mingzhen Huang, Juehuan Liu, et al.  Lasot: A high-quality large-scale single object tracking benchmark.  IJCV, 129:439–461, 2021.
  * [13] Heng Fan, Liting Lin, Fan Yang, Peng Chu, Ge Deng, Sijia Yu, Hexin Bai, Yong Xu, Chunyuan Liao, and Haibin Ling.  Lasot: A high-quality benchmark for large-scale single object tracking.  In CVPR, 2019.
  * [14] Shenyuan Gao, Chunluan Zhou, Chao Ma, Xinggang Wang, and Junsong Yuan.  Aiatrack: Attention in attention for transformer visual tracking.  In ECCV, 2022.
  * [15] Shenyuan Gao, Chunluan Zhou, and Jun Zhang.  Generalized relation modeling for transformer tracking.  In CVPR, 2023.
  * [16] Mingzhe Guo, Zhipeng Zhang, Heng Fan, Liping Jing, Yilin Lyu, Bing Li, and Weiming Hu.  Learning target-aware representation for visual tracking via informative interactions.  In IJCAI, 2022.
  * [17] Tong He, Chunhua Shen, Zhi Tian, Dong Gong, Changming Sun, and Youliang Yan.  Knowledge adaptation for efficient semantic segmentation.  In CVPR, 2019.
  * [18] Geoffrey Hinton, Oriol Vinyals, and Jeff Dean.  Distilling the knowledge in a neural network.  arXiv, 2015.
  * [19] Lianghua Huang, Xin Zhao, and Kaiqi Huang.  Got-10k: A large high-diversity benchmark for generic object tracking in the wild.  TPAMI, 43(5):1562–1577, 2011.
  * [20] Ben Kang, Xin Chen, Dong Wang, Houwen Peng, and Huchuan Lu.  Exploring lightweight hierarchical vision transformers for efficient visual tracking.  In ICCV, 2023.
  * [21] Yutong Kou, Jin Gao, Bing Li, Gang Wang, Weiming Hu, Yizheng Wang, and Liang Li.  Zoomtrack: Target-aware non-uniform resizing for efficient visual tracking.  In NeurIPS, 2023.
  * [22] Matej Kristan, Aleš Leonardis, Jiří Matas, Michael Felsberg, Roman Pflugfelder, Joni-Kristian Kämäräinen, Martin Danelljan, Luka Čehovin Zajc, Alan Lukežič, Ondrej Drbohlav, et al.  The eighth visual object tracking vot2020 challenge results.  In ECCVW, 2020.
  * [23] Liting Lin, Heng Fan, Zhipeng Zhang, Yong Xu, and Haibin Ling.  Swintrack: A simple and strong baseline for transformer tracking.  In NeurIPS, 2022.
  * [24] Sihao Lin, Hongwei Xie, Bing Wang, Kaicheng Yu, Xiaojun Chang, Xiaodan Liang, and Gang Wang.  Knowledge distillation via the target-aware transformer.  In CVPR, 2022.
  * [25] Tsung-Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, and C Lawrence Zitnick.  Microsoft coco: Common objects in context.  In ECCV, 2014.
  * [26] Chuofan Ma, Qiushan Guo, Yi Jiang, Ping Luo, Zehuan Yuan, and Xiaojuan Qi.  Rethinking resolution in the context of efficient video recognition.  NeurIPS, 2022.
  * [27] Christoph Mayer, Martin Danelljan, Goutam Bhat, Matthieu Paul, Danda Pani Paudel, Fisher Yu, and Luc Van Gool.  Transforming model prediction for tracking.  In CVPR, 2022.
  * [28] Matthias Mueller, Neil Smith, and Bernard Ghanem.  A benchmark and simulator for uav tracking.  In ECCV, 2016.
  * [29] Matthias Muller, Adel Bibi, Silvio Giancola, Salman Alsubaihi, and Bernard Ghanem.  Trackingnet: A large-scale dataset and benchmark for object tracking in the wild.  In ECCV, 2018.
  * [30] Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, et al.  Pytorch: An imperative style, high-performance deep learning library.  NeurIPS, 2019.
  * [31] Jianbing Shen, Yuanpei Liu, Xingping Dong, Xiankai Lu, Fahad Shahbaz Khan, and Steven Hoi.  Distilled siamese networks for visual tracking.  TPAMI, 44(12):8896–8909, 2021.
  * [32] Mingxing Tan and Quoc Le.  Efficientnet: Rethinking model scaling for convolutional neural networks.  In ICML, 2019.
  * [33] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin.  Attention is all you need.  In NIPS, 2017.
  * [34] Ning Wang, Wengang Zhou, Jie Wang, and Houqiang Li.  Transformer meets tracker: Exploiting temporal context for robust visual tracking.  In CVPR, 2021.
  * [35] Wenhui Wang, Furu Wei, Li Dong, Hangbo Bao, Nan Yang, and Ming Zhou.  Minilm: Deep self-attention distillation for task-agnostic compression of pre-trained transformers.  NeurIPS, 2020.
  * [36] Yulin Wang, Rui Huang, Shiji Song, Zeyi Huang, and Gao Huang.  Not all images are worth 16x16 words: Dynamic transformers for efficient image recognition.  NeurIPS, 2021.
  * [37] Zhangyang Wang, Shiyu Chang, Yingzhen Yang, Ding Liu, and Thomas S Huang.  Studying very low resolution recognition using deep networks.  In CVPR, 2016.
  * [38] Xing Wei, Yifan Bai, Yongchao Zheng, Dahu Shi, and Yihong Gong.  Autoregressive visual tracking.  In CVPR, 2023.
  * [39] Kan Wu, Houwen Peng, Zhenghong Zhou, Bin Xiao, Mengchen Liu, Lu Yuan, Hong Xuan, Michael Valenzuela, Xi Stephen Chen, Xinggang Wang, et al.  Tinyclip: Clip distillation via affinity mimicking and weight inheritance.  In ICCV, 2023.
  * [40] Qiangqiang Wu, Tianyu Yang, Ziquan Liu, Baoyuan Wu, Ying Shan, and Antoni B Chan.  Dropmae: Masked autoencoders with spatial-attention dropout for tracking tasks.  In CVPR, 2023.
  * [41] Fei Xie, Chunyu Wang, Guangting Wang, Yue Cao, Wankou Yang, and Wenjun Zeng.  Correlation-aware deep tracking.  In CVPR, 2022.
  * [42] Bin Yan, Houwen Peng, Jianlong Fu, Dong Wang, and Huchuan Lu.  Learning spatio-temporal transformer for visual tracking.  In ICCV, 2021.
  * [43] Bin Yan, Houwen Peng, Kan Wu, Dong Wang, Jianlong Fu, and Huchuan Lu.  Lighttrack: Finding lightweight neural networks for object tracking via one-shot architecture search.  In CVPR, 2021.
  * [44] Botao Ye, Hong Chang, Bingpeng Ma, Shiguang Shan, and Xilin Chen.  Joint feature learning and relation modeling for tracking: A one-stream framework.  In ECCV, 2022.
  * [45] Sergey Zagoruyko and Nikos Komodakis.  Paying more attention to attention: Improving the performance of convolutional neural networks via attention transfer.  In ICLR, 2017.