# Tunable Electromagnetically Induced Multi-Transparencies in Hybrid
Optomechanical system Incorporating Atomic Medium

M. Hunza  Department of Physics, COMSATS University Islamabad, 45550,
Pakistan,  M. Asjad<EMAIL_ADDRESS>Department of Applied Mathematics and
Sciences, Khalifa University, 127788, Abu Dhabi, UAE,  T. Abbas  Department of
Physics, COMSATS University Islamabad, 45550, Pakistan,  M. Qasymeh
Electrical and Computer Engineering Department Abu Dhabi University, 5991 Abu
Dhabi, UAE,  B. Teklu  Department of Applied Mathematics and Sciences, Khalifa
University, 127788, Abu Dhabi, UAE,  Center for Cyber-Physical Systems (C2PS),
Khalifa University, 127788, Abu Dhabi, UAE,  H. Eleuch  Department of Applied
Physics and Astronomy, University of Sharjah, Sharjah, UAE,  Institute for
Quantum Science and Engineering,Texas AM University, College Station, TX 77843
USA.

(August 28, 2024)

###### Abstract

We consider a hybrid atom-optomechanical system incorporating N identical
$\Lambda$-type atoms. The system is subjected to dual optical and phononic
drives. We show that by exploiting the optomechanical linear and quadratic
interactions, multiple electromagnetic transparency windows are attained.
Furthermore, owing to the incorporated mechanical pump, the transparency
windows are controlled and tuned. For instance, by adjusting the phase of the
external mechanical pump, additional controlling parameters are enabled, and
the absorption/emission profiles are enhanced. Our present study provides an
efficient approach to modifying propagating signals inside the quantum devices
incorporating cavity-optomechanical systems.

##  I Introduction

Since the achievement of electromagnetically induced transparency (EIT),
several significant advancements in optical physics have been accessible [1,
2]. EIT is now an active area of research in both theoretical and experimental
domains. This phenomenon has ushered into different applications such as the
propagation of ultraslow light pulses [3, 4, 5], cooling of ground-state atoms
[6], enhancement in the efficiency of nonlinear conversion [7, 8] etc. Further
progress has also been made in the context of coherent control of a massive
mechanical oscillator, leading to optomechanical induced transparency (OMIT)
[9, 10, 11] as well as to optomechanical induced absorption (OMIA) [12, 13,
14]. OMIA is analogous to the EIT phenomenon whichhas been extensively studied
in the presence of atomic media [15, 16, 17]. Devices based on nano and micro
mechanical systems can be coupled to many systems through magnetic coupling
[18], optical dipoleforce [19, 20] and by the radiation pressure caused by the
optical field [21, 22, 23, 24, 25, 26] that paves the way for configurability
in quantum systems.

The generic utility in optomechanical induced transparency (OMIT) is the
interaction of the optical mode with the mechanical mode that is noticeable in
different experimental as well as theoretical developments [27, 28]. OMIT
dependent phenomena like, nonlinear quantum domain [29, 30], second and higher
order side-bands [31, 32] include linear coupling. Likewise, configurations
like membrane placed in-between the cavity [33, 34, 35], ultracold atoms [36,
37], have been realized under quadratic optomechanical coupling (QOC).
Furthermore, quadratic coupling of the mechanical oscillator with the cavity
field reveals the phenomena of slow light and two-phonon OMIT [38, 39]. Such
coupled systems can be used to measure displacements with optimum precision in
the field of quantum metrology [40], quantum probes [41] as well as the
detection of gravitational waves [42], engineering the micro-macro sytem
entanglement [43, 44, 45, 46, 47] with important implications for quantum
logic gates based on EIT schemes [48].

Motivated from all of the above stated work, we investigate the behavior of
OMIT, where an atomic ensemble is present, along with linear as well as
quadratic coupling that are interacting in the simultaneous presence of the
strong optical field as well as a phonon pump of the external mechanical mode
with the cavity mode field. Multiple tunable transparency windows are
significantly observed in the presence of atomic media in cavity
optomechanical systems.

The paper is organized as follows. In Section II, the details of the
theoretical model at hand is discussed along with the Langvein equations
whereas Section III, contains the graphical representations of the analytical
solution of the optical response of the weak field. Section IV, contains the
conclusion of the results.

##  II model and equations

We consider an ensemble of N identical $\Lambda$-type three-level atoms
confined inside an optical cavity, that has an oscillating membrane placed
between the optical cavity as shown in Fig. 1. Here, we are not considering
the membrane to be present in the equilibrium position, but rather, a more
generalized scenario where the membrane is oscillating and is coupled to both
the linear and quadratic optomechanical coupling via modes of the cavity. For
the $i$th atom, a classical control field with amplitude $\Omega$ and
frequency $\nu$ induces transition between levels
$|a\rangle\leftrightarrow|c\rangle$ whereas, the cavity field with frequency
$\omega_{o}$ interacts with atoms making a transition between the atomic
levels $|a\rangle\leftrightarrow|b\rangle$. In addition to this, we have an
external mechanical mode that induces a phononic pump that is also responsible
for the vibrations in the membrane. The schematics are presented in Fig. ( 1).

Figure 1: Schematic of the system: Hybrid atom-optomechanical system with a
membrane that is coupled linearly and quadratically to the cavity modes. The
cavity is driven by strong laser field with amplitude $\Omega$ and frequency
$\omega_{l}$ and a weak laser beam of amplitude $\tilde{\epsilon_{m}}$ and
frequency $\omega_{p}$.

Considering $\hbar=1$ , the Hamiltonian of the system is given by,

$H=H_{C}+H_{A}+H_{M}+H_{MC}+H_{AC}+H_{CL}.$ (1)

The first term in the above Hamiltonian represents free energy of the cavity
field which is given as,

$\displaystyle H_{C}$ $\displaystyle=$ $\displaystyle\omega_{o}c^{\dagger}c.$
(2)

Here, ${c}^{\dagger}$ and ${c}$ design respectively the creation and
annihilation operators of the field. The second term in Eq. (1) represents
free energy of an ensemble of N identical $\Lambda$ \- type atoms which can be
written as,

$\displaystyle H_{A}$ $\displaystyle=$
$\displaystyle\sum_{i=1}^{n}(\omega_{a}\sigma_{aa}^{i}+\omega_{b}\
\sigma_{bb}^{i}+\omega_{c}\ \sigma_{cc}^{i}).$ (3)

The Bohr frequencies $\omega_{a}$, $\omega_{b}$ and $\omega_{c}$ relate to the
atomic level $|a\rangle$, $|b\rangle$ and $|c\rangle$ respectively, together
with the respective atomic operators $\sigma_{aa}$, $\sigma_{bb}$ and
$\sigma_{cc}$ that correspond to these atomic levels. The energy for the
mechanical motion of the oscillating mirror is represented by $H_{m}$ in Eq.
(1) is given by,

$H_{M}=\dfrac{\omega_{m}}{2}(p^{2}+q^{2}).$ (4)

Here, $\omega_{m}$ is the frequency, $p$ is the momentum and $q$ is the
displacement of the oscillating mirror. The coupling between cavity field and
oscillating mirror represented by $H_{CM}$ in Eq. (1) is given by,

$H_{CM}=g_{1}qc^{\dagger}c+g_{2}q^{2}c^{\dagger}c.$ (5)

The linear and the quadratic coupling constants between cavity field and
oscillating mirror are represented here as, $g_{1}$ and $g_{2}$ respectively.
These constants are defined as $g_{1}=\frac{\partial\omega_{c}}{\partial
q}x_{o}$ and $g_{2}=\frac{\partial^{2}\omega_{c}}{\partial
q^{2}}\frac{x_{o}^{2}}{2}$. The zero point fluctuations in displacement of
oscillating mirror are represented as $x_{o}$. The interaction of fields (both
quantized and classical) with the atomic ensembles can be represented as,

$H_{AC}=\sum_{i=1}^{N}(gc\sigma_{ab}^{i}+\Omega\exp(-i\nu
t)\sigma_{ac}^{i}+H.c).$ (6)

The interaction between atomic levels $|a\rangle\leftrightarrow|b\rangle$ with
quantized field along with a coupling constant is shown as,
$g=-\mu(\omega_{0}/2V\epsilon_{0})$. The electric dipole between two levels is
represented as $\mu$ whereas, $V$ is the volume of the cavity. The atomic
interaction between levels $|a\rangle\leftrightarrow|c\rangle$ are driven by
classical field having the Rabi frequency $\Omega$. Furthermore the cavity is
driven by a pump field and a stokes field with frequency $\omega_{l}$ and
$\omega_{p}$ respectively. The last term in Eq. (1) accounts for strong
optical pump, probe field and phonon driving and is given by

$\displaystyle
H_{CL}=ic^{\dagger}(\epsilon_{l}e^{-i\omega_{l}t}+\epsilon_{p}e^{-i\omega_{p}t})-q\tilde{\epsilon}_{m}e^{i\delta
t}-H.c,$ (7)

where $\epsilon_{l}=(2\kappa\wp_{l}/\omega_{l})^{1/2}$ is the amplitude of
pump field whereas $\epsilon_{p}=(2\kappa\wp_{p}/\omega_{p})^{1/2}$
corresponds to the probe field’s amplitude. $\wp_{l}(\wp_{p})$ is the power of
pump and probe field. The last term mentioned in Eq.(7) describes the phonon
pump of the mechanical mode where,
$\tilde{\epsilon}_{m}=\epsilon_{m}e^{i\Phi_{m}}$ is the strength of the phonon
pump with phase $\Phi_{m}$ and $\delta=\omega_{p}-\omega_{l}$. Now we define
atomic ensemble operator $A=(1/N)^{1/2}\sum_{i=1}^{n}\sigma_{ba}^{i}$ and
$C=(1/N)^{1/2}\sum_{i=1}^{n}\sigma_{bc}^{i}$ which satisfy the Bosonic
relation $[C,C^{\dagger}]=1$, $[A,A^{\dagger}]=1$. Then the Heisenberg-
Langevin equation for this system reads

$\displaystyle\dot{q}$ $\displaystyle=$ $\displaystyle\omega_{m}p,$
$\displaystyle\dot{p}$ $\displaystyle=$
$\displaystyle-\omega_{m}q-\gamma_{m}p-g_{1}c^{\dagger}c-2g_{2}c^{\dagger}cq+\tilde{\epsilon}_{m}\cos(\delta
t)+\xi,$ $\displaystyle\dot{c}$ $\displaystyle=$
$\displaystyle-[\kappa+i(\omega_{0}+g_{1}q+g_{2}q^{2})]c-iG_{a}A+\epsilon_{l}e^{-i\omega_{l}t}+\epsilon_{p}e^{-i\omega_{p}t}$
$\displaystyle+$ $\displaystyle\sqrt{2\kappa}c_{in},$ $\displaystyle\dot{A}$
$\displaystyle=$ $\displaystyle-(i\omega_{ab}+\gamma_{1})A-iG_{a}c-i\Omega
e^{-i\upsilon t}C+\sqrt{2\gamma_{1}}A_{in},$ $\displaystyle\dot{C}$
$\displaystyle=$ $\displaystyle-(i\omega_{cb}+\gamma_{2})C-i\Omega
e^{i\upsilon t}A+\sqrt{2\gamma_{2}}C_{in},$ (8)

where $G_{a}=gN^{1/2}$ is the effective atom filed coupling. Here, the atomic
decay rate between the three level atom is represented by $\gamma$. More
precisely, $\gamma_{1}(\gamma_{2})$ is the decay rate of transition
$|a\rangle\leftrightarrow|b\rangle(|a\rangle\leftrightarrow|c\rangle)$.
Similarly the transition frequencies for
$|a\rangle\leftrightarrow|b\rangle(|c\rangle\leftrightarrow|b\rangle)$ are
represented as $\omega_{ab}=\omega_{a}-\omega_{b}$ and
$\omega_{cb}=\omega_{c}-\omega_{b}$. The decay rate of the oscillating mirror
is shown by $\gamma_{m}$ whereas, $\kappa$ is the decay rate of cavity field.
Here, $\xi,A_{in}$, $C_{in}$ and $c_{in}$ represent the noise terms of mirror,
atomic parts, and cavity respectively. The cavity field evolution is related
to the oscillating mirror, that oscillates quadratically and is represented as
$q^{2}$ as shown in Eq. (II) which corresponds to the mechanical mode of
oscillating mirror. Therefore we define $Q=\langle q^{2}\rangle$, $P=\langle
p^{2}\rangle$ and $X=\langle pq+qp\rangle$, that we need to determine their
evolution as well. The time evolution of these operators are given as follows,

$\displaystyle\dot{P}$ $\displaystyle=$
$\displaystyle-2\gamma_{m}P-(\omega_{m}+2g_{2}\langle
c^{\dagger}c\rangle)X-2g_{1}\langle c^{\dagger}c\,p\rangle$ $\displaystyle+$
$\displaystyle 2\gamma_{m}(1+2n_{th})+2\langle
p\rangle\tilde{\epsilon}_{m}\cos(\delta t),$ $\displaystyle\dot{X}$
$\displaystyle=$
$\displaystyle-\gamma_{m}X+2P\omega_{m}-2(\omega_{m}+2g_{2}\langle
c^{\dagger}c\rangle)Q-2g_{1}\langle c^{\dagger}cq\rangle$ (9)
$\displaystyle+2\langle q\rangle\tilde{\epsilon}_{m}\cos(\delta t).$

The mean occupation number of phonon of a thermal reservoir is represented as
$n_{th}$. The averages of noise operators follow the vanishing statistical
mean as, $\langle f(t)\rangle=0$ where, $f$=$\xi$, $A_{in}$, $C_{in}$ and
$c_{in}$.

The system of nonlinear quantum Langevin equations (II )- (II) can be
linearized by writing operators as $\langle O\rangle=\langle
O_{s}\rangle+\delta\langle O\rangle$ for ($O=p,q,c,A,C,P,Q,X$) where $O_{s}$
represents the steady state solution and $\delta\langle O\rangle$ is the small
fluctuation around steady state. By defining the following slowly varying
operators, $c\rightarrow ce^{-i\omega_{l}t}$, $A\rightarrow
Ae^{-i\omega_{l}t}$ and $C\rightarrow Ce^{i(\upsilon-\omega_{l})t}$ and
ignoring the fast frequency dependence of the field, the steady state solution
are given by $p_{s}=0$, $q_{s}=\frac{-G_{1}c_{s}}{\omega_{m,eff}}$,
$P_{s}=(1+2n_{th})$, $X_{s}=0$, $A_{s}=\frac{-iG_{a}c_{s}-i\Omega
C_{s}}{i\Delta_{a}+\gamma_{1}}$ and $C_{s}=\frac{-i\Omega
A_{s}}{i\Delta_{b}+\gamma_{2}}$,

$\displaystyle c_{s}$ $\displaystyle=$
$\displaystyle\frac{\epsilon_{l}}{\kappa+i\Delta_{0}+[{G^{2}_{a}}/({i\Delta_{a}+\gamma_{1}+\frac{\Omega^{2}}{i\Delta_{b}+\gamma_{2}}})]},$
$\displaystyle Q_{s}$ $\displaystyle=$
$\displaystyle\frac{\omega_{m}P_{s}}{\omega_{m,eff}}+\frac{G_{1}^{2}|c_{s}|^{2}-G_{1}c_{s}\tilde{\epsilon}_{m}}{\omega^{2}_{m,eff}}.$
(10)

Here, ${\omega_{m,eff}=\omega_{m}+2g_{2}|c_{s}|^{2}}$ is the effective
mechanical frequency. The effective linear and quadratic coupling terms are
redefined as $G_{1}=g_{1}c_{s}$ and $G_{2}=g_{2}c_{s}$, whereas,
$\Delta_{0}=\omega_{0}-\omega{l}+g_{1}q_{s}+g_{2}Q_{s}$,
$\Delta_{b}=\omega_{cb}+\upsilon-\omega_{l}$ and
$\Delta_{a}=\omega_{ab}-\omega_{l}$. Then the corresponding linearized
equations of motion are given by

$\displaystyle\delta\dot{q}$ $\displaystyle=$ $\displaystyle\omega_{m}\delta
p,\,\,\,\,\delta\dot{Q}=\omega_{m}\delta\hat{X},$ $\displaystyle\delta\dot{p}$
$\displaystyle=$ $\displaystyle-\gamma_{m}\delta p-\omega_{m,eff}\delta
q-\frac{G}{2}(\delta c+\delta c^{\dagger})+\tilde{\epsilon}_{m}\cos(\delta
t),$ $\displaystyle\delta\dot{c}$ $\displaystyle=$
$\displaystyle-(\kappa+i\Delta_{0})\delta c-iG_{1}\delta q-iG_{2}\delta
Q-iG_{a}\delta A+\epsilon_{p}e^{-i\delta t},$ $\displaystyle\delta\dot{A}$
$\displaystyle=$ $\displaystyle-(\gamma_{1}+i\Delta_{a})\delta A-iG_{a}\delta
c-i\Omega\delta C,$ $\displaystyle\delta\dot{P}$ $\displaystyle=$
$\displaystyle-2\gamma_{m}\delta P-\omega_{m,eff}\delta
X-[G_{sm}-2\tilde{\epsilon}_{m}\cos(\delta t)]\delta p,$
$\displaystyle\delta\dot{X}$ $\displaystyle=$ $\displaystyle 2\omega_{m}\delta
P-\gamma_{m}\delta X-2\omega_{m,eff}\delta Q$ (11)
$\displaystyle-[G_{sm}-2\tilde{\epsilon}_{m}\cos(\delta t)]\delta q$
$\displaystyle+(4G_{2}Q_{s}+2G_{1}q_{s})(\delta c+\delta c^{\dagger}),$

where $G=2(G_{1}+2G_{2}q_{s})$ and $G_{sm}=2c_{s}G_{1}$. In above equation,
the higher order terms, i.e, $\delta Q\delta c$ and $\delta q\delta c\delta
c^{\dagger}$ are ignored. In order to obtain the solution of Eq.(II), we
consider the ansatz upto first order sideband as [49, 50, 51]:
$\delta{O}=\delta O_{+}e^{-i\delta t}+\delta O_{-}e^{i\delta t}$ where,
$\delta O=(\delta p,\delta q,\delta c,\delta P,\delta Q,\delta X,\delta
A,\delta C)$. By the virtue of this ansatz, we are able to obtain two sets of
equations that arise due to the comparison of coefficients such as $e^{i\delta
t}$ and $e^{-i\delta t}$, that are mentioned in Appendix-A, one gets

$\delta c_{+}=\frac{\epsilon_{p}+i\beta
G_{2}S}{\chi_{c}-\frac{iG_{1}\omega_{m}(G-\tilde{\epsilon}_{m})}{\chi_{d}}+i\alpha
G_{2}S\omega_{m}+\frac{G^{2}_{a}\chi_{b}}{\chi_{a}\chi_{b}+\Omega^{2}}},$ (12)

where,

$\displaystyle\alpha$ $\displaystyle=$ $\displaystyle
GG_{sm}\omega_{m}(2\chi_{d}-\chi_{e}\chi_{g})+G_{t}\chi_{d}\left(-2\gamma^{2}_{m}+3i\gamma_{m}\delta+\delta^{2}\right),$
$\displaystyle\beta$ $\displaystyle=$
$\displaystyle\frac{\tilde{\epsilon}_{m}G_{sm}\omega^{2}_{m}(2\chi_{d}-\chi_{e}\chi_{g})}{\chi_{d}\chi_{f}(\delta+i\gamma_{m})},$
$\displaystyle S$ $\displaystyle=$
$\displaystyle\frac{1}{\gamma_{m}\chi_{d}\chi_{f}-i\delta\chi_{d}\chi_{f}},$
(13)

with $\chi_{a}=\gamma_{1}-i\delta+i\Delta_{a}$,
$\chi_{b}=\gamma_{2}-i\delta+i\Delta_{b}$,
$\chi_{c}=-i\delta+i\Delta_{o}+\kappa$,
$\chi_{d}=-i\gamma_{m}\delta-\delta^{2}+\omega_{m}\omega_{m,eff}$,
$\chi_{e}=2\omega_{m}\omega_{m,eff}+(\gamma_{m}-i\delta)(2\gamma_{m}-i\delta)$,
$\chi_{f}=2\omega_{m}\omega_{m,eff}(2\gamma_{m}-i\delta)-i\delta\chi_{e}$,
$\chi_{g}=-2\gamma^{2}_{m}+3i\gamma_{m}\delta+\delta^{2}+2\omega_{m}\omega_{m,eff}$
and $G_{t}=4(G_{1}q_{s}+2G_{2}Q_{s})$.

In order to study OMIT, we have to find the response of our system to the
probe field of the hybrid optomechanical system. Then by using the standard
input output relation which is given by $c_{out}=\sqrt{\kappa}c-c_{in}$ [52,
53], the transmission of the probe field is defined as ratio of output and
input cavity fields at probe frequency is given by

$\displaystyle t_{P}$ $\displaystyle=$
$\displaystyle\frac{\epsilon_{p}-\sqrt{\kappa}\delta c_{+}}{\epsilon_{p}}.$
(14)

The quadrature $\epsilon_{out}$ the optical components in the output probe
field is in general used to describe the OMIT phenomena which is given as [49,
54],

$\displaystyle\epsilon_{out}$ $\displaystyle=$
$\displaystyle\frac{\sqrt{2\kappa}\delta c_{+}}{\epsilon_{p}}.$ (15)

Here, $\nu_{p}$=Re$[\epsilon_{out}]$ and $\rho_{p}$=Im$[\epsilon_{out}]$
correspond to the absorptive and dispersive coefficients of the output field
at probe frequency. In the next section, the expression of $\delta c_{+}$ is
used to obtain the absorptive(dispersive) profiles of the output field along
with the transmission of the probe field.

##  III Results and discussion

In accordance with the generalized expression for the output probe field
mentioned in Eq. (12) where, all the coupling parameters i.e., LOC ($G_{1}$),
QOC ($G_{2}$), atomic parameter ($G_{a}$) and the driving field $\epsilon_{m}$
are present where we mainly observe the changes in the absorption(dispersion)
and transmission profiles of OMIT with the variation in these aforementioned
parameters. For $G_{1}=G_{2}=G_{a}=0$, a single peak is observed as shown in
Fig. (2). This figure shows the absorption, dispersion and transmission
profiles. The mathematical expression can be extracted from Eq. (12) that
comes out to be as,

$\epsilon_{out}=\frac{\sqrt{2\kappa}\epsilon_{p}}{-i\delta+i\Delta_{o}+\kappa}.$
(16)

From the above expression, it is clear that the denominator consists of a
single root, due to which no splitting of the peak is in absorption
(dispersion) and transmission profiles. The expression of transmission
mentioned in Eq. (14) along with Eq. (12) is utilized for the formation of
transmission profile.

Figure 2: The blue (Red) profile show the real(imaginary) profiles of the
transmission and absorption when all the coupling parameters (LOC and QOC)
along with the atomic parameters are set to be zero. The selected parameters
are taken as, $\Omega=0.7$, $\kappa=0.1$, $\epsilon_{p}=1$, $\hbar=1$,
$\Delta_{o}=1$, $\Phi_{m}=0$, $\omega_{m,eff}=1.0006$, $\Delta_{a}=0.02$,
$\Delta_{b}=0.02$, $\gamma_{1}=0.0001$, $\gamma_{2}=0.0001$,
$\epsilon_{l}=0.05$, $\tilde{\epsilon_{m}}=0$, $\omega_{m}=1$, $n_{th}=0$ and
$\gamma_{m}=0.00016.$

Moreover, in Fig. (3), we observe two peaks in both the transmission and
absorption (dispersion) profiles. By setting the linear optical coupling,
quadratic optical coupling and the amplitude of the driving field
($\epsilon_{m}$) to zero, the expression in Eq. (12) reduces to the following,

$\epsilon_{out}=\frac{\sqrt{2\kappa}(\chi_{a}\chi_{b}+\Omega^{2})}{G^{2}_{a}\chi_{b}+\chi_{c}(\chi_{a}\chi_{b}+\Omega^{2})}.$
(17)

The above mathematical expression gives two roots that are mainly responsible
for the two peaks observed in Fig. (3). In these profiles the atomic parameter
$G_{a}$ is taken into account whereas, linear, quadratic coupling terms
between the mirror and the cavity field, along with the driving field are set
to be zero. Here, the atoms also do not interact directly with the phonon or
quanta of movable mirror.

Figure 3: The blue(red) curves represent the real(imaginary) profiles of both
the transmission and absorption. In this plot the linear and quadratic
coupling are not present whereas, the atomic parameter is present due to which
a transparency window is clearly visible at $\delta=1$. The parameters are
taken as, $\Omega=.01$, $\kappa=0.2$, $\epsilon_{p}=1$, $\hbar=1$,
$\Delta_{o}=1$, $\Phi_{m}=0$, $\omega_{m,eff}=1.006$, $\Delta_{a}=1$,
$\Delta_{b}=1$, $\gamma_{1}=0.30$, $\gamma_{2}=0.01$, $\epsilon_{l}=0.5$,
$\tilde{\epsilon_{m}}=0$, $\omega_{m}=1$, $n_{th}=0$, $G_{a}=1$ and
$\gamma_{m}=0.001$. Figure 4: The blue (red) curves represent the real
(imaginary) profiles of both the transmission and absorption. In this plot the
atomic and quadratic coupling are not present whereas the linear coupling is
present due to which a transparency window is clearly visible at $\delta=1$,
$\Omega=1.0$, $\kappa=0.1$, $\epsilon_{p}=1$, $\hbar=1$, $\Delta_{o}=1$,
$\Phi_{m}=0$, $\omega_{m,eff}=1.006$, $\Delta_{a}=1$, $\Delta_{b}=2$,
$\gamma_{1}=1.01$, $\gamma_{2}=0.01$, $\tilde{\epsilon_{m}}=0$,
$\omega_{m}=1$, $\epsilon_{l}=0.5$, $n_{th}=0$, $G_{1}=0.15$ and
$\gamma_{m}=0.004$.

Correspondingly in Fig. (4), we again observe two peaks in both the
transmission and absorption (dispersion) profiles. These plots are again in
agreement with the phenomenon of EIT. Here, the quadratic coupling and the
atomic parameter are set to zero whereas, only the linear coupling is present.
The mathematical expression in Eq. (12) now contains only the linear coupling
terms, hence its reduced form is,

$\epsilon_{out}=\frac{\sqrt{2\kappa}\chi_{d}}{i\tilde{\epsilon_{m}}G_{1}\omega_{m}-iGG_{1}\omega_{m}+\chi_{c}\chi_{d}}.$
(18)

On the contrary, when both the linear coupling along with the atomic parameter
are present, the absorption and transmission profiles show that two
transparency windows occur at $\delta=1$ and $\delta=3$ as depicted in Fig.
(5). This is due to the presence of linear coupling term whereas, the second
peak becomes visible, owing to the atomic parameter. When the atomic parameter
$G_{a}$ is set to be zero, the second transparency window vanishes. Again, by
considering Eq. (12), we obtain our simplified expression shown below where,
the QOC and the driving field is set to zero. In this case the expression
comes out to be,

$\epsilon_{out}=\frac{\sqrt{2\kappa}\epsilon_{p}\chi_{d}\left(\chi_{a}\chi_{b}+\Omega^{2}\right)}{\chi_{b}\chi_{d}G^{2}_{a}+\left(\chi_{a}\chi_{b}+\Omega^{2}\right)(\chi_{c}\chi_{d}-2iG^{2}_{1}\omega_{m})}.$
(19)

It is clear that three roots are obtained from the above expression that gives
us three different peaks.

Figure 5: The blue (red) curves represent the real(imaginary) profiles of both
the transmission and absorption (dispersion). In this plot the linear coupling
and the atomic parameter are present whereas the quadratic parameter is not
present. The parameters are chosen to be as, $\Omega=1.0$, $\kappa=0.3$,
$\epsilon_{p}=1$, $\hbar=1$, $\Delta_{o}=1$, $\Phi_{m}=0$,
$\omega_{m,eff}=1.006$, $\Delta_{a}=1$, $\Delta_{b}=1$, $\gamma_{1}=0.35$,
$\gamma_{2}=0.075$, $\epsilon_{m}=0$, $\omega_{m}=1$, $n_{th}=0$,
$\epsilon_{l}=0.05$, $G_{a}=2$, $G_{1}=0.15$ and $\gamma_{m}=0.0016$.

Subsequently, the behavior of the quadratic coupling is observed in Fig. (6).
The profiles distinctly show four explicit peak that arise due to the
simultaneous presence of $G_{a}$ and both the $G_{1}$($G_{2}$) coupling terms.

Figure 6: The blue(red) curves represent the real(imaginary) profiles of both
the transmission and absorption(dispersion). In this plot the linear and
quadratic coupling and the atomic parameter are present simultaneously. The
parameters are chosen to be as, $\Omega=0.6$, $\kappa=0.1$, $\epsilon_{p}=1$,
$\hbar=1$, $\Delta_{o}=1$, $\Phi_{m}=0$, $\omega_{m,eff}=1.006$,
$\Delta_{a}=0.02$, $\Delta_{b}=0.03$, $\gamma_{1}=0.001$, $\gamma_{2}=0.001$,
$\epsilon_{m}=0$, $\omega_{m}=1$, $n_{th}=0$, $\epsilon_{l}=0.05$,
$G_{1}=0.19$, $G_{a}=0.22$, $G_{2}=0.26$ and $\gamma_{m}=0.0015$.

Due to atomic factor we observe a dip at $\delta=0.6$ whereas, we observe two
dips at $\delta=1$ and $\delta=2.2$ that arise due to the linear and quadratic
coupling terms respectively that are responsible for the radiation pressure.
Ideally we observe four peaks over here that arise due to four different roots
of $\delta$. Keeping the side-band resolved limit i.e., $\omega_{m}>>\kappa$,
the splitting between the normal modes is observed. Here, single phonon
process is observed at $\delta\approx\omega_{m}$, although the external
driving field of the membrane at the instant is set to zero. In such cases
multiple optomechanical induced transparencies (MOMIT) are observed that are
tunable by further controlling the phase and the intensity of external driving
field. In tunable MOMIT, we consider the complete expression given in Eq. (12)
together with Eq. (II) by considering the linear optical coupling, quadratic
optical coupling, atomic parameter and the external driving part that
generates a phononic pump, due to which the variation in phase changes the
intensity of the quadratic peak which is exhibited in Fig. (7). This
comprehends us that when the phase is increased the intensity of the quadratic
coupling peak decreases. Furthermore, we also observe that increasing the
phase changes the origination interval of the LOC and atomic peak i.e., the
peak shifts towards the left side which makes its tunable for different
purposes such as slowing down the speed of the probe light as well as
modification in signal propagation is quantum devices.

Figure 7: The curves depicts the behavior of the system when the linear,
quadratic and atomic parameters along with the external field are switched on.
The green curve shows the shift towards the left side as well as the intensity
of the QOC decreases as well when $\Phi_{m}$=$\pi$. Similarly when $\Phi_{m}$
decreases from $\pi/2$ to $0$, the peaks shift towards the right side with
rise in intensity of QOC as shown in red and blue curve respectively.
$\Omega=0.5$, $\kappa=0.2$, $\epsilon_{p}=1$, $\hbar=1$, $\Delta_{o}=1$,
$\Phi_{m}=0$, $\Phi_{m}=\pi$, $\Phi_{m}=\frac{\pi}{2}$,
$\omega_{m,eff}=1.006$, $\Delta_{a}=0.1$, $\Delta_{b}=0.1$,
$\gamma_{1}=0.001$, $\gamma_{2}=0.001$, $\epsilon_{m}=0.3$, $\omega_{m}=1$,
$n_{th}=0$, $\gamma_{m}=0.00015$, $G_{1}=0.23$, $G_{a}=0.26$ and
$G_{2}=0.025$.

##  IV Conclusion

In this work, we have studied the optical response of a hybrid optomechanical
system and observed the optomechanical-induced transparency (OMIT) phenomenon.
A three-level $\Lambda$-type atomic ensemble placed inside an optical cavity
is considered. We have shown that multiple transparency windows can be
attained owing to the simultaneous presence of linear optical coupling (LOC),
quadratic optical coupling (QOC), and atomic parameters. The strong coupling
of the dielectric membrane with the optical cavity is the optimal condition to
obtain quantum control of the mechanical system. It then follows that the
induced transparency windows are optimized by controlling the LOC and the QOC.
The constructive and destructive interferences can be also adjusted by
modifying the external driving fields.

##  Appendix A Equation of motions

The set of equations utilizes to acquire the final expression of $\delta
c_{+}$ in Eq. (12) is given by

$\displaystyle-i\delta(\delta q_{+})$ $\displaystyle=$ $\displaystyle\delta
p_{+}\omega_{m}$ (20) $\displaystyle(-i\delta+\gamma_{m})(\delta p_{+})$
$\displaystyle=$ $\displaystyle-\omega_{m,eff}\delta q_{+}-G\delta
c_{+}+\tilde{\epsilon}_{m},$ (21) $\displaystyle\chi_{c}(\delta c_{+})$
$\displaystyle=$ $\displaystyle-i(G_{1}\delta q_{+}+G_{2}\delta
Q_{+})+\epsilon_{p}$ (22) $\displaystyle-$ $\displaystyle iG_{a}\delta A_{+},$
$\displaystyle\chi_{a}(\delta A_{+})$ $\displaystyle=$ $\displaystyle-
iG_{a}\delta c_{+}-i\Omega\delta C_{+},$ (23) $\displaystyle\chi_{b}\delta
C_{+}$ $\displaystyle=$ $\displaystyle-i\Omega\delta A_{+},$ (24)
$\displaystyle-i\delta(\delta Q_{+})$ $\displaystyle=$
$\displaystyle\omega_{m}\delta X_{+},$ (25)
$\displaystyle(-i\delta+2\gamma_{m})(\delta P_{+})$ $\displaystyle=$
$\displaystyle-\omega_{m,eff}\delta X_{+}-G_{sm}\delta p_{+},$ (26)
$\displaystyle(-i\delta+\gamma_{m})(\delta X_{+})$ $\displaystyle=$
$\displaystyle 2\omega_{m}\delta P_{+}-2\omega_{m,eff}\delta Q_{+}$ (27)
$\displaystyle-$ $\displaystyle G_{t}\delta c_{+}-G_{sm}\delta q_{+},$

where $\chi_{a_{o}}=\gamma_{1}+i\delta+i\Delta_{a}$,
$\chi_{b_{o}}=\gamma_{2}+i\delta+i\Delta_{b}$,
$\chi_{c_{o}}=i\delta+i\Delta_{o}+\kappa$. Similarly the equations for $\delta
O_{-}$ can be collected as follows.

$\displaystyle i\delta(\delta q_{-})$ $\displaystyle=$ $\displaystyle\delta
p_{-}\omega_{m},$ (28) $\displaystyle(i\delta+\gamma_{m})(\delta p_{-})$
$\displaystyle=$ $\displaystyle-\omega_{m,eff}\delta q_{-}-G\delta
c_{-}+\tilde{\epsilon}_{m},$ (29) $\displaystyle\chi_{c_{o}}(\delta c_{-})$
$\displaystyle=$ $\displaystyle-i(G_{1}\delta q_{-}+G_{2}\delta Q_{-})$ (30)
$\displaystyle-$ $\displaystyle iG_{a}\delta A_{-},$
$\displaystyle\chi_{a_{o}}(\delta A_{-})=$ $\displaystyle-iG_{a}\delta
c_{-}-i\Omega\delta C_{-},$ (31) $\displaystyle\chi_{b_{o}}(\delta C_{-})$
$\displaystyle=$ $\displaystyle-i\Omega\delta A_{-},$ (32) $\displaystyle
i\delta(\delta Q_{-})$ $\displaystyle=$ $\displaystyle\omega_{m}\delta X_{-},$
(33) $\displaystyle(i\delta+2\gamma_{m})(\delta P_{-})$ $\displaystyle=$
$\displaystyle-\omega_{m,eff}\delta X_{-}-G_{sm}\delta p_{-},$ (34)
$\displaystyle(i\delta+\gamma_{m})(\delta X_{-})$ $\displaystyle=$
$\displaystyle 2\omega_{m}\delta P_{-}-2\omega_{m,eff}\delta Q_{-}$ (35)
$\displaystyle-$ $\displaystyle G_{t}\delta c_{-}-G_{sm}\delta q_{-}.$

## References

  * [1] S. E. Harris, Phys. Today 50, 36 (1997).
  * [2] M. O. Scully and M. S. Zubairy, _Quantum Optics, Cambridge University Press_ (1997).
  * [3] L. V. Hau, S. E. Harris, Z. Dutton and C. H. Behroozi, Nat. 397, 594–598 (1999).
  * [4] M. M. Kash, V. A. Sautenkov, A. S. Zibrov, L. Hollberg, G. R. Welch, M. D. Lukin, Y. Rostovtsev, E. S. Fry, and M. O. Scully, Phys. Rev. Lett. 82, 5229 (1999).
  * [5] N Boutabba, H Eleuch, Applied Mathematics & Information Sciences 7, 1505 (2013).
  * [6] R. Lechner, C. Maier, C. Hempel, P. Jurcevic, B. P. Lanyon, T. Monz, M. Brownnutt, R. Blatt, and C. F. Roos, Phys. Rev. A 93, 053401 (2016).
  * [7] G. Z. Zhang, K. Hakuta, and B. P. Stoicheff, Phys. Rev. Lett. 71, 3099 (1993).
  * [8] P. R. Hemmer, D. P. Katz, J. Donoghue, M. Cronin-Golomb, M. S. Shahriar, and P. Kumar, Opt. Lett. 20, 982–984 (1995).
  * [9] S. Huang and G. S. Agarwal, Phys. Rev. A 81, 033830 (2010).
  * [10] S. Weis, R. Riviere. S. Deleglise, E. Gavartin, O. Aecizet, A. Schliesser, and T. J. kippenberg, Sci. 330, 1520-1523 (2010).
  * [11] A. H. Safavi-Naenin, T. P. Mayer Alegre, J. Chan, M. Eichenfield, M. Winger, Q. Lin, J. T. Hill, D. E. Chang, and O. Painter, Nat. 472, 69 (2011).
  * [12] V. Singh, S. J. Bosman, B. H. Schneider, Y. M. Blanter, A. Castellous-Gomez, and G. A. Steele, Nat. Nanotechnol. 9, 820 (2014).
  * [13] K. N. Qu, and G. S. Agarawal, Phys. Rev. A 87, 031802 (2013).
  * [14] F. Hocke, X. Zhou. A. Schliesser, T. J. Kippenberg, H. Huebl, and R. Gross, New. J. Phys. 14, 123037 (2012).
  * [15] Y. Wu and X. Yang, Phys. Rev. A 76, 013832 (2007).
  * [16] W. -X. Yang, A. -X. Chen, R.-K. lee, and Y. Wu, Phys. Rev. A 84, 013835 (2011).
  * [17] X. -T. Xie and M. A. Macovei, Phys. Rev. Lett. 104, 073902 (2010).
  * [18] P. Rabl, P. Cappellaro, M. V. Gurudev Dutt, L. Jiang, J. R. Maze, and M. D. Lukin, Phys. Rev. B 79, 041302(R) (2009).
  * [19] M. Eichenfield, C. P. Michael, R. Perahia, and O. Painter, Nat. Photonics 1, 416–422 (2007).
  * [20] Mo Li, W. H. P. Pernice, C. Xiong, T. Baehr-Jones, M. Hochberg and H. X. Tang, Nat. 456, 480–484 (2008).
  * [21] E. A. Sete, H. Eleuch and C. H. R. Ooi, Phys. Rev. A 92, 033843 (2015).
  * [22] E. A. Sete and H. Eleuch, Phys. Rev. A 85, 043824 (2012).
  * [23] J. Manninen, M. Asjad, E. Selenius, R. Ojajarvi, P. Kuusela and F. Massel, Phys. Rev. A 98, 043831 (2018).
  * [24] M. Aspelmeyer, T. J. Kippenberg and F. Marquardt, Rev. Mod. Phys. 86, 1391 (2014).
  * [25] W. P. Bowen and G. J. Milburn,_Taylor and Francis Group, LLC_ (2016).
  * [26] E. A. Sete and H. Eleuch, Phys. Rev. A 91, 032309 (2015).
  * [27] Y. Chang, T. Shi, Yu-xi. Liu, C. P. Sun, and F. Nori, Phys. Rev. A 83, 063826 (2011).
  * [28] Y. Han, J. Cheng, and L. Zhou, J. Phys. B: At. Mol. Opt. Phys. 44, 165505 (2011).
  * [29] A. Kronwald and F. Marquardt, Phys. Rev. Lett. 111, 133601 (2013).
  * [30] K. Borkje, A. Nunnenkamp, J. D. Teufel and S. M. Girvin, Phys. Rev. Lett. 111, 053602 (2013).
  * [31] H. Xiong, L. G. Si, A. S. Zheng, X. Yang, and Y. Wu, Phys. Rev. A 86, 013815 (2012).
  * [32] H. Xiong, L. G. Si, X. Y. Lu and Y. Wu, Opt. Express 24, 5773 (2016).
  * [33] J. C. Sankey, C. Yang, B. M. Zwickl, A. M. Jayich and J. G. E. Harris, Nat. Phys. 6, 707 (2010).
  * [34] M. Asjad and D. Vitali, J. Phys. B: At. Mol. Opt. Phys. 47, 045502 (2013).
  * [35] M. Asjad, G. S. Agarwal, M. S. Kim, P. Tombesi, G. Di Giuseppe and D. Vitali, Phys. Rev. A 89, 023849 (2014).
  * [36] K. W. Murch, K. L. Moore, S. Gupta and D. M. Stamper-Kurn, Nat. Phys. 4, 561 (2008).
  * [37] T. P. Purdy, D. Brooks, T. Botter, N. Brahms, Z.-Y. Ma and D. M. Stamper-Kurn, Phys. Rev. Lett. 105, 133602 (2010).
  * [38] M. Karuza, C. Biancofiore, M. Bawaj, C. Molinelli, M. Galassi. R. Natali, P. Tombesi, G. Di. Giuseppe, and D. Vitali, Phys. Rev. A 88, 013804 (2013).
  * [39] X. G. Zhan, L. G. Si, A. S. Zheng, and X. Yang, J. Phys. B 46, 0255021 (2013).
  * [40] D. Rugar, R. Budakian, H. J. Mamin and B. W. Chui, Nat. 430, 329 (2004).
  * [41] A. Candeloro, S. Razavian, M. Piccolini, B. Teklu, S. Olivares, and M. G. A. Paris, Entropy 23, 1353 (2021).
  * [42] V. Braginsky and S. P. Vyatchanin, Phys. Lett. A 293, 228 (2002).
  * [43] D. Vitali, S. Gigan, A. Ferreira, H. R. Bohm, P. Tombesi, A. Guerreiro, V. Vedral, A. Zeilinger and M. Aspelmeyer, Phys. Rev. Lett. 98, 030405 (2007).
  * [44] C. F. Ockeloen-Korppi, E. Damskagg, J. M. Pirkkalainen, M. Asjad, A. A. Clerk, F. Massel, M. J. Woolley and M. A. Sillanpaa, Nat. 556, 478 (2018).
  * [45] M. Asjad, S. Zippilli, P. Tombesi and D. Vitali, Phys. Scr. 90, 074055 (2015).
  * [46] M. Asjad, S. Zippilli and D. Vitali, Phys. Rev. A 93, 062307 (2016).
  * [47] B. Teklu, T. Byrnes, and F. S. Khan, Phys. Rev. A 97, 023829 (2018).
  * [48] A. Feizpour, G. Dmochowski, and A. M. Steinberg, Phys. Rev. A 93, 013834 (2016).
  * [49] S. Weis, R. Riviere, S. Deleglise, E. Gavartin, O. Arcizet, A. Schliesser and T. J. Kippenberg, Sci. 330, 1520 (2010).
  * [50] M. Asjad, J. Russ. Laser Res. 34, 278 (2013).
  * [51] M. Asjad, J. Russ. Laser Res. 34, 159 (2013).
  * [52] D. F. Walls and G. J. Milburn, _Quantum Optics Springer-Verlag, Berlin Heidelberg_ (1994).
  * [53] C. W. Gardiner, P. Zoller, _Quantum Noise, Springer Berlin Heidelberg_ (2004).
  * [54] A. H. Safavi-Naeini, T. P. Mayer Alegre, J. Chan, M. Eichenfield, M. Winger, Q. Lin, J. T. Hill, D. E. Chang and O. Painter, Nat. 472, 69 (2011).
  * [55] S. K. Singh, M. Parvez, T. Abbas, J. -Xin. Peng, M. Mazaheri and M. Asjad. Phys. Lett. A 442, 128181 (2022).


# XLDA: Linear Discriminant Analysis for Scaling Continual Learning to Extreme
Classification at the Edge

Karan Shah  Vishruth Veerendranath  Anushka Hebbar  Raghavendra Bhat

###### Abstract

Streaming Linear Discriminant Analysis (LDA) while proven in Class-incremental
Learning deployments at the edge with limited classes (upto 1000), has not
been proven for deployment in extreme classification scenarios. In this paper,
we present: (a) XLDA, a framework for Class-IL in edge deployment where LDA
classifier is proven to be equivalent to FC layer including in extreme
classification scenarios, and (b) optimizations to enable XLDA-based training
and inference for edge deployment where there is a constraint on available
compute resources. We show upto 42$\times$ speed up using a batched training
approach and upto 5$\times$ inference speedup with nearest neighbor search on
extreme datasets like AliProducts (50k classes) and Google Landmarks V2 (81k
classes).

Continual Learning,Extreme Classification,Edge Optimizations,LDA

##  1 Introduction

Training models using all data previously available as well as data that
arrives in small batches, comes with compute constraints at the edge. It
mandates the need for models that can adapt to incrementally arriving data,
without stressing local compute. In classification, this is commonly known as
the Class-incremental Learning objective.

Standard DNN-based classifiers when trained incrementally, suffer from
catastrophic forgetting. Previous literature proposes various ways of dealing
with forgetting; using external memory (Rebuffi et al., 2017; Lopez-Paz &
Ranzato, 2017; Van de Ven et al., 2020), by constraining parameter
(Kirkpatrick et al., 2017; Zenke et al., 2017) or activation drift (Li &
Hoiem, 2017).

By far, most successful methods operate on storing or generating data for
rehearsal, which requires models to be finetuned on the same data again,
increasing the compute footprint. This gets exacerbated when the number of
classes grows significantly over time, as is the case in many use-cases at the
edge like retail, healthcare and education.

Figure 1: Deployment scenario for FC-XLDA conversion

Handling classification with a large number of classes has been an active area
of research, referred to as _Extreme Multi-label Classification_ (XML) (Bhatia
et al., 2016). These methods (Siblini et al., 2018; Dahiya et al., 2021; Yu et
al., 2022; Dahiya et al., 2023) have been used for search and retrieval on
text data consisting of millions of labels. In the image domain, this has been
explored in a few recognition tasks (Schroff et al., 2015; Deng et al., 2019;
Weyand et al., 2020; Ypsilantis et al., 2021; Song et al., 2020), but not in
Class-IL settings.

In this work, we focus on the task-free Class-IL setting (Aljundi et al.,
2019), without replay or rehearsal, which is pragmatic in real-world
deployments. Since our focus here is on class counts exceeding 1k, we assume
feature extractors to be pretrained on a relevant larger subset on the cloud.
At the edge, the problem reduces to learning a classifier incrementally on the
features of new data.

Linear Discriminant Analysis (LDA) (Fisher, 1936) has been popularly used for
dimensionality reduction, and it lends itself very well to Class-IL (Pang et
al., 2005), as a classifier on top of pretrained model features. Hayes & Kanan
(2020) demonstrated this use of LDA, and achieved competitive or better
accuracy against all other techniques on datasets up to 1k classes, even when
the data was presented only once, in a streaming manner.

In this paper, we present EXtreme LDA (XLDA), a framework of optimizations
that enables scaling LDA-based classifiers to extreme multiclass settings. Our
contributions are as follows:

  * •

Class-IL beyond 1k classes: We extend LDA benchmarks on Class-IL up to 81k
classes, showing that it can achieve competitive or on-par accuracy of a
fully-connected (FC) layer trained until convergence, in Section 3. Benchmarks
are covered in Section 4.

  * •

Optimizing training and inference: LDA can be used as a drop-in replacement of
an FC layer with $O(1)$ time complexity training per sample. We parallelize
this for batches in Section 2.2, which results in a worst-case 42$\times$
speedup in training versus FC layers for 50k classes. We optimize inference
for upto 5$\times$ speedup using nearest neighbor search in Section 2.3. The
experiments for these are detailed in Section 4.

  * •

Translating between LDA and FC: We empirically show that an FC layer can be
converted to an LDA layer, while maintaining accuracy, to enable Class-IL at
the edge, under certain assumptions, in Section 4.1.

##  2 Optimizations in XLDA

Background on SLDA has been included in Appendix B.

###  2.1 Conversion from a Fully-Connected layer

In Deep SLDA (Hayes & Kanan, 2020) authors train two components: _feature
extractor_ and the _LDA classifier_ separately. We posit that a model trained
end-to-end using an FC layer (with parameters $W$ and $b$), can be converted
to an LDA classifier (with parameters $\mu$ and shared covariance
$\Sigma_{s}$), assuming samples are normally distributed with identity
covariance across all classes.

Let $W$ be the weights of the last fully connected layer used to train the
model. In order to replace this with an LDA classifier of equivalent
performance, we initialize the covariance matrix $\Sigma$ to either of the two
options below

$\displaystyle\Sigma$ $\displaystyle=\mathrm{I}$ (1) $\displaystyle\Sigma$
$\displaystyle=\mathrm{Cov}(W)$

where, $I$ $\in\mathbb{R}^{d\times d}$ is an identity matrix, and $Cov(W)$ is
the covariance of $W$. The first initialization $\Sigma=\mathrm{I}$, follows
our assumption of decorrelated dimensions in high-dimensional spaces if the
classes are easily distinguishable. The second initialization
$\Sigma=\mathrm{Cov}(W)$, is empirically observed to handle the ”extreme”
cases where due to a large number of classes and inter-class confusion, the
dimensions of the embeddings would be correlated. Here, we hypothesize that
the correlation in embeddings would be captured implicitly in the correlation
between the weight matrix’s dimensions.

We then simply initialize the means matrix as $\mu=W$. This follows from
SLDA’s Eqn. 19 (detailed in Appendix B.2) and the previous initialization of
$\Sigma=\mathrm{I}$.

Performing the base initialization in this way also helps transfer pre-trained
classifier weights to an SLDA classifier that can be deployed at the edge with
CL-capabilities with no retraining effort as seen in Fig. 1. The assumption of
using information in dense layer for SLDA is further validated by the
equivalence of LDA and fully connected layer shown in Section 3.

###  2.2 Batch Parallel Training

LDA-based classifiers can theoretically be trained in constant time with
respect to the number of classes, as the process only involves updating for
every sample, the corresponding class’s mean vector $\mu_{k}$ and the shared
covariance $\Sigma$. This makes the time complexity of training SLDA $O(1)$
with respect to the number of classes and as a result, highly optimized for
training in extreme multiclass settings. Details of training updates detailed
by Hayes & Kanan (2020) are included in Appendix B.1.

One of the upsides of a fully connected layer is its ability to handle
vectorized batch parallel training efficiently on specialized hardware, which
is much faster than per-sample training strategies. Inspired by chunk updates
described in Pang et al. (2005), we vectorize the training process of Deep
SLDA (Hayes & Kanan, 2020) and incorporate batch-parallelism which updates the
mean and covariance parameters just once for every batch of samples. The
method detailed in Algorithm 1, allows for better utilization of specialized
hardware as well as a significant speedup in training times.

Algorithm 1 Batch Parallel Training in XLDA

m: batch size, n: number of classes d: length of a feature embedding

Input: Batch of data (X, y) where $\textbf{X}\in\mathbb{R}^{m\times d}$,
$\textbf{y}\in\mathbb{R}^{m}$

Initialize means $\boldsymbol{\mu}\in\mathbb{R}^{n\times d}$ and counts
$\textbf{C}\in\mathbb{R}^{n}$ to zero

for every batch of samples (X, y) do

Compute a dictionary of unique labels and their counts in y where $l$ is a
unique label and $s$ is its count in y

U: dict($l:s$) $\leftarrow$ UNIQUE(y)

parallel for label $i$ in U.keys do

$\textbf{C}[i]\leftarrow\textbf{C}[i]+\textsc{U}[i]$

$\boldsymbol{\mu}[i]\leftarrow\boldsymbol{\mu}[i]+\frac{\textbf{X}_{i}-\boldsymbol{\mu}[i]}{\textbf{C}[i]}$

end for

end for

###  2.3 Inference using Nearest Neighbor Search (NNS)

As described in Appendix B.2, while making a prediction in SLDA, $\mu$ and
$\Sigma$ are translated to $W$ and $b$ of an equivalent fully connected layer.
The logits for all N classes are computed as per $y=Wx^{T}+b$ where x denotes
the embedding of input images. Since this then requires the expensive
normalization of logits against all classes, we optimize inference for sub-
linear search using NNS.

Inspired by k-NN Softmax (Song et al., 2020) and ANN-Softmax (Zhao et al.,
2021) , we propose the use of Locality Sensitive Hashing (LSH) for _active
class selection_ in XLDA. We optimize XLDA for inputs $x$ by searching for the
_k-nearest mean vectors_ $\tilde{\mu}$ from all mean vectors $\mu$ present in
the search index. We translate only the means of the active classes to
$\tilde{w}$ and $\tilde{b}$ to obtain the reduced fully connected layer
equivalent. The active classification logits are then computed as
$\tilde{y}=\tilde{w}x^{T}+\tilde{b}$ and the final classification logits as
$y_{i}=\tilde{y}_{i}\forall i\in$ k-active classes, and $0$ for all others.

In essence, this is a two step process of first running a _nearest mean
classifier_ to shortlist the potential classes and then predicting on only
those classes.

Table 1: Accuracies of Base Initialization Model Initialization | CalTech Birds | ImageNet-1k | AliProducts | Landmarks (Clean)
---|---|---|---|---
| (200 classes) | (1000 classes) | ($\sim$50k classes) | ($\sim$80k classes)
Dense Layer (Roof-line) | 72.07% | 83.86% | 81.35% | 72.57%
$\mu=W$ & $\Sigma=I$ |  71.49% (-0.58) |  83.82% (-0.04) | 77.62% (-3.73) | 63.45% (-9.12)
$\mu=W$ & $\Sigma=\mathrm{Cov}(W)$ | 71.13% (-0.94) | 82.26% (-1.6) |  81.77% (+0.42) |  66.71% (-5.86)
Table 2: Training time for 1 epoch. Speed-up factor of XLDA against FC layer shown in parentheses. Dataset | FC Layer | SLDA | XLDA
---|---|---|---
| (bs = 512) | (bs = 1) | (bs = 512)
Caltech-Birds | 0.18s | 7s | 0.36s (0.5$\times$)
ImageNet-1k | 15s | 23m |  11s (1.3$\times$)
iNaturalist | 37s | 9m |  5s (7.4$\times$)
AliProducts | 26m | 1h5m |  36s (42$\times$)

##  3 Equivalence of LDA classifier and Fully Connected layer

A fully-connected layer with weights $W$ and bias $b$, linearly transforms the
input $x$, resulting in a transformed output $Wx^{T}+b$. When activated with a
softmax, this takes the form of the posterior $\hat{y}=p(y|x;W,b)$. Training
classifiers with fully-connected layers, is effectively, learning the
posterior distribution.

LDA classifier, on the other hand, models the likelihood over $x$, assuming
normally-distributed samples $x\sim\mathcal{N}(\mu^{(y)},\Sigma_{s})$, where
$\mu^{y}$ refers to the mean vector belonging to class $y$. Covariance
$\Sigma_{s}$ is shared across classes for multivariate data, which is
imperative for a linear decision boundary. Labels $y\in\mathcal{Y}$ are
assumed to be multinomially distributed
$y\sim\mathrm{Multinomial}(\phi^{(y)})$. In classification, we’re interested
in the posterior $p(y|x)$, we arrive at the posterior for LDA using Bayes’
rule as shown in Eq.2.

$\displaystyle p(y|x)$ $\displaystyle=\frac{p(x|y)p(y)}{p(x)}$ (2)
$\displaystyle p(y=1|x)$
$\displaystyle=\frac{p(x|y=1)p(y=1)}{\Sigma_{j=0}^{1}p(x|y=j)p(y=j)}$ (3)

For simplicity, we take the binary classification case with univariate inputs
and shared variance $\sigma_{s}$. Prior $p(y=1)$ now becomes
$y\sim\mathrm{Bern}(\phi)$, where $\phi$ is the probability of sampling $y=1$
in a single trial.

$\displaystyle\mathcal{N}(\mu,\sigma^{2})$
$\displaystyle=\frac{1}{\sigma\sqrt{2\pi}}e^{-\frac{1}{2}\left(\frac{x-\mu}{\sigma}\right)^{2}}$
(4) $\displaystyle\mathrm{Bern(\phi^{(k)})}$
$\displaystyle=\begin{cases}\phi&\text{if }k=1,\\\ 1-\phi&\text{if
}k=0.\end{cases}$ (5)

Substituting distributions in Eq.2 assuming binary outcomes, the posterior
$p(y|x)$ simplifies to a logistic form, as shown below:

$\displaystyle p(y=1|x)$
$\displaystyle=\frac{1}{1+\frac{p(x|y=0)p(y=0)}{p(x|y=1)p(y=1)}}$ (6)
$\displaystyle=\frac{1}{1+\frac{\phi}{1-\phi}\frac{\exp(\frac{-(x-\mu_{0})^{2}}{2\sigma^{2}})}{\exp(\frac{-(x-\mu_{1})^{2}}{2\sigma^{2}})}}$
(7) $\displaystyle=\frac{1}{1+\alpha
e^{(\frac{-1}{2\sigma^{2}}[\mu_{0}^{2}-\mu_{1}^{2}-2x(\mu_{0}-\mu_{1})])}}$
(8) $\displaystyle\approx\frac{1}{1+\alpha e^{-(wx+b)}}$ (9)

Here $\alpha=\frac{\phi}{1-\phi}$. Assuming equal prior, $\alpha$ is
eliminated, and the resulting expression now resembles a logistic curve with
parameters $w$ and $b$.

Hence, an FC layer with binary outcome essentially learns the posterior
$p(y|x)$, that resembles the logistic function with parameters $W$ and $b$,
which is implicitly a function of $\mu^{(y)}$ and shared variance
$\sigma_{s}$.

For the sake of brevity, we show the equivalent reductions for $W$ and $b$ for
class $k$ in case of multivariate input x below. More details can be found in
Appendix B.

$\displaystyle\hat{\textbf{y}}$
$\displaystyle=\mathrm{W}^{T}\textbf{x}+\mathrm{b}$ (10)
$\displaystyle\Lambda$
$\displaystyle=((1-\beta)\Sigma+\beta\mathrm{I})^{-1},\beta\in(0,1)$ (11)
$\displaystyle\mathrm{w}_{k}$ $\displaystyle=\Lambda\mu_{k},k\in\mathcal{Y}$
(12) $\displaystyle\mathrm{b}_{k}$
$\displaystyle=-\frac{1}{2}(\mu_{k}.\Lambda\mu_{k})$ (13)

Therefore, given parameters $W$ and $b$ from an FC layer, assuming normally-
distributed data and shared identity covariance, one can derive ‘initial’
guesses for $\mu=W$ and $\Sigma_{s}=\mathrm{I}$, to enable CL without
retraining LDA. Experiments on these initializations are discussed in Section
4.1.

Conversely, an LDA layer can be converted to an FC layer for end-to-end
training using $W$ and $b$ initialized based on the above parameters. As we
show in Section 4.1, this provides compute savings up to 42$\times$ for up to
50k classes, at a minimum, since SLDA converges to an optimum starting point
of $W$ and $b$ in a single epoch, where an FC layer typically requires
multiple epochs to converge on a solution. This speedup gets significant, when
accounting for multiple epochs that FC layers require.

##  4 Experiments and Results

Details of the experimental setup and the data used are included in Appendices
C and D.

###  4.1 Conversion from a Fully-Connected Layer

We train a Fully Connected (Dense) layer classifier with the EfficientNetV2-S
(Tan & Le, 2021) backbone and then attempt to transfer the parameters of the
Dense layer to a CL-capable SLDA. For ImageNet-1k we use the pretrained
classifier weights of EfficientNet-V2-S for initialization. In Table 1, we
compare the accuracies of the initialized SLDA model with the Dense layer that
was used to initialize it. Validation sets used have been detailed in Appendix
D.

We notice that a major portion of the accuracy from the Dense layer is
retained in the SLDA classifier and the initialization only results in a small
drop-off in accuracy which can be considered a trade-off for CL capabilities.
Between the two options for initializing $\Sigma$ — $I$ and $\mathrm{Cov}(W)$
— $I$ seems to be the better option for datasets with a smaller number of
classes due to lower inter-class confusion and in turn, lower correlation
between the dimensions of embeddings. $\mathrm{Cov}(W)$ seems to be the better
option for datasets with a large number of fine-grained classes due to higher
inter-class confusion which breaks the assumption of uncorrelated embedding
space.

The results for ImageNet-1k also demonstrate the capability to transfer pre-
trained weights of a Dense classifier head trained without a CL objective or
architecture, to a CL-amenable and edge deployment-ready SLDA classifier
straightaway with almost no drop in accuracy.

We study the improvement by training $\mu$ and $\Sigma$, and the impact of
each component in Appendix A.

###  4.2 Batch Parallelization

On datasets of increasing sample size and class count, the results in Table 2
compare the time taken to train SLDA sample-wise and batch-wise, with the time
to train a fully connected layer with the dataset embeddings as reference. The
FC layer is trained for 1 epoch with cosine decay and a batch size of 512.
However, 1 epoch is not sufficient to train an FC layer, in which case the
speedup for XLDA is even more pronounced (as shown in Table 6). By
implementing vectorized SLDA over batches of 512 samples, training times
significantly decrease to less than a minute, even for the large-scale
AliProducts datasets of around 2.5M samples. XLDA gains exponentially versus
FC as the number of classes grow, due to its $O(1)$ training construction.

###  4.3 Inference optimization

Table 3: Per-sample Inference times and Speed-up factors Dataset | SLDA | XLDA (Ours)
---|---|---
ImageNet-1k | 7 ms |  5 ms (1.4$\times$)
AliProducts | 172 ms |  41 ms (4.2$\times$)
Landmarks | 278 ms |  59 ms (4.7$\times$)

We evaluate the performance of Nearest Neighbor (NN) Search optimization in
XLDA by comparing the time taken to make predictions per-sample (batch size =
1) during inference in Table 3. We report times for inference done purely on
CPU; both the LSH Nearest Neighbor Search and the logit computation. We choose
the number of active classes $k=\frac{C}{10}$ where $C$ is the total number of
classes. We notice that XLDA is faster per-sample, and over the entire test
set we observe upto a $\sim$ 5$\times$ speedup. The speedup is more pronounced
on datasets with a larger number of classes.

##  5 Conclusion and Future Work

In this paper, we’ve discussed XLDA, a framework to extend Class-IL to
classification on a large number of classes, and described a suite of
optimizations to efficiently train and use it for inference at compute
constrained edge devices. Our results for converting an FC layer to an LDA
classifier with simple initializations are promising and in future works, a
conversion that enables end-to-end training for LDA classifiers could be
explored.

## References

  * Aljundi et al. (2019) Aljundi, R., Kelchtermans, K., and Tuytelaars, T.  Task-free continual learning.  In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , pp. 11254–11263, 2019.
  * Bhatia et al. (2016) Bhatia, K., Dahiya, K., Jain, H., Kar, P., Mittal, A., Prabhu, Y., and Varma, M.  The extreme classification repository: Multi-label datasets and code, 2016\.  URL http://manikvarma.org/downloads/XC/XMLRepository.html.
  * Cheng et al. (2020) Cheng, L., Zhou, X., Zhao, L., Li, D., Shang, H., Zheng, Y., Pan, P., and Xu., Y.  Weakly supervised learning with side information for noisy labeled images.  In _The European Conference on Computer Vision (ECCV)_ , August 2020\.
  * Dahiya et al. (2021) Dahiya, K., Saini, D., Mittal, A., Shaw, A., Dave, K., Soni, A., Jain, H., Agarwal, S., and Varma, M.  Deepxml: A deep extreme multi-label learning framework applied to short text documents.  In _Proceedings of the 14th ACM International Conference on Web Search and Data Mining_ , pp. 31–39, 2021.
  * Dahiya et al. (2023) Dahiya, K., Gupta, N., Saini, D., Soni, A., Wang, Y., Dave, K., Jiao, J., Dey, P., Singh, A., Hada, D., et al.  Ngame: Negative mining-aware mini-batching for extreme classification.  In _Proceedings of the Sixteenth ACM International Conference on Web Search and Data Mining_ , pp. 258–266, 2023.
  * Deng et al. (2019) Deng, J., Guo, J., Xue, N., and Zafeiriou, S.  Arcface: Additive angular margin loss for deep face recognition.  In _Proceedings of the IEEE/CVF conference on computer vision and pattern recognition_ , pp. 4690–4699, 2019.
  * Fisher (1936) Fisher, R.  Linear discriminant analysis.  _Ann. Eugenics_ , 7:179, 1936.
  * Hayes & Kanan (2020) Hayes, T. L. and Kanan, C.  Lifelong machine learning with deep streaming linear discriminant analysis.  In _Proceedings of the IEEE/CVF conference on computer vision and pattern recognition workshops_ , pp. 220–221, 2020.
  * Kirkpatrick et al. (2017) Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A. A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.  Overcoming catastrophic forgetting in neural networks.  _Proceedings of the national academy of sciences_ , 114(13):3521–3526, 2017.
  * Li & Hoiem (2017) Li, Z. and Hoiem, D.  Learning without forgetting.  _IEEE transactions on pattern analysis and machine intelligence_ , 40(12):2935–2947, 2017.
  * Lopez-Paz & Ranzato (2017) Lopez-Paz, D. and Ranzato, M.  Gradient episodic memory for continual learning.  _Advances in neural information processing systems_ , 30, 2017.
  * Pang et al. (2005) Pang, S., Ozawa, S., and Kasabov, N.  Incremental linear discriminant analysis for classification of data streams.  _IEEE transactions on Systems, Man, and Cybernetics, part B (Cybernetics)_ , 35(5):905–914, 2005.
  * Rebuffi et al. (2017) Rebuffi, S.-A., Kolesnikov, A., Sperl, G., and Lampert, C. H.  icarl: Incremental classifier and representation learning.  In _Proceedings of the IEEE conference on Computer Vision and Pattern Recognition_ , pp. 2001–2010, 2017.
  * Russakovsky et al. (2015) Russakovsky, O., Deng, J., Su, H., Krause, J., Satheesh, S., Ma, S., Huang, Z., Karpathy, A., Khosla, A., Bernstein, M., Berg, A. C., and Fei-Fei, L.  ImageNet Large Scale Visual Recognition Challenge.  _International Journal of Computer Vision (IJCV)_ , 115(3):211–252, 2015.  doi: 10.1007/s11263-015-0816-y.
  * Schroff et al. (2015) Schroff, F., Kalenichenko, D., and Philbin, J.  Facenet: A unified embedding for face recognition and clustering.  In _Proceedings of the IEEE conference on computer vision and pattern recognition_ , pp. 815–823, 2015.
  * Siblini et al. (2018) Siblini, W., Kuntz, P., and Meyer, F.  Craftml, an efficient clustering-based random forest for extreme multi-label learning.  In _International Conference on Machine Learning_ , pp. 4664–4673. PMLR, 2018.
  * Song et al. (2020) Song, L., Pan, P., Zhao, K., Yang, H., Chen, Y., Zhang, Y., Xu, Y., and Jin, R.  Large-scale training system for 100-million classification at alibaba.  In _Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining_, pp. 2909–2930, 2020.
  * Tan & Le (2021) Tan, M. and Le, Q.  Efficientnetv2: Smaller models and faster training.  In _International conference on machine learning_ , pp. 10096–10106. PMLR, 2021.
  * Van de Ven et al. (2020) Van de Ven, G. M., Siegelmann, H. T., and Tolias, A. S.  Brain-inspired replay for continual learning with artificial neural networks.  _Nature communications_ , 11(1):4069, 2020.
  * Van Horn et al. (2018) Van Horn, G., Mac Aodha, O., Song, Y., Cui, Y., Sun, C., Shepard, A., Adam, H., Perona, P., and Belongie, S.  The inaturalist species classification and detection dataset.  In _Proceedings of the IEEE conference on computer vision and pattern recognition_ , pp. 8769–8778, 2018.
  * Welinder et al. (2010) Welinder, P., Branson, S., Mita, T., Wah, C., Schroff, F., Belongie, S., and Perona, P.  Caltech-UCSD Birds 200.  Technical Report CNS-TR-2010-001, California Institute of Technology, 2010\.
  * Weyand et al. (2020) Weyand, T., Araujo, A., Cao, B., and Sim, J.  Google landmarks dataset v2-a large-scale benchmark for instance-level recognition and retrieval.  In _Proceedings of the IEEE/CVF conference on computer vision and pattern recognition_ , pp. 2575–2584, 2020.
  * Yokoo et al. (2020) Yokoo, S., Ozaki, K., Simo-Serra, E., and Iizuka, S.  Two-stage discriminative re-ranking for large-scale landmark retrieval.  In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops_ , pp. 1012–1013, 2020.
  * Ypsilantis et al. (2021) Ypsilantis, N.-A., Garcia, N., Han, G., Ibrahimi, S., Noord, N. V., and Tolias, G.  The met dataset: Instance-level recognition for artworks.  In _Thirty-fifth Conference on Neural Information Processing Systems Datasets and Benchmarks Track (Round 2)_ , 2021.  URL https://openreview.net/forum?id=fnuAjFL7MXy.
  * Yu et al. (2022) Yu, H.-F., Zhang, J., Chang, W.-C., Jiang, J.-Y., Li, W., and Hsieh, C.-J.  Pecos: Prediction for enormous and correlated output spaces.  In _Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining_ , pp. 4848–4849, 2022.
  * Zenke et al. (2017) Zenke, F., Poole, B., and Ganguli, S.  Continual learning through synaptic intelligence.  In _International conference on machine learning_ , pp. 3987–3995. PMLR, 2017.
  * Zhao et al. (2021) Zhao, K., Song, L., Zhang, Y., Pan, P., Xu, Y., and Jin, R.  Ann softmax: Acceleration of extreme classification training.  _Proceedings of the VLDB Endowment_ , 15(1):1–10, 2021.

##  Appendix A Ablation Study for FC-XLDA conversion

In Table 4, we study the improvement by training the two components of SLDA —
$\mu$ and $\Sigma$ — as opposed to initializing it from $W$, as well as the
relative impact of training each of these components on accuracy. We notice
that training only $\mu$ leads to a more significant boost in performance
compared to training only $\Sigma$. As expected training both $\mu$ and
$\Sigma$ leads to a significant boost in performance, albeit at the cost of
training time and compute.

Table 4: Ablation study for improvement by training each component of SLDA over initializing SLDA SLDA Model Initialization | CalTech Birds | ImageNet-1k | AliProducts
---|---|---|---
$\mu=W$ & $\Sigma=I/\mathrm{Cov}(W)$ (Best Init) | 71.49% | 83.82% | 81.77%
$\mu$ trained & $\Sigma=I$ | 73.20% (+1.71) | 83.64% (-0.18) | 82.66% (+0.89)
$\mu$ trained & $\Sigma=\mathrm{Cov}(W)$ | 73.32% (+1.83) | 81.50% (-2.32) | 84.65% (+2.88)
$\mu=W$ & $\Sigma$ trained | 72.18% (+0.69) | 82.27% (-1.55) | 78.76% (-3.01)
$\mu$ & $\Sigma$ both trained |  74.90% (+3.41) |  83.74% (-0.08) |  86.87% (+5.1)

##  Appendix B Background on SLDA

Streaming Linear Discriminant Analysis (SLDA) was introduced by Hayes & Kanan
(2020) as a linear classifier that makes a prediction based on embeddings from
a fixed CNN-backbone. The SLDA classifier stores a mean vector $\mu_{k}$ per
class, which together make the mean matrix $\mu$, as well as a shared
covariance matrix $\Sigma$. This allows SLDA to learn a per-class
distribution, giving it generative properties that are ideal for Class-IL.

###  B.1 Training

During training, in addition to $\mu_{k}$, an associated count of samples seen
for the class $k$ is stored and is represented by $c_{k}$. On the arrival of a
new sample $(\mathbf{z}_{t},y)$, the mean $\boldsymbol{\mu}_{y}$ parameter and
counter variable $c_{y}$ of the class at a timestep $t$ are updated by

$\displaystyle\boldsymbol{\mu}_{(k=y,t+1)}\leftarrow\frac{c_{(k=y,t)}\boldsymbol{\mu}_{(k=y,t)}+\mathbf{z}_{t}}{c_{(k=y,t)}+1}$
(14) $\displaystyle c_{(k=y,t+1)}=c_{(k=y,t)}+1$

The shared covariance matrix $\Sigma$ is either kept plastic or fixed —
plastic refers to the covariance being updated at every training step after
base initialization and fixed is when it does not change after base-
initialization. In the plastic scenario, $\Sigma$ is updated as per

$\displaystyle\Sigma_{t+1}=\frac{t\Sigma_{t}+\Delta_{t}}{t+1}$ (15)

$\displaystyle\Delta_{t}=\frac{t(z_{t}-\mu_{(k=y,t)})(z_{t}-\mu_{(k=y,t)})^{T}}{t+1}$
(16)

###  B.2 Inference

To use SLDA as a classifier and make predictions, $\mu$ and $\Sigma$ are
converted to an equivalent fully-connected layer $W$ and $b$ as per

$w_{k}=\Lambda\mu_{k}$ (17)

$b_{k}=-\frac{1}{2}(\mu_{k}\cdot\Lambda\mu_{k})$ (18)

where $\Lambda\sim\Sigma^{-1}$ with a shrinkage estimator applied. This
implies,

$\displaystyle W=\Sigma^{-1}\mu$ (19) $\displaystyle\mu=\Sigma W$

Table 5: Dataset Statistics of number of classes and training samples Dataset | # classes | # samples
---|---|---
CalTech Birds (2011) (Welinder et al., 2010) | 200 | 6k
ImageNet-1K (2012) (Russakovsky et al., 2015) | 1K | 1.3m
i-Naturalist (Van Horn et al., 2018) | 8.1K | 0.5m
AliProducts v1 (Cheng et al., 2020) | 50K | 2.5m
Google Landmarks v2 - Clean (Weyand et al., 2020; Yokoo et al., 2020) | 81K | 1.4m

##  Appendix C Experimental Setup

Our experiments were implemented in TensorFlow and run on a Intel Core
i9-10920X CPU with a NVIDIA RTX A6000 GPU.

##  Appendix D Data and Validation Sets

We use the datasets shows in Table 5 for our experiments. Google Landmarks v2
(Weyand et al., 2020) has 200k classes originally, but we use the Clean
version of the dataset created by Yokoo et al. (2020), as it is more amenable
to classification tasks, compared to the original which is better suited for
retrieval. The clean version consists of classes with at least 2 samples,
reducing the class count to 81k.

The validation set used for reporting accuracy were created as described
below. For CalTech Birds, ImageNet-1k and i-Naturalist we use the train test
split created as part of the Tensorflow-Hub dataset. For AliProducts we use
the train and validation splits created for the AliProducts Product
Recognition Challenge 2020, as is. For Google Landmarks (Clean), we create a
1-sample per class validation set, i.e our validation set consists of 81k
images which correspond to the 81k classes in the dataset.

Table 6: Practical training times for FC layer (as opposed to ideal 1 epoch). Speedup with XLDA training for 1 epoch Dataset | Approx. # epochs | FC Layer | Practical Speedup
---|---|---|---
| for FC to converge | (bs = 512) | of XLDA
Caltech-Birds | 25 | 4.5s | 12.5$\times$
iNaturalist | 20 | 740s | 148$\times$
AliProducts | 15 | 390m | 650$\times$