# Crossing the dark matter soliton core: a possible reversed orbital
precession

Man Ho Chan†, Chak Man Lee  Department of Science and Environmental Studies,
The Education University of Hong Kong, Tai Po, New Territories, Hong Kong,
China
Correspondence Email<EMAIL_ADDRESS>

###### Abstract

The ultra-light dark matter (ULDM) model has become a popular dark matter
scenario nowadays. The mass of the ULDM particles is extremely small so that
they can exhibit wave properties in the central dark matter halo region.
Numerical simulations show that a soliton core with an almost constant mass
density would be formed inside the ULDM halo. If our Galactic Centre has a
dark matter soliton core, some of the stars orbiting about the supermassive
black hole (Sgr A*) would be crossing the soliton core boundary. In this
article, we report the first theoretical study on how the dark matter soliton
core near the Sgr A* could affect the surrounding stellar orbital precession.
We show that some particular stellar orbital precession may become retrograde
in direction, which is opposite to the prograde direction predicted by General
Relativity. We anticipate that future orbital data of the stars S2, S12 and
S4716 can provide crucial tests for the ULDM model for $m\sim
10^{-19}-10^{-17}$ eV.

##  I Introduction

In the past few decades, many direct-detection (e.g. XENON-1T) Amole ; Aprile
; Aalbers  and indirect-detection (e.g. radio and gamma-ray detection)
Ackermann ; Chan3 ; Chan4  experiments were deployed to search for the signal
of weakly-interacting massive particle (WIMP) dark matter. However, no
promising signal has been obtained so far Bertone2 . Therefore, some suggest
another extreme that dark matter may be made of bosons with very small mass
(i.e. the ultra-light dark matter, ULDM). This proposal can simultaneously
solve some long-standing problems in particle physics Bertone ; Marsh . Since
the mass of the ULDM is very small, they will exhibit wave nature at small
scales. Interestingly, the wave behaviour of dark matter would undergo
interference and it would finally form a soliton core at the central region of
the ULDM halo. Numerical simulations show that the soliton core formed by the
ULDM would be quite universal, which depends on the mass of the ULDM particle
$m$ and the halo mass $M_{h}$ Schive ; Safarzadeh .

The ULDM model has become more popular nowadays because it can form dark
matter cores observed in many dwarf galaxies (i.e. solving the core-cusp
problem) Hu . The ULDM can behave like cold dark matter on large scales while
it can demonstrate core-like structures at small scales Hu . Various studies
have put constraints on the mass of the ULDM, such as using the data of
galaxies Safarzadeh ; Marsh2 ; Desjacques ; Maleki ; Li ; Chan ,
Lyman-$\alpha$ forest Rogers , halo mass function Schutz , supermassive black
hole Stott ; Monica ; Saha , and gravitational waves Hannuksela .
Nevertheless, many of the constraints are contradictory to each other and no
robust conclusion has been made so far. Overall speaking, recent studies seem
to favour $m\geq 10^{-20}$ eV Rogers ; Hui  and some even suggest $m\geq
10^{-18}$ eV Desjacques .

In this article, we report the first theoretical study on how the ULDM dark
matter soliton core could affect the surrounding stellar orbital precession,
which have not been discussed previously. We surprisingly discover that some
stellar orbits at the deep Galactic Centre might possibly be reversal in the
orbital precession if a dark matter soliton exists. This phenomenon depends on
the position of the stellar orbits and the actual value of $m$. By using the
information of the stellar cluster at the deep Galactic Centre (the S-Star
cluster), we can test a wide range of ULDM mass ($m\sim 10^{-19}-10^{-17}$
eV), which is currently one of the popular ranges in theories (e.g. string
axions) Marsh ; Hui . Future data from GRAVITY collaboration and the orbital
data of the newly discovered stars (e.g. S4716 star) near the Sgr A* can
provide crucial tests to verify or falsify the ULDM soliton model. This can
greatly enhance our understanding of the nature of dark matter.

##  II Equation of motion for ULDM soliton core model

For a Schwarzschild black hole, the spherical symmetric space-time metric is

$ds^{2}=A(r)c^{2}dt^{2}-B(r)dr^{2}-r^{2}(d\theta^{2}+\sin^{2}\theta
d\phi^{2}),$ (1)

where $(r,\theta,\phi)$ are the spherical coordinates, $A(r)=1-r_{s}/r$ with
$r_{s}=2GM_{\rm BH}/c^{2}$, and $B(r)=1/A(r)$. The equation of motion of a
star orbiting about a supermassive black hole on the fixed plane at
$\theta=\pi/2$ is given by:

$\frac{d^{2}u}{d\phi^{2}}+u=\frac{GM_{\rm BH}}{L^{2}}+3\frac{GM_{\rm
BH}}{c^{2}}u^{2},$ (2)

where $u=1/r$ and $L$ is the angular momentum. The solution of $u(\phi)$ is
approximately given by

$u(\phi)\approx\frac{GM_{\rm BH}}{L^{2}}[1+e\cos(\phi-\epsilon\phi)],$ (3)

where $e$ and $\epsilon$ are constant. Therefore, the precession angle of the
stellar orbit per period due to the General Relativistic effect is

$\Delta\phi_{\rm GR}=2\pi\epsilon\approx 6\pi\frac{G^{2}M_{\rm
BH}^{2}}{c^{2}L^{2}}\approx 6\pi\frac{GM_{\rm BH}}{c^{2}a(1-e^{2})}.$ (4)

If the ULDM soliton core exists, the ULDM core mass would be much smaller than
the supermassive black hole mass. Following the method of perturbation, we can
re-write the equation of motion by adding a ULDM component as

$\frac{d^{2}u}{d\phi^{2}}+u\approx\frac{GM(u)}{L^{2}}+3\frac{GM_{\rm
BH}}{c^{2}}u^{2},$ (5)

where $M(u)=M_{\rm BH}+M_{\rm DM}(u)$ and $M_{\rm DM}(r)$ is the enclosed mass
of the ULDM halo. Note that the general relativistic term $3GM_{\rm
BH}u^{2}/c^{2}$ can be regarded as a first order correction term of the
Newtonian equation of motion. Since we have $M_{\rm DM}(u)\ll M_{\rm BH}$,
adding a small term representing the effect of the ULDM component in the
general relativistic term would further generate a second order correction
term for the equation of motion, which is negligible. Therefore, there is no
change in the general relativistic term in Eq. (5) compared with that in Eq.
(2).

In our present numerical simulations, the central ULDM density profile is
given by Schive ; Safarzadeh

$\rho_{\rm DM}(r)=\frac{\rho_{0}}{(1+Ar^{2})^{8}},$ (6)

where $A=9.1\times 10^{-2}r_{c}^{-2}$,

$\rho_{0}=1.9\left(\frac{m}{10^{-23}~{}{\rm
eV}}\right)^{-2}\left(\frac{r_{c}}{\rm kpc}\right)^{-4}~{}M_{\odot}~{}{\rm
pc^{-3}},$ (7)

and $r_{c}$ is the soliton core radius, which is given by

$r_{c}=16\left(\frac{m}{10^{-23}~{}\rm
eV}\right)^{-1}\left(\frac{M_{h}}{10^{9}M_{\odot}}\right)^{-1/3}~{}{\rm kpc},$
(8)

with $M_{h}$ being the total halo mass. Therefore, the enclosed mass of the
ULDM halo is

$M_{\rm DM}(r)=\int_{0}^{r}4\pi r^{\prime 2}\rho_{\rm
DM}(r^{\prime})dr^{\prime}.$ (9)

The final stellar orbit $r(\phi)=1/u(\phi)$ could be obtained by solving Eq.
(5) numerically. Here, the central ULDM density profile is indeed embedded in
a galactic regular dark matter density profile (e.g. the Navarro-Frenk-White
density profile). For $r\gg r_{c}$, the ULDM density would follow the galactic
regular dark matter density profile Schive .

##  III Results

We first use the S2 star as an example to demonstrate the effect on the
orbital precession angle due to the ULDM model. The motion of the S2 star has
been monitored for almost three decades Eckart ; Ghez98 ; Schodel ; Eisenhauer
; Ghez03 ; Ghez08 ; Heibel . The S2 star has finished at least one period
(orbital period $\approx 16$ years) since our first observation on it.
Therefore, we have already obtained rich information about the S2 star and the
constraints on the mass of the supermassive black hole.

If a ULDM soliton core exists in our Galactic Centre, numerical simulations
can predict the size of the core radius $r_{c}$ and the central core density
$\rho_{c}$ Schive ; Safarzadeh . The core radius and the central core density
depend on both $m$ and the total halo mass $M_{h}$ Schive ; Safarzadeh .
Recent observations of the Gaia satellite have constrained the total halo mass
to be $M_{h}=1.08^{+0.20}_{-0.14}\times 10^{12}M_{\odot}$ Cautun . This is
consistent with the results obtained from other studies, such as
$M_{h}=1.17^{+0.21}_{-0.15}\times 10^{12}M_{\odot}$ in Callingham  and
$M_{h}=(1.16\pm 0.24)\times 10^{12}M_{\odot}$ in Deason . In the following
analysis, we will adopt a wider possible range of the halo mass
$M_{h}=(1.0-1.4)\times 10^{12}M_{\odot}$. Therefore, the central core density
can be written in terms of $m$ only.

Recent observations have precisely determined the orbital parameters of the S2
star, including the semimajor axis $a$, eccentricity $e$, and the mass of the
supermassive black hole $M_{BH}=(4.154\pm 0.014)\times 10^{6}M_{\odot}$ Abuter
. By adding the ULDM component to the equation of motion, we can theoretically
simulate the subsequent orbit of the S2 star for different $m$. We plot the
predicted precession angle for the S2 star against $m$ in Fig. 1. We can see
that for $m\sim 10^{-18}-10^{-17}$ eV, the effect due to the ULDM is large
enough to change the orbital precession from prograde (positive precession
angle) to retrograde (negative precession angle). However, recent observations
have constrained the precession angle to be $\Delta\phi=(0.22\pm
0.04)^{\circ}$ per period Abuter , which is in excellent agreement with the
prediction by General Relativity (GR) $\Delta\phi_{\rm GR}=0.2^{\circ}$. Based
on this constraint, we can rule out the range $5.4\times 10^{-19}$ eV $\leq
m\leq 4.8\times 10^{-17}$ eV for $M_{h}=(1.0-1.4)\times 10^{12}M_{\odot}$. The
effect of the uncertainty in $M_{h}$ has a very little effect on the ruled out
range only.

Generally speaking, the reversal in precession occurs when the orbital
semimajor axis is close to the ULDM core radius. The motion of the star would
be affected significantly when it is crossing the ‘boundary’ of the ULDM core.
It is because the ULDM density changes abruptly near the soliton core region.
If $m$ is too small, the orbit is entirely inside the soliton core and the
core density may be too small to affect the orbital precession. If this is the
case, the precession angle would approach the GR prediction. If $m$ is too
large, the core radius is too small so that the whole orbit is outside the
core region. As the ULDM density drops significantly outside the core radius,
the overall effect on precession is similar to adding a small point mass
inside the orbit. In other words, different positions and sizes of the stellar
orbits might generate different changes or behaviours in the precession angle.

Although the S2 star is the only one in the S-Star cluster which has been
confirmed in the orbital angle precession, we expect that some other stars
would soon complete one orbital period so that we can determine the precession
angle more precisely. In the S-Star cluster, we have chosen 2 more
representative stars (S12 and S4716) for analyses. The S12 star has a large
semimajor axis and a high orbital eccentricity. Although it has a relatively
long period ($P\approx 59$ years), it has been monitored for almost 3 decades
already Gillessen . We need not wait for a very long time to determine the
precession angle. Also, it has an appropriate orbital inclination angle so
that less uncertainty would be resulted in orbital determination. The S4716
star is a recently discovered star which has only a very short orbital period
($P\approx 4$ year) Peibker . It has a very small semimajor axis so that it
can help constrain the large regime of $m$ in the ULDM model. The orbital
parameters of these 3 stars are shown in Table 1.

In Fig. 1, we plot the precession angle against $m$ for the 2 stars for
comparison. We can see that a large reverse in the precession angle is
resulted for the S12 star. The maximum retrograde precession is
$0.9^{\circ}-1.1^{\circ}$ per period (negative sign represents the retrograde
precession) when $m\approx 2\times 10^{-18}$ eV. The retrograde precession
would occur if $2.8\times 10^{-19}$ eV $<m<1.8\times 10^{-17}$ eV. The
precession angle predicted by GR is $0.088^{\circ}$ per period (prograde). For
the S4716 star, there is no retrograde precession, except $M_{h}$ being close
to $1.4\times 10^{12}M_{\odot}$. The dark matter soliton core can
significantly suppress the precession angle from the GR predicted value
$\Delta\phi_{\rm GR}=0.26^{\circ}$ to nearly no precession ($\Delta\phi\approx
0^{\circ}$) when $m\approx 1.1\times 10^{-17}$ eV. We expect that the
precession angle of the S4716 star can be precisely determined after 4 years.
Therefore, we can critically examine the ULDM model for $m\sim 10^{-17}$ eV in
the near future. In Fig. 2, we show the predicted orbital precession of the
S2, S12, and S4716 stars after 20 periods for illustrations and the core radii
of the ULDM halo for $m=10^{-18}$ eV ($\approx 0.4$ arcsec) and $m=10^{-17}$
eV ($\approx 0.04$ arcsec) for comparison.

##  IV Discussion

We have simulated the predicted effects on the stellar orbital precession
angle due to the existence of the dark matter soliton core. These effects have
not been discussed or investigated previously. For a certain range of ULDM
mass $m$, some particular stellar orbits might undergo a reversed orbital
precession. Therefore, if a dark matter soliton core exists and $m$ is within
the range $10^{-19}-10^{-17}$ eV, it is possible to see the retrograde
precession of some stars near the supermassive black hole. Even if we cannot
observe any retrograde precession, we can constrain the value of $m$ by using
the observed precession angles of different stars, such as the constraints of
$m$ obtained in our study using the S2 data. Using the recent observational
data of the S2 star Abuter , we can rule out $5.4\times 10^{-19}$ eV $\leq
m\leq 4.8\times 10^{-17}$ eV. Therefore, it is a good method to test the ULDM
model if we have the information of the orbital precession for different stars
inside the S-Star cluster.

Generally speaking, using the data of the stars with larger semimajor axis can
give better constraints for the smaller regime of $m$. The effect of the
potential retrograde precession would also be more obvious. However, the
orbital periods for those stars would be very large ($>60$ years) so that we
need to wait for a very long time to see the effect of orbital precession.
Moreover, the effects due to the neutron stars and white dwarf mass
distributions on those stellar orbital precession would also be larger. This
would enlarge the systematic uncertainties in the analysis.

In fact, some studies have already use this method to examine the alleged
extended mass distribution Heibel ; Rubilar ; Nucita  and dark matter
distribution Zakharov ; Dokuchaev ; Dokuchaev2 ; Arguelles ; Jovanovic ; Chan2
around the Sgr A*. Different mass distributions would somewhat decrease the
precession angle by a small amount only Jovanovic ; Chan2 , except for some
specific dark matter models Arguelles . The overall effect on the precession
depends on many factors, such as the dark matter mass, central density, scale
radius, and the functional form of the distribution. However, the effect on
the stellar orbital precession due to the dark matter soliton core is quite
different. The sharp decrease on the density profile outside the soliton core
can produce a featured variation in the precession angle. For the wave ULDM
scenario, if the total halo mass $M_{h}$ is known, the effect on precession
depends on a single parameter, the ULDM mass $m$, only. For instance, if
$m=4\times 10^{-19}$ eV, we may be able to observe the star S12 undergoing
retrograde precession while the S2 and S4716 stars undergoing prograde
precession. In the coming decade, some stars will finish their first periods
since our first observations on them (e.g. S38, S55, S62, S4711, S4714 and
S4716) Gillessen ; Becerra ; Peibker ; Peibker2 ; Peibker3 . High-quality
observations of these stars can help provide rich information on their orbital
properties and determine the precession angles. These data can critically
examine the prediction of General Relativity and verify the ULDM model.
Besides, the precession data can also help verify or falsify the other dark
matter models (e.g. WIMP model) or constrain the extended mass distribution
near the supermassive black hole.

##  V Acknowledgments

We thank the anonymous referee for useful constructive feedback and comments.
The work described in this paper was partially supported by the Seed Funding
Grant (RG 68/2020-2021R) and the Dean’s Research Fund (activity code: 04628)
from The Education University of Hong Kong.

Table 1: Parameters of the stellar orbits Gillessen ; Peibker  Star | Semimajor axis $a$ (AU) | Eccentricity $e$ |  $\Delta\phi_{\rm GR}$ (deg)
---|---|---|---
S2 | 1002 | 0.886 | 0.21
S12 | 2390 | 0.888 | 0.088
S4716 | 398 | 0.756 | 0.26
Figure 1: The coloured solid lines, dotted lines, and dashed lines represent
the precession angles $\Delta\phi$ against ULDM mass $m$ for $M_{h}=1.08\times
10^{12}M_{\odot}$, $M_{h}=1.0\times 10^{12}M_{\odot}$ and $M_{h}=1.4\times
10^{12}M_{\odot}$ respectively for different stars (Orange: S2; Blue: S12;
Red: S4716). The region bounded by the black dash-dotted lines indicates the
allowed $1\sigma$ range observed for the S2 star Abuter . Figure 2: The
orange, blue and red solid lines represent the current predicted orbits of the
S2, S12 and S4716 stars respectively. The corresponding dotted lines, dot-
dashed lines and dashed lines indicate the precessed orbits after 20 periods
for $m=10^{-17}$ eV, $m=10^{-18}$ eV and $m=10^{-19}$ eV respectively. The
green solid lines represent the dark matter soliton core radii for
$m=10^{-17}$ eV and $m=10^{-18}$ eV. The observed data with error bars for the
S2 Do , S12 Gillessen  and S4716 Peibker  are shown for reference. The
position of the supermassive black hole (Sgr A*) is indicated by the black
dot. Here, $M_{h}=1.08\times 10^{12}M_{\odot}$ is assumed.

## References

  * (1) C. Amole et al., Phys. Rev. D 93, 061101 (2016).
  * (2) E. Aprile et al., Phys. Rev. Lett. 121, 111302 (2018).
  * (3) J. Aalbers et al., First dark matter search results from the LUX-ZEPLIN (LZ) experiment, arXiv:2207.03764.
  * (4) M. Ackermann et al., Phys. Rev. Lett. 115, 231301 (2015).
  * (5) M. H. Chan, L. Cui, J. Liu, and C. S. Leung, Astrophys. J. 872, 177 (2019).
  * (6) M. H. Chan and C. M. Lee, Phys. Lett. B 825, 136887 (2022).
  * (7) G. Bertone and T. M. P. Tait, Nature 562, 51 (2018).
  * (8) G. Bertone, D. Hooper, and Silk J., Phys. Rep. 405, 279 (2005).
  * (9) D. J. E. Marsh, Phys. Rep. 643, 1 (2016).
  * (10) H.-Y. Schive, M.-H. Liao, T.-P. Woo, S.-K. Wong, T. Chiueh, T. Broadhurst, and W.-Y. P. Hwang, Phys. Rev. Lett. 113, 261302 (2014).
  * (11) M. Safarzadeh and D. N. Spergel, Astrophys. J. 893, 21 (2020).
  * (12) W. Hu, R. Barkana, and A. Gruzinov, Phys. Rev. Lett. 85, 1158 (2000).
  * (13) D. J. E. Marsh and J. C. Niemeyer, Phys. Rev. Lett. 123, 051103 (2019).
  * (14) V. Desjacques and A. Nusser, Mon. Not. R. Astron. Soc. 488, 4497 (2019).
  * (15) A. Maleki, S. Baghram, and S. Rahvar, Phys. Rev. D 101, 103504 (2020).
  * (16) Z. Li, J. Shen, and H.-Y. Schive, Astrophys. J. 889, 88 (2020).
  * (17) M. H. Chan and C. F. Yeung, Astrophys. J. 913, 25 (2021).
  * (18) K. K. Rogers and H. V. Peiris, Phys. Rev. Lett. 126, 071302 (2021).
  * (19) K. Schutz, Phys. Rev. D 101, 123026 (2020).
  * (20) M. J. Stott and D. J. E. Marsh, Phys. Rev. D 98, 083006 (2018).
  * (21) R. D. Monica and I. de Martino, Shutting the allowed mass range of the ultralight bosons with S2 star, arXiv:2206.03980.
  * (22) A. K. Saha, P. Parashari, T. N. Maity, A. Dubey, S. Bouri and R. Laha, Bounds on ultralight bosons from the Event Horizon Telescope observation of Sgr A*, arXiv:2208.03530.
  * (23) O. A. Hannuksela, K. W. K. Wong, R. Brito, E. Berti, and T. G. F. Li, Nature Astronomy 3, 447 (2019).
  * (24) L. Hui, Ann. Rev. Astron. Astrophys. 59, 247 (2021).
  * (25) A. Eckart and R. Genzel, Nature 383, 415 (1996).
  * (26) A. M. Ghez, B. L. Klein, M. Morris, and E. E. Becklin, Astrophys. J. 509, 678 (1998).
  * (27) T. Schödel et al., Nature 419, 694 (2002).
  * (28) F. Eisenhauer, R. Sch$\ddot{\rm o}$del, R. Genzel, T. Ott, M. Tecza, R. Abuter, A. Eckart, and T. Alexander, Astrophys. J. 597, L121 (2003).
  * (29) A. M. Ghez, et al., Astrophys. J. 586, L127 (2003).
  * (30) A. M. Ghez et al., Astrophys. J. 689, 1044 (2008).
  * (31) G. Hei$\beta$el, T. Paumard, G. Perrin, and F. Vincent, Astron. Astrophys. 660, A13 (2022).
  * (32) M. Cautun et al., Mon. Not. R. Astron. Soc. 494, 4291 (2020).
  * (33) T. M. Callingham et al., Mon. Not. R. Astron. Soc. 484, 5453 (2019).
  * (34) A. J. Deason et al., Mon. Not. R. Astron. Soc. 501, 5964 (2021).
  * (35) R. Abuter et al. [GRAVITY Collaboration], Astron. Astrophys. 636, L5 (2020).
  * (36) S. Gillessen et al., Astrophys. J. 837, 30 (2017).
  * (37) F. Peißker, A. Eckart, M. Zajac̃ek, and S. Britzen, Astrophys. J. 933, 49 (2022).
  * (38) T. Do et al., Science 365, 664 (2019).
  * (39) G. F. Rubilar and A. Eckart, Astron. Astrophys. 374, 95 (2001).
  * (40) A. A. Nucita, F. De Paolis, G. Ingrosso, A. Qadir, and A. F. Zakharov, Publ. Astron. Soc. Pac. 119, 349 (2007).
  * (41) A. F. Zakharov, A. A. Nucita, F. de Paolis, and G. Ingrosso, Phys. Rev. D 76, 062001 (2007).
  * (42) V. I. Dokuchaev and Y. N. Eroshenko, Physics-Uspekhi 58, 772 (2015).
  * (43) V. I. Dokuchaev and Y. N. Eroshenko, JETP Letters 101, 777 (2015).
  * (44) C. R. Arg$\ddot{\rm u}$elles, M. F. Mestre, E. A. Becerra-Vergara, V. Crespi, A. Krut, J. A. Rueda, and R. Ruffini, Mon. Not. R. Astron. Soc. 511, L35 (2022).
  * (45) P. Borka, D. Jovanović, V. Borka Jovanović, and A. F. Zakharov, Eur. Phys. J. D 75, 145 (2021).
  * (46) M. H. Chan, C. M. Lee, and C. W. Yu, Sci. Rep. 12, 15258 (2022).
  * (47) E. A. Becerra-Vergara, C. R. Arguelles, A. Krut, J. A. Rueda, and R. Ruffini, Mon. Not. R. Astron. Soc. 505, L64 (2021).
  * (48) F. Peißker, A. Eckart, and M. Parsa, Astrophys. J. 889, 61, (2020).
  * (49) F. Peißker, A. Eckart, M. Ali B. Zajac̃ek, and M. Parsa, Astrophys. J. 899, 50 (2020).


# SSAMBA: Self-Supervised Audio Representation Learning with Mamba State Space
Model

###### Abstract

Transformers have revolutionized deep learning across various tasks, including
audio representation learning, due to their powerful modeling capabilities.
However, they often suffer from quadratic complexity in both GPU memory usage
and computational inference time, affecting their efficiency. Recently, state
space models (SSMs) like Mamba have emerged as a promising alternative,
offering a more efficient approach by avoiding these complexities. Given these
advantages, we explore the potential of SSM-based models in audio tasks. In
this paper, we introduce Self-Supervised Audio Mamba (SSAMBA), the first self-
supervised, attention-free, and SSM-based model for audio representation
learning. SSAMBA leverages the bidirectional Mamba to capture complex audio
patterns effectively. We incorporate a self-supervised pretraining framework
that optimizes both discriminative and generative objectives, enabling the
model to learn robust audio representations from large-scale, unlabeled
datasets. We evaluated SSAMBA on various tasks such as audio classification,
keyword spotting, and speaker identification. Our results demonstrate that
SSAMBA outperforms the Self-Supervised Audio Spectrogram Transformer (SSAST)
in most tasks. Notably, SSAMBA is approximately 92.7% faster in batch
inference speed and 95.4% more memory-efficient than SSAST for the tiny model
size with an input token size of 22k. These efficiency gains, combined with
superior performance, underscore the effectiveness of SSAMBA’s architectural
innovation, making it a compelling choice for a wide range of audio processing
applications. Code at https://github.com/SiavashShams/ssamba.

Index Terms—  Audio classification, audio representation learning, state space
models, self-supervised learning, deep learning

##  1 Introduction

Learning robust audio representations is critical for various tasks, including
audio classification, speaker recognition, and emotion recognition [1, 2].
Capturing both short-range and long-range dependencies is necessary for
effective audio representation. While convolutional neural network models have
shown limitations in capturing global dependencies, transformer models have
excelled in image and language tasks due to their self-attention mechanisms
[3, 4]. Building on these advancements, the Audio Spectrogram Transformer
(AST) [5] applied the self-attention mechanism to audio classification,
achieving state-of-the-art performance in various audio classification
benchmarks. AST training requires a large number of labeled audio clips, which
can be difficult to find. To mitigate this, the Self-Supervised Audio
Spectrogram Transformer (SSAST) [2] was introduced, employing an unsupervised
pretraining framework. SSAST utilizes masked spectrogram patch modeling (MSPM)
to pretrain the model on unlabeled audio data, significantly reducing the
reliance on labeled data while maintaining competitive performance to AST.

Fig. 1: A top-down view of Self-Supervised Audio Mamba

Despite the high performance of SSAST, it still suffers from quadratic
computation and memory usage due to its transformer architecture. As a more
efficient alternative to transformers, state space models (SSMs) [6, 7, 8]
have been explored in recent research. SSMs run with subquadratic complexity
but maintain strong sequence modeling ability as transformers and can be
trained in parallel. Notably, the newly proposed SSM model Mamba [9]
incorporates input-selective parameters within SSMs, improving sequence
modeling ability while still enjoying linear complexity relative to sequence
length. Its foundation paper demonstrates Mamba’s high performance and
efficiency in text, audio, and genomics modeling tasks. Subsequent studies
have applied Mamba across a broader range of modalities and tasks. These
models validate the Mamba’s versatility and effectiveness in areas such as
vision [10, 11], biomedical imaging [12, 13], video [14], and graphs [15].
Additionally, Mamba has been utilized in speech and audio applications. For
instance, [16] employs Mamba for long-term multichannel speech enhancement.
[17] combines a hybrid transformer and Mamba model for acoustic and bone
conduction speech enhancement. [18] investigates speech separation using
Mamba. These studies demonstrate that Mamba models can achieve performance
comparable to transformer models.

Given the performance and efficiency demonstrated by these applications, Mamba
has the potential to learn a general audio representation for multiple
downstream tasks. Building on this potential, we propose Self-Supervised Audio
Mamba (SSAMBA). In our approach, audio spectrograms are first split into
patches and then transformed into an embedding sequence. These patches are
subsequently fed into a bidirectional Mamba encoder, which captures the global
audio context using selective state spaces. SSAMBA is trained with a self-
supervised objective on masked spectrogram patches from a large unlabeled
dataset. Once pretraining is complete, SSAMBA can be fine-tuned for specific
downstream tasks using a small labeled dataset. Our experiments demonstrate
that SSAMBA achieves superior or comparable performance to SSAST while
significantly reducing inference costs.

The main contributions of this study are as follows:

  * •

We propose SSAMBA, the first self-supervised, attention-free, and SSM-based
audio representation learning model. SSAMBA incorporates the bidirectional
Mamba to encode and process audio, and it is pretrained without any labeled
data.

  * •

We implement and train SSAMBA in three sizes: Tiny, Small, and Base. All of
them achieve similar or higher performance than the transformer model SSAST in
downstream tasks: audio event classification, keyword spotting, and speaker
identification.

  * •

We show the subquadratic-time computation and memory complexity of SSAMBA,
which makes it a more efficient alternative to SSAST. For example, SSAMBA Tiny
is approximately 92.7% faster in inference speed and 95.4% more memory-
efficient than SSAST of the same size for the input length of 22k patches.

##  2 Self-Supervised Audio Mamba

In this section, we explore the mathematical foundations of the Mamba model,
focusing on its state space model (SSM) framework and efficiency in capturing
long-range dependencies. We describe the architecture of the Self-Supervised
Audio Mamba (SSAMBA) model, which integrates bidirectional SSMs for robust
audio context modeling. Finally, we explain the self-supervised learning
framework adapted from SSAST [19], utilizing masked spectrogram patch modeling
(MSPM) to reduce reliance on labeled data [19].

###  2.1 Mathematical Foundations of the Mamba Model

State space models (SSMs) are a powerful framework for sequence modeling,
drawing inspiration from continuous systems that map a one-dimensional
function or sequence $x(t)\in\mathbb{R}$ to an output $y(t)\in\mathbb{R}$
through a hidden state $h(t)\in\mathbb{R}^{N}$. This is achieved using
evolution parameters $A\in\mathbb{R}^{N\times N}$ and projection parameters
$B\in\mathbb{R}^{N\times 1}$ and $C\in\mathbb{R}^{1\times N}$.

The continuous-time state space model is defined by the following differential
equations:

$h^{\prime}(t)=Ah(t)+Bx(t),$ $y(t)=Ch(t).$

To implement these models in digital systems, we need to discretize them. The
discrete version of the SSM includes a timescale parameter $\Delta$, which
transforms the continuous parameters $A$ and $B$ to their discrete
counterparts. The zero-order hold (ZOH) method is commonly used for this
transformation, defined as follows:

$A_{d}=\exp(\Delta A),$ $B_{d}=(\Delta A)^{-1}(\exp(\Delta A)-I)\cdot\Delta
B.$

After discretization, the state space model for a discrete-time signal with
step size $\Delta$ can be expressed as:

$h_{t}=A_{d}h_{t-1}+B_{d}x_{t},$ $y_{t}=Ch_{t}.$

To compute the output sequence $y_{t}$ efficiently, we use a global
convolution operation. The output $y$ is obtained by convolving the input
sequence $x$ with a structured convolutional kernel $K$, which is precomputed
from the matrices $A$, $B$, and $C$:

$K=(CB,CAB,\ldots,CA^{M-1}B),$ $y=x\ast K,$

where $M$ is the length of the input sequence $x$, and $K\in\mathbb{R}^{M}$ is
the structured convolutional kernel. The Mamba model enhances this framework
by incorporating dynamic updates to the parameters $\Delta_{t}$, $A_{t}$,
$B_{t}$, and $C_{t}$ based on the input $x_{t}$ at each timestep $t$. This
makes the model input-selective and content-aware, allowing it to adjust to
the specific characteristics of the input sequence dynamically. To efficiently
handle these dynamic updates, Mamba employs a selective scan algorithm that
recalculates the convolution dynamically, ensuring efficient and accurate
sequence modeling.

###  2.2 SSAMBA Architecture

Fig. 1 provides a comprehensive overview of the SSAMBA model, illustrating the
following key components:

####  2.2.1 Spectrogram Input Representation

The input audio waveform is first converted into a spectrogram, which
represents the time-frequency domain of the audio data. This transformation is
performed using a short-time Fourier transform (STFT), resulting in a
spectrogram matrix $S$ with dimensions $F\times T$, where $F$ is the number of
frequency bins and $T$ is the number of time frames.

####  2.2.2 Flatten and Linear Projection

Each spectrogram patch $S_{i}$ is flattened into a 1D vector and projected
into a higher-dimensional space using a linear projection layer. This results
in embeddings $E_{i}$, which have dimensions $D$.

####  2.2.3 Positional Encoding

To capture the temporal order and spatial structure of the spectrogram
patches, a learnable positional encoding $P_{i}$ of the same dimension $D$ is
added to each patch embedding $E_{i}$. This positional encoding ensures that
the model retains the positional information of each patch within the
spectrogram.

####  2.2.4 Mamba Encoder

The core component of SSAMBA is the Mamba encoder, which consists of
bidirectional SSMs [10]. The Mamba encoder processes the combined embeddings
$E_{i}+P_{i}$, capturing both forward and backward dependencies. This
bidirectional approach processes information in both temporal directions,
unlike unidirectional SSMs. The bidirectional SSM can be mathematically
described as:

Algorithm 1 Bidirectional Mamba Block Processing

1:Audio embedding sequence: $E_{1},\ldots,E_{M}$

2:Embeddings sequence: $H_{1},\ldots,H_{M}$

3:

4:for $i=1$ to $M$ do

5: if initial layer then

6: $E^{\prime}_{i}\leftarrow E_{i}+P_{i}$ $\triangleright$ Add positional
encoding to the initial layer input

7: else

8: $E^{\prime}_{i}\leftarrow H_{i}$ $\triangleright$ For subsequent layers,
use the output of the previous layer’s corresponding patch

9: end if

10: $x\leftarrow\text{Linear}_{x}(E^{\prime}_{i})$

11: $z\leftarrow\text{Linear}_{z}(E^{\prime}_{i})$

12: for $o\in\\{\text{forward},\text{backward}\\}$ do

13: $x^{\prime}_{o}\leftarrow\text{SiLU}(\text{Conv1D}_{o}(x))$

14: $B_{o}\leftarrow\text{Linear}_{B_{o}}(x^{\prime}_{o})$

15: $C_{o}\leftarrow\text{Linear}_{C_{o}}(x^{\prime}_{o})$

16:
$\Delta_{o}\leftarrow\log(1+\exp(\text{Linear}_{\Delta_{o}}(x^{\prime}_{o})))$

17: $A_{o}\leftarrow\Delta_{o}\times\text{Parameter}_{A_{o}}$

18: $B_{o}\leftarrow\Delta_{o}\times B_{o}$

19: $y_{o}\leftarrow\text{SSM}(A_{o},B_{o},C_{o})(x^{\prime}_{o})$

20: end for

21: $y^{\prime}_{\text{forward}}\leftarrow
y_{\text{forward}}\odot\text{SiLU}(z)$

22: $y^{\prime}_{\text{backward}}\leftarrow
y_{\text{backward}}\odot\text{SiLU}(z)$

23:
$H_{i}\leftarrow\text{Linear}_{T}(y^{\prime}_{\text{forward}}+y^{\prime}_{\text{backward}})+E^{\prime}_{i}$

24:end for

25:return $H_{1},\ldots,H_{M}$

M represents the number of patches in the input sequence, and $z$ is an
intermediate representation that modulates the forward and backward outputs of
the SSM blocks.

###  2.3 Self-Supervised Learning Framework

The SSAMBA model employs a self-supervised pretraining framework designed to
learn robust audio representations by jointly optimizing discriminative and
generative objectives. This section details the key components and methodology
of this framework.

####  2.3.1 Masked Spectrogram Patches

The spectrogram $S$ is then split into a sequence of non-overlapping patches.
Each patch $S_{i}$ is of size $F_{p}\times T_{p}$, where $F_{p}$ and $T_{p}$
are the dimensions of the patch in the frequency and time domains,
respectively. During pretraining, a portion of these patches is randomly
masked. The masked patches embeddings denoted by $[M]$ are used as targets for
the model to predict, forcing the model to learn the underlying structure of
the audio data.

####  2.3.2 Training Objective

The training objective of SSAMBA integrates both discriminative and generative
tasks to harness a comprehensive understanding of the audio spectrogram’s
structure. The overall training strategy involves two primary objectives:

  * •

Discriminative Objective: This objective focuses on correctly identifying the
masked patch. The discriminative task employs a classification head that
outputs a vector for each masked patch, which is then compared against all
other patch embeddings within the batch to compute the InfoNCE loss [20]:

$L_{d}=-\frac{1}{N}\sum_{i=1}^{N}\log\left(\frac{\exp(\langle
c_{i},x_{i}\rangle)}{\sum_{j=1}^{N}\exp(\langle c_{i},x_{j}\rangle)}\right),$

where $c_{i}$ is the output from the classification head for the $i$-th masked
patch, $x_{i}$ is the actual embedding of the $i$-th patch, and $N$ is the
total number of patches.

  * •

Generative Objective: Alongside the discriminative task, the generative
objective aims to reconstruct the original content of masked patches. A
reconstruction head generates predictions for the masked embeddings, which are
then evaluated using the Mean Squared Error (MSE) loss:

$L_{g}=\frac{1}{N}\sum_{i=1}^{N}\|\hat{x}_{i}-x_{i}\|^{2},$

where $\hat{x}_{i}$ is the predicted reconstruction of the masked patch and
$x_{i}$ is the true embedding of the patch.

The total loss $L$ is a weighted sum of the discriminative and generative
losses, with a balancing parameter $\lambda$ controlling the relative
importance of each:

$L=L_{d}+\lambda L_{g}.$

##  3 Results

(a) Average Inference Time vs. Input Size

(b) GPU Memory Usage vs. Input Size

Fig. 2: (a) Inference Time and (b) GPU Memory Usage for different model types
and sizes

###  3.1 Pretraining

For self-supervised pretraining of the SSAMBA model, we strategically mixed
and utilized audio samples from two datasets, focusing solely on the audio
components and excluding any associated labels to foster a robust learning
environment.

Dataset Preparation and Integration:

  * •

AudioSet-2M: We incorporated the entire unbalanced training set from AudioSet
[21], which includes approximately 2 million 10-second audio clips from
YouTube videos covering 527 distinct sound categories. These categories
encompass a wide range of sounds from human and animal activities to natural
and environmental noises. While AudioSet features a considerable presence of
speech, it is often not the dominant component in the clips, which motivated
the inclusion of an additional speech-focused dataset.

  * •

LibriSpeech: To improve speech representation in our training mix, we added
the 960-hour training set from LibriSpeech [22], which features audiobooks
read in English by over 1,000 speakers.

Data Handling:

  * •

To standardize the audio input, we processed all tracks from both AudioSet and
LibriSpeech to ensure a uniform duration of 10 seconds per sample, either by
cutting or padding the waveforms accordingly.

  * •

We downsampled all audio files to 16kHz and converted stereo tracks to mono by
averaging the channels, streamlining the audio input format for consistent
model training.

Training Configuration:

  * •

The dataset loader was designed to randomly select samples from the mixed
dataset pool, ensuring varied batch content for each training iteration.

  * •

We employed the Adam optimizer [23] with a learning rate of $1e-4$ and a batch
size of 64.

  * •

Training was conducted on NVIDIA L40 GPUs. We pretrained the base model on 2
GPUs, and used 1 GPU for the small and tiny sizes.

  * •

The pretraining was limited to 10 epochs, with an early stopping criterion
based on the validation loss; specifically, training was halted if no
significant improvement in the loss was observed during three consecutive
evaluations. This approach helped in preventing overfitting and ensured
efficient use of computational resources.

Table 1: Comparison of Model Specifications

Model Params Depth Embed Dim SSAST-tiny 6M 12 192 SSAST-small 23M 12 384
SSAST-base 89M 12 768 SSAMBA-tiny 7M 24 192 SSAMBA-small 26M 24 384 SSAMBA-
base 99M 24 768

###  3.2 Performance Comparison of SSAMBA and SSAST Models

####  3.2.1 Downstream tasks and dataset

In evaluating the performance of the SSAMBA and SSAST, we focused on a diverse
set of tasks tailored to test the models’ effectiveness in different audio
processing contexts. These tasks included the AudioSet-20K (AS) [21] for
multi-label audio event classification, ESC-50 (ESC) [24] for single-label
audio event classification, Speech Commands V1 (KS1) and V2 (KS2) [25] for
keyword spotting, and VoxCeleb 1 (SID) [26] for speaker identification. For a
fair and comprehensive comparison, three different sizes of both models—Tiny,
Small, and Base—were evaluated. The specifications of these models are shown
in Table 1

During the fine-tuning phase, unlike the pretraining setup, no patches are
masked, which allows the model to adjust to recognizing and classifying
complete audio data. Furthermore, the final output tokens of the encoder are
averaged before being fed to the classification head. This method streamlines
the decision-making process and focuses on the most salient features extracted
across the entire audio clip.

####  3.2.2 Downstream Performance Comparison

The results, summarized in Table 2, illustrate that SSAMBA generally
outperforms SSAST, particularly in the larger model configurations. SSAMBA’s
enhancements in architecture appear to provide superior handling of complex
audio patterns, as evidenced by its consistently higher performance across the
majority of tasks, especially in the ’Base’ model size. Notably, SSAMBA shows
significant improvement in the AudioSet-20K and environmental sound
classification tasks, suggesting robust feature extraction capabilities that
scale well with model size. These improvements highlight SSAMBA’s ability to
effectively capture and leverage audio representations, resulting in better
performance across diverse audio tasks.

####  3.2.3 Efficiency Comparison

Efficiency is a critical factor for deploying deep learning models in real-
world applications, where computational resources and inference times are
often constrained. We compared the inference speed and GPU memory usage of
SSAMBA and SSAST across different model sizes per varying input sizes, as
depicted in Figure 2. These comparisons were conducted with a batch size of 4
during inference.

The comparison shows that SSAMBA not only performs better in terms of accuracy
but also offers significant improvements in efficiency. For instance, when
comparing the Tiny models at an input size of 22k tokens, SSAMBA is
approximately 92.7% faster in inference speed and 95.4% more memory-efficient
than SSAST. These efficiency gains are crucial for real-time processing
applications and deployment on resource-constrained devices.

Table 2: Performance Comparison of SSAMBA and SSAST Models ($\dagger$
indicates using a larger learning rate for the final classification head)

Size Model AS KS1 KS2 ESC SID (mAP) (Acc.) (Acc.) (Acc.) (Acc.) Tiny SSAST
23.2$\dagger$ 94.8 97.1 79.5 64.3 SSAMBA 23.3$\dagger$ 94.0 94.0 80.1 66.1
Small SSAST 25.0$\dagger$ 95.4 97.8 85.4 67.0 SSAMBA 25.6$\dagger$ 96.4 96.3
85.5 67.9 Base SSAST 26.9 96.0 97.9 88.8 68.8 SSAMBA 28.3 96.9 97.4 89.3 70.1

###  3.3 Ablations

In this section, we explore the impact of varying the number of masked patches
during training on the performance of different sizes of the SSAMBA model. The
primary goal of these ablations is to understand how different levels of input
obfuscation during pretraining affect the model’s ability to generalize and
perform across various audio classification tasks. By systematically modifying
the number of patches that are masked, we assess the robustness and
flexibility of the model under varying degrees of information scarcity.

These experiments were conducted across three model sizes: Tiny, Small, and
Base. Each model was tested with three different settings of masked
patches—400, 300, and 250—to investigate how these variations influence
performance metrics in audio event classification (AS-20K) [21], keyword
spotting tasks (KS1 and KS2)[25], and Environmental Sound Classification
(ESC)[24]. The results are shown in Table 3.

Table 3: Impact of Masked Patches on Model Performance ($\dagger$ indicates
using a larger learning rate for the final classification head)

Size Patches AS KS1 KS2 ESC SID (mAP) (Acc.) (Acc.) (Acc.) (Acc.) SSAMBA-Tiny
400 23.3$\dagger$ 95.7 94.0 80.1 66.1 300 22.9$\dagger$ 95.5 94.3 78.1 65.2
250 23.0$\dagger$ 94.1 94.9 78.9 65.1 SSAMBA-Small 400 25.6$\dagger$ 96.4 96.3
85.5 67.9 300 25.1$\dagger$ 96.2 96.1 85.0 67.6 250 25.6$\dagger$ 96.4 96.3
85.3 67.3 SSAMBA-Base 400 28.3 96.9 97.4 89.3 70.1 300 28.5 97.3 97.6 89.1
68.5 250 28.2 97.3 97.7 88.6 68.5

The choice between RMSNorm and LayerNorm, as well as the use of fused add
norm, had little impact on the model’s performance. During our experiments, we
also evaluated the performance of unidirectional models but found that they
significantly underperformed compared to their bidirectional counterparts.
Consequently, we decided to proceed with the bidirectional model
configuration.

##  4 Conclusion

In this paper, we introduced the Self-Supervised Audio Mamba (SSAMBA), a novel
model for audio representation learning that leverages the advantages of state
space models (SSMs) with a bidirectional architecture. Unlike traditional
transformer-based models, which suffer from quadratic complexity, SSAMBA
offers a more efficient and scalable alternative by utilizing the Mamba
architecture. Our approach marks the first instance of a self-supervised,
attention-free, and SSM-based model applied to audio tasks. We demonstrated
the efficacy of SSAMBA through extensive experiments across various downstream
tasks, including audio classification, keyword spotting, environmental sound
classification, and speaker identification. Our results showed that SSAMBA
consistently outperforms the Self-Supervised Audio Spectrogram Transformer
(SSAST) in most tasks, particularly in larger model configurations. Notably,
SSAMBA achieved significant improvements in efficiency, with the Tiny model
being approximately 92.7% faster in inference speed and 95.4% more memory-
efficient than its SSAST counterpart with an input size of 22k patches.

SSAMBA’s robust performance stems from architectural innovations that capture
complex audio patterns and a mixed dataset pretraining strategy with AudioSet
and LibriSpeech. This combination enhances generalization across diverse audio
types. SSAMBA’s efficiency on resource-constrained devices suggests potential
for broad real-world applications, from mobile and edge devices to large-scale
cloud systems.

##  5 Acknowledgement

This work was funded by the National Institutes of Health (NIH- NIDCD) and a
grant from Marie-Josee and Henry R. Kravis.”

## References

  * [1] Abdelrahman Mohamed, Hung-yi Lee, Lasse Borgholt, Jakob D. Havtorn, Joakim Edin, Christian Igel, Katrin Kirchhoff, Shang-Wen Li, Karen Livescu, Lars Maaloe, Tara N. Sainath, and Shinji Watanabe,  “Self-Supervised Speech Representation Learning: A Review,”  IEEE Journal of Selected Topics in Signal Processing, vol. 16, no. 6, pp. 1179–1210, Oct. 2022.
  * [2] Yuan Gong, Cheng-I Lai, Yu-An Chung, and James Glass,  “Ssast: Self-supervised audio spectrogram transformer,”  Proceedings of the AAAI Conference on Artificial Intelligence, vol. 36, no. 10, pp. 10699–10709, Jun. 2022.
  * [3] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Lukasz Kaiser, and Illia Polosukhin,  “Attention is all you need,”  Advances in neural information processing systems, vol. 30, 2017.
  * [4] Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, Jakob Uszkoreit, and Neil Houlsby,  “An image is worth 16x16 words: Transformers for image recognition at scale,”  in International Conference on Learning Representations, 2021.
  * [5] Yuan Gong, Yu-An Chung, and James Glass,  “AST: Audio Spectrogram Transformer,”  in Proc. Interspeech 2021, 2021, pp. 571–575.
  * [6] Rudolph Emil Kalman,  “A new approach to linear filtering and prediction problems,”  1960\.
  * [7] Albert Gu, Isys Johnson, Karan Goel, Khaled Saab, Tri Dao, Atri Rudra, and Christopher Ré,  “Combining recurrent, convolutional, and continuous-time models with linear state-space layers,”  CoRR, vol. abs/2110.13985, 2021.
  * [8] Albert Gu, Karan Goel, and Christopher Ré,  “Efficiently modeling long sequences with structured state spaces,”  CoRR, vol. abs/2111.00396, 2021.
  * [9] Albert Gu and Tri Dao,  “Mamba: Linear-time sequence modeling with selective state spaces,”  arXiv, Dec 2023.
  * [10] Lianghui Zhu, Bencheng Liao, Qian Zhang, Xinlong Wang, Wenyu Liu, and Xinggang Wang,  “Vision Mamba: Efficient Visual Representation Learning with Bidirectional State Space Model,” Feb. 2024,  arXiv:2401.09417 [cs].
  * [11] Jiarun Liu, Hao Yang, Hong-Yu Zhou, Yan Xi, Lequan Yu, Yizhou Yu, Yong Liang, Guangming Shi, Shaoting Zhang, Hairong Zheng, and Shanshan Wang,  “Swin-umamba: Mamba-based unet with imagenet-based pretraining,” 2024.
  * [12] Jun Ma, Feifei Li, and Bo Wang,  “U-mamba: Enhancing long-range dependency for biomedical image segmentation,” 2024.
  * [13] Zhaohu Xing, Tian Ye, Yijun Yang, Guang Liu, and Lei Zhu,  “Segmamba: Long-range sequential modeling mamba for 3d medical image segmentation,” 2024.
  * [14] Kunchang Li, Xinhao Li, Yi Wang, Yinan He, Yali Wang, Limin Wang, and Yu Qiao,  “Videomamba: State space model for efficient video understanding,” 2024.
  * [15] Chloe Wang, Oleksii Tsepa, Jun Ma, and Bo Wang,  “Graph-mamba: Towards long-range graph sequence modeling with selective state spaces,” 2024.
  * [16] Changsheng Quan and Xiaofei Li,  “Multichannel long-term streaming neural speech enhancement for static and moving speakers,”  arXiv preprint arXiv:2403.07675, 2024.
  * [17] Yueyuan Sui, Minghui Zhao, Junxi Xia, Xiaofan Jiang, and Stephen Xia,  “Tramba: A hybrid transformer and mamba architecture for practical audio and bone conduction speech super resolution and enhancement on mobile and wearable platforms,” 2024.
  * [18] Xilin Jiang, Cong Han, and Nima Mesgarani,  “Dual-path Mamba: Short and Long-term Bidirectional Selective Structured State Space Models for Speech Separation,” Apr. 2024,  arXiv:2403.18257 [cs, eess].
  * [19] Yuan Gong, Cheng-I. Jeff Lai, Yu-An Chung, and James Glass,  “SSAST: Self-Supervised Audio Spectrogram Transformer,” Feb. 2022,  arXiv:2110.09784 [cs, eess].
  * [20] Aaron van den Oord, Yazhe Li, and Oriol Vinyals,  “Representation learning with contrastive predictive coding,”  arXiv preprint arXiv:1807.03748, 2018.
  * [21] Jort F Gemmeke, Daniel PW Ellis, Dylan Freedman, Aren Jansen, Wade Lawrence, R Channing Moore, Manoj Plakal, and Marvin Ritter,  “Audio set: An ontology and human-labeled dataset for audio events,”  in 2017 IEEE international conference on acoustics, speech and signal processing (ICASSP). IEEE, 2017, pp. 776–780.
  * [22] Vassil Panayotov, Guoguo Chen, Daniel Povey, and Sanjeev Khudanpur,  “Librispeech: an asr corpus based on public domain audio books,”  in 2015 IEEE international conference on acoustics, speech and signal processing (ICASSP). IEEE, 2015, pp. 5206–5210.
  * [23] Diederik P. Kingma and Jimmy Ba,  “Adam: A method for stochastic optimization,”  in 3rd International Conference on Learning Representations, ICLR 2015, San Diego, CA, USA, May 7-9, 2015, Conference Track Proceedings, Yoshua Bengio and Yann LeCun, Eds., 2015.
  * [24] Karol J Piczak,  “Esc: Dataset for environmental sound classification,”  in Proceedings of the 23rd ACM international conference on Multimedia, 2015, pp. 1015–1018.
  * [25] Pete Warden,  “Speech commands: A dataset for limited-vocabulary speech recognition,”  arXiv preprint arXiv:1804.03209, 2018.
  * [26] A. Nagrani, J. S. Chung, and A. Zisserman,  “Voxceleb: a large-scale speaker identification dataset,”  in INTERSPEECH, 2017.