Vol.0 (20xx) No.0, 000–000

11institutetext: Department of Physics, Technion, Haifa, 3200003, Israel;
<EMAIL_ADDRESS>
Received 20xx month day; accepted 20xx month day

# A pre-explosion effervescent zone for the circumstellar material in SN
2023ixf

Noam Soker

###### Abstract

I present the effervescent zone model to account for the compact dense
circumstellar material (CSM) around the progenitor of the core collapse
supernova (CCSN) SN 2023ixf. The effervescent zone is composed of bound dense
clumps that are lifted by stellar pulsation and envelope convection to
distances of $\approx{\rm tens}\times{~{}\rm AU}$, and then fall back. The
dense clumps provide most of the compact CSM mass and exist alongside the
regular (escaping) wind. I crudely estimate that for a compact CSM within
$R_{\rm CSM}\approx 30{~{}\rm AU}$ that contains $M_{\rm CSM}\approx
0.01M_{\odot}$, the density of each clump is $k_{b}\ga 3000$ times the density
of the regular wind at the same radius and that the total volume filling
factor of the clumps is several percent. The clumps might cover only a small
fraction of the CCSN photosphere in the first days post-explosion, accounting
for the lack of strong narrow absorption lines. The long-lived effervescent
zone is compatible with no evidence for outbursts in the years prior to SN
2023ixf explosion and the large-amplitude pulsations of its progenitor, and it
is an alternative to the CSM scenario of several-years-long high mass loss
rate wind.

###### keywords:

stars: massive – stars: mass-loss – supernovae: general; supernova:
individual: SN 2023ixf

##  1 Introduction

A number of studies conclude that the ejecta of the type II core collapse
supernova (CCSN) SN 2023ixf interacted with a circumstellar material (CSM)
that was extended up to a distance of $R_{\rm CSM}\simeq 20-50{~{}\rm AU}$
(e.g., Berger et al. 2023; Bostroem et al. 2023; Grefenstette et al. 2023;
Jacobson-Galan et al. 2023; Kilpatrick et al. 2023; Singh Teja et al. 2023;
Smith et al. 2023).

Two basic types of models might account for a compact CSM. In one the red
supergiant (RSG) progenitor of a CCSN with a close CSM experiences a pre-
explosion high-mass ejection episode that starts years to weeks before
explosion (e.g., Foley et al. 2007; Pastorello et al. 2007; Smith et al. 2010;
Margutti et al. 2014; Ofek et al. 2014; Svirski & Nakar 2014; Tartaglia et al.
2016; Yaron et al. 2017; Wang, Wang, & Dai 2019; Bruch et al. 2020; Prentice
et al. 2020; Strotjohann et al. 2021). Most of the studies of the CSM of SN
2023ixf take it to result from an outburst that took place few years before
explosion, e.g., Bostroem et al. (2023) take the wind velocity to be $v_{\rm
w}\simeq 10{~{}\rm km}{~{}\rm s}^{-1}$ and deduce the beginning of the high
mass loss rate to be $\tau_{\rm w}\simeq 17{~{}\rm yr}$ before explosion while
Jacobson-Galan et al. (2023) take $v_{\rm w}\simeq 50{~{}\rm km}{~{}\rm
s}^{-1}$ to estimate $\tau_{\rm w}\simeq 3-6{~{}\rm yr}$. Smith et al. (2023)
observationally infer the expansion velocity of the CSM to be $v_{\rm
w}=115{~{}\rm km}{~{}\rm s}^{-1}$ and from that they calculate $\tau_{\rm
w}\simeq 0.9-1.5{~{}\rm yr}$. Smith et al. (2023) consider the CSM to be non-
spherical.

The high mass loss rate phase shortly before explosion might be powered by a
binary interaction that is triggered by the sudden expansion of the RSG star
years to weeks before explosion (e.g., Soker 2013; Smith & Arnett
2014)111Soker (2013) suggested (based on earlier binary models of similar
types of transients) that a sudden pre-CCSN swelling may trigger asymmetric
mass loss in binary systems. Five months later Smith & Arnett (2014) repeated
this suggestion. Later papers, e.g., Smith (2017) and Smith et al. (2023),
wrongly attributed this suggestion to Smith & Arnett (2014).. The binary
interaction likely involves jets (e.g., a review by Soker 2022). A problem
with this type of CSM is that there was no recorded outburst of the progenitor
of SN 2023ixf. Neustadt, Kochanek, & Rizzo Smith (2023) find no outbursts in
the time period of 5600 to 463 days before explosion. Jencson et al. (2023)
find no evidence for an outburst, beside the periodic variability, from 2010
until 10 days before explosion. Soraisam et al. (2023) find that the star
continues its general variability over 16 years until less than two weeks
before explosion.

The second possibility for a compact CSM, up to $\approx 100{~{}\rm AU}$, is a
long-lived extended material around the CCSN progenitor. Dessart et al. (2017)
presented a model of a long-lived complex extended dense zone of $\approx
0.01M_{\odot}$ around RSG progenitors of CCSNe. Moriya et al. (2017) consider
a dense compact CSM that is the acceleration zone of the wind (also Moriya et
al. 2018). In this study I consider a different extended material above the
RSG progenitor of SN 2023ixf, namely, the effervescent CSM zone model that I
proposed to occur in some CCSN progenitors (Soker 2021).

The effervescent zone is an extended zone from the giant surface and up to
$R_{\rm eff}\approx(10-100)R_{\ast}$ where in addition to the regular
(escaping) stellar wind there are many dense clumps that do not reach the
escape velocity, and hence rise and fall back. The average density of the
bound mass might reach tens of times that of the regular wind. Since the
effervescent zone can live for thousands of years and more it removes the need
for many type II CCSN progenitors to experience a very strong outburst just
years to weeks before explosion. Namely, the long-lived effervescent zone can
mimic a short-duration enhanced mass loss rate phase. Again, in many cases the
RSG does experience an outburst before explosion, but not in all cases.

The effervescent CSM model for RSG stars is based on the effervescent zone
model for asymptotic giant branch (AGB) stars (Soker 2008). This model was
motivated by observations of inhomogeneous outflows from AGB stars, including
clumps (dusty or not) and sometimes with inflows alongside the outflow (e.g.,
Lopez et al. 1997; Vlemmings et al. 2002; Diamond & Kemball 2003; Cotton et
al. 2006; Fonfría et al. 2008; Klochkova & Chentsov 2007; Tuyet Nhung et al.
2019; Khouri et al. 2020). A good example is the pulsating AGB star Mira A
that has a radius of $R_{\ast}\simeq 500R_{\odot}$ (e.g., Wood & Karovska
2006) and that possesses an inhomogeneous and clumpy asymmetrical compact CSM,
up to $\approx{\rm several}\times 10{~{}\rm AU}$ (e.g., Planesas et al. 1990;
Ryde & Schöier 2001; Lopez et al. 1997).

The presence of compact CSM around some CCSNe and observations of
inhomogeneous winds, sometimes with clumps, and/or inflow/outflow in some RSGs
(e.g., Lobel & Dupree 2000; Humphreys et al. 2007; Josselin & Plez 2007;
Ohnaka et al. 2011; Kervella et al. 2016; Kamiński 2019) and claims for
extended regions where inflow and outflow coexist around stars that are close
to their Eddington luminosity limit (e.g., Owocki & van Marle 2008; van Marle
et al. 2009) motivated me to develop the effervescent zone model for CCSN
progenitors (see Soker 2021 for more details).

In this Letter I propose that the compact CSM around the progenitor of SN
2023ixf was an effervescent zone rather than the ejecta of a short-lived
enhanced mass loss rate episode. I mostly motivated by the lack of indication
of a pre-explosion outburst and from the large-amplitude pulsations of the
progenitor. My main goal is to estimate plausible parameters for the
effervescent zone (section 2). I discuss the implications in section 3.

##  2 The pre-explosion effervescent zone

###  2.1 An effervescent zone

I do not repeat here the derivations from Soker (2021), but rather only
present the basic ingredients of the effervescent zone model that I present
schematically in Fig. 1 with parameters for the progenitor of SN 2023ixf.

---
Figure 1: A schematic drawing of the effervescent zone crudely scaled to a
possible effervescent zone around the progenitor of SN 2023ixf at explosion.
The thick-blue arrows depict the regular (escaping) wind outflowing with
super-escape velocity $v_{w}$. The red-oval shapes represent the bound dense
clumps that rise and fall within the effervescent zone. The orange sphere at
the center is the RSG progenitor of SN 2023ixf of radius $R_{\ast}\simeq
800R_{\odot}$. The outer edge of the effervescent zone is at $R_{\rm
eff}\approx 30{~{}\rm AU}$. The typical ratio of the density of a clump to
that of the regular wind at a given radius is $k_{b}\equiv\rho_{c}/\rho_{\rm
wind}\approx 3000$, according to a simple model where most clumps are in a
shell near $R_{\rm eff}$ and their solid angle covering fraction is
$f_{S}\simeq 0.5$.

The clumps in the effervescent zone are uplifted by stellar pulsations with
additional uplifted forces by strong convection and possibly magnetic activity
and/or rotation. The progenitor of SN 2023ixf had large-amplitude pulsations
before explosion (e.g., Kilpatrick et al. 2023; Soraisam et al. 2023). The
period is about $2.8{~{}\rm yr}$, which Kilpatrick et al. (2023) note to be
consistent with kappa-mechanism pulsations of RSG stars but with a much larger
amplitude. These are the type of pulsations that can support an effervescent
zone. The stellar radiation cannot accelerate the clumps to escape velocities.
Actually, the stellar radiation is already accelerating the regular (escaping
wind) to about the maximum possible mass loss rate from momentum balance
$\dot{M}_{\rm wc}v_{\rm w}\simeq\eta_{\rm w}L/c$, where $v_{\rm w}$ is the
terminal wind speed, $L$ the stellar luminosity, and $\eta_{\rm w}$ is the
average number of times that a photon transfers its momentum to the wind in
the outward radial direction. Generally $\eta_{\rm w}<1$, but in dense and
opaque winds this factor can be $\eta_{\rm w}\ga 1$. Substituting typical
values gives

$\displaystyle\begin{aligned} \dot{M}_{\rm wc}&\simeq 4\times 10^{-5}\eta_{\rm
w}\left(\frac{L}{2\times 10^{5}L_{\odot}}\right)\\\ &\times\left(\frac{v_{\rm
w}}{100{~{}\rm km}{~{}\rm s}^{-1}}\right)^{-1}M_{\odot}{~{}\rm
yr}^{-1}.\end{aligned}$ (1)

I scale the luminosity according to the inferred values of $L\simeq 1.3\times
10^{5}L_{\odot}$ (Jencson et al. 2023), $L\simeq 1.6\times 10^{5}-3\times
10^{5}L_{\odot}$ Soraisam et al. (2023) and $L\simeq 5.5\times
10^{4}L_{\odot}$ (Kilpatrick et al. 2023). I scale the wind velocity for SN
2023ixf according to Smith et al. (2023) who infer it to be $v_{\rm
w}=115{~{}\rm km}{~{}\rm s}^{-1}$.

###  2.2 The density of the clumps

Consider a clump that is ejected from the star and its density is $k_{b}$
times the density of the regular wind at the same radius. The assumptions
(Soker 2021) are that it expands radially, i.e., its cross section facing the
star varies as $A_{b}(r)\propto r^{2}$, and that its width (along the radial
direction) $l_{b}$ stays constant. The forces that act on a clump are the
stellar gravity (inwards; the negative radial direction), and two outwards
forces, the drag by the regular wind and the radiation pressure. When the wind
mass loss rate is as given by equation (1) for $\eta_{\rm w}\simeq 1$ it turns
out that the expression for the maximum radius inside which the acceleration
is negative (otherwise the clump escapes with the wind) does not depend on
whether it is optically thin or thick and it reads (Soker 2021).

$r_{\rm max}\simeq\frac{v_{\rm
Kep}}{v_{w}}\left(\frac{l_{b}}{R_{\ast}}\right)^{1/2}k^{1/2}_{b}R_{\ast},$ (2)

where $v_{\rm Kep}=v_{\rm esc}/\sqrt{2}$ is the Keplerian velocity on the
stellar surface of the RSG star and $v_{\rm esc}$ is the escape velocity.

The escape velocity of the progenitor of SN 2023ixf is uncertain. Hosseinzadeh
et al. (2023) gives progenitor radius of $R_{\ast}\simeq 410R_{\odot}$. From
the luminosity and effective temperatures that studies infer the progenitor
stellar radius is $R_{\ast}\simeq 980R_{\odot}$ (Jencson et al. 2023) and
$R_{\ast}\simeq 500R_{\odot}$ (Kilpatrick et al. 2023). The Mass of the
progenitor is also uncertain. Jencson et al. (2023) estimate the initial mass
as $M_{\rm init}=17\pm 4M_{\odot}$. On the other hand Kilpatrick et al. (2023)
argues that the luminosity they infer is consistent with $M_{\rm
init}=11M_{\odot}$ and Pledger & Shara (2023) estimate that $M_{\rm
init}=8-10M_{\odot}$. Neustadt, Kochanek, & Rizzo Smith (2023) give the range
of $M_{\rm init}=9-14M_{\odot}$. Soraisam et al. (2023) infer a value of
$M_{\rm init}=20\pm 4M_{\odot}$. I scale here with a mass at explosion of
$M_{\ast}=10M_{\odot}$. Considering the long-period pulsation of $1091\pm 71$
d (Soraisam et al. 2023), as Kilpatrick et al. (2023) also report, I scale the
radius at explosion with $R_{\ast}\simeq 800R_{\odot}$. This gives for the
escape velocity and for the Keplerian velocity $v_{\rm esc}=69{~{}\rm
km}{~{}\rm s}^{-1}$ and $v_{\rm Kep}=49{~{}\rm km}{~{}\rm s}^{-1}$ ,
respectively.

I scale the outer radius of the compact CSM, which here is also the outer
radius of the effervescent zone, with $R_{\rm eff}=R_{\rm CSM}=30{~{}\rm
AU}=6445R_{\odot}$ (section 1). From equation (2) with $r_{\rm max}=R_{\rm
eff}$, I find the condition on the density factor of the clump to stay bound
at the radius of the effervescent zone $R_{\rm eff}$ to be

$k_{b}\ga 250\left(\frac{R_{\rm
eff}}{8R_{\ast}}\right)^{2}\left(\frac{v_{w}}{2v_{\rm
Kep}}\right)^{2}\left(\frac{l_{b}}{R_{\ast}}\right)^{-1}.$ (3)

Equation (3) gives the lower limit on the density ratio of clumps that reach
the radius of the effervescent zone and then fall back. Clumps with lower
density will be dragged out by the wind at that radius. For the wind
parameters used here the density of the clumps at $R_{\rm eff}$ is

$\displaystyle\small\begin{aligned} \rho_{b}(R_{\rm eff})\ga&2.5\times
10^{-14}\left(\frac{\dot{M}_{\rm w}}{4\times 10^{-5}M_{\odot}{~{}\rm
yr}^{-1}}\right)\\\
&\times\left(\frac{R_{\ast}}{800R_{\odot}}\right)^{-2}\left(\frac{v_{\rm
w}}{100{~{}\rm km}{~{}\rm s}^{-1}}\right)^{-1}\\\
&\times\left(\frac{v_{w}}{2v_{\rm
Kep}}\right)^{2}\left(\frac{l_{b}}{R_{\ast}}\right)^{-1}{~{}\rm g}{~{}\rm
cm}^{-3}.\end{aligned}$ (4)

The value of $l_{b}$ is not determined. The model assumes that very large
convective cells that move outward, acting together with the pulsation modes,
e.g., Freytag, Liljegren, & Höfner (2017) and Nhung et al. (2023) for AGB
stars, eject clumps (blobs) with sizes of $\simeq 0.2-0.5R_{\ast}$. Shortly
after it is formed and before it cools a clump expands in all directions due
to its high pressure. For that it is scaled with $l_{b}=R\ast$ (Soker 2021).
Near the surface of the star the density of the clump is $\simeq(R_{\rm
eff}/R_{\ast})^{2}\rho_{b}(R_{\rm eff})$ which for the scaling used here that
is appropriate for the progenitor of SN 2023ixf it is $\ga 2\times
10^{-12}{~{}\rm g}{~{}\rm cm}^{-3}$. This is about 2-3 orders of magnitude
smaller than the density of the photosphere of RSG stars.

###  2.3 Global Properties

The density profile of the clumps in the effervescent zone depends on the
velocity distribution of the clumps when they are ejected from the star and on
the initial density contrast $k_{b}$ (Soker 2021). Because the clumps slow
down as they rise and accelerated as they fall back to the star, the average
density is not $r^{-2}$. It can be shallower than this or steeper, depending
on the initial velocity distribution. These properties span an undetermined
parameter space of the effervescent zone model. I consider therefore global
and average properties.

Observations infer different CSM properties, depending on the type of
observation and analysis. Bostroem et al. (2023) infer a mean density of
$5.6\times 10^{-14}{~{}\rm g}{~{}\rm cm}^{-2}$. Grefenstette et al. (2023)
estimate a mass loss rate of $\dot{M}_{\rm w}\approx 3\times
10^{-4}M_{\odot}{~{}\rm yr}^{-1}$ for a wind velocity of $v_{\rm w}=50{~{}\rm
km}{~{}\rm s}^{-1}$, while Jacobson-Galan et al. (2023) estimate a value of
$\dot{M}_{\rm w}\approx 10^{-2}M_{\odot}{~{}\rm yr}^{-1}$ for $v_{\rm
w}=50{~{}\rm km}{~{}\rm s}^{-1}$. The mass of the compact CSM of SN 2023ixf is
in the range of $0.001-0.03M_{\odot}$. I scale with $M_{\rm
CSM}=0.01M_{\odot}$.

According to the effervescent zone model there is a regular wind in addition
to the dense clumps. For that, the covering fraction of the clumps, which is
the total solid angle the clumps cover divided by $4\pi$, cannot be more than
about half. Considering that the clumps spend a large fraction of the time
near their turning point that is close to $R_{\rm eff}$ I assume a simple
model where all clumps are in a shell of width $l_{b}$ near $R_{\rm eff}$. The
demand on the covering fraction to be $f_{S}<0.5$ yields a constraint on the
density of the clumps there to be

$\displaystyle\small\begin{aligned} \rho(R_{\rm eff})&\ga 2.8\times
10^{-13}\left(\frac{M_{\rm
CSM}}{0.01M_{\odot}}\right)\left(\frac{f_{S}}{0.5}\right)^{-1}\\\
&\times\left(\frac{R_{\rm eff}}{30{~{}\rm
AU}}\right)^{-2}\left(\frac{l_{b}}{800R_{\odot}}\right)^{-1}{~{}\rm g}{~{}\rm
cm}^{-3}.\end{aligned}$ (5)

By equations (4) and (3) this implies that the density factor of the clumps is
$k_{b}\ga 3000$. Since some clumps overlap in the solid angle they cover, the
covering fraction for these parameters is actually $f_{S}<0.5$.

Consider the total volume that the clumps occupy inside the effervescent zone
$V_{b}$. For a wind mass loss rate and velocity as the scaling of equation (1)
the total wind mass inside $R_{\rm eff}=30{~{}\rm AU}$ is $M_{\rm
wind,eff}=5.7\times 10^{-5}M_{\odot}$. The relative volume the clumps occupies
inside this radius is therefore

$\displaystyle\small\begin{aligned} &f_{V}\equiv\frac{V_{b}}{V(R_{\rm
eff})}\simeq 0.06\left(\frac{k_{b}}{3000}\right)^{-1}\left(\frac{R_{\rm
eff}}{30{~{}\rm AU}}\right)^{-1}\\\ &\times\left(\frac{M_{\rm
CSM}}{0.01M_{\odot}}\right)\left(\frac{v_{\rm w}}{100{~{}\rm km}{~{}\rm
s}^{-1}}\right)\left(\frac{\dot{M}_{\rm w}}{4\times 10^{-5}M_{\odot}{~{}\rm
yr}^{-1}}\right)^{-1}.\end{aligned}$ (6)

##  3 Discussion and Summary

This study presents the effervescent zone model as an alternative scenario to
the enhanced mass-loss rate phase few years before explosion as an explanation
for the compact CSM around the progenitor of SN 2023ixf. A schematic drawing
of the long-lived effervescent zone is in Fig. 1. The motivation to propose
the effervescent zone model comes from that there are no indications for
outbursts in the time period of years before SN 2023ixf explosion and the
large-amplitude pulsations of its progenitor. Such pulsations are expected to
facilitated the formation of an effervescent zone. Previous studies of SN
2023ixf ignored the possibility of an effervescent zone.

There are some undetermined properties of the effervescent zone model,
including the density and velocity distributions of the clumps that the RSG
star ejects as a result of its pulsations and envelope convection. For that,
at this stage I limit the study to present the possible properties of
individual clumps (section 2.2) and the global properties in the effervescent
zone (section 2.3). Because the CSM properties of SN 2023ixf are not well
determined, the quantities I derive are all scaled. These include the lower
limit on the ratio of the clump density to the regular wind density (equation
3) and its density at the effervescent radius (equation 4) in order for the
clump to fall back. A stronger lower bound on the clumps’ density comes from
the global demand that the clumps supply the CSM mass, but that they do not
cover more than about half of the solid angle in order to allow the existence
of a regular (escaping) wind (equation 5). The total volume the clumps occupy
within the effervescent zone for the parameters I use here is $f_{V}\simeq
0.06$ (equation 6).

The effervescent zone model has the following implications. First, as said,
there is no need for a pre-explosion outburst during the several years before
explosion. Secondly, the dense gas might show red-shifted and blue-shifted
emission of up to several tens of ${~{}\rm km}{~{}\rm s}^{-1}$, and any value
in between the red and blue-shifted emissions. There are a total of
$N_{c}\approx 50$ clumps in the effervescent zone. About half dominate the
emission towards the observer. Therefore, the spectrum is a combination of
these clumps, which will have one peak at the stellar velocity relative to the
observer, with a spread due to Doppler shifts. The regular wind, however, will
have stronger blue shifted emission. This might lead to a complicated
spectrum. In particular, if the dense clumps have a small covering fraction
along the line of sight to the photosphere of the ejecta then their influence
on the absorption lines is small. Our next step will be to calculate the
absorption and emission properties of the effervescent zone.

For the lack of narrow blueshifted absorption lines Smith et al. (2023)
suggest that most of the dense CSM is not along our line of sight. They
further propose that a binary interaction ejected the dense CSM into a disk or
a torus in the equatorial plane, and that this disk/torus does not cross our
line of sight to the supernova photosphere. They attribute the disappearance
of CSM lines to a flow structure where the supernova ejecta engulf the highly
non-spherical CSM. In the effervescent zone model that I suggest here for SN
2023ixf the dense clumps replace the equatorial mass ejection. The small solid
angle coverage by the dense clumps, $f_{S}<0.5$, and the small filling
fraction $f_{V}$ (equation 6), suggest that the clumps might play the same
role as dense equatorial disk or torus.

I encourage future studies of the ejecta-CSM interaction of SN 2023ixf to
consider the effervescent zone model.

## Acknowledgments

I thank an anonymous referee for useful comments. This research was supported
by a grant from the Israel Science Foundation (769/20).

## References

  * Berger et al. (2023) Berger E., Keating G. K., Margutti R., Maeda K., Alexander K. D., Cendes Y., Eftekhari T., et al., 2023, arXiv, arXiv:2306.09311.
  * Bostroem et al. (2023) Bostroem K. A., Pearson J., Shrestha M., Sand D. J., Valenti S., Jha S. W., Andrews J. E., et al., 2023, arXiv, arXiv:2306.10119.
  * Bruch et al. (2020) Bruch, R. J., Gal-Yam, A., Schulze, S., et al. 2020, arXiv:2008.09986
  * Cotton et al. (2006) Cotton, W. D., Vlemmings, W., Mennesson, B., et al. 2006, A&A, 456, 339
  * Dessart et al. (2017) Dessart, L., John Hillier, D., & Audit, E. 2017, A&A, 605, A83
  * Diamond & Kemball (2003) Diamond, P. J. & Kemball, A. J. 2003, ApJ, 599, 1372
  * Foley et al. (2007) Foley, R. J., Smith, N., Ganeshalingam, M., Li, W., Chornock, R., & Filippenko, A. V. 2007, ApJ, 657, L105
  * Fonfría et al. (2008) Fonfría, J. P., Cernicharo, J., Richter, M. J., & Lacy, J. H. 2008, ApJ, 673, 445
  * Freytag, Liljegren, & Höfner (2017) Freytag B., Liljegren S., Höfner S., 2017, A&A, 600, A137. doi:10.1051/0004-6361/201629594
  * Grefenstette et al. (2023) Grefenstette B. W., Brightman M., Earnshaw H. P., Harrison F. A., Margutti R., 2023, arXiv, arXiv:2306.04827.
  * Hosseinzadeh et al. (2023) Hosseinzadeh G., Farah J., Shrestha M., Sand D. J., Dong Y., Brown P. J., Bostroem K. A., et al., 2023, arXiv, arXiv:2306.06097.
  * Humphreys et al. (2007) Humphreys, R. M., Helton, L. A., & Jones, T. J. 2007, AJ, 133, 2716
  * Jacobson-Galan et al. (2023) Jacobson-Galan W. V., Dessart L., Margutti R., Chornock R., Foley R. J., Kilpatrick C. D., Jones D. O., et al., 2023, arXiv, arXiv:2306.04721.
  * Jencson et al. (2023) Jencson J. E., Pearson J., Beasor E. R., Lau R. M., Andrews J. E., Bostroem K. A., Dong Y., et al., 2023, arXiv, arXiv:2306.08678.
  * Josselin & Plez (2007) Josselin, E. & Plez, B. 2007, A&A, 469, 671
  * Kamiński (2019) Kamiński, T. 2019, A&A, 627, A114
  * Kervella et al. (2016) Kervella, P., Lagadec, E., Montargès, M., et al. 2016, A&A, 585, A28
  * Khouri et al. (2020) Khouri, T., Vlemmings, W. H. T., Paladini, C., et al. 2020, A&A, 635, A200
  * Kilpatrick et al. (2023) Kilpatrick C. D., Foley R. J., Jacobson-Galán W. V., Piro A. L., Smartt S. J., Drout M. R., Gagliano A., et al., 2023, arXiv, arXiv:2306.04722.
  * Klochkova & Chentsov (2007) Klochkova, V. G. & Chentsov, Y. L. 2007, Astronomy Reports, 51, 994
  * Lobel & Dupree (2000) Lobel, A. & Dupree, A. K. 2000, ApJ, 545, 454
  * Lopez et al. (1997) Lopez, B., Danchi, W. C., Bester, M., et al. 1997, ApJ, 488, 807
  * Margutti et al. (2014) Margutti, R., Milisavljevic, D., Soderberg, A. M., et al. 2014, ApJ, 780, 21
  * Moriya et al. (2018) Moriya, T. J., Förster, F., Yoon, S.-C., Gräfener, G., & Blinnikov, S. I., 2018, MNRAS, 476, 2840
  * Moriya et al. (2017) Moriya, T. J., Yoon, S.-C., Gräfener, G., & Blinnikov, S. I. 2017, MNRAS, 469, L108
  * Neustadt, Kochanek, & Rizzo Smith (2023) Neustadt J. M. M., Kochanek C. S., Rizzo Smith M., 2023, arXiv, arXiv:2306.06162.
  * Nhung et al. (2023) Nhung P. T., Hoai D. T., Darriulat P., Diep P. N., Ngoc N. B., Thai T. T., Tuan-Anh P., 2023, RAA, 23, 015004.
  * Ofek et al. (2014) Ofek, E. O., Sullivan, M., Shaviv, N. J., et al. 2014, ApJ, 789, 104
  * Ohnaka et al. (2011) Ohnaka, K., Weigelt, G., Millour, F., et al. 2011, A&A, 529, A163
  * Owocki & van Marle (2008) Owocki, S. P., & van Marle, A. J., 2008, in Massive Stars as Cosmic Engines, IAU Symp 250, ed. F. Bresolin, P. A. Crowther, & J. Puls (Cambridge Univ. Press) Volume 250, p. 71-82.
  * Pastorello et al. (2007) Pastorello, A., Smartt, S. J., Mattila, S., et al. 2007, Nature, 447, 829
  * Planesas et al. (1990) Planesas, P., Bachiller, R., Martin-Pintado, J., & Bujarrabal, V. 1990, ApJ, 351, 263
  * Pledger & Shara (2023) Pledger J. L., Shara M. M., 2023, arXiv, arXiv:2305.14447.
  * Prentice et al. (2020) Prentice, S. J., Maguire, K., Boian, I., et al. 2020, arXiv:2009.10509
  * Ryde & Schöier (2001) Ryde, N. & Schöier, F. L. 2001, ApJ, 547, 384
  * Singh Teja et al. (2023) Singh Teja R., Singh A., Dutta A., Basu J., Anupama G. C., Sahu D. K., Swain V., et al., 2023, arXiv, arXiv:2306.10284
  * Smith (2017) Smith N., 2017, Handbook of Supernovae, (ISBN 978-3-319-21845-8. Springer International Publishing AG, 2017) p. 403
  * Smith & Arnett (2014) Smith N., Arnett W. D., 2014, ApJ, 785, 82.
  * Smith et al. (2010) Smith, N., Miller, A., Li, W., et al. 2010, AJ, 139, 1451
  * Smith et al. (2023) Smith N., Pearson J., Sand D. J., Ilyin I., Bostroem K. A., Hosseinzadeh G., Shrestha M., 2023, arXiv, arXiv:2306.07964.
  * Soker (2008) Soker, N. 2008, New A, 13, 491
  * Soker (2013) Soker N., 2013, arXiv, arXiv:1302.5037.
  * Soker (2021) Soker N., 2021, ApJ, 906, 1.
  * Soker (2022) Soker N., 2022, RAA, 22, 122003.
  * Soraisam et al. (2023) Soraisam M. D., Szalai T., Van Dyk S. D., Andrews J. E., Srinivasan S., Chun S.-H., Matheson T., et al., 2023, arXiv, arXiv:2306.10783
  * Strotjohann et al. (2021) Strotjohann, N. L., Ofek, E. O., Gal-Yam, A., et al. 2021, arXiv:2010.11196
  * Svirski & Nakar (2014) Svirski, G., & Nakar, E. 2014, ApJ, 788, L14
  * Tartaglia et al. (2016) Tartaglia, L., Pastorello, A., Sullivan, M., et al. 2016, MNRAS, 459, 1039
  * Tuyet Nhung et al. (2019) Tuyet Nhung P. T., Hoai D. T., Tuan-Anh P., Le Bertre T., Darriulat P., Diep P. N., Phuong N. T., et al., 2019, RAA, 19, 043.
  * van Marle et al. (2009) van Marle, A. J., Owocki, S. P., & Shaviv, N. J. 2009, MNRAS, 394, 595
  * Vlemmings et al. (2002) Vlemmings, W. H. T., Diamond, P. J., & van Langevelde, H. J. 2002, A&A, 394, 589
  * Wang, Wang, & Dai (2019) Wang S.-Q., Wang L.-J., Dai Z.-G., 2019, RAA, 19, 063.
  * Wood & Karovska (2006) Wood, B. E. & Karovska, M. 2006, ApJ, 649, 410
  * Yaron et al. (2017) Yaron, O., Perley, D. A., Gal-Yam, A., et al. 2017, Nature Physics, 13, 510


# BD-MSA: Body decouple VHR Remote Sensing Image Change Detection method
guided by multi-scale feature information aggregation

Yonghui Tan, Xiaolong Li, Yishu Chen, and Jinquan Ai Yonghui Tan, Xiaolong Li
and Jinquan Ai was with the Key Laboratory of Mine Environmental Monitoring
and Improving around Poyang Lake, Ministry of Natural Resources, East China
University of Technology, Nanchang 330013, China (email:
<EMAIL_ADDRESS>cv tyh@ecut.edu.cn<EMAIL_ADDRESS>(_Corresponding author: Xiaolong Li_).Yishu Chen are with Ningbo Alatu Digital
Technology Co., Ltd, Ningbo 315000, China (email: 2817161223@qq.com).

###### Abstract

The purpose of remote sensing image change detection (RSCD) is to detect
differences between bi-temporal images taken at the same place. Deep learning
has been extensively used to RSCD tasks, yielding significant results in terms
of result recognition. However, due to the shooting angle of the satellite,
the impacts of thin clouds, and certain lighting conditions, the problem of
fuzzy edges in the change region in some remote sensing photographs cannot be
properly handled using current RSCD algorithms. To solve this issue, we
proposed a Body Decouple Multi-Scale by fearure Aggregation change detection
(BD-MSA), a novel model that collects both global and local feature map
information in the channel and space dimensions of the feature map during the
training and prediction phases. This approach allows us to successfully
extract the change region’s boundary information while also divorcing the
change region’s main body from its boundary. Numerous studies have shown that
the assessment metrics and evaluation effects of the model described in this
paper on the publicly available datasets DSIFN-CD and S2Looking are the best
when compared to other models.

###### Index Terms:

Change detection (CD), very high resolution (VHR) images, body decouple,
multi-scale information aggregation.

##  I Introduction

Change detection (CD) is a technique for determining whether a change has
occurred at the same area by examining images of that location at different
times [1]. Binary change detection is a popular technique that analyzes
information between two images to determine whether a pixel in one image has
changed. It then categorizes the pixels in the image as either changed or
unchanged. One of the fundamental and essential issues in remote sensing is
the interpretation of very-high-resolution (VHR) remote sensing (RS) images.
VHR remote sensing image change detection (RSCD) is useful for a variety of
remote sensing applications, including urban land use analysis [2], urban
planning [3], urban sprawl analysis [4], disaster assessment [5], and so on.

The challenge in RSCD is capturing the connections between regions of interest
between bi-temporal images while disregarding interference from other regions.
At the same time, several irritating elements such as seasonality in the bi-
temporal and image quality issues such as noise and contrast are not of
importance and should be ignored when performing CD.

Figure 1: A section of the images in the DSIFN-CD and S2Looking, with the
first column representing the pre-change images, the second representing the
post-change images, and the third representing the change mask. The photos in
the figure’s top row are from S2Looking, while those in the second and third
rows are from DSIFN-CD.

The two primary streams of CD in RS images are the traditional method and the
deep learning method, which has gained popularity in the last decade. Several
strategies for detecting changes in RS images have been proposed before using
deep learning to RS images on a broad scale. Coppin and Bauer [6] employed a
pixel-based change detection method for RSCD, which detects changes in gray
values or colors by comparing images from two points in time pixel by pixel.
Deng et al. [7] discovered and quantified land use change using PCA and a
hybrid classifier that includes both unsupervised and supervised
classification. He et al. [8] combined texture change information with
standard spectral-based change vector analysis (CVA), resulting in integrated
spectral and texture change information. Wu et al. [9] used slow feature
analysis to isolate the most time-undeformed section of a multi-temporal image
and migrate it to a new feature space, effectively concealing the image’s
unaltered pixels.

Although these techniques have produced good results, they have certain
drawbacks because they rely on standard image processing:

  * •

Traditional techniques often necessitate the manual design of features, which
may necessitate domain expertise and experience;

  * •

When dealing with complicated sceneries, varied lighting conditions, and
multi-category changes, traditional approaches have rather weak generalization
capacity;

  * •

For supervised learning, traditional methods often necessitate enormous
amounts of manually labeled data.

Deep learning is a technique that has evolved tremendously quickly in the
previous decade, and deep learning-based computer vision has achieved
exceptional performance in RSCD tasks because to CNNs’ robust feature
extraction capabilities. Deep learning computer vision based RSCD techniques
can be divided into three categories based on the structure of these models:
pure convolution based, attention mechanism based, and Transformer based. The
above can be categorized as 1) Fully Convolutional (FC-EF, FC-Siam-Di, FC-
Siam-Conc) [10], Improved UNet++ [11], IFNet [12], CDNet [13], DTCDSCN [14],
TINY-CD [15], these models simply extract features from RS images using CNNs,
which make it difficult to capture long-term dependencies across images and
may be insensitive to complicated scene changes. 2) MSPSNet [16], DSAMNet
[17], HANet [18], STANet [19], SNUNet [20], ADS-Net [21], DARNet [22], SRCDNet
[23], TFI-GR [24], the strategies described above boost the model’s
sensitivity to crucial regions and help improve change detection accuracy, but
it is challenging to collect global information in bi-temporal images. 3) BIT
[25], ChangeFormer [26], RSP-BIT [27], SwinSUNet [28], MTCNet [29],
TransUNetCD [30], DMATNet [31], FTN [32], AMTNet [33], Hybrid-transcd [34],
when compared to traditional convolutional approaches, Transformer can handle
long-range relationships better, but its ability to extract local contextual
information is poor and computationally expensive.

Although the methods described above produced outstanding results in the RSCD
task, they have certain flaws. Because of their narrow local perceptual domain
and susceptibility to spatial fluctuations, pure convolution-based approaches
have limited degrees of feature extraction for RSCD. Second, when executing
RSCD, the approach based on the attention mechanism can only take into
consideration the local information in the feature map and cannot aggregate
the global information. Third, the Transformer-based solution lacks the link
between contexts in the details, and the arithmetic need is excessively large.

It is worth mentioning that the changing camera angles for different time
phases, as well as the fact that most RS photographs are not shot at an angle
perpendicular to the ground, result in shadows on various features in the
enormous number of RS images. Furthermore, thin clouds appear in some RS
photographs as a result of meteorological conditions. As illustrated in Fig.
1, the majority of the buildings in the image are inclined and cast long
shadows on the ground, and thin clouds can be seen in some of the photographs.
When executing RSCD, the margins of the change region are likely to get
blurred due to the aforementioned issue. As a result, we prefer to address
this issue during model training.

Targeting the two aforementioned primary issues—that is, the inability of
current RSCD methods to effectively aggregate global and local feature
information simultaneously and the blurring of change region edges as a result
of feature shadowing in remote sensing images—we proposed BD-MSA, a model that
can simultaneously aggregate global and local information in multi-scale
feature maps and decouple the change region’s center from its edges during
training.

The contributions of this paper are as follows:

  * (1)

The Overall Feature Aggregation Module (OFAM), which we proposed in this
paper, is a technique that can simultaneously aggregated global and local
information in both channel and spatial dimensions. It can adapt feature
information at different scales in the backbone part while effectively
increasing the model’s accuracy;

  * (2)

Given the large difference in recognition accuracy between the main body and
the edge of the changing region in the RSCD task, this paper designs a
Decouple Module in the prediction head part that can effectively separate the
main body and the edge of the changing region, and the experimental results
show that using this module improves the model’s recognition accuracy for the
edge;

  * (3)

Since the MixFFN module in SegFormer can capture intricate feature
representations in the network, this paper presents the module in the network
decoder, enhancing the feature extraction and generalization capabilities of
the model;

  * (4)

Extensive studies show that the technique presented in this work outperforms
existing models on the public datasets DSIFN-CD and S2Looking, achieving the
SOTA (state-of-the-art) performance.

The rest of the paper is structured as follows. The prior approaches are
introduced in section II. The model’s detail described in this paper is
introduced in Section III. Section IV conducts experiments to compare this
paper’s method with related methods. This paper is summarized in Section V.

##  II Related Work

In this section, we present an overview of existing RSCD works, including:
pure convolution-based, attention mechanism-based, and Transformer-based.

###  II-A Pure Convolutional-based Model

Deep CNNs have achieved amazing performance in the field of computer vision
[35] due to their powerful feature extraction capabilities. RS image
interpretation is essentially an image processing in which deep learning plays
an important role such as image classification [36], object detection [37,
38], semantic segmentation [39, 40], and change detection [41].

In the field of RSCD, the first attempt to use fully convolutional networks is
the work of [10], it devided into three methods namely FC-EF, FC-Siam-Di, and
FC-Siam-Conc, it proposes a CD architecture for the Siamese Network and
demonstrates that this architecture is effective; In [11], an improved UNet++
[42] has been proposed, it adopts the MSOF strategy, which can effectively
combine multi-scale information and help to detect objects with large size and
scale variations on VHR RS images; Zhang et al. [12] proposed a depth-
supervised image fusion network for CD in high-resolution bi-temporal RS
images, which combines the attention module and depth supervision to provide
an effective new way for CD in RS images. For better industrial applications,
Andrea et al. [15] proposed TINY-CD, which employs the Siamese U-Net
architecture and an innovative Mixed and Attention Masking Block (MAMB) to
achieve better performance than existing models while being smaller in size.

###  II-B Attention Mechanism-based Model

Attention mechanisms were first introduced in the context of natural language
processing [43]. Later, computer vision researchers presented attentional
processes that could be applied to images [44, 45, 46].

One can apply attention techniques in the field of RSCD, just like in most
other computer vision jobs. In order to address issues like illumination noise
and scale variations in aerial image change detection, Shi et al. [17]
introduced a deeply supervised attentional metric network for remote sensing
change detection, this network incorporates a metric learning module and a
convolutional block attentional module (CBAM) to enhance feature
differentiation; In order to increase detection accuracy, Guo et al. [16]
suggested a deep multiscale twin network for RSCD. This network is based on a
deep multiscale twin neural network and incorporates a self-attention module
and a parallel convolutional structure. Li et al. [22] proposed a dense
attention refinement network that combines dense hopping connections, a hybrid
attention module that combines a channel attention module and a spatial-
temporal attention module, and a recursive refinement module to effectively
improve the accuracy of CD in high-resolution bi-temporal RS images. In order
to overcome the resolution disparity between bi-temporal images, Liu et al.
[23] created SRCDNet, which learns super-resolution images using adversarial
learning and enriches multiscale features with a stacked attention module made
up of five CBAMs.

Even though attention-based RSCD is more adept at identifying local contextual
information from bi-temporal RS images, it is less effective at capturing the
global information.

###  II-C Transformer-based Model

Transformer is crucial to RSCD because of its potent global feature extraction
capacity. For the first time, BIT [25] brings Transformer, which effectively
describes context in the spatial-temporal domain to the RSCD domain. In order
to model context and improve features, BIT converts the input image into a
limited set of high-level semantic tokens using Transformer encoders and
decoders; Based on BIT, RSP-BIT [27] primarily focuses on using Remote Sensing
Pretraining (RSP) to analyze aerial images. It has been observed that RSP
enhances performance on the scene identification test and helps comprehend the
semantics related to RS; By fusing a multiscale Transformer with a CBAM, Wang
et al. [29] developed MTCNet. It creates a multiscale module to create the
multiscale Transformer after extracting the bi-temporal image features using
the Transformer module.

While Transformer performs RSCD tasks effectively in terms of global
information extraction, its huge number of parameters makes prediction more
time-consuming, and it struggles to extract the semantics across local
contexts.

##  III Methodology

In this section, we proposed BD-MSA, a novel approach in which we first
provide a brief overview of the general structure, followed by a full
description of the modules in our approach in each subsection.

###  III-A Overall Structure

The siamese network is presently a commonly utilized structure in RSCD, which
uses two weight-sharing Backbone in the feature extraction phase to extract
features from the input. In BD-MSA, we feed $I=\left\\{I_{1},I_{2}\right\\}$
into the CNN Backbone to extract the respective deep features of the bi-
temporal images, which are then sent successively through the Decouple Decoder
and the Prediction Mask, and the output is compared with the Mask.

Figure 2: Schematic diagram of BD-MSA.

Fig. 2 depicts the general architecture diagram of BD-MSA. The diagram is
divided into three primary sections: CNN Backbone, Decouple Decoder, and
Prediction Mask. The following equation can illustrate the model training
process:

$\rm\hat{Y}=Predict\left(Decoder\left(Backbone\left\\{I_{1},I_{2}\right\\}\right)\right)$
(1)

where Backbone, Decoder, and Predict represent different parts of the model
diagram, $\rm\hat{Y}$ represents the training result graph, and $\rm I_{1}$,
$\rm I_{2}$ represent the input bi-temporal images. In Algorithm 1, we have
expressed the model training procedure as pseudo-code to help the reader
comprehend.

Algorithm 1 Inference of BD-MSA for Change Detection

0: $\textbf{I}=\left\\{\left(\textbf{I}^{1},\textbf{I}^{2}\right)\right\\}$ (a
pair of bi-temporal image)

0: M (a prediction change mask)

1: // step1: extract high-level features by MiT backbone and OAFM

2: for $i~{}in~{}\left\\{1,2\right\\}$ do

3: for $n~{}in~{}\left\\{1,2,3,4\right\\}$ do

4: $\textbf{MiT}^{i}_{n}={\rm MiT\underline{~{}}Backbone}(\textbf{T}^{i})$

5: $\textbf{F}^{i}_{n}={\rm OAFM}(\textbf{MiT}^{i}_{n})$

6: end for

7: end for

8: // step2: Concat high-level feature to FA Module

9: $\textbf{F}_{FA}={\rm
FA\underline{~{}}Module}(\textbf{F}^{1}_{4},\textbf{F}^{2}_{4})$

10: // step3: Decoupling $\textbf{F}_{FA}$ into $\textbf{F}_{body}$ and
$\textbf{F}_{edge}$ by Body Decouple and Edge Decouple

11: $\textbf{F}_{body}={\rm Body\underline{~{}}Decouple}(\textbf{F}_{FA})$

12: $\textbf{F}_{edge}={\rm Edge\underline{~{}}Decouple}(\textbf{F}_{FA})$

13: $\textbf{M}={\rm Conv}({\rm Concat}(\textbf{F}_{body},\textbf{F}_{edge}))$

###  III-B Overall Feature Aggregation Module (OFAM)

In the feature extraction section, we utilize a feature decoder with shared
weights to send the input diachronic phase through two identical Backbones
with the same weights during the training process. The Backbone we designed,
in particular, can be divided into four stages, in which the input features
are first made to pass through the MiT [47], and the output results are then
pass through the OFAM while extracting both local and global features in the
channel dimension and spatial dimension of the feature map. Fig. 3 depicts the
OFAM module that we designed.

Figure 3: The graphic depicts our OFAM, which is separated into three major
portions, Channel Attention, Spatial Attention, and Fusion, which are
distinguished by various colored backgrounds.

Our designed OFAM is divided into three parts. First, we divide the output of
MiT in channel dimension into two branches, one of which, Local Channel
Attention, is used to extract local features and the other branch, Global
Channel Attention, is used to extract global features; the related
computational formula is as follows:

$\mathbf{F}_{c}^{l}=\mathrm{MiT}\left(\mathbf{F}_{n}^{i}\right)+\mathrm{LCA}\left(\mathrm{MiT}\left(\mathbf{F}_{n}^{i}\right)\right)\times\mathrm{MiT}\left(\mathbf{F}_{n}^{i}\right)$
(2)
$\mathbf{F}_{c}^{g}=\mathrm{GCA}\left(\mathrm{MiT}\left(\mathbf{F}_{n}^{i}\right)\right)\times\mathrm{MiT}\left(\mathbf{F}_{n}^{i}\right)$
(3)

where LCA and GCA denote Local Channel Attention and Global Channel Attention,
respectively, $\mathbf{F}_{c}^{l}$ and $\mathbf{F}_{c}^{g}$ denote locally and
globally extracted channel dimension features.

Following Channel Attention, the obtained $\mathbf{F}_{c}^{l}$ and
$\mathbf{F}_{c}^{g}$ are sent to Spatial Attention, where they are used to
construct global and local attention feature maps $\mathbf{F}_{s}^{g}$,
$\mathbf{F}_{s}^{l}$ in channel dimension. The relevant formulas are as
follows:

$\mathbf{F}_{s}^{l}=\mathrm{LSA}\left(\mathrm{MiT}\left(\mathbf{F}_{n}^{i}\right)\right)\times\mathbf{F}_{c}^{l}+\mathbf{F}_{c}^{l}$
(4)
$\mathbf{F}_{s}^{g}=\mathrm{GSA}\left(\mathrm{MiT}\left(\mathbf{F}_{n}^{i}\right)\right)\times\mathbf{F}_{c}^{g}+\mathbf{F}_{c}^{g}$
(5)

where LSA and GSA correspond to Local Spatial Attention and Global Spatial
Attention in Fig. 3. $\mathbf{F}_{s}^{l}$ and $\mathbf{F}_{s}^{g}$ are the two
output feature layers of Spatial Attention, which weight local and global
information in the spatial dimension, respectively. Unlike Channel Attention,
the topic part of Spatial Attention is symmetric, with only Local Spatial
Attention and Global Spatial Attention differing.

Following the extraction of global and local information in the channel and
spatial dimensions, the features are fused to produce the final output feature
map.

$\mathbf{F}_{n+1}^{i}=\mathrm{MiT}\left(\mathbf{F}_{n}^{i}\right)\times\mathbf{F}_{s}^{l}\times\mathbf{F}_{s}^{g}+\mathbf{F}_{s}^{l}+\mathbf{F}_{s}^{g}$
(6)

Each Attention Module in OFAM is detailed in depth in Fig. 4. The processing
of the feature maps in each section is shown below:

  1. 1.

Part (a) of the diagram depicts a simple convolutional neural network that
incorporates the layers of convolution, pooling, and so on by linking them in
sequence, as shown in Eq:

$\displaystyle\mathbf{F}_{out}=\mathrm{\sigma}\left(\mathrm{Conv}^{3\times
3}\left(\mathrm{LAP}\left(\mathbf{F}_{in}\right)\right)\right)$ (7)

where $\mathrm{\sigma}$ denotes the Softmax activation function,
$\mathrm{LAP}$ denotes Local Channel Attention, $\mathrm{Conv}^{3\times
3}\left(\cdot\right)$ is a convolutional layer with a convolutional kernel
size of 3$\times$3, and $\mathbf{F}_{in}$ and $\mathbf{F}_{out}$ denote the
input and output, respectively.

  2. 2.

Part (b) in Fig. 4 tends to extract global features compared to part (a) in
the design of the pooling layer, and we picked three different sizes of
convolution to extract the input features, which are 3$\times$3, 5$\times$5,
and 7$\times$7\. Part (b) can be written as follows:

$\displaystyle\begin{split}\mathbf{F}_{out}&=\mathrm{\sigma}\left(\mathrm{Conv}^{3\times
3}\left(\mathrm{Concat}\left(\mathrm{Conv}\left(\mathbf{F}_{in}\right)\right)\right)\right)\\\
\mathrm{Conv}\left(\cdot\right)&=\left\\{\mathrm{Conv}^{3\times
3}\left(\cdot\right),\mathrm{Conv}^{5\times
5}\left(\cdot\right),\mathrm{Conv}^{7\times
7}\left(\cdot\right)\right\\}\end{split}$ (8)

where $\mathrm{Concat}$ denotes the splicing of the input feature
$\mathbf{F}_{in}$ in the channel dimension after three different convolutions
and scaling to a uniform size.

  3. 3.

Parts (a) and (b) weight the feature maps solely in the channel dimension, but
they do not examine the relationship between the convolution kernel and the
input feature maps for different convolution sizes, therefore we devised part
(c) to address this issue. This can be stated mathematically as follows:

$\displaystyle\begin{split}\mathbf{F}_{mid}&=\mathrm{\sigma}\left(\prod_{i=1}^{2}{\mathrm{ConvG}_{i}\left(\mathrm{Conv}\left(\mathbf{F}_{in}\right)\right)}\right)\\\
\mathbf{F}_{out}&=\mathbf{F}_{in}\times\mathbf{F}_{mid}\end{split}$ (9)

where $\mathrm{ConvG}_{i}$ denotes that the features are first subjected to a
convolution operation with a convolution kernel size of 3$\times$3, followed
by the GeLU activation function [48].

  4. 4.

We created the module depicted in (d) to use the ability of the interaction
between different convolutional kernels for the extraction of global
information, with the goal of weight extraction of global information at the
spatial level. The following are the calculating formulas:

$\displaystyle\begin{split}\mathbf{F}_{mid}&=\gamma\left(\mathrm{Conv}^{5\times
5}\left(\mathbf{F}_{in}\right)\times\mathrm{Conv}^{7\times
7}\left(\mathbf{F}_{in}\right)\right)\\\
\mathbf{F}_{out}&=\mathrm{ConvS}\left(\mathbf{F}_{mid}\times\mathrm{Conv}^{3\times
3}\left(\mathbf{F}_{in}\right)\right)\end{split}$ (10)

where $\gamma$ denotes the GeLU activation function, and
$\mathrm{Conv}^{5\times 5}$ and $\mathrm{Conv}^{7\times 7}$ denote
convolutional layers with convolutional kernel sizes of 5$\times$5 and
7$\times$7, respectively.

Figure 4: Parts (a), (b), (c), and (d) of the OFAM schematic diagrams depict
Local Channel Attention, Global Channel Attention, Local Spatial Attention,
and Global Spatial Attention, respectively, in Fig. 3.

In the model feature extraction section, we combine the MiT feature extractor
with OFAM. The global and local information in the feature map is retrieved
simultaneously in both channel and spatial dimensions, thereby aggregating the
positional and spectral information in the remote sensing image.

###  III-C FA Module

After Backbone, we created a feature aggregation module called FA (Feature
Alignment) Module to better aggregate the deep features produced by feature
extraction for bi-temporal images. The FA Module construction is depicted in
Fig. 5. We integrate MixFFN from SegFormer [47] after FDAF in ChangerEX [49]
to improve feature representation and contextual comprehension when performing
feature extraction in image altering regions. The following are the relevant
formulas:

Figure 5: A schematic representation of our FA Module, which is separated into
two main portions, FDAF and MixFFN, which are distinguished by various colored
backgrounds.

$\begin{split}\mathbf{F}_{con}&=\mathrm{Concat}\left(\mathbf{F}_{in1},\mathbf{F}_{in2}\right)\\\
\mathbf{F}_{flow}&=\mathrm{Conv}\left(\gamma\left(\mathrm{InsNorm}\left(\mathrm{Conv}\left(\mathbf{F}_{con}\right)\right)\right)\right)\\\
\mathbf{F}_{FDAF}&=\mathrm{Concat}\left(\mathbf{F}_{in}-\mathrm{warp}\left(\mathbf{F}_{flow1},\mathbf{F}_{flow2}\right)\right)\end{split}$
(11)

where $\mathbf{F}_{in1}$, $\mathbf{F}_{in2}$ denote the feature maps generated
by Backbone respectively, $\mathrm{InsNorm}$ denotes the Instance
normalization method [50], $\gamma$ denotes the GeLU activation function, and
warp is the Feature Warp in the upper right corner of the Fig. 5.

In FDAF, we first splice the two input features in channel dimension and then
insert them into the dashed box on the left side of the figure. Borrowing the
idea of flow field in the field of video processing [51], the authors design a
feature alignment method, i.e., warp in Fig. 5, to correct the feature offset
problem caused by the dimensional change of the input feature maps after
feature extraction is performed.

In warp, the semantic flow field $\Delta_{l-1}$ is generated by bilinear
interpolating $\mathbf{F}_{l-1}$ to the same size as $\mathbf{F}_{l}$, then
concatenating the two in the channel dimension, and finally a convolutional
layer. Following that, using a simple addition operation, each position
${p}_{l-1}$ is mapped to a point ${p}_{l}$ in the preceding layer $l$.
Finally, using a bilinear sampling method, the values of the four nearby
pixels are linearly interpolated to approximate the FAM’s final output
$\mathbf{F}_{l}\left({p}_{l-1}\right)$. The following are the relevant
formulas for the aforementioned calculations:

$\begin{split}\Delta_{l-1}&=\mathrm{Conv}_{l}\left(\mathrm{Concat}\left(\mathbf{F}_{l},\mathbf{F}_{l-1}\right)\right)\\\
{p}_{l}&={p}_{l-1}+\frac{\Delta_{l-1}\left({p}_{l-1}\right)}{2}\\\
\mathbf{F}_{l}\left({p}_{l-1}\right)&=\mathbf{F}_{l}\left({p}_{l}\right)=\sum_{{p}\in
N\left({p}_{l}\right)}{\mathrm{\omega}_{p}\mathbf{F}_{l}\left(p\right)}\end{split}$
(12)

where $N\left({p}_{l}\right)$ denotes the neighborhood of the deformation
point ${p}_{\mathrm{l}}$ in $\mathbf{F}_{l}$ and $\mathrm{\omega}_{p}$ denotes
the bilinear kernel weights.

Considering the information interaction between bi-temporal RS images and
inspired by ChangerEx, we introduce FDAF into the method of this paper and
simultaneously insert MixFFN after FDAF to improve the feature expression
ability after information fusion between bi-temporal phases.

###  III-D Feature Decouple Module

Some of the image change edges in the RSCD datasets were found to be blurred.
This is due in part to the long shadows cast by tilt photography on ground
buildings, and in part to blurring of image regions of interest caused by
image quality issues in remote sensing photographs such as overexposure, thin
clouds, and so on.

Meanwhile, in the RSCD datasets, detection accuracy is high relative to the
edges of the modified region due to consistent semantic information throughout
the building, indicating homogeneity. In order to solve the aforementioned
challenges, we expect to decouple the changing region interior and edges
throughout the training process, which will allow us to extract the region
boundary on the one hand and effectively minimize the computation on the
other.

As a result, we use the flow field concept and add the Decouple Module after
feature decoding in the model to successfully extract the boundary of the
changing region throughout the training process. The decouple module is
depicted in Fig. 6.

Figure 6: Illustration of our proposed Decouple Module.

We initially sample the input feature map $\mathbf{F}_{in}$ twice (DownSample
and UpSample) in Fig. 6 to boost its semantic information without affecting
the feature size. In sectionrefsec:FA-Module, we use Warp to correct the
features of $\mathbf{F}_{in}$ to get $\mathbf{F}_{body}$, and then subtract
$\mathbf{F}_{in}$ from $\mathbf{F}_{body}$ to produce $\mathbf{F}_{edge}$.The
following are the relevant formulas:

$\begin{split}\mathbf{F}_{flow}&=\mathrm{ConcatConv}\left(\mathbf{F}_{in},\mathrm{DownUp}\left(\mathbf{F}_{in}\right)\right)\\\
\mathbf{F}_{body}&=\mathrm{Warp}\left(\mathbf{F}_{flow},\mathbf{F}_{in}\right)\\\
\mathbf{F}_{edge}&=\mathbf{F}_{in}-\mathbf{F}_{body}\end{split}$ (13)

where $\mathrm{DownUp}$ indicates that $\mathbf{F}_{in}$ is downsampled before
being upsampled.

After passing the features via the Decouple Module during the model training
process, the features are successfully classified as edge features and body
features. To the best of our knowledge, we are the first in the field of RSCD
to do so. This substantially enhances the model’s prediction capacity and, to
some extent, reduces the number of parameters in the model.

##  IV Experimental Results and Analysis

In this section, we first introduce the dataset, experimental environment, and
validation metrics used in this paper’s experiments, then compare the model of
this paper to other models, conduct ablation experiments to evaluate the
effect of each module, and finally visualize some of the feature maps
generated during the model’s training process.

###  IV-A Experimental Setup

For this experiment, the two public RSCD datasets listed below were employed.

DSIFN-CD [12] is derived from six Chinese cities, including Beijing, and was
manually collected in Google Earth. It is a publicly available binary change
detection dataset with a spatial resolution of 2m that includes changes to
roads, buildings, agriculture, and water bodies. During the experimental
process, we cropped each image to 512$\times$512, and the test set in the
original dataset was of lower quality, so we divided the original training set
into a training set and a validation set, and we used the original validation
set as the test set, and the dataset now has 3000/600/340
training/validation/test respectively.

S2Looking [52] is a publicly available dataset of 5000 pairs of bi-temporal RS
images broken into 3500/500/1000 training/validation/test sets with a spatial
resolution of 0.5 0.8m and a size of 1024$\times$1024 for each image.

Some of the images in DSIFN-CD and S2Looking are shown in Fig. 7. The three
columns in the figure are pre-change image, post-change image, and change
Mask, respectively.

Figure 7: Some of the images in DSIFN-CD and S2Looking.

###  IV-B Implementation Details

This experiment was deployed under PyTorch 2.0.1 and Python 3.8.13. For
hardware, we used Intel Xeon E5-2678 v3 @2.50GHz$\times$2, 32GB of RAM as well
as used an NVIDIA RTX 4090 GPU. And for hyper-parameters, we used BCE Loss as
the paper’s loss function for our experiments and use AdamW as the optimizer,
which is formally defined as:

$\begin{split}\mathcal{L}_{BCE}&=-\frac{1}{H\times
W}\sum_{h=1,w=1}^{H,W}\bigg{[}{Y\left(h,w\right)}\\\
&+\left(1-Y\left(h,w\right)\right)\cdot\log\left(1-\hat{Y}\left(h,w\right)\right)\bigg{]}\end{split}$
(14)

$\begin{split}\theta_{t+1}=\theta_{t}-\frac{\alpha}{\sqrt{\hat{\upsilon}_{t}}+\varepsilon}\hat{m}_{t}-\alpha\lambda\theta_{t}\end{split}$
(15)

in 14, where $H\times W$ is the size of the image to be predicted, $Y(h,w)$ is
the predicted value of the point $(h,w)$ in the image and $\hat{Y}(h,w)$ is
the true value of the point; in 15, $\theta_{t}$ and $\theta_{t+1}$ denote the
parameter values at time steps $t$ and $t+1$, respectively, $\alpha$ is the
learning rate, $\hat{m}_{t}$ and $\hat{\upsilon}_{t}$ are the exponential
moving averages of the first-order and second-order moments, respectively, and
$\varepsilon$ is a very small value.

In this paper, we use the Open-CD development kit [49] based on OpenMMLab [53]
in order to compare the training results of different models in the same
experimental environment.

Evaluation Metrics. We used the following metrics for validation to validate
the training effect of our proposed BD-MSA: F1-score (F1), Precision (Prec.),
Recall (Rec.), and IoU, which are defined as follows:

$\rm Pre.=\frac{TP}{TP+FP}$ (16) $\rm Rec.=\frac{TP}{TP+FN}$ (17) $\rm
F1=2\times\frac{Pre\times Rec}{Pre+Rec}$ (18) $\rm IoU=\frac{TP}{TP+FP+FN}$
(19)

where TP, FP, and FN represents the number of true positive, false positive,
and false negative pixels, respectively.

###  IV-C Comparison With SOTA Methods

We compared the approaches mentioned in this work to some SOTA methods, which
are listed below:

  * •

FC-EF, FC-Siam-Di and FC-Siam-Conc [10] are built on fully convolutional
networks [54] with a model structure similar to that of U-Net [55], and they
use distinct methodologies to analyze paired image data.

  * •

BIT [25] introduces Transformer [43] to classic CNN change detection networks,
which can more effectively capture long-distance interdependence and complex
spatial dynamics.

  * •

ChangeFormer [26], unlike typical fully convolutional network-based
techniques, ChangeFormer combines a hierarchically structured Transformer
encoder and a multilayer perceptron decoder to efficiently capture long range
information at multi-scale, enhancing change detection accuracy.

  * •

ChangerEx-MiT [49] emphasizes the significance of feature interaction and
presents simple but effective interaction mechanisms—AD and feature
“exchange”.

  * •

HANet [18] addresses the challenge of data imbalance between changed and
unchanged pixels in the change detection task by proposing a stepwise
foreground-balanced sampling strategy to improve model learning for changed
pixels and employing a concatenated network structure with hierarchical
attention to integrate multi-scale features for finer detection.

  * •

IFNet [12] collects deep features using a fully convolved two-stream
architecture and then uses a difference discrimination network and an
attention module to identify changes, highlighting the significance of deep
supervision in improving border integrity and object internal compactness.

  * •

SNUNet [20], through tight hopping connections between the encoder and decoder
as well as between decoders, SNUNet is able to maintain high-resolution fine-
grained features while mitigating pixel uncertainty at the borders of changing
targets and deterministic missingness of small targets.

  * •

STANet [19] captures spatial-temporal correlations via a self-attentive method
in order to generate more discriminative features. It was divided into three
variants, STANet-Base, STANet-Bam, and STANet-Pam.

  * •

TINY-CD [15] employs the Siamese U-Net architecture and a new feature mixing
method to optimally utilize low-level information for spatial and temporal
domains, while also offering a new spatial-semantic attention mechanism via
its Mix and Attention Mask Block (MAMB).

###  IV-D Main Results

On the DSIFN-CD and S2Looking datasets, we compared the outcomes of our
proposed BD-MSA with previous SOTA approaches in table I. The top, second
best, and third best performers in each evaluation metric are shown in red,
blue, and bolded black, respectively. The results reveal that our proposed BD-
MSA outperforms the second-best model ChangerEx-MiT on the DSIFN-CD dataset,
with an F1 score, IoU of 83.98% and 72.38%, respectively, which is 3.11% and
4.49% higher. Our suggested BD-MSA achieves an F1 score, IoU of 64.08% and
47.17% on the S2Looking dataset, which is 2.1% and 2.23% higher than the
second-best model IFNet. The results demonstrate that our proposed BD-MSA
performs well in the field of RSCD. In the column of #Param (M), we can see
that our Proposed BD-MSA has a modest number of parameters, which is 3.465M;
while this indication is not the smallest, it is a comparatively small number
of parameters compared to many other techniques.

TABLE I: Comparison of our proposed BD-MSA with other SOTA methods on DSIFN-CD and S2Looking datasets. We use different colors to indicate: best, second best, and third best. Method | Backbone | #Param (M) | Flops | DSIFN-CD | S2Looking
---|---|---|---|---|---
F1 | Prec. | Rec. | IoU | F1 | Prec. | Rec. | IoU
FC-EF [10] | - | 1.353 | 12.976 | 63.44 | 75.97 | 54.47 | 46.46 | 7.65 | 81.36 | 8.95 | 8.77
FC-Siam-Di [10] | - | 1.352 | 17.54 | 63.41 | 73.23 | 55.92 | 46.43 | 13.19 | 83.29 | 15.76 | 15.28
FC-Siam-Conc [10] | - | 1.548 | 19.956 | 67.68 | 66.83 | 68.56 | 51.15 | 13.54 | 68.27 | 18.52 | 17.05
BIT [25] | ResNet18 | 2.99 | 34.996 | 71.04 | 77.22 | 65.78 | 55.09 | 44.51 | 67.41 | 33.23 | 28.63
ChangeFormer [26] | MiT-b1 | 3.847 | 11.38 | 80.23 | 84.4 | 76.46 | 66.99 | 60.92 | 75.79 | 50.93 | 43.8
ChangerEx-MiT [49] | MiT-b0 | 3.457 | 8.523 | 80.87 | 87.93 | 74.87 | 67.89 | 60.01 | 67.52 | 54.0 | 42.87
HANet [18] | - | 3.028 | 97.548 | 75.91 | 75.9 | 75.92 | 61.18 | 43.67 | 44.89 | 42.51 | 27.93
IFNet [12] | VGG-16 | 35.995 | 323.584 | 79.21 | 85.54 | 73.75 | 65.58 | 61.98 | 64.96 | 59.27 | 44.91
SNUNet [20] | - | 3.012 | 46.921 | 76.08 | 78.26 | 74.02 | 61.4 | 48.25 | 60.8 | 39.99 | 31.79
STANet-Base [19] | ResNet18 | 12.764 | 70.311 | 66.28 | 76.07 | 58.71 | 49.56 | 26.92 | 15.87 | 88.54 | 15.55
STANet-Bam [19] | ResNet18 | 12.846 | 391.168 | 61.48 | 70.83 | 54.31 | 44.39 | 27.27 | 16.11 | 88.68 | 15.79
STANet-Pam [19] | ResNet18 | 13.356 | 512 | 37.84 | 76.13 | 25.18 | 23.34 | 23.73 | 13.85 | 82.79 | 13.46
TINY-CD [15] | EfficientNet | 0.285 | 5.791 | 74.71 | 76.37 | 73.12 | 59.63 | 54.5 | 63.81 | 47.56 | 37.46
BD-MSA (Ours) | MiT-b0 | 3.465 | 12.658 | 83.98 | 88.01 | 80.3 | 72.38 | 64.08 | 70.44 | 58.77 | 47.14

We visualized the prediction results in the DSIFN-CD and S2Looking datasets to
compared the method of this research with other methods in prediction results,
as shown in Fig. 8. varied hues in the graphic represent the model’s varied
prediction results for each pixel during the prediction phase. Simply said,
the greater the proportion of white and black patches in the figure to the
total image, the better the model’s prediction outcome.

Figure 8: Comparative experimental visualization results for each model on the
DSIFN-CD and S2Looking test sets, which different colored regions denote FP,
FN, and TN, respectively, and where the white region is TP.

We specifically chose six photographs at random from each of DSIFN-CD and
S2Looking as a test, and it is evident that the method in this work
outperforms the other methods in terms of prediction outcomes. In Fig. 8(a),
(e), and (f), our proposed approach effectively mitigates misclassification
for non-changing regions when making predictions; see Fig. 8, other models’
predictions for the boundary of the changing regions are generally confusing
in (b), (c), and (d), however the model in this research solves the problem to
a degree. Although certain models, such as STANet-Pam, have fewer
mispredictions within the limits of the change region, they have a high missed
detection rate, implying that the model cannot identify the boundaries well.
The improvement of this paper’s model over other models, for S2Looking, is
mostly in the precision of modifying the region’s boundary and the effective
decrease of the adhesion phenomenon between buildings. Refer to Fig. 8, in
(j), (k), BD-MSA predicts the edges of changing zones more accurately; in (g),
(h), and (l), MGIG-BG successfully mitigates the adhesion phenomena between
buildings with more compact layouts.

To compared IoU as well as Params. between multiple models at the same time,
we plotted the color mapping for the test results of different models on both
datasets, as shown in Fig. 9.

Figure 9: Params. and IoU of different models on the two datasets, the top and
bottom parts of the figure show the evaluation results of each model on DSIFN-
CD and S2Looking, respectively.

Each point in the graphic represents a model, with the horizontal axis
representing the model’s parameters and the vertical axis representing the IoU
of each model on the two datasets.The closer the model is to the upper left
corner of the figure, the higher the accuracy detection, while the model takes
less arithmetic power. Our proposed BD-MSA may be seen in the upper left
corner, suggesting that the IoU reaches its maximum value and the number of
parameters is lower than in most models.

Furthermore, the preceding conclusions show that the model in this study
migrates better across devices than alternative models, particularly for
machines with weaker arithmetic capability.

###  IV-E Ablation Studies

We conduct ablation tests on OFAM, MixFFN, and the Decouple Module,
respectively, to validate the influence of different modules on our proposed
model.

The nomenclature of the models in the ablation experiments is as follows:

  * •

Baseline: MiT + FDAF + Predict layer.

  * •

BD-MSA-1-1: Baseline + MixFFN.

  * •

BD-MSA-1-2: Baseline + Decouple.

  * •

BD-MSA-1-3: Baseline + OFAM.

  * •

BD-MSA-2-1: Baseline + MixFFN + Decouple Module.

  * •

BD-MSA-2-2: Baseline + MixFFN + OFAM.

  * •

BD-MSA-2-3: Baseline + Decouple Module + OFAM.

  * •

BD-MSA: Baseline + MixFFN + Decouple Module + OFAM.

The results of each ablation experiments are shown in Table II and III.

TABLE II: Results of ablation experiments on DSIFN-CD Test. We use different colors to indicate: best, second best, and third best. Module | +MixFFN | +Decouple | +OFAM | F1 | IoU
---|---|---|---|---|---
Baseline |  |  |  | 80.64 | 67.56
BD-MSA-1-1 | ✓ |  |  | 80.87 | 67.89
BD-MSA-1-2 |  | ✓ |  | 80.87 | 67.89
BD-MSA-1-3 |  |  | ✓ | 81.7 | 69.06
BD-MSA-2-1 | ✓ | ✓ |  | 81.04 | 68.12
BD-MSA-2-2 | ✓ |  | ✓ | 82.82 | 70.68
BD-MSA-2-3 |  | ✓ | ✓ | 82.77 | 70.6
BD-MSA | ✓ | ✓ | ✓ | 83.98 | 72.38

This show that adding each module improves the assessment metrics F1 and IoU
when compared to the baseline, with F1 being able to synthesize Prec. and Rec.
When only one module is added, adding OFAM results in the greatest improvement
in assessment metrics, which we assume is related to the fact that OFAM is
added to all four phases of the backbone.

TABLE III: Results of ablation experiments on S2Looking Test. We use different colors to indicate: best, second best, and third best. Module | +MixFFN | +Decouple | +OFAM | F1 | IoU
---|---|---|---|---|---
Baseline |  |  |  | 56.72 | 39.59
BD-MSA-1-1 | ✓ |  |  | 60.01 | 42.87
BD-MSA-1-2 |  | ✓ |  | 59.73 | 42.58
BD-MSA-1-3 |  |  | ✓ | 61.94 | 45.93
BD-MSA-2-1 | ✓ | ✓ |  | 63.27 | 46.27
BD-MSA-2-2 | ✓ |  | ✓ | 63.31 | 46.32
BD-MSA-2-3 |  | ✓ | ✓ | 63.29 | 46.28
BD-MSA | ✓ | ✓ | ✓ | 64.08 | 47.14

To visualize the outcomes of each module’s ablation experiments, we exhibit
its effect on the test set evaluation of DSIFN-CD and S2Looking in Fig. 10.
Although the prediction effect of each ablation experimental model for the bi-
temporal images prediction in Fig. 10 is mostly right. BD-MSA outperforms the
other models in predicting the edges of the change region. The parts of the
photography where BD-MSA outperforms the predictions of other models have been
highlighted in yellow boxes.

Figure 10: The results of ablation experiments for each model on the DSIFN-CD
and S2Looking test sets.

In addition to ablation experiments on different modules, we also perform
ablation studies on OFAM modules, specifically adding OFAM modules behind
different stages in the backbone, as shown in Tables IV and V. The results
show that adding OFAM modules to all stages of the backbone has the greatest
effect on the evaluation metrics, whereas OFAM-1 in the table is second-best
in each evaluation index, which we hypothesize is due to the fact that the
first stage of the backbone has the largest feature map, and the addition of
OFAM modules can effectively aggregating information in the feature map, thus
reducing the computational cost of the model.

TABLE IV: The different stages in backbone are followed by the results of the OFAM ablation experiments on the DSIFN-CD test sets. We use different colors to indicate: best, second best, and third best. Methods | stage1 | stage2 | stage3 | stage4 | F1 | IoU
---|---|---|---|---|---|---
OFAM-1 | ✓ |  |  |  | 82.6 | 70.36
OFAM-2 | ✓ | ✓ |  |  | 79.65 | 66.18
OFAM-3 | ✓ | ✓ | ✓ |  | 76.32 | 61.71
OFAM-4 | ✓ | ✓ | ✓ | ✓ | 83.98 | 72.38
TABLE V: The different stages in backbone are followed by the results of the OFAM ablation experiments on the S2Looking test sets. We use different colors to indicate: best, second best, and third best. Methods | stage1 | stage2 | stage3 | stage4 | F1 | IoU
---|---|---|---|---|---|---
OFAM-1 | ✓ |  |  |  | 63.66 | 46.7
OFAM-2 | ✓ | ✓ |  |  | 57.49 | 40.34
OFAM-3 | ✓ | ✓ | ✓ |  | 60.01 | 42.86
OFAM-4 | ✓ | ✓ | ✓ | ✓ | 64.08 | 47.14

Fig. 11 depicts the experimental outcomes of introducing OFAM behind various
phases of the backbone. In general, each model achieves better prediction
results, but BD-MSA outperforms the other models in the subtle aspects shown
in the figure with yellow boxes, such as edge detection, which is more
accurate and can separate buildings with tight layouts very well.

###  IV-F Feature Map Visualization

To investigate whether the modules in this paper’s model are able to aggregate
semantic information in the prediction process for bi-temporal images, we used
Grad-CAM [56] to view some of the feature layers in BD-MSA, and the results
are shown in Fig. 12.

From left to right, the figure is divided into five sections: the original bi-
temporal images, feature maps before and after OFAM for stage 1, feature maps
before and after MixFFN, boundary and body feature maps generated by Decouple
Module, and change labels.

The figure clearly shows that the OFAM Module can transfer the weight in the
feature map from the unimportant road part to the more important building
part; MixFFN can focus the features on the changing region while reducing the
weight of the non-changing region; and Decouple Module can effectively
decouple the feature map and extract the edge features.

Figure 11: Visualization of the results of ablation experiments on DSIFN-CD
and S2Looking test sets for different stages followed by OFAM in backbone.
Figure 12: Visualization of heat maps generated by some modules.

##  V Conclusion

In this study, we suggested a novel approach for RSCD called BD-MSA. In the
training and prediction phase, the approach can combine global and local
information in both channel and spatial dimensions, as well as decouple the
main body of the change region and the edges of the feature maps. The
experimental results suggest that the technique in this research outperforms
previous models on the public datasets DSIFN-CD and S2Looking in terms of SOTA
performance. We further demonstrate, through a series of ablation experiments,
that all modules in this study are superior to the baseline.

We will continue to investigate the following aspects in the future: 1) The
method in this paper has only been validated on two public datasets, DSIFN-CD
and S2Looking, and it will be validated on more public datasets in the future;
2) The method in this paper is essentially a supervised learning method, and
we hope to explore unsupervised learning methods for application to tasks such
as remote sensing image change detection and more domain migration in future
work.

## Acknowledgment

This research was supported by National Natural Science Foundations of China
(No. 42261078), the Jiangxi Provincial Key R&D Program (Grant
number20223BBE51030) and the Science and Technology Research Project of
Jiangxi Bureau of Geology(Grant number 2022JXDZKJKY08) and the Open Research
Fund of Key Laboratory of Mine Environmental Monitoring and Improving around
Poyang Lake of Ministry of Natural Resources(MEMI-2021-2022-31) and the
Graduate Innovative Special Fund Projects of Jiangxi Province(YC2023-S556).

## References

  * [1] A. Singh, “Review article digital change detection techniques using remotely-sensed data,” _International journal of remote sensing_ , vol. 10, no. 6, pp. 989–1003, 1989.
  * [2] I. Onur, D. Maktav, M. Sari, and N. Kemal Sönmez, “Change detection of land cover and land use using remote sensing and gis: a case study in kemer, turkey,” _International Journal of Remote Sensing_ , vol. 30, no. 7, pp. 1749–1757, 2009.
  * [3] R. E. Kennedy, P. A. Townsend, J. E. Gross, W. B. Cohen, P. Bolstad, Y. Wang, and P. Adams, “Remote sensing change detection tools for natural resource managers: Understanding concepts and tradeoffs in the design of landscape monitoring projects,” _Remote sensing of environment_ , vol. 113, no. 7, pp. 1382–1396, 2009.
  * [4] P. Du, S. Liu, P. Gamba, K. Tan, and J. Xia, “Fusion of difference images for change detection over urban areas,” _IEEE journal of selected topics in applied earth observations and remote sensing_ , vol. 5, no. 4, pp. 1076–1086, 2012.
  * [5] Z. Zheng, Y. Zhong, J. Wang, A. Ma, and L. Zhang, “Building damage assessment for rapid disaster response with a deep object-based semantic change detection framework: From natural disasters to man-made disasters,” _Remote Sensing of Environment_ , vol. 265, p. 112636, 2021.
  * [6] P. R. Coppin and M. E. Bauer, “Digital change detection in forest ecosystems with remote sensing imagery,” _Remote sensing reviews_ , vol. 13, no. 3-4, pp. 207–234, 1996.
  * [7] J. Deng, K. Wang, Y. Deng, and G. Qi, “Pca-based land-use change detection and analysis using multitemporal and multisensor satellite data,” _International Journal of Remote Sensing_ , vol. 29, no. 16, pp. 4823–4838, 2008.
  * [8] C. He, A. Wei, P. Shi, Q. Zhang, and Y. Zhao, “Detecting land-use/land-cover change in rural–urban fringe areas using extended change-vector analysis,” _International Journal of Applied Earth Observation and Geoinformation_ , vol. 13, no. 4, pp. 572–585, 2011.
  * [9] C. Wu, B. Du, and L. Zhang, “Slow feature analysis for change detection in multispectral imagery,” _IEEE Transactions on Geoscience and Remote Sensing_ , vol. 52, no. 5, pp. 2858–2874, 2013.
  * [10] R. C. Daudt, B. Le Saux, and A. Boulch, “Fully convolutional siamese networks for change detection,” in _2018 25th IEEE International Conference on Image Processing (ICIP)_. IEEE, 2018, pp. 4063–4067.
  * [11] D. Peng, Y. Zhang, and H. Guan, “End-to-end change detection for high resolution satellite images using improved unet++,” _Remote Sensing_ , vol. 11, no. 11, p. 1382, 2019.
  * [12] C. Zhang, P. Yue, D. Tapete, L. Jiang, B. Shangguan, L. Huang, and G. Liu, “A deeply supervised image fusion network for change detection in high resolution bi-temporal remote sensing images,” _ISPRS Journal of Photogrammetry and Remote Sensing_ , vol. 166, pp. 183–200, 2020.
  * [13] H. Chen, W. Li, and Z. Shi, “Adversarial instance augmentation for building change detection in remote sensing images,” _IEEE Transactions on Geoscience and Remote Sensing_ , vol. 60, pp. 1–16, 2021.
  * [14] Y. Liu, C. Pang, Z. Zhan, X. Zhang, and X. Yang, “Building change detection for remote sensing images using a dual-task constrained deep siamese convolutional network model,” _IEEE Geoscience and Remote Sensing Letters_ , vol. 18, no. 5, pp. 811–815, 2020.
  * [15] A. Codegoni, G. Lombardi, and A. Ferrari, “Tinycd: a (not so) deep learning model for change detection,” _Neural Computing and Applications_ , vol. 35, no. 11, pp. 8471–8486, 2023.
  * [16] Q. Guo, J. Zhang, S. Zhu, C. Zhong, and Y. Zhang, “Deep multiscale siamese network with parallel convolutional structure and self-attention for change detection,” _IEEE Transactions on Geoscience and Remote Sensing_ , vol. 60, pp. 1–12, 2021.
  * [17] Q. Shi, M. Liu, S. Li, X. Liu, F. Wang, and L. Zhang, “A deeply supervised attention metric-based network and an open aerial image dataset for remote sensing change detection,” _IEEE transactions on geoscience and remote sensing_ , vol. 60, pp. 1–16, 2021.
  * [18] C. Han, C. Wu, H. Guo, M. Hu, and H. Chen, “Hanet: A hierarchical attention network for change detection with bi-temporal very-high-resolution remote sensing images,” _IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing_ , 2023.
  * [19] H. Chen and Z. Shi, “A spatial-temporal attention-based method and a new dataset for remote sensing image change detection,” _Remote Sensing_ , vol. 12, no. 10, p. 1662, 2020.
  * [20] S. Fang, K. Li, J. Shao, and Z. Li, “Snunet-cd: A densely connected siamese network for change detection of vhr images,” _IEEE Geoscience and Remote Sensing Letters_ , vol. 19, pp. 1–5, 2021.
  * [21] D. Wang, X. Chen, M. Jiang, S. Du, B. Xu, and J. Wang, “Ads-net: An attention-based deeply supervised network for remote sensing image change detection,” _International Journal of Applied Earth Observation and Geoinformation_ , vol. 101, p. 102348, 2021.
  * [22] Z. Li, C. Yan, Y. Sun, and Q. Xin, “A densely attentive refinement network for change detection based on very-high-resolution bitemporal remote sensing images,” _IEEE Transactions on Geoscience and Remote Sensing_ , vol. 60, pp. 1–18, 2022.
  * [23] M. Liu, Q. Shi, A. Marinoni, D. He, X. Liu, and L. Zhang, “Super-resolution-based change detection network with stacked attention module for images with different resolutions,” _IEEE Transactions on Geoscience and Remote Sensing_ , vol. 60, pp. 1–18, 2021.
  * [24] Z. Li, C. Tang, L. Wang, and A. Y. Zomaya, “Remote sensing change detection via temporal feature interaction and guided refinement,” _IEEE Transactions on Geoscience and Remote Sensing_ , vol. 60, pp. 1–11, 2022.
  * [25] H. Chen, Z. Qi, and Z. Shi, “Remote sensing image change detection with transformers,” _IEEE Transactions on Geoscience and Remote Sensing_ , vol. 60, pp. 1–14, 2021.
  * [26] W. G. C. Bandara and V. M. Patel, “A transformer-based siamese network for change detection,” in _IGARSS 2022-2022 IEEE International Geoscience and Remote Sensing Symposium_. IEEE, 2022, pp. 207–210.
  * [27] D. Wang, J. Zhang, B. Du, G.-S. Xia, and D. Tao, “An empirical study of remote sensing pretraining,” _IEEE Transactions on Geoscience and Remote Sensing_ , 2022.
  * [28] C. Zhang, L. Wang, S. Cheng, and Y. Li, “Swinsunet: Pure transformer network for remote sensing image change detection,” _IEEE Transactions on Geoscience and Remote Sensing_ , vol. 60, pp. 1–13, 2022.
  * [29] W. Wang, X. Tan, P. Zhang, and X. Wang, “A cbam based multiscale transformer fusion approach for remote sensing image change detection,” _IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing_ , vol. 15, pp. 6817–6825, 2022.
  * [30] Q. Li, R. Zhong, X. Du, and Y. Du, “Transunetcd: A hybrid transformer network for change detection in optical remote-sensing images,” _IEEE Transactions on Geoscience and Remote Sensing_ , vol. 60, pp. 1–19, 2022.
  * [31] X. Song, Z. Hua, and J. Li, “Remote sensing image change detection transformer network based on dual-feature mixed attention,” _IEEE Transactions on Geoscience and Remote Sensing_ , vol. 60, pp. 1–16, 2022.
  * [32] T. Yan, Z. Wan, and P. Zhang, “Fully transformer network for change detection of remote sensing images,” in _Proceedings of the Asian Conference on Computer Vision_ , 2022, pp. 1691–1708.
  * [33] W. Liu, Y. Lin, W. Liu, Y. Yu, and J. Li, “An attention-based multiscale transformer network for remote sensing image change detection,” _ISPRS Journal of Photogrammetry and Remote Sensing_ , vol. 202, pp. 599–609, 2023.
  * [34] Q. Ke and P. Zhang, “Hybrid-transcd: A hybrid transformer remote sensing image change detection network via token aggregation,” _ISPRS International Journal of Geo-Information_ , vol. 11, no. 4, p. 263, 2022.
  * [35] S. S. Islam, S. Rahman, M. M. Rahman, E. K. Dey, and M. Shoyaib, “Application of deep learning to computer vision: A comprehensive study,” in _2016 5th international conference on informatics, electronics and vision (ICIEV)_. IEEE, 2016, pp. 592–597.
  * [36] E. Maggiori, Y. Tarabalka, G. Charpiat, and P. Alliez, “Convolutional neural networks for large-scale remote-sensing image classification,” _IEEE Transactions on geoscience and remote sensing_ , vol. 55, no. 2, pp. 645–657, 2016\.
  * [37] G. Cheng and J. Han, “A survey on object detection in optical remote sensing images,” _ISPRS journal of photogrammetry and remote sensing_ , vol. 117, pp. 11–28, 2016.
  * [38] Z. Deng, H. Sun, S. Zhou, J. Zhao, L. Lei, and H. Zou, “Multi-scale object detection in remote sensing imagery with convolutional neural networks,” _ISPRS journal of photogrammetry and remote sensing_ , vol. 145, pp. 3–22, 2018.
  * [39] R. Kemker, C. Salvaggio, and C. Kanan, “Algorithms for semantic segmentation of multispectral remote sensing imagery using deep learning,” _ISPRS journal of photogrammetry and remote sensing_ , vol. 145, pp. 60–77, 2018.
  * [40] X. Yuan, J. Shi, and L. Gu, “A review of deep learning methods for semantic segmentation of remote sensing imagery,” _Expert Systems with Applications_ , vol. 169, p. 114417, 2021.
  * [41] R. Zhang, H. Zhang, X. Ning, X. Huang, J. Wang, and W. Cui, “Global-aware siamese network for change detection on remote sensing images,” _ISPRS journal of photogrammetry and remote sensing_ , 2023.
  * [42] Z. Zhou, M. M. Rahman Siddiquee, N. Tajbakhsh, and J. Liang, “Unet++: A nested u-net architecture for medical image segmentation,” in _Deep Learning in Medical Image Analysis and Multimodal Learning for Clinical Decision Support: 4th International Workshop, DLMIA 2018, and 8th International Workshop, ML-CDS 2018, Held in Conjunction with MICCAI 2018, Granada, Spain, September 20, 2018, Proceedings 4_. Springer, 2018, pp. 3–11.
  * [43] A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, Ł. Kaiser, and I. Polosukhin, “Attention is all you need,” _Advances in neural information processing systems_ , vol. 30, 2017.
  * [44] S. Woo, J. Park, J.-Y. Lee, and I. S. Kweon, “Cbam: Convolutional block attention module,” in _Proceedings of the European conference on computer vision (ECCV)_ , 2018, pp. 3–19.
  * [45] J. Hu, L. Shen, and G. Sun, “Squeeze-and-excitation networks,” in _Proceedings of the IEEE conference on computer vision and pattern recognition_ , 2018, pp. 7132–7141.
  * [46] Y. Li, T. Yao, Y. Pan, and T. Mei, “Contextual transformer networks for visual recognition,” _IEEE Transactions on Pattern Analysis and Machine Intelligence_ , vol. 45, no. 2, pp. 1489–1500, 2022.
  * [47] E. Xie, W. Wang, Z. Yu, A. Anandkumar, J. M. Alvarez, and P. Luo, “Segformer: Simple and efficient design for semantic segmentation with transformers,” _Advances in Neural Information Processing Systems_ , vol. 34, pp. 12 077–12 090, 2021.
  * [48] D. Hendrycks and K. Gimpel, “Gaussian error linear units (gelus),” _arXiv preprint arXiv:1606.08415_ , 2016.
  * [49] S. Fang, K. Li, and Z. Li, “Changer: Feature interaction is what you need for change detection,” _IEEE Transactions on Geoscience and Remote Sensing_ , 2023.
  * [50] X. Huang and S. Belongie, “Arbitrary style transfer in real-time with adaptive instance normalization,” in _Proceedings of the IEEE international conference on computer vision_ , 2017, pp. 1501–1510.
  * [51] Y.-H. Tsai, M.-H. Yang, and M. J. Black, “Video segmentation via object flow,” in _Proceedings of the IEEE conference on computer vision and pattern recognition_ , 2016, pp. 3899–3908.
  * [52] L. Shen, Y. Lu, H. Chen, H. Wei, D. Xie, J. Yue, R. Chen, S. Lv, and B. Jiang, “S2looking: A satellite side-looking dataset for building change detection,” _Remote Sensing_ , vol. 13, no. 24, p. 5094, 2021.
  * [53] M. Contributors, “MMCV: OpenMMLab computer vision foundation,” https://github.com/open-mmlab/mmcv, 2018.
  * [54] J. Long, E. Shelhamer, and T. Darrell, “Fully convolutional networks for semantic segmentation,” in _Proceedings of the IEEE conference on computer vision and pattern recognition_ , 2015, pp. 3431–3440.
  * [55] O. Ronneberger, P. Fischer, and T. Brox, “U-net: Convolutional networks for biomedical image segmentation,” in _Medical Image Computing and Computer-Assisted Intervention–MICCAI 2015: 18th International Conference, Munich, Germany, October 5-9, 2015, Proceedings, Part III 18_. Springer, 2015, pp. 234–241.
  * [56] R. R. Selvaraju, M. Cogswell, A. Das, R. Vedantam, D. Parikh, and D. Batra, “Grad-cam: Visual explanations from deep networks via gradient-based localization,” in _Proceedings of the IEEE international conference on computer vision_ , 2017, pp. 618–626.