$V^{{\ydiagram{1,1,1,1}}\otimes{\ydiagram{1,1}}}$ onto the representation in
the superscript, e.g.
$V^{{\ydiagram{2,2,1,1}}}=\mathcal{Y}^{GL}_{{\ydiagram{2,2,1,1}}}(V^{{\ydiagram{1,1,1,1}}\otimes{\ydiagram{1,1}}})$.
Contraction with the Riemann tensor then immediately gives

$\Omega[V^{{\ydiagram{1,1,1,1}}\otimes{\ydiagram{1,1}}}]_{\mu\nu}=R^{\alpha\beta\gamma\delta}(V^{{\ydiagram{1,1,1,1,1,1}}}_{\mu\nu\alpha\beta\gamma\delta}+V^{{\ydiagram{2,1,1,1,1}}}_{\mu\nu\alpha\beta\gamma|\delta}+V^{{\ydiagram{2,2,1,1}}}_{\mu\nu\alpha\beta|\gamma\delta})=R^{\alpha\beta\gamma\delta}V^{{\ydiagram{2,2,1,1}}}_{\mu\nu\alpha\beta|\gamma\delta}$
(216)

where the first two terms in the parentheses vanish upon contraction with the
Riemann tensor from $R_{[\alpha\beta\gamma]\delta}=0$.

The second of the three tensor products on the right-hand side of eq. (213)
decomposes as

$\ydiagram{2,1,1}\otimes\ydiagram{1,1}=\ydiagram{2,1,1,1,1}\oplus\ydiagram{2,2,1,1}\oplus\ydiagram{2,2,2}\oplus\ydiagram{3,2,1}\oplus\ydiagram{3,1,1,1}$
(217)

In a similar manner to eq. (216), some of the representations on the right-
hand side of eq. (217) give $\Omega[V]=0$ identically from the Bianchi
identity. In particular, we can write the decomposition (217) in terms of the
tensors $V$ as

$V^{{\ydiagram{2,1,1}}\otimes{\ydiagram{1,1}}}_{\mu\nu\alpha|\beta|\gamma\delta}=U^{{\ydiagram{2,1,1,1,1}}}_{\mu\nu\alpha\gamma\delta|\beta}+U^{{\ydiagram{2,2,1,1}}}_{\mu\nu\alpha[\gamma|\delta]\beta}+U^{{\ydiagram{2,2,2}}}_{\mu\nu\alpha|\beta\gamma\delta}+U^{{\ydiagram{3,2,1}}}_{\mu\nu\alpha|\beta[\gamma|\delta]}+\bigg{(}\frac{1}{2}U^{{\ydiagram{3,1,1,1}}}_{\mu\nu\alpha\gamma|\beta|\delta}-(\gamma\leftrightarrow\delta)\bigg{)}$
(218)

where, as before, each term on the right-hand side is the Young projection of
$V^{{\ydiagram{2,1,1}}\otimes{\ydiagram{1,1}}}$ onto the irreducible
$GL(d,\mathbb{R})$ representation indicated in the superscript. Contracting
with $R^{\alpha\beta\gamma\delta}$ to form $\Omega[V]_{\mu\nu}$, we
immediately see that the ${\ydiagram{2,1,1,1,1}}$ and ${\ydiagram{2,2,2}}$
representations give a vanishing result from the Bianchi identity
$R_{[\alpha\beta\gamma]\delta}=0$. Furthermore, the ${\ydiagram{3,1,1,1}}$
representation yields a vanishing result because

$R^{\alpha\beta\gamma\delta}V^{{\ydiagram{3,1,1,1}}}_{\mu\nu\alpha\gamma|\beta|\delta}=\frac{1}{2}R^{\alpha\gamma\beta\delta}V^{{\ydiagram{3,1,1,1}}}_{\mu\nu\alpha\gamma|[\beta|\delta]}=0$
(219)

where we have used $R_{[\alpha\beta\gamma]\delta}=0$ in the first equality and
$V^{{\ydiagram{3,1,1,1}}}_{\mu\nu\alpha\gamma|[\beta|\delta]}=0$ in the
second, which is a criterion for the irreducibility of this representation.
Therefore, only the ${\ydiagram{2,2,1,1}}$ and ${\ydiagram{3,2,1}}$
representations in eq. (218) give non-vanishing $\Omega[V]$.

Finally, the third tensor product on the right-hand side of eq. (213)
decomposes as

$\ydiagram{2,2}\otimes\ydiagram{1,1}=\ydiagram{2,2,1,1}\oplus\ydiagram{3,2,1}\oplus\ydiagram{3,3}$
(220)

Again, in terms of the tensor $V$, this decomposition can be written

$V^{{\ydiagram{2,2}}\otimes{\ydiagram{1,1}}}_{\mu\nu|\alpha\beta|\gamma\delta}=W^{{\ydiagram{2,2,1,1}}}_{\mu\nu\gamma\delta|\alpha\beta}+\bigg{(}\frac{1}{2}(W^{{\ydiagram{3,2,1}}}_{\mu\nu\gamma|\alpha\beta|\delta}-(\gamma\leftrightarrow\delta)\bigg{)}+W^{{\ydiagram{3,3}}}_{\mu\nu|\alpha\beta|\gamma\delta}$
(221)

where each term on the right-hand side is the Young projection of
$V^{{\ydiagram{2,2}}\otimes{\ydiagram{1,1}}}$ onto the irreducible
$GL(d,\mathbb{R})$ representation labelled. All of the terms on the right-hand
side can contribute to non-zero $\Omega[V]$.

Combining the results from the three tensor products on the right-hand side of
eq. (213), we can write the most general $V$ which contributes to non-zero
$\Omega[V]$ as

$\displaystyle\begin{split}V_{\mu\nu|\alpha\beta|\gamma\delta}&=V^{{\ydiagram{2,2,1,1}}}_{\mu\nu\alpha\beta|\gamma\delta}+U^{{\ydiagram{2,2,1,1}}}_{\mu\nu\alpha[\gamma|\delta]\beta}+U^{{\ydiagram{3,2,1}}}_{\mu\nu[\alpha|\beta][\gamma|\delta]}\\\
&\quad+W^{{\ydiagram{2,2,1,1}}}_{\mu\nu\gamma\delta|\alpha\beta}+\bigg{(}\frac{1}{2}W^{{\ydiagram{3,2,1}}}_{\mu\nu\gamma|\alpha\beta|\delta}-(\gamma\leftrightarrow\delta)\bigg{)}+W^{{\ydiagram{3,3}}}_{\mu\nu|\alpha\beta|\gamma\delta}\end{split}$
(222)

Note that there are three tensors in the ${\ydiagram{2,2,1,1}}$ representation
with different index placements. This reflects the fact that this
representation arises with multiplicity three in the full decomposition of
${\ydiagram{1,1}}\otimes{\ydiagram{1,1}}\otimes{\ydiagram{1,1}}$ in eq. (213);
that is, it appears once in each of the three tensor products on the right-
hand side of eq. (213) (see eqs. (214), (217) and (220)). Similarly, the
${\ydiagram{3,2,1}}$ representation appears twice.

However, while these representations appear with multiple different index
placements in the decomposition of $V$, the properties of the Riemann tensor
imply that they all contribute in the same way to $\Omega[V]$. Indeed,
contracting eq. (222) with $R^{\alpha\beta\gamma\delta}$ and using the
identities $R_{[\alpha\beta\gamma]\delta}=0$ and
$R_{\alpha\beta\gamma\delta}=R_{\gamma\delta\alpha\beta}$ we find

$\begin{split}\Omega[V]_{\mu\nu}&=R^{\alpha\beta\gamma\delta}\bigg{(}V^{{\ydiagram{2,2,1,1}}}_{\mu\nu\alpha\beta|\gamma\delta}-\frac{1}{2}U^{{\ydiagram{2,2,1,1}}}_{\mu\nu\alpha\beta|\gamma\delta}+W^{{\ydiagram{2,2,1,1}}}_{\mu\nu\alpha\beta|\gamma\delta}\\\
&\qquad\qquad-\frac{1}{2}U^{{\ydiagram{3,2,1}}}_{\mu\nu\alpha|\beta\gamma|\delta}+W^{{\ydiagram{3,2,1}}}_{\mu\nu\alpha|\beta\gamma|\delta}+W^{{\ydiagram{3,3}}}_{\mu\nu|\alpha\beta|\gamma\delta}\bigg{)}\end{split}$
(223)

This demonstrates that all three terms in the ${\ydiagram{2,2,1,1}}$
representation contribute to $\Omega[V]$ in the same way. Similarly, both the
${\ydiagram{3,2,1}}$ terms contribute in the same way. Therefore, since we are
interested in classifying the possible conserved 2-forms $\Omega[V]$, we can
simply choose one of the index permutations for the ${\ydiagram{2,2,1,1}}$ and
${\ydiagram{3,2,1}}$ representations to include in $V$. This will still
produce all the possible 2-forms $\Omega[V]$. Therefore, without loss of
generality, we are free to set

$V_{\mu\nu|\alpha\beta|\gamma\delta}=V^{{\ydiagram{2,2,1,1}}}_{\mu\nu\alpha\beta|\gamma\delta}+U^{{\ydiagram{3,2,1}}}_{\mu\nu[\alpha|\beta][\gamma|\delta]}+W^{{\ydiagram{3,3}}}_{\mu\nu|\alpha\beta|\gamma\delta}$
(224)

##### Vanishing of first term of eq. (211).

We now find the constraints on $V$ in order for the first term in eq. (211) to
vanish, as required for $\Omega[V]$ to be conserved. We have

$\partial^{\mu}R^{\alpha\beta\gamma\delta}V_{\mu\nu|\alpha\beta|\gamma\delta}=\partial^{\mu}R^{\alpha\beta\gamma\delta}(V^{{\ydiagram{2,2,1,1}}}_{\mu\nu\alpha\beta|\gamma\delta}+U^{{\ydiagram{3,2,1}}}_{\mu\nu\alpha|\beta\gamma|\delta}+W^{{\ydiagram{3,3}}}_{\mu\nu|\alpha\beta|\gamma\delta})$
(225)

The ${\ydiagram{2,2,1,1}}$ contribution projects onto the differential Bianchi
identity $\partial^{[\mu}R^{\alpha\beta]\gamma\delta}=0$ and so vanishes. The
other two terms, however, do not vanish in this manner. Instead, we should
make use of the field equation $R_{\mu\nu}=0$, as well as the Bianchi
identities in eq. (212). In order to allow for the use of the field equation,
we view the irreducible $GL(d,\mathbb{R})$ representations as reducible
$SO(1,d-1)$ representations. They are reducible because $SO(1,d-1)$ has the
additional invariant tensor $\eta_{\mu\nu}$. We can decompose the
${\ydiagram{3,2,1}}$ and ${\ydiagram{3,3}}$ $GL(d,\mathbb{R}$) representations
into $SO(1,d-1)$ representations by removing all possible traces. This
introduces various factors of the $SO(1,d-1)$-invariant metric $\eta_{\mu\nu}$
which can contract with the Riemann tensor to give the Ricci tensor.

We begin with the ${\ydiagram{3,2,1}}$ representation, we have a decomposition

$\begin{split}{\ydiagram{3,2,1}}_{GL}&=\left(\;{\ydiagram{3,2,1}}\oplus\;{\ydiagram{3,1}}\oplus\;{\ydiagram{2,1,1}}\oplus
3\;{\ydiagram{2}}\oplus 2\;{\ydiagram{1,1}}\oplus
3\;\bullet\right)_{SO}\end{split}$ (226)

where $\bullet$ represents the trivial $\mathbf{1}$ representation. The
degeneracies of the latter terms come from the fact that there are multiple
ways to remove a trace from a tableau with more than two columns. Recall that
the irreducible $SO(1,d-1)$ representations are traceless. The decomposition
of $U^{{\ydiagram{3,2,1}}}$ can then be written

$\begin{split}U^{{\ydiagram{3,2,1}}}_{\mu\nu\alpha|\beta\gamma|\delta}&=u^{{\ydiagram{3,2,1}}}_{\mu\nu\alpha|\beta\gamma|\delta}+\bigg{(}\frac{1}{2}\eta_{\beta[\mu}u^{{\ydiagram{3,1}}}_{\nu\alpha]|\gamma|\delta}-(\beta\leftrightarrow\gamma)\bigg{)}+u^{{\ydiagram{2,1,1}}}_{\mu\nu\alpha|[\beta}\eta_{\gamma]\delta}+\eta_{\delta[\mu}u^{{\ydiagram{2,2}}}_{\nu\alpha]|\beta\gamma}\\\
&\quad+\bigg{(}\frac{1}{2}\eta_{\beta[\mu|}\eta_{\gamma|\nu}u^{{\ydiagram{2}}}_{\alpha]|\delta}-(\beta\leftrightarrow\gamma)\bigg{)}+\eta_{\delta[\beta}\eta_{\gamma][\mu}u^{{\ydiagram{1,1}}}_{\nu\alpha]}+\bigg{(}\frac{1}{2}\eta_{\delta[\mu|}\eta_{\beta|\nu}u^{\prime\,{\ydiagram{2}}}_{\alpha]|\gamma}-(\beta\leftrightarrow\gamma)\bigg{)}\\\
&\quad+u^{\prime\,{\ydiagram{1,1}}}_{[\mu\nu}\eta_{\alpha][\beta}\eta_{\gamma]\delta}+\bigg{(}\frac{1}{2}\eta_{\delta[\mu|}\eta_{\beta|\nu}u^{\prime\prime\,{\ydiagram{2}}}_{\alpha]|\gamma}-(\beta\leftrightarrow\gamma)\bigg{)}+\bigg{(}\frac{1}{2}\eta_{\beta[\mu|}\eta_{\gamma|\nu}\eta_{\alpha]\delta}u^{\bullet}-(\beta\leftrightarrow\gamma)\bigg{)}\\\
&\quad+\bigg{(}\frac{1}{2}\eta_{\delta[\mu|}\eta_{\beta|\nu}\eta_{\alpha]\gamma}u^{\prime\,\bullet}-(\beta\leftrightarrow\gamma)\bigg{)}+\bigg{(}\frac{1}{2}\eta_{\delta[\mu|}\eta_{\beta|\nu}\eta_{\alpha]\gamma}u^{\prime\prime\,\bullet}-(\beta\leftrightarrow\gamma)\bigg{)}\end{split}$
(227)

where all the $u$ tensors are traceless. We will not give explicit formulae
for the $u$ tensors, but they are constructed by Young projection of
$U^{{\ydiagram{3,2,1}}}$ onto the irreducible $SO(1,d-1)$ representations. In
particular, each $u$ is traceless. Schematically, each $u$ tensor contains
some number of traces of $U^{{\ydiagram{3,2,1}}}$. This is a generalisation of
the standard decomposition of a symmetric 2-tensor into irreducible
$SO(1,d-1)$ tensors:
$A_{\mu\nu}=(A_{\mu\nu}-\frac{1}{D}\eta_{\mu\nu}A\indices{{}_{\sigma}^{\sigma}})+\frac{1}{D}\eta_{\mu\nu}A\indices{{}_{\sigma}^{\sigma}}$.

Note that there are three $u$ tensors in the ${\ydiagram{2}}$ representation
which we denote separately by $u^{{\ydiagram{2}}}$,
$u^{\prime\,{\ydiagram{2}}}$ and $u^{\prime\prime\,{\ydiagram{2}}}$. This
reflects the three ways in which this representation can arise when two traces
are removed from the ${\ydiagram{3,2,1}}$ representation. Indeed, this gives
the multiplicity in eq. (226). Similarly, there are two terms in the
${\ydiagram{1,1}}$ representation and three terms in the trivial $\bullet$
representation.

We first note that not all of the irreducible $SO(1,d-1)$ representations will
contribute to $\Omega[V]$. Namely, the ${\ydiagram{2,1,1}}$, ${\ydiagram{2}}$,
${\ydiagram{1,1}}$, and $\bullet$ representations vanish by $R_{\mu\nu}=0$
when contracted with $R^{\alpha\beta\gamma\delta}$ and so give $\Omega[V]=0$.
Thus, the only $SO(1,d-1)$ representations which potentially contribute to
non-trivial $\Omega[V]$ are ${\ydiagram{3,2,1}}$, ${\ydiagram{3,1}}$ and
${\ydiagram{2,2}}$, so we restrict to

$U^{{\ydiagram{3,2,1}}}_{\mu\nu\alpha|\beta\gamma|\delta}=u^{{\ydiagram{3,2,1}}}_{\mu\nu\alpha|\beta\gamma|\delta}+\bigg{(}\frac{1}{2}\eta_{\beta[\mu}u^{{\ydiagram{3,1}}}_{\nu\alpha]|\gamma|\delta}-(\beta\leftrightarrow\gamma)\bigg{)}+\eta_{\delta[\mu}u^{{\ydiagram{2,2}}}_{\nu\alpha]|\beta\gamma}$
(228)

The goal of the decomposition into irreducible $SO(1,d-1)$ representations was
to allow for the first term of eq. (211) to vanish by the field equation
$R_{\mu\nu}=0$. However, when we contract eq. (228) with
$\partial^{\mu}R^{\alpha\beta\gamma\delta}$, none of the terms vanish by using
$R_{\mu\nu}=0$ or the Bianchi identities in eq. (212). Therefore, these
representations cannot be allowed in $V$ if we want $\Omega[V]$ to be
conserved. Therefore, the irreducible $GL(d,\mathbb{R})$ representation
${\ydiagram{3,2,1}}$ does not give rise to any non-zero conserved 2-forms
$\Omega[V]$.

We now proceed to the ${\ydiagram{3,3}}$ $GL(d)$ representation in eq. (225),
which decomposes into irreducible $SO(1,d-1)$ representations as

${\ydiagram{3,3}}_{\,GL}=\left(\;{\ydiagram{3,3}}\oplus{\ydiagram{3,1}}\oplus{\ydiagram{1,1}}\oplus{\ydiagram{2}}\oplus\bullet\right)_{SO}$
(229)

Note that, while one may naïvely think that there are multiple traces which
can be removed from the ${\ydiagram{3,3}}$ representation, they are not
independent. This follows from the irreducibility of the ${\ydiagram{3,3}}$
representation, which requires that $W^{{\ydiagram{3,3}}}$ is invariant under
permutations of the three columns. Therefore, it is equivalent to take a trace
between any two columns of this representation. This is why there are no
representations in the decomposition (229) with multiplicity greater than one.

One can then expand the $GL(d,\mathbb{R})$ tensor $W^{{\ydiagram{3,3}}}$
according to the decomposition (229), in a similar manner to eq. (227).

$\begin{split}{W^{{\ydiagram{3,3}}}_{\mu\nu|\alpha\beta|}}^{\gamma\delta}&={w^{{\ydiagram{3,3}}}_{\mu\nu|\alpha\beta|}}^{\gamma\delta}+{w^{{\ydiagram{3,1}}}_{\mu\nu|[\alpha|}}^{[\gamma}\delta_{\beta]}^{\delta]}+{w^{{\ydiagram{3,1}}}_{\alpha\beta|[\mu|}}^{[\gamma}\delta_{\nu]}^{\delta]}+\bigg{(}\frac{1}{2}{w^{{\ydiagram{3,1}}\,\gamma\delta}}_{\mu|[\alpha}\eta_{\beta]\nu}-(\mu\leftrightarrow\nu)\bigg{)}\\\
&\quad+w^{{\ydiagram{1,1}}}_{\mu\nu}\delta_{[\alpha}^{[\gamma}\delta_{\beta]}^{\delta]}+w^{{\ydiagram{1,1}}}_{\alpha\beta}\delta_{[\mu}^{[\gamma}\delta_{\nu]}^{\delta]}+\bigg{(}\frac{1}{2}w^{{\ydiagram{1,1}}\gamma\delta}\eta_{\alpha[\mu}\eta_{\nu]\beta}-(\alpha\leftrightarrow\beta)\bigg{)}\\\
&\quad+{w^{{\ydiagram{2}}}_{[\mu|}}^{[\gamma}\eta_{\nu][\alpha}\delta_{\beta]}^{\delta]}+\delta_{[\mu}^{[\gamma}w^{{\ydiagram{2}}}_{\nu]|[\alpha}\delta_{\beta]}^{\delta]}+{w^{{\ydiagram{2}}}_{[\alpha|}}^{[\gamma}\eta_{\beta][\mu}\delta_{\nu]}^{\delta]}+\bigg{(}\frac{1}{2}\eta_{\mu[\alpha}\delta_{\beta]}^{[\gamma}\delta^{\delta]}_{\nu}w^{\bullet}-(\mu\leftrightarrow\nu)\bigg{)}\end{split}$
(230)

where all the $w$ tensors are traceless and are constructed by Young
projection of $W^{{\ydiagram{3,3}}}$ onto the $SO(1,d-1)$ representations. We
have raised the final two indices using $\eta^{\mu\nu}$ in order to write the
anti-symmetrisations in a more compact form. The different terms corresponding
to the same Young tableau are necessary as the ${\ydiagram{3,3}}$
representation is symmetric under permutations of the three columns.

Again, we are only interested in those representations which contribute non-
trivially to $\Omega[V]$. By contracting eq. (230) with
$R^{\alpha\beta\gamma\delta}$, the ${\ydiagram{2}}$ and $\bullet$
representations give a vanishing result from $R_{\mu\nu}=0$. Therefore, the
$SO(1,d-1)$ representations which contribute to $\Omega[V]$ are
${\ydiagram{3,3}}$, ${\ydiagram{3,1}}$ and ${\ydiagram{1,1}}$.

We must determine which of these representations can yield a conserved 2-form
$\Omega[V]$, which requires that the first term of eq. (211) vanishes.
Contracting eq. (230) with $\partial^{\mu}R^{\alpha\beta\gamma\delta}$, the
${\ydiagram{3,3}}$ and ${\ydiagram{3,1}}$ representations do not give a
vanishing result. Therefore, they cannot appear in $V$. Therefore, the only
$SO(1,d-1)$ representation which gives a non-zero contribution to $\Omega[V]$
and also makes the first term in eq. (211) vanish is ${\ydiagram{1,1}}$.

In summary, we have shown that any $V$ in the ${\ydiagram{2,2,1,1}}_{GL}$
representation will give rise to a non-zero conserved 2-form $\Omega[V]$. The
only other representation which can contribute to a non-zero conserved
$\Omega[V]$ is the $SO(1,d-1)$ representation ${\ydiagram{1,1}}$ which is
contained in the decomposition of the $GL(d,\mathbb{R})$ representation
${\ydiagram{3,3}}$.

##### Representations giving independent $\Omega[V]$.

While the ${\ydiagram{1,1}}$ representation appearing in the decomposition
(230) can give rise to non-zero $\Omega[V]$, we now show that this $\Omega[V]$
can also be produced by a ${\ydiagram{2,2,1,1}}$ tensor $V$. Contracting eq.
(230) with the Riemann tensor, we find the 2-form $\Omega[V]$ corresponding to
the ${\ydiagram{1,1}}$ representation is

$\Omega[w^{{\ydiagram{1,1}}}]=2R_{\mu\nu\alpha\beta}w^{{\ydiagram{1,1}}\,\alpha\beta}$
(231)

which is of the form of the Penrose 2-form in eq. (1). However, we know from
section 2.3 that the Penrose 2-form can also be produced by a
${\ydiagram{2,2,1,1}}$ tensor $V$ given by eq. (32). It follows that the full
set of possible conserved 2-forms $\Omega[V]$ can be accounted for solely by
the ${\ydiagram{2,2,1,1}}$ $GL(d)$ representation. As emphasised throughout,
our intention is to classify the conserved 2-forms $\Omega[V]$, rather than
the tensors $V$ themselves. Therefore, for our purposes it is sufficient to
restrict $V$ to transform in this representation. As mentioned at the
beginning of this appendix, it remains to ensure that the second term in eq.
(117) vanishes in order for $\Omega[V]$ to be conserved. This puts a
differential condition on $V$. This is considered in the main text, and the
details can be found in Appendix C.

##  Appendix C Constraint on $V$ from representation theory

In this appendix we give a derivation of the constraint given in eq. (121) via
representation theory arguments. We require that $\Omega[V]$ is conserved for
some ${[4,2]}$-tensor $V$. From eq. (120), this requires

$0=\partial^{\mu}\Omega[V]_{\mu\nu}=R^{\alpha\beta\gamma\delta}\partial^{\mu}V_{\mu\nu\alpha\beta|\gamma\delta}$
(232)

which imposes a differential constraint on $V$. We begin with a
$GL(d,\mathbb{R})$ decomposition of the tensor product ${\lfloor 3,2\rfloor}$
representation in which $\differential_{L}^{\dagger}V$ transforms:

$\differential_{L}^{\dagger}V\in\ydiagram{1,1,1}\otimes\ydiagram{1,1}=\ydiagram{1,1,1,1,1}\oplus\ydiagram{2,1,1,1}\oplus\ydiagram{2,2,1}$
(233)

However, using the fact that $V$ transforms in the _irreducible_ ${[4,2]}$
$GL(d)$ representation, we have

$V_{[\mu\nu\alpha\beta|\gamma]\delta}=0\implies
V_{\mu[\nu\alpha\beta|\gamma\delta]}=0\implies\partial^{\mu}V_{\mu[\nu\alpha\beta|\gamma\delta]}=0$
(234)

such that the ${\ydiagram{1,1,1,1,1}}$ representation does not contribute to
$\differential_{L}^{\dagger}V$ in eq. (233). The remaining contributions to
$\differential_{L}^{\dagger}V$ can further be decomposed into $SO(1,d-1)$
representations as

$\differential_{L}^{\dagger}V\in\left(\;\ydiagram{2,1,1,1}\oplus\ydiagram{1,1,1}\;\right)_{SO}\oplus\left(\;\ydiagram{2,2,1}\oplus\ydiagram{2,1}\oplus\ydiagram{1}\;\right)_{SO}$
(235)

where the $SO(1,d-1)$ representations are traceless with respect to the
$SO(1,d-1)$-invariant metric $\eta_{\mu\nu}$. We write this decomposition
explicitly as

$\partial^{\mu}V_{\mu\nu\alpha\beta|\gamma\delta}=\frac{1}{2}\left(A^{{\ydiagram{2,1,1,1}}}_{\nu\alpha\beta\gamma|\delta}+\eta_{\gamma[\nu}B^{{\ydiagram{1,1,1}}}_{\alpha\beta]\delta}+C^{{\ydiagram{2,2,1}}}_{\nu\alpha\beta|\gamma\delta}+\eta_{\gamma[\nu}D^{{\ydiagram{2,1}}}_{\alpha\beta]|\delta}+\eta_{\gamma[\nu}E^{{\ydiagram{1}}}_{\alpha}\eta_{\beta]\delta}\right)-(\gamma\leftrightarrow\delta)$
(236)

where the tensors $A$, $B$, $C$, $D$ and $E$ are in the corresponding
representations in eq. (235) (in particular, they are traceless). Inserting
this decomposition into eq. (232) and using the field equation and Bianchi
identity (116), we find that the $A$, $B$ and $E$ contributions vanish,
leaving

$\partial^{\mu}\Omega[V]_{\mu\nu}=R^{\alpha\beta\gamma\delta}C^{{\ydiagram{2,2,1}}}_{\nu\alpha\beta|\gamma\delta}+\frac{1}{3}R\indices{{}^{\alpha\beta}_{\nu}{}^{\delta}}D^{{\ydiagram{2,1}}}_{\alpha\beta|\delta}$
(237)

This vanishes for all field configurations $h_{\mu\nu}$ only if $C=D=0$. That
is, these representations cannot appear in $\differential^{\dagger}_{L}V$ if
$\Omega[V]$ is to be conserved. Therefore, the allowed $SO(1,d-1)$
representations in the decomposition of $\differential_{L}^{\dagger}V$ are

$\differential_{L}^{\dagger}V\in\ydiagram{2,1,1,1}\oplus\ydiagram{1,1,1}\oplus\ydiagram{1}$
(238)

This then yields precisely the constraint which $V$ must satisfy in eq. (121).

## References

  * (1) L.F. Abbott and S. Deser, _Stability of Gravity with a Cosmological Constant_ , _Nuclear Physics B_ 195 (1982) 76.
  * (2) C.M. Hull, _Magnetic Charges for the Graviton_ , 2310.18441.
  * (3) C.M. Hull, _Strongly coupled gravity and duality_ , _Nucl. Phys. B_ 583 (2000) 237 [hep-th/0004195].
  * (4) C.M. Hull, _Duality in gravity and higher spin gauge fields_ , _JHEP_ 09 (2001) 027 [hep-th/0107149].
  * (5) R. Penrose, _Quasilocal mass and angular momentum in general relativity_ , _Proc. Roy. Soc. Lond. A_ 381 (1982) 53.
  * (6) C. Hull, M.L. Hutt and U. Lindström, _Charges and topology in linearised gravity_ , 2401.17361.
  * (7) C.M. Hull and P.K. Townsend, _Unity of superstring dualities_ , _Nucl. Phys. B_ 438 (1995) 109 [hep-th/9410167].
  * (8) C.M. Hull, _Gravitational duality, branes and charges_ , _Nucl. Phys. B_ 509 (1998) 216 [hep-th/9705162].
  * (9) K. Hinterbichler, D.M. Hofman, A. Joyce and G. Mathys, _Gravity as a gapless phase and biform symmetries_ , _JHEP_ 02 (2023) 151 [2205.12272].
  * (10) K. Hinterbichler, A. Joyce and G. Mathys, _Impossible Symmetries and Conformal Gravity_ , 2403.03256.
  * (11) V. Benedetti, H. Casini and J.M. Magán, _Generalized symmetries of the graviton_ , _Journal of High Energy Physics_ 2022 (2022) 45.
  * (12) V. Benedetti, P. Bueno and J.M. Magan, _Generalized Symmetries for Generalized Gravitons_ , _Phys. Rev. Lett._ 131 (2023) 111603 [2305.13361].
  * (13) V. Benedetti, H. Casini and J.M. Magan, _Generalized symmetries and Noether’s theorem in QFT_ , _JHEP_ 08 (2022) 304 [2205.03412].
  * (14) C. Gómez-Fayrén, P. Meessen and T. Ortín, _Covariant generalized conserved charges of General Relativity_ , _JHEP_ 09 (2023) 174 [2307.04041].
  * (15) C. Cheung, M. Derda, J.-H. Kim, V. Nevoa, I. Rothstein and N. Shah, _Generalized Symmetry in Dynamical Gravity_ , 2403.01837.
  * (16) H. Casini and J.M. Magan, _On completeness and generalized symmetries in quantum field theory_ , _Mod. Phys. Lett. A_ 36 (2021) 2130025 [2110.11358].
  * (17) P.d. Medeiros and C. Hull, _Exotic Tensor Gauge Theory and Duality_ , _Communications in Mathematical Physics_ 235 (2003) 255.
  * (18) X. Bekaert and N. Boulanger, _Tensor Gauge Fields in Arbitrary Representations of $GL(D,R)$. Duality and Poincaré Lemma_, _Communications in Mathematical Physics_ 245 (2004) 27.
  * (19) M. Dubois-Violette and M. Henneaux, _Generalized cohomology for irreducible tensor fields of mixed Young symmetry type_ , _Lett. Math. Phys._ 49 (1999) 245 [math/9907135].
  * (20) M. Dubois-Violette, _Lectures on differentials, generalized differentials and on some examples related to theoretical physics_ , in _School: Bariloche 2000: Quantum Symmetries in Theoretical Physics and Mathematics_ , 1, 2000 [math/0005256].
  * (21) M. Dubois-Violette and M. Henneaux, _Tensor fields of mixed Young symmetry type and N complexes_ , _Commun. Math. Phys._ 226 (2002) 393 [math/0110088].
  * (22) P.S. Howe and U. Lindström, _Some remarks on (super)-conformal Killing-Yano tensors_ , _JHEP_ 11 (2018) 049 [1808.00583].
  * (23) U. Lindström and O. Sarıoğlu, _Geometry, conformal Killing-Yano tensors and conserved “currents”_ , _JHEP_ 05 (2023) 176 [2206.08037].
  * (24) R. Penrose and W. Rindler, _Spinors and Space-Time_ , vol. 2 of _Cambridge Monographs on Mathematical Physics_ , Cambridge University Press (1986).
  * (25) V.P. Frolov and D. Kubizňák, _Higher-dimensional black holes: hidden symmetries and separation of variables_ , _Classical and Quantum Gravity_ 25 (2008) 154005.
  * (26) D. Kastor and J. Traschen, _Conserved gravitational charges from Yano tensors_ , _JHEP_ 08 (2004) 045 [hep-th/0406052].
  * (27) D. Marolf, _Chern-Simons terms and the three notions of charge_ , in _International Conference on Quantization, Gauge Theory, and Strings: Conference Dedicated to the Memory of Professor Efim Fradkin_ , pp. 312–320, 6, 2000 [hep-th/0006117].
  * (28) P. de Medeiros and C. Hull, _Geometric second order field equations for general tensor gauge fields_ , _JHEP_ 05 (2003) 019 [hep-th/0303036].
  * (29) M. Henneaux and C. Teitelboim, _Duality in linearized gravity_ , _Phys. Rev. D_ 71 (2005) 024018 [gr-qc/0408101].
  * (30) K. Hinterbichler, _Manifest Duality Invariance for the Partially Massless Graviton_ , _Phys. Rev. D_ 91 (2015) 026008 [1409.3565].
  * (31) K. Hinterbichler and A. Joyce, _Manifest Duality for Partially Massless Higher Spins_ , _JHEP_ 09 (2016) 141 [1608.04385].
  * (32) D. Gaiotto, A. Kapustin, N. Seiberg and B. Willett, _Generalized global symmetries_ , _Journal of High Energy Physics_ 2015 (2015) 172.


* active transformation §1.4.4
  * algebraic long-range order §2.5.4, §6.3
  * amplitude mode §7.3.2
  * Anderson–Higgs mechanism §3.1, §7.3, §7.5
  * antiferromagnet §2.4, §3.3, §4.1, §5.3.2
  * BCS theory §A.5, §7.1
  * BKT phase transition §6.3, §6.4, §7.5
  * Bogoliubov transformation §4.1
  * canonical momentum §1.2.4
  * canted antiferromagnet §3.3
  * cat states §2.7
  * central element Exercise 1.7
  * chiral symmetry §3.3.1, Exercise 3.3
  * classical state §2.5.3
  * cluster decomposition §2.7
  * coherence length §5.4, §6.1, Figure 7.1
  * coherent state §2.5.2
  * Coleman–Weinberg mechanism §7.6
  * colour superconductivity Exercise 7.4
  * complex scalar field §1.3.1, §1.3.2
  * configuration space
    * restricted §2.1, Exercise 2.1
  * conserved quantity §1.2.1
  * constant of motion §1.2.1, §1.2.4
  * constraint §1.5.2, §1.5.3, §7.3
  * continuity equation §1.2.2
  * continuous symmetry §1.2.2, §1.4.1
  * Cooper pair §7.1
  * coordinate transformation §1.5.1
  * correlation length §2.5.4, §5.4
  * coset §1.6.1, §2.5.1
  * critical exponent §5.2.1, §7.6
    * dynamical §A.3
  * critical point §A.3, §A.4, §5.1, §5.4, §5.5, §6.5.1
  * critical slowing down §6.5.2
  * critical temperature §5
  * crystal §2.3, §3.3, §6.4
  * decoherence §A.8
  * density matrix §2.5.4, §5
    * reduced §A.2, §A.8, §2.5.4
  * diagonal subgroup §1.5.3, Exercise 3.3
  * Dirac treatment of Hamiltonian constraints §1.5.3, §7.3
  * discrete symmetry §1.4.1
  * discrete time crystals §A.6
  * dislocation-mediated melting §6.4
  * domain §A.3
  * domain wall §6.1, §6.2
  * duality mapping §6.5.1
  * Ehrenfest classification §5.1, §6.3
  * Elitzur’s theorem §2.3
  * emergence §5.4
  * entanglement §A.2, §2.6
  * entanglement entropy §A.2
  * equation of state §5.2
  * ergodicity breaking §2.1
  * ferrimagnet Exercise 3.4
  * ferromagnet Exercise 2.8, §3.2, §3.3
  * flat direction §5.3.1
  * fluctuation-induced first-order phase transition §7.6
  * flux quantisation §7.4
  * free energy §5.2.1
  * gapped partner mode §3.4, §4.2.5
  * gauge field §1.5.2
  * gauge fix item 2
  * gauge freedom §1.5, §1.5
  * gauge potential §1.5.2
  * gauge transformation §1.5.2, §7.1
  * Ginzburg criterion §5.4
  * Ginzburg temperature §5.4, §5.4, §7.6
  * Ginzburg–Landau theory §5.4, §7.1
  * glass §A.1
  * global gauge symmetry §1.5.2
  * global symmetry §1.4.3
  * Goldstino §3.3.1
  * Goldstone’s theorem §3.1, Theorem 3.1
  * group §1.6.1
    * Abelian §1.6.1
    * non-Abelian §1.6.1
    * representation §1.6.3
  * Heisenberg model §2.4, Exercise 2.8, §4.1, §5.3.2
  * hexatic liquid crystal §6.4
  * Higgs boson §7.3.2
  * higher-form symmetry §A.7
  * Holstein–Primakoff transformation §4.1
  * homotopy group §6.4
  * imaginary time §4.2.1
  * infrared divergence §4.2.4
  * instability
    * against local measurement §2.7
    * thermodynamic §2.7
  * instanton §6.4
  * interpolating field §2.5.2
  * Ising model §2.7, §6.2
  * Josephson effect §2.5.5, §7.2
  * Josephson junction array Exercise 7.3
  * Kibble–Zurek mechanism §A.3, §6.5.2
  * Lagrangian §1.2.3
    * effective §3.2
  * Landau theory §5.2
  * Landau–Ginzburg theory §5.4
  * Lie algebra §1.6.2
    * representation §1.6.3
  * Lie group §1.6.2
  * Lieb-Mattis model §2.4
  * linear spin-wave theory §4.1
  * local symmetry §1.4.3
  * London penetration depth §7.3.1, §7.5
  * long-range order 2.34
    * algebraic §2.5.4, §6.3
    * diagonal §2.5.4
    * off-diagonal §2.5.4, §2.5.4
  * lower critical dimension §4.2
  * magnet
    * classical Exercise 2.1
    * quantum §2.4
  * magnon §2.4
  * many-body localisation §1.4.3
  * Matsubara frequency §4.2.1
  * Maxwell electromagnetism §1.5.2
  * mean-field theory §5.3.2, §5.3.2, §5.4, §7.6
  * measurement problem §A.8
  * Meissner effect §7.3.1, §7.4, §7.6
  * Mermin–Wagner–Hohenberg–Coleman theorem §1.4.1, §4.1, §4.2, §6.3, §7.5
  * minimal coupling §1.5.2, §7.1
  * monopole §6.1, §6.4
  * Nambu–Goldstone mode §1.4.1, §3
    * dispersion relation §1.4.4, §3.2
    * redundant §3.3
    * type-A §3.2, §4.2.4
    * type-B §3.2, §4.2.5
  * Néel state §2.4
  * Noether charge §1.2.4, §2.5.1
  * Noether current §1.2.2, §1.2.3, Exercise 2.4
  * Noether’s second theorem §1.5.2
  * Noether’s theorem Theorem 1.2, §3
  * Noether’s trick Exercise 1.4
  * non-Abelian group §1.6.1
  * non-commuting limits §2.2
  * normal subgroup Exercise 1.7
  * nucleation §A.3, §6.4
  * opalescence Figure 5.1, §5.5
  * order parameter §2.1, §2.5.2, §5.2
  * order parameter operator §2.5.2, §2.5.2
  * passive transformation §1.4.4
  * Peierls substitution §7.1, Exercise 7.1
  * penetration depth Figure 7.1, §7.3.1, §7.5
  * phase diagram §5.1
  * phase rotation §1.3.1
  * phase transition §5
    * continuous 1st item, §5.2.1
    * discontinuous §5.1
    * first-order §5.1, §6.4
    * fluctuation-induced first-order §7.6
    * quantum §A.4
    * second-order 1st item
  * phonon §2.3
  * product state §A.2, §2.1, §2.5.1, §2.6
  * quantum chromodynamics Exercise 3.3, Exercise 7.4
  * quantum corrections §A.4, §2.5.3, §4
  * quantum critical point §A.4
  * quantum criticality §A.4
  * “quantum fluctuations” §A.4, §4
  * quantum Hall effect §A.5, §6.5.3
  * quantum phase transition §A.4, §5.1
  * quasicrystal §A.1
  * quotient set §1.6.1, §2.5.1
  * reduced temperature §5.2.1
  * renormalisation §5.5, §6.3
  * replica symmetry breaking §A.1
  * representation §1.6.3
  * residual symmetry group §2.5.1
  * rigidity §3
  * scalar potential §1.5.2
  * scale invariance §A.3, §A.4, 2nd item, §5.2.1, §5.5
  * Schrödinger field §1.3.1
  * singular limit §2.1, §2.1, §2.2
  * skyrmion §6.5.3
  * soliton §6.5.3
  * spacetime translations §1.3.3
  * stability §A.8
  * staggered magnetisation §2.5.2, §5.3.2
  * Standard Model §3.3.1, Exercise 3.3, §7.3.2, Exercise 7.4
  * structure constants §1.6.2
  * subgroup §1.6.1, §2.5.1
  * superconductor §A.5, §2.5.2, §7
  * supercooling §A.1, §A.3
  * supercurrent §3.3, §7.1, §7.1, §7.3.1
  * superfluid §2.5.2, §2.5.5, §3.3, §6.1
  * supersymmetry §3.3.1
  * symmetry
    * continuous §1.2.2, §1.4.1
    * discrete §1.4.1
    * generator §1.2.2, §1.2.4, §1.6.2
    * global §1.3.1
    * group §1.6.1
    * higher-form §A.7
    * is relative §1.1.1, §1.5.3
    * of states Definition 1.1, §2.5.2
    * of the action §1.2.3
    * of the Hamiltonian Definition 1.2
    * time translation §A.6, §1.3.3
  * symmetry breaking
    * explicit §1.5.3, §3.1
    * spontaneous §2
  * symmetry generator §1.2.2, §1.2.4, §2.5.1
  * symmetry group
    * continuous §1.6.1, §2.5.1
    * discrete §1.6.1
    * finite §1.6.1
    * infinite §1.6.1
    * residual §2.5.1
  * symmetry transformation
    * $SO(2)$ Exercise 1.3
    * translation §1.3.3, §1.4.3
    * $U(1)$ §1.3.1
  * symmetry-protected topological order §A.5
  * thermodynamic limit §2.1
  * time crystal §A.6
    * discrete §A.6
  * time evolution operator §1.2.1
  * time reversal symmetry §1.4.2
  * topological charge §6.1, §6.1
  * topological defect §6, §6.1, §6.1
    * unbinding §6.3
  * topological insulator §A.5
  * topological invariant §A.5, §6.1
  * topological melting §6.2, §6.3
  * topological order §A.5
  * tower of states §2.1, §2.3, §2.4, §3, §3.4.1
  * translation symmetry §1.3.3, §1.4.3
  * universality §A.4, §5.5, §6.5.2
  * universality class §A.4, §5.5
  * upper critical dimension §5.4
  * variance §2.7
  * vector potential §1.5.2
  * vortex §6.1, §6.1
  * Ward–Takahashi identity §1.5.2
  * Watanabe–Brauner matrix §3.2, §3.3, §3.3, Exercise 3.4
  * winding number §6.1, §6.1, §6.4
  * $XY$-model Exercise 4.1, §6.1


# Cross Attention-guided Dense Network for Images Fusion

Zhengwen Shen, Jun Wang
Zaiyu Pan, Yulian Li, Jiangyu Wang
China University of Mining and Technology, Xuzhou, 221000, China
<EMAIL_ADDRESS><EMAIL_ADDRESS>
{pzycumt, lylcumtedu<EMAIL_ADDRESS>Corresponding author

###### Abstract

In recent years, various applications in computer vision have achieved
substantial progress based on deep learning, which has been widely used for
image fusion and shown to achieve adequate performance. However, suffering
from limited ability in modelling the spatial correspondence of different
source images, it still remains a great challenge for existing unsupervised
image fusion models to extract appropriate feature and achieves adaptive and
balanced fusion. In this paper, we propose a novel cross attention-guided
image fusion network, which is a unified and unsupervised framework for multi-
modal image fusion, multi-exposure image fusion, and multi-focus image fusion.
Different from the existing self-attention module, our cross attention module
focus on modelling the cross-correlation between different source images.
Using the proposed cross attention module as core block, a densely connected
cross attention-guided network is built to dynamically learn the spatial
correspondence to derive better alignment of important details from different
input images. Meanwhile, an auxiliary branch is also designed to model the
long-range information, and a merging network is attached to finally
reconstruct the fusion image. Extensive experiments have been carried out on
publicly available datasets, and the results demonstrate that the proposed
model outperforms the state-of-the-art quantitatively and qualitatively. The
source code is available at https://github.com/shenzw21/CADNIF.

##  1 Introduction

Image fusion, which aims at unifying the feature information from different
source images into an efficient representation and advancing the visual
perception performance of humans and machines. Particularly, image fusion has
a wide and various applications in computer vision, such as multi-modal image
fusion: infrared and visible image fusion(Li and Wu 2018), RGB and depth image
fusion for semantic segmentation(Wang et al. 2020), medical image fusion(Ma et
al. 2020), etc; multi-focus image fusion(Zhang et al. 2021); multi-exposure
image fusion(Ram Prabhakar, Sai Srikar, and Venkatesh Babu 2017), etc.
In recent years, traditional methods for image fusion take the lead and group
three main directions: transform domain approach(Cao et al. 2014), sparse
representation(Bin, Chao, and Guoyu 2016; Zhang et al. 2018), various
component analysis(Kuncheva and Faithfull 2013). While many traditional
methods are proposed and perform well in image fusion task, there also exists
shortcoming and limitations: the feature extraction relies on handcrafted,
lack of flexibility and generalization ability. With the development and wide
application of deep learning, the limitations of traditional methods got a
certain breakthrough and many image fusion methods based on deep learning are
proposed(Ram Prabhakar, Sai Srikar, and Venkatesh Babu 2017; Li and Wu 2018;
Ma et al. 2019). Although existing methods proposed based on deep learning
improve the performance and generalization ability, image fusion still suffer
from feature extraction and modelling spatial correspondence learning from
source images issues. For example, design fusion strategy need more artificial
for different fusion tasks, the balance of feature extraction and effective
jointly representation learning from source images. In order to solve the
mentioned problems, in this paper we proposed a novel cross attention-guided
dense network for multi-task image fusion which is constructed by cross
attention-guided densenet, auxiliary network, and merging network. Modelling
the cross-correlation, we proposed the cross attention-guided dense network
focus on the spatial correspondence from source images by dense connected
strategy. Avoiding lack of the global information, we adopted an auxiliary
network for extract the long-range information. To better reconstruct the
fusion image, we adopted a residual connect merging network to aggregate the
fusion feature from source images.
The main contributions of this paper are summarized as follows:

  * •

A. We propose a novel multi-task images fusion method, which is named Cross
Attention-guided Dense Network for Images Fusion(CADNIF). It utilizes the
advantage of the deep neural network model with attention mechanisms, and
improves the ability of feature extraction and modelling spatial
correspondence from source images effectively and robustness.

  * •

B. We propose a new attention-guided dense network to modelling spatial
correspondence from source images. To preserve more global information and
details from source images, we employ an auxiliary network to capture long-
range information and a dilated residual dense blocks with larger receptive
field for reconstructing the fusion image.

  * •

C. Extensive experiments on different datasets validate the superiority of the
proposed CADNIF, which is successfully applied to multi image fusion tasks
with superior performance against the state-of-the-art methods.

##  2 Related Work

In this section, we group the image fusion methods into two categories,
traditional methods and deep learning-based approaches for image fusion. We
also briefly introduce the dense block and attention mechanisms, which are
highly related to our proposed method.

#### Image fusion.

In the field of image fusion, the key to the evaluation of fusion algorithms
is how to effectively extract features and fuse features. Numbers of
traditional image fusion methods have been proposed to solve the feature
extract problem, and group the methods into three main categories: transform
domain approach, such as discrete cosine(DCT), discrete wavelet(DWT), etc;
sparse representation domain approach; various component analysis, such as
principal component analysis(PCA), independent component analysis(ICA), etc.
However, the feature extraction methods lack flexibility and generalizability
because of the increasing complexity. Moreover, traditional methods need to
pay much attention to design the appropriateness of fusion methods to ensure
the features come from different specific source images.
With the wide application of deep learning in high-level vision tasks, the
limitations of traditional methods have a breakthrough to a certain extent.
More and more CNNs based approaches were introduced for feature extraction as
a backbone for image fusion tasks via various fusion strategies. The deep
learning-based approaches for image fusion have been application successful in
several areas, such as multi-modal image fusion, multi-exposure image fusion,
multi-focus image fusion, and medical image fusion. (Ram Prabhakar, Sai
Srikar, and Venkatesh Babu 2017) proposed a deep learning-based unsupervised
Deepfuse framework for exposure and extreme exposure image fusion. (Li and Wu
2018) proposed an infrared and visible image fusion method named DenseFuse
based on Deepfuse and incorporated dense block in the encoding network for
feature extraction. (Ma et al. 2019) proposed a FusionGAN based on the
generative adversarial network for infrared and visible image fusion. (Ma et
al. 2020) extended previous work to adopt a dual-discriminator strategy for
fusing multi-modality images of multi-resolution image fusion. Recently, some
works are focus on unified unsupervised end-to-end image fusion network, (Xu
et al. 2020; Zhang et al. 2020a) are each proposed unified fusion network for
image fusion, such as multi-modal multi-exposure, multi-focus. However, above
methods focus on significance of the difference between the different source
images and do not consider the adaptability balance of the final fusion
information from source images. In this paper, we propose a more generic
unified unsupervised approach to multi image fusion task, which joint learning
spatial correspondence information from source images, and for each cross
attention module outputs, makes it possible for building a balance that each
source images become one.

#### Image fusion with dense block.

While deep convolution neural network has achieved remarkable results in the
field of computer vision, with the wide application and the depth increased,
gradually exposed a lot of problems, such as performance degradation,
vanishing gradient, exploding gradient, and previous features not reused
fully, etc. To address the above problems, (He et al. 2016) proposed a deep
residual learning network, based on He’s work, (Huang et al. 2017) proposed a
new network framework densenet, which each layers can be concatenated by
previous layer and as an input for the next layer and strengthen feature
propagation, encourage feature reuse, and alleviates the problem of the
gradient. Based on the above advantages, the dense block has been widely
incorporated into multi computer vision tasks, such as semantic segmentation,
image classification, object detection, etc. In the image fusion task, a dense
block is widely used in feature extraction. (Li and Wu 2018) incorporate the
dense block into the encoding network, preserve the useful information from
middle layers. (Ma et al. 2020) proposed a novel GAN framework for image
fusion and the architecture of the generator is based on densenet. Instead of
directly using densenet as a feature extraction network, we propose cross
attention-guided dense network, which effective learning cross-correlation
information between different source images.

#### Attention mechanisms in deep learning methods.

In recent years, with the emergence of attention mechanism, which is a
technique that enables models to focus on important information and fully
learn to absorb it, it has been widely used in many computer vision tasks,
natural language processing, image and video understanding. (Bastidas and Tang
2019) proposed a novel channel attention network for multi-spectral imagery.
(Tian, Guo, and Long 2021) proposed a multi-level attention to address the
problem of Crowd Counting. (Song et al. 2021) proposed a cross-modal attention
for MRI and Ultrasound Volume Registration. Our work explores cross attention
for basic mutual feature extraction of source images.

Figure 1: The overall architecture of the proposed CADNIF mainly consists of
three components: Attention-guided Network, Auxiliary network, and Merging
Network. Firstly, the attention-guided network is adopted to extract cross
attention feature from source images. Simultaneously, the auxiliary network
extracts the long-range feature representations via series convolution.
Finally, merging network merging the global feature representations and the
spatial correspondence via a dilated residual dense block and global residual
connection.

##  3 Methods

We propose a novel cross attention-guided image fusion framework, which
focuses on improving the cross significance information representation,
modelling the spatial correspondence, capture the long-range information and
merging the feature from source images to reconstruct the fusion image. As
shown in Figure 1, there are three networks: cross attention-guided dense
network, auxiliary network, and merging networks. The attention-guided network
captures the spatial correspondence feature via using five attention blocks
based on densenet, the responsible of auxiliary for capture the long-range
information, while the merging network relying on a series of dilated residual
dense blocks to utilize the image features effectively and obtain the fusion
images more details. Finally, the proposed attention-guided fusion network
further to fuse the source images and the following experiments show that the
fusion result prefers well.

###  3.1 Cross Attention Guided Densenet

Unlike the previous methods, which use the densenet as a feature extract
network directly, our proposed cross attention guided densenet obtains the
cross-attention information from each source image. As shown in Figure 2, the
attention-guided densenet consists of five major cross attention blocks. Given
input images $I_{i}$ and $I_{j}$, each input image is concatenated by three
one-channel gray images of the same source. We first concatenate the input
images and obtain the 6-channel concatenation image. As shown in Figure 2a,
the input images and the concatenated image feed into the attention-guided
network, and obtain the final cross attention features via dense connection.
As shown in Figure 2b, the cross attention guided block consists of three
convolution layers to obtain feature maps, concatenate operation, and
attention module. Details of the cross attention block are provided below. The
final cross attention maps can be obtained via:

$\begin{split}A_{i}=Attent\left(I_{i},Concat\left(I_{i},I_{j}\right)\right)\end{split}$
(1)
$\begin{split}A_{j}=Attent\left(I_{j},Concat\left(I_{i},I_{j}\right)\right)\end{split}$
(2) $\begin{split}Z_{g}=Concat\left(Concat\left(I_{i},I_{j}\right),A_{i}\circ
I_{i},A_{j}\circ I_{j}\right)\end{split}$ (3)

where $\circ$ denotes the point-wise multiplication, $I_{i}$ and $I_{j}$
denote the original input images, $Z_{i}$ and $Z_{j}$ denote each original
image concatenate with $Concat\left(I_{i},I_{j}\right)$, $A_{i}$ and $A_{j}$
denote the attention map, $Z_{g}$ denote the final cross attention feature
maps.

Figure 2: (a) The architecture of the proposed attention-guided dense network,
the network consists of five attention blocks for modelling the spatial
correspondence. (b) shows the cross attention blocks. (c) shows the attention
module.

###  3.2 Attention Module

Attention modules play an important role in the cross attention block, as
shown in Figure 2c, to obtain the one-to-one correspondence maps for each
source image, the attention module consists of two convolution layers and a
Sigmoid layer. From Eq. 3, the input of the attention module is $A_{i}$ or
$A_{j}$, and obtain the feature maps via two convolution layers. Each
convolution layer applies $3\times 3$ layers. Respectively, a ReLU activation
and a Sigmoid activation in the module after convolution operation. As a
result, we finally obtain the attention map $Z_{g}$ with values in the range
$\left[0,1\right]$.

###  3.3 Auxiliary network

As shown in Figure 3, in the auxiliary network, we first use $1\times 1$
convolution kernel to obtain 32-channels feature maps via concatenating
operation of the input images $I_{i}$ and $I_{j}$, then we use a convolution
block consists of three different kernel size convolution operation branches,
$1\times 1$, $3\times 3$, $5\times 5$, respectively, each of convolutions is
followed by layer normalization. Finally, obtain the final fusion feature maps
via concatenate operation and followed a layer normalization after the
convolution of $1\times 1$ kernel.

Figure 3: The architecture of the auxiliary network.

###  3.4 Merging Network

To better reconstruct the fused image, integrate the cross-correlation
information and long-range information to obtain more details of the feature
maps, we adopt a dilated residual dense block proposed in (Yan et al. 2019) as
the reconstructed network. The merging network takes the concatenated feature
maps after a convolution layer of $1\times 1$ kernel operation as input, which
is concatenated from the attention-guided network and auxiliary network. As
shown in figure 4, the dilated residual dense block consists of three
convolution layers followed by ReLu activation, concatenation-based skip-
connection similar to densenet, and a convolution layer of $1\times 1$ kernel
operation as output. Different from the normal convolution operation, each
convolution layers adopt 2-dilated convolutions, kernel size is $3\times 3$.
Finally, we apply a global residual connection strategy to concatenate the
output with concatenation image from source images.

Figure 4: The detail architecture of the merging network.

###  3.5 Training Loss

As described above section, the proposed CANIF method can be trained to obtain
the context details structural and the background details from the different
sources images. In the proposed method, the loss function consists of Mean
Square Error($MSE$) loss $L_{mse}$ and Structural Similarity($SSIM$) loss
$L_{ssim}$. The $SSIM$ loss is an index to measure the similarity of two
images, the larger the value, the better, the maximum is 1. Generally, if the
value is 1, then the fused image retains the input images more structural
details, the $L_{ssim}$ is defined by Eq. 4,

$\begin{split}L_{ssim}=1-SSIM\left(O,I\right)\end{split}$ (4)

where $I$ denotes the input source image, $O$ denotes the fused image,
$SSIM\left(\cdot\right)$ denotes the measures structural similarity. The $MSE$
loss emphasize the matching of each corresponding pixel between the input
image and output image, the $MSE$ loss $L_{mse}$ is calculated as Eq. 5,

$\begin{split}L_{mse}=\left\|I_{f}-I\right\|_{2}^{2}\end{split}$ (5)

where $I$ denotes the input source image, $I_{f}$ denotes the fused image,
$\left\|\cdot\right\|_{2}^{2}$ denotes the $l_{2}$ normal. Finally, the total
losses $L_{total}$ of the proposed model can be expressed as follows:

$\begin{split}L_{total}=\lambda_{mi}I_{mse_{i}}+\lambda_{mj}I_{mse_{j}}+\lambda_{si}I_{ssim_{i}}+\lambda_{sj}I_{ssim_{j}}\end{split}$
(6)

where $\lambda_{\left(\cdot\right)}$ denotes the contribution of each loss to
the whole objective function. In this paper, for each experiment below,
setting the parameter $\lambda_{mi}$, $\lambda_{mj}$, $\lambda_{si}$,
$\lambda_{sj}$ equal to: 1, 0.5, 0.03, 0.03, in infrared and visible fusion
experiment; 1, 1, 0.01, 0, in MRI and PET fusion experiment; 2, 5, 1, 1, in
Multi-focus fusion experiment; 0.5, 0.7, 1.3, 1, in Multi-exposure fusion
experiment.

##  4 Experiments

To demonstrate the proposed method, we conduct the experiments on five
publicly available datasets:
TNO111http://figshare.com/articles/TNOImageFusionDataset/1008029 and
RoadScene222https://github.com/hanna-xu/RoadScene datasets for visible and
infrared image fusion task, TNO used for training, RoadSence are all used for
testing; MFF and SICE datasets333https://drive.google.com/drive/folders for
multi-exposure image fusion task; MRI and
PET444http://www.med.harvard.edu/AANLIB/home.html datasets for medical image
fusion task; Lytro555https://github.com/sametaymaz/Multi-focus-Image-Fusion-
Dataset for multi-focus image fusion task. For the lack of training data
problem, we adopt the expansion strategy based on(Zhang et al. 2020a). For the
testing, the number of image pairs used for testing dataset is 17, 50, 19, 20,
and 18, respectively.
For all experiments, we set the batchsize, learning rate, and epoch equal to
16, $10^{-5}$, 10, respectively. The proposed method was implemented on NVIDIA
GEFORCE RTX 2080 Ti GPU and based on TensorFlow.

###  4.1 Infrared and visible image fusion

In the infrared and visible image fusion experiment, most TNO datasets are
used to train the proposed image fusion method, image pairs used for testing
is 17, and 50 pairs of RoadSence datasets are used to test our proposed method
to verify its generalization and model robustness. We compare our proposed
method with CE(Zhou et al. 2016), DDLatLRR(Li, Wu, and Kittler 2020),
DenseFuse(Li and Wu 2018), PMGI(Zhang et al. 2020a), U2Fusion(Xu et al. 2020).
To effective quantitatively evaluate the fusion quality, we adopt five related
indicators to compare the existing methods, namely entropy (EN), sum of the
correlations of differences (SCD)(Aslantas and Bendes 2015), standard
deviation (SD), represents the ratio of noise added to the final image(Qabf),
and mutual information (MI). As shown in Table 1 and Table 2, the quantitative
result of TNO and RoadSence dataset, the red font in italic represents the
best value and the bold black font in italic represents the second best value.
Similarly, we can infer that the quantitative value of proposed method
outperform in EN, SCD, SD and MI metrics. It demonstrates that the proposed
method maintains abundant correspondence spatial information and detail
information from source images.
The qualitative fusion results on two typical image pairs of each test dataset
are illustrated Figure 5 and Figure 6, red box highlights are the local fine
features. From the results of TNO dataset and RoadSence dataset, we can
observe that the proposed method capture abundant correspondence semantic
information compare with the state-of-the-art methods. Similarly, the
highlights information obtained by zooming in on local fine features, we can
infer that the local fine information retained well from source images, such
as sky, window, license plate, and the top of the tree, etc. In particular,
for each source image illumination information and texture details, the fusion
result shows that the proposed method performs well by adaptive balance
strategy via cross attention-guided network.

Figure 5: Comparison result of infrared and visible image fusion of TNO datasets. From left to right: infrared image, visible image, the results of CE, DDLatLRR, DenseFuse, PMGI, U2Fusion and proposed. Figure 6: Comparison result of infrared and visible image fusion of RoadSence datasets. From left to right: infrared image, visible image, the results of CE, DDLatLRR, DenseFuse, PMGI, U2Fusion and proposed. Methods | EN | SCD | SD | Qabf | MI
---|---|---|---|---|---
CE | 7.1939 | 1.5846 | 90.6729 | 0.4798 | 14.3879
DDLatLRR | 6.8565 | 1.7255 | 76.3305 | 0.4881 | 13.7131
DenseFuse | 6.8425 | 1.6183 | 89.3237 | 0.5350 | 13.6851
PMGI | 7.1191 | 0.1062 | 99.6564 | 0.1535 | 14.1382
U2Fusion | 6.8959 | 1.5877 | 76.4380 | 0.3941 | 13.7919
Proposed | 7.2841 | 1.8318 | 104.3948 | 0.3256 | 14.5683
Table 1: Quantitative evaluation for each methods via 17 pairs infrared and visible image fusion of TNO dataset. Methods | EN | SCD | SD | Qabf | MI
---|---|---|---|---|---
CE | 7.3635 | 1.5893 | 75.8509 | 0.4587 | 14.7271
DDLatLRR | 7.1485 | 1.6482 | 67.6889 | 0.5017 | 14.2971
DenseFuse | 6.8549 | 1.2950 | 65.5098 | 0.5074 | 13.7099
PMGI | 7.2492 | 1.7410 | 78.2691 | 0.4230 | 14.6985
U2Fusion | 7.1968 | 1.7692 | 68.0393 | 0.4791 | 14.3937
Proposed | 7.5315 | 1.7785 | 97.9758 | 0.4571 | 15.0631
Table 2: Quantitative evaluation for each methods via 50 pairs infrared and
visible image fusion of RoadSence dataset.

###  4.2 MRI and PET image fusion

Methods | EN | SD | CC | Qabf | MI
---|---|---|---|---|---
DCHWT | 5.7501 | 84.9344 | 0.7302 | 0.6849 | 11.1003
SA | 4.9705 | 84.6888 | 0.7079 | 0.7415 | 9.9411
DDCTPCA | 4.8017 | 84.9032 | 0.8030 | 0.4486 | 9.6035
PMGI | 5.3341 | 83.9781 | 0.7646 | 0.6917 | 10.6684
IFCNN | 4.8187 | 98.1931 | 0.7726 | 0.6662 | 9.6374
Proposed | 5.5149 | 80.2054 | 0.8180 | 0.6211 | 11.1298
Table 3: Quantitative evaluation for each methods via 20 pairs of MRI and PET
image fusion. Figure 7: Comparison result of MRI and PET image fusion. From
left to right: MRI image, PET image, the results of DCHWT, Structure
Aware(SA), DDCTPCA, PMGI, IFCNN and Proposed.

In the MRI and PET image fusion experiment, the number of test image pairs is
20. We compare our proposed method with DCHWT(Kumar 2013), Structure Aware(Li
et al. 2018), DDCTPCA(Naidu 2014), PMGI(Zhang et al. 2020a), IFCNN(Zhang et
al. 2020b) by quantitative and qualitative evaluation metrics.
We adopt five related indicators to compare the existing methods, namely
entropy (EN), Feature mutual information (FMI), standard deviation
(SD),represents the ratio of noise added to the final image(Qabf), and mutual
information (MI). As shown in Table 3, we can observe that the proposed fusion
method outperforms in CC, and MI metrics, in EN metric, the result performance
better than other methods but DCHWT. Qualitative results can be seen from
Figure 7, the spatial correspondence details information is preserved, and the
local context detail information is contained from source input images.

###  4.3 Multi-focus Image Fusion

Methods | EN | FMI | SD | MI | CC
---|---|---|---|---|---
VSMWLS | 7.4934 | 0.8850 | 106.0445 | 15.0068 | 0.9564
CBF | 7.4699 | 0.8927 | 106.5535 | 14.9399 | 0.9514
SA | 7.3825 | 0.8943 | 107.3123 | 14.7652 | 0.9495
ConvSR | 7.4615 | 0.8940 | 107.3435 | 14.9230 | 0.9496
MFF | 7.5666 | 0.8826 | 111.8743 | 15.1333 | 0.9515
Proposed | 7.5005 | 0.8788 | 113.1872 | 15.0811 | 0.9530
Table 4: Quantitative evaluation for each specialized methods via 18 pairs of
Far-focused and Near-focused image fusion. Figure 8: Comparison result of
Multi-focus image fusion. From left to right: Far-focused, Near-focused, the
results of VSMWLS, CBF, Structure Aware, ConvSR, MFF and Proposed.

To verify the proposed method ability which extract features, versatility, and
generalization from source images, we conducted fusion experiments based on
the near-focused and far-focused image datasets. At the same time, our
proposed method was compared with other methods, which specialized focused on
the far and near focused images fusion task. We employ five existing methods
to compare with our method: VSMWLS(Ma et al. 2017), CBF(Kumar 2015), Structure
Aware(Li et al. 2018), ConvSR(Liu et al. 2016) and MFF(Zhang et al. 2021),
respectively.
For quantitative analysis, we adopt five related indicators to compare the
existing methods, namely entropy (EN), feature mutual information (FMI),
standard deviation (SD), mutual information (MI), and correlation coefficient
(CC). As shown in Table 4, we can infer that the proposed fusion method
outperform in SD metric, outperform in EN, MI, and CC metrics but MFF and
VSMWLS method. More intuitively for qualitative analysis, from Figure 8, it
demonstrates that our proposed method has a good feature extraction capability
of illumination information and spatial details information for multi-focus
images compared with the specialized methods.

###  4.4 Multi-exposure Image Fusion

In the multi-exposure image fusion experiment, we employ five existing methods
to compare with our methos: FMMR(Li and Kang 2012), DSIFT(Hayat and Imran
2019), DeepFuse(Ram Prabhakar, Sai Srikar, and Venkatesh Babu 2017),
MGFF(Bavirisetti et al. 2019) and IFCNN(Zhang et al. 2020b), respectively.
From Table 5, compare the five existing methods by entropy (EN), Feature
mutual information (FMI), standard deviation (SD), correlation coefficient
(CC), and mutual information (MI). We can observe that the proposed fusion
method outperforms in SD and CC metrics, in EN and MI metric our method
outperforms but MGFF and DeepFuse. Visual comparison results of each image
fusion method can be seen from Figure 9. From the perspective of global image
information, the proposed method adaptive balance the illuminate information
well, similarly the local context detail information is retained and presented
from source images.

Methods | EN | FMI | SD | CC | MI
---|---|---|---|---|---
FMMR | 6.3172 | 0.8736 | 92.5337 | 0.5448 | 12.6344
DISIFT | 6.5245 | 0.8934 | 101.3824 | 0.5445 | 13.0491
DeepFuse | 6.8183 | 0.8926 | 116.3688 | 0.7945 | 13.8566
MGFF | 6.8792 | 0.8874 | 116.2462 | 0.7976 | 13.7586
IFCNN | 6.7259 | 0.8838 | 113.8987 | 0.7572 | 13.4519
Proposed | 6.8598 | 0.8619 | 122.6287 | 0.8202 | 13.8398
Table 5: Quantitative comparisons of the 19 pairs of underexposed and
overexposed image by five metrics. Figure 9: Comparison result of Multi-
exposure image fusion. From left to right: underexposed image, overexposed
image, the fusion results of FMMR, DISIFT, DeepFuse, MGFF, IFCNN and Proposed.

###  4.5 Ablation Study and Visualization

In this section, we take an ablation study which is conducted on infrared and
visible images fusion experiment as an example. Compared with dilated residual
dense block(DRDB) and original densenet without cross attention-guided module.
As shown in Table 6, the bold black font in italic represents the best value.
It is evident that for the reconstructed fusion image, the proposed method can
effect modelling the spatial correspondence, and the DRDB module can obtain
sufficient information from the reconstructed image based on a multi-illusion
image by use dilated convolution to obtain a larger receptive field to
generate illusion and details.
As shown in Figure 10, we visualize the cross attention maps obtained by each
cross-attention module. Taking infrared and visible image fusion task as an
example, we can infer from the figure that the fusion image background, detail
information, and significance features obtained by each attention module are
enhanced. It shows that the cross attention-guided is effective for the
extraction of spatial correspondence feature information from source images.

Methods | EN | SCD | SD | Qabf | MI
---|---|---|---|---|---
Densenet | 7.0573 | 0.5774 | 100.39 | 0.3050 | 14.1147
Without DRDB | 7.1020 | 1.3297 | 97.9295 | 0.2757 | 14.2040
Proposed | 7.2841 | 1.8318 | 104.3948 | 0.3256 | 14.5683
Table 6: Ablation Study on cross attention-guided network for infrared and
visible image fusion. Figure 10: The attention map of cross attention unit
from the source images.

##  5 Conclusion

This work introduced a novel unsupervised framework for the challenging multi-
image fusion task, and it can serve as a unified framework for four tasks
including infrared and visual image fusion, medical image fusion, multi-focus
image fusion, and multi-exposure image fusion. To learn semantic features
better for image fusion effectively, a cross attention-guided dense network is
proposed to learn the spatial correspondence explicitly, and only the
essential details are then learned and aligned attentively. We also proposed
an auxiliary branch and a global merging block to better model the long-range
relationship and better leverage the global information to reconstruct the
fused image. Through quantitative comparison and qualitative analysis, the
proposed method achieves better result compared with the state-of-the-art
fusion methods.

##  6 Acknowledgments

This work was partially supported by the Scientific Innovation 2030 Major
Project for New Generation of AI under Grant 2020AAA0107300.

## References

  * Aslantas and Bendes (2015) Aslantas, V.; and Bendes, E. 2015.  A new image quality metric for image fusion: the sum of the correlations of differences.  _Aeu-international Journal of electronics and communications_ , 69(12): 1890–1896.
  * Bastidas and Tang (2019) Bastidas, A. A.; and Tang, H. 2019.  Channel Attention Networks.  In _2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW)_ , 881–888.
  * Bavirisetti et al. (2019) Bavirisetti, D. P.; Xiao, G.; Zhao, J.; Dhuli, R.; and Liu, G. 2019.  Multi-scale guided image and video fusion: A fast and efficient approach.  _Circuits, Systems, and Signal Processing_ , 38(12): 5576–5605.
  * Bin, Chao, and Guoyu (2016) Bin, Y.; Chao, Y.; and Guoyu, H. 2016.  Efficient image fusion with approximate sparse representation.  _International Journal of Wavelets, Multiresolution and Information Processing_ , 14(04): 1650024.
  * Cao et al. (2014) Cao, L.; Jin, L.; Tao, H.; Li, G.; Zhuang, Z.; and Zhang, Y. 2014.  Multi-focus image fusion based on spatial frequency in discrete cosine transform domain.  _IEEE signal processing letters_ , 22(2): 220–224.
  * Hayat and Imran (2019) Hayat, N.; and Imran, M. 2019.  Ghost-free multi exposure image fusion technique using dense SIFT descriptor and guided filter.  _Journal of Visual Communication and Image Representation_ , 62: 295–308.
  * He et al. (2016) He, K.; Zhang, X.; Ren, S.; and Sun, J. 2016.  Deep residual learning for image recognition.  In _Proceedings of the IEEE conference on computer vision and pattern recognition_ , 770–778.
  * Huang et al. (2017) Huang, G.; Liu, Z.; Van Der Maaten, L.; and Weinberger, K. Q. 2017.  Densely connected convolutional networks.  In _Proceedings of the IEEE conference on computer vision and pattern recognition_ , 4700–4708.
  * Kumar (2013) Kumar, B. S. 2013.  Multifocus and multispectral image fusion based on pixel significance using discrete cosine harmonic wavelet transform.  _Signal, Image and Video Processing_ , 7(6): 1125–1143.
  * Kumar (2015) Kumar, B. S. 2015.  Image fusion based on pixel significance using cross bilateral filter.  _Signal, image and video processing_ , 9(5): 1193–1204.
  * Kuncheva and Faithfull (2013) Kuncheva, L. I.; and Faithfull, W. J. 2013.  PCA feature extraction for change detection in multidimensional unlabeled data.  _IEEE transactions on neural networks and learning systems_ , 25(1): 69–80.
  * Li and Wu (2018) Li, H.; and Wu, X.-J. 2018.  DenseFuse: A fusion approach to infrared and visible images.  _IEEE Transactions on Image Processing_ , 28(5): 2614–2623.
  * Li, Wu, and Kittler (2020) Li, H.; Wu, X.-J.; and Kittler, J. 2020.  MDLatLRR: A novel decomposition method for infrared and visible image fusion.  _IEEE Transactions on Image Processing_.  Doi: 10.1109/TIP.2020.2975984.
  * Li and Kang (2012) Li, S.; and Kang, X. 2012.  Fast multi-exposure image fusion with median filter and recursive filter.  _IEEE Transactions on Consumer Electronics_ , 58(2): 626–632.
  * Li et al. (2018) Li, W.; Xie, Y.; Zhou, H.; Han, Y.; and Zhan, K. 2018.  Structure-aware image fusion.  _Optik_ , 172: 1–11.
  * Liu et al. (2016) Liu, Y.; Chen, X.; Ward, R. K.; and Wang, Z. J. 2016.  Image fusion with convolutional sparse representation.  _IEEE signal processing letters_ , 23(12): 1882–1886.
  * Ma et al. (2020) Ma, J.; Xu, H.; Jiang, J.; Mei, X.; and Zhang, X.-P. 2020.  DDcGAN: A dual-discriminator conditional generative adversarial network for multi-resolution image fusion.  _IEEE Transactions on Image Processing_ , 29: 4980–4995.
  * Ma et al. (2019) Ma, J.; Yu, W.; Liang, P.; Li, C.; and Jiang, J. 2019.  FusionGAN: A generative adversarial network for infrared and visible image fusion.  _Information Fusion_ , 48: 11–26.
  * Ma et al. (2017) Ma, J.; Zhou, Z.; Wang, B.; and Zong, H. 2017.  Infrared and visible image fusion based on visual saliency map and weighted least square optimization.  _Infrared Physics & Technology_, 82: 8–17.
  * Naidu (2014) Naidu, V. 2014.  Hybrid DDCT-PCA based multi sensor image fusion.  _Journal of Optics_ , 43(1): 48–61.
  * Ram Prabhakar, Sai Srikar, and Venkatesh Babu (2017) Ram Prabhakar, K.; Sai Srikar, V.; and Venkatesh Babu, R. 2017.  Deepfuse: A deep unsupervised approach for exposure fusion with extreme exposure image pairs.  In _Proceedings of the IEEE international conference on computer vision_ , 4714–4722.
  * Song et al. (2021) Song, X.; Guo, H.; Xu, X.; Chao, H.; Xu, S.; Turkbey, B.; Wood, B. J.; Wang, G.; and Yan, P. 2021.  Cross-modal Attention for MRI and Ultrasound Volume Registration.  _arXiv preprint arXiv:2107.04548_.
  * Tian, Guo, and Long (2021) Tian, M.; Guo, H.; and Long, C. 2021.  Multi-Level Attentive Convoluntional Neural Network for Crowd Counting.  _arXiv preprint arXiv:2105.11422_.
  * Wang et al. (2020) Wang, Y.; Huang, W.; Sun, F.; Xu, T.; Rong, Y.; and Huang, J. 2020.  Deep multimodal fusion by channel exchanging.  _Advances in Neural Information Processing Systems_ , 33.
  * Xu et al. (2020) Xu, H.; Ma, J.; Jiang, J.; Guo, X.; and Ling, H. 2020.  U2Fusion: A unified unsupervised image fusion network.  _IEEE Transactions on Pattern Analysis and Machine Intelligence_.
  * Yan et al. (2019) Yan, Q.; Gong, D.; Shi, Q.; Hengel, A. v. d.; Shen, C.; Reid, I.; and Zhang, Y. 2019\.  Attention-guided network for ghost-free high dynamic range imaging.  In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , 1751–1760.
  * Zhang et al. (2021) Zhang, H.; Le, Z.; Shao, Z.; Xu, H.; and Ma, J. 2021.  MFF-GAN: An unsupervised generative adversarial network with adaptive and gradient joint constraints for multi-focus image fusion.  _Information Fusion_ , 66: 40–53.
  * Zhang et al. (2020a) Zhang, H.; Xu, H.; Xiao, Y.; Guo, X.; and Ma, J. 2020a.  Rethinking the image fusion: A fast unified image fusion network based on proportional maintenance of gradient and intensity.  In _Proceedings of the AAAI Conference on Artificial Intelligence_ , 12797–12804.
  * Zhang et al. (2018) Zhang, Q.; Liu, Y.; Blum, R. S.; Han, J.; and Tao, D. 2018.  Sparse representation based multi-sensor image fusion for multi-focus and multi-modality images: A review.  _Information Fusion_ , 40: 57–75.
  * Zhang et al. (2020b) Zhang, Y.; Liu, Y.; Sun, P.; Yan, H.; Zhao, X.; and Zhang, L. 2020b.  IFCNN: A general image fusion framework based on convolutional neural network.  _Information Fusion_ , 54: 99–118.
  * Zhou et al. (2016) Zhou, Z.; Dong, M.; Xie, X.; and Gao, Z. 2016.  Fusion of infrared and visible images for night-vision context enhancement.  _Applied optics_ , 55(23): 6480–6490.