RGA
Relation-aware Global Attention
In relation-aware global attention (RGA) stresses the importance of global structural information provided by pairwise relations, and uses it to produce attention maps. 

RGA comes in two forms,  spatial RGA (RGA-S) and channel RGA (RGA-C). RGA-S first reshapes the input feature map $X$ to $C\times (H\times W)$ and the pairwise relation matrix $R \in \mathbb{R}^{(H\times W)\times (H\times W)}$ is computed using 
\begin{align}
    Q &= \delta(W^QX) 
\end{align}
\begin{align}
    K &= \delta(W^KX) 
\end{align}
\begin{align}
    R &= Q^TK
\end{align}
The relation vector $r_i$ at position $i$ is defined by stacking  pairwise relations at all positions:
\begin{align}
    r_i = [R(i, :); R(:,i)]    
\end{align}
and the spatial relation-aware feature $y_i$ can be written as
\begin{align}
    Y_i = [g^c_\text{avg}(\delta(W^\varphi x_i)); \delta(W^\phi r_i)]
\end{align}
where $g^c_\text{avg}$ denotes global average pooling in the channel domain. Finally, the spatial attention score at position $i$ is given by 
\begin{align}
    a_i = \sigma(W_2\delta(W_1y_i))
\end{align}
RGA-C has the same form as RGA-S, except for taking the input feature map as a set of $H\times W$-dimensional features.

RGA uses global relations to generate the attention score for each feature node,  so provides valuable structural information and significantly enhances the representational power. RGA-S and RGA-C are flexible enough to be used in any CNN network; Zhang et al. propose  using them jointly in sequence to better capture both spatial and cross-channel relationships.
Relation-Aware Global Attention for Person Re-identification
https://paperswithcode.com/paper/relation-aware-global-attention
Relation-Aware Global Attention for Person Re-identification