Sandwich Transformer
Sandwich Transformer
A **Sandwich Transformer** is a variant of a [Transformer](https://paperswithcode.com/method/transformer) that reorders sublayers in the architecture to achieve better performance. The reordering is based on the authors' analysis that models with more self-attention toward the bottom and more
feedforward sublayers toward the top tend to perform better in general.
Improving Transformer Models by Reordering their Sublayers
https://paperswithcode.com/paper/improving-transformer-models-by-reordering
Improving Transformer Models by Reordering their Sublayers