Fastformer
Fastformer
**Fastformer** is an type of [Transformer](https://paperswithcode.com/method/transformer) which uses [additive attention](https://www.paperswithcode.com/method/additive-attention) as a building block. Instead of modeling the pair-wise interactions between tokens, [additive attention](https://paperswithcode.com/method/additive-attention) is used to model global contexts, and then each token representation is further transformed based on its interaction with global context representations.
Fastformer: Additive Attention Can Be All You Need
https://paperswithcode.com/paper/fastformer-additive-attention-is-all-you-need
Fastformer: Additive Attention Can Be All You Need