Reformer
Reformer
**Reformer** is a [Transformer](https://paperswithcode.com/method/transformer) based architecture that seeks to make efficiency improvements. [Dot-product attention](https://paperswithcode.com/method/dot-product-attention) is replaced by one that uses locality-sensitive hashing, changing its complexity
from O($L^2$) to O($L\log L$), where $L$ is the length of the sequence. Furthermore, Reformers use reversible residual layers instead of the standard residuals, which allows storing activations only once in the training process instead of $N$ times, where $N$ is the number of layers.
Reformer: The Efficient Transformer
https://paperswithcode.com/paper/reformer-the-efficient-transformer-1
Reformer: The Efficient Transformer