SHA-RNN
Single Headed Attention RNN
**SHA-RNN**, or **Single Headed Attention RNN**, is a recurrent neural network, and language model when combined with an embedding input and [softmax](https://paperswithcode.com/method/softmax) classifier, based on a core [LSTM](https://paperswithcode.com/method/lstm) component and a [single-headed attention](https://paperswithcode.com/method/single-headed-attention) module. Other design choices include a Boom feedforward layer and the use of [layer normalization](https://paperswithcode.com/method/layer-normalization). The guiding principles of the author were to ensure simplicity in the architecture and to keep computational costs bounded (the model was originally trained with a single GPU).
Single Headed Attention RNN: Stop Thinking With Your Head
https://paperswithcode.com/paper/single-headed-attention-rnn-stop-thinking
Single Headed Attention RNN: Stop Thinking With Your Head