Hopfield Layer
Hopfield Layer
A **Hopfield Layer** is a module that enables a network to associate two sets of vectors. This general functionality allows for [transformer](https://paperswithcode.com/method/transformer)-like self-attention, for decoder-encoder attention, for time series prediction (maybe with positional encoding), for sequence analysis, for multiple instance learning, for learning with point sets, for combining data sources by associations, for constructing a memory, for averaging and pooling operations, and for many more. 

In particular, the Hopfield layer can readily be used as plug-in replacement for existing layers like pooling layers ([max-pooling](https://paperswithcode.com/method/max-pooling) or [average pooling](https://paperswithcode.com/method/average-pooling), permutation equivariant layers, [GRU](https://paperswithcode.com/method/gru) & [LSTM](https://paperswithcode.com/method/lstm) layers, and attention layers. The Hopfield layer is based on modern Hopfield networks with continuous states that have very high storage capacity and converge after one update.
Hopfield Networks is All You Need
https://paperswithcode.com/paper/hopfield-networks-is-all-you-need
Hopfield Networks is All You Need