k-Sparse Autoencoder
k-Sparse Autoencoder
**k-Sparse Autoencoders** are autoencoders with linear activation function, where in hidden layers only the $k$ highest activities are kept. This achieves exact sparsity in the hidden representation. Backpropagation only goes through the the top $k$ activated units. This can be achieved with a [ReLU](https://paperswithcode.com/method/relu) layer with an adjustable threshold.
k-Sparse Autoencoders
https://paperswithcode.com/paper/k-sparse-autoencoders
k-Sparse Autoencoders