Barlow Twins
Barlow Twins
**Barlow Twins** is a self-supervised learning method that applies redundancy-reduction — a principle first proposed in neuroscience — to self supervised learning. The objective function measures the cross-correlation matrix between the embeddings of two identical networks fed with distorted versions of a batch of samples, and tries to make this matrix close to the identity. This causes the embedding vectors of distorted version of a sample to be similar, while minimizing the redundancy between the components of these vectors. Barlow Twins does not require large batches nor asymmetry between the network twins such as a predictor network, gradient stopping, or a moving average on the weight updates. Intriguingly it benefits from very high-dimensional output vectors.
Barlow Twins: Self-Supervised Learning via Redundancy Reduction
https://paperswithcode.com/paper/barlow-twins-self-supervised-learning-via
Barlow Twins: Self-Supervised Learning via Redundancy Reduction