SimCSE
SimCSE
**SimCSE** is a contrastive learning framework for generating sentence embeddings. It utilizes an unsupervised approach, which takes an input sentence and predicts itself in contrastive objective, with only standard [dropout](https://paperswithcode.com/method/dropout) used as noise. The authors find that dropout acts as minimal “data augmentation” of hidden representations, while removing it leads to a representation collapse. Afterwards a supervised approach is used, which incorporates annotated pairs from natural language inference datasets into the contrastive framework, by using “entailment” pairs as positives and “contradiction
SimCSE: Simple Contrastive Learning of Sentence Embeddings
https://paperswithcode.com/paper/simcse-simple-contrastive-learning-of
SimCSE: Simple Contrastive Learning of Sentence Embeddings