Trans-Encoder
Trans-Encoder
Unsupervised knowledge distillation from a pretrained language model to *itself*, by alternating between its bi- and cross-encoder forms.
Trans-Encoder: Unsupervised sentence-pair modelling through self- and mutual-distillations
https://paperswithcode.com/paper/trans-encoder-unsupervised-sentence-pair
Trans-Encoder: Unsupervised sentence-pair modelling through self- and mutual-distillations