Virtual Data Augmentation
Virtual Data Augmentation
**Virtual Data Augmentation**, or **VDA**, is a framework for robustly fine-tuning pre-trained language model. Based on the original token embeddings, a multinomial mixture for augmenting virtual data is constructed, where a masked language model guarantees the semantic relevance and the Gaussian noise provides the augmentation diversity. Furthermore, a regularized training strategy is proposed to balance the two aspects.
Virtual Data Augmentation: A Robust and General Framework for Fine-tuning Pre-trained Models
https://paperswithcode.com/paper/virtual-data-augmentation-a-robust-and
Virtual Data Augmentation: A Robust and General Framework for Fine-tuning Pre-trained Models