DHEL
Decoupled Hyperspherical Energy Loss
InfoNCE variants demonstrate direct and indirect coupling between the alignment and uniformity terms thus hurting optimisation. The Decoupled Hyperspherical Energy Loss (DHEL) is an NT-Xent variant that completly decouples alignment from uniformity by discarding the corresponding terms from the denominator. In this way optimisation is more efficient and robust to hyper parameter changes.
Bridging Mini-Batch and Asymptotic Analysis in Contrastive Learning: From InfoNCE to Kernel-Based Losses
https://paperswithcode.com/paper/bridging-mini-batch-and-asymptotic-analysis
Bridging Mini-Batch and Asymptotic Analysis in Contrastive Learning: From InfoNCE to Kernel-Based Losses