Weak-to-Strong Preference Optimization: Stealing Reward from Weak Aligned Model Paper • 2410.18640 • Published Oct 24, 2024