Sophia: A Scalable Stochastic Second-order Optimizer for Language Model Pre-training
Paper • 2305.14342 • Published
Hong Liu
hongliu9903
AI & ML interests
None yet
Organizations
authored 4
papers 11 months ago
Chain of Thought Empowers Transformers to Solve Inherently Serial Problems
Paper • 2402.12875 • Published • 13
Same Pre-training Loss, Better Downstream: Implicit Bias Matters for Language Models
Paper • 2210.14199 • Published
MoCa: Modality-aware Continual Pre-training Makes Better Bidirectional Multimodal Embeddings
Paper • 2506.23115 • Published • 36