BAGUA
BAGUA
**BAGUA** is a communication framework whose design goal is to provide a system abstraction that is both flexible and modular to support state-of-the-art system relaxation techniques of distributed training. The abstraction goes beyond parameter server and Allreduce paradigms, and provides a collection of MPI-style collective operations to facilitate communications with different precision and centralization strategies.
BAGUA: Scaling up Distributed Learning with System Relaxations
https://paperswithcode.com/paper/bagua-scaling-up-distributed-learning-with
BAGUA: Scaling up Distributed Learning with System Relaxations