Linear Warmup
Linear Warmup
**Linear Warmup** is a learning rate schedule where we linearly increase the learning rate from a low rate to a constant rate thereafter. This reduces volatility in the early stages of training.

Image Credit: [Chengwei Zhang](https://www.dlology.com/about-me/)