Linear Warmup With Linear Decay
Linear Warmup With Linear Decay
**Linear Warmup With Linear Decay** is a learning rate schedule in which we increase the learning rate linearly for $n$ updates and then linearly decay afterwards.