Exponential Decay
Exponential Decay
**Exponential Decay** is a learning rate schedule where we decay the learning rate with more iterations using an exponential function:

$$ \text{lr} = \text{lr}\_{0}\exp\left(-kt\right) $$

Image Credit: [Suki Lau](https://towardsdatascience.com/learning-rate-schedules-and-adaptive-learning-rate-methods-for-deep-learning-2c8f433990d1)