Polyak Averaging
Polyak Averaging
**Polyak Averaging** is an optimization technique that sets final parameters to an average of (recent) parameters visited in the optimization trajectory. Specifically if in $t$ iterations we have parameters $\theta\_{1}, \theta\_{2}, \dots, \theta\_{t}$, then Polyak Averaging suggests setting 

$$ \theta\_t =\frac{1}{t}\sum\_{i}\theta\_{i} $$

Image Credit: [Shubhendu Trivedi & Risi Kondor](https://ttic.uchicago.edu/~shubhendu/Pages/Files/Lecture6_flat.pdf)