Label Smoothing
Label Smoothing
**Label Smoothing** is a regularization technique that introduces noise for the labels. This accounts for the fact that datasets may have mistakes in them, so maximizing the likelihood of $\log{p}\left(y\mid{x}\right)$ directly can be harmful. Assume for a small constant $\epsilon$, the training set label $y$ is correct with probability $1-\epsilon$ and incorrect otherwise. Label Smoothing regularizes a model based on a [softmax](https://paperswithcode.com/method/softmax) with $k$ output values by replacing the hard $0$ and $1$ classification targets with targets of $\frac{\epsilon}{k}$ and $1-\frac{k-1}{k}\epsilon$ respectively.

Source: Deep Learning, Goodfellow et al

Image Source: [When Does Label Smoothing Help?](https://arxiv.org/abs/1906.02629)