L1 Regularization
L1 Regularization
**$L_{1}$ Regularization** is a regularization technique applied to the weights of a neural network. We minimize a loss function compromising both the primary loss function and a penalty on the $L\_{1}$ Norm of the weights:

$$L\_{new}\left(w\right) = L\_{original}\left(w\right) + \lambda{||w||}\_{1}$$

where $\lambda$ is a value determining the strength of the penalty. In contrast to [weight decay](https://paperswithcode.com/method/weight-decay), $L_{1}$ regularization promotes sparsity; i.e. some parameters have an optimal value of zero.

Image Source: [Wikipedia](https://en.wikipedia.org/wiki/Regularization_(mathematics)#/media/File:Sparsityl1.png)