Activation Normalization
Activation Normalization
**Activation Normalization** is a type of normalization used for flow-based generative models; specifically it was introduced in the [GLOW](https://paperswithcode.com/method/glow) architecture. An ActNorm layer performs an affine transformation of the activations using a scale and bias parameter per channel, similar to [batch normalization](https://paperswithcode.com/method/batch-normalization). These parameters are initialized such that the post-actnorm activations per-channel have zero mean and unit variance given an initial minibatch of data. This is a form of data dependent initilization. After initialization, the scale and bias are treated as regular trainable parameters that are independent of the data.
Glow: Generative Flow with Invertible 1x1 Convolutions
https://paperswithcode.com/paper/glow-generative-flow-with-invertible-1x1
Glow: Generative Flow with Invertible 1x1 Convolutions