BatchChannel Normalization
BatchChannel Normalization
**Batch-Channel Normalization**, or **BCN**, uses batch knowledge to prevent channel-normalized models from getting too close to "elimination singularities". Elimination singularities correspond to the points on the training trajectory where neurons become consistently deactivated. They cause degenerate manifolds in the loss landscape which will slow down training and harm model performances.
Rethinking Normalization and Elimination Singularity in Neural Networks
https://paperswithcode.com/paper/rethinking-normalization-and-elimination
Rethinking Normalization and Elimination Singularity in Neural Networks