InPlace-ABN
In-Place Activated Batch Normalization
**In-Place Activated Batch Normalization**, or **InPlace-ABN**, substitutes the conventionally used succession of [BatchNorm](https://paperswithcode.com/method/batch-normalization) + Activation layers with a single plugin layer, hence avoiding invasive framework surgery while providing straightforward applicability for existing deep learning frameworks. It approximately halves the memory requirements during training of modern deep learning models.
In-Place Activated BatchNorm for Memory-Optimized Training of DNNs
https://paperswithcode.com/paper/in-place-activated-batchnorm-for-memory
In-Place Activated BatchNorm for Memory-Optimized Training of DNNs