DEQ
Deep Equilibrium Models
A new kind of implicit models, where the output of the network is defined as the solution to an "infinite-level" fixed point equation. Thanks to this we can compute the gradient of the output without activations and therefore with a significantly reduced memory footprint.
Deep Equilibrium Models
https://paperswithcode.com/paper/deep-equilibrium-models
Deep Equilibrium Models