AGLU
Adaptive Generalised Linear Unit
The Adaptive Parametric activation (APA) is defined as: $APA(z,λ,κ) = (λ exp(−κz) + 1) ^{\frac{1}{−λ}}$, where $λ$ and $κ$ are learnable parameters. 
This activation function is a generalisation of the Sigmoid and the Gumbel activation functions  and it is expressive and versatile. For example, APA can be used inside the channel attention mechanism instead of the Sigmoid activation,
or it  can be used inside the intermediate layers using the Adaptive Generalised Linear Unit (AGLU): $AGLU(z,λ,κ) = z APA(z,λ,κ)$.
Adaptive Parametric Activation
https://paperswithcode.com/paper/adaptive-parametric-activation
Adaptive Parametric Activation