CoLU
Collapsing Linear Unit
CoLU is an activation function similar to Swish and Mish in properties. It is defined as:
$$f(x)=\frac{x}{1-x^{-(x+e^x)}}$$
It is smooth, continuously differentiable, unbounded above, bounded below, non-saturating, and non-monotonic. Based on experiments done with CoLU with different activation functions, it is observed that CoLU usually performs better than other functions on deeper neural networks.
Deeper Learning with CoLU Activation
https://paperswithcode.com/paper/deeper-learning-with-colu-activation
Deeper Learning with CoLU Activation