HardELiSH
HardELiSH
**HardELiSH** is an activation function for neural networks.  The HardELiSH is a multiplication of the [HardSigmoid](https://paperswithcode.com/method/hard-sigmoid) and [ELU](https://paperswithcode.com/method/elu) in the negative part and a multiplication of the Linear and the HardSigmoid in the positive
part:

$$f\left(x\right) = x\max\left(0, \min\left(1, \left(\frac{x+1}{2}\right)\right) \right) \text{ if } x \geq 1$$
$$f\left(x\right) = \left(e^{x}-1\right)\max\left(0, \min\left(1, \left(\frac{x+1}{2}\right)\right)\right) \text{ if } x < 0 $$

Source: [Activation Functions](https://arxiv.org/pdf/1811.03378.pdf)
The Quest for the Golden Activation Function
https://paperswithcode.com/paper/the-quest-for-the-golden-activation-function
The Quest for the Golden Activation Function