Hardtanh Activation
Hardtanh Activation
**Hardtanh** is an activation function used for neural networks:

$$ f\left(x\right) = -1 \text{ if } x < - 1 $$
$$ f\left(x\right) = x \text{ if } -1 \leq x \leq 1 $$
$$ f\left(x\right) = 1 \text{ if } x > 1 $$

It is a cheaper and more computationally efficient version of the [tanh activation](https://paperswithcode.com/method/tanh-activation).

Image Source: [Zhuan Lan](https://zhuanlan.zhihu.com/p/30385380)