SiLU
Sigmoid Linear Unit
** Sigmoid Linear Units**, or **SiLUs**, are activation functions for
neural networks. The activation of the SiLU is computed by the sigmoid function multiplied by its input, or $$ x\sigma(x).$$

See [Gaussian Error Linear Units](https://arxiv.org/abs/1606.08415) ([GELUs](https://paperswithcode.com/method/gelu)) where the SiLU was originally coined, and see [Sigmoid-Weighted Linear Units for Neural Network Function Approximation in Reinforcement Learning](https://arxiv.org/abs/1702.03118) and [Swish: a Self-Gated Activation Function](https://arxiv.org/abs/1710.05941v1) where the SiLU was experimented with later.
Sigmoid-Weighted Linear Units for Neural Network Function Approximation in Reinforcement Learning
https://paperswithcode.com/paper/sigmoid-weighted-linear-units-for-neural
Sigmoid-Weighted Linear Units for Neural Network Function Approximation in Reinforcement Learning