Maxout
Maxout
The **Maxout Unit** is a generalization of the [ReLU](https://paperswithcode.com/method/relu) and the [leaky ReLU](https://paperswithcode.com/method/leaky-relu) functions. It is a piecewise linear function that returns the maximum of the inputs, designed to be used in conjunction with [dropout](https://paperswithcode.com/method/dropout). Both ReLU and leaky ReLU are special cases of Maxout. 

$$f\left(x\right) = \max\left(w^{T}\_{1}x + b\_{1}, w^{T}\_{2}x + b\_{2}\right)$$

The main drawback of Maxout is that it is computationally expensive as it doubles the number of parameters for each neuron.
Maxout Networks
https://paperswithcode.com/paper/maxout-networks
Maxout Networks