Welcome to Echo’s documentation!¶
Table of Contents
- About
- Installation
- Torch Examples
- PyTorch Extensions API
- Echo.Activation.Torch.aria2
- Echo.Activation.Torch.mish
- Echo.Activation.Torch.beta_mish
- Echo.Activation.Torch.silu
- Echo.Activation.Torch.eswish
- Echo.Activation.Torch.swish
- Echo.Activation.Torch.elish
- Echo.Activation.Torch.hard_elish
- Echo.Activation.Torch.mila
- Echo.Activation.Torch.sine_relu
- Echo.Activation.Torch.fts
- Echo.Activation.Torch.sqnl
- Echo.Activation.Torch.isru
- Echo.Activation.Torch.isrlu
- Echo.Activation.Torch.bent_id
- Echo.Activation.Torch.soft_clipping
- Echo.Activation.Torch.weightedTanh
- Echo.Activation.Torch.srelu
- Echo.Activation.Torch.brelu
- Echo.Activation.Torch.apl
- Echo.Activation.Torch.soft_exponential
- Echo.Activation.Torch.maxout
- Echo.Activation.Torch.functional
- Keras Extensions API
- Indices and tables
About¶
Echo Package is created to provide an implementation of the most promising mathematical algorithms, which are missing in the most popular deep learning libraries, such as PyTorch, Keras and TensorFlow.
Implemented Activation Functions¶
List of activation functions implemented in Echo:
- PyTorch:
- Weighted Tanh (see
Echo.Activation.Torch.weightedTanh
)- Aria2 (see
Echo.Activation.Torch.aria2
)- SiLU (see
Echo.Activation.Torch.silu
)- E-Swish (see
Echo.Activation.Torch.eswish
)- Swish (see
Echo.Activation.Torch.swish
)- ELiSH (see
Echo.Activation.Torch.elish
)- Hard ELiSH (see
Echo.Activation.Torch.hard_elish
)- Mila (see
Echo.Activation.Torch.mila
)- SineReLU (see
Echo.Activation.Torch.sine_relu
)- Flatten T-Swish (see
Echo.Activation.Torch.fts
)- SQNL (see
Echo.Activation.Torch.sqnl
)- Mish (see
Echo.Activation.Torch.mish
)- ISRU (see
Echo.Activation.Torch.isru
)- ISRLU (see
Echo.Activation.Torch.isrlu
)- Bent’s Identity (see
Echo.Activation.Torch.bent_id
)- Soft Clipping (see
Echo.Activation.Torch.soft_clipping
)- SReLU (see
Echo.Activation.Torch.srelu
)- BReLU (see
Echo.Activation.Torch.brelu
)- APL (see
Echo.Activation.Torch.apl
)- Soft Exponential (see
Echo.Activation.Torch.soft_exponential
)- Maxout (see
Echo.Activation.Torch.maxout
)- Beta Mish (see
Echo.Activation.Torch.beta_mish
)
- Keras:
- Swish (see
Echo.Activation.Keras.custom_activations.swish()
)- ESwish (see
Echo.Activation.Keras.custom_activations.eswish()
)- ISRU (see
Echo.Activation.Keras.custom_activations.isru()
)- Beta Mish (see
Echo.Activation.Keras.custom_activations.beta_mish()
)- Mish (see
Echo.Activation.Keras.custom_activations.mish()
)- Mila (see
Echo.Activation.Keras.custom_activations.mila()
)- SQNL (see
Echo.Activation.Keras.custom_activations.sqnl()
)- Flatten T-Swish (see
Echo.Activation.Keras.custom_activations.fts()
)- ELiSH (see
Echo.Activation.Keras.custom_activations.elish()
)- Hard ELiSH (see
Echo.Activation.Keras.custom_activations.hard_elish()
)- Bent’s Identity (see
Echo.Activation.Keras.custom_activations.bent_id()
)- Weighted Tanh (see
Echo.Activation.Keras.custom_activations.weighted_tanh()
)- Sine ReLU (see
Echo.Activation.Keras.custom_activations.sineReLU()
)- ISRLU (see
Echo.Activation.Keras.custom_activations.isrlu()
)- Soft Clipping (see
Echo.Activation.Keras.custom_activations.soft_clipping()
)- Aria2 (see
Echo.Activation.Keras.custom_activations.aria2()
)
Installation¶
To install Echo package follow the instructions below:
- Clone or download GitHub repository.
- Navigate to Echo folder:
>>> $ cd Echo
- Install the package with pip:
>>> $ pip install .
Torch Examples¶
Simple Activation Functions¶
The following code block contains an example of usage of an activation function from Echo package:
# import activations from Echo
from Echo.Activation.Torch.weightedTanh import weightedTanh
import Echo.Activation.Torch.functional as Func
# use activations in layers of model defined in class
class Classifier(nn.Module):
def __init__(self):
super().__init__()
# initialize layers
self.fc1 = nn.Linear(784, 256)
self.fc2 = nn.Linear(256, 128)
self.fc3 = nn.Linear(128, 64)
self.fc4 = nn.Linear(64, 10)
def forward(self, x):
# make sure the input tensor is flattened
x = x.view(x.shape[0], -1)
# apply activation function from Echo
x = Func.weighted_tanh(self.fc1(x), weight = 1)
x = F.relu(self.fc2(x))
x = F.relu(self.fc3(x))
x = F.log_softmax(self.fc4(x), dim=1)
return x
def main():
# Initialize the model using defined Classifier class
model = Classifier()
# Create model with Sequential
model = nn.Sequential(OrderedDict([
('fc1', nn.Linear(784, 256)),
# use activation function from Echo
('wtahn1', weightedTanh(weight = 1)),
('fc2', nn.Linear(256, 128)),
('bn2', nn.BatchNorm1d(num_features=128)),
('relu2', nn.ReLU()),
('dropout', nn.Dropout(0.3)),
('fc3', nn.Linear(128, 64)),
('bn3', nn.BatchNorm1d(num_features=64)),
('relu3', nn.ReLU()),
('logits', nn.Linear(64, 10)),
('logsoftmax', nn.LogSoftmax(dim=1))]))
PyTorch Extensions API¶
Echo.Activation.Torch.aria2¶
Applies the Aria-2 function element-wise:
See Aria paper: https://arxiv.org/abs/1805.08878
-
class
Echo.Activation.Torch.aria2.
aria2
(beta=1, alpha=1.5)¶ Applies the Aria-2 function element-wise:
\[Aria2(x, \alpha, \beta) = (1+e^{-\beta*x})^{-\alpha}\]Plot:
- Shape:
- Arguments:
- alpha: hyper-parameter which has a two-fold effect; it reduces the curvature in 3rd quadrant as well as increases the curvature in first quadrant while lowering the value of activation (default = 1)
- beta: the exponential growth rate (default = 0.5)
- References:
- See Aria paper:
- https://arxiv.org/abs/1805.08878
- Examples:
>>> m = aria2(beta=0.5, alpha=1) >>> input = torch.randn(2) >>> output = m(input)
-
forward
(input)¶ Forward pass of the function.
Echo.Activation.Torch.mish¶
Applies the mish function element-wise:
-
class
Echo.Activation.Torch.mish.
mish
(inplace=False)¶ Applies the mish function element-wise:
\[mish(x) = x * tanh(softplus(x)) = x * tanh(ln(1 + e^{x}))\]Plot:
- Shape:
- Arguments:
- inplace: (bool) perform the operation in-place
- Examples:
>>> m = mish() >>> input = torch.randn(2) >>> output = m(input)
-
forward
(input)¶ Forward pass of the function.
Echo.Activation.Torch.beta_mish¶
Applies the β mish function element-wise:
-
class
Echo.Activation.Torch.beta_mish.
beta_mish
(beta=1.5)¶ Applies the β mish function element-wise:
\[\beta mish(x) = x * tanh(ln((1 + e^{x})^{\beta}))\]Plot:
- Shape:
- Arguments:
- beta: hyperparameter (default = 1.5)
- References
- β-Mish: An uni-parametric adaptive activation function derived from Mish:
- Examples:
>>> m = beta_mish(beta=1.5) >>> input = torch.randn(2) >>> output = m(input)
-
forward
(input)¶ Forward pass of the function.
Echo.Activation.Torch.silu¶
Applies the Sigmoid Linear Unit (SiLU) function element-wise:
See related paper: https://arxiv.org/pdf/1606.08415.pdf
-
class
Echo.Activation.Torch.silu.
silu
(inplace=False)¶ Applies the Sigmoid Linear Unit (SiLU) function element-wise:
\[silu(x) = x * sigmoid(x)\]Plot:
- Shape:
- Arguments:
- inplace - (bool) if inplace == True operation is performed inplace
- References:
- Related paper:
- Examples:
>>> m = silu(inplace = False) >>> input = torch.randn(2) >>> output = m(input)
-
forward
(input)¶ Forward pass of the function.
Echo.Activation.Torch.eswish¶
Applies the E-Swish function element-wise:
See E-Swish paper: https://arxiv.org/abs/1801.07145
-
class
Echo.Activation.Torch.eswish.
eswish
(beta=1.75)¶ Applies the E-Swish function element-wise:
\[ESwish(x, \beta) = \beta*x*sigmoid(x)\]Plot:
- Shape:
- Arguments:
- beta: a constant parameter (default value = 1.375)
- References:
- See related paper:
- Examples:
>>> m = eswish(beta=1.375) >>> input = torch.randn(2) >>> output = m(input)
-
forward
(input)¶ Forward pass of the function.
Echo.Activation.Torch.swish¶
Applies the Swish function element-wise:
See Swish paper: https://arxiv.org/pdf/1710.05941.pdf
-
class
Echo.Activation.Torch.swish.
swish
(beta=1.25)¶ Applies the Swish function element-wise:
\[Swish(x, \beta) = x*sigmoid(\beta*x) = \frac{x}{(1+e^{-\beta*x})}\]Plot:
- Shape:
- Arguments:
- beta: hyperparameter, which controls the shape of the bump (default = 1.25)
- References:
- See Swish paper:
- Examples:
>>> m = swish(beta=1.25) >>> input = torch.randn(2) >>> output = m(input)
-
forward
(input)¶ Forward pass of the function.
Echo.Activation.Torch.elish¶
Applies the ELiSH (Exponential Linear Sigmoid SquasHing) function element-wise:
See ELiSH paper: https://arxiv.org/pdf/1808.00783.pdf
-
class
Echo.Activation.Torch.elish.
elish
¶ Applies the ELiSH (Exponential Linear Sigmoid SquasHing) function element-wise:
\[\begin{split}ELiSH(x) = \left\{\begin{matrix} x / (1+e^{-x}), x \geq 0 \\ (e^{x} - 1) / (1 + e^{-x}), x < 0 \end{matrix}\right.\end{split}\]Plot:
- Shape:
- References:
- Related paper:
- Examples:
>>> m = elish() >>> input = torch.randn(2) >>> output = m(input)
-
forward
(input)¶ Forward pass of the function.
Echo.Activation.Torch.hard_elish¶
Applies the HardELiSH function element-wise:
See HardELiSH paper: https://arxiv.org/pdf/1808.00783.pdf
-
class
Echo.Activation.Torch.hard_elish.
hard_elish
¶ Applies the HardELiSH function element-wise:
\[\begin{split}HardELiSH(x) = \left\{\begin{matrix} x \times max(0, min(1, (x + 1) / 2)), x \geq 0 \\ (e^{x} - 1)\times max(0, min(1, (x + 1) / 2)), x < 0 \end{matrix}\right.\end{split}\]Plot:
- Shape:
- References:
- See HardELiSH paper:
- Examples:
>>> m = hard_elish() >>> input = torch.randn(2) >>> output = m(input)
-
forward
(input)¶ Forward pass of the function.
Echo.Activation.Torch.mila¶
Applies the Mila function element-wise:
Refer to: https://github.com/digantamisra98/Mila
-
class
Echo.Activation.Torch.mila.
mila
(beta=-0.25)¶ Applies the Mila function element-wise:
\[mila(x) = x * tanh(ln(1 + e^{\beta + x})) = x * tanh(softplus(\beta + x)\]Plot:
- Shape:
- Arguments:
- beta: scale to control the concavity of the global minima of the function (default = -0.25)
- References:
- Examples:
>>> m = mila(beta=-0.25) >>> input = torch.randn(2) >>> output = m(input)
-
forward
(input)¶ Forward pass of the function.
Echo.Activation.Torch.sine_relu¶
Applies the SineReLU function element-wise:
See related Medium article<EMAIL_ADDRESS>-
class
Echo.Activation.Torch.sine_relu.
sine_relu
(epsilon=0.01)¶ Applies the SineReLU function element-wise:
\[\begin{split}SineReLU(x, \epsilon) = \left\{\begin{matrix} x , x > 0 \\ \epsilon * (sin(x) - cos(x)), x \leq 0 \end{matrix}\right.\end{split}\]Plot:
- Shape:
- Arguments:
- epsilon: hyperparameter (default = 0.01) used to control the wave amplitude
- References:
- See related Medium article:
- Examples:
>>> m = sine_relu() >>> input = torch.randn(2) >>> output = m(input)
-
forward
(input)¶ Forward pass of the function.
Echo.Activation.Torch.fts¶
Applies the FTS (Flatten T-Swish) function element-wise:
See Flatten T-Swish paper: https://arxiv.org/pdf/1812.06247.pdf
-
class
Echo.Activation.Torch.fts.
fts
¶ Applies the FTS (Flatten T-Swish) function element-wise:
\[\begin{split}FTS(x) = \left\{\begin{matrix} \frac{x}{1 + e^{-x}} , x \geq 0 \\ 0, x < 0 \end{matrix}\right.\end{split}\]Plot:
- Shape:
- References:
- See Flattened T-Swish paper:
- Examples:
>>> m = fts() >>> input = torch.randn(2) >>> output = m(input)
-
forward
(input)¶ Forward pass of the function.
Echo.Activation.Torch.sqnl¶
Applies the SQNL function element-wise:
See SQNL paper: https://ieeexplore.ieee.org/document/8489043
-
class
Echo.Activation.Torch.sqnl.
sqnl
¶ Applies the SQNL function element-wise:
\[\begin{split}SQNL(x) = \left\{\begin{matrix} 1, x > 2 \\ x - \frac{x^2}{4}, 0 \leq x \leq 2 \\ x + \frac{x^2}{4}, -2 \leq x < 0 \\ -1, x < -2 \end{matrix}\right.\end{split}\]Plot:
- Shape:
- References:
- See SQNL paper:
- Examples:
>>> m = sqnl() >>> input = torch.randn(2) >>> output = m(input)
-
forward
(input)¶ Forward pass of the function.
Echo.Activation.Torch.isru¶
Applies the ISRU (Inverse Square Root Unit) function element-wise:
ISRU paper: https://arxiv.org/pdf/1710.09967.pdf
-
class
Echo.Activation.Torch.isru.
isru
(alpha=1.0)¶ Applies the ISRU function element-wise:
\[ISRU(x) = \frac{x}{\sqrt{1 + \alpha * x^2}}\]Plot:
- Shape:
- Arguments:
- alpha: A constant (default = 1.0)
- References:
- ISRU paper:
- Examples:
>>> m = isru(alpha=1.0) >>> input = torch.randn(2) >>> output = m(input)
-
forward
(input)¶ Forward pass of the function.
Echo.Activation.Torch.isrlu¶
Applies the ISRLU (Inverse Square Root Linear Unit) function element-wise:
ISRLU paper: https://arxiv.org/pdf/1710.09967.pdf
-
class
Echo.Activation.Torch.isrlu.
isrlu
(alpha=1.0)¶ Applies the ISRLU function element-wise:
\[\begin{split}ISRLU(x)=\left\{\begin{matrix} x, x\geq 0 \\ x * (\frac{1}{\sqrt{1 + \alpha*x^2}}), x <0 \end{matrix}\right.\end{split}\]Plot:
- Shape:
- Arguments:
- alpha: hyperparameter α controls the value to which an ISRLU saturates for negative inputs (default = 1)
- References:
- ISRLU paper: https://arxiv.org/pdf/1710.09967.pdf
- Examples:
>>> m = isrlu(alpha=1.0) >>> input = torch.randn(2) >>> output = m(input)
-
forward
(input)¶ Forward pass of the function.
Echo.Activation.Torch.bent_id¶
Applies the Bent’s Identity function element-wise:
Echo.Activation.Torch.soft_clipping¶
Applies Soft Clipping function element-wise:
See SC paper: https://arxiv.org/pdf/1810.11509.pdf
-
class
Echo.Activation.Torch.soft_clipping.
soft_clipping
(alpha=0.5)¶ Applies the Soft Clipping function element-wise:
\[SC(x) = 1 / \alpha * log(\frac{1 + e^{\alpha * x}}{1 + e^{\alpha * (x-1)}})\]Plot:
- Shape:
- Arguments:
- alpha: hyper-parameter, which determines how close to linear the central region is and how sharply the linear region turns to the asymptotic values
- References:
- See SC paper:
- https://arxiv.org/pdf/1810.11509.pdf
- Examples:
>>> m = soft_clipping(alpha=0.5) >>> input = torch.randn(2) >>> output = m(input)
-
forward
(input)¶ Forward pass of the function.
Echo.Activation.Torch.weightedTanh¶
Applies the weighted tanh function element-wise:
-
class
Echo.Activation.Torch.weightedTanh.
weightedTanh
(weight=1, inplace=False)¶ Applies the weighted tanh function element-wise:
\[weightedtanh(x) = tanh(x * weight)\]Plot:
- Shape:
- Arguments:
- weight: hyperparameter (default = 1.0)
- inplace: perform inplace operation (default = False)
- Examples:
>>> m = weightedTanh(weight = 1) >>> input = torch.randn(2) >>> output = m(input)
-
forward
(input)¶ Forward pass of the function.
Echo.Activation.Torch.srelu¶
Script defined the SReLU (S-shaped Rectified Linear Activation Unit):
See SReLU paper: https://arxiv.org/pdf/1512.07030.pdf
-
class
Echo.Activation.Torch.srelu.
srelu
(in_features, parameters=None)[source]¶ SReLU (S-shaped Rectified Linear Activation Unit): a combination of three linear functions, which perform mapping R → R with the following formulation:
\[\begin{split}h(x_i) = \left\{\begin{matrix} t_i^r + a_i^r(x_i - t_i^r), x_i \geq t_i^r \\ x_i, t_i^r > x_i > t_i^l\\ t_i^l + a_i^l(x_i - t_i^l), x_i \leq t_i^l \\ \end{matrix}\right.\end{split}\]with 4 trainable parameters.
- Shape:
Parameters:
\[\{t_i^r, a_i^r, t_i^l, a_i^l\}\]4 trainable parameters, which model an individual SReLU activation unit. The subscript i indicates that we allow SReLU to vary in different channels. Parameters can be initialized manually or randomly.
- References:
- See SReLU paper:
- Examples:
>>> srelu_activation = srelu((2,2)) >>> t = torch.randn((2,2), dtype=torch.float, requires_grad = True) >>> output = srelu_activation(t)
Echo.Activation.Torch.brelu¶
Script defined the BReLU (Bipolar Rectified Linear Activation Unit):
See BReLU paper: https://arxiv.org/pdf/1709.04054.pdf
-
class
Echo.Activation.Torch.brelu.
brelu
[source]¶ Implementation of BReLU activation function:
\[\begin{split}BReLU(x_i) = \left\{\begin{matrix} f(x_i), i \mod 2 = 0\\ - f(-x_i), i \mod 2 \neq 0 \end{matrix}\right.\end{split}\]Plot:
- Shape:
- References:
- See BReLU paper:
- Examples:
>>> brelu_activation = brelu.apply >>> t = torch.randn((5,5), dtype=torch.float, requires_grad = True) >>> t = brelu_activation(t)
-
static
backward
(ctx, grad_output)[source]¶ In the backward pass we receive a Tensor containing the gradient of the loss with respect to the output, and we need to compute the gradient of the loss with respect to the input.
-
static
forward
(ctx, input)[source]¶ In the forward pass we receive a Tensor containing the input and return a Tensor containing the output. ctx is a context object that can be used to stash information for backward computation. You can cache arbitrary objects for use in the backward pass using the ctx.save_for_backward method.
Echo.Activation.Torch.apl¶
Script defined the APL (ADAPTIVE PIECEWISE LINEAR UNITS):
See APL paper: https://arxiv.org/pdf/1412.6830.pdf
-
class
Echo.Activation.Torch.apl.
apl
(in_features, S, a=None, b=None)[source]¶ Implementation of APL (ADAPTIVE PIECEWISE LINEAR UNITS) unit:
\[APL(x_i) = max(0,x) + \sum_{s=1}^{S}{a_i^s * max(0, -x + b_i^s)}\]with trainable parameters a and b, parameter S should be set in advance.
- Shape:
- Parameters:
- S: hyperparameter, number of hinges to be set in advance
- a: trainable parameter, control the slopes of the linear segments
- b: trainable parameter, determine the locations of the hinges
- References:
- See APL paper:
- Examples:
>>> a1 = apl(256, S = 1) >>> x = torch.randn(256) >>> x = a1(x)
-
class
Echo.Activation.Torch.apl.
apl_function
[source]¶ Implementation of APL (ADAPTIVE PIECEWISE LINEAR UNITS) activation function:
\[APL(x_i) = max(0,x) + \sum_{s=1}^{S}{a_i^s * max(0, -x + b_i^s)}\]- Shape:
- Arguments:
- a: variables control the slopes of the linear segments
- b: variables determine the locations of the hinges
- References:
- See APL paper:
- Examples:
>>> apl_func = apl_function.apply >>> t = torch.tensor([[1.,1.],[0.,-1.]]) >>> t.requires_grad = True >>> S = 2 >>> a = torch.tensor([[[1.,1.],[1.,1.]],[[1.,1.],[1.,1.]]]) >>> b = torch.tensor([[[1.,1.],[1.,1.]],[[1.,1.],[1.,1.]]]) >>> t = apl_func(t, a, b)
-
static
backward
(ctx, grad_output)[source]¶ In the backward pass we receive a Tensor containing the gradient of the loss with respect to the output, and we need to compute the gradient of the loss with respect to the input.
-
static
forward
(ctx, input, a, b)[source]¶ In the forward pass we receive a Tensor containing the input and return a Tensor containing the output. ctx is a context object that can be used to stash information for backward computation. You can cache arbitrary objects for use in the backward pass using the ctx.save_for_backward method.
Echo.Activation.Torch.soft_exponential¶
Script implements soft exponential activation:
See related paper: https://arxiv.org/pdf/1602.01321.pdf
-
class
Echo.Activation.Torch.soft_exponential.
soft_exponential
(in_features, alpha=None)[source]¶ Implementation of soft exponential activation:
\[\begin{split}SoftExponential(x, \alpha) = \left\{\begin{matrix} - \frac{log(1 - \alpha(x + \alpha))}{\alpha}, \alpha < 0\\ x, \alpha = 0\\ \frac{e^{\alpha * x} - 1}{\alpha} + \alpha, \alpha > 0 \end{matrix}\right.\end{split}\]with trainable parameter alpha.
- Shape:
- Parameters:
- alpha - trainable parameter
- References:
- See related paper:
- Examples:
>>> a1 = soft_exponential(256) >>> x = torch.randn(256) >>> x = a1(x)
Echo.Activation.Torch.maxout¶
Script implements Maxout activation
See related paper: https://arxiv.org/pdf/1302.4389.pdf
See implementation: https://github.com/Usama113/Maxout-PyTorch/blob/master/Maxout.ipynb
-
class
Echo.Activation.Torch.maxout.
maxout
[source]¶ Implementation of Maxout:
\[maxout(\vec{x}) = max_i(x_i)\]- Shape:
- References:
- See Maxout paper:
https://arxiv.org/pdf/1302.4389.pdf
- Reference to the implementation:
https://github.com/Usama113/Maxout-PyTorch/blob/master/Maxout.ipynb
- Examples:
>>> a1 = maxout.apply >>> x = torch.randn(256) >>> x = a1(x)
-
static
backward
(ctx, grad_output)[source]¶ Defines a formula for differentiating the operation.
This function is to be overridden by all subclasses.
It must accept a context
ctx
as the first argument, followed by as many outputs didforward()
return, and it should return as many tensors, as there were inputs toforward()
. Each argument is the gradient w.r.t the given output, and each returned value should be the gradient w.r.t. the corresponding input.The context can be used to retrieve tensors saved during the forward pass. It also has an attribute
ctx.needs_input_grad
as a tuple of booleans representing whether each input needs gradient. E.g.,backward()
will havectx.needs_input_grad[0] = True
if the first input toforward()
needs gradient computated w.r.t. the output.
-
static
forward
(ctx, input)[source]¶ Performs the operation.
This function is to be overridden by all subclasses.
It must accept a context ctx as the first argument, followed by any number of arguments (tensors or other types).
The context can be used to store tensors that can be then retrieved during the backward pass.
Echo.Activation.Torch.functional¶
Script provides functional interface for custom activation functions.
-
Echo.Activation.Torch.functional.
aria2
(input, beta=1, alpha=1.5)¶ Applies the Aria-2 function element-wise:
\[Aria2(x, \alpha, \beta) = (1+e^{-\beta*x})^{-\alpha}\]See additional documentation for
Echo.Activation.Torch.aria2
.
-
Echo.Activation.Torch.functional.
bent_id
(input)¶ Applies the Bent’s Identity function element-wise:
\[bentId(x) = x + \frac{\sqrt{x^{2}+1}-1}{2}\]See additional documentation for
Echo.Activation.Torch.bent_id
.
-
Echo.Activation.Torch.functional.
beta_mish
(input, beta=1.5)¶ Applies the β mish function element-wise:
\[\beta mish(x) = x * tanh(ln((1 + e^{x})^{\beta}))\]See additional documentation for
Echo.Activation.Torch.beta_mish
.
-
Echo.Activation.Torch.functional.
elish
(input)¶ Applies the ELiSH (Exponential Linear Sigmoid SquasHing) function element-wise:
See additional documentation for
Echo.Activation.Torch.elish
.\[\begin{split}ELiSH(x) = \left\{\begin{matrix} x / (1+e^{-x}), x \geq 0 \\ (e^{x} - 1) / (1 + e^{-x}), x < 0 \end{matrix}\right.\end{split}\]See additional documentation for
Echo.Activation.Torch.elish
.
-
Echo.Activation.Torch.functional.
eswish
(input, beta=1.75)¶ Applies the E-Swish function element-wise:
\[ESwish(x, \beta) = \beta*x*sigmoid(x)\]See additional documentation for
Echo.Activation.Torch.eswish
.
-
Echo.Activation.Torch.functional.
fts
(input)¶ Applies the FTS (Flatten T-Swish) activation function element-wise:
\[\begin{split}FTS(x) = \left\{\begin{matrix} \frac{x}{1 + e^{-x}} , x \geq 0 \\ 0, x < 0 \end{matrix}\right.\end{split}\]See additional documentation for
Echo.Activation.Torch.fts
.
-
Echo.Activation.Torch.functional.
hard_elish
(input)¶ Applies the HardELiSH (Exponential Linear Sigmoid SquasHing) function element-wise:
\[\begin{split}HardELiSH(x) = \left\{\begin{matrix} x \times max(0, min(1, (x + 1) / 2)), x \geq 0 \\ (e^{x} - 1)\times max(0, min(1, (x + 1) / 2)), x < 0 \end{matrix}\right.\end{split}\]See additional documentation for
Echo.Activation.Torch.hard_elish
.
-
Echo.Activation.Torch.functional.
isrlu
(input, alpha=1.0)¶ Applies the ISRLU function element-wise:
\[\begin{split}ISRLU(x)=\left\{\begin{matrix} x, x\geq 0 \\ x * (\frac{1}{\sqrt{1 + \alpha*x^2}}), x <0 \end{matrix}\right.\end{split}\]See additional documentation for
Echo.Activation.Torch.isrlu
.
-
Echo.Activation.Torch.functional.
isru
(input, alpha=1.0)¶ Applies the ISRU function element-wise:
\[ISRU(x) = \frac{x}{\sqrt{1 + \alpha * x^2}}\]See additional documentation for
Echo.Activation.Torch.isru
.
-
Echo.Activation.Torch.functional.
mila
(input, beta=-0.25)¶ Applies the mila function element-wise:
\[mila(x) = x * tanh(softplus(\beta + x)) = x * tanh(ln(1 + e^{\beta + x}))\]See additional documentation for
Echo.Activation.Torch.mila
.
-
Echo.Activation.Torch.functional.
mish
(input, inplace=False)¶ Applies the mish function element-wise:
\[mish(x) = x * tanh(softplus(x)) = x * tanh(ln(1 + e^{x}))\]See additional documentation for
Echo.Activation.Torch.mish
.
-
Echo.Activation.Torch.functional.
silu
(input, inplace=False)¶ Applies the Sigmoid Linear Unit (SiLU) function element-wise:
\[SiLU(x) = x * sigmoid(x)\]See additional documentation for
Echo.Activation.Torch.silu
.
-
Echo.Activation.Torch.functional.
sineReLU
(input, eps=0.01)¶ Applies the SineReLU activation function element-wise:
\[\begin{split}SineReLU(x, \epsilon) = \left\{\begin{matrix} x , x > 0 \\ \epsilon * (sin(x) - cos(x)), x \leq 0 \end{matrix}\right.\end{split}\]See additional documentation for
Echo.Activation.Torch.sine_relu
.
-
Echo.Activation.Torch.functional.
soft_clipping
(input, alpha=0.5)¶ Applies the Soft Clipping function element-wise:
\[SC(x) = 1 / \alpha * log(\frac{1 + e^{\alpha * x}}{1 + e^{\alpha * (x-1)}})\]See additional documentation for
Echo.Activation.Torch.soft_clipping
.
-
Echo.Activation.Torch.functional.
sqnl
(input)¶ Applies the SQNL activation function element-wise:
\[\begin{split}SQNL(x) = \left\{\begin{matrix} 1, x > 2 \\ x - \frac{x^2}{4}, 0 \leq x \leq 2 \\ x + \frac{x^2}{4}, -2 \leq x < 0 \\ -1, x < -2 \end{matrix}\right.\end{split}\]See additional documentation for
Echo.Activation.Torch.sqnl
.
-
Echo.Activation.Torch.functional.
swish
(input, beta=1.25)¶ Applies the Swish function element-wise:
\[Swish(x, \beta) = x*sigmoid(\beta*x) = \frac{x}{(1+e^{-\beta*x})}\]See additional documentation for
Echo.Activation.Torch.swish
.
-
Echo.Activation.Torch.functional.
weighted_tanh
(input, weight=1, inplace=False)¶ Applies the weighted tanh function element-wise:
\[weightedtanh(x) = tanh(x * weight)\]See additional documentation for
Echo.Activation.Torch.weightedTanh
.
Keras Extensions API¶
Echo.Activation.Keras.custom_activations¶
Layers that act as activation functions.
-
class
Echo.Activation.Keras.custom_activations.
aria2
(alpha=1.0, beta=0.5, **kwargs)¶ Aria-2 Activation Function.
\[Aria2(x, \alpha, \beta) = (1+e^{-\beta*x})^{-\alpha}\]Plot:
- Shape:
- Input: Arbitrary. Use the keyword argument input_shape
(tuple of integers, does not include the samples axis) when using this layer as the first layer in a model.
- Output: Same shape as the input.
- Arguments:
- alpha: hyper-parameter which has a two-fold effect; it reduces the curvature in 3rd quadrant as well as increases the curvature in first quadrant while lowering the value of activation (default = 1)
- beta: the exponential growth rate (default = 0.5)
- References:
- See Aria paper:
- https://arxiv.org/abs/1805.08878
- Examples:
>>> X_input = Input(input_shape) >>> X =aria2(alpha=1.0, beta=0.5)(X_input)
-
call
(inputs)¶ This is where the layer’s logic lives.
- # Arguments
- inputs: Input tensor, or list/tuple of input tensors. **kwargs: Additional keyword arguments.
- # Returns
- A tensor or list/tuple of tensors.
-
compute_output_shape
(input_shape)¶ Computes the output shape of the layer.
Assumes that the layer will be built to match that input shape provided.
- # Arguments
- input_shape: Shape tuple (tuple of integers)
- or list of shape tuples (one per output tensor of the layer). Shape tuples can include None for free dimensions, instead of an integer.
- # Returns
- An input shape tuple.
-
get_config
()¶ Returns the config of the layer.
A layer config is a Python dictionary (serializable) containing the configuration of a layer. The same layer can be reinstantiated later (without its trained weights) from this configuration.
The config of a layer does not include connectivity information, nor the layer class name. These are handled by Network (one layer of abstraction above).
- # Returns
- Python dictionary.
-
class
Echo.Activation.Keras.custom_activations.
bent_id
(**kwargs)¶ Bent’s Identity Activation Function.
\[bentId(x) = x + \frac{\sqrt{x^{2}+1}-1}{2}\]Plot:
- Shape:
- Input: Arbitrary. Use the keyword argument input_shape
(tuple of integers, does not include the samples axis) when using this layer as the first layer in a model.
- Output: Same shape as the input.
- Examples:
>>> X_input = Input(input_shape) >>> X = bent_id()(X_input)
-
call
(inputs)¶ This is where the layer’s logic lives.
- # Arguments
- inputs: Input tensor, or list/tuple of input tensors. **kwargs: Additional keyword arguments.
- # Returns
- A tensor or list/tuple of tensors.
-
compute_output_shape
(input_shape)¶ Computes the output shape of the layer.
Assumes that the layer will be built to match that input shape provided.
- # Arguments
- input_shape: Shape tuple (tuple of integers)
- or list of shape tuples (one per output tensor of the layer). Shape tuples can include None for free dimensions, instead of an integer.
- # Returns
- An input shape tuple.
-
get_config
()¶ Returns the config of the layer.
A layer config is a Python dictionary (serializable) containing the configuration of a layer. The same layer can be reinstantiated later (without its trained weights) from this configuration.
The config of a layer does not include connectivity information, nor the layer class name. These are handled by Network (one layer of abstraction above).
- # Returns
- Python dictionary.
-
class
Echo.Activation.Keras.custom_activations.
beta_mish
(beta=1.5, **kwargs)¶ β mish activation function.
\[\beta mish(x) = x * tanh(ln((1 + e^{x})^{\beta}))\]Plot:
- Shape:
- Input: Arbitrary. Use the keyword argument input_shape
(tuple of integers, does not include the samples axis) when using this layer as the first layer in a model.
- Output: Same shape as the input.
- Arguments:
- beta: A constant or a trainable parameter (default = 1.5)
- References
- β-Mish: An uni-parametric adaptive activation function derived from Mish:
- Examples:
>>> X_input = Input(input_shape) >>> X = beta_mish(beta=1.5)(X_input)
-
call
(inputs)¶ This is where the layer’s logic lives.
- # Arguments
- inputs: Input tensor, or list/tuple of input tensors. **kwargs: Additional keyword arguments.
- # Returns
- A tensor or list/tuple of tensors.
-
compute_output_shape
(input_shape)¶ Computes the output shape of the layer.
Assumes that the layer will be built to match that input shape provided.
- # Arguments
- input_shape: Shape tuple (tuple of integers)
- or list of shape tuples (one per output tensor of the layer). Shape tuples can include None for free dimensions, instead of an integer.
- # Returns
- An input shape tuple.
-
get_config
()¶ Returns the config of the layer.
A layer config is a Python dictionary (serializable) containing the configuration of a layer. The same layer can be reinstantiated later (without its trained weights) from this configuration.
The config of a layer does not include connectivity information, nor the layer class name. These are handled by Network (one layer of abstraction above).
- # Returns
- Python dictionary.
-
class
Echo.Activation.Keras.custom_activations.
celu
(alpha=1.0, **kwargs)¶ CELU Activation Function.
\[CELU(x, \alpha) = max(0,x) + min(0,\alpha * (exp(x/ \alpha)-1))\]- Shape:
- Input: Arbitrary. Use the keyword argument input_shape
(tuple of integers, does not include the samples axis) when using this layer as the first layer in a model.
- Output: Same shape as the input.
- Arguments:
- alpha: the α value for the CELU formulation (default=1.0)
- References:
- See CELU paper:
- https://arxiv.org/abs/1704.07483
- Examples:
>>> X_input = Input(input_shape) >>> X = celu(alpha=1.0)(X_input)
-
call
(inputs)¶ This is where the layer’s logic lives.
- # Arguments
- inputs: Input tensor, or list/tuple of input tensors. **kwargs: Additional keyword arguments.
- # Returns
- A tensor or list/tuple of tensors.
-
compute_output_shape
(input_shape)¶ Computes the output shape of the layer.
Assumes that the layer will be built to match that input shape provided.
- # Arguments
- input_shape: Shape tuple (tuple of integers)
- or list of shape tuples (one per output tensor of the layer). Shape tuples can include None for free dimensions, instead of an integer.
- # Returns
- An input shape tuple.
-
get_config
()¶ Returns the config of the layer.
A layer config is a Python dictionary (serializable) containing the configuration of a layer. The same layer can be reinstantiated later (without its trained weights) from this configuration.
The config of a layer does not include connectivity information, nor the layer class name. These are handled by Network (one layer of abstraction above).
- # Returns
- Python dictionary.
-
class
Echo.Activation.Keras.custom_activations.
elish
(**kwargs)¶ ELiSH (Exponential Linear Sigmoid SquasHing) Activation Function.
\[\begin{split}ELiSH(x) = \left\{\begin{matrix} x / (1+e^{-x}), x \geq 0 \\ (e^{x} - 1) / (1 + e^{-x}), x < 0 \end{matrix}\right.\end{split}\]Plot:
- Shape:
- Input: Arbitrary. Use the keyword argument input_shape
(tuple of integers, does not include the samples axis) when using this layer as the first layer in a model.
- Output: Same shape as the input.
- References:
- Related paper:
- Examples:
>>> X_input = Input(input_shape) >>> X = elish()(X_input)
-
call
(inputs)¶ This is where the layer’s logic lives.
- # Arguments
- inputs: Input tensor, or list/tuple of input tensors. **kwargs: Additional keyword arguments.
- # Returns
- A tensor or list/tuple of tensors.
-
compute_output_shape
(input_shape)¶ Computes the output shape of the layer.
Assumes that the layer will be built to match that input shape provided.
- # Arguments
- input_shape: Shape tuple (tuple of integers)
- or list of shape tuples (one per output tensor of the layer). Shape tuples can include None for free dimensions, instead of an integer.
- # Returns
- An input shape tuple.
-
get_config
()¶ Returns the config of the layer.
A layer config is a Python dictionary (serializable) containing the configuration of a layer. The same layer can be reinstantiated later (without its trained weights) from this configuration.
The config of a layer does not include connectivity information, nor the layer class name. These are handled by Network (one layer of abstraction above).
- # Returns
- Python dictionary.
-
class
Echo.Activation.Keras.custom_activations.
eswish
(beta=1.375, **kwargs)¶ E-Swish Activation Function.
\[ESwish(x, \beta) = \beta*x*sigmoid(x)\]Plot:
- Shape:
- Input: Arbitrary. Use the keyword argument input_shape
(tuple of integers, does not include the samples axis) when using this layer as the first layer in a model.
- Output: Same shape as the input.
- Arguments:
- beta: a constant parameter (default value = 1.375)
- References:
- See related paper:
- Examples:
>>> X_input = Input(input_shape) >>> X = eswish(beta=0.5)(X_input)
-
call
(inputs)¶ This is where the layer’s logic lives.
- # Arguments
- inputs: Input tensor, or list/tuple of input tensors. **kwargs: Additional keyword arguments.
- # Returns
- A tensor or list/tuple of tensors.
-
compute_output_shape
(input_shape)¶ Computes the output shape of the layer.
Assumes that the layer will be built to match that input shape provided.
- # Arguments
- input_shape: Shape tuple (tuple of integers)
- or list of shape tuples (one per output tensor of the layer). Shape tuples can include None for free dimensions, instead of an integer.
- # Returns
- An input shape tuple.
-
get_config
()¶ Returns the config of the layer.
A layer config is a Python dictionary (serializable) containing the configuration of a layer. The same layer can be reinstantiated later (without its trained weights) from this configuration.
The config of a layer does not include connectivity information, nor the layer class name. These are handled by Network (one layer of abstraction above).
- # Returns
- Python dictionary.
-
class
Echo.Activation.Keras.custom_activations.
fts
(**kwargs)¶ FTS (Flatten T-Swish) Activation Function.
\[\begin{split}FTS(x) = \left\{\begin{matrix} \frac{x}{1 + e^{-x}} , x \geq 0 \\ 0, x < 0 \end{matrix}\right.\end{split}\]Plot:
- Shape:
- Input: Arbitrary. Use the keyword argument input_shape
(tuple of integers, does not include the samples axis) when using this layer as the first layer in a model.
- Output: Same shape as the input.
- References:
- Flatten T-Swish paper:
- Examples:
>>> X_input = Input(input_shape) >>> X = fts()(X_input)
-
call
(inputs)¶ This is where the layer’s logic lives.
- # Arguments
- inputs: Input tensor, or list/tuple of input tensors. **kwargs: Additional keyword arguments.
- # Returns
- A tensor or list/tuple of tensors.
-
compute_output_shape
(input_shape)¶ Computes the output shape of the layer.
Assumes that the layer will be built to match that input shape provided.
- # Arguments
- input_shape: Shape tuple (tuple of integers)
- or list of shape tuples (one per output tensor of the layer). Shape tuples can include None for free dimensions, instead of an integer.
- # Returns
- An input shape tuple.
-
get_config
()¶ Returns the config of the layer.
A layer config is a Python dictionary (serializable) containing the configuration of a layer. The same layer can be reinstantiated later (without its trained weights) from this configuration.
The config of a layer does not include connectivity information, nor the layer class name. These are handled by Network (one layer of abstraction above).
- # Returns
- Python dictionary.
-
class
Echo.Activation.Keras.custom_activations.
hard_elish
(**kwargs)¶ Hard ELiSH Activation Function.
\[\begin{split}HardELiSH(x) = \left\{\begin{matrix} x \times max(0, min(1, (x + 1) / 2)), x \geq 0 \\ (e^{x} - 1)\times max(0, min(1, (x + 1) / 2)), x < 0 \end{matrix}\right.\end{split}\]Plot:
- Shape:
- Input: Arbitrary. Use the keyword argument input_shape
(tuple of integers, does not include the samples axis) when using this layer as the first layer in a model.
- Output: Same shape as the input.
- References:
- Related paper:
- Examples:
>>> X_input = Input(input_shape) >>> X = hard_elish()(X_input)
-
call
(inputs)¶ This is where the layer’s logic lives.
- # Arguments
- inputs: Input tensor, or list/tuple of input tensors. **kwargs: Additional keyword arguments.
- # Returns
- A tensor or list/tuple of tensors.
-
compute_output_shape
(input_shape)¶ Computes the output shape of the layer.
Assumes that the layer will be built to match that input shape provided.
- # Arguments
- input_shape: Shape tuple (tuple of integers)
- or list of shape tuples (one per output tensor of the layer). Shape tuples can include None for free dimensions, instead of an integer.
- # Returns
- An input shape tuple.
-
get_config
()¶ Returns the config of the layer.
A layer config is a Python dictionary (serializable) containing the configuration of a layer. The same layer can be reinstantiated later (without its trained weights) from this configuration.
The config of a layer does not include connectivity information, nor the layer class name. These are handled by Network (one layer of abstraction above).
- # Returns
- Python dictionary.
-
class
Echo.Activation.Keras.custom_activations.
hard_shrink
(lambd=0.5, **kwargs)¶ Hard-Shrink Activation Function.
\[\begin{split}Hard-Shrink(x) = \left\{\begin{matrix} x, x > \lambda \\ 0 , - \lambda \leq x \leq \lambda \\ x, x <- \lambda \end{matrix}\right.\end{split}\]- Shape:
- Input: Arbitrary. Use the keyword argument input_shape
(tuple of integers, does not include the samples axis) when using this layer as the first layer in a model.
- Output: Same shape as the input.
- Arguments:
- lambda: the λ value for the Hardshrink formulation (default=0.5)
- Examples:
>>> X_input = Input(input_shape) >>> X = hard_shrink()(X_input)
-
call
(inputs)¶ This is where the layer’s logic lives.
- # Arguments
- inputs: Input tensor, or list/tuple of input tensors. **kwargs: Additional keyword arguments.
- # Returns
- A tensor or list/tuple of tensors.
-
compute_output_shape
(input_shape)¶ Computes the output shape of the layer.
Assumes that the layer will be built to match that input shape provided.
- # Arguments
- input_shape: Shape tuple (tuple of integers)
- or list of shape tuples (one per output tensor of the layer). Shape tuples can include None for free dimensions, instead of an integer.
- # Returns
- An input shape tuple.
-
get_config
()¶ Returns the config of the layer.
A layer config is a Python dictionary (serializable) containing the configuration of a layer. The same layer can be reinstantiated later (without its trained weights) from this configuration.
The config of a layer does not include connectivity information, nor the layer class name. These are handled by Network (one layer of abstraction above).
- # Returns
- Python dictionary.
-
class
Echo.Activation.Keras.custom_activations.
hard_tanh
(**kwargs)¶ Hard-TanH Activation Function.
\[\begin{split}Hard-TanH(x) = \left\{\begin{matrix} 1, x > 1 \\ x , -1 \leq x \leq 1 \\ -1, x <- 1 \end{matrix}\right.\end{split}\]- Shape:
- Input: Arbitrary. Use the keyword argument input_shape
(tuple of integers, does not include the samples axis) when using this layer as the first layer in a model.
- Output: Same shape as the input.
- Examples:
>>> X_input = Input(input_shape) >>> X = hard_tanh()(X_input)
-
call
(inputs)¶ This is where the layer’s logic lives.
- # Arguments
- inputs: Input tensor, or list/tuple of input tensors. **kwargs: Additional keyword arguments.
- # Returns
- A tensor or list/tuple of tensors.
-
compute_output_shape
(input_shape)¶ Computes the output shape of the layer.
Assumes that the layer will be built to match that input shape provided.
- # Arguments
- input_shape: Shape tuple (tuple of integers)
- or list of shape tuples (one per output tensor of the layer). Shape tuples can include None for free dimensions, instead of an integer.
- # Returns
- An input shape tuple.
-
get_config
()¶ Returns the config of the layer.
A layer config is a Python dictionary (serializable) containing the configuration of a layer. The same layer can be reinstantiated later (without its trained weights) from this configuration.
The config of a layer does not include connectivity information, nor the layer class name. These are handled by Network (one layer of abstraction above).
- # Returns
- Python dictionary.
-
class
Echo.Activation.Keras.custom_activations.
isrlu
(alpha=1.0, **kwargs)¶ ISRLU Activation Function.
\[\begin{split}ISRLU(x)=\left\{\begin{matrix} x, x\geq 0 \\ x * (\frac{1}{\sqrt{1 + \alpha*x^2}}), x <0 \end{matrix}\right.\end{split}\]Plot:
- Shape:
- Input: Arbitrary. Use the keyword argument input_shape
(tuple of integers, does not include the samples axis) when using this layer as the first layer in a model.
- Output: Same shape as the input.
- Arguments:
- alpha: hyperparameter α controls the value to which an ISRLU saturates for negative inputs (default = 1)
- References:
- ISRLU paper: https://arxiv.org/pdf/1710.09967.pdf
- Examples:
>>> X_input = Input(input_shape) >>> X = isrlu(alpha=1.0)(X_input)
-
call
(inputs)¶ This is where the layer’s logic lives.
- # Arguments
- inputs: Input tensor, or list/tuple of input tensors. **kwargs: Additional keyword arguments.
- # Returns
- A tensor or list/tuple of tensors.
-
compute_output_shape
(input_shape)¶ Computes the output shape of the layer.
Assumes that the layer will be built to match that input shape provided.
- # Arguments
- input_shape: Shape tuple (tuple of integers)
- or list of shape tuples (one per output tensor of the layer). Shape tuples can include None for free dimensions, instead of an integer.
- # Returns
- An input shape tuple.
-
get_config
()¶ Returns the config of the layer.
A layer config is a Python dictionary (serializable) containing the configuration of a layer. The same layer can be reinstantiated later (without its trained weights) from this configuration.
The config of a layer does not include connectivity information, nor the layer class name. These are handled by Network (one layer of abstraction above).
- # Returns
- Python dictionary.
-
class
Echo.Activation.Keras.custom_activations.
isru
(alpha=1.0, **kwargs)¶ ISRU (Inverse Square Root Unit) Activation Function.
\[ISRU(x) = \frac{x}{\sqrt{1 + \alpha * x^2}}\]Plot:
- Shape:
- Input: Arbitrary. Use the keyword argument input_shape
(tuple of integers, does not include the samples axis) when using this layer as the first layer in a model.
- Output: Same shape as the input.
- Arguments:
- alpha: A constant (default = 1.0)
- References:
- ISRU paper:
- Examples:
>>> X_input = Input(input_shape) >>> X = isru(alpha=0.5)(X_input)
-
call
(inputs)¶ This is where the layer’s logic lives.
- # Arguments
- inputs: Input tensor, or list/tuple of input tensors. **kwargs: Additional keyword arguments.
- # Returns
- A tensor or list/tuple of tensors.
-
compute_output_shape
(input_shape)¶ Computes the output shape of the layer.
Assumes that the layer will be built to match that input shape provided.
- # Arguments
- input_shape: Shape tuple (tuple of integers)
- or list of shape tuples (one per output tensor of the layer). Shape tuples can include None for free dimensions, instead of an integer.
- # Returns
- An input shape tuple.
-
get_config
()¶ Returns the config of the layer.
A layer config is a Python dictionary (serializable) containing the configuration of a layer. The same layer can be reinstantiated later (without its trained weights) from this configuration.
The config of a layer does not include connectivity information, nor the layer class name. These are handled by Network (one layer of abstraction above).
- # Returns
- Python dictionary.
-
class
Echo.Activation.Keras.custom_activations.
log_sigmoid
(**kwargs)¶ Log-Sigmoid Activation Function.
\[Log-Sigmoid(x) = log (\frac{1}{1+e^{-x}})\]- Shape:
- Input: Arbitrary. Use the keyword argument input_shape
(tuple of integers, does not include the samples axis) when using this layer as the first layer in a model.
- Output: Same shape as the input.
- Examples:
>>> X_input = Input(input_shape) >>> X = log_sigmoid()(X_input)
-
call
(inputs)¶ This is where the layer’s logic lives.
- # Arguments
- inputs: Input tensor, or list/tuple of input tensors. **kwargs: Additional keyword arguments.
- # Returns
- A tensor or list/tuple of tensors.
-
compute_output_shape
(input_shape)¶ Computes the output shape of the layer.
Assumes that the layer will be built to match that input shape provided.
- # Arguments
- input_shape: Shape tuple (tuple of integers)
- or list of shape tuples (one per output tensor of the layer). Shape tuples can include None for free dimensions, instead of an integer.
- # Returns
- An input shape tuple.
-
get_config
()¶ Returns the config of the layer.
A layer config is a Python dictionary (serializable) containing the configuration of a layer. The same layer can be reinstantiated later (without its trained weights) from this configuration.
The config of a layer does not include connectivity information, nor the layer class name. These are handled by Network (one layer of abstraction above).
- # Returns
- Python dictionary.
-
class
Echo.Activation.Keras.custom_activations.
log_softmax
(**kwargs)¶ Log-SoftMax Activation Function.
\[Log-SoftMax(x) = log(Softmax(-x))\]- Shape:
- Input: Arbitrary. Use the keyword argument input_shape
(tuple of integers, does not include the samples axis) when using this layer as the first layer in a model.
- Output: Same shape as the input.
- Examples:
>>> X_input = Input(input_shape) >>> X = log_softmax()(X_input)
-
call
(inputs)¶ This is where the layer’s logic lives.
- # Arguments
- inputs: Input tensor, or list/tuple of input tensors. **kwargs: Additional keyword arguments.
- # Returns
- A tensor or list/tuple of tensors.
-
compute_output_shape
(input_shape)¶ Computes the output shape of the layer.
Assumes that the layer will be built to match that input shape provided.
- # Arguments
- input_shape: Shape tuple (tuple of integers)
- or list of shape tuples (one per output tensor of the layer). Shape tuples can include None for free dimensions, instead of an integer.
- # Returns
- An input shape tuple.
-
get_config
()¶ Returns the config of the layer.
A layer config is a Python dictionary (serializable) containing the configuration of a layer. The same layer can be reinstantiated later (without its trained weights) from this configuration.
The config of a layer does not include connectivity information, nor the layer class name. These are handled by Network (one layer of abstraction above).
- # Returns
- Python dictionary.
-
class
Echo.Activation.Keras.custom_activations.
mila
(beta=-0.25, **kwargs)¶ Mila Activation Function.
\[mila(x) = x * tanh(ln(1 + e^{\beta + x})) = x * tanh(softplus(\beta + x)\]Plot:
- Shape:
- Input: Arbitrary. Use the keyword argument input_shape
(tuple of integers, does not include the samples axis) when using this layer as the first layer in a model.
- Output: Same shape as the input.
- Arguments:
- beta: scale to control the concavity of the global minima of the function (default = -0.25)
- References:
- Examples:
>>> X_input = Input(input_shape) >>> X = mila(beta=0.5)(X_input)
-
call
(inputs)¶ This is where the layer’s logic lives.
- # Arguments
- inputs: Input tensor, or list/tuple of input tensors. **kwargs: Additional keyword arguments.
- # Returns
- A tensor or list/tuple of tensors.
-
compute_output_shape
(input_shape)¶ Computes the output shape of the layer.
Assumes that the layer will be built to match that input shape provided.
- # Arguments
- input_shape: Shape tuple (tuple of integers)
- or list of shape tuples (one per output tensor of the layer). Shape tuples can include None for free dimensions, instead of an integer.
- # Returns
- An input shape tuple.
-
get_config
()¶ Returns the config of the layer.
A layer config is a Python dictionary (serializable) containing the configuration of a layer. The same layer can be reinstantiated later (without its trained weights) from this configuration.
The config of a layer does not include connectivity information, nor the layer class name. These are handled by Network (one layer of abstraction above).
- # Returns
- Python dictionary.
-
class
Echo.Activation.Keras.custom_activations.
mish
(**kwargs)¶ Mish Activation Function.
\[mish(x) = x * tanh(softplus(x)) = x * tanh(ln(1 + e^{x}))\]Plot:
- Shape:
- Input: Arbitrary. Use the keyword argument input_shape
(tuple of integers, does not include the samples axis) when using this layer as the first layer in a model.
- Output: Same shape as the input.
- Examples:
>>> X_input = Input(input_shape) >>> X = mish()(X_input)
-
call
(inputs)¶ This is where the layer’s logic lives.
- # Arguments
- inputs: Input tensor, or list/tuple of input tensors. **kwargs: Additional keyword arguments.
- # Returns
- A tensor or list/tuple of tensors.
-
compute_output_shape
(input_shape)¶ Computes the output shape of the layer.
Assumes that the layer will be built to match that input shape provided.
- # Arguments
- input_shape: Shape tuple (tuple of integers)
- or list of shape tuples (one per output tensor of the layer). Shape tuples can include None for free dimensions, instead of an integer.
- # Returns
- An input shape tuple.
-
get_config
()¶ Returns the config of the layer.
A layer config is a Python dictionary (serializable) containing the configuration of a layer. The same layer can be reinstantiated later (without its trained weights) from this configuration.
The config of a layer does not include connectivity information, nor the layer class name. These are handled by Network (one layer of abstraction above).
- # Returns
- Python dictionary.
-
class
Echo.Activation.Keras.custom_activations.
relu6
(**kwargs)¶ RELU6 Activation Function.
\[RELU6(x) = min(max(0,x),6)\]- Shape:
- Input: Arbitrary. Use the keyword argument input_shape
(tuple of integers, does not include the samples axis) when using this layer as the first layer in a model.
- Output: Same shape as the input.
- References:
- See RELU6 paper:
- http://www.cs.utoronto.ca/~kriz/conv-cifar10-aug2010.pdf
- Examples:
>>> X_input = Input(input_shape) >>> X = relu6()(X_input)
-
call
(inputs)¶ This is where the layer’s logic lives.
- # Arguments
- inputs: Input tensor, or list/tuple of input tensors. **kwargs: Additional keyword arguments.
- # Returns
- A tensor or list/tuple of tensors.
-
compute_output_shape
(input_shape)¶ Computes the output shape of the layer.
Assumes that the layer will be built to match that input shape provided.
- # Arguments
- input_shape: Shape tuple (tuple of integers)
- or list of shape tuples (one per output tensor of the layer). Shape tuples can include None for free dimensions, instead of an integer.
- # Returns
- An input shape tuple.
-
get_config
()¶ Returns the config of the layer.
A layer config is a Python dictionary (serializable) containing the configuration of a layer. The same layer can be reinstantiated later (without its trained weights) from this configuration.
The config of a layer does not include connectivity information, nor the layer class name. These are handled by Network (one layer of abstraction above).
- # Returns
- Python dictionary.
-
class
Echo.Activation.Keras.custom_activations.
sineReLU
(epsilon=0.01, **kwargs)¶ Sine ReLU Activation Function.
\[\begin{split}SineReLU(x, \epsilon) = \left\{\begin{matrix} x , x > 0 \\ \epsilon * (sin(x)-cos(x)), x \leq 0 \end{matrix}\right.\end{split}\]Plot:
- Shape:
- Input: Arbitrary. Use the keyword argument input_shape
(tuple of integers, does not include the samples axis) when using this layer as the first layer in a model.
- Output: Same shape as the input.
- References:
- See related Medium article:
- Arguments:
- epsilon: hyperparameter (default=0.01)
- Examples:
>>> X_input = Input(input_shape) >>> X = sineReLU(epsilon=0.01)(X_input)
-
call
(inputs)¶ This is where the layer’s logic lives.
- # Arguments
- inputs: Input tensor, or list/tuple of input tensors. **kwargs: Additional keyword arguments.
- # Returns
- A tensor or list/tuple of tensors.
-
compute_output_shape
(input_shape)¶ Computes the output shape of the layer.
Assumes that the layer will be built to match that input shape provided.
- # Arguments
- input_shape: Shape tuple (tuple of integers)
- or list of shape tuples (one per output tensor of the layer). Shape tuples can include None for free dimensions, instead of an integer.
- # Returns
- An input shape tuple.
-
get_config
()¶ Returns the config of the layer.
A layer config is a Python dictionary (serializable) containing the configuration of a layer. The same layer can be reinstantiated later (without its trained weights) from this configuration.
The config of a layer does not include connectivity information, nor the layer class name. These are handled by Network (one layer of abstraction above).
- # Returns
- Python dictionary.
-
class
Echo.Activation.Keras.custom_activations.
soft_clipping
(alpha=0.5, **kwargs)¶ Soft Clipping Activation Function.
\[SC(x) = 1 / \alpha * log(\frac{1 + e^{\alpha * x}}{1 + e^{\alpha * (x-1)}})\]Plot:
- Shape:
- Input: Arbitrary. Use the keyword argument input_shape
(tuple of integers, does not include the samples axis) when using this layer as the first layer in a model.
- Output: Same shape as the input.
- Arguments:
- alpha: hyper-parameter, which determines how close to linear the central region is and how sharply the linear region turns to the asymptotic values
- References:
- See SC paper:
- https://arxiv.org/pdf/1810.11509.pdf
- Examples:
>>> X_input = Input(input_shape) >>> X =soft_clipping(alpha=0.5)(X_input)
-
call
(inputs)¶ This is where the layer’s logic lives.
- # Arguments
- inputs: Input tensor, or list/tuple of input tensors. **kwargs: Additional keyword arguments.
- # Returns
- A tensor or list/tuple of tensors.
-
compute_output_shape
(input_shape)¶ Computes the output shape of the layer.
Assumes that the layer will be built to match that input shape provided.
- # Arguments
- input_shape: Shape tuple (tuple of integers)
- or list of shape tuples (one per output tensor of the layer). Shape tuples can include None for free dimensions, instead of an integer.
- # Returns
- An input shape tuple.
-
get_config
()¶ Returns the config of the layer.
A layer config is a Python dictionary (serializable) containing the configuration of a layer. The same layer can be reinstantiated later (without its trained weights) from this configuration.
The config of a layer does not include connectivity information, nor the layer class name. These are handled by Network (one layer of abstraction above).
- # Returns
- Python dictionary.
-
class
Echo.Activation.Keras.custom_activations.
soft_exponential
(**kwargs)¶ Soft-Exponential Activation Function.
\[\begin{split}SoftExponential(x, \alpha) = \left\{\begin{matrix} - \frac{log(1 - \alpha(x + \alpha))}{\alpha}, \alpha < 0\\ x, \alpha = 0\\ \frac{e^{\alpha * x} - 1}{\alpha} + \alpha, \alpha > 0 \end{matrix}\right.\end{split}\]with 1 trainable parameter.
- Shape:
- Input: Arbitrary. Use the keyword argument input_shape
(tuple of integers, does not include the samples axis) when using this layer as the first layer in a model.
- Output: Same shape as the input.
- Parameters:
- alpha - trainable parameter
- References:
- See Soft-Exponential paper:
- Examples:
>>> X_input = Input(input_shape) >>> X = soft_exponential()(X_input)
-
build
(input_shape)¶ Creates the layer weights.
Must be implemented on all layers that have weights.
- # Arguments
- input_shape: Keras tensor (future input to layer)
- or list/tuple of Keras tensors to reference for weight shape computations.
-
call
(inputs)¶ This is where the layer’s logic lives.
- # Arguments
- inputs: Input tensor, or list/tuple of input tensors. **kwargs: Additional keyword arguments.
- # Returns
- A tensor or list/tuple of tensors.
-
compute_output_shape
(input_shape)¶ Computes the output shape of the layer.
Assumes that the layer will be built to match that input shape provided.
- # Arguments
- input_shape: Shape tuple (tuple of integers)
- or list of shape tuples (one per output tensor of the layer). Shape tuples can include None for free dimensions, instead of an integer.
- # Returns
- An input shape tuple.
-
get_config
()¶ Returns the config of the layer.
A layer config is a Python dictionary (serializable) containing the configuration of a layer. The same layer can be reinstantiated later (without its trained weights) from this configuration.
The config of a layer does not include connectivity information, nor the layer class name. These are handled by Network (one layer of abstraction above).
- # Returns
- Python dictionary.
-
class
Echo.Activation.Keras.custom_activations.
soft_shrink
(lambd=0.5, **kwargs)¶ Soft-Shrink Activation Function.
\[\begin{split}Soft-Shrink(x) = \left\{\begin{matrix} x - \lambda , x > \lambda \\ 0 , - \lambda \leq x \leq \lambda \\ x + \lambda , x <- \lambda \end{matrix}\right.\end{split}\]- Shape:
- Input: Arbitrary. Use the keyword argument input_shape
(tuple of integers, does not include the samples axis) when using this layer as the first layer in a model.
- Output: Same shape as the input.
- Arguments:
- lambda: the λ value for the Softshrink formulation (default=0.5)
- Examples:
>>> X_input = Input(input_shape) >>> X = soft_shrink()(X_input)
-
call
(inputs)¶ This is where the layer’s logic lives.
- # Arguments
- inputs: Input tensor, or list/tuple of input tensors. **kwargs: Additional keyword arguments.
- # Returns
- A tensor or list/tuple of tensors.
-
compute_output_shape
(input_shape)¶ Computes the output shape of the layer.
Assumes that the layer will be built to match that input shape provided.
- # Arguments
- input_shape: Shape tuple (tuple of integers)
- or list of shape tuples (one per output tensor of the layer). Shape tuples can include None for free dimensions, instead of an integer.
- # Returns
- An input shape tuple.
-
get_config
()¶ Returns the config of the layer.
A layer config is a Python dictionary (serializable) containing the configuration of a layer. The same layer can be reinstantiated later (without its trained weights) from this configuration.
The config of a layer does not include connectivity information, nor the layer class name. These are handled by Network (one layer of abstraction above).
- # Returns
- Python dictionary.
-
class
Echo.Activation.Keras.custom_activations.
softmin
(**kwargs)¶ SoftMin Activation Function.
\[SoftMin(x) = Softmax(-x)\]- Shape:
- Input: Arbitrary. Use the keyword argument input_shape
(tuple of integers, does not include the samples axis) when using this layer as the first layer in a model.
- Output: Same shape as the input.
- Examples:
>>> X_input = Input(input_shape) >>> X = softmin()(X_input)
-
call
(inputs)¶ This is where the layer’s logic lives.
- # Arguments
- inputs: Input tensor, or list/tuple of input tensors. **kwargs: Additional keyword arguments.
- # Returns
- A tensor or list/tuple of tensors.
-
compute_output_shape
(input_shape)¶ Computes the output shape of the layer.
Assumes that the layer will be built to match that input shape provided.
- # Arguments
- input_shape: Shape tuple (tuple of integers)
- or list of shape tuples (one per output tensor of the layer). Shape tuples can include None for free dimensions, instead of an integer.
- # Returns
- An input shape tuple.
-
get_config
()¶ Returns the config of the layer.
A layer config is a Python dictionary (serializable) containing the configuration of a layer. The same layer can be reinstantiated later (without its trained weights) from this configuration.
The config of a layer does not include connectivity information, nor the layer class name. These are handled by Network (one layer of abstraction above).
- # Returns
- Python dictionary.
-
class
Echo.Activation.Keras.custom_activations.
sqnl
(**kwargs)¶ SQNL Activation Function.
\[\begin{split}SQNL(x) = \left\{\begin{matrix} 1, x > 2 \\ x - \frac{x^2}{4}, 0 \leq x \leq 2 \\ x + \frac{x^2}{4}, -2 \leq x < 0 \\ -1, x < -2 \end{matrix}\right.\end{split}\]Plot:
- Shape:
- Input: Arbitrary. Use the keyword argument input_shape
(tuple of integers, does not include the samples axis) when using this layer as the first layer in a model.
- Output: Same shape as the input.
- References:
- SQNL Paper:
- Examples:
>>> X_input = Input(input_shape) >>> X = sqnl()(X_input)
-
call
(inputs)¶ This is where the layer’s logic lives.
- # Arguments
- inputs: Input tensor, or list/tuple of input tensors. **kwargs: Additional keyword arguments.
- # Returns
- A tensor or list/tuple of tensors.
-
compute_output_shape
(input_shape)¶ Computes the output shape of the layer.
Assumes that the layer will be built to match that input shape provided.
- # Arguments
- input_shape: Shape tuple (tuple of integers)
- or list of shape tuples (one per output tensor of the layer). Shape tuples can include None for free dimensions, instead of an integer.
- # Returns
- An input shape tuple.
-
get_config
()¶ Returns the config of the layer.
A layer config is a Python dictionary (serializable) containing the configuration of a layer. The same layer can be reinstantiated later (without its trained weights) from this configuration.
The config of a layer does not include connectivity information, nor the layer class name. These are handled by Network (one layer of abstraction above).
- # Returns
- Python dictionary.
-
class
Echo.Activation.Keras.custom_activations.
srelu
(**kwargs)¶ SReLU (S-shaped Rectified Linear Activation Unit): a combination of three linear functions, which perform mapping R → R with the following formulation:
\[\begin{split}h(x_i) = \left\{\begin{matrix} t_i^r + a_i^r(x_i - t_i^r), x_i \geq t_i^r \\ x_i, t_i^r > x_i > t_i^l\\ t_i^l + a_i^l(x_i - t_i^l), x_i \leq t_i^l \\ \end{matrix}\right.\end{split}\]with 4 trainable parameters.
- Shape:
- Input: Arbitrary. Use the keyword argument input_shape
(tuple of integers, does not include the samples axis) when using this layer as the first layer in a model.
- Output: Same shape as the input.
Parameters:
\[\{t_i^r, a_i^r, t_i^l, a_i^l\}\]4 trainable parameters, which model an individual SReLU activation unit. The subscript i indicates that we allow SReLU to vary in different channels. Parameters can be initialized manually or randomly.
- References:
- See SReLU paper:
- Examples:
>>> X_input = Input(input_shape) >>> X = srelu(params=None)(X_input)
-
build
(input_shape)¶ - Adding Trainable Parameters.
- parameters: (tr, tl, ar, al) initialized randomly.
-
call
(inputs)¶ This is where the layer’s logic lives.
- # Arguments
- inputs: Input tensor, or list/tuple of input tensors. **kwargs: Additional keyword arguments.
- # Returns
- A tensor or list/tuple of tensors.
-
compute_output_shape
(input_shape)¶ Computes the output shape of the layer.
Assumes that the layer will be built to match that input shape provided.
- # Arguments
- input_shape: Shape tuple (tuple of integers)
- or list of shape tuples (one per output tensor of the layer). Shape tuples can include None for free dimensions, instead of an integer.
- # Returns
- An input shape tuple.
-
get_config
()¶ Returns the config of the layer.
A layer config is a Python dictionary (serializable) containing the configuration of a layer. The same layer can be reinstantiated later (without its trained weights) from this configuration.
The config of a layer does not include connectivity information, nor the layer class name. These are handled by Network (one layer of abstraction above).
- # Returns
- Python dictionary.
-
class
Echo.Activation.Keras.custom_activations.
swish
(beta=1.0, **kwargs)¶ Swish Activation Function.
\[Swish(x, \beta) = x*sigmoid(\beta*x) = \frac{x}{(1+e^{-\beta*x})}\]Plot:
- Shape:
- Input: Arbitrary. Use the keyword argument input_shape
(tuple of integers, does not include the samples axis) when using this layer as the first layer in a model.
- Output: Same shape as the input.
- Arguments:
- a constant or a trainable parameter (default=1; which is equivalent to Sigmoid-weighted Linear Unit (SiL))
- References:
- See Swish paper:
- Examples:
>>> X_input = Input(input_shape) >>> X = swish(beta=0.5)(X_input)
-
call
(inputs)¶ This is where the layer’s logic lives.
- # Arguments
- inputs: Input tensor, or list/tuple of input tensors. **kwargs: Additional keyword arguments.
- # Returns
- A tensor or list/tuple of tensors.
-
compute_output_shape
(input_shape)¶ Computes the output shape of the layer.
Assumes that the layer will be built to match that input shape provided.
- # Arguments
- input_shape: Shape tuple (tuple of integers)
- or list of shape tuples (one per output tensor of the layer). Shape tuples can include None for free dimensions, instead of an integer.
- # Returns
- An input shape tuple.
-
get_config
()¶ Returns the config of the layer.
A layer config is a Python dictionary (serializable) containing the configuration of a layer. The same layer can be reinstantiated later (without its trained weights) from this configuration.
The config of a layer does not include connectivity information, nor the layer class name. These are handled by Network (one layer of abstraction above).
- # Returns
- Python dictionary.
-
class
Echo.Activation.Keras.custom_activations.
tanh_shrink
(**kwargs)¶ TanH-Shrink Activation Function.
\[TanH-Shrink(x) = x - tanh(x)\]- Shape:
- Input: Arbitrary. Use the keyword argument input_shape
(tuple of integers, does not include the samples axis) when using this layer as the first layer in a model.
- Output: Same shape as the input.
- Examples:
>>> X_input = Input(input_shape) >>> X = tanh_shrink()(X_input)
-
call
(inputs)¶ This is where the layer’s logic lives.
- # Arguments
- inputs: Input tensor, or list/tuple of input tensors. **kwargs: Additional keyword arguments.
- # Returns
- A tensor or list/tuple of tensors.
-
compute_output_shape
(input_shape)¶ Computes the output shape of the layer.
Assumes that the layer will be built to match that input shape provided.
- # Arguments
- input_shape: Shape tuple (tuple of integers)
- or list of shape tuples (one per output tensor of the layer). Shape tuples can include None for free dimensions, instead of an integer.
- # Returns
- An input shape tuple.
-
get_config
()¶ Returns the config of the layer.
A layer config is a Python dictionary (serializable) containing the configuration of a layer. The same layer can be reinstantiated later (without its trained weights) from this configuration.
The config of a layer does not include connectivity information, nor the layer class name. These are handled by Network (one layer of abstraction above).
- # Returns
- Python dictionary.
-
class
Echo.Activation.Keras.custom_activations.
weighted_tanh
(weight=1.0, **kwargs)¶ Weighted TanH Activation Function.
\[Weighted TanH(x, weight) = tanh(x * weight)\]Plot:
- Shape:
- Input: Arbitrary. Use the keyword argument input_shape
(tuple of integers, does not include the samples axis) when using this layer as the first layer in a model.
- Output: Same shape as the input.
- Arguments:
- weight: hyperparameter (default=1.0)
- Examples:
>>> X_input = Input(input_shape) >>> X = weighted_tanh(weight=1.0)(X_input)
-
call
(inputs)¶ This is where the layer’s logic lives.
- # Arguments
- inputs: Input tensor, or list/tuple of input tensors. **kwargs: Additional keyword arguments.
- # Returns
- A tensor or list/tuple of tensors.
-
compute_output_shape
(input_shape)¶ Computes the output shape of the layer.
Assumes that the layer will be built to match that input shape provided.
- # Arguments
- input_shape: Shape tuple (tuple of integers)
- or list of shape tuples (one per output tensor of the layer). Shape tuples can include None for free dimensions, instead of an integer.
- # Returns
- An input shape tuple.
-
get_config
()¶ Returns the config of the layer.
A layer config is a Python dictionary (serializable) containing the configuration of a layer. The same layer can be reinstantiated later (without its trained weights) from this configuration.
The config of a layer does not include connectivity information, nor the layer class name. These are handled by Network (one layer of abstraction above).
- # Returns
- Python dictionary.


Help village families change their lives through the Village Gift program!
100% of the funds you contribute will go directly into the Village Development program, helping families obtain clean water, farm animals, vegetable gardens, fishponds and beehives. Village families who become part of the Pass-On Program are empowered to become donors themselves: When their animals have grown, or their beehive or fishpond is thriving, the family passes on an identical gift to two other families, and those families later do the same. So your gift keeps on growing through the years, creating positive change by empowerment of village families.
Each gift you choose can be made in your name, or that of a friend or family member. You can choose as many gifts as you wish. For each gift we'll send you a thank-you card, and send one to the friend or family member in whose name you gave.


public class BWTAWrapper
extends java.lang.Object
| Modifier and Type | Field and Description |
|---|---|
private static int |
DEFAULT_STEP_SIZE |
| Constructor and Description |
|---|
BWTAWrapper() |
| Modifier and Type | Method and Description |
|---|---|
private static bwta.Region |
forceFindRegion(bwapi.Position position,
int maxIterations,
int stepSize)
Function for initializing a force search for a Position's Region
instance.
|
private static java.util.HashSet<bwapi.Position> |
generatePossibleAdjacentPositions(bwapi.Position position,
java.util.HashSet<bwapi.Position> checkedPositions,
int stepSize)
Function for generating the Positions that are the direct neighbours of a
provided Position instance.
|
static bwta.Region |
getRegion(bwapi.Position position)
Convenience function.
|
static bwta.Region |
getRegion(bwapi.Position position,
int stepSize)
Function for finding the Region of a provided Position.
|
static bwta.Region |
getRegion(bwapi.TilePosition tilePosition)
Convenience function.
|
static bwta.Region |
getRegion(bwapi.TilePosition tilePosition,
int stepSize)
Convenience function.
|
private static boolean |
positionIsValid(bwapi.Position position)
Function for testing if a Position is a valid one.
|
public static bwta.Region getRegion(bwapi.TilePosition tilePosition)
tilePosition
- the TilePosition whose Region is being looked for.getRegion(TilePosition, int)
public static bwta.Region getRegion(bwapi.TilePosition tilePosition, int stepSize)
tilePosition
- the TilePosition whose Region is being looked for.stepSize
- the distance which is being used to determine the Region if
the used BWTA function returns null and the TilePosition is
not inside a Region.getRegion(TilePosition, int)
public static bwta.Region getRegion(bwapi.Position position)
position
- the Position whose Region is being looked for.getRegion(TilePosition, int)
public static bwta.Region getRegion(bwapi.Position position, int stepSize)
position
- the Position whose Region is being looked for.stepSize
- the distance which is being used to determine the Region if
the used BWTA function returns null and the Position is not
inside a Region.private static bwta.Region forceFindRegion(bwapi.Position position, int maxIterations, int stepSize) throws java.lang.Exception
position
- starting Position instance for the flood fill.maxIterations
- the max number of iterations until the search is stopped.stepSize
- the offset that is applied to each Position in every
iteration. Left, top, right and bottom Positions are generated
with this value (X and Y +/- value).java.lang.Exception
- an Exception is thrown when the maximum number of iterations
is reached.private static boolean positionIsValid(bwapi.Position position)
position
- the Position that is going to be checked.private static java.util.HashSet<bwapi.Position> generatePossibleAdjacentPositions(bwapi.Position position, java.util.HashSet<bwapi.Position> checkedPositions, int stepSize)
position
- the Position whose neighbours are being checked.checkedPositions
- the HashSet of already checked Positions which a generated
Position must not be in.stepSize
- the offset that is applied to the X and Y values when
generating the adjacent Positions.


Dolphins-Browns Stats
by EndPlayCle_Landry 7 pass from Mayfield (Seibert kick), 9:59.
Cle_Beckham 35 pass from Mayfield (Seibert kick), 1:51.
Cle_Landry 5 pass from Mayfield (Seibert kick), 10:31.
Cle_Hunt 6 run (Seibert kick), 1:01.
Mia_FG Sanders 36, :00.
Mia_Gesicki 11 pass from Fitzpatrick (Sanders kick), 9:37.
Mia_Fitzpatrick 8 run (Sanders kick), :04.
Cle_FG Seibert 40, 11:21.
Cle_FG Seibert 26, 6:51.
Cle_Chubb 5 run (Seibert kick), 3:26.
Mia_Hurns 19 pass from Fitzpatrick (Sanders kick), :39.
A_67,431.
___
___
RUSHING_Miami, Fitzpatrick 5-45, Laird 3-20, Ballage 7-13, Gaskin 4-10, A.Wilson 1-4. Cleveland, Chubb 21-106, Hunt 8-37, Mayfield 2-5, Gilbert 1-(minus 1).
PASSING_Miami, Fitzpatrick 21-39-2-214. Cleveland, Mayfield 24-34-1-327.
RECEIVING_Miami, D.Parker 6-91, A.Wilson 6-33, Hurns 4-42, Gesicki 3-28, Ballage 1-13, Gaskin 1-7. Cleveland, Landry 10-148, Beckham 6-84, Chubb 3-58, D.Harris 2-20, Hunt 2-9, Carlson 1-8.
MISSED FIELD GOALS_Cleveland, Seibert 46.
First Quarter
Second Quarter
Third Quarter
Fourth Quarter
INDIVIDUAL STATISTICS


Thank you for stopping by my site.
Take a look around and see what I've been up to.
I love hearing from people, so also feel free to reach out!
Hi, I'm Chevy
UX Researcher
Strategist
Do-Gooder
Follow @artscanner
About
Projects
Blog
Contact


Hide an element to all devices except screen readers with .sr-only
. Combine .sr-only
with .sr-only-focusable
to show the element again when it’s focused (e.g. by a keyboard-only user). Can also be used as mixins.
Hide an element to all devices except screen readers with .sr-only
. Combine .sr-only
with .sr-only-focusable
to show the element again when it’s focused (e.g. by a keyboard-only user). Can also be used as mixins.


Alfred Duncan remains Antonio Conte's only midfield option in the winter transfer window
Inter Milan manager Antonio Conte is hopeful of making a January transfer move for Ghana midfielder Alfred Duncan, who remains his only midfield option in the window.
Conte has his sights on Udinese midfielder Rodrigo de Paul and Atlanta's Dejan Kulusevski but it seems the two will be difficult to get in January.
This has forced Conte to try convince Duncan to return to Inter, a club he had his youth football development at.
The 26-year progressed through the ranks at Inter before being sold to Sampdoria in 2014, who subsequently sold him to his current employers Sassuolo.
Inter's management have good relations with their Neroverdi counterparts as the deals for both Matteo Politano and Stefano Sensi in recent seasons go to show.
Duncan is however not the only player Inter are considering with them also taking a future interest in Dejan Kulusevski, Sandro Tonali, Rodrigo de Paul and Gaetano Castrovilli.
But it is particularly difficult to see Inter manage to land any of Kulusevski, Tonali or Castrovilli in the upcoming January window which is why Duncan has become an option for them.


Design and Architecture of Duke's Forest
Duke's Forest is a complex application consisting of three main projects and three subprojects. The following figure shows the architecture of the three main projects that you will deploy: Duke's Store, Duke's Shipment, and Duke's Payment. It also shows how Duke's Store makes use of the Events and Entities projects.
Duke's Forest uses the following Java EE 6 platform features:
Java Persistence API entities:
Bean Validation annotations on the entities for verifying data
XML annotations for Java API for XML Binding (JAXB) serialization
Web services:
A JAX-WS web service for payment, with security constraints
A JAX-RS web service that is EJB based
Enterprise beans:
Local session beans
All enterprise beans packaged within the WAR
Contexts and Dependency Injection (CDI):
CDI annotations for JavaServer Faces components
A CDI managed bean used as a shopping cart, with conversation scoping
Qualifiers
Events and event handlers
Servlets:
A Servlet 3.0 file upload example
A servlet for dynamic image presentation
JavaServer Faces technology, using Facelets for the web front end
Templating
Composite components
Resources packaged in a JAR file so they can be found in the classpath
Security:
Java EE security constraints on the administrative interface business methods (enterprise beans)
Security constraints for customers and administrators (web components)
The Duke's Forest application has two main user interfaces, both packaged within the Duke's Store WAR file:
The main interface, for customers and guests
The administrative interface used to perform back office operations, such as adding new items to the catalog
The Duke's Shipment application also has a user interface, accessible to administrators.
The following figure shows how the web applications and web service interact.
In this figure, the customer interacts with the main interface of Duke's Store, while the administrator interacts with the administration interface. Both interfaces access a façade consisting of managed beans and stateless session beans, which in turn interact with the entities that represent database tables. The façade also interacts with web services APIs that access the Duke's Payment web service. The administrator also interacts with the interface of Duke's Shipment, which can be accessed either directly through Duke's Shipment, or from the administration interface of Duke's Store by means of a web service.
The most fundamental building blocks of the application are the Events and Entities projects, which are bundled into Duke's Store and Duke's Shipment along with the Duke's Resources project.
The Events Project
Events are one of the core components of Duke's Forest. The events project, included in all three of the main projects, is the most simple project of the application. It has only one class, OrderEvent, but this class is responsible for most of the messages between objects in the application.
The application can send messages based on events to different components and react to them based on the qualification of the event. The application supports the following qualifiers:
@LoggedIn: For authenticated users
@New: When a new order is created by the shopping cart
@Paid: When an order is paid for and ready for shipment
The following code snippet from the PaymentHandler class of Duke's Store shows how the @Paid event is handled:
@Inject @Paid Event<OrderEvent> eventManager; ... public void onNewOrder(@Observes @New OrderEvent event) { if (processPayment(convertForWS(event))) { orderBean.setOrderStatus(event.getOrderID(), OrderBean.Status.PENDING_PAYMENT); logger.info("Payment Approved"); eventManager.fire(event); } else { orderBean.setOrderStatus(event.getOrderID(), OrderBean.Status.CANCELLED_PAYMENT); logger.info("Payment Denied"); } } ...
To enable users to add more events to the project easily or to update an event class with more fields for a new client, this component is a separate project within the application.
The Entities Project
The entities project is a Java Persistence API (JPA) project that is used by both Duke's Store and Duke's Shipment. It is generated from the database schema shown in the following figure and is also used as a base for the entities consumed and produced by the web services through JAXB. Each entity has validation rules based on business requirements, specified using Bean Validation.
The database schema contains eight tables:
PERSON, which has a one-to-many relationship with PERSON_GROUPS and CUSTOMER_ORDER
GROUPS, which has a one-to-many relationship with PERSON_GROUPS
PERSON_GROUPS, which has a many-to-one relationship with PERSON and GROUPS (it is the join table between those two tables)
PRODUCT, which has a many-to-one relationship with CATEGORY and a one-to-many relationship with ORDER_DETAIL
CATEGORY, which has a one-to-many relationship with PRODUCT
ORDER_DETAIL, which has a many-to-one relationship with PRODUCT and CUSTOMER_ORDER (it is the join table between those two tables)
CUSTOMER_ORDER, which has a one-to-many relationship with ORDER_DETAIL and a many-to-one relationship with PERSON and ORDER_STATUS
ORDER_STATUS, which has a one-to-many relationship with CUSTOMER_ORDER
The entity classes that correspond to these tables are as follows:
Person, which defines attributes common to customers and administrators. These attributes are the person's name and contact information, including street and email address. The email address has a Bean Validation annotation to ensure that the submitted data is well-formed.
Customer, a specialization of Person with a specific field for CustomerOrder objects.
Administrator, a specialization of Person with fields for administration privileges.
Groups, which represents the group (customer or administrator) to which the user belongs.
Product, which defines attributes for products. These attributes include a name, price, description, associated image, and category.
Category, which defines attributes for product categories. These attributes include a name and a set of tags.
CustomerOrder, which defines attributes for orders placed by customers. These attributes include an amount and a date, along with id values for the customer and the order detail.
OrderDetail, which defines attributes for the order detail. These attributes include a quantity, along with id values for the product and the customer.
OrderStatus, which defines a status attribute for each order.
The Duke's Payment Project
The dukes-payment project is a web project that holds a simple Payment web service. Since this is an example application, it does not obtain any real credit information or even customer status to validate the payment. For now, the only rule imposed by the payment system is to approve all orders above $1000. This application illustrates a common scenario where a third-party payment service is used to validate credit cards or bank payments.
The project uses HTTP Basic Authentication and JAAS (Java Authentication and Authorization Service) to authenticate a customer to a JAX-WS web service. The implementation itself exposes a simple method, processPayment, which receives an OrderEvent to evaluate and approve or deny the order payment. The method is called from the checkout process of Duke's Store.
The Duke's Resources Project
The dukes-resources project contains a number of files used by both Duke's Store and Duke's Shipment, bundled into a JAR file placed in the classpath. The resources are in the src/META-INF/resources directory:
- src/ValidationMessages.properties
Bean Validation messages.
- src/META-INF/resources/css
Two style sheets, default.css and jsfcrud.css.
- src/META-INF/resources/img
Images used by the projects.
- src/META-INF/resources/js
A JavaScript file, util.js.
- src/META-INF/resources/util
Composite components used by the projects.
The Duke's Store Project
Duke's Store, a web application, is the core application of Duke's Forest. It is responsible for the main store interface for customers as well as the administration interface.
The main interface of Duke's Store allows users to perform the following tasks:
Browsing the product catalog
Signing up as a new customer
Adding products to the shopping cart
Checking out
Viewing order status
The administration interface of Duke's Store allows administrators to perform the following tasks:
Product maintenance (create, edit, update, delete)
Category maintenance (create, edit, update, delete)
Customer maintenance (create, edit, update, delete)
Group maintenance (create, edit, update, delete)
The project also uses stateless session beans as façades for interactions with the JPA entities described in The Entities Project, and CDI managed beans as controllers for interactions with Facelets pages. The project thus follows the MVC (Model-View-Controller) pattern and applies the same pattern to all entities and pages, as in the following example:
AbstractFacade is an abstract class that receives a Type<T> and implements the common operations (CRUD) for this type, where <T> is a JPA entity.
ProductBean is a stateless session bean that extends AbstractFacade, applying Product as Type<T>, and injects the PersistenceContext for the Entity Manager. In this bean we can implement any custom method that we would need to interact with the Product entity or call a custom query.
ProductController is a CDI managed bean that interacts with the necessary enterprise beans and Facelets pages to control the way the data will be displayed.
ProductBean begins as follows:
@Stateless public class ProductBean extends AbstractFacade<Product> { private static final Logger logger = Logger.getLogger(ProductBean.class.getCanonicalName()); @PersistenceContext(unitName="forestPU") private EntityManager em; @Override protected EntityManager getEntityManager() { return em; } ...
Enterprise Beans Used in Duke's Store
The enterprise beans used in Duke's Store provide the business logic for the application and are located in the com.forest.ejb package. All are stateless session beans.
AbstractFacade is not an enterprise bean, but is an abstract class that implements common operations for Type<T>, where <T> is a JPA entity.
Most of the other beans extend AbstractFacade, inject the PersistenceContext, and implement any needed custom methods:
AdministratorBean
CategoryBean
GroupsBean
OrderBean
OrderDetailBean
OrderStatusBean
ProductBean
ShoppingCart
UserBean
The ShoppingCart, although it is in the ejb package, is a CDI managed bean with conversation scope, which means that the request information will persist across multiple requests. Also, ShoppingCart is responsible for starting the event chain for customer orders, as described in The Events Project.
Facelets Files Used in the Main Interface of Duke's Store
Like the other case study examples, Duke's Store uses Facelets to display the user interface. The main interface uses a large number of Facelets pages to display different areas of the user interface. The pages are grouped into directories based on the module they handle.
- template.xhtml
The template file, used for both the main and administration interfaces, first performs a browser check to verify that the user's browser supports HTML 5, which is required for Duke's Forest. It divides the screen into several areas and specifies the client page for each area.
- topbar.xhtml
Page for the login area at the top of the screen.
- top.xhtml
Page for the title area.
- left.xhtml
Page for the left sidebar.
- index.xhtml
Page for the main screen content.
- login.xhtml
Login page specified in web.xml. The main login interface is provided in topbar.xhtml, but this page appears if there is a login error.
- admin directory
Pages related to the administration interface, described in Facelets Files Used in the Administration Interface of Duke's Store.
- customer directory
Pages related to customers (Create.xhtml, Edit.xhtml, List.xhtml, Profile.xhtml, View.xhtml).
- order directory
Pages related to orders (Create.xhtml, List.xhtml, MyOrders.xhtml, View.xhtml).
- orderDetail directory
Popup page allowing users to view details of an order (View_popup.xhtml).
- orderStatus directory
Pages related to order status (Create.xhtml, Edit.xhtml, List.xhtml, View.xhtml).
- product directory
Pages related to products (List.xhtml, ListCategory.xhtml, View.xhtml).
Facelets Files Used in the Administration Interface of Duke's Store
The Facelets pages for the administration interface of Duke's Store are found in the web/admin directory.
- administrator directory
Pages related to administrator management (Create.xhtml, Edit.xhtml, List.xhtml, View.xhtml).
- category directory
Pages related to product category management (Create.xhtml, Edit.xhtml, List.xhtml, View.xhtml).
- customer directory
Pages related to customer management (Create.xhtml, Edit.xhtml, List.xhtml, Profile.xhtml, View.xhtml).
- groups folder
Pages related to group management (Create.xhtml, Edit.xhtml, List.xhtml, View.xhtml).
- order directory
Pages related to order management (Create.xhtml, Edit.xhtml, List.xhtml, View.xhtml).
- orderDetail directory
Popup page allowing the administrator to view details of an order (View_popup.xhtml).
- product directory
Pages related to product management (Confirm.xhtml, Create.xhtml, Edit.xhtml, List.xhtml, View.xhtml).
Managed Beans Used in Duke's Store
Duke's Store uses the following CDI managed beans, which correspond to the enterprise beans. The beans are in the com.forest.web package.
AdministratorController
CategoryController
CustomerController
CustomerOrderController
GroupsController
OrderDetailController
OrderStatusController
ProductController
UserController
Helper Classes Used in Duke's Store
The CDI managed beans in the main interface of Duke's Store use the following helper classes, found in the com.forest.web.util package:
- AbstractPaginationHelper
An abstract class with methods used by the managed beans.
- FileUploadServlet, ImageServlet
Classes used for image processing. FileUploadServlet uploads an image and stores its content in the database. ImageServlet retrieves the image content from the database and displays it (JavaServer Faces technology does not provide this functionality, so a servlet is needed).
- JsfUtil
Class used for JavaServer Faces operations, such as queuing messages on a FacesContext instance.
- MD5Util
Class used by the CustomerController managed bean to generate an encrypted password for a user.
Qualifiers Used in Duke's Store
Duke's Store defines the following qualifiers in the com.forest.qualifiers package:
- @LoggedIn
Qualifies a user as having logged in.
- @New
Qualifies an order as new.
- @Paid
Qualifies an order as paid.
Event Handlers Used in Duke's Store
Duke's Store defines event handlers related to the OrderEvent class packaged in The Events Project. The event handlers are in the com.forest.handlers package:
- IOrderHandler
The IOrderHandler interface defines a method, onNewOrder, that is implemented by the two handler classes.
- PaymentHandler
The ShoppingCart bean fires an OrderEvent qualified as @New. The onNewOrder method of PaymentHandler observes these events and, when it intercepts them, processes the payment using the Duke's Payment web service. After a successful response from the web service, PaymentHandler fires the OrderEvent again, this time qualified as @Paid.
- DeliveryHandler
The onNewOrder method of DeliveryHandler observes OrderEvent objects qualified as @Paid (orders paid and ready for delivery) and modifies the order status to PENDING_SHIPMENT. When an administrator accesses Duke's Shipment, it will call the Order Service, a RESTful web service, and ask for all orders on the database that are ready for delivery.
Properties Files Used in Duke's Store
Duke's Store contains one properties file, src/java/Bundle.properties. In this resource bundle are the strings used in the main and administration interfaces. Currently the strings exist only in the default language, English.
Duke's Store uses the ValidationMessages.properties file bundled in dukes-resources.jar for Bean Validation messages.
Deployment Descriptors Used in Duke's Store
Duke's Store uses the following deployment descriptors, located in the web/WEB-INF directory:
- beans.xml
An empty deployment descriptor file used to enable the CDI runtime.
- faces-config.xml
The JavaServer Faces configuration file.
- glassfish-web.xml
The configuration file specific to GlassFish Server.
- jaxws-catalog.xml
A deployment descriptor for a JAX-WS web service client.
- web.xml
The web application configuration file.
The Duke's Shipment Project
Duke's Shipment is a web application, with a login page, a main Facelets page, and some other objects. This application, which is accessible only to administrators, calls the Order Service (the RESTful web service exposed by Duke's Store) and lists all orders under two status headings: Pending and Shipped. The administrator can either approve or deny a pending order. If approved, the order is shipped, and it appears under the Shipped heading. If not approved, the order disappears from the page, and on the customer's Orders list it appears as cancelled.
There is also a gear icon on the Pending list that makes an Ajax call to the Order Service that will refresh the list without refreshing the page. The code looks like this:
<h:commandLink> <h:graphicImage library="img" title="Check for new orders" style="border:0px" name="refresh.png"/> <f:ajax execute="refresh" render="out" /> </h:commandLink>
Enterprise Beans Used in Duke's Shipment
The enterprise bean used in Duke's Store, UserBean, provides the business logic for the application and is located in the com.forest.shipment.session package. It is a stateless session bean.
Like Duke's Store, Duke's Shipment uses the AbstractFacade class. This class is not an enterprise bean, but is an abstract class that implements common operations for Type<T>, where <T> is a JPA entity.
Facelets Files Used in Duke's Shipment
Duke's Shipment has only one page, so it has many fewer Facelets files than Duke's Store.
- template.xhtml
The template file, like the one in Duke's Store, first performs a browser check to verify that the user's browser supports HTML 5, which is required for Duke's Forest. It divides the screen into areas and specifies the client page for each area.
- topbar.xhtml
Page for the login area at the top of the screen.
- top.xhtml
Page for the title area.
- left.xhtml
Page for the left sidebar (not used in Duke's Shipment).
- index.xhtml
Page for the initial main screen content.
- login.xhtml
Login page specified in web.xml. The main login interface is provided in topbar.xhtml, but this page appears if there is a login error.
- admin/index.xhtml
Page for the main screen content after authentication.
Managed Beans Used in Duke's Shipment
Duke's Shipment uses the following CDI managed beans, in the com.forest.shipment package:
- control.ShippingBean
Managed bean that acts as a client to the Order Service.
- web.UserController
Managed bean that corresponds to the UserBean session bean.
Helper Classes Used in Duke's Shipment
The Duke's Shipment managed beans use only one helper class, found in the com.forest.shipment.web.util package:
- JsfUtil
Class used for JavaServer Faces operations, such as queuing messages on a FacesContext instance.
Qualifiers Used in Duke's Shipment
Duke's Shipment includes the same qualifiers described in Qualifiers Used in Duke's Store, although only the @LoggedIn qualifier is used in this project.
Properties Files Used in Duke's Shipment
Like Dukes' Store, Duke's Shipment uses just one properties file, the resource bundle src/java/Bundle.properties. It is identical to the file in Duke's Store.
Deployment Descriptors Used in Duke's Shipment
Duke's Shipment uses the following deployment descriptors:
- web/WEB-INF/beans.xml
An empty deployment descriptor file used to enable the CDI runtime.
- web/WEB-INF/faces-config.xml
The JavaServer Faces configuration file.
- web/WEB-INF/glassfish-web.xml
The configuration file specific to GlassFish Server.
- web/WEB-INF/web.xml
The web application configuration file.
- src/conf/persistence.xml
The Java Persistence API configuration file.