La Unidad Lineal Exponencial Escalada (SELU) es una función de activación used in redes neuronales, particularly in aprendizaje profundo models. It was introduced to help address issues of vanishing and la explosión de gradientes that can occur during training. The SELU function is defined mathematically as follows:
Para una entrada x, the output f(x) es:
f(x) = λ * (x if x > 0 else α * (exp(x) – 1))
donde:
- λ (lambda) es un factor de escala, generalmente establecido en aproximadamente 1.0507.
- α (alpha) es un parámetro, usualmente alrededor de 1.6733.
SELU has a unique property of self-normalization, meaning that when used appropriately in a network, it helps maintain the mean and variance of the activations close to zero and one, respectively. This property facilitates faster convergence during training and can improve overall rendimiento del modelo.
To effectively use SELU, it is recommended to initialize the weights of the neural network using the LeCun normal initialization method and to avoid dropout layers, as SELU is designed to work best in fully connected architectures without such técnicas de regularización.
Overall, the SELU activation function is particularly beneficial for deep networks, as it helps stabilize the training process and can lead to better generalization en datos no vistos.