ELU-Aktivierung
ELU, oder Exponential Linear Unit, ist eine Aktivierungsfunktion used in artificial neuronale Netze to introduce non-linearity into the model. It is particularly valued for its ability to mitigate the ‘dying ReLU’ problem, which occurs when neurons output zero für alle Eingaben, wodurch sie effektiv inaktiv werden und nicht mehr lernen.
Die ELU-Funktion ist mathematisch wie folgt definiert:
For an input x, the ELU activation function is:
ELU(x) = x, if x > 0 ELU(x) = α * (e^x - 1), if x ≤ 0
Here, α is a hyperparameter that determines the value of the output for negative inputs. The exponential component for negative inputs allows ELU to produce outputs that are non-zero and smooth, which helps in maintaining a mean output close to zero. This property is an advantage over the standard ReLU Funktion, die für alle negativen Eingaben Null ausgibt.
Verwendung von ELU in Deep Learning models has been shown to accelerate learning and improve accuracy in certain tasks, especially when dealing with deep architectures. It retains all the benefits of ReLU while providing a gradient for negative inputs, which can lead to better convergence during training.
Zusammenfassend ist ELU Aktivierungsfunktionen provide a robust alternative to traditional activation functions, particularly in deep neural networks, by addressing some of their inherent limitations.