Elastische Gewichtskonsolidierung (EWC)
Elastic Gewicht Consolidation is a regularization technique used in training neuronale Netze, particularly in scenarios involving kontinuierliches Lernen. The goal of EWC is to enable a model to learn new tasks without forgetting previously acquired knowledge, a phenomenon known as katastrophales Vergessen.
Wenn ein neuronales Netzwerk is trained on a new task, it often adjusts its weights significantly, which can lead to a decline in performance on previously learned tasks. EWC addresses this challenge by introducing a penalty term to the loss function during training. This penalty discourages large changes to the weights that are crucial for the performance of older tasks.
The technique works by first estimating the importance of each weight, typically using the Fisher-Informationsmatrix. This matrix quantifies how sensitive the loss is to changes in each weight, effectively highlighting which weights are critical for the tasks already learned. EWC then applies a quadratic penalty to these important weights, making it more costly to change them significantly when learning new tasks.
By balancing the learning of new tasks with the preservation of old knowledge, EWC allows for more robust and efficient training of neural networks in dynamic environments where tasks may be added or changed frequently. As a result, it is particularly useful in applications such as robotics, der Verarbeitung natürlicher Sprache, and any area where models must adapt over time without losing their previous capabilities.