Consolidação de Pesos Elásticos (EWC)
Elástico Peso Consolidation is a regularization technique used in training redes neurais, particularly in scenarios involving aprendizado contínuo. The goal of EWC is to enable a model to learn new tasks without forgetting previously acquired knowledge, a phenomenon known as esquecimento catastrófico.
Quando uma rede neural is trained on a new task, it often adjusts its weights significantly, which can lead to a decline in performance on previously learned tasks. EWC addresses this challenge by introducing a penalty term to the loss function during training. This penalty discourages large changes to the weights that are crucial for the performance of older tasks.
The technique works by first estimating the importance of each weight, typically using the Matriz de informação de Fisher. This matrix quantifies how sensitive the loss is to changes in each weight, effectively highlighting which weights are critical for the tasks already learned. EWC then applies a quadratic penalty to these important weights, making it more costly to change them significantly when learning new tasks.
By balancing the learning of new tasks with the preservation of old knowledge, EWC allows for more robust and efficient training of neural networks in dynamic environments where tasks may be added or changed frequently. As a result, it is particularly useful in applications such as robotics, processamento de linguagem natural, and any area where models must adapt over time without losing their previous capabilities.