Katastrophale Störung ist ein Phänomen, das bei künstlichen neuronale Netze, particularly in those that employ überwachten Lernens. It occurs when a neuronales Netzwerk learns new information and subsequently forgets previously learned information. This challenge arises because traditional neural networks typically adjust their weights based on new learning, which can lead to the overwriting of the weights that encode older knowledge.
Dieses Problem ist besonders problematisch in Anwendungen, bei denen kontinuierliches Lernen is essential, such as in robotics or der Verarbeitung natürlicher Sprache. When a model is trained on a new task, the alterations made to its weights can unintentionally degrade its performance on tasks it had been previously trained on. For example, if a language model is trained to understand English and is later trained to understand Spanish, it may lose some of its proficiency in English as a result of adjusting its internal parameters to accommodate Spanish.
Several strategies have been proposed to mitigate catastrophic interference, including techniques like elastische Gewichts-Konsolidierung, which helps preserve important weights, and rehearsal methods, where old data is revisited during training. Another approach involves using architectures designed for incremental learning, such as modular networks that compartmentalize knowledge in a way that minimizes interference. Understanding and addressing catastrophic interference is crucial for developing robust AI systems that can learn continuously without losing previously acquired knowledge.