Label Smoothing is a regularization technique im maschinellen Lernen, particularly in the training of neuronale Netze. It addresses the challenge of overfitting durch die Veränderung der Art und Weise, wie Labels im Trainingsprozess dargestellt werden.
Bei herkömmlichen überwachten Lernens, each training example is assigned a hard label, which is often a one-hot encoded vector. For instance, if there are three classes, the label for class one would be represented as [1, 0, 0]. While this approach works well, it can lead to models that are overly confident in their predictions, resulting in poor generalization to new, unseen data.
Label smoothing alters this by softening the target labels. Instead of assigning a hard label of [1, 0, 0], label smoothing might adjust the label to [0.9, 0.05, 0.05]. This slight modification means that the model learns to distribute some probability mass over other classes, which encourages it to be less confident about its predictions. As a result, the model is better able to handle uncertainty and variations in the input data.
This technique has been shown to improve performance in various tasks, such as image classification and der Verarbeitung natürlicher Sprache, especially when the training data is limited. By making the model less reliant on exact label matches, label smoothing can lead to improved robustness and accuracy in real-world applications.