スケジュールサンプリングはトレーニング手法です 機械学習で使用される, particularly in the context of sequence-to-sequence models like those found in 自然言語処理 and generative tasks. The primary goal of Scheduled Sampling is to mitigate the problem of exposure bias, which occurs when a model is trained on ground truth data during training but must rely on its own predictions during inference.
In traditional training, a model learns to predict the next element in a sequence based solely on the previous elements, using actual historical data. However, during inference or testing, the model must generate sequences based on its own predictions, which may lead to compounding errors if the initial predictions are incorrect. This discrepancy can negatively affect the model’s performance.
Scheduled Sampling addresses this issue by gradually transitioning the model from training with ground truth data to training with its own predictions. It does this by incorporating a probabilistic sampling strategy where, at each time step, the model either receives the actual previous input (with a certain probability) or its own predicted output from the previous time step. This probability is adjusted over time, starting with a higher likelihood of using ground truth data and decreasing it as training progresses.
This method allows the model to become more robust and better at handling its own predictions during inference, ultimately improving its ability to generate high-quality outputs. Scheduled Sampling has been particularly effective in applications such as 機械翻訳, image captioning, and speech recognition, where maintaining coherence and context over sequences is crucial.