温度 scaling is a post-processing technique applied to the outputs of 機械学習 models, particularly in classification tasks. It is primarily used to calibrate the probabilities produced by models, ensuring they reflect the true likelihoods of the predicted classes. This method is especially useful when a model’s predictions are overly confident or miscalibrated, which can lead to suboptimal decision-making.
温度スケーリングの概念は、統計熱力学に由来します thermodynamics. In this context, ‘temperature’ is a parameter that controls the smoothness of the softmax function, which converts raw model outputs (logits) into probabilities. By adjusting this temperature parameter, we can either sharpen or soften the distribution of predicted probabilities.
温度が1未満（T < 1), the softmax function amplifies the differences between the logits, making the model more confident in its predictions. Conversely, when the temperature is greater than one (T > 1), the softmax function flattens the logits, resulting in more uniform probabilities and reduced confidence. The goal is to find an optimal temperature that minimizes the difference between the predicted probabilities and the true class frequencies.
To implement temperature scaling, one typically performs a calibration step on a separate validation dataset. This involves finding the best temperature value through a simple grid search or 最適化手法 that minimize a loss function, such as the negative log-likelihood. The calibrated model can then provide better-calibrated predictions, improving performance in applications where understanding uncertainty is crucial, such as in medical diagnoses or financial forecasting.