The Brier Score is a statistical measure used to assess the accuracy of probabilistic predictions. It is particularly useful in fields such as meteorology, machine learning, and any domain where predictions involve probabilities. The score is calculated as the mean squared difference between predicted probabilities and the actual outcomes, which are typically binary (0 or 1). This method provides a clear numerical value that reflects how well the predicted probabilities correspond to the actual results.
Mathematically, the Brier Score is defined as:
Brier Score = (1/N) * Σ (f_i – o_i)²
Where:
- N is the total number of predictions.
- f_i represents the predicted probability of the event occurring.
- o_i is the actual outcome (1 if the event occurred, 0 if it did not).
The resulting score ranges from 0 to 1, where a Brier Score of 0 indicates perfect accuracy (all predictions are correct), while a score of 1 indicates complete inaccuracy (all predictions are wrong). One of the strengths of the Brier Score is its sensitivity to both the calibration (how well the predicted probabilities reflect the actual outcomes) and the refinement (how close the predicted probabilities are to the actual outcomes).
In practice, lower Brier Scores are preferred, as they indicate better predictive performance. This metric is particularly useful for evaluating models that produce probabilities, such as logistic regression and various machine learning classifiers.