In the context of machine learning and artificial intelligence, logits refer to the output layer of a neural network before any activation function, such as softmax or sigmoid, is applied. These raw scores represent the model’s confidence for each class in classification tasks. The term is commonly used in the field of deep learning, particularly in multi-class classification problems.
When a neural network processes input data, it produces a set of logits that can be thought of as unbounded real-valued scores. For example, in a classification task with three classes, the network might output logits like [2.0, -1.5, 0.5]. These values do not represent probabilities; they can be positive or negative, and there is no inherent relationship between them.
To convert these logits into probabilities that sum to one—making them interpretable as the likelihood of each class occurring—we typically apply the softmax function. The softmax function exponentiates each logit and normalizes the results to produce a probability distribution. In our example, applying softmax to the logits [2.0, -1.5, 0.5] will yield a set of probabilities that indicates how confident the model is about each class.
Logits are essential for understanding how a model makes predictions, as they provide insight into the decision-making process of the network. By analyzing the logits, researchers and practitioners can better diagnose model behavior, tune hyperparameters, and improve overall performance.