A Baseline for Detecting Misclassified and Out-of-Distribution Examples in Neural Networks

This paper tackles two related detection problems—flagging when a neural network's prediction is misclassified, and flagging when an input is out-of-distribution. The authors propose a simple baseline that reads off the probabilities from the softmax output, using the maximum softmax probability as a confidence signal, based on the observation that correctly classified examples tend to have greater maximum probabilities than misclassified or out-of-distribution examples.

They define several evaluation tasks spanning computer vision, natural language processing, and automatic speech recognition, and show the softmax baseline is effective across all of them. They also demonstrate that the baseline can sometimes be surpassed, establishing both a standard reference point and evidence that these detection tasks remain underexplored and open to future research.