Density Estimation in Representation Space to Predict Model Uncertainty

Abstract

Deep learning models frequently make incorrect predictions with high confidence when presented with test examples that are not well represented in their training dataset. We propose a novel and straightforward approach to estimate prediction uncertainty in a pre-trained neural network model. Our method estimates the training data density in representation space for a novel input. A neural network model then uses this information to determine whether we expect the pre-trained model to make a correct prediction. This uncertainty model is trained by predicting in-distribution errors, but can detect out-of-distribution data without having seen any such example. We test our method for a state-of-the art image classification model in the settings of both in-distribution uncertainty estimation as well as out-of-distribution detection.

A Appendix

We tuned the hyperparameters L, the number of layers; and k the number of nearest numbers fed to the classifier. All models were trained using the Adam optimizer with a learning rate of $10^{-3}$ annealed to $10^{-4}$ after 40000 steps. We train each model for one single epoch before validating. We considered the following values for hyperparameters: $L\in [1, 2, 3]$ (with $L=1$ corresponding to a linear model) and $k\in [10, 50, 100, 200]$.

We chose the hyperparameters used in the paper $L=2, k=[10, 200]$ based on the AUROC  [11] on the in-distribution mistake prediction task using the ILSVRC2012 validation set (Fig. 4).

Fig. 4.

AUROC metric reported for models with $L\in [1, 2, 3]$ and $k\in [10, 50, 100, 200]$.