Beyond Accuracy: Reliability and Uncertainty Estimation in Convolutional Neural Networks

arXiv cs.LG / 3/12/2026

📰 NewsIdeas & Deep AnalysisModels & Research

共有:

Key Points

The paper investigates reliability and uncertainty in CNNs by comparing Monte Carlo Dropout, a Bayesian approximation, with Conformal Prediction for uncertainty quantification.
Experiments on Fashion-MNIST using H-CNN VGG16 and GoogLeNet show VGG16 achieves higher accuracy but tends to be overconfident, whereas GoogLeNet provides better-calibrated uncertainty estimates.
Conformal Prediction yields statistically valid prediction sets, offering practical value for high-stakes decision-making contexts.
The study emphasizes evaluating model performance beyond accuracy to build more reliable and trustworthy deep learning systems.

Abstract

Deep neural networks (DNNs) have become integral to a wide range of scientific and practical applications due to their flexibility and strong predictive performance. Despite their accuracy, however, DNNs frequently exhibit poor calibration, often assigning overly confident probabilities to incorrect predictions. This limitation underscores the growing need for integrated mechanisms that provide reliable uncertainty estimation. In this article, we compare two prominent approaches for uncertainty quantification: a Bayesian approximation via Monte Carlo Dropout and the nonparametric Conformal Prediction framework. Both methods are assessed using two convolutional neural network architectures; H-CNN VGG16 and GoogLeNet, trained on the Fashion-MNIST dataset. The empirical results show that although H-CNN VGG16 attains higher predictive accuracy, it tends to exhibit pronounced overconfidence, whereas GoogLeNet yields better-calibrated uncertainty estimates. Conformal Prediction additionally demonstrates consistent validity by producing statistically guaranteed prediction sets, highlighting its practical value in high-stakes decision-making contexts. Overall, the findings emphasize the importance of evaluating model performance beyond accuracy alone and contribute to the development of more reliable and trustworthy deep learning systems.