Adaptive Coverage Policies in Conformal Prediction

arXiv stat.ML / 4/3/2026

💬 OpinionIdeas & Deep AnalysisModels & Research

共有:

Key Points

The paper addresses a key limitation of standard conformal prediction: using a fixed, user-chosen coverage level can yield either overly conservative or empty/uninformative prediction sets.
It introduces an adaptive coverage policy that varies the coverage level per example based on its characteristics, improving efficiency by allowing prediction-set sizes to change with difficulty.
The method leverages recent techniques such as e-values and post-hoc conformal inference to retain valid statistical guarantees even when coverage is data-dependent.
The authors train a neural network for the coverage policy using a leave-one-out procedure on the calibration set, and provide both theoretical coverage guarantees and experimental validation.

Abstract

Traditional conformal prediction methods construct prediction sets such that the true label falls within the set with a user-specified coverage level. However, poorly chosen coverage levels can result in uninformative predictions, either producing overly conservative sets when the coverage level is too high, or empty sets when it is too low. Moreover, the fixed coverage level cannot adapt to the specific characteristics of each individual example, limiting the flexibility and efficiency of these methods. In this work, we leverage recent advances in e-values and post-hoc conformal inference, which allow the use of data-dependent coverage levels while maintaining valid statistical guarantees. We propose to optimize an adaptive coverage policy by training a neural network using a leave-one-out procedure on the calibration set, allowing the coverage level and the resulting prediction set size to vary with the difficulty of each individual example. We support our approach with theoretical coverage guarantees and demonstrate its practical benefits through a series of experiments.