SegWithU: Uncertainty as Perturbation Energy for Single-Forward-Pass Risk-Aware Medical Image Segmentation

arXiv cs.CV / 4/17/2026

📰 NewsSignals & Early TrendsTools & Practical UsageModels & Research

共有:

Key Points

SegWithU introduces a post-hoc uncertainty estimation framework for medical image segmentation that works with a frozen pretrained segmentation backbone and adds a lightweight uncertainty head.
It uses intermediate backbone features to model uncertainty as perturbation energy in a compact probe space via rank-1 posterior probes, producing voxel-wise uncertainty maps for both calibration and ranking.
The method outputs two uncertainty maps: one aimed at probability tempering (calibration) and another aimed at error detection and selective prediction (ranking).
On ACDC, BraTS2024, and LiTS, SegWithU is reported as the strongest and most consistent single-forward-pass baseline while preserving segmentation quality.
Source code is released on GitHub, enabling practical adoption and further experimentation with the approach.

Abstract

Reliable uncertainty estimation is critical for medical image segmentation, where automated contours feed downstream quantification and clinical decision support. Many strong uncertainty methods require repeated inference, while efficient single-forward-pass alternatives often provide weaker failure ranking or rely on restrictive feature-space assumptions. We present

\textbf{SegWithU}

, a post-hoc framework that augments a frozen pretrained segmentation backbone with a lightweight uncertainty head. SegWithU taps intermediate backbone features and models uncertainty as perturbation energy in a compact probe space using rank-1 posterior probes. It produces two voxel-wise uncertainty maps: a calibration-oriented map for probability tempering and a ranking-oriented map for error detection and selective prediction. Across ACDC, BraTS2024, and LiTS, SegWithU is the strongest and most consistent single-forward-pass baseline, achieving AUROC/AURC of

0.9838/2.4885

0.9946/0.2660

, and

0.9925/0.8193

, respectively, while preserving segmentation quality. These results suggest that perturbation-based uncertainty modeling is an effective and practical route to reliability-aware medical segmentation. Source code is available at https://github.com/ProjectNeura/SegWithU.