Align then Refine: Text-Guided 3D Prostate Lesion Segmentation

arXiv cs.CV / 4/22/2026

📰 NewsDeveloper Stack & InfrastructureModels & Research

共有:

Key Points

The paper targets accurate 3D segmentation of prostate lesions from biparametric MRI, where cross-modal fusion and lesion-level semantics are still difficult for existing models.
It introduces a new multi-encoder U-Net design that adds (1) an alignment loss to strengthen foreground text-image similarity for lesion semantics, and (2) a heatmap loss to better calibrate similarity maps while suppressing background activations.
A final-stage confidence-gated multi-head cross-attention “refiner” performs localized boundary edits only in high-confidence regions to improve segmentation precision.
A phase-scheduled training strategy is used to stabilize optimization across the multiple components, and the method reports consistent gains, achieving new state-of-the-art results on the PI-CAI dataset.
The authors provide open-source code for the proposed approach to support reproduction and further research.

Abstract

Automated 3D segmentation of prostate lesions from biparametric MRI (bp-MRI) is essential for reliable algorithmic analysis, but achieving high precision remains challenging. Volumetric methods must combine multiple modalities while ensuring anatomical consistency, but current models struggle to integrate cross-modal information reliably. While vision-language models (VLMs) are replacing the currently used architectural designs, they still lack the fine-grained, lesion-level semantics required for effective localized guidance. To address these limitations, we propose a new multi-encoder U-Net architecture incorporating three key innovations: (1) an alignment loss that enhances foreground text-image similarity to inject lesion semantics; (2) a heatmap loss that calibrates the similarity map and suppresses spurious background activations; and (3) a final-stage, confidence-gated multi-head cross-attention refiner that performs localized boundary edits in high-confidence regions. A phase-scheduled training regime stabilizes the optimization of these components. Our method consistently outperforms prior approaches, establishing a new state-of-the-art on the PI-CAI dataset through enhanced multi-modal fusion and localized text guidance. Our code is available at https://github.com/NUBagciLab/Prostate-Lesion-Segmentation.