NeoNet: An End-to-End 3D MRI-Based Deep Learning Framework for Non-Invasive Prediction of Perineural Invasion via Generation-Driven Classification

arXiv cs.CV / 4/1/2026

💬 OpinionSignals & Early TrendsIdeas & Deep AnalysisModels & Research

共有:

Key Points

The article introduces NeoNet, an end-to-end 3D deep learning framework to non-invasively predict perineural invasion (PNI) in cholangiocarcinoma using 3D MRI without relying on predefined imaging features.
NeoNet combines three modules: NeoSeg for ROI-based tumor localization, NeoGen to generate balanced synthetic training patches via a 3D latent diffusion model with ControlNet and anatomical-mask conditioning, and NeoCls for final classification.
For classification, the work proposes the PNI-Attention Network (PattenNet), which leverages a frozen diffusion-model encoder plus specialized 3D dual-attention blocks to capture subtle intensity and spatial patterns linked to PNI.
In 5-fold cross-validation, NeoNet outperformed baseline 3D models and reported a best AUC of 0.7903, suggesting improved predictive performance from the generation-driven, feature-free pipeline.

Abstract

Minimizing invasive diagnostic procedures to reduce the risk of patient injury and infection is a central goal in medical imaging. And yet, noninvasive diagnosis of perineural invasion (PNI), a critical prognostic factor involving infiltration of tumor cells along the surrounding nerve, still remains challenging, due to the lack of clear and consistent imaging criteria criteria for identifying PNI. To address this challenge, we present NeoNet, an integrated end-to-end 3D deep learning framework for PNI prediction in cholangiocarcinoma that does not rely on predefined image features. NeoNet integrates three modules: (1) NeoSeg, utilizing a Tumor-Localized ROI Crop (TLCR) algorithm; (2) NeoGen, a 3D Latent Diffusion Model (LDM) with ControlNet, conditioned on anatomical masks to generate synthetic image patches, specifically balancing the dataset to a 1:1 ratio; and (3) NeoCls, the final prediction module. For NeoCls, we developed the PNI-Attention Network (PattenNet), which uses the frozen LDM encoder and specialized 3D Dual Attention Blocks (DAB) designed to detect subtle intensity variations and spatial patterns indicative of PNI. In 5-fold cross-validation, NeoNet outperformed baseline 3D models and achieved the highest performance with a maximum AUC of 0.7903.