Multi-Dataset Cross-Domain Knowledge Distillation for Unified Medical Image Segmentation, Classification, and Detection

arXiv cs.CV / 5/5/2026

📰 NewsIdeas & Deep AnalysisModels & Research

共有:

Key Points

The paper introduces a unified cross-domain transfer learning framework that uses knowledge from multiple heterogeneous medical imaging datasets to improve segmentation, classification, and object detection together.
It applies a teacher–student paradigm where a joint teacher aggregates domain-invariant representations and a task-specific student learns through multi-level knowledge distillation.
The method is originally built for segmentation but is extended to support image-level classification and bounding-box-based detection, forming a general multi-task setup for medical imaging.
Experiments across many datasets and modalities (MRI/CT, including segmentation benchmarks, classification sets, and detection datasets) show consistent gains over both dataset-specific and multi-head baselines.
The results indicate improved robustness to distribution shifts and better generalization across heterogeneous medical domains, suggesting a scalable and task-agnostic distillation approach.

Abstract

We propose a unified cross-domain transfer learning framework that leverages knowledge from multiple heterogeneous medical imaging datasets to improve performance across segmentation, classification, and object detection tasks. Our approach employs a teacher-student paradigm in which a joint teacher model aggregates domain-invariant representations learned from diverse source datasets, while a task-specific student model is trained via multi-level knowledge distillation. Originally developed for medical image segmentation, the framework is extended to support image-level classification and object-level detection, enabling a general multi-task formulation for medical image analysis. We evaluate our method on a broad suite of datasets, including six segmentation benchmarks, BrainMetShare, ISLES, BraTS (MRI) and Lung MSD, LiTS, KiTS (CT), as well as multiple classification datasets for pulmonary disease and dementia, and detection datasets with native bounding-box annotations. Across all tasks and modalities, the proposed approach yields consistent improvements over strong dataset-specific and multi-head baselines, demonstrating enhanced robustness to distributional shifts and superior generalization. These findings highlight the potential of multi-dataset knowledge distillation as a scalable and task-agnostic approach for enhancing segmentation, classification, and object detection performance across heterogeneous medical imaging domains.