Semi-Supervised Flow Matching for Mosaiced and Panchromatic Fusion Imaging

arXiv cs.CV / 4/23/2026

📰 NewsModels & Research

共有:

Key Points

The paper tackles the ill-posed problem of fusing low-resolution mosaiced hyperspectral images with high-resolution panchromatic images for fast, single-shot HR-HSI imaging.
It introduces a semi-supervised flow matching framework that combines an unsupervised prior with conditional flow matching to generate target HR-HSI without relying on fixed protocols or handcrafted assumptions.
The method uses a two-stage training process: pretraining an unsupervised network to create a pseudo HR-HSI, then training a conditional flow model with a random voting mechanism to iteratively refine that estimate.
For inference, it applies a conflict-free gradient guidance strategy to enforce spectral and spatial consistency in the reconstructed HR-HSI.
Experiments on multiple benchmark datasets show that the approach delivers substantially better quantitative and qualitative results than representative baselines, and it is designed to be extensible to other image fusion and restoration tasks.

Abstract

Fusing a low resolution (LR) mosaiced hyperspectral image (HSI) with a high resolution (HR) panchromatic (PAN) image offers a promising avenue for video-rate HR-HSI imaging via single-shot acquisition, yet its severely ill-posed nature remains a significant challenge. In this work, we propose a novel semi-supervised flow matching framework for mosaiced and PAN image fusion. Unlike previous diffusion-based approaches constrained by specific protocols or handcrafted assumptions, our method seamlessly integrates an unsupervised scheme with flow matching, resulting in a generalizable and efficient generative framework. Specifically, our method follows a two-stage training pipeline. First, we pretrain an unsupervised prior network to produce an initial pseudo HR-HSI. Building on this, we then train a conditional flow matching model to generate the target HR-HSI, introducing a random voting mechanism that iteratively refines the initial HR-HSI estimate, enabling robust and effective fusion. During inference, we employ a conflict-free gradient guidance strategy that ensures spectrally and spatially consistent HR-HSI reconstruction. Experiments on multiple benchmark datasets demonstrate that our method achieves superior quantitative and qualitative performance by a significant margin compared to representative baselines. Beyond mosaiced and PAN fusion, our approach provides a flexible generative framework that can be readily extended to other image fusion tasks and integrated with unsupervised or blind image restoration algorithms.