MSGS: Multispectral 3D Gaussian Splatting

arXiv cs.CV / 4/16/2026

💬 OpinionSignals & Early TrendsIdeas & Deep AnalysisModels & Research

共有:

Key Points

The paper introduces MSGS, a multispectral extension of 3D Gaussian Splatting that augments each Gaussian with wavelength-aware spectral radiance modeled via per-band spherical harmonics.
It uses a dual-loss training approach that supervises both RGB and multispectral signals, enabling wavelength-aware view synthesis rather than relying on RGB-only information.
To preserve richer spectral cues during optimization, the method performs spectral-to-RGB conversion at the pixel level, improving rendering fidelity.
Experiments on public and self-captured datasets show improvements over the RGB-only 3DGS baseline in both image quality and spectral consistency, especially for translucent materials and anisotropic reflections.
The authors claim MSGS retains 3DGS’s compactness and real-time efficiency while enabling future work toward integration with physically based shading models.

Abstract

We present a multispectral extension to 3D Gaussian Splatting (3DGS) for wavelength-aware view synthesis. Each Gaussian is augmented with spectral radiance, represented via per-band spherical harmonics, and optimized under a dual-loss supervision scheme combining RGB and multispectral signals. To improve rendering fidelity, we perform spectral-to-RGB conversion at the pixel level, allowing richer spectral cues to be retained during optimization. Our method is evaluated on both public and self-captured real-world datasets, demonstrating consistent improvements over the RGB-only 3DGS baseline in terms of image quality and spectral consistency. Notably, it excels in challenging scenes involving translucent materials and anisotropic reflections. The proposed approach maintains the compactness and real-time efficiency of 3DGS while laying the foundation for future integration with physically based shading models.