Beyond Surface Artifacts: Capturing Shared Latent Forgery Knowledge Across Modalities

arXiv cs.CV / 4/10/2026

📰 NewsSignals & Early TrendsIdeas & Deep AnalysisModels & Research

Key Points

  • The paper argues that multimodal deepfake forensics suffers a generalization bottleneck because existing methods overfit to superficial, modality-specific artifacts rather than shared latent forgery cues.
  • It proposes a modality-agnostic forgery (MAF) detection framework that decouples modality-specific styles to extract essential cross-modal latent forgery knowledge.
  • The study introduces Weak MAF (transferability to semantically related modalities) and Strong MAF (robustness to isolated “dark modalities”) to measure different levels of generalization.
  • To evaluate these limits, it presents the DeepModal-Bench benchmark, aggregating multimodal forgery detection algorithms and incorporating generalized learning approaches.
  • The authors report empirical evidence for universal forgery traces and substantial performance gains on unseen modalities using the MAF framework, positioning it as a step toward universal multimodal defense.

Abstract

As generative artificial intelligence evolves, deepfake attacks have escalated from single-modality manipulations to complex, multimodal threats. Existing forensic techniques face a severe generalization bottleneck: by relying excessively on superficial, modality-specific artifacts, they neglect the shared latent forgery knowledge hidden beneath variable physical appearances. Consequently, these models suffer catastrophic performance degradation when confronted with unseen "dark modalities." To break this limitation, this paper introduces a paradigm shift that redefines multimodal forensics from conventional "feature fusion" to "modality generalization." We propose the first modality-agnostic forgery (MAF) detection framework. By explicitly decoupling modality-specific styles, MAF precisely extracts the essential, cross-modal latent forgery knowledge. Furthermore, we define two progressive dimensions to quantify model generalization: transferability toward semantically correlated modalities (Weak MAF), and robustness against completely isolated signals of "dark modality" (Strong MAF). To rigorously assess these generalization limits, we introduce the DeepModal-Bench benchmark, which integrates diverse multimodal forgery detection algorithms and adapts state-of-the-art generalized learning methods. This study not only empirically proves the existence of universal forgery traces but also achieves significant performance breakthroughs on unknown modalities via the MAF framework, offering a pioneering technical pathway for universal multimodal defense.