Adaptive Guidance for Retrieval-Augmented Masked Diffusion Models

arXiv cs.CL / 4/6/2026

💬 OpinionIdeas & Deep AnalysisModels & Research

共有:

Key Points

The paper addresses how retrieval-augmented generation can fail when retrieved context is noisy, unreliable, or conflicts with what the model already knows, creating “retrieval–prior conflicts.”
It proposes ARAM, a training-free adaptive guidance method for retrieval-augmented masked diffusion models that recalibrates the guidance strength during denoising based on the SNR of the distributional shift caused by retrieved context.
ARAM effectively increases guidance when retrieval provides reliable corrective evidence and reduces it when the retrieved signal is noisy or non-supportive, aiming to prevent harmful reliance on bad context.
The authors report that experiments on multiple knowledge-intensive QA benchmarks show improved question-answering performance over competitive RAG baselines.

Abstract

Retrieval-Augmented Generation (RAG) improves factual grounding by incorporating external knowledge into language model generation. However, when retrieved context is noisy, unreliable, or inconsistent with the model's parametric knowledge, it introduces retrieval-prior conflicts that can degrade generation quality. While this problem has been studied in autoregressive language models, it remains largely unexplored in diffusion-based language models, where the iterative denoising process introduces unique challenges for integrating retrieved context. In this work, we propose Adaptive Retrieval-Augmented Masked Diffusion (ARAM), a training-free adaptive guidance framework for Masked Diffusion Models (MDMs) in RAG settings. ARAM dynamically calibrates the guidance scale during denoising according to the Signal-to-Noise Ratio (SNR) of the distributional shift induced by retrieved context. Intuitively, the model strengthens guidance when the retrieved context provides reliable corrective evidence and suppresses it when the contextual signal is noisy or non-supportive. Extensive experiments on multiple knowledge-intensive QA benchmarks show that ARAM improves overall QA performance over competitive RAG baselines.