Discrete Guidance Matching: Exact Guidance for Discrete Flow Matching

arXiv stat.ML / 4/16/2026

💬 OpinionIdeas & Deep AnalysisModels & Research

共有:

Key Points

The paper argues that commonly used discrete guidance methods rely on first-order approximations that can be inaccurate in discrete state spaces, potentially causing large errors.
It proposes “Discrete Guidance Matching,” deriving an exact transition rate toward the target (desired) distribution from a learned discrete flow matching model.
The resulting guidance mechanism improves sampling efficiency by requiring only a single forward pass per sampling step while providing exact guidance rather than an approximation.
The framework is presented as general and can recover existing guidance approaches as special cases, and it is compatible with masked diffusion models as well.
Experiments show improved performance for energy-guided simulations and for preference alignment in text-to-image generation and multimodal understanding, with code released on GitHub.

Abstract

Guidance provides a simple and effective framework for posterior sampling by steering the generation process towards the desired distribution. When modeling discrete data, existing approaches mostly focus on guidance with the first-order approximation to improve the sampling efficiency. However, such an approximation is inappropriate in discrete state spaces since the approximation error could be large. A novel guidance framework for discrete data is proposed to address this problem: we derive the exact transition rate for the desired distribution given a learned discrete flow matching model, leading to guidance that only requires a single forward pass in each sampling step, significantly improving efficiency. This unified novel framework is general enough, encompassing existing guidance methods as special cases, and it can also be seamlessly applied to the masked diffusion model. We demonstrate the effectiveness of our proposed guidance on energy-guided simulations and preference alignment on text-to-image generation and multimodal understanding tasks. The code is available at https://github.com/WanZhengyan/Discrete-Guidance-Matching.