Pause or Fabricate? Training Language Models for Grounded Reasoning

arXiv cs.CL / 4/22/2026

📰 NewsIdeas & Deep AnalysisModels & Research

共有:

Key Points

The paper argues that “ungrounded reasoning” in large language models—confident but unreliable conclusions when inputs are incomplete—stems less from weak reasoning ability and more from missing inferential boundary awareness.
It proposes GRIL (Grounded Reasoning via Interactive Reinforcement Learning), a multi-turn RL framework that splits reasoning into “clarify and pause” (detect whether premises are sufficient) and “grounded reasoning” (solve once premises are established).
GRIL uses stage-specific rewards to penalize hallucinations, teaching models to proactively stop when information is missing and to resume after clarification.
Experiments on GSM8K-Insufficient and MetaMATH-Insufficient report up to 45% improvements in premise detection, a 30% rise in task success, and over 20% shorter average responses, with further evidence of robustness and out-of-distribution generalization.

Abstract

Large language models have achieved remarkable progress on complex reasoning tasks. However, they often implicitly fabricate information when inputs are incomplete, producing confident but unreliable conclusions -- a failure mode we term ungrounded reasoning. We argue that this issue arises not from insufficient reasoning capability, but from the lack of inferential boundary awareness -- the ability to recognize when the necessary premises for valid inference are missing. To address this issue, we propose Grounded Reasoning via Interactive Reinforcement Learning (GRIL), a multi-turn reinforcement learning framework for grounded reasoning under incomplete information. GRIL decomposes the reasoning process into two stages: clarify and pause, which identifies whether the available information is sufficient, and grounded reasoning, which performs task solving once the necessary premises are established. We design stage-specific rewards to penalize hallucinations, enabling models to detect gaps, stop proactively, and resume reasoning after clarification. Experiments on GSM8K-Insufficient and MetaMATH-Insufficient show that GRIL significantly improves premise detection (up to 45%), leading to a 30% increase in task success while reducing average response length by over 20%. Additional analyses confirm robustness to noisy user responses and generalization to out-of-distribution tasks.