Validity-Calibrated Reasoning Distillation

arXiv cs.AI / 5/7/2026

💬 OpinionIdeas & Deep AnalysisModels & Research

共有:

Key Points

The paper introduces “validity-calibrated reasoning distillation,” aiming to transfer multi-step reasoning from large language models to smaller, more efficient ones while addressing limitations of prior trajectory-imitating approaches.
Instead of token-level or fixed teacher-student hierarchy imitation, the method compares the teacher’s and student’s proposed next actions under the same prefix and uses their relative local validity to scale how strongly the student is updated.
This reframes distillation as allocating local learning signal rather than aligning full reasoning paths, better matching the fact that intermediate steps can be locally under-specified.
Experiments on mathematical reasoning, code generation, and instruction-following benchmarks show consistent improvements over strong distillation baselines.
The results suggest that effective reasoning distillation depends more on principled, context-dependent calibration of supervision than on rigid path imitation.

Abstract

Reasoning distillation aims to transfer multi-step reasoning capabilities from large language models to smaller, more efficient ones. While recent methods have shown promising gains, they typically rely on static teacher-student hierarchies and frame distillation as trajectory imitation. This is misaligned with the structure of reasoning, where intermediate steps are often locally under-specified: global correctness constrains the final answer, but does not uniquely determine each intermediate move. We propose validity-calibrated reasoning distillation, a framework that treats reasoning distillation as a problem of local learning-signal allocation rather than path alignment. Instead of enforcing token-level imitation, we compare the student's and teacher's proposed next-step actions under the same prefix and use their relative local validity to modulate the strength of the distillation update. This yields a dynamic, context-dependent supervision mechanism that preserves the teacher's structural guidance while adapting update strength to local reasoning quality. Across mathematical reasoning, code generation, and instruction-following benchmarks, our method consistently outperforms strong distillation baselines. These results indicate that effective LLM reasoning distillation is governed not by rigid trajectory imitation, but by principled, locally calibrated allocation of learning signal.