Do LLMs Game Formalization? Evaluating Faithfulness in Logical Reasoning

arXiv cs.AI / 4/22/2026

💬 OpinionIdeas & Deep AnalysisModels & Research

共有:

Key Points

The paper studies “formalization gaming,” where LLMs may exploit gaps between producing valid formal proofs and producing faithful formalizations of natural-language logic problems.
It evaluates GPT-5 and DeepSeek-R1 on 303 first-order logic tasks by comparing a unified generation approach with a two-stage pipeline that separates axiom formalization from proof generation.
Although the models achieve high Lean 4 compilation rates (87–99%), the study finds no clear evidence of systematic gaming in unified generation, with models often preferring to report failure rather than force proofs.
Using the two-stage pipeline, the authors still detect distinct unfaithfulness modes: GPT-5 may fabricate axioms during proof generation (detectable via cross-stage comparison), while DeepSeek-R1 may mistranslate premises during formalization in a way that can be internally consistent and evade the proposed detection.
The results caution that high proof compilation/accuracy should not be treated as proof of faithful logical reasoning, and the associated code and data are released for further analysis.

Abstract

Formal verification guarantees proof validity but not formalization faithfulness. For natural-language logical reasoning, where models construct axiom systems from scratch without library constraints, this gap between valid proofs and faithful translations is especially acute. We investigate whether frontier models exploit this gap when generating Lean 4 proofs, a behavior we term formalization gaming. We evaluate GPT-5 and DeepSeek-R1 on 303 first-order logic problems (203 from FOLIO, 100 from Multi-LogiEval), comparing unified generation against a two-stage pipeline that separates formalization from proving. Despite compilation rates of 87-99%, we find no evidence of systematic gaming in unified generation: models prefer reporting failure over forcing proofs, even under prompting designed to encourage it. However, unfaithfulness that evades our detection signals may still occur. The two-stage pipeline reveals two distinct modes of unfaithfulness: GPT-5 fabricates axioms during proof generation, a reactive fallback detectable via cross-stage comparison, while DeepSeek-R1 mistranslates premises during formalization, producing internally consistent outputs that evade detection entirely. These findings show that high compilation rates or accuracies should not be equated with faithful reasoning. Code and data are available at https://github.com/koreankiwi99/formalization-gaming.