When Roles Fail: Epistemic Constraints on Advocate Role Fidelity in LLM-Based Political Statement Analysis

arXiv cs.AI / 5/1/2026

📰 NewsIdeas & Deep AnalysisModels & Research

共有:

Key Points

The paper tests a key assumption behind multi-agent LLM political discourse systems—that evaluator models reliably keep their assigned adversarial advocate roles—using the TRUST pipeline.
It introduces an epistemic stance classifier to infer advocate roles from reasoning text and evaluates role fidelity on 60 political statements (30 English, 30 German) with metrics including Role Drift Index (RDI) and Entropy-based Role Stability (ERS).
Two role-failure modes are identified—Epistemic Floor Effect and Role-Prior Conflict—and they are shown to stem from a single mechanism called Epistemic Role Override (ERO).
Model and component choices materially change role fidelity: Mistral Large outperforms Claude Sonnet by 28 percentage points (67% vs. 39%), and fact-check provider selection can reduce Claude’s fidelity for German inputs.
The authors argue that multi-agent LLM validation that omits explicit role-fidelity measurement can systematically misrepresent the epistemic diversity the system is intended to produce.

Abstract

Democratic discourse analysis systems increasingly rely on multi-agent LLM pipelines in which distinct evaluator models are assigned adversarial roles to generate structured, multi-perspective assessments of political statements. A core assumption is that models will reliably maintain their assigned roles. This paper provides the first systematic empirical test of that assumption using the TRUST pipeline. We develop an epistemic stance classifier that identifies advocate roles from reasoning text without relying on surface vocabulary, and measure role fidelity across 60 political statements (30 English, 30 German) using four metrics: Role Drift Index (RDI), Expected Drift Distance (EDD), Directional Drift Index (DDI), and Entropy-based Role Stability (ERS). We identify two failure modes - the Epistemic Floor Effect (fact-check results create an absolute lower bound below which the legitimizing role cannot be maintained) and Role-Prior Conflict (training-time knowledge overrides role instructions for factually unambiguous statements) - as manifestations of a single mechanism: Epistemic Role Override (ERO). Model choice significantly affects role fidelity: Mistral Large outperforms Claude Sonnet by 28pp (67% vs. 39%) and exhibits a qualitatively different failure mode - role abandonment without polarity reversal - compared to Claude's active switch to the opposing stance. Role fidelity is language-robust. Fact-check provider choice is not universally neutral: Perplexity significantly reduces Claude's role fidelity on German statements (Delta = -15pp, p = 0.007) while leaving Mistral unaffected. These findings have direct implications for multi-agent LLM validation: a system validated without role fidelity measurement may systematically misrepresent the epistemic diversity it was designed to provide.