From Plausible to Causal: Counterfactual Semantics for Policy Evaluation in Simulated Online Communities

arXiv cs.CL / 4/7/2026

📰 NewsIdeas & Deep AnalysisModels & Research

共有:

Key Points

The paper argues that LLM-based social simulations can look realistic but often lack explicit causal semantics, making claims about policy effects unreliable.
It proposes using a causal counterfactual framework that separates necessary causation (whether the outcome would have happened without intervention) from sufficient causation (whether the intervention reliably produces the outcome).
The authors map these causal notions to stakeholder needs: moderators benefit from necessary-causation evidence for incident diagnosis, while platform designers need sufficient-causation evidence to select policies.
They formalize how simulation design choices can support estimation of these causal quantities under explicit assumptions, yielding simulator-conditional causal estimates.
The work emphasizes that policy relevance depends on simulator fidelity and frames establishing this causality-aware evaluation approach as a key step for moving from plausible simulations to policy-changing “wind tunnels.”

Abstract

LLM-based social simulations can generate believable community interactions, enabling ``policy wind tunnels'' where governance interventions are tested before deployment. But believability is not causality. Claims like ``intervention

A

reduces escalation'' require causal semantics that current simulation work typically does not specify. We propose adopting the causal counterfactual framework, distinguishing \textit{necessary causation} (would the outcome have occurred without the intervention?) from \textit{sufficient causation} (does the intervention reliably produce the outcome?). This distinction maps onto different stakeholder needs: moderators diagnosing incidents require evidence about necessity, while platform designers choosing policies require evidence about sufficiency. We formalize this mapping, show how simulation design can support estimation under explicit assumptions, and argue that the resulting quantities should be interpreted as simulator-conditional causal estimates whose policy relevance depends on simulator fidelity. Establishing this framework now is essential: it helps define what adequate fidelity means and moves the field from simulations that look realistic toward simulations that can support policy changes.