Can AI Debias the News? LLM Interventions Improve Cross-Partisan Receptivity but LLMs Overestimate Their Own Effectiveness

arXiv cs.CL / 5/5/2026

💬 OpinionIdeas & Deep AnalysisModels & Research

共有:

Key Points

The study evaluates whether LLM-generated debiasing of liberal news headlines can improve conservative readers’ trust-related judgments through two pre-registered experiments.
In Experiment 1, subtle lexical changes (replacing emotive words with more moderate synonyms) produced no measurable effects on any outcome.
In Experiment 2, a more substantive reframing increased conservatives’ perceived trustworthiness, completeness, and willingness to engage with liberal headlines, without triggering a backfire effect among liberals.
The results also show a mismatch between LLM-simulated “silicon” participants and human readers: effects appeared in silicon but not in humans in Study 1, and were larger in magnitude in silicon for some outcomes in Study 2.
Moderation analyses indicate that LLMs overestimate their own effectiveness because their implicit model of who responds to debiasing does not match the psychological factors that actually predict human responsiveness, implying the need for human oversight.

Abstract

Partisan news media erode cross-partisan trust, but large language models (LLMs) offer a potential means of debiasing such content at scale. Across two pre-registered experiments, we tested whether LLM-generated debiasing of liberal news headlines could improve conservative readers' trust-relevant judgments. Study 1 found that subtle lexical debiasing (replacing emotive words with more moderate synonyms) had no effect on any outcome. Study 2 found that a more substantive reframing intervention significantly increased conservatives' perceived trustworthiness, completeness, and willingness to engage with liberal news headlines, without producing a backfire effect among a sample of liberals. In Study 1, the intervention produced robust effects among LLM-simulated silicon participants, whereas it had no impact on human readers. In Study 2, the intervention's effects among silicon participants aligned directionally with human responses but were significantly larger in magnitude for some outcomes. Moderation analyses revealed that the model's implicit theory of who responds to debiasing diverged from the psychological profile that actually predicted human responsiveness. These findings demonstrate that LLM-based debiasing can improve cross-partisan receptivity when targeting ideological framing rather than surface-level language, but that current models lack both the quantitative accuracy and qualitative psychological fidelity to evaluate their own interventions without human oversight.