Culturally Adaptive Explainable LLM Assessment for Multilingual Information Disorder: A Human-in-the-Loop Approach

arXiv cs.CL / 3/31/2026

💬 OpinionSignals & Early TrendsIdeas & Deep AnalysisModels & Research

共有:

Key Points

The paper argues that assessing “information disorder” (manipulation) is inherently culture- and language-dependent, and that current LLMs often fail by giving fluent but monocultural, English-centric explanations.
It proposes a human-in-the-loop hybrid intelligence loop that evaluates LLMs using human-written rationales from native-speaking annotators, aiming for more culturally appropriate explainable outputs.
Instead of relying on static few-shot prompting, the method uses dynamically retrieved, target-language exemplars from a multilingual Information Disorder (InDor) corpus via in-context learning (ICL).
A pilot compares static vs adaptive prompting for Farsi and Italian news, measuring prediction performance (span/severity) and the quality/cultural fit of generated rationales across evaluator groups.
Overall, the work presents a testbed for culturally grounded explainable AI for multilingual information disorder detection and assessment.

Abstract

Recognizing information disorder is difficult because judgments about manipulation depend on cultural and linguistic context. Yet current Large Language Models (LLMs) often behave as monocultural, English-centric "black boxes," producing fluent rationales that overlook localized framing. Preliminary evidence from the multilingual Information Disorder (InDor) corpus suggests that existing models struggle to explain manipulated news consistently across communities. To address this gap, this ongoing study proposes a Hybrid Intelligence Loop, a human-in-the-loop (HITL) framework that grounds model assessment in human-written rationales from native-speaking annotators. The approach moves beyond static target-language few-shot prompting by pairing English task instructions with dynamically retrieved target-language exemplars drawn from filtered InDor annotations through In-Context Learning (ICL). In the initial pilot, the Exemplar Bank is seeded from these filtered annotations and used to compare static and adaptive prompting on Farsi and Italian news. The study evaluates span and severity prediction, the quality and cultural appropriateness of generated rationales, and model alignment across evaluator groups, providing a testbed for culturally grounded explainable AI.