Psychologically Potent, Computationally Invisible: LLMs Generate Social-Comparison Triggers They Fail to Detect

arXiv cs.CL / 5/5/2026

📰 NewsIdeas & Deep AnalysisModels & Research

共有:

Key Points

The paper introduces XHS-SCoRE, a benchmark to detect whether text-only Xiaohongshu (RedNote) posts trigger upward, downward, or neutral social comparisons from a first-person reader’s perspective.
Experiments with prompted LLM classifiers and supervised Chinese encoder baselines show a gap: generated text can look fluent, yet the social-comparison signal is not reliably detectable via prompt-based classification.
The prompted LLMs fail in consistent, interpretable ways—often neutralizing comparison-triggering content and showing model-specific biases in predicted direction.
A controlled pilot suggests that even when detection is fragile, LLM-generated Xiaohongshu-style posts can still change perceived social standing and comparison-related emotions for readers.
The work provides both a reader-grounded benchmark and a framework for analyzing when socially meaningful relational cues are only partially visible to prompt-based inference.

Abstract

We introduce Xiaohongshu Social Comparison Reader Elicitation (XHS-SCoRE), a reader-grounded benchmark for detecting if a text-only Xiaohongshu (RedNote) post elicits UPWARD, DOWNWARD, or NEUTRAL/no clear social comparison from a first-person reader perspective. The task targets a socially meaningful relational signal that is behaviorally real yet not reducible to sentiment. Across prompted LLM classifiers and supervised Chinese encoder baselines, we find a consistent mismatch between generation fluency and reliable detection ability: the signal is textually learnable in-domain, but not robustly accessible to prompt-based classification. Prompted LLM classifiers exhibit stable, interpretable failure modes, especially neutralization of comparison-triggering posts and model-specific directional skew. A controlled pilot further shows that LLM-generated Xiaohongshu-style posts can shift perceived standing and comparison-related affect even when prompt-based detection of the same construct remains fragile. XHS-SCoRE contributes both a benchmark for reader-grounded comparison detection and a diagnostic framework for studying when socially meaningful relational cues remain only partially visible to prompt-based inference.