DiscoTrace: Representing and Comparing Answering Strategies of Humans and LLMs in Information-Seeking Question Answering

arXiv cs.CL / 4/17/2026

📰 NewsIdeas & Deep AnalysisModels & Research

共有:

Key Points

The paper introduces DiscoTrace, a method for identifying the rhetorical strategies used by answerers in information-seeking question answering.
DiscoTrace represents answers as sequences of question-related discourse acts linked with interpretations of the original question and overlays these on rhetorical structure theory parses.
When applied to answers from nine distinct human communities, the study finds that communities differ in their preferences for how to construct answers.
By comparison, LLMs show limited rhetorical diversity and tend to choose breadth by addressing question interpretations that human answerers typically do not cover.
The authors argue that these insights can inform the development of more pragmatic LLM answerers that adapt strategy selection to conversational context in QA.

Abstract

We introduce DiscoTrace, a method to identify the rhetorical strategies that answerers use when responding to information-seeking questions. DiscoTrace represents answers as a sequence of question-related discourse acts paired with interpretations of the original question, annotated on top of rhetorical structure theory parses. Applying DiscoTrace to answers from nine different human communities reveals that communities have diverse preferences for answer construction. In contrast, LLMs do not exhibit rhetorical diversity in their answers, even when prompted to mimic specific human community answering guidelines. LLMs also systematically opt for breadth, addressing interpretations of questions that human answerers choose not to address. Our findings can guide the development of pragmatic LLM answerers that consider a range of strategies informed by context in QA.