Proposing Topic Models and Evaluation Frameworks for Analyzing Associations with External Outcomes: An Application to Leadership Analysis Using Large-Scale Corporate Review Data

arXiv cs.CL / 4/22/2026

📰 NewsModels & Research

共有:

Key Points

The paper addresses a gap in topic modeling where current methods fail to balance interpretability, topic specificity to concrete characteristics/actions, and consistent sentiment polarity within each topic.
It proposes a leadership-analysis approach that uses large language models to generate topics meeting these requirements, plus an evaluation framework designed specifically for analyzing relationships with external outcomes.
The evaluation framework explicitly includes topic specificity and polarity stance consistency as scoring criteria, and it tests automated evaluation using existing metrics.
Using employee reviews from OpenWork, the method improves interpretability, specificity, and polarity consistency versus prior approaches.
In downstream analysis of external outcomes such as employee morale, the approach yields topics with stronger explanatory power, suggesting broader applicability beyond leadership analytics.

Abstract

Analyzing topics extracted from text data in relation to external outcomes is important across fields such as computational social science and organizational research. However, existing topic modeling methods struggle to simultaneously achieve interpretability, topic specificity (alignment with concrete actions or characteristics), and polarity stance consistency (absence of mixed positive and negative evaluations within a topic). Focusing on leadership analysis using corporate review data, this study proposes a method leveraging large language models to generate topics that satisfy these properties, along with an evaluation framework tailored to external outcome analysis. The framework explicitly incorporates topic specificity and polarity stance consistency as evaluation criteria and examines automated evaluation methods based on existing metrics. Using employee reviews from OpenWork, a major corporate review platform in Japan, the proposed method achieves improved interpretability, specificity, and polarity consistency compared to existing approaches. In analyses of external outcomes such as employee morale, it also produces topics with higher explanatory power. These results suggest that the proposed method and evaluation framework provide a generalized approach for topic analysis in applications involving external outcomes.