Social media polarization during conflict: Insights from an ideological stance dataset on Israel-Palestine Reddit comments

arXiv cs.CL / 4/16/2026

💬 OpinionSignals & Early TrendsIdeas & Deep AnalysisModels & Research

共有:

Key Points

The paper studies ideological polarization in a conflict setting by analyzing 9,969 Israel-Palestine–related Reddit comments collected from October 2023 to August 2024.
It frames the task as stance classification into three categories—Pro-Israel, Pro-Palestine, and Neutral—and evaluates multiple approaches including machine learning, neural networks, pre-trained language models, and prompt engineering for open-source LLMs.
Performance is measured with accuracy, precision, recall, and F1-score, enabling a direct comparison of different model families and prompting strategies.
The best results come from a prompt strategy (“Scoring and Reflective Re-read”) applied to Mixtral 8x7B, which outperforms other tested methods across all reported metrics.
The dataset is made publicly available to support further analysis and external validation of stance-detection methods in highly polarized contexts.

Abstract

In politically sensitive scenarios like wars, social media serves as a platform for polarized discourse and expressions of strong ideological stances. While prior studies have explored ideological stance detection in general contexts, limited attention has been given to conflict-specific settings. This study addresses this gap by analyzing 9,969 Reddit comments related to the Israel-Palestine conflict, collected between October 2023 and August 2024. The comments were categorized into three stance classes: Pro-Israel, Pro-Palestine, and Neutral. Various approaches, including machine learning, pre-trained language models, neural networks, and prompt engineering strategies for open source large language models (LLMs), were employed to classify these stances. Performance was assessed using metrics such as accuracy, precision, recall, and F1-score. Among the tested methods, the Scoring and Reflective Re-read prompt in Mixtral 8x7B demonstrated the highest performance across all metrics. This study provides comparative insights into the effectiveness of different models for detecting ideological stances in highly polarized social media contexts. The dataset used in this research is publicly available for further exploration and validation.