Применение больших языковых моделей для анализа ценностно-патриотического дискурса русскоязычных пользователей

Вестник Российского фонда фундаментальных исследований. Гуманитарные и общественные науки. 2025. Т. 123. № 4. С. 56–69.

Balakina Y. V., Григорьева М. В., Соколова Е. Н.

The article examines the potential of large language models (LLMs) for automated analysis of value-laden and patriotic discourse in Russian-language social media. Using a corpus of posts from VK, Odnoklassniki and Telegram (2023–2025), it investigates the extent to which automatic coding results align with expert annotation based on a specially developed categorical scheme. The codebook comprises eight dimensions: basic values according to Sh. Schwartz; two axes of R. Inglehart (traditionalism vs. secularity; survival vs. self-expression); levels of needs following A. Maslow; types of patriotism (constructive vs. aggressive) inspired by K. D. Ushinsky and V. S. Solovyov; dominant speech act types following J. Austin; as well as binary indicators of explicit patriotism and civic identity. The experiment is conducted on the “Pride and Patriotism” message cluster (N = 456), where the density of value markers is maximal; comparison is implemented via confusion matrices, accuracy, macro/weighted F1 and Cohen’s κ. The results show that LLMs reliably detect explicit patriotic content, but are substantially less consistent with experts in multiclass and fine-grained value classification (Schwartz, Maslow, Inglehart scales, types of patriotism, Austin’s speech acts), exhibiting systematic biases and over-detection of certain categories. It is concluded that, in their current configuration, LLMs can be used as an auxiliary tool for preliminary annotation and hypothesis generation, but not as an autonomous substitute for expert content analysis of value-related discourse.

Keywords: патриотизм, социальные медиа, речевые акты, patriotism, Гражданская идентичность, social media, speech acts, civic identity, большие языковые модели, large language models, LLMs, values, ценности, Russian-language discourse, automatic annotation, русскоязычный дискурс, автоматическая разметка

Publication based on the results of: Modern Technologies of Socio-Political Communication (2025)