• A
  • A
  • A
  • АБВ
  • АБВ
  • АБВ
  • A
  • A
  • A
  • A
  • A
Обычная версия сайта
  • RU
  • EN
  • HSE University
  • Publications
  • Articles
  • Using Generative Pretrained Transformer-3 Models for Russian News Clustering and Title Generation tasks
  • RU
  • EN
Расширенный поиск
Высшая школа экономики
Национальный исследовательский университет
Priority areas
  • business informatics
  • economics
  • engineering science
  • humanitarian
  • IT and mathematics
  • law
  • management
  • mathematics
  • sociology
  • state and public administration
by year
  • 2027
  • 2026
  • 2025
  • 2024
  • 2023
  • 2022
  • 2021
  • 2020
  • 2019
  • 2018
  • 2017
  • 2016
  • 2015
  • 2014
  • 2013
  • 2012
  • 2011
  • 2010
  • 2009
  • 2008
  • 2007
  • 2006
  • 2005
  • 2004
  • 2003
  • 2002
  • 2001
  • 2000
  • 1999
  • 1998
  • 1997
  • 1996
  • 1995
  • 1994
  • 1993
  • 1992
  • 1991
  • 1990
  • 1989
  • 1988
  • 1987
  • 1986
  • 1985
  • 1984
  • 1983
  • 1982
  • 1981
  • 1980
  • 1979
  • 1978
  • 1977
  • 1976
  • 1975
  • 1974
  • 1973
  • 1972
  • 1971
  • 1970
  • 1969
  • 1968
  • 1967
  • 1966
  • 1965
  • 1964
  • 1963
  • 1958
  • More
Subject
News
June 5, 2026
Neural Network Maps as a Method for Constructing Mathematical Models
Scientists from HSE University–Nizhny Novgorod and the Institute of Physics Belgrade, Serbia, are jointly exploring the application of machine learning techniques and neural networks to the study of nonlinear dynamics. Natalya Stankevich, Leading Research Fellow at the Laboratory of Topological Methods in Dynamics of the Faculty of Informatics, Mathematics, and Computer Science at HSE University–Nizhny Novgorod, spoke to the HSE News Service about this international project.
June 5, 2026
‘In the Age of Technology, It Is Interesting to Look into the Past and Think about What We Can Take from It
Polina Tabakova decided to apply for a Philology degree at HSE in Nizhny Novgorod because she grew up in Mari El and did not want to move far away from the Russian forests. In an interview for the Young Scientists of HSE University project, she spoke about the genre of the campus novel, the existential drama of Kolobok, and a blackout version of Eugene Onegin.
June 5, 2026
HSE Scientists Develop Method to Compress Large Language Models Without Losing Quality
Researchers from the AI and Digital Science Institute at the HSE Faculty of Computer Science have developed a new compression method for large language models such as GPT and LLaMA that reduces their size by 25–36% without additional training or significant loss of accuracy. This is the first approach to use mathematical transformations—specifically, rotations of model weights—to make models more amenable to compression with structured matrices. The study results have been published in ACL Findings 2025. The code is available on GitHub.

 

Have you spotted a typo?
Highlight it, click Ctrl+Enter and send us a message. Thank you for your help!

Publications
  • Books
  • Articles
  • Chapters of books
  • Working papers
  • Report a publication
  • Research at HSE

?

Using Generative Pretrained Transformer-3 Models for Russian News Clustering and Title Generation tasks

Komp'juternaja Lingvistika i Intellektual'nye Tehnologii. 2021. Vol. 20. P. 1214–1223.
Tikhonova M., Pisarevskaya D., Shavrina T., Shliazhko O.

The paper presents a methodology for news clustering and news headline generation based on the zero-shot
approach and minimal tuning of the RuGPT-3 architecture (Generative Pretrained Transformer 3 for Russian). The
solution is presented in a competition for news clustering, headline selection and generation.
The following approaches are described: 1) zero-shot unsupervised classification based on pairwise news
perplexity: the method requires no training or model fine-tuning and yields 0.7 F1-measure.
2) fine-tuning: news headlines generation with the best result 0.292 ROUGE and 0.596 BLEU.

Research target: Computer Science
Language: English
Full text
DOI
Text on another site
Keywords: text generationGenerative pretrained transformerEvaluation tracktext clustering ruGPT-3
Similar publications
Artificial intelligence and digital twins for failure prediction in data center cooling systems: a comprehensive literature review (2018–2026)
Hushchyn M., Butorova A., Bobakov V. et al., European Physical Journal: Special Topics 2026 P. 1–19
This paper presents a review of artificial intelligence (AI) methods for failure prediction in data center cooling systems, with a focus on the integration of digital twins (DTs), physics-informed learning, and graph-based models. Positioned within complex network science, this review addresses a limitation of conventional graph approaches—their reliance on pairwise connectivity—whereas real-world failures often arise ...
Added: June 10, 2026
Beyond Delta: Introducing an Angle Metric for Stylometric Similarity
Gorina O. G., / Series rs-8457236 "Research Square Preprints". 2025.
This paper addresses the problem of measuring stylometric similarity between texts. Traditional methods, such as Burrows’s Delta and its modifications, have several limitations, including dependence on a reference corpus and sensitivity to sample size, which reduces their reliability when working with short texts of fewer than 5,000 words. As an alternative, we propose a novel ...
Added: June 8, 2026
Innovations in Information and Decision Sciences. Proceedings of the 13th International Conference on Frontiers in Intelligent Computing: Theory and Applications (FICTA 2025), Volume 4
Springer, 2026.
The book presents the proceedings of the 13th International Conference on Frontiers of Intelligent Computing: Theory and Applications (FICTA 2024), held at Intelligent Systems Research Group (ISRG), London Metropolitan University, London, United Kingdom, during June 6–7, 2025. Researchers, scientists, engineers and practitioners exchange new ideas and experiences in the domain of intelligent computing theories with ...
Added: June 8, 2026
Proceedings of the 43rd International Conference on Machine Learning (ICML 2026)
Seul: PMLR, 2026.
Added: June 4, 2026
OpenAtom Foundation. Консорциум, развивающий Open Source в Китае.
Silakov D., Системный администратор 2026 № 3 С. 28–33
В статье про платформы для разработки открытого ПО в Китае мы рассказали про GitCode – молодой проект, позиционируемый как площадка для разработчиков со всего мира. Сейчас на GitCode размещаются проекты, созданные в КНР, но некоторые из них уже известны и на международной арене. Помочь открытым проектам в становлении, развитии и расширению аудитории призван фонд OpenAtom ...
Added: June 2, 2026
The recognition-by-components method
Slivnitsin P., Mylnikov L., Engineering Applications of Artificial Intelligence 2026 Vol. 179 Article 115185
The paper describes a applied artificial intelligence task of recognition-by-components method of real objects based on the recognition of a limited set of primitives or components. The recognition-by-components makes it possible to determine the components, that compose an object, and increase the number of recognizable objects without degrading the recognition quality. Training is performed on ...
Added: May 29, 2026
Brain-Computer Interfaces for Gait Rehabilitation After Stroke A Scoping Review
Mokienko O., Zisman M. A., Bobrov P. et al., American Journal of Physical Medicine and Rehabilitation 2026 Vol. 105 No. 6 P. 555–563
Brain-computer interfaces (BCIs) represent a promising technology for restoring lower limb motor functions and gait after stroke. The application of BCIs in this field is supported by a limited number of studies. The objective of the review was to systematically and critically evaluate the current evidence on the use of BCIs for lower limb function ...
Added: May 28, 2026
Generalizing the Brady-Yong Algorithm: Efficient Fast Hough Transform for Arbitrary Image Sizes
Kazimirov D., Rybakova E., Vitalii V. Gulevskii et al., IEEE Access 2025 Vol. 13 P. 20101–20132
The Hough (discrete Radon) transform (HT/DRT) is a digital image processing tool that has become indispensable in many application areas, ranging from general image processing to neural networks and X-ray computed tomography. The utilization of the HT in applied problems demands its computational efficiency and increased accuracy. The de facto standard algorithm for the fast ...
Added: May 28, 2026
Universal Comparison Methodology for Hough Transform Approaches
Kazimirov D., Vitalii Gulevskii, Kroshnin A. et al., Mathematics 2026 Article 1136
The Hough transform (HT) is widely used in computer vision, tomography, and neural networks. Numerous algorithms for HT computation have been proposed, making their systematic comparison essential. However, existing comparative methodologies are either non-universal and limited to certain HT formulations, or task-oriented, relying on application-specific criteria that do not fully capture algorithmic properties. This paper ...
Added: May 28, 2026
ИНФОРМАЦИОННЫЕ ТЕХНОЛОГИИ И ТЕХНИЧЕСКИЕ СРЕДСТВА УПРАВЛЕНИЯ (ICCT-2024)
М.: Институт проблем управления им. В.А. Трапезникова РАН, 2024.
В сборник вошли материалы VIII Международной научной конференции «Информационные технологии и технические средства управления» (ICCT-2024). На конференции были рассмотрены вопросы, касающиеся перспектив развития научного приборостроения в телекоммуникационных и управляющих системах, биомедицинской информатики, аппаратного и программного обеспечения информационнокоммуникационных систем, надежности, диагностики и неразрушающего контроля, систем управления и автоматизации, цифровых экосистем, управления производством и логистикой, методов математического ...
Added: May 27, 2026
Non-linear in-band interference cancellation on base of conjugate gradients method
Degtyarev A., Bakhurin S., Yudin N., DSPA 2026 P. 1–6
This paper investigates one possible solution to the problem of self-interference cancellation (SIC) arising in the design of in-band full-duplex (IBFD) communication systems. Self-interference cancellation is performed in the digital domain using multilayer nonlinear models adapted via gradient-based optimization. The presence of local minima and saddle points during the adaptation of multilayer models limits the ...
Added: May 26, 2026
28th European Conference on Artificial Intelligence, 25-30 October 2025, Bologna, Italy – Including 14th Conference on Prestigious Applications of Intelligent Systems (PAIS 2025)
IOS Press, 2025.
Added: May 26, 2026
Comparative Study of Training Methods and Architectures of Echo State Networks
Androsov I., Proceedings of the Institute for System Programming of the RAS 2026 Vol. 38 No. 3 P. 87–114
This paper examines echo state networks (ESNs), one of the most prevalent approaches to implementing reservoir computing. An ESN consists of a recurrent neural network with fixed (untrained) weights and a readout layer that is typically linear and trainable. This approach enables the creation of energyefficient and computationally efficient neural networks capable of real-time learning. However, since ...
Added: May 26, 2026
Рефакторинг исходного кода на основе LLM и расширения UML
Караваева Е. А., Кулигин Л. А., Rezunik L. et al., Труды Института системного программирования РАН 2026 Т. 38 № 3 С. 67–94
В статье представлен метод рефакторинга исходного кода на основе интеграции большой языковой модели (LLM) и расширенной UML-модели программного кода. Предложенный подход позволяет выявлять проблемные участки кода с использованием функций тревожности и структурных метрик классов, а затем выполнять автоматизированный рефакторинг. Ключевой особенностью метода является использование LLM для генерации формальных спецификаций на языке OCL (Object Constraint Language), ...
Added: May 24, 2026
Coping with AI errors with provable guarantees
Tyukin I., Tyukina T., van Helden D. P. et al., Information Sciences 2024 Vol. 678 Article 120856
AI errors pose a significant challenge, hindering real-world applications. This work introduces a novel approach to cope with AI errors using weakly supervised error correctors that guarantee a specific level of error reduction. Our correctors have low computational cost and can be used to decide whether to abstain from making an unsafe classification. We provide ...
Added: May 23, 2026
Overcoming the Curse of Dimensionality with Synolitic AI
Zaikin A., Sviridov I., Sosedka A. et al., Technologies 2026 Vol. 14 No. 2 Article 84
High-dimensional tabular data are common in biomedical and clinical research, yet conventional machine learning methods often struggle in such settings due to data scarcity, feature redundancy, and limited generalization. In this study, we systematically evaluate Synolitic Graph Neural Networks (SGNNs), a framework that transforms high-dimensional samples into sample-specific graphs by training ensembles of low-dimensional pairwise ...
Added: May 23, 2026
TEncDM: Understanding the Properties of the Diffusion Model in the Space of Language Model Encodings
Shabalin A., Meshchaninov V., Chimbulatov E. et al., , in: Proceedings of the 39th Annual AAAI Conference on Artificial IntelligenceVol. 39. Issue 23.: Washington, United States of America: AAAI Press, 2025. Ch. 110 P. 25110–25118.
This paper presents the Text Encoding Diffusion Model (TEncDM), a novel approach to diffusion modeling that operates in the space of pre-trained language model encodings. In contrast to traditionally used embeddings, encodings integrate contextual information. In our approach, we also employ a transformer-based decoder, specifically designed to incorporate context in the token prediction process. We ...
Added: December 18, 2025
Искусственный интеллект как симулякр смысла
Малинов С. А., Галактика медиа: журнал медиа исследований 2025 Т. 7 № 4 С. 154–173
In recent years, artificial intelligence (AI) has been actively integrated into everyday human life. Its popularity continues to grow steadily, and companies increasingly employ AI to optimize and accelerate workflows. Ordinary users leverage large language models (LLMs) and multimodal AI systems to perform a wide range of tasks, including generating texts, images, and videos; planning ...
Added: December 7, 2025
Оценивание студенческих работ в рамках обучения академическому письму на английском языке в контексте развития инструментов искусственного интеллекта
Bakulev A., В кн.: Профессионализм учителя иностранных языков и его реализация. Сборник статей по материалам научно-методического симпозиума с международным участием «Лемпертовские чтения – XXVII» 15-17 мая 2025 года.: Пятигорск: Издательство Пятигорского государственного университета, 2025. С. 270–279.
The paper focuses on assessing students’ written papers in the discipline “Academic Writing in English in the context of AI tools’ capabilities. AI tools, specifically large language models (LLMs) appear to be able to tackle and solve a wide range of educational and research tasks. Foreign language teaching is no exception: AI tools are utilized ...
Added: June 5, 2025
Pixel-Level BPE for Auto-Regressive Image Generation
Razzhigaev A., Voronov A., Kaznacheev A. et al., , in: Proceedings of the First Workshop on Performance and Interpretability Evaluations of Multimodal, Multipurpose, Massive-Scale Models (MMMPIE 2022).: International Conference on Computational Linguistics, 2022. P. 26–30.
Pixel-level autoregression with Transformer models (Image GPT or iGPT) is one of the recent approaches to image generation that has not received massive attention and elaboration due to quadratic complexity of attention as it imposes huge memory requirements and thus restricts the resolution of the generated images. In this paper, we propose to tackle this ...
Added: March 7, 2024
РАЗРАБОТКА СИСТЕМЫ ГЕНЕРАЦИИ ПОВСЕДНЕВНЫХ ДИАЛОГОВ НА РУССКОМ ЯЗЫКЕ: ПИЛОТНОЕ ИССЛЕДОВАНИЕ
Кругликова В. Г., В кн.: Анализ речи: теоретические и прикладные аспекты: сборник научных статей.: [б.и.], 2023.
The article presents a comparative analysis of various language models used to generate texts and evaluates their effectiveness for the task of generating conversational speech. There are such models as GPT-3, BERT, LSTM involved in the comparative analysis. This study is part of a project of developing a system for generating dialogues in Russian. The ...
Added: December 10, 2023
Proceedings of the First Workshop on Performance and Interpretability Evaluations of Multimodal, Multipurpose, Massive-Scale Models (MMMPIE 2022)
Razzhigaev A., Voronov A., Kaznacheev A. et al., International Conference on Computational Linguistics, 2022.
Pixel-level autoregression with Transformer models (Image GPT or iGPT) is one of the recent approaches to image generation that has not received massive attention and elaboration due to quadratic complexity of attention as it imposes huge memory requirements and thus restricts the resolution of the generated images. In this paper, we propose to tackle this ...
Added: June 7, 2023
Sounds Wilde. Phonetically extended embeddings for author-stylized poetry generation
Tikhonov A., Yamshchikov I. P., , in: Proceedings of the Fifteenth Workshop on Computational Research in Phonetics, Phonology, and Morphology.: Association for Computational Linguistics, 2018. P. 117–124.
This paper addresses author-stylized text generation. Using a version of a language model with extended phonetic and semantic embeddings for poetry generation we show that phonetics has comparable contribution to the overall model performance as the information on the target author. Phonetic information is shown to be important for English and Russian language. Humans tend ...
Added: April 7, 2021
Текстема как языковая единица и сверхтекстовый конструкт
Sosnin A., В кн.: Теория и практика лингвистического описания разговорной речи: Сборник материалов Всероссийской научной конференции "Скребневские чтения" 26 октября 2016 г.: Н. Новгород: НГЛУ им. Н.А. Добролюбова, 2016. С. 170–180.
The article examines text as the primary givenness for humanitarian and philological thought and raises the possibility of instituting the textual paradigm and the texteme as a language unit and textual type; it stresses that supertextual structures significantly influence how verbal syntagms are constructed. The article also suggests that the syntax of a text is ...
Added: January 28, 2017
  • About
  • About
  • Key Figures & Facts
  • Sustainability at HSE University
  • Faculties & Departments
  • International Partnerships
  • Faculty & Staff
  • HSE Buildings
  • HSE University for Persons with Disabilities
  • Public Enquiries
  • Studies
  • Admissions
  • Programme Catalogue
  • Undergraduate
  • Graduate
  • Exchange Programmes
  • Summer University
  • Summer Schools
  • Semester in Moscow
  • Business Internship
  • Research
  • International Laboratories
  • Research Centres
  • Research Projects
  • Monitoring Studies
  • Conferences & Seminars
  • Academic Jobs
  • Yasin (April) International Academic Conference on Economic and Social Development
  • Media & Resources
  • Publications by staff
  • HSE Journals
  • Publishing House
  • iq.hse.ru: commentary by HSE experts
  • Library
  • Economic & Social Data Archive
  • Video
  • HSE Repository of Socio-Economic Information
  • HSE1993–2026
  • Contacts
  • Copyright
  • Privacy Policy
  • Site Map
Edit