?
Transformer-based Multi-Party Conversation Generation using Dialogue Discourse Acts Planning
P. 519–529.
Alexander Chernyavskiy, Ilvovsky D.
Recent transformer-based approaches to multi-party conversation generation may produce syntactically coherent but discursively inconsistent dialogues in some cases. To address this issue, we propose an approach to integrate a dialogue act planning stage into the end-to-end transformer-based generation pipeline. This approach consists of a transformer fine-tuning procedure based on linearized dialogue representations that include special discourse tokens. The obtained results demonstrate that incorporating discourse tokens into training sequences is sufficient to significantly improve dialogue consistency and overall generation quality. The suggested approach performs well, including for automatically annotated data. Apart from that, it is observed that increasing the weight of the discourse planning task in the loss function accelerates learning convergence.
Publication based on the results of:
In book
Association for Computational Linguistics, 2023.
Yusupov V., Sukhorukov N., Frolov E., User Modelling and User-Adapted Interaction 2026 Vol. 36 Article 2
Graph-based recommender systems have emerged as a powerful paradigm for personalized recommendations. However, their reliance on full model retraining to incorporate new users or new interactions creates scalability barriers. The task becomes infeasible in real-life recommender systems due to excessive time and resource costs involved. To address this limitation, we propose a fast and efficient ...
Added: March 15, 2026
Anna Volodkevich, Danil Gusak, Klenitskiy A. et al., User Modelling and User-Adapted Interaction 2025 No. 35 Article 13
The goal of modern sequential recommender systems is often formulated in terms of next-item prediction. In this paper, we explore the applicability of transformer-based generative models for the Top-K sequential recommendation task, where the goal is to predict items that a user is likely to interact with in the “near future.” This goal aligns with ...
Added: January 26, 2026
Sherman K., Ignatov D. I., Tatiana I. Shishkovskaya et al., , in: Analysis of Images, Social Networks and Texts, 12th International Conference, AIST 2024, Bishkek, Kyrgyzstan, October 17–19, 2024, Revised Selected PapersVol. 15419.: Springer, 2024. P. 94–108.
More than 3% of people worldwide experience depression. This diagnosis is established through interviews and clinical observations, which is a time- and money-demanding process. Additionally, there are a variety of symptoms associated with depression that are difficult to capture due to the limited capabilities of a human being. Many studies propose methods of automatic mental ...
Added: January 23, 2026
Razzhigaev A., Kurkin M., Goncharova E. et al., , in: Proceedings of the 2nd GenBench Workshop on Generalisation (Benchmarking) in NLP.: Association for Computational Linguistics, 2024. P. 183–195.
We introduce OmniDialog — the first trimodal comprehensive benchmark grounded in a knowledge graph (Wikidata) to evaluate the generalization of Large Multimodal Models (LMMs) across three modalities. Our benchmark consists of more than 4,000 dialogues, each averaging 10 turns, all annotated and cross-validated by human experts. The dialogues in our dataset are designed to prevent ...
Added: February 21, 2025
Razzhigaev A., Mikhalchuk M., Goncharova E. et al., , in: Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) 2024Vol. 1: Long Papers.: Bangkok: Association for Computational Linguistics, 2024. P. 5376–5384.
This paper reveals a novel linear characteristic exclusive to transformer decoders, including models like GPT, LLaMA, OPT, BLOOM and others. We analyze embedding transformations between sequential layers, uncovering an almost perfect linear relationship (Procrustes similarity score of 0.99). However, linearity decreases when the residual component is removed, due to a consistently low transformer layer output ...
Added: February 17, 2025
Razzhigaev A., Mikhalchuk M., Goncharova E. et al., , in: Findings of the Association for Computational Linguistics: EACL 2024.: Association for Computational Linguistics, 2024. P. 868–874.
Added: February 17, 2025
Gorshkov S., Ignatov D. I., Chernysheva A. et al., IEEE Access 2025 Vol. 13 P. 962–979
Identifying potentially high-performing students is crucial for universities aiming to enhance educational outcomes, for companies seeking to recruit top talents early, and for advertising platforms looking to optimize targeted marketing. This paper introduces an algorithm designed to identify students with exceptional academic performance by analyzing their subscriptions to communities on the social network VKontakte. The ...
Added: January 3, 2025
Lyutkin D. A., D. V. Pozdnyakov, Soloviev A. A. et al., Automation and Remote Control, США 2024 Vol. 85 No. 3 P. 297–308
The need for skilled medical support is growing in the era of digital healthcare. This research presents an innovative strategy, utilizing the RuBERT model, for categorizing user inquiries in the field of medical consultation with a focus on expert specialization. By harnessing the capabilities of transformers, we fine-tuned the pretrained RuBERT model on a varied ...
Added: September 26, 2024
Микулинский А. Д., , in: Синергия языков и культур 2022: междисциплинарные исследования.: St. Petersburg: -, 2023. P. 335–351.
The paper is devoted to the issue of the local structure modeling of the eSports commentary spoken genre on an example of the Dota 2 computer discipline. ESports commentary is a spontaneous and creative speech aimed at describing of what is happening on the computer-gaming field. The main factors that force us to study it ...
Added: May 12, 2024
Chernyavskiy A., Ostyakova L., Ilvovsky D., , in: Proceedings of the 5th Workshop on Computational Approaches to Discourse (CODI 2024).: Association for Computational Linguistics, 2024. P. 149–160.
Recent language models have significantly boosted conversational AI by enabling fast and cost-effective response generation in dialogue systems. However, dialogue systems based on neural generative approaches often lack truthfulness, reliability, and the ability to analyze the dialogue flow needed for smooth and consistent conversations with users. To address these issues, we introduce GroundHog, a modified ...
Added: May 9, 2024
Chernyavskiy A., Ilvovsky D., Nakov P., , in: Proceedings of the 18th Conference of the European Chapter of the Association for Computational Linguistics: (Volume 1: Long Papers).: Association for Computational Linguistics, 2024. P. 1452–1462.
Added: May 9, 2024
Lyutkin D., Soloviev A., Zhukov D. et al., Working papers by Cornell University. Series math "arxiv.org" 2023 P. 1–16
The need for skilled medical support is growing in the era of digital healthcare. This research presents an innovative strategy, utilizing the RuBERT model, for categorizing user inquiries in the field of medical consultation with a focus on expert specialization. By harnessing the capabilities of transformers, we fine-tuned the pre-trained RuBERT model on a varied ...
Added: November 27, 2023
Galitsky B., Ilvovsky D., Goncharova E., , in: Компьютерная лингвистика и интеллектуальные технологии: По материалам ежегодной международной конференции «Диалог». Вып. 22.Вып. 22.: [б.и.], 2023.
We extend the concept of a discourse tree (DT) in the discourse representation of text towards data of various forms and natures. The communicative DT to include speech act theory, extended DT to ascend to the level of multiple documents, entity DT to track how discourse covers various entities were defined previously in computational linguistics, we now proceed ...
Added: November 10, 2023
PaperPersiChat: Scientific Paper Discussion Chatbot using Transformers and Discourse Flow Management
Chernyavskiy A., Bregeda M., Nikiforova M., , in: Proceedings of the 24th Meeting of the Special Interest Group on Discourse and Dialogue.: Association for Computational Linguistics, 2023. P. 584–587.
The rate of scientific publications is increasing exponentially, necessitating a significant investment of time in order to read and comprehend the most important articles. While ancillary services exist to facilitate this process, they are typically closed-model and paid services or have limited capabilities. In this paper, we present PaperPersiChat, an open chatbot-system designed for the discussion ...
Added: October 6, 2023
Arutyunov G.A., Avdoshin S. M., Proceedings of the Institute for System Programming of the RAS 2022 Vol. 34 No. 4 P. 79–88
IT industry has been thriving over the past decades. Numerous new programming languages have emerged, new architectural patterns and software development techniques. Tools involved in the process ought to evolve as well. One of the key principles of new generation of instruments for software development would be the ability of the tools to learn using ...
Added: December 26, 2022
Chernyavskiy A., Ilvovsky D., Galitsky B., , in: Proceedings of the Student Research Workshop Associated with RANLP 2021.: INCOMA Ltd, 2021. P. 36–43.
Recent transformer-based approaches to NLG like GPT-2 can generate syntactically coherent original texts. However, these generated texts have serious flaws: global discourse incoherence and meaninglessness of sentences in terms of entity values. We address both of these flaws: they are independent but can be combined to generate original texts that will be both consistent and ...
Added: May 29, 2022
Alexander Chernyavskiy, , in: WSDM 2022 - Proceedings of the 15th ACM International Conference on Web Search and Data Mining.: Association for Computing Machinery (ACM), 2022. P. 1543–1544.
Recent Transformer-based approaches to NLG like GPT-2 can generate syntactically coherent original texts. However, these generated texts have serious flaws. One of them is a global discourse incoherence. We present an approach to estimate the quality of discourse structure. Empirical results confirm that the discourse structure of currently generated texts is inaccurate. We propose the ...
Added: May 29, 2022