• A
  • A
  • A
  • АБВ
  • АБВ
  • АБВ
  • A
  • A
  • A
  • A
  • A
Обычная версия сайта
  • RU
  • EN
  • HSE University
  • Publications
  • Articles
  • Enhancing Emotion Recognition in Speech Based on Self-Supervised Learning: Cross-Attention Fusion of Acoustic and Semantic Features
  • RU
  • EN
Расширенный поиск
Высшая школа экономики
Национальный исследовательский университет
Priority areas
  • business informatics
  • economics
  • engineering science
  • humanitarian
  • IT and mathematics
  • law
  • management
  • mathematics
  • sociology
  • state and public administration
by year
  • 2027
  • 2026
  • 2025
  • 2024
  • 2023
  • 2022
  • 2021
  • 2020
  • 2019
  • 2018
  • 2017
  • 2016
  • 2015
  • 2014
  • 2013
  • 2012
  • 2011
  • 2010
  • 2009
  • 2008
  • 2007
  • 2006
  • 2005
  • 2004
  • 2003
  • 2002
  • 2001
  • 2000
  • 1999
  • 1998
  • 1997
  • 1996
  • 1995
  • 1994
  • 1993
  • 1992
  • 1991
  • 1990
  • 1989
  • 1988
  • 1987
  • 1986
  • 1985
  • 1984
  • 1983
  • 1982
  • 1981
  • 1980
  • 1979
  • 1978
  • 1977
  • 1976
  • 1975
  • 1974
  • 1973
  • 1972
  • 1971
  • 1970
  • 1969
  • 1968
  • 1967
  • 1966
  • 1965
  • 1964
  • 1963
  • 1958
  • More
Subject
News
November 17, 2026
Physicists Show How a Solvent Shapes the Response of Drug Molecules
Many drug molecules can switch between different three-dimensional shapes. Using a mathematical model of ibuprofen dissolved in supercritical CO₂, researchers from HSE University and the Institute of Solution Chemistry of the Russian Academy of Sciences have shown that the solvent can make these molecules appear to respond unusually slowly. The molecules themselves continue to change shape rapidly, but their surroundings rearrange much more slowly, and this lag shows up in the measurements. The study has been published in The Journal of Chemical Physics. 
November 10, 2026
Physicists Train AI to Optimise Particle Beams in Accelerators
Researchers from the AI and Digital Science Institute at the HSE Faculty of Computer Science and the Budker Institute of Nuclear Physics at the Siberian Branch of the Russian Academy of Sciences have developed RLABC (Reinforcement Learning for Accelerator Beamline Control), a framework that uses reinforcement learning to identify optimal settings for a particle accelerator. In a computer simulation, RLABC optimised the accelerator's parameters to achieve a particle transmission rate of 70%. The study has been published in Computer Physics Communications.
November 9, 2026
'Organ-on-a-Chip Technology Could Eliminate the Need for Animal Testing'
The Laboratory of Molecular Physiology at the HSE Faculty of Biology and Biotechnology investigates how molecular regulators known as microRNAs control gene expression and contribute to the development of diseases, including cancer, viral and inflammatory conditions, and pregnancy complications. At HSE University, researchers develop organ-on-a-chip models: miniature devices containing living cells that mimic the functions of human organs, such as the placenta and intestines. These models help researchers understand how diseases develop and how drugs affect them. HSE News Service spoke with Evgeny Knyazev, Head of the Laboratory.

 

Have you spotted a typo?
Highlight it, click Ctrl+Enter and send us a message. Thank you for your help!

Publications
  • Books
  • Articles
  • Chapters of books
  • Working papers
  • Report a publication
  • Research at HSE

?

Enhancing Emotion Recognition in Speech Based on Self-Supervised Learning: Cross-Attention Fusion of Acoustic and Semantic Features

IEEE Access. 2026. Vol. 13. P. 56283–56295.
Deeb B., Andrey V. Savchenko, Makarov I.

Speech Emotion Recognition has gained considerable attention in speech processing and machine learning due to its potential applications in human-computer interaction, mental health monitoring, and customer service. However, state-of-the-art models for speech emotion recognition use many parameters, which leads to computational complexity. In this paper, we introduce a novel deep-learning model to enhance the accuracy of emotional content detection in speech signals while maintaining a lightweight architecture compared to state-of-the-art models. The proposed model incorporates a feature encoder that significantly improves the emotional representation of acoustic features and a cross-attention mechanism to fuse acoustic features, such as Spectrograms, with semantic features extracted from the pre-trained self-supervised learning framework, enriching the emotional representation of speech. An extensive experimental study demonstrates that the proposed model achieves a weighted accuracy of 74.6% on the IEMOCAP dataset, competitive with the state-of-the-art baselines. In addition, our proposed model achieves a latency of 24 milliseconds on moderate devices while containing up to three times fewer parameters.

Research target: Computer Science
Language: English
Full text
DOI
Text on another site
Keywords: распознавание эмоцийspeech emotion recognitioncross-attention mechanismмеханизм внимания feature fusionобъединение признаков
Similar publications
Active learning of collinear magnetic Moment Tensor Potentials using the spin-MLIP package from soft-constrained spin-polarized DFT calculations: a case study of Fe-Pd
Burov A., Котыхов А. С., Aksyonov D. et al., Computer Physics Communications 2027 Vol. 330 Article 110458
Explicit incorporation of magnetic degrees of freedom in machine-learning interatomic potentials (magnetic MLIPs) plays a crucial role in the correct description of magnetic materials and their properties. An important ingredient for fitting of magnetic MLIPs is spin-polarized density functional theory (DFT) calculations with non-equilibrium magnetic moments, i.e. DFT calculations with constraints on magnetic moments. In this study, we ...
Added: November 17, 2026
Toward reliable false discovery rate control in classification problems under distribution shift
Kertesz-Farkas A., Andrey B., Plos One 2026 Vol. 21 No. 11 P. 1–20
Background Domain shifts or batch effects can significantly degrade the performance of machine learning-based classifiers in software production without any warning. This can particularly be problematic in medical applications which involve human lives. The shift changes the shape of the prediction score distribution of the test samples compared to that of the training samples; hence, decision ...
Added: November 17, 2026
Траектории ставок как сигнал сговора: LSTM-классификация аукционов государственных закупок
Ефимов К. Д., Журнал Новой экономической ассоциации 2026 № 72 С. 105–114
This study proposes a method for automatic detection of collusive behavior in electronic public procurement auctions based on analysis of the full bid trajectory. The empirical dataset comprises 508 auction histories from 2016–2020 collected from the RTS tender e-procurement platform; the positive class is labeled using case materials of the Federal Antimonopoly Service (FAS Russia), ...
Added: November 13, 2026
Model-Dependent Prompt-Component Ablation for Medical LLM Decision Support: Application to ENT Biologics Screening
Sergey N. Koltcov, Budkovaia M., Surkov A. et al., IEEE Journal of Biomedical and Health Informatics 2026 P. 1–10
Large language models (LLMs) show promise for clinical decision support, but privacy constraints and unreliable free-text decisions limit deployment. We present Medotor, a locally deployed system for screening patients with chronic rhinosinusitis with nasal polyps (CRSwNP) for biologic therapy, and evaluate prompt design on 163 retrospective cases (132 eligible and 31 ineligible). Five models were ...
Added: November 13, 2026
Impact of 5G NR-V2X Message Losses on Connected Vehicle Warning Functions
Физулин А. В., Stepanyants V., Amerikanov A. et al., FUTURE TRANSPORTATION 2026 Vol. 6 No. 6 Article 264
Cooperative warning functions depend on whether V2X information becomes available in time to influence vehicle control. This article presents an event-based co-simulation framework coupling SUMO, ns-3 (ms-van3t), 5G-LENA NR sidelink and application records, with Sionna RT enabled for the intersection scenario. In 140 emergency-vehicle-warning runs, reducing mean packet reception ratio (PRR) from 0.9412 to 0.0675 ...
Added: November 12, 2026
Structural model of decision support system for sustainable development of oil and gas companies
Stoianova A. D., Trofimets V. Y., Matrokhina K. V., International Journal of Engineering, Transactions A: Basics 2025 Vol. 38 No. 4 P. 701–709
The article considers the issue of managerial decision-making in an oil and gas company, which is related to the class of multi-criteria choice problems. To improve the validity of decisions in the context of companies’ sustainable development it is proposed to implement a Decision Support System (DSS). A brief review of the existing methods of ...
Added: November 10, 2026
EdFitter: a hybrid course recommendation system using large language models
Sergei Koltcov, Ryumin D., Alexandr Axyonov et al., PeerJ Computer Science 2026 Article 12:e4141
Educational institutions face increasing pressure to adapt their curricula to the rapidly evolving demands of the job market, and recommender systems can be important tools to guide students in selecting courses that align with their career goals. We present EdFitter, a hybrid course recommendation system that combines pretrained text embeddings for retrieval with large language ...
Added: November 10, 2026
Assessing density equilibration in confined Liquid–Liquid interfaces
Guskov T., Gerke K., Kondratyuk N. et al., Molecular Simulation 2026 Vol. 52 No. 7 P. 546 – 555
Equilibration is often assumed rather than tested in molecular dynamics studies of confined liquid–liquid interfaces, even though different physical properties may relax on different time scales. In this work, equilibration is examined using an automated truncation method based on the marginal standard error rule. Interfacial geometry, wetting behaviour, and density-related properties are analysed for liquid–liquid ...
Added: November 10, 2026
Influence of nuclear quantum effects on metastable states of warm dense hydrogen
Lukyanchuk V., Norman G., Kondratyuk N. et al., Scientific Reports 2026 Vol. 16 No. 1 Article 16435
Previously, it has been effectively shown that nuclear quantum effects (NQE) considerably influence the prediction of the warm dense hydrogen phase diagram. Here, we study the influence of NQE on the parameters of the conducting state formation in warm dense hydrogen using path integral molecular dynamics (PIMD) within the framework of the finite temperature density ...
Added: November 10, 2026
Machine Learning and Knowledge Discovery in Databases. Research Track and Applied Data Science Track. European Conference, ECML PKDD 2026, Naples, Italy, September 7–11, 2026, Proceedings
Cham: Springer, 2027.
This volume is Part VIII of the refereed proceedings of the European Conference on Machine Learning and Knowledge Discovery in Databases, ECML PKDD 2026, held in Naples, Italy, September 7–11, 2026. It contains 40 papers from the Research Track and the Applied Data Science Track and is published as volume 16948 of Lecture Notes in ...
Added: November 9, 2026
Artificial Intelligence for Software Engineering (AI4SE): A Review of Capabilities, Evaluation, and Verification Challenges
Avdoshin S.M., Pesotskaya E.Y., Proceedings of the Institute for System Programming of the RAS 2026 Vol. 38 No. 6, part 2 P. 133–146
Artificial intelligence has significantly reshaped contemporary software engineering practice, changing how software systems are designed, implemented, tested, and maintained. Recent advances in large language models and AI-assisted development tools have expanded the role of AI from supporting isolated tasks to participating in multiple stages of the software lifecycle. In many development settings, software is now created from high-level specifications ...
Added: November 8, 2026
MFL-ScrollNet: Recurrent Sliding-Window Detection for Panoramic Pipeline Magnetic Flux Leakage Images
Aleksei Samarin, Belous F., Торопов А. Г. et al., Technologies 2026 Vol. 14 No. 11 P. 1–21
Abstract Automated interpretation of magnetic flux leakage (MFL) data is central to condition assessment of long-distance oil and gas pipelines. Yet, full-resolution panoramas are difficult to process without either suppressing small anomalies through resizing or fragmenting elongated objects at sliding-window boundaries. We propose MFL-ScrollNet, a recurrent sliding-window detector that formulates panoramic analysis as an ordered sequence ...
Added: November 8, 2026
Proceedings 2026 IEEE International Conference on Joint Cloud Computing JCC 2026
IEEE Computer Society (Conference Publishing Services, CPS), 2026.
Added: November 7, 2026
Detecting the undetectable: benchmarking AI-generated text
Sergey Avdoshin, Elena Pesotskaya, Saleh H. et al., Procedia Computer Science 2026 Vol. 292 P. 1332–1341
The rapid development of generative large language models (LLMs) has intensified the need for reliable methods capable of distinguishing AI-generated text from human writing, especially in academic and professional settings. This paper presents the results of a systematic benchmarking study evaluating the performance and limitations of contemporary LLM-text detectors applied to Russian-language academic content. We introduce ...
Added: November 7, 2026
PrimeKV: Preserving critical KV cache without attention scores for long-context inference
Zhang Z., Ilvovsky D., Neurocomputing 2027 Vol. 710
Efficient management of the Key-Value (KV) cache is essential for deploying large language models (LLMs) in long-context scenarios, where memory requirements often exceed the capacity of resource-constrained hardware. Many existing training-free compression algorithms typically rely on attention-driven heuristics, but exhibit two key limitations. First, they require explicit attention score computation, which is difficult to reconcile ...
Added: November 5, 2026
An Audio-Gated Cascade for Resource-Efficient Bimodal Emotion Recognition
Kandelov D., Savchenko L., , in: Speech and Computer. 28th International Conference, SPECOM 2026, Ohrid, North Macedonia, September 17–18, 2026, Proceedings, Part II. LNCS, volume 16935.: Springer, 2026. Ch. 3 P. 30–45.
Bimodal (audio–visual) emotion recognition systems typ-ically process speech and video in parallel with heavy neural back-bones, incurring a computational cost that makes them unavailable on mobile, embedded and edge platforms. In this paper we propose a two-stage cascade architecture that decouples the two modalities in time. A lightweight binary audio gate, implemented as a compact ...
Added: October 1, 2026
“Reading the Mind in the Eyes Test”: Development of a Russian-Language Version Expanded for Neuroimaging Applications
Zheltyakova M., Korotkov A., Bursov K. et al., Herald of the Russian Academy of Sciences 2026 Vol. 95 No. 6 P. 1048–1057
The Reading the Mind in the Eyes Test (RMET) is a widely used tool for assessing affective mentalization in both clinical and research contexts. Although translated into many languages, no validated Russian-language version has been available. Furthermore, the limited number of stimuli in the original RMET constrains its applicability in neuroimaging studies. This study presents ...
Added: September 13, 2026
Сравнение точности распознавания эмоций другого человека при текстовой онлайн-коммуникации в мессенджере и наблюдении за офлайн-поведением
Сысоева И. А., Sysoeva T., Новые психологические исследования 2026 № 3 С. 95–114
Currently, instant text messaging is widespread and often replaces regular offline communication. At the same time, such an important component of interaction as understanding the emotions of the interlocutor in online text communication may be difficult because of the lack of nonverbal emotional cues (facial expressions, gestures, etc.). Although it is generally believed that emotional ...
Added: June 22, 2026
Метод распознавания сентимента и эмоций в транскрипциях русскоязычной речи с использованием машинного перевода
Dvoynikova A., Кагиров И. А., Карпов А. А., Информатика и автоматизация (Труды СПИИРАН) 2024
В статье рассматривается проблема распознавания сентимента и эмоций пользователей в русскоязычных текстовых транскрипциях речи с использованием словарных методов и машинного перевода. Количество имеющихся информационных ресурсов для анализа сентимента текстовых сообщений на русском языке очень ограничено, что существенно затрудняет применение базовых методов анализа сентимента, а именно, предобработки текстов, векторизации с помощью тональных словарей, традиционных классификаторов. Для ...
Added: April 25, 2026
Аналитический обзор многомодальных корпусов данных для распознавания эмоций
Dvoynikova A., В кн.: Альманах научных работ молодых ученых Университета ИТМО.: Университет ИТМО, 2023.
В статье раскрываются достоинства и недостатки категориальных и пространственных моделей описания эмоций. Пространственные модели позволяют охватить более широкий спектр человеческих эмоций, что позволяет разработать наиболее эффективную систему распознавания эмоций. В работе проводится аналитический обзор существующих многомодальных корпусов данных, которые имеют разметку по валентности и интенсивности эмоций. В заключении выделяется наиболее репрезентативный корпус данных для автоматического ...
Added: April 25, 2026
Подход к автоматическому распознаванию эмоций в транскрипциях речи
Dvoynikova A., Кондратенко К. О., Известия высших учебных заведений. Приборостроение 2023 Т. 66 № 10 С. 818–827
Аннотация. Исследован актуальный в различных областях вопрос распознавания эмоций в транскрипциях речи. Проанализировано влияние методов предобработки (удаление стоп-слов, лемматизация, стемминг) на точность распознавания эмоций в текстовых данных на русском и английском языках. Для проведения экспериментальных исследований использовались орфографические транскрипции диалогов из многомодальных корпусов RAMAS и CMU-MOSEI на русском и английском языке соответственно. Аннотирование этих корпусов ...
Added: April 25, 2026
Автоматическое определение эмоционального состояния участников предметных разговоров по транскрипциям речи
Dvoynikova A., Мамонтов Д. Ю., Карпов А. А., В кн.: Альманах научных работ молодых ученых Университета ИТМОТ. 3.: Университет ИТМО, 2021. С. 63–68.
В работе проводятся экспериментальные исследования по определению уровня эмоциональных проявлений в текстовых транскрипциях базы данных K-EmoCon. Рассматривается влияние сбалансирования классов при обучении классификаторов на точность определения эмоций. В статье устанавливается базовый стандарт результатов по классификации уровня эмоций дикторов в текстовых транскрипциях. ...
Added: April 24, 2026
A Bimodal Approach for Speech Emotion Recognition using Audio and Text
Verkholyak O., Dvoynikova A., Karpov A., Journal of Internet Services and Information Security 2021 No. 1 P. 80–96
This paper presents a novel bimodal speech emotion recognition system based on analysis of acoustic and linguistic information. We propose a novel decision-level fusion strategy that leverages both emotions and sentiments extracted from audio and text transcriptions of extemporaneous speech utterances. We perform experimental study to prove the effectiveness of the proposed methods using emotional ...
Added: April 24, 2026
Метод улучшения обнаружения атак презентации на биометрическую систему распознавания лиц с помощью сверточной сети с механизмом внимания
Pikul A. S., В кн.: Альманах научных работ молодых ученых университета ИТМО. Материалы Пятьдесят третьей (LIII) научной и учебно-методической конференции Том 1.: СПб.: Университет ИТМО, 2024. С. 338–342.
Предложен новый подход для улучшения распознавания атак презентации на биометрическую систему распознавания лиц с помощью сверточной сети с механизмом внимания. Проверена центральная гипотеза, которая заключалась в том, что с помощью механизма внимания возможно улучшить результаты работы исходной сверточной нейронной сети. В ходе экспериментов гипотеза была подтверждена. Наибольший прирост по качеству был достигнут на наборе данных ...
Added: December 13, 2025
  • About
  • About
  • Key Figures & Facts
  • Sustainability at HSE University
  • Faculties & Departments
  • International Partnerships
  • Faculty & Staff
  • HSE Buildings
  • HSE University for Persons with Disabilities
  • Public Enquiries
  • Studies
  • Admissions
  • Programme Catalogue
  • Undergraduate
  • Graduate
  • Exchange Programmes
  • Summer University
  • Summer Schools
  • Semester in Moscow
  • Business Internship
  • Research
  • International Laboratories
  • Research Centres
  • Research Projects
  • Monitoring Studies
  • Conferences & Seminars
  • Academic Jobs
  • Yasin (April) International Academic Conference on Economic and Social Development
  • Media & Resources
  • Publications by staff
  • HSE Journals
  • Publishing House
  • iq.hse.ru: commentary by HSE experts
  • Library
  • Economic & Social Data Archive
  • Video
  • HSE Repository of Socio-Economic Information
  • HSE1993–2026
  • Contacts
  • Copyright
  • Privacy Policy
  • Site Map
Edit