Everyday Conversations: a Comparative Study of Expert Transcriptions and ASR Outputs at a Lexical Level

Lecture Notes in Computer Science. 2023. Vol. 14338. P. 43–56.

Sherstinova T., Михайловский Н. Э., Kolobov R.

The study examines the outcomes of automatic speech recognition (ASR) applied to field recordings of daily Russian speech. Everyday conversations, captured in real-life communicative scenarios, pose quite a complex subject for ASR. This is due to several factors: they can contain speech from a multitude of speakers, the loudness of the conversation partners’ speech signals fluctuates, there’s a substantial volume of overlapping speech from two or more speakers, and significant noise interferences can occur periodically. The presented research compares transcripts of these recordings produced by two recognition systems: the NTR Acoustic Model and OpenAI’s Whisper. These transcripts are then contrasted with expert transcription of the same recordings. The comparison of three frequency lists (the expert transcription, the acoustic model, and Whisper) reveals that each model has its unique characteristics at the lexical level. At the same time, both models perform worse in recognizing the following groups of words typical for spontaneous unprepared dialogues: discursive words, pragmatic markers, backchannel responses, interjections, conversational reduced word forms, and hesitations. These findings aim to foster improvements in ASR systems designed to transcribe conversational speech, such as work meetings and daily life dialogues.

Keywords: vocabulary, hesitations, whisper, Interjections, pragmatic markers, Word lists, ASR Systems, Acoustic Model, Field Recordings, Everyday Conversations, Dialogues, Russian Language, Backchanneling, Discursive Words, Conversational Word Forms

Publication based on the results of: Текст как Big Data: моделирование конвергентных процессов в языке и речи цифровыми методами (2023)