Disambiguation in context in the Russian National Corpus: 20 yeas later

P. 307–318.

Lyashevskaya O., Afanasev I., Stefan Rebrikov, Yana Shishkina, Suleymanova E., Trofimov I., Vlasova N.

An updated annotation of the Main, Media, and some other corpora of the Russian National Corpus (RNC) features the part-of-speech and other morphological information, lemmas, dependency structures, and constituency types. Transformer-based architectures are used to resolve the homonymy in context according to a schema based on the manually disambiguated subcorpus of the Main corpus (morphology and lexicon) and UD-SynTagRus (syntax). The paper discusses the challenges in applying the models to texts of different registers, orthographies, and time periods, on the one hand, and making the new version convenient for users accustomed to the old search practices, on the other. The re-annotated corpus data form the basis for the enhancement of the RNC tools such as word and n-gram frequency lists, collocations, corpus comparison, and Word at a glance.

Keywords: Russian language, автоматическая обработка естественного языка, лемматизация, Russian National Corpus, NLP evaluation, morphological tagging, морфологическая разметка, dependency parsing, lemmatization, синтаксическая разметка корпуса, disambiguation, automated syntactic parsing, historical corpora, автоматическая обработка исторических текстов, Natural Language Processing (NLP)

In book

Компьютерная лингвистика и интеллектуальные технологии: По материалам ежегодной международной конференции «Диалог». Вып. 22.

Вып. 22. , [б.и.], 2023.