BERT-like Models for Slavic Morpheme Segmentation

P. 6795–6815.

Morozov D., Astapenka L., Glazkova A., Garipov T., Lyashevskaya O.

Automatic morpheme segmentation algorithms are applicable in various tasks, such as building tokenizers and language education. For Slavic languages, the development of such algorithms is complicated by the rich derivational capabilities of these languages. Previous research has shown that, on average, these algorithms have already reached expert-level quality. However, a key unresolved issue is the significant decline in performance when segmenting words containing roots not present in the training data. This problem can be partially addressed by using pre-trained language models to better account for word semantics. In this work, we explored the possibility of fine-tuning BERT-like models for morpheme segmentation using data from Belarusian, Czech, and Russian. We found that for Czech and Russian, our models outperform all previously proposed approaches, achieving word-level accuracy of 92.5-95.1%. For Belarusian, this task was addressed for the first time.

Keywords: морфологический анализ, словообразование, word formation, славянские языки, Slavic languages, morphological analysis, morpheme segmentation for Russian, automatic morpheme segmentation, word segmentation, morpheme segmentation for Belarusian, morpheme segmentation for Czech, морфемная сегментация, морфемная сегментация русского языка, морфемная сегментация белорусского языка, морфемная сегментация чешского языка

In book

Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)

Vol. 1: Long papers. , Association for Computational Linguistics, 2025.