Morpheme Segmentation for Russian: Evaluation of Convolutional Neural Network Models

P. 160–166.

Maltina L., Malafeev A.

This paper is aimed at evaluating the performance of existing models of morphemic analysis for Russian based on convolutional neural networks. The models were trained on a relatively small amount of annotated training data (38,368 words). We tuned the hyperparameters to accommodate the harder task setting, which helped improve the accuracy of the model. In addition to testing 15 different configurations on the available test set, a new sample of 800 words containing roots that are missing in the training sample (e.g. neologisms and recent loan words) was manually created and annotated for morphemic structure (the new dataset is made available to the community). The effectiveness of the models was evaluated on this sample, and it turned out that the performance of the CNN models was much worse on this set (an almost 30% drop in word accuracy). We performed a classification of errors made by the best model both on the standard test set and the new one.

Keywords: convolutional neural networks, model evaluation, error analysis, morpheme segmentation for Russian, words with out-of-vocabulary roots, parameter tuning, морфемный анализ слов русского языка, свёрточные нейронные сети

Publication based on the results of: Эффективные методы распознавания мультимедийных данных для задач анализа предпочтений пользователей мобильных устройств (2019)

In book

Analysis of Images, Social Networks and Texts. 8th International Conference, AIST 2019, Kazan, Russia, July 17–19, 2019, Revised Selected Papers. Communications in Computer and Information Science

Vol. 1086. , Springer, 2020.