Improving Distributional Semantic Models Using Anaphora Resolution during Linguistic Preprocessing

P. 288–300.

Kutuzov A. B., Козлова О. С.

In natural language processing, distributional semantic models are known as an efficient data driven approach to word and text representation, which allows computing meaning directly from large text corpora into word embeddings in a vector space. This paper addresses the role of linguistic preprocessing in enhancing performance of distributional models, and particularly studies pronominal anaphora resolution as a way to exploit more co-occurrence data without directly increasing the size of the training corpus.
We replace three different types of anaphoric pronouns with their antecedents in the training corpus and evaluate the extent to which this affects the performance of the resulting models in lexical similarity tasks. CBOW and SkipGram distributed models trained on Russian National Corpus are in the focus of our research, although the results are potentially applicable to other distributional semantic frameworks and languages as well. The trained models are evaluated against RUSSE '15 and SimLex-999 gold standard data sets. As a result, we find that models trained on corpora with pronominal anaphora resolved perform significantly better than their counterparts trained on baseline corpora.

Keywords: natural language processing, автоматическая обработка естественного языка, искусственные нейронные сети, анафора, anaphora resolution, distributional semantics, семантическая близость, semantic similarity of words, дистрибутивная семантика, word2vec, neural embeddings, vector space model, word2vec, word embeddings, разрешение анафоры, векторные репрезентации лексики

In book

Компьютерная лингвистика и интеллектуальные технологии: По материалам ежегодной международной конференции «Диалог» (Москва,1–4 июля 2016 г.)

Вып. 15. , М.: Изд-во РГГУ, 2016.