АВТОМАТИЧЕСКАЯ КЛАССИФИКАЦИЯ ТЕКСТОВ С ИСПОЛЬЗОВАНИЕМ СЕМАНТИКО-СИНТАКСИЧЕСКИХ СВЯЗЕЙ СЛОВ

Вестник компьютерных и информационных технологий. 2018. № 12(174). С. 28–35.

Lebedev I., Спивак А. И., Лапшин С. В.

Abstract. We present the method for improving the quality metrics of text classification. The result achieved by using of additional semantico-syntactic features for text classifier. These features calculated from a semantico-syntactic representation of text. In our research, we used Stanford CoreNLP parser and its “Universal++Dependencies” representation of parse tree. It allowed us to handle some dependencies between words without additions preprocessing of parse tree and get a more complete set of semantico-syntactic features. In comparison with statistical features, such as TF–IDF (Term Frequency – Inverse Document Frequency) for words or n-grams, our features allows to build more “meaningful” numerical model of texts. At the same time, semantico-syntactic features can be used both for the training of a separate classifier, and are added to statistical features and used in training together.We performed an experiment on English texts from arXiv.org. We have taken the titles and abstracts of 4500 papers from three lexically close subject areas without intersection in subjects and used them for training and evaluation of two classifiers to check our idea. The first classifier trained on statistical features. The second trained on both statistical and semantico-syntactic features. Both of them used support vector machine method and tuned separately for maximum accuracy using cross-validation. The experiment showed a decrease of classification error count by 12.15 % compared with the classifier that trained on the statistical features.

Keywords: семантический анализ, classification, тематическая классификация текста, semantic analysis