Об энтропийных критериях отбора признаков в задачах анализа данных

Информационные технологии и вычислительные системы. 2018. № 2. С. 60–69.

Dubnov Y. A.

The paper considers the problem of reducing the dimension of the feature space for describing objects
in data analysis problems using the example of binary classification. The article provides a detailed
overview of existing approaches to solving this problem and proposes several modifications. In which
the dimensionality reduction is considered as the problem of extracting the most relevant information
from the characteristic description of objects and is solved in terms of the Shanon's entropy. To identify
the most significant features information criteria such as crossentropy, mutual information and Kullback-
Leibler divergence are used.

Keywords: entropy, энтропия, классификация, classification, dimensionality reduction, feature selection, понижение размерности, отбор признаков