Рекомендации товаров по набору фотографий на основе нейросетевых методов агрегации векторов признаков изображений

Дёмочкин Кирилл Владиславович; Савченко Андрей Владимирович

Book chapter

Рекомендации товаров по набору фотографий на основе нейросетевых методов агрегации векторов признаков изображений

Дёмочкин К. В., Савченко А. В.

In press

In this paper we focus on the problem of user prediction in visual product recommender systems based on the given set of photos of products purchased by the user previously. We studied neural aggregation methods for image features extracted by the deep neural networks. We propose the novel two-stage algorithm. At first, the image features are learned by fine-tuning the convolutional neural network. At the second stage, we sequentially combine the known learnable pooling techniques (neural aggregation network and context gating) in order to compute a single descriptor for particular user as a weighted average of image features. It is experimentally shown for the Amazon product dataset that F1-measure for our approach is more than 20% higher when compared to conventional averaging of the feature vector.

Language: Russian

Full text (PDF, 524 Kb)

Keywords: recommender system Image recognition Convolutional Neural Network

Publication based on the results of: Разработка и апробация эффективных методов классификации для больших баз мультимедийных данных(2017)

In book

Сборник трудов V Международной конференции и молодёжной школы "Информационные технологии и нанотехнологии" (ИТНТ 2019)

2019.

The Adaptive Approach to Abnormal Situations Recognition Using Images from Condition Monitoring Systems

Savchenko A., Milov V. R. Optical Memory and Neural Networks (Information Optics). 2016. Vol. 25. No. 2. P. 79-87.

Decision support in equipment condition monitoring systems with image processing is analyzed. Long-run accumulation of information about earlier made decisions is used to realize the adaptiveness of the proposed approach. It is shown that unlike conventional classification problems, the recognition of abnormalities uses training samples supplemented with reward estimates of earlier decisions and can be tackled using reinforcement learning algorithms. We consider the basic stages of contextual multi-armed bandit algorithms during which the probabilistic distributions of each state are evaluated to evaluate the current knowledge of the states, and the decision space is explored to increase the decision-making efficiency. We propose a new decision-making method, which uses the probabilistic neural network to classify abnormal situation and the softmax rule to explore the decision space. A modelling experiment in image processing was carried out to show that our approach allows a higher accuracy of abnormality detection than other known methods, especially for small-size initial training samples.

Added: Jul 10, 2016

Image Recognition Using Kullback-Leibler Information Discrimination

Savchenko A. In bk.: SCAKD'11 - Soft Computing applications and Knowledge Discovery. Workshop co-located with the 13th International Conference on Rough Sets,Fuzzy Sets, Data Mining, and Granular Computing (RSFDGrC-2011) and the 4th International Conference on Pattern Recognition and Machine Intelligence (PReMI-2011), June 2011, Moscow, Russia. Vol. 758. M.: Higher School of Economics Publishing House, 2011. P. 102-112.

The problem of automatic image recognition based on the minimum information discrimination principle is formulated and solved. Color histograms comparison in the Kullback–Leibler information metric is proposed. It’s combined with method of directed enumeration alternatives as opposed to complete enumeration of competing hypotheses. Results of an experimental study of the Kullback-Leibler discrimination in the problem of face recognition with a large database are presented. It is shown that the proposed algorithm is characterized by increased accuracy and reliability of image recognition.

Added: Aug 31, 2012

Image retrieval using minimum information discrimination criterion

Savchenko A. In bk.: ACIT - Control, Diagnostics, and Automation (ACIT-CDA 2010). Anaheim: ACTA Pre, 2010. P. 345-350.

The problem of automatic image recognition based on the minimum information discrimination principle is formulated and solved. Discrimination calculation in the Kullback–Leibler information metric based on colour histograms comparison is proposed. It’s combined with a method of directed enumeration of the set of alternatives as opposed to the method of complete enumeration of competing hypotheses. Results of an experimental study of the discrimination in the problem of face images recognition are presented. It is shown that the proposed algorithm is characterized by increased accuracy and reliability of automatic image recognition.

Added: Aug 31, 2012

Communications in Computer and Information Science

Vol. 436: Analysis of Images, Social Networks and Texts. Third International Conference, AIST 2014 Yekaterinburg, Russia, April 10–12, 2014 Revised Selected Papers. Cham: Springer, 2014.

The CCIS series is devoted to the publication of proceedings of computer science conferences. Its aim is to efficiently disseminate original research results in informatics in printed and electronic form. While the focus is on publication of peer-reviewed full papers presenting mature work, inclusion of reviewed short papers reporting on work in progress is welcome, too. Besides globally relevant meetings with internationally representative program committees guaranteeing a strict peer-reviewing and paper selection process, conferences run by societies or of high regional or national relevance are also considered for publication.

Added: Oct 15, 2014

Directed enumeration method in image recognition

Savchenko A. Pattern Recognition. 2012. Vol. 45. No. 8. P. 2952-2961.

The article is devoted to the problem of image recognition in real-time applications with a large database containing hundreds of classes. The directed enumeration method as an alternative to exhaustive search is examined. This method has two advantages. First, it could be applied with measures of similarity which do not satisfy metric properties (chi-square distance, Kullback-Leibler information discrimination, etc). Second, the directed enumeration method increases recognition speed even in the most difficult cases which seem to be very important in practical terms. In these cases many neighbors are located at very similar distances. In this paper we present the results of an experimental study of the directed enumeration method with comparison of color- and gradient-orientation histograms in solving the problem of face recognition with well-known datasets (Essex, FERET). It is shown that the proposed method is characterized by increased computing efficiency of automatic image recognition (3-12 times in comparison with a conventional nearest neighbor classifier).

Added: Jun 9, 2012

Image Recognition with a Large Database Using Method of Directed Enumeration Alternatives Modification

Savchenko A. In bk.: Rough Sets, Fuzzy Sets, Data Mining and Granular Computing: 13th International Conference, RSFDGrC 2011, Moscow, Russia, June 25-27, 2011. Proceedings. Vol. 6743. Berlin; Heidelberg: Springer, 2011. P. 338-341.

A new modification of the method of directed alternatives' enumeration using the Kullback-Leibler discrimination information is proposed for half-tone image recognition.Results of an experimental studyin the problem of face images recognition with a large database are pre-sented. It is shown that the proposed modification is characterized by increased speed of image recognition (5-10 times vs exhaustive search).

Added: Aug 31, 2012

Group-Level Emotion Recognition Using Transfer Learning From Face Identification

Alexandr Rassadin, Alexey Gruzdev, Andrey Savchenko. Computer Science > Computer Vision and Pattern Recognition. cs.CV. Cornell University Library, 2017. No. 1709.01688.

In this paper we describe our algorithmic approach, which was used for submissions in the fifth Emotion Recognition in the Wild (EmotiW 2017) group-level emotion recognition sub-challenge. We extracted feature vectors of detected faces using the Convolutional Neural Network trained for face identification task, rather than traditional pre-training on emotion recognition problems. In the final pipeline an ensemble of Random Forest classifiers was learned to predict emotion score using available training set. In case when the faces have not been detected, one member of our ensemble extracts features from the whole image. During our experimental study, the proposed approach showed the lowest error rate when compared to other explored techniques. In particular, we achieved 75.4% accuracy on the validation data, which is 20% higher than the handcrafted feature-based baseline. The source code using Keras framework is publicly available.

Added: Oct 17, 2017

About neural-network algorithms application in viseme classification problem with face video in audiovisual speech recognition systems

Savchenko A., Khokhlova Y. I. Optical Memory and Neural Networks (Information Optics). 2014. Vol. 23. No. 1. P. 34-42.

The paper considers the phoneme recognition by facial expressions of a speaker in voice-activated control systems. We have developed a neural network recognition algorithm by using the phonetic words decoding method and the requirement for isolated syllable pronunciation of voice commands. The paper presents the experimental results of viseme (facial and lip position corresponding to a particular phoneme) classification of Russian vowels. We show the dependence of the classification accuracy on the used classifier (multilayer feed-forward network, support vector machine, k-nearest neighbor method), image features (histogram of oriented gradients, eigenvectors, SURF local descriptors) and the type of camera (built-in or Kinect one). The best accuracy of speaker-dependent recognition is shown to be 85% for a built-in camera and 96% for Kinect depth maps when the classification is performed with the histogram of oriented gradients and the support vector machine.

Added: Mar 26, 2014