Emotion Recognition of a Group of People in Video Analytics Using Deep Off-the-Shelf Image Embeddings

Ch. 19. P. 191–198.

Tarasov Alexander V., Savchenko A.

In this paper we address the group-level emotion classification problem in video analytic systems.We propose to apply the MTCNN face detector to obtain facial regions on each video frame. Next, off-the-shelf image features are extracted from each located face using preliminary trained convolutional neural networks. The features of the whole frame are computed as a mean average of image embeddings of individual faces. The resulted frame features are recognized with an ensemble of state-of-the-art classifiers computed as a weighted sum of their outputs. Experimental results with EmotiW 2017 dataset demonstrate that the proposed approach is 2–20% more accurate when compared to the conventional group-level emotion classifiers.

Keywords: распознавание эмоций, face recognition, распознавание лиц, видеоанализ, Image and Video Analysis and Understanding, Group-level Emotion Recognition, Convolutional Neural Network, сверточные нейронные сети

Publication based on the results of: Разработка и апробация эффективных методов классификации для больших баз мультимедийных данных (2017)

In book

Proceedings of Analysis of Images, Social Networks and Texts – 7th International Conference, AIST 2018, Moscow, Russia, July 5-7, 2018, Revised Selected Papers. Lecture Notes in Computer Science

Vol. 11179. , Berlin: Springer, 2018.