Organizing Multimedia Data in Video Surveillance Systems Based on Face Verification with Convolutional Neural Networks

P. 223–230.

Anastasiia D. Sokolova, Angelina S. Kharchevnikova, Savchenko A.

In this paper we propose the two-stage approach of organizing information in video surveillance systems. At first, the faces are detected in each frame and a video stream is split into sequences of frames with face region of one person. Secondly, these sequences (tracks) that contain identical faces are grouped using face verification algorithms and hierarchical agglomerative clustering. Gender and age are estimated for each cluster (person) in order to facilitate the usage of the organized video collection. The particular attention is focused on the aggregation of features extracted from each frame with the deep convolutional neural networks. The experimental results of the proposed approach using YTF and IJB-A datasets demonstrated that the most accurate and fast solution is achieved for matching of normalized average of feature vectors of all frames in a track.

Keywords: кластеризация, clustering, видеоанализ, video surveillance, Deep Convolutional Neural Networks, сверточные нейронные сети, face verification, верификация лиц

Publication based on the results of: Разработка и апробация эффективных методов классификации для больших баз мультимедийных данных (2017)

In book

Analysis of Images, Social Networks and Texts. 6th International Conference, 2017, Revised Selected Papers

Vol. 10716. , Cham: Springer, 2018.