Распознавание выражений лиц на основе адаптации классификатора видеоданных пользователя

Компьютерная оптика. 2023. Т. 47. № 5. С. 806–815.

Churaev E., Savchenko A.

In this paper, an approach that can significantly increase the accuracy of facial emotion recogni- tion by adapting the model to the emotions of a particular user (e.g., smartphone owner) is consid- ered. At the first stage, a neural network model, which was previously trained to recognize facial expressions in static photos, is used to extract visual features of faces on each frame. Next, the face features of video frames are aggregated into a single descriptor for a short video fragment. After that a neural network classifier is trained. At the second stage, it is proposed to perform adapting (fine- tuning) for this classifier using a small set of video data with the facial expressions of a particular user. After emotion classification, the user can adjust the predicted emotions to further improve the accuracy of personal model. As part of an experimental study for the RAVDESS dataset, it has been shown that the approach with model adaptation for a specific user can significantly (up to 20-50%) improve the accuracy of facial expression recognition on video.

Keywords: распознавание лиц, Speaker-dependent emotion recognition, facial expression classification, neural network classifier adaptation, распознавание выражений лиц, адаптация нейросетевого классификатора