Нейросетевое обучение метрик: сравнение функций потерь

Доклады Российской академии наук. Математика, информатика, процессы управления (ранее - Доклады Академии Наук. Математика). 2023. Т. 514. № 2. С. 60–71.

D'yakonov A., Васильев Р. Л.

An overview of deep metric learning methods is presented. Although they have appeared in recent
years, these methods were compared only with their predecessors, with neural networks of outdated architec-
tures used for representation learning (representations on which the metric is calculated). The described
methods were compared on different datasets from several domains, using pre-trained neural networks com-
parable in performance to SotA (state of the art): ConvNeXt for images and DistilBERT for texts. Labeled
datasets were used, divided into two parts (train and test) so that the classes did not overlap (i.e., for each class
its objects are fully in train or fully in test). Such a large-scale honest comparison was made for the first time
and led to unexpected conclusions, viz. some “old” methods, for example, Tuplet Margin Loss, are superior
in performance to their modern modifications and methods proposed in very recent works.

Keywords: метрика, machine learning, metric, deep learning, similarity, Машинное обучение и анализ данных, глубокое обучение, автоматическое машинное обучение, схожесть