What is a Fair Value of Your Recommendation List?

P. 1–12.

Бобриков В. В., Ненова Е. Н., Ignatov D. I.

We propose a new quality metric for recommender systems. The main feature of our approach is the fact, that we take into account the set of requirements, which are important for business application of a recommender. Thus, we construct a general criterion, named “audience satisfaction”, which thoroughly describe the result of interaction between users and recommendation service. During the criterion construction we had to deal with a number of common recommenders’ problems: a) Most of users rate only a random part of the objects they consume and a part of the objects that were recommended to them; b) Attention of users is distributed very unevenly over the list of recommendations and it requires a special behavioral model; c) The value of the user’s rate measures the level of his/her satisfaction, hence these values should be naturally incorporated in the criterion intrinsically; d) Different elements may often dramatically differ from each other by popularity (long tail – short head problem) and this effect prevents accurate measuring of user’s satisfaction. The final metric takes into account all these issues, leaving opportunity to adjust the metric performance based on proper behavioral models and parameters of short head problem treatment.

 

Keywords: оценка качества, метрики качества, Показатель AUC, рекомендательные системы и алгоритмы, quality metric, explicit feedback, movie recommendation, AUC, cold start, recommendations for novices, холодный старт, рекомендации для новичков, явный отклик, Recommender Systems

Publication based on the results of: Mining Data with Complex Structure and Semantic Technologies (2016)

In book

Proceedings of the Third Workshop on Experimental Economics and Machine Learning (EEML 2016), Moscow, Russia, July 18, 2016

Vol. 1627. , Aachen: CEUR Workshop Proceedings, 2016.