Ranking journals using social choice theory methods: A novel approach in bibliometrics

Journal of Informetrics. 2018. Vol. 12. No. 2. P. 416–429.

Subochev A., Fuad Aleskerov, Pislyakov V.

We use data on economic, management and political science journals to produce quan- titative estimates of (in)consistency of the evaluations based on six popular bibliometric indicators (impact factor, 5-year impact factor, immediacy index, article influence score, SNIP and SJR). We advocate a new approach to the aggregation of journal rankings. Since the rank aggregation is a multicriteria decision problem, ranking methods from social choice theory may solve it. We apply either a direct ranking method based on the majority rule (the Copeland rule, the Markovian method) or a sorting procedure based on a tournament solution, such as the uncovered set and the minimal externally stable set. We demonstrate that the aggregate rankings reduce the number of contradictions and represent the set of the single-indicator-based rankings better than any of the six rankings themselves.

Keywords: коллективный выбор, social choice, многокритериальный выбор, импакт-фактор, impact factor, externally stable set, внешнеустойчивое множество, правило большинства, правило Коупланда, SNIP, SJR, rank aggregation, majority rule, Copeland rule, tournament solution, агрегирование ранжирований, турнирное решение, ordinal method, multiple criteria, ординальный метод

Publication based on the results of: Modern context of decision making and data analysis methods: human factor, uncertainty, risks, network models, big data (2018)