• A
  • A
  • A
  • АБВ
  • АБВ
  • АБВ
  • A
  • A
  • A
  • A
  • A
Обычная версия сайта
  • RU
  • EN
  • HSE University
  • Publications
  • Book chapter
  • Does It Look Sequential? An Analysis of Datasets for Evaluation of Sequential Recommendations
  • RU
  • EN
Расширенный поиск
Высшая школа экономики
Национальный исследовательский университет
Priority areas
  • business informatics
  • economics
  • engineering science
  • humanitarian
  • IT and mathematics
  • law
  • management
  • mathematics
  • sociology
  • state and public administration
by year
  • 2027
  • 2026
  • 2025
  • 2024
  • 2023
  • 2022
  • 2021
  • 2020
  • 2019
  • 2018
  • 2017
  • 2016
  • 2015
  • 2014
  • 2013
  • 2012
  • 2011
  • 2010
  • 2009
  • 2008
  • 2007
  • 2006
  • 2005
  • 2004
  • 2003
  • 2002
  • 2001
  • 2000
  • 1999
  • 1998
  • 1997
  • 1996
  • 1995
  • 1994
  • 1993
  • 1992
  • 1991
  • 1990
  • 1989
  • 1988
  • 1987
  • 1986
  • 1985
  • 1984
  • 1983
  • 1982
  • 1981
  • 1980
  • 1979
  • 1978
  • 1977
  • 1976
  • 1975
  • 1974
  • 1973
  • 1972
  • 1971
  • 1970
  • 1969
  • 1968
  • 1967
  • 1966
  • 1965
  • 1964
  • 1963
  • 1958
  • More
Subject
News
July 6, 2026
Ancient Craniiform Brachiopod: A Newly Discovered Species with a Unique Shell Shape and Lifestyle
Scientists from HSE University, MSU, and Tallinn University of Technology have studied a fossil species of ancient brachiopods that lived in a warm sea in what is now northern Estonia more than 445 million years ago. These ancient brachiopods developed a cup-shaped shell with a protective 'cap' that shielded them from overgrowth by other marine organisms. The study has been published in Palaeogeography, Palaeoclimatology, Palaeoecology.
July 2, 2026
Researchers Discover How Spelling Errors Slow Down Reading in Russian
Psycholinguists from the Centre for Language and Brain at HSE University–St Petersburg have shown that words that are frequently misspelled are processed more slowly by readers, even when presented with the correct spelling. The researchers confirmed this effect for the first time using Russian-language materials and found that response speed is most strongly linked to how confidently individuals can distinguish the correct spelling of a word from an incorrect one. The study has been published in The Mental Lexicon.
July 2, 2026
HSE Develops App for Assessing Phonological Processing in Children
Researchers at the HSE Centre for Language and Brain have developed a new digital tool for assessing children's phonological processing skills—the ZARYA (Sound Analysis of the Russian Language) test battery. It is the first standardised application in Russia designed to provide a fast and reliable assessment of children's ability to distinguish speech sounds, retain them in working memory, and perform phonemic analysis. The app runs on Android tablets and smartphones and is available for download from RuStore. Details of the test validation have been published in the Journal of Speech, Language, and Hearing Research.

 

Have you spotted a typo?
Highlight it, click Ctrl+Enter and send us a message. Thank you for your help!

Publications
  • Books
  • Articles
  • Chapters of books
  • Working papers
  • Report a publication
  • Research at HSE

?

Does It Look Sequential? An Analysis of Datasets for Evaluation of Sequential Recommendations

P. 1067–1072.
Klenitskiy Anton, Volodkevich Anna, Pembek A., Vasilev A.

Sequential recommender systems are an important and demanded area of research. Such systems aim to use the order of interactions in a user’s history to predict future interactions. The premise is that the order of interactions and sequential patterns play an essential role. Therefore, it is crucial to use datasets that exhibit a sequential structure to evaluate sequential recommenders properly.

We apply several methods based on the random shuffling of the user’s sequence of interactions to assess the strength of sequential structure across 15 datasets, frequently used for sequential recommender systems evaluation in recent research papers presented at top-tier conferences. As shuffling explicitly breaks sequential dependencies inherent in datasets, we estimate the strength of sequential patterns by comparing metrics for shuffled and original versions of the dataset. Our findings show that several popular datasets have a rather weak sequential structure.

Language: English
DOI
Text on another site
Keywords: datasetsrecommender systemsSASRecData CharacteristicsSequential Recommendations

In book

RecSys '24: Proceedings of the 18th ACM Conference on Recommender Systems
Association for Computing Machinery (ACM), 2024.
Similar publications
Pre-trained LLMs Meet Sequential Recommenders: Efficient User-Centric Knowledge Distillation
Severin N., Kartushov D., Urzhumov V. et al., , in: Advances in Information Retrieval: 48th European Conference on Information Retrieval, ECIR 2026, Delft, The Netherlands, March 29 – April 2, 2026, Proceedings, Part II. (LNCS, volume 16484).: Cham: Springer Publishing Company, 2026. P. 508–517.
Sequential recommender systems have achieved significant success in modeling temporal user behavior but remain limited in cap-turing rich user semantics beyond interaction patterns. Large Language Models (LLMs) present opportunities to enhance user understanding with their reasoning capabilities, yet existing integration approaches cre-ate prohibitive inference costs in real time. To address these limitations, we present a ...
Added: June 18, 2026
SMMR: Sampling-Based MMR Reranking for Faster, More Diverse, and Balanced Recommendations and Retrieval
Liakhnovich K., Lashinin O., Babkin A. et al., Proceedings of the 48th International ACM SIGIR Conference on Research and Development in Information Retrieval 2025 P. 2754–2758
Relevance and diversity are critical objectives in modern information retrieval (IR), particularly in recommender systems. Achieving a balance between relevance (exploitation) and diversity (exploration) optimizes user satisfaction and business goals such as catalog coverage and novelty. While existing post-processing reranking methods address this trade-off, they usually rely on greedy strategies, leading to suboptimal outcomes for ...
Added: February 3, 2026
An Analysis of Sequential Patterns in Datasets for Evaluation of Sequential Recommendations
Klenitskiy A., Anna Volodkevich, Pembek A. et al., ACM Transactions on Recommender Systems 2026
Sequential recommender systems are an important and in-demand area of research. These systems aim to use the order of interactions in a user’s history to predict future interactions. The premise is that the order of interactions and sequential patterns play an essential role. Therefore, it is crucial to use datasets that exhibit a sequential structure ...
Added: January 28, 2026
Autoregressive generation strategies for Top-K sequential recommendations
Anna Volodkevich, Danil Gusak, Klenitskiy A. et al., User Modelling and User-Adapted Interaction 2025 No. 35 Article 13
The goal of modern sequential recommender systems is often formulated in terms of next-item prediction. In this paper, we explore the applicability of transformer-based generative models for the Top-K sequential recommendation task, where the goal is to predict items that a user is likely to interact with in the “near future.” This goal aligns with ...
Added: January 26, 2026
Encode Me If You Can: Learning Universal User Representations via Event Sequence Autoencoding
Klenitskiy A., Fatkulin A., Denisova D. et al., , in: RecSysChallenge '25: Proceedings of the Recommender Systems Challenge 2025.: Association for Computing Machinery (ACM), 2025. P. 26–30.
Building universal user representations that capture the essential aspects of user behavior is a crucial task for modern machine learning systems. In real-world applications, a user’s historical interactions often serve as the foundation for solving a wide range of predictive tasks, such as churn prediction, recommendations, or lifetime value estimation. Using a task-independent user representation ...
Added: January 26, 2026
Benefiting from Negative yet Informative Feedback by Contrasting Opposing Sequential Patterns
Ivanova V., Frolov E., Vasilev A., , in: RecSys '25: Proceedings of the Nineteenth ACM Conference on Recommender Systems.: ACM, 2025. P. 1142–1147.
We consider the task of learning from both positive and negative feedback in a sequential recommendation scenario, as both types of feedback are often present in user interactions. Meanwhile, conventional sequential learning models usually focus on considering and predicting positive interactions, ignoring that reducing items with negative feedback in recommendations improves user satisfaction with the ...
Added: January 26, 2026
Time to Split: Exploring Data Splitting Strategies for Offline Evaluation of Sequential Recommenders
Gusak D., Volodkevich A., Klenitskiy A. et al., , in: RecSys '25: Proceedings of the Nineteenth ACM Conference on Recommender Systems.: ACM, 2025. P. 874–883.
Modern sequential recommender systems, ranging from lightweight transformer-based variants to large language models, have become increasingly prominent in academia and industry due to their strong performance in the next-item prediction task. Yet common evaluation protocols for sequential recommendations remain insufficiently developed: they often fail to reflect the corresponding recommendation task accurately, or are not aligned ...
Added: January 26, 2026
Let It Go? Not Quite: Addressing Item Cold Start in Sequential Recommendations with Content-Based Initialization
Pembek A., Fatkulin A., Klenitskiy A. et al., , in: RecSys '25: Proceedings of the Nineteenth ACM Conference on Recommender Systems.: ACM, 2025. P. 626–631.
Many sequential recommender systems suffer from the cold start problem, where items with few or no interactions cannot be effectively used by the model due to the absence of a trained embedding. Content-based approaches, which leverage item metadata, are commonly used in such scenarios. One possible way is to use embeddings derived from content features ...
Added: January 26, 2026
MM '25: Proceedings of the 33rd ACM International Conference on Multimedia
Association for Computing Machinery (ACM), 2025.
ACM Multimedia 2025, held in Dublin, Ireland, wasmarked by record-breaking attendance (over 2,000 registrations) and submissions (over 7,100 abstracts). The conference featured a comprehensive five-day program including workshops, grand challenges, tutorials, paper presentations, SIGMM award talks, lunchtime panels, an industry program, Interactive Art sessions, and distinguished keynote speeches from Shalini De Mello (NVIDIA), Tat-Seng Chua ...
Added: November 8, 2025
Ultra Fast Warm Start Solution for Graph Recommendations
Yusupov V., Rakhuba M., Frolov E., , in: CIKM '25: Proceedings of the 34rd ACM International Conference on Information and Knowledge Management.: ACM, 2025. Ch. 1 P. 5469–5473.
In this work, we present a fast and effective Linear approach for updating recommendations in a scalable graph-based recommender system UltraGCN. Solving this task is extremely important to maintain the relevance of the recommendations under the conditions of a large amount of new data and changing user preferences. To address this issue, we adapt the ...
Added: October 3, 2025
Leveraging Geometric Insights in Hyperbolic Triplet Loss for Improved Recommendations
Yusupov V., Rakhuba M., Frolov E., , in: RecSys '25: Proceedings of the Nineteenth ACM Conference on Recommender Systems.: ACM, 2025. Ch. 1 P. 1217–1221.
Recent studies have demonstrated the potential of hyperbolic geometry for capturing complex patterns from interaction data in recommender systems. In this work, we introduce a novel hyperbolic recommendation model that uses geometrical insights to improve representation learning and increase computational stability at the same time. We reformulate the notion of hyperbolic distances to unlock additional ...
Added: October 3, 2025
MTS Kion Implicit Contextualised Sequential Dataset for Movie Recommendation
I. Safilo, D. Tikhonovich, Petrov A. et al., Doklady Mathematics 2023 Vol. 108 No. 2 P. S456–S464
We present a new movie and TV show recommendation dataset collected from the real users of MTS Kion video-on-demand platform. In contrast to other popular movie recommendation datasets, such as MovieLens or Netflix, our dataset is based on the implicit interactions registered at the watching time, rather than on explicit ratings. We also provide rich ...
Added: May 24, 2025
Sim4Rec: Flexible and Extensible Simulator for Recommender Systems for Large-Scale Data
Anna Volodkevich, Ivanova V., Vasilev A. et al., , in: Advances in Information Retrieval: 47th European Conference on Information Retrieval, ECIR 2025, Lucca, Italy, April 6–10, 2025, Proceedings, Part IV.: Springer, 2025. P. 425–430.
Simulators for recommender systems are widely used for recommender systems performance evaluation and feedback loop effects analysis. Existing simulators often propose inflexible pipelines, are focused on narrow research tasks, or are not adapted to work with industrial large data volumes. To address these challenges, we developed the Sim4Rec simulation framework. The Sim4Rec models key aspects ...
Added: April 10, 2025
Scalable Cross-Entropy Loss for Sequential Recommendations with Large Item Catalogs
Gleb Mezentsev, Danil Gusak, Ivan V Oseledets et al., , in: RecSys '24: Proceedings of the 18th ACM Conference on Recommender Systems.: Association for Computing Machinery (ACM), 2024. P. 475–485.
Added: January 16, 2025
User response modeling in recommender systems: a survey
M. Shirokikh, Shenbin I., Alekseev A. et al., Journal of Mathematical Sciences 2024 Vol. 285 No. 2 P. 255–284
Over the last several decades, recommender systems have become an integral part of both our daily lives and the research frontier at machine learning. In this survey, we explore various approaches to developing simulators for recommendation systems, especially for modeling the user response function. We consider simple probabilistic models, approaches based on generative adversarial networks, ...
Added: November 24, 2024
Quality Metrics in Recommender Systems: Do We Calculate Metrics Consistently?
Tamm Y., Damdinov R., Vasilev A., , in: RecSys '21: Proceedings of the 15th ACM Conference on Recommender Systems.: Association for Computing Machinery (ACM), 2021. P. 708–713.
Added: November 24, 2024
RecSys '21: Proceedings of the 15th ACM Conference on Recommender Systems
Association for Computing Machinery (ACM), 2021.
Added: November 24, 2024
Turning Dross Into Gold Loss: is BERT4Rec really better than SASRec?
Klenitskiy Anton, Alexey Vasilev, , in: RecSys '23: Proceedings of the 17th ACM Conference on Recommender Systems.: Association for Computing Machinery (ACM), 2023. P. 1120–1125.
Added: November 24, 2024
  • About
  • About
  • Key Figures & Facts
  • Sustainability at HSE University
  • Faculties & Departments
  • International Partnerships
  • Faculty & Staff
  • HSE Buildings
  • HSE University for Persons with Disabilities
  • Public Enquiries
  • Studies
  • Admissions
  • Programme Catalogue
  • Undergraduate
  • Graduate
  • Exchange Programmes
  • Summer University
  • Summer Schools
  • Semester in Moscow
  • Business Internship
  • Research
  • International Laboratories
  • Research Centres
  • Research Projects
  • Monitoring Studies
  • Conferences & Seminars
  • Academic Jobs
  • Yasin (April) International Academic Conference on Economic and Social Development
  • Media & Resources
  • Publications by staff
  • HSE Journals
  • Publishing House
  • iq.hse.ru: commentary by HSE experts
  • Library
  • Economic & Social Data Archive
  • Video
  • HSE Repository of Socio-Economic Information
  • HSE1993–2026
  • Contacts
  • Copyright
  • Privacy Policy
  • Site Map
Edit