• A
  • A
  • A
  • АБВ
  • АБВ
  • АБВ
  • A
  • A
  • A
  • A
  • A
Обычная версия сайта
  • RU
  • EN
  • HSE University
  • Publications
  • Book chapter
  • Nonasymptotic Analysis of Stochastic Gradient Descent with the Richardson–Romberg Extrapolation
  • RU
  • EN
Расширенный поиск
Высшая школа экономики
Национальный исследовательский университет
Priority areas
  • business informatics
  • economics
  • engineering science
  • humanitarian
  • IT and mathematics
  • law
  • management
  • mathematics
  • sociology
  • state and public administration
by year
  • 2027
  • 2026
  • 2025
  • 2024
  • 2023
  • 2022
  • 2021
  • 2020
  • 2019
  • 2018
  • 2017
  • 2016
  • 2015
  • 2014
  • 2013
  • 2012
  • 2011
  • 2010
  • 2009
  • 2008
  • 2007
  • 2006
  • 2005
  • 2004
  • 2003
  • 2002
  • 2001
  • 2000
  • 1999
  • 1998
  • 1997
  • 1996
  • 1995
  • 1994
  • 1993
  • 1992
  • 1991
  • 1990
  • 1989
  • 1988
  • 1987
  • 1986
  • 1985
  • 1984
  • 1983
  • 1982
  • 1981
  • 1980
  • 1979
  • 1978
  • 1977
  • 1976
  • 1975
  • 1974
  • 1973
  • 1972
  • 1971
  • 1970
  • 1969
  • 1968
  • 1967
  • 1966
  • 1965
  • 1964
  • 1963
  • 1958
  • More
Subject
News
July 20, 2026
Scientists Create Open Dataset for Studying Concentration
A team of Russian researchers, including scientists from HSE University–St Petersburg, has developed the first open multimodal dataset containing recordings of brain activity, heart function, and video observations to help researchers understand what happens in the human brain during deep concentration. In the future, the dataset could accelerate the development of neural interfaces, rehabilitation technologies, and AI systems. The article has been published in Scientific Data.
July 20, 2026
‘Science Is Universal-It Knows No Borders
Fuad Aleskerov, Tenured Professor and Director of the International Centre of Decision Choice and Analysis at HSE University, together with his colleagues, has developed methods of network analysis in bibliometrics that have made it possible to identify patterns in the appearance and citation of publications in academic journals, as well as their influence on each other. When one or a number of studies are frequently cited by a wide range of journals, this is an indicator that the research is of high quality. By contrast, extensive cross-citation within a limited group of journals increases the likelihood of identifying a network of predatory publications.
July 20, 2026
Scientists Propose Method for More Efficient Resource Use in Machine Learning
An international group of researchers, including mathematicians from the AI and Digital Science Institute at the HSE Faculty of Computer Science, has provided a theoretical justification for a simple and computationally efficient method of estimating uncertainty in Stochastic Gradient Descent (SGD). The paper has been published on the scientific preprint server arXiv.org and presented at AISTATS 2026.

 

Have you spotted a typo?
Highlight it, click Ctrl+Enter and send us a message. Thank you for your help!

Publications
  • Books
  • Articles
  • Chapters of books
  • Working papers
  • Report a publication
  • Research at HSE

?

Nonasymptotic Analysis of Stochastic Gradient Descent with the Richardson–Romberg Extrapolation

.
Sheshukova M., Belomestny D., Durmus A., Moulines E., Naumov A., Samsonov S.
Language: English
Text on another site
Keywords: цепи МарковаMarkov chainsRichardson-Romberg extrapolationstochastic gradient descentстохастический градиентный спускРичардсон-Ромберг
Publication based on the results of:
Development of theoretical foundations and methods of generative artificial intelligence and their application to heterogeneous domain area (2025)

In book

Proceedings of the 13th International Conference on Learning Representations (ICLR 2025)
ICLR, 2025.
Similar publications
High-Order Error Bounds for Markovian LSA with Richardson–Romberg Extrapolation
Levin I., Naumov A., Samsonov S., , in: Proceedings of the AAAI Conference on Artificial Intelligence. AAAI-26: AAAI Technical Track on Planning, Routing, and Scheduling; AAAI Technical Track on Reasoning under Uncertainty; AAAI Technical Track on Search and Optimization. Main Track, volume 40 no. 43.: American Association for Artificial Intelligence (AAAI) Press, 2026. P. 36696–36704.
In this paper, we study the bias and high-order error bounds of the Linear Stochastic Approximation (LSA) algorithm with Polyak-Ruppert (PR) averaging under Markovian noise. We focus on the version of the algorithm with constant step size and propose a novel decomposition of the bias via a linearization technique. We analyze the structure of the ...
Added: April 17, 2026
Об одном применении теоремы А.Н. Колмогорова
Соболев В. Н., Фролов А. А., Чебышевский сборник 2025 Т. 26 № 5 С. 203–220
In the article, on the class K 0 of infinite binary sequences without the runs of ones, a consistent probability distribution P is constructed which is induced by a time-homogeneous Markov chain with a one-step transition matrix P𝜑 , and is completely determined by the golden ratio 𝜑. Using a Markov chain to construct a probability measure P ...
Added: February 11, 2026
A Revival of Conservative Ideology and the Projected Religious Landscape in Russia
Skorobogatov A., Economics of Transition and Institutional Change 2026 Vol. 34 No. 2 P. 387–409
This paper analyzes the dynamics of the public attitude towards religion using longitudinal data from Russian respondents. Applying Markov chains and regression analysis, we determine the relative success of religious groups in retaining and attracting members. Based on this information, we estimate and explain the projected religious composition of Russia. According to our results, the ...
Added: November 3, 2025
Refined Analysis of Constant Step Size Federated Averaging and Federated Richardson-Romberg Extrapolation
Paul M., Durmus A., Dieuleveut A. et al., , in: Proceedings of The 28th International Conference on Artificial Intelligence and Statistics, 3-5 May 2025, Splash Beach Resort in Mai Khao, Thailand, PMLR: vol. 258Vol. 258.: PMLR, 2025. Ch. 258 P. 5023–5031.
In this paper, we present a novel analysis of FedAvg with constant step size, relying on the Markov property of the underlying process. We demonstrate that the global iterates of the algorithm converge to a stationary distribution and analyze its resulting bias and variance relative to the problem’s solution. We provide a first-order bias expansion in ...
Added: May 18, 2025
Nonasymptotic Analysis of Stochastic Gradient Descent with the Richardson-Romberg Extrapolation
Sheshukova M., Belomestny D., Durmus A. et al., / Series arXiv "math". 2024.
We address the problem of solving strongly convex and smooth minimization problems using stochastic gradient descent (SGD) algorithm with a constant step size. Previous works suggested to combine the Polyak-Ruppert averaging procedure with the Richardson-Romberg extrapolation technique to reduce the asymptotic bias of SGD at the expense of a mild increase of the variance. We ...
Added: October 13, 2024
Rosenthal-type inequalities for linear statistics of Markov chains
Durmus A., Moulines E., Naumov A. et al., / Series arXiv "math". 2023.
In this paper, we establish novel deviation bounds for additive functionals of geometrically ergodic Markov chains similar to Rosenthal and Bernstein-type inequalities for sums of independent random variables. We pay special attention to the dependence of our bounds on the mixing time of the corresponding chain. Our proof technique is, as far as we know, ...
Added: June 18, 2023
Local Limit Theorems and Strong Approximations for Robbins-Monro Procedures
Konakov V., Mammen E., / Series arXiv "math". 2023. No. 2304.10673.
The Robbins-Monro algorithm is a recursive, simulation-based stochastic procedure to approximate the zeros of a function that can be written as an expectation. It is known that under some technical assumptions, Gaussian limit distributions approximate the stochastic performance of the algorithm. Here, we are interested in strong approximations for Robbins-Monro procedures. The main tool for ...
Added: April 24, 2023
Local-Global MCMC kernels: the best of both worlds
Samsonov S., Lagutin E., Gabrie M. et al., , in: Thirty-Sixth Conference on Neural Information Processing Systems : NeurIPS 2022.: Curran Associates, Inc., 2022. P. 5178–5193.
Added: February 1, 2023
BR-SNIS: Bias Reduced Self-Normalized Importance Sampling
Cardoso G., Samsonov S., Thin A. et al., , in: Thirty-Sixth Conference on Neural Information Processing Systems : NeurIPS 2022.: Curran Associates, Inc., 2022. P. 716–729.
Added: February 1, 2023
Mathematical Model for Assessing the Reliability of Water Supply Networks
Runev E. V., Springer Nature Switzerland 2022 Vol. 402 No. 1 P. 343–351
The book presents latest developments in the field of high-speed railway, Hyperloop transportation technologies and Maglev system. In recent years, railway transport has received a powerful impetus in its development. With the advent of the 4th Industrial revolution, the transport sector is moving towards full digitalization. TransSiberia is a platform where both the rail industry ...
Added: November 1, 2022
  • About
  • About
  • Key Figures & Facts
  • Sustainability at HSE University
  • Faculties & Departments
  • International Partnerships
  • Faculty & Staff
  • HSE Buildings
  • HSE University for Persons with Disabilities
  • Public Enquiries
  • Studies
  • Admissions
  • Programme Catalogue
  • Undergraduate
  • Graduate
  • Exchange Programmes
  • Summer University
  • Summer Schools
  • Semester in Moscow
  • Business Internship
  • Research
  • International Laboratories
  • Research Centres
  • Research Projects
  • Monitoring Studies
  • Conferences & Seminars
  • Academic Jobs
  • Yasin (April) International Academic Conference on Economic and Social Development
  • Media & Resources
  • Publications by staff
  • HSE Journals
  • Publishing House
  • iq.hse.ru: commentary by HSE experts
  • Library
  • Economic & Social Data Archive
  • Video
  • HSE Repository of Socio-Economic Information
  • HSE1993–2026
  • Contacts
  • Copyright
  • Privacy Policy
  • Site Map
Edit