### ?

## On the Stability of Random Matrix Product with Markovian Noise: Application to Linear Stochastic Approximation and TD Learning

P. 1711-1752.

Publication based on the results of:

### In book

Vol. 134: Conference on Learning Theory. , PMLR, 2021

Konakov V., Mammen E., / Cornell University. Series arXiv "math". 2023. No. 2304.10673.

The Robbins-Monro algorithm is a recursive, simulation-based stochastic procedure to approximate the zeros of a function that can be written as an expectation. It is known that under some technical assumptions, Gaussian limit distributions approximate the stochastic performance of the algorithm. Here, we are interested in strong approximations for Robbins-Monro procedures. The main tool for ...

Added: April 24, 2023

Konakov V., Kozhina A., Menozzi S., ESAIM: Probability and Statistics 2017 Vol. 21 P. 88-112

We study the sensitivity of the densities of non degenerate diffusion processes and related Markov Chains with respect to a perturbation of the coefficients. Natural applications of these results appear in models with misspecified coefficients or for the investigation of the weak error of the Euler scheme with irregular coefficients. ...

Added: April 14, 2017

Molchanov S., Whitmeyer J., Applicable Analysis 2015

...

Added: June 22, 2016

Durmus A., Moulines E., Naumov A. et al., Journal of Theoretical Probability 2024 Vol. - No. -

In this paper, we establish moment and Bernstein-type inequalities for additive functionals of geometrically ergodic Markov chains. These inequalities extend the corresponding inequalities for independent random variables. Our conditions cover Markov chains converging geometrically to the stationary distribution either in V-norms or in weighted Wasserstein distances. Our inequalities apply to unbounded functions and depend explicitly on ...

Added: September 7, 2021

Veretennikov A., Veretennikova M., Известия РАН. Серия математическая 2022 Т. 86 № 1 С. 98-133

We continue the work of improving the rate of convergence of ergodic homogeneous Markov chains. The setting is more general than in previous papers: we are able to get rid of the assumption about a common dominating measure and consider the case of inhomogeneous Markov chains as well as more general state spaces. We give examples ...

Added: March 14, 2022

Runev E. V., Springer Nature Switzerland 2022 Vol. 402 No. 1 P. 343-351

The book presents latest developments in the field of high-speed railway, Hyperloop transportation technologies and Maglev system. In recent years, railway transport has received a powerful impetus in its development. With the advent of the 4th Industrial revolution, the transport sector is moving towards full digitalization. TransSiberia is a platform where both the rail industry ...

Added: November 1, 2022

Olshanski G., Selecta Mathematica, New Series 2021 Vol. 27 Article 41

Using Okounkov’s q-integral representation of Macdonald polynomials we construct an infinite sequence Ω1,Ω2,Ω3,… of countable sets linked by transition probabilities from Ω𝑁 to Ω𝑁−1 for each 𝑁=2,3,…. The elements of the sets Ω𝑁 are the vertices of the extended Gelfand–Tsetlin graph, and the transition probabilities depend on the two Macdonald parameters, q and t. These ...

Added: June 4, 2021

Douc R., Moulines E., Priouret P. et al., Switzerland : Springer Publishing Company, 2019

This book covers the classical theory of Markov chains on general state-spaces as well as many recent developments. The theoretical results are illustrated by simple examples, many of which are taken from Markov Chain Monte Carlo methods. The book is self-contained, while all the results are carefully and concisely proven. Bibliographical notes are added at ...

Added: December 29, 2018

Konakov V., Mammen E., Probability Theory and Related Fields 2000 Vol. 117 No. 4 P. 551-587

We consider triangular arrays of Markov chains that converge weakly to a diffusion process. Local limit theorems for transition
densities are proved. ...

Added: October 15, 2012

Cardoso G., Samsonov S., Thin A. et al., , in : Thirty-Sixth Conference on Neural Information Processing Systems : NeurIPS 2022. : Curran Associates, Inc., 2022. P. 716-729.

Added: February 1, 2023

Samsonov S., Lagutin E., Gabrie M. et al., , in : Thirty-Sixth Conference on Neural Information Processing Systems : NeurIPS 2022. : Curran Associates, Inc., 2022. P. 5178-5193.

Added: February 1, 2023

Kelbert M., Sazonov I., Gravenor M., Mathematical Biosciences 2016 Vol. 274 P. 45-57

We consider the epidemic dynamics in stochastically interacting population centers coupled by a random migration. ...

Added: February 15, 2016

Sheresheva M. Y., Kolesnik N. A., Industrial Marketing Management 2011 Vol. 40 P. 979-987

This article investigates contemporary distribution processes in the industrial market. The main trend in distribution during the recent decades manifests itself in a growing number of network-type distribution chains. Based on the evolutionary trends in distribution research, we came up with the idea to investigate distribution networks processes using mathematical tools of probability theory. We ...

Added: July 27, 2012

Durmus A., Moulines E., Naumov A. et al., / Cornell University. Series arXiv "math". 2023.

In this paper, we establish novel deviation bounds for additive functionals of geometrically ergodic Markov chains similar to Rosenthal and Bernstein-type inequalities for sums of independent random variables. We pay special attention to the dependence of our bounds on the mixing time of the corresponding chain. Our proof technique is, as far as we know, ...

Added: June 18, 2023

Durmus A., Moulines E., Naumov A. et al., , in : Advances in Neural Information Processing Systems 34 (NeurIPS 2021). : Curran Associates, Inc., 2021. P. 30063-30074.

This paper provides a non-asymptotic analysis of linear stochastic approximation (LSA) algorithms with fixed stepsize. This family of methods arises in many machine learning tasks and is used to obtain approximate solutions of a linear system $\bar{A}\theta = \bar{b}$ for which $\bar{A}$ and $\bar{b}$ can only be accessed through random estimates $\{({\bf A}_n, {\bf b}_n): ...

Added: February 17, 2022

Blank M., Discrete and Continuous Dynamical Systems 2021 Vol. 41 No. 4 P. 1649-1665

We study qualitative properties of the set of recurrent points of
finitely generated free semigroups of measurable maps. In the case of a single
generator the classical Poincare recurrence theorem shows that these properties are closely related to the presence of an invariant measure. Curious, but
otherwise it turns out to be possible that almost all points are ...

Added: October 21, 2020

Konakov V., Mammen E., Probability Theory and Related Fields 2009 Vol. 143 No. 1 P. 137-176

We consider triangular arrays of Markov chains that converge weakly to a diffusion process. Second order Edgeworth type expansions for transition densities are proved. The paper differs from recent results in two respects. We allow nonhomogeneous diffusion limits and we treat transition densities with time lag converging to zero. Small time asymptotics are motivated by ...

Added: December 4, 2012

Belomestny D., Moulines E., Samsonov S., Statistics and Computing 2022 Vol. 32 No. 1 Article 16

In this paper, we propose an efficient variance reduction approach for additive functionals of Markov chains relying on a novel discrete-time martingale representation. Our approach is fully non-asymptotic and does not require the knowledge of the stationary distribution (and even any type of ergodicity) or specific structure of the underlying density. By rigorously analyzing the ...

Added: August 31, 2020

Konakov V., Mammen E., Bernoulli: a journal of mathematical statistics and probability 2005 Vol. 11 No. 4 P. 591-641

We consider triangular arrays of Markov chains that converge weakly to a diffusion process. We prove Edgeworth-type expansions of order o(n-1-δ),δ>0, for transition densities. For this purpose we apply the parametrix method to represent the transition density as a functional of densities of sums of independent and identically distributed variables. Then we apply Edgeworth expansions ...

Added: December 4, 2012

Bezhaeva Z., Куликов В. Л., Олехова Е. Ф. et al., Труды Математического института им. В.А. Стеклова РАН 2017 Т. 297 С. 38-45

We define an invariant Erdős measure on the compact abelian group of A-adic integers. We also define an A-invariant Erdős measure on the n-dimensional torus. We show the connection between these invariant measures and functions of countable stationary Markov chains. ...

Added: September 7, 2017

Litvin Y. V., Аудит и финансовый анализ 2010 № 4 С. 266-272

In real situations, the work of project-oriented businesses takes place in conditions of high uncertainty. In particular, the moments of the receipt of project execution time, as well as costs and other factors are yutsya-random numbers with given or unknown to the laws of the distributions. Management capabilities offered by the use of stochastic process ...

Added: November 8, 2012

Igor Kheifets, Saikkonen P., Econometric Reviews 2020 Vol. 39 No. 39 P. 407-414

Smooth transition autoregressive models are widely used to capture nonlinearities in univariate and multivariate time series. Existence of stationary solution is typically assumed, implicitly or explicitly. In this paper we describe conditions for stationarity and ergodicity of vector STAR models. The key condition is that the joint spectral radius of certain matrices is below 1, ...

Added: February 23, 2021