• A
  • A
  • A
  • АБB
  • АБB
  • АБB
  • А
  • А
  • А
  • А
  • А
Обычная версия сайта
Найдено 12 публикаций
Сортировка:
по названию
по году
Статья
Shnourkoff P. V., Novikov D. A. Working papers by Cornell University. Series math "arxiv.org". 2018. No. arXiv:1811.10993 [q-fin.GN]. P. 1-15.
Добавлено: 26 февраля 2019
Статья
Bonelli G., Gavrylenko P., Tanzini A. et al. Working papers by Cornell University. Series math "arxiv.org". 2019.
Добавлено: 13 ноября 2019
Статья
Dunin-Barkowski P., Popolitov A., Shadrin S. et al. Working papers by Cornell University. Series math "arxiv.org". 2017. Vol. 1712. No. 08614. P. 1-38.
Добавлено: 2 января 2018
Статья
Gayfullin S., Gaifullin A. A. Working papers by Cornell University. Series math "arxiv.org". 2013.
Добавлено: 15 ноября 2013
Статья
Le Gouic T., Paris Q., Rigollet P. et al. Working papers by Cornell University. Series math "arxiv.org". 2019. P. 1-17.
Добавлено: 14 ноября 2019
Статья
Naumov A., Moulines E., Каледин М. Л. et al. Working papers by Cornell University. Series math "arxiv.org". 2020. P. 1-61.

Linear two-timescale stochastic approximation (SA) scheme is an important class of algorithms which has become popular in reinforcement learning (RL), particularly for the policy evaluation problem. Recently, a number of works have been devoted to establishing the finite time analysis of the scheme, especially under the Markovian (non-i.i.d.) noise settings that are ubiquitous in practice. In this paper, we provide a finite-time  analysis for linear two timescale SA. Our bounds show that there is no discrepancy in the convergence rate between Markovian and martingale noise, only the constants are affected by the mixing time of the Markov chain. With an appropriate step size schedule, the transient term in the expected error bound is $o(1/k^c)$ and the steady-state term is ${\cal O}(1/k)$, where $c>1$ and $k$ is the iteration number. Furthermore, we present an asymptotic expansion of the expected error with a matching lower bound of $\Omega(1/k)$. A simple numerical experiment is presented to support our theory.

Добавлено: 5 февраля 2020
Статья
Polyakov N. L. Working papers by Cornell University. Series math "arxiv.org". 2018. P. 1-22.
Добавлено: 9 октября 2018
Статья
Nikolai L. Poliakov, Saveliev D. I. Working papers by Cornell University. Series math "arxiv.org". 2018. P. 1-46.
Добавлено: 18 февраля 2019
Статья
P. V. Shnurkov. Working papers by Cornell University. Series math "arxiv.org". 2017. No. 1709.03442v1. P. 1-16.
Добавлено: 13 декабря 2017
Статья
P. V. Shnurkov, K. A. Adamova. Working papers by Cornell University. Series math "arxiv.org". 2019. No. arXiv:1906.05824v1. P. 1-14.
Добавлено: 17 июня 2019
Статья
Paris Q. Working papers by Cornell University. Series math "arxiv.org". 2020. P. 1-17.
Добавлено: 10 февраля 2020
Статья
Belomestny D., Iosipoi L., Zhivotovskiy N. Working papers by Cornell University. Series math "arxiv.org". 2019.
Добавлено: 22 октября 2018