This paper studies the exponential stability of  random matrix products driven by a general (possibly unbounded) state space Markov chain. It is a  cornerstone in the analysis of stochastic algorithms in machine learning (e.g. for parameter tracking in online-learning or reinforcement learning). The existing results impose strong  conditions  such as uniform boundedness of the matrix-valued functions and uniform ergodicity of the Markov chains.  Our main contribution is an exponential stability result for the p-th moment of random matrix product, provided  that (i) the underlying Markov chain satisfies a super-Lyapunov drift condition, (ii) the growth of the matrix-valued functions is controlled by an appropriately defined function (related to the drift condition). Using this result, we give finite-time p-th moment bounds for constant and decreasing stepsize linear stochastic approximation schemes with Markovian noise on general state space. We illustrate these findings for linear value-function estimation in reinforcement learning. We provide finite-time p-th moment bound for various members of temporal difference (TD) family of algorithms.

We undertake a precise study of the non-asymptotic properties of vanilla generative adversarial networks (GANs) and derive theoretical guarantees in the problem of  estimating an unknown $d$-dimensional density $p$  under a proper choice of the class of generators and discriminators. We prove that the resulting  density estimate converges to $p$ in terms of  Jensen-Shannon (JS) divergence at the rate $n^{-2\beta/(2\beta+d)}$ where $n$ is the sample size and $\beta$ determines the smoothness of $p$. This is the first result in the literature on density estimation using vanilla GANs with JS rates faster than $n^{-1/2}$ in the regime $\beta>d/2$.