Dimension-free Structured Covariance Estimation

N. Puchkin; M. Rakhuba

?

Dimension-free Structured Covariance Estimation

P. 4276–4306.

Given a sample of i.i.d. high-dimensional centered random vectors, we consider a problem of estimation of their covariance matrix Σ with an additional assumption that Σ can be represented as a sum of a few Kronecker products of smaller matrices. Under mild conditions, we derive the first non-asymptotic dimension-free high-probability bound on the Frobenius distance between Σ and a widely used penalized permuted least squares estimate. Because of the hidden structure, the established rate of convergence is faster than in the standard covariance estimation problem.

Language: English

Full text

Text on another site

Keywords: Kronecker product structure covariance matrices covariance estimation

Publication based on the results of:

Development and theoretical analysis of new effective stochastic machine learning algorithms (2024)

In book

Proceedings of Machine Learning Research. Volume 247: The Thirty Seventh Annual Conference on Learning Theory, 30-3 July 2023, Edmonton, Canada

PMLR, 2024.

Sharper dimension-free bounds on the Frobenius distance between sample covariance and its expectation

Puchkin N., Noskov F., Spokoiny V., Bernoulli: a journal of mathematical statistics and probability 2025 Vol. 31 No. 2 P. 1664–1691

We study properties of a sample covariance estimate $\widehat \Sigma$ given a finite sample of $n$ i.i.d. centered random elements in $\mathbb{R}^d$ with the covariance matrix $\Sigma$. We derive dimension-free bounds on the squared Frobenius norm of $(\widehat\Sigma - \Sigma)$ under reasonable assumptions. For instance, we show that $\smash{\|\widehat\Sigma - \Sigma\|_{\rm F}^2}$ differs from its ...

Added: February 12, 2025

Gaussian and robust Kronecker product covariance estimation: Existence and uniqueness

Soloveychik I., Trushin D., Journal of Multivariate Analysis 2016 Vol. 149 P. 92–113

We study the Gaussian and robust covariance estimation, assuming the true covariance matrix to be a Kronecker product of two lower dimensional square matrices. In both settings we define the estimators as solutions to the constrained maximum likelihood programs. In the robust case, we consider Tyler’s estimator defined as the maximum likelihood estimator of a ...

Added: February 20, 2018