?
Convergence of an alternating maximization procedure
Journal of Machine Learning Research. 2016. No. 17(63). P. 1–53.
Andresen A., Spokoiny V.
We derive two convergence results for a sequential alternating maximization procedure to approximate the maximizer of random functionals such as the realized log likelihood in MLE estimation. We manage to show that the sequence attains the same deviation properties as shown for the profile M-estimator by Andresen and Spokoiny (2013), that means a finite sample Wilks and Fisher theorem. Further under slightly stronger smoothness constraints on the random functional we can show nearly linear convergence to the global maximizer if the starting point for the procedure is well chosen. ©2016 Andreas Andresen, and Vladimir Spokoiny.
Stanislav Morozov, Calcolo 2026 Vol. 63 No. 2 Article 23
The approximation of tensors in a low-para metric format is a crucial component in many mathematical modelling and data analysis tasks. Among the widely used low-parametric representations, the canonical polyadic (CP) decomposition is known to be very efficient. Nowadays, most algorithms for CP approximation aim to construct the approximation in the Frobenius norm; however, some ...
Added: May 22, 2026
Stanislav Morozov, Zheltkov D., Osinsky A., Russian Journal on Numerical Analysis and Mathematical Modelling 2024 Vol. 39 No. 5 P. 311–328
Nowadays, low-rank approximations are a critical component of many numerical procedures. Traditionally the problem of low-rank approximation of matrices is solved in unitary invariant norms such as Frobenius or spectral norm due to the existence of efficient methods for constructing approximations. However, recent results discover the potential of low-rank approximations in the Chebyshev norm, which ...
Added: February 18, 2026
Stanislav Morozov, Smirnov M., Zamarashkin N., Linear Algebra and its Applications 2023 Vol. 679 P. 4–29
The problem of low rank approximation is ubiquitous in science. Traditionally this problem is solved in unitary invariant
norms such as Frobenius or spectral norm due to existence of efficient methods for building approximations. However, recent results reveal the potential of low rank approximations in Chebyshev norm, which naturally arises in many applications. In this paper ...
Added: April 10, 2025
Guminov S., Dvurechensky P., Tupitsa N. et al., , in: Proceedings of the 38th International Conference on Machine Learning (ICML 2021)Vol. 139.: PMLR, 2021. P. 3886–3898.
Added: October 30, 2022
Ivanova A., Pasechnyuk D., Grishchenko D. et al., , in: Optimization and Applications: 12th International Conference, OPTIMA 2021, Petrovac, Montenegro, September 27 – October 1, 2021, Proceedings.: Switzerland: Springer, 2021. Ch. 268319 P. 20–37.
In this paper, we present a generic framework that allows accelerating almost arbitrary non-accelerated deterministic and randomized algorithms for smooth convex optimization problems. The major approach of our envelope is the same as in Catalyst [37]: an accelerated proximal outer gradient method, which is used as an envelope for a non-accelerated inner method for the ...
Added: October 30, 2022
Tupitsa N., Dvurechensky P., Gasnikov A. et al., Journal of Inverse and Ill-posed problems 2021 Vol. 29 No. 5 P. 721–739
We consider alternating minimization procedures for convex and non-convex optimization problems with the vector of variables divided into several blocks, each block being amenable for minimization with respect to its variables while maintaining other variables' blocks constant. In the case of two blocks, we prove a linear convergence rate for alternating minimization procedure under the ...
Added: September 29, 2021
Tupitsa, N., Dvurechensky P., Gasnikov A. et al., , in: 2020 IEEE 59th Conference on Decision and Control (CDC).: IEEE, 2020. P. 6132–6137.
We study multimarginal optimal transport (MOT) problems, which include, as a particular case, the Wasserstein barycenter problem. In MOT problems, one has to find an optimal coupling between m probability measures, which amounts to finding a tensor of order m. We propose a method based on accelerated alternating minimization and estimate the complexity to find ...
Added: February 5, 2021
Tupitsa N., Gasnikov A., Dvurechensky P. et al., , in: Mathematical Optimization Theory and Operations Research. MOTOR 2020. Communications in Computer and Information ScienceVol. 1275.: Springer, 2020. P. 192–204.
In this paper we experimentally check a hypothesis, that dual problem to discrete entropy regularized optimal transport problem possesses strong convexity on a certain compact set. We present a numerical estimation technique of parameter of strong convexity and show that such an estimate increases the performance of an accelerated alternating minimization algorithm for strongly convex ...
Added: October 28, 2020
Vorontsov K. V., Potapenko A., Machine Learning 2015 Vol. 101 No. 1 P. 303–323
Probabilistic topic modeling of text collections has been recently developed mainly within the framework of graphical models and Bayesian inference. In this paper we introduce an alternative semi-probabilistic approach, which we call additive regularization of topic models (ARTM). Instead of building a purely probabilistic generative model of text we regularize an ill-posed problem of stochastic matrix factorization ...
Added: February 19, 2015
Vorontsov K. V., Potapenko A., Машинное обучение и анализ данных 2013 Т. 1 № 6 С. 657–686
Probabilistic topic models discover a low-dimensional interpretable representation of text corpora by estimating a multinomial distribution over topics for each document and a multinomial distribution over terms for each topic. A uni ed family of expectation-maximization (EM) like algorithms with smoothing, sampling, sparsing, and robustness heuristics that can be used in any combinations is considered. The ...
Added: February 19, 2015
Vorontsov K. V., Potapenko A., Компьютерные исследования и моделирование 2012 Т. 4 № 4 С. 693–706
We propose a generalized probabilistic topic model of text corpora which can incorporate heuristics of Bayesian regularization, sampling, frequent parameters update, and robustness in any combinations. Well- known models PLSA, LDA, CVB0, SWB, and many others can be considered as special cases of the proposed broad family of models. We propose the robust PLSA model ...
Added: February 19, 2015
N.A. Novikov, Pattern Recognition and Image Analysis 2014 Vol. 24 No. 3 P. 443–451
This paper considers an approach to solving the problem of binary classification of objects. This approach is based on representing one of the classes by a sequence of Gaussian mixtures with further introduction of threshold decision rules. A method of constructing hierarchical sequences of Gaussian mixtures using the partial EM algorithm is proposed. We compare ...
Added: January 16, 2015
Konstantin Vorontsov, Anna Potapenko, , in: Communications in Computer and Information ScienceVol. 436: Analysis of Images, Social Networks and Texts. Third International Conference, AIST 2014 Yekaterinburg, Russia, April 10–12, 2014 Revised Selected Papers.: Cham: Springer, 2014. P. 29–46.
Probabilistic topic modeling of text collections is a powerful tool for statistical text analysis. In this tutorial we introduce a novel non-Bayesian approach, called Additive Regularization of Topic Models. ARTM is free of redundant probabilistic assumptions and provides a simple inference for many combined and multi-objective topic models. ...
Added: December 5, 2014
Mozgunov P., , in: COMPSTAT 2014. 21st International Conference on Computational Statistics hosting the 5th IASC World Conference. Geneva, Switzerland, August 19–22, 2014. Book of Abstracts.: Geneva: [б.и.], 2014. P. 419–427.
In this paper we consider the behavior of Kalman Filter state estimates in the case of distribution with heavy tails .The simulated linear state space models with Gaussian measurement noises were used. Gaussian noises in state equation are replaced by components with alpha-stable distribution with different parameters alpha and beta. We consider the case when ...
Added: November 14, 2014
К.В. Воронцов, Потапенко А. А., Машинное обучение и анализ данных 2013 Т. 1 № 6 С. 657–686
Probabilistic topic models discover a low-dimensional interpretable representation of text corpora
by estimating a multinomial distribution over topics for each document and a multinomial
distribution over terms for each topic. A uni ed family of expectation-maximization (EM) like
algorithms with smoothing, sampling, sparsing, and robustness heuristics that can be used in
any combinations is considered. The known models PLSA (probabilistic ...
Added: May 6, 2014