• A
  • A
  • A
  • АБВ
  • АБВ
  • АБВ
  • A
  • A
  • A
  • A
  • A
Обычная версия сайта
  • RU
  • EN
  • HSE University
  • Publications
  • Articles
  • Automatic differentiation for Riemannian optimization on low-rank matrix and tensor-train manifolds
  • RU
  • EN
Расширенный поиск
Высшая школа экономики
Национальный исследовательский университет
Priority areas
  • business informatics
  • economics
  • engineering science
  • humanitarian
  • IT and mathematics
  • law
  • management
  • mathematics
  • sociology
  • state and public administration
by year
  • 2028
  • 2027
  • 2026
  • 2025
  • 2024
  • 2023
  • 2022
  • 2021
  • 2020
  • 2019
  • 2018
  • 2017
  • 2016
  • 2015
  • 2014
  • 2013
  • 2012
  • 2011
  • 2010
  • 2009
  • 2008
  • 2007
  • 2006
  • 2005
  • 2004
  • 2003
  • 2002
  • 2001
  • 2000
  • 1999
  • 1998
  • 1997
  • 1996
  • 1995
  • 1994
  • 1993
  • 1992
  • 1991
  • 1990
  • 1989
  • 1988
  • 1987
  • 1986
  • 1985
  • 1984
  • 1983
  • 1982
  • 1981
  • 1980
  • 1979
  • 1978
  • 1977
  • 1976
  • 1975
  • 1974
  • 1973
  • 1972
  • 1971
  • 1970
  • 1969
  • 1968
  • 1967
  • 1966
  • 1965
  • 1964
  • 1963
  • 1958
  • More
Subject
News
August 25, 2026
Scientists Develop Algorithm for More Reliable Processors in Data Centres
Researchers from HSE MIEM and Samara University have developed the LRF-3D algorithm to automatically bypass idle nodes in three-dimensional networks-on-chip. Thanks to its hierarchical architecture, the algorithm outperforms existing solutions in both speed and path accuracy, improving processor reliability for use in data centres, supercomputers, and AI computing. The source code and test results are publicly available.
August 24, 2026
Researchers Develop Method for Direct Generation of Regulatory DNA
Researchers at HSE University have developed a model for generating promoters and enhancers—DNA sequences that regulate gene activity. The model works directly with DNA nucleotides, without first transforming them into a continuous numerical representation. This solution could be useful for applications in synthetic biology and gene therapy. The study results were presented at the ICLR 2026 Workshop ‘Generative AI in Genomics (Gen^2): Barriers and Frontiers.’
August 21, 2026
Social Integration: At the Crossroads of Knowledge and Values
The International Laboratory for Social Integration Research (ILSIR) at HSE University studies the challenges faced by vulnerable groups and explores ways to help them participate fully in everyday life. To develop effective solutions, the laboratory’s researchers combine cutting-edge methods with practical fieldwork. In this interview with the HSE News Service, Laboratory Head Elena Iarskaia-Smirnova discusses the laboratory’s work.

 

Have you spotted a typo?
Highlight it, click Ctrl+Enter and send us a message. Thank you for your help!

Publications
  • Books
  • Articles
  • Chapters of books
  • Working papers
  • Report a publication
  • Research at HSE

?

Automatic differentiation for Riemannian optimization on low-rank matrix and tensor-train manifolds

SIAM Journal of Scientific Computing. 2022. Vol. 44. No. 2. P. A843–A869.
Novikov A., Rakhuba M., Oseledets I.

In scientific computing and machine learning applications, matrices and more general multidimensional arrays (tensors) can often be approximated with the help of low-rank decompositions. Since matrices and tensors of fixed rank form smooth Riemannian manifolds, one of the popular tools for finding low-rank approximations is to use Riemannian optimization. Nevertheless, efficient implementation of Riemannian gradients and Hessians, required in Riemannian optimization algorithms, can be a nontrivial task in practice. Moreover, in some cases, analytic formulas are not even available. In this paper, we build upon automatic differentiation and propose a method that, given an implementation of the function to be minimized, efficiently computes Riemannian gradients and matrix-by-vector products between an approximate Riemannian Hessian and a given vector.

Research target: Computer Science Mathematics
Language: English
DOI
Text on another site
Keywords: Riemannian optimizationtensor trainLow-rank matrix approximationautomatic differentiation
Similar publications
A unified frequency-domain framework for tilted slice localization and ischemic stroke detection
Khodadoust J., Kulikova S., Khodadoust F., Biomedical Signal Processing and Control 2027 Vol. 129 P. 111284–111284
Acute ischemic stroke (AIS) analysis from two-dimensional (2D) clinical imaging is hindered by uncontrolled slice tilt and geometric inconsistencies that violate the assumptions of pose-agnostic deep learning (DL) models. This paper proposes a unified geometry-aware, frequency-domain framework for tilted slice localization and ischemic stroke segmentation that explicitly decouples pose estimation from lesion analysis. The method ...
Added: September 2, 2026
Proceedings of the 2026 Fourth International Conference on Distributed Computing and High Performance Computing (DCHPC)
IEEE, 2026.
On behalf of the Organizing Committee, it is my great pleasure to extend a warm welcome to all participants of the Fourth International IEEE Conference on Distributed Computing and High-Performance Computing (DCHPC 2026), held in Tehran from May 10–11, 2026. This conference is jointly organized by the School of Computer Science at the Institute for Research in Fundamental Sciences (IPM) ...
Added: September 2, 2026
Orbifold Saito theory of A and D type singularities
Basalaev A., Rarovskii A., Journal of Singularities 2026 Vol. 30 P. 61–80
Saito theory associates to an isolated singularity rich structure that plays an important role in mirror symmetry. In this note we construct Saito theory for A and D type Landau-Ginzburg orbifolds. Namely, for the pairs (f,G), where f defines an isolated singularity of A and D type and G is a group of symmetries of ...
Added: September 1, 2026
Non-axiomatizability of modal predicate logics of Dedekind-complete linear orders with constant domains
Rybakov M., Shkatov D., Journal of Logic and Computation 2026 Vol. 36 No. 6 Article exag026
We prove Pi-1-1-hardness, and thus lack of recursive axiomatizability, of constant-domain modal predicate logics defined by a class of Dedekind complete linear Kripke frames containing a frame with an infinitely increasing chain of worlds. The result holds even for the language with one unary predicate letter, one propositional letter, and two individual variables. ...
Added: September 1, 2026
Semi-Interlaced Polytopes
Селянин Ф. И., Moscow Mathematical Journal 2026 Vol. 26 No. 2 P. 167–187
Minkowski mixed volume of n subpolytopes D1,…,Dn of a polytope P⊂Rn clearly does not exceed the normalized volume n!Vol(P). Equality holds if and only if the subpolytopes are interlaced, i.e., each proper face F⊊P intersects at least dim(F)+1 of the polytopes Di. Efficiently computing mixed volumes for more general collections of subpolytopes is crucial for estimating the complexity of numerically solving polynomial systems. Motivated by relaxing the bound dim(F)+1 to dim(F), we ...
Added: August 31, 2026
A new spin on polynomial relations among kappa classes
Kazaryan M., Dunin-Barkowski P., Bychkov B. et al., International Mathematics Research Notices 2026 Vol. 14 Article rnag146
We prove a recent conjecture of the fourth named author with P. Norbury that states a system of universal polynomial relations among the kappa classes on the moduli spaces of algebraic curves. The proof involves localization and materialization analysis of the spin Gromov–Witten theory of the projective line and is dictated by Z 2 -equivariant ...
Added: August 31, 2026
Any Topological Recursion on a Rational Spectral Curveis KP Integrable
Kazaryan M., Dunin-Barkowski P., Bychkov B. et al., Communications in Mathematical Physics 2026 Vol. 407 No. 69
We prove that for any initial data on a genus zero spectral curve the cor responding correlation differentials of topological recursion are KP integrable. As an application we prove KP integrability of partition functions associated via ELSV-type formulas to the r-th roots of the twisted powers of the log canonical bundles ...
Added: August 31, 2026
Сплетенный мир: кошка перевернулась. Доклад Римскому клубу
Gromov V., Переслегин С. Б., Переслегина Е. Б. et al., СПб.: Полакс, 2026.
Механизм происходящих в мире изменений носит эволюционный, а не экологический характер. Иначе говоря, Человечество столкнулось с кризисом развития, который имеет три независимые составляющие: кризис индустриального общества (фазовый кризис), кризис научного мышления (эпистемный кризис) и кризис формата существования разума (социосистемный кризис). Доклад посвящён аспектам этого триединого кризиса и возможным путям его преодоления, не сводящимся к первичному ...
Added: August 31, 2026
Multiplicity-free products of Schubert divisors
Devyatov R. A., Mathematical notes 2026 Vol. 119 No. 3 P. 782–786
Let G/B be a flag variety over ℂ, where G is a simple algebraic group with a simply laced Dynkin diagram, and B is a Borel subgroup. We say that the product of classes of Schubert divisors in the Chow ring is multiplicity free if it is possible to multiply it by a Schubert class ...
Added: August 30, 2026
Mukai models of Fano varieties
Bayer A., Kuznetsov A., Macrì E., Journal fuer die reine und angewandte Mathematik 2026 Vol. 2026 No. 836 P. 111–162
We give a self-contained and simplified proof of Mukai’s classification of prime Fano threefolds of index 1 and genus g ≥ 6 with at most factorial terminal singularities, and of its extension to higher dimension. ...
Added: August 30, 2026
Mukai bundles on Fano threefolds
Bayer A., Kuznetsov A., Macrì E., Compositio Mathematica 2026 Vol. 162 No. 1 P. 59–99
We give a proof of Mukai’s theorem on the existence of certain exceptional vector bundles on prime Fano threefolds. To our knowledge this is the first complete proof in the literature. The result is essential for Mukai’s biregular classification of prime Fano threefolds, and for the existence of semiorthogonal decompositions in their derived categories. Our ...
Added: August 30, 2026
Full exceptional collections on the symplectic isotropic Grassmannians
Guseva L., Novikov A., Advances in Mathematics 2026 Vol. 503 Article 111211
We prove that the Kuznetsov–Polishchuk exceptional collections on rational homogeneous spaces of the symplectic groups Sp(2n,C) are full and consist of vector bundles. To achieve this, we construct several classes of complexes, which we call generalized staircase complexes, symplectic staircase complexes and secondary staircase complexes — each of which may be of independent interest. ...
Added: August 30, 2026
Exceptional pairs on del Pezzo surfaces and spaces of compatible Feigin-Odesskii brackets
Polishchuk A., Rains E., Journal of the Institute of Mathematics of Jussieu 2026 Vol. 25 No. 1 P. 339–373
We prove that for every relatively prime pair of integers (d,r) with r>0, there exists an exceptional pair (O,V) on any del Pezzo surface of degree 4, such that V is a bundle of rank r and degree d. As an application, we prove that every Feigin-Odesskii Poisson bracket on a projective space can be ...
Added: August 30, 2026
Analog of theta-lifting for a curve over dual numbers over a finite field
Kazhdan D., Polishchuk A., Pure and Applied Mathematics Quarterly 2026 Vol. 22 No. 3 P. 1115–1166
We continue the study of automorphic functions associated with a curve C over the ring k[ε]/(ε²), where k is a finite field, begun in arXiv:2303.16259. Namely, we study an example of theta-lifting in this framework and show that it can be understood in terms of the orbit decomposition of the space of automorphic functions S(SL₂(F)\SL₂(A_C)) ...
Added: August 30, 2026
LoRA meets Riemannion: Muon Optimizer for Parametrization-independent Low-Rank Adapters
Vladimir Bogachev, Aletov V., Alexander Molozhavenko et al., , in: The Fourteenth International Conference on Learning Representations (ICLR 2026).: ICLR, 2026. Ch. 20503 P. 1–26.
This work presents a novel, fully Riemannian framework for Low-Rank Adaptation (LoRA) that geometrically treats low-rank adapters by optimizing them directly on the fixed-rank manifold. This formulation eliminates the parametrization ambiguity present in standard Euclidean optimizers. Our framework integrates three key components to achieve this: (1) we derive Riemannion, a new Riemannian optimizer on the fixed-rank ...
Added: April 29, 2026
Optimization on the Extended Tensor-Train Manifold with Shared Factors
Alexander Molozhavenko, Rakhuba M., Computational and Applied Mathematics 2026 Vol. 45 No. 6 Article 221
This paper studies tensors that admit decomposition in the Extended Tensor Train (ETT) format, with a key focus on the case where some decomposition factors are constrained to be equal. This factor sharing introduces additional challenges, as it breaks the multilinear structure of the decomposition. Nevertheless, we show that Riemannian optimization methods can naturally handle ...
Added: December 22, 2025
Fast gradient-free activation maximization for neurons in spiking neural networks
Pospelov N., Chertkov A., Beketov M. et al., Neurocomputing 2025 Vol. 618 Article 129070
Elements of neural networks, both biological and artificial, can be described by their selectivity for specific cognitive features. Understanding these features is important for understanding the inner workings of neural networks. For a living system, such as a neuron, whose response to a stimulus is unknown and not differentiable, the only way to reveal these ...
Added: December 14, 2024
Training a Tucker Model With Shared Factors: a Riemannian Optimization Approach
Peshekhonov I., Aleksey Arzhantsev, Rakhuba M., , in: Proceedings of The 27th International Conference on Artificial Intelligence and Statistics (AISTATS 2024), 2-4 May 2024, Palau de Congressos, Valencia, Spain. PMLR: Volume 238Vol. 238.: Valencia: PMLR, 2024. Ch. 238 P. 3304–3312.
Added: November 29, 2024
Classification Using Marginalized Maximum Likelihood Estimation and Black-Box Variational Inference
Shalileh S., , in: Data Analysis and Optimization. In Honor of Boris Mirkin's 80th Birthday.: Springer, 2023. P. 349–361.
Based upon variational inference (VI) a new set of classification algorithms has recently emerged. This set of algorithms aims (A) to increase generalization power, (B) to decrease computational complexity. However, the complex math and implementation considerations have led to the emergence of black-box variational inference methods (BBVI). Relying on these principles, we assume the existence ...
Added: October 17, 2023
Low-rank tensor approximation of singularly perturbed boundary value problems in one dimension
Marcati C., Rakhuba M., Ulander J. ., Calcolo 2022 Article 2
We derive rank bounds on the quantized tensor train (QTT) compressed approximation of singularly perturbed reaction diffusion boundary value problems in one dimension. Specifically, we show that, independently of the scale of the singular perturbation parameter, a numerical solution with accuracy 0 < 𝜀 < 1 can be represented in the QTT format with a number of parameters that ...
Added: October 31, 2021
Spectral Tensor Train Parameterization of Deep Learning Layers
Obukhov A., Rakhuba M., Liniger A. et al., , in: International Conference on Artificial Intelligence and Statistics, 13-15 April 2021, VirtualVol. 130.: PMLR, 2021. P. 3547–3555.
Added: October 31, 2021
О ТТ-рангах приближенных тензоризаций некоторых гладких функций
Высоцкий Л. И., Журнал вычислительной математики и математической физики 2021 Т. 61 № 5 С. 776–786
Tensorizations of functions are studied, that is, tensors with elements 𝐴(𝑖1,…,𝑖𝑑)=𝑓(𝑥(𝑖1,…,𝑖𝑑)) 𝐴(𝑖1,…,𝑖𝑑)=𝑓(𝑥(𝑖1,…,𝑖𝑑)), where 𝑓(𝑥) is some function defined on an interval and {𝑥(𝑖1,…,𝑖𝑑)} is a grid on this interval. For tensors of this type, the problem of approximation by tensors admitting a tensor train (ТТ) decomposition with low ТТ ranks is posed. For the class ...
Added: October 31, 2021
Tensor-Train Numerical Integration of Multivariate Functions with Singularities
Высоцкий Л. И., Смирнов А. В., Тыртышников Е. Е., Lobachevskii Journal of Mathematics 2021 Vol. 42 P. 1608–1621
Numerical integration is a classical problem emerging in many fields of science. Multivariate integration cannot be approached with classical methods due to the exponential growth of the number of quadrature nodes. We propose a method to overcome this problem. Tensor-train decomposition of a tensor approximating the integrand is constructed and used to evaluate a multivariate ...
Added: October 31, 2021
2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW)
IEEE, 2020.
Dimensionality reduction problem is stated as finding a mapping f:X ∈ R m → Z ∈ R n , where ≪ m while preserving some relevant properties of the data. We formulate topology-preserving dimensionality reduction as finding the optimal orthogonal projection to the lower-dimensional subspace which minimizes discrepancy between persistent diagrams of the original data and the projection. This ...
Added: October 14, 2021
  • About
  • About
  • Key Figures & Facts
  • Sustainability at HSE University
  • Faculties & Departments
  • International Partnerships
  • Faculty & Staff
  • HSE Buildings
  • HSE University for Persons with Disabilities
  • Public Enquiries
  • Studies
  • Admissions
  • Programme Catalogue
  • Undergraduate
  • Graduate
  • Exchange Programmes
  • Summer University
  • Summer Schools
  • Semester in Moscow
  • Business Internship
  • Research
  • International Laboratories
  • Research Centres
  • Research Projects
  • Monitoring Studies
  • Conferences & Seminars
  • Academic Jobs
  • Yasin (April) International Academic Conference on Economic and Social Development
  • Media & Resources
  • Publications by staff
  • HSE Journals
  • Publishing House
  • iq.hse.ru: commentary by HSE experts
  • Library
  • Economic & Social Data Archive
  • Video
  • HSE Repository of Socio-Economic Information
  • HSE1993–2026
  • Contacts
  • Copyright
  • Privacy Policy
  • Site Map
Edit