?
ProcrustesGPT: Compressing LLMs with Structured Matrices and Orthogonal Transformations
P. 26937–26949.
Large language models (LLMs) demonstrate impressive results in natural language processing tasks but require a significant amount of computational and memory resources. Structured matrix representations are a promising way for reducing the number of parameters of these models. However, it seems unrealistic to expect that weight matrices of pretrained models can be accurately represented by structured matrices without any fine-tuning.To overcome this issue, we utilize the fact that LLM output is invariant under certain orthogonal transformations of weight matrices.This insight can be leveraged to identify transformations that significantly improve the compressibility of weights within structured classes.The proposed approach is applicable to various types of structured matrices that support efficient projection operations. Code is available at: https://github.com/GrishKate/ProcrustesGPT.
In book
Association for Computational Linguistics, 2025.
Biryukova K., Chelnokova D., Erkenova J. et al., , in: Analysis of Images, Social Networks and Texts. AIST 2024Issue 2364.: Cham: Springer, 2024. P. 109–121.
Visual Question Answering is one of the essential parts of machine reasoning. Datasets are created to train a model to perform this task. However, there are only a few datasets for the Russian language. Moreover, existing sets may have strong biases, allowing models to score high without reasoning. In this paper, we adapt the idea ...
Added: June 14, 2026
Tikhonova M., Chekalina V., Chervyakov A. et al., , in: Proceedings of the Seventh Workshop on Teaching Natural Language Processing (TeachNLP 2026).: Association for Computational Linguistics, 2026. P. 41–44.
Modern LLM education is increasingly centered on system building: grounding generation with retrieval, enabling tool use, and deploying models under latency and cost constraints. We present an updated release of our open course on Transformer-based LLMs and multimodal models (Nikishina et al, 2024). The update introduces topics which became importance since the first edition, namely ...
Added: June 13, 2026
Churin I., Apishev M., Tikhonova M. et al., , in: Proceedings of the 6th Workshop on Computational Approaches to Discourse, Context and Document-Level Inferences (CODI 2025).: Suzhou: Association for Computational Linguistics, 2025. P. 1–13.
Recent progress in Natural Language Processing (NLP) has driven the creation of Large Language Models (LLMs) capable of tackling a vast range of tasks. A critical property of these models is their ability to handle large documents and process long token sequences, which has fostered the need for a robust evaluation methodology for long-text scenarios. ...
Added: June 11, 2026
Derkacheva A., Sakirkina M., Kraev G. et al., /. 2026.
Comprehensive data on natural hazards and their consequences are crucial for effective for risk assessment, adaptation planning, and emergency response. However, many countries face challenges with fragmented, inconsistent, and inaccessible data, particularly regarding local-scale events. To address this data gap in Russia, we developed an end-to-end processing pipeline that scrapes news from various online sources, ...
Added: April 28, 2026
A.V. Demidovskij, Burmistrova E. O., E.I. Zharikov, Optical Memory and Neural Networks (Information Optics) 2025 Vol. 34 P. S166–S174
Large Language Models (LLMs) require a lot of computational resources for inference. That is why the latest advancements in hardware design may offer many possibilities for speeding the LLM up. For example, TPU optimize calculations on data, transformed into the Coordinate sparse tensor format. The SparseCore processing unit that performs the calculations is heavily tailored ...
Added: December 22, 2025
A. V. Demidovskij, A. I. Trutnev, Optical Memory and Neural Networks (Information Optics) 2025 Vol. 34 No. Suppl. 1 P. S16–S29
Large Language Models (LLMs) are widely employed across a broad range of applications due to their versatility and state-of-the-art performance. However, as usage scenarios grow, there is a pressing demand for task-specific adaptation of LLMs through fine-tuning. While full fine-tuning (FT) remains the most preferred in terms of quality, its high memory and computation requirements ...
Added: December 22, 2025
Малинов С. А., Галактика медиа: журнал медиа исследований 2025 Т. 7 № 4 С. 154–173
In recent years, artificial intelligence (AI) has been actively integrated into everyday human life. Its popularity continues to grow steadily, and companies increasingly employ AI to optimize and accelerate workflows. Ordinary users leverage large language models (LLMs) and multimodal AI systems to perform a wide range of tasks, including generating texts, images, and videos; planning ...
Added: December 7, 2025
Ignatenko V., Surkov A., Zakharov V. et al., Sci 2025 Vol. 7 No. 3 Article 130
Large language models (LLMs) have recently demonstrated remarkable capabilities in natural language processing, mathematical reasoning, and code generation. However, their potential for solving differential equations—fundamental to applied mathematics, physics, and engineering—remains insufficiently explored. For the first time, we applied LLMs as translators from the textual form of an equation into the textual representation of its ...
Added: October 10, 2025
Yusupov V., Rakhuba M., Frolov E., , in: CIKM '25: Proceedings of the 34rd ACM International Conference on Information and Knowledge Management.: ACM, 2025. Ch. 1 P. 5469–5473.
In this work, we present a fast and effective Linear approach for updating recommendations in a scalable graph-based recommender system UltraGCN. Solving this task is extremely important to maintain the relevance of the recommendations under the conditions of a large amount of new data and changing user preferences. To address this issue, we adapt the ...
Added: October 3, 2025
Yusupov V., Rakhuba M., Frolov E., , in: RecSys '25: Proceedings of the Nineteenth ACM Conference on Recommender Systems.: ACM, 2025. Ch. 1 P. 1217–1221.
Recent studies have demonstrated the potential of hyperbolic geometry for capturing complex patterns from interaction data in recommender systems. In this work, we introduce a novel hyperbolic recommendation model that uses geometrical insights to improve representation learning and increase computational stability at the same time. We reformulate the notion of hyperbolic distances to unlock additional ...
Added: October 3, 2025
Vladimir Zakharov, Anton Surkov, Sergei Koltcov, Procedia Computer Science 2025 Vol. 258 P. 1169–1178
The rapid development of large language models (LLMs), including their successful application to solving mathematical problems requiring complex reasoning, presents a potential avenue for using LLMs in solving differential equations. While these equations are currently being solved successfully both numerically and via the symbolic approach, it is possible that fine-tuned LLMs, if they treat solving ...
Added: August 21, 2025
Matkin N., Smirnov A., Usanin M. et al., , in: 12th International Conference, AIST 2024, Bishkek, Kyrgyzstan, October 17–19, 2024, Revised Selected Papers.: Cham: Springer, 2025.
The labor market is undergoing rapid changes, with increasing demands on job seekers and a surge in job openings. Identifying essential skills and competencies from job descriptions is challenging due to varying employer requirements and the omission of key skills. This study addresses these challenges by comparing traditional Named Entity Recognition (NER) methods based on ...
Added: July 26, 2025
Lekili Y., Polishchuk A., Advances in Mathematics 2023 Vol. 418 Article 108942
For an appropriate choice of a -grading structure, we prove that the wrapped Fukaya category of the symmetric square of a -punctured sphere, i.e. the Weinstein manifold given as the complement of generic lines in is quasi-equivalent to the derived category of coherent sheaves on a singular surface constructed as the boundary of a toric Landau-Ginzburg model . We do this ...
Added: December 2, 2024
Kodryan M., Kropotov D., Vetrov D., , in: Proceedings of The 26th International Conference on Artificial Intelligence and Statistics (AISTATS 2023), Volume 206Vol. 206.: Valencia: PMLR, 2023. P. 3718–3732.
Tensor decomposition methods have proven effective in various applications, including compression and acceleration of neural networks. At the same time, the problem of determining optimal decomposition ranks, which present the crucial parameter controlling the compressionaccuracy trade-off, is still acute. In this paper, we introduce MARS - a new efficient method for the automatic selection of ...
Added: June 9, 2023
Kodryan M., Kropotov D., Vetrov D., / Series QTNML 2020 "First Workshop on Quantum Tensor Networks in Machine Learning, NeurIPS 2020". 2020.
Tensor decomposition methods have recently proven to be efficient for compressing and accelerating neural networks. However, the problem of optimal decomposition structure determination is still not well studied while being quite important. Specifically, decomposition ranks present the crucial parameter controlling the compression-accuracy trade-off. In this paper, we introduce MARS - a new efficient method for ...
Added: February 5, 2021
Pavlov A., Mathematische Zeitschrift 2021 No. 297 P. 223–254
We show that for maximal Cohen–Macaulay modules over the homogeneous coordinate ring of a smooth Calabi–Yau varieties X, the computation of Betti numbers can be reduced to computations of dimensions of certain HomHom spaces in the bounded derived category Db(X). In the simplest case of a smooth elliptic curve E embedded in P2 as a smooth cubic, we get explicit values for Betti ...
Added: October 31, 2020
Polishchuk A., Arnold Mathematical Journal 2019 Vol. 5 No. 1 P. 23–35
We present an improved construction of the fundamental matrix factorization in the FJRW-theory given in Polishchuk and Vaintrob (J Reine Angew Math 714:1—22, 2016). The revised construction is coordinate-free and works for a possibly nonabelian finite group of symmetries. One of the new ingrediants is the category of dg-matrix factorizations over a dg-scheme. ...
Added: September 4, 2019
Grachev A., Ignatov D. I., Savchenko A., Applied Soft Computing Journal 2019 Vol. 79 P. 354–362
Recurrent neural networks have proved to be an effective method for statistical language modeling. However, in practice their memory and run-time complexity are usually too large to be implemented in real-time offline mobile applications. In this paper we consider several compression techniques for recurrent neural networks including Long–Short Term Memory models. We make particular attention ...
Added: June 12, 2019
Efimov A., International Mathematics Research Notices 2018 No. 12 P. 3834–3869
In this article, we will show that for a smooth quasi-projective variety X over complex numbers, and regular function W on X, the periodic cyclic homology of the DG category of matrix factorizations of W on X is identified (under the Riemann–Hilbert correspondence) with the vanishing cohomology, with the monodromy twisted by a sign. Also, ...
Added: October 14, 2018
Katzarkov L. V., Deliu D., Favero D. et al., Advances in Mathematics 2016 Vol. 295 P. 195–249
Building upon ideas of Eisenbud, Buchweitz, Positselski, and others, we introduce the notion of a factorization category. We then develop some essential tools for working with factorization categories, including constructions of resolutions of factorizations from resolutions of their components and derived functors. Using these resolutions, we lift fully-faithfulness and equivalence statements from derived categories of ...
Added: October 23, 2017
Shitov Y., Linear and Multilinear Algebra 2016 Vol. 64 No. 2 P. 219–220
We give a tropical proof of a recent result of Mohindru on the CP-rank conjecture over min-max semiring. ...
Added: December 16, 2015
Efimov A., Positselski L., Algebra and Number Theory 2015 Vol. 9 No. 5 P. 1159–1292
We define the triangulated category of relative singularities of a closed subscheme in a scheme. When the closed subscheme is a Cartier divisor, we consider matrix factorizations of the related section of a line bundle, and their analogues with locally free sheaves replaced by coherent ones. The appropriate exotic derived category of coherent matrix factorizations ...
Added: September 4, 2015