?
Renormalization approach to the task of determining the number of topics in topic modeling
P. 234–247.
Topic modeling is a widely used approach for clustering text documents, however, it possesses a set of parameters that must be determined by a user, for example, the number of topics. In this paper, we propose a novel approach for fast approximation of the optimal topic number that corresponds well to human judgment. Our method combines renormalization theory and Renyi entropy approach. The main advantage of this method is computational speed which is crucial when dealing with big data. We apply our method to Latent Dirichlet Allocation model with Gibbs sampling procedure and test our approach on two datasets in different languages. Numerical results and comparison of computational speed demonstrate significant gain in time with respect to standard grid search methods.
Keywords: Renormalizationренормализациявероятностное тематическое моделированиеprobabilistic topic modeling
Publication based on the results of:
- Р. М., Sergei Koltcov, Surkov A. et al., Computers, Materials and Continua 2026 Vol. 86 No. 3 Article 99
Activation pruning reduces neural network complexity by eliminating low-importance neuron activations, yet identifying the critical pruning threshold—beyond which accuracy rapidly deteriorates—remains computationally expensive and typically requires exhaustive search. We introduce a thermodynamics-inspired framework that treats activation distributions as energy-filtered physical systems and employs the free energy of activations as a principled evaluation metric. Phase-transition–like phenomena ...
Added: December 19, 2025
Blokh A., Levin G., Oversteegen L. et al., Communications in Mathematical Physics 2025 Vol. 406 Article 141
The modulus of a polynomial-like (PL) map is an important invariant that controls distortion of the straightening map and, hence, geometry of the corresponding PL Julia set. Lower bounds on the modulus, called complex a priori bounds, are known in a great variety of contexts. For any rational function we complement this by an upper bound ...
Added: May 31, 2025
Fedor Ozhegov, Reviews in Mathematical Physics 2024 Vol. 36 No. 7 Article 2450017
In this paper, we study Feynman checkers, one of the most elementary models of electron motion. It is also known as a one-dimensional quantum walk or an Ising model at an imaginary temperature. We add the simplest non-trivial electromagnetic field and find the limits of the resulting model for small lattice step and large time, ...
Added: May 17, 2024
Blokh A., Oversteegen L., Timorin V., Nonlinearity 2024 Vol. 37 No. 3 Article 035003
We establish a version of the Pommerenke–Levin–Yoccoz inequality for the modulus of a polynomial-like (PL) restriction of a polynomial and give two applications. First we show that if the modulus of a PL restriction of a polynomial is bounded from below then this restricts the combinatorics of the polynomial. The second application concerns parameter slices ...
Added: February 16, 2024
Skripchenko A., Успехи математических наук 2023 Т. 78 № 6(474) С. 3–46
Research on dynamical and topological properties of interval exchange transformations and their natural generalizations is an important problem that belongs to the intersection of the several branches of mathematics: dynamical systems, low-dimensional topology, algebraic geometry, number theory and geometric group theory. The goal of the current survey is to make a systematic presentation of the ...
Added: December 28, 2023
Blokh A., Oversteegen L., Timorin V., Moscow Mathematical Journal 2023 Vol. 23 No. 4 P. 441–461
A cubic polynomial $P$ with a non-repelling fixed point $b$ is said to be immediately renormalizable if there exists a (connected) QL invariant filled Julia set $K^*$ such that $b\in K^*$. In that case, exactly one critical point of $P$ does not belong to $K^*$. We show that if, in addition, the Julia set of $P$ has no (pre)periodic cutpoints, then ...
Added: November 29, 2023
Милкова М. А., Экономическая наука современной России 2020 № 1 С. 143–157
Nowadays the process of information accumulation is so rapid that the concept of the usual iterative search requires revision. Being in the world of oversaturated information in order to comprehensively cover and analyze the problem under study, it is necessary to make high demands on the search methods. An innovative approach to search should flexibly ...
Added: June 29, 2023
Shanenko, A. A., Saraiva, T. T., Vagov, A. et al., Physical Review B: Condensed Matter and Materials Physics 2022 Vol. 105 No. 21 Article 214527
Chainlike structured superconductive materials (such as A2Cr3As3, with A=K,Rb,Cs) exhibit the multiband electronic structure of single-particle states, where coexisting quasi-one-dimensional (Q1D) and conventional higher-dimensional energy bands take part in the creation of the aggregate superconducting condensate. When the chemical potential approaches the edge of a Q1D band in a single-band superconductor, the corresponding mean-field critical temperature increases ...
Added: July 3, 2022
Koltcov Sergei, Ignatenko V., Entropy 2020 Vol. 22 No. 5 P. 1–23
In practice, to build a machine learning model of big data, one needs to tune model parameters. The process of parameter tuning involves extremely time-consuming and computationally expensive grid search. However, the theory of statistical physics provides techniques allowing us to optimize this process. The paper shows that a function of the output of topic ...
Added: May 18, 2020
Koltsov S., Ignatenko V., Boukhers Z. et al., Entropy 2020 Vol. 22 No. 4 P. 1–13
Topic modeling is a popular technique for clustering large collections of text documents. A variety of different types of regularization is implemented in topic modeling. In this paper, we propose a novel approach for analyzing the influence of different regularization types on results of topic modeling. Based on Renyi entropy, this approach is inspired by ...
Added: April 1, 2020
Алексейчук Н. Н., Sarkisyan V., Emelyanov A. et al., , in: Digital Transformation and Global Society. Fourth International Conference, DTGS 2019, St. Petersburg, Russia, June 19–21, 2019, Revised Selected Papers.: Springer, 2019. P. 68–81.
In this paper, we present a project on the analysis of an extensive corpus of strategic planning documents, devoted to various aspects of the development of Russian regions. The main purposes of the project are: 1) to extract different aspects of goal setting and planning, 2) to form an ontology of goals and criteria of ...
Added: October 30, 2019
Koltsov S., Ignatenko V., Koltsova O., Entropy 2019 Vol. 21 No. 7 P. 1–29
Topic modeling is a popular approach for clustering text documents. However, current tools have a number of unsolved problems such as instability and a lack of criteria for selecting the values of model parameters. In this work, we propose a method to solve partially the problems of optimizing model parameters, simultaneously accounting for semantic stability. ...
Added: July 5, 2019
Ignatenko V., Sergei Koltcov, Staab S. et al., Journal of Physics: Conference Series 2019 Vol. 1163 No. 1 P. 1–6
In the framework of this paper we apply multifractal formalism to the analysis of
statistical behaviour of topic models under variation of the number of topics. Fractal analysis
of topic models allows to show that self-similar fractal clusters exist in large textual collections.
We provide numerical results for 3 topic models (PLSA, ARTM, LDA Gibbs sampling) on
2 datasets, ...
Added: November 30, 2018
Nagornyy O. S., , in: Digital Transformation and Global Society Third International Conference, DTGS 2018, St. Petersburg, Russia, May 30 –June 2, 2018, Revised Selected Papers, Part IIssue 858.: Cham: Springer, 2018. Ch. P. 83–94.
The paper reveals the topic structure of ethnic discussions in the Russian-speaking social media and explores how these topics are related to the post-Soviet ethnic groups. Analyzed more than 2.6 million texts from Russian-speak- ing social media published for two-year period from 2014 to 2015 and contained at least one of the post-Soviet ethnonyms, we ...
Added: September 3, 2018
N. A. Chirkova, K. V. Vorontsov, Journal of machine learning and data analysis 2016 Vol. 2 No. 2 P. 187–200
Probabilistic topic models uncover the latent semantics of text collections and represent each document by a multinomial distribution over topics. Hierarchical models divide topics into subtopics recursively, thus simplifying information retrieval, browsing and understanding of large multidisciplinary collections. The most of existing approaches to hierarchy learning rely on Bayesian inference. This makes difficult the incorporation ...
Added: October 19, 2017
K. V. Vorontsov, Journal of machine learning and data analysis 2016 Vol. 2 No. 2 P. 187–200
Probabilistic topic models uncover the latent semantics of text collections and represent each document by a multinomial distribution over topics. Hierarchical models divide topics into subtopics recursively, thus simplifying information retrieval, browsing and understanding of large multidisciplinary collections. The most of existing approaches to hierarchy learning rely on Bayesian inference. This makes difficult the incorporation ...
Added: October 19, 2017