In applied investigations, the invariance of the Lyapunov dimension under a diffeomorphism is often used. However, in the case of irregular linearization, this fact was not strictly considered in the classical works. In the present work, the invariance of the Lyapunov dimension under diffeomorphism is demonstrated in the general case. This fact is used to obtain the analytic exact upper bound of the Lyapunov dimension of an attractor of the Shimizu–Morioka system.
Topic modeling is a popular technique for clustering large collections of text documents. A variety of different types of regularization is implemented in topic modeling. In this paper, we propose a novel approach for analyzing the influence of different regularization types on results of topic modeling. Based on Renyi entropy, this approach is inspired by the concepts from statistical physics, where an inferred topical structure of a collection can be considered an information statistical system residing in a non-equilibrium state. By testing our approach on four models—Probabilistic Latent Semantic Analysis (pLSA), Additive Regularization of Topic Models (BigARTM), Latent Dirichlet Allocation (LDA) with Gibbs sampling, LDA with variational inference (VLDA)—we, first of all, show that the minimum of Renyi entropy coincides with the “true” number of topics, as determined in two labelled collections. Simultaneously, we find that Hierarchical Dirichlet Process (HDP) model as a well-known approach for topic number optimization fails to detect such optimum. Next, we demonstrate that large values of the regularization coefficient in BigARTM significantly shift the minimum of entropy from the topic number optimum, which effect is not observed for hyper-parameters in LDA with Gibbs sampling. We conclude that regularization may introduce unpredictable distortions into topic models that need further research.
A proposal for a new method of classification of objects of various nature, named “2”-soft classification, which allows for referring objects to one of two types with optimal entropy probability for available collection of learning data with consideration of additive errors therein. A decision rule of randomized parameters and probability density function (PDF) is formed, which is determined by the solution of the problem of the functional entropy linear programming. A procedure for “2”-soft classification is developed, consisting of the computer simulation of the randomized decision rule with optimal entropy PDF parameters. Examples are provided.
Topic modeling is a popular approach for clustering text documents. However, current tools have a number of unsolved problems such as instability and a lack of criteria for selecting the values of model parameters. In this work, we propose a method to solve partially the problems of optimizing model parameters, simultaneously accounting for semantic stability. Our method is inspired by the concepts from statistical physics and is based on Sharma–Mittal entropy. We test our approach on two models: probabilistic Latent Semantic Analysis (pLSA) and Latent Dirichlet Allocation (LDA) with Gibbs sampling, and on two datasets in different languages. We compare our approach against a number of standard metrics, each of which is able to account for just one of the parameters of our interest. We demonstrate that Sharma–Mittal entropy is a convenient tool for selecting both the number of topics and the values of hyper-parameters, simultaneously controlling for semantic stability, which none of the existing metrics can do. Furthermore, we show that concepts from statistical physics can be used to contribute to theory construction for machine learning, a rapidly-developing sphere that currently lacks a consistent theoretical ground.
We introduce the general class of symmetric two-qubit states guaranteeing the perfect correlation or anticorrelation of Alice and Bob outcomes whenever some spin observable is measured at both sites. We prove that, for all states from this class, the maximal violation of the original Bell inequality is upper bounded by (3/2) and specify the two-qubit states where this quantum upper bound is attained. The case of two-qutrit states is more complicated. Here, for all two-qutrit states, we obtain the same upper bound (3/2) for violation of the original Bell inequality under Alice and Bob spin measurements, but we have not yet been able to show that this quantum upper bound is the least one. We discuss experimental consequences of our mathematical study.
We present a general approach for quantifying tolerance of a nonlocal N-partite state to any local noise under different classes of quantum correlation scenarios with arbitrary numbers of settings and outcomes at each site. This allows us to derive new precise bounds in d and N on noise tolerances for: (i) an arbitrary nonlocal N-qudit state; (ii) the N-qudit Greenberger–Horne–Zeilinger (GHZ) state; (iii) the N-qubit W state and the N-qubit Dicke states, and to analyse asymptotics of these precise bounds for large N and d.
In practice, to build a machine learning model of big data, one needs to tune model parameters. The process of parameter tuning involves extremely time-consuming and computationally expensive grid search. However, the theory of statistical physics provides techniques allowing us to optimize this process. The paper shows that a function of the output of topic modeling demonstrates self-similar behavior under variation of the number of clusters. Such behavior allows using a renormalization technique. A combination of renormalization procedure with the Renyi entropy approach allows for quick searching of the optimal number of topics. In this paper, the renormalization procedure is developed for the probabilistic Latent Semantic Analysis (pLSA), and the Latent Dirichlet Allocation model with variational Expectation–Maximization algorithm (VLDA) and the Latent Dirichlet Allocation model with granulated Gibbs sampling procedure (GLDA). The experiments were conducted on two test datasets with a known number of topics in two different languages and on one unlabeled test dataset with an unknown number of topics. The paper shows that the renormalization procedure allows for finding an approximation of the optimal number of topics at least 30 times faster than the grid search without significant loss of quality.
Cardiac activity is involved in the processes of organization of goal-directed behaviour. Each behavioural act is aimed at achieving an adaptive outcome and it is subserved by the actualization of functional systems consisting of elements distributed across the brain and the rest of the body. This paper proposes a system-evolutionary view on the activity of the heart and its variability. We have compared the irregularity of the heart rate, as measured by sample entropy (SampEn), in behaviours that are subserved by functional systems formed at different stages of individual development, which implement organism-environment interactions with different degrees of differentiation. The results have shown that SampEn of the heart rate was higher during performing tasks that included later acquired knowledge (foreign language vs. native language; mathematical vocabulary vs. general vocabulary) and decreased in the stress and alcohol conditions, as well as at the beginning of learning. These results are in line with the hypothesis that irregularity of the heart rate reflects the properties of a set of functional systems subserving current behaviour, with higher irregularity corresponding to later acquired and more complex behaviour.