?
Putting MRFs on a Tensor Train
P. 811–819.
In the paper we present a new framework for dealing with probabilistic graphical models. Our approach relies on the recently proposed Tensor Train format (TT-format) of a tensor that while being compact allows for efficient application of linear algebra operations. We present a way to convert the energy of a Markov random field to the TT-format and show how one can exploit the properties of the TT-format to attack the tasks of the partition function estimation and the MAP-inference. We provide theoretical guarantees on the accuracy of the proposed algorithm for estimating the partition function and compare our methods against several state-of-the-art algorithms.
In book
Issue 32: Proceedings of The 31st International Conference on Machine Learning. , Beijing: Microtome Publishing, 2014.
Kostylev Ilya, Kalyagin Valeriy, Operations Research Forum 2025 Vol. 6 Article 188
Graphical modelling has become a useful tool in modern data mining. Graphical model selection by observations is an important challenge for various practical applications of this technique. Optimization-based approaches are a basis for many fast and effcient algorithms to solve this problem. Many known properties of graphical model selection algorithms are related with the case ...
Added: November 26, 2025
Kodryan M., Kropotov D., Vetrov D., , in: Proceedings of The 26th International Conference on Artificial Intelligence and Statistics (AISTATS 2023), Volume 206Vol. 206.: Valencia: PMLR, 2023. P. 3718–3732.
Tensor decomposition methods have proven effective in various applications, including compression and acceleration of neural networks. At the same time, the problem of determining optimal decomposition ranks, which present the crucial parameter controlling the compressionaccuracy trade-off, is still acute. In this paper, we introduce MARS - a new efficient method for the automatic selection of ...
Added: June 9, 2023
Kodryan M., Kropotov D., Vetrov D., / Series QTNML 2020 "First Workshop on Quantum Tensor Networks in Machine Learning, NeurIPS 2020". 2020.
Tensor decomposition methods have recently proven to be efficient for compressing and accelerating neural networks. However, the problem of optimal decomposition structure determination is still not well studied while being quite important. Specifically, decomposition ranks present the crucial parameter controlling the compression-accuracy trade-off. In this paper, we introduce MARS - a new efficient method for ...
Added: February 5, 2021
Shitov Y., Linear Algebra and its Applications 2018 Vol. 544 P. 299–305
An n×n matrix A is called permutative if the rows of A are distinct permutations of a family of n distinct elements. For all n⩾3, we show that the minimal rank of a non-negative permutative matrix equals 3. The minimal rank of a generic permutative n×n matrix equals the smallest integer r such that r!⩾n. ...
Added: January 30, 2019
Izmailov P., Novikov A., Kropotov D., , in: Proceedings of Machine Learning Research. Proceedings of The International Conference on Artificial Intelligence and Statistics (AISTATS 2018).: [б.и.], 2018. P. 726–735.
We propose a method (TT-GP) for approximate inference in Gaussian Process (GP) models. We build on previous scalable GP research including stochastic variational inference based on inducing inputs, kernel interpolation, and structure exploiting algebra. The key idea of our method is to use Tensor Train decomposition for variational parameters, which allows us to train GPs ...
Added: December 10, 2018
Khrulkov V., Novikov A., Oseledets I., , in: Proceedings of the 6th International Conference on Learning Representations (ICLR 2018).: [б.и.], 2018. P. 1–12.
Added: February 4, 2018
Izmailov P., Novikov A., Kroptov D., / Series arXiv "math". 2017.
We propose a method (TT-GP) for approximate inference in Gaussian Process (GP) models. We build on previous scalable GP research including stochastic variational inference based on inducing inputs, kernel interpolation, and structure exploiting algebra. The key idea of our method is to use Tensor Train decomposition for variational parameters, which allows us to train GPs ...
Added: October 20, 2017
Grechikhin I., /. 2017.
Graphical models are used in a variety of problems to uncover hidden structures. There is a huge number of different identification procedures, constructed for different purposes. However, it is important to research different properties of such procedures and compare them in order to find out the best procedure or the best use case for some ...
Added: October 20, 2017
Kohli P., Osokin A., Jegelka S., , in: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR 2013).: Portland: IEEE, 2013. P. 1971–1978.
We discuss a model for image segmentation that is able to overcome the short-boundary bias observed in standard pairwise random field based approaches. To wit, we show that a random field with multi-layered hidden units can encode boundary preserving higher order potentials such as the ones used in the cooperative cuts model of [11] while ...
Added: October 19, 2017
Delong A., Veksler O., Osokin A. et al., , in: Advances in Neural Information Processing Systems 26 (NIPS 2012).: Lake Tahoe: Curran Associates, Inc., 2012.
Inference in high-order graphical models has become important in recent years. Several approaches are based, for example, on generalized message-passing, or on transformation to a pairwise model with extra 'auxiliary' variables. We focus on a special case where a much more efficient transformation is possible. Instead of adding variables, we transform the original problem into ...
Added: October 18, 2017
A. G. Rassadin, A. V. Savchenko, , in: CEUR Workshop ProceedingsVol. 1901: Proceedings of the International conference Information Technology and Nanotechnology. Session Image Processing, Geoinformation Technology and Information Security.: CEUR-WS, 2017. P. 207–213.
In this paper, we consider the problem of insufficient runtime and memory space complexities of deep convolutional neural networks for visual emotion recognition. A survey of recent compression methods and efficient neural networks architectures is provided. We experimentally compare the computational speed and memory consumption during the training and the inference stages of such methods ...
Added: October 17, 2017
Kirillov A., Gavrikov M., Lobacheva E. et al., , in: Proceedings of the 27th British Machine Vision Conference.: -, 2016. P. 1–12.
The Shape Boltzmann Machine (SBM) and its multilabel version MSBM have been recently introduced as deep generative models that capture the variations of an object shape. While being more flexible MSBM requires datasets with labeled parts of the objects for training. In the paper we present an algorithm for training MSBM using binary masks of ...
Added: February 24, 2017
Novikov A., Trofimov M., Oseledets I., / Series stat :: arxiv :: Cornell University "stat :: arxiv :: Cornell University". 2017.
Modeling interactions between features improves the performance of machine learning solutions in many domains (e.g. recommender systems or sentiment analysis). In this paper, we introduce Exponential Machines (ExM), a predictor that models all interactions of every order. The key idea is to represent an exponentially large tensor of parameters in a factorized format called Tensor ...
Added: September 19, 2016
Arefiev N., / Series WP BRP "Basic research program". 2016. No. 124/EC/2016.
The literature on graphical models and the literature on identification pursue similar goals, but do not use entirely each other’s results, because represent them in different languages. To ease the communication between these fields, I translate the most important theorems on identification of linear Gaussian Simultaneous Equations Models (SEMs) and Structural Vector Autoregressions (SVARs) into ...
Added: April 14, 2016
Кириллов А. Н., Гавриков М. И., Lobacheva E. et al., Интеллектуальные системы. Теория и приложения 2015 Т. 19 № 2 С. 75–95
In this paper we consider the Shape Boltzmann Machine(SBM) and its multi-label version MSBM. We present an algorithm for training MSBM using only binary masks of objects and the seeds which approximately correspond to the locations of objects parts. ...
Added: September 30, 2015
Osokin A., Vetrov D., IEEE Transactions on Pattern Analysis and Machine Intelligence 2015 Vol. 37 No. 7 P. 1347–1359
In this paper we address the problem of finding the most probable state of a discrete Markov random field (MRF), also known as the MRF energy minimization problem. The task is known to be NP-hard in general and its practical importance motivates numerous approximate algorithms. We propose a submodular relaxation approach (SMR) based on a ...
Added: July 6, 2015
Vetrov D., Kohli P., Osokin A. et al., Lecture Notes in Computer Science 2015 Vol. 8932 P. 406–420
Structured-output learning is a challenging problem; particularly so because of the difficulty in obtaining large datasets of fully labelled instances for training. In this paper we try to overcome this difficulty by presenting a multi-utility learning framework for structured prediction that can learn from training instances with different forms of supervision. We propose a unified ...
Added: April 9, 2015
Vetrov D., Osokin A., Rodomanov A. et al., Journal of Machine Learning Research 2014
In the paper we present a new framework for dealing with probabilistic graphical models. Our approach relies on the recently proposed Tensor Train format (TT-format) of a tensor that while being compact allows for efficient application of linear algebra operations. We present a way to convert the energy of a Markov random field to the ...
Added: March 18, 2015