?
Community Embeddings with Bayesian Gaussian Mixture Model and Variational Inference
P. 88–96.
Anton I.N. Begehr, Peter B. Panfilov
Graphs, such as social networks, emerge naturally from various real-world situations. Recently, graph embedding methods have gained traction in data science research. The graph and community embedding algorithm ComE aims to preserve first-, second- and higher-order proximity. ComE requires prior knowledge of the number of communities K. In this paper, ComE is extended to utilize a Bayesian Gaussian mixture model with variational inference for learning community embeddings (ComE BGMM+VI), similar to ComE+. ComE BGMM+VI takes K as the maximum number of communities and drops components through the trade-off hyperparameter weight concentration prior. The advantage of ComE BGMM+VI over the non-Bayesian ComE for an unknown number of communities K is shown for the small Karate club dataset and explored for the larger DBLP dataset.
In book
Vol. 2: CBI Forum and Workshop Papers. , IEEE, 2022.
Shalileh S., , in: Data Analysis and Optimization. In Honor of Boris Mirkin's 80th Birthday.: Springer, 2023. P. 349–361.
Based upon variational inference (VI) a new set of classification algorithms has recently emerged. This set of algorithms aims (A) to increase generalization power, (B) to decrease computational complexity. However, the complex math and implementation considerations have led to the emergence of black-box variational inference methods (BBVI). Relying on these principles, we assume the existence ...
Added: October 17, 2023
Giordano M., Maddalena L., Manzo M. et al., Annals of Mathematics and Artificial Intelligence 2023 Vol. 91 P. 259–285
As the number of graph-level embedding techniques increases at an unprecedented speed, questions arise about their behavior and performance when training data undergo perturbations. This is the case when an external entity maliciously alters training data to invalidate the embedding. This paper explores the efects of such attacks on some graph datasets by applying diferent ...
Added: November 18, 2022
Makarov I., Oborevich A., , in: Proceedings of IEEE 21st International Symposium on Computational Intelligence and Informatics (CINTI'21), 18-20 Nov. 2021.: NY: IEEE, 2021. P. 000127–000130.
Graph visualization is an effective and efficient way to discover complex inter-connections between elements within the nested structure of data. To accomplish this type of representation machine learning algorithms use a technique called graph embedding and node embedding in particular. However, in this paper, we will compare well-known techniques to yet largely under-explored setting of ...
Added: January 19, 2022
Gerasimova O., Makarov I., Лапидус А. А., , in: Analysis of Images, Social Networks and Texts. 10th International Conference, AIST 2021, Tbilisi, Georgia, December 16–18, 2021, Revised Selected Papers.: Cham: Springer, 2022. P. 1–14.
The work is devoted to academic papers recommendation task considered as link prediction on a static citation network. We compare several graph embeddings, text-based and fusion models in the link prediction problem on academic papers citation dataset. We showed that fusion models of graph and text information outperform other approaches based on graph or text information alone. We prove ...
Added: December 3, 2021
Dubnov Y. A., Bulychev A., Информационные технологии и вычислительные системы 2020 № 1 С. 25–33
The article considers the problem of separating a mixture of Gaussian components, which consists in determining, from available observations, the parameters of the mixture components. An approach to solving this problem is proposed, based on Bayesian estimation using the most informative prior distributions (Maximal Data Information Prior - MDIP). The novelty of the described approach ...
Added: October 30, 2020
Sobolev A., Vetrov D., , in: Advances in Neural Information Processing Systems 32 (NeurIPS 2019).: [б.и.], 2019. P. 1–13.
Variational Inference is a powerful tool in the Bayesian modeling toolkit, however, its effectiveness is determined by the expressivity of the utilized variational distributions in terms of their ability to match the true posterior distribution. In turn, the expressivity of the variational family is largely limited by the requirement of having a tractable density function. ...
Added: March 13, 2020
Vetrov D., Izmailov P., Maddox W. et al., , in: Proceedings of the 35th Uncertainty in Artificial Intelligence Conference (UAI-2019).: [б.и.], 2019. P. 1–11.
Bayesian inference was once a gold standard for learning with neural networks, providing accurate full predictive distributions and well calibrated uncertainty. However, scaling Bayesian inference techniques to deep neural networks is challenging due to the high dimensionality of the parameter space. In this paper, we construct low-dimensional subspaces of parameter space, such as the first ...
Added: January 14, 2020
Molchanov D., Kharitonov V., Sobolev A. et al., , in: Proceedings of Machine Learning Research, Volume 89: The 22nd International Conference on Artificial Intelligence and Statistics (AISTATS 2019).: PMLR, 2019. P. 2593–2602.
We extend the existing framework of semi-implicit variational inference (SIVI) and introduce doubly semi-implicit variational inference (DSIVI), a way to perform variational inference and learning when both the approximate posterior and the prior distribution are semi-implicit. In other words, DSIVI performs inference in models where the prior and the posterior can be expressed as an ...
Added: November 20, 2019
Козачок А. В., Копылов С. А., Шелупанов А. А. et al., Journal of Computer Virology and Hacking Techniques 2019 Vol. 15 No. 3 P. 219–232
Printed documents protection problem against leakage is still one of the relevant. Existing security tools allow us to protect electronic text documents, however are ineffective in protecting their printed versions. This research presents the marking approach for text electronic documents invariant to the print-and-scan transformation. During marker embedding source text line spacing values are changing ...
Added: September 3, 2019
Makarov I., Gerasimova O., Sulimov P. et al., PeerJ Computer Science 2019 P. 1–20
We present a study on co-authorship network representation based on network embedding together with additional information on topic modeling of research papers and new edge embedding operator. We use the link prediction (LP) model for constructing a recommender system for searching collaborators with similar research interests. Extracting topics for each paper, we construct keywords co-occurrence ...
Added: January 21, 2019
Izmailov P., Novikov A., Kropotov D., , in: Proceedings of Machine Learning Research. Proceedings of The International Conference on Artificial Intelligence and Statistics (AISTATS 2018).: [б.и.], 2018. P. 726–735.
We propose a method (TT-GP) for approximate inference in Gaussian Process (GP) models. We build on previous scalable GP research including stochastic variational inference based on inducing inputs, kernel interpolation, and structure exploiting algebra. The key idea of our method is to use Tensor Train decomposition for variational parameters, which allows us to train GPs ...
Added: December 10, 2018
Izmailov P., Kropotov D., Journal of machine learning and data analysis 2017 Vol. 3 No. 1 P. 20–35
Background: Gaussian processes (GP) provide an elegant and effective approach to learning in kernel machines. This approach leads to a highly interpretable model and allows using the Bayesian framework for model adaptation and incorporating the prior knowledge about the problem. The GP framework is successfully applied to regression, classification, and dimensionality reduction problems. Unfortunately, the ...
Added: December 6, 2018
Makarov I., Gerasimova O., Sulimov P. et al., , in: Proceedings of Analysis of Images, Social Networks and Texts – 7th International Conference, AIST 2018, Moscow, Russia, July 5-7, 2018, Revised Selected Papers. Lecture Notes in Computer ScienceVol. 11179.: Berlin: Springer, 2018. P. 20–31.
In this paper, we consider new formulation of graph embedding algorithm, while learning node and edge representation under common constraints. We evaluate our approach on link prediction problem for co-authorship network of HSE researchers’ publications. We compare it with existing structural network embeddings and feature-engineering models. ...
Added: September 5, 2018
Makarov I., Gerasimova O., Sulimov P. et al., , in: Proceedings of Analysis of Images, Social Networks and Texts – 7th International Conference, AIST 2018, Moscow, Russia, July 5-7, 2018, Revised Selected Papers. Lecture Notes in Computer ScienceVol. 11179.: Berlin: Springer, 2018. P. 32–38.
Co-authorship networks contain invisible patterns of collaboration among researchers. The process of writing joint paper can depend of different factors, such as friendship, common interests, and policy of university. We show that, having a temporal co-authorship network, it is possible to predict future publications. We solve the problem of recommending collaborators from the point of ...
Added: September 5, 2018
Figurnov M., Struminsky K., Vetrov D., / Series arXiv:1611.09226 "arxiv.org". 2016.
Variational inference is a powerful tool for approximate inference. However, it mainly focuses on the evidence lower bound as variational objective and the development of other measures for variational inference is a promising area of research. This paper proposes a robust modification of evidence and a lower bound for the evidence, which is applicable when ...
Added: November 30, 2016
Bartunov S., Vetrov D., , in: JMLR Workshop and Conference ProceedingsIssue 32: Proceedings of The 31st International Conference on Machine Learning.: Beijing: Microtome Publishing, 2014. P. 1404–1412.
Recently proposed distance dependent Chinese Restaurant Process (ddCRP) generalizes extensively used Chinese Restaurant Process (CRP) by accounting for dependencies between data points. Its posterior is intractable and so far only MCMC methods were used for inference. Because of very different nature of ddCRP no prior developments in variational methods for Bayesian nonparametrics are appliable. In ...
Added: March 4, 2015
Bartunov S. O., Vetrov D., Journal of Machine Learning Research 2014 Vol. 32 No. 1 P. 1404–1412
Recently proposed distance dependent Chinese Restaurant Process (ddCRP) generalizes extensively used Chinese Restaurant Process (CRP) by accounting for dependencies between data points. Its posterior is intractable and so far only MCMC methods were used for inference. Because of very different nature of ddCRP no prior developments in variational methods for Bayesian nonparametrics are appliable. In ...
Added: July 9, 2014