Multimodal Clustering of Boolean Tensors on MapReduce: Experiments Revisited

Ignatov Dmitry I.; Egurnov Dmitrii; Точилкин Дмитрий Сергеевич

Book chapter

Multimodal Clustering of Boolean Tensors on MapReduce: Experiments Revisited

P. 137-151.

Ignatov D. I., Egurnov D., Точилкин Д. С.

This paper presents further development of distributed multimodal clustering. We introduce a new version of multimodal clustering algorithm for distributed processing in Apache Hadoop on computer clusters. Its implementation allows a user to conduct clustering on data with modality greater than two. We provide time and space complexity of the algorithm and justify its relevance. The algorithm is adapted for MapReduce distributed processing model. The program implemented by means of Apache Hadoop framework is able to perform parallel computing on thousands of nodes.

Language: English

Full text (PDF, 479 Kb)

Text on another site

Keywords: FCA (Formal Concept Analysis)Triadic Formal Concept Analysis Map Reduce big data Boolean Tensors Data Mining

Publication based on the results of: Discovering and Representing Knowledge for Recommender Systems(2019)

In book

Supplementary Proceedings ICFCA 2019 Conference and Workshops

Edited by: F. Le Ber, D. Cristea, R. Missaoui et al. Vol. 2378. CEUR Workshop Proceedings, 2019.

Efficient Mining of Subsample-Stable Graph Patterns

Buzmakov A. V., Kuznetsov S., Napoli A. In bk.: 2017 IEEE 17th International Conference on Data Mining (ICDM). New Orleans: IEEE, 2017. Ch. 89. P. 757-762.

A scalable method for mining graph patterns stable under subsampling is proposed. The existing subsample stability and robustness measures are not antimonotonic according to definitions known so far. We study a broader notion of antimonotonicity for graph patterns, so that measures of subsample stability become antimonotonic. Then we propose gSOFIA for mining the most subsample-stable graph patterns. The experiments on numerous graph datasets show that gSOFIA is very efficient for discovering subsample-stable graph patterns.

Added: Sep 26, 2017

Triclustering in Big Data Setting

Egurnov D., Ignatov D. I., Точилкин Д. С. Lecture Notes in Computer Science. LNCS. Springer, 2020

In this paper, we describe versions of triclustering algorithms adapted for efficient calculations in distributed environments with MapReduce model or parallelisation mechanism provided by modern programming languages. OAC-family of triclustering algorithms shows good parallelisation capabilities due to the independent processing of triples of a triadic formal context. We provide the time and space complexity of the algorithms and justify their relevance. We also compare performance gain from using a distributed system and scalability.

Added: Nov 10, 2020

Mapping the Radical Innovations in Food Industry: A Text Mining Study

Kuzminov I., Bakhtin P. D., Khabirova E. et al. Science, Technology and Innovation. WP BRP. Высшая школа экономики, 2018. No. WP BRP/80/STI/2018.

The article presents the results of the study of radical innovations in the global food industry which were obtained through semantic analysis of heterogeneous unstructured text data sources by applying innovative big data text mining system. The approach used allows performing rapid, yet comprehensive aggregation of the whole polyphony of existing knowledge of the technology development in any sector for traditional foresight, future oriented technology analysis, and horizon scanning studies. The sources for the analysis include research papers, patent applications with both full-text data and additional structured metadata, analytical reports by main international organizations and national key players, various media and news resources, including all the major technology innovation, disruption and venture capital news websites. Their processing with an introduced approach for trend- and technology-mapping helps to identify ongoing and emerging technology-related trends, weak signals on possible scientific breakthroughs in the global food industry, including most promising startup strategies and food innovation controversies. This kind of analysis can be performed on a regular basis owing to constant accumulation of textual data and serve as a framework for constant science and technology (S&T) monitoring for early warning on changing technology landscape and its implications on agriculture and food markets.

Added: Mar 20, 2018

Formal Concept Analysis Meets Information Retrieval 2013

Edited by: C. Carpineto, A. Napoli, S. Kuznetsov. Vol. 977. Aachen: CEUR Workshop Proceedings, 2013.

Formal Concept Analysis (FCA) is a mathematically well-founded theory aimed at data analysis and classication, introduced and detailed in the book of Bernhard Ganter and Rudolf Wille, \Formal Concept Analysis", Springer 1999. The area came into being in the early 1980s and has since then spawned over 10000 scientic publications and a variety of practically deployed tools. FCA allows one to build from a data table with objects in rows and attributes in columns a taxonomic data structure called concept lattice, which can be used for many purposes, especially for Knowledge Discovery and Information Retrieval. The \Formal Concept Analysis Meets Information Retrieval" (FCAIR) workshop collocated with the 35th European Conference on Information Retrieval (ECIR 2013) was intended, on the one hand, to attract researchers from FCA community to a broad discussion of FCA-based research on information retrieval, and, on the other hand, to promote ideas, models, and methods of FCA in the community of Information Retrieval. This volume contains 11 contributions to FCAIR workshop (including 3 abstracts for invited talks and tutorial) held in Moscow, on March 24, 2013. All submissions were assessed by at least two reviewers from the program committee of the workshop to which we express our gratitude. We would also like to thank the co-organizers and sponsors of the FCAIR workshop: Russian Foundation for Basic Research, National Research University Higher School of Economics, and Yandex.

Added: Oct 10, 2013

Proc. 11th International Conference on Formal Concept Analysis (ICFCA 2013)

Edited by: P. Cellier, F. Distel, B. Ganter. Vol. 7880. Springer, 2013.

Added: Jun 2, 2013

Метод анализа многомерных временных рядов с использованием корректировки предварительно рассчитанной обратной матрицы: исследование в сравнении с другими методами Data Mining

Перминов Г. И. Бизнес-информатика. 2008. № 1. С. 36-44.

Added: Sep 28, 2012

Proceedings of the 11th IEEE International Conference “Application of Information and Communication Technologies” (AICT-2017)

Vol. 1, 2. M.: 2017.

Added: Oct 27, 2017

Data vs. Big Data: metodic experiments and research ethics in project "Index of Digital Literacy"

Olga Logunova. The Russian Journal of Communication. 2017. Vol. 9. No. 3. P. 303-304.

In 2015-2016 the Department of Communication, Media and Design of the National Research University “Higher School of Economics” in collaboration with non-profit organization ROCIT conducted research aimed to construct the Index of Digital Literacy in Russian Regions. This research was the priority and remain unmatched for the momentIn 2015-2016 the Department of Communication, Media and Design of the National Research University “Higher School of Economics” in collaboration with non-profit organization ROCIT conducted research aimed to construct the Index of Digital Literacy in Russian Regions. This research was the priority and remain unmatched for the moment

Added: Oct 14, 2017

Информация на стол руководителя

Дорофеев А. Н. Грузовое и пассажирское автохозяйство. 2009. № 4. С. 11-16.

The article examines the approaches of OLAP-applications for business analysis trucking company. Examples of using multi-dimensional tables to support decision-making.

Added: Jan 12, 2013

Big Data и юриспруденция: на одном ли мы пути?

Чураков В. Д. В кн.: Право и информация: вопросы теории и практики: Сборник материалов международной научно-практической конференции. Вып. 7. СПб.: Президентская библиотека имени Б.Н. Ельцина, 2018. С. 136-143.

The article is dedicated to the analysis of Big Data perspective in jurisprudence. It is proved that Big Data have to be used as the explanatory and predictable tool. The author describes issues concerning Big Data application in legal research. The problems are technical (data access, technical imperfections, data verification) and informative (interpretation of data and correlations). It is concluded that there is the necessity to enhance Big Data investigations taking into account the abovementioned limits.

Added: Oct 22, 2018