Frequent Itemset Mining for Clustering Near Duplicate Web Documents

D. I. Ignatov; S. Kuznetsov

АБВ
АБВ
АБВ

Обычная версия сайта

Priority areas

by year

Subject

News

August 13, 2026

‘Working with AI Solves a Wide Range of Engineering Problems

Artificial intelligence is a working tool based on a balanced combination of algorithms and engineering. Experts and doctoral students from the HSE Moscow Institute of Electronics and Mathematics explain how AI technologies can improve an application, device, or system, and what engineering tasks are solved in the process.

August 12, 2026

‘I Would Like My Research to Help Make the World a Calmer and Better Place

Whatever task Saraa Ali, Junior Research Fellow at the Laboratory of Methods for Big Data Analysis (LAMBDA) of the AI and Digital Science Institute (HSE Faculty of Computer Science), is working on, she thinks about how it can benefit people. She told the Young Scientists of HSE University project about her large family, diagnosing three-phase motors, and her dream of building a children’s home in her native country.

August 11, 2026

‘The Peak of Stupidity and ‘The Valley of Despair: HSE Economists Propose an Explanation for the Dunning–Kruger Effect

The Dunning–Kruger effect, which describes a sharp surge in self-confidence among beginners followed by an equally rapid decline as they gain experience, can be explained by the nature of the learning process and the acquisition of new knowledge. This conclusion was reached by Andrey Vorchik of the HSE Faculty of Economic Sciences together with independent researcher Murat Mamyshev. They developed a mathematical model of learning and demonstrated how subjective confidence is formed and changes as knowledge accumulates, as well as how teachers can reduce the ‘valley of despair’ experienced by learners.

Have you spotted a typo?
Highlight it, click Ctrl+Enter and send us a message. Thank you for your help!

Publications

?

Frequent Itemset Mining for Clustering Near Duplicate Web Documents

P. 185–200.

Ignatov D. I., Kuznetsov S.

A vast amount of documents in the Web have duplicates, which is a challenge for developing efficient methods that would compute clusters of similar documents. In this paper we use an approach based on computing (closed) sets of attributes having large support (large extent) as clusters of similar documents. The method is tested in a series of computer experiments on large public collections of web documents and compared to other established methods and software, such as biclustering, on same datasets. Practical efficiency of different algorithms for computing frequent closed sets of attributes is compared.

Language: English

Full text

Keywords: Clustering Near Duplicate Frequent Itemset Mining Web Documents

In book

Conceptual Structures: Leveraging Semantic Technologies. 17th International Conference on Conceptual Structures, ICCS 2009, Moscow, Russia, July 26-31, 2009, Proceedings

Vol. 5662. , Berlin, Heidelberg: Springer, 2009.

Near-Duplicate Detection for Online-Shops Owners: An FCA-Based Approach

Ignatov D. I., Chubis Y., Konstantinov A. V., Lecture Notes in Computer Science 2013 Vol. 7814 P. 722–725

We proposed a prototype of near-duplicate detection system for web-shop owners. It’s a typical situation for this online businesses to buy description of their goods from so-called copyrighters. Copyrighter can cheat from time to time and provide the owner with some almost identical descriptions for different items. In this paper we demonstrated how we can ...

Added: October 10, 2013