Frequent Itemset Mining for Clustering Near Duplicate Web Documents

P. 185–200.

Ignatov D. I., Kuznetsov S.

A vast amount of documents in the Web have duplicates, which is a challenge for developing efficient methods that would compute clusters of similar documents. In this paper we use an approach based on computing (closed) sets of attributes having large support (large extent) as clusters of similar documents. The method is tested in a series of computer experiments on large public collections of web documents and compared to other established methods and software, such as biclustering, on same datasets. Practical efficiency of different algorithms for computing frequent closed sets of attributes is compared.

Keywords: Clustering Near Duplicate, Frequent Itemset Mining, Web Documents

In book

Conceptual Structures: Leveraging Semantic Technologies. 17th International Conference on Conceptual Structures, ICCS 2009, Moscow, Russia, July 26-31, 2009, Proceedings

Vol. 5662. , Berlin, Heidelberg: Springer, 2009.