DCE-IVI: Density-based clustering ensemble by selecting internal validity index

Abstract

As each clustering algorithm cannot efficiently partition datasets with arbitrary shapes, the thought of clustering ensemble is proposed to consistently integrate clustering results to obtain better division. Most of ensemble research employs a single algorithm with different parameters to clustering. And this can be easily integrated, however it is hardly to divide complex datasets. Other available methods integrate different algorithms, it can divide datasets from different aspects, but fail to take outliers into account, which produces negative effects on the partition results. In order to solve these problems, we clustering datasets with three different density-based algorithms. The innovation of this paper is described as: (1) by setting dynamic thresholds, lower frequency evidence in the co-association matrix is gradually deleted to obtain multiple reconstructed matrices; (2) these reconstructed matrices are analyzed by hierarchical clustering to obtain basic clustering results; (3) an internal validity index is designed by the compactness within clusters and the correlation between clusters, which is used to select the final clustering result. By this innovation, the clustering effect is significantly improved. Finally, a series of experiments are designed, and the results verify the improvement and effectiveness of the proposed technique (DCE-IVI).