Denoising Cluster Analysis

Abstract

Clustering or cluster analysis is an important and common task in data mining and analysis, with applications in many fields. However, most existing clustering methods are sensitive in the presence of limited amounts of data per cluster in real-world applications. Here we propose a new method called denoising cluster analysis to improve the accuracy. We first construct base clusterings with artificially corrupted data samples and later learn their ensemble based on mutual information. We develop multiplicative updates for learning the aggregated cluster assignment probabilities. Experiments on real-world data sets show that our method unequivocally improves cluster purity over several other clustering approaches.

Appendix: Proof of Theorem 1

Proof

We use W for current estimate, $\widetilde{W}$ for variable, and $W^\text {new}$ for the new estimate, respectively. The objective function $\widetilde{\mathcal {J}}$ fulfills the theorem conditions in [16]. Therefore, we can construct the majorization function

$$\begin{aligned} \nonumber G(\widetilde{W},W)=&\sum _{ik}\left[ \mathcal {\nabla }^+_{ik}\widetilde{W}_{ik}-\mathcal {\nabla }^-_{ik}W_{ik}\log \widetilde{W}_{ik}+\frac{B_{ik}}{A_{ik}}W_{ik}-\frac{W_{ik}}{A_{ik}}\log \widetilde{W}_{ik}\right] +\text {constant}\end{aligned}$$

such that $G(\widetilde{W},W)\ge \widetilde{\mathcal {J}}(\widetilde{W},\lambda )$ and $G(W,W)=\widetilde{\mathcal {J}}(W,\lambda )$. Let $W^\text {new}$ be the minimum of $G(\widetilde{W},W)$, which is implemented by zeroing $\partial G/\partial \widetilde{W}$ and yields Eq. 12. Therefore $\widetilde{\mathcal {J}}(W^\text {new},\lambda ) \le G(W^\text {new},W) \le G(W,W) =\widetilde{\mathcal {J}}(W,\lambda )$.