Xiaojia Zhao
Skip Abstract Section
Abstract
Ensemble clustering (EC), utilizing multiple basic partitions (BPs) to yield a robust consensus clustering, has shown promising clustering performance. Nevertheless, most current algorithms suffer from two challenging hurdles: (1) a surge of EC-based methods only focus on pair-wise sample correlation while fully ignoring the high-order correlations of diverse views. (2) they deal directly with the co-association (CA) matrices generated from BPs, which are inevitably corrupted by noise and thus degrade the clustering performance. To address these issues, we propose a novel Double High-Order Correlation Preserved Robust Multi-View Ensemble Clustering (DC-RMEC) method, which preserves the high-order inter-view correlation and the high-order correlation of original data simultaneously. Specifically, DC-RMEC constructs a hypergraph from BPs to fuse high-level complementary information from different algorithms and incorporates multiple CA-based representations into a low-rank tensor to discover the high-order relevance underlying CA matrices, such that double high-order correlation of multi-view features could be dexterously uncovered. Moreover, a marginalized denoiser is invoked to gain robust view-specific CA matrices. Furthermore, we develop a unified framework to jointly optimize the representation tensor and the result matrix. An effective iterative optimization algorithm is designed to optimize our DC-RMEC model by resorting to the alternating direction method of multipliers. Extensive experiments on seven real-world multi-view datasets have demonstrated the superiority of DC-RMEC compared with several state-of-the-art multi-view ensemble clustering methods.
- [1] . 2018. Multi-view low-rank sparse subspace clustering. Pattern Recognition 73 (2018), 247–258.Google Scholar
Cross Ref
- [2] . 1970. Analysis of individual differences in multidimensional scaling via an n-way generalization of “Eckart-Young” decomposition. Psychometrika 35, 3 (1970), 283–319.Google ScholarCross Ref
- [3] . 2012. Low-rank matrix recovery with structural incoherence for robust face recognition. In Proceedings of the 2012 IEEE Conference on Computer Vision and Pattern Recognition. IEEE, 2618–2625.Google ScholarDigital Library
- [4] . 2012. Marginalized denoising autoencoders for domain adaptation. International Conference on Machine Learning (2012), 767–774.Google Scholar
- [5] . 2022. Low-rank tensor graph learning for multi-view subspace clustering. IEEE Transactions on Circuits and Systems for Video Technology 32, 1 (2022), 92–104.Google ScholarDigital Library
- [6] . 2002. Finding median partitions using information-theoretical-based genetic algorithms. Journal of Universal Computer Science 8, 2 (2002), 153–172.Google Scholar
- [7] . 2013. Sparse subspace clustering: Algorithm, theory, and applications. IEEE Transactions on Pattern Analysis and Machine Intelligence 35, 11 (2013), 2765–2781.Google ScholarDigital Library
- [8] . 2018. Flexible affinity matrix learning for unsupervised and semisupervised classification. IEEE Transactions on Neural Networks and Learning Systems 30, 4 (2018), 1133–1149.Google ScholarCross Ref
- [9] . 2004. Learning generative visual models from few training examples. In Proceedings of the Workshop on Generative-Model Based Vision, IEEE Proc. CVPR, 2004.Google Scholar
- [10] . 2004. Solving cluster ensemble problems by bipartite graph partitioning. In Proceedings of the 21st International Conference on Machine Learning. 36.Google ScholarDigital Library
- [11] . 2020. A unified tensor framework for clustering and simultaneous reconstruction of incomplete imaging data. ACM Transactions on Multimedia Computing, Communications, and Applications 16, 3 (2020), 1–24.Google ScholarDigital Library
- [12] . 2014. Ensemble clustering by means of clustering embedding in vector spaces. Pattern Recognition 47, 2 (2014), 833–842.Google ScholarDigital Library
- [13] . 2005. Combining multiple clusterings using evidence accumulation. IEEE Transactions on Pattern Analysis and Machine Intelligence 27, 6 (2005), 835–850.Google ScholarDigital Library
- [14] . 2016. Ensemble clustering using factor graph. Pattern Recognition 50 (2016), 131–142.Google ScholarDigital Library
- [15] . 2015. Robust ensemble clustering using probability trajectories. IEEE Transactions on Knowledge and Data Engineering 28, 5 (2015), 1312–1326.Google ScholarDigital Library
- [16] . 2017. Locally weighted ensemble clustering. IEEE Transactions on Cybernetics 48, 5 (2017), 1460–1473.Google ScholarCross Ref
- [17] . 2023. Fast Multi-view clustering via ensembles: Towards scalability, superiority, and simplicity. IEEE Transactions on Knowledge and Data Engineering (2023).Google ScholarDigital Library
- [18] . 2018. Enhanced ensemble clustering via fast propagation of cluster-wise similarities. IEEE Transactions on Systems, Man, and Cybernetics: Systems 51, 1 (2018), 508–520.Google ScholarCross Ref
- [19] . 2011. A link-based approach to the cluster ensemble problem. IEEE Transactions on Pattern Analysis and Machine Intelligence 33, 12 (2011), 2396–2409.Google ScholarDigital Library
- [20] . 2021. Spectral ensemble clustering with LDA-based co-training for multi-view data analysis. In Proceedings of the 2021 17th International Conference on Computational Intelligence and Security (CIS). IEEE, 367–371.Google ScholarCross Ref
- [21] . 2013. Third-order tensors as operators on matrices: A theoretical and computational framework with applications in imaging. SIAM Journal on Matrix Analysis and Applications 34, 1 (2013), 148–172.Google ScholarDigital Library
- [22] . 2011. Co-regularized multi-view spectral clustering. In Proceedings of the Advances in Neural Information Processing Systems.Google Scholar
- [23] . 1993. Distortion invariant object recognition in the dynamic link architecture. IEEE Transactions on Computers 42, 3 (1993), 300–311.Google Scholar
- [24] . 2009. Spectral clustering of linear subspaces for motion segmentation. In Proceedings of the 2009 IEEE 12th International Conference on Computer Vision. IEEE, 678–685.Google ScholarCross Ref
- [25] . 2020. Ensemble learning for spectral clustering. In Proceedings of the 2020 IEEE International Conference on Data Mining (ICDM). IEEE, 1094–1099.Google ScholarCross Ref
- [26] . 2015. Large-scale multi-view spectral clustering via bipartite graph. In Proceedings of the 29th AAAI Conference on Artificial Intelligence.Google ScholarCross Ref
- [27] . 2012. Robust recovery of subspace structures by low-rank representation. IEEE Transactions on Pattern Analysis and Machine Intelligence 35, 1 (2012), 171–184.Google ScholarDigital Library
- [28] Guangcan Liu, Zhouchen Lin, and Yong Yu. 2010. Robust subspace segmentation by low-rank representation. In Proceedings of the 27th International Conference on Machine Learning. 8.Google Scholar
- [29] . 2015. Clustering with partition level side information. In Proceedings of the 2015 IEEE International Conference on Data Mining. IEEE, 877–882.Google ScholarDigital Library
- [30] . 2018. Consensus guided multi-view clustering. ACM Transactions on Knowledge Discovery from Data 12, 4 (2018), 1–21.Google ScholarDigital Library
- [31] . 2015. Spectral ensemble clustering. In Proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. 715–724.Google ScholarDigital Library
- [32] . 2016. Infinite ensemble for image clustering. In Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. 1745–1754.Google ScholarDigital Library
- [33] . 2018. Infinite ensemble clustering. Data Mining and Knowledge Discovery 32, 2 (2018), 385–416.Google ScholarDigital Library
- [34] . 2017. Spectral ensemble clustering via weighted k-means: Theoretical and practical evidence. IEEE Transactions on Knowledge and Data Engineering 29, 5 (2017), 1129–1143.Google ScholarDigital Library
- [35] . 2001. On spectral clustering: Analysis and an algorithm. In Proceedings of the Advances in Neural Information Processing Systems.Google Scholar
- [36] Feiping Nie, Jing Li, and Xuelong Li. 2016. Parameter-free auto-weighted multiple graph learning: A framework for multiview clustering and semi-supervised classification. In Proceedings of the International Joint Conference on Artificial Intelligence.1881–1887.Google Scholar
- [37] . 2012. The TFC model: Tensor factorization and tag clustering for item recommendation in social tagging systems. IEEE Transactions on Systems, Man, and Cybernetics: Systems 43, 3 (2012), 673–688.Google ScholarCross Ref
- [38] . 2009. Motion segmentation in the presence of outlying, incomplete, or corrupted trajectories. IEEE Transactions on Pattern Analysis and Machine Intelligence 32, 10 (2009), 1832–1845.Google ScholarDigital Library
- [39] . 2002. Cluster ensembles—a knowledge reuse framework for combining multiple partitions. Journal of Machine Learning Research 3, Dec (2002), 583–617.Google Scholar
- [40] . 2021. Scalable multi-view subspace clustering with unified anchors. In Proceedings of the ACM International Conference on Multimedia. 3528–3536.Google ScholarDigital Library
- [41] . 2015. ClustHOSVD: Item recommendation by combining semantically enhanced tag clustering with tensor HOSVD. IEEE Transactions on Systems, Man, and Cybernetics: Systems 46, 9 (2015), 1240–1251.Google ScholarCross Ref
- [42] . 2021. From ensemble clustering to subspace clustering: Cluster structure encoding. IEEE Transactions on Neural Networks and Learning Systems 34, 5 (2021), 2670–2681.Google Scholar
- [43] . 2019. Adversarial graph embedding for ensemble clustering. In Proceedings of the International Joint Conferences on Artificial Intelligence Organization.Google ScholarCross Ref
- [44] . 2017. From ensemble clustering to multi-view clustering. In Proceedings of the International Joint Conference on Artificial Intelligence.Google ScholarCross Ref
- [45] . 2019. Marginalized multiview ensemble clustering. IEEE Transactions on Neural Networks and Learning Systems 31, 2 (2019), 600–611.Google ScholarCross Ref
- [46] . 2019. Robust spectral ensemble clustering via rank minimization. ACM Transactions on Knowledge Discovery from Data 13, 1 (2019), 1–25.Google ScholarDigital Library
- [47] . 2016. Robust spectral ensemble clustering. In Proceedings of the 25th ACM International Conference on Information and Knowledge Management. 367–376.Google ScholarDigital Library
- [48] . 1966. Some mathematical notes on three-mode factor analysis. Psychometrika 31, 3 (1966), 279–311.Google ScholarCross Ref
- [49] . 2010. Stacked denoising autoencoders: Learning useful representations in a deep network with a local denoising criterion. Journal of Machine Learning Research 11, 12 (2010), 3371–3408.Google ScholarDigital Library
- [50] . 2019. GMC: Graph-based multi-view clustering. IEEE Transactions on Knowledge and Data Engineering 32, 6 (2019), 1116–1129.Google ScholarCross Ref
- [51] . 2021. Fast parameter-free multi-view subspace clustering with consensus anchor guidance. IEEE Transactions on Image Processing 31 (2021), 556–568.Google ScholarDigital Library
- [52] . 2009. An HOG-LBP human detector with partial occlusion handling. In Proceedings of the 2009 IEEE 12th International Conference on Computer Vision. IEEE, 32–39.Google ScholarCross Ref
- [53] . 2016. Shifting multi-hypergraphs via collaborative probabilistic voting. Knowledge and Information Systems 46, 3 (2016), 515–536.Google ScholarDigital Library
- [54] . 2016. Iterative views agreement: An iterative low-rank based structured optimization method to multi-view spectral clustering. In Proceedings of the International Joint Conference on Artificial Intelligence 2016. Association for the Advancement of Artificial Intelligence (AAAI), 2153–2159.Google Scholar
- [55] . 2018. Beyond low-rank representations: Orthogonal clustering basis reconstruction with optimized graph structure for multi-view spectral clustering. Neural Networks 103 (2018), 1–8.Google ScholarDigital Library
- [56] . 2018. Multi-view spectral clustering via structured low-rank matrix factorization. IEEE Transactions on Neural Networks and Learning Systems 29, 10 (2018), 4833–4843.Google ScholarCross Ref
- [57] . 2014. K-means-based consensus clustering: A unified view. IEEE Transactions on Knowledge and Data Engineering 27, 1 (2014), 155–169.Google ScholarCross Ref
- [58] . 2009. Adapting the right measures for k-means clustering. In Proceedings of the 15th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. 877–886.Google ScholarDigital Library
- [59] . 2014. Robust multi-view spectral clustering via low-rank and sparse decomposition. In Proceedings of the AAAI Conference on Artificial Intelligence.Google ScholarCross Ref
- [60] . 2018. On unifying multi-view self-representations for clustering by tensor multi-rank minimization. International Journal of Computer Vision 126, 11 (2018), 1157–1179.Google ScholarDigital Library
- [61] . 2019. Discovering latent topics by Gaussian latent Dirichlet allocation and spectral clustering. ACM Transactions on Multimedia Computing, Communications, and Applications 15, 1 (2019), 1–18.Google ScholarDigital Library
- [62] . 2019. Robust subspace discovery by block-diagonal adaptive locality-constrained representation. In Proceedings of the 27th ACM International Conference on Multimedia. 1569–1577.Google ScholarDigital Library
- [63] . 2006. Learning with hypergraphs: Clustering, classification, and embedding. In Proceedings of the Advances in Neural Information Processing Systems.Google Scholar
- [64] . 2015. Learning a robust consensus matrix for clustering ensemble via kullback-Leibler divergence minimization. In Proceedings of the 24th International Joint Conference on Artificial Intelligence.Google ScholarDigital Library
Index Terms
- Double High-Order Correlation Preserved Robust Multi-View Ensemble Clustering
Recommendations
Multi-view Ensemble Clustering via Low-rank and Sparse Decomposition: From Matrix to Tensor
As a significant extension of classical clustering methods, ensemble clustering first generates multiple basic clusterings and then fuses them into one consensus partition by solving a problem concerning graph partition with respect to the co-association ...
Multi-view clustering with dual tensors
AbstractMulti-view clustering methods based on tensor have achieved favorable performance thanks to the powerful capacity of capturing the high-order correlation hidden in multi-view data. However, many existing works only pay attention to exploring the ...
NOODLE: Joint Cross-View Discrepancy Discovery and High-Order Correlation Detection for Multi-View Subspace Clustering
Benefiting from the effective exploration of the valuable topological pair-wise relationship of data points across multiple views, multi-view subspace clustering (MVSC) has received increasing attention in recent years. However, we observe that existing ...
Comments