Skip to main content
SpringerLink
Log in

A multiple kinds of information extraction method for multi-view low-rank subspace clustering

  • Original Article
  • Published:
International Journal of Machine Learning and Cybernetics Aims and scope Submit manuscript

Abstract

Recently, multi-view subspace clustering has attracted intensive attentions due to the remarkable clustering performance by extracting abundant complementary information from multi-view data, making its clustering performance much better than that of single view data. However, at present, multi-view subspace clustering methods develop either a shared consistency representation that models the common properties from all views, or a series of specificity representations each of which mines the intrinsic difference in each view, or global spatial structure of all features, or local geometric structure of multiple features. More seriously, only one kind of information is extracted and utilized in some research work. In this paper, to cope with the issue, we present a multiple kinds of information extraction method (MKIE) for multi-view subspace clustering, which combines the consistency and specificity regularizations with the graph regularizations. We construct some graph structures for the shared consistency representation and all the specificity representations, which model the local geometric structures of multiple features. The model of MKIE makes full use of four kinds of valid information: global spatial structure information, local geometric structure information, consistent feature information and specific feature information. In addition, we design an effective optimization algorithm based on Alternating Direction Method of Multipliers. Extensive experiments performed on four benchmark multi-view datasets validate the effectiveness of MKIE which is compared with ten state-of-the-art multi-view clustering methods.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4

Similar content being viewed by others

Data availability

The BBC-SPORTS dataset [78] is available in ML Resources whose URL is http://mlg.ucd.ie/datasets/segment.html. The ALOI dataset [79] is available in ELKI Data Mining whose URL is http://elki.dbs.ifi.lmu.de/wiki/DataSets/MultiView. The Prokaryotic-phyla dataset [80] is available in ProTraits whose URL is http://protraits.irb.hr/. The Caltech101 dataset [81] is available in Caltech Vision Lab whose URL is http://www.vision.caltech.edu/datasets/.

Notes

  1. http://mlg.ucd.ie/datasets/segment.html.

  2. http://elki.dbs.ifi.lmu.de/wiki/DataSets/MultiView.

  3. http://protraits.irb.hr/.

  4. http://www.vision.caltech.edu/datasets/.

References

  1. Mirkin B (2005) Clustering for data mining: a data recovery approach. Chapman & Hall/CRC, Los Angeles

  2. Fu L, Lin P, Vasilakos AV et al (2020) An overview of recent multi-view clustering. Neurocomputing 402:148–161

    Google Scholar 

  3. Xie Y, Lin B, Qu Y et al (2020) Joint deep multi-view learning for image clustering. IEEE Trans Knowl Data Eng 33(11):3594–3606

    Google Scholar 

  4. Zhao J (2022) A novel low-rank matrix approximation algorithm for face denoising and background/foreground separation. Comput Appl Math 41(4):1–38

    MathSciNet  Google Scholar 

  5. Bertsimas D, Orfanoudaki A, Wiberg H (2021) Interpretable clustering: an optimization approach. Mach Learn 110(1):89–138

    MathSciNet  Google Scholar 

  6. Ghadiri S, Mazlumi K (2020) Adaptive protection scheme for microgrids based on SOM clustering technique. Appl Soft Comput 88:106062

    Google Scholar 

  7. Kouhi A, Seyedarabi H, Aghagolzadeh A (2020) Robust FCM clustering algorithm with combined spatial constraint and membership matrix local information for brain MRI segmentation. Expert Syst Appl 146:113159

    Google Scholar 

  8. Lu C, Feng J, Lin Z et al (2018) Subspace clustering by block diagonal representation. IEEE Trans Pattern Anal Mach Intell 41(2):487–501

    PubMed  Google Scholar 

  9. Zhao J, Zhao L (2020) Low-rank and sparse matrices fitting algorithm for low-rank representation. Comput Math Appl 79(2):407–425

    MathSciNet  Google Scholar 

  10. Baker Y, Tang T, Allen G (2020) Feature selection for data integration with mixed multiview data. Annals of Applied Statistics 14(4):1676–1698

    MathSciNet  Google Scholar 

  11. Liu X, Li M, Tang C et al (2020) Efficient and effective regularized incomplete multi-view clustering. IEEE Trans Pattern Anal Mach Intell 43(8):2634–2646

    Google Scholar 

  12. Li R, Zhang C, Fu H, et al (2019) Reciprocal multi-layer subspace learning for multi-view clustering. Proceedings of the IEEE/CVF International Conference on Computer Vision 8172–8180

  13. Yang Z, Xu Q, Zhang W et al (2019) Split multiplicative multi-view subspace clustering. IEEE Trans Image Process 28(10):5147–5160

    ADS  MathSciNet  Google Scholar 

  14. Zhao J, Feng Q, Zhao L (2019) Alternating direction and Taylor expansion minimization algorithms for unconstrained nuclear norm optimization. Numerical Algorithms 82(1):371–396

    MathSciNet  Google Scholar 

  15. Zhang X, Sun H, Liu Z et al (2019) Robust low-rank kernel multi-view subspace clustering based on the Schatten p-norm and correntropy. Inf Sci 477:430–447

    Google Scholar 

  16. Peng X, Huang Z, Lv J, et al (2019) COMIC: multi-view clustering without parameter selection. International Conference on Machine Learning 5092–5101

  17. Huang S, Kang Z, Tsang I et al (2019) Auto-weighted multi-view clustering via kernelized graph learning. Pattern Recogn 88:174–184

    ADS  Google Scholar 

  18. Yu H, Wang X, Wang G et al (2020) An active three-way clustering method via low-rank matrices for multi-view data. Inf Sci 507:823–839

    Google Scholar 

  19. Yin M, Gao J, Xie S et al (2018) Multiview subspace clustering via tensorial t-product representation. IEEE Transactions on Neural Networks and Learning Systems 30(3):851–864

    MathSciNet  PubMed  Google Scholar 

  20. Chen Y, Xiao X, Zhou Y (2020) Multi-view subspace clustering via simultaneously learning the representation tensor and affinity matrix. Pattern Recogn 106:107441

    Google Scholar 

  21. Gao Q, Xia W, Wan Z et al (2020) Tensor-SVD based graph learning for multi-view subspace clustering. Proceedings of the AAAI Conference on Artificial Intelligence 34(04):3930–3937

    Google Scholar 

  22. Zhu X, Zhang S, He W et al (2018) One-step multi-view spectral clustering. IEEE Trans Knowl Data Eng 31(10):2022–2034

    Google Scholar 

  23. Wang Y, Wu L, Lin X et al (2018) Multiview spectral clustering via structured low-rank matrix factorization. IEEE Transactions on Neural Networks and Learning Systems 29(10):4833–4843

    PubMed  Google Scholar 

  24. Wang Y, Wu L (2018) Beyond low-rank representations: Orthogonal clustering basis reconstruction with optimized graph structure for multi-view spectral clustering. Neural Netw 103:1–8

    PubMed  Google Scholar 

  25. Sharma K, Seal A (2021) Multi-view spectral clustering for uncertain objects. Inf Sci 547:723–745

    MathSciNet  Google Scholar 

  26. Wang H, Yang Y, Liu B et al (2019) A study of graph-based system for multi-view clustering. Knowl-Based Syst 163:1009–1019

    Google Scholar 

  27. Zhang C, Fu H, Hu Q et al (2018) Generalized latent multi-view subspace clustering. IEEE Trans Pattern Anal Mach Intell 42(1):86–99

    PubMed  Google Scholar 

  28. Brbić M, Kopriva I (2018) Multi-view low-rank sparse subspace clustering. Pattern Recogn 73:247–258

    ADS  Google Scholar 

  29. Luo S, Zhang C, Zhang W et al (2018) Consistent and specific multi-view subspace clustering. Proceedings of the AAAI Conference on Artificial Intelligence 32(1):3110–3126

    Google Scholar 

  30. Zhang G, Zhou Y, He X et al (2020) One-step kernel multi-view subspace clustering. Knowl-Based Syst 189:105126

    Google Scholar 

  31. Zheng Q, Zhu J, Li Z et al (2020) Feature concatenation multi-view subspace clustering. Neurocomputing 379:89–102

    Google Scholar 

  32. Wang H, Yang Y, Liu B (2019) GMC: Graph-based multi-view clustering. IEEE Trans Knowl Data Eng 32(6):1116–1129

    Google Scholar 

  33. Zhao L, Chen Z, Yang Y et al (2018) Incomplete multi-view clustering via deep semantic mapping. Neurocomputing 275:1053–1062

    Google Scholar 

  34. Jing P, Su Y, Li Z et al (2021) Learning robust affinity graph representation for multi-view clustering. Inf Sci 544:155–167

    MathSciNet  Google Scholar 

  35. Wu J, Xie X, Nie L et al (2020) Unified graph and low-rank tensor learning for multi-view clustering. Proceedings of the AAAI Conference on Artificial Intelligence 34(04):6388–6395

    Google Scholar 

  36. Tang C, Liu X, Zhu X et al (2020) CGD: Multi-view clustering via cross-view graph diffusion. Proceedings of the AAAI Conference on Artificial Intelligence 34(04):5924–5931

    Google Scholar 

  37. Li L, He H (2020) Bipartite graph based multi-view clustering. IEEE Transactions on Knowledge and Data Engineering 34(7):3111–3125

  38. Xia W, Wang Q, Gao Q et al (2021) Self-supervised graph convolutional network for multi-view clustering. IEEE Transact Multimedia 24:3182–3192

  39. Li J, Zhou G, Qiu Y et al (2020) Deep graph regularized non-negative matrix factorization for multi-view clustering. Neurocomputing 390:108–116

    Google Scholar 

  40. Li Z, Tang C, Liu X et al (2021) Consensus graph learning for multi-view clustering. IEEE Transact Multimedia 24:2461–2472

  41. Wang Y, Chang D, Fu Z et al (2021) Consistent multiple graph embedding for multi-view clustering. IEEE Transact Multimedia 25:1008–1018

  42. Huang S, Tsang I, Xu Z et al (2021) Measuring diversity in graph learning: a unified framework for structured multi-view clustering. IEEE Transact Knowledge Data Eng 34(12):5869–5883

  43. Yang B, Zhang X, Chen B et al (2022) Efficient correntropy-based multi-view clustering with anchor graph embedding. Neural Netw 146:290–302

    PubMed  Google Scholar 

  44. Shi S, Nie F, Wang R et al (2021) Multi-view clustering via nonnegative and orthogonal graph reconstruction. IEEE Transactions on Neural Networks and Learning Systems 34(1):201–214

  45. Shi S, Nie F, Wang R et al (2021) Fast multi-view clustering via prototype graph. IEEE Transactions on Knowledge and Data Engineering 35(1):443–455

  46. Mei Y, Ren Z, Wu B et al (2022) Robust graph-based multi-view clustering in latent embedding space. Int J Mach Learn Cybern 13(2):497–508

    Google Scholar 

  47. Wang R, Li L, Tao X et al (2022) Contrastive and attentive graph learning for multi-view clustering. Inf Process Manage 59(4):102967

    Google Scholar 

  48. Li M, Liang W, Liu X (2021) Multi-view clustering with learned bipartite graph. IEEE Access 9:87952–87961

    Google Scholar 

  49. Shu X, Zhang X, Wang Q (2022) Self-weighted graph learning for multi-view clustering. Neurocomputing 501:188-196

  50. Gu Z, Feng S (2022) Individuality meets commonality: a unified graph learning framework for multi-view clustering. ACM Trans Knowl Disc Data 17(1):1–21

  51. Liu L, Chen P, Luo G et al (2022) Scalable multi-view clustering with graph filtering. Neural Comput Appl 34(19):16213–16221

  52. Lu X, Feng S (2022) Structure diversity-induced anchor graph fusion for multi-view clustering. ACM Trans Knowl Disc Data 17(2):1–18

  53. Lu H, Gao Q, Zhang X et al (2022) A multi-view clustering framework via integrating k-means and graph-cut. Neurocomputing 501:609–617

  54. Jiang T, Gao Q, Gao X (2021) Multiple graph learning for scalable multi-view clustering. arXiv preprint arXiv: 2106.15382

  55. Wang C, Geng L, Zhang J et al (2022) Multi-view clustering via robust consistent graph learning. Digit Signal Process 128:103607

  56. Yang B, Zhang X, Lin Z et al (2022) Efficient and robust multi-view clustering with anchor graph regularization. IEEE Transactions on Circuits and Systems for Video Technology 32(9):6200–6213

  57. Jiang G, Peng J, Wang H et al (2022) Tensorial multi-view clustering via low-rank constrained high-order graph learning. IEEE Transactions on Circuits and Systems for Video Technology 32(8):5307–5318

  58. Gao Q, Xia W, Gao X et al (2021) Effective and efficient graph learning for multi-view clustering. arXiv preprint arXiv: 2108.06734

  59. Huang S, Kang Z, Xu Z (2020) Auto-weighted multi-view clustering via deep matrix decomposition. Pattern Recogn 97:107015

    Google Scholar 

  60. Xu J, Ren Y, Li G et al (2021) Deep embedded multi-view clustering with collaborative training. Inf Sci 573:279–290

    MathSciNet  Google Scholar 

  61. Wang Q, Cheng J, Gao Q et al (2020) Deep multi-view subspace clustering with unified and discriminative learning. IEEE Trans Multimedia 23:3483–3493

    Google Scholar 

  62. Liu J, Cao F, Gao X et al (2020) A cluster-weighted kernel K-means method for multi-view clustering. Proceedings of the AAAI Conference on Artificial Intelligence 34(04):4860–4867

    Google Scholar 

  63. El Hajjar S, Dornaika F, Abdallah F et al (2022) Consensus graph and spectral representation for one-step multi-view kernel based clustering. Knowl-Based Syst 241:108250

  64. Zhu W, Lu J, Zhou J (2019) Structured general and specific multi-view subspace clustering. Pattern Recogn 93:392–403

    ADS  Google Scholar 

  65. Li Z, Tang C, Chen J et al (2019) Diversity and consistency learning guided spectral embedding for multi-view clustering. Neurocomputing 370:128–139

    Google Scholar 

  66. Mi Y, Ren Z, Mukherjee M et al (2021) Diversity and consistency embedding learning for multi-view subspace clustering. Appl Intell 51(10):6771–6784

    Google Scholar 

  67. Si X, Yin Q, Zhao X et al (2022) Consistent and diverse multi-view subspace clustering with structure constraint. Pattern Recogn 121:108196

    Google Scholar 

  68. Wang S, Liu X, Zhu X et al (2021) Fast parameter-free multi-view subspace clustering with consensus anchor guidance. IEEE Trans Image Process 31:556–568

    ADS  PubMed  Google Scholar 

  69. Kang Z, Zhou W, Zhao Z et al (2020) Large-scale multi-view subspace clustering in linear time. Proceedings of the AAAI Conference on Artificial Intelligence 34(04):4412–4419

    Google Scholar 

  70. Kumar A, Daumé H (2011) A co-training approach for multi-view spectral clustering. Proceedings of the 28th International Conference on Machine Learning 393–400

  71. Kumar A, Rai P, Daume H (2011) Co-regularized multi-view spectral clustering. Adv Neural Inform Proces Syst 24:1413–1421

  72. Zhang C, Fu H, Liu S, et al (2015) Low-rank tensor constrained multiview subspace clustering. Proceedings of the IEEE International Conference on Computer Vision 1582–1590

  73. Cao X, Zhang C, Fu H, et al (2015) Diversity-induced multi-view subspace clustering. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition 586–594

  74. Ng A, Jordan M, Weiss Y (2001) On spectral clustering: Analysis and an algorithm. Adv Neural Inform Proces Syst 14:849–856

  75. Hiriart-Urruty J, Lemaréchal C (2013) Convex analysis and minimization algorithms I: Fundamentals. Springer Science & Business Media, New York

    Google Scholar 

  76. Yang J, Yin W, Zhang Y et al (2009) A fast algorithm for edge-preserving variational multichannel image restoration. SIAM J Imag Sci 2(2):569–592

    MathSciNet  Google Scholar 

  77. Bartels R, Stewart G (1972) Solution of the matrix equation AX + XB = C [F4]. Commun ACM 15(9):820–826

    Google Scholar 

  78. Xia R, Pan Y, Du L, et al (2014) Robust multi-view spectral clustering via low-rank and sparse decomposition. In: Proceedings of AAAI Conference on Artificial Intelligence 2149–2155

  79. Geusebroek J, Burghouts G, Smeulders A (2005) The Amsterdam library of object images. Int J Comput Vision 61(1):103–112

    Google Scholar 

  80. Brbić M, Piškorec M, Vidulin V, et al (2016) The landscape of microbial phenotypic traits and associated genes. Nucleic Acids Res gkw964

  81. Fei-Fei L, Fergus R, Perona P (2004) Learning generative visual models from few training examples: An incremental Bayesian approach tested on 101 object categories. Conference on Computer Vision and Pattern Recognition Workshop 2004:178–178

    Google Scholar 

  82. Zhan K, Zhang C, Guan J et al (2017) Graph learning for multiview clustering. IEEE Transact Cybern 48(10):2887–2895

    Google Scholar 

  83. Wang X, Lei Z, Guo X et al (2019) Multi-view subspace clustering with intactness-aware similarity. Pattern Recogn 88:50–63

    ADS  Google Scholar 

  84. Zhang Z, Liu L, Shen F et al (2019) Binary multi-view clustering. IEEE Trans Pattern Anal Mach Intell 41(7):1774–1782

    PubMed  Google Scholar 

  85. Liu J, Liu X, Yang Y, et al (2021) One-pass multi-view clustering for large-scale data. Proceedings of the IEEE/CVF International Conference on Computer Vision 12344–12353

  86. Tang C, Li Z, Wang J, et al (2022) Unified one-step multi-view spectral clustering. IEEE Transact Knowledge Data Eng 35(6):6449-6460

  87. Huang D, Wang C, Lai J (2017) Locally weighted ensemble clustering. IEEE Transact Cybern 48(5):1460–1473

    Google Scholar 

  88. Demšar J (2006) Statistical comparisons of classifiers over multiple data sets. J Mach Learn Res 7:1–30

    MathSciNet  Google Scholar 

  89. Zhao J, Kang F, Zou Q et al (2023) Multi-view clustering with orthogonal mapping and binary graph. Expert Syst Appl 213:118911

    Google Scholar 

  90. Zhao J, Wang X, Zou Q et al (2023) On improvability of hash clustering data from different sources by bipartite graph. Pattern Anal Appl 26(2):555–570

    Google Scholar 

Download references

Acknowledgements

The authors would like to thank R&D Program of Beijing Municipal Education Commission (Grant KM202011232018), Key Research and Cultivation Project of Scientific Research on Campus of Beijing Information Science and Technology University (Grant 2021YJPY236), National Natural Science Foundation of China (Grant 81903408) and the reviewers for their valuable suggestions.

Author information

Authors and Affiliations

  1. School of Applied Science, Beijing Information Science and Technology University, Beijing, 100192, China

    Jianxi Zhao, Qingrong Zou & Fangyuan Kang

  2. Department of Epidemiology and Health Statistics, School of Public Health, Capital Medical University, Beijing, 100069, China

    Xiaonan Wang

  3. School of Statistics, Renmin University of China, Beijing, 100872, China

    Fan Wang & Jingfu Peng

Authors
  1. Jianxi Zhao
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Xiaonan Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Qingrong Zou
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Fangyuan Kang
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Fan Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Jingfu Peng
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Xiaonan Wang.

Ethics declarations

Conflict of interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zhao, J., Wang, X., Zou, Q. et al. A multiple kinds of information extraction method for multi-view low-rank subspace clustering. Int. J. Mach. Learn. & Cyber. 15, 1313–1330 (2024). https://doi.org/10.1007/s13042-023-01969-5

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s13042-023-01969-5

Keywords

Search

Navigation