Skip to main content
Log in

Unsupervised feature selection based on decision graph

  • New Trends in data pre-processing methods for signal and image classification
  • Published:
Neural Computing and Applications Aims and scope Submit manuscript

Abstract

In applications of algorithms, feature selection has got much attention of researchers, due to its ability to overcome the curse of dimensionality, reduce computational costs, increase the performance of the subsequent classification algorithm and output the results with better interpretability. To remove the redundant and noisy features from original feature set, we define local density and discriminant distance for each feature vector, wherein local density is used for measuring the representative ability of each feature vector, and discriminant distance is used for measuring the redundancy and similarity between features. Based on the above two quantities, the decision graph score is proposed as the evaluation criterion of unsupervised feature selection. The method is intuitive and simple, and its performances are evaluated in the data classification experiments. From statistical tests on the averaged classification accuracies over 16 real-life dataset, it is observed that the proposed method obtains better or comparable ability of discriminant feature selection in 98% of the cases, compared with the state-of-the-art methods.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11

Similar content being viewed by others

Notes

  1. http://www.uk.research.att.com/facedatabase.html.

  2. http://www.cad.zju.edu.cn/home/dengcai/Data/FaceData.html.

  3. http://www.iis.ee.ic.ac.uk/icvl/code.htm.

  4. http://www.cad.zju.edu.cn/home/dengcai/Data/FaceData.html.

  5. http://rvl1.ecn.purdue.edu/~aleix/aleix_face_DB.html.

  6. http://www.darmstadt.gmd.de/mobile/hm/projects/MPEG7/Documents/N2466.html.

  7. http://archive.ics.uci.edu/ml/datasets.html.

References

  1. Chen B, Hong J, Wang Y (1997) The problem of finding optimal subset of features[J]. Chin J Comput 20(2):133–138

    Google Scholar 

  2. Kira K, Rendell LA (1992) The feature selection problem: traditional methods and a new algorithm[C]. In: Proceedings of the ninth national conference on artificial intelligence, Menlo Park, CA, USA, pp 129–134

  3. Robnik-Šikonja M, Kononenko I (2003) Theoretical and empirical analysis of ReliefF and RReliefF[J]. Mach Learn 53(1–2):23–69

    Article  MATH  Google Scholar 

  4. Almuallim H, Dietterich TG (1994) Learning boolean concepts in the presence of many irrelevant features[J]. Artif Intell 69(1):279–305

    Article  MathSciNet  MATH  Google Scholar 

  5. Hall MA (1999) Correlation-based feature selection for machine learning[D]. The University of Waikato, Hillcrest

    Google Scholar 

  6. Yu L, Liu H (2003) Feature selection for high-dimensional data: a fast correlation-based filter solution[C]. ICML 3:856–863

    Google Scholar 

  7. Nie F, Xiang S, Jia Y, Zhang C, Yan S (2008) Trace ratio criterion for feature selection[C]. In: Proceedings of conference on artificial intelligence (AAAI), pp 671–676

  8. Wei HL, Billings SA (2007) Feature subset selection and ranking for data dimensionality reduction[J]. IEEE Trans Pattern Anal Mach Learn 29(1):162–166

    Article  Google Scholar 

  9. Li WH, Chen WM, Yang LP, Gong WG (2007) Face feature selection and recognition based on different types of margin[J]. J Electron Tech 29(7):1744–1748

    Google Scholar 

  10. Dash M, Liu H (2003) Consistency-based search in feature selection[J]. Artif Intell 151(1):155–176

    Article  MathSciNet  MATH  Google Scholar 

  11. Dash M, Choi K, Scheuermann P et al (2002) Feature selection for clustering-A filter solution [C]. In: Proceedings of the 2nd IEEE international conference data mining, Piscataway, pp 115–122

  12. Mitra P, Murthy CA, Pal SK (2002) Unsupervised feature selection using feature similarity[J]. IEEE Trans Pattern Anal Mach Intell 24(3):301–312

    Article  Google Scholar 

  13. Peng H, Long F, Ding C (2005) Feature selection based on mutual information criteria of max-dependency, max-relevance, and min-redundancy[J]. Pattern Anal Mach Intell IEEE Trans 27(8):1226–1238

    Article  Google Scholar 

  14. Singh P (2014) Devanagri handwritten numeral recognition using feature selection approach[J]. Int J Intell Syst Appl 12:40–47. doi:10.5815/ijisa.2014.12.06

    Google Scholar 

  15. Junling X, Yuming Z, Lin C, Baowen X (2012) An unsupervised feature selection approach based on mutual information[J]. J Comput Res Dev 49(2):372–382

    Google Scholar 

  16. Bandhyapadhyay S, Bhadra T, Mitra P, Maullik U (2014) Integration of dense subgraph finding with feature clustering for unsupervised feature selection[J]. Pattern Recogn Lett 40:104–112

    Article  Google Scholar 

  17. He X, Cai D, Niyogi P (2005) Laplacian score for feature selection[C]. In: Advances in neural information processing systems, pp 507–514

  18. Zhao Z, Liu H (2007) Spectral feature selection for supervised and unsupervised learning[C]. In: Proceedings of the international conference on machine learning (ICML), Corvallis, Oregon, pp 1151–1157

  19. Cai D, Zhang C, He X (2010) Unsupervised feature selection for multi-cluster data[C]. In: Proceedings of the 16th ACM SIGKDD international conference on knowledge discovery and data mining. ACM, pp 333–342

  20. Yang Y, Shen H T, Ma Z, et al (2011) l2, 1-norm regularized discriminative feature selection for unsupervised learning[C]. In: IJCAI proceedings-international joint conference on artificial intelligence, vol 22, no 1, p 1589

  21. Li Z, Yang Y, Liu J et al (2012) Unsupervised feature selection using nonnegative spectral analysis[C]. In: Proceedings of the twenty-sixth AAAI conference on artificial intelligence, Canada, pp 1026–1032

  22. Du L, Shen Z, Li X et al (2013) Local and global discriminative learning for unsupervised feature selection[C]. Data mining (ICDM), 2013 IEEE 13th international conference on IEEE, pp 131–140

  23. Zhao Z, Wang L, Liu H et al (2013) On similarity preserving feature selection[J]. Knowl Data Eng IEEE Trans 25(3):619–632

    Article  Google Scholar 

  24. Liu Tao W, Gongyi CZ (2005) An effective unsupervised feature selection method for text clustering[J]. J Comput Res Dev 42(3):381–386

    Article  Google Scholar 

  25. Lu Y, Cohen I, Zhou X S et al (2007) Feature selection using principal feature analysis[C]. In: Proceedings of the 15th international conference on Multimedia. ACM, pp 301–304

  26. Song Q, Ni J, Wang G (2013) A fast clustering-based feature subset selection algorithm for high-dimensional data[J]. Knowl Data Eng IEEE Trans 25(1):1–14

    Article  Google Scholar 

  27. Yan H, Yang J (2015) Sparse discriminative feature selection[J]. Pattern Recogn 48(5):1827–1835

    Article  Google Scholar 

  28. Rodriguez A, Laio A (2014) Clustering by fast search and find of density peaks[J]. Science 344(6191):1492–1496

    Article  Google Scholar 

  29. Brahim AB, Limam M (2016) A hybrid feature selection method based on instance learning and cooperative subset search[J]. Pattern Recogn Lett 69:28–34

    Article  Google Scholar 

  30. Şen B, Peker M, Çavuşoğlu A et al (2014) A comparative study on classification of sleep stage based on EEG signals using feature selection and classification algorithms[J]. J Med Syst 38(3):1–21

    Google Scholar 

Download references

Acknowledgements

We thank the editors and anonymous reviewers for their very useful comments and suggestions. This work is supported in part by Science Computing and Intelligent Information Processing of Guangxi Higher Education Key Laboratory (No. GXSCIIP201406), Doctoral Starting up Foundation of Northwest A&F University (No. 2452015302), the Fundamental Research Funds for the Central Universities (No. 2452015197), National Natural Science Foundation of China (No. 61402481) and Hebei Province Natural Science Foundation of China (No. F2015403046).

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Jinrong He.

Ethics declarations

Conflict of interest

The authors declared that they have no conflicts of interest to this work. All authors of this manuscript have directly participated in planning, execution and analysis of this study. The contents of this manuscript have not been copyrighted or published previously, or under consideration for publication elsewhere.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

He, J., Bi, Y., Ding, L. et al. Unsupervised feature selection based on decision graph. Neural Comput & Applic 28, 3047–3059 (2017). https://doi.org/10.1007/s00521-016-2737-2

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00521-016-2737-2

Keywords

Navigation