Skip to main content
SpringerLink
Log in

Sparse L1-norm-based linear discriminant analysis

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Linear discriminant analysis (LDA) is a well-known feature extraction method, which has been widely used for many pattern recognition problems. However, the objective function of conventional LDA is based on L2-norm, which makes LDA sensitive to outliers. Besides, the basis vectors learned by conventional LDA are dense and it is often hard to explain the extracted features. In this paper, we propose a novel sparse L1-norm-based linear discriminant analysis (SLDA-L1) which not only replaces L2-norm in conventional LDA with L1-norm, but also use the elastic net to regularize the basis vectors. Then L1-norm used in SLDA-L1 is for both robust and sparse modelling simultaneously. We also propose an efficient iterative algorithm to solve SLDA-L1 which is theoretically shown to arrive at a locally maximal point. Experiment results on some image databases demonstrate the effectiveness of the proposed method.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

Similar content being viewed by others

References

  1. Ahonen T, Hadid A, Pietikäinen M (2006) Face description with local binary patterns: Application to face recognition. IEEE Trans Pattern Anal Mach Intell 28(12):2037–2041

    Article  MATH  Google Scholar 

  2. Bair E, Hastie T, Paul D, Tibshirani R (2006) Prediction by supervised principal components. J Am Stat Assoc 101(473):119–137

    Article  MathSciNet  MATH  Google Scholar 

  3. Barshan E, Ghodsi A, Azimifar Z, Jahromi MZ (2011) Supervised principal component analysis: Visualization, classification and regression on subspaces and submanifolds. Pattern Recogn 44(7):1357–1371

    Article  MATH  Google Scholar 

  4. Belhumeour PN, Hespanha JP, Kriegman DJ (1997) Eigenfaces vs. Fisherfaces: Recognition using class specific linear projection. IEEE Trans Pattern Anal Mach Intell 19(7):711–720

    Article  Google Scholar 

  5. Cai D, He X, Han J (2007) Spectral regression: a unified approach for sparse subspace learning. Proceeding of the 2007 International Conference on Data Mining (ICDM 07), Omaha, NE, 73–87

  6. Cai D, He X, Han J (2008) Sparse projections over graph. Proceedings of the 21st AAAI conference on artifician intelligence

  7. Ding C, Zhou D, He X, Zha H (2006) R1-PCA: Rotational invariant L1-norm principal component analysis for robust subspace factorization. Proceedings of the 23rd Internal Conference on Machine Learning, 281–288

  8. Duda RO, Hart PE, Stork DG (2000) Pattern Classification, 2nd edn. John Wiley & Sons, New York

    MATH  Google Scholar 

  9. Fukunaga K (1990) Introduction to Statistical Pattern Recognition, 2nd edn. Academic Press, Boston, USA

    MATH  Google Scholar 

  10. Gu B, Sheng VS (2016) A robust regularization path algorithm for ν-support vector classification. IEEE Trans Neural Netw Learn Syst. https://doi.org/10.1109/TNNLS.2016.2527796

  11. Gu B, Sheng VS, Tay KY, Romano W, Li S (2015) Incremental support vector learning for ordinal regression. IEEE Trans Neural Netw Learn Syst 26(7):1403–1416

    Article  MathSciNet  Google Scholar 

  12. Gu B, Sheng VS, Wang Z, Ho D, Osman S, Li S (2015) Incremental learning for ν-support vector regression. Neural Netw 67(7):140–150

    Article  Google Scholar 

  13. Gu B, Sun X, Sheng VS (2016) Structural minimax probability machine. IEEE Trans Neural Netw Learn Syst. https://doi.org/10.1109/TNNLS.2016.2544779

  14. Jain AK, Duin RPW, Mao J (2000) Statistical pattern recognition: A review. IEEE Trans on Pattern Analysis and Machine Intelligence 22(1):4–37

    Article  Google Scholar 

  15. Jenatton R, Obozinski G, Bach F (2010) Structured sparse principal component analysis. Proceeding of the 13th international conference on artificial intelligence and statistics, 366–373

  16. Kawulok M, Wu J, Hancock ER (2011) Supervised relevance maps for increasing the distinctiveness of facial images. Pattern Recogn 44(4):929–939

    Article  Google Scholar 

  17. Ke Q, Kanade T (2005) Robust L1 norm factorization in the presence of outliers and missing data by alternative convex programming, Proc IEEE Conf Comput Vis Pattern Recognit, San Diego, CA, USA, 20-26 June, vol. 1, p 1-8

  18. Kwak N (2008) Principal component analysis based on L1-norm maximization. IEEE Trans on Pattern Anal Mach Intell 30(9):1672–1680

    Article  Google Scholar 

  19. Leng L, Zhang J, Xu J, Khan MK, Alghathbar K (2010) Dynamic weighted discrimination power analysis in DCT domain for face and palmprint recognition. Int Conf Inf Commun Technol Convergence:467–471

  20. Leng L, Zhang J, Khan MK, Chen X, Alghathbar K (2010) Dynamic weighted discrimination power analysis: A novel approach for face and palmprint recognition in DCT domain. Int J Phys Sci 5(17):2543–2554

    Google Scholar 

  21. Leng L, Zhang J, Chen G, Khan MK, Alghathbar K (2011) Two-directional two-dimensional random projection and its variations for face and palmprint recognition. Int Conf Comput Sci Appl, Santander, Spain, June 20-23, p 458–470

  22. Li H, Jiang T, Zhang K (2004) Efficient and robust feature extraction by maximum margin criterion. Advances in Neural Information Processing Systems, Cambridge, MA, 97-104

  23. Li H, Jiang T, Zhang K (2006) Efficient and robust feature extraction by maximum margin criterion. IEEE Trans Neural Netw 17(1):1157–1165

    Article  Google Scholar 

  24. Li X, Hua W, Wang H, Zhang Z (2010) Linear discriminant analysis using rotational invariant L1 norm. Neurocomputing 13-15(73):2571–2579

    Article  Google Scholar 

  25. Meng D, Zhao Q, Xu Z (2012) Improve robustness of sparse PCA by L1-norm maximization. Pattern Recogn 45(1):487–497

    Article  MATH  Google Scholar 

  26. Nie F, Huang H, Ding C, Luo D, Wang H (2011) Principal component analysis with non-greedy L1-norm maximization. The 22nd International Joint Conference on Artificial Intelligence (IJCAI), Barcelona, 1-6

  27. Pang Y, Li X, Yuan Y (2010) Robust tensor analysis with L1-Norm. IEEE Trans Circuits Syst Video Technol 20(2):172–178

    Article  Google Scholar 

  28. Wang H (2012) Structured sparse linear graph embedding. Neural Netw 27:38–44

    Article  Google Scholar 

  29. Wang H, Wang J (2013) 2DPCA with L1-norm for simultaneously robust and sparse modelling. Neural Netw 46(10):190–198

    Article  MATH  Google Scholar 

  30. Wang H, Tang Q, Zheng W (2012) L1-norm-based common spatial patterns. IEEE Trans Biomed Eng 59(3):653–662

    Article  Google Scholar 

  31. Wang H, Lu X, Hu Z, Zheng W (2014) Fisher discriminant analysis with L1-norm. IEEE Trans Cybernetics 44(6):828–842

    Article  Google Scholar 

  32. Wen X, Shao L, Xue Y, Fang W (2015) A rapid learning algorithm for vehicle classification. Inf Sci 295(1):395–406

    Article  Google Scholar 

  33. Xia J, Chanussot J, Du P, He X (2014) (Semi-) supervised probabilistic principal component analysis for hyperspectral remote sensing image classification. IEEE J Sel Top Appl Earth Observations Remote Sens 7(6):2224–2236

    Article  Google Scholar 

  34. Xuelong L, Pang Y, Yuan Y (2009) L1-Norm-Based 2DPCA. IEEE Trans Syst Man Cybern B Cybern 40(4):1170–1175

    Article  Google Scholar 

  35. Yan S, Xu D, Zhang B, Zhang H-J, Yang Q, Lin S (2007) Graph embedding and extensions: A general framework for dimensionality reduction. IEEE Trans Pattern Anal Mach Intell 29(1):40–51

    Article  Google Scholar 

  36. Yang J, Zhang D, Frangi AF, Yang JY (2004) Two-dimensional PCA: A new approach to appearance-based face representation and recognition. IEEE Trans Pattern Anal Mach Intell 26(1):131–137

    Article  Google Scholar 

  37. Zhao G, Pietikäinen M (2007) Dynamic texture recognition using local binary patterns with an application to facial expressionas. IEEE Trans Pattern Anal Mach Intell 29(6):915–928

    Article  Google Scholar 

  38. Zheng W, Lin Z, Wang H (2014) L1-norm kernel discriminant analysis via Bayes error bound optimizatin for robust feature extraction. IEEE Trans Neural Netw Learn Syst 25(4):793–805

    Article  Google Scholar 

  39. Zhong F, Zhang J (2013) Linear discriminant analysis based on L1-norm maximization. IEEE Trans Image Process 22(8):3018–3027

    Article  MathSciNet  MATH  Google Scholar 

  40. Zhou H, Hastie T, Tibshirani R (2006) Sparse principal component analysis. J Comput Graph Stat 15(2):265–286

    Article  MathSciNet  Google Scholar 

  41. Zhou T, Tao D, Wu X (2011) Manifold elastic net: A unified framework for sparse dimension reduction. Data Min Knowl Disc 22:340–371

    Article  MathSciNet  MATH  Google Scholar 

  42. Zhou Z, Wang Y, Wu QMJ, Yang C-N, Sun X (2016) Effective and efficient global context verification for image copy detection. IEEE Trans Inf Forensics and Secur. https://doi.org/10.1109/TIFS.2016.2601065

Download references

Acknowledgments

This research is supported by supported by NSFC of China (No. 61572033, 71371012), the Natural Science Foundation of Education Department of Anhui Province of China (No.KJ2015ZD08), the Social Science and Humanity Foundation of the Ministry of Education of China (No. 13YJA630098).

Author information

Authors and Affiliations

  1. School of Computer and Information, AnHui Polytechnic University, WuHu, AnHui, 241000, China

    Gui-Fu Lu, Jian Zou, Yong Wang & Zhongqun Wang

  2. School of Computer and Software, Nanjing University of Information Science and Technology, Nanjing, 210044, China

    Gui-Fu Lu

Authors
  1. Gui-Fu Lu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jian Zou
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Yong Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Zhongqun Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Gui-Fu Lu.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Lu, GF., Zou, J., Wang, Y. et al. Sparse L1-norm-based linear discriminant analysis. Multimed Tools Appl 77, 16155–16175 (2018). https://doi.org/10.1007/s11042-017-5193-9

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-017-5193-9

Keywords

Search

Navigation