Skip to main content
SpringerLink
Log in

Mutual neighborhood and modified majority voting based KNN classifier for multi-categories classification

  • Theoretical Advances
  • Published:
Pattern Analysis and Applications Aims and scope Submit manuscript

Abstract

In this paper, two techniques for improving the performance of the k-Nearest Neighbors (KNN) based classifiers are proposed: mutual neighborhood (MN) for searching the neighbors of the query sample, and two-stage modified majority voting (MMV) based decision. In MN, two samples are the neighbors of each other, if each of them exists in the k-neighborhood of the other. Selecting the MN-based neighbors depends on the data distribution and makes to select the data with the same category and/or more similarity. Also, the number of neighbors is variable in MN. Moreover, a two-stage method is proposed to improve majority voting based classifiers which we call it modified majority voting. In MMV, if there is any ambiguous, the mean vectors of each category with majority voting are computed and then the decision is made based on the minimum Euclidean distance of the mean vectors from the query sample. By the proposed techniques, some new and extended KNN-based classifiers are defined. Two different kinds of databases are used in our experiments: eight datasets of UCI machine learning repository and fifteen datasets of UCR time series classification archive. The results exhibit the proposed techniques increase the recognition rates of the KNN-based classifies. In some cases, the rate of improvement is more than 10%.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

References

  1. Deng W, Hu J, Guo J (2018) Compressive binary patterns: designing a robust binary face descriptor with random-field eigen filters. IEEE Trans Pattern Anal Mach Intell 41(3):758–767

    Article  Google Scholar 

  2. Liu Z, Wu XJ, Shu Z (2019) Sparsity augmented discriminative sparse representation for face recognition. Pattern Anal Appl 22:1527–1535. https://doi.org/10.1007/s10044-019-00792-5

    Article  MathSciNet  Google Scholar 

  3. Malach T, Pomenkova J (2020) Optimal face templates: the next step in surveillance face recognition. Pattern Anal Appl 23:1021–1032. https://doi.org/10.1007/s10044-019-00842-y

    Article  Google Scholar 

  4. Hajizadeh R, Aghagolzadeh A, Ezoji M (2018) Fusion of LLE and stochastic LEM for Persian handwritten digits recognition. Int J Doc Anal Recognit (IJDAR) 21(1–2):109–122

    Article  Google Scholar 

  5. Hajizadeh R, Aghagolzadeh A, Ezoji M (2020) Local distances preserving based manifold learning. Expert Syst Appl 139:112860

    Article  Google Scholar 

  6. Dash KS, Puhan NB, Panda G (2018) Unconstrained handwritten digit recognition using perceptual shape primitives. Pattern Anal Appl 21:413–436. https://doi.org/10.1007/s10044-016-0586-3

    Article  MathSciNet  Google Scholar 

  7. Li Y, Cui WG, Huang H, Guo YZ, Li K, Tan T (2019) Epileptic seizure detection in EEG signals using sparse multiscale radial basis function networks and the Fisher vector approach. Knowl-Based Syst 164:96–106

    Article  Google Scholar 

  8. Lin D, Sun L, Toh KA, Zhang JB, Lin Z (2018) Biomedical image classification based on a cascade of an SVM with a reject option and subspace analysis. Comput Biol Med 96:128–140

    Article  Google Scholar 

  9. Shen SY, Singhania R, Fehringer G, Chakravarthy A, Roehrl MH, Chadwick D, Zuzarte PC, Borgida PC, Wang A, Li TT, Kis O (2018) Sensitive tumour detection and classification using plasma cell-free DNA methylomes. Nature 563:579–583

    Article  Google Scholar 

  10. Karthiga R, Narasimhan K (2021) Medical imaging technique using curvelet transform and machine learning for the automated diagnosis of breast cancer from thermal image. Pattern Anal Appl. https://doi.org/10.1007/s10044-021-00963-3

    Article  Google Scholar 

  11. You S, Xu C, Xu C, Tao D (2018) Learning with single-teacher multi-student. In: The thirty-second AAAI conference on artificial intelligence (AAAI-18), pp 4390–4397.

  12. Uebele V, Abe S, Lan MS (1995) A neural-network-based fuzzy classifier. IEEE Trans Syst Man Cybern 25(2):353–361

    Article  Google Scholar 

  13. Cover T, Hart P (1967) Nearest neighbor pattern classification. IEEE Trans Inf Theory 13(1):21–27

    Article  Google Scholar 

  14. Kanisha B, Lokesh S, Kumar PM, Parthasarathy P, Babu GC (2018) Speech recognition with improved support vector machine using dual classifiers and cross fitness validation. Pers Ubiquitous Comput 22(5–6):1083–1091

    Article  Google Scholar 

  15. Wang H, Zheng B, Yoon SW, Ko HS (2018) A support vector machine-based ensemble algorithm for breast cancer diagnosis. Eur J Oper Res 267(2):687–699

    Article  MathSciNet  Google Scholar 

  16. Abid F, Hamami L (2018) A survey of neural network based automated systems for human chromosome classification. Artif Intell Rev 49(1):41–56

    Article  Google Scholar 

  17. Beritelli F, Capizzi G, Sciuto GL, Napoli C, Scaglione F (2018) Automatic heart activity diagnosis based on Gram polynomials and probabilistic neural networks. Biomed Eng Lett 8(1):77–85

    Article  Google Scholar 

  18. Gerhana YA, Zulfikar WB, Ramdani AH, Ramdhani MA (2018) Implementation of nearest neighbor using HSV to identify skin disease. IOP Conf Ser Mater Sci Eng. 288(1):012153

    Article  Google Scholar 

  19. Maillo J, Triguero I, Herrera F (2015) A map reduce-based k-nearest neighbor approach for big data classification. IEEE Trustcom/BigDataSE/ISPA 2:167–172

    Article  Google Scholar 

  20. Wang JG, Neskovic P, Cooper LN (2006) Neighborhood size selection in the k-nearest-neighbor rule using statistical confidence. Pattern Recognit 39(3):417–423

    Article  Google Scholar 

  21. Zhang S, Li X, Zong M, Zhu X, Cheng D (2017) Learning k for kNN classification. ACM Trans Intell Syst Technol 8(3):43

    Article  Google Scholar 

  22. Zhang SC, Li X, Zong M, Zhu X, Wang R (2018) Efficient knn classification with different numbers of nearest neighbors. IEEE Trans Neural Netw Learn Syst 29(5):1774–1785

    Article  MathSciNet  Google Scholar 

  23. Gou JP, Du L, Zhang Y, Xiong T (2012) A new distance-weighted k-nearest neighbor classifier. J Inf Comput Sci 9(6):1429–1436

    Google Scholar 

  24. Li W, Du Q, Zhang F, Hu W (2015) Collaborative-representation based nearest classifier for hyperspectral imagery. IEEE Geosci Remote Sens Lett 12(2):389–393

    Article  Google Scholar 

  25. Zeng Y, Yang Y, Zhao L (2009) Pseudo nearest neighbor rule for pattern classification. Expert Syst Appl 36:3587–3595

    Article  Google Scholar 

  26. Mitani Y, Hamamoto Y (2006) A local mean-based nonparametric classifier. Pattern Recognit Lett 27(10):1151–1159

    Article  Google Scholar 

  27. Gou JP, Qiu WM, Mao QR, Zhan YZ, Shen XZ, Rao YB (2017) A multi-local means based nearest neighbor classifier. In: 2017 IEEE 29th international conference on tools with artificial intelligence (ICTAI), pp 448–452

  28. Pan ZP, Wang YD, Ku WP (2017) A new k-harmonic nearest neighbor classifier based on the multi-local means. Expert Syst Appl 67:115–125

    Article  Google Scholar 

  29. Gou JP, Zhan YZ, Rao YB, Shen XJ, Wang XM, He W (2014) Improved pseudo nearest neighbor classification. Knowl-Based Syst 70:361–375

    Article  Google Scholar 

  30. Dudani SA (1976) The distance-weighted k-nearest neighbor rule. IEEE Trans Syst Man Cybern 6(4):325–327

    Article  Google Scholar 

  31. Gou J, Qiu W, Yi Z, Shen X, Zhan Y, Ou W (2019) Locality constrained representation-based K-nearest neighbor classification. Knowl-Based Syst 167:38–52

    Article  Google Scholar 

  32. Dua D, Graff C (2019) UCI Machine learning repository. http://archive.ics.uci.edu/ml. University of California, School of Information and Computer Science, Irvine, CA

  33. Chen Y, Keogh E, Hu B, Begum N, Bagnall A, Mueen A, Batista G (2015) The UCR time series classification archive. http://www.cs.ucr.edu/~eamonn/time_series_data/

Download references

Funding

Funding was provided by Babol Noshirvani University of Technology (No. BNUT/389059/400).

Author information

Authors and Affiliations

  1. Faculty of Engineering Modern Technologies, Amol University of Special Modern Technologies, Amol, Iran

    Rassoul Hajizadeh

  2. Faculty of Electrical and Computer Engineering, Babol Noshirvani University of Technology, Babol, Iran

    Rassoul Hajizadeh, Ali Aghagolzadeh & Mehdi Ezoji

Authors
  1. Rassoul Hajizadeh
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Ali Aghagolzadeh
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Mehdi Ezoji
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Ali Aghagolzadeh.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Hajizadeh, R., Aghagolzadeh, A. & Ezoji, M. Mutual neighborhood and modified majority voting based KNN classifier for multi-categories classification. Pattern Anal Applic 25, 773–793 (2022). https://doi.org/10.1007/s10044-022-01069-0

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10044-022-01069-0

Keywords

Search

Navigation