Skip to main content
SpringerLink
Log in

Kernel Optimization for Reducing Core Vector Machine Classification Error

  • Published:
Neural Processing Letters Aims and scope Submit manuscript

Abstract

Core vector machine (CVM) is considered as a popular machine learning technique, mostly used for classification and regression. Its generalized form can be either applied to kernel methods or used with linear and nonlinear kernels. As a classifier, both the kernel function and its parameters can significantly affect the accuracy of CVM classification. In this paper, a metaheuristic kernel core vector machine (MKCVM) was applied by using comprehensive learning particle swarm optimization. It was done through an innovative parameter selection technique in order to obtain a proper kernel function based on a nonlinear combination of standard kernel functions. The CVM true classification rate and geometric mean accuracy were adopted as the fitness criteria of the proposed approach, to be maximized. In addition, some benchmark datasets were used to evaluate the proposed approach. Based on the experimental results, the proposed method could decrease CVM classification error and increase its geometric mean accuracy. This has a comparative performance rather than support vector machine with lower memory and computational complexity. Moreover, MKCVM showed the acceptable capability to deal with imbalanced data. So, it had a good ability for handling binary and multiclass data classification.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

References

  1. Afzal AL, Asharaf S (2018) Deep kernel learning in core vector machines. Pattern Anal Appl 21(3):721–729. https://doi.org/10.1007/s10044-017-0600-4

    Article  MathSciNet  Google Scholar 

  2. Alizadeh M, Ebadzadeh M (2011) Kernel evolution for support vector classification, pp 93–99. https://doi.org/10.1109/EAIS.2011.5945924

  3. Alpaydin E (2010) Introduction to machine learning, 2nd edn. MIT Press

  4. Amudha P, Karthik S, Sivakumari S (2015) Intrusion detection based on core vector machine and ensemble classification methods. In: 2015 international conference on soft-computing and networks security (ICSNS), pp 1–5. Coimbatore, INDIA. https://doi.org/10.1109/ICSNS.2015.7292408

  5. Anh Tran Quang, Qian-Li Zhang, Xing Li (2002) Evolving support vector machine parameters. In: Proceedings. International conference on machine learning and cybernetics, vol 1, pp 548–551. https://doi.org/10.1109/ICMLC.2002.1176817

  6. Asharaf S, Murty MN, Shevade SK (2006) Cluster based core vector machine. In: Sixth international conference on data mining (ICDM’06), pp 1038–1042. https://doi.org/10.1109/ICDM.2006.34

  7. Asharaf S, Murty MN, Shevade SK (2007) Multiclass core vector machine. In: Proceedings of the 24th international conference on machine learning, vol. 27, pp 41–48. https://doi.org/10.1145/1273496.1273502

  8. Canu S, Grandvalet YVGAR (2005) SVM and Kernel Methods Matlab Toolbox. Published in Perception Systems and Information, Rouen, France. Accessed 12 April 2019

  9. Chen J, Zhang X, Guo K (2016) A core set based large vector-angular region and margin approach for novelty detection. Math Probl Eng 2016:929–940

    MathSciNet  MATH  Google Scholar 

  10. Diego IM, noz AM, Moguerza JM (2010) Methods for the combination of kernel matrices within a support vector framework. Mach Learn 78(1–2):137–174. https://doi.org/10.1007/s10994-009-5135-5

    Article  MathSciNet  MATH  Google Scholar 

  11. Ding S, Liu X (2009) Evolutionary computing optimization for parameter determination and feature selection of support vector machines. In: 2009 international conference on computational intelligence and software engineering, pp 1–5. Wuhan, China. https://doi.org/10.1109/CISE.2009.5366095

  12. Dioşan L, Rogozan A, Pécuchet JP (2007) Evolving kernel functions for svms by genetic programming. In: Sixth international conference on machine learning and applications (ICMLA 2007), pp 19–24. IEEE Computer Society, USA. https://doi.org/10.1109/ICMLA.2007.70

  13. Dioşan L, Rogozan A, Pécuchet JP (2012) Improving classification performance of support vector machine by genetically optimising kernel shape and hyper-parameters. Appl Intell 36(2):280–294. https://doi.org/10.1007/s10489-010-0260-1

    Article  Google Scholar 

  14. Dua D, Graff C (2017) Uci machine learning repository. http://archive.ics.uci.edu/ml

  15. Friedrichs F, Igel C (2005) Evolutionary tuning of multiple svm parameters. In: Trends in Neurocomputing: 12th European symposium on artificial neural networks 2004, vol 64, pp 107–117. https://doi.org/10.1016/j.neucom.2004.11.022

  16. Gangurde H, Metre K (2015) Clustering based feature selection from high dimensional data. Int J Recent Innov Trends Comput Commun 3:3556–3560

    Google Scholar 

  17. Gangurde HD (2014) Feature selection using clustering approach for big data. IJCA Proceedings on Innovations and Trends in Computer and Communication Engineering, Innovations and Trends in Computer and Communication Engineering (ITCCE)(4), 1–3

  18. Genton MG (2002) Classes of kernels for machine learning: a statistics perspective. J Mach Learn Res 2:299–312

    MathSciNet  MATH  Google Scholar 

  19. Gomathi R, Ganapathy S (2017) A new approach for classification of prices in the electricity market using core vector machine, pp 13909–13916

  20. Gu S, Guo YS (2012) Learning svm classifiers with indefinite kernels. In: in Proceedings of the 26th conference on artificial intelligence (AAAI 2012), AAAI’12. AAAI Press, pp 942–948

  21. Gu X, Chung FL, Wang S (2019) Extreme vector machine for fast training on large data. Int J Mach Learn Cybern 11:33–53. https://doi.org/10.1007/s13042-019-00936-3

    Article  Google Scholar 

  22. Howley T, Madden M (2005) The genetic kernel support vector machine: description and evaluation. Artif Intell Rev 24:379–395. https://doi.org/10.1007/s10462-005-9009-3

    Article  Google Scholar 

  23. Hsu CW, Chang CC, Lin CJ (2003) A practical guide to support vector classification. Tech. rep., Department of Computer Science and Information Engineering, University of National Taiwan, Taipei

  24. Hu M, Chen Y, Kwok JT (2009) Building sparse multiple-kernel svm classifiers. IEEE Trans Neural Netw 20(5):827–839. https://doi.org/10.1109/TNN.2009.2014229

    Article  Google Scholar 

  25. Hu W, lai Chung F, Wang S (2012) The maximum vector-angular margin classifier and its fast training on large datasets using a core vector machine. Neural Netw 27:60–73. https://doi.org/10.1016/j.neunet.2011.10.005

    Article  MATH  Google Scholar 

  26. Huang CL, Wang CJ (2006) A ga-based feature selection and parameters optimization for support vector machines. Expert Syst Appl 31:231–240. https://doi.org/10.1016/j.eswa.2005.09.024

    Article  Google Scholar 

  27. Jian L, Xia Z, Liang X, Gao C (2011) Design of a multiple kernel learning algorithm for ls-svm by convex programming. Neural Netw Off J Int Neural Netw Soc 24:476–83. https://doi.org/10.1016/j.neunet.2011.03.009

    Article  MATH  Google Scholar 

  28. Kubat M, Matwin S (1997) Addressing the curse of imbalanced training sets: one-sided selection. In: In Proceedings of the fourteenth international conference on machine learning. Morgan Kaufmann, pp 179–186

  29. Kuo RJ, Chen CM (2011) Evolutionary-based support vector machine. In: 2011 IEEE international conference on industrial engineering and engineering management, pp 472–475. https://doi.org/10.1109/IEEM.2011.6117962

  30. Lessmann S, Li N, Voß S (2008) A case study of core vector machines in corporate data mining. Waikoloa, HI, USA, pp 1–9

  31. Li J, Liu H (2017) Challenges of feature selection for big data analytics. IEEE Intell Syst 32(2):9–15. https://doi.org/10.1109/MIS.2017.38

    Article  MathSciNet  Google Scholar 

  32. Liang JJ, Qin AK, Suganthan PN, Baskar S (2006) Comprehensive learning particle swarm optimizer for global optimization of multimodal functions. IEEE Trans Evol Comput 10(3):281–295. https://doi.org/10.1109/TEVC.2005.857610

    Article  Google Scholar 

  33. Loosli G, Canu S (2007) Comments on the “core vector machines: Fast svm training on very large data sets’’. J Mach Learn Res 8:291–301

    MATH  Google Scholar 

  34. Lu M, Chen C, Huo J (2009) Optimization of combined kernel function for svm by particle swarm optimization. In: 2009 IEEE international conference on machine learning and cybernetics, vol 2, pp 1160–1166. Baoding, China

  35. Lu S, Zhu C, Jiao C (2015) Density weighted core support vector machine. Adv Comput Sci Int J 4(6):150–155

    Google Scholar 

  36. Meng YY (2009) Parameters selection of hybrid kernel based on ga. In: 2009 international workshop on intelligent systems and applications, pp 1–4. https://doi.org/10.1109/IWISA.2009.5072997

  37. Mei Dong A (2014) Research and implementation of support vector machine and its fast algorithm. Int. J. Multimedia Ubiquit. Eng. 9:79–90. https://doi.org/10.14257/ijmue.2014.9.10.08

    Article  Google Scholar 

  38. Ming Y, Yue-qiao A (2011) Svm parameters optimization based on artificial bee colony algorithm and its application in handwriting verification. In: 2011 international conference on electrical and control engineering, pp 5026–5029. https://doi.org/10.1109/ICECENG.2011.6057135

  39. Montana DJ (1995) Strongly typed genetic programming. Evol Comput 3(2):199–230. https://doi.org/10.1162/evco.1995.3.2.199

    Article  Google Scholar 

  40. Phienthrakul T, Kijsirikul B (2007) Gpes: An algorithm for evolving hybrid kernel functions of support vector machines. In: 2007 IEEE congress on evolutionary computation, pp 2636–2643. Singapore. https://doi.org/10.1109/CEC.2007.4424803

  41. Rojas SA, Fernandez-Reyes D (2005) Adapting multiple kernel parameters for support vector machines using genetic algorithms. In: 2005 IEEE congress on evolutionary computation, vol 1, pp 626–631. Edinburgh, Scotland. https://doi.org/10.1109/CEC.2005.1554741

  42. Rojo-Álvarez JL, Martinez-Ramon M, Muñoz J, Camps-Valls G (2018) Digital signal processing with kernel methods. Wiley. https://doi.org/10.1002/9781118705810

  43. Sruthika SNT (2016) Improved multiobjective binary biogeography based optimization using cvm for feature selection using gene expression data, pp 95–101

  44. Samadzadegan F, Soleymani A, Abbaspour RA (2010) Evaluation of genetic algorithms for tuning svm parameters in multi-class problems. In: 2010 11th international symposium on computational intelligence and informatics (CINTI), pp 323–328. Budapest, Hungary. https://doi.org/10.1109/CINTI.2010.5672224

  45. Schleif F (2014) Matlab implementation of the Core Vector Machine of I. Tsang et al. http://www.techfak.uni-bielefeld.de/~fschleif/software.xhtml

  46. Schleif FM, Tino P (2017) Indefinite core vector machine. Pattern Recogn 71:187–195. https://doi.org/10.1016/j.patcog.2017.06.003

    Article  Google Scholar 

  47. Scholkopf B, Smola AJ (2002) Learning with kernels: support vector machines, regularization, optimization, and beyond. MIT Press, Cambridge

    Google Scholar 

  48. Shawe-Taylor J, Cristianini N (2004) Kernel methods for pattern analysis. Cambridge University Press

  49. Smits GF, Jordan EM (2002) Improved svm regression using mixtures of kernels. In: Proceedings of the 2002 international joint conference on neural networks (IJCNN’02), vol 3, pp 2785–2790. Honolulu, Hawaii. https://doi.org/10.1109/IJCNN.2002.1007589

  50. Song H, Ding Z, Guo C, Li Z, Xia H (2008) Research on combination kernel function of support vector machine. In: 2008 international conference on computer science and software engineering, vol 1, pp 838–841. https://doi.org/10.1109/CSSE.2008.1231

  51. Sonnenburg S, Rätsch G, Schäfer C, Schölkopf B (2006) Large scale multiple kernel learning. J Mach Learn Res 7:1531–1565

    MathSciNet  MATH  Google Scholar 

  52. Sullivan K, Luke S (2007) Evolving kernels for support vector machine classification. In: Proceedings of the 9th annual conference on genetic and evolutionary computation, GECCO ’07, pp 1702–1707. Association for Computing Machinery, New York, NY, USA. https://doi.org/10.1145/1276958.1277292

  53. Szafranski M, Grandvalet Y, Rakotomamonjy A (2008) Learning with groups of kernels

  54. Tax DMJ, Duin RPW (2004) Support vector data description. Mach Learn 54(1):45–66. https://doi.org/10.1023/B:MACH.0000008084.60811.49

    Article  MATH  Google Scholar 

  55. Tian X, Gasso G, Canu S (2012) A multiple kernel framework for inductive semi-supervised svm learning. Neurocomputing 90:46–58. https://doi.org/10.1016/j.neucom.2011.12.036

    Article  Google Scholar 

  56. Tsang I, Kwok J, Cheung PM (2005) Very large svm training using core vector machines. In: In Proceedings of the tenth international workshop on artificial intelligence and statistics (AISTATS), pp 349–356

  57. Tsang IW, Kwok JT, Zurada PM (2006) Generalized core vector machines. IEEE Trans Neural Netw 17(5):1126–1140. https://doi.org/10.1109/TNN.2006.878123

    Article  Google Scholar 

  58. Tsang IW, Kocsor A, Kwok JT (2007) Simpler core vector machines with enclosing balls. In: Proceedings of the 24th international conference on machine learning, ICML ’07, pp 911–918. Association for Computing Machinery, New York, NY, USA. https://doi.org/10.1145/1273496.1273611

  59. Tsang IW, Kocsor A, Kwok JT (2008) Large-scale maximum margin discriminant analysis using core vector machines. IEEE Trans Neural Netw 19(4):610–624. https://doi.org/10.1109/TNN.2007.911746

    Article  Google Scholar 

  60. Tsang IW, Kwok JT, Cheung PM (2005) Core vector machines: fast svm training on very large data sets. J Mach Learn Res 6(13):363–392

    MathSciNet  MATH  Google Scholar 

  61. Vapnik VN (1999) The nature of statistical learning theory, 2nd edn. Springer, New York

    MATH  Google Scholar 

  62. Wei L, Wei B, Wang B (2012) Text classification using support vector machine with mixture of kernel. J Softw Eng Appl 05:55–58. https://doi.org/10.4236/jsea.2012.512B012

    Article  Google Scholar 

  63. Weston J, Schölkopf B, Eskin E, Leslie C, Noble W (2003) Dealing with large diagonals in kernel matrices. Ann Inst Stat Math 55:391–408

    Article  MathSciNet  MATH  Google Scholar 

  64. Yongqing W, Xiaotai N, Liang C (2010) Multiclass core vector machine with smaller core sets. In: 2010 Chinese control and decision conference, pp 525–530. https://doi.org/10.1109/CCDC.2010.5498996

  65. Zhang Q, Shan G, Duan X, Zhang Z (2009) Parameters optimization of support vector machine based on simulated annealing and genetic algorithm. In: 2009 IEEE international conference on robotics and biomimetics (ROBIO), pp 1302–1306. https://doi.org/10.1109/ROBIO.2009.5420717

  66. Zheng C, Jiao L (2004) Automatic parameters selection for svm based on ga. In: Fifth world congress on intelligent control and automation, vol 2, pp 1869–1872. Hangzhou, P.R. China. https://doi.org/10.1109/WCICA.2004.1341000

Download references

Author information

Authors and Affiliations

  1. Department of Computer Engineering, North Tehran Branch, Islamic Azad University, Tehran, Iran

    Babak Afshin & Kamran Layeghi

  2. Department of Mathematics and Computer Sciences, Amirkabir University of Technology, Tehran, Iran

    Mohammad Ebrahim Shiri

  3. Department of Mathematics and Computer Sciences, Shahed University, Tehran, Iran

    Hamid Haj Seyyed Javadi

Authors
  1. Babak Afshin
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Mohammad Ebrahim Shiri
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Kamran Layeghi
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Hamid Haj Seyyed Javadi
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Mohammad Ebrahim Shiri.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Afshin, B., Shiri, M.E., Layeghi, K. et al. Kernel Optimization for Reducing Core Vector Machine Classification Error. Neural Process Lett 55, 10011–10036 (2023). https://doi.org/10.1007/s11063-023-11236-x

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11063-023-11236-x

Keywords

Search

Navigation