Skip to main content
SpringerLink
Log in

Fast clustering-based weighted twin support vector regression

  • Foundations
  • Published:
Soft Computing Aims and scope Submit manuscript

Abstract

Construction of an effective model for regression to fit data samples with noise or outlier is a challenging work. In this paper, in order to reduce the influence of noise or outlier on regression and further improve the prediction performance of standard twin support vector regression (TSVR), we proposed a fast clustering-based weighted TSVR, termed as FC-WTSVR. First, we use a fast clustering algorithm to quickly classify samples into different categories based on their similarities. Secondly, to reflect the prior structural information and distinguish contributions of samples located at different positions to regression, we introduce the covariance matrix and weighted diagonal matrix into the primal problems of FC-WTSVR, respectively. Finally, to shorten the training time, we adopt the successive over-relaxation algorithm to solve the quadratic programming problems. The results show that the proposed FC-WTSVR can obtain better prediction performance and anti-interference capability than some state-of-the-art algorithms.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

References

  • Anagha P, Balasundaram S, Meena Y (2018) On robust twin support vector regression in primal using squared pinball loss. J Intell Fuzzy Syst 35(5):5231–5239

    Google Scholar 

  • Balasundaram S, Gupta D (2014) Training Lagrangian twin support vector regression via unconstrained convex minimization. Knowl Based Syst 59:85–96

    MATH  Google Scholar 

  • Bruno WJ, Socci ND, Halpern AL (2000) Weighted neighbor joining: a likelihood-based approach to distance-based phylogeny reconstruction. Mol Biol Evol 17(1):189–197

    Google Scholar 

  • Chang KW, Hsieh CJ, Lin CJ (2008) Coordinate descent method for large-scale L2-loss linear support vector machines. J Mach Learn Res 9(3):1369–1398

    MathSciNet  MATH  Google Scholar 

  • Chen XB, Yang J, Liang J, Ye QL (2012) Smooth twin support vector regression. Neural Comput Appl 21(3):505–513

    Google Scholar 

  • Chen XB, Yang J, Chen L (2014) An improved robust and sparse twin support vector regression via linear programming. Soft Comput 18(12):2335–2348

    MATH  Google Scholar 

  • Chen SG, Gao JF, Huang Z (2019) Weighted linear loss projection twin support vector machine for pattern classification. IEEE Access 7:57349–57360

    Google Scholar 

  • Cheng HX, Wang J (2016) Density-weighted twin support vector regression. Control Decis 31(4):755–758

    MATH  Google Scholar 

  • Cristianini N, Shawe-Talyor J (2000) An introduction to support vector machines and other kernel-based learning methods. Cambridge University Press, Cambridge

    MATH  Google Scholar 

  • Deng NY, Tian YJ (2009) Support vector machine: theory, algorithm and extension. Science Press, Beijing

    Google Scholar 

  • Fang JW, Pan F, Gu BJ (2019) Twin support vector regression based on fruit fly optimization algorithm. Int J Innov Comput Inf Control 15(5):1851–1864

    Google Scholar 

  • Gu BJ, Pan F (2015) A soft sensor modelling of biomass concentration during fermentation using accurate incremental online ν-support vector regression learning algorithm. Am J Biochem Biotechnol 1(3):149–159

    Google Scholar 

  • Gu BJ, Shen GL, Pan F, Chen H (2019) Least squares twin projection support vector regression. Int J Innov Comput Inf Control 15(6):2275–2288

    Google Scholar 

  • Gupta U, Gupta D (2019) An improved regularization based Lagrangian asymmetric ν-twin support vector regression using pinball loss function. Appl Intell 49(10):3606–3627

    Google Scholar 

  • Hua XP, Xu S, Gao J, Ding SF (2019) L1-norm loss-based projection twin support vector machine for binary classification. Soft Comput 23(21):10649–10659

    Google Scholar 

  • Jayadeva KR, Chandra S (2007) Twin support vector machines for pattern classification. IEEE Trans Pattern Anal Mach Intell 29(5):905–910

    MATH  Google Scholar 

  • Jiang SY, Song XY, Wang H, Han JJ, Li QH (2006) A clustering-based method for unsupervised intrusion detections. Pattern Recogn Lett 27(7):802–810

    Google Scholar 

  • Kalidas Y, Chandra N (2008) Pocketdepth: a new depth based algorithm for identification of ligand binding sites in proteins. J Struct Biol 161(1):0–42

    Google Scholar 

  • Liu R, Wang H, Yu XM (2018) Shared-nearest-neighbor-based clustering by fast search and find of density peaks. Inf Sci 450:200–226

    MathSciNet  Google Scholar 

  • López J, Maldonado S (2018) Robust twin support vector regression via second-order cone programming. Knowl Based Syst 152:83–93

    Google Scholar 

  • López J, Barbero Á, Dorronsoro JR (2011) Clipping algorithms for solving the nearest point problem over reduced convex hulls. Pattern Recogn 44(3):607–614

    MATH  Google Scholar 

  • Mangasarian OL, Musicant DR (2001) Lagrangian support vector machines. J Mach Learn Res 1(3):161–177

    MathSciNet  MATH  Google Scholar 

  • Mavroforakis ME, Theodoridis S (2006) A geometric approach to support vector machine (SVM) classification. IEEE Trans Neural Netw 17(3):671–682

    Google Scholar 

  • Niu JY, Chen J, Xu YT (2017) Twin support vector regression with Huber loss. J Intell Fuzzy Syst 32(6):4247–4258

    MATH  Google Scholar 

  • Pan XL, Luo Y, Xu YT (2015) K-nearest neighbor based structural twin support vector machine. Knowl Based Syst 88:34–44

    Google Scholar 

  • Pang XY, Xu YT (2019) A safe screening rule for accelerating weighted twin support vector machine. Soft Comput 23(17):7725–7739

    MATH  Google Scholar 

  • Pang XY, Xu C, Xu YT (2018) Scaling KNN multi-class twin support vector machine via safe instance reduction. Knowl Based Syst 148:17–30

    Google Scholar 

  • Parastalooi N, Amiri A, Aliheydari P (2016) Modified twin support vector regression. Neurocomputing 211:84–97

    Google Scholar 

  • Peng XJ (2010) TSVR: an efficient twin support vector machine for regression. Neural Netw 23(3):365–372

    MATH  Google Scholar 

  • Peng XJ, Wang YF, Xu D (2013) Structural twin parametric-margin support vector machine for binary classification. Knowl Based Syst 49:63–72

    Google Scholar 

  • Peng XJ, Xu D, Shen JD (2014) A twin projection support vector machine for data regression. Neurocomputing 138:131–141

    Google Scholar 

  • Peng XJ, Xu D, Kong LY, Chen DJ (2016) L1-norm loss based twin support vector machine for data recognition. Inf Sci 340–341:86–103

    MATH  Google Scholar 

  • Platt J (2000) Fast training of support vector machines using sequential minimal optimization. MIT Press, Cambridge

    Google Scholar 

  • Qi ZQ, Tian YJ, Shi Y (2013) Structural twin support vector machine for classification. Knowl Based Syst 43:74–81

    Google Scholar 

  • Quan Y, Yang J, Yao LX, Ye CZ (2004) Successive over-relaxation for support vector regression. J Softw 15(2):200–206

    MATH  Google Scholar 

  • Rodriguez A, Laio A (2014) Clustering by fast search and find of density peaks. Science 344(6191):1492–1496

    Google Scholar 

  • Shao YH, Zhang CH, Yang ZM, Ling J, Deng NY (2013) An ε-twin support vector machine for regression. Neural Comput Appl 23(1):175–185

    Google Scholar 

  • Shao YH, Chen WJ, Zhang JJ, Wang Z, Deng NY (2014) An efficient weighted Lagrangian twin support vector machine for imbalanced data classification. Pattern Recogn 47(9):3158–3167

    MATH  Google Scholar 

  • Tanveer M, Sharma A, Suganthan PN (2019a) General twin support vector machine with pinball loss function. Inf Sci 494:311–327

    MathSciNet  Google Scholar 

  • Tanveer M, Tiwari A, Choudhary R, Jalan S (2019b) Sparse pinball twin support vector machines. Appl Soft Comput J 78:164–175

    Google Scholar 

  • Vapnik VN (1999) An overview of statistical learning theory. IEEE Trans Neural Netw 10(5):988–999

    Google Scholar 

  • Wang LD, Gao C, Zhao NN, Chen XB (2019) A projection wavelet weighted twin support vector regression and its primal solution. Appl Intell 49(8):3061–3081

    Google Scholar 

  • Xu YT, Wang LS (2012) A weighted twin support vector regression. Knowl Based Syst 33:92–101

    MathSciNet  Google Scholar 

  • Xu YT, Wang LS (2014) K-nearest neighbor-based weighted twin support vector regression. Appl Intell 41(1):299–309

    Google Scholar 

  • Xu GB, Cao Z, Hu BG, Principe JC (2017) Robust support vector machines based on the rescaled hinge loss function. Pattern Recogn 63:139–148

    MATH  Google Scholar 

  • Xue ZX, Zhang RX, Qin CD, Zeng XQ (2018) A rough υ-twin support vector regression machine. Appl Intell 48(11):4023–4046

    Google Scholar 

  • Ye YF, Cao H, Bai L, Wang Z, Shao YH (2013) Exploring determinants of inflation in china based on L1-ε-twin support vector regression. Proc Comput Sci 17:514–522

    Google Scholar 

  • Ye YF, Bai L, Hua XY, Shao YH, Wang Z et al (2016) Weighted Lagrange ε-twin support vector regression. Neurocomputing 197:53–68

    Google Scholar 

  • Yeung DS, Wang DF, Ng WWY, Tsang ECC, Wang XZ (2007) Structured large margin machines: sensitive to data distributions. Mach Learn 68(2):171–200

    Google Scholar 

Download references

Acknowledgement

This research was funded by the National Natural Science Foundation of China (21878124, 31771680 and 61773182).

Author information

Authors and Affiliations

  1. Key Laboratory of Advanced Process Control for Light Industry, Ministry of Education, Jiangnan University, Wuxi, 214122, China

    Binjie Gu, Jianwen Fang & Feng Pan

  2. National Engineering Laboratory for Cereal Fermentation Technology, Jiangnan University, Wuxi, 214122, China

    Zhonghu Bai

Authors
  1. Binjie Gu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jianwen Fang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Feng Pan
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Zhonghu Bai
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Binjie Gu.

Ethics declarations

Conflict of interest

The authors declare no conflict of interests.

Ethical approval

This article does not contain any studies with human participants or animals performed by any of the authors.

Informed consent

Informed consent was obtained from all individual participants included in the study.

Additional information

Communicated by A. Di Nola.

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Gu, B., Fang, J., Pan, F. et al. Fast clustering-based weighted twin support vector regression. Soft Comput 24, 6101–6117 (2020). https://doi.org/10.1007/s00500-020-04746-6

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00500-020-04746-6

Keywords

Search

Navigation