Abstract
Support vector machine (SVM) has attracted considerable attentions recently due to its successful applications in various domains. However, by maximizing the margin of separation between the two classes in a binary classification problem, the SVM solutions often suffer two serious drawbacks. First, SVM separating hyperplane is usually very sensitive to training samples since it strongly depends on support vectors which are only a few points located on the wrong side of the corresponding margin boundaries. Second, the separating hyperplane is equidistant to the two classes which are considered equally important when optimizing the separating hyperplane location regardless the number of training data and their dispersions in each class. In this paper, we propose a new SVM solution, adjusted support vector machine (ASVM), based on a new loss function to adjust the SVM solution taking into account the sample sizes and dispersions of the two classes. Numerical experiments show that the ASVM outperforms conventional SVM, especially when the two classes have large differences in sample size and dispersion.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Batista, G., Prati, R. C., & Monard, M. C. (2004). A study of the behavior of several methods for balancing machine learning training data. SIGKDD Explorations, 6(1), 20–29.
Boser, B. E., Guyon, I. M., & Vapnik, V. N. (1992). A training algorithm for optimal margin classifiers. In 5th annual ACM workshop on COLT. Pittsburgh, PA.
Burges, C. J. C. (1998). A tutorial on support vector machines for pattern recognition. Knowledge Discovery and Data Mining, 2(2), 121–167.
Chawla, N. V., Japkowicz, N., & Kolcz, A. (2004). Editorial: special issue on learning from imbalanced data sets. SIGKDD Explorations, 6(1), 1–6.
Cortes, C., & Vapnik, V. (1995). Support vector networks. Machine Learning, 20, 273–297.
Eitrich, T., & Lang, B. (2006). Efficient optimization of support vector machine learning parameters for unbalanced datasets. Journal of Computational and Applied Mathematics, 196, 425–436.
Elkan, C. (2001). The foundations of cost-sensitive learning. In Proceedings of the seventeenth international conference on machine learning (pp. 239–246).
Guyon, I., Boser, B., & Vapnik, V. (1993). Automatic capacity tuning of very large VC-dimension classifiers. Advances in Neural Information Processing Systems, 5, 147–155.
Guyon, I., Weston, J., Barnhill, S., & Vapnik, V. (2002). Gene selection for cancer classification using support vector machines. Machine Learning, 46, 389–422.
Hall, P., Marron, J., & Neeman, A. (2005). Geometric representation of high dimension, low sample size data. Journal of the Royal Statistical Society Series B, 67, 427–444.
Hastie, T., Tibshirani, R., & Friedman, J. (2001). The elements of statistical learning: data mining, inference, and prediction. Berlin: Springer.
Lee, Y., & Mangasarian, O. L. (2001). SSVM: smooth support vector machine for classification. Computational Optimization and Applications, 20(1), 5–22.
Lin, Y. (2002a). Support vector machines and the Bayes rule in classification. Data Mining and Knowledge Discovery, 6(3), 259–275.
Lin, Y. (2002b). A note on margin-based loss function in classification (Technical Report). Department of Statistics, University of Wisconsin.
Lin, Y., Lee, Y., & Wahba, G. (2002). Support vector machines for classification in nonstandard situations. Machine Learning, 46, 191–202.
Luenberger, D. G. (1984). Linear and nonlinear programming (2nd ed.). Reading: Addison-Wesley.
Osuna, E., Freund, R., & Girosi, F. (1997). Support vector machines: training and applications (AI Memo 1602). MIT.
Shen, X., Tseng, G. C., Zhang, X., & Wong, W. H. (2003). On ψ-learning. Journal of American Statistical Association, 98, 724–734.
Statnikov, A., Aliferis, C., Tsamardinos, I., Hardin, D., & Levy, S. (2005). A comprehensive evaluation of multicategory classification methods for microarray gene expression cancer diagnosis. Bioinformatics, 21(5), 631–643.
Vapnik, V. (1998). Statistical learning theory. New York: Wiley.
Weiss, G. M. (2004). Mining with rarity: a unifying framework. SIGKDD Explorations, 6(1), 7–19.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Wang, S., Jiang, W. & Tsui, KL. Adjusted support vector machines based on a new loss function. Ann Oper Res 174, 83–101 (2010). https://doi.org/10.1007/s10479-008-0495-y
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10479-008-0495-y