Hamed Masnadi-Shirazi
ABSTRACT
A cost-sensitive extension of boosting, denoted as asymmetric boosting, is presented. Unlike previous proposals, the new algorithm is derived from sound decision-theoretic principles, which exploit the statistical interpretation of boosting to determine a principled extension of the boosting loss. Similarly to AdaBoost, the cost-sensitive extension minimizes this loss by gradient descent on the functional space of convex combinations of weak learners, and produces large margin detectors. It is shown that asymmetric boosting is fully compatible with AdaBoost, in the sense that it becomes the latter when errors are weighted equally. Experimental evidence is provided to demonstrate the claims of cost-sensitivity and large margin. The algorithm is also applied to the computer vision problem of face detection, where it is shown to outperform a number of previous heuristic proposals for cost-sensitive boosting (AdaCost, CSB0, CSB1, CSB2, asymmetric-AdaBoost, AdaC1, AdaC2 and AdaC3).
- Chawla, N. V., Lazarevie, A., Hall, L. O., & Bowyer, K. (2003). Smoteboost: Improving prediction of the minority class in boosting. In Proceedings of Principles of Knowledge Discovery in Databases.Google Scholar
Cross Ref
- Domingos, P. (1999). Metacost: a general method for making classifiers cost-sensitive. Proceedings of the fifth ACM SIGKDD. ACM Press. Google ScholarDigital Library
- Duda, R., Hart, P. E., & Stork, D. (2001). Pattern classification. New York: Wiley and Sons. Google ScholarDigital Library
- Fan, W., Stolfo, S., Zhang, J., & Chan, P. (1999). Adacost: Misclassification cost-sensitive boosting. ICML. Google ScholarDigital Library
- Freund, Y., & Schapire, R. (1997). A decision-theoretic generalization of on-line learning and an application to boosting. Journal of Computer and System Sciences, 55, 119--139. Google ScholarDigital Library
- Freund, Y., & Schapire, R. (2004). A discussion of "Process consistency for AdaBoost" by Wenxin Jiang, "On the Bayes-risk consistency of regularized boosting methods" by Gabor Lugosi and Nicolas Vayatis, "Statistical behavior and consistency of classification methods based on convex risk minimization" by Tong Zhang. Annals of Statistics.Google Scholar
- Friedman, J., Hastie, T., & Tibshirani, R. (2000). Additive logistic regression: A statistical view of boosting. Journal of Annals of Statistics.Google ScholarCross Ref
- Guo, H., & Viktor, H. L. (2004). Learning from imbalanced data sets with boosting and data generation: the databoost-im approach. SIGKDD Explor. Newsl. Google ScholarDigital Library
- Hastie, Tibshirani, & Friedman (2001). The elements of statistical learning. New York: Springer-Verlag Inc.Google Scholar
- Mason, L., Baxter, J., Bartlett, P., & Frean, M. (2000). Boosting Algorithms as Gradient Descent. NIPS.Google Scholar
- Park, S.-B., Hwang, S., & Zhang, B.-T. (2003). Mining the risk types of human papillomavirus (hpv) by adacost. International Conference on Database and expert Systems Applications.Google ScholarCross Ref
- Schapire, R. E., Freund, Y., Bartlett, P., & Lee, W. S. (1998). Boosting the margin: A new explanation for the effectiveness of voting methods. The Annals of Statistics.Google Scholar
- Sun, Y., Wong, A. K. C., & Wang, Y. (2005). Parameter inference of cost-sensitive boosting algorithms. Machine Learning and Data Mining in Pattern Recognition, 4th International Conference. Google ScholarDigital Library
- Ting, K. M. (2000). A comparative study of cost-sensitive boosting algorithms. ICML. Google ScholarDigital Library
- Viaene, S., Derrig, R. A., & Dedene, G. (2004). Costsensitive learning and decision making for massachusetts pip claim fraud data. International Journal of Intelligent Systems. Google ScholarDigital Library
- Viola, P., & Jones, M. (2001). Robust real-time object detection. Proc. 2nd Intl Workshop on Statistical and Computational Theories of Vision Modeling, Learning, Computing and Sampling. Vancouver, Canada.Google Scholar
- Viola, P., & Jones, M. (2002). Fast and robust classification using asymmetric adaboost and a detector cascade. NIPS.Google Scholar
- Zadrozny, B., Langford, J., & Abe, N. (2003). A simple method for cost-sensitive learning. Technical Report RC22666, IBM.Google Scholar
- Asymmetric boosting
Recommendations
Multi-class boosting with asymmetric binary weak-learners
We introduce a multi-class generalization of AdaBoost with binary weak-learners. We use a vectorial codification to represent class labels and a multi-class exponential loss function to evaluate classifier responses. This representation produces a set ...
Boosting recombined weak classifiers
Boosting is a set of methods for the construction of classifier ensembles. The differential feature of these methods is that they allow to obtain a strong classifier from the combination of weak classifiers. Therefore, it is possible to use boosting ...
Boosting feature selection
ICAPR'05: Proceedings of the Third international conference on Advances in Pattern Recognition - Volume Part IIt is possible to reduce the error rate of a single classifier using a classifier ensemble. However, any gain in performance is undermined by the increased computation of performing classification several times. Here the AdaboostFS algorithm is proposed ...
Comments