Abstract
A new multiple regression method based on optimal convex combinations of simple univariate regressions is discussed, where the simple regressions are searched with an ordinary least squares technique. Convex combination is considered optimal if it correlates with the response variable in the best way. It is shown that the developed approach is equivalent to a least squares technique variant regularized by constraints on signs of regression parameters. A method of optimal convex combination search is discussed that is based on unexpandable and irreducible ensembles. The developed method is compared with elastic networks at simulated data.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Breiman, L.: Random forests – random features. Statistics department, University of California, Berkley (1999)
Brown, G., Wyatt, J.L., Tino, P.: Diversity in Regression Ensembles. Journal of Machine Learning 6, 1621–1650 (2005)
Efron, B., Hastie, T., Jonnstone, I., Tibshirani, R.: Least Angle Regression. Annals of Statistics 32(2), 407–499 (2004)
Friedman, J.H., Hastie, T., Tibshirani, R.: Regularization Paths for Generalized Linear Models via Coordinate Descent. Journal of Statistical Software 33(1) (2010)
Kuncheva, L.I.: Combining Pattern Classifiers. Methods and Algorithms. Wiley Interscience, New Jersey (2004)
Senko, O.V.: The Use of Collective Method for Improvement of Regression Modeling Stability. InterStat. Statistics on the Internet (2004), http://statjournals.net
Senko, O.V.: An Optimal Ensemble of Predictors in Convex Correcting Procedures. Pattern Recognition and Image Analysis 19(3), 465–468 (2009)
Senko, O.V., Dokukin, A.A.: Optimal Forecasting Based on Convex Correcting Procedures. In: New Trends in Classification and Data Mining, pp. 62–72. ITHEA, Sofia (2010)
Senko, O., Dokukin, A.: On Some Properties Of Regression Models Based On Correlation Maximization Of Convex Combinations. International Journal “Information Models and Analyses” 1(1), 112–122 (2012)
Tibshirani, R.: Regression shrinkage and selection via the lasso. J. Roy. Stat. Soc. 58, 267–288 (1996)
Zhuravlev, Y.I., Kuznetsova, A.V., Ryazanov, V.V., Senko, O.V., Botvin, M.A.: The Use of Pattern Recognition Methods in Tasks of Biomedical Diagnostics and Forecasting. Pattern Recognition and Image Analysis 58(2), 195–200 (2008)
Zou, H., Hastie, T.: Regularization and variable selection via the elastic net. J. Roy. Stat. Soc. 67(2), 301–320 (2005)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2014 Springer International Publishing Switzerland
About this paper
Cite this paper
Senko, O., Dokukin, A. (2014). Multiple Regression Method Based on Unexpandable and Irreducible Convex Combinations. In: Perner, P. (eds) Machine Learning and Data Mining in Pattern Recognition. MLDM 2014. Lecture Notes in Computer Science(), vol 8556. Springer, Cham. https://doi.org/10.1007/978-3-319-08979-9_4
Download citation
DOI: https://doi.org/10.1007/978-3-319-08979-9_4
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-08978-2
Online ISBN: 978-3-319-08979-9
eBook Packages: Computer ScienceComputer Science (R0)