Abstract
This paper proposes a non-linear ordinal logistic regression method based on the combination of a linear regression model and an evolutionary neural network with hybrid basis functions, combining Sigmoidal Unit and Radial Basis Functions neural networks. The process for obtaining the coefficients is carried out in several steps. Firstly we use an evolutionary algorithm to determine the structure of the hybrid neural network model, in a second step we augment the initial feature space (covariate space) adding the non-linear transformations of the input variables given by the hybrid hidden layer of the best individual of the evolutionary algorithm. Finally, we apply an ordinal logistic regression in the new feature space. This methodology is tested using 10 benchmark problems from the UCI repository. The hybrid model outperforms both the RBF and the SU pure models obtaining a good compromise between them and better results in terms of accuracy and ordinal classification error.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Angeline, P.J., Sauders, G.M., Pollack, J.B.: An evolutionary algorithm that constructs recurrent neural networks. IEEE Trans. Neural Netw. 5(1), 54–65 (1994)
Asuncion, A., Newman, D.: UCI machine learning repository (2007). http://www.ics.uci.edu/mlearn/MLRepository.html
Baccianella, S., Esuli, A., Sebastiani, F.: Evaluation measures for ordinal regression. In: Proceedings of the Ninth International Conference on Intelligent Systems Design and Applications (ISDA 09), Pisa, Italy, December 2009
Bishop, C.M.: Improving the generalization properties of radial basis function neural networks. Neural Comput. 8, 579–581 (1991)
Buchtala, O., Klimek, M., Sick, B.: Evolutionary optimization of radial basis function classifiers for data mining applications. IEEE Trans. Neural Netw. Part B 35(5), 928–947 (2005)
Chu, W., Keerthi, S.S.: Support vector ordinal regression. Neural Comput. 19(3), 792–815 (2007)
Cohen, S., Intrator, N.: A hybrid projection-based and radial basis function architecture: initial values and global optimisation. Pattern Anal. Appl. 5, 113–120 (2002)
Donoho, D.: Projection-based approximation and a duality with kernel methods. Ann. Stat. 5, 58–106 (1989)
Dorado-Moreno, M., Gutiérrez, P.A., Hervás-Martínez, C.: Ordinal classification using hybrid artificial neural networks with projection and kernel basis functions. In: Corchado, E., Snášel, V., Abraham, A., Woźniak, M., Graña, M., Cho, S.-B. (eds.) HAIS 2012, Part II. LNCS, vol. 7209, pp. 319–330. Springer, Heidelberg (2012)
Friedman, M.: A comparison of alternative tests of significance for the problem of m rankings. Ann. Math. Stat. 11, 86–92 (1940)
Gutiérrez, P.A., Hervás-Martínez, C., Carbonero-Ruz, M., Fernandez, J.C.: Combined projection and kernel basis functions for classification in evolutionary neural networks. Neurocomputing 27(13–15), 2731–2742 (2009)
Gutiérrez, P.A., Hervás-Martínez, C., Martínez-Estudillo, F.J.: Logistic regression by means of evolutionary radial basis function neural networks. IEEE Trans. Neural Netw. 22(2), 246–263 (2011)
Hastie, T., Tibshirani, R.: Generalized Additive Models. Chapman and Hall, London (1990)
Igel, C., Hüsken, M.: Empirical evaluation of the improved rprop learning algorithms. Neurocomputing 50(6), 105–123 (2003)
Lee, S.H., Hou, C.L.: An art-based construction of RBF networks. IEEE Trans. Neural Netw. 13(6), 1308–1321 (2002)
Lippmann, R.P.: Pattern classification using neural networks. IEEE Trans. Neural Netw. 27, 47–64 (1989)
Maniezzo, V.: Genetic evolution of the topology and weight distribution of neural networks. IEEE Trans. Neural Netw. 5, 39–53 (1994)
Martínez-Estudillo, A.C., Martínez-Estudillo, F.J., Hervás-Martínez, C., García, N.: Evolutionary product unit based neural networks for regression. Neural Netw. 19(4), 477–486 (2006)
Maul, T.: Early experiments with neural diversity machines. Neurocomputing 113, 136–48 (2013)
McCullagh, P.: Regression models for ordinal data (with discussion). J. Roy. Stat. Soc. 42(2), 109–142 (1980)
PASCAL: Pascal (pattern analysis, statistical modelling and computational learning) machine learning benchmarks repository (2011). http://mldata.org/
van Rooij, A.J.F., Jain, L.C., Johnson, R.P.: Neural Networks Training Using Genetic Algorithms. Series in Machine Perception and Artificial Intelligence, vol. 26. World Scientific, Singapore (1996)
Schmitt, M.: On the complexity of computing and learning with multiplicative neural networks. Neural Comput. 14, 241–301 (2001)
Smola, A., Scholkopf, B.: A tutorial on support vector regression. Stat. Comput. 14(3), 199–222 (2004)
Soltesz, I.: Diversity in the Neuronal Machine: Order and Variability in Interneuronal Microcircuits. Oxford University Press, New York (2002)
Yao, X., Liu, Y.: A new evolutionary system for evolving artificial neural networks. IEEE Trans. Neural Netw. 8, 694–713 (1997)
Yao, X.: Evolving artificial neural networks. Proc. IEEE 87(9), 1423–1447 (1999)
Acknowledgements
This work has been partially subsidised by the TIN2014-54583-C2-1-R project of the Spanish MINECO, FEDER funds and P11-TIC-7508 project of the “Junta de Andalucía(Spain)”.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2015 Springer International Publishing Switzerland
About this paper
Cite this paper
Dorado-Moreno, M., Gutiérrez, P.A., Sánchez-Monedero, J., Hervás-Martínez, C. (2015). Overcoming the Linearity of Ordinal Logistic Regression Adding Non-linear Covariates from Evolutionary Hybrid Neural Network Models. In: Puerta, J., et al. Advances in Artificial Intelligence. CAEPIA 2015. Lecture Notes in Computer Science(), vol 9422. Springer, Cham. https://doi.org/10.1007/978-3-319-24598-0_27
Download citation
DOI: https://doi.org/10.1007/978-3-319-24598-0_27
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-24597-3
Online ISBN: 978-3-319-24598-0
eBook Packages: Computer ScienceComputer Science (R0)