Abstract
Extreme Learning Machine(ELM) proposed by Huang et al is a new and simple algorithm for single hidden layer feedforward neural network(SLFN) with extreme fast learning speed and good generalization performance.When new hidden nodes are added to existing network retraining the network would be time consuming, and EM-ELM is proposed to calculate the output weight incrementally.However there are still two issues in EM-ELM:1.the initial hidden layer output matrix may be nearly singular thus the computation will loss accuracy;2.the algorithms can’t always get good generalization performance due to overfitting.So we propose the improved version of EM-ELM based on regularization method called Incremental Regularized Extreme Learning Machine(IR-ELM).When new hidden node is added one by one,IR-ELM can update output weight recursively in a fast way.Empirical studies on benchmark data sets for regression and classification problems have shown that IR-ELM always get better generalization performance than EM-ELM with the similar training time.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Huang, G.B., Zhu, Q.Y., Siew, C.K.: Extreme Learning Machine: a new Learning Scheme of Feedforward Neural Networks. In: Proceedings. 2004 IEEE International Joint Conference on Neural Networks, vol. 2. IEEE (2004)
Huang, G.B., Zhu, Q.Y., Siew, C.K.: Extreme Learning Machine: Theory and Applications. Neurocomputing 70, 489–501 (2006)
Huang, G.B., Zhu, Q.Y., Siew, C.K.: Real-time Learning Capability of Neural Networks. IEEE Transactions on Neural Networks 17(4), 863–878 (2006)
Rumelhart, D.E., Hinton, G.E., Williams, R.J.: Learning Representations by Back-propagating Errors. MIT Press, Cambridge (1988)
Huang, G.B., Zhou, H., Ding, X.: Extreme Learning Machine for Regression and Multiclass Classification. IEEE Transactions on Systems, Man, and Cybernetics, Part B: Cybernetics 42(2), 513–529 (2012)
Sun, Z.L., Choi, T.M., Au, K.F.: Sales Forecasting Using Extreme Learning Machine with Applications in Fashion Retailing. Decision Support Systems 46(1), 411–419 (2008)
Nizar, A.H., Dong, Z.Y., Wang, Y.: Power Utility Nontechnical Loss Analysis with Extreme Learning Machine Method. IEEE Transactions on Power Systems 23(3), 946–955 (2008)
Zhang, R., Huang, G.B., Sundararajan, N.: Multicategory Classification Using an Extreme Learning Machine for Microarray Gene Expression Cancer Diagnosis. IEEE/ACM Transactions on Computational Biology and Bioinformatics 4(3), 485–495 (2007)
Huang, G.B., Chen, L., Siew, C.K.: Universal Approximation Using Incremental Constructive Feedforward Networks with Random Hidden Nodes. IEEE Transactions on Neural Networks 17(4), 879–892 (2006)
Feng, G., Huang, G.B., Lin, Q.: Error Minimized Extreme Learning Machine with Growth of Hidden Nodes and Incremental Learning. IEEE Transactions on Neural Networks 20(8), 1352–1357 (2009)
Deng, W., Zheng, Q., Chen, L.: Regularized Extreme Learning Machine. Computational Intelligence and Data Mining. In: IEEE Symposium on. IEEE CIDM 2009, pp. 389–395 (2009)
Vapnik, V.: The Nature of Statistical Learning Theory. Springer, 2000
Vapnik, V.N.: An Overview of Statistical Learning Theory. IEEE Transactions on Neural Networks 10(5), 988–999 (1999)
Frank, A., Asuncion, A.: UCI Machine Learning Repository. Univ. California, Sch. Inform. Comput. Sci., Irvine, CA (2011), http://archive.ics.uci.edu/ml
Martnez-Martnez, J.M., Escandell-Montero, P., Soria-Olivas, E.: Regularized Ex- treme Learning Machine for Regression Problems. Neurocomputing 74(17), 3716–3721 (2011)
Haykin, S.S.: Neural Networks and Learning Machines. Pearson Education, Upper Saddle River (2009)
Bartlett, P.L.: The Sample Complexity of Pattern Classification with Neural Networks: the Size of the Weights is More Important than the Size of the Network. IEEE Transactions on Information Theory 44(2), 525–536 (1998)
Girosi, F., Jones, M., Poggio, T.: Regularization Theory and Neural Networks Architectures. Neural Computation 7(2), 219–269 (1995)
Jin, Y., Okabe, T., Sendhoff, B.: Neural Network Regularization and Ensembling Using Multi-objective Evolutionary Algorithms. In: Congress on Evolutionary Computation, CEC 2004, vol. 1, pp. 1–8. IEEE (2004)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2015 Springer International Publishing Switzerland
About this paper
Cite this paper
Xu, Z., Yao, M. (2015). A Fast Incremental Method Based on Regularized Extreme Learning Machine. In: Cao, J., Mao, K., Cambria, E., Man, Z., Toh, KA. (eds) Proceedings of ELM-2014 Volume 1. Proceedings in Adaptation, Learning and Optimization, vol 3. Springer, Cham. https://doi.org/10.1007/978-3-319-14063-6_2
Download citation
DOI: https://doi.org/10.1007/978-3-319-14063-6_2
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-14062-9
Online ISBN: 978-3-319-14063-6
eBook Packages: EngineeringEngineering (R0)