Abstract
An ensemble of neural networks exhibits higher generalization performance compared to a single neural network. In this paper, a new design method for a neural network ensemble is proposed. The hierarchical pair competition-based parallel genetic algorithm (HFC-PGA) is employed to train the neural networks forming the ensemble. The aim of the HFC-PGA is to achieve not only the best neural network, but also a diversity of potential neural networks. A set of component neural networks is selected to build an ensemble such that the generalization error is minimized and the negative correlation is maximized. Finally, some experiments are carried out using several data sets to illustrate and quantify the performance of the proposed method.
Similar content being viewed by others
References
Hassan YF (2011) Rough sets for adapting wavelet neural networks as a new classifier system. Appl Intell 35(2):260–268
Bruzzone L, Prieto DF, Serpico SB (1999) A neural-statistical approach to multitemporal and multisource remote-sensing image classification. IEEE Trans Geosci Remote Sens 37:1350–1359
Dong G, Xie M (2005) Color clustering and learning for image segmentation based on neural networks. IEEE Trans Neural Netw 16:925–936
Tumar K, Ghosh J (1996) Error correlation and error reduction in ensemble classifiers. Connect Sci 8:385–404
Optiz D, Maclin R (1999) Popular ensemble methods: an empirical study. J Artif Intell Res 11:169–198
Hansen L, Salamon P (1992) Neural network ensembles. IEEE Trans Pattern Anal Mach Intell 12:333–342
Huang F, Zhou Z, Zhang H, Chen T (2000) Pose invariant face recognition. In: Proc 4th IEEE int conf automatic face and gesture recognition, pp 245–250
Hansen L, Liisberg L, Salamon P (1992) Ensemble methods for handwritten digit recognition. In: Proc IEEE workshop on neural networks for signal processing, pp 333–342
Cherkauer K (1996) Human expert level performance on a scientific image analysis task by a system using combined artificial neural networks. In: Proc workshop on integrating multiple learned models for improving and scaling machine learning algorithms, pp 15–21
Zhou Z, Jiang Y, Yang Y, Chen S (2002) Lung cancer cell identification based on artificial neural network ensembles. Artif Intell Med 24, 25–36
Lee H, Kim E, Park M (2007) A genetic feature weighting scheme for pattern recognition. Integr Comput-Aided Eng 14(2):161–171
Korkmaz EE (2010) Multi-objective genetic algorithms for grouping problems. Appl Intell 33(2):179–192
Kim K-J, Cho S-B (2008) Evolutionary ensemble of diverse artificial neural networks using speciation. Neurocomputing 71(7–9):1604–1608
Ishigami H, Fukuda T, Shibata T, Arai F (1995) Structure optimization of fuzzy neural network by genetic algorithm. Fuzzy Sets Syst 71:257–264
Leung F, Lam H, Tam K (2003) Tuning of the structure and parameters of a neural network using an improved genetic algorithm. IEEE Trans Neural Netw 14(1):79–88
Tsai J, Chou J, Liu T (2006) Tuning the structure and parameters of a neural network by using hybrid Taguchi-genetic algorithm. IEEE Trans Neural Netw 17(1):69–80
Kang J-G, Kim S, An S-Y, Oh S-Y (2012) A new approach to simultaneous localization and map building with implicit model learning using neuro evolutionary optimization. Appl Intell 36(1):242–269
Yao X (1999) Evolving artificial neural networks. Proc IEEE 87(9):1423–1447
Optitz D, Shavlik J (1999) Generating accurate and diverse members of a neural-network ensemble. Adv Neural Inf Process Syst 8:535–541
Rogers A, Prugel-Bennett A (1999) Genetic drift in genetic algorithm selection schemes. IEEE Trans Evol Comput 3(4):298–303
Choi J, Oh S, Pedrycz W (2008) Structural and parametric design of fuzzy inference systems using hierarchical fair competition-based parallel genetic algorithms and information granulation. Int J Approx Reason 49:631–648
Michalewicz Z (1996) Genetic algorithms + data structures = evolution programs. Springer, Berlin
Wang H, Chen S, Zheng W (2008) Locality-preserved maximum information projection. IEEE Trans Neural Netw 19(4):571–585
Hong J, Cho S (2006) Efficient huge-scale feature selection with speciated genetic algorithm. Pattern Recognit Lett 27:143–150
Lee H, Hong S, Nizami IF, Kim E (2010) An efficient design of nearest neighbor classifier for various-scale problems. Pattern Recognit Lett 31(9):1020–1027
Juo H, Chang H (2004) A new symbiotic evolution-based fuzzy-neural approach to fault diagnosis of marine propulsion systems. Eng Appl Artif Intell 17:919–930
Lee H, Hong S, Kim E (2009) A new genetic feature selection with neural network ensemble. Int J Comput Math 86(7):1105–1117
Houck CR, Joines JA, Kay MG (1995) A genetic algorithm for function optimization: a Matlab implementation. NCSU-IE Technical Report 95-09, North Carolina State University
Canuto A, Santos A, Vergas R (2011) Ensembles of ARTMAP-based neural networks: an experimental study. Appl Intell 35(1):1–17
Hu JJ, Goodman ED (2002) The hierarchical fair competition (HFC) model for parallel evolutionary algorithms. In: Proc congress on evolutionary computation, pp 45–94
Choi J, Oh S, Pedrycz W (2008) Identification of fuzzy models using a successive tuning method with a variant identification ratio. Fuzzy Sets Syst 159:2873–2889
Liu Y, Yao X, Higuchi T (2000) Evolutionary ensembles with negative correlation learning. IEEE Trans Evol Comput 4(4):380–387
Lee H, Hong S, Kim E (2008) An efficient gait recognition based on a selective neural network ensemble. Int J Imaging Syst Technol 18(4):237–241
Kim K-J, Cho S-B (2007) Ensemble classifiers based on correlation analysis for DNA microarray classification. Appl Soft Comput 7(1):398–410
Murphy PM, Aha DW (1994) UCI repository for machine learning databases. Technical report, Dept of Information and Computer Science, Univ of California, Irvine, Calif
Lossos L et al (2000) Ongoing immunoglobulin somatic mutation in germinal center B cell-like but not in activated B cell-like diffuse large cell lymphomas. Proc Natl Acad Sci USA 97:10209–10213
Tan H et al (2002) Prediction of central nervous system embryonal tumour outcome based on gene expression. Nature 415:436–442
Hwang JP, Rou K, Park S, Kim E, Kang H (2006) PCA based vehicle detection for ACC. In: Proc 8th international conference on electronics, information, and communication, pp 98–101
Lee S-I, Ahn J-H, Cho S-B (2001) Exploiting diversity of neural ensembles with speciation evolution. In: Proc international joint conference on neural networks, vol 2, pp 808–813
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Lee, H., Kim, E. & Pedrycz, W. A new selective neural network ensemble with negative correlation. Appl Intell 37, 488–498 (2012). https://doi.org/10.1007/s10489-012-0342-3
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10489-012-0342-3