Abstract
MRBP algorithm is a training algorithm based on the MapReduce model for Back Propagation Network Networks (BPNNs), that employs the data parallel capability of the MapReduce model to improve the training efficiency and has shown a good performance for training BPNNs with massive training patterns. However, it is a coarse-grained pattern parallel algorithm and lacks the capability of fine-grained structure parallelism. As a result, when training a large scale BPNN, its training efficiency is still insufficient. To solve this issue, this paper proposes a novel MRBP algorithm, Fine-grained Parallel MRBP (FP-MRBP) algorithm, which has the capability of fine-grained structure parallelism. To the best knowledge of the authors, it is the first time to introduce the fine-grained parallelism to the classic MRBP algorithm. The experimental results show that our algorithm has a better training efficiency when training a large scale BPNN.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Gupta, J.N.D., Sexton, R.S.: Comparing backpropagation with a genetic algorithm for neural network training. Omega 27(6), 679–684 (1999)
Yang, S.E., Huang, L.: Financial crisis warning model based on BP neural network. Syst. Eng.-Theory Pract. 25(12), 12–19 (2005)
Li, J., Cheng, J., Shi, J., Huang, F.: Brief introduction of back propagation neural network algorithm and its improvement. In: Jin, D., Lin, S. (eds.) Advances in Computer Science and Information Engineering. AINSC, vol. 169, pp. 553–558. Springer, Heidelberg (2012). doi:10.1007/978-3-642-30223-7_87
Dean, J., Ghemawat, S.: MapReduce: simplified data processing on large clusters. In: Proceedings of the Conference on Symposium on Opearting Systems Design & Implementation, pp. 107–113 (2004)
White, T.: Hadoop: The Definitive Guide. O’Reilly Media Inc., Sebastopol (2012)
Chu, C., Kim, S.K.: Map-Reduce for machine learning on multicore. In: Advances in Neural Information Processing Systems, vol. 19, pp. 281–288 (2006)
Liu, Z., Li, H., Miao, G.: MapReduce-based backpropagation neural network over large scale mobile data. In: International Conference on Natural Computation, ICNC 2010, 10–12 August 2010, Yantai, Shandong, China, pp. 1726–1730 (2010)
Liu, Y., Yang, J., Huang, Y., Xu, L., Li, S., Qi, M.: MapReduce based parallel neural networks in enabling large scale machine learning. Comput. Intell. Neurosci. 1–13, 2016 (2015)
Turchenko, V.: Computational grid vs. parallel computer for coarse-grain parallelization of neural networks training. In: Meersman, R., Tari, Z., Herrero, P. (eds.) OTM 2005. LNCS, vol. 3762, pp. 357–366. Springer, Heidelberg (2005). doi:10.1007/11575863_55
Turchenko, V., Paliy, I., Demchuk, V.: Coarse-grain parallelization of neural network-based face detection method. In: Proceedings of the 4th IEEE Workshop on Intelligent Data Acquisition, 6–8 September 2007, pp. 155–158 (2007)
Turchenko, V., Grandinetti, L.: Efficiency analysis of parallel batch pattern NN training algorithm on general-purpose supercomputer. In: Proceedings of the International Work-Conference on Artificial Neural Networks, pp. 223–226 (2009)
Sudhakar, V., Murthy, C.S.R.: Efficient mapping of backpropagation algorithm onto a network of workstations. IEEE Trans. Syst. Man Cybern. 28(6), 841–848 (1998)
Suresh, S., Omkar, S.N., Mani, V.: Parallel implementation of back-propagation algorithm in networks of workstations. IEEE Trans. Parallel Distrib. Syst. 16(1), 24–34 (2005)
Ganeshamoorthy, K., Ranasinghe, D.N.: On the performance of parallel neural network implementations on distributed memory architectures. In: Proceedings of the IEEE International Symposium on Cluster Computing, pp. 90–97 (2008)
Chu, L., Wah, B.W.: Optimal mapping of neural-network learning on message-passing multicomputers. J. Parallel Distrib. Comput. 14(3), 319–339 (1992)
Acknowledgment
This work was supported by National Science Foundation of China (Nos. 61100066, 91530323), National Key R&D Plan of China (No. 2016YFB0200603). The authors would like express sincere gratitude to all the authors of the references in this paper. The authors also extend their thanks to all anonymous referees for providing valuable comments on this paper.
Author information
Authors and Affiliations
Corresponding authors
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2017 Springer International Publishing AG
About this paper
Cite this paper
Ren, G., Hua, Q., Deng, P., Yang, C. (2017). FP-MRBP: Fine-grained Parallel MapReduce Back Propagation Algorithm. In: Lintas, A., Rovetta, S., Verschure, P., Villa, A. (eds) Artificial Neural Networks and Machine Learning – ICANN 2017. ICANN 2017. Lecture Notes in Computer Science(), vol 10614. Springer, Cham. https://doi.org/10.1007/978-3-319-68612-7_77
Download citation
DOI: https://doi.org/10.1007/978-3-319-68612-7_77
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-68611-0
Online ISBN: 978-3-319-68612-7
eBook Packages: Computer ScienceComputer Science (R0)