Abstract
In this paper, it is investigated simulating/implementing a fully connected multilayered feedforward neural network using the backpropagation learning algorithm on a distributed-memory multiprocessor system. Each layer is partitioned into p disjoint sets and each set is mapped on a processor of a p-processor system. A fully distributed backpropagation algorithm, necessary communication among the processors, and its time complexity are investigated. The p-processor speed-up of the backpropagation algorithm over a single processor is analyzed theoretically for some popular processor interconnection topologies, which can be used as a basis in determining the most cost-effective or optimal number of processors.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
Similar content being viewed by others
References
M. Annaratone, C. Pommerell, and R. Ruhl, “Interprocessor Communication Speed and Performance in Distributed-Memory Parallel Processors,” Proc. of the 16th Annual Intl Symp. on Computer Architecture, pp. 315–324, 1989.
DARPA Neural Network Study, AFCEA International Press, 1988.
E. Deprit, “Implementing Recurrent Back-Propagation on the Connection Machine,” Neural Networks, Vol. 2, pp. 295–314, 1989.
J. A. Feldman and et al., “Computing with Structured Connectionist Networks,” Comm. of ACM, Vol. 31, No. 2, pp. 170–187, 1988.
J. Hicklin and H. Demuth, “Modeling Neural Networks on the MPP,” Proc. of the 2nd Symp. on the Frontiers of Massively Parallel Computation, pp. 39–42, 1988.
S. L. Johnsson and C. T. Ho, “Optimum Broadcasting and Personalized Communication in Hypercubes,” IEEE Trans. on Computers, Vol. 38, No. 9, pp. 1249–1268, 1989.
D. A. Pomerleau and et al., “Neural Network Simulation at Warp Speed: How We Got 17 Million Connections Per Second”, Proc. of IEEE 2nd Intl Conf on Neural Networks, Vol. II, pp. 143–150, 1988.
D. E. Rumelhart, G. E. Hinton, and R. J. Williams, “Learning Internal Representations by Error Propagation,” In D. E. Rumelhart and J. L. McClelland (Eds.), Parallel Distributed Processing: Explorations in the Microstructure of Cognition, Vol.1, pp.318–362, MIT Press, 1987.
Author information
Authors and Affiliations
Rights and permissions
Copyright information
© 1990 Springer Science+Business Media Dordrecht
About this chapter
Cite this chapter
Yoon, H., Nang, J.H. (1990). Multilayer Neural Networks on Distributed-Memory Multiprocessors. In: International Neural Network Conference. Springer, Dordrecht. https://doi.org/10.1007/978-94-009-0643-3_37
Download citation
DOI: https://doi.org/10.1007/978-94-009-0643-3_37
Publisher Name: Springer, Dordrecht
Print ISBN: 978-0-7923-0831-7
Online ISBN: 978-94-009-0643-3
eBook Packages: Springer Book Archive