Abstract
Multicasting large amounts of data efficiently to all nodes of a PC cluster is an important operation. In the form of a partition cast it can be used to replicate entire software installations by cloning. Optimizing a partition cast for a given cluster of PCs reveals some interesting architectural tradeoffs, since the fastest solution does not only depend on the network speed and topology, but remains highly sensitive to other resources like the disk speed, the memory system performance and the processing power in the participating nodes. We present an analytical model that guides an implementation towards an optimal configuration for any given PC cluster. The model is validated by measurements on our cluster using Gigabit- and Fast Ethernet links. The resulting simple software tool, Dolly, can replicate an entire 2 GByte Windows NT image onto 24 machines in less than 5 minutes.
Chapter PDF
Keywords
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.
References
Henri Bal. The Distributed ASCI Supercomputer (DAS). http://www.cs.vu.nl/~bal/das.html.
D. J. Becker, T. Sterling, D. Savarese, J. E. Dorband, U. A. Ranawake, and C. V. Packer. Beowulf: A Parallel Workstation for Scientific Computation. In Proceedings of 1995 ICPP Workshop on Challenges for Parallel Processing, Oconomowc, Wisconsin, U.S.A., August 1995. CRC Press.
Nanette J. Boden, Robert E. Felderman, Alan E. Kulawik, Charles L. Seitz, Jakov N. Seizovic, and Wen-King Su. Myrinet — A Gigabit per Second Local Area Network. IEEE-Micro, 15(1):29–36, February 1995.
William J. Bolosky, Joseph S. Barrera III, Richard P. Draves, Robert P. Fitzgerald, Garth A. Gibson, Michael B. Jones, Steven P. Levi, Nathan P. Myhrvold, and Richard F. Rashid. The Tiger Video Fileserver. In Sixth International Workshop on Network and Operating System Support for Digital Audio and Video, Zushi, Japan, April 1996. IEEE Computer Society.
Dolphin Interconnect Solutions. PCI SCI Cluster Adapter Specification, 1996.
S. Floyd, V. Jacobson, S. McCanne, L. Zhang, and C-G. Liu. A Reliable Multicast Framework For Lightweight Sessions and Application Level Framing. In Proceedings of ACM SIGCOMM’ 95, pages 342–356, August 1995.
H. Hellwagner and A. Reinefeld, editors. SCI Based Cluster Computing. Springer, Berlin, Spring 1999.
Norman C. Hutchinson, Stephen Manley, Mike Federwisch, Guy Harris, Dave Hitz, Steven Kleiman, and Sean O Malley. Logical vs. Physical File System Backup. In Proceedings of the 3rd Symposium on Operating Systems Design and Implementation, New Orleans, Louisiana, pages 239–249. The USENIX Association, February 1999.
Steve Kotsopoulos and Jeremy Cooperstock. Why Use a Fishing Line When you Have a Net? An Adaptive Multicast Data Distribution Protocol. In Proccedings of the USENIX 1996 Annual Technical Conference, San Diego, California, January 1996. The USENIX Association.
Sanjoy Paul, Krishan K. Sabnani, and David M. Kristol. Multicast Transport protocols for High Speed Networks. In Proceedings of International Conference on Network Protocols, pages 4–14. IEEE Computer Society Press, 1994.
F. Rauch. Zuverlassiges Multicastprotokoll. Master’s thesis, ETH Zurich, 1997. English title: Reliable Multicast Protocol. See also http://www.cs.inf.ethz.ch/. Contains a survey about reliable IP multicast.
Felix Rauch, Christian Kurmann, Thomas Stricker, and Blanca Maria Müller. Patagonia — A Dual Use Cluster of PCs for Computation and Education. In 2. Workshop Cluster Computing, Karlsruhe, March 1999.
Rich Seifert. Gigabit Ethernet: Technology and Applications for High-Speed LANs. Addison-Wesley, May 1998. ISBN: 0201185539.
T. Stricker and T. Gross. Optimizing Memory System Performance for Communication in Parallel Computers. In Proc. 22nd Intl. Symposium on Computer Architecture, pages 308–319, Santa Marguerita di Ligure, June 1995. ACM.
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2000 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Rauch, F., Kurmann, C., Stricker, T.M. (2000). Partition Cast — Modelling and Optimizing the Distribution of Large Data Sets in PC Clusters. In: Bode, A., Ludwig, T., Karl, W., Wismüller, R. (eds) Euro-Par 2000 Parallel Processing. Euro-Par 2000. Lecture Notes in Computer Science, vol 1900. Springer, Berlin, Heidelberg. https://doi.org/10.1007/3-540-44520-X_159
Download citation
DOI: https://doi.org/10.1007/3-540-44520-X_159
Published:
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-67956-1
Online ISBN: 978-3-540-44520-3
eBook Packages: Springer Book Archive