Regular Article
Measurement and Prediction of Communication Delays in Myrinet Networks

https://doi.org/10.1006/jpdc.2001.1761Get rights and content

Abstract

This paper describes a series of experiments carried out to determine if it is possible to accurately predict the delays of inter-node communication in a PC cluster system interconnected with a Myrinet switch network. Prediction accuracy is affected not only by the software and hardware overhead involved in network communication, but also interference from concurrent message streams. Based on extensive measurements using a 14-node Myrinet cluster system, it is determined that (1) the simple linear model typically used to model communication delay in networks is insufficient and (2) communication delay behavior with n message streams sharing a common link is more complicated than a simple divide-by-n solution. A piecewise-linear model, based on parameters obtained through experiments, is proposed as a more accurate communication delay prediction method when there is no sharing of communication links. However, if two or more message streams share a common link, then the communication delay is more accurately predicted as being one of a set of discrete values.

References (8)

  • N.J. Boden et al.

    Myrinet—A gigabit per second local area network

    IEEE Micro.

    (Feb. 1995)
  • R.A.F. Bhoedjang et al.

    User-level network interface protocols

    Computer

    (Nov. 1998)
  • L. Prylli et al.

    Technical Report

    (1997)
There are more references available in the full text version of this article.

Cited by (16)

  • Extending τ-Lop to model concurrent MPI communications in multicore clusters

    2016, Future Generation Computer Systems
    Citation Excerpt :

    The authors use the model to study the effects of multi-stage switches on that kind of networks in [31]. Paper [32] analytically estimates the communication delays in Myrinet networks, and the work in [33] addresses hierarchical Ethernet networks. Regarding contention modeling, a sound work studying this issue in high performance networks is carried out in [35] for Infiniband and in [36,37] for Ethernet and Myrinet networks.

  • Wave-based passive control for transparent micro-teleoperation system

    2006, Robotics and Autonomous Systems
    Citation Excerpt :

    Although the presented varying compensation gain has been successfully validated on a piece-linear model [20], our future work will consist of developing realistic models of network delays and using these models to develop switching strategies for adjusting the communication gains on-line for unknown varying time-delays such as those encountered in internet networks.

  • An Oracle for Guiding Large-Scale Model/Hybrid Parallel Training of Convolutional Neural Networks

    2021, HPDC 2021 - Proceedings of the 30th International Symposium on High-Performance Parallel and Distributed Computing
View all citing articles on Scopus

This research was supported by Korea Research Foundation Grant KRF-1998-016-E00069.

2

To whom correspondence should be addressed.

View full text