Abstract
The process of obtaining useful message passing applications tracefiles for performance analysis in supercomputers is a large and tedious task. When using hundreds or thousands of processors, the tracefile size can grow up to 10 or 20 GB. It is clear that analyzing or even storing these large traces is a problem. The methodology we have developed and implemented performs an automatic analysis that can be applied to huge tracefiles, which obtains its internal structure and selects meaningful parts of the tracefile. The paper presents the methodology and results we have obtained from real applications.
Funded by project TIN2004-07739-CO2-01 and by a FPI grant from spanish gov.
Chapter PDF
References
Paraver: performance visualization and analysis, http://www.cepba.upc.es/Paraver/
KOJAK: Kit for Objective Judgment and Knowledge-based Detection of Performance Bottlenecks, http://www.fz-juelich.de/zam/kojak/
Knuepfer, A., Brunst, H., Nagel, W.E.: High Performance Event Trace Visualization. In: Proc. PDP 2005, pp. 258–263 (2005)
Brunst, H., Kranzlmuller, D., Nagel, W.E.: Tools for Scalable Parallel Program Analysis - Vampir VNG and DeWiz. In: DAPSYS 2004, pp. 93–102 (2004)
Kranzlmuller, D., Scarpa, M., Volkert, J.: DeWiz - A Modular tool Architecture for Parallel Program Analysis. In: Proc. Euro-Par 2003, pp. 74–80 (2003)
Freitag, F., Corbalán, J., Labarta, J.: A Dynamic Periodicity Detector: Application to Speedup Computation. In: IPDPS 2001 (2001)
Mohr, B., Traff, J.L.: Initial Design of a Test Suite for Automatic Performance Analysis Tools. In: IPDPS (2003)
Nataraj, A., Malony, A., Shende, S., Morris, A.: Kernel-Level Measurement for Integrated Parallel Performance Views: the KTAU Project. In: IEEE International Conference on Cluster Computing (2006)
Vetter, J.S., Worley, P.H.: Asserting Performance Expectations. In: Supercomputing, ACM/IEEE 2002, Conference (2002)
The Message Passing Interface (MPI) standard, http://www-unix.mcs.anl.gov/mpi/
OMPItrace manual, http://www.cepba.upc.es/paraver/docs/OMPItrace.pdf
Badia, R.M., Labarta, J., Sirvent, R., Perez, J.M., Cela, J.M., Grima, R.: Programming grid applications with GRID Superscalar. Journal of Grid Computing 1(2) (2003)
Dimemas: performance prediction for message passing applications. 3rd edn., pp. 40–45, McGraw-Hill, New York, (1999), http://www.cepba.upc.es/Dimemas/
Press, W.H., Flannery, B.P., Teukolsky, S.A., Vetterling, W.T.: Correlation and Autocorrelation Using the FFT. In: 13.2 in Numerical Recipes in FORTRAN: The Art of Scientific Computing, 2nd edn., pp. 538–539. Cambridge University Press, Cambridge, England (1992)
Serra, J.: Image Analysis and Mathematical Morphology. Academic Press, London (1982)
Simon, B., Odom, J., DeRose, L., Ekanadham, K., Hollingsworth, J.K., Sbaraglia, S.: Using Dynamic Tracing Sampling to Measure Long Running Programs. In: Proceedings of the 2005 ACM/IEEE conference on Supercomputing (2005)
Sherwood, T., Perelman, E., Hamerly, G., Calder, B.: Automatically Characterizing Large Scale Program Behavior. In: 10th International Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS) (2002)
De Chevigne, A., Kawahara, H.: YIN, a fundamental frequency estimator for speech and music. Journal of Acoustical Society of America (2002)
Hoyas, S., Jimeńez, J.: Scaling of velocity fluctuations in turbulent channels up to Re=2003. Physics of fluids (2006)
Teysser, R.: Cosmological hydrodynamics with adaptive mesh refinement - A new high resolution code called RAMSES. Astronomy & Astrophysics (2002)
Springel, V., Yoshida, N., White, S.D.M.: Gadget: a code for collisionless and gasdynamical cosmological simulations. New Astronomy 6 (2001)
Linpack benchmark, http://www.netlib.org/linpack/
Author information
Authors and Affiliations
Editor information
Rights and permissions
Copyright information
© 2007 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Casas, M., Badia, R.M., Labarta, J. (2007). Automatic Structure Extraction from MPI Applications Tracefiles . In: Kermarrec, AM., Bougé, L., Priol, T. (eds) Euro-Par 2007 Parallel Processing. Euro-Par 2007. Lecture Notes in Computer Science, vol 4641. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-74466-5_2
Download citation
DOI: https://doi.org/10.1007/978-3-540-74466-5_2
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-74465-8
Online ISBN: 978-3-540-74466-5
eBook Packages: Computer ScienceComputer Science (R0)