Systems support for irregular parallel applications

Abstract

Software developers for distributed memory multiprocessors often complain about the lack of libraries and tools for developing and performancing tuning their applications. While some tools exist for regular array-based computations, support for applications with pointerbased data structures, asynchronous communication patterns, or unpredictable computational costs is seriously lacking. The Multipol distributed data structure library addresses this need.

The purpose of this talk is to describe our experience building Multipol and associated software. First, I will describe several irregular applications that helped motivate the Multipol design and are now part of a challenging new irregular application benchmark suite. The Multipol application benchmarks are medium sized applications from CAD, Cell Biology, Symbolic Algebra, and Genetics. Second, I will present an overview of the Multipol design, which is built on a lightweight runtime layer with support for multi-threading, remote invocation, application-specific caching, global snapshots, dynamic message aggregation, and scheduling. The Multipol data structures include sets, queues, lists, hash tables, and graphs, with each data structure used in at least one application. The library is designed for flexibility and performance portability: rather than setting system-wide policies for load balancing, caching, or communication, it provides several alternatives that the application programmer may select and mechanism for extending the library. Finally, I will describe some of the scheduling and load balancing algorithms used in our applications.