Using coordination to parallelize sparse-grid methods for 3-D CFD problems

Abstract

In this paper, we investigate the good parallel computing properties of sparse-grid solution techniques. To this end, an existing sequential computational fluid dynamics (CFD) code for a standard 3-D problem from computational aerodynamics is restructured into a parallel application. The restructuring is organized according to a master/worker protocol. The coordinator modules developed thereby are implemented in the coordination language Manifold and are applicable to other problems than the present CFD problem only. Performance results are given for both the sequential and the parallel versions of the code. The results are promising. The paper contributes to the state-of-the-art in improving the efficiency of large-scale computations. We also present a theoretical analysis of speed-up through parallelization in a multi-user single-machine environment.

Introduction

One of the major challenges in science and technology is the fast numerical solution of partial differential equations. Important examples of such equations are those of fluid mechanics. When partial differential equations are solved numerically, they must be discretized, i.e., their solution, which is a set of functions defined over an area, is approximated by a set of - say - O(N^d) real numbers, where d is the space dimension of the problem (d=1, 2 or 3). Thus, the original differential equations are transformed into a system of O(N^d) algebraic equations with the aforementioned O(N^d) real numbers as the unknowns. For d=3 the size of the system can be very large. To solve these large systems, various techniques have been developed. Among these, the multigrid methods are optimal in the sense that the amount of computational work to solve the algebraic system is only linear with the number of unknowns. For all other known solution methods, the amount of work grows faster than linearly with the number of unknowns. For literature on multigrid techniques, see, e.g., Refs. 1, 2, 3, where Ref. [1] is recommended for an elementary introduction.

Novel multigrid techniques to speed up the solution of systems of discrete equations are the so-called sparse-grid techniques; see Ref. [4] and its references. Sparse-grid techniques are very attractive from the viewpoint of computational efficiency, particularly for 3-D problems. The gain in efficiency is achieved through a strong reduction of the number of grid points. Of course, this goes at the expense of numerical accuracy. Fortunately, the sparse-grid-of-grids approach has a better ratio of discrete accuracy over number of grid points [5] than a standard multigrid method (which in turn already has a much better performance in this sense than a single-grid method).

The efficiency of sparse-grid methods can still be improved further; an advantage of the methods is their good suitability for implementation on a parallel computer or a cluster of workstations. In this paper we present the parallel implementation of an existing sparse-grid solution method for the steady, 3-D Euler equations of gas dynamics 6, 7. Our starting point is a sequential Fortran 77 code describing this standard problem. If, for instance, entire subroutines of this code can be plugged into a new parallel structure, the resulting renovated software can take advantage of the improved performance offered by modern parallel computing environments, without rethinking or rewriting the bulk of the existing code [8]. The good parallel computing properties of sparse-grid solution techniques allow us to perform such a coarse-grain restructuring. The restructuring is organized according to a master/worker protocol and essentially consists of picking out the computation subroutines in the original Fortran 77 code, and glueing them together with coordination modules written in Manifold. Hardly any rewriting or changes to these subroutines is necessary: within the new structure, they have the same input/output and calling sequence conventions as they had in the old structure, and they still manipulate the same global data. The Manifold glue modules are separately compiled programs that have no knowledge of the computation performed by the Fortran modules - they simply encapsulate the protocol necessary to coordinate the cooperation of the computation modules running in a parallel computing environment. Manifold is a coordination language developed at CWI (Centrum voor Wiskunde en Informatica) in the Netherlands. It is very well suited for managing complex, dynamically changing interconnections among sets of independent concurrent cooperating processes 9, 10.

The rest of this paper is organized as follows. In Section 2, we introduce the discrete equations under consideration. In Section 3, we describe the concept of sparse-grid methods. For this, first standard multigrid methods are described. In Section 4, we briefly describe the sequential implementation of the 3-D CFD code and pay attention to its good parallel computing properties. Next, in Section 5we show how we can restructure this sequential 3-D software into a parallel code, using the coordination language Manifold. In Section 6, we give an analysis of the speed-up figures in a multi-user single-machine environment and show performance results for the test case of a half-wing in transonic flight: the standard test case of the ONERA M6 wing (Fig. 1) at a far-field Mach number of 0.84 and 3.06° angle of attack. Finally, the conclusion of the paper is in Section 7.