Our adaptive mesh software infrastructure consists of three primary components: numerical operations, grid management facilities, and display routines. The numerical routines define the elliptic partial differential equation to be solved. The display library contains some simple graphing and plotting facilities for visualizing data. The grid hierarchy management routines comprise the most complex and interesting portion of the adaptive mesh API. These facilities manage all aspects of the grid hierarchy: data structure bookkeeping, error estimation, grid generation, workload balancing and processor assignment, and communication. An important observation is that the grid management facilities are independent of the numerical details of a particular partial differential equation; the same routines may be used to solve a number of different numerical problems.
One feature of our adaptive mesh library is that its facilities are independent of problem dimension. Scientists using the library see the same abstractions and interface whether they are working in two or three spatial dimensions. Numerical details differ, but the interfaces for grid generation, error estimation, load balancing, and grid hierarchy management are identical. Dimension independence provides programmers the freedom to develop and debug simpler, faster 2d versions of their applications on workstations and then, when confident that the code is working, recompile for 3d on a supercomputer. In practice, we have found dimension independence particularly useful; the adaptive mesh libraries and a materials design application [15] were first developed on workstations in 2d.
Our adaptive mesh software is layered on top of the LPARX software abstractions [16]. LPARX provides run-time parallel support such as distributed data management, coarse-grain parallel execution, interprocessor communication, and synchronization. The adaptive mesh API builds on this foundation and add facilities specifically tailored towards adaptive mesh applications.
LPARX's concept of structural abstraction and its support for first-class data decompositions have been vital to our success. Structural abstraction enables the application to represent and manipulate the structure of data---the ``floorplan'' describing where data is located---separately from the data itself. For example, when adding a new level to the adaptive mesh hierarchy, refinement regions at the new level are represented as first-class, language-level objects. The structure of the new refinement level is determined by grid generation routines. Refinement patch descriptions are then manipulated by load balancing and processor assignment algorithms. Only then does the code actually allocate the data associated with the refinement patches. In contrast, languages such as HPF [14] provide very limited run-time control over the dynamic allocation and placement of irregular distributed data. Structural abstraction enables our API to represent and modify dynamic refinement structures at run-time.
In the following sections, we describe grid hierarchy management and our model of coarse-grain parallel numerical computation. Further details can be found elsewhere [15].