Recall from Section 2 that structured adaptive mesh methods store data using a composite grid implemented as a hierarchy of levels (see Figure 2). To represent this structure, our API uses three data types: a Grid , an IrregularGrid , and a CompositeGrid . The Grid represents a single, logically rectangular array at one level of the composite grid hierarchy. A collection of Grids at one level of the hierarchy is an IrregularGrid , and a set of IrregularGrid s organized into levels is a CompositeGrid .
Figure 2: A composite grid is represented using a Grid , an IrregularGrid ,
and a CompositeGrid . A CompositeGrid consists of IrregularGrid objects organized into
levels. Each IrregularGrid is a collection of Grids . Real grid hierarchies
in multiple dimensions are vastly more complicated (see
Figure 1).
LPARX defines the basic building blocks for IrregularGrid and CompositeGrid , which provide specialized abstractions appropriate for adaptive mesh applications. Each object provides facilities appropriate for its role in the adaptive mesh hierarchy. For example, IrregularGrid defines communication operations only among Grids at the same level of refinement; an IrregularGrid has no notion of ``level.'' Instead, communication between levels is managed by CompositeGrid .
Communication among Grids in the hierarchy employs LPARX's copy-on-intersection operation, a high-level facility which copies data between the logically overlapping portions of two Grids . Data motion involves no explicit computations involving subscripts; all bookkeeping details and interprocessor communication are managed by the run-time system and are completely hidden from the user.
The parallelism in our adaptive mesh application lies at the level of the IrregularGrid . We can parallelize across one level of the hierarchy, but there is little opportunity for parallelism across levels. Therefore, the Grids in an IrregularGrid are distributed across processors, and we compute over these Grids in parallel. Following the LPARX model, each Grid in an IrregularGrid is assigned to one processor. Of course, a single processor may be responsible for many Grids .