CASA Parallelization Implementation Plan

Introduction

The project will be approached with an iterative/incremental development model, deploying the parallelization framework first, with sub-optimal parallelization of tasks at the python/task level. Deeper parallelization (at the C++ level) and optimization of resource use (I/O shaping, processing efficiency) will be added in subsequent cycles.

The rough divisions between iterations aer described below.

Iteration 1 Development

• Parallelization of common tasks at the python level to leverage the MMS data structure.
• Parallel infrastructure deployment (including MPI).
• Guidelines for what resources to request for typical use cases.
• Integration with the B&T system such that CASA can be built and tested routinely on the cluster nodes.
• Existing parallel implementations to be tested include:

Iteration 1 Testing

• Testing of existing parallel implementations
  • MMS filling and split
  • Mstransform
  • Simple_cluster in Imager
  • Openmp in flagging/gridding.
  • Other existing HPC solutions/implementation.
• Cluster resource optimization tool/heuristics research.
• Planning for next iteration

Iteration 2 Development

• Parallelization of the pipeline (high priority given automated production of pipeline products).

Iteration 3 Development

• I/O shaping and sizing.
• Processing efficiency improvements.
• Deeper task parallelization. (C++ level)

Iteration 4 Development

• GPU integration.