Casa Parallelization meeting minutes

Monday August 27th Room 317, 3:30pm MDT

Attendees:

Targets/tasks

• Benchmarks against various hardware James
  • Using CASA 3.4
  • Done for now with calibration/flagging focusing on continuum imaging
  • No benefit from async or striping input/output measurement sets
  • Updated graph here: CASA-bench.pdf
  • Completely dominated by major cycle
  • Moving to full polarization and MS-MFS then pcube
• Complete pclean vs clean parametrization Kumar
  • In progress, nearly done
  • Need to define an EVLA and ALMA pcube data set
• Examine temp lattice memory consumption
• Refactoring visiterator (plotms) vs visibilityiterator (imaging and calibration) Jim
  • Waiting for feedback from others
• Add selfcal to benchmark

Discussion

• Early adopters of parallel CASA (Nurur Rahman, Joanna Corby)
• Reduction in image sizes in MS-MFS
• Description of work reducing memory foot print in pclean

Deferred Items

• Memory requirements in imaging
• Describe and implement engine vs threaded gridder determination as a function of memory
  • Need dynamic image partitioning scheme for 2, 3, 4 or more threads.
  • (Total memory - some delta ) / memory per engine = number of engines
  • int (Total cores / number engines) = number of threads per engine
• Model column data tests James
  • Pending completion of test script
• Memory resident subtable tests *James *
  • Pending completion of test script
  • Check that it affects number of FDs as well
• Spectral imaging tests James
  • Understand channel chunk choice effect on performance James
  • Pending completion of test script

CASA HPC Initiatives for the 3.5 Cycle

• Tentative list from last meeting
  • Reduce in memory requirements during imaging (maybe removed)
  • Parallelize multi-field imaging
  • Parallelize MFS terms > 1 in the gridding step
  • Parallelize Cube imaging scatter gather
  • Expand threaded gridder case to >4 threads
  • Implement core affinity for threaded gridder case (first out)
  • Parallel framework (partitioning, parallel go) work in ESO
  • Filler to write a MMS in parallel