Casa Parallelization meeting minutes

Monday August 6th Room 317, 3:30pm MDT

Attendees: James, Jeff, Kumar, Bryan, Brian, Sanjay, Jim

Targets/tasks

• Benchmarks against various hardware James
  • Using CASA 3.4
    • 3.5 is close to working
  • No major surprises, auto flagging is dominate non-imaging task but parallelizes well
  • 8k X 8k Stokes I limited to 2 engines.
  • Preliminary graph here: CASA-bench.pdf
  • Continue with calibration benchmarks up through 4 nodes then switch to imaging.
• Begin completion of pclean vs clean parametrization Kumar
  • In progress, nearly done
  • Need to define an EVLA and ALMA pcube data set
• Refactoring visiterator (plotms) vs visibilityiterator (imaging and calibration) Jim
  • visiterator will be deprecated leads toward removal of scratch column
  • Waiting for feedback from others

Discussion

• Where to focus for 3.5 release beyond ESO effort (original 3.5 target list is at bottom of page).
  • Long disucssion on memory.

Deferred Items

• Memory requirements in imaging
• Describe and implement engine vs threaded gridder determination as a function of memory
  • Need dynamic image partitioning scheme for 2, 3, 4 or more threads.
  • (Total memory - some delta ) / memory per engine = number of engines
  • int (Total cores / number engines) = number of threads per engine
• Model column data tests James
  • Pending completion of test script
• Memory resident subtable tests *James *
  • Pending completion of test script
  • Check that it affects number of FDs as well
• Spectral imaging tests James
  • Understand channel chunk choice effect on performance James
  • Pending completion of test script

CASA HPC Initiatives for the 3.5 Cycle

• Tentative list from last meeting
  • Reduce in memory requirements during imaging (maybe removed)
  • Parallelize multi-field imaging
  • Parallelize MFS terms > 1 in the gridding step
  • Parallelize Cube imaging scatter gather
  • Expand threaded gridder case to >4 threads
  • Implement core affinity for threaded gridder case (first out)
  • Parallel framework (partitioning, parallel go) work in ESO
  • Filler to write a MMS in parallel