Casa Parallelization meeting minutes

Monday July 23rd Room 317, 3:30pm MDT

Attendees:

Targets/tasks

• Benchmarks against various hardware James
  • Started will take many weeks to complete.
• Begin completion of pclean vs clean parametrization Kumar
  • In progress
• SSH key management, consider fork() for localhost Jeff to send to ESO
  • No information
• Refactoring visiterator (plotms) vs visibilityiterator (imaging and calibration) Jim
  • visiterator will be deprecated leads toward removal of scratch column

Discussion

• Status of HPC effort in Garching: Parallel Framework including partitioning Kern
  • Justo is working on cluster specification
  • Multims is possibly in standard tasks
  • Code in active not ready for testing

Deferred Items

• Memory requirements in imaging
• Describe and implement engine vs threaded gridder determination as a function of memory
  • Need dynamic image partitioning scheme for 2, 3, 4 or more threads.
  • (Total memory - some delta ) / memory per engine = number of engines
  • int (Total cores / number engines) = number of threads per engine
• Model column data tests James
  • Pending completion of test script
• Memory resident subtable tests *James *
  • Pending completion of test script
  • Check that it affects number of FDs as well
• Spectral imaging tests James
  • Understand channel chunk choice effect on performance James
  • Pending completion of test script

CASA HPC Initiatives for the 3.5 Cycle

• Tentative list from last meeting
  • Reduce in memory requirements during imaging (maybe removed)
  • Parallelize multi-field imaging
  • Parallelize MFS terms > 1 in the gridding step
  • Parallelize Cube imaging scatter gather
  • Expand threaded gridder case to >4 threads
  • Implement core affinity for threaded gridder case (first out)
  • Parallel framework (partitioning, parallel go) work in ESO
  • Filler to write a MMS in parallel