Casa Parallelization meeting minutes

Monday April 9th Room 317, 3:30pm MDT

Attendees:

Targets/tasks for April 9th - April 15th

• Test model data column changes James

Discussion/Agenda

• C++ vs Fortran performance in pclean
  • Going to revert to Fortran
• Discussion of focus over next 3+ months
  • Final cluster order in July
  • Initial cluster scaling calculations
    • Need to better understand effects of data size, image size, scales, terms and channels on MS-MFS and pcube
• General CASA memory handling (how does it handle over mem situations) (Memory in general has become high nail)
  • test memory and temp image use
  • Considering diskless/no swap partition cluster
  • Memory requirements as a function of imaging algorithm and image size Sanjay
  • Jim to look at memory consumption profiling
  • Contact Ger regarding memory assignment and swap avoidance
• Update on Partitioning status (null selections, non-conforming tasks) Jeff
  • No update
• Update on asdm2ms_xl schedule for stable inclusion
• Model data tests
  • James to get with Kumar
• Memory resident subtable tests James
  • Check that it affects number of FDs as well
• Spectral imaging tests James
  • Understand channel chunk choice effect on memory James
  • Awaiting calibrated data set(s) from Drew to test

CASA HPC Initiatives for the 3.4 Cycle

• Memory Footprint Issues
  • Deployment of Multi-Field MS-MFS (Urvashi) Deferred
  • Deployment of Parallel Gridder and splitting of the Grid (Jim)
  • Design of MS-MFS Minor Cycle Parallelization (Jim et. al.) using Friday meetings Deferred
• Initial Deployment of ASP Clean (Sanjay)
• Performance Issues:
  • Virtual Model Data Column (Kumar) Done
    • Removes model data column
    • removes need for Setjy
  • Memory Resident Subtables Deployment In progress needs testing
  • VisBuffer Processing Framework
    • Add Write capability (Jim) Need to revisit but seems to work in not imaging
    • Deployment in tasks (Jeff to Assign)
  • Parallel Filler (and partitioning) (Tak) In progress
• User Interface:
  • Parallel Imager in Task (Kumar)
  • Parallelization of other Tasks (assigned to Europe)

CASA HPC Initiatives for the 3.5 Cycle

• Tentative list from last meeting
  • Parallelize multi-field imaging
  • Parallelize MFS terms > 1 in the gridding step
  • Parallelize Cube imaging scatter gather
  • Expand threaded gridder case to >4 threads
  • Implement core affinity for threaded gridder case

Deferred Topics

ARDG Targets

• What is negative side effect of CVel in filler, non-reversible
• What needs to be done to acceptably implement Auto-boxing with large number of channels