Casa Parallelization meeting minutes

Monday February 6th Room 317, 3:30pm MDT

Attendees:

Targets/tasks for January 6th - February 13th

Discussion/Agenda

• Update on Partitioning status (null selections, non-conforming tasks) Jeff
• Update on asdm2ms_xl schedule for stable inclusion
  • Does not handle large syspower tables
  • Still serial, eventually multi-threaded across spw, multi-process across scans
• Model data tests
  • No benchmarking done last week James
• Begin re-examination of multithreaded gridder
• Pcube and pclean as task after model data work Kumar
• Lustre file I/O failure Jim/James
  • Need to test whether it's actually fixed
• Revisit async iterator for writing Jim/Justo
  • Refactoring seems to be working in active
  • Justo is testing flagging
  • Pointer rewrite in active
• Memory resident subtables Jim
  • Recent tests with storage manager inconclusive, James needs to retest
    • Check that it affects number of FDs as well
  • flagging infrastructure most likely next target (Justo might do)
  • Partition worth looking at
• Spectral imaging Kumar
  • Understand channel chunk choice effect on memory James
• Need 400+GB continuum case with complex field
  • Stalled for partitioning

CASA HPC Initiatives for the 3.4 Cycle

• Memory Footprint Issues
  • Deployment of Multi-Field MS-MFS (Urvashi) Deferred
  • Deployment of Parallel Gridder and splitting of the Grid (Jim)
  • Design of MS-MFS Minor Cycle Parallelization (Jim et. al.) using Friday meetings Deferred
• Initial Deployment of ASP Clean (Sanjay)
• Performance Issues:
  • Virtual Model Data Column (Kumar) Done
    • Removes model data column
    • removes need for Setjy
  • Memory Resident Subtables Deployment In progress needs testing
  • VisBuffer Processing Framework
    • Add Write capability (Jim) Need to revisit but seems to work in not imaging
    • Deployment in tasks (Jeff to Assign)
  • Parallel Filler (and partitioning) (Tak) In progress
• User Interface:
  • Parallel Imager in Task (Kumar)
  • Parallelization of other Tasks (assigned to Europe)

Deferred Topics

ARDG Targets

• What is negative side effect of CVel in filler, non-reversible
• What needs to be done to acceptably implement Auto-boxing with large number of channels