Casa Parallelization meeting minutes

Friday July 30

Attendees:

Discussion

• Cluster and Lustre status
  • New Lustre hardware is in, currently being set up and tested
  • Limited availability to new lustre and 4 nodes possible by 1st or 2nd week of August
  • Current CASA dev cluster will be limited to old lustre environment for now
  • Cluster scheduling software Torque being installed, should eventually be used on both clusters
• CASA testing
  • Sanjay had some difficulty in getting Imaging to run to completion, apparently simple fix
  • Need a benchmark data set for historic comparisons
• General discussion of various issues
  • Autoflagger is non-trivial due to locking issues
  • Need to parallelize multi-ms filler
    • Ger Van Diepen(sp?) has a multi-ms ms viewer, Sanjay will look into
  • Calibration issues
    • If calibration is going to occur in time domain might be worthwhile to move ahead with imaging in time domain
    • Some discussion of self calibration I missed most of this, if anybody has anything to add feel free
• I/O manager discussion
  • Discrete I/O process, interprocess communication from it to CASA tasks
  • Converts I/O from sync to async
  • Alleviates disk and i/o scheduler contention on workstations and nodes
  • Potentially relatively easy compared to threading or alternatives

Targets/tasks

• Further Lustre testing James
• Setup HPC wiki James
• Python modification to correct masking error Sanjay
• Check on multi-ms viewer Sanjay
• Calibration from one MS to another Bryan
• Time domain partitioning in continuum imaging Kumar
• Start parallel filler discussion ?
• Enumerate CASA tasks that need parallelization everyone
• I/O manager discussion everyone