Casa Parallelization meeting minutes

Friday August 13

Attendees:

Targets/tasks for August 13-20

• Time domain partitioning in continuum imaging Kumar
• Enable multiple MSes per imaging process Kumar
• Examine Multiscale-clean performance issue in object copy Sanjay
• Implement proto-parallel calibration task Sanjay (full version to be assigned)
• Further Lustre testing James
• Create CASA parallelization priority list James/Bryan/Brian
• Understand planning for time segregation of filler and MS independent calibration Brian
• Look into status of M. Caillat and parallel filler (Bryan or Brian ?)
• Check on multi-ms viewer Sanjay stalled till Ger gets back

Discussion

• Further Lustre testing James
  • Not really possible to fully test, client bound
  • Initial results are positive, much lower latency as expected
  • Still capped at around 2.1 GB/s aggregate through put
• Create description for calibration from one MS to another Bryan
  • Ticket identified, not sure if it's been forwarded on
  • New target begin working on calibration independent of MS *
  • Sanjay may work on separate proof of concept version to advance state of the art.
  • Needed to advance split MSes
• Time domain partitioning in continuum imaging Kumar
  • Started but slightly sidetracked to revisit some channel based issues
  • Desired to make I/O more efficient
• Minor debugging of current frequency based continuum imaging Kumar
  • Inefficiency when a nodes image channel spans two spectral windows
  • Potential bug if #channels is not integer divisible by number of cores
• Examine Multiscale-clean performance issue Sanjay
  • Non-urgent FFT process spends 75% of time in object copy
  • 35% in summation of images as I += I(s), potentially solveable with I = I + I(0) + I(1) ... I(n)
  • Interactive masking now works
• Create CASA parallelization priority list James/Bryan/Brian
  • 7 Antennas at high site, by October should have 10'ish ALMA antennas
  • Mosaic'ing for both EVLA and ALMA
  • Want end-to-end processing filler, flagging, calibration, imaging, plots
  • Brian Understand planning for time segregation of filler and MS independent calibration
• I/O manager draft James
  • Need Continuum imaging processes to support multiple MSes
  • Kumar will look into the above, James will look into implementing a simple I/O manager
• Check on multi-ms viewer Sanjay stalled till Ger gets back
• Potential need for scheduler by early to mid-september