Casa Parallelization meeting minutes

Monday November 8th Room 317, 3:30pm MDT

Attendees: Kumar, Jim, James, Bryan, Brian, Greg, Jeff

Targets/tasks for November 8 - November 14

• Identify somebody to test scan selection in filler Jeff/Brian
• Create async I/0 design doc Jim
• Complete timing scripts Greg
• Generate procedures and first timings for test cases James
• Time domain partitioning with multiple MSes per imaging process Kumar

Discussion/Agenda

• Clean status
  • Single MS and MS equal to number of nodes is in daily build, full multi-ms is not.
  • No task interface
• Filler status for filling multi-ms, scan selection is in but not tested
  • Focus should be on serial then multi-ms to improve large scan filling
  • Need to assign somebody to test scan selection.
• Async I/O development plan Jeff/Jim
  • Plan exists with dates
  • Which should be done first, calibration (not advantageous due to IO boundedness) or imaging
  • Decision on first task deferred till next Monday but likely imaging.
• Script generation of 4 data sets Greg
  • Small continuum and spectral cases nearly done, working on flagging
    • Long scan case will include pre-filled version
  • Two large to be done using Steve's original script as a basis
  • Generate baseline timings for test data sets Greg
    • re-assign to James after single proof run
• Benchmark procedures for serial and parallel tests James
  • Keep track of environment, versions, procedural details
  • Generate Lustre based data but then run against local disk to avoid contention
• Timing tests of 4 data sets James
  • run in parallel on all 4 sets
• Time domain partitioning with multiple MSes per imaging process Kumar
  • One MS per node is implemented
  • Nearly completed developing multiple ms per core mode
  • In the multiple ms per core each core has its own MS though we should compare to the multi-core per MS case
• Cluster scheduler scripting with CV James on hold for scripts
  • Discuss with Jim semantics for cluster/CASA interaction
• Cluster resource definitions James/Jim on hold
• Discussion of what's likely to be in 3.2
  • Imaging as a user task
  • Calibration or flagging implemented
  • Probably acceptable if things are usable for pipelines but not necessarily by users.