Casa Parallelization meeting minutes

Monday April 4th Room 317, 3:30pm MDT

Attendees: Bryan, Brian, Kumar, Sanjay, James, Jim

Targets/tasks for April 4th - April 10th

• Understand imaging memory consumption Kumar
• Continue to develop (debug) async imaging Jim

Discussion/Agenda

• Imaging framework discussion last week
  • Memory dominated by number of images which in turn grow linearly with number of engines
  • Discussion of a threaded gridder approach, reduces memory footprint per node
  • async code is unnecessarily calculating parallactic angle each time
  • aspclean possibly replaces ms-mfs
• Imaging issues
  • Identify pimager.py memory issue as a function of version Kumar
    • Seems to be a function of number of MSes
  • Imaging seems to use slightly more memory than expected
  • filehandle limits a function of time/channel/field selection and reference ms
• Discussion of threaded grid cpu utilization Sanjay
  • threads appear to be idle 60+% of the time.
• Clarify and document parallel imaging framework Jeff/Kumar/Sanjay
• Resolve async issues with pcontmultims Jim
  • Don't test with debug builds
  • James to run using Jim's code (or test with single engine)
• Profile split Jim
• Further develop parallel script frame work Jeff/Honglin
  • Pending reference-MS approach testing, auxiliary table creation problems
• Cluster scheduler scripting with CV James
  • First pass of CASA/Torque interface Dan
  • Chris Mattman here from JPL to discuss OODT April 4th/5th
  • Clean up and make available James
• Test async performance against parallel continuum case James
  • Pending resolution to pcontmultims async issues
• Further refine continuum test timings James
• Examine streaming framework design Jim
• What is flagbackup=True in flagging and applycal ?
• General discussion on what's next
  • Where to draw the line on short term functional/usable vs longer term design.
    • Pending user interface, memory issues, filehandle issues

Older Issues

• Filling I/O issues update
  • Michel is optimistic of having a streamed I/O approach and use sub-ms infrastructure
  • Waiting on Francois to change data access methods
• Plot-ms suffers similar problem to split but needs lower level solution Ger
• Timing scripts
  • Large spectral (Bryan talked with Claire, Miriam available)
    • Less interesting in absence of parallel cube imaging
• Time domain partitioning with multiple MSes per imaging process Kumar
  • Cube case regression status
  • Trying to identify 10% small reads, possibly metadata column (either main or sub table)