Casa Parallelization meeting minutes

Tuesday March 15th Room 317, 3:00pm MDT

Attendees: Jim, Jeff, Brian, James, Sanjay, Dan

Targets/tasks for March 14th - March 20th

Discussion/Agenda

• Imaging issues
  • Current pimager.py has memory leak, defer for now
  • Imaging seems to use more memory than expected
  • New inexplicable error "This constructor takes no arguments"
• Identify/remove small read causes in split James/Jim
  • Modify source MS to use StManAipsIO
  • Modify split to generate MMS with StManAipsIO
• Examine streaming framework design Jim
• Clarify and document parallel imaging framework Jeff/Kumar/Sanjay
  • Some confusion as to details of the outer loop
• Test async performance against parallel continuum case James
• Further refine continuum test timings James
• Further develop parallel script frame work Jeff/Honglin
  • Pending reference-MS approach testing, auxiliary table creation problems
  • Some discussion about who should be involved in user interface
• Cluster scheduler scripting with CV James
  • Dan Klopp here from March 8th to 28th
  • Discuss with Jim semantics for cluster/CASA interaction later this week
  • Meet Thursday 3pm
• Confirm memmap vs cache performance behavior James
• General discussion on what's next
  • Becoming test bound, need to identify testers

Older Issues

• Continued discussion of threaded gridding approach Sanjay/Jeff/James defer for now
• Filling I/O issues update
  • Michel is optimistic of having a streamed I/O approach and use sub-ms infrastructure to test by friday 2/4/11
    • When is less clear now
  • Waiting on Francois to change data access methods
• Plot-ms suffers similar problem to split but needs lower level solution Ger
• Timing scripts
  • Large spectral (Bryan talked with Claire, Miriam available)
    • Less interesting in absence of parallel cube imaging
• Time domain partitioning with multiple MSes per imaging process Kumar
  • Cube case regression status
  • Trying to identify 10% small reads, possibly metadata column (either main or sub table)