Casa Parallelization meeting minutes

Monday June 20th Room 317, 3:30pm MDT

Attendees: Bryan, Brian, Sanjay, James, Jim, Jeff, Kumar

Targets/tasks for June 20th - June 26th

Discussion/Agenda

• Examine async thread failure problem Jim/James
  • In progress
  • Concurrent access to subtables is triggering problem
  • Memory resident subtables may help
• Move subtables to in memory storage rather than storage manager Jim
  • Suggested by Ger in favor of AIPSio storage manager
• Work toward implementing streaming framework in applycal Jim
  • deferred for subtable storage manager rework
• Spectral imaging Kumar
  • Parallelization status
  • Split on spectral window by default (includes continuum imaging as well)
  • Give Kumar small spectral
• Complete parallel user interface frame work Jeff/Honglin
  • In testing: flagging, calibration seem to work
  • Can just use pcont and pcube no need for p*multims variant
  • Discussion of path specification issue
• Refine and distribute cluster scheduler interface James
  • Re-addressing this week,
  • OODT presentation sometime this week
• Parallel gridder status Sanjay
  • Needs further optimization on hold
• Need 400+GB continuum case with complex field Bryan
  • Bryan will look into observing (spectral line and extended emission)
  • 3c147 in L band 8 hour observation set up by Sanjay
• MS-MFS performance vs image size James
  • Appears to be an L1-L2 cache issue on older systems (newer Nehalems have one cache line per core)
  • Consider other parallelization approaches
• Other issues
  • OODT presentation sometime this week, possibly Thursday afternoon

Priorities based on 3.3 release plans

• Parallel framework
• BDF size independent filler aka streaming filler
• Async I/O
• Multims spectral, continuum imaging
• auto-flagging

ARDG Targets

• What is negative side effect of CVel in filler, non-reversible
• What needs to be done to acceptably implement Auto-boxing with large number of channels
• Broader parallelization of minor cycle for MS-MFS