Monday October 3rd Room 317, 3:30pm MDT

Attendees:

Targets/tasks for October 3rd - 10th

Discussion/Agenda

• Lustre file I/O failure Jim/James
  • Being logged but no hits reported
• Revisit async iterator for writing Jim/Gusto
• Move subtables to in-memory storage manager Jim
  • Status
  • Now put in calibrator tool
  • Now put in flagging infrastructure (Gusto might do)
• Examine async thread failure problem and memory leak Jim/James
  • On hold for in-memory storage manager
  • Seems stable in recent tests
  • pcube does show thread failure
• Spectral imaging Kumar
  • Parallelization status
  • Should be in release
  • Masking issue with pcube complicating task creation
• Complete parallel user interface frame work Jeff/Honglin
  • Setjy taking longer than the serial case, seems to be read seek problem
    • Pending James looking at
• Refine and distribute cluster scheduler interface James
  • In progress, working with Kscott on multiple queue options
  • Awaiting proto parallel script
• Need 400+GB continuum case with complex field Sanjay
  • Formalized reduction status
  • Scripted reduction status

Priorities based on 3.3 release plans

• Parallel framework
• BDF size independent filler aka streaming filler
• Async I/O
• Multims spectral, continuum imaging
• auto-flagging

CASA HPC Initiatives for the 3.4 Cycle

• Memory Footprint Issues
  • Deployment of Multi-Field MS-MFS (Urvashi)
  • Deployment of Parallel Gridder and splitting of the Grid (Jim)
  • Design of MS-MFS Minor Cycle Parallelization (Jim et. al.) using Friday meetings
• Initial Deployment of ASP Clean (Sanjay)
• Performance Issues:
  • Virtual Data Column (Kumar)
  • Memory Resident Subtables Deployment (Identify list of tasks)
  • VisBuffer Processing Framework
    • Add Write capability (Jim)
    • Deployment in tasks (Jeff to Assign)
  • Parallel Filler (and partitioning) (Tak)
• User Interface:
  • Parallel Imager in Task (Kumar)
  • Parallelization of other Tasks (Kumar and Jeff)

Deferred Topics

• Work toward implementing streaming framework in applycal Jim postpone till after release
  • Needs testing
  • Needs flagger added
• Test version of auto-flagging deferred
  • Few weeks from suitable for testing
• Parallel gridder testing James deferred
• MS-MFS performance vs image size James Deferred
  • Appears to be an L1-L2 cache issue on older systems (newer Nehalems have one cache line per core)
  • Consider other parallelization approaches

ARDG Targets

• What is negative side effect of CVel in filler, non-reversible
• What needs to be done to acceptably implement Auto-boxing with large number of channels
• Broader parallelization of minor cycle for MS-MFS