Casa Parallelization meeting minutes

Monday September 12th Room 317, 3:30pm MDT

Attendees: Sanjay, Brian, Bryan, Jim, Jeff, James, Kumar

Targets/tasks for September 12 - 18th

Discussion/Agenda

  • Lustre file I/O failure Jim/James
  • Revisit async iterator for writing Jim/Gusto
  • Move subtables to in-memory storage manager Jim
    • Status
    • Now put in calibrator tool
    • Now put in flagging infrastructure (Gusto might do)
  • Examine async thread failure problem and memory leak Jim/James
    • On hold for in-memory storage manager
    • Seems stable in recent tests
    • pcube does show thread failure
  • Work toward implementing streaming framework in applycal Jim postpone till after release
    • Needs testing
    • Needs flagger added
  • Spectral imaging Kumar
    • Parallelization status
    • Should be in release
    • Masking issue with pcube for making a task
  • Imaging advances
    • Add in weights to pcube and pcont (mosaicing) is in and may make release
    • Add in stopping threshold to both in place
  • Complete parallel user interface frame work Jeff/Honglin
    • Setjy taking longer than the serial case, seems to be read seek problem
      • Pending James looking at
  • Refine and distribute cluster scheduler interface James
    • In progress, working with Kscott on multiple queue options
    • Awaiting proto parallel script
  • Need 400+GB continuum case with complex field Sanjay
    • Formalized reduction status
    • Scripted reduction status
  • Internal users
    • Miriam is working on a parallel Casa guide

Deferred Topics

  • Test version of auto-flagging deferred
    • Few weeks from suitable for testing
  • Parallel gridder testing James deferred
  • MS-MFS performance vs image size James Deferred
    • Appears to be an L1-L2 cache issue on older systems (newer Nehalems have one cache line per core)
    • Consider other parallelization approaches

Priorities based on 3.3 release plans

  • Parallel framework
  • BDF size independent filler aka streaming filler
  • Async I/O
  • Multims spectral, continuum imaging
  • auto-flagging

ARDG Targets

  • What is negative side effect of CVel in filler, non-reversible
  • What needs to be done to acceptably implement Auto-boxing with large number of channels
  • Broader parallelization of minor cycle for MS-MFS

-- JamesRobnett - 2011-10-03
Topic revision: r1 - 2011-10-03, JamesRobnett
This site is powered by FoswikiCopyright © by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding NRAO Public Wiki? Send feedback