Casa Parallelization meeting minutes

Monday January 31st Room 317, 3:30pm MDT

Attendees: Bryan, Brian, Jeff, Jim, Kumar, James

Targets/tasks for January 31th - February 6th

  • Merge async jim
  • parallelize cont kumar
  • refine parallel split and filler jeff
  • further timing james
  • Uniform MS size discussion Jeff and Kumar
  • Check on summer student availability Kumar/Jeff

Discussion/Agenda

  • Continue code development of async I/O Jim
    • working toward getting imaging to make use of it
    • Outstanding issues
      • Fixed msiter bug fixed for both async I/O and test cases
      • Double delete call bug for mdirection data. Temporary solution.
        • run tests against corrected/uncorrected code to see what performance hit is
    • Async I/O code to be checked in this week
  • Integrate async I/O with large continuum test case Kumar
    • Awaiting async I/O in active
  • Parallel imaging for large continuum
    • modified script with #node and #core selection
    • 1 ms per all cores vs 1 ms per core
    • add full beam code later
  • Parallel split test case Jeff
    • Plan to recreate as a task
    • Multi-ms still to be done
    • Only time domain for now
  • Filling I/O issues
    • Michel is optimistic of having a streamed I/O approach and use sub-ms infrastructure to test by friday
  • Split I/O issues
    • not clear whether it's in test or not
  • Plot-ms suffers similar problem to split but needs lower level solution Ger
  • Jim will be examining sub-ms/split/cvel refactoring for async I/O
  • Complete remaining timing scripts Gregg
    • Small spectral status (needed further cleaning)
      • does not include continuum subtraction as a separate timed step using uvconsub2.
    • Large continuum essentially done
    • Large spectral
  • Generate procedures and first timings for test cases James
    • Tile size substantially affects performance, need to characterize behavior
    • Still need other data sets
  • Time domain partitioning with multiple MSes per imaging process Kumar
    • Cube case regression status
    • Trying to identify 10% small reads, possibly metadata column (either main or sub table)
  • Cluster scheduler scripting with CV James on hold for scripts
    • Discuss with Jim semantics for cluster/CASA interaction
  • Cluster resource definitions James/Jim on hold

Task Assigned Status User testable User usable Notes
Imaging Kumar testing no no  
Calibration tbd        
Flagging tbd        
Filling tbd        

-- JamesRobnett - 2011-02-07
Topic revision: r1 - 2011-02-07, JamesRobnett
This site is powered by FoswikiCopyright © by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding NRAO Public Wiki? Send feedback