Monday Morning Meeting 13 November 2017

  • DIAL-IN NUMBERS & PASSCODES:
  • IP: 192.33.117.12##8110
  • Phone: (434) 817-6524

Attendance

  • Socorro:
  • CV:
  • Garching:
  • SCO:

News / Meetings / Visitors

  • CUC Meeting held 11/8-11/9. Committee members were very impressed with the documentation effort over the last year.
  • MSv3 MOU signed
  • Upcoming Build/Test Process Review Meeting planned

Users

Build, Release

  • 5.1.2 / 5.2 plans?
  • Volunteer reviewers for CAS-10683?
  • 10.13 now part of the master builds.
  • Switched TS3 to PR build
  • Trouble with Casa data updates

Verification Testing

  • HPC vs Serial Pipeline testing
    • CAS-10857 (see ticket for additional detail)
      • Federico re-ran E2E5.1.00041 (did Urvashi or Federico look at the results?)
      • Anything else related to this ticket?
    • Next issues...
      • I've noticed (looking at the By Topic tab in the weblog) that the various serial vs. parallel runs are producing different warnings (more often then not)
      • I am planning on creating a single ticket, then have a collection of these issues as sub-tickets
      • I will collect identical warnings in identical stages
      • This may be on the order of 10 tickets
  • Trying to build CASA in python 3 using the module 2to3
    • It's missing some Boost libraries (any idea Darrell?)
  • Testing latest casaguides on OSX
    • They all passed, will be going in the test suite soon
    • Outstanding issue: running the test with PowerNap enabled on Macs

Validation Testing

  • Current validation tally
    • 122 Under Validation, 85 Ready to Validate.
    • Oct 20 - 27: 8 tickets assigned for validation. 5 tickets resolved.
    • Nov 3 - 9: 6 tickets assigned for validation. 5 tickets resolved.
  • Current Testing Efforts for 5.2 deliverables
  • Current Testing Efforts for 5.3 deliverables
    • imaging issues/autoboxing refinements
    • statwt2 (CAS-10530)
    • systematic tests of mstransform
    • plone/CASAdocs:
      • we're moving forward with this -- will be distributing the final list of deprecated tasks for 5.3 soon, and
      • tickets are being created/bumped and will be assigned for developer input for ~60 tasks to be published in 5.3 release
    • further parallelization testing (non-ALMA pipeline specific)
    • polarization calibration
    • testing snapshot -- please see https://safe.nrao.edu/wiki/bin/view/Software/CASAUserTesting for the up-to-date list of tickets and testing status
      • testing ongoing:
        • plotants tickets (Pam, few more)
        • plotms/plotbandpass capability (Pam, CAS-9053)
        • statwt2 (Dave, CAS-10530 and others)
        • imhistory tickets (Dave, several)
        • phase center shift in plotms (CAS-8431)
        • averaging in flagdata (CAS-6215)
        • frequency interpolation of 'linearflag' for BP tables with good solutions (George, CAS-10772)
        • ms tool selectinit (Pam, Sandra, CAS-10818)
        • simulations (Remy, several)
        • Implement ACCOR equivalent (George, Walter, CAS-10366)
        • immath 'poli' and 'pola' implementation (Dave, CAS-9123, CAS-9124)
        • plotcal functionality in plotms (Pam, CAS-8270)

Architecture

HPC

  • Performance:
    • Started to run parallel tests for different number of cores.
      • Return on "investment" speedup/number of cores.
      • One aim is to better identify where (and hopefully find why) the speed up ratios do not scale up ideally with the number of cores (varies depending on tasks).
    • Started to look at run times for individual tclean calls, distinguishing by single calls or by pipeline stages. Long-term: characterize runtime as a function of tclean parameters.
    • Cross-checked slowdown in parallel against test runs on the ESO ARC cluster/lustre. Based on the very special T015 dataset, apparently similar behavior (or even worse).
  • Functionality:
  • Presented the CASA HPC and Parallelisation Status to the CUC. Slides are here.

Development

  • CASA 5.2
    • CAS-10853 : Selection error ( Update from TT ? )
    • CAS-10849 : Restoring Beam in Parallel Cube : Imager code checked into branch. Verified that the proposed solution of a parallel run followed by a serial restore gives the correct numbers (same as full serial). Waiting for resolution of a refconcat issue (setting restoringbeam appropriately in the header for refconcat images) - KG/DM ?
    • CAS-10857 : Numerical differences at stage 26 (hif_findcontinuum) - investigation ongoing.
    • CAS-10794 / CAS-10418 : Refconcat image analysis inefficiency - none yet (KG busy with above).

  • CASA 5.3

Pipeline

  • CAS 5.2 / Pipeline
    • selection error: waiting on CAS-10853
    • restoring beam in parallel cube: waiting on CAS-10849
    • refconcat images: waiting on CAS-10794 / CAS-10418
    • ASDM / MS spw warning messages for main array data understood, probably harmless but need to be looked at
  • CASA 5.3 / Pipeline
    • Observing sessions development branch regressions testing in preparation for merging to trunk
    • Updates to ALMA Cycle 6 planning and ICT tickets
    • ALMA Archive / Pipeline / AQUA interface discussions and meeting this Thursday
    • Preparations for ALMA Cycle 5 lifecycle presentations to DMS
    • CASA callibrary status: waitng on update from George
  • CUC meeting pipeline update
  • issues with orphaned tasks , e.g. plotweather
  • missing plotms plots from VLA pipeline

AOB

Developer Reports

Monday Meeting
  • Sanjay Bhatnagar
  • Sandra Castro
  • Lindsey Davis
    • presented pipeline status update at CUC
    • sessions branch testing
    • cycle 6 planning discussions
  • Bjorn Emonts
  • Pam Ford
    • CAS-10818 (ms.selectinit with invalid ddid warning not exception): pull request merged
    • CAS-8087 (plotms RA & Dec) question from Renaud: Investigated azel() and azel0() from VIVB2, values from ANTENNA and FIELD tables not POINTING
    • CAS-10774 (plotms failures in pipeline): xvfb errors fixed, investigated dbus timeouts on some individual plots
    • CAS-8854 (support plotting cal table types in plotms) in progress. Created test cal tables using casaguides, added support for KAntPos and EGainCurve
  • Enrique Garcia
    • Working on CAS-10458: Vi/VB2 vb->visCubeCorrected() segfaults if no column
  • Bob Garwood
    • CAS-10644 Filler needs to detect duplicate (sets of) rows and retain only one copy. POINTING duplicates are now handled. .Working on DATA duplicates for WVR data. More complicated than expected.
    • GBO issues.
    • CAS-10386 Add new shared ALMA code to support 2 new ASDM tables in asdm2MS.
    • Looking at long-opened Jira tickets (mine and Michel Callait, which should probably be close or reassigned to me) that pre-date my joining CASA last year.
    • sick (out today/working from home).
  • Kumar Golap
  • David Mehringer
  • George Moellenbrock
    • CAS-10865: pushed through verificaiton/validation/pull to master
    • Polarization stuff CAS-9105 and related followup on "short scheme" after QUESO meeting.
  • Dirk Petry
  • Martin Pokorny
  • Federico M Pouzols
    • ALMA pipeline tests for various performance concerns.
    • Work on CAS-10768: Resource Predictor for Imager, refactoring code, adding more metrics, producing plots, going through exceptions. Preparing report(s).
  • Urvashi Rao
    • Code for CAS-10847
  • Darrell Schiebel
  • Ville Suoranta * CAS-10878 init_dbus.py patch * Worked on CAS-10869 (Non-feature/bugfix branches in Bamboo)
  • Takahiro Tsutsumi
Friday NAOJ Meeting
  • Kanako Sugimoto
  • Wataru Kawasaki
  • Takeshi Nakazato
  • Renaud Miel

-- PamFord - 2017-11-09
Topic revision: r13 - 2017-11-13, BobGarwood
This site is powered by FoswikiCopyright © by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding NRAO Public Wiki? Send feedback