Monday Morning Meeting 18 May 2017

  • DIAL-IN NUMBERS & PASSCODES:
  • IP: 192.33.117.12##8110
  • Phone: (434) 817-6524

Minutes in blue

Attendance

  • Socorro: George, Pam, Lindsey, Martin, Juergen, Kumar, Tak, Urvashi
  • CV: Anand, Akeem, Ville, Jen, David
  • Garching: Sandra, Enrique, Federico
  • SCO:

News / Meetings / Visitors

  • CASA Lead search – committee meeting this Friday to identify candidates for phone screening. Screening should happen over the next two weeks.
  • CASA PM -- Anand to provide part-time CASA PM support through end of September. Focus on wrapping up the documentation effort and delivering 5.1.
  • CASA Coordination Committee – I hope to close the loop with Jeff on Friday and do a kickoff meeting next week. (This is the interim management committee that will assist Morgan until a new group lead is hired.)

Build, Release

  • Are we going to update CASA Core for 5.0 to address the field name vs. field id discrepancy? This will make a difference to VLASS.
    • VLASS can use a CASA 5.1 prerelease for testing, and updating CASA Core for 5.0 may introduce bugs. Therefore, we will not update CASA Core for 5.0.
    • A discussion is needed regarding how to handle CASA Core and branching. This should probably wait for the next group lead.
  • Sub-tasks and Sub-Engineering tasks do not create Bamboo branches, and therefore do not get tested (beyond test suite 1) and packaged.
    • Currently, the ticket type must be either bug, feature, or engineering task to result in a Bamboo branch.
    • It is not obvious to Ville how to link Bamboo packages for sub-tasks.
    • Decision: We will stay with the current configuration for now. Email Ville if a subtask needs to be tested and packaged. Ville will let us know if this becomes a problem.
  • Ville thinks that it may be possible to create a new branch outside of the BitBucket interface and have the branch associated with the correct JIRA ticket. Akeem is testing this possibility.
  • There were a couple Bamboo outages over the past week. Not clear what the cause was. Settings have been tweaked. If this continues, the next step will be to escalate to Atlassian.
  • Note that, as currently configured, Bamboo branches (and associated packages) will expire after 30 days. This expiration date is intended to conserve compute resources.
    • Ville will investigate the possibility of adding a notification before the branch expires.
  • CASA 5.0:
    • A tclean bug has been discovered related to masks that causses files to be left open. Only occurs for NFS-mounted disks. Urvashi and Tak are aware and are in communication with Dirk.
  • CASA Docs:
    • Please do not yet update Plone with 5.1 content.
    • Need a knowledgeable developer to review LEL pages for accuracy.
    • Need to merge visibility data selection pages.
    • Jen, Juergen, Anand planning for final documentation review and clean up on week of June 5th.

Verification Testing

Validation Testing

  • testing in the new framework
  • plone changes to XML in the new framework
    • Changes do require their own branch. In the pull request, a not can be included indicating where the change is desired.
  • thanks to Urvashi, James for updating their chapters with reviewer feedback

Architecture

HPC

  • Parallelization will be available in CASA 5.0.
  • Testing the ALMA pipeline in parallel
    • Nothing new in the tests. The last test run automatically in the pipelines used case-prerelease-5.1.0-2, which is 6 days old.
  • mpi4casa framework
    • On-going investigation of a problem reported by Kana that running the interferometry pipeline fails sometimes in CentOS7. We haven't seen this at ESO in our development dockers that run CentOS too. TBV
    • Implemented a way to disable the heartbeat mechanism in mpi4casa to allow easier debugging of tclean without timeouts from the framework. Implementation is under verification and will soon be available for validation.

Development

Pipeline

  • Pipeline face to face meeting in CV May 8-12
    • ALMA development and testing planning, coordination with VLA / VLASS
  • CASA 5.0 / pipeline testing
    • field name vs field id data selection performance when field tables are large, e.g. the VLASS use case CAS-10143
    • recent autoboxing mask handling behavior change
    • on-going NAASC / developer pipeline regression testing on the pipeline trunk in preparation for pipeline branching
    • C5P1 release notes preparation
    • on-going VLASS prototype imaging tests
Planning to cut a branch for the pipeline. This will allow continued testing while infrastructure changes are made.

AOB

Developer Reports

  • Question about status of SD in CASA 5.0. Anand will relay question to Morgan.
Monday Meeting
  • Sanjay Bhatnagar
  • Sandra Castro
    • Not much work on CASA lately, only Plone documentation for mstransform task description and examples tabs.
  • Lindsey Davis
    • Face to face meeting participation, presentations, discussions, ...
    • Preparation for pipeline branching, testing, bug fixing, release notes
  • Pam Ford
    • vacation
    • continued work on atmospheric overlays in plotms (CAS-9053)
  • Enrique Garcia
    • continue working on CAS-9679 to remove spw parsing from FreqAxisTVI
  • Bob Garwood
  • Kumar Golap
  • Jeff Kern
  • David Mehringer
    • (CAS-9408) create plone docs for imsmooth. Package branch verified. Ready to validate.
    • (CAS-9999) StatWt Rework: Flexible time and frequency-range binning. Package branch verified. Ready to validate.
    • (CAS-10109) imdev needs to mask pixels outside the computable region and/or are masked in the input image: Package branch verified. Ready to validate.
    • Responded to LEL question from Juergen. Raised point again if we should stop supporting binary region files and "regions in images".
    • (CAS-10151) Investigate not caching column vectors in MSMetaData. casacore PR submitted. Martin has asked questions.
    • (CAS-10135) wrong spectral increment in exportfits: Initially assigned to me, but I don't know what is going on without code diving. Dirk is our image FITS domain expert, so I punted to him.
    • (CAS-9098) gencal requantizer calculations extremely slow: Discussed various MSMetaData issues with George. Created CAS-10150 and 10151 to remedy some of the caching and memory issues.
    • (CAS-10002) StatWt Rework: Add support for specifying minimum number of unflagged points: Under verification. tStatWtTVI is failing which I thought I had fixed...
    • tStatWtTVI failing sometimes / on some platforms: This exposed a bug in StatWtTVI which I fixed.
  • George Moellenbrock
    • CAS-8564: Followed-up Frank Schinzel's validation of this ticket to show that it is, in fact, working correctly. The 'garbage' phases he described in the new regime were actually the effect of the dispersive delay.
    • CAS-9472: Some design work on how to generically incorporate TEC in the QU solve a basis-independent way.
    • Various incidental discussions on VLASS-related calibration issues (CAS-10124, CAS-10138, CAS-9107, CAS-10139)
    • Some science/admin work related to a conference at which I'll be speaking at the end of June.
    • One day off.
  • Dirk Petry
  • Martin Pokorny
  • Federico M Pouzols
    • CAS-8809: "channel chunk logic, skip channels". Status is fixed. Opened follow-up engineering task to keep track of potential issue: CAS-10160.
    • CAS-9077: "Cannot capture return values from tasks run in an mpicasa client". Status is fixed/clarified but deeper, more disruptive changes planned for 5.1: CAS-9871 "parameter defaults in the MPI task.._pg.py"
    • CAS-9143: "Allow to disable heartbeat mechanism form mpi4casa (for debugging)" fix ready.
    • CAS-10036: "flagdata not flagging data" - still working on it, very specific issue.
  • Urvashi Rao
  • Darrell Schiebel
  • Ville Suoranta
  • Takahiro Tsutsumi
    • Disabled saving of temp masks to disk (for 5.0) and a minor documentation update (CAS-9208)
    • Continue testing on fluxscale modification for the robust flux density bootstrapping (CAS-9186)
Friday NAOJ Meeting
  • Kanako Sugimoto
    • suppress log messages in blfunc='variable' of sdbaseline ( CAS-9825): enabled verbose parameter for this. development complete. under testing.
    • discussion of residual atmospheric lines in TP calibration going on at ( CSV-3320)
  • Wataru Kawasaki
    • CAS-9918: bugfix done. user testing underway.
    • CAS-9955/CAS-10077: these require some modification in casa-data repository. setting up GIT-based casa-data repository on my development environment.
    • passed 3 tickets (CAS-9825/CAS-9918/CAS-10044) to CASA testers at NAOJ for validation.
  • Masaya Kuniyoshi
  • Takeshi Nakazato
    • migration of SDGrid class to VI/VB2 based framework (CAS-9957)
    • investigation on residual atmospheric feature seen in single dish calibrated spectra
  • Renaud Miel
-- JaredCrossley - 2017-05-17
Topic revision: r18 - 2017-05-18, WataruKawasaki
This site is powered by FoswikiCopyright © by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding NRAO Public Wiki? Send feedback