Thursday Morning Meeting 04 May 2017

  • DIAL-IN NUMBERS & PASSCODES:
  • IP: 192.33.117.12##8110
  • Phone: (434) 817-6524

Attendance

  • Socorro: George, Urvashi, Lindsey, Pam, Martin, Kumar
  • CV: Anand, Dave, Akeem, Andy, Bob, Ville, Jen
  • Garching: Sandra, Dirk, Federico
  • SCO:

News / Meetings / Visitors

  • Jeff and Morgan are at an SKA meeting today and tomorrow.
  • CASA Lead search - two committees have been formed, one technically-oriented and one stakeholder-oriented
  • Pipeline Working Group F2F: May 8 - 12
  • CARTA F2F: June 19 - 23
  • Federico, Sandra, and Enrique will be visiting Socorro so Federico and Enrique can meet Socorro team members. Will work on requirements for cvel refactoring.
    • Federico staying June 5 - 16
    • Sandra staying only the first week
    • Lindsey will only be available the last day or so

Build, Release

Turning off Subversion writes and sync at branch time.
  • The svn repository will become read only when the CASA 5.0 branch is cut. This does not apply to the pipeline.
  • Darrell has given Ville instructions on how to do this, but contact Ville if there is an issue.
CASA-5.0 Status. Holding the branch for:
  • CAS-10086: memory issue; differences between VI1 and VI2; code efficiency improved by 40%
    • Kumar will have a look at Sandra's data today
    • Will hold branch until the issue is understood
  • Calibration side of VI2/VB2 should not hold up branch at this point (GM).
    • Some minor differences are expected.
  • imager's vivb2 related fixes in response to testing (for serial runs) - Should this also hold the branch or not ?
    • CAS-8809 - Federico was investigating chanchunking problem; failure with VI2 happens in serial mode
  • Autoboxing
    • From the code side, it appears to be ready (UR).
    • Jen agrees
  • CAS-8530 - VI/VB2
Tarball
  • Ville has taken the pipeline out of the taball because it will not be included in the 5.0 release. Pipeline will go back in after the 5.0 branch is created.

Verification Testing

  • Casa guide extraction script
    • Josh has updated Mediawiki to 1.27.3 (1.23.17) on a test server
    • Akeem has updated the extraction script
    • Some further is needed (by the NAASC analysts)
    • Juergen and Sabrina have copies, so once testing concludes, it will be ready to be deployed, operationally
  • Osiris memory usage tracking for a 47 ASDM SD ALMA observation
    • Talked to James in Socorro about including Osiris in ganglia (this is complete)
    • Lustre has been down, still in recovery
  • Akeem is attending Intel Xeon phi processing class (online)
    • All about optimization
  • M100 automated test?
    • Sandra has updated the M100 test; will be in when Ville accepts pull request

Validation Testing

  • IT report re: upgrades to 10.12, looking at end of May, ~coinciding with CASA release
  • some questions from the SWST about VI1 and CI build numbering
    • VI1 will still be available in the background; there will be switces for imaging and calibration. uvcontsub relies on VI1 for now. The recommendation to users should be to use VI2 unless they suspect a problem.
  • performance improvement question (if we don't discuss it above)
  • current testing snapshot: 192 tickets under validation (141 plone), 90 ready for validation (35 plone)
  • Testers have been complaining about dates not going into builds. The build dates are somewhat arbitrary; the version number is what we need to use.

Architecture

  • Warnings Summary
  • The warning summary is still being produced automatically by Jim's code. Who is responsible for this in the long term?

HPC

  • Testing the ALMA pipeline:
    • CAS-10078 coordinate system mismatch in tclean parallel cube error is fixed. But the datasets now all show the convergence problem, which Urvashi says it can happen with some partitions that never exit. This is all related to the Epic in CAS-10060 (multiple iteration control changes).
  • Working on a small fix that affects the log origin of some tasks when running in parallel (CAS-10074). There is a bigger issue about the log origin when running in parallel, which we will tackle next in CAS-9548)

Development

  • For Statwt rework, George is looking into write back feature of VI2 / TVI. Dave has a workaround for now.

Pipeline

  • CASA 5.0 / pipeline testing
    • tclean vi/vb2 testing
      • mysterious timing improvement, differences in the reported channel sensitivities
      • channel chunk heuristics problem
    • image analysis collapse method problems resolved
  • Pipeline F2F meeting preparations

AOB

  • If changes need to be made to release after branch cut, you will need to branch from the pre-release branch.

Developer Reports

Thursday Meeting
  • Sanjay Bhatnagar
  • Sandra Castro
    • I have modified the expected RMS in the alma-m100 regressions, which have deviated in the past months from the expected hard-coded values. I have already asked for a pull commit for this branch (CAS-10100).
    • Started to fill up the mstransform task Plone documentation (description and example tabs).
  • Lindsey Davis
    • pipeline F2F meeting preparations
    • ALMA SCOPS ticket debugging
    • Doc command testing -- does bring up a browser, but there are still error messages
  • Pam Ford
    • CAS-9053: continued work to add atmospheric overlays to plotms bandpass plots
    • CAS-2140: extended uv plot work (square plots with conjugates plotted) in plotms to uwave/vwave plots
    • CAS-8050: responded to validation testing issues for new ms tool functions
  • Enrique Garcia
    • CAS-9864: deeper analysis on the issues found during validation for test cases #2 and #3. The concerns are somehow unrelated with the original problem of the ticket.
  • Bob Garwood
    • CAS-9882 : write a test_importvla. This is done and waiting for testing.
    • asdm module + cmake: Darrell suggested a solution that works with the original plan for the submodule location, no need for a separate cmake
    • providing feedback to Rachel on the formal proposal to ICT management for the asdm code move
    • started trying some asdm module work flow use cases to flesh out the details and demonstrate the basic ideas will work as imagined
  • Kumar Golap
    • Responding to issues w/ VI2.
    • VLASS
  • Jeff Kern
  • David Mehringer
    • Working on statwt time binning; George just replied to Dave's question; working on unrelated doc for plone.
  • George Moellenbrock
    • Consultations with Dave M. on VI2 in statwt, including a meeting with managers. Dave is largely un-stuck now. Exploring the write-back regime in the TVI context more deeply---will document "best-practices" soon.
    • Some continuing examination of minor differences between VI1 and VI2 in calibration solving, which seem to be subtle consequences of now using a channelized weight (e.g., fields with non-zero spectral index in their models correctly yield a weight gradient with channel when divided into the data)...
    • CAS-9472: Some study of how to generalize the (pre-D-solve) Q,U model estimation in a basis agnostic way.
  • Dirk Petry
    • Working on CAS-9993 (FITS problem with PC matrix): found solution; need more tests before I can commit to casacore.
  • Martin Pokorny
  • Federico M Pouzols
    • CAS-9089: automatic generation of antenna position corrections from web service / ALMA TMC DB. Service time out issues. Trying to get it sorted out: http://ictjira.alma.cl/browse/ICT-9872
    • Validation of fix for CAS-10086: looks good. Files transferred for more checks by experts.
    • Based on that, preparing a tclean memory performance test for the Garching CI/Jenkins.
    • Validating CAS-8809: looks fixed (for VI1), but there seems to be a VI issue with VI2.
  • Urvashi Rao
    • fixing refimager regression tests, some work on cas-10078 for parallel cube iteration control + non casa work.
  • Darrell Schiebel
  • Ville Suoranta
    • Bamboo upgrade test and verification
    • Cleaned up Bamboo agent maintenance scripts
    • Fixed error handling on the CI build script
    • Added rh6tst to the Bamboo test machines
    • Changed make-app script to pull distro data from Git instead of svn
    • Resumed docker work. Docker build works, but regression testing in docker doesn't yet.
    • Fixed Gradle deprecation messages
    • Untangled unnecessarily complex closure juggling in the Gradle test framework
    • Contacted Anthony to see what the status of the Carta packages is -> Should be available on Friday
    • Debated with Darrell and Jeff about allowing SVN commits after branch
    • Fixed CrashReporter environment variable naming
    • Tested xdg-open browser issues with NRAO machines. Opened helpdesk ticket on this.
  • Takahiro Tsutsumi
    • Working on fluxscale mod for flux bootstrapping with RFI

Friday NAOJ Meeting
  • Kanako Sugimoto
  • Wataru Kawasaki
  • Masaya Kuniyoshi
  • Takeshi Nakazato
CONFLICT original 12:
  • Renaud Miel
    CONFLICT version 13:
* Renaud Miel* Renaud Miel
CONFLICT end
  • Renaud Miel
Topic revision: r18 - 2017-05-11, JaredCrossley
This site is powered by FoswikiCopyright © by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding NRAO Public Wiki? Send feedback