Thursday Morning Meeting 20 April 2017

• DIAL-IN NUMBERS & PASSCODES:
• IP: 192.33.117.12##8110
• Phone: (434) 817-6524

Attendance

• Socorro: Jeff, George, Lindsey, Juergen, Pam, Martin, Urvashi, Tak, Kumar
• CV: Darrell, Morgan, Akeem, Andy, Dave, Anand, Ville, Bob
• Garching: Sandra, Fedrico
• SCO:

News / Meetings / Visitors

• Anand Transition
  • Will continue with some CASA work (e.g. documentation) through the 5.0 release.
  • Morgan will be meeting w/ Morgan to discuss future of PM in CASA group.
• Obsmode: April 24-26
• Pipeline WG Face to Face: May 8-12
• CARTA Face to Face: June 19-23
  • CARTA group is starting work on interactive ("guided") clean.
• Jeff's replacement advertisement should close at end of month. Review of applications will begin ASAP.
• Susan replacement search will precede Jim replacement search. Jim replacement has been advertised as a junior systems position.
  

Build, Release

• New CI build Plan in place. The change was to move some of the build tasks from Bamboo to the Gradle/Groovy scripts. This will put us into a better position to start using Docker. It will also provide better flexibility for future workflow changes. The most visible change for the developers should be restart of the CI build numbering and the name change of the Plan to "CI - Test Suite - Gen 2". The older plan will be removed after the new one has been running successfully for a while. A couple of test machines had trouble with the JDK dependency. Still investigating.
• CASA-5.0 Status. Holding the branch for these:
  
  • Autoboxing: waiting on testing; Urvashi and Tak put in a change to prevent deconvolution when there is nothing to deconvolve.
  • Polarization Solve: under VI/VB2 polarization solutions are returning strange results.

Verification Testing

• CARTA Confluence page is wide open. Might want it restricted
  • https://open-confluence.nrao.edu/display/CAR
  • Who is our interface with the CARTA team?
    • We are in the process of trying to hire someone who will be our interface.
  • The team is composed of people in Chile, Canada, and South Africa, and ASIAA. In ASIAA, Chin-Fey Lee is the team lead; Grimmer Kang is the lead developer; Anthony Morgan is the systems guy.
  • For 5.0, CARTA will be included but not advertised. The CARTA team is planning for a stand-alone release in June. CARTA documentation should remain private for 5.0 release.
    
• Task level unit tests
  • About a half dozen new tests are making their way in to the framework
  • Pinged the tickets, to get updates on activities
  • Anybody have contact info for Nissim Kanekar? I have his nrao.edu address, but I'm not sure how responsive he will be to this email.
    • I want to assign him to write the unit test for importgmrt [ His email address is nkanekar@ncra.tifr.res.in ]
• CASAguide work
  • Coordinating between the test group, and NAASC analysts, as well as with Juergen (EVLA)
  • Current status

Validation Testing

Architecture

• Warnings Summary

HPC

• ALMA pipeline testing in parallel
  • VI2 in calibration causes a failure in applycal due to the use of VBContinuumSubtractor in AMueller. We are waiting for a decision from the pipeline on what to do to test the solution in parallel.
  • Using latest casa-prerelease 175, setting VI1CAL =1 and using VI2 for imaging:
    • A dataset (uid___A002_Xb020f7_X9e2) does not converge when running tclean cube, doing 90000 iterations, inside stage 32. The pipeline never finishes so it needs to be interrupted.
    • It works through the end for 3 other test datasets.
    • It seems to be unrelated to VI2 in imaging.

Development

• Followup on CASA memory limits: for release 5.0, limited to an environment variable checked by some imaging tasks including plotms. In the future, may be expanded to a hierarchical setting (.casarc / environment variable / script, with more local setting overriding more global).

Pipeline

• VI/VB2
  • applycal does not work with A Mueller caltables unless VI1CAL is set
  • setting / deleting VI1CAL from Python using os.environ['VI1CAL'] ='1' / del os.environ['VI1CAL'] works around the A Mueller caltable problem in serial mode, not clear why this does not work for HPC mode, Python / C++ environment issue ?
  • polarization calibration problems
• VLASS imaging pipeline development and testing
• Primary beam issues for VLASS imaging ?

AOB

Developer Reports

Thursday Meeting

• Sanjay Bhatnagar
• Sandra Castro
  • Verifying strange log origin in some parallel logs of the pipeline (CAS-10074).
• Lindsey Davis
  • Completed representative source identification heuristics development
  • VI/VB2 testing
• Pam Ford
  • Fixed plotms seg fault when changing visibility axis datacolumn and averaging (CAS-10066)
• Enrique Garcia
• Bob Garwood
  • Decided to go with plan B for the location of the asdm submodule (i.e. placed at same level as casacore and asap, not subdirectory in code/alma), I hate cmake.
  • Attacked alma warnings. One big one fixed. The rest can be fixed but are in the shared code and need to wait for the ALMA code development process to run it's course.
  • Started work on test_importvla.
• Kumar Golap
• Jeff Kern
• David Mehringer
• George Moellenbrock
  • Working on discrepancies in new VI2 solve regime that have emerged in the polarization calibration case. Current theory is that it is related to the net nominal handling of weight accumulation in VI2 vs. some weight conditioning (e.g., leveling over correlations) that occurred in the VI1 regime.....
  • Also addressed Lindsey's uvcontsub problem.
• Dirk Petry
• Martin Pokorny
• Federico M Pouzols
  • CAS-9853: regridding in mstransform like in tclean. Looking fine. Awaiting science validation.
  • CAS-9089: automatic generation of antenna position corrections from web service for ALMA. Checked requirements and started work.
  • CAS-9519: tests of ALMA pipeline in parallel. Issues in applycal and tclean.
• Urvashi Rao
  • cas-9208 : tests + waiting for more info from testers
  • meetings with CARTA people about interactive imaging
• Darrell Schiebel
  • tests with JanusGraph
  • debugging ccache mods
  • parsing large XML files
  • (quick) retrospective of IterBot
• Ville Suoranta
  • Moved CI build logic from Bamboo to Groovy
  • Moved most of the build servers to use data from the NetApp server. Servers running Test Suite 4 are still using local disk
  • As a part of this stumbled upon an issue with with OSX read locks and based on my findings provided feedback on CAS-9476
  • Fixed Carta build issues at Sinica
  • Investigated ASDM refactoring CMake problems
  • Provided release note logs for Juergen
• Takahiro Tsutsumi

Friday NAOJ Meeting

• Kanako Sugimoto
• Wataru Kawasaki
• Masaya Kuniyoshi
• Takeshi Nakazato
• Renaud Miel