Federico, Sandra, and Enrique will be visiting Socorro so Federico and Enrique can meet Socorro team members. Will work on requirements for cvel refactoring.
Federico staying June 5 - 16
Sandra staying only the first week
Lindsey will only be available the last day or so
Build, Release
Turning off Subversion writes and sync at branch time.
The svn repository will become read only when the CASA 5.0 branch is cut. This does not apply to the pipeline.
Darrell has given Ville instructions on how to do this, but contact Ville if there is an issue.
CASA-5.0 Status. Holding the branch for:
CAS-10086: memory issue; differences between VI1 and VI2; code efficiency improved by 40%
Kumar will have a look at Sandra's data today
Will hold branch until the issue is understood
Calibration side of VI2/VB2 should not hold up branch at this point (GM).
Some minor differences are expected.
imager's vivb2 related fixes in response to testing (for serial runs) - Should this also hold the branch or not ?
CAS-8809 - Federico was investigating chanchunking problem; failure with VI2 happens in serial mode
Autoboxing
From the code side, it appears to be ready (UR).
Jen agrees
CAS-8530 - VI/VB2
Tarball
Ville has taken the pipeline out of the taball because it will not be included in the 5.0 release. Pipeline will go back in after the 5.0 branch is created.
Verification Testing
Casa guide extraction script
Josh has updated Mediawiki to 1.27.3 (1.23.17) on a test server
Akeem has updated the extraction script
Some further is needed (by the NAASC analysts)
Juergen and Sabrina have copies, so once testing concludes, it will be ready to be deployed, operationally
Osiris memory usage tracking for a 47 ASDM SD ALMA observation
Talked to James in Socorro about including Osiris in ganglia (this is complete)
Lustre has been down, still in recovery
Akeem is attending Intel Xeon phi processing class (online)
All about optimization
M100 automated test?
Sandra has updated the M100 test; will be in when Ville accepts pull request
Validation Testing
IT report re: upgrades to 10.12, looking at end of May, ~coinciding with CASA release
some questions from the SWST about VI1 and CI build numbering
VI1 will still be available in the background; there will be switces for imaging and calibration. uvcontsub relies on VI1 for now. The recommendation to users should be to use VI2 unless they suspect a problem.
performance improvement question (if we don't discuss it above)
current testing snapshot: 192 tickets under validation (141 plone), 90 ready for validation (35 plone)
Testers have been complaining about dates not going into builds. The build dates are somewhat arbitrary; the version number is what we need to use.
The warning summary is still being produced automatically by Jim's code. Who is responsible for this in the long term?
HPC
Testing the ALMA pipeline:
CAS-10078 coordinate system mismatch in tclean parallel cube error is fixed. But the datasets now all show the convergence problem, which Urvashi says it can happen with some partitions that never exit. This is all related to the Epic in CAS-10060 (multiple iteration control changes).
Working on a small fix that affects the log origin of some tasks when running in parallel (CAS-10074). There is a bigger issue about the log origin when running in parallel, which we will tackle next in CAS-9548)
Development
For Statwt rework, George is looking into write back feature of VI2 / TVI. Dave has a workaround for now.
Pipeline
CASA 5.0 / pipeline testing
tclean vi/vb2 testing
mysterious timing improvement, differences in the reported channel sensitivities
If changes need to be made to release after branch cut, you will need to branch from the pre-release branch.
Developer Reports
Thursday Meeting
Sanjay Bhatnagar
Sandra Castro
I have modified the expected RMS in the alma-m100 regressions, which have deviated in the past months from the expected hard-coded values. I have already asked for a pull commit for this branch (CAS-10100).
Started to fill up the mstransform task Plone documentation (description and example tabs).
Doc command testing -- does bring up a browser, but there are still error messages
Pam Ford
CAS-9053: continued work to add atmospheric overlays to plotms bandpass plots
CAS-2140: extended uv plot work (square plots with conjugates plotted) in plotms to uwave/vwave plots
CAS-8050: responded to validation testing issues for new ms tool functions
Enrique Garcia
CAS-9864: deeper analysis on the issues found during validation for test cases #2 and #3. The concerns are somehow unrelated with the original problem of the ticket.
Bob Garwood
CAS-9882 : write a test_importvla. This is done and waiting for testing.
asdm module + cmake: Darrell suggested a solution that works with the original plan for the submodule location, no need for a separate cmake
providing feedback to Rachel on the formal proposal to ICT management for the asdm code move
started trying some asdm module work flow use cases to flesh out the details and demonstrate the basic ideas will work as imagined
Kumar Golap
Responding to issues w/ VI2.
VLASS
Jeff Kern
David Mehringer
Working on statwt time binning; George just replied to Dave's question; working on unrelated doc for plone.
George Moellenbrock
Consultations with Dave M. on VI2 in statwt, including a meeting with managers. Dave is largely un-stuck now. Exploring the write-back regime in the TVI context more deeply---will document "best-practices" soon.
Some continuing examination of minor differences between VI1 and VI2 in calibration solving, which seem to be subtle consequences of now using a channelized weight (e.g., fields with non-zero spectral index in their models correctly yield a weight gradient with channel when divided into the data)...
CAS-9472: Some study of how to generalize the (pre-D-solve) Q,U model estimation in a basis agnostic way.
Dirk Petry
Working on CAS-9993 (FITS problem with PC matrix): found solution; need more tests before I can commit to casacore.
Martin Pokorny
Federico M Pouzols
CAS-9089: automatic generation of antenna position corrections from web service / ALMA TMC DB. Service time out issues. Trying to get it sorted out: http://ictjira.alma.cl/browse/ICT-9872
Validation of fix for CAS-10086: looks good. Files transferred for more checks by experts.
Based on that, preparing a tclean memory performance test for the Garching CI/Jenkins.
Validating CAS-8809: looks fixed (for VI1), but there seems to be a VI issue with VI2.
Urvashi Rao
fixing refimager regression tests, some work on cas-10078 for parallel cube iteration control + non casa work.
Darrell Schiebel
Ville Suoranta
Bamboo upgrade test and verification
Cleaned up Bamboo agent maintenance scripts
Fixed error handling on the CI build script
Added rh6tst to the Bamboo test machines
Changed make-app script to pull distro data from Git instead of svn
Resumed docker work. Docker build works, but regression testing in docker doesn't yet.
Fixed Gradle deprecation messages
Untangled unnecessarily complex closure juggling in the Gradle test framework
Contacted Anthony to see what the status of the Carta packages is -> Should be available on Friday
Debated with Darrell and Jeff about allowing SVN commits after branch