ALMA pipeline wants a Nov/Dec 2017 product from CASA. We will call it 5.2 and have its development happen on a 5.2 branch (taken from the 5.1 branch when we release 5.1). All other development for the March 2018 release (to be called 5.3) will happen on master. Only fixes required for the 5.2 branch will have to go on 5.2 and master.
All tickets currently labeled as fixVersion=5.2 for CASA should go to 5.3. Anand (?) will automatically do this in JIRA. Anand - any instructions for us ? When do you plan to do this ?
CAS-10481 : Serial vs Parallel HPC tests : At least one dataset where there was earlier a ~40% difference in rms noise is now showing nearly identical results for serial vs parallel, after being re-run with the latest master containing fixes for parallel data selection issues. Bjorn / Andy - any more details ?
21 out of 30 serial tests finished, all others running smoothly so far and expected to finish by this weekend. Technical datasets "TEC" also fine.
Not many issues in serial mode, and all fixed. Good landscape for release:
Bug in imager field ids, only affecting one test dataset CAS-10620 the fix needs to be merged into 5.1 and master.
Issue with transitions in pipeline CAS-10614. Pervasive issue, but fixed in pipeline r40738 (exactly prerelease 5-1.0-61).
... but just found new issue possibly in pipeline, hopefully minor: CAS-10636.
For serial vs. parallel validation, updated results for 3 problematic datasets ready CAS-10481. They seem to match better now.
We can start now / next week parallel parallel re-testing for 5.2 but resource availability is not so great any longer.
Note/Warning: pipeline updates after r40738 (r40753 as of 24th in the morning) could introduce bugs, and there seems to be very limited test coverage.
Imager_BugFixes_5.1 -- No blockers for 5.1
Open : CAS-10317 : Mosaic slowdown. VLA case : being worked on. ALMA case : all is as expected
Open : New : CAS-10624 : Seg-fault from mosaic major cycle upon restart in build #48. Already fixed by KG in build #54 or #57. Requesting retesting by Todd.
Open : CAS-10538 : Ongoing w.r.to fix for parallel (array access?) error. Serial mode is OK.
Open : CAS-10250 : 7m+12m image cube coordsys definition (ambiguity)? : Tak - any update on this. Move to next release ?
Fixed : CAS-10264 : Fix a log message. Under verification by Amanda.
Fixed : New : CAS-10620 : Error in Field table access : Fixed by KG.
Fixed : New : CAS-10603 : Mosaic residual scaling off : Fixed by KG + setting of the default of conjbeams to False in the interfaces/python.
Anything else ?
Planning for the March 2018 release :
Please continue setting fixVersion=5.2 for tickets planned for the March 2018 release. Someone (Anand?) will do a mass edit in JIRA to move "5.2" to "5.3" at some point.
Received more detailed requirements from Remy for ALMA for the March 2018 release. No major changes to existing plans. Will update notes and talk to folks when needed.
CAS-10613 : SD imaging of solar data for Cyc5 : Note from Kana : the issue is being actively investigated. If not for 5.1, then this may have to go into our December product (5.2) as data will flow in/after December.
Participated in readiness review developer / PWG telecon August 18
Added support for an error on exit file creation to the pipeline processing request execution modules
Cycle 6 pipeline infrastructure / framework development planning
CAS-10597 (remove ms tool deprecation warnings for 5.1) - Ready for pull request after 5.1 branch merge
CAS-10604 (redo ms tool warnings for 5.3) - Ready to Verify
CAS-7049 (antenna selection for cal tables) - rescheduled after discussion with Sanjay
CAS-10613 - sdimaging cannot image Solar data taken in Cy5 E2E. This is due to duplicate rows in the MS POINTING table. That's caused by duplicate values in the ASDM at subscan boundaries. Discussions are ongoing as to where the appropriate "fix" should be. It's not clear yet whether the filler will be changed to handle this. As commented above, if the filler is changed to handle this that will need to appear in the December product as it's probably too late for any fix to appear in 5.1.
Federico M Pouzols
Parallel (little) and serial (mostly) pipeline tests.
CAS-10584 for more information about partitions in parallel pipeline runs
CAS-10264 : fix log message with product of image shape :
CAS-10423 : trying to write a unit test script that imports tests from another file, so as to not duplicate the code.... still trying.
Email and phone conversations about the December release and planning for the March 2018 release.
Friday NAOJ Meeting
Minutes of the meeting :
casa 5.1.0-63 will be the one to first connect to the pipeline, for a pipeline release candidate (for testing).
March release is 5.3. December product is 5.2 => Anand will do relabelling today -- 5.2 -> 5.3 fixVersion in JIRA. ( As of 10am MT, he has done this. ).
SD imaging issue : Talked later. Decisions made. Emails sent around about it. No last minute action needed from CASA for the 5.1 release.
George pointed out that we must add notes on confluence about 5.1.1 , 5.2, 5.3 with clear information about all the release dates and how they relate to the various ALMA/VLA deadlines. Also, the various releases are going to be called, and when branches will be made.
Docs : Kelly made a snapshot of docs for the 5.1 release. For 5.2, will clone the 5.1 docs.
test_predictcomp failure --> Need to follow up. Tak will send me email about what format this is that has broken. Need to follow up about casacore edits.
CrashReporter -> are we paying attention to reports from it right now ? Not really, but it has been used a couple of times to diagnoze problems. Juergen pointed out that we should iron out usability issues by the October meeting of the CUC.
*Lindsey : A Qt app error upon casa exit, is messing up shell scripts that look for exit criteria from casa. Darrell : may have already been fixed --- Ticket from Rafael on this. -- need to retest
HPC testing : looking good. Serial OK. Parallel - continue retests of the earlier failed tests. Resume full suite of serial vs parallel tests after all 5.1 fixes are in, and hand this to the PLWG as their starting point.
Newsletter contributions due on Sep 11 - if you want to contribute