We have created a page in Confluence to follow up our tests in CV.
CAS-10398 (exportfits has issues with concatenated images) seems fixed. Resuming the parallel tests now.
The PWG uses custom scripts to run the pipeline and some of the settings they do are not trivial to do when running in parallel. For example, changing ulimit on the shell will only affect where mpicasa is executed, and will not affect the MPI remote servers. We have created CAS-10416 to figure out how to integrate parallel CASA in these scripts.
Agreement pending between Urvashi and Remy/Crystal whether we (ESO team) can get nodes in CV for the parallel tests or not.
Parallel pipeline tests :
The pipeline working group will not have time to test/commission this in time for our casa 5.1 release (or even their own cycle 5 release). They plan to do a pipeline patch for December. But, we still need testing before our casa 5.1 release, specifically to minimize the chances of needing a casa 5.1.1 patch from our side. So we're doing our own tests.
Sandra is coordinating with the PLWG on an agreed-upon list of ALMA pipeline test cases and setting up the tests to run in serial/parallel with some comparison metrics, and weblogs posted somewhere. In CV, Darrell/Andy will take this over once Sandra goes on vacation (in 9 days) and Andy will do some of the scientific validation (with input from Lindsey and the imaging team on what to look for).
This has begun. One critical bug turned up immediately (CAS-10398) for which a solution has already been pushed in (thanks to Tak) and ready for Sandra's tests. Follow up work will still be required for a better long-term solution, but it's OK for now.
Confirmed that VLASS does not need parallelization for now, and individual users who do need it should be covered by the above tests.
Master is still failing on acceptance tests. MP is working on possible vi2 issues via CAS-10362. Although the failure is manifesting itself in 'virtualmodel', MP's fix on the branch exposed another failure in vi2 from channel chunking of cubes. This brings this issue into the critical path for the ALMA pipeline.
Autoboxing goes on : Update from Tak ?
Documentation -- Anand to review remaining tickets to identify any critical work for 5.1. I think the only high priority item for this cycle is "branching" the documentation along with the code at release time to allow continued work on documentation for the subsequent cycle. Will update / create documentation that describes the current doc system and release / package process.
On-going ALMA Cycle 5 e2e tests in Chile
One going ALMA Cycle 5 pipeline testing at the NASSC - focus on autoboxing and imaging sensitivity pre-checking
On going VLASS testing, many enhancement and improvements
CASA 5.1. parallel pipeline testing - ALMA parallel pipeline not target for start of Cycle 5, will come in as patch
latest issue was slowdown in exportfits - should be resolved
possible packaging issue in gencal ALMA antenna position correction support
(UR) Whoever the first person is to check if this wiki page exists or not, please create the page from the template stored above the calendar list on https://safe.nrao.edu/wiki/bin/view/Software/CASA/CASAMeetingPage#Monday_Morning_Meetings . Lately, I've been doing it, but I usually get to it only late on Wednesday night, leaving little time for Socorro folks to add things in. - (MG) - thanks Urvashi! I should probably be doing it - I'll put a reminder on my calendar to make sure I do.
(MG) FY2018 POP goals
(FMP) ALMA Web services:
Pipeline would like to know if the SOAP client functionality is not going to be included for CASA 5.1, as it affects PL workflow and development (see CAS-9089).
Should we escalate a request for harmonization of ALMA web services?
Improvements to ALMA low SNR gain calibration heuristics
CASA lead job search interviews and discussions
Followup on pipeline tickets
CASA-10013: Merging changes from StatWtTVI in mstransform.
Federico M Pouzols
cvel1/cvel2 regridding question from users: CAS-10411.
Tests of pipeline locally and in CV cluster.
Issues in pipeline, already fixed in recent revisions CAS-10397. Issue in calibration plotting currently affecting most if not all datasets: CAS-10413.