Pipeline face to face meeting at 5/7 - 5/11 at NAOJ
CASA 5.3 / 5.4 / Pipeline development planning uncertainties
CASA 5.5+ / 6.0 / Pipeline Cycle 7 planning uncertainties including Python 2 to 3 migration
New ALMA Cycle 8 pipeline processing "Fire and forget goal'
Good progress on CASA Callibrary testing, ALMA IF good, VLA IF problem with antenna correction caltable, single dish issue
Development
Progress on ALMA Archive / AQUA / Pipeline interface issues, adoption and extension of pipeline manifest
Progress on separating the ALMA interferometry pipeline into separate calibration and imaging recipes
Some new CASA 5.3 / Pipeline HPC / MMS bugs
Migration from plotcal to plotms is complete
Progress on web log error reporting
Progress on speeding up remote web log viewing
HPC
5.3
Fix for CAS-11196: "parallel setjy breaks PL", found by Remy while testing parallel pipeline with CASA 5.3.
Was making the pipeline fail in parallel mode at an early stage in calibration.
After changes in mstool and nrow(), nrow(True) won't produce 0 after a 'NullSelection'. setjy mishandles the error/exception and returns an empty dict which confuses the pipeline.
This problem was already caught in the ESO/Jenkins tests of the pipeline back in October and reported in CAS-10818. Another example on how useful it is to have automated tests of the pipeline!! The problem wasn't solved for setjy back then but deferred to 5.3.
After fix for CAS-11196, pipeline fails (both serial and parallel) at the beginning of imaging: CAS-11207: pipeline stage 23, hif_mstransform, failing with "KeyError: 'time'" - fixed today by Stewart.
Problem found after validation; under investigation. CAS-10697: uvcontsub does not work in parallel mode with 'combine=spw'
CAS-10386 Add new shared ALMA code to support 2 new ASDM tables in asdm2MS. Ready to validate. VLA expects this for 5.3.
CAS-10643 importasdm fails at the bdflags2MS step in at least one case. Ready to validate.
CAS-10644 Filler needs to detect duplicate (sets of) rows and retain only one copy. Done.
CAS-10738 Review the existing Ephemeris handling code in asdm2MS and fix found bugs. Ready to validate.
CASA 5.4
CAS-10693 remove boost dependency. Nearly finished. (shared ALMA code is done and will be checked in to ALMA/ACS repo shortly).
Kumar Golap
David Mehringer
George Moellenbrock
Dirk Petry
Martin Pokorny
Federico M Pouzols
CAS-10768: "Resource Predictor for Imager" - do we include it already in 5.3? With a switch in casarc, disabled by default. Or otherwise it would be very useful early in 5.4, so memory and (detailed) time profiles for tclean can be produced in general pipeline tests.
Fixed CAS-11196: parallel setjy breaks PL, fixed issue, now pipeline can run through calibration, but...
CAS-11207 "pipeline stage 23, hif_mstransform, failing with "KeyError: 'time'"": issue in pipeline mstransform / tablereader (both serial and parallel modes).
Working on test additions and cvel/mstransform tickets: CAS-10635, CAS-10340.