CARTA - ASIAA plans to post the pre-release on github, timing TBD
Pipeline - ICT agreement to meet to align the Pipeline/Archive/AQUA interface, as an ICT-only issue
Build, Release
Packaged versions copied to CV/Socorro. Preliminary email sent out to sci-staff for last minute checks. VLASS says they need a later version of the pipeline (r40809).
We need a new Plone page describing the final steps of our release process (scripts, where executables get copied, etc).
Verification Testing
Only had time to begin the recruiting process to replace Akeem (ATR is done, iterating with Darrell on the job description)
Validation Testing
Current validation tally
116 Under Validation, 90 Ready to Validate.
2 tickets went RtV this week so far. 1 tickets went through Validation to Resolved this week so far.
Aug 25 - Sept 1: 27 tickets assigned for validation. 33 tickets resolved.
Current Testing Efforts for 5.1 deliverables
final packages built (5.1.0-68), and final testing shows a couple of small issues (still cannot start casadocs on lustre, run plotcal in "casa -pipeline" mode). now there is an issue with VLASS restore script?
Current Testing Efforts for 5.2 deliverables
coordination for the next level of science validation testing between PLWG/HPC group/Bjorn/(user testers as needed); time to start discussing once 5.1 is out
HPC needs for testing going through the 5.2 release, as far as nodes required and disk space anticipated (current numbers, future projections over the next months)
Current Testing Efforts for 5.3 deliverables
statwt2 (CAS-10530)
VLA testing scheduled early in the development cycle
systematic tests of mstransform
plone/CASAdocs
further parallelization testing (non-ALMA pipeline specific)
polarization calibration
miscellaneous
reporting peak residual in a channel that has no mask: CAS-10692
The error comes from a change in the pipeline to force applycal to run in serial on an MMS inside the hif_uvcontsub stage. Because the pipeline needs to run applycal with the old VI/VB for that correction, a VI1CAL environment variable is set inside the pipeline. This setting does not propogate to the MPIServers at run time, therefore bypassing the parallel processing allows applycal to run using the old VIVB. There are 2 ways to bypass the cluster, 1) by making the task process each Sub-MS, one after the other. 2) by making the task process the MMS as a monolithic MS. This second approach is causing the error message, which we are investigating at the moment. The first we will need to do in the pipeline once the 5.2 branch is cut is to set the bypass to 1 instead of 2.
Slowness of stats operations on concatenated images ( CAS-10428) becomes a top priority performance issue, in the light of recent parallel tests. The penalty (~10x slowdown in pipeline tasks) can compromise parallelization speedup for medium size datasets. Examples, with dataset T020: pipeline.hif.heuristics.imageparams_base takes 35m (serial) => 5h9m (parallel); with T030, 19m (serial) => 4h42m (parallel). Other pipe.hif and pipe.infra tasks also affected.
For validation: many more links to serial + parallel weblogs available now with fresh re-runs. First 20 datasets done. Last 10 datasets halfway through.
Can we allocate some 5.2 effort to improve automated test coverage (~0% now) of the many issues fixed in the last few weeks?
Development
CASA 5.2
Ongoing discussion about how much of the pipeline scripts/setup we (CASA) need to replicate for our September tests of serial/parallel
New bug : CAS-10685 : uvcontinuum fitting : Error when CORRECTED_DATA is encountered. Pipeline bug or CASA bug ?
New bug : CAS-10672 : CountedPtr dereference error in some datasets.
ALMA correlator upgrade project : We need to produce a note about how CASA will be impacted by (or will handle) this (by Monday Sep 11).
Project timescale : 2021 - 2022
Correlator specifications :
Increase spectral resolution by a factor of 8 => nchan = ~3800 will go to 32K. Limits are 65K, 32K or 16K depending on polarization mode.
Increase bandwidth by a factor of 2 => 16 GHz x 2 polarizations. Process the entire 4-12 GHz IF bandwidth.
Increase time resolution from 16msec to 1msec.
CASA's impact
Spectral resolution : NChan : Our spectral-domain parallelization schemes need to handle this. As of now, we expect current designs to scale. But we should verify this over the next year or two and make plans if we need something new. All modules : filling, flagging, plotting, calibration, imaging.
Double bandwidth : More use of wideband analysis algorithms. VLA has done this so all basic algorithms are in place. As of now, we expect existing algorithms to suffice. A factor of 2 increase is still modest in terms of a sensitivity increase (1.4 times) and fractional bandwidth (8% currently to 16% later (for comparison, EVLA handles 60% fractional bandwidth) ). Some commissioning of algorithms may still be required.
Narrow channels and/r high time resolution => Less sensitivity per visibility. Any algorithmic changes needed ? Can/should we rely on averaging in time and frequency when needed ? Are our averagers expected to scale appropriately ?
Anything else ?
Calibration : renewed interest in bpoly and bspline to get parameterized fits across channels and time, transfering solutions across spws/scans
Calibration parallelization
General topic of splitting and combining across frequency ranges (list of MS, multi-MS, imager's cube partitioning, channel chunking, the need for calibration, flagging, etc to gather stats and/or use data that cross SPW boundaries or 'split' boundaries).
Image storage format for large cubes. Neither of our methods will scale well. CASA Image format or Reference Concatenated images. Need to figure something out here.
Displays : plotms may need cacheless operations. For image viewing, we have no solution right now. CARTA is meant to support this, but it remains to be seen if this is going to be feasible.
We should ask if any heuristics discussions are happening - i.e. to think about how to break the spectra into pieces, keeping in mind the downstream processing.
Dev items that we will already be doing in the next year or two : Streamlining frequency splitting for parallelization, image formats, cacheless plotms, crossing split boundaries during processing. Dev items we don't yet have a plan for : A scaleable image format, and image display options.
CAS-7049 (cal table selection) in the home stretch. tCTSelection.cc updated and successful. Need to create google test.
Enrique Garcia
Bob Garwood
CAS-10621 : Target in image appear at incorrect coordinates. I believe I've found a bug in how the filler handles the ephemeris in the ASDM. But that may not be the cause of the target being at the wrong coordinates wrt the image center. Dirk is also looking in to this. I'm also gaining a better understanding in how ephemeris data is handled by the filler. I've also discovered an unrelated bug in how ephemeris related filler arguments are handled.
Some more discussions on solar VLA data. Eventually this will turn into a new ticket.
Started looking at how to remove boost dependencies from the filler code.
we will release a re-packaged version version 69, wit the new pipeline revision.
existing package is availabale at all sites, we won't change that yet. We will rebuild in the next couple of days, and then announce later.
Validation : Ongoing discussion (outside CASA) of how to allocate hardware resources for the next few months, and how the HPC testing of CASA will fit into that. No problems as of now.