Pipeline WG / Development Team Face 2 Face: Oct 20-25
CASA-4.2.2 release TODAY
Build, Release, Testing
"...due to the fact that Jenkins compiles in parallel the log file does not produce a description of the build error. One possibility would be adding a sequential build after a failing parallel build in order to produce a properly described build error"
Could we just remove the parallel build
Changes to JIRA Workflow - "Testing" as a status
Need to check Jira configuration.
4.3 Release Work:
Test build 4.3: OSX (EL-5)
Migration to googlecode (casacore)
Statistics framework status & meeting
HPC team meeting on Thursday
ALMA transitioned to Pipeline for calibration and flagging!
Initial Draft of pipeline imaging document
Bugs in parallel imaging
CAS-6835: "Implement cvel2". Have the first working version. Pending a decision from Jeff about passall=True. The only failing test refers to adding support to new OBJECT in mstransform; pending CAS-6778.
Consulted Crystal, Juergen, Dirk and Anita about the necessity to have passall=True in cvel2. Only Anita answered. She has use-cases for this parameter but they could easily be met by splitting out the non-selected data.
CAS-6543: "Heuristics of MMS cases". Committed to revision 30903.
CAS-6178: "Implement split2". Ready to test.
CAS-6914: "The plot in flagdata does not work with FLOAT_DATA column". Resolved.
created CAS-6891: "apply strict checks in the input lists given to flagcmd"
CAS-6072: "numpy.f2py does not work inside CASA when other numpy installed on the system". I have reproduced the problem reported by Schilke's ALMA study group. Indeed, the CASA release package is not self-contained, because when calling f2py.compile inside CASA, it still uses the system numpy instead of the CASA installation. I have requested to re-open this ticket.
Algorithm to determine number of w-planes (can set to -1 for automatic determination) for data sets
Needs to be implemented in tCLEAN
Justo González Villalba
Added support for combination, separation, and partition of Calibration Device and System Power sub-tables (which are not supported in casacore).
Created ALMA and EVLA tests for the fixes related with combination, separation, and partition of sub-tables with Spectral Window column (SPECTRAL_WINDOW, DDI, FEED, SOURCE, SYSCAL, FREQOFFSET, CALDEVICE and SYSPOWER)
Preparing a technical note for Science Testing (Juergen) describing this changes.
Three plus days of failed attempts to get a developer build on a 10.8 MAC (with help from Jim & Darrell).
WEIGHT_SPECTRUM: applycal context in good shape, solving context pending
Advised Jason Kooi on TEC testing
Bug in 4.2.2 EVLA switched power is not correctly handling weights.
(CAS-3046) "Support and use of ephemeris table when available" and (CAS-2988) "Make importasdm and MeasTable work with the JPL Horizon ephemerides": followed an alert by Arielle Moulet which turned out to be false. The ALMA science data don't have ephemerides attached, yet.
(CAS-6827) "tb.putcolkeyword is changing the type key values": intense work, approaching a solution (problem is related to the fact that casa variants don't know uInt and that the ValueHolder system in our casacore version does not support uInt; however, the googlecode casacore version already does support it; should switch to casacore from googlecode soon)
(CAS-6917) "cs.frequencytofrequency does not observe doppler definition": found that this is indeed the case. The straight relativistic doppler shift is applied. Corrected docu and removed redundant optional doppler parameter.
(CAS-5689) "spectral fitting results seem off": tested and found not yet resolved
two wvrgcal tickets CAS-6826 (new) and CAS-4867 (reopened): CAS-5897 needs to be resolved first before these ticket can be.
(CAS-6823) "Memory consumption in wvrgcal seems large": could not reproduce, waiting for input from Lindsey
tidied up some old tickets
Interactive GUI for Imager :
Modified the old Clean Viewer panel to add new runtime parameter modification options. Launched the GUI in-between major/minor cycles, via iteration-controller class.
Enabled interactive mask drawing and tested Tak's mask-making functions for regridding/expanding/evaluating an input mask. Modified peak residual calcs to always operate within masks. Made log messages say sensible things when channels are skipped because of intentional empty masks on some planes, etc....
Enabled field-based action controls for multi-field imaging (stop editing masks on some fields but not others, or stop deconvolving some fields but continue on others, etc...)
Discussions about imaging pipeline heuristics, some user support, debugged regression failure
Started (and monitored/debugged) parallel re-runs of CTB80 wideband mosaic tests with interactive masking ( 40+ processes, 300GB dataset, MT-MFS+WB-AWP ).
Vacation! (Pedaled a mountain bike from Durango, CO to Moab, UT.)
Mask handling code: added expand mask method (to generate a spectral cube mask from an input continuum mask). Also worked on to support a list of masks for the input
Makemask bug (CAS-6912 ) fix
Checked in the rotation measure parameter name change for setjy that I had thought I had checked in a few weeks ago...