Socorro: Rob, Lindsey, Tak, Kumar, Sanjay, Urvashi,
CV: Dave, Andy, Darrell
Garching: Justo, Sandra, Dirk
Jeff out today. George, Jim & Susan on leave.
News / Meetings / Visitors
NAOJ Visit last week
Summary of meeting noted below under development.
Mark in Chile
Jeff heading to Chile on Wednesday
Build, Release, Testing
RHEL Pre-release (EL 5)
EL6 deployed in the AOC? No, 4.3 will be first EL6 deployed.
ACTION: Rob to follow up with the AOC deployment.
OSX 10.8 build
Working on script to generate the package. Should be out this week.
CAS-6285 to be merged by Sandra.
Will need another pre-release.
Lindsey to confirm Todd's changes are merged.
Sandra to merge CAS-6285.
4.3 (trunk) test package
Linux test build works for Darrell and Dirk. Sandra ran functional tests, all passed.
User tester meeting
Change in practice - For now, developers merging their changes onto the release branch should be considered the standard operating procedure.
ACTION: Need an official announcement on CASA 4.2.2 release.
Common Statistics Framework
Meeting to discuss possible implementation.
Ger probably won't want it integrated here.
Need to address casacore integration as a broader issue.
Scimath equivalent under code.
To replace imagestatistics would require implementation in casacore. Could have code statistics package to build over casacore statistics package?
Tasks: calstat, imstat, sdstat, statwt, visstat
Pipeline also has some stand alone statistics.
Imaging and flagging also have some statistics implemented. (rflag, imager, etc).
ACTION: Pick a time for next week.
Status review and discussion.
Meeting time, place, and attendees.
ACTION: Rob to send agenda.
NAOJ Meeting Last Week (Jeff)
Given current observing schedule, aiming for parallel SD testing Dec 2014
Discussed migration plan from Scan table to MS for CASA single dish capabilities
Total project estimated at 1.5 years
Phase 1 (which is core functionality) targeted for 4.4 cycle.
Will require increased technical communication with NAOJ colleagues
4.2.2 Pipeline release.
Plotbandpass fix was only blocker.
Plotms is much more stable now. Will need to test with higher data volumes.
OSX 10.8 Testing
OSX workstation allocated to Lindsey for testing. No other progress.
Lindsey has a concern about some of the flagdata tickets reported below. Sandra and Lindsey to discuss. CAS-6791 in particular is contentious.
CAS-6746 needs a follow up. Does not affect EVLA, but is affecting ALMA Cycle 2 data.
Merge to trunk, test first. Watch regression results.
Can merge to 4.2.2 if it doesn't break anything.
CAS-6746: "inconsistencies between FLAG_ROW and FLAG cube."
We continued to debug this problem and found that the problem is very likely to be due to the Store Manager used for FLAG_ROW inside SubMS. Justo has sent a technical memo explaining his findings.
Working on translating/improving these ALMA reduction scripts to make better use of flagdata's features. Found a difference between list and interface flagging. Am working on this. I will post my suggestions in CSV-3014 as requested by E. Villard.
CAS-6074: "the pipeline needs a way to control the creation of flagdata agents in list mode". They've requested to have a strict mode to raise and exception as soon as the first agent fails to be created. This should be added to 4.3 or 4.4 priority list.
CAS-6805: "importasdm should only overwrite online text files if overwrite=True". Asked Jeff for his feedback on this. There was a requirement that the flag commands should be appended to the existing file. The pipeline doesn't want this.
CAS-6791: "Add a check for flagdata parameters in the current parser". Implemented a strict check on the flag commands parsed to the flagdata task. This will stop the task execution if any parameter in the list is not a flagdata parameter. It will also stop if any parameter has the wrong type. In testing phase.
User support for partition and flagdata (display mode).
Verifying the new casapy-test today.
Pipeline 4.2.2 release debugging and testing
Regression testing analysis
Cleanup of remaining 4.2.2 ALMA interferometry pipeline release bugs / issues
Debugging EVLA pipeline 4.2.2 pipeline
Analysis of initial pipeline results for ALMA spectral scan data sets
Analysis of initial pipeline results for ALMA data sets with ephemeris tables
Attended several single dish meeting sessions
Submitted comments on new build and test system requirements
Discussions on enables ALMA pipeline use for QA0 unset and failed data
Review of SSA group OODT work
Worked mainly on converting some FTMachines and VisModelData to directly support vb2
Justo González Villalba
CAS-6746: "inconsistencies between FLAG_ROW and FLAG cube": Completed the investigation about FLAG_ROW/FLAG inconsistenciestogether with Sandra
Reached the conclusion that the problem comes from using an Incremental Store Manager for FLAG_ROW, which seems to have problems with RW access when working with large discontinuous sub-chunks.
Verified that a fix based on using a Standard Store Manager for FLAG_ROW in SubMS fixes the problem (SubMS is used by asdm2MS and split).
Produced a technical report explaining in detail the reasoning of our investigation, which we hope can be the basis for a fix in the casacore's Incremental Store Manager.
Important: We think this fix should go to 4.2.2 as well
NAOJ / JAO / Single Dish Pipeline meeting
Plotms: CAS-6802, CAS-6753
Modified MSSummary to account for the end half-intervals correctly in MS start/stop/elapsed times reporting, CAS-5359.
Investigated CAS-5553, imregrid leaves open image.im and regridded table. Couldn't reproduce, reassigned to requester asking if this was still an issue for them.
Worked with Susan to resolve CAS-6793, Conversion of frequency units broken in Viewer/Profiler
redhat test version - Scott had created an independent script for starting casa with linux, likely to either work toward a independent script for OSX & linux or an independent script for linux versions. I incorporated this script into the automated build script.
HPC Sub-project research & planning
OSX release - work on script to create application from basic template which provides locations for executables and resources but pulls dependencies via otool