CAS-10672: RuntimeError: Error in running Major Cycle : CountedPtr: null dereference error - Under verification
CAS-10538 : The two imager errors are not reproduceable, and a test has been added to try to trigger one of them. RESOLVED
CAS-10453 : Applycal errors are a harmless(?) manifestation of using subMSs. We need to understand why the SEVERE messages are not filtered out from the logs as in the past. Maybe something in the pipeline is resetting the log filter, which then exposes the messages to the logger. If not solved in 5.2, at least it will be added to list of "differences" between serial parallel runs.
For 5.2 (pipeline)
CAS-10662: "MPI buffer error", fixed by Stewart in pipeline r40843 (we'll need this included in 5.2). Validation throughout the next tests for 5.2. The pipeline was sending over parts of the context.
CAS-10697 : parallel uvcontsub failure on subMS. This does not affect the ALMA pipeline, therefore strictly speaking, it should go to 5.3. Old uvcontsub was parallelised a few years ago but needs a better error handling to advise the user that when combine=spw, all the Sub-MSs need to have all the selected spws. If the MMS is partitioned with the default options, most likely the Sub-MSs will not contain all the spws. The best implementation means, adding a new parameter called keepmms, in order to align uvcontsub with the other tasks such as split, which also can create an MMS.
CAS-10615 : Info about partitions for the pipeline. This was not requested by the pipeline. It is a convenient way to show the contents of an MMS when partition/importasdm finishes to run. There is already a task called listpartition which does this.
Discussions with Bjorn on his VLA HPC testing. His tests show slowdown when running gaincal and setjy, both in serial mode on MMS. TBV
CASA 5.2 : 8 issues being tracked. 5 CASA-specific ones are below. ( Need not discuss if already covered by the HPC group's report )
CAS-10672, CAS-10538 : The two imager errors are not reproduceable, and a test has been added to try to trigger one of them.
CAS-10453 : Applycal errors are a harmless(?) manifestation of using subMSs,
CAS-10697 : parallel uvcontsub failure on subMS ?
CAS-10615 : Info about partitions for the pipeline ?
CASA 5.3 :
Iteration control + Autoboxing : Plans have been set up for a clear division of work for R&D vs production, as well as how to handle overlapping features. This is to avoid the issues we have had over the past year on this topic.
statwt2 : Some dev in response to testing. Some updates to interface requirements.
ALMA has done it's planned changes to the SDM (new tables - CalAntennaSolutions, Pulsar, more cols to the CalBandpass table + a channel column to the flag table (i.e. a new Flag.xml tag which will require edits to our parser used by imports and flagger). Bob - any surprises yet ?
No. But I haven't looked in detail. It will take some time before this progresses through the ALMA acceptance tests and shows up in the main trunk. Also, ALMA's move to git is happening now. So, I'm going to wait awhile (probably about a month) for the dust to settle before I look at this. The related CASA tickets (CAS-10386, and CAS-10422) are for 5.3 with the requirement that they be available for testing in a 5.3 pre-release by Feb. 2018. I see no problems with meeting that timeline at this point.
Polarization Calibration dev update from George ?
Imager : new bugs to be investigated : mosaic sidelobe/wrapping, two spectral coordinate system generation issues.
Anything else ?
CASA 5.1 / Pipeline
Out the door
CASA 5.2 / Pipeline
Pipeline plans to branch for 5.2 after today's meeting
First pipeline HPC issue fixed on pipeline trunk, will be backported to HPC branch
CASA 5.4 / Pipeline
Development has resumed
ALMA heuristics development plan is conservative and evolutionary
ALMA heuristics ticket will go into jira shortly, framework tickets are already there
VLASS heuristics for single epoch imaging will require effort
Heuristics planning meeting with PWG
Ongoing CASA callibrary tests, problems with data / callibrary selection
Pipeline refactoring, code cleanup work
3 tickets under validation (CAS-10727 plotms add/close plot seg fault, CAS-8270 plotms axis label bug with cal table overplot, CAS-10736 cal library parser bug)
1 pull request merged (CAS-10750 plotms constant name conflict)
2 tickets responded to validation bug reports (CAS-9053 atm/tsky overlay, CAS-10604 ms tool deprecation inline help)
Developer builds on OSX 10.11 (cloned git code), OSX 10.12 (upgraded laptop from 10.11)
CAS-10738 - Review the existing ephemeris handling code in asdm2MS and fix found bugs. I'm now testing these fixes to make sure that the examples I know of behave as expected. An entire use case in the code was blocked by poor command line argument handling and it may not exist in any data to date, but it's also not excluded by the ASDM so I've revived that code. Any suggestions for ephemeris data to test should be sent to me.
Discussions on the planned move of the ASDM and generated and shared code to a separate model. The full move is being deferred until ALMA developers are more comfortable with git and related changes to the work flow. In the interim I will reorganize the code within the ALMA code tree so that when re revisit this issue next year the actual move to a separate package and git submodule will be relatively simple. This reorganization will also involve the work on removing the boost dependency in the shared code (at least for CASA builds) - CAS-10693.
CAS-10732 and CAS-10665 - both dealing with parts of importasdm being slow. I'll likely spend more time on this in the next few weeks trying to understand where the filler spends most of it's time and looking for low-hanging fruit.
Some GBT work: accepted some contributed gbtgridder code that adds a new "eqweight" option. Some anonymous person created a new JIRA user and submitted a GBT post processing ticket saying "did you know anyone can create a new user account and submit a ticket?". Discussions ongoing on whether that's a feature or a bug and if it's a bug, what to do about. GBTPP-15.
Minor newsletter article revisions
Finished building CASA on loaner OSX10.12 laptop! Relatively smooth compared to previous experience. Thanks Ville!
MSv3 suggestions submitted to Kumar and Martin
Cal library discussions with Lindsey
CAS-9108 (linear feed Xf refactor and related): work continuing
CAS-10772 (repeated use of bandpass+applycal causes creeping flags; see also CAS-10771): testing fix now, soon ready to verify and validate
Some travel prep (@JIVE/ASTRON next week for CASA/VLBI workshop)
Federico M Pouzols
Timing of pipeline tasks in serial versus parallel test runs.
New test scripts and adapting analysisUtils/picklePipeRun to compare parallel/serial runs.
Numerous emails/discussions on various topics (primarily R&D vs production issues w.r.to the PLWG)
Began work on CAS-10768 : step 1 of a resource predictor for imager.
Started timing analysis of some of underlying functions used in automask (CAS-10743)
Some science time (gathering information needed for responding the referees report for a paper that I am co-authoring)