Code that was introduced some years ago as a "temporary fix" to mitigate a memory hungry feature/issue in casacore.
Response from Ger: that feature of casacore::TiledStorageManager is still there and there is no intention to change this TSM behaviour - there could be an option to disable auto-sizing when too large.
One quick solution that reduces the slowdown from ~1000% to ~50% for a typical example: use a simple Table instead of full MeasurementSet (that's all ROTiledStManAccessor needs), or ad-hoc new minimalistic MeasurementSet constructor.
... but do we need elms.setMemoryResidentSubtables (mrSubtables), or other tables opened? This is not done in the VI2. Not clear why this was introduced in the Vi1.
The issue of caches, memory resident tables, how MeasurementSet is constructed etc. seems to call for a bit of architectural/design re-thinking.
Stats viewer issues: 17% of all NRAO helpdesk tickets related to visualization, mostly viewer (% for ALMA helpdesk substantially lower). Of these 17% critical/bugs, 44% major, 28% minor, 11% user 'error'. Bugs/critical: HD10827: plotms crashes under certain condition --> fixed in 5.1; HD11209: plotting rotation angles wrong; HD11237 Table array conformance error for data set with different size spw, even if only 1 spw is plotted
Build, Release
Verification Testing
Emergency update on the two OSX 10.13 machines due to a gaping security flaw
Since NRAO does not officially support 10.13, it's up to us to manage these machines
Added support for "import as imaging measurement" set to the pipeline importdata tasks with default=False
Added check for empty pipeline callibrary to pipeline applycal tasks
Changed interpolation for calmode='a' solint='inf' tables from linear,linear to nearest,linear, pipeline callibrary handles same table different apply rules transparently
Callibrary discussions with George and Juergen
ICT Archive / Pipeline tickets refactored into separate archive and pipeline error handling tickets
Bjorn Emonts
Pam Ford
CAS-10900: Completed CTSummary class for plotms summaries for cal tables, ready for testing
CAS-10774: Failed to reproduce pipeline plotms failures which appear to be plotms crashes (plots succeeded for me). Contacted others to request enabling core files for user vlapipe. Notified Darrell that crash report poster is not working on el6.
Pull requests merged for CAS-10818 (ms.selectinit with invalid DDID) and CAS-10848 (ms.getdata with capitalized item) bug fixes
Enrique Garcia
Working on CAS-10458: Vi/VB2 vb->visCubeCorrected() segfaults if no column. Improving unit tests on VI/VB2 while at that.
Fixes on casacore related to MsFactory MS simulator.
Bob Garwood
CAS-6832 importasdm should be able to import data up until the first corrupt scan. This was a problem for some cycle 2 data (ICT-331). The appropriate fix to asdm2MS is being negotiated.
CAS-10949 Change the behavior of asdm2MS when ASDMBinary file(s) are missing. Ongoing discussions.
CAS-10386 Add new shared ALMS code to support 2 new ASDM tables in asdm2MS. Testing with new Pulsar table from the VLA. Appears ready for validation.
CAS-10644 Filler needs to detect duplicate (sets of) rows and retain only one copy. Working for Pointing, back to working on this problem for the main table for WVR data (duplicate DATA values).
Kumar Golap
David Mehringer
George Moellenbrock
Dirk Petry
Martin Pokorny
Progress on asynchronous VI2: basically working, now addressing performance issues
Federico M Pouzols
Profiling and trying to understand CAS-10795, casacore storage managers, etc.