Socorro: Jeff, Sanjay, Lindsey, Pam, George, Urvashi, Kumar, Tak, Susan, Jim, Martin
CV: Dave, Darrell, Mark, Akeem, Ville
Garching: Dirk, Sandra, Justo
SCO:
News / Meetings / Visitors
Martin transitioning in
Build, Release, Testing
4.4 Release still working on 10.10 build
All issues resolved on Linux
ASDM Code has been backed out.
ASAP Code
10.9 has same compilers as 10.10!
Sensitivity calculator nearly complete and checked in.
Changes to frequency labeling when combine spectral windows
Pam biggest issues is getting flagging data back into MS, reindexing, phase center shift, model data if no model data column.
Model data -> minor issue
Phase shifting -> in conversations
Flagging is not trivial
Indexing can be solved by internal work-around
Issue with nterms >1 not writing model column, when exiting based on niter. Document the issue.
New casarc variable to enable multi-file constructor. Casa core table.
Architecture
Technical Forums:
casacore will be moving to github, Jim will be able to give people access.
Unit tests may not be sufficient to catch all failures.
Git transition: in discussion
HPC
Due to too many absences we didn't have a meeting last week.
The last test of the ALMA pipeline + MMS showed that the fix for the wrong tileshape that affected plotms is fixed in both VI/VB2 and old VI/VB. Tasks that create the data columns were affected by this problem.
The next batch of tests will use a bigger MS which takes longer to process so that the parallelization can be of any help. The other issue is that most of the tasks used by the parallel (calibration tasks) run in sequential. The parallel tasks (flagdata, setjy and applycal) are already quite fast in sequential.
The layout of the CASA binaries has been fixed so now the mpicasa wrapper works without the need to set environmental variables for the use in cluster nodes. This is available in the latest casa-stable-4.4.112.tar.gz.
We will start a series of lectures in the HPC group to allow everybody to understand the issues in each development part: mpi, imaging, pipeline, etc.
Development
Pipeline
pipeline testing against 4.4.110, table column keywords and lazy filler issues have been fixed
pipeline release testing against 4.3.0, simple spectral mapping and flux calibration heuristics issues, flagging issues
im.sensitivity, needs the ALMA weights aware version, role of im.defineimage unclear
ALMA Tokyo conference, paper on ALMA pipeline in preparation by Hiroku Shinnaga and many co-authors
tclean testing
AOB
Developer Reports
Monday Meeting
Sanjay Bhatnagar
Testing OpenMP and MPI
Sandra Castro
Mostly tested the ALMA pipeline with MMS and setting different values of OMP_NUM_THREADS to verify the fix in plotms. I started a new pipeline batch of tests with a bigger MS.
Extended QtDBusXML class to handle array types in Records for calibration libraries. Tested with George. On-the-fly calibration now works in plotms!
Assigned tickets to Justo for MSTransformVI requirements for plotms. One is a blocker: implement writeFlag() in new framework to allow users to flag data in the plot and have these flags written to the input MS.
Mac system in my office upgraded to OSX 10.10, working on installing casa trunk there for plotms bugs
Recompiled the latest plotms with RelWithDebInfo build type for updated performance tests
Kumar Golap
Tested, debugged and documented linearmosaic tool lm
Fixed tile shape issue for the MMS case in the CORRECTED/MODEL columns initialization (by setjy, applycal)
Discussions about phaseShift interactions with other transformations, OpenMP defaults, TVI performance
Investigated logfilter issues in the parallel context, sent a proposal to Tak.
Jim Jacobs
Submitted paper to A&C special issue on formats about MeasurementSets
Got AVI2 passing my unit tests and test_mstransform regression test. Need to profile it now.
Took a quick look into request from Kana about workings of VI2 regarding rowIDs; I thought there was a logic error, but on further examination I think I was mistaken. Will have to look further.
Jeff Kern
Susan Loveland
CARTA Histogram and Animator
David Mehringer
Discussion ongoing with Marsha re: NRAO's subscription to Safari. We are authenticated using IP addresses, which means I cannot use the Safari app on my phone. Also, there are issues displaying Safari books with my phone's browsers. So far, I'm not impressed with the quality of accessibility we get via NRAO vs the quality of accessibility I have gotten by using a personal account.
Looked into the imregrid issue in "(CAS-6095) IMREGRID and IMSTAT cannot read Galactic Coordinates in GBT data". I'm unsure if this type of regridding using CAR projections is even possible. Certainly, wcslib is having issues with CAR projections.
Responded to a question Kumar passed on from a user re: debugging messages. Suggest users don't try to understand what debugging messages mean (IMO putting debugging messages in production code is not a good idea anyway for performance reasons).
Looked at "(CAS-7257) discrepancy between spectra extracted by imval and by spectral profiler in viewer". imval seems to be doing what is advertised IMO. Have put the ticket in Input Required and waiting for response from Dirk.
Responded to Justo re: his requests for additions to msmd.summary() data structure.
Completed "(CAS-7389) Refactor LatticeStatistics to remove as much beam/image code as possible"
Communicated with Ger re: removing several hundred lines of plotting code from LatticeStatistics since we no longer use it. He's fine with me removing this code.
"(CAS-7207) Flux density not working for multiple beam one per channel image". Started work on this, but realized I needed to refactor LatticeStatistics (as mentioned above) and that this fix was better located in ImageStatistics than in ImageStatsCalculator, where I first tried to put it.
George Moellenbrock
CAS-7285: With Pam, finished wiring plotms for OTF calibration. Needed to fix flagging in the cal library framework---it hadn't been upgraded to do chan-dep flagging last year.
Re-implemented the code underlying im.sensitivity to do a more straightforward calculation of imaging sensitivity based on (presumed) calibrated weights. Almost ready to check in.
CAS-4436: Analyzed a fix for combine='spw' frequency labeling in CalTables. For v4.4, we will support the case of a single spw aggregation.
Various discussions with Pam, Jim, Justo re VI2/VB2 issues in plotms.
User support: discussions with Crystal re calibrated data weights in combination of 7m and 12m arrays.
Dirk Petry
improved radial velocity correction in cvel (CAS-7270) and dTdL calculation in wvrgcal (new refant parameter, related to CAS-6952), working on CAS-7200 (wvrgcal with MMS)
meeting with Ger van Diepen and Des Small at ESO (HILADO): wondering when we will really switch to a common casacore repository; + advice for fringe fitting task work
Urvashi Rao
cas-7350 : Looked at problem of model col not being written sometimes with mtmfs in clean. Verified and made edits to ensure it happens correctly on the last major cycle, in tclean.
Went through bug-reports sent in by naasc testers who inadvertently tried out tclean few months ago while under daily code dev - fixed most of them. Added responses to their wiki. Also had some conversation with pipeline folk about testing tclean.
Looked at code regarding a bug reported and worked on by Tak : calc of sumwt properly for non stokes-I imaging with tclean
Discussed with Jeff/Tak/KG a bit about what we will focus on in the refactored imager in the short term.
Built casa on laptop, after iterating with Soc helpdesk about os updates and libsakura. Can't yet test parallel runs as mpi libs are needed now ?
Darrell Schiebel
Julian Taylor
Takahiro Tsutsumi
setjy unit tests modifications - mostly for testing properly for MMS
tclean related discussion wih Urvashi
Investigated tclean 'cube' issues previously a tester reported in NAASC wiki