Monday Morning Meeting March 16, 2015

• DIAL-IN NUMBERS & PASSCODES:
• USA Toll Free Number: 866-901-8266
• USA Toll Number: +1-203-566-3863
• PARTICIPANT PASSCODE: 725760

Attendance

• Socorro: Jeff, Sanjay, Lindsey, Pam, George, Urvashi, Kumar, Tak, Susan, Jim, Martin
• CV: Dave, Darrell, Mark, Akeem, Ville
• Garching: Dirk, Sandra, Justo
• SCO:

News / Meetings / Visitors

• Martin transitioning in

Build, Release, Testing

• 4.4 Release still working on 10.10 build
• All issues resolved on Linux
  • ASDM Code has been backed out.
  • ASAP Code
• 10.9 has same compilers as 10.10!
• Sensitivity calculator nearly complete and checked in.
• Changes to frequency labeling when combine spectral windows
• Pam biggest issues is getting flagging data back into MS, reindexing, phase center shift, model data if no model data column.
  • Model data -> minor issue
  • Phase shifting -> in conversations
  • Flagging is not trivial
  • Indexing can be solved by internal work-around
• Issue with nterms >1 not writing model column, when exiting based on niter. Document the issue.
• New casarc variable to enable multi-file constructor. Casa core table.

Architecture

• Technical Forums:
  • casacore will be moving to github, Jim will be able to give people access.
    • Unit tests may not be sufficient to catch all failures.
• Git transition: in discussion

HPC

• Due to too many absences we didn't have a meeting last week.
• The last test of the ALMA pipeline + MMS showed that the fix for the wrong tileshape that affected plotms is fixed in both VI/VB2 and old VI/VB. Tasks that create the data columns were affected by this problem.
• The next batch of tests will use a bigger MS which takes longer to process so that the parallelization can be of any help. The other issue is that most of the tasks used by the parallel (calibration tasks) run in sequential. The parallel tasks (flagdata, setjy and applycal) are already quite fast in sequential.
• The layout of the CASA binaries has been fixed so now the mpicasa wrapper works without the need to set environmental variables for the use in cluster nodes. This is available in the latest casa-stable-4.4.112.tar.gz.
• We will start a series of lectures in the HPC group to allow everybody to understand the issues in each development part: mpi, imaging, pipeline, etc.

Development

Pipeline

• pipeline testing against 4.4.110, table column keywords and lazy filler issues have been fixed
• pipeline release testing against 4.3.0, simple spectral mapping and flux calibration heuristics issues, flagging issues
• im.sensitivity, needs the ALMA weights aware version, role of im.defineimage unclear
• ALMA Tokyo conference, paper on ALMA pipeline in preparation by Hiroku Shinnaga and many co-authors
• tclean testing

AOB

Developer Reports

Monday Meeting

• Sanjay Bhatnagar
  • Testing OpenMP and MPI
• Sandra Castro
  • Mostly tested the ALMA pipeline with MMS and setting different values of OMP_NUM_THREADS to verify the fix in plotms. I started a new pipeline batch of tests with a bigger MS.
• Lindsey Davis
  • pipeline / CASA 4.4.110 testing
  • pipeline flux calibration error handing improvements
  • pipeline snr code packaging
  • resumed work on pipeline results document
• Joe Depa
• Pam Ford
  • Extended QtDBusXML class to handle array types in Records for calibration libraries. Tested with George. On-the-fly calibration now works in plotms!
  • Assigned tickets to Justo for MSTransformVI requirements for plotms. One is a blocker: implement writeFlag() in new framework to allow users to flag data in the plot and have these flags written to the input MS.
  • Mac system in my office upgraded to OSX 10.10, working on installing casa trunk there for plotms bugs
  • Recompiled the latest plotms with RelWithDebInfo build type for updated performance tests
• Kumar Golap
  • Tested, debugged and documented linearmosaic tool lm
  • Fixed a bug related to ms iter and AsdmDataManager
  • Working on a bug of mosaic and outlier field
  • support, discussion etc..
• Justo González Villalba
  • Fixed tile shape issue for the MMS case in the CORRECTED/MODEL columns initialization (by setjy, applycal)
  • Discussions about phaseShift interactions with other transformations, OpenMP defaults, TVI performance
  • Investigated logfilter issues in the parallel context, sent a proposal to Tak.
• Jim Jacobs
  • Submitted paper to A&C special issue on formats about MeasurementSets
  • Got AVI2 passing my unit tests and test_mstransform regression test. Need to profile it now.
  • Took a quick look into request from Kana about workings of VI2 regarding rowIDs; I thought there was a logic error, but on further examination I think I was mistaken. Will have to look further.
• Jeff Kern
• Susan Loveland
  • CARTA Histogram and Animator
• David Mehringer
  • Discussion ongoing with Marsha re: NRAO's subscription to Safari. We are authenticated using IP addresses, which means I cannot use the Safari app on my phone. Also, there are issues displaying Safari books with my phone's browsers. So far, I'm not impressed with the quality of accessibility we get via NRAO vs the quality of accessibility I have gotten by using a personal account.
  • Looked into the imregrid issue in "(CAS-6095) IMREGRID and IMSTAT cannot read Galactic Coordinates in GBT data". I'm unsure if this type of regridding using CAR projections is even possible. Certainly, wcslib is having issues with CAR projections.
  • Responded to a question Kumar passed on from a user re: debugging messages. Suggest users don't try to understand what debugging messages mean (IMO putting debugging messages in production code is not a good idea anyway for performance reasons).
  • Looked at "(CAS-7257) discrepancy between spectra extracted by imval and by spectral profiler in viewer". imval seems to be doing what is advertised IMO. Have put the ticket in Input Required and waiting for response from Dirk.
  • Fixed test_boxit which I broke with a PixelValueManipulator commit.
  • Responded to Justo re: his requests for additions to msmd.summary() data structure.
  • Completed "(CAS-7389) Refactor LatticeStatistics to remove as much beam/image code as possible"
  • Communicated with Ger re: removing several hundred lines of plotting code from LatticeStatistics since we no longer use it. He's fine with me removing this code.
  • "(CAS-7207) Flux density not working for multiple beam one per channel image". Started work on this, but realized I needed to refactor LatticeStatistics (as mentioned above) and that this fix was better located in ImageStatistics than in ImageStatsCalculator, where I first tried to put it.
• George Moellenbrock
  • CAS-7285: With Pam, finished wiring plotms for OTF calibration. Needed to fix flagging in the cal library framework---it hadn't been upgraded to do chan-dep flagging last year.
  • Re-implemented the code underlying im.sensitivity to do a more straightforward calculation of imaging sensitivity based on (presumed) calibrated weights. Almost ready to check in.
  • CAS-4436: Analyzed a fix for combine='spw' frequency labeling in CalTables. For v4.4, we will support the case of a single spw aggregation.
  • Various discussions with Pam, Jim, Justo re VI2/VB2 issues in plotms.
  • User support: discussions with Crystal re calibrated data weights in combination of 7m and 12m arrays.
• Dirk Petry
  • improved radial velocity correction in cvel (CAS-7270) and dTdL calculation in wvrgcal (new refant parameter, related to CAS-6952), working on CAS-7200 (wvrgcal with MMS)
  • meeting with Ger van Diepen and Des Small at ESO (HILADO): wondering when we will really switch to a common casacore repository; + advice for fringe fitting task work
• Urvashi Rao
  • cas-7350 : Looked at problem of model col not being written sometimes with mtmfs in clean. Verified and made edits to ensure it happens correctly on the last major cycle, in tclean.
  • Went through bug-reports sent in by naasc testers who inadvertently tried out tclean few months ago while under daily code dev - fixed most of them. Added responses to their wiki. Also had some conversation with pipeline folk about testing tclean.
  • Looked at code regarding a bug reported and worked on by Tak : calc of sumwt properly for non stokes-I imaging with tclean
  • Discussed with Jeff/Tak/KG a bit about what we will focus on in the refactored imager in the short term.
  • Built casa on laptop, after iterating with Soc helpdesk about os updates and libsakura. Can't yet test parallel runs as mpi libs are needed now ?
• Darrell Schiebel
• Julian Taylor
• Takahiro Tsutsumi
  • setjy unit tests modifications - mostly for testing properly for MMS
  • tclean related discussion wih Urvashi
  • Investigated tclean 'cube' issues previously a tester reported in NAASC wiki

Friday NAOJ Meeting

• Kanako Sugimoto
• Wataru Kawasaki
• Masaya Kuniyoshi
• Takeshi Nakazato
• Shinnosuke Kawakami