Monday Morning Meeting 18 July 2016

• DIAL-IN NUMBERS & PASSCODES:
• IP: 192.33.117.12##8110
• Phone: (434) 817-6524

Attendance

• Socorro: Jeff, George, Pam, Lindsey, Susan, Kumar, Jim, Sanjay, Urvashi, Tak
• CV: Anand, Andy, Ville, Jen, Akeem, Darrell, Dave
• Garching: Sandra, Julien, Justo

News / Meetings / Visitors

• No Meeting Next week
• ACDC Visitors: CV next week (July 26-29), Socorro Following week (Aug 1-3)
• ALMA ICT Admin Meeting / Pipeline Review (July 25-28)
• Casa.nrao.edu is moving to Charlottesville. If you have things that link to casa.nrao.edu or svn.cv.nrao.edu, please email Jeff so he can compile a list of things that need to be updated.

Build, Release

• NRAO upgrades mac to 10.11 on August 5th. A list of machines to stay on 10.10 has been given to CIS.

Verification Testing

• Transition to Bamboo CI test framework
  • Plone page (work in progress): https://casa.nrao.edu/plonedocs/stable/casa-processes/test-definition-and-processes
  • Collection of which tests will be in which test suite is eventually going to be here: https://safe.nrao.edu/wiki/bin/view/Software/CASA/ContinuousIntegrationTestDiscussionForBamboo
• tsdbaseline moved from critical to accepted, since it was preventing a stable from being created
  • NAOJ found a bug; working on a fix.
• Long test run times in Charlottesville
  • Julian was able to see a slowdown in tests on the ESO side using a similar configuration to CV, but still not as extensive
  • I will look in to how the disks are organized on local CV machines
  • Julian has provided a suggested configuration to speed things up in CV
• NGC3256 removed from accepted test list

Validation Testing

• Recent weeks have been focused on pipeline testing. "Regular" s/w testing expected this week.

Architecture

• Warnings Summary
• Continuing to make casa compatible with the casacore namespace.

HPC

• Testing the pipeline in parallel: tclean stalled with a read-lock coming from imageconcat. I replaced the virtualmove by virtualcopy in the task and it works. Informed Tak about it.
• Testing CAS-7702 reported by James last year that tclean was slower when running serially on MMS than on MS. At the time Justo suspected that it could be the OMP_NUM_THREADS setting. Will post the test results soon.
• Julian found a logic problem in the chanchunks calculation (CAS-8809). I'm investigating if this is causing the failures that Felix is seeing when running pipeline 37554 that shows a bad_alloc for many datasets.
• Revised JIRA HPC open tickets.

Development

Pipeline

AOB

• Moving casa.nrao.edu to CV

Developer Reports

Monday Meeting

• Sanjay Bhatnagar
• Sandra Castro
  • Added parallelisation documentation to Plone - on-going.
  • Verified a slice error problem when running flagdata summary mode on an MS (reported by the ALMA pipeline) (CAS-8631). Jim is looking at this as it seems to be in the VI/VB2.
• Lindsey Davis
• Pam Ford
  • Plotcal-to-plotms conversion
    • added axes to plotms and set default axes for calibration tables (CAS-8380)
    • ran some CASAGuides tutorials to make test cal tables
    • meeting with George today to discuss further how to make plotms smarter than plotcal.
  • Debugged 2 test_plotms failures and tested fixes by Jim and Justo.
  • Began Plone documentation on plotms.
• Kumar Golap
• Justo González Villalba
  • Fixed ephmerides propagation problem in mstransform reported by ALMA Pipeline
  • Fixed channel drop problem in MSTRansformIterator reported by plotms (Pam)
  • Fine tuned ALMA M100 parallel regression using tclean to set it ready for Jenkins, now it includes writing to the model column for phase calibrators as per request from Sanjay
• Jim Jacobs
  • Looking into problem reported by Justo where an existing MMS now throws an exception after a May timeframe mod. The VI appears to be expecting one cube shape while a different one is served up.
• Jeff Kern
• Susan Loveland
  • CARTA
    • Finished first draft of spectral fitting code (Ville helped by updating docker image with Gnu Scienfitic Library).
    • Performance analysis revealed bottleneck when doing percentile->intensity conversions. Preliminary implementation of cache system resulted in test image loading 6X faster.
  • Viewer: Plone work
• David Mehringer
  • (CAS-8825) Investigate multithreading for computing quantile values: WIP.
  • (CAS-5996) immath unit- (and if relevant beam-) awareness: RTT. Updated log messages and documentation as requested.
  • (CAS-8783) add parallel support to imstat/ia.statistics(): RTT. Completed implementation.
  • Provided user with information in imstat::axes parameter which resolved his use case issue.
  • (CAS-2278) problems importing stokes I data: IR. Currently not supported as assumptions must be made on how to decompose stokes parameters into correlations. Best solution IMO is to give users complete control and responsibility to use tools in the exporting package to convert their data sets from stokes to correlations. User seems to agree this is the best solution, from dialog in the ticket.
  • Queried Jeff re: plone documentation details. Upon re-reviewing my assignments, I realized I have no non-task, non-tool method related writing assignments.
• George Moellenbrock
• Dirk Petry
• Martin Pokorny
• Urvashi Rao
  • CAS-6675 : Added more EVLA beam models + got vpmanager to use them.
  • CAS-7362 : Fixed a bug with MTMFS not making anything other than Stokes I. Also removed restriction for single stokes plane from the MTMFS minor cycle. (Earlier work was only tested for cube)
• Darrell Schiebel
• Ville Suoranta
  • Resolved Carta CI build issues (on CircleCI /Ubuntu docker image)
  • First stab at firing Bamboo builds via post calls. Still investigating how to get this properly integrated to a Jira workflow
  • Enabled plan branches for the CI build (will create a branch automatically for bugs and feature branches)
  • Updated OSX ports and build systems to wcs-5.15
  • Provided Akeem with a OSX build using qwt6 -> will upgrade build systems if test results look good
  • Resolved CAS-8522 (version number cheat) & CAS-8721(matplolib rc directory)
• Julian Taylor
  • fixed calibration performance regression, gh-460
  • pointed out large performance problem in tclean, CAS-8824
• Takahiro Tsutsumi
  • automask modifications - mostly done, ready for testing (except for interactive mode)
  • Fixed CAS-8790, VI2 image coord issue
  • Looked at VI2 data selection issue for cube imgaing
  • Added a new standard, Stevens-Reynolds 2016 (currently contains only a single source, 1934-638), in setjy (CAS-8711)
  • Initial edit on flux calibrator models in the Plone documentation

Friday NAOJ Meeting

• Kanako Sugimoto
  • A quick F2F brain stroming discussion about things to be investigated and defined for Cycle 5 with Tsuyoshi, Rie, and George.
  • I'm going to give a presentation of the plan to support processing of NRO and ASTE data by CASA/pipeline in 'NRO/ALMA Science/Development Workshop 2016' next week at Nobeyama.
  • Update of ATM library for CASA 4.7 ( CAS-8448): the update contains the fix to a bug reported by Todd, i.e., transmission of an ozon isotope.
  • the fix to segfault of CASA caused by atmosphere tool methods due to negative channel/spw IDs ( CAS-8254): introduced check for negative IDs throughout the tool kit. I understand Todd requested additional check for IDs exceeding permissive range. I believe it is better the check is done in ATM library level but I can introduce it in tool level.
  • Changes to a behavior of tsdfit: avoid task to fail when one of the fit in spectra failed. Just show information of failed fit. Pring warnings if no fit converged.
  • fixed a bug in sdimaging found in user tests of CAS-8415
• Wataru Kawasaki
• Masaya Kuniyoshi
• Takeshi Nakazato
  • Bug fix on CAS-7451 (clipping capability for sdimaging)
  • Discussion with tester of CAS-8419 (support moving source in tsdcal with mode='otf')
  • Discussion with tester from Nobeyama about a test of multi-beam (multi-feed) data handling in CASA
  • Added new task sdgaincal that performs gain calibration of single dish data (CAS-8416)
  • coordination of libsakura update to version 3
• Renaud Miel
  • CAS-8082
    • Prototype / Header Items selection widget: tested modelview combo-box
    • Investigating required data structure
  • libsakura 3.0 packaging + test