Monday Morning Meeting 4 April 2016

• DIAL-IN NUMBERS & PASSCODES:
• IP: 192.33.117.12##8110
• Phone: (434) 817-6524

Attendance

• Socorro: Jeff, George, Susan, Pam, Martin, Sanjay, Urvashi, Kumar
• CV: Akeem, Andy, Dave, Jen, Ville
• Garching: Sandra, Justo,
• SCO:

News / Meetings / Visitors

• CARTA team in CV.

Build, Release

• Asap & Casacore checkout problems
  • Failing during checkout
• Disabled tSimpleSimVi2_GT unit test due to Jenkins problems. The test works in one workspace, but not others. Possibly a problem with parallel test runs, but couldn't replicate the problem on a local machine.
• No clean 4.6.0-19 test run on RHEL6 yet.
  • Testing infrastructure issues. This will be the release candidate.
  • Issue with XCLASS and build my tasks

Verification Testing

• NGC 3256 regression is failing in 4.7
  • looks like applycal
  • CAS-8475 opened
• Can now submit jobs to the cluster automatically - pipeline and MPI tests currently being run autonomously
• Discussion with ALMA casaguide lead (Jim Braatz) and Jen
  • He will look at automatically produced calibrated data being done by the current scripts for validation
• Prerelease testing is turned off on linux due to hardware limitations - focusing on getting RH7 accepted testing in place
• Testing spreadsheet to be sent around to the team.

Validation Testing

• CAS-8313 appears resolved, but still two plotms tickets under test waiting on new build (CAS-8337 and CAS-8261, thanks to Pam for very quick fix turnaround)
• updates on stakeholders' meeting and status of CASA testing architecture?

Architecture

• Technical Forums:

HPC

• Still working on a change to the MMS structure, to have links to all sub-tables instead of a copy for each SubMS, (CAS-7896). There are some issues with virtualconcat when I apply these modifications. TBV.
• Dave asked if there is any policy on whether or not developers may use the C++11 concurrency API. After a lot of discussions by email it seems that concurrency will be needed but some considerations need to be taken, especially when two different frameworks are operating at the same time. The discussion then diverged to the use of FFTW in CASA.

Development

• Threading Model
  • Stick to open-MP for now.
• Documentation

Pipeline

• Pipeline callibrary module improvements for VLASS, use of interval trees rather element trees
• ALMA imaging pipeline, separate calibration and imaging MS list support, UV continuum regions definition and subtraction support
• New flagging heuristics testing, ALMA Tsys flagging, EVLA target flagging

AOB

Developer Reports

Monday Meeting

• Sanjay Bhatnagar
• Sandra Castro
  • Updated the cookbook and the release notes.
• Lindsey Davis
  • Finished prototyping new components of the ALMA interferometry imaging pipeline
    • new pipeline tasks hif_mstransform, hif_uvcontfit, hif_uvcontsub
    • added support for continuum regions file to hif_uvcontfit
• Pam Ford
  • CAS-8374 in progress (port casaqt to Qwt 6.0) - compiles on RH7 but gets seg fault
  • CAS-8261 RTT (seg fault with antenna averaging) - fixed in 4.6 and trunk
  • CAS-3622 RTT (display the reference frame on the plotms frequency axis)
  • CAS-2656 RTT (plot the SNR column for cal tables)
• Kumar Golap
• Justo González Villalba (last 3 weeks)
  • Prepared mstransform article for Newsletter
  • Investigated tclean issues in M100 regression
    • Apparently Memory Resident Subtables was the cause for the tclean-clean differences, reported it to Jim
    • Temporary file creation concurrency problems have appeared again, I reported it to Urvashi
  • XCLASS
    • Discussed input image format for XCLASS line identification algorithm
    • Tested XCLASS v1.7.1 with CASA 4.6.0-14 pre-release, found problems with buildmytask, reported it to Darrell
  • mstransform 4.6 fixes
    • Added reindex parameter to mstransform interface, and make it compatible with the re-gridding code
    • Fixed seg. fault that when working with single channel SPWs
    • Fixed phase shifting problem that appears in conjunction time average in plotms (CAS-8431)
    • Fixed flag row problem in SPW and DDI subtables that appears after combining/separating SPWs (CAS-8371)
    • Updated cookbook
• Jim Jacobs
• Jeff Kern
  • Documentation Effort
• Susan Loveland
  • Viewer:
    • CAS-7860: New task-based interface for image moments (inline help)
    • CAS-8458: Viewer spectral profile tool line list display should include line temperature
    • CAS-8457: Viewer spectral profile tool line list displays insufficient frequency precision.
  • CARTA: Layer group work.
• David Mehringer
  • (CAS-8463) ia.moments() should write history to created images. RTT
  • (CAS-8465) ia.pad() should write history to created image. RTT
  • (CAS-8466) ia.crop() should write history to created image. RTT
  • (CAS-8446) specsmooth output does not have synthesized beam. RTT. Fixed
  • (CAS-8468) ia.putchunk() should write history to image. RTT
  • (CAS-8470) ia.pv() should write history to created image. RTT
  • ia.maskhandler() should write history to image. RTT
  • (CAS-8469) ia.putregion() should write history. RTT
  • Fixed imagetest regression test failure
  • Provided 0.5 TB MS to Brian Kent for his testing purposes
  • (CAS-4735) imsmooth needs additional option for single dish use. RTT. Updated logger message, documentation as requested.
  • (CAS-7081) After changing RECEPTOR_ANGLE in MS either exportuvfits or importuvfits resets it to zero. RTT. Could not reproduce the issue in a recent build reported by tester, but discovered that exportuvfits is not honoring spectral window dependence in the FEED table. So looks like I will have to modify it to produce one AN table for each spectral window described in the FEED table, since FITS AN tables do not have a corresponding column but instead use FREQID as an AN table keyword.
  • (CAS-8447) modify MSMetaData to allow multi-threading. WIP. Iterating with Julian on casacore modifications. Original method to get scan and subscan data took 500 seconds on Steve's 0.5 TB dataset. New serial implementation takes 150 seconds. Parallel implementation takes 20 seconds on my 8 core Xeon box.
  • Monitored discussion re: C++11 concurrency API I asked our HPC gurus. It sounds like there are no major objections to using this API in our code.
• George Moellenbrock
• Dirk Petry
• Martin Pokorny
• Urvashi Rao
  • Fixed a couple of bugs in model scattering for multi-term parallel runs.
  • Added a unit-test suite for parallel tclean for Akeem/Andy to try out.
  • Added more inline documentation for widebandpbcor (to address two frequently asked questions from users)
  • Continued testing joint wideband mosaics with real ALMA data (the MosaicFT + MTMFS combination).
  • Non casa work (paper drafts).
• Darrell Schiebel
• Ville Suoranta
• Julian Taylor
  • improved efficiency of tiled storage manager gh-334
  • made HostInfo::numCpus() affinity aware gh-353
• Takahiro Tsutsumi
  • Worked on to modify automask in tclean based on the comments on CAS-7924

Friday NAOJ Meeting

• Kanako Sugimoto
  • Sent release notes and known issues of casa 4.6 to Juergen (I believe Erik is still working on cookbook)
  • finalized casa 4.7 development targets, assignments, and shchedule. Updated casa 4.7 planning wiki
  • changed default intent selection in sdimaging ( CAS-8238): the new default, intent='OBSERVE_TARGET#ON_SOURCE'.
• Wataru Kawasaki
• Masaya Kuniyoshi
  • I'm finishing off refactoring the visstat2 unit tests (CAS-7449) today.
  • discussed CAS7449 (visstat2) with Kana, leading to an idea that the implementation of timeaverage in visstat2 including its unit tests should be a new ticket for Martin.
• Takeshi Nakazato
  • Collected documents required for implementation of NRO filler, added supplementary comments, and provided to Wataru who is supposed to be working on the implementation
  • asap NRO filler bug fix (proper string handling)
  • follow up of discussion on CAS-8416
  • started defining unit test for CAS-7451
• Renaud Miel
  • CAS-7339: tsdcal.calmode=otf
    • committed code yesterday
    • next step: add Python unit tests
  • CAS-8262: Sakura rpm for RHEL 7
    • Built, tested, rpm-packaged, rpm-installed Sakura rpm on RHEL 7.2 virtual machine
    • Next step: upload rpm to JIRA for testing at NRAO