Monday Morning Meeting Nov. 17, 2014

• DIAL-IN NUMBERS & PASSCODES:
• USA Toll Free Number: 866-901-8266
• USA Toll Number: +1-203-566-3863
• PARTICIPANT PASSCODE: 725760

Attendance

• Socorro: Rob, Lindsey, Susan, Sanjay, Jim, Pam, Tak
• CV: Mark, Dave, Andy, Akeem, Darrell
• Garching: Justo, Sandra, Dirk
• SCO:

News / Meetings / Visitors

Build, Release, Testing

• Welcome Akeem Wells to the test group.

• 4.3 Release
  • Were down to 2 blockers last week, waiting for testing. New issues have come up.
    • Importasdm - seems to work, stable over the weekend.
    • (other)
  • CAS-7138, fixed by Susan.
  • Justo's mstransform weightspectrum is implemented and committed. Ready for science testing.
    • Justo and George will test in parallel.
  • George and Lindsey working on a new applycal / vi/vb2 issue (may be fixed, waiting on testing).
  • Pipeline also found some plotms segfaults. (EL5 only).
  • Run setup script for pipeline also not working.
  • Tsys calcs for ALMA also raised by Crystal. George has a report below. (CAS-7141).
    • No agreement that this is a blocker, George will assess ticket. May want online system to flag "bad" tsys values.
  • New SD imaging failure.

• Mark, Darrell and Rob to talk about AOC deployment scripts after the meeting.

Architecture

• Technical Forums: No apparent interest
• Dave's migration of the components library out of casacore will take place on Tuesday barring any late problems.
  • Large change, could cause build problems (only on trunk, not 4.3).

Development

• Problem in setjy, affects 4.2.2 as well as 4.3. MMS compatibility issue, corrupts MMS. Address on trunk since only an HPC issue.
  • See Sandra's report below.

Pipeline

• Focus on wrapping up November targets. Missing some dependencies (from JAO).
• Wrapping up SNR computations; working on plotms integration; 4.3 regressions, etc.

AOB

Developer Reports

Monday Meeting

• Sanjay Bhatnagar
  • Worked on measuring performance of parallel imaging as a function of nodes. 64 nodes with 70-80% scaling efficiency. Higher number of nodes did not work, working issue with James.
  • Worked on msselection for caltables.
• Sandra Castro
  • Added backwards support to split2 to accept selection parameters as int. No support for Array of ints though. Also verified that split2 doesn't have the temporary path problem seen in old split, when time and frequency averaging are requested together.
  • Several tests using the pipeline with MMS. Major stopper now is that setjy corrupts the MMS when there are NULL MS selections in the Sub-MSs. There is one easy fix which is to do like applycal and create the MODEL column at open time (cb.open(addcorr=False, addmode=True)). The other thing to do is to remove all the code that raises exceptions in the middle of the task and do that only at the end, so that ParallelTaskHelper can catch these NULL exceptions and handle them.
  • I need to be able to make changes in CASA, re-build and re-run the pipeline with my local changes. I haven't found a way to do this.
  • Run into the write-lock problem of plotms when running the cycle2 script on MMSs.
• Lindsey Davis
• Pam Ford
  • Resolved PlotMS issues:
    • Two Mac tickets involving /Volumes in the file browser and an error regarding Objective-C autorelease thread pools
    • Two tickets involving iteration indexing
    • Two old already-resolved tickets
    • Menu bug on Mac reported to Susan, tested, and resolved
• Kumar Golap
  • Worked on understanding a small decorrelation that occurs in msuvbin which is a function of distance from phasecenter...not clear if it is to be expected or not.
  • Some time on reviewing the VLASS documents.
• Justo González Villalba
  • Completed weight channel average feature in mstransform
  • Created unit tests for generation of SIGMA_SPECTRUM and WEIGHT_SPECTRUM for time and channel average separately and together in mstransform
• Jim Jacobs
  • Put a mod in that appears to allow subtables to be read locked while the main table is write locked. Requires the use of a new MeasurementSet constructor.
    • Opt-in implementation, so will need assistance from the rest of the team to implement.
  • Looked into cas-6119 (memory leak in measures tool). Looks like there might be a small leak but it appears it might be associated with any use of the record data type across the python/tool boundary. Each call to a record-using tool seems to leak about 300-500 bytes.
• Jeff Kern
• Susan Loveland
  • Plotms: CAS-7137, CAS-7138, CAS-7147
  • Carta region selection
• David Mehringer
  • Stats framework implementation. Trying to figure out exactly where low level code needs to be replaced for image statistics to continue to work. Hoping someone who is in charge of casacore on our side has told our partners this will be happening so I don't get emails from partners asking me what is going on when I start committing casacore files. Also did some performance tuning necessary for very large datasets.
  • More components relocation work. Since no one involved in ASAP took ownership, I went ahead and hacked ASAP CMakeLists.txt file to try to support new name for components library. Sent around email to most affected parties describing changes and risks involved. Will send hacked ASAP file to Tak to commit on Tuesday roughly simultaneously with my components-related commits.
  • (CAS-5878) Provide a way to obtain pointing direction of an antenna corresponding to MS rows Kana responded that she needs essentially arbitrary ways of selecting on an MS. This is best accomplished via MSSelection which msmd does not support. So this method is better
    implemented in the ms tool by someone who understands how MSSelection works. That is not me, so hopefully this ticket will not be assigned to me again.
  • Looked over target list. Noted that none of my targets represent particularly low hanging fruit, so I only anticipate being able to implement three or four at most this cycle given their complexity. The stats framework target itself will likely take at least two more months to implement.
• George Moellenbrock
  • CAS-7136: Migration to VI2/VB2 in applycal internals broke applymode='*strict' (discovered in Lindsey's pipeline testing). This is now fixed.
  • A bunch of JIRA clean-up (more to come...)
  • Considered CAS-7141: Edge channel Tsys are, if not flagged, participating in the channel average used to calibrate weights. Request is for median, but this will cause legitimately high Tsys (e.g., atm lines) to be underrepresented, and thus incorrectly bias the weight calibration. More on this in the coming week.
  • Some residual data examination (Ionosphere related) for one of the EVLA workshop participants.
• Dirk Petry
  • Worked on importasdm with Michel. Working on a casa tutorial for next week.
• Urvashi Rao
• Darrell Schiebel
  • Working on automated builds for OSX. (George willing to be a guinea pig).
• Rob Selina
• Julian Taylor
• Takahiro Tsutsumi
  • Continued investigating the setjy issue for Butler-JPL-Horizons 2012 (CAS-7066). The models of the solar system objects looks reasonable so long as the current trunk or the prerelease was used but the original ticket submitter also reported an issue when it is imaged.
  • Commented on a way to compute angular diameters of the SS objects in CAS-5755
  • Had an initial look at the table lock issue in the pipeline (CAS-7138). Made setjy to clean up the temporary file mentioned in the ticket although it turned out to be not a source of the issue.
  • Attended the Software Discussion on git
  • Spent ~1 day on email exchanges/proof reading for a science paper that I am co-authoring

Friday NAOJ Meeting

• Kanako Sugimoto
  • User support: updated mirroring and CASA installation scripts at EA-ARC. I will hand over the maintenance to Shinnosuke next week.
  • Responded to user test feedback of CASA 4.3 tickets: CAS-6600 sdtpimaging, CAS-6109 sdlist, CAS-6595 sdplot. A couple more to go.
  • Discussion of new sd tools and sdutils ( CAS-6989): decision is to create new sd tool (name TBD) but not to create new sdutil.
  • Chaged ~2/3 of "Open" tickets to "Uscheduled". 11 more to go (need to be revisited).
• Wataru Kawasaki
  • worked on CAS-6985, committed a fix to the trunk ant then set this ticket RTT. An NRO staff Atsushi Nishimura will be verifying once the first casapy-test after CASA4.3 release becomes available.
  • wrote a text for rasterutil module to be included in the CASA 4.3 cookbook and sent it to Bunyo.
  • reviewed the SD part of the CASA 4.3 cookbook. looks OK.
• Masaya Kuniyoshi
  • [JIRA] CAS-4142: "Create regression tests for ALMA SD data" (99.9% finished)
• Takeshi Nakazato
  • importasdm lazy and bdfflags issues: confirmed that fix for CAS-7054 is merged into release branch, no progress on packaging issues for Mac distribution
  • unit test code review
  • 4.3 user test: resolved CAS-6596, CAS-6585, CAS-6571, CAS-6589, found another bug related to CAS-6121 ticket created (CAS-7134)
  • offline discussion related to NRO frequency/velocity issue (CAS-6986)
  • started discussion for introducing libsakura into casa
• Shinnosuke Kawakami
  • CAS-6929: Test building GC-casacore + casa-code with gcc4.4.7 on RHEL6.4 is completed by a few source code modification. But test building gcwrap is 30% completed.