Monday Morning Meeting, May 9, 2016

DIAL-IN NUMBERS & PASSCODES:
  • IP: 192.33.117.12##8110
  • Phone: (434) 817-6524

Attendance

  • Socorro: Jeff, Lindsey, Susan, Martin, George, Kumar, Jim
  • CV: Andy, Akeem, Ville, Anand, David, Jen, Darrell
  • Garching: Dirk, Sandra, Julian

News / Meetings / Visitors

  • Next week: NRAO users committee and ANASAC in Socorro
  • In 2 weeks: pipeline working group F2F in Socorro.
  • In 3 weeks: Data reduction workshop. Tutorials will be run on CASA 4.5.2.

Build, Release

  • git synchronization will be turned on this week. Contact Ville if any strange issues appear in SVN.
  • No word from Pat and Josh about installation of Atlassian tools. Jeff will contact them.
  • Stephan's group will begin using Atlassian tools about the same time as the CASA group.

Verification Testing

  • M100 Analysis Regression has been failing on OSX 10.11
    • ASDM conversion error
    • Dirk will look into this, but it will take him a few days to acquire a machine.
  • MPI tests continue to run very long (not failing)
    • Tests are passing, but exit from CASA is failing. Problem started about 2 weeks ago; seen most recently last night. The viewer is suspect.
  • Pipeline was not working on OSX 10.11 - it couldn't find the pipeline installation due to the directory structure
    • After working on it last week, it appears to be working now
  • Evolving approach to new casaguides (see https://casaguides.nrao.edu/index.php/M100_Band3_SingleDish_4.3)
    • Not being designed to run as a single continous script. Multiple paths are being included, plus new authors do not have CASA testing in mind and therefore some commands needed for testing are missing.
    • Jeff: User's committee has requested that we organize the CASA Guides because they are currently too complicated. CASA Guides should be kept uniform in look and feel and ability to extract a script and make it run.
  • Slowdown in imaging regression test run time
    • Will investigate that to see which revision the slowdown started
  • Test spreadsheet
    • Re-assigning some tests from Jeff to other developers
  • NGC3256 RMS issue -- Andy has not had time to follow up on feedback from George. Andy will attempt to quantify what is happening and send it back to George. This test will continue to fail until this is fixed.

Validation Testing

  • Shrinking the backlog: 45 tickets assigned, 30 tickets closed in the last two weeks (vast majority at the NAASC and VLA)
  • Testing effort to updating (ALMA) CASAguides for SISS in June
  • Significant testing effort this week to new C4R1 pipeline tickets (calibration and imaging), benchmark imaging datasets to be started imminently

Architecture

  • Warnings: You can look at the Warnings Summary page for more details. Looks like there's been a bit of back sliding: flagging has gone from zero warnings up to 16, single dish has gone from 0 to 4 and synthesis from zero to 2. However, with so few, it's likely they can be quickly removed wink
    • Jeff: This needs to be fixed. Please be vigilant and get rid of any warnings.

HPC

  • Have been trying to run the ALMA pipeline in parallel for the X291 3 EBs to test the changes in the MMS structure. The pipeline has run out of memory whenever I tried it in parallel. The machine contains 64 Gb RAM and 32 cores. We will try to debug the failure, but so far it seems to be from tclean. The latest pipeline run went until step 29 before failing.
    • This was tested with latest 4.7 stable; 8 compute engines; tclean was using 98% of RAM and another process consumed the remainer.
    • This is a reminder that some work needs to be done to prevent tasks from assuming they can use all available RAM.

Development

Pipeline

  • pipeline C4R1 branch created and released for pipeline working group testing
  • issues
    • NaNs created in tclean for non-obvious reasons
      • In general, we need to decide when NaNs should be generated.
      • Tak is looking into the tclean problem.
    • Pam is still working on a plotms issue with averaging by antenna.

AOB

  • Lindsay: The cal library does not handle antenna positions.
    • This is not a generic problem; there is something to be fixed.

Developer Reports

Monday Meeting
  • Sanjay Bhatnagar
  • Sandra Castro
    • Attended some of the Obsmode discussions about cycle 5.
    • Have been running parallel tests on the pipeline.
  • Lindsey Davis
    • final C4R1 release testing, debugging, and documentation
    • created C4R1 branch
  • Pam Ford
  • Kumar Golap
    • working on mosaic phase shifting problem
  • Justo Gonz├ílez Villalba
  • Jim Jacobs
    • CrashReporter - Almost integrated with the build + release system with Ville's help. We'll have the needed web client very soon and can then start the first testing. Once we're happy about its workings within the firewall we can let it become internet accessible.
    • A couple tickets that have to do with setting the tile cache size are awaiting Casacore mods before I can commit the corresponding casa-code mods in. Ger is on vacation, so the CC mods will have to wait until then.
    • The tClean work needs some additional features added to the VI2 framework so that clients will not need to do an end-around of the framework to get things done. This will involve adding access to several of the columns as measures rather than native C++ types (time, time_centroid, UVW).
  • Jeff Kern
    • ngVLA Cost Model presentation
    • Preparation for NRAO users committee
    • SKA SDP PDR
      • They passed the PDR and are moving forward. They are planning to use CASA for the first few antennas.
  • Susan Loveland
    • CAS-8585: Spectral Viewer always shows K when launched
    • CARTA: fix of memory leaks, profile work
  • David Mehringer
    • (CAS-3847) Global Image module object to record memory allocation/de-allocation. Asked Ger if he might be able to implement this, but he said it would be very complicated to do, so he likely wouldn't be able to. If he thinks it would be difficult for him to implement, it will likely be even more so for me.
    • Amanda expressed interest in having image analysis tasks/tool methods include any warning messages in image history. Pointed out that this was far beyond the scope of the current image history effort and would require a feature request to management for prioritization, as it is a very non-trivial thing to implement.
      • Jeff: We should come back to this at some point.
    • (CAS-5691) refactor ImageMoments and related classes. WIP. Some refactoring work. Realized MSMoments was no longer being used in our code. Takeshi N says it could be removed, so I removed it from trunk.
    • (CAS-6906) avgchan in exportuvfits not changing number of channels in output: IR. Assigned to Juergen for CSSC input on expected behavior. I'm thinking all the averaging and selection behavior should utlimately be removed from exportuvfits, as all this is possible elsewhere and just adds unnecessarily to complexity and ambiguity in behavior of this code.
    • (CAS-8497) exportuvfits possibly incorrectly writing antenna coordinates. Reassigned to Jeff as there is nothing more I can do with this until my pull request has been removed. I made that request almost a month ago.
      • We've asked nicely for a month without response. It's about time to flush this in.
    • (CAS-8354) Create task imhistory to wrap ia.history(). RTT. Completed various requested documentation changes.
    • (CAS-7986) Create MS Consistency Validation Tool. RTT. Made modification to improve performance.
  • George Moellenbrock
    • testing and calibration mainly
    • some discussions with Chris on polarization calibration
  • Dirk Petry
  • Martin Pokorny
    • I/O testing
  • Urvashi Rao
  • Darrell Schiebel
  • Ville Suoranta
    • Troubleshooted issues with the Pipeline test on OSX10.11.Provided Akeem with instructions on how the packaging scripts do the Pipeline installation.
    • Breakpad (CrashReporter) was breaking bad. Getting Crash Reporter to build on Mac and packaging took longer than expected. Still need to test the make-app changes on Linux and test the Mac report creation.
    • Worked on Bitbucket/Subgit
  • Julian Taylor
  • Takahiro Tsutsumi
    • Fixed summary plot error in tclean (CAS-8576)
    • Investigating "the clean flux issue" (CAS-8584)
    • Looked into ways to make the automask code more modular
Friday NAOJ Meeting
  • Kanako Sugimoto
  • Wataru Kawasaki
  • Masaya Kuniyoshi
  • Takeshi Nakazato
  • Renaud Miel
Topic revision: r15 - 2016-05-09, JimJacobs
This site is powered by FoswikiCopyright © by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding NRAO Public Wiki? Send feedback