Thursday Morning Meeting 06 July 2017

  • DIAL-IN NUMBERS & PASSCODES:
  • IP: 192.33.117.12##8110
  • Phone: (434) 817-6524

Attendance

  • Socorro: Pam, Urvashi, Martin, George, Tak, Lindsey
  • CV: Bob, Akeem, Darrell, Ville, Andy, Dave
  • Garching: Sandra, Federico, Enrique
  • SCO:

News / Meetings / Visitors

  • Morgan in Garching next week for ALMA ICT meeting
    • Lindsey: If an ICT meeting, is there an agenda?
    • Morgan: It is a regularly scheduled management meeting, and there is an agenda
    • Lindsey: interested in release issues and cycle 5 discussion
    • Morgan: Probably a lot of talk about B&R for ALMA software. Rafael will present an overview of OODT, and how it will be used in VLASS. Also address issue of aligning different architecture issues (e.g. Archive and Pipeline). I will send out a link, hopefully everyone will be able to see the agenda. The agenda is pretty dynamic, so anything online may be misleading.

Build, Release

  • 5.0 release
    • Morgan: Calling in folks in the IT side to getting the proper versions of CASA in place on system networks. trying to get it out the door. Packaged it up since the 30th. Juergen has the release notes done.
    • Darrell: Once the tar balls are available on the website, they get installed automatically, don't they?
    • Morgan: Mike Hatz and K. Scott might be the IT folks to talk to about local NRAO installation
    • Urvashi: Andrew Mertland in Socorro
    • Morgan: one update from Japan for a known issue is incoming
    • Ville: Working on removing ASAP, it is pretty close to being done (currently under user test in Japan)

Verification Testing

  • Tests not catching recent bugs
    • Adding mpi4casa and refimager_parallel to Test Suite 2 for now
    • Could be related to ASAP removal
      • CAS-10077 sdimprocess had a dependency in mpi4casa, which caused a break in mpi4casa
    • mpi4casa (stripped down version) would have caught this issue
      • Sandra: possible to select some subtests within a single test
      • Ville: A separate wrapper script would keep things more clear in the test system (reports would be less cryptic)
      • Akeem: Not sure robot framework could take the html, which could be significant work
      • Darrell: for 5.1. we don’t have the time to modify the test framework.
      • Urvashi: write a wrapper test script
    • Long term plan
      • George: Hasn’t gotten used to the various test layers. It would be nice to be able to launch tests locally and manually, to get an idea on test coverage. Automated tests tend to obscure what is actually getting testing. What is the policy on people committing code, then going on vacation?
      • Ville: The issue was not knowing where Kumar was, and who was doing what. One thing we have in the test system is not all tests are run before the merge. The good news is a recent Bamboo update brings in a new feature: branch creation in bamboo at pull request. Might get better test coverage cheaply because of that.
  • Plan for MPI on OSX?
    • Sandra: We don’t have a cluster running OS X.
    • Ville: On Mac we exclude MPI tests. Is this how we want to do it? When somebody runs MPI on Mac, should be there be a warning?
    • Darrell: Could a user run the Mac as a client, and actually use computing power on a cluster node?
    • Sandra: I'll create some JIRA tickets to have a roadmap on how to deal with MPI/OSX interaction

Validation Testing: Jen is AoD

Architecture

HPC

  • Lindsey: HPC is holding up pipeline testing.

Development

  • Imaging
    • Made a JIRA label called 'Imager_BugFixes_5.1'
    • CAS-10280 (a fix for a huge slowdown during data selection for some datasets) broke an acceptance test that got run only after the code was merged to master. Changes were backed out (as KG left on travel), but some investigation continues as the resulting build causes other tests to fail. A new ticket reporting a new bug (CAS-10352) could be related.
    • VLASS pointed out an inefficiency in how the Field table is read. KG knows what to fix as he did something similar for the Pointing table recently (1-2 days of work when he gets back).
    • Parallel continuum : SB investigating possible normalization/precision issues.
    • Parallel cube : TT investigating parallel cube non-convergence issues. This is blocking parallel pipeline tests at ESO ( CAS-10078 )
  • Statwt mostly under validation. One development detail needs discussion with stakeholders as the initial requirement document doesn't address it. Following up.
    • Urvashi: Morgan - Who to contact for statwt requirements document?
    • Morgan: I don’t know, Jeff usually does this, I will check in to it.
    • Urvashi: If she hears nothing from Morgan, she will poke around locally in Socorro.
  • Polarisation calibration - George ?
    • George: Polarization testing - 2 high priority items for VLASS: refant (ready for validation), also tdc corrections - discrepancy was only in linear feeds, non-linear feeds should be OK. Needs to do a little work to verify/validate that. Third thing that George can get done before he goes on vacation. Mostly clean up and streamlining (not changing heuristics) - these things may slip. Away from next Thursday through the end of the month.
  • plotcal - Pam ?
    • Pam has not started on it for 5.1
  • Do we have an initial 5.2 requirements list from the stakeholders ?
    • Urvashi: 5.2 initial requirements list?
    • Morgan: we should work on that, but there is not much in JIRA
    • Urvashi: Juergen usually provides a spreadsheet to Jeff for future release planning, then Jeff puts them in to JIRA.
  • Urvashi: getting NAOJ folks to refactor single dish to use refimager.
  • George: owes a review of VI2 stuff for Takeshi

Pipeline

  • ALMA cycle 5 workflow and lifecycle schedule, support, and deployment issue may impact pipeline
  • Much iteration on Cycle 5 ALMA recipe and data product naming conventions
  • CASA 5.1 / Cycle 5 pipeline testing
    • New pipeline corrected amplitude / model based flagging tasks and imaging precondition check task under test
    • Tclean convergence problems still blocking HPC testing
    • Tclean with autoboxing seems more sensitive to calibration errors than it is without particularly for calibrator sources
    • Intensive VLASS testing

AOB

  • Plone being included in CASA package
    • Morgan: some discussion on including plone documentation. Darrell and Kelly have been working on this. More to come on that. This is wrapping up plone documentation with the CASA package.
  • Open position statuses:
    • Susan Loveland replacement:
      • Having interviews for the Susan Loveland replacement (these are phone interviews).
    • CASA lead:
      • Morgan: 3 candidates coming in to onsite interviews.
      • Urvashi: Can we name names?
      • Morgan: not yet, due to confidentiality issues.

Developer Reports

Monday Meeting
  • Sanjay Bhatnagar
  • Sandra Castro
    • Working with Sanjay on how best to add the EXPOSURE value into the Matrix returned by MSSel getTimeList(). This is needed for flagdata when used in list mode. CAS-10142. This work raised the issue that it is not obvious and clear how to use someone's casacore fork and branch. Federico and I went through a trial and error procedure until we got it working. This use-case should be added to the git Plone documentation.
    • Cleanup of JIRA tickets; assigned fix versions and closed unnecessary old tickets.
  • Lindsey Davis
    • ALMA Cycle 5 workflow discussions
    • Work on ALMA workflow issues including standard recipes and product names with ALMA ICT (Rachel) and PWG
  • Pam Ford
    • CAS-9053: adding sky temperature overlay to plotms for bandpass plots
    • CAS-10316: wrapping up validation testing and documentation for new ms tool functions
  • Enrique Garcia
    • Merge CAS-9864 in 5.0 and 5.1
  • Bob Garwood
    • CAS-7861 : CASA dies on VLA ephemeris datasets. Fixed a problem introduced into the filler awhile ago. The main fix for this issue has been in the code for some time, but unverified. This is now Under Validation, waiting for the tester to check if both issues are fixed.
    • ALMA ASDM tickets
    • GBT issues
  • Kumar Golap
  • Jeff Kern
  • David Mehringer
  • George Moellenbrock
    • At conference last week.
    • Revised expectations for 5.1, bearing in mind vacation last 2 weeks of July.
  • Dirk Petry
  • Martin Pokorny
  • Federico M Pouzols
    • CAS-10359 Opened to compile documents and keep track of discussions/issues related to data weights, spawning from CAS-9853 discussions.
    • CAS-9089 ready for validation of calibrations in pipeline, but giving a last chance to zeep, though it looks like we'll have to go ahead with suds (very old, unmaintained) as it is the only Python SOAP client package able to handle the non-standard responses from the TMCDB AntennaPadService at the moment.
  • Urvashi Rao
    • Discussions with various people about 5.1 details
    • Continued email conversation with Takeshi about code details for incorporating single dish imaging into the refactored imager.
    • Continued work with two summer students.
  • Darrell Schiebel
  • Ville Suoranta
    • CAS-10313 - Patching Macports bug in Tk/Tkinter
    • Rolling out 5.0.0 packages
    • Various Jira/Bamboo/Bitbucket issues and discussions
    • Working on CAS-10355 - CMakeList issues
  • Takahiro Tsutsumi
Friday NAOJ Meeting
  • Kanako Sugimoto
  • Wataru Kawasaki
  • Masaya Kuniyoshi
  • Takeshi Nakazato
  • Renaud Miel

-- UrvashiRV - 2017-07-06
Topic revision: r13 - 2017-07-06, AndyHale
This site is powered by FoswikiCopyright © by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding NRAO Public Wiki? Send feedback