CASA User Testing Meeting Minutes - 2015-07-07, 09:00 Socorro / 11:00 Charlottesville

Post-meeting minutes notes appended in blue text.

Agenda

Participants

Present: Mark R., Crystal, Amanda, Akeem, Mark L., Juergen, Steve

Apologies: Anand (dentist)

Locations

https://staff.nrao.edu/cal/RoomListing.htm

  • Socorro: AOC-317; IP: 146.88.6.8
  • Charlottesville: Room 331; IP: 192.33.117.19


Minutes (Main Points Only)

Status

  • 4.4.0 has been released for all supported platforms.
  • 4.5 feature freeze scheduled for August 1st 2015. Test/stable/monthly packages are being produced.
    • The same general-purpose CASA launcher script has now been deployed at CV, SOC and GB. Details are on the appropriate drop-down menu of the public CASA pages at http://casa.nrao.edu
      • For anyone using test machines zuul01, 03 and 04 are running RHEL6, zuul02 is running RHEL5. REMEMBER: Use /usr/bin/casa on these machines to make sure that you are running the new test/stable packages (there are RPM installations on these).

Number of CASA Jira tickets with "Ready To Test"/"Under Test" status (including EA, EU, JAO, Software-only tickets, but excluding Pipeline) prior to review: 59.

Number of CASA Jira tickets with "Ready To Test"/"Under Test" status (including EA, EU, JAO, Software-only tickets) following review/meeting: 52.

Testing Tracking

  • New ticket assignments.
  • The "nag list", "input required" items
  • Major test coverage areas for 4.5 testing
      • tclean testing to be handled as a short-term project all of its own. In principle, a strategy has been agreed upon for ALMA between Jeff & Crystal. Similar approach for the VLA testing effort?
        • Dedicated parent tickets with systematic subtasks specifically to track tclean testing efforts have been set up: CAS-7664 and CAS-7669. The main questions to be addressed will be: (1) Does the user interface work as expected? and (2) Does it actually do the right thing?
      • An ALMA Pipeline testing package intended purely for internal NRAO testing use only has been put together that makes use of tclean. It is expected that the Pipeline people in Europe will be comparing manual and the new basic automatic imaging. The PA Pipeline people will be performing the "deep dive" testing (e.g. terms:2 at B3, etc.).
      • For the 4.5 CASA release, it is anticipated that one or two CASA Guides will be produced that essentially outline "This is how you do it with the old clean, and here is the tclean equivalent."
      • It will be necessary to ensure that the clean regression tests are shown to work with tclean (modulo any appropriate tweaking).
      • Crystal now has access on lustre to a CASA 4.5 test/stable package containing tclean that she can run and start testing in earnest by no later than June 29th (the next imaging group meeting will take place on June 30th). The aim is to at least ensure that it works well enough for ~5 or 6 of the most common-use cases.
      • Parallelization is also expected to feature more prominently in CASA 4.5. It would be good if this could be relied upon to help speed up flagdata, although our most immediate concern is that it is not responsible for additional breakage (particularly if any such breakage makes it more difficult to test/debug any tclean issues).
        • For a limited group: HPC work (e.g. number of open files, etc.). The Pipeline group are also involved in testing this. This probably does not need a lot of additional dedicated science user testing yet, beyond routinely trying to use split2, day-to-day MSTransform-type work, etc. Amanda had been looking into testing routine split2, cvel2 and other MS_Transform-related issues.
    • Weight column (including weight column -> initialized weight spectrum). See CAS-4469. Testers to work with George. Steve was to look into this.

  • Any Other Business:
    • A discussion took place motivated by the late discovery of CAS-7699. Concern was expressed that a problem so close to common use cases was not detected by the current user testing practices. It was decided that a wiki page will be created for the NA Imaging Team to collect together a list of common use cases to be tested. Socorro staff will then review the list, and add any VLA-specific entries for which they also see a need. Juergen will provide a copy of his old plotms testing automation targets to use as a starting point.
    • Crystal also provided some background on the recent Clark vs. Hogbom clean discussions of the Imaging and Pipeline groups. Full details on request.
    • Mark R. really needs to get going on the tools documentation again soon. TThe documentation on the imager (in particular) is just plain wrong in places.

PART 1: Items to be Covered During Meetings (as time permits)

Propose to Close Items

  • TBA

Items Overdue for Status Updates (i.e. the "Nag List")

  • CAS-7488 Support single dish data transformations (time/chan avg) in plotms/mstransform. Was listed as a 4.4 Blocker Pushed back to 4.5. Currently Under Test. Passed to Erik Muller for NAOJ testing assingment. Ticket pinged.
  • CAS-7318 (Feb): 4.5 item (former 4.4 item). Verify unpolarized calibrator mode for linear basis (ALMA). Under Test. Tester: Steve. Ticket pinged. Steve and George to nag each other about this one. Ticket pinged repeatedly. Steve will nag George about this.
  • CAS-7109 (Mar 2015): Linear full mosaics. Available in stable now. Under Test. 4.5 item (former 4.4 item. Testing started, Input requested from developer. Tester: Steve. Kumar's working on this at the moment.
  • CAS-4983 (Mar 2015): frequencies in gain tables derived using combine='spw' are incorrect. 4.5 item (formerly 4.4). Available in stable now. Under Test. Tester: was Claire, now Crystal. Some duplication with CAS-7097 (assigned to Claire). Claire reports: "Have tested gaincal with solving for individual spws, looks OK, labels for delays now match complex gain, at least in plotcal plots (used to be edge frequency instead of center frequency). Still have to test with averaged spws." Crystal felt that Claire's testing so far was already almost sufficient. While the remaining testing should be done before the release, it is probably less urgent than other issues right now.
  • CAS-6385 (May 2015): simobserve listobs file has an incorrect intent? Under Test. Tester: Catarina. Ticket pinged for confirmation of final fix.
  • CAS-7522 (May 2015): channel conformance error in MS reported by viewer. Under Test. 4.5 item (was 4.4 item). Tester: Juergen. Ticket pinged repeatedly. Resolved as done.
  • CAS-7243 (June 2015): plotms crashes with combination of adding plot and closing GUI. 4.5 item (was 4.4 item). Reopened and Under Test (again). Tester: Juergen. Ticket pinged. Resolved as done.
  • CAS-7185 (June 2015): reset rest frequency in viewer to original value. 4.5 item (was 4.4 item). Reopened and Under Test (again). Tester: Juergen Resolved as done.
  • CAS-7435 (Jun 2015): mstransform sets all antenna indices to zero. 4.4 item. Tester: Josh M. Ticket pinged for final comment, to try to close it out as a 4.4 item.
  • CAS-1390 (Aug 2014): ImageRegrid required for Complex images. 4.4 item. Tester: Preshanth. Ticket pinged for final comment, to try to close it out as a 4.4 item. Preshanth's away on travel for a couple of weeks.

Other Items Currently "Under Test"

  • CAS-7149 (Jun 2015): mstransform(chanaverage=True) gets the weights wrong if spws differ in number of channels and chanbin is set to the maximum number of channels (instead of a vector). 4.4 item. Tester: Amanda Crystal will help Amanda identify a suitable dataset
  • CAS-7314 (Mar 2015): Add capability to average across fields. Officially Under Test already, but needs the VI/VB2 version of PlotMS to test, so this would be an advance assignment. Intended science use case for this: Different observations of the same field source might carry different field IDs. May also have mosaicking applications. 4.5 item (was 4.4). Tester: Steve. Ticket pinged. Steve will do this.
  • CAS-7227 (May 2015): better reporting of tabular spectral axes. 4.5 item. Under Test. Tester: Remy. Ticket pinged repeatedly.
  • CAS-7573 (May 2015): imval should work on pV images. 4.5 item. Under Test. Tester: Juergen. Ticket pinged. Resolved as done.
  • CAS-7485 (May 2015): Modify SubImageFactory interface. 4.5 item. Under Test. Tester: Erica. Looks OK so far. May be able to resolve?
  • CAS-6232 (Mar): writing version numbers in GUIs. Under Test in 4.5 (and 4.4 and 4.3). Most recent commit to trunk was r29373. Tester: Juergen. However, this also needs to be checked in the table browser, the work on which is currently unscheduled. Spawned another ticket CAS-7033) for its underlying cause (just pinged). The Browsetable help file is now in place, but the reported version number is still 4.3, and is a bit buried.
  • CAS-7560 (Jul 2015): Include Prolate Spheroidal in AW-Projection. Under Test in 4.5. Tester: Preshanth Preshanth's away on travel for a couple of weeks.
  • CAS-7642 (Jul 2015): Flagdata and RESIDUAL data column. Under Test in 4.5. Tester: Josh Marvil(?)
  • CAS-6421 (Jul 2015): modify casapy launching script to drop into the python interpreter. Under Test in 4.4 public release or later. Tester: Todd

"Ready To Test" Items Pending Tester Assignment

  • CAS-7248 (Jan 2015): two channel lines appear in spectral tool when using "channel positioning" button. 4.4/4.5 item. Under Test. Tester: DA (was Nate; ticket with Anand)
  • CAS-3263 (Jul 2015): Autoranging of y-axis in plotms fails to include points close to zero. 4.5 item. Under Test. Tester: DA (ticket with Anand)
  • CAS-7501 (Jul 2015): Allow users to specify callib filename in plotms GUI. 4.5 item. RTT Tester: TBD Drew
  • CAS-2370 (Dec 2014): plotms suggestions. 4.4/4.5 item. Under Test in public 4.4 release. Tester: DA (ticket currently with Anand)
  • CAS-7589 (Jun 2015): ImageFitter segfaults with empty estimates file. 4.5 item. Tester: DA (ticket currently with Anand)
  • CAS-7620 (Jun 2015): ImageProfileFitter segfaults for a small channel range. 4.5 item. Tester: DA (ticket currently with Anand)
  • CAS-7621 (Jun 2015): Residual image not correct when fitting a single change of an image cube with a 2D fit. 4.5 item. Tester: DA (ticket passed to Anand)
  • CAS-3953 (May 2015): Subtraction / Fixing of outlier fields in clean. Subtask of CAS-6622. Ready to test in tclean via interactive viewer. Tester: TBD. Ticket with Urvashi. VLA item. Steve to test.

  • tclean testing (to be handled by the Imaging Group)
    • CAS-5048 (May 2015): Clarify when peak residual calcs use/ignore masks, and use consistent thresholding rules. Older ticket, now ready to be tested using new tClean in 4.5. Tester: TBD (was Emmanuel, back in the 4.1 cycle). Ticket with Urvashi
    • CAS-3729 (May 2015): clean to write out only those images that are necessary for a given run. Subtask of CAS-4640. Tester: TBD. Ticket with Urvashi
    • CAS-3290 (May 2015): Logging : clean logging for cube cleaning needs improvement. Subtask of CAS-6615. Tester: TBD. Ticket with Urvashi
    • CAS-4133 (May 2015): clean inline docs for masks and new region format... Subtask of CAS-6617. Tester: TBD (Brian Kent, who has history with this ticket? Or DA?). Ticket with Urvashi
    • CAS-1623 (May 2015): CLEAN should have option to continue from previous CLEAN state. Subtask of CAS-4640. Tester: TBD. Ticket with Urvashi
    • CAS-2673 (May 2015): The 'cyclefactor' parameter in the clean task. Subtask of CAS-6622. Tester: TBD (Is this really user testable? Maybe not...). Ticket with Urvashi

Single Dish (to be skipped during the meeting)

  • See SD ticket list below

Input Required

  • CAS-5583: Frequency in exportuvfits output. Currently with Brian K., who is liaising with the user in question.
  • CAS-1608: importuvfits does not load data with muliple antenna tables properly. Subtask of CAS-7087. Sample dataset requested by developer.
  • CAS-4469 (Jun+ 2013): Split with channel averaging is creating spurious spectral features and weight is wrong when there has been channel flagging. Subtask of CAS-5245. 4.5 item. Critical item. was "Under Test". Tester: Crystal. Some stuff works. Ticket back with developer now. Extensive comments provided by George, now "Input Required" from Crystal.
  • CAS-5806 Mathematical prediction of post-gridding single dish beam. Formerly a 4.3+ Blocker, now "Critical". "Input Required?* Developer has done some modelling, but... ? This is a "blocker of a different kind..." High priority (blocking further development). Currently Input Required from Kana. Crystal and Kumar had been in discussion, but this ticket has been quiet since (effectively) December. 4.5+ item now.
  • CAS-4735 (Nov): imsmooth needs additional option for single dish use. Subtask of CAS-5844 (above). Input Required again. Tester: Tsuyoshi. Critical item. Code committed to trunk. 4.3 item. Some testing done, mostly works as intended. Input Required from Crystal for some remaining changes. Any remaining future work will have to be for 4.5(+).
  • CAS-6124: imfit subtasks:
    • CAS-4538: migrate imfit input estimates from file to input parameters. Input Required (was "On Hold"). Subtask of CAS-6124. Ticket passed to Juergen, pending a CSSC discussion of what needs to be done and a scheduling of the work. Pushed back to 4.5. Ticket pinged. Juergen: More thought about imfit still needed by Science.
  • CAS-4748 (Dec 2012): Polarization selection does not select SIGMA. Input Required. Passed to Jeff for prioritization. Sanjay thinks it's a low-priority item. Version 4.5+.
  • CAS-5886 (Dec 2013): bizarre clean boxes for large images. Awaiting input from one developer to another. Version 4.5+.
  • CAS-2825 Support for Lissajous mapping. Pushed back to 4.5+. Input from Tsuyoshi needed, but he is not able to provide it yet.
  • CAS-5162 (Sep 2013): Improvements in flagging documentation and examples. Input Required from Science (via Jeff). This ticket has been left untouched since May 2013... Version 4.5+.
  • CAS-6794 clean is very slow handling this simple, small ACA mosaic. Input Required from Juergen. Kumar commented on this.

Other Blockers

  • CAS-6688 Turn on new vi/vb within imager and test. Was a 4.4 item, but now Unscheduled. Currently with Jim. Some scope modification suggested for this ticket.
  • Viewer registration tickets:
    • CAS-6560 contour image does not unzoom when master coordinate image is re-registered. Subtask of CAS-6604. Viewer issue. Currently Unscheduled.
    • CAS-6720 2 panels don't register. Viewer issue. Currently Unscheduled.

Additional work-in-progress 4.5 Blocker items to be added in the future as appropriate.


PART 2: Items Listed for Informational Purposes (note that these will not normally be specifically discussed during the meeting unless someone raises them)

Unable to Test Items

  • CAS-4758 (Sep): 4.4 item. fluxscale fails if one of the reference fields is flagged. Previously "Under Test" (r31121+) in current (new-style) stable. Tester: Claire. No testing progress on this since October. Claire had been trying to find a suitable dataset for testing. Steve thought that this should actually be fairly easy to test and that Claire and/or he may have been able to test this one. The previous suggestion was to ask Drew M. about this, and (if appropriate) reassign it to Brian Kent. Ticket pinged. Place On Hold, pending test data availability. No recent updates.
  • CAS-5379 - Subtle inconsistencies with other programs in Galactic Coordinates (Adam and Harvey). In a nutshell, shifts of ~0.2" are seen when taking data into other packages (e.g. DS9, IDL which both seem to agree with each other). It is unclear yet if this is an incorrect implementation in one or more packages, or possibly just a precision error. This is a subtle issue, and was pushed back. Ticket stalled. Needs to be looked at by someone with lots of time. To be left as-is for now. This needs significant work, including new science papers on how to actually define the Galactic coordinate system? Needs casacore discussion; this can be part of that. Passed to Jeff for now. Pushed back to some indefinite future release. Unscheduled.
  • CAS-4636 (Oct 2012): Elevation dependent opacity application w/ Tsys. Still awaiting input from Science. Had been "On Hold" since August, awaiting data. Previously switched to "Input Required" and passed to Crystal for comment. Bumped back to 4.4+. Work on this is Unscheduled until further notice (more than a year, now).
  • CAS-3532 (Sep): Incorporation of Alma Primary Beam in Mosaic. Currently Unscheduled. Tester was Scott S., but ticket passed to Crystal for input. Some test imaging done. Ticket with developer.

Upcoming Ready to (Re)Test Items

Not Yet Assigned To Testers

  • TBA

Single Dish

See below "assigned" SD list.

Already Assigned To Testers

  • CAS-7531 (May 2015): poisson fitter for 2d histogram does not appear to be working. Was "Ready To Test". Appears to mostly work; couple of questions raised, so passed back to developer. Tester: Amanda
  • CAS-4592: plotms on 10.7 issues. Previously half fixed due to Qt updates. Shown to till be an issue under 10.10. Currently Unscheduled. Tester: was Nathan, to be Juergen. Exported plots look OK, only the plots in the GUI appear corrupted.
  • CAS-6951 (Oct): 4.4 item now. mstransform creates a 4-polarization DATA column for single-polarization ALMA WVR data, and a 1-polarization DATA column for four-polarization ALMA data. Fix already included in current new-style test/stable packages newer than 4.2.23. Scheduled again now. Tester: Todd. Ticket currently back with developer(s) again.
  • CAS-5893 (May): Now a 4.5 item. Add the ability to write a spectral profile to an ASCII file in the ia tool. Was "Under Test" in 4.3. Now Scheduled. Tester: Assigned to Juergen for testing and a possible CSSC discussion. Might also need a task adding for this, but this will be 4.5 work. Recent testing done, passed back to developer for 4.5 work.
  • CAS-5127 (May): Document ia toolkit methods. Status Scheduled. Mark should start working on this again... Most of Juergen's prioritized list of 26 items had been done. Dates to be added his future documentation updates. Ongoing. ia.regrid docs will require more work. Still needs code commits. Mark to restart work on this under Linux soon.

Single Dish

  • CAS-6018 (Jun 2014): The bdflags2MS task does not work with single dish data. Provisionally passed to Erik M. Under Test in 4.4 public release.
  • CAS-7598: (Jul 2015): beam size definition fails for data without REST_FREQUENCY. 4.5 item. Ready To Test. Ticket currently with Erik for reassignment/testing.
  • CAS-6572: (Aug 2014): A new baseline task that handles baseline table. Was Under Test in 4.4. Mostly done. Ticket with Kana. Some duplication with CAS-5855). Ticket pinged (resolve or move to 4.5?).

-- MarkRawlings - 2015-07-06
Topic revision: r6 - 2015-07-07, MarkRawlings
This site is powered by FoswikiCopyright © by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding NRAO Public Wiki? Send feedback