CASA User Testing Tracking - 2015-05-26 - NOTES ONLY

NOTES ONLY. NO MEETING DUE TO MARK R. BEING AWAY ON TRAVEL.

Notes

Status

  • 4.4 feature freeze officially passed. Test/stable/monthly packages are being produced for RHEL5 and 6.
  • OS X test packages production ongoing (for 10.10 and 10.9) *Pre-release packages currently being used for user testing. OS X pre-release packages now available at: https://svn.cv.nrao.edu/casa/osx_distro/prerelease/10.10/
  • 4.3.1 is the current public release version for both Linux (RHEL5 and 6) and OS X (10.8). 4.2.2 is still currently being used for pipelined ALMA data reduction at the NAASC, but 4.3 has now been adopted for "manual calibration" reductions.
    • The same general-purpose CASA launcher script has now been deployed at CV, SOC and GB. Details are on the appropriate drop-down menu of the public CASA pages at http://casa.nrao.edu
      • For anyone using test machines zuul01, 03 and 04 are running RHEL6, zuul02 is running RHEL5. REMEMBER: Use /usr/bin/casa on these machines to make sure that you are running the new test/stable packages (there are RPM installations on these).

Number of CASA Jira tickets with "Ready To Test"/"Under Test" status (including EA, EU, JAO, Software-only tickets, but excluding Pipeline) prior to review: 70.

Number of CASA Jira tickets with "Ready To Test"/"Under Test" status (including EA, EU, JAO, Software-only tickets) following review: 57.

Testing Tracking

  • 4.4 feature freeze deadline (March 1st) has passed. No new features can be added to 4.4 from now on. 4.4 pre-release branch to be announced soon, user testing to be conducted on the 4.4 stable packages for now (Linux only for now, OS X still to come, but testing can be done using OS X test packages).
  • New ticket assignments.
  • The "nag list", "input required" items
  • Major test coverage areas for 4.4 pre-release testing
    • Reworked statistics framework - check for changes in behavior. Peter Teuben (plus?) See CAS-7034 Mark R. has repeatedly hassled Peter about the testing of this. Since he led the group that requested this, they should probably also set requirements criteria for associated regression testing as well.
    • PlotMS - Designated tester: Drew Brisbin
      • New PlotMS code implementing VI/VB2 and on-the-fly calibration. Related tickets (with revision 32835+) are: CAS-3034, CAS-4864, CAS-7117, CAS-7118, CAS-7119, (CAS-7386 and CAS-7393) (see below)
      • Besides testing the new functionality, including on-the-fly calibration and plotting sigma and sigma spectrum, please put PlotMS through its paces and make sure none of the old functionality was broken as a result of the underlying code change.
      • Check averaging and flagging. The Pipeline group will also be looking for any such issues.
      • Remaining related issues for release 4.4 included the creation of virtual model data when there is no MODEL_DATA column or SOURCE_MODEL column (CAS-7390), and improving the performance of the VI/VB2 iteration through the data when averaging (CAS-7412).
      • Also, regarding testing the OTF calibration: A callib parameter has been added to the plotms task in order to use this feature, e.g. plotms("ngc5921.ms", callib="callib.txt", ydatacolumn="corrected") where callib can be a filename or a python dictionary. PlotMS now has a Calibration tab where the OTF calibration can be enabled/disabled, but callib can only be set through the casapy interface for this release.
      • Any other issues should be reported.
    • Viewer: Amanda has been doing user testing of this recently.
    • Weight column (including weight column -> initialized weight spectrum). See CAS-4469. Testers to work with George. Steve was to look into this.
    • New ATM library in 4.4 (CAS-7220). Should be (more or less) a drop-in replacement: testers just need to watch for any oddities during routine use.
    • For a limited group: HPC work (e.g. number of open files, etc.). Pipeline group are also involved in testing this. Probably does not need a lot of dedicated science user testing yet, beyond routinely trying to use split2, day-to-day MSTransform-type work, etc. Amanda had been looking into testing routine split2, cvel2 and other MS_Transform-related issues.
  • Any Other Business:
    • Crystal had previously reported that she had run into severe performance problems with the VIewer under 4.3.1 on her desktop machine: she only had two normal, normal-sized cubes open, but found the Viewer was taking several minutes to redisplay. In addition, the CPU usage was extremely high. It is not clear if this behavior has been seen to be a problem elsewhere yet. The problem does not appear to be present in the current 4.4 stable packages, but should be watched for in the 4.4 pre-release packages.

PART 1: Items to be Covered During Meetings (as time permits)

Propose to Close Items

  • TBA

Items Overdue for Status Updates (i.e. the "Nag List")

  • CAS-7550 (May 2015): plotms interator bug? 4.4 Blocker item.. Under Test. Tester: Brian Kent
  • CAS-7554 (May 2015): plotms frequency plotting shift. 4.4 Blocker item.. Under Test. Tester: Brian Kent
  • CAS-7532 (May 2015): 2-d source fitter only appears to fit one source. 4.4 Blocker item.. Under Test. Tester: Amanda
  • CAS-7515 (May 2015): Test setjy source model flux scaling fixes for 4.4. release. 4.4 Blocker item.. Under Test. Tester: Juergen
  • CAS-7318 (Feb): 4.4 item. Verify unpolarized calibrator mode for linear basis (ALMA). Under Test. Tester: Steve. Ticket pinged. Steve and George to nag each other about this one. Ticket pinged repeatedly.
  • CAS-4469 (Jun+ 2013): Split with channel averaging is creating spurious spectral features and weight is wrong when there has been channel flagging. Subtask of CAS-5245. Critical item. "Under Test". Tester: Crystal. Some stuff works. Ticket back with developer now. Ticket pinged for an update.
  • CAS-3580 (Aug): 4.4 item. Prediction of expected image based noise based on Tsys. Under Test in 4.3. Tester: Eric V. (to be tested at the JAO). Kept open at Remy's express request. Recent testing continuing by Eric. Tracked elsewhere(?)
  • CAS-4758 (Sep): 4.4 item. fluxscale fails if one of the reference fields is flagged. Under Test (r31121+) in current (new-style) stable. Tester: Claire. No testing progress on this since October. Claire had been trying to find a suitable dataset for testing. Steve thought that this should actually be fairly easy to testand that Claire and/or he may have been able to test this one. The previous suggestion was to ask Drew M. about this, and (if appropriate) reassign it to Brian Kent. Ticket pinged.
  • CAS-6707 (Sep): mstransform problems with CALDEVICE and SYSPOWER sub-tables when creating an MMS. Under Test in 4.3, but 4.4 item now. Tester: Previously passed to Steve for VLA user tester assignment. Previously discussed at length: Feature was already shipped in 4.3. To resolve or not to resolve (that is the question)? Steve had talked to James Robnett to get a sense of the status of parallelization. May be more of a developer/Pipeline testing item? Ticket pinged.
  • CAS-7034 (Mar 2015): Development of new Common Statistics Framework. Under Test now in current stable package. Tester: Peter Teuben. %Sounds like it largely works so far, but a possible issue has been identified with half-fit during full cube access.
  • CAS-7364 (Feb 2015): wvrgcal has problems with purely numerical field names. 4.4 item. Under Test. Tester: Thomas Stanke. Ticket pinged repeatedly.
  • CAS-7207 (Mar 2015): Flux density not working for multiple beam one per channel image. Available in stable now. Under Test. Tester: was Juergen, now Emmanuel. Mostly works, but Emmanuel reported a failure mode. Fix in, assigned to him for testing.
  • CAS-7109 (Mar 2015): Linear full mosaics. Available in stable now. Under Test. Tester: Steve. Ticket pinged.
  • CAS-4983 (Mar 2015): frequencies in gain tables derived using combine='spw' are incorrect. 4.4 item. Available in stable now. Under Test. Tester: was Claire, not Crystal. Some duplication with CAS-7097 (assigned to Claire). Ticket pinged.
  • CAS-6385 (May 2015): simobserve listobs file has an incorrect intent? Under Test. Tester: Catarina. Ticket pinged.

Other Items Currently "Under Test"

  • PlotMS Items Drew Brisbin is the science user tester assigned to work closely with Pam on this during the next couple of weeks or so..
    • CAS-3034 (Mar 2015): Ability to apply calibration on-the-fly. PlotMS testing item. Currently RTT. Available in current stable package. Tester: Drew B. Looks to work. Minor correction recommended for the QR guide, but should then be resolved.
  • CAS-6882 (Mar 2015): imstat sums over multiple planes in an image cube to produce a "flux". RTT, available in current stable. 4.4 item. Ongoing. Tester: Catarina. Mostly done? Developer pinged for info.
  • CAS-7268 (Mar 2015): importuvfits creates inconsistent MS. Under Test. Tester: was Nate, passed to Anand for DA re-assignment. Fix should be available now.
  • CAS-7314 (Mar 2015): Add capability to average across fields. Officially Under Test already, but needs the VI/VB2 version of PlotMS to test, so this would be an advance assignment. Question raised during previous meeting: What is the intended science use case for this? Answer: Different observations of the same field source might carry different field IDs. May also have mosaicking applications. Tester: Steve Ticket pinged.
  • CAS-4283: implement ia.continuumsub()/imcontsub using ImageProfileFitter. RTT. Available in the current stable package now. Tester: Catarina. Help recently requested from developer for testing.
  • CAS-7473: Fix raw pointer issue in LatticeApply and StatsTiledCollapser. RTT. Available in the current stable package now. Tester: Erica. Testing ongoing.
  • CAS-7522 (May 2015): channel conformance error in MS reported by viewer. Under Test. Tester: Juergen
  • CAS-7499 (May 2015): ia tool used in imstat online help examples. Under Test. Tester: Ashley (or other DA?)
  • CAS-7482 (April 2015): Plotms cannot create Az-El plot for MS that doesn't have DATA column. Under Test. Close to done? Tester: Takeshi
  • CAS-7517 (May 2015): plotms plots WTSP even when that column is not initialized. Under Test. Tester: Crystal(?)
  • CAS-7227 (May 2015): better reporting of tabular spectral axes. 4.5 item. Under Test. Tester: Remy.
  • CAS-7544 (May 2015): importfits fails for double precision data when defaultaxes=True. 4.5 item. Under Test. Tester: Brian M.
  • CAS-7573 (May 2015): imval should work on pV images. 4.5 item. Under Test. Tester: Juergen.
  • CAS-7351 (May 2015): poisson fitter for 2d histogram does not appear to be working. Ready To Test, pending a new 4.5 package. Tester: Amanda

"Ready To Test" Items Pending Tester Assignment

  • CAS-7485 (May 2015): Modify SubImageFactory interface. Under Test. Passed to Anand for DA user testing assignment.
  • CAS-7478 (April 2015): uvcontsub seg faults. Under Test. Tester: Was Nathan, passed to Anand for DA user testing reassignment.
  • CAS-7578 (May 2015): plotbandpass(overlay='time', showfdm=True, showatm=True) shows multiple copies of the atmospheric curve and FDM labels when the next antenna's solutions are completely flagged. Under Test. Tester: DA (TBD)
  • CAS-5329: IMMATH cannot parse strings with "_" on Mac OS. This should (in practice) just require a quick check of some documentation updates. Under Test. Passed to Anand for DA user testing assignment.

Single Dish (to be skipped during the meeting)

  • See SD ticket list below

Input Required

  • CAS-5806 Mathematical prediction of post-gridding single dish beam. Formerly a 4.3+ Blocker, now "Critical". "Input Required?* Developer has done some modelling, but... ? This is a "blocker of a different kind..." High priority (blocking further development). Currently Input Required from Kana. Crystal and Kumar had been in discussion, but this ticket has been quiet since (effectively) December. Post-4.4 item now.
  • CAS-4735 (Nov): imsmooth needs additional option for single dish use. Subtask of CAS-5844 (above). Open again. Tester: Tsuyoshi. Critical item. Code committed to trunk. 4.3 item. Some testing done, mostly works as intended. Input Required from Crystal for some remaining changes. Any remaining future work will have to be for post-4.4.
  • CAS-6124: imfit subtasks:
    • CAS-4538: migrate imfit input estimates from file to input parameters. Input Required (was "On Hold"). Subtask of CAS-6124. Ticket passed to Juergen, pending a CSSC discussion of what needs to be done and a scheduling of the work. Pushed back to 4.5. Ticket pinged. Juergen: More thought about imfit still needed by Science.
  • CAS-4748 (Dec 2012): Polarization selection does not select SIGMA. Input Required. Passed to Jeff for prioritization. Sanjay thinks it's a low-priority item. Version 4.5+.
  • CAS-5886 (Dec 2013): bizarre clean boxes for large images. Awaiting input from one developer to another. Version 4.5+.
  • CAS-2825 Support for Lissajous mapping. Pushed back to 4.5+. Input from Tsuyoshi requested. Discussion was ongoing, as of March.
  • CAS-5162 (Sep 2013): Improvements in flagging documentation and examples. Input Required from Science (via Jeff). This ticket has been left untouched since May 2013... Version 4.5+.
  • CAS-6794 clean is very slow handling this simple, small ACA mosaic. Input Required from Juergen.

Other Blockers

  • CAS-7488 Cube error when trying to plot averaged float data. New item. Currently Scheduled. Currently being worked on.
  • CAS-7460 plotms frequencies wrong when chan. averaging. Subtask of CAS-4864. New item. Fix already committed. Currently Under Test by developer. Close to done?
  • CAS-7491 plotms frequencies wrong when time avg interval is entire scan. Subtask of CAS-4864. Ready To Test (by developers. Currently blocked by casacore mods commits.
    • CAS-6688 Turn on new vi/vb within imager and test. Was a 4.4 item, but Unscheduled. Currently with Jim. Some scope modification suggested for this ticket.
  • Viewer registration tickets:
    • CAS-6560 contour image does not unzoom when master coordinate image is re-registered. Subtask of CAS-6604. Viewer issue. Currently Unscheduled.
    • CAS-6720 2 panels don't register. Viewer issue. Currently Open.


PART 2: Items Listed for Informational Purposes (note that these will not normally be specifically discussed during the meeting unless someone raises them)

Unable to Test Items

  • CAS-6232 (Mar): writing version numbers in GUIs. Under Test in 4.3. Most recent commit to trunk was r29373. Tester: Juergen. However, this also needs to be checked in the table browser, the work on which is currently unscheduled. Spawned another ticket CAS-7033) for its underlying cause (just pinged).
  • CAS-5379 - Subtle inconsistencies with other programs in Galactic Coordinates (Adam and Harvey). In a nutshell, shifts of ~0.2" are seen when taking data into other packages (e.g. DS9, IDL which both seem to agree with each other). It is unclear yet if this is an incorrect implementation in one or more packages, or possibly just a precision error. This is a subtle issue, and was pushed back. Ticket stalled. Needs to be looked at by someone with lots of time. To be left as-is for now. This needs significant work, including new science papers on how to actually define the Galactic coordinate system? Needs casacore discussion; this can be part of that. Passed to Jeff for now. Pushed back to some indefinite future release. Unscheduled.
  • CAS-4636 (Oct 2012): Elevation dependent opacity application w/ Tsys. Still awaiting input from Science. Had been "On Hold" since August, awaiting data. Previously switched to "Input Required" and passed to Crystal for comment. Bumped back to 4.4+. Work on this is Unscheduled until further notice (more than a year, now). * CAS-6243: sdimaging does not accept type numpy.string_ for fieldid. Unscheduled, pending identification of a suitable example case. Commit was r29988. Discussion ongoing. Pushed back to 4.4(+). Tester: Todd. Todd hasn't seen this happening, but hasn't specifically tested for it either. Maybe think about resolving if there's nothing else reported by the 4.4 public release?
  • CAS-3532 (Sep): Incorporation of Alma Primary Beam in Mosaic. Currently Unscheduled. Tester was Scott S., but ticket passed to Crystal for input. Some test imaging done. Ticket with developer. Pushed back to 4.5.

Upcoming Ready to (Re)Test Items

Not Yet Assigned To Testers

  • TBA

Single Dish

See below "assigned" SD list.

Already Assigned To Testers

  • CAS-4592: plotms on 10.7 issues. Previously half fixed due to Qt updates. Was "ARTT" in 4.2.1 and 4.2.2 pre-release. Was "Under Test" , shown to till be an issue under 10.10. Tester: was Nathan, to be Juergen. Exported plots look OK, only the plots in the GUI appear corrupted.
  • CAS-6951 (Oct): 4.4 item now. mstransform creates a 4-polarization DATA column for single-polarization ALMA WVR data, and a 1-polarization DATA column for four-polarization ALMA data. Fix already included in current new-style test/stable packages newer than 4.2.23. Scheduled again now. Tester: Todd. Ticket currently back with developer(s) again.
  • CAS-5893 (May): Now a 4.5 item. Add the ability to write a spectral profile to an ASCII file in the ia tool. Was "Under Test" in 4.3. Now Scheduled. Tester: Assigned to Juergen for testing and a possible CSSC discussion. Might also need a task adding for this, but this will be 4.5 work. Recent testing done, passed back to developer for 4.5 work.
  • CAS-5127 (May): Document ia toolkit methods. Status Scheduled. Mark should start working on this again... Most of Juergen's prioritized list of 26 items had been done. Dates to be added his future documentation updates. Ongoing. ia.regrid docs will require more work. Still needs code commits. Mark to restart work on this under Linux soon.

Single Dish

  • CAS-7446: (May 2015): Provide a way to obtain pointing directions of an antenna corresponding to MS rows with data selection and transformations. Subtask of CAS-6988. Ready To Test. Ticket passed to Erik for testing reassignment.
  • CAS-6572: (Aug 2014): A new baseline task that handles baseline table. Under Test in latest 4.3 casapy-test/new-style stable packages. Mostly done. Ticket with Kana. Ticket pinged.

-- MarkRawlings - 2015-05-25

This topic: Software > WebHome > LegacyCASA > CASAUserTestingMain > CASAUserTestingMinutes20150526
Topic revision: 2015-05-26, MarkRawlings
This site is powered by FoswikiCopyright © by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding NRAO Public Wiki? Send feedback