CASA Build and Test Group Meeting

DIAL-IN NUMBERS & PASSCODES:

USA Toll Free Number: 866-901-8266

USA Toll Number: +1-203-566-3863

Participants

Mark Rawlings, Darrell Schiebel, Shinnosuke Kawakami, Andy Hale

Post-meeting edits and additions are in blue text.

Agenda

  • Status of current CASA packages
    • Redefinition of "test" and "stable" currently being implemented (see also below).
    • Release - 4.2.2 out for OS X (10.8) and RHEL 5 and 6. 4.2.2 release has been successfully installed in Japan.
  • Staffing
    • Still receiving applications for B&T engineer. Currently in a "holding pattern": consideration of hiring a consultant short-term
    • Interviewing ongoing for Testing Automation position (Andy Hale's group). Mark and Darrell are on the hiring committee for this position as well. More phone interviewing this week? One on-site interview next week. One of the more promising candidates had taken a different position in Socorro. Phone interviews being planned for the following week.
    • Quite a lot of internal changes to the CASA development team, including Rob Selina's upcoming departure.
  • Build and Test Review Status
    • Requirements spreadsheet draft circulated to CASA team last week.
    • Review timeline effectively on hold, pending above decisions regarding B&T hiring strategy.
  • Casacore unification - Planned to occur after the 4.3 release. For reference, Jim's proposed plan was as follows:
    • 0.5. Attempt to build against the GC casacorem and identify problems. DONE. Shinnosuke has effectively completed this task.
    • 0.75. Iteratively fix bugs and attempt to build again. This is Shinnosuke's B&T primary assignment for the short-term. See also below.
    • 1. Merge our Casacore into GC casacore. This will require us to flush all casacore mods in and then let Ger to the final merge (finally).
    • 2. Build CASA against GC casacore (platforms?)
    • 3. Test CASA/GC casacore against our various regression tests, etc.
    • 4. Create a Jenkins job that will handle B+T of codebase using GC casacore.
    • 5. Keep a cached, readonly copy of GC casacore locally(?) to prevent a glitch at googlecode from allowing us to build. Also we should maintain periodic a backup copies.
    • 6. Go live and live happily ever after. wink
  • B&T news from Charlottesville.
    • Darrell has been working on increasing automation of the generation of the CASA test packages. Test packages can now be built daily. They are then regression tested, and sufficiently-successful packages (from a testing perspective) are promoted to be "stable" packages. This system also has the side-benefit of providing Socorro developers with nighty builds. The system now also incorporates a unified launch script for CASA, which can also report (RPM) CASA versions available to the user. It was noted that the nightly "test" builds are not currently coupled to Jenkins. There are pros and cons to hooking it into Jenkins: A "one-stop shop" for controlling all the builds is attractive. It might help with distributed testing runs, and handover of the package building between staff. It might also provide a single point of failure, and could limit our options for having (say) Europe maintain the regular package production for Fedora, Ubuntu, etc. This issue can probably be discussed in more detail a little further into the future, perhaps following the hiring of a new Build and Test engineer. Darrell also noted that the new system produces one build per day (for each supported platform: OS X, RHEL5, RHEL6), whereas the current Jenkins system provides a more rapid response to individual code commits.
    • There was some discussion of at which stage automated testing should be done before a public release. Socorro had historically primarily favored testing on the code tree, but there had been some concern expressed that breakage might occur during the binary packaging, and so ideally, tests should perhaps also be run on the user-facing finished packages as well. Andy thought that this could be something that his new hire might be able to work on with the B&T group. The specifics of the division of work can be discussed at some point in the future. Darrell was not opposed in principle to the possibility of using Jenkins to manage the automated testing (even of the binaries).
    • Next step of the plan is to set up a second "slow" RPM repository that will serve the most recent test package, and a version of the stable package that is not updated more frequently than weekly (updates proposed to take place on Sunday evening in Charlottesville, Monday morning in Tokyo). This will be for the benefit of developers running jobs that take > 24 hours to complete. tarball packages will just accumulate as previously, so anyone wishing to stick with a particular private build for longer still has this option. This strategy has since been slightly revised. The plan will instead be to have "test", "stable" and "monthly" packages. Mark will write up a full description of these.
    • Deployment of all of this at NRAO is now primarily stalled at this point by the version numbering discussion. A version numbering convention has now been settled upon. Mark will be writing up instructions for developers, user testers and end users on the new casa launch script and packaging conventions in the next few days. This information will be communicated to the appropriate software deployment staff in Socorro, Japan, Europe and Chile. in terms of the practicalities of the deployment, Charlottesville should be easy enough (it already works on a number of individual systems there. Socorro would be more difficult, due to their preference for tarballs over RPMs. Darrell suggested that it might make sense to roll out there for the first pre-release package.Japan currently uses a system that auto-installs from a tarball to an NFS drive via a shell script. All but one (running RHEL5) of the Linux machines in Japan are now running RHEL6. Jeff and Kana will be having a meeting on Friday to discuss several issues, including this one.
  • B&T news from EA.
    • Many of the CASA 4.3 Single Dish jobs have been done. Kana is keen for them to commence testing. There is some keenness to get going with the 4.3 release branch for this.
    • Shinnosuke has worked on removing the "asynch = T" option from the tasking system (see CAS-6461). This is more-or-less done, and could be tested by appropriate users (developers?) under RHEL 6.
      • tget command doesn't work properly. That is, it can't get last parameter from taskname.last file. This will be Shinnosuke's higher priority task.
    • Shinnosuke has been working on the casacore unification issue (also see above). The work for this is tracked on Jira ticket CAS-6929. He has been able to build GC-casacore, (latest version 1.7.0) on RHEL6 successfully. However GC-casacore + code (with casa-data/trunk) had failed with many errors at the cmake stage. Version checking had caused issues, but Jim had some ideas about this. There were, however, currently many additional errors that Shinnosuke was going to look into.
      • there are differences under the GC-casacore/casa/IO , Logging, BasicSL, Exceptions,System against NRAO-casacore, investigation(changing and re-build) is now in progress. Ger has since been helping Shinnosuke with this. Darrell suggested that Shinnosuke might want to keep the changes he puts in on a side-branch, to make it easier to keep them isoalted and readily discussed with Ger.
  • Next meeting: October 21st, 2014
  • Any Other Business
    • None

Action Items Arising

  • Mark to send Bunyo the URLs for the new-style tarballs (they can choose whether they prefer to use the RHEl5 or RHEL6 builds).
  • Darrell to provide help to Shinnosuke on creating a development branch to capture code changes that he implements for the casacore reunification effort.

-- MarkRawlings - 2014-10-06
Topic revision: r4 - 2014-10-09, MarkRawlings
This site is powered by FoswikiCopyright © by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding NRAO Public Wiki? Send feedback