CASA Build Automation

Desired Build Automation

This section describes an idealized automated build and test system for CASA that will

  • give developers quicker feedback on code changes,
  • give NRAO CASA users more consistent updates to try out, and
  • give everyone better visibility into the state of CASA development.

Terms used in this page

These should be defined elsewhere. Temporarily including working, fluffy definitions here to get discussion started.

Subsystem
one of "casacore", "code", or "asap". This is code either developed entirely by CASA developers, or developed with an outside organization.
Module
a related chunk of code within a subsystem.
Unit
a related chunk of code within a module.

Unit test
tests exercising a small set of code within one module. Developers are expected to run unit tests before checking in changes.
Module tests
tests exercising a module in isolation from other modules (do these tests exist in CASA?).
Inter-module tests
tests exercising several modules together, but not a full subsystem (do these exist in CASA?).
Subsystem tests
tests exercising a large fraction of a subsystem, but not other sub-systems (do these exist in CASA?).
End-to-end tests
tests exercising a large fraction of CASA, crossing subsystem or module boundaries. These tests often start with reading data in a telescope specific format, and end with some reduced data.

NRAO OS list
OSs used by most CASA users and developers an NRAO. This includes
  • OS X 10.6 64 bit,
  • RHEL 5.5 64 bit, and
  • RHEL 5.5 32 bit.
Full OS list
all OSs supported by CASA development. This includes the NRAO OS list, plus
  • Fedora Core 14 64 bit,
  • OS X 10.5 32 bit, and
  • Ubuntu 10.10 64 bit.

General notes

In each build or test step, notify developers who checked in changes of failures as soon as they are detected. If no failures are detected, notify developers who checked in changes of success at end of tests.

Active branch

On check in to the Active branch, for the NRAO OS list:

  • Build incrementally (to report build results to developers quickly).
  • On successful build, run all module tests for changed subsystems. Start with tests in changed modules, then run tests on unchanged modules.
  • On failed module test, run unit tests in failed module to get some hint of source of failure.

Once a day, for the NRAO OS list:

  • Build clean end-to-end build for all subsystems.
  • On successful build, run all module and end-to-end tests for all subsystems.
  • On successful test, tag changes as ready for test branch and distribute this build within NRAO (do we distribute this outside of NRAO?).
  • On failed module test, run unit tests in failed module to get some hint of source of failure.

Test branch

Every Tuesday and Thursday, for the NRAO OS list (manual step?):

  • Merge latest code tagged ready for test to Test branch.
  • Build incremental build for all subsystems.
  • On successful build, run end-to-end tests.
  • On successful test, commit changes to Test branch, tag as ready for stable.

Once a day, for the NRAO OS list:

  • Build clean end-to-end build for all subsystems.
  • On successful build, run all end-to-end, module and unit tests for all subsystems.
  • On successful test, distribute this build within NRAO (do we distribute this outside of NRAO?).
  • On failed test, get developer to provide fix as soon as possible for Test and Active branches.

Stable branches

Once every two weeks (manual step?):

  • Chose the latest Test branch build with minimal test failures. Ideally, none.
  • Create a new Stable branch based on this build.

Once a day, for Full OS list:

  • Build clean end-to-end build for all subsystems (possibly build on a subset of OSs, TBD).
  • On successful build, run all end-to-end, module, and unit tests for all subsystems.
  • On successful test,
    • distribute this build within NRAO
    • create distribution packages for release outside of NRAO
    • install distribution packages as they would be installed by an end-user
    • run post-installation tests included in distribution packages.
  • On failed test, get developer to provide fix as soon as possible for Stable and Active branches. fixes will be merged into Test as part of routine merges.

To Do to make this work

  • Identify computers to run builds/tests
  • Identify or create hooks in change control system to determine on demand:
    • which subsystems changed between two revisions
    • which modules changed between two revisions
  • Identify or create hooks in build system to run each on demand:
    • a subsystem build
    • a module build
    • end to end build
    • all unit tests
    • all module tests
    • all inter-module tests
    • all subsystem tests
    • CASA end to end tests
    • a subsystem's unit tests
    • a subsystem's module tests
    • a subsystem's inter-module tests
    • a subsystem's tests
    • module unit tests
    • module tests

Existing Build Automation

This section documents existing (as of 2011-01-26) build automation supporting CASA development and test. This is all I have found. There is probably more.

Chancellorsville

Socorro

Computers

  • ballista
    • aips2adm - all jobs commented out
    • aips2mgr - update and build i686 CASA Active, run some tests, report daily SVN changes.
  • dave
    • aips2adm - update IERS?
    • aips2mgr - nothing
  • tarzan - does this exist?
  • rishi
    • aips2adm - nothing
    • aips2mgr - update and build x86_64 CASA Active
Jobs

  • Update IERS data
    • aips2adm@dave
  • Build and install CASA Active
    • aips2mgr@ballista
    • aips2mgr@rishi
  • Build and install CASA Test
  • Build and install CASA Stable

-- ScottRankin - 2011-01-26

This topic: Main > TWikiUsers > ScottRankin > ScottRankinCasaBuildAutomation
Topic revision: 2011-02-23, ScottRankin
This site is powered by FoswikiCopyright © by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding NRAO Public Wiki? Send feedback