CASA Continuous Build and Test Server

Status: OBSOLETE - see Build A Public CASA Continuous Build Test Server

Goals

  • Improve CASA developer access to continuous build and test (CBT) results.
  • Reduce the time between committing a change and receiving test results.
  • Improve cross platform test coverage.
  • Improve total test coverage.

Requirements

  1. Security
    1. Build and test results must be readable by any Internet user.
    2. Build and test configuration changes must be restricted to CASA developers.
      • Build and test configuration changes may be restricted to specific CASA developers logged in from specific NRAO sites.
    3. Build products may only be readable by CASA developers.
      • This requires more thought about requirements. To be implemented later.
      • It may be possible to create re-distributable build products within the continuous build and test system. If so, these build products must not be allowed to leak out where they may waste users time or damage their data.
    4. Build and test jobs must not run on the computer running continuous build and test server (No builds or tests on the web server).
      1. The continuous build and test server must have ssh access to computers running build and test jobs.
      2. ---- address -----
  2. Resources
    1. 200GB disk space (to start) for build products, logs, reports, etc.
    2. New apache virtual host (ccbts.nrao.edu?).
  3. Maintenance
    1. Jenkins war file updates - change request to web helpdesk.
  4. Configurable builds
    1. Must be able to build CASA build components separately.
    2. Must be able to add or remove build components, or change build component order as dependencies change.
    3. Changes in upstream build components must trigger rebuild, relink, and retest of downstream build components as appropriate.
  5. Must be able to build in parallel on:
    1. RHEL 5.7/5.8
    2. RHEL 6.x - when available to CASA developers
    3. OS X 10.6
    4. OS X 10.7
    5. OS X 10.8 - when available to CASA developers
    6. several compilers on each host, as appropriate
      1. GCC
      2. Intel
      3. LLVM/Clang
    7. active/trunk, test, and stable branches
    8. 32 and 64 bit
      1. Slower 32 bit jobs must not hold up faster 64 bit jobs
  6. Must be able to test in parallel on:
    1. All platforms above
    2. Fedora core (version TBD)
    3. Maybe Ubuntu (version TBD)

Design Decisions

  1. We will keep the existing CASA Jenkins server running as we build the new CBT system.
  2. We will use Jenkins to coordinate build and test jobs in the new CBT system.
  3. Admin jobs will run in the "casaadm" user account.
  4. Build jobs will run in the "casabuild" user account.
  5. Test jobs will run in the "casatest" user account.

Jenkins Jobs

The following diagram roughly illustrates the Jenkins jobs used to build CASA. Boxes in the diagram are jobs defined in Jenkins. Ellipses are ssh sessions launched on remote computers by Jenkins.

Jenkins will monitor the CASA change control server. When Jenkins detects a commit to a CASA build component, it will start a job to build and test that change on all supported OS/Hardware combinations. If that change fails to build, Jenkins will notify the committer and end this job. If that change builds successfully, Jenkins will run commit acceptance tests for that build component. If these tests fail, Jenkins will notify the committer and end this job. If these tests pass, Jenkins will save the build products (libraries, apps, scripts, etc.), then start a job to build and test the following build component with the new change. This will continue until all CASA build components have been built and tested with this change.

jenkins-jobs.gif

Updating Jenkins Jobs diagram

jenkins-jobs.gif, was created using the Graphviz dot program. To update this diagram:

  1. Download jenkins-jobs.dot from this page.
  2. Edit jenkins-jobs.dot
  3. Run "dot -Tgif jenkins-jobs.dot -o jenkins-jobs.gif".
  4. Upload the updated files as attachments to this page.

-- ScottRankin - 2012-07-03
Topic attachments
I Attachment Action Size Date Who Comment
jenkins-jobs.dotdot jenkins-jobs.dot manage 2 K 2012-07-03 - 13:47 ScottRankin Jenkins Jobs Graphviz
jenkins-jobs.gifgif jenkins-jobs.gif manage 34 K 2012-07-03 - 13:45 ScottRankin Jenkins Jobs
Topic revision: r14 - 2013-07-31, ScottRankin
This site is powered by FoswikiCopyright © by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding NRAO Public Wiki? Send feedback