CASA Software Engineering Project: Build a CASA 3rd Party Package Deployment Pipeline

Develop a CASA deployment pipeline to

  • deliver CASA 3rd party packages as needed all developers, on all supported platforms,
  • improve consistency and quality of delivered 3rd party packages,
  • reduce manual work required to deliver 3rd party packages,
  • deliver CASA 3rd party packages to all CASA developers for each CASA release before development starts on that release, and
  • allow developers to work on > 1 CASA release at a time.

Status: See CAS-5388.

Team

Customers
  • ESO Network Admins
  • JAO Network Admins
  • NAOJ Network Admins
  • NRAO Network Admins
  • CASA Developers

Collaborators
  • TBD

Plan

At this time (2013-11-20) we are in crisis. We can not build the CASA trunk or release-4_2_0 branch on OS X.

Resolve this crisis in any way possible. Learn from the experience, then generalize to Linux distributions.

Deliverables

  • Developers 3rd party package installation instructions.
  • Package maintainer's instructions.

Requirements

See also:

Scope

CASA 3rd party packages are software packages added to CASA development systems that provide libraries or programs required to develop CASA but are:

  • not provided by the host OS at all,
  • provided by the host OS, but not installed by default,
  • provided by the host OS, but the host OS version contains bugs that CASA can not effectively work around,
  • provided by the host OS, but the host OS version is old enough, or new enough, that the interfaces are too different from the equivalent interfaces provided by that package on other OSs.

Versions

  • To limit the number of 3rd party packages maintained by CASA, developers shall use 3rd party packages available in the host OS package repository whenever possible. We call these native packages.
  • When suitable 3rd party packages are not available in the host OS package repository, CASA shall provide a mechanism to install required 3rd party packages on all CASA supported host OSs.
    • CASA shall provide the same major.minor version of that 3rd party package on every supported OS.

Source Code Cache

  • Source code used to build 3rd party packages shall be kept in change control at:

https://svn.cv.nrao.edu/svn/casa-3rd-party/trunk

  • All copies of a CASA provided 3rd party package, regardless of destination, shall be built from the same source.

General

  • As much as possible, CASA developers must be able to install required 3rd party packages without requiring administrator privilege on development systems.
    • Depending on development site policies that we have no influence over,
      • developers may have to ask local administrators to install packages available from the host OS package repository, and
      • developers may have to ask local administrators to create some directories and give developers control over those directories.

Package Managers

Based on known requirements, we must use the following package management systems to install native packages:

Based on known requirements, we must use one of the following package management systems:

Design

Source Code and Package Specifications

The 3rd party package source repository (https://svn.cv.nrao.edu/svn/casa-3rd-party/trunk) is organized as:
  • one directory for the source for each provided package containing
    • the source for the provided package as either
      • a tar package from the upstream maintainers, or
      • if this package is CASA fork (for example, dbus-cpp), the unpacked source code
    • the source for other 3rd party packages required only by this package
    • any patches that must be applied to either the primary or required packages for all platforms
  • a Debian package specification directory containing
    • one package specification file for each 3rd party package we provide for Debian based distributions
      • Note: if package specification files become difficult to maintain and support Debian, Ubuntu, etc., we may split them into separate directories, one for each distribution.
    • any patches that are only required to produce Debian packages.
  • a Macports ports directory containing
    • one port file for each 3rd party package we provide for Macports based distributions
    • any patches that are only required to produce Macports packages.
  • an RPM spec directory containing
    • one spec file for each 3rd party package we provide for RPM based distributions
      • Note: if spec files become difficult to maintain and support RHEL 5, RHEL 6, Fedora 17, etc., we may split them into separate directories, one for each distribution.
    • any patches that are only required to produce RPM packages.

EXCEPTION:

We have, in a private repository, the source for a package TBD, that we may not distribute. Source for this package shall not be kept in the repository above, but the package specification files shall be. Source will have to be checked out at package build time. Details TBD.

Implementation

(Note: replace this note with a one paragraph summary of the implementation for this project, a short bullet list of implementation details, or link(s) other pages documenting the implementation for this project. Often, this will be a bullet list including the location for any source code in change control and any user accounts or computers required by this project.)

Tasks

Project tasks are tracked with Jira sub-tasks under this project's master task. See CAS-5388.

-- ScottRankin - 2013-07-16
Topic revision: r6 - 2013-11-20, ScottRankin
This site is powered by FoswikiCopyright © by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding NRAO Public Wiki? Send feedback