Project Scope

(Need to define the B&T System limits)

B & T is the category of all work involved in taking the raw source code and producing valid executable software. The executable systems supported are targeted at developers, the CASA project as a whole (e.g., Jenkins), and end-users.

The following systems or functions are within the scope of the Build & Test System:
  • Version Control System (extends beyond just the B&T group)
    • We are currently using SVN (a centralized system VCS) for this.
      • SVN Advantages: Familiarity; already in place; widely supported by development environments, etc.; C tools that fit in well with the Unix philosophy
      • SVN Disadvantages: SVN merge is a major pain, and works solely on version history, not content. Branch diversion can rapidly become problematic (as we have seen), leading to major delays and broken builds. In prinicple, this could be mitigated by (e.g.) just avoiding merging and having developers commit to all relevant branches directly, but this could be quite labor-intensive and still difficult to track.
    • Questions:

  • Issue Tracking System
    • Currently using Jira
      • Generally seems to be liked (or at least accepted) by all stakeholders. Also already in heavy use by other related projects.
      • Plugins for most of the other relevant major B&T component options.
      • Adjustments are being made regarding some details of usage, but seems to work for CASA.
      • No current plans to replace Jira.

  • Developer Build Environment
    • Basic Build System
    • Build Tool-Chain
      • Currently using Cmake with gcc/clang. Cmake was originally selected for cross-platform compatibility. Seems to require custom scripts in addition to config files, though.
      • Do we want to stay with Cmake, or switch to something else (if something else, what?)
      • Some newer tools, e.g. gradle ( http://www.gradle.org/overview ) allow management of (and automated selection from) versioned libraries of binaries. Might be worth looking into?

  • Developer Test Environment
    • Unit test framework - MGR question: How are we doing nowadays with this?

  • External Package Integration
    • Qt, Qwt, Boost, python, ipython, D-Bus, etc. Probably need to get a full list in here.
    • This is something the B&T group need to get a much better handle on. Need to start imposing stricter policies, procedures, and timeline limits on this for any given 6-month release cycle.

  • Continuous Build and Test Servers (CI - Continuous Integration)
    • Currently using Jenkins with Robot Framework. Lots of benefits: Already familiar, existing set-up (but needs work), open source, mature, good integration with other tools, feature complete
      • First question: Should we continue to use Jenkins, or move to something else? (If something else, then what? https://en.wikipedia.org/wiki/Comparison_of_continuous_integration_software ). MGR's initial inclination is to stay with Jenkins.
      • Assuming that we stay with Jenkins, we need to improve build and test automation.
        • We currently have two Jenkins servers, casa-jenkins and casa-cbts. The latter was eventually intended to replace the former (which does not really follow Jenkins usage conventions, and uses a Perl script to send out notifications, rather than using the built-in features of Jenkins). We should aim for a single functioning server.
        • Four main problems currently:
          • No cross-platform testing.
          • It is not currently clear to developers how to control which tests are run.
          • It is not publicly available.
          • Every commit (even null ones) trigger a complete, monolithic e2e build. To address this will require some changes in CASA's architecture, to allow modular builds (e.g. make it possible to only build gcwrap, without requiring a rebuild of everything else). We cannot currently build CASA in one place, tar it up, move it somewhere else, untar it, and have it work. We should probably either build from source on all supported platforms, or abandon some Unix conventions and use a cleverer package management system (e.g. Nix, or possible a portable package creation system). Anything we want to use should ideally be "more Pythonic".
      • It is also possible to support continuous builds in Jenkins for (say) individual developers, if it is needed (e.g. for a personal branch).

  • Integrated Testing & Validation
    • Module Tests (equivalent to component interface testing) - Under development. Is this within B&T group's scope, or Andy Hale's group?
    • Integration testing - Not currently done as a formal, separate item. Should we? If so, is this within B&T group's scope, or Andy Hale's group?
    • Automated GUI testing - Not currently done, but should be implemented. Available tools to be surveyed and compared (DogTail, etc).
    • End-to-End System Regressions
    • Acceptance testing
      • User testing by scientists at the various sites (NRAO, ESO/Arc Nodes, NAOJ, plus new stakeholders: Australia, South Africa)
      • More details to be added on this

  • Release Tools / Delivery (i.e. software package distribution to end users)
    • Currently producing rpms, tar files and OS X .dmg files
    • Ideally, packaging process should be able to generate all of these routinely (or whichever subset of these the project ultimately decides to deliver)
    • Ideally, packaging process should be able to generate deliverable packages for all advertised, supported platforms
    • Currently, the (probably unnecessarily) large number of internal module dependencies result in the generation of single large downloads.
    • If CASA's internal architecture can be made more modular in the future (as is the stated goal), then in principle, it could be possible to deliver tailored, smaller installations directly to end-users using a Zero Install-type ( https://en.wikipedia.org/wiki/Zero_Install ) technology.

  • Other considerations
    • Software
      • Licensing
      • Cross-tool integration (including Jira)
      • General maturity and support
      • Ease of migration from existing systems
      • Some newer tools/toolchains available may span several of the above categories
    • Hardware requirements
      • Linux and OS X machine availability for developers that need it
      • Fast machines for running builds
      • Not-so-fast machines for running (continuous) tests
    • Staffing
      • We should be able to indicate how approximately many lines of code are present, and what typical industry standards are regarding staffing for B&T operations for similarly-sized software products.
    • Timelines
      • Realistically achievable frequency of package deliveries
      • Key deadlines for six-month delivery cycles

  • Code review software also considered
    • We don't currently have a standard platform for this. Do we need one?
    • Generally deemed to be beyond the scope of the B&T group - more a consideration for Architecture
    • However, it would still be nice from the B&T perspective to have some way to automatically periodically audit all the internal module dependencies

Concept of Operations

(Need to define how the B&T system will be used. The primary objective is to ensure that the operational needs of users (developers) are clearly understood and that the rationale for the derived performance requirements is documented. This information can edify the flow-down into lower level specifications as part of the architectural design process.)

CASA-Process-Workflow.png

-- RobSelina - 2014-05-29
Topic attachments
I Attachment Action Size Date Who CommentSorted ascending
CASA-Process-Workflow.pdfpdf CASA-Process-Workflow.pdf manage 122 K 2014-05-29 - 10:36 RobSelina  
CASA-Process-Workflow.pngpng CASA-Process-Workflow.png manage 124 K 2014-05-29 - 10:36 RobSelina CASA Workflow
Topic revision: r5 - 2014-06-19, MarkRawlings
This site is powered by FoswikiCopyright © by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding NRAO Public Wiki? Send feedback