Improve CASA Testing

I created this page after discussions with different CASA developers and stake holders on different aspects of CASA testing, and realizing CASA does not have an accessible, written plan for improving CASA testing to reduces the amount of time lost by users due to bugs that could be found during development.

On this page, I focus on automated testing. Interactive testing by astronomers is out of scope of this page.

There are several pockets of good, even excellent, testing withing CASA. But, these generally focus on helping the original developer find problems. They generally do not fit into routine automated testing well, and do not provide enough information for non-experts to understand and fix problems.

Status: DRAFT



Approaches That Can Not Work

Some approaches to software testing can not work on CASA, due to resource constraints or missing information.

Exhaustive Testing

CASA is far too large and complex to test exhaustively. Most non-trivial software is too expensive to test exhaustively, and CASA is far from trivial.

Therefore, CASA must focus limited testing testing resources on the subset of CASA that is most useful to users.

Testing to Requirements

Software requirements are a conventional means of documenting user requirements, and the relative value of software features.

While some subsets of CASA are based on formal, written requirements, much of CASA is not. In discussions with Jeff Kern, he has told me that gaps will not be filled.

Therefore, we must find some other way to identify the subset of CASA that is most useful to users.

Approaches That Could Work

-- ScottRankin - 2013-10-21
Topic revision: r3 - 2014-01-28, ScottRankin
This site is powered by FoswikiCopyright © by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding NRAO Public Wiki? Send feedback