CASA Python Coverage
I've synchronized a build from jenkins to warp/atejeda from:
- rsync -av /home/warp/casa/jenkins/work/jobs/casa-trunk/workspace/build/
- cd update casainit.sh to use the local installation rather than the jenkins paths
- source casainit.sh # nothing weird, I've updated to use a local installation.
- An example to test:
- cd testing && casapy --nologger -c /home/warp/casa/atejeda/coverage/build/linux-x86_64/python/2.7/runUnitTest.py test_clean[test11]
Install the python coverage package
- wget --no-check-certificate http://pypi.python.org/packages/source/c/coverage/coverage-3.7.1.tar.gz && tar xzf coverage-3.7.1.tar.gz
- mkdir -p $PACKAGES_DIR
- cd coverage-3.7.1
- PYTHONPATH=$PACKAGES_DIR python setup.py install --force --prefix $INSTALL_DIR
- PYTHONPATH=$PACKAGES_DIR/coverage-3.7.1-py2.7-linux-x86_64.egg python -c "import coverage" > /dev/null 2>&1 && echo "ok" || echo "not ok"
- copy $PACKAGES_DIR/coverage-3.7.1-py2.7-linux-x86_64.egg/coverage to <your CASA installation>/python/2.7/
- copy $INSTALL_DIR/bin contents to <your CASA installation>/bin
By defining this environments variables one can enable the coverage reports to stdout, seems there's a bug with HTML and XML report generation. A stdout_report
example is attached.
- environment variables:
Intrusive means that python/2.7/runUnitTest.py was modified in order to enable the coverage for python/2.7/__casac__ file tree, the configuration allows the possibility to define which packages should be covered.
Basically, add these code snippets
# set environment variables
CASA_DIR = os.environ["CASAPATH"].split()
PYTHON_DIR = os.path.join(CASA_DIR, os.environ["CASAPATH"].split(), "python", PYVER)
TESTS_DIR = os.path.join(PYTHON_DIR, "tests")
# configure the coverage
py_coverage_tree = [ os.path.join(PYTHON_DIR, "__casac__") ]
coverage_instance = coverage.coverage(branch=True, source=py_coverage_tree)
coverage_instance.xml_report(outfile="%s/nosedir/xml/coverage.xml" % os.getcwd())
coverage_instance.html_report(directory="%s/nosedir/html/coverage" % os.getcwd())
Coverage reports are generated as XML for Jenkins Cobertura Plugin
and HTML (as pybot report style) per test following the usual test regression directory schema (everything is generated under nose directory):
| `-- coverage
| |-- index.html
for an example.
Example of the HTML report
HTML report extended
Jenkins Cobertura reports example
Either way, intrusive and not intrusive, the implementation/configuration for the coverage is done at runUnitTest.py, in that way the coverage will enabled by default for all tests using that script as is done in https://casa-jenkins.nrao.edu/job/casa-trunk/,
in order to keep the reports it is needed to move the generated report before running each test.