CASA Python Coverage
=
=
Environment setup
I've synchronized a build from jenkins to warp/atejeda from:
- rsync -av /home/warp/casa/jenkins/work/jobs/casa-trunk/workspace/build/
- cd update casainit.sh to use the local installation rather than the jenkins paths
- source casainit.sh # nothing weird, I've updated to use a local installation.
- An example to test:
- cd testing && casapy --nologger -c /home/warp/casa/atejeda/coverage/build/linux-x86_64/python/2.7/runUnitTest.py test_clean[test11]
Install the python coverage package
- wget --no-check-certificate http://pypi.python.org/packages/source/c/coverage/coverage-3.7.1.tar.gz && tar xzf coverage-3.7.1.tar.gz
- INSTALL_DIR=$PWD
- PACKAGES_DIR=$INSTALL_DIR/lib/python2.7/site-packages
- mkdir -p $PACKAGES_DIR
- cd coverage-3.7.1
- PYTHONPATH=$PACKAGES_DIR python setup.py install --force --prefix $INSTALL_DIR
- PYTHONPATH=$PACKAGES_DIR/coverage-3.7.1-py2.7-linux-x86_64.egg python -c "import coverage" > /dev/null 2>&1 && echo "ok" || echo "not ok"
- copy $PACKAGES_DIR/coverage-3.7.1-py2.7-linux-x86_64.egg/coverage to <your CASA installation>/python/2.7/
- copy $INSTALL_DIR/bin contents to <your CASA installation>/bin
Non intrusive
By defining this environments variables one can enable the coverage reports to stdout, seems there's a bug with HTML and XML report generation. A
stdout_report example is attached.
- environment variables:
- NOSE_WITH_COVERAGE
- NOSE_COVER_ERASE
- NOSE_COVER_HTML
- NOSE_COVER_BRANCHES
- NOSE_COVER_XML
- Optional
- NOSE_COVER_HTML_DIR
- NOSE_COVER_XML_FILE
Intrusive
Intrusive means that python/2.7/runUnitTest.py was modified in order to enable the coverage for python/2.7/__casac__ file tree, the configuration allows the possibility to define which packages should be covered.
Basically, add these code snippets
# set environment variables
CASA_DIR = os.environ["CASAPATH"].split()[0]
PYTHON_DIR = os.path.join(CASA_DIR, os.environ["CASAPATH"].split()[1], "python", PYVER)
TESTS_DIR = os.path.join(PYTHON_DIR, "tests")
and
# configure the coverage
py_coverage_tree = [ os.path.join(PYTHON_DIR, "__casac__") ]
coverage_instance = coverage.coverage(branch=True, source=py_coverage_tree)
coverage_instance.start()
main(testnames)
coverage_instance.stop()
coverage_instance.xml_report(outfile="%s/nosedir/xml/coverage.xml" % os.getcwd())
coverage_instance.html_report(directory="%s/nosedir/html/coverage" % os.getcwd())
del coverage_instance)
Coverage reports are generated as XML for
Jenkins Cobertura Plugin and HTML (as pybot report style) per test following the usual test regression directory schema (everything is generated under nose directory):
nosedir/
|-- clean.last
|-- html
| `-- coverage
| |-- index.html
|-- test_clean.py
|-- test_clean.pyc
`-- xml
|-- coverage.xml
`-- nose.xml
Check
nosedir_example.tar.bz2 for an example.
Example of the HTML report
HTML report extended
Jenkins Cobertura reports example
Either way, intrusive and not intrusive, the implementation/configuration for the coverage is done at runUnitTest.py, in that way the coverage will enabled by default for all tests using that script as is done in
https://casa-jenkins.nrao.edu/job/casa-trunk/, in order to keep the reports it is needed to move the generated report before running each test.
Resources
--
AlexisTejeda - 2015-04-21