CASA Python Coverage

==


Environment setup

I've synchronized a build from jenkins to warp/atejeda from:
  • rsync -av /home/warp/casa/jenkins/work/jobs/casa-trunk/workspace/build/
  • cd update casainit.sh to use the local installation rather than the jenkins paths
  • source casainit.sh # nothing weird, I've updated to use a local installation.
  • An example to test:
    • cd testing && casapy --nologger -c /home/warp/casa/atejeda/coverage/build/linux-x86_64/python/2.7/runUnitTest.py test_clean[test11]

Install the python coverage package

  • wget --no-check-certificate http://pypi.python.org/packages/source/c/coverage/coverage-3.7.1.tar.gz && tar xzf coverage-3.7.1.tar.gz
  • INSTALL_DIR=$PWD
  • PACKAGES_DIR=$INSTALL_DIR/lib/python2.7/site-packages
  • mkdir -p $PACKAGES_DIR
  • cd coverage-3.7.1
  • PYTHONPATH=$PACKAGES_DIR python setup.py install --force --prefix $INSTALL_DIR
  • PYTHONPATH=$PACKAGES_DIR/coverage-3.7.1-py2.7-linux-x86_64.egg python -c "import coverage" > /dev/null 2>&1 && echo "ok" || echo "not ok"
  • copy $PACKAGES_DIR/coverage-3.7.1-py2.7-linux-x86_64.egg/coverage to <your CASA installation>/python/2.7/
  • copy $INSTALL_DIR/bin contents to <your CASA installation>/bin

Non intrusive

By defining this environments variables one can enable the coverage reports to stdout, seems there's a bug with HTML and XML report generation. A stdout_report example is attached.
  • environment variables:
    • NOSE_WITH_COVERAGE
    • NOSE_COVER_ERASE
    • NOSE_COVER_HTML
    • NOSE_COVER_BRANCHES
    • NOSE_COVER_XML
    • Optional
      • NOSE_COVER_HTML_DIR
      • NOSE_COVER_XML_FILE

Intrusive

Intrusive means that python/2.7/runUnitTest.py was modified in order to enable the coverage for python/2.7/__casac__ file tree, the configuration allows the possibility to define which packages should be covered.

Basically, add these code snippets
# set environment variables
CASA_DIR = os.environ["CASAPATH"].split()[0]
PYTHON_DIR = os.path.join(CASA_DIR, os.environ["CASAPATH"].split()[1], "python", PYVER)
TESTS_DIR = os.path.join(PYTHON_DIR, "tests")

and
# configure the coverage
py_coverage_tree = [ os.path.join(PYTHON_DIR, "__casac__") ]
coverage_instance = coverage.coverage(branch=True, source=py_coverage_tree)
coverage_instance.start()

main(testnames)

coverage_instance.stop()
coverage_instance.xml_report(outfile="%s/nosedir/xml/coverage.xml" % os.getcwd())
coverage_instance.html_report(directory="%s/nosedir/html/coverage" % os.getcwd())
del coverage_instance)

Coverage reports are generated as XML for Jenkins Cobertura Plugin and HTML (as pybot report style) per test following the usual test regression directory schema (everything is generated under nose directory):

nosedir/
|-- clean.last
|-- html
|   `-- coverage
|       |-- index.html
|-- test_clean.py
|-- test_clean.pyc
`-- xml
    |-- coverage.xml
    `-- nose.xml

Check nosedir_example.tar.bz2 for an example.

Example of the HTML report


coverage html report.png

HTML report extended


coverage html report extended.png

Jenkins Cobertura reports example


cobertura report.png

cobertura report extended.png

Either way, intrusive and not intrusive, the implementation/configuration for the coverage is done at runUnitTest.py, in that way the coverage will enabled by default for all tests using that script as is done in https://casa-jenkins.nrao.edu/job/casa-trunk/, in order to keep the reports it is needed to move the generated report before running each test.

Resources

-- AlexisTejeda - 2015-04-21

Topic attachments
I Attachment Action Size Date Who Comment
cobertura_report.pngpng cobertura_report.png manage 85 K 2015-06-02 - 15:23 AlexisTejeda  
cobertura_report_extended.pngpng cobertura_report_extended.png manage 41 K 2015-06-02 - 15:23 AlexisTejeda  
coverage_html_report.pngpng coverage_html_report.png manage 99 K 2015-06-02 - 15:23 AlexisTejeda  
coverage_html_report_extended.pngpng coverage_html_report_extended.png manage 122 K 2015-06-02 - 15:23 AlexisTejeda  
nosedir_example.tar.bz2bz2 nosedir_example.tar.bz2 manage 819 K 2015-06-02 - 15:26 AlexisTejeda  
This topic: Software > WebHome > SoftwareTestGroup > TestCoverageAnalysisDocumentation
Topic revision: 2015-06-02, AlexisTejeda
This site is powered by FoswikiCopyright © by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding NRAO Public Wiki? Send feedback