CASA Python Test Profile

==

Environment setup

I've synchronized a build from jenkins to warp/atejeda from:
  • rsync -av /home/warp/casa/jenkins/work/jobs/casa-trunk/workspace/build/
  • cd update casainit.sh to use the local installation rather than the jenkins paths
  • source casainit.sh # nothing weird, I've updated to use a local installation.
  • An example to test:
    • cd testing && casapy --nologger -c /home/warp/casa/atejeda/coverage/build/linux-x86_64/python/2.7/runUnitTest.py test_clean[test11]

Install the needed packages

  • wget --no-check-certificate https://pypi.python.org/packages/source/p/psutil/psutil-2.1.3.tar.gz && tar xzf psutil-2.1.3.tar.gz
  • INSTALL_DIR=$PWD
  • PACKAGES_DIR=$INSTALL_DIR/lib/python2.7/site-packages
  • mkdir -p $PACKAGES_DIR
  • cd psutil-2.1.3
  • PYTHONPATH=$PACKAGES_DIR python setup.py install --force --prefix $INSTALL_DIR
  • PYTHONPATH=$PACKAGES_DIR python -c "import psutil" > /dev/null 2>&1 && echo "ok" || echo "not ok"
  • unzip $PACKAGES_DIR/psutil-2.1.3-py2.7-linux-x86_64.egg and copy the contents to <your CASA installation>/python/2.7/ # can be automated
  • # in your <your CASA installation>/python/2.7/
    • wget --no-check-certificate
  • Problem with this?: OSError: [Errno 16] Device or resource busy: '/users/atejeda/tmp/pymp-SgvsQx/.nfs0000000000c3c0a6000001ce' remove the directory tmp/

Intrusive

Intrusive means that python/2.7/runUnitTest.py was modified in order to enable the coverage for python/2.7/__casac__ file tree, the configuration allows the possibility to define which packages should be covered.

Add this code snippet:
import psprofile
    xmlfile = xmldir+'nose.xml'
    try:
        if (HAVE_MEMTEST and MEM):
            regstate = nose.run(argv=[sys.argv[0],"-d","-s","--with-memtest","--verbosity=2",
                            "--memtest-file="+xmlfile], suite=list, addplugins=[memTest.MemTest()])
        else:
            regstate = nose.run(argv=[sys.argv[0],"-d","-s","--with-xunit","--verbosity=2",
                            "--with-psprofile", "--psprofile-file=%s/psprofile.json" % xmldir,
                            "--xunit-file="+xmlfile], suite=list, addplugins = [psprofile.PSProfile()])

Performance reports are generated in a single json line in a single file per test, the performance data contains per second for the process:
  • Used mem (rss, vms) in bytes
  • IO counters [read count, write count, read bytes, write bytes ]
  • Used CPU
  • Used file descriptors
{
   "test19":{
      "mem":[
         [
            161497088,
            1026613248
         ],
         [
            173121536,
            1192132608
         ],
         [
            179576832,
            1166028800
         ]
      ],
      "fds":[
         17,
         49,
         16
      ],
      "ioc":[
         [
            6203,
            400,
            4096,
            102400
         ],
         [
            9077,
            873,
            16384,
            2891776
         ],
         [
            10747,
            1947,
            16384,
            6459392
         ]
      ],
      "cpu":[
         245.1,
         742.2,
         0.0
      ],
      "time":[
         1428700861.345394,
         1428700862.389876,
         1428700863.390347
      ]
   }
}

The data can be process by using several python plotting libraries or like in this example was generated by octave:

test 01 listobs on the initial data set cropped.png

The directory layout is the same, the only different is that the json file is generated under xm/ directory.
nosedir/
|-- clean.last
|-- html
|   `-- coverage
|       |-- index.html
|-- test_clean.py
|-- test_clean.pyc
`-- xml
    |-- coverage.xml
    |-- nose.xml
    `-- psprofile.json

The implementation/configuration for the performance is done at runUnitTest.py, in that way the coverage will enabled by default for all tests using that script as is done in https://casa-jenkins.nrao.edu/job/casa-trunk/, in order to keep the reports it is needed to move the generated report before running each test.

Resources

-- AlexisTejeda - 2015-04-21
Topic revision: r2 - 2015-06-02, AlexisTejeda
This site is powered by FoswikiCopyright © by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding NRAO Public Wiki? Send feedback