Benchmarking

Page for starting to think about benchmarking tests, in particular identifying where are the common bottlenecks etc. Feel free to grouse about poor performance, but give details! Free format for now...

Untarring

Tests r/w speed of disks, minimal CPU usage

13.7GB ASDM (TwHyd Band 7)

with Lustre (on multivac02): 1m55s

desktop (gauss; T3500) to local disk: 5m50s

- much faster on Lustre, as hoped/anticipated

CASA Guide Benchmarking and Testing

Adam L and Jared C have worked on revising the casaguide extraction script to have a "benchmarking" mode. The extraction script and two other python programs are attached at the bottom of the page.

The idea is the following:

1) Put casa_call.py into your CASA path. We'll toss an updated version here when once we have one. This little program defines a "Call" class that will be used to dump a line of text to the benchmark file for each task call.

2) [Optional and usually not necessary] Use the list_all_tasks.py to make sure that the script extractor has an up to date list of tasks. In order for the script extractor to work at the command line this list is currently cut and pasted into the script extractor. If the script extractor were run in CASA then this would not be necessary. Seems like a fairly arbitrary point.

3) Run the script extractor as usual but add a second command line parameter "benchmark" after the initial call. This will force the script to produce "benchmarking mode" output where each task is wrapped with an invocation of the Call class. The script also attempts to suppress user interaction . Table locks turned out to be an issue so we entirely shut down plotms (this is configurable). CLEAN gets run but the script attempts to force it to interactive = False. This could come back to bite us, but seems like a reasonable first cut.

4) Run the CASA script as usual ("execfile"). It will dump its benchmarking output to the currently defined variable out_file or "benchmark.txt" if none is defined. It should back up previous runs with the same name as, e.g., "benchmark.txt.1" ....

5) casa_call.py includes a routine to produce a basic summary of a single benchmarking run. We can work out a more sophisticated approach later.

Outstanding issues:

1) This is CASA focused. I assume we'll reach another solution for tar, rm, cp, etc.

2) Interactive input to CLEAN is currently lost.

-- MarkLacy - 2011-07-20

Topic attachments
I Attachment Action Size Date Who Comment
casa_call.py.txttxt casa_call.py.txt manage 2 K 2011-08-23 - 13:20 AdamLeroy Benchmarking utility. Needs to be in CASA path (specify with ~/.casa/init.py). WIP.
extractCASAscript_v2.py.txttxt extractCASAscript_v2.py.txt manage 13 K 2011-08-26 - 14:01 AdamLeroy Script extractor with benchmark mode implemented.
list_all_tasks.py.txttxt list_all_tasks.py.txt manage 396 bytes 2011-08-23 - 13:19 AdamLeroy Useful CASA doodad to give you all tasks in a list. Use to keep extractor up to date.
This topic: ALMA > WebHome > NAASC > DataServicesGroup > BenchMarking
Topic revision: 2011-08-26, AdamLeroy
This site is powered by FoswikiCopyright © by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding NRAO Public Wiki? Send feedback