Casa HPC/Parallelization meeting agenda/minutes

Thursday [2 April 2015], [ESO Centaurus, C.2.01], [15:00 UT]

How to connect

• Dial in: +49 89 6834
• Video connection: Use this one (46104@134.171.42.27). Other option is 46104@eso.org

Attendees:

Agenda

1. News
   • From casa-test-4.4.115 on, mpicasa sets OMP_NUM_THREADS to 1.
   • Race condition that caused partition to fail (bug in openMPI) seems to be fixed in casa-test-4.4.123 onwards.
   • We want to have a face to face meeting of the HPC team in Socorro, the week of 22nd June.
2. Status of open items:
   • Monitoring slowness of gaincal (James)
   • Number of open files problem (MultiFile implementation) - CAS-4860 (Julian)
   • Jim/Jeff to find a way to have the CASA trunk build in the cluster without the use of tricks and patches as it is currently done.
3. Testing:
   • Imaging (conclusions of Sanjay’s report and next steps)
   • EVLA pipeline - status and next steps
   • ALMA pipeline - status and next steps. See the following results: https://safe.nrao.edu/wiki/bin/view/HPC/3EBSALMAPipeline

Minutes

• George will commit a fix for the gaincal tileshape/caching issues.
• Julian tested the MultiFile implementation from Ger, which seems to work. Sandra will run the ALMA pipeline script which failed previously with the "number of open files" and verify too.
• Sanjay proposes 2 options for the continuing his tests: 1) multithreaded gridders; 2) cube image scaling tests. James can help him with the scaling tools. Sanjay says there is nothing else needed to be implemented in the MPI framework to continue his tests.
• Testing the EVLA pipeline with split2 has not contributing factor in time. There are no problems running this script in sequential, but in parallel it shows a problem which seems to come from importevla leaving the MS in the cache.
• We should get rid of importevla and use importasdm for EVLA too, so that the lazy mode can be used.
• Sandra reported a problem with a write-lock from plotms while running the ALMA pipeline. She also showed several improvements when running the pipeline in parallel.
• We should aim to run plotms in parallel (instead of parallelizing it). Plotms is a great contributor to time in EVLA pipeline runs.

Action Item List

Useful Links

• Major blockers
• Current status
• CASA parallelization task list
• Long-term issue chart
• Bleeding tests area
• Documentation (see Reference Materials topic)