Casa HPC/Parallelization meeting agenda/minutes

Thursday [29/01/2015], [ESO Centaurus, C.2.01], [9:00 AM MST]

How to connect

• Dial in: +49 89 307 6834
• Video connection: 46104@eso.org (46104@134.171.42.27)

Attendees:

Agenda

1. Partition tests with new balanced mode
2. Status of test system after the inclusion of MPI packages. Ok
3. Status of HPC deliverables in 4.4
4. Draft of Quick Guide to running functional tests with MMS
5. Imaging team report
6. AOB

Minutes

1. The video connection worked this time.
2. SC modified partition to use the new balanced mode as the default in the task (as separationaxis='auto'). All the partition tests pass. Some mstransform_mms tests fail and the code is being fixed. SC also run the pipeline script with the balanced mode by removing the setjy calls. The script gives a bad_alloc error in applycal due to lack of memory. SC will try on a smaller MS and with fewer subMSs. The current test used 8 subMSs with about 5Gb each.
3. JG also tested the balanced mode in the cluster in Socorro with a cycle2 script. It runs fine. He also tested lazy mode with MMS in the same script which also runs without problems until the end.
4. JG and Andy worked together to choose an ALMA band 3 script with a small MS which can be used to test in parallel automatically. Justo will work on it.
5. JT reported that he fixed the application of online flags slowness both on normal MSs and on MMSs.
6. JT reported that it is time to try the MPI packages in the casa tarball again. He will contact Darrell about it.
7. TT will work on the parallelization of the cube images script.
8. JG and SB reported that the list of MPI functions seems to be complete with respect to what is available in parallel_go and used in tclean. SB would like to use the same interface as that of parallel_go, so Justo will create wrappers for these functions in mpi4casa. He will test them independently of tclean.
9. LD reports that the new release of the pipeline is faster in several aspects.
10. JG explained the new balanced mode.
11. JK requested the default of numsubms to be set to "auto" instead of 0. As soon as all tests pass, SC will set the new default in partition and release the quick-guide for the developers to test with MMSs.

Action Item List

• SC to send James the pipeline script and MS so that he can try to reproduce and debug the applycal crash.
• JT to send email to Darrell asking to create another casapy-test with the MPI packages.
• JK check if other people in the team also prefer separated parallelisation logs

Useful Links

• Major blockers
• Current status
• CASA parallelization task list
• Long-term issue chart
• Bleeding tests area
• Documentation (see Reference Materials topic)