 |
|
You are here: NRAO Public Wiki>HPC Web>CasaParallelization>CasaParallelTaskList (2017-01-12, SandraCastro)
Edit wiki text
Edit
Attach
Print version
CASA Parallelization Task List
| Item No. | System Element | Description | Status | Prerequisites | People | Target Cycle | Notes |
|---|---|---|---|---|---|---|---|
| 1 | Environment Management | ||||||
| 1.01 | Choose a preferred MPI implementation (CAS-5801) | DONE | Rob | 4.3 | |||
| 1.01.1 | Testing of mpi4casa vs the finally chosen preferred MPI lib. | DONE | Justo | 4.3 | (in case it is not OpenMPI) | ||
| 1.01.1 | Script that generates a hostfile from the Torque resource environmental variables | Justo | 4.3 | If required | |||
| 1.02 | Integrate preferred MPI implementation + mpi4py in CASA dev. Environment (update yum repo) | DONE | 1.01 | Julian | 4.4 | CAS-7019 | |
| 1.03 | Modify cmake to allow compilation using MPI and disable GIL | DONE | 1.02 | Julian | 4.4 | ||
| 1.04 | Integrate mpi4casa tests in Jenkins environment | DONE | 4.4 | ||||
| 1.10 | Logging: Use MPI at C++ level to sync. logging (CAS-6705) | DONE | Sandra | 4.4 | |||
| 2 | User Interface | ||||||
| 2.01 | Definition of use case and requirements | TBD | 4.4+ | ||||
| 2.02 | Implementation of simple parameter management tool. | TBD | 4.4+ | ||||
| 2.03 | Multi-MS support in the viewer. | TBD | 4.4+ | ||||
| 3 | Filling & Data Import | MSTransform | Sandra | ||||
| 3.01 | MSTransform: Create Cvel2 wrapper | DONE | Sandra | 4.3 | |||
| 3.02 | MSTransform: Heuristics of input/output MMS and transformations such as the combination of spws and time averaging across scans. Implementation of user warnings.\ | DONE | Sandra | 4.3 | |||
| 3.03 | MSTransform: Some subtables still need to be reviewed and consolidated for the MMS output creation. | DONE | Justo | 4.3 | CAS-6545, CAS-6529, CAS-6706, CAS-6707 | ||
| 3.05 | MSTransform: Improve error handling for MMS processing. Deal with NULL MS selections, failed SubMSs, etc. | DONE | Sandra | 4.3 | |||
| 3.06 | MSTransform: Create new regression tests. Incorporate into nightly system. | Sandra | 4.4 | CAS-7189 | |||
| 3.07 | MSTransform: Create new heuristics tests to verify all typical transformations. | Sandra | 4.3,4.4 | many tests are already included | |||
| 3.08 | Documentation on MSTransform best practices for MMS partitioning. | Sandra | 4.4 | in progress | |||
| 3.09 | Lazy Filler & MSTransform integration | TBD | 4.4, 4.5 | tested. TBI in 4.5 | |||
| 4 | Flagging | ||||||
| 4.01 | Flagdata: Compression of online flags when using list mode with MMS | DONE | Sandra, Justo | 4.4 | CAS-7201 | ||
| 5 | Calibration | ||||||
| 5.01 | Generalized Application of CalTables | TBD | 4.4 | ||||
| 6 | Imaging | ||||||
| 6.01 | Implement imager parallelization at C++ level (CAS-6680) | 4.4, 4.5 | CAS-6680, CAS-7324 | ||||
| 7 | Pipeline | ||||||
| 7.01 | Implement pipeline heuristics to determine best parallelization approach (settings for cluster, partition, and re-tiling prior to imaging) depending on the data reduction scenario.\ | 4.5 | |||||
| 7.02 | Introduce parallelization awareness in the section of the pipeline scripts which use direct tool access..\ | 4.5 | |||||
| 8 | Testing | ||||||
| 8.02 | Test effects of retiling according to the imager parallelization strategy (CAS-5122) | 4.4 |
Edit | Attach | Print version | History: r12 < r11 < r10 < r9 | Backlinks | View wiki text | Edit wiki text | More topic actions
Topic revision: r12 - 2017-01-12, SandraCastro
ALMA
Copyright © by the contributing authors. All material on this collaboration platform is the property of the contributing authors. Ideas, requests, problems regarding NRAO Public Wiki? Send feedback