CASA Parallelization Task List

Item No. System Element Description Status Prerequisites People Target Cycle Notes
1 Environment Management            
1.01   Choose a preferred MPI implementation (CAS-5801) DONE   Rob 4.3  
1.01.1   Testing of mpi4casa vs the finally chosen preferred MPI lib. DONE   Justo 4.3 (in case it is not OpenMPI)
1.01.1   Script that generates a hostfile from the Torque resource environmental variables     Justo 4.3 If required
1.02   Integrate preferred MPI implementation + mpi4py in CASA dev. Environment (update yum repo) DONE 1.01 Julian 4.4 CAS-7019
1.03   Modify cmake to allow compilation using MPI and disable GIL DONE 1.02 Julian 4.4  
1.04   Integrate mpi4casa tests in Jenkins environment     DONE 4.4  
1.10   Logging: Use MPI at C++ level to sync. logging (CAS-6705) DONE   Sandra 4.4  
2 User Interface            
2.01   Definition of use case and requirements     TBD 4.4+  
2.02   Implementation of simple parameter management tool.     TBD 4.4+  
2.03   Multi-MS support in the viewer.     TBD 4.4+  
3 Filling & Data Import MSTransform     Sandra    
3.01   MSTransform: Create Cvel2 wrapper DONE   Sandra 4.3  
3.02   MSTransform: Heuristics of input/output MMS and transformations such as the combination of spws and time averaging across scans. Implementation of user warnings.\ DONE   Sandra 4.3  
3.03   MSTransform: Some subtables still need to be reviewed and consolidated for the MMS output creation. DONE   Justo 4.3 CAS-6545, CAS-6529, CAS-6706, CAS-6707
3.05   MSTransform: Improve error handling for MMS processing. Deal with NULL MS selections, failed SubMSs, etc. DONE   Sandra 4.3  
3.06   MSTransform: Create new regression tests. Incorporate into nightly system.     Sandra 4.4 CAS-7189
3.07   MSTransform: Create new heuristics tests to verify all typical transformations.     Sandra 4.3,4.4 many tests are already included
3.08   Documentation on MSTransform best practices for MMS partitioning.     Sandra 4.4 in progress
3.09   Lazy Filler & MSTransform integration     TBD 4.4, 4.5 tested. TBI in 4.5
4 Flagging            
4.01   Flagdata: Compression of online flags when using list mode with MMS DONE   Sandra, Justo 4.4 CAS-7201
5 Calibration            
5.01   Generalized Application of CalTables     TBD 4.4  
6 Imaging            
6.01   Implement imager parallelization at C++ level (CAS-6680)       4.4, 4.5 CAS-6680, CAS-7324
7 Pipeline            
7.01   Implement pipeline heuristics to determine best parallelization approach (settings for cluster, partition, and re-tiling prior to imaging) depending on the data reduction scenario.\       4.5  
7.02   Introduce parallelization awareness in the section of the pipeline scripts which use direct tool access..\       4.5  
8 Testing            
8.02   Test effects of retiling according to the imager parallelization strategy (CAS-5122)       4.4  
Topic revision: r12 - 2017-01-12, SandraCastro
This site is powered by FoswikiCopyright © by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding NRAO Public Wiki? Send feedback