CAS-10834: In flagdata, mode = summary behavior different between serial and parallel runs
Closed: not a bug. The casalog showed the flag summary of one of the MPIServers, not the consolidated total.
CAS-10794: Slowdown of pipeline tasks in parallel mode
Chat with Ger....... He suggests looking into differences in tiling parameters.
There seem to be big differences in tile shapes and other parameters between standard / concatenated images ( CAS-10796).
But there doesn't seem to be any differences between single MSs (serial) and Multi-MSs (parallel) for which we also see severe performance issues, for example in MS selection ( CAS-10837).
We started to re-run some of the E2E5 datasets with different number of cores to understand:
how they affect the speedup of pre-imaging tasks and tclean. Task tclean tends to scale well with the number of cores (as seen in past tests and benchmarks). Other tasks such as flagdata, applycal and split do not scale as well with N_cores, therefore it is wise to think that the choice of N_cores for flagdata+others is different than that given to tclean.
Verify in the existing datasets how the processing time of tclean varies with different parameters such as: specmode, gridder, imsize, nchan, deconvolver
Crystal created CAS-10838 and set it to 5.2 release, but this should be moved to 5.3. The new feature is unrelated to the pipeline or parallelisation. It is to expose the hidden parameter "reindex" in mstransform so that manual ALMA processing looks like pipeline products.
I would like this ticket to be accepted in the CASA context first, because it means people will start creating MSs without any reindexing. I don't know what will happen if they try to mix reindexed and non-reindexed MSs, for example in imaging!
Performance with reference-concat images + slowness of 'concat' itself
tclean speedup variation analysis (covered in HPC section)
Model writes for parallel run
statwt2 incremental requirements
CASA 5.1.1-3 / Pipeline
Morgan gave go ahead on Friday, sanity checks pass, acceptance ?
CASA 5.2 / Pipeline
What to do about virtually concatenated images
CASA 5.3 / Pipeline
Temporary issue with ms.selectinit
Warning issued to pipeline group concerning behavior change of ms.nrows
non HPC pipeline efficiency issues
use of imhead task rather than ia tool in findContuum.py module, Todd to investigate
too many ms.open calls in hif_makeimlist (?), under investigation
ALMA backwards compatibility issues iin hifa_restoredata
Reported that the change in behaviour of ms.nrow() broke the ALMA pipeline in 5.3.0-prerelease. Now it seems that setjy also needs to be updated with the new ms.nrow(). (pford: setjy fixed, ready to re-test with new tarballs in CAS-10818)
Discussions with Kana on the parallelisation of the SD pipeline
attended ICT leads meeting October 16-18, presented update on the pipeline
further discussion with Remy and others on restore data backwards compatibility issues
followup with Remy and Dirk on some non-HPC pipeline efficiency issues
performed sanity check on the 5.1.1-3 prerelease tarball
CAS-10818 (ms tool changes to selectinit and nrow) - fixed selectinit, fixed setjy nrow call, sent email to group about change in nrow
CAS-10822 (plotms spawns new windows) - in progress, forks new process when dbus (or connection to it) fails
CAS-9053 (atm/tsky overlays) - committed changes in response to validation testing report
Hosted candidate for CARTA position
CAS-10732/CAS-10695 : both tickets for investigating filler speed (asdm2MS, bdflags2MS). Mostly just getting my bearings on how best to profile these things. Also, every time I look into this code I have a wtf distraction that I end up following that distracts from why I was there in the first place.
Moving my ALMA ASDM workspace over to git.
Some gbtidl work for GBO - mostly driven by the unexpected move of the IDL installation here in CV which broke several things (a few things are still broken).
Some followup on CASA-VLBI meeting
Prep for QUESO talk next week
Federico M Pouzols
Away at ASTRON this week for ERIS2017.
Trying to investigate various slowdown-in-parallel issues (tickets under CAS-10794).
Last 3 (longer) runs for E2E5 datasets ready this week. Weblogs will be transferred soon, just waiting for last serial run to finish.
Friday NAOJ Meeting
resumed work for sideband separartion algorithm ( CAS-8091)
investigated unexpected changes of pipeline results and found they come from change of ms.getdata. reported to casa-staff (and pipeline developers as heads-up).