Thursday Morning Meeting 8 June 2017

  • DIAL-IN NUMBERS & PASSCODES:
  • IP: 192.33.117.12##8110
  • Phone: (434) 817-6524

Attendance

  • Socorro: Jeff, George, Martin, Pam, Sandra, Federico, Enrique, Juergen, Kumar
  • CV: Darrell, Jen, Anand, Ville, Morgan
  • Garching:
  • SCO:

News / Meetings / Visitors

  • Pipeline SsS change (temporary) - John Hibbard -> Remy Indebetouw
  • NRAO Budget Summit June 13-14, Jeff in CV
  • CARTA meeting - Socorro, June 19-23
  • CASA 5.0
    • Jeff: there are no blocker tickets.
    • Review of "blocker" tickets in JIRA:
      • CAS-10231 -- Crystal's ticket on mosaic; Tak is investigating; maybe user error.
      • CAS-10233 -- ready to merge
      • Documentation -- only functionality is a blocker
      • CAS-9233 -- Anything pipeline can postponed to 5.1
      • CAS-6688 -- supersceded by cas-10050
      • CAS-9208 - 5.0; other sibling tickets are 5.1.
    • Jeff expects to release the week after next.
    • Release process:
      • Jeff decides we are done.
      • Ville creates tarballs.
      • Spot check on tarballs.
      • Make an announcement.
      • (However, this process may be a bit different on account of the new documentation.)
  • CASA 5.1
    • Verifying dates
      • July 1: CASA 5.1-P2 2nd release for testing. After this point, bug fixes only. Needs all new features verified before Cy5 E2E tests so we are not testing them during E2Eonline
      • Aug 1: CASA 5.1 freeze
      • Aug: E2Eonline tests including PL
      • ~Aug15: Benchmark tests with final CASA 5.1 release candidate
      • ???: test of all associated DR workflow (SACM scripts, data packer, scriptForPI.py) - will need to "bless" PL/CASA version used for this test (probably before PL Acceptance test report finalized)
      • Aug 30: PL Acceptance test report due (feature completion; E2Eonline performance; Benchmarks)
      • Sept 01: final CASA 5.1 release

Build, Release

Verification Testing

Validation Testing

  • EA validation of VB2 in their processes?
  • CAS-10118 -- George, can we close?
  • clarification of process for fixes that need to go into both 5.0 and 5.1, if not discussed during Build/Release

MUST-HAVES FOR 5.0 (I know it looks like a long list, but my sense is that most of these things are in pretty good shape):
  • plone
    • (literally hundreds of tickets, may continue to descope individual pages to 5.1)
  • autoboxing
    • CAS-9208 C++ implementation of the new autoboxing algorithm
    • CAS-10111 (CAS-10060) Re-order (auto)mask generation call sequence
  • mstransform issues with averaging
    • CAS-9853 mstransform (regridms=True) implementation of correct gridding (JDM)
  • simulations (Andrew has been testing Remy's new features for quite some time, I believe it's almost done)
    • CAS-8592, CAS-3458 (as well as 9 related documentation tickets)
  • VI/VB2 validation: tclean
    • CAS-8588 segfault when imaging multiple ms in tclean if columns don't match (just need to final check and close)
    • CAS-10086 tclean with vi2 is now more memory hungry (just need to final check and close)
    • CAS-10050 tclean testing ticket with 10 subtickets:
      • CAS-10231 multiscale mosaic will no longer stop on threshold (currently "open", possibly not a bug)
      • CAS-10191 polarization (Andrew)
      • CAS-10192 single dish imaging (Catarina)
      • CAS-10193 self cal (has been validated in mfs, not cube... probably okay to slip to 5.1?)
      • CAS-10194 basic testing
      • CAS-10195 VLA/awproject (asked Claire and Steve for help, but nothing yet) (overlaps with CAS-9493?)
      • CAS-10196 stopping/restarting/startmodel (okay to slip to 5.1?)
      • CAS-10197 model column (Remy)
      • CAS-10198 VLASS (asked Claire for help)
      • CAS-10206 MMSes and lists of MSes (Rachel)
  • VI/VB2 validation: calibration
    • CAS-10118 ALMA polarization calibration tests for VIVB2 changes (Andrew, waiting on George's okay)
    • CAS-10200 testing new iterator: manual calibration path (Rachel)
  • MSSelector testing
    • CAS-8050 Remove obsolete MS selection framework MSSelector from ms tool (Tom)
    • CAS-8283 MSContinuumSubtractor::subtract (Tom)
  • miscellaneous (some will be finished if there is time/effort, otherwise pushed to 5.1)
    • CAS-9196 AW projection with MS + MFS + mosaic (marked for 5.0, no reply from Steve)
    • CAS-7086 Migrate visstat to VI/VB2, new statistics framework (BrianM and Martin Pokorny, almost done I think)
    • CAS-2140 plotms produces distorted UV plot (Amanda)
    • CAS-10228 at tool in CASA 5.x returns a spectrum that is shifted by 0.5 channel from the request (Todd)
    • CAS-9661 CASA 5.0 GUI testing (Jeremy)
    • CAS-9864 MSTRANSFORM floating point exception when combining spw's and channel averaging (done, just needs doc updates; Enrique)
    • CAS-9518 Reference frequency in the image coordinate system (tclean with mtmfs) (tried to get EricG to test this)

Architecture

HPC

  • We attended parts of the Pipeline Working group meeting in Garching last week. Based on several discussions with Dirk M. and Stewart, we plan to do:
    • run parallel tests on tclean using the mfs mode, which should work without convergence issues;
    • verify how to run the pipeline in parallel when invoked with pipelineMakeRequest and then the PPR (use-case from Felix);
    • verify a failure (probably a bug) that Stewart found when trying to run sessions in parallel that called flagdata. The issue is that flagdata is internally parallelised, but the sessions approach would try to fork jobs in MPIServers, but currently it can only do this from an MPIClient. His use-case for these sessions come from the SD pipeline, which will have many ASDMs to be processed all together.

Development

Pipeline

AOB

Developer Reports

Thursday Meeting
  • Sanjay Bhatnagar
  • Sandra Castro
    • Working on CAS-10142: add support in flagdata to take the exposure time into account when flagging a timerange. This led me to using the git framework in a different way. I need to use a branch from a fork in casacore. Should I add this information to the Plone git page?
    • Discussions about Dave's new implementation of statwt2 and the integration into mstransform.
  • Lindsey Davis
  • Pam Ford
    • Continued work on overplotting atmospheric transmission curve CAS-9053
  • Enrique Garcia
    • Visit to Socorro. Working and discussing about a number of topics:
      • Extending the VI/VB2 to cope with TVIs that need to access several SPWs simultanously.
      • AsyncIO support in VI/VB2 and its impact on mstransform.
      • Integration of StatWtTVI in mstransform.
      • Removal of spw parsing in all frequency TVIs.
  • Bob Garwood
  • Kumar Golap
  • Jeff Kern
  • David Mehringer
  • George Moellenbrock
    • CAS-9975: tec_maps now makes plot disk file---Ready to Validate (Brian Kent)
    • CAS-10139: New rerefant task, with refantmode='strict' (cf traditional, called 'flex') that insists on a single specifed refant; also available in gaincal. Nearly ready for validation.
    • Various VI2-related meetings with ESO colleagues visiting Socorro.
  • Dirk Petry
  • Martin Pokorny
  • Federico M Pouzols
    • Several discussions in Socorro.
    • CAS-9089: automatic generation of antenna position corrections from web service / ALMA TMCDB
      • Requires the Python package suds (for SOAP queries). Is it ok to add it as a dependency for CASA 5.1?
      • Still need to understand several points in order to do the right calculations of position corrections for gencal.
  • Urvashi Rao
  • Darrell Schiebel
    • crash reporter command line flag, message fix, and enable logic
    • documentation download/URL
    • cmake issues
  • Ville Suoranta
  • Takahiro Tsutsumi
    • (CAS-10210) Investigated and fixed the segfault in automask. It was caused by recursive calls in the “connected-components (pixels)” code. Rewrote it without the recursion.
    • Merged help documentation update done in 5.0 to master
    • Looked into CAS-10231 a bit, appeared to be a user error

Friday NAOJ Meeting
  • Kanako Sugimoto
  • Wataru Kawasaki
  • Masaya Kuniyoshi
  • Takeshi Nakazato
  • Renaud Miel
Topic revision: r12 - 2017-06-29, JaredCrossley
This site is powered by FoswikiCopyright © by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding NRAO Public Wiki? Send feedback