Monday Morning Meeting May 5th

  • DIAL-IN NUMBERS & PASSCODES:
  • USA Toll Free Number: 866-901-8266
  • USA Toll Number: +1-203-566-3863
  • PARTICIPANT PASSCODE: 725760

Attendance

  • Socorro: Jeff, Lindsey, Sandra, Juergen, Tak, Urvashi, George, Jim, Sanjay
  • CV: Dave, Darrell, Andy, Mark
  • Garching: Dirk, Sandra
  • SCO:

  • (Kumar on leave.)

News / Meetings / Visitors

  • Dirk Petry Visit: 12-23 May
  • Imaging Synthesis Workshop May 13-20
  • CASA User Committee Meeting: End of May.
    • Jeff to present. Juergen gets a pass this round.
    • Not CUC, old format.
  • CASA Admin Calendar

Developer Reports

Monday Meeting
  • Sanjay Bhatnagar
    • Checked-in MSSelection changes to time and SPW selection as commented code.
    • Helping Andy to test these changes (made a build for him with these changes)
    • MSSelection related discussions with SD group.
      • Updated the MSSelection document to (hopefully) clarify somethings.
    • Started debugging the W-term part of AW-Projection code (this used to work, but doesn't now).
    • Usual impromptu discussions with Jeff, Urvashi.
  • Sandra Castro
    • Resuming work on CAS-5833. Working on the design for this implementation. Steve hasn't answered my questions, so I am working with what I know. I think I will need to create a new datahandler class inside the flagger. My idea is to have a class to handle sub-tables, FlagTableHandler. I wanted to use the TableProxy (TaQL) directly for the selection of antenna, spw, time, but this will not work, as I would have to interpret every syntax, similar to what MSSelection already does. Second plan: use MSSelection for the selections, but then there is no class in MeasurementSets to handle the SYSPOWER table. I am back to the beginning. NOTE: There is no iterator for sub-tables either.
    • Created CAS-6493: Add proper conversion of numpy types across Python/XML/C++. Until this is not implemented automatically, I have added work-arounds in split2/mstransform to convert numpy types from the chanbin/width parameters.
    • Fixed the two mstransform failing tests in "CAS-6408 test_mstransform.py failed in OS X package CASA-intel64b-10.7-29047-2.dmg:" There were two missing commits in that branch which led to the failures.
  • Lindsey Davis: working on blocker tickets to full ALMA interferometry pipeline acceptance mostly web log and plotting, debugging and testing pipeline ASDM / CASA / pipeline problems
  • Kumar Golap: (On Vacation)
  • Justo Gonz├ílez Villalba (last 3 weeks)
    • New MPI-based parallel processing framework is integrated in ParallelTaskHelper, and thus working with all the tasks
    • I have added several new features in MPICommandClient
      • Execution modes: Blocking / Non-Blocking / Single target server / Multiple target servers / Undefined target server
      • Parameters can now be specified in a dictionary, and it is not necessary to convert them to strings (all Python pickable objects are supported)
      • When a server is in timeout condition the corresponding commands are marked so that the client response is not blocked globally
    • Added checks at startup time to guarantee that the provided MPI implementation will work as expected
      • Thread-Safety level
      • MPI version
    • Created a set of more than 50 unit tests to cover all MPICommandclient/Server cases, including server timeout detection and recovery
  • Jim Jacobs
    • Began Carta (Alma visualization project) GUI design
  • Jeff Kern:
    • PEP Process
    • Started looking at long term planning for CASA
  • Susan Loveland: Fixed bugs in 4.2.1, 4.2.2, and trunk for viewer, plotms, and feather. Worked on CARTA user interface and floundered around trying to build Skel2.
  • David Mehringer: much of the week spent on refactoring code for ImageAnalysis::getFreqProfile() rewrite, minor imregrid modification to remove repeated error messages, updated imfit/ia.fitcomponents() docs per user request, modified imfit/ia.fitcomponents() to use null value (empty string) of noisefwhm by default, investigated a reported defect in ia.decimate() which I couldn't reproduce
  • George Moellenbrock
    • Completed regression adjustments for new weight calculation regime, including nchan correction in applycal
    • Began investigating new weights consequences for split and mstransform (some changes related to nchan correction and other effects are likely)
    • Further analysis of the 'bpdip' effect CAS-4071, which seems to persist in some cases despite previous advice to use phase-only for pre-bandbass gaincal.
    • Some additional testing of the new ionosphere corrections
    • Studied new laptop options
  • Dirk Petry
    • No major CASA activities last week.
  • Scott Rankin
  • Urvashi Rao : (a) Reworking internals of imagestore classes for better handling of reference images (for facet imaging) and to enable image allocation only 'on demand' without any extra setup logic elsewhere in the code (b) 'tclean' task : tested TT's spectral frame code, added ability to image data or corrected cols, added optional auto imagename increments, etc.
  • Darrell Schiebel: RHEL6 build, casacore includes in RPM, update of casainfo, purchase of OSX build host, sleep/awake crash of DBus VO service, addition of fetch via libcurl, viewer OSX defects
  • Rob Selina: Tracking 4.2.1 progress. Tracking 4.2.2 progress. Setting up PC to update documentation folders (VNC to gygax, SVN checkout, etc). Naming convention requirements (below).
  • Takahiro Tsutsumi
    • Fixed some issues in cube/cubedata modes in the refactored imaging code
    • Fixed a bug in setjy output dictionary and updated setjy unit tests to be more consistent with current task interface
    • Re-tested setjy with Crystal's data set on different casapy builds just to be sure the reported issue was non-reproducible. Also looked at VisModelData code a bit for source_id=-1 handling issue but the fix was not trivial (for me)
    • Started looking into im.setjy code to refactor inefficient part of the code that is doing per-spw processing
Friday NAOJ Meeting
  • Kanako Sugimoto
  • Wataru Kawasaki
  • Takeshi Nakazato: Implemented a function to fill weight in sdsave based on interferomerty tasks/tools. Replied to testers comments for some "Ready To Test" tickets.
  • Shinnosuke Kawakami: verification for test data of tsdstat unittest, investigation of XSLT file related to async dependencies of CASA

Build, Release, Testing

  • CASA 4.2.1 Status
    • EL Ready to go
    • OSX
      • Juergen completed testing of dmg on 10.7
      • Nathan testing dmg on 10.8.
      • Outstanding tickets are related to regression test failures. Top of the list are import failures, in progress debug by Scott and Michel.
      • Mark will ping Michel and Scott for an update.
  • CASA 4.2.2
    • Push milestones back 2 weeks. Merging on hold while Scott prioritizes 4.2.1 debugging.
    • Stable for pipeline development & testing around 5/23.
      • ACTION: Rob to update the schedule accordingly.
  • CASA 4.3 Test Packages
    • Importasdm issue needs to be checked and resolved. Likely a packaging issue. Impacts the pipeline.
  • CASA Package Naming Convention & Version Identification

Architecture

  • Technical Forums:
    • Next will be Justo - MPI - May 22nd
      • Big commit coming before then, including a manual describing changes implemented and future changes required.
      • Will distribute findings for coordination soon.
      • Will replace implementation of simple cluster.
    • Need to reschedule Lindsey's Pipeline presentation

Development

  • Timing results from Emmanuel (See e-mail from Jeff)
    • Use of OTF in AIPS saves time.
      • Getting corrected data column virtualized will help performance.
      • Factor of 3-4 improvement available in how we run through the data.
    • Import data penalty is larger than expected.
      • Lazy filler will provide a gain, but size of gain to be determined with a test.
    • Use case could be optimized to make certain steps faster.
    • Flagging can be completed on the fly to provide a performance boost too.
    • Parallelization and reducing the number of passes through the data are the 1st order implementation change.
    • George will send out an ASDM for a second use case showing the flagging performance concern.
  • 4.2.2
    • Setjy fix by Tak (or Kumar?) CAS-6164
      • Bump to 4.3 since Kumar is on vacation.
    • Casacore include fix by Darrell CAS-6202
    • CASA 4.2.2. Plan
    • Need to consider testing approach to Weights since it is a significant change.
      • Application of bandpass calibration will be the most significant case.
      • Splitting and channel averaging may also warrant inspection. Implementation is changing in these cases.
      • Mark will consider testing approach. Testing after the summer school. Seek input from Crystal.
  • 4.3
    • Imaging
      • Simulation broken due to changes and need repair (Remy?)
    • VO
    • MPI
    • Requiring Clarification

Pipeline

AOB

  • Use of global variables in python. Task treatment is different depending on whether you pass arguments.
    • Does not use globals at the moment if you pass without an argument.
    • Was a design choice. May need to be revisited as the tasking system is redesigned.

-- JeffKern - 2014-04-28
Topic revision: r21 - 2014-05-05, RobSelina
This site is powered by FoswikiCopyright © by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding NRAO Public Wiki? Send feedback