Monday Morning Meeting: January 30, 2017

• DIAL-IN NUMBERS & PASSCODES:
• IP: 192.33.117.12##8110
• Phone: (434) 817-6524

Attendance

• Socorro: Jeff, Lindsey, George, Juergen, Martin, Pam, Tak, Kumar
• CV: Andy, Darrell, Jen, Ville, Dave
• Garching: Sandra, Justo, Fredrico, Enrique, Dirk
• SCO:

News / Meetings / Visitors

• git Part 2 - Tuesday 1600 UTC / 9am MT / 11am ET, Socorro SO-317; Charlottesville ER-230, Video 192.33.117.12##8108; Audio only 434-817-6523
• NAASC Scientist Search over next 3 weeks.

Build, Release

• Another memory fix for ALMA pipeline, will need to see.

Verification Testing

• Draft plone documentation about the test infrastructure soon to be operational under bamboo: https://casa.nrao.edu/casadocs/stable/casa-processes/casa-test-framework-primer-1
  • Akeem has reviewed it and provided input, I'm giving Darrell and Ville a chance to provide feedback before calling it final
  • Answers JIRA ticket: https://open-jira.nrao.edu/browse/CAS-9381
• Any chance for Jeff and/or Anand to review my task list mapped to developers for test writing?

Validation Testing

• Autoboxing Testing
• Plone Testing

Architecture

• Warnings Summary
  • code/mstransform/TVI/UVContSubTVI.cc:143:30: warning: comparison between signed and unsigned integer expressions [-Wsign-compare]
  • code/synthesis/TransformMachines/CFBuffer.cc:221:27: warning: unused parameter 'miscInfo' [-Wunused-parameter]
  • code/singledish/Filler/DataAccumulator.h:466:98: warning: 'casa::sdfiller::DataChunk::get_num_pol_' will be initialized after pcorr_type_

HPC

• mpi4casa framework:
  • CAS-9287 finally resolved (Open MPI 2.0.1, IPython and atexit. Thanks Darrell).
  • CAS-9426 ready to verify (MPI get filtered casa dict).
• ALMA pipeline testing ( CAS-9519):
  • The imaging pipeline failed in one dataset because the image is not square (imsize=[294,120]. Created CAS-9587.
  • The calibration pipeline ran successfully in parallel on several datasets. Some of them will be included in the ESO Jenkins to run automatically.
• EVLA pipeline testing
  • Run pipeline in parallel on Band C data. Both serial and parallel runs give the same error coming from applycal. It fails because intent SYSTEM_CONFIGURATION included in selection doesn't exist. I suggested to Brian and Lindsey that perhaps using an error handler (similar to the one installed in flagdata) would make sense in applycal.
  • Run in parallel on a LONG_RUN dataset. It didn't give any errors and it didn't finish the processing. It stopped while running applycal. Serial version also gives the same applycal error.
  • Run also on band K, but the it's also affected by applycal error, despite what Juergen said. Will wait for a fix to this.
  • Memory monitoring of the pipeline CAS-9120
    • Still collecting data in serial mode

Development

• [pford] With code compiled in Debug mode using new third-party packages on RH6, I would get a seg fault in gdb when setting a breakpoint or running a backtrace on a core file! (did not happen with the latest tarball though) Then I tried /opt/rh/devtoolset-4/root/usr/bin/gdb and it worked, don't know why. YMMV

Pipeline

• First version of edit image list task implemented, for VLASS support
• First version of circular polarization task implemented, VLASS support
• Serial / HPC testing, various CASA 5.0 issues, library version, CASA version handling, user switches handling
• Assorted CASA 4.7.1 / Pipeline C4R2 bug fixes, including one major memory problem

AOB

Developer Reports

Monday Meeting

• Sanjay Bhatnagar
• Sandra Castro
  • HPC as above
  • Implemented minor feature in CAS-7619: flagdata mode='rflag' and action='calculate' should return timedev/freqdev dictionary
  • Fixed the code that creates MMS for some tests, which got broken by the latest changes in casa.
  • Had issues with the latest prerelease packages that have the pipeline built-in ( CAS-9588 and CAS-9572). In one, the pipeline requested a minimum version of CASA, which was not the one in the tarball. In another tarball the pipeline had an import error. In the tarball 112, building with the trunk of the pipeline showed a different error. Using the latest prerelease package 116, I got another error coming from init_pipeline.py pipeline.initcli().
• Lindsey Davis
  • Changed pipeline scipy.lib.blas imports to pipeline.linalg.blas imports
  • Investigated problems user defined casa switches, e.g. --testpipe in latest casa tarball
  • Investigated new casa version determination and comparison routines in latest tarball
  • Followed up on test data acquisition issues with Rachel, e.g.updated ALMA SBSummary and ExecBlock tables
  • Reviewed pipeline processing request and results schemas for Cycle 5
• Pam Ford
  • Prepare git talk, answer questions
  • Restarting plotms with procmgr - CAS-9432
  • Plotting same axes with diff datacolumns (e.g. Amp:corrected vs Amp:data) - CAS-9482
• Enrique Garcia
  • Started working on CAS-9441 with the help of Justo. Feedback sent to Pam.
• Bob Garwood
• Kumar Golap
• Justo Gonzalez
• Jeff Kern
  • LSST Commissioning Review
• David Mehringer
• George Moellenbrock
  • VI2 in solve (CAS-9097): Established basic numerical agreement with old VI framework, except in edge cases when correlation-dependent flags and low baseline count (expected, but needs a bit more verification). Also, there is potentially a problem with the channel-averaging layer in non-trivially flagged cases. The combine='spw' (or 'field') case requires a bit more casacore surgery in MSIter and BaseTableIter, and is very close.
  • More discussions with Takeshi on SD w/ VI2.
  • git -- Good discussions w/ Pam in wake of her presentation
  • Not as much plone doc as I would have liked...
• Dirk Petry
• Martin Pokorny
• Federico M Pouzols
  • HPC as above
  • Started work on CAS-9473: new mode in flagdata to flag all integrations in which a specified antenna is flagged
• Urvashi Rao
  • Documentation for imager.
• Darrell Schiebel
  • resolved plotants et al. issue
  • proposed alternative bamboo workflow
  • generation of intel compiler based distro
• Ville Suoranta
  • Bamboo testing and fixes
  • Tweaked Subgit translation settings
  • Enabled ccache for packaging builds
  • Updated Carta packaging script
• Takahiro Tsutsumi
  • automask
    • Fixed a bug that the pixel mask in residual image was not copied when creating a working copy of residual inside automask code. This probably fixes the issue that the image statistics was off in the mosaic.
    • Worked on a new pruneRegions code. Wrote the connected components labeling using the depth-first search algorithm. Also ‘pruning’ part is also mostly done, just need to plug into the main automask code.
  • CAS-9538 - Perley & Butler 2017 - need it for 5.0?

Friday NAOJ Meeting

• Kanako Sugimoto
• Wataru Kawasaki
• Masaya Kuniyoshi
• Takeshi Nakazato
• Renaud Miel