Monday Morning Meeting DATE

  • DIAL-IN NUMBERS & PASSCODES:
  • USA Toll Free Number: 866-901-8266
  • USA Toll Number: +1-203-566-3863
  • PARTICIPANT PASSCODE: 725760

Attendance

  • Socorro: Jeff, Lindsey, Pam, Jim, Tak, Juergen, Susan, Sanjay, Urvashi, Martin, George
  • CV: Andy, Ville, Akeem, Mark, Darrell
  • Garching: Dirk, Julian, Justo, Sandra
  • SCO:

News / Meetings / Visitors

  • Peter Quinn, Andreas Wicenec visit
  • CASA Users Survey
    • Reliability: what to do
    • Forum: How do we get more information to the users.

Build, Release, Testing

  • Last things before branch: Linux build, wvrgcal on OSX.
  • User Testing:
    • Lack of user testing, need mstransform, plotms
  • plotms performance checked in, pipeline can follow up
  • Mark to provide svn output

Architecture

  • Technical Forums:
  • casacore suggestion management,

HPC

  • Latest minutes are in https://safe.nrao.edu/wiki/bin/view/HPC/Apr0915
  • Latest tests with EVLA pipeline running on a very large MS still shows "Exception Reported: Table::lock cannot sync table, another process changed the number of columns". Running on a smaller MS does not show this problem.
  • Latest tests with ALMA pipeline shows that it is more beneficial to run each EB on a separated node, simultaneously, having each node run with MPI using many cores. See test details in here: https://safe.nrao.edu/wiki/bin/view/HPC/3EBSSeparatedALMAPipeline
  • Latest tests using a cycle2 reduction script using ALMA MS still shows that the tile caching commit does not fix the MMS cases.

Development

Pipeline

  • Development: web log infrastructure upgraded to handle multiple ALMA observing sessions, ASDMs more gracefully
  • EVLA commissioning:flux scale / model column issue, handling cross-hands selection in plots
  • HPC testing: flag data efficiency in pipeline and CASA
  • Plotms testing: TBD with version later than 4.4.131 (last Friday)

AOB

Developer Reports

Monday Meeting
  • Sanjay Bhatnagar
  • Sandra Castro
    • Verified test failures in OSX 10.10. Made ParallelDataHelper more robust for the case when a simple_cluster fails to be created due to lack of resources.
    • Moved the location of hidden parameters in mstransform and partition to avoid clash with positions of other parameters, as reported by a helpdesk user.
  • Lindsey Davis
    • pipeline flagdata usage analysis (with Brian Kent), improved hif_fluxcalflag efficiency
    • input to EVLA commissioning( with Brian Kent), flagdata calls, setjy calls, calibrator imaging
    • new pipeline web log infrastructure testing
    • plotms testing, speed issue not resolved as of last Friday
    • Verified the MultiFile implementation that previously failed due to the limit in number of open files. It is fixed for this case of running the ALMA pipeline with 3 EBs, creating a Multi-MS each with 42 subMSs
    • pipeline manifest file discussions with Kana and Takeshi
    • observing modes meeting input
  • Pam Ford
    • CAS-7412 - Implemented Justo's new method MSTransformIteratorFactory::getVisBufferStructure() in plotms for estimating memory requirements of non-, channel-, and time-averaged data with better performance. Implemented plain VI2 for other averaging with countChunks method as before.
    • Investigated open files with write-lock (CAS-7428). Now all files are closed in plotms after cache is loaded, except those opened by getVisBufferStructure(). Sent email to Justo.
  • Kumar Golap
    • Lots of testing and discussion about tile cache etc
    • started working on speeding up convolution function calculations faster for mosaics for CAS-3532 and CAS-6794
  • Justo Gonz├ílez Villalba
    • Fixed antenna reindexing problem reported for SPW separation
    • Fixed iPython-cluster security issues and system command backwards incompatibilities reported for MAC 10.10
    • Work in progress setting up developer env. in MAC 10.10 to investigate seg. fault in split
    • Tested VI/VB fixes from George in the MMS context, the problem is still present, sent a report to George/Jim with the test results so that they can reproduce them.
  • Jim Jacobs
  • Jeff Kern
  • Susan Loveland
    • CARTA:
      • Finished basic custom toolbar (No Search)
      • Writing selenium tests for Commands
    • Viewer: CAS-7439 (Help not working)
  • David Mehringer
    • Fixed broken UVFITS-related test failures.
    • (CAS-7087) Parent ticket for UVFits Bug Fixes for 4.4 Cycle: Updated with official version of Eric's memo (had been draft).
    • (CAS-7328) imregrid does not rotate beam in J2000-GALACTIC conversion: Discussion with Juergen re: precision issues or possible direction coordinate conversion issues in CASA.
    • Much discussion about a Linux->OSX client so I can actually get a development environment on which to debug OSX test failures. Tracy installed tigerVNC on my machine which allows me to connect to razor, but I still need an account to proceed farther. Josh said he needed to talk to Darrell, which is the last update I had about that.
    • (CAS-7408) immoments crashes with underscores in mask names: kicked back to Juergen since I have no idea what else I can do with this ticket.
    • (CAS-5053) ms.getscansummary() fails to list all spws of a scan: Worked with Catarina to help her test this.
    • Discussed with Jim my role as coordinator on a couple of casacore modules.
    • (CAS-4283) implement ia.continuumsub()/imcontsub using ImageProfileFitter: completed implementation, have been working on improving performance.
    • [JIRA] (CAS-6971) imfit logger messages should indicate noise scheme being assumed: worked with Catarina on helping her test this.
    • Became aware of imagetest failure on Saturday, first on my to do list today.
  • George Moellenbrock
    • More work/testing on CAS-7440---making the tile cache full-spw-width.
    • CAS-7453/SCOPS-1710 -- Effects related to Tsys calibration sampling, etc. Coments on these tickets pending.
    • Some EVLA polarization user support (Kurtz, et al.)
  • Dirk Petry
    • (CAS-7416) Perform ALMA HPC tests with realistic conditions and setup - still working on finalising the test scripts
    • (CAS-7424) improve leap second message - done
    • (CAS-5689) spectral fitting results seem off - testing and found additional tester
    • (CAS-7225) mosaic clean in alma-m100-analysis-regression fails on Fedora Core 20 - seems resolved
    • (CAS-7421) Make the distances parameter of im.fixvis available in task fixvis (and make it work) - found that underlying code is very old and little tested and doesn't seem to be easily resuscitated
    • (CAS-7227) better reporting of tabular spectral axes - added main fix
    • working on getting my new OSX 10.10 machine work as test platform (in order to find reason for wvrgcal failure on 10.10)
    • alma-m100-analysis regression now automatically run on each new test build both on EL6 and FC20 (and pass recently)
  • Martin Pokorny
  • Urvashi Rau
    • Added imager weighting calcs (with gather/scatter) for parallel runs. Plus some other minor code changes.
    • Second talk/discussion/emails about Imager features for HPC folk. Discussion/emails with JK/MP about iteration control and interaction for the refactored Imager. Some user-support emails.
  • Darrell Schiebel
  • Ville Suoranta
  • Julian Taylor
    • fixed simple_cluster not respecting cpuset/affinity on gnu/linux
    • finer grained GIL locking for plotting to avoid stalling MPI heartbeat thread (CAS-7383)
  • Takahiro Tsutsumi
    • cube tclean
    • makemask bug fixes in 'expand' mode after feedback from a tester (CAS-7302)
Friday NAOJ Meeting
  • Kanako Sugimoto
    • new SsS Erik Muller
    • completed porting asap to 10.10 ( CAS-7383)
    • completed CASA 4.5 planning and ticket generation
    • Prepared review presentation of CASA 4.4 SD developments
  • Wataru Kawasaki
    • CAS-7001: committed updates to the trunk (r32913). developer testing needed for blfunc='variable' mode.
    • CAS-6999 user testing: strange feature in cspline baseline fitting reported. found a bug in libsakura and commited a fix (r1346). libsakura.so needs to be updated for CASA4.4.
  • Masaya Kuniyoshi
    • adding more unite tests for CAS-6999
    • verified Weight_Spectrum[i,j]=1/(sigma[i])**2 X 1/FPARAM[i,j]**2 (George, Thanks!) [CAS-6995]
    • committed test_tsdcal after replacing dobt=True with wtmode='nyq' in initweights due to the new interface.
    • installation of CASA 4.3.1 on local machines at NAOJ for casa users after validation including unite&regression tests.
  • Takeshi Nakazato

-- KanaSugimoto - 2015-04-09
Topic revision: r22 - 2015-04-13, PamFord
This site is powered by FoswikiCopyright © by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding NRAO Public Wiki? Send feedback