Notes Concerning Changes

Decisions

  • with respect to python packages, CASAtools only depends on numpy
  • tests for CASAtools and CASAtasks may notrequire a window manager (i.e. X11 Display or Quartz session)
    • tests to fix:
      1. test_flagcmd
  • no dynamically loaded storage managers -- all needed Table storage managers should be compiled in and initialized at startup

Intentional CASA Changes

'taskinit.py' no longer exists
  • there is no grab-bag from which tasks (and other python files) include a potpourri of symbols

Tool Synonyms No Longer Exist
  • previously 'rgtool' and 'regionmanager' referred to the same tool; now the name specified in the tool XML file is used everywhere (e.g. in this case 'regionmanager')

The public development interface for CASA is only the tasks and tools

CASA should start and work in serial mode without mpi4casa being available
  • previously mpi4casa and all of its dependencies were required for casa to start in any environment, now casa should start even if openmpi and mpi4casa have not been installed on the system

Changes to XML

  • <any> with limittypeattribute
    • <any limittypes="string stringArray" type="variant"/> becomes:
      • <type>string</type>
      • <type>stringArray</type>
  • multiple <type> elements automatically implies a variant or <any>
  • the parameter types doubleArray, intArray, etc. now mean numpy ndarrray and the corresponding doubleVec, intVec, etc. types imply a python list or a C++ vector
  • multiple default values for sub-parameters (based upon some parameter flag) are no longer supported, the defaults are included directly in the python signature and in the typical case there is no inp / go layer to mediate the various layers of defaults
  • <type> elements have precedence over the type attribute in <param>
  • with the old generation, one <output> parameter along with a <returns type="void"> was interpreted as a return type like the single output parameter; with the new generation, this should be written as <returns>doubleArray</returns> (where doubleArray was the type of the single output parameter)

What Is Complete (mostly)

The tools that make up CASAtools all build and all tests pass. Missing tools (as identified by non-deprecated tasks in CASAtasks) are added as they are encountered.

These tasks have been ported and their tests seem to pass:

  • applycal
  • calstat
  • concat
  • cvel
  • cvel2
  • flagcmd
  • flagmanager
  • imhistory
  • imstat
  • listobs
  • listpartition
  • setjy
  • slsearch
  • smoothcal
  • specfit
  • splattotable
  • spxfit
  • vishead

These tests have been ported, but their tests fail (these require maintainer examination):

  • bandpass
  • flagdata
  • imhead
  • immath
  • immoments
  • mstransform
  • partition
  • specflux
  • split
  • imsmooth
  • sdbaseline
  • sdfixscan
  • sdimaging
  • sdsmooth

These test have been ported, but have no unit test:

  • specsmooth
  • uvsub
  • tclean
  • importuvfits
  • blcal

Things to Fix

Tools which have no unittests:
  • imagemetadata
  • mstransformer
  • imager
  • synthesisimager
  • synthesisdeconvolver
  • spectralline
  • vpmanager
  • singledishms
  • vlafiller
  • atcafiller
  • sakura
  • simulator

Tasks which have no unittests:
  • accor
  • blcal
  • fringefit
  • importuvfits
  • uvsub
  • specsmooth
  • imsubimage
  • delmod
  • clearcal
  • importfits
  • exportfits
  • ft
  • imdev
  • importgmrt
  • impv
  • imrebin
  • imreframe
  • listsdm
  • polcal
  • rerefant
  • rmfit
  • rmtables
  • feather
  • simobserve
  • widebandpbcor

Tasks which have tests but not in python unittest format
  • listcal
  • polcal
  • accum

Tasks whose tests are fundamentally broken%endColor%
  • imval

Tasks which should have been removed (but still persist):
  • caltabconvert

Configuration and Build Issues
  • add m4 discovery for wcslib
  • add m4 discovery for rpfits
  • add m4 check for swig version (/usr/bin/swig on RHEL7 is too old)

Tasks Should Not Use Other Tasks to Implement Testing (unless there are insurmountable difficulties avoiding it)
  • test_imhead uses test_immoments
  • test_split uses partition, listobs, flagdata, importasdm, flagcmd
  • test_flagcmd uses flagdata but test_flagdata also uses flagcmd
  • test_flagdata uses flagcmd, flagdata, mstransform, setjy
  • test_mstransform uses cvel, cvel2, listpartition, listobs, setjy, flagdata, split, applycal, importasdm, flagcmd
  • test_partition uses flagdata, flagmanager, split, setjy, listpartition, listobs
  • task_listpartition and its test program both use partitionhelper
  • task_partition and its test program both use partitionhelper

Duplicates

  • imager and synthesisimager
  • cvel and cvel2
  • clean, tclean, and tclean2

Removed Tasks

These are the tasks which will not be included in CASA 6.0:

  • task_autoclean
  • task_boxit
  • task_clearplot
  • task_csvclean
  • task_importevla
  • task_mosaic
  • task_oldhanningsmooth
  • task_oldpartition
  • task_oldplotants
  • task_plotuv
  • task_plotxy
  • task_ssoflux
  • task_visstatold
  • task_widefield
statwt2 is renamed statwt

Wayward Tasks

Tasks that do not appear in tasklist but are available for use:
  • importnro
  • msuvbin

Backlog

While casadocs will guide porting tasks for use with CASAtasks, the tasklist provide the list of all known tasks. These are the tasks which have not yet been ported to CASAtasks according to tasklist [strike through implies not included in CASA6, red implies postponed until the other tasks are working]:
  • accor
  • accum
  • asdmsummary
  • browsetable
  • caltabconvert
  • clean
  • clearcal
  • clearstat
  • conjugatevis
  • deconvolve
  • delmod
  • exportasdm
  • exportfits
  • exportuvfits
  • feather
  • fixplanets
  • fixvis
  • fluxscale
  • fringefit
  • ft
  • gaincal
  • gencal
  • hanningsmooth
  • imcollapse
  • imcontsub
  • imdev
  • imfit
  • impbcor
  • importasap
  • importasdm
  • importatca
  • importfits
  • importfitsidi
  • importgmrt
  • importmiriad
  • importvla
  • impv
  • imrebin
  • imreframe
  • imregrid
  • imsmooth
  • imsubimage
  • imtrans
  • imval
  • imview
  • initweights
  • listcal
  • listfits
  • listhistory
  • listsdm
  • listvis
  • makemask
  • msview
  • oldsplit (not needed?)
  • plotants
  • plotbandpass
  • plotcal
  • plotms
  • plotprofilemap
  • plotweather
  • polcal
  • predictcomp
  • rerefant
  • rmfit
  • rmtables
  • sdbaseline
  • sdcal
  • sdfit
  • sdfixscan
  • sdgaincal
  • sdimaging
  • sdsidebandsplit
  • sdsmooth
  • simalma
  • simanalyze
  • simobserve
  • specflux
  • statwt
  • tclean2
  • testconcat
  • tsdimaging
  • uvcontsub
  • uvcontsub3
  • uvmodelfit
  • virtualconcat
  • visstat
  • widebandpbcor
  • wvrgcal

Questions

  • spectralline tool (i.e. Splatalog) is the only thing that depends on SQLite, perhaps this should be separable?

To Do

  1. Understand what CASA CLI functionality would be lost if we were to use only ipython. This is the list of known functionality or features that would be lost:
    • unified collection of packages etc. that are known to work well together (e.g. a matplotlib version that works with everything else)
    • "inp" for setting and checking parameters
    • "go" filling task arguments from global variables
    • protection against overwriting system variables (e.g. "applycal = 90")
    • unique CASA command line prompt
  2. Explore CASA CLI as a module so that CASA can be started like "python -m CASA" or (even though the "CASA.start( )" will create a new environment without access to any setup done in the ellipsis):
                     import CASA
                     ...
                     CASA.start( ) 
  3. segmentation violation on linux with -fopenmp see valgrind output for task_setjy

Python 3 Changes

  • integer division: "1 / 2" becomes "1 // 2"
  • range should be used instead of xrange
  • parens must be used to invoke print, e.g. print("hello world!")
  • fetched dictionary keys and values cannot be indexed, e.g. (mydict.keys( ))[0]; must instead convert to a list, e.g. list(mydict.keys( ))[0]
  • dictionary objects do not have has_key function, use "mykey" in mydict instead
  • "print >> fd, pstr" becomes "print(pstr,file=fd)"
  • "except AssertionError, e:" becomes "except AssertionError as e:"

Bad Python Habits

  • use os.path.join(datapath,nep) instead of datapath+nep
-- DarrellSchiebel - 2018-03-09
Topic attachments
I Attachment Action Size Date Who Comment
segv-setjy-with-openmp.txttxt segv-setjy-with-openmp.txt manage 326 K 2018-04-18 - 20:26 DarrellSchiebel valgrind analysis of setjy segv with openmp enabled
Topic revision: r59 - 2018-09-09, DarrellSchiebel
This site is powered by FoswikiCopyright © by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding NRAO Public Wiki? Send feedback