Using plotbandpass

Return to directory of Todd's CASA extensions

Plotbandpass has been installed as a task in casapy as of version 4.1.0 (released in May 2013), but it does not have all the very latest features of the analysisUtils version (mainly the groupByBaseband option). Please see the casaguide page if you are using the casa task, as some of the argument names are different here in the analysisUtils version.

The utility called plotbandpass provides extended capabilities beyond those of casa's plotcal. This utility was written to facilitate rapid visualization of ALMA's bandpass and spectral Tsys measurements in SV and Early Science data, but it can be used on EVLA data as well. It is distributed with the analysisUtils python package which provides numerous other python extensions callable from within casa. The command-line options and some relevant examples are given below. The current list of options can always be obtained by giving option: help=True. It has been tested on dual-pol ALMA data, and single, dual and full-pol EVLA data. Regarding the showatm=True option, since most people reducing ALMA data do not load the ASDM_CALWVR table into the ms, the atmospheric transmission curve will be computed by default for 1.0 mm, but you can set the PWV to any value using the pwv option. If you want the fully correct curve for ALMA, then copy the CalWVR.xml file from your ASDM to the present working directory and specify pwv='auto' (which is now the default). Most of the regular casa list syntax is accepted (commas and tildes) but negation via exclamation point is not supported, nor is channel selection within an spw. Let me know if you want either of these capabilities. If you find any problems or unexpected behavior, please report: 1) the version number you are using, which is printed as the first line of output; 2) the exact command; and 3) a pointer to the dataset.

CASA <1>: import analysisUtils as au
CASA <2>: au.plotbandpass(help=True)
$Id: Plotbandpass.txt,v 1.49 2014/04/07 03:58:50 ToddHunter Exp $
Usage: plotbandpass(caltable='', antenna='', field='', spw='', yaxis='amp',
   xaxis='chan', figfile='', plotrange=[0,0,0,0], help=False, caltable2='',
   overlay='', showflagged=False, timeranges='', buildpdf=False, caltable3='',
   markersize=3, density=108, interactive=True, showpoints='auto',
   showlines='auto', subplot='22', zoom='', poln='', showatm=False, pwv='auto',
   gs='gs', convert='convert', chanrange='', debug=False, vm='',
   solutionTimeThresholdSeconds=60.0, phase='', ms='', showtsky=False,
   showfdm=False, showatmfield='', lo1=None, showimage=False,
   showatmPoints=False, parentms='', pdftk='pdftk', channeldiff=False
    edge=8, resample=1, vis='', platformingThreshold=10, platformingSigma=5,
    basebands=None, showBasebandNumber=False, scans='',
    figfileSequential=False, groupByBaseband=False, cleanup=False, 
    caltable2amplitudeOffset=0, xcolor='b', ycolor='g', regression=False)
 antenna: must be ID (int or string or list), or a single antenna name or list
 atm: overlay the atmospheric transmission curve for the weather and elevation
 basebands: show only spws from the specified baseband or list of basebands (default:None=all)
 buildpdf: True/False, if True and figfile is set, assemble pngs into a pdf
 caltable: a bandpass table, of type B or BPOLY
 caltable2: a second cal table, of type BPOLY or B, to overlay on a B table
 caltable2amplitudeOffset: constant value to add to caltable2 before displaying it
 caltable3: a third cal table, of type BPOLY, to overlay on the first two
 chanrange: set xrange ('5~100') over which to autoscale y-axis for xaxis='freq'
 channeldiff: set to value > 0 (sigma) to plot derivatives of amplitude
 cleanup: remove pngs after making pdf when buildpdf=True
 convert: full path for convert command (in case it's not found)
 density: dpi to use in creating PNGs and PDFs (default=108)
 edge: the number of edge channels to ignore in finding outliers (for channeldiff>0)
 field: must be an ID, source name, or list thereof; can use trailing *: 'J*'
 figfile: the base_name of the png files to save: base_name.antX.spwY.png
 figfileSequential: naming scheme, False: name by spw/antenna (default), True: figfile.1.png, figfile.2.png, etc."
 groupByBaseband: group spws for display by baseband
 gs: full path for ghostscript command (in case it's not found)
 help: print this message
 interactive: if False, then figfile will run to completion automatically
 lo1: specify the LO1 setting (in GHz) for the observation (used if showimage=T)
 overlay: 'antenna' or 'time', make 1 plot with different items in colors
 showflagged:  show the values of data, even if flagged
 markersize: size of points (default=3)
 ms: name of the ms for this table, in case it does not match the string in the caltable
 parentms: name of the parent ms, in case the ms has been previously split
 pdftk: full path for pdftk command (in case it's not found)
 phase: the y-axis limits to use for phase plots when yaxis='both'
 platformingThreshold: the value in percentage to use to detect platforming (for channeldiff>0)
 plotrange: define axis limits: [x0,x1,y0,y1] where 0,0 means auto
 poln: polarizations to plot (e.g. 'XX','YY','RR','LL' or '' for both)
 pwv: define the pwv to use for the showatm option: 'auto' or value in mm
 regression: Boolean, if True, then don't show version/date on plot
 resample: channel expansion factor to use when computing MAD of derivative (for channeldiff>0)
 scans: show only solutions for the specified scans (int, list, or comma-delimited string)
 showatm: compute and overlay the atmospheric transmission curve
 showatmfield: for overlay='time', use first observation of this fieldID or name
 showatmPoints: draw atmospheric curve with points instead of a line
 showBasebandNumber: put the BBC_NO in the title of each plot, and force all in showfdm
 showfdm: when showing TDM spws with xaxis='freq', draw locations of FDM spws
 showimage: also show the atmospheric curve for the image sideband (in black)
 showtsky: compute and overlay the sky temperature curve instead of transmission
 showlines: draw lines connecting the data (default=T for amp, F for phase)
 showpoints: draw points for the data (default=F for amp, T for phase)
 solutionTimeThresholdSeconds: consider 2 solutions simultaneous if within this interval (default=60)
 spw: must be single ID or list or range (e.g. 0~4, not the original ID)
 subplot: 11,22,32 or 42 for RowsxColumns (default=22), any 3rd digit is ignored
 timeranges: show only these timeranges, the first timerange being 0
 vis: alternative way to specify the name of the ms (when necessary)
 vm: the result from ValueMapping('my.ms'), or as returned from a previous call to plotbandpass
 xaxis: 'chan' or 'freq'
 xcolor: color for XX polarization points (default = 'blue')
 yaxis: 'amp', 'tsys', 'phase', or 'both' amp&phase == 'ap'; append 'db' for dB
 ycolor: color for YY polarization points (default = 'green')
 zoom: 'intersect' will zoom to overlap region of caltable with caltable2
Return value: the ValueMapping structure for the specified ms.

Display a simple bandpass solution

au.plotbandpass(caltable='bandpass.bcal',yaxis='both',antenna='DV07',figfile='bandpass')
Here we have specified a single antenna, but we could give a list, or leave it blank for all antennas. We have asked for both phase and amplitude which are shown in adjacent plots. Note how the ouput figfile name will be modified to include antenna number, spw, and timerange of the first panel for your convenience: bandpass.DV07.spw0.t1.png. The default style for amplitude is lines, and phase is points, but this can be changed with showpoints or showlines.

Maybe you prefer to see your phase plots on a common -180 to +180 degree scale. You can do that with plotrange and by limiting yourself to showing only the phase plots.

au.plotbandpass(caltable='bandpass.bcal',yaxis='phase',antenna='DV07',figfile='bandpass_phase',plotrange=[0,0,-180,180])

The first 3 plots are from the 3 different time ranges. The fourth plot is the first timerange on the second spw (spw=1). Notice how the upper half of the channels are missing. This is because they were flagged prior to the solution. The x-axis range reflects both flagged and unflagged data to visually remind you of this situation.

Display different (fixed) y-axis ranges for amplitude and phase

Alternatively, if you want to show both amplitude and phase plots on the same page, you can set a fixed amplitude range with the plotrange parameter, and a different fixed phase range with phase=[-180,180].
au.plotbandpass(caltable='bandpass.bcal',yaxis='both',antenna='DV07',figfile='bandpass_phase',plotrange=[0,0,0,5],phase=[-180,180])

Display a bandpass solution with a BPOLY solution overlaid

You can overlay a BPOLY solution on top of the regular B solution to see how well the polynomial reproduces the shape.

au.plotbandpass('bandpass_b_skipspw19high.bcal',overlay='',yaxis='phase',field='0',xaxis='freq',caltable2='bandpass_bpoly_skipspw19high.bcal',showpoints=True,spw=0,figfile='bpoly_overlay',antenna='4~7')

Note: You cannot overlay two BPOLY solutions, unless you are overlaying three solutions and caltable is a B solution and caltable2 and caltable3 are BPOLY solutions.

Overlay a bandpass solution with a smoothed version created by smoothbandpass

au.plotbandpass(caltable='bandpass.bcal',caltable2='bandpass.bcal_smooth',xaxis='freq',yaxis='both',overlay='',figfile='smooth',antenna='DV07')

The "c1" and "c2" labels in the legend correspond to caltable and caltable2:

Display a Tsys solution for individual times for one field

A Tsys table is simply a bandpass table with amplitude values set to the system temperature and phase set to zero. There is no inherent way in the header to differentiate a Tsys table from another bandpass table. Therefore, in order to decide whether to make the yaxis label "Amplitude" or "Tsys", we simply look for the letters "tsys" in the caltable name.

au.plotbandpass('X3c1.tsys',yaxis='amp',field='2',xaxis='chan',figfile='tsys_chan.png')

Overlay Tsys values for all antennas, one plot per timerange

A quick way to check whether any antenna shows an anomalous system temperature is to overlay all of them together.

 au.plotbandpass('X3c1.tsys',overlay='antenna',yaxis='amp',field='2',xaxis='chan',figfile='tsys_antenna_overlay.png',subplot=11)

Overlay multiple spectral windows

There are two new overlay options: 'spw' and 'baseband'. Using overlay='spw' will plot all spws from a given baseband on the same frame. Using overlay='baseband' will show all spws, regardless of which baseband they originate in. If one uses xaxis='chan', they will be on top of one another, but if one uses xaxis='freq' they will be spaced according to their frequency. For datasets where the spws are spaced adjacent to one another, like typical VLA continuum setups, the result is very useful. Here is an example of overlay='spw' on VLA Q-band data taken with the 3-bit digitizers:

au.plotbandpass(caltable='bandpass.b',spw='2~65',xaxis='freq',yaxis='amp', figfile='bandpass.b_amp.png',overlay='spw', interactive=False,subplot=42) 

Below is an example of overlay='baseband', which is also useful for datasets from telescopes that do not have a column for the baseband number. This plot was made for one sideband of an SMA dataset which has 24 spws of bandwidth 104 MHz and spaced by ~82 MHz for a total bandwidth of 1.968 GHz. If 24 spws is too many to view at once, then the number of spws shown can be limited by also specifying the spw parameter. Then one can make successive calls to plotbandpass(overlay='baseband' or 'spw') with the next series of spws selected.

Overlay Tsys values for all timeranges, one plot per antenna

au.plotbandpass('X3c1.tsys',overlay='time',yaxis='amp',field='2',xaxis='chan',figfile='tsys_overlay.png',subplot=11)

Overlay Tsys from TDM and interpolated result for FDM

au.plotbandpass(caltable='X3c1.tsys.fdm',caltable2='X3c1.tsys',yaxis='amp',field='1',xaxis='freq',figfile='tsys.png')

The "c1" and "c2" labels in the legend correspond to caltable and caltable2:

If you would like to zoom in on the range of intersection, use the zoom='intersect' option. The example below shows that the interpolation from TDM onto FDM is working as expected:
 au.plotbandpass(caltable='X3c1.tsys.fdm',caltable2='X3c1.tsys',yaxis='amp',field='1',xaxis='freq',figfile='tsyszoom.png',zoom='intersect')

Compute and overlay the atmospheric transmission cure

Sometimes it is instructive to compare the atmospheric transmission curve to the bandpass solution, for example to confirm that absorption features are due to ozone lines. Because Juan Pardo's atm program is contained in casa, the curve can be computed with showatm=True option. By default it will use 1.0 mm as the PWV, however you can change this by entering a different value with pwv='2.0' or by setting pwv='auto', which will use the actual values of PWV as measured by the WVRs via TelCal and written into the ASDM_CALWVR table. In this case, the median value from all antennas is used. In all cases, the transmission curve uses the airmass of the source for the scan closest to the midpoint of the solution interval. The curve is drawn such that 0% is at the bottom and 100% is at the top of the y-axis, regardless of what plotrange is set for the amplitude.

au.plotbandpass('bandpass2_60m.bcal',showatm=True,xaxis='freq',figfile='showatm.png',subplot=11,plotrange=[230.6,232.5,0,1.2])

Clipping off the edge channels

Sometimes the edge of the band shows extreme amplitude values. In this case, it is useful to limit the xrange of the display. If you are using xaxis='chan', you can simply use plotrange=[8,120,0,0] to clip 8 channels off each edge. But if you are using xaxis='freq', you must set chanrange='8~120' and leave plotrange at the default. Below is an example of the full display, and the display with chanrange set.

 au.plotbandpass('bandpass2_20m.bcal',xaxis='freq',overlay='time',showatm=T,antenna='',plotrange=[0,0,0,0],subplot=11,figfile='limitedrange',chanrange='10~120')

Plotting the normalized derivative

One can display the derivative of the caltable spectra instead of the spectra themselves. For amplitude and Tsys, the derivative is normalized by the median of the spectrum and the units displayed are percent per channel. Here is an example on a Tsys table, first plotting the value, then its derivative. The edges and the ozone line show up strongly in the derivative:

au.plotbandpass('uid___A002_X54d35d_X761.ms.tsys',xaxis='freq',interactive=False,figfile='X761_tsys_ant0',antenna='0',overlay='',chanrange='5~122',timeranges=0)
au.plotbandpass('uid___A002_X54d35d_X761.ms.tsys',xaxis='freq',interactive=False,figfile='X761_tsys_deriv_ant0',antenna='0',overlay='',chanrange='',timeranges=0,channeldiff=6,plotrange=[0,0,-10,10])

How to speed up the performance on large datasets in casa versions < 4.0

Note: In casa 4.0, the msmd tool was introduced, which is vastly faster than ValueMapping. Plotbandpass now uses that tool to gather the information it needs.

In older versions of casa, the plotbandpass function first needs to call ValueMapping to learn about the dataset and create a structure for future reference. If you plan to run the function many times on the same dataset, then you can store the return value of the structure from ValueMapping, like so:
vm = au.ValueMapping(your_ms)
Now, you can pass the result back to plotbandpass via the optional 'vm' argument to avoid having to run ValueMapping inside of plotbandpass each time. It will do a sanity check to see if the ms name in the vm structure matches the ms name stored inside the caltable.
# All subsequent calls to plotbandpass will now run faster, as long as the vm=vm argument is passed.
au.plotbandpass(caltable='bandpass.bcal', vm=vm)
-- ToddHunter - 2011-10-07

Return to directory of Todd's CASA extensions

Topic attachments
I Attachment ActionSorted descending Size Date Who Comment
X761_tsys_ant0.DA41.spw9.t1.pngpng X761_tsys_ant0.DA41.spw9.t1.png manage 127 K 2013-02-20 - 10:20 ToddHunter  
X761_tsys_deriv_ant0.DA41.spw9.t1.pngpng X761_tsys_deriv_ant0.DA41.spw9.t1.png manage 125 K 2013-02-20 - 10:21 ToddHunter  
bandpass.DV07.spw0.t1.pngpng bandpass.DV07.spw0.t1.png manage 380 K 2011-10-07 - 14:49 ToddHunter  
bandpass.b_amp.ea01.spw50.t0.pngpng bandpass.b_amp.ea01.spw50.t0.png manage 189 K 2013-08-06 - 22:13 ToddHunter example of overlay='spw' on 3-bit VLA data
bandpass_phase.DV07.spw0.t1.pngpng bandpass_phase.DV07.spw0.t1.png manage 126 K 2011-10-07 - 14:52 ToddHunter  
bpoly_overlay.DV09.spw0.t1.pngpng bpoly_overlay.DV09.spw0.t1.png manage 218 K 2011-10-07 - 14:53 ToddHunter  
fullrange.DV02.spw0.t1.pngpng fullrange.DV02.spw0.t1.png manage 156 K 2011-10-18 - 23:28 ToddHunter  
limitedrange.DV02.spw0.t1.pngpng limitedrange.DV02.spw0.t1.png manage 330 K 2011-10-18 - 23:22 ToddHunter  
showatm.DV02.spw1.t3.pngpng showatm.DV02.spw1.t3.png manage 144 K 2011-10-16 - 14:06 ToddHunter  
sma.1.spw00.t0.pngpng sma.1.spw00.t0.png manage 127 K 2013-08-07 - 11:27 ToddHunter  
smooth.DV07.spw0.t1.pngpng smooth.DV07.spw0.t1.png manage 345 K 2011-10-07 - 14:47 ToddHunter  
tsys.DV04.spw0.t1.pngpng tsys.DV04.spw0.t1.png manage 248 K 2011-10-07 - 14:55 ToddHunter  
tsys_antenna_overlay.spw0.t1.pngpng tsys_antenna_overlay.spw0.t1.png manage 224 K 2011-10-07 - 14:57 ToddHunter  
tsys_chan.DV04.spw0.t1.pngpng tsys_chan.DV04.spw0.t1.png manage 204 K 2011-10-07 - 15:00 ToddHunter  
tsys_overlay.DV04.spw0.pngpng tsys_overlay.DV04.spw0.png manage 349 K 2011-10-07 - 12:07 ToddHunter  
tsyszoom.DV04.spw0.t1.pngpng tsyszoom.DV04.spw0.t1.png manage 218 K 2011-10-07 - 15:05 ToddHunter  
Topic revision: r49 - 2014-04-06, ToddHunter
This site is powered by FoswikiCopyright © by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding NRAO Public Wiki? Send feedback