bandpassPreAverage

Return to directory of Todd's CASA extensions

Computes the minimum required pre-averaging bandwidth to achieve the specified accuracy level in the bandpass solution for the specified field. The Tsys values are taken from the SYSCAL table, so no tsys table is required. Bands 3, 4, 6, 7, 8, 9 and 10 are supported, using sensitivities scaled appropriately from the online ALMA sensitivity calculator. See also gaincalSNR.

Usage:

au.bandpassPreAverage(vis, tsysTable='', flux=None, field=None, intent='BANDPASS', 
                       spw=None,  requiredSnr=50, computeUnflaggedAntennas=False, 
                       verbose=False, tsysChanTol=1, atmcal=None, minChannelsInSolution=8)

Required inputs:
  • vis: the measurement set

Optional inputs:
  • tsysTable: if it is specified, it is only used for tsysspwmap; otherwise, an internal mapping is performed which is probably less prone to failures
  • flux: the flux density value to use in units of Jy (default=query ALMA catalog for flux on date of vis observation)
  • field: the field ID or name to use (default=the first BANDPASS calibrator)
  • intent: the intent to use (instead of BANDPASS)
  • spw: the spw to use (default=all with BANDPASS intent and OBSERVE_TARGET intent, if there are any OBSERVE_TARGET spws present)
  • requiredSnr: required by the science case (the default of 50 means 2% accuracy)
  • computeUnflaggedAntennas: if True, then run the agent flagger (af) tool to find the number of unflagged antennas (<90% flagged)
  • tsysChanTol: only used if tsysTable is specified
  • atmcal: an instance of class Atmcal; if None, then run Atmcal if False, then first try the faster, less robust method
  • minChannelsInSolution: warn if required value is smaller than this

Returns:
  • a dictionary keyed by spw ID and other useful values (see example below)
    • A new key added in May 2015 is 'solintRecommended', which is the largest value among the smallestCommonSolintEvenlyDivisible determined for each spw, but ignoring those spws whose channel widths are wider than the solintRecommended (i.e. typically TDM spws). This single value can be used in the bandpass task to produce solutions that have an equal number of channels per solution interval across each spw. It may do more smoothing than strictly necessary on the narrowest windows in exchange for not producing solutions on the wider spws that are not evenly-divisible into those wider spws.

Examples:

Here are examples of applying the script to Cycle 1 and 2 datasets recently reduced by Crystal. The results are generally quite consistent with expectations.

Example 1: Band 9 TDM

The script recommends 5 channels, while the data reducer used 4. The plots of the 4-channel bandpass solution show peak to peak phase of about 7.5 deg, suggesting an rms of 1.5 deg, or a S/N of 57/1.5 = 38. Thus, to get S/N=50 would require 7 channels, which is close to the script's prediction.
CASA <2>: au.bandpassPreAverage('uid___A002_X87b480_Xa11.ms')
Choosing field = 0 = J1924-2914
Choosing spws =  [17 19 21 23]
Analyzing the Atmospheric Cal scans may take a minute...
scans = [3, 6, 9, 11]
times = ['02:40:02', '02:47:51', '02:53:02', '02:55:14']
Atmcal class initialization completed in 9 seconds
Median tsys value for spw 9 = 683.5 K
Median tsys value for spw 11 = 699.3 K
Median tsys value for spw 13 = 696.7 K
Median tsys value for spw 15 = 657.7 K
spw 17 (2000MHz) requires solint='73.47MHz' (25 channels in solution) to reach S/N=50
spw 19 (2000MHz) requires solint='71.97MHz' (25 channels in solution) to reach S/N=50
spw 21 (2000MHz) requires solint='73.90MHz' (25 channels in solution) to reach S/N=50
spw 23 (2000MHz) requires solint='68.28MHz' (25 channels in solution) to reach S/N=50
  Out[2]: 
{17: {'Tsys_spw': 9,
      'band': 9,
      'bandwidth': 2000000000.0,
      'chanwidth_Hz': 15625000.0,
      'fluxDensity_Jy': 1.7268998605566674,
      'frequency_Hz': 680422797796.6355,
      'medianTsys': 615.04434204101562,
      'minutes': 5.2539999802907307,
      'sensitivity_mJy': 74.894777504960757,
      'snrPerChannel': 23.057680629898176,
      'solint': '73.473173MHz',
      'solintChannels': '5',
      'totalChannels': 128},
 19: {'Tsys_spw': 11,
      'band': 9,
      'bandwidth': 2000000000.0,
      'chanwidth_Hz': 15625000.0,
      'fluxDensity_Jy': 1.7240159780780551,
      'frequency_Hz': 682177927949.52966,
      'medianTsys': 607.6998291015625,
      'minutes': 5.2539999802907307,
      'sensitivity_mJy': 74.000426277117143,
      'snrPerChannel': 23.297379012682331,
      'solint': '71.969072MHz',
      'solintChannels': '5',
      'totalChannels': 128},
 21: {'Tsys_spw': 13,
      'band': 9,
      'bandwidth': 2000000000.0,
      'chanwidth_Hz': 15625000.0,
      'fluxDensity_Jy': 1.7330687749746487,
      'frequency_Hz': 676693297796.6355,
      'medianTsys': 619.0185546875,
      'minutes': 5.2539999802907307,
      'sensitivity_mJy': 75.378722729020708,
      'snrPerChannel': 22.99148502694673,
      'solint': '73.896861MHz',
      'solintChannels': '5',
      'totalChannels': 128},
 23: {'Tsys_spw': 15,
      'band': 9,
      'bandwidth': 2000000000.0,
      'chanwidth_Hz': 15625000.0,
      'fluxDensity_Jy': 1.7298570063829515,
      'frequency_Hz': 678630797796.6355,
      'medianTsys': 593.90887451171875,
      'minutes': 5.2539999802907307,
      'sensitivity_mJy': 72.321083171285451,
      'snrPerChannel': 23.919124694052957,
      'solint': '68.276220MHz',
      'solintChannels': '5',
      'totalChannels': 128},
 'field': 0,
 'fieldname': 'J1924-2914',
 'totalAntennas': 32,
 'vis': 'uid___A002_X87b480_Xa11.ms'}

Example 2: Band 9 FDM (1875 MHz)

For 3 of the 4 spws, 4 channels is recommended by the script. The data reducer used 20. The plot of the full-resolution bandpass solution has a peak-to-peak phase of 15 degrees, so an rms ~ 3 deg, thus S/N = 57/3 ~ 19. So, to get S/N=50 requires (50/19)**2 = 6.9 channels, again close to the prediction.
CASA <2>: au.bandpassPreAverage('uid___A002_X8602fa_X208d.ms',computeUnflaggedAntennas=True)
Running the af tool, this can take a few minutes.
DA48 total=426812900  flagged=426702452  fraction=0.999741226
DA65 total=426812900  flagged=426751708  fraction=0.999856630
Choosing field = 0 = J0522-3627
Choosing spws =  [17 19 21 23]
Analyzing the Atmospheric Cal scans may take a minute...
scans = [3, 6, 9, 11, 13, 15, 18, 20, 22, 24, 27, 29, 31, 33]
times = ['12:34:06', '12:47:17', '12:52:44', '12:55:28', '12:58:09', '13:01:23', '13:09:23', '13:12:05', '13:14:46', '13:17:59', '13:25:49', '13:28:30', '13:31:13', '13:34:36']
Atmcal class initialization completed in 13 seconds
Median tsys value for spw  9 = 456.5 K
Median tsys value for spw 11 = 426.8 K
Median tsys value for spw 13 = 468.2 K
Median tsys value for spw 15 = 371.1 K
spw 17 (1875MHz) requires solint='1.80MHz' (960 channels in solution) to reach S/N=50
spw 19 (1875MHz) requires solint='1.58MHz' (960 channels in solution) to reach S/N=50
spw 21 (1875MHz) requires solint='1.90MHz' (960 channels in solution) to reach S/N=50
spw 23 (1875MHz) requires solint='1.19MHz' (1280 channels in solution) to reach S/N=50
  Out[2]: 
{17: {'Tsys_spw': 9,
      'band': 9,
      'bandwidth': 1875000000.0,
      'chanwidth_Hz': 488281.25,
      'fluxDensity_Jy': 6.0715893874837885,
      'frequency_Hz': 662250281329.81836,
      'medianTsys': 456.50286102294922,
      'minutes': 10.560000006357829,
      'sensitivity_mJy': 233.38799096972596,
      'snrPerChannel': 26.015003438079074,
      'solint': '1.803692MHz',
      'solintChannels': '4',
      'totalChannels': 3840},
 19: {'Tsys_spw': 11,
      'band': 9,
      'bandwidth': 1875000000.0,
      'chanwidth_Hz': 488281.25,
      'fluxDensity_Jy': 6.0699726416871513,
      'frequency_Hz': 664120197416.81567,
      'medianTsys': 426.81556701660156,
      'minutes': 10.560000006357829,
      'sensitivity_mJy': 218.21030316741286,
      'snrPerChannel': 27.817076249741586,
      'solint': '1.577564MHz',
      'solintChannels': '4',
      'totalChannels': 3840},
 21: {'Tsys_spw': 13,
      'band': 9,
      'bandwidth': 1875000000.0,
      'chanwidth_Hz': 488281.25,
      'fluxDensity_Jy': 6.0676041026494145,
      'frequency_Hz': 666870074015.31555,
      'medianTsys': 468.18775939941406,
      'minutes': 10.560000006357829,
      'sensitivity_mJy': 239.36191838533406,
      'snrPerChannel': 25.349078682104942,
      'solint': '1.899703MHz',
      'solintChannels': '4',
      'totalChannels': 3840},
 23: {'Tsys_spw': 15,
      'band': 9,
      'bandwidth': 1875000000.0,
      'chanwidth_Hz': 488281.25,
      'fluxDensity_Jy': 6.0821030002952901,
      'frequency_Hz': 650229993479.15125,
      'medianTsys': 371.11605834960938,
      'minutes': 10.560000006357829,
      'sensitivity_mJy': 189.73381915007249,
      'snrPerChannel': 32.055977303047747,
      'solint': '1.187933MHz',
      'solintChannels': '3',
      'totalChannels': 3840},
 'field': 0,
 'fieldname': 'J0522-3627',
 'unflaggedAntennas': 29,
 'vis': 'uid___A002_X8602fa_X208d.ms'}

Example 3: Band 3 FDM (58 MHz)

25 channels is recommended by the script. The data reducer used 16. The peak-to-peak phase scatter in the 16-channel solint solution is ~8 degrees, so rms~1.6 and S/N=57/1.6 =35.6. To get S/N=50 requires 16*(50/35.6)**2 = 32 channels. This is close to the prediction.
CASA <4>: au.bandpassPreAverage('uid___A002_X74fea5_X1311.ms')
Choosing field = 0 = J1107-4449
Choosing spws =  [33 35 37 39]
Analyzing the Atmospheric Cal scans may take a minute...
scans = [3, 6, 9, 12, 14, 19, 21, 25]
times = ['10:22:03', '10:24:44', '10:32:37', '10:38:11', '10:39:50', '10:56:18', '11:05:30', '11:14:28']
Atmcal class initialization completed in 10 seconds
Median tsys value for spw 25 = 57.8 K
Median tsys value for spw 27 = 58.6 K
Median tsys value for spw 29 = 58.4 K
Median tsys value for spw 31 = 58.6 K
spw 33 (  59MHz) requires solint='0.38MHz' (153 channels in solution) to reach S/N=50
spw 35 (  59MHz) requires solint='0.37MHz' (153 channels in solution) to reach S/N=50
spw 37 (  59MHz) requires solint='0.38MHz' (147 channels in solution) to reach S/N=50
spw 39 (  59MHz) requires solint='0.37MHz' (153 channels in solution) to reach S/N=50
  Out[35]: 
{33: {'Tsys_spw': 25,
      'band': 3,
      'bandwidth': 58593750.0,
      'chanwidth_Hz': 15258.7890625,
      'fluxDensity_Jy': 1.4892239105283507,
      'frequency_Hz': 89184960592.038651,
      'medianTsys': 57.790096282958984,
      'minutes': 5.24160000483195,
      'sensitivity_mJy': 148.07921992795264,
      'snrPerChannel': 10.056940543399179,
      'solint': '0.377162MHz',
      'solintChannels': '25',
      'totalChannels': 3840},
 35: {'Tsys_spw': 27,
      'band': 3,
      'bandwidth': 58593750.0,
      'chanwidth_Hz': 15258.7890625,
      'fluxDensity_Jy': 1.5187627643236232,
      'frequency_Hz': 86750809734.063309,
      'medianTsys': 58.556076049804688,
      'minutes': 5.24160000483195,
      'sensitivity_mJy': 150.04193834599678,
      'snrPerChannel': 10.122255024600893,
      'solint': '0.372311MHz',
      'solintChannels': '25',
      'totalChannels': 3840},
 37: {'Tsys_spw': 29,
      'band': 3,
      'bandwidth': 58593750.0,
      'chanwidth_Hz': 15258.7890625,
      'fluxDensity_Jy': 1.4958597130746831,
      'frequency_Hz': 88628045308.734863,
      'medianTsys': 58.401288986206055,
      'minutes': 5.24160000483195,
      'sensitivity_mJy': 149.64531765998171,
      'snrPerChannel': 9.9960341991689816,
      'solint': '0.381772MHz',
      'solintChannels': '26',
      'totalChannels': 3840},
 39: {'Tsys_spw': 31,
      'band': 3,
      'bandwidth': 58593750.0,
      'chanwidth_Hz': 15258.7890625,
      'fluxDensity_Jy': 1.5291361369527268,
      'frequency_Hz': 85922804440.804184,
      'medianTsys': 58.598058700561523,
      'minutes': 5.24160000483195,
      'sensitivity_mJy': 150.14951314815906,
      'snrPerChannel': 10.184089877426786,
      'solint': '0.367803MHz',
      'solintChannels': '25',
      'totalChannels': 3840},
 'field': 0,
 'fieldname': 'J1107-4449',
 'totalAntennas': 27,
 'vis': 'uid___A002_X74fea5_X1311.ms'}
-- ToddHunter - 2014-09-10
Topic revision: r8 - 2015-05-13, ToddHunter
This site is powered by FoswikiCopyright © by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding NRAO Public Wiki? Send feedback