default parameter values and inline help for findContinuumCycle8.findContinuum:

img='', pbcube=None, psfcube=None, minbeamfrac=0.3, spw='', transition='',
baselineModeA='min', baselineModeB='min', sigmaCube=3,
nBaselineChannels=0.19, sigmaFindContinuum='auto', sigmaFindContinuumMode='auto',
verbose=False, png='', pngBasename=False, nanBufferChannels=1,
source='', useAbsoluteValue=True, trimChannels='auto',
percentile=20, continuumThreshold=None, narrow='auto',
separator=';', overwrite=True, titleText='', # 25
maxTrim=maxTrimDefault, maxTrimFraction=1.0,
meanSpectrumFile='', centralArcsec='auto', alternateDirectory='.',
plotAtmosphere='transmission', airmass=1.5, pwv=1.0,
channelFractionForSlopeRemoval=0.75, mask='',
invert=False, meanSpectrumMethod='mom0mom8jointMask', peakFilterFWHM=10, # 38
skyTempThreshold=1.2,
skyTransmissionThreshold=0.08, maxGroupsForSkyThreshold=5,
minBandwidthFractionForSkyThreshold=0.2, regressionTest=False,
quadraticFit=True, triangleFraction=0.83, maxMemory=-1, # 46
tdmSkyTempThreshold=0.65, negativeThresholdFactor=1.15,
vis='', singleContinuum=None, applyMaskToMask=False,
plotBaselinePoints=True, dropBaselineChannels=2.0,
madRatioUpperLimit=1.5, madRatioLowerLimit=1.15,
projectCode='', useIAGetProfile=True, useThresholdWithMask=False,
dpi=dpiDefault, normalizeByMAD='auto', returnSnrs=False, # 61
overwriteMoments=False,minPeakOverMadForSFCAdjustment=[19,19],
minPixelsInJointMask=3, userJointMask='', signalRatioTier1=0.967,
snrThreshold=23, mom0minsnr=MOM0MINSNR_DEFAULT, mom8minsnr=MOM8MINSNR_DEFAULT, overwriteMasks=True,
rmStatContQuadratic=False, quadraticNsigma=1.8, bidirectionalMaskPhase2=False,
makeMovie=False, returnAllContinuumBoolean=True, outdir='', allowBluePruning=True,
avoidance='', returnSigmaFindContinuum=False,
enableRejectNarrowInnerWindows=True, avoidExtremaInNoiseCalcForJointMask=False, # 81
buildMom8fc=True, momentdir='',
amendMaskIterations='auto', skipchan=1, # 85
checkIfMaskWouldHaveBeenAmended=False, fontsize=10, thresholdForSame=0.01, # 88
keepIntermediatePngs=True, returnWarnings=False, enableOnlyExtraMask=True,
useMomentDiff=True, smallBandwidthFraction=0.05, smallSpreadFraction=0.33,
skipAmendMask=False, useAnnulus=True, cubeSigmaThreshold=7.5,
npixThreshold=7, nbin=1

inline help
This task calls functions to:
1) compute a representative 'mean' spectrum of a dirty cube
2) find the continuum channels
3) plot the results and writes a text file with channel ranges and frequency ranges
It calls runFindContinuum up to 2 times.
If meanSpectrumMethod = 'mom0mom8jointMask', then it runs it a second
time with a reduced field size if it finds only one range of continuum
channels.

Please note that the channel ranges do not necessarily map back to
the visibility spectra in the measurement set(s). In order to produce
channel ranges that are guaranteed to map back correctly, then you must
supply a list of all desired measurement sets to the vis parameter. In
this case, the final line in the .dat file created will contain the
channel list in the correct frequency frame and order.

Returns:
--------
* A channel selection string suitable for the spw parameter of clean.
* The name of the final png produced.
* The aggregate bandwidth in continuum in GHz.
If returnAllContinuumBoolean = True, then it also returns a Boolean desribing
if it thinks the whole spectrum is continuum (True) or not (False)
Cycle 7 pipeline usage should set returnAllContinuumBoolean = True
If returnWarnings = True, then it also returns a list of warning strings,
which may be empty [], for tooLittleBandwidth and/or tooLittleSpread
If returnSnrs = True, then it also returns two more lists: mom0snrs and mom8snrs,
and the max baseline (floating point meters) inferred from the psf image
if returnSigmaFindContinuum = True (new in PL2020), then it also returns 7 more things:
1) the final value used for sigmaFindContinuum
2) the Boolean intersectionOfSelections
3) number of central ranges that were (or would have been) dropped, summed
over all runs of findContinuumChannels by the final run of runFindContinuum.
4) the result of amendMaskYesOrNo (if it was run; False otherwise)
5) the result of extraMaskYesOrNo (if it was run; False otherwise)
6) the result of extraMaskYesOrNo test 2 (if it was run; False otherwise)
7) the name of the finalMomDiff image (if amendMask=True) else the final mom8fc image
8) the path code (for amendMask)
9) the 4-letter code (for amendMask)

Required Inputs:
----------------
img: the image cube to operate upon (as opposed to specifying a
pre-existing meanSpectrumFile from a previous run)
--or--
meanSpectrumFile: an alternative pre-existing ASCII text file to use
for the mean spectrum, instead of operating on the img to create one.
Note: if you specify meanSpectrumFile and vis, you must also specify img, but
only its metadata will be used.
Parameters relevant only to meanSpectrumFile:
* invert: if True, then invert the sign of intensity and add the minimum

Optional inputs (see also General Parameters below):
---------------------------------------------------
spw: the spw name or number (integer or string integer) to put in the x-axis label;
also, it will be used to select the spw for which to generate topo channels for the
*.dat file if vis is also specified; if vis is specified and spw is
not, then it will search the name of the image for an spw number
and use it for generating the topo channel list.
transition: the name of the spectral transition (for the plot title)
avoidance: a CASA channel selection string to avoid selecting (for manual use)
meanSpectrumMethod: 'auto', 'mom0mom8jointMask', 'peakOverMad', 'peakOverRms',
'meanAboveThreshold', 'meanAboveThresholdOverRms', 'meanAboveThresholdOverMad',
where 'peak' refers to the peak in a spatially-smoothed version of cube
'auto' invokes 'meanAboveThreshold' unless sky temperature range
(max-min) from the atmospheric model is more than skyTempThreshold
in which case it invokes 'peakOverMad' instead
ALMA Cycle 5 used 'auto', while ALMA Cycle 6 will use 'mom0mom8jointMask'

Parameters relevant only to 'mom0mom8jointMask' (ALMA Cycle 6 onward)
---------------------------------------------------------------------
* minPeakOverMadForSFCAdjustment: a list of two values, first is for spws with
channels > 2500, second is for spws with fewer channels
* pbcube: the primary beam response for img, but only used by
meanSpectrumMethod='mom0mom8jointMask'. If not specified,
it will be searched for by changing file suffix from .residual to .pb

* psfcube: the psf response for img, but only used by
meanSpectrumMethod='mom0mom8jointMask'. If not specified,
it will be searched for by changing file suffix from .residual to .psf,
if still not found, then pruneMask will default to 6 pixels
* minbeamfrac: only used if psfcube is specified and meanSpectrumMethod='mom0mom8jointMask'
floating point value or 'auto' for 0.3 for 12m and 0.1 for 7m (maxBaseline<60m)
Set to zero to prevent pruning of the joint mask entirely; otherwise pruneMask
will use np.max([4,minbeamfrac*pixelsPerBeam])
* minPixelsInJointMask: if fewer than these pixels are found (after pruning), then
use all pixels above pb=0.3 (or equivalent higher value on mitigated images).
This option was essentially obseleted by the addition of pruneMask with the
knowledge of beamsize from psfcube. To avoid having any effect, it should be set
to < minbeamfrac*minPixelsPerBeam, which is typically 0.5*7=3.5; so use 3 or less.
* userJointMask: if specified, use this joint mask instead of computing one
(this option is meant for test purposes only, typically *.mask2_bi)
* normalizeByMAD: can be True, False or 'auto' (default);
if 'auto', then if atmospheric transmission varies enough across the spw,
set to True, otherwise set to False.
If True, then request normalization of the resulting mean spectrum by the
xmadm spectrum using an outer annulus and outside the joint mask. The
nominal annulus is the 0.2-0.3 pb response, but the range will
automatically be scaled upward with minimum value in the pb cube,
in order to preserve the fractional number of pixels contained in
a single field's 0.2-0.3 annulus.
* mom0minsnr: sets the threshold for the mom0 image (i.e. median + mom0minsnr*scaledMAD)
* mom8minsnr: sets the threshold for the mom8 image (i.e. median + mom8minsnr*scaledMAD)
* snrThreshold: if SNR is higher than this, then run a phase 2 mask calculation
* bidirectionalMaskPhase2: True=extend mask to negative values beyond threshold; False=Cycle6 behavior
(True was tried but gives worse results in some cases)
* rmStatContQuadratic: if True, then fit and remove a quadratic using the subset of channels that lack signal
* makeMovie: if True, then make a png for each quadratic fit as channels are
removed one-by-one (in the rmStatContQuadratic option)
* quadraticNsigma: the stopping threshold to use when ignoring outliers prior to
fitting the quadratic (in the rmStatContQuadratic option). Tto more quickly test
different values, be sure to set the meanSpectrumFile option and overwrite=False)
* allowBluePruning: use continuum range pruning heuristic added in Cycle 6 (default=True)
* amendMaskIterations (new in PL2020): if 'auto', then choose 0 for TDM and 3 for FDM;
if 0, then do not attempt any amendMask (i.e. mimic the Cycle 7 pipeline release)

if >=1, then when finished the initial findContinuum trial, create a mom8fc image and
mom0fc image scaled by nchan and chanwidth in kms, and subtract them to create momDiff
image. If the momDiff image has sufficient signal remaining in it, then expand the
joint mask, and repeat the findContinuum process with this expanded mask as the
userJointMask; if >=2, and there is still sufficient signal in the new momDiff image from
the second run of findContinuum, then check if further analysis of the extra portion of the
mask is necessary. If so, first check if intersecting the ranges of the first two rounds
eliminates the excess, otherwise create a new spectrum from it and pass it into a third
round of findContinuum with this as the meanSpectrum file. If >=3, and there is stil
sufficient signal in the new momDiff, image then set sigmaFindContinuumMode='autolower'
and sigmaFindContinuum=most recent value, and run a final round of findContinuum.
* skipAmendMask: if True, then skip the test for amend mask, and move on to onlyExtraMask
* useAnnulus: if amendMaskIterations>0, sets whether median/MAD in imageSNR be assessed in
annulus but outside the jointmask (True) or simply outside the joint mask (False)
* enableOnlyExtraMask: if False, then do not do the extraMask or onlyExtraMask iterations
* useMomentDiff: if False, use the mom8fc image to assess AmendMask decision instead of momDiff
* checkIfMaskWouldHaveBeenAmended (new in PL2020): if True, then if amendMaskIterations=0,
then it will still check to see if the mask would have been amended and return the
result (assuming that returnSigmaFindContinuum==True) -- for test purposes only

Parameters relevant only to 'auto' and 'mom0mom8jointMask':
-----------------------------------------------------------
* skyTempThreshold: rms value in Kelvin (for FDM spectra<1000MHz), above which
the meanSpectrumMethod is changed in 'auto' mode to peakOverMad; and above
which normalizeByMAD is set True in meanSpectrumMethod='mom0mom8jointMask'
when normalizeByMAD='auto' mode
* tdmSkyTempThreshold: rms value in Kelvin (for TDM and 1875MHz FDM spectra)
* skyTransmissionThreshold: threshold on (max-min)/mean, above which the
meanSpectrumMethod is changed in 'auto' mode to peakOverMad; and above
which normalizeByMAD is set True in meanSpectrumMethod='mom0mom8jointMask'
when normalizeByMAD='auto' mode
* overwriteMoments: if True, then overwrite any existing moment0 or moment8 image
* overwriteMasks: if True, then overwrite any existing moment0 or moment8 mask image
* rmStatContQuadratic: if True, then fit and remove quadratic after removing outliers
* outdir: directory to write the .mom0 and .mom8 images, .png, .dat and meanSpectrum
(default is same directory as the cube)
* avoidExtremaInNoiseCalcForJointMask (new in PL2020): experimental Boolean to avoid
pixels <5%ile and >95%ile in the chauvenet imstat of mom0 and mom8 images


Parameters relevant only to 'auto':
-----------------------------------
* triangleFraction: remove a triangle shape if the MAD of the dual-linfit residual
is less than this fraction*originalMAD; set this parameter to zero to turn it off.

Parameters relevant only to 'peakOverMad' or 'meanAboveThreshold*':
-------------------------------------------------------------------
* maxMemory: behave as if the machine has this many GB of memory
* nanBufferChannels: when removing or replacing NaNs, do this many extra channels
beyond their extent
* channelFractionForSlopeRemoval: if this many channels are initially selected,
then fit and remove a linear slope and re-identify continuum channels
Set to 1 to turn off.
* quadraticFit: if True, fit quadratic polynomial to the noise regions when
deemed appropriate; Otherwise, fit only a linear slope
* centralArcsec: radius of central box within which to compute the mean spectrum
default='auto' means start with whole field, then reduce to 1/10 if only
one window is found (unless mask is specified); or 'fwhm' for the central square
with the same radius as the PB FWHM; or a floating point value in arcseconds
* mask: a mask image preferably equal in shape to the parent image that is used to determine
the region to compute the noise (outside the mask, i.e. mask=0) and
(in the 'meanAboveThresholdMethod') the mean spectrum (inside the mask, i.e. mask=1).
The spatial union of all masked pixels in all channels is used as the mask for each channel.
Option 'auto' will look for the <filename>.mask that matches the <filename>.image
and if it finds it, it will use it; otherwise it will use no mask.
If the mask does not match in shape but is multi-channel, it will be regridded to match
and written out as <filename>.mask.regrid.
If it matches spatially but is single-channel, this channel will be used for all.
To convert a .crtf file to a mask, use:
makemask(mode='copy', inpimage=img, inpmask='chariklo.crtf', output=img+'.mask')
* applyMaskToMask: if True, apply the mask inside the user mask image to set its masked pixels to 0

Parameters relevant only to 'peakOverRms' and 'peakOverMad':
------------------------------------------------------------
* peakFilterFWHM: value (in pixels) to presmooth each plane with a Gaussian kernel of
this width. Set to 1 to not do any smoothing.
Default = 10 which is typically about 2 synthesized beams.


Parameters relevant only to 'meanAboveThreshold':
-------------------------------------------------
* useAbsoluteValue: passed to meanSpectrum, then avgOverCube -- take absolute value of
the cube before producing mean spectrum
* useIAGetProfile: if True, then for baselineMode='min', use ia.getprofile instead of
ia.getregion and subsequent arithmetic (faster, less memory exhaustive)
* continuumThreshold: if specified, begin by using only pixels above this intensity level
* sigmaCube: multiply this value by the MAD to get the threshold above
which a voxel is included in the mean spectrum.
* baselineModeA: 'min' or 'edge', method to define the baseline in meanSpectrum()
Parameters relevant to 'min':
-----------------------------
* percentile: control parameter for 'min'

Parameters relevant to 'edge':
* nBaselineChannels: if integer, then the number of channels to use in:
a) computing the mean spectrum with the 'meanAboveThreshold' methods.
b) computing the MAD of the lowest/highest channels in findContinuumChannels
if float, then the fraction of channels to use (i.e. the percentile)
default = 0.19, which is 24 channels (i.e. 12 on each side) of a TDM window

Parameters relevant only when amendMaskIterations > 0 (new for PL2020)
----------------------------------------------------------------------
* keepIntermediatePngs: if True, keep changing the png name to avoid overwriting them; only
the final one is returned
* thresholdForSame: fractional value used to determine if the residual in the mom8fc image
improved ('L'), worsened ('H') or stayed the same ('S')
* cubeSigmaThreshold: minimum cube SNR for onlyExtraMask to trigger
* npixThreshold: minimum momDiff pixels for onlyExtraMask to trigger

Parameters for function findContinuumChannels (which is called by all meanSpectrum heuristics):
-----------------------------------------------------------------------------------------------
baselineModeB: 'min' (default) or 'edge', method to define the baseline
Parameters only relevant if baselineModeB='min':
* dropBaselineChannels: percentage of extreme values to drop
nBaselineChannels: if integer, then the number of channels to use in computing
standard deviation/MAD of the baseline channels (i.e. the blue points in the plot)

if float, then it is the fraction of channels to use (i.e. the percentile)
sigmaFindContinuum: starting value for sigmaFindContinuum; passed to findContinuumChannels,
'auto' starts with value:
TDM: singleContinuum: 9, meanAboveThreshold: 4.5, peakOverMAD: 6.5,
mom0mom8jointMask: 7.2
FDM: meanAboveThreshold: 3.5, peakOverMAD: 6.0,
mom0mom8jointMask: nchan<750: 4.2 (strong lines) or 4.5 (weak lines)
nchan>=759: 2.6 (strong lines or 3.2 (weak lines)
and adjusts it depending on results
sigmaFindContinuumMode (new in PL2020): 'auto', 'autolower', 'fixed'
trimChannels: after doing best job of finding continuum, remove this many
channels from each edge of each block of channels found (for margin of safety)
If it is a float between 0..1, then trim this fraction of channels in each
group (rounding up). If it is 'auto' (default), use 0.1 but not more than
maxTrim channels, and not more than maxTrimFraction
narrow: the minimum number of channels that a group of channels must have to survive
if 0<narrow<1, then it is interpreted as the fraction of all
channels within identified blocks
if 'auto', then use int(ceil(log10(nchan)))
maxTrim: in trimChannels='auto', this is the max channels to trim per group for
TDM spws; it is automatically scaled upward for FDM spws.
maxTrimFraction: in trimChannels='auto', the max fraction of channels to trim per group
singleContinuum: if True, treat the cube as having come from a Single_Continuum setup;
For testing purpose. This option is overridden by the contents of vis (if specified).
negativeThresholdFactor: scale the nominal negative threshold by this factor (to
adjust the sensitivity to absorption features: smaller values=more sensitive)
signalRatioTier1: threshold for signalRatio, above which we desensitize the level to
consider line emission in order to avoid small differences in noise levels from
affecting the result (e.g. that occur between serial and parallel tclean cubes)
signalRatio=1 means: no lines seen, while closer to zero: more lines seen
enableRejectNarrowInnerWindows (new in PL2020): if True, then remove any inner groups
of channels that are narrower than both edge windows (when there are 3-15 groups)

General parameters:
-------------------
verbose: if True, then print additional information during processing
momentdir (new in PL2020): alternative directory to look for (and write) the moment 0&8 images
png: the name of the png to produce ('' yields default name)

pngBasename: if True, then remove the directory from img name before generating png name
source: the name of the source, to be shown in the title of the spectrum.
if None, then use imhead('object')
overwrite: if True, or ASCII file does not exist, then recalculate the mean spectrum
writing it to <img>.meanSpectrum
if False, then read the mean spectrum from the existing ASCII file
separator: the character to use to separate groups of channels in the string returned
titleText: default is img name and transition and the control parameter values
plotAtmosphere: '', 'tsky', or 'transmission'
airmass: for plot of atmospheric transmission
skipchan (newly-exposed in PL2020): used in making the spectral plot, the number of highest
and lowest intensity channels to avoid when constructing the y-axis range
pwv: in mm (for plot of atmospheric transmission)
vis: comma-delimited list or python list of measurement sets to use to convert channel
ranges to topocentric frequency ranges (for use in uvcontfit or uvcontsub)
plotBaselinePoints: if True, then plot the baseline-defining points as black dots
fontsize (newly-exposed in PL2020): size to use for channel ranges, axis labels and plot legend
dpi: dots per inch to use in writing the png (106 produces 861x649 pixels)
150 produces 1218x918
projectCode: a string to prepend to the title of the plot (useful for regression testing
where you can put the page number from the full PDF when running only a subset)
buildMom8fc (new in PL2020): if True, then when finished, generate "mom8fc" and "mom0fc"
images using selected channels
smallBandwidthFraction: threshold for returning a warning for too little bandwidth found
smallSpreadFraction: threshold for returning a warning for too little bandwidth spread found

-- ToddHunter - 2021-12-01
Topic revision: r2 - 2022-04-22, ToddHunter
This site is powered by FoswikiCopyright © by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding NRAO Public Wiki? Send feedback