analyzemscal

Return to directory of Todd's CASA extensions

This function generates the Cycle 0 QA2 plots for a pipeline-reduction (along with other useful plots and information) and displays them a webpage side-by-side with the results from manual calibration. It must be run from inside a normal casapy session, not the pipeline environment's casapy. You do not need to be in any specific directory when you run this function. For help in modifying the PPR for your project prior to running the pipeline, see modifyPPR. Updates to this function are announced at CSV-2684. See also analyzemsimage.

Usage:

import analyzemscal as amc
amc.analyzemscal(asdm, pipecaldir, manualcaldir='', manualqa2dir='', webpagedir='web', 
                  target_source=0, uvrange='0~40m', refant='', ampGainTableManual='', 
                  wvrstat_intent='BANDPASS', force_au_plotbandpass=False, debug_plotms=False, 
                  debug=False, phase_cal='', plotlevel=1, wvrstat_spw='', wvrstat_pol='', correlation='', 
                  bandpass_cal='', showFlagStats=True, edgeChannels=10, avgchannelTarget=5, 
                  solutionTimeThresholdSeconds=30.0, nsigma_coherence=5, unwrap_phase=False, 
                  autoscale_yaxis=False, wvr_refant='', nsigma_discrepancy=5, 
                  match_man_refant=True, ignoreZeroCoords=False)

• asdm: the name of the ASDM to plot (any path will be stripped, as will ".ms" if present)
• pipecaldir: required argument, the absolute directory to find the pipeline result for a single execution (or just '.' if it is the current working directory)
• pipelinedir: optional argument, the pipeline-YYYYMMDDTHHMMSS subdirectory (i.e. within pipecaldir) to search for the casa_commands.log file (by default it will use the most recent one containing the asdm)
• pipeqa2dir: optional argument (default = pipecaldir+'qa2/'+asdm)
• manualcaldir: optional argument, the absolute directory to find the manual results produced by uid___scriptForCalibration.py and es.generateQA2Report
• manualqa2dir: the absolute directory to find the manual QA2 result (default = manualcaldir+'/qa2')
• webpagedir: the absolute (or relative) directory in which to write a subdirectory (named the asdm name) where the html and pngs will be written (default='web/')
• webpagedir_suffix: optional suffix to add to the ASDM name to distinguish multiple runs in the same webpagedir
• webarea: if True, precede webpagedir with "/home/safe.nrao.edu/content/alma/PipelineTestResults/"
• target_source: integer, the Nth science target, which is used to pick a phase calibrator for display on the phase vs. time plot (default=0)
• clean: True will cause all old plots and directories created by this script to be removed prior to creating new plots.
• uvrange: used when generating the average target spectrum. This corresponds to the parameter uvrange in es.generateQA2Report.
• refant: if non-blank, do not try to determine the refant that the pipeline used, but tell it that this one was used
• ampGainTableManual: in case the manual reduction's split.flux_inf table is called something else, specify it here. This corresponds to the parameter ant_amp_temporal_caltable in es.generateQA2Report.
• force_au_plotbandpass: default=True: always use au.plotbandpass(). False: if the task plotbandpass is present in casa, then use it.
• solutionTimeThresholdSeconds: passed to plotbandpass (default=30)
• debug_plotms: if True (default), it will detect missing plotms plotfiles and create a python script to run manually to try to make them again.
• phase_cal: the field ID to use as the phase calibrator, instead of calling es.getPhaseCal. This is useful if there are not 2 observations of the same phase calibrator in the dataset, in which case es.getPhaseCal returns nothing.
• bandpass_cal: the field ID to use as the bandpass calibrator, instead of using the result from es.getIntentsAndSourceNames. This is useful if the manual reducer used the phase calibrator as the bandpass calibrator, e.g. due to CSV-1807.
• plotlevel: 1=basic plots only; 2=build the per-antenna plots as multipage PDFs
• wvrstat_intent: the scan intent to use in es.wvr_stat() for the statistics and the thumbnail image
• wvrstat_spw: the one spw to use in es.wvr_stat() for the statistics and the thumbnail image
• wvrstat_pol: the one polarization to use in es.wvr_stat() ('X' or 'Y') for the statistics and the thumbnail image
• wvr_refant: override the pipeline's choice of refant with the specified string
• nsigma_coherence: value to pass to es.wvr_stat for the report of outlier antennas
• unwrap_phase: value to pass to es.wvr_stat for purposes of plotting
• autoscale_yaxis: value to pass to es.wvr_stat for purposes of plotting
• correlation: the correlations to plot in all calls to plotms (default='' meaning all), use 'XX,YY' for 4-polarization datasets to avoid the cross-terms
• showFlagStats: default=True, set to False to speed up the processing (mainly for debugging purposes)
• edgeChannels: default=10, Num of channels to ignore in counting flagged Tsys channels
• avgchannelTarget: default=5, Num of channels to average in target spectrum
• nsigma_coherence: value to define outlier antennas in es.wvr_stat
• nsigma_discrepancy: value to define outliers in extractWvrgcalOutputFromLog
• ignoreZeroCoords: if True, then don't stop if planets have 0,0 coordinates

Typical Examples:

1. Pipeline and manual calibration results side-by-side:

This example will write the webpage output into directory: ./web_results/uid___A002_X49a70a_X4d8/
It will contain results for both the pipeline calibration and the manual calibration:

 amc.analyzemscal(asdm='uid___A002_X49a70a_X4d8',
    pipecaldir='/lustre/naasc/pipeproc/pipeline/root/2011.0.00958.S_2012_11_07T14_44_44.813/MOUS_uid___A002_X314ea1_X231/working',
    manualcaldir='/lustre/naasc/thunter/Cycle0/2011.0.00958.S/X4d8_working/Calibration',
    webpagedir='web_results')

Please note that the 'manualcaldir' option expects the manual reduction files to be in the state prior to packaging. It will not work from the limited products that are stored from the archive. If you only have the archive products, then you should first run importasdm on the ASDM, then run the uid___scriptForCalibration.py, then run es.generateQA2Report('uid___.ms'). Only then can you run analyzemscal with the manualcaldir parameter set.

2. Pipeline results only:

This example will write the webpage output into the absolute directory: /home/user/web_results/uid___A002_X49a70a_X4d8/
It will contain results for only the pipeline calibration.

 amc.analyzemscal(asdm='uid___A002_X49a70a_X4d8',
    pipecaldir='/lustre/naasc/pipeproc/pipeline/root/2011.0.00958.S_2012_11_07T14_44_44.813/MOUS_uid___A002_X314ea1_X231/working',
    webpagedir='/home/user/web_results')

Example for Charlottesville users

For Charlottesville users, this example will write the results into directory named by your UID in the standard ALMA/NAASC web area. An example result is here:
https://safe.nrao.edu/alma/PipelineTestResults/2013Sep_heuristics/uid___A002_X369097_X419/

 amc.analyzemscal(asdm='uid___A002_X49a70a_X4d8',
    pipecaldir='/lustre/naasc/pipeproc/pipeline/root/2011.0.00958.S_2012_11_07T14_44_44.813/MOUS_uid___A002_X314ea1_X231/working',
    manualcaldir='/lustre/naasc/thunter/Cycle0/2011.0.00958.S/X4d8_working/Calibration',
    webpagedir='Mar2013_heuristics', webarea=True)

Example results on Cycle 0 data: on the NRAO wiki (but password protected)

-- ToddHunter - 2013-01-02