ALMA Correlator Quantization Correction

Meetings

• 2010 Mar 25
• 2010 Apr 14

New memos

• Four-Level van Vleck Correction via Table Lookup by F. Schwab (postscript version)

2010 Apr 14 (Wednesday) 9:00 EDT

• Attendees:
  1. Fred Schwab
  2. Todd Hunter
  3. Jeff Kern
  4. Steve Scott
  5. Jim Pisano
  6. Rodrigo Amestica
• Discussion and action items
  • Fred pointed out that some of the lag zero data in the Rn0.txt file indicate that the threshold settings are far from optimal (1-2 instead of 3.55, see time series plot and histograms). This appears to be all data observed in TDM mode.
  • Fred will work out how large the matrix needs to be to precalculate the QC.

2010 Mar 25 (Thursday) 10:30 EDT, CV-331, 434-293-6691

• Attendees:
  1. Fred Schwab
  2. Ed Fomalont
  3. Todd Hunter
  4. Jim Pisano
  5. Rodrigo Amestica
• Discussion
  • current implementation takes ~90us in average to compute Van Vleck curves and apply the correction to each item in the dump. It has been measured that of this timing quota 66% is taken by the Van Vleck curves instantiation and 33% by the correction itself. Speeding up the Van Vleck curves computation seems to accept two major approaches:
    1. use a pre-calculated table to interpolate the curves using both input sigma levels as lookup key.
    2. at the beginning of a sub-scan compute the 'operational point' based on measured sigma levels (a set of 4 or 5 Van Vleck curves.) Following dumps to be corrected by selecting the closest curve or interpolating a curve from the original set.
  • Discussing how to speed up the correction algorithm itself needs more input data at this moment. Rodrigo will work on collecting more timing information (instrument qc algorithm itself to see where the timing is going and how is the polynomial aproach working? how many times applies polynomial and spline?)
  • what happen if qc is completely disabled? Ask Chile to run a test on this (details of the test to be refined offline.) How important is the correction compared against its costly timing quota? Knowing an answer to these details would let dimension the actually 'needed' algorithm.
  • Ed should revive some test environment to contrast results with data produced by the correlator software. Present Ed some real data to clarify what's the 'type' of lags we are expecting to observe at the correlator's input.

Background material

• Gianni's paper (2008-07-23) (a.k.a. ALMA Memo 583 (2008-11-05))
• Fred's memo on van Vleck Correction for the GBT Correlator
• Convenient formulas for quantization efficiency (Radio Science 2007)
• Correlators with 2-bit quantization (Cooper 1970)
• Relevant correlator software (as of 23 Mar 2010)
  • QuantizationCorrection.h
  • QuantizationCorrection.cpp
• Timing tests run on March 24
  • total qc (instantiate object + apply correction) timing quota is close to 90us per item. FDM modes require such a correction in a sub-band basis (a factor 32 for modes involving 32 sub-bands), this means that for a few different array sizes we would scale as it follows:
    • 32*90*2*(2+1)/2 = 8640 [us]
    • 32*90*3*(3+1)/2 = 17280 [us]
    • 32*90*5*(5+1)/2 = 43200 [us]
    • 32*90*32*(32+1)/2 = 1520640 [us] (each cdp node 'sees' at most only 32 antennas)
• Data variations of lag zero
  • The attached file contains data showing variations of lag0 across all correlator dumps in a sub-scan. Number of dumps and basic statistic figures (mean, rms, min and max.) For FDM modes there is one entry per each TFB sub-band in that specific correlator mode.
  • My conclusion from AIV-989 was that variations across dumps in a sub-scan were the cause for platforming, this coupled with the fact that originally the software was reusing the quantization correction computed for the first dump only. The data in the file represents the actual lag0 variations we are dealing with.

Reference material

• TFB Programming Manual (local PDF)
• JIRA ticket AIV-989 on "Frequent platforming and scalloping in autocorrelation spectras" (closed Oct 2009)
• JIRA ticket AIV-1622 on "...autocorrelation is spuriously high by a factor of two" (closed Jan 2010)
• Correlator design document
• ESSR detailed requirements
• Thompson, Emerson and Schwab 2007