Here is an initial list of requirements that have not been implemented yet, lifted from documents written at JIVE and NRAO:
  • EOP corrections (JIVE HP.3, NRAO 2.7.2 and 2.7.3)
  • Delay and rate windows (JIVE HP.5, NRAO 2.4.3.8)
    Implemented, but not per-antenna (NRAO 2.4.3.8.2); per-antenna windows requested by EHT
  • Smoothing and editing solutions (JIVE HP.6, NRAO 2.2.8)
  • Automatic import of gain curves (JIVE HP.11, NRAO 2.2.4.1)
  • Ionospheric correction (JIVE MP.2, NRAO 2.9.1)
  • Decorrelation correction (JIVE MP.3, NRAO 2.2.5)
  • Phase-cal tone correction (JIVE MP.11, NRAO 2.3)
    Prototype implementation in Python
  • Plotting of gain curves (JIVE LP.2)
  • Use pointing direction when appropriate (JIVE LP.3)
  • Parallactic angle correction for X-Y mounts (JIVE LP.4)
  • Interpolation should (optionally) respect scan boundaries (NRAO 2.2.1.3)
    Also raised by Michael Janssen as important for mm-VLBI
  • Implement overlapping solution intervals (JIVE HP4 and MP7, NRAO 2.4.3.2.1 and 2.4.2.2.2)
  • Weighting of visibility data (JIVE HP10 and MP5)
    Implemented but not yet on master
Deliverables for RINGS:
  • Fringefitting wide-bandwidth (with gaps)
    Proposed solution also includes handling irregular grids needed for VLBI with ALMA
  • Fringefitting dispersive delay (NRAO 2.4.1.2.1)
Wishlist from EHT/rPICARD:
  • Fringefit improvements
    • Support for irreguar grids with multiple spectral windows
      Related to fringe fitting large bandwidths
    • Polarization stacking (NRAO 2.4.3.7 and 2.6.4.1)
    • Baseline stacking
    • 1D fringe search (NRAO 2.4.3.6)
      Solving for delay only (instead of just zeroing rates)
  • Polarization calibration (NRAO 2.6.1 also NRAO 2.4.1.3?)
    Prototype implementation (by Ivan Marti-Vidal) in (mostly) Python
  • Proper handling of solution intervals that result in intervals containing a small number of remaining visibilities (NRAO 2.4.3.2.2)
The JIVE document also points at VLBI Scientific Memo #37 that describes a better amplitude calibration strategy. CASA seems to have (all) the ingredients to implement this strategy, but an explicit description of the steps involved is probably necessary.

-- MarkKettenis - 2019-01-22

Generic
* Interpolation during application of various calibration tables
(including, but not limited to fringe fitting) should have the
capability of respecting scan boundaries.
 
* Geometry-dependent calibration (parallactic angle, gaincurve,
opacity, etc.) should use the POINTING.DIRECTION instead of
FIELD.DELAY_DIR/PHASE_DIR/REFERENCE_DIR.
Question: How does mean vs. apparent position factor into this?


Amplitude calibration
* Antenna gain
** Elevation gain curve tables naturally imported with visibility
data should be usable.
** Application of different elevation gain curves on different
sub-bands should be 2.2.4.3 supported. This is especially
important in cases where dual-band observing (e.g., S- band and
X-band simultaneously) is employed.
** It should be possible to plot gain curves as function elevation.
Question: Should it also be possible to plot the gain curve as
function of time (following antenna pointing)?
Currently gain curves are not automatically imported when using the
importfitsidi task. Instead helper scripts are available to create
gain curve tables from GAIN_CURVE tables included in FITS-IDI files or
from text files in ANTAB/VLOG format. A challenge here is that gain
curves need to be converted from "total power" representation into
"voltage" representation, which isn't always possible for
(ill-defined) gain curves that aren't positive across the entire
elevation range.
* Decorrelation correction
** Correlator averaging parameters, possibly as a function of
baseline, should be stored with the visibility database.
** It shall be possible to optionally correct for decorrelation
without requiring an additional pass through the data (on-the-fly
application).
* Calibration editing
** It shall be possible to manually edit input calibration data.

Pulse Cal data handling

Phase (or equivalently, pulse) calibration is a VLBI technique used to
correct the sampled data for instrumental effects. For example, in
geodesy one would like the observations to measure the baseline delay
to a fixed point on each antenna, typically the intersection of
axes. However, the data are sampled on the ground after passing
through cables, connectors, down-converters, and filters. By injecting
a series of pulses as close to the front end as is practical, a series
of tones is produced in the frequency domain, which can be used to
solve for both delay and phase effects between the front end and the
samplers.

For example, in the broadband system used for the NASA Space Geodesy
Program, pulses are injected at a 5MHz rate, producing tones (or
rails) at 5MHz intervals. Thes eare extracted in the correlation
software and written to a calibration file that accompanies the
visibility data, one per antenna. The phase cal data consist of
triplets of frequency, amplitude, and phase, tabulated every second.
The fringe-fitting software (e.g., HOPS FOURFIT) finds via FFT the
best-fit line of phase as a function of frequency, using all of the
tones in a channel or any desired subset thereof (in the case where
known RFI corrupts tones). The slope determines a delay, which is then
differenced on thebaseline and applied to the complex visibility data.
The visibilities are also adjusted by the differential phase (at
mid-band) of the two fits. This process allows data that have passed
through different anti-aliasing filters and samplers to be registered
with one another, thus allowing phase-coherent delay solutions across
multiple wide IFs. This technique has been applied with success to
group delayextraction over a frequency span of about 6 GHz. The comb
frequency structure causes ambiguities in measured delay; any delay
measurement solely determined by a comb with frequency interval Q can
only determine delay modulo 1/Q. Resolution of this ambiguity can come
from fringe fitting some data. Usually only a very small amount of
data for an entire experiment is required as delays typically don’t
change by more than 10s of nanoseconds and the ambiguities are
typically 200 or 1000 nanoseconds. Continuity of delay through time
can be used to extend the period of ambiguity resolution.
* Data Structure
** Import of Pulse Cal data shall be supported
*** Pulse cal data attached to a FITS file shall be importable.

*** Each pulse cal measurement contains a real and imaginary value
and the time interval corresponding to that measurement.

*** A pulse cal set is the collection of all pulse cal measurements
made over one time interval at one antenna.

** Between 0 and B+1 pulse cal tones per sub-band must be supported
where B is sub-band bandwidth in MHz.
** A cadence as fast as one pulse cal set per visibility integration
time should be supported.
** Time averaged pulse cal data should be supported; averaging
intervals may be integer multiples of the visibility integration
time or not.
** An optional "cable cal" value, containing an additional
instrumental delay correction, should be handled along with pulse
cal data.
** Different antennas may have different pulse cal intervals and/or
number of tones.
** Single precision floating point is sufficient for the real and
imaginary parts of each pulse cal measurement; time should be
accurately representable to at least 1ms.
* Pulse Cal Data Selection
** It should be possible to plot a time series of pulse cal
amplitude or phase as a function of time
foraselectionoftonestones. Similarlythecablecalvaluesshouldbeplottable.
** It should be possible to view the amplitude or phase of the time
series of a pulse cal set as a raster image.
** It should be possible to flag certain pulse cal values based on a
priori information or interactive editing processing; flagged
values should be ignored in computations involving the pulse cal
data.
* Calculations to perform
** It shall be possible to determine the delay as a function of time
based on the Pulse Cal data.
*** Solutions should be determined separately for each antenna and
separately for each spectral window.

*** It shall be possible to determine a single delay value from
multiple subbands.

** In cases where cable cal data is present there should be the
option to include the cable cal correction in the computed delay.
** The determined delays should be stored in a table that can be
further edited and applied as necessary.
** It shall be possible to time average pulse cal values.
** It shall be possible to form a bandpass calibration table based
on pulse cal sets.
** It shall be possible to form a gain calibration table by
extracting the amplitude and/or phase of a single tone of each
spectral window.
** It shall be possible to use fringe-fit determined delays to
resolve pulse cal delay ambiguities.
Note: This has been partly prototyped in Python by Des Small.


** Fringe fitting
* General Delay Fitting Requirements
** It shall be possible to determine and correct cross-polarized
delays and phases.
*** It shall be possible to include in the fit a dispersive
multi-band delay component proportional to $1/freq^2$. This is
relevant for ionospheric calibration.

*** Modes of operation
*** It shall be possible to overlap time ranges by a specified
amount (see the SOLSUB parameter for AIPS task FRING).
*** In cases where the cadence does not evenly span the data valid
period of a scan an intelligent algorithm to shift the intervals
shall be invoked.
*** It shall be possible for delay and rate windows, both center and
width, to be specified by the user on a per-antenna basis.
** It should be possible to smooth calibration solutions obtained
using the CASA fringefit task.
*** It should be possible to smooth parameters of these solutions
(phase, delay, rate) individually.
** It should be possible to combine data by IF polarization prior to
determination of the delay paramters.
** It shall be possibble to use alternative weighting schemes in the
CASA fringefit task (see the WEIGHTIT parameter for AIPS task
FRING. At least the standard weighting by $1/\sigma^2$ as
weighting by $1/\sigma$ shall be supported.
** It shall be possible to use baseline stacking techniques to
assist fringe detection.
** It shall be possible to combine spectral windows (in order to do
a multi-band fit) even if there are large gaps (in frequency)
between spectral windows.
** It shall be possible to combine spectral windows even if the
channels of different spectral windows do not align on a single
frequency grid.
Note: A possible approach has been prototyped in Python by Des Small.


** Polarization

The following requirements relate to the handling of data correlated
with full polarization (i.e., all 4 polarization products).
* Polatization calibration
** It must be possible to determine and correct for antenna
polarization leakages.
Note: Prototype implementation (by Ivan Marti-Vidal) in (mostly) Python.
** It mustl be possible to apply the (co)-parallactic angle correction
for X-Y mounts. This is necessary to support calibration of data that
includes the Hobart 26m antenna which is part of the Australian Long
Baseline Array (LBA).
Note: Implementation (for casacore) exists but has not been verified yet.

*** Calibration
** It shall be possible to transfer delay calibration information
across polarizations to increase coherence time:
*** It shall be possible to use fringe solutions in one polarization
(e.g., RR) to assist in fringe detection in another polarization
(e.g., RL) via baseline stacking techniques.
*** It shall be possible to use the visibility phases as a function
of time on a baseline in one polarization to attempt to extend
the coherence time in another polarization.

* Model accountability and manipulation
** Delay model adjustments
Earth Orientation Parameters (EOPs) are used to describe the
orientation and spin phase of the earth relative to a standard model
of a uniformly spinning orb. The deviations from uniform motion are
unpredictable as they are largely driven by transfer of angular
momentum between the earth's crust, the oceans, atmosphere and earth
core. Typically final best estimates for the EOPs are only available a
week or two after observation, which may be after correlation.
** It shall be possible to apply delay corrections to the data with
improved EOPs[1].
* Delay model replacement
** It shall be possible to derive a delay correction from the
difference between the current delay model table and an external
delay model.
[1] The equivalent functionality is implemented in AIPS task CLCOR
when using OPCODE = 'EOPS'.


** Miscellaneous requirements
* Ionospheric Correction
** It shall be possible to derive a dispersive delay calibration
table from external ionosphere data. (Equivalent to AIPS task
TECOR)
** External ionosphere data in IONEX format should be downloaded on
demand from cddis.gsfc.nasa.gov. (For reference implementation,
see AIPS task TECOR)
Topic attachments
I Attachment Action Size Date WhoSorted ascending Comment
requirements.pdfpdf requirements.pdf manage 135 K 2020-01-15 - 06:26 MarkKettenis Requirements document (draft)
Topic revision: r7 - 2020-01-15, MarkKettenis
This site is powered by FoswikiCopyright © by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding NRAO Public Wiki? Send feedback