The VLBA-DiFX Correlator

What is VLBA-DiFX

VLBA-DiFX is the name that NRAO has given to its new VLBI correlator. It is a software correlator that uses Adam Deller's Distributed FX (DiFX) as the base and has software written around it for nearly seamless integration with the VLBA. The correlator runs on a cluster of commodity computers containing multi-core Intel processors. A bank of about 24 Mark5 units feed the correlator directly from data recorded at the antennas.

VLBA-DiFX is one of three components of the recent VLBA Sensitivity Upgrade Project, the other two being a digital back end to replace the aging analog baseband-converters, samplers, and formatters, and the Mark5C recorder which will provide 4 times the record rate of the Mark5A units that are currently in use.

Users whose observations are correlated using DiFX are encouraged to include the following statement in any publication of those results: "This work made use of the Swinburne University of Technology software correlator, developed as part of the Australian Major National Research Facilities Programme and operated under licence", and to cite the following paper: Deller, A. T., Tingay, S. J., Bailes, M., & West, C. 2007, PASP, 119, 318.

Capabilities

The capabilities of VLBA-DiFX match or exceed those of its predecessor including:

• Very short integration times (in principle as short as twice the reciprocal frequency resolution)
• High spectral resolution (15 Hz has been demonstrated)
• Pulsar gating and binning
• Support of data recorded in Mark4, VLBA and Mark5B formats

Information about specific observation modes

The following types of observing have some complexities worthy of note:

• Pulsar observations (See pulsar observig guide)
• Use of oversampling
• Multiple sub-array observing

Identifying DiFX data in the VLBA archive

Data from both the hardware and DiFX correlators can be downloaded from the VLBA data archive. Some projects are correlated on both correlators making identification of the various data types more complicated. Three types of data may be present: data straight out of the VLBA hardware correlator, hardware correlator data that has gone through a pipeline to simplify post-processing, and the DiFX software correlator output. The file names can be used to identify the type of each file as follows: files ending in .idifits are from DiFX; files ending in .uvfits are data straight from the VLBA hardware correlator; files without a file extension are from the pipeline.

Information about the output data

The VLBA-DiFX correlator will produce FITS-IDI data just as the VLBA hardware correlator has always done. This means that you should still use DIGICOR=1 in FITLD and run ACCOR just as you would have before. There are some minor differences in what you will get, both of which should make your life easier:

• VBGLU is not needed for projects observed at 512 Mbps data rate.
• FXPOL is not needed for any polarization mode.

It is recommended to periodically visit http://www.aips.nrao.edu/problems.shtml#Ref_Frame to learn about other potential AIPS related issues. Please make sure to use a recent version of AIPS.

Gotchas

There are some differences between the VLBA hardware correlator and the DiFX software correlator that you should know about:

• AIPS. A recent version of AIPS is required to properly load DiFX data. 2008DEC31 (final) or newer is needed. 2014DEC31 or newer is recommended.

• Integration times. The hardware correlator supported integration times that are multiples of 131072 microseconds. For speed-up factor 2 or speed-up factor 4 the integration times must be multiples of 262144 and 524288 microseconds respectively. On the software correlator, any integration time can be specified. Certain integration times are better than others; if the requested integration time is not an exact multiple of the short term accumulate (sub-integration) time then visibilities will have variable weight as some integrations will have more than the average number of sub-integrations, and some will have fewer. Correlation resulting in spectral resolution no higher than 31.25 kHz can have integration times that are integer numbers of sub-integrations in length and are exact multiples of 1 second. In these cases use of 1, 2 or 4 seconds is preferred. For higher spectral resolution, integration times of 1024, 2048 and 4096 milliseconds are recommended. Integration times that are multiples of 524288 microseconds continue to be supported if there is a desire to match the time resolution of projects correlated on the hardware correlator. Note that unless specific measures are made, matching of integration time does not imply matching of integration interval; the hardware correlator always uses midnight as the beginning of the time binning over the course of the day whereas DiFX uses the experiment start time as that reference point.

• Specification of source and antenna positions. Standard practice has allowed users to prepare sched input files (.key files) that suggest "database positions" be used for calibrators and antennas. With DiFX this is not an option. Any antennas or sources without explicit positions will be assigned positions from the sched catalogs used at the time sched is run. In most cases the database values and the sched catalogs in Socorro will be identical, but it is possible that they differ. If you require certain positions to be used, be sure to specify them in the .key file.

• U,V,W values. The VLBA hardware correlator assigned the standard geometric U,V,W baseline vector values in the FITS files. These values are adequate for imaging and astrometry near the delay center. In cases where a shift of thousands of synthesized beams is needed, the appropriateness of the purely geometric U,V,W values diminishes. The VLBA DiFX correlator assigns U,V,W derived from derivatives of the delay model to ensure consistency of the baseline vectors with the correlator model, which is needed when shifting large amounts. As a result, expect differences of the U,V,W values of typically 1 part in 10,000 to 100,000.

• Data archive. The online VLBA archive is the preferred source for you to access your recently correlated data. Filenames for data correlated with DiFX will differ from those correlated with the hardware correlator.

References

The VLBA Observational Status Summary

The DiFX Wiki

The VLBA Sensitivity Upgrade Project