DiFX Correlator

About DiFX
DiFX is a collection of code for correlating baseband radio interferometry data, producing visibilities. It was originally developed for the Australian LBA, but it also supports Mk5 and K5 style recording formats. In principle, it could be easily modified to work with any baseband data. Presently it produces visbilities in the RPFITS format, but in future support is planned is planned for FITS-IDI. DiFX has been developed by Adam Deller at Swinburne University, with assistance from many parties including Steven Tingay, Craig West, Walter Brisken (NRAO) and Warwick Wilson (ATNF).

Spectral Resolution

DiFX currently supports powers-of-2 numbers of spectral points spanning each individual baseband channel, up to 4096 for routine DiFX processing, and up to 32,768 if required and adequately justified. (The latter limit is the maximum resolution currently supported by AIPS.) Oversampled data (essential for extremely high spectral resolution with the existing VLBA baseband subsystem can be decimated appropriately. Currently, both the number of spectral points, and the oversampling factor, must be the same for all basebands at any given time, although multiple passes with different baseband subsets are possible. The actual spectral resolution obtained, and statistical independence of the spectral points, depends on subsequent smoothing and other processing.
A new DiFX capability supports spectal zooming, selection of a subset of correlated spectral channels from any or all basebands. Only the selected channels are included in the output dataset. This capability will be of value mainly in maser studies, where the recorded band may be much wider than the maser emission in two main categories of observations:
• Maser astrometry with in-beam continuum calibrators. Wideband observing is required for maximum sensitivity on the calibrators, while zooming allows high spectral resolution at the frequencies where maser emission appears.
• Multiple maser transitions. When wide bands are used to cover a large number of widely separated maser transitions, spectral zooming allows the empty portions of high-resolution spectrum to be discarded.
In proposing observations that will use spectral zooming, the required number of channels before zooming should be specified in the Proposal Submission Tool. Initially, the location and width of the "zoom" subbands will have to be communicated directly to VLBA operations before correlation. The output data rate must be justified if it exceeds the current 10 MB/s soft limit

Integration Period

DiFX accommodates a nearly continuous range of correlator integration periods over the range of practical interest. Individual integrations are quantized in multiples of the indivisible internal FFT interval, which is equal to the number of spectral points requested, divided by the baseband channel bandwidth. Since the latter are powers-of-2 MHz, the internal FFT interval is always a power-of-2 microseconds.
For most cases, with low to moderate spectral resolution, and/or wideband baseband channels, the FFT intervals are fairly short, and it is straightforward to find an integration period in any desired range that is an optimal integral multiple of the FFT interval, where optimal refers to the performance of DiFX. Extreme cases of very high spectral resolution (many spectral points across a narrow baseband channels - resolution of less than about 100 Hz) imply FFT intervals long enough that only limited choices of integral multiples are available.
For flexibility in these situations (although the option exists in all cases), integration periods other than an integral multiple of the FFT interval can be approximated, in a long-term mean, by an appropriate sequence of nearby optimal integral multiples. In this case, output records are time-tagged as if correlated with exactly the requested period.

Multiple Phase Centers

The field of view of the VLBA using typical correlation parameters (e.g., 0.5 MHz spectral resolution, 2 second averaging) is very small, around 1/10,000 of the primary beam area. Thus, to image targets which are widely spaced in the primary beam requires multiple processing passes in typical correlator implementations. If the visibilities are maintained at high time and frequency resolution, it is possible to perform a uv shift after correlation, essentially repointing the correlated dataset to a new phase center. However, this approach requires prohibitively large visibility datasets.
A new feature in the VLBA DiFX correlator implements multiple uv shifts inside the correlator, to generate as many phase centers as are necessary, in a single correlation pass. The output consists of one dataset of normal size for each phase center. This mode consumes around three times the correlator resources of a normal continuum correlation, due to the need for finer frequency resolution before the uv shift, but the additional cost is only weakly dependent on the number of phase centers. For example, correlating 200 phase centers requires only 20% more correlator time than 2 phase centers. This mode thus should only be requested for imaging of three or more sources within any single antenna pointing.
This mode is requested in the NRAO Proposal Submission Tool by setting the Number of Fields item in the resource section to the maximum number of phase centers required for any antenna pointing specified in this resource. Updates to the SCHED program, to be announced subsequently, will support specification of the actual phase center locations. The requested spectral resolution and integration time should correspond to the desired initial number of frequency channels per subband (required to minimize bandwidth smearing) and the desired integration between uv-shifts (to minimize time smearing).

Output Rate

Correlation parameters must result in an output rate less than 10 MBytes per second of observing time for routine DiFX processing; higher rates may be considered if required and adequately justified. Observers should ensure that their data-analysis facilities can handle the dataset volumes that will result from the correlation parameters they specify.
A rough parameterization for the output rate R, in Byte/s, is given by:

where N_stn, N_chn and N_spc are the numbers of observing stations, baseband channels, and spectral points per channel, respectively T_int is the correlator integration period, again referred to observing time and N_pht is the number of phase centers.
The polarization factor p=1 for single- or dual-polar (LL & RR) output; p=2 for cross-polar processing, delivering all four Stokes parameters.
Approximate output data rates are also predicted by the SCHED software.

Capabilities
DiFX can be configured to produce visibilties with pretty much arbitrary time and frequency resolution (although you need to be aware of the fundamental limitations of FFT window sizes etc), and can produce an arbitrary combination of products on each baseline (ie full stokes on one baseline, parallel only on another etc). It can channelise using an FFT or a polyphase filterbank (slower, but better channel spectral response - good for isolating RFI or spectral lines). Finally, it can pulsar bin visibilities into any number of bins, using incoherent dedispersion. This requires some a priori information on the pulsar ephemerides, provided in the form of a polyco file produced by the pulsar software package TEMPO. If you're just trying to recover the best possible SNR from a pulsar observation (eg to do astrometry) then DiFX gives you the option to sum your bins, after weighting each by the power contained in that bin. This gives the best possible SNR, by integrating over the whole pulse period.

CORRELATOR
The correlator is situated in the DSOC, at the end of the data path. Its role is to reproduce the signals recorded at the VLBA stations and any others involved in the observation, and to combine them in two-station baseline pairs, to yield the visibility function which is the fundamental measurement produced by the VLBA.

VLBA observations are processed using the DiFX software correlator. DiFX was developed at Swinburne University in Melbourne, Australia (Deller et al. 2007), and adapted to the VLBA operational environment by NRAO staff (Brisken 2008).

Software correlation has become feasible in recent years, and is especially well suited to applications like VLBI with bandwidth-limited data-transmission systems and non-real-time processing. Among its several advantageous aspects are:

• flexible allocation of processing resources to support correlation of varying numbers of stations, frequency and time resolution, and various special processing modes, with no fundamental fixed limits other than the finite performance of the processing cluster
• optimization of resource usage to minimize processing time
• integration of control and processing functions
• continuously scalable, incremental upgrade paths
• relatively straightforward implementation of special modes and tests

These and other virtues of software correlation are discussed in more detail by Deller et al. (2007).

Despite the absence of fixed limits cited in item (1) above, the initial operational use of DiFX occurs in a situation of limited processing capacity and uncertainty as to the scaling of load with various parameters. Thus, NRAO has established temporary guidelines for the extremes of spectral resolution, integration period, and output rate, that will be supported in routine DiFX processing. These are are specified in the appropriate sections below. Exceptions will be considered for proposals including a sufficiently compelling scientific justification.

DiFX processes 2-bit samples with substantially greater efficiency than 1-bit samples over double the bandwidth, basically because only half as many samples must be correlated. Since these two cases have nearly equivalent sensitivity, a specific justification also will be required for proposals requesting the wider-bandwidth, 1-bit mode.

Operation of DiFX is governed primarily by an observation description in VEX format. This format is used for both station and correlator control functions in a number of VLBI arrays, and the NRAO SCHED software.

In addition to the Mark 5A recordings from VLBA stations, DiFX can also process input data recorded in Mark 5B format.

Correlator output is written according to the FITS Interferometry Data Interchange Convention (Greisen 2009). In addition to the fundamental visibility function measurements and associated meta-data, the FITS files include amplitude and phase calibration measurements, weather data, and editing flags, that are logged at the VLBA stations (Ulvestad 1999). AIPS (Section 22.1) release 31DEC08 or later is required to handle DiFX data properly.