A Flexible Format for exchanging Pulsar Data

Clink here to down-load a PostScript document with full details of the format...


Introduction

The European Pulsar Network (EPN) is an association of European astrophysical research institutes that co-operate in the subject of pulsar research. All institutes have up until now developed their own individual hardware and software facilities tailored to their own requirements and will, of course, continue to do so in future. Contact and co-operation has always existed between the scientists of the member institutes and outside, but the lack of a common standard format for pulsar data has hampered previous collaborative research efforts.

We we have developed a flexible format for exchanging data between EPN pulsar groups. We have written a suite of Fortran-77 sub-routines which can be easily incorporated within existing software which can read and write data in this format. The format has proved so successful that we now advocate its use as a useful world-wide utility for pulsar data exchange. Astrophysical applications of such a format currently being pursued by EPN groups include the establishment of a data bank of pulse profiles as well as simultaneous observations of pulsars by several European observatories.

The EPN format

The underlying principles of the format result from a number of requirements. This was essentially a balance between efficient data storage and providing sufficient information about the data for potential users. Specifically, the following requirements had to be met:

Operating system independence:
To make the data format as portable as possible between present and future operating systems, we have opted to use only ASCII-data throughout. We have arrange these data so that words are aligned over 80-byte boundaries, this simplifies inspection and printing of the files.

Completeness:
The data should contain all information for the identification of the source and the observing circumstances useful for further analyses of the data by others.

Compactness:
Descriptive information should not dominate the format. The measured values that form the bulk of a block of data are given as scaled four-character hexadecimal numbers, giving a dynamic range of 65536:1.

Versatility:
The format should be suitable for continuously sampled multi-channel filterbank search data, synchronous integrated and single-pulse data as well as processed data. In addition, we have designed the format, so that it can be used for observations of pulsars outside the radio regime i.e. variable units for the observing frequency and bandwidth, as well as topocentric telescope coordinates which are time variable for satellite observatories. Space is left for more descriptors and future adaptations and expansions.

Simplicity and ease of access:
We describe a data format consisting of a standardised fixed length header with a variable length data structure attached to it. The header fully describes the structure of the data, this structure is not changed within one file but can vary between files. That way it is possible to calculate the length of a data block within each file after reading its header. The file can then be opened for random access with fixed block length, faster than a sequential read.

Many of the above mentioned requirements were already met by a format in use at Jodrell Bank to which we made suitable modifications and extensions to make it more flexible.

The basic structure of an EPN file consists of a common fixed length header followed by a number of individual data streams of equal length. The header describes the data, containing information on the pulsar itself, the observing system used to make the observation as well as some free-form information about the processing history of the data. The onus is on the site-specific conversion process to ensure correct conversion to the standardised entries and reference to common catalogues (e.g. the Princeton Pulsar Catalog).

The data streams themselves may be outputs of different polarisation channels, or individual channels (bands) of a filterbank or a combination thereof. In total, there may be a number of data streams of i.e. different frequencies for each polarisation. Each data stream starts with a small, fixed length sub-header in front of the actual data values. The number of streams and their length may vary between different EPN files, but is constant within each file. A character field and an ordinal number is provided for each stream for its identification.

Format Compatible Software

To incorporate the capability to read and write data in this format within existing analysis software, a simple routine which can read and write data in this format. In addition, we have written some sample programs which can plot the data and display the header parameters. The software are written in Fortran-77 and have been packaged into a single UNIX tar file which is freely available via the Internet. To down-load the package, click here.

To uncompress and extract the contents of the tar file, issue the commands:

gunzip epnsoft.tar.gz
tar xvf epnsoft.tar

The present package contains some sample data and two example programs - ``plotepn'' and ``viewepn'' which plot and view EPN files respectively. The ASCII file 00README in this packages gives further details of the software and how to use it.


Aris Karastergiou: akara@mpifr-bonn.mpg.de

Data archive at the MPIfR Pulsar Group