CORRELATOR REPORT, EVN MkIV DATA PROCESSOR AT JIVE EVN TOG MEETING, March 2006, Dwingeloo 15 March 2006 (statistics cover 16 Jun 2005 - 10 Mar 2006) Bob Campbell Huib van Langevelde Arpad Szomoru PERSONNEL On January 1 2006 Arpad Szomoru assumed the position of head of the JIVE R&D position. This encompasses the e-VLBI programme covered under the EXPReS proposal but also the maintenance and development of the correlator. Mark Kettenis was promoted from the ALBUS project to the correlator development team. SCIENCE OPERATIONS The table below summarizes projects correlated, distributed, and released from 16 June to 10 March. The table lists the number of experiments as well as the network hours and correlator hours for both user and test/NME experiments. Here, correlator hours are the network hours multiplied by any multiple correlation passes required (e.g., continuum/line, >16 station, 2 head stacks, different phase centers, etc.) User Experiments Test & Network Monitoring N Ntwk_hr Corr_hr N Ntwk_hr Corr_hr Correlated 28 367 474 13 65 68 Distributed 32 420 578 13 65 68 Released 24 332 494 13 72 75 The following table summarizes by session the user experiments still in the queue, with an additional column for experiments not yet distributed (entries = remaining to do / total). The actual correlator time is typically between 1.5-2.5 times these estimates, depending on the number of re-dos or other problems. A significant difference in this table compared to previous versions is that now there is neither a backlog of experiments to correlate nor to distribute. N_to.corr Corr.hrs N_to.dist Feb/Mar'06 17/17 not yet determined The Feb/Mar'06 session lasted until 6 March. At the date of this report, there's no experiment for which we've received the media (or field-system logs) from all stations. We have begun sending out the pre-correlation e-mails to PIs, by which we confirm the correlation parameters & archive classification of the individual sources. A priori, about 60% of the experiments would call for multiple correlation passes, leading to an estimate of about 320 correlator hours, which would be a single-session high since the large May/Jun'03 session. Nt has apparently successfully returned to observations, following a two-session hiatus with a azimuth-drive casualty. Some stations have experienced weather-related problems in individual sessions, but that's not unusual for a winter session. TECHNICAL DEVELOPMENTS We currently have 10 DPUs and 16 Mk5A units attached to station units (SU) for operations. All of the Mk5A units are housed inside temperature- controlled cabinets, and are fully connected to their SU. Through a simple re-connection of 2 cables, we can re-connect a DPU an SU. In the Feb/Mar'06 session, we expect only 1 tape station (GBT -- even though we sent disks to Socorro to supply them.....) Disk-servo'ing at the beginning of jobs and coming out of gaps in the schedule is now nearly instantaneous. Some attention now turns to the interaction between sched, the field-system logs, and the correlator- control files, to get the playback to start from the beginning of good headers on disk rather than when sched (or the field-system) thinks the antenna is on source (the latter condition more rigorously a job for post-correlation flagging that already takes place in the pipeline). Further progress has been made on e-VLBI. Tests have been done that reached 256 Mb/s fringes and 512 Mb/s data transfer. But the connectivity remains rather erratic. Earlier this year, Medicina became the sixth European telescope with a high-speed internet connection to JIVE. Medicina participated in the e-VLBI test last January, but some problems still remain to be solved. Much effort has gone in preparing the contract with the EU for the EXPReS project. This aims to transform the EVN into a fully functional 16-station real-time e-VLBI network. The PCInt development concentrates on making the high-speed output (1/8 and 1/16 second integrations) available in the normal data path. Deriving new data products from this output (new phase centers, spectral averaging and such) is part of the ALBUS project. It is now possible to channel phase cal detections out of the EVN correlator in such a way that the AIPS task PCCOR can work with these. Some more testing is necessary before this option can be made available to general users. Promising results have been obtained with the ionospheric calibration project. Various data products have been collected that are now being evaluated with real data. The calibration algorithm is implemented in ParselTongue and uses some special AIPS extension tables. In addition we have been working to improve the accountability of the correlator model, specifically to be able to undo tropospheric corrections. This is necessary to import improved calibration from the Bonn WVR project. ParselTongue, the Python interface to AIPS, is being upgraded regularly to address bug reports and enhancement request from its users. There are quite a number of people interested in this tool all over the world. USER SUPPORT The EVN Archive at JIVE is up and running. This provides web access to the station feedback, standard plots, pipeline results, and FITS files. More than half of the PIs use the Archive to access their distributed FITS files, rather than having a DAT or DVD sent to them. Public access to source-specific information is governed by the EVN Archive Policy -- the complete raw FITS files and pipeline results for sources identified by the PI as "private" have a one-year proprietary period, starting from distribution of the last experiment resulting from a proposal. PIs can access proprietary data via a password they arrange with JIVE. PIs receive a one-month warning prior to the expiration of their proprietary period. We have also increased the storage available on the archive machine, as it was rapidly getting too small. The top 7 experiments in terms of FITS-file size are: GG060 (268.2 GB), EL032 (260.2 GB), GG053A (228.8 GB), GG053C (224.9 GB), GG053B (221.2 GB), GW017 (162.3 GB), and EB026 (94.2 GB). Such increases in storage capacity will likely prove to be a continuing requirement. Provisions have been made to store a copy of all the user data outside the main Dwingeloo building. There are two independent ways to search the Archive other than by direct entry via a specific experiment. The EVN catalogue of observations (Bologna) can be used to search for observations of particular sources, provides a link to the relevant experiments on the EVN data archive for experiments correlated at JIVE. In addition, a FITS-finder utility for the archived data is operational. A database contains all the meta-data for the projects that are on-line. Searches can be key to source names or coordinates, observing frequency, and/or participating telescopes, among other characteristics. New archive features include interfaces between the EVN Archive and both the Aladin Sky Atlas and the Sloan Digital Sky Survey.