TOG Meeting, 1 April 2004, Kotzting Performance and Reliability of the EVN ====================================== Session 3/2003 -------------- The November session was very busy with NMEs, ftp fringe tests, and eVLBI tests in between user experiments. The ftp fringe tests proved to be a useful way of checking telescopes between user experiments. Effelsberg recorded all the EVN experiments on Mk5. In the C-band session, for the first time, the Lovell Telescope participated. It produced fringes in the fringe test, but soon after that there was an LO problem. There were no Jb fringes in the D03C3 ftp test. Unfortunately, there was not much time left till the end of the C-band session to fix the LO, and so all user experiments were lost. This case shows that mid-session ftp tests are very useful, but we still have to reduce the turnaround time for the report. The other great loss was Torun, which could not observe in the C-band and L-band session because the recorder was broken. Comments on individual stations: Ef: In EC017C and EK015B some data were lost due to Mk5 problems. Some data were also lost in EC017C due to a computer problem (unrelated to the VLBI system). In EM050 there were troubles with the gregorian subreflector, lost half an hour of the data. Wb: Due to a Capstan motor failure N03C1 and EH016A were recorded on Mk5. In GT005 and D03C3 lost some data due to correlator and hardware problems. Jb: In F03C1 only pass 1 data was good. The following passes overwrote pass 2 data. Lost in the C-band session due to an LO problem. The L-band session went smoothly. On: Had some tracking problems in GM048C. In experiments EH015A, EM050, EF011, EH015B, and EJ006 the even numbered channels did not produce good data (at least for some the time) because of troubles with IF2. Produced very low sampler statistics data in all experiments. Sensitivity was badly affected. Mc: In F03X1 the S/X setup was used, no LCP data were recorded. In N03C1 the patch panel was not configured properly, which resulted in low amplitudes from VC#02 and VC#04. In EH016 Tsys was wrong for the same VCs (incorrect setting of IFD at start). In GT005 VC#05 and VC#06 had similar problems. Nt: The receiver was set to the wrong frequency in GG049A. The Bonn correlator reported very low X-band amplitudes in EC017C. It was probably due to a pointing error. D03C3 was lost due to an FS problem. Lost some data in EK015B because the vacuum was lost in the tape recorder. No feedback from EH016B. Tracks 2 and 3 were dead during the session. Piggyback-mode experiments (NMEs) produced strange spectrum channels (which were also very insensitive). Ur: Troubles with the cooled C-band receiver. In most of the C-band experiments the old uncooled receiver was used. The upper sidebands of VCs #2 and #4 produced low amplitude signal during the session. This was due to very low sampler statistics. The problem was not related to bad recording - check power in VCs! Sh: In EC017C there were antenna control and recorder problems. No fringes were found in the Bonn correlator. There were also antenna control problems in EH013 and EJ006, some data was lost. Session 1/2004 -------------- In the February session we had 4 NMEs and 4 fringe tests (one of these was a mid-session C-band ftp fringe test). There were a couple of outright failures due to LO problems: Mc and Ur in the K-band, and Ef in the X-band. The Ef problem was realized in the ftp test, and was quickly fixed for the C-band session. Several stations used Mk5 in most of the experiments. The low sampler statistics problem at Onsala that affected the May and November sessions in 2003 was fixed at the station. Notes on individual telescopes: Cm: Recorded on tape - track 15 was dead, so it was remapped to a system track. Started late in EB027A due to problems with MERLIN control, and in EK019 due to problems with starting the tape. Recorder problems in N04L1 (vacuum was lost, three scans were affected). Lower numbered tracks were sometimes dead (2-4). Ef: Lost N04X1 and GM052A due to an LO problem. Ef was late in EK019 (2 scans were lost). The FS-formatter communication crashed during EB026. In EB027B and N04L1 there were disk-pack and Mk5 problems, some scans were lost. Wb: In N04C1 VCs 7 and 8 (numbers according to the vexfile) did not produce fringes. Wb has a 100 MHz IF limitation in the C-band. Auto disk bank switching failed in EB027A, EB026, EB027B, and GG053B. No fringes from VC#4 lower sideband and on hs2 in N04X1. No fringes from VC#7 and 8 upper sidebands in N04L1. Polarization leakage was very high in N04L1 (~30%), the other experiments were OK. Jb: The two headstack experiments were lost (mainly the NMEs) because only the headstack 1 cables were connected. Headstack 1 data could be recovered at JIVE for EA029. On: There was no signal from VC#8 in N04K1. Lost EK019 and most of EB027A due to tracking problems. For the 21cm experiments and N04L1 used uncooled HEMTs, because one of the cooled HEMTs was wrong at 21cm. Disk bank switching failed in EB025 and EB027B. In EB025 the Mk5 had to be restarted, several scans were lost. The wrong schedule was started in GG053A (only 20 minutes were lost). Mc: Lost in the K-band session due to an LO problem. In EB027A headstack 2 position was apparently not right. Recorder lost vacuum in EB025, some scans were lost. In EB027B lost 3 scans because of pointing problems. In N04L1 VCs 1,2,7 and 8 were scheduled outside of the IF band (cannot do 128 MHz in the L-band). Nt: The tape was changed late in EB027B. Problems with GPS-formatter data at the beginning of the session. In N04X1 there was a large amplitude variation between hs2 channels. Tr: Troubles with BBC#5 in EB025 and EB027B: instable signal levels. Sometimes a bit poorer 2nd headtsack playback. Apparently there were no hs2 fringes in N04C1 (not pipelined yet). Ur: Lost in K-band due to an LO offset. No fringes from VC#2 lower sideband in N04X1 and N04L1 (others may also be affected). Sometimes a bit poorer 2nd headstack playback. Sh: BBC#1 lower sideband produced low amplitude signal in N04L1. Also data missing from BBC#7 and BBC#8. Hh: Due to recorder problems lost two scans in GM052B. There were no fringes from VC#1 upper sideband, and probably from all the headstack 2 channels in N04L1 (not pipelined yet). Mh: In F04K1 the TPI of IF distributor C was stuck. Phasecal was not connected during the session. High parity errors on the higher numbered tracks (25-33). Ro: There were some recording problems in GM052A and EB027B. Mk5 tests and ftp fringe tests ------------------------------ We carried out Mk5 three Mk5 tests in September-October. In D03L1 Cm, Wb, Nt and On participated. Only Onsala seemed to be OK at 1 Gbps. Cm and Nt failed at 1 Gbps for an unkown reason, while Westerbork had only 4 disks in the disk pack, which caused a problem at the highest datarate. In D03C4 Mc, Ef, Cm, Wb and Nt joined the observations. The data were checked in Bonn. All the stations produced fringes at 1 Gbps. In D03C3 (observed after D03C4, during session 3/2003) Ef, Mc, Wb, On and Jb participated. The first four stations produced fringes up to 1 Gbps recording rate. Jb did not produce fringes because of an LO offset (see above), but the recording seemed to be good. We carried out ftp test in all of these Mk5 tests, and also during session 3/2003 in the fringe tests and NMEs. D03C4 was the first experiment from which the ftp data was processed with the CRL Software Correlator. A perl script was written to run the software on certain baseline combinations. The processing steps involved transferring K5 files to Mk5 format, correlation, and making the ftp web pages from the fringe plots. In session 3/2003 we reported back on fringes tipically within a day after the data arrived to JIVE. This way we quickly realized that there was a problem with Jb in the session. In session 1/2004 there was only one ftp fringe test, F04C1. We reported on fringes to Hh, Jb, Nt, On, Wb and Mc. For Ef there were no fringes. Thanks to the quick identification of the problem, the LO offset in Ef was fixed before the C-band session started. Network Monitoring Experiments ------------------------------ In session 3/2003 there were three NMEs: F03X1, N03C1 and N03L3. F03X1 was a general NME recorded at 256 Mbps in RCP only. N03C1 was processed fully for checking polarization purity of the stations. In particular we were interested in the characteristics of the new adding box in Westerbork. The polarization leakage in Wb was much less then earlier, it was about 5% in the C-band. This was a great improvement. In N03L3 the Wb leakage was a bit higher in VC#2 (~10%), the other channels were good. One scan of N03L3 was recorrelated with a source position offset of 2 arcminutes. We used 512 lags and 1/4s integration time. The results were consistent with the normal correlation. The goal of the NMEs in session 1/2004 was to prepare at the stations for the 1 Gbps observations in 2004, and check the available bandwidth at each telescope. In order to let stations join without a Mk5 recorder, we scheduled 512 Mbps experiments using 1-bit sampling. N04K1 and N04X1 was recorded in LCP and RCP, respectively (256 MHz bandwidth). N04L1 and N04C1 was recorded in both polarizations (128 MHz/polarization). It seems that the stations can observe in the K and X-bands without any problem. The C and L-bands are more problematic. In the C-band Wb has a 100 MHz IF-band limitation, while in the L-band Mc has even stronger limitations, practically only half of the channels produced good data. There were also failures like missing channels at other stations as well. Note that Wb again had high polarization leakage (~30%) in N04L1, the reason was not known. At the end of session 1/2004 we carried out fringe tests in the UHF(F04U1) and 5cm (F04M1) bands. In F04U1 there were fringes to all stations: Wb, On, Nt, Ur and Tr. In F04M1 there were fringes to Wb, Nt and Hh, there were no fringes to Mc. Zsolt Paragi for the Science Operations and Support Group (JIVE)