Observing Run Report: May 18–21
Date: May 28, 2003
From: Mark Colavita, Andrew Booth, Peter Wizinowich
Summary
The run was Sunday May 18 (1st half), Mon-Tue May 19-20 (full), and Wed May 21 (Caltech, full).
The run went fairly well. The number of engineering tests was somewhat limited because of the bad weather Sunday night, but we did get most of the major ones done, including our nightly performance test; a long lock test to look for evidence of the visibility variations we saw in April; a test with the fringe-tracker target dither; and a vibration identification test to correlate a suspect machine with some lines seen in the data. Several internal tests were also performed during the weather downtime including calibrating the spectrometer wavelengths.
5 science sources were observed, including a Seyfert galaxy for internal team – the first extra-galactic object detected with infrared interferometry – 1 main sequence binary for the Kulkarni team, 2 YSO's for the Monnier team, and 1 YSO binary for the Caltech team.
General priorities that were identified for this run
· Science observing (~2.5 nights)
· Engineering and calibration tests (~1 night)
These priorities were achieved, although the quantity was limited due to weather on Sunday and Wednesday.
Future scheduled runs
· June off
· Mon Jul 14 - Thu Jul 17: 0.5, 1, 1, 1
· Wed Aug 06 - Fri Aug 08: 0.5, 1, 1
· Sept off
· Sun Oct 12 - Thu Oct 16: 0.5, 1, 1, 1, C (C= Caltech science full night)
· Fri Nov 14 - Sun Nov 16: 1.0, 1, 1
· Dec off
· Mon Jan 05 - Thu Jan 08: 0.5, 1, C, C
Engineering summary
References
Original engineering plan: http://keck-lib.jpl.nasa.gov/Get/File-3642.
Detailed engineering logs during the run: http://keck-lib.jpl.nasa.gov/Get/File-3658.
May punch-list with final status: http://keck-lib.jpl.nasa.gov/Get/File-3642.
New July punch-list: http://keck-lib.jpl.nasa.gov/Get/File-3657,
Engineering summary from Andrew Booth
First half night was mostly clouded out. The second and third nights were clear (second night started out partly cloudy). The third night started relatively clear but later became cloudy with bad seeing, so we were unable to get good images on AO after midnight.
No major telescope, AO, or IF technical problems. Some time lost to: telescope – pointing errors on K1; loss of s/w control on K1 (twice, needed complete re-init; seems to be network issue); K1 AO rotator faults during the AO nighttime setup; poor tip-tilt performance (driving 1Hz oscillations) on K1; IF – CT poor alignment/rotator problems; poor auto-peaking on faint/low Strehl sources.
Significant issues
CT – difficult to align and K1 CT often noisy on scope. Noted that the auto-align algorithm often drives the light off the hole in the mirror (as seem on the align camera return). Perhaps need to change range of algorithm. Contributed to perhaps 5% of time not on fringes after lock on AO.
Aligner CPU failures (and less often TO). Had to reboot align 3-4 times per night. Even after reboot, sometimes the seq state of the CT aligner was not reported. Symptom of failure is usually the WL and Las voltage control panels blanking.
The slider corner cube init takes a long time and places it a long way from on-beam. I wonder if its init point can be moved to a better place?
Two major melt downs of K1 telescope control s/w (had to be restarted completely). Noted on the second occasion that there was a problem with one IF application failing to find required keywords. John comments that “gets” on CA are much worse for the tel systems than are monitors. Tel s/w has been overwhelmed before by gets. Need to check the use of CA get in seq and other apps that use tel az and el.
Auto-peaking is not great, especially on K1. Noted that the required image moves seemed bigger on last night when we were at large zenith angles. Probably the DCR needs to be tuned. Especially noisy on faint stars: 1) maybe need to make parameters dependent on star mag, 2) maybe need to make algorithm intelligent and judge when to finish based on SNR.
Aligner mode in for FATCAT works well. Needs a background facility and a way to report the true pixel position, as well as the relative one. Need to make the transition from alignment mode to fringe-tracking mode more benign. Need to get either 1 image or 4 working. Need to link the Make files for the aligner and track waveforms so that the pixels are at known offsets.
Diagnosed the problem with lock offsets for the fringe tracker: going to idle mode causes values to be zeroed when they should persist. Fixing this will make fringe finding much more efficient. (Also needs a seq modification to do an update after the first lock of a sequence).
Would be good to be able to limit the fringe-tracker search space to small range, instead of an infinite spiral, when observing faint, low-visibility targets. Missed fringes can cause long search excursions that are erroneous. Likewise or instead, would be good to make the search restart without loosing previous locks in a sequence.
Operators often made mistakes with the tracking on fdl1 to get to the initial offset for fringes on a new target. We need to 1) fix the jumps so that we don’t have to do this, 2) make the sequencer take care of it [this was on the April punchlist, but did not get completed in time]. Also, note that the track on the FDL reports lock before we’ve reach low laser error, so the sequencer will have to be a little careful.
Daytime tests completed
D0. Did a long lock on internal fringes. Also did shorter locks on a) fringes with nominal spec wavelengths, b) wavelengths from FTS scan on Sunday, c) same, but with FT dither on, 4) same but with mismatched stroke.
D1. Not done
D2. FATCAT PZT aligner stability. PZT box was used at start of every night to align WL and spec pixel (using 2.1W filter – cannot see narrow filters on align image). Find at most about 0.5 pixel drift over 24hrs. Note, for correct alignment of spec pixels, 2.1W is two pixels to right of (bigger pixel number) the first read out pixel (which is nominally 2.3um).
D5. Check of 29Hz code clean up. Done, all OK.
D7. CT rotator home check. Done. No missed homings noted in repeated homes.
D8. Characterize ND filters for FATCAT. Wedge is ~1KAT pixel. Trans is about 40x.
D4. Hand paddle test. Not done
D9. FATCAT lock offset test. Done… see above for result.
D10. Auto-peaking variability test. – Not done
D3. tsvr crash test. Done Starting 3-4 GUIs simultaneously crashes tsvr on telemetry connections. Will discuss with Rich and Glenn.
D6. FATCAT out of socket test. Done. No evidence that GUIs or seq (when correctly started and shut down) cause excess socket use… all sockets cleaned up on correct shutdown.
D?. 29Hz par-osc at KAT 20Hz. Not done.
Nighttime test summary
N6. KAT 20Hz. Consider as done. On 3rd night have several scans on calibrators with KAT at 20Hz and 100Hz. Check to see if data are consistent.
N7, N8, N9. KAT search, DCR tuning and autopeak tuning. Not done. During first night single telescope time the K1 telescope had pointing problems until after K2 phasing was complete.
N3 Dome fan test. Done see engineering log
N0. Done on all 3 eng nights, but not on Caltech night.
N4 Glycol chiller test. Done on Wed pm - See log by Chris N.
Stroke checks. Done on Wed night. See fdl.log file for records
Uptime monitoring
Averaged over the run, the facility uptime was 83% (vs. req. of 80%; goal of 90%), and the interferometer uptime was 95% (vs. req. of 90%). Most of the facility losses were from K1 problems on the first night. Details are in the (embedded) spreadsheet, which includes worksheets for each night, and also in the engineering log.
Scans per hour
From Rafael: as for April 03 run, I have calculated the averages using for each night chunks of time corresponding only to normal observing (no engineering tests or funny business). But this time I have also subtracted from the time intervals in each chunk a time given by 15mins times the number of entries in the *.stats file in that interval (to correct for small instrument down periods or bad weather).
I did not use 2003139 (1/2 eng night) or 2003142 (CIT clouded night).
These are the results:
2003140
average: 3.3 scans/hour
peak: 4 scans/hour
2003141
average: 4.9 scans/hour
peak: 6 scans/hour
Phasing summary
The two telescopes were phased prior to the April run and again prior to the May run. The K1 telescope was rephased subsequent to the previous week's segment exchange using tests 598 and 596. Test 962 option A was run on K2 on the first night of the run. K2 starting conditions looked good.
Peter’s spreadsheet
is here: 
Other notes
Problems with the kawela disk pack in the week before the run severely impacted run preparation; resolving or working around these problems needs to get a higher priority, even though we may be a week from the run, because of constraints on software checkout and interference with other summit activities.