Images

No light in images

Symptom
Images taken using a grating (spectral or zeroth order image) look like darks.
Problem
The grating is at the wrong tilt, or the grating cover was not removed when the grating was installed.
Solution
  1. Re-home both grating tilts with the following commands on polo:
    	unstick_grating 3
    	unstick_grating 4
  2. Have summit support personnel check for and, if necessary, remove the cover from the grating.

Images contaminated by red light

Symptom
Images show excess red light
Problem
A red LED inside or outside of the instrument has been left on.
Solution
Send someone up to locate the source of the illumination. One likely place is a red LED on the DEIMOS handpaddle which is lit when the paddle is left in MANUAL mode. Switch it to COMPUTER mode in order to disable the LED. Another thing to check is the plastic housing enclosing the optical encoders on the grating tilt stages, which can be deformed and thus leak light.

Direct images and spectra shifted vertically

Symptom
Spectra and/or direct images taken on the grating appear to be shifted vertically on the CCD images relative to their customary position.
Problem
The grating tilt encoder has lost its positional reference, most likely due to momentary loss of power to the tilt ensoder.
Solution
Re-home both tilt encoders by running the following commands on polo:
	unstick_grating 3
	unstick_grating 4

Bias jumps in images

Symptom
Images show a significant jump or drop in the bias level in one or more amplifier.
Problem
The CCD readout electronics may be too warm.
Solution
  1. Check DEIMOS temperatures by issuing the temps command on polo.
  2. If electronics temps are too high, then have summit staff remove covers on the electronics as needed to improve airflow.

NaN values in images

Symptom
DEIMOS images show data values as NaN.
Problem
The CCD readout electronics are malfunctioning, perhaps because they are too warm.
Solution
  1. Check DEIMOS temperatures by issuing the temps command on polo.
  2. If electronics temps are too high, then have summit staff remove covers on the electronics as needed to improve airflow.
  3. If data values are bad in only one amplifier, it may be a malfunction on the Leach board. Try increasing the bias level by decreasing the value of the VIDOFFn keyword for the affected amplifier.

Computers & Software

FACSUM and compass roses wrong

Symptom
Compass roses display the wrong orientation. FACSUM reports incorrect drive angle on the rotator. The DCS keywords ROTPDEST (demanded position) and ROTPPOSN (actual position) differ by many degrees.
Problem
The DEIMOS rotator control computer (roto) has lost its DCS connection.
Solution
See corrective actions described below.

ds9 will not load images

Symptom
Newly acquired images do not appear on the the ds9 image display GUI, even though the GUI appears to be alive and responds to mouse input. The ds9relay widget is present and active.
Problem
The X server you are sitting at is no longer secure, possibly because someone has used the xhost command to open an insecure line of communication.
Solution
  1. Try issuing the command
    	\xhost -
    to disable all insecure xhost access.
  2. If this does not work, log out and log in again.

ds9 will not load images

Symptom
Newly acquired images do not appear on the the ds9 image display GUI, even though the GUI appears to be alive and responds to mouse input. The ds9relay widget is not visible on the screen with ds9.
Problem
The ds9relay task has died and needs to be restarted.
Solution
  1. Run the ctx command on polo and verify that ds9relay is not running.
  2. From the OpenWindows menu, select the item DEIMOS Control Menu > Subcomponents... > Restart ds9.
  3. Verify that a ds9relay appears. If it does not appear or dies, see the item above regarding xhost problems.

Images are not saved to proper directory

Symptom
An output directory is defined (outdir commands returns valid result) but images are not going there.
Problem
The output directory does not exist.
Solution
Create the output directory using the mkdir command.

Images are written incorrectly

Symptom
Images are saved with strange filenames (e.g., d0217_0001.fits.old.PdGWS5) and FRAMENO keyword does not increment between frames.
Problem
Either the ADDFRAME keyword is set incorrectly or the DEIMOS daemons are in a bad state.
Solution
  1. Run testAll to verify that the ADDFRAME keyword has the correct value.
  2. Re-load DEIMOS default settings from Dashboard Initialize panel.
  3. Shutdown all DEIMOS GUIs.
  4. Kill and restart all DEIMOS daemons using the recommend procedure.
  5. Restart all DEIMOS GUIs and verify that images write properly.

DS9 Regions option inactive

Symptom
DS9 regions option above the image and below the title bar are inactive.
Problem
Regions options are unavailable.
Solution

Images will not start

Symptom
Clicking the Expose button on the Dashboard window appears to have no effect, although testAll indicates that all systems are functional. Watching the output on the science CCD crate reveals the message:
Image aborted: insufficient memory on
      host computer
Problem
The host computer is improperly configured.
Solution
  1. Have the sysadmin add the following lines to the /etc/system file on polo:
    *       Below are changes added locally
    
    *       set the maximum shared memory segment size up for DEIMOS
    *       images to be passed from lickserv to other apps
    set shmsys:shminfo_shmmax=268435456
    
    *       set the maximum number of users up to a useful level
    set maxusers=128
    
    *       by default system uses 2% of RAM; or, as reported by "sysdef -i"
    *       47415296        maximum memory allowed in buffer cache (bufhwm)
    *       which matches what is expected for a system with 2.25 Gbyte RAM
    *       we need enough for a DEIMOS image, and some room to spare
    *       this is measured in kbyte
    set bufhwm=163840
    
    *       We believe that the default paging algorithm causes amazing
    *       slowdowns to happen when writing files that exceed about 80 MB.
    *       If we understand correctly, this may be a remedy.
    *       see   http://www.sun.com/sun-on-net/performance/priority_paging.html
    set priority_paging=1
  2. Reboot the host computer.

DCS keywords missing in image headers

Symptom
Keywords such as RA, Dec, ROTPPOSN, etc., are not found in the image headers. These keywords can still be obtained from the polo command line via show -s dcs commands.
Problem
The watch_ccd process has lost its connection to DCS and needs to be restarted.
Solution
Use this procedure to restart watch_ccd:
  1. Wait for the exposure to end; do not begin another
  2. rlogin keamano -l kics
  3. deimos restart watch_ccd
Newly-acquired images should now have these keywords in the image header. You can verify this by using the fitshead command:
	polo{deimos}112: fitshead d0812_0081.fits rotposn
	d0812_0081.fits:  ROTPOSN = 270.00146389
If a ROTPOSN value is shown, that's good news. If no output is received, then the keyword is missing from the header and the problem has not been fixed.

Keywords missing from image headers

Symptom
Keywords are found (or reported) to be missing from image headers.
Problem
Communication with a keyword service is interrupted.
Solution
  1. View logfile. In general, if there is a problem with keywords missing from a DEIMOS FITS header, the best place to start is to look at the appropriate watch_ccd log file in /local/kroot/var/log on keamano:
    • If it's an FCS image that is missing one or more keywords in its FITS header, look at the logfile df_watch_ccdlog by issuing the command
      	view_logfile df_watch_ccd
      or by selecting the item DEIMOS Engineering > Log files... > Log view... > df_watch_ccd log from the OpenWindows pulldown menu.
    • If it's a science CCD image, look at the logfile ds_watch_ccdlog by issuing the command
      	view_logfile ds_watch_ccd
      or by selecting the item DEIMOS Engineering > Log files... > Log view... > ds_watch_ccd log from the OpenWindows pulldown menu.
  2. Locate errors. Use the timestamp of the FITS file to key to the appropriate timestamped messages in the log file. In such cases where keywords are missing from a FITS file, the corresponding watch_ccdlog file should have error messages about either
    • reading the missing keyword, or
    • reading some other keyword from the same service.
  3. If the former: then try to show the missing keyword; e.g., type the following at a polo prompt:
    	show -s deimot tmirrval
    If that fails, then there is probably a problem either with the dispatcher that serves that keyword or the underlying hardware that that dispatcher controls.
  4. In the latter case: when the FITS file is missing keyword X from service Y but the watch_ccdlog file instead reports an error reading keyword Y from service Y, then the problem may be more indirect. Recall that once watch_ccd gets an error reading any keyword from service Y, it will not attempt to read any more keywords from that service for that image. Thus, an error reading keyword Z will not only cause keyword Z (from service Y) to be missing from that FITS file but any other keywords from that same service which have not yet been collected by watch_ccd for this image will also be missing. In such cases, you should then check the dispatcher that serves keyword Z along with its underlying hardware.

CCD crate will not boot

Symptom
CCD crate is unresponsive. Attempts to reboot via RESET button or power cycling the unit do not succeed. Connecting via telnet to the crate shows the following message printed out every few seconds:
	Timeout waiting for ARP/RARP packet
Problem
The crate is asking for an IP address and is not being given one because the YP service does not know about it.
Solution
  1. Ensure that the appropriate entry for the VME crate appears in the master YP database from which the 'ethers' database is pushed. For the FCS crate, that entry is:
    	00:80:42:0b:4e:8c fcsvmep.keck.hawaii.edu
  2. Change the nsswitch.conf file on keamano so that it resolves hardware ethernet addresses directly from the /etc/ethers file.

Dashboard will not run

Symptom
Dashboard window comes up with many imcomplete components listed, and numerous complaints about keywords it can't access.
Problem
One or more computers or keyword libraries which server keywords are not running.
Solution
Use the testAll or checktl commands to test keyword libraries. You may need to reboot CCD crates or restart software.

No Sounds from Speakers (OLD xbeeper)

Symptom
The xbeeper sounds are very faint and are not eminating from the speakers.
Problem
The sounds on the local machine (hanauma) are not configured correctly.
Solution
  1. Open an xterm on the local machine (hanauma)
  2. Type audiocontrol in the xterm
  3. In the popup audiocontrol window select "line out"
  4. In the popup audiocontrol window deselect "speaker" (yes ... deselecting is correct)
  5. Ensure the volume is set to maximum value.
  6. Kill gui
  7. Test sounds by clicking "apply" in xbeeper

No Sounds from Speakers

Symptom
kEventSounds/soundplay utility does not echo event sounds.
Problem
The soundplay utility may not be running on your machine.
Solution
See the common VNC troubleshooting entry for a solution. trouble.html#vnc9

Roto is hung

Symptom
testAll indicates several apparently unrelated problems: Other symtpoms may include:
Problem
The DEIMOS rotator control computer (roto) is hung.
Solution
Cycle power to roto as described in the instructions on rebooting roto and in the instructions for resetting the rotator controller.

Mechanism Moves

Move times out

Symptom
Mechanism move fails with error message "Error code -1: Someone needs to define an error message for this condition"
Problem
A motor move timed out or otherwise failed in a probably innocuous manner.
Solution
Re-try the move.

No light from grating

Symptom
Spectrum appears to be a dark image, although the shutter appeared to open.
Problem
Grating tilt encoder controller lost power thus sending grating to the wrong tilt for imaging, or technician forgot to remove the grating cover when inserting the grating into the instrument.
Solution
  1. Re-home both grating tilts with the following commands on polo:
    	unstick_grating 3
    	unstick_grating 4
  2. Have summit support personnel check for and, if necessary, remove the cover from the grating.

Dewar filter wheel will not rotate

Symptom
Attempts to change the dewar filter wheel to another position fail. Inspection of the Filter Wheel Select window show that the filter wheel is stuck between two positions and cannot reach either of them.
Problem
A filter has not been loaded properly, and the filter holder is protruding from the wheel.
Solution
Have technician remove and inspect the offending filter, then re-install it properly.

Tent mirror not positioning reliably

Symptom
The tent mirror does not position reliably; the piezo controller may not appear to be functioning; FCS may not be tracking well.
Problem
The piezo dispatcher and/or controller operating the tent tent mirror may be in a bad state, possibly due to a power glitch.
Solution
First, restart the piezo dispatcher following the instructions obtained from the intrcon GUI:
  1. Log into keamano as kics
  2. Issue the command
    	deimos restart dispatcher.piezo
If this fails to fix the problem, have summit technician cycle power on the piezo controller as follows:
  1. The tent mirror actuator is driven by a Physik Instrument P-845 Piezo controller. This controller is located in DEIMOS barrel electronics bay #2.
  2. The piezo controller is located in the outboard position (i.e., farthest from the center of the rotation axis) of this bay while the DEIMOS PXL camera electronics unit is located in the inboard position.
  3. To access the piezo controller, rotate DEIMOS so that bay #2 is accessible, then remove the panel from the rear of the bay (i.e., the panel on the side of the electronics ring that faces away from the telescope).
  4. The main power button is the square red pushbutton located in the lower right corner of the piezo controller's front panel. Switch the controller's power off for 30 seconds, then turn it back on.
Note that you can watch the output of the tent mirror control software (dispatcher.piezo) using the following command from a polo window:
view_logfile -tail tmirr

Filter move succeeds but filter is "Unknown"

Symptom
Following a filter move, the filter name turns to Unknown. Inspecting the filter wheel position on the filter detail panel shows that the wheel appears to be at the proper position.
Problem
The filter wheel is unbalanced and thus when the filter rotation motor is depowered at the end of the move, the wheel turns enough that its raw encoder position no longer matches the defined position within the tolerance of 1000 units.
Solution
Possible solutions are:
  1. Have summit staff exchange filters in an attempt to balance the wheel.
  2. Have Lick software personnel modify the configuration file to increase the tolerance to an acceptable level.

Loss of air pressure

Symptom
Dashboard pops up a warning message "Air pressure has been lost!"
Problem
The pneumatic system on DEIMOS lost pressure, and pressure may still be out.
Solution
  1. If an exposure is in progress, abort it immediately. The loss of air pressure has likely caused the grating to unclamp, and if the exposure continues it will be ruined. If air pressure remains off, the CCD shutter will not close and the exposure will be ruined anyway. But if you're near the end of a long exposure, it's worth trying to salvage it.
  2. On the main Dashboard, open the Tricorder panel and check the barrel and cradle air pressure indicators at OKAY, then a problem exists. Have summit staff investigate the loss of air pressure.
Note that the air pressure is interrupted momentarily once per day around 8:30am by the air water trap purge system. Occasionally, this system may trigger one or more additional purges following at 30 minute intervals if the original purge did not clear the trap.
See also
Notes on air purge

Grating stage will not tilt (timeout)

Symptom
Attempts to change tilt of a grating stage fail. Trying to home the stage using the modify -s deimot G3TLTCAL=1 command yields a timeout error.
Problem
The motor control process for this stage is hung, or the drive belt has snapped.
Solution
  1. Type the command unstick_grating n, where n is the number of the misbehaing grating (3 or 4), in a polo window to return the grating to normal operation.
  2. If this times out, then suspect that the drive belt on the titl stage is broken.

Grating will not GO ZERO

Symptom
Attempts to change tilt of a grating stage to zeroth order imaging position fail. Issuing the gozero command results in an error indicating that the move would exceed the allowable software range.
Problem
Accumulated tilt corrections are too large.
Solution
  1. Check the value of the grating tilt offsets by issuing the command
    	smot gXtltoff
    where X is the grating numger (3 or 4). Typically these parameters are 100 or less.
  2. If the offset is large, (i.e., absolute value greater than 100), then reset the tilt offset and recalibrate the tilt on the stage by issuing the command;
    	gratecal
  3. Re-try the GO ZERO operation.

Grating tilt in reverse limit

Symptom
Attempts to move the grating tilt stage fail. The G3TLTLIM or G4TLTLIM keyword indicates motor in secondary limit. The grating subpanel on the dashboard indicates that the tilt of the grating stage is at a limit.
Problem
The tilt stage has exceeded the allowed range of travel and has hit the secondary limit switch. This cuts power to the motor and prevents further motor moves.
Solution
It is necessary for summit personnel to assist in moving the stage out of a limit. This operation presents a physical hazard to the operator because the stage could move suddenly when it is backed off the limit switch. Accordingly, this task should only be performed by a qualified instrument technician!
  1. Send the grating to the unload position, either by opening the grating access hatch and pressing the appropriate button, or by issuing the command
    	unload_grating 3
    to send slider 3 to the unload position, or
    	unload_grating 4
    to send slider 4 to the unload position. The technician can then access the stage through the grating changeout hatch.
  2. Open the grating changeout hatch. De-power the motor by unplugging the black plastic connector carrying DC power to the motor.
  3. Manually rotate the wheel on the friction drive mechanism to move the stage in the appropriate direction. An indication of the positive direction should be written on the wheel. If the grating tilt stage is in the negative secondary limit, you will need to move in the positive direction, and vice versa. You can monitor the current position using the G3TLTRAW or G4TLTRAW keywords to verify that the stage is moved in the proper direction. The technician may be able to detect the "click" of the mechanical limit switch when the stage moves off the limit.
  4. With hands clear of the tilt stage, restore power the motor by plugging in the black plastic connector carrying DC power to the motor. BEWARE! The stage may begin a tilt move when power is restored.
  5. When the stage is returned to a working state, re-home the tilt stage from the Initialize panel on the main DEIMOS dashboard GUI.

Grating tilt in "bogus" secondary limit

Symptom
Grating stage(s) fails to move, reporting a secondary tilt limit. However, the grating subpanel on the dashboard indicates that the stage isn't close to a tilt limit.
Problem
The secondary limit switch is falsely triggered.
Solution
Ensure via visual inspection that the stage is not in a limit. If needed, manually tilt the stage via the drive wheel to move the tilt for verification. This can be done by sending the affected grating to the unload position (e.g., modify -s deimot gratepos=-3 for the grating 3 unload position) and accessing it through the grating access hatch at the side of the instrument. If the limit switch is giving a bogus error, you can jumper out the secondary limit switch as follows:
  1. Pull back DEIMOS from the science position.
  2. Remove front cladding to access the grating slide area.
  3. Locate the slider in question.
  4. Locate the wiring interconnect panel on teh grating slider.
  5. Disconnect the appropriate wires (detailed description TBD -- ask Wagner).

Grating will not unclamp (clamps won't release)

Symptom
Repeated attempts to unclamp the current DEIMOS grating fail.
Problem
The pneumatic clamps holding the grating will not release.
Solution
  1. Try using the unclamp_grating command to unclamp the grating. By default, this will try 25 times to unclamp. If no success after 25 tries, go to next step.
  2. Have summit personnel disconect the air supply line at the rear of the instrument via this procedure. This should remove the air pressure and allow the clamps to release.

Grating will not unclamp (pin still in)

Symptom
Repeated attempts to unclamp the current DEIMOS grating fail.
Problem
The grating pin remains in even after all clamps are released.
Solution
This is a tricky condition, since with the pin in you can't clamp up, nor can you unclamp. Here's one thing that has worked.
  1. Rotate DEIMOS to ROTATVAL=-90°.
  2. Follow the instructions on the grating clamps page to do the following:
    • close clamps 1, 2, 4, and 5
    • open clamps 5, 4, 2, and 1
    then re-start the motor control software.
This should release the pin.

Grating will not move to unload position

Symptom
The grating will not move to the unload position, because the select stage hits a limit and the tilt move does not execute. GRSELORD=-3 as desired, but GRSELLIM shows a primary limit.
Problem
The move cannot complete due to the limit switch striker being out of position.
Solution
Use keyword-level commands to move the tilt to the unload position as described in this procedure.

Grating select move fails (pin not in)

Symptom
The grating fails to move into position. Move Failed is written below the grating box on the DEIMOS dashboard. Under the grating diagnostics subpanel, the Grating Message reads:
 Can't set GRATEPOS=4, grating locating pin not in
And the pin is out, the position reads -999, and the named position is unknown.
Problem
We are at an unfavorable rotator postion where the gravity load prevents the grating from moving into position.
Solution
If you are trying to move to slider 4, execute the command
clampup_slider4
to attempt to clamp up. Otherwise, simply re-try the move from the dashboard GUI. If the pin will not engage, then rotate to a PA that is 90 degrees away from this location and retry the grating move. If this fails, rotate another 90 and try again. If it is still failing, there may be additional problems.

Grating select move fails (positioning error)

Symptom
Grating move fails with error:
Can't set GRATEPOS=3, motor shut off - positioning error.
Problem
The move failed, most likely due to excessive vibration in the lead screw generating an overtorque condition in the controller. This can happen in the vicinity of the slider 3 load position at certain DEIMOS position angles.
Solution
  1. Rotate DEIMOS to the position at which the grating system is horizontal. This corresponds to “Physical PA” (ROTATVAL) of -90 or +90 as displayed on the DEIMOS rotator subpanel.
  2. Home the grating select stage using the command
    modify -d deimot GRSELCAL=1
    or alternatively use the Initialize panel on the dashboard GUI and click the Grating selectbutton.
  3. If the calibration succeeds, then re-try the original move. If it fails, suspect a mechanical problem.

Troubleshooting motors

See Motor troubleshooting

Slitmask: select move fails with motor powered off error

Symptom
Slitmask move fails; SLMSKMSG and/or SLSELMSG keywords report that move failed and motor is powered off.
Problem
The motor move has failed due to a bad reading on a switch, usually either the one on the slitmask arm or on the door comb.
Solution
Based on one successful recovery from this error, the following procedure is suggested:
  1. Rotate DEIMOS to the slitmask unload position (ROTATVAL=-90) and try the move again. If this works, then quit.
  2. Attempt to calibrate (home) the stage (mmot slselcal=1). This may succeed, in which case you can quit.
  3. Have summit support attempt to move the slitmask selector motor manually.
  4. Attempt to calibrate the stage.

Slitmask: misaligned comb killed motor power

Symptom
A slitmask move fails. The slitmask status reads "UNKNOWN" after trying to select a slitmask and move it into the optical path. Slitmask detail panel on DEIMOS dashboard indicates comb error when attempting to move a mask. Tklogger window warning indicates “misaligned comb killed motor power.”
Problem
A slitmask is sticking on the mask pusher mechanism, triggering the comb sensor. The slitmask system cannot be moved until the mask is freed from the pusher bar.
Solution

Slitmask move fails - slitmask jammed

Symptom
Attempting a slitmask move fails.
Problem
Slitmask may be jammed.
Solution
See procedure for dealing with jammed slitmask.

Slitmask move fails - fiducial hole blocked

Symptom
Attempting a slitmask move fails, but inspection of the masks shows that the masks are not jammed.
Problem
A slitmask may be missing its fiducial hole.
Solution

Slitmask move fails - mask position is read both at beginning and end

Symptom
Attempting a slitmask move fails, error says that slitmask position is detected both at beginning and end)
Problem
Slitmask selector might be jammed, or might have lost its calibration.
Solution

Electronics

Motor controller fault

Symptom
The testAll script reports that the clock on one of the motor controllers has failed, or that the Bekins/Uhaul/Monitor processes on one of the DEIMOS motor controllers are not running.
Problem
The motor controller has faulted.
Solution
Attempt to reset the motor controller as follows:
  1. From the main DEIMOS dashboard, click the daemons button to raise the daemon subpanel.
  2. Use the displaeyd status information on the daemons subpanel to verify the fault reportd by testAll.
  3. Use the Ctrlr 1 Reset or Ctrlr 2 Reset buttons as required to attempt to reset the controller.
If this fails, proceed to the next item.

Motor controller hung

Symptom
The testAll script reports that the clock on one of the motor controllers has failed, or that the Bekins/Uhaul/Monitor processes on one of the DEIMOS motor controllers are not running. Attempting to reset the controller from the dashboard's daemons subpanel (as described above) fails.
Problem
The motor controller has hung.
Solution
Attempt to reset the motor controller as follows:
  1. Log in as kics to keamano.
  2. Use the reset_motor_controller script to reset the appropriate motor controller:
    	reset_motor_controller n
    where n=1 or 2.
  3. After the script completes, use the command
    	deimos status dispatcher2.n
    (where n=1 or 2) to verify that the appropriate dispatcher was restarted properly.
  4. If the dispatcher is reported to be not running, use the command
    	deimos start dispatcher2.n
    (where n=1 or 2) to restart.
If this fails, proceed to the next item.

Problem communicating with Galil controller

Symptom
Although the Lantronix terminal server is pingable, attempts to communicate with the Galil controller fail with this error:
    [1009] kics@keamano% telnet 192.168.6.4 3005
   Trying 192.168.6.4...
   telnet: Unable to connect to remote host: Connection refused
Problem
The port on the Lantronics unit connected to the Galil controller is hung.
Solution
Follow the Lantronix port reset procedure.

Problem communicating with Lantronix unit

Symptom
Motor moves fail. Running testAll indicates a problem communicating to the Lantronix one of the Lantronix units (deits2 or deits3).
Problem
The most likely problems are:
Solution

Ion pump is off

Symptom
The testAll command reports that one or both of the ion pumps is off.
Problem
The ion pumps need to be turned on.
Solution
State the solution to the problem here, numbering the steps to be taken:
  1. Go to the DEIMOS dashbaord GUI and click the Tricorder panel.
  2. Verify that the status of IONPUMP1 and/or IONPUMP2 is OFF. Also check the dewar temp; if the dewar temp is well above -100 C then you may not be able to start the ion pump.
  3. Click the Restart button under the appropriate ion pump to attempt to restart the ion pump. The status should change from OFF to starting.
  4. If the startup is successful then the status will eventually change to on. If the startup fails, the status will return to off, and you should try again to start the ion pump. Even if the dewar is cold, numerous restarts may be required before the ion pump will stay on.

Rotator

Rotator won't go into DCS Control Mode

Symptom
Rotator will not go into DCS control mode; attempt to modify ROTATMOD to DCS yields an error about INSTRUME and FOCALSTN not being set properly for DEIMOS.
Problem
The DEIMOS rotator control computer (roto) has lost its DCS connection.
Solution
Follow these steps to re-establish the DCS connection:
  1. Open a window on polo (or keamano) as user dmoseng (or kics).
  2. Enter this command:
    	deirot_restart
    which will repeatedly reset the daemons until they are all alive.
  3. Optional: once the restart completes, verify that all is well by issuing the command deirot status; the output should resemble this.
  4. The observer must click the Allow DCS Control button and then select the yes option to enable DCS control of the rotator.
  5. The OA must do a Standby/Init on the rotator to re-initialize the rotator.
  6. Note: If restarting the deirot process fails to fix the problem, the real problem may be that keamano has stopped forwarding IP traffic to roto. See this document for details.

Rotator reporting wrong angle

Symptom
Compass roses display the wrong orientation. FACSUM reports incorrect drive angle on the rotator. The DCS keywords ROTPDEST (demanded position) and ROTPPOSN (actual position) differ by many degrees.
Problem
The DEIMOS rotator control computer (roto) has lost its DCS connection.
Solution
See corrective actions described above.

Can't find stars

Symptom
At the beginning of the night, pointing appears to be way off; stars not visible in guider.
Problem
Either the rotator has not been initialized and placed into DCS mode, or the DCS does not know the rotator position.
Solution
  1. Run testAll to verify that the instrument and telescope settings are correct (e.g., selected instrument is DEIMOS).
  2. If the rotator was not initialized at the start of the night, then follow these instructions to do so.
  3. If you have not yet commanded a rotator move, have the OA send the rotator to any angle (e.g., 0°).
  4. Verify on FACSUM that the rotator is in position angle mode.

Rotator spontaneously homes itself

Symptom
When sending a rotator move, the rotator unexpectedly executes a large slew to the home position.
Problem
The rotator software determined that the discrepancy between the readings on its two absolute encoders was too large and decided to home itself in order to reset the encoders before proceeding.
Solution
Allow the move to complete, then proceed with observing. Note that properly initializing the rotator at the start of each night should generally prevent this from happening.

Can't do STANDBY/INIT on rotator

Symptom
At the start of the night, the OA cannot perform the STANDBY/INIT operation on the rotator.
Problem
The rotator software on roto is not communicating properly with DCS.
Solution
  1. Restart rotator software as kics on keamano via
    	deirot restart
    and attempt another STANDBY/INIT.
  2. If that fails, try
    	deirot stop
    then wait 5 - 10 seconds, then issue the command
    	deirot start
    and attempt another STANDBY/INIT.
  3. If that fails, restart DCSGUI.
  4. If that fails, reboot TDC and AUX, then restart DCSGUI and perform another deirot restart.

DCS Rotator Fault

Symptom
At the start of a telescope slew, the rotator faults. The rotator logfile indicates
	Ignoring bad DCS demand #####,D (#successive bad demands=1): Demand - NNN.N (physical angle) is outside limits
Problem
The DEIMOS rotator received a request to go to an invalid rotator position. Usually this indicates a problem in the rotator angle interpolation routine.
Solution
  1. Issue the following command in order to ensure that the rotator is now giving appropriate demands:
    	modify -s dcs rotdest=0 rotmode=stationary
    Failing to issue this command could cause the rotator to fault again after re-initializing.
  2. Perform an INIT (not necessary to do STANDBY/INIT) on the rotator.

Rotator moves are all outside of limits

Symptom
When the OA attempts to send a rotator move, SKY complains that the requested angle is out of range for the DEIMSO rotator although it is clearly a legal request.
Problem
The rotators limits are not properly defined.
Solution
  1. Verify the problem by displaying the keyword values:
    	show -s dcs rotccwlm rotcwlm
    The correct values are:
                          rotccwlm = -330.00 deg
                           rotcwlm = 402.00 deg
  2. If you get zero values back, have the OA re-load DEIMOS as the current instrument.

Rotator server errors are high and status toggles from tracking to slew

Symptom
Problem
watch_rotator, the program that monitors DCS position broadcasts of ROTPDEST and extrapolates the required (x, vel, accel), is issuing velocity and accel values (NaN's). Typically, this occurs when watch_rotator is receiving bad values from DCS (perhaps due to one of those infamously sketchy EPICS/CA connections), and therefore it cannot fit a curve with appropriate velocity and acceleration.
Solution
  1. Log in to keamano as user kics.
  2. Enter this command:
    	deirot restart
  3. After 30 seconds, use the testAll command to check the status of the system. If problems are shown, issue the command deirot status from roto as kics; the output should resemble this.
  4. The observer must click the Allow DCS Control button and then select the yes option to enable DCS control of the rotator.
  5. The OA must do a Standby/Init on the rotator to re-initialize rotator control.

Flashing handpaddle lights

Symptom
LEDs on the DEIMOS rotator handpaddle are flashing.
Problem
The rotator needs to be re-initialized.
Solution
Re-initialize the rotator:
  1. Close all DEIMOS hatches and release the band brake.
  2. Switch the toggle switch on the DEIMOS handpaddle to COMPUTER.
  3. Open an xterm on polo.
  4. Type the following commands:
    	modify -s deirot rotatlck=unlocked
    	rotatcal
    Answer 'y' when prompted whether you are sure you want to home the rotator.
  5. Allow the homing operation to complete.

Rotator disabled due to ESTOP or power outage

Symptom
Rotator will not move or initialize. The Rotation subpanel on the DEIMOS dashboard indicates ESTOP from DCS.
Problem
The rotator is disabled because the Keck II facility ESTOP circuit was de-activated, either because someone pressed a facility ESTOP button or because of a power outage.
Solution
  1. Follow the DEIMOS Rotator Controller Reset Procedure.
  2. Verify that the ESTOP from DCS indicator on the dashboard's Rotation subpanel is no longer lit.

Rotator computer will not boot

Symptom
Rotator is inoperative from the handpaddle. Attempts to log into the rotator computer fail. The Linux PC, roto, does not have any lit power indicator when power cycled..
Problem
The DEIMOS rotator computer is dead.
Solution
Complete the procedure to replace the DEIMOS rotator computer.

DEIMOS will not rotate under computer control

Symptom
DEIMOS will not rotate via handpaddle or computer control.
Problem
The DEIMOS rotator computer is dead.
Solution
For servicing, rotate DEIMOS manually using the manual rotation procedure.

Flexure Compensation System

No light on FCS images

Symptom
No light is visible in an FCS exposure
Problem
Probably one of the following:
  1. CCD shutter is not open
  2. I or Z-band filter is in use
  3. CuAr lamp burned out
  4. Grating cover left on
  5. Grating is tilted side-to-side
Solution
Follow these steps to recover:
  1. Verify that the CCD shutter is open.
  2. If the I or Z band filter is in use, ensure that teh CuAr lamp is turned on; the LED does not emit at these wavelengths.
  3. If still no light, try switching to the LED lamp by requesting a direct-mode calibration image on the FCS Setup GUI.
  4. If still no light, acquire a spectrum with the science array and check whether any light is received. If not, send someone up to check whether the grating cover has mistakenly been left in place.
  5. If still no light, check whether the science spectra appear shifted to the left or right relative to the expected positions based on the slit layout. If so, the grating may not be properly seated.

FCS cannot track (offset too large)

Symptom
FCS image shows light but software cannot lock on to image and complains that the offset is too large.
Problem
The grating tilt is not correct and the image is shifted.
Solution
  1. Verify that central wavelength and filter are set to exactly the correct values for FCS. If they are not, then the FCS GUI will show Lockout status.
  2. Execute the fcsfix.tcl script via the desktop menu option: DEIMOS Troubleshooting -> DEIMOS Trouble Recovery -> Fix FCS

FCS cannot track (xcorr failed)

Symptom
FCS image shows light but software reports "X-correlation failed. Image contrast too low or grating/slider misregistered" errors
Problem
Potential causes are:
  1. The grating tilt encoder has lost its position so the actual position does not match the reported position.
  2. Either the FCS reference image or the new FCS image contains a substantial cosmic ray.
  3. Grating became unclamped due to a loss in DEIMOS air pressure.
  4. Grating was not properly installed in its mount by summit technician.
Solution
  1. Execute the fcsfix.tcl script via the desktop menu option: DEIMOS Troubleshooting -> DEIMOS Trouble Recovery -> Fix FCS
  2. Re-home the grating tilt mechanism by either clicking on the RE-Cal button in the grating detail panel or typing "mmot g#tltcal=1" at a polo prompt, and where "#" is either "3" or "4".
  3. Inspect the FCS reference image for cosmic rays and if needed, re-take the reference image.
  4. Unclamp and re-clamp the grating via the dashboard.
  5. Have summit staff remove and re-install the suspect grating to ensure that it is seated properly in the slider.

FCS won't permit "match previous" option

Symptom
You want to match against a previous observation, but the FCS GUI setup panel does not allow you to select the “Match Previous” option.
Problem
Either the FCSTRACK script is not running, or there are no saved FCS setups which match the current setup.
Solution

FCS attempt to match previous fails

Symptom
An attempt to use the "Match Previous" feature of FCS fails with the error Dewar X translation stage hit a limit.
Problem
The grating is either improperly seated now, or was improperly seated when the original FCS reference exposure was taken. Alternatively, changes to the grating cell, grating system, or grating parameters (e.g., GRATOFFx) may cause the grating to clamp up in a slightly different position and thus shift the images. See also next item.
Solution
  1. Compare Y positions of the same reference spots on the original FCS reference image and the new reference image. Verify that they are separated significantly in Y. If not, then the problem is elsewhere.
  2. Have summit staff unload and re-install the grating.
  3. Re-try the ``Match Previous'' operation.
  4. If ``Match Previous'' continues to fail, then some change in the grating system may be the cause. In this case, matching previous may not be possible.

FCS cannot match previous

Symptom
When attempting to use the "Match Previous" function on the FCS setup GUI to match to the previous night's FCS reference image, the attempt fails with an error "unable to match". Comparing the current FCS image to the previous reference image shows a large spatial offset.
Problem
The grating tilt encoder has slipped and the new image is not properly registered.
Solution
  1. Recalibrate the grating tilt by typing the following command in a polo window:
    modify -s deimot g3tltcal=1
    Note: if using slider 4, specify g4tltcal=1 instead.
  2. Re-try the match previous operation.
  3. If FCS still cannot match, use the fcsmov command to steer the current image so that it matches the previous reference image.

FCS ping-pongs in X

Symptom
FCS issues moves but does not converge on the target, which keeps bouncing back and forth in the X direction.
Problem
The piezo controller/actuator is not functioning properly.
Solution
  1. Inspect cabling at the FCS controller (located in DEIMOS barrel electronics bay #2), particularly the high voltage line coming out the center of the controller.
  2. If cabling looks fine, check that the FCS controller channel is working properly by using one of the other channels on the controller (consult Lick first).

FCSTRACK script dies on startup

Symptom
When attempting to run fcstrack, the script dies with an Subscript out of range. error.
Problem
The script attempted to read an undefined keyword.
Solution
Load the default DEIMOS settings as follows:
  1. On the DEIMOS dashboard, click the Initialize button to bring up the Initialize subpanel.
  2. On the Initialize subpanel, click on Default DEIMOS configuration

FCS Lockout (cannot track)

Symptom
FCS refuses to lock in due to a lockout condition, which is claimed to be a focus mismatch although the focus is correct.
Problem
The focus tolerance keywords are not initialized properly.
Solution
  1. Check the keyword values via:
    	show -s deifcs fcsfoto1 fcsfoto2
  2. If they are zero, kill and restart the FCSTRACK script, which should reset them to reasonable values like 200 units. If not, then you can also perform this step manually:
    	modify -s deifcs fcsfoto1=200 fcsfoto2=200
  3. Kill and restart FCSGUI.

FCS Lockout (cannot track)

Symptom
FCS remains in lockout condition even when the grating, central wavelength, filter and focus values all look correct. It complains that there is no setup for the current configuration. Beware especially after a mask alignment.
Problem
Usually, the central wavelength of the grating is slightly off from reference value. There is finite resolution in the tilt mechanism and sometimes the "Go Back" function of the grating does not set the exact wavelength.
Solution
Move off the central wavelength, then back again.
  1. In the main Dashboard GUI, change the central wavelength by 1 Angstrom.
  2. Return to your desired central wavelength
This should solve the problem.

FCS CCD failure

Symptom
FCS cannot track. Looking at FCS images reveals that one of the two FCS CCDs is not reading out (i.e., half of the FCS image displayed on FIGDISP contains all zero-valued pixels), although the other CCD looks normal.
Problem
The signal chain for one of the FCS CCDs is interrupted.
Solution
An expert technician at the summit will need to check and repair the cabling to make a permanent repair. As a temporary measure, as long as one of the CCDs is reading out the support astronomer can enable partial FCS compensation by telling the FCS software to use only the one working CCD. To do this:
  1. Log in to polo as user kics.
  2. Edit the file /local/kroot/bin/fcstrack.offline_select. Under normal operation (both CCDs working), line 53 of this script reads:
    	set offline = 0
    If CCD 1 (left-hand CCD as viewed with FIGDISP) is malfunctioning, change this line to read:
    	set offline = 1
    If CCD 2 (right-hand CCD as viewed with FIGDISP) is malfunctioning, change this line to read:
    	set offline = 2
  3. Kill and restart the fcstrack script. You should get a warning message:
    	*** FCS CCD #1 has been marked offline ***
    Verify that FCS is now able to track.

FCS dewar stage at negative limit

Symptom
FCS complains of a lockout situation because the dewar stage is at a negative limit. FCS cannot re-position the dewar stage to correct for differences between the FCS reference image and the current FCS image.
Problem
One problem may be that the grating is improperly clamped either currently or when the reference image was acquired in the afternoon. If the grating is improperly clamped, the result could be large shifts between FCS images.
Solution

FCS fails to cross correlate images

Symptom
FCS images are flipped left/right or the spectrum appear shifted left/right relative to the FCS reference image. When FCS is tracking, large cross correlation errors are reported by the FCS error handler.
Problem
This problem is attributed to attenuated signal in the fiber optic digital communications link between the FCS CCD controller (inside DEIMOS) and FCS CCD VME crate (in the Keck-2 computer room).
Solution
This problem may not be solved at night. A technician will need to clean the FCS fibers and reseat the boards during the day.

FCS cannot save images. FCS ccd crate crashes

Symptom
When taking reference image or starting FCS tracking, the ccd reads out but no image is saved and a 30 seconds timeout warning appears. The FCS ccd crate become unresponsive.
Problem
Wrong directory permissions: the account running the fcstrack script does not have write permission to the current output directory.
Solution
Compare the group membership and the write permission of the current outdir with the other numbered (user) accounts. In at least one case, we found that the outdir belonged to group deimos rather than to the group instr. Have the SA check the proper group memberships, and if neither you nor the SA can change them, request the help of the software on call person. Unfortunately, this problem also crashes the FCS ccd crate which needs to be rebooted and restarted.

FCS saves images with wrong name

Symptom
When taking reference image or starting FCS tracking, the CCD reads out and displays an image on FIGDISP but the image is saved under the wrong name (e.g., fcs0504_2057.fits.old.rNAkGG)
Problem
The FCS watch_ccd daemon is in a bad state.
Solution
Restart the watch_ccd program.

FCS power is disabled

Symptom
The testAll command indicates that power to the the FCS CCD system is disabled:
  Checking FCS CCD 15V Power.......ERROR! Current value 'disabled' should be 'enabled'
  Checking FCS CCD 30V Power.......ERROR! Current value 'disabled' should be 'enabled'
Problem
Although the power may be off, the more likely issue is that the FCS CCD system was reading out at the time you ran testAll
Solution
Re-run testAll and check whether the system still reports an error. If necessary, stop the FCS system by issuing the command
mfcs fcsmode=off
and then re-run testAll to check status.

TV Guider

Guider is not responsive

Symptom
Guider cannot be selected from Xguide.
Problem
Guider is powered off or requires a power cycle.
Solution
Cycle power to the guider off and on by executing the following command on polo:
	tvpower cycle
This should take about one minute to complete.

Bad guider images

Symptom
Guider can be selected from Xguide but images from the guider do not show stars or only a portion of the field (top 1/3) show stars.
Problem
Either the guider is not properly configured or the shutter is broken.
Solution
Follow these steps to determine source of problem:
  1. Verify that the DEIMOS hatch is open from the main Dashboard panel.
  2. On the Dashboard's TV Guider subpanel, verify that the TV filter is set to a reasonable value (e.g., R). If it is set to Home, select R.
  3. On the Dashboard's TV Guider subpanel, verify that the TV focus is set to a reasonable value for the selected filter (e.g., 970 for R). Note that 1 is not a reasonable value.
  4. Note that the default value for the guider filter and focus is R & 970. If you suspect that the defaults should be in use, reselect these values.
  5. If still no good image, close the hatch, turn on the Qz lamp, and see whether any light is seen on the image.
  6. If no light is seen, the guider shutter may be inoperative. Replacing this is an extensive job for the day crew.

Bad guider images

Symptom
Light is seen on the guider images, but the images appear "smeared" horizontally.
Problem
The serial clock card in the guider controller is malfunctioning.
Solution
Repairing this is a full day's work for a talented daycrew. Have daytime summit staff pull DEIMOS, remove access panels on front of instrument, and check serial card in the guider head.

Poor guiding in vertical angle mode

Symptom
When offset guiding in vertical angle mode, the guide star is stationary but the target does not stay in the slit.
Problem
The wrong guiding mode has been selected, so the center of rotation is set to the guide star rather than the slit.
Solution
Please see table on guiding modes. This is a case of offset guiding. Center of rotation needed to be set and then start guiding with "On Star" button.

AUTOFOC and Mira

AUTOFOC returns abnormal telescope focus value

Symptom
After running AUTOFOC to focus the telescope, the secondary mirror position is more than a few tenths of a mm away from the nominal value of 0.8 mm.
Problem
The TV camera focus is not set correctly.
Solution
  1. On Dashboard, click on the TV Guider button to call up the guider window.
  2. Check the displayed TV camera focus value, which should be around 1000 units.
  3. Change this value as needed and click the MOVE button to make the change.
  4. Re-run AUTOFOC and verify that the value has changed.

Mira images are badly out of focus

Symptom
When running Mira, the images are badly out of focus.
Problem
The telescope is badly out of focus, possibly because the DEIMOS TV camera focus is not set correctly.
Solution
See the description for AUTOFOC returns abnormal telescope focus value.

AUTOFOC crashes

Symptom
Autofoc crashes with the following error message:
	can't connect autofoc to aut_srv_task
Problem
The wrong version of AUTOFOC was run. There is a SunOS version for use with the SunOS guider, and a Solaris version for use with the Solaris guider.
Solution
Run the correct version of AUTOFOC.

MIRA fails to reset parameters

Symptom
Data directory, filename prefix, etc., are not properly reset after running MIRA.
Problem
The MIRA script failed to complete.
Solution
Have the OA or SA restore parameters as follows:
  1. Open a polo window as dmoseng
  2. Execute the command
    	restore_state -verify
    This should restore the pre-MIRA settings.

MIRA image shows gradient

Symptom
MIRA images show a steep top-to-bottom gradient in the brightness of the segments.
Problem
This is not generally a problem, just a benign symptom of using the 900-l/mm grating.
Solution
As long as all segments can be identified and MIRA completes without error, no additional action is required.

Slitmask Alignment

Xbox Fails (Image not Written)

Symptom
When xbox is run, it fails before analyzing the first box, with an error akin to
	extension BluSlits not found
Subsequent attempts to analyze this same image yield the same error.
Problem
The xbox program attempted to read the image before it was fully written.
Solution
  1. Type flpr and re-run xbox.
  2. If that fails, restart IRAF and re-run xbox.
  3. If that fails, inspect the image header (execute the command fv -cmap 2 filename on polo) and verify that the specified extension exists in the image header.

xbox Fails (Missing FITS Chunks)

Symptom
When xbox is run, it fails before analyzing the first box, with an error akin to
	extension DESIslits not found
Subsequent attempts to analyze this same image yield the same error.
Problem
The FITS header extension called DesiSlits is not being read by the software, either because it does not exist in the image header or because IRAF can't read it.
Solution
If you are trying to get a slitmask aligned and are pressed for time, proceed straight to the procedure for aligning slitmasks without FITS chunks. Otherwise, you can proceed as follows: If FITS chunks are missing, it is likely that something went wrong in the process of scanning the masks. Follow the procedure for dealing with a mask scan error, then take these additional steps:

xbox Fails (Bad box)

Symptom
When xbox is run, it crashes with a floating point error when it tries to analyze a certain box.
Problem
A truncated box (i.e., one that is at the edge of the CCD) may cause this to occur. The mask must be re-analyzed with the offending box excluded.
Solution
  1. Identfy the bad box in the ds9 display (not the one being used with IRAF; refer to the one displaying the outcoming DEIMOS images). Note the PANE corodinates of the bad box.
  2. In IRAF, run the command get_boxes image, where image is the name of an alignment image, to generate a "boxfile" (text file) listing the boxes (generally named box.maskname).
  3. Edit the boxfile. Locate the line corresponding to the bad box by comparing the PANE coordinates. Delete the line and save the file.
  4. In IRAF, run xbox from the command line using the box file as input:
    	xbox image input=boxfile

xbox reports STAR SKIPPED

Symptom
When xbox is run, it beeps and complains STAR SKIPPED when attempting to locate certain stars.
Problem
The alignment boxes were not found at the expected locations in the alignment image, probably due to either uncorrected flexure or changing to a different grating.
Solution
  1. In IRAF, execute the command
    	tune_qmodel last
    to adjust the pointing model so that the guide boxes are found. Note that after you mark the location of the alignment box, the program will invoke xbox in practice mode.
  2. Re-run do_xbox to compute and send telescope moves.

Mask Milling

Mask cannot be milled , xaxis exceeded

Symptom
When the summit mask milling maestro attempts to mill a mask, the sequence fails with an error like: "Block 43, error 401 x-axis software over travel. press esc to continue."

Symptom 2 is that the Milltronics control panel display does not

Problem
On occasion, the memory card in the Milltronics system may become corrupted and lose the stored parameters from its memory. This may cause the mill to drive the table into a limit or refuse to mill a mask because the coordinates are illegal. This results in an x-axis over travel condition.
Solution
Page 40 of the Slitmask Milling Manual describes the Parameter Recover Procedure. Please see: slitmask Milling System Manual of Operations

DREMEL

Bad response when scanning barcodes

Symptom
When scanning a filter or grating barcode, the instrument responds with a "bad" signal (3 short honks).
Problem
The stage was not at a defined position. Unless the position is defined, DREMEL will not accept the barcode.
Solution
Verify that you have a green light from the stage's position indicator before you scan. If not, re-select the position using the local controls on the instrument.

No response when scanning barcodes

Symptom
When a barcode is scanned using the handheld barcode scanner, the scanner beeps an acknowledgement but DEIMOS does not give the "honk" response.
Problem
Either DREMEL or the bargun dispatcher is hung or otherwise unresponsive.
Solution
  1. Verify that the DREMEL and Hand Held Barcode Scanner lights on the intrcon GUI are on. If not, restart these processes using the instructions available by clicking with the left mouse on the name of the process.
  2. Verify that DREMEL is really responsive:
    • On the DEIMOS dashboard, double-click on the Slitmask box to launch the Slitmask Panel.
    • On the Slitmask Panel, click the dremel button at the lower left to launch the dremel monitor panel.
    • Click the Hello? button and verify that the message Hello! appears at the top of the panel. If not, restart dremel per the instructions on intrcon.
  3. Watch the dremel and bargun logs by issuing the following commands in a polo window:
    	view_logfile -tail dremel
    	view_logfile -tail bargn
    Also, issue the following command on polo to monitor broadcasts from the bar gun dispatcher:
    	cshow -s deimot bargnget
  4. Have a summit technician scan the barcode on a grating or filter, or scan one of the barcodes on the laminated sheets attached to the instrument.
  5. If the bar gun beeps but no entry appears in the bar gun log, restart the bargun dispatcher as follows:
    • Log into keamano as kics
    • Type the command:
      	deimos restart dispatcher.bargun
  6. If the bar gun is functional but no entry appears in the dremel log, restart dremel as follows:
    • Log into keamano as kics
    • Type the command:
      	deimos restart dremel
  7. Repeat the above steps and verify that dremel now recognizes the bar gun signals.

Slitmask scanning error (bad barcodes)

Symptom
The automated Count/Scan operation to get the slitmask barcodes terminates with an error about mask scanning, indicating either that an incorrect number of barcodes were received or that invalid barcodes were scanned.
Problem
The barcode reader incorrectly scanned one or more barcodes.
Solution
Follow these steps to manually fix the barcode list. Note: unless you really know what you're doing, it is best to leave this to a DEIMOS expert.
  1. In the main DEIMOS dashboard, double-click in the brown slitmask box to load the slitmask subpanel.
  2. On the slitmask subpanel, click the dremel button to load the DREMEL subpanel.
  3. On the DREMEL subpanel, click the Emergency/Diagnostic Tools button to launch the dremel Emergency/Diagnostic Tools subpanel.
  4. On the dremel Emergency/Diagnostic Tools subpanel, click the Get barcodes button. The current barcode list will be copied into the text box below that button.
  5. In the text box, enter a comma-separated list of the correct slitmask barcodes starting from slot 2 and continuing through the last occupied slot. Be sure there is a one-to-one correspondence between barcodes and occupied slots (as indicated by the DREMPOPS keyword in the DREMEL subpanel). Do not press the Tell dremel button yet!
  6. On the DREMEL subpanel, verify that the DREMFITS keyword reads
    	Completed around...
    which indicates that the FITS-writing process has run to completion. If not, then the process may be hung and you have a bigger problem which may require a keamano reboot.
  7. On the dremel Emergency/Diagnostic Tools subpanel, click the Tell dremel button to force regeneration of FITS chunks.
  8. On the dremel sunpanel, click Commit! to save changes.
  9. Wait for the DREMFITS and DREMCMIT keywords to read
    	Completed around...
  10. Remember that the dashboard must be restarted in order to show the revised list of gratings, filters, and slitmasks.

Review Config shows wrong data

Symptom
When the Review Config (aka tkremel) widget is launched from the DREMEL panel, it does not show the expected changes to the current configurations on filters, gratings, and slitmasks.
Problem
This is a known feature of tkremel with a simple solution.
Solution
  1. Use the Quit button to exit tkremel.
  2. Re-launch tkremel and all should now be well.

Review Config fails to launch

Symptom
Clicking the Review Config button on the DEIMOS does not bring up the tkremel window previewing the new configuration.
Problem
A possible cause is that the resulting configuration is invalid and cannot be displayed, perhaps because the same name appears in two different positions.

As an example, the config below lists two Long1.0B masks with different barcodes. Because the names are the same, dremel fails.

SLITMASK2: Long0.7 BC=8506
SLITMASK3: Long1.0B BC=9090
SLITMASK4: Long0.8 BC=7441
SLITMASK5: LVMslitG BC=8850
SLITMASK6: Long1.2 BC=7359
SLITMASK7: Long1.5 BC=7360
SLITMASK8: GOH_X BC=2126
SLITMASK9: E0b BC=9024-no change
SLITMASK10: uds_ft BC=8852-no change
SLITMASK11: Long1.0B BC=7358-no change
SLITMASK12: GOH_X BC=2124-no change
SLITMASK13: Empty_13 (Bad port)-no change 
Solution
  1. Inspect the file /local/kroot/data/deimot/dyna/dremel.cfg. Check the listings for filters, gratings, and slitmasks looking for identically-named elements in two positions of the same sytem.
  2. If this fails to indicate the problem, then run the dashboard in development mode:
    • Launch dashboard via the command dashboard-dev.
    • From the slitmask panel, click the dremel button to launch the DREMEL subpanel.
    • Click the Review Config button again and check for messages like this:
      ERROR duplicates in stage position data!
      These seem to be duplicated
      GRATE.nam.600ZD GRATE.nam.600ZD GRSEL.nam.600ZD GRSEL.nam.600ZD
      FATAL, CANNOT PROCEED
      # found duplicates in stage data, cannot process
  3. If identical entries are found, fix the problem by either:
    • Removing the duplicate from the instrument if there are two masks for instance with the same name.
    • If duplicate names are not in the instrument, then try re-scanning the elements in the affected stage.
  4. Check that Dremel works as expected
    • From the main DEIMOS dashboard, double-click in the slitmask box to bring up the slitmask panel.
    • Click the dremel button to bring up the DREMEL panel.
    • Click on Review Config and verify that it runs and displays the new configuration properly.

DREMEL can't access database

Symptom
DREMEL dies while attempting to initialize, or at the completion of a slitmask scan when attempting to read recods from slitmask database. The DREMEL panel on the DEIMOS dashboard indicates an unusual failure on the top status line.
Problem
The summit instrument Sybase server is not responding properly and requires a reboot.
Solution
  1. Use ping waiaha to determine whether waiaha is alive.
  2. Have summit staff determine whether waiaha is functioning properly, and reboot if needed.
  3. Restart DREMEL by logging into keamano as kics and running:
    	deimos stop dremel
    	deimos start dremel