Summary

Computers & Software
Red & Blue Mechanism Moves
Guiders
CCD Images
Crates
Shutter
ADC
temps
- Temperature warnings
Reconfig software
- Reconfig software hangs
Ion Pump
- Ion pump current not valid
- Ion pump values all zeros

Computers & Software

XLRIS response is slow

Symptom

LRIS widgets such as XLRIS are slow to respond to keyboard and mouse input on hanauma, but the response time from local windows on hanauma is normal.

Problem

Either lrisserver is overloaded or the network connection to the summit is overloaded.

Solution

If you are running a Netscape window on lrisserver, then kill it.
Check the CPU load on lrisserver by typing the following command in a lrisserver window:
```
  ps -aux | more
```
Check listings under the %CPU column to identify any jobs consuming large fractions of CPU time. Contact computer support to halt such processes.
If no high-CPU processes are found, then another user may be transferring large amounts of data to or from the summit, or the network may be slow, or perhaps lrisserver needs rebooting. Contact computer support for assistance.

Can't reconfigure gratings, slitmasks, filters, etc.

Symptom

Attempts to use the configure command to change the grating, slitmask or filter list give error messages.

Problem

The grating/slitmask/filter list is corrupted in memory and on disk.

Solution

Log into lrisserver as user kics.
Go to directory /usr/common/music/info.
Inspect appropriate file (graname.str, redfilt.str, or slitname.str) for obvious errors.
Create a backup copy of the offending file; e.g.,
```
cp slitname.str backups/slitname.str.11oct1999
```
Edit the file (emacs, vi, etc.) to remove the offending entry.
Determine the process number (PID) of infoman using ct.

Use the following command to halt and restart infoman:

kill -9 154 ; /usr/local/music/bin/infoman l_info v=lris &

Re-try the configure command.

XLRIS dies immediately after startup

Symptom

When XLRIS is started, the window appears and then vanishes within a few seconds without any warning messages.

Problem

The grating/slitmask/filter list is empty or corrupted.

Solution

Type the following command to check the configuration lists:
```
consort
```
This will print out sorted lists of the configurations for all configurable stages. Check for a list which has either
- no entries
- invalid entries
- duplicate entries (two positions with the same string)
If one (or more) of these lists is empty or corrupted, fix it as follows:
- delete bad entries using the syntax
```
configure -d keyword alias
```
- create new entries using the syntax
```
configure keyword alias=N
```
  where N is the stage number to which to assign the string alias
Restart XLRIS.

Blue LICKSERV dies immediately after startup

Symptom

The blue lickserv process dies. Attempting to restart it appears to work briefly, but it dies within a minute of being restarted.

Problem

The observer attempted to window the blue CCD, which is not permitted, and the blue crate is in a bad state.

Solution

reboot the blue CCD crate (connect to the crate from the desktop menu using Engineering > CCD Crates... > TIP line to BLUE CCD crate) and type the command reboot
Restart the blue side lickserv process after the reboot has completed using the pulldown menu item LRIS Control Menu > Subcomponents... > Re-start LICKSERVER... > Re-start Lickserver (BLUE)

It is not necessary to cycle power on the blue saddle bag.

Can't show Keywords

Symptom

Show keywords give solicite the following output

startproc: connect failed
startproc: connect failed
startproc: connect failed
startproc: connect failed
startproc: giving up on connect; exiting ...
Sorry, the show command was not able to contact the control system: Can't 
connect to traffic:

After starting a crate session with start_crate [red | blue], the following output is printed to the screen:

Got network configuration
Broadcast message sent on ln0
Broadcast not sent on lo0
tropen: our host = ted.
read_remote:  pathname=/local/kroot/data/music/services
read_remote:  service file open
tropen:  remote host = lrisserverp
startproc: parameter hostname = lrisserverp
startproc: gethostname returned ted
read_remote:  pathname=/local/kroot/data/music/services
read_remote:  service file open
startproc: connect failed
read_remote:  pathname=/local/kroot/data/music/services
read_remote:  service file open
startproc: connect failed
read_remote:  pathname=/local/kroot/data/music/services
read_remote:  service file open
startproc: connect failed
read_remote:  pathname=/local/kroot/data/music/services
read_remote:  service file open
startproc: connect failed
read_remote:  pathname=/local/kroot/data/music/services
read_remote:  service file open
startproc: connect failed
startproc: giving up on connect; exiting ...
"cserv.c", line 106: cserv, da8398, "cserv ()",
Error, #41: Can't make connection to traffic controller. Unable to make 
connection to traffic controller, retrying...
Timeout waiting for ISERV, retrying

Problem

The low_level_software is not running or has died.

Solution

Restart the low level software

From the pulldown menu:
LRIS Control Menu -> Subcomponents -> Re-start low-level software
Or in a lrisserver window type
```
restart_low_level_software
```

Xpose will not start exposure

Symptom

The Xpose window will not allow you to start an exposure.

Problem

Either the GUI is in a confused state or the relevant watch_ccd process isn't running.

Solution

Try each of the following to see if that re-enables control in Xpose.

Click ABORT in the Xpose window, which should re-enable the START button.
Restart Xpose from the background menu.
From the background menu select LRIS Engineering -> Reset blue CCD flags or LRIS Engineering -> Reset blue CCD flags as appropriate.
Check that the watch_ccd processes are running by typing ctx. There should be a watch_ccd_red and a watch_ccd_blue process. If either is missing, restart it by logging in as lris@manuka and running lris start watch_ccd.

Images do not appear on ds9

Symptom

After images acquisition completes, no image appears on the ds9 image display.

Problem

The ds9relay widget is not transferring data to ds9 becuase the ds9relay is running under some other account. Keep in mind that the ds9relay runs on the linux server (lris).

Solution

Both the ds9relay_red/blue and the ds9_red/blue should run only on the linux server. If they are running on the Sun system they will not display images on the vm-lris.
So please run lris stop ds9relay_red/blue and lris stop ds9_red/blue on lrisserver.
restart the ds9 from the background menu

Images do not appear on ds9

Symptom

After images acquisition completes, no image appears on the ds9 image display.

Problem

The ds9relay widget is not transferring data to ds9 becuase there is a problem with the X server. Keep in mind that the ds9relay runs on the linux server (lris).

Solution

X server may be insecure. Type xhost to check.
If that's not the problem, then shut down and restart LRIS software.
If that doesn't fix it, then log out and log back in.

ctx shows that caRepeater is missing

Symptom

Software cannot be started complaining that another user is running it. Running ctx shows a missing caRepeater.

Problem

The EPICS channel repeater is not running.

Solution

Preferred: from a lris terminal on manuka lris start repeater
Backup: Showing a dcs keyword should restart caRepeater i.e., show -s dcs AIRMASS

Some submenues of XLRIS do not open a popup menu

Symptom

pressing the little G on gratings, or trying to adjust the focus fails to open a popup menu

Problem

The XLRIS is not working properly on the new vm system

Solution

Check the grating configuration by running consort on an lrisserver terminal.
Change the focus by using the focusr value or focusb value commands on a lrisserver terminal

XPOSE Blue will not open the CMD popup menu

Symptom

The popup menu under CMD does not start.

Problem

The blue XPOSE GUI is not working well on the linux VNC system

Solution

The only solution is to restart the blue XPOSE from the background menu.

Red & Blue Mechanism Moves

Grating/turret move fails

Symptom: When changing gratings or adjusting grating tilt, the move fails soon after being initiated.
Problem: The grating brake drive has failed to reach the unclamp limit.
Solution: The solution is to restart the move by selecting 'set' in either the grating selection subwindow, or the wavelength selection subwindow. This will normally drive the brake unclamp to limit, prior to completing the rest of the compound move sequence.

Grating move fails

Symptom: When attempting to change gratings, the move fails. Running check_turret on the motor crate reveals that the grating is ``not at home.''
Problem: The homing operation fails because the puka (hole) in the rotor is plugged and the optical homing switch cannot detect light through it.
Solution: The puka must be cleaned by technical staff. If this does not fix problem, then see below.

The homing operation fails due to some failure of the hardware. The likely hardware failure is that the home switch has failed. The software will not permit you to move a grating if the home switches for the grating tilt are not set.

Solution

An instrument technician will have to examine the home switches. In the mean time, avoid the toublesome port. If the grating is needed for observing, the grating will need to be swapped with a second grating in a port that is opperating. Follow these instructions for manually moving the grating turret away from the failing port.

Have an instrument technician confirm that the grating is tilted to a safe location. An instrument technician has access to the grating port ...
- Safe (homed) grating position: INSERT PICTURE
- Unsafe (tilted) grating position: INSERT PICTURE
If the grating is unsafe, change the tilt at the API level.
- ```
telnet tsred 3016 
```
- ```
:0 
```
- ```
stat 
```
  - and check the current position
- ```
mov  1000  
```
  - move 1000 steps (you may need to move more or less than this value to reach an acceptable tilt).
Once the grating is tilted to a safe position, in a lrisserver window release the grating detent by:
```
 m gdetent=0 
```
At the grating service port, an instrument technician will be able to manually rotate the turret away from the troublesome port.
Swap gratings if necessary.
Update SIAS and the instrument software with any grating changes.

Mechanism move fails

Symptom

When attempting to change gratings, slitmasks, or filters (from either GUI or keyword level), the computer responds as if the move completed properly, but the instrument is unchanged.

Problem

The software is confused. It believes that the stage you selected is already in place.

Solution

Issue the following command from a lrisserver command line to force the software to re-read the instrument settings:
```
m init=3
```
Try the move again.
If it still fails try selecting a different filters, gratings, or slitmasks until one works. Then, return to the slitmask, grating, or filter you desired.

Mechanism move fails (mread select timeout)

Symptom

When attempting to change a mechanism (from either GUI or keyword level), the move fails immediately. The following message appears in either the motor log window or the window from which the keyword command was issued:

move failed:
	mread select timeout

Problem

The motor control software is hung.

Solution

Issue the following command from a lrisserver window to restart the motor control software and all other low-level software:
```
restart_low_level_software
```

Mechanism move fails (drive fault)

Symptom

A requested mechanism move fails with the following message printed in the motorlog:

DRIVE FAULT

Problem

One of the API motor controller is confused and needs to be reset.

Solution

Consult the following webpage to identify the APIs associated with the failing mechanism API ports
open the power control GUI and power cycle the suggested APIs.
Try to move again the mechanism.

If this procedure fails, then a full power cycle might be needed. This is the procedure:

Issue the following command from a lrisserver window to cycle power to the entire instrument:
```
allpower cycle
```
When it comes back up, issue the following command to turn the guiders back on:
```
guiderpower on
```
Use the tdl command to check communication with the crates:
```
tdl red
tdl blue
```
See below for information on dealing with crate problems.

Mirror gets set to GRANGLE=0

Symptom

When completing a mirror move using either XLRIS or a script such as R-band, the mirror gets set to grating angle 0 rather than the nominal angle for imaging.

Problem

Either the mirror move is failing due to a bad limit switch, or the MIRGRANG keyword has been set to zero.

Solution

Check the value of the MIRGRANG keyword using the show command. If it is zero, reset it to the appropriate value for imaging (currently 13.75 deg) using the modify command.
If MIRGRANG is okay, then check the motor log output to verify that the move completed successfully:
```
cdlog
more motorlog
```

Grating tilts stops prematurely

Symptom

Although the mirror (grating position 1 in optical port) can be commanded to any GRANGLE, other gratings cannot be tilted more than about 5 degrees.

Problem

The turret switch is not reporting the proper grating position; hence, the electronics are always trying to drive the mirror stage (corresponding to turret switch output 0000) regardless of which stage is actually in the service or optical port. The likely problem is no signal from the turret switch, which multiplexes the signals to the proper grating station.

To verify the diagnosis, checking the bits on the API unit as follows:

Log into lrisserver
Open a connection to the correct port:
```
telnet tsred 3016
```
Select API 2:
```
:2
```
Show bits on input word 1:
```
ver i1
ver i1
```

Bits 3-6 of word I1 are supposed to encode the position of the grating turret. If these bits are all set to 1 and do not change when gratings are changed, then no signal from the switches is the likely problem.

Solution

Check for a broken wire connecting the turret switch to the electronics.

Trapdoor won't close

Symptom: Attempts to close the trapdoor from within XLRIS fail.
Problem: Due to an overflexible design the trapdoor will not often make it to its closed limit in certain orientations; it is essentially closed for optical purposes, however.
Solution: Try again to close the trapdoor using XLRIS. If it repeatedly fails, then assume that the door is closed and proceed.

Slitmasks won't move

Symptom: Attempts to move the slitmasks result in an error: #25 LIMIT SWITCH ENCOUNTERED. Attempt to move using the keyord (modify -s lris SLITNAME) results in Move encountered an unexpected limit.
Problem: The door of the slitmask cassette is not properly closed.
Solution: Move the telescope to horizon, and have a summit staff close the door properly. When properly closed, the LEDs door indicators will be dark. If the door is ajar, the LEDs will be lit.

Blue mechanism move fails due to latch out of position

Symptom

When trying to move a blue filter, dichroic, or grism, the move fails with the following error:

Error reading crate:  Carousel cell retainer latch is in an i
ntermediate position.  Check air pressure!
See log files for further details.

Problem

The cell retainer latch for the appropriate stage has moved from the latched position to an undefined position in the middle of the move. The carousel cannot be rotated until the latch is latched, but the latch cannot be moved until the carousel is in a valid position.

Solution

Determine which carousel (blue filter, dichroic, or grism) is stuck.
Have summit personnel remove the dust cover on the appropriate carousel and push the latch into the latched position.
Rotate the carousel to a defined position by issuing the command
```
m dichchar=1
```
for example, assuming the dichroic carousel is stuck. Use grischar=1 for the grism and blufchar=1 for the blue filter.
Attempt to deploy the stage.

Red focus or movable guider position invalid

Symptom

Following installation or removal of the polarimeter module, attempts to move the red focus stage or the moveable guider stage fail with a serial port timeout error. Motor log may show an entry similar to this:

Nov 13 16:43:01 [9881] sta_init:  Getting stage position for red_camera_focus   
  
Nov 13 16:43:05 [9881] lrs_mcs.c,1456,mcs_timeout():  Serial port (select) timeo
ut, stage red_camera_focus     , retrying select/read
Nov 13 16:43:09 [9881] lrs_mcs.c,1456,mcs_timeout():  Serial port (select) timeo
ut, stage red_camera_focus     , retrying select/read
Nov 13 16:43:13 [9881] lrs_mcs.c,1456,mcs_timeout():  Serial port (select) timeo
ut, stage red_camera_focus     , retrying select/read
Nov 13 16:43:13 [9881] lrs_mcs.c,1468,mcs_timeout():  Aborting because of serial
 port timeout, stage red_camera_focus     
Nov 13 16:43:13 [9881] lrs_encoder.c,168,read_abs_encoder:  Serial port timeout,
 trying to read AR encoder, stage red_camera_focus

Problem

The polarimeter install/remove script was not completed properly, leaving the encoder is powered off. The system cannot communicate with the red focus and movable guider encoders.

Solution

Turn on power to the polarimeter encoder using the LRIS Power Control GUI.

Invalid controller unit number

Symptoms

Attempt to make motor move fails with the following error message:

Error setting grating: Error reading crate:  
Controller unit number != unit number expected.

Problem

The API unit used for this motor is not echoing command input back to the motor control software.

Solution

Enable command echoing on the appropriate API unit by issuing the following command by connecting to the server:

Log into lrisserver
Open a connection to the correct port:
```
telnet tsred 3016
```
Select the correct API n
```
:n
```
where n is the appropriate API unit number as determined from consulting the Motor Stage Data file.
Set the echo mode:
```
ECHO
	
```

Now, if one types a command in the server, the server would echo that command. If this is not the case then issue the echo command again

Disconnect from the server

Red Filter fails to deploy

Symptoms

When attempting to move the red filter into the beam (deploy), the move fails, and in the motor log, there is the following error:

Dec 1 16:15:08 [734] BEGIN: red_filter_jukebox complete move 
Dec 1 16:15:09 [734] red_filter_jukebox : Moving grabber into box 
Dec 1 16:15:09 [734] red_filter_grabber : Applying power to motor controller 
Dec 1 16:15:09 [734] lrs_mcs.c,345,mcs_com: Error message from API unit received, stage red_filter_grabber 
Dec 1 16:15:09 [734] lrs_mcs.c,346,mcs_com: Error message: S=0,MO 1 
3 "#25 LIMIT SWITCH ENCOUNTERED" 
3> 

Dec 1 16:15:09 [734] red_filter_grabber : Releasing magnetic brake 
Dec 1 16:15:09 [734] red_filter_grabber : Configuring motion-control parameters 
Dec 1 16:15:10 [734] red_filter_grabber : Current encoder position = 1277800 
Dec 1 16:15:10 [734] red_filter_grabber : starting servo loop w/ encoder destination = 1440000. 
Dec 1 16:15:10 [734] lrs_mcs.c,345,mcs_com: Error message from API unit received, stage red_filter_grabber 
Dec 1 16:15:10 [734] lrs_mcs.c,346,mcs_com: Error message: PAUSE,MOV 207616 
3 "#25 LIMIT SWITCH ENCOUNTERED" 
3> 

Dec 1 16:15:10 [734] lrs_encoder.c,543,move_abs_encoder: Bad mcs_com ret=-3079 sending API servo move command, stage red_filter_grabber 

Dec 1 16:15:10 [734] lrs_encoder.c,548,move_abs_encoder: dx=162200, db=50, count=0, max_tries=5, stage red_filter_grabber 
Dec 1 16:15:11 [734] red_filter_grabber : Applying magnetic brake 
Dec 1 16:15:11 [734] lrs_changers.c,194,move_sc: Bad move_float_changer ret=-3079 

Dec 1 16:15:11 [734] lrs_changers.c,196,move_sc: Error moving red_filter_grabber into jukebox 
Dec 1 16:15:11 [734] s_redfnum.c,172: Bad move_sc ret=-3079 
Dec 1 16:15:11 [734] END: red_filter_jukebox complete move

Problem

Either the pin securing the filters is not sufficiently tight, or the door to the jukebox is not securely latched.

Solution

Have summit staff check for a red LED indication on the red jukebox mechanism. If lit, then confirm that the pin and door on the red filter jukebox mechanism are both secure. Note that if the pin is even a fraction of a turn loose, it will activate the LED.

Red Filter fails to deploy

Symptoms

When attempting to move the red filter into the beam (deploy), the move fails, and in the motor log, there is the following error

Aug 14 09:44:51 [2650] red_filter_grabber   :
          Current encoder position = -9520000
Aug 14 09:44:51 [2650] lrs_mcs.c,345,mcs_com:  
          Error message from API unit received, stage red_filter_grabber   
Aug 14 09:44:51 [2650] lrs_mcs.c,346,mcs_com:  
          Error message:  PAUSE,MOV  140288003 "#25 LIMIT SWITCH ENCOUNTERED"

In addition:

Homing the filter grabber and selector is successful.
Keyword moves fail.
The encoder is reading negative numbers.
Possible rollover errors reported (none observed this time).

Problem

It may be that the encoder has rolled over and is now reporting negative encoder values as seen above. This will prevent all moves initiated via keywords and guis.

Solution

OPTION 1: This problem will require manual intervention to address. An instrument technician will need to back off the encoder more than one full turn until it is reading positive values. For the red filter grabber encoder, you can read the encoder value by running on lrisserver absread tsred 13 0 1 0.

OPTION2: If you want to put in the clear filter for spectroscopy, you can manually do this by:

home the filter grabber and selector
telnet tsred 3015
at the telnet prompt type
- :3 to select the grabber mech.
- reset 1 2 3 4 5 6 7 8
- set 6 8
- ver o1 and make sure only bits 6 and 8 are set
- mov -208000
- set 2

On XLRIS the red filter will say "unknown", but it is in the clear position.

The steps presented above to control the hardware at the API level are found in the Motor Control Hardware Troubleshooting guide.

Relevant K1 Nightlog tickets are: K1-18283

Polarimeter waveplate will not rotate to all angles.

Symptoms

The wave plate may move to an angle of 0 degrees and some other angles, but not all the normal angles. As an example, it may move from -90 to +30 but not to the normal angle of +90. Check the plangle and plangler keywords. For a plangle=0, the plangler should be 1946876.

Problem

The parameter file that holds the default encoder value for the home or 0 degree angle is corrupt. This has happened at least twice in the past when a new software version is released. Two known corruptions were that 1) the file dropped the least significant digit for the encoder value and 2) the encoder value was deleted. It is unknown why this happens during a new software release.

Solution

ll /local/kroot/data/lris/plangle.dat to determine which file to edit
The file should contain 1946876. This should be the only number in the file.
Save the file
Try moving the rotator, you should not have to restart the software.
Recheck the plange and plangler keywords.

Calibration Wheel in unknown position

Symptoms

we notice that the calibration unit is in an UNKNOWN position. The calnum is -9999999.

Problem

For some reason, the calibration unit has lost is position. The position is defined by a set of three magnet switches and it is secured in a position by using a detent system. If the position is undefined that measn that the magent switches are not defined and that the detent is not fully engaged. There is no hoiming sequence so we would need to drive it to a known position.

Solution

Cam Diagnostic and slit guider image of the
polarimeter calibration unit in the open posiion. Note also
that the waveplate is at plangle = 0

start taking images with the slitguider (300 milliseconds)
configure lris in the following way:
- trapdoor in the closed position
- slitmask in open
- turn on the Kr lamp
- move the wheel until we see the open position on the slitguider
- telnet tsred 3016
- at the telnet prompt type
  - :1 to select the calibration wheel.
  - reset 1 2 3 4 5 6 7 8
  - set 1 6 7 8
  - ver o1 and make sure that these bits are selected
  - mov 1000
- continue moving until the open position is reached. Verify by applying small moved (10 steps) to make sure the open position is reached and the detent is secured.
- log out from the telnet session
mov the calibration unit to open: calname open
If the detect is fully engaged and the magnetic swiches are set then the new calname position will be open. If not, then the calibration wheel was not in the correct position and the user would need to apply small motions hoping that the magnet switches will be set to Open. Some iteration might be neede. Please note that the telnet session has to be closed before using keywords.

Grism fails to move: Unlatch Failed

Symptoms

Attempts to move the Grism results in the following error as seen in the motor log.

      UNLATCH failed, stage grism_carousel       
      Both carousel positions switches are not set.  Latch will
      not move., #1=1, #2=0, stage grism_carousel

Problem

The flags that are used to indicate that the carousel is in position are not "made" suggesting that the carousel is missaligned. Because the flag are not made, the carousel will not deploy. The grism will not unlatch if one or both of the flags are not made ( #1=1 and #2=0,#1=0 and #2=1, or #1=0 and #2=0).

Solution

Try to nudge the carousel into position at the API level. Please use caution.

determine which flags are not made by issuing the command in a lrisserver window

 cari1 tsblue 16

If in position the output for bits 6 and 7 will read

                   I1 Bit 6 set.      Cell is in position according to Switch #1.
                   I1 Bit 7 set.      Cell is in position according to Switch #2.

telnet tsblue 3016 log in to the terminal server on the blue side
check the flags by stat and looking at
```
                         I1 - STATES, INPUT SET 1   = 11110111
	  
```
Bits 6 and 7 should be set to one if the carousel is in position. If either is 0 then procede to the next step
set 4 - this releases the carousel brake.
mov -500 move 500 steps in the negative direction.

check the flags by stat and looking at

                         I1 - STATES, INPUT SET 1   = 11110111

Try the other direction if the negative direction fails.

If you are unable to set the flag, this position will have to be avoided until someone is able to look inside the instrument and verify the position of the element and the flags.
If the flags are set, try to deploy the grism at the keyword level m gtran=1

Grism fails to move: transport at positive limit

Symptom

Grism fails to change when commanded. The motor log will contain the error

transport at positive limit (DEPLOYED), stage grism_transport

Problem

The transport stage has difficulty lifting the heaviest grisms at certain rotation angles. The 600/4000 and 1200/3400 grisms are the worst offenders.

Solution

Rotate the instrument to a new drive angle and retry. In the past drive angles around +90 were problematic and angles around -90 worked.

LRIS red focus reports wrong control mode

Symptom

Attempts to move the red focus mechanism yield the following error:

	Error setting redfocus: lris dispatcher: Wrong control mode for request
	See log files for further details.

Problem

The red focus mechanism is now in position (POS) mode.

Solution

Follow these steps to reset red focus mechanism to POS mode:

Set the control mode to POS and home the stage via

        modify -s lredfoc rfocmod=Pos  rfoccal=Homed

See the red-side focus documentation for further information.

LRIS red focus is inoperable (timeout, communication problems), after the removal of the polarimeter

Symptom

Attempts to move the red focus mechanism yield the following error:

	Error setting redfocus: lris dispatcher: No communications with device.
	See log files for further details.

Problem

The communication cable for the focus Galil is probably connected to the wrong jack or left umplugged. The cable is a coax cable that plugs in to J15 on the telescope CASS interface panel.

Solution

Follow these steps to reset red focus mechanism to POS mode:

Power off the red focus stage
Plug the cable to the correct jack (J15)
Power on the red focus stage and home it

Guiders

Guider images are saturated

Symptom

Guider images are saturated (signal > 16000ADUs).

Problem

LRIS lamps are on, OR
Sky is bright.

Solutions

Verify that all LRIS lamps are off by showing the lamps keyword on lrisserver:

lrisserver{lris}2: s lamps
 
        Lamp status:  0=off, 1=on, <0=unknown
 
                         lamps = 
        lamp1 status 0
        lamp2 status 0
        lamp3 status 0
        lamp4 status 0
        lamp5 status 0
        lamp6 status 0

If any show status equal to 1 (on) then turn off using the XLRIS gui.

It may be that the object is bright, so the OA should try using an ND filter.

MIRA images have the wrong orientation

Symptom

MIRA images appear to be flipped or have a randon orientation. Analysis is impossible.

Problem

The ROTPPOSN keyword is missing in the MIRA image headers.

Solutions

This is usually a temporary problem, due to a race condition with DCS. Repeat MIRA and see if the orientation is correct. In fact, the problem might only affect one of the two MIRA images, and the MIRA gui can work with just one image. Remove the wrong image from the list and push Analyze.
If the problem persists and MIRA is required, you might have to restart the DCS computer or call software support.

IQL/IQM does not move

Symptom

Starting the IQM procedure, the stars remain single or the procedure does not start at all.

Problem

The IQ prism is stuck and does not enter the light path.

Solutions

Login to k1server as k1obs
go to: cd magiq/lris
execute the command: ./rununstickOffsetGuiderIQL
this command resets the galil (ignore the error message), and then moves the IQL in and out
It will then ask you to confirm that the IQL is moving by providing you with the expected and measured encoder position
if the measured encoder positions are within 50 counts of the expected ones, answer yes, otherwise answer no
if you answer no, the procedure will be repeated
Note that if you do NOT reset the galil with this procedure, all the other offset guider motors might be disabled (filter for example)

CCD Images

Images show severe light contamination

Symptoms

LED-contaminated
image

Images taken with LRIS show severe light contamination with an illumination pattern strongest at the top, resembling the image at right. Spectra of the contamination shows it to be predominantly red in color.

Problem

The light-emitting diodes (LEDs) used in the grating tilt optical homing switches have been left on, probably due to a failure in homing the grating tilt.

Solution

Please consult your instrument specialist for assistance.

Start a motor crate session.
At the API level on the motor crate, type:
```
  :0
  reset 1
  quit
```

Random image rows are reversed

Symptom: CCD images read out in dual-amp mode show strange patterns, with the rows from the left-side of the chip randomly appearing on the right and vice versa.
Problem: Fiber connections between the CCD and the crate are flaky, causing random bits to be dropped and the sense of certain rows to appear reversed.
Solution: Ask Instrument Technician to test all fiber connections to the CCD for light loss.

No light in half or entire image

Symptom

Images taken in single-amp mode appear to have no signal, and images taken in dual-amp moce have single in only one amp but not the other.

Problem

One of the CCD amplifiers is not working, perhaps because the saddlebag is overheating or a card in the saddlebag is poorly mounted.

Solution

Check the saddlebag temperature using the command
```
show -s lris utbtemp
```
If it is above 25°C, then the saddlebag should be inspected for failed glycol or failed fan.
If the saddlebag temperature is okay, then the saddlebag should be opened and the cards reseated.

Images saturated but no lights on

Symptom

Images are saturated (signal above 65000DN) but the trapdoor is closed and no lights are on.

Problem

One or both of the fibers allowing the saddlebag to communicate with the VME crate in the control room are broken.

Solution

Change to a new (good) set of fibers between the NAS deck interconnect panel and the control room interconnect panel.
If this does not fix problem, check the transmission of the other fibers connecting the Leach board in the saddlebag to the VME crate in the control room.

Bias level very high on half or all of image

Symptom

Images show an elevated bias level on half of the image, or on the entire image. Most commonly this is seen in dual-amp mode where the right-side amplifier suddenly has a bias level of over 40,000, as seen in the pre- and post-scan columns. Aside from the elevated bias level, the image may look normal (light can be detected).

Problem

The A/D converter for one amplifier has a stuck bit which is raising all pixel values by some constant amount.

Solution

Try taking a dark image to verify the elevated bias level.
Power cycle the offending ccd crate using the following command on lrisserver (assuming the red side is causing the problem):
```
ccdpower red cycle
```
Remember to issue the lda 0 command on the CCD crate to re-enable temperature control.
If problem persists and affects only one amp, switch to single-amp readout mode and use the good amplifier.
If this is not possible, attempt to shorten exposure times so that the elevated bias level and accordingly reduced dynamic range of the image do not cause saturated pixels for science target.

Image regions are filled with zeros

Symptom

After reading out part of an image successfully, the readout pauses and the remainder of the image if filled with zeros.

Problem

Communications between the CCD crate and the saddlebag were lost during image readout.

Solution

Use the tdl command on the CCD crate to check communications.
It is likely you will find that the crate can't talk to the VME interface card (i.e., tdl 0,1 fails). If so, the card must be reset. The image data cannot be recovered.
After resetting the card, use the tdl command to verify that communications with the saddlebag are restored.
If the problem persists, try switching to single amp readout mode in order to reduce the data flow rate and hence prevent lockups.

Images have central depression

Symptom

Illuminated images show a circular patch in the image center which has reduced flux (20% or more). Objects seen in images may look fuzzy.

Problem

Water vapor has condensed onto the dewar window.

Solution

If the problem is on the red side, check that the dewar window heater is enabled and operating at maximum power by showing the keywords REDWHEAT and WHEATPWR. Normal settings are:
```
	redwheat = on
	wheatpwr = 4400 milliwatts
```
If needed, use the following commands to change them:
```
	m redwheat = on
	m wheatpwr = 4400
```
This should eventually clear the window over a period of several hours.
Only in an emergency situation, the window can be defogged by removing the dewar from the instrument, blowing warm air over the dewar window with a heat gun to clear the condensation, and re-installing the dewar. Do not attempt this procedure unless absolutely necessary and with the assistance of a technician who has removed the dewar before. The dewars are heavy and difficult to manipulate, and disconnecting/reconnecting the dewar cables presents risk to the CCD.

Bias jumps in image

Symptom: Bias level jumps randomly between two values during readout, affecting both data columns and pre/post-scan columns, as shown in this image.
Problem: A cable in the Leach CCD saddlebag readout electronics is loose.
Solution: Inspect and reseat cables in the affected saddlebag.

Bias jumps all over image

Symptom: Bias level fluctuates wildly by dozens of DN across the entire image, as seen in this image.
Problem: A connector is loose, possibly J6 MS connector at cass interconnect panel.
Solution: Inspect and reseat cables at the cass panel.

Image did not readout - Exposure did not complete on the Blue side.

Symptom

The exposure did not readout, but crate seems to be OK. (NOTE: If the crate is NOT ok then go to Crate Crash instructions below). "testAll", "tdl blue", "ctx" all indicate that all processes are running normal. Blue exposures will finished, but the images are not displayed in figdisp and the data is not written to disk. The "Start" button on the XPOSE gui became inactive and needs to be restarted to see the start button active again.

Problem

We don't know what the problem is at this time. Somehow the software got tied in knots.

Solution

These are some things to try

Run LRIS Engineering --> Reset blue CCD flags from the background menu.
take an exposure using the command line: goib
reboot the blue crate
Do a full software shutdown using the desktop menu
restart XPOSE, figdisp, and lickserve on the blue side

Blue image is messed up

Symptom

The blue exposure reads out but looks freakishly discombobulated.

Freakishly discombobulated LRIS blue image

Problem

The blue-side windowing and binning are in an inconsistent state. This can happen if the observer uses Xpose to change the binning without making corresponding changes to the windowing.

Solution

Use one of these options on the CMD... menu in Xpose to reset the windowing and binning to an appropriate value:

Window full frame mosaic (1x1)
Binning 1x2 (spectral)
Binning 2x2 (spatial and spectral)

Blue image has no variation in pixel values on each amp

Symptom

The blue exposure reads out the correct size, but every pixel any given amp has the same value.

Problem

The +30V power supply on the LRIS blue side is powered off.

Solution

Execute the command

	ccdpower blue on

to enable the high voltage power.

Image shows a dark bar with sawtooth pattern

Symptom: The direct image shows a dark bar running across that looks like a hacksaw blade.
Problem: The slitmask move has failed and the grabber bar for the slitmask is in beam.
Solution: Try resending the slitmask move again.

Crates

Crate Crash: exposure stops integrating, or image never readout - Exposure did not complete

Symptoms

Possible symptoms:

Elapsed exposure time stops updating and timer is stuck
Image never reads out
The exposure did not completely readout leaving a partial image in the ds9 window

Problem

The crate crashed. Note that it is possible to save the image if the crash occured before the image readout.

Solution

IMPORTANT: DO NOT try to reboot the Archon without calling the support astronomer. The red side crate requires special handling.

Hard Reboot the Crate: Go to LRIS Engineering -> CCD Crates -> Power cycle blue CCD crate
start_crate blue in an lrisserver xterm window to log into the crate. Monitor the crate session to ensure that the crate boots appropriately.
If reboot is stuck at the VMXWORKS BOOT prompt, type @ and return to initiate the rest of the boot sequence
At the end of the boot sequence, the prompt will read done.
log out of the crate: ^] followed by quit at the telnet prompt.
If the crash occurred before readout began, it is possible to save the exposure. The observer must decide whether it would be valuable to save the image. Run the recover_image blue script at the command line or from the engineering portion of the pull down menu. Script will readout the current data.
take a 1 sec test frame to ensure everything is okay. If the recover image script is run, it prompts you to acquire a test image.
If xpose gui will not take exposure run reset_ccd_flags [red|blue]
If still having problems, run testAll and take appropriate actions.
Note: The same power cycle command is available to OAs in the LRIS specific pull-down menu

Can't boot crate

Symptom

When crate is rebooted, the boot sequence doesn't complete. Logging in via TIP session (start_crate [red|blue] command) a pressing Enter yields only the following prompt:

[VxWorks Boot]:

Problem

The startup script hung for some reason.

Solution

start a crate session using one of the two methods below
- From the desktop menu select
```
	Engineering -> CCD crates ... -> Tip line to [Red|Blue] CCD crate
```
- or in a lrisserver window execute the command
```
	start_crate [red|blue]
```
Confirm that when you press Enter that you only get the following prompt:
```
[VxWorks Boot]: 
```
In the TIP window to the CCD crate, type @ and press Enter. This should cause the startup script to complete. "Done" will appear when the boot sequence is finished.
If the crate was booted after it hung during an exposure or readout, you should reset the ccd crate flags after the boot sequence is complete.
```
 reset_ccd_flags [red|blue|both] 
```
e.g. to reset the flags on the blue side
```
 reset_ccd_flags blue 
```

Can't boot crate

Symptom

When crate is rebooted, the boot sequence fails before the startup script is run. The following error message appears on the crate console (or TIP line):

Attaching network interface ln0... done.
interrupt: ln0 lnInt: no carrier
Loading... interrupt: ln0 lnInt: no carrier
interrupt: ln0 lnInt: no carrier
interrupt: ln0 lnInt: no carrier
interrupt: ln0 lnInt: no carrier
 
Error loading file: errno = 0x3c.
Can't load boot file!!
 
[VxWorks Boot]:

Problem

Communications between the crate and lrisserver are broken.

Solution

Check ethernet connections between the crates and lrisserver.

Sudden inoperation (CCD cannot start an exposure)

Symptom:

CCD system suddenly cannot start an exposure after an extended period of normal operation.

Problem:

It is likely that too many serv_ tasks have piled up on the crate. Verify the problem by starting a CCD crate session (see the solution below) and using the i command to check processes running. Here is an example showing an excesss of suspended tasks:

-> i

  NAME        ENTRY       TID    PRI   STATUS      PC       SP     ERRNO  DELAY
---------- ------------ -------- --- ---------- -------- -------- ------- -----
tExcTask   _excTask       3f8ea0   0 PEND          37f60   3f8cf0  3d0001     0
tLogTask   _logTask       3f6d48   0 PEND          37f60   3f6b98   d0003     0
tShell     _shell         3c5818   1 READY         525cc   3c5208   30065     0
tRlogind   _rlogind       3d3138   2 PEND          6f5cc   3d2d68       0     0
tTelnetd   _telnetd       3d1260   2 PEND          6f5cc   3d0fc0       0     0
tNetTask   _netTask       3f1410  50 PEND          6f5cc   3f1268  3d0002     0
tPortmapd  _portmapd      3cfd20 100 PEND          6f5cc   3cfa30      16     0
MLOG_STDOUT_start_mlog_   2b0da8 100 PEND          37f60   2b0878  3d0001     0
cserv      _cserv         298550 100 PEND          6f5cc   298028  3d0002     0
ccdClock   _ccdClock      27fcf8 100 PEND          6f5cc   27f880  3d0002     0
broad_mon  _broadcast_m   2767f0 100 DELAY         70f2c   2766c0  3d0002  2595
responder  _responder     24b2f8 100 PEND          37f60   24a010  3d0001     0
serv_4644  _s_show_auto   3b7ed0 100 SUSPEND       70924   3b7d40  3d0002     0
serv_4624  _s_show_numa   231a68 100 SUSPEND       70924   2318d8  3d0002     0
serv_4644  _s_show_auto   22f1a0 100 SUSPEND       70924   22f010  3d0002     0
serv_4632  _s_show_voff   22c8d8 100 SUSPEND       70924   22c748  3d0002     0
serv_4644  _s_show_auto   22a010 100 SUSPEND       70924   229e80  3d0002     0
serv_4622  _s_show_ccdg   226710 100 SUSPEND       70924   226580  3d0002     0
serv_4624  _s_show_numa   222e10 100 SUSPEND       70924   222c80  3d0002     0
serv_4624  _s_show_numa   21f510 100 SUSPEND       70924   21f380  3d0002     0
serv_4604  _s_show_binn   21cc48 100 SUSPEND       70924   21cab8  3d0002     0
serv_4630  _s_show_voff   21a380 100 SUSPEND       70924   21a1f0  3d0002     0
serv_4604  _s_show_binn   216a80 100 SUSPEND       70924   2168f0  3d0002     0
serv_4604  _s_show_binn   2141b8 100 SUSPEND       70924   214028  3d0002     0
serv_4606  _s_show_elap   2118f0 100 SUSPEND       70924   211760  3d0002     0
serv_4604  _s_show_binn   20f028 100 SUSPEND       70924   20ee98  3d0002     0
serv_4904  _s_show_raw_   20c760 100 SUSPEND       70924   20c5d0  3d0002     0
serv_4610  _s_show_wind   209e98 100 SUSPEND       70924   209d08  3d0002     0
serv_4606  _s_show_elap   206598 100 SUSPEND       70924   206408  3d0002     0
serv_4606  _s_show_elap   203cd0 100 SUSPEND       70924   203b40  3d0002     0
serv_4502  _s_set_binni   2003d0 100 SUSPEND       70924   200240  3d0002     0
serv_4816  _s_set_utb_d   1fcad0 100 PEND          6f5cc   1fc898  3d0002     0
rccd       _rccd          395390 150 SUSPEND      3fe730   395010  3d0002     0
value = 3951892 = 0x3c4d14

Solution:

We need to reboot the CCD crate.

First, start a crate session using one of the two methods below
- From the desktop menu select
```
	Engineering -> CCD crates ... -> Tip line to [Red|Blue] CCD crate
```
- or in a lrisserver window execute the command
```
	start_crate [red|blue]
```
Reboot the crate by either
- resetting the toggle switch on the VME Sun1E board (white reset button)
- or type "reboot":
```
-> reboot
```
  in the Tip session. Note that sometimes only the reset btn will work.
Watch the output from the crate.
- Upon successful completion of the crate boot, the Tip line session will print done and pressing Enter gives you a -> prompt.
- if boot sequence looks hung, pressing Enter will get you to the
```
[VxWorks Boot]:
		
```
  . If that happens, type @ in the tip session to cause the boot sequence to complete.

VME message exchange errors on CCD crate

Symptom

vmemsgxchng (VME message exchange) errors such as the following are seen on the CCD crate output:

vmemsgxchng:  DSP reply timeout.  Check power/hardware.
E1:Time=209340:"util_mem_rd.c", line 31: broad_mon, 24da70, "util_mem_rd ()",
Error, #1: Undefined error condition. Bad return 12 from vmemsgxchng() sending 
RDM packet

E1:Time=209340:"get_dsp_data.c", line 108: broad_mon, 24da70, "get_dsp_data ()",
Error, #1: Undefined error condition. util_mem_rd returned -1007 reading loc 
40000d of camera 0

E1:Time=209340:"broadcast_ccd_analog_inputs.c", line 72: broad_mon, 24da70, 
"broadcast_ccd_analog_inputs ()",
Error, #1: Undefined error condition. get_dsp_data returned 3 reading UTB adc 
chan 6 from camera 0

Problem

These messages indicate an inability of the CCD crate to communicate with the utility board in the LRIS saddlebag. Possible causes are:

the saddlebag has no power
fiber communications between the VME interface board in the control room and the utility board are bad

Solution

Use the tdl command on the CCD crate to determine the failure point and take actions recommended at the bottom of that web page.

Repetitive messages on the blue crate

Symptom

After a power cycle and reboot the blue crate shows the follwoing output:

Getting digital input word masked with 1 from utility board for camera
0
s_show_utb_dig_input: value = 0

Problem

These messages are produced after a reboot and power cycle of the blue crate. It is unclear if they cause some communication issues that in turn cause more blue crate crashes.

Solution

Rebooting the lrisserver manuka seems to solve the problem.

Unable to take multi-exposures with the red camera while using the GUI or the goir script

Symptom

Trying to start multiexposures fails. The first exposure ends very fast and the second exposure starts before the first one has been read. The following error appears:

Error setting STARTEX: STARTEX: ARCHON_ERROR_STARTEX_EXPOSUID(-13058) cannot start exposure - cycle in progress
ERROR trying to start exposure

Problem

The lrisplus keyword RED_DETECTOR does not change and remains on Idle.

Solution

restart the lrisplus daemon

run the following command lris stop! lrisplus

wait a few seconds

run lris start lrisplus Check that the keyword changes during an exposure. If noit then powercycle the Archon and restart lrisplus.

Shutter

Can't start an exposure

Symptom

When the START button on the Xpose widget is pressed, the exposure does not start. Logging into the CCD crate reveals shutter error messages such as:

check_shutter_status: returning -1103
E1:Time=41488:"s_cshutter.c", line 491: serv_4714, 3b9310, "check_shutter_status ()",
Error, #1: Undefined error condition. Camera shutter is partially open!

E1:Time=41488:"s_cshutter.c", line 318: serv_4714, 3b9310, "do_shutter ()",
Error, #1010: Error number not found in error message table. Bad return from 
check_shutter_status.

E1:Time=41488:"s_set_erase.c", line 234: serv_4714, 3b9310, "set_erase ()",
Error, #1010: Error number not found in error message table. Bad return from 
close_shutter()

E1:Time=41488:"s_expose.c", line 254: serv_4714, 3b9310, "s_expose ()",
Error, #1: Undefined error condition. Bad return from set_erase()

Problem

The shutter is stuck partially open.

Solution

Cycle the shutter open and closed manually as directed in this procedure. Then try pressing the ABORT key on the Xpose widget and attempt another exposure.
If this fails, have technician cycle shutter open and closed manually via the toggle switch on the LRIS shutter controller.
If this fails, have technician cycle power on the LRIS shutter controller.
If this fails, check the cabling from the Leach saddlebag to the shutter control box, and from there to the shutter housing in front of the dewar.

Shutter will not operate

Symptom

Attempts to start exposure or cycle the shutter fail. Logging into the CCD crate reveals shutter error messages such as:

safe_do_shutter:  calling do_shutter.
"s_cshutter.c", line 304: serv_4740, fdb698, "do_shutter ()",
Error, #1010: Error number not found in error message table. CSH command 
unsuccessful, reply was 455252

"s_cshutter.c", line 406: serv_4740, fdb698, "safe_do_shutter ()",
Error, #1010: Error number not found in error message table. Bad return 
from do_shutter()

"s_cshutter.c", line 153: serv_4740, fdb698, "s_cshutter ()",
Error, #1010: Error number not found in error message table. Bad return 
from safe_do_shutter()

Problem

The shutter is not operating.

Solution

Execute the test_shutter command from lrisserver to verify operation of the shutter.

If the shutter fails to move, the script will print the following message:

------------------------------------------------------------------------
	    ERROR: The $side shutter is not functioning properly
------------------------------------------------------------------------

I tried to cycle the $side shutter open and closed, and it FAILED to work.
Should I power cycle the $side CCD saddlebag and the $side shutter controller
echo -n and then try again? (y/n) [y]:

Press Enter to power cycle the components. The script will try again to open the shutter. Quit the script by answering n if it does not succeed on the second try.
If still not working, get onto the CCD crate and use the tdl 0,3 command to verify that the crate can communicate with the utility board, the card that actually commands the shutter to open. If tdl fails, try cycling power to the saddlebag, crate, and/or VME interface card to restore communications.
If you can talk to the utility board, have technician cycle shutter open and closed manually via the toggle switch on the LRIS shutter controller, which is located in the red-side enclosure. The controller has three lights:
- red means closed
- green means open
- amber means unknown
An amber light is bad news and could indicate a mechanical problem, possibly a broken linkage.
If the shutter can opened and closed manually, the problem is not mechanical and hence may be electrical or electronic.
- Check the cabling from the Leach saddlebag to the shutter control box, and from there to the shutter housing in front of the dewar.
- If the cabling looks fine, use a voltmeter to check the ?? cable for evidence of the +5V signal from the utility card to the shutter controller which triggers the open shutter operation. If this is not found, then try to gain access to the utility card and probe ?? for the +5V signal.

Shutter will not operate - Blue Side

Symptom

Exposures on the blue side look like bias images or they have " star trails" above a star caused by the erase and below the star caused by the readout.

Problem

The shutter is not operating normally.

Solution

Execute the cycle_shutter blue 5 command from lrisserver to cycle the shutter open and closed. This may free the shutter..
Execute the test_shutter command from lrisserver to verify operation of the shutter.
If this fails, have technician cycle power on the LRIS shutter controller,
If this fails, have technician cycle shutter open and closed manually via the toggle switch on the LRIS shutter controller.
- Removing the blue side electronics bay cover which is at the bottom of the instrument when the dewars ate at the fill position (3 O'clock when standing behind LRIS).
- locate the shutter control box at the left. It has red and black buttons with a toggle switch for manual and auto modes.
- switch to manual and open and close the shutter.
- Technician should hear a click as the shutter opens and closed.
- If it opens, observes can try automatic shutter mode or choose to observe in a shutterless mode.
If this fails, check the cabling from the Leach saddlebag to the shutter control box, and from there to the shutter housing in front of the dewar.
Instructions for using the trapdoor as a shutter may be found here

ADC

ADC ERROR! galildisp.tcl not running

Symptom: adctestAll reports the following errors:
Checking ADC daemons: Checking galil...................ERROR! galildisp.tcl not running Checking pulizzi.................ERROR! pulizziDispatcher not running
Problem: The ADC daemons are not running.
Solution: Log in to adcserver using
ssh kics@adcserver
and run
adc start daemons

ADC not tracking

Symptom

The ADC Status GUI reports that the ADC Mode is Halt and the Status is Halted when it should be tracking (i.e., the telescope is above an elevation of 30°).

Problem

The ADC mode has changed to Halted for some reason, such as an ESTOP signal.

Solution

Follow these steps to put the ADC back into tracking mode:

Log into lrisserver as lriseng account on a local terminal.
Execute the adctestAll command to check ADC status. Note the ADC Status line, which should be OK if the system is in Track mode.
Check the relevant ADC logfile in /net/adcserver/usr/local/kroot/var/log/k1adclog for clues that might indicate the reason the ADC halted (e.g., ESTOP signal detected).
Verify the current mode via
```
	show -s k1adc adcmod
```
If the reported mode is not Track then execute this command on the LRIS host to put the system into tracking mode via
```
	adcreset
```
If this fails to return the system to tracking mode, you can try to re-home the ADC via
```
	adcinit
```
followed by another
```
	adcreset
```
which should help the system return to Track mode.
Verify that the ADC Mode displayed on the ADC GUI changes to Track and that, after slewing to the correct position, the ADC Status shows as Tracking.
If continued attempts to have the ADC track fail, then null and stow the ADC by executing this command:
```
	adcstow
```

ADC System is "In Forward Soft Limit"

Symptom

Guis report that the system is in a forward software limit. The status message reads:

 In fwd-soft limit

Problem

The ADC is in a forward software limit

Solution

To clear the limit status

Select the "track" option to see if it will restart.
Move to elevation that is legal and select "track" to initiate tracking.

ADC System will not track

Symptom

The ADC will not change modes

Problem

The ADC daemons are not running appropriately

Solution

To fix the problem.

login to host adcserver as user kics using the appropriate account
restart the ADC daemons:
```
	adc restart daemons 
```
Re-initialize mechanisms and enable tracking by issuing this command from any LRIS account on lrisserver:
```
	adcreset
```
From the desktop menus, restart the ADC gui.

Status never says Track

Symptom

Obsserver ADC GUI status never says that it is tracking. Engineering gui indicates that the separation is oscillating between values.

Problem

There is a delay between information being received for updating the position, and this delay is keeping the mechanism motion out of phase with requested position updates.

Solution

To fix the problem

Reset the ADC by executing this command from any LRIS account on lrisserver:
```
	adcreset
```
If problem persists, halt the ADC again, place it into position mode, and send it to the 20 mm separation:
```
	madc adcmod=halt
	madc adcmod=pos
	madc adcval=20
```
Re-enable tracking:
```
	madc adcmod=track
```

Did not reach intended position

Symptom: When in position mode, the ADC does not reach the intended positon because it times out.
Problem: During long moves, the software may time out before the ADC reaches the destination. This is due to a combination of a relatively short time out and slow slew speed.
Solution: Re-execute the move command.

Primary hardware limit set

Symptom

ADC will not move because the primary hardware limit is set

Problem

The optics have moved too close to the end of travel and have triggered the primary limit switch.

Solution

Remote recovery is possible as long as the secondary limits are not triggered.

Change to Position mode by issuing this command on the LRIS host computer:
```
	madc adcmod=pos
```
Set the destination to 20 mm:
```
	madc adcval=20
```
This should get the ADC out of the limit.
Re-initialize via:
```
	madc adccal=homed
```
Start Track mode, to begin tracking:
```
	madc adcmod=Track
```

Primary and Secondary limits set

Symptom: ADC will not move or initialize. The status indicates that the primary and secondary hardware limits are set
Problem: The primary and secondary hardware limits are set.
Solution: This requires manual intervention at the ADC to solve. If this occurs at night, the protocol is to operate LRIS with the ADC stuck in this state. If this occurs in the afternoon when trained staff members are available, then they should address the problem.
To move the ADC out of the primary and secondary limits, a trained staff member has to manually turn the lead screw on the back of the ADC. The ADC does not need to be removed from the CASS position, however, if LRIS in in beam, LRIS may need to be pulled back to gain access to the lead screw. Someone should monitor the ADC status while the lead screw is turned. As the lead screw is tunred, the secondary limit will dissapear and then the primary limit will dissapear in the status message window on the eng and observer guis. After the limits are cleared, ensure that the technician is safely outside the instrument and then re-initialize the ADC. Set the mode to track to test whether the ADC will track appropriately.

temps

Temperature warnings

Symptom

The teperature of the CCD and saddle bags on either or both the red and blue side are out of range. The LRIS dewar status monitor task "/home/lrisserver/lris/bin/monitor_tempdet2" running on the LRIS host computer "lrisserver" may have detected the problem and is now sending e-mail warnings.

Problems

The observers could be acquiring several short exposures in a row.
The saddle bag is powered down
We have lost communication to the crate
The CCD controller has been powered up without temperature control running, and the temperature control loop has just been started.
The dewar is dry of LN2, and is starting to warm up.

Solution

Try running "testAll" on lrisserver
Try running "tdl red" or "tdl blue" on lrisserver and follow instructions if there are problems.

If both tests pass, run "temps" on lrisserver. Typical output is:

[69] lris@lrisserver: temps
System               Current Temp  Acceptable Range  Status
-------------------  ------------  ----------------  ------
Blue CCD temp              -120.0  -121.0 to -119.0  OK
Red CCD temp               -100.0  -101.0 to  -99.0  OK
Blue saddlebag temp          -3.5   -10.0 to    0.0  OK
Red saddlebag temp           -4.5    -8.0 to    0.0  OK

if several temps output indicates that the temperature is converging to the acceptable range, then continue to monitor these messages until the CCD temperature converges.
If several temps output indicates that the temperature is rising, i.e. continuing to warm up, then the dewar is indeed dry and an immediate LN2 fill is necessary, to prevent warm up and loss of vacuum. Contact the support astronomer on duty if this happens.

Reconfig software

Reconfig software hangs

Symptom

Reconfig software hangs while trying to retireve slitmask information from the slitmask database.

Problem

It may be that polo is hung. Polo serves the slitmask retrieval software because it is authenticated with the slitmask database.

Solution

ping polo to see if it is alive
if polo is alive, try to log in as lriseng
if it hangs on login, contact computer support

An example of how polo was hung from 30 July 2008. A nirspec host computer (waikoko) disk was cross mounted on polo. Waikoko was having problems and this hung polo. JC unmounted the waikoko disk and then polo was fine. Then the reconfig software ran normally.

Ion Pump

Ion pump current not valid

Symptom

Current status is not displaying valid values. If it is a problem during the night, troubleshooting that would result in lost sky time should not be completed. The Red dewar should be fine without continually monitoring it. It is not a critical component.

Problem

It may be that the ion pump monitoring software is lost following a power cycle or the ion pump electronics need to be power cycled.

Solution

Start the power gui: LRIS Engineering -> LRIS Power Control
Cycle power on the ion pump. This will cycle power and restart the software.

As lriseng@lrisserver run ionStart

Following a software restart, you may need to wait up to five minutes before a valid measurment is recorded.

For further information see: ion pump technical doc

Ion pump values all zeros

Symptom

Ion pump electronics are not responding appropriately. Under normal opperation, lrmode = 1 and lrstat =3. An example of common bad data from an ionshow is provided below. If it is a problem during the night, troubleshooting that would result in lost sky time should not be completed. The Red dewar should be fine without continually monitoring it. It is not a critical component.

          --- Current lris ion pump keyword values
                         lrcur = 0.000000000 amp
                        lrvolt = 0.000000 KV
                        lrmode = 0 
                        lrstat = 0  (MAJOR: value is less than "low" limit)

Problem

The high voltage is not set appropriately.

Solution

First try restarting the software a few times. If this fails, then staff will need to power cycle the high voltage current on the ion pump electronics located in the Red side electronics bay.

If it is working "lrmode = 1" and "lrstat = 3"

Following a software restart, you may need to wait up to five minutes before a valid measurment is recorded.

For further information see: ion pump technical doc