NIRES Troubleshooting

Summary

  • Computers & Software
  • SCAM images
  • SPEC images (H2RG)
  • Servers
  • FCS
  • temps

    Computers & Software

    Server (niresserver1 or niresserver2) is unresponsive

    Symptom
    xterms are unresponsive and disappear.
    Problem
    The server is probably rebooting itself.
    Solution
    1. wait for the server to reboot
    2. Restart the software.

    Slit Viewer Camera (scam) Images

    Unable to take a test image with the SVC

    Symptom
    A test image fails to be taken.
    Problem
    Test image are stored in /tmp and they might have the wrong permission.
    Solution
    1. Log into niresserver1 as user niresrun.
    2. Go to directory /tmp.
    3. Inspect the permissions for file TEMPFITS.fits.
    4. Delete it if the owner is not niresrun.

    Images appears to be overwritten

    Symptoms
    An image taken with the scam seems to be partially overwritten on a previous image.
    Problem
    There seems to be a race condition that happens at random.
    Solution
    There is no solution. The next image should be OK.

    SCAM images show vertical pattern

    Symptom
    SCAM images show a vertical pattern on each quadrant.
    Problem
    The images are saturated.
    Solution
    The maximum exposure time under clear conditions is 30 seconds. If the exposure time is longer please adjust accordingly. Also check the sky conditions as there could be clouds present that will lower the maximum exposure time.

    Spectroscopic Detector (H2RG) Images

    Excessive noise that oscillates among amplifiers

    Symptom
    Excessive noise that appears as horizontal bands that oscillates among amplifiers and changes in time.
    Problem
    It is not clear what the source of this noise is. But we know that at least sometimes is triggered by an aborted exposure.
    Solution
    1. Log into niresserver2 as user niresrun.
    2. run the restart_h2rg.
    3. check that two nsds rpc processes are running with ct .
    4. as the nires number account restart the Quick Look tool
    5. take a test image
    6. on an xterm type restore_state state_master to restore the PROGID, OBSERVER, and OUTDIR.

    Servers

    nsds server crash: Unable to take exposures or exposure never reads out.

    Symptom
    The exposure completed but there was no image.
    Problem
    The nsds server program crashed.
    Solution
    Restart the nsds server.
    1. Check if two rpc processes are running for nsds by typing ct.
    2. If there is one or none then log on to niresserver2 as niresrun on an xterm.
    3. type restart_h2rg to restart the nsds daemons
    4. check that two nsds rpc processes are running with ct .
    5. as the nires number account restart the Quick Look tool
    6. take a test image
    7. on an xterm type restore_state state_master to restore the PROGID, OBSERVER, and OUTDIR.

    nids server crash: unable to take images

    Symptom
    Unable to take an exposure with the SCAM.
    Problem
    The nids server program crashed.
    Solution
    Restart the nids server.
    1. Check if two rpc processes are running for nids by typing ct.
    2. If there is one or none then log on to niresserver1 as niresrun on an xterm.
    3. type restart_h1 to restart the nids daemons
    4. check that two nids rpc processes are running with ct .
    5. If this fails run a testAll
    6. Check the output for tdl, it should look like this:
      Checking coonection to Leach board:
              Checking whether H1 board is alive...OK
              Checking whether H2RG board is alive...OK
      
      If not then suspect a fiber communication problem. Ask the summit crew to perform the following:
    7. reset the fiber board on the niresserver1 by pressing the white buttom
    8. check the communication with testAll
    9. If the communication has not been restored then reboot the niresserver1 (call Software support).
    10. as the nires number account restart the Quick Look tool
    11. take a test image
    12. on an xterm type restore_state state_master to restore the PROGID, OBSERVER, and OUTDIR.

    FCS

    FCS not working

    Symptom
    The FCS Status reports that the FCS Mode is Halt and the Status is Halted when it should be tracking.
    Problem
    The FCS mode has changed to for some reason.
    Solution
    Follow these steps to put the FCS back into tracking mode:
    1. Log into niresserver1 as any NIRES account.
    2. Execute the restart_FCS& script.
    3. After 30 seconds the FCS should be back to tracking
    4. If continued attempts to have the FCS track fail, then suspect that the nfcs daemon is not running.
    5. check this by running ct
    6. If there are less than two rpc processes then restart the nfcs daemon by:
    7. log on to niresserver1 as user niresrun
    8. start the dameons with nires start nfcs
    9. verify that the daemons is running with ct
    10. Execute the restart_FCS& script.

    temps

    Temperature warnings

    Symptom
    The temperature of the H2RG is out of range. The NIRES dewar status monitor task "monitor_tempdet" running on the NIRES host computer "niresserver1" may have detected the problem and is now sending e-mail warnings.
    Problems
      There are several reasons for the temperature warning.
    • The main dewar is dry of LN2, and is starting to warm up.
    • The aux tank of LN2 is dry and the H2RG is warming up.
    Solution
    1. Try running "temps" on niresserver1. Typical output is:
      [1006] nireseng@niresserver1: temps
      System               Current Temp  Acceptable Range  Status
      -------------------  ------------  ----------------  ------
      Detector temp 1              73.9    73.0 to   75.0  OK
      Detector temp 2              74.0    73.0 to   75.0  OK
      Auxiliary Tank                1.0    72.5 to   74.5  ERROR: temp too low!
      H1 cold finger               77.8    76.0 to   78.0  OK
      
      
      	    
    2. If several temps output indicates that the temperature is rising, i.e. continuing to warm up, then the dewar is indeed dry and an immediate LN2 fill is necessary, to prevent warm up and loss of vacuum. Contact the support astronomer on duty if this happens.
Last modified: 03/24/2023 07:16
Send questions or comments to:NIRES Support

The information on this page is the property of the W. M. Keck Observatory. The contents of this page or any part thereof shall not be copied or otherwise reproduced or transferred to other documents or used or disclosed to others for any purpose other than observing support at the W. M. Keck Observatory and the subsequent analysis and publication of scientific data obtained from observations conducted at the W. M. Keck Observatory. All rights reserved. © W. M. Keck Observatory.