A bunch of this page isn't filled out, but all of these questions AND MORE are answered on the Internal AO Optics Troubleshooting Page, so seek your answers there!

• AO Performance
• User Guide for Optimization Tool (also named bandwidth widget)
• Checklist for bad AO performance
• How do I measure the seeing
• Seeing is either large or very fast, what should I do?
• The seeing is OK but LGSAO Performance is NOT, TT gain is very low
• Checking the WFS centroid origin calibrations
• Stars appear elongated, what should I do?
• The Guide Star is a 2"-5" binary (NGS only)
• Telescope/AO interactions
• AO/DCS communications: general
• Dither failure: light lost on WFS:
• Dither failure 2: light lost on WFS:
• Generalized offset/dither problem in NGS and LGS
• Dither failure: timeouts in AO/DCS handshake
• About "Restart AO/DCS communications":
• How do I check the TT offloading parameters
• SKY/AO conflict for non-sidereal (planetary) targets
• NGS AO Operations
• How to set the WFS parameters manually?
• How to insert/remove a neutral density filter (WND)
• Why is the PSF observation so important to AO observers?
• How much should you care about the B-V magnitude
• No sounds???
• Poor Correction on Extended Objects
• Pyramid WFS Operations
• Restarting the PyWFS camera (SHAPHIRA)
• Bad IQ in open- and closed-loop confirmation image
• LGS AO Operations
• What are the main differences between LGS AO and NGS AO
• Where are the LTCS pages?
• NGS and LGS operations procedures
• Laser Manual Acquisition
• Acquisition when background is uneven (e.g., M31)
• Laser spot not detected on ACAM image. M3 misaligned.
• Noise on the LBWFS
• Cannot measure the unstacked LGS spot size on ACAM
• Express LGS AO Checkout
• Checking STRAP
• TSS: DAR and TSS FOCUS control in LGS mode
• LGS AO VNC session Xserver seems dead
• LGS AO VNC Norse Viking Attack
• K2 only: Cannot take LBWFS images
• Transponder-Based Aircraft Detection
• TBAD Basics
• TBAD xshow Timestamp Incorrect
• Wavefront Controller
• Wavefront Controller Crash
• Powering up the tip-tilt mirror
• Loading new WFS centroid origin calibrations
• Querying TRS
• AO Rotator
• Who controls the AO rotator?
• How to detect a ROTATOR FAULT in NGS or LGS AO
• Recover from a ROTATOR FAULT in NGS or LGS AO
• OBS Device FAULTs
• Why do these stages fault?
• Repeated FSM FAULTs
• FCS FAULT in tracking mode (LGS AO)
• TSS FAULT in LGS AO mode
• Other Resources and Help Pages
• Where can I learn about all these acronyms?
• Observing without ACAM
• I cannot see light on the WFS
• Problems with User Interfaces (IDL, etc)

AO Performance

User Guide for Optimization Tool (also named bandwidth widget)

The optimization tool has its dedicated web page.

Checklist for bad AO performance

Symptom

The image quality is bad.

Problem

The list of possible problems includes:

1. Bad or fast seeing
2. Telescope windshake or telescope vibrations, low elevation
3. Problem with the WFS background
4. Problem with the reconstructor
5. Vignetted pupil
6. ??
7. Tip-tilt loop unstable
8. Tip-tilt offloading problem
9. ??
10. Bad calibrations
11. ACS snap

Solution

Note that proper troubleshooting should be performed on a 10th magnitude B-A type star (NGS) or one of the LGS checkout stars.

1. Measure the seeing using the atmospheric characterization tool, acamtool, open loop images on NIRC2 or using the LBWFS (read the aolbfwhm keyword).
2. Use the bandwidth widget (optmization tool) to optimize the TT and DM gains and check for any problems with vibrations.
3. Record new background.
4. Trigger a new reconstructor using MAORI→WFS with both loops closed and check that reconstructor updates after a few seconds. If not, then restart the reconstructor from MAORI→Menu. Check the pupil on the WFS intensity display with the reconstructor mask on and off (the pupil should look the same).
5. Check windshake, check tracking errors for El and Az.
6. Check for possible vibration produced by the secondary vacuum valve, the telescope drives, etc.
7. Check that tip-tilt mirror is powered up. N.B. if the TT graph shows the mirror position changing, it is VERY likely powered up.
8. Check for problem with the tip-tilt mirror (broken actuator?).
9. Check for problem with the DM using the WYKO interferometer. Open the WYKO hatch and look at the WYKO when the DM loop is opened or closed. You should see the DM move constantly in closed loop. You should also see that it is not producing strange shapes, such as waffle or a lot of focus.
10. Check for calibrations: DM to lenslet, COG, SFP home problem, WLS centering, FSS vignetting, etc.

    COG file

    If you see poor image quality, you can try an old COG file (this is quick and can be done with loops closed)

    Current COG	Old COG

    In this case, we decided the old COG was better than the current one. We made the old one the default by opening a terminal as kNobsao@kNaoserver

    1. cdcog
    2. cp -p 24NIRC2-N2x2.cog31Aug2016 24NIRC2-N2x2.cog
    3. The change takes effect when one next runs Setup Bench from AOAcq.

11. Check ACS snap. If you suspect the cals are fine or you see non-diffraction limited performance, check the snap that ACS is using (OA's are very helpful here).

    Bad (DEIMOS) Snap	Good (PCS) Snap

How do I measure the seeing

Symptom

Problem

Solution

Seeing is either large or very fast, what should I do?

Symptom

Problem

Solution

The seeing is OK but LGSAO performance is NOT (TT gain is very low)

Symptom

Problem

Solution

Checking the WFS centroid origin calibrations

Symptom

Bad AO performance, even for bright star (Vmag10). Artifacts in the image.

Problem

Bad calibrations or corrupted file for the centroid origins?

Solution

1. From an xterm on k2aoserver as k2obsao, type cdcog.
2. Then dir will list all centroid origins calibration files sorted by date.
3. The name reflects the instrument and the date. For a NIRC2 night in NGS AO mode with the 2.4 arcsec plate scale and 2x2 binning the default file would be 24NIRC2-N2x2.cog. Find a file with the same name but a different date appended (e.g., 24NIRC2-N2x2.cog16Feb2007) that you trust. In this example,
4. cp -p 24NIRC2-N2x2.cog16Feb2007 24NIRC2-N2x2.cog
5. In IDL, type LOADCOG,'24NIRC2-N2x2.cog'
6. The new calibration file should be loaded on the system.
7. Try other calibration files if necessary or use different binning modes or plate scale.

Stars appear elongated, what should I do?

Symptom

Stars appear elongated on science detector.

The TT mirror and TT residual graph could show a lot of jitter.

Telescope servo-error tracking values could be large.

Problem

Many possible reasons, but the most likely are:

1. Tip-tilt mirror oscillations.
2. Problem with TT offloading: could happen when star goes through meridian.
3. Wind shake: do you see the same effect when you point into the wind vs other directions?
4. Lots of vibrations (produced by the secondary-mirror vacuum valve?): experience has shown it could be worse at lower elevation
5. Reference star is a binary or has a peculiar morphology: 2-5 arcseconds binary used as a reference star may produce some elongation. If it is a close binary (less than 100 mas separattion) then the image may appear to be elongated.

The guide star is a 2"-5" binary (NGS only)

Symptom

We are guiding on a 2 to 5 arcsec binary of comparable magnitude and the image quality is poor.

Problem

Both stars are in the wavefront sensor, so the star is not centered on the cross hairs of the wavefront sensor.

Solution

A possible solution is to introduce a smaller field stop in front of the wavefront sensor.

1. Open the loops using MAORI.
2. From MAORI → WFS, left click on the entry by the FSS row and select "0.6". FSS is the field stop stage.
3. Once FSS reads back "0.6", you may reclose the loops and continue.
4. Next time a star is acquired, the field stop will automatically go back to the "2.4" named position.

Telescope/AO Interactions

AO/DCS communications: general

Symptom

Problem

Solution

Dither failure: light lost in WFS

Symptom

Problem

Solution

Dither failure 2: Light lost in WFS

Symptom

Problem

Solution

Generalized offset/dither problem in NGS and LGS

Symptom

Problem

Solution

Dither failure: timeouts in AO/DCS handshake

Symptom

Problem

Solution

About "Restart AO/DCS communications"

Symptom

Problem

Solution

How do I check the TT offloading parameters?

Symptom

Problem

Solution

SKY/AO conflict for non-sidereal (planetary) targets

Symptom

Problem

Solution

NGS AO Operations

How to set the WFS parameters manually?

Symptom

Problem

Solution

How to insert/remove a neutral density filter (WND)

Symptom

Problem

Solution

Why is the PSF observation so important to AO observers?

Symptom

Problem

Solution

How much should you care about the B-V magnitude?

Symptom

Problem

Solution

No sounds???

Symptom

• You never hear Keck II AO loops are closed or The wavefront controller has crashed.

Problem

The IDL tools are sending sounds to the wrong computers or the volume is too low

Solution

From a summit computer (e.g. k2aoserver, k2vncserver, etc.) as k2obsao:

1. Type sounds to list to where IDL is sending sounds.
2. If your computer is not listed, type sounds your_computer1 your_computer2 to define the sound hosts. N.B. you may define 1 or more sound hosts
3. Type volume volume1 volume2 to define how loud you want the sounds (must be an integer between 0 and 100).
4. Make sure your speakers are turned up.

Typical list of hosts for K2AO sounds are:

k2display-tc
ro2display
shared-t620

Poor correction on extended objects

Symptom

Problem

Solution

Pyramid WFS Operations

Restarting the PyWFS camera (SAPHIRA)

Symptom

PyWFS Camera (in the PyWFS status) shows red and says it is powered off. Clicking on it and rebooting/restarting the camera server does NOT resolve this.

Problem

Communications between the SAPHIRA and the RTC have been lost.

Solution

1. Connect to the RTC: ssh -X prtc@prtcserver
2. Type: showRTProcess. This opens a window, and the second process from the top is partially crashed
3. Ctrl-c to restart the process
4. Connect to the PyWFS server: ssh -X pwfs@vm-pwfsserver
5. Issue command: prtcRestartCOM. It kes about 5 sec, then you should be able to reacquire with the camera.

Bad IQ in open- and closed-loop confirmation image

Symptom

The image quality after image sharpening is good, but the opne- and closed-loop image after having calculated new slopes looks bad.

Problem

It is likely the slopes were obtained with low light on the fiber source.

Solution

1. Check if PyWFS is configured for SND=OPEN after image sharpening and tuning the PyWFS focus. If it is not:
   • Open vm-pwfsserver terminal: ssh -X pwfs@vm-pwfsserver
   • Type: prtcSetupForMag 1.
2. Measure the reference slopes.

LGS AO Operations

What are the main differences between LGS AO and NGS AO?

Where are the LTCS pages?

Symptom

Problem

Solution

NGS and LGS operations procedures

Laser manual acquisition

Symptom

Problem

Solution

Acquisition when background is uneven (e.g M31)

Symptom

Problem

Solution

Laser spot not detected in ACAM (M3 misaligned)

Symptom

Problem

Solution

Noise on the LBWFS

Symptom

Problem

Solution

Cannot measure the unstacked LGS spot size on ACAM

Symptom

Problem

Solution

Express LGS AO checkout

Symptom

Problem

Solution

Checking STRAP

Symptom

Problem

Solution

TSS: DAR and TSS FOCUS control in LGS mode

Symptom

Problem

Solution

LGS AO VNC session Xserver seems dead

Symptom

Problem

Solution

LGS AO VNC Norse Viking Attack

Symptom

Problem

Solution

K2 only: Cannot take LBWFS images

Symptom

Problem

Solution

Transponder-Based Aircraft Detection

TBAD basics

TBAD xshow timestamp incorrect

Symptom

Problem

Solution

Wavefront Controller

Wavefront controller crash

Symptom

Problem

Solution

Powering up the tip-tilt mirror

Symptom

Problem

Solution

Loading new WFS centroid origin calibrations

Symptom

Problem

Solution

Quering TRS

Symptom

Problem

Solution

AO Rotator

Who controls the AO rotator?

Symptom

Problem

Solution

How to detect a ROTATOR FAULT in NGS or LGS AO

Symptom

Problem

Solution

How to recover from a ROTATOR FAULT in NGS or LGS AO

Symptom

Problem

Solution

OBS Device Faults

Why do these statges fault?

Symptom

Problem

Solution

Repeated FSM FAULTs

Symptom

Problem

Solution

FCS FAULT in tracking mode (LGS AO)

Symptom

Problem

Solution

TSS FAULT in LGS AO mode

Symptom

Problem

Solution

OBS Reboot

Symptom

Nothing is working

Problem

OBS is hosed

Solution

Reboot OBS

Other Resources and Help Pages

Where can I learn about all these acronisms?

Symptom

Problem

Solution

Observing without ACAM

Symptom

Problem

Solution

I cannot see light on the WFS

I Can't See Light on the WFS!

Problems with User Interfaces (IDL, etc)

Problems with User Interfaces