Director’s Introduction

Hilton Lewis, Director, WMKO

Aloha, and welcome to the spring 2017 Observers’ Newsletter. In this edition we report on some of the major activities in the last 6 months. Read about our next major instrument, the long-anticipated Keck Cosmic Web Imager. Science commissioning commenced with a very successful First Light on April 11th, and will continue over the next several months. We anticipate that we will be ready for shared-risk science observing in the latter part of semester 2017B.  We report also on MOSFIRE, which was damaged in engineering testing last September, but is now fully repaired and back in the hands of the astronomers, continuing to enable remarkable science. You can read about the accident investigation and follow-up, and how the observatory has embraced the incident as an opportunity for institutional learning, with full transparency and accountability.

After many delays NIRES (the near-infrared echelle spectrometer) constructed at Caltech and destined for Keck 2 has finally arrived at the observatory.  We look forward to commissioning NIRES later this summer – a long-awaited and powerful new capability for our observers. We also look forward to the completion of the OSIRIS imager upgrade and the start of the Keck 1 deployable tertiary installation in the same period.

As you read you will find there has been progress in many areas.  While considering all the exciting new developments keep in mind that the observatory continues to operate reliably, efficiently and effectively night after night, enabling a constant stream of new science data.  Many are the unsung staff who ensure the smooth operation of the twin Keck telescopes, the solid underpinning of the scientific productivity of the Keck Observatory and of its astronomers. Next time you visit please stop by and talk to the people who make it all possible. I think you will be as impressed as I am by their skill and dedication.

Now please continue reading about how your observatory is moving forward.

A hui hou kākou – Hilton Lewis, Observatory Director.

✶

Keck Visiting Scholars Program

Anne Kinney, Chief Scientist, WMKO

Through the generous gift of a donor, we are able to offer funded visits to WMKO Headquarters in Waimea for graduate students and Post Docs in the first 3 years of their appointment. These visits aim to inspire and develop the next generation of scientists and instrumentalists by providing hands on experience in observational astronomy, hardware and software, involving active instrument operations and real time problem solving. In our Keck Visiting Scholars Program, funds would be provided (for airfare, room and board, and a rental car, but not for salary) for these early career scientists to be resident at Keck Observatory, working with our Support Astronomers and instrumentalists for periods of one to three months.

Today’s early career astronomer has often little experience with telescopes, scientific instruments, night time observing, or data reduction. Queue observing is in common usage on many ground based telescopes, space based telescopes come with fully developed data pipelines, and even Keck observers rarely attend in person.

There is an expression in astronomy: “Only a theoretician believes the data, and only an observer believes the theory.” With a better depth of knowledge of data obtained through time at the telescope, the observer obtains a better understanding of the data and their uncertainties.

Implementation

We are beginning what we hope is a multi-year program with a pilot program where the first year’s applications will be by invitation. In that manner, we will be better able to target opportunities which benefit the visiting scientist, their mentor, and Keck Observatory. We hope to demonstrate to the community the concrete benefits of having students and Post Docs performing their own observations on site at Keck Observatory, working directly with Support Astronomers, and even helping other scientists perform their observations. We hope to create interest for the second and third years, where we would invite the entire Keck early career community to apply.

Applications are due April 15, with one brief not-to-exceed-one-page program description by the graduate student or post doc, and a brief not-to-exceed-one-page recommendation by the mentor or advisor. The selection criteria is likelihood of success of the visit for the scholar, their mentor, and for Keck Observatory.

Students must supply their own laptops, surf boards, and scuba gear. ✶

New HIRES Echelle Format Simulator

Greg Doppmann, Support Astronomer, WMKO

An improved planning tool for HIRES is now available to the Keck Observing community. The Echelle Format Simulator (EFS) has been an essential tool for planning observations that require specific grating angles and slit widths. Previously, the on-line version of the EFS was problematic and not supported by most browsers. Thanks to the efforts of Oliver Grayson, a talented sophomore at a Waimea high school, the HIRES EFS has been now re-written in Javascript and runs seamlessly in your favorite browser. Further improvements to this tool include: 1) a slit length setting that accurately shows when blue orders begin to overlap and better predicts which portions of orders fall within the CCD gaps, 2) more accurate wavelength coverage of all orders on the three CCD chips with wavelength and order number shown from the mouse position anywhere on the echellogram, and 3) a better zoom capability for viewing where orders fall on each CCD chip. Users can now access this new tool from the Echelle Format Simulator Page within the HIRES Observing page and click on “New HIRES EFS”.

Figure 1. HIRES echellogram showing 47 orders spanning three CCD chips from red to blue wavelengths in the optical (top to bottom).  The width of the spectral order is determined by the choice of slit length, now a modeled parameter in the new EFS.

Figure 2. New HIRES EFS control GUIs showing relevant instrument settings for configuring the echelle and cross disperser grating angles and for viewing the resultant wavelength coverage on the detector.  ✶

Time Doman Astronomy

Randy Campbell, Observing Support Manager, WMKO

Time Domain Astronomy, TDA, is becoming an important operational capability for W.M. Keck Observatory in the near term and likely well into the future. WMKO is developing a deployable tertiary on Keck I, K1DM3, to be commissioned in 2017, ushering in a new era of TDA to enable new modes of operation without significantly impacting the traditional classic modes of operation that are highly valued at Keck.
TDA is currently implemented at Keck in two forms:

• Targets of Opportunity, ToOs - observations needing rapid response observations within hours or days.
• Cadence Observations - periodic series of observations of objects that may be evolving on time scales of weeks or months.

ToO’s

WMKO has recently expanded its ToO program to enable ToO science on a limited basis.  It is authorized by an inter-institutional agreement to allow TAC approved programs the ability to interrupt the current schedule observation between the major partners UC, CIT, NASA, and UH. Although the ToO program is still in a trial period, it has been successful in capturing transient events on short time scales and getting important follow up observations. Observers interested in applying for ToO time should submit a proposal with the option that specifies “ToO” on the coversheet to ensure it is tracked correctly by the TAC and Observatory. Future improvements such at the K1 deployable tertiary will likely satisfy the future needs of WMKO in the era of ZTF, LSST, PanStarrs, and other TDA-oriented science operations.

Cadence

Cadence programs are defined as those that require a periodic series of observations over time scales of days, weeks, or months. A trial program for cadence observations has also been recently implemented at WMKO. The challenge for the classically operated observatory is to squeeze in series of shorter observing blocks of time into the scheduled nights. This is now possible due to a cross-institutional agreement, similar to the one for ToO’s. Another advantage of this flexibility is inserting engineering time that is often cadence in nature, being more efficient and timely when scheduled in shorter periodic blocks of time. Observers interested in applying for cadence time should submit a proposal with the option that specifies “Cadence” to ensure the TACs and Observatory correctly account for these programs.
Surely the WMKO observing support for cadence represents an increase in workload, particularly for support astronomers and can potentially reduce our efficiency. Thus, the program is limited so that a maximum of only 10% of the nights can be impacted by cadence and this modest change leaves the vast majority of whole night (and even half night) of classically scheduled programs unaffected.

TDA Scheduling Database

The implementation of TDA at Keck has prompted changes to the system of proposals, the method TACs use to inform the observatory of TDA allocations, observatory scheduling, and tracking institutional night balance. New database software is being developed to handle the information flow and to improve the scheduling process.  Additionally, a new Keck proposal cover sheet has been developed to help  distinguish TDA programs and to improve the scheduling in general. An important entry on the new cover sheet is the coordinate field that best represents your primary field or fields of interest. If your proposal is only allocated a single observing run, this RA can be used to ensure your night is optimized for observability using factors such as airmass, telescope limits, moon position, etc.

As the demand for time domain astronomy grows in the future, WMKO may need to scale up the TDA program and/or make additional modifications to the process of proposing, scheduling, and supporting these observations in order to meet the demand.  WMKO is committed to meeting the scientific needs of the community and adding a small amount of TDA to the classically operated observatory is currently one of those needs. ✶

KCWI Installed on the Keck II Telescope and First Light

Luca Rizzi, Support Astronomer, WMKO

It is with great pleasure that we announce the successful delivery and installation of the Keck Cosmic Web Imager on right Nasmyth focus of the Keck II telescope, and the achievement of first light during the initial commissioning night.

After passing a rigorous pre-ship review, the instrument was accepted by WMKO and carefully prepared for shipping. The logistic challenges related to the transportation of such a valuable cargo are staggering on their own, but this instrument had a special requirement: it had to travel in an air conditioned environment. As it turns out, the epoxy used for some of the optics is perfect for the Maunakea environment, but deteriorates at high temperatures such as the one that are likely to be experienced on a sunny highway in Southern California and inside the cargo hold of a ship. Unfortunately, KCWI is too big to fit into a standard refrigerated container: a special crate had to be designed and built, with an air conditioners permanently attached to it. The next challenge was to find ways to power the air conditioner across multiple transportation environments: on the road while on a trailer, at the dock in San Diego, on the ship, at the dock again in Hilo, and then on the road on the way to Maunakea. A tiger team was created for this purpose, made of a number of Caltech staff (Ray Zarzaca, Matt Matuszewski, Hector Rodriguez) and WMKO staff (Cindy Chong, Ean James, Sam Park, Steve Milner, Greg Doppmann, Luca Rizzi), and after detailed discussions with the shipping and crating companies, we finalized a plan and hired two marine surveyors to execute it. I am happy to report that the plan was followed to the letter and without a hitch and that the instrument successfully made its way from Caltech to Keck II where it was delivered on January 20 without ever experiencing excessive vibrations or temperature changes.

Following the delivery, the instrument underwent a thorough inspection to reveal any possible damage, and was then lifted to the right Nasmyth deck. On the deck, the instrument lives on a sophisticated transport cart which is able to both transport the instrument from the stowed location to the deployed position, and to raise and lower it safely onto the kinematic mounts. Once on the cart, we started the installation procedure which included establishing communication with the computer room via long fiber cables, connecting cooling glycol and clean air supplies, and a careful power up and test of the individual components.

Figure 3. KCWI installed on the transport cart.

Once the instrument operations had been verified, we proceeded to the optical alignment. First, the instrument was mechanically aligned to the rotational axis of the telescope using tracking lasers to achieve a coarse alignment. Secondly, a LED target was placed inside the Keck II Adaptive Optics bench at the opposite side of the telescope (left Nasmyth), to accurately track the elevation rotational axis of the telescope and align it with the rotation axis of the K-mirror of KCWI. The last step was to verify the internal alignment of the optics. During this phase, a small misalignment was discovered that resulted in the spectra being partially off the detector in some of the conjurations. The cause of the misalignment was identified and promptly corrected.

Despite losing a couple of days to bad weather and to a long power outage, the team was able to finalize the installation on schedule.

On the night of April 11, a team of WMKO and CIT staff started the rigorous sequence of tests required to interface the instrument with the telescope control system, which worked flawlessly. At 2:30 am, on April 12, we obtained the first scientific spectrum, a high resolution integral field spectrum of M3. The instrument performed perfectly and, paired with the collecting power of the Keck telescope, it was able to surprise even the seasoned team who built it.

Figure 4. The KCWI commissioning team immediately after first light. From left to right: Chris Martin, Luca Rizzi, Greg Doppmann, Shui Kwok, Don Neill, Patrick Morrissey, Matt Matuszewki.

To learn more about KCWI, visit the instrument web page. An exposure time calculator is also provided.

The timely and successful installation and initial commissioning of KCWI is the result of careful planning and of the dedication and perseverance of both the Caltech and WMKO redoubtable teams: Chris Martin, Matt Matuszewski, Don Neill, Patrick Morrissey, Hector Rodriguez, Ray Zarzaca, Bob Weber, Cindy Chong, Ean James, Shui Kwok, Sam Park, Greg Doppmann, Steve Milner and the entire Software/IT and Summit teams. Stay tuned for the results of our future commissioning nights and for the great scientific discoveries that KCWI will make.

New Support Astronomer Sherry Yeh joins WMKO

Randy Campbell, Observing Support Manager, WMKO

Welcome Dr. Sherry Yeh, Keck’s new Support Astronomer, to Keck. Sherry’s most recent position was research fellow at the Subaru Observatory where she was working on the queue science operations of Hyper-Suprime camera, HSC, the prime focus wide field optical imager.  Sherry received her PhD in Astronomy and Astrophysics at the University of Toronto in 2013. Her research interests includes massive star cluster formation using observations in the near and mid infrared. She'll initially be learning to support Keck’s near infrared multi-object spectrometer on Keck 1, MOSFIRE, and then branching out to other instruments and AO science support.  Sherry and her family recently moved to Waimea from Hilo and are getting settled in.  We’re very happy to have Sherry join the Keck team.

Mainland Observing Updates

Josh Walawender, Support Astronomer, WMKO

Keck Observatory has offered mainland observing for more than 10 years now and the demand for it has increased steadily during that time. Currently over 70% of Keck telescope nights have some form of mainland observing in use and mainland only observing is currently used on more than half of all nights.

Through the end of semester 16B, there have been over 3,650 nights in which mainland observing has been used and 4,522 program-nights of use.

To accommodate mainland observers, Keck has been working with various institutions to expand the number of mainland observing sites available to the users. While Keck has generated the specifications for the infrastructure, it is the host institutions who build and maintain the site.

There are currently 13 mainland observing sites which support "mainland only" observing (where no users are at the Waimea operations center) and 3 additional sites support "eavesdrop mode" (where observers can participate from that site, but the lack of backup systems means someone must be in Waimea).

A new mainland only site is now in operation at USRA, serving astronomers in the greater Baltimore-Washington DC area. Additionally, both the UCSB and ANU sites have recently been upgraded from eavesdrop to mainland only mode

The "mainland only" qualification requires that the site have an ISDN backup system to cover cases in which the network connection fails. The ISDN system allows telescope operations during many types of network failures between the site and Maunakea.

Keck now has a new Mainland Observing Coordinator: Josh Walawender has taken over these duties from Greg Doppmann. We are also bringing Gloria Martin on to the mainland observing team and she will be handling many of the mainland observing requests. As part of this transition, we will be trying to simplify some of the mainland observing approval processes, so that observers see quicker turnaround times on their mainland observing requests.

Figure 5. Mainland use over time. ✶

The Return of MOSFIRE

Marc Kassis, Support Astronomer, WMKO

After a five-month service mission to re-bond a loose lens, MOSFIRE went back on sky on February 11 2017 and was returned to science operations. Although our engineering nights to quantify the performance of the instrument were lost to weather, we do not expect the performance of MOSFIRE to have changed noticeably, and astronomers who have used MOSFIRE in 2017 report no losses in throughput or image degradation following the service mission.

MOSFIRE's optical performance was degraded during an incident that occurred during engineering testing on 13 Sept 2016. Calibration data and on-sky testing obtained that evening showed that MOSFIRE's optical quality was severely degraded and it was concluded that the instrument would need to be opened to make a repair. MOSFIRE was taken off sky for the next five months resulting in approximately 40 MOSFIRE nights being reassigned to LRIS, HIRES, and OSIRIS.

Figure 6. November 3, 2016 MOSFIRE is lifted using the shutter crane from the Keck I deck to the cleanroom on the dome floor. Inside the clean room is the instrument development test stand used to open the instrument. Keck staff completed the clean room in a month.

The Summit crew erected a clean room, originally planned for Keck II to support KCWI commissioning. While the clean room was erected, we acquired spectroscopic and imaging calibration data, completed visual inspections of the front end optics, and modeled the spectroscopic and imaging data to try to pinpoint where in the instrument the failure had occurred. Modeling completed by Caltech staff has indicated that an out of position lens in the collimator assembly was mostly likely the source of the problem.

As soon as the clean room was complete and tested, we craned MOSFIRE from the storage position on the Keck I deck to the dome floor just outside the cleanroom. We then rolled MOSFIRE into the clean room and secured it to the same fixture that was used to build and develop the instrument at Caltech. We then spent a week cleaning the instrument and room to remove five years of grime and eventually obtained a level of cleanliness that matched that of the Caltech facility in which MOSFIRE was built.

To reach the collimator, we removed the electronics cabinet, instrument cable wrap, the dewar shell, and shielding from the back of the instrument. After the instrument was open, tt became obvious that the problem was in the collimator lens assembly because the second optic in the assembly was visibly loose. We then removed and disassembled the collimator lens assembly to access this lens.

Figure 7. With MOSFIRE open in the background Dwight Chan (Keck) and Ken Magnone (UCLA) install the lifting bolts to prepare the collimator lens assembly for installation in the instrument.

Six pads are bonded along the edge of each lens to maintain its position in the collimator assembly. All lenses in the system were still secure except for one. This particular lens is made from ZnSe and all six bonds had failed, thus it was free to roll around while the instrument was rotated. We suspect that the failure is specific to this lens material and feel confident that the bonds on the rest of the optical system remain strong. We found that the ZnSe lens had a small chip along the edge and a few scratches in the coating. Because the damage to the lens was assessed to be minimal, the ZnSe lens and the lens mount were shipped to UCLA where it was cleaned; its surfaces were re-prepared, and then re-bonded.

While the instrument was open for this work, we took the opportunity to address a chronic issue with one of the filter wheels. We found that the position switches for this filter wheel were activating at the very edge of the switch roller and occasionally missed activating. We shifted the switch so that the activation is in the middle of the roller, and now MOSFIRE reliably selects the medium band filters J2, J3, H1, and H2.

Critical to the success of MOSFIRE's recovery was the collaboration with the MOSFIRE development team at Caltech and UCLA. From Caltech, Dr. Chuck Steidel, Keith Matthews, Jason Fucik, Bob Weber, and Hector Rodriguez and from UCLA Dr. Ian McLean, Ken Magnone, Ted Aliado, and Jason Weiss worked along side Keck's instrument engineers Steve Milner, Dwight Chan, and Nick Suominen and support astronomers with modeling the data, opening the instrument, and re-bonding the lens. ✶

MOSFIRE Accident Investigation

Rich Matsuda, Chief of Operations, WMKO

On September 13, 2016, MOSFIRE was damaged and deemed unusable for science.  MOSFIRE, WMKO’s most highly-demanded instrument and used on about one-third of the nights on the Keck I Telescope, went out of service for over 5 months resulting in a significant and regrettable loss of science opportunity with direct negative impact to many observers.

The failure was due to the instrument being ‘shaken’ causing a serious problem with the internal optics of the instrument. A Cassegrain-mounted instrument, MOSFIRE sits in a module that rotates.  A new rotator control system was implemented as part of the Telescope Control System Upgrade (TCSU) project, and at the time of the incident the rotator control parameters were being tuned to address a tracking performance issue.  This caused the rotator to go into an unstable oscillation, shaking MOSFIRE. Later it was found that a lens in the collimator assembly broke free of its mounting bracket causing severe image quality problems.

The criticality of this failure caused the observatory to not only investigate the proximate technical causes of the accident, but to also look for underlying systemic issues with our management and work processes that could pose unacceptable risk to other high-value assets.  Two in-depth reviews were held.

On October 18, 2016, an incident investigation was held, chaired by Jason Chin (WMKO senior project manager) and included Peter Gillingham (instrumentation engineer, Australian Astronomical Observatory), Kevin Ho (instrumentation manager, Canada France Hawaii Telescope), and Josh Walawender (WMKO support astronomer). The investigation focused on fully understanding the accident and related circumstances.  Key participants in the incident were interviewed and the incident report filed by the TCSU team and related documentation were carefully reviewed.
On December 1, 2016, an external review was held to broadly examine our management and engineering practices and work culture at WMKO.  The external review panel was chaired by Tom Greene (NASA Ames, Keck SSC NASA rep) and included Bill Irace (JPL - retired), Alan Tokunaga (IRTF Director), Mark Sirota (Telescope Controls Group Leader), Jason Chin (WMKO senior project manager) and Randy Campbell (WMKO science operations manager). 

Both committees provided thorough and insightful reports to the WMKO Director, Hilton Lewis.  While the overall professionalism and dedication of the staff and organization were commended, serious deficiencies were identified, as follows:

• We accept too much risk when handling high-value assets.  Engineering and management practices for identifying and mitigating risk are inadequate.
• We have technical expertise gaps in servo control and systems engineering.
• Our work culture is deficient in the area of awareness and communication of equipment safety concerns.

A number of recommendations were made by the review committees to address the findings.  In January, Director Lewis accepted the recommendations and directed the formation of five teams led by senior managers to make process improvements:

• TCSU Follow Up – properly validate and test the rotator control system, implement system safeguards, and strengthen project management.
• Project Oversight – establish a head of major projects to lead project oversight, assuring all major projects have appropriate plans and project teams.
• Technical Expertise – analyze and identify expertise gaps, develop internal capabilities, and assure projects are staffed for success.
• Equipment Safety – implement appropriate system safeguards, improve equipment safety incident awareness and follow up, and address deficiencies in equipment safety work culture.
• Operations and Maintenance – address deficiencies in rotator maintenance discovered during the investigation.

These five teams are expected to formulate and implement improvements by the end of April 2017 and are being held accountable by the director.

In conducting this investigation process, our management team consciously chose to establish full transparency of the investigation process and follow up as an essential element to full accountability and to promote lasting institutional learning.  The full unabridged external review report was shared with all staff members and the Science Steering Committee (SSC) and discussed at the February SSC and Board meetings. 

In addition, the accident and investigation findings were shared with our peer observatories on Maunakea, resulting in the following heartfelt acknowledgement by a manager from another observatory:  “You set an example of how to truly learn from mistakes and become better and stronger.  We often pretend and believe we don’t make mistakes when making mistakes is what all humans do and should always be part of considerations in all our activities.”

While we would never wish to have such an accident occur, the observatory is committed to learning from this mistake and emerging an even stronger organization.  We are very grateful to our colleagues who served on the investigation and external review committees, and appreciate the open and forthright response by our staff in the investigation process.
We are relieved the instrument is now back in service and in the hands of our observers. ✶

Adaptive Optics Development Update

Peter Wizinowich, Optical Systems Manager, WMKO

The new TOPTICA/MPBC Raman-fiber amplifier laser that was commissioned on Keck II in early 2016 is continuing to operate very reliably providing an ~ 8th magnitude laser guide star. In addition to its reliability, the efficiency of the new laser is saving the observatory $25K of electricity cost per year.

The first two papers from the NIRC2 L-band vortex coronagraph were published in January 2017 along with a press release. A series of additional science papers are currently in the works.

The Keck I near-infrared tip-tilt sensor (TRICK) had suffered from reliability issues that appear to be largely resolved. The system clearly provides better tip-tilt performance than the visible (STRAP) tip-tilt sensor on faint stars. The projects to implement a near-infrared pyramid wavefront sensor and fiber injection unit on Keck II are being developed in parallel with a joint preliminary design review scheduled for March 29. The pyramid wavefront sensor is being implemented to support a 150 M-star NIRC2 L-band vortex survey for planets around M-stars. The pyramid wavefront sensor will also support injecting exoplanet light into a single mode fiber feeding NIRSPEC.

The WMKO and UCLA-led PSF reconstruction demonstration projects were successfully closed out with a review in September 2016. The review also covered plans for a UC Office of the President funded science verification phase in FY17. Two science teams, studying low mass binaries and QSO host galaxies respectively, are evaluating the improvements in quantitative science with the reconstructed PSFs from AO telemetry.

Both the Keck I and II low bandwidth wavefront sensor and Keck II acquisition cameras are being upgraded this year to replace aging systems. Further improvements are planned for the low bandwidth wavefront sensor camera to reduce dark and read noise.

A series of modest AO improvements are also underway in FY16 including optimization of the laser beam train alignments for efficiency, reduction of heat sources on the AO benches, reduced on-sky focus errors, AO control loop gain optimizations, better understanding of the impact of segment piston errors, and the procurement of new wavefront sensor dichroics to reduce non-common path aberrations. A re-calibration of the Keck II image sharpening algorithm has considerably reduced the halo speckles with a 100 nm rms wavefront error reduction.

Two of the Keck science instruments used with AO are undergoing detector upgrades as well. The OSIRIS spectrograph detector has already been replaced with an H2RG and the imager detector upgrade will be taking place this year. NIRSPEC's spectrograph and slit viewing detectors will be replaced in 2018. Science productivity continues to be high with 78 refereed science papers published in 2016 using Keck AO; including 61% of all laser guide star AO based papers published worldwide.

Back Issues

Please see the Keck Observers’ Newsletter Archive.