{ "cells": [ { "cell_type": "markdown", "metadata": { "id": "HozF_O0VM6-B" }, "source": [ "This notebook is a brief guide of downloading OSIRIS raw (level 0) data in a given period. It requires PI credentials to access proprietary data. For more detail about PyKOA, please see:\n", "http://koa.ipac.caltech.edu/UserGuide/PyKOA/notebooks/PyKOA_OSIRIS_introduction.html" ] }, { "cell_type": "markdown", "metadata": { "id": "mmPy9VTCNrt-" }, "source": [ "First, we need to install the KOA python client PyKOA. This line can be removed if PyKOA is already installed." ] }, { "cell_type": "code", "execution_count": 1, "metadata": { "colab": { "base_uri": "https://localhost:8080/" }, "id": "T3H-B5qiOHGo", "outputId": "507b7c68-12c3-4c22-d153-3c1a3f684e2a" }, "outputs": [ { "output_type": "stream", "name": "stdout", "text": [ "Collecting pykoa\n", " Downloading pykoa-1.7.2-py3-none-any.whl (29 kB)\n", "Requirement already satisfied: requests in /usr/local/lib/python3.10/dist-packages (from pykoa) (2.31.0)\n", "Collecting xmltodict (from pykoa)\n", " Downloading xmltodict-0.13.0-py2.py3-none-any.whl (10.0 kB)\n", "Collecting bs4 (from pykoa)\n", " Downloading bs4-0.0.1.tar.gz (1.1 kB)\n", " Preparing metadata (setup.py) ... \u001b[?25l\u001b[?25hdone\n", "Requirement already satisfied: lxml in /usr/local/lib/python3.10/dist-packages (from pykoa) (4.9.3)\n", "Requirement already satisfied: beautifulsoup4 in /usr/local/lib/python3.10/dist-packages (from bs4->pykoa) (4.11.2)\n", "Requirement already satisfied: charset-normalizer<4,>=2 in /usr/local/lib/python3.10/dist-packages (from requests->pykoa) (3.2.0)\n", "Requirement already satisfied: idna<4,>=2.5 in /usr/local/lib/python3.10/dist-packages (from requests->pykoa) (3.4)\n", "Requirement already satisfied: urllib3<3,>=1.21.1 in /usr/local/lib/python3.10/dist-packages (from requests->pykoa) (2.0.4)\n", "Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.10/dist-packages (from requests->pykoa) (2023.7.22)\n", "Requirement already satisfied: soupsieve>1.2 in /usr/local/lib/python3.10/dist-packages (from beautifulsoup4->bs4->pykoa) (2.4.1)\n", "Building wheels for collected packages: bs4\n", " Building wheel for bs4 (setup.py) ... \u001b[?25l\u001b[?25hdone\n", " Created wheel for bs4: filename=bs4-0.0.1-py3-none-any.whl size=1256 sha256=505318b97126e76878967a1c83ba56e9b7036dca4ca6c5f73c90160af6d31d43\n", " Stored in directory: /root/.cache/pip/wheels/25/42/45/b773edc52acb16cd2db4cf1a0b47117e2f69bb4eb300ed0e70\n", "Successfully built bs4\n", "Installing collected packages: xmltodict, bs4, pykoa\n", "Successfully installed bs4-0.0.1 pykoa-1.7.2 xmltodict-0.13.0\n" ] } ], "source": [ "!pip install pykoa" ] }, { "cell_type": "markdown", "metadata": { "id": "hhVJIWD0OKsS" }, "source": [ "The next steop is to import pre-requisite packages" ] }, { "cell_type": "code", "execution_count": 2, "metadata": { "id": "4G2fXjioOKBa" }, "outputs": [], "source": [ "import sys\n", "import io\n", "import os\n", "from pykoa.koa import Koa\n", "from astropy.table import Table,Column" ] }, { "cell_type": "markdown", "metadata": { "id": "9jJgI5RBOOOX" }, "source": [ "Then we need to assign directory that stores the query results:" ] }, { "cell_type": "code", "execution_count": 3, "metadata": { "id": "OB_GgjpsOXVk" }, "outputs": [], "source": [ "try:\n", " os.mkdir('./output')\n", "except:\n", " print(\" Directory exists already\", flush=True)" ] }, { "cell_type": "markdown", "metadata": { "id": "Q3-Uu0RiOWm9" }, "source": [ "After that, we use a cookie for KOA log in. We can also spell out the credentials, but for security purpose, it is better to enter the userid and password interactively" ] }, { "cell_type": "code", "execution_count": 4, "metadata": { "colab": { "base_uri": "https://localhost:8080/" }, "id": "olSx2E55OcgS", "outputId": "ded7fc2e-b4b6-4343-d79b-a73badea098b" }, "outputs": [ { "output_type": "stream", "name": "stdout", "text": [ "Userid: clee\n", "Password: ··········\n", "Successfully login as clee\n" ] } ], "source": [ "Koa.login ('./tapcookie.txt')\n", "#Koa.login ('./tapcookie.txt', userid='clee', password='salami')" ] }, { "cell_type": "markdown", "metadata": { "id": "kBQdEWm9OdOs" }, "source": [ "We can then proceed to query OSIRIS data taken in a given period. Note the time period is specified in UT date. The output table will be saved in the designated directory and in IPAC table format." ] }, { "cell_type": "code", "execution_count": 5, "metadata": { "colab": { "base_uri": "https://localhost:8080/" }, "id": "VaeWQLWROveh", "outputId": "bce768d8-9e12-435b-e107-96be0f0cf93d" }, "outputs": [ { "output_type": "stream", "name": "stdout", "text": [ "submitting request...\n", "Result downloaded to file [./output/osiris_daterange.tbl]\n", " koaid instrume targname object ... filter instr scale\n", " ... \n", "---------------------- -------- ----------- -------- ... ------ ----- -----\n", "OS.20210702.02021.fits OSIRIS -- ... Kn3 spec 0.02\n", "OS.20210702.02111.fits OSIRIS -- ... Kn3 spec 0.02\n", "OS.20210702.02198.fits OSIRIS -- ... Kn3 spec 0.035\n", "OS.20210702.02269.fits OSIRIS -- ... Kn3 spec 0.05\n", "OS.20210702.02333.fits OSIRIS -- ... Kn3 spec 0.1\n", "OS.20210702.06867.fits OSIRIS -- ... Kbb spec 0.02\n", "OI.20210702.06936.fits OSIRIS -- ... Kp imag 0.02\n", "OS.20210702.19914.fits OSIRIS HIP76068 HIP76068 ... Kbb spec 0.05\n", "OS.20210702.20019.fits OSIRIS HIP76068 HIP76068 ... Kbb spec 0.05\n", "OS.20210702.20102.fits OSIRIS HIP76068 HIP76068 ... Kbb spec 0.02\n", " ... ... ... ... ... ... ... ...\n", "OS.20210702.37975.fits OSIRIS s15552+5854 HIP84147 ... Drk spec 0.02\n", "OS.20210702.38023.fits OSIRIS s15552+5854 HIP84147 ... Drk spec 0.02\n", "OS.20210702.38070.fits OSIRIS s15552+5854 HIP84147 ... Drk spec 0.02\n", "OS.20210702.38117.fits OSIRIS s15552+5854 HIP84147 ... Drk spec 0.02\n", "OS.20210702.38164.fits OSIRIS s15553+5903 HIP84147 ... Drk spec 0.02\n", "OS.20210702.38212.fits OSIRIS s15553+5903 HIP84147 ... Drk spec 0.02\n", "OS.20210702.38288.fits OSIRIS s15553+5903 HIP84147 ... Drk spec 0.02\n", "OS.20210702.38365.fits OSIRIS hr5949 HIP84147 ... Drk spec 0.02\n", "OS.20210702.38442.fits OSIRIS hr5949 HIP84147 ... Drk spec 0.02\n", "OS.20210702.38519.fits OSIRIS hr5949 HIP84147 ... Drk spec 0.02\n", "Length = 309 rows\n" ] } ], "source": [ "Koa.query_datetime ('osiris', \\\n", " '2021-07-02 00:00:00/2021-07-02 23:59:59', \\\n", " './output/osiris_daterange.tbl', overwrite=True, format='ipac', \\\n", " cookiepath='./tapcookie.txt')\n", "\n", "rec = Table.read ('./output/osiris_daterange.tbl',format='ipac')\n", "print (rec)\n" ] }, { "cell_type": "markdown", "metadata": { "id": "KV5BDsU4Osxd" }, "source": [ "Once we have the query result, we can download files to the designated directory, in this case to dnload_dir_osiris/. If the download function is called several times, it will check the designated directory and only download new files." ] }, { "cell_type": "code", "execution_count": 6, "metadata": { "colab": { "base_uri": "https://localhost:8080/" }, "id": "KxRXJeruPLET", "outputId": "22ce2adf-8153-4c1b-8e74-372bea505dda" }, "outputs": [ { "output_type": "stream", "name": "stdout", "text": [ "Start downloading 3 koaid data you requested;\n", "please check your outdir: dnload_dir_osiris for progress ....\n", "\n", "A total of 3 new lev0 FITS files downloaded.\n" ] } ], "source": [ "Koa.download ('./output/osiris_daterange.tbl',\n", " 'ipac', \\\n", " 'dnload_dir_osiris', \\\n", "# start_row=0, \\\n", "# end_row = 2, \\\n", " cookiepath='./tapcookie.txt')" ] } ], "metadata": { "colab": { "provenance": [] }, "kernelspec": { "display_name": "Python 3 (ipykernel)", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.11.3" } }, "nbformat": 4, "nbformat_minor": 0 }