{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Using Leafmap and Earthaccess to Explore OPERA Products"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The Leafmap library provides a suite of tools for interactive mapping and visualization in Jupyter Notebooks Leafmap version 0.30.0 and and later offer tools specifically for accessing NASA Earthdata by building on the newly developed NASA Earthaccess library. Earthaccess provides streamlined access to NASA Earthdata and simplifies the authentication and querying process over previously developed approaches.This notebook is designed to leverage tools within Earthaccess and Leafmap to facility easier access and vizualization of OPERA data products for a user-specified area of interest (AOI). \n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Import Libraries"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import earthaccess\n",
    "import leafmap\n",
    "import pandas as pd\n",
    "import geopandas as gpd\n",
    "from shapely import box\n",
    "from datetime import datetime"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Authentication \n",
    "A [NASA Earthdata Login](https://urs.earthdata.nasa.gov/) account is required to download the data used in this tutorial. You can create an account at the link provided. After establishing an account, the code in the next cell will verify authentication. If this is your first time running the notebook, you will be prompted to enter your Earthdata login credentials, which will be saved in ~/.netrc."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "leafmap.nasa_data_login()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## View NASA Earthdata datasets\n",
    "A tab separated values (TSV) file, made available through the opengeos Github repository, catalogues metadata for more than 9,000 datasets available through NASA Earthdata. In the next cell we load the TSV into a pandas dataframe and view the metadata for the first five (5) Earthdata products"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "### View Earthdata datasets\n",
    "earthdata_url = 'https://github.com/opengeos/NASA-Earth-Data/raw/main/nasa_earth_data.tsv'\n",
    "earthdata_df = pd.read_csv(earthdata_url, sep='\\t')\n",
    "earthdata_df.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## View the available OPERA products\n",
    "Note above that the `earthdata_df` contains a number of columns with metadata about each available product. the `ShortName` column will be used to produce a new dataframe containing only OPERA products. Let's view the available products and their metadata."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "opera_df = earthdata_df[earthdata_df['ShortName'].str.contains('OPERA', case=False)]\n",
    "opera_df"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Create an interactive Leaflet map to define an area of interest (AOI)\n",
    "Leafmap provides the capability to create an interactive map in the Jupyter Notebook and to use the cursor to define an area of interest (AOI). When the user defines an AOI, the boundary is saved as an object. Give this a try in the next cell."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "m = leafmap.Map(center=[42,-100], zoom=4, basemap='Google Satellite')\n",
    "m"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "### This cell initializes the AOI for future use. If the user didn't define an AOI in the previous cell, the AOI defaults to an bounding box over Ridgecrest, California (USA)\n",
    "if m.user_roi is not None:\n",
    "    AOI = tuple(m.user_roi_bounds())\n",
    "else:\n",
    "    AOI = (-117.33, 35.541, -117.880, 35.991) #Defaults to Ridgecrest, CA"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Query Earthdata and return metadata for OPERA products within the AOI\n",
    "The `earthaccess` library makes it simple to quickly query NASA's Common Metadata Repository (CMR) and return the associated metadata as a Geodataframe. `Leafmap` has recently added functionality that builds on `earthaccess` to enable interactive viewing of this data. \n",
    "In the next cell, the user should specify which OPERA product and the date range of interest. The AOI defined previously is used as the boundary in the query."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### View OPERA Product Shortnames"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "### Print the available OPERA datasets\n",
    "print('Available OPERA datasets:', opera_df['ShortName'].values)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Query the OPERA DSWx-HLS dataset for the AOI\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "dswx_results, dswx_gdf = leafmap.nasa_data_search(\n",
    "    short_name='OPERA_L3_DSWX-HLS_PROVISIONAL_V1',\n",
    "    cloud_hosted=True,\n",
    "    bounding_box= AOI,\n",
    "    temporal=(\"2023-10-01\", str(datetime.now().date())),\n",
    "    count=-1,  # use -1 to return all datasets\n",
    "    return_gdf=True,\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### See the available DSWx-HLS layers\n",
    "Functionality within earthaccess enables more more asthetic views of the available layers, as well as displaying the thumbnail. These links are clickable and will download in the browser when clicked. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "dswx_results[0]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The `leafmap.nasa_data_search()` function returns a Geodataframe containing the metadata for all granules which intersect the AOI from the specified time range. Let's look at the first five granules."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### View the DSWx-HLS metadata and footprints"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "dswx_gdf.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "### Plot the location of the tiles \n",
    "dswx_gdf.explore(fill=False)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Query the OPERA DIST-ALERT-HLS dataset for the AOI"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "dist_results, dist_gdf = leafmap.nasa_data_search(\n",
    "    short_name='OPERA_L3_DIST-ALERT-HLS_PROVISIONAL_V0',\n",
    "    cloud_hosted=True,\n",
    "    bounding_box= AOI,\n",
    "    temporal=(\"2023-06-01\", str(datetime.now().date())),\n",
    "    count=-1,  # use -1 to return all datasets\n",
    "    return_gdf=True,\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### See the available DIST-ALERT-HLS layers"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "dist_results[0]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### View the DIST-ALERT-HLS metadata and footprints"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "dist_gdf.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "### Plot the location of the tiles \n",
    "dist_gdf.explore(fill=False)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Query the OPERA RTC-S1 dataset for the AOI"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "rtc_results, rtc_gdf = leafmap.nasa_data_search(\n",
    "    short_name='OPERA_L2_RTC-S1_V1',\n",
    "    cloud_hosted=True,\n",
    "    bounding_box= AOI,\n",
    "    temporal=(\"2023-06-01\", str(datetime.now().date())),\n",
    "    count=-1,  # use -1 to return all datasets\n",
    "    return_gdf=True,\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### See the available RTC layers"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "rtc_results[0]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### View the RTC-S1 metadata and footprints"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "rtc_gdf.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "rtc_gdf.explore(fill=False)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Query the OPERA CSLC-S1 dataset  for the AOI\n",
    "**Note: This will only work for AOIs over North America as this is the extent of CSLC coverage**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "cslc_results, cslc_gdf = leafmap.nasa_data_search(\n",
    "    short_name='OPERA_L2_CSLC-S1_V1',\n",
    "    cloud_hosted=True,\n",
    "    bounding_box= AOI,\n",
    "    temporal=(\"2023-06-01\", str(datetime.now().date())),\n",
    "    count=-1,  # use -1 to return all datasets\n",
    "    return_gdf=True,\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### See the available CSLC-S1 layers"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "cslc_results[0]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### View the CSLC-S1 metadata and footprints"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "cslc_gdf.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "cslc_gdf.explore(fill=False)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Download data with leafmap\n",
    "*<font color='red'>Important note: As of Jan. 2024, OPERA RTC and CSLC may not be accessible using the earthaccess library due to additional athentication measures required by ASF DAAC. This will likely be resolved soon, but currently the notebook should be used only to access OPERA products distributed by LP DAAC and PO.DAAC. For additional details regarding the current status of using earthaccess to access data from ASF DAAC, see the linked Github Issue ([#439](https://github.com/nsidc/earthaccess/issues/439#issuecomment-1915518987)).</font>*\n",
    "\n",
    "Let's download some data from one of our above queries. In the cell below we specify data from the DSWx query, but feel free to modify to any of the others above. *Note: We also filter to include the layer we would like to from the product we would like. So modify this step accordingly if a different product is chosen."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Create a subdirectory\n",
    "This will be where the files are downloaded. It will be a subdirectory inside of a directory called `data`, and the directory name will be the datetime that it was created."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import os\n",
    "from datetime import datetime\n",
    "\n",
    "def create_data_directory():\n",
    "    # Get the current date and time\n",
    "    current_datetime = datetime.now().strftime(\"%m_%d_%Y_%H_%M_%S\")\n",
    "\n",
    "    # Define the base directory\n",
    "    base_directory = \"data\"\n",
    "\n",
    "    # Create the full path for the new directory\n",
    "    new_directory_path = os.path.join(base_directory, f\"data_{current_datetime}\")\n",
    "\n",
    "    # Create the new directory\n",
    "    os.makedirs(new_directory_path, exist_ok=True)\n",
    "    print(f\"Directory '{new_directory_path}' created successfully.\")\n",
    "\n",
    "    return new_directory_path\n",
    "\n",
    "directory_path = create_data_directory()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Download the data\n",
    "The below will download the data to your newly created subdirectory. Look on your file system for a directory `/data/datetime/` where `datetime` is the date and time the directory was created."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "dswx_data = leafmap.nasa_data_download(dswx_results[:5], out_dir=directory_path)     # Downloads the first 5 granules. Remove [:5] to download all granules or modify to keep as many as you like"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## View the files using Leafmap"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Load in images from data folder\n",
    "We load in data from only band 1 below. If you'd like load data from a different band change the `B01` to suit your needs."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import os\n",
    "\n",
    "# Get the current directory\n",
    "current_directory = os.getcwd()\n",
    "\n",
    "# Construct the path to the data directory\n",
    "data_directory = os.path.join(current_directory, directory_path)\n",
    "\n",
    "# Create a list of file paths and a list of corresponding dates\n",
    "images = [os.path.join(data_directory, filename) for filename in os.listdir(data_directory) if os.path.isfile(os.path.join(data_directory, filename)) and 'B01' in filename]\n",
    "image_dates = [image[25:33] for image in os.listdir(data_directory) if 'B01' in image]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Display the first image in the directory"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "m = leafmap.Map()\n",
    "m.add_raster(images[0])\n",
    "m"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Create a split map with the first and last granule\n",
    "Create a map of the first and last image within the directory"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "leafmap.split_map(\n",
    "    left_layer=images[0],\n",
    "    right_layer=images[-1],\n",
    "    left_label=\"First\",\n",
    "    right_label=\"Last\",\n",
    "    label_position=\"bottom\",\n",
    "    zoom=10,\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Draw a polygon around the feature of interest interactively"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "m = leafmap.Map()\n",
    "m.add_raster(images[0])\n",
    "m"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Save output as a geojson\n",
    "m.save_draw_features(\"test_output.geojson\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Create a timelapse (.gif) of the images over this region"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "leafmap.create_timelapse(\n",
    "    images,\n",
    "    out_gif='dswx.gif',\n",
    "    fps=1,\n",
    "    progress_bar_color='blue',\n",
    "    add_text=True,\n",
    "    text_xy=('3%', '3%'),\n",
    "    text_sequence=[str(date) for date in image_dates],\n",
    "    font_size=20,\n",
    "    font_color='red',\n",
    "    mp4=False,\n",
    "    reduce_size=False,\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "leafmap.show_image('dswx.gif',height=\"450px\", width=\"450px\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## View OPERA data in Leafmap GUI"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "m = leafmap.Map()\n",
    "m.add(\"nasa_earth_data\")\n",
    "m"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### View the results as a Geodataframe"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "m._NASA_DATA_GDF.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Download the Displayed Footprints"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# leafmap.nasa_data_download(m._NASA_DATA_RESULTS[:5], out_dir=\"data\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Make a timeslider to visualize granules\n",
    "Make an interactive timeslide to cycle through the images. This isn't the perfect solution, as the granules in our list do not have identical footprints."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "m = leafmap.Map()\n",
    "m.add_time_slider(\n",
    "    images,\n",
    "    time_interval=1,\n",
    "    position='bottomright',\n",
    "    zoom_to_layer=True,\n",
    ")\n",
    "m"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Conclusions\n",
    "`earthaccess` and `leafmap` provide means for very simple access to OPERA data and metadata. Additional code may be required for more sophisticated filtering (cloud cover, spatial overlap). This notebook provides a guide for learning more about OPERA data products and simple funtionality for their exploration. For more details and application see the main [OPERA Applications Github repository](https://github.com/OPERA-Cal-Val/OPERA_Applications)."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "opera_app",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.13"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}