{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "0",
   "metadata": {},
   "source": [
    "[![image](https://jupyterlite.rtfd.io/en/latest/_static/badge.svg)](https://demo.leafmap.org/lab/index.html?path=notebooks/57_national_map.ipynb)\n",
    "[![image](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/opengeos/leafmap/blob/master/docs/notebooks/57_national_map.ipynb)\n",
    "[![image](https://mybinder.org/badge_logo.svg)](https://mybinder.org/v2/gh/opengeos/leafmap/HEAD)\n",
    "\n",
    "### Downloading various shapes from the National Map\n",
    "\n",
    "The national map (TNM) is a catalog of *topological* datasources maintained by the USGS. \n",
    "\n",
    "- It contains a wide range of dataformats (such as GeoTiff, LAZ, ...) and datasets.\n",
    "- It provides an endpoint that can be used to search for published datasets and files.\n",
    "- This API supports a wide range of searchable parameters (bounding box, polygon, dates, keyword, ...)\n",
    "- It returns detailed information regarding the properties of datasets, file,\n",
    "- as well as various download links (file, thumbnail, xml descriptions, ...).\n",
    "\n",
    "We've created a thin wrapper to expose this treasure trove. \n",
    "\n",
    "- For more details about TNM, see https://apps.nationalmap.gov/tnmaccess/#/\n",
    "- The same data is also downloable using https://apps.nationalmap.gov/downloader/"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "1",
   "metadata": {},
   "outputs": [],
   "source": [
    "# !pip install leafmap"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "2",
   "metadata": {},
   "outputs": [],
   "source": [
    "import leafmap"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3",
   "metadata": {},
   "source": [
    "### Usage\n",
    "\n",
    "A class groups the functionalities together."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "4",
   "metadata": {},
   "outputs": [],
   "source": [
    "TNM = leafmap.The_national_map_USGS()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5",
   "metadata": {},
   "source": [
    "### Datasets"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "6",
   "metadata": {},
   "outputs": [],
   "source": [
    "TNM.datasets"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7",
   "metadata": {},
   "source": [
    "### Formats\n",
    "\n",
    "Note that any format (f.e. 'All') is specific to one or more datasets."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "8",
   "metadata": {},
   "outputs": [],
   "source": [
    "TNM.prodFormats"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9",
   "metadata": {},
   "source": [
    "### Looking for files"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "10",
   "metadata": {},
   "outputs": [],
   "source": [
    "TNM.find_details().keys(), TNM.find_details()[\"total\"]"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "11",
   "metadata": {},
   "source": [
    "### A detail"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "12",
   "metadata": {},
   "outputs": [],
   "source": [
    "TNM.find_details()[\"items\"][0]"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "13",
   "metadata": {},
   "source": [
    "### Using parameters"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "14",
   "metadata": {},
   "outputs": [],
   "source": [
    "params = {\n",
    "    \"q\": \"National Elevation Dataset (NED) 1/3 arc-second\",\n",
    "    \"polyCode\": \"01010002\",\n",
    "    \"polyType\": \"huc8\",\n",
    "}\n",
    "\n",
    "TNM.find_details(**params)[\"total\"]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "15",
   "metadata": {},
   "outputs": [],
   "source": [
    "params = {\n",
    "    \"prodFormats\": \"LAS,LAZ\",\n",
    "    \"datasets\": \"Lidar Point Cloud (LPC)\",\n",
    "    \"polygon\": [\n",
    "        (-104.94262695312236, 41.52867510196275),\n",
    "        (-102.83325195312291, 40.45065268246805),\n",
    "        (-104.94262695312236, 40.45065268246805),\n",
    "        (-104.94262695312236, 41.52867510196275),\n",
    "    ],\n",
    "}\n",
    "\n",
    "TNM.find_details(**params)[\"total\"]"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "16",
   "metadata": {},
   "source": [
    "Available parameters"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "17",
   "metadata": {},
   "outputs": [],
   "source": [
    "help(TNM.find_details)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "18",
   "metadata": {},
   "source": [
    "### Max items\n",
    "\n",
    "Defaults to about 50. You only retrieve about 1000 items in one call."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "19",
   "metadata": {},
   "outputs": [],
   "source": [
    "len(TNM.find_details()[\"items\"])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "20",
   "metadata": {},
   "outputs": [],
   "source": [
    "len(TNM.find_details(max=1000000)[\"items\"])"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "21",
   "metadata": {},
   "source": [
    "Use offset to retrieve more batches."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "22",
   "metadata": {},
   "outputs": [],
   "source": [
    "params = {\n",
    "    \"q\": \"National Elevation Dataset (NED) 1/3 arc-second\",\n",
    "    \"polyCode\": \"01010002\",\n",
    "    \"polyType\": \"huc8\",\n",
    "    \"max\": 2,\n",
    "}\n",
    "\n",
    "TNM.find_details(**params, offset=0)[\"items\"][0] == TNM.find_details(\n",
    "    **params, offset=1\n",
    ")[\"items\"][0]"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "23",
   "metadata": {},
   "source": [
    "### Select a region from leafmap"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "24",
   "metadata": {},
   "outputs": [],
   "source": [
    "m = leafmap.Map(center=[40, -100], zoom=4)\n",
    "m"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "25",
   "metadata": {},
   "outputs": [],
   "source": [
    "region = m.user_roi_bounds()\n",
    "if region is None:\n",
    "    region = [-115.9689, 35.9758, -115.3619, 36.4721]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "26",
   "metadata": {},
   "outputs": [],
   "source": [
    "TNM.find_details(q=\"LAZ\", bbox=region)[\"total\"]"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "27",
   "metadata": {},
   "source": [
    "### Error handling"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "28",
   "metadata": {},
   "outputs": [],
   "source": [
    "bool(TNM.find_details(start=\"01-01-2010\", q=\"NED\", bbox=region))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "29",
   "metadata": {},
   "outputs": [],
   "source": [
    "bool(TNM.find_details(start=\"2021-12-01\", end=\"2020-01-01\", q=\"NED\", bbox=region))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "30",
   "metadata": {},
   "outputs": [],
   "source": [
    "bool(TNM.find_details(start=\"2021-12-01\", end=\"2022-01-01\", q=\"NED\", bbox=region))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "31",
   "metadata": {},
   "outputs": [],
   "source": [
    "bool(\n",
    "    TNM.find_details(\n",
    "        start=\"2020-12-01\",\n",
    "        end=\"2022-01-01\",\n",
    "        q=\"NED\",\n",
    "        dateType=\"dateCreated\",\n",
    "        bbox=region,\n",
    "    )\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "32",
   "metadata": {},
   "source": [
    "### Downloading files"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "33",
   "metadata": {},
   "outputs": [],
   "source": [
    "help(TNM.download_tiles)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "34",
   "metadata": {},
   "outputs": [],
   "source": [
    "params = {\n",
    "    \"q\": \"National Elevation Dataset (NED) 1/3 arc-second\",\n",
    "    \"polyCode\": \"01010002\",\n",
    "    \"polyType\": \"huc8\",\n",
    "    \"max\": 0,\n",
    "}\n",
    "\n",
    "TNM.download_tiles(API=params)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "35",
   "metadata": {},
   "source": [
    "It can also be accessed without invoking the class."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "36",
   "metadata": {},
   "outputs": [],
   "source": [
    "params = {\n",
    "    \"q\": \"National Elevation Dataset (NED) 1/3 arc-second\",\n",
    "    \"polyCode\": \"01010002\",\n",
    "    \"polyType\": \"huc8\",\n",
    "    \"max\": 0,\n",
    "}\n",
    "\n",
    "leafmap.download_tnm(API=params)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "37",
   "metadata": {},
   "outputs": [],
   "source": [
    "region = [-115.9689, 35.9758, -115.3619, 36.4721]\n",
    "\n",
    "leafmap.download_ned(region=region, return_url=True) == leafmap.download_tnm(\n",
    "    region=region, return_url=True, API={\"q\": \"NED\"}\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "38",
   "metadata": {},
   "source": [
    "### List of files"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "39",
   "metadata": {},
   "outputs": [],
   "source": [
    "TNM.find_tiles(API=params)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "40",
   "metadata": {},
   "source": [
    "### Dataset metadata"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "41",
   "metadata": {},
   "outputs": [],
   "source": [
    "TNM.datasets_full[0]"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "42",
   "metadata": {},
   "source": [
    "### Read the docs"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "43",
   "metadata": {},
   "outputs": [],
   "source": [
    "help(TNM)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}