{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## >>This script no longer works as the USGS ftp site has been shut down<<\n",
    "\n",
    "### Download watershed boundaries for a specific HUC\n",
    "Here we use the `urllib` and `zipfile` packages to download and unzip HUCs for a state specified in the first code box. Once downloaded, we see how the `geopandas` package is used to analyze and visualize the data. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#Set the state\n",
    "state = 'Maryland'"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#Import modules\n",
    "import os\n",
    "import urllib\n",
    "import zipfile #https://pymotw.com/2/zipfile/\n",
    "import geopandas as gpd #http://geopandas.org/io.html"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#Allow maps to be shown inline...\n",
    "%matplotlib inline"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The data we want is stored on an FTP server:ftp://rockyftp.cr.usgs.gov/vdelivery/Datasets/Staged/Hydrography/NHD/State/HighResolution/Shape). Open the link (*you'll have to copy and paste the link...*) and view the files. The zip files contain the shapefiles we want, and the all have a set name of `NHD_H_***_State_Shape.zip`, where `***` is the state name. \n",
    "\n",
    "So, what we want to do is create two variables. The first points to the FTP folder containing the file we want, and the second is the name of the actual file we want. Separating the name is a bit easier to manage and also gives us a proper output file name. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#Set the URLs\n",
    "ftpFolder = 'ftp://rockyftp.cr.usgs.gov/vdelivery/Datasets/Staged/Hydrography/NHD/State/HighResolution/Shape/'\n",
    "stateFile = 'NHD_H_{}_State_Shape.zip'.format(state)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "With these variables set, we construct the full URL, which is just the ftpFolder and the stateFile together with a forward slash between them. \n",
    "\n",
    "Then we use the `urllib.request.urlretrieve` command, passing it the URL we want to retrieve and a filename used to store the file locally. This take a few moments to complete since we are downloading the data in this step. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#Get the file (this can take a few minutes...)\n",
    "url = ftpFolder + \"/\" + stateFile\n",
    "if os.path.exists(stateFile):\n",
    "    print(\"{} already downloaded\".format(stateFile))\n",
    "else:\n",
    "    print(\"Downloading {}\".format(stateFile))\n",
    "    data = urllib.request.urlretrieve(url,stateFile)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We just downloaded the zip file, so we can use the Python `zipfile` package to unpack the contents into a folder we create. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#Unzip the file\n",
    "outFolder = stateFile[:-4]\n",
    "if not os.path.exists(outFolder): os.mkdir(outFolder)\n",
    "zip_ref = zipfile.ZipFile(stateFile)\n",
    "zip_ref.extractall(outFolder)\n",
    "zip_ref.close()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Here we use `GeoPandas` to read the shapefile's contents into a GeoDataFrame. (More on GeoPandas later...)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#HUC8\n",
    "shp = outFolder + os.sep + 'Shape' + os.sep + 'WBDHU8.shp'\n",
    "gdf = gpd.read_file(shp)\n",
    "gdf.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The `plot` method of a geodataframe plots our spatial data to the screen. The `figsize` property sets how big we want to show it. More on the plot function is [here](http://geopandas.org/reference.html?highlight=plot#geopandas.GeoDataFrame.plot).\n",
    "\n",
    "GIS without any ArcGIS! "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#Plot the geodataframe\n",
    "gdf.plot(figsize=(20,10));"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## ► Exercise\n",
    "The FTP folder ftp://rockyftp.cr.usgs.gov/vdelivery/Datasets/Staged/LndCvr/Shape/ contains land cover shapes for all US states. Use the example above to write code blocks to download and display land cover for Hawaii. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "state = 'Delaware'"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#Set the URLs\n",
    "ftpFolder = \n",
    "stateFile = "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#Get the file (this can take a few minutes...)\n",
    "url = \n",
    "data = "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#Unzip the file\n",
    "outFolder = stateFile[:-4]\n",
    "if not os.path.exists(outFolder): os.mkdir(outFolder)\n",
    "zip_ref = zipfile.ZipFile(stateFile)\n",
    "zip_ref.extractall(outFolder)\n",
    "zip_ref.close()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#LandCover\n",
    "shp = outFolder + os.sep + 'Shape' + os.sep + 'LandCover_Woodland.shp'\n",
    "gdf = gpd.read_file(shp)\n",
    "gdf.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#Plot\n",
    "gdf.plot(figsize=(20,10));"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.13"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}