{
 "cells": [
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "[![image](https://colab.research.google.com/assets/colab-badge.svg)](https://gishub.org/whitebox-colab)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# A tutorial for the whitebox Python package\n",
    "\n",
    "This notebook demonstrates the usage of the **whitebox** Python package for geospatial analysis, which is built on a stand-alone executable command-line program called [WhiteboxTools](https://github.com/jblindsay/whitebox-tools).\n",
    "\n",
    "* GitHub repo: https://github.com/opengeos/whitebox-python\n",
    "* WhiteboxTools: https://github.com/jblindsay/whitebox-tools\n",
    "* WhiteboxGeo: https://www.whiteboxgeo.com\n",
    "* User Manual: https://www.whiteboxgeo.com/manual/wbt_book/intro.html\n",
    "* PyPI: https://pypi.org/project/whitebox/\n",
    "* Documentation: https://whitebox.readthedocs.io\n",
    "* Binder: https://gishub.org/whitebox-cloud\n",
    "* Free software: [MIT license](https://opensource.org/licenses/MIT)\n",
    "\n",
    "**Launch this tutorial as an interactive Jupyter Notebook on the cloud - [MyBinder.org](https://gishub.org/whitebox-cloud).**\n",
    "\n",
    "![whitebox-gif](https://i.imgur.com/LF4UE1j.gif)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Table of Content\n",
    "\n",
    "* [Installation](#Installation)\n",
    "* [About whitebox](#About-whitebox)\n",
    "* [Getting data](#Getting-data)\n",
    "* [Using whitebox](#Using-whitebox)\n",
    "* [Displaying results](#Displaying-results)\n",
    "* [whitebox GUI](#whitebox-GUI)\n",
    "* [Citing whitebox](#Citing-whitebox)\n",
    "* [Credits](#Credits)\n",
    "* [Contact](#Contact)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Installation\n",
    "\n",
    "\n",
    "**whitebox** supports a variety of platforms, including Microsoft Windows, macOS, and Linux operating systems. Note that you will need to have **Python 3.x** installed. Python 2.x is not supported. The **whitebox** Python package can be installed using the following command:\n",
    "\n",
    "`pip install whitebox`\n",
    "\n",
    "If you have installed **whitebox** Python package before and want to upgrade to the latest version, you can use the following command:\n",
    "\n",
    "`pip install whitebox -U`\n",
    "\n",
    "If you encounter any installation issues, please check [Troubleshooting](https://github.com/opengeos/whitebox#troubleshooting) on the **whitebox** GitHub page and [Report Bugs](https://github.com/opengeos/whitebox#reporting-bugs)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%pip install whitebox"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## About whitebox\n",
    "\n",
    "**import whitebox and call WhiteboxTools()**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import whitebox\n",
    "\n",
    "wbt = whitebox.WhiteboxTools()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Prints the whitebox-tools help...a listing of available commands**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "print(wbt.help())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Prints the whitebox-tools license**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": false
   },
   "outputs": [],
   "source": [
    "print(wbt.license())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Prints the whitebox-tools version**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print(\"Version information: {}\".format(wbt.version()))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Print the help for a specific tool.**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "print(wbt.tool_help(\"ElevPercentile\"))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Tool names in the whitebox Python package can be called either using the snake_case or CamelCase convention (e.g. lidar_info or LidarInfo). The example below uses snake_case.** "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "import os, pkg_resources\n",
    "\n",
    "# identify the sample data directory of the package\n",
    "data_dir = os.path.dirname(pkg_resources.resource_filename(\"whitebox\", 'testdata/'))\n",
    "\n",
    "# set whitebox working directory\n",
    "wbt.set_working_dir(data_dir)\n",
    "wbt.verbose = False\n",
    "\n",
    "# call whiteboxtools\n",
    "wbt.feature_preserving_smoothing(\"DEM.tif\", \"smoothed.tif\", filter=9)\n",
    "wbt.breach_depressions(\"smoothed.tif\", \"breached.tif\")\n",
    "wbt.d_inf_flow_accumulation(\"breached.tif\", \"flow_accum.tif\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**You can search tools using keywords. For example, the script below searches and lists tools with 'lidar' or 'LAS' in tool name or description.**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "lidar_tools = wbt.list_tools(['lidar', 'LAS'])\n",
    "for index, tool in enumerate(lidar_tools):\n",
    "    print(\"{} {}: {} ...\".format(str(index + 1).zfill(3), tool, lidar_tools[tool][:45]))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**List all available tools in whitebox-tools**. Currently, **whitebox** contains 372 tools. More tools will be added as they become available."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "all_tools = wbt.list_tools()\n",
    "for index, tool in enumerate(all_tools):\n",
    "    print(\"{} {}: {} ...\".format(str(index + 1).zfill(3), tool, all_tools[tool][:45]))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Getting data"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This section demonstrates two ways to get data into Binder so that you can test **whitebox** on the cloud using your own data. \n",
    "\n",
    "* [Getting data from direct URLs](#Getting-data-from-direct-URLs) \n",
    "* [Getting data from Google Drive](#Getting-data-from-Google-Drive)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Getting data from direct URLs\n",
    "\n",
    "If you have data hosted on your own HTTP server or GitHub, you should be able to get direct URLs. With a direct URL, users can automatically download the data when the URL is clicked. For example https://github.com/opengeos/whitebox/raw/master/examples/testdata.zip"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Import the following Python libraries and start getting data from direct URLs."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import os\n",
    "import zipfile\n",
    "import tarfile\n",
    "import shutil\n",
    "import urllib.request"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Create a folder named *whitebox* under the user home folder and set it as the working directory."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "work_dir = os.path.join(os.path.expanduser(\"~\"), 'whitebox')\n",
    "if not os.path.exists(work_dir):\n",
    "    os.mkdir(work_dir)\n",
    "os.chdir(work_dir)\n",
    "print(\"Working directory: {}\".format(work_dir))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Replace the following URL with your own direct URL hosting your data."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "url = \"https://github.com/opengeos/whitebox/raw/master/examples/testdata.zip\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Download data the from the above URL and unzip the file if needed."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# download the file\n",
    "zip_name = os.path.basename(url)\n",
    "zip_path = os.path.join(work_dir, zip_name)\n",
    "\n",
    "print('Downloading {} ...'.format(zip_name))\n",
    "urllib.request.urlretrieve(url, zip_path)\n",
    "print('Downloading done.'.format(zip_name))\n",
    "\n",
    "# if it is a zip file\n",
    "if '.zip' in zip_name:\n",
    "    print(\"Decompressing {} ...\".format(zip_name))\n",
    "    with zipfile.ZipFile(zip_name, \"r\") as zip_ref:\n",
    "        zip_ref.extractall(work_dir)\n",
    "    print('Decompressing done.')\n",
    "\n",
    "# if it is a tar file\n",
    "if '.tar' in zip_name:\n",
    "    print(\"Decompressing {} ...\".format(zip_name))\n",
    "    with tarfile.open(zip_name, \"r\") as tar_ref:\n",
    "        tar_ref.extractall(work_dir)\n",
    "    print('Decompressing done.')\n",
    "\n",
    "print('Data directory: {}'.format(os.path.splitext(zip_path)[0]))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You have successfully downloaded data to Binder. Therefore, you can skip to [Using whitebox](#Using-whitebox) and start testing whitebox with your own data. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Getting data from Google Drive\n",
    "\n",
    "Alternatively, you can upload data to [Google Drive](https://www.google.com/drive/) and then [share files publicly from Google Drive](https://support.google.com/drive/answer/2494822?co=GENIE.Platform%3DDesktop&hl=en). Once the file is shared publicly, you should be able to get a shareable URL. For example, https://drive.google.com/file/d/1xgxMLRh_jOLRNq-f3T_LXAaSuv9g_JnV.\n",
    " \n",
    "To download files from Google Drive to Binder, you can use the Python package called [google-drive-downloader](https://github.com/ndrplz/google-drive-downloader), which can be installed using the following command:\n",
    "\n",
    "`pip install googledrivedownloader requests`"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Replace the following URL with your own shareable URL from Google Drive.**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "gfile_url = 'https://drive.google.com/file/d/1xgxMLRh_jOLRNq-f3T_LXAaSuv9g_JnV'"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Extract the file id from the above URL.**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "file_id = gfile_url.split('/')[5]  #'1xgxMLRh_jOLRNq-f3T_LXAaSuv9g_JnV'\n",
    "print('Google Drive file id: {}'.format(file_id))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Download the shared file from Google Drive.**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from google_drive_downloader import GoogleDriveDownloader as gdd\n",
    "\n",
    "dest_path = './testdata.zip'  # choose a name for the downloaded file\n",
    "gdd.download_file_from_google_drive(file_id, dest_path, unzip=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You have successfully downloaded data from Google Drive to Binder. You can now continue to [Using whitebox](#Using-whitebox) and start testing whitebox with your own data. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Using whitebox"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Here you can specify where your data are located. In this example, we will use [DEM.tif](https://github.com/opengeos/whitebox/blob/master/examples/testdata/DEM.tif), which has been downloaded to the testdata folder."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**List data under the data folder.**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "data_dir = './testdata/'\n",
    "print(os.listdir(data_dir))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In this simple example, we smooth [DEM.tif](https://github.com/opengeos/whitebox/blob/master/examples/testdata/DEM.tif) using a [feature preserving denoising](https://github.com/jblindsay/whitebox-tools/blob/master/src/tools/terrain_analysis/feature_preserving_denoise.rs) algorithm. Then, we fill depressions in the DEM using a [depression breaching](https://github.com/jblindsay/whitebox-tools/blob/master/src/tools/hydro_analysis/breach_depressions.rs) algorithm. Finally, we calculate [flow accumulation](https://github.com/jblindsay/whitebox-tools/blob/master/src/tools/hydro_analysis/dinf_flow_accum.rs) based on the depressionless DEM."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import whitebox\n",
    "\n",
    "wbt = whitebox.WhiteboxTools()\n",
    "# set whitebox working directory\n",
    "wbt.set_working_dir(data_dir)\n",
    "wbt.verbose = False\n",
    "\n",
    "# call whiteboxtool\n",
    "wbt.feature_preserving_smoothing(\"DEM.tif\", \"smoothed.tif\", filter=9)\n",
    "wbt.breach_depressions(\"smoothed.tif\", \"breached.tif\")\n",
    "wbt.d_inf_flow_accumulation(\"breached.tif\", \"flow_accum.tif\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Displaying results\n",
    "\n",
    "This section demonstrates how to display images on Jupyter Notebook. Three Python packages are used here, including [matplotlib](https://matplotlib.org/), [imageio](https://imageio.readthedocs.io/en/stable/installation.html), and [tifffile](https://pypi.org/project/tifffile/). These three packages can be installed using the following command:\n",
    "\n",
    "`pip install matplotlib imageio tifffile`\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!pip install matplotlib imageio tifffile"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Import the libraries.**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# comment out the third line (%matplotlib inline) if you run the tutorial in other IDEs other than Jupyter Notebook\n",
    "import matplotlib.pyplot as plt\n",
    "import imageio\n",
    "\n",
    "%matplotlib inline"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Display one single image.**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "raster = imageio.imread(os.path.join(data_dir, 'DEM.tif'))\n",
    "plt.imshow(raster)\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Read images as numpy arrays.**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "original = imageio.imread(os.path.join(data_dir, 'DEM.tif'))\n",
    "smoothed = imageio.imread(os.path.join(data_dir, 'smoothed.tif'))\n",
    "breached = imageio.imread(os.path.join(data_dir, 'breached.tif'))\n",
    "flow_accum = imageio.imread(os.path.join(data_dir, 'flow_accum.tif'))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Display multiple images in one plot.**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "fig = plt.figure(figsize=(16, 11))\n",
    "\n",
    "ax1 = fig.add_subplot(2, 2, 1)\n",
    "ax1.set_title('Original DEM')\n",
    "plt.imshow(original)\n",
    "\n",
    "ax2 = fig.add_subplot(2, 2, 2)\n",
    "ax2.set_title('Smoothed DEM')\n",
    "plt.imshow(smoothed)\n",
    "\n",
    "ax3 = fig.add_subplot(2, 2, 3)\n",
    "ax3.set_title('Breached DEM')\n",
    "plt.imshow(breached)\n",
    "\n",
    "ax4 = fig.add_subplot(2, 2, 4)\n",
    "ax4.set_title('Flow Accumulation')\n",
    "plt.imshow(flow_accum)\n",
    "\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## whitebox GUI\n",
    "\n",
    "WhiteboxTools also provides a Graphical User Interface (GUI) - **WhiteboxTools Runner**, which can be invoked using the following Python script. *__Note that the GUI might not work in Jupyter notebooks deployed on the cloud (e.g., MyBinder.org), but it should work on Jupyter notebooks on local computers.__*\n",
    "\n",
    "```python\n",
    "import whitebox\n",
    "whitebox.Runner()\n",
    "\n",
    "```\n",
    "\n",
    "![](https://wetlands.io/file/images/whitebox.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Citing whitebox\n",
    "\n",
    "If you use the **whitebox** Python package for your research and publications, please consider citing the following papers to give Prof. [John Lindsay](http://www.uoguelph.ca/~hydrogeo/index.html) credits for his tremendous efforts in developing [Whitebox GAT](https://github.com/jblindsay/whitebox-geospatial-analysis-tools) and [WhiteboxTools](https://github.com/jblindsay/whitebox-tools). Without his work, this **whitebox** Python package would not exist!  \n",
    "\n",
    "* Lindsay, J. B. (2016). Whitebox GAT: A case study in geomorphometric analysis. Computers & Geosciences, 95, 75-84. http://dx.doi.org/10.1016/j.cageo.2016.07.003"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Credits\n",
    "\n",
    "This interactive notebook is made possible by [MyBinder.org](https://mybinder.org/). Big thanks to [MyBinder.org](https://mybinder.org/) for developing the amazing binder platform, which is extremely valuable for reproducible research!\n",
    "\n",
    "This tutorial made use a number of open-source Python packages, including [ Cookiecutter](https://github.com/audreyr/cookiecutter), [numpy](http://www.numpy.org/), [matplotlib](https://matplotlib.org/), [imageio](https://imageio.readthedocs.io/en/stable/installation.html), [tifffile](https://pypi.org/project/tifffile/), and [google-drive-downloader](https://github.com/ndrplz/google-drive-downloader). Thanks to all developers of these wonderful Python packages!\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Contact\n",
    "\n",
    "If you have any questions regarding this tutorial or the **whitebox** Python package, you can contact me (Qiusheng Wu) at qwu18@utk.edu or https://wetlands.io/#contact"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.9"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}