{ "cells": [ { "attachments": {}, "cell_type": "markdown", "metadata": {}, "source": [ "[![image](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/opengeos/whitebox-python/blob/master/examples/whitebox.ipynb)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "# A tutorial for the whitebox Python package\n", "\n", "This notebook demonstrates the usage of the **whitebox** Python package for geospatial analysis, which is built on a stand-alone executable command-line program called [WhiteboxTools](https://github.com/jblindsay/whitebox-tools).\n", "\n", "* GitHub repo: https://github.com/opengeos/whitebox-python\n", "* WhiteboxTools: https://github.com/jblindsay/whitebox-tools\n", "* WhiteboxGeo: https://www.whiteboxgeo.com\n", "* User Manual: https://www.whiteboxgeo.com/manual/wbt_book/intro.html\n", "* PyPI: https://pypi.org/project/whitebox/\n", "* Documentation: https://whitebox.readthedocs.io\n", "* Binder: https://gishub.org/whitebox-cloud\n", "* Free software: [MIT license](https://opensource.org/licenses/MIT)\n", "\n", "**Launch this tutorial as an interactive Jupyter Notebook on the cloud - [MyBinder.org](https://gishub.org/whitebox-cloud).**\n", "\n", "![whitebox-gif](https://i.imgur.com/LF4UE1j.gif)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Table of Content\n", "\n", "* [Installation](#Installation)\n", "* [About whitebox](#About-whitebox)\n", "* [Getting data](#Getting-data)\n", "* [Using whitebox](#Using-whitebox)\n", "* [Displaying results](#Displaying-results)\n", "* [whitebox GUI](#whitebox-GUI)\n", "* [Citing whitebox](#Citing-whitebox)\n", "* [Credits](#Credits)\n", "* [Contact](#Contact)\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Installation\n", "\n", "\n", "**whitebox** supports a variety of platforms, including Microsoft Windows, macOS, and Linux operating systems. Note that you will need to have **Python 3.x** installed. Python 2.x is not supported. The **whitebox** Python package can be installed using the following command:\n", "\n", "`pip install whitebox`\n", "\n", "If you have installed **whitebox** Python package before and want to upgrade to the latest version, you can use the following command:\n", "\n", "`pip install whitebox -U`\n", "\n", "If you encounter any installation issues, please check [Troubleshooting](https://github.com/opengeos/whitebox#troubleshooting) on the **whitebox** GitHub page and [Report Bugs](https://github.com/opengeos/whitebox#reporting-bugs)." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "%pip install whitebox" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## About whitebox\n", "\n", "**import whitebox and call WhiteboxTools()**" ] }, { "cell_type": "code", "execution_count": 1, "metadata": {}, "outputs": [], "source": [ "import whitebox\n", "\n", "wbt = whitebox.WhiteboxTools()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "**Prints the whitebox-tools help...a listing of available commands**" ] }, { "cell_type": "code", "execution_count": 2, "metadata": { "scrolled": true }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "WhiteboxTools Help\n", "\n", "The following commands are recognized:\n", "--cd, --wd Changes the working directory; used in conjunction with --run flag.\n", "--compress_rasters Sets the compress_raster option in the settings.json file; determines if newly created rasters are compressed. e.g. --compress_rasters=true\n", "-h, --help Prints help information.\n", "-l, --license Prints the whitebox-tools license. Tool names may also be used, --license=\"Slope\"\n", "--listtools Lists all available tools. Keywords may also be used, --listtools slope.\n", "--max_procs Sets the maximum number of processors used. -1 = all available processors. e.g. --max_procs=2\n", "-r, --run Runs a tool; used in conjunction with --wd flag; -r=\"LidarInfo\".\n", "--toolbox Prints the toolbox associated with a tool; --toolbox=Slope.\n", "--toolhelp Prints the help associated with a tool; --toolhelp=\"LidarInfo\".\n", "--toolparameters Prints the parameters (in json form) for a specific tool; --toolparameters=\"LidarInfo\".\n", "-v Verbose mode. Without this flag, tool outputs will not be printed.\n", "--viewcode Opens the source code of a tool in a web browser; --viewcode=\"LidarInfo\".\n", "--version Prints the version information.\n", "\n", "Example Usage:\n", ">> ./whitebox_tools -r=lidar_info --cd=\"/path/to/data/\" -i=input.las --vlr --geokeys\n", "\n", "\n" ] } ], "source": [ "print(wbt.help())" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "**Prints the whitebox-tools license**" ] }, { "cell_type": "code", "execution_count": 3, "metadata": { "scrolled": false }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "WhiteboxTools License\n", "Copyright 2017-2023 John Lindsay\n", "\n", "Permission is hereby granted, free of charge, to any person obtaining a copy of this software and\n", "associated documentation files (the \"Software\"), to deal in the Software without restriction,\n", "including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense,\n", "and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so,\n", "subject to the following conditions:\n", "\n", "The above copyright notice and this permission notice shall be included in all copies or substantial\n", "portions of the Software.\n", "\n", "THE SOFTWARE IS PROVIDED \"AS IS\", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT\n", "NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND\n", "NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES\n", "OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN\n", "CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.\n", "\n" ] } ], "source": [ "print(wbt.license())" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "**Prints the whitebox-tools version**" ] }, { "cell_type": "code", "execution_count": 4, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Version information: WhiteboxTools v2.4.0 (c) Dr. John Lindsay 2017-2023\n", "\n", "WhiteboxTools is an advanced geospatial data analysis platform developed at\n", "the University of Guelph's Geomorphometry and Hydrogeomatics Research \n", "Group (GHRG). See www.whiteboxgeo.com for more details.\n", "\n" ] } ], "source": [ "print(\"Version information: {}\".format(wbt.version()))" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "**Print the help for a specific tool.**" ] }, { "cell_type": "code", "execution_count": 16, "metadata": { "scrolled": true }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "thread 'main' panicked at whitebox-tools-app/src/main.rs:72:21:\n", "Unrecognized tool name Conditionedlatinhypercube.\n", "note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace\n", "\n" ] } ], "source": [ "print(wbt.tool_help(\"ConditionedLatinHypercube\"))\n", "print(wbt.tool_help(\"lidar_info\"))" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "**Tool names in the whitebox Python package can be called either using the snake_case or CamelCase convention (e.g. lidar_info or LidarInfo). The example below uses snake_case.** " ] }, { "cell_type": "code", "execution_count": null, "metadata": { "scrolled": true }, "outputs": [], "source": [ "import os\n", "from importlib_resources import files\n", "\n", "# identify the sample data directory of the package\n", "data_dir = str(files(\"whitebox\").joinpath(\"testdata\"))\n", "\n", "# set whitebox working directory\n", "wbt.set_working_dir(data_dir)\n", "wbt.verbose = False\n", "\n", "# call whiteboxtools\n", "wbt.feature_preserving_smoothing(\"DEM.tif\", \"smoothed.tif\", filter=9)\n", "wbt.breach_depressions(\"smoothed.tif\", \"breached.tif\")\n", "wbt.d_inf_flow_accumulation(\"breached.tif\", \"flow_accum.tif\")" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "**You can search tools using keywords. For example, the script below searches and lists tools with 'lidar' or 'LAS' in tool name or description.**" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "lidar_tools = wbt.list_tools(['lidar', 'LAS'])\n", "for index, tool in enumerate(lidar_tools):\n", " print(\"{} {}: {} ...\".format(str(index + 1).zfill(3), tool, lidar_tools[tool][:45]))" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "**List all available tools in whitebox-tools**. Currently, **whitebox** contains 372 tools. More tools will be added as they become available." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "all_tools = wbt.list_tools()\n", "for index, tool in enumerate(all_tools):\n", " print(\"{} {}: {} ...\".format(str(index + 1).zfill(3), tool, all_tools[tool][:45]))" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Getting data" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "This section demonstrates two ways to get data into Binder so that you can test **whitebox** on the cloud using your own data. \n", "\n", "* [Getting data from direct URLs](#Getting-data-from-direct-URLs) \n", "* [Getting data from Google Drive](#Getting-data-from-Google-Drive)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Getting data from direct URLs\n", "\n", "If you have data hosted on your own HTTP server or GitHub, you should be able to get direct URLs. With a direct URL, users can automatically download the data when the URL is clicked. For example https://github.com/opengeos/whitebox-python/raw/master/examples/testdata.zip" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Import the following Python libraries and start getting data from direct URLs." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "import os\n", "import zipfile\n", "import tarfile\n", "import shutil\n", "import urllib.request" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Create a folder named *whitebox* under the user home folder and set it as the working directory." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "work_dir = os.path.join(os.path.expanduser(\"~\"), 'whitebox')\n", "if not os.path.exists(work_dir):\n", " os.mkdir(work_dir)\n", "os.chdir(work_dir)\n", "print(\"Working directory: {}\".format(work_dir))" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Replace the following URL with your own direct URL hosting your data." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "url = \"https://github.com/opengeos/whitebox-python/raw/master/examples/testdata.zip\"" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Download data the from the above URL and unzip the file if needed." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "# download the file\n", "zip_name = os.path.basename(url)\n", "zip_path = os.path.join(work_dir, zip_name)\n", "\n", "print('Downloading {} ...'.format(zip_name))\n", "urllib.request.urlretrieve(url, zip_path)\n", "print('Downloading done.'.format(zip_name))\n", "\n", "# if it is a zip file\n", "if '.zip' in zip_name:\n", " print(\"Decompressing {} ...\".format(zip_name))\n", " with zipfile.ZipFile(zip_name, \"r\") as zip_ref:\n", " zip_ref.extractall(work_dir)\n", " print('Decompressing done.')\n", "\n", "# if it is a tar file\n", "if '.tar' in zip_name:\n", " print(\"Decompressing {} ...\".format(zip_name))\n", " with tarfile.open(zip_name, \"r\") as tar_ref:\n", " tar_ref.extractall(work_dir)\n", " print('Decompressing done.')\n", "\n", "print('Data directory: {}'.format(os.path.splitext(zip_path)[0]))" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "You have successfully downloaded data to Binder. Therefore, you can skip to [Using whitebox](#Using-whitebox) and start testing whitebox with your own data. " ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Getting data from Google Drive\n", "\n", "Alternatively, you can upload data to [Google Drive](https://www.google.com/drive/) and then [share files publicly from Google Drive](https://support.google.com/drive/answer/2494822?co=GENIE.Platform%3DDesktop&hl=en). Once the file is shared publicly, you should be able to get a shareable URL. For example, https://drive.google.com/file/d/1xgxMLRh_jOLRNq-f3T_LXAaSuv9g_JnV.\n", " \n", "To download files from Google Drive to Binder, you can use the Python package called [google-drive-downloader](https://github.com/ndrplz/google-drive-downloader), which can be installed using the following command:\n", "\n", "`pip install googledrivedownloader requests`" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "**Replace the following URL with your own shareable URL from Google Drive.**" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "scrolled": true }, "outputs": [], "source": [ "gfile_url = 'https://drive.google.com/file/d/1xgxMLRh_jOLRNq-f3T_LXAaSuv9g_JnV'" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "**Extract the file id from the above URL.**" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "file_id = gfile_url.split('/')[5] #'1xgxMLRh_jOLRNq-f3T_LXAaSuv9g_JnV'\n", "print('Google Drive file id: {}'.format(file_id))" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "**Download the shared file from Google Drive.**" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "from google_drive_downloader import GoogleDriveDownloader as gdd\n", "\n", "dest_path = './testdata.zip' # choose a name for the downloaded file\n", "gdd.download_file_from_google_drive(file_id, dest_path, unzip=True)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "You have successfully downloaded data from Google Drive to Binder. You can now continue to [Using whitebox](#Using-whitebox) and start testing whitebox with your own data. " ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Using whitebox" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Here you can specify where your data are located. In this example, we will use [DEM.tif](https://github.com/opengeos/whitebox-python/blob/master/examples/testdata/DEM.tif), which has been downloaded to the testdata folder." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "**List data under the data folder.**" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "data_dir = './testdata/'\n", "print(os.listdir(data_dir))" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "In this simple example, we smooth [DEM.tif](https://github.com/opengeos/whitebox-python/blob/master/examples/testdata/DEM.tif) using a [feature preserving denoising](https://github.com/jblindsay/whitebox-tools/blob/master/src/tools/terrain_analysis/feature_preserving_denoise.rs) algorithm. Then, we fill depressions in the DEM using a [depression breaching](https://github.com/jblindsay/whitebox-tools/blob/master/src/tools/hydro_analysis/breach_depressions.rs) algorithm. Finally, we calculate [flow accumulation](https://github.com/jblindsay/whitebox-tools/blob/master/src/tools/hydro_analysis/dinf_flow_accum.rs) based on the depressionless DEM." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "import whitebox\n", "\n", "wbt = whitebox.WhiteboxTools()\n", "# set whitebox working directory\n", "wbt.set_working_dir(data_dir)\n", "wbt.verbose = False\n", "\n", "# call whiteboxtool\n", "wbt.feature_preserving_smoothing(\"DEM.tif\", \"smoothed.tif\", filter=9)\n", "wbt.breach_depressions(\"smoothed.tif\", \"breached.tif\")\n", "wbt.d_inf_flow_accumulation(\"breached.tif\", \"flow_accum.tif\")" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Displaying results\n", "\n", "This section demonstrates how to display images on Jupyter Notebook. Three Python packages are used here, including [matplotlib](https://matplotlib.org/), [imageio](https://imageio.readthedocs.io/en/stable/installation.html), and [tifffile](https://pypi.org/project/tifffile/). These three packages can be installed using the following command:\n", "\n", "`pip install matplotlib imageio tifffile`\n" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "!pip install matplotlib imageio tifffile" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "**Import the libraries.**" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "# comment out the third line (%matplotlib inline) if you run the tutorial in other IDEs other than Jupyter Notebook\n", "import matplotlib.pyplot as plt\n", "import imageio\n", "\n", "%matplotlib inline" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "**Display one single image.**" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "raster = imageio.imread(os.path.join(data_dir, 'DEM.tif'))\n", "plt.imshow(raster)\n", "plt.show()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "**Read images as numpy arrays.**" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "original = imageio.imread(os.path.join(data_dir, 'DEM.tif'))\n", "smoothed = imageio.imread(os.path.join(data_dir, 'smoothed.tif'))\n", "breached = imageio.imread(os.path.join(data_dir, 'breached.tif'))\n", "flow_accum = imageio.imread(os.path.join(data_dir, 'flow_accum.tif'))" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "**Display multiple images in one plot.**" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "fig = plt.figure(figsize=(16, 11))\n", "\n", "ax1 = fig.add_subplot(2, 2, 1)\n", "ax1.set_title('Original DEM')\n", "plt.imshow(original)\n", "\n", "ax2 = fig.add_subplot(2, 2, 2)\n", "ax2.set_title('Smoothed DEM')\n", "plt.imshow(smoothed)\n", "\n", "ax3 = fig.add_subplot(2, 2, 3)\n", "ax3.set_title('Breached DEM')\n", "plt.imshow(breached)\n", "\n", "ax4 = fig.add_subplot(2, 2, 4)\n", "ax4.set_title('Flow Accumulation')\n", "plt.imshow(flow_accum)\n", "\n", "plt.show()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## whitebox GUI\n", "\n", "WhiteboxTools also provides a Graphical User Interface (GUI) - **WhiteboxTools Runner**, which can be invoked using the following Python script. *__Note that the GUI might not work in Jupyter notebooks deployed on the cloud (e.g., MyBinder.org), but it should work on Jupyter notebooks on local computers.__*\n", "\n", "```python\n", "import whitebox\n", "whitebox.Runner()\n", "\n", "```\n", "\n", "![](https://wetlands.io/file/images/whitebox.png)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Citing whitebox\n", "\n", "If you use the **whitebox** Python package for your research and publications, please consider citing the following papers to give Prof. [John Lindsay](http://www.uoguelph.ca/~hydrogeo/index.html) credits for his tremendous efforts in developing [Whitebox GAT](https://github.com/jblindsay/whitebox-geospatial-analysis-tools) and [WhiteboxTools](https://github.com/jblindsay/whitebox-tools). Without his work, this **whitebox** Python package would not exist! \n", "\n", "* Lindsay, J. B. (2016). Whitebox GAT: A case study in geomorphometric analysis. Computers & Geosciences, 95, 75-84. http://dx.doi.org/10.1016/j.cageo.2016.07.003" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Credits\n", "\n", "This interactive notebook is made possible by [MyBinder.org](https://mybinder.org/). Big thanks to [MyBinder.org](https://mybinder.org/) for developing the amazing binder platform, which is extremely valuable for reproducible research!\n", "\n", "This tutorial made use a number of open-source Python packages, including [ Cookiecutter](https://github.com/audreyr/cookiecutter), [numpy](http://www.numpy.org/), [matplotlib](https://matplotlib.org/), [imageio](https://imageio.readthedocs.io/en/stable/installation.html), [tifffile](https://pypi.org/project/tifffile/), and [google-drive-downloader](https://github.com/ndrplz/google-drive-downloader). Thanks to all developers of these wonderful Python packages!\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Contact\n", "\n", "If you have any questions regarding this tutorial or the **whitebox** Python package, you can contact me (Qiusheng Wu) at qwu18@utk.edu or https://wetlands.io/#contact" ] } ], "metadata": { "kernelspec": { "display_name": "Python 3 (ipykernel)", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.11.9" } }, "nbformat": 4, "nbformat_minor": 2 }