{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "9df9ed03-ec56-40ea-bae6-571d942af22e",
   "metadata": {},
   "source": [
    "## Working with filter libraries"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "213bddad-e06a-41b2-80d4-ae783e1adc5d",
   "metadata": {},
   "outputs": [],
   "source": [
    "import adaptivefiltering as af"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4c2122e1-bd37-46cd-878d-4e741b6cb474",
   "metadata": {},
   "source": [
    "`adaptivefiltering` stores filter pipelines in `.json` files for later reuse. JSON stands for *JavaScript Object Notation* and is a widely used format for storing custom data structures. We will see in [Creating filter pipelines](filtering.ipynb) how these files are created from scratch. Now, we will learn how to use these files. This will enable you to leverage a library of community-contributed filter pipelines and allow you to organize your filter pipelines locally."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "15164cc4-7900-4e3b-b8dc-8527a8e52cb2",
   "metadata": {},
   "source": [
    "### Adding filter libraries"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "dcd0bff4-47f9-4fd3-8169-892780e76234",
   "metadata": {},
   "source": [
    "Filter libraries are directories that contain a number of `.json` files that contain filter pipeline definitions. `adaptivefiltering` internally keeps a per-session list of known filter libraries which by default contains the current working directory and the path to the [library of community-contributed filters](https://github.com/ssciwr/adaptivefiltering-library/), which has been installed as a separate Python package. We can manually register directories as filter libraries like this:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "060cd2ca-ecfd-4d8a-b231-d49a12a24927",
   "metadata": {},
   "outputs": [],
   "source": [
    "af.add_filter_library(path=\"/home/user/somepath\", recursive=False)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6f273999-c99a-485d-85a8-05cec856c5ed",
   "metadata": {},
   "source": [
    "The `recursive` parameter specifies whether filter pipelines in subdirectories of the given directory should also be taken into account (defaults to `False`)."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "04addc93-fabd-415c-8453-b28fec1da551",
   "metadata": {},
   "source": [
    "### Browsing filters in filter libraries"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1a4cff2f-775a-4c74-bb93-3cdac415e52e",
   "metadata": {},
   "source": [
    "We can search our filter libraries for filter pipelines matching certain criteria by using the `select_pipeline_from_library` and `select_pipelines_from_library` functions. The former will allow you to select exactly one pipeline, where as the latter allows you to select multiple pipelines by holding `CTRL` pressed while clicking on the filters. Clicking the *Finalize* button will make the user interface vanish, but you can also use the returned `pipeline` object before that:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "36a6ca95-c71f-44b6-9169-98224b3be314",
   "metadata": {},
   "outputs": [],
   "source": [
    "pipeline = af.select_pipeline_from_library()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ce9edefe-a4ca-498f-8df2-838474ee0398",
   "metadata": {},
   "source": [
    "In the left most column, we see filtering criterions we can use to access the filter pipelines in our filtering libraries. The middle column contains a list of filter pipelines that match the given criteria. Clicking one of these will select it and show its metadata in the third column. The returned `pipeline` object can then be passed to other functions `adaptivefiltering`, e.g. to [select the best filter pipeline for a given dataset](selection.ipynb)."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "cc720a9a-4134-4c67-8e41-ca984425926f",
   "metadata": {},
   "source": [
    "### Proprietary filtering backends"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3b863308-f9d3-4c3b-90c4-e7af62c1d0dd",
   "metadata": {},
   "source": [
    "`adaptivefiltering` does not implement its own ground point filtering algorithms. Instead, algorithms from existing packages are accessible through a common interface. Currently, the following backends are available:\n",
    "\n",
    "* [PDAL](https://pdal.io/): The Point Data Abstraction Library is an open source library for point cloud processing.\n",
    "* [OPALS](https://opals.geo.tuwien.ac.at/html/stable/index.html) is a proprietary library for processing Lidar data. It can be tested freely for datasets <1M points.\n",
    "* [LASTools](https://rapidlasso.com/) has a proprietary tool called `lasground_new` that can be used for ground point filtering.\n",
    "\n",
    "PDAL is always available when using `adaptivefiltering` and is used internally for many tasks that are not directly related to ground point filtering. In order to enable the OPALS backend, `adaptivefiltering` needs to be given the information where your OPALS installation (potentially including your license key) is located. This can either be done by setting the environment variable `OPALS_DIR` or by setting the path at runtime:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "5a897001-2022-444f-a10d-3f55b857551a",
   "metadata": {},
   "outputs": [],
   "source": [
    "af.set_opals_directory(\"/path/to/opals\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1b0c91e7-52eb-4efd-8c35-64233ef6fb15",
   "metadata": {},
   "source": [
    "Similarly, you can set the path to your installation of LASTools either through the environment variable `LASTOOLS_DIR` or at runtime:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "068a2cee-c6a5-4143-bcab-f422fa699540",
   "metadata": {},
   "outputs": [],
   "source": [
    "af.set_lastools_directory(\"/path/to/lastools\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e90e7ff2-06c0-4af1-9573-4d10e1a678ee",
   "metadata": {},
   "source": [
    "Please note that LASTools only ships Windows binaries. Therefore, you will need [Wine](https://www.winehq.org/) installed on your system to successfully use the LASTools backend on Linux."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f0fc62a7-d5fd-4e84-a64c-bc0d5b9b0924",
   "metadata": {},
   "source": [
    "### Creating new filter libraries"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b8844523-8b0c-4983-b4a5-3784857f6425",
   "metadata": {},
   "source": [
    "As filter libraries are just directories on the file system, you are free to organizing them yourself by moving and copying files and then making the respective directories known to `adaptivefiltering` using `add_filter_library` as seen above. However, there is also additional functionality available that eases the management process. If you are editing filter pipelines a lot, you might want to set the current library path like this:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "80661544-002c-47b3-9f11-a2178c6de6b7",
   "metadata": {},
   "outputs": [],
   "source": [
    "af.set_current_filter_library(\"mylibrary\", create_dirs=True, name=\"My testing library\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "30208243-e665-4b2e-878e-f46935c4ca44",
   "metadata": {},
   "source": [
    "The library defined by `set_current_filter_library` will be used as the default path to store filter pipelines using [save_filter](filtering.ipynb) or [select_best_pipeline](selection.ipynb). Here, the `name` parameter is used to define a display name for the filter library. It is stored in JSON file called `library.json` in the directory. Such file could also be added manually to filter library directories. If you do not want to generate `library.json` or do not want to override an existing one, you may set `name=None`."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9696b679-6d54-43f8-8c18-41ad9c98fb7f",
   "metadata": {},
   "source": [
    "### Sharing filter pipelines with others"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "017c88f6-910f-4760-bd7f-8fbeeddb555c",
   "metadata": {},
   "source": [
    "As each filter pipeline is stored as a separate file, filter pipelines can be shared easily by sharing these files via your favorite method. If you want to share your filter pipeline with a wider community, you should consider contributing it to `adaptivefiltering`'s community contribution library. This will make your filter pipeline accessible to all useres of `adaptivefiltering`. You can find the details of the process [on GitHub](https://github.com/ssciwr/adaptivefiltering-library/issues/3)."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "20ad768b-d035-472a-b2d5-804e6384816c",
   "metadata": {},
   "source": [
    "### Resetting filter libraries"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "674258b5-3c01-4ceb-a8f0-6ad710b8d5b4",
   "metadata": {},
   "source": [
    "If, for some reason, you want to reset the currently registered filter libraries to the default ones, you can do so like this:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "2cd70830-e8e5-4ddc-87bf-cf9581553f84",
   "metadata": {},
   "outputs": [],
   "source": [
    "af.reset_filter_libraries()"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.10"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}