{
 "cells": [
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Example: Working with models from CLI"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "HydroMT has the following high-level functionality from the Command Line Interface (CLI) for setting up models from raw data or adjusting models:\n",
    "\n",
    "- **building** a model: building a model from scratch.\n",
    "- **updating** a model: adding or changing model components of an existing model.\n",
    "- **clipping** a model: changing the spatial domain of an existing model (e.g. select subbasins from a larger model).\n",
    "\n",
    "Here we show how to build and update a hypothetical distributed model from the command line interface (CLI) based on the generic HydroMT **grid_model**."
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Lets first check which models are available in our environment:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!hydromt --models"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Build a model from CLI"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To build a model you always follow the next four steps. \n",
    "\n",
    "1. Prepare or use a pre-defined data catalog with all the required data sources\n",
    "2. Define your model region, see the overview of [model region options](https://deltares.github.io/hydromt/latest/user_guide/model_region.html).\n",
    "3. Prepare a model configuration which describes the complete pipeline to build your model, see [preparing a model configuration](https://deltares.github.io/hydromt/latest/user_guide/model_config.html). \n",
    "4. Build you model using the CLI or Python interface\n",
    "\n",
    "Here we focus steps 2-4 and use data from the [predefined **artifact_data** data catalog](https://deltares.github.io/hydromt/latest/user_guide/data_existing_cat.html)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Using the **hydromt build** method we can setup a complete model from scratch. Let's get an overview of the method and its arguments. \n",
    "\n",
    "Note the required `MODEL` (i.e. name of the model), `MODEL_ROOT` (i.e. folder where to save the model) arguments.\n",
    "As of version v0.7.0 the `REGION` (i.e. area of interest) argument is optional. This argument can be used by adding -r or --region flag."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!hydromt build --help"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!hydromt build grid_model ./tmp_grid_model --region \"{'bbox': [11.70, 45.35, 12.95, 46.70]}\" --opt \"setup_grid.res\"=0.05 -vv"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The example above means the following: run **hydromt build** with:\n",
    "\n",
    "- `grid_model`: i.e. build a generic GridModel instance\n",
    "- `./tmp_grid_model`: output model folder\n",
    "- `--region \"{'bbox': [11.70, 45.35, 12.95, 46.70]}\"`: set the region of interest using a bounding box defined by its [xmin, ymin, xmax, ymax] coordinates (in WGS84)\n",
    "- `--opt \"setup_grid.res\" = 0.05`: choose the resolution of the grid to generate within the bounding box. Unit (degree or meter) depends on the chosen coordinate system.\n",
    "- `-vv` : give some extra verbosity (2 * v) to display feedback on screen. Now debug messages are provided."
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As we did not specify a hydromt configuration, besides the hydromt.log file, an empty model simulation configuration file, only the model region (area of interest) and the grid definition have been defined and are respectively saved in the geoms folder and the grid.nc file. To build a complete model we need the use a hydromt configuration .yaml file. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# print MODEL_ROOT folder\n",
    "import os\n",
    "\n",
    "\n",
    "def print_dir(root):\n",
    "    for path, _, files in os.walk(root):\n",
    "        print(path)\n",
    "        for name in files:\n",
    "            if name.endswith(\".xml\"):\n",
    "                continue\n",
    "            print(f\" - {name}\")\n",
    "\n",
    "\n",
    "print_dir(\"tmp_grid_model\")"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The **configuration file** exists of available  **setup_methods** for your model, listed in the order of execution. Most methods start with reading input data using the DataAdapter, transforming the data using workflows (e.g. reprojection, reclassification, aggregation, etc...) and adding the new model data to the right model component. An overview of the available methods can be found in the online API reference for the [GridModel](https://deltares.github.io/hydromt/latest/api.html#gridmodel), [VectorModel](https://deltares.github.io/hydromt/latest/api.html#vectormodel), and [MeshModel](https://deltares.github.io/hydromt/latest/api.html#meshmodel)\n",
    "\n",
    "Note that these methods for the generic model classes are still quite limited. To get an idea of potential setup_ methods, checkout the [model plugins](https://deltares.github.io/hydromt/latest/plugins.html)\n",
    "\n",
    "The configuration yaml file can be passed to the hydromt build method using the `-i` flag."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# check the content of the .yaml file (for sake of the example only)\n",
    "fn_yaml = \"grid_model_build.yaml\"\n",
    "with open(fn_yaml, \"r\") as f:\n",
    "    txt = f.read()\n",
    "print(txt)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In this configuration, you see that we will prepare quite a lot of data for our grid model using some of the generic model methods for grid. We will prepare:\n",
    "\n",
    "- A grid with constant values 0.01 using [setup_grid_from_constant](https://deltares.github.io/hydromt/latest/_generated/hydromt.GridModel.setup_grid_from_constant.html)\n",
    "- A couple of grid based on reprojection of MERIT Hydro IHU (elevation, basins) and VITO (landuse) using [setup_grid_from_rasterdataset](https://deltares.github.io/hydromt/latest/_generated/hydromt.GridModel.setup_grid_from_rasterdataset.html). Note that to use the same method twice within the same configuration file, you can end the method name with a number.\n",
    "- A grid generated by mapping roughness values to the land use classes in VITO using [setup_grid_from_raster_reclass](https://deltares.github.io/hydromt/latest/_generated/hydromt.GridModel.setup_grid_from_raster_reclass.html)\n",
    "- A couple of lake properties including the fraction of the grid cells covered by the lake geometry using [setup_grid_from_geodataframe](https://deltares.github.io/hydromt/latest/_generated/hydromt.GridModel.setup_grid_from_geodataframe.html)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!hydromt build grid_model ./tmp_grid_model1 -r \"{'bbox': [11.70, 45.35, 12.95, 46.70]}\" -i grid_model_build.yaml -d artifact_data -d data/vito_reclass.yml -vv"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The example above means the following: run **hydromt build** with:\n",
    "\n",
    "- `grid_model`: i.e. build a generic GridModel instance\n",
    "- `./tmp_grid_model1`: output model folder\n",
    "- `-r \"{'bbox': [11.70, 45.35, 12.95, 46.70]}\"`: set the region of interest using a bounding box defined by its [xmin, ymin, xmax, ymax] coordinates (in WGS84)\n",
    "- `-i grid_model_build.yaml`: use this .yaml file to configure the model build\n",
    "- `-d artifact_data -d data/vito_reclass.yml`: parse the pre-defined artifact_data and the local vito_reclass data catalogs\n",
    "- `-vv` : give some extra verbosity (2 * v) to display feedback on screen. Now debug messages are provided.\n",
    "\n"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's check some of the outputs that were produced:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Files created\n",
    "print_dir(\"tmp_grid_model1\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# checkout the content of the hypothetical model simulation configuration\n",
    "fn_yaml = \"tmp_grid_model1/model.ini\"\n",
    "with open(fn_yaml, \"r\") as f:\n",
    "    txt = f.read()\n",
    "print(txt)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# List of variables in grid.nc\n",
    "import xarray as xr\n",
    "ds = xr.open_dataset(\"tmp_grid_model1/grid/grid.nc\")\n",
    "\n",
    "print(f\"Variables available in grid.nc: {list(ds.data_vars)}\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Plot one of the variable (change the name below to plot a different variable)\n",
    "var = \"elevtn\"\n",
    "ds[var].plot()"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Update a model from CLI"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Using the **hydromt update** method we can update an existing model with new components or modify existing components. Let's get an overview of the method and its arguments. \n",
    "\n",
    "Note that the `MODEL` (i.e. name of the model), and `MODEL_ROOT` (i.e. folder of existing model) are still required. There is an optional `-o --model-out` option to save the updated model in a different directory."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!hydromt update --help"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In this basic example we use the **hydromt update** method to update the GridModel instance with an upstream area raster map. Then we write only the updated model map component to file. \n",
    "\n",
    "The `%%writefile` magic saves the content below to a file. The content of the .yaml file start from the second line."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%%writefile ./tmp_grid_model1/grid_model_update.yaml\n",
    "setup_grid_from_rasterdataset:\n",
    "    raster_fn: merit_hydro_1k\n",
    "    variables: uparea\n",
    "    reproject_method: max\n",
    "\n",
    "write_grid:\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!hydromt update grid_model ./tmp_grid_model1 -o ./tmp_grid_model1_update -i ./tmp_grid_model1/grid_model_update.yaml -d artifact_data -vv"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The example above means the following: run **hydromt update** with:\n",
    "\n",
    "- `grid_model`: i.e. update a generic GridModel instance\n",
    "- `./tmp_grid_model1`: the folder of the to-be updated model\n",
    "- `./tmp_grid_model1_update`: the folder of the updated model\n",
    "- `-i ./tmp_grid_model1/grid_model_update.yaml`: the hydromt configuration listing the methods to be executed\n",
    "- `-vv` : give some extra verbosity (2 * v) to display feedback on screen. Now debug messages are provided."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# note the difference with the original model\n",
    "print_dir(\"tmp_grid_model1_update\")"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "hydromt-dev1",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.10"
  },
  "orig_nbformat": 4,
  "vscode": {
   "interpreter": {
    "hash": "4107911daee865ab8b23da348e7348b24a775bdb6009ea2571458b9eceb9e334"
   }
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}