{
 "cells": [
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Example: Working with models from CLI"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "HydroMT has the following high-level functionality from the Command Line Interface (CLI) for setting up models from raw data or adjusting models:\n",
    "\n",
    "- **building** a model: building a model from scratch.\n",
    "- **updating** a model: adding or changing model components of an existing model.\n",
    "\n",
    "Here we show how to build and update a hypothetical gridded model that contains a ``grid`` and a ``config`` from the command line interface (CLI) based on the generic HydroMT **example_model**."
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Lets first check which models are available in our environment:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!hydromt --models"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Build a model from CLI"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To build a model you always follow the next four steps.\n",
    "\n",
    "1. Prepare or use a pre-defined data catalog with all the required data sources\n",
    "2. Define your model region, see the overview of [model region options](https://deltares.github.io/hydromt/latest/user_guide/models/model_region.html).\n",
    "3. Prepare a [model workflow file](https://deltares.github.io/hydromt/latest/user_guide/models/model_workflow.html) which describes the complete pipeline to build your model.\n",
    "4. Build you model using the CLI or Python interface\n",
    "\n",
    "Here we focus on steps 2-4 and use data from the [predefined artifact_data catalog](https://deltares.github.io/hydromt/latest/user_guide/data_catalog/data_existing_cat.html#available-pre-defined-data-catalogs).\n",
    "\n",
    "Using the **hydromt build** method we can setup a complete model from scratch. Let’s get an overview of the method and its arguments.\n",
    "\n",
    "> Note the required MODEL (i.e. name of the model) and MODEL_ROOT (i.e. folder where to save the model) arguments."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!hydromt build --help"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As a first step, we want to create a grid for our example model. We will do this by calling the ``example_model.grid.create_from_region`` method. We will based our grid from a bounding box with a resolution of 0.05 degree in EPSG 4326.\n",
    "\n",
    "For this we will create the following workflow file below, where we define the region for our grid.\n",
    "\n",
    "> The %%writefile magic saves the content below to a file. The content of the .yaml file start from the second line."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%%writefile tmp_build_example_model_min.yml\n",
    "\n",
    "steps:\n",
    "  - grid.create_from_region:\n",
    "      region:\n",
    "        bbox: [11.70, 45.35, 12.95, 46.70]\n",
    "      res: 0.05\n",
    "      crs: 4326"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now let's call the HydroMT ``build`` command with our newly created **build_example_model.yml** workflow file and using the **artifact_data** catalog:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!hydromt build example_model ./tmp_example_model_min -i tmp_build_example_model_min.yml -d artifact_data --fo -v"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The example above means the following: run **hydromt build** with:\n",
    "\n",
    "- ``example_model``: i.e. build a generic GridModel instance\n",
    "- ``./tmp_example_model_min``: output model folder\n",
    "- ``-i tmp_build example_model_min.yml``: path to the model build workflow file\n",
    "- ``-d artifact_data``: data catalog to use\n",
    "- ``-vv`` : give some extra verbosity (1 * v) to display feedback on screen. Now INFO messages are provided.\n",
    "\n",
    "Let's see which files where created for our model:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# print MODEL_ROOT folder\n",
    "import os\n",
    "\n",
    "\n",
    "def print_dir(root):\n",
    "    for path, _, files in os.walk(root):\n",
    "        print(path)\n",
    "        for name in files:\n",
    "            if name.endswith(\".xml\"):\n",
    "                continue\n",
    "            print(f\" - {name}\")\n",
    "\n",
    "\n",
    "print_dir(\"tmp_example_model_min\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can see that in this case, not many files were created, only the grid and the hydromt log file.\n",
    "\n",
    "So now let's add more steps to our workflow file to also add a configuration file with some settings, re-create our grid and populate it with some data (e.g. elevation and landuse).\n",
    "We will use the following workflow file:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%%writefile tmp_build_example_model.yml\n",
    "\n",
    "steps:\n",
    "  - config.update:\n",
    "      data:\n",
    "        header.settings: value\n",
    "        timers.end: \"2010-02-15\"\n",
    "        timers.start: \"2010-02-05\"\n",
    "  - grid.create_from_region:\n",
    "      region:\n",
    "        bbox: [11.70, 45.35, 12.95, 46.70]\n",
    "      res: 0.05\n",
    "      crs: 4326\n",
    "  - grid.add_data_from_rasterdataset:\n",
    "      raster_data: merit_hydro_ihu\n",
    "      variables:\n",
    "        - elevtn\n",
    "        - basins\n",
    "      reproject_method:\n",
    "        - average\n",
    "        - mode\n",
    "  - grid.add_data_from_rasterdataset:\n",
    "      raster_data: vito_2015\n",
    "      fill_method: nearest\n",
    "      reproject_method: mode\n",
    "      rename:\n",
    "        vito: landuse\n",
    "  - write:\n",
    "      components:\n",
    "        - grid\n",
    "        - config"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In this configuration, you see that we will prepare quite a lot of data for our grid model using some of the generic model methods for grid. We will prepare:\n",
    "\n",
    "- Update some config options using [config.update](https://deltares.github.io/hydromt/latest/_generated/hydromt.model.components.ConfigComponent.update.html#hydromt.model.components.ConfigComponent.update)\n",
    "- Create a regular grid from a bounding box region using [grid.create_from_region](https://github.com/Deltares/hydromt/blob/fbb17de5046e657a6257959c5ee48634eab5adab/hydromt/model/example/example_grid_component.py#L58)\n",
    "- A couple of grid based on reprojection of MERIT Hydro IHU (elevation, basins) and VITO (landuse) using [grid.add_data_from_rasterdataset](https://github.com/Deltares/hydromt/blob/fbb17de5046e657a6257959c5ee48634eab5adab/hydromt/model/example/example_grid_component.py#L156). Note that you can use the same several times in the same workflow file.\n",
    "\n",
    "Let's now build this model:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!hydromt build example_model ./tmp_example_model -i tmp_build_example_model.yml -d artifact_data --fo -v"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The example above means the following: run **hydromt build** with:\n",
    "\n",
    "- ``example_model``: i.e. build a generic GridModel instance\n",
    "- ``./tmp_example_model``: output model folder\n",
    "- ``-i tmp_build_example_model.yaml``: use this .yaml file to configure the model build\n",
    "- ``-d artifact_data``: parse the pre-defined artifact_data\n",
    "- ``-vv``: give some extra verbosity (1 * v) to display feedback on screen. Now INFO messages are provided.\n",
    "\n",
    "Let’s check some of the outputs that were produced:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Files created\n",
    "print_dir(\"tmp_example_model\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# checkout the content of the hypothetical model simulation configuration\n",
    "fn_yaml = \"tmp_example_model/settings.toml\"\n",
    "with open(fn_yaml, \"r\") as f:\n",
    "    txt = f.read()\n",
    "print(txt)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# List of variables in grid.nc\n",
    "from hydromt.model import ExampleModel\n",
    "\n",
    "model = ExampleModel(root=\"./tmp_example_model\", mode=\"r\")\n",
    "ds = model.grid.data\n",
    "\n",
    "print(f\"Variables available in grid.nc: {list(ds.data_vars)}\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Plot one of the variable (change the name below to plot a different variable)\n",
    "var = \"elevtn\"\n",
    "ds[var].plot()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Update a model from CLI"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Using the **hydromt update** method we can update an existing model with new components or modify existing components. Let’s get an overview of the method and its arguments.\n",
    "\n",
    "Note that the ``MODEL`` (i.e. name of the model), and ``MODEL_ROOT`` (i.e. folder of existing model) are still required. There is an optional ``-o`` ``--model-out`` option to save the updated model in a different directory."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!hydromt update --help"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In this basic example we use the **hydromt update** method to update the ExampleModel instance with an upstream area raster map. Then we write only the updated model map component to file."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%%writefile tmp_update_example_model.yml\n",
    "\n",
    "steps:\n",
    "  - grid.add_data_from_rasterdataset:\n",
    "      raster_data: merit_hydro_ihu\n",
    "      variables:\n",
    "        - uparea\n",
    "      reproject_method:\n",
    "        - max\n",
    "  - grid.write:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!hydromt update example_model ./tmp_example_model -o ./tmp_example_model_update -i tmp_update_example_model.yml -d artifact_data --fo -v"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The example above means the following: run **hydromt update** with:\n",
    "\n",
    "- ``example_model``: i.e. update a ExampleModel instance\n",
    "- ``./tmp_example_model``: the folder of the to-be updated model\n",
    "- ``-o ./tmp_example_model_update``: the folder of the updated model\n",
    "- ``-i tmp_update_example_model.yml``: the hydromt workflow file listing the methods to be executed\n",
    "- ``-d artifact_data``: use the pre-defined artifact_data catalog\n",
    "- ``-v``: give some extra verbosity (2 * v) to display feedback on screen. Now debug messages are provided.\n",
    "\n",
    "Let's check the content of the updated model and that we should now have an uparea variable available in the updated model grid:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# note the difference with the original model\n",
    "print_dir(\"tmp_example_model_update\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# List of variables in grid.nc\n",
    "from hydromt.model import ExampleModel\n",
    "\n",
    "model = ExampleModel(root=\"./tmp_example_model_update\", mode=\"r\")\n",
    "ds = model.grid.data\n",
    "\n",
    "print(f\"Variables available in grid.nc: {list(ds.data_vars)}\")"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.11.14"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}