{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "7d244468-ea2b-43e2-b4f6-f286c176f72f",
   "metadata": {},
   "source": [
    "# Frequently Ask Questions and best practices\n",
    "This section gives an overview over the most commonly askes questions and the best practice when it comes to plugin development. \n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "fec1f8ea",
   "metadata": {},
   "source": [
    "### How can I make use of `Freva` from jupyter notebooks?\n",
    "The easiest way to use freva in a jupyter notebook is to log on the HPC system where freva is installed and load the freva module. The name of the specific freva module should be\n",
    "communicated by your admin team. If the name of the freva module is `freva` then simply load it with:\n",
    "\n",
    "```console\n",
    "module load freva\n",
    "```\n",
    "\n",
    "After that you can install a new jupyter kernel via:\n",
    "\n",
    "```console\n",
    "jupyter-kernel-install python --name freva --display-name FrevaKernel\n",
    "```\n",
    "\n",
    "After you have installed the kernel you can use freva in your jupyter notebook:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "b0a1cf8a",
   "metadata": {},
   "outputs": [],
   "source": [
    "import freva\n",
    "# Get an overview of the availalbe freva plugins\n",
    "freva.get_tools_list()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "b7a20427",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Get the documentation of a plugin\n",
    "freva.plugin_doc(\"animator\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2a8a6689",
   "metadata": {},
   "source": [
    "### How can I use freva in my own python environment?\n",
    "If you don't want to create a new jupyter kernel that points to the python environment where the freva system is installed you can also install freva into your own python environment. You can either install freva via `pip`: \n",
    "\n",
    "```console\n",
    "python3 -m pip install freva\n",
    "```\n",
    "or `conda`:\n",
    "\n",
    "```console\n",
    "conda install -c conda-forge freva\n",
    "```\n",
    "\n",
    "Afterwards you can use freva in your own environment. \n",
    "\n",
    "The only difference to the above approach of using the freva python environment is that you will have to tell freva to use the correct configuration. \n",
    "\n",
    "```python\n",
    "import freva\n",
    "# In order to use freva we have to tell where to get the configuration\n",
    "freva.config(\"/path/to/the/freva/config/file.conf\")\n",
    "# Talk to your admins to get the location of the config file.\n",
    "```\n",
    "\n",
    "Please talk to your admins on where the central freva configuration is located."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b68392b5-5375-40b8-84d5-ac3daeaeb9a3",
   "metadata": {
    "tags": []
   },
   "source": [
    "### How can I use `Freva` in my analysis workflows?\n",
    "You can use `Freva` without creating or applying data analysis plugins. One example would be using the `databrowser` command in you data analysis workflow:\n",
    "\n",
    "```console\n",
    "\n",
    "# Use the databrowser search and pipe the output to ncremap\n",
    "freva databrowser project=observations experiment=cmorph time_frequency=30min | ncremap -m map.nc -O drc_rgr\n",
    "\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "514cd933-a94e-479b-8377-cfcd55a1b9ad",
   "metadata": {},
   "source": [
    "Below you can find a python example, which you could use in a notebook"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "dfad0452-92fd-48aa-a74e-feabd79c02ec",
   "metadata": {},
   "outputs": [],
   "source": [
    "import freva\n",
    "import xarray as xr\n",
    "\n",
    "# Open the data with xarray\n",
    "dset = xr.open_mfdataset(freva.databrowser(project=\"obs*\", time_frequency='30min'), combine='by_coords')['pr']\n",
    "dset"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "93272058-47a9-4fcf-9707-7610b8e1bc50",
   "metadata": {},
   "source": [
    "### Best practice: Using the `Freva` module in a plugin\n",
    "Like above you can use the Freva python module within your wrapper API code of a plugin"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "26c898ed-a77a-442b-a93e-56fb69c88158",
   "metadata": {},
   "outputs": [],
   "source": [
    "import json\n",
    "\n",
    "from evaluation_system.api import plugin, parameters\n",
    "import freva\n",
    "\n",
    "class MyPlugin(plugin.PluginAbstract):\n",
    "    \"\"\"An analysis plugin that uses the databrowser search.\"\"\"\n",
    "    \n",
    "    __parameters__ = parameters.ParameterDictionary(\n",
    "        parameters.SolrField(name=\"project\", facet=\"project\", help=\"Set the project name\"),\n",
    "        parameters.SolrField(name=\"variable\", facet=\"variable\", help=\"Set the variable name\"),\n",
    "    )\n",
    "    \n",
    "    def run_tool(self, config_dict):\n",
    "        \"\"\"Main plugin method that makes Freva calls.\"\"\"\n",
    "        # Search for files\n",
    "        files = list(freva.databrowser(**config_dict))\n",
    "        # Save the files to a json file\n",
    "        with open(\"/tmp/some_json_file.json\", \"w\") as f:\n",
    "            json.dumps(files, f)\n",
    "        self.call(\"external_command /tmp/some_json_file.json\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "737ba4df-422e-4e88-afff-61eb15eb2108",
   "metadata": {},
   "source": [
    "### Best practice: Making calls to external commands with complex command line parameters\n",
    "\n",
    "If you have a plugin that makes calls to a command line interface (like a shell script) you should avoid long command line\n",
    "argument calls like this:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "6ab5a3ab-701a-4a17-beaa-1d9c073dead4",
   "metadata": {},
   "outputs": [],
   "source": [
    "import json\n",
    "\n",
    "from evaluation_system.api import plugin, parameters\n",
    "import freva\n",
    "class MyPlugin(plugin.PluginAbstract):\n",
    "    \n",
    "    def run_tool(self, config_dict):\n",
    "        result = self.call('%s/main.py %s %s %s %s %s %s %s %s %s %s %s %s %s %s %s %s %s %s %s %s %s' \n",
    "                   % (self.class_basedir, config_dict['inputdir'], config_dict['project'], \n",
    "                      config_dict['product'], config_dict['institute'], config_dict['model'],\n",
    "                      config_dict['experiment'], config_dict['time_frequency'], \n",
    "                      config_dict['variable'], config_dict['ensemble'], config_dict['cmor_table'],\n",
    "                      config_dict['outputdir'], config_dict['cache'], config_dict['cacheclear'], \n",
    "                      config_dict['seldate'], config_dict['latlon'], config_dict['area_weight'],\n",
    "                      config_dict['percentile_threshold'], config_dict['threshold'],\n",
    "                      config_dict['persistence'], config_dict['sel_extr'], config_dict['dryrun'])\n",
    "        )\n",
    "        \n",
    "        "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "313a5447-ca08-4cbf-9112-cf2e14e21284",
   "metadata": {},
   "source": [
    "Such call are very hard to read and should be avoided. Instead you can use Freva as much as possible within the wrapper code and save relevant results into a json file that is passed and a single argument to the tool. Using the above example the code could be simplified as follows:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "0ce2c05f-bf41-4cde-8f10-43f4093fa962",
   "metadata": {},
   "outputs": [],
   "source": [
    "import json\n",
    "from tempfile import NamedTemporaryFile\n",
    "from evaluation_system.api import plugin, parameters\n",
    "import freva\n",
    "\n",
    "class MyPlugin(plugin.PluginAbstract):\n",
    "    \n",
    "    def run_tool(self, config_dict):\n",
    "        config_dict['input_files'] = list(\n",
    "            freva.databrowser(\n",
    "                product=config_dict.pop('product'), project=config_dict.pop('project'),\n",
    "                institute=config_dict.pop('institute'), model=config_dict.pop('model'),\n",
    "                experiment=config_dict.pop('experiment'), variable=config_dict.pop('variable'),\n",
    "                ensemble=config_dict.pop('ensemble'), cmor_table=config_dict.pop('cmor_table')\n",
    "            )\n",
    "        )\n",
    "        with NamedTemporaryFile(suffix='.json') as tf:\n",
    "            with open(tf.name, 'w') as f:\n",
    "                json.dump(config_dict, f)\n",
    "            self.call(f\"{self.class_basedir}/main.py {tf.name}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6c79d551-ade4-4a2e-8f73-39d1b5c3069a",
   "metadata": {},
   "source": [
    "Here we use json because most scripting and programming languages have a `json` parser functionality which can be conveniently used."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "freva",
   "language": "python",
   "name": "freva"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.11.4"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}