{
 "cells": [
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#hide\n",
    "# default_exp script"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Script - CLI\n",
    "\n",
    "> A fast way to turn your python function into a script."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Part of [fast.ai](https://www.fast.ai)'s toolkit for delightful developer experiences."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Overview"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Sometimes, you want to create a quick script, either for yourself, or for others. But in Python, that involves a whole lot of boilerplate and ceremony, especially if you want to support command line arguments, provide help, and other niceties. You can use [argparse](https://docs.python.org/3/library/argparse.html) for this purpose, which comes with Python, but it's complex and verbose.\n",
    "\n",
    "`fastcore.script` makes life easier. There are much fancier modules to help you write scripts (we recommend [Python Fire](https://github.com/google/python-fire), and [Click](https://click.palletsprojects.com/en/7.x/) is also popular), but fastcore.script is very fast and very simple. In fact, it's <50 lines of code! Basically, it's just a little wrapper around `argparse` that uses modern Python features and some thoughtful defaults to get rid of the boilerplate.\n",
    "\n",
    "For full details, see the [docs](https://fastcore.script.fast.ai) for `core`."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Example"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Here's a complete example (available in `examples/test_fastcore.py`):\n",
    "\n",
    "```python\n",
    "from fastcore.script import *\n",
    "@call_parse\n",
    "def main(msg:str,     # The message\n",
    "         upper:bool): # Convert to uppercase?\n",
    "    \"Print `msg`, optionally converting to uppercase\"\n",
    "    print(msg.upper() if upper else msg)\n",
    "````"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "If you copy that info a file and run it, you'll see:\n",
    "\n",
    "```\n",
    "$ examples/test_fastcore.py --help\n",
    "usage: test_fastcore.py [-h] [--upper] msg\n",
    "\n",
    "Print `msg`, optionally converting to uppercase\n",
    "\n",
    "positional arguments:\n",
    "  msg          The message\n",
    "\n",
    "optional arguments:\n",
    "  -h, --help   show this help message and exit\n",
    "  --upper      Convert to uppercase? (default: False)\n",
    "```\n",
    "\n",
    "As you see, we didn't need any `if __name__ == \"__main__\"`, we didn't have to parse arguments, we just wrote a function, added a decorator to it, and added some annotations to our function's parameters. As a bonus, we can also use this function directly from a REPL such as Jupyter Notebook - it's not just for command line scripts!\n",
    "\n",
    "You should provide a default (after the `=`) for any *optional* parameters. If you don't provide a default for a parameter, then it will be a *positional* parameter."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Param annotations"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "If you want to use the full power of `argparse`, you can do so by using `Param` annotations instead of type annotations and [docments](https://fastcore.fast.ai/docments.html), like so:\n",
    "\n",
    "```python\n",
    "from fastcore.script import *\n",
    "@call_parse\n",
    "def main(msg:Param(\"The message\", str),\n",
    "         upper:Param(\"Convert to uppercase?\", store_true)):\n",
    "    \"Print `msg`, optionally converting to uppercase\"\n",
    "    print(msg.upper() if upper else msg)\n",
    "````\n",
    "\n",
    "If you use this approach, then each parameter in your function should have an annotation `Param(...)` (as in the example above). You can pass the following when calling `Param`: `help`,`type`,`opt`,`action`,`nargs`,`const`,`choices`,`required` . Except for `opt`, all of these are just passed directly to `argparse`, so you have all the power of that module at your disposal. Generally you'll want to pass at least `help` (since this is provided as the help string for that parameter) and `type` (to ensure that you get the type of data you expect). `opt` is a bool that defines whether a param is optional or required (positional) - but you'll generally not need to set this manually, because fastcore.script will set it for you automatically based on *default* values."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## setuptools scripts"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "There's a really nice feature of pip/setuptools that lets you create commandline scripts directly from functions, makes them available in the `PATH`, and even makes your scripts cross-platform (e.g. in Windows it creates an exe). fastcore.script supports this feature too. The trick to making a function available as a script is to add a `console_scripts` section to your setup file, of the form: `script_name=module:function_name`. E.g. in this case we use: `test_fastcore.script=fastcore.script.test_cli:main`. With this, you can then just type `test_fastcore.script` at any time, from any directory, and your script will be called (once it's installed using one of the methods below).\n",
    "\n",
    "You don't actually have to write a `setup.py` yourself. Instead, just use [nbdev](https://nbdev.fast.ai). Then modify `settings.ini` as appropriate for your module/script. To install your script directly, you can type `pip install -e .`. Your script, when installed this way (it's called an [editable install](http://codumentary.blogspot.com/2014/11/python-tip-of-year-pip-install-editable.html)), will automatically be up to date even if you edit it - there's no need to reinstall it after editing. With nbdev you can even make your module and script available for installation directly from pip and conda by running `make release`."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## API details"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#export\n",
    "import inspect,functools,argparse,shutil\n",
    "from fastcore.imports import *\n",
    "from fastcore.utils import *\n",
    "from fastcore.docments import docments"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from fastcore.test import *"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#export\n",
    "def store_true():\n",
    "    \"Placeholder to pass to `Param` for `store_true` action\"\n",
    "    pass"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#export\n",
    "def store_false():\n",
    "    \"Placeholder to pass to `Param` for `store_false` action\"\n",
    "    pass"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#export\n",
    "def bool_arg(v):\n",
    "    \"Use as `type` for `Param` to get `bool` behavior\"\n",
    "    return str2bool(v)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#export\n",
    "def clean_type_str(x:str):\n",
    "    x = str(x)\n",
    "    x = re.sub(\"(enum |class|function|__main__\\.|\\ at.*)\", '', x)\n",
    "    x = re.sub(\"(<|>|'|\\ )\", '', x) # spl characters\n",
    "    return x"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "class Test: pass\n",
    "\n",
    "test_eq(clean_type_str(argparse.ArgumentParser), 'argparse.ArgumentParser')\n",
    "test_eq(clean_type_str(Test), 'Test')\n",
    "test_eq(clean_type_str(int), 'int')\n",
    "test_eq(clean_type_str(float), 'float')\n",
    "test_eq(clean_type_str(store_false), 'store_false')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Param -"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#export\n",
    "class Param:\n",
    "    \"A parameter in a function used in `anno_parser` or `call_parse`\"\n",
    "    def __init__(self, help=None, type=None, opt=True, action=None, nargs=None, const=None,\n",
    "                 choices=None, required=None, default=None):\n",
    "        if type in (store_true,bool):  type,action,default=None,'store_true' ,False\n",
    "        if type==store_false: type,action,default=None,'store_false',True\n",
    "        if type and isinstance(type,typing.Type) and issubclass(type,enum.Enum) and not choices: choices=list(type)\n",
    "        store_attr()\n",
    "\n",
    "    def set_default(self, d):\n",
    "        if self.default is None:\n",
    "            if d==inspect.Parameter.empty: self.opt = False\n",
    "            else: self.default = d\n",
    "        if self.default is not None: self.help += f\" (default: {self.default})\"\n",
    "\n",
    "    @property\n",
    "    def pre(self): return '--' if self.opt else ''\n",
    "    @property\n",
    "    def kwargs(self): return {k:v for k,v in self.__dict__.items()\n",
    "                              if v is not None and k!='opt' and k[0]!='_'}\n",
    "    def __repr__(self):\n",
    "        if not self.help and self.type is None: return \"\"\n",
    "        if not self.help and self.type is not None: return f\"{clean_type_str(self.type)}\"\n",
    "        if self.help and self.type is None: return f\"<{self.help}>\"\n",
    "        if self.help and self.type is not None: return f\"{clean_type_str(self.type)} <{self.help}>\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "test_eq(repr(Param(\"Help goes here\")), '<Help goes here>')\n",
    "test_eq(repr(Param(\"Help\", int)), 'int <Help>')\n",
    "test_eq(repr(Param(help=None, type=int)), 'int')\n",
    "test_eq(repr(Param(help=None, type=None)), '')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Each parameter in your function should have an annotation `Param(...)`. You can pass the following when calling `Param`: `help`,`type`,`opt`,`action`,`nargs`,`const`,`choices`,`required` (i.e. it takes the same parameters as `argparse.ArgumentParser.add_argument`, plus `opt`). Except for `opt`, all of these are just passed directly to `argparse`, so you have all the power of that module at your disposal. Generally you'll want to pass at least `help` (since this is provided as the help string for that parameter) and `type` (to ensure that you get the type of data you expect).\n",
    "\n",
    "`opt` is a bool that defines whether a param is optional or required (positional) - but you'll generally not need to set this manually, because fastcore.script will set it for you automatically based on *default* values. You should provide a default (after the `=`) for any *optional* parameters. If you don't provide a default for a parameter, then it will be a *positional* parameter.\n",
    "\n",
    "Param's `__repr__` also allows for more informative function annotation when looking up the function's doc using shift+tab. You see the type annotation (if there is one) and the accompanying help documentation with it."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def f(required:Param(\"Required param\", int),\n",
    "      a:Param(\"param 1\", bool_arg),\n",
    "      b:Param(\"param 2\", str)=\"test\"):\n",
    "    \"my docs\"\n",
    "    ..."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Help on function f in module __main__:\n",
      "\n",
      "f(required: int <Required param>, a: bool_arg <param 1>, b: str <param 2> = 'test')\n",
      "    my docs\n",
      "\n"
     ]
    }
   ],
   "source": [
    "help(f)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "p = Param(help=\"help\", type=int)\n",
    "p.set_default(1)\n",
    "test_eq(p.kwargs, {'help': 'help (default: 1)', 'type': int, 'default': 1})"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## anno_parser -"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#export\n",
    "class _HelpFormatter(argparse.HelpFormatter):\n",
    "    def __init__(self, prog, indent_increment=2):\n",
    "        cols = shutil.get_terminal_size((120,30))[0]\n",
    "        super().__init__(prog, max_help_position=cols//2, width=cols, indent_increment=indent_increment)\n",
    "    def _expand_help(self, action): return self._get_help_string(action)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#export\n",
    "def anno_parser(func, prog=None, from_name=False):\n",
    "    \"Look at params (annotated with `Param`) in func and return an `ArgumentParser`\"\n",
    "    p = argparse.ArgumentParser(description=func.__doc__, prog=prog, formatter_class=_HelpFormatter)\n",
    "    for k,v in docments(func, full=True, returns=False, eval_str=True).items():\n",
    "        param = v.anno\n",
    "        if not isinstance(param,Param): param = Param(v.docment, v.anno)\n",
    "        param.set_default(v.default)\n",
    "        p.add_argument(f\"{param.pre}{k}\", **param.kwargs)\n",
    "    p.add_argument(f\"--pdb\", help=argparse.SUPPRESS, action='store_true')\n",
    "    p.add_argument(f\"--xtra\", help=argparse.SUPPRESS, type=str)\n",
    "    return p"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This converts a function with parameter annotations of type `Param` into an `argparse.ArgumentParser` object. Function arguments with a default provided are optional, and other arguments are positional."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "usage: progname [-h] [--b B] [--c {aa,bb,cc}] required a\n",
      "\n",
      "my docs\n",
      "\n",
      "positional arguments:\n",
      "  required        Required param\n",
      "  a               param 1\n",
      "\n",
      "optional arguments:\n",
      "  -h, --help      show this help message and exit\n",
      "  --b B           param 2 (default: test)\n",
      "  --c {aa,bb,cc}  param 3 (default: aa)\n"
     ]
    }
   ],
   "source": [
    "_en = str_enum('_en', 'aa','bb','cc')\n",
    "def f(required:Param(\"Required param\", int),\n",
    "      a:Param(\"param 1\", bool_arg),\n",
    "      b:Param(\"param 2\", str)=\"test\",\n",
    "      c:Param(\"param 3\", _en)=_en.aa):\n",
    "    \"my docs\"\n",
    "    ...\n",
    "\n",
    "p = anno_parser(f, 'progname')\n",
    "p.print_help()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "It also works with type annotations and docments:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "usage: progname [-h] [--b B] [--c {aa,bb,cc}] required a\n",
      "\n",
      "my docs\n",
      "\n",
      "positional arguments:\n",
      "  required        Required param\n",
      "  a               param 1\n",
      "\n",
      "optional arguments:\n",
      "  -h, --help      show this help message and exit\n",
      "  --b B           param 2 (default: test)\n",
      "  --c {aa,bb,cc}  param 3 (default: aa)\n"
     ]
    }
   ],
   "source": [
    "def g(required:int,  # Required param\n",
    "      a:bool_arg,    # param 1\n",
    "      b=\"test\",      # param 2\n",
    "      c:_en=_en.aa): # param 3\n",
    "    \"my docs\"\n",
    "    ...\n",
    "\n",
    "p = anno_parser(g, 'progname')\n",
    "p.print_help()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#export\n",
    "def args_from_prog(func, prog):\n",
    "    \"Extract args from `prog`\"\n",
    "    if prog is None or '#' not in prog: return {}\n",
    "    if '##' in prog: _,prog = prog.split('##', 1)\n",
    "    progsp = prog.split(\"#\")\n",
    "    args = {progsp[i]:progsp[i+1] for i in range(0, len(progsp), 2)}\n",
    "    annos = type_hints(func)\n",
    "    for k,v in args.items():\n",
    "        t = annos.get(k, Param()).type\n",
    "        if t: args[k] = t(v)\n",
    "    return args"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Sometimes it's convenient to extract arguments from the actual name of the called program. `args_from_prog` will do this, assuming that names and values of the params are separated by a `#`. Optionally there can also be a prefix separated by `##` (double underscore)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "exp = {'a': False, 'b': 'baa'}\n",
    "test_eq(args_from_prog(f, 'foo##a#0#b#baa'), exp)\n",
    "test_eq(args_from_prog(f, 'a#0#b#baa'), exp)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#export\n",
    "SCRIPT_INFO = SimpleNamespace(func=None)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## call_parse -"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#export\n",
    "def call_parse(func):\n",
    "    \"Decorator to create a simple CLI from `func` using `anno_parser`\"\n",
    "    mod = inspect.getmodule(inspect.currentframe().f_back)\n",
    "    if not mod: return func\n",
    "\n",
    "    @functools.wraps(func)\n",
    "    def _f(*args, **kwargs):\n",
    "        mod = inspect.getmodule(inspect.currentframe().f_back)\n",
    "        if not mod: return func(*args, **kwargs)\n",
    "        if not SCRIPT_INFO.func and mod.__name__==\"__main__\": SCRIPT_INFO.func = func.__name__\n",
    "        if len(sys.argv)>1 and sys.argv[1]=='': sys.argv.pop(1)\n",
    "        p = anno_parser(func)\n",
    "        args = p.parse_args().__dict__\n",
    "        xtra = otherwise(args.pop('xtra', ''), eq(1), p.prog)\n",
    "        tfunc = trace(func) if args.pop('pdb', False) else func\n",
    "        tfunc(**merge(args, args_from_prog(func, xtra)))\n",
    "\n",
    "    if mod.__name__==\"__main__\":\n",
    "        setattr(mod, func.__name__, _f)\n",
    "        SCRIPT_INFO.func = func.__name__\n",
    "        return _f()\n",
    "    else: return _f"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "@call_parse\n",
    "def test_add(\n",
    "    a:int,  # param a\n",
    "    b:int  # param 1\n",
    "): return a + b"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "`call_parse` decorated functions work as regular functions and also as command-line interface functions."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "test_eq(test_add(1,2), 3)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This is the main way to use `fastcore.script`; decorate your function with `call_parse`, add `Param` annotations (as shown above) or type annotations and docments, and it can then be used as a script."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Export -"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Converted 00_test.ipynb.\n",
      "Converted 01_basics.ipynb.\n",
      "Converted 02_foundation.ipynb.\n",
      "Converted 03_xtras.ipynb.\n",
      "Converted 03a_parallel.ipynb.\n",
      "Converted 03b_net.ipynb.\n",
      "Converted 04_dispatch.ipynb.\n",
      "Converted 05_transform.ipynb.\n",
      "Converted 06_docments.ipynb.\n",
      "Converted 07_meta.ipynb.\n",
      "Converted 08_script.ipynb.\n",
      "Converted index.ipynb.\n",
      "Converted parallel_win.ipynb.\n"
     ]
    }
   ],
   "source": [
    "#hide\n",
    "from nbdev.export import notebook2script\n",
    "notebook2script()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "jupytext": {
   "split_at_heading": true
  },
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}