{ "cells": [ { "cell_type": "markdown", "metadata": { "toc": true }, "source": [ "

Table of Contents

\n", "
" ] }, { "cell_type": "code", "execution_count": 1, "metadata": {}, "outputs": [ { "data": { "text/html": [ "\n", "\n" ], "text/plain": [ "" ] }, "execution_count": 1, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# code for loading the format for the notebook\n", "import os\n", "\n", "# path : store the current path to convert back to it later\n", "path = os.getcwd()\n", "os.chdir(os.path.join('..', '..', 'notebook_format'))\n", "\n", "from formats import load_style\n", "load_style(plot_style = False)" ] }, { "cell_type": "code", "execution_count": 2, "metadata": { "scrolled": true }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Ethen 2017-09-06 08:25:08 \n", "\n", "CPython 3.5.2\n", "IPython 6.1.0\n" ] } ], "source": [ "os.chdir(path)\n", "\n", "# magic to print version\n", "%load_ext watermark\n", "%watermark -a 'Ethen' -d -t -v" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "# Decorators" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Basic concepts: \n", "\n", "- In python, functions are `First class functions` allow us to treat functions like any other object, so for example: we can pass functions as arguments to another function, return functions and we can assign functions to variables. \n", "- `Closures` allows us to take advantage of `First class functions` that returns an inner function that remembers and has access to variables local to the scope in which they were created.\n", "\n", "More detailed description in the following link. [Youtube: Programming Terms: First-Class Functions](https://www.youtube.com/watch?v=kr0mpwqttM0&feature=cards&src_vid=swU3c34d2NQ&annotation_id=98878b78-dceb-4942-8d49-bcbed34e5263)." ] }, { "cell_type": "code", "execution_count": 3, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Hello John\n", "Hello John\n", "Hello John\n", "Hello John\n", "Hello John\n" ] } ], "source": [ "# we can assign functions to variables\n", "def greet(name) :\n", " return 'Hello ' + name\n", "\n", "greet_someone = greet\n", "print(greet_someone('John'))\n", "\n", "\n", "# we can define functions inside other functions\n", "def greet(name) :\n", " def get_message() :\n", " return 'Hello '\n", "\n", " result = get_message() + name\n", " return result\n", "\n", "print(greet('John'))\n", "\n", "\n", "# functions can be passed as parameters to other functions\n", "def greet(name):\n", " return 'Hello ' + name \n", "\n", "def call_func(func):\n", " other_name = 'John'\n", " return func(other_name) \n", "\n", "print(call_func(greet))\n", "\n", "\n", "# functions can return other functions \n", "def compose_greet_func():\n", " def get_message():\n", " return 'Hello John'\n", "\n", " return get_message\n", "\n", "greet = compose_greet_func()\n", "print(greet())\n", "\n", "\n", "# scoping, access the inner functions.\n", "# note that python only allows read access to the outer scope and not assignment\n", "def compose_greet_func(name):\n", " def get_message():\n", " return 'Hello ' + name\n", "\n", " return get_message\n", "\n", "greet = compose_greet_func('John')\n", "print(greet())" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Decorators Example" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "A **decorator** is basically a function that takes another function as an argument, adds some kind of functionality and then returns another function. So why would we want to do something like this? Well, it's because this allows us to easily add or alter the functionality to our existing function method or class without having to directly use its subclasses. In short, **decorators** are simply wrappers to existing functions.\n", "\n", "In the example below, the `p_decorate` function takes another function as an argument and generates a new function which augments the work of the original function, and returning the generated function so we can use it anywhere else." ] }, { "cell_type": "code", "execution_count": 4, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "

Hello John

\n" ] } ], "source": [ "def get_text(name):\n", " return 'Hello {}'.format(name)\n", "\n", "def p_decorate(func):\n", " def func_wrapper(name):\n", " return '

{0}

'.format(func(name))\n", " \n", " return func_wrapper\n", "\n", "# pass in the function to the decorator function,\n", "# and note that the decorator function returns\n", "# the wrapper function that's waiting to be executed\n", "my_get_text = p_decorate(get_text)\n", "\n", "print(my_get_text('John'))" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "And in python, there's neat shortcut for that, which is to mention the name of the decorating function before the function to be decorated. and perpend the decorator with an @ symbol.\n", "\n", "So instead of calling `p_decorate(get_text)`. It becomes:" ] }, { "cell_type": "code", "execution_count": 5, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "

Hello John

\n" ] } ], "source": [ "@p_decorate\n", "def get_text(name):\n", " return 'Hello {}'.format(name)\n", "\n", "# adding the decorator alters the functionality of \n", "# `get_text` without having to change any piece of code\n", "# in `get_text`\n", "print(get_text('John'))" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Using multiple decorators. Note the ordering matters." ] }, { "cell_type": "code", "execution_count": 6, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "

Hello John

\n", "

Hello John

\n" ] } ], "source": [ "def get_text(name):\n", " return 'Hello {}'.format(name)\n", "\n", "def p_decorate(func):\n", " def func_wrapper(name):\n", " return '

{0}

'.format(func(name))\n", " \n", " return func_wrapper\n", "\n", "def strong_decorate(func):\n", " def func_wrapper(name):\n", " return '{0}'.format(func(name))\n", " \n", " return func_wrapper\n", "\n", "def div_decorate(func):\n", " def func_wrapper(name):\n", " return '
{0}
'.format(func(name))\n", " \n", " return func_wrapper\n", "\n", "\n", "# the original way \n", "get_text1 = div_decorate(p_decorate(strong_decorate(get_text)))\n", "print(get_text1('John'))\n", "\n", "\n", "# the decorator way\n", "@div_decorate\n", "@p_decorate\n", "@strong_decorate\n", "def get_text(name):\n", " return 'Hello {}'.format(name)\n", "\n", "print(get_text('John'))" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "We can build **decorators** for class's methods in a similar fashion. We can do this by putting `*args` and `**kwargs` as parameters for the wrapper, then it can accept any arbitrary number of arguments and keyword arguments. This approach will make our decorators work for both functions and methods alike." ] }, { "cell_type": "code", "execution_count": 7, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "

John Doe

\n" ] } ], "source": [ "# You can also use it with class's methods.\n", "def p_decorate(func):\n", " def func_wrapper(*args, **kwargs):\n", " return '

{0}

'.format(func(*args, **kwargs))\n", " \n", " return func_wrapper\n", "\n", "\n", "class Person:\n", " def __init__(self):\n", " self.name = 'John'\n", " self.family = 'Doe'\n", "\n", " @p_decorate\n", " def get_fullname(self):\n", " return self.name + ' ' + self.family\n", "\n", "\n", "my_person = Person()\n", "print(my_person.get_fullname())" ] }, { "cell_type": "code", "execution_count": 8, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "wrapper executed this before display_info\n", "display info ran with arguments (John, 25)\n" ] } ], "source": [ "# Another example of using *args and **kwargs when working the function\n", "def decorator_function1(original_function):\n", " def wrapper_function(*args, **kwargs):\n", " print('wrapper executed this before {}'.format(original_function.__name__))\n", " return original_function(*args, **kwargs)\n", " return wrapper_function\n", "\n", "\n", "@decorator_function1\n", "def display_info(name, age):\n", " print('display info ran with arguments ({}, {})'.format(name, age))\n", " \n", "\n", "display_info('John', 25)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Looking back at the example above, you can notice how redundant the decorators in the example are. 3 decorators, `div_decorate`, `p_decorate`, `strong_decorate` each with the same functionality but wrapping the string with different tags. We can definitely do much better than that. Why not have a more general implementation for one that takes the tag to wrap with as a argument? Yes please!" ] }, { "cell_type": "code", "execution_count": 9, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "

Hello John

\n" ] } ], "source": [ "def tags(tag_name):\n", " def tags_decorator(func):\n", " def func_wrapper(name):\n", " return '<{0}>{1}'.format(tag_name, func(name))\n", " \n", " return func_wrapper\n", " \n", " return tags_decorator\n", "\n", "\n", "@tags('p')\n", "def get_text(name):\n", " return 'Hello ' + name\n", "\n", "print(get_text('John'))" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "In this case, our **decorators** expect to receive an additional argument, that is why we will have to build a function that takes those extra arguments and generate our decorator on the fly.\n", "\n", "At the end of the day decorators are just wrapping our functions, in case of debugging that can be problematic since the wrapper function does not carry the name, module and docstring of the original function." ] }, { "cell_type": "code", "execution_count": 10, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "func_wrapper\n" ] } ], "source": [ "# it prints out the name of the decorator instead \n", "# of the original function because it got overridden\n", "print(get_text.__name__)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Fortunately, python includes the `functools` module which contains `wraps`. `Wraps` is a **decorator** for updating the attributes of the wrapping function(func_wrapper) to those of the original function(get_text). This is as simple as decorating func_wrapper by `@wraps(func)`. Here is the updated example:" ] }, { "cell_type": "code", "execution_count": 11, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "

Hello John

\n", "get_text\n", "returns some text\n" ] } ], "source": [ "from functools import wraps\n", "\n", "\n", "def tags(tag_name):\n", " def tags_decorator(func): \n", " \n", " @wraps(func)\n", " def func_wrapper(name):\n", " return '<{0}>{1}'.format(tag_name, func(name))\n", " \n", " return func_wrapper\n", " \n", " return tags_decorator\n", "\n", "\n", "@tags('p')\n", "def get_text(name):\n", " \"\"\"returns some text\"\"\"\n", " return 'Hello ' + name\n", "\n", "print(get_text('John'))\n", "print(get_text.__name__) # get_text\n", "print(get_text.__doc__ ) # returns some text" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Practical Use Cases" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Adding timing and logging to functions." ] }, { "cell_type": "code", "execution_count": 12, "metadata": {}, "outputs": [], "source": [ "import time\n", "import logging\n", "\n", "\n", "def logger(func):\n", " \"\"\"\n", " create logging for the function,\n", " re-define the format to add specific logging time\n", " \"\"\"\n", " @wraps(func)\n", " def wrapper(*args, **kwargs):\n", " logging.basicConfig(\n", " filename = '{}.log'.format( func.__name__ ),\n", " format = '%(asctime)s -- %(levelname)s:%(name)s: %(message)s',\n", " datefmt = '%Y/%m/%d-%H:%M:%S',\n", " level = logging.INFO)\n", " \n", " # custom the logging information\n", " logging.info('Ran with args: {} and kwargs: {}'.format(args, kwargs))\n", " return func(*args, **kwargs)\n", "\n", " return wrapper\n", "\n", "\n", "def timer(func):\n", " \"\"\"time the running time of the passed in function\"\"\"\n", " @wraps(func)\n", " def wrapper(*args, **kwargs):\n", " t1 = time.time()\n", " result = func(*args, **kwargs)\n", " t2 = time.time() - t1\n", " print('{} ran in: {} sec'.format(func.__name__, t2))\n", " return result\n", " \n", " return wrapper" ] }, { "cell_type": "code", "execution_count": 13, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "display info ran with arguments (John, 25)\n", "display_info ran in: 1.002450942993164 sec\n" ] } ], "source": [ "@timer\n", "def display_info(name, age):\n", " time.sleep(1) # manually add a second to the timing\n", " print('display info ran with arguments ({}, {})'.format(name, age))\n", "\n", "display_info('John', 25)" ] }, { "cell_type": "code", "execution_count": 14, "metadata": {}, "outputs": [], "source": [ "# http://stackoverflow.com/questions/18786912/get-output-from-the-logging-module-in-ipython-notebook\n", "# ipython notebook already call basicConfig somewhere, thus reload the logging\n", "from importlib import reload\n", "reload(logging)\n", "\n", "\n", "@logger\n", "def display_info(name, age):\n", " print('display info ran with arguments ({}, {})'.format(name, age))\n", " \n", "\n", "display_info('John', 25)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Both time and log the function" ] }, { "cell_type": "code", "execution_count": 15, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "display info ran with arguments (Tom, 22)\n", "display_info ran in: 1.0016038417816162 sec\n" ] } ], "source": [ "@logger\n", "@timer\n", "def display_info(name, age):\n", " time.sleep(1) # manually add a second to the timing\n", " print('display info ran with arguments ({}, {})'.format(name, age))\n", " \n", "display_info('Tom', 22)" ] }, { "cell_type": "code", "execution_count": 16, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "'display_info'" ] }, "execution_count": 16, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# good idea to add @wraps to your decorator for debugging\n", "display_info.__name__" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The good thing about writing the timing and logging function as **decorators** is that we can use them without having the re-write the logic every time we wish to use timing or logging functionality." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "# Reference \n", "\n", "- [Blog: Guide to python function decorators](http://thecodeship.com/patterns/guide-to-python-function-decorators/)\n", "- [Youtube: Decorators - Dynamically Alter The Functionality Of Your Functions](https://www.youtube.com/watch?v=FsAPt_9Bf3U)" ] } ], "metadata": { "anaconda-cloud": {}, "kernelspec": { "display_name": "Python 3", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.6.4" }, "toc": { "nav_menu": { "height": "148px", "width": "252px" }, "number_sections": true, "sideBar": true, "skip_h1_title": false, "title_cell": "Table of Contents", "title_sidebar": "Contents", "toc_cell": true, "toc_position": {}, "toc_section_display": "block", "toc_window_display": true }, "varInspector": { "cols": { "lenName": 16, "lenType": 16, "lenVar": 40 }, "kernels_config": { "python": { "delete_cmd_postfix": "", "delete_cmd_prefix": "del ", "library": "var_list.py", "varRefreshCmd": "print(var_dic_list())" }, "r": { "delete_cmd_postfix": ") ", "delete_cmd_prefix": "rm(", "library": "var_list.r", "varRefreshCmd": "cat(var_dic_list()) " } }, "types_to_exclude": [ "module", "function", "builtin_function_or_method", "instance", "_Feature" ], "window_display": false } }, "nbformat": 4, "nbformat_minor": 1 }