{ "cells": [ { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "from functools import *" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "
\n", "

10.2. functools — Higher-order functions and operations on callable objects

\n", "

Source code: Lib/functools.py

\n", "
\n", "

The functools module is for higher-order functions: functions that act on\n", "or return other functions. In general, any callable object can be treated as a\n", "function for the purposes of this module.

\n", "

The functools module defines the following functions:

\n", "
\n", "
\n", "functools.cmp_to_key(func)
\n", "

Transform an old-style comparison function to a key function. Used\n", "with tools that accept key functions (such as sorted(), min(),\n", "max(), heapq.nlargest(), heapq.nsmallest(),\n", "itertools.groupby()). This function is primarily used as a transition\n", "tool for programs being converted from Python 2 which supported the use of\n", "comparison functions.

\n", "

A comparison function is any callable that accept two arguments, compares them,\n", "and returns a negative number for less-than, zero for equality, or a positive\n", "number for greater-than. A key function is a callable that accepts one\n", "argument and returns another value to be used as the sort key.

\n", "

Example:

\n" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "sorted(iterable, key=cmp_to_key(locale.strcoll)) # locale-aware sort order\n", "\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "\n", "

For sorting examples and a brief sorting tutorial, see Sorting HOW TO.

\n", "
\n", "

New in version 3.2.

\n", "
\n", "
\n", "
\n", "
\n", "@functools.lru_cache(maxsize=128, typed=False)
\n", "

Decorator to wrap a function with a memoizing callable that saves up to the\n", "maxsize most recent calls. It can save time when an expensive or I/O bound\n", "function is periodically called with the same arguments.

\n", "

Since a dictionary is used to cache results, the positional and keyword\n", "arguments to the function must be hashable.

\n", "

If maxsize is set to None, the LRU feature is disabled and the cache can\n", "grow without bound. The LRU feature performs best when maxsize is a\n", "power-of-two.

\n", "

If typed is set to true, function arguments of different types will be\n", "cached separately. For example, f(3) and f(3.0) will be treated\n", "as distinct calls with distinct results.

\n", "

To help measure the effectiveness of the cache and tune the maxsize\n", "parameter, the wrapped function is instrumented with a cache_info()\n", "function that returns a named tuple showing hits, misses,\n", "maxsize and currsize. In a multi-threaded environment, the hits\n", "and misses are approximate.

\n", "

The decorator also provides a cache_clear() function for clearing or\n", "invalidating the cache.

\n", "

The original underlying function is accessible through the\n", "__wrapped__ attribute. This is useful for introspection, for\n", "bypassing the cache, or for rewrapping the function with a different cache.

\n", "

An LRU (least recently used) cache works\n", "best when the most recent calls are the best predictors of upcoming calls (for\n", "example, the most popular articles on a news server tend to change each day).\n", "The cache’s size limit assures that the cache does not grow without bound on\n", "long-running processes such as web servers.

\n", "

Example of an LRU cache for static web content:

\n" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "for n in 8, 290, 308, 320, 8, 218, 320, 279, 289, 320, 9991:\n", " pep = get_pep(n)\n", " print(n, len(pep))\n", "get_pep.cache_info()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "\n", "

Example of efficiently computing\n", "Fibonacci numbers\n", "using a cache to implement a\n", "dynamic programming\n", "technique:

\n" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "[fib(n) for n in range(16)]\n", "fib.cache_info()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "\n", "
\n", "

New in version 3.2.

\n", "
\n", "
\n", "

Changed in version 3.3: Added the typed option.

\n", "
\n", "
\n", "
\n", "
\n", "@functools.total_ordering
\n", "

Given a class defining one or more rich comparison ordering methods, this\n", "class decorator supplies the rest. This simplifies the effort involved\n", "in specifying all of the possible rich comparison operations:

\n", "

The class must define one of __lt__(), __le__(),\n", "__gt__(), or __ge__().\n", "In addition, the class should supply an __eq__() method.

\n", "

For example:

\n" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "@total_ordering\n", "class Student:\n", " def _is_valid_operand(self, other):\n", " return (hasattr(other, \"lastname\") and\n", " hasattr(other, \"firstname\"))\n", " def __eq__(self, other):\n", " if not self._is_valid_operand(other):\n", " return NotImplemented\n", " return ((self.lastname.lower(), self.firstname.lower()) ==\n", " (other.lastname.lower(), other.firstname.lower()))\n", " def __lt__(self, other):\n", " if not self._is_valid_operand(other):\n", " return NotImplemented\n", " return ((self.lastname.lower(), self.firstname.lower()) <\n", " (other.lastname.lower(), other.firstname.lower()))\n", "\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "\n", "
\n", "

Note

\n", "

While this decorator makes it easy to create well behaved totally\n", "ordered types, it does come at the cost of slower execution and\n", "more complex stack traces for the derived comparison methods. If\n", "performance benchmarking indicates this is a bottleneck for a given\n", "application, implementing all six rich comparison methods instead is\n", "likely to provide an easy speed boost.

\n", "
\n", "
\n", "

New in version 3.2.

\n", "
\n", "
\n", "

Changed in version 3.4: Returning NotImplemented from the underlying comparison function for\n", "unrecognised types is now supported.

\n", "
\n", "
\n", "
\n", "
\n", "functools.partial(func, *args, **keywords)
\n", "

Return a new partial object which when called will behave like func\n", "called with the positional arguments args and keyword arguments keywords. If\n", "more arguments are supplied to the call, they are appended to args. If\n", "additional keyword arguments are supplied, they extend and override keywords.\n", "Roughly equivalent to:

\n" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "def partial(func, *args, **keywords):\n", " def newfunc(*fargs, **fkeywords):\n", " newkeywords = keywords.copy()\n", " newkeywords.update(fkeywords)\n", " return func(*args, *fargs, **newkeywords)\n", " newfunc.func = func\n", " newfunc.args = args\n", " newfunc.keywords = keywords\n", " return newfunc\n", "\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "\n", "

The partial() is used for partial function application which “freezes”\n", "some portion of a function’s arguments and/or keywords resulting in a new object\n", "with a simplified signature. For example, partial() can be used to create\n", "a callable that behaves like the int() function where the base argument\n", "defaults to two:

\n" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "from functools import partial\n", "basetwo = partial(int, base=2)\n", "basetwo.__doc__ = 'Convert base 2 string to an int.'\n", "basetwo('10010')" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "\n", "
\n", "
\n", "
\n", "class functools.partialmethod(func, *args, **keywords)
\n", "

Return a new partialmethod descriptor which behaves\n", "like partial except that it is designed to be used as a method\n", "definition rather than being directly callable.

\n", "

func must be a descriptor or a callable (objects which are both,\n", "like normal functions, are handled as descriptors).

\n", "

When func is a descriptor (such as a normal Python function,\n", "classmethod(), staticmethod(), abstractmethod() or\n", "another instance of partialmethod), calls to __get__ are\n", "delegated to the underlying descriptor, and an appropriate\n", "partial object returned as the result.

\n", "

When func is a non-descriptor callable, an appropriate bound method is\n", "created dynamically. This behaves like a normal Python function when\n", "used as a method: the self argument will be inserted as the first\n", "positional argument, even before the args and keywords supplied to\n", "the partialmethod constructor.

\n", "

Example:

\n" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "class Cell(object):\n", " def __init__(self):\n", " self._alive = False\n", " @property\n", " def alive(self):\n", " return self._alive\n", " def set_state(self, state):\n", " self._alive = bool(state)\n", " set_alive = partialmethod(set_state, True)\n", " set_dead = partialmethod(set_state, False)\n", "c = Cell()\n", "c.alive\n", "c.set_alive()\n", "c.alive" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "\n", "
\n", "

New in version 3.4.

\n", "
\n", "
\n", "
\n", "
\n", "functools.reduce(function, iterable[, initializer])
\n", "

Apply function of two arguments cumulatively to the items of sequence, from\n", "left to right, so as to reduce the sequence to a single value. For example,\n", "reduce(lambda x, y: x+y, [1, 2, 3, 4, 5]) calculates ((((1+2)+3)+4)+5).\n", "The left argument, x, is the accumulated value and the right argument, y, is\n", "the update value from the sequence. If the optional initializer is present,\n", "it is placed before the items of the sequence in the calculation, and serves as\n", "a default when the sequence is empty. If initializer is not given and\n", "sequence contains only one item, the first item is returned.

\n", "

Roughly equivalent to:

\n" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "def reduce(function, iterable, initializer=None):\n", " it = iter(iterable)\n", " if initializer is None:\n", " value = next(it)\n", " else:\n", " value = initializer\n", " for element in it:\n", " value = function(value, element)\n", " return value\n", "\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "\n", "
\n", "
\n", "
\n", "@functools.singledispatch(default)
\n", "

Transforms a function into a single-dispatch generic function.

\n", "

To define a generic function, decorate it with the @singledispatch\n", "decorator. Note that the dispatch happens on the type of the first argument,\n", "create your function accordingly:

\n" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "from functools import singledispatch\n", "@singledispatch\n", "def fun(arg, verbose=False):\n", " if verbose:\n", " print(\"Let me just say,\", end=\" \")\n", " print(arg)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "\n", "

To add overloaded implementations to the function, use the register()\n", "attribute of the generic function. It is a decorator, taking a type\n", "parameter and decorating a function implementing the operation for that\n", "type:

\n" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "@fun.register(int)\n", "def _(arg, verbose=False):\n", " if verbose:\n", " print(\"Strength in numbers, eh?\", end=\" \")\n", " print(arg)\n", "@fun.register(list)\n", "def _(arg, verbose=False):\n", " if verbose:\n", " print(\"Enumerate this:\")\n", " for i, elem in enumerate(arg):\n", " print(i, elem)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "\n", "

To enable registering lambdas and pre-existing functions, the\n", "register() attribute can be used in a functional form:

\n" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "def nothing(arg, verbose=False):\n", " print(\"Nothing.\")\n", "fun.register(type(None), nothing)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "\n", "

The register() attribute returns the undecorated function which\n", "enables decorator stacking, pickling, as well as creating unit tests for\n", "each variant independently:

\n" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "@fun.register(float)\n", "@fun.register(Decimal)\n", "def fun_num(arg, verbose=False):\n", " if verbose:\n", " print(\"Half of your number:\", end=\" \")\n", " print(arg / 2)\n", "fun_num is fun" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "\n", "

When called, the generic function dispatches on the type of the first\n", "argument:

\n" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "fun(\"Hello, world.\")\n", "fun(\"test.\", verbose=True)\n", "fun(42, verbose=True)\n", "fun(['spam', 'spam', 'eggs', 'spam'], verbose=True)\n", "fun(None)\n", "fun(1.23)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "\n", "

Where there is no registered implementation for a specific type, its\n", "method resolution order is used to find a more generic implementation.\n", "The original function decorated with @singledispatch is registered\n", "for the base object type, which means it is used if no better\n", "implementation is found.

\n", "

To check which implementation will the generic function choose for\n", "a given type, use the dispatch() attribute:

\n" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "fun.dispatch(float)\n", "fun.dispatch(dict) # note: default implementation" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "\n", "

To access all registered implementations, use the read-only registry\n", "attribute:

\n" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "fun.registry.keys()\n", "fun.registry[float]\n", "fun.registry[object]" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "\n", "
\n", "

New in version 3.4.

\n", "
\n", "
\n", "
\n", "
\n", "functools.update_wrapper(wrapper, wrapped, assigned=WRAPPER_ASSIGNMENTS, updated=WRAPPER_UPDATES)
\n", "

Update a wrapper function to look like the wrapped function. The optional\n", "arguments are tuples to specify which attributes of the original function are\n", "assigned directly to the matching attributes on the wrapper function and which\n", "attributes of the wrapper function are updated with the corresponding attributes\n", "from the original function. The default values for these arguments are the\n", "module level constants WRAPPER_ASSIGNMENTS (which assigns to the wrapper\n", "function’s __module__, __name__, __qualname__, __annotations__\n", "and __doc__, the documentation string) and WRAPPER_UPDATES (which\n", "updates the wrapper function’s __dict__, i.e. the instance dictionary).

\n", "

To allow access to the original function for introspection and other purposes\n", "(e.g. bypassing a caching decorator such as lru_cache()), this function\n", "automatically adds a __wrapped__ attribute to the wrapper that refers to\n", "the function being wrapped.

\n", "

The main intended use for this function is in decorator functions which\n", "wrap the decorated function and return the wrapper. If the wrapper function is\n", "not updated, the metadata of the returned function will reflect the wrapper\n", "definition rather than the original function definition, which is typically less\n", "than helpful.

\n", "

update_wrapper() may be used with callables other than functions. Any\n", "attributes named in assigned or updated that are missing from the object\n", "being wrapped are ignored (i.e. this function will not attempt to set them\n", "on the wrapper function). AttributeError is still raised if the\n", "wrapper function itself is missing any attributes named in updated.

\n", "
\n", "

New in version 3.2: Automatic addition of the __wrapped__ attribute.

\n", "
\n", "
\n", "

New in version 3.2: Copying of the __annotations__ attribute by default.

\n", "
\n", "
\n", "

Changed in version 3.2: Missing attributes no longer trigger an AttributeError.

\n", "
\n", "
\n", "

Changed in version 3.4: The __wrapped__ attribute now always refers to the wrapped\n", "function, even if that function defined a __wrapped__ attribute.\n", "(see bpo-17482)

\n", "
\n", "
\n", "
\n", "
\n", "@functools.wraps(wrapped, assigned=WRAPPER_ASSIGNMENTS, updated=WRAPPER_UPDATES)
\n", "

This is a convenience function for invoking update_wrapper() as a\n", "function decorator when defining a wrapper function. It is equivalent to\n", "partial(update_wrapper, wrapped=wrapped, assigned=assigned, updated=updated).\n", "For example:

\n" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "from functools import wraps\n", "def my_decorator(f):\n", " @wraps(f)\n", " def wrapper(*args, **kwds):\n", " print('Calling decorated function')\n", " return f(*args, **kwds)\n", " return wrapper\n", "@my_decorator\n", "def example():\n", " \"\"\"Docstring\"\"\"\n", " print('Called example function')\n", "example()\n", "example.__name__\n", "example.__doc__" ] } ], "metadata": {}, "nbformat": 4, "nbformat_minor": 2 }