{ "cells": [ { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "#|default_exp foundation" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "#|export\n", "from fastcore.imports import *\n", "from fastcore.basics import *\n", "from functools import lru_cache\n", "from contextlib import contextmanager\n", "from copy import copy\n", "from configparser import ConfigParser\n", "import random,pickle,inspect" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "#|hide\n", "from fastcore.test import *\n", "from nbdev.showdoc import *\n", "from fastcore.nb_imports import *" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "# Foundation\n", "\n", "> The `L` class and helpers for it" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Foundational Functions" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "#|export\n", "@contextmanager\n", "def working_directory(path):\n", " \"Change working directory to `path` and return to previous on exit.\"\n", " prev_cwd = Path.cwd()\n", " os.chdir(path)\n", " try: yield\n", " finally: os.chdir(prev_cwd)" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "#|export\n", "def add_docs(cls, cls_doc=None, **docs):\n", " \"Copy values from `docs` to `cls` docstrings, and confirm all public methods are documented\"\n", " if cls_doc is not None: cls.__doc__ = cls_doc\n", " for k,v in docs.items():\n", " f = getattr(cls,k)\n", " if hasattr(f,'__func__'): f = f.__func__ # required for class methods\n", " f.__doc__ = v\n", " # List of public callables without docstring\n", " nodoc = [c for n,c in vars(cls).items() if callable(c)\n", " and not n.startswith('_') and c.__doc__ is None]\n", " assert not nodoc, f\"Missing docs: {nodoc}\"\n", " assert cls.__doc__ is not None, f\"Missing class docs: {cls}\"" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "`add_docs` allows you to add docstrings to a class and its associated methods. This function allows you to group docstrings together seperate from your code, which enables you to define one-line functions as well as organize your code more succintly. We believe this confers a number of benefits which we discuss in [our style guide](https://docs.fast.ai/dev/style.html).\n", "\n", "Suppose you have the following undocumented class:" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "class T:\n", " def foo(self): pass\n", " def bar(self): pass" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "You can add documentation to this class like so:" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "add_docs(T, cls_doc=\"A docstring for the class.\",\n", " foo=\"The foo method.\",\n", " bar=\"The bar method.\")" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Now, docstrings will appear as expected:" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "test_eq(T.__doc__, \"A docstring for the class.\")\n", "test_eq(T.foo.__doc__, \"The foo method.\")\n", "test_eq(T.bar.__doc__, \"The bar method.\")" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "`add_docs` also validates that all of your public methods contain a docstring. If one of your methods is not documented, it will raise an error:" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "class T:\n", " def foo(self): pass\n", " def bar(self): pass\n", "\n", "f=lambda: add_docs(T, \"A docstring for the class.\", foo=\"The foo method.\")\n", "test_fail(f, contains=\"Missing docs\")" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "#|hide\n", "class _T:\n", " def f(self): pass\n", " @classmethod\n", " def g(cls): pass\n", "add_docs(_T, \"a\", f=\"f\", g=\"g\")\n", "\n", "test_eq(_T.__doc__, \"a\")\n", "test_eq(_T.f.__doc__, \"f\")\n", "test_eq(_T.g.__doc__, \"g\")" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "#|export\n", "def docs(cls):\n", " \"Decorator version of `add_docs`, using `_docs` dict\"\n", " add_docs(cls, **cls._docs)\n", " return cls" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Instead of using `add_docs`, you can use the decorator `docs` as shown below. Note that the docstring for the class can be set with the argument `cls_doc`:" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "@docs\n", "class _T:\n", " def f(self): pass\n", " def g(cls): pass\n", " \n", " _docs = dict(cls_doc=\"The class docstring\", \n", " f=\"The docstring for method f.\",\n", " g=\"A different docstring for method g.\")\n", "\n", " \n", "test_eq(_T.__doc__, \"The class docstring\")\n", "test_eq(_T.f.__doc__, \"The docstring for method f.\")\n", "test_eq(_T.g.__doc__, \"A different docstring for method g.\")" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "For either the `docs` decorator or the `add_docs` function, you can still define your docstrings in the normal way. Below we set the docstring for the class as usual, but define the method docstrings through the `_docs` attribute:" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "@docs\n", "class _T:\n", " \"The class docstring\"\n", " def f(self): pass\n", " _docs = dict(f=\"The docstring for method f.\")\n", "\n", " \n", "test_eq(_T.__doc__, \"The class docstring\")\n", "test_eq(_T.f.__doc__, \"The docstring for method f.\")" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [ { "data": { "text/markdown": [ "---\n", "\n", "### is_iter\n", "\n", "> is_iter (o)\n", "\n", "Test whether `o` can be used in a `for` loop" ], "text/plain": [ "---\n", "\n", "### is_iter\n", "\n", "> is_iter (o)\n", "\n", "Test whether `o` can be used in a `for` loop" ] }, "execution_count": null, "metadata": {}, "output_type": "execute_result" } ], "source": [ "show_doc(is_iter)" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "assert is_iter([1])\n", "assert not is_iter(array(1))\n", "assert is_iter(array([1,2]))\n", "assert (o for o in range(3))" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "#|export\n", "def coll_repr(c, max_n=10):\n", " \"String repr of up to `max_n` items of (possibly lazy) collection `c`\"\n", " return f'(#{len(c)}) [' + ','.join(itertools.islice(map(repr,c), max_n)) + (\n", " '...' if len(c)>max_n else '') + ']'" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "`coll_repr` is used to provide a more informative [`__repr__`](https://stackoverflow.com/questions/1984162/purpose-of-pythons-repr) about list-like objects. `coll_repr` and is used by `L` to build a `__repr__` that displays the length of a list in addition to a preview of a list.\n", "\n", "Below is an example of the `__repr__` string created for a list of 1000 elements:" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "test_eq(coll_repr(range(1000)), '(#1000) [0,1,2,3,4,5,6,7,8,9...]')\n", "test_eq(coll_repr(range(1000), 5), '(#1000) [0,1,2,3,4...]')\n", "test_eq(coll_repr(range(10), 5), '(#10) [0,1,2,3,4...]')\n", "test_eq(coll_repr(range(5), 5), '(#5) [0,1,2,3,4]')" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "We can set the option `max_n` to optionally preview a specified number of items instead of the default:" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "test_eq(coll_repr(range(1000), max_n=5), '(#1000) [0,1,2,3,4...]')" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "#|export\n", "def is_bool(x):\n", " \"Check whether `x` is a bool or None\"\n", " return isinstance(x,(bool,NoneType)) or risinstance('bool_', x)" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "#|export\n", "def mask2idxs(mask):\n", " \"Convert bool mask or index list to index `L`\"\n", " if isinstance(mask,slice): return mask\n", " mask = list(mask)\n", " if len(mask)==0: return []\n", " it = mask[0]\n", " if hasattr(it,'item'): it = it.item()\n", " if is_bool(it): return [i for i,m in enumerate(mask) if m]\n", " return [int(i) for i in mask]" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "test_eq(mask2idxs([False,True,False,True]), [1,3])\n", "test_eq(mask2idxs(array([False,True,False,True])), [1,3])\n", "test_eq(mask2idxs(array([1,2,3])), [1,2,3])" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "#|export\n", "def cycle(o):\n", " \"Like `itertools.cycle` except creates list of `None`s if `o` is empty\"\n", " o = listify(o)\n", " return itertools.cycle(o) if o is not None and len(o) > 0 else itertools.cycle([None])" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "test_eq(itertools.islice(cycle([1,2,3]),5), [1,2,3,1,2])\n", "test_eq(itertools.islice(cycle([]),3), [None]*3)\n", "test_eq(itertools.islice(cycle(None),3), [None]*3)\n", "test_eq(itertools.islice(cycle(1),3), [1,1,1])" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "#|export\n", "def zip_cycle(x, *args):\n", " \"Like `itertools.zip_longest` but `cycle`s through elements of all but first argument\"\n", " return zip(x, *map(cycle,args))" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "test_eq(zip_cycle([1,2,3,4],list('abc')), [(1, 'a'), (2, 'b'), (3, 'c'), (4, 'a')])" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "#|export\n", "def is_indexer(idx):\n", " \"Test whether `idx` will index a single item in a list\"\n", " return isinstance(idx,int) or not getattr(idx,'ndim',1)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "You can, for example index a single item in a list with an integer or a 0-dimensional numpy array:" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "assert is_indexer(1)\n", "assert is_indexer(np.array(1))" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "However, you cannot index into single item in a list with another list or a numpy array with ndim > 0. " ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "assert not is_indexer([1, 2])\n", "assert not is_indexer(np.array([[1, 2], [3, 4]]))" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## `L` helpers" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "#|export\n", "class CollBase:\n", " \"Base class for composing a list of `items`\"\n", " def __init__(self, items): self.items = items\n", " def __len__(self): return len(self.items)\n", " def __getitem__(self, k): return self.items[list(k) if isinstance(k,CollBase) else k]\n", " def __setitem__(self, k, v): self.items[list(k) if isinstance(k,CollBase) else k] = v\n", " def __delitem__(self, i): del(self.items[i])\n", " def __repr__(self): return self.items.__repr__()\n", " def __iter__(self): return self.items.__iter__()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "`ColBase` is a base class that emulates the functionality of a python `list`:" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "class _T(CollBase): pass\n", "l = _T([1,2,3,4,5])\n", "\n", "test_eq(len(l), 5) # __len__\n", "test_eq(l[-1], 5); test_eq(l[0], 1) #__getitem__\n", "l[2] = 100; test_eq(l[2], 100) # __set_item__\n", "del l[0]; test_eq(len(l), 4) # __delitem__\n", "test_eq(str(l), '[2, 100, 4, 5]') # __repr__" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## L -" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "#|export\n", "class _L_Meta(type):\n", " def __call__(cls, x=None, *args, **kwargs):\n", " if not args and not kwargs and x is not None and isinstance(x,cls): return x\n", " return super().__call__(x, *args, **kwargs)" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "#|export\n", "class L(GetAttr, CollBase, metaclass=_L_Meta):\n", " \"Behaves like a list of `items` but can also index with list of indices or masks\"\n", " _default='items'\n", " def __init__(self, items=None, *rest, use_list=False, match=None):\n", " if (use_list is not None) or not is_array(items):\n", " items = listify(items, *rest, use_list=use_list, match=match)\n", " super().__init__(items)\n", "\n", " @property\n", " def _xtra(self): return None\n", " def _new(self, items, *args, **kwargs): return type(self)(items, *args, use_list=None, **kwargs)\n", " def __getitem__(self, idx): return self._get(idx) if is_indexer(idx) else L(self._get(idx), use_list=None)\n", " def copy(self): return self._new(self.items.copy())\n", "\n", " def _get(self, i):\n", " if is_indexer(i) or isinstance(i,slice): return getattr(self.items,'iloc',self.items)[i]\n", " i = mask2idxs(i)\n", " return (self.items.iloc[list(i)] if hasattr(self.items,'iloc')\n", " else self.items.__array__()[(i,)] if hasattr(self.items,'__array__')\n", " else [self.items[i_] for i_ in i])\n", "\n", " def __setitem__(self, idx, o):\n", " \"Set `idx` (can be list of indices, or mask, or int) items to `o` (which is broadcast if not iterable)\"\n", " if isinstance(idx, int): self.items[idx] = o\n", " else:\n", " idx = idx if isinstance(idx,L) else listify(idx)\n", " if not is_iter(o): o = [o]*len(idx)\n", " for i,o_ in zip(idx,o): self.items[i] = o_\n", "\n", " def __eq__(self,b):\n", " if b is None: return False\n", " if risinstance('ndarray', b): return array_equal(b, self)\n", " if isinstance(b, (str,dict)) or callable(b): return False\n", " return all_equal(b,self)\n", "\n", " def sorted(self, key=None, reverse=False): return self._new(sorted_ex(self, key=key, reverse=reverse))\n", " def __iter__(self): return iter(self.items.itertuples() if hasattr(self.items,'iloc') else self.items)\n", " def __contains__(self,b): return b in self.items\n", " def __reversed__(self): return self._new(reversed(self.items))\n", " def __invert__(self): return self._new(not i for i in self)\n", " def __repr__(self): return repr(self.items)\n", " def _repr_pretty_(self, p, cycle):\n", " p.text('...' if cycle else repr(self.items) if is_array(self.items) else coll_repr(self))\n", " def __mul__ (a,b): return a._new(a.items*b)\n", " def __add__ (a,b): return a._new(a.items+listify(b))\n", " def __radd__(a,b): return a._new(b)+a\n", " def __addi__(a,b):\n", " a.items += list(b)\n", " return a\n", "\n", " @classmethod\n", " def split(cls, s, sep=None, maxsplit=-1): return cls(s.split(sep,maxsplit))\n", " @classmethod\n", " def range(cls, a, b=None, step=None): return cls(range_of(a, b=b, step=step))\n", "\n", " def map(self, f, *args, **kwargs): return self._new(map_ex(self, f, *args, gen=False, **kwargs))\n", " def argwhere(self, f, negate=False, **kwargs): return self._new(argwhere(self, f, negate, **kwargs))\n", " def argfirst(self, f, negate=False): \n", " if negate: f = not_(f)\n", " return first(i for i,o in self.enumerate() if f(o))\n", " def filter(self, f=noop, negate=False, **kwargs):\n", " return self._new(filter_ex(self, f=f, negate=negate, gen=False, **kwargs))\n", "\n", " def enumerate(self): return L(enumerate(self))\n", " def renumerate(self): return L(renumerate(self))\n", " def unique(self, sort=False, bidir=False, start=None): return L(uniqueify(self, sort=sort, bidir=bidir, start=start))\n", " def val2idx(self): return val2idx(self)\n", " def cycle(self): return cycle(self)\n", " def map_dict(self, f=noop, *args, **kwargs): return {k:f(k, *args,**kwargs) for k in self}\n", " def map_first(self, f=noop, g=noop, *args, **kwargs):\n", " return first(self.map(f, *args, **kwargs), g)\n", "\n", " def itemgot(self, *idxs):\n", " x = self\n", " for idx in idxs: x = x.map(itemgetter(idx))\n", " return x\n", " def attrgot(self, k, default=None):\n", " return self.map(lambda o: o.get(k,default) if isinstance(o, dict) else nested_attr(o,k,default))\n", "\n", " def starmap(self, f, *args, **kwargs): return self._new(itertools.starmap(partial(f,*args,**kwargs), self))\n", " def zip(self, cycled=False): return self._new((zip_cycle if cycled else zip)(*self))\n", " def zipwith(self, *rest, cycled=False): return self._new([self, *rest]).zip(cycled=cycled)\n", " def map_zip(self, f, *args, cycled=False, **kwargs): return self.zip(cycled=cycled).starmap(f, *args, **kwargs)\n", " def map_zipwith(self, f, *rest, cycled=False, **kwargs): return self.zipwith(*rest, cycled=cycled).starmap(f, **kwargs)\n", " def shuffle(self):\n", " it = copy(self.items)\n", " random.shuffle(it)\n", " return self._new(it)\n", "\n", " def concat(self): return self._new(itertools.chain.from_iterable(self.map(L)))\n", " def reduce(self, f, initial=None): return reduce(f, self) if initial is None else reduce(f, self, initial)\n", " def sum(self): return self.reduce(operator.add, 0)\n", " def product(self): return self.reduce(operator.mul, 1)\n", " def setattrs(self, attr, val): [setattr(o,attr,val) for o in self]" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "#|export\n", "add_docs(L,\n", " __getitem__=\"Retrieve `idx` (can be list of indices, or mask, or int) items\",\n", " range=\"Class Method: Same as `range`, but returns `L`. Can pass collection for `a`, to use `len(a)`\",\n", " split=\"Class Method: Same as `str.split`, but returns an `L`\",\n", " copy=\"Same as `list.copy`, but returns an `L`\",\n", " sorted=\"New `L` sorted by `key`. If key is str use `attrgetter`; if int use `itemgetter`\",\n", " unique=\"Unique items, in stable order\",\n", " val2idx=\"Dict from value to index\",\n", " filter=\"Create new `L` filtered by predicate `f`, passing `args` and `kwargs` to `f`\",\n", " argwhere=\"Like `filter`, but return indices for matching items\",\n", " argfirst=\"Return index of first matching item\",\n", " map=\"Create new `L` with `f` applied to all `items`, passing `args` and `kwargs` to `f`\",\n", " map_first=\"First element of `map_filter`\",\n", " map_dict=\"Like `map`, but creates a dict from `items` to function results\",\n", " starmap=\"Like `map`, but use `itertools.starmap`\",\n", " itemgot=\"Create new `L` with item `idx` of all `items`\",\n", " attrgot=\"Create new `L` with attr `k` (or value `k` for dicts) of all `items`.\",\n", " cycle=\"Same as `itertools.cycle`\",\n", " enumerate=\"Same as `enumerate`\",\n", " renumerate=\"Same as `renumerate`\",\n", " zip=\"Create new `L` with `zip(*items)`\",\n", " zipwith=\"Create new `L` with `self` zip with each of `*rest`\",\n", " map_zip=\"Combine `zip` and `starmap`\",\n", " map_zipwith=\"Combine `zipwith` and `starmap`\",\n", " concat=\"Concatenate all elements of list\",\n", " shuffle=\"Same as `random.shuffle`, but not inplace\",\n", " reduce=\"Wrapper for `functools.reduce`\",\n", " sum=\"Sum of the items\",\n", " product=\"Product of the items\",\n", " setattrs=\"Call `setattr` on all items\"\n", " )" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "#|export\n", "#|hide\n", "# Here we are fixing the signature of L. What happens is that the __call__ method on the MetaClass of L shadows the __init__\n", "# giving the wrong signature (https://stackoverflow.com/questions/49740290/call-from-metaclass-shadows-signature-of-init).\n", "def _f(items=None, *rest, use_list=False, match=None): ...\n", "L.__signature__ = inspect.signature(_f)" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "#|export\n", "Sequence.register(L);" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "`L` is a drop in replacement for a python `list`. Inspired by [NumPy](http://www.numpy.org/), `L`, supports advanced indexing and has additional methods (outlined below) that provide additional functionality and encourage simple expressive code. For example, the code below takes a list of pairs, selects the second item of each pair, takes its absolute value, filters items greater than 4, and adds them up:" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "from fastcore.utils import gt" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "d = dict(a=1,b=-5,d=6,e=9).items()\n", "test_eq(L(d).itemgot(1).map(abs).filter(gt(4)).sum(), 20) # abs(-5) + abs(6) + abs(9) = 20; 1 was filtered out." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Read [this overview section](https://fastcore.fast.ai/tour.html#L) for a quick tutorial of `L`, as well as background on the name. \n", "\n", "You can create an `L` from an existing iterable (e.g. a list, range, etc) and access or modify it with an int list/tuple index, mask, int, or slice. All `list` methods can also be used with `L`." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "(#12) [11,10,9,'j',7,'k',5,4,3,2...]" ] }, "execution_count": null, "metadata": {}, "output_type": "execute_result" } ], "source": [ "t = L(range(12))\n", "test_eq(t, list(range(12)))\n", "test_ne(t, list(range(11)))\n", "t.reverse()\n", "test_eq(t[0], 11)\n", "t[3] = \"h\"\n", "test_eq(t[3], \"h\")\n", "t[3,5] = (\"j\",\"k\")\n", "test_eq(t[3,5], [\"j\",\"k\"])\n", "test_eq(t, L(t))\n", "test_eq(L(L(1,2),[3,4]), ([1,2],[3,4]))\n", "t" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Any `L` is a `Sequence` so you can use it with methods like `random.sample`:" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "assert isinstance(t, Sequence)" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "import random" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "[5, 0, 11]" ] }, "execution_count": null, "metadata": {}, "output_type": "execute_result" } ], "source": [ "random.seed(0)\n", "random.sample(t, 3)" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "#|hide\n", "# test set items with L of collections\n", "x = L([[1,2,3], [4,5], [6,7]])\n", "x[0] = [1,2]\n", "test_eq(x, L([[1,2], [4,5], [6,7]]))\n", "\n", "# non-idiomatic None-ness check - avoid infinite recursion\n", "some_var = L(['a', 'b'])\n", "assert some_var != None, \"L != None\"" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "There are optimized indexers for arrays, tensors, and DataFrames." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "import pandas as pd" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "arr = np.arange(9).reshape(3,3)\n", "t = L(arr, use_list=None)\n", "test_eq(t[1,2], arr[[1,2]])\n", "\n", "df = pd.DataFrame({'a':[1,2,3]})\n", "t = L(df, use_list=None)\n", "test_eq(t[1,2], L(pd.DataFrame({'a':[2,3]}, index=[1,2]), use_list=None))" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "You can also modify an `L` with `append`, `+`, and `*`." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "t = L()\n", "test_eq(t, [])\n", "t.append(1)\n", "test_eq(t, [1])\n", "t += [3,2]\n", "test_eq(t, [1,3,2])\n", "t = t + [4]\n", "test_eq(t, [1,3,2,4])\n", "t = 5 + t\n", "test_eq(t, [5,1,3,2,4])\n", "test_eq(L(1,2,3), [1,2,3])\n", "test_eq(L(1,2,3), L(1,2,3))\n", "t = L(1)*5\n", "t = t.map(operator.neg)\n", "test_eq(t,[-1]*5)\n", "test_eq(~L([True,False,False]), L([False,True,True]))\n", "t = L(range(4))\n", "test_eq(zip(t, L(1).cycle()), zip(range(4),(1,1,1,1)))\n", "t = L.range(100)\n", "test_shuffled(t,t.shuffle())" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "test_eq(L([]).sum(), 0)\n", "test_eq(L([]).product(), 1)" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "def _f(x,a=0): return x+a\n", "t = L(1)*5\n", "test_eq(t.map(_f), t)\n", "test_eq(t.map(_f,1), [2]*5)\n", "test_eq(t.map(_f,a=2), [3]*5)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "An `L` can be constructed from anything iterable, although tensors and arrays will not be iterated over on construction, unless you pass `use_list` to the constructor." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "test_eq(L([1,2,3]),[1,2,3])\n", "test_eq(L(L([1,2,3])),[1,2,3])\n", "test_ne(L([1,2,3]),[1,2,])\n", "test_eq(L('abc'),['abc'])\n", "test_eq(L(range(0,3)),[0,1,2])\n", "test_eq(L(o for o in range(0,3)),[0,1,2])\n", "test_eq(L(array(0)),[array(0)])\n", "test_eq(L([array(0),array(1)]),[array(0),array(1)])\n", "test_eq(L(array([0.,1.1]))[0],array([0.,1.1]))\n", "test_eq(L(array([0.,1.1]), use_list=True), [array(0.),array(1.1)]) # `use_list=True` to unwrap arrays/arrays" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "If `match` is not `None` then the created list is same len as `match`, either by:\n", "\n", "- If `len(items)==1` then `items` is replicated,\n", "- Otherwise an error is raised if `match` and `items` are not already the same size." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "test_eq(L(1,match=[1,2,3]),[1,1,1])\n", "test_eq(L([1,2],match=[2,3]),[1,2])\n", "test_fail(lambda: L([1,2],match=[1,2,3]))" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "If you create an `L` from an existing `L` then you'll get back the original object (since `L` uses the `NewChkMeta` metaclass)." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "test_is(L(t), t)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "An `L` is considred equal to a list if they have the same elements. It's never considered equal to a `str` a `set` or a `dict` even if they have the same elements/keys." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "test_eq(L(['a', 'b']), ['a', 'b'])\n", "test_ne(L(['a', 'b']), 'ab')\n", "test_ne(L(['a', 'b']), {'a':1, 'b':2})" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### `L` Methods" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [ { "data": { "text/markdown": [ "---\n", "\n", "[source](https://github.com/fastai/fastcore/blob/master/fastcore/foundation.py#L112){target=\"_blank\" style=\"float:right; font-size:smaller\"}\n", "\n", "### L.__getitem__\n", "\n", "> L.__getitem__ (idx)\n", "\n", "Retrieve `idx` (can be list of indices, or mask, or int) items" ], "text/plain": [ "---\n", "\n", "[source](https://github.com/fastai/fastcore/blob/master/fastcore/foundation.py#L112){target=\"_blank\" style=\"float:right; font-size:smaller\"}\n", "\n", "### L.__getitem__\n", "\n", "> L.__getitem__ (idx)\n", "\n", "Retrieve `idx` (can be list of indices, or mask, or int) items" ] }, "execution_count": null, "metadata": {}, "output_type": "execute_result" } ], "source": [ "show_doc(L.__getitem__)" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "t = L(range(12))\n", "test_eq(t[1,2], [1,2]) # implicit tuple\n", "test_eq(t[[1,2]], [1,2]) # list\n", "test_eq(t[:3], [0,1,2]) # slice\n", "test_eq(t[[False]*11 + [True]], [11]) # mask\n", "test_eq(t[array(3)], 3)" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [ { "data": { "text/markdown": [ "---\n", "\n", "[source](https://github.com/fastai/fastcore/blob/master/fastcore/foundation.py#L122){target=\"_blank\" style=\"float:right; font-size:smaller\"}\n", "\n", "### L.__setitem__\n", "\n", "> L.__setitem__ (idx, o)\n", "\n", "Set `idx` (can be list of indices, or mask, or int) items to `o` (which is broadcast if not iterable)" ], "text/plain": [ "---\n", "\n", "[source](https://github.com/fastai/fastcore/blob/master/fastcore/foundation.py#L122){target=\"_blank\" style=\"float:right; font-size:smaller\"}\n", "\n", "### L.__setitem__\n", "\n", "> L.__setitem__ (idx, o)\n", "\n", "Set `idx` (can be list of indices, or mask, or int) items to `o` (which is broadcast if not iterable)" ] }, "execution_count": null, "metadata": {}, "output_type": "execute_result" } ], "source": [ "show_doc(L.__setitem__)" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "t[4,6] = 0\n", "test_eq(t[4,6], [0,0])\n", "t[4,6] = [1,2]\n", "test_eq(t[4,6], [1,2])" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [ { "data": { "text/markdown": [ "---\n", "\n", "[source](https://github.com/fastai/fastcore/blob/master/fastcore/foundation.py#L166){target=\"_blank\" style=\"float:right; font-size:smaller\"}\n", "\n", "### L.unique\n", "\n", "> L.unique (sort=False, bidir=False, start=None)\n", "\n", "Unique items, in stable order" ], "text/plain": [ "---\n", "\n", "[source](https://github.com/fastai/fastcore/blob/master/fastcore/foundation.py#L166){target=\"_blank\" style=\"float:right; font-size:smaller\"}\n", "\n", "### L.unique\n", "\n", "> L.unique (sort=False, bidir=False, start=None)\n", "\n", "Unique items, in stable order" ] }, "execution_count": null, "metadata": {}, "output_type": "execute_result" } ], "source": [ "show_doc(L.unique)" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "test_eq(L(4,1,2,3,4,4).unique(), [4,1,2,3])" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [ { "data": { "text/markdown": [ "---\n", "\n", "[source](https://github.com/fastai/fastcore/blob/master/fastcore/foundation.py#L167){target=\"_blank\" style=\"float:right; font-size:smaller\"}\n", "\n", "### L.val2idx\n", "\n", "> L.val2idx ()\n", "\n", "Dict from value to index" ], "text/plain": [ "---\n", "\n", "[source](https://github.com/fastai/fastcore/blob/master/fastcore/foundation.py#L167){target=\"_blank\" style=\"float:right; font-size:smaller\"}\n", "\n", "### L.val2idx\n", "\n", "> L.val2idx ()\n", "\n", "Dict from value to index" ] }, "execution_count": null, "metadata": {}, "output_type": "execute_result" } ], "source": [ "show_doc(L.val2idx)" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "test_eq(L(1,2,3).val2idx(), {3:2,1:0,2:1})" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [ { "data": { "text/markdown": [ "---\n", "\n", "[source](https://github.com/fastai/fastcore/blob/master/fastcore/foundation.py#L161){target=\"_blank\" style=\"float:right; font-size:smaller\"}\n", "\n", "### L.filter\n", "\n", "> L.filter (f=, negate=False, gen=False, **kwargs)\n", "\n", "Create new `L` filtered by predicate `f`, passing `args` and `kwargs` to `f`" ], "text/plain": [ "---\n", "\n", "[source](https://github.com/fastai/fastcore/blob/master/fastcore/foundation.py#L161){target=\"_blank\" style=\"float:right; font-size:smaller\"}\n", "\n", "### L.filter\n", "\n", "> L.filter (f=, negate=False, gen=False, **kwargs)\n", "\n", "Create new `L` filtered by predicate `f`, passing `args` and `kwargs` to `f`" ] }, "execution_count": null, "metadata": {}, "output_type": "execute_result" } ], "source": [ "show_doc(L.filter)" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "[0, 1, 2, 3, 1, 5, 2, 7, 8, 9, 10, 11]" ] }, "execution_count": null, "metadata": {}, "output_type": "execute_result" } ], "source": [ "list(t)" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "test_eq(t.filter(lambda o:o<5), [0,1,2,3,1,2])\n", "test_eq(t.filter(lambda o:o<5, negate=True), [5,7,8,9,10,11])" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [ { "data": { "text/markdown": [ "---\n", "\n", "[source](https://github.com/fastai/fastcore/blob/master/fastcore/foundation.py#L157){target=\"_blank\" style=\"float:right; font-size:smaller\"}\n", "\n", "### L.argwhere\n", "\n", "> L.argwhere (f, negate=False, **kwargs)\n", "\n", "Like `filter`, but return indices for matching items" ], "text/plain": [ "---\n", "\n", "[source](https://github.com/fastai/fastcore/blob/master/fastcore/foundation.py#L157){target=\"_blank\" style=\"float:right; font-size:smaller\"}\n", "\n", "### L.argwhere\n", "\n", "> L.argwhere (f, negate=False, **kwargs)\n", "\n", "Like `filter`, but return indices for matching items" ] }, "execution_count": null, "metadata": {}, "output_type": "execute_result" } ], "source": [ "show_doc(L.argwhere)" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "test_eq(t.argwhere(lambda o:o<5), [0,1,2,3,4,6])" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [ { "data": { "text/markdown": [ "---\n", "\n", "[source](https://github.com/fastai/fastcore/blob/master/fastcore/foundation.py#L158){target=\"_blank\" style=\"float:right; font-size:smaller\"}\n", "\n", "### L.argfirst\n", "\n", "> L.argfirst (f, negate=False)\n", "\n", "Return index of first matching item" ], "text/plain": [ "---\n", "\n", "[source](https://github.com/fastai/fastcore/blob/master/fastcore/foundation.py#L158){target=\"_blank\" style=\"float:right; font-size:smaller\"}\n", "\n", "### L.argfirst\n", "\n", "> L.argfirst (f, negate=False)\n", "\n", "Return index of first matching item" ] }, "execution_count": null, "metadata": {}, "output_type": "execute_result" } ], "source": [ "show_doc(L.argfirst)" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "test_eq(t.argfirst(lambda o:o>4), 5)\n", "test_eq(t.argfirst(lambda o:o>4,negate=True),0)" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [ { "data": { "text/markdown": [ "---\n", "\n", "[source](https://github.com/fastai/fastcore/blob/master/fastcore/foundation.py#L156){target=\"_blank\" style=\"float:right; font-size:smaller\"}\n", "\n", "### L.map\n", "\n", "> L.map (f, *args, gen=False, **kwargs)\n", "\n", "Create new `L` with `f` applied to all `items`, passing `args` and `kwargs` to `f`" ], "text/plain": [ "---\n", "\n", "[source](https://github.com/fastai/fastcore/blob/master/fastcore/foundation.py#L156){target=\"_blank\" style=\"float:right; font-size:smaller\"}\n", "\n", "### L.map\n", "\n", "> L.map (f, *args, gen=False, **kwargs)\n", "\n", "Create new `L` with `f` applied to all `items`, passing `args` and `kwargs` to `f`" ] }, "execution_count": null, "metadata": {}, "output_type": "execute_result" } ], "source": [ "show_doc(L.map)" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "test_eq(L.range(4).map(operator.neg), [0,-1,-2,-3])" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "If `f` is a string then it is treated as a format string to create the mapping:" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "test_eq(L.range(4).map('#{}#'), ['#0#','#1#','#2#','#3#'])" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "If `f` is a dictionary (or anything supporting `__getitem__`) then it is indexed to create the mapping:" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "test_eq(L.range(4).map(list('abcd')), list('abcd'))" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "You can also pass the same `arg` params that `bind` accepts:" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "def f(a=None,b=None): return b\n", "test_eq(L.range(4).map(f, b=arg0), range(4))" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [ { "data": { "text/markdown": [ "---\n", "\n", "[source](https://github.com/fastai/fastcore/blob/master/fastcore/foundation.py#L169){target=\"_blank\" style=\"float:right; font-size:smaller\"}\n", "\n", "### L.map_dict\n", "\n", "> L.map_dict (f=, *args, gen=False, **kwargs)\n", "\n", "Like `map`, but creates a dict from `items` to function results" ], "text/plain": [ "---\n", "\n", "[source](https://github.com/fastai/fastcore/blob/master/fastcore/foundation.py#L169){target=\"_blank\" style=\"float:right; font-size:smaller\"}\n", "\n", "### L.map_dict\n", "\n", "> L.map_dict (f=, *args, gen=False, **kwargs)\n", "\n", "Like `map`, but creates a dict from `items` to function results" ] }, "execution_count": null, "metadata": {}, "output_type": "execute_result" } ], "source": [ "show_doc(L.map_dict)" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "test_eq(L(range(1,5)).map_dict(), {1:1, 2:2, 3:3, 4:4})\n", "test_eq(L(range(1,5)).map_dict(operator.neg), {1:-1, 2:-2, 3:-3, 4:-4})" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [ { "data": { "text/markdown": [ "---\n", "\n", "[source](https://github.com/fastai/fastcore/blob/master/fastcore/foundation.py#L181){target=\"_blank\" style=\"float:right; font-size:smaller\"}\n", "\n", "### L.zip\n", "\n", "> L.zip (cycled=False)\n", "\n", "Create new `L` with `zip(*items)`" ], "text/plain": [ "---\n", "\n", "[source](https://github.com/fastai/fastcore/blob/master/fastcore/foundation.py#L181){target=\"_blank\" style=\"float:right; font-size:smaller\"}\n", "\n", "### L.zip\n", "\n", "> L.zip (cycled=False)\n", "\n", "Create new `L` with `zip(*items)`" ] }, "execution_count": null, "metadata": {}, "output_type": "execute_result" } ], "source": [ "show_doc(L.zip)" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "t = L([[1,2,3],'abc'])\n", "test_eq(t.zip(), [(1, 'a'),(2, 'b'),(3, 'c')])" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "t = L([[1,2,3,4],['a','b','c']])\n", "test_eq(t.zip(cycled=True ), [(1, 'a'),(2, 'b'),(3, 'c'),(4, 'a')])\n", "test_eq(t.zip(cycled=False), [(1, 'a'),(2, 'b'),(3, 'c')])" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [ { "data": { "text/markdown": [ "---\n", "\n", "[source](https://github.com/fastai/fastcore/blob/master/fastcore/foundation.py#L183){target=\"_blank\" style=\"float:right; font-size:smaller\"}\n", "\n", "### L.map_zip\n", "\n", "> L.map_zip (f, *args, cycled=False, **kwargs)\n", "\n", "Combine `zip` and `starmap`" ], "text/plain": [ "---\n", "\n", "[source](https://github.com/fastai/fastcore/blob/master/fastcore/foundation.py#L183){target=\"_blank\" style=\"float:right; font-size:smaller\"}\n", "\n", "### L.map_zip\n", "\n", "> L.map_zip (f, *args, cycled=False, **kwargs)\n", "\n", "Combine `zip` and `starmap`" ] }, "execution_count": null, "metadata": {}, "output_type": "execute_result" } ], "source": [ "show_doc(L.map_zip)" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "t = L([1,2,3],[2,3,4])\n", "test_eq(t.map_zip(operator.mul), [2,6,12])" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [ { "data": { "text/markdown": [ "---\n", "\n", "[source](https://github.com/fastai/fastcore/blob/master/fastcore/foundation.py#L182){target=\"_blank\" style=\"float:right; font-size:smaller\"}\n", "\n", "### L.zipwith\n", "\n", "> L.zipwith (*rest, cycled=False)\n", "\n", "Create new `L` with `self` zip with each of `*rest`" ], "text/plain": [ "---\n", "\n", "[source](https://github.com/fastai/fastcore/blob/master/fastcore/foundation.py#L182){target=\"_blank\" style=\"float:right; font-size:smaller\"}\n", "\n", "### L.zipwith\n", "\n", "> L.zipwith (*rest, cycled=False)\n", "\n", "Create new `L` with `self` zip with each of `*rest`" ] }, "execution_count": null, "metadata": {}, "output_type": "execute_result" } ], "source": [ "show_doc(L.zipwith)" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "b = [[0],[1],[2,2]]\n", "t = L([1,2,3]).zipwith(b)\n", "test_eq(t, [(1,[0]), (2,[1]), (3,[2,2])])" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [ { "data": { "text/markdown": [ "---\n", "\n", "[source](https://github.com/fastai/fastcore/blob/master/fastcore/foundation.py#L184){target=\"_blank\" style=\"float:right; font-size:smaller\"}\n", "\n", "### L.map_zipwith\n", "\n", "> L.map_zipwith (f, *rest, cycled=False, **kwargs)\n", "\n", "Combine `zipwith` and `starmap`" ], "text/plain": [ "---\n", "\n", "[source](https://github.com/fastai/fastcore/blob/master/fastcore/foundation.py#L184){target=\"_blank\" style=\"float:right; font-size:smaller\"}\n", "\n", "### L.map_zipwith\n", "\n", "> L.map_zipwith (f, *rest, cycled=False, **kwargs)\n", "\n", "Combine `zipwith` and `starmap`" ] }, "execution_count": null, "metadata": {}, "output_type": "execute_result" } ], "source": [ "show_doc(L.map_zipwith)" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "test_eq(L(1,2,3).map_zipwith(operator.mul, [2,3,4]), [2,6,12])" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [ { "data": { "text/markdown": [ "---\n", "\n", "[source](https://github.com/fastai/fastcore/blob/master/fastcore/foundation.py#L173){target=\"_blank\" style=\"float:right; font-size:smaller\"}\n", "\n", "### L.itemgot\n", "\n", "> L.itemgot (*idxs)\n", "\n", "Create new `L` with item `idx` of all `items`" ], "text/plain": [ "---\n", "\n", "[source](https://github.com/fastai/fastcore/blob/master/fastcore/foundation.py#L173){target=\"_blank\" style=\"float:right; font-size:smaller\"}\n", "\n", "### L.itemgot\n", "\n", "> L.itemgot (*idxs)\n", "\n", "Create new `L` with item `idx` of all `items`" ] }, "execution_count": null, "metadata": {}, "output_type": "execute_result" } ], "source": [ "show_doc(L.itemgot)" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "test_eq(t.itemgot(1), b)" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [ { "data": { "text/markdown": [ "---\n", "\n", "[source](https://github.com/fastai/fastcore/blob/master/fastcore/foundation.py#L177){target=\"_blank\" style=\"float:right; font-size:smaller\"}\n", "\n", "### L.attrgot\n", "\n", "> L.attrgot (k, default=None)\n", "\n", "Create new `L` with attr `k` (or value `k` for dicts) of all `items`." ], "text/plain": [ "---\n", "\n", "[source](https://github.com/fastai/fastcore/blob/master/fastcore/foundation.py#L177){target=\"_blank\" style=\"float:right; font-size:smaller\"}\n", "\n", "### L.attrgot\n", "\n", "> L.attrgot (k, default=None)\n", "\n", "Create new `L` with attr `k` (or value `k` for dicts) of all `items`." ] }, "execution_count": null, "metadata": {}, "output_type": "execute_result" } ], "source": [ "show_doc(L.attrgot)" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "# Example when items are not a dict\n", "a = [SimpleNamespace(a=3,b=4),SimpleNamespace(a=1,b=2)]\n", "test_eq(L(a).attrgot('b'), [4,2])\n", "\n", "#Example of when items are a dict\n", "b =[{'id': 15, 'name': 'nbdev'}, {'id': 17, 'name': 'fastcore'}]\n", "test_eq(L(b).attrgot('id'), [15, 17])" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [ { "data": { "text/markdown": [ "---\n", "\n", "[source](https://github.com/fastai/fastcore/blob/master/fastcore/foundation.py#L136){target=\"_blank\" style=\"float:right; font-size:smaller\"}\n", "\n", "### L.sorted\n", "\n", "> L.sorted (key=None, reverse=False)\n", "\n", "New `L` sorted by `key`. If key is str use `attrgetter`; if int use `itemgetter`" ], "text/plain": [ "---\n", "\n", "[source](https://github.com/fastai/fastcore/blob/master/fastcore/foundation.py#L136){target=\"_blank\" style=\"float:right; font-size:smaller\"}\n", "\n", "### L.sorted\n", "\n", "> L.sorted (key=None, reverse=False)\n", "\n", "New `L` sorted by `key`. If key is str use `attrgetter`; if int use `itemgetter`" ] }, "execution_count": null, "metadata": {}, "output_type": "execute_result" } ], "source": [ "show_doc(L.sorted)" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "test_eq(L(a).sorted('a').attrgot('b'), [2,4])" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [ { "data": { "text/markdown": [ "---\n", "\n", "[source](https://github.com/fastai/fastcore/blob/master/fastcore/foundation.py#L152){target=\"_blank\" style=\"float:right; font-size:smaller\"}\n", "\n", "### L.split\n", "\n", "> L.split (s, sep=None, maxsplit=-1)\n", "\n", "Class Method: Same as `str.split`, but returns an `L`" ], "text/plain": [ "---\n", "\n", "[source](https://github.com/fastai/fastcore/blob/master/fastcore/foundation.py#L152){target=\"_blank\" style=\"float:right; font-size:smaller\"}\n", "\n", "### L.split\n", "\n", "> L.split (s, sep=None, maxsplit=-1)\n", "\n", "Class Method: Same as `str.split`, but returns an `L`" ] }, "execution_count": null, "metadata": {}, "output_type": "execute_result" } ], "source": [ "show_doc(L.split)" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "test_eq(L.split('a b c'), list('abc'))" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [ { "data": { "text/markdown": [ "---\n", "\n", "[source](https://github.com/fastai/fastcore/blob/master/fastcore/foundation.py#L154){target=\"_blank\" style=\"float:right; font-size:smaller\"}\n", "\n", "### L.range\n", "\n", "> L.range (a, b=None, step=None)\n", "\n", "Class Method: Same as `range`, but returns `L`. Can pass collection for `a`, to use `len(a)`" ], "text/plain": [ "---\n", "\n", "[source](https://github.com/fastai/fastcore/blob/master/fastcore/foundation.py#L154){target=\"_blank\" style=\"float:right; font-size:smaller\"}\n", "\n", "### L.range\n", "\n", "> L.range (a, b=None, step=None)\n", "\n", "Class Method: Same as `range`, but returns `L`. Can pass collection for `a`, to use `len(a)`" ] }, "execution_count": null, "metadata": {}, "output_type": "execute_result" } ], "source": [ "show_doc(L.range)" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "test_eq_type(L.range([1,1,1]), L(range(3)))\n", "test_eq_type(L.range(5,2,2), L(range(5,2,2)))" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [ { "data": { "text/markdown": [ "---\n", "\n", "[source](https://github.com/fastai/fastcore/blob/master/fastcore/foundation.py#L190){target=\"_blank\" style=\"float:right; font-size:smaller\"}\n", "\n", "### L.concat\n", "\n", "> L.concat ()\n", "\n", "Concatenate all elements of list" ], "text/plain": [ "---\n", "\n", "[source](https://github.com/fastai/fastcore/blob/master/fastcore/foundation.py#L190){target=\"_blank\" style=\"float:right; font-size:smaller\"}\n", "\n", "### L.concat\n", "\n", "> L.concat ()\n", "\n", "Concatenate all elements of list" ] }, "execution_count": null, "metadata": {}, "output_type": "execute_result" } ], "source": [ "show_doc(L.concat)" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "test_eq(L([0,1,2,3],4,L(5,6)).concat(), range(7))" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [ { "data": { "text/markdown": [ "---\n", "\n", "[source](https://github.com/fastai/fastcore/blob/master/fastcore/foundation.py#L113){target=\"_blank\" style=\"float:right; font-size:smaller\"}\n", "\n", "### L.copy\n", "\n", "> L.copy ()\n", "\n", "Same as `list.copy`, but returns an `L`" ], "text/plain": [ "---\n", "\n", "[source](https://github.com/fastai/fastcore/blob/master/fastcore/foundation.py#L113){target=\"_blank\" style=\"float:right; font-size:smaller\"}\n", "\n", "### L.copy\n", "\n", "> L.copy ()\n", "\n", "Same as `list.copy`, but returns an `L`" ] }, "execution_count": null, "metadata": {}, "output_type": "execute_result" } ], "source": [ "show_doc(L.copy)" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "t = L([0,1,2,3],4,L(5,6)).copy()\n", "test_eq(t.concat(), range(7))" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [ { "data": { "text/markdown": [ "---\n", "\n", "[source](https://github.com/fastai/fastcore/blob/master/fastcore/foundation.py#L170){target=\"_blank\" style=\"float:right; font-size:smaller\"}\n", "\n", "### L.map_first\n", "\n", "> L.map_first (f=, g=, *args, **kwargs)\n", "\n", "First element of `map_filter`" ], "text/plain": [ "---\n", "\n", "[source](https://github.com/fastai/fastcore/blob/master/fastcore/foundation.py#L170){target=\"_blank\" style=\"float:right; font-size:smaller\"}\n", "\n", "### L.map_first\n", "\n", "> L.map_first (f=, g=, *args, **kwargs)\n", "\n", "First element of `map_filter`" ] }, "execution_count": null, "metadata": {}, "output_type": "execute_result" } ], "source": [ "show_doc(L.map_first)" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "t = L(0,1,2,3)\n", "test_eq(t.map_first(lambda o:o*2 if o>2 else None), 6)" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [ { "data": { "text/markdown": [ "---\n", "\n", "[source](https://github.com/fastai/fastcore/blob/master/fastcore/foundation.py#L194){target=\"_blank\" style=\"float:right; font-size:smaller\"}\n", "\n", "### L.setattrs\n", "\n", "> L.setattrs (attr, val)\n", "\n", "Call `setattr` on all items" ], "text/plain": [ "---\n", "\n", "[source](https://github.com/fastai/fastcore/blob/master/fastcore/foundation.py#L194){target=\"_blank\" style=\"float:right; font-size:smaller\"}\n", "\n", "### L.setattrs\n", "\n", "> L.setattrs (attr, val)\n", "\n", "Call `setattr` on all items" ] }, "execution_count": null, "metadata": {}, "output_type": "execute_result" } ], "source": [ "show_doc(L.setattrs)" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "t = L(SimpleNamespace(),SimpleNamespace())\n", "t.setattrs('foo', 'bar')\n", "test_eq(t.attrgot('foo'), ['bar','bar'])" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Config" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "#|export\n", "def save_config_file(file, d, **kwargs):\n", " \"Write settings dict to a new config file, or overwrite the existing one.\"\n", " config = ConfigParser(**kwargs)\n", " config['DEFAULT'] = d\n", " config.write(open(file, 'w'))" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "#|export\n", "def read_config_file(file, **kwargs):\n", " config = ConfigParser(**kwargs)\n", " config.read(file, encoding='utf8')\n", " return config['DEFAULT']" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Config files are saved and read using Python's `configparser.ConfigParser`, inside the `DEFAULT` section." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "{'user': 'fastai',\n", " 'lib_name': 'fastcore',\n", " 'some_path': 'test',\n", " 'some_bool': 'True',\n", " 'some_num': '3'}" ] }, "execution_count": null, "metadata": {}, "output_type": "execute_result" } ], "source": [ "_d = dict(user='fastai', lib_name='fastcore', some_path='test', some_bool=True, some_num=3)\n", "try:\n", " save_config_file('tmp.ini', _d)\n", " res = read_config_file('tmp.ini')\n", "finally: os.unlink('tmp.ini')\n", "dict(res)" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "#|export\n", "class Config:\n", " \"Reading and writing `ConfigParser` ini files\"\n", " def __init__(self, cfg_path, cfg_name, create=None, save=True, extra_files=None, types=None):\n", " self.types = types or {}\n", " cfg_path = Path(cfg_path).expanduser().absolute()\n", " self.config_path,self.config_file = cfg_path,cfg_path/cfg_name\n", " self._cfg = ConfigParser()\n", " self.d = self._cfg['DEFAULT']\n", " found = [Path(o) for o in self._cfg.read(L(extra_files)+[self.config_file], encoding='utf8')]\n", " if self.config_file not in found and create is not None:\n", " self._cfg.read_dict({'DEFAULT':create})\n", " if save:\n", " cfg_path.mkdir(exist_ok=True, parents=True)\n", " save_config_file(self.config_file, create)\n", "\n", " def __repr__(self): return repr(dict(self._cfg.items('DEFAULT', raw=True)))\n", " def __setitem__(self,k,v): self.d[k] = str(v)\n", " def __contains__(self,k): return k in self.d\n", " def save(self): save_config_file(self.config_file,self.d)\n", " def __getattr__(self,k): return stop(AttributeError(k)) if k=='d' or k not in self.d else self.get(k)\n", " def __getitem__(self,k): return stop(IndexError(k)) if k not in self.d else self.get(k)\n", "\n", " def get(self,k,default=None):\n", " v = self.d.get(k, default)\n", " if v is None: return None\n", " typ = self.types.get(k, None)\n", " if typ==bool: return str2bool(v)\n", " if not typ: return str(v)\n", " if typ==Path: return self.config_path/v\n", " return typ(v)\n", "\n", " def path(self,k,default=None):\n", " v = self.get(k, default)\n", " return v if v is None else self.config_path/v" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "`Config` is a convenient wrapper around `ConfigParser` ini files with a single section (`DEFAULT`).\n", "\n", "Instantiate a `Config` from an ini file at `cfg_path/cfg_name`:" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "{'user': 'fastai', 'lib_name': 'fastcore', 'some_path': 'test', 'some_bool': 'True', 'some_num': '3'}" ] }, "execution_count": null, "metadata": {}, "output_type": "execute_result" } ], "source": [ "save_config_file('../tmp.ini', _d)\n", "try: cfg = Config('..', 'tmp.ini')\n", "finally: os.unlink('../tmp.ini')\n", "cfg" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "You can create a new file if one doesn't exist by providing a `create` dict:" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "{'user': 'fastai', 'lib_name': 'fastcore', 'some_path': 'test', 'some_bool': 'True', 'some_num': '3'}" ] }, "execution_count": null, "metadata": {}, "output_type": "execute_result" } ], "source": [ "try: cfg = Config('..', 'tmp.ini', create=_d)\n", "finally: os.unlink('../tmp.ini')\n", "cfg" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "If you additionally pass `save=False`, the `Config` will contain the items from `create` without writing a new file:" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "cfg = Config('..', 'tmp.ini', create=_d, save=False)\n", "test_eq(cfg.user,'fastai')\n", "assert not Path('../tmp.ini').exists()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Keys can be accessed as attributes, items, or with `get` and an optional default:" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "test_eq(cfg.user,'fastai')\n", "test_eq(cfg['some_path'], 'test')\n", "test_eq(cfg.get('foo','bar'),'bar')" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Extra files can be read _before_ `cfg_path/cfg_name` using `extra_files`, in the order they appear:" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "with tempfile.TemporaryDirectory() as d:\n", " a = Config(d, 'a.ini', {'a':0,'b':0})\n", " b = Config(d, 'b.ini', {'a':1,'c':0})\n", " c = Config(d, 'c.ini', {'a':2,'d':0}, extra_files=[a.config_file,b.config_file])\n", " test_eq(c.d, {'a':'2','b':'0','c':'0','d':'0'})" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "If you pass a dict `types`, then the values of that dict will be used as types to instantiate all values returned. `Path` is a special case -- in that case, the path returned will be relative to the path containing the config file (assuming the value is relative). `bool` types use `str2bool` to convert to boolean." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "_types = dict(some_path=Path, some_bool=bool, some_num=int)\n", "cfg = Config('..', 'tmp.ini', create=_d, save=False, types=_types)\n", "\n", "test_eq(cfg.user,'fastai')\n", "test_eq(cfg['some_path'].resolve(), (Path('..')/'test').resolve())\n", "test_eq(cfg.get('some_num'), 3)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "# Export -" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "#|hide\n", "import nbdev; nbdev.nbdev_export()" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [] } ], "metadata": { "jupytext": { "split_at_heading": true }, "kernelspec": { "display_name": "Python 3 (ipykernel)", "language": "python", "name": "python3" } }, "nbformat": 4, "nbformat_minor": 4 }