{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "# Advanced features\n", "\n", "Here are yet more tools that the average user won't need, but might come in handy one day." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "
\n", " \n", "Click [here](https://mybinder.org/v2/gh/sciris/sciris/HEAD?labpath=docs%2Ftutorials%2Ftut_advanced.ipynb) to open an interactive version of this notebook.\n", " \n", "
\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Nested dictionaries\n", "\n", "Nested dictionaries are a useful way of storing complex data (and in fact are more or less the basis of JSON), but can be a pain to interact with if you don't know the structure in advance. Sciris has several functions for working with nested dictionaries. For example:" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "import sciris as sc\n", "\n", "# Create the structure\n", "nest = {}\n", "sc.makenested(nest, ['key1','key1.1'])\n", "sc.makenested(nest, ['key1','key1.2'])\n", "sc.makenested(nest, ['key1','key1.3'])\n", "sc.makenested(nest, ['key2','key2.1','key2.1.1'])\n", "sc.makenested(nest, ['key2','key2.2','key2.2.1'])\n", "\n", "# Set the value for each \"twig\"\n", "count = 0\n", "for twig in sc.iternested(nest):\n", " count += 1\n", " sc.setnested(nest, twig, count)\n", "\n", "# Convert to a JSON to view the structure more clearly\n", "sc.printjson(nest)\n", "\n", "# Get all the values from the dict\n", "values = []\n", "for twig in sc.iternested(nest):\n", " values.append(sc.getnested(nest, twig))\n", "print(f'{values = }')" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Sciris also has a `sc.search()` function, which can find either keys or values that match a certain pattern:" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "print(sc.search(nest, 'key2.1.1'))\n", "print(sc.search(nest, value=5))" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "There's even an `sc.iterobj()` function that can make arbitrary changes to an object:" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "def increment(obj):\n", " return obj + 1000 if isinstance(obj, int) and obj !=3 else obj\n", "\n", "sc.iterobj(nest, increment, inplace=True)\n", "sc.printjson(nest)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Context blocks\n", "\n", "Sciris contains two context block (i.e. \"`with ... as`\") classes for catching what happens inside them.\n", "\n", "`sc.capture()` captures all text output to a variable:" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "import sciris as sc\n", "import numpy as np\n", "\n", "def verbose_func(n=200):\n", " for i in range(n):\n", " print(f'Here are 5 random numbers: {np.random.rand(5)}')\n", "\n", "with sc.capture() as text:\n", " verbose_func()\n", "\n", "lines = text.splitlines()\n", "target = '777'\n", "for l,line in enumerate(lines):\n", " if target in line:\n", " print(f'Found target {target} on line {l}: {line}')" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The other function, `sc.tryexcept()`, is a more compact way of writing `try ... except` blocks, and gives detailed control of error handling:" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "def fickle_func(n=1):\n", " for i in range(n):\n", " rnd = np.random.rand()\n", " if rnd < 0.005:\n", " raise ValueError(f'Value {rnd:n} too small')\n", " elif rnd > 0.99:\n", " raise RuntimeError(f'Value {rnd:n} too big')\n", "\n", "sc.heading('Simple usage, exit gracefully at first exception')\n", "with sc.tryexcept():\n", " fickle_func(n=1000)\n", "\n", "sc.heading('Store all history')\n", "tryexc = None\n", "for i in range(1000):\n", " with sc.tryexcept(history=tryexc, verbose=False) as tryexc:\n", " fickle_func()\n", "tryexc.disp()" ] }, { "cell_type": "markdown", "metadata": { "tags": [] }, "source": [ "## Interpolation and optimization\n", "\n", "Sciris includes two algorithms that complement their SciPy relatives: interpolation and optimization.\n", "\n", "### Interpolation\n", "\n", "The function `sc.smoothinterp()` smoothly interpolates between points but does _not_ use spline interpolation; this makes it somewhat of a balance between `numpy.interp()` (which only interpolates linearly) and `scipy.interpolate.interp1d(..., method='cubic')`, which takes considerable liberties between data points:" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "import sciris as sc\n", "import numpy as np\n", "import pylab as pl\n", "from scipy import interpolate\n", "\n", "# Create the data\n", "origy = np.array([0, 0.2, 0.1, 0.9, 0.7, 0.8, 0.95, 1])\n", "origx = np.linspace(0, 1, len(origy))\n", "newx = np.linspace(0, 1)\n", "\n", "# Create the interpolations\n", "sc_y = sc.smoothinterp(newx, origx, origy, smoothness=5)\n", "np_y = np.interp(newx, origx, origy)\n", "si_y = interpolate.interp1d(origx, origy, 'cubic')(newx)\n", "\n", "# Plot\n", "kw = dict(lw=2, alpha=0.7)\n", "pl.plot(newx, np_y, '--', label='NumPy', **kw)\n", "pl.plot(newx, si_y, ':', label='SciPy', **kw)\n", "pl.plot(newx, sc_y, '-', label='Sciris', **kw)\n", "pl.scatter(origx, origy, s=50, c='k', label='Data')\n", "pl.legend();" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "As you can see, `sc.smoothinterp()` gives a more \"reasonable\" approximation to the data, at the expense of not exactly passing through all the data points." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Optimization\n", "\n", "Sciris includes a gradient descent optimization method, [adaptive stochastic descent](https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0192944) (ASD), that can outperform SciPy's built-in [optimization methods](https://docs.scipy.org/doc/scipy/reference/optimize.html) (such as simplex) for certain types of optimization problem. For example:" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "# Basic usage\n", "import numpy as np\n", "import sciris as sc\n", "from scipy import optimize\n", "\n", "# Very simple optimization problem -- set all numbers to 0\n", "func = np.linalg.norm\n", "x = [1, 2, 3]\n", "\n", "with sc.timer('scipy.optimize()'):\n", " opt_scipy = optimize.minimize(func, x)\n", "\n", "with sc.timer('sciris.asd()'):\n", " opt_sciris = sc.asd(func, x, verbose=False)\n", "\n", "print(f'Scipy result: {func(opt_scipy.x)}')\n", "print(f'Sciris result: {func(opt_sciris.x)}')" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Compared to SciPy's simplex algorithm, Sciris' ASD algorithm was ≈3 times faster and found a result ≈8 orders of magnitude smaller." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Animation\n", "\n", "And finally, let's end on something fun. Sciris has an `sc.animation()` class with lots of options, but you can also just make a quick movie from a series of plots. For example, let's make some lines dance:" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "pl.figure()\n", "frames = [pl.plot(pl.cumsum(pl.randn(100))) for i in range(20)] # Create frames\n", "sc.savemovie(frames, 'dancing_lines.gif'); # Save movie as a gif" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "This creates the following movie, which is a rather delightful way to end:\n", "\n", "![](dancing_lines.gif)\n", "\n", "We hope you enjoyed this series of tutorials! Remember, [write to us](mailto:info@sciris.org) if you want to get in touch." ] } ], "metadata": { "kernelspec": { "display_name": "Python 3 (ipykernel)", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.9.16" }, "toc": { "base_numbering": 1, "nav_menu": {}, "number_sections": true, "sideBar": true, "skip_h1_title": false, "title_cell": "Table of Contents", "title_sidebar": "Contents", "toc_cell": false, "toc_position": {}, "toc_section_display": true, "toc_window_display": false } }, "nbformat": 4, "nbformat_minor": 4 }