{ "cells": [ { "cell_type": "markdown", "metadata": { "toc": "true" }, "source": [ "# Table of Contents\n", "
" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "# *\"Living in a noisy world...\"*, using James Powell's (`dutc`) `rwatch` module\n", "\n", "Goal : I want to write a [context manager](https://docs.python.org/3.5/library/stdtypes.html#typecontextmanager) for [Python 3.5+](https://www.Python.org/), so that inside the context manager, every number is seen noisy (with a white Gaussian noise, for instance).\n", "\n", "> It will be like being drunk, except that your it will be my Python interpretor and not me !\n", "\n", "For instance, I will like to have this feature:" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "```python\n", ">>> x = 120193\n", ">>> print(x)\n", "120193\n", ">>> np.random.seed(1234)\n", ">>> with WhiteNoise():\n", ">>> print(x)\n", "120193.47143516373249306\n", "```" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "----\n", "## Requirements and links" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "First, we will need [numpy](https://docs.scipy.org/doc/numpy/user/whatisnumpy.html) to have some random number generator, as `numpy.random.normal` to have some 1D Gaussian noise." ] }, { "cell_type": "code", "execution_count": 1, "metadata": { "collapsed": true }, "outputs": [], "source": [ "import numpy as np" ] }, { "cell_type": "code", "execution_count": 2, "metadata": { "collapsed": false }, "outputs": [ { "data": { "text/plain": [ "0.47143516373249306" ] }, "execution_count": 2, "metadata": {}, "output_type": "execute_result" } ], "source": [ "np.random.seed(1234)\n", "np.random.normal()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Then, the core part will be to install and import [James Powell (`dutc`)](https://github.com/dutc/) [`rwatch`](https://github.com/dutc/rwatch) module.\n", "If you don't have it installed :\n", "\n", "1. Be sure to have CPython 3.5. `rwatch` patches the CPython eval loop, so it's fixed to specific versions of Python & the author lazy about keeping it updated.\n", "2. Then `pip install dutc-rwatch`. It should work, but it fails for me.\n", "3. (alternative) You can just `cd /tmp/ && git clone https://github.com/dutc/rwatch && cd rwatch/src/ && make` and copy the `rwatch.so` dynamic library wherever you need..." ] }, { "cell_type": "code", "execution_count": 5, "metadata": { "collapsed": false }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "gcc `python3-config --cflags` `python3-config --includes` -fPIC -DPy_BUILD_CORE -c -o ceval.o ceval.c\n", "gcc `python3-config --cflags` `python3-config --includes` -L`python3-config --prefix`/lib -Wl,--export-dynamic -fPIC -shared -o rwatch.so rwatch.c hook.c ceval.o -ldl `python3-config --libs`\n", "-rwxrwxr-x 1 lilian lilian 453K mai 14 01:20 ./rwatch.so\n", "./rwatch.so: ELF 64-bit LSB shared object, x86-64, version 1 (SYSV), dynamically linked, BuildID[sha1]=f41ef744646e6247119bd702039725fc3a1c8e66, not stripped\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "Clonage dans 'rwatch'...\n", "In file included from ceval.c:300:0:\n", "ceval_gil.h:114:46: warning: initialization makes integer from pointer without a cast [-Wint-conversion]\n", " static _Py_atomic_address gil_last_holder = {NULL};\n", " ^~~~\n", "ceval_gil.h:114:46: note: (near initialization for ‘gil_last_holder._value’)\n", "ceval_gil.h: In function ‘create_gil’:\n", "ceval_gil.h:145:5: warning: initialization makes integer from pointer without a cast [-Wint-conversion]\n", " _Py_atomic_store_relaxed(&gil_last_holder, NULL);\n", " ^~~~~~~~~~~~~~~~~~~~~~~~\n", "ceval_gil.h: In function ‘drop_gil’:\n", "ceval_gil.h:181:9: warning: initialization makes integer from pointer without a cast [-Wint-conversion]\n", " _Py_atomic_store_relaxed(&gil_last_holder, tstate);\n", " ^~~~~~~~~~~~~~~~~~~~~~~~\n", "ceval_gil.h: In function ‘take_gil’:\n", "ceval_gil.h:243:9: warning: initialization makes integer from pointer without a cast [-Wint-conversion]\n", " _Py_atomic_store_relaxed(&gil_last_holder, tstate);\n", " ^~~~~~~~~~~~~~~~~~~~~~~~\n" ] } ], "source": [ "%%bash\n", "tmpdir=$(mktemp -d)\n", "cd $tmpdir\n", "git clone https://github.com/dutc/rwatch\n", "cd rwatch/src/\n", "make\n", "ls -larth ./rwatch.so\n", "file ./rwatch.so\n", "# cp ./rwatch.so /where/ver/you/need/ # ~/publis/notebook/ for me" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Anyhow, if `rwatch` is installed, we can import it, and it enables two new functions in the `sys` module:" ] }, { "cell_type": "code", "execution_count": 6, "metadata": { "collapsed": false }, "outputs": [ { "data": { "text/plain": [ "{}" ] }, "execution_count": 6, "metadata": {}, "output_type": "execute_result" } ], "source": [ "import rwatch\n", "from sys import setrwatch, getrwatch\n", "\n", "setrwatch({}) # clean any previously installed rwatch\n", "getrwatch()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Finally, we need the [`collections`](https://docs.python.org/3/library/collections.html) module and its [`defaultdict`](https://docs.python.org/3/library/collections.html#collections.defaultdict) magical datastructure." ] }, { "cell_type": "code", "execution_count": 7, "metadata": { "collapsed": true }, "outputs": [], "source": [ "from collections import defaultdict" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "----\n", "## Defining a debugging context manager, just to try\n", "\n", "This is the first example given in [James presentation]() at PyCon Canada 2016.\n", "\n", "We will first define and add a `rwatch` for just one object, let say a variable `x`, and then for any object using `defaultdict`.\n", "From there, writing a context manager that enables this feature only locally is easy." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Watching just one object\n", "1. Write the function, that needs two argument `frame, obj`, and should return `obj`,\n", "2. Install it...\n", "3. Check it!" ] }, { "cell_type": "code", "execution_count": 15, "metadata": { "collapsed": true }, "outputs": [], "source": [ "def basic_view(frame, obj):\n", " print(\"Python saw the object {} from frame {}\".format(obj, frame))\n", " return obj" ] }, { "cell_type": "code", "execution_count": 16, "metadata": { "collapsed": true }, "outputs": [], "source": [ "x = \"I am alive!\"" ] }, { "cell_type": "code", "execution_count": 17, "metadata": { "collapsed": true }, "outputs": [], "source": [ "setrwatch({\n", " id(x): basic_view\n", "})" ] }, { "cell_type": "code", "execution_count": 18, "metadata": { "collapsed": false, "scrolled": false }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Python saw the object I am alive! from frame \n", "Python saw the object I am alive! from frame \n", "I am alive!\n" ] } ], "source": [ "print(x)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "That's awesome, it works!" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Can we delete the `rwatch` ?\n", "Sure!" ] }, { "cell_type": "code", "execution_count": 19, "metadata": { "collapsed": true }, "outputs": [], "source": [ "def delrwatch(idobj):\n", " getrwatch().pop(idobj, None)" ] }, { "cell_type": "code", "execution_count": 20, "metadata": { "collapsed": false }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Python saw the object I am alive! from frame \n", "Python saw the object I am alive! from frame \n", "I am alive!\n", "Python saw the object I am alive! from frame \n", "I am alive!\n", "I am alive!\n" ] } ], "source": [ "print(x)\n", "delrwatch(id(x))\n", "print(x) # no more rwatch on this!\n", "print(x) # no more rwatch on this!" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "We can also delete rwatches that are not defined, without a failure:" ] }, { "cell_type": "code", "execution_count": 21, "metadata": { "collapsed": false }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "I am Zorro !\n", "I am Zorro !\n" ] } ], "source": [ "y = \"I am Zorro !\"\n", "print(y)\n", "delrwatch(y) # No issue!\n", "print(y)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### More useful debuggin information\n", "What is this `frame` thing?\n", "It is described in the documentation of the [`inspect`](https://docs.python.org/3/library/inspect.html) module.\n", "\n", "We can actually use it to display some useful information about the object and where was it called etc." ] }, { "cell_type": "code", "execution_count": 22, "metadata": { "collapsed": true }, "outputs": [], "source": [ "from inspect import getframeinfo\n", "\n", "def debug_view(frame, obj):\n", " info = getframeinfo(frame)\n", " msg = '- Access to {!r} (@{}) at {}:{}:{}'\n", " print(msg.format(obj, hex(id(obj)), info.filename, info.lineno, info.function))\n", " return obj" ] }, { "cell_type": "code", "execution_count": 23, "metadata": { "collapsed": false }, "outputs": [ { "data": { "text/plain": [ "{140613264650544: