{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "\n", "
\n", "Thierry Parmentelat,
Mohamed Naoufal Mahfoudi,
Thierry Turletti,
and Walid Dabbous,
Inria\n", "Licence CC BY-NC-ND\n", "
" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "# R2lab's radiomap" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Foreword on Jupyter notebooks" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### How to run a notebook" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "**For people not familiar with notebooks**, let us simply stress that:\n", "\n", "* in order to evaluate a code cell, you first **select it** (click in it), and then press ***Shift-Enter*** - or, equivalent, click the right arrow button in the menubar. The next cell gets selected, so you can essentially run you way through the document by selecting the first cell, and then pressing ***Shift-Enter*** until you're done.\n", "\n", "* you can also run the whole notebook in a single click - although this is not the recommended technique - from the menubar with ***Cell → Run All***" ] }, { "cell_type": "markdown", "metadata": { "run_control": { "frozen": false, "read_only": false } }, "source": [ "### Where to run notebooks" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "There are several ways to run a notebook:\n", "\n", "1. some public sites offer the ability to host notebooks; the present notebook for example \n", " \n", " can be run on mybinder.org\n", " \n", " \n", "1. you can also simply install jupyter on your own machine, and run the notebook from there; please refer to [Jupyter's installation instructions](http://jupyter.readthedocs.io/en/latest/install.html) for details." ] }, { "cell_type": "markdown", "metadata": { "run_control": { "frozen": false, "read_only": false } }, "source": [ "The method that you chose is important, because :\n", "\n", "* on the one hand, using a public infrastructure removes the burden of having to install anything, so you can start playing around right away;\n", "\n", "* however, your ssh private key is of course unreachable from mybinder or any other public infrastructure, so as far as actually triggering experiments on R2lab is concerned, you will not have the necessary credentials until you go for the second option. You can still use the visualisation tools though, as some pre-gathered data are part of the git repository." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "*****" ] }, { "cell_type": "markdown", "metadata": { "run_control": { "frozen": false, "read_only": false } }, "source": [ "## Purpose" ] }, { "cell_type": "markdown", "metadata": { "run_control": { "frozen": false, "read_only": false } }, "source": [ "R2lab's radiomap is a set of measurements that act as a calibration of the testbed.\n", "The goal is to measure and visualize **received power** at **all node locations** \n", "when a radio signal is sent **from any given sender** node.\n", "\n", "Additionally, that same experiment can be carried out with **various settings** \n", "for the emitted signal, like emission power, Tx rate, channel frequency, \n", "and with various antenna setups (single antenna, multiple antennas)." ] }, { "cell_type": "markdown", "metadata": { "run_control": { "frozen": false, "read_only": false } }, "source": [ "## Workflow" ] }, { "cell_type": "markdown", "metadata": { "run_control": { "frozen": false, "read_only": false } }, "source": [ "End to end experiment involves 2 successive stages:\n", "\n", "1. **data acquisition** : per se, including post-processing (aggregation); this can be carried out with the `acquiremap.py` python script, or interactively through the first part of the present notebook; this of course requires at the very least a reservation in the testbed, and as pointed out above is unfortunately not possible from a publicly hosted notebook;\n", "1. **visualization** : interactively, through the second part of this notebook.\n", "\n", "For convenience, this git repository also contains a directory `datasample` that contains one (partial) dataset obtained by running the first-stage acquisition script, so that visualization can be performed right away, as a way to give a quick sense of the results. " ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "# by default we want to use the pre-shipped data\n", "datadir = 'datasample'" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "*****" ] }, { "cell_type": "markdown", "metadata": { "run_control": { "frozen": false, "read_only": false } }, "source": [ "# Data acquisition with `acquiremap.py`" ] }, { "cell_type": "markdown", "metadata": { "run_control": { "frozen": false, "read_only": false } }, "source": [ "The àcquiremap.py`python script is designed to expose to the outside 3 levels of scenarios:\n", "\n", "* a `one_run` python function, that runs a complete set of measurements on all nodes, with a specific combination of environment settings (like transmission power, number of antennas, and similar)\n", "\n", "* a `all_runs` python function, that calls `one_run` with all possible values for the environment settings\n", "\n", "* the script itself, when invoked from the command-line, calls `all_runs` with the environment settings specified as command-line options.\n", "\n", "Additionally, all these functions can be instructed to perform node initializations (load a fresh image on all nodes, and turn off unused nodes). When this feature is turned on with in multiple-runs mode (be it from python or from the shell), nodes initialization is performed only once before the the first invokation of `one_run`." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "##### Digression " ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "As a side note, we recommend using the following trick: " ] }, { "cell_type": "code", "execution_count": 1, "metadata": { "run_control": { "frozen": false, "read_only": false } }, "outputs": [], "source": [ "# for convenience, we use this notebook extension\n", "# that will reload any imported python module\n", "# this is handy if you want to use a tex editor to\n", "# change the code in separate python files while you run the notebook\n", "%load_ext autoreload\n", "%autoreload 2\n", "import nest_asyncio\n", "nest_asyncio.apply()" ] }, { "cell_type": "markdown", "metadata": { "run_control": { "frozen": false, "read_only": false } }, "source": [ "## `one_run`" ] }, { "cell_type": "markdown", "metadata": { "run_control": { "frozen": false, "read_only": false } }, "source": [ "Back to data acquisition, the `one_run` python function performs data collection on a set of nodes (default is all nodes), for one given setting of the environment variables." ] }, { "cell_type": "code", "execution_count": null, "metadata": { "run_control": { "frozen": false, "read_only": false } }, "outputs": [], "source": [ "from acquiremap import one_run" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "run_control": { "frozen": false, "read_only": false } }, "outputs": [], "source": [ "one_run(wireless_driver, tx_power, phy_rate, antenna_mask, channel, run_name='myradiomap', slicename='inria_admin', load_images=False, node_ids=None, parallel=None, verbose_ssh=False, verbose_jobs=False, dry_run=False)" ] }, { "cell_type": "markdown", "metadata": { "run_control": { "frozen": false, "read_only": false } }, "source": [ "The `one_run` function implements a scenario in which each node sends a number of ping packets to every other node.\n", "\n", "In addition, all nodes run a tcpdump process, and at the end of the run, \n", "every pcap file (called `fit.pcap` at node `N`) is analyzed locally at node N to \n", "retrieve the RSSI values received from each other node on its antenna(s), and the result \n", "is stored in file `result-N.txt` (still on node N).\n", "\n", "When all nodes are done, the results are fetched from all nodes and centralized on this laptop in a directory called `t{}-r{}-a{}-ch{}` containing all the files retrieved from all nodes, i.e., `fitN.pcap`, `result-N.txt` for all N nodes.\n", "\n", "With:\n", "\n", "* `t{}` identifies Tx power for sender nodes in dBm (e.g. 5 to 14):\n", "* `r{}` identifies PHY Tx rate used on all nodes (1 to 54 Mbps)\n", "* `a{}` identifies the antenna mask on all nodes\n", "* `ch{}` denotes the WiFi channel used for transmission\n", "\n", "At that point, the post-processing function `processmap.py` is invoked \n", "to generate intermediate files `rssi-.txt` (one per node),\n", "and eventually one consolidated file `RSSI.txt`, that will be used to plot the radiomap." ] }, { "cell_type": "markdown", "metadata": { "run_control": { "frozen": false, "read_only": false } }, "source": [ "**IMPORTANT NOTE**: Both Atheros 93xx (with `ath9k` driver) and Intel 5300 (with `iwlwifi` driver) a/b/g/n NICs are now supported in these scripts. However, both cards do no have the same features and in particular, the `iwlwifi` driver limits the number of wireless stations in the same IBSS (parameter `IWLAGN_STATION_COUNT`) to a dozen! So, if you run the script with the 37 nodes, you will observe strange behaviors... Also the Intel 5300 cards are not allowed to use the 5GHz band in Ad Hoc mode. One good point is that it is possible with these cards to decrease the TX power to 0dBm (the lower bound for Atheros cards is 5dBm)." ] }, { "cell_type": "markdown", "metadata": { "run_control": { "frozen": false, "read_only": false } }, "source": [ "## `all_runs`" ] }, { "cell_type": "markdown", "metadata": { "run_control": { "frozen": false, "read_only": false } }, "source": [ "This function simply calls `one_run` with several combinations (a cartesian product) of environment settings." ] }, { "cell_type": "code", "execution_count": 2, "metadata": { "run_control": { "frozen": false, "read_only": false } }, "outputs": [], "source": [ "from acquiremap import all_runs" ] }, { "cell_type": "code", "execution_count": 3, "metadata": { "run_control": { "frozen": false, "read_only": false } }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Help on function all_runs in module acquiremap:\n", "\n", "all_runs(wireless_driver, tx_powers, phy_rates, antenna_masks, channels, *args, **kwds)\n", " calls one_run with the cartesian product of\n", " tx_powers, phy_rates, antenna_masks and channels, that are expected to\n", " be lists of strings\n", " \n", " All other arguments to one_run may/must be specified as well\n", " \n", " Example:\n", " all_runs([5, 14], [1], [1], [1, 40], ...)\n", " will call one_run exactly 4 times\n", "\n" ] } ], "source": [ "help(all_runs)" ] }, { "cell_type": "markdown", "metadata": { "run_control": { "frozen": false, "read_only": false } }, "source": [ "For example, instead of expecting a paramater `tx_power` that is a simple string, it expects parameter `tx_powers` that is a list of `tx_power` strings to consider. So for example\n", "\n", " all_runs(tx_powers=[5], phy_rates=[1, 54], antenna_masks=[3, 7])\n", " \n", "would result in 4 runs of `one_run` with the 2 possible values for `phy_rates` multiplied by the 2 possible values for `antenna_masks`" ] }, { "cell_type": "markdown", "metadata": { "run_control": { "frozen": false, "read_only": false } }, "source": [ "## Shell interface" ] }, { "cell_type": "markdown", "metadata": { "run_control": { "frozen": false, "read_only": false } }, "source": [ "The command-interface lets you essentially call `all_runs` directly from your shell" ] }, { "cell_type": "code", "execution_count": 4, "metadata": { "run_control": { "frozen": false, "read_only": false } }, "outputs": [], "source": [ "#!./acquiremap.py --help" ] }, { "cell_type": "markdown", "metadata": { "run_control": { "frozen": false, "read_only": false } }, "source": [ "So by default this simple shell script will run the scenario \n", "with all values for Tx power, PHY Tx rate and Antenna configurations. \n", "Book R2lab for at least 2 hours to run it." ] }, { "cell_type": "markdown", "metadata": { "run_control": { "frozen": false, "read_only": false } }, "source": [ "## Run your own" ] }, { "cell_type": "markdown", "metadata": { "run_control": { "frozen": false, "read_only": false } }, "source": [ "Any of the `one_run` or `all_runs` functions accept a keyword-only parameter named `run_name`, that specifies the name of the subdirectory where results will be stored.\n", "It defaults to `myradiomap` but you are encouraged \n", "to provide your own in order to isolate your results.\n", "\n", "Tweak the following cell to run your own data collection campaign:" ] }, { "cell_type": "code", "execution_count": 5, "metadata": { "run_control": { "frozen": false, "read_only": false } }, "outputs": [], "source": [ "# set this to True if you want to run your own data collection experiment\n", "\n", "use_my_data = True" ] }, { "cell_type": "code", "execution_count": 6, "metadata": { "run_control": { "frozen": false, "read_only": false } }, "outputs": [], "source": [ "# if that is the case you need to set these as well\n", "\n", "if use_my_data:\n", " # this plays the same role as 'datasample' -- see below\n", " # but to store your own data\n", " datadir = \"mymap-intel\"\n", " # enter here the name of the slice for which you have a valid reservation\n", " slicename = 'inria_admin'" ] }, { "cell_type": "code", "execution_count": 7, "metadata": { "run_control": { "frozen": false, "read_only": false } }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "13:47:24.056 SCHEDULER 1D + 0R + 152I = 153: Emergency exit upon exception in critical job\n" ] }, { "ename": "TypeError", "evalue": "prepare() got an unexpected keyword argument 'config'", "output_type": "error", "traceback": [ "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m", "\u001b[0;31mTypeError\u001b[0m Traceback (most recent call last)", "\u001b[0;32m/var/folders/6n/2x3z_fxs0pxcyh_g3bkjzfyr0000gn/T/ipykernel_26803/2447732765.py\u001b[0m in \u001b[0;36m\u001b[0;34m\u001b[0m\n\u001b[1;32m 3\u001b[0m all_runs(run_name=datadir, tx_powers=[0,15], phy_rates=[1,54], \n\u001b[1;32m 4\u001b[0m \u001b[0mantenna_masks\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;36m7\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mchannels\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;36m1\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mnode_ids\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;36m5\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;36m7\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;36m8\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;36m9\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;36m10\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;36m11\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;36m12\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;36m19\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;36m20\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;36m21\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;36m22\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;36m32\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;36m33\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;36m34\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;36m36\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m----> 5\u001b[0;31m wireless_driver='iwlwifi', load_images=True, slicename=slicename)\n\u001b[0m", "\u001b[0;32m~/fit-r2lab/r2lab-demos/radiomap/acquiremap.py\u001b[0m in \u001b[0;36mall_runs\u001b[0;34m(wireless_driver, tx_powers, phy_rates, antenna_masks, channels, *args, **kwds)\u001b[0m\n\u001b[1;32m 355\u001b[0m \u001b[0;31m# record any failure\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 356\u001b[0m if not one_run(wireless_driver, tx_power, phy_rate, antenna_mask,\n\u001b[0;32m--> 357\u001b[0;31m channel, *args, **kwds):\n\u001b[0m\u001b[1;32m 358\u001b[0m \u001b[0moverall\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0;32mFalse\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 359\u001b[0m \u001b[0;31m# make sure images will get loaded only once\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n", "\u001b[0;32m~/fit-r2lab/r2lab-demos/radiomap/acquiremap.py\u001b[0m in \u001b[0;36mone_run\u001b[0;34m(wireless_driver, tx_power, phy_rate, antenna_mask, channel, run_name, slicename, load_images, node_ids, parallel, verbose_ssh, verbose_jobs, dry_run)\u001b[0m\n\u001b[1;32m 312\u001b[0m \u001b[0;31m# if not in dry-run mode, let's proceed to the actual experiment\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 313\u001b[0m \u001b[0;31m# ok = scheduler.orchestrate(jobs_window=jobs_window)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 314\u001b[0;31m \u001b[0mok\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mscheduler\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0morchestrate\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m 315\u001b[0m \u001b[0;31m# give details if it failed\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 316\u001b[0m \u001b[0;32mif\u001b[0m \u001b[0;32mnot\u001b[0m \u001b[0mok\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n", "\u001b[0;32m~/miniconda3/envs/r2lab-demos/lib/python3.7/site-packages/asynciojobs/purescheduler.py\u001b[0m in \u001b[0;36mrun\u001b[0;34m(self, *args, **kwds)\u001b[0m\n\u001b[1;32m 750\u001b[0m \"\"\"\n\u001b[1;32m 751\u001b[0m \u001b[0mloop\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0masyncio\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mget_event_loop\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 752\u001b[0;31m \u001b[0;32mreturn\u001b[0m \u001b[0mloop\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mrun_until_complete\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mco_run\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m*\u001b[0m\u001b[0margs\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m**\u001b[0m\u001b[0mkwds\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m 753\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 754\u001b[0m \u001b[0;31m# define the alias for legacy\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n", "\u001b[0;32m~/miniconda3/envs/r2lab-demos/lib/python3.7/site-packages/nest_asyncio.py\u001b[0m in \u001b[0;36mrun_until_complete\u001b[0;34m(self, future)\u001b[0m\n\u001b[1;32m 87\u001b[0m raise RuntimeError(\n\u001b[1;32m 88\u001b[0m 'Event loop stopped before Future completed.')\n\u001b[0;32m---> 89\u001b[0;31m \u001b[0;32mreturn\u001b[0m \u001b[0mf\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mresult\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m 90\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 91\u001b[0m \u001b[0;32mdef\u001b[0m \u001b[0m_run_once\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n", "\u001b[0;32m~/miniconda3/envs/r2lab-demos/lib/python3.7/asyncio/futures.py\u001b[0m in \u001b[0;36mresult\u001b[0;34m(self)\u001b[0m\n\u001b[1;32m 176\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m__log_traceback\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0;32mFalse\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 177\u001b[0m \u001b[0;32mif\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_exception\u001b[0m \u001b[0;32mis\u001b[0m \u001b[0;32mnot\u001b[0m \u001b[0;32mNone\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 178\u001b[0;31m \u001b[0;32mraise\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_exception\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m 179\u001b[0m \u001b[0;32mreturn\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_result\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 180\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n", "\u001b[0;32m~/miniconda3/envs/r2lab-demos/lib/python3.7/asyncio/tasks.py\u001b[0m in \u001b[0;36m__step\u001b[0;34m(***failed resolving arguments***)\u001b[0m\n\u001b[1;32m 247\u001b[0m \u001b[0;31m# We use the `send` method directly, because coroutines\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 248\u001b[0m \u001b[0;31m# don't have `__iter__` and `__next__` methods.\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 249\u001b[0;31m \u001b[0mresult\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mcoro\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0msend\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;32mNone\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m 250\u001b[0m \u001b[0;32melse\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 251\u001b[0m \u001b[0mresult\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mcoro\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mthrow\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mexc\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n", "\u001b[0;32m~/miniconda3/envs/r2lab-demos/lib/python3.7/site-packages/asynciojobs/scheduler.py\u001b[0m in \u001b[0;36mco_run\u001b[0;34m(self)\u001b[0m\n\u001b[1;32m 132\u001b[0m \u001b[0mexc\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mjob\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mraised_exception\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 133\u001b[0m \u001b[0;32mif\u001b[0m \u001b[0mexc\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 134\u001b[0;31m \u001b[0;32mraise\u001b[0m \u001b[0mexc\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m 135\u001b[0m \u001b[0;31m# we should not reach this point\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 136\u001b[0m \u001b[0;32mraise\u001b[0m \u001b[0mValueError\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m\"Internal error in Scheduler.co_run()\"\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n", "\u001b[0;32m~/miniconda3/envs/r2lab-demos/lib/python3.7/asyncio/tasks.py\u001b[0m in \u001b[0;36m__step\u001b[0;34m(***failed resolving arguments***)\u001b[0m\n\u001b[1;32m 247\u001b[0m \u001b[0;31m# We use the `send` method directly, because coroutines\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 248\u001b[0m \u001b[0;31m# don't have `__iter__` and `__next__` methods.\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 249\u001b[0;31m \u001b[0mresult\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mcoro\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0msend\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;32mNone\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m 250\u001b[0m \u001b[0;32melse\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 251\u001b[0m \u001b[0mresult\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mcoro\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mthrow\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mexc\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n", "\u001b[0;32m~/miniconda3/envs/r2lab-demos/lib/python3.7/site-packages/asynciojobs/window.py\u001b[0m in \u001b[0;36mwrapped\u001b[0;34m()\u001b[0m\n\u001b[1;32m 36\u001b[0m \u001b[0;32mawait\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mqueue\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mput\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;36m1\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 37\u001b[0m \u001b[0mjob\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_running\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0;32mTrue\u001b[0m \u001b[0;31m# pylint: disable=w0212\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m---> 38\u001b[0;31m \u001b[0mvalue\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0;32mawait\u001b[0m \u001b[0mjob\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mco_run\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m 39\u001b[0m \u001b[0;31m# release slot in the queue\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 40\u001b[0m \u001b[0;32mawait\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mqueue\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mget\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n", "\u001b[0;32m~/miniconda3/envs/r2lab-demos/lib/python3.7/site-packages/apssh/sshjob.py\u001b[0m in \u001b[0;36mco_run\u001b[0;34m(self)\u001b[0m\n\u001b[1;32m 218\u001b[0m \u001b[0mresult\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0;32mawait\u001b[0m \u001b[0mcommand\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mco_run_local\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mnode\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 219\u001b[0m \u001b[0;32melse\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 220\u001b[0;31m \u001b[0mresult\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0;32mawait\u001b[0m \u001b[0mcommand\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mco_run_remote\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mnode\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m 221\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 222\u001b[0m \u001b[0;31m# has the command failed ?\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n", "\u001b[0;32m~/miniconda3/envs/r2lab-demos/lib/python3.7/site-packages/apssh/commands.py\u001b[0m in \u001b[0;36mco_run_remote\u001b[0;34m(self, node)\u001b[0m\n\u001b[1;32m 234\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_verbose_message\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mnode\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m\"Run: -> {}\"\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mformat\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mcommand\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 235\u001b[0m \u001b[0;31m# need an ssh connection\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 236\u001b[0;31m \u001b[0mconnected\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0;32mawait\u001b[0m \u001b[0mnode\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mconnect_lazy\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m 237\u001b[0m \u001b[0;32mif\u001b[0m \u001b[0;32mnot\u001b[0m \u001b[0mconnected\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 238\u001b[0m \u001b[0;32mreturn\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n", "\u001b[0;32m~/miniconda3/envs/r2lab-demos/lib/python3.7/site-packages/apssh/sshproxy.py\u001b[0m in \u001b[0;36mconnect_lazy\u001b[0;34m(self)\u001b[0m\n\u001b[1;32m 252\u001b[0m \u001b[0;32masync\u001b[0m \u001b[0;32mwith\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_connect_lock\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 253\u001b[0m \u001b[0;32mif\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mconn\u001b[0m \u001b[0;32mis\u001b[0m \u001b[0;32mNone\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 254\u001b[0;31m \u001b[0;32mawait\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_connect\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m 255\u001b[0m \u001b[0;32mreturn\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mconn\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 256\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n", "\u001b[0;32m~/miniconda3/envs/r2lab-demos/lib/python3.7/site-packages/apssh/sshproxy.py\u001b[0m in \u001b[0;36m_connect\u001b[0;34m(self)\u001b[0m\n\u001b[1;32m 261\u001b[0m \u001b[0;32mif\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mgateway\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 262\u001b[0m \u001b[0;32mreturn\u001b[0m \u001b[0;32mawait\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_connect_tunnel\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 263\u001b[0;31m \u001b[0;32mreturn\u001b[0m \u001b[0;32mawait\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_connect_direct\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m 264\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 265\u001b[0m \u001b[0;32masync\u001b[0m \u001b[0;32mdef\u001b[0m \u001b[0m_connect_direct\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n", "\u001b[0;32m~/miniconda3/envs/r2lab-demos/lib/python3.7/site-packages/apssh/sshproxy.py\u001b[0m in \u001b[0;36m_connect_direct\u001b[0;34m(self)\u001b[0m\n\u001b[1;32m 289\u001b[0m \u001b[0mconfig\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;32mNone\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 290\u001b[0m ),\n\u001b[0;32m--> 291\u001b[0;31m timeout=self.timeout)\n\u001b[0m\u001b[1;32m 292\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 293\u001b[0m \u001b[0;32masync\u001b[0m \u001b[0;32mdef\u001b[0m \u001b[0m_connect_tunnel\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n", "\u001b[0;32m~/miniconda3/envs/r2lab-demos/lib/python3.7/asyncio/tasks.py\u001b[0m in \u001b[0;36mwait_for\u001b[0;34m(fut, timeout, loop)\u001b[0m\n\u001b[1;32m 440\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 441\u001b[0m \u001b[0;32mif\u001b[0m \u001b[0mfut\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mdone\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 442\u001b[0;31m \u001b[0;32mreturn\u001b[0m \u001b[0mfut\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mresult\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m 443\u001b[0m \u001b[0;32melse\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 444\u001b[0m \u001b[0mfut\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mremove_done_callback\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mcb\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n", "\u001b[0;32m~/miniconda3/envs/r2lab-demos/lib/python3.7/asyncio/futures.py\u001b[0m in \u001b[0;36mresult\u001b[0;34m(self)\u001b[0m\n\u001b[1;32m 176\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m__log_traceback\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0;32mFalse\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 177\u001b[0m \u001b[0;32mif\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_exception\u001b[0m \u001b[0;32mis\u001b[0m \u001b[0;32mnot\u001b[0m \u001b[0;32mNone\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 178\u001b[0;31m \u001b[0;32mraise\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_exception\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m 179\u001b[0m \u001b[0;32mreturn\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_result\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 180\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n", "\u001b[0;32m~/miniconda3/envs/r2lab-demos/lib/python3.7/asyncio/tasks.py\u001b[0m in \u001b[0;36m__step\u001b[0;34m(***failed resolving arguments***)\u001b[0m\n\u001b[1;32m 247\u001b[0m \u001b[0;31m# We use the `send` method directly, because coroutines\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 248\u001b[0m \u001b[0;31m# don't have `__iter__` and `__next__` methods.\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 249\u001b[0;31m \u001b[0mresult\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mcoro\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0msend\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;32mNone\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m 250\u001b[0m \u001b[0;32melse\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 251\u001b[0m \u001b[0mresult\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mcoro\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mthrow\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mexc\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n", "\u001b[0;32m~/miniconda3/envs/r2lab-demos/lib/python3.7/site-packages/asyncssh/connection.py\u001b[0m in \u001b[0;36mcreate_connection\u001b[0;34m(client_factory, host, port, **kwargs)\u001b[0m\n\u001b[1;32m 5893\u001b[0m \"\"\"\n\u001b[1;32m 5894\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m-> 5895\u001b[0;31m \u001b[0mconn\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0;32mawait\u001b[0m \u001b[0mconnect\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mhost\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mport\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mclient_factory\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mclient_factory\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m**\u001b[0m\u001b[0mkwargs\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m 5896\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 5897\u001b[0m \u001b[0;32mreturn\u001b[0m \u001b[0mconn\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mconn\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mget_owner\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n", "\u001b[0;32m~/miniconda3/envs/r2lab-demos/lib/python3.7/site-packages/asyncssh/connection.py\u001b[0m in \u001b[0;36mconnect\u001b[0;34m(host, port, tunnel, family, flags, local_addr, options, **kwargs)\u001b[0m\n\u001b[1;32m 5632\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 5633\u001b[0m \u001b[0;32mif\u001b[0m \u001b[0;32mnot\u001b[0m \u001b[0moptions\u001b[0m \u001b[0;32mor\u001b[0m \u001b[0mkwargs\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m-> 5634\u001b[0;31m \u001b[0moptions\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mSSHClientConnectionOptions\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0moptions\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m**\u001b[0m\u001b[0mkwargs\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m 5635\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 5636\u001b[0m return await _connect(host, port, loop, tunnel, family, flags, local_addr,\n", "\u001b[0;32m~/miniconda3/envs/r2lab-demos/lib/python3.7/site-packages/asyncssh/misc.py\u001b[0m in \u001b[0;36m__init__\u001b[0;34m(self, options, **kwargs)\u001b[0m\n\u001b[1;32m 210\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 211\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mkwargs\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mupdate\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mkwargs\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 212\u001b[0;31m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mprepare\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m**\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mkwargs\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m 213\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 214\u001b[0m \u001b[0;32mdef\u001b[0m \u001b[0mprepare\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n", "\u001b[0;31mTypeError\u001b[0m: prepare() got an unexpected keyword argument 'config'" ] } ], "source": [ "# run the data collection\n", "if use_my_data:\n", " all_runs(run_name=datadir, tx_powers=[0,15], phy_rates=[1,54], \n", " antenna_masks=[7], channels=[1], node_ids=[5,7,8,9,10,11,12,19,20,21,22,32,33,34,36],\n", " wireless_driver='iwlwifi', load_images=True, slicename=slicename)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "*****" ] }, { "cell_type": "markdown", "metadata": { "run_control": { "frozen": false, "read_only": false } }, "source": [ "# Plotting R2lab Radio-Maps" ] }, { "cell_type": "markdown", "metadata": { "run_control": { "frozen": false, "read_only": false } }, "source": [ "## Data naming scheme" ] }, { "cell_type": "markdown", "metadata": { "run_control": { "frozen": false, "read_only": false } }, "source": [ "The git repository comes with a pre-populated dataset collected by us, in the `datasample` directory. This contains all the RSSI information to run this visualization.\n", "\n", "If you have collected RSSI data, you will be able to " ] }, { "cell_type": "markdown", "metadata": { "run_control": { "frozen": false, "read_only": false } }, "source": [ "The `datasample` directory contains a collection of `RSSI.txt` files in the following subdirectories:\n", "\n", "* datasample/t14-r1-a1-ch1/RSSI.txt\n", "* datasample/t14-r1-a3-ch1/RSSI.txt\n", "* datasample/t14-r1-a7-ch1/RSSI.txt\n", "* datasample/t14-r54-a1-ch1/RSSI.txt\n", "* datasample/t14-r54-a3-ch1/RSSI.txt\n", "* datasample/t14-r54-a7-ch1/RSSI.txt\n", "* datasample/t5-r1-a1-ch1/RSSI.txt\n", "* datasample/t5-r1-a3-ch1/RSSI.txt\n", "* datasample/t5-r1-a7-ch1/RSSI.txt\n", "* datasample/t5-r54-a1-ch1/RSSI.txt\n", "* datasample/t5-r54-a3-ch1/RSSI.txt\n", "* datasample/t5-r54-a7-ch1/RSSI.txt\n", "\n", "Where\n", "\n", "* `t5` means an emission power of 5dBm\n", "\n", "* `r1` means PHY rate=1Mbps\n", "\n", "* `a1` means 1 single valid antenna - and so 2 values in RSSI.txt\n", "* `a3` means 2 valid antennas - and so 3 values in RSSI.txt\n", "* `a7` means 3 valid antennas - and so 4 value in RSSI.txt\n", "\n", "* `ch1` means channel 1, i.e. 2412 MHz frequency" ] }, { "cell_type": "markdown", "metadata": { "run_control": { "frozen": false, "read_only": false } }, "source": [ "This naming scheme is implemented in a helper function in `acquiremap`:" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "run_control": { "frozen": false, "read_only": false } }, "outputs": [], "source": [ "from acquiremap import naming_scheme" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "run_control": { "frozen": false, "read_only": false } }, "outputs": [], "source": [ "# here's what the naming scheme looks like\n", "naming_scheme('example-run', tx_power=5, phy_rate=1, antenna_mask=3, channel=40)" ] }, { "cell_type": "markdown", "metadata": { "run_control": { "frozen": false, "read_only": false } }, "source": [ "## Preparation" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "#### Interactive notebook" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "run_control": { "frozen": false, "read_only": false } }, "outputs": [], "source": [ "# interactive_output is used to refresh\n", "# a visualization based on user input\n", "from ipywidgets import interactive_output, fixed" ] }, { "cell_type": "markdown", "metadata": { "run_control": { "frozen": false, "read_only": false } }, "source": [ "#### Importing plotly" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "run_control": { "frozen": false, "read_only": false } }, "outputs": [], "source": [ "# plotly is one visualization library\n", "import plotly\n", "import plotly.plotly as py\n", "import plotly.graph_objs as go" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "run_control": { "frozen": false, "read_only": false } }, "outputs": [], "source": [ "# using plotly in offline mode is a requirement \n", "# in interactive mode - too slow otherwise\n", "import plotly.offline as pyoff\n", "pyoff.init_notebook_mode()" ] }, { "cell_type": "markdown", "metadata": { "run_control": { "frozen": false, "read_only": false } }, "source": [ "#### Importing a few utilities to retrieve RSSI data from files and convert it to arrays" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "In order to leave low-details out of the notebook, we have chosen to ship such small matters in separate files as python modules, right in this directory." ] }, { "cell_type": "code", "execution_count": null, "metadata": { "run_control": { "frozen": false, "read_only": false } }, "outputs": [], "source": [ "# to extract data from a RSSI file\n", "from rssi import read_rssi" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "### R2lab common utilities\n", "# a mapping node -> gridx, gridy\n", "from r2lab import R2labMap\n", "# how to initialize a dataframe\n", "from r2lab import MapDataFrame" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "#### Importing the dashboard" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "run_control": { "frozen": false, "read_only": false } }, "outputs": [], "source": [ "# this module deals with gory details of ipywidgets\n", "# its only purpose is to to come up with some \n", "# reasonably compact layout for our control buttons\n", "from dashboard import dashboard" ] }, { "cell_type": "markdown", "metadata": { "run_control": { "frozen": false, "read_only": false } }, "source": [ "*****" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## 2D heatmaps using `bokeh` heatmaps" ] }, { "cell_type": "markdown", "metadata": { "run_control": { "frozen": false, "read_only": false } }, "source": [ "With this in place, we can define a first visualization angle that relies on a 2D representation using `bokeh`.\n", "This approach gives IMHO better results than with `plotly` - that we will see next - as we are able to *alter* the figure when a change is made via the dashboard, instead of having to *redraw* it, which leads to a flickering effect." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "# just in case we want to start running from here\n", "try:\n", " # is this variable defined ?\n", " datadir\n", "except:\n", " # if not let's use the data that ships with the git repo\n", " datadir = 'datasample'" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "#### Importing bokeh" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "# again just in case\n", "from ipywidgets import interactive_output, fixed\n", "from dashboard import dashboard\n", "from acquiremap import naming_scheme\n", "from rssi import read_rssi" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "from bokeh.plotting import figure, show\n", "from bokeh.io import output_notebook, push_notebook\n", "\n", "from bokeh.models import ColorBar, ColumnDataSource, LinearColorMapper\n", "\n", "output_notebook()" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "run_control": { "frozen": false, "read_only": false } }, "outputs": [], "source": [ "# convert data into x, y and z\n", "from rssi import rssi_to_heatmap" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "# from https://bokeh.pydata.org/en/latest/docs/gallery/unemployment.html\n", "# this is the colormap from the original NYTimes plot\n", "# \n", "# xxx this colormap does not reflect our use case very well and needs more work\n", "# 0 means very strong power, while -100 means we can hardly notice the signal\n", "# \n", "colors = [\"#75968f\", \"#a5bab7\", \"#c9d9d3\", \"#e2e2e2\", \"#dfccce\", \"#ddb7b1\", \"#cc7878\", \"#933b41\", \"#550b1d\"]\n", "colormapper = LinearColorMapper(palette=colors, low=-100, high=0.)" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "# one global static object is good enough\n", "r2labmap = R2labMap()\n", "\n", "def init_bokeh():\n", " \n", " df = MapDataFrame(r2labmap, {'rssi': 0.})\n", " cds = ColumnDataSource(df) \n", "\n", " tools = \"hover,save,pan,box_zoom,reset,wheel_zoom\"\n", " fig = figure(\n", " title = 'bokeh-based R2lab radiomap',\n", " plot_width = 900, plot_height=500,\n", " tools = tools, toolbar_location = 'right'\n", " )\n", " # create the rectangles that make the heatmap\n", " fig.rect(x='x', y='y', width=1, height=1,\n", " fill_color={'field':'rssi',\n", " 'transform':colormapper,\n", " },\n", " source = cds)\n", " # show the figure, return handle for updates\n", " handle = show(fig, notebook_handle=True)\n", " \n", " # return as a tuple:\n", " # * dataframe is where animation will publish updates \n", " # * handle is an internal bokeh object needed to actually\n", " # redisplay the changes\n", " return df, cds, handle" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "run_control": { "frozen": false, "read_only": false } }, "outputs": [], "source": [ "def update_bokeh(datadir, sender, power, rate, antenna_mask, channel, rssi_rank):\n", " global dataframe, cds, handle\n", " # locate corresponding data file\n", " filename = str(naming_scheme(datadir, power, rate, antenna_mask, channel) / \"RSSI.txt\")\n", " # read that file\n", " rssi_dict = read_rssi(filename, sender, rssi_rank)\n", " if not rssi_dict:\n", " return\n", "\n", " for node_id, rssi in rssi_dict.items():\n", " dataframe.loc[node_id]['rssi'] = rssi\n", " \n", " cds.data = cds.from_df(dataframe)\n", " push_notebook(handle)" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "run_control": { "frozen": false, "read_only": false } }, "outputs": [], "source": [ "# create data and figure\n", "dataframe, cds, handle = init_bokeh()\n", "\n", "# interactively update it with an UI\n", "interactive_output(update_bokeh, dashboard(datadir, continuous_sender=True))" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "# debug\n", "# dataframe" ] }, { "cell_type": "markdown", "metadata": { "run_control": { "frozen": false, "read_only": false } }, "source": [ "*****" ] }, { "cell_type": "markdown", "metadata": { "run_control": { "frozen": false, "read_only": false } }, "source": [ "## 2D radiomaps using `plotly` heatmaps" ] }, { "cell_type": "markdown", "metadata": { "run_control": { "frozen": false, "read_only": false } }, "source": [ "Using `plotly` tends to a simpler code, at the cost of a less pleasant visual effect when the dashboard is used to tweak an experimental setting.\n", "\n", "Here we leverage `plotly`'s `Heatmap` tool:" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "run_control": { "frozen": false, "read_only": false } }, "outputs": [], "source": [ "def radiomap2D(datadir, sender, power, rate, antenna_mask, channel, rssi_rank):\n", " # locate corresponding data file\n", " filename = str(naming_scheme(datadir, power, rate, antenna_mask, channel) / \"RSSI.txt\")\n", " # read that file\n", " rssi_dict = read_rssi(filename, sender, rssi_rank)\n", " if not rssi_dict:\n", " return\n", "\n", " X, Y, Z, T = rssi_to_heatmap(rssi_dict)\n", " heatmap = go.Heatmap( x=X, y=Y, z=Z, text=T, \n", " zmin=-100, zmax=0, zauto=False, opacity=1)\n", " axis = [heatmap]\n", " layout = go.Layout(\n", " title=\"R2lab Radio-Map: Rx power (in dBm) when fit{:02d} is transmitting
from {}\"\n", " .format(sender, filename))\n", " figure = go.Figure(data = axis, layout=layout)\n", " pyoff.iplot(figure)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "**NOTE: you need to play with at least one control for the figure to actually show up**" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "run_control": { "frozen": false, "read_only": false } }, "outputs": [], "source": [ "# interactively call radiomap2D\n", "interactive_output(radiomap2D, dashboard(datadir))" ] }, { "cell_type": "markdown", "metadata": { "run_control": { "frozen": false, "read_only": false } }, "source": [ "*****" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### 3D surface using ipyvolume" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "import ipyvolume as ipv" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "from rssi import rssi_to_3d" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "# colors: see https://ipyvolume.readthedocs.io/en/latest/mesh.html#Colors\n", "from matplotlib import cm\n", "\n", "def colormap(Z):\n", " colormap = cm.coolwarm\n", " znorm = Z - Z.min()\n", " znorm /= znorm.ptp()\n", " znorm.min(), znorm.max()\n", " color = colormap(znorm)\n", " # just doing this magic as-is from the above link\n", " return color[...,:3]" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "run_control": { "frozen": false, "read_only": false } }, "outputs": [], "source": [ "def radiomap3Dipv(datadir, sender, power, rate, antenna_mask, channel, rssi_rank):\n", " # locate corresponding data file\n", " filename = str(naming_scheme(datadir, power, rate, antenna_mask, channel) / \"RSSI.txt\")\n", " # read that file\n", " rssi_dict = read_rssi(filename, sender, rssi_rank)\n", " if not rssi_dict:\n", " return\n", "\n", " X, Y, Z, T = rssi_to_3d(rssi_dict)\n", " ipv.figure()\n", " ipv.plot_surface(X, Y, Z, color=colormap(Z))\n", " ipv.show()" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "interactive_output(radiomap3Dipv, dashboard(datadir))" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### 3D with plotly (broken)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Alternatively, we could use a 3D view based on `plotly` to explore the same results; here is what you would see if you used plotly's Surface object instead. " ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "def radiomap3D(datadir, sender, power, rate, antenna_mask, channel, rssi_rank):\n", " # locate corresponding data file\n", " filename = str(naming_scheme(datadir, power, rate, antenna_mask, channel) / \"RSSI.txt\")\n", " # read that file\n", " rssi_dict = read_rssi(filename, sender, rssi_rank)\n", " if not rssi_dict:\n", " return\n", "\n", " X, Y, Z, T = rssi_to_3d(rssi_dict)\n", " surface = go.Surface(x=X, y=Y, z=Z, text=T, zmin=-100, zmax=0,\n", " #connectgaps=False,\n", " )\n", " data = go.Data([surface])\n", " axis = dict(\n", " showbackground=True, # show axis background \n", " backgroundcolor=\"rgb(204, 204, 204)\", # set background color to grey \n", " gridcolor=\"rgb(255, 255, 255)\", # set grid line color \n", " zerolinecolor=\"rgb(255, 255, 255)\", # set zero grid line color \n", " )\n", " title = \"R2lab Radio-Map: Rx power (in dBm)\"\\\n", " \"when fit{:02d} is transmitting
from {}\"\\\n", " .format(sender, filename)\n", " layout = go.Layout(\n", " autosize=True,\n", " title = title,\n", " scene=go.Scene( \n", " xaxis=go.XAxis(axis), # set x-axis style \n", " yaxis=go.YAxis(axis), # set y-axis style \n", " zaxis=go.ZAxis(axis, title=\"RSSI (dBm) \") # set z-axis style \n", " )\n", " )\n", " \n", " figure = go.Figure(data=data, layout=layout)\n", " pyoff.iplot(figure)\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "**NOTE: again, you need to play with at least one control for the figure to actually show.**\n", "\n", "**NOTE2:** somehow this seems to have broken recently, it looks like `Surface` won't accept the `zmin` and `zmax` parameters any more.." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "# interactively call radiomap3D\n", "interactive_output(radiomap3D, dashboard(datadir))" ] }, { "cell_type": "markdown", "metadata": { "run_control": { "frozen": false, "read_only": false } }, "source": [ "*****" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "# Conclusion" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Of course this experiment has no scientific value in itself. On the contrary it has been chosen on purpose to be as straightforward as possible, so that we can exclusively focus on the matter of reproducibility. \n", "\n", "Hopefully we have illustrated a possible path for structuring research artifacts, with the objective of maximizing reproducibility, at least in the context of using the R2lab testbed.\n", "\n", "Additionally, the `acquiremap.py` script showcases a real-scale use of `nepi-ng` for orchestrating this sort of experimentation." ] } ], "metadata": { "kernelspec": { "display_name": "Python 3 (ipykernel)", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.7.5" }, "notebookname": "Purpose", "toc": { "colors": { "hover_highlight": "#DAA520", "running_highlight": "#FF0000", "selected_highlight": "#FFD700" }, "moveMenuLeft": true, "nav_menu": { "height": "12px", "width": "252px" }, "navigate_menu": true, "number_sections": true, "sideBar": true, "threshold": "2", "toc_cell": false, "toc_section_display": "block", "toc_window_display": false, "widenNotebook": false }, "version": "1.0" }, "nbformat": 4, "nbformat_minor": 2 }