{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "# Application 1: ProvStore Documents\n", "Identifying owners of provenance documents from their provenance network metrics\n", "\n", "* **Goal**: To determine if the provenance network analytics method can identify the owner of a provenance document from its provenance network metrics.\n", "* **Training data**: In order to ensure that there are sufficient samples to represent a user's provenance documents the Training phase, we limit our experiment to users who have at least 20 documents. There are fourteen such users (the authors were excluded to avoid bias), who we named $u_{1}, u_{2}, \\ldots, u_{14}$. Their numbers of documents range between 21 and 6,745, with the total number of documents in the data set is 13,870.\n", "* **Classification labels**: $\\mathcal{L} = \\left\\{ u_1, u_2, \\ldots, u_{14} \\right\\} $, where $l_{x} = u_i$ if the provenance document $x$ belongs to user $u_i$. Hence, there are 14 labels in total.\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Reading data\n", "For each provenance document, we calculate the 22 provenance network metrics. The dataset provided contains those metrics values for 13,870 provenance documents along with the owner identifier (i.e. $u_{1}, u_{2}, \\ldots, u_{14}$)." ] }, { "cell_type": "code", "execution_count": 1, "metadata": { "collapsed": true }, "outputs": [], "source": [ "import pandas as pd" ] }, { "cell_type": "code", "execution_count": 2, "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
labelentitiesagentsactivitiesnodesedgesdiameterassortativityaccacc_e...mfd_e_amfd_e_agmfd_a_emfd_a_amfd_a_agmfd_ag_emfd_ag_amfd_ag_agmfd_derpowerlaw_alpha
0u_3175931496-0.1963620.4447090.466667...584250003-1.0
1u_270290-1-1.0000000.0000000.000000...00000000-1-1.0
2u_270290-1-1.0000000.0000000.000000...00000000-1-1.0
3u_270290-1-1.0000000.0000000.000000...00000000-1-1.0
4u_270290-1-1.0000000.0000000.000000...00000000-1-1.0
\n", "

5 rows × 23 columns

\n", "
" ], "text/plain": [ " label entities agents activities nodes edges diameter assortativity \\\n", "0 u_3 17 5 9 31 49 6 -0.196362 \n", "1 u_2 7 0 2 9 0 -1 -1.000000 \n", "2 u_2 7 0 2 9 0 -1 -1.000000 \n", "3 u_2 7 0 2 9 0 -1 -1.000000 \n", "4 u_2 7 0 2 9 0 -1 -1.000000 \n", "\n", " acc acc_e ... mfd_e_a mfd_e_ag mfd_a_e mfd_a_a \\\n", "0 0.444709 0.466667 ... 5 8 4 2 \n", "1 0.000000 0.000000 ... 0 0 0 0 \n", "2 0.000000 0.000000 ... 0 0 0 0 \n", "3 0.000000 0.000000 ... 0 0 0 0 \n", "4 0.000000 0.000000 ... 0 0 0 0 \n", "\n", " mfd_a_ag mfd_ag_e mfd_ag_a mfd_ag_ag mfd_der powerlaw_alpha \n", "0 5 0 0 0 3 -1.0 \n", "1 0 0 0 0 -1 -1.0 \n", "2 0 0 0 0 -1 -1.0 \n", "3 0 0 0 0 -1 -1.0 \n", "4 0 0 0 0 -1 -1.0 \n", "\n", "[5 rows x 23 columns]" ] }, "execution_count": 2, "metadata": {}, "output_type": "execute_result" } ], "source": [ "df = pd.read_csv(\"provstore/data.csv\")\n", "df.head()" ] }, { "cell_type": "code", "execution_count": 3, "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
entitiesagentsactivitiesnodesedgesdiameterassortativityaccacc_eacc_a...mfd_e_amfd_e_agmfd_a_emfd_a_amfd_a_agmfd_ag_emfd_ag_amfd_ag_agmfd_derpowerlaw_alpha
count13870.00000013870.0000013870.00000013870.00000013870.00000013870.00000013870.00000013870.00000013870.00000013870.000000...13870.00000013870.00000013870.00000013870.00000013870.00000013870.00000013870.00000013870.00000013870.00000013870.000000
mean9.9133382.086951.83619313.83648219.2126890.868926-0.6286900.3478350.3411420.323606...1.3127611.7549391.0735400.7092290.7521270.0174480.0149240.0303532.185436-0.916534
std28.9319152.2771618.57082343.352894134.6403661.9439050.3767180.3945310.4095770.395727...1.7693291.3148741.6226061.3433631.0776280.2009020.1523510.2097595.2111180.612437
min0.0000000.000000.0000001.0000000.000000-1.000000-1.0000000.0000000.0000000.000000...0.0000000.0000000.0000000.0000000.0000000.0000000.0000000.000000-1.000000-1.000000
25%2.0000001.000000.0000005.0000005.000000-1.000000-1.0000000.0000000.0000000.000000...0.0000001.0000000.0000000.0000000.0000000.0000000.0000000.0000001.000000-1.000000
50%4.0000001.000001.0000007.0000009.0000001.000000-0.5929490.0000000.0000000.000000...1.0000002.0000000.0000000.0000001.0000000.0000000.0000000.0000002.000000-1.000000
75%5.0000003.000002.00000010.00000013.0000002.000000-0.3500000.6741470.7500000.666667...2.0000002.0000002.0000001.0000001.0000000.0000000.0000000.0000002.000000-1.000000
max1188.00000051.000001580.0000002776.0000006853.00000010.0000001.0000001.0000001.0000001.000000...52.00000044.00000051.00000052.00000043.0000004.0000005.0000006.000000303.0000008.184413
\n", "

8 rows × 22 columns

\n", "
" ], "text/plain": [ " entities agents activities nodes edges \\\n", "count 13870.000000 13870.00000 13870.000000 13870.000000 13870.000000 \n", "mean 9.913338 2.08695 1.836193 13.836482 19.212689 \n", "std 28.931915 2.27716 18.570823 43.352894 134.640366 \n", "min 0.000000 0.00000 0.000000 1.000000 0.000000 \n", "25% 2.000000 1.00000 0.000000 5.000000 5.000000 \n", "50% 4.000000 1.00000 1.000000 7.000000 9.000000 \n", "75% 5.000000 3.00000 2.000000 10.000000 13.000000 \n", "max 1188.000000 51.00000 1580.000000 2776.000000 6853.000000 \n", "\n", " diameter assortativity acc acc_e acc_a \\\n", "count 13870.000000 13870.000000 13870.000000 13870.000000 13870.000000 \n", "mean 0.868926 -0.628690 0.347835 0.341142 0.323606 \n", "std 1.943905 0.376718 0.394531 0.409577 0.395727 \n", "min -1.000000 -1.000000 0.000000 0.000000 0.000000 \n", "25% -1.000000 -1.000000 0.000000 0.000000 0.000000 \n", "50% 1.000000 -0.592949 0.000000 0.000000 0.000000 \n", "75% 2.000000 -0.350000 0.674147 0.750000 0.666667 \n", "max 10.000000 1.000000 1.000000 1.000000 1.000000 \n", "\n", " ... mfd_e_a mfd_e_ag mfd_a_e mfd_a_a \\\n", "count ... 13870.000000 13870.000000 13870.000000 13870.000000 \n", "mean ... 1.312761 1.754939 1.073540 0.709229 \n", "std ... 1.769329 1.314874 1.622606 1.343363 \n", "min ... 0.000000 0.000000 0.000000 0.000000 \n", "25% ... 0.000000 1.000000 0.000000 0.000000 \n", "50% ... 1.000000 2.000000 0.000000 0.000000 \n", "75% ... 2.000000 2.000000 2.000000 1.000000 \n", "max ... 52.000000 44.000000 51.000000 52.000000 \n", "\n", " mfd_a_ag mfd_ag_e mfd_ag_a mfd_ag_ag mfd_der \\\n", "count 13870.000000 13870.000000 13870.000000 13870.000000 13870.000000 \n", "mean 0.752127 0.017448 0.014924 0.030353 2.185436 \n", "std 1.077628 0.200902 0.152351 0.209759 5.211118 \n", "min 0.000000 0.000000 0.000000 0.000000 -1.000000 \n", "25% 0.000000 0.000000 0.000000 0.000000 1.000000 \n", "50% 1.000000 0.000000 0.000000 0.000000 2.000000 \n", "75% 1.000000 0.000000 0.000000 0.000000 2.000000 \n", "max 43.000000 4.000000 5.000000 6.000000 303.000000 \n", "\n", " powerlaw_alpha \n", "count 13870.000000 \n", "mean -0.916534 \n", "std 0.612437 \n", "min -1.000000 \n", "25% -1.000000 \n", "50% -1.000000 \n", "75% -1.000000 \n", "max 8.184413 \n", "\n", "[8 rows x 22 columns]" ] }, "execution_count": 3, "metadata": {}, "output_type": "execute_result" } ], "source": [ "df.describe()" ] }, { "cell_type": "code", "execution_count": 4, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "u_3 6745\n", "u_8 4449\n", "u_5 1327\n", "u_2 487\n", "u_12 312\n", "u_14 150\n", "u_9 141\n", "u_6 71\n", "u_7 66\n", "u_4 34\n", "u_1 25\n", "u_13 21\n", "u_10 21\n", "u_11 21\n", "Name: label, dtype: int64" ] }, "execution_count": 4, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# The number of each label in the dataset\n", "df.label.value_counts()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ " ## Experiment" ] }, { "cell_type": "code", "execution_count": 5, "metadata": { "collapsed": true }, "outputs": [], "source": [ "from analytics import balance_smote, test_classification" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "**Balancing the data**\n", "\n", "With an unbalanced like the above, the resulted trained classifier will typically be skewed towards the majority labels. In order to mitigate this, we balance the dataset using the [SMOTE Oversampling Method](https://www.jair.org/media/953/live-953-2037-jair.pdf)." ] }, { "cell_type": "code", "execution_count": 6, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Original data shapes: (13870, 22) (13870,)\n", "Balanced data shapes: (94430, 22) (94430,)\n" ] } ], "source": [ "df = balance_smote(df)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "**Cross Validation tests**: We now run the cross validation tests on the dataset (`df`) using all the features (`combined`), only the generic network metrics (`generic`), and only the provenance-specific network metrics (`provenance`). Please refer to [Cross Validation Code.ipynb](Cross%20Validation%20Code.ipynb) for the detailed description of the cross validation code." ] }, { "cell_type": "code", "execution_count": 7, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Accuracy: 98.13% ±0.0080 <-- combined\n", "Accuracy: 92.32% ±0.0157 <-- generic\n", "Accuracy: 98.11% ±0.0079 <-- provenance\n" ] } ], "source": [ "results, importances = test_classification(df)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "**Result**: The outputs above is the accuracy of the classifier in identifying the owner of a provenance document from ProvStore using all provenance network metrics (i.e. `combined`), only generic network metrics, and only provenance-specific network metrics.\n", "\n", "The individual accuracy scores are stored in `results` and the importance of every feature in each test in `imps` (both are pandas Dataframe objects)." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Saving experiments' results (optional)\n", "\n", "Optionally, we can save the test results to save time the next time we want to re-explore them:" ] }, { "cell_type": "code", "execution_count": 8, "metadata": { "collapsed": true }, "outputs": [], "source": [ "results.to_pickle(\"provstore/results.pkl\")\n", "importances.to_pickle(\"provstore/importances.pkl\")" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Next time, we can reload the results as follows:" ] }, { "cell_type": "code", "execution_count": 9, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "((3000, 2), (1000, 22))" ] }, "execution_count": 9, "metadata": {}, "output_type": "execute_result" } ], "source": [ "import pandas as pd\n", "results = pd.read_pickle(\"provstore/results.pkl\")\n", "importances = pd.read_pickle(\"provstore/importances.pkl\")\n", "results.shape, importances.shape # showing the shape of the data (for checking)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### The importance of features\n", "\n", "We plot the importance of each feature (from the `combined` test, which used all 22 features)." ] }, { "cell_type": "code", "execution_count": 10, "metadata": { "collapsed": true }, "outputs": [], "source": [ "%matplotlib inline\n", "import seaborn as sns\n", "sns.set_style(\"whitegrid\")\n", "sns.set_context(\"talk\")" ] }, { "cell_type": "code", "execution_count": 11, "metadata": { "collapsed": true }, "outputs": [], "source": [ "# Rename the columns with Math notation for consistency with the metrics symbols in the paper\n", "feature_name_maths_mapping = {\n", " \"entities\": \"$n_e$\", \"agents\": \"$n_{ag}$\", \"activities\": \"$n_a$\", \"nodes\": \"$n$\", \"edges\": \"$e$\",\n", " \"diameter\": \"$d$\", \"assortativity\": \"$r$\", \"acc\": \"$\\\\mathsf{ACC}$\",\n", " \"acc_e\": \"$\\\\mathsf{ACC}_e$\", \"acc_a\": \"$\\\\mathsf{ACC}_a$\", \"acc_ag\": \"$\\\\mathsf{ACC}_{ag}$\",\n", " \"mfd_e_e\": \"$\\\\mathrm{mfd}_{e \\\\rightarrow e}$\", \"mfd_e_a\": \"$\\\\mathrm{mfd}_{e \\\\rightarrow a}$\",\n", " \"mfd_e_ag\": \"$\\\\mathrm{mfd}_{e \\\\rightarrow ag}$\", \"mfd_a_e\": \"$\\\\mathrm{mfd}_{a \\\\rightarrow e}$\",\n", " \"mfd_a_a\": \"$\\\\mathrm{mfd}_{a \\\\rightarrow a}$\", \"mfd_a_ag\": \"$\\\\mathrm{mfd}_{a \\\\rightarrow ag}$\",\n", " \"mfd_ag_e\": \"$\\\\mathrm{mfd}_{ag \\\\rightarrow e}$\", \"mfd_ag_a\": \"$\\\\mathrm{mfd}_{ag \\\\rightarrow a}$\",\n", " \"mfd_ag_ag\": \"$\\\\mathrm{mfd}_{ag \\\\rightarrow ag}$\", \"mfd_der\": \"$\\\\mathrm{mfd}_\\\\mathit{der}$\", \"powerlaw_alpha\": \"$\\\\alpha$\"\n", "}\n", "importances.rename(columns=feature_name_maths_mapping, inplace=True)" ] }, { "cell_type": "code", "execution_count": 12, "metadata": {}, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAX8AAAEuCAYAAABmlhI1AAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3XtYVPW6B/DvDIIzCGpFUj6WIp0iTUMwDMMbhe5toyZB\nulEwPaZQJOk2o067yC6Wx+0NE9tImxR3XmgrKqaZKZqE11OKZI/KqKAFEYUKM4Cwzh9uRkcuM2vW\nDIthfT/P4/Mwa37v/N41M7yMv3nXWipBEAQQEZGiqOVOgIiIWh+LPxGRArH4ExEpEIs/EZECsfgT\nESkQiz8RkQJ1kDsBaxw7dkzuFIiInFJgYGCT260q/gUFBXjrrbdw9uxZ9OzZE++88w78/f2bHf/e\ne+/B1dUVr732mmnb119/jWXLluHSpUu499578corryAsLEzyDhARUdNa+uBscdmnuroasbGxCA8P\nx5EjRxAdHY24uDhUVlY2Gvv7778jMTERa9euNduu1+sxb948vPHGGzh27Bhef/11zJs3D+fOnbNh\nd4iISCqLxT8vLw9qtRpRUVFwdXVFREQEvLy8kJOT02hsVFQUXFxcMGrUKLPtly5dwnPPPYfg4GCo\nVCqEhITAx8cHJ0+etN+eEBGR1Swu++j1evj6+ppt8/HxQWFhYaOx6enp8Pb2RmJiotn2kJAQhISE\nmG4XFRXhzJkz8PPzszVvIiKSwGLxr6qqglarNdum0WhgNBobjfX29rY4YUlJCV544QWMHz9eVPFv\naj4iIrKNxeKv1WobFV6j0Qh3d3fRkxUUFCA2NhbDhw9HUlKSqNhTp06Jno+IiJpmsfj37t0bGRkZ\nZtv0ej10Op2oifbv34/Zs2fjpZdewrRp08RlCaBv376iY4iIlKylD80Wi39wcDBqamqwdu1aTJw4\nEVlZWSgrKzNbw7fkzJkzmDVrFt5//308/fTTVsfdSqPRNNqWm5uLJUuWAABmz56NwYMH2/TYRERK\nY7Hbx83NDampqcjOzkZQUBAyMjKQkpICd3d3TJ8+HatWrbI4yZo1a2A0GvHmm29iwIABpn8bNmyw\nOXFBEJCcnIzy8nKUl5cjOTkZvDQBEZF1VM5wMZdjx441OsjLYDA0Wnravn17oy+niYiUqqna2cAp\nTu/Q4NeUm989CIKAu7Sd8JvhxsFmd2k74eo/M3FNpTKNuTtucqvnSETkDJz2xG4qlQrTBgSjq0aL\nrhotpg24cQAZERFZ5lSf/G8X2P0+BHafKHcaREROx2k/+RMRke1Y/ImIFIjFn4hIgVj8iYgUiMWf\niEiBWPyJiBSIxZ+ISIFY/ImIFIjFn4hIgVj8iYgUiMWfqJ3Jzc1FZGQkIiMjkZubK3c61Eax+BO1\nI7zOBVnLqU/sRkTA+C/2mX4WamtwtbTUdLu0tBTjN+yGytXNtG3zs8NbMTtqq/jJn6g96eAKlUdn\n002VR2egg6uMCVFbxeJP1I6oVCpohv4ZKncPqNw9bvzM61xQE7jsQ9TOuPZ6EK7PPyh3GtTG8ZM/\nEZECsfgTESkQiz8RkQKx+BMRKRCLPxGRArH4ExEpEIs/EZECWVX8CwoKEBERAX9/f4wbNw7ff/99\ni+Pfe+89fPTRR2bbcnNzodPp4O/vj6ioKOj1etuzJiIiSSwW/+rqasTGxiI8PBxHjhxBdHQ04uLi\nUFlZ2Wjs77//jsTERKxdu9Zse1lZGeLj4zFnzhwcPnwYgwcPRnx8PE84RUQkE4vFPy8vD2q1GlFR\nUXB1dUVERAS8vLyQk5PTaGxUVBRcXFwwatQos+1fffUVHn74YYSGhsLNzQ1xcXEoLS3FyZMn7bcn\nRERkNYund9Dr9fD19TXb5uPjg8LCwkZj09PT4e3tjcTERLPthYWFZo/h4uKC++67D4WFhejfv79V\niRqNRqvGSY0hau/4e0GAFcW/qqoKWq3WbJtGo2nyDeTt7d3kYxgMBnh4eJht02q1MBgMVid66tQp\n3G/16JsxRGSOvxcEWFH8tVpto0JvNBrh7u5u9SRNPYbBYBD1GH379sXVwz9aPb4hhqjdO58najh/\nL5SjpT/0Fot/7969kZGRYbZNr9dDp9NZnUDv3r2xc+dO0+26ujpcvHgRDzzwgNWPodFocNXq0Tdj\niMgcfy8IsOIL3+DgYNTU1GDt2rWora1FZmYmysrKEBISYvUkYWFhyM/Px1dffYWamhqkpKTgnnvu\nQZ8+fSQlT0REtrFY/N3c3JCamors7GwEBQUhIyMDKSkpcHd3x/Tp07Fq1SqLk9x9991YuXIlVqxY\ngUGDBiE3NxfJycm8yAQRkUysupiLn58f1q9f32j76tWrmxz/4YcfNtr2+OOPY+vWrSLTIyIiR+Dp\nHYiIFIjFn4hIgVj8iYgUiMWfiEiBWPyJiBSIxZ+ISIFY/ImIFIjFn4hIgVj8iYgUiMWfiEiBWPyJ\niBSIxZ+ISIFY/ImIFIjFn4hIgVj8iYgUiMWfiEiBWPyJiBSIxZ+ISIFY/ImIFIjFn4hIgVj8iYgU\niMWfiEiBWPyJiBSIxZ9aRW5uLiIjIxEZGYnc3Fy50yFSPBZ/cjhBEJCcnIzy8nKUl5cjOTkZgiDI\nnRaRollV/AsKChAREQF/f3+MGzcO33//fZPj0tPTMWTIEAQEBGDu3Lmoqqoy3bdp0yY8+eSTCAwM\nxMSJE5Gfn2+fPaA2z2g0orS01HS7tLQURqNRxoyIyGLxr66uRmxsLMLDw3HkyBFER0cjLi4OlZWV\nZuP27t2LtLQ0rFmzBjk5OaioqMDChQsBAKdPn8aiRYuwevVqHDlyBKGhoUhISHDMHhERkUUWi39e\nXh7UajWioqLg6uqKiIgIeHl5IScnx2xcVlYWIiIi4OPjA09PTyQkJCArKwt1dXW4cOEC6uvrUVdX\nB0EQoFarodFoHLZTRETUsg6WBuj1evj6+ppt8/HxQWFhodm2wsJChIWFmY2pqqpCSUkJQkJC0KtX\nLzz99NNwcXFBp06dsGbNGlGJ2rJMwKUFebyR/YzZ7braxuv7iVnPwMVVZbr9wdNbHJ4X3cDfCwKs\nKP5VVVXQarVm2zQaTaM3kMFgMPs03xDTsP2BBx7AW2+9hQcffBCpqamIj49Hdna21f8DOHXqFO63\naqR5DDkHvlath881AVYUf61W26jQG41GuLu7m23TaDSorq423TYYDACATp06YcWKFbjnnnvQr18/\nAMBLL72EjRs3Ijc3F6GhoVYl2rdvX1w9/KNVY2+NIRmcN7+p7gC4egC1127cdvW4se1WfK0kOJ8n\najifa+Vo6Q+9xeLfu3dvZGRkmG3T6/XQ6XRm23x9fc2WgvR6PTp37oxu3brh8uXLZv97UKlUcHFx\ngYuLi9U7odFocNXq0TdjSH4qlQo9hrqgaF8dAKDHUBeoVCqzMXytWg+fawKs+MI3ODgYNTU1WLt2\nLWpra5GZmYmysjKEhISYjRs7diw2bNiAM2fO4Nq1a1i+fDl0Oh3UajWGDx+OzMxMnDp1CtevX8c/\n//lP1NXVITAw0GE7ZgkPOmpdnXup0fd5V/R93hWde/HwEiK5Wfzk7+bmhtTUVCQlJWHx4sXo2bMn\nUlJS4O7ujunTp2PgwIGIjY1FaGgoiouLMXPmTFy5cgXDhg3DvHnzAAATJkzAlStX8PLLL+PKlSt4\n+OGHsXr1anh4eDh8B5ty60FHAJCcnIzg4OBGn0aJiNori8UfAPz8/LB+/fpG21evXm12OyYmBjEx\nMY3GqVQqzJgxAzNmzLAxTelKUj40/Wy8fr3RQUcXV7wPTYebT4d3XGKr5kdE1JoU+f/vji4uuEt7\nc93zLq0GHUV8/0BE5OwUWfxVKhWmDvBDV40bumrcMHWAH5d8iEhRrFr2aY8Cut+Nld2HyZ0GEZEs\nFPnJn4hI6Vj8iYgUiMWfiEiBWPyJiBSIxd8GPDqYiJwdi79IvCQhEbUHim31FONc8jjTz8brAkpL\nb17FrLS0FAVLx0HTwfw4Ad+Xs1otP2p/cnNzsWTJEgDA7NmzMXjwYJkzovaGn/xF6ugC3KG9Wejv\n0KrQkQcHkx3xf5fUGlj8RVKpVJj0aEd07qhC5443fubRwWRPvOA9tQYu+9ig/z0dsOjPfOpaixKW\nQMZm3lwmFAQBKg8PCNduXP1G5eGB57bvMvuQsTViXKPHIBKDFYzaNCWeflulUsFtyAjU5OwBALgN\nGdGu91cpSpYeFjXe+5UgB2VyA4s/tWnNLYHcfl3p9qZDLx906DVd7jSoHeOaP5ED8FgQautY/Ins\njN065Ay47ENtzp+zYk0/CzX1je4fvz0BKrebn1u+HLeqVfJqiS5znelnobYWxtuWqnTrP4PK1dUs\nZnvEpFbLj+h2/ORPZG8dOkDl0cl0U+XRCejAz1nUtrD4U9vmqgI8bimcHh1ubGvDVCoVXIcEA+5a\nwF0L1yHtuzuJnBM/jlCbplKp0GHE3bi+58YySocRdztFIXXpdT+0ve6XOw2iZrH4U5un9vGA23QP\nudMgale47ENEpEAs/kRECsTiT0SkQFYV/4KCAkRERMDf3x/jxo3D999/3+S49PR0DBkyBAEBAZg7\ndy6qqqpM9x09ehTjx4/HgAEDMGbMGHz33Xf22QMiIhLNYvGvrq5GbGwswsPDceTIEURHRyMuLg6V\nlZVm4/bu3Yu0tDSsWbMGOTk5qKiowMKFCwEAJSUliIuLQ2xsLI4fP46ZM2fi5Zdf5mlqiYhkYrH4\n5+XlQa1WIyoqCq6uroiIiICXlxdycnLMxmVlZSEiIgI+Pj7w9PREQkICsrKyUFdXh6ysLAwePBij\nRo2CSqWCTqfDZ599BrWaq05ERHKwWH31ej18fX3Ntvn4+KCwsNBsW2FhIR544AGzMVVVVSgpKcGp\nU6fg7e2Nl156CYMGDcKECRNQV1cHNzc3O+0GERGJYbHPv6qqqtHpczUaTaMlG4PBAI1GY7rdEGMw\nGFBRUYH9+/cjOTkZS5cuxcaNGzFjxgzs2rULXbp0sSpRW5aIpCwrSV2S4pKWOHK+VnKRa5+d9flS\nGke/ThaLv1arbZSE0WiEu7u72TaNRoPq6mrTbYPBAADo1KkT3NzcMHToUISEhAAAJk2ahLS0NBw/\nfhwjRoywKtFTp05B7PGSp06dMv3cQ0JsV5Gxt8eTZVKeL2d9ruXaZ2d9vpydlBrkCBaLf+/evZGR\nkWG2Ta/XQ6fTmW3z9fU1WwrS6/Xo3LkzunXrBh8fH1y8eNFsfH19vajT3Pbt2xdXD/9o9fiGmAYV\nh3fbHHvJhtOx3xqvOOfFh5g9X8USYuWiPy06xCxvvbidNos9n2d7LLWaigMnRI23x+vU0h8Qi8U/\nODgYNTU1WLt2LSZOnIisrCyUlZWZPsU3GDt2LN5++22MGjUK9957L5YvXw6dTge1Wo1x48ZhwoQJ\n2LdvH4YOHYp169ahuroagwYNsnonNBoNrlo9+mZMgwoJsbaQGq80Up4vZ32u5dpnZ32+nF1r1yBL\nLH7h6+bmhtTUVGRnZyMoKAgZGRlISUmBu7s7pk+fjlWrbpxLPTQ0FC+88AJmzpyJ4cOHw9PTE/Pm\nzQMA9OnTBykpKVi6dCkCAwOxefNmrFq1Cp06dWppaiIichCrTuzm5+eH9evXN9q+evVqs9sxMTGI\niYlp8jFCQkIa/W+BiIjkwUZ7IiIFYvEnIlIgFn8iIgVi8SciUiAWfyIiBWLxJyJSIBZ/IiIF4gXc\niZqRm5uLJUuWAABmz56NwYMHy5xR2/Lu5suixv9tfHcHZUK24Cd/oiYIgoDk5GSUl5ejvLwcycnJ\nos5FRdTWsfgTNcFoNKK0tNR0u7S0lKdCpnaFyz5E//H0F5+YfhYEAfBwB6795zrUHu54NvszqFQq\n05jsZ2e2dopEdsNP/kRNUKlUcB0aALhrAHcNXIcGmBV+ImfHT/5EzXDp1R0uz4+VOw0ih+AnfyIi\nBWLxJyJSIBZ/IiIFYvEnIlIgFn8iIgVi8SciUiAWfyIiBWLxJyJSIBZ/BcnNzUVkZCQiIyORm5sr\ndzpEJCMWf4XgWSqJ6FY8vUM79ln6SNPPtbUCSkvrTbdLS0uRtnokXF1vnq9myvNftWp+RCQffvJX\niA4dAHf3m7fd3W9sIyJlYvFXCJVKhccHqaHVAlot8PggNc9SSaRgVhX/goICREREwN/fH+PGjcP3\n33/f5Lj09HQMGTIEAQEBmDt3LqqqqhqN+e677+Dn54fKykppmZNoPXqoEBnhgsgIF/TowcJPpGQW\ni391dTViY2MRHh6OI0eOIDo6GnFxcY2K9969e5GWloY1a9YgJycHFRUVWLhwodmYiooKvPHGG/yi\nkYhIZhaLf15eHtRqNaKiouDq6oqIiAh4eXkhJyfHbFxWVhYiIiLg4+MDT09PJCQkICsrC3V1daYx\nSUlJGD16tP33goiIRLH4lZ9er4evr6/ZNh8fHxQWFpptKywsRFhYmNmYqqoqlJSUoHv37ti6dSuu\nXLmCv/71r1i9erXoRG25fqqUa65KvV6rM17vVc6c5Xyt5JpXrn121udLaRz9fFks/lVVVdBqtWbb\nNBpNo8QMBgM0Go3pdkOMwWDA5cuXsWzZMvzrX/9CbW2tTYmeOnUK99sQ06CHhNiuImNvj3cWcuYs\nZW658pY6r1z7bL/n616Z5nVOUmqQI1gs/lqttlGhNxqNcL+1bxA3/iBUV1ebbhsMBlP8a6+9htmz\nZ8Pb2xvFxcU2Jdq3b19cPfyj6JgGFYd32xx7yYaDYW+Nl0v+SXHj7ZbzefEhZnOLfIvYL++jts+r\nPy16OvN4cTttFns+z/ZYCXZcLJdlXmdVceCEqPH2eL5a+gNisfj37t0bGRkZZtv0ej10Op3ZNl9f\nX7OlIL1ej86dOwMAfvjhB/z4449ISkpCff2NA42GDRuGVatWYeDAgVbthEajwVWrRprHNKiQEGsL\nqfFykDNnKXPLlbec7xElPl/OrrVrkCUWv/ANDg5GTU0N1q5di9raWmRmZqKsrAwhISFm48aOHYsN\nGzbgzJkzuHbtGpYvXw6dTofu3bvjxIkTOHr0KI4ePYqtW7cCAHJycqwu/EREZF8Wi7+bmxtSU1OR\nnZ2NoKAgZGRkICUlBe7u7pg+fTpWrVoFAAgNDcULL7yAmTNnYvjw4fD09MS8efMcvgNERCSeVQf4\n+/n5Yf369Y223961ExMTg5iYmBYfq0ePHvjpp59EpEhERPbG0zsQESkQiz8RkQKx+BMRKRCLPxGR\nArH4ExEpEIs/EZECsfgTESkQiz8RkQKx+BMRKRCLPxGRArH4ExEpEIs/EZECsfgTESkQiz8RkQKx\n+BMRKRCLPxGRArH4ExEpEIs/EZECsfjLIDc3F5GRkYiMjERubm6rxRIRNWDxb2WCICA5ORnl5eUo\nLy9HcnIyBEFweCwR0a2suoA7SZP7D53p5+paAaWlRtPt0tJS7FupQ0dXlWnb4Bnbm3wco9GI0tJS\ns1ij0QitVuuArImoPWPxb2VuHYAu7ipUVN34xN7FXQW3Fl6F7Z/+2fRzTW3jT/lfrhkPt1v+cOim\nfWm/ZImo3eKyTytTqVQYE+gKDw3goQHGBLpCpVJZDgTg2gHwdL9529P9xjYiIrFYOmTwUHcXvDZO\n/FKNSqVC6GMdsPvQdQBA6GMdrP7DQUR0KxZ/J9O7hxoze7jJnQYROTmrln0KCgoQEREBf39/jBs3\nDt9//32T49LT0zFkyBAEBARg7ty5qKqqMt23ceNGjBw5EgEBAXj22Wdx9OhR++wBERGJZrH4V1dX\nIzY2FuHh4Thy5Aiio6MRFxeHyspKs3F79+5FWloa1qxZg5ycHFRUVGDhwoUAgLy8PCxevBjLli3D\n0aNHMXnyZMTGxuL33393zF4REVGLLBb/vLw8qNVqREVFwdXVFREREfDy8kJOTo7ZuKysLERERMDH\nxweenp5ISEhAVlYW6urq8Msvv+C///u/8fDDD0OtVmP8+PFwcXHB2bNnHbZjRETUPItr/nq9Hr6+\nvmbbfHx8UFhYaLatsLAQYWFhZmOqqqpQUlKCZ555xmzssWPHUFlZ2ehxW2I0Gi0PskOMPWLlnFvO\nfZbCGfNW4ntECjnfX87I0c+XxeJfVVXV6CAijUbTKDGDwQCNRmO63RBjMBjMxp09exazZs3CrFmz\ncOedd1qd6KlTp3C/1aNvxjToISG2q8jY2+OVECuVM+YtdV7nf53vlWle5ySlBjmCxeKv1WobFXqj\n0Qh3d3ezbRqNBtXV1abbDUW/U6dOpm3ffvstZs+ejalTp2LGjBmiEu3bty+uHv5RdEyDisO7bY69\nZMMpdG6NP37M9tiff7A9Nv+k7bGSnBcfYjZ3sYRYKc6La0Iwm1d/WvR05vHidtos9nye7bES7LhY\nLsu8zqriwAlR4+3xfLX0B8Ri8e/duzcyMjLMtun1euh0OrNtvr6+ZktBer0enTt3Rrdu3QAAX3zx\nBd5//33Mnz+/Uaw1NBoNrtoQ06BCQqwtpMQ7Y6xUzpi3Et8jUsj5/moLWrsGWWLxC9/g4GDU1NRg\n7dq1qK2tRWZmJsrKyhASEmI2buzYsdiwYQPOnDmDa9euYfny5dDpdFCr1fjuu+/wzjvv4B//+IdN\nhZ+IiOzL4id/Nzc3pKamIikpCYsXL0bPnj2RkpICd3d3TJ8+HQMHDkRsbCxCQ0NRXFyMmTNn4sqV\nKxg2bBjmzZsHAEhNTUVtbS1eeOEFs8detmwZhg4d6pg9IyKiZll1hK+fnx/Wr1/faPvq1avNbsfE\nxCAmJqbRuE8//dTG9IioPdr4RZnomOee9XJAJsrFE7sRESkQiz8RkQKx+BMRKRCLPxGRArH4ExEp\nEIs/EZECsfiTVXJzcxEZGYnIyEjk5tpwvgsialNY/MkiQRCQnJyM8vJylJeXIzk5GYLQ+GLyROQ8\nWPzJIqPRiNLSUtPt0tJSnp6XyMnxGr7UpCX/GmX6+Xpt40/5KzaOQwfXmxePnx21q1XysmT05g9F\njd8xPtFBmTiHyC/yRY3f9OwjDsqEWhuLP1nk0gHQdAKM/7lyp6bTjW1Ezuj0yhJR4/1e9HZQJvLi\nsg9ZpFKp0G+wGh21QEct0G+wGiqVynIgEbVZ/PxGVvG+X42wKH5WIGov+NtMRKRALP5ERArE4k9E\npEAs/kRECsTiT0SkQOz2ISKy0i+LCkWNv2dubwdlIh0/+RMRKRCLPxGRArH4ExEpEIs/EZECsfgT\nESkQiz8RkQJZ1epZUFCAt956C2fPnkXPnj3xzjvvwN/fv9G49PR0pKWlobKyEqGhoZg/fz7c3d0B\nANu3b8eSJUvw22+/YdCgQXj//ffh5eVl370hImqnSlfsFDW+W/yfWrzf4if/6upqxMbGIjw8HEeO\nHEF0dDTi4uJQWVlpNm7v3r1IS0vDmjVrkJOTg4qKCixcuBAAcPr0abz99ttYvHgx8vLy4OXlhddf\nf13UjhARkf1YLP55eXlQq9WIioqCq6srIiIi4OXlhZycHLNxWVlZiIiIgI+PDzw9PZGQkICsrCzU\n1dVh27ZtePLJJ/Hoo49Co9Fg7ty5OHDgAMrKyhy2Y0RE1DyLxV+v18PX19dsm4+PDwoLzY90Kyws\nxAMPPGA2pqqqCiUlJY3uu+OOO9ClSxfo9Xqp+RMRkQ0srvlXVVVBq9WabdNoNI0u4G0wGKDRaEy3\nG2IMBkOj+xruNxgMVidqywXDpVxkXOoFyuWa2xlj5Zyb++wcsfaIl2Petvx8qQRBaHx17lv885//\nxMGDB7F69WrTtlmzZsHPzw8vvviiaduYMWMQFxeH0aNHAwAqKysREBCAnJwcJCUlISAgADNmzDCN\nHzRoED7++GMMHDjQ4k4cO3bM4hgiImosMDCwye0WP/n37t0bGRkZZtv0ej10Op3ZNl9fX7OlIL1e\nj86dO6Nbt27w9fU1W+IpLy9HRUVFo+UksckTEZFtLK75BwcHo6amBmvXrkVtbS0yMzNRVlaGkJAQ\ns3Fjx47Fhg0bcObMGVy7dg3Lly+HTqeDWq2GTqfDV199haNHj6K6uhqLFy/G0KFDcccddzhsx4iI\nqHkWl32AG62aSUlJ+Omnn9CzZ08kJSXB398f06dPx8CBAxEbGwsAWLNmDdLT03HlyhUMGzYM7733\nnmntf8eOHVi2bBl+/fVXDBw4EAsWLMBdd93l2L0jIqImWVX8iYiofeHpHYiIFIjFn4hIgVj8iYgU\niMWfiEiBWPyJiBTI6Yv/pUuXUFRUJHcaFtXV1SEnJwc5OTn4+eefRcefO3cOmzdvxu7du60af+LE\nCdTU1Iiep61yltf5dsy79ThjzmLZc/+cutXzww8/RHFxMS5evIjNmzdj165dptNLWPLll1+iT58+\n6Nmzp+h5lyxZgn379kEQBDz00EN45JFHMGXKlBZjXn75ZRw+fBgeHh64fPky7rzzTjzyyCN45JFH\n8PLLL7cY++9//xvvvfce+vfvjx9++AH/93//h/z8fNTV1eHRRx9tMiYwMBDV1dXw8fFBnz598PDD\nD6NPnz7o06cPPDw8LO5jXV0dRowYgT179sDV1dXi+FuVlZXh8OHDTb4WO3bswKBBg0Qd4yHldQZs\ne70aSHmfyJW3lJyl5u2Mz7VcOQPAL7/8gnvuucfq8WFhYRgxYgTi4uLMDpL98ssv8ec//1nU3E79\nyf/kyZNYsWIFunTpAhcXF2RmZlodu3//fkyZMgVjxoxBVFQUPvzwQ5w4ccKq2AMHDiArKwtdunTB\nyJEjcerUKYsx3377LbZu3Yo9e/bg8OHDWLRoEQYOHIhz585ZjP3444+Rnp6O9PR0dOhw44wcHTt2\nNF0voSnHjh3Dl19+iYSEBHh7e2PRokWIiYnBY489hrCwMMyaNQurVq1qNt7FxQUdO3ZEXV2dxfxu\nl5qaigsXLjR5X3FxMVJTU0U9npTXGbDt9Wog5X0iV95ScpaatzM+13LlDAD/8z//Y/VYANi8eTPO\nnTuHsLAwrFixArt378Znn32GN998U9TjAE5e/F1dXVFdXQ2VSgUAqK2ttTr2559/xjfffINt27Zh\n6tSpKC9yTVnCAAASjUlEQVQvR2pqKtavX28xtuEvrlqtRlhYmFUF0tvb2xTn6emJ4OBgvPDCC1i6\ndKnF2GvXrqF///4AYNrXXr164ezZsy3G3XfffXjqqadQVFSExMREnDhxArt370Z4eDgOHDiAgwcP\nthgfExNjdkI/a+Xk5CAyMrLJ+8LDw7F3715RjyfldQZse70aSHmfyJW3lJyl5u2Mz7VcOQPAQw89\nZNUHwAazZ89GUVERBg8ejIKCAsyZMwerV6/GBx98YPVjNHDq4j9t2jRMmzYNv/32G7Kzs0XFGo1G\n04scFhaGq1evYtmyZcjKyrIYO3ToUFRVVaFXr17YvHkzCgoKLMY8//zz+Pzzz0Xl2MDX1xeHDh0C\nADSs0nXo0AHXr1+3Kn7//v2YPHky3Nzc0KNHD8TFxVl1eo2PP/4Yq1atQmxsLLKzs1FcXGzVfGVl\nZc1eotPLy0v0RXykvM6Aba9XAynvE7nylpKz1Lyd8bmWK2cAOHToEKKiojB79mx89tlnFs9gfPz4\ncWzZsgXLly/HypUrsW3bNnTv3h2XL1+2OmcTwclduHBB+Pjjj4UlS5YIly9ftjpu586dwuTJk4Ut\nW7YIW7ZsEcaMGSMIgiBERUVZ/RhXrlwRFi5cKOzZs8fi2CeeeELo16+fMGfOHCE7O1soKiqyep4D\nBw4IgwcPFnbv3i089thjgiAIwp49ewSdTmdVfFhYmHDixAmzbbW1tcKQIUNajMvLyxPS0tKEOXPm\nCCNHjhT8/PyEQYMGCdOmTWsx7vHHHxdKSkqavK+kpEQICgqyKu9b2fo630rM69VA6vtEjrzt8d6W\nmrczPtdy5CwIgnD+/HkhPz9fyMzMFN59990Wx44ZM0Y4d+6c2bbff//dpt8ppyr+169fF/bt2yfs\n27fP5l+kWxUVFQkrV64UFi9eLJw7d06ora0VlixZ4pB5Dx48KHzyySdCQkKCEBYWZnUhbZCVlSUE\nBQUJfn5+Qnh4uODv7y9kZ2dbFZudnS2MGDFC2Ldvn1k+gwYNErUPV69eFfLy8oRPP/20xXEvvvii\n8Pe//73J+xYvXiy8+OKLouaVm7Xvk7bEGXMWBOfMW0rOCxYsEF566SVhzJgxwvXr14Xt27e3OH7L\nli1CaGiosGfPHuH69euCIAjCjz/+KAQHB4vO26m6fZrrmOnXrx/i4+NFPZaYb/ildOo059q1a8jP\nz8ePP/6IqVOnWhVjMBhw+PBh/P777+jXr5/V10MAbnQMffTRR6ivr8ddd92FoqIixMbG2px/S06e\nPIlJkyZh7NixePrpp+Ht7Y2SkhJkZ2dj27Zt+Ne//oW+ffs2ipPaJWTvLiNAWieIXJwxZ8A585aa\n86RJk7Bu3TpER0dj7dq1mDZtGj799NMWYz7//HMsXboUNTU1uPvuu3H58mVERUXhjTfeEJW7xYu5\ntCXffvstdu7cCW9vb1y9ehX5+fnIz88X9e18g4Zv+KOjozFy5MgW++ftOW8DDw8PPP7443j88cet\njtFqtRg2bJhN84WHh2P06NE4fPgwfvvtNzzwwAPo16+fTY9lSb9+/ZCSkoL58+cjMzMTKpUKgiCg\nZ8+eSElJabLwAze6hLp27drkfcXFxThx4gQSExObnVdqfFPEvE+aU1JSAm9vb9FxtsbbI2db5pUa\nK/dzLUfOtnxR/Ze//AXPPfccjh07hp9//hn33nsvHnvsMVHzAk72ha+UjpnbifmG357zykmj0WDo\n0KEYP368wwp/gyeeeAK7du3Czp07sW7dOuzcuRO7du3C4MGDm42R2iVk7y4jQFonCHDjS/OwsDDk\n5OSIntvWeKk52zqv1Fg5n2u5crb1i2oXFxcEBQVh3LhxCAoKMv3xEMOpir+UjpnbifmG357zKkFZ\nWRl27NgB4EZLakBAAHr16gXgxvLLb7/91myclC4he3cZAdI6Qb755hv88ccf8PHxwbp16/DLL7+I\nmtvWeCk5S81bSqxcz7VcOTfEL1iwAE8//TTOnDnT4rE7dif6WwIZSemYaY413/A7Yt727IMPPhBW\nrlzZ5H2ffPKJsGDBgibvk9ol5Iguowa2dILU19cLgiAIEyZMMP0shtR4W3KWOq/UnAWh9Z9rOXKe\nNm2asGjRIiE7O1vQ6/U2zSmVU635L1y4EPn5+SgoKMDSpUtRVFSELl26oG/fvkhLS2sxtq6uDt9+\n+y0A4MEHH8S9994L4MYyzquvvuqweZUoJycHGRkZTd4XHh6OSZMmNbn2HhAQgIyMDMyZM6fRfevW\nrcPAgQNbnFdqfEuseZ/c7tb/itvy33Kp8bbkLHVeqTkDrf9cy5HzokWLcPr0aezYsQOLFi2CVqtF\nly5d0KdPH5uO1rWFUxX/wYMHm60Z39oxY8krr7xic8eOlHmVyNbll9jYWEyaNAnl5eXNdgm1REq8\nIzqFiJpzxx13IDg4GKtWrcLXX38NtVqNrVu3Wn1wmD04VfG/nZiOGXt27NjSqaMkrq6uKC0tRbdu\n3RrdV1paajo/0e1s7RKyR7wjOoUaCBK7qW2Nl9plJCVvKbFS8na2nKurq3H9+nW4ublh7NixrVr8\nnWrNX4pRo0YJ1dXVcqehCPY4yEuv1wvHjh2zeT1UbPyoUaOEX3/9tcn7fv31V2HkyJE25SEIgrBx\n40abY22NX7FihdCvXz+zA/taY16psVLzdracv/76ayEmJkbYunWrkJmZKYwePdqmHGzhVAd5SbF+\n/XpUV1e3+YNG2gNnPMhr4MCBOHr0aLOPHRgYaPG8K23FN998g++++w6HDx+Gt7c35s+fL+q0wXJx\nxrztkfOlS5ewdetWGAwGjBs3TtTBm1IopviHhITgypUrCAsLw5NPPon+/fujR48ecqfVbh08eBDz\n58/HhQsXzJZf3n777WZ7/RcsWICuXbsiLi6u0X3/+Mc/UF5e3uLSi5T44OBgZGVlNbtUNWbMGNPJ\n9cS4dOkS6uvrcd9994mOtTVeEASoVCpMnDgRn3/+uc1fYrY2KXlLOa++lFhrcrb2pGvdu3cXPb8U\nTr3mLwY7dlpXw0Fe58+fR3l5Oe6880706tULZ86cwXvvvddkR4OtXUL2iHdEp5DUi7nYGi+1e0XK\nKQukxErJe//+/fjoo4/g6ekJT09P9O/fH6NHjzadCt1RsdbkHBoaatX+tHYDiWKKPzt25NGrVy90\n794dO3bsQGJiIn744Qf4+/s3OVbOg7ykdho15eTJk6bztjRcYERM8Zcabysppyyw16klxGo4r75a\nrcbu3buxe/dupKam4oknnsDEiRMdFmuNW48a3r9/PzZv3oz4+Hj06NEDxcXFWLlyJZ555hnJ84il\nmOJ/O3bsON7Zs2exfv16bNu2DQaDAfX19UhNTUVISEiT423tErJHvNROo+bykXIxF6nxtq7o3n7K\ngp07d7ZKbANb8m44r37DvP/+97/x8ccfY9KkSRYLuJRYa3K+tQsoLS0N69atM333dP/99+PBBx/E\n5MmTERERYdVc9qLY4k+Os2XLFmzcuBHHjx/HQw89hPj4eIwZMwY6nQ4PP/xws3FyH+TV3FKVrRrO\n2/LHH3/YdDEXqfG2FhMppyyQeroDwLa8p06dimnTppliL126BLVaDbXa8hlspMSKzbmsrAwajcZs\nm0ajsenUI1Ip5gtfaj1+fn7o2rUrPvroI7OzkIaEhCArK6vZjhtbu4TsEe+og7wuXryI7du3o6am\nBhMmTDAdWd5a8VJcvXoVq1atQmBgIEJDQ1st1lbFxcXYtm0bjEYjxo4di549e2LFihV45ZVXHBor\nRmxsLFxdXfHaa6+he/fuuHTpEv73f/8XNTU1LV5T2xFY/MnuUlJSsGnTJpSWlmLYsGF49tlnMXz4\ncAwdOrTF4g/Y1iVkj3hbO4WkdnI4qhPEUpeQlHnbaveKHMR2CpWVleGvf/0rDh06ZFrOCwoKwt//\n/vdmv69yFBZ/cghBELB//35s2rQJ+/btQ9euXXHlyhVs2LChxaWfBk11CW3YsMHq856Ijf/Tn/6E\njIyMJn8By8rKMGnSJOzatavRfX5+fpI6OaTGN8WaLiEp8zoiZ0C+LiMpsa+//jq+++470Z1CJSUl\npqOCpRyFLUlrHU1GylVSUiKsWLFCGDFihNCnTx9h1qxZVsVVV1cLmzdvFiZMmCD4+fkJEydOFDWv\nmPjAwMAWHysgIKDJ7b/88ovp38aNG4W//OUvwsGDB4ULFy4IBw8eFCZNmiRs2rSp2ceVGt+UhuvH\nTp48WRAEQZg6dapd53VEzoIgCOPHjzfl/dVXXwmvvvpqm4+dMmWKUFdXJwiCYIqNj48XPv/8c9OY\n8PBw08/JyclWP7ajsfhTq6mvrxf27dsnxMXFtTjuzJkzwrvvvisEBQUJ/fr1E/r27SscOHDA6nls\nibfH6aBHjRollJWVmW379ddfhVGjRlmVt9T4BlOmTBGMRqMQHR0tCMLNPwKOmNdeOQuCYLqedUxM\njCAIgjBnzpw2HzthwgShpqbGdDs2Nlaoq6sz+6ARGBhoOlX0gAEDrH5sR2O3D7UalUqFYcOGNXsp\nSlu7hOwRb4+DvKR2ctirE0Rsl5CUee3ZvSJXl5GUWGs6hR599FFMmjQJvr6+qKmpwd/+9rcmH+vd\nd9+1el574Jo/tRm2dgnZI15qpxEgvZPDnp0gYrqEpMzriO4VubqMbI211ClUUVGB9evXo6ioCFu2\nbMGYMWOafJwFCxaIylcqFn9qM6R0CdkjXmqnkdRODjHx9uy4kZK32Fi5uozaSodSbGxsq7d0NofF\nn9oUQWKXkNR4QHqnkdRODmviHdFxIyVva2Pl6jJqKx1KdXV1+PLLL5Gfn4/Kykqz+1p72Ydf+FKb\nZWuXkD3ire0UktrJYWu81I4bKXlLiZWry6itdCi9+eabQnBwsJCQkCAkJiaa/WttLP7U5lnbJWSP\neLGdQlI7OezRCWJLx42Uee3VvSJXl5GcHUoDBw4ULl++LHoeR2C3D7V5lrqE7BFva6eQ1E4Oe3SC\n2NJxI2Vee3WvyNVlJGeH0h133IE777xT9DyOYP2Zi4jascTERBQWFuKTTz4xnZK4uWv63mrx4sUY\nNmyY6ayO169fb/Kfo+KBG1chS0xMRHFxMerr61FUVIQ33nijxfZUKfPaI2db85Y79nZTpkyBu7s7\n5s6di7Nnz+LVV19tcXxCQgLef/99/PHHH6Lnsjd+4UsE6Z1CgPRODlvjpXYZSclbSmxrdhlJibVn\np9BPP/2E+Ph4FBcXw8XFxey+/Px8q+axFxZ/ov8QJHYKSe3kkBpva7eOlHnt0b3SGl1GUmLt2Sk0\nduxYPPTQQ9DpdNBqtWb3BQUFWU7ajrjmT/Qft343UFpaik2bNuGLL75AREQEnnrqKSxbtqzF+KSk\nJOzZswdBQUGNfrGtISb+2WefxRdffAEAWLFiBeLj421uLZWSt9hYKXnLFWvPK3EVFRVh8+bNjT71\ny0K2r5qJnICYTiGpnRxi4u15vhgpeYuNlavLqC10KAmCIERHRwuFhYU2z29P/ORP1AIxnUZSOznE\nxNvzfDFS8hYbK1eXUVvoUAKA4OBgxMbGYsKECbj77rvN7mvutA+O4pKUlJTUqjMStVN33nknPv/8\ncwwYMKBRgbB3/PDhw3Ht2jVUVFTg9OnT+K//+i/U19c3+vfUU085NG+xsVLyliv2VsePH8fevXvR\nv39/eHh4oLi4GO+++y68vb2h0+ks7v9rr72G+vp6nDhxAnl5eaZ/hw4dsvoaAvbCL3yJ7ERqJ4et\n8VK7jKTkLSVWri4juTqU2hoWfyI7kdrJYWu81I4bKXlLiZWry0juDqW2gmv+RHYitZPD1nipXUZS\n8pYS25pdRlJi7dlZ1Zaw+BPZSb9+/XDx4kX4+Pi0avzOnTuxdevWFs/Z74h5pcZKybs1Yy9cuABB\nEKBSqfDpp58iPj5e9JxtEYs/kZ1I7eSwNV5ql5GUvKXEtmaXkZTYtnolLqm45k9kJ81d/UmlUmHP\nnj0Oi8/OzsahQ4cwZ84cq85HZK95pcZKybs1Y9vqlbikYvEncnJt6XwxYsjVZSRXh1Jbw+JP5OTa\n0vlixJCry0iuDqW2hmv+RE6uTZ0vRgS5uozk6lBqa1j8iZyc1C4jucjVZSRXh1Jbw+JP5OTa0vli\nxJCry0iuDqW2hmv+RE5OapeRXOTqMpKrQ6mtYfEnIrKSs3ZWNYXFn4jISs7aWdUUFn8iIisNGDAA\nR48edbrOqqao5U6AiMhZNHQKtQfs9iEispKzdlY1hcs+RERWctbOqqaw+BMRKRDX/ImIFIjFn4hI\ngVj8iYgUiMWfiEiBWPyJiBSIxZ+ISIFY/ImIFOj/AaB4uxoTJ1cGAAAAAElFTkSuQmCC\n", "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "plot = sns.barplot(data=importances)\n", "for i in plot.get_xticklabels():\n", " i.set_rotation(90)" ] }, { "cell_type": "markdown", "metadata": { "collapsed": true }, "source": [ "From the above chart, the three most important features for this application are: $n_e$, $\\mathrm{mfd}_{e \\rightarrow ag}$, and $\\mathsf{ACC}$." ] } ], "metadata": { "kernelspec": { "display_name": "Python 3", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.6.2" } }, "nbformat": 4, "nbformat_minor": 2 }