{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "# Reproducibility\n", "---\n", "You can follow this Jupyter Notebook in order to reproduce the figure 3 of our paper about reproducibility in bioinformatics ([Kim et al. BioRxiv 2017](http://www.biorxiv.org/content/early/2017/06/20/143503)).\n", "\n", "Our StratiPy project is based on NBS (Network Based Stratification) method ([Hofree et al. Nat. Meth. 2013](http://www.nature.com/nmeth/journal/v10/n11/full/nmeth.2651.html)).\n", "\n", "---\n", "\n", "# 1 - Before starting work\n", "\n", "## 1.1 - Virtual environment\n", "We recommend that you use this Python 3 project in a virtual environment.\n", "For anaconda users, you can activate a virtual environment with:\n", "```\n", "~$ conda create -n python=3.6 anaconda\n", "```\n", "\n", "Then you can activate your virtual environment with:\n", "```\n", "~$ source activate \n", "```\n", "\n", "When you want to deactivate your virtual environment:\n", "```\n", "~$ source deactivate\n", "```\n", "\n", "More details on [Conda's environments](https://conda.io/docs/user-guide/tasks/manage-environments.html).\n", "\n", "---\n", "\n", "## 1.2 - Data\n", "\n", "- General input data was downloaded from [original author's NBS website](http://chianti.ucsd.edu/~mhofree/NBS/). In the original study, authors worked on:\n", " - 4 types of cancer data provided by [TCGA](https://cancergenome.nih.gov/). Here we only work on uterine endometrial carcinoma (uterine cancer) with 248 patients' somatic mutation data.\n", " - 3 types of Protein-Protein Interaction (PPI) networks. Here we only utilize [STRING database](https://string-db.org/).\n", "- [data](../data/) folder includes input and output data such as mutation profiles, PPI networks, similarity matrices, etc...\n", "- In this case study, the code will take general input data from [data](../data/) folder but output will be placed in the specific [reproducibility data](reproducibility_data/). Original MATLAB results (100 and 1000 permutations of bootstrap) are also in reproducibility data folder.\n", "- Result's filename will be constituted by parameters' value. If there is same filename, the code will not create new file. If you want to create a new one, you have to remove previous file from the specific [reproducibility data](reproducibility_data/).\n", "\n", "---\n", "\n", "## 1.3 - Parameters\n", "There are several tuning parameters of NBS. In order to reproduce same result, you only use same values outlined in the original NBS study except two ajustable parameters.\n", "\n", "1. **Graph regulator factor (lambda)** is the most influential parameter in this case study. It was thought that this factor had to be a constant value of 1 until we found that its value was changing and converged to 1800. Here you run StratiPy with lambd = 1 and 1800 in order to compare them.\n", "2. **Permutation number of bootstrap** utilized in the original study is 1000. But we recommand to start with 100 permutations since this step is highly time consuming (optimization ongoing ).\n", "\n", "Details about each parameter are explained in docstring of [reproducibility.py](reproducibility.py)." ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": true }, "outputs": [], "source": [ "# Defalut settings\n", "data_folder = '../data/' \n", "patient_data = 'TCGA_UCEC' # Uterine endometrial carcinoma (uterine cancer) with 248 patients' somatic mutation data\n", "ppi_data = 'STRING' # STRING PPI network database\n", "influence_weight = 'min' # Influence weight of propagation on the network\n", "simplification = True # Simplification after propagation\n", "compute = True\n", "overwrite = False\n", "alpha = 0.7 # Diffusion (propagation) factor\n", "tol = 10e-3 # Convergence threshold during diffusion\n", "ngh_max = 11 # Number of best influencers in PPI network\n", "keep_singletons = False\n", "min_mutation = 10\n", "max_mutation = 200000\n", "qn = 'mean' # Quantile normalization (QN) after diffusion is based on the mean of ranked values\n", "n_components = 3 # Desired number of subgroups (clusters)\n", "run_bootstrap = True\n", "run_consensus = True\n", "tol_nmf = 1e-3 # Convergence threshold of NMF and GNMF algorithm\n", "linkage_method = 'average' # Linkage method of hierarchical clustering" ] }, { "cell_type": "markdown", "metadata": { "collapsed": true }, "source": [ "# 2 - Let's launch!\n", "Import code" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": true }, "outputs": [], "source": [ "import sys\n", "import os\n", "sys.path.append(os.path.dirname(os.path.abspath('.')))\n", "from stratipy import load_data, formatting_data, filtering_diffusion, clustering, hierarchical_clustering\n", "from nbs_functions import all_functions\n", "import scipy.sparse as sp\n", "from scipy.io import loadmat, savemat\n", "import numpy as np\n", "import time\n", "import datetime\n", "from IPython.display import Image, display\n", "from importlib import reload" ] }, { "cell_type": "markdown", "metadata": { "collapsed": true }, "source": [ "\n", "\n", "## 2.1 - With Graph regulator factor (lambda) = 1 " ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": true }, "outputs": [], "source": [ "lambd = 1\n", "n_permutations = 100" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### ! ! ! Clustering part will take few hours ! ! !" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": true }, "outputs": [], "source": [ "lambd1_perm100 = all_functions(data_folder, patient_data, ppi_data, influence_weight,\n", " simplification, compute, overwrite, alpha, tol, ngh_max,\n", " keep_singletons, min_mutation, max_mutation, qn, n_components,\n", " n_permutations, run_bootstrap, run_consensus, lambd, tol_nmf,\n", " linkage_method)" ] }, { "cell_type": "markdown", "metadata": { "collapsed": true }, "source": [ "\n", "## 2.2 - With Graph regulator factor (lambda) = 1800" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": true }, "outputs": [], "source": [ "lambd = 1800" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### ! ! ! Clustering part will take few hours ! ! ! (yes, once again) " ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": true }, "outputs": [], "source": [ "lambd1800_perm100 = all_functions(data_folder, patient_data, ppi_data, influence_weight,\n", " simplification, compute, overwrite, alpha, tol, ngh_max,\n", " keep_singletons, min_mutation, max_mutation, qn,\n", " n_components, n_permutations, run_bootstrap, run_consensus,\n", " lambd, tol_nmf, linkage_method)" ] }, { "cell_type": "markdown", "metadata": { "collapsed": true }, "source": [ "# 3 - Reproducibility analyses\n", "## 3.1 - Permutation number of bootstrap: 100 vs 1000, original work\n", "\n", "Due to time-comsuming task of bootstrap, we launched StratiPy with 100 permutations instead of 1000 which was proposed in the original NBS work. However you can always launch StratiPy with 1000 permutations if you want (you have to set \"n_permutations = 1000\").\n", "\n", "\n", "In [reproducibility data](reproducibility_data/), you can find results provided by original work with 100 and 1000 permutations.\n", "Before comparing testing results of StratiPy (Python), we have to verify whether any significant difference exists between 100 and 1000 permutations\n", "\n", "![1](img/confusion_matrices_1.svg)" ] }, { "cell_type": "markdown", "metadata": { "collapsed": true }, "source": [ "### Load cluster index of each patients from original results (NBS)" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": true }, "outputs": [], "source": [ "from confusion_matrices import get_cluster_idx, repro_confusion_matrix\n", "%matplotlib inline\n", "result_folder_repro = \"reproducibility_output/\"" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": true }, "outputs": [], "source": [ "# 100 and 1000 permutations of bootstrap\n", "nbs_100 = get_cluster_idx(result_folder_repro, method='nbs',\n", " n_permutations=100)\n", "nbs_1000 = get_cluster_idx(result_folder_repro, method='nbs',\n", " n_permutations=1000, replace_1by2=True)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Confusion matrix\n", "plot and save" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAX8AAAE+CAYAAABlQBCuAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3XmcHFW99/HPd5Zsk42sZA8JaxJli7KIGNwAZbvCo8ii\niIoSecDt+qDCNXBRXLiKyKLxgnCR5SqyKKAIKGiULUEDJCRBSCZAErJnliwz0/N7/jg1Saene6Ym\n3TPdNf1751WvTFWfrjqnpubXp0+dOkdmhnPOufJSUewMOOec63ke/J1zrgx58HfOuTLkwd8558qQ\nB39XdJLmSHpLkkk6rwD7mxzta2YBslfyJD0h6fpi58Mli7y3j8tG0mjgG8BJwHhgPfAC8BMze7iA\nx5kBvAh8BHgK2GJm2/LcZyUwElhvZi3557JnSZoMLAfeYWbzY6QfBjSbWX03Z831IlXFzoArPVHw\n+RtQD3wdWEj4lvg+4KfAxAIebt/o//utQDURM0sBawqxr1ImqY+ZNZnZxmLnxSWPN/u4bG6M/p9p\nZr8ys6Vm9rKZXQ+8vS2RpImS7pNUHy33Shqf9vocSS9JOlPSq1Ga+yWNaHsduC9K3irJou23Snow\nPUNt+0pbf5ukxyXVSWqQtFDScdFr7Zp9JB0r6RlJ26Mmph9J6pP2+hOSbpT0HUnrJa2VdI2knH8j\nks6Ljn2ipCWStkr6raQhks6Q9IqkLZJul9Q/7X0nSPqrpE2SNkp6RNJBabteHv3/XFSOJ9LPi6T/\nJ+kN4I20vF8f/XyApEZJn8w4XpOko3KVxZUfD/5uN1ETwgnADWbWkPm6mW2O0lUADwCjgeOiZSxw\nvySlvWUy8DHg34APAocC345euwb4bPTzmGiJ605gNfBO4BBgDrA9R5nGAb8H/hEd/9PAx4GrM5Ke\nDbQARwMXAV+M8t6RvsBXove+D5gJ/Ab4JHA6cBqh6Wx22ntqgGujvM8CtgC/S/swemf0/wmEc/KR\ntPe+h/ABfEJ0vN2Y2VLgS8BPJE2RNBK4Ffi2mT3VSVlcGfFmH5dpX0DAy52kex8hCE01sxUAks4C\n/hW99liUrgo4z8y2RGnmAp8CMLMGSZujn7vaTDMJuMbMlkTr/+og7WxgFTDbzFqBlyVdCvxM0uVm\ntjVKt9jM/iP6eZmkz0ZluauDfVcBX4iCLpLuJATf0Wa2Ptr2AOHD8b+isv4mfQeSPgXUEYL+PGBd\n9NKGLOdlO3C+me3IlSEzmyvpRMIH5AbgVeCqDsrgypDX/F0mdZ4EgIOAVW2BH8DMXiME2Wlp6Wrb\nAn9kFTAq30wCPwT+W9KfJH1T0oGd5PXpKPC3mQf0Ydc9Bwg3tNPFyeuOtsAfeQtY0xb407bt3I+k\nqZLujJrC6qLXK4h3L+WljgJ/ms8QynYscE50H8S5nTz4u0yvAEYImHsq/cZtc5bXOrvuWmn/IVS9\n207M5hA+ZO4nNNO8IOn8Luc0/7xm9iayGPt5kNAb6XPAEYSmqBbCh1FnGmOkAZgBDAH6AeNivseV\nEQ/+bjdRz5FHgIskDcx8XdLQ6MeXgbFRz6C216YQ2v0X55mNdbRv/z8kS15fMbPrzOzDwM2E2m42\nLwNHZty8PQZoIjSJ9BhJw4EDge+Y2WNm9jIwiN2bYJui/yv38BhDgdsJ91RuAG6XNHjPc+16Iw/+\nLpsvEGre8yX9n6gHyYGSLmRX08hj0c93SJoZ9ay5A3ge+FOex/8TcKik8yXtK+lrwLvaXpTUX9IN\nkmZFPXuOIATzXB86NxI+lG6UdJCkDwPfBa5Pa+/vKZsIz0x8NirbewjdZ9O/QawFtgHHSxotaUgX\nj/FTwgfofwD/j9Bl94a8c+56FQ/+rp2o7f4w4FHge4Qg/ydCj50vRWkMOJUQZP4cLWuA0/Ltr29m\njwBXEHoFLSD0GLoxLUkK2IvQi2UpobvoU8CXc+zvTeBEQvPKP4FbCDdxv5FPPvdEdN/hY4Sb5S8R\ngvLlwI60NC3AxYRvMqsIvapikXQucApwtpk1R/cHzgLOkHRmocrhks+f8HXOuTLkNX/nnCtDHvyd\nc64MefB3zrky5MHfOefKkAd/55wrQx78nXOuDHnwd865MuTB35WcbOP5lyJJFZJ+JmlDNO7+rGLn\nKakk7RXNszA1bducaB6CzGHCXQ6Sfi3pK7ESm1mnC2FkwN8CbxIGqTovS5rZhEkothOeynz3nqTp\n7iVOWQpVnlIobw+e1ycIwyXk/T7CgGRDi12mGHk/iTAOz9HA3kCfLGluja6ztmU9YWC3A0vhd7Cn\nv7duyPsPgF9kbBtMeMrZgEOKnccc+Y4TGyuB/0yLBcsJQ2xXZaQrRMx5G7ARGNJZ3uPW/AcSHkW/\nhDDmyG4kfQz4MfAdwiP0fwd+L2liV9L0kA7LAoUpTwmVl/QZq5LAzLZYNGlMidsXWG1mfzezNWbW\nlCPdY+yarOaDQH92zWBW8rr7+pE0gBDkb07fbmZ1hA/PHYSgVoo6jSeE8ZW+QBiy48Ao7WzCFKlA\n4WKomb0IvAac02nO9+CTroGMTzfgGeDnGdteAa7uSposx7qAMNZ5Zcb2O4Hfpn3yPh3lawvwLDBj\nT8tSqPLsSXmjNE8QBub6MWEQsE2EWlFFWhoBXyOMSLmNMAH6ORn7uIkwquM64LmM7f9FqB2sI1yI\nfQljzGwGVgLnZslTZu38VuDBtJ8tY5lMmG3qr1EZ2kYLPShjH9net3PfUbq+hJmv3iLUeJ4GjsmS\nxxsJfxjrCYOjXdN23rp6nXR2zCx5X5FjP7uVJdp2UvSe/l04Xlfz03Yuc5a7g/c8Qfbrp7Pf5xN0\ncu3mOEdnRPtTltf6R7/P73U1VvX0Qu548iBwW8a22zKu8YLFUMKAfvM6y2/ebf5RreBw4I8ZL/2R\n8HU4Vpocfk1oAvhA2vEGEgYU+6WkKsKgV/OAgwljo19LGPhrjxSiPHmUt83ZhPsxRxHGfL+AMKVg\nm6sIUxF+gTCm/dWEWak+nJbmHMKHxLuBT2Tsu55wrr5LOF/3A8sIUxDeRpgkpStTKl5CGFjtF+yq\n4b5O59MV5npfpu8TBkM7n1DjeRH4Q5Y8Zp2GcQ+vk86OeQlwJWEe3THAOzrY106SBkX7fdHM0muK\nnR0vTn4yz+XqTsrd0fnPdv109vuEzq/dbN4NLLAocmX4D2A4HdT8JX1DYS7ljpZ3d5KHnNTBPM4x\nzQOOa5twSNI04L3Aw9F6oWPos8A7lTZvdFb5froRhso14Ngsnz5L46bp4Hj3ArenrZ9DuOj6AcOi\n/b6nUJ/UhShPnuV9ghCIlbbtMuCN6OcaQm0/s63vWuDhtH28kGPfT6Wti1Cz+23atmpCO/YZGe/L\nWfPPlSbL8WsIgeeYuPuO3tMEfCLt9UqiqQlzlS3a9ijw3129TrpwzK+So8afUZaW6FpriPKxkrRv\nHZ0dr4vn4Pq09U7LneP8Z71+Ovt9dnbtdrCf+8moGUfbZxC+5fwWWNnB+4cRmuA6WvrneO9Awsi1\nywnt9rcD7yf8HQwnzBgX97rJVfMXYYTaVsJEP5bxeytoDCWMGGuEKVZz5jcJvX1+CZwWtQtCqFn8\nxsy2W5h45FbgEUkPSfpyMdrUu8HTFv0WI08B46IJOaYRPvj+kF6zAS4Epqa9Z0GOfe+cqjA6xlpC\nLbJtWzPh63reUy3mOV1hm6mEP8S/peUxRTgn0zLSZp2GcQ+uk64cM46/ECajOYRQa34c+KOkCTGP\nt0f5yfPvo931E/P32dG1m0t/QpBPP5aAucDPCR/gE3LNa2BmG83sX50sudrjvwQMBT4KnEmoDP0P\noYLV9u01199SXB8jfHs6izBU+ieA2ZI+ned+c2kra4c1/0IE//WET//RGdtHE8Z3j5sml4cINadT\nJY0ifCr/su1FM/sU4evsXwjjmC+VdHzXi7FTIcqTT3k70/Y7O5ldAeUQYDrhZmKbXNP9ZZtisLNp\nBzudVjGHfKYrjCOzmSBnOQp4nWRrmujM1rQg9Bzh5uZgQpNIvsfr8PU8yp3t+umu3+d6wvwM6T4P\nTCDMudD2oT4j25vzbPb5iZl9zsyeM7O/mtmXgfGED7TB0WsNeZbvB8A1Zna3mb1oZrcTvlG03fAt\ndAwdFv2/rqNM5R38LfRwWEBau3zkA4S70bHSdLD/HYS2/7MJn6BrCF8v09MsNLPvmdms6LVPdr0k\nO/eVd3nyKW/kiIx+zUcSJkuvI8xWtQOYlKV2Uxtj33si27SKB2esN5E27aDiTVfY7n1ZvBqlSZ/J\nq5LQptyl6SK7cJ0U7Ji5shItbd9mOzte3PxkPZedlLuz8992vLi/z46u3Vz+Qdo3mOg+xtXAF8ys\n3sxWAHXkbvf/KbtXhLIt87O90bL0KjOzVjNbZWFSnUIYQPv7Syl2VUwKHUNnAG+a2VsdZSrzF5dV\ndJN132i1Apgo6RBgo5mtJHyK3S7pWcJX088T2qh+mrabOGly+SXhq/I+wF0WZkNC0j6EWkhbP9sp\nhPaum/IoS9y8dpYmn/KOBa6VdCPhgv93QtsvZlYv6RrgmuiP7C+EdssjgVYzmxtj/131pyg/pxDu\naXyOUCtbkZZmBeEm02RC2+dGdk1X+DphEvEf0H7C82zv28nMGiXdBHxP0npC2+yXCDWeG4mhq9dJ\nIY6Zoa+kvaOf9yLcjK4BfhfneF3Izwp2P5dDgM92Uu7M9+x2/tOkTz/Z0e8z57XbgUeisg03sw3A\ndcCjZvbbtDQvkaPmHzVv5cp3t4oZT34HXCppObCI8I3py4TmpTaFjKHvJpzTjsW8kTGL9l3CDLg1\nLc1swoW0g/AJdWyW/XSaJsfxFb3PgLenbR9NuCH8ZrTPlYReEdX5lKVQ5dmT8rKru9z1hK6Xmwhd\nMyszzsf/Zde3gHWEm5sfSNtHu5uv2bYT/qjmZGxbA1yUtl5N6Aq6PlquoP0N3/0J7btb2dVl8L3R\n/rdH/x9P+w4D2d6Xue/0bo47yN3VM+uN4z28TuIcM+4N3/TrrI7QG+P0rhwvZn4yz+URnZU7x/nP\ndf10+PskxrXbwXl6itB77cPRe8dkvH4T8GSceNGTC/Fi46Dod1dLaI9/jdAluV83xJx+hA4xR3aW\nd5/GscRIegJ4ycwuKnZenOuKfK5dSScQng+YZuFmttsDkr4AnGpmH+wsbRJ6+zjnejkz+wPh2+X4\nYucl4ZoJrQKditXm75xz3c3Mrit2HpLOunDPz5t9nHOuDHmzj3POlSEP/s45V4Y8+DvnXBny4O+c\nc2XIg79zzpUhD/7OOVeGPPg751wZ8uDvnHNlyIO/c86VIQ/+zjlXhjz4O+dcGfLg75xzZciDv3PO\nlSEP/q6oJD0h6TPFzodz5caDvysIScdI+rukLZI2SvqbpHcUO1/Ouex8MheXN0mDCXPlXgj8CuhD\nmER6Rw/no8rMMicUd85l4TV/Vwj7A5jZXWaWMrNtZvZHM3tB0hxJv2xLKGmyJJOUXvGYKulZSXWS\nHpA0LC39JyTVStog6XJJKyS9P3ptjqR7JP1SUh1wnqS+kq6VtCparpXUN0p/nqR56RmP8rJv9POt\nkn4q6VFJ9ZKelDQpek2SfiRpbZTPFyXN6K4T6lx38+DvCmEZkJJ0m6QTJe3Vxfd/AjgfGAO0ANcB\nSJoG3AicHb02BBiX8d5TgXuAocAdwDeBI4FDgIOBdwKXdSEvZwP/CYwA/hntE+CDwLGED7ohwEeB\nDV3Yr3MlxYO/y5uZ1QHHAAb8HFgn6beSRsfcxe1m9pKZNQKXAx+VVAmcAfzOzOaZWRPwH9Ex0j1l\nZvebWauZbSME7yvNbK2ZrQOuAM7tQnEeMrO/mNkOwgfJUZImECbGHgQcSJj+9GUzW92F/TpXUjz4\nu4KIguF5ZjYemAGMBa6N+fbX036uBaoJNe+x6a+Z2Vba17Zfz1gfG+0jfX9jY+Zjt/2ZWQOwERhr\nZn8CrgduANZKmhvd63AukTz4u4IzsyXArYQPgUZgQNrLe2d5y4S0nycSatnrgdXA+LYXJPUHhmce\nLmN9FTApY3+rop93y4ukDvMiaSAwrO39ZnadmR0OTCM0//x7lvc7lwge/F3eJB0o6SuSxkfrE4CP\nA08T2s2PlTRR0hDg61l2cY6kaZIGAFcC95hZitCWf7KkoyX1AeYA6iQ7dwGXSRopaQShqajthvNC\nYLqkQyT1i/aX6UNRt9U+hLb/p83sdUnvkHSEpGrCh8h2oDXWCXKuBHnwd4VQDxwBPCOpkRD0XwK+\nYmaPAv8LvAAsIHQJzXQ74ZvCGqAfcDGAmS0C/i9wN+FbQAOwlo67kF4FzI+O9yLwfLQNM1tG+HB5\nDHgFmJfl/XcC3yI09xwOnBNtH0y4n7GJ0JS0AfhBB/lwrqTJLPNbs3OlKWqG2QzsZ2bLu2H/twJv\nmFlXegc5l0he83clTdLJkgZIqgGuIdTmVxQ3V84lnwd/V+pOJdxwXQXsB5xp/nXVubx5s49zzpUh\nr/k751wZ8uBfoiRdJGm+pB3RjUiXJhrD5+Zo3J96Sf+UdGKx81VKojGP1kRjES3zobNdOg/+pWsV\noYviLcXOSImqIjyN+x7CWDuXAb+SNLmIeSo13wWmmNlg4BTgKkmHFzlPrkR48C9RZnavmd2PDx6W\nlZk1mtkcM1sRjevzILCc0DffAdF4SVvbVqNlahGz5EqIB3/XK0SDyO0PLCp2XkqJpBslbQWWEB6U\ne7jIWXIlwoO/S7xoyIU7gNuicYVcxMxmE0YjfTdwLz08wY4rXR78XaJJqiAMD9EEXFTk7JSkaIKd\neYRB8i4sdn5cafBpHF1iSRJwMzAa+JCZNRc5S6WuCm/zdxGv+ZcoSVXRyJOVQKWkfhlTHzq4CTgI\nODmayMVFJI2SdKakgZIqJR1PGGn18WLnzZUGf8K3REmaQxhdMt0VZjan53NTeqK5dVcQ2rDTJ23/\nnJndkfVNZUTSSMKQ2AcTKnm1wHVm9vOiZsyVDA/+zjlXhrzZxznnypAHf+ecK0Me/J1zrgx58HfO\nuTLkwd8558qQB/8SJ+mCYueh1Pk56pifn451Zfh0SV9KGyb7Fkl9eyibBefBv/T5H27n/Bx1zM9P\nx2INnx49KHcp8D5gEjAFuKLbc9dNPPg758paF4ZP/yRws5ktMrNNwJXAed2dv+7iwwVkGDGs0iZP\nqC52NnaaOK6KmQf3K5kn8Za9MKDYWWinHwMYrGElc45KTamdn+000mQ7lM8+jj+uxjZsTMVKu+CF\nHYuA7Wmb5prZ3D047HTggbT1hcBoScPNLHHzbnjwzzB5QjXPPjKh2NkoWcePPaTYWXAJ94zlP7zQ\nho0pnn1kYqy0lWNe2W5mM/M+KAwEtqSt10X/DyKBky558HfOJY4BrbT29GEbgMFp60Oi/+t7OiOF\n4G3+zrnEMYxmS8VaCmgRYaC8NgcDbyWxyQc8+DvnEqo15r/OdGH49P8BPi1pmqS9gMuBWwtZpp7k\nwd85lziGkbJ4SwyXAdsI3TjPiX6+TNJESQ2SJgKY2R+A7wN/JgyRvZz2w64nhrf5O+cSqZXCdGCK\n5siYk+PlgRlpfwj8sCAHLjIP/s65xDEgVaDgX648+DvnEqlQNf9y5cHfOZc4BjT7LIR58eDvnEsc\nw7zZJ08e/J1zyWOQ8tifFw/+zrnECU/4unx48HfOJZBIkdfYcGXPg79zLnHCDV8P/vnw4O+cS5zQ\nz9+Dfz48+DvnEqnVa/558eDvnEscr/nnz4O/cy5xDJHycSnz4mfPOZdIraZYSxyShkm6T1KjpFpJ\nZ+VI11fSjyStkrRJ0o2SSmfe1y7w4O+cSxxDNFllrCWmG4AmYDRwNnCTpOlZ0l0KzARmAPsDhxGG\nhE4cD/7OucQJD3lVxFo6I6kGOB243MwazGweYaL2c7MkPxn4iZltNLN1wHXA+YUrWc/xNn/nXCJ1\n4YbvCEnz09bnmtnctPX9gRYzW5a2bSEwK8a+BYyXNMTMtnSauoR48HfOJY6ZSFnshov1Zjazg9cH\nAnUZ2+qAQVnS/gG4RNKfCdM+XhxtHwB48HfOue7WWriung3A4IxtQ4D6LGm/DQwF/gnsAH4OHAq8\nVajM9BRv83fOJU644VsVa4lhGVAlab+0bQcDi9od12ybmV1kZuPMbAqwAVhgZokbZ85r/s65xGm7\n4VuQfZk1SroXuFLSZwg1+VOAozPTShoXHX41cARwOfDpgmSkh3nN3zmXSClTrCWm2UB/YC1wJ3Ch\nmS2SNFFSg6SJUbqpwN+BRuA24FIz+2OBi9YjvObvnEucQj/ha2YbgdOybF9JuCHctv4XYHLBDlxE\nHvydc4nUGr+3j8vCg79zLnHCwG4e/PPhwd85lziGaI4/dIPLwoN/kd1wy2Zu+996XlyygzNPG8Qv\nfjw6Z9prf7aZ79+wia3bWjn9pIHc+N1R9O3b+4e1bbYmFjOfDbxFH/qyLzPYe+f9t93V2jJqWUqK\nFKMYz0EcSoV6d5Aox/NjRlce8nJZ9OqzJ+kiSfMl7ZB0a7Hzk82Y0VV844t78akzM58x2d0jf27k\ne9dv4tFfj2X5c5NZXtvMnGs29FAui2sJ/6CCCo7lZGbwTl7meRqyPEm/wdZQy1IO41iO4UNso4FX\nWVyEHPes8jw/ojXm4rLr1cEfWAVcBdxS7Izk8pEPD+S0EwcyfK+Oa1//8+t6zv/4YKYf0Je9hlZy\n2ZeHcdv/ZnsAsXdJWQtreYMpTKdKVQzVCEYyltWsbJd2FbWMZTIDNYRq9WEK01jNip7PdA8q1/Nj\nhJp/nMVl16vPjJnda2b3E57CS7TFS5t4+/Q+O9cPntaXt9al2LAxVcRcdb9G6hEV1GjXMCuDGEpj\nu6FYoJE6BjJ05/pAhtDEDppsR4/ktRjK+fykqIi1uOz8zCREQ2MrQwbt+nUNjn6ub0zcU+VdkqKF\nqoxbU1VU0UJzjrTVaemqd27vrcr1/BjxJnLxeX5z8xu+gKQLgAsAJo4rzVMysKaCuoZdgX5LXfh5\nUE3v/vyupIqWjODUQvNuQSw9bSot6LUFwMpefJmX6/kxoDneuD0uh94dOWIys7lmNtPMZo4cXpo9\nH6Yd0IcXFjXtXF+4eAejR1YyfFhp5rdQahiE0cpW23V/o54t1LQbhBFqGEx92qi6DWymD33po749\nktdiKN/zI1IxF5edB/8ia2kxtm9vJZUyUinYvr2VlhZrl+7cMwZxy111LF7axKbNKb79o4188mPZ\nhhvvXSpVxSjG8SqLSVkLm20961jFGNp3ZRzDJFaxnAaro9maeI2XGdM7nsTPqVzPjxGe8I2zuOx6\n9ZmRVCWpH2HShUpJ/SSV1HfFb1+7kZp9XuN712/mjt/UU7PPa3z72o2sfKOZwVNfZeUb4av5Ce+t\n4d9nD+V9Z7zJPu9YweSJ1cz56vAi575nHMhhtJLiSX7HizzDQRzGQA1hu23lz3Yf220rACO0N5M4\ngOd5knk8TH9qmMq0Iue++5Xr+Slkzb8LE7hL0lWS3pS0RdITOeb6LXkya1/L7C0kzQG+lbH5CjOb\nk+s9Mw/uZ88+MqE7s5Vox489pNhZcAn3jD1OnW3Mqz1m3PShNvtXx8RKe9mMhxZ0MpMXku4iVIY/\nDRwCPAQcbWaLMtJ9FLgWOAaoJXQlP97MDutyIYqsV9f8zWyOmSljmVPsfDnn8hNu+FbGWjrTxQnc\n9wHmmdlrZpYCfgnJ/PrUq4O/c663UiEf8so1gXu25py7gamS9pdUDXySMK9v4pRU+7dzzsURbvjG\nbjkaIWl+2vpcM5ubtt6VCdxXA/OApUAKeB14b9yMlBIP/s65ROrC07vrO2nz78oE7v8BvBOYAKwB\nzgH+JGm6WXRnPSG82cc5lzgFfsI39gTuhJvBd5vZG2bWYma3AnuRwHZ/D/7OuURqpSLW0hkzawTa\nJnCvkXQMYQL327Mkfw74P5JGS6qQdC5QDfyrgEXrEd7s45xLHDNobi1o3XU2YfTftYSBIHdO4A4s\nBqZF8/l+DxgF/BOoIQT9081scyEz0xM8+DvnEic0+xRlAvftwBeiJdE8+DvnEsnH7cmPB3/nXOJ0\nsauny8KDv3MugQrb7FOOPPg75xLJ5+fNjwd/51zihN4+vXsui+7mwd85lzhtD3m5PefB3zmXSN7s\nkx8P/s65xPHePvnz4O+cSyTv7ZMfD/7OucQxEy0e/PPiwd85l0je7JMfD/7OucTxNv/8+fcm51wi\nFXA8fyQNk3SfpEZJtZLOypHup5Ia0pYdkrJN+lLyvObvnEucbujnfwPQBIwmTNjykKSFZrbbhC5m\n9nng823rkm4FWguZkZ7iwd85l0iF6ucvqQY4HZhhZg3APEkPAOcCl8Z430kFyUgP8+DvnEscM2gp\n3GQu+wMtZrYsbdtCYFYn7zsdWAf8pVAZ6Uke/J1zidSFZp8Rkuanrc81s7lp6wOBuoz31AGDOtnv\nJ4H/MTOLm5FS4sHfOZc4XWzzX29mMzt4vQEYnLFtCJDzRm40veMs4LNxM1FqvLePcy6RzBRriWEZ\nUCVpv7RtBwOLcqSHcD/gb2b2Wh5FKCoP/s65RGpFsZbOmFkjcC9wpaQaSccApwC3d/C2TwC3FqIc\nxeLB3zmXOGaF7ecPzAb6A2uBO4ELzWyRpIlRf/6JbQklHQWMB35d6HL1JG/zd84lkEgVrrcPZrYR\nOC3L9pWEG8Lp254Cagp28CLx4O+cS6SY7fkuBw/+GZa9MIDjxx9e7GyUrAfffLbYWSh5J43z66e7\n+dg++fPg75xLHgvt/m7PefB3ziWST+OYHw/+zrnEsQLf8C1HHvydc4nkzT758eDvnEsk7+2THw/+\nzrnEMfPgny8P/s65RPKunvnx4O+cSyRv88+PB3/nXOIYotV7++TFg79zLpG84p8f/+h0ziWPFXQ8\nfyQNk3SfpEZJtZLO6iDtFEkPSqqXtF7S9wtWrh7kNX/nXDIVtup/A9AEjAYOAR6StNDMdpvQRVIf\n4NEo/ceAFGEO4MTx4O+cS6RCdfWUVEOYjH2GmTUA8yQ9QJit69KM5OcBq8zsh2nbXihIRnqYN/s4\n5xLHgNZSsFjKAAAYPklEQVRWxVqIJnBPWy7I2N3+QIuZLUvbthCYnuXQRwIrJP0+avJ5QtLbuqWQ\n3cxr/s655DGgcBO4DwTqMrbVAYOypB0PHEeY5vFx4BLgAUkHmllT3AyVAq/5O+cSySzeEkMDMDhj\n2xCgPkvabcA8M/t9FOyvAYYDB+VRlKLw4O+cSyaLuXRuGVAlab+0bQcDi7KkfSH2XkucB3/nXALF\n6+YZ56awmTUC9wJXSqqRdAyhWef2LMl/CRwp6f2SKoEvAuuBlwtXtp7hwd85l0yFq/kDzAb6A2uB\nO4ELzWyRpImSGiRNBDCzpcA5wE+BTcCpwClJa+8Hv+HrnEsiA2st3MBuZrYROC3L9pWEG8Lp2+4l\nfFNINA/+zrmE8lE98+HB3zmXTL3itmvxePB3ziWTB/+8ePB3ziVP1x7ycll48HfOJZJP5pIfD/7O\nuWQqYG+fcuTB3zmXSPKaf148+DvnkqdrD3C5LDz4O+cSSH7DN08e/J1zyeQ1/7x48C+CZtvB4tbn\n2MAa+tCXffV29q6YlDVtbetSam0JKVoYpQkcpMOpUCUAr7e+wipbTgNb2FsTmV5xRE8Wo9ts3JTi\n819Zz2NPbmP4sAr+8+vDOPMjA9ul27HDuOw7G7nnt41s22589LQa/uvK4VRXhxrhkleauOQbG/jH\nCzsYMbySqy8fxqkn1vR0cbpdszWxmPls4K1wPTGDvcNQNO3U2jJqWUqKFKMYz0EcuvN6SpzWYmcg\n2XxgtyJYYs9ToQqOrTiVGRVH8rItoMG2tEu3wVZTay9zWMUsjqk4mW3WwKv20s7X+6o/+1RMY6z2\n6cHcd79LvrGBPtVi5QsTufX6UVz89fUsXtp+3KwfXL+ZBQt3sOBP43hx3nj++WITV1+7GYCWFuOM\nT73Fh94/gNWLJ3HD90fwqYvW8cqrzT1dnG63hH9QQQXHcjIzeCcv83yO62kNtSzlMI7lGD7ENhp4\nlcVFyHEBtPXzj7O4rHpt8JfUV9LNkmol1Uv6p6QTi52vlLWw1t5git5GlaoZqpGM1FhW24p2aVfZ\nCsZqCgM1hGr1YUrF9N3SjdJ4Rmk81fTtuQJ0s8atrdz/cCPf+tpeDKyp4F1H9OOk42u4856Gdmkf\nfnQrs88fzLC9Khk5vJLZ5w/mtrvD/BtL/9XM6jUpLr5gMJWV4rhj+nPUO/pyx2/a7yfJUtbCWt5g\nCtOpUhVDNYKRjGU1K9ulXUUtY5m863piGqtZ0fOZLhBZvCXWvqRhku6T1BjFjLNypDtPUioa6bNt\nmVXAYvWYXhv8CU1arwPvIczKcxnwK0mTi5gnGqlHiBrtmiFuEENptMxZ5KDRtjCQoTvXBzKUJrbT\nZDt6JK/F8MqrzVRViv2mVu/c9vZpfbLW/DMZ8ObqFFvqsrcHmMHiJYkbebdD4XqqaH89tZuVEBqp\ny7iehtDEjuReT4Ud0vkGoAkYDZwN3CQp2xy+AE+Z2cC05Yk9LUIx9drgb2aNZjbHzFaYWauZPQgs\nBw4vZr5StFBF9W7bqqimhfbNESlaqFL1bunatvdWDVuNwYN2vywHDRL1je3/ij9wXH+u/+861m1I\nsWZtCzfeHALe1m2t7D+1mpEjKvnhjVtobjYefWIrf316O1u39a67hOF62v3WXRVVua8nyut6ikNS\nDXA6cLmZNZjZPOAB4Nzi5qx79drgn0nSaGB/skzNJukCSfMlzW+me2tBlVn+MFtobveB0JY2Zc27\npWvb3lsNHCDq6nevudfVGYNq2rfdXnrxUA6Z0YcjPvAms05ZzcknDKC6GkaPrKS6Wvz6ltH8/vFt\nTDpkJT/+WR2nn1zDuDEJvbmZQ7iedg/eHV5P9J7rqQvNPiPa/r6j5YKMXe0PtJjZsrRtC4FcNf9D\nJa2XtEzS5ZISeQLLIvhLqgbuAG4zsyWZr5vZXDObaWYzu7v9vIZBGMZW2zU3dD2bqVHm/NFQoyHU\ns3nnegOb6UM/+qj3tPFn2m9qNS0p41+v7QpSLyzewbQD+rRL279/Bdd+ZwSvPT+RJU9PYNheFRz2\n9r5UVIQPirdN68Nj945h1aJJPHjX3qyobWHmob3r3IXrqTXjetpCTbv5yKGGwdSz60ZwuJ76JvN6\nMsLwDnEWWN/29x0tczP2NhDatZPVAYNo7y/ADGAU4dvCx4F/L2jZekivD/6SKghzcTYBFxU5O1Sq\nilEax6v2EilrYbOtY52tYowmt0s7RpNDV07bQrM18Vrrot3StVorKUsBhmGkLEWrJbv/W82ACk47\nsYYrfrCJxq2t/O2Z7Tz0x62cdUb7rp5vrm5h1ZoWzIxnFmzn6h9t5rKv7GrTfnFxE9u3t7J1ays/\numkLq9em+MRHs/09J1elqhjFOF5lcXQ9rWcdqxhD+66eY5jEKpbTYHXheuJlxjC55zNdKIVr82+A\ndp+WQ4D6zIRm9pqZLY+akl8ErgTO2MMSFFUiv67EJUnAzYSbOB8ys5Lo53egDmdx63M8afdTTV8O\n0uEM1BC2WyNPtf6BoypOoJ9qGKExTNKBPN/659AvW+OZqhk797PcFrPcdrVirbFa9tH03dIk0Y+v\nHs7nvryeCW9bybC9Krju6hFMO6APK99o4dBZb/CPJ8YzcXwVr9U28+mL17NufYrxY6u46pvD+MCs\nATv3c+c9Dfzirnqam413HdGPh+/em759e1/XvwM5jMXM50l+RzV9OIjDoutpK0/xCEdxPP00gBHa\nm0l2AM/zZNTPfxxTmVbs7O+xAo7tswyokrSfmb0SbTuYLE3EWRgJnVJM1ovHRZX0U+AQ4P1mFquP\n32ANsyMqP9i9GUuwB19/tthZKHknjStqn4KS94w9Tp1tzCtg9p0wwcZ/8Uux0r721a8sMLOZHaWR\ndDchkH8GOBR4CDjazBZlpDsReN7M3pJ0IHAP8Gszu2IPilFUvbbZR9Ik4HOE4L8mrU/u2UXOmnOu\nEArb1XM20B9YC9wJXGhmiyRNjOJGWzva+4AXJDUCDxMmcv9OQcrTw3pts4+Z1ZLQr2POuY515QGu\nOMxsI3Balu0rCTeE29a/Cny1cEcunl4b/J1zvZxP5pIXD/7OuUTyyVzy48HfOZdMHvzz4sHfOZc8\nBW7zL0ce/J1zyeTBPy8e/J1ziaRkP8xedL22n79zzrncvObvnEsmb/bJiwd/51zy+A3fvHnwd84l\nkwf/vHjwd84lkwf/vPgNX+dc4ojQ2yfOEmt/MSdwz3jP45IsqTN5JTLTzrkyV/g2//QJ3A8BHpK0\nMHNI5zbR6MDt58pMEK/5O+eSqUBDOnd1AndJQ4BvAV/LuwxF5MHfOZdMhRvPv6sTuH8HuAlYsyfZ\nLhUe/J1zidQ2pn9nCzBC0vy05YKMXcWewF3STOBdwE+6oUg9ytv8nXPJFL/Nf30n0zjGmsBdUgVw\nI3CJmbWEKcKTy2v+zrnksYL29tk5gXvatmwTuA8GZgL/K2kN8Fy0/Q1J786zRD3Oa/7OuWQqUG8f\nM2uUdC9wpaS2CdxPAY7OSLoFGJu2PgF4FjgcWFeY3PQcD/7OuUQqcFfP2cAthAncN5A2gTuwGJgW\nzee78yavpH7Rj2+ZWUtBc9MDPPg755KpCBO4Z7y2gvC8WSJ58HfOJU/8bpwuBw/+zrnEET6qZ748\n+DvnEsmDf348+DvnksmDf148+DvnksmDf148+Dvnksdn8sqbB3/nXDJ58M+LB3/nXCLFnajFZefB\nP5vWVLFzULJOmvDOYmeh5D345rPFzkJJO/qExoLsx5t98uPB3zmXPP6QV948+DvnksmDf148+Dvn\nEsef8M2fj+fvnEsktVqsJda+pGGS7pPUKKlW0lk50p0paamkOklrJd0mKXMimETw4O+cS5648/fG\n/3ZwA9AEjAbOBm6SlG0O378D7zGzwcAUQuvJVXtajGLyZh/nXCIVqtlHUg1wOjDDzBqAeZIeAM4F\nLk1PGw3xnC4F7FuYnPQsD/7OuWQqXJv//kCLmS1L27YQmJUtsaRjgIcI0zpuBf6tYDnpQR78nXOJ\n1IWa/whJ89PW55rZ3LT1gUBdxnvqgEHZdmZm84AhksYBnwVWxM5JCfHg75xLpvjBf72Zzezg9QZC\nLT7dEKC+w8ObvSnpD8DdwGGxc1Mi/Iavcy55LAzvEGeJYRlQJWm/tG0HA4tivLcKmNr1AhSfB3/n\nXOK09fOPs3TGzBqBe4ErJdVEbfqnALe3O650djSpO5ImAd8GHi9g0XqMB3/nXDKZxVvimQ30B9YC\ndwIXmtkiSRMlNbQFfGAa8HdJjcDfgKWEdv/E8TZ/51wiFfIJXzPbCJyWZftKwg3htvVvAt8s3JGL\nx4O/cy55fGC3vHnwd84lko/nnx8P/s65RPLgnx8P/s655DG6cjPXZeHB3zmXSD6kc348+DvnksmD\nf148+DvnEscnc8mfB3/nXPJY/IlaXHYe/J1zyeSxPy8e/J1zieTNPvnx4O+cSx4DvNknLx78nXPJ\n5LE/Lz6qp3MukQo1pDOApGGS7pPUKKlW0lk50n1S0gJJdZLekPR9SYmsRHvwd84lklot1hLTDUAT\nMBo4G7hJ0vQs6QYAXwRGAEcA7wO+mn9pel4iP7Gcc2WugKN6SqoBTgdmmFkDME/SA8C5wKW7Hdbs\nprTVNyXdARxXmJz0LA/+zrnECQ95xY7+nU3gvj/QYmbL0rYtBGbF2PexxJvuseR48HfOJVP8UT07\nm8B9IFCXsa0OGNTRTiWdD8wEPhM7JyXEg79zLpG6UPPvTAMwOGPbEKA+57Gl04Crgfeb2fpCZaQn\nefAvsmZrYjHz2cBb9KEv+zKDvXdOF7q7WltGLUtJkWIU4zmIQ6lQZQ/nuPs12w4Wtz7HBtaEc6K3\ns3fFpKxpa1uXUmtLSNHCKE3gIB2+85y83voKq2w5DWxhb01kesURPVmMbrVxU4rPf2U9jz25jeHD\nKvjPrw/jzI8MbJduxw7jsu9s5J7fNrJtu/HR02r4ryuHU10tAJa80sQl39jAP17YwYjhlVx9+TBO\nPbGmp4vTdYWdyWsZUCVpPzN7Jdp2MDmacySdAPwc+LCZvViwXPSwXt3bR9IvJa2JumUtk1RyX8+W\n8A8qqOBYTmYG7+RlnqfBtrRLt8HWUMtSDuNYjuFDbKOBV1lchBx3vyX2PBWq4NiKU5lRcSQv24Ic\n52Q1tfYyh1XM4piKk9lmDbxqL+18va/6s0/FNMZqnx7Mfc+45Bsb6FMtVr4wkVuvH8XFX1/P4qVN\n7dL94PrNLFi4gwV/GseL88bzzxebuPrazQC0tBhnfOotPvT+AaxePIkbvj+CT120jldebe7p4uyB\neD194vT2MbNG4F7gSkk1ko4BTgFuz0wr6b3AHcDpZvZsgQvVo3p18Ae+C0wxs8GEX+ZVkg4vcp52\nSlkLa3mDKUynSlUM1QhGMpbVrGyXdhW1jGUyAzWEavVhCtNYzYqez3Q3S1kLa+0NpuhtVKmaoRrJ\nSI1lta1ol3aVrWCspuw6JxXTd0s3SuMZpfFU07fnCtADGre2cv/DjXzra3sxsKaCdx3Rj5OOr+HO\nexrapX340a3MPn8ww/aqZOTwSmafP5jb7g6tGUv/1czqNSkuvmAwlZXiuGP6c9Q7+nLHb9rvpySZ\nxVvimQ30B9YCdwIXmtkiSRMlNUg7v45fTmgSejja3iDp9wUvWw/o1c0+ZmnVwF1fFKcCC4qTo901\nUo+ooEa77isNYiibWJclbR0jGbtzfSBDaGIHTbaDPuo9wS2cE7U/J5blnNgWRmrczvWBDKWJ7b3u\nnGR65dVmqirFflOrd257+7Q+/OXv2zp9rwFvrk6xpS773VIzWLyk/TeIkmOFncbRzDYCp2XZvpJw\nQ7htPZHdOrPp7TV/JN0oaSuwBFgNPFzkLO2UooWqjM/fKqpoof3X7pC2Oi1d9c7tvUlmOSGUNec5\nUe8/J5kathqDB+3+pztokKhvbF/L/cBx/bn+v+tYtyHFmrUt3Hhz6NSydVsr+0+tZuSISn544xaa\nm41Hn9jKX5/eztZtCRk3obA1/7LT64O/mc0mdNl6N6Fdb0dmGkkXSJovaX5z+5e7TSVVtGQEqhaa\n2wW/trSptADYFgwre9mXt8osH34dnhPr/eck08ABoq5+92pvXZ0xqEbt0l568VAOmdGHIz7wJrNO\nWc3JJwyguhpGj6ykulr8+pbR/P7xbUw6ZCU//lkdp59cw7gxCelEYDEXl1WvD/4AZpYys3nAeODC\nLK/PNbOZZjazJ9uHaxiE0cpW29WjrJ4t1LTrdQY1DKaeXTc9G9hMH/r2uuaNcE4s45xspkZZzomG\nUM/mnevhnPTrdeck035Tq2lJGf96bdcH3wuLdzDtgD7t0vbvX8G13xnBa89PZMnTExi2VwWHvb0v\nFRXhg+Jt0/rw2L1jWLVoEg/etTcraluYeWgyzp9aW2MtLruyCP5pqght/iWhUlWMYhyvspiUtbDZ\n1rOOVYyhfVfPMUxiFctpsDqarYnXeJkxTO75THezSlUxSuN41V6Kzsk61tkqxmhyu7RjNDl05bQt\n4Zy0LtotXau1krIUYBhGylK0WvKDQc2ACk47sYYrfrCJxq2t/O2Z7Tz0x62cdUb7rp5vrm5h1ZoW\nzIxnFmzn6h9t5rKvDN35+ouLm9i+vZWtW1v50U1bWL02xSc+2uGzTaXBCA95xVlcVr32+7GkUcB7\ngQeBbcD7gY9HS8k4kMNYzHye5HdU04eDOIyBGsJ228pTPMJRHE8/DWCE9maSHcDzPBn18x/HVKYV\nO/vd4kAdzuLW53jS7qeavhykw6Nz0shTrX/gqIoT6KcaRmgMk3Qgz7f+OZwTjWeqZuzcz3JbzHLb\n1VV7jdWyj6bvliapfnz1cD735fVMeNtKhu1VwXVXj2DaAX1Y+UYLh856g388MZ6J46t4rbaZT1+8\nnnXrU4wfW8VV3xzGB2YN2LmfO+9p4Bd31dPcbLzriH48fPfe9O3bvvmo1Agr5ENeZUnWS0+gpJHA\nPYSHNSqAWuA6M/t5R+8brGF2hN7XAzlMqIqEtAcX0YOvJ7r7d7c7+oQ3WbBwR16fMENqxtqRB10Q\nK+0fF1yxoJPhHcpSr635m9k64D3Fzodzrpv00oprT+m1wd8514u1tfm7PebB3zmXSN6TJz8e/J1z\nCeQPcOXLg79zLnkMD/558uDvnEsmb/XJS7k95OWc6yVkFmuJtS9pmKT7JDVKqpV0Vo50MyQ9Imm9\npER/9fDg75xLpsIO7HYD0ASMBs4GbpI0PUu6ZuBXwKcLUoYi8mYf51zymEGqMO0+kmqA04EZZtYA\nzJP0AHAucOnuh7WlwFJJ+xbk4EXkwd85l0zxa/UjJM1PW59rZnPT1vcHWsxsWdq2hcCs/DJY2jz4\nO+eSKX7wX9/J8A4DgbqMbXWEoeB7LQ/+zrnkMSDG/LwxNUC7cdSHAPVZ0vYafsPXOZdABtYab+nc\nMqBK0n5p2w4GFuVI3yt4zd85lzxGwW74mlmjpHuBKyV9BjgUOAU4OjOtJAF9gT7Rer+wC+u5KQAL\nxGv+zrlkKmxXz9lAf2AtcCdwoZktkjRRUoOkthmWJhHmB2n7VrANWFrAUvUYr/k755KpgMM7mNlG\n4LQs21cSbgi3ra8ASn+2mxg8+DvnEsgHdsuXB3/nXPIY4EM658WDv3MumbzmnxcP/s65BCrc8A7l\nyoO/cy55DCxeH36Xgwd/51wyFe4J37Lkwd85l0ze5p8XD/7OueQx894+efLg75xLJq/558WDv3Mu\ngQxLpYqdiUTz4O+cS57CDulclnxgN+dcMhVuSOfYE7hHab8kaY2kOkm3SOpbsDL1IA/+zrnEMcBa\nLdYSU6wJ3CUdT5jX932EET6nAFcUokw9zYO/cy55rHCTuaRN4H65mTWY2TygbQL3TJ8EbjazRWa2\nCbgSOK9wBes5Hvydc4lkqVSsJYZcE7i3q/lH2xZmpBstafgeF6RI/IZvhno2rX/M7qktdj5Klnew\n6FS/scXOQcmblO8O6tn0yGN2z4iYyftJmp+2PtfM5qatd2UC94HAlox0RGk3xMxPSfDgn8HMRhY7\nD865jpnZCQXcXVcmcM9MOyT6P3GTvXuzj3Ou3HVlAvdF0Wvp6d4ys0TV+gFk/pScc67MSbqb0Imo\nbQL3h4CjzWxRRroTgFuB9wKrgfuAp83s0h7NcAF4zd8552JO4G5mfwC+D/wZqAWWA98qUp7z4jV/\n55wrQ17zd865MuTB3znnypAHf+ecK0Me/J1zrgx58HfOuTLkwd8558qQB3/nnCtDHvydc64MefB3\nzrky9P8B6Xt8C91I7HYAAAAASUVORK5CYII=\n", "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "repro_confusion_matrix(result_folder_repro, data1=nbs_100, data2=nbs_1000,\n", " plot_title='Confusion matrix\\n100 vs 1000 permutations of Bootstrap',\n", " lambd=1800, tilt=True)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "**Confusion matrix**: each row or column corresponds to a subgroup of patients (here three subgroups). The diagonal elements show the frequency of correct classifications for each\n", "subgroup: **a high value indicates a correct prediction.**\n", "\n", "\n", "As you can see, there is no significant difference between 2 results (100 vs 1000 permutations). We will now focus only on 100 permutations of Bootstrap." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## 3.2 - Graph regulator factor (lambda): 1 vs 1800, NBS vs StratiPy\n", "\n", "![2](img/confusion_matrices_2.svg)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Load cluster index of each patients from previously obtained results (StratiPy)" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": true }, "outputs": [], "source": [ "# lambda 1 and 1800, both 100 permutations of bootstrap\n", "stp_100_lamb1 = get_cluster_idx(result_folder_repro, method='stratipy',\n", " n_permutations=100, replace_1by2=True, lambd=1)\n", "stp_100_lamb1800 = get_cluster_idx(result_folder_repro, method='stratipy',\n", " n_permutations=100, replace_1by2_2by3_3by1=True, lambd=1800)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Confusion matrices" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAASwAAAE+CAYAAAA3Yy3lAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3XmYHFW5+PHv29Mzk1mSyTLZQxKSANnYQgBBVgUCIouK\nXpRFfrKIwL3gBbmKRgKCiqIiyiKbcBFE5CK7showrCbsJCRkG7JnMllmS2amu9/fH6dm0tPT012d\n9Ex3pd/P89STdPWpqlPdNW+fOlV1XlFVjDEmCEK5roAxxvhlAcsYExgWsIwxgWEByxgTGBawABGZ\nJSLrRERF5JwsrG+st67pWahe3hOR2SLy+17c3iwR+TBNmYL6DvwSkWrvczkq13XZEXkbsERkqIj8\nVkSWiEiLiKwSkb+LyBeyvJ2pwNXAhcBw4C9ZWO0Kb13vZmFdvW4H/ti/DPygJ+uU4EbgyPYXInKv\niDzVi9vvVSJylPd9VOe6LrkWznUFkhGRscCrQAPuD+E9XHD9PHA7MDqLm5vg/fuYZukeD1WNAmuz\nsa58JiIlqtqqqht7c7uq2gg09sa22vexN7bV3fZzte28pKp5NwHPAKuAyiTv9Y/7/2jgb7jA1gA8\nCoyKe38W8CFwOrDEK/MYUB33vsZP3vx7gacStjsL+DDu9d7Ai0A97o/nPeBo772x3vqmx5U/AngT\n2AasA34DlMS9Pxu4FfgpsAFYj2tJhFJ8Tud42z4B+BhoBp4AqoDTgE+ALcD9QFnccscD/wI2ARuB\nZ4FJce9rwjQ7/nMB/gdYCayPq/vvvf/vBTQB30zYXitwSDf78RBwe9zr67ztfiZu3grgzMTvItl3\nCBwV9x18BXje+2zmA8emOfZmA7d5n30t8G9vfhVwh/e9NAAvJ3y/7d/FScAi73v+JzAuYf3fBhZ7\nn8di4PyE9xW4GHcsNwGPJNm/e72yAlyJO7a3Ah+0f0Zx6zsQmOfV5x3gxPbPKNd/5zsUG3JdgSQH\nzEAgBlyVplzI+wJeA6Z70xvAXEDiDuZGXFDbBzgEqAH+4L1fCZznfYHDgGHxf5gJ2+v4I/FefwD8\nCZiIa6V9Ce8PkoSABYz0Dr7bgUnAF3EtsF8l/KFsAa4F9gS+BkSAr6f4DM4B2oAXgAO8/VvtvX7S\n2+ejcYHp8rjlvuJNe3hlHvb+eEriDnIFZnify8C4z6UBeACYCuwdV/ffx63/AlwgHwcM9vb16hT7\ncSHwcdzrObhg8X3v9QSvPqMSvwvvO/wLLigN86aSuO/gY1wQ2QO4D6gjyQ9hwvfQAPzK+24n4QLD\nHOBp4CCvPj/x9nF4wncxF/gssD/wCq5boP14/JJX5hLvO/5P7/VJcdtXXFA8z/v8xuNOuRWY7O1f\nlVf2emAh7gdhd+AbuOPsxLjPZj3wV+/7mgEswAJWVgPWQd4H+qU05Y4FosDYuHnjcMHumLgDe1v7\nF+zN+yGwOO71aXgtq7h595I+YNUT14pIKNv+x9IesK7HtXZCcWXOAVqA8rg/lNcT1vM8cFeKz+Ac\nbzt7xc270ftcqlPtT8J6KrxlDktW/4T11AKlCfNnExewvHl/w/2API07vS9Ksf2J3vaGA+Xe5/I/\nwLPe++clfGeJ30Wy76t9H74dN2+kN++wFHWZDbyfMO9zuB++soT57wJXJnwXn417f4z3ubYfj68C\n9yT5TOfEvVbgdwlljvLmx3+nFbhW1eEJZW8CnvH+fwGwmbgADZxJgANWPna6i89yk4DVqrq8fYaq\nLsW1MCbHlatR1S1xr1cDQ3a2ksCvgbtE5CUR+aGITExT1zdUNRY3bw6uJTAhbt77Ccv5qWuLqi6M\ne70OWKuqGxLmdaxHRMaLyIPeBY167/0Q/voGP1TVFh/lzsPt2xG405RodwVV9WNcK+wo4FDcKc5f\ngM+KSLE3f7aPbSYT/5mu9v5N95nOS3h9AC6Q1opIY/uEa7WMjysXA95qf6GqNXQ+Hifhgla8OXQ+\nXsG10tKZDPQB/pFQp+/E1WkSLvjG9/e97mPdeSsfO90/wf0CTML9Su8Ijft/W5L30gXqGF0DZ3Gn\nlajOEpEHcP1HM4CrReRCVb2nl+saSbJMuvU8heuD+jaurzCC69/x08Hb5KMMuD/mKu//I4Flacq/\njDt9XQ/8U1WXi8gG3Onpkez4VciOz0JVVUQg/WeauI8hXFA/PEnZ+oTXmqRMOonL+PmM2/fhJODT\nhPcSv/9dRt61sNRdcXoWuEREKhPfF5H+3n8XACO8K4rt740DRuD++HZGLe70JN5+Ser6iarerKon\nAnfjWhXJLAA+IyLxn/dhuI7XJTtZ14yIyCDcKdhPVfUFVV0A9KXzj1f7VbGiHdxGf1xH/43ALcD9\nItIvzWKzcQHrKLa3pmYD5wOjSN3Cat3Ruvr0NjAUiKnq4oRpfVy5EK5LAwARGY07Hhd4sxbg+rfi\nHUb64zXZ9zEfd+o8JkmdauK2t7eIVMQt95k028preRewPBfjWjhzReSrIrKXiEwUke+wvYn/gvf/\nB0RkunfP0AO4g+ulndz+S8D+IvItEZkgIlcSd6CJSJmI3OLdHzNWRA4m9YF3K+7AvVVEJonIicDP\ncf0+zTtZ10xtwl2FPN/btyNxFwPiW2rrcf0jM7z74aqSrCeV23FB/8e4vqgGXOBKZTbuFPIgOges\nM4ElqroyxbLLganecVLtnUZm0wu4U7nHReQEEdldRA4RkWtEJL7VFQFu8t7bD9fJ/5G3PMAvgbNE\n5GIR2UNE/hM4A/hFmu3X4FphJ4rIYBGpVNUG3A/CjXHH6X4icqGIXOAt96BXp3tEZIqIHIvrww2s\nvAxYXl/UNFyn8w24wPQS7irLd70yCpyC+8P4pzetBU713tuZ7T8LXIPrLJ+H68C9Na5IFBiA6zBd\niDt1fR34727Wtwp36rg/rqP2HuDPwFU7U88d4fWj/Qfu6uCHuEAyE/dr3V4mAvwXrsW4Gnjc7/pF\n5CzgZOAMVW3z+ru+AZwmIqenqFd7P9YiVa31Zs/Gtfxmp9nsnbjWxFzc8ZDYitkp3vH0BdwxeCfu\nO38YdwvH6riiLbhj5n9xt7CEgC+3H4+q+hjuyuB3cT9ulwIXqeqTaba/Cndz8/W4U9P2pwpm4i5A\nXIELjM/jrv4u85ZrxF2R3gP3Q34j7gcksGQn/7aNMYD3SNfvVbVLN4bJnrxsYRljTDIWsIwxgWGn\nhMaYwLAWljEmMCxgGWMCwwKWMSYwLGAZYwKjoAPWrj5Spend71hEBnhDbY+PmzdLRFpF5DHxHmTM\nBRH5q4hcnqvtZ0veB6weHi/8UtyjH2YH9PZY7gFwFW5ol/jnQ38NXIR7KmPfntioiBwhIk94w4h3\nl5fgWuCHO/CYVV7J+4DVk1R1i6puznU9dsSuNHTurrAvIlKOe5Tp7vj5qlqPe4SrBTdKbU+oxD1m\ndSnuGdAuVPUDYClB/4HO9YBcqSbcF504POxYkg8Ydy9xg7jhY8jhTJfBDZr2v7jB3NYA38MN1XJv\nmv2YjXsg+Le4h4834R6Eja9LumGLZ5N86N6UyyUs+yuvTC3u4C7FPUu4GTdEyVlxy6QcfjfFd+Nn\n2N6k+5JQ5gLcc3NFCfMfBJ7IYN+Tfcfpjp20+5Ckvqd5dZAk75V5x9MNvfA30wic0817PyZusMAg\nTvnewroU91DxH3HDvQzHje3t1xm4p9UPxQ1Lexnuwd8dXeZXuLGZvgQcgxvYLdkYSd2tN4Qbxvjb\nuD/Iy+Ler8CNFnkQboiVLcCTCa2PM3F/TIcDZ2ewXPv2G4CDcSNF3IQb334Rbnjp+3ADErYPq3Md\ncC5u5IzJwM+AP3gjTUD330265VLtS7y/4sbTOrZ9hjfc0Cm4oakz2fdM+d2HeIcD89SLDAl+DAwi\nRQtLRK6KH4ivm8nvsdadt4CDRKRsJ9eTO7mOmD5+MWbT9Rcx2bx76fpLmnLI4UyWwTW7W4HT496r\nwP263+tjHxYR9+sL/AhYmWKZxGGLZ5MwdK+f5ZLtFy5Q1OK1VLx5xd7+nYaP4XeTfQ8ZLudnXx4F\n7o97fSYuKPXJYN+TfcfdHjt+9yHJth8D7ksyfypumO4ngE9TLD8QN7xOqqmsu+Xj1pOqhbUPriU8\nPpO/wXya8nHE0WzakSGHu1tmPO6POn4I3CZJk9AzzhvqHTWe14GfiEg/Va33riz9BNcCGoxrjSUO\nW5w4dC8+l+u0X6qqIrIed6rTPq9NRDZ5+xo//G58nYtxY091J5PluuxLEn8C7hORcnXjhp0B/J+q\nboOM9j0TO7rvZbhT2A7eVcE7cEPSPI8bT6tKOw/ZDXQMXNnT6dLa+7cC28IKasBKO4SxZ0eGHN6R\nZbLBz7DFyYbO9TvccbL96m5fd3T43UyW8zMM8NO4/TlFRF7EnYbPiHt/R4Z6Tnfs7Oi+b8CNkRbv\nQmA3r86DvHlT6TquOyJyFenHRztBVf+VpkwqA71/a1OWymNBCFjJhr9NNoTxvqT+BdxZS3AH7IG4\nqy3tV4am4m+Y44NFROJaWZ/BJdGojxu2+CJV/ae37mmk+X52dDkf4offTTV6a+J343c5X1S1RUT+\nimtZVeMG+JsNO7Xv6Y6dHd2Hd3CZc/DqMhzX93W2utFBG8Ql/NibJAELd1Hm4TTbWJVBfZKZCqxS\n1XVpS+apIASs5biOwrG48/ONuJEfbxKRk3GjP34b90u2vKcqoaqNInIPcIO45AhrcP1QIfwlHhiB\nq/OtuIP2e7jOXeg8bPEKXNKGX9I1wUSiHV0uJVVtEJH24XcFl1+vEhdkY6p6h1d0OV2/Gz/LZeJP\nuIS1uwN/1u2Zh3Z031MeOxnse6JnccfGIFWtA24GnlfVJ+LKfIgLGl3szCmhdzGiPftSCBjtDdG8\nUVXjW4mHe/UMrHy/SgjuD6AV98tXi+ufuCduak9pv6MZdjJxBe4y+hO4IZk/wA3Lu83Hsg/gWiNv\n4vo07sZlf0Z9DFuczI4u51PK4Xc9yb4bP8tl4l+4lsVktl8d3Jl993PsZLwP6u5zegs43buaeCxu\nmOl479Mz92JNx7Xw3sH1T13j/f/a9gIi0gd3dfvOHth+r7HxsHaCiJTiEgT8UlV/laLcbFw+v0t6\nq26m94nI8bh77SZrijyMuSAiFwOnqOpxua7LzgjCKWHeEJH9cfkS38Klxvof79+/5LJeJj+o6j9E\n5BZcWrKadOV7WRsuAUagWcDK3H/jsqVEcBlwjtDUKahMAVHVm3Ndh2R2sP8w79gpoTEmMILQ6W6M\nMYAFLGNMgFjAMsYEhgUsY0xgWMAyxgSGBSxjTGBYwDLGBIYFLGNMYFjAMsYEhgUsY0xgWMAyxgSG\nBSxjTGBYwDLGBIYFrF2Elzb+vFzXw5ieZAErz4jIYSLymohsEZGNIvKqiByY63oZkw9sAL88IiL9\ncKmrvoPLoFKCSxyQjTHaM6lHWFV3KpGFMT3BWlj5ZU8AVf2zqkZVdauqPqeq74vILBHpSMIgImNF\nREUk/kdnvIi8JSL1IvK4iAyMK3+2iNSISJ2IzBSR5SJyjPfeLBF5RET+5KWiOkdESkXkJhFZ7U03\neWPYIyLniMic+Ip7dZng/f9eEbldRJ4XkQYReVlExnjviYj8RkTWe/X8QESSZpIxJpEFrPyyCIiK\nyH0icoKIJCbmTOds4Fu4vHsRXKopRGQycCsuv99woAqXFiveKcAjQH9chp8f4lJb7YfL23cQLq2Z\nX2fgsjJX44aSfsCbfxxwBC44VwFfA+oyWK8pYBaw8oiq1gOH4fIc3gnUisgTIjLU5yruV9UPVbUJ\nl6rqayJSBJwGPKmqc1S1FfgxXXMpvq6qj6lqTFW34gLOtaq6XlVrcamjzspgd55W1VdUtQUX/A4R\nkd1wyRD64pKgiqouUNU1GazXFDALWHnG+wM+R1VH4ZJujgBu8rn4irj/1+BSsFd76+h4T1Wb6dqq\nWZHwegSdM7/UePP8it9ee5LVEV425d/j8giuF5E7vL47Y9KygJXHVPVj4F5c4GoCyuPeHpZkkd3i\n/j8a15ppz1I9qv0NESkDBiVuLuH1amBMwvpWe//vVBcRSVkXLzPxwPblVfVmVT0Alxx1T1wWbGPS\nsoCVR0RkoohcLiKjvNe7AV8H3sBLKSYio0WkCvhBklWcKSKTRaQcl/X3ES+h5yPASSJyqIiU4LIa\nS5rq/Bn4kYgMFpFq3Glke6f/e8AUEdnPyyg8K8nyX/Bu0SjB9WW9oaorRORAETlYRIpxgW8bEEuy\nvDFdWMDKLw3AwcCbItKEC1QfAper6vO4hK3vA/Nwtz8kuh/XIlsL9MFLla6qH+GSaD6Ea201AutJ\nfbvEdcBcb3sfAG9781DVRbiA+ALwCTAnyfIPAlfjTgUPAM705vfD9c9twp1m1gG/TFEPYzpYXsIC\n5J2ibQb2UNVlPbD+e4GVqprJVUVj0rIWVoEQkZNEpFxEKoAbca2m5bmtlTGZsYBVOE7BdXqvBvYA\nTldrXpuAsVNCY0xgWAvLGBMYu3zAEpFLRGSuiLR4ncEmjvfM4N3ec4YNIvKuiJyQ63rlE+8Zy7Xe\ns4+LbBif3NnlAxauz+Y64J5cVyRPhXF3pR+Je7bvR8DDIjI2h3XKNz8HxqlqP+Bk4DoROSDHdSpI\nu3zAUtVHVfUx7AHbpFS1SVVnqepy7znCp4BluHunDOA9n9nc/tKbxuewSgVrlw9YJjPeg9Z7Ah/l\nui75RERuFZFm4GPczbfP5LhKBckClungPS7zAHCf9xyj8ajqRbhRJg4HHqWXB1U0jgUsA4CIhHCP\n9rQCl+S4OnnJG1RxDu5B8u/kuj6FyIZINoiIAHcDQ4EvqGpbjquU78JYH1ZO7PItLBEJeyMKFAFF\nItInYVhhA7cBk4CTvMH7jEdEhojI6SJSKSJFIjIDN4LGi7muWyHa5e90F5FZuFED4l2jqrN6vzb5\nxxtrfTmuTyY+8cS3VfWBpAsVEBEZjBueZ1/cD3wNcLOq3pnTihWoXT5gGWN2Hbv8KaExZtdhAcsY\nExgWsIwxgWEByxgTGBawjDGBUTABS0QuyHUd8p19RqnZ55N7BROwADvY0rPPKDX7fHKskAKWMSbg\neuzG0QEDQzpiVP48AbNpY4wBA/MnPq/8JDHxcu61RpopCZenL9hLdFt+DYjQRgvFlOa6Gh220USr\ntqRLiJvSjKMrtG5j1FfZee+3PKuqx+/M9nZWj0WUEaPCPPzU4J5afeBdPuPsXFch70UXLs51FfLa\nm7rzjzPWbYzy1rOjfZUtGv5J9U5vcCflTxPIGNPrFIgRy3U1fLOAZUwBU5Q29XdKmA8sYBlT4KyF\nZYwJBEWJBmjEFgtYxhS4GBawjDEBoEA0QAErf25MMsbkRAz1NfkhIgNF5G8i0uRlE/9GN+VKReQ3\nIrJaRDZ5adSK063fApYxBUyBNlVfk0+34DIvDQXOAG4TkSlJyn0fmA5MxeXBnIbLOp6SBSxjCpii\nRH1O6YhIBfAVYKaqNnop0R4HzkpS/CTgd6q6UVVrgZuBb6XbhgUsYwqZQtTn5MOeQERVF8XNew9I\n1sJKJMAoEalKVcgCljEFzN3p7m8CqkVkbtyUOHpFJVCfMK8elzE70T+AS0VksIgMA/7Lm5/yYVa7\nSmhMQROi+H5+eoOqTk/xfiPQL2FeFdCQpOz1QH/gXVyKuTuB/YF1qSpgLSxjCpjrdBdfkw+LgLCI\n7BE3b1/goy7bVd2qqpeo6khVHQfUAfNUNeVt99bCMqaAufuwdmqEmu3rUm0SkUeBa0XkPFyL6WTg\n0MSyIjLS2/wa4GBgJnBuum1YC8uYAhdT8TX5dBFQBqwHHgS+o6ofichoEWkUkfaxbMYDrwFNwH3A\n91X1uXQrtxaWMQUsmy0sAFXdCJyaZP6nuE759tevAGMzXb8FLGMKmCJEA3SiZQHLmAKXwelezlnA\nMqaAKUKrFuW6Gr5ZwDKmgLkbR+2U0BgTENnsdO9pFrCMKWCqQlSthWWMCYiYtbCMMUHgOt2DEwaC\nU1NjTNZZp7sxJlCidh+WMSYI7E53Y0ygxOwqoTEmCNzDz8EJWMGpqTEm6xShTYt8TX5kkOZLROQ6\nEVklIltEZHY32XU6CXQLa8vmGDOv3Mzrr7TQf2CIy67sy4mndh0SWlX53Y0NPPbXZpqblYlTivnR\nT6qYsKdLg7bkkzaun7mF+R+2MWBgiMuv6scxx5f19u5kXWt0Kx+teYa6puUUF5Wxx+AjGVHV9Zho\naKll4fqXqN+2lrboVmZM/H6n92s2zWP1lg9oaKlleN9J7D3ii721C72qTVuZz1zqWEcJpUxgKsM6\nhm/qrEYXUcNCokQZwigmsT8hCc4zee1UyfaNo/FpvvYDnhaR91Q1cdTRr+Ky5BwG1ADXAffj0n11\ny1dNReQSb9D5FhG5N7P695zrZm6huBhenjeUG37bn5/8aAuLF7V1Kffs09v428PN3PdINa++N4x9\np5Xwg8s2AxCJKP91/kaO/HwfXn1vGLN+1p8fXLaZ5Usjvb07Wbdg7XOEpIij9vhP9hlxEgvWPUdj\nS22XciFCDOs7kanDTki6nj7hSsYNOpRRVfv0dJVz6mPeIUSIIziJqRzEAt6mUbd0KVena6lhIdM4\ngsP4AltpZAnzc1DjbBBiPqe0a8oszdfuwBxVXaqqUeBPwOR02/AbWlfjIuA9Psv3uObmGM//fSv/\neXk/yitCTDuwlKOP7cOTj27tUnbVigjTDixht9FhioqEk75UxpLFLrAtWxJh/boYZ59XQVGRcPBn\nS9lveglPPtrc27uUVZFYK+saFjJh8BGEQyUMKN+NIZUTWL2ly/DaVJQOYlT/fakoHZx0XUP77sXQ\nvntSXBT8Vmd3ohphPSsZxxTCEqa/VDOYEazh0y5lV1PDCMZSKVUUSwnjmMwalvd+pbNAcS0sP5MP\nmaT5eggYLyJ7ehmfv4nLpJOSr1qo6qOq+hhuoPi8ULM0SrhIGDtu+1ntXpOKk7awTjipjBU1UZYv\njdDWpjz+SDOHHdmn23WrKp8sCnYLq7l1IyIhKkoGdszrWzokaQvLQBMNCCEqZHtGqr70p6lL1ipo\nop5K+ne8rqSKVlpo1ZZeqWu2RQn5mshumq81wBxgIbAVd4r43XR1DWwfVnNzjIq+nZuplX2Fpqau\nGR8HDyli/wNL+OLR6ykqgmHDi7j7z4MAGDsuzKBBIf74h0bOOreSt15vYe6brRx0SGmv7EdPicba\nCIc670NRUSmRWGuOapTfokQIJ/w5hAkToesPoCtbHFeuuGM+BOu4UTIarz2bab5+DBwE7AasBc4E\nXhKRKara7elNVnvbROSC9ui7aWPKbD07rbw8RFND5+DUUK9UVHT98G/7bQMfvtvKC28MZd6i4Xzn\nsr6c+/U6tm6NUVws/PbOgbzyUgtHTV/LfXc2MePEMoYOC14HaryiUDGRWOdf/Ei0hXCoJEc1ym9F\nhInQuVUdoa1TYIovG40LZO1BrSiAv/8uzVfY1+SD7zRfuA75h1R1papGVPVeYABp+rGyGrBU9Q5V\nna6q0wcM7Nk7JsaMKyISVWqWbT/IFi5o67jyF+/j+W0cf3IZw4YXEQ4Lp361nPr6GEs+ccvuNamY\nex+u5tX3hnPH/YNY+WmEvffrup4gKS8ZiGqMptaNHfMaWtZT2U0/VaGroC9KjGbd3hhoYAsVXRoM\nUEE/GtjeGd/IZkoopUSC1bpyXCJVP1M6qtoEtKf5qhCRw3Bpvu5PUvzfwFdFZKiIhETkLKAYWJxq\nG4G9D6u8PMQxx/fh979uoLk5xtv/bmH2C9s46ctdO4an7lvCc09vY0NtlFhMeeLRZiJtMHqM+9VY\nuKCNlm3K1q0x/viHRjasj3HqaSkzZue9cKiEoX33YnHtv4jEWtnUvILaxsVJb2tQVaKxCO5iDURj\nEWKx7T8EMY1578dQXNlY6nyXgVMkYYYwkiXMJ6oRNusGalnNcLre1jCcMaxmGY1aT5u2spQFDM88\nAUxeUNyd7n4mn/ym+boB1yH/LrAZ13/1FVXdnGrlvtp5IhL2yhYBRSLSB3c1IKc90zOv68/M723m\nyGnrqBoQYuZ17t6qNasinHxMLU+8MJjhI8Oce2ElGzdEOe2EWrZuVUaPCfOb2wfQr8p9CU8+2syj\nDzXTFoEDDizhzgcGUVIanAdCuzNp2HF8tOYZZn/yO4qLypg09DgqSweztW0Lry69i8+OO4+y4iq2\ntW3hlaW3dyz3wqIb6RPux5ETLgJg6YZXWVL3asf7a+o/YvygzzJh8OG9vk89aSLTmM9cXuZJiilh\nEtOolCq2aTOv8yyHMIM+Uk61DGOM7sXbvOzdhzWS8emvyOetHKX52gZc7E2+iWrXTuouhURmAVcn\nzL5GVWd1t8yUfUr04afs9KM7l884O9dVyHvRhSnPDgrem/oi9bpxp6LNyCn99aKHD/NV9kdTn56X\nptO9x/lqYXmBaVaP1sQY0+tcp3twLjAF77KGMSaLbEx3Y0xAuE734PTXWsAypsAFaXgZC1jGFLAM\n73TPOQtYxhQ4S0JhjAkEVWiLWcAyxgSAOyW0gGWMCYhs3une0yxgGVPA7LYGY0yA2CmhMSZA/IzX\nni+CE1qNMVnnrhIW+Zr8yCDN1+3ecDPtU4uIJBuZtBNrYRlTwHrgxlFfab5U9ULgwvbXXjautIOs\nWcAypsBl65QwLs3XVFVtBOaISHuar+/7WC5twksLWMYUsCxfJewuzddRaZb7ClALvJJuAxawjClw\nGVwlrBaRuXGv71DVO+JeZ5LmK943gf9VH6OJWsAypoCpChH/ASubab4A8MZ4Pwo4308F7CqhMQUu\npuJr8iGTNF/tzgJeVdWlfjZgAcuYAtbeh5WNgJVhmq92ZwP3+q2vBSxjClwWW1jgP80XInIIMAr4\nq9+VWx+WMQUs2/dh+U3z5c17HajIZP0WsIwpcEF6NMcCljEFTBUiNoCfMSYobHgZY0wgWBIKY0yg\nqAUsY0xQWKe7MSYQVK0PyxgTGELUrhIaY4LC+rCAlQuquPyAtONxFaxnPngk11XIezNG7JfrKuzy\nLGuOMSanlZs5AAAWbUlEQVQ41PVjBYUFLGMKnF0lNMYEggas0z04NTXG9AhVf5MfftN8eWXHichT\nItIgIhtE5Bfp1m8tLGMKXJavEvpK8yUiJcDzXvn/AKK4JBYpWcAypoC51lNO0nydA6xW1V/HzXs/\n3TbslNCYApfBiKPVIjI3brogYVXdpfmakmSznwGWi8jfvdPB2SKyd7q6WgvLmAKXwW0N6bLmZJLm\naxRwNG7M9xeBS4HHRWSiqrZ2twFrYRlTwBQhFgv5mnzIJM3XVmCOqv7dC1A3AoOASak2YAHLmAKn\nPicfMknz9b7/1W5nAcuYQuZ1uvuZ0q4qszRffwI+IyLHiEgRcBmwAViQahsWsIwpdFlsYuEzzZeq\nLgTOBG4HNgGnACen6r8C63Q3puBl8z6sDNN8PYprkflmAcuYAqZALGbPEhpjgkABG17GGBMUNryM\nMSY4LGAZY4LB3y0L+cICljGFzlpYxphAUFC7SmiMCQ4LWMaYoLBTQmNMYFjAMsYEgt04aowJErtx\n1BgTHAG6SmjDyxhT4ET9Tb7W5TPNl4icIyJRb8iZ9umodOu3FpYxhSyzsa788JXmy/O6qh6Wycqt\nhWVMQRPX6e5nSrem7Wm+Zqpqo6rOAdrTfGWFBSxjCp3/EUezmeYLYH8vxdciEZkpImnP+AJ9Stga\n28ZHja9Q17qS4lAf9ig/iBF9JiQtu3zr+yxrfo8oEYaV7M7kysMJSREAb21+ki2R9Yh3x29pUQWH\nD/iPXtuPnrJxU5Tz/ns9z7/cTPXAIq6/ahDf+HLXjEstLcoPrt/Aw080snWbcvqpldz0k8EUF7vP\n43NfXskbb7cQdh8XI4eHWTBnTG/uSq9o01bmM5c61lFCKROYyjA3om8XNbqIGhYSJcoQRjGJ/TuO\np8CJ+S6ZzTRfrwBTgRpcQPsLEAF+lqoCgQ5YCxpfJUSIowadRUOkjrfr/06/8EAqwwM7ldvQuoJl\nze9xYNWJlIYqeKf+ORY3z2XPioM7ykyq/Cyj+kzs7V3oUZdcVUtJibDmg91598MWTjprDftOKWHK\nXqWdyt3w+03Me6+F9/85mmgUTvnmaq6/aSOzvjeoo8zN11dz3hlVvb0Lvepj3iFEiCM4iUY28w5z\nqNQqKqXzftfpWmpYyDSOoJQy3uM1ljCfPUibBzT/ZPc+LN9pvlR1adzLD0TkWuB7pAlYaU8JRaRU\nRO72evwbRORdETkhfd17VkTbWNe6jAkV0wlLMQOKhzGkZCyrWz7pUnZVyyJG9tmLyvBAikOljC+f\nxqpti5KsddfR1Bzj0acbufbKgVRWhDjs4DJOnlHBnx7pmiLuqeeauPjcKgYOKGJwdRGXnNufPz6U\nLJXcriuqEdazknFMISxh+ks1gxnBGj7tUnY1NYxgLJVSRbGUMI7JrGF571c6S7J4lTCTNF+JFB8P\nNfrpwwoDK4AjcdHyR8DDIjLWx7I9pjm6BUGoKOrfMa9v0SAaI5u6lG2MbKJveHtroW94EK26ldbY\nto55i5re4qW6+3hz8+NsbF3ds5XvBYuWtBEuEvYcX9Ixb98pJXy0MGVSEsDdSLhydYQt9dGOeT/8\naR1DJi/l8JNXMvu15h6pcy410YAQokK2n730pT9NXc5woIl6Ktl+3FVSRSsttGpLr9Q167KUNSeT\nNF8icoKIDPX+PxGYieugTyltwFLVJlWdparLVTWmqk8By4AD0u9Cz4lqG2Ep6TSvKFRMRNvSlm3/\nf9Qru2fFwRwx4HSOGngmo/pM4u2GZ2mOdj1Qg6SxKUa/vp2/3r6VIRobu3ZYzDi6nN/dtYXaDVHW\nro/w+7s3A9C81R2lP/tRNYvfHMuKd3bnvDP7ccrZa1iyvOvnHGRRIoQTekjChImQ5HgiQpjiuHLF\nHfONvzRfwOeB90WkCXgGF+h+mm7lGfdheVFxT5I087yrBhcA9AlVJr6dVUVSTCQhhVkk1kpYipOW\njcaVbV+uyCvbv3hIx3sj++zJmpbF1LZ+ypiyqT1R9V5RWRGivqFzcKpviFFZ2fU36qpLB7C5Psa0\nYz+ltEQ474x+vPNhC0MHu07kg6f16Sj7za/146G/NfL3F5u45Nz+XdYVVEWEiSQEnAhtnQJTfNlo\nXCBrD2pFAe0S9ntTqB9+03yp6hXAFZmuP6PbGkSkGHgAuE9VP05SqTtUdbqqTi8J9em6giwqL6pC\nUZqiWzrmNUTrqAwP6FK2MjyAhkjd9nKROkqkjO7qKAEaH6g7e44vJhJVPlm6PVC/91ErU/Yq6VK2\nrCzE7346mBXv7M7iN8cycEARB+xTSiiU/HMQCdbzZ35U0BclRrNu77trYAsVXfqQoYJ+NLD9uGtk\nMyWUUiKlXcrmPcU9muNnygO+A5aIhHDnoq3AJT1WI5/CUszQkrEsbppLRNvY1LaW2tYaRpTu0aXs\niNI9WbltIY2RTbTFWljS/DYj++wJQFushQ2tK4hqhJjGWL3tEza1raG6ZFRv71JWVZSH+NIXKrn6\nlxtpao4x582tPPlcE2ee1vUK86o1EVavjaCqvDFvG9fftImrr3B9fpu3RHn2n01s2xYjElEe+L8G\n/vXGVmYcXd7bu9SjiiTMEEayhPlENcJm3UAtqxlO19sahjOG1SyjUetp01aWsoDhjO39SmdLdjM/\n9yhfbVgREeBu3O32X1BN0lGUA5MqD+OjxpeZXXc/xaFSJlUcTmV4IFujjby66WE+O+BrlBVVMrhk\nN3Yv35d/b3mKKBGGluzOhHJ3O4kS45PmuTRFN3d04u/X77hOnflBdcvPBnPud9czbOoyBg0o4paf\nD2bKXqV8urKNqUd+yocvj2b0qGKWLG/jnP9ax/oNUXYbGeanVw3iuKNcQGprgx/fsJGPF7dSVCRM\nnFDMo38c3qkzf1cxkWnMZy4v8yTFlDCJaVRKFdu0mdd5lkOYQR8pp1qGMUb34m1e9u7DGsl4Jue6\n+jssm6eEPU3UR9teRG7HPRd0jKo2+llxVfFgPaT/l3eyeruuZz54KddVyHszRuyX6yrktTf1Rep1\n406dq5XutpuOuuy7vsouveLyeWluHO1xfu7DGgN8Gxew1sY9WX1Gj9fOGNPzdqVTQlWtIUij1Btj\nfMtk6Jh8EMzrsMaY7MmTK4B+WMAypsBZC8sYExwWsIwxgWB9WMaYQLGAZYwJCvE/gF/O2RDJxpjA\nsIBlTKHL4o2jftN8JSzzoojoLj+muzFmJ2W/0z2TNF94T8x0HcOnG9bCMqbQZamFlWmaLxGpAq4G\nrvRbVQtYxhS63KX5+ilwG7DWb1XtlNCYAiZkdJUwa2m+RGQ68FngUsD34HMWsIwpZNntw/KV5ssb\nDPRW4FJVjbjh9vyxU0JjCl32rhL6TfPVD5gO/EVE1gL/9uavFJHDU23AWljGFLostbBUtUlE2tN8\nnQfsj0vzdWhC0S3AiLjXuwFv4TJx1abahgUsYwpclm9ruAi4B5fmq464NF/AfGCyl0Gno6NdRNqz\nwaxT1ZS50ixgGVPocpDmK+G95fgcJNQCljGFTIP1LKEFLGMKnY3WYIwJChsPyxgTHBawjDGBkEcp\nvPywgGVMARPslNAYEyAWsIwxwWEByxgTGBawjDGBYGm+jDGBYgHLGBMU9mgOoJEo0bqNPbX6wJt0\n+0W5rkL++2tD+jIFrPXK17KyniCdEtoAfsYUMr+D92U5zZeInC4iC0WkXkTWi8h9IpI4WmkXFrCM\nKXRZDFh0TvN1BnCbiCRLQvEacKSq9gPG4c72rku3cuvDMqaAZfNO97g0X1NVtRGYIyLtab6+H1/W\nGx8rXhSYkG4bFrCMKXAS8x2xqkVkbtzrO1T1jrjX3aX5OirpdkUOA57GjfHeDHwpXQUsYBlTyDI7\n3ctami8AL9FqlYiMBM4HlqergPVhGVPgRP1NPvhK85VIVVcB/wAeSrcBC1jGFLreT/OVTBgYn66Q\nBSxjCly2Wliq2gS0p/mq8PqoTgbu77JNkTO8TDqIyBjgeuDFdNuwgGVMocvubQ0XAWW4NF8PEpfm\nS0Qa24MUMBl4TUSagFeBhbh+rJSs092YQpblrDl+03yp6g+BH2a6fgtYxhQwG3HUGBMsGpyIZQHL\nmAJnLSxjTDBY1hxjTJDYeFjGmMCwgGWMCQbFOt2NMcFhne7GmOCwgGWMCQK7cdQYExyqmQzgl3MW\nsIwpdMGJVxawjCl0QToltOFljClkCsTU3+RDBmm+viki87w0XytF5BcikrYBZQHLmEKXmzRf5cBl\nQDVwMPB54Ip0K7dTQmMKXI7SfN0W93KViDwAHJ1uGxawjClwuUrzleAIfIz9bgHLmEKWwzRf7UTk\nW8B04Lx0FbCAZUwBczeOZu0yYcZpvkTkVOBnwDGquiHdBqzT3ZhCF/M5pZdRmi8ROR64EzhJVT/w\nswELWMYUOFH1NaWTYZqvzwEPAF9R1bf81nWXOSVs01bmM5c61lFCKROYyrCOjEKd1egialhIlChD\nGMUk9ickRb1c454X3drE2if+QvOSRRSVV1D9+S/Qb+8DUi6z4n9vY+uyT9hj5i+RkPtMNr31L+rf\n/Tet69fQd+o0hp369d6ofo+LNjSz9rbHaX5vCUV9y6k+4xj6Hb5Pl3Jb/vkO6257HCkp7pg38vvf\noHzq7gC0rKxl/V1P07J0NUX9Kqg+6zj6Hjyp1/Zjp2R/xNGLgHtwab7qiEvzBcwHJnsZdGbiThef\nEZH2Zf+lqiekWrmvgCUifwKOwd07sRb4haretQM702M+5h1ChDiCk2hkM+8wh0qtolKqOpWr07XU\nsJBpHEEpZbzHayxhPnuwd45q3nPWP/MoUhRm/BXX0LJ2FasevIvSoSMpHTIsafn69+dBNNplfrhv\nFYOOOJamxQvRSFtPV7vXrL/raSRcxPi7vkfL8rWs+tkDlI4dRuluQ7qU7bPnboy+7twu8zUaZfUN\nf6b/cdMZNfNsts5fzqqfP0jpLy+kZER1b+zGTsrus4QZpPlKewtDMn5PCX8OjFPVfrgm3nUikvqn\nuhdFNcJ6VjKOKYQlTH+pZjAjWMOnXcqupoYRjKVSqiiWEsYxmTUs7/1K97BYawsN89+n+ujjCZWU\nUjZ6HBV7TaX+/blJy0e3baXu5eeoPvaLXd7rO2kfKifuTVF5eU9Xu9fEtrXS8OYCqk//HKGyUsom\njaFi+kTqX34vo/W0rtpAZFMD/b94CFIUonzvcZTtNTrj9eSUqr8pD/hqYanqh/EvvWk8MK8nKpWp\nJhoQQlTI9qunfenPJmqTlK1nMCM6XldSRSsttGoLJVLaK/XtDa11tUgoRMmg7a2F0qEj2FqzOGn5\nDS8+Q//phxKuTLzIs2tqXV3nPp+4VlDp2KFs/Wh50vIty9aw+P/dQFFlGf2O2IeBXz4cKequG0Fp\nWbE++5XuCVlOpNrTfHe6i8itItIMfAysAZ7psVplKEqEcELsDRMmQtfTF1e2OK5cccf8XUmstZVQ\naZ9O80KlpcRaWrqU3bZ6BdtWLKP/wYf1VvVyLratlVB55x+oUFkpsa2tXcqWTR7D2F9fzPi7v8eI\nK/6Dhlc/ZNPjrwJQMqKacL8KNj3+KhqJ0vTuYprn16AtATp1DlALy3fAUtWLcDeAHY67EtDlyBeR\nC0RkrojMbev6do8pIkwkIeBEaOsUmOLLRuMCWXtQK9p1rj8AECopIdayrdO8WMs2QqWd/0hVY6x7\n+hEGH39qRyd7IQj1KSHW3PkYjTW3ECor6VK2ZOhAiocOQEIhSscMZeBpR9LwxnwAJFzEiCtPp+nt\nRSw575dsevI1+h4yhfCgALVUs/ssYY/K6K9UVaO454POBL4D3Jzw/h3AHQD9ZGCv7WIFfVFiNGsD\n5d5pYQNbqOhyDxtU0I8GtjCU3QBoZDMllO5Sp4MAJYMGo7EYrXW1lAwaDEDLutWUDO7c4R5raaFl\n9UrWPOJdeVZ3frD019cy/KvfpHzMuF6td28pGTHIfT5r6igZPgiAlpq1lCTpcE8kIp3+gEvHDmO3\na7/V8frTq+6i31H7Zr3OPUViwTkn3NH7sMK4Pqy8UCRhhjCSJcwnqhE26wZqWc1wut7WMJwxrGYZ\njVpPm7aylAUMZ2zvV7qHhUpK6Ttpb+pm/4NYawtbP11K08KP6LdP5ycrQqV9GHf51Yy58HLGXHg5\nI79xPgCjL/guZaPc56exKLFIGxqLoRrz/t/1amKQhPqU0PegSdQ99BKxba1sXVBD09yF9Duya6Bp\nevsTIpsbAWhdVUvdIy9TeeBeHe+3LF9LrLWNWEsrGx9/lcjmBvodvX+v7ctOUbJ542iPS9vCEpEh\nwOeAp4CtuNsbvu5NeWMi05jPXF7mSYopYRLTqJQqtmkzr/MshzCDPlJOtQxjjO7F27zs3Yc1kvFM\nznX1e8SQE7/C2sf/wpIbr6aorJwhJ36F0iHDaNuyieW33MDYi/+H4qoBnTraNeJOrcOVfTtOEete\neZ6NLz/XUabh/XkMPPI4qo86vnd3KMuGnH8ia299nCXn/oKivuUMOf+LlO42hLbazSz/7i2M/c3F\nFA/uT/MHS1l7y9+IbWslXFVJ3yP2YeCXj+hYT/0r77HlxbfRaIyyiaMZNfNsQsXB6GIQ/N0Umi9E\n01RWRAYDj+BusQ8BNcDNqnpnquX6yUA9WD6frXrucj798aG5rkL+27vbR9AMUHPlH9i2ZJWkL9m9\nqooR+plJF/gq+9y8a+alefi5x6X9GVDVWuDIXqiLMSYXAtTCCka71RjTM9r7sALCApYxBS5IVwkt\nYBlT0PLnplA/bHgZYwqZktU73TPImjNVRJ4VkQ0i/keVt4BlTKHL7n1YfrPmtAEPA12HwEjBTgmN\nKXDZug8rw6w5C4GFIjIhk21YC8uYQpe9U8LusuYka2HtEGthGVPIVCHq+3wvXZqvHcqakwkLWMYU\nOv+nhOnSfGWcNSdTdkpoTKHL3ilhRllzdoQFLGMKmQIx9TelW1VmWXNERPoAJd7rPiLpx3iygGVM\nQVM3BpqfyZ+LgDJc1pwHicuaIyKNXvYcgDG40V/aW19bgYXpVm59WMYUMiWTTvf0q/OfNWc5LvF0\nRixgGVPoAvRojgUsYwqdBSxjTDAE6+FnC1jGFDIFbHgZY0xgWAvLGBMMGT2ak3MWsIwpZOqS6QaF\nBSxjCp2Pu9jzhQUsYwqd9WEZYwJB1a4SGmMCxFpYxphgUDQazXUlfLOAZUwhax9eJiBseBljCl0W\nh5fxm+bLK/tdEVkrIvUico+Nh2WMSUkBjamvySdfab5EZAYuk87ncWNjjQOuSbdyC1jGFDLN3gB+\ncWm+Zqpqo6rOAdrTfCX6JnC3qn6kqpuAa4Fz0m3DApYxBU6jUV+TD5mk+ZrivRdfbqiIDEq1gR7r\ndG9g04YX9JGanlp/4F3zSK5rYIJvzM6uoIFNz76gj1T7LN4ni2m+KoEtCeXwytZ1V4EeC1iqOrin\n1m2MyQ5VPT6Lq8skzVdi2Srv35QpweyU0BiTLZmk+frIey++3DpV7bZ1BSAaoLtcjTH5TUQewl18\nPA/YH3gaOFRVP0oodzxwL/A5YA3wN+ANVf1+qvVbC8sYk02+0nyp6j+AXwD/BGqAZcDV6VZuLSxj\nTGBYC8sYExgWsIwxgWEByxgTGBawjDGBYQHLGBMYFrCMMYFhAcsYExgWsIwxgWEByxgTGP8febDO\nrH6xdrkAAAAASUVORK5CYII=\n", "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "# between NBS and StratiPy (lambda = 1)\n", "repro_confusion_matrix(result_folder_repro, data1=nbs_100, data2=stp_100_lamb1,\n", " plot_title='Confusion matrix with reported\\ntuning parameter value',\n", " lambd=1)" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAASwAAAE+CAYAAAA3Yy3lAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3XeYG9XV+PHvUdvudVnbuGCMbQwuFIPpzYROQktIBQK/\nBFIIb0jeNF4ChBBIhfQALyF5IZQQIISemGqIKQGbahuwccPdXrftRdL5/XFn11qtdjXyastY5/M8\n89g7ujNzRxod3bkzc4+oKsYYEwSh/q6AMcb4ZQHLGBMYFrCMMYFhAcsYExiBD1gico2IbBARFZEL\n87C+8d66ZuahegOeiMwRkd/34fauEZEFWcrscp+BiMzy9qkq099BIiK/F5E5/bHtXglYIjJSRH4j\nIktFpFlE1ojIP0XktDxvZzrwA+ArwCjgb3lY7SpvXW/mYV19bie+7B8H/qc365TmBuDYtj9E5HYR\neawPt98lP8HU9K9IvlcoIuOBF4Fa3BfhLVxgPB64BRiXx81N8v59SPN0f4aqJoD1+VjXQCYiMVVt\nUdUtfbldVa0D6vpym2YXoqp5nYAngDVAeYbXBqf8fxzwD1xgqwUeBMamvH4NsAD4DLDUK/MQUJXy\nuqZO3vzbgcfStnsNsCDl732BZ4Aa3JfnLeA477Xx3vpmppQ/BvgP0ARsAH4FxFJenwPcBPwYqAY2\n4loSoW7epwu9bZ8KvAc0AI8AlcA5wBJgO3AnUJKy3CnAv4GtwBZgNjAl5XVNm+akvi/A94DVwMaU\nuv/e+//eQD1wQdr2WoDDu9iPe4FbUv6+ztvuYSnzVgHnpX8WmT5DYFbKZ/AJ4CnvvVkEnJjl2Ov2\nvfHKjAbuBjZ7630TOM77PNLrcmHKe3pO2npWAN9O+fu/gbe9928NcBsdj/dZ3nqq0v8GynDHYvo2\nTgRagZFd7O/t9OBY916fCjyO+35tBP4K7Jbyehh3LG/1pl8DN+MdV3095fWUUESG4g6aP6j7Je1A\nVbd55ULAw8BI3MFyHO5AekhEJGWR8cCngbOBk4AZwPXeazcAF3v/H+VNft0DrAMOAQ7AfchNXezT\nGOCfwBve9r8IfBb4SVrRc4E4cARwKfANr+7dKQK+5S17PDAT+DtwAe7LehbwMeCSlGXKcAfNIbiD\nfjvwqIjEvNcP8f49BfeefDxl2WOB/bzXjk+vjKq+D3wT+J2ITBCR4bgvxfWq+nIX+zDHq0ebWbig\nPQtARCYBY71y6W4A7gOeZsdn+FLK69cDvwX2B14D7hWR8i7qAVneGxEpA57HHVdnAW1dCuC6E24E\n3k+pSy5dDEncZz4N+JxXh9/5WVBV63GB4gtpL30BF5A25FCPdF0e6yIyCngB1zA4BDgBKAce9r6j\n4I7Pi4EvA4fjAti5PahPz+Qz+uF2WoGzs5Q7EUgA41PmTcB96Cek/FI0AZUpZb4PfJDy9zl4Lasc\nf3VqSGlFpJUdT0oLC/elWUJKawn3a9wMlHp/zwFeTlvPU8Bt3bwHF3rb2Ttl3g3e+1LV3f6krafM\nW+aoTPVPW88moCht/hy8FlbKvH8Ar+B+eV8Ewt1sfx9ve6OAUu99+R4w23v9orTPLP2zyPR5te3D\nl1PmjfHmHZXD8Zj+3lyMa0lUdVG+Q91S5mdtYWVY5hTvvQh5f8+iixaW9/dM3A/eGO/vIUAj8LFu\ntpHpvUt/f7s71q8FnkmbN8Sr1yHe32uB76e8HgIWsyu0sADJXgSAKcBaVV3RNkNVl3lvztSUcitV\ndXvK32uBET2tJPBL4DYReVZEvi8i+2Sp6yuqmkyZNxeIsaMPDdzpQCo/dW1W16ppswFYr6rVafPa\n1yMiE0XkHu+CRo33egh/fYMLVLXZR7mLcPt2DO5ULtFVQVV9D9fnNwvXulyKa5kcKSJRb/4cH9vM\nJPU9Xev92+V76uO9mQG8nfb+5oWIfEREnhKR1SLS1sURA3bzs7yqzgPewbWuwbXStuBa9z3R3bF+\nEHCMiNS1TbjTd4CJIlKJ+yFqb11734P/9LBOOy3fAWsJLjpP6cE6UjvPWzO8lq3OSToHzmiHlahe\ngwuMD+G+ZG+LSHpz3I+e1jWeYZls63kMGI5roh+K+xLGcV+ObOp9lAF3qlQJFONaNtk8jzutnwU8\n5/0QVQMH405D5/jcbrr290K9n3e6f0978t50R+nmmBKRPXCt0XeBT+ICQdvxlMu2b8O1vPGWv6O7\nHwt6fqyHvHofkDbthXsvB5y8Bix1V5xmA5dm6msQkcHef98FRntXFNtem4Drx1rUw2psonN/1gEZ\n6rpEVX+rqh8F/oRrVWTyLnBYyjk9wFG4juilPaxrTkRkGO4U7Meq+rSqvgtU0PFqb4v3b3gntzEY\n19F/A/AH4E4RGZRlsTnsCFhzUuZdTNf9V21adrauqXy+N28A+3Vz71NXdelwTInISDoeYzNxgemb\nqvqyqi7GHcu5uhsYKyKXAgcC/5elfE+P9ddxfW4rVfWDtKnWO7tZBxzWti6vj/mQ9G30ld64D+tr\nuKg/T0Q+KSJ7i8g+IvJVdjTxn/b+f7eIzPTuGbob9wY+28PtPwvMEJEviMgkEfkucGTbiyJSIiJ/\n8G7cGy8ih+ICUFeB8ibcwXeTiEwRkY8CP8X1+zT0sK652opruVzs7duxuFtFUltqG3F9Hyd798NV\n5riNW3BfhKtxfVG1uMDVnTm4U8hD6BiwzgOWqurqbpZdAUz3jpMq7zRyZ/h5b+7BvT8Pi8jR3oWF\nM0TkuJS67CEiB3p1KfLmPwt8zTtWZ+D6jlIv0izBfZe+ISJ7ishncR3wOVF3Uep+XOf/C6q6JMsi\nPT3W/4BrSf9NRA713o8TRORWEanwyvwG+K6InCMie+MuauRygSuv8h6wvL6oA3Gdzj/DBaZncVf6\nvumVUeBM3BfjOW9aD5yV0vTf2e3PBn6I6yyfj+vAvSmlSALXsXg77orQP3Dn6P/dxfrW4G49mIG7\nBP5n3BWdK3pSz53h9R98GnelbwHugLsK17nbViYOfB33K7oWdzXWFxE5HzgDOFdVW73+rs8B54jI\nZ7qpV1s/1mJV3eTNnoNr3czJstk/4lqx83DHw5HdF++yDn7em3rcKepq4FGv3A/ZcWr/d9xtOc94\ndfmsN/9bwDJvXx7AnbptTFnv28BluGNoEe69//bO7AeuBRTz/u1WT491VV2Le7+TwL+Ahbj3rZkd\n79uNuJbebbi+qxCucdEvpIfxwRiTRyLyaeB/gdH90IIf8PJ+p7sxJnciUoq7ongF8EcLVpkF/uFn\nY3YR38Wdtm0BftTPdRmw7JTQGBMY1sIyxgSGBSxjTGBYwDLGBIYFLGNMYOwSAWsgjVppekdffsYi\nMkTcsNsTU+ZdIyItIpI+BNIuSUTuF5Fv9Xc90vVZwJLeHTv8MtxjIGYn9PJnE0RXAE+oauqzor/E\njUt2Jm58rrwTkWNE5BFxQ4pnzFEgImER+ZGILBeRJu/f60QkklbukpQy80Xk6Azr6q7MtcD3d+LR\nrl61S7SwVHW79xxW4MiOgfcCb1fYF+8GzotIezRGVWtwj7g040bx7A3luMeFLsM9D5rJ93DP634d\n97D3ZbhA2j4uv3e3/G9wI+DOwA2K+E8RGee3jKq+g3scaWA1BPpi0C3cB50+/Ox4Mg8edzspg5Lh\nY/jhXJfBDez2F9yQseuA7+CG07g9y37MwT1Q+xt2DBn7i7S6ZBvCeA5uiNkbcM+rveZnubRlb/TK\nbMIdsEW4Z8C2AR8C56csI7ibEpfivgTv4A1XnOWz6Xa57vYlrcyXcONShdPm3wM8ksO+Z/qMsx07\nWfchQ33P8eogGV4r8Y6nn/XBd6YOb4jmtPmP4YadSZ13R9p+/wd3t3xqmSXAT3IsczUwt7f3NZep\nr1pYl+Eeuvw/dgw/u6rbJTrameGHu1vmRtxDsGfjhoU9COjUZO5mvSHccLFfxn0hU5/MzzaEMbhf\nLfG2+fkclmvbfi1uvKefess8hBsFcibu4L1N3PC34MZY/yLuV3kqbmjn//VGnYCuP5tsy3W3L6nu\nx40IcGLbDG/ooTOBu3Lc91z53YdURwPz1fvGprkaGEY3LSwRuUJSBsTrYvJ7rGUyFzhOvIH4RGQq\n8BHcQ9ttrdyDgCfTlnsS913wVcbzKnCIiJT0oL751VeRkcy/iJnm3U7nX9Juhx/OZRlcs7sF+EzK\na2W4X/fbfezDYlJ+fYErgdXdLJM+TO8c3KiX2d6vDstl2i9coNiE11Lx5kW9/TvHW0cjcHTaun+N\n66PJ+DnkuJyffXkQuDPl7/NwQak4h33P9Bl3eez43YcM236ItBaMN386bkiZR4APu1l+KG6one6m\nkq6WT1lPVy0swY3OkMQNcKjAdSmvj/bmHZO23NXA+37LeH/v55WbmK2+fTUF5eHnnRl+uKtlJuK+\n1K+2vaCq9eI/H90r6n2anpeBH4nIIFWt8a4s/QjXAhqOa42lD2E8P32lPpfrsF+qqiKyEXeq0zav\nVUS2evs6FTdq6L9EJLXOUdzYT13JZblO+5LBXcAdIlKq7qHec4G/q2pbMgS/+56Lnd33EtwpbDvv\nquCtuKFwnsKNp1WpHYfvBtoHsezN1GmfxrVkP4cbDuYA4DcislxVsw5Jk6O2frQB08Lq74CVdYhX\nz84MP7wzy+TDY7jxlr6MS/cUx42RlHp6k2moYj/LQeb96mpf2/b3dFzfVnfrSZXLcn6GXX4ctz9n\nisgzuNPwk1Ne97vvqbIdOzu779W4MaRSfQXY3avzMG/edFyCjg5E5Aqyj5V2qqr+O0uZrvwCuEFV\n7/X+fkfcEM3/g7tQUI1rnY5MW24kO/Jt+ikDrrUIrhU/IPRlwMo0/GymIV73p/tfwJ5aijtgD8Zd\nBWm7MjQdf0MeHyoiktLKOgyXUKNGdgzTe4mqPuet+0CyvM87u5wPi3BXtfZQ1e5Gck3/bPwu54uq\nNovI/biWVRXuSzEHerTv2Y6dnd2HN9gxrnpbKqyfAJ9X1VqgVlyCi33JELBwF2Xuy7KNNTnUJ10p\nLtikSuAFaFVtEZH5uD7D+1PKnIgboNBXGc90YI32LM1YXvVlwFqB68Abjzs/34IbifTXInIGbmiN\nL+N+yVb0ViVUtU5E/gz8TESqcVcJr8R94H6GrhiNq/NNuIP2O7jOXeg4TO8qXAKHX9A52US6nV2u\nW6paKyI3ADd4pzUv4PrwDgOSqnqrV3QFnT8bP8vl4i7cSJ57An/VHVmIdnbfuz12ctj3dLNxx8Yw\nVd2My4v4lKo+klJmAe7L3ElPTgm9ixFtmZhCwDgROQDYoqptrcRHgctFZDnulHAGbgTRv6Ss6pe4\nsfhfxQXVr+CO21tyLHM07v0YOPqqswyYjOvvaWDHpfMo7nJ8tTf9kJ27fJ3TMrgD907c6cwGXHP6\nGeDmLPswB/eB/h53C8FW3BXHcEqZj+AO6Cbv35NJ6UDNVDc/y3WzXwuAa9LmrQcu1R2dtP/FjhbH\nJlw/zIkp5TN9Nn6Wy7gvXbx3ggsmCuy3E/ue/hn7OXay7kMXdX0Zd2Xxo97nPCrt9ZuB53vhOzKL\nzreYKCkXg3CJNX4NrMT1MS3D3b5TnLauS7z3uxnXz3hMhu11WQbX/7edlAzeA2Gy8bAAcckGVgK/\nUNUbuyk3B5fb79K+qpvpeyJyCu5eu6nafZqtXZaIfA04U1VP6u+6pOrvTvd+IS7zyRTclcIK3N3D\nFeSWmtzsolT1XyLyB1yKspX9XZ9+0oprnQ4oBRmwPP8N7I3rK3kT1xzuLh2VKSCq+tv+rkN/0p3r\np+x1dkpojAmMXeLhZ2NMYbCAZYwJDAtYxpjAsIBljAkMC1jGmMCwgGWMCQwLWMaYwLCAZYwJDAtY\nxpjAsIBljAkMC1jGmMCwgGWMCQwLWMaYwLCAtYvw0s1f1N/1MKY3WcAaYETkKBF5SUS2i8gWEXlR\nRA7u73oZMxAU8gB+A46IDMKlvPoqLvNKDJcIoLmP6xFR1R4lwDCmN1gLa2CZDKCqf1XVhKo2quqT\nqvq2iFwjIm2p3RGR8SKiIpL6ozNRRF4VkRoReVhEhqaU/7yIrBSRzSJylYisEJETvNeuEZEHROQu\nL4XVhSJSJCK/FpG13vRrb+x7RORCEZmbWnGvLpO8/98uIreIyFMiUisiz3u58xDnVyKy0avnOyKS\nMQONMeksYA0si4GEiNwhIqeKSHpCz2w+D3wBl68vjktRhYhMBW7C5QUcBVTi0mmlOhN4ABgM3A18\nH5cS6wBcvr9DcOnQ/DoXl825CjcE9d3e/JOAY3DBuRL4FLA5h/WaAmYBawBR1RrgKFxqpz8Cm0Tk\nERFJz9DblTtVdYGq1gNXAZ8SkTBwDvCoqs5V1RbgajrnYHxZVR9S1aSqNuICzrWqulFVN+HSaJ2f\nw+48rqovqGozLvgdLiK745IbVOCSp4qqvquq63JYrylgFrAGGO8LfKGqjsUl6xyNy0Pnx6qU/6/E\n5e6r8tbR/pqqNtC5VbMq7e/RdMwYs9Kb51fq9tqSs45Wl4X597icghtF5Fav786YrCxgDWCq+h4u\nOeh0XNLX0pSXd8uwyO4p/x+Ha820Zbce2/aCiJQAw9I3l/b3WmCPtPWt9f7foS4i0m1dvIzGQ9uW\nV9XfqupBwFTcqeF3MixvTCcWsAYQEdlHRL4lImO9v3cHPgu8gpeKTETGiUglLlt1uvNEZKqIlALX\nAg94iUAfAE4XkSNEJAZcg8uK3J2/AleKyHARqcKdRrZ1+r8FTBORA0Sk2FtfutO8WzRiuL6sV1R1\nlYgcLCKHikgUF/iagGSG5Y3pxALWwFILHAr8R0TqcYFqAfAtVX0Kl+j1bVxa8ccyLH8nrkW2Hpdq\n/OsAqroQlxTzXlxrqw7YSPe3S1wHzPO29w7wujcPVV2MC4hPA0uAuRmWvwf4Ae5U8CDgPG/+IFz/\n3FbcaeZm4Bfd1MOYdpaXsAB5p2jbgL1UdXkvrP92YLWq5nJV0ZisrIVVIETkdBEpFZEy4AZcq2lF\n/9bKmNxYwCocZ+I6vdcCewGfUWtem4CxU0JjTGBYC8sYExi7fMASkUtFZJ6INHudwSaF98zgn7zn\nDGtF5E0RObW/6zWQeM9YrveefVxsw/j0n10+YOH6bK4D/tzfFRmgIri70o/FPdt3JXCfiIzvxzoN\nND8FJqjqIOAM4DoROaif61SQdvmApaoPqupD2AO2Galqvapeo6orvOcIHwOW4+6dMoD3fGZD25/e\nNLEfq1SwdvmAZXLjPWg9GVjY33UZSETkJhFpAN7D3Xz7RD9XqSBZwDLtvMdl7gbu8J5jNB5VvQQ3\nysTRwIP08aCKxrGAZQAQkRDu0Z4W4NJ+rs6A5A2qOBf3IPlX+7s+hciGSDaIiAB/AkYCp6lqaz9X\naaCLYH1Y/WKXb2GJSMQbUSAMhEWkOG1YYQM3A1OA073B+4xHREaIyGdEpFxEwiJyMm4EjWf6u26F\naJe/011ErsGNGpDqh6p6Td/XZuDxxlpfgeuTSU088WVVvTvjQgVERIbjhufZH/cDvxL4rar+sV8r\nVqB2+YBljNl17PKnhMaYXYcFLGNMYFjAMsYEhgUsY0xgWMAyxgRGwQQsEflSf9dhoLP3qHv2/vS/\ngglYgB1s2dl71D17f/pZIQUsY0zA9dqNo1VDwzp+92ivrHtnbNqcYPiwcH9Xo93it0uzF+pjrTQT\npai/q9FOQgPr97RFm4hJcX9Xo11jso4WbcqWELdbJx9Xppu3JHyVnf9282xVPaUn2+upXnumbvzu\nUV6dvXv2ggXq5NEH9HcVBrxQycAL6gPJK42P93gdm7ckeHX2OF9lw6OWVPV4gz1kDwEbU8AUSJLs\n72r4ZgHLmAKmKK3q75RwILCAZUyBsxaWMSYQFCURoBFbLGAZU+CSWMAyxgSAAgkLWMaYoLAWljEm\nEBRotT4sY0wQKGqnhMaYgFBIBCdeWcAyppC5O92DY2A9XWqM6WNCwufka20iQ0XkHyJSLyIrReRz\nXZQrEpFfichaEdkqIjeJSNbREixgGVPAXKe7+Jp8+gPQgssifi5ws4hMy1DucmAmMB2YDBwIXJlt\n5RawjClg7j6s/LSwRKQM+ARwlarWqepc4GHg/AzFTwd+p6pbVHUT8FvgC9m2YX1YxhS4pP/WU5WI\nzEv5+1ZVvTXl78lAXFUXp8x7C5jlY90CjBWRSlXd3lUhC1jGFLC2FpZP1ao6s5vXy4GatHk1QEWG\nsv8CLhOR54Aw8HVvfilgAcsY05kiJPLXM1QHDEqbVwnUZih7PTAYeBNoBv4IzAA2dLcB68MypsAl\nVXxNPiwGIiKyV8q8/YGF6QVVtVFVL1XVMao6AdgMzFfVbu+ysBaWMQVMEVo0P7kOVLVeRB4ErhWR\ni3AtpjOAI9LLisgY3BnpOuBQ4Crgi9m2YS0sYwqYu3E05Gvy6RKgBNgI3AN8VVUXisg4EakTkbYB\n5CcCLwH1wB3A5ar6ZLaVWwvLmAKXQ6d7Vqq6BTgrw/wPcZ3ybX+/AIzPdf0WsIwpYKpCQoNzomUB\ny5gCl8xjC6u3WcAypoC5TvfghIHg1NQYk3dtne5BYQHLmAKX8P9oTr+zgGVMAcvzne69zgKWMQUu\naVcJjTFB4B5+toBljAkARWjN06M5fSHQAWvL1gQX/fdGnnq+gaqhYa6/Yhif+3jnkSyam5X/ub6a\n+x6po7FJ+cxZ5fz6R8OJRl1n46CJSzuUb2xSvnphJb+9fnif7EdfadUWFjGPzWwgRhGTmM5u7U9K\ndLRSF7OS90mQYARjmcIMQhKcA9uPVm1mYfNLVCfWEZMi9orNYFRkQsayK1sXsbx1AQlNMDIyjqmx\nw9rfj3ea/s3m5DoSmqBIihkfnc7Y6F4Z1zPQqLLr3TgqIpcCFwL7An9V1Qt7sU6+XXrFJmIxYd07\ne/LmgmZOP38d+0+LMW3vog7lfvb7rcx/q5m3nxtHIgFnXrCW63+9hWu+MwyAmqUT28vW1ScZvd9y\nzvlYObua93iDECGO4XTq2MYbzKVcKymXyg7lNut6VvI+B3IMRZTwFi+xlEXsxb79VPPe8W7zfxBC\nzCr9JLXJLbzR9CwVoaGUhwZ3KFcdX8PylgXMLDmJIinhzaY5fND6JpNjBwEwPjadqXI4YYlQn9zO\na02zGRQayqDwsP7YrRxJoG4c9Rta1wLXAX/uxbrkpL4hyYOP13Htd4dSXhbiqENLOOPkMu56oPPQ\nO489Wc/XvljJ0CFhhleFufSLg/m/ezMN0QN/f7yOEVVhjj6suLd3oU8lNM5GVjOBaUQkwmCpYjij\nWceHncquZSWjGU+5VBKVGBOYyjpW9H2le1FcW9mQ+JBJsRlEJMqQ8EiGR3ZnbXxpp7Jr40sZE51E\neWgwUSliQmy/DuUqQkMIS+pvv9CgmY+vgUZxLSw/00Dgqxaq+qCqPoQbs2ZAWLy0lUhYmDwx1j5v\n/2kxFr7fknVZVVi9Ns72mkSn1+68r5bzP1mBSHB+dfyopxYhRJnsOGWuYDD1nQaIhHpqKGdHK6Oc\nSlpopkWb+6SufaEhWYMglIV2jDdXERpCXbLzYJd1up2K0JAO5Vq0iRZtap+3qPkVnq6/mxcbH6ZI\nSqgKj+ndHcijBCFfkx85ZM0REblORNaIyHYRmdNFsooOBkbY3Al19UkGVXSsfkV5iLq6zuN/nXxc\nKb+7bTubqhOs3xjn93/aBkBDY8cMkitXtfL8y418/lPpgyYGX4I4kbQegAgR4rR2UTaaUi7aPn9X\nkSBOJC2rVIQoCc3wfmgrEWIp5WLe/B3vx9Siwzi+9LMcXHwyI8PjCBGM/j7F3+B9OYz77jdrzidx\nSSeOBoYCLwN3Zlt5XgOWiHxJROaJyLxNmzu3XvKpvCxETW3H4FRTm6S8vPMuXXHZEA6YXsSBJ37I\nUaev5sxTyohGYeTwjgfVXQ/UctQhxew5Lmt6tMAJEyGeFnDitHYITKllEymBrC2ohYN9jaaDMBHi\nacEpTivhDKnxwhLtENjjtHjzO74fIiGGhEfSpA2sir/fC7XOP5fmK+JryibHrDl7AnNVdZmqJoC7\ngKnZtpHXgKWqt6rqTFWdOXxY7/7CTJ4YJZ5QlizbcQr41sIWpu0d61S2pCTE7348nFVv7MkH/xnP\n0CFhDtqviFCo46/GnQ/Ucv4u2LoCKKMCJdmhb6WW7ZR1GoIbyhhEbUoegDq2EaOImBR1KhtUpaFB\nKEp9cscpcW1iK+Whyk5ly6WS2uTWHeWSW4lJMTHJ3M+pJGlMBqMPK8dEqlVtDRJv+lLayrrKmpOp\nhXUvMFFEJnsJVC/AJaboVmBPCctKQ5x9Wjk/+MUW6huSzP1PI48+Wc9553S+rWHNujhr18dRVV6Z\n38T1v97KD77d8QrOS681smZdnE+evutdHQTXGhjBGJayiITG2abVbGIto+h8W8Mo9mAty6nTGlq1\nhWW8y6jcx1ob0CISZWR4HEtb3iSurWxNbGBTYhWjIxM7lR0dmcia+BLqktto1WaWtbzdXq5ZG1kX\nX05cW1FNUh1fw7r4CoaGR/X1Lu0Uxd3p7mfCy5qTMt2atrpcsuasA+YC7wONuFPEb2arr9/bGiJe\n2TAQFpFiXCTt106NP/xkOF/85kZ2m76cYUPC/OGnw5m2dxEfrm5l+rEfsuD5cYwbG2XpilYu/PoG\nNlYn2H1MhB9fMYyTZpV2WNdf7qvl7NPKqchwSrmr2IcDWcQ8nudRosSYwoGUSyVN2sDLzOZwTqZY\nSqmS3dhD9+Z1nvfuwxrDxOyt9cCZUnQoC5pfYk7D/cQkxpSiQykPDaYxWcdLjY9wRMkZlITKqYqM\nYbxOZ17jkyRw92FNih4AgCCsbl3Mu82voECJlLFPbCYjIrv3787lII8jjuaSNedq4BBgd2A9cB7w\nrIhMU9WGrjYgqtrVazsKiVwD/CBt9g9V9Zqulpm5f7G+Ojs4H1pfO3n0Af1dhQEvVFqavVABe6Xx\ncbYnqnsUbcZMG6yX3HeUr7JXTn98fnd5Cb0+rK3ANFVd4s27E1ijqpenlX0MeEpVf5Mybxtwgqqm\nJmvtwO+Wmh92AAAa2UlEQVRtDdeoqqRN1/hZ1hgzcLlO97CvKeu6VOuBtqw5ZSJyFC5rTqarf68B\nnxSRkSISEpHzgSjwQXfb2HUu+xhjdkLex3S/BHeD+UbcfZvtWXOARcBULyHFz4ARuESqZbhA9QlV\n3dbdyi1gGVPAXKd7v2TNaQK+5k2+WcAypsDZ8DLGmEBou9M9KCxgGVPgLAmFMSYQVKE1aQHLGBMA\n7pTQApYxJiDyeKd7r7OAZUwBy/dtDb3NApYxBc1OCY0xARKkMd0tYBlTwNxVwmCMjgoWsIwpaHbj\nqDEmUIJ0Shic3jZjTN61XSXMVxKKHLLm3CIidSlTs4hkHVfaWljGFLg8XyVMzZpzAPC4iLylqgtT\nC6nqV4CvtP0tIrcDnVNepbGAZUwBUxXieQpYKVlzpqtqHTBXRNqy5lzuY7mPZduGBSxjClweO927\nypozK8tynwA2AS9k24AFLGMKWI53uleJSOp467emZc7JJWtOqguAv6iPBBMWsIwpcDkErOruklCQ\nW9YcALyhk2cBF/upgF0lNKaA5TlV/WIgIiJ7pczbH1jYRXlw/VsvquoyPxuwgGVMgUsivqZscsya\n0+bzwO1+62oBy5gCpgrxZMjX5NMlQAkua849pGTN8e63ak81LiKHA2OB+/2u3PqwjClw/ZE1x5v3\nMi7Fl28WsIwpYPYsoTEmUNQCljEmKIL08LMFLGMKmKoNkWyMCQwhYWm+jDFBYX1YwJJFFZy270d6\na/WB992lz/d3FQa8Xx57Sn9XYWBb3/OhjS1rjjEmONT1YwWFBSxjCpxdJTTGBIJap7sxJkjslNAY\nExh2ldAYEwiqwQpYwTl5Ncb0iv5I8+WVnSAij4lIrYhUi8jPs63fWljGFLg892H5SvMlIjHgKa/8\np4EELolFtyxgGVPAFCGZp6uEOab5uhBYq6q/TJn3drZt2CmhMQVOfU54WXNSpi+lraqrNF/TMmz2\nMGCFiPzTOx2cIyL7ZqurtbCMKWS5dbpny5qTS5qvscBxuDHfnwEuAx4WkX1UtaWrDVgLy5hCl0MT\nK4tc0nw1AnNV9Z9egLoBGAZM6W4DFrCMKXCq4mvyIZc0X2/jNwymsIBlTAFTIJkUX1PWdeWW5usu\n4DAROUFEwsA3gGrg3e62YQHLmEKmgIq/yR9fab5U9X3gPOAWYCtwJnBGd/1XYJ3uxhS8fN6HlWOa\nrwdxLTLfLGAZU+js4WdjTDD47lAfECxgGVPorIVljAkEBfVxBXCgsIBlTMGzgGWMCQo7JTTGBIYF\nLGNMILTdOBoQFrCMKXCWhMIYExx2ldAYExRiLSxjTCD4H+tqQLDRGowpaD5Hashz1hwRuVBEEt4I\nDm3TrGzrtxaWMYWuH7LmeF5W1aNyWXmgA1ZLsomFdS+wuWU10VAxe5UewujiSRnLrmh8m+UNb5Eg\nzm6xPZlafjQhCQPw6rZH2R7fiHh3/BaFyzh6yKf7bD96S822BL+8fD3z/11P5ZAwX/jOcD5yZvoI\ntqCq3P7Lap58oIbG+iSTphVx6Q9HMn5yEQDrV7fyu6s28O4bjURjwtGnVvDVq0YQjgSnszaTlkQT\nC7Y+zeamlURDJUyuPILRZftkLLui9nWW1cwnoXF2K53EtCHHERL39WmI17Bo67Nsa15PSMLsVjqJ\nfQYfS0gCcgKTzM9qcsyas1MCHbDerXuRECFmDTuf2vhmXq/5J4MiQymPDO1QrrplFcsb3uLgyo9S\nFCrjjZon+aBhHpPLDm0vM6X8SMYWZz5Yg+r3V28gGhXue3USSxc1ceUX1zBhSlF7IGrzwhO1zL5/\nO7+6bxwjxkS5/cZqfv6tddz06HgAfnfVBiqHhbn3PxOpq0ly+fmreOSubZx94ZB+2Kv8WbTtOUKE\nOG70xdS2bmL+pkeoiA2nIjqsQ7lNjStZVjOPg0d8guJwGa9XP8aS7a+w92DXOFi09VlioVKOG3MR\n8WQzr236Bx/Wvc34igP6Y7dyk9/7sLrKmjOri/IzRKQa2IIblfQnqhrvbgNZfwJEpEhE/uSdj9aK\nyJsicqq/+veeuLayoWU5k8pmEpEoQ6K7MSI2nrXNSzqVXdO8mDHFe1MeGUo0VMTE0gNZ07Q4w1p3\nHY0NSebOruWCb1ZRUhZi+sGlHHFiOc/8Iz2pCaxf1cr0maWMGhcjHBaOP2sQK5fsGPhx/epWZn20\nglhRiKHDI8w8toyVS5r7cnfyLp5sZUPDB+xVeTiRUIwhRWMYUTKBtfWdR+hd27CIsWXTqIgOIxoq\nZtKgQ1mTUq4xXsOo0r0IS4SicBlVxXtQ17q5L3enR0T9TWRP85VL1pwXgOnACFyr7LPAd7LV1U+b\nNQKsAo7FZcC4ErhPRMb7WLbXNCS2Iwhl4cHt8yrCw6iLb+1Uti6+lYrIjl/NisgwWrSRlmRT+7zF\n9a/y7OY7+M+2h9nSsrZ3K98H1ixvIRwWxk6Itc+bMKWIFRkCzazTB7H2wxZWL2sh3qo89eB2Zh5b\n1v762f9vCHMer6WpMUn1+lZee76emceUdVpPkDTEtyISoiy6o5VYER1OXeuWTmXrWrdQERueUq6K\nlmQDLYlGAPaomMG6hiUkkq00xeuoblzJ8OI9en8n8sV/1pxqVZ2ZMt2atibfWXNUdZmqLlfVpKq+\nA1wLnJOtqllPCb2B5a9JmfWYiCwHDgJWZFu+tyS0lYjEOswLh6LEtTVr2bb/J7QVKGZy2aGUhwcT\nkjDrmpfyeu1sjhj8CUrDnft7gqKxPklpecffo9LyEI31nTsshg6PMH1mCV84YTmhMAwfFeXnd49t\nf32/Q0p44t5tnLXfEpIJOPETgzjypPJO6wmSeIbjJxKKEc8wpHg8mXb8hGLt64hRwtCiMayuW8DT\na25GUUaXTmFEycTe3YGBqT1rjqq2nep0lTUnneJj2IicewVFZCTuXLVTJUTkS23NxdTWS28IS7TT\nwRVPthCRaMayiZSybcuFvbKDoyOIhGKEJMyY4skMjoxkU8uHvVj73ldSFqKhrmNwqq9NUlLW+SO/\n63fVvP9WE3e/OIHH353M+V8fxnfPXU1TY5JkUrniwtUcdXIFjyzYiwfmT6Jue5Lbfrapr3alV0Qy\nHj/NnYIYQCTUsWw82dK+DlVl3qaHGFk6kRPHXsJHRn+JeLKZxdtf7N0dyKMcTgm7lUvWHBE51Ysl\niMg+wFXAw9m2kVPAEpEocDdwh6q+l6HCt7Y1F2Oh4lxWnbPScCWKUp/Y3j6vNrGZ8kjnjuDyyBBq\n4zv6FGrjm4lJCV3VUQI0PlBXxuwZI5FQ1izf8UVb9m4z4/cq6lR22aJmZn1sEMNHRQlHhJPOqaRu\ne4IPl7RQuy3BxrVxzjx/MLGiEIOGhDnpnEG8Oqe+L3cn70ojQ1BNUt+6owuhtrWa8ujQTmXLo0Op\nbalOKbeJWKiUWLiE1mQTTYlaxpXvT0gixMIljCmbyqbGFX2xGz2nuEdz/Ez++MqaAxwPvC0i9cAT\nuED342wr9x2wRCSEi5QtwKV+l+stEYkyMjaeD+rnEddWtrauZ1PLSkYX7dWp7Oiiyaxuep+6+FZa\nk80sbXidMcWTAWhNNlPdsoqExklqkrVNS9jauo6q2NhO6wmSktIQR55cwR2/qqaxIcmC1xp4+ek6\njj+782nu5P2KeeGJWrZuipNMKk//YzvxuDJ6fJTKoRF22z3KY/dsIxFX6moSPPVgDRP27hz4giQS\nijKyZBJLal4hnmxla/MaNjYuY3RZ58TDo0unsLp+IXWtm2lNNvFBzauM8crFwiWUhAexqu4dkpqk\nNdnMmoZ3qYgN67SeASt/mZ9R1S2qepaqlqnqOFW9x5v/oaqWe9lzUNVvq+pIr9wEVb1aNUN/Thpf\ntzWIiAB/wt0MdpqfFfeFKeVHsbDueeZsvpNoqIgpZUdTHhlKY6KOF7fex5FDPkVJuJzhsd3Zs3R/\nXtv+GAnijIztyaTSmQAoSZY0zKM+sa29E/+AQSd16MwPqv+6diQ3fm89nzr4AwYNDvP1H7l7qzau\naeWik5dz2+w9GTEmyqe/MpRtmxN89WMraGpURu8R5eqbxlA+yN2n9oObR3Pzjzbyt1u2EAoLBxxe\nyleuHNHPe9dzU4ccx4KtT/Hc2luJhoqZOuQ4KqLDaIzXMHf9XRy123mURAYxvGQ8ew46iFc3/p2E\nJtitZCJ7VR7Wvp4ZVR/j3W3Ps6xmHiLCsKLd2WfwMf24Z7kJ0rOEoj7GlhCRW3B3rZ7g3RCWVWV0\nuB4++OM9rN6u61uvPt/fVRjwfnnsKf1dhQHtpfV/ZXvLhh71XxTtvruO/cY3fZVd9u1vzVfVmT3Z\nXk/5uQ9rD+DLuIC1PuW5n3N7vXbGmN6Xx1PC3ubntoaVBGmUemOMb36vAA4UgX40xxiTBzaAnzEm\nKKyFZYwJDgtYxphAsD4sY0ygWMAyxgSF5GkAv74QkCERjTHGWljGGDslNMYEQsA63e2U0JhCl8dH\nc/ym+Upb5hkRURHJ2oCyFpYxha7/0nzhPZPcedTNLlgLy5gCJrirhH6mrOvakebrKlWtU9W5uFFE\nz++ifCXwA+C7futrLSxjCllufVhVIjIv5e9b0xJR5Jrm68fAzcB6vxWwgGVMofMfsKqzjIflO82X\niMwEjgQuA3wP72sBy5hCl78+LF9pvrzh1m8CLlPVuBvQ2B/rwzKmwOUraw4pab5S5mVK8zUImAn8\nTUTWA69581eLyNHdbcBaWMYUujy1sFS1XkTa0nxdBMzApfk6Iq3odmB0yt+7A6/icp12mz/OApYx\nhUzz/izhJcCfcWm+NpOS5gtYBEz1Mue0d7SLSFu+vQ2qGu9u5RawjCl0ebwPS1W3AGdlmP8hrlM+\n0zIr8DkMuwUsYwpckB7NsYBlTKGzgGWMCYQBlMLLDwtYxhQwwU4JjTEBYgHLGBMcFrCMMYFhAcsY\nEwgBG3HUApYxhc4CljEmKIKU5qsXA5ZAKNx7qw+4n0/ct7+rMODdu+r+/q7CgHb8aVvzsh47JTTG\nBEPAbhy18bCMKXT9kDVHRD4jIu+LSI2IbBSRO0QkffC/TixgGVPA2u50z9MAftAxa865wM0iMi1D\nuZeAY1V1EDABd7Z3XbaV2ymhMQVOkvk5J0zJmjNdVeuAuSLSljXn8tSy3nAzqRLApGzbsIBlTCHL\nbx9WTllzROQo4HHckMkNwNnZNmABy5gCl8c0X76z5gB4eQsrRWQMcDGwIlsFLGAZU+jyl+bLV9ac\nTptXXSMi/wLuBQ7srqx1uhtT4Poha04mEWBitkIWsIwpdHm6rUFV64G2rDllXh/VGcCd6WVF5Fwv\nMQUisgdwPfBMtm1YwDKmkHlZc/xMPl0ClOCy5txDStYcEalrC1LAVOAlEakHXgTex/Vjdcv6sIwp\nYPkecdRv1hxV/T7w/VzXbwHLmEKnwXk2xwKWMQXOHn42xgRDwB5+toBlTIGz8bCMMYFhAcsYEwyK\ndbobY4LDOt2NMcFhAcsYEwSWqt4YExyqeRvAry9YwDKm0AUnXlnAMqbQBemU0EZrMKaQKZBUf5MP\nOWTNuUBE5ntZc1aLyM9FJGsDygKWMYUuj2m+8J81pxT4BlAFHAocD3w728rtlNCYApevU8Ics+bc\nnPLnGhG5Gzgu2zYsYBlT4PJ4lTCnrDlpjsHHUMoWsIwpZLmd7uU1a04bEfkCMBO4KFsFLGAZU8Dc\njaO+I1bes+aIyFnAT4ATVLU6WwWs092YQpf0OWWXU9YcETkF+CNwuqq+42cDFrCMKXCi6mvKJses\nOR8B7gY+oaqv+q1roE8JW5JNLKyZw+aWVURDxexVfiijiydnLLui4S2WN7xBQuPsVjSRqRXHEJJw\n++vrmpawtH4eTYk6YqFS9h10HENio/tqV/pEq7awiHlsZgMxipjEdHZrT2LS0UpdzEreJ0GCEYxl\nCjM6vF+7gq1bk1z27W3MeaGZoUNDXHl5BeecXdqpXHOzcu1Panjo0UaampSPn1nCj39YSTQqHcot\nXRbnmBM3cvppJdzyuyF9tRs9k/8RRy8B/ozLmrOZlKw5wCJgqpeQ4irc6eITIu3v479V9dTuVu4r\nYInIXcAJuHsn1gM/V9XbdmJn8urd2n8TkhCzqi6kNl7N69ufYFCkivLI0A7lqps/ZHn9Gxw85AyK\nQmW8sf1ffFD/GpPLD3Ovt6xicd0r7F95IpWRkTQn6/tjd3rde7xBiBDHcDp1bOMN5lKulZRLZYdy\nm3U9K3mfAzmGIkp4i5dYyiL2Yt9+qnnv+O6V24nGYNGbI1mwsJXPXrCF6VOj7LN3tEO53/yhjrfe\nbmXu0yNIJJVzL9zCjb+p5fJvd+yu+d6V25mxf6wvdyEP8vssYQ5Zc7LewpCJ31PCnwITVHUQrol3\nnYgctDMbzJe4trKheRmTyg4hEooyJDaKEUXjWdu0uFPZNU3vM6ZkH8ojQ4mGiphYdhBrmt5rf31p\n3WtMLJvJ4OhuiAjF4XKKw+Wd1hNkCY2zkdVMYBoRiTBYqhjOaNbxYaeya1nJaMZTLpVEJcYEprKO\nFX1f6V5U35DksSca+Z/vDKK8LMRhhxRx6knF3Pf3xk5lZz/dxEX/r4whQ0JUDQtz8RfKuOdvDR3K\nPPhwI5WDhKOPDFrAwg3g52caAHwFLFVdoKptn1BbIzJrWune1BDfhhCiLDK4fV5FZBh18S2dytYl\ntlIRqUopV0VLspGWZBOqSbbHN9GSbOSFzXczp/ovLKr9NwmN98l+9JV6at37JTuuMFcwmPpOV6Gh\nnhrK2fG+llNJC820aHOf1LUvLF2WIBIWJk3YcZIxbWqU9xa3Zl1WFdauS1JT43qia2uT/PSGGn50\ndWWWJQeg/CdS7VW+O91F5CYRaQDeA9YBT/RarXxIaCuRUMeme1hixLXzAZdIthKRHb98EYl662ih\nOdmIkmRD8zIOHXwWRwz9JLXxapbVz+/dHehjCeJE0noAIkSIk+H9Ik6EaEq5aPv8XUV9fZKKio59\nUBXlQl1d55bE8bOKuPVPdVRvTrBhY4I//tl1GTQ0urI/+UUt532mlNGjA9rHt6u1sABU9RLcDWBH\n464EdPq5FZEvicg8EZnXkuzctM6nsESJJzt+2eLa0h6MOpQNRUloS4dybh0xwt7zluNKplMULiMW\nKmGPkv3Y1NL5VCnIwkSIpwWcOK0dAlNq2URKIGsLauFgX6PpoKwsRG1txy9hTa1SXi6dyn7zvyrY\nd1qUWSdt4rSzqjntlGKiURgxPMQ7C1t5fm4zX7k4wF0I+X2WsFfldFuDqiZUdS4wFvhqhtdvVdWZ\nqjozFirJVx0zKo0MRklSH9/WPq+2dXOnDneA8vAQauObd5SLbyYWKiEWKiYaKqI4VIa7hc4ROh+0\nQVdGBUqSBt1xD18t2ynrdJ8flDGIWra3/13HNmIUEZOiPqlrX5g4IUw8oSxdtiOIL1zUyj6TOwfw\nkhLhZ9cPZsH83Zj/0kiGDA6x/75RQiHhxZebWbUqwQGHbmDqjPXc9L/1PPZEI8edsqkvd6dHJJn0\nNQ0EO3sfVoR+7sOKSJSRRRP4oP414trK1pZ1bGpZkfG2htHFe7O66V3q4ltoTTaztH4+Y4r3SXl9\nHz5sfIfmZAOtyWZWNL7F8Ngefbk7vS4sEUYwhqUsIqFxtmk1m1jLKDrf1jCKPVjLcuq0hlZtYRnv\nMorxfV/pXlRWGuKjpxbz0xtrqW9I8sqrzfzrqSY+9YnOP7Tr1iVYtz6BqjJvfgs3/qaW733L9QV+\n/txSXntxBM/NHs5zs4dzwfmlnHh8Mfff3fmHc0BS8nnjaK/L2sYXkRHAR4DHgEbc7Q2f9aZ+NaXi\naBbWPMecTbcTDRUzpeJoyiNDaUzU8uKWezly6GcoCVcwvGgceyZm8NrWR0gQZ2TRBCaVHdy+noll\nB9GqTczd/FdCEma3oklMKDuwH/esd+zDgSxiHs/zKFFiTOFAyqWSJm3gZWZzOCdTLKVUyW7soXvz\nOs9792GNYSJT+7v6efeL6wfz9W9vY8r+GxgyJMQvflzJPntHWb0mzpHHbeLF54YzdkyE5SvjfO0b\n26iuTjJ6dIir/mcQxx1bDEBpSYjSlBhXVioUFQlVw4LRnyX4uyl0oBDNUlkRGQ48gLvFPgSsBH6r\nqn/sbrnK6Ag9fOg5+arnLiexKTinDP3l3lUv9XcVBrTjT9vEm2+19Kj/orJstB425Uu+yj45/4fz\nszxL2OuytrBUdRNwbB/UxRjTHwLUwtp1LvsYY3LX1ocVEBawjClwA+UKoB8WsIwpaAPnplA/LGAZ\nU8iUQAUsGw/LmEKXx/uwckjzNV1EZotItYj/NBgWsIwpcPkawM/jN81XK3Af8MVc6mqnhMYUujyd\nEuaY5ut94H0RmZTLNixgGVPIVCHh+yphtqw5PUnz5YsFLGMKXf6y5uxUmq9cWMAyptDl7yphzmm+\ncmWd7sYUMgWS6m/KLqc0XzvDApYxBU1Bk/6mbGvKLc2XiEgxEPP+LhbJPuCaBSxjCpniOt39TP5c\nApTg0nzdQ0qaLxGp89J9AeyBG66qrfXVCLyfbeXWh2VMocvjne45pPlaAbkP7WsBy5hCF6BHcyxg\nGVPQ7OFnY0xQKGDDyxhjAsNaWMaYYMjp0Zx+ZwHLmEKmoD7usRooLGAZU+j83cU+IFjAMqbQWR+W\nMSYQVO0qoTEmQKyFZYwJBkUTif6uhG8WsIwpZG3DywSEjdZgTKHL0/Ay4D9rjlf2myKyXkRqROTP\nNryMMaZbCmhSfU0++cqaIyIn4xJTHI8bamYC8MNsK7eAZUwh0/wN4JeSNecqVa1T1blAW9acdBcA\nf1LVhaq6FbgWuDDbNixgGVPgNJHwNfnQVdacTHkJp3mvpZYbKSLDuttAr3W618Q3Vc/eePPK3lq/\n2fVVje3vGgx4e/R0BbVsnf20PlDls3hxljRfuWTNKQe2p5XDK7u5qwr0WsBS1eG9tW5jTH6o6il5\nXF0uWXPSy1Z6/3abYcdOCY0x+ZJL1pyF3mup5TaoapetKwDRAN3laowZ2ETkXtzFx4uAGcDjwBGq\nujCt3CnA7cBHgHXAP4BXVLVDSvt01sIyxuSTr6w5qvov4OfAc8BKYDnwg2wrtxaWMSYwrIVljAkM\nC1jGmMCwgGWMCQwLWMaYwLCAZYwJDAtYxpjAsIBljAkMC1jGmMCwgGWMCYz/D0sv+I1ZdnzlAAAA\nAElFTkSuQmCC\n", "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "# between NBS and StratiPy (lambda = 1800)\n", "repro_confusion_matrix(result_folder_repro, data1=nbs_100, data2=stp_100_lamb1800,\n", " plot_title='Confusion matrix with actually used\\ntuning parameter value',\n", " lambd=1800)" ] } ], "metadata": { "kernelspec": { "display_name": "Python 3", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.6.1" } }, "nbformat": 4, "nbformat_minor": 2 }