{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Applying Rocket to Raw Audio\n",
    "\n",
    "Note: this notebook is extremely messy as a result of me trying to rapidly prototype rocket for raw audio and not following best practices. It also uses the beta version of fastai v2 for removing silence and preprocessing. If you are interested in experimenting yourself and can't make sense of something here, please reach out to [me](https://forums.fast.ai/u/madeupmasters/summary) by PM, or in the [Deep Learning with Audio](https://forums.fast.ai/t/deep-learning-with-audio-thread) or [Time Series](https://forums.fast.ai/t/time-series-sequential-data-study-group) threads\n",
    "\n",
    "This notebook will apply the findings of the recent [Rocket Paper](https://arxiv.org/abs/1910.13051) by Angus Dempster, François Petitjean, Geoffrey I. Webb to 1D raw audio signals for the task of voice recognition. Some of this code is also adapted from [Ignacio Oguiza](https://forums.fast.ai/u/oguiza) and his [Time Series Module for FastAI v1](https://github.com/timeseriesAI/timeseriesAI)\n",
    "\n",
    "Initially the signals were too long and slow to train at a sample rate of 16000 (was going to take ~30-40 minutes for 1s clips). Training a 3800 audio 10 class dataset (small problem, trains to 99%+ accuracy in 2 minutes using typical audio pipeline of spectrogram + CNN), . To speed things up I added a stride which sped up results without a drop in accuracy, but still only led to 85% accuracy after 4 minutes of training. Removing silence and doubling the amount of time to 2s allowed us to get great results (95% accuracy in 6s, 98.6% in 20 seconds, 99.2% in 1 min 20 sec).\n",
    "\n",
    "Unfortunately so far, I have not been able to scale the results to harder problems, such as a 250 speaker dataset. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Summary of Findings\n",
    "\n",
    "In order to spare you having to go through this network, here's a summary of the interesting results\n",
    "\n",
    " - 95% accuracy in 6s, 98.6% in 20 seconds, 99.2% in 1 min 20 sec on a 10 class problem using raw audio and no augmentation\n",
    " - Having a stride of 5-7 seems to be an optimal balance between computational cost and accuracy\n",
    " - Bigger filter sizes are better only up to 7x7 or 9x9 and then they actually decrease accuracy with increased cost\n",
    " - A variety of filter sizes (7,9,11) doesnt seem to beat out just individual filter sizes 7,9,11 but more testing is needed with a harder dataset\n",
    " - Testing tons of individual kernels on random subsets of the data and then selecting the best ones actually results in less accuracy vs the same number of random kernels (this is likely due to increased correlation between the good random kernels, making them worse in an ensemble)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%reload_ext autoreload\n",
    "%autoreload 2\n",
    "%matplotlib inline"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from local.torch_basics import *\n",
    "from local.test import *\n",
    "from local.basics import *\n",
    "from local.data.all import *\n",
    "from local.vision.core import *\n",
    "from local.notebook.showdoc import show_doc\n",
    "from local.audio.core import *\n",
    "from local.audio.augment import *\n",
    "from local.vision.learner import *\n",
    "from local.vision.models.xresnet import *\n",
    "from local.metrics import *\n",
    "from local.callback.schedule import *\n",
    "import torchaudio\n",
    "from fastprogress import progress_bar as pb\n",
    "import time\n",
    "from sklearn.model_selection import train_test_split\n",
    "from sklearn.linear_model import RidgeClassifierCV\n",
    "from rocket import generate_kernels, apply_kernel, apply_kernels"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "PosixPath('/home/jupyter/.fastai/data/ST-AEDS-20180100_1-OS/ST-AEDS-20180100_1-OS')"
      ]
     },
     "execution_count": null,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "p10speakers = Config()['data_path'] / 'ST-AEDS-20180100_1-OS'\n",
    "untar_data(URLs.SPEAKERS10, fname=str(p10speakers)+'.tar', dest=p10speakers)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "get_audio = AudioGetter(\"\", recurse=True, folders=None)\n",
    "files_10 = get_audio(p10speakers)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(#3842) [/home/jupyter/.fastai/data/ST-AEDS-20180100_1-OS/f0004_us_f0004_00446.wav,/home/jupyter/.fastai/data/ST-AEDS-20180100_1-OS/m0002_us_m0002_00128.wav,/home/jupyter/.fastai/data/ST-AEDS-20180100_1-OS/f0003_us_f0003_00279.wav,/home/jupyter/.fastai/data/ST-AEDS-20180100_1-OS/f0001_us_f0001_00168.wav,/home/jupyter/.fastai/data/ST-AEDS-20180100_1-OS/f0005_us_f0005_00286.wav,/home/jupyter/.fastai/data/ST-AEDS-20180100_1-OS/m0005_us_m0005_00282.wav,/home/jupyter/.fastai/data/ST-AEDS-20180100_1-OS/f0005_us_f0005_00432.wav,/home/jupyter/.fastai/data/ST-AEDS-20180100_1-OS/f0005_us_f0005_00054.wav,/home/jupyter/.fastai/data/ST-AEDS-20180100_1-OS/m0004_us_m0004_00110.wav,/home/jupyter/.fastai/data/ST-AEDS-20180100_1-OS/m0003_us_m0003_00180.wav...]"
      ]
     },
     "execution_count": null,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "files_10"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "audio_opener = OpenAudio(files_10)\n",
    "p10_labeler = lambda x: str(x).split('/')[-1][:5] #grab the label from each file"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "CLIP_LENGTH = 2000"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "\n",
       "    <div>\n",
       "        <style>\n",
       "            /* Turns off some styling */\n",
       "            progress {\n",
       "                /* gets rid of default border in Firefox and Opera. */\n",
       "                border: none;\n",
       "                /* Needs to be in here for Safari polyfill so background images work as expected. */\n",
       "                background-size: auto;\n",
       "            }\n",
       "            .progress-bar-interrupted, .progress-bar-interrupted::-webkit-progress-bar {\n",
       "                background: #F44336;\n",
       "            }\n",
       "        </style>\n",
       "      <progress value='3842' class='' max='3842', style='width:300px; height:20px; vertical-align: middle;'></progress>\n",
       "      100.00% [3842/3842 00:12<00:00]\n",
       "    </div>\n",
       "    "
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "sigs, labels = [],[]\n",
    "cropper = CropSignal(CLIP_LENGTH, pad_mode='repeat')\n",
    "remove_silence = RemoveSilence()\n",
    "for i in pb(range(len(files_10))):\n",
    "    sigs.append(cropper(remove_silence(audio_opener(i))).sig)\n",
    "    labels.append(p10_labeler(files_10[i]))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(3842, 3842)"
      ]
     },
     "execution_count": null,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "len(sigs), len(labels)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "total_size = len(sigs)\n",
    "train_size = int(total_size*.8)\n",
    "train_idxs = torch.randperm(total_size)[:train_size]\n",
    "valid_idxs = [i for i in range(total_size) if i not in train_idxs]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "assert len(train_idxs) + len(valid_idxs) == len(sigs)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "x_train = [sigs[idx].squeeze(0).numpy() for idx in train_idxs]\n",
    "y_train = [labels[idx] for idx in train_idxs]\n",
    "x_valid = [sigs[idx].squeeze(0).numpy() for idx in valid_idxs]\n",
    "y_valid = [labels[idx] for idx in valid_idxs]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[3073, 3073, 769, 769]"
      ]
     },
     "execution_count": null,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "list(map(len, (x_train, y_train, x_valid, y_valid)))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "((3073, 32000), (769, 32000))"
      ]
     },
     "execution_count": null,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "np_x_train = np.stack(x_train).astype(np.float64)\n",
    "np_x_valid = np.stack(x_valid).astype(np.float64)\n",
    "np_x_train.shape, np_x_valid.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "o2i_f = lambda x: 5*(x[0]=='m') + int(x[-1]) - 1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "np_y_train = np.array(list(map(o2i_f, y_train)))\n",
    "np_y_valid = np.array(list(map(o2i_f, y_valid)))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([0, 7, 2, ..., 0, 7, 1])"
      ]
     },
     "execution_count": null,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "np_y_train"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "((3073, 32000), (3073,), (769, 32000), (769,))"
      ]
     },
     "execution_count": null,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "np_x_train.shape, np_y_train.shape, np_x_valid.shape, np_y_valid.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "-4.49039649777175e-05"
      ]
     },
     "execution_count": null,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "np_x_train.mean()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Normalize the training data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "np_x_train = (np_x_train - np_x_train.mean(axis = 1, keepdims = True)) / (np_x_train.std(axis = 1, keepdims = True) + 1e-8)\n",
    "np_x_valid = (np_x_valid - np_x_valid.mean(axis = 1, keepdims = True)) / (np_x_valid.std(axis = 1, keepdims = True) + 1e-8)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(-8.10809639770585e-20, 0.9999995545301024)"
      ]
     },
     "execution_count": null,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "np_x_train.mean(), np_x_train.std()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "dtype('float64')"
      ]
     },
     "execution_count": null,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "np_x_train.dtype"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Start Here"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def timing_test(runs, candidate_lengths, stride, num_kernels, seq_length, show_progress=True):\n",
    "    times, scores = [],[]\n",
    "    for i in range(runs):\n",
    "        kernels = generate_kernels(seq_length, num_kernels, candidate_lengths, stride)\n",
    "        start = time.time()\n",
    "        x_train_tfm = apply_kernels(np_x_train, kernels)\n",
    "        x_valid_tfm = apply_kernels(np_x_valid, kernels)\n",
    "        classifier = RidgeClassifierCV(alphas=np.logspace(-3, 3, 7), normalize=True)\n",
    "        classifier.fit(x_train_tfm, np_y_train)\n",
    "        score = classifier.score(x_valid_tfm, np_y_valid)\n",
    "        t = time.time()-start\n",
    "        scores.append(score)\n",
    "        times.append(t)\n",
    "        if(show_progress): print(\"Finished Run\", i+1, \"Score:\", round(score, 3), \"Time:\", round(t,3))\n",
    "    return times, scores"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Initial attempts"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "timing_test(5, np.array((7,9,11)), stride=5, num_kernels=200, seq_length=16000)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "| Kernel Sizes     | Strides | Results |  Time  |\n",
    "| :--------------: | :-----: | :-----: | :-----:|      \n",
    "| {7,9,11}         | 7       | .85     |  4:02 |\n",
    "| {7,9,11}         | 5       | .899    |  5:20  |\n",
    "| {7,9,11}         | 3       | .903    |  8:15  |\n",
    "|{800,1000,1200}   | 400     | .46     |  3:43  |\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Silence removed results 10000 kernels**\n",
    "\n",
    "| Kernel Sizes     | Strides | Results |  Time  |\n",
    "| :--------------: | :-----: | :-----: | :-----:|      \n",
    "| {7,9,11}         | 5       | .979     |  5:08 |\n",
    "| {7,9,11}         | 5       | .976     |  5:10 |\n",
    "| {7,9,11}         | 5       | .980     |  5:26 |\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Silence removed results 2000 kernels**\n",
    "\n",
    "| Kernel Sizes     | Strides | Results |  Time  |\n",
    "| :--------------: | :-----: | :-----: | :-----:|      \n",
    "| {7,9,11}          | 5       | .972     |  1:10 |\n",
    "| {7,9,11}         | 5       | .979     |  1:03 |\n",
    "| {7,9,11}         | 5       | .976     |  1:01 |"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Silence removed results 1000 kernels**\n",
    "\n",
    "| Kernel Sizes     | Strides | Results |  Time  |\n",
    "| :--------------: | :-----: | :-----: | :-----:|      \n",
    "| {7,9,11}         | 5       | .974     |  0:31 |\n",
    "| {7,9,11}         | 5       | .974     |  0:31 |\n",
    "| {7,9,11}         | 5       | .966     |  0:31 |"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Silence removed results 200 kernels**\n",
    "\n",
    "| Kernel Sizes     | Strides | Results |  Time  |\n",
    "| :--------------: | :-----: | :-----: | :-----:|      \n",
    "| {7,9,11}         | 5       | .949     |  0:06 |\n",
    "| {7,9,11}         | 5       | .950     |  0:06 |\n",
    "| {7,9,11}         | 5       | .934     |  0:06|"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 1s Audio, silence removed, testing strides"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Silence removed results 200 kernels**\n",
    "\n",
    "| Kernel Sizes     | Strides | Results |  Time  |\n",
    "| :--------------: | :-----: | :-----: | :-----:|      \n",
    "| {7,9,11}         | 1       | .942     |  0:30 |\n",
    "| {7,9,11}         | 1       | .954     |  0:28 |\n",
    "| {7,9,11}         | 1       | .950     |  0:28|"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Silence removed results 200 kernels**\n",
    "\n",
    "| Kernel Sizes     | Strides | Results |  Time  |\n",
    "| :--------------: | :-----: | :-----: | :-----:|      \n",
    "| {7,9,11}         | 3       | .960     |  0:10 |\n",
    "| {7,9,11}         | 3       | .948     |  0:10 |\n",
    "| {7,9,11}         | 3       | .954     |  0:10 |"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Up clip length to 2 seconds"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Silence removed results 200 kernels**\n",
    "\n",
    "| Kernel Sizes     | Strides | Results |  Time  |\n",
    "| :--------------: | :-----: | :-----: | :-----:|      \n",
    "| {7,9,11}         | 5       | .987     |  0:15 |\n",
    "| {7,9,11}         | 5       | .980     |  0:13 |\n",
    "| {7,9,11}         | 5       | .986     |  0:13 |"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Silence removed results 200 kernels**\n",
    "\n",
    "| Kernel Sizes     | Strides | Results |  Time  |\n",
    "| :--------------: | :-----: | :-----: | :-----:|      \n",
    "| {7,9,11}         | 3       | .986     |  0:20 |\n",
    "| {7,9,11}         | 3       | .986     |  0:20 |\n",
    "| {7,9,11}         | 3       | .990     |  0:20 |"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Silence removed results 1000 kernels**\n",
    "\n",
    "| Kernel Sizes     | Strides | Results |  Time  |\n",
    "| :--------------: | :-----: | :-----: | :-----:|      \n",
    "| {7,9,11}         | 3       | .992     |  1:40 |\n",
    "| {7,9,11}         | 3       | .992     |  1:40 |\n",
    "| {7,9,11}         | 3       | .992     |  1:40 |"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Check impact of the kernel size options, do we really need to choose from 3?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "times_ks2, scores_ks2 = timing_test(10, np.array((2,)), stride=1, num_kernels=100, seq_length=32000, show_progress=False)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "times_ks3, scores_ks3 = timing_test(10, np.array((3,)), stride=1, num_kernels=100, seq_length=32000, show_progress=False)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "times_ks5, scores_ks5 = timing_test(10, np.array((5,)), stride=1, num_kernels=100, seq_length=32000, show_progress=False)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "times_ks7, scores_ks7 = timing_test(10, np.array((7,)), stride=1, num_kernels=100, seq_length=32000, show_progress=False)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "times_ks9, scores_ks9 = timing_test(10, np.array((9,)), stride=1, num_kernels=100, seq_length=32000, show_progress=False)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "times_ks11, scores_ks11 = timing_test(10, np.array((11,)), stride=1, num_kernels=100, seq_length=32000, show_progress=False)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "times_ksorig, scores_ksorig = timing_test(10, np.array((7,9,11,)), stride=1, num_kernels=100, seq_length=32000, show_progress=False)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def mn(x): return round(sum(x)/len(x), 3)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "all_scores = [scores_ks2, scores_ks3, scores_ks5, scores_ks7, scores_ks9, scores_ks11, scores_ksorig,]\n",
    "all_times = [times_ks2, times_ks3, times_ks5, times_ks7, times_ks9, times_ks11, times_ksorig,]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "mean_times = list(map(mn, all_times))\n",
    "mean_scores = list(map(mn, all_scores))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[<matplotlib.lines.Line2D at 0x7fad81118310>]"
      ]
     },
     "execution_count": null,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXoAAAD4CAYAAADiry33AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAah0lEQVR4nO3dfXRchXnn8e+jN0u2JVm2hTGSbdnhJRgMNsgmgWKztMlCm+KAu40hm5Nku3F2G9psD2zXbM+hu6QJacO25HQ5e9YJLuGEDUvdNKUpqWFdCJCceCRjDNjGidHIsmxjyx5Z8otkvcyzf2hkj0ZjaWSNdEd3fp9zfHTn3qvRM3PQj6vnuXOvuTsiIhJeBUEXICIiE0tBLyIScgp6EZGQU9CLiIScgl5EJOSKgi4g1dy5c72uri7oMkREppQdO3Ycd/fqdNtyLujr6upobGwMugwRkSnFzA5cbJtaNyIiIZdR0JvZXWa2z8z2m9nGNNsXmdk2M3vHzF4zs9rE+n9lZm8n/es2s09n+0WIiMjFjRr0ZlYIPAXcDSwF7jezpSm7PQE86+43AI8BjwO4+6vuvtzdlwN3AmeBl7NYv4iIjCKTI/pVwH53b3L3HuB5YG3KPkuBbYnlV9NsB/gd4CfufvZSixURkbHLJOhrgINJj1sT65LtAtYllu8Fys1sTso+64EfpPsBZrbBzBrNrLGtrS2DkkREJFOZBL2lWZd6JbSHgTVmthNYAxwC+s4/gdl8YBmwNd0PcPdN7l7v7vXV1WnPDhIRkUuUyemVrcCCpMe1wOHkHdz9MHAfgJnNBNa5e0fSLr8L/L27946vXBERGatMgr4BuMrMFjNwpL4eeCB5BzObC8TcPQ48AmxOeY77E+tFJI/E486pc310dvXS0dV74Wv3wFeAe26s4fLK0oArDbdRg97d+8zsQQbaLoXAZnffbWaPAY3u/iJwB/C4mTnwOvCVwe83szoG/iL4adarF5EJ19MXHxLOg4Hd2dVLZ3ffwLqzKdu7B9adOtfHaLe8+NbWfdxzYw0bVi/hmsvLJ+dF5RnLtRuP1NfXuz4ZK5I97s6Znv7zR9PJR9YdibDuHLbuwnJ3b3zE5y8tLqCitJjKsoF/FWVJy6VFVAxbV0zl9IHl2OkeNv8syv9tOEhXbz9rrq5mw+ol3PqROZilGw/KxZjZDnevT7tNQS+S+/r64+cDefjRdd+QgB4e2n30x0f+PS8vLboQwucDu2hYeFck7TO4fVpR4bhf38mzPTy3vYW/+Vkzx0+f47orKtiwegm/uWw+xYX6AH8mFPQiAXN3unvjw9ofF772jdgeOX2ub8TnLy6080FdMSSci0Y42h74OrO0iMKC3Dh67u7t5x/ePsSm15v4oO0MNbPK+OJtdaxftZCZ03Lu0lw5RUEvkgWjDRbTHV0Pruvs6qWnf+QWyIySwgshPSSwi4aF85DALiuirLgwVK2OeNx5dd8xNr3exPZojPLSIj57yyK+eFsd8yo0uE1HQS+SMNJgcbDNcamDxQJjSFAnB3HFsHXD+9hqUaS36+BJNr3RxE/ePUJhgbF2eQ1ful2D21QKegmN1MFiai86NbhTj6y7evtHfP5pRQXD+9KlQ3vVqWE+eMQ9c1pRqI6qc83B2FmefnPo4PbLq5fwcQ1uAQW95Jj+uI+h7TH0aLuzq5e+DAeL6cK4MiWok9sjFaXFlBaPf7AoE0uD2/QU9BKIeNx55ufNbN39IR1dvZxKHHFnPFi8yFkg6VoguThYlImlwe1QCnqZdEc7u3nohV28uf8419dUML+y7OKhPX1oWJcWF+hPccmYBrcDFPQyqbbu/pCNf/cO3b1x/vS3l/KZlQsU3DIp3j54ku/k6eBWQS+T4mxPH1/78V5+EGlhWU0lT65fzkeqZwZdluShlhNnh33iNuyDWwW9TLj3DnXwh8/vJHr8DBtWL+GhT1xDSVH+DsYkN7Sf6eG57Qd45ucHOH76HNfXVPCl28M5uFXQy4SJx53vvtnEt7buY/aMEv7qd5dz65Vzgy5LZIh8GNwq6GVCJA9c//V18/jmfTdQNaMk6LJELirMg1sFvWTdy7s/5L9o4CpTWNgGtwp6yZqunn6+9k97+D/bW7i+poJvr1+hgatMaamD2zuuqWbD7VNvcKugl6zQwFXCbKoPbhX0Mi7xuPP0m1H+Yuv7GrhK6HX39vOjnYf4zhtTa3CroJdLpoGr5Kt43PmX94+x6Y0mIlNgcKugl0uSPHB99LeXsl4DV8lTbx88yXdeb+In7+Xu4FZBL2OigatIei0nzvL0m0280Niac4NbBb1k7L1DHXz1+Z00aeAqclEXBrfNHD/dc35w+1vL5lMU0OBWQS+j0sBVZOzSDW7/3a8t5jMrF0z64FZBLyM62tnNw3+7izd+pYGryKVIHdxWlBbx2Y8t4gu3Tt7gVkEvF6WBq0h2pQ5uP728hi+tXsLV8yZ2cKugl2E0cBWZWGkHt6uX8PElEzO4VdDLEBq4ikyeyRrcKugF0MBVJEiDg9tNbzTRNAGDWwW9aOAqkiMmanCroM9zr+w5yh9v2aWBq0iO2dnSznffiJ4f3P7OzQv4xr3XX9Lv50hBn7tX6JFx6+rp58/+aQ/PaeAqkpNWLKziqc9WnR/cxp0JOQhT0IfU4MD1g7YzfHn1Eh76pAauIrlq4Zzp/Pe110/Y8yvoQyZ14Prcv7+F2zRwFclrCvoQ0cBVRNJR0IfE4MC1q7efb9y7jPtXaeAqIgMU9FNc6sD1yc+s4MrLNHAVkQsyms6Z2V1mts/M9pvZxjTbF5nZNjN7x8xeM7PapG0LzexlM9trZnvMrC575ee39w518Km/foPntrfw5dVL+OF/vE0hLyLDjHpEb2aFwFPAJ4BWoMHMXnT3PUm7PQE86+7fM7M7gceBzyW2PQt83d1fMbOZQDyrryAPaeAqImORSetmFbDf3ZsAzOx5YC2QHPRLgT9KLL8K/Cix71KgyN1fAXD301mqO28lD1w/uXQef75OA1cRGVkmrZsa4GDS49bEumS7gHWJ5XuBcjObA1wNnDSzH5rZTjP7VuIvhCHMbIOZNZpZY1tb29hfRZ54Zc9R7nrydRqaY3zj3mX878/drJAXkVFlEvTpTt1IvW7Cw8AaM9sJrAEOAX0M/MVwe2L7SmAJ8IVhT+a+yd3r3b2+uro68+rzRFdPP3/y9+/ypWcbuWJWGT/+g9t54JaFOqtGRDKSSeumFViQ9LgWOJy8g7sfBu4DSPTh17l7h5m1AjuT2j4/Aj4GPJ2F2vPC7sMdfPX5t9l/7PTAJYU/eTXTiob9USQiclGZBH0DcJWZLWbgSH098EDyDmY2F4i5exx4BNic9L1VZlbt7m3AnYCuWJaB1IHr93/vFn7tKg1cRWTsRg16d+8zsweBrUAhsNndd5vZY0Cju78I3AE8bmYOvA58JfG9/Wb2MLDNBvoMO4DvTMxLCY9jnd08pIGriGSJLlOcY5I/4frop67TJ1xFJCO6TPEUkPwJ1+uuGLiksD78JCLZoKDPARq4ishEUtAHKB53Nv8syl/88z5mTS/WwFVEJoSCPiDJA9dPJAauszVwFZEJoKAPgC4pLCKTSUE/iTRwFZEgKOgniQauIhIUBf0E08BVRIKmoJ9AGriKSC5Q0E8QDVxFJFco6LOsrz/Of/vH3Xz/Fxq4ikhuUNBn2f/be5Tv/6KFL95Wx8a7P6qBq4gETkGfZb9oilFWXMgjd19LSVFG914XEZlQSqIsa2iOsWLhLIW8iOQMpVEWdXb3sudIJ6sWzw66FBGR8xT0WbTjQDvusKpOQS8iuUNBn0WRaIyiAmPFwqqgSxEROU9Bn0UN0Rg31FZSVqIzbUQkdyjos6S7t59drSdZqf68iOQYBX2W7Gw5SW+/c4uCXkRyjII+SxqaY5jBzYsU9CKSWxT0WRKJxvjo5RVUlhUHXYqIyBAK+izo7Y/zVks7q+p0to2I5B4FfRbsPtzJ2Z5+Vi2eE3QpIiLDKOizIBI9AcDKxTqiF5Hco6DPgki0ncVzZ3BZeWnQpYiIDKOgH6d43GlojumyByKSsxT04/SrY6fp6OrVB6VEJGcp6MdpsD+vD0qJSK5S0I/T9miM+ZWl1FaVBV2KiEhaCvpxcB/oz6+sm60bf4tIzlLQj0NL7CxHO8/pRiMiktMU9OMQicYAFPQiktMU9OMQicaoml7MldUzgy5FROSiFPTjEGmOUV83m4IC9edFJHdlFPRmdpeZ7TOz/Wa2Mc32RWa2zczeMbPXzKw2aVu/mb2d+PdiNosP0tHObg6cOKvTKkUk5xWNtoOZFQJPAZ8AWoEGM3vR3fck7fYE8Ky7f8/M7gQeBz6X2Nbl7suzXHfg1J8XkakikyP6VcB+d29y9x7geWBtyj5LgW2J5VfTbA+dhuYYM0oKWTq/IuhSRERGlEnQ1wAHkx63JtYl2wWsSyzfC5Sb2eA1e0vNrNHMfmFmn073A8xsQ2Kfxra2tjGUH5xINMZNi6ooKtSYQ0RyWyYplW7S6CmPHwbWmNlOYA1wCOhLbFvo7vXAA8CTZvaRYU/mvsnd6929vrq6OvPqA3LybA/vf3hKFzITkSlh1B49A0fwC5Ie1wKHk3dw98PAfQBmNhNY5+4dSdtw9yYzew1YAXww7soD1NjcDqg/LyJTQyZH9A3AVWa22MxKgPXAkLNnzGyumQ0+1yPA5sT6KjObNrgPcBuQPMSdkiLNMUoKC7hxwaygSxERGdWoQe/ufcCDwFZgL/CCu+82s8fM7J7EbncA+8zsl8A84OuJ9dcCjWa2i4Eh7TdTztaZkiLRGDcuqKS0uDDoUkRERpVJ6wZ3fwl4KWXdo0nLW4Atab7v58CycdaYU86c6+O9Qx18ec2SoEsREcmIThkZo50tJ+mLOys1iBWRKUJBP0aR5hgFBjcv0o3ARWRqUNCPUSR6guuuqKS8tDjoUkREMqKgH4Oevjg7W06qbSMiU4qCfgzePXSSc31xnT8vIlOKgn4MItGBD0qtrFN/XkSmDgX9GESiJ7jyspnMmTkt6FJERDKmoM9Qf9xpbG5Xf15EphwFfYbe/7CTU+f6dKMREZlyFPQZ0o1GRGSqUtBnqKE5Rs2sMq6YVRZ0KSIiY6Kgz4C7E4nG1LYRkSlJQZ+BpuNnOH66h5UKehGZghT0GWhQf15EpjAFfQYi0RhzZ5awZO6MoEsRERkzBX0GIs0xVtbNxizd7XNFRHKbgn4Uh0520drepbaNiExZCvpRDPbn9YlYEZmqFPSjiDTHKJ9WxLXzK4IuRUTkkijoRxGJxri5rorCAvXnRWRqUtCP4MTpc+w/dlr9eRGZ0hT0I2hoHrj+vD4RKyJTmYJ+BJFojGlFBSyrmRV0KSIil0xBP4KG5hgrFs6ipEhvk4hMXUqwizjV3cvuwx2s0mmVIjLFKegv4q2Wk8QdVi2eE3QpIiLjoqC/iEj0BEUFxk2L1J8XkalNQX8RkWiM62oqmV5SFHQpIiLjoqBPo7u3n10HO3RapYiEgoI+jV0HT9LTH9f1bUQkFBT0aTQ0D17IrCrgSkRExk9Bn8b2aIxr5pUza3pJ0KWIiIybgj5FX3+ctw606/o2IhIaCvoUe450cqanX0EvIqGRUdCb2V1mts/M9pvZxjTbF5nZNjN7x8xeM7PalO0VZnbIzP5ntgqfKBHdCFxEQmbUoDezQuAp4G5gKXC/mS1N2e0J4Fl3vwF4DHg8ZfvXgJ+Ov9yJF4nGWDRnOvMqSoMuRUQkKzI5ol8F7Hf3JnfvAZ4H1qbssxTYllh+NXm7md0MzANeHn+5EysedxoSNwIXEQmLTIK+BjiY9Lg1sS7ZLmBdYvleoNzM5phZAfA/gP883kInwwdtp2k/26u2jYiESiZBn+4eep7y+GFgjZntBNYAh4A+4PeBl9z9ICMwsw1m1mhmjW1tbRmUNDG2J/rz+kSsiIRJJhdyaQUWJD2uBQ4n7+Duh4H7AMxsJrDO3TvM7OPA7Wb2+8BMoMTMTrv7xpTv3wRsAqivr0/9n8ikiURjXFY+jYWzpwdVgohI1mUS9A3AVWa2mIEj9fXAA8k7mNlcIObuceARYDOAu382aZ8vAPWpIZ8r3J1INMaqxbMx043ARSQ8Rm3duHsf8CCwFdgLvODuu83sMTO7J7HbHcA+M/slA4PXr09QvROmtb2LDzu71Z8XkdDJ6Bq87v4S8FLKukeTlrcAW0Z5jmeAZ8Zc4STR+fMiElb6ZGxCJBqjsqyYqy8rD7oUEZGsUtAnRJpjrKyroqBA/XkRCRcFPXDsVDfR42fUthGRUFLQAw3RdgB9IlZEQklBz8CNRsqKC7m+pjLoUkREsk5Bz8AnYm9aNIviQr0dIhI+eZ9sHV29vP9hJ6vq5gRdiojIhMj7oN9xIIa7zp8XkfDK+6DfHo1RXGisWDgr6FJERCZE3gd9QzTGDbWzKC0uDLoUEZEJkddB39XTzzutHTqtUkRCLa+DfmdLO31x1/XnRSTU8jroI80xzODmuqqgSxERmTD5HfTRGNdeXkFFaXHQpYiITJi8DfqevjhvtbTrtEoRCb28Dfr3DnfQ3RtX0ItI6OVt0A/eaERn3IhI2OVt0DdEYyyZO4Pq8mlBlyIiMqHyMujjcaehOaa2jYjkhbwM+n1HT9HZ3aegF5G8kJdBr/68iOSTvA36KypLqa0qC7oUEZEJl3dB7+4DNwJfPBsz3QhcRMIv74K++cRZ2k6dU39eRPJG3gV9Q6I/rwuZiUi+yLug3x6NMXtGCR+pnhl0KSIikyLvgr6hOcbKuir150Ukb+RV0B/p6KIldlanVYpIXsmroI+c78/PCbgSEZHJk1dB39AcY0ZJIdfOLw+6FBGRSZNXQR+Jxri5bjZFhXn1skUkz+VN4rWf6eGXR0/rtEoRyTt5E/QNzbq+jYjkp7wJ+kg0RklRATfUVgZdiojIpMqboG9ojrG8dhalxYVBlyIiMqkyCnozu8vM9pnZfjPbmGb7IjPbZmbvmNlrZlabtH6Hmb1tZrvN7D9k+wVk4sy5Pt473Knr24hIXho16M2sEHgKuBtYCtxvZktTdnsCeNbdbwAeAx5PrD8C3Oruy4FbgI1mdkW2is/UWy3t9MddQS8ieSmTI/pVwH53b3L3HuB5YG3KPkuBbYnlVwe3u3uPu59LrJ+W4c/Lukg0RoHBTYuqgvjxIiKByiR4a4CDSY9bE+uS7QLWJZbvBcrNbA6AmS0ws3cSz/Hn7n54fCWP3fZojOtrKpk5rWiyf7SISOAyCfp0V//ylMcPA2vMbCewBjgE9AG4+8FES+dK4PNmNm/YDzDbYGaNZtbY1tY2phcwmnN9/bx98KROqxSRvJVJ0LcCC5Ie1wJDjsrd/bC73+fuK4A/SazrSN0H2A3cnvoD3H2Tu9e7e311dfUYX8LI3mntoKcvrv68iOStTIK+AbjKzBabWQmwHngxeQczm2tmg8/1CLA5sb7WzMoSy1XAbcC+bBWfCd0IXETy3ahB7+59wIPAVmAv8IK77zazx8zsnsRudwD7zOyXwDzg64n11wLbzWwX8FPgCXd/N8uvYUSRaIyrLpvJ7Bklk/ljRURyRkbTSXd/CXgpZd2jSctbgC1pvu8V4IZx1njJ+uPOjgPtrF0+6Wd0iojkjFB/MnbvkU5On+tTf15E8lqog367+vMiIuEO+oZojNqqMq6YVRZ0KSIigQlt0Ls7keaY2jYikvdCG/QftJ0mdqZHNxoRkbwX2qCPRNsB9edFREIc9CeYO3Mai+fOCLoUEZFAhTboG5rbWbW4CrN0l+oREckfoQz61vazHDrZxSq1bUREwhn0g9e3WbV4TsCViIgEL5RB39Aco7y0iGsuLw+6FBGRwIUy6LdHY6ysm01hgfrzIiKhC/rjp8/R1HZGp1WKiCSELugbzvfnFfQiIhDCoI80xygtLmBZTWXQpYiI5ITwBX00xooFVZQUhe6liYhcklClYWd3L3uOdKptIyKSJFRBv+NAO+7qz4uIJAtV0EeiMYoKjBULZwVdiohIzghV0DdEY1xfU8n0koxuhSsikhdCE/Tdvf3saj2p68+LiKQITdB3dvfym8vms+bq6qBLERHJKaHpcVxWXsq3168IugwRkZwTmiN6ERFJT0EvIhJyCnoRkZBT0IuIhJyCXkQk5BT0IiIhp6AXEQk5Bb2ISMiZuwddwxBm1gYcGMdTzAWOZ6mcqU7vxVB6P4bS+3FBGN6LRe6e9tIAORf042Vmje5eH3QduUDvxVB6P4bS+3FB2N8LtW5EREJOQS8iEnJhDPpNQReQQ/ReDKX3Yyi9HxeE+r0IXY9eRESGCuMRvYiIJFHQi4iEXCiC3swWmNmrZrbXzHab2VeDrikXmFmhme00sx8HXUvQzGyWmW0xs/cT/518POiagmJmf5T4PXnPzH5gZqVB1zSZzGyzmR0zs/eS1s02s1fM7FeJr1VB1phtoQh6oA94yN2vBT4GfMXMlgZcUy74KrA36CJyxLeBf3b3jwI3kqfvi5nVAH8I1Lv79UAhsD7YqibdM8BdKes2Atvc/SpgW+JxaIQi6N39iLu/lVg+xcAvcU2wVQXLzGqB3wK+G3QtQTOzCmA18DSAu/e4+8lgqwpUEVBmZkXAdOBwwPVMKnd/HYilrF4LfC+x/D3g05Na1AQLRdAnM7M6YAWwPdhKAvck8MdAPOhCcsASoA34m0Qr67tmNiPoooLg7oeAJ4AW4AjQ4e4vB1tVTpjn7kdg4MARuCzgerIqVEFvZjOBvwP+k7t3Bl1PUMzsU8Axd98RdC05ogi4Cfhf7r4COEPI/jTPVKL3vBZYDFwBzDCzfxtsVTLRQhP0ZlbMQMg/5+4/DLqegN0G3GNmzcDzwJ1m9v1gSwpUK9Dq7oN/5W1hIPjz0W8AUXdvc/de4IfArQHXlAuOmtl8gMTXYwHXk1WhCHozMwb6r3vd/S+Drido7v6Iu9e6ex0Dg7Z/cfe8PWpz9w+Bg2Z2TWLVrwN7AiwpSC3Ax8xseuL35tfJ08F0iheBzyeWPw/8Q4C1ZF1R0AVkyW3A54B3zeztxLr/6u4vBViT5JY/AJ4zsxKgCfhiwPUEwt23m9kW4C0GzlbbScg//p/KzH4A3AHMNbNW4E+BbwIvmNnvMfA/w38TXIXZp0sgiIiEXChaNyIicnEKehGRkFPQi4iEnIJeRCTkFPQiIiGnoBcRCTkFvYhIyP1/1mDwEZhg4tQAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.plot([2,3,5,7,9,11], mean_scores[:6])\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[<matplotlib.lines.Line2D at 0x7fad819cd1d0>]"
      ]
     },
     "execution_count": null,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXAAAAD4CAYAAAD1jb0+AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAgAElEQVR4nO3dd3iV5f3H8fedQdgjZAAhIYsVNoQtQ0BAqOKkjrpKRey0WFvl1zparaOK2mpFKhZrrYqAohJEHICIooyQhBVI2GSxEyD7/v2RYNECCTHJc55zPq/rykXynHNyvjxJPnnyPfcw1lpERMR9/JwuQEREakYBLiLiUgpwERGXUoCLiLiUAlxExKUC6vPJQkJCbHR0dH0+pYiI661bt+6gtTb0u8frNcCjo6NZu3ZtfT6liIjrGWN2n+24WigiIi6lABcRcSkFuIiISynARURcqsoAN8Y0NMZ8ZYzZaIzZZIx5qPL4XGPMTmNMcuVb77ovV0RETqvOKJQiYJS1tsAYEwisMsYsqbztHmvt/LorT0REzqXKALcVyxUWVH4YWPmmJQxFRBxWrR64McbfGJMM5ALLrLVrKm96xBiTYox52hgTdI7HTjXGrDXGrM3Ly6ulskVE3OHoyWIefHcT+YUltf65qxXg1toya21voD0wwBjTHbgP6AL0B4KB353jsbOttYnW2sTQ0P+ZSCQi4rVWpucx7pmV/PvL3azJPFzrn/+CRqFYa48Cy4Hx1tosW6EI+CcwoNarExFxoVPFZdy/KI2bX/6KZg0DefunQxmTEF7rz1NlD9wYEwqUWGuPGmMaAWOAx40xba21WcYYA1wBpNV6dSIiLpO89yjT30wm8+AJplwUwz3jOtMw0L9Onqs6o1DaAq8YY/ypuGKfZ6193xjzSWW4GyAZmFYnFYqIuEBJWTl/+2QHz3+6g/BmQfznJwMZEh9Sp89ZnVEoKUCfsxwfVScViYi4zI7cfH795kZS9x/jqr4RPHBZN1o0Cqzz563X1QhFRLxJebll7updPP7BVho38OeFG/tyaY+29fb8CnARkRo4cPQU98zfyOc7DjGqSxiPXd2DsGYN67UGBbiIyAWw1vJO8n7uX7SJsnLLo1f14Lr+kVSM56hfCnARkWo6cqKY/3snlaTUbPp1aMXMyb3o0LqJY/UowEVEquHTbbn8dn4KR08W89vxnbljeBz+fvV/1X0mBbiIyHmcKCrlkaQt/GfNHjqHN2Pubf3p1q6F02UBCnARkXNat/sI0+cls+fwSaYOj2X6JZ3qbFJOTSjARUS+o7i0nGc/TueF5Rm0bdGIN24fxMDY1k6X9T8U4CIiZ0jPyefXbyaz6cBxru3XnvsvS6BZw7qflFMTCnARESom5bz8+U6eWLqNZkEBzL6pH2O7tXG6rPNSgIuIz9t35CR3z9vImp2HuSQhnEev6kFI07NuceBRFOAi4rOstcxft4+H3tsMwBPX9OTafu0dmZRTEwpwEfFJhwqKmPF2Kks35TAgJpinru1FZHBjp8u6IApwEfE5H23O4d6FKRw/VcqMCV2YclGs45NyakIBLiI+o6ColD+9t5k31+6la9vm/PsnvejSprnTZdWYAlxEfMJXOw9z91vJ7D9yijtHxnHXmI4EBXjOpJyaUICLiFcrKi1j5rJ0Zq/MJLJVY+bdMZjE6GCny6oVCnAR8Vpbso7z6zeT2Zqdz/UDIvn9xASaBHlP7HnP/0REpFJZuWX2ykxmLttGi0YNmHNLIqO71v6u8E5TgIuIV9l7+CTT5yXz9a4jjO/Whkeu7E5rF0zKqQkFuIh4BWst89bu5Y/vbcbPGGZO7sWVfSJcMymnJhTgIuJ6eflF3LcwhY+25DI4tjVPTu5FRMtGTpdV5xTgIuJqH6RlM+PtVAqKSvnDDxK4bUg0fi6clFMTCnARcaXjhSU89O5mFqzfR/eI5jw9uTcdw5s5XVa9UoCLiOt8kXGI37y1kaxjp/jFqHh+MaojDQL8nC6r3inARcQ1CkvKeHLpNuZ8vpMOwY2Zf+cQ+ka1crosxyjARcQV0vYfY/q8ZNJzCvjRoChmTOhK4wa+HWG+/b8XEY9XWlbOiyszeeajdFo1bsDc2/ozsnOY02V5BAW4iHisXQdPMH1eMuv3HGViz7Y8PKk7rZo0cLosj6EAFxGPY63lP1/t4eH3txDob3j2ut5c3qudV0/KqQkFuIh4lNzjhfx2QQrLt+UxrGMIT1zTk7YtvH9STk0owEXEYyxOyeL/3knlVHEZD16WwM2DfWdSTk0owEXEccdOlfDAojTeST5Ar/YteGpyb+LDmjpdlsdTgIuIoz7fcZDfvLWR3Pwi7hrTkZ9dHE+gv+9NyqkJBbiIOKKwpIzHlmxl7updxIY2YeGdQ+gV2dLpslylygA3xjQEVgJBlfefb619wBgTA7wBBAPrgZustcV1WayIeIeUfUf59ZvJZOSd4NYh0fxufBcaNXD3/pROqM4VeBEwylpbYIwJBFYZY5YA04GnrbVvGGNmAVOAF+qwVhFxudKycp7/NIO/fbKdkKZBvDplAMM6hjpdlmtVGeDWWgsUVH4YWPlmgVHADZXHXwEeRAEuIueQkVfA9Hkb2bj3KJN6t+OPl3enReNAp8tytWr1wI0x/sA6IB54HsgAjlprSyvvsg+IOMdjpwJTAaKior5vvSLiMtZaXv1yN39O2kJQgD/P3dCHH/Rs53RZXqFaAW6tLQN6G2NaAm8DXc92t3M8djYwGyAxMfGs9xER75R9rJB75m/ks+0HGdEplCeu6Ul484ZOl+U1LmgUirX2qDFmOTAIaGmMCai8Cm8PHKiD+kTEpRYl7+cP76RRUmZ5+Iru3DgwSlPha1l1RqGEAiWV4d0IGAM8DnwKXEPFSJRbgEV1WaiIuMPBgiIeeHcTi1Oy6B3Zkqd/2JuYkCZOl+WVqnMF3hZ4pbIP7gfMs9a+b4zZDLxhjHkY2ADMqcM6RcTDWWtZsH4/Dy/ezMmiMu6+pBN3jowjQJNy6kx1RqGkAH3OcjwTGFAXRYmIu+w5dJIZb6eyasdBEju04rGrexAf5lv7UzpBMzFFpMZKy8r55+e7eGrZNgL8/PjTFd25cUCUFqCqJwpwEamRTQeOce+CVFL3H2NM1zD+OKk77Vpq2df6pAAXkQtSWFLGsx9vZ/bKTFo1DuS5G/owsUdbjTBxgAJcRKrti4xD3LcwhV2HTnJtv/b838SutGysLc6cogAXkSodO1nCo0u28MbXe4kKbsxrPxnI0PgQp8vyeQpwETmvJalZ3P/uJg4VFHHH8FjuGtNJKwd6CAW4iJxV9rFC7l+Uxoebc+jWrjn/vLU/3SNaOF2WnEEBLiLfUl5uef3rPTyWtJXisnLuvbQLP7koRhNyPJACXES+kZFXwH0LU/lq52EGx7bm0at6EK1p8B5LAS4ilJSV8+KKDP76yQ4aBvjxxNU9uTaxvYYGejgFuIiPS957lHsXpLA1O5+JPdrywOUJhDXTkq9uoAAX8VEnikp56sN05q7eSVizhsy+qR9ju7Vxuiy5AApwER+0Ij2PGQtT2X/0FD8aFMVvx3eheUNtb+Y2CnARH3L4RDEPv7+ZhRv2ExfahLemDaZ/dLDTZUkNKcBFfIC1lnc3HuCh9zZz/FQJvxwVz08vjqdhoCbkuJkCXMTL7Ttykt+/k8bybXn0jmzJY1f3oEub5k6XJbVAAS7ipcrKLf/6Yhd/WboNgAcuS+DmwdH4a61ur6EAF/FC27Lz+d2CFJL3HmVk51AevqI77Vs1drosqWUKcBEvUlRaxvOf7ODvyzNo3iiQZ6/rzeW92mlCjpdSgIt4ia93HebeBSlk5J3gqj4R/P4HCQQ30Vrd3kwBLuJyxwtLeOKDrfz7yz1EtGzEKz8ewIhOoU6XJfVAAS7iYss25/CHd9LIzS9kykUxTL+kE02C9GPtK/SVFnGh3PxCHnp3M4tTs+jSphmzbupH78iWTpcl9UwBLuIi1lreWruPhxdvprC0nHvGdWbq8FgCtVa3T1KAi7jEroMnmPF2KqszDjEgJphHr+pBXGhTp8sSBynARTxcaVk5L63aydPL0mng78efr+zBdf0j8dOEHJ+nABfxYGn7j/G7BSlsOnCcsQnh/OmK7oQ311rdUkEBLuKBThWX8cxH6by0aifBTRow60d9Gd+9rdNliYdRgIt4mM93HOS+hansOXyS6wdEcu+lXWnRSGt1y/9SgIt4iKMni3lk8RbeWrePmJAmvH77IAbHtXa6LPFgCnARh1lrWZyaxYPvbuLIyRJ+OjKOX47uqLW6pUoKcBEHZR07xR/eSeOjLbn0iGjBv348kIR2WqtbqkcBLuKA8nLLa2t28/gH2ygtL+f3E7ty65BoAjQhRy6AAlyknu3IzefeBams3X2EYR1DeOSKHkS11lrdcuEU4CL1pLi0nBeWZ/D8pztoHOTPU9f24qq+EVqrW2qsygA3xkQC/wLaAOXAbGvts8aYB4HbgbzKu86w1ibVVaEibrZu9xHuW5hCek4Bl/dqx/2XJRDSNMjpssTlqnMFXgrcba1db4xpBqwzxiyrvO1pa+2TdVeeiLsVFJXy5NJtvPLFLto2b8jLtyYyqku402WJl6gywK21WUBW5fv5xpgtQERdFybidp9uzeX376Rx4NgpbhkczW/Gdaap1uqWWnRB303GmGigD7AGGAr83BhzM7CWiqv0I2d5zFRgKkBUVNT3LFfE8x0qKOKh9zbz7sYDdAxryvxpQ+jXoZXTZYkXMtba6t3RmKbACuARa+1CY0w4cBCwwJ+AttbaH5/vcyQmJtq1a9d+z5JFPJO1lrc37OdP72+moKiUn1/ckWkjYwkK0IQc+X6MMeustYnfPV6tK3BjTCCwAHjNWrsQwFqbc8bt/wDer6VaRVxn18ET/GFRGp9tP0i/Dq147KoedAxv5nRZ4uWqMwrFAHOALdbamWccb1vZHwe4EkirmxJFPNehgiL+9skO/v3lboIC/PjjpG78aGAHrdUt9aI6V+BDgZuAVGNMcuWxGcD1xpjeVLRQdgF31EmFIh7oVHEZL3++k1nLMzhZUsYP+0dy15iOhDXTWt1Sf6ozCmUVcLbLCY35Fp9TVm5ZsH4fMz9MJ/t4IZckhPO78Z2JD1O7ROqfxjSJVIO1lhXpeTy2ZCtbs/PpFdmSZ6/rzcBYLfcqzlGAi1Qhbf8xHl2yhc93HKJD68Y8f0NfJvRooynw4jgF+AV47pPtbM8tYMpFMfRs39LpcqSO7Ttykqc+TOftDftp1TiQBy5L4MaBHWgQoBUDxTMowKvpZHEpz326g8KSchYlH2BofGumjYjjovgQXYl5mWMnS/j78h38c/UuDHDnyDjuHBlH84ba1kw8iwK8mj7dmkdhSTmzb+rHzoMnmLNqJzfN+YruEc25Y3gcE3q0xV9Dx1ytqLSMV7/Yzd8+2cHxwhKu7tue6Zd0ol3LRk6XJnJWCvBqSkrNIqRpA0Z3Dcffz3Dr0GjeXr+f2Ssz+cXrG3jyw23cPiyWa/q111ZYLlNebnkv5QB/WbqNfUdOMbxTKPeO76KdccTjKcCr4VRxGZ9szeWqvhHfXGUHBfhz3YAork2MZNnmbF5Ykcnv30njmY/SuW1oDD8a1EE7ibvAFxmHeHTJFlL2HSOhbXNendKDYR1DnS5LpFoU4NWwfFsup0rKmNCj7f/c5u9nGN+9LeO6teHLzMPMWpHBX5Zu4++f7uCGgVFMuSiWNi00ucPTpOfk89iSrXyyNZd2LRoyc3IvrugdoRmU4ioK8GpYnJpFcJMGDIwJPud9jDEMjmvN4LjWbDpwjBdXZDJn1U7mrt7FFb0juGNErCZ7eICc44U8vSydeWv30iQogHsv7cKtQ6LV9hJXUoBXobCkon0yqXdEtTec7dauBX+9vg/3jOvMPz7L5M2v9/LWun2MTQhn2sg4+kZpadH6VlBUyosrMvjHZ5mUlVtuGxrDzy+Op1WTBk6XJlJjCvAqLN+Wy8niMiaepX1SlcjgxvxxUnd+Nbojr6zexStf7ObDzTkMiAnmzhFxjOwcqiGIdaykrJw3vtrDMx9t59CJYi7r1Y57xnbWJsLiFRTgVVicmk1wkwYMij13+6QqrZsGMX1sZ+4YEccbX+9lzmeZ3Db3a7q0acYdI2L5Qc92BFbz6l6qx1rL0k05PPHBVjIPnmBgTDAvT+hKr0hNwBLvoQA/j8KSMj7eksOk3u2q3T45nyZBAUy5KIabB3fg3eQDzFqRwa/f3MiTS9O5fVgMk/tH0riBviTf17rdh/lz0lbW7T5CfFhT5tySyKguYfprR7yO0uI8lm/L42Tx2UeffB+B/n5c3a89V/aJ4JOtucxakcGD723m2Y+3c8uQaG4ZHK3ebA3sPHiCJz7YypK0bEKbBfHoVT24tl/7WvnlK+KJFODnkZSaRavGgQyuoxXn/PwMYxLCGZMQztpdFUMQn/loOy+uyOSH/SP5ybAY2rdSr7YqBwuK+OvH2/nPmj0EBfgx/ZJO/GRYjP6aEa+n7/BzON0+uaxX7bRPqpIYHcxL0cGk5+Qza0UG//5yN69+uZtJvdpxx4g4OrfREMTvOlVcxpxVmcxakcmpkjKuHxDJr0Z3IrRZkNOlidQLBfg5rEzP40RxGZfWcvukKp3CmzFzcm/uHtuZOZ/t5I2v97Bww35GdQlj2og4+ke38vleblm5ZcG6fTy1bBs5x4sYmxDO7y7tQlxoU6dLE6lXCvBzSErNomXjQIbEObNgf0TLRtx/WQK/HB3Pv77YzdzVu5j84hf0jWrJtBFxjOka7nOzBq21LN9WsanCtpx8+kS15Lkb+tI/uuYjhETcTAF+FoUlZXy0JZcJPdo4PryvZeMG/HJ0R24fFstb6/Yye2UmU19dR3xYU+4YHsuk3hE+sT516r6KTRVWZxwiunVj/n5jXy7trk0VxLcpwM/is+0HKSgqrfXRJ99Howb+3Dw4mhsGRLE4NYsXlmdwz/wUZi5LZ8pFMVw3IIqmQd735dx7+CRPfriNRckHCG7SgIcu78b1A6J84peWSFW87ye+FiSlZtGiUSBD40OcLuV/BPj7Mal3BJf3aseK9Dxmrcjg4cVb+OvH27l5cDS3Do0mpKn7X8Q7erKY5z/dwSurd2MM/OziOO4YoU0VRM6kAP+OotIyPtqcw/juzrdPzscYw8jOYYzsHMaGPUeYtSKD55fv4B+fZTI5MZLbh8W6crp4YUnFpgrPfVqxqcI1fdszfWwn2rbQpgoi36UA/47P0g+SX1TKhJ6e0z6pSp+oVrx4UyIZeQXMXpHJG1/v4bU1u5nYsx3TRsTSrV0Lp0usUnm55d2NFZsq7D96ihGdQrn30i50batNFUTORQH+HUmpWTRvGMDQOM9rn1QlLrQpj1/Tk+ljO/Hyqp28tmYP7208wLCOIdw5Io7Bca098kW/1TsO8uclW0jbf5xu7Zrz+NU9uaij+86/SH1TgJ+hqLSMZZtzGNe9jatfJAtv3pD7JnTlpxfH89qa3by8ahc3vLSGnu1bMG1EHOO6tfGI/Tu3Zefz6JItLN+WR0TLRjz9w15M6qVNFUSqSwF+hs93VLZPerRxupRa0aJRID8dGc+Ph8awcP1+Zq/M4KevrScmpAlTh8dyZZ8IRzYyyD5WyMxl25i/bh9NgwKYMaELNw/WpgoiF0oBfobFKdk0axjARfHetSdiw0B/bhgYxQ/7R/JBWjazVmRw38JUZi5L58dDY7hxUFS9jO7ILyzhxRWZvLQqk/Jy+PHQGH6mTRVEakwBXqm4tJxlm7O5JCHc1e2T8/H3M0zs2ZYJPdqwOuMQs1Zk8PgHWyv27xwUxZShMYQ1r/39O0vKyvnPmj08+/F2Dp8o5vJe7bhnXGcig903SkbEkyjAK32+4yDHC0trtPOO2xhjGBofwtD4ENL2H2PWigz+sTKTf67axVV9I5g6PJbYWlhXxFrLB2nZPLF0GzsPnmBQbDAzJnSlZ3ttqiBSGxTglRanZtEsKMDnRj90j2jBczf0ZfehE8xemclb6/bx5tq9jO/Whmkj4mq8g83aXYf5c9IW1u85Ssewprx8ayIXd9amCiK1SQFORfvkw00V7ZOgAN98Ia1D6yY8cmUP7hrTibmrd/LqF7tZkpbN4NjWTBsZx/COIdUK34y8Ap74YCtLN+UQ1iyIx6/uwdV9tamCSF1QgAOfZ1S0Tzxp7ROnhDYL4p5xXbhzZDyvr9nDS6syueXlr0ho25xpI+OY0L3NWcM4L79yU4Wv9tAwwI+7L+nEFG2qIFKn9NMFJKVk0TQogGGdfKt9cj5NgwK4fXgsNw/pwKINB5i1MoNfvr6BvwQ3YuqwWK5NjKRhoD8ni0t56bOdvLgig8LScm4YEMWvxnT0ivVYRDydzwd4SVk5H27OYUzXMJ9tn5xPUIA/k/tHck2/9izbksOsFRn8YdEmnvloO5f1akdSaha5+UWM79aGe8Z31qYKIvXI5wN8dcYhjp0qUfukCn5+hnHd2jA2IZyvdlbs3zl39S76RrXk7zf2JVGbKojUuyoD3BgTCfwLaAOUA7Ottc8aY4KBN4FoYBcw2Vp7pO5KrRun2yfDO3nX5J26YoxhYGxrBsa25nhhCc2CAjSyRMQh1RkaUArcba3tCgwCfmaMSQDuBT621nYEPq782FVKyspZujmb0V3DNI27Bpo3DFR4izioygC31mZZa9dXvp8PbAEigEnAK5V3ewW4oq6KrCtfZBzi6Em1T0TEnS5ocK4xJhroA6wBwq21WVAR8kDYOR4z1Riz1hizNi8v7/tVW8uSUrNo0sCfEWqfiIgLVTvAjTFNgQXAXdba49V9nLV2trU20VqbGBrqOUFZUlbO0k3ZjO4arvaJiLhStQLcGBNIRXi/Zq1dWHk4xxjTtvL2tkBu3ZRYN77MPMQRtU9ExMWqDHBT8SrVHGCLtXbmGTe9C9xS+f4twKLaL6/uJKVm0biBPyM7e85fBSIiF6I648CHAjcBqcaY5MpjM4DHgHnGmCnAHuDauimx9pWWlbN0Uw6jumj0iYi4V5UBbq1dBZxrrNjo2i2nfqzZeZjDJ4p9YulYEfFePrlE3OLULBoF+jOy81kHzoiIuILPBXhpWTlL07IZ1TWMRg3UPhER9/K5AP9q52EOqX0iIl7A5wL8dPvkYrVPRMTlfCrAy8otSzdlM6qL2ici4n4+FeBrdh7iYEGxJu+IiFfwqQBPSs2iYaAfF3fR5B0RcT+fCfCycssHaTlc3DlM+zSKiFfwmQD/etdhDhYUqX0iIl7DZwI8KTWLoAA/RnXR6BMR8Q4+EeBl5ZYladlc3DmMJkFqn4iId/CJAF+76zB5+UVM6Kn2iYh4D58I8NPtk9Fqn4iIF/H6AC+vbJ+M7Byq9omIeBWvD/C1u4+Qm6/RJyLifbw+wJNSs2gQ4MforuFOlyIiUqu8OsAr2idZjOgUSlO1T0TEy3h1gK/fc4Sc40VaOlZEvJJXB/jib9onGn0iIt7HawO8vNyyJDWb4R1DadYw0OlyRERqndcG+Ia9R8g+XsjEnm2cLkVEpE54bYAvTsmmgb9Gn4iI9/LKAD89+mR4pxCaq30iIl7KKwN8w96jZB0r1OQdEfFqXhngS1KzCPQ3ap+IiFfzugC3tmLtk2EdQ2nRSO0TEfFeXhfgyXuPsv/oKbVPRMTreV2AJ1W2Ty5JUPtERLybVwW4tZak1Gwuig9R+0REvJ5XBfjGfcfUPhERn+FVAX66fTI2QbMvRcT7eU2AW2tZnJLF0PgQWjRW+0REvJ/XBHiK2ici4mO8JsCT0rII8DOM1egTEfERXhHgFaNPshgSH0LLxg2cLkdEpF5UGeDGmJeNMbnGmLQzjj1ojNlvjEmufJtQt2WeX9r+4+w9fIqJPfTipYj4jupcgc8Fxp/l+NPW2t6Vb0m1W9aFWZyahb+fRp+IiG+pMsCttSuBw/VQS4180z6Ja02rJmqfiIjv+D498J8bY1IqWyytznUnY8xUY8xaY8zavLy87/F0Z7fpwHH2HD6pjYtFxOfUNMBfAOKA3kAW8NS57mitnW2tTbTWJoaGhtbw6c7tm/ZJN7VPRMS31CjArbU51toya2058A9gQO2WVe06vmmfBKt9IiI+pkYBbow5s19xJZB2rvvWpU0HjrP70ElN3hERnxRQ1R2MMa8DI4EQY8w+4AFgpDGmN2CBXcAddVjjOS1JOz36RJN3RMT3VBng1trrz3J4Th3UckFOLx07KDaY1k2DnC5HRKTeuXYm5pasfHYePKH2iYj4LNcGeFJqFn4Gxmn0iYj4KFcG+OnRJ4NiWxOi9omI+ChXBvjW7Hwy1T4RER/nygA/3T4Z313tExHxXa4LcGsti1OzGBij9omI+DbXBXh6TgGZeSeYoKVjRcTHuS7AF6dmYQyMU/tERHyc6wI8KTWLAdHBhDVr6HQpIiKOclWAp+fksyO3gIk9NfpERMRVAb44paJ9otEnIiIuC/Ck1Cz6q30iIgK4KMC35+SzPbdAO++IiFRyTYCfHn1yqdonIiKAiwI8KTWL/h2CCWuu9omICLgkwHfk5pOeU8ClmrwjIvINVwR4Umo2AJd2V/9bROQ0VwR4m+YNmZzYnjYt1D4RETmtyi3VPMHk/pFM7h/pdBkiIh7FFVfgIiLyvxTgIiIupQAXEXEpBbiIiEspwEVEXEoBLiLiUgpwERGXUoCLiLiUsdbW35MZkwfsruHDQ4CDtViO2+l8/JfOxbfpfHybN5yPDtba0O8erNcA/z6MMWuttYlO1+EpdD7+S+fi23Q+vs2bz4daKCIiLqUAFxFxKTcF+GynC/AwOh//pXPxbTof3+a158M1PXAREfk2N12Bi4jIGRTgIiIu5fEBboyJNMZ8aozZYozZZIz5ldM1Oc0Y42+M2WCMed/pWpxmjGlpjJlvjNla+T0y2OmanGSM+XXlz0maMeZ1Y4xPbWNljHnZGJNrjEk741iwMWaZMWZ75b+tnKyxNnl8gAOlwN3W2q7AIOBnxpgEh2ty2q+ALU4X4SGeBT6w1nYBeuHD58UYEwH8Eki01nYH/IHrnK2q3s0Fxn/n2L3Ax8W8J28AAAIDSURBVNbajsDHlR97BY8PcGttlrV2feX7+VT8gEY4W5VzjDHtgYnAS07X4jRjTHNgODAHwFpbbK096mxVjgsAGhljAoDGwAGH66lX1tqVwOHvHJ4EvFL5/ivAFfVaVB3y+AA/kzEmGugDrHG2Ekc9A/wWKHe6EA8QC+QB/6xsKb1kjGnidFFOsdbuB54E9gBZwDFr7YfOVuURwq21WVBxQQiEOVxPrXFNgBtjmgILgLustcedrscJxpgfALnW2nVO1+IhAoC+wAvW2j7ACbzoz+MLVdnbnQTEAO2AJsaYHzlbldQlVwS4MSaQivB+zVq70Ol6HDQUuNwYswt4AxhljPm3syU5ah+wz1p7+i+y+VQEuq8aA+y01uZZa0uAhcAQh2vyBDnGmLYAlf/mOlxPrfH4ADfGGCp6nFustTOdrsdJ1tr7rLXtrbXRVLw49Ym11mevsKy12cBeY0znykOjgc0OluS0PcAgY0zjyp+b0fjwi7pneBe4pfL9W4BFDtZSqwKcLqAahgI3AanGmOTKYzOstUkO1iSe4xfAa8aYBkAmcJvD9TjGWrvGGDMfWE/F6K0NePE08rMxxrwOjARCjDH7gAeAx4B5xpgpVPySu9a5CmuXptKLiLiUx7dQRETk7BTgIiIupQAXEXEpBbiIiEspwEVEXEoBLiLiUgpwERGX+n/0Kd3/f6JW4QAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.plot([2,3,5,7,9,11], mean_times[:6])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[0.924, 0.956, 0.967, 0.968, 0.969, 0.962, 0.967]"
      ]
     },
     "execution_count": null,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "mean_scores"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Conclusion: It appears 7,9,11 do well, it's hard to tell if the variety helps without testing a higher number of kernels\n",
    "\n",
    "Let's rerun the last experiment, this time with longer strides"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "times_ks2, scores_ks2 = timing_test(10, np.array((2,)), stride=7, num_kernels=100, seq_length=32000, show_progress=False)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "times_ks3, scores_ks3 = timing_test(10, np.array((3,)), stride=7, num_kernels=100, seq_length=32000, show_progress=False)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "times_ks5, scores_ks5 = timing_test(10, np.array((5,)), stride=7, num_kernels=100, seq_length=32000, show_progress=False)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "times_ks7, scores_ks7 = timing_test(10, np.array((7,)), stride=7, num_kernels=100, seq_length=32000, show_progress=False)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "times_ks9, scores_ks9 = timing_test(10, np.array((9,)), stride=7, num_kernels=100, seq_length=32000, show_progress=False)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "times_ks11, scores_ks11 = timing_test(10, np.array((11,)), stride=7, num_kernels=100, seq_length=32000, show_progress=False)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "times_ksorig, scores_ksorig = timing_test(10, np.array((7,9,11,)), stride=7, num_kernels=100, seq_length=32000, show_progress=False)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "all_scores_s7 = [scores_ks2, scores_ks3, scores_ks5, scores_ks7, scores_ks9, scores_ks11, scores_ksorig,]\n",
    "all_times_s7 = [times_ks2, times_ks3, times_ks5, times_ks7, times_ks9, times_ks11, times_ksorig,]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "mean_times_s7 = list(map(mn, all_times_s7))\n",
    "mean_scores_s7 = list(map(mn, all_scores_s7))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[0.905, 0.937, 0.949, 0.95, 0.958, 0.951, 0.957]"
      ]
     },
     "execution_count": null,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "mean_scores_s7"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[<matplotlib.lines.Line2D at 0x7fad80fe3c90>]"
      ]
     },
     "execution_count": null,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXoAAAD6CAYAAACvZ4z8AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAgAElEQVR4nO3de3zU9Z3v8dcnCUmAhGvCLQkQJCigIBrwggqt66P0plW8trVWPcezu7Wnp2etq90et3Ufrt2uPbvdR9vzqF3p2u1FES+11q21aOJdCSJ3EpBbLkiGSwgh5DKZz/kjExxiNGOY5JfMvJ+Phw9+l+/MfGYkb775zvf3/Zm7IyIiySst6AJERKR/KehFRJKcgl5EJMkp6EVEkpyCXkQkySnoRUSSXFxBb2bLzKzSzHaY2V09nJ9mZqvNbIOZlZlZYcy5qWb2JzPbamZbzGx64soXEZHeWG/z6M0sHagCLgNqgDXADe6+JabNY8Az7v6wmX0SuNndb4yeKwPuc/fnzSwHiLh784e9Xl5enk+fPv3U3pWISIpZu3btAXfP7+lcRhyPXwTscPedAGb2CHAFsCWmzRzgm9HtF4Gnom3nABnu/jyAuzf19mLTp0+noqIijrJERKSLme35sHPxDN0UANUx+zXRY7HWA8uj21cCuWY2HpgFNJjZE2a2zsz+OfobgoiIDJB4gt56ONZ9vOcOYImZrQOWALVAmM7fGC6Onl8IzAC++oEXMLvNzCrMrCIUCsVfvYiI9CqeoK8BimL2C4G62AbuXufuV7n7AuDvoseORB+7zt13unuYziGdc7q/gLs/6O6l7l6an9/jEJOIiPRRPEG/Bigxs2IzywSuB56ObWBmeWbW9Vx3AytiHjvWzLrS+5OcPLYvIiL9rNegj/bEbweeA7YCK919s5nda2aXR5stBSrNrAqYCNwXfWwHncM2q81sI53DQD9P+LsQEZEP1ev0yoFWWlrqmnUjIvLxmNlady/t6ZyujBURSXIKehEJxJa6Rn7x6i52HzgWdClJL54LpkREEiIScV7YVs9Dr+zi9Z0HAfje77dwXvE4ri0t4jNnTWZ4pi61STSN0YtIvzvWGubxt2v4xau72XXgGFNGZ3PThdO5dPYEntu8n5UV1ew52ExuVgafP3sK15UWMa9wNGY9XcYjPfmoMXoFvYj0m7qG4zz8+m5+++ZeGlvCnF00hlsvKmbZmZMYlv7+yLG78+auQ6xcU82zm/bR0h7hjEm5XFtaxBcWFDBuZGZwb2KIUNCLyIBat/cwD72yi//a9B7uzqfPnMwtFxVz7rSxvT62saWd36+vY+WaatbXHCEzPY3L5kzk2oVFXDQzj/Q09fJ7oqAXkX4X7ojw3Ob9PPTKTt7e20BudgY3LJrKVy6YRuHYEX16zm3vNfLommqeXFdLQ3M7U0Znc3VpEdecW0jRuL49Z7JS0ItIv2lsaefRt6r5j9d2U9twnGnjR3DzhdO5urSInKzEzPdoDXfw/Jb9rKyo4eXtIdxh8czxXFtaxKfmTiJ7mL7AVdCLSMLtOXiMX7y6m8cqqjnW1sF5xeO49aJiLp09sV+HV2objrOqoobH1lZTc/g4o4cP4wtnT+HahUXMnTK63153sFPQi0hCuDtv7TrEQ6/s4vmt+8lIMz4/bwq3XFTMmQUDG7KRiPP6zoM8uqaaP25+j7ZwhLlTRnHdwiKumF/A6BHDBrSeoCnoReSUtIUjPLOhjhWv7mJTbSNjRwzjS+dN48YLpjFxVHbQ5dHQ3Mbv3qnj0TXVbNnXSFZGGsvOnMR1pUWcP2M8aSnwBa6CXkT65NCxNn7z5h5++foe6o+2MnNCDrcsLubKBQWD9sKmTbVHeHRNNU+9U8vRljBF44Zz7blFLD+3kCljhgddXr9R0IvIx7Kj/igPvbKbJ96uoTUc4ZJZ+dyyeDpLZuUPmYuYWto7eG7zezy6pprX3j1ImsHFJflct7CIv5g9kcyM5FoBRkEvIr1yd17efoCHXtlFeVWIrIw0rjqngJsXFzNrYm7Q5Z2SvQebeWxtNavW1rDvSAvjRmbyhbMLuG5hEadPGtrvrYuCXkQ+VEt7B0+tq2XFq7uo2t9Efm4WXzl/Gl88byrjc7KCLi+hOiLOy9tDrKyo5vkt+2nvcOYXjeG60iI+P38yudlD9wtcBb2IfED90RZ+9foefvXmXg4da2PO5FHcelExn5s/mayMwTn+nkgHm1p5cl0tKyuqqdrfxPBh6XzmrMlcW1rIouJxQ2aIqouCXkRO2Fx3hBWv7Ob36+toj0S49IyJ3HpRMefPGHrhlgjuzvqazi9wf7++jqbWMMV5I7mmtJCrzylkwiCYVRQPBb1IiotEnNXb6nnolZ28sfMQIzLTuebcQr66uJjivJFBlzdoNLeFeXbje6ysqOatXYdITzM+cXo+15YW8YkzJpy0ENtgo6AXSVHHWsOsWlvTeYOPg80nlge+ftFURg8fuuPRA2HXgWOsrKjm8bU11B9tJS8ni+XnFHBNaREzJ+QEXd4HKOhFUkxdw3Eefm03v33r5OWBP33mJDIGca90MAp3RCirDPFoRTUvbKunI+KUThvLtQuL+OxZkxmZoPV8TpWCXiRFxC4PDLDszEncelEx50ztfXlg6V390RaefLuWRyuq2Rk6xsjMdD43r3OdnXOmjgn0Ow4FvUgS+7DlgW+6cDoFSXwlaJDcnbV7DvPommr+sHEfzW0dzJyQw3WlRVx5TgF5AUxLVdCLJKEjx9tZuaZ/lweW3jW1hvnDhs51dt7e20BGmnHp7Alct7CIS0ryB2yoTEEvkkSCWh5Yerd9/1FWVlTzxNu1HDzWxsRRWVx9biHXlhYxbXz/zm5S0IsMcV33VH3olV38OeDlgaV3beEIL2yrZ2VFNWWV9UQczisex3ULi/j0mZP7ZUE4Bb3IENW1PPBDr+xic937ywN/5YJpQ+ZCnlT33pEWHn+7hpUV1ew52ExuVgaXnz2F6xYWcVbB6IR9gaugFxliui8PXDIhh1su6lweWLfNG5oiEeet3YdYuaaaZzfto6U9whmTcrm2tIgrFxQwdmTmKT2/gl5kiOhpeeBbLyrmkpK8lFyeIFk1trTz9Dt1rKyoZkPNETLT07hs7kSuKy3ikln5fXrOjwr6uL6aN7NlwI+AdODf3f373c5PA1YA+cAh4MvuXhM91wFsjDbd6+6X9+ldiCQpd+el7QdY0W154FsWF1MyxJcHlp6Nyh7Gl8+fxpfPn8bWfY2srKjmyXW11DUc73PQf5Ree/Rmlg5UAZcBNcAa4AZ33xLT5jHgGXd/2Mw+Cdzs7jdGzzW5e9zXC6tHL6mipb2DJ9fVsuKVXWyvT+7lgaV3reEO6htbKRo3ok+PP9Ue/SJgh7vvjD7ZI8AVwJaYNnOAb0a3XwSe6lOlIimg/mgL//n6Hn4dszzwD6+ZnzLLA0vPsjLS+xzyvYkn6AuA6pj9GuC8bm3WA8vpHN65Esg1s/HufhDINrMKIAx83931j4AkjUjEaW7voKklTFNrO0dbwhxtCdPUGqapJczR6J9Nre00tYYJHW2jvKqecMRTfnlgGTjxBH1PfwO7j/fcAfzYzL4KvATU0hnsAFPdvc7MZgAvmNlGd3/3pBcwuw24DWDq1Kkfo3yRvumIOMfaukL4/XA+2tL+gWMn9rudb2oJ09QWJp75DCMy08nJyiAnO4MvLprKzYuLma7lgWWAxBP0NUBRzH4hUBfbwN3rgKsAzCwHWO7uR2LO4e47zawMWAC82+3xDwIPQucYfV/eiKSGcEeEY60dHI32kGN7zUe7es4n9aTfD+2jLe8/5lhbR1yvl5OVcSKgc7IyyM3OYNKobHKzM8jJGkZOdga5Mee79nOzh504NjIzXStGSqDiCfo1QImZFdPZU78e+GJsAzPLAw65ewS4m84ZOJjZWKDZ3VujbRYDP0hg/TJEtHdEPthTjg51nDh20vmTg7zr/PH23gParDOgYwN41PBhFIwZfiKsY4O7K7C79nNPBHQGaVpSQJJAr0Hv7mEzux14js7plSvcfbOZ3QtUuPvTwFLgfjNzOoduvhZ9+GzgZ2YWAdLoHKPf8oEXkaRUf7SF7z69mRe21dPSHum1fZrR2RPOej9sx47MpGjciBP7Xee796S7Ajs3O4Phw9IV0CIxdMGUJJy789Q7tXz36S0cb+/gutIiJuRmRXvSJwd5bGAPH5auLyVF+uiUL5gSidf+xha+/cRGVm+r55ypY/jB1fMH5W3XRFKJgl4Swt1ZtbaGf3hmC20dEb7z2dncvLhYy+aKDAIKejlldQ3H+faTGymrDLFw+lh+cPV8ijV1UGTQUNBLn7k7j66p5r4/bCUccf7+83O46YLp+iJUZJBR0Euf1DYc567HN/Dy9gOcP2Mc/7R8Xr/fQUdE+kZBLx+Lu/Obt/byj3/YigP/cMVcvnTeNPXiRQYxBb3ErfpQM3/7+AZee/cgi2eO5/tXzeu3RZhEJHEU9NKrSMT51Zt7+P5/bSPNjH+88ixuWFSkOe8iQ4SCXj7SnoPHuHPVBt7cdYiLS/L4/vJ5FIwZHnRZIvIxKOilR5GI8x+v7eYHz21jWFoaP1g+j2tKC9WLFxmCFPTyATtDTdy5agMVew7zidPz+cerzmLyaPXiRYYqBb2c0BFxVryyiwf+VElWRho/vGY+V51ToF68yBCnoBcAdtQ38a1V61m3t4G/mD2R+648k4mjsoMuS0QSQEGf4sIdEX7+8i7+5c9VjMhM50fXn83l86eoFy+SRBT0Kaxq/1G+9dh61tccYdncSdz7hblMyFUvXiTZKOhTUHtHhJ+Vv8u/rd5BTnYGP/7iAj571mT14kWSlII+xWzd18i3Vq1nU20jnz1rMt+7Yi55OVlBlyUi/UhBnyLaOyL89MV3+fGL2xmVPYyffukcPnPW5KDLEpEBoKBPAZvrjnDHYxvYuq+Ry+dP4buXz2XcyMygyxKRAaKgT2Jt4Qg/fmE7Py17lzEjMvnZjefyqbmTgi5LRAaYgj5Jbaw5wrdWrWfbe0e5akEB93x+DmNGqBcvkooU9EmmNdzBj/68nZ+9tJO8nEweuqmUS2dPDLosEQmQgj6JvFPdwLceW8/2+iauObeQ73xuDqOHDwu6LBEJmII+CbS0d/Avf67i5y/tZOKobH5x80I+cfqEoMsSkUFCQT/Erd1zmG+tWs/O0DGuX1jEtz87m1HZ6sWLyPsU9EPU8bYOfvinSh56dRdTRg/nP29dxMUl+UGXJSKDkIJ+CHpr1yHuXLWe3Qeb+dJ5U7n7M7PJydL/ShHpmdJhCGluC/ODP1by8Ou7KRw7nN/8t/O4cGZe0GWJyCCnoB8i3th5kDtXbWDvoWZuumAady47g5HqxYtIHNLiaWRmy8ys0sx2mNldPZyfZmarzWyDmZWZWWG386PMrNbMfpyowlPFsdYw9/xuE9c/+AZm8Mht5/O9K85UyItI3HpNCzNLB34CXAbUAGvM7Gl33xLT7AHgl+7+sJl9ErgfuDHm/D8A5YkrOzW8uuMAf/v4BmobjnPL4mLu+NQsRmQq4EXk44knNRYBO9x9J4CZPQJcAcQG/Rzgm9HtF4Gnuk6Y2bnAROCPQGkCak56R1vauf+/tvGbN/dSnDeSx/7HBZROHxd0WSIyRMUT9AVAdcx+DXBetzbrgeXAj4ArgVwzGw8cBn5IZ+/+0g97ATO7DbgNYOrUqfHWnpReqgpx1+MbeK+xhdsumcH/vmwW2cPSgy5LRIaweIK+p9sOebf9O4Afm9lXgZeAWiAM/DXwrLtXf9Tdi9z9QeBBgNLS0u7PnRIaW9q575mtPFpRzWn5I1n1VxdyztSxQZclIkkgnqCvAYpi9guButgG7l4HXAVgZjnAcnc/YmYXABeb2V8DOUCmmTW5+we+0E1lL26r5+4nNlJ/tIW/Wnoa37i0RL14EUmYeIJ+DVBiZsV09tSvB74Y28DM8oBD7h4B7gZWALj7l2LafBUoVci/70hzO/c+s4XH365h1sQcfnbjYuYXjQm6LBFJMr0GvbuHzex24DkgHVjh7pvN7F6gwt2fBpYC95uZ0zl087V+rDkp/HnLfr795EYOHmvj9k/M5OuXziQrQ714EUk8cx9cQ+KlpaVeUVERdBn95vCxNr73+8089U4dZ0zK5Z+vns9ZhaODLktEhjgzW+vuPc5s1KTsAfTHTe/xnac20dDcxjcuLeFrn5hJZkZc16yJiPSZgn4AHGxq5e+f3swzG/YxZ/IoHr5lIXOnqBcvIgNDQd/Pnt24j//z1CYaW9r5m8tm8ZdLT2NYunrxIjJwFPT95EBTK/f8bhPPbnyPswpG8+trzuOMSaOCLktEUpCCvh/8fn0d9/xuE8daO7hz2encdvEMMtSLF5GAKOgT7I2dB/n6b9cxv2gMD1w9j5KJuUGXJCIpTkGfYH/avJ/MjDQe+e/nMzxT8+JFJHgaT0iwsqp6zp8xXiEvIoOGgj6Bqg81szN0jKWzdJNuERk8FPQJVFYVAmDJ6Qp6ERk8FPQJVF4ZomjccGbkjQy6FBGRExT0CdIa7uC1dw+wZFY+H7X2vojIQFPQJ8ja3Ydpbutg6awJQZciInISBX2ClFWFyExP44LTxgddiojISRT0CVJWWc/C4rGMzNKlCSIyuCjoE6Cu4ThV+5s0bCMig5KCPgHKNa1SRAYxBX0ClFeGmDI6m5IJOUGXIiLyAQr6U9TeEeHVHQdYcrqmVYrI4KSgP0Vv7znM0dYwSzQ+LyKDlIL+FJVVhchIMxbP1LRKERmcFPSnqLwyxLnTxpKbPSzoUkREeqSgPwX1jS1s2deo2TYiMqgp6E9B17RKzZ8XkcFMQX8KyqpCTMjNYvZk3S5QRAYvBX0fhTsivFwV0mqVIjLoKej7aH1NA40tYZaermEbERncFPR9VFYZIs3gopl5QZciIvKR4gp6M1tmZpVmtsPM7urh/DQzW21mG8yszMwKY46vNbN3zGyzmf1lot9AUMqrQpwzdSyjR2hapYgMbr0GvZmlAz8BPg3MAW4wszndmj0A/NLd5wH3AvdHj+8DLnT3s4HzgLvMbEqiig/KgaZWNtQcYYluAi4iQ0A8PfpFwA533+nubcAjwBXd2swBVke3X+w67+5t7t4aPZ4V5+sNei9vj06r1Pi8iAwB8QRvAVAds18TPRZrPbA8un0lkGtm4wHMrMjMNkSf45/cve7USg5eWWWIvJxM5k4ZFXQpIiK9iifoe5o76N327wCWmNk6YAlQC4QB3L06OqQzE7jJzCZ+4AXMbjOzCjOrCIVCH+sNDLSOiPNSVYhLSvJJS9O0ShEZ/OIJ+hqgKGa/EDipV+7ude5+lbsvAP4ueuxI9zbAZuDi7i/g7g+6e6m7l+bnD+5x7421Rzjc3K5lD0RkyIgn6NcAJWZWbGaZwPXA07ENzCzPzLqe625gRfR4oZkNj26PBRYDlYkqPgjllSHM4OISBb2IDA29Br27h4HbgeeArcBKd99sZvea2eXRZkuBSjOrAiYC90WPzwbeNLP1QDnwgLtvTPB7GFBlVfXMKxzDuJGZQZciIhKXjHgaufuzwLPdjt0Ts70KWNXD454H5p1ijYPG4WNtrK9u4OufLAm6FBGRuCXFdMeB8vKOA0Qclmp8XkSGEAX9x1BWWc/YEcOYVzgm6FJEROKmoI9TJOK8VHWAi0vySde0ShEZQhT0cdqyr5EDTa1a9kBEhhwFfZy67iZ1iYJeRIYYBX2cyirrObNgFPm5WUGXIiLysSjo43DkeDtv723QvWFFZEhS0Mfh1R0H6Ii4lj0QkSFJQR+H8soQudkZLCjStEoRGXoU9L1wd8qrQlxckkdGuj4uERl6lFy9qNx/lPcaWzQ+LyJDloK+F2WVmlYpIkObgr4X5ZUhzpiUy6TR2UGXIiLSJwr6j9DUGqZizyHdG1ZEhjQF/Ud4dccB2jtcyx6IyJCmoP8I5VUhcrIyOHfa2KBLERHpMwX9h3B3yitDXHjaeDIz9DGJyNClBPsQ74aaqG04rvF5ERnyFPQfomtapZY9EJGhTkH/IcqrQpRMyKFgzPCgSxEROSUK+h40t4V5c+chzbYRkaSgoO/BGzsP0tYR0fi8iCQFBX0PyipDDB+WzsJiTasUkaFPQd+D8qrOaZVZGelBlyIicsoU9N3sOnCMPQebWarZNiKSJBT03ZRV1gOwRMsSi0iSUNB3U14VYkbeSKaOHxF0KSIiCaGgj9HS3sHr7x7U2vMiklTiCnozW2ZmlWa2w8zu6uH8NDNbbWYbzKzMzAqjx882s9fNbHP03HWJfgOJ9OauQ7SGIxqfF5Gk0mvQm1k68BPg08Ac4AYzm9Ot2QPAL919HnAvcH/0eDPwFXefCywD/tXMBu0dtssq68nKSOP8GeODLkVEJGHi6dEvAna4+053bwMeAa7o1mYOsDq6/WLXeXevcvft0e06oB4YtN3l8qoQ588YT/YwTasUkeQRT9AXANUx+zXRY7HWA8uj21cCuWZ2UrfYzBYBmcC7fSu1f1UfamZn6JiWPRCRpBNP0FsPx7zb/h3AEjNbBywBaoHwiScwmwz8J3Czu0c+8AJmt5lZhZlVhEKhuItPpLKqztfV+LyIJJt4gr4GKIrZLwTqYhu4e527X+XuC4C/ix47AmBmo4A/AN9x9zd6egF3f9DdS929ND8/mKAtr6ynaNxwivNGBvL6IiL9JZ6gXwOUmFmxmWUC1wNPxzYwszwz63quu4EV0eOZwJN0flH7WOLKTqzWcAevvXuQpbMmYNbTLzAiIkNXr0Hv7mHgduA5YCuw0t03m9m9ZnZ5tNlSoNLMqoCJwH3R49cClwBfNbN3ov+dneg3caoqdh+mua1DwzYikpQy4mnk7s8Cz3Y7dk/M9ipgVQ+P+xXwq1Ossd+VV4XITE/jgtM0rVJEko+ujKVz/vyi4nGMyIzr3z0RkSEl5YO+ruE4VfubNK1SRJJWygd9uaZVikiSS/mgL6usZ8robGZOyAm6FBGRfpHSQd/eEeHVHQdZcrqmVYpI8krpoF+75zBNrWGNz4tIUkvpoC+vCpGRZiyeqWmVIpK8UjroyypDnDttLLnZw4IuRUSk36Rs0O9vbGHrvkaWnq57w4pIckvZoO+aVqnxeRFJdikd9BNHZTF7cm7QpYiI9KuUDPpwR4SXq0IsmZWvaZUikvRSMujX1zTQ2BJmySyNz4tI8kvJoC+rDJGeZlxUkhd0KSIi/S5lg35B0RhGD9e0ShFJfikX9AeaWtlYe0SLmIlIyki5oH/pxLRKjc+LSGpIuaAvrwqRl5PJ3Cmjgi5FRGRApFTQd0Scl6pCXFKST1qaplWKSGpIqaDfWHuEw83tLNH4vIikkJQK+rLKeszg4hIFvYikjpQK+vKqEPMLxzBuZGbQpYiIDJiUCfrDx9p4p7pB0ypFJOWkTNC/vOMA7lqtUkRST8oEfVllPWNHDGNe4ZigSxERGVApEfSR6LTKi0vySde0ShFJMSkR9Fv2NXKgqU3j8yKSklIi6Msq6wFNqxSR1JQSQV9eFeKsgtHk52YFXYqIyICLK+jNbJmZVZrZDjO7q4fz08xstZltMLMyMyuMOfdHM2sws2cSWXi8jhxv5+29DZptIyIpq9egN7N04CfAp4E5wA1mNqdbsweAX7r7POBe4P6Yc/8M3JiYcj++V3ccoCPiGp8XkZQVT49+EbDD3Xe6exvwCHBFtzZzgNXR7Rdjz7v7auBoAmrtk7LKekZlZ3B2kaZVikhqiifoC4DqmP2a6LFY64Hl0e0rgVwzGx9vEWZ2m5lVmFlFKBSK92G9cnfKo9MqM9JT4usIEZEPiCf9epp47t327wCWmNk6YAlQC4TjLcLdH3T3Uncvzc9P3BDLtveOsr+xVatVikhKy4ijTQ1QFLNfCNTFNnD3OuAqADPLAZa7+5FEFdlX5SfuJqWgF5HUFU+Pfg1QYmbFZpYJXA88HdvAzPLMrOu57gZWJLbMvimrrGf25FFMHJUddCkiIoHpNejdPQzcDjwHbAVWuvtmM7vXzC6PNlsKVJpZFTARuK/r8Wb2MvAYcKmZ1ZjZpxL8HnrU1BqmYvdh9eZFJOXFM3SDuz8LPNvt2D0x26uAVR/y2ItPpcC+enXHAcKaVikikrxXxpZVhsjJyuDcaWODLkVEJFBJGfTunatVLp45nmGaVikiKS4pU3BHfRO1DcdZMmtC0KWIiAQuKYP+xLRKjc+LiCRn0JdVhiiZkEPBmOFBlyIiErikC/rmtjBv7Tqk2TYiIlFJF/Svv3uQto6IxudFRKKSLujLq0KMyExnYbGmVYqIQJIFvbtTVhniwtPGk5WRHnQ5IiKDQlIF/e6Dzew91KxlD0REYiRV0HfdBFzj8yIi70uyoA8xI28kU8ePCLoUEZFBI2mCvqW9gzd2HtRFUiIi3SRN0Dceb+dTcydx2ZyJQZciIjKoxLVM8VAwYVQ2/3bDgqDLEBEZdJKmRy8iIj1T0IuIJDkFvYhIklPQi4gkOQW9iEiSU9CLiCQ5Bb2ISJJT0IuIJDlz96BrOImZhYA9p/AUecCBBJUz1OmzOJk+j5Pp83hfMnwW09y9xzVgBl3Qnyozq3D30qDrGAz0WZxMn8fJ9Hm8L9k/Cw3diIgkOQW9iEiSS8agfzDoAgYRfRYn0+dxMn0e70vqzyLpxuhFRORkydijFxGRGEkR9GZWZGYvmtlWM9tsZt8IuqbBwMzSzWydmT0TdC1BM7MxZrbKzLZF/55cEHRNQTGzb0Z/TjaZ2W/NLDvomgaSma0ws3oz2xRzbJyZPW9m26N/jg2yxkRLiqAHwsDfuPts4Hzga2Y2J+CaBoNvAFuDLmKQ+BHwR3c/A5hPin4uZlYA/E+g1N3PBNKB64OtasD9B7Cs27G7gNXuXgKsju4njaQIenff5+5vR7eP0vlDXBBsVcEys0Lgs8C/B11L0MxsFHAJ8BCAu7e5e0OwVQUqAxhuZhnACKAu4HoGlLu/BBzqdvgK4OHo9sPAFwa0qH6WFEEfy8ymAwuAN4OtJHD/CtwJRIIuZBCYAYSAX0SHsv7dzEYGXVQQ3L0WeADYC+wDjqCffNIAAAF0SURBVLj7n4KtalCY6O77oLPjCEwIuJ6ESqqgN7Mc4HHgf7l7Y9D1BMXMPgfUu/vaoGsZJDKAc4D/5+4LgGMk2a/m8YqOPV8BFANTgJFm9uVgq5L+ljRBb2bD6Az5X7v7E0HXE7DFwOVmtht4BPikmf0q2JICVQPUuHvXb3mr6Az+VPQXwC53D7l7O/AEcGHANQ0G+81sMkD0z/qA60mopAh6MzM6x1+3uvv/DbqeoLn73e5e6O7T6fyi7QV3T9lem7u/B1Sb2enRQ5cCWwIsKUh7gfPNbET05+ZSUvSL6W6eBm6Kbt8E/C7AWhIuI+gCEmQxcCOw0czeiR77trs/G2BNMrh8Hfi1mWUCO4GbA64nEO7+ppmtAt6mc7baOpL8qtDuzOy3wFIgz8xqgL8Hvg+sNLNb6fzH8JrgKkw8XRkrIpLkkmLoRkREPpyCXkQkySnoRUSSnIJeRCTJKehFRJKcgl5EJMkp6EVEkpyCXkQkyf1/C7mCZQLBByIAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.plot([2,3,5,7,9,11], mean_scores_s7[:6])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[<matplotlib.lines.Line2D at 0x7fad8a14bed0>]"
      ]
     },
     "execution_count": null,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXQAAAD6CAYAAACxrrxPAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAgAElEQVR4nO3dd3hUZd7G8e9DQkgIHUINofcOAQRc1sLaULCAomtB3UV9d3VdO3aw97K4KrbFdS0YUIooFuwFNygkIQQINYFAgEAKIf33/kF2N0ZKgEnOzOT+XFeuzJxzmNwMyZ2HM888x5kZIiIS+Op4HUBERHxDhS4iEiRU6CIiQUKFLiISJFToIiJBQoUuIhIkqlTozrkmzrk451yKc26Vc25Epf3OOfescy7VOZfgnBtcPXFFRORgQqt43DPAR2Y2wTkXBtSvtP90oFv5x3Dg+fLPB9WiRQvr2LHjkaUVEanlli1bttPMog6077CF7pxrBIwGJgOYWRFQVOmw8cDrtv9dSj+Uj+jbmFnGwR63Y8eOxMfHV/GvICIiAM65TQfbV5VTLp2BHcBrzrmfnXMvO+ciKx3TDkircD+9fJuIiNSQqhR6KDAYeN7MBgF7gdsqHeMO8Od+taaAc26Kcy7eORe/Y8eOIw4rIiIHV5VCTwfSzWxp+f049hd85WPaV7gfDWyt/EBmNtPMYs0sNirqgKeARETkKB220M1sG5DmnOtRvulkILnSYfOBS8tnuxwHZB/q/LmIiPheVWe5XAv8q3yGy3rgcufc1QBm9gKwCDgDSAXygcurIauIiBxClQrdzJYDsZU2v1BhvwF/8mEuERE5QnqnqIhIkFChi4jUkJLSMp77PJXE9OxqefyqnkMXEZFjsH5HHje+u4KfN+8ht6CEftGNff41VOgiItWorMx4/fuNPPxRCvVCQ3j2wkGMG9C2Wr6WCl1EpJps2bOPm99dwXfrdnFCjygeOa8/rRqFV9vXU6GLiPiYmRG3LJ3pC5IpM+Ohc/sxaWh7nDvQm+p9R4UuIuJDO3ILmTo3kU9XbWdYp2Y8MXEA7ZtVXqC2eqjQRUR85MPEDO54P4m8whLuHNuLK0Z1ok6d6h2VV6RCFxE5Rtn5xdw9P4l5y7fSr11jnjx/AN1aNazxHCp0EZFj8OWaHdwSt4JdeUVcP6YbfzqxK3VDvHmLjwpdROQo7C0s4YFFq3hz6Wa6tWzAy5cOrZa55UdChS4icoT+vTGLG2evIG13PlNGd+aG33UnvG6I17FU6CIiVVVQXMqTn6zhpa/XE900gnemjGBYp2Zex/ovFbqISBUkbcnmhtnLWbM9j4uGx3DHGb2IrOdfFepfaURE/ExxaRl//3wdf1uyluYNwvjH5UM5oUdLr2MdkApdROQgUjNzuWH2ChLSsxk/sC3TxvWhSf0wr2MdlApdRKSSsjLj1W838Oji1USGhfD33w/mjH5tvI51WCp0EZEK0rLyufHdFfy4IYsxvVry4Ln9aNmw+hbU8iUVuogI+xfUevvfady/MBnnHI9O6M/EIdHVvqCWL6nQRaTW255TwG1zEvh89Q5GdG7OYxP7E920ZhbU8iUVuojUavNXbOWu95MoKC7l3rN6c+mIjjW6oJYvqdBFpFbavbeIO+cl8UFCBgPbN+GJ8wfQJaqB17GOSZUK3Tm3EcgFSoESM4uttP8EYB6woXzTXDOb7ruYIiK+syRlO7fOSWRPfhE3n9qDq0Z3JtSjBbV86UhG6Cea2c5D7P/azM481kAiItUlt6CY+xeu4p34NHq2bsisy4fRu20jr2P5jE65iEit8P26Xdz07goysvdxzQlduH5MN+qFer+gli9VtdAN+Ng5Z8CLZjbzAMeMcM6tALYCN5nZSl+FFBE5WgXFpTz60Wpe/XYDHZvX592rRzCkg/8sqOVLVS30UWa21TnXEvjEOZdiZl9V2P8T0MHM8pxzZwDvA90qP4hzbgowBSAmJuYYo4uIHNqKtD3cMHs563bs5dIRHbjt9J7UDwveExNVehXAzLaWf84E3gOGVdqfY2Z55bcXAXWdcy0O8DgzzSzWzGKjoqKOObyIyIEUlZTxxMerOff578gvKuWNK4czfXzfoC5zqMII3TkXCdQxs9zy26cA0ysd0xrYbmbmnBvG/l8Uu6ojsIjIoazelssNs5ezcmsO5w5uxz1n9aFxRF2vY9WIqvy6agW8V/7211DgTTP7yDl3NYCZvQBMAK5xzpUA+4BJZmbVlFlE5FdKy4yXvl7Pkx+voWF4KC9eMoRT+7T2OlaNOmyhm9l6YMABtr9Q4fYMYIZvo4mIVM3GnXu56d0VxG/azal9WvHAOf1o0aCe17FqXHCfUBKRoGZmvLF0Mw9+sIrQEMdTFwzg7IHtAmpBLV9SoYtIQMrI3sctcQl8vXYnv+nWgkcn9KdN4wivY3lKhS4iAcXMeH/5Fu6et5KSUuO+s/ty8fCYWjsqr0iFLiIBY1deIXe8l8RHK7cxpENTnpg4gI4tIr2O5TdU6CISED5euY3b30skZ18Jt53ekz/+pjMhAbrMbXVRoYuIX8spKGba/GTm/JRO7zaNeOMPA+jZOngW1PIlFbqI+K1vU3dy87sr2J5byLUndeXak7oRFhr4y9xWFxW6iPid/KISHv4whde/30TnqEjmXDOSge2beB3L76nQRcSvLNu0mxtnL2fjrnyuGNWJW07rQXjd4Frmtrqo0EXELxSWlPL0p2t58ct1tGkcwVt/PI4RXZp7HSugqNBFxHPJW3O4YfZyUrblckFse+48sxcNw2vHglq+pEIXEc+UlJbx4lfrefrTNTSOCOOVy2I5uVcrr2MFLBW6iHhi/Y48bnx3BT9v3sPY/m24f3xfmkaGeR0roKnQRaRGlZUZr3+/kYc/SqFeaAjPXjiIcQPaeh0rKKjQRaTGbNmzj5vfXcF363ZxQo8oHjmvP60ahXsdK2io0EWk2pkZccvSmb4gmTIzHjq3H5OGtteCWj6mQheRarUjt5CpcxP5dNV2hnVqxhMTB9C+WX2vYwUlFbqIVIv/LHM7bUEy+UWl3Dm2F1eM6kQdLahVbVToIuJzGdn7uOO9JJakZDI4pgmPTuhP15YNvY4V9FToIuIzZsbs+DTuX7iK4rIy7jqzN5NHdtQytzVEhS4iPpGWlc/UuYl8k7qT4Z2a8eiE/nRorotP1CQVuogck7Iy442lm3j4wxQccN/Zffn9sBidK/dAlQrdObcRyAVKgRIzi6203wHPAGcA+cBkM/vJt1FFxN9s2LmXW+MS+HFjFr/p1oKHzu1HdFPNYPHKkYzQTzSznQfZdzrQrfxjOPB8+WcRCUKlZcar32zg8Y9XExZah0cn9GfikGjNK/eYr065jAdeNzMDfnDONXHOtTGzDB89voj4ibXbc7k5LoHlaXsY06slD5zTT+/29BNVLXQDPnbOGfCimc2stL8dkFbhfnr5NhW6SJAoLi1j5lfreebTtUTWC+GZSQMZN6CtRuV+pKqFPsrMtjrnWgKfOOdSzOyrCvsP9C9qlTc456YAUwBiYmKOOKyIeGPl1mxuiUtg5dYcxvZrw7TxfWjRoJ7XsaSSKhW6mW0t/5zpnHsPGAZULPR0oH2F+9HA1gM8zkxgJkBsbOyvCl9E/EthSSnPLUnl71+so0n9MJ7//WBO79fG61hyEIctdOdcJFDHzHLLb58CTK902Hzgz865t9n/Ymi2zp+LBLblaXu4JW4Fa7bnce6gdtx1Zm+tV+7nqjJCbwW8V36eLBR408w+cs5dDWBmLwCL2D9lMZX90xYvr564IlLdCopLeeqTNbz09XpaNgzn1cmxnNRTVxEKBIctdDNbDww4wPYXKtw24E++jSYiNS1+Yxa3xCWwfudeLhzWnqln9KKRru0ZMPROUREhv6iERz9azazvN9KuSQRvXDmc47u18DqWHCEVukgt913qTm6dm0Ba1j4mj+zIzaf2ILKeqiEQ6V9NpJbKKSjmoUUpvPXjZjo2r8/sq0YwrFMzr2PJMVChi9RCn6/O5Pa5iWzPKWDK6M78dUx3IsJCvI4lx0iFLlKL7MkvYvrCZOb+tIVuLRvw92tGMiimqdexxEdU6CK1xOKV27jz/SSy9hZx7Uld+fNJXakXqlF5MFGhiwS5XXmF3DN/JQsTMujdphGvTR5K33aNvY4l1UCFLhKkzIwFCRncO38luQXF3HRKd676bRfqhtTxOppUExW6SBDKzCngjveT+CR5OwOiG/PYxOPo3koXaQ52KnSRIGJmxC1L576FyRSWlHH7GT25YlQnQjUqrxVU6CJBYsuefdw+N5Ev1+xgaMemPHJefzpHNfA6ltQgFbpIgCsrM97692YeWpRCmRnTxvXhkuM66CLNtZAKXSSAbd6Vz61zEvh+/S5GdW3Ow+f2p30zXaS5tlKhiwSg0jJj1ncbeWzxakLrOB4+tx8XDG2vy8HVcip0kQCTmpnHrXMSWLZpNyf2iOKBc/rRtkmE17HED6jQRQJESWkZL329gac+XUNE3RCePH8A5wxqp1G5/JcKXSQApGzL4Za4BBLSszm1TyvuO7svLRuGex1L/IwKXcSPFZWU8fwX65jx+VoahdfluYsGc0a/1hqVywGp0EX8VGJ6NjfHrSBlWy7jB7blnrP60EwXaZZDUKGL+JmC4lKe/WwtL361nuaRYbx0aSy/662LNMvhqdBF/MiyTbu5JW4F63bsZeKQaO4c25vG9XWRZqkaFbqIH9hXVMrjH6/m1W830KZROLOuGMZvu0d5HUsCTJUL3TkXAsQDW8zszEr7JgOPAVvKN80ws5d9FVIkmP2wfhe3zklg0658Lj4uhltP60nDcI3K5cgdyQj9L8AqoNFB9r9jZn8+9kgitUNeYQmPfJjCP3/YREyz+rz1x+MY0aW517EkgFWp0J1z0cBY4AHghmpNJFILfLVmB1PnJrI1ex9XHt+JG0/pTv0wnQGVY1PV76CngVuAQ62Qf55zbjSwBvirmaUdaziRYJO9r5gHPkhmdnw6XaIiibt6JEM66CLN4huHLXTn3JlAppktc86dcJDDFgBvmVmhc+5qYBZw0gEeawowBSAmJuaoQ4sEok+Tt3PH+4nszCvimhO68JeTuxFeVxdpFt9xZnboA5x7CLgEKAHC2X8Ofa6ZXXyQ40OALDM75FVoY2NjLT4+/qhCiwSS3XuLuHfBSuYt30rP1g15dEJ/+kc38TqWBCjn3DIziz3QvsOO0M1sKjC1/IFOAG6qXObOuTZmllF+dxz7XzwVqfUWJWZw97wk9uQXc/2YbvzfCV0JC9Xl4KR6HPWrMM656UC8mc0HrnPOjWP/KD4LmOybeCKBaUduIXfPS+LDpG30a9eYf145nF5tDjZBTMQ3DnvKpbrolIsEo9Iy4+1/b+axxavJLyrl+jHdmPKbzrpIs/jMMZ1yEZGq+ffGLO6Zt5LkjByGdWrGg+f0pWvLQ00ME/EtFbrIMcrI3sdDi1KYv2IrbRuHM+OiQYzt10ZL3EqNU6GLHKWC4lJe+WYDM5akUmrGdSd345rfdiEiTFMRxRsqdJEjZGZ8uiqT+xYmszkrn9P6tOaOsb1o36y+19GkllOhixyB1Mxcpi1I5uu1O+nWsgFvXDmc47u18DqWCKBCF6mSnIJinv10Lf/4biMRYSHcc1ZvLj6uA3U1e0X8iApd5BDKyoy4Zek8ujiFXXuLmDS0PTed0oPmDep5HU3kV1ToIgfx0+bdTJu/khXp2QyOacJrk4fRL/qQK1qIeEqFLlJJZk4Bj3y0mjk/pdOyYT2evmAg4we21TRE8XsqdJFyRSVlvPbtBp79bC3FpcY1J3ThTyd2pUE9/ZhIYNB3qgjw+epM7luQzPqdezm5Z0vuPLM3nVpEeh1L5Iio0KVW27BzL/ctTGZJSiadW0Ty2uVDObFHS69jiRwVFbrUSnmFJcxYksor36ynXmgIt5/Rk8kjO2lpWwloKnSpVcyM95dv4aFFKWTmFjJhSDS3nNaDlg3DvY4mcsxU6FJrJKZnc8/8JH7avIcB0Y158ZIhDIrR9TwleKjQJejtzCvk8cWreSc+jeaRYTw6oT8TBkdTp46mIUpwUaFL0CouLeOf32/iqU/XsK+olD8c34lrT+5Go/C6XkcTqRYqdAlK36zdybQFK1mbmcfo7lHcfWZvurZs4HUskWqlQpegkpaVz/0fJLN45XZimtXn5UtjOblXS73LU2oFFboEhfyiEp7/Yh0vfrWeEOe4+dQeXHl8J8Lr6mITUnuo0CWgmRkLEzJ4cNEqMrILGD+wLbed3pM2jSO8jiZS41ToErCSt+Zw74KV/Lghiz5tG/HshYMY2rGZ17FEPFPlQnfOhQDxwBYzO7PSvnrA68AQYBdwgZlt9GFOkf/avbeIJz5ZzZtLN9M4oi4PntOPC4a2J0TTEKWWO5IR+l+AVUCjA+y7EthtZl2dc5OAR4ALfJBP5L9KSst468fNPP7xGvIKS7h0REf+OqY7jetrGqIIVLHQnXPRwFjgAeCGAxwyHri3/HYcMMM558zMfBFS5If1u7h3/kpStuUyonNz7h3Xhx6tG3odS8SvVHWE/jRwC3Cwn6B2QBqAmZU457KB5sDOY04otdqWPft4cNEqPkjIoF2TCJ7//WBO69ta0xBFDuCwhe6cOxPINLNlzrkTDnbYAbb9anTunJsCTAGIiYk5gphS2xQUlzLzq/X8/YtUzOD6Md24anQXIsI0DVHkYKoyQh8FjHPOnQGEA42cc2+Y2cUVjkkH2gPpzrlQoDGQVfmBzGwmMBMgNjZWp2PkV8yMxSu3c/8HyaTv3sfYfm2YekZPopvW9zqaiN87bKGb2VRgKkD5CP2mSmUOMB+4DPgemAAs0flzOVJrtucybcFKvk3dRY9WDXnzj8MZ2aWF17FEAsZRz0N3zk0H4s1sPvAK8E/nXCr7R+aTfJRPaoHsfcU8/ekaXv9+Ew3qhTJ9fB8uGhZDaIguNiFyJI6o0M3sC+CL8tt3V9heAEz0ZTAJfqVlxuz4NB5bvJrd+UVcNCyGG0/pQbPIMK+jiQQkvVNUPBG/MYt7F6wkaUsOQzs25Z6zhtG3XWOvY4kENBW61KjtOQU8/GEK7/28hdaNwnlm0kDGDWiraYgiPqBClxpRWFLKK99sYMaSVEpKjT+f2JX/O7EL9cP0LSjiK/ppkmplZixJyWT6wmQ27crnd71bcefYXnRoHul1NJGgo0KXarNuRx73LUzmi9U76BIVyetXDGN09yivY4kELRW6+FxuQTF/W5LKq99sIKJuCHeO7cVlIztSV9MQRaqVCl18pqzMmPvzFh7+MIVdewuZOCSam0/tSVTDel5HE6kVVOjiEyvS9nDP/JUsT9vDoJgmvHJZLAPaN/E6lkitokKXY5JXWMLji1cz6/uNtGhQjycmDuCcQe2oo4tNiNQ4FboctU+Tt3PXvCS25RRw6XEduOnUHjQM18UmRLyiQpcjlplTwLQFyXyQmEH3Vg2YcdFIhnRo6nUskVpPhS5VVlZmvBOfxoOLVlFYUsZNp3RnyuguhIVq9oqIP1ChS5WkZuZx+9xEftyYxXGdm/HgOf3oHNXA61giUoEKXQ6pqKSM579Yx3OfpxIRFsKj5/VnYmy01l4R8UMqdDmo+I1ZTJ2byNrMPM4a0Ja7z+ytOeUifkyFLr+SU1DMIx+m8K+lm2nXJILXJg/lxJ4tvY4lIoehQpdf+ChpG/fMT2JHbiFXHt+JG37Xnch6+jYRCQT6SRUAtmUXcPe8JD5O3k6vNo2YeYne6SkSaFTotVxZmfGvpZt45KPVFJeWcdvpPbny+E5aSEskAKnQa7E123O5bU4CP23ew/FdW/DAOX21TrlIAFOh10IFxaU893kqL3y5jgb1Qnny/P3rr2gqokhgU6HXMj+s38XtcxNZv3Mv5w5qxx1je9G8gaYiigQDFXotkZ1fzEMfruLtf6fRvlmErh4kEoQOW+jOuXDgK6Be+fFxZnZPpWMmA48BW8o3zTCzl30bVY6GmfFBYgb3zk9md34RV43uzPVjuhMRFuJ1NBHxsaqM0AuBk8wszzlXF/jGOfehmf1Q6bh3zOzPvo8oR2vLnn3c9X4SS1Iy6deuMf+4fCh92zX2OpaIVJPDFrqZGZBXfrdu+YdVZyg5NqVlxqzvNvL4x6sxgzvH9mLyyI6EaiqiSFCr0jl051wIsAzoCjxnZksPcNh5zrnRwBrgr2aWdoDHmQJMAYiJiTnq0HJwyVtzmDo3gRXp2fy2exT3n92X9s3qex1LRGqA2z8Ar+LBzjUB3gOuNbOkCtubA3lmVuicuxo438xOOtRjxcbGWnx8/FHGlsoKikt55rO1zPxqPU3r1+Xus/pwVv82moooEmScc8vMLPZA+45olouZ7XHOfQGcBiRV2L6rwmEvAY8cRU45St+m7uT29xLZtCuf82Ojuf2MXjSpH+Z1LBGpYVWZ5RIFFJeXeQQwhkqF7ZxrY2YZ5XfHAat8nlR+ZffeIu7/YBVzfkqnY/P6vPnH4Yzs0sLrWCLikaqM0NsAs8rPo9cBZpvZQufcdCDezOYD1znnxgElQBYwuboCy/6piPOWb2X6wmRy9hXzpxO7cO1J3Qivq6mIIrXZEZ1D9yWdQz86aVn53PF+El+t2cHA9k14+Lx+9GzdyOtYIlJDfHYOXbxTUlrGq99u4MlP1hDiHNPG9eHi4zoQUkcveorIfir0AJC0JZtb5ySwcmsOY3q1Yvr4PrRtEuF1LBHxMyp0P5ZfVMJTn6zhlW820LxBPf7++8Gc3re1piKKyAGp0P3Ul2t2cMd7iaTv3sdFw2O49bSeNI6o63UsEfFjKnQ/szOvkPsWJjNv+Va6REUy+6oRDOvUzOtYIhIAVOh+wsyIW5bOA4tWsbewhL+c3I3/O7EL9UI1FVFEqkaF7gc27tzL7e8l8t26XcR2aMpD5/ajW6uGXscSkQCjQvdQcWkZL329nmc+XUtYSB0eOKcvFw6NoY6mIorIUVChe2R52h5um5NAyrZcTuvTmmnj+9CqUbjXsUQkgKnQa1heYQmPL17NrO830qphOC9eMoRT+7T2OpaIBAEVeg36bNV27no/iYycAi45rgM3n9qDhuGaiigivqFCrwGZuQVMW5DMBwkZdG/VgLiLRjKkQ1OvY4lIkFGhV6OyMuOd+DQeWrSKgpIybjqlO1NGdyEsVJeCExHfU6FXk3U78pg6N5EfN2QxvFMzHjq3H52jGngdS0SCmArdx4pKynjhy3XMWJJKRFgIj57Xn4mx0Vp/RUSqnQrdh5ZtyuK2OYmszczjzP5tuOesPkQ1rOd1LBGpJVToPpBTUMxjH63mjaWbaNs4glcnx3JSz1ZexxKRWkaFfowWr9zG3fOS2JFbyOUjO3HjKd2JrKenVURqnprnKGXmFHDXvCQWr9xOrzaNmHlJLAPaN/E6lojUYir0o/DZqu3cHJfA3sISbj2tJ3/4TSfqhmgqooh4S4V+BAqKS3n4wxT+8d1GerVpxN8uHETXlpqKKCL+QYVeRamZeVz71s+sysjh8lEdufW0noTX1VrlIuI/Dlvozrlw4CugXvnxcWZ2T6Vj6gGvA0OAXcAFZrbR52k9YGbMjk/j3vnJRISFaAaLiPitqozQC4GTzCzPOVcX+MY596GZ/VDhmCuB3WbW1Tk3CXgEuKAa8tao7H3F3D43kQ8SMxjVtTlPnT+QllriVkT81GEL3cwMyCu/W7f8wyodNh64t/x2HDDDOefK/2xAWrYpi+veWs72nAJuPa0nV43urAtPiIhfq9I5dOdcCLAM6Ao8Z2ZLKx3SDkgDMLMS51w20BzYWelxpgBTAGJiYo4teTUpLTOe+zyVZz5bS7smEcRdM5KBmo4oIgGgSoVuZqXAQOdcE+A951xfM0uqcMiBhq6/Gp2b2UxgJkBsbKzfjd4zsvdx/dvLWbohi/ED23L/2X21XrmIBIwjmuViZnucc18ApwEVCz0daA+kO+dCgcZAlq9C1oTFK7dx65wEikrKeGLiAM4d3E4LaolIQKnKLJcooLi8zCOAMex/0bOi+cBlwPfABGBJoJw/Lygu5YEPVvHPHzbRr11jnr1wEJ1aRHodS0TkiFVlhN4GmFV+Hr0OMNvMFjrnpgPxZjYfeAX4p3Mulf0j80nVltiHVm/L5bq3fmb19lymjO7MTaf00MUnRCRgVWWWSwIw6ADb765wuwCY6Nto1cfM+NfSzdy3MJmG4aHMumIYv+0e5XUsEZFjUuveKbonv4hb4hL4OHk7o7tH8cTEAVqzXESCQq0q9B/W7+Kv7yxnZ14hd47txRWjOmluuYgEjVpR6CWlZTy7JJUZS9bSoXkkc68ZRb/oxl7HEhHxqaAv9PTd+Vz/9nLiN+1mwpBopo3rowtQiEhQCupm+yAhg9vmJmAGz0wayPiB7byOJCJSbYKy0POLSrhvYTJv/ZjGgPZN+NukQcQ0r+91LBGRahV0hZ68NYdr3/qJ9Tv3cs0JXbjhd911NSERqRWCptDNjFnfbeTBRSk0qV+XN64czqiuLbyOJSJSY4Ki0LP2FnHzuyv4LCWTk3q25LEJ/WneQHPLRaR2CfhC/y51J9e/s5w9+cXce1ZvLhvZUYtqiUitFLCFXlxaxlOfrOH5L9fRuUUk/7h8GL3bNvI6loiIZwKy0Dfvyue6t39medoeJg1tz91n9aZ+WED+VUREfCbgWvCL1Zn8+c2fcQ6eu2gwY/u38TqSiIhfCLhC79g8ksEdmvLgOX2Jbqq55SIi/xF4hd4iktevGOZ1DBERv6N33IiIBAkVuohIkFChi4gECRW6iEiQUKGLiAQJFbqISJBQoYuIBAkVuohIkHBm5s0Xdm4HsOko/3gLYKcP4wQ6PR+/pOfjf/Rc/FIwPB8dzCzqQDs8K/Rj4ZyLN7NYr3P4Cz0fv6Tn43/0XPxSsD8fOuUiIhIkVOgiIkEiUAt9ptcB/Iyej1/S8/E/ei5+Kaifj4A8hy4iIr8WqCN0ERGpJKAK3TnX3jn3uXNulXNupXPuL15n8ppzLsQ597NzbqHXWbzmnGvinItzzqWUf4+M8DqTl5xzfy3/OUlyzr3lnOv+LkoAAAJfSURBVAv3OlNNcs696pzLdM4lVdjWzDn3iXNubfnnpl5m9LWAKnSgBLjRzHoBxwF/cs719jiT1/4CrPI6hJ94BvjIzHoCA6jFz4tzrh1wHRBrZn2BEGCSt6lq3D+A0yptuw34zMy6AZ+V3w8aAVXoZpZhZj+V385l/w9sO29Tecc5Fw2MBV72OovXnHONgNHAKwBmVmRme7xN5blQIMI5FwrUB7Z6nKdGmdlXQFalzeOBWeW3ZwFn12ioahZQhV6Rc64jMAhY6m0STz0N3AKUeR3ED3QGdgCvlZ+Cetk5F+l1KK+Y2RbgcWAzkAFkm9nH3qbyC63MLAP2DxCBlh7n8amALHTnXANgDnC9meV4nccLzrkzgUwzW+Z1Fj8RCgwGnjezQcBeguy/00ei/NzweKAT0BaIdM5d7G0qqW4BV+jOubrsL/N/mdlcr/N4aBQwzjm3EXgbOMk594a3kTyVDqSb2X/+xxbH/oKvrcYAG8xsh5kVA3OBkR5n8gfbnXNtAMo/Z3qcx6cCqtCdc47950hXmdmTXufxkplNNbNoM+vI/he7lphZrR2Bmdk2IM0516N808lAsoeRvLYZOM45V7/85+ZkavGLxBXMBy4rv30ZMM/DLD4X6nWAIzQKuARIdM4tL992u5kt8jCT+I9rgX8558KA9cDlHufxjJktdc7FAT+xf3bYzwT5uyQrc869BZwAtHDOpQP3AA8Ds51zV7L/l95E7xL6nt4pKiISJALqlIuIiBycCl1EJEio0EVEgoQKXUQkSKjQRUSChApdRCRIqNBFRIKECl1EJEj8P64jOgbY0o6gAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.plot([2,3,5,7,9,11], mean_times_s7[:6])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def mn(x): return round(sum(x)/len(x),3)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Test 100 kernels each of larger kernels"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Kernel Size 7: Score: 0.952 Time: 5.322s\n",
      "Kernel Size 11: Score: 0.962 Time: 6.52s\n",
      "Kernel Size 15: Score: 0.956 Time: 7.862s\n",
      "Kernel Size 19: Score: 0.965 Time: 8.23s\n",
      "Kernel Size 23: Score: 0.943 Time: 9.699s\n",
      "Kernel Size 27: Score: 0.953 Time: 11.522s\n",
      "Kernel Size 31: Score: 0.941 Time: 12.626s\n",
      "Kernel Size 35: Score: 0.948 Time: 12.208s\n",
      "Kernel Size 39: Score: 0.947 Time: 11.937s\n",
      "Kernel Size 43: Score: 0.931 Time: 12.709s\n",
      "Kernel Size 47: Score: 0.936 Time: 14.581s\n",
      "Kernel Size 51: Score: 0.926 Time: 15.842s\n",
      "Kernel Size 55: Score: 0.932 Time: 17.233s\n",
      "Kernel Size 59: Score: 0.926 Time: 18.059s\n",
      "Kernel Size 63: Score: 0.932 Time: 20.527s\n",
      "Kernel Size 67: Score: 0.925 Time: 20.773s\n",
      "Kernel Size 71: Score: 0.935 Time: 18.156s\n",
      "Kernel Size 75: Score: 0.915 Time: 19.595s\n",
      "Kernel Size 79: Score: 0.912 Time: 21.77s\n",
      "Kernel Size 83: Score: 0.926 Time: 23.429s\n",
      "Kernel Size 87: Score: 0.923 Time: 25.025s\n",
      "Kernel Size 91: Score: 0.923 Time: 26.44s\n",
      "Kernel Size 95: Score: 0.902 Time: 26.552s\n",
      "Kernel Size 99: Score: 0.928 Time: 29.478s\n"
     ]
    }
   ],
   "source": [
    "scores_ks, times_ks = [],[]\n",
    "for kernel_size in range(7,100,4):\n",
    "    times, scores = timing_test(1, np.array((kernel_size,)), stride=7, num_kernels=100, seq_length=32000, show_progress=False)\n",
    "    print(f\"Kernel Size {kernel_size}: Score: {mn(scores)} Time: {mn(times)}s\")\n",
    "    scores_ks.append(mn(scores))\n",
    "    times_ks.append(mn(times))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYgAAAEGCAYAAAB/+QKOAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAgAElEQVR4nO3deXxU9bn48c+TfV9IQiALCRAUEkiCAgqoWLeqValLq9aqvbdea1u7Xm9rl2strfXX1t7a7ba11nvF1rpQF6pcFXEXVFD2PUBCFkjCloWQ/fn9MRM6JJNkksyZyfK8X6+8mDnnzDnfzGuYJ9/zfL/PV1QVY4wxpruQYDfAGGPM8GQBwhhjjFcWIIwxxnhlAcIYY4xXFiCMMcZ4FRbsBvhLamqq5ubmBrsZxhgzonz44YeHVDXN275REyByc3NZt25dsJthjDEjioiU9bbPbjEZY4zxygKEMcYYryxAGGOM8coChDHGGK8sQBhjjPHKAoQxxhivLEAYY4zxygLEMHesqZWn1pZjZdmNMYFmAWKY+8Uru/jW3zex42BDsJtijBljLEAMY7UNLTy1rhyALZV1QW6NMWassQAxjP3v6n20dnQSERrC1qr6YDfHGDPGjJpaTKNNQ3MbS9eUcdnMCRyoa2abBQhjTIBZD2KYevz9/TQ0t3PHoqnMzEhk24F6OjstUW2MCRwLEMNQc1sHD7+zj3PyUinMSqIgI4HGlnb2H2kKdtOMMWOIBYhh6Nn1ldQ2tPDF86cCUJCRCGB5CGNMQFmAGGY6OpU/vrmHwqxEFkxNAeC0CXGEhQhbq2wkkzEmcCxADFJ7R6cjk9de2nKQ0sNNfHHRVEQEgMiwUPLGx1kPwhgTUBYgBqGzU7n2D2v48uMf+TVIqCq/f7OEKamxXFIw4ZR9BRmJFiCMMQFlAWIQ3t1ziI3lx1ix+SB/eX+/3877TskhtlTW84VFUwgNkVP2FWQkcKixhZr6Zr9dzxhj+mIBYhAeXV1GSmwE505L5b4Xt1FS0+iX8/7+jT2kJ0TyydmZPfYVZCQAlqg2xgSOBYgBKj/SxKod1dw4bxIPfKqI6PBQvv7kelrbO4d03g3lx1i95zC3nTOFyLDQHvvzTwYIS1QbYwLD0QAhIpeKyE4RKRGRu73szxGRVSKySUTeEJEsj32TROQVEdkuIttEJNfJtvrqL++XESLCZ86aRHpCFPdfM4stlfX8atWuIZ33D2/sISEqjBvPmuR1f3xUODkpMdaDMMYEjGMBQkRCgd8BlwH5wI0ikt/tsAeApapaCCwB7vfYtxT4uarOAOYBNU611VfNbR08ubacS/LTyUiKBuDSmRP59Jwsfv/GHtaWHhnUeUtqGnl520FuXZBLXGTv1U8KMhLYYj0IY0yAONmDmAeUqOpeVW0FngAWdzsmH1jlfvx61353IAlT1ZUAqtqoqkGfRrx8YxXHmtq4ZX7uKdvvubKArOQYvvHkBhqa2wZ83ofe2kNkWAifW5Db53EFGYmUHzlB3YmBX8MYYwbKyQCRCZR7PK9wb/O0EbjW/fhqIF5EUoDTgGMi8oyIrBeRn7t7JKcQkdtFZJ2IrKutrXXgV/gnVeXR1aWclh7H2VPGnbIvLjKMX15fTNWxE9y7fNuAznug7gTPrq/k+jnZpMRF9nlsVx7CCvcZYwLByQAhXrZ1nzRwF7BIRNYDi4BKoB1Xldlz3fvnAlOAz/U4mepDqjpHVeekpaX5sek9fbT/GFur6rllfu7JCWyezsxJ5s4LpvH3jyp4cdMBn8/757f30alw27lT+j22wBLVxpgAcjJAVADZHs+zgCrPA1S1SlWvUdXZwPfc2+rcr13vvj3VDjwHnOFgW/u1dE0p8ZFhXO1lCGqXr1yQR1F2Et99djMH6/qfr3CsqZXHP9jPVUUZZI+L6ff48fFRjI+PtB6EMSYgnAwQa4FpIjJZRCKAG4DlngeISKqIdLXhO8AjHq9NFpGubsEFwMDu3fhRTUMzKzYf4Lo5WcT2kUQODw3hweuLaW3v5K6nN/ZbnnvpmjKaWjv4wqL+ew9dCjISbCSTMSYgHAsQ7r/87wReBrYDT6nqVhFZIiJXuQ87H9gpIruAdOA+92s7cN1eWiUim3HdrvqTU23tzxMflNPWodx8dk6/x05OjeWeK/N5p+QQ/7O6tNfjmlrb+Z9393Hh9PFMn5Dgc1sKMhIpqW2kua3D59cYY8xgOLqinKquAFZ023aPx+NlwLJeXrsSKHSyfb5o6+jkr++Xcd5paUxJi/PpNTfMzWbV9hp++tIOFualeA0AT60t52hTG3e4S3r7qiAjgY5OZefBBoqykwb0WmOMGQibSd2PV7ZWU13fwq3z++89dBER/t+1s0iICuPrT2zo8dd+W0cnf3p7H3NykpmbO66Xs3hna0MYYwLFAkQ/Hl1TSva4aM4/ffyAXpcaF8nPritkx8EGfvHKzlP2/WNjFZXHTpxcEGggssdFEx8VZiOZjDGOswDRhx0H6/lg3xFuPjunR3VVX1wwPZ3Pnj2JP729j3dLDgGuUuF/eHMPp6fH87EBBh1w9U7yJ1qi2hjjPAsQfVi6pozIsBA+PSe7/4N78b3L85mSFsu/P7WRuqY2XttRw67qRu44fwohgwg64LrNtONgPR39jJIyxpihsADRi7oTbTz7USWfLM4kKSZi0OeJjgjlweuLOdTYwnef28wf3txDZlI0VxRmDPqcBRkJNLd1srfWP2XGjTHGGwsQvVj2YQUn2jq4eQDJ6d4UZiXxjYtP48VNB1hXdpTbz5tCeOjg3/qCTFsbwhjjPAsQXnR2Ko+tKWVOTjIzMxP9cs47Fk1lXu44xsdHDumWFcDUtDgiwkIsUW2McZSj8yBGqrd211J6uIlvXnK6384ZGiL85bazaGxpJzqi54JAAxEeGsL0CfFsqbQehDHGOdaD8GLpmjLS4iO5tGCCX88bERbCuNjB5zM8uUpu1KFqiWpjjDMsQHRTdvg4r++s4TPzJhERNnzfnvyMROqb26k4eiLYTTHGjFLD9xswSP7yXhmh7iVFh7OZGZaoNsY4ywKEhxOtriVFL505gfSEqGA3p0/TJyQQIrDNEtXGGIdYgPDw/IZK6pvbubWfpT+Hg+iIUKamxVkPwhjjGAsQbqrKo2vKmDExgTk5ycFujk9sbQhjjJMsQLitKzvK9gP13Do/x+uSosNRQUYiB+ubOdzYEuymGGNGIQsQbo+uLiUhKozFxb0vKTrcFPgxUa2qfOPJDTzzUcWQz2WMGR0sQADV9c28tOUg18/NHvIktkDK92OAWFd2lGfXV/Lb10tsboUxBrAAAcDj7++nQ5XP+rCk6HCSFBNBZlK0X0puPOpeHnVv7XHLaxhjAAsQtLZ38vgH+/nY6ePJSYkNdnMGrCAjgW1D/ELv6kFdd2YW4aHC8o1VfmqdMWYkG/MBoraxhazkaG7xQ9XWYCjISGTf4eMcb2kf9Dm6elBfuSCPRaelsXxDFZ221oQxY96YDxCZSdE8+6WFLDotLdhNGZSCjARUYfuBwfUiuvegrirO5GB9Mx+UHvFzS40xI82YDxBdRsrQ1u661obYUjm4PMRLWw9S29Bysgd10YzxxESE8vwGu81kzFjnaIAQkUtFZKeIlIjI3V7254jIKhHZJCJviEiWx74OEdng/lnuZDtHsgkJUYyLjRh0Ynnp6lJyU2I4b5qrBxUTEcbF+ems2HyA1vZOfzbVGDPCOBYgRCQU+B1wGZAP3Cgi+d0OewBYqqqFwBLgfo99J1S12P1zlVPtHOlEZNAzqrdU1rGu7Cg3z889ZX3sxcUZ1J1o461dtf5sqjFmhHGyBzEPKFHVvaraCjwBLO52TD6wyv34dS/7jQ8KMhLZXdMw4L/4H1tTRnR4KNedmXXK9nOnpZEcE87zNprJmDHNyQCRCZR7PK9wb/O0EbjW/fhqIF5EUtzPo0RknYi8JyKf9HYBEbndfcy62tqx+9duQUYCbR3KruoGn19zrKmV5zZUcvUZmSRGh5+yLzw0hMtnTWTltoNDGh1ljBnZnAwQ3rK+3cdO3gUsEpH1wCKgEuj6RpqkqnOAzwAPisjUHidTfUhV56jqnLS0kTkKyR+6Sm4MZD7EU+vKaWnv7HV47+LiTJrbOlm5rdovbTTGjDxOBogKINvjeRZwyj0LVa1S1WtUdTbwPfe2uq597n/3Am8Asx1s64iWmxJLbESozzOqOzqVx94r46zJ45g+IcHrMXNykslIjOL5DZX+bKoxZgRxMkCsBaaJyGQRiQBuAE4ZjSQiqSLS1YbvAI+4tyeLSGTXMcBCYJuDbR3RQkKEGRN9T1S/sbOG8iMn+lz3IiREuLI4g7d2H7JqscaMUY4FCFVtB+4EXga2A0+p6lYRWSIiXaOSzgd2isguIB24z719BrBORDbiSl7/P1W1ANGHgowEth+o92kG9KNrypiQEMXF+el9Hre4KJOOTmXFloP+aqYxZgQJc/LkqroCWNFt2z0ej5cBy7y8bjUwy8m2jTYFGYk8uqaM0sPHmZIW1+txe2sbeWtXLd+8+DTCQ/v++2DGxHimjY9j+YZKbvZjIcPOTqWmoYX0hMgRO0HRmLHA0QBhAsez9HdfAeKx98oIDxVumJfd6zFdRITFxRk88MouKo42kZUc45e2/mLlTn73+h7S4iOZm5vM3NxxzM0dx4yJCYSGWMAwZriwADFKnJYeT3iosLWqniuLMrwec7ylnWXrKrh81kTGx0f5dN6rijJ54JVd/GPjAb54fo+BZANWUtPAH9/cyzl5qaTGRbC29CgrNrtuYcVFhnFGTjLzcpOZkzuO4uwkosJHzvocxow2FiBGiYiwEKaNj+9zJNOz6ytpaGnnlvm5Pp93UkoMsyclsXxj1ZADhKryg+VbiY4I5cEbikmNiwSg8tgJ1pUe4YN9R1hXepQHXtkFQHioUJiVxJzcZObljmNOzjgSY8L7uoQxxo8sQIwiBRkJvLajBlXtcW9fVVm6ppSZmQmcMSlpQOddXJTBvf/Yxq7qBk5Ljx90+17cfIB3Sw6zZHHByeAAroq6mcWZJ5d7PdbUyrrSo6wtO8LafUd45J19/PHNvUSFh/DMFxeevJ1mjHGWVXMdRQoyEjh8vJWD9c099r239wi7qhu5ZX7ugBPDnyjMIERg+RAqvDa2tPOjF7ZRkJHATWf1nfBOiongovx0vnPZDJ750kI2/eDj/PW2s2jrUP5vy4FBt8EYMzAWIEaRgsxEALZW9pwPsXRNKUkx4VzVS36iL2nxkSzMS+X5jZWDXq/616t2U13fwpLFMweciI6OCGVhXiqFWYm8U3JoUNc3xgycBYhRZMbEBEToMWGu6tgJXtlWzfVzswed9F1cnEn5kROsLz824Nfuqm7gkXf2cf2cbM7MSR7U9QHOyUtlU0Ud9c1tgz6HMcZ3FiBGkbjIMCanxPZIVD/+/n46VflsP7d2+vLxgnQiwkIGfJtJVbnn+S3ERobxrUtPH/T1ARZMTaWjU3l/r612Z0wgWIAYZfK7rQ3R0t7B3z7Yz4XT08keN/h5DPFR4Vw0YzwvbKqivcP3suLLN1bx3t4j/MfHTyfFIzE9GGfkJBEVHsK7dpvJmICwADHKFGQkUnnsBMeaWgFYsfkAh4+3cuuCoc+Evqook0ONrazec9in4xua27jvxe0UZiVy47xJQ75+ZFgoc3PHWYAwJkAsQIwy3Ut/P7q6jClpsSycmjrkc59/ehrxUWE+r1f94Ku7qW1s4UeDSEz3ZmFeKrtrGqnxMlLLGONfFiBGmQKPkhsby4+xofwYt5ydc8qSooMVFR7KZTMn8PLWgzS3dfR57I6D9fzv6lJumDuJouyBzbvoyzl5rkD37h7rRRjjNAsQo0xKXCQTEqLYWlXH0jVlxEaEcm23JUWHYnFxJo0t7by2o6bXY1SVe57bSnxUGN/6+NAS093lT0wgKSacd0t8u81ljBk8CxCjUEFGAh/sO8I/NlVxzRlZxEf5rzzF2VNSSIuP7HMhoec2VPJB6RG+fel0kmMj/HZtcK1TsWBqCu+WHBr0nAxjjG8sQIxCBRkJVNU109rHkqKDFRoiXFmYwes7aqk70XM+Qt2JNu57cQdF2UlcP6f/irGDsWBqKgfqmtl36Lgj5zfGuFiAGIXyM1wzqhdMTWHaEGon9WZxcQatHZ287GUhoV+u3MXh4y38ePFMv+Q9vDmZh7DRTMY4ygLEKHRmTjJp8ZHcsWjo5bm9KcxKJDclhuc3nnqbaVtVPUvXlHLTWZOYlZXoyLUBclJiyEyKtjyEMQ6zADEKpcVHsvZ7F3HeaWmOnF9EuKo4k9V7Dp8cbtrZ6ZoxnRQTwV2X+Dcx7e36C/NSWL3nEB0+LLFqjBkcCxBmUK4qykAV/rHJVV31mfWVrCs7yt2XTScpxr+JaW8W5qVS39ze5/oXxpihsQBhBiVvfBwzMxNYvqGSuqY27l+xnTMmJXHdGf4bUtuXBe6Jf1bd1RjnWIAwg7a4KJONFXV846kNHG1qZYmDienu0uIjOT09ntWWhzDGMRYgzKBdUTQREXhtRw03n53DzEznEtPeLMxLZW3pkX5ndRtjBscChBm0iYnRzJ+SQmpcBN90ODHtzcK8FFraO/mo7GjAr23MWOBogBCRS0Vkp4iUiMjdXvbniMgqEdkkIm+ISFa3/QkiUikiv3WynWbwfn3jbJ6/8xwSo/03W9tXZ01JITRELA9hjEMcCxAiEgr8DrgMyAduFJH8boc9ACxV1UJgCXB/t/0/At50qo1m6FLjIslMig7KteMiwyjOTuJdH8uPG2MGxskexDygRFX3qmor8ASwuNsx+cAq9+PXPfeLyJlAOvCKg200I9zCvFQ2VxzzWvbDGDM0/QYIEblTRAazkHAmUO7xvMK9zdNG4Fr346uBeBFJEZEQ4BfAf/TTtttFZJ2IrKutrR1EE81It3BqCp0K7+21XoQx/uZLD2ICsFZEnnLnFHwdx+jtuO7TXu8CFonIemARUAm0A18CVqhqOX1Q1YdUdY6qzklLc2bWsBneZk9KJjo8lNWWhzDG78L6O0BVvy8i/wlcAvwL8FsReQr4s6ru6eOlFYBnOc8s4JSlyFS1CrgGQETigGtVtU5E5gPnisiXgDggQkQaVbVHotuMbRFhIcybPM4S1cY4wKcchLoK7x90/7QDycAyEflZHy9bC0wTkckiEgHcACz3PEBEUt23kwC+Azzivt5NqjpJVXNx9TKWWnAwvTknL5U9tcc5WGfLkBrjT77kIL4qIh8CPwPeBWap6heBM/ln/qAHVW0H7gReBrYDT6nqVhFZIiJXuQ87H9gpIrtwJaTvG8ovY8amBXkpgJX/Nsbf+r3FBKQC16hqmedGVe0UkSv6eqGqrgBWdNt2j8fjZcCyfs7xv8D/+tBOM0bNmJDAuNgI3t1zyK/Lqxoz1vlyi2kFcKTriYjEi8hZAKq63amGGeOrkBBhvi1Daozf+RIgfg80ejw/7t5mzLBxTl4q1fUt7Km1ZUiN8RdfAoSox59lqtqJb7emjAmYhVNtGVJj/M2XALHXnagOd/98DdjrdMOMGYhJKTFkj4u2AGGMH/kSIO4AFuCaxFYBnAXc7mSjjBmMhVNTWbP3MO0dncFuijGjQr8BQlVrVPUGVR2vqumq+hlVrQlE44wZiIV5qTQ0t7Olqn5I51lXeoSP9lsJcWP6zSWISBTweaAAiOrarqr/6mC7jBmwBVP/OR+iODtpUOf4YN8Rbnr4PZJiInj32xcQEWZLppixy5dP/2O46jF9HFfp7SygwclGGTMYKXGRzJiYMOg8RNnh43zhsXXERoZR29DCS1sP+rmFxowsvgSIPFX9T+C4qj4KfAKY5WyzjBmchVNTWFd2dMDLkNY3t/H5R9fRqfD3Ly4gNyWGpatLnWnkKNba3mmFE0cRXwJEV6H9YyIyE0gEch1rkTFDsHBaKq3tnawr9T2H0N7RyZf/+hGlh47zh8+eydS0OG6en8u6sqNsqaxzsLWjz8Pv7OUzD7/PtiHmgczw4EuAeMi9HsT3cRXb2wb81NFWGTNI83LHETbAZUiXvLCNt3cf4r6rZzLfnce47swsosNDeWxNWT+vNl3aOzr5i/v9Wl9uSf7RoM8A4a60Wq+qR1X1LVWd4h7N9McAtc+YAYmNDOOMScms3uNbgHh0dSlL15TxhfOmcP3cSSe3J0aHc/UZmTy3oZJjTa1ONXdUeXV7DVXuiroby48FuTXGH/oMEO5Z03cGqC3G+MWCvBQ2V9b1+8X+xs4afviPrVycn863Lp3eY/8t83Noae/kqXV9rltl3JauKSUjMYpzp6WysdxuzY0GvtxiWikid4lItoiM6/pxvGXGDNI5ealoP8uQ7jzYwJ2Pr2f6hAQevL6Y0JCeCyBOn5DAWZPH8dh7ZXR0WhHAvpTUNLB6z2FuOjuHMyYls6umgcaW9mA3ywyRLwHiX4EvA28BH7p/1jnZKGOGoig7idiI0F7zEIcaW/j8o2uJiQjlz5+bQ2xk79OBbpmfS/mRE7yx0+aG9mXpmjIiQkO4YW42xdlJqGIJ/lHAl5nUk738TAlE44wZjPDQEM6aksLqkp49iOa2Dr7w2Iccamzh4VvnMDExus9zXVKQTnpCJI9asrpXDc1t/P3DCq4omkhKXCSFWYmA5SFGA19mUt/ibbuqLvV/c4zxjwVTU3htRw1Vx06QkeQKAqrK3X/fxIdlR/nvm86gMKv/2dbhoSHcdFYO/7VyF3trG5mSFud000ecZz6q5HhrB7fOzwVcExazx0WzscICxEjnyy2muR4/5wL3Alf19QJjgu2caT3Lf//2tRKe21DFf3z8dC6fNdHnc90wL5vwUOGx96wX0Z2q8uiaUoqykyjyKG9SlJVkiepRwJdbTF/x+Pk3YDYQ4XzTjBm809PjSY2LOBkgXthUxS9W7uKa2Zl86fypAzrX+PgoLp81kWXrKjjux8Trmj2Huei/3mSTw39pt3d08m9L1/GjF7b5/dzvlhxmb+1xbp2fc8r24uwkKo+doKah2e/XNIEzmEpkTcA0fzfEGH8SERZMTeXdPYdZv/8o//7URubkJHP/tbMQ6TliqT+3zM+loaWdZ9dX+qV9dU1tfOPJDZTUNPL1JzbQ1OrciJ/fv7GHlduq+fM7+3i/j5Fdg/HomlJSYiN69Mi6ehObrBcxovUbIETkHyKy3P3zArATeN75phkzNAvzUqhtaOGWP3/A+IRI/njzmUSGhQ7qXGdMSmJmZgJL15QOed1rVeW7z23mUGML37t8BnsPHecnK5xZ3n1D+TEeXLWbT8yaSGZSNPc8v5U2P62XUX6kiVXbq7lhXjZR4ae+rwUZCYSGiOO9I+MsX3oQDwC/cP/cD5ynqnc72ipj/GBhXurJx4/cOpeUuMhBn0tEuGV+LruqG3lv75Ehteu5DZW8uOkA37j4NP7tvCn827mT+ct7+3ltR/WQzttdU2s733hyAxMSovjJNbO458p8dlY38KifihD+9f39ANx0Vk6PfTERYZyWHs+GCutBjGS+BIj9wPuq+qaqvgscFpFcX04uIpeKyE4RKRGRHkFFRHJEZJWIbBKRN0Qky2P7hyKyQUS2isgdA/idjAEgKzmGr144jYdvncO09Pghn++qogySYsJZuqZ00OcoP9LEPc9tZW5uMncscuVC7vr46UyfEM+3lm3iUGPLkNvZ5ccvbqf08HF+8ekiEqPDuSQ/nfNPT+PBV3dTXT+03EBzWwdPrt3PJfkTTo4S6644O5GN5ceG3OMyweNLgHga8OyTdri39UlEQoHfAZcB+cCNIpLf7bAHgKWqWggswdVDATgALFDVYlxLnN4tIhk+tNWYU3zz4tM4a0qKX84VFR7K9XOzeWVbNVXHTgz49R2dyr8/tREF/uvT/5y9HRkWyoM3FFPf3M7df9/kly/UV7dV8/j7+7n9vCmc7f79RYQfXlVAa0fnkG9p/WNjFUeb2rhlQc/eQ5eirCTqTrRRdrhpSNcyweNLgAhT1ZNFbdyPfRnFNA8oUdW97tc8ASzudkw+sMr9+PWu/araqqpdf0pF+thOYxz32bNy6FTlcfftlYH441t7+KD0CEsWF5A9LuaUfdMnJPDtS6fz6vYa/vbB0Go/1Ta08O2/byJ/YgLfvPi0U/blpMRyx6KpPL+hyueCht11DW2dNj6O+X0E365Etc2HGLl8+eKtFZGT8x5EZDHgyycrE/D8pFe4t3naCFzrfnw1EC8iKe7rZIvIJvc5fqqqVd0vICK3i8g6EVlXW1vrQ5OMGZrscTFcOD2dv32wn5Z23xcl2lJZx3+9sotPFE7k6tnd/xu4/MuCXM7JS+VHL2xjb23joNqnqnz775tobGnnVzcUe03Kf+n8qWQlDz5hvb78GFsq67llQW6fI8KmjY8jOjyUDTajesTyJUDcAXxXRPaLyH7g28AXfHidt09O977zXcAiEVkPLAIqgXYAVS1333rKA24VkfQeJ1N9SFXnqOqctLQ0H5pkzNDduiCHw8dbWbH5gE/Hn2jt4GtPrCc1LpL7Pjmz1y/VkBDhgU8VEREWwjee3DCoL++/vr+f13bU8J3Lpvead4kKD+XeKwsoqWnkf97dN+BrLF1dSnxkGNf0Eui6hIWGMCsz0UpujGC+TJTbo6pn47odVKCqC1S1xIdzVwDZHs+zgFN6AapaparXqOps4HvubXXdjwG24prFbUzQLZyaypS0WB5d7dvM6vv/bzt7al3J4qSYvu/OTkiM4v5rZrGxoo7frNo9oHbtqW3kxy9u47zT0rjFXfaiNxflp3PRjPE8+OpuDtT5nk+pbWjhxc0HuPbMrD6LHHYpyk5kS1W934bWmsDyZR7ET0QkSVUbVbVBRJJF5Mc+nHstME1EJotIBHADrhXpPM+d6l6UCOA7wCPu7VkiEu1+nAwsxDX/wpigCwkRbjk7hw3lx/r96/j1HTUsXVPGbedMPmXYbV8unzWRa8/I4revl/BhmW9Dats6Ovn6ExuIDg/l59cVEuKlfHl3P7iygI5O5ccv+p6wfuKD/bR1KDfP7z057akoO4nW9k52Hmzw+Rpm+PDlFguSbWgAABtpSURBVNNlqnryf4GqHgUu7+9FqtqOa7Ghl4HtwFOqulVElnjkNM4HdorILiAduM+9fQbwvohsBN4EHlDVzT7+TsY47tozs4iNCGVpH1VeDze28B/LNjF9Qjx3ffz0AZ3/3qvyyUiK5utPbvBpXYVfvbqbzZV13H/NLNITony6Rva4GL50fh4vbjrAO7v7Tyu2dXTy1/f3c+60VKb6WLSwyF0Q0fIQI5MvASJURE7OMHL/Ze/TjCNVXaGqp6nqVFW9z73tHlVd7n68TFWnuY+5rWvkkqquVNVCVS1y//vQwH81Y5wTHxXONWdk8Y9NVRz2MndBVbn7mc3Un2jjl9cX95hp7Mv5f3l9MZVHT/DD5Vv7PHZt6RH++40SPj0ni0tn+l6EEOALi6aQkxLDPcu30Nre922glduqOVjffLJqqy+ykqNJiY2wPISDVm6r5uWtBx05ty8B4i/AKhH5vIh8HlgJPOpIa4wZQW6Zn0NreydPelmS9Mm15azcVs23Lj2dGRMTBnX+ubnj+NL5eTz9YQUvbfGeEG9odtV0ykqO4Z4rCwZ8jajwUO69qoC9tcf58zt9J6wfXV1KVnI0H5s+3ufziwhF2Uk21NVBf3prLw+/vdeRc/uSpP4Z8GNct33ygZcA325AGjOKTUuPZ8HUFP763n7aPZKwpYeOs+SFbSyYmsK/Lpw8pGt87aJpFGYlcvczm73Ofr53+Taqjp3gl9cXE+dD0tibj50+nkvy0/n1qt29TgDccbCe9/cd4eazc7wuz9qXoqwkdtc02hKkDtld00DeeGfWKfF1AtpBXLOprwUuxJVTMGbMu2V+LpXHTrBqh2tJ0vaOTr7+5AbCQoRffLrIp2RxX8JDQ/jl9cU0t3Vw19Mb6fRYG/vFTQf4+0cV3HnBNM7MSR7Sde65Mh9Fey0JvnRNGZFhIXx6TrbX/X0pyk5EFTZbXSa/O9zYwtGmNvLGD72UjDe9BggROU1E7hGR7cBvcU1YE1X9mKr+1pHWGDPCXDRjPBmJUSfrM/329RI2lB/jJ9fM6nc5U19NTYvj+5/I5+3dh05e52BdM999djNF2Ul85YK8IV8jKzmGr1wwjf/bcpA3d5066bTuRBvPflTJ4uIMkmMHvhRMV6LabjP53+4a14TKYPQgduDqLVypqueo6m9w1WEyxriFhYZw09k5vFtymKfXlfOb10q4ZnYmVxT6t3TYTWdN4oLp47n//3aw82ADdz29kdb2Th68vpjwUP9Uornt3MlMTo3l3uVbT5klvuzDCk60dfQ7t6I3ybER5KTEWKLaASXuADEtCAHiWly3ll4XkT+JyIV4nx1tzJh2w9xsIkJD+I9lm5iQEMW9iweeLO6PiPDTawuJiwzjU39YzTslh7jnynwmp8b67RqRYa6E9b5Dx3n4bVfCurNTeWxNKWfmJDMzM3HQ53YtQRr8ANHW0dnvaK2RpKSmkdiIUCYm+ja0eaB6DRCq+qyqXg9MB94AvgGki8jvReQSR1pjzAiUEhfJlUUZiMAvry8mISrckeukxUfy02sLqW9u56IZ6dwwd+D5gP4sOi2Ny2ZO4Dev7ab8SBNv7a6l9HATt/g4Ma43RdlJVNU1UzPEMuNDdc/zW5j3k1d5aYszw0IDraSmkbzxcYNaJdEXvoxiOq6qf1XVK3CVy9gA2IJBxnj40ScLeOEr5zBv8jhHr3NRfjrPfXkhv7lxtmNfCv95RT6C8KMXtrF0TRmpcZFcNsD5Fd0VZ7t6HxuDmKhu6+jkhU0HON7Szh1/+ZDvPLPJ0aVeA2F3TQNTHbq9BAMso62qR1T1j6p6gVMNMmYkiokIoyBj8LdgBqI4O4noiMEtneqLjKRovnrhNF7ZVs1rO2r4zFmTiAgbWp6jICOR0BAJ6m2mtaVHaGhu58HrZ/PF86fyxNpyrvj1OyN2WdT65jaq61uY5tAIJrB1FowxXnz+nMlMSYslLES46axJQz5fVHgo0yfEB3Uk06rtNUSEhfCx6Wl8+9LpPH7b2Zxo6+Ca/17Nf79RQkfnyFr5rsThEUxgAcIY40VEWAgP3Xwmv//smT7XdupPUbYrUd0ZhC9iVeXV7dUsmJpCTIRrQuH8qSm89LXz+HjBBH720k5uevi9Qa0UGCwl1c6OYAILEMaYXuSNj+fi/B7LsAxacVYS9c3tlB4+7rdz+mpPbSNlh5u4cMapv09iTDi//cxsfn5dIZsr6rj0wbd4cZNv63wEW0ltIxFhIT1WJ/QnCxDGmIAI5hKkr253zXS/0EsdKRHhU3OyefGr5zIlLY4vP/4Rdz29cdiXBtld3cCU1NgBlz4ZCAsQxpiAyBsfR0xEKBvLAz+SadX2agoyEshI6n12e25qLE/fMZ+vXpDHMx9V8Ilfv836/UcD2MqBKaltdDT/ABYgjDEBEhoizMpMDPjaEEePt/Jh2dEet5e8CQ8N4ZuXnM6TX5hPe4dy3R/W8JtVu4ddAvtEawcVR084OoIJYHDlH40xZhCKs5P4n3dLaW3vHPLQWV+9vrOGTnXVzfLV3Nxx/N/Xz+U/n9vCL1bu4h+bqgacrI8MC2XJ4oI+ey2Dtae2EVWYlu5sD8IChDEmYIqyk2jt6GTHwXoK3UX8nLZqew3j4yOZOcB5KglR4fzqhtl87PTx/PX9sgHlJDoVNpYf4py8FD43xJLv3gRiiCtYgDDGBNDJRHX5sYAEiNb2Tt7cVcuVRRMHXXr9k7Mz+eTszAG9RlWZe9+rbK6sH9Q1+1NS00hoiJCb4r9aXN5YDsIYEzAZiVGkxkWyIUCJ6vf3HaaxpZ0Lp/tvuK4vRFz5ls2VzuRbdtc0kJMS4/htOgsQxpiAERGKsxMDNtR11fYaIsNCWJiXGpDreZqVmUhJTaMj9Z5KahodnSDXxQKEMSagirKS2FPbSH1zm6PX6Zo9fU5eqqO1q3ozKyuJToXtB/x7m6m1vZPSw02O5x/AAoQxJsCKspNQhS0OV3bdVd1IxdETXOTH2eADMcu9fsYmP/+eZYeP09Gpjg9xBYcDhIhcKiI7RaRERHqUCBeRHBFZJSKbROQNEclyby8WkTUistW973on22mMCZzCLNcX5waHbzO9ur0a8D57OhDSEyJJjYtkc6V/A4TTy4x6cixAiEgo8DvgMiAfuFFE8rsd9gCwVFULgSXA/e7tTcAtqloAXAo8KCKBGRNnjHFUUkwEk1NjHS/9vWp7NYVZiYz3U7HBgRIRCrMS2eznHkRJTSMirrXKneZkD2IeUKKqe1W1FXgCWNztmHxglfvx6137VXWXqu52P64CaoA0B9tqjAmgoqxER0tuHGpsYX35sYCPXupuZmYie2r9m6jeXdNIZlJ0QPIqTgaITKDc43mFe5unjbjWvga4GogXkRTPA0RkHhAB7Ol+ARG5XUTWici62tpavzXcGOOsouwkDtY3c7DOmSVIX99RgypcOIDZ004ozEykU2Fblf8S1YEawQTOBghvs1K6FzS5C1gkIuuBRUAlcDLUishE4DHgX1S1x0rjqvqQqs5R1TlpadbBMGakcLqy66vbq5mYGEVBRoIj5/fVrCz/Jqo7OpU9ASjS18XJAFEBeK6qngVUeR6gqlWqeo2qzga+595WByAiCcCLwPdV9T0H22mMCbD8iQmEObQEaXNbB2/vPsQF08c7tm63r9ITohgfH8kWPyWqK4420dreGZARTOBsgFgLTBORySISAdwALPc8QERSRaSrDd8BHnFvjwCexZXAftrBNhpjgiAqPJQZExMc6UG8t/cwTa0dXORD9dZAmJWZyCY/BYjd7lXkpo70HoSqtgN3Ai8D24GnVHWriCwRkavch50P7BSRXUA6cJ97+6eB84DPicgG90+xU201xgReUXYim8rr/L4E6artNUSHhzJ/akr/BwdAV6L6uB8WIArkEFdwuFifqq4AVnTbdo/H42XAMi+v+wvwFyfbZowJrqKsJP7y3n72Hjruty88VWXV9mrOnZZKVHjgZ097U5iViCpsO1DP3NxxQzpXSU0j4+MjSYwO91Pr+mYzqY0xQeFZ2dVfth9ooKquedjcXgL/zqguqWlwfA0ITxYgjDFBMTUtjtiIUL/mIVZtr0YEPhak2dPejE+IIj1h6IlqVaWkppG8AEyQ62IBwhgTFKEhwqysRL/2IF7dUUNRVhJp8ZF+O6c/zMpMZNMQA+GBumaOt3aQlx6YEUxgAcIYE0RF2UlsO1BPS3vHkM9VU9/MxvJjA1paNFBmZSax99DxAa1K193JVeSsB2GMGQuKs5Jo61C2H2gY8rle21EDwIXDKP/QZVZWAqqwdQi3mbpGMFkOwhgzJvgzUf3q9hoyk6KZPiFwt2B8NdOdqB5KZdeSmkaSY8JJiY3wV7P6ZQHCGBM0ExOjSIuPHHKAaG7r4J2SWi6aEfzZ096Mj49iQkLUkBLVJTUN5I2PC+jvZwHCGBM0IkJRVtKQ14ZYvecQzW2dw/L2UpeZQ5hRrarsrmkkL0AlNrpYgDDGBFVxdiJ7a49Td2LwS5C+ur2G2IhQzpoytIloTirMSmTfoeM0DGKp1cPHWznW1BawGdRdLEAYY4KqKw/xbsmhQb1eVXltew3nnZZGZNjwmD3tzaxM14zqrYMo/d01gilQZb67WIAwxgTVGZOSyUqO5qt/W8/v39hDxwBrM22tqudgffOwvr0E/0xUDyYPEegaTF0sQBhjgio2MowXvnIOlxSk89OXdnDTw+9RdeyEz69fuc09e/r04b0mTFp8JBMTowY1kmlPTSOxEaFMTAzs8qkWIIwxQZcUE8HvPnMGP7+ukE0VdVz64Fu8uOmAT69dtaOaMyYlkxI3vGZPezMzc3BrVO8OwggmsABhjBkmRIRPzclmxVfPZXJaHF9+/CPuenpjn7OPD9Y1s6WyPuhLi/qqMDORvYNIVJcEYQQTWIAwxgwzuamxLLtjPl+9II9nPqrg8l+9zfr9R70eu2pHNQAXD/P8Q5eZWV15CN8T1fXNbVTXtwQ8/wAWIIwxw1B4aAjfvOR0nvzCfDo6lev+sIbfrNrdI4G9ansNk8bFBOXLczBmDSJRHawRTGABwhgzjM3NHceKr53LJ2ZN5Bcrd3HDQ2soP9IEwInWDt4tOcSFw3T2tDepcZFkDDBRXVIdnBFMYAHCGDPMJUaH8+sbZ/Pg9cVsP9DA5b96m+c3VPJOySFa2juH1eJAvpiZmTigALG7poGIsBCyx8U42CrvHF1y1Bhj/OWTszM5MyeZrz+5ga89sYGU2AjiI8OGvIxnoBVmJfLKtmrqm9tIiOp/6dCSmkampMYSGhL4XpL1IIwxI0b2uBievP1svnnxaRw70caFM8YTETayvsYGOmFud00j0wK4SJAn60EYY0aUsNAQvnrhNK6enUliTP9/gQ83nonqBVNT+zy2qbWdymMn+NSZ2YFoWg8WIIwxI1Iw7sn7Q0pcJJlJ0Wz2Yajr3trjqAZ2kSBPI6tvZowxo8CszEQ2+1DivCRINZi6OBogRORSEdkpIiUicreX/TkiskpENonIGyKS5bHvJRE5JiIvONlGY4wJtFlZiZQebuq3xPnumgZCQ4TclNgAtexUjgUIEQkFfgdcBuQDN4pIfrfDHgCWqmohsAS432Pfz4GbnWqfMcYES1eiur81qktqGslJiQlaIt7Jq84DSlR1r6q2Ak8Ai7sdkw+scj9+3XO/qq4Chr6SuTHGDDOzfFyjendNY1BmUHdxMkBkAuUezyvc2zxtBK51P74aiBeRFF8vICK3i8g6EVlXW1s7pMYaY0ygjIuNIDMpus8lSFvbOyk73BTUMiJOBghvszq6rwRyF7BIRNYDi4BKoPfSjd1PpvqQqs5R1TlpacO7FrwxxngqzErscy5E6eHjdHQq04JQxbWLkwGiAvAcvJsFVHkeoKpVqnqNqs4GvufeNrhVvY0xZgSZmZlI2eEm6pq8J6qDPYIJnA0Qa4FpIjJZRCKAG4DlngeISKqIdLXhO8AjDrbHGGOGjZMT5qq8/028u7oREZiaNgoDhKq2A3cCLwPbgadUdauILBGRq9yHnQ/sFJFdQDpwX9frReRt4GngQhGpEJGPO9VWY4wJtP4S1SW1jWQlRxMdERrIZp3C0ZnUqroCWNFt2z0ej5cBy3p57blOts0YY4IpOTaCrOToXpcg3V3dQF4Qew9gM6mNMSZoCrO8l/7u6FT2HjoetCJ9XSxAGGNMkMzMTGT/kZ6J6vIjTbS2d1oPwhhjxqrCzCSgZx7i5AimIBXp62IBwhhjgmRmZgLQM0DsHgZDXMEChDHGBE1STATZ46LZXHlqZdfdNQ2kJ0T6tOKckyxAGGNMEBVmJvXoQeypaQx67wEsQBhjTFDNzEyk/MgJjjW1AqCqlNQ0BrXERhcLEMYYE0SFWadOmDtQ18zx1g6mWg/CGGPGtpkZpwaIrgR1MMt8d7EAYYwxQZQYE05OSszJGdXDoUhfFwsQxhgTZDMz/zmjuqSmgeSYcFJiI4LcKgsQxhgTdLMyE6k4eoKjx1spcY9gEvG2pE5gWYAwxpggK/So7Lq7ppG8YTCCCSxAGGNM0BW4A8QbO2s51tQ2LPIPYAHCGGOCLjE6nNyUGJZvrASGxwgmsABhjDHDwszMRA41uibLWQ/CGGPMSV0rzMVGhDIxMSrIrXGxAGGMMcPALPeM6rz0+GExggksQBhjzLAw092DCPYiQZ4cXZPaGGOMbxKiwvn+J2YwN3dcsJtykgUIY4wZJm47d0qwm3AKu8VkjDHGK0cDhIhcKiI7RaRERO72sj9HRFaJyCYReUNEsjz23Soiu90/tzrZTmOMMT05FiBEJBT4HXAZkA/cKCL53Q57AFiqqoXAEuB+92vHAT8AzgLmAT8QkWSn2mqMMaYnJ3sQ84ASVd2rqq3AE8DibsfkA6vcj1/32P9xYKWqHlHVo8BK4FIH22qMMaYbJwNEJlDu8bzCvc3TRuBa9+OrgXgRSfHxtYjI7SKyTkTW1dbW+q3hxhhjnA0Q3mZ6aLfndwGLRGQ9sAioBNp9fC2q+pCqzlHVOWlpaUNtrzHGGA9ODnOtALI9nmcBVZ4HqGoVcA2AiMQB16pqnYhUAOd3e+0bDrbVGGNMN072INYC00RksohEADcAyz0PEJFUEelqw3eAR9yPXwYuEZFkd3L6Evc2Y4wxAeJYD0JV20XkTlxf7KHAI6q6VUSWAOtUdTmuXsL9IqLAW8CX3a89IiI/whVkAJao6pG+rvfhhx8eEpEyh36d4SYVOBTsRgwD9j642PvgYu+Dy0Dfh5zedohqj1v7ZpgTkXWqOifY7Qg2ex9c7H1wsffBxZ/vg82kNsYY45UFCGOMMV5ZgBiZHgp2A4YJex9c7H1wsffBxW/vg+UgjDHGeGU9CGOMMV5ZgDDGGOOVBYhhTESyReR1EdkuIltF5Gvu7eNEZKW7FPrKsVLpVkRCRWS9iLzgfj5ZRN53vw9PuidkjmoikiQiy0Rkh/tzMX8Mfx6+4f5/sUVE/iYiUWPhMyEij4hIjYhs8djm9TMgLr92L7mwSUTOGMi1LEAMb+3Av6vqDOBs4Mvukul3A6tUdRquarg91toYpb4GbPd4/lPgl+734Sjw+aC0KrB+BbykqtOBIlzvx5j7PIhIJvBVYI6qzsQ1GfcGxsZn4n/pWd26t8/AZcA098/twO8HciELEMOYqh5Q1Y/cjxtwfRlk4iqL/qj7sEeBTwanhYHjXkzqE8DD7ucCXAAscx8y6t8HEUkAzgP+DKCqrap6jDH4eXALA6JFJAyIAQ4wBj4TqvoW0L2yRG+fgcW41txRVX0PSBKRib5eywLECCEiucBs4H0gXVUPgCuIAOOD17KAeRD4FtDpfp4CHFPVdvdzryXhR5kpQC3wP+5bbQ+LSCxj8POgqpW4Fhzbjysw1AEfMvY+E116+wz4tHRCbyxAjADuSrd/B76uqvXBbk+gicgVQI2qfui52cuho33MdhhwBvB7VZ0NHGcM3E7yxn2PfTEwGcgAYnHdTulutH8m+jOk/ycWIIY5EQnHFRz+qqrPuDdXd3UT3f/WBKt9AbIQuEpESnGtTHgBrh5Fkvv2AngpJz8KVQAVqvq++/kyXAFjrH0eAC4C9qlqraq2Ac8ACxh7n4kuvX0G+l12oS8WIIYx9332PwPbVfW/PHYtB251P74VeD7QbQskVf2Oqmapai6uRORrqnoTrmVqr3MfNhbeh4NAuYic7t50IbCNMfZ5cNsPnC0iMe7/J13vxZj6THjo7TOwHLjFPZrpbKCu61aUL2wm9TAmIucAbwOb+ee99+/iykM8BUzC9R/lU/2VQx8tROR84C5VvUJEpuDqUYwD1gOfVdWWYLbPaSJSjCtRHwHsBf4F1x96Y+7zICI/BK7HNdpvPXAbrvvro/ozISJ/w7VUQipQDfwAeA4vnwF38PwtrlFPTcC/qOo6n69lAcIYY4w3dovJGGOMVxYgjDHGeGUBwhhjjFcWIIwxxnhlAcIYY4xXFiDMmCUijR6PL3dXwpzk8DXfEJEeC8qLyBXu8hkbRWSbiHzBvf0OEbnFyTYZ05uw/g8xZnQTkQuB3wCXqOp+H18T5lHzZ6jXD8e1TOQ8Va0QkUggF0BV/+CPaxgzGBYgzJgmIucCfwIuV9U97m1pwB9wTToCVw2sd0XkXlx1f3KBQyLyCnAVrkqiU4FnVfVb7nNcAvwQiAT24JqgdLLH0k08rv+LhwHcE7t2us9zL9AIPA6s8HjNLFzF+5q8tXVw74Yxp7IAYcaySFwlCc5X1R0e23+Fa02Bd9y3nF4GZrj3nQmco6onRORzQDGuKrstwE4R+Q1wAvg+cJGqHheRbwPfBJZ4a4R7xutyoExEVgEvAH9T1U6PY6rc10JEvgwsUtUyEXm8j7YaMyQWIMxY1gasxrWozNc8tl8E5LuqFACQICLx7sfLVfWEx7GrVLUOQES2ATlAEpAPvOs+RwSwpq+GqOptIjLLfe27gIuBz3U/TkQW4iopcW5fbXWvH2LMkFiAMGNZJ/Bp4FUR+a6q/sS9PQSY3y0Q4P4SPt7tHJ51fjpw/Z8SYKWq3jiQxqjqZmCziDwG7KNbgHBX6fwzcJXH7SqvbTXGH2wUkxnTVLUJuAK4SUS6lqd8Bbiz6xh3gbyBeA9YKCJ57tfHiMhpvR0sInHuIoRdioGybseE4yrG9m1V3eWxa6htNaZXFiDMmOeufHop8H0RWYx7rWP3Iu/bgDsGeL5aXH/9/01ENuEKGNP7eIkA3xKRnSKyAVdy+3PdjlkAzAV+KCIb3D8ZQ22rMX2xaq7GGGO8sh6EMcYYryxAGGOM8coChDHGGK8sQBhjjPHKAoQxxhivLEAYY4zxygKEMcYYr/4/qqvyrDgQWCQAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.xlabel(\"Kernel Size\")\n",
    "plt.ylabel(\"Accuracy\")\n",
    "plt.plot(np.arange(7,100,4), scores_ks);"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAX4AAAEGCAYAAABiq/5QAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAgAElEQVR4nO3deXhU5fn/8ffNkrAk7GHfI7ssIiAgKm7VWitqXUAriCjutdZWW5e61X6rtS61aquCoLK4Cy5VqetPkX0n7DsEQliSQCAhy/37YwaNSJIBMjNJ5vO6rrkyc+acObfj4TNnnnnO85i7IyIisaNKtAsQEZHIUvCLiMQYBb+ISIxR8IuIxBgFv4hIjKkW7QJC0ahRI2/btm20yxARqVDmzp27w92TDl1eIYK/bdu2zJkzJ9pliIhUKGa24XDL1dQjIhJjwhb8ZlbDzGaZ2UIzW2pmDwaXtzOzmWa2ysxeN7O4cNUgIiI/Fc4z/lzgDHfvCfQCzjWz/sCjwJPu3gHYDYwKYw0iInKIsAW/B+wNPqwevDlwBvBWcPl44MJw1SAiIj8V1jZ+M6tqZguA7cA0YA2Q4e75wVU2Ay2K2Xa0mc0xsznp6enhLFNEJKaENfjdvcDdewEtgX5Al8OtVsy2L7h7H3fvk5T0k95IIiJylCLSq8fdM4Avgf5APTM72I20JZAaiRpERCQgnL16ksysXvB+TeAsYBnwBXBJcLURwJRw1SAiUlHl5BXwwNSlpGbsL/PXDucZfzPgCzNbBMwGprn7B8BdwO/MbDXQEBgTxhpERCqksd+uY9z09Wzcta/MXztsV+66+yLghMMsX0ugvV9ERA5j595cnvtiDWd1aUL/9g3L/PV15a6ISDnzz89WsT+vgD/+vHNYXl/BLyJSjqxN38uEmRsZ1q8VxzVOCMs+FPwiIuXIox8vJ75aFW47s2PY9qHgFxEpJ2at28UnS9O4cXAySYnxYduPgl9EpBxwdx75aBlN69Rg1KD2Yd2Xgl9EpBz4YNFWFm7K4I6fdaRmXNWw7kvBLyISZbn5BTz2yXI6N03k4t4tw74/Bb+ISJS9+t0GNu3azz2/6ELVKhb2/Sn4RUSiKGPfAZ75fDWndkzilA6RGZBSwS8iEkX/+nw1e3LyuPu88FysdTgKfhGRKNm4cx+vfLeBS05sSeemdSK2XwW/iEiUPPbJcqpWMX53dqeI7lfBLyISBQs2ZfDBoq1cd0o7mtatEdF9K/hFRCLM3fnrh8tolBDP6NOSI75/Bb+ISIR9mpLGrPW7uP3sDiTEh210/GIp+EVEIiivoJBH/7uc4xoncHmfVlGpQcEvIhJBk2ZtZO2ObP70885UqxqdCFbwi4hESFZOHk/9bxUD2jfkjM6No1aHgl9EJEL+/eUadmUf4O7zumAW/qEZiqPgFxGJgNSM/Yz5Zh0XndCC7i3rRrUWBb+ISAQ8/ukKHLjjZ+GbWStUke9HJCJSCeXmF5C5P4/MfXlk7M8jY18emfvzyNh3gPS9ubw7fwvXn5pMy/q1ol2qgl9E5Ei8O38zHy/Z9n2wZwZDfn9eQbHbmEGXpnW4cXDkL9Y6HAW/iEgICgudv3+6gue/XEOrBjVpVqcmrRrUonvN6tSrVZ26NatTt1Yc9WoG7terVZ16NeOoW6s6ifHVqBKBcfZDpeAXESlFbn4Bf3hzEVMXpjKsX2seHtItan3wy4KCX0SkBJn78hj96hxmrtvFned24sbTkqPaFbMsKPhFRIqxadc+Ro6bzYad2Tw9tBdDerWIdkllQsEvInIYizdnMnLcbA7kF/DKNScxILlhtEsqMwp+EZFDfL48jZsnzKdB7TgmXXcSHZokRrukMqXgFxEpYsLMDdz33hK6Nq/D2BF9aVwnspOkRIKCX0SEH3fXPL1TEv+6oje1ozBWfiRUzv8qEZEjUNm6a5YmbP9lZtbKzL4ws2VmttTMbgsuf8DMtpjZguDtvHDVICJSmsx9eQwfM4upC1O589xO/PWi4yt16EN4z/jzgTvcfZ6ZJQJzzWxa8Lkn3f3xMO5bRKRUlbW7ZmnCFvzuvhXYGry/x8yWAbHxropIubd6+16ueHEGOXmVr7tmaSLyfcbM2gInADODi24xs0VmNtbM6hezzWgzm2Nmc9LT0yNRpojEiFVpexj6wgwK3XnzhoExFfoQgeA3swTgbeC37p4FPA8kA70IfCP4x+G2c/cX3L2Pu/dJSkoKd5kiEiNWbAuEvhlMHt2fTk0rVx/9UIQ1+M2sOoHQn+Du7wC4e5q7F7h7IfAi0C+cNYiIHLRsaxbDXpxB1SrG5NH9Oa5x7IU+hLdXjwFjgGXu/kSR5c2KrHYRsCRcNYiIHLQ0NZNhL84grmoVXr9+AMlJCdEuKWrC2avnZOAqYLGZLQguuxsYZma9AAfWA9eHsQYREZZsyeTKl2ZSO64qk0b3p03D2tEuKarC2avnG+BwY5d+FK59iogcatHmDH790kwSa1Rn0nX9ad0w+lMfRpuu3BWRSmv+xt0MHzuLujUDod+qgUIfItSdU0Qk0uZu2M3wMbOoXyuOyaMV+kXpjF9EKp0563dx9cuzaZgQx6Tr+tO8Xs1ol1Su6IxfRCqVWet2MXzsLJIS43l99ACF/mHojF9EKo0Za3dyzbjZNK1bg0nX9adJJRxLvyzojF9EKoXpq3dw9cuzaF6vJpNHK/RLojN+Eanwpq/ZwTXjZ9O6QS0mXNufpMT4aJdUrpV6xm9mlwaHVcbM7jWzd8ysd/hLExEp3ax1uxg1bg6t6tdi4nUK/VCE0tRzX3BY5UHAOcB4AgOtiYhE1byNuxn58iya1a3BhOtOolGCQj8UoQR/QfDvL4Dn3X0KEBe+kkRESrdocwYjxs6iUWI8E6/rT+NEtemHKpTg32Jm/wEuAz4ys/gQtxMRCYulqZlcNSZwRe7E6/rTtK5C/0iEEuCXAZ8A57p7BtAA+ENYqxIRKcaKbXu4asyswIBr1/WnhfrpH7Fie/WYWYMiD78ssiwXmBPeskREfmr19r1c+dIMqlUxJmrsnaNWUnfOuQSGTj7cCJsOtA9LRSIih7FuRzZXvDgDCIR+20axPbTysSg2+N29XSQLEREpzqZd+7jixRnkFzqTruvPcY1jdxKVshBKP34zs1+b2X3Bx63NTNMlikhEbMnYz9AXZrDvQAGvjTopJufILWuh/Lj7HDAAuCL4eA/wbNgqEhEJ2paZwxUvziArJ4/XRp1E1+Z1ol1SpRDKkA0nuXtvM5sP4O67zUz9+EUkrLbvCYT+zr0HeHVUP7q3rBvtkiqNUM7488ysKoEfdDGzJKAwrFWJSEzbuTeXK1+cybasHF4e2ZcTWtePdkmVSijB/0/gXaCxmT0CfAP8NaxViUjMyth3gCtfmsmm3fsYM6Ivfds2KH0jOSKlNvW4+wQzmwucSaBr54XuvizslYlIzHF3fvfGQtamZzPm6j4MSG4Y7ZIqpVCHZV4FZB1c38xau/vGsFUlIjHppf+3js+Xb+fBC7pxSoekaJdTaZUa/GZ2K3A/kEZgwDYj0N7fI7yliUgsmb9xN49+vJxzuzVl+IA20S6nUgvljP82oJO77wx3MSISmzL35XHLxPk0rVuDRy/pgdnhBgyQshJK8G8CMsNdiIjEJnfnzrcXkpaVw5s3DKBuzerRLqnSCyX41wJfmtmHBAZoA8DdnwhbVSISM175bgOfLE3jnvO6qNtmhIQS/BuDtzg0AYuIlKElWzJ55MNlnNm5MdeeouHBIqXE4A9euJXg7hp/X6QScHfSsnLZkrGP7i3qEVctenMq7cnJ4+aJ82iYEMfjl/ZUu34ElRj87l6gidVFKp7s3HzW7chm7Y5s1qbvZW16Nmt37GVdejbZBwKzqY48uS33/7JbVOpzd/70zmI2797P5NH9qV9bjQmRFEpTzwIzmwq8CWQfXOju74StKhEJibvz3dqdLN+6h7U7ggGfns22rJzv1zGDFvVq0j4pgT5tGpCcVJtZ63czfvp6ftW7Jce3iPwYOJNnb+KDRVv5wzmddGVuFIQS/A2AncAZRZY5oOAXibLnv1rDYx+vACCxRjXaJyUwMLkh7ZNq0z4pgfZJtWnbsDY1qlf90XYX9GrBd2t2cs+7i3nnppOpWiVyzSzLtmbxwNSlnNKhETeelhyx/coPQhmyYWQkChGRI7N6+x6emraKc7o14S8XdqdRQlzI7eR1a1bnvvO7cNvkBUycuYGrBrQNb7FB2bn53DJxHnVqVueJy3pRJYIfOPKDUCZi6Whmn5nZkuDjHmZ2bwjbtTKzL8xsmZktNbPbgssbmNk0M1sV/Kv+WyJHqKDQufOtRdSKr8pfLuxOUmL8Ef84ekHP5gw6rhGPfbyC7XtySt+gDPx5ylLW7sjm6aG9SEqMj8g+5adC+Un/ReBPQB6Auy8ChoawXT5wh7t3AfoDN5tZV+CPwGfu3gH4LPhYRI7AuOnrmbcxgwd+2e2oA9TMePjC48ktKOQvH4R/3MW35m7m7Xmb+c0ZHRiY3Cjs+5PihRL8tdx91iHL8kvbyN23uvu84P09wDKgBTAEGB9cbTxwYejlisiGndn8/ZPlnNm5MUN6NT+m12rXqDY3DU5m6sJUvl6ZXkYV/tTq7Xu4770l9G/fgN+c2SFs+5HQhBL8O8wsmR8mYrkE2HokOzGztsAJwEygibtvhcCHA9C4mG1Gm9kcM5uTnh6+A1KkIiksdO56exHVq1ThkYu6l0nf9xtOS6Zdo9rcN2UJOXkFZVDlj+XkFXDzhPnUiqvK00NPiOgPyXJ4oQT/zcB/gM5mtgX4LXBjqDswswTgbeC37p4V6nbu/oK793H3PklJGp5VBGDS7I3MWLuLe37RhaZ1a5TJa9aoXpWHhxzPhp37eO7LNWXymkU9+P5SVqTt4YnLe9GkTtnULMcmlODf4u5nAUlAZ3cfRGBs/lKZWXUCoT+hSL//NDNrFny+GbD9yMsWiT1bMvbzfx8t5+TjGnJ531Zl+tqDOjTiwl7N+feXa1iTvrfMXvfNOZuYNGsTNw5O5rSOOoErL0IJ/nfMrJq7Z7v7HjNrCkwrbSMLfAcdAyw7ZEC3qcCI4P0RwJQjLVok1rg7d7+zmIJC528Xh2fY4nt+0ZX46lW4770luPsxv96Yb9Zx59uLGNC+IXec3bEMKpSyEkrwvwe8ZWZVg231nxLo5VOak4GrgDPMbEHwdh7wN+BsM1sFnB18LCIleHveFr5amc5d53aiVYNaYdlHUmI8d53bmelrdjJlQepRv05hofPIhyk8/EEK53Rtyssj+1KtavTGBJKfCuUCrhfNLI7AB0Bb4Hp3nx7Cdt8QmK3rcM48kiJFYtn2rBween8pfdrUZ3iYL7S6ol9r3pq7mb98mMLpnRpTt9aRjY2fm1/A799cxPsLUxk+oA33/7Kbfswth4r9GDaz3x28ATWAVsACoH9wmYiEmbtz73tLyM0v5LFLeoT9StcqVYxHLjqeXdkHePST5Ue0bVZOHlePnc37C1O569zOPHiBQr+8Kun7V2KRWwLwLrC6yDIRCbMPF2/l05Q0bj+7I+2TEiKyz27N6zLy5HZMnLmRuRt2h7TNtswcLvv3d8xev4snL+/JjYOTNcxyOVZsU4+7PxjJQkTkx3buzeX+KUvp0bIu1w6K7CQlt5/dkY8Wb+Wedxfzwa2DSmyjX5m2h6vHziIrJ5+XR/bllA7qvVPe6RcXkXLqwfdTyMrJ47FLekT8x9GE+Grc/8tuLN+2h3HT1xe73sy1O7nk+enkFTqvX99foV9BKPhFyqFpKWlMXZjKLad3oHPTOlGp4ZxuTTizc2OemLaS1Iz9P3n+o8VbuWrMLBolxvPOjQPp1jzy4/rL0VHwi5QzmfvzuOfdxXRumsiNg6M3Xr2Z8cAF3Sh054GpS3/03MvfruPmifPo3rIub98wMGxdTCU8SurVU83Mrjezj81skZktNLP/mtkNwStyRSQMHvkwhZ3ZB/j7JT2jOicuQKsGtbjtzI58mpLGtJQ0Cgud//toGQ++n8LZXZow4dqTNG1iBVRSP/5XgQzgAWBzcFlLAlfbvgZcHtbKRGLQ1yvTeWPOZm4cnEz3luWj6eTaU9rx7vzNPDB1KVMXpvL+wlSu6t+GB9Rds8IqKfh7u3unQ5ZtBmaY2cow1iQSk/bm5vOndxbTPqk2t5WjoYurVw2MBHrpv79jS8Z+/nBOJ25Sd80KraTg321mlwJvu3shgJlVAS4FQuvcKyKlcne+WLGdxz9ZSWrmft68fsBP5siNtr5tG/DXi7rToHZ1zj2+WbTLkWNUUvAPBR4FnjOz3QSGX6gHfE5oM3CJSAncnW9W7+Afn65kwaYMWjWoyTPDTqBP2wbRLu2wrjipdbRLkDJS0gVc6wm245tZQ8DcfUeE6hKp1L5bs5Mnp61k1vpdNK9bg/+7uDuXnNiS6hrMTCKgxEHazKwzgakSWwBuZqnAFHc/skE8RASAuRt28Y9PVzJ9zU4aJ8bz0JBuXN63FfHVylfTjlRuxQa/md0FDAMmAwfn3G0JTDazye6u4ZRFQrRgUwZPTlvJVyvTaZQQx33nd+XKk1qXu7Z8iQ0lnfGPArq5e17RhWb2BLAUjaMvUqqlqZk8OW0l/1u2nfq1qvPHn3dm+IA21IordUR0kbAp6egrBJoDGw5Z3iz4nIgUY/X2vTz+yQo+XrqNOjWqccfZHRk5qB0J8Qp8ib6SjsLfAp8FZ8raFFzWGjgOuCXchYlUVJ8tS+PWSfOpYsZvzuzAqEHtqFtTF7tL+VFSr56Pzawj0I/Aj7tG4AKu2e5eEKH6RCqUcd+u46EPUujWvC4vjehDkzo1ol2SyE+U+L0zeOHWjEOXm1mCu+8NW1UiFUxBofPwBymMm76es7s24emhvdSOL+XW0R6ZKQSafURiXnZuPrdNns//lm1n1KB23H1eF41hI+VaSd05i5tX1whMxSgS89Kychg1fjYpqVk8NKRb2CdDFykLJZ3x/xX4O5B/mOd0eaHEvGVbs7hm3Gyy9ucxZkRfTu/cONoliYSkpOCfB7zn7nMPfcLMrg1fSSLl35crtnPLxPkkxFfjjRsGaPYpqVBKCv6RwM5inusThlpEKoQJMzfw5ylL6dQkkTFX96FZ3ZrRLknkiJTUnXNFCc+lhacckfKrsND528fLeeHrtZzRuTH/HHaCLsiSCklHrUgI9h8o4PbXF/Dx0m0MH9CGP5/flWoaSVMqKAW/SCnS9+Ry7StzWLQ5gz+f35WRJ7fV7FNSoZV4ymJmVc3s9kgVI1LeLNmSyYXPfsvKbXv4z69P5JpB7RT6UuGVGPzBoRmGRKgWkXLljdmbuPj56bg7r1/fn591axrtkkTKRChNPd+a2b+A14HsgwvdfV7YqhKJopy8Ah58fymTZm1i0HGN+OewE2hQOy7aZYmUmVCCf2Dw70NFljlwRtmXIxJdm3fv48bX5rF4SyY3n57M787upOEXpNIpNfjd/fRIFCISbV+tTOe2yfMpKHBeHN6Hs7s2iXZJImFRan80M6trZk+Y2Zzg7R9mVuplimY21sy2m9mSIsseMLMtZrYgeDvvWP8DRI5VYaHzzGeruPrlWTStU4Optw5S6EulFkpH5LHAHuCy4C0LeDmE7cYB5x5m+ZPu3it4+yjUQkXCIXN/Hte9Mod/TFvJkJ7NeeemgbRrVDvaZYmEVSht/Mnu/qsijx80swWlbeTuX5tZ26MtTCTcUlKzuOG1uWzN3M9DQ7pxVf826qopMSGUM/79Zjbo4AMzOxnYfwz7vMXMFgWbguoXt5KZjT7YvJSenn4MuxP5qbfnbuai574lN7+AyaMHMHyALsqS2GHuXvIKZj2BV4CD7fq7gRHuvqjUFw+c8X/g7scHHzcBdhDoFfQw0Mzdryntdfr06eNz5swpbTWRUuXmF/DwBym8NmMj/ds34JlhvUlKjI92WSJhYWZz3f0ng2qWNBHLbe7+NJDg7j3NrA6Au2cdbRFFB3czsxeBD472tUSO1K7sA1wzbjYLNmVw/ant+cM5nTTejsSkko76kcG/z0Ag8I8l9AHMrFmRhxcBS4pbV6Qs5eQVcN0rc0jZmsXzV/bmT+d1UehLzCrpx91lZrYeSDKzos06Bri79yjphc1sEjAYaGRmm4H7gcFm1otAU8964PqjL10kNIWFzh1vLmTuht08e0Vvft69WekbiVRiJY3HP8zMmgKfABcc6Qu7+7DDLB5zpK8jcqz+/ukKPly0lT/+vDO/6KHQFymxO6e7bwN6RqgWkTI3adZGnv9yDcP6teb6U9tHuxyRckGNnFJpfb0ynXvfW8KpHZN4eEg3ddcUCVLwS6W0fFsWN02YR4fGCTx7xQn6IVekiJD/NZiZrmOXCiEtK4drXp5N7fiqvDyyL4k1qke7JJFyJZRB2gaaWQqwLPi4p5k9F/bKRI5Cdm4+o8bPJmN/HmNG9KVZ3ZrRLkmk3AnljP9J4BxgJ4C7LwRODWdRIkejoND5zaT5pKRm8ewVvTm+RamDyIrEpJCaetx90yGLCsJQi8hRc3ceen8pny3fzoMXdOP0zo2jXZJIuRXK6JybzGwg4GYWB/yGYLOPSHkx9tv1jP9uA9cOasdVA9pGuxyRci2UM/4bgJuBFsBmoFfwsUi58MnSbfzlwxTO6daEu8/rEu1yRMq9UKZe3AFcGYFaRI7Ywk0Z3DZ5Pj1a1uOpy0+giubHFSlVqcFvZu2AW4G2Rdd39yMexkGkLG3atY9R4+fQKCGel4b3oWZc1WiXJFIhhNLG/x6BMXbeBwrDW45IaHYHh1g+kF/A5NEnaUx9kSMQSvDnuPs/w16JlCvbMnN4dcZ6WjeoxcDkRrRqUCvaJQGwY28uY79Zx6vfbSAnv4Dx1/TjuMaJ0S5LpEIJJfifNrP7gU+B3IML3X1e2KqSqHF3pixI5c9TlpCVk//98pb1azIwuSEDkxsxILkhTerUiGhdqRn7eeHrtUyevZHc/ELOO74ZN59+HF2b14loHSKVQSjB3x24CjiDH5p6PPhYKpGde3O5970l/HfJNk5sU5/HL+1JQWEh09fsZPrqnXyyNI035mwG4LjGCcEPgob0b9+QerXiwlLT2vS9/PurNbw7fwvucNEJLbhhcDLJSQlh2Z9ILAhlzt3lQA93PxCZkn5Kc+6G36dLt3H3u4vJ2p/P7Wd3ZPSp7al6SA+ZgkJn2dYspq/ZwfQ1O5m1bhf7DhRgBl2b1fn+G0GPlnVpmHBsbe4pqVk89+VqPlq8lepVqzCsX2uuO7U9LeppCAaRUBU3524owf86cKu7bw9XcaVR8IdPVk4eD05N4e15m+narA5PXN6Tzk1Daz7JKyhk0eYMvl29k+lrdjBvQwYHCgJfChvWjqNDkwQ6NkmkQ5NEOjYO3K9fu+RvBnM37ObZL1bz+fLtJMRX46oBbbjm5Hb68VbkKBxL8H8J9ABm8+M2/oh151Twh8e3q3fwhzcXkrYnl5sGJ3PrGR2Iq3b0wxfn5BUwb8NuUrZmsTJtD6u272VV2l725v7wW0GjhHg6fv+BEPjbsXEii7Zk8OwXq5mxdhcNasdxzcltuWpAW+rW1MiaIkeruOAPpY3//jDUI1G070A+f/vvcl75bgPtk2rz9o0D6dWq3jG/bo3qVRl4XCMGHtfo+2XuztbMnMAHQdpeVqbtYeX2vbw5ZxPZB3485FPTOjX48/ldGdqvFbXiQjk0ReRohHLl7leRKEQiY+6G3dzxxgLW79zHNSe3485zO1GjevgufDIzmterSfN6NRnc6YeB09ydLRn7g98K9tCgdjy/7NmM+Gq6CEsk3IoNfjP7xt0HmdkeAr14vn8KcHdXP7oKJDe/gKf+t4r/fLWGZnVrMum6/gxIbhi1esyMlvVr0bJ+LU7vpJE0RSKppDP+2gDurqtjKrj1O7K54bW5LN+2h6F9W3HPL7poViqRGFZS8Jf8q69UCOl7crlq7Ez25uQz9uo+nNG5SbRLEpEoKyn4G5vZ74p70t2fCEM9Uoayc/O5Ztxsduw5wOTR/elZBj/gikjFV1LwVwUSCLTpSwWTV1DITRPmkbI1ixeHn6jQF5HvlRT8W939oYhVImXG3bn7ncV8tTKdv13cXc07IvIjJV2tozP9CurJ/63izbmb+c2ZHRjar3W0yxGRcqak4D8zYlVImZk0ayP//GwVl/Vpye1ndYh2OSJSDhUb/O6+K5KFyLH7fHka9763hNM6JvHIRd0x05c2Efmpox+YRcqVhZsyuHnCfLo2q8NzV/amelX9rxWRw1M6VALrd2RzzbjZNEqMY+zVfakdr3FuRKR4YQt+MxtrZtvNbEmRZQ3MbJqZrQr+rR+u/ceKHXtzGfHyLArdGT+yn4YvFpFShfOMfxxw7iHL/gh85u4dgM+Cj+Uo7TuQz6hxs0nLymHM1X1pr1mpRCQEYQt+d/8aOPQH4iHA+OD98cCF4dp/ZZdfUMgtE+ezeEsmzwzrTe/W+vIkIqGJdBt/E3ffChD8q2EZj4K7c9+UJXy+fDsPDTmes7vqAi0RCV25/XHXzEab2Rwzm5Oenh7tcsqVZz5fzaRZm7j59GR+3b9NtMsRkQom0sGfZmbNAIJ/i53H191fcPc+7t4nKSkpYgWWdxNnbuSJaSu5uHcLfv+zTtEuR0QqoEgH/1RgRPD+CGBKhPdfYbk7z3+5hrvfXczgTkn87eIeukBLRI5K2Dp8m9kkYDDQyMw2E5i792/AG2Y2CtgIXBqu/VcmhYXOIx8tY8w367igZ3Mev7TnMU2KLiKxLWzB7+7DinlKYwAdgQP5hfzhrYVMWZDK1QPb8ufzu1Klis70ReTo6RLPciw7N58bJ8zj65Xp3HluJ248LVnNOyJyzBT85dTOvblcM242i7dk8tivenBZ31bRLklEKgkFfzm0efc+ho+ZxZaM/fznqj7qpy8iZUrBX86s2LaH4WNnsv9AAa+OOol+7RpEuyQRqWQU/OXI7PW7GDVuNjXjqvLGDQPo3LROtEsSkUpIwV9OTEtJ45aJ82hRvyavXNOPlvVrRbskEamkFPzlwOuzN/KndxbTvWU9Xr66Lw1qx0W7JBGpxD9jXdUAAAoaSURBVBT8UeTuPPflGv7+yQpO7ZjE81f21iQqIhJ2SpkoSM3Yz9wNu5mWksbUhakM6dWcv1+iq3FFJDIU/GGWV1DIsq1ZzN2w+/vb1swcAGpWr8oNpyVz5zmddDWuiESMgr+MZew7wLyNgYCfs343CzdnkJNXCEDzujU4sU19+rSpz4ltGtC5WaImRReRiFPwl4HM/Xn849MVfLt6B2vSswGoWsXo1rwOQ/u2pk/b+vRuXZ/m9WpGuVIREQX/MVu3I5tR42ezcec+Tu2YxMW9W3Jim/r0aFmXWnF6e0Wk/FEyHYNvVu3g5onzqGIw4dqTOKl9w2iXJCJSKgX/UXr1u/U88H4KyUm1GTOiL60a6IIrEakYFPxHKK+gkAffX8prMzZyZufGPDW0F4k1qke7LBGRkCn4j0DGvgPcNGEe09fs5PrT2nPnOZ2pqm6YIlLBKPhDtHr7Hq4dP4fUjBz+cWlPfnViy2iXJCJyVBT8IfhyxXZunTif+OpVmDT6JE5so6GSRaTiUvCXwN0Z++16HvkwhU5N6/DSiD60UF98EangFPzFOJBfyJ+nLGHy7E2c060JT1zWSwOoiUiloCQ7jF3ZB7jhtbnMWreLW884jtvP6qixdESk0lDwH2JpaibXvzqX9D25PD20F0N6tYh2SSIiZUrBH+TuvDpjA3/5cBn1a1Xn9esH0KtVvWiXJSJS5hT8BAZZu+utRXy8dBund0ri8Ut70jAhPtpliYiERcwH//yNu7l10ny2ZeZwz3ldGDWondrzRaRSi9ngLyx0XvpmLY99vIKmdWvw5g0DOKF1/WiXJSISdjEZ/LuyD3DHGwv4YkU653ZryqOX9KBuTY23IyKxIeaCf8bandw2eT67s/N4eEg3ft2/DWZq2hGR2BEzwV9Q6Dz7xWqe+t9K2jSszdir+9Kted1olyUiEnExEfzbs3L47esLmL5mJxf2as5fLupOgq7CFZEYVenT7+uV6dz++gL2HSjgsUt6cOmJLdW0IyIxLSrBb2brgT1AAZDv7n3CsZ9/fb6Kxz9dSacmifzrihPo0CQxHLsREalQonnGf7q77wjnDto2qs3Qvq24/5fdqBlXNZy7EhGpMCp1U8/5PZpzfo/m0S5DRKRcqRKl/TrwqZnNNbPRUapBRCQmReuM/2R3TzWzxsA0M1vu7l8XXSH4gTAaoHXr1tGoUUSkUorKGb+7pwb/bgfeBfodZp0X3L2Pu/dJSkqKdIkiIpVWxIPfzGqbWeLB+8DPgCWRrkNEJFZFo6mnCfBusC99NWCiu38chTpERGJSxIPf3dcCPSO9XxERCYhWrx4REYkSBb+ISIwxd492DaUys3RgQ7TriIBGQFivZq4g9D78QO9FgN6HgCN9H9q4+0+6RVaI4I8VZjYnXOMWVSR6H36g9yJA70NAWb0PauoREYkxCn4RkRij4C9fXoh2AeWE3ocf6L0I0PsQUCbvg9r4RURijM74RURijIJfRCTGKPijxMxamdkXZrbMzJaa2W3B5Q3MbJqZrQr+rR/tWiPBzKqa2Xwz+yD4uJ2ZzQy+D6+bWVy0aww3M6tnZm+Z2fLgcTEgFo8HM7s9+G9iiZlNMrMasXI8mNlYM9tuZkuKLDvsMWAB/zSz1Wa2yMx6h7ofBX/05AN3uHsXoD9ws5l1Bf4IfObuHYDPgo9jwW3AsiKPHwWeDL4Pu4FRUakqsp4GPnb3zgTGs1pGjB0PZtYC+A3Qx92PB6oCQ4md42EccO4hy4o7Bn4OdAjeRgPPh7oTBX+UuPtWd58XvL+HwD/yFsAQYHxwtfHAhdGpMHLMrCXwC+Cl4GMDzgDeCq5S6d8HM6sDnAqMAXD3A+6eQQweDwQGj6xpZtWAWsBWYuR4CE5IteuQxcUdA0OAVzxgBlDPzJqFsh8FfzlgZm2BE4CZQBN33wqBDwegcfQqi5ingDuBwuDjhkCGu+cHH28m8KFYmbUH0oGXg01eLwXnq4ip48HdtwCPAxsJBH4mMJfYOx6KKu4YaAFsKrJeyO+Lgj/KzCwBeBv4rbtnRbueSDOz84Ht7j636OLDrFrZ+x1XA3oDz7v7CUA2lbxZ53CC7ddDgHZAc6A2gSaNQ1X24yEUR/3vRMEfRWZWnUDoT3D3d4KL0w5+XQv+3R6t+iLkZOACM1sPTCbwlf4pAl9bD84X0RJIjU55EbMZ2OzuM4OP3yLwQRBrx8NZwDp3T3f3POAdYCCxdzwUVdwxsBloVWS9kN8XBX+UBNuxxwDL3P2JIk9NBUYE748ApkS6tkhy9z+5e0t3b0vgR7zP3f1K4AvgkuBqsfA+bAM2mVmn4KIzgRRi7Hgg0MTT38xqBf+NHHwfYup4OERxx8BUYHiwd09/IPNgk1BpdOVulJjZIOD/AYv5oW37bgLt/G8ArQn8I7jU3Q/9sadSMrPBwO/d/Xwza0/gG0ADYD7wa3fPjWZ94WZmvQj8wB0HrAVGEjg5i6njwcweBC4n0PNtPnAtgbbrSn88mNkkYDCB4ZfTgPuB9zjMMRD8YPwXgV5A+4CR7j4npP0o+EVEYouaekREYoyCX0Qkxij4RURijIJfRCTGKPhFRGKMgl8qJTPbW+T+ecGRDVuHeZ9fmtlPJsI2s/ODwzAsNLMUM7s+uPwGMxsezppEDqda6auIVFxmdibwDPAzd98Y4jbViowLc6z7r05gurx+7r7ZzOKBtgDu/u+y2IfIkVLwS6VlZqcALwLnufua4LIk4N8ELoaBwBhJ35rZAwTGhmkL7DCzT4ELCIwOmQy86+53Bl/jZ8CDQDywhsCFM99/wzhEIoF/ZzsBghcdrQi+zgPAXmAi8FGRbboTGLRt3+FqPbp3Q+QHCn6prOIJXNo+2N2XF1n+NIFx3b8JNv18AnQJPnciMMjd95vZ1UAvAqOm5gIrzOwZYD9wL3CWu2eb2V3A74CHDldE8ArLqcAGM/sM+ACY5O6FRdZJDe4LM7sZOM3dN5jZxBJqFTlqCn6prPKA6QQm7LityPKzgK6Bq90BqGNmicH7U919f5F1P3P3TAAzSwHaAPWArsC3wdeIA74rqRB3v9bMugf3/XvgbODqQ9czs5MJDE9wSkm1BudvEDlqCn6prAqBy4D/mdnd7v7X4PIqwIBDAp5guGYf8hpFx4IpIPDvxYBp7j7sSIpx98XAYjN7FVjHIcEfHHVxDHBBkWajw9YqcqzUq0cqLXffB5wPXGlmB6fq+xS45eA6wYHRjsQM4GQzOy64fS0z61jcymaWEBx87qBewIZD1qlOYBCuu9x9ZZGnjrVWkcNS8EulFhzJ8lzgXjMbQnA+1+Dk1CnADUf4eukEztYnmdkiAh8EnUvYxIA7zWyFmS0g8KPw1YesMxDoCzxoZguCt+bHWqtIcTQ6p4hIjNEZv4hIjFHwi4jEGAW/iEiMUfCLiMQYBb+ISIxR8IuIxBgFv4hIjPn/RQ7U+RchXeMAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.xlabel(\"Kernel Size\")\n",
    "plt.ylabel(\"Time for 100 kernels\")\n",
    "plt.plot(np.arange(7,100,4), times_ks);"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 3x3 kernel is twice as fast as 9x9, that means we can double the kernels and run in same time, but it still has worse results"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "times_ks3, scores_ks3 = timing_test(10, np.array((2,)), stride=7, num_kernels=200, seq_length=32000, show_progress=False)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.935"
      ]
     },
     "execution_count": null,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "mn(scores_ks3)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Drop down to stride 1 conv, run 2500 kernels for filter size 7,9,11 and {7,9,11} to see if variety helps"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "times_7, scores_7 = timing_test(1, np.array((7,)), stride=1, num_kernels=2500, seq_length=32000, show_progress=False)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "times_9, scores_9 = timing_test(1, np.array((9,)), stride=1, num_kernels=2500, seq_length=32000, show_progress=False)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "times_11, scores_11 = timing_test(1, np.array((11,)), stride=1, num_kernels=2500, seq_length=32000, show_progress=False)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "times_orig, scores_orig = timing_test(1, np.array((7,9,11,)), stride=1, num_kernels=2500, seq_length=32000, show_progress=False)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(0.996, 0.996, 1.0, 0.995)"
      ]
     },
     "execution_count": null,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "mn(scores_7), mn(scores_9), mn(scores_11), mn(scores_orig)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Conclusion: Doesnt appear to help but needs to be retested on a tougher dataset"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "([600.959691286087],\n",
       " [723.5468919277191],\n",
       " [877.7471699714661],\n",
       " [831.6202943325043])"
      ]
     },
     "execution_count": null,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "times_7, times_9, times_11, times_orig"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## What is the predictive power of a single kernel?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Finished Run 1 Score: 0.243 Time: 0.101\n",
      "Finished Run 2 Score: 0.212 Time: 0.141\n",
      "Finished Run 3 Score: 0.226 Time: 0.161\n",
      "Finished Run 4 Score: 0.224 Time: 0.194\n",
      "Finished Run 5 Score: 0.211 Time: 0.107\n",
      "Finished Run 6 Score: 0.182 Time: 0.122\n",
      "Finished Run 7 Score: 0.257 Time: 0.189\n",
      "Finished Run 8 Score: 0.28 Time: 0.167\n",
      "Finished Run 9 Score: 0.19 Time: 0.146\n",
      "Finished Run 10 Score: 0.164 Time: 0.122\n",
      "Finished Run 11 Score: 0.186 Time: 0.172\n",
      "Finished Run 12 Score: 0.289 Time: 0.225\n",
      "Finished Run 13 Score: 0.251 Time: 0.226\n",
      "Finished Run 14 Score: 0.139 Time: 0.057\n",
      "Finished Run 15 Score: 0.274 Time: 0.157\n",
      "Finished Run 16 Score: 0.211 Time: 0.157\n",
      "Finished Run 17 Score: 0.213 Time: 0.21\n",
      "Finished Run 18 Score: 0.231 Time: 0.197\n",
      "Finished Run 19 Score: 0.226 Time: 0.108\n",
      "Finished Run 20 Score: 0.226 Time: 0.233\n"
     ]
    }
   ],
   "source": [
    "times, scores = timing_test(20, np.array((7,9,11)), stride=5, num_kernels=1, seq_length=32000)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.22178153446033813"
      ]
     },
     "execution_count": null,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "sum(scores)/len(scores)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Answer: about 22% on a 10-class probelm"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Can we apply kernels to a small subset of our data and get enough data to see which ones will be predictive? if so we could generate 100x as many random kernels for a large dataset, select the best 1% and use those on full data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def get_good_kernels(runs, candidate_lengths, stride, num_kernels, seq_length, thresh, subset_size=300, show_progress=True):\n",
    "    good_kernels, scores = [], []\n",
    "    for i in range(runs):\n",
    "        candidate_lengths = np.array((7,))\n",
    "        kernels = generate_kernels(seq_length, num_kernels, candidate_lengths, stride)\n",
    "        idxs = torch.randperm(len(np_x_train))[:subset_size]\n",
    "        #print(idxs)\n",
    "        np_x_train_subset = np_x_train[idxs]\n",
    "        np_y_train_subset = np_y_train[idxs]\n",
    "        x_train_tfm = apply_kernels(np_x_train_subset, kernels)\n",
    "        x_valid_tfm = apply_kernels(np_x_valid, kernels)\n",
    "        classifier = RidgeClassifierCV(alphas=np.logspace(-3, 3, 7), normalize=True)\n",
    "        classifier.fit(x_train_tfm, np_y_train_subset)\n",
    "        score = classifier.score(x_valid_tfm, np_y_valid)\n",
    "        if score > thresh:\n",
    "            good_kernels.append(kernels)\n",
    "        scores.append(score)\n",
    "        if(show_progress): print(\"Finished Run\", i+1, \"Score:\", round(score, 3))\n",
    "    return good_kernels, scores"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "k, s = get_good_kernels(2000, np.array((7,)), stride=5, num_kernels=1,seq_length=32000, thresh=0.275, show_progress=False)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def merge_kernels(k):\n",
    "    num_kernels = len(k)\n",
    "    strides = np.zeros(num_kernels, dtype = np.int32)\n",
    "    weights = np.zeros((num_kernels, 7)) # see note\n",
    "    lengths = np.zeros(num_kernels, dtype = np.int32) # see note\n",
    "    biases = np.zeros(num_kernels)\n",
    "    dilations = np.zeros(num_kernels, dtype = np.int32)\n",
    "    paddings = np.zeros(num_kernels, dtype = np.int32)\n",
    "    for i in range(num_kernels):\n",
    "        #weights, lengths, biases, dilations, paddings, strides\n",
    "        weights[i], lengths[i], biases[i], dilations[i], paddings[i], strides[i] = k[i]\n",
    "    return weights, lengths, biases, dilations, paddings, strides"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "165"
      ]
     },
     "execution_count": null,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "len(k)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "kernels = merge_kernels(k)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Finished Run 1 Score: 0.974 Time: 9.498\n",
      "Finished Run 2 Score: 0.978 Time: 9.682\n",
      "Finished Run 3 Score: 0.974 Time: 10.002\n",
      "Finished Run 4 Score: 0.979 Time: 9.684\n",
      "Finished Run 5 Score: 0.98 Time: 9.885\n",
      "Finished Run 6 Score: 0.979 Time: 9.818\n",
      "Finished Run 7 Score: 0.979 Time: 9.717\n",
      "Finished Run 8 Score: 0.978 Time: 9.658\n",
      "Finished Run 9 Score: 0.977 Time: 9.609\n",
      "Finished Run 10 Score: 0.978 Time: 9.866\n"
     ]
    }
   ],
   "source": [
    "times, scores = timing_test(10, np.array((7,)), stride=5, num_kernels=168, seq_length=32000, show_progress=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def score_kernels(k):\n",
    "    x_train_tfm = apply_kernels(np_x_train, k)\n",
    "    x_valid_tfm = apply_kernels(np_x_valid, k)\n",
    "    classifier = RidgeClassifierCV(alphas=np.logspace(-3, 3, 7), normalize=True)\n",
    "    classifier.fit(x_train_tfm, np_y_train)\n",
    "    score = classifier.score(x_valid_tfm, np_y_valid)\n",
    "    return(score)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.9687906371911573"
      ]
     },
     "execution_count": null,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "score_kernels(kernels)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Result is that 168 \"high accuracy\" kernels drawn from 2000 random kernels actually performs worse than just 168 random kernels! The correlation between the high accuracy kernels must be higher than among the random ones, so they are picking out the same features more frequently"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(0.4967490247074122, 0.5357607282184655, 0.52)"
      ]
     },
     "execution_count": null,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "min(s), max(s), mn(s)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}