{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "# Sampling" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "In this first exercise, we will investigate how to evaluate the Q-value of each action available in a 5-armed bandit. It is mostly to give you intuition about the limits of sampling and the central limit theorem.\n", "\n", "Let's start with importing numpy and matplotlib:" ] }, { "cell_type": "code", "execution_count": 1, "metadata": {}, "outputs": [], "source": [ "import numpy as np\n", "import matplotlib.pyplot as plt " ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Sampling a n-armed bandit\n", "\n", "Let's now create the n-armed bandit. The only thing we need to do is to randomly choose 5 true Q-values $Q^*(a)$.\n", "\n", "![](../img/bandit-example.png)\n", "\n", "To be generic, let's define `nb_actions=5` and create an array corresponding to the index of each action (0, 1, 2, 3, 4) for plotting purpose." ] }, { "cell_type": "code", "execution_count": 2, "metadata": {}, "outputs": [], "source": [ "nb_actions = 5\n", "actions = np.arange(nb_actions)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "**Q:** Create a numpy array `Q_star` with `nb_actions` values, normally distributed with a mean of 0 and standard deviation of 1 (as in the lecture). " ] }, { "cell_type": "code", "execution_count": 3, "metadata": {}, "outputs": [], "source": [ "rng = np.random.default_rng()\n", "Q_star = rng.normal(0, 1, nb_actions)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "**Q:** Plot the Q-values. Identify the optimal action $a^*$.\n", "\n", "*Tip:* you could plot the array `Q_star` with `plt.plot`, but that would be ugly. Check the documentation of the `plt.bar` method." ] }, { "cell_type": "code", "execution_count": 4, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Optimal action: 2\n" ] }, { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAA18AAAINCAYAAADBd8UhAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjkuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8hTgPZAAAACXBIWXMAAA9hAAAPYQGoP6dpAAAl0ElEQVR4nO3dfbDWdZ3/8ddB4oAmOMgRJFDEGPEOUVQWbROFBGVMyrxp2VGRzBxtJKiEpoVVc7G00k1DXRV1zWzd1EVNW8O7VimQOlnekFYsrHhQS88BWrk9vz8cz2/ZgIXD4XNxHR6Pme8M1/fmut7nzDXAc77X93vVNDc3NwcAAIDtqkOlBwAAANgZiC8AAIACxBcAAEAB4gsAAKAA8QUAAFCA+AIAAChAfAEAABQgvgAAAAroWOkBqtX69euzdOnS7L777qmpqan0OAAAQIU0Nzdn+fLl6d27dzp02PT5LfHVSkuXLk3fvn0rPQYAALCDWLJkSfr06bPJ7eKrlXbfffck7/2Cu3btWuFpAACASmlqakrfvn1bGmFTxFcrvf9Rw65du4ovAADg/7wcyQ03AAAAChBfAAAABYgvAACAAsQXAABAAeILAACgAPEFAABQgPgCAAAoQHwBAAAUIL4AAAAKEF8AAAAFiC8AAIACxBcAAEAB4gsAAKAA8QUAAFCA+AIAAChAfAEAABQgvgAAAAoQXwAAAAV0rPQAALQ//aY8XOkR2A4WXTWm0iMAVDVnvgAAAAoQXwAAAAWILwAAgALEFwAAQAHiCwAAoADxBQAAUID4AgAAKEB8AQAAFCC+AAAAChBfAAAABYgvAACAAsQXAABAAeILAACgAPEFAABQgPgCAAAoQHwBAAAUIL4AAAAKEF8AAAAFdKz0AMCOpd+Uhys9Am1s0VVjKj0CABBnvgAAAIoQXwAAAAWILwAAgALEFwAAQAHiCwAAoADxBQAAUID4AgAAKEB8AQAAFCC+AAAAChBfAAAABYgvAACAAsQXAABAAeILAACgAPEFAABQgPgCAAAoQHwBAAAUIL4AAAAKEF8AAAAFiC8AAIACxBcAAEAB7Sa+brjhhvTr1y+dO3fO0KFDM2/evM3uf++992bgwIHp3LlzDj300PzoRz8qNCkAALAzahfx9YMf/CCTJk3K9OnT84tf/CKHHXZYRo0alTfeeGOj+z/77LP59Kc/nQkTJuSXv/xlxo4dm7Fjx+Y3v/lN4ckBAICdRbuIr29961s5//zzM378+Bx00EG58cYbs+uuu+a2227b6P7XXXddRo8enS996Us58MADc8UVV+SII47I9ddfX3hyAABgZ1H18bV69eosWLAgI0eObFnXoUOHjBw5MnPnzt3oMXPnzt1g/yQZNWrUJvdPklWrVqWpqWmDBQAAYEt1rPQA2+qtt97KunXr0rNnzw3W9+zZMy+//PJGj2loaNjo/g0NDZt8nRkzZuSyyy7b9oG3k35THq70CLSxRVeN2alel/bF+4i24t+39qdSfz94L7U/1fhvTdWf+Spl6tSpaWxsbFmWLFlS6ZEAAIAqUvVnvnr06JFddtkly5Yt22D9smXL0qtXr40e06tXr63aP0lqa2tTW1u77QMDAAA7pao/89WpU6cMGTIkc+bMaVm3fv36zJkzJ8OGDdvoMcOGDdtg/yR57LHHNrk/AADAtqr6M19JMmnSpJxzzjk58sgjc/TRR+faa6/NypUrM378+CTJ2WefnQ996EOZMWNGkuSSSy7Jcccdl29+85sZM2ZM7rnnnjz33HO5+eabK/ljAAAA7Vi7iK8zzzwzb775ZqZNm5aGhoYMHjw4jz76aMtNNRYvXpwOHf7/Sb5jjjkmd999d7761a/mK1/5SgYMGJAHHngghxxySKV+BAAAoJ1rF/GVJBdffHEuvvjijW578skn/2Ld6aefntNPP307TwUAAPCeqr/mCwAAoBqILwAAgALEFwAAQAHiCwAAoADxBQAAUID4AgAAKEB8AQAAFCC+AAAAChBfAAAABYgvAACAAsQXAABAAeILAACgAPEFAABQgPgCAAAoQHwBAAAUIL4AAAAKEF8AAAAFiC8AAIACxBcAAEAB4gsAAKAA8QUAAFCA+AIAAChAfAEAABQgvgAAAAoQXwAAAAWILwAAgALEFwAAQAHiCwAAoADxBQAAUID4AgAAKEB8AQAAFCC+AAAAChBfAAAABYgvAACAAsQXAABAAeILAACgAPEFAABQgPgCAAAoQHwBAAAUIL4AAAAKEF8AAAAFiC8AAIACxBcAAEAB4gsAAKAA8QUAAFCA+AIAAChAfAEAABQgvgAAAAoQXwAAAAWILwAAgALEFwAAQAHiCwAAoADxBQAAUID4AgAAKEB8AQAAFCC+AAAAChBfAAAABYgvAACAAsQXAABAAeILAACgAPEFAABQgPgCAAAoQHwBAAAUIL4AAAAKEF8AAAAFiC8AAIACxBcAAEAB4gsAAKAA8QUAAFCA+AIAAChAfAEAABQgvgAAAAoQXwAAAAWILwAAgALEFwAAQAHiCwAAoADxBQAAUID4AgAAKEB8AQAAFCC+AAAAChBfAAAABYgvAACAAsQXAABAAeILAACgAPEFAABQQNXH15/+9KeMGzcuXbt2zR577JEJEyZkxYoVmz1m+PDhqamp2WD53Oc+V2hiAABgZ9Sx0gNsq3HjxuX111/PY489ljVr1mT8+PH57Gc/m7vvvnuzx51//vm5/PLLWx7vuuuu23tUAABgJ1bV8fXSSy/l0Ucfzfz583PkkUcmSb7zne/k5JNPzjXXXJPevXtv8thdd901vXr1KjUqAACwk6vqjx3OnTs3e+yxR0t4JcnIkSPToUOH/PznP9/ssd/73vfSo0ePHHLIIZk6dWr+/Oc/b+9xAQCAnVhVn/lqaGjIXnvttcG6jh07pnv37mloaNjkcX/zN3+TfffdN717987zzz+fSy+9NAsXLsx99923yWNWrVqVVatWtTxuamra9h8AAADYaeyQ8TVlypR8/etf3+w+L730Uquf/7Of/WzLnw899NDsvffeGTFiRH73u99l//333+gxM2bMyGWXXdbq1wQAAHZuO2R8TZ48Oeeee+5m9+nfv3969eqVN954Y4P1a9euzZ/+9Ketup5r6NChSZJXX311k/E1derUTJo0qeVxU1NT+vbtu8WvAQAA7Nx2yPiqq6tLXV3d/7nfsGHD8s4772TBggUZMmRIkuTxxx/P+vXrW4JqS9TX1ydJ9t57703uU1tbm9ra2i1+TgAAgP+pqm+4ceCBB2b06NE5//zzM2/evDzzzDO5+OKLc9ZZZ7Xc6fC1117LwIEDM2/evCTJ7373u1xxxRVZsGBBFi1alNmzZ+fss8/ORz/60QwaNKiSPw4AANCOVXV8Je/dtXDgwIEZMWJETj755HzkIx/JzTff3LJ9zZo1WbhwYcvdDDt16pSf/OQnOfHEEzNw4MBMnjw5p512Wh588MFK/QgAAMBOYIf82OHW6N69+2a/ULlfv35pbm5uedy3b9889dRTJUYDAABoUfVnvgAAAKqB+AIAAChAfAEAABQgvgAAAAoQXwAAAAVU/d0OAYD2a9FVYyo9AkCbceYLAACgAPEFAABQgPgCAAAoQHwBAAAUIL4AAAAKEF8AAAAFiC8AAIACxBcAAEAB4gsAAKAA8QUAAFCA+AIAAChAfAEAABQgvgAAAAoQXwAAAAWILwAAgALEFwAAQAHiCwAAoADxBQAAUID4AgAAKEB8AQAAFCC+AAAAChBfAAAABYgvAACAAsQXAABAAeILAACgAPEFAABQgPgCAAAoQHwBAAAUIL4AAAAKEF8AAAAFiC8AAIACxBcAAEAB4gsAAKAA8QUAAFCA+AIAAChAfAEAABQgvgAAAAoQXwAAAAV0bIsnWbNmTRoaGvLnP/85dXV16d69e1s8LQAAQLvR6jNfy5cvz8yZM3Pcccela9eu6devXw488MDU1dVl3333zfnnn5/58+e35awAAABVq1Xx9a1vfSv9+vXLrFmzMnLkyDzwwAOpr6/Pb3/728ydOzfTp0/P2rVrc+KJJ2b06NF55ZVX2npuAACAqtKqjx3Onz8/Tz/9dA4++OCNbj/66KNz3nnn5cYbb8ysWbPy05/+NAMGDNimQQEAAKpZq+Lr+9///hbtV1tbm8997nOteQkAAIB2ZZtvuLFu3brccsstWbhwYfr06ZPDDjssgwcPzp577tkW8wEAALQL2xxfn//85/PDH/4wI0eOzPXXX5+ampqsXbs2H/rQhzJ48ODMnj27LeYEAACoatv8PV/33Xdf7rzzznzve99LbW1tnnvuuVx33XV59913s++++7bFjAAAAFVvm898rVixIgcddFCS5AMf+EA6duyYiy++OGvWrMnSpUu3eUAAAID2YJvPfPXv378lsj70oQ/ltddeS5Kccsopueuuu7b16QEAANqFbY6vT37yk3nkkUeSJMcdd1xuu+22JMmLL76Y//7v/97WpwcAAGgXtvljh3//93/f8ucvf/nLOeqoo1JXV5empqZMmDBhW58eAACgXdjm+Pqf9tlnn7zwwgt58MEHs+eee+aUU05py6cHAACoWm0aX0nSo0ePjB8/vq2fFgAAoKq16pqvxYsXb9X+79+EAwAAYGfVqvg66qijcsEFF2T+/Pmb3KexsTH/9E//lEMOOSQ//OEPWz0gAABAe9Cqjx2++OKLufLKK/Oxj30snTt3zpAhQ9K7d+907tw5b7/9dl588cW88MILOeKII/KNb3wjJ598clvPDQAAUFVadeZrzz33zLe+9a28/vrruf766zNgwIC89dZbeeWVV5Ik48aNy4IFCzJ37lzhBQAAkG284UaXLl3yqU99Kp/61Kfaah4AAIB2aZu/ZBkAAID/m/gCAAAoYKvj66c//WmS5JlnnmnzYQAAANqrrY6vRx55JHPnzs3DDz+8PeYBAABol7Yqvi677LKsXbs2J5xwQtatW5fLL798e80FAADQrmxVfE2fPj0DBgzIFVdckQEDBmTatGnbay4AAIB2Zas/drh27dp88YtfzLp167bHPAAAAO3SVsfXhRdemCS54IIL2nwYAACA9sqt5gEAAArouK1P8Pvf/z6/+tWv0rFjxwwaNCj77rtvW8wFAADQrrQ6vtatW5fPfOYzufPOO9Pc3JwkqampyXHHHZfrrrsuhx56aJsNCQAAUO1a/bHDf/iHf8js2bNz0003ZeHChamvr88tt9yS5cuX59hjj83jjz/elnMCAABUtVaf+brjjjvy7W9/O2effXbLukGDBmX8+PG55pprMnbs2Lzyyivp0qVLfvnLX+a4445rk4EBAACqUavPfC1ZsiR//dd/vdFtX/ziF3PmmWfmvPPOy5AhQ/Lss8+2ekAAAID2oNXx1b1797z99tub3P6Zz3wmjzzySI4//vhMnjy5tS8DAADQLrQ6voYPH5677rprk9t79uyZjh075uabb06nTp1a+zIAAADtQquv+br00kszdOjQDBkyJOPGjfuL7c8991z69OmzTcMBAAC0F60+8zV48ODMnDkz55xzTk499dT8+7//e5YtW5bGxsbMnj07X/jCF3LmmWe25awAAABVa5u+ZPm8885L//79M3HixIwePTo1NTVJkubm5owaNSrTp09vkyEBAACq3TbFV/LetV/19fUty+rVq3PYYYdl6NChbTEfAABAu7DN8fW+wYMHZ/DgwW31dAAAAO1Kq6/5AgAAYMuJLwAAgAKqPr6uvPLKHHPMMdl1112zxx57bNExzc3NmTZtWvbee+906dIlI0eOzCuvvLJ9BwUAAHZqVR9fq1evzumnn54LL7xwi4/5xje+kX/8x3/MjTfemJ///OfZbbfdMmrUqLz77rvbcVIAAGBn1mY33KiUyy67LEly++23b9H+zc3Nufbaa/PVr341p556apLkzjvvTM+ePfPAAw/krLPO2l6jAgAAO7GqP/O1tf7whz+koaEhI0eObFnXrVu3DB06NHPnzt3kcatWrUpTU9MGCwAAwJba6eKroaEhSdKzZ88N1vfs2bNl28bMmDEj3bp1a1n69u27XecEAADalx0yvqZMmZKamprNLi+//HLRmaZOnZrGxsaWZcmSJUVfHwAAqG475DVfkydPzrnnnrvZffr379+q5+7Vq1eSZNmyZdl7771b1i9btmyzXxJdW1ub2traVr0mAADADhlfdXV1qaur2y7Pvd9++6VXr16ZM2dOS2w1NTXl5z//+VbdMREAAGBr7JAfO9waixcvTn19fRYvXpx169alvr4+9fX1WbFiRcs+AwcOzP33358kqampycSJE/O1r30ts2fPzq9//eucffbZ6d27d8aOHVuhnwIAAGjvdsgzX1tj2rRpueOOO1oeH3744UmSJ554IsOHD0+SLFy4MI2NjS37fPnLX87KlSvz2c9+Nu+8804+8pGP5NFHH03nzp2Lzg4AAOw8apqbm5srPUQ1ampqSrdu3dLY2JiuXbtWepz0m/JwpUegjS26akylRwCAdsP/ldqfHen/SlvaBlX/sUMAAIBqIL4AAAAKEF8AAAAFiC8AAIACxBcAAEAB4gsAAKAA8QUAAFCA+AIAAChAfAEAABQgvgAAAAoQXwAAAAWILwAAgALEFwAAQAHiCwAAoADxBQAAUID4AgAAKEB8AQAAFCC+AAAACuhY6QEAAGB7W3TVmEqPAM58AQAAlCC+AAAAChBfAAAABYgvAACAAsQXAABAAeILAACgAPEFAABQgPgCAAAoQHwBAAAUIL4AAAAKEF8AAAAFiC8AAIACxBcAAEAB4gsAAKAA8QUAAFCA+AIAAChAfAEAABQgvgAAAAoQXwAAAAWILwAAgALEFwAAQAHiCwAAoADxBQAAUID4AgAAKEB8AQAAFCC+AAAAChBfAAAABYgvAACAAsQXAABAAeILAACgAPEFAABQgPgCAAAoQHwBAAAUIL4AAAAKEF8AAAAFiC8AAIACxBcAAEAB4gsAAKAA8QUAAFCA+AIAAChAfAEAABQgvgAAAAoQXwAAAAWILwAAgALEFwAAQAHiCwAAoADxBQAAUID4AgAAKEB8AQAAFCC+AAAAChBfAAAABYgvAACAAsQXAABAAeILAACgAPEFAABQgPgCAAAoQHwBAAAUIL4AAAAKEF8AAAAFiC8AAIACxBcAAEAB4gsAAKAA8QUAAFCA+AIAAChAfAEAABQgvgAAAAoQXwAAAAVUfXxdeeWVOeaYY7Lrrrtmjz322KJjzj333NTU1GywjB49evsOCgAA7NQ6VnqAbbV69eqcfvrpGTZsWG699dYtPm706NGZNWtWy+Pa2trtMR4AAECSdhBfl112WZLk9ttv36rjamtr06tXr+0wEQAAwF+q+o8dttaTTz6ZvfbaKwcccEAuvPDC/PGPf9zs/qtWrUpTU9MGCwAAwJbaKeNr9OjRufPOOzNnzpx8/etfz1NPPZWTTjop69at2+QxM2bMSLdu3VqWvn37FpwYAACodjtkfE2ZMuUvbojxv5eXX3651c9/1lln5eMf/3gOPfTQjB07Ng899FDmz5+fJ598cpPHTJ06NY2NjS3LkiVLWv36AADAzmeHvOZr8uTJOffccze7T//+/dvs9fr3758ePXrk1VdfzYgRIza6T21trZtyAAAArbZDxlddXV3q6uqKvd5//dd/5Y9//GP23nvvYq8JAADsXHbIjx1ujcWLF6e+vj6LFy/OunXrUl9fn/r6+qxYsaJln4EDB+b+++9PkqxYsSJf+tKX8rOf/SyLFi3KnDlzcuqpp+bDH/5wRo0aVakfAwAAaOd2yDNfW2PatGm54447Wh4ffvjhSZInnngiw4cPT5IsXLgwjY2NSZJddtklzz//fO64446888476d27d0488cRcccUVPlYIAABsNzXNzc3NlR6iGjU1NaVbt25pbGxM165dKz1O+k15uNIj0MYWXTWm0iMAALAFtrQNqv5jhwAAANVAfAEAABQgvgAAAAoQXwAAAAWILwAAgALEFwAAQAHiCwAAoADxBQAAUID4AgAAKEB8AQAAFCC+AAAAChBfAAAABYgvAACAAsQXAABAAeILAACgAPEFAABQgPgCAAAoQHwBAAAUIL4AAAAKEF8AAAAFiC8AAIACxBcAAEAB4gsAAKAA8QUAAFCA+AIAAChAfAEAABQgvgAAAAoQXwAAAAWILwAAgALEFwAAQAHiCwAAoADxBQAAUID4AgAAKEB8AQAAFCC+AAAAChBfAAAABYgvAACAAsQXAABAAeILAACgAPEFAABQgPgCAAAoQHwBAAAUIL4AAAAKEF8AAAAFiC8AAIACxBcAAEAB4gsAAKAA8QUAAFCA+AIAAChAfAEAABQgvgAAAAoQXwAAAAWILwAAgALEFwAAQAHiCwAAoADxBQAAUID4AgAAKEB8AQAAFCC+AAAAChBfAAAABYgvAACAAsQXAABAAeILAACgAPEFAABQgPgCAAAoQHwBAAAUIL4AAAAKEF8AAAAFiC8AAIACxBcAAEAB4gsAAKAA8QUAAFCA+AIAAChAfAEAABQgvgAAAAoQXwAAAAWILwAAgALEFwAAQAHiCwAAoICOlR6AtrHoqjGVHgEAANgMZ74AAAAKEF8AAAAFiC8AAIACxBcAAEAB4gsAAKCAqo6vRYsWZcKECdlvv/3SpUuX7L///pk+fXpWr1692ePefffdXHTRRdlzzz3zwQ9+MKeddlqWLVtWaGoAAGBnVNXx9fLLL2f9+vW56aab8sILL+Tb3/52brzxxnzlK1/Z7HFf+MIX8uCDD+bee+/NU089laVLl+aTn/xkoakBAICdUU1zc3NzpYdoS1dffXVmzpyZ3//+9xvd3tjYmLq6utx999351Kc+leS9iDvwwAMzd+7c/NVf/dUWvU5TU1O6deuWxsbGdO3atc3mBwAAqsuWtkFVn/namMbGxnTv3n2T2xcsWJA1a9Zk5MiRLesGDhyYffbZJ3Pnzt3kcatWrUpTU9MGCwAAwJZqV/H16quv5jvf+U4uuOCCTe7T0NCQTp06ZY899thgfc+ePdPQ0LDJ42bMmJFu3bq1LH379m2rsQEAgJ3ADhlfU6ZMSU1NzWaXl19+eYNjXnvttYwePTqnn356zj///DafaerUqWlsbGxZlixZ0uavAQAAtF8dKz3AxkyePDnnnnvuZvfp379/y5+XLl2a448/Psccc0xuvvnmzR7Xq1evrF69Ou+8884GZ7+WLVuWXr16bfK42tra1NbWbtH8AAAA/9sOGV91dXWpq6vbon1fe+21HH/88RkyZEhmzZqVDh02fzJvyJAh+cAHPpA5c+bktNNOS5IsXLgwixcvzrBhw7Z5dgAAgI3ZIT92uKVee+21DB8+PPvss0+uueaavPnmm2loaNjg2q3XXnstAwcOzLx585Ik3bp1y4QJEzJp0qQ88cQTWbBgQcaPH59hw4Zt8Z0OAQAAttYOeeZrSz322GN59dVX8+qrr6ZPnz4bbHv/Dvpr1qzJwoUL8+c//7ll27e//e106NAhp512WlatWpVRo0blu9/9btHZAQCAnUu7+56vUnzPFwAAkOzE3/MFAACwIxJfAAAABYgvAACAAqr6hhuV9P6lck1NTRWeBAAAqKT3m+D/up2G+Gql5cuXJ0n69u1b4UkAAIAdwfLly9OtW7dNbne3w1Zav359li5dmt133z01NTWVHmen0NTUlL59+2bJkiXuMMk28V6irXgv0Va8l2gr3kuV0dzcnOXLl6d3797p0GHTV3Y589VKHTp0+IvvFqOMrl27+suENuG9RFvxXqKteC/RVryXytvcGa/3ueEGAABAAeILAACgAPFF1aitrc306dNTW1tb6VGoct5LtBXvJdqK9xJtxXtpx+aGGwAAAAU48wUAAFCA+AIAAChAfAEAABQgvgAAAAoQX1SNG264If369Uvnzp0zdOjQzJs3r9IjUWWefvrpnHLKKendu3dqamrywAMPVHokqtSMGTNy1FFHZffdd89ee+2VsWPHZuHChZUeiyozc+bMDBo0qOXLcIcNG5ZHHnmk0mPRDlx11VWpqanJxIkTKz0K/4v4oir84Ac/yKRJkzJ9+vT84he/yGGHHZZRo0bljTfeqPRoVJGVK1fmsMMOyw033FDpUahyTz31VC666KL87Gc/y2OPPZY1a9bkxBNPzMqVKys9GlWkT58+ueqqq7JgwYI899xzOeGEE3LqqafmhRdeqPRoVLH58+fnpptuyqBBgyo9ChvhVvNUhaFDh+aoo47K9ddfnyRZv359+vbtm89//vOZMmVKhaejGtXU1OT+++/P2LFjKz0K7cCbb76ZvfbaK0899VQ++tGPVnocqlj37t1z9dVXZ8KECZUehSq0YsWKHHHEEfnud7+br33taxk8eHCuvfbaSo/F/+DMFzu81atXZ8GCBRk5cmTLug4dOmTkyJGZO3duBScDeE9jY2OS9/7jDK2xbt263HPPPVm5cmWGDRtW6XGoUhdddFHGjBmzwf+Z2LF0rPQA8H956623sm7duvTs2XOD9T179szLL79coakA3rN+/fpMnDgxxx57bA455JBKj0OV+fWvf51hw4bl3XffzQc/+MHcf//9Oeiggyo9FlXonnvuyS9+8YvMnz+/0qOwGeILALbBRRddlN/85jf5j//4j0qPQhU64IADUl9fn8bGxvzrv/5rzjnnnDz11FMCjK2yZMmSXHLJJXnsscfSuXPnSo/DZogvdng9evTILrvskmXLlm2wftmyZenVq1eFpgJILr744jz00EN5+umn06dPn0qPQxXq1KlTPvzhDydJhgwZkvnz5+e6667LTTfdVOHJqCYLFizIG2+8kSOOOKJl3bp16/L000/n+uuvz6pVq7LLLrtUcELe55ovdnidOnXKkCFDMmfOnJZ169evz5w5c3wuHqiI5ubmXHzxxbn//vvz+OOPZ7/99qv0SLQT69evz6pVqyo9BlVmxIgR+fWvf536+vqW5cgjj8y4ceNSX18vvHYgznxRFSZNmpRzzjknRx55ZI4++uhce+21WblyZcaPH1/p0agiK1asyKuvvtry+A9/+EPq6+vTvXv37LPPPhWcjGpz0UUX5e67786//du/Zffdd09DQ0OSpFu3bunSpUuFp6NaTJ06NSeddFL22WefLF++PHfffXeefPLJ/PjHP670aFSZ3Xff/S+uOd1tt92y5557uhZ1ByO+qApnnnlm3nzzzUybNi0NDQ0ZPHhwHn300b+4CQdsznPPPZfjjz++5fGkSZOSJOecc05uv/32Ck1FNZo5c2aSZPjw4RusnzVrVs4999zyA1GV3njjjZx99tl5/fXX061btwwaNCg//vGP87GPfazSowHbie/5AgAAKMA1XwAAAAWILwAAgALEFwAAQAHiCwAAoADxBQAAUID4AgAAKEB8AQAAFCC+AGAr3X777dljjz0qPQYAVUZ8AbBTmDt3bnbZZZeMGTNmq47r169frr322g3WnXnmmfntb3/bhtMBsDMQXwDsFG699dZ8/vOfz9NPP52lS5du03N16dIle+21VxtNBsDOQnwB0O6tWLEiP/jBD3LhhRdmzJgxuf322zfY/uCDD+aoo45K586d06NHj3ziE59IkgwfPjz/+Z//mS984QupqalJTU1Nko1/7HDmzJnZf//906lTpxxwwAH553/+5w2219TU5JZbbsknPvGJ7LrrrhkwYEBmz57dsv3tt9/OuHHjUldXly5dumTAgAGZNWtW2/8yAKgY8QVAu/cv//IvGThwYA444ID87d/+bW677bY0NzcnSR5++OF84hOfyMknn5xf/vKXmTNnTo4++ugkyX333Zc+ffrk8ssvz+uvv57XX399o89///3355JLLsnkyZPzm9/8JhdccEHGjx+fJ554YoP9Lrvsspxxxhl5/vnnc/LJJ2fcuHH505/+lCT5u7/7u7z44ot55JFH8tJLL2XmzJnp0aPHdvytAFBaTfP7//oAQDt17LHH5owzzsgll1yStWvXZu+99869996b4cOH55hjjkn//v1z1113bfTYfv36ZeLEiZk4cWLLuttvvz0TJ07MO++80/L8Bx98cG6++eaWfc4444ysXLkyDz/8cJL3znx99atfzRVXXJEkWblyZT74wQ/mkUceyejRo/Pxj388PXr0yG233bZ9fgkAVJwzXwC0awsXLsy8efPy6U9/OknSsWPHnHnmmbn11luTJPX19RkxYsQ2vcZLL72UY489doN1xx57bF566aUN1g0aNKjlz7vttlu6du2aN954I0ly4YUX5p577sngwYPz5S9/Oc8+++w2zQTAjkd8AdCu3XrrrVm7dm169+6djh07pmPHjpk5c2Z++MMfprGxMV26dCk2ywc+8IENHtfU1GT9+vVJkpNOOqnl+rKlS5dmxIgR+eIXv1hsNgC2P/EFQLu1du3a3HnnnfnmN7+Z+vr6luVXv/pVevfune9///sZNGhQ5syZs8nn6NSpU9atW7fZ1znwwAPzzDPPbLDumWeeyUEHHbRV89bV1eWcc87JXXfdlWuvvXaDjzECUP06VnoAANheHnroobz99tuZMGFCunXrtsG20047LbfeemuuvvrqjBgxIvvvv3/OOuusrF27Nj/60Y9y6aWXJnnvmq+nn346Z511Vmprazd6E4wvfelLOeOMM3L44Ydn5MiRefDBB3PfffflJz/5yRbPOm3atAwZMiQHH3xwVq1alYceeigHHnjgtv0CANihOPMFQLt16623ZuTIkX8RXsl78fXcc8+le/fuuffeezN79uwMHjw4J5xwQubNm9ey3+WXX55FixZl//33T11d3UZfZ+zYsbnuuutyzTXX5OCDD85NN92UWbNmZfjw4Vs8a6dOnTJ16tQMGjQoH/3oR7PLLrvknnvu2eqfGYAdl7sdAgAAFODMFwAAQAHiCwAAoADxBQAAUID4AgAAKEB8AQAAFCC+AAAAChBfAAAABYgvAACAAsQXAABAAeILAACgAPEFAABQgPgCAAAo4P8BsYNTiDEe9LgAAAAASUVORK5CYII=", "text/plain": [ "
" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "print(\"Optimal action:\", Q_star.argmax())\n", "\n", "plt.figure(figsize=(10, 6))\n", "plt.bar(actions, Q_star)\n", "plt.xlabel('Actions')\n", "plt.ylabel('$Q^*(a)$')\n", "plt.show()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Great, now let's start evaluating these Q-values with random sampling.\n", "\n", "**Q:** Define an action sampling method `get_reward` taking as arguments:\n", "* The array `Q_star`.\n", "* The index `a` of the action you want to sample (between 0 and 4).\n", "* An optional variance argument `var`, which should have the value 1.0 by default.\n", " \n", "It should return a single value, sampled from the normal distribution with mean `Q_star[a]` and variance `var`." ] }, { "cell_type": "code", "execution_count": 5, "metadata": {}, "outputs": [], "source": [ "def get_reward(Q_star, a, var=1.0):\n", " return float(rng.normal(Q_star[a], var, 1)[0])" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "**Q:** For each possible action `a`, take `nb_samples=10` out of the reward distribution and store them in a numpy array. Compute the mean of the samples for each action separately in a new array `Q_t`. Make a bar plot of these estimated Q-values." ] }, { "cell_type": "code", "execution_count": 6, "metadata": {}, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAA10AAAINCAYAAADFghUcAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjkuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8hTgPZAAAACXBIWXMAAA9hAAAPYQGoP6dpAAAoZElEQVR4nO3dfbSVdZ3//9cB5IAlx0GO3CthLlERMVRCG8UgIVklZYoNs1AyK5e6NOgGWg2MVoOlFU4a6uTtmOk4qXmTNkbezCQFYqfMG9IKIfCAlp4DNHJ7fn/463yHROLg+bDZ8Hisda3FvvZ17f3erL2U5/rsfe2alpaWlgAAAFBEh0oPAAAAsCsTXQAAAAWJLgAAgIJEFwAAQEGiCwAAoCDRBQAAUJDoAgAAKEh0AQAAFNSp0gNUk02bNmX58uXZa6+9UlNTU+lxAACACmlpacmqVavSp0+fdOiw9bUs0dUGy5cvT//+/Ss9BgAAsJNYunRp+vXrt9VjRFcb7LXXXkle/4vt1q1bhacBAAAqpbm5Of37929thK0RXW3wl48UduvWTXQBAADb9LUjF9IAAAAoSHQBAAAUJLoAAAAKEl0AAAAFiS4AAICCRBcAAEBBogsAAKAg0QUAAFCQ6AIAAChIdAEAABQkugAAAAoSXQAAAAWJLgAAgIJEFwAAQEGiCwAAoCDRBQAAUJDoAgAAKEh0AQAAFNSp0gMAlTdg2n2VHoECFl8yrtIjAACx0gUAAFCU6AIAAChIdAEAABQkugAAAAoSXQAAAAWJLgAAgIJEFwAAQEGiCwAAoCDRBQAAUJDoAgAAKEh0AQAAFFT10XXllVdmwIAB6dKlS4YPH5758+e/6bE33HBDampqNtu6dOmyA6cFAAB2N1UdXbfddlumTJmSmTNn5oknnsjhhx+eMWPGZOXKlW96Trdu3fLiiy+2bi+88MIOnBgAANjdVHV0feMb38jZZ5+dyZMn55BDDslVV12VPffcM9ddd92bnlNTU5NevXq1bj179tyBEwMAALubqo2udevWZeHChRk9enTrvg4dOmT06NGZN2/em563evXq7L///unfv39OPvnkPPXUU2967Nq1a9Pc3LzZBgAA0BZVG10vv/xyNm7c+IaVqp49e6axsXGL5xx00EG57rrr8oMf/CA333xzNm3alGOOOSZ/+MMftnj8rFmzUldX17r179+/3V8HAACwa6va6NoeI0aMyKRJkzJ06NAcf/zxueOOO1JfX5+rr756i8dPnz49TU1NrdvSpUt38MQAAEC161TpAbZXjx490rFjx6xYsWKz/StWrEivXr226TH22GOPHHHEEXn++ee3eH9tbW1qa2vf8qwAAMDuq2pXujp37pxhw4Zl7ty5rfs2bdqUuXPnZsSIEdv0GBs3bsyTTz6Z3r17lxoTAADYzVXtSleSTJkyJWeccUaOPPLIHH300Zk9e3bWrFmTyZMnJ0kmTZqUvn37ZtasWUmSiy++OO9+97vzzne+M6+++mouvfTSvPDCC/n4xz9eyZcBAADswqo6uiZMmJCXXnopM2bMSGNjY4YOHZoHHnig9eIaS5YsSYcO/28x75VXXsnZZ5+dxsbG/N3f/V2GDRuWxx57LIccckilXgIAALCLq2lpaWmp9BDVorm5OXV1dWlqakq3bt0qPQ60mwHT7qv0CBSw+JJxlR4BAHZZbWmDqv1OFwAAQDUQXQAAAAWJLgAAgIJEFwAAQEGiCwAAoCDRBQAAUJDoAgAAKEh0AQAAFCS6AAAAChJdAAAABYkuAACAgkQXAABAQaILAACgINEFAABQkOgCAAAoSHQBAAAUJLoAAAAKEl0AAAAFiS4AAICCRBcAAEBBogsAAKAg0QUAAFCQ6AIAAChIdAEAABQkugAAAAoSXQAAAAWJLgAAgIJEFwAAQEGiCwAAoCDRBQAAUJDoAgAAKEh0AQAAFCS6AAAAChJdAAAABYkuAACAgkQXAABAQaILAACgINEFAABQkOgCAAAoSHQBAAAUJLoAAAAKEl0AAAAFiS4AAICCRBcAAEBBogsAAKAg0QUAAFCQ6AIAAChIdAEAABQkugAAAAoSXQAAAAWJLgAAgIJEFwAAQEGiCwAAoCDRBQAAUJDoAgAAKEh0AQAAFCS6AAAAChJdAAAABYkuAACAgkQXAABAQaILAACgINEFAABQUNVH15VXXpkBAwakS5cuGT58eObPn7/V42+//fYMGjQoXbp0yWGHHZYf/vCHO2hSAABgd1TV0XXbbbdlypQpmTlzZp544okcfvjhGTNmTFauXLnF4x977LF89KMfzVlnnZVf/OIXGT9+fMaPH59f//rXO3hyAABgd1HT0tLSUukhttfw4cNz1FFH5YorrkiSbNq0Kf3798/555+fadOmveH4CRMmZM2aNbn33ntb97373e/O0KFDc9VVV/3N52tubk5dXV2amprSrVu39nshUGEDpt1X6REoYPEl4yo9AgDsstrSBlW70rVu3bosXLgwo0ePbt3XoUOHjB49OvPmzdviOfPmzdvs+CQZM2bMmx6/du3aNDc3b7YBAAC0RadKD7C9Xn755WzcuDE9e/bcbH/Pnj3z7LPPbvGcxsbGLR7f2Ni4xeNnzZqViy66qH0GLsQKxa6nEqsTVkRoL/6btGuqxH8jvJd2PZX6f4330q6nGv/dUrUrXTvC9OnT09TU1LotXbq00iMBAABVpmpXunr06JGOHTtmxYoVm+1fsWJFevXqtcVzevXq1abja2trU1tb2z4DAwAAu6WqXenq3Llzhg0blrlz57bu27RpU+bOnZsRI0Zs8ZwRI0ZsdnySPPjgg296PAAAwFtVtStdSTJlypScccYZOfLII3P00Udn9uzZWbNmTSZPnpwkmTRpUvr27ZtZs2YlSS644IIcf/zx+frXv55x48bl1ltvzeOPP55rrrmmki8DAADYhVV1dE2YMCEvvfRSZsyYkcbGxgwdOjQPPPBA68UylixZkg4d/t9i3jHHHJNbbrklX/ziF/OFL3whBx54YO66664MHjy4Ui8BAADYxVV1dCXJeeedl/POO2+L9z388MNv2Hfqqafm1FNPLTwVAADA66r2O10AAADVQHQBAAAUJLoAAAAKEl0AAAAFiS4AAICCRBcAAEBBogsAAKAg0QUAAFCQ6AIAAChIdAEAABQkugAAAAoSXQAAAAWJLgAAgIJEFwAAQEGiCwAAoCDRBQAAUJDoAgAAKEh0AQAAFCS6AAAAChJdAAAABYkuAACAgkQXAABAQaILAACgINEFAABQkOgCAAAoSHQBAAAU1KnSAwAA/LXFl4yr9AgA7cZKFwAAQEGiCwAAoCDRBQAAUJDoAgAAKEh0AQAAFCS6AAAAChJdAAAABYkuAACAgkQXAABAQaILAACgINEFAABQkOgCAAAoSHQBAAAUJLoAAAAKEl0AAAAFiS4AAICCRBcAAEBBogsAAKAg0QUAAFBQp0oPAMCuY/El4yo9AgDsdKx0AQAAFCS6AAAAChJdAAAABYkuAACAgkQXAABAQaILAACgINEFAABQkOgCAAAoSHQBAAAUJLoAAAAKEl0AAAAFiS4AAICCRBcAAEBBogsAAKAg0QUAAFBQ1UbXn/70p0ycODHdunXL3nvvnbPOOiurV6/e6jkjR45MTU3NZtunPvWpHTQxAACwO+pU6QG218SJE/Piiy/mwQcfzPr16zN58uR84hOfyC233LLV884+++xcfPHFrbf33HPP0qMCAAC7saqMrmeeeSYPPPBAFixYkCOPPDJJ8q1vfSsnnXRSLrvssvTp0+dNz91zzz3Tq1evHTUqAACwm6vKjxfOmzcve++9d2twJcno0aPToUOH/PznP9/qud/97nfTo0ePDB48ONOnT8+f//znNz127dq1aW5u3mwDAABoi6pc6WpsbMy+++672b5OnTqle/fuaWxsfNPz/uEf/iH7779/+vTpk1/96lf5/Oc/n0WLFuWOO+7Y4vGzZs3KRRdd1K6zAwAAu5edKrqmTZuWr371q1s95plnntnux//EJz7R+ufDDjssvXv3zqhRo/Lb3/42BxxwwBuOnz59eqZMmdJ6u7m5Of3799/u5wcAAHY/O1V0TZ06NWeeeeZWjxk4cGB69eqVlStXbrZ/w4YN+dOf/tSm72sNHz48SfL8889vMbpqa2tTW1u7zY8HAADw13aq6Kqvr099ff3fPG7EiBF59dVXs3DhwgwbNixJ8pOf/CSbNm1qDalt0dDQkCTp3bv3ds0LAADwt1TlhTQOPvjgjB07NmeffXbmz5+fn/70pznvvPNy+umnt165cNmyZRk0aFDmz5+fJPntb3+bL33pS1m4cGEWL16cu+++O5MmTcpxxx2XIUOGVPLlAAAAu7CqjK7k9asQDho0KKNGjcpJJ52U97znPbnmmmta71+/fn0WLVrUenXCzp0758c//nFOPPHEDBo0KFOnTs0pp5ySe+65p1IvAQAA2A3sVB8vbIvu3btv9YeQBwwYkJaWltbb/fv3zyOPPLIjRgMAAGhVtStdAAAA1UB0AQAAFCS6AAAAChJdAAAABYkuAACAgkQXAABAQaILAACgINEFAABQkOgCAAAoSHQBAAAUJLoAAAAKEl0AAAAFiS4AAICCRBcAAEBBogsAAKAg0QUAAFCQ6AIAAChIdAEAABQkugAAAAoSXQAAAAWJLgAAgIJEFwAAQEGiCwAAoCDRBQAAUJDoAgAAKKjT9p64fv36NDY25s9//nPq6+vTvXv39pwLAABgl9Cmla5Vq1Zlzpw5Of7449OtW7cMGDAgBx98cOrr67P//vvn7LPPzoIFC0rNCgAAUHW2Obq+8Y1vZMCAAbn++uszevTo3HXXXWloaMhvfvObzJs3LzNnzsyGDRty4oknZuzYsXnuuedKzg0AAFAVtvnjhQsWLMijjz6aQw89dIv3H3300fnYxz6Wq666Ktdff33++7//OwceeGC7DQoAAFCNtjm6vve9723TcbW1tfnUpz613QMBAADsSly9EAAAoKDtunrhxo0b853vfCeLFi1Kv379cvjhh2fo0KHZZ5992ns+AACAqrZd0XX++efn+9//fkaPHp0rrrgiNTU12bBhQ/r27ZuhQ4fm7rvvbu85AQAAqtJ2fbzwjjvuyE033ZTvfve7qa2tzeOPP57LL788r732Wvbff//2nhEAAKBqbddK1+rVq3PIIYckSfbYY4906tQp5513XtavX5/ly5e364AAAADVbLtWugYOHNgaV3379s2yZcuSJB/4wAdy8803t990AAAAVW67ouvDH/5w7r///iTJ8ccfn+uuuy5J8vTTT+d///d/2286AACAKrddHy/853/+59Y/f+5zn8tRRx2V+vr6NDc356yzzmqv2QAAAKredkXX/7Xffvvlqaeeyj333JN99tknH/jAB9pjLgAAgF3CNkfXkiVLst9++23xvh49emTy5Mmb7Vu2bFn69u371qYDAACoctv8na6jjjoqn/zkJ7NgwYI3PaapqSn/9m//lsGDB+f73/9+uwwIAABQzbZ5pevpp5/OV77ylbzvfe9Lly5dMmzYsPTp0yddunTJK6+8kqeffjpPPfVU3vWud+VrX/taTjrppJJzAwAAVIVtXunaZ5998o1vfCMvvvhirrjiihx44IF5+eWX89xzzyVJJk6cmIULF2bevHmCCwAA4P/X5gtpdO3aNR/5yEfykY98pMQ8AAAAu5Tt+p2uv+a3uQAAALasXaLrPe95zxv2Pfvss+3x0AAAAFXtLf1O1z333JOnn346q1evztKlS9O/f//W+yZMmJBf/vKXb3lAAACAavaWomvw4MFZunRpXn755UyaNClLlixJ375907t37+yxxx7tNSMAAEDV2q7o+t3vfpdf/vKX6dSpU8aNG5fBgwfnuOOOS/L6jyK/8MILGTx4cLsOCgAAUI3aFF0bN27Mxz/+8dx0001paWlJktTU1OT444/P5ZdfnsMOOyx9+/ZN3759iwwLAABQbdp0IY1/+Zd/yd13352rr746ixYtSkNDQ77zne9k1apVOfbYY/OTn/yk1JwAAABVqU0rXTfeeGO++c1vZtKkSa37hgwZksmTJ+eyyy7L+PHj89xzz6Vr1675xS9+keOPP77dBwYAAKgmbVrpWrp0af7+7/9+i/d95jOfyYQJE/Kxj30sw4YNy2OPPdYuAwIAAFSzNkVX9+7d88orr7zp/R//+Mdz//3354QTTsjUqVPf8nAAAADVrk3RNXLkyNx8881ven/Pnj3TqVOnXHPNNencufNbHg4AAKDatek7XZ///OczfPjwDBs2LBMnTnzD/Y8//nj69evXbsMBAABUuzatdA0dOjRz5szJGWeckZNPPjn/9V//lRUrVqSpqSl33313Pv3pT2fChAmlZgUAAKg6bf5x5I997GMZOHBgLrzwwowdOzY1NTVJkpaWlowZMyYzZ85s9yEBAACqVZujK3n9u10NDQ2t27p163L44Ydn+PDh7T0fAABAVduu6PqLoUOHZujQoe00CgAAwK6nTd/pAgAAoG1EFwAAQEFVGV1f+cpXcswxx2TPPffM3nvvvU3ntLS0ZMaMGendu3e6du2a0aNH57nnnis7KAAAsNuryuhat25dTj311JxzzjnbfM7Xvva1/Ou//muuuuqq/PznP8/b3va2jBkzJq+99lrBSQEAgN3dW7qQRqVcdNFFSZIbbrhhm45vaWnJ7Nmz88UvfjEnn3xykuSmm25Kz549c9ddd+X0008vNSoAALCbq8qVrrb6/e9/n8bGxowePbp1X11dXYYPH5558+ZVcDIAAGBXV5UrXW3V2NiYJOnZs+dm+3v27Nl635asXbs2a9eubb3d3NxcZkAAAGCXtdOsdE2bNi01NTVb3Z599tkdOtOsWbNSV1fXuvXv33+HPj8AAFD9dpqVrqlTp+bMM8/c6jEDBw7crsfu1atXkmTFihXp3bt36/4VK1Zs9cedp0+fnilTprTebm5uFl4AAECb7DTRVV9fn/r6+iKP/Y53vCO9evXK3LlzWyOrubk5P//5z7d6BcTa2trU1tYWmQkAANg97DQfL2yLJUuWpKGhIUuWLMnGjRvT0NCQhoaGrF69uvWYQYMG5c4770yS1NTU5MILL8yXv/zl3H333XnyySczadKk9OnTJ+PHj6/QqwAAAHYHO81KV1vMmDEjN954Y+vtI444Ikny0EMPZeTIkUmSRYsWpampqfWYz33uc1mzZk0+8YlP5NVXX8173vOePPDAA+nSpcsOnR0AANi91LS0tLRUeohq0dzcnLq6ujQ1NaVbt26VHidJMmDafZUegXa2+JJxlR4BAHYZ/q2069lZ/q3Uljaoyo8XAgAAVAvRBQAAUJDoAgAAKEh0AQAAFCS6AAAAChJdAAAABYkuAACAgkQXAABAQaILAACgINEFAABQkOgCAAAoSHQBAAAUJLoAAAAKEl0AAAAFiS4AAICCRBcAAEBBogsAAKAg0QUAAFCQ6AIAAChIdAEAABQkugAAAAoSXQAAAAWJLgAAgIJEFwAAQEGiCwAAoCDRBQAAUJDoAgAAKEh0AQAAFCS6AAAAChJdAAAABYkuAACAgkQXAABAQaILAACgINEFAABQkOgCAAAoSHQBAAAUJLoAAAAKEl0AAAAFiS4AAICCRBcAAEBBogsAAKAg0QUAAFCQ6AIAAChIdAEAABQkugAAAAoSXQAAAAWJLgAAgII6VXoAAAAoZfEl4yo9AljpAgAAKEl0AQAAFCS6AAAAChJdAAAABYkuAACAgkQXAABAQaILAACgINEFAABQkOgCAAAoSHQBAAAUJLoAAAAKEl0AAAAFiS4AAICCRBcAAEBBogsAAKCgqoyur3zlKznmmGOy5557Zu+9996mc84888zU1NRsto0dO7bsoAAAwG6vU6UH2B7r1q3LqaeemhEjRuTaa6/d5vPGjh2b66+/vvV2bW1tifEAAABaVWV0XXTRRUmSG264oU3n1dbWplevXgUmAgAA2LKq/Hjh9nr44Yez77775qCDDso555yTP/7xj1s9fu3atWlubt5sAwAAaIvdJrrGjh2bm266KXPnzs1Xv/rVPPLII3n/+9+fjRs3vuk5s2bNSl1dXevWv3//HTgxAACwK9hpomvatGlvuNDFX2/PPvvsdj/+6aefng9+8IM57LDDMn78+Nx7771ZsGBBHn744Tc9Z/r06Wlqamrdli5dut3PDwAA7J52mu90TZ06NWeeeeZWjxk4cGC7Pd/AgQPTo0ePPP/88xk1atQWj6mtrXWxDQAA4C3ZaaKrvr4+9fX1O+z5/vCHP+SPf/xjevfuvcOeEwAA2P3sNB8vbIslS5akoaEhS5YsycaNG9PQ0JCGhoasXr269ZhBgwblzjvvTJKsXr06n/3sZ/Ozn/0sixcvzty5c3PyySfnne98Z8aMGVOplwEAAOwGdpqVrraYMWNGbrzxxtbbRxxxRJLkoYceysiRI5MkixYtSlNTU5KkY8eO+dWvfpUbb7wxr776avr06ZMTTzwxX/rSl3x8EAAAKKqmpaWlpdJDVIvm5ubU1dWlqakp3bp1q/Q4SZIB0+6r9Ai0s8WXjKv0CAAA/A1taYOq/HghAABAtRBdAAAABYkuAACAgkQXAABAQaILAACgINEFAABQkOgCAAAoSHQBAAAUJLoAAAAKEl0AAAAFiS4AAICCRBcAAEBBogsAAKAg0QUAAFCQ6AIAAChIdAEAABQkugAAAAoSXQAAAAWJLgAAgIJEFwAAQEGiCwAAoCDRBQAAUJDoAgAAKEh0AQAAFCS6AAAAChJdAAAABYkuAACAgkQXAABAQaILAACgINEFAABQkOgCAAAoSHQBAAAUJLoAAAAKEl0AAAAFiS4AAICCRBcAAEBBogsAAKAg0QUAAFCQ6AIAAChIdAEAABQkugAAAAoSXQAAAAWJLgAAgIJEFwAAQEGiCwAAoCDRBQAAUJDoAgAAKEh0AQAAFCS6AAAAChJdAAAABYkuAACAgkQXAABAQaILAACgINEFAABQkOgCAAAoSHQBAAAUJLoAAAAKEl0AAAAFiS4AAICCRBcAAEBBogsAAKAg0QUAAFBQVUbX4sWLc9ZZZ+Ud73hHunbtmgMOOCAzZ87MunXrtnrea6+9lnPPPTf77LNP3v72t+eUU07JihUrdtDUAADA7qgqo+vZZ5/Npk2bcvXVV+epp57KN7/5zVx11VX5whe+sNXzPv3pT+eee+7J7bffnkceeSTLly/Phz/84R00NQAAsDuqaWlpaan0EO3h0ksvzZw5c/K73/1ui/c3NTWlvr4+t9xySz7ykY8keT3eDj744MybNy/vfve7/+ZzNDc3p66uLk1NTenWrVu7zr+9Bky7r9Ij0M4WXzKu0iMAAPA3tKUNqnKla0uamprSvXv3N71/4cKFWb9+fUaPHt26b9CgQdlvv/0yb968LZ6zdu3aNDc3b7YBAAC0xS4RXc8//3y+9a1v5ZOf/OSbHtPY2JjOnTtn77333mx/z54909jYuMVzZs2albq6utatf//+7Tk2AACwG9ipomvatGmpqanZ6vbss89uds6yZcsyduzYnHrqqTn77LPbdZ7p06enqampdVu6dGm7Pj4AALDr61TpAf6vqVOn5swzz9zqMQMHDmz98/Lly3PCCSfkmGOOyTXXXLPV83r16pV169bl1Vdf3Wy1a8WKFenVq9cWz6mtrU1tbe02zw8AAPDXdqroqq+vT319/TYdu2zZspxwwgkZNmxYrr/++nTosPVFu2HDhmWPPfbI3Llzc8oppyRJFi1alCVLlmTEiBFveXYAAIAt2ak+Xritli1blpEjR2a//fbLZZddlpdeeimNjY2bfTdr2bJlGTRoUObPn58kqaury1lnnZUpU6bkoYceysKFCzN58uSMGDFim65cCAAAsD12qpWubfXggw/m+eefz/PPP59+/fptdt9froC/fv36LFq0KH/+859b7/vmN7+ZDh065JRTTsnatWszZsyYfPvb396hswMAALuXXeZ3unYEv9PFjuB3ugAAdn675e90AQAA7IxEFwAAQEGiCwAAoCDRBQAAUJDoAgAAKEh0AQAAFCS6AAAAChJdAAAABYkuAACAgkQXAABAQaILAACgINEFAABQkOgCAAAoSHQBAAAUJLoAAAAKEl0AAAAFiS4AAICCRBcAAEBBogsAAKAg0QUAAFCQ6AIAAChIdAEAABQkugAAAAoSXQAAAAWJLgAAgIJEFwAAQEGiCwAAoCDRBQAAUJDoAgAAKEh0AQAAFCS6AAAACupU6QF4axZfMq7SIwAAAFthpQsAAKAg0QUAAFCQ6AIAAChIdAEAABQkugAAAAoSXQAAAAWJLgAAgIJEFwAAQEGiCwAAoCDRBQAAUJDoAgAAKEh0AQAAFCS6AAAAChJdAAAABYkuAACAgkQXAABAQaILAACgINEFAABQkOgCAAAoqFOlB6gmLS0tSZLm5uYKTwIAAFTSX5rgL42wNaKrDVatWpUk6d+/f4UnAQAAdgarVq1KXV3dVo+padmWNCNJsmnTpixfvjx77bVXampqKj3ObqO5uTn9+/fP0qVL061bt0qPQxXzXqK9eC/RXryXaA/eR5XR0tKSVatWpU+fPunQYevf2rLS1QYdOnRIv379Kj3Gbqtbt27+Q0K78F6ivXgv0V68l2gP3kc73t9a4foLF9IAAAAoSHQBAAAUJLrY6dXW1mbmzJmpra2t9ChUOe8l2ov3Eu3Fe4n24H2083MhDQAAgIKsdAEAABQkugAAAAoSXQAAAAWJLgAAgIJEFzu9K6+8MgMGDEiXLl0yfPjwzJ8/v9IjUWUeffTRfOADH0ifPn1SU1OTu+66q9IjUYVmzZqVo446KnvttVf23XffjB8/PosWLar0WFShOXPmZMiQIa0/ZDtixIjcf//9lR6LXcAll1ySmpqaXHjhhZUehb8iutip3XbbbZkyZUpmzpyZJ554IocffnjGjBmTlStXVno0qsiaNWty+OGH58orr6z0KFSxRx55JOeee25+9rOf5cEHH8z69etz4oknZs2aNZUejSrTr1+/XHLJJVm4cGEef/zxvPe9783JJ5+cp556qtKjUcUWLFiQq6++OkOGDKn0KGyBS8azUxs+fHiOOuqoXHHFFUmSTZs2pX///jn//PMzbdq0Ck9HNaqpqcmdd96Z8ePHV3oUqtxLL72UfffdN4888kiOO+64So9DlevevXsuvfTSnHXWWZUehSq0evXqvOtd78q3v/3tfPnLX87QoUMze/bsSo/F/2Gli53WunXrsnDhwowePbp1X4cOHTJ69OjMmzevgpMBJE1NTUle/8cybK+NGzfm1ltvzZo1azJixIhKj0OVOvfcczNu3LjN/s3EzqVTpQeAN/Pyyy9n48aN6dmz52b7e/bsmWeffbZCUwG8vup+4YUX5thjj83gwYMrPQ5V6Mknn8yIESPy2muv5e1vf3vuvPPOHHLIIZUeiyp066235oknnsiCBQsqPQpbIboAoI3OPffc/PrXv87//M//VHoUqtRBBx2UhoaGNDU15T//8z9zxhln5JFHHhFetMnSpUtzwQUX5MEHH0yXLl0qPQ5bIbrYafXo0SMdO3bMihUrNtu/YsWK9OrVq0JTAbu78847L/fee28effTR9OvXr9LjUKU6d+6cd77znUmSYcOGZcGCBbn88stz9dVXV3gyqsnChQuzcuXKvOtd72rdt3Hjxjz66KO54oorsnbt2nTs2LGCE/IXvtPFTqtz584ZNmxY5s6d27pv06ZNmTt3rs+9AztcS0tLzjvvvNx55535yU9+kne84x2VHoldyKZNm7J27dpKj0GVGTVqVJ588sk0NDS0bkceeWQmTpyYhoYGwbUTsdLFTm3KlCk544wzcuSRR+boo4/O7Nmzs2bNmkyePLnSo1FFVq9eneeff7719u9///s0NDSke/fu2W+//So4GdXk3HPPzS233JIf/OAH2WuvvdLY2JgkqaurS9euXSs8HdVk+vTpef/735/99tsvq1atyi233JKHH344P/rRjyo9GlVmr732esP3St/2trdln3328X3TnYzoYqc2YcKEvPTSS5kxY0YaGxszdOjQPPDAA2+4uAZszeOPP54TTjih9faUKVOSJGeccUZuuOGGCk1FtZkzZ06SZOTIkZvtv/7663PmmWfu+IGoWitXrsykSZPy4osvpq6uLkOGDMmPfvSjvO9976v0aEAhfqcLAACgIN/pAgAAKEh0AQAAFCS6AAAAChJdAAAABYkuAACAgkQXAABAQaILAACgINEFANvohhtuyN57713pMQCoMqILgF3avHnz0rFjx4wbN65N5w0YMCCzZ8/ebN+ECRPym9/8ph2nA2B3ILoA2KVde+21Of/88/Poo49m+fLlb+mxunbtmn333bedJgNgdyG6ANhlrV69OrfddlvOOeecjBs3LjfccMNm999zzz056qij0qVLl/To0SMf+tCHkiQjR47MCy+8kE9/+tOpqalJTU1Nki1/vHDOnDk54IAD0rlz5xx00EH593//983ur6mpyXe+85186EMfyp577pkDDzwwd999d+v9r7zySiZOnJj6+vp07do1Bx54YK6//vr2/8sAoGJEFwC7rP/4j//IoEGDctBBB+Uf//Efc91116WlpSVJct999+VDH/pQTjrppPziF7/I3Llzc/TRRydJ7rjjjvTr1y8XX3xxXnzxxbz44otbfPw777wzF1xwQaZOnZpf//rX+eQnP5nJkyfnoYce2uy4iy66KKeddlp+9atf5aSTTsrEiRPzpz/9KUnyT//0T3n66adz//3355lnnsmcOXPSo0ePgn8rAOxoNS1/+b8PAOxijj322Jx22mm54IILsmHDhvTu3Tu33357Ro4cmWOOOSYDBw7MzTffvMVzBwwYkAsvvDAXXnhh674bbrghF154YV599dXWxz/00ENzzTXXtB5z2mmnZc2aNbnvvvuSvL7S9cUvfjFf+tKXkiRr1qzJ29/+9tx///0ZO3ZsPvjBD6ZHjx657rrryvwlAFBxVroA2CUtWrQo8+fPz0c/+tEkSadOnTJhwoRce+21SZKGhoaMGjXqLT3HM888k2OPPXazfccee2yeeeaZzfYNGTKk9c9ve9vb0q1bt6xcuTJJcs455+TWW2/N0KFD87nPfS6PPfbYW5oJgJ2P6AJgl3Tttddmw4YN6dOnTzp16pROnTplzpw5+f73v5+mpqZ07dp1h82yxx57bHa7pqYmmzZtSpK8//3vb/3+2PLlyzNq1Kh85jOf2WGzAVCe6AJgl7Nhw4bcdNNN+frXv56GhobW7Ze//GX69OmT733vexkyZEjmzp37po/RuXPnbNy4cavPc/DBB+enP/3pZvt++tOf5pBDDmnTvPX19TnjjDNy8803Z/bs2Zt9XBGA6tep0gMAQHu7995788orr+Sss85KXV3dZvedcsopufbaa3PppZdm1KhROeCAA3L66adnw4YN+eEPf5jPf/7zSV7/Ttejjz6a008/PbW1tVu8uMVnP/vZnHbaaTniiCMyevTo3HPPPbnjjjvy4x//eJtnnTFjRoYNG5ZDDz00a9euzb333puDDz74rf0FALBTsdIFwC7n2muvzejRo98QXMnr0fX444+ne/fuuf3223P33Xdn6NChee9735v58+e3HnfxxRdn8eLFOeCAA1JfX7/F5xk/fnwuv/zyXHbZZTn00ENz9dVX5/rrr8/IkSO3edbOnTtn+vTpGTJkSI477rh07Ngxt956a5tfMwA7L1cvBAAAKMhKFwAAQEGiCwAAoCDRBQAAUJDoAgAAKEh0AQAAFCS6AAAAChJdAAAABYkuAACAgkQXAABAQaILAACgINEFAABQkOgCAAAo6P8DQNU1edJMvvEAAAAASUVORK5CYII=", "text/plain": [ "
" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "nb_samples = 10\n", "rewards = np.zeros((nb_actions, nb_samples))\n", "\n", "for a in actions:\n", " for play in range(nb_samples):\n", " rewards[a, play] = get_reward(Q_star, a, var=1.0)\n", "\n", "Q_t = np.mean(rewards, axis=1)\n", " \n", "plt.figure(figsize=(10, 6))\n", "plt.bar(actions, Q_t)\n", "plt.xlabel('Actions')\n", "plt.ylabel('$Q_t(a)$')\n", "plt.show()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "**Q:** Make a bar plot of the difference between the true values `Q_star` and the estimates `Q_t`. Conclude. Re-run the sampling cell with different numbers of samples." ] }, { "cell_type": "code", "execution_count": 7, "metadata": {}, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAA2AAAAINCAYAAABYjxyUAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjkuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8hTgPZAAAACXBIWXMAAA9hAAAPYQGoP6dpAAAtc0lEQVR4nO3de5TXBZ3/8dfAyIDJJS5CCDpiHAUNJlFZNC/JKIjHtDSxpRURyUxcFS1lt8DLFlamWLp411w13fKyqEkZppTOysVoFRUtM1lxQI/KACZymd8f/prdWYFwZvgMMzwe53zPcT7X98z5Hup5Pt/P51tSW1tbGwAAALa6Ns09AAAAwPZCgAEAABREgAEAABREgAEAABREgAEAABREgAEAABREgAEAABREgAEAABSktLkHaMk2bNiQpUuXpmPHjikpKWnucQAAgGZSW1ublStXpnfv3mnTZtPXuQRYIyxdujR9+/Zt7jEAAIBtxJIlS9KnT59NrhdgjdCxY8ckH/yRO3Xq1MzTAAAAzaWmpiZ9+/ata4RNEWCN8NePHXbq1EmAAQAAf/PWJA/hAAAAKIgAAwAAKIgAAwAAKIgAAwAAKIgAAwAAKIgAAwAAKIgAAwAAKIgAAwAAKIgAAwAAKIgAAwAAKIgAAwAAKIgAAwAAKIgAAwAAKIgAAwAAKIgAAwAAKIgAAwAAKIgAAwAAKIgAAwAAKEhpcw8AbHvKL3youUegib1y2dHNPQIAEFfAAAAACiPAAAAACiLAAAAACiLAAAAACiLAAAAACiLAAAAACiLAAAAACiLAAAAACiLAAAAACiLAAAAACiLAAAAACiLAAAAACiLAAAAACiLAAAAACiLAAAAACiLAAAAACiLAAAAACiLAAAAACiLAAAAACiLAAAAACiLAAAAACiLAAAAACiLAAAAACiLAAAAACiLAAAAACiLAAAAACtKqAuyaa65JeXl52rdvn6FDh2bu3LlbtN9dd92VkpKSHHfccVt3QAAAYLvWagLs7rvvzqRJkzJ16tQ8/fTTGTx4cEaMGJHly5dvdr9XXnkl559/fg4++OCCJgUAALZXrSbArrjiikyYMCHjxo3LwIEDc+2112bHHXfMzTffvMl91q9fnzFjxuTiiy9Ov379CpwWAADYHrWKAHv//fezYMGCVFZW1i1r06ZNKisrU1VVtcn9Lrnkkuy8884ZP378Fp1nzZo1qampqfcCAADYUq0iwN58882sX78+PXv2rLe8Z8+eqa6u3ug+v/3tb3PTTTflhhtu2OLzTJs2LZ07d6579e3bt1FzAwAA25dWEWAf1cqVK/MP//APueGGG9K9e/ct3m/y5MlZsWJF3WvJkiVbcUoAAKC1KW3uAZpC9+7d07Zt2yxbtqze8mXLlqVXr14f2v6Pf/xjXnnllRxzzDF1yzZs2JAkKS0tzeLFi7PHHnt8aL+ysrKUlZU18fQAAMD2olVcAWvXrl2GDBmS2bNn1y3bsGFDZs+enWHDhn1o+7322ivPPPNMFi5cWPf63Oc+l89+9rNZuHChjxYCAABbRau4ApYkkyZNytixY7PffvvlgAMOyPTp07N69eqMGzcuSXLyySdnl112ybRp09K+ffvss88+9fbv0qVLknxoOQAAQFNpNQE2evTovPHGG5kyZUqqq6tTUVGRWbNm1T2Y49VXX02bNq3igh8AANBCldTW1tY29xAtVU1NTTp37pwVK1akU6dOzT0ONJnyCx9q7hFoYq9cdnRzjwAArdqWtoFLQgAAAAURYAAAAAURYAAAAAURYAAAAAURYAAAAAURYAAAAAURYAAAAAURYAAAAAURYAAAAAURYAAAAAURYAAAAAURYAAAAAURYAAAAAURYAAAAAURYAAAAAURYAAAAAURYAAAAAUpbe4BAAA2p/zCh5p7BLaCVy47urlHgGbhChgAAEBBBBgAAEBBBBgAAEBBBBgAAEBBBBgAAEBBBBgAAEBBBBgAAEBBBBgAAEBBBBgAAEBBBBgAAEBBBBgAAEBBBBgAAEBBBBgAAEBBBBgAAEBBBBgAAEBBBBgAAEBBBBgAAEBBBBgAAEBBBBgAAEBBBBgAAEBBBBgAAEBBBBgAAEBBBBgAAEBBBBgAAEBBBBgAAEBBBBgAAEBBBBgAAEBBBBgAAEBBBBgAAEBBBBgAAEBBBBgAAEBBBBgAAEBBBBgAAEBBBBgAAEBBBBgAAEBBBBgAAEBBBBgAAEBBBBgAAEBBBBgAAEBBBBgAAEBBBBgAAEBBBBgAAEBBBBgAAEBBBBgAAEBBBBgAAEBBBBgAAEBBBBgAAEBBBBgAAEBBBBgAAEBBBBgAAEBBWlWAXXPNNSkvL0/79u0zdOjQzJ07d5Pb3nDDDTn44IPz8Y9/PB//+MdTWVm52e0BAAAaq9UE2N13351JkyZl6tSpefrppzN48OCMGDEiy5cv3+j2jz32WL70pS/l17/+daqqqtK3b98ceeSRee211wqeHAAA2F60mgC74oorMmHChIwbNy4DBw7Mtddemx133DE333zzRre/44478rWvfS0VFRXZa6+9cuONN2bDhg2ZPXt2wZMDAADbi1YRYO+//34WLFiQysrKumVt2rRJZWVlqqqqtugY7777btauXZuuXbtucps1a9akpqam3gsAAGBLtYoAe/PNN7N+/fr07Nmz3vKePXumurp6i45xwQUXpHfv3vUi7v+aNm1aOnfuXPfq27dvo+YGAAC2L60iwBrrsssuy1133ZX77rsv7du33+R2kydPzooVK+peS5YsKXBKAACgpStt7gGaQvfu3dO2bdssW7as3vJly5alV69em9338ssvz2WXXZZf/epXGTRo0Ga3LSsrS1lZWaPnBQAAtk+t4gpYu3btMmTIkHoP0PjrAzWGDRu2yf2+973v5dJLL82sWbOy3377FTEqAACwHWsVV8CSZNKkSRk7dmz222+/HHDAAZk+fXpWr16dcePGJUlOPvnk7LLLLpk2bVqS5Lvf/W6mTJmSO++8M+Xl5XX3iu20007Zaaedmu33AAAAWq9WE2CjR4/OG2+8kSlTpqS6ujoVFRWZNWtW3YM5Xn311bRp8z8X/GbMmJH3338/J5xwQr3jTJ06NRdddFGRowMAANuJVhNgSTJx4sRMnDhxo+see+yxej+/8sorW38gAACA/6VV3AMGAADQEggwAACAgggwAACAgggwAACAgggwAACAgggwAACAgggwAACAgggwAACAgggwAACAgggwAACAgggwAACAgggwAACAgggwAACAgggwAACAgggwAACAgggwAACAgggwAACAgggwAACAgggwAACAgggwAACAgggwAACAgggwAACAgggwAACAgggwAACAgggwAACAgggwAACAgggwAACAgggwAACAgggwAACAgggwAACAgggwAACAgggwAACAgggwAACAgggwAACAgggwAACAgggwAACAgggwAACAgggwAACAgggwAACAgggwAACAgggwAACAgggwAACAgggwAACAgggwAACAgggwAACAgggwAACAgggwAACAgggwAACAgpQ2xUHWrl2b6urqvPvuu+nRo0e6du3aFIcFAABoVRp8BWzlypWZMWNGDj300HTq1Cnl5eUZMGBAevTokd122y0TJkzIvHnzmnJWAACAFq1BAXbFFVekvLw8t9xySyorK3P//fdn4cKFefHFF1NVVZWpU6dm3bp1OfLIIzNy5Mi89NJLTT03AABAi9OgjyDOmzcvc+bMyd57773R9QcccEBOPfXUXHvttbnlllvym9/8Jv3792/UoAAAAC1dgwLsJz/5yRZtV1ZWlq9+9asNOQUAAECr4ymIAAAABWn0UxDXr1+fG2+8MYsXL06fPn0yePDgVFRUpFu3bk0xHwAAQKvR6AA766yzcs8996SysjJXX311SkpKsm7duuyyyy6pqKjIzJkzm2JOAACAFq/RH0G89957c9ttt+WOO+5IWVlZ5s+fn6uuuirvvfdedtttt6aYEQAAoFVo9BWwVatWZeDAgUmSHXbYIaWlpZk4cWLWrl2bpUuXNnpAAACA1qLRV8D69etXF1q77LJLXnvttSTJMccck9tvv72xhwcAAGg1Gh1gX/jCF/Lwww8nSQ499NDcfPPNSZLnnnsuf/nLXxp7eAAAgFaj0R9BvOiii+r++xvf+Eb233//9OjRIzU1NRk/fnxjDw8AANBqNDrA/rddd901ixYtygMPPJBu3brlmGOOacrDAwAAtGgNCrBXX301u+6660bXde/ePePGjau37LXXXssuu+zSkFMBAAC0Gg26B2z//ffP6aefnnnz5m1ymxUrVuSGG27IPvvsk3vuuafBAwIAALQWDboC9txzz+Xb3/52jjjiiLRv3z5DhgxJ79690759+7z99tt57rnnsmjRouy777753ve+l1GjRjX13AAAAC1Og66AdevWLVdccUVef/31XH311enfv3/efPPNvPTSS0mSMWPGZMGCBamqqhJfAAAA/1+jHsLRoUOHnHDCCTnhhBPqLf/LX/6SDh06NGowAACA1qbR3wO2MZ/5zGc+tOyFF17YGqeq55prrkl5eXnat2+foUOHZu7cuZvd/qc//Wn22muvtG/fPp/61Kfy85//fKvPCAAAbL+aNMAeeOCBfPe7382qVauyZMmSeutGjx7dlKf6kLvvvjuTJk3K1KlT8/TTT2fw4MEZMWJEli9fvtHtn3zyyXzpS1/K+PHj87vf/S7HHXdcjjvuuDz77LNbdU4AAGD71aQBts8++6Rjx4558803M3bs2Oyxxx455JBDMnr06Oywww5NeaoPueKKKzJhwoSMGzcuAwcOzLXXXpsdd9wxN99880a3v+qqqzJy5Mh8/etfz4ABA3LppZdm3333zdVXX71V5wQAALZfTfpFzLvvvnu+9rWvZZ999skhhxyS5IPvAPvzn/+cffbZpylPVc/777+fBQsWZPLkyXXL2rRpk8rKylRVVW10n6qqqkyaNKneshEjRuT+++/f5HnWrFmTNWvW1P1cU1PTuMEBAIDtSqMD7OWXX87vf//7lJaWZtCgQdltt93q4itJdtlll63+Jcxvvvlm1q9fn549e9Zb3rNnz03ee1ZdXb3R7aurqzd5nmnTpuXiiy9u/MBbSfmFDzX3CGwFr1x29HZxTlon/y61Pv5NoiXzb1Lr1NL+jWjwRxDXr1+fcePGpX///jn++ONz7LHHpl+/fjn88MPzzDPPNOWM24zJkydnxYoVda//e58bAADA5jQ4wL7zne9k5syZue6667J48eIsXLgwN954Y1auXJmDDjoojz76aFPOuVndu3dP27Zts2zZsnrLly1bll69em10n169en2k7ZOkrKwsnTp1qvcCAADYUg0OsB//+Me58sorc9ppp6V///4ZNGhQxo0bl3nz5mXKlCk57rjjsmzZstTU1OTxxx9vypk/pF27dhkyZEhmz55dt2zDhg2ZPXt2hg0bttF9hg0bVm/7JHnkkUc2uT0AAEBjNTjAlixZkoMPPnij684///yMHj06p556aoYMGZInn3yywQNuqUmTJuWGG27Ij3/84zz//PM544wzsnr16owbNy5JcvLJJ9d7SMfZZ5+dWbNm5Qc/+EFeeOGFXHTRRZk/f34mTpy41WcFAAC2Tw1+CEfXrl3z9ttvZ/fdd9/o+tNOOy3Dhg3LaaedlvPOO6/BA26p0aNH54033siUKVNSXV2dioqKzJo1q+5BG6+++mratPmf3jzwwANz55135pvf/Gb+6Z/+Kf3798/999+/VZ/WCAAAbN8aHGCHHXZYbr/99uy7774bXd+zZ8+Ulpbm+uuvb/BwH9XEiRM3eQXrscce+9CyL37xi/niF7+4lacCAAD4QIMD7IILLsjQoUMzZMiQjBkz5kPr58+fnz59+jRqOAAAgNakwfeAVVRUZMaMGRk7dmyOPfbY/PKXv8yyZcuyYsWKzJw5M+eee25Gjx7dlLMCAAC0aI36IuZTTz01/fr1yznnnJORI0empKQkSVJbW5sRI0Zk6tSpTTIkAABAa9CoAEs+uBds4cKFda/3338/gwcPztChQ5tiPgAAgFaj0QH2VxUVFamoqGiqwwEAALQ6Db4HDAAAgI9GgAEAABREgAEAABSkSQPsxRdfzLp165rykAAAAK1GkwbYgAED8vLLLzflIQEAAFqNJg2w2trapjwcAABAq+IeMAAAgIIIMAAAgIIIMAAAgIIIMAAAgIIIMAAAgIIIMAAAgII0aYBdcMEF6datW1MeEgAAoNUobcqDTZs2rSkPBwAA0Kr4CCIAAEBBBBgAAEBBBBgAAEBBmuQesLVr16a6ujrvvvtuevToka5duzbFYQEAAFqVBl8BW7lyZWbMmJFDDz00nTp1Snl5eQYMGJAePXpkt912y4QJEzJv3rymnBUAAKBFa1CAXXHFFSkvL88tt9ySysrK3H///Vm4cGFefPHFVFVVZerUqVm3bl2OPPLIjBw5Mi+99FJTzw0AANDiNOgjiPPmzcucOXOy9957b3T9AQcckFNPPTXXXnttbrnllvzmN79J//79GzUoAABAS9egAPvJT36yRduVlZXlq1/9akNOAQAA0Oo0+iEc69evz4033pjFixenT58+GTx4cCoqKtKtW7emmA8AAKDVaHSAnXXWWbnnnntSWVmZq6++OiUlJVm3bl122WWXVFRUZObMmU0xJwAAQIvX6O8Bu/fee3PbbbfljjvuSFlZWebPn5+rrroq7733XnbbbbemmBEAAKBVaPQVsFWrVmXgwIFJkh122CGlpaWZOHFi1q5dm6VLlzZ6QAAAgNai0VfA+vXrVxdau+yyS1577bUkyTHHHJPbb7+9sYcHAABoNRodYF/4whfy8MMPJ0kOPfTQ3HzzzUmS5557Ln/5y18ae3gAAIBWo9EfQbzooovq/vsb3/hG9t9///To0SM1NTUZP358Yw8PAADQajQ6wP63XXfdNYsWLcoDDzyQbt265ZhjjmnKwwMAALRoTRpgSdK9e/eMGzeuqQ8LAADQ4jXoHrBXX331I23/1wdzAAAAbM8aFGD7779/Tj/99MybN2+T26xYsSI33HBD9tlnn9xzzz0NHhAAAKC1aNBHEJ977rl8+9vfzhFHHJH27dtnyJAh6d27d9q3b5+33347zz33XBYtWpR999033/ve9zJq1KimnhsAAKDFadAVsG7duuWKK67I66+/nquvvjr9+/fPm2++mZdeeilJMmbMmCxYsCBVVVXiCwAA4P9r1EM4OnTokBNOOCEnnHBCU80DAADQajX6i5gBAADYMgIMAACgIB85wH7zm98kSZ544okmHwYAAKA1+8gB9vDDD6eqqioPPfTQ1pgHAACg1fpIAXbxxRdn3bp1Ofzww7N+/fpccsklW2suAACAVucjBdjUqVPTv3//XHrppenfv3+mTJmyteYCAABodT7yRxDXrVuX888/P+vXr98a8wAAALRaHznAzjjjjCTJ6aef3uTDAAAAtGYeQw8AAFCQ0sYe4OWXX87vf//7lJaWZtCgQdltt92aYi4AAIBWp8EBtn79+px22mm57bbbUltbmyQpKSnJoYcemquuuiqf+tSnmmxIAACA1qDBH0H8zne+k5kzZ+a6667L4sWLs3Dhwtx4441ZuXJlDjrooDz66KNNOScAAECL1+ArYD/+8Y9z5ZVX5uSTT65bNmjQoIwbNy6XX355jjvuuLz00kvp0KFDfve73+XQQw9tkoEBAABaqgZfAVuyZEkOPvjgja47//zzM3r06Jx66qkZMmRInnzyyQYPCAAA0Fo0OMC6du2at99+e5PrTzvttDz88MP57Gc/m/POO6+hpwEAAGg1Ghxghx12WG6//fZNru/Zs2dKS0tz/fXXp127dg09DQAAQKvR4HvALrjgggwdOjRDhgzJmDFjPrR+/vz56dOnT6OGAwAAaE0afAWsoqIiM2bMyNixY3Psscfml7/8ZZYtW5YVK1Zk5syZOffcczN69OimnBUAAKBFa9QXMZ966qnp169fzjnnnIwcOTIlJSVJktra2owYMSJTp05tkiEBAABag0YFWPLBvWALFy6se73//vsZPHhwhg4d2hTzAQAAtBqNDrC/qqioSEVFRVMdDgAAoNVp8D1gAAAAfDQCDAAAoCACDAAAoCACDAAAoCACDAAAoCACDAAAoCCtIsDeeuutjBkzJp06dUqXLl0yfvz4rFq1arPbn3XWWdlzzz3ToUOH7LrrrvnHf/zHrFixosCpAQCA7U2rCLAxY8Zk0aJFeeSRR/Lggw9mzpw5+cpXvrLJ7ZcuXZqlS5fm8ssvz7PPPptbb701s2bNyvjx4wucGgAA2N402RcxN5fnn38+s2bNyrx587LffvslSX70ox9l1KhRufzyy9O7d+8P7bPPPvvknnvuqft5jz32yLe//e18+ctfzrp161Ja2uL/LAAAwDaoxV8Bq6qqSpcuXeriK0kqKyvTpk2bPPXUU1t8nBUrVqRTp06bja81a9akpqam3gsAAGBLtfgAq66uzs4771xvWWlpabp27Zrq6uotOsabb76ZSy+9dLMfW0ySadOmpXPnznWvvn37NnhuAABg+7PNBtiFF16YkpKSzb5eeOGFRp+npqYmRx99dAYOHJiLLrpos9tOnjw5K1asqHstWbKk0ecHAAC2H9vszU7nnXdeTjnllM1u069fv/Tq1SvLly+vt3zdunV566230qtXr83uv3LlyowcOTIdO3bMfffdlx122GGz25eVlaWsrGyL5gcAAPi/ttkA69GjR3r06PE3txs2bFjeeeedLFiwIEOGDEmSPProo9mwYUOGDh26yf1qamoyYsSIlJWVZebMmWnfvn2TzQ4AALAx2+xHELfUgAEDMnLkyEyYMCFz587NE088kYkTJ+akk06qewLia6+9lr322itz585N8kF8HXnkkVm9enVuuumm1NTUpLq6OtXV1Vm/fn1z/joAAEArts1eAfso7rjjjkycODHDhw9PmzZtcvzxx+eHP/xh3fq1a9dm8eLFeffdd5MkTz/9dN0TEj/5yU/WO9af/vSnlJeXFzY7AACw/WgVAda1a9fceeedm1xfXl6e2traup8PO+ywej8DAAAUocV/BBEAAKClEGAAAAAFEWAAAAAFEWAAAAAFEWAAAAAFEWAAAAAFEWAAAAAFaRXfAwYAAH/LK5cd3dwjgCtgAAAARRFgAAAABRFgAAAABRFgAAAABRFgAAAABRFgAAAABRFgAAAABRFgAAAABRFgAAAABRFgAAAABRFgAAAABRFgAAAABRFgAAAABRFgAAAABRFgAAAABRFgAAAABRFgAAAABRFgAAAABRFgAAAABRFgAAAABRFgAAAABRFgAAAABRFgAAAABRFgAAAABRFgAAAABRFgAAAABRFgAAAABRFgAAAABRFgAAAABSlt7gFoOq9cdnRzjwAAAGyGK2AAAAAFEWAAAAAFEWAAAAAFEWAAAAAFEWAAAAAFEWAAAAAFEWAAAAAFEWAAAAAFEWAAAAAFEWAAAAAFEWAAAAAFEWAAAAAFEWAAAAAFEWAAAAAFEWAAAAAFEWAAAAAFEWAAAAAFEWAAAAAFEWAAAAAFEWAAAAAFEWAAAAAFEWAAAAAFEWAAAAAFEWAAAAAFEWAAAAAFEWAAAAAFEWAAAAAFEWAAAAAFEWAAAAAFaRUB9tZbb2XMmDHp1KlTunTpkvHjx2fVqlVbtG9tbW2OOuqolJSU5P7779+6gwIAANu1VhFgY8aMyaJFi/LII4/kwQcfzJw5c/KVr3xli/adPn16SkpKtvKEAAAASWlzD9BYzz//fGbNmpV58+Zlv/32S5L86Ec/yqhRo3L55Zend+/em9x34cKF+cEPfpD58+fnE5/4RFEjAwAA26kWfwWsqqoqXbp0qYuvJKmsrEybNm3y1FNPbXK/d999N3//93+fa665Jr169SpiVAAAYDvX4q+AVVdXZ+edd663rLS0NF27dk11dfUm9zv33HNz4IEH5thjj93ic61ZsyZr1qyp+7mmpuajDwwAAGy3ttkrYBdeeGFKSko2+3rhhRcadOyZM2fm0UcfzfTp0z/SftOmTUvnzp3rXn379m3Q+QEAgO3TNnsF7Lzzzsspp5yy2W369euXXr16Zfny5fWWr1u3Lm+99dYmP1r46KOP5o9//GO6dOlSb/nxxx+fgw8+OI899thG95s8eXImTZpU93NNTY0IAwAAttg2G2A9evRIjx49/uZ2w4YNyzvvvJMFCxZkyJAhST4IrA0bNmTo0KEb3efCCy/MaaedVm/Zpz71qVx55ZU55phjNnmusrKylJWVfYTfAgAA4H9sswG2pQYMGJCRI0dmwoQJufbaa7N27dpMnDgxJ510Ut0TEF977bUMHz48t912Ww444ID06tVro1fHdt111+y+++5F/woAAMB2Ypu9B+yjuOOOO7LXXntl+PDhGTVqVD7zmc/k+uuvr1u/du3aLF68OO+++24zTgkAAGzvWvwVsCTp2rVr7rzzzk2uLy8vT21t7WaP8bfWAwAANFaruAIGAADQEggwAACAgggwAACAgggwAACAgggwAACAgggwAACAgggwAACAgggwAACAgggwAACAgggwAACAgggwAACAgggwAACAgggwAACAgggwAACAgggwAACAgggwAACAgggwAACAgggwAACAgggwAACAgggwAACAgggwAACAgggwAACAgggwAACAgggwAACAgggwAACAgggwAACAgggwAACAgggwAACAgggwAACAgggwAACAgggwAACAgggwAACAgggwAACAgggwAACAgggwAACAgggwAACAgggwAACAgggwAACAgggwAACAgggwAACAgggwAACAgggwAACAgggwAACAgggwAACAgggwAACAgggwAACAgggwAACAgggwAACAgggwAACAgggwAACAgggwAACAgggwAACAgggwAACAgggwAACAgggwAACAgggwAACAgggwAACAgggwAACAgggwAACAgggwAACAgggwAACAgggwAACAgggwAACAgggwAACAgggwAACAgggwAACAgggwAACAgggwAACAgpQ29wBN4a233spZZ52VBx54IG3atMnxxx+fq666KjvttNNm96uqqso///M/56mnnkrbtm1TUVGRX/ziF+nQoUNBkwO0bq9cdnRzjwAA25RWcQVszJgxWbRoUR555JE8+OCDmTNnTr7yla9sdp+qqqqMHDkyRx55ZObOnZt58+Zl4sSJadOmVfxJAACAbVBJbW1tbXMP0RjPP/98Bg4cmHnz5mW//fZLksyaNSujRo3Kf//3f6d3794b3e/v/u7vcsQRR+TSSy9t8LlramrSuXPnrFixIp06dWrwcQAAgJZtS9ugxV/uqaqqSpcuXeriK0kqKyvTpk2bPPXUUxvdZ/ny5Xnqqaey884758ADD0zPnj1z6KGH5re//e1mz7VmzZrU1NTUewEAAGypFh9g1dXV2XnnnestKy0tTdeuXVNdXb3RfV5++eUkyUUXXZQJEyZk1qxZ2XfffTN8+PC89NJLmzzXtGnT0rlz57pX3759m+4XAQAAWr1tNsAuvPDClJSUbPb1wgsvNOjYGzZsSJKcfvrpGTduXD796U/nyiuvzJ577pmbb755k/tNnjw5K1asqHstWbKkQecHAAC2T9vsUxDPO++8nHLKKZvdpl+/funVq1eWL19eb/m6devy1ltvpVevXhvd7xOf+ESSZODAgfWWDxgwIK+++uomz1dWVpaysrItmB4AAODDttkA69GjR3r06PE3txs2bFjeeeedLFiwIEOGDEmSPProo9mwYUOGDh260X3Ky8vTu3fvLF68uN7yF198MUcddVTjhwcAANiIbfYjiFtqwIABGTlyZCZMmJC5c+fmiSeeyMSJE3PSSSfVPQHxtddey1577ZW5c+cmSUpKSvL1r389P/zhD/Ozn/0sf/jDH/Ktb30rL7zwQsaPH9+cvw4AANCKbbNXwD6KO+64IxMnTszw4cPrvoj5hz/8Yd36tWvXZvHixXn33Xfrlp1zzjl57733cu655+att97K4MGD88gjj2SPPfZojl8BAADYDrT47wFrTr4HDAAASLaj7wEDAABoKQQYAABAQQQYAABAQQQYAABAQQQYAABAQQQYAABAQQQYAABAQQQYAABAQQQYAABAQUqbe4CWrLa2NskH33oNAABsv/7aBH9thE0RYI2wcuXKJEnfvn2beRIAAGBbsHLlynTu3HmT60tq/1aisUkbNmzI0qVL07Fjx5SUlDT3ONuNmpqa9O3bN0uWLEmnTp2aexxaMO8lmoL3EU3Fe4mm4r3UPGpra7Ny5cr07t07bdps+k4vV8AaoU2bNunTp09zj7Hd6tSpk39UaBLeSzQF7yOaivcSTcV7qXibu/L1Vx7CAQAAUBABBgAAUBABRotTVlaWqVOnpqysrLlHoYXzXqIpeB/RVLyXaCreS9s2D+EAAAAoiCtgAAAABRFgAAAABRFgAAAABRFgAAAABRFgtCjXXHNNysvL0759+wwdOjRz585t7pFogebMmZNjjjkmvXv3TklJSe6///7mHokWaNq0adl///3TsWPH7LzzzjnuuOOyePHi5h6LFmjGjBkZNGhQ3ZfmDhs2LA8//HBzj0ULd9lll6WkpCTnnHNOc4/C/yHAaDHuvvvuTJo0KVOnTs3TTz+dwYMHZ8SIEVm+fHlzj0YLs3r16gwePDjXXHNNc49CC/b444/nzDPPzH/+53/mkUceydq1a3PkkUdm9erVzT0aLUyfPn1y2WWXZcGCBZk/f34OP/zwHHvssVm0aFFzj0YLNW/evFx33XUZNGhQc4/CRngMPS3G0KFDs//+++fqq69OkmzYsCF9+/bNWWedlQsvvLCZp6OlKikpyX333ZfjjjuuuUehhXvjjTey88475/HHH88hhxzS3OPQwnXt2jXf//73M378+OYehRZm1apV2XffffOv//qv+Zd/+ZdUVFRk+vTpzT0W/4srYLQI77//fhYsWJDKysq6ZW3atEllZWWqqqqacTKAD6xYsSLJB//HGRpq/fr1ueuuu7J69eoMGzasucehBTrzzDNz9NFH1/v/TGxbSpt7ANgSb775ZtavX5+ePXvWW96zZ8+88MILzTQVwAc2bNiQc845JwcddFD22Wef5h6HFuiZZ57JsGHD8t5772WnnXbKfffdl4EDBzb3WLQwd911V55++unMmzevuUdhMwQYADTSmWeemWeffTa//e1vm3sUWqg999wzCxcuzIoVK/Kzn/0sY8eOzeOPPy7C2GJLlizJ2WefnUceeSTt27dv7nHYDAFGi9C9e/e0bds2y5Ytq7d82bJl6dWrVzNNBZBMnDgxDz74YObMmZM+ffo09zi0UO3atcsnP/nJJMmQIUMyb968XHXVVbnuuuuaeTJaigULFmT58uXZd99965atX78+c+bMydVXX501a9akbdu2zTghf+UeMFqEdu3aZciQIZk9e3bdsg0bNmT27Nk+Iw80i9ra2kycODH33XdfHn300ey+++7NPRKtyIYNG7JmzZrmHoMWZPjw4XnmmWeycOHCutd+++2XMWPGZOHCheJrG+IKGC3GpEmTMnbs2Oy333454IADMn369KxevTrjxo1r7tFoYVatWpU//OEPdT//6U9/ysKFC9O1a9fsuuuuzTgZLcmZZ56ZO++8M//xH/+Rjh07prq6OknSuXPndOjQoZmnoyWZPHlyjjrqqOy6665ZuXJl7rzzzjz22GP5xS9+0dyj0YJ07NjxQ/egfuxjH0u3bt3cm7qNEWC0GKNHj84bb7yRKVOmpLq6OhUVFZk1a9aHHswBf8v8+fPz2c9+tu7nSZMmJUnGjh2bW2+9tZmmoqWZMWNGkuSwww6rt/yWW27JKaecUvxAtFjLly/PySefnNdffz2dO3fOoEGD8otf/CJHHHFEc48GbAW+BwwAAKAg7gEDAAAoiAADAAAoiAADAAAoiAADAAAoiAADAAAoiAADAAAoiAADAAAoiAADgAa49dZb06VLl+YeA4AWRoABsN2oqqpK27Ztc/TRR3+k/crLyzN9+vR6y0aPHp0XX3yxCacDYHsgwADYbtx0000566yzMmfOnCxdurRRx+rQoUN23nnnJpoMgO2FAANgu7Bq1arcfffdOeOMM3L00Ufn1ltvrbf+gQceyP7775/27dune/fu+fznP58kOeyww/LnP/855557bkpKSlJSUpJk4x9BnDFjRvbYY4+0a9cue+65Z/7t3/6t3vqSkpLceOON+fznP58dd9wx/fv3z8yZM+vWv/322xkzZkx69OiRDh06pH///rnlllua/o8BQLMRYABsF/793/89e+21V/bcc898+ctfzs0335za2tokyUMPPZTPf/7zGTVqVH73u99l9uzZOeCAA5Ik9957b/r06ZNLLrkkr7/+el5//fWNHv++++7L2WefnfPOOy/PPvtsTj/99IwbNy6//vWv62138cUX58QTT8x//dd/ZdSoURkzZkzeeuutJMm3vvWtPPfcc3n44Yfz/PPPZ8aMGenevftW/KsAULSS2r/+rw8AtGIHHXRQTjzxxJx99tlZt25dPvGJT+SnP/1pDjvssBx44IHp169fbr/99o3uW15ennPOOSfnnHNO3bJbb70155xzTt5555264++99965/vrr67Y58cQTs3r16jz00ENJPrgC9s1vfjOXXnppkmT16tXZaaed8vDDD2fkyJH53Oc+l+7du+fmm2/eOn8EAJqdK2AAtHqLFy/O3Llz86UvfSlJUlpamtGjR+emm25KkixcuDDDhw9v1Dmef/75HHTQQfWWHXTQQXn++efrLRs0aFDdf3/sYx9Lp06dsnz58iTJGWeckbvuuisVFRX5xje+kSeffLJRMwGw7RFgALR6N910U9atW5fevXuntLQ0paWlmTFjRu65556sWLEiHTp0KGyWHXbYod7PJSUl2bBhQ5LkqKOOqrvfbOnSpRk+fHjOP//8wmYDYOsTYAC0auvWrcttt92WH/zgB1m4cGHd6/e//3169+6dn/zkJxk0aFBmz569yWO0a9cu69ev3+x5BgwYkCeeeKLesieeeCIDBw78SPP26NEjY8eOze23357p06fX+0gjAC1faXMPAABb04MPPpi3334748ePT+fOneutO/7443PTTTfl+9//foYPH5499tgjJ510UtatW5ef//znueCCC5J8cA/YnDlzctJJJ6WsrGyjD8b4+te/nhNPPDGf/vSnU1lZmQceeCD33ntvfvWrX23xrFOmTMmQIUOy9957Z82aNXnwwQczYMCAxv0BANimuAIGQKt20003pbKy8kPxlXwQYPPnz0/Xrl3z05/+NDNnzkxFRUUOP/zwzJ07t267Sy65JK+88kr22GOP9OjRY6PnOe6443LVVVfl8ssvz957753rrrsut9xySw477LAtnrVdu3aZPHlyBg0alEMOOSRt27bNXXfd9ZF/ZwC2XZ6CCAAAUBBXwAAAAAoiwAAAAAoiwAAAAAoiwAAAAAoiwAAAAAoiwAAAAAoiwAAAAAoiwAAAAAoiwAAAAAoiwAAAAAoiwAAAAAoiwAAAAAry/wC3T7SoR/nsIQAAAABJRU5ErkJggg==", "text/plain": [ "
" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "plt.figure(figsize=(10, 6))\n", "plt.bar(actions, Q_t - Q_star)\n", "plt.xlabel('Actions')\n", "plt.ylabel('$Q^*(a) - Q_t(a)$')\n", "plt.show()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "**Q:** To better understand the influence of the number of samples on the accuracy of the sample average, create a `for` loop over the preceding code, with a number of samples increasing from 1 to 100. For each value, compute the **mean square error** (mse) between the estimates `Q_t` and the true values `Q^*`.\n", "\n", "The mean square error is simply defined over the `N = nb_actions` actions as:\n", "\n", "$$\\epsilon = \\frac{1}{N} \\, \\sum_{a=0}^{N-1} (Q_t(a) - Q^*(a))^2$$\n", "\n", "At the end of the loop, plot the evolution of the mean square error with the number of samples. You can append each value of the mse in an empty list and then plot it with `plt.plot`, for example. " ] }, { "cell_type": "code", "execution_count": 8, "metadata": {}, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAzoAAAH5CAYAAABJUkuHAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjkuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8hTgPZAAAACXBIWXMAAA9hAAAPYQGoP6dpAABeQ0lEQVR4nO3dd3xV9f3H8fcdyc0ggxCSEAhT9pYljrpQHMWq1aJ1UNpqtWhRfi5slVoHaqu1Koqj1tEiuLXOIgqI7KXsvSGb5Gbf3HvP74+be5NIArlJ7j1JfD0fjzwg9557zzfJCdz3/X6+n6/FMAxDAAAAANCGWM0eAAAAAAA0N4IOAAAAgDaHoAMAAACgzSHoAAAAAGhzCDoAAAAA2hyCDgAAAIA2h6ADAAAAoM2xmz2AhvB6vTp8+LDi4uJksVjMHg4AAAAAkxiGoaKiIqWnp8tqrX/eplUEncOHDysjI8PsYQAAAABoIQ4cOKAuXbrUe3+rCDpxcXGSfF9MfHy8yaMBAAAAYBan06mMjIxARqhPqwg6/nK1+Ph4gg4AAACAEy5poRkBAAAAgDaHoAMAAACgzSHoAAAAAGhzCDoAAAAA2hyCDgAAAIA2h6ADAAAAoM0h6AAAAABocwg6AAAAANocgg4AAACANoegAwAAAKDNIegAAAAAaHMIOgAAAADanKCDzuLFizVhwgSlp6fLYrHogw8+OO7x7733ns477zx17NhR8fHxGjt2rL744ovGjhcAAAAATijooFNSUqKhQ4dq1qxZDTp+8eLFOu+88/Tpp59qzZo1OvvsszVhwgStW7cu6MECAAAAQENYDMMwGv1gi0Xvv/++Lr300qAeN3DgQE2cOFH3339/g453Op1KSEhQYWGh4uPjGzFSAAAAAG1BQ7OBPYxjkiR5vV4VFRUpKSmp3mMqKipUUVER+NzpdIZjaAAAAADaiLA3I/jb3/6m4uJi/eIXv6j3mJkzZyohISHwkZGREcYRAgAAAGjtwhp05syZowceeEBvvfWWUlJS6j1u+vTpKiwsDHwcOHAgjKNsvDkr9mv6exvk9Ta6GhAAAABAMwhb6drcuXP129/+Vm+//bbGjRt33GMdDoccDkeYRtZ8/va/bcovcelXp3ZX37Q4s4cDAAAA/GiFZUbnzTff1OTJk/Xmm2/q4osvDscpw67S41V+iUuSVF7pMXk0AAAAwI9b0DM6xcXF2rlzZ+DzPXv2aP369UpKSlLXrl01ffp0HTp0SK+//rokX7napEmT9I9//ENjxoxRZmamJCk6OloJCQnN9GWY72hVyJF8oQcAAACAeYKe0Vm9erWGDx+u4cOHS5KmTZum4cOHB1pFHzlyRPv37w8c/+KLL8rtdmvKlCnq1KlT4GPq1KnN9CW0DHm1gg5rdAAAAAAzBT2jc9ZZZ+l4W++8+uqrtT5fuHBhsKdolfKKmdEBAAAAWoqwt5duq/JKqvf9IegAAAAA5iLoNBNmdAAAAICWg6DTTGrO6LhYowMAAACYiqDTTPJrNiNwM6MDAAAAmImg00xya5Suub0EHQAAAMBMBJ1mkldM6RoAAADQUhB0mgmlawAAAEDLQdBpJnRdAwAAAFoOgk4zqHB7VFThDnxO0AEAAADMRdBpBjXL1iTW6AAAAABmI+g0g5pla5LkZkYHAAAAMBVBpxnk1ui4JlG6BgAAAJiNoNMMfli6VknpGgAAAGAqgk4z+GHpmosZHQAAAMBUBJ1mkFvyg9I19tEBAAAATEXQaQb5VTM6SbGRkiS3l9I1AAAAwEwEnWaQV7VGJzU+ShKlawAAAIDZCDrNwB900uIdkihdAwAAAMxG0GkGeVXtpdMSoiXRXhoAAAAwG0GnGfi7rqVVla7RXhoAAAAwF0GniUpdbpVVeiRJnRJYowMAAAC0BASdJvLP5jjsViXEREiS3AQdAAAAwFQEnSbyNyLoEBupSLvv20npGgAAAGAugk4T5VdtFtqhnUMRVn/QYUYHAAAAMBNBp4lyq0rXOrSLVITNIok1OgAAAIDZCDpN5F+jkxQbqQg7MzoAAABAS0DQaSJ/6VpyO4cibb5vp5s1OgAAAICpCDpN5J/R6RAbqQgbMzoAAABAS0DQaaLckurSNbt/jY6boAMAAACYiaDTRHWVrtFeGgAAADAXQaeJajUjoHQNAAAAaBEIOk1gGEb1hqE12ku7vYYMg1kdAAAAwCwEnSYornAH1uN0iHUE2ktLlK8BAAAAZiLoNIG/bC0m0qboSFtgjY5E+RoAAABgJoJOE9QsW5Mku9USuI+gAwAAAJiHoNMEecW+jmsdYh2SJJvVIktV1nERdAAAAADTEHSaIDCjE+ub0bFYLDU6r7FGBwAAADALQacJ8n9QuiYpsE7HzYwOAAAAYBqCThPk+kvX2jkCt/lbTLNGBwAAADAPQacJ/F3X/KVrkmSvmtFxuSldAwAAAMxC0GmC45WuMaMDAAAAmIeg0wS5P+i6JlG6BgAAALQEBJ0m8M/oJNUoXfN3XaO9NAAAAGAegk4jeb1GIOgk12pG4O+6xhodAAAAwCwEnUZyllfK7fWFmfaxEYHbI+ys0QEAAADMRtBpJP9moXFRdjnstsDtEVbW6AAAAABmI+g0kr+1dM2yNanmGh1K1wAAAACzEHQaKa+q41rNRgRSjdI1NzM6AAAAgFkIOo3kL13r8IOgE1nVXtrtJegAAAAAZiHoNJK/dK0DpWsAAABAi0PQaaS8Ev9mobVndOw2StcAAAAAsxF0GilQutbuB2t0bHRdAwAAAMxG0GkkfzOCH5auRdrYRwcAAAAwG0GnkQJrdH7YdY01OgAAAIDpCDqNlF9v6ZrvW+pmRgcAAAAwDUGnETxeQ/ml/hmdH3Rds7NGBwAAADAbQacRCkpdMqoq09rHRNS6L8LqX6ND6RoAAABgFoJOI/g7rrWPiQi0k/arXqPDjA4AAABgFoJOI+TW03FNqlG6xj46AAAAgGmCDjqLFy/WhAkTlJ6eLovFog8++OCEj1m4cKFOPvlkORwOnXTSSXr11VcbMdSWw9+IIOkHHdck2ksDAAAALUHQQaekpERDhw7VrFmzGnT8nj17dPHFF+vss8/W+vXrddttt+m3v/2tvvjii6AH21L4W0sntzs26PhL1yq9rNEBAAAAzGIP9gEXXnihLrzwwgYfP3v2bPXo0UNPPPGEJKl///5asmSJ/v73v2v8+PHBnr5F8G8WWteMjt1G6RoAAABgtpCv0Vm2bJnGjRtX67bx48dr2bJl9T6moqJCTqez1kdL4m9G8MPW0lKNGR1K1wAAAADThDzoZGZmKjU1tdZtqampcjqdKisrq/MxM2fOVEJCQuAjIyMj1MMMyvFK16rX6FC6BgAAAJilRXZdmz59ugoLCwMfBw4cMHtIteSV+EvX6p/Rob00AAAAYJ6g1+gEKy0tTVlZWbVuy8rKUnx8vKKjo+t8jMPhkMNxbIhoKQKla3U2I/Ct0XETdAAAAADThHxGZ+zYsVqwYEGt2+bPn6+xY8eG+tQhc9yua3ZK1wAAAACzBR10iouLtX79eq1fv16Sr330+vXrtX//fkm+srPrr78+cPxNN92k3bt366677tLWrVv13HPP6a233tLtt9/ePF9BmFV6vCosq5RUT+malWYEAAAAgNmCDjqrV6/W8OHDNXz4cEnStGnTNHz4cN1///2SpCNHjgRCjyT16NFDn3zyiebPn6+hQ4fqiSee0Msvv9xqW0sfrSpbs1qkxOiIY+73l66xRgcAAAAwT9BrdM466ywZRv1lWa+++mqdj1m3bl2wp2qRcqvK1pJiHbJaLcfcX126RtABAAAAzNIiu661ZPmBPXSOXZ8j1Wgv7WaNDgAAAGAWgk6Q/K2l6+q4JlW3l3Z7mdEBAAAAzELQCZK/dK1Du7rbX9v9a3TcBB0AAADALASdIOX7Z3ROVLpGe2kAAADANASdIPn30Kkv6ETYaEYAAAAAmI2gE6RA17V61+j4StfcXuO43ekAAAAAhA5BJ0jVpWt1r9Hxt5eWKF8DAAAAzELQCVJeVXvp5HpmdPxrdCTK1wAAAACzEHSClBfYMLTuoGOvsYkoQQcAAAAwB0EnCOWVHhVXuCXV317aZrXIUpV1XAQdAAAAwBQEnSDkV5WtRdgsio+y13mMxWKp0XmNNToAAACAGep+tY46JURHaNYvT1ZJhVsWi6Xe4yJtVrncXlWyaSgAAABgCoJOEGIddl08pNMJj6tuMU3QAQAAAMxA6VoI2KtK11xuStcAAAAAMxB0QiAysEaHGR0AAADADASdEPCXrhF0AAAAAHMQdELA33WN9tIAAACAOQg6IeAPOm7aSwMAAACmIOiEAKVrAAAAgLkIOiEQQTMCAAAAwFQEnRCoXqND6RoAAABgBoJOCETYq2Z03MzoAAAAAGYg6IRAJGt0AAAAAFMRdEIgsEbHS+kaAAAAYAaCTgjYbZSuAQAAAGYi6IQA7aUBAAAAcxF0QiCS9tIAAACAqQg6IUB7aQAAAMBcBJ0Q8AcdNzM6AAAAgCkIOiHAGh0AAADAXASdEAi0l6Z0DQAAADAFQScEqtfoMKMDAAAAmIGgEwIR9qrSNfbRAQAAAExB0AkB2ksDAAAA5iLohEBgjY6XNToAAACAGQg6IWC3UboGAAAAmImgEwIRlK4BAAAApiLohEAk7aUBAAAAUxF0QoD20gAAAIC5CDohEOFfo0PQAQAAAExB0AkB/4yOm9I1AAAAwBQEnRCgGQEAAABgLoJOCPhL11ijAwAAAJiDoBMCEXZmdAAAAAAzEXRCINBe2s0aHQAAAMAMBJ0QCDQj8DKjAwAAAJiBoBMCdv8aHTdBBwAAADADQScEAqVrtJcGAAAATEHQCQHaSwMAAADmIuiEgL+9tNtryOtlVgcAAAAIN4JOCPjbS0tSJQ0JAAAAgLAj6IRAhLX62+pmnQ4AAAAQdgSdEPCXrkms0wEAAADMQNAJAZvVIktV1nERdAAAAICwI+iEgMViqdF5jdI1AAAAINwIOiES2EuHTUMBAACAsCPohEh1i2mCDgAAABBuBJ0QsVfN6LjclK4BAAAA4UbQCZFA6RrNCAAAAICwI+iEiL90jaADAAAAhF+jgs6sWbPUvXt3RUVFacyYMVq5cuVxj3/qqafUt29fRUdHKyMjQ7fffrvKy8sbNeDWwt91jfbSAAAAQPgFHXTmzZunadOmacaMGVq7dq2GDh2q8ePHKzs7u87j58yZo3vuuUczZszQli1b9M9//lPz5s3Tvffe2+TBt2S0lwYAAADME3TQefLJJ3XDDTdo8uTJGjBggGbPnq2YmBi98sordR6/dOlSnXbaafrlL3+p7t276/zzz9fVV199wlmg1i7QdY0ZHQAAACDsggo6LpdLa9as0bhx46qfwGrVuHHjtGzZsjofc+qpp2rNmjWBYLN79259+umnuuiii+o9T0VFhZxOZ62P1iaCZgQAAACAaezBHJybmyuPx6PU1NRat6empmrr1q11PuaXv/ylcnNzdfrpp8swDLndbt10003HLV2bOXOmHnjggWCG1uJUr9GhdA0AAAAIt5B3XVu4cKEeeeQRPffcc1q7dq3ee+89ffLJJ3rwwQfrfcz06dNVWFgY+Dhw4ECoh9nsIuxVMzpuZnQAAACAcAtqRic5OVk2m01ZWVm1bs/KylJaWlqdj7nvvvt03XXX6be//a0kafDgwSopKdGNN96oP/7xj7Jaj81aDodDDocjmKG1OJG0lwYAAABME9SMTmRkpEaMGKEFCxYEbvN6vVqwYIHGjh1b52NKS0uPCTM2m02SZBhtt6wrsEbH23a/RgAAAKClCmpGR5KmTZumSZMmaeTIkRo9erSeeuoplZSUaPLkyZKk66+/Xp07d9bMmTMlSRMmTNCTTz6p4cOHa8yYMdq5c6fuu+8+TZgwIRB42iK7jdI1AAAAwCxBB52JEycqJydH999/vzIzMzVs2DB9/vnngQYF+/fvrzWD86c//UkWi0V/+tOfdOjQIXXs2FETJkzQww8/3HxfRQsUQekaAAAAYBqL0Qrqx5xOpxISElRYWKj4+Hizh9Mg97z7veauOqA7zu+jW87pbfZwAAAAgDahodkg5F3XfqxoLw0AAACYh6ATImwYCgAAAJiHoBMi/jU6boIOAAAAEHYEnRCpntGhdA0AAAAIN4JOiFSv0WFGBwAAAAg3gk6IRNir2kuzjw4AAAAQdgSdEImkGQEAAABgGoJOiLBGBwAAADAPQSdE7FVd15jRAQAAAMKPoBMi7KMDAAAAmIegEyKRlK4BAAAApiHohAjtpQEAAADzEHRCJII1OgAAAIBpCDoh4p/RcVO6BgAAAIQdQSdEaEYAAAAAmIegEyL+0jXW6AAAAADhR9AJkQg7MzoAAACAWQg6IRJoL+1mjQ4AAAAQbgSdEGGNDgAAAGAegk6I2GkvDQAAAJiGoBMigdI12ksDAAAAYUfQCRFK1wAAAADzEHRCxN9e2u015PUyqwMAAACEE0EnRPztpSWp0susDgAAABBOBJ0QibBWf2vdrNMBAAAAwoqgEyL+0jWJdToAAABAuBF0QsRmtchSlXVcBB0AAAAgrAg6IWKxWGp0XqN0DQAAAAgngk4IBfbScTOjAwAAAIQTQSeE7FXrdFijAwAAAIQXQSeEKF0DAAAAzEHQCaFA6RozOgAAAEBYEXRCKILSNQAAAMAUBJ0Q8peu0V4aAAAACC+CTgixRgcAAAAwB0EnhPyla25mdAAAAICwIuiEUATNCAAAAABTEHRCqHqNDqVrAAAAQDgRdEIowl41o+NmRgcAAAAIJ4JOCEXSXhoAAAAwBUEnhOxW1ugAAAAAZiDohFCgdI01OgAAAEBYEXRCKILSNQAAAMAUBJ0QiqS9NAAAAGAKgk4I0V4aAAAAMAdBJ4TYMBQAAAAwB0EnhPxrdNwEHQAAACCsCDohVD2jQ+kaAAAAEE4EnRCqXqPDjA4AAAAQTgSdEIqwV7WXdhN0AAAAgHAi6IQQ7aUBAAAAcxB0Qshu9W8YyhodAAAAIJwIOiEUYWdGBwAAADADQSeE2EcHAAAAMAdBJ4QiaS8NAAAAmIKgE0K0lwYAAADMQdAJoQibvxkBQQcAAAAIJ4JOCLFGBwAAADAHQSeE/EHHzRodAAAAIKwIOiHkL11jjQ4AAAAQXo0KOrNmzVL37t0VFRWlMWPGaOXKlcc9vqCgQFOmTFGnTp3kcDjUp08fffrpp40acGvCPjoAAACAOezBPmDevHmaNm2aZs+erTFjxuipp57S+PHjtW3bNqWkpBxzvMvl0nnnnaeUlBS988476ty5s/bt26fExMTmGH+LFmgv7aZ0DQAAAAinoIPOk08+qRtuuEGTJ0+WJM2ePVuffPKJXnnlFd1zzz3HHP/KK68oPz9fS5cuVUREhCSpe/fuxz1HRUWFKioqAp87nc5gh9ki2Om6BgAAAJgiqNI1l8ulNWvWaNy4cdVPYLVq3LhxWrZsWZ2P+eijjzR27FhNmTJFqampGjRokB555BF5PJ56zzNz5kwlJCQEPjIyMoIZZotB1zUAAADAHEEFndzcXHk8HqWmpta6PTU1VZmZmXU+Zvfu3XrnnXfk8Xj06aef6r777tMTTzyhhx56qN7zTJ8+XYWFhYGPAwcOBDPMFiNQukbXNQAAACCsgi5dC5bX61VKSopefPFF2Ww2jRgxQocOHdJf//pXzZgxo87HOBwOORyOUA8t5JjRAQAAAMwRVNBJTk6WzWZTVlZWrduzsrKUlpZW52M6deqkiIgI2Wy2wG39+/dXZmamXC6XIiMjGzHs1sHfXtrtNeT1GrJaLSaPCAAAAPhxCKp0LTIyUiNGjNCCBQsCt3m9Xi1YsEBjx46t8zGnnXaadu7cKa+3elZj+/bt6tSpU5sOOVJ1e2lJqvQyqwMAAACES9D76EybNk0vvfSSXnvtNW3ZskU333yzSkpKAl3Yrr/+ek2fPj1w/M0336z8/HxNnTpV27dv1yeffKJHHnlEU6ZMab6vooWKsNYIOqzTAQAAAMIm6DU6EydOVE5Oju6//35lZmZq2LBh+vzzzwMNCvbv3y9rjRf4GRkZ+uKLL3T77bdryJAh6ty5s6ZOnaq77767+b6KFspfuiZJbtbpAAAAAGFjMQyjxU81OJ1OJSQkqLCwUPHx8WYPp8EMw1DPez+VYUgr/3iuUuKizB4SAAAA0Ko1NBsEXbqGhrNYLDU6r7X4PAkAAAC0GQSdEAvspeOmdA0AAAAIF4JOiNmr1umwlw4AAAAQPgSdEKN0DQAAAAg/gk6IBUrXmNEBAAAAwoagE2IRlK4BAAAAYUfQCTF/6ZqLoAMAAACEDUEnxFijAwAAAIQfQSfEAqVrtJcGAAAAwoagE2L+GR23l6ADAAAAhAtBJ8Sq1+hQugYAAACEC0EnxCLsVWt0KF0DAAAAwoagE2KRtJcGAAAAwo6gE2J2KxuGAgAAAOFG0AmxQOkaa3QAAACAsCHohFgEpWsAAABA2BF0QizSRukaAAAAEG4EnRCjvTQAAAAQfgSdEItgRgcAAAAIO4JOiAXW6LCPDgAAABA2BJ0Q88/ouL2UrgEAAADhQtAJseo1OszoAAAAAOFC0AmxCDulawAAAEC4EXRCjPbSAAAAQPgRdELMbvVvGMoaHQAAACBcCDohFmFnRgcAAAAIN4JOiLGPDgAAABB+BJ0Qq16jQ+kaAAAAEC4EnRCjvTQAAAAQfgSdEIuw+ZsREHQAAACAcCHohBhrdAAAAIDwI+iEmD/ouFmjAwAAAIQNQSfE/KVrrNEBAAAAwoegE2LsowMAAACEH0EnxALtpd2UrgEAAADhQtAJMTtd1wAAAICwI+iEGPvoAAAAAOFH0AmxSLquAQAAAGFH0Akx9tEBAAAAwo+gE2L+9tJuryGvl1kdAAAAIBwIOiFmt1V/iyu9zOoAAAAA4UDQCbHImkGHdToAAABAWBB0QsxfuiZJbtbpAAAAAGFB0Akxm9UiS1XWocU0AAAAEB4EnRCzWCw1Oq9RugYAAACEA0EnDPzrdCrdzOgAAAAA4UDQCQN71Tod9tIBAAAAwoOgEwb+0jXW6AAAAADhQdAJA3/pmps1OgAAAEBYEHTCIILSNQAAACCsCDphQOkaAAAAEF4EnTCw014aAAAACCuCThhE+kvXaC8NAAAAhAVBJwz8pWtuL0EHAAAACAeCThhUr9GhdA0AAAAIB4JOGETYq9boULoGAAAAhAVBJwwiaS8NAAAAhBVBJwzsVn/XNYIOAAAAEA4EnTDwl66xRgcAAAAIj0YFnVmzZql79+6KiorSmDFjtHLlygY9bu7cubJYLLr00ksbc9pWK6KqdM3NjA4AAAAQFkEHnXnz5mnatGmaMWOG1q5dq6FDh2r8+PHKzs4+7uP27t2rO+64Q2eccUajB9taRdooXQMAAADCKeig8+STT+qGG27Q5MmTNWDAAM2ePVsxMTF65ZVX6n2Mx+PRNddcowceeEA9e/Zs0oBbI9pLAwAAAOEVVNBxuVxas2aNxo0bV/0EVqvGjRunZcuW1fu4v/zlL0pJSdFvfvObBp2noqJCTqez1kdrZqfrGgAAABBWQQWd3NxceTwepaam1ro9NTVVmZmZdT5myZIl+uc//6mXXnqpweeZOXOmEhISAh8ZGRnBDLPFCZSusY8OAAAAEBYh7bpWVFSk6667Ti+99JKSk5Mb/Ljp06ersLAw8HHgwIEQjjL0/KVrbi+lawAAAEA42IM5ODk5WTabTVlZWbVuz8rKUlpa2jHH79q1S3v37tWECRMCt3m9vlkNu92ubdu2qVevXsc8zuFwyOFwBDO0Fq16jQ4zOgAAAEA4BDWjExkZqREjRmjBggWB27xerxYsWKCxY8cec3y/fv20YcMGrV+/PvBxySWX6Oyzz9b69etbfUlaQ0XYq9boULoGAAAAhEVQMzqSNG3aNE2aNEkjR47U6NGj9dRTT6mkpESTJ0+WJF1//fXq3LmzZs6cqaioKA0aNKjW4xMTEyXpmNvbMtpLAwAAAOEVdNCZOHGicnJydP/99yszM1PDhg3T559/HmhQsH//flmtIV360+rYrf6ua6zRAQAAAMIh6KAjSbfccotuueWWOu9buHDhcR/76quvNuaUrVqEnTU6AAAAQDgx9RIGga5rBB0AAAAgLAg6YVC9RofSNQAAACAcCDphQHtpAAAAILwIOmFgt/mbERB0AAAAgHAg6IQB7aUBAACA8CLohEF1MwLW6AAAAADhQNAJg4iq0jXW6AAAAADhQdAJA/8+OpSuAQAAAOFB0AmDwBodN6VrAAAAQDgQdMKArmsAAABAeBF0woB9dAAAAIDwIuiEQSRd1wAAAICwIuiEQQT76AAAAABhRdAJA397abfXkNfLrA4AAAAQagSdMLDbqr/NlV5mdQAAAIBQI+iEQWTNoMM6HQAAACDkCDph4C9dk6RKNzM6AAAAQKgRdMLAZrXIUpV1KF0DAAAAQo+gEwYWi6VG5zVK1wAAAIBQI+iEiX+dDqVrAAAAQOgRdMLEXrVOh710AAAAgNAj6ISJv3TNRdABAAAAQo6gEyb+0jU3a3QAAACAkCPohEkEpWsAAABA2BB0woTSNQAAACB8CDphYqe9NAAAABA2BJ0wifSXrtFeGgAAAAg5gk6YVG8YStABAAAAQo2gEyaBoOOldA0AAAAINYJOmMQ6bJKkZbtyTR4JAAAA0PYRdMLk2lO6SZLeXHlA76w5aPJoAAAAgLaNoBMmZ/VN0W3jekuS7n1/gzYeKjR5RAAAAEDbRdAJoz+c01vn9kuRy+3V795Yo/wSl9lDAgAAANokgk4YWa0WPTlxmLp3iNGhgjL94c11ctOFDQAAAGh2BJ0wS4iO0AvXjVRMpE1Ldubqb//bbvaQAAAAgDaHoGOCvmlxevyKIZKk2Yt26dMNR0weEQAAANC2EHRM8tMh6brxJz0lSXe8/Z12ZBWZPCIAAACg7SDomOiu8X11aq8OKnV5dOMba+QsrzR7SAAAAECbQNAxkd1m1TNXD1fnxGjtyS3RC4t2mT0kAAAAoE0g6JisQzuHpp3XR5K0dFeeyaMBAAAA2gaCTgswukeSJGnDwUKVuTwmjwYAAABo/Qg6LUCX9tFKi4+S22to3YGjZg8HAAAAaPUIOi2AxWLRqKpZndV7CToAAABAUxF0WojR3dtLklbtzTd5JAAAAEDrR9BpIUZ2983orN13VG6P1+TRAAAAAK0bQaeF6Jsap7gou0pcHm05wuahAAAAQFMQdFoIq9Wikd185WsrKV8DAAAAmoSg04JUNyQg6AAAAABNQdBpQUZXrdNZtTdfhmGYPBoAAACg9SLotCCDuyQo0m5VbrFLe3JLzB4OAAAA0GoRdFoQh92mYV0SJbGfDgAAANAUBJ0WZlQPGhIAAAAATUXQaWH8++nQkAAAAABoPIJOCzOiW3tZLNLevFJlF5WbPRwAAACgVSLotDDxURHqlxYvSVq1h3U6AAAAQGMQdFqg0d1963RWUb4GAAAANApBpwXybxxK0AEAAAAah6DTAo2qakiw5YhTReWVJo8GAAAAaH0IOi1QanyUuibFyGtIa/cXmD0cAAAAoNUh6LRQI/3rdPZQvgYAAAAEq1FBZ9asWerevbuioqI0ZswYrVy5st5jX3rpJZ1xxhlq37692rdvr3Hjxh33ePiM7s46HQAAAKCxgg468+bN07Rp0zRjxgytXbtWQ4cO1fjx45WdnV3n8QsXLtTVV1+tr7/+WsuWLVNGRobOP/98HTp0qMmDb8v8DQnWHyhQhdtj8mgAAACA1sViGIYRzAPGjBmjUaNG6dlnn5Ukeb1eZWRk6NZbb9U999xzwsd7PB61b99ezz77rK6//voGndPpdCohIUGFhYWKj48PZritlmEYGvnQl8orcendm8dqRLcks4cEAAAAmK6h2SCoGR2Xy6U1a9Zo3Lhx1U9gtWrcuHFatmxZg56jtLRUlZWVSkqq/4V7RUWFnE5nrY8fG4vFUr1OZy8bhwIAAADBCCro5ObmyuPxKDU1tdbtqampyszMbNBz3H333UpPT68Vln5o5syZSkhICHxkZGQEM8w2w99mmoYEAAAAQHDC2nXt0Ucf1dy5c/X+++8rKiqq3uOmT5+uwsLCwMeBAwfCOMqWwx90Vu87Kq83qApDAAAA4EfNHszBycnJstlsysrKqnV7VlaW0tLSjvvYv/3tb3r00Uf15ZdfasiQIcc91uFwyOFwBDO0NmlgerxiIm0qLKvUjuxi9U2LM3tIAAAAQKsQ1IxOZGSkRowYoQULFgRu83q9WrBggcaOHVvv4x5//HE9+OCD+vzzzzVy5MjGj/ZHxm6zanjXREnSyjbeZvrdNQc1Zc5alVfSYQ4AAABNF3Tp2rRp0/TSSy/ptdde05YtW3TzzTerpKREkydPliRdf/31mj59euD4xx57TPfdd59eeeUVde/eXZmZmcrMzFRxcXHzfRVt2I9lnc6T87frk++PaNH2HLOHAgAAgDYgqNI1SZo4caJycnJ0//33KzMzU8OGDdPnn38eaFCwf/9+Wa3V+en555+Xy+XSFVdcUet5ZsyYoT//+c9NG/2PwMldfZ3XNh4qNHkkoVNYVqlDBWWSpN05JSaPBgAAAG1B0EFHkm655Rbdcsstdd63cOHCWp/v3bu3MadAlf6dfL3B9+SVqNTlVkxko35kLdr2rKLA33flMNMHAACApgtr1zUEr2OcQ8ntHDIMaVtm0Ykf0AptzSToAAAAoHkRdFqB/p183da2HGmbQWdbZvWGsLuyi2UYtNIGAABA0xB0WoEBVeVrW444T3Bk61RzpspZ7lZeicvE0QAAAKAtIOi0Av3bcNAxDCNQuma3WiT5ZnUAAACApiDotAL+oLM1s0heb9sq6zpSWK6icrfsVovG9PS10t5F5zUAAAA0EUGnFejZMVaRNquKK9w6eLTM7OE0K3/ZWs+Oseqb6gt0NCQAAABAUxF0WoEIm1W9U9tJkja3sfI1f9la37R49UqJlUTQAQAAQNMRdFqJtrpOx99xrV9anHp19IU5Ng0FAABAUxF0Wom2GnQCMzqp1UHnwNFSlVd6zBwWAAAAWjmCTisR2EsnM/RBp6TCrc83HlGlxxvS81R6vIEytb5pcUpuF6n4KLsMQ9qbx6wOAAAAGo+g00r499I5kF+movLKkJ7r2a936qZ/r9UzC3aE9Dx7cktU6TEUG2lT58RoWSwW9aya1dmVTdABAABA4xF0WonEmEh1SoiSVF3uFSqr9+ZLkt5ZczCk7az9Hdf6pMXJWrWHTvU6HRoSAAAAoPEIOq1IONbpeL2GNh/2Pf/hwnKtqgo9oeAPOv3S4gK30XkNAAAAzYGg04oE1umEMOjszy9Viau6EcAH6w+F7Fw1GxH4+Wd02DQUAAAATUHQaUX8Mzqbj4SudM2/T09MpE2S9Mn3R1ThDk0HtG1ZvnP1TYsP3Naro29GZ3dOsQwjdGVzAAAAaNsIOq2IP+hsy3TKE6K1M5sOF0qSJgxJV1p8lJzlbn29NafZz1Nc4daB/DJJtUvXuibFyma1qMTlUZazotnPCwAAgB8Hgk4r0r1DrKIirCqv9Ias/bJ/fc6gzvH62bB0SdIH65q/fG17lm9WKiXOofaxkYHbI+1WdUuKkcQ6HQAAADQeQacVsVktgTKvUK3T8ZeuDUiP18+GdZYkfbU1W4VlzdvS2t+IoG+N2Ry/QItpgg4AAAAaiaDTygwIYUOC3OIKZTkrZLFI/dLi1b9TnPqmxsnl8eqzDUea9Vx1dVzzC3ReyyboAAAAoHEIOq1MdYvp5m9I4C9b69EhVrEOuywWiy4d7pvVae7ua1szj21E4NcruWovnVw6rwEAAKBxCDqtTCj30vGXrfVPrw4fl1St01m+O1+HC8qa5TyGYTCjAwAAgJAi6LQy/mBwpLBcBaWuZn1u/4zOgE7VQadzYrTG9EiSJH303eFmOU9OUYWOllbKapFOSml3zP09q2Z0DheWq6TC3SznBAAAwI8LQaeViYuKUEZStKTqGZjmUrMRQU2B8rVm6r7m3yi0e3KsoiJsx9zfPjZSHao6se2hfA0AAACNQNBphfqnNf86nTKXR7urupwN/EHQuWhQJ0XarNqaWdQsJXPHK1vz61m1cSid1wAAANAYBJ1WKBTrdLZmOuU1pOR2DqXERdW6LyEmQmf36yipeZoS+Gd0+qTWH3R6BVpMM6MDAACA4BF0WqGGBp1Sl1vO8obtf1Nf2ZrfZVXlax+tPyyv12joUOu0Lct3ruPN6PRiLx0AAAA0AUGnFfI3C9iRVaxKj7fOY5zllbroH9/ozMe/1tGSEzctqKsRQU1n9U1RfJRdRwrLtWJPfiNHLnm8hnZk+cJLXa2l/ei8BgAAgKYg6LRCXdpHq53DLpfHq931lHb9+cNN2ptXqqOllfpiU+YJn3NTVdD54focv6gImy4a3ElS05oS7M0rUYXbq6gIq7omxdR7nL/z2p7ckibPIAEAAODHh6DTClmtlkDZV13la598f0Tv1Qgjn248ftDxeI3ABp71la5J1d3XPt14ROWVnqDHLVU3IuiTGieb1VLvcV3aRyvSZlWF26tDzbR/DwAAAH48CDqtVH3rdLKd5frjBxskSZcM9W32uXRn7nH33NmTW6LySq+iI2zq3iG23uNGd09SekKUisrd+nprdqPG7W9E0Pc4jQgkyW6zqnuyb8aHdToAAAAIFkGnlfIHnZp76RiGoTvf+V4FpZUa1DleT/xiqPqlxcntNTR/c1a9z+V/jv6djj/LYrVadMmwqqYEjdw8dFvVzFHf4zQi8KPzGgAAABqLoNNK9e/kL12r3kvn3yv2a9H2HDnsVv39F8MUYbPqwkG+dTWfHad8bdPhQknHL1vz++kQ3/Mt2p7TqPK16j10TnwuOq8BAACgsQg6rVTftDhZLFJucYVyiiq0O6dYD3+yWZJ0z4X91LuqNOyiwWmSpG925NTbarq641rCCc87MD1e6QlRKnV59O3O3KDGXOpya19+aWD8J+LfNHQ3QQcAAABBIui0UjGRdvWoWk+z8VChbp+3XuWVXp1+UrImje0eOK53apxOSmmnSo+hBVuOLV8zDKM66DRgRsdisWjcgFRJOm45XF12ZBXLMKQOsZHqGOc44fGUrgEAAKCxCDqtmH+dzv0fbdR3BwsVH2XXX68cIusP1tlcNMg3q/PphmPL13KKKpRX4pLVcvwNPGs6ryrofLklO6jWz/6ytYbM5kjVMzo5RRUqLGvYxqcAAACARNBp1fzrdA7k+9ovP3TZYHVKiD7muAsHV6+rKa5w17rPv39Or47tFBVha9B5x/TooDiHXbnFFVp3oKDB490aZNCJi4pQarxv5ofyNQAAAASDoNOK+Wd0JF8raX876R/qlxanHsmxcrm9+uoHbaH9HdcaUrbmF2m36ux+KZKCK1/bluUMjKeh/BuH1rcxamMVV7j18je79eH6xm9+CgAAgJaLoNOKDe/aXu0cdnVOjNaDPxtU73EWi0UXVpWvfbbhSK37qhsRNDzoSNXla//bfPzNSP0Mw6hRutbwc/VK8ZWvNVfnNZfbq1e/3aMzH/9aD32yRVPnrtf6IGalAAAA0DoQdFqxpNhIfXXHmfrstjOUEBNx3GMvqipf+3pbtkpd1eVr/tbSA9NP3HGtprP6dlSEzaLdOSUNCiHLduUpt9glh92qPqntGnye5mox7fUa+nD9IZ375EL9+b+blVfiUqTdd/k//MlmGUbD1xoBAACg5SPotHIpcVGKjzp+yJF8baG7JsWovNKrhdtyJPnKt/bm+do9+9f7NFRcVIRO6dlBUsPK115YvFuSdOXILoqJtDf4PE3tvGYYhhZtz9FPn1miqXPX60B+mTrGOfTwZYO0YNqZioqwatXeo/piU3Ad5AAAANCyEXR+JCwWiy4c7O++5itf21q1PictPkod2p243fMPnT/Q93wnCjrbMou0aHuOLBbpt6f3DOoc/s5r+/JK5PZ4G/y4LGe5/r18nya+sFyTXlmpzUecinPYdef4vlp051m6Zkw3ZSTF6MYzfON59LMtcrkb/vwAAABo2Qg6PyIXDfKVr321NVvllZ5GNSKo6bz+vnU6a/cfVU5RRb3HvVg1m3PBwDR1T44N6hzpCdGKirCq0mPowNGyeo8zDENbM516ZsEOXfLsEo15ZIH+9MFGrdybr0ibVb89vYcW33W2ppx9Uq0ZpRvP7KXkdg7tzSvVnBX7GjSmCrdHmw87KXcDAABowRpeQ4RWb0iXBHVOjNahgjIt2p6jTYd8QWdgI4NOWkKUhnRJ0PcHC7VgS5auGt31mGMyC8v10Xe+zmY3/iS42RxJslot6pncTpuPOPXUl9vVpX20DEPyGpIhQzJ8JXiLd+QE2mxLksUiDc9I1LgBqfrZsM7qnHhs221Jauewa9p5fXTv+xv0jwU7dNnJXZQQXX8pYFF5pa59eYW+O1ioq0Zl6OHLBsv2g32LAAAAYD6Czo+Iv/vay0v26LMNRwLrXoLtuFbTef1T9f3BQs3fXHfQ+de3e1TpMTS6e5KGd23fqHP0S4vT5iNOfbj+8HGPc9itOqN3ssb1T9U5/VOUEhfVoOf/xcgu+te3e7Qju1jPfb1T0y/qX+dxZS6PfvPaan130NfAYe6qA3KWV+rvE4fJYW/YHkQAAAAID4LOj8yFgzvp5SV79OWWbLmq1rw0tnRNks4bmKon5m/Xkp25KnW5a5WFFZVXas6K/ZIaN5vjd/t5fZQc5wisobFYJKvFIkvV321Wq4Z3TdQZvZODanTgZ7dZde/F/TX5X6v0r2/36tpTfOt3anK5vbr5P2u0ck++4hx23Xx2Lz01f4c+3ZCpovLVmn3tCMU6+HWCObKc5Zr21npdNLiTrhnTzezhAADQIvDK7EdmeEai0uKjlOksl+Qr3cpoH3OCR9Wvb2qcuibFaH9+qRZvz9UFVfv1SNKbK/erqMKtXh1jdU7VBqONkZEUo3vrmWVpLmf16ajTT0rWkp25evyLbXrm6uGB+zxeQ7fPW6+F23IUFWHVK5NHaVT3JA3tkqgbXl+tb3bk6pqXV+jVyaOUGBMZ0nECdXnmqx36dmeelu7KU6eEKJ3TL9XsIQEAYDqaEfzIWK2WWmFkQKd4WZuwxsRisQQ2D63Zfc3l9uqVJXsl+WZzmnKOcLBYLJp+UT9ZLNJ/vzusdfuPSvLtvzP9ve/1yYYjirBZ9MJ1IzWqe5Ik6bSTkjXnhlOUGBOh9QcK9IsXlimrKkCieXm8hv69fJ/eXXNQhWWVZg+nRckuKtdbqw9KkgxDmjp3vfbkNq4dOwAAbQlB50fIv3mo1LSyNT9/0FmwNSvQAvq/3x1WprNcHeMcunR45yafIxwGpifo5yd3kSQ98ukWGYahBz/ZrLdWH5TVIj191XCd2adjrccMy0jU278bq7T4KG3PKtbPn1+qvbzIbHazF+3Snz7YqP97+zuNfGi+fv3qKkJPlVeW7JXL7dXQjESN7NZeReVu3fj6ahVXuE/8YAAA2jCCzo/QiG7t1THOt29OUxoR+I3s1l6JMREqKK3U6n1HZRiGXvrG11L6V6d2b1UL9e84v29gE9FfVa3ZkaTHrxiqC2sExJp6p8bp7ZvGqnuHGB08WqYrZi/TtsyiMI76+EpdjXvBW1zhVoXb08yjCd7qvfl6cv52SVLXpBhVegx9tTU7EHp+8+oqvbf2oIrKf3yhx1leqf8s97VFv+Xsk/TctScrNd6hHdnFmjZvvbxeWqADAH68CDo/QjarRTMmDNBFg9N00ZC6X7wHw26z6tx+1eVri3fkamtmkWIibbq2lS2MTkuICmwiumh7jiTpgUsG6ooRXY77uIykGL1906ka0CleucUV+u3rq1RQ6gr5eE9k1tc7NXDGF7r1zXUqLG1YEPB6Db2yZI9OfnC+fvbst6bODBSUujR17np5vIYuHZauRXeepfm3/0S3jeut3intVOkxtGBrtqa99Z3OeWKRdmYXmzZWM7yxbJ+KKtzqk9pO5/bzdRqcfe0IRdqs+t/mLD379U6zhwgAgGkIOj9SPx2SrueuGaF2zdQprOY6nRcX75IkXTWqqxJi6t+TpqW68cxegRmvO8f31aRTuzfocR3jHJpzwxh1TYrRgfwy3WbyO+qLtufob//bJsPwlRJe+I/FWrYr77iPyXaW61evrtJfPt4sl9urrZlF+uP7G0zZHNUwDN397vc6VFCm7h1i9NBlg2WxWNQ7NU63jeuj+dPO1P9u/4mmnttbXdpHK6eoQr98afmPpnSwvNKjf327R5J081m9Auvghndtr4cuHSRJ+vuX27VgS1a9zwEAQFtmMVrB9u5Op1MJCQkqLCxUfHzTS63Q/Epdbg3/y3xVVLWAtlktWnTnWerShI5uZtqTW6ID+aU6o3eyLJbgGilsOlyoy59bqgq3V384t7emndcnRKOs35HCMl389BLll7h0wcA0bcl0al9eqSwWX3OI/zuvryLttd/nmL85S3e/+73yS1xy2K26fmw3vfLtXnm8hh69fHCd+ySF0hvL9+m+DzYqwmbRezefpsFdEuo9Nr/EpatfXK5tWUVKT4jSvN+NPaZFeLhUerx66svtWruvQB6voUqvV26PIbfXkNvjldtrqGtSjB6+bFCTfj/eWLZX9324SZ0To7XwzrMUYav987zvg416Y/k+xTns+uCW09SrY7umfmlo4ZbsyNXhgjJdObJL0P9u1afC7dH7aw9pZPf2OiklrlmeEz5HCssUabOqQzuH2UMBWp2GZgNmdNAsYiLtOv2k5MDnFw/u1GpDjiT1SI7VT/p0bNSLhYHpCZp5+WBJ0tMLdjTLO+oer6Hyyoatl6n0eHXLnHXKL3FpUOd4PXXVMH36hzM0cWSGDEN6YdFuXTrrW+3M9q0jKnN59Mf3N+iG11crv8Sl/p3i9fGtp+uPFw/QHef3lSTN+GiTthxxnvDchmHojeX7NOU/a7X+QEGjv94tR5x68OPNkqS7L+h33JAjSUmxkfr3b8eoV8dYHS4s19UvLdfhgrJGn7+xDMPQ9Pc2aNbXu7Rsd55W7s3Xuv0F2nCoUFuOOLUju1h7cku0aHuOLp21VN818nvk9nj1wmLfOrjfndnzmJAjSff9dIBGdW+vogpfc4LWvobpcEGZXly8Sw9+vFlLd+bKw/qjWt5dc1DXvbJCd737vR7/YluzPe8D/92se97boAnPfKsvNmU22/M2Vn6JS3/9YqsWbMkyZaa5uazck68z/7pQZ/11obZmnvjfVgCNw4wOms3clft1z3sbJEkf33q6BnU+/ovTtu7+Dzfq9WX7FB9l139vPV3dOsQ26nmW7MjVPe99r8KySj16+RBdfIJ1VQ99vFkvL9mjuCi7Prn1DHXtUB04P9+YqXve+14FpZVy2K36/Vkn6aPvDmlXjq/c64YzeuiO8X0DDSS8XkO/fm2VFm7LUc/kWH106+n1lju6PV7d/9GmwCaxknTpsHTdeUE/dU6MbvDXW+pya8IzS7Qrp0Tn9EvRPyeNbHDgzHKWa+ILy7Q3r1TdO8Tord+NVUp8VIPP3VSPfb5Vzy/cJZvVonsu6Kf0xGjZbRZF2CyyW62y2ywyDOnBjzdra2aRoiKsemri8Fot3xvig3WHdNu89eoQG6kld5+j6Mi6G37kFFVowjNLlOks19ieHfS3XwwN6mdhtqMlLn268Yg+XH9YK/fk17ovNd6hS4am62fDOmtgenyzzWAEY+nOXB0qKNPPT+5iagv9/353WFPnrlPN7PfAJQMbXHZbn0++P6Ipc9YGPrdYpLvG99NNZ/Y05ftd6nLr6heX67uDhZKkIV0SdPu4Pjqrb+PelDLL7pxiXf78UhVUrZvslBCl939/mtISwvdvVWtw8GipEmMim63EHm1LQ7MBQQfNprCsUr98abmGdEnQzMuHmD0c07ncXl314jKt3V+gfmlxev/3p9X7grQuzvJKzfx0i95ceaDW7ZPGdtO9F/evs5vd5xszddO/10iSXrhuhMYPPPYFdJazXHe8/Z2+2ZEbuC0lzqEnfzFMp/dOPub4/BKXLvrHN8p0lutnw9L11MRhx7yoKCqv1JQ567R4e44sFgU2XzUMyWG36oYzeuqms3o16D+su9/5XvNWH1BqvEOf/uGMoMs6DheU6RcvLNPBo2U6KaWd5t54ipLDUBryypI9+kvVLNRjPx+siaPqL/UrrnDrljlrtXCb7/v1x4v66zen92jQizWv19CF//hG27KKdOf4vppy9knHPf67qj2eKtxeRUfYdNu43vr16T3qnAVqCUpdbs3fnKUP1x/W4u05ctd49T6mR5K6JsXoi02ZcpZXN8k4KaWdLh3mCz3hKFk8XFCmv/x3sz6vmuG4alSGHrlssClh5/ONmZoyZ608XkNXjcpQ58RoPTF/uywWadYvT661nUAw9ueV6uKnv1FRhVu/O7OnSis8eqOqw98VI7rokcsGH1P+Gkpuj1c3vL5aX2/LUVyUXW6PobKqWe5hGYm6/bw++kkjSo3DLa+4Qpc/v1T78ko1NCNRxeWV2pVTov6d4vXW705RXFTrW9cq+Wazm/N7/86ag7rrne/UMc6heTeOVffkxr1RiLaLoAO0AJmF5frpM98ot9ilS4el6+91hIS6fL0tW/e+t0FHCn0bkF4/tptiIu2avcjX6GFIlwTN+uXJtV7U7csr0U+fXqKiCrduOKOH/njxgHqf3+s19NqyvXpy/nad0TtZD186WO1jI+s9ftXefF314vI61+scKijTb15dpa2ZRYqOsOkfVw3T+QPTtPFQoR78eLNWVL0T3zHOoTvO76MrRmTIVs8Lwg/XH9LUuetlsUhzfnuKxvbqcMLvVV0O5JfqFy8s05HCcvVLi9ObN5xy3K+vqT767rD+8OY6SWpQ+JB8L9z+/N9N+vdy3wzYtad01Z8nDJT9BAHky81Z+u3rq9XOYde395yjhOgTvzDallmkP32wQav2+jbC7ZPaTg9dOlijeyTV+xiP19DGqpI7Z3mlisrdKip3y1leKWeZW0XllTIkXTOmqy4Zmt7kFznbs4r07+X79N7aQ7U6/Q3qHK9Lhqbrp0PSlV41G1Xh9mjhthx9uP6QvtySLVfV2kCLRfr1aT105/i+iopo/rb2lR6v/vXtHj315Q6VujyyWS0yDENeQ7p6dIYevjS8Yefrrdm68Y3VqvQYunx4Z/31yqGyWqT7P9ykN5bvU6TNqtd/M1qn9Azu98jl9urK2Uv13cFCjezWXnNvPEV2m1WvLd2rB/67SV5DGt0jSbOvHaGkEP5e+RmGoXve3aB5qw/IYbdqzg2nqFuHGL24eLdeX7ZX5ZW+n//JXX2B5/STWmbgKa/06JqXV2jNvqPKSIrWezefpvJKjy57bqlyiyv0kz4d9c9JI1vsmxB1yS4q173vbdSaffl68NJB+umQ9CY/57xVvuoQ/6vTTglRmnvjKQ2qijAMQ68t3au1+wv0m9N7aGhGYpPHg5aJoAO0EMt35+mal1fI4zX05wkD9KvTetR7bEGpS3/5eLPeW3tIktStQ4we//kQjal6ofL11mzd/tZ6FZRWKi7Krr9dOVTjB6apvNKjnz+/VJsOOzWyW3u9eeMpDfrPMph34Z5fuEuPfb5VDrtVH0w5Tf07xev7gwX6zWurlVNUoY5xDr0yaVSt9TSGYeh/m7M089Mt2ptXKknq1TFWXdrHyKi633ecZMjQ+v0FKnF59IdzTtK0qvVBjbUnt0QTX1im7KIK9UuL053j++qsvin1hqzG+mZHjn796ipVegxNGttNf75kYIO/p4Zh6J9L9ujhT7fIMKQz+3TUs78cXu+7uoZh6OfPL9Xa/QX63Zk9Nf3C/g0ep2EYemfNQc38bKvyS3ytz39+chfde1E/dWjnkGEY2pNbom935mrJzlwt25VXa9bkeE7t1UF/+dkgnZQSXMMDl9urLzZl6o3l+2qVpnXrEKOfDeusS4amn/A5neWV+nxDpt5fd0jLdvu6CvZLi9M/rhquvmnNt3h+1d58/en9jdqW5VvbNrJbez102SBtOeLU/731nbyG9MsxXfXQzwY1KOwUllaqxOWWx+trVOHxeqsaVhjyGoa6J8cq/jjv7i/Zkatfv7ZKLrdXFw/ppH9MHBYIyR6vod//Z42+2JSluCi73r5prPqlNfz/zkc+3aIXF+9WQnSEPp16Rq1yx4XbsnXLnHUqrnCrW4cY/XPSqKB/7sH6+/zt+seCHbJapNnXjtD5NWaqs4vK9cKi3fr38n2BZjg/6dNRf7tiSFjLVk/E6zV069x1+uT7I4qPsuu9358aaO7w/cECTXxhucoqPbpqVIZmXj64RQa1H/pyc5buqmpg4/eHc07SbeP6NDrw/3v5Pv3pg42SfG8erNp7VDuzixvUZKbU5dad73yvT74/ErjtosFp+r/z+zZLM5acogrFRdlD8iYKgkfQAVqQl7/ZrYc+2SK71aJbz+mtWIdNVotFNqtFVqtFNotFZZUezV60SzlFFYF3pu84v+8x5W6HCsp065y1Wru/QJL0m9N7qKTCrbmrDigpNlKf/OF0dUpo/nUYP1yv84dze+ue975XeaVX/dLi9M9fjap3/YfL7dUby/fpH19uP+GL59HdkzTnhjEnnNloiJ3ZRbrqxeXKLfb9R5yeEKWJo7pq4qiME9bDF5ZVam9uibq0j663fG7DwUJd9eIylbg8unhIJz191fBGBakvNmVq6tx1Kq/0qm9qnH59ened1TdFqT94obZid54mvrhckXarltx1dqNeyB0tcenxL7YGSiIToiN0Zp+OWr03X4erZhD94qLsOrlreyXFRiouyq74qAjFRdkVV/Xn7pwSPbdwpyrcXkXYLLrxJz11y9m9T1iieSC/VG+tPqA3Vx5QbnGFJF+nxvP6p+q6sd10aq8OjXqh92VV58C8Epci7Vbdc0E//erU7k2aZckuKtfjn2/TO2sOSpLax0Ro+kX9dUWNdTnvrT2o/3v7OxmGb2buwZ8Nqnf8O7KK9Lf/bdMXm47fpMRutWhEt/Y6u1+Kzu6boj6p7QLPuWJ3nib9a6XKK706f0CqZl1z8jFvbJRXenTtyyu0et9RpcVH6b3fnxqYETuer7dla/K/Vkmqv/x1e1aRfv3qKh08Wqa4KLtuPquX4hx2Oew2RdqtctitVX/a1K1DTJPKCWuu/Xz4skG6pp692bKd5Xpu4S7NWblfLrdXHWIj9bcrh+rsfimNPveJFJZVav2BAu3MLtawjESd3DWx3p+7f/1ehM2i13895pjZ6i83Z+nGN1bLa0h3nN9Ht5zTO2Tj9jMMQ6UujwrLKlVQWilneaU6J0af8OdV6nLroU+2BNZj9kuL04hu7fWfqs/HD0zVk78Yptgg19a8tnSvZny0SZLv/7U/XdxfOcUVuurF5dqdU6LOidGa97tT6mx0tD+vVDe+sVpbM4tkt1r0kz4d9fW2bBmG79+WX4zsoqnn9gl6HVSlx/dmzOvLfG/GJMZE6Hc/6aVJp/qqLGAegg7QghiGoVvfXKePa7zTVJ9eHWP1+BVDNaJb+3qPqfR49fjnW/XSN3sCt1ks0uu/Hq0zendsljHXJb/EpYuf/iZQUif53j2ddZxZiJqOlri0aHuOKj1eWSwWWSRZrZJFFlksksNu09n9Ota5/qixDhWU6ZUle/Tu2oOBxb82q0Xn9EvRL8d01U96d9TRUpc2HirUpsPOwJ/780sDz9E5MVpDuiRoSJdEDemSoEGdE5Rf4tIVzy9VXolLp/bqoH9NHtWkcdecHfMbmB6vc/ql6Ky+KRqWkahfv7pKi7bn6JdjuuqRywY3/psiae3+o/rj+xtrddOLtFk1olt7nd47WaedlKxB6fEnDJwH8ks146NN+mprtiTf9+qBSwZqXNXeWoZhaH9+qVbsydfKqo+a39uUOIeuGt1VV4/OaJaAnlNUobve+U5fb/Nt+HtG72T97cqhx4TGmio9Xh3IL9XunBLtzi32/Vn1d39IlnzvMN81vl+dZZDvrjmoO97xhZ3rTummv/ys9szeoYIyPTV/u95dezDQNCDSZpXNapHdZpHdapHNapXdapHHMGpdB5IvpJ/ZN0UD0uP16KdbVOLy6Oy+HTX7uhH1XncFpS5dMXuZdmYXq3dKO71901glxtRfapZZWK6Lnv5G+SUu/erU7vrzJQPrPTavuEI3vrFGa/YdrfcYyffv0gUD0zTl7JOCblCzYEuWbnxjjTxeQ7eec5L+rwGzvDuyinTrm+u0NdM38/br03ro7gv7Hvd3s6DUpffXHdKO7GJ1bOdQanyU0hIcSomLUmp8lDpU/bx35RRr7f6jWruvQGv3H9XOnGLVfAXVpX10oEFGzdnEN1fu1/SqsPbElUP183o2oPa3jJekv08cqsuG131cmcuj3OIKtY+NVGyk7bhvChSUurQ1s0jbs4q0NbNIO7OKlVtSIWdZpQrLKlXpOfYl4IBO8bpwUJouHJx2TEvxDQcLNXXeOu2uo4HNO2sO6t73Nsjl8b359dL1Ixsccv+5ZE+g0+bvzuypey7oF/i6spzluurF5dqTW6KMpGjNvXFsrTfVluzI1S1vrlVBaaWS2zn0/LUna1T3JG3LLNJfv9iqL7f4/m1y2K2afFoP3XxmrxPu8ZftLNeclfs1Z8V+Zf/gd1GSOsRG6uazeunaU7rVO8OTW1yhLzZl6vONmdqfX6qUuKprKz5KaQlRVddZlBKjI1RQVqm84grlFruUV+xSXkmF8opdKqpwa1hGos7rn6pBnc1puiL53jjZcsSpTYedGtol8YSdUMMhpEFn1qxZ+utf/6rMzEwNHTpUzzzzjEaPHl3v8W+//bbuu+8+7d27V71799Zjjz2miy66qMHnI+igLSh1ufXi4t3an1cqj2HI4/WVqHi8hjxeyWsYGpaRqBt/0rPBU+PzN2fp/95aL2e5W1PP7a3bw7Bnz+q9+ZpYtV7nl2O66i+XnHhdSUtQXunRZxuP6M0VB7Ryb3WZVEykTaWuult3J7eLrPVCt6aoCKvKK70amB6vuTc2zyLibGe53lx5QF9ty9b3BwtqvYhqHxOho6WVslqkr+84q9Fd/Gpye7x6d+1BHcgv0+geSRrVPSmohhl+hmFo/uYsPfDfzTpU1db7nH4pinXYtXJPnrKctV8oWC3SmB4ddN3YbjpvQGqzr0kwDEP/Xr5PD32yRRVurxJjIvTIZYPVNy1O+/JKtDe31PdnXqn25pXo4NGy47arHtIlQTMmDDzumw+SbwH1nVVhx1/GmF/i0qyvd+nfy/fJ5fGVVo0fmKo7zu+r3qn1l9btyyvRwm05+npbtpbtyguUZfmdflKyXp408oT/VhwqKNPPn1uqTGe5RnRrr9vG9daIbu2PeTfa4zV0zcvLtXx3vgamx+u93596wuBeXunRS4t3a2dOsVxuryrcXlW4PYG/l7k82pFdHDj+zD4dNeXsk467Nsxv3f6juvql5Sqv9OqKEV301yuGNPhFXnmlR49+tlWvLt0ryffC/emrh9cqsTMMQyv35OvNlfv16cbMwDqvutitFkXarXX+O9E1KUa9OsZq5Z58ldS4v19anCYMTVd6YpTuePt7ebxGg/6NnvnpFr2weLcibBY9c/VwRUfatTvH15beF8CLa828OuxWJbdzKCk2Uh3aRapDrEPtHDbtzi3R9qyiY3736hJhsyghOkLtHHYd+MHvwkkp7XThoDRdMChNi7fn6on/bZPbayg13qEnrjy2gc2affn63RtrlVtcoaTYSM2+dsQJf94vLNqlmZ9tlSRNObuX7ji/7zE/68zCck18cZn25ZWqa1KM5v3uFKXFR+mlb3br0c+2ymtIQ7skaPZ1I455w2TV3nw99tlWra4K5bGRNvXoGKuUuCilxDmUEl/1Z5xDETar3ll7UF9szAw0QukY59DVo32VACt25+kfC3ZoX1UpdkqcQ1POPklXjc6Qw25TtrNcn2/K1Kcbjmjlnnw1Zxf8TglRGtc/VeMGpOqUnkm1fj+zi8q16bBTmw87telwobYc8c1s9esUr35pceqXFqe+aXHqnBh9wt+jwtJKbTrse8PP/+eunOLA1/L7s3rprgv6Nd8X1kghCzrz5s3T9ddfr9mzZ2vMmDF66qmn9Pbbb2vbtm1KSTl2injp0qX6yU9+opkzZ+qnP/2p5syZo8cee0xr167VoEGDmvWLAX6Msp3l2ppZ1KjNTRtr5Z58OcsqdW7/lFZRS/5DO7KKNGflfr275qCc5W5ZLL69kwalJ2hQ53gNTE/QwPR4JcZEylleqY2HCrXhYKG+r/rTPyPRNSlG79w8Vilxzb8WILe4IvBCd/H2HBVVlfxdMjRdT189vNnP1xxKXW49vWCnXv5md61uaRE2i4Z0SdToHkka3SNJI7q1P+76k+ayM7tIt81br42HTrxPSXSETT2SY9WzY6x6dmynXh1j1SPZ9xFMiH1r9QHd/e73MgzfbNLafUcDL4BP6Zmkuy/op+Fdjx+Yfqi80qNlu/O0cGu2vtmRq54d2+npq4c1uHRma6ZTV85eFriG7FaLBndJ0JgeHTSmZ5JGdmuvV5bs1d+/3K6YSJs+vvV09WymDWa3ZRbp+YU79dF3hwMvlEZ3T9Lvz+6lM6v2KitzeZTlLPd9FFUEytDyS1w6s09HvdzIBfpfbs7Sne98p6OllYqOsOnPlwzQeQPS9O6ag3pz1f7ArIQk9e8Ur7P7dlRBWaWyCsuVVVSuLGeFcosrAm84REfYNKRLgk7u1l4nd22v4V0TAx0dy1wefbU1Wx+uP6SF23ICodbv8uGd9cQvhp7w38uaa3mOJ8JmqXM2pi6dE6PVLy1OfdLi1Dc1TmkJUUqIjlBCdIQSYyIUHVE9K5Rf4tL8zZn6bGOmvt2ZW+c5LhiYppmX19/A5nBBmW54fbU2HXYqwmbRjAkDdWafjr439QxD3sC6NENfbsnSU1/ukCRNPbe3bhvXu97v0eGCMl314nLtz/dtHzCoc0KgSuKKEV300KWD6g3+hmFowZZsPf7FVm3PKq7zmB8a1b29rhvbXRcMTKvVYbDS49V7aw/q6QU7A2/sdEqIUufEaK3Zf7TWG1RDuyTowsGdNLRLovJKKpRZ6LvOM50Vyios1xFnmQpLK9U+NlIdYiPVoZ1DyVWBtUO7SNltVn27I1eLd+TUCtrtHL69CyvcHm087DxmBrg+cQ67+qbFKTU+SiUut0oqfE1mfH/3qLjCXW/o7xAbqQHpvuYwV47MaND5QilkQWfMmDEaNWqUnn32WUmS1+tVRkaGbr31Vt1zzz3HHD9x4kSVlJTo448/Dtx2yimnaNiwYZo9e3azfjEAEIzySo925RSrW4fYoPZqOFriKwfplxYX0m5ufpUer9bsO6otR5y6fHiXE5ZdmG1HVQe1pFiHRvdI0rCMxEbNFDUHl9urv3+5Xf/8Zo8ibBZ16xCr7skxvj87+P7s1iFGqXFRzdYx7a1VB3TXu98HPh/UOV53je8X1jcjfmjT4UK9/M0erdidd8xaLKtFVc1Bjl8y1RT78ko0e9FuvbvmYCAEpFW92CqqZ93e4M4JmnvjKUGv9agpy1muaW+t17c7fY0qrBYFAldMpE2XDE3X1aO7akiXhDp/NpUer3KLK1RS4Va3DrENClyFpZX6fNMRffTdYS3dlReYfWtoaWt5pUc3vL5aK3bnq1uHmKoA3s4Xwqv+3j4mQmWVHuUVu5Rb7Ctzyi9xVZWludU1KUZ90+LUJ7Vdo2ebC8sqtWBLlj7bmKnF23Nkt/pCy5Uju5zwOi51uXXn29/rkw0nLteWpP87r49uPffE65IOFZRpYtX2AZKvDPn+nw7Q9WO7Neh3y+M1tDXTqSxnubKdFcpyVii7qFzZVQG7sKxSp/TsoOvHdteA9OO/3nS5vZq3+oBmfbVTmc7q36nhXRN10aBOumBQWrO1uy+v9GjZrjz9b3OWFmzJOqaczmKReibHBt6kG5AeL7fX0LbMIm094tTWzCLtyilucDju0j5aA9Or3/QbmJ6g1HhHi3pjMyRBx+VyKSYmRu+8844uvfTSwO2TJk1SQUGBPvzww2Me07VrV02bNk233XZb4LYZM2bogw8+0HfffVfneSoqKlRRUf1DdDqdysjIIOgAABql0uOV3WoJ23/UH64/pHfWHNTEURm6aFAnUzcUrckwDB08WqYVe/K1YneeVtRYM3XFiC7625VDQ3r+zMJyvfTNbs1ZsT+wD47kmy1JS6guJereIUaTT+vRLO2rvV5DLyzeHSi7Gtw5QVeP7qpLhqWHfDPKkgq3oiNsjfr5N/feNE1RXumR3WoJqkzZMAzN+nqnXly8W5Uew9d8xyLZbdaqZjxSVIRNk0/tftxupD90IL9U1/1zhUpcHj1z9fCg26c3t/JKjz5af1jlbo/G9U9tUNOPpvB6DX1/qFDf7sxVfJRdA9IT1L9T3AlneF1ur3bnFmvrkSIdLXUp1mFXnMOu2KqPdg67Yh02JURHtIr9nEISdA4fPqzOnTtr6dKlGjt2bOD2u+66S4sWLdKKFSuOeUxkZKRee+01XX311YHbnnvuOT3wwAPKyqq768yf//xnPfDAA8fcTtABAKB5HSks0+6cEo3pkRS29XZHS1zanlWkDu0cSo13qJ3DHvIX9btziqsWyvM6orVzV80Ktob1oQiNhgadFtkbb/r06Zo2bVrgc/+MDgAAaF6dEqJD0pL+eNrHRgb2BwuX5lp3BPMRcNBQQQWd5ORk2Wy2Y2ZisrKylJZ2bK99SUpLSwvqeElyOBxyOOretwIAAAAATiSoSBwZGakRI0ZowYIFgdu8Xq8WLFhQq5StprFjx9Y6XpLmz59f7/EAAAAA0FRBl65NmzZNkyZN0siRIzV69Gg99dRTKikp0eTJkyVJ119/vTp37qyZM2dKkqZOnaozzzxTTzzxhC6++GLNnTtXq1ev1osvvti8XwkAAAAAVAk66EycOFE5OTm6//77lZmZqWHDhunzzz9XaqpvF+z9+/fLaq2eKDr11FM1Z84c/elPf9K9996r3r1764MPPmjwHjoAAAAAEKyg99ExA/voAAAAAJAang1oWwEAAACgzSHoAAAAAGhzCDoAAAAA2hyCDgAAAIA2h6ADAAAAoM0h6AAAAABocwg6AAAAANocgg4AAACANoegAwAAAKDNIegAAAAAaHMIOgAAAADaHIIOAAAAgDbHbvYAGsIwDEmS0+k0eSQAAAAAzOTPBP6MUJ9WEXSKiookSRkZGSaPBAAAAEBLUFRUpISEhHrvtxgnikItgNfr1eHDhxUXFyeLxWLqWJxOpzIyMnTgwAHFx8ebOha0Plw/aAquHzQW1w6agusHTRGK68cwDBUVFSk9PV1Wa/0rcVrFjI7ValWXLl3MHkYt8fHx/LKj0bh+0BRcP2gsrh00BdcPmqK5r5/jzeT40YwAAAAAQJtD0AEAAADQ5hB0guRwODRjxgw5HA6zh4JWiOsHTcH1g8bi2kFTcP2gKcy8flpFMwIAAAAACAYzOgAAAADaHIIOAAAAgDaHoAMAAACgzSHoAAAAAGhzCDoAAAAA2hyCThBmzZql7t27KyoqSmPGjNHKlSvNHhJaoJkzZ2rUqFGKi4tTSkqKLr30Um3btq3WMeXl5ZoyZYo6dOigdu3a6ec//7mysrJMGjFaqkcffVQWi0W33XZb4DauHRzPoUOHdO2116pDhw6Kjo7W4MGDtXr16sD9hmHo/vvvV6dOnRQdHa1x48Zpx44dJo4YLYXH49F9992nHj16KDo6Wr169dKDDz6oms15uX7gt3jxYk2YMEHp6emyWCz64IMPat3fkGslPz9f11xzjeLj45WYmKjf/OY3Ki4ubtZxEnQaaN68eZo2bZpmzJihtWvXaujQoRo/fryys7PNHhpamEWLFmnKlClavny55s+fr8rKSp1//vkqKSkJHHP77bfrv//9r95++20tWrRIhw8f1uWXX27iqNHSrFq1Si+88IKGDBlS63auHdTn6NGjOu200xQREaHPPvtMmzdv1hNPPKH27dsHjnn88cf19NNPa/bs2VqxYoViY2M1fvx4lZeXmzhytASPPfaYnn/+eT377LPasmWLHnvsMT3++ON65plnAsdw/cCvpKREQ4cO1axZs+q8vyHXyjXXXKNNmzZp/vz5+vjjj7V48WLdeOONzTtQAw0yevRoY8qUKYHPPR6PkZ6ebsycOdPEUaE1yM7ONiQZixYtMgzDMAoKCoyIiAjj7bffDhyzZcsWQ5KxbNkys4aJFqSoqMjo3bu3MX/+fOPMM880pk6dahgG1w6O7+677zZOP/30eu/3er1GWlqa8de//jVwW0FBgeFwOIw333wzHENEC3bxxRcbv/71r2vddvnllxvXXHONYRhcP6ifJOP9998PfN6Qa2Xz5s2GJGPVqlWBYz777DPDYrEYhw4daraxMaPTAC6XS2vWrNG4ceMCt1mtVo0bN07Lli0zcWRoDQoLCyVJSUlJkqQ1a9aosrKy1vXUr18/de3alesJkqQpU6bo4osvrnWNSFw7OL6PPvpII0eO1JVXXqmUlBQNHz5cL730UuD+PXv2KDMzs9b1k5CQoDFjxnD9QKeeeqoWLFig7du3S5K+++47LVmyRBdeeKEkrh80XEOulWXLlikxMVEjR44MHDNu3DhZrVatWLGi2cZib7ZnasNyc3Pl8XiUmppa6/bU1FRt3brVpFGhNfB6vbrtttt02mmnadCgQZKkzMxMRUZGKjExsdaxqampyszMNGGUaEnmzp2rtWvXatWqVcfcx7WD49m9e7eef/55TZs2Tffee69WrVqlP/zhD4qMjNSkSZMC10hd/5dx/eCee+6R0+lUv379ZLPZ5PF49PDDD+uaa66RJK4fNFhDrpXMzEylpKTUut9utyspKalZryeCDhBCU6ZM0caNG7VkyRKzh4JW4MCBA5o6darmz5+vqKgos4eDVsbr9WrkyJF65JFHJEnDhw/Xxo0bNXv2bE2aNMnk0aGle+utt/Sf//xHc+bM0cCBA7V+/XrddtttSk9P5/pBq0XpWgMkJyfLZrMd09koKytLaWlpJo0KLd0tt9yijz/+WF9//bW6dOkSuD0tLU0ul0sFBQW1jud6wpo1a5Sdna2TTz5ZdrtddrtdixYt0tNPPy273a7U1FSuHdSrU6dOGjBgQK3b+vfvr/3790tS4Brh/zLU5c4779Q999yjq666SoMHD9Z1112n22+/XTNnzpTE9YOGa8i1kpaWdkxDL7fbrfz8/Ga9ngg6DRAZGakRI0ZowYIFgdu8Xq8WLFigsWPHmjgytESGYeiWW27R+++/r6+++ko9evSodf+IESMUERFR63ratm2b9u/fz/X0I3fuuedqw4YNWr9+feBj5MiRuuaaawJ/59pBfU477bRjWtlv375d3bp1kyT16NFDaWlpta4fp9OpFStWcP1ApaWlslprvyy02Wzyer2SuH7QcA25VsaOHauCggKtWbMmcMxXX30lr9erMWPGNN9gmq2tQRs3d+5cw+FwGK+++qqxefNm48YbbzQSExONzMxMs4eGFubmm282EhISjIULFxpHjhwJfJSWlgaOuemmm4yuXbsaX331lbF69Wpj7NixxtixY00cNVqqml3XDINrB/VbuXKlYbfbjYcfftjYsWOH8Z///MeIiYkx/v3vfweOefTRR43ExETjww8/NL7//nvjZz/7mdGjRw+jrKzMxJGjJZg0aZLRuXNn4+OPPzb27NljvPfee0ZycrJx1113BY7h+oFfUVGRsW7dOmPdunWGJOPJJ5801q1bZ+zbt88wjIZdKxdccIExfPhwY8WKFcaSJUuM3r17G1dffXWzjpOgE4RnnnnG6Nq1qxEZGWmMHj3aWL58udlDQgskqc6Pf/3rX4FjysrKjN///vdG+/btjZiYGOOyyy4zjhw5Yt6g0WL9MOhw7eB4/vvf/xqDBg0yHA6H0a9fP+PFF1+sdb/X6zXuu+8+IzU11XA4HMa5555rbNu2zaTRoiVxOp3G1KlTja5duxpRUVFGz549jT/+8Y9GRUVF4BiuH/h9/fXXdb7WmTRpkmEYDbtW8vLyjKuvvtpo166dER8fb0yePNkoKipq1nFaDKPGlrcAAAAA0AawRgcAAABAm0PQAQAAANDmEHQAAAAAtDkEHQAAAABtDkEHAAAAQJtD0AEAAADQ5hB0AAAAALQ5BB0AAAAAbQ5BBwAAAECbQ9ABAAAA0OYQdAAAAAC0Of8PFQFnEqH9jegAAAAASUVORK5CYII=", "text/plain": [ "
" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "errors = []\n", "for nb_sample in range(1, 100):\n", " \n", " rewards = np.zeros((nb_actions, nb_sample))\n", "\n", " for a in actions:\n", " for play in range(nb_sample):\n", " rewards[a, play] = get_reward(Q_star, a, var=1.0)\n", "\n", " Q_t = np.mean(rewards, axis=1)\n", " error = np.mean((Q_star - Q_t)**2)\n", " errors.append(error)\n", " \n", "plt.figure(figsize=(10, 6))\n", "plt.plot(errors)\n", "plt.show()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The plot should give you an indication of how many samples you at least need to correctly estimate each action (30 or so). But according to the central limit theorem (CLT), the variance of the sample average also varies with the variance of the distribution itself.\n", "\n", "> The distribution of sample averages is normally distributed with mean $\\mu$ and variance $\\frac{\\sigma^2}{N}$.\n", "\n", "$$S_N \\sim \\mathcal{N}(\\mu, \\frac{\\sigma}{\\sqrt{N}})$$\n", "\n", "**Q:** Vary the variance of the reward distribution (as an argument to `get_reward`) and re-run the previous experiment. Do not hesitate to take more samples. Conclude." ] }, { "cell_type": "code", "execution_count": 9, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "0.1252024683141708\n" ] }, { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAz8AAAH5CAYAAACve4DDAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjkuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8hTgPZAAAACXBIWXMAAA9hAAAPYQGoP6dpAABOi0lEQVR4nO3deXxU9b3/8fcsmZnsIQnZIEBYBBQUBcEoWqu5InqtVmprSy21Xr1toRW9tyq3xWsXRa1trYqi3tblV62treJSxSJYEI2RVdkMIFtYkgBZJttMZjm/P5IMGYgIzElyxryej8c8SM6cOflOchLmPZ/v93NshmEYAgAAAIAvOHtvDwAAAAAAegLhBwAAAECfQPgBAAAA0CcQfgAAAAD0CYQfAAAAAH0C4QcAAABAn0D4AQAAANAnOHt7ACcjHA5r3759Sk1Nlc1m6+3hAAAAAOglhmGooaFBBQUFstuPXduJy/Czb98+FRYW9vYwAAAAAFhERUWFBg4ceMx94jL8pKamSmp7gmlpab08GgAAAAC9xev1qrCwMJIRjiUuw0/HVLe0tDTCDwAAAIDjWg5DwwMAAAAAfQLhBwAAAECfQPgBAAAA0CcQfgAAAAD0CYQfAAAAAH0C4QcAAABAn0D4AQAAANAnEH4AAAAA9AmEHwAAAAB9AuEHAAAAQJ9A+AEAAADQJxB+AAAAAPQJhB8AAAAAfQLhBwAAAECfQPgBAAAA0CcQfgAAAAD0CYQfAAAAAH0C4SdGv3p9k7722Pv6V3l1bw8FAAAAwDEQfmK0pbpRq3bV6lBja28PBQAAAMAxEH5iZOvtAQAAAAA4LoQfkxi9PQAAAAAAx0T4iZGN0g8AAAAQFwg/JjEMaj8AAACAlRF+YtRR+CH6AAAAANZG+ImRjXlvAAAAQFwg/JiF0g8AAABgaYSfGFH3AQAAAOID4cckBqUfAAAAwNIIPzHqWPJDszcAAADA2gg/MWPiGwAAABAPCD8mofADAAAAWBvhJ0Z0ugYAAADiA+HHJKz5AQAAAKyN8BMjCj8AAABAfCD8mIRW1wAAAIC1EX5iRKtrAAAAID4QfmJkY+IbAAAAEBcIPyah8AMAAABYG+EnRrS6BgAAAOID4ccsLPoBAAAALI3wE6NIw4PeHQYAAACAz0H4iRENDwAAAID4cMLhZ/ny5briiitUUFAgm82mhQsXRu4LBAK6/fbbNXbsWCUnJ6ugoEDf+c53tG/fvqhj1NTUaPr06UpLS1NGRoZuuOEGNTY2xvxkehOz3gAAAABrO+Hw09TUpDPOOEPz588/6r7m5matWbNGc+fO1Zo1a/TSSy+pvLxcX/nKV6L2mz59ujZu3KjFixfr9ddf1/Lly3XTTTed/LPoTRR+AAAAgLjgPNEHTJ06VVOnTu3yvvT0dC1evDhq2yOPPKKJEydq9+7dGjRokDZv3qxFixZp5cqVmjBhgiTp4Ycf1mWXXaYHHnhABQUFJ/E0ep9B6QcAAACwtG5f81NfXy+bzaaMjAxJUmlpqTIyMiLBR5JKSkpkt9tVVlbW5TH8fr+8Xm/UzSo6Cj9EHwAAAMDaujX8+Hw+3X777frmN7+ptLQ0SVJlZaVycnKi9nM6ncrMzFRlZWWXx5k3b57S09Mjt8LCwu4c9gmxcaEfAAAAIC50W/gJBAL6+te/LsMw9Nhjj8V0rDlz5qi+vj5yq6ioMGmU5mHWGwAAAGBtJ7zm53h0BJ9du3Zp6dKlkaqPJOXl5am6ujpq/2AwqJqaGuXl5XV5PLfbLbfb3R1DjRl1HwAAACA+mF756Qg+W7du1dtvv62srKyo+4uLi1VXV6fVq1dHti1dulThcFiTJk0yezg9hsIPAAAAYG0nXPlpbGzUtm3bIp/v2LFD69atU2ZmpvLz8/W1r31Na9as0euvv65QKBRZx5OZmSmXy6XRo0fr0ksv1Y033qgFCxYoEAho1qxZuvbaa+Oy01vHkh+6vQEAAADWdsLhZ9WqVfryl78c+fzWW2+VJM2YMUN33XWXXn31VUnSuHHjoh73zjvv6MILL5QkPffcc5o1a5Yuvvhi2e12TZs2TQ899NBJPoXexbQ3AAAAID6ccPi58MILj1nlOJ4KSGZmpp5//vkT/dIAAAAAcNK6/To/X3S0ugYAAADiA+HHJCz5AQAAAKyN8BOjjrqPQb83AAAAwNIIP7Fi1hsAAAAQFwg/JmHaGwAAAGBthJ8Y2Sj9AAAAAHGB8GMSCj8AAACAtRF+YtTR6ZppbwAAAIC1EX5ixKQ3AAAAID4QfkxCq2sAAADA2gg/MbJR+gEAAADiAuHHJKz5AQAAAKyN8BMjWl0DAAAA8YHwEyOmvQEAAADxgfBjEoN5bwAAAIClEX5iROUHAAAAiA+EH5NQ+AEAAACsjfATs7bSD9kHAAAAsDbCT4yY9gYAAADEB8KPSZj2BgAAAFgb4SdGFH4AAACA+ED4MYnBqh8AAADA0gg/MWLNDwAAABAfCD8mYc0PAAAAYG2EnxjZaHUNAAAAxAXCT4yY9gYAAADEB8KPWZj3BgAAAFga4SdGFH4AAACA+ED4MQl1HwAAAMDaCD8xsrUv+mHWGwAAAGBthB8AAAAAfQLhxyQGE98AAAAASyP8xIhW1wAAAEB8IPyYhDU/AAAAgLURfmJka292TfYBAAAArI3wEyOmvQEAAADxgfBjEqa9AQAAANZG+IkRhR8AAAAgPhB+TEKrawAAAMDaCD8xiqz5IfsAAAAAlkb4iZGNjgcAAABAXCD8mITCDwAAAGBthJ8YUfcBAAAA4gPhxyQGva4BAAAASyP8xKq99EP2AQAAAKyN8BMjGxPfAAAAgLhA+DEJhR8AAADA2gg/MaLTNQAAABAfCD8mYc0PAAAAYG2Enxh1FH4MJr4BAAAAlkb4iRHT3gAAAID4QPgxCdPeAAAAAGsj/MSIVtcAAABAfCD8AAAAAOgTTjj8LF++XFdccYUKCgpks9m0cOHCqPsNw9Cdd96p/Px8JSYmqqSkRFu3bo3ap6amRtOnT1daWpoyMjJ0ww03qLGxMaYn0ls61vwYzHsDAAAALO2Ew09TU5POOOMMzZ8/v8v777//fj300ENasGCBysrKlJycrClTpsjn80X2mT59ujZu3KjFixfr9ddf1/Lly3XTTTed/LPoRUx6AwAAAOKD80QfMHXqVE2dOrXL+wzD0IMPPqif/exnuvLKKyVJzz77rHJzc7Vw4UJde+212rx5sxYtWqSVK1dqwoQJkqSHH35Yl112mR544AEVFBTE8HR6D3UfAAAAwNpMXfOzY8cOVVZWqqSkJLItPT1dkyZNUmlpqSSptLRUGRkZkeAjSSUlJbLb7SorK+vyuH6/X16vN+pmGfS6BgAAAOKCqeGnsrJSkpSbmxu1PTc3N3JfZWWlcnJyou53Op3KzMyM7HOkefPmKT09PXIrLCw0c9imYMkPAAAAYG1x0e1tzpw5qq+vj9wqKip6e0gRHXUfg4lvAAAAgKWZGn7y8vIkSVVVVVHbq6qqIvfl5eWpuro66v5gMKiamprIPkdyu91KS0uLulkFs94AAACA+GBq+CkqKlJeXp6WLFkS2eb1elVWVqbi4mJJUnFxserq6rR69erIPkuXLlU4HNakSZPMHE6PYtobAAAAYG0n3O2tsbFR27Zti3y+Y8cOrVu3TpmZmRo0aJBmz56tX/3qVxoxYoSKioo0d+5cFRQU6KqrrpIkjR49WpdeeqluvPFGLViwQIFAQLNmzdK1114bl53ebDS7BgAAAOLCCYefVatW6ctf/nLk81tvvVWSNGPGDD399NO67bbb1NTUpJtuukl1dXWaPHmyFi1aJI/HE3nMc889p1mzZuniiy+W3W7XtGnT9NBDD5nwdHoPhR8AAADA2myGEX8Ttrxer9LT01VfX9/r638eWrJVv128Rd+aNEj3fHVsr44FAAAA6GtOJBvERbe3eBB/ERIAAADoWwg/MTq84of0AwAAAFgZ4SdGtLoGAAAA4gPhxyRMewMAAACsjfATIxulHwAAACAuEH5MQuUHAAAAsDbCj0kMGh4AAAAAlkb4iRGz3gAAAID4QPgxCdPeAAAAAGsj/MTIJko/AAAAQDwg/JiEwg8AAABgbYSfGHWs+WHaGwAAAGBthJ8YMekNAAAAiA+EH5PQ6hoAAACwNsJPjGh1DQAAAMQHwo9ZKPwAAAAAlkb4iVFHq2uyDwAAAGBthJ8YMe0NAAAAiA+EH5MY9LoGAAAALI3wAwAAAKBPIPyYhLoPAAAAYG2EnxjZ2hf9MOsNAAAAsDbCT4zodwAAAADEB8KPSSj8AAAAANZG+IkRra4BAACA+ED4MQmtrgEAAABrI/zEqKPwQ/QBAAAArI3wEyMb894AAACAuED4MQulHwAAAMDSCD8xovADAAAAxAfCj0kMSj8AAACApRF+YhRpeED2AQAAACyN8BMr5r0BAAAAcYHwYxIqPwAAAIC1EX5iRN0HAAAAiA+EH5PQ8AAAAACwNsJPjDqW/DDtDQAAALA2wk+MbEx8AwAAAOIC4cckFH4AAAAAayP8xIhO1wAAAEB8IPyYhDU/AAAAgLURfmJ0uPBD+gEAAACsjPADAAAAoE8g/MSIVtcAAABAfCD8xIhW1wAAAEB8IPyYhMIPAAAAYG2En1hR+AEAAADiAuHHJAaLfgAAAABLI/zEqKPwQ/QBAAAArI3wEyObjXlvAAAAQDwg/JiEWW8AAACAtRF+YkTdBwAAAIgPhB+TUPgBAAAArI3wE6OOJT90ewMAAACsjfATI/odAAAAAPHB9PATCoU0d+5cFRUVKTExUcOGDdMvf/nLqMqIYRi68847lZ+fr8TERJWUlGjr1q1mDwUAAAAAIkwPP/fdd58ee+wxPfLII9q8ebPuu+8+3X///Xr44Ycj+9x///166KGHtGDBApWVlSk5OVlTpkyRz+czezjdzkbLAwAAACAuOM0+4Pvvv68rr7xSl19+uSRpyJAh+vOf/6wPP/xQUlvV58EHH9TPfvYzXXnllZKkZ599Vrm5uVq4cKGuvfZas4fUI1jyAwAAAFib6ZWfc889V0uWLNGWLVskSR999JFWrFihqVOnSpJ27NihyspKlZSURB6Tnp6uSZMmqbS0tMtj+v1+eb3eqJtVRBoe0O8NAAAAsDTTKz933HGHvF6vRo0aJYfDoVAopLvvvlvTp0+XJFVWVkqScnNzox6Xm5sbue9I8+bN089//nOzhwoAAACgDzG98vPXv/5Vzz33nJ5//nmtWbNGzzzzjB544AE988wzJ33MOXPmqL6+PnKrqKgwccTmYNobAAAAYG2mV35+8pOf6I477ois3Rk7dqx27dqlefPmacaMGcrLy5MkVVVVKT8/P/K4qqoqjRs3rstjut1uud1us4dqChu9rgEAAIC4YHrlp7m5WXZ79GEdDofC4bAkqaioSHl5eVqyZEnkfq/Xq7KyMhUXF5s9nB5D5QcAAACwNtMrP1dccYXuvvtuDRo0SKeddprWrl2r3/72t/re974nqa1SMnv2bP3qV7/SiBEjVFRUpLlz56qgoEBXXXWV2cPpdh11HxoeAAAAANZmevh5+OGHNXfuXP3whz9UdXW1CgoK9J//+Z+68847I/vcdtttampq0k033aS6ujpNnjxZixYtksfjMXs43Y5ZbwAAAEB8sBlG/E3Y8nq9Sk9PV319vdLS0np1LK9/vE+znl+rSUWZ+st/xu+0PQAAACAenUg2MH3NT19jE6UfAAAAIB4QfkwSd+UzAAAAoI8h/MTIdrjjAQAAAAALI/zEiElvAAAAQHwg/JiEVtcAAACAtRF+YkSrawAAACA+EH5MEn8NwwEAAIC+hfATs7bSD9kHAAAAsDbCT4yY9gYAAADEB8KPSQzmvQEAAACWRviJEYUfAAAAID4QfkxC3QcAAACwNsJPjGzti36Y9QYAAABYG+EnRkx7AwAAAOID4cckFH4AAAAAayP8xIhW1wAAAEB8IPyYhUU/AAAAgKURfmLUUfkh+gAAAADWRvgBAAAA0CcQfmJkE62uAQAAgHhA+IkVDQ8AAACAuED4MYnBqh8AAADA0gg/MaLwAwAAAMQHwo9JWPMDAAAAWBvhJ0Y2Gw0PAAAAgHhA+IkR094AAACA+ED4MQmFHwAAAMDaCD8xslH6AQAAAOIC4cckBot+AAAAAEsj/MTIxqofAAAAIC4QfmLEtDcAAAAgPhB+TMKsNwAAAMDaCD8xovADAAAAxAfCj0kMml0DAAAAlkb4iVV76YdpbwAAAIC1EX5iRLc3AAAAID4QfkxC4QcAAACwNsJPjGh1DQAAAMQHwo9JDBb9AAAAAJZG+IlRR+GH6AMAAABYG+EnRjbmvQEAAABxgfBjFko/AAAAgKURfmJE4QcAAACID4Qfk1D4AQAAAKyN8BOjSMMDur0BAAAAlkb4iRHT3gAAAID4QPgxCXUfAAAAwNoIPzGj9AMAAADEA8KPSVjyAwAAAFgb4SdGHWt+DCa+AQAAAJZG+IkRk94AAACA+ED4MQnT3gAAAABrI/zEyEavawAAACAuEH5MQuUHAAAAsLZuCT979+7Vt7/9bWVlZSkxMVFjx47VqlWrIvcbhqE777xT+fn5SkxMVElJibZu3dodQ+l21H0AAACA+GB6+KmtrdV5552nhIQEvfnmm9q0aZN+85vfqF+/fpF97r//fj300ENasGCBysrKlJycrClTpsjn85k9nG7HrDcAAAAgPjjNPuB9992nwsJCPfXUU5FtRUVFkY8Nw9CDDz6on/3sZ7ryyislSc8++6xyc3O1cOFCXXvttWYPqUcYzHsDAAAALM30ys+rr76qCRMm6JprrlFOTo7OPPNMPfnkk5H7d+zYocrKSpWUlES2paena9KkSSotLe3ymH6/X16vN+pmFTYmvgEAAABxwfTws337dj322GMaMWKE3nrrLf3gBz/Qj3/8Yz3zzDOSpMrKSklSbm5u1ONyc3Mj9x1p3rx5Sk9Pj9wKCwvNHnbMqPsAAAAA1mZ6+AmHwzrrrLN0zz336Mwzz9RNN92kG2+8UQsWLDjpY86ZM0f19fWRW0VFhYkjjk3Hmh9mvQEAAADWZnr4yc/P16mnnhq1bfTo0dq9e7ckKS8vT5JUVVUVtU9VVVXkviO53W6lpaVF3QAAAADgRJgefs477zyVl5dHbduyZYsGDx4sqa35QV5enpYsWRK53+v1qqysTMXFxWYPp8cYTHwDAAAALM30bm+33HKLzj33XN1zzz36+te/rg8//FBPPPGEnnjiCUmSzWbT7Nmz9atf/UojRoxQUVGR5s6dq4KCAl111VVmD6fb0eoaAAAAiA+mh5+zzz5bL7/8subMmaNf/OIXKioq0oMPPqjp06dH9rntttvU1NSkm266SXV1dZo8ebIWLVokj8dj9nB6DGt+AAAAAGuzGXF4gRqv16v09HTV19f3+vqfTfu8uuyhd5WT6taHPy35/AcAAAAAMM2JZAPT1/z0VXGXIAEAAIA+hvATI1pdAwAAAPGB8BMjGh4AAAAA8YHwYxpKPwAAAICVEX5iZBOlHwAAACAeEH5MwpofAAAAwNoIPzGKNDzo3WEAAAAA+ByEnxgx6Q0AAACID4Qfk8ThtWIBAACAPoXwEyNaXQMAAADxgfBjEuo+AAAAgLURfmLWVvph1hsAAABgbYSfGDHtDQAAAIgPhB+T0PAAAAAAsDbCT4wo/AAAAADxgfBjEuo+AAAAgLURfmJk61j0Q/oBAAAALI3wEyOmvQEAAADxgfBjEgo/AAAAgLURfmJEq2sAAAAgPhB+TEKrawAAAMDaCD8xsrWv+iH6AAAAANZG+IkR094AAACA+ED4MUnnWW+GYcgXCPXeYAAAAAAchfDTDf77xY81au4ibatu6O2hAAAAAGhH+DGJ0WnVz9/X7JEkPbF8e28NBwAAAMARCD8x6ljzQ7M3AAAAwNoIPzGyHaPjQUcnOAAAAAC9j/BjEgo/AAAAgLURfmJ0rNoObbABAAAA6yD8mIXSDwAAAGBphJ8YRRoekH4AAAAASyP8xOhYTQ2Y9gYAAABYB+HHJLS6BgAAAKyN8BOjY1d3KP0AAAAAVkH4MQmFHwAAAMDaCD8x6qjtGMx7AwAAACyN8BOrY8xso+EBAAAAYB2EH5N0Vfch+wAAAADWQfiJ0bFaXQMAAACwDsKPSVjyAwAAAFgb4SdGrOsBAAAA4gPhpxsRjAAAAADrIPzEqHO+ObLdNeuBAAAAAOsg/MTIRnkHAAAAiAuEHxMd2fSAXAQAAABYB+EnRuQbAAAAID4QfkxEt2sAAADAugg/Meo8te3ohgcAAAAArILwEyM6ugEAAADxgfBjoiOnvdEJDgAAALAOwk+syDcAAABAXCD8mOjIVtcAAAAArIPwE6Oohgf0ewMAAAAsi/ATo2PNemPJDwAAAGAd3R5+7r33XtlsNs2ePTuyzefzaebMmcrKylJKSoqmTZumqqqq7h5Kt2PaGwAAAGBd3Rp+Vq5cqccff1ynn3561PZbbrlFr732ml588UUtW7ZM+/bt09VXX92dQ+k2R3Z0O/JaPwAAAACsodvCT2Njo6ZPn64nn3xS/fr1i2yvr6/XH/7wB/32t7/VRRddpPHjx+upp57S+++/rw8++KDLY/n9fnm93qibVZF9AAAAAGvqtvAzc+ZMXX755SopKYnavnr1agUCgajto0aN0qBBg1RaWtrlsebNm6f09PTIrbCwsLuGfcI6130MQwp3Sj9cABUAAACwjm4JPy+88ILWrFmjefPmHXVfZWWlXC6XMjIyorbn5uaqsrKyy+PNmTNH9fX1kVtFRUV3DPukHNnUIGx89n0AAAAAeo/T7ANWVFTo5ptv1uLFi+XxeEw5ptvtltvtNuVY3cmg2TUAAABgWaZXflavXq3q6mqdddZZcjqdcjqdWrZsmR566CE5nU7l5uaqtbVVdXV1UY+rqqpSXl6e2cPpdkdObWPNDwAAAGBNpld+Lr74Yq1fvz5q2/XXX69Ro0bp9ttvV2FhoRISErRkyRJNmzZNklReXq7du3eruLjY7OH0KMOQwuq85gcAAACAVZgeflJTUzVmzJiobcnJycrKyopsv+GGG3TrrbcqMzNTaWlp+tGPfqTi4mKdc845Zg+n23Ve12OIyg8AAABgVaaHn+Pxu9/9Tna7XdOmTZPf79eUKVP06KOP9sZQTBfV7Y3SDwAAAGAZPRJ+/vWvf0V97vF4NH/+fM2fP78nvnyPMQyDyg8AAABgUd12nZ++4sjqjkH6AQAAACyJ8GMiQ0de54d5bwAAAIBVEH5i1LnVtWFEr/kBAAAAYB2EnxgdPe2t88cEIQAAAMAqCD9mMqIDD9kHAAAAsA7CT4yOXNXTec1PmPADAAAAWAbhJ0b2TvPeguFw1JofQ6QfAAAAwCoIPzGy221yOdq+jf7gEeGH7AMAAABYBuHHBO6Etm+jLxCi4QEAAABgUYQfE3gSHJIkXyAcHX56aTwAAAAAjkb4MYGno/ITDEVNe+OaPwAAAIB1EH5M4HF2VH5CrPkBAAAALIrwY4KOaW/+QDiqvTXZBwAAALAOwo8J3M7DDQ86Rx4aHgAAAADWQfgxQaThQTAUXfkh+wAAAACWQfgxQUfDg7Zpb6z5AQAAAKyI8GMCd0Knhgfhw9vp9gYAAABYB+HHBJFub8EjKj+9NSAAAAAARyH8mCBynZ9AKGo7lR8AAADAOgg/Jog0PDhizQ+lHwAAAMA6CD8m6Fz54To/AAAAgDURfkzQsebHHwxFVX6Y9gYAAABYB+HHBJ2nvRlc5wcAAACwJMKPCdydpr0ZdHsDAAAALInwY4JIq+sj1vww7Q0AAACwDsKPCQ5Xfuj2BgAAAFgV4ccEHWt+aHgAAAAAWBfhxwROu02SFAobUdUesg8AAABgHYQfE9g7wo9hHHGdH9IPAAAAYBWEHxM4bB2VHx0x7a23RgQAAADgSIQfExye9hbd8IBpbwAAAIB1EH5MYO+05ic68JB+AAAAAKsg/JjA0R5+2vodMO0NAAAAsCLCjwnstsOVn3D48HaDeW8AAACAZRB+TODoNO2NhgcAAACANRF+TOCMCj+Ht5N9AAAAAOsg/JggMu3NMKKmujHtDQAAALAOwo8JIg0PwtGXNSX7AAAAANZB+DGBo/27GDKi1/wYTHwDAAAALIPwY4Kobm+d1/yQfQAAAADLIPyYwGlv+za2XeS0c7c30g8AAABgFYQfE9g7pr0d0eqa7AMAAABYB+HHBJGGB4YRFXgIPwAAAIB1EH5M4PisNT80PAAAAAAsg/BjgsOVn7Z21x2o/AAAAADWQfgxQUf4kaRgmIYHAAAAgBURfkxgjwo/4cjHRB8AAADAOgg/JuhY8yNJgRDT3gAAAAArIvyYIGraW6hT5Yf0AwAAAFgG4ccEn7Xmh+gDAAAAWAfhxwTR094OV35oeAAAAABYB+HHBFEND1jzAwAAAFgS4cckHVPfAp27vRF+AAAAAMsg/JikY+pb58oP094AAAAA6zA9/MybN09nn322UlNTlZOTo6uuukrl5eVR+/h8Ps2cOVNZWVlKSUnRtGnTVFVVZfZQelRH5adztzcAAAAA1mF6+Fm2bJlmzpypDz74QIsXL1YgENAll1yipqamyD633HKLXnvtNb344otatmyZ9u3bp6uvvtrsofSow9PeWPMDAAAAWJHT7AMuWrQo6vOnn35aOTk5Wr16tS644ALV19frD3/4g55//nlddNFFkqSnnnpKo0eP1gcffKBzzjnH7CH1iI6eB0G6vQEAAACW1O1rfurr6yVJmZmZkqTVq1crEAiopKQkss+oUaM0aNAglZaWdnkMv98vr9cbdbOaw9PeuM4PAAAAYEXdGn7C4bBmz56t8847T2PGjJEkVVZWyuVyKSMjI2rf3NxcVVZWdnmcefPmKT09PXIrLCzszmGflK6mvVH5AQAAAKyjW8PPzJkztWHDBr3wwgsxHWfOnDmqr6+P3CoqKkwaoXki4SfYqeEB2QcAAACwDNPX/HSYNWuWXn/9dS1fvlwDBw6MbM/Ly1Nra6vq6uqiqj9VVVXKy8vr8lhut1tut7u7hmqKjlbXLYFQZBvZBwAAALAO0ys/hmFo1qxZevnll7V06VIVFRVF3T9+/HglJCRoyZIlkW3l5eXavXu3iouLzR5Oj7Hbjw4/THsDAAAArMP0ys/MmTP1/PPP65VXXlFqampkHU96eroSExOVnp6uG264QbfeeqsyMzOVlpamH/3oRyouLo7bTm/S4Wlvvs6VH7IPAAAAYBmmh5/HHntMknThhRdGbX/qqaf03e9+V5L0u9/9Tna7XdOmTZPf79eUKVP06KOPmj2UHtUx7a25tfO0N9IPAAAAYBWmhx/jOModHo9H8+fP1/z5883+8r2mo/LT0in8hMOftTcAAACAntbt1/npKxxdrPkBAAAAYB2EH5PYbUdXfvbWtei6P5SpoqZZe2qbj6sqBgAAAKB7dFur677msyo/7249qPPvf0eSdOu/naIfXzyix8cGAAAAgMqPaTpaXR/Lbxdv6YGRAAAAAOgK4cckzuMIPwAAAAB6D+HHJB2trgEAAABYE+HHJHa+kwAAAICl8ZLdJA6mvQEAAACWRvgxiYPSDwAAAGBpvGI3iYPCDwAAAGBphB+TtIbCvT0EAAAAAMdA+DGJtyXY20MAAAAAcAyEH5PUtwR6ewgAAAAAjoHwYxKvj/ADAAAAWBnhxyQNPqa9AQAAAFZG+DFJKGz09hAAAAAAHAPhBwAAAECfQPgBAAAA0CcQfkzym2vO6O0hAAAAADgGwo9Jpo0fqN9fO663hwEAAADgMxB+THT52Hz9++n5+s8vDe3toQAAAAA4AuHHRE6HXY986yzdMLmot4cCAAAA4AiEn25gt9l6ewgAAAAAjkD46QZEHwAAAMB6CD/dwEblBwAAALAcwk83SHI5ensIAAAAAI5A+OkGngTCDwAAAGA1hJ9u4rAz9Q0AAACwEsJPNzklN7XL7YZh9PBIAAAAAEiEn27z+2vHdbk9FCb8AAAAAL2B8NNNTslN1Xt3XHTU9iDhBwAAAOgVhJ9ulNDFuh/CDwAAANA7CD/dyOk4+tsbChF+AAAAgN5A+OlGTsfRlZ9AONwLIwEAAABA+OlGCfajv71BKj8AAABAryD8dKOuKj9BKj8AAABAryD8dCNnVw0Pjqj8fLynTnvrWnpqSAAAAECf5eztAXyR2WzH7va2r65FX3nkPUnS9nsuk72LsAQAAADAHFR+eljnaW87DzZFPl5bUStJCoTCam4N9vi4AAAAgC86wk83mzC4X9Tnnae9+UOHg9A/N1ZJkr7+eKkm3/eOmvwEIAAAAMBMhJ9u9tT1Z+vaswsjn3ee9tboOxxwdhxskj8Y0trddappatWqXbU9Ok4AAADgi47w081SPQm6d9rpGpKVJEkKdZr21rm6U+n1qdrrj3zewtQ3AAAAwFSEnx7icrZ9qxeu3RfZ1tgp/Oyv96m6wRf5vLLep/e3HdSGvfU9N0gAAADgC4zw00OuGd829e2vqypkGIYCobDKdtRE7j/Y6Nee2sMtr5dtOaBv/V+Z/v3hFT0+VgAAAOCLiPDTQ2acO0SS5A+GVdsc0N3/2KzFm6oi9xuG9PGew1Wed8oPRD4Oh6OvDQQAAADgxBF+eojLaVdWskuSVOX16en3dx61z7qKui4f23TE+h/DMPSTFz/Sna9sMHuYAAAAwBcWFzntQTlpHh1qalWl19fl/as/o8Nbkz+kFLdTNptN1V6f/ufl9Xp7c7Uk6dLT8nTO0CwukAoAAAB8Dio/PSg3zS1JqqqPDj8lo3OP+bjyqgZN+NXb+vVbn+iOlw4HH0n61v+V6a+rKo7r62+rbpQvEDrBUZ+8cNiQYTBlDwAAANZA+OlBuakeSdKWqsao7VeOK5CzU+Xm99eO06wvD1eCo23bA2+V61BTq+a/86lKPz101HH/b8UONfqDeub9nbrw1+/o5bV7jtpn5c4alfx2mX7wp9XHPd4GX0DBThdiPRG+QEglv12m659eeVKPBwAAAMxG+OlBuelt4WfVrpqo7YkJDuVneCKfjx2Qrv+eMlJF2cmS2tYIdXB2Mb1tW3Wjxv38n/rfVzdq56Fm/emD3Uft8+b6SkltjRT217ccdX8HXyCkP32wS8u2HNAZP/+n5r35yQk8w8Pe23ZQ2w826V/lB046QHW2t65FlfVdTxcEAAAAjgdrfnrQsP5tYaZzVzepraFBRqJLFWoLJblpbUEo2d3246lvCUTt25Vgp45wna8X1MHQ4fv/tmqP8jMSdfnYfCW6HFH7zX5hnRZtrIx8/ocVO/TTy0af8Jqi6obDF2ytaWpVTprnGHsfW3NrUOfdu1SStO3uqXI6YsvsD769Rc2tIc2ZOko2G2ulAAAA+goqPz3o8rH5Gp6TctT24mFZcnQKFx2hJ6X9X3/wcOXkWF2vUz1t+1fUtGjpJ1Vqbg9Kuw416an3dkb2+83iLfrvFz/SzOfXRLa1tIb0z42VUcGnQ3lVQ+Tjjfvq9cBb5fIFQmryB/W1x97XXa9ujNz/05fX69onSlVeefgxBxr9OlJza1B3vrJBK3fW6JV1e7Vpn/czn9fOg83HPNaJ2FvXogff3qonlm+Puq7SyWgNhrV5v5d1TQAAAHGCyk8Pcjrsmjw8W9uq29b8fPfcIfrRRcOVleKOrO/pLNl1Yj+e/3fDJF01/z1J0veeXqUvj+yv/7pkpKY99n6X+y/9pFq+QEieBIdm/2Wt3tpY1eV+9y/6RI9fN0Eup11fffR9tQbDCoYNnZKbolW7arVqV61uu3SkbLLpubK2KXcb9h4OM4caWyMfG4ahNbvr9Ob6/Xq2dJeeLd0VuW/HvMu0cmetXv1or35yySilJyVIagsskX0ONik/PTFyrE37vSrKTlbScX6vlm85fP2k/fU+FWYmHdfjuvLz1zbqubLdWvDts3TpmPyTPg4AAAB6BuGnh3Ws45GkYTkpykpp6wA3fnCmVu6MbnXdUQE6HjdfPELjCjOiti3bckD76nxRlaMjvf/pQaUnuj4z+Eht64R+8fpG/eiiEWptP9aiDfu1rTo1ss9HFfWRSpUkNfoPT8872OhXfXNAaytq9eLqPfrHx/u7/Dq/e3urXlxVof31PrmdDs3991MltVWuOnzryTLdN22sSj89pPqWgN4pP6CCdI9e+uF5uueNzVq1s0b/+PH56td+TaUOb6zfr7Lth7S37vCUwF+/9Yn+3w2T5EmInvrX2fNlu/XM+zv1+HXjNaTTz84wjEjQ++XrmxUMGyoZnXvMY3XFFwhpT22zhuekHnO/pZ9U6b1th3TLv50S9X3uUN3g071vfqIrxw3Ql07pf0JjOFFVXp/mv7NNN10wVAP7nXx47LDzYJNW7arV1WcOoGU7AADoVoSfHtY5/AzMSIx8/OOLh8uQocvHHq4gpLi7fiE9fdIg3XbpKP3wudV6b1tb97eC9oYJQ/sna/uBtrAQNqKnrElS/1S3DnRaj/P3NXu1+RhTzn551RjNXbhBL6/Zq5bWwyFq56Fm7Tx0eDra6l01yktP7OoQ+ufGKt35ysaoQNSVh5ZsjXz8hxU7dGp+mjKTXfrVPzZH7Xf739dHfb6v3qdz5i2JfL54U5W+etYA1Ta16u3N1Zo0NFOz/7IuEtw6rNxZq1FzF+lnl4/Wf5w/NOrxs19YK5fTrtrmtvVWv1+yVb/7xrjIPu9uPRj5eG9di2Y9v1YXjcrRnKmjNDwnRWt21+mhJVv1wwuHackn1frmxEEqyk6WYRhR64zmvLReL6/dqyeuG69LTsuLGp9hGPIFwkp0OfRff/1Itc0Bbalq0P+7YZIk6fFln2rlzlrN/ffRmv5/ZdpT26Ky7TW67dKRGp6TolPz0yRJr6zbpwZfQN8+Z7BsNpve23ZQ/9xYqVsvGan0xIRj/ky6cuOzq/Txnnp9tKder8w874Qf39mBBr+u+2OZKmpadLDRr+9/aVhMx+tKOGzo5r+sk01tnRRPdp3Xv8qrleJ2asKQTFPG1RoMK8Fhi4t1Z2t312pgvyT1T3X39lAAAIiJzejFBQvz58/Xr3/9a1VWVuqMM87Qww8/rIkTJ37u47xer9LT01VfX6+0tLQeGKl5Kmqadf7970iSFs0+X6PyPnv8897crMeXbZck/fnGc1T66UFVen266yunKcnlVJM/qNP+9y1Jiky92lrVoLc3V6tfUoLueKktJFwzfqDW761XksuhkXmp+vOHXV8X6PtfGqYbJhfp7LvfjmzbdvdUjfvF4khwsduOve7oWBx2m0JHPDg/3aP9R3Rxczvtx6xWfZ40j1PNraGoJhCfx5Ng14TBmXrs22fpG49/oE37owPhgIxE/e0HxdpX59O9b24+qkrX2fRJgyJVoQ7D+ifrzZsv0Iw/fqhtBxr1H5OL5HbadddrmyRJw3NS9PatX5JhGKry+rV6V60+3lunx5dt1yWn5uqfmw5X5k7JTZHdZtMnldHB9kiFmYmqqDk8ZfDvPzhX4wf307hf/FN17aHutII0zZk6Wh/urNG3Jg5SXrpHgVBYYcPQut112rzfq2njByrVk6BGf1Ard9bo+qcOty/fee/lkqR1FXV6ZOk2TT9nkIqHZsnttMtms6m2qVUNvqA8CXbtONikBl9QfyrbpUtPy9PpAzN05fwVCoTafk5JLodK51ys/37xI509pJ9uuuBwENpT26ysZLcSXQ41twaVmOCQzWZTOGwoEA5rb22LMpJcevDtLZpYlKnLx+bLZrOprrlVuw4168r26aCS9LfvF2v84H461NSqN9fv18Wjc1WQkaidB5u0/WCjvnRKjuy2tkYdq3bVak9tiy4c2V8X/2aZJOnx68YrIzFBk4Zmfeb3vjUYlst59JLKjj+3H++p1zeeKNXXJxTqF1eOidqnYypqZwcb/UrzJESO+d62g1q8qUoJDpuuPmugRucf/9/BrVUNavQHo8LMmt21Mgxp/OB+kqSn39uhtzdX6/fXjtP2g026ZkGpxhVmaGGnsBsOG9pd06ya5lblpXlU0P5mzoqtB9XoDxxzKujbm6r07tYDuu3SUZEK9/76lsiU1mAorJfX7tWZg/opN82tVM+xQ3prMCy7TZFmKIFQWD/40xqlJybogWtOP+mA2dH98t9PL1Be+ok3bTnU6NfXHy/VhMGZuu9rp5/UGCpqmtUSCOmU3FR9eqBRd726Uf9x/tDjrvCGwoZqmlqPCq7BUFitofBxTxk+GV2dy/Gq85tRHWqaWuVtCUTNCgC+yIKhcMxNrLrLiWSDXgs/f/nLX/Sd73xHCxYs0KRJk/Tggw/qxRdfVHl5uXJyco752HgOP6GwoYl3vy1/MKxVPys55n8MNz67SovbX/R+es9lUU0RpLY/xkVz3pAkLZx5XtS0N8MwNOel9dpc2aCnvnu2+iUlyGazqdrr09cfL9VpBen6x/ro6Wf/950JKjk1V0Pu+IckaVReqhbNvkAX/+Zf+rS9mnT/tNO1elet/tJ+YdWXf3iuvvpo12uKiodmqXR7W2VqQEainvneRM3444eRNTydKy776lp0+UPvql+yS7NLTtGP/7z287+Zks4fkR1VhYlVXppHle2txX8yZaS2H2jS39ccfd2k7tAvKUHBsKEG37ErZCcrM9mlodnJWrXrs4NbUXayDjT4o6p0eWkeTSzK1Osf7zsq+A7JStL15xXpyXejG0gMzkrS9ecO0YNLtkaC1om664pTNXZghvbWtejmF9aq81+qkbmpGto/WRv3ebW7pvmox541KENfOaNA9y76RL7A0UE62eVQU+vhC/7O/PIwPfv+LjW0P+/sFJcOdlqr1pWrxhUoM9mtov7J2lPbrA+212h0XqrSEhP09Hs7lZ/h0dVnDlRGUoLGDEjTB9tr9OS72+UPhNXS6WLDM4oH68KROfrTB7u05JNq2WzSuMIMeZwOfWlkf+061BR5w6J4aJb+57LRuuKRFZHHZ6e4NHVMvsYVZqi+JaBkt0N763z6++o9unRMnr7/pWHacbBJd7z0caQq3FnHOe+w2/Tzr5ymrVUNeqZ9Ld4ZA9OVk+aJ/B0akJGoc4ZmacKQfnpl3V59sP1w2/5T89OUleLSe9sOKmy0nW+3/NspumhUjrJTXFqzq067DjXpUFOrfv1WuaS2vzEzzh2iP67Yoa3tayFPzU+TLxDS9oNtY7XbpAXfHq8V2w4q1eNUUXaKth9oVF66Rx6nQzXNrZr/zjYlu5y6Y+oolVc16M31+yOV6RsmF6nBF5A/GNZXzijQoMwk7TzULLfTLofdpqLsZKV4nAqHDYXChlxOu7ZUNej5sgq9/+lB7a/36ZTcFP3pPyZp8aYqlVc2aMyAdAVCYX31zAEqr2zQB9trNCQrSa9/vF+5aR7dXDJCpZ8e0vc7XVftLzedo2S3U3vrWvToO9t0RmGGLjk1T4kuhzwJdrUGw/IFwvp4T51OLUjTkKxk/fG9HZFmNbf+2yl6tnRn5Lz88cUjNKx/stZV1Gnlzhp9eWSOPt5Tr6LsZJ1akKZzh2Xp5TV79bc1e1RR06yZXx6uouxkFWYm6WCDX3e8tF5eX0A3nT9Ucy4brUZ/UOv31OvVj/ZpZG6Kxg5MV3aKWzbZ1JEd7XabGnwBPbVipwZnJ+mcoVmySXI7HVq9q0bbDzZp8vBsnVqQpj+u2KE/vrdT4wf1U266R0OykvSNswvVL8klQ5K3JaAFyz7Viq0HVdPcqhE5KZo6Jl+TR2SrpTWkvHSPnHab/r5mjy44pb+qvH49sfxTHWps1f1fO125aR75A2F5fQHd/1a50hMTNHl4ltZV1Ov8EdmaVJSpKq9fDb6Adhxs0pdG9teOA036x/r9ykn1qHT7QeWkehQKGyrKTlZumlvlVQ3KTHJpYL8kXTiyf/vU6mqlJybo6fd3aefBJl1XPFjXjB+oJZ9U6943P1GCw6anr5+ocYUZqvT6lJ3i1p7aZi1cu1c7DzVrRvEQDc5KUlaKSx9V1OtQk1+5aR6NHZAuu82mjfvq29bQ5qTK6bDJ6bDprlc3qqU1pB9dPELvbjmg7FS3MhJdOtTk16Z9XvVLdinNk6BTclPkcto1PCdF73xyQIs2VuqGyUWR1wKHGv0yJK3eVaskl0OnFaQrM9klXyCkh5du1bPv79KXR+Xou+cN0VmD2t74aPQH9cb6/Rqek6JxAzN0oNGvRJdDpZ8e0oqtB3XusCzVNgc0saitAj6wX6JW76qV027TmYP6yeW0q8kfVG37myKLNlZqS2WDbDabLhzZX4Myk/Sv8gPKTHEpEAwrK8WtmqZWhcKGBvZLVEZSgipqWhQ2DE0qypQhqbLepzW7a/XG+v0qyEjUN84u1McV9UpPSlBNU6uKspM1cUimNu1v+950vBkSNtr+1svW9gZJZrJLizdVKSfVo6wUl9xOuw41terU/DTVNQeUmdz2Pc5J9SgYDqu+JaD+Ke7I78bummZNGpqlaq9Pg7OS1egP6O9r9mrikEyNyE3RtupGvbx2r6q8Pn3/S8N04cgchcOGFm+uUm1Tq84Z2tbcKjfNI5fTrvrmQOQNrU3769XoDykcNtQ/1a3sFLfSExNks0meBIcMw1Btc0DPlu7UkKxkXTw6R8+8v1O7DjVryml58voCunBkjpJcDu061CxfIKTTB6YrbLQtHXA57WpuDSnNk6C9dS0KhsIamZeqTw80yW6TDLW9GepqfwOpyutXSyCkzGSXZEit7ZcsuW/RJ/r7mj165JtnafLwbO041KSCDI/6p7h7fRZDXISfSZMm6eyzz9YjjzwiSQqHwyosLNSPfvQj3XHHHVH7+v1++f2Hp2p5vV4VFhbGZfiRJK8voFDIOGpdypH+X+lOzX1lo4ZmJ2vpf1/Y5T4vrqrQntoWzS4ZccIn3rtbD+ieNz7R5vYqx/t3XKSCjETd+coGvbhqj1764bkanZ+mJ5dv191vbNY3JxZq3tWna9ehJn310fd11bgBuvOKU/WdP34Y1Uigw99/cK6++8cPlehy6NHpZ2nCkEwZhqHqBr8+qWzQBSOyo8Zc3xyQ02FTstup97cd1MJ1e1W2o0ZXnF6gC0f219cWlEYd35Ng1xs/Pl8/f22TlnXx9ccVZmhdRZ0k6XvnFWnRhv1K8Th125RR+o9nV0mSRuSkKNntVGFmkv7R/uLebpN+dvmp+t7kIknSK+v26uYX1h11/PumjdVT7+3UtupG3X7pKBUPy9K/P3z4Rek5QzO1amftMStQLqddI3JStPEYUw87XHBKf23e741MW3TYbTp3WJbe3XpQw3NSlJfm0Ypt0UHQk2Dv8sW/1HZ9qc4vwgH0TSlup5pbgydd1f8iimWWw/FIcTsVCIWjZjnYbJIZr8gS299UPfLvu83WFlYq632RinuH3DS3fIFw1KU1XA575EXv8fAk2JXqSdDBRr8pzyPBYTtqnD0hyeVQIBSO+Wsnu9r+j+3qPEp1O9XgD8pmk2z6/E6+DrvthN9IPNFZNKlup8KGodYTfO5up12bf3Fpr6/ZtXz4aW1tVVJSkv72t7/pqquuimyfMWOG6urq9Morr0Ttf9ddd+nnP//5UceJ1/BzvFqDYb21sVLFw7KUndJ9c+3/uqpC/mBY150zWNLR5f1gKKwN+7w6fUB6lyd3fUtAK3fU6JnSnbr6rAHavL9B4wf305TT8hQKG7JJpvxSVDf4ZLfZ1OwPKTfdrZbWkDKSDgfIcNjQPW9sVlH/ZE2f1PZc3t16QPvqWvT1CYUKto/F6bBrw9769neZDj9+5c4a/XVlhb59zmCdcUTziB0Hm5SZ5FJNc6vqmlu1papB14wvVFNrUC2toUgJ+PFln+rJd3fo8evO0vjBmQqEwnLabdq8v0GvfbxPF57SX3UtAQ3NTlZ9S0BpiQkakJGoB/5ZrtqmVm2tbtTAfom68fyhWr7lgEpOzVVWilvPfbBL3z5nsJwOm9burtO26kZNLMrUmIJ0/WXlbl02Nl976lp09z82a9ehZuWnezS7ZITOHZatzZVeDc9J0bLyA/rpy+sVCBl64JozdPnp+frnxkp9sL1GqR6nNu33Khw2dMlpuWrwtU2LGj+4n+56baM+qqjTRaNydEpuqlLcTqUnJuitjZXKSHLpxfYq4CPfOksb99UrI8mlTfu8+uN7O3TO0Ez926l5WrmjRr5gSIMzk/SDC4dr0/569U/xKNnt0F2vbdKN5xfpk/0Nqm1uewfwjQ37o6bsXX56vi45NVcfbD+kgf2SlJPq1str9+qU3FQVZrZ9ft+iT5ST6tbvvjFOP1u4QVurGnXBKdkKG9KSzVXqn+rWhCGZGpqdrPV76/XJ/gZ9+5xB+sf6/Vq7u04TizL19QmF+vVb5frqmQOU4nbq8eXbNW38ANU3B2SztTUm+d3iLdpb16IzCjM0MCNRizdX6dT8NE07a4BeXL1Hm/Z5FQwbGp2fJn97BSPBYdOInFQluhwaU5CmzfsbdNnYPO2v97Wt1dpxKPKOfmayS4bRVoUoyEjUJ5UNmj5pkAoyEjX/nW1qbj06sB75IuG0gjQNyEhU6fZDavQHlep2KjfNE3lXfdLQTP25rEJ761pUmJmozfsb1OQPKhg2NDwnRfUtAfnbXzh5u6hEDshI1IjcFNU0taqlNaRh/VN01uAM7a/3KSfVo4raZtU1t6q8skE7DjYpbLRVqIZkJcsXDGnDXq/y0z0anpOinYealJPqUXNrSHtqmtXgD+q84Vka0/4u9YptB6Oqu6lup8YNytCe2ha1tIaU4nEqMcGhBl9AOw8167zhWUrzJGj1rloZapsm6guEtXpXbdSbAQ67TQUZHu2v8x31BoXN1lYF3dk+9gEZidpX3xJ5QVeQ7tG+9um6qW6nUjxOVTf4j5rW25knwa4Ud4L8wZAafEElu9qmbnoSHLLb2t4IcTntka8ptb1QlRSpqrocdhUPy1JNU6v6Jbu0aV+9hvZP0Yc72ipww/ona3R+mrZUNWhLVaNOyU3R1WcN1MZ9Xq3eWaO0xAQdaGh7R9fltOuikTl65aN9R417QEaiWkNhNfuDMtT2gtyQoXBYChuG8jPaqi6HmlqVnpigxvaprb5AOPJi2eWw69IxedpW3Sh/MKTa5oBqmg5XU+02aezADJ1ZmKE1u2vV6A+q2R9Spdcnt9OuYHsl7ngUZSerf6pbOw82qbrBH3ljx2G3KdXjVOiIinqax6mLR+cqJ80tw5A+qqhTgy+oc4dlaW9diz7eUx+ZoZCf7lF1g18jclJ0yWl5WvpJVVQ308+Tl+ZRTVOrWkNheRLscjsdavQHI88tPTFBdpsi60s76zxVPM3jlMtp18HGVnkS7BpTkK69dS2qbW6VLxA+5gvd4TkpCoeNSDVVavsZTx2Tp7qWgF5dt+9zQ05msktN/qAykhJU3dAWbjq+ps3WFuY+a9ZCdopLKW6nmlpDUWuO7ba2xk45qW7ZbTbtrWuRLxBS/9S2/987/vY47DZlJCaowd/2e1PbHNCAjER5WwKRar2kSIXS5bDLZpMCoeM/hz5PV9P2HXabDMNQYoJDvmD4M7+Wy2mX3Sb5AuHPDNVZyS4Fw4b8wZBssqk19NnH6zA4K0nD+6doaXn1cYXNjp9XksshT4Ij8vvotLf9Hfq8ddldSXDYNDQ7RW/dcsEJP9Zslg8/+/bt04ABA/T++++ruLg4sv22227TsmXLVFZWFrX/F63yA/SW5tagHHab3E7z5uEHQ22tz4+cwlnX3Npetj/54LvzYJMMRTcK+SyhsCG7TSf19YKh8AldPLdz44pw2IgK96GwETVFtbk1qEDIOGZzCcMwFDba3qlNdjlkGG1vGBiGoQMNfvVPbZtS4A+GVN8ckNvpUHpSQtQbFb72F3u+QEgpbqdsNpsCobBag+HP7BzZ+XmEwoaqG3yRdTdS25qNnYeaVJSdfNLnjNcX0P46n4bnpEReLOyv9yk/3XPcPyuvL6Bkl1P1LQGlepxK6OJnZRiG9ta1aEBG4jGP6w+G5A+GZRhtLzxD7S+y7ba2FzMHGv2SIeWkebTjYJNS3E71T3XLFwipviUQuQh1MBRWVYNfOaluJTjskQYWobDR9kaLTZHv2aFGv9ITE477HKttam2fDtf2+EZ/UC6HXS2tocglADprbg2qoqZFI/NSI9+LRn8wch4cy766Fu2ra5EnwaGcVLeS3M4uO0p26Hy+d5w/wVBYDnvbCzZfIKwGX0ApbmfUm0sdY2rytwUvp8OmtCPWcvkCIZVXNmjsgHQZkg41+ZWd7FZNc6uSXU61BsNKS3Sqrv3NCKfDLofNdtTFuju+Z0kJjsjvUaXXpySXU2nt18M71vfFMAztqW1pD8iJqmtuVYrbKafDHvmd7Jfskk1Skz8kh8MWWYvYEggpI9Elh90W+TvgD4bUGgxHfh7BUFjlVQ1yO+0amp0ie/vvbX1LQK3BsPLSPUpw2OX1BRQIhlXXEtDgzCQ5Hfb28yscObfC4bbnlpnskifBofrmgLy+gAxD6pecIEOKfJ+rvD7tONikgf0So35PDjX6VV7VoOwUt3LTPErztP2uHWxsVf8Utw40+jWsf3LU96ylte3nuK+uRVkpLnmcDm070Kgmf9v/Mafmp6mqwa8UlzPqnO34P6i8skGj89OO+l0Ot//uhMKHf2YZiQlRf2M71lQ2+AJq8AVV3xJQemKCslPccthtsulwEGpqDSkxwaGwYaiy/e9O2Gh70b6lqlHZKS41t4aU5HKo0uvTKbmp2rTPK6fD1vamR51P9S0BnTkoQ26nXQca/EpPStDe2hblpnkif1tbg2FV1DZrYL9EhcPS9oONSnY5Zagt/DnsNlV5/RqSlaTa5oCqG3wqyEhUfXNACQ77UWsKW4OHp0cfaPCryR9Uv/bfJ2f7+dZx7hxo8CvV0/Z70fE3KMFhV11LQKFwWP1TPe3rcKVdh5o1ov16k96WoJLdDjkddvkCIa3dXaf0xASlJTrltNvVP9Ud+dtot9nk9QV0qKlVQ7KSVd3g06HG1sh60yOXZfSGL1z4OVI8r/kBAAAAYJ4TyQbH/1anibKzs+VwOFRVFX1tmaqqKuXl5X3GowAAAADg5PVK+HG5XBo/fryWLDl8bZZwOKwlS5ZEVYIAAAAAwCy9dpHTW2+9VTNmzNCECRM0ceJEPfjgg2pqatL111/fW0MCAAAA8AXWa+HnG9/4hg4cOKA777xTlZWVGjdunBYtWqTc3NzeGhIAAACAL7Beu85PLGh4AAAAAECKg4YHAAAAANDTCD8AAAAA+gTCDwAAAIA+gfADAAAAoE8g/AAAAADoEwg/AAAAAPoEwg8AAACAPoHwAwAAAKBPIPwAAAAA6BMIPwAAAAD6BMIPAAAAgD6B8AMAAACgT3D29gBOhmEYkiSv19vLIwEAAADQmzoyQUdGOJa4DD8NDQ2SpMLCwl4eCQAAAAAraGhoUHp6+jH3sRnHE5EsJhwOa9++fUpNTZXNZuvVsXi9XhUWFqqiokJpaWm9OhbED84bnCjOGZwozhmcKM4ZnCirnDOGYaihoUEFBQWy24+9qicuKz92u10DBw7s7WFESUtL4w8FThjnDU4U5wxOFOcMThTnDE6UFc6Zz6v4dKDhAQAAAIA+gfADAAAAoE8g/MTI7Xbrf//3f+V2u3t7KIgjnDc4UZwzOFGcMzhRnDM4UfF4zsRlwwMAAAAAOFFUfgAAAAD0CYQfAAAAAH0C4QcAAABAn0D4AQAAANAnEH4AAAAA9AmEnxjNnz9fQ4YMkcfj0aRJk/Thhx/29pDQC+bNm6ezzz5bqampysnJ0VVXXaXy8vKofXw+n2bOnKmsrCylpKRo2rRpqqqqitpn9+7duvzyy5WUlKScnBz95Cc/UTAY7Mmngl5y7733ymazafbs2ZFtnDPoyt69e/Xtb39bWVlZSkxM1NixY7Vq1arI/YZh6M4771R+fr4SExNVUlKirVu3Rh2jpqZG06dPV1pamjIyMnTDDTeosbGxp58KekAoFNLcuXNVVFSkxMREDRs2TL/85S/Vudkv50zftnz5cl1xxRUqKCiQzWbTwoULo+436/z4+OOPdf7558vj8aiwsFD3339/dz+1rhk4aS+88ILhcrmMP/7xj8bGjRuNG2+80cjIyDCqqqp6e2joYVOmTDGeeuopY8OGDca6deuMyy67zBg0aJDR2NgY2ef73/++UVhYaCxZssRYtWqVcc455xjnnntu5P5gMGiMGTPGKCkpMdauXWu88cYbRnZ2tjFnzpzeeEroQR9++KExZMgQ4/TTTzduvvnmyHbOGRyppqbGGDx4sPHd737XKCsrM7Zv32689dZbxrZt2yL73HvvvUZ6erqxcOFC46OPPjK+8pWvGEVFRUZLS0tkn0svvdQ444wzjA8++MB49913jeHDhxvf/OY3e+MpoZvdfffdRlZWlvH6668bO3bsMF588UUjJSXF+P3vfx/Zh3Omb3vjjTeMn/70p8ZLL71kSDJefvnlqPvNOD/q6+uN3NxcY/r06caGDRuMP//5z0ZiYqLx+OOP99TTjCD8xGDixInGzJkzI5+HQiGjoKDAmDdvXi+OClZQXV1tSDKWLVtmGIZh1NXVGQkJCcaLL74Y2Wfz5s2GJKO0tNQwjLY/Pna73aisrIzs89hjjxlpaWmG3+/v2SeAHtPQ0GCMGDHCWLx4sfGlL30pEn44Z9CV22+/3Zg8efJn3h8Oh428vDzj17/+dWRbXV2d4Xa7jT//+c+GYRjGpk2bDEnGypUrI/u8+eabhs1mM/bu3dt9g0evuPzyy43vfe97UduuvvpqY/r06YZhcM4g2pHhx6zz49FHHzX69esX9X/T7bffbowcObKbn9HRmPZ2klpbW7V69WqVlJREttntdpWUlKi0tLQXRwYrqK+vlyRlZmZKklavXq1AIBB1vowaNUqDBg2KnC+lpaUaO3ascnNzI/tMmTJFXq9XGzdu7MHRoyfNnDlTl19+edS5IXHOoGuvvvqqJkyYoGuuuUY5OTk688wz9eSTT0bu37FjhyorK6POm/T0dE2aNCnqvMnIyNCECRMi+5SUlMhut6usrKznngx6xLnnnqslS5Zoy5YtkqSPPvpIK1as0NSpUyVxzuDYzDo/SktLdcEFF8jlckX2mTJlisrLy1VbW9tDz6aNs0e/2hfIwYMHFQqFol50SFJubq4++eSTXhoVrCAcDmv27Nk677zzNGbMGElSZWWlXC6XMjIyovbNzc1VZWVlZJ+uzqeO+/DF88ILL2jNmjVauXLlUfdxzqAr27dv12OPPaZbb71V//M//6OVK1fqxz/+sVwul2bMmBH5uXd1XnQ+b3JycqLudzqdyszM5Lz5Arrjjjvk9Xo1atQoORwOhUIh3X333Zo+fbokcc7gmMw6PyorK1VUVHTUMTru69evX7eMvyuEH8BkM2fO1IYNG7RixYreHgosrKKiQjfffLMWL14sj8fT28NBnAiHw5owYYLuueceSdKZZ56pDRs2aMGCBZoxY0Yvjw5W9Ne//lXPPfecnn/+eZ122mlat26dZs+erYKCAs4Z9ElMeztJ2dnZcjgcR3VeqqqqUl5eXi+NCr1t1qxZev311/XOO+9o4MCBke15eXlqbW1VXV1d1P6dz5e8vLwuz6eO+/DFsnr1alVXV+uss86S0+mU0+nUsmXL9NBDD8npdCo3N5dzBkfJz8/XqaeeGrVt9OjR2r17t6TDP/dj/d+Ul5en6urqqPuDwaBqamo4b76AfvKTn+iOO+7Qtddeq7Fjx+q6667TLbfconnz5kninMGxmXV+WOn/K8LPSXK5XBo/fryWLFkS2RYOh7VkyRIVFxf34sjQGwzD0KxZs/Tyyy9r6dKlR5V2x48fr4SEhKjzpby8XLt3746cL8XFxVq/fn3UH5DFixcrLS3tqBc7iH8XX3yx1q9fr3Xr1kVuEyZM0PTp0yMfc87gSOedd95RbfS3bNmiwYMHS5KKioqUl5cXdd54vV6VlZVFnTd1dXVavXp1ZJ+lS5cqHA5r0qRJPfAs0JOam5tlt0e/3HM4HAqHw5I4Z3BsZp0fxcXFWr58uQKBQGSfxYsXa+TIkT065U0Sra5j8cILLxhut9t4+umnjU2bNhk33XSTkZGREdV5CX3DD37wAyM9Pd3417/+Zezfvz9ya25ujuzz/e9/3xg0aJCxdOlSY9WqVUZxcbFRXFwcub+jbfEll1xirFu3zli0aJHRv39/2hb3IZ27vRkG5wyO9uGHHxpOp9O4++67ja1btxrPPfeckZSUZPzpT3+K7HPvvfcaGRkZxiuvvGJ8/PHHxpVXXtllW9ozzzzTKCsrM1asWGGMGDGCtsVfUDNmzDAGDBgQaXX90ksvGdnZ2cZtt90W2Ydzpm9raGgw1q5da6xdu9aQZPz2t7811q5da+zatcswDHPOj7q6OiM3N9e47rrrjA0bNhgvvPCCkZSURKvrePTwww8bgwYNMlwulzFx4kTjgw8+6O0hoRdI6vL21FNPRfZpaWkxfvjDHxr9+vUzkpKSjK9+9avG/v37o46zc+dOY+rUqUZiYqKRnZ1t/Nd//ZcRCAR6+NmgtxwZfjhn0JXXXnvNGDNmjOF2u41Ro0YZTzzxRNT94XDYmDt3rpGbm2u43W7j4osvNsrLy6P2OXTokPHNb37TSElJMdLS0ozrr7/eaGho6MmngR7i9XqNm2++2Rg0aJDh8XiMoUOHGj/96U+jWg5zzvRt77zzTpevYWbMmGEYhnnnx0cffWRMnjzZcLvdxoABA4x77723p55iFJthdLrELwAAAAB8QbHmBwAAAECfQPgBAAAA0CcQfgAAAAD0CYQfAAAAAH0C4QcAAABAn0D4AQAAANAnEH4AAAAA9AmEHwAAAAB9AuEHAAAAQJ9A+AEAAADQJxB+AAAAAPQJ/x8kh/Kt8w03eAAAAABJRU5ErkJggg==", "text/plain": [ "
" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "errors = []\n", "for nb_sample in range(1, 1000):\n", " \n", " rewards = np.zeros((nb_actions, nb_sample))\n", "\n", " for a in actions:\n", " for play in range(nb_sample):\n", " rewards[a, play] = get_reward(Q_star, a, var=10.0)\n", "\n", " Q_t = np.mean(rewards, axis=1)\n", " error = np.mean((Q_star - Q_t)**2)\n", " errors.append(error)\n", " \n", "print(error)\n", "\n", "plt.figure(figsize=(10, 6))\n", "plt.plot(errors)\n", "plt.show()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "**A:** the higher the variance of the distribution, the more samples we need to get correct estimates. " ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Bandit environment\n", "\n", "In order to prepare the next exercise, let's now implement the n-armed bandit in a Python class. As reminded in the tutorial on Python, a class is defined using this structure:\n", "\n", "```python\n", "class MyClass:\n", " \"\"\"\n", " Documentation of the class.\n", " \"\"\"\n", " def __init__(self, param1, param2):\n", " \"\"\"\n", " Constructor of the class.\n", " \n", " :param param1: first parameter.\n", " :param param2: second parameter.\n", " \"\"\"\n", " self.param1 = param1\n", " self.param2 = param2\n", " \n", " def method(self, another_param):\n", " \"\"\"\n", " Method to do something.\n", " \n", " :param another_param: another parameter.\n", " \"\"\"\n", " return (another_param + self.param1)/self.param2\n", "```\n", "\n", "You can then create an object of the type `MyClass`:\n", "\n", "```python\n", "my_object = MyClass(param1= 1.0, param2=2.0)\n", "```\n", "\n", "and call any method of the class on the object:\n", "\n", "```python\n", "result = my_object.method(3.0)\n", "```\n", "\n", "**Q:** Create a `Bandit` class taking as arguments:\n", "\n", "* nb_actions: number of arms.\n", "* mean: mean of the normal distribution for $Q^*$.\n", "* std_Q: standard deviation of the normal distribution for $Q^*$.\n", "* std_r: standard deviation of the normal distribution for the sampled rewards.\n", "\n", "The constructor should initialize a `Q_star` array accordingly and store it as an attribute. It should also store the optimal action.\n", "\n", "Add a method `step(action)` that samples a reward for a particular action and returns it." ] }, { "cell_type": "code", "execution_count": 10, "metadata": {}, "outputs": [], "source": [ "class Bandit:\n", " \"\"\"\n", " n-armed bandit.\n", " \"\"\"\n", " def __init__(self, nb_actions, mean=0.0, std_Q=1.0, std_r=1.0):\n", " \"\"\"\n", " :param nb_actions: number of arms.\n", " :param mean: mean of the normal distribution for $Q^*$.\n", " :param std_Q: standard deviation of the normal distribution for $Q^*$.\n", " :param std_r: standard deviation of the normal distribution for the sampled rewards.\n", " \"\"\"\n", " # Store parameters\n", " self.nb_actions = nb_actions\n", " self.mean = mean\n", " self.std_Q = std_Q\n", " self.std_r = std_r\n", " \n", " # Initialize the true Q-values\n", " self.Q_star = rng.normal(self.mean, self.std_Q, self.nb_actions)\n", " \n", " # Optimal action\n", " self.a_star = self.Q_star.argmax()\n", " \n", " def step(self, action):\n", " \"\"\"\n", " Sampled a single reward from the bandit.\n", " \n", " :param action: the selected action.\n", " :return: a reward.\n", " \"\"\"\n", " return float(rng.normal(self.Q_star[action], self.std_r, 1)[0])" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "**Q:** Create a 5-armed bandits and sample each action multiple times. Compare the mean reward to the ground truth as before." ] }, { "cell_type": "code", "execution_count": 11, "metadata": {}, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAA18AAAINCAYAAADBd8UhAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjkuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8hTgPZAAAACXBIWXMAAA9hAAAPYQGoP6dpAABMnUlEQVR4nO3de1xVdb7/8fcGBNQEUxDEu+aIpkLiZdAmLDlC+tAwx9ScUUltxiNlYpqYg6k5VCcVJxzJvOWU6TiZY2XOQcrKkTRFpqykbCzwslEzQTHBYP/+6Oee9hGQ2177wuv5eKzHaX/3d6392Xvke3jvz1oLk8VisQgAAAAAYFceji4AAAAAABoCwhcAAAAAGIDwBQAAAAAGIHwBAAAAgAEIXwAAAABgAMIXAAAAABiA8AUAAAAABiB8AQAAAIABvBxdgKsqLy/X6dOn1axZM5lMJkeXAzRYFotFly5dUkhIiDw8XOf7JNYQwHmwjgCoq+quI4SvWjp9+rTatWvn6DIA/H/5+flq27ato8uoNtYQwPmwjgCoq5utI4SvWmrWrJmknz5gPz8/B1cDNFxFRUVq166d9WfSVbCGAM6DdQRAXVV3HSF81dL19r6fnx8LHuAEXO2UG9YQwPmwjgCoq5utI65zYjMAAAAAuDDCFwAAAAAYwOXD1wcffKARI0YoJCREJpNJO3bsqHL+3r17ZTKZbtjMZrMxBQMAAABokFw+fBUXFyssLEyrVq2q0X65ubk6c+aMdWvVqpWdKgQAAAAANwhf9957r55++mmNGjWqRvu1atVKwcHB1s2V/q4HgPpFBx0AABihwSaO8PBwtW7dWv/1X/+lf/7zn44uB4AD0UEHAABGaHC3mm/durXS09PVt29flZSUaO3atRo8eLAOHDigPn36VLpfSUmJSkpKrI+LioqMKBeAAe69917de++9Nd6vVatWat68ef0XBAAA3FKDC1/dunVTt27drI8HDhyor7/+WitWrNBf/vKXSvdLSUnRokWLjCgRgIsIDw9XSUmJevbsqaeeekqDBg2qdC5f4AAAgAZ72uHP9e/fX8ePH69yTlJSkgoLC61bfn6+QdUBcDbXO+ivv/66Xn/9dbVr106DBw9WdnZ2pfukpKTI39/furVr187AigEAgDNocJ2viuTk5Kh169ZVzvHx8ZGPj49BFQFwZrXpoCclJSkxMdH6uKioiAAGAEAD4/Lh6/LlyzZdqxMnTignJ0ctWrRQ+/btlZSUpFOnTmnTpk2SpNTUVHXq1Em33367rl69qrVr1+rdd9/V//7v/zrqLQBwA/3799e+ffsqfZ4vcAAAgMuHr0OHDunuu++2Pr7+zfKkSZO0ceNGnTlzRnl5edbnS0tLNXv2bJ06dUpNmjRR7969tWfPHptjAEBNVaeDDgAAGjaXD1+DBw+WxWKp9PmNGzfaPJ47d67mzp1r56oAuBI66AAAwAguH74AoK7ooAMAACOYLFW1jVCpoqIi+fv7q7CwUH5+fo4uB2iwXPVn0VXrBtyRq/48umrdgDuq7s8jt5oHAAAAAAMQvgAAAADAAIQvAAAAADAAN9wAcIOO8952dAmSpG+eGe7oEgDUgrOsIRLrCADnQvgCAACA0yC8w51x2iEAAAAAGIDwBQAAAAAGIHwBAAAAgAEIXwAAAABgAMIXAAAAABiA8AUAAAAABiB8AQAAAIABCF8AAAAAYADCFwAAAAAYgPAFAAAAAAYgfAEAAACAAQhfAAAAAGAAwhcAAAAAGIDwBQAAAAAGIHwBAAAAgAEIXwAAAABgAMIXAAAAABiA8AUAAAAABiB8AQAAAIABCF8AAAAAYADCFwAAAAAYgPAFAAAAAAYgfAEAAACAAQhfAAAAAGAAwhcAAAAAGIDwBQAAAAAGIHwBAAAAgAEIXwAAAABgAMIXAAAAABiA8AUAAAAABiB8AQAAAIABCF8AAAAAYADCFwAAAAAYgPAFAAAAAAYgfAEAAACAAQhfAAAAAGAAwhcAAAAAGIDwBQAAAAAGIHwBAAAAgAEIXwAAAABgAJcPXx988IFGjBihkJAQmUwm7dix46b77N27V3369JGPj49uu+02bdy40e51AgAAAGjYXD58FRcXKywsTKtWrarW/BMnTmj48OG6++67lZOTo8cee0xTp07VP/7xDztXCgAAAKAhc/nwde+99+rpp5/WqFGjqjU/PT1dnTp10rJly9S9e3clJCTo17/+tVasWGHnSgE4KzroAADACC4fvmoqKytL0dHRNmMxMTHKyspyUEUAHI0OOgAAMEKDC19ms1lBQUE2Y0FBQSoqKtIPP/xQ6X4lJSUqKiqy2QC4BzroAG5m1apV6tixo3x9fTVgwAAdPHiwyvnbtm1TaGiofH191atXL+3atcvm+cmTJ8tkMtlssbGx9nwLAJxAgwtftZWSkiJ/f3/r1q5dO0eXBMBBatNB5wscwHVt3bpViYmJWrhwobKzsxUWFqaYmBidPXu2wvn79+/X+PHjNWXKFB05ckRxcXGKi4vT0aNHbebFxsbqzJkz1u21114z4u0AcKAGF76Cg4NVUFBgM1ZQUCA/Pz81bty40v2SkpJUWFho3fLz8+1dKgAnVZsOOl/gAK5r+fLlmjZtmuLj49WjRw+lp6erSZMmWr9+fYXzV65cqdjYWM2ZM0fdu3fXkiVL1KdPH6WlpdnM8/HxUXBwsHW79dZbjXg7AByowYWvyMhIZWZm2oxlZGQoMjKyyv18fHzk5+dnswFAdfEFDuCaSktLdfjwYZtut4eHh6Kjoyvtdle3O7537161atVK3bp10/Tp0/Xdd99VWQsddMD1uXz4unz5snJycpSTkyPppwvhc3JylJeXJ+mnX3gmTpxonf/73/9e//73vzV37lwdO3ZMf/7zn/XXv/5Vs2bNckT5AFxQbTrofIEDuKbz58+rrKyswm632WyucJ/KuuM/nx8bG6tNmzYpMzNTzz77rN5//33de++9Kisrq7QWOuiA6/NydAF1dejQId19993Wx4mJiZKkSZMmaePGjTpz5ow1iElSp06d9Pbbb2vWrFlauXKl2rZtq7Vr1yomJsbw2gG4psjIyBsunq9OBx0Arhs3bpz1v3v16qXevXurS5cu2rt3r4YMGVLhPklJSdbfcySpqKiIAAa4GJcPX4MHD5bFYqn0+Yr+9s7gwYN15MgRO1YFwJVcvnxZx48ftz6+3kFv0aKF2rdvr6SkJJ06dUqbNm2S9FMHPS0tTXPnztVDDz2kd999V3/961/19ttvO+otALCTgIAAeXp6VtjtDg4OrnCfyrrjlc2XpM6dOysgIEDHjx+vNHz5+PjIx8enhu8AgDNx+dMOAaCuDh06pDvuuEN33HGHpJ866HfccYeSk5MlqdIOekZGhsLCwrRs2TI66ICb8vb2VkREhM314uXl5crMzKy0212b68tPnjyp7777Tq1bt66fwgE4JZfvfAFAXdFBB1CVxMRETZo0SX379lX//v2Vmpqq4uJixcfHS5ImTpyoNm3aKCUlRZI0c+ZMRUVFadmyZRo+fLi2bNmiQ4cOac2aNZJ+6rYvWrRIo0ePVnBwsL7++mvNnTtXt912G1/iAG6O8AUAAFCFsWPH6ty5c0pOTpbZbFZ4eLh2795tvalGXl6ePDz+czLRwIEDtXnzZi1YsEDz589X165dtWPHDvXs2VOS5OnpqU8++UQvv/yyLl68qJCQEA0dOlRLlizhtELAzRG+AAAAbiIhIUEJCQkVPrd3794bxsaMGaMxY8ZUOL9x48b6xz/+UZ/lAXARXPMFAAAAAAYgfAEAAACAAQhfAAAAAGAAwhcAAAAAGIDwBQAAAAAGIHwBAAAAgAEIXwAAAABgAMIXAAAAABiA8AUAAAAABiB8AQAAAIABCF8AAAAAYADCFwAAAAAYgPAFAAAAAAYgfAEAAACAAQhfAAAAAGAAwhcAAAAAGIDwBQAAAAAGIHwBAAAAgAEIXwAAAABgAMIXAAAAABiA8AUAAAAABiB8AQAAAIABCF8AAAAAYADCFwAAAAAYgPAFAAAAAAbwcnQBQEPQcd7bji7B6ptnhju6BAAAgAaJzhcAAAAAGIDwBQAAAAAGIHwBAAAAgAG45gsAABfgLNeOct0oANQenS8AAAAAMADhCwAAAAAMQPgCAAAAAAMQvgAAAADAAIQvAAAAADAA4QsAAAAADED4AgAAAAADEL4AAAAAwACELwAAAAAwAOELAAAAAAxA+AIAAAAAAxC+AAAAAMAAhC8AAAAAMIDbhK9Vq1apY8eO8vX11YABA3Tw4MFK527cuFEmk8lm8/X1NbBaAAAAAA2NW4SvrVu3KjExUQsXLlR2drbCwsIUExOjs2fPVrqPn5+fzpw5Y92+/fZbAysGAAAA0NC4Rfhavny5pk2bpvj4ePXo0UPp6elq0qSJ1q9fX+k+JpNJwcHB1i0oKMjAigE4G7rnAADA3lw+fJWWlurw4cOKjo62jnl4eCg6OlpZWVmV7nf58mV16NBB7dq103333afPPvvMiHIBOCG65wAAwAguH77Onz+vsrKyGzpXQUFBMpvNFe7TrVs3rV+/Xn//+9/1yiuvqLy8XAMHDtTJkycrfZ2SkhIVFRXZbADcA91zAABgBJcPX7URGRmpiRMnKjw8XFFRUdq+fbsCAwP14osvVrpPSkqK/P39rVu7du0MrBiAvRjVPecLHAAA4PLhKyAgQJ6eniooKLAZLygoUHBwcLWO0ahRI91xxx06fvx4pXOSkpJUWFho3fLz8+tUNwDnYFT3nC9wAACAy4cvb29vRUREKDMz0zpWXl6uzMxMRUZGVusYZWVl+vTTT9W6detK5/j4+MjPz89mA9Aw1aZ7zhc4AADAy9EF1IfExERNmjRJffv2Vf/+/ZWamqri4mLFx8dLkiZOnKg2bdooJSVFkrR48WL98pe/1G233aaLFy/qf/7nf/Ttt99q6tSpjnwbABzAqO65j4+PfHx86lQrAABwbS7f+ZKksWPH6vnnn1dycrLCw8OVk5Oj3bt3W08jysvL05kzZ6zzv//+e02bNk3du3fXsGHDVFRUpP3796tHjx6OegsAHMSo7jkAAIBbdL4kKSEhQQkJCRU+t3fvXpvHK1as0IoVKwyoCoAroHsOAACM4DbhCwBqa+zYsTp37pySk5NlNpsVHh5+Q/fcw+M/Jwpc756bzWbdeuutioiIoHsOAABuivAFAKJ7DgAA7M8trvkCAAAAAGdH+AIAAAAAAxC+AAAAAMAAhC8AAAAAMADhCwAAAAAMQPgCAAAAAAMQvgAAAADAAIQvAAAAADAA4QsAAAAADED4AgAAAAADEL4AAAAAwACELwAAgJtYtWqVOnbsKF9fXw0YMEAHDx6scv62bdsUGhoqX19f9erVS7t27ap07u9//3uZTCalpqbWc9UAnA3hCwAAoApbt25VYmKiFi5cqOzsbIWFhSkmJkZnz56tcP7+/fs1fvx4TZkyRUeOHFFcXJzi4uJ09OjRG+a+8cYb+uijjxQSEmLvtwHACRC+AAAAqrB8+XJNmzZN8fHx6tGjh9LT09WkSROtX7++wvkrV65UbGys5syZo+7du2vJkiXq06eP0tLSbOadOnVKjzzyiF599VU1atTIiLcCwMEIXwAAAJUoLS3V4cOHFR0dbR3z8PBQdHS0srKyKtwnKyvLZr4kxcTE2MwvLy/Xb3/7W82ZM0e33357tWopKSlRUVGRzQbAtRC+AAAAKnH+/HmVlZUpKCjIZjwoKEhms7nCfcxm803nP/vss/Ly8tKjjz5a7VpSUlLk7+9v3dq1a1eDdwLAGRC+AAAADHT48GGtXLlSGzdulMlkqvZ+SUlJKiwstG75+fl2rBKAPRC+AAAAKhEQECBPT08VFBTYjBcUFCg4OLjCfYKDg6uc/+GHH+rs2bNq3769vLy85OXlpW+//VazZ89Wx44dK63Fx8dHfn5+NhsA10L4AgAAqIS3t7ciIiKUmZlpHSsvL1dmZqYiIyMr3CcyMtJmviRlZGRY5//2t7/VJ598opycHOsWEhKiOXPm6B//+If93gwAh/NydAEAAADOLDExUZMmTVLfvn3Vv39/paamqri4WPHx8ZKkiRMnqk2bNkpJSZEkzZw5U1FRUVq2bJmGDx+uLVu26NChQ1qzZo0kqWXLlmrZsqXNazRq1EjBwcHq1q2bsW8OgKEIXwAAAFUYO3aszp07p+TkZJnNZoWHh2v37t3Wm2rk5eXJw+M/JxMNHDhQmzdv1oIFCzR//nx17dpVO3bsUM+ePR31FgA4CcIXAADATSQkJCghIaHC5/bu3XvD2JgxYzRmzJhqH/+bb76pZWUAXAnXfAEAAACAAQhfAAAAAGAAwhcAAAAAGIDwBQAAAAAGIHwBAAAAgAEIXwAAAABgAMIXAAAAABiA8AUAAAAABiB8AQAAAIABCF8AAAAAYADCFwAAAAAYgPAFAAAAAAYgfAEAAACAAQhfAAAAAGAAwhcAAAAAGIDwBQAAAAAGIHwBAAAAgAEIXwAAAABgAMIXAAAAABiA8AUAAAAABiB8AQAAAIABCF8AAAAAYADCFwAAAAAYwG3C16pVq9SxY0f5+vpqwIABOnjwYJXzt23bptDQUPn6+qpXr17atWuXQZUCAAAAaIjcInxt3bpViYmJWrhwobKzsxUWFqaYmBidPXu2wvn79+/X+PHjNWXKFB05ckRxcXGKi4vT0aNHDa4cAAAAQEPhFuFr+fLlmjZtmuLj49WjRw+lp6erSZMmWr9+fYXzV65cqdjYWM2ZM0fdu3fXkiVL1KdPH6WlpRlcOQBnQfccAADYm8uHr9LSUh0+fFjR0dHWMQ8PD0VHRysrK6vCfbKysmzmS1JMTEyl8wG4N7rngHuxlP2ogi3zde3CKUeXAgA2XD58nT9/XmVlZQoKCrIZDwoKktlsrnAfs9lco/mSVFJSoqKiIpsNgHugew64F5Onl0rPfuPoMgDgBl6OLsBVpKSkaNGiRbXat+O8t+u5mtr55pnhN53jLLVKN6/XlWqtzmfvTFyt3rq43j1PSkqyjlWne56YmGgzFhMTox07dlT6OiUlJSopKbE+5gscwL6a3n63Ln+SIelhR5cCAFYuH74CAgLk6empgoICm/GCggIFBwdXuE9wcHCN5ktSUlKSzS9bRUVFateuXR0qB+AMquqeHzt2rMJ9atM9d4cvcCTX+hLH3Wp1pS9FnKHWRx7ZrU2bNqlv376KiIhQ06ZNbZ5fvny5gyoD0JC5/GmH3t7eioiIUGZmpnWsvLxcmZmZioyMrHCfyMhIm/mSlJGRUel8SfLx8ZGfn5/NBgDVlZSUpMLCQuuWn5/v6JIAt3b06FH16dNHzZo105dffqkjR45Yt5ycHEeXB6CBcvnOlyQlJiZq0qRJ6tu3r/r376/U1FQVFxcrPj5ekjRx4kS1adNGKSkpkqSZM2cqKipKy5Yt0/Dhw7VlyxYdOnRIa9asceTbAOAARnXPfXx85OPjU/eCAVTLe++95+gSAOAGLt/5kqSxY8fq+eefV3JyssLDw5WTk6Pdu3dbTwvKy8vTmTNnrPMHDhyozZs3a82aNQoLC9Pf/vY37dixQz179nTUWwDgIEZ1zwE4zsmTJ3Xy5ElHlwEA7tH5kqSEhAQlJCRU+NzevXtvGBszZozGjBlj56oAuAK654D7KS8v19NPP61ly5bp8uXLkqRmzZpp9uzZevLJJ+Xh4RbfPwNwMW4TvgCgtsaOHatz584pOTlZZrNZ4eHhN3TPf/6L2vXu+YIFCzR//nx17dqV7jngZJ588kmtW7dOzzzzjAYNGiRJ2rdvn5566ildvXpVS5cudXCFABoiwhcAiO454G5efvllrV27ViNHjrSO9e7dW23atNF///d/E74AOAQ9dwAA4HYuXLig0NDQG8ZDQ0N14cIFB1QEAIQvAADghsLCwpSWlnbDeFpamsLCwhxQEQBw2iEAAHBDzz33nIYPH649e/ZY70SalZWl/Px87dq1y8HVAWio6HwBAAC3ExUVpS+//FKjRo3SxYsXdfHiRd1///3Kzc3Vr371K0eXB6CBovMFAADcyrVr1xQbG6v09HRurAHAqdD5AgAAbqVRo0b65JNPHF0GANyA8AUAANzOb37zG61bt87RZQCADU47BAAAbufHH3/U+vXrtWfPHkVERKhp06Y2zy9fvtxBlQFoyAhfcFnfPDPc0SUAAJzU0aNH1adPH0nSl19+afOcyWRyREkAQPgCAADupaysTIsWLVKvXr106623OrocALDimi8AAOBWPD09NXToUF28eNHRpQCADcIXAABwOz179tS///1vR5cBADY47RAA0GBx7aj7evrpp/X4449ryZIlFd5ww8/Pz0GVAWjICF8AAMDtDBs2TJI0cuRImxtsWCwWmUwmlZWVOao0AA0Y4QsAALid9957z9ElAMANCF8AAMDtREVFOboEALgBN9wAAABu6cMPP9RvfvMbDRw4UKdOnZIk/eUvf9G+ffscXBmAhorwBQAA3M7rr7+umJgYNW7cWNnZ2SopKZEkFRYW6o9//KODqwPQUBG+AACA23n66aeVnp6ul156SY0aNbKODxo0SNnZ2Q6sDEBDRvgCAABuJzc3V3fdddcN4/7+/vzxZQAOUy/h69q1a8rPz1dubq4uXLhQH4cEAACoteDgYB0/fvyG8X379qlz584OqAgA6hC+Ll26pNWrVysqKkp+fn7q2LGjunfvrsDAQHXo0EHTpk3Txx9/XJ+1AgAAVMu0adM0c+ZMHThwQCaTSadPn9arr76qxx9/XNOnT3d0eQAaqFrdan758uVaunSpunTpohEjRmj+/PkKCQlR48aNdeHCBR09elQffvihhg4dqgEDBuiFF15Q165d67t2AACACs2bN0/l5eUaMmSIrly5orvuuks+Pj56/PHH9cgjjzi6PAANVK3C18cff6wPPvhAt99+e4XP9+/fXw899JDS09O1YcMGffjhh4QvAJW6du2azGazrly5osDAQLVo0cLRJQFwcSaTSU8++aTmzJmj48eP6/Lly+rRo4duueUWR5cGoAGrVfh67bXXqjXPx8dHv//972vzEgDc3KVLl/TKK69oy5YtOnjwoEpLS2WxWGQymdS2bVsNHTpUDz/8sPr16+foUgG4MG9vb/Xo0cPRZQCApFqGr58rKyvT2rVrlZubq7Zt2yosLEzh4eFq2bJlfdQHwA1x6jIAAGiI6hy+HnnkEb3++uuKjo5WWlqaTCaTfvzxR7Vp00bh4eHauXNnfdQJwI1w6jIAAGiI6hy+tm/frk2bNikmJkY7d+7U/v379f7772vx4sXq0KFDfdQIwM1w6jIAAGiI6hy+rl/AKkmNGjWSl5eXEhISdO3aNZ0+fbrOBQIAAACAO6jzH1nu3LmzNWS1adNGp06dkiSNGDFCr7zySl0PD8DNlZWV6cUXX1RiYqKWL1+uzMxMfffdd44uC4Ab+Mtf/qJBgwYpJCRE3377rSQpNTVVf//73x1cGYCGqs7h6/7779c777wjSYqKitL69eslSZ9//rl++OGHuh4egJt75JFHlJycrIKCAs2bN0/Dhg1Tq1at1L59e40cOdLR5QFwUatXr1ZiYqKGDRumixcvqqysTJLUvHlzpaamOrY4AA1WncPXU089paeeekqSNHfuXGVmZiowMFBjxozRgw8+WNfDA3Bz168bffXVV+Xj46NDhw5p5cqVunr1KteNAqi1F154QS+99JKefPJJeXp6Wsf79u2rTz/9tMbHW7VqlTp27ChfX18NGDBABw8erHL+tm3bFBoaKl9fX/Xq1Uu7du2yef6pp55SaGiomjZtqltvvVXR0dE6cOBAjesC4FrqHL5+rn379vrss8/03HPPadu2bVq1alV9Hh6AG6rsutGkpCT5+vo6uDoArurEiRO64447bhj38fFRcXFxjY61detWJSYmauHChcrOzlZYWJhiYmJ09uzZCufv379f48eP15QpU3TkyBHFxcUpLi5OR48etc75xS9+obS0NH366afat2+fOnbsqKFDh+rcuXM1e6MAXEq9hi9JCggIUHx8vEaOHCmTyVTfhwfgZrhuFIA9dOrUSTk5OTeM7969W927d6/RsZYvX65p06YpPj5ePXr0UHp6upo0aWK91OL/WrlypWJjYzVnzhx1795dS5YsUZ8+fZSWlmad8+CDDyo6OlqdO3fW7bffruXLl6uoqEiffPJJjWoD4FpqFb7y8vJqNP/6L1MA8H9x3SgAe0hMTNSMGTO0detWWSwWHTx4UEuXLlVSUpLmzp1b7eOUlpbq8OHDio6Oto55eHgoOjpaWVlZFe6TlZVlM1+SYmJiKp1fWlqqNWvWyN/fX2FhYZXWUlJSoqKiIpsNgGup1a3m+/Xrp7i4OE2dOlX9+vWrcE5hYaH++te/auXKlXr44Yf16KOP1qlQAO7p+jWj0k/Xjfbr10+BgYEqKirSlClTHFcYAJc2depUNW7cWAsWLNCVK1f04IMPKiQkRCtXrtS4ceOqfZzz58+rrKxMQUFBNuNBQUE6duxYhfuYzeYK55vNZpuxt956S+PGjdOVK1fUunVrZWRkKCAgoNJaUlJStGjRomrXDsD51Cp8ff7551q6dKn+67/+S76+voqIiFBISIh8fX31/fff6/PPP9dnn32mPn366LnnntOwYcPqu24Abuj6daNvvvmmWrZsqREjRji6JAAubMKECZowYYKuXLmiy5cvq1WrVo4uycbdd9+tnJwcnT9/Xi+99JIeeOABHThwoNI6k5KSlJiYaH1cVFSkdu3aGVUugHpQq9MOW7ZsqeXLl+vMmTNKS0tT165ddf78eX311VeSflrsDh8+rKysLIIXgBtUdepyRdeNcuoygJq65557dPHiRUlSkyZNrIGmqKhI99xzT7WPExAQIE9PTxUUFNiMFxQUKDg4uMJ9goODqzW/adOmuu222/TLX/5S69atk5eXl9atW1dpLT4+PvLz87PZALiWOt1wo3Hjxvr1r3+t1NRUvfHGG9q9e7deeeUVzZ49Wz179qyvGgG4mX79+ul3v/udPv7440rnFBYW6qWXXlLPnj31+uuvG1gdAHewd+9elZaW3jB+9epVffjhh9U+jre3tyIiIpSZmWkdKy8vV2ZmpiIjIyvcJzIy0ma+JGVkZFQ6/+fHLSkpqXZtAFxPrU47BIC64NRlAPby87sFfv755zbXWZWVlWn37t1q06ZNjY6ZmJioSZMmqW/fvurfv79SU1NVXFys+Ph4SdLEiRPVpk0bpaSkSJJmzpypqKgoLVu2TMOHD9eWLVt06NAhrVmzRpJUXFyspUuXauTIkWrdurXOnz+vVatW6dSpUxozZkxdPwIATozwBcBw109dXrp0qd5++23t27dP3377rX744QcFBARowoQJiomJoYMOoMbCw8NlMplkMpkqPL2wcePGeuGFF2p0zLFjx+rcuXNKTk6W2WxWeHi4du/ebb2pRl5enjw8/nMy0cCBA7V582YtWLBA8+fPV9euXbVjxw7rmubp6aljx47p5Zdf1vnz59WyZUv169dPH374oW6//fY6vHsAzq7G4evDDz/Ur371K/3zn//UoEGD7FETgAbi+qnLv/71rx1dCgA3ceLECVksFnXu3FkHDx5UYGCg9Tlvb2+1atVKnp6eNT5uQkKCEhISKnxu7969N4yNGTOm0i6Wr6+vtm/fXuMaALi+Goevd955R15eXnr77bcJXwDq3Q8//KDGjRs7ugwALqpDhw6Sfrp+CgCcTY3C16JFi/Tjjz/qnnvu0aOPPqrFixcrOTnZXrUBaIDuvPNOHT582Gbs2LFjCg0NdVBFAFzRpk2bqnx+4sSJBlUCAP9Ro/C1cOFCvfTSS1qyZImaN2+uqVOn2qsuAA3Mm2++qc8//1yXL19Wfn6+zd+uGTt2rP71r385sDoArmbmzJk2j69du6YrV67I29tbTZo0IXwBcIga32r+xx9/1OOPP66ysjJ71FNjFy5c0IQJE+Tn56fmzZtrypQpunz5cpX7DB482Hox7vXt97//vUEVA6hIz5491axZM50/f14TJ05Uly5ddNddd2ns2LFq1KiRo8sD4GK+//57m+3y5cvKzc3VnXfeqddee83R5QFooGp8zdf06dMlSb/73e/qvZjamDBhgs6cOaOMjAxdu3ZN8fHxevjhh7V58+Yq95s2bZoWL15sfdykSRN7lwqgAv/+97/1r3/9S15eXho+fLh69uypu+66S9JPf1z522+/5a6HAOpF165d9cwzz+g3v/mNjh075uhyADRAdfojy472xRdfaPfu3Vq7dq0GDBigO++8Uy+88IK2bNmi06dPV7lvkyZNFBwcbN34K/GAscrKyhQfH6+uXbtq9OjRuu+++9S5c2c99dRT+vTTTyVJbdq00cCBA+3680n3HGhYvLy8bvo7AgDYS53/ztfPv7Xu3bu39S5DRsjKylLz5s3Vt29f61h0dLQ8PDx04MABjRo1qtJ9X331Vb3yyisKDg7WiBEj9Ic//IHuF2CgP/7xj9q5c6defPFFRUVF6YcfftDhw4f15z//WYMGDdKOHTsq/Bs99Y3uOeCedu7cafPYYrHozJkzSktL427NABym1uGrrKxMU6dO1aZNm2SxWCRJJpNJUVFRWrlypXr16lVvRVbGbDarVatWNmNeXl5q0aKFzV+0/78efPBBdejQQSEhIfrkk0/0xBNPKDc3t8q/uVFSUqKSkhLr46Kiorq/AaABe/nll7VixQqbi9579+6t+Ph4Pf/884qLi9NXX32lxo0b68iRI4qKiqr3Gq53zz/++GPrlzgvvPCChg0bpueff14hISGV7nu9ew7AOcXFxdk8NplMCgwM1D333KNly5Y5pigADV6tTzv8+bfWubm5ysnJ0dq1a3Xp0iUNGjRI7777bq2Lmjdv3g2n9PzfrS7naj/88MOKiYlRr169NGHCBG3atElvvPGGvv7660r3SUlJkb+/v3X7+Z3YANRcfn6+fvWrX1X43OOPP66xY8fqoYceUkREhPbv32+XGm7WPa/Kq6++qoCAAPXs2VNJSUm6cuVKlfNLSkpUVFRkswGwn/LycputrKxMZrNZmzdvVuvWrR1dHoAGqtadL3t+az179mxNnjy5yjmdO3dWcHCwzp49azP+448/6sKFCzX6RnrAgAGSpOPHj6tLly4VzklKSlJiYqL1cVFREQEMqIMWLVro+++/V6dOnSp8furUqYqMjNTUqVM1e/Zsu9RgZPc8JSVFixYtqrfaAQCA66l1+LrZt9a5ubl66KGH9OWXX+qhhx6qUfgKDAxUYGDgTedFRkbq4sWLOnz4sCIiIiRJ7777rsrLy62BqjpycnIkqcpvwnx8fOTj41PtYwKo2uDBg/XKK6+oT58+FT4fFBQkLy8vrVmzpsbHnjdvnp599tkq53zxxRc1Pu51Dz/8sPW/e/XqpdatW2vIkCH6+uuv+QIHcKCf/4zdzPLly+1YCQBUrNbhyxm+te7evbtiY2M1bdo0paen69q1a0pISNC4ceOs12qcOnVKQ4YM0aZNm9S/f399/fXX2rx5s4YNG6aWLVvqk08+0axZs3TXXXepd+/edqkTwI2eeOIJDRgwQBEREZowYcINzx86dEht27at1bGdsXvOFziA/R05cqRa80wmk50rAYCK1Tp82fNb65p49dVXlZCQoCFDhsjDw0OjR4/Wn/70J+vz165dU25urvV6DG9vb+3Zs0epqakqLi5Wu3btNHr0aC1YsMCudQKwFR4ertWrV2vSpEn661//qhkzZigsLEy+vr56//33NWvWLP3mN7+p1bGdsXsOwP7ee+89R5cAAFWqdfiy57fWNdGiRYsqbwndsWNH690YJaldu3Z6//337V4XgJt76KGH1LlzZz322GOKjY21fhttsVgUExOjhQsX2vX16Z4DDcPJkyclyZDfSwCgKrW+2+HPv7W+77779L//+78qKChQYWGhdu7cqVmzZmns2LH1WSsANzR48GDl5OQoOztb69at0+rVq5WVlaV33nlHvr6+dn/9V199VaGhoRoyZIiGDRumO++806ZjX1n3fOjQoQoNDdXs2bM1evRovfnmm3avFUD1lZeXa/HixfL391eHDh3UoUMHNW/eXEuWLFF5ebmjywPQQNXpjyw7+ltrAO4jPDxc4eHhhr8u3XPAPT355JNat26dnnnmGesfVd63b5+eeuopXb16VUuXLnVwhQAaojqFL+k/31pf30pLSxUWFlaj6yUAAADq08svv6y1a9dq5MiR1rHevXurTZs2+u///m/CFwCHqHP4us5R31qjfn3zzHBHlwAAQJ1duHBBoaGhN4yHhobqwoULDqgIAOpwzRcAAICzCgsLU1pa2g3jaWlpCgsLc0BFAFCPnS8AAABn8dxzz2n48OHas2ePIiMjJUlZWVnKz8/Xrl27HFwdgIaKzhcAAHA7UVFR+vLLLzVq1ChdvHhRFy9e1P3336/c3Fz96le/cnR5ABooOl8AAMAthYSEcGMNAE6FzhcAAHA7u3fv1r59+6yPV61apfDwcD344IP6/vvvHVgZgIaM8AUAANzOnDlzVFRUJEn69NNPlZiYqGHDhunEiRNKTEx0cHUAGipOOwQAAG7nxIkT6tGjhyTp9ddf14gRI/THP/5R2dnZGjZsmIOrA9BQEb4AAPWKvxcIZ+Dt7a0rV65Ikvbs2aOJEydKklq0aGHtiAGA0QhfAADA7dx5551KTEzUoEGDdPDgQW3dulWS9OWXX6pt27YOrg5AQ8U1XwAAwO2kpaXJy8tLf/vb37R69Wq1adNGkvTOO+8oNjbWwdUBaKjofAEAALfTvn17vfXWWzeMr1ixwgHVAMBPCF8AAMAtlZWV6Y033tAXX3whSerevbvi4uLk5cWvPwAcg9UHAAC4nc8++0wjRoxQQUGBunXrJkl69tlnFRgYqDfffFM9e/Z0cIUAGiKu+QIAAG5n6tSp6tmzp06ePKns7GxlZ2crPz9fvXv31sMPP+zo8gA0UHS+AACA28nJydGhQ4d06623WsduvfVWLV26VP369XNgZQAaMjpfAADA7fziF79QQUHBDeNnz57Vbbfd5oCKAIDwBQAA3ERRUZF1S0lJ0aOPPqq//e1vOnnypE6ePKm//e1veuyxx/Tss886ulQADRSnHQIAALfQvHlzmUwm62OLxaIHHnjAOmaxWCRJI0aMUFlZmUNqBNCwEb4AAIBbeO+99xxdAgBUifAFAADcQlRUVLXmHT161M6VAEDFuOYLAAC4vUuXLmnNmjXq37+/wsLCHF0OgAaK8AUAANzWBx98oEmTJql169Z6/vnndc899+ijjz5ydFkAGihOOwQAAG7FbDZr48aNWrdunYqKivTAAw+opKREO3bsUI8ePRxdHoAGjM4XAABwGyNGjFC3bt30ySefKDU1VadPn9YLL7zg6LIAQBKdLwAA4EbeeecdPfroo5o+fbq6du3q6HIAwAadLwAA4Db27dunS5cuKSIiQgMGDFBaWprOnz/v6LIAQBLhCwAAuJFf/vKXeumll3TmzBn97ne/05YtWxQSEqLy8nJlZGTo0qVLji4RQANG+AIAAG6nadOmeuihh7Rv3z59+umnmj17tp555hm1atVKI0eOdHR5ABoowhcAAHBr3bp103PPPaeTJ0/qtddec3Q5ABowwhcAAGgQPD09FRcXp507dzq6FAANFOELAAAAAAxA+AIAAAAAAxC+AAAAAMAAhC8AAAAAMADhCwAAAAAMQPgCAAAAAAMQvgAAAADAAIQvAAAAADAA4QsAAAAADED4AgAAAAADEL4AAAAAwACELwAAAAAwAOELAAAAAAzg8uFr6dKlGjhwoJo0aaLmzZtXax+LxaLk5GS1bt1ajRs3VnR0tL766iv7FgoAAACgQXP58FVaWqoxY8Zo+vTp1d7nueee05/+9Celp6frwIEDatq0qWJiYnT16lU7VgoAAACgIXP58LVo0SLNmjVLvXr1qtZ8i8Wi1NRULViwQPfdd5969+6tTZs26fTp09qxY4d9iwXgdOieAwAAo7h8+KqpEydOyGw2Kzo62jrm7++vAQMGKCsrq9L9SkpKVFRUZLMBcH10zwEAgFEaXPgym82SpKCgIJvxoKAg63MVSUlJkb+/v3Vr166dXesEYAy65wAAwChOGb7mzZsnk8lU5Xbs2DFDa0pKSlJhYaF1y8/PN/T1ATgHuucAAKC2nDJ8zZ49W1988UWVW+fOnWt17ODgYElSQUGBzXhBQYH1uYr4+PjIz8/PZgPQ8NA9BxqmVatWqWPHjvL19dWAAQN08ODBKudv27ZNoaGh8vX1Va9evbRr1y7rc9euXdMTTzyhXr16qWnTpgoJCdHEiRN1+vRpe78NAA7mlOErMDBQoaGhVW7e3t61OnanTp0UHByszMxM61hRUZEOHDigyMjI+noLAByI7jmA+rR161YlJiZq4cKFys7OVlhYmGJiYnT27NkK5+/fv1/jx4/XlClTdOTIEcXFxSkuLk5Hjx6VJF25ckXZ2dn6wx/+oOzsbG3fvl25ubkaOXKkkW8LgAN4ObqAusrLy9OFCxeUl5ensrIy5eTkSJJuu+023XLLLZKk0NBQpaSkaNSoUTKZTHrsscf09NNPq2vXrurUqZP+8Ic/KCQkRHFxcY57IwDqzezZszV58uQq59RH97x169bW8YKCAoWHh1e6n4+Pj3x8fGr1mgAca/ny5Zo2bZri4+MlSenp6Xr77be1fv16zZs374b5K1euVGxsrObMmSNJWrJkiTIyMpSWlqb09HT5+/srIyPDZp+0tDT1799feXl5at++vf3fFBqcjvPednQJVt88M9zRJTiMy4ev5ORkvfzyy9bHd9xxhyTpvffe0+DBgyVJubm5KiwstM6ZO3euiouL9fDDD+vixYu68847tXv3bvn6+hpaOwD7CAwMVGBgoF2O/fPu+fWwdb17XpM7JgJwDaWlpTp8+LCSkpKsYx4eHoqOjq70Os+srCwlJibajMXExFR5U57CwkKZTKYq/+RFSUmJSkpKrI+5dhRwPU552mFNbNy4URaL5YbtevCSfro72c+/BTeZTFq8eLHMZrOuXr2qPXv26Be/+IXxxQNwuLy8POXk5Nh0z3NycnT58mXrnNDQUL3xxhuSZNM937lzpz799FNNnDiR7jngps6fP6+ysrIaXedpNptrNP/q1at64oknNH78+CqvKefaUcD1uXznCwDqgu45AEe6du2aHnjgAVksFq1evbrKuUlJSTYdtaKiIgIY4GIIXwAatI0bN2rjxo1VzrFYLDaPr3fPFy9ebMfKADiDgIAAeXp61uguycHBwdWafz14ffvtt3r33Xdveidlrh0FXJ/Ln3YIAABgL97e3oqIiLC5S3J5ebkyMzMrvUtyZGSkzXxJysjIsJl/PXh99dVX2rNnj1q2bGmfNwDAqdD5AgAAqEJiYqImTZqkvn37qn///kpNTVVxcbH17ocTJ05UmzZtlJKSIkmaOXOmoqKitGzZMg0fPlxbtmzRoUOHtGbNGkk/Ba9f//rXys7O1ltvvaWysjLr9WAtWrSo9Z/TAeD8CF8AAABVGDt2rM6dO6fk5GSZzWaFh4dr9+7d1ptq5OXlycPjPycTDRw4UJs3b9aCBQs0f/58de3aVTt27FDPnj0lSadOndLOnTsl6YY/UfHz600BuB/CFwAAwE0kJCQoISGhwuf27t17w9iYMWM0ZsyYCud37NjxhmtJATQMXPMFAAAAAAYgfAEAAACAAQhfAAAAAGAAwhcAAAAAGIDwBQAAAAAGIHwBAAAAgAEIXwAAAABgAMIXAAAAABiAP7IMAAAA1ELHeW87ugSrb54Z7ugSUA10vgAAAADAAIQvAAAAADAA4QsAAAAADED4AgAAAAADEL4AAAAAwACELwAAAAAwAOELAAAAAAxA+AIAAAAAAxC+AAAAAMAAhC8AAAAAMADhCwAAAAAMQPgCAAAAAAMQvgAAAADAAIQvAAAAADAA4QsAAAAADED4AgAAAAADEL4AAAAAwACELwAAAAAwAOELAAAAAAxA+AIAAAAAAxC+AAAAAMAAhC8AAAAAMADhCwAAAAAMQPgCAAAAAAMQvgAAAADAAIQvAAAAADAA4QsAAAAADED4AgAAAAADEL4AAAAAwACELwAAAAAwAOELAAAAAAxA+AIAAAAAA7h8+Fq6dKkGDhyoJk2aqHnz5tXaZ/LkyTKZTDZbbGysfQsFAAAA0KB5ObqAuiotLdWYMWMUGRmpdevWVXu/2NhYbdiwwfrYx8fHHuUBAAAAgCQ36HwtWrRIs2bNUq9evWq0n4+Pj4KDg63brbfeaqcKATgzuucAAMAoLh++amvv3r1q1aqVunXrpunTp+u7776rcn5JSYmKiopsNgCu73r3fPr06TXaLzY2VmfOnLFur732mp0qBAAA7sLlTzusjdjYWN1///3q1KmTvv76a82fP1/33nuvsrKy5OnpWeE+KSkpWrRokcGVArC36z/XGzdurNF+17vnAAAA1eWUna958+bdcErP/92OHTtW6+OPGzdOI0eOVK9evRQXF6e33npLH3/8sfbu3VvpPklJSSosLLRu+fn5tX59AK6P7jkAAKgpp+x8zZ49W5MnT65yTufOnevt9Tp37qyAgAAdP35cQ4YMqXCOj48PN+UAIInuOQAAqB2nDF+BgYEKDAw07PVOnjyp7777Tq1btzbsNQHYz7x58/Tss89WOeeLL75QaGhorY4/btw463/36tVLvXv3VpcuXbR3795Kv8BJSkpSYmKi9XFRUZHatWtXq9cHAACuySnDV03k5eXpwoULysvLU1lZmXJyciRJt912m2655RZJUmhoqFJSUjRq1ChdvnxZixYt0ujRoxUcHKyvv/5ac+fO1W233aaYmBgHvhMA9YXuOQDY6jjvbUeXIEn65pnhji4BLsJd/826fPhKTk7Wyy+/bH18xx13SJLee+89DR48WJKUm5urwsJCSZKnp6c++eQTvfzyy7p48aJCQkI0dOhQLVmyhF+MADdB9xwAADgjlw9fGzduvOldyiwWi/W/GzdurH/84x92rgqAq6B7DgAAjOLy4QsA6oLuOQAAMArhC0CDRvccAAAYxSn/zhcAAAAAuBvCFwAAAAAYgPAFAAAAAAYgfAEAAACAAQhfAAAAAGAAwhcAAAAAGIDwBQAAAAAGIHwBAAAAgAEIXwAAAABgAMIXAAAAABiA8AUAAAAABiB8AQAAAIABCF8AAAAAYADCFwAAwE2sWrVKHTt2lK+vrwYMGKCDBw9WOX/btm0KDQ2Vr6+vevXqpV27dtk8v337dg0dOlQtW7aUyWRSTk6OHasH4CwIXwAAAFXYunWrEhMTtXDhQmVnZyssLEwxMTE6e/ZshfP379+v8ePHa8qUKTpy5Iji4uIUFxeno0ePWucUFxfrzjvv1LPPPmvU2wDgBAhfAAAAVVi+fLmmTZum+Ph49ejRQ+np6WrSpInWr19f4fyVK1cqNjZWc+bMUffu3bVkyRL16dNHaWlp1jm//e1vlZycrOjoaKPeBgAnQPgCAACoRGlpqQ4fPmwTkjw8PBQdHa2srKwK98nKyrohVMXExFQ6H0DD4eXoAgAAAJzV+fPnVVZWpqCgIJvxoKAgHTt2rMJ9zGZzhfPNZnOdaikpKVFJSYn1cVFRUZ2OB8B4dL4AAABcQEpKivz9/a1bu3btHF0SgBoifAEAAFQiICBAnp6eKigosBkvKChQcHBwhfsEBwfXaH51JSUlqbCw0Lrl5+fX6XgAjEf4AgAAqIS3t7ciIiKUmZlpHSsvL1dmZqYiIyMr3CcyMtJmviRlZGRUOr+6fHx85OfnZ7MBcC1c8wUAAFCFxMRETZo0SX379lX//v2Vmpqq4uJixcfHS5ImTpyoNm3aKCUlRZI0c+ZMRUVFadmyZRo+fLi2bNmiQ4cOac2aNdZjXrhwQXl5eTp9+rQkKTc3V9JPXbO6dsgAOC/CFwAAQBXGjh2rc+fOKTk5WWazWeHh4dq9e7f1php5eXny8PjPyUQDBw7U5s2btWDBAs2fP19du3bVjh071LNnT+ucnTt3WsObJI0bN06StHDhQj311FPGvDEAhiN8AQAA3ERCQoISEhIqfG7v3r03jI0ZM0Zjxoyp9HiTJ0/W5MmT66k6AK6Ca74AAAAAwACELwAAAAAwAOELAAAAAAxA+AIAAAAAAxC+AAAAAMAAhC8AAAAAMADhCwAAAAAMQPgCAAAAAAMQvgAAAADAAIQvAAAAADAA4QsAAAAADED4AgAAAAADEL4AAAAAwACELwAAAAAwAOELAAAAAAxA+AIAAAAAAxC+AAAAAMAAhC8AAAAAMADhCwAAAAAMQPgCAAAAAAMQvgAAAADAAC4dvr755htNmTJFnTp1UuPGjdWlSxctXLhQpaWlVe539epVzZgxQy1bttQtt9yi0aNHq6CgwKCqAQAAADRELh2+jh07pvLycr344ov67LPPtGLFCqWnp2v+/PlV7jdr1iy9+eab2rZtm95//32dPn1a999/v0FVAwAAAGiIXDp8xcbGasOGDRo6dKg6d+6skSNH6vHHH9f27dsr3aewsFDr1q3T8uXLdc899ygiIkIbNmzQ/v379dFHHxlYPQBHo3sOAACM5NLhqyKFhYVq0aJFpc8fPnxY165dU3R0tHUsNDRU7du3V1ZWVqX7lZSUqKioyGYD4NrongMAACN5ObqA+nT8+HG98MILev755yudYzab5e3trebNm9uMBwUFyWw2V7pfSkqKFi1aVF+lAnACsbGxio2NtT7u3LmzcnNztXr16krXkevd882bN+uee+6RJG3YsEHdu3fXRx99pF/+8peG1A4AAFyPU3a+5s2bJ5PJVOV27Ngxm31OnTql2NhYjRkzRtOmTav3mpKSklRYWGjd8vPz6/01ADge3XMAAGAvTtn5mj17tiZPnlzlnM6dO1v/+/Tp07r77rs1cOBArVmzpsr9goODVVpaqosXL9p0vwoKChQcHFzpfj4+PvLx8alW/QBcE91zAABgT07Z+QoMDFRoaGiVm7e3t6SfOl6DBw+23jjDw6PqtxQREaFGjRopMzPTOpabm6u8vDxFRkba9X0BMAbdcwAA4IycsvNVXdeDV4cOHfT888/r3Llz1ueud7FOnTqlIUOGaNOmTerfv7/8/f01ZcoUJSYmqkWLFvLz89MjjzyiyMhIrtUA3ATdcwAA4IxcOnxlZGTo+PHjOn78uNq2bWvznMVikSRdu3ZNubm5unLlivW5FStWyMPDQ6NHj1ZJSYliYmL05z//2dDaAdhPYGCgAgMDqzX31KlTuvvuu2vVPR89erQkuucAAKB6nPK0w+qaPHmyLBZLhdt1HTt2lMVi0eDBg61jvr6+WrVqlS5cuKDi4mJt3769ym+sAbin693z9u3bW7vnZrPZ5tqtU6dOKTQ0VAcPHpQkm+75e++9p8OHDys+Pp7uOQAAuCmX7nwBQF3QPQcAAEYifAFosCZPnnzTa8Oud89/7nr3fNWqVXasDgAAuBuXPu0QAAAAAFwF4QsAAAAADED4AgAAAAADEL4AAAAAwACELwAAAAAwAOELAAAAAAxA+AIAAAAAAxC+AAAAAMAAhC8AAAAAMADhCwAAAAAMQPgCAAAAAAMQvgAAAADAAIQvAAAAADAA4QsAAAAADED4AgAAAAADEL4AAAAAwACELwAAAAAwgJejC2gIvnlmuKNLAODCWEMAAHAPdL4AAAAAwACELwAAAAAwAOELAAAAAAxA+AIAAAAAAxC+AAAAAMAAhC8AAAAAMADhCwAAAAAMQPgCAAAAAAMQvgAAAADAAIQvAAAAADAA4QsAAAAADED4AgAAAAADEL4AAAAAwACELwAAAAAwAOELAAAAAAxA+AIAAAAAAxC+AAAAbmLVqlXq2LGjfH19NWDAAB08eLDK+du2bVNoaKh8fX3Vq1cv7dq1y+Z5i8Wi5ORktW7dWo0bN1Z0dLS++uore74FAE6A8AUAAFCFrVu3KjExUQsXLlR2drbCwsIUExOjs2fPVjh///79Gj9+vKZMmaIjR44oLi5OcXFxOnr0qHXOc889pz/96U9KT0/XgQMH1LRpU8XExOjq1atGvS0ADkD4AgAAqMLy5cs1bdo0xcfHq0ePHkpPT1eTJk20fv36CuevXLlSsbGxmjNnjrp3764lS5aoT58+SktLk/RT1ys1NVULFizQfffdp969e2vTpk06ffq0duzYYeA7A2A0L0cX4KosFoskqaioyMGVAA3b9Z/B6z+TroI1BHAeVa0jpaWlOnz4sJKSkqxjHh4eio6OVlZWVoXHy8rKUmJios1YTEyMNVidOHFCZrNZ0dHR1uf9/f01YMAAZWVlady4cRUet6SkRCUlJdbHhYWFNvVXpbzkyk3nGMGVapVuXi+11o4r/Tuo7v+fru7vI4SvWrp06ZIkqV27dg6uBID008+kv7+/o8uoNtYQwPlUtI6cP39eZWVlCgoKshkPCgrSsWPHKjyO2WyucL7ZbLY+f32ssjkVSUlJ0aJFi24Yd6V1xD/V0RXUjCvVS632UdNab/b7COGrlkJCQpSfn69mzZrJZDLZ9bWKiorUrl075efny8/Pz66v1ZDwudqPkZ+txWLRpUuXFBISYtfXqW9GriES/97thc/VPoz+XF1lHUlKSrLpqJWXl+vChQtq2bIlv4u4MD5b+3DWdYTwVUseHh5q27atoa/p5+fHD6Ud8Lnaj1GfrSt1vK5zxBoi8e/dXvhc7cPIz7WydSQgIECenp4qKCiwGS8oKFBwcHCF+wQHB1c5//r/LSgoUOvWrW3mhIeHV1qjj4+PfHx8bMaaN29e6Xx74N+6/fDZ2oczrCM/xw03AAAAKuHt7a2IiAhlZmZax8rLy5WZmanIyMgK94mMjLSZL0kZGRnW+Z06dVJwcLDNnKKiIh04cKDSYwJwD3S+AAAAqpCYmKhJkyapb9++6t+/v1JTU1VcXKz4+HhJ0sSJE9WmTRulpKRIkmbOnKmoqCgtW7ZMw4cP15YtW3To0CGtWbNGkmQymfTYY4/p6aefVteuXdWpUyf94Q9/UEhIiOLi4hz1NgEYgPDlAnx8fLRw4cIbTjVA3fC52g+frfPhfxP74HO1D2f7XMeOHatz584pOTlZZrNZ4eHh2r17t/WGGXl5efLw+M/JRAMHDtTmzZu1YMECzZ8/X127dtWOHTvUs2dP65y5c+equLhYDz/8sC5evKg777xTu3fvlq+vr+Hvrzqc7X8Td8Jnax/O+rmaLK52f2YAAAAAcEFc8wUAAAAABiB8AQAAAIABCF8AAAAAYADCFwAAAAAYgPDlAlatWqWOHTvK19dXAwYM0MGDBx1dkktLSUlRv3791KxZM7Vq1UpxcXHKzc11dFlu55lnnrHeThmOxRpS/1hHjME64jxYR+oXa4gxnHENIXw5ua1btyoxMVELFy5Udna2wsLCFBMTo7Nnzzq6NJf1/vvva8aMGfroo4+UkZGha9euaejQoSouLnZ0aW7j448/1osvvqjevXs7upQGjzXEPlhH7I91xHmwjtQ/1hD7c9o1xAKn1r9/f8uMGTOsj8vKyiwhISGWlJQUB1blXs6ePWuRZHn//fcdXYpbuHTpkqVr166WjIwMS1RUlGXmzJmOLqlBYw0xButI/WIdcS6sI/bHGlK/nHkNofPlxEpLS3X48GFFR0dbxzw8PBQdHa2srCwHVuZeCgsLJUktWrRwcCXuYcaMGRo+fLjNv1s4BmuIcVhH6hfriPNgHTEGa0j9cuY1xMvRBaBy58+fV1lZmYKCgmzGg4KCdOzYMQdV5V7Ky8v12GOPadCgQerZs6ejy3F5W7ZsUXZ2tj7++GNHlwKxhhiFdaR+sY44F9YR+2MNqV/OvoYQvtCgzZgxQ0ePHtW+ffscXYrLy8/P18yZM5WRkSFfX19HlwMYhnWk/rCOoCFiDak/rrCGEL6cWEBAgDw9PVVQUGAzXlBQoODgYAdV5T4SEhL01ltv6YMPPlDbtm0dXY7LO3z4sM6ePas+ffpYx8rKyvTBBx8oLS1NJSUl8vT0dGCFDQ9riP2xjtQv1hHnwzpiX6wh9csV1hCu+XJi3t7eioiIUGZmpnWsvLxcmZmZioyMdGBlrs1isSghIUFvvPGG3n33XXXq1MnRJbmFIUOG6NNPP1VOTo5169u3ryZMmKCcnByHL3YNEWuI/bCO2AfriPNhHbEP1hD7cIU1hM6Xk0tMTNSkSZPUt29f9e/fX6mpqSouLlZ8fLyjS3NZM2bM0ObNm/X3v/9dzZo1k9lsliT5+/urcePGDq7OdTVr1uyGc9WbNm2qli1bcg67A7GG2AfriH2wjjgn1pH6xxpiH66whhC+nNzYsWN17tw5JScny2w2Kzw8XLt3777hwldU3+rVqyVJgwcPthnfsGGDJk+ebHxBgB2xhtgH6wgaEtaR+sca0nCZLBaLxdFFAAAAAIC745ovAAAAADAA4QsAAAAADED4AgAAAAADEL4AAAAAwACELwAAAAAwAOELAAAAAAxA+AIAAAAAAxC+0OBs3LhRzZs3d3QZAFwY6wiAumANabgIX3AJWVlZ8vT01PDhw2u0X8eOHZWammozNnbsWH355Zf1WB0AV8A6AqAuWENQHwhfcAnr1q3TI488og8++ECnT5+u07EaN26sVq1a1VNlAFwF6wiAumANQX0gfMHpXb58WVu3btX06dM1fPhwbdy40eb5N998U/369ZOvr68CAgI0atQoSdLgwYP17bffatasWTKZTDKZTJIqbvWvXr1aXbp0kbe3t7p166a//OUvNs+bTCatXbtWo0aNUpMmTdS1a1ft3LnT+vz333+vCRMmKDAwUI0bN1bXrl21YcOG+v8wANQK6wiAumANQb2xAE5u3bp1lr59+1osFovlzTfftHTp0sVSXl5usVgslrfeesvi6elpSU5Otnz++eeWnJwcyx//+EeLxWKxfPfdd5a2bdtaFi9ebDlz5ozlzJkzFovFYtmwYYPF39/fevzt27dbGjVqZFm1apUlNzfXsmzZMounp6fl3Xfftc6RZGnbtq1l8+bNlq+++sry6KOPWm655RbLd999Z7FYLJYZM2ZYwsPDLR9//LHlxIkTloyMDMvOnTuN+HgAVAPrCIC6YA1BfSF8wekNHDjQkpqaarFYLJZr165ZAgICLO+9957FYrFYIiMjLRMmTKh03w4dOlhWrFhhM/Z/F7yBAwdapk2bZjNnzJgxlmHDhlkfS7IsWLDA+vjy5csWSZZ33nnHYrFYLCNGjLDEx8fX5u0BMADrCIC6YA1BfeG0Qzi13NxcHTx4UOPHj5ckeXl5aezYsVq3bp0kKScnR0OGDKnTa3zxxRcaNGiQzdigQYP0xRdf2Iz17t3b+t9NmzaVn5+fzp49K0maPn26tmzZovDwcM2dO1f79++vU00A6g/rCIC6YA1BfSJ8wamtW7dOP/74o0JCQuTl5SUvLy+tXr1ar7/+ugoLC9W4cWPDamnUqJHNY5PJpPLycknSvffeaz2n+/Tp0xoyZIgef/xxw2oDUDnWEQB1wRqC+kT4gtP68ccftWnTJi1btkw5OTnW7V//+pdCQkL02muvqXfv3srMzKz0GN7e3iorK6vydbp3765//vOfNmP//Oc/1aNHjxrVGxgYqEmTJumVV15Ramqq1qxZU6P9AdQ/1hEAdcEagvrm5egCgMq89dZb+v777zVlyhT5+/vbPDd69GitW7dO//M//6MhQ4aoS5cuGjdunH788Uft2rVLTzzxhKSf/rbGBx98oHHjxsnHx0cBAQE3vM6cOXP0wAMP6I477lB0dLTefPNNbd++XXv27Kl2rcnJyYqIiNDtt9+ukpISvfXWW+revXvdPgAAdcY6AqAuWENQ3+h8wWmtW7dO0dHRNyx20k8L3qFDh9SiRQtt27ZNO3fuVHh4uO655x4dPHjQOm/x4sX65ptv1KVLFwUGBlb4OnFxcVq5cqWef/553X777XrxxRe1YcMGDR48uNq1ent7KykpSb1799Zdd90lT09PbdmypcbvGUD9Yh0BUBesIahvJovFYnF0EQAAAADg7uh8AQAAAIABCF8AAAAAYADCFwAAAAAYgPAFAAAAAAYgfAEAAACAAQhfAAAAAGAAwhcAAAAAGIDwBQAAAAAGIHwBAAAAgAEIXwAAAABgAMIXAAAAABiA8AUAAAAABvh/Dzg/O3q+z/4AAAAASUVORK5CYII=", "text/plain": [ "
" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "nb_actions = 5\n", "bandit = Bandit(nb_actions)\n", "\n", "all_rewards = []\n", "for t in range(1000):\n", " rewards = []\n", " for a in range(nb_actions):\n", " rewards.append(bandit.step(a))\n", " all_rewards.append(rewards)\n", " \n", "mean_reward = np.mean(all_rewards, axis=0)\n", "\n", "plt.figure(figsize=(10, 6))\n", "plt.subplot(131)\n", "plt.bar(range(nb_actions), bandit.Q_star)\n", "plt.xlabel(\"Actions\")\n", "plt.ylabel(\"$Q^*(a)$\")\n", "plt.subplot(132)\n", "plt.bar(range(nb_actions), mean_reward)\n", "plt.xlabel(\"Actions\")\n", "plt.ylabel(\"$Q_t(a)$\")\n", "plt.subplot(133)\n", "plt.bar(range(nb_actions), np.abs(bandit.Q_star - mean_reward))\n", "plt.xlabel(\"Actions\")\n", "plt.ylabel(\"Absolute error\")\n", "plt.show()" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [] } ], "metadata": { "kernelspec": { "display_name": ".venv", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.12.7" } }, "nbformat": 4, "nbformat_minor": 4 }