{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "# Multi Armed Bandit Problem\n", "> An introduction to multi-armed bandits\n", "\n", "- toc: true \n", "- badges: true\n", "- comments: true\n", "- categories: [AI]\n", "- image: images/chart-preview.png" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Problem Description\n", "\n", "Imagine you are at a casino, and you have N slot machines to play, each slot machine gives rewards according to a fixed probability distribution. What strategy should you play with to maximise your total reward ?\n", "\n", "This problem is known as [Multi Armed Bandit](https://en.wikipedia.org/wiki/Multi-armed_bandit) problem." ] }, { "cell_type": "code", "execution_count": 1, "metadata": {}, "outputs": [], "source": [ "# Importing numpy for math, and matplotlib for plots\n", "import matplotlib.pyplot as plt\n", "import numpy as np\n", "%matplotlib inline" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Arms\n", "\n", "An arm when pulled, gives a random number from a normal distribution with fixed mean(mu) and deviation(sigma).\n", "When pulled many times the frequency of the rewards look like this:\n", "\n", "![normal distribution](https://upload.wikimedia.org/wikipedia/commons/thumb/7/74/Normal_Distribution_PDF.svg/350px-Normal_Distribution_PDF.svg.png)\n", "X axis is the magnitude of reward \n", "Y axis is it's frequency.\n", "\n", "The Arm class provides an arm with these properties." ] }, { "cell_type": "code", "execution_count": 2, "metadata": {}, "outputs": [], "source": [ "class Arm:\n", "\n", " def __init__(self, mu=None, sigma=None):\n", " if mu is None:\n", " self.mu = np.absolute(np.random.uniform())\n", " else:\n", " self.mu = mu\n", " \n", " \n", " if sigma is None:\n", " self.sigma=np.absolute(np.random.uniform())\n", " else:\n", " self.sigma = sigma\n", "\n", "\n", " def pull(self):\n", " reward = np.random.normal(self.mu, self.sigma, 1)\n", " return reward\n", "\n", "\n", "def get_arms(k):\n", " # returns a list of arms\n", " arms = []\n", " for i in range(k):\n", " arms.append(Arm())\n", " return arms" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Agents\n", "\n", "An agent here is a player who pulls arms to play.\n", "It has a policy, which is a list of probabilities associated with each arm.\n", "\n", "\n", "The agent class makes designing agents **fast**. The object is initialised with arms and whether it should play all arms once as part of the initialisation.\n", "\n", "Features provided by this class: \n", "Attributes:\n", "* expectations[i]: gives the expected reward on playing arm[i]\n", "* times_played[i]: gives the number of times the agent has played arm[i]\n", "* N = Total number of times agent has played\n", "* reward_history : list of rewards earned by the agent\n", "* choice_history : list of choices made by the agent \n", "\n", "Methods:\n", "* gamble(i): Plays for i iterations while updating it's policy.\n", "* play(i): Pulls arm[i] and updates reward_history, N , times_played\n", "* select_arm(): returns index of an arm by sampling probability distribution given by the policy\n" ] }, { "cell_type": "code", "execution_count": 3, "metadata": {}, "outputs": [], "source": [ "class agent:\n", " def __init__(self, arms, play_once=1):\n", " self.expectations = np.zeros(len(arms))\n", " self.times_played = np.zeros(len(arms))\n", " self.arms = arms\n", "\n", " self.number_of_arms = len(arms)\n", " self.N = 0\n", "\n", " self.reward_history = []\n", " self.choice_history = []\n", "\n", " if play_once == 1:\n", " for i in range(self.number_of_arms):\n", " self.expectations[i] = self.play(i)\n", "\n", " def play(self, index):\n", " reward = self.arms[index].pull()\n", "\n", " self.times_played[index] += 1\n", " self.N += 1\n", "\n", " self.choice_history.append(index)\n", " self.reward_history.append(reward)\n", "\n", " return reward\n", "\n", " def policy(self):\n", " pass\n", "\n", " def update_expectations(self, reward, index):\n", " self.expectations[index] += (reward - self.expectations[index])/self.N\n", "\n", " def select_arm(self):\n", " options = range(self.number_of_arms)\n", " i = np.random.choice(options, p=self.policy(), replace=False)\n", " return i\n", "\n", " def gamble(self, iterations):\n", " for i in range(iterations):\n", " index = self.select_arm()\n", " reward = self.play(index)\n", " self.update_expectations(reward, index)\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Example agents\n", "\n", "To make a new agent we [inherit]() the agent class." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Time to make some agents! \n", "\n", "### First up: epsilon-greedy\n", "\n", "This agent plays the arm with the highest expected reward with _1 - epsilon_ probability, and plays a random arm with _epsilon_ probability\n", "\n", "So \n", " epsilon = 1 => random choices \n", " epsilon = 0 => greedy choices \n", " \n", " \n", "\n" ] }, { "cell_type": "code", "execution_count": 4, "metadata": {}, "outputs": [], "source": [ "class epsilon_greedy(agent):\n", "\n", " def __init__(self, arms, play_once=1, epsilon=0.1):\n", " super().__init__(arms, play_once)\n", " self.epsilon = epsilon\n", " \n", " def __str__(self):\n", " return \"Epsilon-Greedy Agent, epsilon= \"+str(self.epsilon)\n", " \n", " def policy(self):\n", " temp = np.zeros_like(self.expectations)\n", " temp[np.argmax(self.expectations)] = 1-self.epsilon\n", " ans = temp + self.epsilon/self.number_of_arms\n", " return ans" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Beta-Softmax\n", "\n", "This agent plays an arm[i] with probability proportional to: e^(expected_reward(arm[i])/beta) \n", "We normalise the whole thing by the sum over all the arms.\n" ] }, { "cell_type": "code", "execution_count": 5, "metadata": {}, "outputs": [], "source": [ "class softmax(agent):\n", "\n", " def __init__(self, arms, play_once=1, beta=1):\n", " super().__init__(arms, play_once)\n", " self.beta = beta\n", " \n", " def __str__(self):\n", " return \"Softmax agent, beta= \"+ str(self.beta)\n", "\n", " def policy(self):\n", " temp = np.exp(self.expectations/self.beta)\n", " ans = temp / np.sum(temp, axis=0)\n", " return ans" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Upper Confidence Bound (UCB1)\n", "\n", "UCB1 agent plays the arm with the highest metric, where metric of arm i is :\n", "metric[i] = expected_reward[i] + sqrt(2*log(N)/times_played[i])\n", "\n", "__Note__ Best peformance when rewards are between 0 and 1" ] }, { "cell_type": "code", "execution_count": 6, "metadata": {}, "outputs": [], "source": [ "class ucb(agent):\n", "\n", " def __init__(self, arms, play_once=1):\n", " super().__init__(arms, play_once)\n", "\n", " def __str__(self):\n", " return \"UCB1 agent\"\n", " \n", " def policy(self):\n", " temp = self.expectations + np.sqrt(2*np.log(self.N)/self.times_played)\n", " ans = np.zeros_like(temp)\n", " ans[np.argmax(temp)] = 1\n", " return ans" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Metrics\n", "\n", "Metric : A scalar number, makes comparison easier. \n", "To compare the performance of our agents we can use these metrics\n", "\n", "* avg_reward[i] : this gives the average reward till i+1 iteration.\n", "* max_reward : this tells us the maximum expected reward\n", "\n", "* euclid_distance : we can think of as learnt policy and optimal policy as vectors and compute the distance between them , smaller is better \n", "* cosine_simmilarity : compute the cos(q) between the policies. larger is better" ] }, { "cell_type": "code", "execution_count": 7, "metadata": {}, "outputs": [], "source": [ "def maxreward(arms):\n", " #Max rewards\n", " a= [arm.mu for arm in arms]\n", " return max(a)\n", "\n", "def avg_reward(rewards):\n", " ans = []\n", " ans.append(rewards[0])\n", " for i in range(1,len(rewards)):\n", " ans.append(ans[i-1]+rewards[i])\n", " for i in range(len(ans)):\n", " ans[i]/=i+1\n", " return ans\n", "\n", "def cosine_similarity(a,b):\n", " temp = a*b\n", " temp/=(euclid_distance(a)* euclid_distance(b))\n", " return np.sum(temp, axis=0)\n", " \n", "def euclid_distance(a):\n", " return np.sqrt(np.sum(a*a, axis=0))" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Test\n", "\n", "This function takes a list of agents and the number of iterations. Makes each agent play, and prints its metrics." ] }, { "cell_type": "code", "execution_count": 8, "metadata": {}, "outputs": [], "source": [ "def test(agents, iterations):\n", " for agent in agents:\n", " \n", " agent.gamble(iterations)\n", " \n", " temp = [ arm.mu for arm in levers] \n", " optimal_policy = np.zeros_like(agent.expectations)\n", " optimal_policy[temp.index(max(temp))] = 1\n", " \n", " avg_rewards_earned = avg_reward(agent.reward_history)\n", " \n", " print(agent)\n", " print(\"maximum possible reward:\", maxreward(levers))\n", " print(\"average reward:\", avg_rewards_earned[-1])\n", " print(\"cosine similarity\" ,cosine_similarity(agent.policy(), optimal_policy))\n", " euclid_norm = euclid_distance(agent.policy()-optimal_policy)/len(optimal_policy)\n", " print(\"euclidian norm \",euclid_norm)\n", " \n", " \n", " plt.plot(avg_rewards_earned)\n", " plt.ylabel('Average Reward')\n", " plt.xlabel('Iteration')\n", " plt.show()\n", " print(\"\\n\")\n", " \n", " # print(\"optimal policy:\" , optimal)\n", " # print(\"learnt policy:\" ,agent.policy())\n", " \n", " \n", " \n", " # plt.scatter(range(len(agent.choice_history)),y=agent.choice_history)\n", " # plt.title(\"Choices\")\n", " # plt.xlabel(\"time\")\n", " # plt.ylabel(\"arm\")\n", " # plt.show()\n", " # print(\"\\n\")\n", " \n", " " ] }, { "cell_type": "code", "execution_count": 9, "metadata": {}, "outputs": [], "source": [ "levers = get_arms(10)\n", "\n", "agents = [\n", " epsilon_greedy(levers, epsilon=1),\n", " epsilon_greedy(levers, epsilon=0),\n", " softmax(levers, beta=0.1),\n", " ucb(levers)\n", "\n", "]\n" ] }, { "cell_type": "code", "execution_count": 10, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "Text(0.5, 1.0, 'distribution of expected value of arms')" ] }, "execution_count": 10, "metadata": {}, "output_type": "execute_result" }, { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXQAAAEICAYAAABPgw/pAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAgAElEQVR4nO3dd3xU15nw8d+j3gVCBUn0IgECFNsY3C3ibiM7iVNspzmbTfZ9symbYidO8nrTnLbZZLNJNmU3ZVOM45Zd3OLYMTIOLoAxRQIkusoIFUAzkkD9vH/cO2IYq4ykGd0pz/fz4YNm7p17H12NHp0557nniDEGpZRSkS/O6QCUUkoFhyZ0pZSKEprQlVIqSmhCV0qpKKEJXSmlooQmdKWUihKa0ENIRH4jIt+wv75SRGqDeOxnROSD9td3i8jfgnjs94rIX4J1vAmc93IROSgiXSLytuk+/3QTka+IyO9DcNzh9910EpG3i0iD/fO7YLrPrzShTxtjzEvGmNLx9gv0l9wYc5Mx5r+nGpeILBARIyIJPsf+gzHm+qkeexK+BvzYGJNhjPkfB84/ISJyTESudTqOMPI94OP2z+8Np4OJRZrQI4xYovXnNh+ocToINWlB+/n5NjBU4KI1MThCRC4QkZ0i0ikifwRSfLZViEijz+PPi0iTvW+tiFwjIjcCXwTeY39s3W3vWyUiD4jIVuAMsMh+7u/PP738WETcInJARK7x2XBeS9LvU8AW+/8O+5yX+nfhiMhlIrLdPvZ2EbnMZ1uViHxdRLba38tfRCR3jGv0ERE5JCKnRGSTiBTZzx8GFgFP2HEkj/DaIhF5TETaROSoiHzSZ9vTIvKvPo8fEpFf2V/fbcc32vXJFpFfikiz/TP5hojE+8W83/7+9onIhSLyO2CeT7z32vteIiIvi0iHiOwWkQqf4ywUkRft4zwHjHWd9ovIBp/HCfb3faH9+BEROWF/P1tEpGyU47ypO87+RLbE/jpZRL4nIvUi0iIiPxOR1FGOFSciXxaR4yLSKiK/ta9dsoh0AfHAbvtnOdLrfyhWl4xHRF4XkSt9tn1FRB4Vkd+LiAe4237uEfu5ThHZKyIlInKfff4GEbne5xh3i8gRe9+jIvLe0a5v1DLG6L8g/AOSgOPAp4FE4J1AP/ANe3sF0Gh/XQo0AEX24wXAYvvrrwC/9zt2FVAPlAEJ9vGrgL+3t98NDPic+z2AG8ixtx8DrvU53vA57HMbIMFn+93A3+yvc4DTwPvtc99pP57lE9thoARItR9/e5Rr9FagHbgQSAZ+BGzx2X5enH6vjQNeB+63r/Ui4Ahwg719NtBqn+O99rbMAK/Pn4CfA+lAPrAN+Ad727uAJuBiQIAlwPxRrmsxcBK42Y73Ovtxnr39FeD79vd+FdDp/7P2Odb9wB98Ht8C7Pd5/HdApn2sfwN2+Wz7Defed8M/S5/tBlhif/0DYJP9c84EngC+NUpMfwccsq99BvA48LuRjjvK698HzLLfR58FTgApPu/JfuBt9rVLtZ/rAW6wX/Nb4CjwJfvn+BHgqP36dMADlNqPC4Eyp/PCdP9zPIBo+Wf/groA8XnuZUZO6Euwks+1QKLfcb7i/0uOlSS/NsJzvgnd/9zbgPfbX/snnuFzMH5Cfz+wze/crwB3+8TxZZ9tHwP+PMo1+iXwXZ/HGfYv8YKR4vR77Tqg3u+5+4Bf+zy+HesPZTtwhd/3M+L1AQqAXiDVZ9udwGb762eBT40Sk/91/Tw+Cc7n9R/Eas0PAOk+2x70/1n7bFuClfDT7Md/AO4fZd8Z9s8w2378GwJI6Fh/oLqxGxP2tkuxk+QI5/kr8DGfx6X2zy/B97gT+J05DZT7vCe3+G3/CvCcz+NKoAuItx9n2uecgZXQO+z3QGqgMUTbP+1yCZ4ioMnY7zTb8ZF2NMYcAv4J6w3bancPFI1z/IZxto907vGOGYgi3vx9HMdqjXqd8Pn6DFaiHvdYxpgurBZs8Sj7+5oPFNldGR0i0oHVPVXgs88TWB/7a40x/lU/o12f+VitvWaf4/4cq6UOMBfrE0gg5gPv8ovxCqzWYhFw2hjT7RfDiOz3yH6gUkTSgFux/gAgIvEi8m0ROWx3TxyzXzZqF84o8oA04HWfeP9sPz8S//fCcayWc8HIu59PRD5ndyW57XNl+8U80nu8xefrs0C7MWbQ5zFAhn1d3wP8H6yf5VMisiyQuKKJJvTgaQaKRUR8nps32s7GmAeNMVdgJQEDfMe7abSXjHP+kc7tsr/uxvrF9Zo9geO67Bh9zcPqhpio844lIulYH8EDOVYDVstxhs+/TGPMzT77PICVBAtF5E6/1492fRqwWui5PsfNMsZ4+6QbgMWjxOR/7RqwWui+MaYbY76N9f6YaX/PvjGMZSPWp4XbgH12kge4y37uWqykuMB+XvwPgN/PXkR8f/btWEmxzCfebGPMaH+Q/d8L3k8dLSPvfo7dX34v8G5gpjFmBla3l2/MU5r61RjzrDHmOqw/oAeA/5zK8SKRJvTgeQXrzf1JEUkUkXcAa0faUURKReStYg389WD9Ug3Zm1uABTLxSpZ8n3O/C1gOPG1v2wXcYW9bg9W/79Vmn3vRKMd9GigRkbvsgbn3ACuAJycYH1gJ6kMi8hb7e/8m8Jox5lgAr90GdIo1mJxqt1JXisjFACJyFfAh4ANYXRw/EhHflv+I18cY0wz8BfhXEcmyB/4Wi8jV9uv+C/iciFwkliUi4k1qLZx/3X6P1aK+wY4vRazB8DnGmOPADuCrIpIkIldgdSGM5SHgeuD/YrfObZlYf4ROYiXrb45xjN1AmX3NU7A+FQJgjBnCSno/EJF8+zoWi8gNoxxrI/BpsQZ3M+zz/tEYMzDO9+GNeQDr/ZYgIvcDWQG8LiAiUiAit9l/MHuxumaGxnlZ1NGEHiTGmD7gHVh9lqewPv49PsruycC3sVpIJ7CSzX32tkfs/0+KyM4JhPAasNQ+5gPAO40xJ+1t/w+rlXka+Co+ycEYc8bef6v9sfsSv+/rJLABaxDrJFYra4Mxpn0CsXmP9bwdy2NYLdbFwB0BvnbQjuMtWANj7VjJNltEsrAGzD5ujGkyxryE1V//a59W+VjX5wNYA637sK7Ro1itPIwxj9j7P4jVp/0/WAOIAN8Cvmxft88ZYxqwWs5fxEpcDcA9nPs9uwtrLOAU8M92zGN9z81YDYXLgD/6bPotVndHkx3zq2Mcow6rvv954CDg3xX1eayBzlft7pvnsfrGR/Ir4HdYlVFHsRojnxjre/DxLFZ3Tp0dew/jdyNORBzwGaxPEaeAq7H+EMYUOb9bUanoIyJ3Yw0gX+F0LEqFkrbQlVIqSmhCV0qpKKFdLkopFSW0ha6UUlHCsQlwcnNzzYIFC5w6vVJKRaTXX3+93Rgz4s1fjiX0BQsWsGPHDqdOr5RSEUlERr3DWLtclFIqSmhCV0qpKDFuQheRX9lzD1ePsl1E5N/FmuN6j9jzNSullJpegbTQfwPcOMb2m7BuqV4KfBT46dTDUkopNVHjJnRjzBasuRFGcxvwW2N5FZghIoXBClAppVRggtGHXsz5k+w0Msr81iLyURHZISI72tragnBqpZRSXtM6KGqM+YUxZo0xZk1e3mhz6CullJqMYCT0JqxVXbzmMLnFD5SKCq6Oszz2eiM6rYaabsG4sWgT8HEReQhrrme3PY+zUjHnmb3NfP6xPXh6Bpibk8bahTnjv0ipIBk3oYvIRqwFjnNFpBFrYv5EAGPMz7BWtLkZa5L8M1irxigVU872DfL1p/bx4Gv1rCrOZn+zh6raVk3oalqNm9CNMf5rM/pvN8A/Bi0ipSLM/mYPn9z4Bgdbu/iHqxfx2etKed8vX6Oqto17b4y5dYqVg/ROUaUmyRjDf798jNt+spWOs/387sNrue+m5SQlxFFRmse+Zg8tnh6nw1QxRBO6UpNwqruPj/x2B/+8qYbLF8/imU9dyZVLz1VuVZTkA/BirZbnqunj2GyLSkWqlw+18+mHd3G6u5/7N6zgQ5cv4Nxa1JblhZkUZCVTVdfKuy+eO8qRlAouTehKBah/cIgfPFfHT188zMLcdH5198WUFWWPuK+IUFGSz9PVzQwMDpEQrx+GVejpu0ypANSfPMO7fvYK/1F1mHdfNJcnP3HFqMncq6I0j86eAXbWd0xTlCrWaQtdqXH8764mvvSnakTgx3ddwIbVRQG97vKluSTEiZYvqmmjLXSlRtHVO8BnH97Npx7aRensTJ751JUBJ3OArJRELpw/k806MKqmiSZ0pUawt9HNhn9/iT+90cgnr1nKHz96CXNmpk34OOtL89mv5YtqmmhCV8rH0JDhP7cc4R0/3UrvwBAPfuQSPnNdyaQHNStKrVJGLV9U00H70JWytXb28NmHd/PSwXZuKCvgO7evZkZa0pSOuWx2JrOzUrR8UU0LTehKAVW1rXzukd109gzwwNtXctfaeW+qLZ8MEeHqkjye3ttM/+AQiVq+qEJI310qpvUODPKNJ/dx96+3Mys9mSc+cQXvXTc/KMnca/2yPDp7B9h5/HTQjqnUSLSFrmLWkbYuPrHxDWpcHj5w6Xy+ePNyUhLjg36ey5fY5Yt1baxbNCvox1fKS1voKuYYY3h4RwMbfvQ3mjrO8ov3X8TXblsZkmQOkJmSyEXzZ1KlA6MqxDShq5ji6ennkw/t4t5H97B6TjbPfOpKri+bHfLzVtjliyfcWr6oQkcTuooZrx8/zc0/fImn9zZzzw2l/OHvL6EwO3Vazj1cvljXOi3nU7FJE7qKeoNDhh+/cJB3//wVAB7+h0v5x/VLiI8L3sDneIbLF7XbRYWQDoqqqHbC3cM//fENXj1yig2rC/nmO1aRlZI47XGICBWleTy1R8sXVejou0pFrb/UnODGH25hT6Ob775zNT+68wJHkrlXRalVvvi6li+qENEWuoo6Pf2DPPDUfn736nHKirL40Z0XsCgvw+mwzpUv1rZxiZYvqhDQFrqKKnUtndz246387tXj/P0VC3n8Y5eFRTIHq3xxzYKZVNXqwKgKDU3oKmo0u89y24+3crK7l9986GK+vGEFyQmhqS2frIrSfA6c6NTyRRUSmtBV1Nh29BRn+wf55QcvpqI03+lwRqTliyqUNKGrqFHj8pAUH8eKoiynQxlVaYFVvrj5gJYvquDThK6iRnWTm2WFmWFdEigirF+Wx9ZD7fQPDjkdjooy4fvOV2oCjDFUN7nHXbg5HFxdkq/liyokNKGrqNB4+iyengFWFodvd4vX5UtmDZcvKhVMmtBVVKhucgOwMgJa6Fq+qEJFE7qKCtUuN/FxQunsTKdDCYi3fLHZfdbpUFQU0YSuokJ1k4el+Rkhm9M82NbbZZW6eLQKJk3oKuJ5B0RXFod/d4tXSUEGhdk6+6IKLk3oKuK1eHo52d3HyjCuP/fnnX3xb4fa6RvQ8kUVHAEldBG5UURqReSQiHxhhO3zRGSziLwhIntE5Obgh6rUyIYHRCOohQ5W+WKXli+qIBo3oYtIPPAT4CZgBXCniKzw2+3LwMPGmAuAO4D/CHagSo2m2uVGBJYXRk4LHazyxcR4oUqnAVBBEkgLfS1wyBhzxBjTBzwE3Oa3jwG8v03ZgCt4ISo1tuomD4ty00lPjqzZoDNTElkzP0cHRlXQBJLQi4EGn8eN9nO+vgK8T0QagaeBT4x0IBH5qIjsEJEdbW36JlbBUeOKrAFRXxWleVq+qIImWIOidwK/McbMAW4Gficibzq2MeYXxpg1xpg1eXl5QTq1imXtXb00u3si4oaikXhnhdRqFxUMgST0JmCuz+M59nO+Pgw8DGCMeQVIAXKDEaBSY6lxeQAoi4Bb/kdSUpBBUXaK3jWqgiKQhL4dWCoiC0UkCWvQc5PfPvXANQAishwroWuTQ4Wct8IlEiblGomIcHVpPlsPndTyRTVl4yZ0Y8wA8HHgWWA/VjVLjYh8TURutXf7LPAREdkNbATuNsaYUAWtlFeNy828nDSyU51b/HmqKkrztHxRBUVAZQHGmKexBjt9n7vf5+t9wOXBDU2p8VU3eSJihsWxXL4k1ypfrG3l0sW6eLSaPL1TVEUs95l+6k+didjuFq+M5ATWzM/RgVE1ZZrQVcSqaY7MO0RHsn5ZHrUtnbg6tHxRTZ4mdBWxaprsCpcImsNlNN7yxRfrtJWuJk8TuopY1S43hdkp5GYkOx3KlC3Nt8oXNx/Q8kU1eZrQVcSKlDVEA3GufFFnX1STpwldRaTu3gGOtHdHfIWLr/WleXT3DbLj+CmnQ1ERShO6ikj7mz0YExlriAbqMrt8USfrUpOlCV1FpEidA30sGckJXLxAyxfV5GlCVxGp2uUhNyOJgqzIHxD1VVGq5Ytq8jShq4jkHRAVEadDCar1OvuimgJN6Cri9PQPcrC1K6oGRL2W5GdQPCNVZ19Uk6IJXUWc2hOdDA6ZqBoQ9bLKF/O0fFFNiiZ0FXGqXdE3IOqrosQuXzym5YtqYjShq4hT3eQhKyWBOTNTnQ4lJLzli1U6DYCaIE3oKuJ41xCNtgFRr4zkBNYuzNF+dDVhmtBVROkfHOJAc2fUdrd4VZTkU9fSRZOWL6oJ0ISuIsrBli76BoeiYobFsVSUWouoaytdTYQmdBVRon1A1Otc+aL2o6vAaUJXEaWmyU16UjwLZ6U7HUpIiQgVpXm8rOWLagI0oauIUu3ysKIoi7i46BwQ9VVRmq/li0G0pa6NP73RSF1LJwOD0flHMqBFopUKB4NDhn0uD++5eK7ToUyLyxbPIik+js21rVy2JNfpcCLa2b5BPvLbHfTan3ZSEuMonZ1FWZH3XzbLZmeSkhjvcKRTowldRYyj7V2c7R+M+v5zr/TkBC5eOJOq2ja+dIvT0US2V4600zswxDffvoqUxDhqXB5qXG6e2O3iwdfqAYiPExbnpVNWlE1ZURYrCq1En52W6HD0gdOEriJGtb2GaDTO4TKa9aX5fOOp/TR1nKV4RnTeSDUdqmrbSE2M5/aLiklOiOcdF1rPG2NoPH2WGpfbTvIeXj7czp/eaBp+bfGM1OFWfFlRFmXFWczOSgnL+yA0oauIUd3kJjkhjiV5GU6HMm0qSvP4xlP7qapt5b3r5jsdTkQyxrC5tpXLl8wiOeH8LhURYW5OGnNz0rhxZeHw8+1dvcOt+H0uD/tcHp7b34Ix1vac9CSrFW8n+hWFWSzMTSfe4bEdTegqYlS73CwrzCIhPnbG8hfnnStf1IQ+OUfau2k4dZaPXrU44NfkZiRzdUkeV5fkDT/X1TvAgWbPcKKvcXn41d+O0j9oZfm0pHiWzc4815IvyqZkdsab/oiEkiZ0FRGGhgw1TR5ufUuR06FMK2/54p/eaKJ3YHBak0O08NbyV/gk58nISE5gzYIc1izIGX6ub2CIg62d1Nit+BqXm8d3NvK7VwcBSIgTluRn+CT5LJYXZZGVEpp+eU3oKiI0nD5DZ+9AzAyI+qoozecPr9Wz49hpLtdqlwmrqm1lSX4Gc3PSgn7spIQ4O1mfe18ODRmOnzpzXr/8i3WtPLazcXifr91WxgcuXRD0eDShq4gwPCAahXOgj8dbvlhV26oJfYLO9A3w2pFTfODS6euuiosTFuamszA3nQ2rz32ibPX0DHfXXDhvZkjOrQldRYRql5uEOKFkduwMiHqlD8++qOWLE/XK4ZP0DQ5RYS/t56T8rBTys1JYvyx0scTO6JKKaNVNbkoKMmO2D7miNI+DrV00nj7jdCgRZXNtK2lJ8Vy8MDQt4nCjCV2FPWMMNS5PTNWf+zs3+6JO1hUoYwxVtW1ctjg3ZhoCASV0EblRRGpF5JCIfGGUfd4tIvtEpEZEHgxumCqWNbt7ONXdF5MDol6L8zKYM1NnX5yIw23dNJ4+O/zHMBaM24cuIvHAT4DrgEZgu4hsMsbs89lnKXAfcLkx5rSION9hpaJGdZM1ZW5ZDA6IennLFx/fqeWLgfLOJR9LCT2QFvpa4JAx5ogxpg94CLjNb5+PAD8xxpwGMMborPwqaKpdHuIElhdmOh2KoypK8jnTN8j2o6edDiUiVNW2sTQ/gzkzg1+uGK4CSejFQIPP40b7OV8lQImIbBWRV0XkxpEOJCIfFZEdIrKjrU0/OqrA1DS5WZyXQVpSbBdlXbbkXPmiGlt37wDbjp6KqdY5BG9QNAFYClQAdwL/KSIz/HcyxvzCGLPGGLMmLy+2LrSavGp7UehYl5aUwLpFOVTVaWNoPC/b5Yrrw6BccToFktCbAN8JqOfYz/lqBDYZY/qNMUeBOqwEr9SUtHb20OLpjfo1RAN1dUkeh7R8cVxVta2kJ8Wfd5t+LAgkoW8HlorIQhFJAu4ANvnt8z9YrXNEJBerC+ZIEONUMarG5Z0yV1vowPANMlrtMrrhcsUluSQlxFZl9rjfrTFmAPg48CywH3jYGFMjIl8TkVvt3Z4FTorIPmAzcI8x5mSoglaxo8aucFmhLXQAFuel2+WL2o8+mkOtXTR1nI257hYI8NZ/Y8zTwNN+z93v87UBPmP/Uypoqps8LJiVFrLZ6SKNli+Ob3h2xRgbEAW9U1SFuWqXmzLtbjnP+lItXxxLVV0rJQUZFMXgCk+a0FXY6jjTR+PpszE5w+JYLl2s5Yuj6bLLFWOxuwU0oaswdm5AVPvPfXnLFzdrQn+Tlw+10z9ouDoGu1tAE7oKY3rL/+iuLsnjcFs3Dae0fNFXVV2bVa44P7bKFb00oauwVe3yUDwjlZz0JKdDCTveObX1JqNzjDFUHWjliqWxV67oFZvftYoINU1uvaFoFIty05mbk8qL2u0y7GBrFy53T1gsZuEUTegqLHX29HOkvVtvKBqFiFBRks/WQyfp6R90OpywEIuzK/rThK7C0v7mTkAHRMdSUZrH2f5Bth875XQoYWHzgTZKCzIpzI69ckUvTegqLHkHRLVkcXSXLp5FUkKcTgOAVa644/gpKpbFbuscNKGrMFXtcpOXmUx+VorToYSttKQE1i3M0Xp0YKtdrlhRErv956AJXYWpmiYPK3VAdFwVpflavojVf56RnMCaBbGxGPRoNKGrsHO2b5CDrZ06IBqAc4tHx24r3Tu74hVLckmMj+2UFtvfvQpLB054GDJ6Q1EgvOWLsdyPXtfSRbO7J6arW7w0oauwU623/AdMRFhfms/Lh2O3fNE7BUKs3u7vSxN6hBscMgwNGafDCKqaJjcz0hIpjsHZ8iYj1ssXq2pbWTY7tssVvTShRzBjDO/++Svc+9gep0MJqmqXm5VF2YiI06FEhEsXWbe6bz4Qe90unT397Dh2OqbvDvWlCT2C7W508/rx0zyx20VX74DT4QRF38AQtSc6KdPuloClJsVb5Yt1sTcwuvVQOwNDRvvPbZrQI9jG1+qJE+gdGOL5fS1OhxMUdS2d9A8avaFogtaX5nMkBssXq2rbyExO4KL5sV2u6KUJPUJ19vSzabeLd140h6LsFJ7Y7XI6pKCocdl3iGrJ4oTEYvnicLniUi1X9NKrEKH+d5eLs/2DvHfdfDaUF7HlYBsdZ/qcDmvKqps8ZCQnMD8nzelQIsrC3HTm5aSxOYbKFw+c6OSER8sVfWlCj0DGGB58rZ4VhVmsnpNN5eoi+gcNz9accDq0Kat2uVlRlEVcnA6IToR38eiXD7fHTPniucWgdUDUSxN6BNrb5GZfs4c7181DRFhZnMWCWWk8sbvZ6dCmZGBwiP3NHu0/n6T1pfn09A+x7WhslC9W1bayvDCLAp3vZ5gm9Aj04Gv1pCbGc9tbigCrdVZZXsTLh9tp7exxOLrJO9LeTU//kN5QNEmXLIqd2Rc9Pf3sOH5au1v8aEKPMN7B0FvLi8hKSRx+vrK8iCEDz+yN3G6X4SlzdUB0UlKT4rlk0ayYKF/cerCdwSHDeu1uOY8m9AizabeLM32D3Llu3nnPlxRkUlqQGdHVLjUuDymJcSzKTXc6lIhVUZLHkbZu6k9Gd/liVW0bmSkJXDhvhtOhhBVN6BFm47Z6lhdmUT7nza3YyvJCdhw/TVPHWQcim7rqJjfLC7NI0BK0SRsuX4ziVroxhqq6Vq5cmqvvFT96NSLI3kY31U0e7lo7d8Tb4ivLrT71p/ZEXit9aMiwz6UDolO1MDed+bPSoroffX9zJy2eXq1uGYEm9Ajy4LZ6UhLjuO2C4hG3z5+VTvmc7Iisdqk/dYbO3gEdEJ0ia/Ho6C5f9H76qCjRAVF/mtAjRFfvAJt2NVG5+vzBUH+V5UXsbXJztL17GqObumr7DlGdA33qKuzyxdeitHyx6kAbKwqzdHnCEWhCjxBP7HbRPcJgqL9bVhcC8GSEDY5WN3lIjBdKCjKdDiXinStfjL5+dPfZfl6vP836GF8MejSa0CPExm31LJudyQVzxx7VL8xOZe2CHJ6IsH70Gpeb0tmZJCXoW3KqUpPiuXTRLF6Mwn70rYesckXtPx+Z/vZEgOomN3sa3dy5dl5Ac4RXlhdS19JF7YnOaYhu6owxVDe5dUA0iCpK8zjS3s3xk5HV9TaezQdayUpJGLdhE6sCSugicqOI1IrIIRH5whj73S4iRkTWBC9EtXFbPckJcbxtlMFQfzetKiROYNPuphBHFhwudw+nz/RTpjcUBY23BRtN1S7GGF6sa+PKkjwtVxzFuFdFROKBnwA3ASuAO0VkxQj7ZQKfAl4LdpCxrLt3gP/d5WLD6iKyU0cfDPWVm5HM5UtyeWJ3M8aE//J0w3eIFmmFS7CcK1+Mnn70fc0eWjt7tbplDIH8mVsLHDLGHDHG9AEPAbeNsN/Xge8AkTuZSBh6co+1GtFd6+ZO6HWVq4uoP3WGPY3uEEUWPDVNbuLjhOWFmtCD6a3L8tl6+CQn3NHxK+n9tKGLQY8ukIReDDT4PG60nxsmIhcCc40xT411IBH5qIjsEJEdbW3R81EwlB7c1kBJQQYXzpvYiiw3rJxNYrxExFQA1S4PS/IySEmMdzqUqPJ3ly/EGMMP/3rQ6VCCoqq2lZXFWeRnarniaKbcESUiccD3gc+Ot68x5hfGmDXGmDV5efpXdjw1Lje7GzoCHgz1lZ2ayNUl+S0u6LEAABgASURBVDy5p5mhofDudqlucusaoiEwNyeNu9bO4+EdDRxp63I6nClxn+lnZ30HFSVa3TKWQBJ6E+D7eX+O/ZxXJrASqBKRY8AlwCYdGJ0672Do2wMcDPVXWV7ICU8PO46fDnJkwdPq6aG1s1crXELk429dSnJCHN9/rs7pUKbkpUNtdrmiNgTHEkhC3w4sFZGFIpIE3AFs8m40xriNMbnGmAXGmAXAq8CtxpgdIYk4RpzpG+B/3nBxy6pCZqQlTeoY1y4vICUxLqy7XWpcHkCnzA2VvMxkPnzFQp7c0zw8+ByJqmrbyE5N5C1arjimcRO6MWYA+DjwLLAfeNgYUyMiXxORW0MdYKx6cnczXb0D494ZOpb05ASuWV7A03ubGRgcCmJ0weNNMiu0wiVkPnLVImakJfIvz9Y6HcqkDA3Z5Yo6u+K4Aro6xpinjTElxpjFxpgH7OfuN8ZsGmHfCm2dT92D2+pZmp/BmvkTGwz1V7m6iJPdfbxy5GSQIguuapebRbnpZCQnOB1K1MpKSeRjFYt5sa6NV8P0fTCWfc0e2jp1dsVA6J+7MLTP5WHXJAdD/VWU5pGRnMCmXeHZ7VLd5NEbiqbBBy5dwOysFL775wMRcW+CL28t/dVafz4uTehh6KHt9SQlxPGOCyc3GOorJTGe68sK+HPNCXoHwms61dPdfTR1nNUbiqZBSmI8n7p2KTvrO3h+f2TdbFRV28aq4mzyMpOdDiXsaUIPM2f7BvnTzqYpDYb6qywvorNngC117UE5XrDogOj0etdFc1iYm873nq1lMMxLWb06zvSxs14Xgw6UJvQw8+QeF529A9y5dvKDof6uWJLLzLTEsKt2OTcHurbQp0NCfByfvb6E2pZO/ndXZMzz89LBdoYM2n8eIE3oYWbjtnoW56Vz8YKpDYb6SoyP46ZVhTy3r4UzfQNBO+5UVTe5mTMzNWifRNT4bl5ZSFlRFt9/ro6+gfCsfPJVVdvGjDQtVwyUJvQwcuCEh531wRkM9Ve5uoiz/YO8cCB8+k9rdA3RaRcXJ9x74zIaT59l47Z6p8MZk1Wu2MqVS/OIjwvu70O00oQeRh7a1kBSfBy3Xzgn6MdeuzCH/MzksOl26ezp52h7t64h6oCrluaybmEOP3rhEN294fOJzV+Ny0N7Vx/rtf88YBGX0Lt7B3hmb+Qtgjyes32DPL6zkZtWzWZmevC7IOLjhFtWF7K5tg1PT3/Qjz9R++wBUS1ZnH4iViu9vauXX2896nQ4o/KWK16l5YoBi7iE/tOqw3zswZ3sOBZdC+A+vbcZT09wB0P9VZYX0TcwxHM1LSE7R6CqvRUu2uXiiIvmz+Ta5QX8/MUjnO7uczqcEW2ubWX1nGxyM7RcMVARl9D/b8Viimekcs+je+jpD6+66qnYuK2eRbnprFuYE7JzXDB3BsUzUsNivdGaJjcFWclaW+yge24opatvgJ+9eNjpUN6k40wfuxo6tLplgiIuoacnJ/Dd21dztL2bf/1LZM5N4a+upZMdx0+HZDDUl4hQWV7E3w62c8rhVlm1S9cQdVrp7EzefkExv3n5WNgtgrFluFxRu1smIuISOsBlS3K5a908fvm3o7wexlPDBmrjtnprMPSi4A+G+qssL2RgyPBMtXPjEGf7BjnU2qX952Hg09eWMBSGi2BUHWhlZloi5XO0XHEiIjKhA9x30zIKs1O599HdEd310tM/yGOvN3LDytnkhGAw1N+KwiwW56U7Wu2y/4SHIaNriIaDuTlpvHfd/LBaBMM7u+JVJVquOFERm9AzUxL59u2rONzWzb89H16ti4k4Nxg6sTVDJ8vb7fLa0VO0eJz5mF3jXRRaW+hh4R/XLwmrRTCqXW5Odvdpd8skRGxCB7hyaR53XDyXX2w5zK6GDqfDmZSN2+pZMCuNSxfNmrZzblhdhDHw1B5nul2qmzzkpCdRmK1rQ4aDcFsEo6q2DRG4aqkm9ImK6IQO8MVbllOQlcI9j+wOu9kEx3OwpZPtx0I/GOpvSX4GKwqzHKt2qXa5KSvKmtbvWY0tnBbBsMoVZzBLyxUnLOITelZKIt96xyoOtnbx72E2sDOejdsaSIyXaRkM9VdZXsQb9R00nDozreftHRikrqVTu1vCTLgsgnG62y5X1JuJJiXiEzpYM7G966I5/OzFI+xpjIyul57+QR7b2cgNZbMduXFiw+pCAJ6c5m6Xgy1d9A8aLVkMQ+GwCMaWg20YLVectKhI6ABf3rCC3Iwk7nlkT0R0vfy5+gTus/3cFcI7Q8cyNyeNC+bNmPZql+rhAVGtcAk34bAIRlVtGznpSazWcsVJiZqEnp1qdb3UtnTykxcOOR3OuB60B0MvmcbBUH+Vq4vY1+zhUOv0latVu9xkpiQwLydt2s6pAufkIhhDQ4YtdW1ctTRXyxUnKWoSOsBblxXwjguL+UnV4bAYrR/NodYuth09xR1r5xHn4Bv3ltWFiDCtrfTqJo8OiIYxJxfB2NvkLVfU2/0nK6oSOsA/byhjVnoSn3tkd9hO4P/QtnoS44V3OjAY6qsgK4VLFs7iiT2uaekzHRgcYn+zzoEe7pxaBGNzbatVrqgDopMWdQk9Oy2RB96+igMnOvmPqvDrevEOhl6/wpnBUH+V5UUcaetmX7Mn5Oc63NZN78CQVriEOacWwaiqbaN8zoxpuWM6WkVdQge4bkUBb3tLET9+4dDwvNvh4tmaE5w+0x/SaXIn4saVs0mIE57YHfpqFx0QjRxXLc3lkkXTtwjGqe4+djd2aHXLFEVlQgf458oyZqRZXS/9g+HT9bJxWz3zctK4bLFzg6G+ctKTuGJpLk/sDn23S7XLTWpiPAtzM0J6HjV1070IxpY6q1xxvfafT0nUJvSZ6Ul8420r2dfs4WdV4THf85G2Ll49coo71s51dDDUX+XqIpo6zvJGiKdPqGnysKIoSysYIsSF82Zy3YrpWQSjqraVWelJrNLuuCmJ2oQOVndCZXkR//7CQQ6ccL7r5aHtDSTEOT8Y6u+6sgKSEuJCWu0yNGSocbl1hsUI87nrQ78IxuCQYcvBdq4qyQurhk4kiuqEDvDVW8vISknknkf2MOBg10vvwCCPvt7IdSsKyM8Mr0mpslISWV+ax1N7mkNWe3zsZDfdfYM6B3qEmY5FMPY0dnBKZ1cMiqhP6DnpSXz9bSvZ2+Tm51uOOBbHszUtnOruC5vBUH+V5UW0dvay7Who1mrVNUQjV6gXwaiqbSNOZ1cMiqhP6AA3ryrkllWF/PD5g9S1dDoSw8bX6pkzM5UrluQ6cv7xXLOsgLSkeDaFqNulpslNUnwcSwt0QDTShHoRjKq6NsrnzmCmlitOWUwkdICv3lZGRkoC9zw6/V0vR9q6eOXISe50+M7QsaQmxXPdigKeqW4OSVVQtcvNssJMEuNj5i0XVUK1CMbJrl72NHZodUuQBPTbJSI3ikitiBwSkS+MsP0zIrJPRPaIyF9FZH7wQ52a3IxkvnprGbsbOvivv4W+DMvXH7c3EB8nvCvMBkP9Va4uouNMP3871B7U4xpj7Fv+tbslUoVqEQydXTG4xk3oIhIP/AS4CVgB3CkiK/x2ewNYY4xZDTwKfDfYgQbDhtWF3FBWwPefq5u2Cal6BwZ55PVGrl2eT35WeA2G+ruyJJeslISgV7s0nj6L+2y/3lAU4UKxCEZVbRu5GUk6thIkgbTQ1wKHjDFHjDF9wEPAbb47GGM2G2O8KyW8CoRlU1RE+PrbVpKWFM89j+6eltnkntsX3oOhvpIT4rlx5Wz+UtMS1IW3a1z2HaL6SxvRgr0IxqDPYtDh2hUZaQJJ6MVAg8/jRvu50XwYeGYqQYVSfmYKX721jDfqO/jVNHS9bNxWT/GM1IgZwa8sL6Krd4Cq2ragHbO6yUN8nFA6OzNox1TOCOYiGLsbO+g406+zKwZRUEeoROR9wBrgX0bZ/lER2SEiO9ragpcwJurW8iKuXV7A9/5SG5JRe69j7d1sPXSSO8PsztCxXLpoFrPSk4K63mi1y83S/AxSEuODdkzljGAugnGuXDE8K78iUSAJvQmY6/N4jv3ceUTkWuBLwK3GmN6RDmSM+YUxZo0xZk1ennMtVhHhm29fSXJCHPc+uidkXS8PeQdD18wdf+cwkRAfx82rCvnr/pagTMpkDYi6dYbFKPKui+awKAiLYFTVtnLBvJnMSNNyxWAJJKFvB5aKyEIRSQLuADb57iAiFwA/x0rmzqxdNUH5WSl85dYydhw/zW9ePhb04/cNDPHo6w1csyyfgjAfDPVXWV5ET/8Qz+9vmfKxWjt7ae/q01v+o0hCfByfmeIiGO1dvexpdOti0EE2bkI3xgwAHweeBfYDDxtjakTkayJyq73bvwAZwCMisktENo1yuLDy9guKeeuyfP7l2QMca+8O6rGf399Ce1cfd64L/8FQf2vmz6QwOyUoU+qemzJXW+jR5OaVhawsnvwiGFvqrC5X7T8ProD60I0xTxtjSowxi40xD9jP3W+M2WR/fa0xpsAY8xb7361jHzE8WF0vq0iMt7pehoLY9RJpg6G+4uKEDasLebGuFfeZ/ikdq7rJgwgsL9QWejSJixPuuWHyi2Bsrm0jNyOZMv3kFlQxf9ve7OwU/t+GFWw7dorfvnIsKMesP3mGlw62856L50bsVLGV5UX0DxqerTkxpeNUu9wsyk0nPTkhSJGpcDHZRTAGhwwvHWzjai1XDLqYT+hgDfJcXZLHd/5cS/3JM+O/YBwPba8nTuDdETQY6m9VcTbzZ6VNudqlRgdEo9ZkF8HY1eAtV4y8T6/hThM61hvzW+9YRUKccO9ju6fU9dI/OMTDOxp567ICZmdH1mCoLxGhcnURWw+10941YtHSuE529eJy9+gNRVFsMotgVNW26uyKIaIJ3VY0I5Uv3bKcV4+c4g+vHZ/0cZ7f10J7Vy93rYvc1rlXZXkRQwae2Tu5wdEae8rcMr3lP6pNdBGMqto2Lpw3k+y0xBBHFns0oft4z8VzuXJpLt965gANpybX9fLgtnoKs1O4uiTyR+9LZ2dSUpAx6WqXavuWf52UK7pNZBGMts5e9ja5tbslRDSh+xARvn37auJE+PxjeyZ8a3PDqcgfDPVXubqIbcdO0ew+O+HX1jR5mJeTRnaqtsSiXaCLYLyo5YohpQndT/GMVO67eRkvHz7JgxMsx4qGwVB/G8qLAHhqz8Rb6dUut86wGCMCXQSjqraVvMxkVmgZa0hoQh/BXWvncdniWXzzqf00ng6s68U7GLq+NJ+iGakhjnD6LMxNZ1Vx9oSn1HWf7ef4yTPa3RJDxlsEY2BwiJcOtmu5YghpQh+BiPCd21djgPse3xtQ18tf97fS1tkbEdPkTtSt5UXsbnRP6G7afd41RLVkMWaMtwjGroYO3Ge1XDGUNKGPYm5OGvfdtIyXDrbzx+0N4+6/cVs9s7NSovLNesvqQgCenEBNes3wgKh+tI4lYy2CUVXbRnyccOWS6PsdCRea0Mfw3nXzuWRRDg88tR9Xx+iDgg2nzrDlYBvvvnguCVG4ZmbRjFQuXjBzQtUu1U1uCrNTyM1IDmFkKtyMtQhGVV0rF86boeWKIRR92SeI4uKE795ezsCQGbPr5eEdDQhW2WO0qiwvoralk9oTnQHtX+3SNURj1UiLYLR29lDd5NHqlhDThD6OebPS+PyNpbxY18Yjrze+afvA4BB/3N5ARWk+xVE0GOrvppWFxElg3S5n+gY43NalFS4xaqRFMF6s9ZYrandLKGlCD8AHLl3A2gU5fP3JfW+6ceKFA620RulgqK+8zGQuW5zLE7td4w4S72/2YIyuIRrL/BfBqKprI1/LFUNOE3oA4uKE775zNf2DQ3zxT+d3vWzcVk9BVjLrY6DlUVleyLGTZ6hu8oy5n3e7VrjELt9FMB7b2chLddbsiiJarhhKmtADtCA3nXtuWMYLB1r50xvWKi1NHWepqmvjPWuiczDU3w1ls0mMl3FnYKxucpObkURBlg6IxjLvIhhf2VSDp2eA9cu0/zzUoj8LBdHdly1gzfyZfGVTDa2enuFyxndH8WCorxlpSVy1NI8nd7vGnJHSOyCqrbHY5l0E40zfIPFxwuVLdDHoUNOEPgHxdtdL78AQ9z2+l4e3N3B1SR5zZqY5Hdq0ufUtRbjcPeysPz3i9p7+QQ62dOqAqAKsRTCuLsnjqqW5OqfPNNBlZCZoUV4Gn7u+lAee3g/AV28rczii6XXt8gJSEuPYtNvFmgU5b9pe19LJwJDRAVEFWHdd//KDa4jTT2vTQlvok/B3VyzkwnkzKMxO4a0x1i+YnpzANcsKeHpvMwODb14cWAdElb+E+Didu2WaaAt9EuLjhN9+eB1n+gZIjIHBUH+V5YU8tbeZV4+c4oql5/eLVrvcZKUkMGdm9NbkKxWuYi8bBUlGcgL5mZG7xNxUVJTmk5GcMOIMjN41RHVAVKnppwldTVhKYjzXryjgmepm+gbOdbv0Dw6x/0Sndrco5RBN6GpSKsuL8PQM8NLBtuHnDrV20TcwpDMsKuUQTehqUi5fksuMtMTzul28c2BrC10pZ2hCV5OSlBDHTStn89y+Fs72DQJQ4/KQnhTPwlnpDkenVGzShK4mrbK8iO6+QTbXWjPqVTe5WVGUpSVqSjlEE7qatHULZ5GXmcymXS4Ghwz7mnUOdKWcpAldTVp8nHDLqkJeqG1lb5ObM32D2n+ulIM0oaspqSwvom9giB/YK73rHC5KOUcTupqSC+fNoHhGKi/WtZGcEMeSvAynQ1IqZmlCV1MiImwoLwRgWWFWTMwLr1S4Cui3T0RuFJFaETkkIl8YYXuyiPzR3v6aiCwIdqAqfFWuLgJgpd5QpJSjxk3oIhIP/AS4CVgB3CkiK/x2+zBw2hizBPgB8J1gB6rCV1lRFp+5roT3rpvvdChKxbRAWuhrgUPGmCPGmD7gIeA2v31uA/7b/vpR4BrR2ZlihojwyWuWskJb6Eo5KpCEXgw0+DxutJ8bcR9jzADgBmb5H0hEPioiO0RkR1tbm/9mpZRSUzCtI1jGmF8YY9YYY9bk5eVN56mVUirqBZLQmwDfVZDn2M+NuI+IJADZwMlgBKiUUiowgST07cBSEVkoIknAHcAmv302AR+0v34n8IIxZvRl4ZVSSgXduEvQGWMGROTjwLNAPPArY0yNiHwN2GGM2QT8EvidiBwCTmElfaWUUtMooDVFjTFPA0/7PXe/z9c9wLuCG5pSSqmJ0Nv6lFIqSmhCV0qpKCFOjV2KSBtwfJIvzwXagxhOpNPrcT69HufotThfNFyP+caYEeu+HUvoUyEiO4wxa5yOI1zo9TifXo9z9FqcL9qvh3a5KKVUlNCErpRSUSJSE/ovnA4gzOj1OJ9ej3P0Wpwvqq9HRPahK6WUerNIbaErpZTyowldKaWiRMQl9PGWw4sVIjJXRDaLyD4RqRGRTzkdUzgQkXgReUNEnnQ6FqeJyAwReVREDojIfhG51OmYnCIin7Z/T6pFZKOIpDgdUyhEVEIPcDm8WDEAfNYYswK4BPjHGL4Wvj4F7Hc6iDDxQ+DPxphlQDkxel1EpBj4JLDGGLMSa5LBqJxAMKISOoEthxcTjDHNxpid9tedWL+s/itJxRQRmQPcAvyX07E4TUSygauwZkLFGNNnjOlwNipHJQCp9noNaYDL4XhCItISeiDL4cUcEVkAXAC85mwkjvs34F5gyOlAwsBCoA34td0F9V8iku50UE4wxjQB3wPqgWbAbYz5i7NRhUakJXTlR0QygMeAfzLGeJyOxykisgFoNca87nQsYSIBuBD4qTHmAqAbiMkxJxGZifVJfiFQBKSLyPucjSo0Ii2hB7IcXswQkUSsZP4HY8zjTsfjsMuBW0XkGFZX3FtF5PfOhuSoRqDRGOP91PYoVoKPRdcCR40xbcaYfuBx4DKHYwqJSEvogSyHFxNERLD6R/cbY77vdDxOM8bcZ4yZY4xZgPW+eMEYE5WtsEAYY04ADSJSaj91DbDPwZCcVA9cIiJp9u/NNUTpAHFAKxaFi9GWw3M4LKdcDrwf2Csiu+znvmivLqUUwCeAP9iNnyPAhxyOxxHGmNdE5FFgJ1Z12BtE6RQAeuu/UkpFiUjrclFKKTUKTehKKRUlNKErpVSU0ISulFJRQhO6UkpFCU3oSikVJTShK6VUlPj/XoiGGgd7pq4AAAAASUVORK5CYII=\n", "text/plain": [ "
" ] }, "metadata": { "needs_background": "light" }, "output_type": "display_data" } ], "source": [ "plt.plot([ arm.mu for arm in levers] )\n", "plt.title(\"distribution of expected value of arms\")" ] }, { "cell_type": "code", "execution_count": 11, "metadata": { "scrolled": false }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Epsilon-Greedy Agent, epsilon= 1\n", "maximum possible reward: 0.9851042878107023\n", "average reward: [0.47962497]\n", "cosine similarity 0.3162277660168379\n", "euclidian norm 0.09486832980505139\n" ] }, { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYIAAAEGCAYAAABo25JHAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAgAElEQVR4nO3de5xdVX338c/3nDlzySSThGTCJSEkaDCCysVAUVBRBAMqVKsVtFqvtNZbtRextkK17cvqo1VbFaP1Qfso1nqNlQqoCFRFCAJCuCVAgHDL5J65zznn9/yx94SZ5MzkJJl9Jpn9fb9e88rea++z92+dnJnfWXvtvZYiAjMzy6/CZAdgZmaTy4nAzCznnAjMzHLOicDMLOecCMzMcq5psgPYW3Pnzo1FixZNdhhmZgeVW265ZWNEdNbadtAlgkWLFrFq1arJDsPM7KAi6aGxtvnSkJlZzjkRmJnlnBOBmVnOORGYmeWcE4GZWc45EZiZ5ZwTgZlZzuUmEdz35A4+ffW9bOwemOxQzMwOKJklAklflbRB0p1jbJ8p6UeSbpe0WtJbsooFYM2T3Xzu52vZ3DOY5WnMzA46WbYILgeWj7P9XcBdEXE8cAbwKUnNGcZjZmY1ZJYIIuJ6YPN4uwAzJAmYnu5bzioeMzOrbTL7CP4NeCbwGHAH8L6IqNbaUdJFklZJWtXV1dXIGM3MprzJTAQvA24DjgBOAP5NUketHSNiRUQsi4hlnZ01B8+rm6doNjMbbTITwVuA70ViLfAgsDSrk0lZHdnM7OA2mYngYeBMAEmHAs8AHpjEeMzMcimz+QgkXUFyN9BcSeuBS4ASQERcBnwMuFzSHYCAD0bExqziMTOz2jJLBBFx4R62PwacndX5zcysPrl5snhY4N5iM7ORcpMI3FdsZlZbbhKBmZnV5kRgZpZzTgRmZjnnRGBmlnO5SwQeYsLMbLTcJAIPMWFmVltuEoGZmdXmRGBmlnNOBGZmOZe7RODOYjOz0XKUCNxbbGZWS44SgZmZ1eJEYGaWc5klAklflbRB0p3j7HOGpNskrZZ0XVaxmJnZ2LJsEVwOLB9ro6RZwBeA8yLiOOC1Gcayk+cjMDMbLbNEEBHXA5vH2eX1JJPXP5zuvyGrWMBPFpuZjWUy+wiOAWZL+oWkWyS9aRJjMTPLrczmLK7z3M8FzgTagF9LujEi7tt1R0kXARcBLFy4sKFBmplNdZPZIlgPXBURPRGxEbgeOL7WjhGxIiKWRcSyzs7OhgZpZjbVTWYi+CFwuqQmSdOA3wPuzvqkfrLYzGy0zC4NSboCOAOYK2k9cAlQAoiIyyLibkk/AX4HVIGvRMSYt5rudzxZHdjM7CCXWSKIiAvr2OeTwCezisHMzPbMTxabmeWcE4GZWc45EZiZ5VxuEoH8aLGZWU25SQRmZlabE4GZWc45EZiZ5VzuEoGfLDYzGy03icBdxWZmteUmEZiZWW1OBGZmOedEYGaWc04EZmY5l7tE4MnrzcxGy00i8AgTZma15SYRmJlZbZklAklflbRB0rizjkk6WVJZ0muyisXMzMaWZYvgcmD5eDtIKgL/DFydYRxmZjaOzBJBRFwPbN7Dbu8BvgtsyCqOXXmICTOz0Satj0DSfOBVwBfr2PciSaskrerq6trH8+3Ty8zMprzJ7Cz+DPDBiKjuaceIWBERyyJiWWdnZwNCMzPLj6ZJPPcy4FvpzGFzgXMllSPiB5MYk5lZ7kxaIoiIxcPLki4H/ttJwMys8TJLBJKuAM4A5kpaD1wClAAi4rKszrsn7is2Mxsts0QQERfuxb5vziqOYfKMBGZmNfnJYjOznHMiMDPLuTEvDUn6V8a5pB4R780kIjMza6jxWgSrgFuAVuAkYE36cwLQnH1o2Qg/WmxmNsqYLYKI+BqApHcCp0dEOV2/DLihMeFNIPcVm5nVVE8fwWygY8T69LTMzMymgHpuH/04cKuka0m+V78QuDTLoMzMrHHGTQRKxn/4KfA/wO+lxR+MiCeyDszMzBpj3EQQESHpyoh4NvDDBsWUKXcVm5mNVk8fwW8lnZx5JBlzX7GZWW319BH8HvAGSQ8BPSR/UyMinpNpZGZm1hD1JIKXZR6FmZlNmj0mgoh4CEDSPJKHy8zMbArZYx+BpPMkrQEeBK4D1pHcRXRQ8oPFZmaj1dNZ/DHgVOC+dDKZM4EbM40qA/KkxWZmNdWTCIYiYhNQkFSIiGtJppk0M7MpoJ5EsFXSdOB64BuSPkty99C4JH1V0gZJd46x/Q2SfifpDkm/knT83oVuZmYToZ5EcD7QC7wf+AlwP/DKOl53ObB8nO0PAi9KH1b7GLCijmOamdkEq+f20QuA6yNiDfC1eg8cEddLWjTO9l+NWL0RWFDvsc3MbOLUkwgWAl+StJhkjoLrgRsi4rYJjONtjHMnkqSLgIsAFi5cuJ+n8m1DZmYj7fHSUERcEhEvAY4lmYfgr0gmrJkQkl5Mkgg+OE4MKyJiWUQs6+zs3Lfz7GN8ZmZT3R5bBJL+FjiNZB6CW4G/ZIImppH0HOArwDnpnUlmZtZg9VwaejVQBn5M8kDZryNiYH9PLGkh8D3gjRFx3/4ez8zM9k09Q0ycJKmDpFVwFrBC0oaIOH2810m6AjgDmCtpPXAJUEqPeRnwEWAO8IX0Ya9yRPj5BDOzBqvn0tCzgBcALyJ5kOwR6rg0FBEX7mH724G31xfmxPEQE2Zmo9U7VeUNwOeAmyNiKNuQsuERJszMaqvn0tArJLUBCw/WJGBmZmOrZ/TRVwK3kTxVjKQTJK3MOjAzM2uMeoaYuBQ4BdgKkD5ItjjDmMzMrIHqHX102y5lB22X60EbuJlZRurpLF4t6fVAUdIS4L3Ar/bwmgOO/GyxmVlN9bQI3gMcBwwA3wS2A3+eZVBmZtY49dw11At8OP0Bdj4V/HCGcZmZWYOM2yKQ9DxJr0knrkfScyR9E/hlQ6IzM7PMjZkIJH0S+CrwB8CPJf0DcDXwG2BJY8KbeH6y2MxstPEuDb0cODEi+iXNJhla4lkRsa4hkU0wP1lsZlbbeJeG+iOiHyAitgBrDtYkYGZmYxuvRXD0Lk8QLx65HhHnZReWmZk1yniJ4Pxd1j+VZSBmZjY5xkwEEXFdIwNplHBvsZnZKPU8UDYluK/YzKy2zBKBpK9K2iDpzjG2S9LnJK2V9DtJJ2UVi5mZja3uRCBp2l4e+3Jg+TjbzyF5HmEJcBHwxb08vpmZTYB65iN4vqS7gHvS9eMlfWFPr4uI64HN4+xyPvD1SNwIzJJ0eJ1xm5nZBKmnRfAvwMuATQARcTvwwgk493ySh9SGrU/LdiPpIkmrJK3q6urar5O6q9jMbLS6Lg1FxCO7FFUyiGW886+IiGURsayzs3PfDuLeYjOzmuqZj+ARSc8HQlIJeB9w9wSc+1HgyBHrC9IyMzNroHpaBH8KvIvkss2jwAnp+v5aCbwpvXvoVGBbRDw+Acc1M7O9UM98BBuBN+ztgSVdAZwBzJW0HrgEKKXHvAy4EjgXWAv0Am/Z23OYmdn+22MikPS5GsXbgFUR8cOxXhcRF4533Ege8Z2IloWZme2Hei4NtZJcDlqT/jyH5Hr+2yR9JsPYMuERJszMRquns/g5wGkRUQGQ9EXgBuB04I4MY5tQnrzezKy2eloEs4HpI9bbgUPSxDCQSVRmZtYw9bQIPgHcJukXJHfjvxD4J0ntwE8zjM3MzBqgnruG/l3SlcApadHfRMRj6fJfZRaZmZk1RL2DzvUDjwNbgKdLmoghJiZFeJAJM7NR6rl99O0kTxMvAG4DTgV+Dbwk29AmlievNzOrrZ4WwfuAk4GHIuLFwInA1kyjMjOzhqknEfRHRD+ApJaIuAd4RrZhmZlZo9Rz19B6SbOAHwDXSNoCPJRtWGZm1ij13DX0qnTxUknXAjOBn2QaVZbcV2xmNsq4iUBSEVgdEUsBIuK6hkSVAfcVm5nVNm4fQfr08L2SFjYoHjMza7B6+ghmA6sl3QT0DBdGxHmZRWVmZg1TTyL4u8yjMDOzSVNPZ/F1ko4ClkTETyVNA4rZh5YN9xWbmY22x+cIJL0D+A7wpbRoPsmtpHskabmkeyWtlXRxje0LJV0r6VZJv5N07t4EvzfkR4vNzGqq54GydwGnAdsBImINMG9PL0rvOPo8cA5wLHChpGN32e1vgW9HxInABcAX6g/dzMwmQj2JYCAiBodXJDVR3xWWU4C1EfFA+vpvAefvsk8AHenyTOAxzMysoepJBNdJ+hugTdJZwH8BP6rjdfOBR0asr0/LRroU+KN0cvsrgffUOpCkiyStkrSqq6urjlObmVm96kkEFwNdJNNS/gnJH+y/naDzXwhcHhELgHOB/5C0W0wRsSIilkXEss7Ozv06oecsNjMbrZ7bR38f+HpEfHkvj/0ocOSI9QVp2UhvA5YDRMSvJbUCc4ENe3muPXJfsZlZbfW0CF4J3CfpPyS9Iu0jqMfNwBJJiyU1k3QGr9xln4eBMwEkPRNoJWl9mJlZg+wxEUTEW4Cnk/QNXAjcL+krdbyuDLwbuAq4m+TuoNWSPipp+KnkvwDeIel24ArgzRG+eGNm1kh1fbuPiCFJ/0Nyl08byeWit9fxuitJ+hRGln1kxPJdJLemmpnZJKnngbJzJF0OrAH+APgKcFjGcWXGcxabmY1WT4vgTcB/An8SEQMZx5MZ9xWbmdVWz1hDF45cl3Q6cGFEvCuzqMzMrGHq6iOQdCLweuC1wIPA97IMyszMGmfMRCDpGJK7hC4ENpJcHlJEvLhBsZmZWQOM1yK4B7gBeEVErAWQ9P6GRGVmZg0z3l1DrwYeB66V9GVJZzIF+lz9lIKZ2WhjJoKI+EFEXAAsBa4F/hyYJ+mLks5uVIATxUNMmJnVVs+TxT0R8c2IeCXJeEG3Ah/MPDIzM2uIesYa2ikitqQjgZ6ZVUBmZtZYe5UIzMxs6sldInBfsZnZaDlKBO4tNjOrJUeJwMzManEiMDPLOScCM7OcyzQRSFou6V5JayVdPMY+fyjpLkmrJX0zy3gAPAGamdlo9c4/vNckFYHPA2cB64GbJa1MZyUb3mcJ8CHgtIjYImledvFkdWQzs4Nbli2CU4C1EfFARAwC3wLO32WfdwCfj4gtABGxIcN4zMyshiwTwXzgkRHr69OykY4BjpH0S0k3Slpe60CSLpK0StKqrq6ujMI1M8unye4sbgKWAGeQzHvwZUmzdt0pHdZiWUQs6+zsbHCIZmZTW5aJ4FHgyBHrC9KykdYDKyNiKCIeBO4jSQyZcVexmdloWSaCm4ElkhZLagYuAFbuss8PSFoDSJpLcqnogSyCcV+xmVltmSWCiCgD7wauAu4Gvh0RqyV9VNJ56W5XAZsk3UUy58FfRcSmrGIyM7PdZXb7KEBEXAlcuUvZR0YsB/CB9MfMzCbBZHcWm5nZJMtfInBvsZnZKLlJBIX00eJwJjAzGyU3iWB4iIlqdXLjMDM70OQmETzVIqjt/q5u1m7Y0biAzMwOEJneNXQg2dkiGGP00TM/dR0A6z7+8kaFZGZ2QMhfi6BGItjRP7Rz+dKVqxsWk5nZgSCHLYLdtz33H366c/nyX63j0vOOG7V9Y/cAzU0FOlpLWYY4YYYqVVZc/wD3PLGD0542h9se2cqGHQMc2tHKj25/jO6BMkvmTee9Zy7h8W197OgvM39WGycvPoSndU5nW+8Q01qKlIq7f0+4ed1mbnpwM3+47Eh6Bsqs29TDkYdM44iZbezoH+KRLX2ctHAWkqhUg219Q8yeVuKJ7f30DVa478kdnLRwNvM6WgGoVoOhapVN3YO0tzQxs62x7/GGHf3MbCshxGClypaeQTpntFCQaCqIru4BdvQPcdSc9prvhx2YBstVHtnSS1upSOeMFnoHK2zvG2KgXN25bUd/mektRdpbmpje0kSpWEBKvjRWqkE1gpamIsUCdLSWkERE0FQsMKO1aZ8/D8NfRjVibPyIIAKGqlU29wwyVA5mTisxVKlSKhZoby7SlOHnLzeJ4KkWwe7bBsuje5AXXfxjbv27s2gtFWktFViWJopPvOY5vOrE+TU/AD0DZdpbmvjZ3U/ytq+tAuDq97+Qp3VOp1gY/R/+0KZeChKHzWylVBSS6BkoI0FrU5FCQQxVkpjq+bCt29jDOZ+9gb6hym7bfnT7YzVfs2ZDN++54tbdyp/W2c79XT2jyp55eAdrN+ygGlBJM+knr7p33JhaSwX6h8bume9oTT56/ekv5rD5s9qY1lykrblIQeJZ8ztoLhbpHhjixIWz2d43RFtzkaPnTmfdph7ufHQbdzy6jY3dA7zomE6O7pzOULlKz2CFtlKR5qYCbaUCPYMVigUxb0YLG7sHuPGBzdzz+HYe29Y/ZozFgnbWtyCYP7uN2dOaKRULPOOwGUQES+bNoLmpwM3rNvPs+TN5eHMvpWKBSjWoVIMFs9uY19HCMw7toFyt0jdYoVINtvYNceTsabQ1F+loa2JT9yB9QxX6Byt0zmjh6fOmj/pDMZ7ewTIPb+4F4PCONlDyZWCoUqWtVGTWtGYGyhW29Q6xuXeQHf1lHt7Uy9Gd7fQMVLjr8W3c+vBWKtVgZluJ447o4Lj5MxkYqtLV3U/vYIVyJdjSO0hTQTQ3FWhpKtLSVGBeRwuSaGkqMDBUpW+ogoDmpuRzO1QJugfKlIqipSn5f21Nt5WrweaeQR7b2kf/UJWZbU1MT79sdfcPsWD2NObOaGEw/YwMVioMDFXZ2JPEUakGW3oG2dg9wBPb+9neV2Zb3xD9QxXWb+ljsJLtnSHtzUVmtpUoFET/UJWWpgItpeT/fnpLE3OmtzCzrcRjW/vY3DPIYLlKqSge29ZPRCCJappwan1B3VWpKP70RU/jL85+xoTXRQfbjF3Lli2LVatW7fXr1m7YwUs/fT3/euGJvPL4I3aW/+r+jbz+y7/hsI5WNuzor+s/5HlHz+GKi04FoG+wwjM/8hMA3vuSp/O5n6/dbf87Lj2bGa0lHt3ax2kf//lexf3TD7yQp8+bAcAjm3tZv6WPpYfNYFv67ebq1U/wqWvu2+11Jy6cxTMP72DpYTM4/elz6Rmo8Olr7uVVJy3gpc+cx/+78SFueWgLbz1tMTv6y7z968l7Omtaie19Q+O+DzPbShzd2c7iOe08bd501m/p44ltffQOVmhuKrBg9jRuenATEXDErDbamos0FcQxh87g0I5WunYMsLlnYOcfuqaCOGrONO7v6mFzzyA7+ocYqgRPbu9nzYZuWpoKBLsn7GFHd7YzWK6yfktf3e9r54wWFs9p57j5HbSWikxLk8aW3iFWP7aNE4+cRbkaHD6rjaLE49v6WLeply09g2ztG+ShTb00Fwts6hkcddy2UpGBcoWWpmLNxFyv5mKB1lKBw2e2MXNaiWnNRTZsH2Ddpp6d33KHKlW295fp2jEw7rGmNRfpHRw/lpltJbb3DzGnvZmN3YPj7nugmdHaxLwZLcye1szMthLFgjhiVhvPnj+TgXKVDTv6md7SREdbidZSkVIh+RI2p72FnsEy3QPJz1C5SjX9Zl4siIJEf7lCNWBb3xDValBQksC295XZ3j/E5p5BKtWgvaWJgXKFgXKVcqXKUCXYsKOfHf1lDu1o5dCOViKSLwdHzGqjqZh89osSxULyZTBZhkPaW2gqiu19Q5SKBQbLSYLtG6pwyqJDePHSfZu/S9ItEbGs5rb8JIJuXvrp6/jsBSdw/glPTYuw/DPXc88TOzhiZis/ePdpnPKPP6vreB86ZylvOPUonnXJVXvc90XHdPJHpx7FO76+93EDXP6Wk3nz/715zO1zp7fwmdedgATPPWo2raXiPp2nlk3dA9zf1cNjW/t48dJ5Db90M1Cu0FQoUI3gtw9t4c7HttNaKlAqFJgzvZmTFx+y85JdpRr0DpYJkj+k2/uHWPNkN0/rnL7zFwugvSX5w1HvN+7xPLK5l4Fyld7BMofPbKNzRsvOpn81kv6nGx/YzEC5QjWC2dOaaU2TzuNb+xmqVHl4cy8LZrfR3tJEe3MTD2zs5qFNvQyWq9zf1c2m7kGam5L6Hjl7Gjv6h9jWN8S05ibaW4ocNaedhYdMA2D9lj5KRTFQrjKjtYnugTJPbutn7vQWZrU3M3taifaWJpqLBdZv6WVmWzMnLpzFoemluojg4c29PLAxSThzp7fQ0dpEsSBmpO/zQLnCYLlKz0CFJ7b37/wD19ZcpLVUTN6PgTItpQKtpSLtzU1UIugfqtA/VKU/TZBNBdHRVuKIWW20NxfZ1jdEz0CFIDnWw5t62d4/RHOxmLZCCsn70N7MUDUoSsxuL9HSNHGf96nMiQB4oKubl3zqOv7ldcfzqhMX7CxfdPGPATjuiA5+/N4X8OxLr2JHf3m316/5x3O46cHNvOErvxn3PDNamrjj718GJL9Uiz80aqglOlqb+M47n8/c6S0MlCscPrNt5+WHXS8hvf1rq/jZPWNP2nbq0Ydw7rMP57zjj2DWtOY9vANmlmfjJYLc9BEM/7H9xx/fMyoRDBv+Fn3Hpckf8eEEcd7xR/Chc5dSKhY47elzWffxl3PqP/2MJ7Y/dW35i284iaWHd/D9367n/Wcds7Nc0qjLRbsmoWEjE8DI137pjc/lU9fcxxd/cT83ffhM5s1o3dfqm5mNKTeJYCC9vryxu/b11GMOnTFq/bvvfD6zp5U4unP6bvuufPdp/N0P76RULPB/Xnv8ziTygRqdOO85cwmvPmkBi+a273XMTcUCH1y+lA8uX7rXrzUzq1duEsFYZk0r0VYqcskrjx1V/tyjZo/5mnkdrXzpjTVbWLspFQv7lATMzBol1zdGRwS9AxXOO+GICe1gNTM7mGSaCCQtl3SvpLWSLh5nvz+QFJLq+5q9Dwo17hC57ZGtDFaqTG/OfcPIzHIss0QgqQh8HjgHOBa4UNKxNfabAbwPGP92nP2O56nlJ9OO3tde9msAbli7MctTm5kd0LJsEZwCrI2IByJiEPgWcH6N/T4G/DMw9iOeE2DkXbJ96cM1zzgs6SB+xwuOzvLUZmYHtCwTwXzgkRHr69OynSSdBBwZET8e70CSLpK0StKqrq6ufQqmveWpPoCCxBd+sZZlaYfwWcceuk/HNDObCibt4rikAvBp4M172jciVgArIHmgbF/Od9Scp+7cufquJ/jET8YfK8fMLC+ybBE8Chw5Yn1BWjZsBvAs4BeS1gGnAiuz7DAe1j2w+5PDZmZ5lWUiuBlYImmxpGbgAmDl8MaI2BYRcyNiUUQsAm4EzouIfRuQZy/87xp3DpuZDcssEUREGXg3cBVwN/DtiFgt6aOSzsvqvPVY9dCWncsjRyI1M8ujTPsIIuJK4Mpdyj4yxr5nZBnLWJYeNmPPO5mZTWG5frIYkrHazczyLPeJYOQoomZmeZT7RNBUYwhoM7M8yX0iaC760pCZ5VvuE8GbT1s02SGYmU2q3A67+aU3PpfjF8xq+By8ZmYHmty2CNqbmzhspqd+NDPLbSKoNU+wmVkeORGYmeVcbhPBvBktkx2CmdkBIVeJ4Pt/9vydy55Q3swskatEcOLC2ZMdgpnZASdXicDMzHbnRGBmlnNOBGZmOZfpk8WSlgOfBYrAVyLi47ts/wDwdqAMdAFvjYiHsoxpxRufi+RbR83MhmWWCCQVgc8DZwHrgZslrYyIu0bsdiuwLCJ6Jb0T+ATwuqxiAjj7uMOyPLyZ2UEny0tDpwBrI+KBiBgEvgWcP3KHiLg2InrT1RtJJrg3M7MGyjIRzAceGbG+Pi0by9uA/8kwHjMzq+GAGH1U0h8By4AXjbH9IuAigIULFzYwMjOzqS/LFsGjwJEj1hekZaNIeinwYeC8iBiodaCIWBERyyJiWWdnZybBmpnlVZaJ4GZgiaTFkpqBC4CVI3eQdCLwJZIksCHDWMzMbAyZJYKIKAPvBq4C7ga+HRGrJX1U0nnpbp8EpgP/Jek2SSvHOJyZmWUk0z6CiLgSuHKXso+MWH5pluc3M7M985PFZmY5p4iY7Bj2iqQuYF+fPp4LbJzAcA4GrnM+5K3Oeasv7H+dj4qImnfbHHSJYH9IWhURyyY7jkZynfMhb3XOW30h2zr70pCZWc45EZiZ5VzeEsGKyQ5gErjO+ZC3OuetvpBhnXPVR2BmZrvLW4vAzMx24URgZpZzuUkEkpZLulfSWkkXT3Y8+0PSVyVtkHTniLJDJF0jaU367+y0XJI+l9b7d5JOGvGaP073XyPpjyejLvWQdKSkayXdJWm1pPel5VO5zq2SbpJ0e1rnv0/LF0v6TVq3/0zH8UJSS7q+Nt2+aMSxPpSW3yvpZZNTo/pIKkq6VdJ/p+tTvb7rJN2RDrGzKi1r/Oc6Iqb8D8lUmfcDRwPNwO3AsZMd137U54XAScCdI8o+AVycLl8M/HO6fC7JPA8CTgV+k5YfAjyQ/js7XZ492XUbo76HAyelyzOA+4Bjp3idBUxPl0vAb9K6fBu4IC2/DHhnuvxnwGXp8gXAf6bLx6af9xZgcfp7UJzs+o1T7w8A3wT+O12f6vVdB8zdpazhn+u8tAj2OFvawSQirgc271J8PvC1dPlrwO+PKP96JG4EZkk6HHgZcE1EbI6ILcA1wPLso997EfF4RPw2Xd5BMojhfKZ2nSMiutPVUvoTwEuA76Tlu9Z5+L34DnCmJKXl34qIgYh4EFhL8vtwwJG0AHg58JV0XUzh+o6j4Z/rvCSCvZ0t7WB0aEQ8ni4/ARyaLo9V94PyPUkvAZxI8g15Stc5vUxyG7CB5Jf7fmBrJCP7wuj4d9Yt3b4NmMPBVefPAH8NVNP1OUzt+kKS3K+WdIuSCbhgEj7XB8QMZTaxIiIkTbn7giVNB74L/HlEbE++ACamYp0jogKcIGkW8H1g6SSHlBlJrwA2RMQtks6Y7Hga6PSIeFTSPOAaSfeM3Nioz3VeWgR1zZZ2kHsybSaS/js80c9YdT+o3hNJJZIk8I2I+F5aPKXrPCwitgLXAs8juRww/AVuZPw765Zunwls4uCp82nAeZLWkVy6fQnwWaZufQGIiEfTfzeQJPtTmITPdV4SwR5nS5sCVgLDd8uYrXcAAAMdSURBVAv8MfDDEeVvSu84OBXYljY7rwLOljQ7vSvh7LTsgJNe+/134O6I+PSITVO5zp1pSwBJbcBZJH0j1wKvSXfbtc7D78VrgJ9H0pO4ErggvctmMbAEuKkxtahfRHwoIhZExCKS38+fR8QbmKL1BZDULmnG8DLJ5/FOJuNzPdm95o36Ielxv4/kOuuHJzue/azLFcDjwBDJ9cC3kVwf/RmwBvgpcEi6r4DPp/W+A1g24jhvJelMWwu8ZbLrNU59Tye5lvo74Lb059wpXufnALemdb4T+EhafjTJH7a1wH8BLWl5a7q+Nt1+9IhjfTh9L+4FzpnsutVR9zN46q6hKVvftG63pz+rh/8uTcbn2kNMmJnlXF4uDZmZ2RicCMzMcs6JwMws55wIzMxyzonAzCznnAgstyR1p/8ukvT6CT723+yy/quJPL7ZRHIiMINFwF4lghFPu45lVCKIiOfvZUxmDeNEYAYfB16Qjgn//nSwt09Kujkd9/1PACSdIekGSSuBu9KyH6QDhq0eHjRM0seBtvR430jLhlsfSo99ZzoO/etGHPsXkr4j6R5J39DIwZTMMuRB58ySMd//MiJeAZD+Qd8WESdLagF+KenqdN+TgGdFMsQxwFsjYnM6DMTNkr4bERdLendEnFDjXK8GTgCOB+amr7k+3XYicBzwGPBLkvF3/nfiq2s2mlsEZrs7m2RMl9tIhrueQzJmDcBNI5IAwHsl3Q7cSDLw1xLGdzpwRURUIuJJ4Drg5BHHXh8RVZJhNBZNSG3M9sAtArPdCXhPRIwauCsdHrlnl/WXAs+LiF5JvyAZA2dfDYxYruDfT2sQtwjMYAfJFJjDrgLemQ59jaRj0tEhdzUT2JImgaUk0wcOGxp+/S5uAF6X9kN0kkw7ekCOjmn54W8cZskIn5X0Es/lJOPgLwJ+m3bYdvHUdIEj/QT4U0l3k4x0eeOIbSuA30n6bSTDKQ/7Psm8AreTjKj61xHxRJpIzCaFRx81M8s5XxoyM8s5JwIzs5xzIjAzyzknAjOznHMiMDPLOScCM7OccyIwM8u5/w/Vb49l+Dtm/QAAAABJRU5ErkJggg==\n", "text/plain": [ "
" ] }, "metadata": { "needs_background": "light" }, "output_type": "display_data" }, { "name": "stdout", "output_type": "stream", "text": [ "\n", "\n", "Epsilon-Greedy Agent, epsilon= 0\n", "maximum possible reward: 0.9851042878107023\n", "average reward: [0.98686237]\n", "cosine similarity 1.0\n", "euclidian norm 0.0\n" ] }, { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYoAAAEGCAYAAAB7DNKzAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAgAElEQVR4nO3de5zcdX3v8dd7Zmf2kjtJuIaQoLEUFIGzAgpFVFSwCD2tbaH2SKuVakVr77S01tqec2w9vRx7aC21HNGHgpZWTCuKqCjUCiRc5Z4QbgmBhEBue53Lp3/8frOZHXYnk+zO/Dab9/Px2Ed+t53f5zuZ/X3me/l9f4oIzMzMJpPLOgAzM5vZnCjMzKwpJwozM2vKicLMzJpyojAzs6a6sg5gui1ZsiRWrFiRdRhmZgeUu+6664WIWDrRvlmXKFasWMHatWuzDsPM7IAi6anJ9rnpyczMmnKiMDOzppwozMysKScKMzNrKtNEIelqSVskPTDJfkn6tKT1ku6XdEqnYzQzO9hlXaP4HHBuk/3nAavSn0uBv+9ATGZmVifTRBERtwIvNjnkQuDzkbgdWCjpiM5EZ2ZmMPPvozgKeKZufWO6bXP9QZIuJalxsHz58o4Fl5VypUpXPkdEIKkt55jstZ98YYDbN2zj0PndSCIiKFeC3SNlAOb1FFjUV2DZoj4Wzy3ywKYdVCM4cdlCCvn9/14SEWzaPkRfsYuFvQUAcrn2lN1sMrXHMrTr726mmumJoiURcRVwFUB/f/+secBG7WIdEdz99Etc/R9PsnnHEHc/vX3cccV8jqMW9fLm4w7l7ScczryeLgZHK2zdNcIdT2zjgU07yEkMlSosnlOkt5hntBwctbCHf7t/M6etPISeQp5li3oZKVe5Y8M27tu4gx87bB5HLuxh6+4RNmwdQMDAaGW/y/OaoxZwzOI+ivkcyxf38cjmXQyMlpnbncS7c7hEqVJlTrGLfE5IEAGVarDhhQG27hoZV+bDFnQzp9hFIZ+jr5inu5CnUq1yyJxuunJK4y2zddcIhy/o4dntw+wcLnHskjkcs3gORy7sZfGcIgv6CmzePszgaJneYp5iPkc1gmpATyFHPpdj13CJvmKekVKV3mKevmIX3V05tuwaoa+YHzuuuytHuRIMjJbJSXTlRFde5HNi6dxuDl/QQyGfQ4JCLkepWiUCegr5vb5/g6NlhktVBkbKlKtBbyHP9qFRdgyW2DVcJp8XxXyOud1dBNCVS86bU/JelipVypWgu5Abe49LlerY65cqtT+doFxNvgBUIxguVXl+5zClSpVdw2UGRyvp9go9hXx6DshJlNLfqVSTn4hgXk+BcjXYMVRiuFRJ9kWyLyL5TO0cKpHPJe/XUKnC7pEyu4bLVKpBXzGPBEKUq8FIucJoucpIuUqpUqWYz1HsylGtJvEXunLkc4KASP+OaiXL50Qhl/z/BlC73FfTWPI5jf3/DJcqjJSrDJeqjJST5dFy8n4V8qKnK09PMc/8ni7m9hToSc/bV8yzdF4PEcFo+p4X8jl6CjlGy1Uq6fszOFpJPx858kq++AyOVNg1UmLnUJmdwyX6il30FfOUq8FoOfm/37Z7hJFylXJa3q405q68OOnohXzhfaft99/oZJT1g4skrQD+PSJePcG+fwC+FxHXpuuPAmdHxObGY2v6+/tjJtyZ/d1Hnue9n1vL3158MuefeMTLvoEMjJT5wfoXkJIL2qbtQwDc+cSLDI6WKeRzfOuh51/2un3FPH3FPC/sHuWQOUVWLO5j6+4RKpXg2R3Dk8Zz7JI5zO3pYsdQCQFbdo1QqlQ5YkEvlfSPuFYr+PEj5nPInAJbd41QrgTzewu8Yulc5vV0sWxRL6cfu5iBkTK59AIxMFJhQW+BfE6MVqrsHCrx1LZBtg2McswhfewaLrHhhQGeSH+GRitsGxhlUV+BIxb0MlyqkM+Jw+b3UMiLgdEK1WpywQHoyiWJ8DVHLaAawc7hMqPlKpt3DDE0Whm7gJWqkZRtZ/I+SGJOdz69oJY4fH4PC/sKPL51gI0vDTJcqk7ybnVeTyHH0nnddOWSWtdIqcL83gIvDowylF6Yp5KkZ4JCXvQU8uMSGEBvMc/C3gKVtHbaU8gzr6eLeT1dSGIw/VwmyS9Jxt1duT0X30qVkVKVfE4ESY279lmo/X3V/vzKlaBUqY77IiIlx+UkypU9yaC3mKe7K093oXbOPMWuHAJGK1WGSxWGSxV2DJUYGKkwlCbBgZEyL+weJZ9L4i3kNZbYar9f6MrRW8hTqSZJuZZY+4p55vcWmNvdxYLeAoOjFQZHy3Tlkxj6inkWz+mmp5CjK61Rl6tJmUqV4MiFPVx61iv26/9H0l0R0T/Rvpleo1gNXCbpOuA0YEezJJGliOC1f/Itdg6XueC1R7L6vmcB+PC19/Dha+/hmMV9/O3FJ3PLI1v5628/tk+vfezSOWzYOsAvnr6cP73w1ZNWe7927yae3jZIJYKdQ2V2j5S45A0rOOHIBRPGC+Or0LUENZUmolYNjpbpLeQzq8JXq8HO4RIv7B5l53CJud1Jk1YAo+UqubRGMlyqUK4G83sKDJUqFLtyDI1Wkp9ShUPmFChVgsHRMqX0ItRTSJJTBJSrVSrVoFQJNu8YYmuaoCtVGCpVmNudp1KFncMlXhwYHTt3d1eO7YMlTjhyQXrBhEPn9VDsylHMi7lprXFBb4GFvUXm9XRRieRb5+7hMlJyAYm0ZlRLur2F/FitpBox9n9diaC7K0dE8s22djGvfVutJfEFvQX6isllo6eQY6RcHVfO7q7k238+J/JpTWbHUInurqTWdbA12cwWmdYoJF0LnA0sAZ4H/hgoAETEZ5R8qv4fycioQeCXI6JpdSGLGsXT2wY561O3vGz7/J4udg6XW3qNvmKeS96wgmOXzOGEIxewcskceot7b44wM5sOM7ZGEREX72V/AB/qUDj75ZkXxyeJ4w6fxyPP7aKnkOP+j78dSL4tfuyGB7jh3qSWce37T+f1r1icSbxmZvsq8z6K6dbJGsUZn/zuWN8CwDXvPZU3vmrCWXrNzGa0GVujOJANjVbGksS87i7u/eO3JSMtzMxmGSeKfTRcqnDcH31zbP2IBT3c9rtvcpIws1kr6yk8Djj/8+sPj1u/8SM/QVcHRgmZmWXFV7h9sGOwxBduTx4C9ZOvOYLD5/ewaE4x46jMzNrLTU/74H9/I6lNHHf4PK58tyeyNbODg2sU++C6Ncm0U//4ngkHBpiZzUpOFC0q182Jc/QhfRlGYmbWWU4ULXrlFd8A4JwfPyzjSMzMOsuJYh99+uKTsg7BzKyjnChacN8zybTePYXc2IRoZmYHCyeKFvzS/78TYEZNS21m1ilOFHtxwz2beGmwBMCjf9bs8d5mZrOTE8VefPTL944td3d52m8zO/g4UezFKcsXAnD9B16fcSRmZtlwomjiiRcGuPvp7fy3YxbRv+KQrMMxM8uEE0UTb/o/3wPguSbPojYzm+2cKFpw/Qfd7GRmBy8nihYcsaA36xDMzDLjRDGJZ9On1/2P04/JOBIzs2w5UUzi0ed3AXD+iUdkHImZWbacKCbx9LZBAFYunZNxJGZm2XKimMS2gVEkWDynO+tQzMwylWmikHSupEclrZd0+QT7l0u6RdI9ku6X9I5OxbZjcJT5PQXyOXXqlGZmM1JmiUJSHrgSOA84HrhY0vENh/0h8JWIOBm4CPi7TsV37Zpn2DFU6tTpzMxmrCxrFKcC6yNiQ0SMAtcBFzYcE8D8dHkB8GyngusteF4nMzOALB+ucBTwTN36RuC0hmM+DnxL0oeBOcA5nQktSRRvPd5PszMzm+md2RcDn4uIZcA7gC9IelnMki6VtFbS2q1bt075pBHBtoERlsx1R7aZWZaJYhNwdN36snRbvfcBXwGIiB8CPcCSxheKiKsioj8i+pcuXTrlwHYMlShVgiVzi1N+LTOzA12WiWINsErSSklFks7q1Q3HPA28BUDSj5MkiqlXGfZi664RAJbOc43CzCyzRBERZeAy4CbgYZLRTQ9K+oSkC9LDfgt4v6T7gGuBX4qIaHds29PRTov6XKMwM8uyM5uIuBG4sWHbx+qWHwLO6HRcI+mzsXuLHvlkZjbTO7MzMVKuANDd5bfHzMxXwgmMlJMahZ+RbWbmRDEh1yjMzPbwlXACtT6K7oLfHjMzXwknUGt6Kub99piZ+Uo4gbGmJ8/3ZGbmRDGRsaYn91GYmTlRTOQvb34MgC4/i8LMzImiGcmJwszMicLMzJpyojAzs6Yynetppjp8fg9vfNXUpys3M5sNXKOYwEi54pvtzMxSvhpOYKRc9dBYM7OUr4YTSBKFb7YzMwMnipcpV6pUquEahZlZylfDBmNTjLuPwswMcKJ4GT+LwsxsPCeKBn4WhZnZeL4aNtg5VAagr9u3mJiZgRPFywyOJoliXo8ThZkZOFG8TLkagGeONTOrcaJoUK4kiSLvRGFmBjSZ60nS3wIx2f6I+EhbIspYuZqMeir4MahmZkDzGsVa4C6gBzgFWJf+nAQUp+Pkks6V9Kik9ZIun+SYn5P0kKQHJX1pOs7bTK3pyTUKM7PEpDWKiLgGQNIHgTMjopyufwa4baonlpQHrgTeCmwE1khaHREP1R2zCvh94IyIeEnSoVM9795U0qanQs41CjMzaK2PYhEwv259brptqk4F1kfEhogYBa4DLmw45v3AlRHxEkBEbJmG8zZVa3pyjcLMLNHKGNBPAvdIugUQcBbw8Wk491HAM3XrG4HTGo55FYCkHwB54OMR8c3GF5J0KXApwPLly6cU1Niop7wThZkZ7CVRKHlo9LeBb7DnIv57EfFcuwNLdQGrgLOBZcCtkl4TEdvrD4qIq4CrAPr7+yftgG9FbdSTh8eamSWaJoqICEk3RsRrgK9N87k3AUfXrS9Lt9XbCNwRESXgCUmPkSSONdMcy5g991G4j8LMDFrro7hb0uvacO41wCpJKyUVgYuA1Q3H3EBSm0DSEpKmqA1tiGVMpdZH4aYnMzOgtT6K04B3S3oKGCDpp4iIOHEqJ46IsqTLgJtI+h+ujogHJX0CWBsRq9N9b5P0EFABficitk3lvHtTGhv15ERhZgatJYq3t+vkEXEjcGPDto/VLQfwm+lPR1R8H4WZ2Th7TRQR8RRAeg9DT9sjytieUU/uozAzgxb6KCRdIGkd8ATwfeBJklFQs1K5kvRReNSTmVmila/NfwqcDjwWESuBtwC3tzWqDHkKDzOz8VpJFKW0AzknKRcRtwD9bY4rM7X7KDwpoJlZopXO7O2S5gK3Al+UtIVk9NOsVBse6wqFmVmila/NFwKDwG8A3wQeB97ZzqCyVK4GhbxIbko3M7NWahQXAbdGxDrgmjbHk7lyNdw/YWZWp5VEsRz4B0krSZ5RcStwW0Tc29bIMlKuhKfvMDOrs9crYkT8cUS8GTie5DkUv0PyQKNZqVyteuZYM7M6e61RSPpD4AyS51DcA/w20/DgopmqXA3fQ2FmVqeVpqefBsrA10luuPthRIy0NaoMVdz0ZGY2TitNT6cA5wB3kjy29EeS/qPdgWWlVK26M9vMrE4rTU+vBn4CeCPJjXbPMIubnirVcB+FmVmdVh+FehvwaWBN+hChWct9FGZm47Uye+z5knqB5bM9SUAyKaD7KMzM9mhl9th3AveS3JWNpJMkNT6JbtZw05OZ2XitfHX+OHAqsB0gvdFuZRtjylSp4qYnM7N6rc4eu6NhW7QjmJmg4ik8zMzGaaUz+0FJvwDkJa0CPgL8Z3vDyk5yZ7b7KMzMalq5In4YOAEYAb4E7AQ+2s6gslR205OZ2TitjHoaBK5IfwCQtBx4uo1xZaZcDfpcozAzG9P0iijp9ZLeJenQdP1ESV8CftCR6DJQrlZdozAzqzNpopD0KeBq4GeAr0v6M+BbwB3Aqs6E13lDoxV6Cq5RmJnVNGt6+kng5IgYlrSIZOqOV0fEkx2JLCPDpSo9hXzWYZiZzRjNvjoPR8QwQES8BKyb7iQh6VxJj0paL+nyJsf9jKSQ1D+d559IpRoUfGe2mdmYZjWKYxvuwF5Zvx4RF0zlxJLywJUkM9JuBNZIWh0RDzUcNw/4dZImr7YrV4O878w2MxvTLFFc2LD+l9N87lOB9RGxAUDSdek5H2o47k+BPyd5sl7bVdyZbWY2zqSJIiK+3+ZzH0XS71GzETit/gBJpwBHR8TXJU2aKCRdClwKsHz58ikFVfad2WZm48zYxnhJOeCvgN/a27ERcVVE9EdE/9KlS6d03oqnGTczGyfLRLEJOLpufVm6rWYe8Grge5KeBE4HVre7QzupUczY/Glm1nEtXxEl9U3zudcAqyStlFQELgLqO8t3RMSSiFgRESuA24ELImLtNMcxTjIpYDvPYGZ2YGnleRRvkPQQ8Ei6/lpJfzfVE0dEGbgMuAl4GPhKRDwo6ROSpjSiagoxpYnCmcLMrKaV2WP/Gng76bf9iLhP0lnTcfKIuBG4sWHbxyY59uzpOGcz1XTydPdRmJnt0dJX54h4pmFTpQ2xZK5crQJ41JOZWZ1WahTPSHoDEJIKJDe/PdzesLJRSasUrlGYme3RSo3iA8CHSO572ASclK7POuU0UbhGYWa2RyvPo3gBeHcHYslcpeIahZlZo70mCkmfnmDzDmBtRHxt+kPKzliNwuNjzczGtHJF7CFpblqX/pxIcnPc+yT9TRtj6zj3UZiZvVwrndknAmdERAVA0t8DtwFnAj9qY2wdNzbqSU4UZmY1rdQoFgFz69bnAIekiWOkLVFlpOLObDOzl2mlRvEXwL2SvgcIOAv4X5LmAN9uY2wdN9b05OdRmJmN2WuNIiL+CXgDcAPwVeDMiPhsRAxEREeeEdEp19+1EYC7n3op40jMzGaOVof3DAObgZeAV07XFB4zzZonXwRg3ZbdGUdiZjZztDI89ldI7sZeBtxLMt33D4E3tze0zqvN9eS+bDOzPVqpUfw68DrgqYh4E3AysL2tUWWkGkmmyDlTmJmNaSVRDEfEMICk7oh4BPix9oaVjVqCKPqGOzOzMa1cETdKWkjSmX2zpK8BT7U3rGy8+7Tkedu/9qZXZhyJmdnM0cpcT/89Xfy4pFuABcA32xpVRq66dQMA83paGTVsZnZwaHpFlJQHHoyI4wAi4vsdiSojjzy3C0huFjEzs0TTpqf07utHJS3vUDwzgtyZbWY2ppU2lkXAg5LuBAZqGyMik+dad4Jn8DAz26OVRPFHbY9ihnGNwsxsj1Y6s78v6RhgVUR8W1IfkG9/aNlxjcLMbI+9Do+V9H7geuAf0k1HkQyVnbXk7mwzszGt3EfxIeAMYCdARKwDDm1nUFlzy5OZ2R6tJIqRiBitrUjqAqJ9IWUv57YnM7MxrSSK70v6A6BX0luBfwb+bTpOLulcSY9KWi/p8gn2/6akhyTdL+k7aV9J2zlNmJnt0UqiuBzYSvLY018FbgT+cKonTm/muxI4DzgeuFjS8Q2H3QP0R8SJJP0kfzHV87bCkwKame3RyvDYnwI+HxH/OM3nPhVYHxEbACRdB1wIPFQ7ICJuqTv+duAXpzmGCTlPmJnt0UqN4p3AY5K+IOn8tI9iOhwFPFO3vjHdNpn3Ad+YaIekSyWtlbR269atUw7MicLMbI9WHoX6y8ArSfomLgYel/TZdgdWT9IvAv3ApybaHxFXRUR/RPQvXbp06udzL4WZ2ZiWagcRUZL0DZLRTr0kzVG/MsVzbwKOrltflm4bR9I5wBXAGyNiZIrnbEneo57MzMa0csPdeZI+B6wDfgb4LHD4NJx7DbBK0kpJReAiYHXDuU8mudHvgojYMg3nbElfcVbfeG5mtk9aqVG8B/gy8KvT+Y0+IsqSLgNuIpkS5OqIeFDSJ4C1EbGapKlpLvDP6fxLT7dzMsL+YxbxxAsD9BScKMzMalqZ6+ni+nVJZwIXR8SHpnryiLiRZLht/baP1S2fM9Vz7ItiV46VS+Z08pRmZjNeS30UaRPQLwA/CzwB/Gs7g8pKhO+hMDNrNGmikPQqklFOFwMvkDQ/KSLe1KHYOq4a4duyzcwaNKtRPALcBpwfEesBJP1GR6LKSOA8YWbWqNmop58GNgO3SPpHSW9htl9H3fRkZvYykyaKiLghIi4CjgNuAT4KHCrp7yW9rVMBdlI1wndlm5k1aOXO7IGI+FJEvJPkprh7gN9re2QZCDx9h5lZo1bmehoTES+l02W8pV0BZSki3PRkZtZgnxLFbFed1Y9jMjPbP04UdZKmJ9cozMzqOVGkypUqj2/Zzc6hUtahmJnNKE4UqR1DJXaPlLn3me1Zh2JmNqM4UZiZWVNOFGZm1pQThZmZNeVEkfJoJzOziTlRmJlZU04UZmbWlBNFyg1PZmYTc6IwM7OmnCjMzKwpJ4qUBz2ZmU3MicLMzJpyojAzs6acKFLyuCczswllmigknSvpUUnrJV0+wf5uSV9O998haUXnozQzO7hlligk5YErgfOA44GLJR3fcNj7gJci4pXAXwN/3tkozcwsyxrFqcD6iNgQEaPAdcCFDcdcCFyTLl8PvEWelMnMrKOyTBRHAc/UrW9Mt014TESUgR3A4sYXknSppLWS1m7dunX/onH6MTOb0KzozI6IqyKiPyL6ly5dmnU4ZmazSpaJYhNwdN36snTbhMdI6gIWANs6Ep2ZmQHZJoo1wCpJKyUVgYuA1Q3HrAYuSZffBXw3IqIdwbjnw8xsYl1ZnTgiypIuA24C8sDVEfGgpE8AayNiNfBPwBckrQdeJEkmZmbWQZklCoCIuBG4sWHbx+qWh4Gf7XRcZma2x6zozJ4OtZannoLfEjOzer4qmplZU04UZmbWlBNFyjd8m5lNzInCzMyacqJo4OnGzczGc6JoELTlfj4zswOWE0XK9Qgzs4k5UTRw05OZ2XhOFA3c9GRmNp4TRcqjY83MJuZE0cBNT2Zm4zlRNHDTk5nZeE4UZmbWlBNFAzc9mZmN50TRwE1PZmbjOVGYmVlTThQN3PRkZjaeE4WZmTXlRGFmZk05UZiZWVNOFGZm1pQThZmZNeVEYWZmTWWSKCQdIulmSevSfxdNcMxJkn4o6UFJ90v6+SxiNTM72GVVo7gc+E5ErAK+k643GgTeExEnAOcCfyNpYQdjNDMzsksUFwLXpMvXAD/VeEBEPBYR69LlZ4EtwNKORWhmZkB2ieKwiNicLj8HHNbsYEmnAkXg8Un2XyppraS1W7dund5IzcwOcl3temFJ3wYOn2DXFfUrERGSJp2JT9IRwBeASyKiOtExEXEVcBVAf3//fs3qV5u6o7eY359fNzObtdqWKCLinMn2SXpe0hERsTlNBFsmOW4+8HXgioi4vU2hAkmCuPy843jr8U0rN2ZmB52smp5WA5eky5cAX2s8QFIR+Crw+Yi4vhNBfeCNr+AVS+d24lRmZgeMrBLFJ4G3SloHnJOuI6lf0mfTY34OOAv4JUn3pj8nZROumdnBSxGz60E9/f39sXbt2qzDMDM7oEi6KyL6J9rnO7PNzKwpJwozM2vKicLMzJpyojAzs6acKMzMrCknCjMza2rWDY+VtBV4agovsQR4YZrCORAcbOUFl/lgcbCVearlPSYiJpx4ddYliqmStHayscSz0cFWXnCZDxYHW5nbWV43PZmZWVNOFGZm1pQTxctdlXUAHXawlRdc5oPFwVbmtpXXfRRmZtaUaxRmZtaUE4WZmTXlRJGSdK6kRyWtl3R51vFMhaSrJW2R9EDdtkMk3SxpXfrvonS7JH06Lff9kk6p+51L0uPXSbpkonPNBJKOlnSLpIckPSjp19Pts7nMPZLulHRfWuY/SbevlHRHWrYvpw8AQ1J3ur4+3b+i7rV+P93+qKS3Z1Oi1knKS7pH0r+n67O6zJKelPSj9Jk8a9Ntnf1sR8RB/wPkgceBY4EicB9wfNZxTaE8ZwGnAA/UbfsL4PJ0+XLgz9PldwDfAAScDtyRbj8E2JD+uyhdXpR12SYp7xHAKenyPOAx4PhZXmYBc9PlAnBHWpavABel2z8DfDBd/jXgM+nyRcCX0+Xj0897N7Ay/TvIZ12+vZT9N4EvAf+ers/qMgNPAksatnX0s+0aReJUYH1EbIiIUeA64MKMY9pvEXEr8GLD5guBa9Lla4Cfqtv++UjcDixMn2P+duDmiHgxIl4CbgbObX/0+y4iNkfE3enyLuBh4Chmd5kjInanq4X0J4A3A7VHBzeWufZeXA+8RZLS7ddFxEhEPAGsJ/l7mJEkLQN+Evhsui5meZkn0dHPthNF4ijgmbr1jem22eSwiNicLj8HHJYuT1b2A/I9SZsXTib5hj2ry5w2wdwLbCH5w38c2B4R5fSQ+vjHypbu3wEs5gArM/A3wO8C1XR9MbO/zAF8S9Jdki5Nt3X0s921P1HbgS0iQtKsGxctaS7wL8BHI2Jn8uUxMRvLHBEV4CRJC4GvAsdlHFJbSTof2BIRd0k6O+t4OujMiNgk6VDgZkmP1O/sxGfbNYrEJuDouvVl6bbZ5Pm0Ckr675Z0+2RlP6DeE0kFkiTxxYj413TzrC5zTURsB24BXk/S1FD7Algf/1jZ0v0LgG0cWGU+A7hA0pMkzcNvBv4vs7vMRMSm9N8tJF8ITqXDn20nisQaYFU6eqJI0vG1OuOYpttqoDbS4RLga3Xb35OOljgd2JFWaW8C3iZpUTqi4m3pthknbXf+J+DhiPirul2zucxL05oEknqBt5L0zdwCvCs9rLHMtffiXcB3I+nlXA1clI4QWgmsAu7sTCn2TUT8fkQsi4gVJH+j342IdzOLyyxpjqR5tWWSz+QDdPqznXWP/kz5IRkt8BhJO+8VWcczxbJcC2wGSiRtke8jaZv9DrAO+DZwSHqsgCvTcv8I6K97nfeSdPStB34563I1Ke+ZJO249wP3pj/vmOVlPhG4Jy3zA8DH0u3Hklz01gP/DHSn23vS9fXp/mPrXuuK9L14FDgv67K1WP6z2TPqadaWOS3bfenPg7VrU6c/257Cw8zMmnLTk5mZNeVEYWZmTTlRmJlZU04UZmbWlBOFmZk15URh1oSk3avOcUQAAAHvSURBVOm/KyT9wjS/9h80rP/ndL6+2XRxojBrzQpgnxJF3d3CkxmXKCLiDfsYk1lHOFGYteaTwE+kzwT4jXRCvk9JWpPO+/+rAJLOlnSbpNXAQ+m2G9IJ3R6sTeom6ZNAb/p6X0y31WovSl/7gfQ5BD9f99rfk3S9pEckfVH1E1qZtYknBTRrzeXAb0fE+QDpBX9HRLxOUjfwA0nfSo89BXh1JFNYA7w3Il5Mp9pYI+lfIuJySZdFxEkTnOungZOA1wJL0t+5Nd13MnAC8CzwA5L5j/5j+otrtodrFGb7520kc+rcSzKl+WKSOYMA7qxLEgAfkXQfcDvJxGyraO5M4NqIqETE88D3gdfVvfbGiKiSTFWyYlpKY9aEaxRm+0fAhyNi3MRq6fTXAw3r5wCvj4hBSd8jmYNof43ULVfw37B1gGsUZq3ZRfKY1ZqbgA+m05sj6VXp7J6NFgAvpUniOJLHU9aUar/f4Dbg59N+kKUkj7adkbOb2sHB30bMWnM/UEmbkD5H8hyEFcDdaYfyVvY8jrLeN4EPSHqYZKbS2+v2XQXcL+nuSKbLrvkqybMl7iOZFfd3I+K5NNGYdZxnjzUzs6bc9GRmZk05UZiZWVNOFGZm1pQThZmZNeVEYWZmTTlRmJlZU04UZmbW1H8Bf4Pb8dOIuDMAAAAASUVORK5CYII=\n", "text/plain": [ "
" ] }, "metadata": { "needs_background": "light" }, "output_type": "display_data" }, { "name": "stdout", "output_type": "stream", "text": [ "\n", "\n", "Softmax agent, beta= 0.1\n", "maximum possible reward: 0.9851042878107023\n", "average reward: [0.91348264]\n", "cosine similarity 0.9992727823574249\n", "euclidian norm 0.008915931500017809\n" ] }, { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYoAAAEGCAYAAAB7DNKzAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAgAElEQVR4nO3deZxcdZnv8c9TVV29ZukknYTJQkcIYkAEbIIIMsjOqOC4jIAz4oiDOuLM6OhMBhzkMndBvbPoDNcRlSt6ZRBXogYRBQSUJQ2EJUDIAiGdBNIknfTetT33j3O6qW66K0W6q06l6/t+verV5/zO6TrPL69OPXV+57eYuyMiIjKRWNQBiIhIZVOiEBGRgpQoRESkICUKEREpSIlCREQKSkQdwFSbN2+et7a2Rh2GiMhB5eGHH37Z3VvGOzbtEkVrayvt7e1RhyEiclAxs60THVPTk4iIFKREISIiBSlRiIhIQUoUIiJSkBKFiIgUpEQhIiIFKVGIiEhB024chYhIOWRzwRINmVyOTNbJ5Jx0NsdAKksyEaOuJj5ynrvjgDskYsas+hpiMcPdSWVz9A9l6R3K0D2YJptzUpkcZgZAKpMj5042fP99A2n29qeJGcHvprLUxGPUJmLMa6rl3cctmvK6KlGISMVyd3IefBgPpnMMpbMMhK/BdPChPJjJMpjKK0tnGUxn6ewZoqOrn739abr6U8xuSJKMx4jFIGZG71CG3sEMC2bWMbM+QTYXXCsZjxGPGYm4MZjO0jOYoXsgTc9QhkzWGUhn6R5IM5TJHXC9EjEjHjMyOR9JOFPhTYtnKVGIyMGjP5XhqR3dvNw7xFAmRzbn9A1lGMrkqE3EiMdidHT109kzxN6BNN0DafpTWV7uHWJPX2rSH6JNtQkWN9fT3JDksJYm9g0E39YHMzlyDjNqE8yqr2FX9xA79w0QjxnukM7mSGeD69Yn48yoSzC7IcniOQ3UxIz6ZJyZdTXUJ+MYQUIZ/uCviceoT8bJZJ3+VAaAeMyImRELbhBIZZ1dPYPgwbHG2gQNyTiNyQQz6xPEY8HdQXAH4iQTMeI2fJ0Ys+prmFVfgwM1caMxmSCdyzGUyZGMl+ZpghKFyDTk7nQPZkiGH1weNl30DGboGcyQymaZ3ZBkdn0NOYfuweAb8p7eFH2pDP2pDNkc7Ng7AEAibmzp7GNPX4rOniGee7mPRNyoTcQY/izP5TxoInGnbyhLV3+K/S2gmYgZ85pqmd1Qw8z6GuY2JTliwQzmNNYEH5CxVz4kaxNBXepr4tTVvPKzriYor0vER37WJWMk47GR5pvprjYWpzYRL9n7K1GIlEgmm2Pjrl66+lKkwrbrnqEMqUyOhTPrmD+zloFUlv50lv6hLM+82M2L+wZJZ4MmjblNtXQPpJnTmCSddXqH0uzuTdE7lCFmRtDqDbkcbN87wEA6SyJmmMGevtTIt+JEzMiFTTiTUZsIvs3W1cR586HNeFjH4c/i4Ftz8M26PhlnwYw6jvqDmRwyuw7DSCaMhmSCxmSCoWyWbM6Z21hLMqE+NZVOiULkAHX1pdjW1c+2PQO81D3I9r0DPL2zmz19KV7sHqQ/lSX1GtqxzaC5IUljbZy+oSz9qQyGMZDO0lSboLE2aJ6oq4nTkIwTG27LMDihtZmmugSZrOMOzY1JZjfU0DeUGWm+SSZiNNUmSMSMupp48PB0IE0iPpwAYsTMWDirjoZkgpjBIbPqyboTN2NeU5LElDVt1EzR+0g5KFFI1RpuKhlum05lc3QPpukeyLCtq59NL/Wy4aUecu7Mrk8ykM7SN5QhmYjx5PZ9PPNiz6j3q03EeP3CGSyZ00BbazMNyQRHLpzBotn1JOIx6muC9m53WL9jHzXxGA3JOHVhc8ri5npm1L36A9Tdq6YJRSqTEoVMW3v6UnR09fNS9xC9Q2mWzWuibyjDum17eei5Pdy/eTepbI6auI0004w1f0Yt6WzQ46Y+GSebc2riMZbPb+Jz57yew+c3saS5gYWz6mhuqCn6A33p3Iai66EkIVFTogj1DmW45mfrefexi3jr4fOiDkeK5B50V6xNxHn0hS6+v3Ybz77Uw5aX++gZzEz4e0vm1PNnJx060n7fn8rS3JBkwcxamuoSLG5uYNm8RuY11ZaxNiKVSYkiNJTOckt7B0cvmqVEUSH29qd4YMseduwdoKk2wbMv9XDfppdZOqeBTZ29zKiroWNPP7v7UqN+77ilsznnqIUsm9dI69xG/mB2HXv703QPpunoGuDCE5YwVwlApGhKFFJxXtw3yHcfeJ4bf7+V3qFX3xVs3d1PImYsml3Paa+fz9ymJO3P7+HiEw/l7KMWMHOcdn4ROXBKFBIJd2fTrl46e4Z45sUebvjdczSFA6Dat3aRc+fsFQu45KRWkmG3zM6eIY4/tHlkagQRKQ8lCimJzp4hOnuG6B5Mk4gZ/aks63d0c/eGXXT2DrFz7yAD6eyrfm/+jFouPWUZf3rioa964Lt8wYxyhS8ieZQoZNI2vtTDTQ+9wBMd+9jTn2JLZ1/B8085fB5vPWwuKw6ZRevcBpa1NLJgRh2ZnFMTN/XyEakwShRVpnswzY8f7uCW9g6e2tnNkjn1NNXW8P43Lyabcx7fvo+fPbaD017fwgmtc3j0hb3cs7GThmScXM5ZuWwOPYMZVvzBTO7fvJt4zHhqZzcGNCYT9AxlqIkbHzl5GUcvmsXcxuTIgK35M+s4rKVxwkSQjClBiFQiJYpprKsvxSMvdPG9B1/gzmd2ETNeNY3Dtj0DwADX/PypUeV3b+jk7g2dI/upTI5DZtXx66d3AfDgc3sAeNvyeXzq9OV8+K2tzGlMlrQ+IhINJYpp6Mnt+7h13XZufmgbPXm9hnIORy6cwd+d+3retHg2O/cNMpTJsW8gRV0iztK5DSTjMdbv6Gb5giYe3trFikNmjno2kM7mWL0uuONQF1OR6qBEcRBzd/b2p2luTDKYzvK/b99AJuf810MvMJTJ8cZFs7js1Ndx5hsWUJ+Mk8rkRk3ANtEH/fyZdQAsbn716OGaeIz3vnlxaSokIhUp0kRhZucCXwHiwDfd/doxx5cCNwKzw3NWufuasgdaYQbTWd549e0j006cd/RCbnvyxZHjcxqT/L+PvpkTWueM+j3N0ikiByKyRGFmceA64CygA1hrZqvdPb+x/PPALe7+NTNbAawBWssebAXIZHN85Tcb+fc7N73q2HCS+Ptzj+SMN8yndW6jkoKITJko7yhWApvcfQuAmd0MXADkJwoHZobbs4AdpQ5qfwutRGFX9yCf+N4jPLy1a6TsA21LuPKdb2Dtc3voHcrw9iPna0SyiJRElIliEbAtb78DOHHMOVcDvzKzTwGNwJnjvZGZXQZcBrB06dIDCqYS++73DmU4/z/uGxmX0JiMc/8VZ1ATC1b0AjjjDQuiDFFEqkClt09cBHzb3RcDfwR818xeFbO7X+/ube7e1tLSUvYgS+ULt64fSRJnvmEBv1t1+shavSIi5RLlHcV2YEne/uKwLN+lwLkA7n6/mdUB84BdZYkwIv2pDFfdup4fPdLBJ99+GJ8758ioQxKRKhZlolgLLDezZQQJ4kLg4jHnvACcAXzbzN4A1AGdTGPfuu85/ikc/LZ0TgOfOn15xBGJSLWLLFG4e8bMLgduJ+j6eoO7rzeza4B2d18N/C3wDTP7NMGD7Q+7V+Lj5qmx5omdI0nif/zx0Vy8cmlFPjsRkeoS6TiKcEzEmjFlV+VtPwWcXO64ovDjRzr4zC2PAXD3Z0+jdV5jxBGJiAQ0MrsCvOOr97J+RzdHLpzBDz5+EjPUzVVEKkil93qa9v75VxtYv6MbgO9/TElCRCqPEkWEbl23fWSk9YNXnMGseiUJEak8anoqs+17Bzj52jtHld3ysZNYEE7EJyJSaZQoyugDX79/ZB2HYTd99ERWLpszwW+IiERPiWKMUvW+fWF3/6gk8Z7jF42sAiciUsmUKEKlHK3wRMc+3vUf9wHwtQ8eT10yzmlHtGiMhIgcFJQoymA4ScQMznvjIRFHIyLy2qjXU4n94vGdI9tb/tc7IoxEROTAKFGU0O83v8wnb3oEgJ9/6pSIoxEROTBKFCX0xdueAeCcoxboobWIHLSUKEokk83xWMc+AL7+Z20RRyMicuCUKEpkTbiO9TUXHBVxJCIik6NeTyXwxV8+w9fu3kwiZvzpiYdGHY6IyKTojmKKuTtfu3szAJmcE4tprISIHNx0RzGFjrn6droHMyP73710ZYTRiIhMDSWKKZSfJB684gxN9Cci04KansY40JmeNu3qHdm+49OnKkmIyLShRBGa7LRL/3HnRgD+74dPYPmCGVMQkYhIZVCimAKZbI6frtsBwKlHtEQcjYjI1FKimALDA+sA4urlJCLTjBLFFHh4a7DOxJ1/+4cRRyIiMvWUKKbA+h3dLJpdz+tamqIORURkyilRTJK7c+u6HaSyuahDEREpCSWKSdrcGXSLfdvyeRFHIiJSGkoUk7Slsw+AS05qjTYQEZESiTRRmNm5ZrbBzDaZ2aoJzvkTM3vKzNab2U3ljnF/vnXfcwAcOrch4khEREojsik8zCwOXAecBXQAa81stbs/lXfOcuAfgJPdvcvM5kcT7fgG01kefC7o8TS7IRlxNCIipRHlHcVKYJO7b3H3FHAzcMGYc/4CuM7duwDcfVeZYyzolvZtACTjasETkekrykkBFwHb8vY7gBPHnHMEgJn9DogDV7v7L8e+kZldBlwGsHTp0kkF5UVM9pTK5LjiJ0/ww4c7AHjsC2dP6poiIpWs0mePTQDLgdOAxcA9ZvZGd9+bf5K7Xw9cD9DW1nZA8/oZxY+oPuLzt43ar0/GD+SSIiIHhSjbTLYDS/L2F4dl+TqA1e6edvfngGcJEkdkegbTo/bfc9yiiCIRESmPKBPFWmC5mS0zsyRwIbB6zDk/JbibwMzmETRFbSlnkGN994Gto/Y/c/YREUUiIlIekTU9uXvGzC4Hbid4/nCDu683s2uAdndfHR4728yeArLA59x9d1QxwysPrn92+Sl09g6yuFndYkVkepswUZjZv1NgHR93/6vJXtzd1wBrxpRdlbftwGfCV0XY3NnL7IYajl40E7NZUYcjIlJyhZqe2oGHgTrgeGBj+DoWqNpBA49t28cR82dgk13pSETkIDHhHYW73whgZp8ATnH3TLj/n8C95Qmvsgxlsjy1szvqMEREyqqYh9nNwMy8/aawrOrs3DsIwJ+f3BptICIiZVTMw+xrgUfN7C7AgFOBq0sZVKXasXcAgLNWLIg4EhGR8imYKCxoiP81cBuvjJr+e3d/sdSBVaIXu4M7ioUz6yKORESkfAomCnd3M1vj7m8Ebi1TTJEqNKx7T18KgLlNteUJRkSkAhTzjOIRMzuh5JFErYhOTLv7UtTEjZl1lT7ziYjI1CnmE+9E4INmthXoI/hIdXc/pqSRVaDbnthJOuvqGisiVaWYRHFOyaM4SDy/uz/qEEREym6/icLdtwKEiwbpKa6ISJXZ7zMKMzvfzDYCzwG/BZ4n6AU1LW18qYfWVb/gtid2jirf2x88yNYiRSJSbYr51Psn4C3As+6+DDgDeKCkUUXo5rXBWkpfun3DqPLdYY+nL7+/6h7NiEiVKyZRpMMZW2NmFnP3u4C2EscVuUwuN2q/K0wUzVobW0SqTDEPs/eaWRNwD/A9M9tF0PtpWhuTJ7hrQ7Bc95xGJQoRqS7F3FFcAPQDnwZ+CWwG3lXKoCrB2DuK6+7aDChRiEj1KeaO4kLgHnffCNxY4ngqRjY3frmankSk2hSTKJYCXzezZQRrVNwD3Ovu60oaWcRe7h0im3PiMaM/lRkpr0/GI4xKRKT89tv05O5fcPfTgRUE61B8jmBBo2nv6tXrAdjbn444EhGR6BQzjuLzZnYb8CvgcOCzwOJSB1Zu483K8aNHOgB4YU8wIvtL71XXWBGpPsU0Pb0HyAC/IBhwd7+7D5U0qgoRD7PHhdcHw0YOm98YZTgiIpEopunpeOBM4CHgLOAJM7uv1IFVgrF3GYtmN0QTiIhIhPZ7R2FmRwNvA/6QYKDdNqpkzezuwcyo/fkztA6FiFSfYpdCvRf4KrDW3avqye5AKgvAYS2NxGKaXlxEqk8xs8e+08zqgaXVliQA2rfuAWBz57QfjC4iMq5iej29C1hHMCobMzvWzFaXOrBKkc0Fi6M21WpVOxGpTsVM4XE1sBLYCxAOtFtWwpgqSiYbJIpvfGjaz4MoIjKuYmeP3TemzKfi4mZ2rpltMLNNZraqwHnvNTM3s7J/Wn/0O+0AzG3S1B0iUp2KSRTrzexiIG5my83s34HfT/bCZhYHrgPOIxj1fZGZrRjnvBnAXwMPTvaak6GmJxGpVsUkik8BRwFDwE1AN/A3U3DtlcAmd9/i7ingZoKZasf6J+CLwOAUXPOANdUpUYhIdSpmwF2/u1/p7ieEryuB+VNw7UUEYzKGdYRlI8zseGCJu/+i0BuZ2WVm1m5m7Z2dnVMQ2qs1JpUoRKQ6FUwUZnaSmb3PzOaH+8eY2U3A70odmJnFgH8B/nZ/57r79e7e5u5tLS0tB3a9vO2/Ov1w/uVP3jTqeFxjKESkSk2YKMzsy8ANwHuBX5jZfyeYGPBBYPkUXHs7sCRvf3FYNmwGcDRwt5k9T7Bu9+pyPNBumVHL0jmarkNEBAoPuHsHcJy7D5pZM0Ez0dHu/vwUXXstsDxc52I7wQJJFw8fDHtazRveN7O7gc+6e/sUXX9iZrz50OaSX0ZE5GBQqOlp0N0HAdy9C9g4hUkCd88AlwO3A08Dt7j7ejO7xszOn6rrHCgz40MnHRp1GCIikSt0R/G6MSOwl+Xvu/ukP8zdfQ2wZkzZVROce9pkr/da/eVph/Od+7fy6TOPKPelRUQqRqFEMbar6j+XMpBKtHBWHY/+41nMqq+JOhQRkchMmCjc/bflDKRSNTdqRLaIVLdiBtyJiEgVU6IQEZGCik4UZqaBBSIiVaiY9SjeamZPAc+E+28ys/9T8shERKQiFHNH8a/AOcBuAHd/DDi1lEGJiEjlKKrpyd23jSnKliCWSJlpLicRkfEUMyXqNjN7K+BmVkOwNsTTpQ1LREQqRTF3FB8HPkkwBfh24NhwX0REqsB+7yjc/WXgg2WIRUREKtB+E4WZfXWc4n1Au7vfOvUhiYhIJSmm6amOoLlpY/g6hmDtiEvN7N9KGJuIiFSAYh5mHwOc7O5ZADP7GnAvcArwRAljExGRClDMHUUz0JS33wjMCRPHUEmiEhGRilHMHcWXgHXhCnNGMNjuf5pZI/DrEsYmIiIVoJheT98yszXAyrDoCnffEW5/rmSRiYhIRSh2UsBBYCfQBRxuZprCQ0SkShTTPfajBKOxFwPrgLcA9wOnlza06Jx39MKoQxARqRjF3FH8NXACsNXd3w4cB+wtaVQRyJ/paV5TbWRxiIhUmmISxaC7DwKYWa27PwO8vrRhiYhIpSim11OHmc0GfgrcYWZdwNbShiUiIpWimF5PfxxuXm1mdwGzgF+WNCoREakYBROFmcWB9e5+JIC7/7YsUYmISMUo+IwiHH29wcyWlikeERGpMMU8o2gG1pvZQ0DfcKG7n1+yqEREpGIUkyj+sVQXN7Nzga8AceCb7n7tmOOfAT4KZIBO4CPurgfpIiJltN/useFzieeBmnB7LfDIZC8cPv+4DjgPWAFcZGYrxpz2KNDm7scAPySYd0pERMpov4nCzP6C4EP662HRIoKuspO1Etjk7lvcPQXcDFyQf4K73+Xu/eHuAwSjw0VEpIyKGXD3SeBkoBvA3TcC86fg2ouAbXn7HWHZRC4FbhvvgJldZmbtZtbe2dk5BaGJiMiwYhLFUPiNHwAzSwBeupBezcz+FGgDvjzecXe/3t3b3L2tpaWlnKGJiEx7xSSK35rZFUC9mZ0F/AD42RRcezuwJG9/cVg2ipmdCVwJnO/uJVsoyWz/54iIVKNiEsUqgh5HTwAfA9YAn5+Ca68FlpvZMjNLAhcCq/NPMLPjCJ6NnO/uu6bgmiIi8hoV0z323cB33P0bU3lhd8+Y2eXA7QTdY29w9/Vmdg3Q7u6rCZqamoAfWPCV/wWN3xARKa9iEsW7gH81s3uA7wO/dPfMVFzc3dcQ3KHkl12Vt33mVFxHREQOXDHjKP4cOJzg2cRFwGYz+2apAxMRkcpQzB0F7p42s9sIejvVEzRHfbSUgYmISGUoZsDdeWb2bWAj8F7gm4DWChURqRLF3FF8iODZxMdK2T1VREQqUzELF12Uv29mpwAXufsnSxaViIhUjKKeUYTjGS4G3g88B/y4lEGJiEjlmDBRmNkRBL2cLgJeJmh+Mnd/e5liExGRClDojuIZ4F7gne6+CcDMPl2WqEREpGIU6vX0HmAncJeZfcPMzgCm7YxINn2rJiIyKRMmCnf/qbtfCBwJ3AX8DTDfzL5mZmeXK0AREYlWMSOz+9z9Jnd/F8EMr48Cf1/yyEREpCIUM3vsCHfvCtd+OKNUAYmISGV5TYlCRESqjxKFiIgUpEQhIiIFKVGIiEhBShQiIlKQEoWIiBSkRCEiIgUpUYiISEFKFCHTVE8iIuNSohARkYKUKEREpCAlChERKUiJQkREClKiEBGRgiJNFGZ2rpltMLNNZrZqnOO1Zvb98PiDZtZa/ihFRKpbZInCzOLAdcB5wArgIjNbMea0S4Eudz8c+Ffgi+WNUkREoryjWAlscvct7p4CbgYuGHPOBcCN4fYPgTPMNOJBRKScokwUi4BtefsdYdm457h7BtgHzB37RmZ2mZm1m1l7Z2dnicIVEalO0+Jhdrg8a5u7t7W0tEQdjojItBJlotgOLMnbXxyWjXuOmSWAWcDuskQnIiJAtIliLbDczJaZWRK4EFg95pzVwCXh9vuAO93dyxijiEjVS0R1YXfPmNnlwO1AHLjB3deb2TVAu7uvBr4FfNfMNgF7CJKJiIiUUWSJAsDd1wBrxpRdlbc9CLy/3HGJiMgrpsXDbBERKR0lChERKUiJQkREClKiEBGRgpQoRESkICWKMTSTlIjIaEoUIiJSkBLFGLqhEBEZTYlCREQKUqII6dmEiMj4lChERKQgJQoRESlIiWIMrbQqIjKaEoWIiBSkRDGG7idEREZTohARkYKUKMbQIwoRkdGUKEREpCAlChERKUiJYgzT42wRkVGUKEREpCAlChERKUiJIuQedQQiIpVJiWIsPaIQERlFiUJERApSohARkYIiSRRmNsfM7jCzjeHP5nHOOdbM7jez9Wb2uJl9oCyxleMiIiIHkajuKFYBv3H35cBvwv2x+oEPuftRwLnAv5nZ7FIFpIfZIiLjiypRXADcGG7fCLx77Anu/qy7bwy3dwC7gJZSB6a5nkRERosqUSxw953h9ovAgkInm9lKIAlsnuD4ZWbWbmbtnZ2dBxSQo1sKEZHxJEr1xmb2a2DhOIeuzN9xdzezCT+lzewQ4LvAJe6eG+8cd78euB6gra1tUp/4msJDRGS0kiUKdz9zomNm9pKZHeLuO8NEsGuC82YCvwCudPcHShRqcK0wQdQn46W8jIjIQSeqpqfVwCXh9iXArWNPMLMk8BPgO+7+w1IHVJ+Ms+q8I/nBx08q9aVERA4qUSWKa4GzzGwjcGa4j5m1mdk3w3P+BDgV+LCZrQtfx5YyqI//4WEc1tJUykuIiBx0zKdZv9C2tjZvb2+POgwRkYOKmT3s7m3jHdPIbBERKUiJQkREClKiEBGRgpQoRESkICUKEREpSIlCREQKUqIQEZGCpt04CjPrBLZO4i3mAS9PUTgHg2qrL6jO1aLa6jzZ+h7q7uPO0D3tEsVkmVn7RINOpqNqqy+oztWi2upcyvqq6UlERApSohARkYKUKF7t+qgDKLNqqy+oztWi2upcsvrqGYWIiBSkOwoRESlIiUJERApSogiZ2blmtsHMNpnZqqjjmQwzu8HMdpnZk3llc8zsDjPbGP5sDsvNzL4a1vtxMzs+73cuCc/faGaXjHetSmBmS8zsLjN7yszWm9lfh+XTuc51ZvaQmT0W1vm/heXLzOzBsG7fD1eKxMxqw/1N4fHWvPf6h7B8g5mdE02NimdmcTN71Mx+Hu5P6zqb2fNm9kS4eFt7WFbev213r/oXEAc2A68DksBjwIqo45pEfU4FjgeezCv7ErAq3F4FfDHc/iPgNsCAtwAPhuVzgC3hz+Zwuznquk1Q30OA48PtGcCzwIppXmcDmsLtGuDBsC63ABeG5f8JfCLc/kvgP8PtC4Hvh9srwr/3WmBZ+P8gHnX99lP3zwA3AT8P96d1nYHngXljysr6t607isBKYJO7b3H3FHAzcEHEMR0wd78H2DOm+ALgxnD7RuDdeeXf8cADwGwzOwQ4B7jD3fe4exdwB3Bu6aN/7dx9p7s/Em73AE8Di5jedXZ37w13a8KXA6cDw2vMj63z8L/FD4EzzMzC8pvdfcjdnwM2Efx/qEhmthh4B/DNcN+Y5nWeQFn/tpUoAouAbXn7HWHZdLLA3XeG2y8CC8Ltiep+UP6bhM0LxxF8w57WdQ6bYNYBuwj+428G9rp7JjwlP/6RuoXH9wFzOcjqDPwb8HdALtyfy/SvswO/MrOHzeyysKysf9uJA4laDm7u7mY27fpFm1kT8CPgb9y9O/jyGJiOdXb3LHCsmc0GfgIcGXFIJWVm7wR2ufvDZnZa1PGU0Snuvt3M5gN3mNkz+QfL8betO4rAdmBJ3v7isGw6eSm8BSX8uSssn6juB9W/iZnVECSJ77n7j8PiaV3nYe6+F7gLOImgqWH4C2B+/CN1C4/PAnZzcNX5ZOB8M3ueoHn4dOArTO864+7bw5+7CL4QrKTMf9tKFIG1wPKw90SS4MHX6ohjmmqrgeGeDpcAt+aVfyjsLfEWYF94S3s7cLaZNYc9Ks4OyypO2O78LeBpd/+XvEPTuc4t4Z0EZlYPnEXwbOYu4H3haWPrPPxv8T7gTg+ecq4GLgx7CC0DlgMPlacWr427/4O7L3b3VoL/o3e6+weZxnU2s0YzmzG8TfA3+STl/tuO+ol+pbwIegs8SwY+Mp8AAAKgSURBVNDOe2XU8UyyLv8F7ATSBG2RlxK0zf4G2Aj8GpgTnmvAdWG9nwDa8t7nIwQP+jYBfx51vQrU9xSCdtzHgXXh64+meZ2PAR4N6/wkcFVY/jqCD71NwA+A2rC8LtzfFB5/Xd57XRn+W2wAzou6bkXW/zRe6fU0besc1u2x8LV++LOp3H/bmsJDREQKUtOTiIgUpEQhIiIFKVGIiEhBShQiIlKQEoWIiBSkRCFSgJn1hj9bzeziKX7vK8bs/34q319kqihRiBSnFXhNiSJvtPBERiUKd3/ra4xJpCyUKESKcy3wtnBNgE+HE/J92czWhvP+fwzAzE4zs3vNbDXwVFj203BCt/XDk7qZ2bVAffh+3wvLhu9eLHzvJ8N1CD6Q9953m9kPzewZM/ue5U9oJVIimhRQpDirgM+6+zsBwg/8fe5+gpnVAr8zs1+F5x4PHO3BFNYAH3H3PeFUG2vN7EfuvsrMLnf3Y8e51nuAY4E3AfPC37knPHYccBSwA/gdwfxH9019dUVeoTsKkQNzNsGcOusIpjSfSzBnEMBDeUkC4K/M7DHgAYKJ2ZZT2CnAf7l71t1fAn4LnJD33h3uniOYqqR1SmojUoDuKEQOjAGfcvdRE6uF01/3jdk/EzjJ3fvN7G6COYgO1FDedhb9H5Yy0B2FSHF6CJZZHXY78IlwenPM7Ihwds+xZgFdYZI4kmB5ymHp4d8f417gA+FzkBaCpW0rcnZTqQ76NiJSnMeBbNiE9G2CdRBagUfCB8qdvLIcZb5fAh83s6cJZip9IO/Y9cDjZvaIB9NlD/sJwdoSjxHMivt37v5imGhEyk6zx4qISEFqehIRkYKUKEREpCAlChERKUiJQkREClKiEBGRgpQoRESkICUKEREp6P8DRIuN1tzJDCgAAAAASUVORK5CYII=\n", "text/plain": [ "
" ] }, "metadata": { "needs_background": "light" }, "output_type": "display_data" }, { "name": "stdout", "output_type": "stream", "text": [ "\n", "\n", "UCB1 agent\n", "maximum possible reward: 0.9851042878107023\n", "average reward: [0.89258379]\n", "cosine similarity 0.0\n", "euclidian norm 0.1414213562373095\n" ] }, { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYIAAAEGCAYAAABo25JHAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAgAElEQVR4nO3deXxddZ3/8dcn+94kTVpKt7RSKEsp1BZa2WEERJZRXEBnxAVh3HAZFdAZRZ2fgyujDgNTGQb9/QRxt6PI9pNVKLZACy0UWkohLaXpljR7cnM/88c5SW/aJL2hOfcmOe/n43EfOVvO/XzzuDmfe77f7/l+zd0REZH4ysl2ACIikl1KBCIiMadEICISc0oEIiIxp0QgIhJzedkOYLhqamq8rq4u22GIiIwpTz755A53rx1o35hLBHV1daxcuTLbYYiIjClm9spg+1Q1JCISc0oEIiIxp0QgIhJzSgQiIjGnRCAiEnNKBCIiMadEICISc7FJBImeJL9YWU8yqWG3RURSjbkHyt6oWx59mev/tA53572LZmQ7HBGRUSM2dwS7WrsAaGzrznIkIiKjS2wSgYiIDCw2iUBTcoqIDCw2iUBERAYWm0RgZtkOQURkVIpNIlDVkIjIwGKTCHrpxkBEpL/YJQLdGIiI9Be7RCAiIv3FLhGoakhEpL/YJQIREelPiUBEJOaUCEREYk6JQEQk5mKTCNRtVERkYLFJBL0MdRsSEUkVWSIws1vNrMHM1gyy//1m9oyZPWtmj5nZ/KhiSeXo1kBEJFWUdwS3AecOsf9l4DR3nwd8A1gaYSwiIjKIyKaqdPeHzaxuiP2PpawuB6ZFFUsqVQ2JiPQ3WtoIPgL8KdtBiIjEUdYnrzezMwgSwclDHHMFcAXAjBmaeF5EZCRl9Y7AzI4FbgEucvedgx3n7kvdfaG7L6ytrc1cgCIiMZC1RGBmM4DfAH/v7i9mKw4RkbiLrGrIzO4ATgdqzGwz8FUgH8Ddbwa+AkwE/iOcRjLh7gujikedRkVEBhZlr6FLD7D/cuDyqN5fRETSM1p6DUVOnUZFRAYWm0SgqiERkYHFJhH00gxlIiL9xS4RaBRSEZH+YpcIRESkv9glAlUNiYj0F7tEICIi/SkRiIjEnBKBiEjMKRGIiMRcbBKBuo2KiAwsNolAREQGpkQgIhJzSgQiIjGnRCAiEnNKBCIiMadEICISc0oEIiIxp0QgIhJzsUkErjnKREQGFJtEICIiA4tNIjBNXy8iMqDYJIIFMysBmDKhOMuRiIiMLrFJBFMmFAFQXpSX5UhEREaX2CSCXmoyFhHpL0aJQG0EIiIDiVEiEBGRgSgRiIjEXOwSgWuqMhGRfiJLBGZ2q5k1mNmaQfabmf3QzDaY2TNmtiCqWIL3i/LsIiJjV5R3BLcB5w6x/23AnPB1BXBThLGIiMggIksE7v4wsGuIQy4CfuqB5UClmU2JKh4RERlYNtsIpgL1Keubw237MbMrzGylma3cvn17RoITEYmLMdFY7O5L3X2huy+sra09uHONUEwiIuPFoOMtmNmPGOK66e5XHeR7bwGmp6xPC7dFQm3FIiIDG+qOYCXwJFAELADWh6/jgIIReO9lwAfC3kOLgSZ33zoC5xURkWEY9I7A3X8CYGYfA05290S4fjPwyIFObGZ3AKcDNWa2GfgqkB+e+2bgLuA8YAPQBnzoYAoiIiJvTDpDcVYBFeztAVQWbhuSu196gP0OfCKN9x9ZaiQQEeknnURwPfC0mT1AUNV+KnBdlEFFwfREmYjIgIZMBBZcPe8H/gScGG6+2t1fjzowERHJjCETgbu7md3l7vOA32coJhERyaB0niN4yswWRR5JhrgaCURE+kmnjeBE4P1m9grQStBO4O5+bKSRjTC1EIiIDCydRHBO5FGIiEjWHDARuPsrAGY2ieDhMhERGUcO2EZgZhea2XrgZeAhYBNBL6IxSfPSiIj0l05j8TeAxcCL7j4LOAtYHmlUEdBjBCIiA0snEXS7+04gx8xy3P0BYGHEcYmISIak01jcaGZlwMPAz8ysgaD3kIiIjAPp3BFcRDAo3GeBu4GXgAuiDCpKaiMQEekvnTuCS4CH3X098JOI44mM6UkCEZEBpZMIZgD/aWazCOYoeBh4xN1XRRqZiEjMJXqSdCSS5Bjk5hh5OTnk5oz8l9p0niP4KoCZFQMfBb4A/BuQO+LRiIiMM8mk09yZoKO7h/auHl7d1cbuti72tHfTNOAr0bevpTPR71xXnjaba9925IjHeMBEYGb/BJxEMA/B08DnSWNiGhGRsSqZdFq7EjhQWpDHnvZuOhI9JHqcpvZudrZ20djWxa7WLna3drGrrYvdrd3sau0CIJFMsqOli50tnTR3JoZsmyzOz2VCcT4TivOpKM5jamURR04p79tWUpBL0qEn6Rw/vTKS8qZTNfROIAH8keCBssfdvTOSaDJAbcUi8eHubG/p5PWmDna0dLKjuYu2rgStXT00tXf3ffPe0xH+bE/Q1N5Nc0c3yfBikWP0LQ/EDKpKCqgqyae6tICepJOXm8MxUydQXRJczMuL8ikqyKUkP5fp1SVUlxb0XfgL87JfuZJO1dACM6sguCt4K7DUzBrc/eTIoxtBeqBMZHRr60qwbU9w0a4qzQegvauHwrxcWrsStHQmaOlI0NoZLLd29tDS2U1LZw+tncH25vBnZyJJrhlbGtv3q17pVZSfQ0VR7zfxfCaVF3FYbR4Vvd/Oi/KD6pzuHmrKCinKDy7Y1aUFTCwroLq0gOqSAiqK8yOpt8+kdKqGjgFOAU4jeJCsHlUNicgQOhM97GrtorkjQf2uNprau9m0s43XGtvpTCTp7O6hqydJVyLJ9uZONu9up727Z9jvU5yfS2lhHmWFwc/SwjwOqSiiIC+HRNJZ8qaJzJxYwrSqEiaWFVBbVhgelzsqvomPFulOVfkI8ENghbt3RxuSiIxmHd09NLZ1s7O1k12tXazb2sz6hmZe3dVGT9J5cVsLTe37XyZyDCaVF1FckEthXg4FeTnk5+Ywq6aUU+bUMrGsgEMqiphcUcSuti5yzSjKz6GjO8mE4nxKC3MpCy/2pYV5lBbkkpebzqNQciDpVA2dH/YYmjEekoDriTIRIOiaWL+7na2N7f3qyZvau2nrCqpbNm5vZUdLJ2ZGVyLJtj0dJAaoMK8pK2DmxFJyc4zz5h3C1MpiKksKKC3MpW5iKROK8zm0srivekVGl3Sqhi4AvgsUALPM7Djg6+5+YdTBiUj6mtq7WV3fyI6WTjbtaGVrUwcOHFpZTP2uNjZubwkONGPL7jZ2tXYN2AiamxM8fllVWsDM6hLmTask0ZOkKD+XyRVFlBXmUl1aSHVp0Ag695ByJpYVZrKoMsLSqRq6DjgBeBDA3VeFD5eJSBb0JJ3nt+5hVX0ja1/bw+tN7dTvbmdDQ0vfMTkGteWFdPc4u1q7qCjKY/70SnLMSLpz5JGTqS0vZHpVCdOrS4KuimEPl9KC4Fu7qYdFbKSTCLrdvWmfD4XqV0RGkLuzo6WLzbvbeHVXG6vrm2hs7+JNtWV0dvewYXsLtWWFrHltD8+9tqevYbW8MI9p1SXMrC7hwvmHctz0SiZXFFFXU9LXGNqVSJKXY+SM8Z4tEp10EsFaM3sfkGtmc4CrgMeiDSs6ymCSSe5OT9Jp7exhe0sHPcmgR83qzU28vL2Vl7a30NqZoH53G9v27H08pzAvh5KCXH7z1JZ+5ztueiXvXTSd42dUsmBGFVMriw94gS/IU4OqDC2dRPAp4MtAJ3A7cC/BZDVjiu5yZaS1dCZ4qaGFTTtb2dPeTWNbN6/uaiPHjF1tXdTvaqN+VxutXQN3iyzOz2VaVTHFBbksmT2RedMqqSjK45ipE5hdW0p+Tg7bmjuoKilQI6tEKp1eQ20EieDLvdvMbAbwaoRxiYw6Hd09rKpvZPnGnSzfuJOnXm2kK5Hsd0xteSGd3T3UlBcyu6aMxbMnUhX2nqktL6QzkaSkIJfjplcytbL4gPXwUyYUR1kkEeAAicDMlgBTCYahbjCzY4FrCB4wm56B+ESypjPRw9Ov7n/hN4NjDp3AZUtmsqiumpkTS6ksye/r4y4y1gz6qTWz7wDnA6uAq83sHuBy4F+BD2cmvJGnxwiko7uHna1d/HldAxu2NdPRnaSiOI/JFUWs2dLElsZ2kg4v72hlV2sXZnD0oRV8YPFMFs+eyKJZ1Uwozs92MURGzFBfX94OHO/uHWZWRTC0xDHuvindk5vZucAPCIasvsXdr99n/wyCyW4qw2Oucfe7hleENGPRxDSx1NTWzZrXmtjT3k1DcyfrXm/mD6tfozkcfyY/1+ju2fvtYFJ5IQ3NnRxSUcTJh9Vw3rwpLJk9kQkluvDL+DVUIuhw9w4Ad99tZuuHmQRygRsJBqrbDKwws2Xu/lzKYf8E/MLdbzKzo4C7gLphlkGkT3dPkuUbd7K7rZs/PvMa9z/fQE/KU1NlhXmcOXcSM6pLqCkr4OI3T6MoP5dV9Y1MmVCUVr29yHgzVCKYbWbLUtZnpa6n8WTxCcAGd98IYGY/J5j/ODUROFARLk8AXks3cBEIumc+v7WZO1e8ypbGDu5/flvfvomlBXzk5Fksnl1NXk4ONWWFHDapbMDulIvqqjMZtsioMlQiuGif9e8N89xTCaqTem0GTtznmOuAe83sU0Ap8DcDncjMrgCuAJgxY8Yww5Dxwt1xh7buHp6pb+Sh9du5c0U9jW3BEFhlhXksmT2RyRWFXH7KbA6fXK4+9CJpGDQRuPtDGXj/S4Hb3P17YQ+l/2tmx7h7vz557r4UWAqwcOHCg2zuVWvxWJFMOtuaO/j1k5v55ZObea2xncK83H7jy58yp4YzjpjEm2dWMWdyGSUF6rUjMlxR/tdsoX8X02nhtlQfAc4FcPfHzawIqAEaRjoYVfuODY1tXfz08Ve48YENdKb00T9sUhmHTy7nyCkVzKopZUZ1CSfMqmZyRVEWoxUZH6JMBCuAOeEAdVuAS4D37XPMq8BZwG1mdiRQBGyPMCYZhZJJ5/ert/Cvd62joTkYZqEgL4dFdVXMn1bJuxZOY+4hFQc4i4i8UWknAjMrCZ8yTou7J8zsk8A9BF1Db3X3tWb2dWCluy8D/hH4sZl9lqDO5oOuCQPGveaOblo6Eyx9eCPPbm5i5Su7+/adNXcS/3j2ERx1qC78IpmSznwEbwFuAcqAGWY2H7jS3T9+oN8Nnwm4a59tX0lZfo5gLuSMUZrJntebOrjtsU3c/NBL/bafOKuav1s8k7fPm6IRMkWyIJ07ghuAc4BlAO6+2sxOjTSqCKiNIPOee20PV/38aXIMmjsSbG3qAGBGdQnHTa/k75fMZMGMqjE/8bfIWJdW1ZC71+/zkM3wZ5mWWGho7uD+5xq45dGNbNze2rfdDBbVVfGJMw7j9CMmZTFCEdlXOomgPqwecjPLBz4NPB9tWDLatXUl+P2q17jhvhcpLcyjYU8Hs2pLWbNlDxA09r7rzdO44tTZzKoppbUzQWVJQZajFpGBpJMI/oFgvKCpBL1/7gU+EWVQUVITwRvXlUjyzbue57bHNvXfEfb0WbNlD0tmT+Szbz2cRXVV/YZqUBIQGb3SmY9gB/D+DMQSKQ0698bs6ejm2OvuBYKx9reHF/2plcVc9paZnHv0FKZXF/c1wquxV2TsSafX0A8H2NxE0AX09yMfkmTbjpZOEj3Opp2tXLJ0ed/25o5urjxtNl88Z+5+DbxqjBcZu9KpGioC5gK/DNcvBl4G5pvZGe7+maiCk8z75cp6vvCrZ/pt++Y75nH+/CmUF+ZpZE6RcSidRHAscJK79wCY2U3AI8DJwLMRxhYJPUcwsBdeb+Y9//k4Te3BAG5zJpUxq6aUfz7/KKZXl2Q5OhGJUjqJoIrgYbKmcL0UqHb3HjPrjCyyEaYvsoO7e83r/MP/exKA9504g8+ffQTVpWrcFYmLdBLBt4FVZvYgYMCpwDfNrBS4P8LYJELJpPOrJzfzxV/vrQZ64POnM6umNItRiUg2pNNr6L/M7C6CiWYAvuTuvRPIfCGyyCQSLZ0JvnvPC/zsiVf6pmicUJzPrz/2FiUBkZhKd9C5DmArQcPxYWZ2mLs/HF1Y0fGYPkmwfONO7l7zer9nAE6oq+amv1vAxLLC7AUmIlmXTvfRywmeJp4GrAIWA48DZ0Yb2siKYxOBu3P1r5/hFys399v+9YuO5gNL6rITlIiMOuncEXwaWAQsd/czzGwu8M1ow5KD0ZVIcvlPV/Lwi3undphdU8qZcyfxubMP1yxeItJPOleEDnfvMDPMrNDd15nZEZFHNsJ668P//c8bOP/YQ7McTXTqd7Vxwb8/2jeP75lzJ/GFc47g8MnlGuVTRAaUTiLYbGaVwO+A+8xsN/BKtGGNvPbuYJ7bda83ZzmS6Kzf1sxbbwiabqpLC7j706cwSVM5isgBpNNr6B3h4nVm9gAwAbg70qgiMN4fJGvtTPQlgXOOnsx33z2f8qL8LEclImPBkInAzHKBte4+F8DdH8pIVBEYr3ngxW3NnH3D3g5c5807hP94/5uzGJGIjDVDJoLw6eEXzGyGu7+aqaCiMNbvCF7e0coLr+/hpoc2smhmFYdMKOJf/th/Wojywjy+dfGxWYpQRMaqdIeYWGtmfwX6ppxy9wsjiyoCPkYzgbtz/Dfu62v8BVhd39jvmAvmH8oN75lPXm5OpsMTkXEgnUTwz5FHIQPq6O5h4b/cT0tn0NBdmJfDB5bM5MePvAzAHz51MkcfWqERQUXkoKTTWPyQmc0E5rj7/WZWAuRGH9rISo7BG4Kr7ni6Lwn87PITOemwGgC+/PajshmWiIwz6TxZ/FHgCqAaeBPBlJU3A2dFG9rIGgtVQ9+6ex03PfgSNWWFXH3uEdz73DYAHr/2TKZMKM5ydCIyXqVTNfQJggHnngBw9/VmNinSqCIwWtNA6hDQvXa0dPZNDrP82rM4ZIKeBRCR6KTTutjp7l29K2aWx+i9rg4qOQrvCB5dv2O/JHDZkplMKA76/1997lwlARGJXDp3BA+Z2ZeAYjN7K/Bx4H+iDWvkjYY2gld2tnLeDx6htauH6985j9v/urdHbmlBLlee9iauOmsOX7voGFo7E5QWakwgEYleOleaa4CPEExLeSVwF3BLlEFFIdt3BMmkc9p3Huxbv+Y3wSyf5YV5PPu1c/Y7XklARDIlnavN3wI/dfcfRx1MlLLdWPzc1j0Dbv/vDy3KcCQiIv2lkwguAG4ws4eBO4G73T0RbVgjryKL4+60dib4zJ2rKMjL4Ylrz6JK8wGLyChywMZid/8QcBjwS+BS4CUzS6tqyMzODYeo2GBm1wxyzHvM7DkzW2tmtw8n+OFYWFcNwEdOnhXVWwyoo7uHo796DxsaWphdU6okICKjTloV0e7ebWZ/IugtVExQXXT5UL8TDlh3I/BWYDOwwsyWuftzKcfMAa4FTnL33VF3Sy3IzSE/w8Mw/O7pLX3LN7z3uIy+t4hIOg54VTSzt5nZbcB64GKChuJD0jj3CcAGd98Ydj/9OXDRPsd8FLjR3XcDuHvDMGIfPsvsnMW/fXpzX6Pw2q+dw5FTKjL23iIi6UrnjuADBG0DV7p75zDOPRWoT1nfDJy4zzGHA5jZXwiGrbjO3feb68DMriB4upkZM2YMI4R9zgMZewLitr+8zHX/E9z8HD+jUr2ARGTUSmesoUtT183sZOBSd//ECL3/HOB0YBrwsJnNc/d+w2u6+1JgKcDChQvf8KXcLDN54KzvPchL24OBWi9eMI3vvltDQ4vI6JVWhbmZHW9m3zGzTcA3gHVp/NoWYHrK+rRwW6rNwDJ373b3l4EXCRJDJAzLSDfS3iQA8O13HavRQUVkVBs0EZjZ4Wb2VTNbB/wIeBUwdz/D3X+UxrlXAHPMbJaZFQCXAMv2OeZ3BHcDmFkNQVXRxuEXIz1m0U9Q05nowSwYKmLT9W/XhPEiMuoNVTW0DngEON/dNwCY2WfTPbG7J8zsk8A9BPX/t7r7WjP7OrDS3ZeF+842s+eAHuAL7r7zDZblgIzoqoY2NDTzN9/fO2XkolnVEb2TiMjIGioRvJPgW/wDZnY3Qa+fYX29dfe7CIakSN32lZRlBz4XviJnZpHdEaQmAYDjpldG80YiIiNs0ETg7r8DfmdmpQTdPj8DTDKzm4Dfuvu9GYpxxAR3BOllgh0tnbyys43q0gIu/8kKjpxSwYdOmsWbZ1b1HbNmSxM/fmQjzR39H7SuLS9kWlXJSIYuIhKZdHoNtQK3A7ebWRXwbuBqYMwlAobRRvCemx9n4469jb4vbW/lD89s5ScfPoHTDq/lt09v5rN3ru73O18+70jesWAqNWWFIxm1iEikhvWYrbvvdvel7j6mZifrNZx6rdQkkOqyW//Klsb2/ZLA2UdN5vJTZikJiMiYE6unnII2goNvJDjp+j/3W7/jo4tZ8qaJB31eEZFsiFkieGO9hi6Yfyhnzq3lovlTmf2lvW3f71wwlY+ffhiHTSobuSBFRDIsXomA9NoIXt3Z1rdcVpjHjy49vm99amUxWxrbAfj+ezSInIiMfZkdijPLzCytXkMrNu3qW9534ph/fec8AH5wiZKAiIwPsbsjSEdOmB4f+PzpzKop7bfv1MNrWfeNcynKzx3Z4EREsiRWdwSQXtVQb4+gskFGDFUSEJHxJFaJYLiNxROKsze9pYhIpsQqEcCBh5jYuL2lb7kgL2Z/HhGJpVhd6SyNmWn+8MzWjMQiIjJaxCsRcOA2gj+GiWD2Po3EIiLjVbwSQRpjDb2wrRmA2z+6OAMRiYhkX7wSAUM/R5A6/MQhE4oyEZKISNbFKxEMckfQk3Qu+NGjPPTi9swHJSKSZfFKBAzcVPzHZ7fy7JYmPvjfKwD47w8uGuAoEZHxKV6JYJAZyp56ZXe/9dJBHiQTERmPYpUIYOAZym57bFO/9erSggxFIyKSfbFKBC2dCVo7Ewc8TsNKi0icxCoRNLV3c8/abUMeo0nnRSRuYlkZvr25k9ry/aeU/M+/fzNnHzU5CxGJiGRPrO4Iei36P/f3LSeTjhlcddYczjn6EMyGM7OxiMjYF8tEkGpLYzvuUFEUy5sjERElglO+/QCAHiYTkdiKfSLoVZCrP4WIxFOsrn7Hz+jfI+h7977Qt3zj+xdkOhwRkVEhVolgXz/684a+ZU0/KSJxFatEkNofqCc5nEkrRUTGr0gTgZmda2YvmNkGM7tmiOMuNjM3s4VRxpOqqb07U28lIjKqRZYIzCwXuBF4G3AUcKmZHTXAceXAp4Enooql11OvNvYtP/hCQ9/yo1efEfVbi4iMWlHeEZwAbHD3je7eBfwcuGiA474BfAvoiDCW/XzuF6sBqCkrZGplcSbfWkRkVIkyEUwF6lPWN4fb+pjZAmC6u/9xqBOZ2RVmttLMVm7fPrL9/b918Tw9TSwisZa1xmIzywG+D/zjgY5196XuvtDdF9bW1o5oHIV56i0kIvEWZSLYAkxPWZ8WbutVDhwDPGhmm4DFwLJMNhgDFOTFquOUiMh+orwKrgDmmNksMysALgGW9e509yZ3r3H3OnevA5YDF7r7yghj2k+hEoGIxFxkV0F3TwCfBO4Bngd+4e5rzezrZnZhVO87XLojEJG4i3TITXe/C7hrn21fGeTY06OMZTC6IxCRuIv9VTA50Gz2IiIxEvtEMKO6NNshiIhkVewTQX6uniEQkXiLfSLQw2QiEnexTASahEZEZK9YXhGPOrQi2yGIiIwasUwEq+qDUUjrJpZkORIRkeyLZSK446OLAbj7M6dmORIRkeyL9IGy0WrJmyay6fq3ZzsMEZFRIZZ3BCIispcSgYhIzCkRiIjEnBKBiEjMKRGIiMScEoGISMwpEYiIxJwSgYhIzCkRiIjEnBKBiEjMKRGIiMRcrMYauvWDC+lKJLMdhojIqBKrRHDm3MnZDkFEZNRR1ZCISMwpEYiIxJwSgYhIzCkRiIjEnBKBiEjMKRGIiMScEoGISMwpEYiIxJy5e7ZjGBYz2w688gZ/vQbYMYLhjAUqczzErcxxKy8cfJlnunvtQDvGXCI4GGa20t0XZjuOTFKZ4yFuZY5beSHaMqtqSEQk5pQIRERiLm6JYGm2A8gClTke4lbmuJUXIixzrNoIRERkf3G7IxARkX0oEYiIxFxsEoGZnWtmL5jZBjO7JtvxHAwzu9XMGsxsTcq2ajO7z8zWhz+rwu1mZj8My/2MmS1I+Z3LwuPXm9ll2ShLOsxsupk9YGbPmdlaM/t0uH08l7nIzP5qZqvDMn8t3D7LzJ4Iy3anmRWE2wvD9Q3h/rqUc10bbn/BzM7JTonSY2a5Zva0mf0hXB/v5d1kZs+a2SozWxluy/zn2t3H/QvIBV4CZgMFwGrgqGzHdRDlORVYAKxJ2fZt4Jpw+RrgW+HyecCfAAMWA0+E26uBjeHPqnC5KttlG6S8U4AF4XI58CJw1DgvswFl4XI+8ERYll8Al4TbbwY+Fi5/HLg5XL4EuDNcPir8vBcCs8L/g9xsl2+Icn8OuB34Q7g+3su7CajZZ1vGP9dxuSM4Adjg7hvdvQv4OXBRlmN6w9z9YWDXPpsvAn4SLv8E+NuU7T/1wHKg0symAOcA97n7LnffDdwHnBt99MPn7lvd/alwuRl4HpjK+C6zu3tLuJofvhw4E/hVuH3fMvf+LX4FnGVmFm7/ubt3uvvLwAaC/4dRx8ymAW8HbgnXjXFc3iFk/HMdl0QwFahPWd8cbhtPJrv71nD5daB3gubByj4m/yZhFcDxBN+Qx3WZw2qSVUADwT/3S0CjuyfCQ1Lj7ytbuL8JmMjYKvO/AV8EkuH6RMZ3eSFI7vea2ZNmdkW4LeOf61hNXh8X7u5mNu76BZtZGfBr4DPuvif4AhgYj2V29x7gODOrBH4LzM1ySJExs/OBBnd/0sxOz3Y8GXSyu28xs0nAfWa2LnVnpj7Xcbkj2AJMT1mfFm4bT7aFt4mEPxvC7YOVfUz9TcwsnyAJ/MzdfxNuHtdl7uXujcADwBKC6oDeL3Cp8feVLdw/AdjJ2CnzScCFZraJoOr2TMxx9HMAAANFSURBVOAHjN/yAuDuW8KfDQTJ/gSy8LmOSyJYAcwJeyAUEDQuLctyTCNtGdDbW+Ay4Pcp2z8Q9jhYDDSFt533AGebWVXYK+HscNuoE9b9/hfwvLt/P2XXeC5zbXgngJkVA28laBt5AHhXeNi+Ze79W7wL+LMHLYnLgEvCXjazgDnAXzNTivS5+7XuPs3d6wj+P//s7u9nnJYXwMxKzay8d5ng87iGbHyus91qnqkXQYv7iwT1rF/OdjwHWZY7gK1AN0F94EcI6kf/P7AeuB+oDo814Maw3M8CC1PO82GCxrQNwIeyXa4hynsyQV3qM8Cq8HXeOC/zscDTYZnXAF8Jt88muLBtAH4JFIbbi8L1DeH+2Snn+nL4t3gBeFu2y5ZG2U9nb6+hcVvesGyrw9fa3utSNj7XGmJCRCTm4lI1JCIig1AiEBGJOSUCEZGYUyIQEYk5JQIRkZhTIpDYMrOW8Gedmb1vhM/9pX3WHxvJ84uMJCUCEagDhpUIUp52HUy/RODubxlmTCIZo0QgAtcDp4Rjwn82HOztO2a2Ihz3/UoAMzvdzB4xs2XAc+G234UDhq3tHTTMzK4HisPz/Szc1nv3YeG514Tj0L835dwPmtmvzGydmf3MUgdTEomQBp0TCcZ8/7y7nw8QXtCb3H2RmRUCfzGze8NjFwDHeDDEMcCH3X1XOAzECjP7tbtfY2afdPfjBnivdwLHAfOBmvB3Hg73HQ8cDbwG/IVg/J1HR764Iv3pjkBkf2cTjOmyimC464kEY9YA/DUlCQBcZWargeUEA3/NYWgnA3e4e4+7bwMeAhalnHuzuycJhtGoG5HSiByA7ghE9mfAp9y938Bd4fDIrfus/w2wxN3bzOxBgjFw3qjOlOUe9P8pGaI7AhFoJpgCs9c9wMfCoa8xs8PD0SH3NQHYHSaBuQTTB/bq7v39fTwCvDdsh6glmHZ0VI6OKfGhbxwiwQifPWEVz20E4+DXAU+FDbbb2TtdYKq7gX8ws+cJRrpcnrJvKfCMmT3lwXDKvX5LMK/AaoIRVb/o7q+HiUQkKzT6qIhIzKlqSEQk5pQIRERiTolARCTmlAhERGJOiUBEJOaUCEREYk6JQEQk5v4Xp8BVdb9iPfQAAAAASUVORK5CYII=\n", "text/plain": [ "
" ] }, "metadata": { "needs_background": "light" }, "output_type": "display_data" }, { "name": "stdout", "output_type": "stream", "text": [ "\n", "\n" ] } ], "source": [ "test(agents, 5000)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "\n", "### Experimental stuff:\n", "\n", "Below are a few agents I wrote for fun." ] }, { "cell_type": "code", "execution_count": 12, "metadata": {}, "outputs": [], "source": [ "\n", "class softmax_with_exponentiation(agent):\n", "\n", " def __init__(self, arms, play_once=1, beta=1, exp=1):\n", " super().__init__(arms, play_once)\n", " self.beta = beta\n", " self.exp = exp\n", "\n", " def policy(self):\n", " temp = np.exp(self.expectations/self.beta)\n", " ans = temp / np.sum(temp, axis=0)\n", " ans = ans**self.exp\n", " ans /= np.sum(ans, axis=0)\n", " return ans\n", "\n", "\n", "class softmax_with_reccurence(agent):\n", "\n", " def __init__(self, arms, play_once=1, beta=1):\n", " super().__init__(arms, play_once)\n", " self.old_policy = np.ones_like(self.expectations)/self.l\n", " self.beta = beta\n", "\n", " def policy(self):\n", " temp = np.exp(self.expectations/self.beta)\n", " new_policy = temp / np.sum(temp, axis=0)\n", "\n", " result = np.multiply(new_policy, self.old_policy)\n", " result /= np.sum(result, axis=0)\n", " self.old_policy = result\n", "\n", " return result\n", "\n", "\n", "class greedy_with_reccurence(agent):\n", " # alpha = number < 1; will sum over a number of observations and will keep\n", " # osiclating.\n", " # alpha = N will allow the algo to converge to an arm, greedy doesn't\n", " # really need this, kind of always give one answer.\n", "\n", " def __init__(self, arms, play_once=1, alpha=1):\n", " super().__init__(arms, play_once)\n", " self.old_policy = np.ones_like(self.expectations)\n", " self.alpha = alpha\n", "\n", " def policy(self):\n", " new_policy = np.zeros_like(self.expectations)\n", " new_policy[np.argmax(self.expectations)] = 1\n", "\n", " new_policy = (1-self.alpha)*new_policy + self.alpha*self.old_policy\n", "\n", " new_policy /= np.sum(new_policy, axis=0)\n", " self.old_policy = new_policy\n", "\n", " return new_policy\n", "\n", "# class magic(agent):\n", "# def __init__(self, arms, play_once=1, exp=1):\n", "# super().__init__(arms, play_once)\n", "# self.old_policy = np.ones_like(self.expectations)/self.l\n", "# self.exp = exp\n", "#\n", "# def policy(self):\n", "# new_policy = f(old_policy, g(expectations))\n" ] } ], "metadata": { "kernelspec": { "display_name": "Python 3", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.7.3" } }, "nbformat": 4, "nbformat_minor": 2 }