{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Dynamic programming"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "import matplotlib.pyplot as plt"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The goal of this exercise is to find the optimal policy for the recycling robot.\n",
    " \n",
    "In this problem, a recycling robot has to search for empty cans to collect (each can defines a \"reward\" given to the robot). It can also decide to stay where it is to save its battery and wait that somebody brings it a can (which gives less cans in average than actively searching for them). \n",
    "\n",
    "The robot has two battery levels, *high* and *low*. \n",
    "\n",
    "* In the *high* level, the robot can either search or wait. \n",
    "\n",
    "* In the *low* state, three actions are possible: search, wait and recharge. \n",
    "\n",
    "State-action transitions are probabilistic, i.e. they bring the robot in different states based on different probabilities $\\alpha$ and $\\beta$.\n",
    "\n",
    "This problem defines a finite MDP, with two states *high* and *low* corresponding to the battery level. The actions *search* and *wait* are possible in the *high* and *low* states, while the action *recharge* is only possible in the *low* state.\n",
    "\n",
    "$$\n",
    "\\begin{aligned}\n",
    "    \\mathcal{S} &=& \\{ \\text{high}, \\text{low} \\} \\\\\n",
    "    \\mathcal{A}(\\text{high} ) &=& \\{ \\text{search}, \\text{wait} \\} \\\\\n",
    "    \\mathcal{A}(\\text{low} ) &=& \\{ \\text{search}, \\text{wait}, \\text{recharge} \\}\n",
    "\\end{aligned}\n",
    "$$\n",
    "\n",
    "The action *search* brings on average a reward of $\\mathcal{R}^\\text{search}$, the action *wait* a reward of $\\mathcal{R}^\\text{wait}$, the action *recharge* brings no reward, but allows to get in the *high* state.\n",
    "\n",
    "Note that if the robot decides to search in the *low* state, there is a probability $1 - \\beta$ that it totally empties its battery, requiring human intervention. This is punished with a negative reward of -3.\n",
    "\n",
    "The transition and reward probabilities of each transition is defined in the following table, completely defining a MDP.\n",
    "\n",
    "| s             |  s'           |  a           | p(s' / s, a)      |  r(s, a, s')                 |\n",
    "|:-------------:|:-------------:|:------------:|:-----------------:|:-----------------------------:|\n",
    "| high          | high          | search       | $\\alpha$          | $\\mathcal{R}^\\text{search}$   |\n",
    "| high          | low           | search       | $1 - \\alpha$      | $\\mathcal{R}^\\text{search}$   |\n",
    "| low           | high          | search       | $1 - \\beta$       | $-3$                          |\n",
    "| low           | low           | search       | $\\beta$           | $\\mathcal{R}^\\text{search}$   |\n",
    "| high          | high          | wait         | $1$               | $\\mathcal{R}^\\text{wait}$     |\n",
    "| high          | low           | wait         | $0$               | $\\mathcal{R}^\\text{wait}$     |\n",
    "| low           | high          | wait         | $0$               | $\\mathcal{R}^\\text{wait}$     |\n",
    "| low           | low           | wait         | $1$               | $\\mathcal{R}^\\text{wait}$     |\n",
    "| low           | high          | recharge     | $1$               | $0$                           |\n",
    "| low           | low           | recharge     | $0$               | $0$                           |\n",
    "\n",
    "The goal of this exercise is to find the optimal policy $\\pi^*$ of the robot, i.e to find for each state the action that should be performed systematically in order to gather the maximum of reward on the long term. \n",
    "\n",
    "We will apply here two **dynamic programming** methods, policy iteration and value iteration, to solve the Bellman equations.\n",
    "\n",
    "The Bellman equation for the state function is:\n",
    "\n",
    "$$V^{\\pi} (s)  = \\sum_{a \\in \\mathcal{A}(s)} \\pi(s, a) \\, \\sum_{s' \\in \\mathcal{S}} p(s' | s, a) \\, [ r(s, a, s') + \\gamma \\, V^{\\pi} (s') ]$$\n",
    "\n",
    "**Q:** On paper, adapt the Bellman equation to the problem. First, for every state $s$ and possible action $a$, find the optimal value of the action with the form:\n",
    "\n",
    "$$Q^{\\pi} (s, a) = f( V^\\pi (\\text{high}), V^\\pi (\\text{low}), \\alpha, \\beta, \\gamma, \\mathcal{R}^{\\text{search}}, \\mathcal{R}^{\\text{wait}} )$$\n",
    "\n",
    "Deduce the Bellman equation for the two states $V^\\pi (\\text{high})$ and $V^\\pi (\\text{low})$."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**A:**\n",
    "\n",
    "$$Q^\\pi(\\text{high}, \\text{search}) = \\alpha \\, (\\mathcal{R }^\\text{search} + \\gamma \\, V^\\pi(\\text{high} )) + (1- \\alpha)\\, (\\mathcal{R}^\\text{search} + \\gamma \\, V^\\pi(\\text{low}))$$\n",
    "\n",
    "$$Q^\\pi(\\text{high}, \\text{wait}) = \\mathcal{R}^\\text{wait} + \\gamma \\, V^\\pi(\\text{high})$$\n",
    "\n",
    "$$Q^\\pi(\\text{low}, \\text{search}) = \\beta * (\\mathcal{R }^\\text{search} + \\gamma \\,  V^\\pi(\\text{low})) + (1- \\beta) \\, (-3 + \\gamma \\, V^\\pi(\\text{high}))$$\n",
    "\n",
    "$$Q^\\pi(\\text{low}, \\text{wait}) = \\mathcal{R}^\\text{wait} + \\gamma \\, V^\\pi(\\text{low})$$\n",
    "\n",
    "$$Q^\\pi(\\text{low}, \\text{recharge}) = \\gamma \\, V^\\pi(\\text{high})$$\n",
    "\n",
    "$$V^\\pi(\\text{high}) = \\pi(\\text{high}, \\text{search}) \\, Q^\\pi(\\text{high}, \\text{search}) +  \\pi(\\text{high}, \\text{wait}) \\, Q^\\pi(\\text{high}, \\text{wait})$$\n",
    "\n",
    "$$V^\\pi(\\text{low}) = \\pi(\\text{low}, \\text{search}) \\, Q^\\pi(\\text{low}, \\text{search}) +  \\pi(\\text{low}, \\text{wait}) \\, Q^\\pi(\\text{low}, \\text{wait}) + \\pi(\\text{low}, \\text{recharge}) \\,  Q^\\pi(\\text{low}, \\text{recharge})$$\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Policy Iteration\n",
    "\n",
    "Now that we have the Bellman equations for the two states high and low, we can solve them using **iterative policy evaluation** for a fixed policy $\\pi$. \n",
    "\n",
    "### Iterative policy evaluation\n",
    "\n",
    "Let's start by setting the parameters of the MDP. In the rest of the exercise, you will modify these parameters to investigate how it changes the optimal policy."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Transition probabilities\n",
    "alpha = 0.3\n",
    "beta = 0.2\n",
    "\n",
    "# Discount parameter\n",
    "gamma = 0.7\n",
    "\n",
    "# Expected rewards\n",
    "r_search = 6.0\n",
    "r_wait = 2.0"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "There are many ways to represent states and actions in a MDP. The suggestion for this exercise is to use dictionaries here the keys are the actions' name and the vaues are indices:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "nb_states = 2\n",
    "nb_actions = 3\n",
    "\n",
    "s = {'high': 0, 'low': 1}\n",
    "a = {'search': 0, 'wait': 1, 'recharge': 2}"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Using dictionaries, you can access numpy arrays with `s['high']` or `a['recharge']` instead of 0 and 2, what will make the code readable.\n",
    "\n",
    "The next step is to initialize numpy arrays where we will store the V and Q values. `V` will have only two elements for high and low, while `Q` will be a 2x3 matrix with one element for each state-action pair. Notice that (high, recharge) is not a possible action, so this element will not be be updated."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "V = np.zeros(nb_states)\n",
    "Q = np.zeros((nb_states, nb_actions))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You can then access the individual values with `V[s['high']]` or `Q[s['low'], a['wait']]`.\n",
    "\n",
    "We can now evaluate a policy $\\pi$. In dynamic programming, the policies are deterministic, as we want to estimate the optimal policy.\n",
    "\n",
    "To implement the policy, we just need to assign the index of an action to each state, i.e. $\\pi(s)$. The following cell creates an initial policy $\\pi$ where the agent **searches** in both states high and low. We here make sure that the array contains integers (0, 1 or 2), but that is not even necessary."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "pi = np.array([a['search'], a['search']], dtype=int)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Q:** Evaluate this policy using iterative policy evaluation.\n",
    "\n",
    "We would normally only need to update the V-value of the two states using:\n",
    "\n",
    "$$\n",
    "     V (s) \\leftarrow \\sum_{a \\in \\mathcal{A}(s)} \\pi(s, a) \\, \\sum_{s' \\in \\mathcal{S}} p(s' | s, a) \\, [ r(s, a, s') + \\gamma \\, V (s') ] \\quad \\forall s \\in \\mathcal{S}\n",
    "$$\n",
    "\n",
    "The code will be more readable if you first update the Q-values of the 5 state-action pairs:\n",
    "\n",
    "$$\n",
    "     Q (s, a) \\leftarrow  \\sum_{s' \\in \\mathcal{S}} p(s' | s, a) \\, [ r(s, a, s') + \\gamma \\, V (s') ] \\quad \\forall s \\in \\mathcal{S}\n",
    "$$\n",
    "\n",
    "and only then update the two V-values:\n",
    "\n",
    "$$\n",
    "     V (s) \\leftarrow \\sum_{a \\in \\mathcal{A}(s)} \\pi(s, a) \\, Q(s, a)\n",
    "$$\n",
    "\n",
    "These updates should normally be applied until the V-values converge. For simplicity, we could decide to simply apply 50 updates or so, and hope that it is enough.\n",
    "\n",
    "Record the V-value of the two states after each update and plot them. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[[11.28888873  9.90222206  0.        ]\n",
      " [ 5.9555554   6.16888873  7.90222206]]\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAzYAAAH5CAYAAABTbqsJAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjYuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/av/WaAAAACXBIWXMAAA9hAAAPYQGoP6dpAAA/NElEQVR4nO3deXxU9b3/8fdMlslCEpJASAIBgiL7oqAICsSqtIBel3vV1lbRVq2VWim3VZBetV4l2mu91tLaor2Utm5tVdTWWvlZARVQQJFN2ZcIhLBkgySTZOb8/jjJkEAICZmZ75zM6/noeZx1zvkEjmHe/Z7v97gsy7IEAAAAAA7mNl0AAAAAAHQUwQYAAACA4xFsAAAAADgewQYAAACA4xFsAAAAADgewQYAAACA4xFsAAAAADherOkCTuT3+7Vv3z6lpKTI5XKZLgcAAACAIZZlqbKyUrm5uXK7W2+Tibhgs2/fPuXl5ZkuAwAAAECEKCoqUq9evVo9JuKCTUpKiiS7+NTUVMPVAAAAADCloqJCeXl5gYzQmogLNo2Pn6WmphJsAAAAALSpiwqDBwAAAABwPIINAAAAAMcj2AAAAABwvIjrY9NWPp9PdXV1pstwlLi4OMXExJguAwAAAAg6xwUby7JUXFyssrIy06U4UteuXZWdnc07ggAAANCpOC7YNIaarKwsJSUl8QW9jSzLUlVVlUpKSiRJOTk5hisCAAAAgsdRwcbn8wVCTWZmpulyHCcxMVGSVFJSoqysLB5LAwAAQKfhqMEDGvvUJCUlGa7EuRr/7OifBAAAgM7EUcGmEY+fnTn+7AAAANAZOTLYAAAAAEBTBJswKSgo0IwZM0653+VyadGiRW0+35IlS+RyuRgdDgAAAJDDBg/ozPbv36/09HTTZQAAAACORLCJENnZ2aZLAAAAAByLR9HCyO/3695771VGRoays7P10EMPBfad+Cja8uXLNXLkSCUkJGj06NFatGiRXC6X1q5d2+yca9as0ejRo5WUlKRx48Zp8+bN4flhAAAAgAji+BYby7JUXeczcu3EuJh2jTK2cOFCzZw5Ux999JFWrFihW265RRdddJEuv/zyZsdVVlbqyiuv1JQpU/TCCy9o9+7dp+yfM2fOHP385z9X9+7ddeedd+rb3/62Pvzww478WAAAAIDjOD7YVNf5NPiBfxq59qaHv6qk+Lb/EQ4fPlwPPvigJKl///6aN2+e3n333ZOCzfPPPy+Xy6Vnn31WCQkJGjx4sPbu3avbb7/9pHM++uijmjhxoiRp1qxZmjp1qmpqapSQkNCBnwwAAABwFh5FC6Phw4c3W8/JyVFJSclJx23evFnDhw9vFk4uuOCC054zJydHklo8JwAAANCZOb7FJjEuRpse/qqxa7dHXFxcs3WXyyW/33/ScZZlnfSIm2VZpz1n42daOifgdD6/pTqfX/V+S3X1fvksSz6/pXq/JZ/Pali39/v8x/f5G49pMvmtxrmaLDdMfslnWbIsS74my/4mx1sNc78lWWpYb7pf9n+zjcf4LUuy/9ewXbKafNYKfMbe1rhfgf1NjlXjPntb0/MeP0fD/oZtDQcHPhP4fMN51eycarZ+fMvJxzRe4+Rtzf/uTvztdarfZ6f6fGPdpzvuTD/XVmf4sSB8+EwvaeCiCIkzvWeBjrp/6iCd19s5o/Y6Pti4XK52PQ7mBAMHDtTzzz8vr9crj8cjSVq9erXhqhCtfH67H1t1rT1V1dWrqtanmlqfqmp9qqrzqabOp9p6v7z1/ob56dYb5j6/6ur9qvf7Ve+zVOuz5/V+v+p8lup99rzO7+cfdgAAwqy8us50Ce3SuRJBJ3HjjTdqzpw5uuOOOzRr1izt2bNHTzzxhCS1a7ACRC/LslRT51dlTZ0qauqbzSsb16uPr1c0zKtqfaqqrVdNnV9Vtfa6tz5yWwBj3S653S7Ful2KaZhiA3O33G4p1u2297mOH+N2u+R2STEul9wul9xu2dsb1132ust1/HMKHC+5XfY+l0vN1t0uNWxzNWyTXDq+3dVkmz1Xk+MkNZ6jyf7Gz6lxucm+E7fZ667A56SmxzU9R5MPq/m57HXXCest79cprhE4fdPlE351NV136YSdLRx/Kq39TmztFK2dv6V62vK503HSb2/+qQEgSUNyU02X0C4EmwiUmpqqN998U9/73vc0cuRIDRs2TA888IBuvPFGBgWIUpZlqby6ToeO1urwUa+OHKvVoWP28uGjtTp8zBvYV1pVp8qaOtX5gtvE4XLZj18mxccooWGeGB+rxDi3EuJi5Il1yxMbo/hYtzyx7ob58XVPC9vjY9yKi3Urzu1SbIxbsTEuxTfMY91uxcW4FNewHudumMe4A+GFoA8AABoRbMJkyZIlJ21r+t6aE585HzdunD777LPA+vPPP6+4uDj17t1bklRQUHDSZ0aOHHnaZ9cRebz1Pu0rq9He0mp9WVqlvWXV2ltarYNHvc2CTL2//X+3bpfUxROr1MQ4pSTEKSUhVqkJcUpNiLWXE+1tjfuS42OVGB8TCDCJ8TFKio9VYlyMEuLcBAkAABCxCDYR6g9/+IP69eunnj176rPPPtN9992n66+/XomJiaZLQzsd9dZrb2m19pZV2eGlIbh8WVqtvWXVOljpbfO5UhNildnFo8zkeGV2iVdmF4+6JdvzzC7xykz2KD05zg4viXFKjm/fu5YAAACcimAToYqLi/XAAw+ouLhYOTk5uu666/Too4+aLgutKK+u05YDlfqiuFKbiyu0ubhS20qOqrTq9B3vEuNi1Cs9UT3TE9Wzqz3PTk0IhJhuXezA4olt30h8AAAA0YJgE6Huvfde3XvvvabLQAu89T5tKzmqzcWV9nTAnu8vrznlZ1ITYtUrPSkQXHql21PPrva29KQ4WlYAAAA6gGADtKKypk6rd5dq/Zfl2lxcqS+KK7TrcJV8p+jvkpuWoAHZKRqQnaoB2V10To8U9c5IUkpCXIvHAwAAIDgINkATFTV1Wr3riD7acUQrdxzWhn0VLYaYtMQ4DchO0cDsFJ3To2GenaJUAgwAAIARBBtEtfJqO8is3HFYH+08og17y3VijumdkaTRfdI1MKehJaZHinqkenh0DAAAIIIQbBBVyqvrtGqnHWRW7jysTfsqTgoyfTOTNCY/UxeelaEx+ZnK7cpIdAAAAJGOYINOb395td5Yu09/W7dfG/aV68RX/fTrlqwx/TJ0Yb9MjcnPVHYaL0EFAABwGoINOqWKmjq9vb5Yr326Vyt3Hm4WZvp1T24IMXaY6ZFKkAEAAHA6gk2YFBQUaOTIkXrqqadMl9Jpeet9WrL5oF5fu1f/7/MS1db7A/su6Juhq8/tqcsGZSmLIAMAANDpEGzgaH6/pdW7S/Xap3v11vr9Kq8+/jLM/llddPW5PXXVyFz1Sk8yWCUAAABCjWADR9pyoFKLPt2r19fu096y6sD2Hqke/duIXF19bk8Nzkll5DIAAIAo4TZdQDQqLS3VzTffrPT0dCUlJWny5MnaunWrJMmyLHXv3l2vvPJK4PiRI0cqKysrsL5ixQrFxcXp6NGjYa/dpDqfXy+v2qMpv3hfk/53mX69ZLv2llWriydW143qpedvG6Plsy7VnKmDNSQ3jVADAAAQRZzfYmNZUl2VmWvHJUln8OX5lltu0datW/XGG28oNTVV9913n6ZMmaJNmzYpLi5OEyZM0JIlS/Tv//7vKi0t1aZNm5ScnKxNmzZp8ODBWrJkiUaNGqUuXbqE4IeKPHU+v177ZK9++d5WFR2xW2di3S4VDMjS1efm6rJBPZQQF2O4SgAAAJjk/GBTVyXNzTVz7fv3SfHJ7fpIY6D58MMPNW7cOEnS888/r7y8PC1atEjXXXedCgoKNH/+fEnSsmXLNGLECPXu3VtLliwJBJuCgoJg/zQRp97n12uf7tW897Zp92E7vHbrEq87JvTTf4zKU0ZyvOEKAQAAECmcH2wc5vPPP1dsbKzGjBkT2JaZmakBAwbo888/l2SPoHbPPffo0KFDWrp0qQoKCtS7d28tXbpUd9xxh5YvX64ZM2YY+glCr97n1+tr9+mX/9qqXQ2BJjM5Xt+d2E/furCPkuK5bQEAANCc878hxiXZLSemrt1O1olvh2yyvbFPyNChQ5WZmamlS5dq6dKlevjhh5WXl6dHH31Uq1atUnV1tS6++OIOlR6JfH5Lb3y2V0+/u007Dx2TJGUk2y00N48l0AAAAODUnP9N0eVq9+NgJg0ePFj19fX66KOPAo+iHT58WFu2bNGgQYMkSS6XSxMmTNDrr7+uDRs2aPz48UpJSVFdXZ1+85vf6LzzzlNKSorJHyOofH5Lb362T0+/u1U7GgJNelKc7phwlm4e20fJHuffpgAAAAgtvjGGWf/+/XXVVVfp9ttv129/+1ulpKRo1qxZ6tmzp6666qrAcQUFBfrhD3+oc889V6mpqZKkCRMm6Pnnn9fMmTNNlR9UPr+lv62zA832g3ag6ZoUp9vH99O0cX3VhUADAACANuKbowELFizQPffcoyuuuEK1tbWaMGGC3nrrLcXFxQWOueSSS+Tz+ZoNEjBx4kQtWrRIEydONFB1cL29oVhPvLNZ20rsIavTEuN0+/h8TRvXVykJcaf5NAAAANCcyzpVpw9DKioqlJaWpvLy8kBLRaOamhrt3LlT+fn5SkhIMFShs5n+M6yqrdeDr2/UX9Z8KUlKTYjVbeP76ZaL+iqVQAMAAIAmWssGJ6LFBmGzubhS01/4RNtKjsrtkr478Sx9r+AsAg0AAAA6jGCDkLMsSy+tKtJDb2yUt96vrBSPfvH1czX2rEzTpQEAAKCTINggpCpr6nT/axv05mf2kNwTz+mun18/Qt26eAxXBgAAgM6EYIOQWf9lub7/4ifafbhKMW6XfvzVAbpjfD+53S7TpQEAAKCTIdgg6CzL0u+X79Lctz5Xnc9Sz66Jevob52pUn3TTpQEAAKCTcmSw8fv9pktwrFD/2ZVV1erHf12nxZsOSJImDe6h//mPEUpLYoAAAAAAhI6jgk18fLzcbrf27dun7t27Kz4+Xi4XjzW1hWVZqq2t1cGDB+V2uxUfHx/0a6zZfUQ/eHGt9pZVKz7GrfunDNS0cX35OwIAAEDIOSrYuN1u5efna//+/dq3b5/pchwpKSlJvXv3ltvtDto5/X5Lv122Q0+8s1k+v6W+mUmad+N5GtozLWjXAAAAAFrjqGAj2a02vXv3Vn19vXw+n+lyHCUmJkaxsbFBbUE5dNSrmX/+TMu2HJQk/duIXD16zVCl8G4aAAAAhJHjgo0kuVwuxcXFKS6OL88mrfuyTN9ZuFoHK71KiHPrp/82RNePzuPRMwAAAISdI4MNzPusqEzf+t1HqqypV/+sLpp343kakJ1iuiwAAABEKYIN2m1tUZluagg15/dN14JbL1AXD7cSAAAAzOHbKNplbVGZbnruI1V6CTUAAACIHHwjRZt9uqdUN//uY1V663VB3wwtuPV8JRNqAAAAEAH4Voo2aRZq8jO04BZCDQAAACJHu19msmzZMl155ZXKzc2Vy+XSokWLmu23LEsPPfSQcnNzlZiYqIKCAm3cuDFY9cKATwg1AAAAiHDtDjbHjh3TiBEjNG/evBb3/+xnP9OTTz6pefPmadWqVcrOztbll1+uysrKDheL8GsaasbkZ+j3PH4GAACACNTub6iTJ0/W5MmTW9xnWZaeeuopzZkzR9dee60kaeHCherRo4deeOEFffe73+1YtQirNbtLNe3/PtbRhlCz4NbzlRRPqAEAAEDkaXeLTWt27typ4uJiTZo0KbDN4/Fo4sSJWr58eYuf8Xq9qqioaDbBvDW7jwRCzYX9CDUAAACIbEENNsXFxZKkHj16NNveo0ePwL4TFRYWKi0tLTDl5eUFsyScgTW7j+jm39mhZmy/TP3fLYQaAAAARLagBptGLper2bplWSdtazR79myVl5cHpqKiolCUhDZavcsONcdqfYQaAAAAOEZQv7FmZ2dLsltucnJyAttLSkpOasVp5PF45PF4glkGztCqXUd0y//ZoWbcWZn63bTzlRgfY7osAAAA4LSC2mKTn5+v7OxsLV68OLCttrZWS5cu1bhx44J5KQTZql12n5pjtT5ddDahBgAAAM7S7habo0ePatu2bYH1nTt3au3atcrIyFDv3r01Y8YMzZ07V/3791f//v01d+5cJSUl6cYbbwxq4QiexlBT1RBqnruZUAMAAABnaXewWb16tS655JLA+syZMyVJ06ZN0+9//3vde++9qq6u1l133aXS0lKNGTNG77zzjlJSUoJXNYKmvKpO3/vTGlXV+nTx2d307M2jCTUAAABwHJdlWZbpIpqqqKhQWlqaysvLlZqaarqcTm/2q+v14sd7dFb3ZP3t7vGEGgAAAESM9mSDkIyKBmdYveuIXvx4jyRp7jXDCDUAAABwLIJNlKqt9+v+19ZLkq4f3Utj+mUarggAAAA4cwSbKPXcBzu05cBRZSTHa/bkQabLAQAAADqEYBOF9hyu0i/+31ZJ0k+mDlJ6crzhigAAAICOIdhEGcuy9JPXN8hb79e4szJ1zbk9TZcEAAAAdBjBJsq8uW6/lm05qPgYtx65eqhcLpfpkgAAAIAOI9hEkfKqOj385iZJ0vRLzla/7l0MVwQAAAAEB8Emijz+zy906KhX/bon686CfqbLAQAAAIKGYBMl1uw+ohc+Ov7OGk8s76wBAABA50GwiQJ1Pr/uf3WDJOm6Ub10Ie+sAQAAQCdDsIkCz76/Q5sPVCojOV73T+GdNQAAAOh8CDadXNN31syZwjtrAAAA0DkRbDqxpu+sGdsvU9eexztrAAAA0DkRbDqxpu+sefQa3lkDAACAzotg00nxzhoAAABEE4JNJ8U7awAAABBNCDad0JrdpbyzBgAAAFGFYNPJ2O+sWS+Jd9YAAAAgehBsOpnn3t/JO2sAAAAQdQg2nciew1X6xbtbJPHOGgAAAEQXgk0n0fjOmpo63lkDAACA6EOw6STe2XSAd9YAAAAgahFsOoHaer8K3/pcknT7hHzeWQMAAICoQ7DpBP64crd2Ha5Sty4efa/gbNPlAAAAAGFHsHG4sqpaPf3uVknSf046R108sYYrAgAAAMKPYONwT7+7TeXVdRqYnaLrR+eZLgcAAAAwgmDjYDsOHtUfVuySJM2ZOkgxbgYMAAAAQHQi2DjYY//4QvV+SwUDumt8/+6mywEAAACMIdg41Modh/XOpgOKcbs0Z8og0+UAAAAARhFsHMjvt/TI3zdJkr5xQZ7690gxXBEAAABgFsHGgV77dK827K1QiidWMy47x3Q5AAAAgHEEG4eprvXpf/65WZJ01yVnq1sXj+GKAAAAAPMINg7z7Ps7VFxRo55dE3XrRX1NlwMAAABEBIKNgxyoqNEzS7ZLkmZNHqiEuBjDFQEAAACRgWDjID9/Z7Oq63w6t3dXXTE8x3Q5AAAAQMQg2DjEpn0V+suaLyVJP5k6WC4XL+MEAAAAGhFsHMCy7OGdLUu6YniORvVJN10SAAAAEFEINg7wry9KtHz7YcXHunXf1waaLgcAAACIOASbCFfn8+vRtz6XJN16UV/lZSQZrggAAACIPASbCPfix3u04+AxZSTHa/olZ5suBwAAAIhIBJsIVl5dp/9dvEWS9MPLz1FqQpzhigAAAIDIRLCJYL96b5tKq+p0dlYXfeP8PNPlAAAAABGLYBOh9hyu0u8/3CVJmjNlkGJj+KsCAAAAToVvyxHq8be/UK3Pr/H9u6lgQHfT5QAAAAARjWATgVbvOqK/r98vt0uaM3UQL+MEAAAAToNgE2H8fkv//Xd7eOcbzs/TwOxUwxUBAAAAkY9gE2H+tn6/PisqU3J8jH54+TmmywEAAAAcgWATYd5Yu1eS9O2L85WVkmC4GgAAAMAZCDYRxOe39NHOI5Kkywf3MFwNAAAA4BwEmwiyaV+FKmvqleKJ1eAc+tYAAAAAbUWwiSArdxyWJJ2fn8F7awAAAIB24NtzBGkMNmP7ZRquBAAAAHAWgk2E8PktfdzQv+ZCgg0AAADQLgSbCLFpX4UqvQ39a3LpXwMAAAC0B8EmQjQ+hnZBfoZi3C7D1QAAAADOQrCJECsagg2PoQEAAADtF/RgU19fr5/85CfKz89XYmKi+vXrp4cfflh+vz/Yl+o06n1+rWroXzP2LIINAAAA0F6xwT7h448/rt/85jdauHChhgwZotWrV+vWW29VWlqa7rnnnmBfrlPYtL+hf01CrAbx/hoAAACg3YIebFasWKGrrrpKU6dOlST17dtXL774olavXh3sS3Uajf1rxtC/BgAAADgjQX8U7eKLL9a7776rLVu2SJI+++wzffDBB5oyZUqLx3u9XlVUVDSbos3KHQzzDAAAAHRE0Fts7rvvPpWXl2vgwIGKiYmRz+fTo48+qm984xstHl9YWKif/vSnwS7DMep9ft5fAwAAAHRQ0FtsXn75Zf3pT3/SCy+8oE8++UQLFy7UE088oYULF7Z4/OzZs1VeXh6YioqKgl1SRNu4r0JHvfVKpX8NAAAAcMaC3mLz4x//WLNmzdLXv/51SdKwYcO0e/duFRYWatq0aScd7/F45PF4gl2GYxx/f00m/WsAAACAMxT0Fpuqqiq53c1PGxMTw3DPp7Ay8P6aDMOVAAAAAM4V9BabK6+8Uo8++qh69+6tIUOG6NNPP9WTTz6pb3/728G+lOPV+/xatatUEv1rAAAAgI4IerD55S9/qf/6r//SXXfdpZKSEuXm5uq73/2uHnjggWBfyvHoXwMAAAAER9CDTUpKip566ik99dRTwT51p7OC/jUAAABAUAS9jw3arrF/zdizeAwNAAAA6AiCjSH1Pr9WBd5fw8ABAAAAQEcQbAzZsK9Cx2p9SkuM06Bs+tcAAAAAHUGwMeT4+2sy5KZ/DQAAANAhBBtDVmxvfH8N/WsAAACAjiLYGFDn82v1LvrXAAAAAMFCsDFgw95y+tcAAAAAQUSwMWDlDru1Zgz9awAAAICgINgY0DhwAP1rAAAAgOAg2IRZnc+vVYH+NQQbAAAAIBgINmG2fm+5qmp96poUp4HZKabLAQAAADoFgk2YNT6GRv8aAAAAIHgINmHWOHAAj6EBAAAAwUOwCaPm768h2AAAAADBQrAJo6b9awb0oH8NAAAAECwEmzBasZ3+NQAAAEAoEGzCiPfXAAAAAKFBsAkTu39NqSRp7FkEGwAAACCYCDZhsu7LclXX+ZSeFKdzsuhfAwAAAAQTwSZMjr+/JpP+NQAAAECQEWzC5Hj/mgzDlQAAAACdD8EmDGrrj/evuZD+NQAAAEDQEWzCYP3eMlXX+ZSRHE//GgAAACAECDZhsHLHEUm8vwYAAAAIFYJNGPD+GgAAACC0CDYh1qx/DcEGAAAACAmCTYit+/J4/5r+WV1MlwMAAAB0SgSbEGs6zDP9awAAAIDQINiEWOPAATyGBgAAAIQOwSaEauv9Wr2bYAMAAACEGsEmhNZ9WaaaOj/9awAAAIAQI9iE0Irtx/vXuFz0rwEAAABChWATQit38v4aAAAAIBwINiHirfdpzW77/TVjCTYAAABASBFsQmTdl+WqqfMrMzleZ9O/BgAAAAgpgk2IrNx+/DE0+tcAAAAAoUWwCZHj/WsyDFcCAAAAdH4EmxDYVlKpj3fa768Zexb9awAAAIBQI9gEmd9vadYr61Xns/SVgVk6OyvFdEkAAABAp0ewCbIXPt6j1btLlRwfo/++eqjpcgAAAICoQLAJouLyGj3+jy8kST/66gD17JpouCIAAAAgOhBsgujBNzao0luvEXlddfPYvqbLAQAAAKIGwSZI3t5QrH9uPKBYt0uPXTtMMW6GeAYAAADChWATBBU1dXrwjQ2SpDsm9NOgnFTDFQEAAADRhWATBD97+wsdqPCqb2aSfnBpf9PlAAAAAFGHYNNBq3cd0Z9W7pEkzb12mBLiYgxXBAAAAEQfgk0HeOt9mvXqeknS9aN7adxZ3QxXBAAAAEQngk0HPLNku7aVHFW3LvG6f8og0+UAAAAAUYtgc4a2lVTq1+9tlyQ9eOUQdU2KN1wRAAAAEL0INmfA77c065X1qvX5dcmA7rpieI7pkgAAAICoRrA5Ay+u2qPVu0uVFB+jR64ZJpeLd9YAAAAAJhFs2ulARY0ee+sLSdKPJg1Qz66JhisCAAAAQLBppwdf36hKb71G5HXVtHF9TZcDAAAAQASbdvnnxmK9vbFYsW6XHrt2mGLcPIIGAAAARAKCTRtV1NTpgdc3SJLumNBPg3JSDVcEAAAAoBHBpo1+9vYXOlDhVd/MJP3g0v6mywEAAADQREiCzd69e/Wtb31LmZmZSkpK0siRI7VmzZpQXCosVu86oj+t3CNJmnvNMCXExRiuCAAAAEBTscE+YWlpqS666CJdcskl+sc//qGsrCxt375dXbt2DfalwsJb79OsV9dLkq4b1Uvjzu5muCIAAAAAJwp6sHn88ceVl5enBQsWBLb17ds32JcJm2eWbNe2kqPq1iVec6YOMl0OAAAAgBYE/VG0N954Q6NHj9Z1112nrKwsnXvuuXr22WdPebzX61VFRUWzKVJsK6nUr9/bLkl64Moh6poUb7giAAAAAC0JeovNjh079Mwzz2jmzJm6//779fHHH+sHP/iBPB6Pbr755pOOLyws1E9/+tNglxEU731xULU+vy4Z0F1XDs8xXQ6AULOshsknWX7J7zu+bPkb9vlPmFrYJrV8nKzjxzcuB7adZv9Jc52w7YT1xp+n2bJO8Xm1fFyz9VNt06mPa3HfCftbOteJn2/rcR06V1uc6ec6cs2OMHFNRBQj9x06lf6TpK55pqtoM5dlBfeuj4+P1+jRo7V8+fLAth/84AdatWqVVqxYcdLxXq9XXq83sF5RUaG8vDyVl5crNdX8kMrLtx9S74wk9UpPMl0KEJksS/LXS/Vee/J5pfoaqb7WnvtqT1j3Njm2tmGqs8/RbLlO8tc1bKtvWG5ynN9nLzfOrabrjcuN2088tpXwAgAAbN/8q9T/cqMlVFRUKC0trU3ZIOgtNjk5ORo8eHCzbYMGDdIrr7zS4vEej0cejyfYZQTNuLMYLACdiGXZ4aKmQqopl7wN85pyyVsp1R6T6o5JddVSbZW9XFsl1TVMjcu1Dcc0Lls+0z9ZmLkkl1tyNc6bTCftczXZfsJy4Fi1sr+1eePn1PIxUvPjAuut7WvLutq2X64Wjm3LvhP2t/e4Vo8P4edOp9XzAkAESu5uuoJ2CXqwueiii7R58+Zm27Zs2aI+ffoE+1JA9LIsO4gcOygdOyRVHWpYPihVHWkILmUNwaWiSYCpsFs+QskdK8UmSDHx9jw2/oR1jz3FeOx9MfGSO06KiW2YxzdZbpgal92xJ6zHSK4Ye7s71l53N1kP7Guy3dWw7HLZyy738fMEllva52oy5wsqAACRJujB5oc//KHGjRunuXPn6vrrr9fHH3+s+fPna/78+cG+FND51FVLZUVS+R7paMnxsHLscJPlhhDj857+fKfickueFCkhTfKkSQmpkidVik+W4pOkuIYpPkmKS5biEu19gW2N+xv2xSYcDy1u3vMEAADCL+h9bCTpb3/7m2bPnq2tW7cqPz9fM2fO1O23396mz7bnOTrAcWoqpPIiO7yU7bEDTGC5yA4s7RGXLCV3s5uKk7vby0kZdmBpGloS0uzg0rgc34VWBwAAEPHakw1CEmw6gmADx6upkA5+IR3YKB3aYoeWst12gKkpO/3n47tIaXlSak7zwNK4nNStYb2b3WICAADQSRkdPACIGvVe6dBWqWSTPR3YJJV8brfCtCYx3Q4uXXvbU2A5z15OTKc1BQAAoJ0INsDp+P1S2S47tBzYJJVstJcPb7OHDm5JSq7UY7DUfaCU3rdJiOll920BAABAUBFsgBP5/XYLzK4PpF3vS7s/lKpLWz7Wk2YHmKxBUtZgqccQO8wkZYS3ZgAAgChHsAHaEmRiPFL3c6SsIXaI6dEwT+3JY2MAAAARgGCD6NOWIBOXLPW+UOp7sdR3vJQ70n5vCgAAACISwQbRobJY2vSGtHMpQQYAAKATItig86oplz5/U1r3Z7tlxvIf3xeXLPUZezzI5IwgyAAAADgYwQadS71X2vqOHWa2/FPyeY/vyxsjnfM1KX8CQQYAAKCTIdjA+fw++/GydX+2Hzfzlh/f132gNOw6adh/2MMuAwAAoFMi2MCZLEva/5m0/i/Shlelyn3H96X2lIb+ux1osocxahkAAEAUINjAWSqLpU/+KK3/s3Roy/HtCWnS4Kul4ddLvcdJbrexEgEAABB+BBs4w9GD0odPSauek+pr7G2xCdKAyXbLzNmXSbEeoyUCAADAHIINIlvVEWnFPGnlb6S6Y/a2XhdIo2+VBl4hJaSarQ8AAAARgWCDyFRTIa18xg413gp7W+650iU/kc6+lH4zAAAAaIZgg8hSe0z6+Fn7sbPGl2hmDZG+MkcaMIVAAwAAgBYRbBAZ6mqkNQuk938uHTtob8vsL10yWxp8DYMBAAAAoFUEG5hVXyt9+kdp2RPHh2zu2kcqmG0PChDDLQoAAIDT41sjzPDVS+tekpY+LpXtsbel9pQm3iuN/KYUE2e2PgAAADgKwQbht/8z6dXvSgc/t9eTs6QJP5LOmybFJZitDQAAAI5EsEH4+P32KGfvPiz566TEDOniH0rn3ybFJ5muDgAAAA5GsEF4lO+VFt0p7Vxmrw+8QrryaSk502xdAAAA6BQINgi9jYukN++RasqkuCTpa49J593M0M0AAAAIGoINQsdbKf1jlrT2T/Z67rnStc9J3c42WxcAAAA6HYINQuPL1dIrt0mlOyW5pPEz7SGcGe0MAAAAIUCwQXD56qUPnpSWPCZZPim1l3Ttb6W+F5uuDAAAAJ0YwQbBU7pbevUOqWilvT7kWumK/5USuxotCwAAAJ0fwQbBse7P0t//U/JWSPEp0tQnpOE3MEAAAAAAwoJgg46pLrMDzYa/2ut5Y6Rr50vpfU1WBQAAgChDsMGZO7BReuEGqbxIcsVIE++Txv+nFMNtBQAAgPDiGyjOzJGd0h+vkY4esFtnrn1OyjvfdFUAAACIUgQbtF/lAemPV9uhJmuIdOvfpcR001UBAAAgirlNFwCHqS6T/nStVLpL6tpHuulVQg0AAACMI9ig7WqrpBe/Lh3YICVnSTcvklKyTVcFAAAAEGzQRr466S+3SHtWSJ40u6Umo5/pqgAAAABJBBu0hd8vvT5d2vpPKTZBuvFlKXuY6aoAAACAAIINWmdZ0j/vl9a9bA/pfP0fpD5jTVcFAAAANEOwQevef0L66Bl7+epnpHO+arYeAAAAoAUEG5zaqt9J/3rEXv7a49KIG8zWAwAAAJwCwQYt2/Cq9Pf/tJcn3CtdeKfZegAAAIBWEGxwsm3vSq/eIcmSRn9HuuR+0xUBAAAArSLYoLmiVdLL35L8ddKQa6Up/yO5XKarAgAAAFpFsMFxJZ9LL1wn1VVJZ31Fuua3kjvGdFUAAADAaRFsYCvdLf3xGqm6VOp1vnTDn6TYeNNVAQAAAG1CsIF09KAdair3S90HSTf+WYpPNl0VAAAA0GYEm2hXXyu9+HXpyHYprbd006tSUobpqgAAAIB2IdhEu3d/Ku1dLSV0lW5eJKXmmq4IAAAAaDeCTTTb/La0Yp69fPWvpcyzzNYDAAAAnCGCTbQq3ystanjp5pjvSQOnmq0HAAAA6ACCTTTy1UuvfMceAS1npHT5T01XBAAAAHQIwSYaLSmU9qyQ4lOk6xZIsR7TFQEAAAAdQrCJNtv/Jb3/c3v5356WMvqZrQcAAAAIAoJNNKk8IL16hyRLGnWrNPRa0xUBAAAAQUGwiRZ+n/TqbdKxg1LWEOlrhaYrAgAAAIKGYBMt3n9S2rlMikuSrvu9FJdouiIAAAAgaAg20WDXh9KSufby1J9L3c8xWw8AAAAQZASbzu7YYemV2yTLL434hjTyRtMVAQAAAEFHsOnM/H5p0fekyn1SZn9pyhOmKwIAAABCIuTBprCwUC6XSzNmzAj1pXCilb+Stv5TivHY/Wo8XUxXBAAAAIRESIPNqlWrNH/+fA0fPjyUl0FLvlwt/b+H7OXJj0nZQ42WAwAAAIRSyILN0aNH9c1vflPPPvus0tPTT3mc1+tVRUVFswkdVF0q/eVWyV8vDbnGfmcNAAAA0ImFLNhMnz5dU6dO1WWXXdbqcYWFhUpLSwtMeXl5oSopOliW9MbdUvkeKb2vdOUvJJfLdFUAAABASIUk2Lz00kv65JNPVFh4+pdAzp49W+Xl5YGpqKgoFCVFj1XPSZ+/KbnjpP9YICWkma4IAAAACLnYYJ+wqKhI99xzj9555x0lJCSc9niPxyOPxxPsMqLT/s+kf95vL1/+sNTzPLP1AAAAAGES9GCzZs0alZSUaNSoUYFtPp9Py5Yt07x58+T1ehUTExPsy6K2yu5X46uVBkyRLvye6YoAAACAsAl6sLn00ku1fv36ZttuvfVWDRw4UPfddx+hJlQ2/FU6sl1KyZGu+hX9agAAABBVgh5sUlJSNHRo86GFk5OTlZmZedJ2BNGa39vzMXdKSRlGSwEAAADCLeQv6EQY7F8n7V1jDxgw8pumqwEAAADCLugtNi1ZsmRJOC4TvRpbawZdIXXpbrQUAAAAwARabJyu9pi07s/28qhbjJYCAAAAmEKwcboNr0q1lVJ6vtR3gulqAAAAACMINk63ZoE9H3WL5OavEwAAANGJb8JOxqABAAAAgCSCjbN9stCeD5zKoAEAAACIagQbp2o6aMDoW83WAgAAABhGsHGqDa9K3goGDQAAAABEsHGuxnfXjJrGoAEAAACIenwjdqLi9dLe1ZI7lkEDAAAAABFsnKmxtWbgFVKXLKOlAAAAAJGAYOM0TQcNGHWL0VIAAACASEGwcZqNrzUMGtBXyp9ouhoAAAAgIhBsnCYwaMAtDBoAAAAANOCbsZMUb5C+XMWgAQAAAMAJCDZOEhg0YCqDBgAAAABNEGycorZKWveyvTzqVrO1AAAAABGGYOMUG19l0AAAAADgFAg2TtH4GNp50xg0AAAAADgB35CdgEEDAAAAgFYRbJzgk4X2fOBUKaWH2VoAAACACESwiXS1VdJnjYMG3GK0FAAAACBSEWwi3cbXJG95w6ABBYaLAQAAACITwSbSMWgAAAAAcFp8U45kBzZKX37MoAEAAADAaRBsIllja82AKQwaAAAAALSCYBOpmg4aMPpWs7UAAAAAEY5gE6k2LbIHDejah0EDAAAAgNMg2ESq1Qvs+SgGDQAAAABOh2/MkajZoAHfMl0NAAAAEPEINpFozUJ7zqABAAAAQJsQbCJNbZW07iV7edQtRksBAAAAnIJgE2k2LZJqGgYN6HeJ6WoAAAAARyDYRBK/T/rgf+3lUbcwaAAAAADQRnxzjiTrXpYObZES06XzbzNdDQAAAOAYBJtIUV8rLSm0ly+aISWkGi0HAAAAcBKCTaT49A9S2R6pSw/pgjtMVwMAAAA4CsEmEtRVS8uesJfH/0iKTzJbDwAAAOAwBJtIsOo5qXK/lJYnjZpmuhoAAADAcQg2pnkrj4+ENvE+KdZjth4AAADAgQg2pq18Rqo6LGWeLY34hulqAAAAAEci2JhUdURa/kt7uWC2FBNrth4AAADAoQg2Ji1/WvJWSD2GSkOuNV0NAAAA4FgEG1MqD0gf/dZevmSO5OavAgAAADhTfJs25YMnpboqqecoacBk09UAAAAAjkawMaGsSFr9f/byV/5LcrnM1gMAAAA4HMHGhGU/k3y1Ut/xUr8C09UAAAAAjkewCbfD26VPn7eXaa0BAAAAgoJgE25LCiXLJ/WfJPUeY7oaAAAAoFMg2ITTgU3S+r/ay1/5idlaAAAAgE6EYBNO7z0qyZIGXyXljDBdDQAAANBpEGzCZe8a6Yu/SXJJBfebrgYAAADoVAg24fKvR+z58BukrIFmawEAAAA6GYJNOOz6UNr+L8kdKxXMMl0NAAAA0OkQbELNsqR//be9fO5NUka+2XoAAACATohgE2rb3pX2rJBiPNKEH5uuBgAAAOiUCDah1LS15vzbpLSeZusBAAAAOqmgB5vCwkKdf/75SklJUVZWlq6++mpt3rw52Jdxhs/flPavleKSpYt/aLoaAAAAoNMKerBZunSppk+frpUrV2rx4sWqr6/XpEmTdOzYsWBfKrL5fQ3vrZF04fekLt3N1gMAAAB0YrHBPuHbb7/dbH3BggXKysrSmjVrNGHChGBfLnKt/6t08AspIU0ad7fpagAAAIBOLejB5kTl5eWSpIyMjBb3e71eeb3ewHpFRUWoSwqP95+w5+N+ICV2NVoKAAAA0NmFdPAAy7I0c+ZMXXzxxRo6dGiLxxQWFiotLS0w5eXlhbKk8KipkA5tsZfP/47ZWgAAAIAoENJg8/3vf1/r1q3Tiy++eMpjZs+erfLy8sBUVFQUypLCo2y3PU/KlBLTzdYCAAAARIGQPYp2991364033tCyZcvUq1evUx7n8Xjk8XhCVYYZpbvseTov4wQAAADCIejBxrIs3X333Xrttde0ZMkS5edH4Zf7IzvteXpfo2UAAAAA0SLowWb69Ol64YUX9PrrryslJUXFxcWSpLS0NCUmJgb7cpEp0GLT12QVAAAAQNQIeh+bZ555RuXl5SooKFBOTk5gevnll4N9qchFsAEAAADCKiSPokW90oZH0TKi8DE8AAAAwICQjooWlfw+qWyPvUyLDQAAABAWBJtgq9gr+eulmHgpJcd0NQAAAEBUINgEW+OIaF37SO4Ys7UAAAAAUYJgE2wMHAAAAACEHcEm2Ag2AAAAQNgRbIKNYAMAAACEHcEm2BjqGQAAAAg7gk2w0WIDAAAAhB3BJpiqy6TqUnu5ax+jpQAAAADRhGATTI2tNclZkqeL0VIAAACAaEKwCSYeQwMAAACMINgEE8EGAAAAMIJgE0yNI6IRbAAAAICwItgEU2OLDUM9AwAAAGFFsAkmHkUDAAAAjCDYBIuvXiorspcJNgAAAEBYEWyCpbxIsnxSbILUJdt0NQAAAEBUIdgES+NjaF37SG7+WAEAAIBw4ht4sNC/BgAAADCGYBMsDPUMAAAAGEOwCRaGegYAAACMIdgEC4+iAQAAAMYQbIKFYAMAAAAYQ7AJhqojUk25vdy1j9laAAAAgChEsAmGxtaaLtlSfJLRUgAAAIBoRLAJBh5DAwAAAIwi2AQDQz0DAAAARhFsgoGhngEAAACjCDbBwKNoAAAAgFEEm2A4ssueE2wAAAAAIwg2HVVfK1V8aS+n8ygaAAAAYALBpqPKiyTLL8UmSl2yTFcDAAAARCWCTUc17V/jcpmsBAAAAIhaBJuOYqhnAAAAwDiCTUcx1DMAAABgHMGmoxjqGQAAADCOYNNRDPUMAAAAGEew6QjLatJiw6NoAAAAgCkEm46oOiLVVtrLXXubrQUAAACIYgSbjmgcES0lV4pLMFsLAAAAEMUINh3BwAEAAABARCDYdERjiw1DPQMAAABGEWw6ghYbAAAAICIQbDqCoZ4BAACAiECw6QiGegYAAAAiAsHmTNV7pYq99jItNgAAAIBRBJszVbZHkiXFJUvJ3UxXAwAAAEQ1gs2ZanwMLSNfcrmMlgIAAABEO4LNmWJENAAAACBiEGzOFMEGAAAAiBgEmzN1pOHlnAQbAAAAwDiCzZliqGcAAAAgYhBszoRl8SgaAAAAEEEINmfi2EGp7pgkl9Q1z3Q1AAAAQNQj2JyJxtaatF5SrMdoKQAAAAAINmeGx9AAAACAiBKyYPPrX/9a+fn5SkhI0KhRo/T++++H6lLhFxgRrY/ZOgAAAABIClGwefnllzVjxgzNmTNHn376qcaPH6/Jkydrz549obhc+NFiAwAAAESUkASbJ598Ut/5znd02223adCgQXrqqaeUl5enZ555JhSXCz+GegYAAAAiStCDTW1trdasWaNJkyY12z5p0iQtX778pOO9Xq8qKiqaTRGPYAMAAABElKAHm0OHDsnn86lHjx7Ntvfo0UPFxcUnHV9YWKi0tLTAlJcX4cMn11VLlfvsZR5FAwAAACJCyAYPcLlczdYtyzppmyTNnj1b5eXlgamoqChUJQVHWUM/IU+qlJRhthYAAAAAkqTYYJ+wW7duiomJOal1pqSk5KRWHEnyeDzyeBz0LpjAY2h9pBaCGgAAAIDwC3qLTXx8vEaNGqXFixc327548WKNGzcu2JcLv8BQz32NlgEAAADguKC32EjSzJkzddNNN2n06NEaO3as5s+frz179ujOO+8MxeXCi6GeAQAAgIgTkmBzww036PDhw3r44Ye1f/9+DR06VG+99Zb69OkEL7RkRDQAAAAg4oQk2EjSXXfdpbvuuitUpzenlEfRAAAAgEgTslHROiXL4lE0AAAAIAIRbNrj6AGpvkZyuaWuvU1XAwAAAKABwaY9Gltr0npJMXFGSwEAAABwHMGmPRjqGQAAAIhIBJv2oH8NAAAAEJEINu3BUM8AAABARCLYtAdDPQMAAAARiWDTHjyKBgAAAEQkgk1b1VbZwz1LUgaPogEAAACRhGDTVmW77XlCmpSYbrYWAAAAAM0QbNqKoZ4BAACAiEWwaSv61wAAAAARi2DTVgz1DAAAAEQsgk1bMdQzAAAAELEINm3Fo2gAAABAxCLYtIXfL5U2jIrGUM8AAABAxCHYtEXlfsnnlVwxUmov09UAAAAAOAHBpi0aH0PrmifFxBotBQAAAMDJCDZtQf8aAAAAIKIRbNqCoZ4BAACAiEawaQuGegYAAAAiGsGmLXgUDQAAAIhoBJu2aAw2DPUMAAAARCSCzel4K6VjB+1lWmwAAACAiESwOZ3GF3MmpksJaWZrAQAAANAigs3p0L8GAAAAiHgEm9MJjIhG/xoAAAAgUhFsTocWGwAAACDiEWxOh2ADAAAARDyCzekw1DMAAAAQ8Qg2rfH7jo+KRosNAAAAELEINq2p2Cf56yR3nJTa03Q1AAAAAE6BYNOaxsfQuvaW3DFGSwEAAABwagSb1gSGeu5rtAwAAAAArYs1XUBE6z9J+sZLUlyS6UoAAAAAtIJg05qUbGnAZNNVAAAAADgNHkUDAAAA4HgEGwAAAACOR7ABAAAA4HgEGwAAAACOR7ABAAAA4HgEGwAAAACOR7ABAAAA4HgEGwAAAACOR7ABAAAA4HgEGwAAAACOR7ABAAAA4HgEGwAAAACOR7ABAAAA4HgEGwAAAACOR7ABAAAA4HgEGwAAAACOR7ABAAAA4Hixpgs4kWVZkqSKigrDlQAAAAAwqTETNGaE1kRcsKmsrJQk5eXlGa4EAAAAQCSorKxUWlpaq8e4rLbEnzDy+/3at2+fUlJS5HK5TJejiooK5eXlqaioSKmpqabLgYNw76AjuH/QEdw/6AjuH3REsO8fy7JUWVmp3Nxcud2t96KJuBYbt9utXr16mS7jJKmpqfzHjTPCvYOO4P5BR3D/oCO4f9ARwbx/TtdS04jBAwAAAAA4HsEGAAAAgOMRbE7D4/HowQcflMfjMV0KHIZ7Bx3B/YOO4P5BR3D/oCNM3j8RN3gAAAAAALQXLTYAAAAAHI9gAwAAAMDxCDYAAAAAHI9gAwAAAMDxCDYAAAAAHI9g04pf//rXys/PV0JCgkaNGqX333/fdEmIQMuWLdOVV16p3NxcuVwuLVq0qNl+y7L00EMPKTc3V4mJiSooKNDGjRvNFIuIUlhYqPPPP18pKSnKysrS1Vdfrc2bNzc7hvsHp/LMM89o+PDhgbd7jx07Vv/4xz8C+7l30B6FhYVyuVyaMWNGYBv3EE7loYceksvlajZlZ2cH9pu6dwg2p/Dyyy9rxowZmjNnjj799FONHz9ekydP1p49e0yXhghz7NgxjRgxQvPmzWtx/89+9jM9+eSTmjdvnlatWqXs7GxdfvnlqqysDHOliDRLly7V9OnTtXLlSi1evFj19fWaNGmSjh07FjiG+wen0qtXLz322GNavXq1Vq9era985Su66qqrAl8euHfQVqtWrdL8+fM1fPjwZtu5h9CaIUOGaP/+/YFp/fr1gX3G7h0LLbrgggusO++8s9m2gQMHWrNmzTJUEZxAkvXaa68F1v1+v5WdnW099thjgW01NTVWWlqa9Zvf/MZAhYhkJSUlliRr6dKllmVx/6D90tPTreeee457B21WWVlp9e/f31q8eLE1ceJE65577rEsi98/aN2DDz5ojRgxosV9Ju8dWmxaUFtbqzVr1mjSpEnNtk+aNEnLly83VBWcaOfOnSouLm52L3k8Hk2cOJF7CScpLy+XJGVkZEji/kHb+Xw+vfTSSzp27JjGjh3LvYM2mz59uqZOnarLLrus2XbuIZzO1q1blZubq/z8fH3961/Xjh07JJm9d2JDenaHOnTokHw+n3r06NFse48ePVRcXGyoKjhR4/3S0r20e/duEyUhQlmWpZkzZ+riiy/W0KFDJXH/4PTWr1+vsWPHqqamRl26dNFrr72mwYMHB748cO+gNS+99JI++eQTrVq16qR9/P5Ba8aMGaM//OEPOuecc3TgwAE98sgjGjdunDZu3Gj03iHYtMLlcjVbtyzrpG1AW3Av4XS+//3va926dfrggw9O2sf9g1MZMGCA1q5dq7KyMr3yyiuaNm2ali5dGtjPvYNTKSoq0j333KN33nlHCQkJpzyOewgtmTx5cmB52LBhGjt2rM466ywtXLhQF154oSQz9w6PorWgW7duiomJOal1pqSk5KT0CbSmcYQQ7iW05u6779Ybb7yh9957T7169Qps5/7B6cTHx+vss8/W6NGjVVhYqBEjRugXv/gF9w5Oa82aNSopKdGoUaMUGxur2NhYLV26VE8//bRiY2MD9wn3ENoiOTlZw4YN09atW43+/iHYtCA+Pl6jRo3S4sWLm21fvHixxo0bZ6gqOFF+fr6ys7Ob3Uu1tbVaunQp9xJkWZa+//3v69VXX9W//vUv5efnN9vP/YP2sixLXq+Xewendemll2r9+vVau3ZtYBo9erS++c1vau3aterXrx/3ENrM6/Xq888/V05OjtHfPzyKdgozZ87UTTfdpNGjR2vs2LGaP3++9uzZozvvvNN0aYgwR48e1bZt2wLrO3fu1Nq1a5WRkaHevXtrxowZmjt3rvr376/+/ftr7ty5SkpK0o033miwakSC6dOn64UXXtDrr7+ulJSUwP+7lZaWpsTExMA7Jbh/0JL7779fkydPVl5eniorK/XSSy9pyZIlevvtt7l3cFopKSmB/nyNkpOTlZmZGdjOPYRT+dGPfqQrr7xSvXv3VklJiR555BFVVFRo2rRpZn//hHTMNYf71a9+ZfXp08eKj4+3zjvvvMAQrEBT7733niXppGnatGmWZdnDHj744INWdna25fF4rAkTJljr1683WzQiQkv3jSRrwYIFgWO4f3Aq3/72twP/RnXv3t269NJLrXfeeSewn3sH7dV0uGfL4h7Cqd1www1WTk6OFRcXZ+Xm5lrXXnuttXHjxsB+U/eOy7IsK7TRCQAAAABCiz42AAAAAByPYAMAAADA8Qg2AAAAAByPYAMAAADA8Qg2AAAAAByPYAMAAADA8Qg2AAAAAByPYAMAAADA8Qg2AAAAAByPYAMAAADA8Qg2AAAAABzv/wNdBS7hqvoFeQAAAABJRU5ErkJggg==",
      "text/plain": [
       "<Figure size 1000x600 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "V = np.zeros(nb_states)\n",
    "Q = np.zeros((nb_states, nb_actions))\n",
    "\n",
    "V_high_history = []\n",
    "V_low_history = []\n",
    "\n",
    "for k in range(50):\n",
    "    \n",
    "    Q[s['high'], a['search']] = alpha * (r_search + gamma * V[s['high']]) + (1 - alpha) * (r_search + gamma*V[s['low']])\n",
    "    Q[s['high'], a['wait']] = r_wait + gamma * V[s['high']]\n",
    "    \n",
    "    Q[s['low'], a['search']] = beta * (r_search + gamma * V[s['low']]) + (1 - beta) * (-3 + gamma*V[s['high']])\n",
    "    Q[s['low'], a['wait']] = r_wait + gamma * V[s['low']]\n",
    "    Q[s['low'], a['recharge']] = gamma * V[s['high']]\n",
    "    \n",
    "    V[s['high']] = Q[s['high'], pi[s['high']]]\n",
    "    V[s['low']] = Q[s['low'], pi[s['low']]]\n",
    "    \n",
    "    V_high_history.append(V[s['high']])\n",
    "    V_low_history.append(V[s['low']])\n",
    "    \n",
    "print(Q)\n",
    "    \n",
    "plt.figure(figsize=(10, 6))\n",
    "plt.plot(V_high_history, label=\"high\")\n",
    "plt.plot(V_low_history, label=\"low\")\n",
    "plt.legend()\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Q:** Do the V-values converge? How fast? What do the final values represent? Change the value of $\\gamma$ and conclude on its importance (do not forget to reset the V and Q arrays to 0!). "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**A:** The V-values converge quite fast (~15 iterations) to their true value. The high state has a higher value than the low state, as there is no risk in that state to receive the punishment of -3.\n",
    "\n",
    "The final value is the expected return in that state, that is:\n",
    "\n",
    "$$R_t = \\sum_{k=0}^\\infty \\gamma^k \\, r_{t+k+1}$$\n",
    "\n",
    "$\\gamma$ completely changes the scale of the return. Small values of $\\gamma$ lead to small returns (only a couple of rewards count in the sum), while high values lead to high returns (there are a lot of rewards to be summed, especially because the task is continuing)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Q:** Print the Q-values at the end of the policy evaluation. What would the greedy policy with respect to these Q-values? \n",
    "\n",
    "**Q:** Change the initial policy to this policy and evaluate it. What happens? Compare the final value of the states under both policies. Which one is the best?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**A:** The greedy policy w.r.t. the Q-values is searching in high, recharging in low, as the Q-values are maximal for these actions.\n",
    "\n",
    "If we evaluate this policy, we observe that:\n",
    "\n",
    "* the value of both states is higher: this is a better policy, as we collect more return on average.\n",
    "* the greedy policy does not change after the evaluation: we have found the optimal policy already!"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Policy iteration\n",
    "\n",
    "Improving the policy is now straightforward. We just to look at the Q-values in each state, and change the policy so that it takes the action with the maximal Q-value. If this does not change the policy (we still take the same actions), we have found the optimal policy, we can stop.\n",
    "\n",
    "**Q:** Implement policy iteration.\n",
    "\n",
    "Do not forget to reset the V and Q arrays at the beginning of the cell, as well as the original policy.\n",
    "\n",
    "Use an infinite loop that you will quit when the policy has not changed between two iterations. Something like:\n",
    "\n",
    "```python\n",
    "while True:\n",
    "    # 1 - Policy evaluation\n",
    "    for k in range(50):\n",
    "        # Update the values\n",
    "    \n",
    "    # 2 - Policy improvement\n",
    "    \n",
    "    if pi != pi_old:\n",
    "        break\n",
    "```\n",
    "\n",
    "**Beware:** if you simply assign the policy to another array and modify the policy:\n",
    "\n",
    "```python\n",
    "pi_old = pi\n",
    "pi[s['high']] = a['search']\n",
    "```\n",
    "\n",
    "`pi_old` will also change! You need to `.copy()` the policy."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Greedy policy after iteration 1\n",
      "pi(high)= 0\n",
      "pi(low)= 2\n",
      "-\n",
      "Greedy policy after iteration 2\n",
      "pi(high)= 0\n",
      "pi(low)= 2\n",
      "-\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAzYAAAH6CAYAAADV+tmnAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjYuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/av/WaAAAACXBIWXMAAA9hAAAPYQGoP6dpAABH0UlEQVR4nO3deXxU9b3/8ffMZDJZSAJJCEkggaCUfVFQK264oYhbbfVWq1LttVqXSunPhdq69SLVVq+3UrW091rvVau3VZF63aiyqKjsiKDsS1hCWLIvk1nO74+TGQgEyISZOWcmr+ej53HO+Z6TOZ/EU8ib8/1+j8MwDEMAAAAAkMCcVhcAAAAAAMeLYAMAAAAg4RFsAAAAACQ8gg0AAACAhEewAQAAAJDwCDYAAAAAEh7BBgAAAEDCI9gAAAAASHgpVhdwqGAwqJ07dyorK0sOh8PqcgAAAABYxDAM1dXVqbi4WE7n0Z/J2C7Y7Ny5UyUlJVaXAQAAAMAmysvL1adPn6OeE3GwWbBggX77299q6dKl2rVrl958801deeWV7Z576623aubMmfr3f/93TZ48uUOfn5WVJcksPjs7O9LyAAAAACSJ2tpalZSUhDPC0UQcbBoaGjRy5EjddNNN+u53v3vE82bNmqUvvvhCxcXFEX1+qPtZdnY2wQYAAABAh4aoRBxsJkyYoAkTJhz1nB07dujOO+/U+++/r4kTJ0Z6CQAAAACISNTH2ASDQd1www265557NHTo0GOe7/V65fV6w/u1tbXRLgkAAABAkov6dM+PP/64UlJS9NOf/rRD50+fPl05OTnhhYkDAAAAAEQqqk9sli5dqv/4j//QsmXLOjxV89SpUzVlypTwfmiA0LEEAgH5fL5O19oVud1uuVwuq8sAAAAAoi6qwebjjz9WZWWlSktLw22BQEA///nP9fTTT2vLli2HfY3H45HH4+nwNQzDUEVFhaqrq6NQcdfTvXt3FRYW8o4gAAAAJJWoBpsbbrhBF1xwQZu2iy66SDfccINuuummqFwjFGoKCgqUkZHBL+gdZBiGGhsbVVlZKUkqKiqyuCIAAAAgeiIONvX19dqwYUN4f/PmzVqxYoVyc3NVWlqqvLy8Nue73W4VFhZq4MCBx11sIBAIh5pDr4NjS09PlyRVVlaqoKCAbmkAAABIGhEHmyVLlujcc88N74fGx0yaNEl/+ctfolZYe0JjajIyMmJ6nWQW+tn5fD6CDQAAAJJGxMFm3LhxMgyjw+e3N67meNH9rPP42QEAACAZRX26ZwAAAACIN4INAAAAgIRHsImTcePGafLkyUc87nA4NGvWrA5/3rx58+RwOJj2GgAAAFCUp3tG5+3atUs9evSwugwAAAAgIRFsbKKwsNDqEgAAAICElfBd0QzDUGOL35IlktnhJCkYDOree+9Vbm6uCgsL9fDDD4ePHdoVbeHChRo1apTS0tI0ZswYzZo1Sw6HQytWrGjzmUuXLtWYMWOUkZGhsWPHau3atcfx0wQAAAASU8I/sWnyBTTkwfctufaaRy9SRmrHf4QvvviipkyZoi+++EKfffaZfvjDH+qMM87QhRde2Oa8uro6XXbZZbrkkkv0yiuvaOvWrUccn/PAAw/oySefVM+ePXXbbbfp5ptv1qeffno83xYAAACQcBI+2CSSESNG6KGHHpIkDRgwQDNmzNCHH354WLB5+eWX5XA49Kc//UlpaWkaMmSIduzYoVtuueWwz5w2bZrOOeccSdL999+viRMnqrm5WWlpabH/hgAAAACbSPhgk+52ac2jF1l27UiMGDGizX5RUZEqKysPO2/t2rUaMWJEm3By6qmnHvMzi4qKJEmVlZUqLS2NqDYAQNcTDBry+oPy+gPm2nfQduvaFzDk8wflDwbVEjDkDwTlC5jtgaAhf9BQIBg01wFzP2iYxwKGoWDQUCAoBY0D7UHD7EputpnHjIPWhsxtGZKhw4+ZPcEP3je7hh841rY95MAxo83+ocfD+zKOcuwoIuup3u71klWEvfhhsV9MHKyTSxNncquEDzYOhyOi7mBWcrvdbfYdDoeCweBh5xmGIYfDcVjbsT4z9DXtfSYAILkYhqF6r1/76lu0t96r6kafapt9qmkyl9omv7lu9qnB61dDS0BNLX41tgRaF7+affx9AeDIapp8VpcQkcRIBF3MoEGD9PLLL8vr9crj8UiSlixZYnFVAIB4qmn0aev+Bm3d16ht+xu1vapRlbVe7W1o0d46r/bWe+X1Ry+YpDgd8qQ45XG7zHWKU25X65LiVKrLoRSnue12OpTicijF5VSK0yGX0yGXw2wLbTtb1y7ngW2n0yGnQ3I6zLXD4QhvOx0OOVrbHJIcR2hzqLXtoP3W/0k68I98B/bNxWxzhNt0yPnh/UN+Lo5DG9qc6+jQeYd/XedEcg0gGoYWZ1tdQkQINjZ03XXX6YEHHtCPf/xj3X///dq2bZt+97vfSTr8D2AAQOIKBg1t29+obyrqtLaiTusr67Rtf6O27mvs8L+UZqS6lNctVT0yUpWT7lZ2mlvZ6W5zOz1FOeludfOkKCM1RRmprtbF3E5v3U91OZXiSviJUgF0cQQbG8rOztY//vEP/eQnP9GoUaM0fPhwPfjgg7ruuuuYFAAAElQwaGjNrlp9sXm/1lbUam1FndbtrleTL3DEr+mZ5VHf3AyV5mWopEeGemWnKb9bqvKzPOrZzaO8bqkJ0x0bAGKNPw3jZN68eYe1HfzemkPH0IwdO1YrV64M77/88styu93hSQHGjRt32NeMGjUq4nfrAABip7bZp0/W79Xcbyo1b90e7anzHnZOaopTAwq6aWBhlgb2ylJZfqZK8zJUmptBaAGACPAnpk3993//t/r376/evXtr5cqVuu+++3TNNdcoPT3d6tIAAEexcU+95qzZrbnfVGrp1ir5gwf+wSkj1aVv98/TsOJsDSzM1sDCLPXLy6AbGABEAcHGpioqKvTggw+qoqJCRUVFuvrqqzVt2jSrywIAtGPTnnq9s2qX3v5yl76pqGtzrH/PTJ07sEDnDizQKWU95EmJ7FUBAICOIdjY1L333qt7773X6jIAAEewZW+D/m/VLv3fl7u0ZldtuD3F6dDYE/N1/qACjRvYU33zMi2sEgC6DoINAAAR2LSnXr966yt9umFfuC3F6dAZJ+Zr4vAijR/aS90zUi2sEAC6JoINAAAd0OIP6o/zN+qZuRvU4g/K5XRo7Al5unREkcYPKVSPTMIMAFiJYAMAwDEs3VqlqW98qXW76yVJ53yrp/7tymEqyc2wuDIAQAjBBgCAI6hr9umJ99bqpS+2yjCkvMxUPXjZEF0+spgXJgOAzRBsAAA4hGEYen/1bj08e7UqapslSd8b3UcPXDKYLmcAYFMEGwAADrJxT70e+ccaLVi3R5LUNy9D078zXGNPzLe4MgDA0RBs4mTcuHEaNWqUnn76aatLAQC0o97r1zMfrtd/fbpZvoChVJdTt5xdprvOG6A0N++eAQC7I9gAALo0wzA0a8UOTX/nG1XWeSVJ5w0q0K8uHaKyfN5BAwCJgmADAOiyVu+s0UNvrdaSrVWSzG5nD146ROcP7mVxZQCASDmtLuC4GYbU0mDNYhidKrmqqko33nijevTooYyMDE2YMEHr169v/XYM9ezZU6+//nr4/FGjRqmgoCC8/9lnn8ntdqu+vv74fnYA0IWt3lmj7zy7UEu2Vind7dI9Fw3U+5PPJtQAQIJK/Cc2vkbpsWJrrv2LnVJq5N0UfvjDH2r9+vWaPXu2srOzdd999+mSSy7RmjVr5Ha7dfbZZ2vevHn67ne/q6qqKq1Zs0aZmZlas2aNhgwZonnz5mn06NHq1q1bDL4pAEh+Lf6gfv6/K9XiD+r0/nl68pqRKu6ebnVZAIDjkPhPbBJMKND8+c9/1llnnaWRI0fq5Zdf1o4dOzRr1ixJ5kQD8+bNkyQtWLBAI0eO1HnnnRdumzdvnsaNG2dJ/QCQDGZ8tF7fVNQpNzNVz1x3EqEGAJJA4j+xcWeYT06sunaEvv76a6WkpOi0004Lt+Xl5WngwIH6+uuvJZnB5u6779bevXs1f/58jRs3TqWlpZo/f75+/OMfa+HChZo8eXK0vgsA6FJWba/RH+ZtlCT9+ophyu/msbgiAEA0JH6wcTg61R3MKsYRxuUYhhF+i/WwYcOUl5en+fPna/78+Xr00UdVUlKiadOmafHixWpqatKZZ54Zz7IBICl4/QH9v7+tVCBoaOKIIk0cUWR1SQCAKKErWpwNGTJEfr9fX3zxRbht3759WrdunQYPHixJcjgcOvvss/XWW2/pq6++0llnnaXhw4fL5/Pp+eef18knn6ysrCyrvgUASFi//3C91u6uU363VP36imFWlwMAiCKCTZwNGDBAV1xxhW655RZ98sknWrlypa6//nr17t1bV1xxRfi8cePG6ZVXXtGIESOUnZ0dDjsvv/wy42sAoBNWllfrudYuaP925XDlZqZaXBEAIJoINhZ44YUXNHr0aF166aU6/fTTZRiG3nnnHbnd7vA55557rgKBQJsQc8455ygQCOicc86xoGoASFzNvoB+/reVChrS5SOLdfGwQqtLAgBEmcM40qAPi9TW1ionJ0c1NTXKzs5uc6y5uVmbN29WWVmZ0tLSLKowsfEzBNAV/ebdb/T8/I3K7+bRnJ+drR48rQGAhHC0bHAontgAAJLasm1VmrnA7IL22HeGEWoAIEkRbAAASau22af/19oF7aqTemv8ULqgAUCyItgAAJLSxj31uvIPn2rTngYVZHn00GVDrS4JABBDif8eGwAADvHh17s1+dUVqvP6VZyTppk3jlFOhvvYXwgASFgJGWyCwaDVJSQsfnYAkplhGHp23kb97oO1Mgzp1H65evb6k5XfzWN1aQCAGEuoYJOamiqn06mdO3eqZ8+eSk1NlcPhsLqshGAYhlpaWrRnzx45nU6lpjJ4FkByafD6dc/fV+qdVRWSpOu/XaoHLx2q1BR6XQNAV5BQwcbpdKqsrEy7du3Szp07rS4nIWVkZKi0tFROJ3/RA0ge5fsbdct/L9E3FXVyuxx69IphuvbUUqvLAgDEUUIFG8l8alNaWiq/369AIGB1OQnF5XIpJSWFp1wAksrWfQ268g+fqqrRp/xuHj1//cka0y/X6rIAAHGWcMFGkhwOh9xut9xuBoICQFf3tyXbVdXo06DCLL1w0ykqykm3uiQAgAXojwQASGgLN+6VJN18ZhmhBgC6MIINACBh1TX7tHJ7jSRp7Al5FlcDALBSxMFmwYIFuuyyy1RcXCyHw6FZs2aFj/l8Pt13330aPny4MjMzVVxcrBtvvJGB/gCAmFi8Zb8CQUN98zLUp0eG1eUAACwUcbBpaGjQyJEjNWPGjMOONTY2atmyZfrVr36lZcuW6Y033tC6det0+eWXR6VYAAAO9umGfZKksSfkW1wJAMBqEU8eMGHCBE2YMKHdYzk5OZozZ06btmeeeUannnqqtm3bptJSpt4EAETPwo2hYEM3NADo6mI+K1pNTY0cDoe6d+/e7nGv1yuv1xver62tjXVJAIAksK/eq693mX9nnE6wAYAuL6aTBzQ3N+v+++/Xddddp+zs7HbPmT59unJycsJLSUlJLEsCACSJzzftlyQNKsxSfjePxdUAAKwWs2Dj8/n0/e9/X8FgUM8+++wRz5s6dapqamrCS3l5eaxKAgAkkU9bp3lmfA0AQIpRVzSfz6drrrlGmzdv1kcffXTEpzWS5PF45PHwL20AgMh8xvgaAMBBoh5sQqFm/fr1mjt3rvLy+AsHABBdO6ubtHlvg1xOh07rn2t1OQAAG4g42NTX12vDhg3h/c2bN2vFihXKzc1VcXGxvve972nZsmV6++23FQgEVFFRIUnKzc1Vampq9CoHAHRZodnQhvfOUVaa2+JqAAB2EHGwWbJkic4999zw/pQpUyRJkyZN0sMPP6zZs2dLkkaNGtXm6+bOnatx48Z1vlIAAFot3GCOrznjRHoFAABMEQebcePGyTCMIx4/2jEAAI6XYRgHvb+GiQMAAKaYTvcMAEC0bdrboIraZqWmODW6bw+rywEA2ATBBgCQUEJPa0aX9lCa22VxNQAAuyDYAAASCuNrAADtIdgAABJGMGjos03mE5vTGV8DADgIwQYAkDDW7KpVdaNP3TwpGtknx+pyAAA2QrABACSMz1rH15xalqsUF3+FAQAO4G8FAEDC+HSjOb5m7AmMrwEAtEWwAQAkBF8gqEWb90vi/TUAgMMRbAAACWFlebUaWwLKzUzVoMIsq8sBANgMwQYAkBBC7685vX+enE6HxdUAAOyGYAMASAiftr6/ZizvrwEAtINgAwCwvaaWgJZvq5bE+BoAQPtSrC4AsEogaMgXCMrrDyoQNOQPBOUPGuH2QNAI7weChgKGoWBrWzC0b0hBw5BhGAoED2yH2oOGZBiGjPB5rWtJMiRDZpshta4P7Ct0XujYIfshB9oOajzkvANfeeSvb087H3nQ1x3tKzvvaNdE11VR06yWQFDFOWnql5dhdTkAABsi2MDWAkFDtU0+VTf5VNXYopomnxq8fjV4/ar3Bg7a9quxJaCmloCa/QE1+wJq9gXV7AvI6w/K6wuoJRBUiz+olkBQvoAZVgAklrEn5svhYHwNAOBwBBvEnWEYqm70aXddsypqmlVZ69Xu2mZV1DZrd61Xe+q9qmlsUVWjT7XNvrj9C77DIbmdTrmcDqW4HEpxOuRyOlvXDjmdksthbrucDjkdB9ZOh+QI75vbDsk85pQccshxULvDYR4LbSt0vHU/dL4O2ldoP1yv46Dtw4+Fv6/DNtr53g85ePDvjcfzK2Si/P556PcPe0pzO/WvZ/W3ugwAgE0RbBAzNU0+bd7boM1767V5T4M272sMbze0BCL6rG6eFHXPcCsn3a1MT4q6eVJa1y5lpprbmR6X0t0uedwupbldSktxmmu3S2lup1JTnHK7nEp1HbSdYgaXVJeTWZYAAAASGMEGUVHT5NOq7TVaub1aK8qr9eX2au2u9R71a3IzU1WQ5VFhTpp6ZaWpV06aemV71LObRz0yU9Ujw62c9FR1z3DL7WKeCwAAABwZwQadsqO6SXO/qdTSrVVaub1am/Y0tHter2yPyvIzVZbfTWX5Ga3rTPXpka40tyvOVQMAACBZEWzQIcGgoS931OjDr3frn19X6utdtYedU5qboZEl3TWyT45GlXTXoKJsdfNwiwEAACD2+K0TR+QPBLVg/R59sHq3PvymUnvqDnQtczqkk0t76IwT8zWqtLtG9umu3MxUC6sFAABAV0awwWGqG1v06uJy/c9nW7Wjuinc3s2TorO/la/zB/XSuYMKCDIAAACwDYINwr7eVasXF27RrBU71OwLSpJ6ZLh1+chiXTCkl04ry1NqCoP4AQAAYD8Emy4uGDT0wZrd+svCzfp80/5w+5CibP3wjH66fGQxg/wBAABgewSbLmxtRZ1+OWuVFm+pkiS5nA5dPLRQk8b20yn9evB2bwAAACQMgk0X1Nji1+8/3KA/f7xJ/qChjFSXJo3tpxu+3VfF3dOtLg8AAACIGMGmi/nw69168K3V4UkBxg/ppYcuH6reBBoAAAAkMIJNF7GzukmP/GO13l+9W5LUu3u6Hr58qC4c0sviygAAAIDjR7DpAt5asUNT31ilxpaAUpwO/eisMt19/gBlpPKfHwAAAMmB32yTmGEYenbeRv32/bWSpDF9e+jfvjNMgwqzLa4MAAAAiC6CTZLyBYL65Ztf6bUl5ZKkW84q09QJg+V0MtMZAAAAkg/BJgnVNft0+8vL9PH6vXI6pIcvH6obT+9ndVkAAABAzBBsksyumibd9MJifVNRp3S3SzOuO0nnD2aCAAAAACQ3gk0SWb2zRjf/ZbF213rVM8uj/5p0iob3ybG6LAAAACDmCDZJYuHGvbrlxSVqaAloQEE3vXDTKerTI8PqsgAAAIC4INgkgfL9jfrJS8vU0BLQ2BPy9Nz1o5WT7ra6LAAAACBuCDYJrtkX0G0vLVVNk08jS7rrhZtOkSfFZXVZAAAAQFw5rS4AnWcYhn456yut3lmr3MxUPfeDkwk1AAAA6JIINgnsr4vK9fel2+V0SM9ce5KKu6dbXRIAAABgCYJNglpRXq2HZ6+WJN1z0SCdcWK+xRUBAAAA1iHYJKB99V7d/tJStQSCumhoL912Tn+rSwIAAAAsRbBJMP5AUHf9dbl21jSrf36mfnv1SDkcDqvLAgAAACxFsEkwT85Zp4Ub9ykj1aXnbxit7DSmdQYAAAAINgnk/dUVem7eRknS498doW/1yrK4IgAAAMAeCDYJoq7Zpwfe/EqS9KMzy3TZyGKLKwIAAADsg2CTIJ75aIP21nvVPz9T9108yOpyAAAAAFsh2CSATXvq9cKnmyVJv7p0iFJT+M8GAAAAHIzfkBPAr99eI1/A0LkDe+rcQQVWlwMAAADYTsTBZsGCBbrssstUXFwsh8OhWbNmtTluGIYefvhhFRcXKz09XePGjdPq1aujVW+XM/ebSs1du0dul0O/unSI1eUAAAAAthRxsGloaNDIkSM1Y8aMdo8/8cQTeuqppzRjxgwtXrxYhYWFuvDCC1VXV3fcxXY1Lf6gfv32GknSTWeUqX/PbhZXBAAAANhTSqRfMGHCBE2YMKHdY4Zh6Omnn9YDDzygq666SpL04osvqlevXnrllVd06623Hl+1XcyLC7do094G5Xfz6K7zTrS6HAAAAMC2ojrGZvPmzaqoqND48ePDbR6PR+ecc44WLlzY7td4vV7V1ta2WSDtqfPq9x+ulyTde/FAZfEiTgAAAOCIohpsKioqJEm9evVq096rV6/wsUNNnz5dOTk54aWkpCSaJSWs377/jeq8fo3ok6PvndzH6nIAAAAAW4vJrGgOh6PNvmEYh7WFTJ06VTU1NeGlvLw8FiUllC+3V+tvS7dLkh66bKiczvZ/dgAAAABMEY+xOZrCwkJJ5pOboqKicHtlZeVhT3FCPB6PPB5PNMtIaIZh6OHZq2UY0ndO6q3RfXtYXRIAAABge1F9YlNWVqbCwkLNmTMn3NbS0qL58+dr7Nix0bxU0pq1YoeWbatWRqpL908YZHU5AAAAQEKI+IlNfX29NmzYEN7fvHmzVqxYodzcXJWWlmry5Ml67LHHNGDAAA0YMECPPfaYMjIydN1110W18GQUDBp68oN1kqQ7zj1RvbLTLK4IAAAASAwRB5slS5bo3HPPDe9PmTJFkjRp0iT95S9/0b333qumpibdfvvtqqqq0mmnnaYPPvhAWVlZ0as6SS3esl/bq5qU5UnRj84ss7ocAAAAIGE4DMMwrC7iYLW1tcrJyVFNTY2ys7OtLieu7n/9S726uFzXjOmjJ7430upyAAAAAEtFkg1iMisaItfsC+j/Vu2SJH3nJKZ3BgAAACJBsLGJj76pVF2zX8U5aTqtLNfqcgAAAICEQrCxiTeW7ZAkXXFSb95bAwAAAESIYGMD+xtaNG9tpSTpqpN6W1wNAAAAkHgINjbw9pc75Q8aGtY7WwN6MXscAAAAECmCjQ28udzshnblKJ7WAAAAAJ1BsLHY5r0NWr6tWk6HdPmoYqvLAQAAABISwcZioac1Zw3oqYKsNIurAQAAABJTitUFdGWGYWhWa7D5DpMGAADQlmGYiwzJCB603d5aR9g22n7eoZ/ftuEox4/1tUf5nI6y1zvTj0OyfB9Qeq7kTpx/eCfYWGjZtipt29+ojFSXxg/tZXU5AIBk5G+RWuqlloYDi+/g7SbJ3yz5vW3XgRZzO+Azt4O+1u3Qvr+dJXBgbQSkYLB13bpvBFu3Q0GlnaW9QALAGj/4uzTgQqur6DCCjYVC7665eFihMlL5TwEAOAZ/i9SwR6qvkOorpfrdUt1uc91cLTXXSs01bRd/k9VVJ7GD3jvniOQddJ18X11E1wCiIbHuOX6btojXH9DbX+6SJF11Uh+LqwEA2EJzjVS9TaouN9c15VL1VnO/plxq3Nf5z3Z5pNRMKbWblJphbrtb1yke83iKR0pJO7B2pUoud+s6VXKlmGun29x2uiVnSuviPLDtcElOV+va2Xbf4WxdHAdtH9ImRzvbjgPbR1xL4V/EDt7v0DEdcgxAoiHYWGTuN3tU0+RTr2yPTj8hz+pyAADx1lQl7Vwh7VzeuqyQarYd++ucKVJmgdStQMoqNNfdepl94dNy2lmypdQsM4gAQBLjTzmLhCYNuGJUb7mc/OsQACS1gF+q+FLa9rm0fZEZZKq2tH9uRp6UUyJ1L5G6923dLpVy+kjZxWaAcTKpKQAcimBjgZpGnz76plISs6EBQFLy1ks7lkhbP5O2fSZtX2IO2D9UjzKp+CSpeJS5LhwhpXePd7UAkBQINhZ4e9VOtQSCGlSYpcFF2VaXAwCIBsOQNs2TPn9W2vChOQvYwdJypJJvS6WnScUnS0UjpYxcS0oFgGREsLHAm8t4dw0AJA2/V1r1d+mzP0iVqw+055RIpd9uXcZKPQfRhQwAYohgE2c7q5u0ZGuVHA5zfA0AIEE17JOW/Je0aKbUYHYvljtTOul66dQfS/knWlsfAHQxBJs4e++rCknS6NIeKsxJnDe5AgBa7VopLf6z9OX/mi+ylKSsYum0W6XRk6T0HtbWBwBdFMEmzkLBZsLwIosrAQB0mK9ZWjPLDDTbFx9oLxoljb1LGnKF+b4XAIBlCDZxVFnXrMVb90uSLh5WaHE1AIBj2r/Z7G62/CWpyfzzW063NORy6ZR/lUpP54WOAGATBJs4en/1bhmGNLJPjnp3T7e6HADAkfiapbdul756Q5JhtuWUSKN/KJ18o/lSTACArRBs4ui9r3ZJohsaANiav0X63xul9e+b+ydeYD6dGTBecrqsrQ0AcEQEmzjZ39CizzeZ3Rgm0A0NAOwp4Jde/5EZalLSpR/8TSo7y+qqAAAdwIT6cTJnTYUCQUODi7LVNy/T6nIAAIcKBs3uZ1/Pllyp0vdfJtQAQAIh2MTJu62zoV3C0xoAsB/DkP7vZ9KXr0nOFOnqF6UTz7e6KgBABAg2cVDT5NOnG/ZKkiYMJ9gAgK0YhvTeVGnpXySHU7pqpjToEqurAgBEiGATBx99s1u+gKETC7rpxIIsq8sBABzso19LXzxnbl8+Qxr2XWvrAQB0CpMHxME7q+iGBgC2YxjSgt9KHz9p7l/yO+mkH1hbEwCg0wg2Mdbg9WvBuj2SpIuHMc0zANhC+SKz+9mOJeb+hb+WTr3F2poAAMeFYBNjc9dWyusPqm9ehgYX0Q0NACxVvU3658PSV6+b++5M6YKHpNNutbQsAMDxI9jE2Lut3dAmDCuSw+GwuBoA6KK8ddIn/y599gfJ3yzJYXY7O+9XUhbdhAEgGRBsYqjZF9DctZWSeCknAFhmxV+lOQ9KDeafx+p3lnTRNKlopLV1AQCiimATQ/PX7VFjS0C9u6drRJ8cq8sBgK5n41xp1m3mdm5/cyzNoIkST9ABIOkQbGLovdaXcl48rJBuaABghWUvmuvhV0tXPCulpFpbDwAgZniPTYx4/QH9c81uSXRDAwBLNFVJ37xjbp9+J6EGAJIcwSZGFm7YpzqvXwVZHp1c2sPqcgCg61n9phTwSgVDGE8DAF0AwSZG3v1qlySzG5rTSTc0AIi7FX811yOvZUwNAHQBBJsY8AeC+qC1G9rFdEMDgPjbu0HavkhyOKUR11hdDQAgDgg2MbBpb4OqG33q5knRqf1yrS4HALqela1Pa044n/fUAEAXQbCJgc17GyRJ/XtmKsXFjxgA4ioYlL58zdweda21tQAA4obfumNgS2uw6ZeXaXElANAFbflYqimXPDnSwIlWVwMAiBOCTQxs2dcabPIJNgAQd6FuaMO+I7nTrK0FABA3BJsYCHVFK8vPsLgSAOhivPXSmtnm9sjrrK0FABBXBJsY2ExXNACwxtezJV+DlHuCVHKq1dUAAOKIYBNljS1+7a71SpLK6IoGAPG14hVzzbtrAKDLiXqw8fv9+uUvf6mysjKlp6erf//+evTRRxUMBqN9KVvasrdRktQjw63uGakWVwMAXUj1NnPiAEka+S/W1gIAiLuUaH/g448/rueff14vvviihg4dqiVLluimm25STk6O7r777mhfznaYOAAALLKydYrnfmdJ3UutrQUAEHdRDzafffaZrrjiCk2caE6x2a9fP/31r3/VkiVLon0pWwpPHMD4GgCIH8M4MBvaKCYNAICuKOpd0c4880x9+OGHWrdunSRp5cqV+uSTT3TJJZe0e77X61VtbW2bJZGFJw7giQ0AxE/5Imn/RsmdKQ2+3OpqAAAWiPoTm/vuu081NTUaNGiQXC6XAoGApk2bpmuvbf/tz9OnT9cjjzwS7TIss4VgAwDxt7J10oAhl0uebtbWAgCwRNSDzWuvvaaXXnpJr7zyioYOHaoVK1Zo8uTJKi4u1qRJkw47f+rUqZoyZUp4v7a2ViUlJdEuK25CY2z6E2wAILr2bpBa6g5vDwalr940t0e2/49oAIDkF/Vgc8899+j+++/X97//fUnS8OHDtXXrVk2fPr3dYOPxeOTxeKJdhiXqmn3aW98iiSc2ABBVy1+W3rr96OfklJgTBwAAuqSoB5vGxkY5nW2H7rhcri4x3XNoquf8bh5180T9RwsAXVf55+bakyN5sg4/7kqRxv1CcvJ6NgDoqqL+2/dll12madOmqbS0VEOHDtXy5cv11FNP6eabb472pWxn0956SVJZfobFlQBAkqnaaq4n/IZZzwAA7Yp6sHnmmWf0q1/9SrfffrsqKytVXFysW2+9VQ8++GC0L2U7oSc2/ZjqGQCiq7o12HTva20dAADbinqwycrK0tNPP62nn3462h9te6GJA8p6EmwAIGqCAalmu7ndg2ADAGgfnZGjiJdzAkAM1O6Ugn7J6ZayiqyuBgBgUwSbKAo9sWFGNACIoupt5jqnj+R0WVsLAMC2CDZRUtXQoupGnyTG2ABAVIXH15RaWwcAwNYINlGyufVpTWF2mtJT+RdFAIia0IxojK8BABwFwSZKtoTG19ANDQCiK9QVjRnRAABHQbCJklCwYXwNAEQZUz0DADqAYBMlm/eZ77Dh5ZwAEGWhJzZ0RQMAHAXBJko2762XxMQBABBVAZ9Uu8PcZvIAAMBREGyiwDAMbdkbemJDsAGAqKnZLhlBKSVN6tbL6moAADZGsImCvfUtqvf65XBIJbl0RQOAqAmNr8kpkRwOa2sBANgawSYKQi/m7N09XWlupnoGgKhhqmcAQAcRbKJgM1M9A0BsMNUzAKCDCDZREAo2TBwAAFEWnuqZiQMAAEdHsIkC3mEDADHCVM8AgA4i2ETBga5oTBwAAFFVxRMbAEDHEGyOUzBoaGv45ZzdLK4GAJKIr0mqrzC3u/eztBQAgP0RbI7T7rpmNfkCcjkd6tMj3epyACB51Gw316ndpIxca2sBANgeweY4hbqhlfRIl9vFjxMAoubgbmi8wwYAcAz8Jn6ctuw1u6ExcQAARFl4RjQmDgAAHBvB5jiFXs7JVM8AEGVM9QwAiADB5jiFuqL170mwAYCoYqpnAEAECDbHiZdzAkCMVNEVDQDQcQSb4xAIGtoWnuqZYAMAUUVXNABABAg2x2FndZNaAkGlupwq7s5UzwAQNd56qXGfuU2wAQB0AMHmOIQmDijJTZfLyVSkABA1ofE1aTlSendLSwEAJAaCzXHY0jq+piy/m8WVAECSCQUbxtcAADqIYHMcNoWDTYbFlQBAkgmNr2FGNABABxFsjkPoiQ0v5wSAKGNGNABAhAg2x2FLaEY0pnoGgOiqJtgAACJDsOkkfyCo8v1msOGJDQBEGVM9AwAiRLDppO1VTfIHDXlSnCrMTrO6HABILqHJAxhjAwDoIIJNJy1Yv0eSNKgoW06megaA6GmqlpprzG2e2AAAOohg00mvL9shSbp8ZLHFlQBAkgk9rcnIl1Lp6gsA6BiCTSdsqKzXyvJquZwOXTGKYAMAUcVUzwCATiDYdMIby7ZLksZ9q6fyu3ksrgYAkkwVEwcAACJHsIlQMGjozeVmN7SrTu5jcTUAkIRCXdGY6hkAEAGCTYQ+37RPu2qalZ2WovMHF1hdDgAkH7qiAQA6gWATob+3dkO7dGSx0twui6sBgCQUfmJDVzQAQMcRbCLQ4PXrva8qJEnfpRsaAESfYRw0xqafpaUAABILwSYC731VocaWgPrlZejk0u5WlwMAyadxn+RrMLdz+AckAEDHEWwi8MZysxvaVSf3kcPBSzkBIOpC42uyiiR3mrW1AAASCsGmg3ZWN2nhxn2SpO+c1NviagAgSYW7oTFxAAAgMgSbDnpz+Q4ZhnRaWa5KcjOsLgcAkhMTBwAAOolg0wGGYYRfysmkAQAQQ0z1DADoJIJNB3y5vUYb9zQoze3UhOGFVpcDAMmLJzYAgE4i2HTA661Pay4aWqisNLfF1QBAEmOMDQCgkwg2x9DiD2r2yp2S6IYGADEVDB54YkNXNABAhGISbHbs2KHrr79eeXl5ysjI0KhRo7R06dJYXCrmPvqmUtWNPhVkeXTGiflWlwMAyauhUgp4JYdTymb2SQBAZFKi/YFVVVU644wzdO655+rdd99VQUGBNm7cqO7du0f7UnERmjTgOyf1lsvJu2sAIGZC3dCy+0guuv0CACIT9WDz+OOPq6SkRC+88EK4rV+/fkc83+v1yuv1hvdra2ujXVKn7W9o0dy1lZLMl3ICAGKIiQMAAMch6sFm9uzZuuiii3T11Vdr/vz56t27t26//Xbdcsst7Z4/ffp0PfLII9EuIyr+sXKnfAFDw3pna2BhltXlwAqGIRlBKRiQjIC5DvoPagseZTEObMto2ybjkG0daAuvD2kL1dOmXe0fO7j+Q9va+x7bNnTwZ3PcJyS+w352OC6b5plrxtcAADrBYRjR/Zs5LS1NkjRlyhRdffXVWrRokSZPnqw//vGPuvHGGw87v70nNiUlJaqpqVF2dnY0S4vY1n0N+tuS7erfM5MnNnbga5KaqqWWeslb17qul1oapJY6c+1rMhd/80HrRsnvNZeATwq0mP34Q9v+FinoMwNL0C8FWtehNgDxNe4X0rj7rK4CAGADtbW1ysnJ6VA2iHqwSU1N1ZgxY7Rw4cJw209/+lMtXrxYn3322TG/PpLikeCCAalhj1S3S6qrkGp3muv63VJT1YGlcb+59jdZXfHhHM4jL3JIzta1wyk5HAfaD92WQ3LokP3QmK6D2qR22tXOMbVtP+axo36TRzl0POPOGLOGdqTlSJf9h5RbZnUlAAAbiCQbRL0rWlFRkYYMGdKmbfDgwXr99dejfSkkAl+TtH+TtG+DtG9j67LB7Etfv9vs3hUJh1NKzZI83aTUbgfWqd2k1EzJnX5gSUmX3GkH1i6POSA5xSO5Us3tUJvLLTlTJKdbcroO2m9dHE6z3eFqbXMdCCsAAACwXNSDzRlnnKG1a9e2aVu3bp369qXPdNJr2CvtWNq6LJP2fCPVlB/9axxOKbNAyiqUsovNdbdeUnqulJErpfcwl4xcs82TRZgAAADAYaIebH72s59p7Nixeuyxx3TNNddo0aJFmjlzpmbOnBntS8FKwYAZYMq/OBBmQjMaHSotR8o70VxyT5DyTpB6lEnZRWaocUX9NgQAAEAXE/UxNpL09ttva+rUqVq/fr3Kyso0ZcqUI86KdijG2NhYc6208SNp3XvS+g+kxn2Hn5M/UOo9Wup9slQ4XMobYD5t4SkLAAAAImTp5AHHi2BjM9XbpLXvmsuWT8yZwkLScqR+Z7UGmdFS8SizDQAAAIgCSycPQBIIBqVNH0mfPy9tmNP2WO4J0sAJ0rculkq/zdvBAQAAYAsEGxzgrZdW/lVaNFPau6610SH1HWsGmYETpPwBlpYIAAAAtIdgA2n/Zmnxn6Vl/yN5a8y21CzppOulU28xB/sDAAAANkaw6cqaa6SPpkmL/yQZQbMt9wTptFulUdeZUysDAAAACYBg0xUZhrTq79L7v5AaKs22E86TTvuJdOIFktNpbX0AAABAhAg2Xc2eddI7P5c2LzD3806UJj4p9R9naVkAAADA8SDYdBUtjdLHv5M+/b05ZXNKmnT2/5PG/lRK8VhdHQAAAHBcCDZdQfki6fUfme+kkaQB46UJT0i5ZdbWBQAAAEQJwSbZff22GWr8zVJ2H2nC49KgiZLDYXVlAAAAQNQQbJLZoj9J795rznj2rYul7/6n5OlmdVUAAABA1BFskpFhSB8+In3y7+b+6B9KlzwpufjPDQAAgOTEb7rJxt8izb5T+vI1c//cX5qTBND1DAAAAEmMYJNMmmul/71B2jRPcriky38vnXS91VUBAAAAMUewSRZ1FdJL35N2r5LcmdI1/y0NuMDqqgAAAIC4INgkg5YG6eXWUJPZU/rB36Tik6yuCgAAAIgbgk2iMwxp1u1SxSopI1/60QdSbn+rqwIAAADiyml1AThOH/9OWjNLcrqlf3mJUAMAAIAuiWCTyL55R/ro38ztib+T+p5ubT0AAACARQg2iarya+mNW8ztU24x31UDAAAAdFEEm0TUuF/667VSS73U7yzp4ulWVwQAAABYimCTaAJ+6e83SVWbpe6l0tUvSi631VUBAAAAliLYJJoPfmm+gNOdKV37qpSZZ3VFAAAAgOUINolkxSvSF8+Z21f9Ueo11Np6AAAAAJsg2CSKpmrpvanm9jn3S4Mvs7QcAAAAwE4INoli4e+l5mqp5yDpnHutrgYAAACwFYJNIqirkD571tw+/0HJ6bK2HgAAAMBmCDaJYP7jkr9J6nOqNPASq6sBAAAAbIdgY3f7NkpLXzS3L3hYcjgsLQcAAACwI4KN3X30b5IRkAaMl/qdYXU1AAAAgC0RbOxs53Jp9RuSHNL5D1ldDQAAAGBbBBs7++cj5nr41VLhMGtrAQAAAGyMYGNXm+ZJm+ZKTrd03gNWVwMAAADYGsHGjgxD+ufD5vaYm6Ue/aysBgAAALA9go0drXnLHF/jzpTOvsfqagAAAADbI9jYTcAvffRrc3vsnVK3ntbWAwAAACQAgo3dLP8fad8GKSNPOv1Oq6sBAAAAEgLBxk4CfmnB78zts++R0rKtrQcAAABIEAQbO/nmbal2u5SRL42+yepqAAAAgIRBsLGTL/5orsfcLLnTrK0FAAAASCAEG7vYtVLatlByppjBBgAAAECHEWzs4ouZ5nrIlVJ2kaWlAAAAAImGYGMHDXulVX8zt0+7zdpaAAAAgAREsLGDpS9IAa9UfLLUZ4zV1QAAAAAJh2BjtYBPWvyf5vZpt0kOh7X1AAAAAAmIYGO1r2dLdbukzAJp6JVWVwMAAAAkJIKN1Q6e4jnFY20tAAAAQIKKebCZPn26HA6HJk+eHOtLJZ4dy6TyLySnmymeAQAAgOMQ02CzePFizZw5UyNGjIjlZRLXotYpnoddJWX1srYWAAAAIIHFLNjU19frBz/4gf70pz+pR48esbpM4qqvlL563dw+7VZrawEAAAASXMyCzR133KGJEyfqggsuOOp5Xq9XtbW1bZYuYelfpECL1OcUqfdoq6sBAAAAElpKLD701Vdf1bJly7R48eJjnjt9+nQ98sgjsSjDvvwtbad4BgAAAHBcov7Epry8XHfffbdeeuklpaWlHfP8qVOnqqamJryUl5dHuyT7+Xq2VF8hdSuUBl9udTUAAABAwov6E5ulS5eqsrJSo0cf6F4VCAS0YMECzZgxQ16vVy6XK3zM4/HI4+li0xx/8by5PuVHUkqqtbUAAAAASSDqweb888/XqlWr2rTddNNNGjRokO677742oaZLqq+Uti+W5JBG/9DqagAAAICkEPVgk5WVpWHDhrVpy8zMVF5e3mHtXdK2z811wRCpW4G1tQAAAABJIuYv6MQhyr8w16WnWVsHAAAAkERiMivaoebNmxePyySG0BOb0tOtrQMAAABIIjyxiaeWRmnXCnO7hCc2AAAAQLQQbOJp5zIp6JeyiqTupVZXAwAAACQNgk08hbqhlZwmORzW1gIAAAAkEYJNPDG+BgAAAIgJgk28BIPS9kXmNjOiAQAAAFFFsImXPd9IzTWSO1PqNdzqagAAAICkQrCJl/LWbmh9RkuuuMyyDQAAAHQZBJt4YXwNAAAAEDMEm3g5eEY0AAAAAFFFsImH2l1S9VbJ4ZT6nGJ1NQAAAEDSIdjEQ2h8TcFQKS3b2loAAACAJESwiYdtX5jr0m9bWwcAAACQpAg28RB6YkOwAQAAAGKCYBNrLQ3Sri/NbSYOAAAAAGKCYBNr25dIRkDK7i11L7G6GgAAACApEWxirZzxNQAAAECsEWxiLfz+GoINAAAAECsEm1gKBqTti83tUsbXAAAAALFCsImlyjWSt1ZK7Wa+wwYAAABATBBsYinUDa3PKZIrxdpaAAAAgCRGsIklJg4AAAAA4oJgE0vhiQMYXwMAAADEEsEmVmp2SDXlksMl9RljdTUAAABAUiPYxEp569OawmGSJ8vaWgAAAIAkR7CJlW2t42t4fw0AAAAQcwSbWNn2mbnm/TUAAABAzBFsYqG5Vtr9lbnNExsAAAAg5gg2sbBprmQEpdz+Uk5vq6sBAAAAkh7BJhbWvmeuvzXB2joAAACALoJgE23BgLT+fXN74MXW1gIAAAB0EQSbaNu+RGrcJ3lypNLTra4GAAAA6BIINtG27l1zPeACyeW2thYAAACgiyDYRBvjawAAAIC4I9hEU9UWac/XksNlPrEBAAAAEBcEm2gKPa0pPV1K72FtLQAAAEAXQrCJptD4GmZDAwAAAOKKYBMtzbXSlk/NbcbXAAAAAHFFsImWjR9KQZ+Ud6KUf6LV1QAAAABdCsEmWsKzodENDQAAAIg3gk00BAPS+g/M7YF0QwMAAADijWATDeWLpKb9Ulp3qeTbVlcDAAAAdDkEm2gIzYY24ELJlWJtLQAAAEAXRLCJBsbXAAAAAJYi2Byv/ZukvWslZ4p04gVWVwMAAAB0SQSb4xV6WlN6upTe3dJSAAAAgK6KYHO8QuNrBl5ibR0AAABAF0awOR5N1dLWheb2QMbXAAAAAFaJerCZPn26TjnlFGVlZamgoEBXXnml1q5dG+3L2MOGf0pBv5Q/UMrtb3U1AAAAQJcV9WAzf/583XHHHfr88881Z84c+f1+jR8/Xg0NDdG+lPXWtY6v4WkNAAAAYKmov3Tlvffea7P/wgsvqKCgQEuXLtXZZ5992Pler1derze8X1tbG+2SYiPgl9bPMbe/NcHaWgAAAIAuLuZjbGpqaiRJubm57R6fPn26cnJywktJSUmsS4qOii+l5moprbtUcqrV1QAAAABdWkyDjWEYmjJlis4880wNGzas3XOmTp2qmpqa8FJeXh7LkqKnfre5zu0vOV3W1gIAAAB0cVHvinawO++8U19++aU++eSTI57j8Xjk8XhiWUZsNO4z1xntP4kCAAAAED8xCzZ33XWXZs+erQULFqhPnz6xuox1wsEmz9o6AAAAAEQ/2BiGobvuuktvvvmm5s2bp7Kysmhfwh4INgAAAIBtRD3Y3HHHHXrllVf01ltvKSsrSxUVFZKknJwcpaenR/ty1qErGgAAAGAbUZ884LnnnlNNTY3GjRunoqKi8PLaa69F+1LWaqwy1zyxAQAAACwXk65oXQJd0QAAAADbiPl7bJIWwQYAAACwDYJNZxFsAAAAANsg2HRGMCA1tY6xSWfyAAAAAMBqBJvOaKqW1DqWiFnRAAAAAMsRbDoj1A3NkyO53NbWAgAAAIBg0ym8wwYAAACwFYJNZzTtN9dMHAAAAADYAsGmM5gRDQAAALAVgk1nEGwAAAAAWyHYdAZjbAAAAABbIdh0RiNjbAAAAAA7Idh0Bk9sAAAAAFsh2HQGY2wAAAAAWyHYdAZd0QAAAABbIdh0Bk9sAAAAAFsh2EQq4Jeaq81tgg0AAABgCwSbSDVVtW44pLTuVlYCAAAAoBXBJlKhbmjp3SVXiqWlAAAAADARbCLF+BoAAADAdgg2kQo/seEdNgAAAIBdEGwixRMbAAAAwHYINpFq4h02AAAAgN0QbCIVfjknXdEAAAAAuyDYRIquaAAAAIDtEGwiRbABAAAAbIdgEymCDQAAAGA7BJtIEWwAAAAA2yHYRIrJAwAAAADbIdhEIuCTvLXmNk9sAAAAANsg2EQi9LTG4ZTScqytBQAAAEAYwSYSofE16T0kp8vaWgAAAACEEWwiwcQBAAAAgC0RbCJBsAEAAABsiWATCYINAAAAYEsEm0gw1TMAAABgSwSbSIQnDyDYAAAAAHZCsIlEU+iJDV3RAAAAADsh2ESCMTYAAACALRFsIkGwAQAAAGyJYBMJgg0AAABgSwSbSDArGgAAAGBLBJuO8jVLLfXmNk9sAAAAAFsh2HRUaEY0h0tKy7G2FgAAAABtEGw6qvGgqZ4dDmtrAQAAANAGwaajwhMHML4GAAAAsBuCTUcxIxoAAABgWzELNs8++6zKysqUlpam0aNH6+OPP47VpeKDJzYAAACAbcUk2Lz22muaPHmyHnjgAS1fvlxnnXWWJkyYoG3btsXicvFx8BgbAAAAALYSk2Dz1FNP6Uc/+pH+9V//VYMHD9bTTz+tkpISPffcc7G4XHzQFQ0AAACwragHm5aWFi1dulTjx49v0z5+/HgtXLjwsPO9Xq9qa2vbLLZEsAEAAABsK+rBZu/evQoEAurVq1eb9l69eqmiouKw86dPn66cnJzwUlJSEu2SooNgAwAAANhWzCYPcBzyrhfDMA5rk6SpU6eqpqYmvJSXl8eqpOPTxBgbAAAAwK5Sov2B+fn5crlchz2dqaysPOwpjiR5PB55PJ5olxF9ockD0pkVDQAAALCbqD+xSU1N1ejRozVnzpw27XPmzNHYsWOjfbn4YbpnAAAAwLai/sRGkqZMmaIbbrhBY8aM0emnn66ZM2dq27Ztuu2222JxudhraZR8jeY2XdEAAAAA24lJsPmXf/kX7du3T48++qh27dqlYcOG6Z133lHfvn1jcbnYC42vcbolT5a1tQAAAAA4TEyCjSTdfvvtuv3222P18fF18Ixo7UyAAAAAAMBaMZsVLakw1TMAAABgawSbjgjNiMbEAQAAAIAtEWw6opF32AAAAAB2RrDpCKZ6BgAAAGyNYNMRjLEBAAAAbI1g0xEEGwAAAMDWCDYdQbABAAAAbI1g0xHMigYAAADYGsGmI3hiAwAAANgaweZYDENqYrpnAAAAwM4INsfia5T8zeY2wQYAAACwJYLNsYS6oaWkSe4Ma2sBAAAA0C6CzbGEgk16ruRwWFsLAAAAgHYRbI6FiQMAAAAA2yPYHAtTPQMAAAC2R7A5Fp7YAAAAALZHsDmWRqZ6BgAAAOyOYHMsPLEBAAAAbI9gcywEGwAAAMD2CDbHEg42TB4AAAAA2BXB5liYFQ0AAACwPYLNsdAVDQAAALA9gs3RGAbBBgAAAEgABJuj8dZJQZ+5nU5XNAAAAMCuCDZH09Q6vsadIaVmWFsLAAAAgCMi2BwN3dAAAACAhECwORpmRAMAAAASAsHmaHhiAwAAACSEFKsLsLWys6Xv/1XyZFldCQAAAICjINgcTXaxuQAAAACwNbqiAQAAAEh4BBsAAAAACY9gAwAAACDhEWwAAAAAJDyCDQAAAICER7ABAAAAkPAINgAAAAASHsEGAAAAQMIj2AAAAABIeAQbAAAAAAmPYAMAAAAg4RFsAAAAACQ8gg0AAACAhEewAQAAAJDwCDYAAAAAEh7BBgAAAEDCI9gAAAAASHgpVhdwKMMwJEm1tbUWVwIAAADASqFMEMoIR2O7YFNXVydJKikpsbgSAAAAAHZQV1ennJyco57jMDoSf+IoGAxq586dysrKksPhsLoc1dbWqqSkROXl5crOzra6HCQI7ht0BvcNOot7B53BfYPOiPd9YxiG6urqVFxcLKfz6KNobPfExul0qk+fPlaXcZjs7Gz+T4+Icd+gM7hv0FncO+gM7ht0Rjzvm2M9qQlh8gAAAAAACY9gAwAAACDhEWyOwePx6KGHHpLH47G6FCQQ7ht0BvcNOot7B53BfYPOsPN9Y7vJAwAAAAAgUjyxAQAAAJDwCDYAAAAAEh7BBgAAAEDCI9gAAAAASHgEGwAAAAAJj2BzFM8++6zKysqUlpam0aNH6+OPP7a6JNjI9OnTdcoppygrK0sFBQW68sortXbt2jbnGIahhx9+WMXFxUpPT9e4ceO0evVqiyqGHU2fPl0Oh0OTJ08Ot3Hf4Eh27Nih66+/Xnl5ecrIyNCoUaO0dOnS8HHuHRzK7/frl7/8pcrKypSenq7+/fvr0UcfVTAYDJ/DfQNJWrBggS677DIVFxfL4XBo1qxZbY535D7xer266667lJ+fr8zMTF1++eXavn173L4Hgs0RvPbaa5o8ebIeeOABLV++XGeddZYmTJigbdu2WV0abGL+/Pm644479Pnnn2vOnDny+/0aP368Ghoawuc88cQTeuqppzRjxgwtXrxYhYWFuvDCC1VXV2dh5bCLxYsXa+bMmRoxYkSbdu4btKeqqkpnnHGG3G633n33Xa1Zs0ZPPvmkunfvHj6HeweHevzxx/X8889rxowZ+vrrr/XEE0/ot7/9rZ555pnwOdw3kKSGhgaNHDlSM2bMaPd4R+6TyZMn680339Srr76qTz75RPX19br00ksVCATi800YaNepp55q3HbbbW3aBg0aZNx///0WVQS7q6ysNCQZ8+fPNwzDMILBoFFYWGj85je/CZ/T3Nxs5OTkGM8//7xVZcIm6urqjAEDBhhz5swxzjnnHOPuu+82DIP7Bkd23333GWeeeeYRj3PvoD0TJ040br755jZtV111lXH99dcbhsF9g/ZJMt58883wfkfuk+rqasPtdhuvvvpq+JwdO3YYTqfTeO+99+JSN09s2tHS0qKlS5dq/PjxbdrHjx+vhQsXWlQV7K6mpkaSlJubK0navHmzKioq2txHHo9H55xzDvcRdMcdd2jixIm64IIL2rRz3+BIZs+erTFjxujqq69WQUGBTjrpJP3pT38KH+feQXvOPPNMffjhh1q3bp0kaeXKlfrkk090ySWXSOK+Qcd05D5ZunSpfD5fm3OKi4s1bNiwuN1LKXG5SoLZu3evAoGAevXq1aa9V69eqqiosKgq2JlhGJoyZYrOPPNMDRs2TJLC90p799HWrVvjXiPs49VXX9WyZcu0ePHiw45x3+BINm3apOeee05TpkzRL37xCy1atEg//elP5fF4dOONN3LvoF333XefampqNGjQILlcLgUCAU2bNk3XXnutJP7MQcd05D6pqKhQamqqevTocdg58fr9mWBzFA6Ho82+YRiHtQGSdOedd+rLL7/UJ598ctgx7iMcrLy8XHfffbc++OADpaWlHfE87hscKhgMasyYMXrsscckSSeddJJWr16t5557TjfeeGP4PO4dHOy1117TSy+9pFdeeUVDhw7VihUrNHnyZBUXF2vSpEnh87hv0BGduU/ieS/RFa0d+fn5crlch6XLysrKw5IqcNddd2n27NmaO3eu+vTpE24vLCyUJO4jtLF06VJVVlZq9OjRSklJUUpKiubPn6/f//73SklJCd8b3Dc4VFFRkYYMGdKmbfDgweFJbfgzB+255557dP/99+v73/++hg8frhtuuEE/+9nPNH36dEncN+iYjtwnhYWFamlpUVVV1RHPiTWCTTtSU1M1evRozZkzp037nDlzNHbsWIuqgt0YhqE777xTb7zxhj766COVlZW1OV5WVqbCwsI291FLS4vmz5/PfdSFnX/++Vq1apVWrFgRXsaMGaMf/OAHWrFihfr37899g3adccYZh00pv27dOvXt21cSf+agfY2NjXI62/6653K5wtM9c9+gIzpyn4wePVput7vNObt27dJXX30Vv3spLlMUJKBXX33VcLvdxn/+538aa9asMSZPnmxkZmYaW7Zssbo02MRPfvITIycnx5g3b56xa9eu8NLY2Bg+5ze/+Y2Rk5NjvPHGG8aqVauMa6+91igqKjJqa2strBx2c/CsaIbBfYP2LVq0yEhJSTGmTZtmrF+/3nj55ZeNjIwM46WXXgqfw72DQ02aNMno3bu38fbbbxubN2823njjDSM/P9+49957w+dw38AwzNk6ly9fbixfvtyQZDz11FPG8uXLja1btxqG0bH75LbbbjP69Olj/POf/zSWLVtmnHfeecbIkSMNv98fl++BYHMUf/jDH4y+ffsaqampxsknnxyexhcwDHMqxPaWF154IXxOMBg0HnroIaOwsNDweDzG2Wefbaxatcq6omFLhwYb7hscyT/+8Q9j2LBhhsfjMQYNGmTMnDmzzXHuHRyqtrbWuPvuu43S0lIjLS3N6N+/v/HAAw8YXq83fA73DQzDMObOndvu7zWTJk0yDKNj90lTU5Nx5513Grm5uUZ6erpx6aWXGtu2bYvb9+AwDMOIz7MhAAAAAIgNxtgAAAAASHgEGwAAAAAJj2ADAAAAIOERbAAAAAAkPIINAAAAgIRHsAEAAACQ8Ag2AAAAABIewQYAAABAwiPYAAAAAEh4BBsAAAAACY9gAwAAACDh/X86ELHuu9ZXCgAAAABJRU5ErkJggg==",
      "text/plain": [
       "<Figure size 1000x600 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "V = np.zeros(nb_states)\n",
    "Q = np.zeros((nb_states, nb_actions))\n",
    "\n",
    "pi = np.array([a['search'], a['search']], dtype=int)\n",
    "\n",
    "V_high_history = []\n",
    "V_low_history = []\n",
    "\n",
    "t = 1\n",
    "while True:\n",
    "    # Policy evaluation\n",
    "    for k in range(50):\n",
    "\n",
    "        Q[s['high'], a['search']] = alpha * (r_search + gamma * V[s['high']]) + (1 - alpha) * (r_search + gamma*V[s['low']])\n",
    "        Q[s['high'], a['wait']] = r_wait + gamma * V[s['high']]\n",
    "\n",
    "        Q[s['low'], a['search']] = beta * (r_search + gamma * V[s['low']]) + (1 - beta) * (-3 + gamma*V[s['high']])\n",
    "        Q[s['low'], a['wait']] = r_wait + gamma * V[s['low']]\n",
    "        Q[s['low'], a['recharge']] = gamma * V[s['high']]\n",
    "\n",
    "        V[s['high']] = Q[s['high'], pi[s['high']]]\n",
    "        V[s['low']] = Q[s['low'], pi[s['low']]]\n",
    "\n",
    "        V_high_history.append(V[s['high']])\n",
    "        V_low_history.append(V[s['low']])\n",
    "        \n",
    "    # Policy improvement\n",
    "    pi_old = pi.copy()\n",
    "    pi[s['high']] = Q[s['high'], :2].argmax()\n",
    "    pi[s['low']] = Q[s['low'], :].argmax()\n",
    "    \n",
    "    print('Greedy policy after iteration', t)\n",
    "    print('pi(high)=', pi[s['high']])\n",
    "    print('pi(low)=', pi[s['low']])\n",
    "    print('-')\n",
    "    \n",
    "    # Exit if the policy does not change\n",
    "    if pi[s['high']] == pi_old[s['high']] and pi[s['low']] == pi_old[s['low']]:\n",
    "        break\n",
    "    t += 1\n",
    "    \n",
    "plt.figure(figsize=(10, 6))   \n",
    "plt.plot(V_high_history, label=\"high\")\n",
    "plt.plot(V_low_history, label=\"low\")\n",
    "plt.legend()\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Value iteration\n",
    "\n",
    "In value iteration, we merge the policy evaluation and improvement in a single update rule: \n",
    "\n",
    "$$\n",
    "    V (s) \\leftarrow \\max_{a \\in \\mathcal{A}(s)} \\sum_{s' \\in \\mathcal{S}} p(s' | s, a) \\, [ r(s, a, s') + \\gamma \\, V (s') ]\n",
    "$$\n",
    "\n",
    "The value of state takes the value of its greedy action. The policy is therefore implicitly greedy w.r.t the Q-values.\n",
    "\n",
    "The algorithm becomes:\n",
    "\n",
    "* while not converged:\n",
    "\n",
    "    * for all states $s$:\n",
    "    \n",
    "        * Update the value estimates with:\n",
    "        \n",
    "        $$\n",
    "            V (s)  \\leftarrow \\max_{a \\in \\mathcal{A}(s)} \\sum_{s' \\in \\mathcal{S}} p(s' | s, a) \\, [ r(s, a, s') + \\gamma \\, V (s') ]\n",
    "        $$\n",
    "        \n",
    "**Q:** Modify your previous code to implement value iteration. Use a fixed number of iterations (e.g. 50) as in policy evaluation. Visualize the evolution of the V-values and print the greedy policy after each iteration. Conclude."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Greedy policy after iteration 49\n",
      "pi(high)= 0\n",
      "pi(low)= 2\n",
      "V= [13.4228186   9.39597296]\n",
      "Q= [[13.4228186  11.39597296  0.        ]\n",
      " [ 7.63221457  8.57718102  9.39597296]]\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAzYAAAH5CAYAAABTbqsJAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjYuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/av/WaAAAACXBIWXMAAA9hAAAPYQGoP6dpAABFyElEQVR4nO3deXxU9b3/8fdMlskeSCAkgQTCLjuCUhABq2JRqbZVa7VV6+1iS62Utor1V7WL0trWayu3Wu2t1VvQLipqV6lKEAVkEaSA7EsEQgKB7JlMZs7vj5OZJBAgy0y+c5LX8/GYznfOOTPnk3iM8+53OS7LsiwBAAAAgIO5TRcAAAAAAJ1FsAEAAADgeAQbAAAAAI5HsAEAAADgeAQbAAAAAI5HsAEAAADgeAQbAAAAAI4Xa7qAUwUCAR0+fFipqalyuVymywEAAABgiGVZqqysVG5urtzus/fJRF2wOXz4sPLy8kyXAQAAACBKFBUVacCAAWc9JuqCTWpqqiS7+LS0NMPVAAAAADCloqJCeXl5oYxwNlEXbILDz9LS0gg2AAAAANo0RYXFAwAAAAA4HsEGAAAAgOMRbAAAAAA4XtTNsWkrv98vn89nugxHiYuLU0xMjOkyAAAAgLBzXLCxLEvFxcU6efKk6VIcqVevXsrOzuYeQQAAAOhWHBdsgqEmKytLSUlJfEFvI8uyVFNTo5KSEklSTk6O4YoAAACA8HFUsPH7/aFQk5mZabocx0lMTJQklZSUKCsri2FpAAAA6DYctXhAcE5NUlKS4UqcK/i7Y34SAAAAuhNHBZsghp91HL87AAAAdEeODDYAAAAA0BzBpovMmjVL8+fPP+N+l8ulZcuWtfnzVqxYIZfLxepwAAAAgBy2eEB3duTIEfXu3dt0GQAAAIAjEWyiRHZ2tukSAAAAAMdiKFoXCgQCuvvuu5WRkaHs7Gw9+OCDoX2nDkV79913NWHCBCUkJGjy5MlatmyZXC6XNm3a1OIzN2zYoMmTJyspKUnTpk3Tjh07uuaHAQAAAKKI43tsLMtSrc9v5NyJcTHtWmXs2Wef1YIFC7R27VqtXr1at912my666CJdfvnlLY6rrKzU3LlzdeWVV2rp0qU6cODAGefn3HffffrFL36hvn376o477tDtt9+ud955pzM/FgAAAOA4jg82tT6/Rt3/LyPn3vbDK5QU3/Zf4bhx4/TAAw9IkoYNG6bFixfrjTfeOC3YLFmyRC6XS08//bQSEhI0atQoHTp0SF/+8pdP+8yHHnpIM2fOlCQtXLhQV111lerq6pSQkNCJnwwAAABwFoaidaFx48a1eJ2Tk6OSkpLTjtuxY4fGjRvXIpxceOGF5/zMnJwcSWr1MwEAAIDuzPE9NolxMdr2wyuMnbs94uLiWrx2uVwKBAKnHWdZ1mlD3CzLOudnBt/T2mcCPUkgYMkXCKjBb9mPQEANAUs+f0D+gKWGgKVAwJLfsvcHrGbbgo9WtgUsKWBZoYc/YL+2TmufeqxkWU377Xbja9n/fgeabbcatwcCjc+N+6SmfVaL90uS1ewcUvAvht1u2mA1+4ym/U1/Y6zQ/zS+L/T5zZ7VrB613Ne0Ra0c02zfae879d1n/rt36uc3vff041v7iLN9bGuf0Zb3nUuH39qJc3b8lAZOiojozDULSNL3rjpP5+c7Z9Vexwcbl8vVruFgTjBy5EgtWbJEXq9XHo9HkrR+/XrDVQHn1uAPqNbnV50voDqfX7U+v2rrG599ftU1tut8weP88jYEVB98+P3N2vazt9lrn7/pWF8wsPjtwNIQaAoxAf5jDgBAp5XX+kyX0C7dKxF0EzfddJPuu+8+feUrX9HChQt18OBB/fznP5ekdi1WALRVbb1fZTX1qqzzqaquQZXeBlXVNaiq8bmyztdym7dBlc32V9c3qM7nl88fvYkixu1SbOMjJvRwK8Ytxbhciolx2c+N+9wul2Jb2eZ2ueR2q6ntsj/b5bKPdbvV1HbZx7latBV67XI1fY6k0OcFt8sludS0zaXG90uSyyWXmraHjmn8rOD2pnbLzwhytficZtsaX7iaDmzx+tTPP3Vf6G1qubP5XzDXKZ95yqlOa7f4vDPsb01rfzfP9razfeap52+Pjv75dtJfff4TBXQvo3PTTJfQLgSbKJSWlqbXXntNX/va1zRhwgSNHTtW999/v2666SYWBcA51TcEdLKmXmU19SqrrteJap/drqrXieC20D77uDpf+IcvJsbFKDE+RolxMUqIczdrNz0nxLnliY1RXIxb8bH2wxPrVnyz18F2XEzjvtim17FuO3zEut2Ki3EpNsatOLf9bG+398W6XXK7+cYFAEB3RrDpIitWrDhtW/P71pw6lnzatGnavHlz6PWSJUsUFxen/Px8SdKsWbNOe8+ECRPOOSYdzuZt8OtouVeHy2t1pLxWR8rrdORknY6U1+rwyToVV9SprLq+Q58dH+NWWmKsUjyxSklofPbEKTWh5ba0hGA7TimeWKUmxCoxPkZJzYKLJ9ZN7yIAAOhSBJso9dxzz2nw4MHq37+/Nm/erHvuuUc33HCDEhMTTZeGCPI2+LXvWLX2lFTroxM1OlJep8MnGwNMeZ2OVXnb9Dlul9Q7KV69k+OVkRSv3slxykj2KCM5Tr2T4pWR3LQvI9l+JMW3775MAAAA0YRgE6WKi4t1//33q7i4WDk5Obr++uv10EMPmS4LYVLlbdDukqoWjz2lVTpwvPqcE989sW7lpCcoJz1ROb0SlJueqOz0BOX2srdlpyUoPTGOoVcAAKBHIdhEqbvvvlt333236TLQScervNp1SnjZdbRKxRV1Z3xPqidWQ7JSNDAzSTnpiaHAYoeZBGUkx9OzAgAAcAqCDRAm/oClHcWV2nDwhDbsL9P6Ayf00YnaMx7fJ8WjYVkpGnrKIyvVQ3ABAABoJ4IN0EFV3gZtOnhS6w+UacOBE9p08KQqvQ0tjnG5pAG9EzW07ykBpm+q0pPizvDJAAAAaC+CDdAGlmXp0MlabThwQhsOnND6/Sf0YXHFafNhkuNjNDG/tyYN7K3Jg3prQl4vpSYQYAAAACKNYAOcgbfBr1W7junvW4r1zu5jrc6L6d8rUZMH2UFm0sDeGpmdphgm7QMAAHQ5gg3QTJ3Pr8KdpfrHliP69/YSVTUbWhbjdml0bprdGzMwQ5MG9lZ2OjdMBQAAiAYEG/R4NfUNWrGjVH/fckRvfliimnp/aF92WoI+MSZbs0f104T8XkqK518ZAACAaMS3tC4ya9YsTZgwQY899pjpUiB74v8b24/qH1uKtWJniep8gdC+/r0SNWdMtuaMzdHEvF7cDwYAAMABCDboMcprfXpj+1H9fUuxVu4qVX1DU5jJz0jSnLHZunJMjsYNSGe5ZQAAAIch2KDbO3C8Wk+t3Ks/b/ioRZgp6JOsK8dma86YHI3OTSPMAAAAOJjbdAE90YkTJ3TLLbeod+/eSkpK0pw5c7Rr1y5J9rLCffv21Ysvvhg6fsKECcrKygq9Xr16teLi4lRVVdXltTvJ1sPluvP593XJz1doydqDqm8IaGhWir556TD9c/7FevPbM/XdK0ZqTH96aAAAAJzO+T02liX5asycOy7JvgNjO912223atWuXXn31VaWlpemee+7RlVdeqW3btikuLk4zZszQihUr9JnPfEYnTpzQtm3blJycrG3btmnUqFFasWKFJk2apJSUlAj8UM5mWZbW7C3TE4V7tHJnaWj7zOF99bVZQzSlIIMQAwAA0A05P9j4aqSHc82c+3uHpfjkdr0lGGjeeecdTZs2TZK0ZMkS5eXladmyZbr++us1a9YsPfXUU5KklStXavz48crPz9eKFStCwWbWrFnh/mkcLRCw9Pq2o3qycI82FZ2UJLld0tXjcvXVmYM1OjfdbIEAAACIKOcHG4fZvn27YmNjNWXKlNC2zMxMjRgxQtu3b5dkr6B211136dixYyosLNSsWbOUn5+vwsJCfeUrX9G7776r+fPnG/oJokt9Q0DLNh3Sbwr3aE9ptSQpPtatGyYP0FcuHqL8zCTDFQIAAKArOD/YxCXZPSemzt1OlmWdcXtwiNSYMWOUmZmpwsJCFRYW6oc//KHy8vL00EMPad26daqtrdX06dM7VbrTVXsb9Px7B/Xbt/epuKJOkpSaEKtbpg7UbdMK1DfVY7hCAAAAdCXnBxuXq93DwUwaNWqUGhoatHbt2tBQtOPHj2vnzp0677zzJEkul0szZszQK6+8ov/85z+6+OKLlZqaKp/PpyeffFLnn3++UlNTTf4YxpTX+PS/q/bq2dUHVF7rkyT1TfXoS9MLdNOUfKUmxBmuEAAAACY4P9g4zLBhw3TNNdfoy1/+sn7zm98oNTVVCxcuVP/+/XXNNdeEjps1a5a+9a1vaeLEiUpLS5MkzZgxQ0uWLNGCBQtMlW/U8m1H9b2Xt6i00ivJXq75KzMG61MT+yshLsZwdQAAADCJYGPAM888o7vuuktXX3216uvrNWPGDP39739XXFxTb8Mll1wiv9/fYpGAmTNnatmyZZo5c6aBqs0pr/HpB69t1UvvH5IkDe6brO/MHqErRmcrxs0KZwAAAJBc1pkmfRhSUVGh9PR0lZeXh3oqgurq6rRv3z4VFBQoISHBUIXO5rTf4Rvbj+rel7aopNIrt0v68sWD9a3Lh9NDAwAA0AOcLRucih4bRKXyWp9++No2vbjxI0nS4D7J+tn14zVpYG/DlQEAACAaEWwQdd7aUaJ7X9yi4oo6uVzSl6YX6NuzR9BLAwAAgDMi2CBqVNT59KPXtunPG+xemoI+yfrZdeM0eVCG4coAAAAQ7Qg2iAqFO0u18MUPdKTc7qW5/aICfWf2CCXG00sDAACAcyPYwKjKOp8e+tt2vbCuSJI0KDNJP7t+vC6glwYAAADt4MhgEwgETJfgWNH0u3t7V6nu+csHOlxeJ0n64kWDdPcVI+mlAQAAQLs5KtjEx8fL7Xbr8OHD6tu3r+Lj4+VycR+TtrAsS/X19SotLZXb7VZ8fLyxWhr8Af3gtW36vzUHJEn5GUn62XXjNGVwprGaAAAA4GyOCjZut1sFBQU6cuSIDh8+bLocR0pKSlJ+fr7cbreR83sb/Lpz6ft6fdtRSdJt0wbp7k+MUFK8oy5FAAAARBnHfZuMj49Xfn6+Ghoa5Pf7TZfjKDExMYqNjTXWy1Vb79dX/7BBK3eWKj7WrcWfm6jZo7ON1AIAAIDuxXHBRpJcLpfi4uIUFxdnuhS0UZW3Qbf/fp3e21emxLgYPX3LZE0f1sd0WQAAAOgmHBls4CzlNT7d8sx72lx0UqmeWP3uixew6hkAAADCimCDiDpW5dUX/vc9bT9SoV5JcXru9gs1bkAv02UBAACgmyHYIGKKy+t082/XaE9ptfqkeLTkS1M0IjvVdFkAAADohgg2iIiishrd9Ns1KiqrVW56gpZ8+WMq6JNsuiwAAAB0UwQbhN2e0ird/PRaFVfUaWBmkpZ8aYoG9E4yXRYAAAC6MYINwmr7kQp94X/X6lhVvYZmpWjJl6aoX1qC6bIAAADQzRFsEDabi07qlt+9p/Jan0blpOn//utCZaZ4TJcFAACAHoBgg7B4b1+Zbv/9OlV5GzQxv5d+/8ULlZ7IfYYAAADQNQg26LS3d5Xqy8+tV50voKmDM/XbWycr2cOlBQAAgK7Dt090yutbi/WNpe+r3h/QJSP66onPT1JCXIzpsgAAANDDEGzQYSt2lOhrSzbKH7A0Z0y2fnnjRMXHuk2XBQAAgB6IYIMOOVpRpwV/2ix/wNI1E3L1i+vHKzaGUAMAAAAz+CaKdvMHLM1/YZPKqus1KidNj1w3jlADAAAAo/g2inZ7YsVurd57XEnxMXr8ponyxDKnBgAAAGa1O9isXLlSc+fOVW5urlwul5YtWxba5/P5dM8992js2LFKTk5Wbm6ubrnlFh0+fDicNcOg9fvL9N//3iVJ+sEnR2tI3xTDFQEAAAAdCDbV1dUaP368Fi9efNq+mpoabdy4Ud///ve1ceNGvfTSS9q5c6c++clPhqVYmFVe49NdL2ySP2Dp2gm5um7SANMlAQAAAJI6sHjAnDlzNGfOnFb3paena/ny5S22Pf7447rwwgt18OBB5efnd6xKGGdZlu558QMdOlmrgZlJ+vGnxsrlcpkuCwAAAJDUBauilZeXy+VyqVevXq3u93q98nq9odcVFRWRLgkdsGTtQf1za7HiYlx6/HMTlcINOAEAABBFIrp4QF1dnRYuXKibbrpJaWlprR6zaNEipaenhx55eXmRLAkdsKO4Uj/66zZJ0t1XjNS4Ab3MFgQAAACcImLBxufz6cYbb1QgENCvf/3rMx537733qry8PPQoKiqKVEnogNp6v76xdKO8DQHNGtFX/zW9wHRJAAAAwGkiMp7I5/Pphhtu0L59+/Tmm2+esbdGkjwejzweTyTKQBj88K9btaukSn1TPfr59ePldjOvBgAAANEn7MEmGGp27dqlt956S5mZmeE+BbrI3z44ouffK5LLJT322Qnqk0IABQAAQHRqd7CpqqrS7t27Q6/37dunTZs2KSMjQ7m5ubruuuu0ceNG/fWvf5Xf71dxcbEkKSMjQ/Hx8eGrHBFVVFajhS99IEn62swhumhoH8MVAQAAAGfmsizLas8bVqxYoUsuueS07bfeeqsefPBBFRS0Pgfjrbfe0qxZs875+RUVFUpPT1d5eflZh7Ahcnz+gG74zWq9f/CkJub30p++OlVxMRFdZwIAAAA4TXuyQbt7bGbNmqWzZaF25iREof9evlPvHzyp1IRY/erGiYQaAAAARD2+saKFVbuO6YnCPZKkn35mnPIykgxXBAAAAJwbwQYhx6q8+tafNsmypM9dmK8rx+aYLgkAAABoE4INJEmBgKVv/2mzSiu9Gt4vRfdfPcp0SQAAAECbEWwgSfrfVftUuLNUnli3Hv/c+UqMjzFdEgAAANBmBBtoc9FJPfKvDyVJ988dpRHZqYYrAgAAANqHYNPD+QOWvvuXzfL5Lc0Zk62bLsw3XRIAAADQbgSbHu61zYe182iV0hPjtOjTY+VyuUyXBAAAALQbwaYHa/AH9Ni/d0qSvjJjsHolxRuuCAAAAOgYgk0P9tLGQ9p/vEaZyfG6bdog0+UAAAAAHUaw6aHqGwL65Ru7JElfmzVEyZ5YwxUBAAAAHUew6aH+uL5Ih07WKivVo89/bKDpcgAAAIBOIdj0QHU+vxa/affWfOPjQ5UQxz1rAAAA4GwEmx7oD2sO6GiFV/17JeqzF+SZLgcAAADoNIJND1PtbdCThXskSXd+fKg8sfTWAAAAwPkINj3Ms6v361hVvQZmJukzkwaYLgcAAAAIC4JND1JR59NvCvdKku66dJjiYvjHDwAAgO6Bb7Y9yO9W7VN5rU9D+ibrmgn9TZcDAAAAhA3Bpoc4WVOv/317nyTpW5cPV4zbZbgiAAAAIHwINj3EUyv3qtLboJHZqbpyTI7pcgAAAICwItj0AMeqvHrmnf2SpG/PHiE3vTUAAADoZgg2PcCTK/ao1ufX+AHpuuy8LNPlAAAAAGFHsOnmjlbU6f/WHJAkLZg9Qi4XvTUAAADofgg23dz/vLVb3oaAJg/srRnD+pguBwAAAIgIgk039tGJGj3/3kFJ9twaemsAAADQXRFsurHH39gtn9/StCGZmjok03Q5AAAAQMQQbLqp/ceq9ZeNH0mSvj17uOFqAAAAgMgi2HRTv3xjl/wBS5eM6KtJAzNMlwMAAABEFMGmG9p1tFLLNh2SJC24fIThagAAAIDII9h0Q4/9e5csS7pidD+NHZBuuhwAAAAg4gg23cy2wxX625Yjcrmkb13O3BoAAAD0DASbbubR5TslSVePy9XI7DTD1QAAAABdg2DTjWwqOql/bz8qt0uaf9kw0+UAAAAAXYZg040Ee2s+NXGAhvRNMVwNAAAA0HUINt3Efw6Va+XOUsW6XbrrUnprAAAA0LMQbLqJlzbayztfMSZb+ZlJhqsBAAAAuhbBphto8Af02geHJUmfntjfcDUAAABA1yPYdAPv7jmu0kqveifFacbwvqbLAQAAALocwaYbWPa+PQzt6nG5iovhHykAAAB6Hr4FO1xNfYP+tbVYknTtxFzD1QAAAABmEGwcbvm2o6qu9ysvI1Hn5/c2XQ4AAABgBMHG4V7ZZC8acO2E/nK5XIarAQAAAMwg2DjY8SqvCneWSpKumcBqaAAAAOi5CDYO9rctR+QPWBrbP11Ds1JMlwMAAAAYQ7BxsJcbV0O7lnvXAAAAoIcj2DjUgePVev/gSbld0tzxOabLAQAAAIwi2DjUsvftRQMuGtpHWakJhqsBAAAAzCLYOJBlWXplkz0M7VMMQwMAAAAINk70wUfl2nusWglxbs0enW26HAAAAMA4go0DBRcNmD0qWymeWMPVAAAAAOYRbBymwR/QXz9ovCnnxFzD1QAAAADRgWDjMKt2H9OxqnplJMfr4mF9TZcDAAAARAWCjcO8ssnurbl6XI7iYvjHBwAAAEgEG0epqW/Qv7YWS+KmnAAAAEBzBBsHWb7tqGrq/RqYmaSJeb1MlwMAAABEDYKNgwRXQ7tmQn+5XC7D1QAAAADRg2DjEMeqvHp71zFJ0rUTWA0NAAAAaI5g4xB/3XxY/oCl8QPSNbhviulyAAAAgKhCsHGIlzcF713DogEAAADAqQg2DrDvWLU2F51UjNulq8cxDA0AAAA4FcHGAZY1LhowfWgf9U31GK4GAAAAiD4EmyhnWZZe2WQHm08xDA0AAABoFcEmym0qOqn9x2uUGBejy0f1M10OAAAAEJXaHWxWrlypuXPnKjc3Vy6XS8uWLWux37IsPfjgg8rNzVViYqJmzZqlrVu3hqveHueVxkUDrhjdT8meWMPVAAAAANGp3cGmurpa48eP1+LFi1vd/8gjj+jRRx/V4sWLtW7dOmVnZ+vyyy9XZWVlp4vtaXz+gF7bbAebaxiGBgAAAJxRu7sA5syZozlz5rS6z7IsPfbYY7rvvvv06U9/WpL07LPPql+/flq6dKm++tWvdq7aHmbV7mM6Xl2vzOR4XTy0j+lyAAAAgKgV1jk2+/btU3FxsWbPnh3a5vF4NHPmTL377rutvsfr9aqioqLFA7bgamhzx+cqNobpUAAAAMCZhPXbcnFxsSSpX7+Wk9z79esX2neqRYsWKT09PfTIy8sLZ0mOVe1t0Otbj0rippwAAADAuUSkG8DlcrV4bVnWaduC7r33XpWXl4ceRUVFkSjJcV7fVqxan1+DMpM0fkC66XIAAACAqBbWZbays7Ml2T03OTk5oe0lJSWn9eIEeTweeTzcdPJUL79vLxpw7cT+ZwyFAAAAAGxh7bEpKChQdna2li9fHtpWX1+vwsJCTZs2LZyn6tZKK71atatUknTtBIahAQAAAOfS7h6bqqoq7d69O/R637592rRpkzIyMpSfn6/58+fr4Ycf1rBhwzRs2DA9/PDDSkpK0k033RTWwruz1zYfVsCSJuT10qA+yabLAQAAAKJeu4PN+vXrdckll4ReL1iwQJJ066236ve//73uvvtu1dbW6utf/7pOnDihKVOm6PXXX1dqamr4qu7mXtlkr4b2KRYNAAAAANrEZVmWZbqI5ioqKpSenq7y8nKlpaWZLqfL7S2t0sd/UagYt0trv3ep+qQw/wgAAAA9U3uyATdHiTKvbrYXDZgxrA+hBgAAAGgjgk2UeXvXMUnSJ8ZkG64EAAAAcA6CTRSp9jZoc9FJSdK0IX3MFgMAAAA4CMEmiqw/cEINAUv9eyUqLyPJdDkAAACAYxBsosiavcclSVOHZBquBAAAAHAWgk0UWb3HDjYfG0ywAQAAANqDYBMlqrwN2nKoXBI9NgAAAEB7EWyixLr9ZfIHLOVnJKl/r0TT5QAAAACOQrCJEmtCw9AyDFcCAAAAOA/BJkqsZuEAAAAAoMMINlGgos6n/zTOr2HhAAAAAKD9CDZRYN2+MgUsaVBmknLSmV8DAAAAtBfBJgoEl3lmGBoAAADQMQSbKLBmH/evAQAAADoj1nQBPV15jU9bD1dIkqYSbIDwsyzJ75MCvsZnv2T5pUBDs7a/bdutgP0IND5b/mbbgm2r9e2ymu23mvarWfu07c2ODb5u8RxoZVuzdvDnP+29av3zmh8faqvZa53yXp352Fbf32xfq9vO9P5W9rW6/yzHtvl9Z/vIc70vAufsFBPn7CAjvx8A5zRrodT/fNNVtBnBxrD39pfJsqTBfZOVlZZguhyg8yxLavBKvhrJV2s/GhqfQ9ua7/NKfq/UUG8/+32nbws9N+4PHuf3NQYRn+RvaBZeGpqeLb/p3wgAAM504ZdNV9AuBBvDVu9hGBqigGVJ3krJW9H4fEq77gzbg4/66pahxQn/T7ErRnLHSu6YxnbwEdvy9anHuVySy9342t34aGy73advCz1cpzy7Jbla369TjpWr2fZT9oe2qZVtwe2nHNem/Wpqh4459fXZ9p3j/Wfadqb3t7qvFa3ub2XbWT/nLPvOdf6z6uB7O3XOjjJxTpyVkesAPV7WKNMVtAvBxrDQ/WsINggnX61Uc7zZo0yqPnbKtsbtwXbAF/463HFSXJIUl9j4aN5OlGIT7EdMvBQbL8V4Wnn22Ptj4pvasR4pJq4xcMQ1tUPbgu3gvpiWx4UCAAAA6C4INgadrKnXh8X2/Bp6bNAmliXVnpAqDknlh+znU9uVxY29Jh3gjpMS0iRPauMjrfGResqjcVvw2LhkKT6pKbzEJtjtmLjw/vwAAABnQLAxaM1ee37N0KwU9U31mC4H0cCy7HBybKd08mBTYCn/SKo4bLfbGlrccVJSZuMjQ0ru0+x1s+1JmVJSH7sdx32UAACAMxFsDFrDMLSey98gndgvHdshlX4ole6028d2SfVV535/UqaU1l9KH2A/p+U2a+fYQcWTynArAADQYxBsDAoFG27M2X35au2wcmynVLqjMcjslMr2SP761t/jipEyBksZBY3hpb+UNqDxuTHE0LMCAADQAsHGkONVXn1YXClJmlKQYbgahIVl2cPHit6TitZIB9dKJVsb70HSithEqc8wqe8Iqc8Iqe9w+zljsD1xHgAAAG1GsDFk7b4ySdKIfqnKTGF+jSP5fVLxB3aQObhGKlorVR45/bjE3i2DS98RUp/hUnqevTwwAAAAOo1gYwjD0Byo9oT00fqmEHNow+kT+d2xUvY4Kf9jUt4UKe9CKTWHuS4AAAARRrAxpOnGnAxDi1qBgB1gtr0i7V0hlW4//ZiE9MYAM8UOM7nn28seAwAAoEsRbAworfRqV0mVXC5pSgE9NlHF3yAdfNcOM9v/KlUVt9yfMVjK+5iUP8V+7jOc4WQAAABRgGBjwNp9dm/NyOw09U5mkrhxfp+0/+2mMFNzrGmfJ10aeaU04kopf6qU0tdcnQAAADgjgo0BDEOLAg310r5Cadsy6cO/2fNnghJ7SyOvlkZdIxXMZIUyAAAAByDYGLCaG3Oa4auT9rxp98zs+IfkLW/al9RHOm+uHWYGTZdi4szVCQAAgHYj2HSxkoo67S2tZn5NV6o8Kq19Qlr3u5ZhJiW7KcwMnCa5Y8zVCAAAgE4h2HSxYG/NqJw0pSfRKxBRZXuldx+X3l8i+b32trT+dpAZdY004EIm/gMAAHQTBJsutoZhaJF3ZLO06jF7/owVsLcNuFC6eIE07ArCDAAAQDdEsOlia/aWSZI+RrAJL8uS9q+SVv23tOeNpu3DZkvTv2WvaMZNMgEAALotgk0XKi6v075j1XK7pAtZES08AgFpx9/tQHNovb3N5ZbGfEa66C4pe6zZ+gAAANAlCDZdKDgMbUz/dKUlML+mUxrqpS1/lt55TDq2094WmyBN/Lw09RtSRoHR8gAAANC1CDZdqOn+NQxD67D6GmnD76XVi6WKQ/Y2T7p04ZekKXdIKVlGywMAAIAZBJsuxP1rOmn/KumVedKJ/fbrlH7S1HnSpC9KCWlGSwMAAIBZBJsucuhkrQ6W1SjG7dLkQb1Nl+Ms9dXSv38gvfcb+3Vaf2nm3dK4G6W4BLO1AQAAICoQbLrImj1N82tSmV/Tdvvfaeyl2We/Pv8WafaPpYR0s3UBAAAgqhBsugjD0Nqpvlp644fS2t9Isuxemk/+Shp6menKAAAAEIUINl0kdGPOIQSbczrwrrTs6029NBO/IF3xEL00AAAAOCOCTRcoKqvRRydqFet2afJA5tecUX2N9OaPpDVPKNRLM/dX0jB6aQAAAHB2BJsuEByGNm5AupI9/MpbdXCN3UtTtsd+PfHz0hUP00sDAACANuFbdhdgGNpZ1NdIb/5YWvNrSZaUmmvPpRl2uenKAAAA4CAEmwizLCu0Iho35jzFwbXSsq819dJM+Lw9lyaxl9GyAAAA4DwEmwg7WFajw+V1iotxafLADNPlRIeAX3rjB9I7v5LdS5Njz6UZPtt0ZQAAAHAogk2EBYehTcjrpcT4GMPVRAF/g91Ls+VP9usJN9tzaeilAQAAQCcQbCJsNcPQmjTUSy/+l7T9VckdK33qN9LY60xXBQAAgG6AYBNBlmVxY84gX630p1ukXa9LMfHSDc9JI+aYrgoAAADdBMEmgvYfr9HRCq/iY9w6vyffv8ZbJT1/o7T/bSk2UfrcUmnIx01XBQAAgG6EYBNBwWFoE/J7KSGuh86vqT0pLb1BKlorxadKN/9JGjjNdFUAAADoZgg2EbSmpw9Dqz4u/eFT0pHN9o02P/+yNGCS6aoAAADQDRFsIqT5/JoeuXBA5VHpuWuk0u1SUh/plmVS9ljTVQEAAKCbIthEyJ7SapVWehUf69bE/F6my+la5R9Jz37SvvFmao50yytS3xGmqwIAAEA3RrCJkOAwtEn5vXvW/JqyvdKz10jlB6X0fOnWV6SMwaarAgAAQDdHsImQHjkMrXSHPfys8oiUMUS69VUpfYDpqgAAANADEGwiwLIsrd1bJkmaOqSHBJviLdJz10o1x6SsUdIXlkmp/UxXBQAAgB6CYBMBpZVeHavyyu2Sxg1IN11O5H20wV79rK5cyhlvh5qkDNNVAQAAoAch2ETAh8WVkqRBfZK7//ya/e9ISz8r1VdKeVOkm/9sL+0MAAAAdCGCTQTsPGoHmxH9Ug1XEmEH10p/+IzUUCsVzJBufF7ypJiuCgAAAD0QwSYCgj02I7K7cbCpKZP+crsdaoZeJn32D1JcoumqAAAA0EO5TRfQHe1oDDYju2uwsSzp1Tulio/s1c+u/z2hBgAAAEaFPdg0NDTo//2//6eCggIlJiZq8ODB+uEPf6hAIBDuU0Ulf8DSrhI72AzvrkPR1v1W+vCvUky8dN3vJE83/TkBAADgGGEfivbTn/5UTz75pJ599lmNHj1a69ev1xe/+EWlp6frrrvuCvfpos7BshrV+QJKiHNrYGay6XLC78gH0r++Z7cv/5GUO8FoOQAAAIAUgWCzevVqXXPNNbrqqqskSYMGDdLzzz+v9evXh/tUUWlHcYUkaVhWqmLcLsPVhJm3SvrLFyV/vTR8jjTlq6YrAgAAACRFYCja9OnT9cYbb2jnzp2SpM2bN2vVqlW68sorWz3e6/WqoqKixcPJdhRXSeqmw9D+/l3p+G4pNVe69teSq5sFNwAAADhW2Hts7rnnHpWXl2vkyJGKiYmR3+/XQw89pM997nOtHr9o0SL94Ac/CHcZxuw4agezbrdwwOYXpM1LJZdb+sxvuQEnAAAAokrYe2z++Mc/6g9/+IOWLl2qjRs36tlnn9XPf/5zPfvss60ef++996q8vDz0KCoqCndJXapbLvV8bLf01wV2e+ZCadBFZusBAAAAThH2Hpvvfve7WrhwoW688UZJ0tixY3XgwAEtWrRIt95662nHezweeTyecJdhRJ3Pr/3HqiV1o2DT4LXn1fiqpYHTpRnfMV0RAAAAcJqw99jU1NTI7W75sTExMT1iuefdJVUKWFKvpDhlpXaPsKblD0jFH0iJGdJnnpbcMaYrAgAAAE4T9h6buXPn6qGHHlJ+fr5Gjx6t999/X48++qhuv/32cJ8q6gRvzDmiX6pc3WFi/Yd/l9Y+Ybc/9aSUlmu2HgAAAOAMwh5sHn/8cX3/+9/X17/+dZWUlCg3N1df/epXdf/994f7VFFnx1E72HSLhQPKD0mvfN1uT/2GNPwKs/UAAAAAZxH2YJOamqrHHntMjz32WLg/OuoFe2yGOz3Y+BukF78k1Z6QciZIlz5guiIAAADgrMI+x6YnCwYbx/fYrPyZdPBdKT5Vuu53Umy86YoAAACAsyLYhEl5jU/FFXWSHH5zzn1vSysfsdtzH5MyhxgtBwAAAGgLgk2YBOfX9O+VqNSEOMPVdFD1cemlL0tWQJr4eWnsdaYrAgAAANqEYBMmO4orJDn4/jWWJS37mlR5ROozXJrziOmKAAAAgDYj2ITJh8Glnp0abNY8Ie36lxTjka57RopPNl0RAAAA0GYEmzDZebTpHjaOc/h9aXnjctyfeFjKHmO2HgAAAKCdCDZhYFmWc3tsAn5p2delgE86b640+b9MVwQAAAC0G8EmDI6U16myrkGxbpeG9E0xXU77/OclqWSblJAuzf2V5HKZrggAAABoN4JNGARXRCvok6z4WAf9Sv0N0opFdnvanVJShtl6AAAAgA5y0Lfw6LXDqcPQtvxJKtsjJWZIU+4wXQ0AAADQYQSbMAgGm5FOCjZ+n1T4U7s9fb7kcVDtAAAAwCkINmHQ1GOTZriSdti0RDqxX0ruK13wJdPVAAAAAJ1CsOmkBn9Au0urJDloqecGr1T4M7s9fQH3rAEAAIDjEWw6af/xatU3BJQUH6MBvRNNl9M2G5+TKj6SUnOkybebrgYAAADoNIJNJwXvXzO8X6rcbgcsleyrld7+hd2++NtSXILZegAAAIAwINh00s7g/BqnDENb/4xUeURKGyCdf4vpagAAAICwINh00odOWuq5vlpa9ajdnvldKdZjth4AAAAgTAg2nRS8Oacjlnp+72mpulTqPUiacLPpagAAAICwIdh0Qk19gw6W1UiShkd7sPFWSu/80m7PvEeKiTNbDwAAABBGBJtO2HW0SpYl9UmJV5+UKB/WtfZJqbZMyhwqjb3BdDUAAABAWBFsOmGHU+bX1J6U3n3cbs9cKMXEGi0HAAAACDeCTScE59cMj/YV0db8Wqorl/qOlMZ82nQ1AAAAQNgRbDoh2GMT1QsH1JRJq39tt2fdK7ljzNYDAAAARADBphOalnpOM1zJWbz7uFRfKfUbI533SdPVAAAAABFBsOmg41VeHavySpKGZaUYruYMqo9Ja39jty/5nuTmHzcAAAC6J77pdlBwfk1+RpKSPVE6GX/Vf0u+ail3ojTiStPVAAAAABFDsOmgqF8RrbJYWvdbu33JfZLLZbYeAAAAIIIINh2082iULxzw9qNSQ5004EJp6GWmqwEAAAAiimDTQcGFA6JyqefyQ9KGZ+z2Jd+jtwYAAADdHsGmAwIBSzujeannt38u+eulgRdJg2eZrgYAAACIOIJNBxw6Wavqer/iY9wa1CfZdDktnTggbfw/u83cGgAAAPQQBJsOCC4cMLhvsuJiouxXuPIRKeCze2oGXWS6GgAAAKBLRNm3cmfYEa0LBxzfI2163m5fcp/ZWgAAAIAuRLDpgA9DSz2nGa7kFIWPSJZfGnq5lHeh6WoAAACALkOw6YCdoWCTYriSZsr2SVv+ZLcv+Z7ZWgAAAIAuRrBpp/qGgPaUVkmKsh6bzS9IVkAafInU/3zT1QAAAABdimDTTnuPVakhYCnVE6vc9ATT5dgsS/rgj3Z7/OfM1gIAAAAYQLBpp+CKaMOzU+WKlqWUP1ovndgnxSVL511tuhoAAACgyxFs2mlHaH5NFK2I9sEL9vN5V0vxUXZfHQAAAKALEGzaKRhsomap54Z66T8v2e1xnzVbCwAAAGAIwaadgvewGdEvSoLNnjek2jIppZ9UMNN0NQAAAIARBJt2qPI26KMTtZKiaCja5sZhaGOuk2JizdYCAAAAGEKwaYfgMLR+aR71Soo3XI2kunJpxz/s9niGoQEAAKDnIti0w87gMLRouX/Ntlclv1fqO1LKHme6GgAAAMAYgk07hFZE65diuJJGwXvXjLtBipalpwEAAAADCDbt8GFxhaQo6bE5WSTtf9tuj73BbC0AAACAYQSbNrIsK7qWet7yZ/t54HSpV57ZWgAAAADDCDZtVFrl1Ykan9wuaWiW4aFoltU0DI1FAwAAAACCTVsFe2sGZSYrIS7GbDHFW6TSD6UYj3TeJ83WAgAAAEQBgk0bhRYOiIZhaMHemhGfkBJ7GS0FAAAAiAYEmzYKBpvh/QwHm4C/aX7NuBvN1gIAAABECYJNG+04GiULB+wrlKqOSom9paGXma0FAAAAiBIEmzbwB6xmN+c0HGw2Nw5DG/1pKTbebC0AAABAlCDYtEFRWY3qfAF5Yt0amJlsrpD6amn7a3Z7PMPQAAAAgCCCTRt82Di/Zli/FMW4XQYL+Zvkq5Z6D5IGXGCuDgAAACDKEGzaILQiWr80s4UEV0Mb91nJZTBgAQAAAFGGYNMGO6Nh4YCqEmnPm3Z7HDflBAAAAJoj2LTBh8UVkqThJoPNlr9IVkDqP1nKHGKuDgAAACAKEWzOoc7n1/7jNZIM99gEh6GxaAAAAABwGoLNOewprZI/YKlXUpyyUj1miijdIR3ZJLljpdGfMlMDAAAAEMUINucQXDhgeL9UuUxN2A/21gy9TEruY6YGAAAAIIoRbM4hGGyMDUMLBKQP/my3WTQAAAAAaFVEgs2hQ4f0+c9/XpmZmUpKStKECRO0YcOGSJwq4nY0rog2wlSwKVojlR+U4lOlEXPM1AAAAABEudhwf+CJEyd00UUX6ZJLLtE//vEPZWVlac+ePerVq1e4T9Ulmu5hYyjYbH7Bfh51jRSXaKYGAAAAIMqFPdj89Kc/VV5enp555pnQtkGDBoX7NF2ivManI+V1kgwt9eyrk7Yus9vjGYYGAAAAnEnYh6K9+uqrmjx5sq6//nplZWVp4sSJevrpp894vNfrVUVFRYtHtAgOQ+vfK1FpCXFdX8Cuf0necimtvzRwetefHwAAAHCIsAebvXv36oknntCwYcP0r3/9S3fccYe++c1v6rnnnmv1+EWLFik9PT30yMvLC3dJHRYMNsP7pZgp4IM/2c9jr5fcrPMAAAAAnInLsiwrnB8YHx+vyZMn69133w1t++Y3v6l169Zp9erVpx3v9Xrl9XpDrysqKpSXl6fy8nKlpaWFs7R2q/I2aEdxhdwulybm9+7ak9eUST8fLgV80tdWS/1Gde35AQAAAMMqKiqUnp7epmwQ9jk2OTk5GjWq5Zfw8847Ty+++GKrx3s8Hnk8hm58eQ4pnlhNGphh5uRbX7ZDTb+xhBoAAADgHMI+vumiiy7Sjh07WmzbuXOnBg4cGO5TdW/BYWgsGgAAAACcU9iDzbe+9S2tWbNGDz/8sHbv3q2lS5fqqaee0rx588J9qu6rbJ99/xq5pDHXma4GAAAAiHphDzYXXHCBXn75ZT3//PMaM2aMfvSjH+mxxx7TzTffHO5TdV9b/mw/D54ppeWYrQUAAABwgLDPsZGkq6++WldffXUkPrr7syzpgz/a7XE3mq0FAAAAcAjWEI42hzZKx3dLsYnSeYRDAAAAoC0INtFm60v283lXS55Us7UAAAAADkGwiTZ7C+3nEXPM1gEAAAA4CMEmmtSUSUf/Y7cHXWy2FgAAAMBBCDbR5OBqSZbUZ4SUkmW6GgAAAMAxCDbRZP8q+3nQRWbrAAAAAByGYBNN9r9tPw+abrYOAAAAwGEINtGipkwqbpxfM5BgAwAAALQHwSZahObXDJdS+5muBgAAAHAUgk20CM2vobcGAAAAaC+CTbQg2AAAAAAdRrCJBrUnpOItdpv5NQAAAEC7EWyiwYHG+TWZw5hfAwAAAHQAwSYaMAwNAAAA6BSCTTTg/jUAAABApxBsTGs+v4ZgAwAAAHQIwca0g2vUNL8m23Q1AAAAgCMRbEwLza+5yGwdAAAAgIMRbEwLza+52GwdAAAAgIMRbEyqPSkd+cBuD6THBgAAAOgogo1JB4P3rxkqpeWYrgYAAABwLIKNSdy/BgAAAAgLgo1JzK8BAAAAwoJgY0rtyab71zC/BgAAAOgUgo0pB9dIVkDKGML8GgAAAKCTCDamhIahMb8GAAAA6CyCjSmhhQOYXwMAAAB0FsHGhNqTUnHj/WsGMb8GAAAA6CyCjQkt5tfkmq4GAAAAcDyCjQkHuH8NAAAAEE4EGxO4MScAAAAQVgSbrlZXLh3ZbLe5fw0AAAAQFgSbrhaaXzNYSu9vuhoAAACgWyDYdDXuXwMAAACEHcGmq3H/GgAAACDsCDZdifk1AAAAQEQQbLrSwbX2/JreBcyvAQAAAMKIYNOVmF8DAAAARATBpisxvwYAAACICIJNV6mrkI5sstuDmF8DAAAAhBPBpqsE71/Tu0BKH2C6GgAAAKBbIdh0FebXAAAAABFDsOkqB96xnwk2AAAAQNgRbLpCXYV0eJPd5v41AAAAQNgRbLpC0VrJ8ku9B0m98kxXAwAAAHQ7BJuuwPwaAAAAIKIINl2B+9cAAAAAEUWwiTRvJfNrAAAAgAgj2ETawcb5Nb0GMr8GAAAAiBCCTaSF5tcwDA0AAACIFIJNpIXm17BwAAAAABApBJtI8lZKh9+324OYXwMAAABECsEmklrMr8k3XQ0AAADQbRFsIon5NQAAAECXINhEUmh+DcPQAAAAgEgi2ESKt6ppfg33rwEAAAAiimATKUVrGufX5Eu9B5quBgAAAOjWCDaREhqGxvwaAAAAINIINpHC/WsAAACALkOwiYSG+mbza6aZrQUAAADoAQg2kXBspxRokDzp9j1sAAAAAEQUwSYSjm61n/uNklwus7UAAAAAPQDBJhJKgsFmtNk6AAAAgB4i4sFm0aJFcrlcmj9/fqRPFT2CPTZZo8zWAQAAAPQQEQ0269at01NPPaVx48ZF8jTR5+g2+7nfGLN1AAAAAD1ExIJNVVWVbr75Zj399NPq3bt3pE4TfWrKpMrDdjvrPLO1AAAAAD1ExILNvHnzdNVVV+myyy4763Fer1cVFRUtHo5W0thb0ytfSkgzWwsAAADQQ8RG4kNfeOEFbdy4UevWrTvnsYsWLdIPfvCDSJRhRmhFNIahAQAAAF0l7D02RUVFuuuuu/SHP/xBCQkJ5zz+3nvvVXl5eehRVFQU7pK6FgsHAAAAAF0u7D02GzZsUElJiSZNmhTa5vf7tXLlSi1evFher1cxMTGhfR6PRx6PJ9xlmHOUpZ4BAACArhb2YHPppZdqy5YtLbZ98Ytf1MiRI3XPPfe0CDXdTiAglWy32wQbAAAAoMuEPdikpqZqzJiW80uSk5OVmZl52vZu5+R+yVctxXikjCGmqwEAAAB6jIjfoLNHCd6/pu8IKSYi6zIAAAAAaEWXfPtesWJFV5zGPFZEAwAAAIygxyacSoLBhhXRAAAAgK5EsAknVkQDAAAAjCDYhEt9jVS2125nEWwAAACArkSwCZfSDyUrICX1kVKyTFcDAAAA9CgEm3ApaVwRrd8oyeUyWwsAAADQwxBswoUV0QAAAABjCDbhEgw2WayIBgAAAHQ1gk24sCIaAAAAYAzBJhyqSqSaY5JcUt+RpqsBAAAAehyCTTgc/Y/9nDlEik8yWwsAAADQAxFswuFo44pozK8BAAAAjCDYhAMrogEAAABGEWzCoSQYbOixAQAAAEwg2HSWv0Eq+dBusyIaAAAAYATBprPK9kp+rxSXLPUaZLoaAAAAoEci2HRWcEW0rPMkN79OAAAAwAS+iXdWSeOKaMyvAQAAAIwh2HQWK6IBAAAAxhFsOisYbLiHDQAAAGAMwaYzvJXSyQN2mxXRAAAAAGMINp1Rst1+Ts2RkjLM1gIAAAD0YASbzgiuiEZvDQAAAGAUwaYzjjauiMb8GgAAAMAogk1nsCIaAAAAEBUINh1lWVJJMNjQYwMAAACYRLDpqIpDUl255I6V+gw3XQ0AAADQoxFsOio4vyZzmBTrMVsLAAAA0MMRbDqKFdEAAACAqEGw6aiSxh4b5tcAAAAAxhFsOooV0QAAAICoQbDpiIZ66dhOu809bAAAAADjCDYdcWynFGiQPOlS+gDT1QAAAAA9HsGmI5rPr3G5zNYCAAAAgGDTIayIBgAAAEQVgk1HBO9hw/waAAAAICoQbDqCFdEAAACAqEKwaa+aMqnysN3OOs9sLQAAAAAkEWzaL7hwQK98KSHNbC0AAAAAJBFs2i80v4aFAwAAAIBoQbBpL1ZEAwAAAKIOwaa9mt/DBgAAAEBUINi0RyDQNBSNFdEAAACAqEGwaY+TByRftRTjkTKGmK4GAAAAQCOCTXsE71/Td4QUE2u2FgAAAAAhBJv2CM2vYeEAAAAAIJoQbNqDFdEAAACAqESwaY/QPWxYEQ0AAACIJgSbtvLVSmV77DYrogEAAABRhWDTVqUfSlZASsqUUrJMVwMAAACgGYJNWwVXROs3WnK5zNYCAAAAoAWCTVuF5tewcAAAAAAQbQg2bcWKaAAAAEDUIti0VegeNqyIBgAAAEQbgk1bVJVI1aWSXFLf80xXAwAAAOAUBJu2CC4ckDFYik8yWwsAAACA0xBs2qL5imgAAAAAog7Bpi1C82sINgAAAEA0Iti0BSuiAQAAAFGNYHMu/gapdIfdzmJFNAAAACAaEWzOpWyv1FAnxSVJvQtMVwMAAACgFQSbcylpXDgg6zzJza8LAAAAiEZ8Uz8XVkQDAAAAol7Yg82iRYt0wQUXKDU1VVlZWbr22mu1Y8eOcJ+m6xxtXBEti2ADAAAARKuwB5vCwkLNmzdPa9as0fLly9XQ0KDZs2eruro63KfqGqyIBgAAAES92HB/4D//+c8Wr5955hllZWVpw4YNmjFjRrhPF1neSunkAbtNsAEAAACiVtiDzanKy8slSRkZGa3u93q98nq9odcVFRWRLqntSrbbz6k5UlLr9QMAAAAwL6KLB1iWpQULFmj69OkaM2ZMq8csWrRI6enpoUdeXl4kS2qf4MIB3L8GAAAAiGouy7KsSH34vHnz9Le//U2rVq3SgAEDWj2mtR6bvLw8lZeXKy0tLVKltU1VqXRovX0Pm8EzzdYCAAAA9DAVFRVKT09vUzaI2FC0O++8U6+++qpWrlx5xlAjSR6PRx6PJ1JldE5KX2nEHNNVAAAAADiHsAcby7J055136uWXX9aKFStUUFAQ7lMAAAAAQAthDzbz5s3T0qVL9corryg1NVXFxcWSpPT0dCUmJob7dAAAAAAQ/jk2Lper1e3PPPOMbrvttnO+vz3j6AAAAAB0X0bn2ERwLQIAAAAAaFVEl3sGAAAAgK5AsAEAAADgeAQbAAAAAI5HsAEAAADgeAQbAAAAAI5HsAEAAADgeAQbAAAAAI5HsAEAAADgeAQbAAAAAI5HsAEAAADgeAQbAAAAAI5HsAEAAADgeAQbAAAAAI4Xa7qAU1mWJUmqqKgwXAkAAAAAk4KZIJgRzibqgk1lZaUkKS8vz3AlAAAAAKJBZWWl0tPTz3qMy2pL/OlCgUBAhw8fVmpqqlwul+lyVFFRoby8PBUVFSktLc10OXAQrh10BtcPOoPrB53B9YPOCPf1Y1mWKisrlZubK7f77LNooq7Hxu12a8CAAabLOE1aWhr/cqNDuHbQGVw/6AyuH3QG1w86I5zXz7l6aoJYPAAAAACA4xFsAAAAADgeweYcPB6PHnjgAXk8HtOlwGG4dtAZXD/oDK4fdAbXDzrD5PUTdYsHAAAAAEB70WMDAAAAwPEINgAAAAAcj2ADAAAAwPEINgAAAAAcj2ADAAAAwPEINmfx61//WgUFBUpISNCkSZP09ttvmy4JUWjlypWaO3eucnNz5XK5tGzZshb7LcvSgw8+qNzcXCUmJmrWrFnaunWrmWIRVRYtWqQLLrhAqampysrK0rXXXqsdO3a0OIbrB2fyxBNPaNy4caG7e0+dOlX/+Mc/Qvu5dtAeixYtksvl0vz580PbuIZwJg8++KBcLleLR3Z2dmi/qWuHYHMGf/zjHzV//nzdd999ev/993XxxRdrzpw5OnjwoOnSEGWqq6s1fvx4LV68uNX9jzzyiB599FEtXrxY69atU3Z2ti6//HJVVlZ2caWINoWFhZo3b57WrFmj5cuXq6GhQbNnz1Z1dXXoGK4fnMmAAQP0k5/8ROvXr9f69ev18Y9/XNdcc03oywPXDtpq3bp1euqppzRu3LgW27mGcDajR4/WkSNHQo8tW7aE9hm7diy06sILL7TuuOOOFttGjhxpLVy40FBFcAJJ1ssvvxx6HQgErOzsbOsnP/lJaFtdXZ2Vnp5uPfnkkwYqRDQrKSmxJFmFhYWWZXH9oP169+5t/fa3v+XaQZtVVlZaw4YNs5YvX27NnDnTuuuuuyzL4u8Pzu6BBx6wxo8f3+o+k9cOPTatqK+v14YNGzR79uwW22fPnq13333XUFVwon379qm4uLjFteTxeDRz5kyuJZymvLxckpSRkSGJ6wdt5/f79cILL6i6ulpTp07l2kGbzZs3T1dddZUuu+yyFtu5hnAuu3btUm5urgoKCnTjjTdq7969ksxeO7ER/XSHOnbsmPx+v/r169die79+/VRcXGyoKjhR8Hpp7Vo6cOCAiZIQpSzL0oIFCzR9+nSNGTNGEtcPzm3Lli2aOnWq6urqlJKSopdfflmjRo0KfXng2sHZvPDCC9q4caPWrVt32j7+/uBspkyZoueee07Dhw/X0aNH9eMf/1jTpk3T1q1bjV47BJuzcLlcLV5blnXaNqAtuJZwLt/4xjf0wQcfaNWqVaft4/rBmYwYMUKbNm3SyZMn9eKLL+rWW29VYWFhaD/XDs6kqKhId911l15//XUlJCSc8TiuIbRmzpw5ofbYsWM1depUDRkyRM8++6w+9rGPSTJz7TAUrRV9+vRRTEzMab0zJSUlp6VP4GyCK4RwLeFs7rzzTr366qt66623NGDAgNB2rh+cS3x8vIYOHarJkydr0aJFGj9+vH75y19y7eCcNmzYoJKSEk2aNEmxsbGKjY1VYWGhfvWrXyk2NjZ0nXANoS2Sk5M1duxY7dq1y+jfH4JNK+Lj4zVp0iQtX768xfbly5dr2rRphqqCExUUFCg7O7vFtVRfX6/CwkKuJciyLH3jG9/QSy+9pDfffFMFBQUt9nP9oL0sy5LX6+XawTldeuml2rJlizZt2hR6TJ48WTfffLM2bdqkwYMHcw2hzbxer7Zv366cnByjf38YinYGCxYs0Be+8AVNnjxZU6dO1VNPPaWDBw/qjjvuMF0aokxVVZV2794der1v3z5t2rRJGRkZys/P1/z58/Xwww9r2LBhGjZsmB5++GElJSXppptuMlg1osG8efO0dOlSvfLKK0pNTQ39v1vp6elKTEwM3VOC6wet+d73vqc5c+YoLy9PlZWVeuGFF7RixQr985//5NrBOaWmpobm8wUlJycrMzMztJ1rCGfyne98R3PnzlV+fr5KSkr04x//WBUVFbr11lvN/v2J6JprDvc///M/1sCBA634+Hjr/PPPDy3BCjT31ltvWZJOe9x6662WZdnLHj7wwANWdna25fF4rBkzZlhbtmwxWzSiQmvXjSTrmWeeCR3D9YMzuf3220P/jerbt6916aWXWq+//npoP9cO2qv5cs+WxTWEM/vsZz9r5eTkWHFxcVZubq716U9/2tq6dWtov6lrx2VZlhXZ6AQAAAAAkcUcGwAAAACOR7ABAAAA4HgEGwAAAACOR7ABAAAA4HgEGwAAAACOR7ABAAAA4HgEGwAAAACOR7ABAAAA4HgEGwAAAACOR7ABAAAA4HgEGwAAAACO9/8B+0L9NCU4DD8AAAAASUVORK5CYII=",
      "text/plain": [
       "<Figure size 1000x600 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "V = np.zeros(nb_states)\n",
    "Q = np.zeros((nb_states, nb_actions))\n",
    "\n",
    "pi = np.array([a['search'], a['search']], dtype=int)\n",
    "\n",
    "V_high_history = []\n",
    "V_low_history = []\n",
    "\n",
    "for k in range(50):\n",
    "    \n",
    "    # Policy evaluation\n",
    "    Q[s['high'], a['search']] = alpha * (r_search + gamma * V[s['high']]) + (1 - alpha) * (r_search + gamma*V[s['low']])\n",
    "    Q[s['high'], a['wait']] = r_wait + gamma * V[s['high']]\n",
    "\n",
    "    Q[s['low'], a['search']] = beta * (r_search + gamma * V[s['low']]) + (1 - beta) * (-3 + gamma*V[s['high']])\n",
    "    Q[s['low'], a['wait']] = r_wait + gamma * V[s['low']]\n",
    "    Q[s['low'], a['recharge']] = gamma * V[s['high']]\n",
    "\n",
    "    V[s['high']] = Q[s['high'], :2].max()\n",
    "    V[s['low']] = Q[s['low'], :].max()\n",
    "\n",
    "    V_high_history.append(V[s['high']])\n",
    "    V_low_history.append(V[s['low']])\n",
    "        \n",
    "    # Compute the greedy policy\n",
    "    pi_old = pi.copy()\n",
    "    pi[s['high']] = Q[s['high'], :2].argmax()\n",
    "    pi[s['low']] = Q[s['low'], :].argmax()\n",
    "    \n",
    "print('Greedy policy after iteration', k)\n",
    "print('pi(high)=', pi[s['high']])\n",
    "print('pi(low)=', pi[s['low']])\n",
    "print(\"V=\", V)\n",
    "print(\"Q=\", Q)\n",
    "    \n",
    "plt.figure(figsize=(10, 6))  \n",
    "plt.plot(V_high_history, label=\"high\")\n",
    "plt.plot(V_low_history, label=\"low\")\n",
    "plt.legend()\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Q:**  Change the value of the discount factor $\\gamma =0.3$ so that the agent becomes short-sighted: it only takes into account the immediate rewards, but forgets about the long-term. Does it change the strategy? Explain why."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Greedy policy after iteration 49\n",
      "pi(high)= 0\n",
      "pi(low)= 1\n",
      "V= [7.25274725 2.85714286]\n",
      "Q= [[7.25274725 4.17582418 0.        ]\n",
      " [0.71208791 2.85714286 2.17582418]]\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAy0AAAH5CAYAAACMINEWAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjYuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/av/WaAAAACXBIWXMAAA9hAAAPYQGoP6dpAAAqk0lEQVR4nO3de5TddX3v/9eeaxIyM5CYkMRMMMQQCBBQghpQiXKxSD3YrtJab1A89nhWtKb8XHjrEdqqobXtartoVayHny7B+DsLQbpOuaoJVS4GbARCGoKwIEhCwJKZJCSTzMz398ckYy6TZHaSmf1N5vFYa699+87s94xf4zz9fj97V4qiKAIAAFBSdbUeAAAAYH9ECwAAUGqiBQAAKDXRAgAAlJpoAQAASk20AAAApSZaAACAUmsY7hfs7e3NCy+8kJaWllQqleF+eQAAoCSKosjGjRszZcqU1NXt+3jKsEfLCy+8kPb29uF+WQAAoKTWrFmTqVOn7vP5YY+WlpaWJH2Dtba2DvfLAwAAJdHZ2Zn29vb+RtiXYY+WnaeEtba2ihYAAOCAy0YsxAcAAEpNtAAAAKUmWgAAgFITLQAAQKmJFgAAoNRECwAAUGqiBQAAKDXRAgAAlJpoAQAASk20AAAApSZaAACAUhMtAABAqYkWAACg1EQLAABQaqIFAAAoNdECAACUmmgBAABKraHWAzB0Nnd156WNXXnl1W3Z1t2bbT29fdc7bnd1735/r9s9ventLdJbFOkt0nfd+5vbxc7Hdjxf7Lrdjvs77bxZpOi/v+tjv7m9H/t5stj/Vw6JYvhfEgDgsPjvbzsxv3XapFqPMWii5QizZVtPXtrYlZc2deWljV15eVPfZeftvutteWljV7Zs76n1uAAAlNClb+iq9QhVES0lVhRFVq/flLseX5d7Vr6YX67flM3bqguR0Y31GXdMU5ob69JUX5fmhro07bzU77xdn8b6St9z9bs+X5+G+koqlaSuUkndjuvKLrfrKtlxf9fn+x6rJKlU+uaoVJK+R37zWN/tHY/tum122aB/u71/tj0fGmgbAAD2NntyW61HqIpoKZne3iLLn9+Qu1asy90rXswzL2/ea5vmhrpMaGnOa8b2XSa0NGfC2Kb+x3a9PqbZf8QAABzZ/EVbAtt7evPg07/OXSvW5Z4nXsyLnb85XNdUX5dzXz8+7zp1Us6ePi4TW5oztrmh/wgFAAAc7URLjWzZ1pOlT76Uu1asyw9XvpjOrd39z41tbsj8WRPyrlMnZf6sCWkZ1VjDSQEAoLZEyzDq6S3yr794If/22Nrct/qlbN3e2//c+GOacuHs4/OuUyflnNePT3NDfQ0nBQCA8hAtw6Qoinzmlkfzfx55vv+xqceNzrtOnZR3nTopZ51wXOrrnPIFAAB7Ei3D5GtLn87/eeT51FWS/zl/Rt59+uTMntxqbQoAAByAaBkGdz6+Nn91538mSb7w27NzxbnTazwRAAAcOepqPcDR7rHnO7Lwe8uTJB+ed4JgAQCAKomWIbS2Y0s+8q1l2bq9N28/aUK+8Nuzaz0SAAAccUTLENnc1Z2P/L8PZ/3GrsycODbXv/8Naaj36wYAgGr5K3oI9PQW+eTi5XlibWfGH9OU/33F2Wn1WSsAAHBQRMsQ+Ks7/zP3rnwxTQ11ueHDc9M+bkytRwIAgCOWaDnMFv/sudxw39NJkq/83pycdcJxNZ4IAACObKLlMPrpUy/nz257PEnyyfNn5tIzX1vjiQAA4MgnWg6Tp9Zvyv/8ziPp7i3y386YkoUXzKz1SAAAcFSoKlpe97rXpVKp7HVZsGDBUM13RHhl87Z85FvL0rm1O2+cdmz++vfm+KR7AAA4TBqq2XjZsmXp6enpv//444/nwgsvzGWXXXbYBztSdHX35H9855E8++tXM/W40bnhw3MzqrG+1mMBAMBRo6pomTBhwm73r7vuusyYMSPnnXfePr+mq6srXV1d/fc7OzurHLG8iqLI577/eH72zH9lbHNDvnn52XnN2OZajwUAAEeVg17Tsm3btnznO9/JlVdeud9ToRYtWpS2trb+S3t7+8G+ZOn885Jf5pafP5+6SnL9+9+QWZNaaj0SAAAcdQ46Wm677bZs2LAhV1xxxX63++xnP5uOjo7+y5o1aw72JUvljsfW5it3rUqSXPvfTs38WRNrPBEAABydqjo9bFff/OY3c/HFF2fKlCn73a65uTnNzUfXKVOPPr8hf/r/LU+SXHHO6/Lhea+r6TwAAHA0O6hoefbZZ3Pvvffm+9///uGe54jwv257PFu392b+rAn5s0tOqfU4AABwVDuo08NuvPHGTJw4MZdccsnhnqf0tm7vyeMv9L2ZwJd+5/Q01PuoGwAAGEpV/8Xd29ubG2+8MZdffnkaGg767LIj1uoXN6Wnt8hxYxozpW1UrccBAICjXtXRcu+99+a5557LlVdeORTzlN4TazuSJLOntPoASQAAGAZVHyq56KKLUhTFUMxyRHhix6lhsye31ngSAAAYGSzIqNITa3dEyxTRAgAAw0G0VKG3t8jKtRuTJLMnt9V4GgAAGBlESxXWvPJqNnV1p6mhLidOOKbW4wAAwIggWqqwcz3LrONb0uitjgEAYFj4y7sKK9dahA8AAMNNtFTBInwAABh+oqUK/W93LFoAAGDYiJZBemXztrzQsTVJcvKklhpPAwAAI4doGaSd61lOGD8mLaMaazwNAACMHKJlkJ6wCB8AAGpCtAxS/3oW0QIAAMNKtAySdw4DAIDaEC2DsHV7T55avymJaAEAgOEmWgbhqfWb0t1b5LgxjZnUOqrW4wAAwIgiWgZh189nqVQqNZ4GAABGFtEyCDvXs5wyyalhAAAw3ETLIOx6pAUAABheouUAensL7xwGAAA1JFoO4PlXtmRTV3ea6usyY8LYWo8DAAAjjmg5gCfWdiRJTpo0No31fl0AADDc/BV+AP3rWSY7NQwAAGpBtBxA/3oW0QIAADUhWg7gN+8c1lbjSQAAYGQSLfvxyuZteaFja5Lk5MktNZ4GAABGJtGyHyt3nBo2bdyYtI5qrPE0AAAwMomW/bCeBQAAak+07IcPlQQAgNoTLfvh7Y4BAKD2RMs+dHX35Kn1m5I40gIAALUkWvZh9Yub0t1b5NgxjZncNqrW4wAAwIglWvZh10X4lUqlxtMAAMDIJVr2wXoWAAAoB9GyD945DAAAykG0DKAoiqx8QbQAAEAZiJYBPP/Klmzs6k5TfV1mTBhb63EAAGBEEy0DWLHjKMtJk8amsd6vCAAAaslf5APYuZ7llElODQMAgFoTLQN4wnoWAAAoDdEygJVrvd0xAACUhWjZw4ZXt+VXG7YkSU5xpAUAAGpOtOxh53qW9nGj0zqqscbTAAAAomUP/etZnBoGAAClIFr28ET/epa2Gk8CAAAkomUv3jkMAADKRbTsoqu7J0+t35REtAAAQFmIll2sfnFTunuLtI1uzJS2UbUeBwAAiGjZza6fz1KpVGo8DQAAkIiW3fQvwndqGAAAlIZo2YW3OwYAgPIRLTsUReFICwAAlJBo2eH5V7Zk49buNNXXZcaEsbUeBwAA2EG07LDzKMvM48emqcGvBQAAysJf5ztYzwIAAOUkWnawngUAAMpJtOzgSAsAAJSTaEnS8er2/GrDliTJKY60AABAqYiW/ObUsPZxo9M6qrHG0wAAALsSLdllPYtTwwAAoHRES36znuUU0QIAAKUjWuJICwAAlNmIj5Zt3b15av3GJN7uGAAAymjER8vq9RuzvadI66iGvPbY0bUeBwAA2MOIj5b+z2eZ0ppKpVLjaQAAgD2Jlv71LG01ngQAABiIaNnlSAsAAFA+IzpaiqLwzmEAAFByIzpann9lSzZu7U5jfSWvnzi21uMAAAADGNHRsnLHUZaZE1vS1DCifxUAAFBaI/ov9f5Tw6xnAQCA0qo6Wn71q1/lgx/8YMaPH58xY8bkzDPPzCOPPDIUsw25/kX41rMAAEBpNVSz8SuvvJJzzz0373jHO3LHHXdk4sSJ+eUvf5ljjz12iMYbWo60AABA+VUVLX/1V3+V9vb23Hjjjf2Pve51r9vv13R1daWrq6v/fmdnZ3UTDpGOLdvz/CtbkiSnONICAAClVdXpYbfffnvmzp2byy67LBMnTswb3vCGfOMb39jv1yxatChtbW39l/b29kMa+HDZuQh/6nGj0za6scbTAAAA+1IpiqIY7MajRo1Kklx11VW57LLL8rOf/SwLFy7M17/+9Xz4wx8e8GsGOtLS3t6ejo6OtLbW7gjHrzZsyR2PrU1dpZIr3zq9ZnMAAMBI1dnZmba2tgO2QVXR0tTUlLlz5+b+++/vf+xP/uRPsmzZsjzwwAOHdTAAAODoNtg2qOr0sMmTJ2f27Nm7PXbKKafkueeeO7gpAQAADqCqaDn33HOzatWq3R578sknc8IJJxzWoQAAAHaqKlr+9E//NA8++GC+/OUv56mnnsrNN9+cG264IQsWLBiq+QAAgBGuqmg5++yzc+utt+a73/1uTjvttPzlX/5l/v7v/z4f+MAHhmo+AABghKtqIf7hYCE+AACQDNFCfAAAgOEmWgAAgFITLQAAQKmJFgAAoNRECwAAUGqiBQAAKDXRAgAAlJpoAQAASk20AAAApSZaAACAUhMtAABAqYkWAACg1EQLAABQaqIFAAAoNdECAACUmmgBAABKTbQAAAClJloAAIBSEy0AAECpiRYAAKDURAsAAFBqogUAACg10QIAAJSaaAEAAEpNtAAAAKUmWgAAgFITLQAAQKmJFgAAoNRECwAAUGqiBQAAKDXRAgAAlJpoAQAASk20AAAApSZaAACAUhMtAABAqYkWAACg1EQLAABQaqIFAAAoNdECAACUmmgBAABKTbQAAAClJloAAIBSEy0AAECpiRYAAKDURAsAAFBqogUAACg10QIAAJSaaAEAAEpNtAAAAKUmWgAAgFITLQAAQKmJFgAAoNRECwAAUGqiBQAAKDXRAgAAlJpoAQAASk20AAAApSZaAACAUhMtAABAqYkWAACg1EQLAABQaqIFAAAoNdECAACUmmgBAABKTbQAAAClJloAAIBSEy0AAECpiRYAAKDURAsAAFBqVUXLtddem0qlsttl0qRJQzUbAABAGqr9glNPPTX33ntv//36+vrDOhAAAMCuqo6WhoaGqo6udHV1paurq/9+Z2dntS8JAACMYFWvaVm9enWmTJmS6dOn533ve1+efvrp/W6/aNGitLW19V/a29sPelgAAGDkqRRFUQx24zvuuCOvvvpqTjrppLz44ov54he/mP/8z//MihUrMn78+AG/ZqAjLe3t7eno6Ehra+uh/wQAAMARqbOzM21tbQdsg6qiZU+bN2/OjBkzcvXVV+eqq646rIMBAABHt8G2wSG95fExxxyT008/PatXrz6UbwMAALBPhxQtXV1dWblyZSZPnny45gEAANhNVdHyqU99KkuXLs0zzzyThx56KL/3e7+Xzs7OXH755UM1HwAAMMJV9ZbHzz//fP7wD/8wL7/8ciZMmJC3vOUtefDBB3PCCScM1XwAAMAIV1W0LF68eKjmAAAAGNAhrWkBAAAYaqIFAAAoNdECAACUmmgBAABKTbQAAAClJloAAIBSEy0AAECpiRYAAKDURAsAAFBqogUAACg10QIAAJSaaAEAAEpNtAAAAKUmWgAAgFITLQAAQKmJFgAAoNRECwAAUGqiBQAAKDXRAgAAlJpoAQAASk20AAAApSZaAACAUhMtAABAqYkWAACg1EQLAABQaqIFAAAoNdECAACUmmgBAABKTbQAAAClJloAAIBSEy0AAECpiRYAAKDURAsAAFBqogUAACg10QIAAJSaaAEAAEpNtAAAAKUmWgAAgFITLQAAQKmJFgAAoNRECwAAUGqiBQAAKDXRAgAAlJpoAQAASk20AAAApSZaAACAUhMtAABAqYkWAACg1EQLAABQaqIFAAAoNdECAACUmmgBAABKTbQAAAClJloAAIBSEy0AAECpiRYAAKDURAsAAFBqogUAACg10QIAAJSaaAEAAEpNtAAAAKUmWgAAgFITLQAAQKmJFgAAoNRECwAAUGqiBQAAKDXRAgAAlJpoAQAASk20AAAApXZI0bJo0aJUKpUsXLjwMI0DAACwu4OOlmXLluWGG27InDlzDuc8AAAAuzmoaNm0aVM+8IEP5Bvf+EaOO+64/W7b1dWVzs7O3S4AAACDdVDRsmDBglxyySW54IILDrjtokWL0tbW1n9pb28/mJcEAABGqKqjZfHixfn5z3+eRYsWDWr7z372s+no6Oi/rFmzpuohAQCAkauhmo3XrFmTT37yk7n77rszatSoQX1Nc3NzmpubD2o4AACASlEUxWA3vu222/I7v/M7qa+v73+sp6cnlUoldXV16erq2u25gXR2dqatrS0dHR1pbW09+MkBAIAj2mDboKojLeeff34ee+yx3R77oz/6o5x88sn59Kc/fcBgAQAAqFZV0dLS0pLTTjttt8eOOeaYjB8/fq/HAQAADodD+nBJAACAoVbVkZaBLFmy5DCMAQAAMDBHWgAAgFITLQAAQKmJFgAAoNRECwAAUGqiBQAAKDXRAgAAlJpoAQAASk20AAAApSZaAACAUhMtAABAqYkWAACg1EQLAABQaqIFAAAoNdECAACUmmgBAABKTbQAAAClJloAAIBSEy0AAECpiRYAAKDURAsAAFBqogUAACg10QIAAJSaaAEAAEpNtAAAAKUmWgAAgFITLQAAQKmJFgAAoNRECwAAUGqiBQAAKDXRAgAAlJpoAQAASk20AAAApSZaAACAUhMtAABAqYkWAACg1EQLAABQaqIFAAAoNdECAACUmmgBAABKTbQAAAClJloAAIBSEy0AAECpiRYAAKDURAsAAFBqogUAACg10QIAAJSaaAEAAEpNtAAAAKUmWgAAgFITLQAAQKmJFgAAoNRECwAAUGqiBQAAKDXRAgAAlJpoAQAASk20AAAApSZaAACAUhMtAABAqYkWAACg1EQLAABQaqIFAAAoNdECAACUmmgBAABKTbQAAAClJloAAIBSEy0AAECpiRYAAKDURAsAAFBqogUAACi1qqLlq1/9aubMmZPW1ta0trZm3rx5ueOOO4ZqNgAAgOqiZerUqbnuuuvy8MMP5+GHH8473/nOXHrppVmxYsVQzQcAAIxwlaIoikP5BuPGjctXvvKVfOQjHxnw+a6urnR1dfXf7+zsTHt7ezo6OtLa2nooLw0AABzBOjs709bWdsA2OOg1LT09PVm8eHE2b96cefPm7XO7RYsWpa2trf/S3t5+sC8JAACMQFUfaXnssccyb968bN26NWPHjs3NN9+cd7/73fvc3pEWAABgIIM90tJQ7TeeNWtWli9fng0bNuSWW27J5ZdfnqVLl2b27NkDbt/c3Jzm5uZqXwYAACDJYVjTcsEFF2TGjBn5+te/PqjtB1tTAADA0W3I17TsVBTFbqd/AQAAHE5VnR72uc99LhdffHHa29uzcePGLF68OEuWLMmdd945VPMBAAAjXFXR8uKLL+ZDH/pQ1q5dm7a2tsyZMyd33nlnLrzwwqGaDwAAGOGqipZvfvObQzUHAADAgA55TQsAAMBQEi0AAECpVf05LRyliiLp2Z70dCXd2/que7YlvT19zxW9Oy49u9zeeSkGeGzH433ffPfbO1+v//aej+813MDzHjaH83sBABwBJs5Ojjuh1lMMmmg5GnRtSjavTza9tON6fbL5pR3X65NXX9kRIztCZMBrb1sNADBiXPK3ydn/vdZTDJpoORK8+l/JU/cmL68eOE62v3r4X7OuIalv6ruuVJJK3R6X+l1uD/R8JUllx3X6bicD3N/Pc7uqDPDYntsNuA0AAHs5ZkKtJ6iKaCmjokhefjJZdUfy5J3Jmof6Trfan8YxfTvf2InJMROTsRN2XE9MxoxLGkYnDU19IVLfvON2c9LQ3PfYntd19cPzswIAwAGIlrLo2Z48e39fpKy6I3nlmd2fP/60ZOrZydjjdw+SYyb0PdY8tjZzAwDAEBMttfTqfyWr70mevCN56odJV+dvnqtvSl73tmTWxclJ70qOnVa7OQEAoIZEy3B7+alk1f9NVt2ZrHlw99O+xrymL1BO+q1kxjuS5pbazQkAACUhWobTfX+T/Ogvd39s4qnJrN9KTro4ee0brSUBAIA9iJbhsvQryY+/2Hf7xPnJrEv6jqocQe+PDQAAtSBahsOuwXL+NcnbrqrtPAAAcASpq/UARz3BAgAAh0S0DKX7BAsAABwq0TJU7vtK8iPBAgAAh0q0DAXBAgAAh41oOdx2C5YvCBYAADhEouVw2itY/p/azgMAAEcB0XK43Pc3ggUAAIaAaDkcdv2ke8ECAACHlWg5VIIFAACGlGg5FLsGyzv/l2ABAIAhIFoO1p7B8vZP1XYeAAA4SomWg/HgVwULAAAME9FSrZ7uZMl1fbff8WeCBQAAhphoqdaaB5OtG5Ix431wJAAADAPRUq0n7+y7nnlRUldf21kAAGAEEC3VWrUjWk56V23nAACAEUK0VOPXv0x+vTqpa0xmnF/raQAAYEQQLdVYdUff9evOTUa11nYWAAAYIURLNXauZznpt2o7BwAAjCCiZbC2bEiee6DvtmgBAIBhI1oG66l7k97uZMLJybjptZ4GAABGDNEyWE4NAwCAmhAtg9HTnay+p++2aAEAgGElWgZjzUPJ1g3J6HFJ+5tqPQ0AAIwoomUwntzxVsczL0rq6ms7CwAAjDCiZTBW7VjPMsupYQAAMNxEy4H8+pfJr1cndQ3JjHfWehoAABhxGmo9QOntfNewE85NRrXVdhYAAGqmp6cn27dvr/UYR5TGxsbU1x/68grRciCrdqxnmXVxbecAAKAmiqLIunXrsmHDhlqPckQ69thjM2nSpFQqlYP+HqJlf7ZsSJ57oO/2Se+q6SgAANTGzmCZOHFixowZc0h/fI8kRVHk1Vdfzfr165MkkydPPujvJVr255c/THq7k9fMSsadWOtpAAAYZj09Pf3BMn78+FqPc8QZPXp0kmT9+vWZOHHiQZ8qZiH+/njXMACAEW3nGpYxY8bUeJIj187f3aGsBxIt+9LTnay+u+/2SdazAACMZE4JO3iH43cnWvbl+Z8lWzcko49Lpp5d62kAAGDEEi37svNdw2ZelNRb+gMAALUiWvZl5+eznGQ9CwAAR5758+dn4cKF+3y+UqnktttuG/T3W7JkSSqVSk3e+tkhhIH8+pfJy08mdQ3J68+v9TQAAHDYrV27Nscdd1ytxxgU0TKQJ+/quz7hnGRUW21nAQCAITBp0qRajzBoTg8byJM71rN41zAAAPZQFEVe3dZdk0tRFFXN2tvbm6uvvjrjxo3LpEmTcu211/Y/t+fpYffff3/OPPPMjBo1KnPnzs1tt92WSqWS5cuX7/Y9H3nkkcydOzdjxozJOeeck1WrVh3Cb3NwHGnZ09aO5Nn7+277fBYAAPawZXtPZn/hrpq89hN/8a6MaRr8n/Df+ta3ctVVV+Whhx7KAw88kCuuuCLnnntuLrzwwt2227hxY97znvfk3e9+d26++eY8++yz+1wP8/nPfz5/+7d/mwkTJuRjH/tYrrzyyvz0pz89lB/rgETLnp66N+ntTl5zUjLuxFpPAwAAB23OnDm55pprkiQzZ87M9ddfnx/+8Id7RctNN92USqWSb3zjGxk1alRmz56dX/3qV/noRz+61/f80pe+lPPOOy9J8pnPfCaXXHJJtm7dmlGjRg3ZzyFa9rRzPYt3DQMAYACjG+vzxF+8q2avXY05c+bsdn/y5MlZv379XtutWrUqc+bM2S083vSmNx3we06ePDlJsn79+kybNq2q2aohWnbV052svrvv9izrWQAA2FulUqnqFK1aamxs3O1+pVJJb2/vXtsVRbHXJ9fva/3Mrt9z59cM9D0PJwvxd/X8z5ItrySjjk2mDlyWAABwtDn55JPz6KOPpqurq/+xhx9+uIYT7U607GrVjncNm3lRUn9k1DMAAByq97///ent7c0f//EfZ+XKlbnrrrvyN3/zN0my1xGYWhAtu9q5nsW7hgEAMIK0trbmX//1X7N8+fKceeaZ+fznP58vfOELSTKkC+wHy+GEnf7r6eTlVUldQzLj/FpPAwAAh2TJkiV7Pbbr57LsuWblnHPOyS9+8Yv++zfddFMaGxv7F9jPnz9/r68588wzq/7smIMhWnZadWff9bR5yehjazoKAAAMt29/+9s58cQT89rXvja/+MUv8ulPfzq///u/n9GjR9d6NNHS78kd0eJdwwAAGIHWrVuXL3zhC1m3bl0mT56cyy67LF/60pdqPVYS0dJna0fy7I5P8fT5LAAAjEBXX311rr766lqPMSAL8ZPkqR8mvd3Ja05Kxs+o9TQAAMAuREvym1PDTqrNJ5sCAAD7Jlp6e5LVd/fdPsl6FgAAKBvRsuZnyZZXklHHJu1vrvU0AADAHkTLk3f0Xc+8MKn3vgQAAFA2omXn57N41zAAAI4i8+fPz8KFC2s9xmFRVbQsWrQoZ599dlpaWjJx4sS8973vzapVq4ZqtqH3X08nL69K6hqS119Q62kAAIABVBUtS5cuzYIFC/Lggw/mnnvuSXd3dy666KJs3rx5qOYbWk/e1Xc9bV4y+tiajgIAAAysqmi58847c8UVV+TUU0/NGWeckRtvvDHPPfdcHnnkkaGab2it2rGexalhAAAcxV555ZV8+MMfznHHHZcxY8bk4osvzurVq5MkRVFkwoQJueWWW/q3P/PMMzNx4sT++w888EAaGxuzadOmYZ89OcQ1LR0dHUmScePG7XObrq6udHZ27nYpha0dybM/7bs9y1sdAwAwSEWRbNtcm0tRHNTIV1xxRR5++OHcfvvteeCBB1IURd797ndn+/btqVQqefvb354lS5Yk6QucJ554Itu3b88TTzyRJFmyZEnOOuusjB079nD9Fqty0G+XVRRFrrrqqrz1rW/Naaedts/tFi1alD//8z8/2JcZOr/8UdLbnYyfmYyfUetpAAA4Umx/NfnylNq89udeSJqOqepLVq9endtvvz0//elPc8455yRJbrrpprS3t+e2227LZZddlvnz5+eGG25Iktx3330544wzMm3atCxZsiSzZ8/OkiVLMn/+/MP90wzaQR9p+fjHP55HH3003/3ud/e73Wc/+9l0dHT0X9asWXOwL3l4zbok+dBtyYUlDCoAADhMVq5cmYaGhrz5zb/5TMLx48dn1qxZWblyZZK+dxpbsWJFXn755SxdujTz58/P/Pnzs3Tp0nR3d+f+++/PeeedV6sf4eCOtHziE5/I7bffnvvuuy9Tp07d77bNzc1pbm4+qOGGVENTMuMdtZ4CAIAjTeOYviMetXrtKhX7OKWsKIpUKpUkyWmnnZbx48dn6dKlWbp0af7iL/4i7e3t+dKXvpRly5Zly5Yteetb33pIox+KqqKlKIp84hOfyK233polS5Zk+vTpQzUXAACUU6VS9SlatTR79ux0d3fnoYce6j897Ne//nWefPLJnHLKKUnSv67lBz/4QR5//PG87W1vS0tLS7Zv356vfe1reeMb35iWlpaa/QxVnR62YMGCfOc738nNN9+clpaWrFu3LuvWrcuWLVuGaj4AAOAQzJw5M5deemk++tGP5ic/+Ul+8Ytf5IMf/GBe+9rX5tJLL+3fbv78+bn55pszZ86ctLa29ofMTTfdVNP1LEmV0fLVr341HR0dmT9/fiZPntx/+d73vjdU8wEAAIfoxhtvzFlnnZXf/u3fzrx581IURf7t3/4tjY2N/du84x3vSE9Pz26Bct5556Wnp6em61mSpFLs6yS3IdLZ2Zm2trZ0dHSktbV1OF8aAACqsnXr1jzzzDOZPn16Ro0aVetxjkj7+x0Otg0O6XNaAAAAhppoAQAASk20AAAApSZaAACAUhMtAABAqYkWAAA4gN7e3lqPcMQ6HL+7hsMwBwAAHJWamppSV1eXF154IRMmTEhTU1MqlUqtxzoiFEWRbdu25aWXXkpdXV2ampoO+nuJFgAA2Ie6urpMnz49a9euzQsvvFDrcY5IY8aMybRp01JXd/AneYkWAADYj6ampkybNi3d3d3p6emp9ThHlPr6+jQ0NBzy0SnRAgAAB1CpVNLY2JjGxsZajzIiWYgPAACUmmgBAABKTbQAAAClNuxrWoqiSJJ0dnYO90sDAAAlsrMJdjbCvgx7tGzcuDFJ0t7ePtwvDQAAlNDGjRvT1ta2z+crxYGy5jDr7e3NCy+8kJaWlpp/ME9nZ2fa29uzZs2atLa21nQWjjz2Hw6WfYdDYf/hUNh/OBRDsf8URZGNGzdmypQp+/0cl2E/0lJXV5epU6cO98vuV2trq//ictDsPxws+w6Hwv7DobD/cCgO9/6zvyMsO1mIDwAAlJpoAQAASm1ER0tzc3OuueaaNDc313oUjkD2Hw6WfYdDYf/hUNh/OBS13H+GfSE+AABANUb0kRYAAKD8RAsAAFBqogUAACg10QIAAJSaaAEAAEptxEbLP//zP2f69OkZNWpUzjrrrPz7v/97rUeihO6777685z3vyZQpU1KpVHLbbbft9nxRFLn22mszZcqUjB49OvPnz8+KFStqMyyls2jRopx99tlpaWnJxIkT8973vjerVq3abRv7EAP56le/mjlz5vR/6vS8efNyxx139D9vv6EaixYtSqVSycKFC/sfsw+xL9dee20qlcpul0mTJvU/X6t9Z0RGy/e+970sXLgwn//85/Mf//Efedvb3paLL744zz33XK1Ho2Q2b96cM844I9dff/2Az//1X/91/u7v/i7XX399li1blkmTJuXCCy/Mxo0bh3lSymjp0qVZsGBBHnzwwdxzzz3p7u7ORRddlM2bN/dvYx9iIFOnTs11112Xhx9+OA8//HDe+c535tJLL+3/w8B+w2AtW7YsN9xwQ+bMmbPb4/Yh9ufUU0/N2rVr+y+PPfZY/3M123eKEehNb3pT8bGPfWy3x04++eTiM5/5TI0m4kiQpLj11lv77/f29haTJk0qrrvuuv7Htm7dWrS1tRVf+9rXajAhZbd+/foiSbF06dKiKOxDVOe4444r/uVf/sV+w6Bt3LixmDlzZnHPPfcU5513XvHJT36yKAr/9rB/11xzTXHGGWcM+Fwt950Rd6Rl27ZteeSRR3LRRRft9vhFF12U+++/v0ZTcSR65plnsm7dut32pebm5px33nn2JQbU0dGRJBk3blwS+xCD09PTk8WLF2fz5s2ZN2+e/YZBW7BgQS655JJccMEFuz1uH+JAVq9enSlTpmT69Ol53/vel6effjpJbfedhiH97iX08ssvp6enJ8cff/xujx9//PFZt25djabiSLRzfxloX3r22WdrMRIlVhRFrrrqqrz1rW/NaaedlsQ+xP499thjmTdvXrZu3ZqxY8fm1ltvzezZs/v/MLDfsD+LFy/Oz3/+8yxbtmyv5/zbw/68+c1vzre//e2cdNJJefHFF/PFL34x55xzTlasWFHTfWfERctOlUplt/tFUez1GAyGfYnB+PjHP55HH300P/nJT/Z6zj7EQGbNmpXly5dnw4YNueWWW3L55Zdn6dKl/c/bb9iXNWvW5JOf/GTuvvvujBo1ap/b2YcYyMUXX9x/+/TTT8+8efMyY8aMfOtb38pb3vKWJLXZd0bc6WGvec1rUl9fv9dRlfXr1+9VjbA/O99Jw77EgXziE5/I7bffnh//+MeZOnVq/+P2Ifanqakpr3/96zN37twsWrQoZ5xxRv7hH/7BfsMBPfLII1m/fn3OOuusNDQ0pKGhIUuXLs0//uM/pqGhoX8/sQ8xGMccc0xOP/30rF69uqb//oy4aGlqaspZZ52Ve+65Z7fH77nnnpxzzjk1mooj0fTp0zNp0qTd9qVt27Zl6dKl9iWS9P0/Tx//+Mfz/e9/Pz/60Y8yffr03Z63D1GNoijS1dVlv+GAzj///Dz22GNZvnx5/2Xu3Ln5wAc+kOXLl+fEE0+0DzFoXV1dWblyZSZPnlzTf39G5OlhV111VT70oQ9l7ty5mTdvXm644YY899xz+djHPlbr0SiZTZs25amnnuq//8wzz2T58uUZN25cpk2bloULF+bLX/5yZs6cmZkzZ+bLX/5yxowZk/e///01nJqyWLBgQW6++eb84Ac/SEtLS///M9XW1pbRo0f3f26CfYg9fe5zn8vFF1+c9vb2bNy4MYsXL86SJUty55132m84oJaWlv61czsdc8wxGT9+fP/j9iH25VOf+lTe8573ZNq0aVm/fn2++MUvprOzM5dffnlt//0Z0vcmK7F/+qd/Kk444YSiqampeOMb39j/FqSwqx//+MdFkr0ul19+eVEUfW/9d8011xSTJk0qmpubi7e//e3FY489VtuhKY2B9p0kxY033ti/jX2IgVx55ZX9/xs1YcKE4vzzzy/uvvvu/uftN1Rr17c8Lgr7EPv2B3/wB8XkyZOLxsbGYsqUKcXv/u7vFitWrOh/vlb7TqUoimJoswgAAODgjbg1LQAAwJFFtAAAAKUmWgAAgFITLQAAQKmJFgAAoNRECwAAUGqiBQAAKDXRAgAAlJpoAQAASk20AAAApSZaAACAUvv/AW78l266IlNWAAAAAElFTkSuQmCC",
      "text/plain": [
       "<Figure size 1000x600 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Transition probabilities\n",
    "alpha = 0.3\n",
    "beta = 0.2\n",
    "\n",
    "# Discount parameter\n",
    "gamma = 0.3\n",
    "\n",
    "# Expected rewards\n",
    "r_search = 6.0\n",
    "r_wait = 2.0\n",
    "\n",
    "V = np.zeros(nb_states)\n",
    "Q = np.zeros((nb_states, nb_actions))\n",
    "\n",
    "pi = np.array([a['search'], a['search']], dtype=int)\n",
    "\n",
    "V_high_history = []\n",
    "V_low_history = []\n",
    "\n",
    "for k in range(50):\n",
    "    \n",
    "    # Policy evaluation\n",
    "    Q[s['high'], a['search']] = alpha * (r_search + gamma * V[s['high']]) + (1 - alpha) * (r_search + gamma*V[s['low']])\n",
    "    Q[s['high'], a['wait']] = r_wait + gamma * V[s['high']]\n",
    "\n",
    "    Q[s['low'], a['search']] = beta * (r_search + gamma * V[s['low']]) + (1 - beta) * (-3 + gamma*V[s['high']])\n",
    "    Q[s['low'], a['wait']] = r_wait + gamma * V[s['low']]\n",
    "    Q[s['low'], a['recharge']] = gamma * V[s['high']]\n",
    "\n",
    "    V[s['high']] = Q[s['high'], :2].max()\n",
    "    V[s['low']] = Q[s['low'], :].max()\n",
    "\n",
    "    V_high_history.append(V[s['high']])\n",
    "    V_low_history.append(V[s['low']])\n",
    "        \n",
    "    # Compute the greedy policy\n",
    "    pi_old = pi.copy()\n",
    "    pi[s['high']] = Q[s['high'], :2].argmax()\n",
    "    pi[s['low']] = Q[s['low'], :].argmax()\n",
    "    \n",
    "print('Greedy policy after iteration', k)\n",
    "print('pi(high)=', pi[s['high']])\n",
    "print('pi(low)=', pi[s['low']])\n",
    "print(\"V=\", V)\n",
    "print(\"Q=\", Q)\n",
    "    \n",
    "plt.figure(figsize=(10, 6))  \n",
    "plt.plot(V_high_history, label=\"high\")\n",
    "plt.plot(V_low_history, label=\"low\")\n",
    "plt.legend()\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**A:** The agent now decides to wait in the low state (r=2) instead of recharging (r=0) and then be in the high state (r=6). The agent is so greedy that it cannot stand not getting reward for one step, although he will collect much more reard later. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Q:** Change $\\gamma$ to 0.99 (far-sighted agent). What does it change and why?  "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Greedy policy after iteration 49\n",
      "pi(high)= 0\n",
      "pi(low)= 2\n",
      "V= [141.37219244 137.82818773]\n",
      "Q= [[141.37219244 139.82818773   0.        ]\n",
      " [135.92647479 136.31962304 137.82818773]]\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAz8AAAH5CAYAAACve4DDAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjYuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/av/WaAAAACXBIWXMAAA9hAAAPYQGoP6dpAAB9WklEQVR4nOzdd3iUVfrG8e9MekIKIZAQSCAhoXeQ3qsoYMe29lWwgqAgFgQVEFRExb6W1bW7AqKIgPTee4cAoYRQQnomycz7++PV+MOyzCQzpN2f6/La65zMnDzDRuD2vOc5FsMwDERERERERCo4a2kXICIiIiIiciko/IiIiIiISKWg8CMiIiIiIpWCwo+IiIiIiFQKCj8iIiIiIlIpKPyIiIiIiEiloPAjIiIiIiKVgndpF1AcDoeDEydOEBwcjMViKe1yRERERESklBiGQWZmJtHR0Vit/3tvp1yGnxMnThATE1PaZYiIiIiISBmRnJxM7dq1/+drymX4CQ4OBswPGBISUsrViIiIiIhIacnIyCAmJqYoI/wv5TL8/PaoW0hIiMKPiIiIiIg4dRxGDQ9ERERERKRSUPgREREREZFKQeFHREREREQqhXJ55sdZdrudgoKC0i6jXPHx8cHLy6u0yxARERERcbsKGX4MwyAlJYXz58+XdinlUlhYGFFRUbpDSUREREQqlAoZfn4LPjVq1CAwMFB/iXeSYRjk5OSQmpoKQM2aNUu5IhERERER96lw4cdutxcFn2rVqpV2OeVOQEAAAKmpqdSoUUOPwImIiIhIhVHhGh78dsYnMDCwlCspv377tdN5KRERERGpSCpc+PmNHnUrPv3aiYiIiEhFVGHDj4iIiIiIyP+n8FOG9OjRgxEjRvzt1y0WC7NmzXJ6vSVLlmCxWNT1TkRERESECtjwoCI7efIkVatWLe0yRERERETKJYWfciQqKqq0SxARERERKbf02FsZ43A4GD16NOHh4URFRTF+/Piir/3xsbdVq1bRsmVL/P39adu2LbNmzcJisbBly5YL1ty4cSNt27YlMDCQTp06sXfv3kvzYUREREREypBKsfNjGAa5BfZS+d4BPl4udU/797//zciRI1m7di2rV6/mzjvvpHPnzvTt2/eC12VmZjJo0CCuuOIKPv/8c44cOfK354WeeuopXnnlFapXr86wYcO4++67WblyZUk+loiIiIhIuVMpwk9ugZ3G434ule+967n+BPo6/8vcvHlznn32WQASExOZMWMGv/zyy5/Cz2effYbFYuH999/H39+fxo0bc/z4ce69994/rTlx4kS6d+8OwBNPPMGVV15JXl4e/v7+JfhkIiIiIiLlix57K2OaN29+wbhmzZqkpqb+6XV79+6lefPmFwSYdu3aXXTNmjVrAvzlmiIiIiIiziq0O0q7BJdVip2fAB8vdj3Xv9S+tyt8fHwuGFssFhyOP/9gGYbxp8fpDMO46Jq/veev1hQRERERuZhDp7P4YEUSqw6e5ecR3fD1Lj/7KZUi/FgsFpcePSsPGjZsyGeffYbNZsPPzw+ADRs2lHJVIiIiIlIRGYbB+sNpvL/8EAt3n+K3/+a+ZG8q/ZqUn47ELse0ZcuWMWjQIKKjoy966ebQoUOxWCxMnz79gnmbzcbDDz9MREQEQUFBDB48mGPHjrlaSqV2yy234HA4uO+++9i9ezc///wzL7/8MoBLDRZERERERP5Ood3BD9tOcPVbqxjy7moW7DKDT++GNfjyvg70bRxZ2iW6xOXwk52dTYsWLZgxY8b/fN2sWbNYu3Yt0dHRf/raiBEjmDlzJl9++SUrVqwgKyuLgQMHYreXTke28igkJIQ5c+awZcsWWrZsyVNPPcW4ceMA1MhAREREREoky1bIhyuS6PHyEh76fDNbk8/j623l5nYxLBzZjQ/uvIwO8dXK3X90d/lZsAEDBjBgwID/+Zrjx4/z0EMP8fPPP3PllVde8LX09HQ++OADPv30U/r06QPAf/7zH2JiYli4cCH9+//5bI7NZsNmsxWNMzIyXC27XFiyZMmf5v7/ztofz/R06tSJrVu3Fo0/++wzfHx8iI2NBaBHjx5/ek/Lli3/9myQiIiIiFRuKel5fLzqMJ+vPUJGXiEA4UG+/KNDHW7vWIeIKn6lXGHJuP0gjMPh4LbbbuPxxx+nSZMmf/r6xo0bKSgooF+/fkVz0dHRNG3alFWrVv1l+Jk8eTITJkxwd6nl3ieffEJ8fDy1atVi69atjBkzhiFDhhAQEFDapYmIiIhIObL7ZAbvLz/EnK0nKLCb/6E8LiKIe7rEcV3r2gT4utbEq6xye/iZMmUK3t7ePPLII3/59ZSUFHx9falateoF85GRkaSkpPzle8aOHcvIkSOLxhkZGcTExLiv6HIqJSWFcePGkZKSQs2aNbnhhhuYOHFiaZclIiIiIuWAYRgs33+G95cfYvn+M0Xz7eqG88+ucfRpFInVWr4ea7sYt4afjRs38tprr7Fp0yaXn//7q9bNv/Hz8yvqaCa/Gz16NKNHjy7tMkRERESkHMkvdDBn6wneX36IPSmZAFgtMKBZTe7tGk/LmLDSLdCD3Bp+li9fTmpqatGZEwC73c6oUaOYPn06hw8fJioqivz8fNLS0i7Y/UlNTaVTp07uLEdERERERH6VkVfAF2uP8tHKw6Rk5AEQ6OvFjZfFcHfnOGLCA0u5Qs9za/i57bbbipoY/KZ///7cdttt3HXXXQC0adMGHx8fFixYwJAhQwA4efIkO3bsYOrUqe4sR0RERESk0juZnstHKw/z+dqjZNnMJgbVg/24s1Nd/tG+DqGBPqVc4aXjcvjJysriwIEDReOkpCS2bNlCeHg4sbGxVKtW7YLX+/j4EBUVRYMGDQAIDQ3lnnvuYdSoUVSrVo3w8HAee+wxmjVr9qfgJCIiIiIixbP7ZAbvLzvE91tPUOgwmxgk1KjCfV3juapVNH7eJWhiUJgPJ7dCzGVuqvbScDn8bNiwgZ49exaNf2tEcMcdd/Dxxx87tcarr76Kt7c3Q4YMITc3l969e/Pxxx/j5VUxukiIiIiIiJQGwzBYeeAs7y47eEETg/Zx4QztHk+P+jVK1sQg9zxs/BjWvgs5Z+HRHVClRonrvlRcDj9/dXfM/3L48OE/zfn7+/PGG2/wxhtvuPrtRURERETkDwrsDn7cdpL3lh1i10nzTszfmhjc1zWeFiVtYpB2BNa+A5s+gfwscy6oBpzZX7HDj4iIiIiIlA1ZtkK+XHeUD1ckcSLdbGIQ4GM2MbinixuaGBzfCKtmwK7ZYNjNueqNoNND0OwG8C5fHZkVfsqQHj160LJlS6ZPn17apYiIiIhIGZaamcfHKw/znzVHyMgzmxhEVPHjzk51uLV9HaoG+RZ/cYcD9v8Mq96AIyt/n4/vAR0fhoTe4OK1NmWFwo+IiIiISDlx8HQW/1p+iP9uPE6+3QFAfEQQ93aL55pWtfD3KcEZ+oJc2PoFrH4Lzu4356ze0PR66Pgg1Gzuhk9QuhR+RERERETKuE1H03h36UHm7zrFb8fvW8WGMax7Pfo2iixZE4PsM7DufVj/vtnEAMAvFNreCe2GQmitEtdfVij8lFFpaWkMHz6cOXPmYLPZ6N69O6+//jqJiYkYhkGNGjV45513uO666wBo2bIlJ06cIDU1FYDVq1fTrVs30tLSqFKlSml+FBEREREpBofDYNGeVN5bdoh1h88VzfdpVIOh3evRtk5VLCV5/OzMAVg9w9ztKTTPCxEaCx3uh9a3gV9wCT9B2VM5wo9hQEFO6Xxvn8BiPRN55513sn//fr7//ntCQkIYM2YMV1xxBbt27cLHx4du3bqxZMkSrrvuOtLS0ti1axdBQUHs2rWLxo0bs2TJEtq0aaPgIyIiIlLO2ArtzN5ygveWHeJAqtlZzcfLwjWtanFft3gSapQwlBxdAytfh71zgV+3kaJbm00MGl0FXhU3IlTcT/b/FeTApOjS+d5PngDfIJfe8lvoWblyJZ06dQLgs88+IyYmhlmzZnHDDTfQo0cP3nvvPQCWLVtGixYtiI2NZcmSJUXhp0ePHu7+NCIiIiLiIRl5BXyx9igfrkziVIYNgGA/b27pEMvdneOIDPEv/uIOuxl2Vr4Ox9b9Pl9/AHR6GOp0KrdNDFxROcJPObN79268vb1p37590Vy1atVo0KABu3fvBszOcMOHD+fMmTMsXbqUHj16EBsby9KlS7nvvvtYtWoVI0aMKKVPICIiIiLOSs3I44OVSXy+5iiZNrNzW2SIH3d3juPm9rGE+PsUf/GCXNjyOax+E84dNOe8fKHFTdDxIajewA2foPyoHOHHJ9DcgSmt7+2iv7tE1jCMouc6mzZtSrVq1Vi6dClLly7lueeeIyYmhokTJ7J+/Xpyc3Pp0qVLiUoXEREREc85dDqL95Yd4rtNv3duS6hRhaHd4rmqZS18va3FXzz7rNnAYN17vzcx8A+Dy/4J7e6D4MiSf4ByqHKEH4vF5UfPSlPjxo0pLCxk7dq1RY+9nT17ln379tGoUSMALBYL3bp1Y/bs2ezYsYOuXbsSHBxMQUEB77zzDq1btyY4uOIdUhMREREp77Ykn+edJQf5eVdKUee2NnWqMqx7PXo3rFGyzm1nD5q7PFs+h8Jccy4sFjo8CK3+AX6V+zx45Qg/5UxiYiJXXXUV9957L++++y7BwcE88cQT1KpVi6uuuqrodT169ODRRx+lVatWhISEANCtWzc+++wzRo4cWVrli4iIiMgfGIbB0n2neWfpQdYcurBz27Du9WhbN7xk3+DYBlj5GuyeQ1ETg5otofMjFb6JgSv0q1BGffTRRwwfPpyBAweSn59Pt27dmDt3Lj4+vz/z2bNnT+x2+wWNDbp3786sWbPo3r17KVQtIiIiIv9fod3Bj9tP8s7SQ+w+mQGAt9XCVS1rMbR7PPUjS/CkjsMB+382mxgcXfX7fGJ/s4lB3S6VoomBKyzG3x0wKcMyMjIIDQ0lPT29aMfjN3l5eSQlJREXF4e/fwk6YlRi+jUUERERKZncfDtfb0jm/eWHOJZmPn4W6OvFze1iuadLHNFhAcVfvNAG278xQ8+Zveac1Qea32i2q67RyA2foPz4X9ngj7TzIyIiIiLiJmnZ+Xyy+gj/Xn2Yc9n5AFQL8uXOTnW5rWMdwgJ9i794Xjps/BjWvA2ZJ805vxBoeze0HwYhNUv+ASo4hR8RERERkRI6mZ7Lv5Yn8cW6o+Tk2wGICQ/gvq7x3NA2Bn8fr+IvnnHCDDwbPoL8THMuuCZ0eADa3An+/3u3Q36n8CMiIiIiUkwHUrN4d+lBZm05ToHdPE3SuGYIw3rU44qmUXh7laBddeoeWPUGbPsKHAXmXPWG0Hk4NL0evEuwi1RJKfyIiIiIiLhoa/J53v5Du+r2ceE80DOBbokRRXczusww4Ohqs3Pbvnm/z9fpbIaehL5gLUGgquQUfkREREREnGAYBisOnOHtJQdZdfBs0XzfxpEM616PNnWqFn9xhwP2/miGnmPrf520QKOB0Gk4xFxWsuIFqMDhx+FwlHYJ5ZZ+7URERER+Z3cYzNuRwttLD7Dj+IXtqod1jyexJO2qC22w9UtY9TqcPWDOeflBy1ug40MQkeCGTyC/qXDhx9fXF6vVyokTJ6hevTq+vr7F33asZAzDID8/n9OnT2O1WvH11XOkIiIiUnnZCu3M3HScd5cdIulMNgD+PlZuuiyWf3aNo3bVwOIvnpcBGz40GxlkpZhz/qFw2T/Nzm1VarjhE8gfVbjwY7VaiYuL4+TJk5w4caK0yymXAgMDiY2NxarnSUVERKQSyrIV8vnaI/xreRKpmTYAQgN8uKNTXe7sVJfwoBL8B+LMlF87t30INnMXieBo6PggtLkD/EqwiyQXVeHCD5i7P7GxsRQWFmK320u7nHLFy8sLb29v7ZaJiIhIpZOWnc9Hqw7z71WHSc81u6tFhvhxb9d4bmoXSxW/EvzV+exB8zzP1i/Abt7/Q0QDs4lBsxvUue0SqZDhB8BiseDj44OPj09plyIiIiIiZVhKeh7vLz90wR09cRFBDOsez9WtauHnXYI7eo5vhBXTYfcc4Ne2cDHtofMIqH+5OrddYhU2/IiIiIiI/C9JZ7J5d+lB/rvp2AV39DzYM4HLm0bhZS1Bu+qDv5ih5/Dy3+frX26GnjodS1y7FI/Cj4iIiIhUKjtPpPP2koPM3X4Sx6+bMe3iwnmgRz26169e/Mf/7YWwaxasnA4p2805q7f5WFunRyCysTvKlxJQ+BERERGRSmH94XO8tfgAi/eeLprr1bAGD/SoR9u64cVfuCAPtn5unulJO2zO+QSZDQw6PABhMSUrXNxG4UdEREREKizDMFiy7zRvLT7A+sNpAFgtcGXzaO7vXo/G0SHFXzwv3ezatvotyE415wLCocP9ZsvqwBIEKvEIhR8RERERqXB+u5j0zcUH2HXSbCnt62Xluja1GNqtHnUjgoq/eOYpWPs2rP/g93bVIbWh08PQ+jbwLcHa4lEKPyIiIiJSYRTYHczafJy3lx7k0GnzYtJAXy9uaRfLP7vGExXqX/zFzyXBqjdg83/Abt7/Q/WGZhODZteDl7oMl3UKPyIiIiJS7uUV2Pl6QzLvLj3E8fO5gHkx6Z2/XkxatSQXk6ZsNzu37fwODIc5V6stdB0J9QeoXXU5ovAjIiIiIuVWlq2Q/6w5wr+WJ3Emy9yNiajix71d47i1Q53iX0xqGHB0Nax4FfbP/30+oQ90eRTqdAZdCl/uKPyIiIiISLmTlp3Px6sO8/Gqw6TnFgBQKyyAod3jGdI2Bn+fYl5Mahhm2Fn+CiSvNecsVmhyjfl4W83m7vkAUioUfkRERESk3EjNyONfK5L4z5oj5OTbAYiPCOL+HvW4ulUtfLyK+Qjab3f0rHgVTu0w57x8oeWtZiODavXc8wGkVCn8iIiIiEiZl3wuh3eXHeTrDcfILzTP3TSqGcJDPRO4vGkUXtZiPoJWaIMtv93Rk2TO+VaBtndDxwchOMpNn0DKAoUfERERESmzDp3O4q0lB5m1+TiFDgOA1rFhPNQrgZ4NamAp7rkbWxZs/AhWvwmZJ8253+7oaXcvBFR10yeQskThR0RERETKnD0pGcxYdIAft5/EMDMPXRIieLBnAh3iw4sfenLOwdp3Ye07kHfenAup9esdPbfrjp4KTuFHRERERMqMrcnnmbH4AAt2nSqa69Mokod6JdAyJqz4C2ecMHd5NnwEBeb9P4TXMzu3Nb8RvEvQClvKDYUfERERESl16w+f441FB1i27zRgdpG+ollNHuyRQOPokOIvfPageZ5n6xdgzzfnoppB11HQaDBYi9kVTsolhR8RERERKRWGYbDywFneWLSftUnnAPCyWriqZTQP9EggoUaV4i9+aicsn3bhxaR1OkOXkZDQW3f0VFIKPyIiIiJySRmGwS+7U5mx+ABbks8D4ONl4Ya2MQzrVo/YaoHFX/zYBvOOnr1zf59L7Gfu9MR2KFnhUu4p/IiIiIjIJWF3GMzbkcKMxQfYfTIDAH8fKze3i+W+bvHUDA0o3sKGAUnLzNCTtPTXSQs0udrc6dHFpPIrhR8RERER8ahCu4M5204wY9EBDp42mw0E+XpxW8e63NMljurBfsVb2DBg38+w/GU4tt6cs3pD85ugywiISHTPB5AKQ+FHRERERDyiwO5g5qbjvLXkAIfP5gAQ4u/N3V3iuLNTXcICi9lhzWGHXbPMMz2ndphzXn5mq+rOj0BYrHs+gFQ4Cj8iIiIi4la2QjvfbDjG20sOcvx8LgDhQb7c0yWO2zvWIdjfp3gLF+bDtq9gxatw7qA551sFLrsHOjwIwZFu+gRSUSn8iIiIiIhb5BXY+XLdUd5ZeoiUjDwAIqr4MbRbPLd2iCXQt5h/9SzIhU2fwMrXIeOYORdQFdrfD+3uhcBwN30CqegUfkRERESkRHLyC/lszVHeXXaIM1k2AKJC/BnWPZ6b2sXi71PMu3RsmbD+A1g9A7LN+3+oEgmdHoY2d4FfCVphS6Wk8CMiIiIixZKZV8Anq4/wwYokzmWbF4jWCgvg/h71uKFtbfy8ixl6ctNg7Xuw5i3IO2/OhcZCl+HQ8h/g4++eDyCVjsKPiIiIiLgkPaeAj1Yl8dHKw6TnFgBQp1ogD/ZI4JrWtfDxshZv4ewzsPpNWPc+5Geac9USzDt6mt0AXsU8KyTyK4UfEREREXFKWnY+H65M4uOVh8m0FQJQr3oQD/VKYFDzaLyLG3oyTsKqN2DDh1BoNkigRmPo9hg0vhqsxdxBEvkDl39Cly1bxqBBg4iOjsZisTBr1qyirxUUFDBmzBiaNWtGUFAQ0dHR3H777Zw4ceKCNWw2Gw8//DAREREEBQUxePBgjh07VuIPIyIiIiLudzbLxpR5e+gyZRFvLDpApq2QBpHBzLilFfMf7c41rWoXL/ikHYEfHoXXmsOaN83gE90Kbvochq2Eptcp+Ihbubzzk52dTYsWLbjrrru47rrrLvhaTk4OmzZt4plnnqFFixakpaUxYsQIBg8ezIYNG4peN2LECObMmcOXX35JtWrVGDVqFAMHDmTjxo14eekHXERERKQsOJNl4/1lh/h0zRFy8u0ANK4ZwiO9E+nXOBKr1VLMhQ/Aimlm22qHuYNEbEfo9jjU6wWWYq4rchEWwzCMYr/ZYmHmzJlcffXVf/ua9evX065dO44cOUJsbCzp6elUr16dTz/9lBtvvBGAEydOEBMTw9y5c+nfv/9Fv29GRgahoaGkp6cTEhJS3PJFRERE5C+kZubx3tJD/GftEfIKHAA0qxXKI70T6dOoBpbihpNTu2D5y7BzJhjmusT3NENP3c5uql4qG1eygcfP/KSnp2OxWAgLCwNg48aNFBQU0K9fv6LXREdH07RpU1atWvWX4cdms2Gz2YrGGRkZni5bREREpNI5lZHH20sO8sW6o9gKzXDSonYow/sk0rNBCULPya2wdCrs+eH3ufoDzDM9tdu6oXIR53g0/OTl5fHEE09wyy23FKWwlJQUfH19qVq16gWvjYyMJCUl5S/XmTx5MhMmTPBkqSIiIiKV1sn0XN5ecpAv1yeT/2voaRUbxvDeiXSvX734oefYBjP07P/51wkLNL7KDD1RzdxTvIgLPBZ+CgoKuOmmm3A4HLz11lsXfb1hGH/7L9bYsWMZOXJk0TgjI4OYmBi31SoiIiJSGR0/n8tbiw/wzYZj5NvN0NO2TlWG90mkS0JE8UPPkVVm6Dm02BxbrND0ejP0VG/gpupFXOeR8FNQUMCQIUNISkpi0aJFFzx7FxUVRX5+PmlpaRfs/qSmptKpU6e/XM/Pzw8/Pz9PlCoiIiJS6SSfy+GtJQf4duMxCuzm8e/2ceEM75NIx/hqxQs9hgFJy2DZS3B4uTln9YbmN0HXkVCtnhs/gUjxuD38/BZ89u/fz+LFi6lWrdoFX2/Tpg0+Pj4sWLCAIUOGAHDy5El27NjB1KlT3V2OiIiIiPzqt9DzzYZjFDrM0NOpXjUe6Z1Ih/hqF3n33zAMOPALLJsKyWvNOasPtPoHdHkUqtZxU/UiJedy+MnKyuLAgQNF46SkJLZs2UJ4eDjR0dFcf/31bNq0iR9++AG73V50jic8PBxfX19CQ0O55557GDVqFNWqVSM8PJzHHnuMZs2a0adPH/d9MhEREREBzNDz5mJzp+e30NMlIYLhfRK5rG548RY1DNj7kxl6Tmw257z8oM0d0Hk4hNZ2U/Ui7uNyq+slS5bQs2fPP83fcccdjB8/nri4uL983+LFi+nRowdgNkJ4/PHH+fzzz8nNzaV379689dZbTp/jUatrERERkYtLPpfDjEUH+O+m30NP18QIhvdOpG1xQ4/DAbu/h2Uvw6nt5pxPILS9Gzo9DMFRbqpexDmuZIMS3fNTWhR+RERERP7e0bPmTo97Q48dds2CpS/B6d3mnG8VaHcvdHwIgiLcU7yIi8rUPT8iIiIicmkcPZvDjMX7+e+m49j/X+gZ0SeRNnVKEHp2fGc2Mjiz15zzC4X2Q6HD/RBYzHVFSoHCj4iIiEg5d+RsNjMWHeC7zb+Hnm71qzO8dyJt6lS9yLv/hr0Qtn8Dy1+Gs7+e9/YPhQ4PmsEnIMw9xYtcQgo/IiIiIuXU0bM5vLFov5tDTwFs+8o805OWZM4FVDUfbWt3H/jryIGUXwo/IiIiIuVM8jkz9Pz/x9u616/O8D6JtI4tZugpzIetX8DyV+D8EXMusJrZxOCyf4JfsJuqFyk9Cj8iIiIi5cRftawueeixwZbPYPmrkH7UnAuqDp0eMTu4+VVxU/UipU/hR0RERKSMO34+lxmLDvDNhuQLureN6FO/+I+3FeTB5k9hxauQcdycqxJp3tHT5i7wDXRT9SJlh8KPiIiISBl14nwuby4+wNcbkimw/3456Yg+JWhZXZAHmz6BFdMg86Q5F1wTOo8wLyj1CXBP8SJlkMKPiIiISBlzMj2XtxYf5Kv1yeTbHQB0qleNR/vW5zJ3hp6QWtDlUWh1G/j4u6l6kbJL4UdERESkjEhJz+OtJQf4ct3voadDfDiP9qlP+/hqxVu0IA82/dt8vO3/h56uI83Q4+3npupFyj6FHxEREZFSlpqRx1tLDvL5uqPkF5qhp12cGXo61itu6MmFjb+GnqwUcy6k9q+h5x8KPVIpKfyIiIiIlJIzWTbeWXKQT9ccwfZr6LmsbtWi0GOxWFxftCAXNn4MK6ZfGHq6jYKWtyr0SKWm8CMiIiJyiZ3LzufdZQf5ZNURcgvsALSpY4aezgklDT2vQtYpc06hR+QCCj8iIiIil8j5nHz+tTyJj1YmkZ1vhp4WMWGM7FufbokRxQ89Gz6CldN/Dz2hMdD1t9Dj674PIFLOKfyIiIiIeFhGXgEfrkjig+VJZNoKAWhaK4SRfevTs0ENN4aeWHOnp8UtCj0if0HhR0RERMRDsmyFfLwyifeWHSIjzww9DaOCebRvffo1jixm6Mn7f4+3/XqmR6FHxCkKPyIiIiJulpNfyCerj/Du0oOk5RQAkFijCiP61GdA0yis1mKGnj/e0xMaA90eU+gRcZLCj4iIiIib5BXY+c+aI7yz9CBnsvIBiI8IYnifRAY2j8arOKGn0GaGnuXTIPOEOVfUyOAfCj0iLlD4ERERESkhW6GdL9cl8+biA6Rm2gCoUy2Q4b0TGdwiGm8vq+uLFubD5k/N0JNxzJzT5aQiJaLwIyIiIlJMBXYH/914jNd/2c+J9DwAaoUFMLx3Ite0roVPcUPPls9g+SuQnmzOBdc0u7e1vl2hR6QEFH5EREREXGR3GMzecpzpC/dz9FwOAFEh/jzUK4EhbWPw9S5G6LEXwJbPYdnLkH7UnKsSZe70tL4DfPzd+AlEKieFHxEREREnORwGP24/yfSF+zh4OhuAiCq+PNAjgVvax+Lv4+X6ovZC2PYlLJ0K54+Yc1Uiocuj0OZO8Alw3wcQqeQUfkREREQuwjAMFuw6xbQF+9iTkglAWKAPw7rX4/aOdQj0LcZfqRx22P4NLJ0C5w6Zc0E1zNDT9i6FHhEPUPgRERER+RuGYbB032mmLdjHtmPpAAT7efPPrvHc3aUuwf4+ri/qcMDO72DJi3B2vzkXGAFdRkDbe8A30H0fQEQuoPAjIiIi8hdWHTzDK/P3sfFIGgCBvl7c1bku93aNJyywGO2lHQ7Y/b0Zek7vNucCqkLn4XDZveBXxY3Vi8hfUfgRERER+X82Hknjlfl7WXXwLAB+3lZu61CHYT3qEVGlGJ3WDAP2zoXFk+HUdnPOPxQ6Pgzth4J/iBurF5H/ReFHREREBNh5Ip1X5u9j0Z5UAHy8LNzcLpYHeyYQGVKMTmuGAfsXwOKJcHKLOecbDB0fgA4PQECY22oXEeco/IiIiEildiA1k1cX7OfH7ScB8LJauL51bR7unUDtqsU4f2MYcHARLJ4ExzeYcz5B0GEYdHwIAsPdWL2IuELhR0RERCqlo2dzmP7LPmZtPo7DAIsFBjWPZkSfROKrF/P8TdIyM/QcXW2OvQOg3b3muZ6gCPcVLyLFovAjIiIilUpKeh5vLNrPV+uTKXQYAPRrHMnIfvVpGFXM8zdH18CiF+DwcnPs5QeX3QOdR0BwpHsKF5ESU/gRERGRSuFslo23lxzk0zVHsBU6AOiaGMFj/RrQIiaseIse32Se6Tmw0BxbfcyLSbuOgpCabqlbRNxH4UdEREQqtPTcAv61/BAfrkgiO98OwGV1q/JYvwa0j69WvEVTdsCSybDnB3Ns8YJWt0K3xyEs1k2Vi4i7KfyIiIhIhZRtK+TjVYd5d+lBMvIKAWhWK5RR/erTvX51LBaL64ue3meGnp3fmWOLFZoNge6joVo9N1YvIp6g8CMiIiIViq3Qzudrj/Lm4gOcycoHoH5kFUb2bUD/JpHFCz3nDsHSqbDtKzDMR+Zocg30GAvVG7ixehHxJIUfERERqRAK7Q6+23yc1xbu5/j5XABiwwMZ2bc+g1pE42UtRug5nwzLXoItn4HD3D2iwZXQcyxENXNj9SJyKSj8iIiISLnmcBj8tCOFVxbs5dDpbAAiQ/x4pHciQ9rG4ONldX3RzBRY/gps/Bjs5u4RCX2g55NQq437iheRS0rhR0RERMolwzBYuu80L8/fy47jGQBUDfThgR4J3NaxDv4+Xq4vmn0WVr4K696Hwjxzrm5X6PU0xHZwY/UiUhoUfkRERKTcWX/4HC/N28u6w+cACPL14p9d4/ln1ziC/X1cXzAvHVa/CavfgvxMc652OzP0xHd3Y+UiUpoUfkRERKTc2HE8nZfn72XJ3tMA+HpbuaNjHe7vkUB4kK/rC+Znw9p3YeVrkHfenItqDr2egcS+UJzmCCJSZin8iIiISJl38HQW0xbs48dtJwHwsloY0jaGR3onUDM0wPUFC22w4SPzXE92qjkX0cA809NoMFiLcU5IRMo8hR8REREps06cz+W1hfv5dtMx7A4DiwUGt4jm0T71qRsR5PqC9gLY8rnZtjrjmDlXta7ZsrrZDWAtxjkhESk3FH5ERESkzDmXnc9biw/wyZoj5Bea9+r0aVSDUf0a0KhmiOsLOhyw47+wZJJ5Zw9AcDR0fxxa3QZexTgnJCLljsKPiIiIlBlZtkI+WJ7E+8sPkWUz79VpFxfOmMsb0KZOuOsLGgbs+QEWT4LUXeZcYAR0HQlt7wafYjwyJyLllsKPiIiIlDpboZ3P1hzlzcUHOJtt3qvTJDqEx/s3oHv96lhcbTxgGHBwESx6Hk5sNuf8QqHzw9D+fvCr4uZPICLlgcKPiIiIlBq7w+C7TceYvnA/x8/nAhAXEcTIvvW5sllNrNZidFs7uhZ+eQ6OrDDHPkHQ4X7o9BAEVHVj9SJS3ij8iIiIyCVnGAY/7zzFK/P3sj81C4DIED+G967PDW1r4+NVjG5rKdth0Quwb5459vKDy+6BLiOhSnU3Vi8i5ZXCj4iIiFxSqw6eYcq8vWxNPg9AaIAPD/Soxx2d6uLvU4xua2cPwuKJZkMDAIsXtLoVuo+B0NruK1xEyj2FHxEREbkkth9LZ+rPe1i+/wwAAT5e3NMljnu7xRMaUIxua+nHzJbVm/8Dht2ca3It9HwKIhLcWLmIVBQKPyIiIuJRh05n8cr8ffy43byg1MfLwi3tYnmwVwI1gv1dXzD7DCyfBuv/BXabOZfYH3o9DTWbu7FyEaloFH5ERETEI1Iz8pj+y36+Wp9cdEHp1S1r8Wif+sRWC3R9wbx0WP2m+U++eU6IOp2h9ziI7eDe4kWkQnL5NOGyZcsYNGgQ0dHRWCwWZs2adcHXDcNg/PjxREdHExAQQI8ePdi5c+cFr7HZbDz88MNEREQQFBTE4MGDOXbsWIk+iIiIiJQN6bkFTJ23h24vLebztUexOwx6NazB3Ee68uqNLV0PPgW5sPI1eK0FLJ1iBp+aLeAf/4U7f1TwERGnuRx+srOzadGiBTNmzPjLr0+dOpVp06YxY8YM1q9fT1RUFH379iUzM7PoNSNGjGDmzJl8+eWXrFixgqysLAYOHIjdbi/+JxEREZFSlVdg571lB+k2dTFvLTlIXoGD1rFhfD20Ix/eeRmNaoa4tqC9ADZ8CK+3ggXjIDcNIurDDf+G+5ZCQh9w9f4fEanULIZhGMV+s8XCzJkzufrqqwFz1yc6OpoRI0YwZswYwNzliYyMZMqUKQwdOpT09HSqV6/Op59+yo033gjAiRMniImJYe7cufTv3/+i3zcjI4PQ0FDS09MJCXHxN1IRERFxq0K7g+82HefVhfs4mZ4HQGKNKjzevwF9G0e6fkGpwwE7vzM7uJ07ZM6FxkCPsdD8RvDSU/si8jtXsoFbf/dISkoiJSWFfv36Fc35+fnRvXt3Vq1axdChQ9m4cSMFBQUXvCY6OpqmTZuyatWqvww/NpsNm81WNM7IyHBn2SIiIlIMhmEwf9cpXvp5Lwd+vasnOtSfEX3rc13r2ni5ekGpYcCBhfDLBPPOHoDACOj2OLS9C7z93PwJRKSycWv4SUlJASAyMvKC+cjISI4cOVL0Gl9fX6pWrfqn1/z2/j+aPHkyEyZMcGepIiIiUgJrD51lyrw9bDp6HoCwQB8e7JHAbR3rFO+unqNrzdBzZKU59g2Gzo9Ah/vBL9h9hYtIpeaRfeM/bm8bhnHRLe//9ZqxY8cycuTIonFGRgYxMTElL1RERERcsvtkBlPn7WHx3tMA+PtYuadLHPd1q1e8u3pO7YRfnod9P5ljLz9ody90GQlB1dxYuYiIm8NPVFQUYO7u1KxZs2g+NTW1aDcoKiqK/Px80tLSLtj9SU1NpVOnTn+5rp+fH35+2uoWEREpLcfScpg2fx8ztxzHMMDLauGmy2IY3juRGiHFuKvnXBIsmQzbvgYMsFih1T+g+xgIre32+kVEwM3hJy4ujqioKBYsWECrVq0AyM/PZ+nSpUyZMgWANm3a4OPjw4IFCxgyZAgAJ0+eZMeOHUydOtWd5YiIiEgJpWXn8+biA3yy+gj5dgcAVzaryWP9GxAXEeT6glmpsOwl2PAROArMucZXQc+noXp9N1YuIvJnLoefrKwsDhw4UDROSkpiy5YthIeHExsby4gRI5g0aRKJiYkkJiYyadIkAgMDueWWWwAIDQ3lnnvuYdSoUVSrVo3w8HAee+wxmjVrRp8+fdz3yURERKTY8grsfLgyibeXHCQzrxCAjvHVeGJAQ1rEhBVjwXRY+TqseQsKcsy5+J7mBaW1WruvcBGR/8Hl8LNhwwZ69uxZNP7tLM4dd9zBxx9/zOjRo8nNzeWBBx4gLS2N9u3bM3/+fIKDfz+s+Oqrr+Lt7c2QIUPIzc2ld+/efPzxx3h5FeOApIiIiLhNod3Bfzcd49UF+0nJMNtWN4wK5okBDelev7rrbasL8mD9+7D8FfOeHoBabaD3sxDf3c3Vi4j8byW656e06J4fERER9zIMg4W7U5k6bw/7f21bXSssgFH96nN1y1pYXW1b7bDD1i9g8WTIOGbORdQ3d3oaDtTlpCLiNqV2z4+IiIiUPxuPnOPFn/aw/rC5MxMW6MNDPRP4R4ditK02DNg7F355Dk7vMedCapkXlLa4WReUikip0u9AIiIildSB1CymztvD/F2nALNt9d2d4xjWox4h/sVoW31kFSwcD8lrzbF/GHQdZbau9glwW90iIsWl8CMiIlLJnMrIY/rCfXy1PhmHAVYL3HhZDMN71ycqtBhtq0/thIUTYP/P5tg7wLyctPNwCAhza+0iIiWh8CMiIlJJZOYV8N6yQ/xreRK5BXYA+jWOZPTlDUioEXyRd/+FtCOweBJs+wrzrh4vaH27eVdPSM2Lvl1E5FJT+BEREang8gsdfLHuKK//sp+z2fkAtKlTlbEDGtK2brjrC2afgWUvw/p//b+7eq6GXs9ARIL7ChcRcTOFHxERkQrKMAx+2pHC1Hl7OHzWvFsnPiKIMQMa0q9xpOttq21Z5j09K1+H/ExzLq479Bmvu3pEpFxQ+BEREamA1iWdY9Lc3WxJPg9ARBU/RvRJ5MbLYvDxsrq2mL0ANv0blkyB7FRzrmYLM/TU6+XWukVEPEnhR0REpAI5kJrJiz/tZeFus4NboK8X93WL596u8QT5ufjHvmHArtlm2+pzB825qnXNx9uaXAtWF0OUiEgpU/gRERGpAP7Ywc3LauGmy2IY3ieRGsHF6OB2eAUsGAfHN5rjwAizkUGbO8Hb1621i4hcKgo/IiIi5dhfdXDr3ySS0Zc3pF71Kq4veGqneVfP/vnm2CcIOj0MnR4Cv2J0hBMRKUMUfkRERMqhAruDL9cdZfrC3zu4tY4N48krGhWvg9v5ZLNt9dYvAAOs3uYuT/cxUKWGW2sXESktCj8iIiLliGEYzN91iik/7eHQmWwA4iKCGHN5A/o3iXK9g1vOOVgxDda+B3abOdfkGvNcT7V6bq5eRKR0KfyIiIiUE5uPpjF57h7WHT4HQLUgX4b3SeTmdrGud3AryIW175rBJy/dnKvbFfpOgFpt3Fy5iEjZoPAjIiJSxh09m8PUn/fww7aTAPh5W/ln1ziGda9HsL+Pa4s57OajbYsnQcZxcy6yKfSZAAm9wdWdIxGRckThR0REpIw6n5PPG4sO8MnqwxTYDSwWuK51bUb1q0/N0ADXFjMMOLDQ7OCWusucC42BXk9DsyFqWy0ilYLCj4iISBmTV2Dnk9WHmbHoABl5hQB0TYxg7IBGNI4OcX3BE5vN0JO0zBz7h0G3x+Cye8GnGG2wRUTKKYUfERGRMsLhMJiz7QQv/byXY2m5ADSMCmbsFY3oXr+66wumHYZfnocd35pjLz9oPxS6joSAqu4rXESknFD4ERERKQPWHDrLpLm72XbMbD4QGeLHqH4NuK51bbysxejgtuwlWPc+OAoACzS/EXo9BWGx7i9eRKScUPgREREpRQdPZzF57h4W7j4FQJCvF/f3qMc9XeIJ8PVybbGCXFjzNqyYDrZfO7jV62U2M6jZ3L2Fi4iUQwo/IiIipeBslo3XftnPZ2uPYncYeFkt3NIuluF9Eomo4ufaYg47bP0SFk/8fx3cmpltqxN6u794EZFySuFHRETkEsorsPPRysO8tfgAmTazmUGfRpE8MaAhCTWquLaYYcCBX37t4LbTnAuNMS8obXaDOriJiPyBwo+IiMgl8Fszg6nz9nL8vNnMoGmtEJ68ohGd6kW4vuDJrTD/GUhaao79Q6HrY9DuPnVwExH5Gwo/IiIiHrYu6RwTf9zF1l+bGdQM9efx/g24umUtrK42MzifDItegG1fAQZ4+ZqBp+soCAx3f/EiIhWIwo+IiIiHHDqdxZR5e/h55+/NDB7omcDdneNcb2aQlw4rXoXVb4HdZs41u8F8xK1qHTdXLiJSMSn8iIiIuNm57Hxe/2U//1lzhEKHgdUCN7WL5dE+9ake7GIzg8J82PgRLJ0COWfNuTpdoN/zUKu1+4sXEanAFH5ERETcxFZo59+rDvPGogNk5pnNDHo2qM6TVzQiMTLYtcUMA3Z/DwvHw7lD5lxEfej7HNS/HCwuPi4nIiIKPyIiIiVlGAZzt6fw4rzdJJ8zmxk0qhnCU1c0oktiMZoZJK+D+U9D8lpzHFQDeo6FVreDl/7oFhEpLv0OKiIiUgKbj6bxwo+72XgkDYDIED9G9WvAda1r4+VqM4OzB+GXCbBrtjn2CYSOD0HnR8DPxZ0jERH5E4UfERGRYjh+Ppep8/Ywe8sJAAJ8vBjaPZ77usUT6OviH68552DpVFj/L3AUgMUKLW+Fnk9BSE0PVC8iUjkp/IiIiLggy1bI20sO8K/lSdgKHVgscF3r2jzWrwFRoS7er1OQB+veg2Uvg81sg01CX+g7ASKbuL94EZFKTuFHRETECXaHwdcbknll/l7OZOUD0CE+nKevbEzTWqGuLWYYsOO/5iNu54+ac1HNoO/zUK+nmysXEZHfKPyIiIhcxPL9p5n44272pGQCEBcRxNgBDenbOBKLq13Xjq6Bn5+C4xvMcXA09H4Gmt8EVqubKxcRkf9P4UdERORv7D+VyaS5u1m89zQAoQE+jOiTyK3t6+Dr7WJQOXcIFjxrtq8G8AmCLo9CxwfBN9DNlYuIyF9R+BEREfmDs1k2pi/cz+frjmJ3GHhbLdzRqS4P90ogLNDXtcVyzsGyl2Dd+783M2h1m9nMIDjSMx9ARET+ksKPiIjIr2yFdj5eeZgZiw6QaTMvKe3fJJInBjQiLiLItcUKbWbgWTYV8n5rZtDHPNcT2djNlYuIiDMUfkREpNIzDIOfd55i0tzdHD2XA0CT6BCevrIxHetVc3Ux2DULFo6HtMPmXI0m0O95SOjtzrJFRMRFCj8iIlKp7TiezvM/7GJt0jkAagT78Xh/85JSq6uXlCavh5+fhGPrzHGVSOj1tHlnj9XLzZWLiIirFH5ERKRSSs3I4+X5e/lm4zEMA/y8rQztFs/Q7vUI8nPxj8e0I+ZOz87vzLFPIHR6BDo9DH5V3F67iIgUj8KPiIhUKnkFdj5YkcSbiw+Qk28H4KqW0Yy+vCG1wgJcXCwDlr8Ca94Guw2wmLs8vZ6GkJruL15EREpE4UdERCoFwzCYs+0kU37aw/HzuQC0jAnjmYGNaVOnqmuL2Qth8yewaCLknDHn4rpBv4lQs7mbKxcREXdR+BERkQpvS/J5nv9hFxuPpAEQHerPmAENGdQ82vVzPQcWws9Pw+nd5rhaAvR7AepfDq5eeCoiIpeUwo+IiFRYJ9NzmTpvLzM3HwcgwMeL+3vU496u8QT4utiAIHU3zH/aDD8AAVWhx1hoezd4+bi5chER8QSFHxERqXBy8+28s/Qg7y47SF6BA4DrWtfm8f4NiAr1d22xrNOwZBJs/BgMB1h9oP1Q6PaYGYBERKTcUPgREZEKwzAMZm85wZR5eziZngfAZXWr8szAxjSvHebaYgV5sPZtWPYK5Geac40GQZ8JUK2eewsXEZFLQuFHREQqhC3J55kwZyebj54HoFZYAE9e0YgrmkVhceUsjmGYLasXjIf0o+ZczZbQfxLU7ezuskVE5BJS+BERkXItJT2PqfP28N2v53oCfb14oEc9/tk1Hn8fF8/1HNsIP4+F5LXmOKQW9H4Wmt0AVqubKxcRkUtN4UdERMqlvAI77y87xFtLDpJbYN7Xc13r2oy+vAGRIS6e60k/BgsnwPavzbFPIHR5FDo+BL6Bbq5cRERKi8KPiIiUK4Zh8OP2k0ye+/t9PW3qVGXcwMa0iAlzbbH8bFj5Gqx8HQrNtcxLSp/RJaUiIhWQwo+IiJQbO46n89ycXaw7fA6AmqH+PDGgIYNbRLt2rsfhgG1fwS8TIPOkORfbCS6fBNGtPFC5iIiUBW5/gLmwsJCnn36auLg4AgICiI+P57nnnsPhcBS9xjAMxo8fT3R0NAEBAfTo0YOdO3e6uxQREakgUjPzePybrQyasYJ1h8/h72NlRJ9EFo3qwVUta7kWfI6ugX/1glnDzOATVgeGfAJ3zVXwERGp4Ny+8zNlyhTeeecd/v3vf9OkSRM2bNjAXXfdRWhoKMOHDwdg6tSpTJs2jY8//pj69evzwgsv0LdvX/bu3UtwcLC7SxIRkXLKVmjnwxWHmbFoP9n55rmeq1pGM+byhkSHBbi2WNoRWPgs7Jxpjn2Dzbt62g8DHxfPCImISLlkMQzDcOeCAwcOJDIykg8++KBo7rrrriMwMJBPP/0UwzCIjo5mxIgRjBkzBgCbzUZkZCRTpkxh6NChF/0eGRkZhIaGkp6eTkhIiDvLFxGRMsAwDBbsOsULP+7m6LkcAFrUDmXcoCa0qePixaK2TFg+DVa/CXYbWKzQ+nbo+RRUqeGB6kVE5FJyJRu4feenS5cuvPPOO+zbt4/69euzdetWVqxYwfTp0wFISkoiJSWFfv36Fb3Hz8+P7t27s2rVqr8MPzabDZvNVjTOyMhwd9kiIlJG7DuVyfM/7GL5/jMA1Aj2Y8zlDbmmVS2sVlfO9dhh839g0QuQnWrOxXUz7+uJauaBykVEpKxze/gZM2YM6enpNGzYEC8vL+x2OxMnTuTmm28GICUlBYDIyMgL3hcZGcmRI0f+cs3JkyczYcIEd5cqIiJlSHpOAa8u3Mena45gdxj4eln5Z9c4HuyZQJCfi39cHV4B856AlO3mOLwe9HsBGgwAV84HiYhIheL28PPVV1/xn//8h88//5wmTZqwZcsWRowYQXR0NHfccUfR6/54ONUwjL89sDp27FhGjhxZNM7IyCAmJsbdpYuISCkotDv4Yn0y0+bvJS2nAIB+jSN5+srGxFZz8Y6dc0mw4BnYPccc+4VCjzFw2b3g7evmykVEpLxxe/h5/PHHeeKJJ7jpppsAaNasGUeOHGHy5MnccccdREVFAeYOUM2av9+hkJqa+qfdoN/4+fnh5+fn7lJFRKSUrT54lglzdrInJROA+pFVGDewCV0SI1xbyJYJy1/59VxPvnmup81d5rmeoGoeqFxERMojt4efnJwcrNYLO2h7eXkVtbqOi4sjKiqKBQsW0KqV2VI0Pz+fpUuXMmXKFHeXIyIiZVDyuRwm/7SbudvNR6FDA3wY2bc+t7aPxdvLhVsYHA7Y+jn88hxknTLn4rrD5ZMhsokHKhcRkfLM7eFn0KBBTJw4kdjYWJo0acLmzZuZNm0ad999N2A+7jZixAgmTZpEYmIiiYmJTJo0icDAQG655RZ3lyMiImVITn4hby85yLvLDpFf6MBqgVvb12Fk3/pUDXLxsbQjq81zPSe3mOOqcdB/IjS4Qud6RETkL7k9/Lzxxhs888wzPPDAA6SmphIdHc3QoUMZN25c0WtGjx5Nbm4uDzzwAGlpabRv35758+frjh8RkQrKMAy+33qCyXP3kJKRB0DH+Go8O7gxDaNcvLLgfDIsGAc7vzPHfiHQ7XFoPxS89Yi0iIj8Pbff83Mp6J4fEZHyY8fxdMZ/v5MNR9IAqF01gKevbET/JlF/2+jmL+Vnw4rpsOp1KMwDLOZ9Pb2e1n09IiKVWKne8yMiIgJwNsvGy/P38eX6oxgGBPh48WDPevyzazz+Pl7OL+RwwPZvYOF4yDxhztXpYp7rqdncI7WLiEjFpPAjIiJuVWh38J81R5i2YB8ZeYUADG4RzdgrGlIzNMC1xY5tgJ/GwPEN5jgs1ryvp9FgnesRERGXKfyIiIjbrDpwhvFzdrLvVBYAjWqGMGFwE9rFhbu2UMZJ+GUCbP3CHPtWga4jocOD4OPv5qpFRKSyUPgREZESSz6Xw6S5u/lph9m6umqgD6P6NeDmdrF4WV3YoSnIgzVvwbKXoSDbnGt5K/QeB8FRHqhcREQqE4UfEREpttx8O+8sPcg7Sw9i+7V19W0d6vBo3/qEBbrQutowYO9c+PlJSDtsztVqCwOmQu02HqldREQqH4UfERFxmWEY/LQjhYk/7ub4+VwAOsSH8+ygJjSq6WIXztTdMG8sHFpsjqtEQd8J0GwIWF248FREROQiFH5ERMQle1MyGf/9TlYfOgtAdKg/T13ZmCuaudi6OjcNFk+G9f8Cww5evtDpYegyEvyqeKh6ERGpzBR+RETEKek5Bby6cB+frjmC3WHg621lWPd63N+9HgG+LrSuthfCpo9h0UTIPWfONRxodnELj/NI7SIiIqDwIyIiF+FwGHy9IZmpP+/lXHY+AJc3ieKpKxsREx7o2mJJy2HeE3Bqhzmu3si8r6deTzdXLSIi8mcKPyIi8re2JJ/n2dk72HosHYCEGlUYP6gJXRIjXFso7QgseAZ2zTbH/mHQ8yloezd46Y8iERG5NPQnjoiI/MmZLBtT5+3h6w3HAKji582IPonc0akuPl4uNCHIz4GV02Hla1CYBxarGXh6PgWBLt79IyIiUkIKPyIiUqTQ7uDTNUeYtmAfmXmFAFzbuhZPDGhIjWAXLhc1DNg1C35+GjLMAEXdrnD5ixDV1P2Fi4iIOEHhR0REAFh98Czjv9/J3lOZADStFcKEwU1oU8fFHZpTO+GnMXB4uTkOjTGbGTS+ClzpBiciIuJmCj8iIpXcyfRcJv64mx+2nQQgLNCHx/s34KbLYvGyuhBWcs7Bkt9aVzvA2x86j4DOw8HXxcYIIiIiHqDwIyJSSdkK7XywIok3fjlAboEdqwVuaR/LqL4NqBrk6/xCDjts+jf88vzvrasbDTZ3e6rW8UzxIiIixaDwIyJSCS3ek8pzP+wi6Uw2AG3rVGX84CY0rRXq2kJHVsNPj0PKdnNcvREMmALx3d1csYiISMkp/IiIVCLJ53KYMGcXC3efAqB6sB9PXtGQq1vWwuLKeZyME7BgHGz/xhz7h/7auvoeta4WEZEyS39CiYhUAnkFdt5ZepC3lxzEVujA22rhrs51eaR3IsH+Ps4vVJAHa96EZa9AQTZggTZ3QK9nIMjFu39EREQuMYUfEZEKbuGuU0z4YSfJ53IB6JxQjQmDm5BQI9i1hfbOg3lPQFqSOY5pDwOmQnRL9xYsIiLiIQo/IiIV1JGz2UyYs4tFe1IBiArx55mBjbmiWZRrj7idPWiGnv3zzXGVKOj3PDS7Qa2rRUSkXFH4ERGpYHLz7by95ADvLDtEfqEDHy8L/+waz0M9Ewjyc+G3/fxsWPYyrJ4B9nyw+kDHB6HbY+Dn4q6RiIhIGaDwIyJSQRiGwfxdp3huzi6OnzcfceuaGMH4wU2oV72KKwvBzu9g/jOQcdycS+gDl78IEYkeqFxEROTSUPgREakAks5kM/77nSzddxqAWmEBPDOwEf2buPiI26md8NMYOLzcHIfVMUNPgwF6xE1ERMo9hR8RkXIsJ7+QNxcf4P1lSeTbHfh6WbmvWzwP9kwgwNfL+YVyz8OSybDufTDs4O0PXUZC50fAJ8Bj9YuIiFxKCj8iIuWQYRjM25HC8z/s4kR6HgA9GlTn2UFNiIsIcn4hhwO2fAYLx0POGXOu0WDoPxHCYt1fuIiISClS+BERKWeSzmQzbvYOlu83w0rtqgGMG9iYvo0jXXvE7fhGmPu4+b8AEfXN1tX1enqgahERkdKn8CMiUk7k5tt5c/EB3lt2yHzEzdvKsG7xPNAzAX8fFx5xyz4Dv0yATZ8CBvgGQ48x0G4oePt6rH4REZHSpvAjIlLGGYbBgl2nmPD/urh1r1+dCYObUNelR9zssOFDWPQ85KWbc81vgr4TIDjKA5WLiIiULQo/IiJl2NGzOYyfs7PoolKzi1tj+jdx8RG3o2th7ihI2W6Oo5rBFS9DbAcPVC0iIlI2KfyIiJRBeQV23ll6kLeWHCy6qPTervE81CuBQF8XfuvOOg0LnzWbGgD4h0KvZ6Dt3WB14VE5ERGRCkDhR0SkjFm8J5Vnv9/J0XM5AHRJiGDCVS5eVGovhA0fwKKJYPv1EbdWt0Gf8RAU4f6iRUREygGFHxGRMiL5XA7P/bCLBbtOARAV4s8zAxtzRTMXLyo9ssrs4nZqhzmu2RKufAVqt3V/0SIiIuWIwo+ISCmzFdp5f9khZiw+QF6BA2+rhXu6xPFI70SC/Fz4bTrzFCwYB9u+NMf+YdDnWWh9hx5xExERQeFHRKRULd9/mmdn7+TQmWwAOsSH89xVTakfGez8IvYCWPc+LJkMtgzAAm3ugF7jIKiaZwoXEREphxR+RERKwamMPJ7/YRc/bDsJQPVgP56+shGDW0S79ojb4RXmI26pu8xxdGu48mWo1cYDVYuIiJRvCj8iIpdQod3Bv1cf4dUF+8iyFWK1wB2d6vJo3/qE+Ps4v1BmCsx/GrZ/Y44Dws1mBq1uA6vVI7WLiIiUdwo/IiKXyMYj53hq5g72pGQC0Co2jBeubkqT6FDnF7EXwrr3YPEkyM8ELGbb6l5PQ2C4ZwoXERGpIBR+REQ87Fx2PlN+2sNXG5IBCAv04YnLGzKkbQxWqytd3FbDj6Mgdac5rtXWfMQtupUHqhYREal4FH5ERDzE4TD4ekMyL87bw/mcAgBubBvDmAENCQ/ydX6hrFSzi9vWL8yxHnETEREpFoUfEREP2Hkinadn7WDz0fMANIwKZuI1TWlTx4VH0xx2WP8BLHrh14tKf+3i1vtZPeImIiJSDAo/IiJulJlXwLQF+/j3qsM4DAjy9WJkvwbc0bEO3l4u7NIkr4cfR0LKNnNcsyVcOQ1qq4ubiIhIcSn8iIi4gWEY/LDtJM//sIvUTBsAA5vX5OkrGxMV6u/8QtlnYOF42PypOfYPhd7joM1duqhURESkhBR+RERKKOlMNuNm72D5/jMAxEUE8dxVTeiaWN35RRx22PRvWDgB8s6bcy3/AX0nQFCE+4sWERGphBR+RESKKa/AzjtLD/LWkoPkFzrw87byYM8EhnaPx8/bhV2a45vMR9xObDbHkc3gylcgtr1nChcREamkFH5ERIph+f7TPDNrB4fP5gDQvX51nruqCXWqBTm/SG4a/PI8bPgQMMAvxLyvp+094KXfnkVERNxNf7qKiLggNTOPF37YzfdbTwAQGeLHuIFNuKJZFBaLk3f2GAZs/RLmPw055qNyNL8R+j4PwZEeqlxEREQUfkREnGB3GHy29ggvzdtLpq0QqwXu6FSXkX3rE+zv4/xCp3aZF5UeXWWOqzc0H3Gr28UzhYuIiEgRhR8RkYvYfiydp2ZtZ9uxdABa1A5l4jXNaFor1PlFbFmwdAqseQscheATCN3HQIcHwNuFC09FRESk2BR+RET+RkZeAdPm7+OT1eadPcH+3ozu34Bb2tfBy+rCI26758C8JyDjuDnXcCBc/iKExXiueBEREfkThR8RkT8wDIMft5/kuTm/39lzVctonrqyETWCXbiz59whmDsaDiwwx2F14IqXoH5/D1QtIiIiF+PCdePOO378OP/4xz+oVq0agYGBtGzZko0bNxZ93TAMxo8fT3R0NAEBAfTo0YOdO3d6ohQREZccPpPN7R+u46HPN5OaaSMuIoj/3NOe125q5XzwKciDpVPhrY5m8PHyhW6PwwNrFHxERERKkdt3ftLS0ujcuTM9e/bkp59+okaNGhw8eJCwsLCi10ydOpVp06bx8ccfU79+fV544QX69u3L3r17CQ4OdndJIiIXZSu0897SQ7yx+AD5hQ58va082MO8s8ffx4U7ew78AnMfh3MHzXF8D7jiZYhI9EjdIiIi4jyLYRiGOxd84oknWLlyJcuXL//LrxuGQXR0NCNGjGDMmDEA2Gw2IiMjmTJlCkOHDv3Te2w2GzabrWickZFBTEwM6enphISEuLN8EamE1hw6y1Mzt3PwdDYAXRMjeO6qpsRFuHBnT8YJ+PlJ2DnTHFeJgssnQZNrwdkW2CIiIuKyjIwMQkNDncoGbn/s7fvvv6dt27bccMMN1KhRg1atWvH+++8XfT0pKYmUlBT69etXNOfn50f37t1ZtWrVX645efJkQkNDi/6JidEhYREpuXPZ+Tz2zVZuem8NB09nE1HFj9duasknd7dzPvjYC2HN2zDjMjP4WKxmB7eH1kPT6xR8REREyhC3h59Dhw7x9ttvk5iYyM8//8ywYcN45JFH+OSTTwBISUkBIDLywov8IiMji772R2PHjiU9Pb3on+TkZHeXLSKViGEYfL0hmd6vLOHbjccAuKV9LL+M7M5VLWs5f1npsY3wfk+zk1t+FtS+DO5bCpdPBn/tSouIiJQ1bj/z43A4aNu2LZMmTQKgVatW7Ny5k7fffpvbb7+96HV//MuFYRh/+xcOPz8//Pz83F2qiFRCB1IzeXLmDtYlnQOgYVQwE69pRps6VZ1fJPc8/PIcbPgQMMA/DPqMh9Z3gNUjfWRERETEDdwefmrWrEnjxo0vmGvUqBH//e9/AYiKigLMHaCaNWsWvSY1NfVPu0EiIu6SV2DnzcUHeGfpQQrsBgE+Xozok8jdXeLw8XIysBgGbP/GPNuTfdqca3Ez9H0eqlT3XPEiIiLiFm4PP507d2bv3r0XzO3bt486deoAEBcXR1RUFAsWLKBVq1YA5Ofns3TpUqZMmeLuckREWL7/NE/P2sGRszkA9GpYgwmDmxATHuj8Imf2w48jIWmZOY6oD1e+AnHdPFCxiIiIeILbw8+jjz5Kp06dmDRpEkOGDGHdunW89957vPfee4D5uNuIESOYNGkSiYmJJCYmMmnSJAIDA7nlllvcXY6IVGKpmXm88MNuvt96AoDIED/GD2rC5U2jnD/XU5ALy6fByulgzwdvf/POnk6PgLev54oXERERt3N7+LnsssuYOXMmY8eO5bnnniMuLo7p06dz6623Fr1m9OjR5Obm8sADD5CWlkb79u2ZP3++7vgREbdwOAy+WH+UKT/tISOvEIsF7uhYl1H96hPs7+P8QgcWwo+PQVqSOU7oC1e8BOFxnilcREREPMrt9/xcCq708haRymVvSiZPztzOxiNpADStFcKka5rRvHaY84tknISfx/5+Z09wTRgwBRoNVutqERGRMsaVbOD2nR8RkdKQV2DnjUX7eXfpIQodBkG+Xozs14A7OtbB29mGBg47rHsfFr0A+ZnmnT3th0HPJ8FPO9MiIiLlncKPiJR7K/af4alZ24saGvRpFMlzVzUhOizA+UVObIY5I+DkFnNcqw0MfBVqtnB7vSIiIlI6FH5EpNw6m2XjhR93M3PzccBsaDBhcFP6N4l0vqGBLRMWTYR174LhAL9Q6DMO2twFVi8PVi8iIiKXmsKPiJQ7hmHwzcZjTJq7m/M5BVgscHuHOjzWv4HzDQ0MA3bPgZ/GQKbZDY6m10P/SRCsO8dEREQqIoUfESlXDp3O4smZ21lz6BwADaOCmXxtM1rFVnV+kfNHYe7jsG+eOa5aF66cBgm93V+wiIiIlBkKPyJSLtgK7byz5BBvLj5Avt2Bv4+VEX3qc0+XOHycbWhgL4A1b8OSyVCQA1Yf6Dwcuj0GPi6cDxIREZFySeFHRMq8dUnneHLmdg6kZgHQrX51Jl7dlJjwQOcXSV4PP4yAUzvMcWwns6FBjYbuL1hERETKJIUfESmz0nMKmPzTbr5cnwxARBVfnhnYmMEtop1vaJB7Hn55DjZ8CBgQUBX6Pg8tbwWrkztGIiIiUiEo/IhImWMYBj9sO8mEObs4k2UD4KbLYnhiQEPCAn2dXQR2fgfzxkLWKXOuxc3Q7wUIivBQ5SIiIlKWKfyISJlyLC2HcbN3smhPKgD1qgcx+drmtIsLd36Rc0nw4yg4+Is5rpZgPuIW180DFYuIiEh5ofAjImWC3WHw8arDvDJ/Lzn5dny9rDzQsx7396iHn7eT9+3YC2D1DFgyBQpzwcsXuo6CLo+Ct59nP4CIiIiUeQo/IlLqdp5IZ+x329l2LB2Ay+pWZfK1zUioEez8IsnrYc5wSN1pjut2NXd7IhI9ULGIiIiURwo/IlJqcvPtTP9lH/9anoTdYRDs783YAY246bIYrFYnGxrkpZsNDdZ/gNnQIBz6TzTP9zjbFEFEREQqBYUfESkVy/ef5qmZOzh6LgeAK5pFMX5QE2qE+Du3gGHA7u9h7mjISjHnWtzya0ODah6qWkRERMozhR8RuaTOZefzwg+7+G7zcQBqhvrz3FVN6ds40vlFzifD3Mdh30/mOLye+YhbfHcPVCwiIiIVhcKPiFwShmEwc/Nxnv9hF2k5BVgscEfHujzWvwFV/Jz8rcheCOvehUUToSAbrD5mM4Ouo8DHyR0jERERqbQUfkTE446ezeGpWdtZvv8MAA2jgpl8bTNaxVZ1fpETm82GBie3muPYjjBwOtRo6P6CRUREpEJS+BERjym0O/hwZRLTFuwjr8CBr7eV4b0Tua9bPD5eVucWsWXB4omw9h0wHOAfCn2fg1a3g9XJNURERERQ+BERD9l5Ip0x/93GjuMZAHSMr8aka5sRFxHk/CJ755mXlWYcM8dNr4fLJ0OVGh6oWERERCo6hR8Rcau8Ajuv/bKf95Ydwu4wCPH35ukrG3ND29pYnG09nXkKfhoNu2aZ47A6cOU0SOzjsbpFRESk4lP4ERG3WX3wLGO/28bhs2b76iub1eTZwY2pEexkMwKHAzZ/AgvGmff3WLyg00PQ/QnwDfRg5SIiIlIZKPyISIml5xYwee5uvlyfDEBkiB/PX9WUfk2inF/kzH6zocGRleY4uhUMeh1qNvdAxSIiIlIZKfyISInM23GSZ2bv5HSmDYBb28cyZkBDQvx9nFugMB9WTodlL4E9H3wCodfT0G4oeOm3KBEREXEf/c1CRIrlVEYe42bv4OedpwCIjwjixeua0y4u3PlFjq6FOY/A6T3mOKGPebanah0PVCwiIiKVncKPiLjE4TD4akMyk+buJjOvEG+rhWHd6/FQrwT8fbycWyQvHX55DtZ/ABgQGAEDpkDT68DZpggiIiIiLlL4ERGnHTqdxdjvtrM26RwALWqH8uJ1zWlUM8T5RXb/AHMfg8yT5rjlP6Df8xDowo6RiIiISDEo/IjIRRXYHby//BDTF+4nv9BBgI8Xo/rV567OcXhZndypyThphp49P5jj8HgYOB3iu3usbhEREZH/T+FHRP6nHcfNy0p3njAvK+2aGMGka5oRE+5k62mHAzZ+BAvHgy0DrN7Q6RHoPhp8AjxXuIiIiMgfKPyIyF/642WloQE+jBvYmGtb13L+stLT+8yGBkdXm+Nabcz21VFNPVe4iIiIyN9Q+BGRP1l76CxPfLedpDPZAFzZvCbjBzWherCfcwv8qX11EPR+BtrdB1YnmyKIiIiIuJnCj4gUycwr4MWf9vDZ2qNAMS8rTV4P3z8Mp3eb44S+MHAahMV6oGIRERER5yn8iAgAv+w+xdOzdnAyPQ+Am9vF8MSARoQGOHlZqS0LFj0Pa9/FbF9dDQZMVftqERERKTMUfkQqubNZNibM2cX3W08AUKdaIJOvbUanehHOL7J/AfzwKKQnm+MWN0O/iRBUzQMVi4iIiBSPwo9IJWUYBrO3nGDCnJ2k5RRgtcC9XeMZ0ac+Ab5OnsvJPgPznoDt35jjsFizfXVCb4/VLSIiIlJcCj8ildCJ87k8NXM7i/eeBqBhVDBTr29O89phzi1gGLDtK5g3FnLPgcUKHR6Ank+Cb5DnChcREREpAYUfkUrE4TD4bO0RXvxpD9n5dny9rDzSO4Gh3evh42V1bpG0w+YjbgcXmePIZjD4dajV2mN1i4iIiLiDwo9IJZF0Jpsx/93GuqRzALSpU5Up1zUjoUawcws47LD2HVj0AhTkgJcf9HgCOj0MXk42RRAREREpRQo/IhWc3WHw4YokXp6/F1uhg0BfL0b3b8DtHetitTrZhe3ULpj9IJzYZI7rdIFBr0FEgucKFxEREXEzhR+RCmzfqUxGf7uNLcnnAeicUI0Xr21OTHigcwsU2mD5NFj+CjgKwC8U+j0HrW4Hq5OPyYmIiIiUEQo/IhVQgd3Bu0sP8vovB8i3Owj28+bpgY0Y0jYGi7N37hzbALMf+v2y0gZXwpWvQEhNzxUuIiIi4kEKPyIVzI7j6Yz+dhu7TmYA0LthDV64pik1QwOcWyA/GxZNhDVvAQYEVYcrXoLGV+uyUhERESnXFH5EKghboZ03fjnA20sPYncYhAX6MH5QE65qGe38bs+hJfD9I3D+iDlufhNcPhkCwz1Wt4iIiMilovAjUgFsPprG499u40BqFgBXNItiwuCmVA/2c26B3PMw/2nY/Kk5DqkNg6ZDYl+P1CsiIiJSGhR+RMqx3Hw7r8zfy4crk3AYEFHFj+evasKAZi6cy9n9A/w4CrJSzPFl90KfZ8HPyRbYIiIiIuWEwo9IObXm0FnG/HcbR87mAHBtq1o8M7AxVYN8nVsgKxXmPg67Zpnjagkw+A2o08kzBYuIiIiUMoUfkXImy1bIlJ/28Oka81xOzVB/Jl3TjJ4Nazi3gGHAtq9g3hOQmwYWL+g8HLqPAR9/D1YuIiIiUroUfkTKkRX7zzDmv9s4fj4XgJvbxTD2ikaE+Ps4t8D5ZPhhBBxYaI6jmsHgGRDd0iP1ioiIiJQlCj8i5UBGXgGTftzNl+uTAahdNYAp1zWnc0KEcws4HLDxQ1jwLORngZcf9BgDnR4BLyeDk4iIiEg5p/AjUsYt3pPK2O+2k5KRB8DtHesw5vKGBPk5+a/v2YPw/cNwZKU5jmlv7vZUr++hikVERETKJqunv8HkyZOxWCyMGDGiaM4wDMaPH090dDQBAQH06NGDnTt3eroUkXIlPaeAkV9v4a6P15OSkUedaoF8dV8HnruqqXPBx14IK1+HtzuZwccnCAZMhbt+UvARERGRSsmjOz/r16/nvffeo3nz5hfMT506lWnTpvHxxx9Tv359XnjhBfr27cvevXsJDlZ7XZH5O1N4atYOTmfasFjg7s5xPNavAQG+Xs4tcGoXzH4QTmwyx/E9YNDrULWOx2oWERERKes8tvOTlZXFrbfeyvvvv0/VqlWL5g3DYPr06Tz11FNce+21NG3alH//+9/k5OTw+eefe6ockXLhXHY+D3+xmfs+3cjpTBv1qgfx7bBOPDOwsXPBpzAflrwI73Yzg49fqPmI222zFHxERESk0vNY+HnwwQe58sor6dOnzwXzSUlJpKSk0K9fv6I5Pz8/unfvzqpVq/5yLZvNRkZGxgX/iFQ0P247Sd9pS5mz9QRWCwzrXo8fH+lKmzpVL/5mgOMb4b0esGQyOAqgwZXw4FpofRtYLB6tXURERKQ88Mhjb19++SWbNm1i/fr1f/paSop5i3xkZOQF85GRkRw5cuQv15s8eTITJkxwf6EiZcDpTBvjZu/gpx3mvxv1I6vw0vUtaBET5twCBbmweBKsngGGAwIj4Iqp0ORahR4RERGR/8ft4Sc5OZnhw4czf/58/P3//sJEyx/+UmYYxp/mfjN27FhGjhxZNM7IyCAmJsY9BYuUEsMw+H7rCZ79fifncwrwtlp4oEc9HuyVgJ+3k2d7Dq+E7x+Cc4fMcbMb4PIpEFTNc4WLiIiIlFNuDz8bN24kNTWVNm3aFM3Z7XaWLVvGjBkz2Lt3L2DuANWsWbPoNampqX/aDfqNn58ffn5+7i5VpNSczrTx9Kzt/LzzFACNa4bw0g3NaRId6twCtkxYOB7W/8scB0fDwFehweWeKVhERESkAnB7+Onduzfbt2+/YO6uu+6iYcOGjBkzhvj4eKKioliwYAGtWrUCID8/n6VLlzJlyhR3lyNSphiGwQ/bTjJu9g7Sft3tebhXIg/0rIePl5NH8A4tgdkPQ/pRc9z6Duj3PPg7GZxEREREKim3h5/g4GCaNm16wVxQUBDVqlUrmh8xYgSTJk0iMTGRxMREJk2aRGBgILfccou7yxEpM85k2Xhm1u9nexrXDOHlG1rQODrEuQXyMmDBONj4kTkOizU7ucV391DFIiIiIhWLR+/5+TujR48mNzeXBx54gLS0NNq3b8/8+fN1x49UWD9uO8kzs3dwLjsfb6uFB3sm8GDPBHy9ndztOfALfP8IZBwzx5fdC33Gg18Vj9UsIiIiUtFYDMMwSrsIV2VkZBAaGkp6ejohIU7+V3ORUnA2y8a42Tv5cftJABpGBfPyDS1oWsvJR9Ty0mH+07DpE3Ncta652xPX1TMFi4iIiJQzrmSDUtn5EakMftp+kqdn7eBsdj5eVgsP9qjHQ70Snd/t2b8Q5jwCGcfNcfth0Hsc+AZ5rmgRERGRCkzhR8TN0rLzGff9TuZsPQGY9/a8ckNLmtV2crcn9zz8/BRs+Y85Do+Hq96EOp08U7CIiIhIJaHwI+JGP+9M4amZOziTZcPLamFY93ge6Z3o/L09++bDnOGQeQKwQIcHoNfT4Bvo0bpFREREKgOFHxE3OJ+Tz/jvdzJri7nbk1ijCi/f0IIWMWHOLZCbBvOehK2fm+PwenD1WxDbwTMFi4iIiFRCCj8iJbRg1ymenLmd05k2rBYY2r0ew3sn4u/j5G7P3p9gzgjISgEs0PFBc7fHJ8CTZYuIiIhUOgo/IsWUnlPAhDk7+W6z2ZCgXvUgXr6hBa1iqzq3QM45mDcWtn1pjqslmrs9Me08VLGIiIhI5abwI1IMi/acYux32zmVYe723Ns1nkf71nd+t2fPj/DDo5B1CixW6PQw9Bir3R4RERERD1L4EXFBem4Bz/+wi283mpeNxkcE8dINLWhTx4Xdnp9Gw/ZvzHFEA3O3p3ZbD1UsIiIiIr9R+BFx0pK9qTzx3+2kZORhscA/u8Qxql8D53d7ds8xd3uyT5u7PZ2HQ/cnwMffs4WLiIiICKDwI3JRGXkFTPxhN19tSAagbrVAXr6hBW3rhju3QPZZ+Olx2PFfc1y9obnbU6uNhyoWERERkb+i8CPyPyzff5ox327jRLq523NXpzge79+AAF8nd3t2zoIfR0HOGbB4QZcR0H0MePt5smwRERER+QsKPyJ/IctWyMQfd/PFuqMAxIYH8tL1zWkfX83JBU7D3Mdg1yxzXKMxXPUm1GrtmYJFRERE5KIUfkT+YOWBM4z+dhvHz+cCcGenuoy+vAGBvk7+67LjOzP45Jw1d3u6joRuj2u3R0RERKSUKfyI/CrbVsjkn3bznzXmbk/tqgG8dH0LOtZzdrcn1XzEbff35rhGE/NsT3RLzxQsIiIiIi5R+BEB1h8+x6ivt3L0XA4At3WowxMDGhLk58S/IoZhNjOY+zjkngOrN3QdBV0fA29fD1cuIiIiIs5S+JFKzVZoZ9qCfby37BCGAbXCAph6fXM6J0Q4t0D2GfhxJOyabY4jm8HVb0LNFp4rWkRERESKReFHKq2dJ9IZ+dVW9p7KBOD6NrUZN6gxIf4+zi2w50eYM9y8t8fqbe70dB2l3R4RERGRMkrhRyqdQruDd5cdYvrCfRTYDaoF+TL52mb0axLl3AK552HeE7D1C3NcvRFc8zZEt/JYzSIiIiJScgo/Uqkknclm5Ndb2Hz0PAD9m0Qy8ZpmRFRxshPbgV9g9kOQeQKwQOdHoMeT4OPvsZpFRERExD0UfqRScDgM/rP2CJPm7iavwEGwnzfjBzfh2ta1sFgsF1/AlgULnoENH5rj8Hi4+h2Ibe/ZwkVERETEbRR+pMI7mZ7L6G+3sXz/GQA6J1TjpetbEB0W4NwCh1fC7Acg7bA5bjcU+jwLvkGeKVhEREREPELhRyoswzCYteU442bvJDOvEH8fK2MHNOK2DnWwWp3Y7SnIhUUvwOo3AQNCY+CqNyG+u8drFxERERH3U/iRCulslo2nZ+3gpx0pALSMCWPakBbEV6/i3ALHNsKsYXBmnzludRv0nwT+IR6qWEREREQ8TeFHKpyFu07xxHfbOZNlw9tqYUSfRIZ1r4e3l/Xiby7Mh6VTYMWrYNihShQMfh3q9/d84SIiIiLiUQo/UmFk5BXw3JxdfLvxGAD1I6swbUhLmtYKdW6BlO0w8344td0cN7sBBkyFwHAPVSwiIiIil5LCj1QIK/afYfS3WzmRnofFAvd2jWdk3/r4+3hd/M32Algx3dzxcRRAYDW4cho0udrTZYuIiIjIJaTwI+VaTn4hL/60h09WHwGgTrVAXrmhBW3rOrlbk7obZg6Dk1vMccOBMPBVqFLDMwWLiIiISKlR+JFya8Phc4z6ZitHzuYAcHvHOjwxoCGBvk78WNsLYfUbsHgS2PPBPwyueMl81M2Ze39EREREpNxR+JFyJ6/AzqsL9/HeskMYBkSH+jP1+hZ0SYxwboEz+83dnuMbzHFifxj0GoTU9FzRIiIiIlLqFH6kXNlxPJ2RX29h36ksAK5vU5txgxoT4u9z8Tc77LDmbVj0PBTmgV8IXP4itLxFuz0iIiIilYDCj5QLBXYHby4+wIxFByh0GERU8WPytc3o2zjSuQXOHoTZD8LR1ea4Xi8Y/AaE1vZc0SIiIiJSpij8SJm3/1QmI7/eyvbj6QBc2awmz1/dlPAg34u/2eGA9e/DgmehMBd8q0D/idD6Du32iIiIiFQyCj9SZtkdBh+sOMTL8/eRX+ggLNCH569qyqAW0c4tkHYYZj8Eh5eb47hucNWbEBbrsZpFREREpOxS+JEyKflcDiO/3sL6w2kA9GpYgxevbUaNEP+Lv9kwYMOHMP8ZKMgGn0Do+xy0vQesVg9XLiIiIiJllcKPlCmGYfDfTccZ//1OsmyFVPHzZtzAxtzQtjYWZx5Tyzhhnu05uMgcx3aCq9+E8HjPFi4iIiIiZZ7Cj5QZ53PyeWrmDn7cfhKAdnXDeWVIC2LCA51bYOdMmDMC8s6Dtz/0fhbaD9Nuj4iIiIgACj9SRqw8cIZRX28lJSMPb6uFR/vWZ1j3enhZndjtyUuHuaNh25fmuGZLuPZ9qF7fozWLiIiISPmi8COlylZo5+Wf9/L+8iQA4iOCmH5TS5rXDnNugcMrYeZQSE8GixW6jITuY8DbiU5wIiIiIlKpKPxIqdmbksnwLzezJyUTgFvbx/LUlY0I9HXix7LQBotegFVvAAZUrQvXvAuxHTxas4iIiIiUXwo/csk5HAYfrzrMi/P2kF/ooFqQL1Oua04fZy8sPbULvrsPTm03x61ug8sng1+w54oWERERkXJP4UcuqVMZeTz2zVaW7z8DQM8G1Zl6fQuqB/td/M0OB6x9GxZOALsNAqvBoNeh0UAPVy0iIiIiFYHCj1wy83akMPa7baTlFODnbeXpKxvxjw51nGthnX4MZt0PScvMcWJ/GPwGBDu5WyQiIiIilZ7Cj3hctq2Q5+bs4qsNyQA0rRXC9BtbklDDycfUtn8LP440u7r5BEL/idDmLnAmNImIiIiI/ErhRzxq09E0Hv1qC0fO5mCxwLDu9Xi0T318vZ24eyc3DX58DHZ8a45rtYFr3oOIBM8WLSIiIiIVksKPeER+oYPXftnH20sO4jCgVlgArwxpQYf4as4tcHARzH4IMo6DxQu6PQ7dHgMvH88WLiIiIiIVlsKPuN2elAwe/Woru09mAHBtq1o8O7gJoQFOBBdbFiwYBxs+MMfh8eaFpbXberBiEREREakMFH7EbewOg/eXH2La/H3k2x1UDfRh0jXNGNCspnMLHFllNjVIO2yO290HfcaDb5CnShYRERGRSkThR9ziyNlsRn29lQ1H0gDo06gGk69t7lwL64I8WPQ8rH4TMCCkNlz9JsT38GjNIiIiIlK5KPxIiRiGwefrjjLxx93k5Nup4ufNuEGNuaFNbedaWB/fCDPvhzN7zXHLf8Dlk8A/1LOFi4iIiEil40TLLddMnjyZyy67jODgYGrUqMHVV1/N3r17L3iNYRiMHz+e6OhoAgIC6NGjBzt37nR3KeJhpzLyuPOj9Tw1cwc5+Xbax4Xz0/CuDGkbc/HgU5gPiybCv/qawadKJNz8lbnjo+AjIiIiIh7g9vCzdOlSHnzwQdasWcOCBQsoLCykX79+ZGdnF71m6tSpTJs2jRkzZrB+/XqioqLo27cvmZmZ7i5HPOT7rSfo9+oylu47ja+3lWcGNuaLezsQEx548Tef2gn/6gXLpoJhhybXwgNroMHlni9cRERERCoti2EYhie/wenTp6lRowZLly6lW7duGIZBdHQ0I0aMYMyYMQDYbDYiIyOZMmUKQ4cOveiaGRkZhIaGkp6eTkhIiCfLlz9Iy87nmdk7+GHbSQCa1Qpl2pAWJEY6cWGpvRBWvQ6LJ4GjAAKqwpWvQNPrPFy1iIiIiFRUrmQDj5/5SU9PByA8PByApKQkUlJS6NevX9Fr/Pz86N69O6tWrfrL8GOz2bDZbEXjjIwMD1ctf2Xx3lTGfLuN1EwbXlYLD/VM4KFeCfh4ObGBeOYAzBoGx9ab4/oDYNBrEBzp2aJFRERERH7l0fBjGAYjR46kS5cuNG3aFICUlBQAIiMv/EtvZGQkR44c+ct1Jk+ezIQJEzxZqvwPufl2XvhxF5+tPQpAvepBTBvSkhYxYRd/s8MB696DheOhMBf8QuDyF6HlLeBMQwQRERERETfxaPh56KGH2LZtGytWrPjT1/54IN4wjL89JD927FhGjhxZNM7IyCAmJsa9xcpf2puSycNfbGLfqSwA7u4cx+jLG+Dv43XxN2emwMxhcGixOY7vAYNnQJj+vxMRERGRS89j4efhhx/m+++/Z9myZdSuXbtoPioqCjB3gGrW/P3yy9TU1D/tBv3Gz88PPz8n7osRtzEMg8/WHuX5H3ZhK3RQPdiPV4e0pEtihHML7JtvPuaWcxa8A6Df89D2HrC6vceGiIiIiIhT3P43UcMweOihh/juu+9YtGgRcXFxF3w9Li6OqKgoFixYUDSXn5/P0qVL6dSpk7vLkWJIzynggc828fSsHdgKHfRoUJ2fhnd1LvgU2uCnJ+DzG8zgE9kUhi6Fdvcq+IiIiIhIqXL7zs+DDz7I559/zuzZswkODi464xMaGkpAQAAWi4URI0YwadIkEhMTSUxMZNKkSQQGBnLLLbe4uxxx0YbD5xj+5RaOn8/Fx8vC6P4NuadLHFarE+dzTu+Fb++BU9vNcfth0GcC+Ph7tmgRERERESe4Pfy8/fbbAPTo0eOC+Y8++og777wTgNGjR5Obm8sDDzxAWloa7du3Z/78+QQHO9EuWTzC7jB4e8kBXl24H7vDoE61QN64uRXNa4dd/M2GAZs+gZ/GmE0NAqvBVW/p3h4RERERKVM8fs+PJ+ieH/c6lZHHiC+3sPrQWQCuahnNC1c3Jdjf5+Jvzk2DOcNh12xzHN8DrnkXgqM8V7CIiIiIyK/K1D0/UrYt2nOKx77ZxrnsfAJ9vXjuqqZc17rW33beu8CR1fDdvZCeDFZv6PUMdHpEZ3tEREREpExS+KmkbIV2pvy0lw9XJgHQuGYIb9zSinrVq1z8zfZCWP4yLJ0ChgOqxsF1H0DtNh6uWkRERESk+BR+KqGkM9k8/MUmdhzPAODOTnUZe0VD/LyduLvnfLK523N0tTlufhNc+TL46byWiIiIiJRtCj+VzHebjvHMrB1k59upGujDS9e3oE/jv75f6U92zYbvH4a8dPANhoHToPkQzxYsIiIiIuImCj+VxLnsfJ6ZvYMft50EoH1cOK/d1IqoUCfaUNsy4ecnzY5uALXawHX/gvB4D1YsIiIiIuJeCj+VwC+7TzHmv9s5k2XDy2rhkV6JPNQrAS9n7u5JWgazH4TzRwELdBkBPZ8CLyc6wYmIiIiIlCEKPxVYZl4BL/ywm682JAOQUKMK04a0cO7unvwcWDge1r1rjsNizbt74rp6rF4REREREU9S+KmgVh88y2PfbOX4+VwsFrincxyP9W+Av48TTQ2OroVZ98O5g+a4zZ3Q7wU1NRARERGRck3hp4LJK7Azdd7vLaxrVw3g5Rta0CG+2sXfXJAHiyfC6hlmC+vgaBj8BiT28XDVIiIiIiKep/BTgWxNPs/Ir7dw8HQ2ADddFsPTAxtTxc+J/5tPbIaZw+D0HnPc4ma4/EUICPNcwSIiIiIil5DCTwVQYHfwxi/7eXPJQewOg+rBfky5rhm9GjrRwrow37ywdNnLYNghqAYMeg0aXuH5wkVERERELiGFn3Ju36lMRn69pejC0oHNa/L8VU2pGuR78Ten7IBZwyBluzluci1c8TIEOfGInIiIiIhIOaPwU07ZHQYfrDjEy/P3kV/oICzQh+evasqgFtFOvLkQVk6HJS+CowACwuHKV6DptR6vW0RERESktCj8lEPJ53IY+fUW1h9OA6Bng+pMua45NUKcuLD09D5zt+f4RnPc4AoYOB2CnXhETkRERESkHFP4KWcW7jrFyK+3kJFXSJCvF08PbMxNl8VgsVzkwlLDgE3/hp/GQGEe+IXCgCnQ4ia42HtFRERERCoAhZ9yotDu4JUF+3h7iXn3TqvYMF6/qRUx4YEXf3N+NvwwErZ9aY7r9YLBMyC0lgcrFhEREREpWxR+yoHTmTYe/mITaw6dA+DOTnV58opG+HpbnXjzPvj6dji9GyxW6PUMdB4BVifeKyIiIiJSgSj8lHHrks7x0OebSM20EeTrxYvXNXeuqQHA9m/h+0egIBuqRML1H0LdLp4tWERERESkjFL4KaMMw+Bfy5N4cd4e7A6DxBpVePsfbUioUeXiby60wbyxsOEDc1y3K1z3gZoaiIiIiEilpvBTBmXkFTD6m23M25kCwFUto5l0TTOC/Jz4vyvtMHxzJ5zYbI67PgY9nwSrl8fqFREREREpDxR+ypjdJzO4/z8bOXw2Bx8vC+MGNuYfHepcvJsbwN6fYOZQyEuHgKpw7fuQ2NfzRYuIiIiIlAMKP2XItxuP8fSs7eQVOKgVFsCbt7amZUzYxd9oL4RFz8HK18xxrbZww8cQFuPJckVEREREyhWFnzIgr8DOhDk7+WJdMgDd61dn+o0tqRrke/E3Z5yEb++Go6vMcfv7oe9z4O3Ee0VEREREKhGFn1KWfC6H+z/byI7jGVgs8Gif+jzUMwGr1YnH3A4tgf/+E7JPg28wXDUDmlzt6ZJFRERERMolhZ9StHDXKUZ+vYWMvEKqBvrw+s2t6JpY/eJvdDhg+cuweBJgQGRTGPIJVKvn8ZpFRERERMorhZ9SkJ5bwMQfd/H1hmMAtIoN481bWhMdFnDxN587ZN7dc3i5OW51G1zxEvg48V4RERERkUpM4ecSW7wnlbHfbSclIw+LBe7pHMfoyxvi623932902GHde/DLc1CQAz6BcMXL0OrWS1O4iIiIiEg5p/BziaTnFvD8D7v4dqO52xMXEcTU65tzWd3wi7/59D74/iFIXmuO63aFwW9AeJwHKxYRERERqVgUfi6BRXtOMfa77ZzKsBXt9ozq14AA34tcPGovhNVvwOLJYLeZTQ36PQet7wTrRXaKRERERETkAgo/HpSeU8CEH3by3abjAMRHBPHSDc1pU8eJ3Z5TO2H2g3Biszmu1xsGvaa7e0REREREiknhx0MW7jrFkzO3k5pp7vbc2zWekX3r4+9zkd2ewnxY8SosewkcBeAfCpe/CC1uBosT7a9FREREROQvKfy42fmcfCbM2cXMzb/u9lQP4qXrW9CmTtWLv/nEFnO359QOc9zgCrhyGoTU9FzBIiIiIiKVhMKPG83fmcKTM3dwJsuG1QL3dovn0T5O7PYU5MGyqbBiOhh2CAg321c3vU67PSIiIiIibqLw4wZp2fmMn7OT2VtOAJBQowovXd+cVrFO7PYkrzd3e87sNcdNroEBL0EVJy47FRERERERpyn8lNDPO1N4auZ2zmTlY7XA0O71GN478eK7PfYC886e1TPAcEBQDbjyFWg8+NIULiIiIiJSySj8lNDukxmcyconsUYVXrqhBS1jwi7+ppxz8M2dkLTUHDe/CS6fDIFOdIETEREREZFiUfgpoQd6JBDs78M/OsTi532R3R6A03vhi5vg3CHwCYJr3tFuj4iIiIjIJaDwU0K+3lbu6RLn3Iv3L4Bv7wZbBoTGws1fQFRTzxYoIiIiIiKAws+lYRjm2Z4F48zzPbGd4MZPISiitCsTEREREak0FH48rdAGPzwKWz4zx61vhyteAW/f0q1LRERERKSSUfjxpKxU+OofkLwWLFboPxnaD9XdPSIiIiIipUDhx1NOboUvboGMY+AXCjd8BAm9S7sqEREREZFKS+HHE3bNhpnDoCAHqiXAzV9CRGJpVyUiIiIiUqkp/LiTYcDSqbBkkjmu1wuu/xACqpZuXSIiIiIiovDjNvk5MOt+2DXLHHd4APo+D176JRYRERERKQv0N3N3SD8GX9wMKdvA6gMDp5ld3UREREREpMxQ+Cmp5PXw5S2QnQqB1eDG/0CdTqVdlYiIiIiI/IHCT0lt+8oMPjWawM1fQNU6pV2RiIiIiIj8BYWfkuo/ydzx6fQw+FUp7WpERERERORvKPyUlLcv9Bxb2lWIiIiIiMhFWEvzm7/11lvExcXh7+9PmzZtWL58eWmWIyIiIiIiFViphZ+vvvqKESNG8NRTT7F582a6du3KgAEDOHr0aGmVJCIiIiIiFZjFMAyjNL5x+/btad26NW+//XbRXKNGjbj66quZPHnyBa+12WzYbLaicUZGBjExMaSnpxMSEnLJahYRERERkbIlIyOD0NBQp7JBqez85Ofns3HjRvr163fBfL9+/Vi1atWfXj958mRCQ0OL/omJiblUpYqIiIiISAVRKuHnzJkz2O12IiMjL5iPjIwkJSXlT68fO3Ys6enpRf8kJydfqlJFRERERKSCKNVubxaL5YKxYRh/mgPw8/PDz8/vUpUlIiIiIiIVUKns/ERERODl5fWnXZ7U1NQ/7QaJiIiIiIi4Q6mEH19fX9q0acOCBQsumF+wYAGdOnUqjZJERERERKSCK7XH3kaOHMltt91G27Zt6dixI++99x5Hjx5l2LBhpVWSiIiIiIhUYKUWfm688UbOnj3Lc889x8mTJ2natClz586lTp06pVWSiIiIiIhUYKV2z09JuNLLW0REREREKq4yf8+PiIiIiIjIpabwIyIiIiIilYLCj4iIiIiIVAoKPyIiIiIiUiko/IiIiIiISKWg8CMiIiIiIpVCqd3zUxK/defOyMgo5UpERERERKQ0/ZYJnLnBp1yGn8zMTABiYmJKuRIRERERESkLMjMzCQ0N/Z+vKZeXnDocDk6cOEFwcDAWi6W0yyEjI4OYmBiSk5N16aq4RD87UhL6+ZGS0M+PlIR+fqS4PPGzYxgGmZmZREdHY7X+71M95XLnx2q1Urt27dIu409CQkL0G4AUi352pCT08yMloZ8fKQn9/Ehxuftn52I7Pr9RwwMREREREakUFH5ERERERKRSUPhxAz8/P5599ln8/PxKuxQpZ/SzIyWhnx8pCf38SEno50eKq7R/dsplwwMRERERERFXaedHREREREQqBYUfERERERGpFBR+RERERESkUlD4ERERERGRSkHhR0REREREKgWFnxJ66623iIuLw9/fnzZt2rB8+fLSLknKoGXLljFo0CCio6OxWCzMmjXrgq8bxv+1d38hTTVgGMCf0ZqZDcn+eBxmrFpFpYNcf45URtZgRBTdFEkMuhJmOKSLqAu9ECddBIVRSBB1EeuipG4KB9mBiGCaozEijCS70EYX1RqoZO938dGhMeem0Xfmt+cHB9z7HuS9eDjudduZoL29HTabDcXFxdi/fz9isZgxw1JeCQQC2LFjB6xWK1avXo1jx47h7du3KecwP5TJ9evXUVNTo3+TuqqqePz4sd5ndmguAoEATCYT/H6/XmOGKJP29naYTKaUQ1EUvW9Udrj8/IF79+7B7/fj4sWLGBoawt69e+HxeDA6Omr0aJRnkskknE4nuru7Z+xfunQJly9fRnd3N8LhMBRFwaFDh5BIJP7jSSnfaJoGn8+Hly9fIhQK4cePH3C73Ugmk/o5zA9lUllZia6uLgwMDGBgYAAHDhzA0aNH9ScYzA7lKhwOo6enBzU1NSl1Zohms3XrVoyNjelHNBrVe4ZlR2jedu7cKU1NTSm1zZs3y/nz5w2aiBYCANLb26s//vnzpyiKIl1dXXptYmJCSktL5caNGwZMSPksHo8LANE0TUSYH5q75cuXy82bN5kdylkikRCHwyGhUEjq6+ulpaVFRHj9odm1tbWJ0+mcsWdkdvjKzzxNTU1hcHAQbrc7pe52u/HixQuDpqKFaGRkBOPj4ylZKioqQn19PbNEab5+/QoAKCsrA8D8UO6mp6cRDAaRTCahqiqzQznz+Xw4fPgwDh48mFJnhiib4eFh2Gw22O12nDx5Eu/fvwdgbHbMf/W3/499/vwZ09PTKC8vT6mXl5djfHzcoKloIfqVl5my9OHDByNGojwlImhtbcWePXuwbds2AMwPZReNRqGqKiYmJrBs2TL09vZiy5Yt+hMMZodmEwwG8erVK4TD4bQerz80m127duHOnTvYuHEjPn36hI6ODtTV1SEWixmaHS4/f8hkMqU8FpG0GlEumCXKprm5Ga9fv8bz58/TeswPZbJp0yZEIhF8+fIF9+/fh9frhaZpep/ZoUw+fvyIlpYW9PX1YcmSJRnPY4ZoJh6PR/+5uroaqqpi/fr1uH37Nnbv3g3AmOzwbW/ztHLlSixatCjtVZ54PJ62xRLN5tedT5glms3Zs2fx6NEj9Pf3o7KyUq8zP5SNxWLBhg0b4HK5EAgE4HQ6ceXKFWaHshocHEQ8HkdtbS3MZjPMZjM0TcPVq1dhNpv1nDBDlIuSkhJUV1djeHjY0OsPl595slgsqK2tRSgUSqmHQiHU1dUZNBUtRHa7HYqipGRpamoKmqYxSwQRQXNzMx48eICnT5/Cbren9JkfmisRweTkJLNDWTU0NCAajSISieiHy+VCY2MjIpEI1q1bxwxRziYnJ/HmzRtUVFQYev3h297+QGtrK06fPg2XywVVVdHT04PR0VE0NTUZPRrlme/fv+Pdu3f645GREUQiEZSVlaGqqgp+vx+dnZ1wOBxwOBzo7OzE0qVLcerUKQOnpnzg8/lw9+5dPHz4EFarVf8vWWlpKYqLi/Xv3GB+aCYXLlyAx+PBmjVrkEgkEAwG8ezZMzx58oTZoaysVqv++cJfSkpKsGLFCr3ODFEm586dw5EjR1BVVYV4PI6Ojg58+/YNXq/X2OvPX72XXAG4du2arF27ViwWi2zfvl2//SzR7/r7+wVA2uH1ekXk31s+trW1iaIoUlRUJPv27ZNoNGrs0JQXZsoNALl165Z+DvNDmZw5c0b/G7Vq1SppaGiQvr4+vc/s0Fz9fqtrEWaIMjtx4oRUVFTI4sWLxWazyfHjxyUWi+l9o7JjEhH5u+sVERERERGR8fiZHyIiIiIiKghcfoiIiIiIqCBw+SEiIiIiooLA5YeIiIiIiAoClx8iIiIiIioIXH6IiIiIiKggcPkhIiIiIqKCwOWHiIiIiIgKApcfIiIiIiIqCFx+iIiIiIioIHD5ISIiIiKigvAPx1zmMXQLbdEAAAAASUVORK5CYII=",
      "text/plain": [
       "<Figure size 1000x600 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Transition probabilities\n",
    "alpha = 0.3\n",
    "beta = 0.2\n",
    "\n",
    "# Discount parameter\n",
    "gamma = 0.99\n",
    "\n",
    "# Expected rewards\n",
    "r_search = 6.0\n",
    "r_wait = 2.0\n",
    "\n",
    "V = np.zeros(nb_states)\n",
    "Q = np.zeros((nb_states, nb_actions))\n",
    "\n",
    "pi = np.array([a['search'], a['search']], dtype=int)\n",
    "\n",
    "V_high_history = []\n",
    "V_low_history = []\n",
    "\n",
    "for k in range(50):\n",
    "    \n",
    "    # Policy evaluation\n",
    "    Q[s['high'], a['search']] = alpha * (r_search + gamma * V[s['high']]) + (1 - alpha) * (r_search + gamma*V[s['low']])\n",
    "    Q[s['high'], a['wait']] = r_wait + gamma * V[s['high']]\n",
    "\n",
    "    Q[s['low'], a['search']] = beta * (r_search + gamma * V[s['low']]) + (1 - beta) * (-3 + gamma*V[s['high']])\n",
    "    Q[s['low'], a['wait']] = r_wait + gamma * V[s['low']]\n",
    "    Q[s['low'], a['recharge']] = gamma * V[s['high']]\n",
    "\n",
    "    V[s['high']] = Q[s['high'], :2].max()\n",
    "    V[s['low']] = Q[s['low'], :].max()\n",
    "\n",
    "    V_high_history.append(V[s['high']])\n",
    "    V_low_history.append(V[s['low']])\n",
    "        \n",
    "    # Compute the greedy policy\n",
    "    pi_old = pi.copy()\n",
    "    pi[s['high']] = Q[s['high'], :2].argmax()\n",
    "    pi[s['low']] = Q[s['low'], :].argmax()\n",
    "    \n",
    "print('Greedy policy after iteration', k)\n",
    "print('pi(high)=', pi[s['high']])\n",
    "print('pi(low)=', pi[s['low']])\n",
    "print(\"V=\", V)\n",
    "print(\"Q=\", Q)\n",
    "    \n",
    "plt.figure(figsize=(10, 6))  \n",
    "plt.plot(V_high_history, label=\"high\")\n",
    "plt.plot(V_low_history, label=\"low\")\n",
    "plt.legend()\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**A:** The optimal policy stays the same (search in high, recharge in low) but the V values grow very high. The difference between the values of the high and low state is comparatively very small: the high state is always only one action away from the low state, it is nothing with such a high gamma."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Q:**  Change the parameters to:\n",
    "\n",
    "$$\\alpha = 0.01 \\quad \\beta = 0.2 \\quad \\gamma = 0.7 \\quad \\mathcal{R}^{\\text{search}} = 6 \\quad  \\mathcal{R}^{\\text{wait}} = 5$$\n",
    "\n",
    "Find the optimal policy. What is the optimal action to be taken in the state *high*, although the probability to stay in this state is very small? Why?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Greedy policy after iteration 49\n",
      "pi(high)= 0\n",
      "pi(low)= 1\n",
      "V= [17.67371571 16.66666637]\n",
      "Q= [[17.67371571 17.37160091  0.        ]\n",
      " [11.030614   16.66666637 12.37160091]]\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAzYAAAH5CAYAAABTbqsJAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjYuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/av/WaAAAACXBIWXMAAA9hAAAPYQGoP6dpAABOFUlEQVR4nO3deXhU9d3//9dsmSxkIUBIAgmbyCKroAiiQLVqVKRqXaq3pdq7atW6cNettkoXRW3rbdW21vauy69au6jULrb4VRYVVBZRBGQNWyCEJWTPZDJzfn+czGRfJpmZM5M8H9c11zlzzpkzb+CIeXE+5/2xGYZhCAAAAADimN3qAgAAAACgpwg2AAAAAOIewQYAAABA3CPYAAAAAIh7BBsAAAAAcY9gAwAAACDuEWwAAAAAxD2n1QW05Pf7dfDgQaWmpspms1ldDgAAAACLGIahiooK5ebmym7v+J5MzAWbgwcPKi8vz+oyAAAAAMSI/fv3a+jQoR0eE3PBJjU1VZJZfFpamsXVAAAAALBKeXm58vLyghmhIzEXbALDz9LS0gg2AAAAALr0iArNAwAAAADEPYINAAAAgLhHsAEAAAAQ92LuGZuu8vl88nq9VpcRV1wulxwOh9VlAAAAAGEXcrBZtWqVfvrTn2r9+vU6dOiQ3njjDX3lK18J7q+srNR9992npUuX6tixYxo+fLhuv/12ffvb3w5LwYZhqLi4WCdOnAjL+fqajIwMZWdnM0cQAAAAepWQg01VVZUmT56s66+/Xpdffnmr/XfddZeWL1+uP/zhDxo+fLiWLVumW265Rbm5uVqwYEGPCw6EmqysLCUnJ/MDehcZhqHq6mqVlJRIknJyciyuCAAAAAifkINNQUGBCgoK2t2/Zs0aLVy4UHPnzpUk3XjjjfrNb36jdevWtRlsPB6PPB5P8H15eXm75/b5fMFQM2DAgFBL7/OSkpIkSSUlJcrKymJYGgAAAHqNsDcPmD17tt58800VFRXJMAwtX75c27dv1/nnn9/m8UuWLFF6enrwlZeX1+65A8/UJCcnh7vsPiPwe8fzSQAAAOhNwh5snnrqKY0fP15Dhw5VQkKCLrjgAv3qV7/S7Nmz2zz+/vvvV1lZWfC1f//+Tr+D4Wfdx+8dAAAAeqOwd0V76qmn9OGHH+rNN9/UsGHDtGrVKt1yyy3KycnRueee2+p4t9stt9sd7jIAAAAA9CFhvWNTU1Oj733ve3riiSc0f/58TZo0Sbfddpuuuuoq/exnPwvnV8WduXPn6s4772x3v81m09KlS7t8vhUrVshms9EdDgAAAFCY79h4vV55vV7Z7c3zksPhkN/vD+dX9TqHDh1S//79rS4DAAAAiEshB5vKykrt3Lkz+L6wsFAbN25UZmam8vPzNWfOHN19991KSkrSsGHDtHLlSr300kt64oknwlp4b5OdnW11CQAAAEDcCnko2rp16zR16lRNnTpVkrRo0SJNnTpVDz74oCTp1Vdf1WmnnaZrr71W48eP16OPPqqHH35YN998c3grj0N+v1/33HOPMjMzlZ2drcWLFwf3tRyKtnr1ak2ZMkWJiYmaPn26li5dKpvNpo0bNzY75/r16zV9+nQlJydr1qxZ2rZtW3R+MQAAAEAMCfmOzdy5c2UYRrv7s7Oz9fzzz/eoqFAYhqEary9q39dUkssRUpexF198UYsWLdJHH32kNWvW6Bvf+IbOPPNMffnLX252XEVFhebPn68LL7xQr7zyivbu3dvu8zkPPPCAfv7zn2vQoEG6+eabdcMNN+iDDz7oyS8LAAAAiDth74oWbTVen8Y/+B9LvnvLj85XckLXfwsnTZqkhx56SJI0evRoPfPMM3rnnXdaBZuXX35ZNptNv/3tb5WYmKjx48erqKhI3/rWt1qd8+GHH9acOXMkSffdd58uuugi1dbWKjExsQe/MgAAACC+hH0eG7Rv0qRJzd7n5OSopKSk1XHbtm3TpEmTmoWT008/vdNz5uTkSFKb5wQAAAB6s7i/Y5PkcmjLj8637LtD4XK5mr232WxtdoszDKPVELf2hv81PWfgM3SgA7rHMAx5fYbq/X5z6fPL5zfk9Teu+/yGfIbRuN701c52vyH5DEOGYchvGPL5JX/D++br5rH+huMC64Zh1mYYatymptvMYw01Hus3ZK432WZ+pnGb35AU3N/k2CbHmb8xCn5fw9vgcYHft+DfUE0+Fzhvw7c0WVdwf+O7xmObb23+91/z7W0f3/Izrf+c29jW6gytj+vglO2eo6ufbf+cPdCjD3fn66L8hYiY7l6v6H2+d9E4nZofP1174z7Y2Gy2kIaDxYOxY8fq5ZdflsfjCU5eum7dOourAiLH5zeflav1+lRT17AMrNf729jmk8frV53Pr7r6Ji9fi2U76/X+hsDSEF4CwcXP/8wBAAgqq/FaXUJIelci6CWuueYaPfDAA7rxxht13333ad++fcEJTkNpVgBEWq3Xp9LqOpXX1KvS41VFbb0qPfWqbFi2eu+pV2Wtt9m22oaAEqvsNsnpsMtpt8lhtwWXdpu5bm9477Db5LA1WW84pnFdwfc2m02Ohvc2m00Ou7lut5nnswf3BbY3vrc1vLepYdniOJvNJpsat9vU5LOS1LC/6bbAeRVYV/N9UuO2gMbvCexvfo7AtsAbW/Bzrc/X9G+1pvtb7mw8h63Vtqafbeu9TbZ293Wk5d+5nX2so/O2rKGrevLXfjz9H4P/vQGx55TcNKtLCAnBJgalpaXp73//u7797W9rypQpmjhxoh588EFdc801NAVAxHh9fp2o9qq0uk7Hq+pUWlWn49Xm8ljwvddcVtWptLpO1XXh70iY5HIoKcGhRKddiQkO873LocSGV2Cf22VXgsOhBKddCU673E67XA6bEhx2JTgbtyc4zH2B966GkOJ02OS0m59xOuxyBYKLo2Gb3R4MLgAAIPYRbKJkxYoVrbY1nbem5XjwWbNm6dNPPw2+f/nll+VyuZSfny+p7bbbU6ZM6XBcOfour8+vw+W1Ki6r1cGyWhWX1ejgiVodKqsJbjta6enWuGqn3ab0JJdSE53ql+hUP7dT/dwN792N24LvG7alul1KdjuUnNAYXNxOO3clAQBAtxBsYtRLL72kkSNHasiQIfr0009177336sorr1RSUpLVpSEG1fv82nu8WrtKKrX3WLUONgksh07U6EgXQ4vNJmUkudQ/JUGZyQnqn5KgASkJzd5nprjUPzlBmQ3bU91OwggAALAcwSZGFRcX68EHH1RxcbFycnJ0xRVX6OGHH7a6LFisuq5eu49UaWdJpXYdqdTOEvO151iVvL6Ok4vLYVN2eqJy0pOU02xprmenJyozJUEOhl4BAIA4RLCJUffcc4/uueceq8uARcqqvdp2uCIYXAIhpuhETbufSXI5NCorRSMG9lNuRqJyG8JKYDkgJYHnRQAAQK9FsAEsZhiGdh2p0oa9pVq397jW7y3VriNV7R6fmZKgkwb106isfjqp4TVqUIpy05MILgAAoM8i2ABRVlPn02cHTmjd3lJt2Fuq9ftKdaK6dZ/4IRlJweBihhdzmZmSYEHVAAAAsY1gA0TY4fJardtTqvV7S7V+73FtPliu+hYzQbqddk3Oy9C0Yf01Lb+/Th3WnwADAAAQAoINEGb1Pr8+Ljyutz4v1rtflLT5XExWqlvTh/fXtGGZmjasv8bnpCnBabegWgAAgN6BYAOEgdfn1+pdx/TWpkNatuWwjlfVBffZbdK4nDTzbkzDa0hGEi2SAQAAwohgA3RTrden93cc1VufF+vtLcUqr60P7stIdum88YN1wYRsnT5igPq5+U8NAAAgkvhpK0rmzp2rKVOm6Mknn7S6FPRATZ1PK7aVBIeZVXoaw8zAfm6df8pgFUzI0YyRmXI5GFoGAAAQLQQboBNVnnq980WJ3tp0SCu2HVGN1xfcl52WqAsmZKtgQramD89kcksAAACLEGyAdpRU1Or5D/boDx/uVUWTYWZD+yfpwok5umBCtqYMzWDuGAAAgBjAWBkLlJaW6utf/7r69++v5ORkFRQUaMeOHZLMyRoHDRqk1157LXj8lClTlJWVFXy/Zs0auVwuVVZWRr32vqDwaJXuf32TZj+2XL9esUsVtfUaNiBZt84bpX98Z7beu2eevnfhOJ2a359QAwAAECPi/46NYUjeamu+25UsdaOz1Te+8Q3t2LFDb775ptLS0nTvvffqwgsv1JYtW+RyuXT22WdrxYoVuvzyy1VaWqotW7YoJSVFW7Zs0fjx47VixQpNmzZN/fr1i8Avqu/67MAJPbtyl976vFhGwzQzp+Zn6OY5o3TuuMGEGAAAgBgW/8HGWy09kmvNd3/voJSQEtJHAoHmgw8+0KxZsyRJL7/8svLy8rR06VJdccUVmjt3rp577jlJ0qpVqzR58mTl5+drxYoVwWAzd+7ccP9q+iTDMPTejqN6duUurd51LLj9S2OzdPOcUTpteH/aMgMAAMSB+A82cWbr1q1yOp2aMWNGcNuAAQM0ZswYbd26VZLZQe2OO+7Q0aNHtXLlSs2dO1f5+flauXKlbrzxRq1evVp33nmnRb+C3qHe59dbnxfr2ZW7tPlguSTJYbdpweRc3ThnpMZmp1lcIQAAAEIR/8HGlWzeObHqu0NkBMY4tbE9cGdgwoQJGjBggFauXKmVK1fqRz/6kfLy8vTwww9r7dq1qqmp0ezZs3tUel9V6/XpL+v267fvFWrfcXMIY5LLoatPz9M3Z4/Q0P6h/5kCAADAevEfbGy2kIeDWWn8+PGqr6/XRx99FByKduzYMW3fvl3jxo2TJNlsNp199tn629/+ps8//1xnnXWWUlNT5fV69eyzz+rUU09Vamqqlb+MuFPr9en/3i/U798v1LGqOklS/2SXFs4aroUzh6t/SoLFFQIAAKAn4j/YxJnRo0drwYIF+ta3vqXf/OY3Sk1N1X333achQ4ZowYIFwePmzp2ru+66S1OnTlVamjks6uyzz9bLL7+sRYsWWVV+XNqwr1R3/+VT7TpSJUkakpGkb501QleelqfkBP4TAAAA6A1o92yB559/XtOmTdPFF1+smTNnyjAM/etf/5LL5QoeM2/ePPl8vmZNAubMmSOfz6c5c+ZYUHX8qfX69Mi/tuqrv16tXUeqNCjVrSeunKwVd8/VN84cQagBAADoRWxGew99WKS8vFzp6ekqKysL3qkIqK2tVWFhoUaMGKHExESLKoxvfeX3cP3e47r7L59p91HzLs1lpw7RgxePV0YyQ84AAADiRUfZoCX+yRq9Sk2dTz9ftk3/90GhDEManObWI5dO1DnjBltdGgAAACKIYINeY92e47r7r5+psOEuzVenDdUPLhqv9GRXJ58EAABAvCPYIO7V1Pn00/9s0/OrG+/SPHrZJM0bm2V1aQAAAIgSgg3i2seFx3XPXz/VnmPmnDRXTh+qBy4ar/Qk7tIAAAD0JQQbxKXquno9/u9tenHNHhmGlJOeqCWXTdTcMdylAQAA6IviMtj4/X6rS4hbveH37qPdx3TPa59pb8Ndmqum5+mBi8cpLZG7NAAAAH1VXAWbhIQE2e12HTx4UIMGDVJCQoJsNpvVZcUFwzBUV1enI0eOyG63KyEh/toeG4ahJ//fDv3inR2SzLs0j14+SXNOHmRxZQAAALBaXAUbu92uESNG6NChQzp48KDV5cSl5ORk5efny26Pr7lZ/X5DP/rHFr2weo8k6erT8vS9i7hLgzhmGJLhl/w+c2n4JcPXZJvRuF1N1lttN9rZ3rAtcEzw2Da2dbpU2/sUWLR1nJoc09b7lvs6OL7Tbe38/rY8b4efa2Nbq+M6mfatw2nhujllnCVTzVnwnbE1pR6AgJPOkdKHWl1Fl8VVsJHMuzb5+fmqr6+Xz+ezupy44nA45HQ64+4ul89v6P7XP9Of1x2QJP14wSm6buZwa4tC9BmG5PNK3mqpvrbhVSf5PE2WHslXZ74C682WHvMcPq/krzdfPq/k90q+hvf+JvtbHmf4Gt4HAkhgPbDd3+IYX5NjWwQYAABi3bV/JdhEms1mk8vlksvFv9b3dnX1ft31543652eHZLdJP/3qZF0+LX7+A+uzfF7JUyF5yhuWTV9Nt1WaQcVb0xhYAuvBZZNtfTIQ2CSbXbIFlvY2ttmab+twvcmyrW3tLtViW8N7qe3jmm4Prne2r8n+Lm9XB9s7+kwb52jzvLYuHNPJebv02Qj9g1O3/yErvv4BLOri7B8IgW5LGWh1BSGJy2CDvqHW69OtL2/QO1+UyOWw6amrp6pgYo7VZfUt3lqp+liL1/HW22pPNA8v9bWRrctml5yJkiNBcrolh1tyJpjvg9ta7mtyjN0l2R2Sw2WuO1yS3Wm+AuuBfXan5HA27rc7JZtDstubrDsa1u1N1hu22+yN+wL7bXZzfyCYBLcFtgeOja8howAAWIlgg5hU5anXt15ap9W7jsnttOs3102jlXO4GIYZRsoOSOVFUvlBc73ikFR1tHl48Vb17LtcyZI7tcUrrXE9IUVypUiupBav5OZLZ2LzbQ4X/2IKAACaIdgg5pRVe/WNFz7WJ/tOKCXBof/7xmk6Y+QAq8uKD4ZhhpLyA42BJRheihrXfZ6un9PulJIHNHlltng/QErMkBLTmgeYhFTzTgcAAEAU8FMHYsrRSo++/n8fa8uhcqUnufTiDadrSl6G1WXFHr9PKt0jHd0uHdnWZLlD8pR17Rz9BktpuVLakIZXrtQvq3WAcadxdwQAAMQ8gg1ixqGyGv3X7z7SriNVGtjPrT/89+kam51mdVnW8tZIx3Y2hpej26Uj281tHd11SRnUGFjSh7RYz5VSc83nTQAAAHoJgg1iwr5j1brmdx/qQGmNctMT9Yf/nqGRg/pZXVZ0eWukg59I+z+S9q+VSjZLpXvV7pwSzkRpwGhp4Ghp0Bhp4MnmMnOU5EqMaukAAABWI9jAcjtLKnTt7z7S4XKPhg1I1sv/PUND+ydbXVbkVRSbIWbfR+by0KfmHCotJWY0BJfR0sAxjSEmI9/spgUAAACCDaz1eVGZvv77j3W8qk4nD+6nP3xzhrLSeuHdBr9POry54W5Mw+vEvtbHpWRJ+TOkvDOknMlmiEkZxDMuAAAAnSDYwDLr9x7XN55fq4raek0amq4Xrz9d/VN6yXMfhiEd3CBtXybt/1A6sE6qq2xxkE0aPEHKO13Km2EGmoxhhBgAAIBuINjAEh/sPKr/fnGdarw+nTa8v37/jdOUmuiyuqye8fulAx9LW96Utr4ple1vvj8hVRo6Xco/wwwzQ6abLZIBAADQYwQbRN224gp988W1qvX6ddbogfrNddOUnBCnl6KvXtq3uiHM/F2qLG7c50qRRp8rjTjbHFqWNY5nYgAAACIkTn+aRLyqqfPptlc2qNbr1+yTBup3C6fL7YyzH/Z9XqlwpRlmvviHVH2scZ87TRpTII27RDrpHMmVZF2dAAAAfQjBBlG1+M3N2lFSqaxUt568ekr8hBpvrbR7uRlmtv1Tqm0yCWZSf2nsRdK4BdLIOZLTbV2dAAAAfRTBBlHzt41F+tO6/bLZpCevmqKB/eIgABzZJq1+Wtq8VKqraNyekiWNu9i8MzN8tuSI8+eDAAAA4hzBBlGx52iVHnjjc0nSd+adpFknDbS4ok7s/1h6/0nz7kxAaq40/hIzzOSfwfMyAAAAMYRgg4jz1Pv0nT9+okpPvU4fnqnbzxltdUltMwxpxzIz0Oxb3bDRZg4zm3mr2QDAbreyQgAAALSDYIOIe+ytbdpUVKaMZJd+8bUpcjpiLBz4vNLnr0kf/EIq2WJus7ukyVdJs+6QBp1sbX0AAADoVMg/Ya5atUrz589Xbm6ubDabli5d2uqYrVu36pJLLlF6erpSU1N1xhlnaN++NmZZR6/3/7Yc1u8/KJQk/eyrk5WTHkNdwuqqpA9/LT01VXrjJjPUJKRKs74j3fmZtOCXhBoAAIA4EfIdm6qqKk2ePFnXX3+9Lr/88lb7d+3apdmzZ+ub3/ymfvjDHyo9PV1bt25VYmJiWApG/DhUVqPv/vVTSdINZ47QueMHW1xRg6pj0sfPSR//RqopNbelZEln3CxN/6aUlGFpeQAAAAhdyMGmoKBABQUF7e5/4IEHdOGFF+rxxx8Pbhs5cmT3qkPcqvf5dccfN+pEtVcTh6Tr3oIxVpckndgnrX5G2vCSVF9jbus/QjrzdmnyNZKL8A0AABCvwvqwg9/v1z//+U+dfPLJOv/885WVlaUZM2a0OVwtwOPxqLy8vNkL8e+pd3bo4z3H1c/t1NNfm2rtfDX1ddLyJeaQs49/Y4aanCnSFS9I31kvTb+BUAMAABDnwhpsSkpKVFlZqUcffVQXXHCBli1bpksvvVSXXXaZVq5c2eZnlixZovT09OArLy8vnCXBAqt3HtXTy3dKkh6+dIKGD0yxrphDn0q/nSetfFTy10sjzpa+/jfpxhXSKZfSshkAAKCXCGtXNL/fL0lasGCB7rrrLknSlClTtHr1aj377LOaM2dOq8/cf//9WrRoUfB9eXk54SaOHa306I4/bZRhSFdNz9OCKUOsKaS+Tlr1U+n9J8xAk5QpXfQz6ZTLJJvNmpoAAAAQMWENNgMHDpTT6dT48eObbR83bpzef//9Nj/jdrvldsfBDPTolN9vaNGfP9WRCo9GZ/XT4ktOsaaQgxulpbdIJZvN9+MXSBf+XOo3yJp6AAAAEHFhDTYJCQk67bTTtG3btmbbt2/frmHDhoXzqxCDfvvebq3afkRup13PXHOqkhKiPMyr3mPepXnvCcnwSckDpIt+bg45AwAAQK8WcrCprKzUzp07g+8LCwu1ceNGZWZmKj8/X3fffbeuuuoqnX322Zo3b57+/e9/6+9//7tWrFgRzroRYzbsK9VP/2MG2sWXnKIx2anRLaBog/S3Wxsn2DzlUunCn0kpA6NbBwAAACxhMwzDCOUDK1as0Lx581ptX7hwoV544QVJ0u9//3stWbJEBw4c0JgxY/TDH/5QCxYs6NL5y8vLlZ6errKyMqWlpYVSGixSVuPVRU+9pwOlNbp4Uo6e/tpU2aL1HEu9R1r5mPT+kw13aQY23KX5SnS+HwAAABETSjYIOdhEGsEmvhiGoVte3qC3Pi9Wfmay/nH7bKUluqLz5UUbzGdpjmw130+4XCr4qZQyIDrfDwAAgIgKJRuE9Rkb9D1/+Gif3vq8WC6HTU9/bWp0Qo231mzf/MFT5l2alEHSRU9I4y+J/HcDAAAgJhFs0G3bD1fox/8wn2m594KxmpyXEfkvPfSZ9Pq3pCNfmO8nXiEVPC4lZ0b+uwEAABCzCDbotiX/2qq6er/mjRmkb84eEfkv3LtaevlKqa5CSsmSLv5fadzFkf9eAAAAxDyCDbpl/d7jWr7tiBx2mx6af0rkmwXsfEd69VqpvkYaNlu66v/jLg0AAACCCDbolp8v2y5JumLaUA0fmBLZL9v6D+mv10u+OumkL5uhxpUU2e8EAABAXLFbXQDiz+qdR7V61zElOOz6zjmjI/tln/1Z+vPXzVAzfoF09SuEGgAAALRCsEFIDMPQz98279Z87fQ8DcmIYMhY93vp9RvNzmeTr5Eu/73kTIjc9wEAACBuEWwQkhXbjmj93lIluuy6dd5Jkfui1U9L/7hLkiGd9i1pwS8lByMnAQAA0DZ+UkSXGYahny3bJklaOHO4stISI/El0opHzXlqJGn2XdI5D0mRbk4AAACAuEawQZf9Z3OxNh8sV0qCQzfNGRX+LzAMadn3pTXPmO/PeVA663/C/z0AAADodQg26BKf39ATDc/WfHP2CGWmhPlZF79P+uciaf0L5vuCx6UZN4X3OwAAANBrEWzQJf/47KC2H65UWqJT3zxrZHhP7vNKS78tbfqLZLNL85+STr0uvN8BAACAXo1gg07V+/z634a7NTfNGaX0JFcYT+6R/nK9tO2fkt0pXfZbacJl4Ts/AAAA+gSCDTr12oYD2nOsWgNSEvSNWcPDd+K6KunVa6XdyyWH25x48+Tzw3d+AAAA9BkEG3TIU+/TU+/slCR9e+4opbjDdMnUlkkvXynt/1BypUjXvCqNODs85wYAAECfQ7BBh/60dr+KTtRocJpb/3XGsPCc1FsjvfQV6eAGKTFduvY1Ke+08JwbAAAAfRLBBu2qqfPp6XfNuzW3fWm0El2O8Jz43/eboSZ5gHTdUilnUnjOCwAAgD7LbnUBiF1/+HCvjlR4NCQjSVdNzwvPSTe/Ia1/XpJNuvz/CDUAAAAIC4IN2lTpqdevV+6SJN1x7mglOMNwqRwvlN683Vw/a5E0al7PzwkAAACIYIN2PP9+oY5X1WnkwBRdNnVIz09YXyf99QbJUy7lnSHN/V7PzwkAAAA0INiglbJqr557b7ck6c4vnyynIwyXybs/amgWkCFd/jvJweNdAAAACB+CDVr57Xu7VVFbrzGDU3XxxJyen3DH29Lqp831Bb+UMsL0vA4AAADQgGCDZo5VevT7DwolSYvOO1l2u61nJyw/KL1xk7l++k3SuIt7WCEAAADQGsEGzTy7cpeq63yaOCRd540f3LOT+X3S6zdK1cek7EnSeT8OT5EAAABACwQbBB0ur9VLa/ZKkv7nvJNls/Xwbs2qn0p73pNcKdJXn5ec7jBUCQAAALRGsEHQM+/ulKfer+nD+mvOyYN6drI970srHzPXL/5faeBJPS8QAAAAaAfBBpKk/cer9erafZKk754/pmd3a6qOSq/9t2T4pSnXSpOvClOVAAAAQNsINpAkPf3uDnl9hmafNFBnjBzQ/RMZhrT021LFIWngydKFPw1fkQAAAEA7CDbQ7iOVem1DkSSzE1qPrPmltGOZ5HCbz9UkpIShQgAAAKBjBBvoqXd2yOc3dM7YLJ2a37/7JypaL/2/xeb6BY9I2RPCUh8AAADQGYJNH1dW7dW/NhVLku44d3T3T1RbJv3lesnvlcZdIk3/ZpgqBAAAADpHsOnj/rnpkOp8fo3NTtWkoRndO4lhSH+/QzqxV8rIly55Wuppq2gAAAAgBASbPu6NTw5Iki6dOqT7J1n/grT5DcnulC7/vZSUEZbaAAAAgK4i2PRh+49Xa+2eUtls0oIp3Qw2h7dI/77PXP/SD6S808JXIAAAANBFBJs+bOknZie0WaMGKDs9MfQT1FVLf71eqq+VRp0jzbo9zBUCAAAAXUOw6aMMw9AbG81gc+nUod07yXs/l458IfXLli79jWTncgIAAIA1+Em0j/rsQJl2H6lSosuuCyZkh36CqqPSh7821y98XOo3KLwFAgAAACEg2PRRbzQMQztvfLb6uZ2hn+D9/5W8VVLOFLO9MwAAAGAhgk0f5PX59fdPD0rqZje08kPS2t+Z61/6Aa2dAQAAYDmCTR/03o4jOlZVpwEpCTpr9MBunODnZsOAvDOkk84Jf4EAAABAiAg2fdAbn5h3a+ZPzpXTEeIlcGKfOW+NJH3p+9ytAQAAQEwg2PQxFbVeLdtcLEm67NRuDENb+Zjk90oj5kgjzgpzdQAAAED3EGz6mH9/XixPvV8jB6Vo4pD00D58dKe08Y/m+pe+H/7iAAAAgG4i2PQxgW5ol00dIluow8hWPioZPmn0+VLe6RGoDgAAAOgegk0fcqisRmt2H5MkLZgS4jC0w1ukTX8117/0QJgrAwAAAHqGYNOH/G3jQRmGdPrwTOVlJof24RWPSDKk8QuknMkRqQ8AAADoLoJNH7K0YRjapaE2DTj4ibT175Js0tzvhb8wAAAAoIcINn3EloPl+qK4QgkOuy6ckBPah9992FxOulLKGhv+4gAAAIAeItj0EUs3mndrzhmXpfRkV9c/uO9Daefbks0hzbk3QtUBAAAAPUOw6QN8fkN/awg2X5ka4jC0d39iLqdeKw0YFebKAAAAgPAg2PQBa3Yd0+FyjzKSXZo3JqvrH9y9UtrznuRIkM6+J3IFAgAAAD1EsOkDXv/kgCTpook5SnB28Y/cMBrv1ky7XsrIi1B1AAAAQM8RbHq56rp6/efzYknSZaF0Q9vxtnTgY8mZJJ21KELVAQAAAOFBsOnl3t5yWFV1PuVnJuvU/P5d+5BhSO/+2Fw//VtSanbkCgQAAADCgGDTy72+obFpgM1m69qHtr4pFX8mJfSTzrwzcsUBAAAAYRJysFm1apXmz5+v3Nxc2Ww2LV26tN1jb7rpJtlsNj355JM9KBHddaTCo/d2HJEkXdrVbmh+n7T8EXP9jFuklAERqg4AAAAIn5CDTVVVlSZPnqxnnnmmw+OWLl2qjz76SLm5ud0uDj3z5qcH5TekKXkZGjEwpWsf+vw16cgXUmKGNPPWiNYHAAAAhIsz1A8UFBSooKCgw2OKiop022236T//+Y8uuuiibheHnln6iTkMrct3a3xeacUSc/3M26WkjMgUBgAAAIRZyMGmM36/X9ddd53uvvtunXLKKZ0e7/F45PF4gu/Ly8vDXVKftLOkQpuKyuS02zR/chfvmn36R+n4bil5oHT6TZEtEAAAAAijsDcPeOyxx+R0OnX77bd36fglS5YoPT09+MrLY76UcHij4W7N3DGDlJmS0PkH6j3SysfN9bMWSe5+EawOAAAACK+wBpv169frF7/4hV544YUud+C6//77VVZWFnzt378/nCX1SX6/oaWfHJRkdkPrkvUvSmX7pdQcafoNEawOAAAACL+wBpv33ntPJSUlys/Pl9PplNPp1N69e/U///M/Gj58eJufcbvdSktLa/ZCz6zdc1xFJ2qU6nbq3HGDO/9AXbX03s/M9bO/K7mSIlsgAAAAEGZhfcbmuuuu07nnntts2/nnn6/rrrtO119/fTi/Ch0IDEMrmJitRJej8w+s/Z1UeVjKyJemfj3C1QEAAADhF3Kwqays1M6dO4PvCwsLtXHjRmVmZio/P18DBjSf98Tlcik7O1tjxozpebXoVK3Xp39uOiRJunTq0M4/4PdJH/7KXJ9zr+TswvM4AAAAQIwJOdisW7dO8+bNC75ftGiRJGnhwoV64YUXwlYYuufdL0pUUVuv3PREzRiR2fkHCldJFYfMeWsmXhHx+gAAAIBICDnYzJ07V4ZhdPn4PXv2hPoV6IHAMLQFU4fIbu9CA4fP/mQuT7lUcrojWBkAAAAQOWFv9wzrlFbVacW2EkldnJSzrkra+ndzffLVEawMAAAAiCyCTS/yj02H5PUZOiU3TScPTu38A1/8S6qrlDKGSXkzIl8gAAAAECEEm17kjQ0HJHXxbo3UOAxt0lVSF+cdAgAAAGIRwaaX2H+8Whv2nZDdJl0yObfzD1SWSLveNdcnXRXZ4gAAAIAII9j0Eu/tOCpJmj4sU1lpiZ1/4PPXJMMnDZkmDTwpwtUBAAAAkUWw6SXW7D4mSZo5akAnRzb49FVzOYmmAQAAAIh/BJtewDAMfdgQbM4Y2YVgc2SbdGijZHdKEy6LbHEAAABAFBBseoFdR6p0pMKjBKddU/MzOv9AoGnASedKKQMjWhsAAAAQDQSbXiAwDG1afn8luhwdH+z3S5/9xVyfdGWEKwMAAACig2DTC3y4K4Tna/atkcr2SQmp0pgLI1wZAAAAEB0EmzgX8vM1gWFo4xdIrqQIVgYAAABED8Emzu0oqdSxqjoluuyanJfe8cHeWmnzUnN9MnPXAAAAoPcg2MS5NQ3D0KYPy5Tb2cnzNTv+I3nKpLQh0rDZUagOAAAAiA6CTZxbE8rzNZ82DEObeIVk548eAAAAvQc/3cYxv9/QR4WB52syOz64+ri0Y5m5PolhaAAAAOhdCDZxbNvhCpVWe5Wc4NCkoRkdH7z5dcnvlbInSoPHR6U+AAAAIFoINnEs+HzN8Ey5HJ38UX72Z3M56eoIVwUAAABEH8EmjgUm5pzZWZvn47ul/R9JNrs08atRqAwAAACILoJNnPL7DX1ceFxSF56v+ewv5nLkXCk1O7KFAQAAABYg2MSpLYfKVVbjVT+3UxOHdDB/jWFIn71qrtM0AAAAAL0UwSZOfdgwDO204f3l7Oj5mqL15lA0V7I09uIoVQcAAABEF8EmTnV5/ppPG+7WjL1YcveLcFUAAACANQg2ccjX7PmaDoJNfZ30+Wvm+mSGoQEAAKD3ItjEoc0Hy1ThqVdqolOn5HbwfM2ud6Sa41JKljRibrTKAwAAAKKOYBOHAsPQZozIlMNua//AwDC0iVdIDmcUKgMAAACsQbCJQ4H5azochlZbJm17y1yfdGUUqgIAAACsQ7CJM16fX2sbnq/psHHAlr9JPo80aKyUMzlK1QEAAADWINjEmc+LylRV51N6kkvjstPaP/CzP5vLSVdKtg6GqwEAAAC9AMEmzgSGoc0YkSl7e8/XnNgv7XnPXJ/IMDQAAAD0fgSbONOl+Ws2/cVcDpstZeRFoSoAAADAWgSbOFJX79e6PaWSOgg2hiF99idznblrAAAA0EcQbOLIpqITqvH61D/ZpZOzUts+qPgz6cgXksMtjV8Q3QIBAAAAixBs4khgGNoZIwe0/3zNpw13a8YUSIkdTN4JAAAA9CIEmzgSaBzQ7jA0X730+V/N9clXR6kqAAAAwHoEmzjhqfc1Pl/T3sSchSukysNSUqY06pzoFQcAAABYjGATJz7dXyZPvV8D+yXopKx+bR8UmLtmwuWSMyF6xQEAAAAWI9jEicDzNTNGDpCtrQk3PZXS1r+b65PohgYAAIC+hWATJ9bsPiqpo2FoqyRvtZQxTBo6PYqVAQAAANYj2MSBWq9PG/adkNRB44A975nLUfOktu7oAAAAAL0YwSYOfLLvhOrq/RqU6tbIgSltHxQINsPPil5hAAAAQIwg2MSBYJvn9p6vqT4uFX9urhNsAAAA0AcRbOLAh7s6mb9m7weSDGngGCl1cPQKAwAAAGIEwSbG1dT59Mn+zuavCQxDmx2lqgAAAIDYQrCJcRv2lcrrM5SdlqhhA5LbPmjP++ZyBMPQAAAA0DcRbGLcmibD0Np8vqbqqFSy2Vzn+RoAAAD0UQSbGNe0cUCbAndrssZLKQOjVBUAAAAQWwg2MazKU69P95+Q1IX5a3i+BgAAAH0YwSaGrd9bqnq/oSEZSRraP6ntgwJ3bBiGBgAAgD6MYBPDAsPQzmhv/prKEunIF5Js3LEBAABAn0awiWFrOpu/JjAMbfAEKTkzSlUBAAAAsYdgE6MqPfXaVFQmSTpjZDuhhflrAAAAAEkEm5i1tvC4fH5DeZlJGtqf+WsAAACAjhBsYtSHnbV5Lj8kHdshySYNmxW9wgAAAIAYRLCJUcH5a9p9vqbhbk3OJCmpf5SqAgAAAGITwSYGldd69Xnw+Zr2gs0qc0mbZwAAAIBgE4s+3n1cfkMaPiBZOenMXwMAAAB0JuRgs2rVKs2fP1+5ubmy2WxaunRpcJ/X69W9996riRMnKiUlRbm5ufr617+ugwcPhrPmXu/DzoahlRVJx3dLNrs0bGYUKwMAAABiU8jBpqqqSpMnT9YzzzzTal91dbU2bNigH/zgB9qwYYNef/11bd++XZdccklYiu0rmk7M2abA/DU5U6TE9OgUBQAAAMQwZ6gfKCgoUEFBQZv70tPT9fbbbzfb9vTTT+v000/Xvn37lJ+f3+ozHo9HHo8n+L68vDzUknqVE9V12nLI/D1otyNaYP4a2jwDAAAAkqLwjE1ZWZlsNpsyMjLa3L9kyRKlp6cHX3l5eZEuKaZ9VHhchiGNGpSirLTEtg8K3LHh+RoAAABAUoSDTW1tre677z5dc801SktLa/OY+++/X2VlZcHX/v37I1lSzPu48LikDoahndgnndgr2RxS/hlRrAwAAACIXSEPResqr9erq6++Wn6/X7/61a/aPc7tdsvtdkeqjLiz5aA5DG1yXkbbBwSGoQ05VXKnRqcoAAAAIMZFJNh4vV5deeWVKiws1Lvvvtvu3Ro0ZxiGth2ukCSNy27n94xhaAAAAEArYQ82gVCzY8cOLV++XAMGtDOkCq0cqfToeFWd7DZp9OB+rQ8wjMY7NsNnR7c4AAAAIIaFHGwqKyu1c+fO4PvCwkJt3LhRmZmZys3N1Ve/+lVt2LBB//jHP+Tz+VRcXCxJyszMVEJCQvgq74W+OGTerRk+IEWJLkfrA0r3SOUHJLuL52sAAACAJkIONuvWrdO8efOC7xctWiRJWrhwoRYvXqw333xTkjRlypRmn1u+fLnmzp3b/Ur7gG3FZrAZm9POszOBYWhDpkkJKVGqCgAAAIh9IQebuXPnyjCMdvd3tA8d21psNg4YM7id52uYvwYAAABoU8TnsUHXBe7YjMlu446NYTRpHMDzNQAAAEBTBJsYUe/za0dJpSRpXFtD0Y7vlioOSY4EKW9GlKsDAAAAYhvBJkbsOValunq/khMcyuuf3PqAwlXmcuhpkispusUBAAAAMY5gEyO+aBiGNnpwqux2W+sDmL8GAAAAaBfBJkYEnq8Z197zNcxfAwAAALSLYBMjth7qoHHA0R1SVYnkcJtD0QAAAAA0Q7CJEdsON7R6bivY7Gl4vibvdMmVGMWqAAAAgPhAsIkBlZ567T9eI0kam93GHDbB+WvOjmJVAAAAQPwg2MSAwPM1WaluZaYkNN9pGNKe9811GgcAAAAAbSLYxIAOJ+Y88oVUfVRyJklDTo1yZQAAAEB8INjEgG3F5vM143I6GIaWP0NyuqNYFQAAABA/CDYxYGvgjs3gDhoHMAwNAAAAaBfBxmKGYbQ/FM3vb3y+hsYBAAAAQLsINhY7XO5RWY1XDrtNJ2X1a76zZItUUyq5UqTcqdYUCAAAAMQBgo3FtjY8XzNiYIoSXY7mO/cEnq85Q3K4olwZAAAAED8INhbrsCNacP4anq8BAAAAOkKwsVgg2Ixr9XyNT9obmL+G52sAAACAjhBsLLb1kDkUbUx2i1bPhz+XasukhFQpZ7IFlQEAAADxg2BjIa/Pr11HKiVJY1vesQkMQxs2U3I4o1wZAAAAEF8INhYqPFolr89QP7dTQzKSmu8MNA5g/hoAAACgUwQbCwWGoZ08uJ/sdlvjDl+9tHe1uU7jAAAAAKBTBBsLNXZEa/F8TfGnkqdccqdL2ZMsqAwAAACILwQbCwU7ouW0eL5mT0M3tGGzJHuLuW0AAAAAtEKwsdAXgTs2g9tpHMAwNAAAAKBLCDYWKa/1quhEjSRpbNOhaD6vtG+NuU7jAAAAAKBLCDYW2d5wtyYnPVHpya7GHQc3SnWVUmKGNHiCJbUBAAAA8YZgY5GtwcYBLZ+vCbR5ni3Z+eMBAAAAuoKfnC2yrdhs9dwq2BzcYC7zz4hyRQAAAED8IthYJNgRrWWr58NbzCXD0AAAAIAuI9hYwDCMxo5oTe/Y1FVJx3eb64NPsaAyAAAAID4RbCxwsKxWFbX1ctptGjWoX+OOI19IMqTkgVK/LMvqAwAAAOINwcYCgedrRg3qpwRnkz+C4DA07tYAAAAAoSDYWGDroXY6oh3ebC4JNgAAAEBICDYW2NZeq+cSgg0AAADQHQQbCwQ7ouU0CTaG0XjHJmu8BVUBAAAA8YtgE2V19X7tOlIpSRrTtNVzZYlUfUySTRo01priAAAAgDhFsImyXUcqVe83lJroVG56YuOOwDC0AaOkhGRrigMAAADiFMEmygLD0MZmp8pmszXuCHREYxgaAAAAEDKCTZRtbWj1TEc0AAAAIHwINlHW2BEtrfkOOqIBAAAA3UawibJgR7Smd2x89dKRbeY6Q9EAAACAkBFsoqis2qtDZbWSpJObBpvju6X6WsmVLPUfYVF1AAAAQPwi2ETRFw3P1wzJSFJaoqtxR2AYWtY4yc4fCQAAABAqfoqOom2HGzuiNUNHNAAAAKBHCDZRtPVQoHEAHdEAAACAcCLYRNG29lo90xENAAAA6BGCTZT4/Ya2H66UJI3LadLq2VMple4x17MINgAAAEB3EGyipOhEjSo99XI5bBoxMKVxR8lWc9lvsJQywJriAAAAgDhHsImSLxrmrxk1qJ9cjia/7QxDAwAAAHqMYBMlgedrmg1Dk+iIBgAAAIQBwSZKthbTEQ0AAACIFIJNlGxrK9gYBkPRAAAAgDAg2ERBrdenwqNVkqRx2U2GolUUSzWlks0hDRxjUXUAAABA/CPYRMHOkkr5/IbSk1wanOZu3BEYhjZglORKtKY4AAAAoBcg2ERB02FoNputcQfD0AAAAICwCDnYrFq1SvPnz1dubq5sNpuWLl3abL9hGFq8eLFyc3OVlJSkuXPnavPmzeGqNy59EeiI1qpxQKAjGsEGAAAA6ImQg01VVZUmT56sZ555ps39jz/+uJ544gk988wzWrt2rbKzs/XlL39ZFRUVPS42Xn0RvGPTstVz4I4NrZ4BAACAnnCG+oGCggIVFBS0uc8wDD355JN64IEHdNlll0mSXnzxRQ0ePFivvPKKbrrppp5VG6fa7Ijm80pHt5nrDEUDAAAAeiSsz9gUFhaquLhY5513XnCb2+3WnDlztHr16jY/4/F4VF5e3uzVmxyvqlNJhUdSi2BzbKfkq5MS+knp+RZVBwAAAPQOYQ02xcXFkqTBgwc32z548ODgvpaWLFmi9PT04CsvLy+cJVku8HxNXmaS+rmb3CALDEPLGifZ6eEAAAAA9EREfqJu1vlL5hC1ltsC7r//fpWVlQVf+/fvj0RJlgkOQxvc4vmakobGAQxDAwAAAHos5GdsOpKdnS3JvHOTk5MT3F5SUtLqLk6A2+2W2+1uc19v8MUhM9iMy2nZES1wx4ZgAwAAAPRUWO/YjBgxQtnZ2Xr77beD2+rq6rRy5UrNmjUrnF8VN7443EbjAKmx1TMd0QAAAIAeC/mOTWVlpXbu3Bl8X1hYqI0bNyozM1P5+fm688479cgjj2j06NEaPXq0HnnkESUnJ+uaa64Ja+HxwO83tL1hKNrYpsGmtkwq22euZxFsAAAAgJ4KOdisW7dO8+bNC75ftGiRJGnhwoV64YUXdM8996impka33HKLSktLNWPGDC1btkypqantnbLX2ne8WjVenxKcdg0fkNK4o2SruUzNlZIzrSkOAAAA6EVCDjZz586VYRjt7rfZbFq8eLEWL17ck7p6hcDEnKOz+snpaDLqj4k5AQAAgLCiz3AEtTkxp0RHNAAAACDMCDYRFJjDZlx2i1bPdEQDAAAAwopgE0Ft3rExDDqiAQAAAGFGsImQmjqf9hyrktSiI1p5keQpk+xOaeDJFlUHAAAA9C4EmwjZUVIhvyFlpiRoUGqTCUgDw9AGjJacvXdiUgAAACCaCDYREuiINmZwqmw2W+MOOqIBAAAAYUewiRA6ogEAAADRQ7CJkGBHtJwWwYaOaAAAAEDYEWwipPGOTZNWz/V10tHt5jpD0QAAAICwIdhEQKWnXkcr6yRJIwelNO44tkPy10vuNCk9z6LqAAAAgN6HYBMBRaU1kqS0RKfSEl2NO4LD0MZLTRsKAAAAAOgRgk0EFJ2oliQN7Z/cfAcd0QAAAICIINhEwIGGOzZD+yc130FHNAAAACAiCDYR0Bhs2rljQ0c0AAAAIKwINhEQeMZmSNM7NjWlUnmRuZ41zoKqAAAAgN6LYBMBB0oDz9g0CTYlW81lep6UlBH9ogAAAIBejGATAW0+Y9O0IxoAAACAsCLYhFlNnU/Hqsw5bIZmNHnGho5oAAAAQMQQbMIs0Oo51e1UWpKzcUewI9oEC6oCAAAAejeCTZjtb9I4wBaYhNMwpMMNwYahaAAAAEDYEWzCrKit52tO7JPqKiS7Sxo42qLKAAAAgN6LYBNmbc5hExiGNmiM5HBZUBUAAADQuxFswqzNVs+HPzeXDEMDAAAAIoJgE2ZFJxqescloGmwCjQMINgAAAEAkEGzCrMOhaHREAwAAACKCYBNGtV6fjlR4JDUZilbvkY7uMNcZigYAAABEBMEmjA42DENLTnAoI7mhScCRbZLhkxLTpbRcC6sDAAAAei+CTRgdaNLqOTiHTdNhaIFtAAAAAMKKYBNGbT5fQ0c0AAAAIOIINmFUdMJs9UxHNAAAACC6CDZh1HQoWhAd0QAAAICII9iEUauhaNXHpYpD5nrWOIuqAgAAAHo/gk0YFTUEmyGBOzaHN5vLjHzJnWpRVQAAAEDvR7AJE0+9T4craiU1GYrGMDQAAAAgKgg2YXLoRK0MQ0p02TUgJcHcSEc0AAAAICoINmFS1DA555CMJnPY0BENAAAAiAqCTZgcKDVbPQcbB/j9UslWc52haAAAAEBEEWzCpFWr5xN7JG+V5HBLmaOsKwwAAADoAwg2YdK6I1rDMLRBJ0sOp0VVAQAAAH0DwSZMWs1hQ0c0AAAAIGoINmHS+IxN4I4NHdEAAACAaCHYhIHX51dxecMcNhkthqLREQ0AAACIOIJNGBSX1cpvSAlOuwb2c0u+eun4bnPnoHHWFgcAAAD0AQSbMNgfGIaWkSS73SaVF0mGT3IkSKk5FlcHAAAA9H4EmzBo1RHtxD5zmZ4n2fktBgAAACKNn7rDoNUcNmX7zWVGnkUVAQAAAH0LwSYMWrV6Dtyxyci3qCIAAACgbyHYhEHRCfMZmyEZLYaiEWwAAACAqCDYhEGroWjBYDPMoooAAACAvoVg00P1Pr8OlTXMYdNyKFo6z9gAAAAA0UCw6aHDFR75/IZcDpuyUhvmsCkvMncyFA0AAACICoJNDx04bj5fkxuYw6bikOSvl+wuKTXb4uoAAACAvoFg00PtPl+TPkSyOyyqCgAAAOhbCDY9VHSiYXLOjJZz2DAMDQAAAIgWgk0PHSg1h6Ixhw0AAABgnbAHm/r6en3/+9/XiBEjlJSUpJEjR+pHP/qR/H5/uL8qJrQeirbXXNLqGQAAAIgaZ7hP+Nhjj+nZZ5/Viy++qFNOOUXr1q3T9ddfr/T0dN1xxx3h/jrLtRqKRqtnAAAAIOrCHmzWrFmjBQsW6KKLLpIkDR8+XH/84x+1bt26cH+V5Xx+Qwcbgs3QzMBQNJ6xAQAAAKIt7EPRZs+erXfeeUfbt2+XJH366ad6//33deGFF7Z5vMfjUXl5ebNXvCipqJXXZ8hpt2lwqlvy+6SyA+ZOgg0AAAAQNWG/Y3PvvfeqrKxMY8eOlcPhkM/n08MPP6yvfe1rbR6/ZMkS/fCHPwx3GVFR1PB8TU5GopwOu1RWJPm9kt0ppeZYXB0AAADQd4T9js2f/vQn/eEPf9Arr7yiDRs26MUXX9TPfvYzvfjii20ef//996usrCz42r9/f7hLiphA44BWrZ7TciVH2DMjAAAAgHaE/afvu+++W/fdd5+uvvpqSdLEiRO1d+9eLVmyRAsXLmx1vNvtltvtDncZUdF+q2c6ogEAAADRFPY7NtXV1bLbm5/W4XD0ynbPgY5orVs983wNAAAAEE1hv2Mzf/58Pfzww8rPz9cpp5yiTz75RE888YRuuOGGcH+V5VoNRWNyTgAAAMASYQ82Tz/9tH7wgx/olltuUUlJiXJzc3XTTTfpwQcfDPdXWa5xcs4WrZ6ZwwYAAACIqrAHm9TUVD355JN68sknw33qmOL3G20MReOODQAAAGCFsD9j01ccrfSort4vu03KTk+U/P7GrmgEGwAAACCqCDbdtD8wh016klwOu1R5WPLVSTaHlDbE4uoAAACAvoVg002BYWhD+jOHDQAAAGA1gk03BeewoSMaAAAAYDmCTTc1dkRjDhsAAADAagSbbiqi1TMAAAAQMwg23RQYijaEVs8AAACA5Qg23WAYzGEDAAAAxBKCTTccq6pTrdcvm81s9yzDYA4bAAAAwEIEm24INA4YnJqoBKddqjoi1ddKsjGHDQAAAGABgk03BFs9txyGlpYrORMsqgoAAADouwg23VBEq2cAAAAgphBsuiEwFI2OaAAAAEBsINh0Q+NQNOawAQAAAGIBwaYbaPUMAAAAxBaCTYgMw2gcipZBsAEAAABiAcEmRKXVXlXX+SRJuRnMYQMAAADEAoJNiAId0bJS3Up0OaTqY5LXfOZG6UMtrAwAAADouwg2IQo0DhjSstVzao7kdFtUFQAAANC3EWxCdCA4h02gIxrP1wAAAABWI9iEqHVHNFo9AwAAAFYj2IQoOBSNjmgAAABAzCDYhKhxKBrBBgAAAIgVBJsQGIYR7IrGMzYAAABA7CDYhKC8pl4VnnpJDUPRmMMGAAAAiAkEmxDsb3i+ZmC/BCUlOKSaUqmu0tzJHDYAAACAZQg2IQh0RBsSHIbWMIdNv8GSK8miqgAAAAAQbEIQbBxARzQAAAAgphBsQhBo9cwcNgAAAEBsIdiEoIhWzwAAAEBMItiEIDAUbQjBBgAAAIgpBJsQNA5Fa2geEGz1PMyiigAAAABIBJsuK6/1qry2xRw2wTs2PGMDAAAAWIlg00WB52v6J7uU4nZKtSckT7m5k+YBAAAAgKUINl0UbPUcnMOm4W5NyiApIdmiqgAAAABIBJsuK2qv1TONAwAAAADLEWy6KNgRreXknAxDAwAAACxHsOmiA8xhAwAAAMQsgk0XFZ1o5xkbgg0AAABgOYJNFwXmsAlOzllGsAEAAABiBcGmC6o89Sqt9kpqEmy4YwMAAADEDIJNFwSGoaUnuZSW6JJqy8yXRPMAAAAAIAYQbLrgQHutnpMHSO5+FlUFAAAAIIBg0wW0egYAAABiG8GmC4pK6YgGAAAAxDKCTRcwhw0AAAAQ2wg2XdB+q+dhFlUEAAAAoCmCTRc0Ts7Z8o4Nz9gAAAAAsYBg04maOp+OVtZJ4hkbAAAAIFYRbDpRdMIchpbqdio9ySV5KqSaUnMnXdEAAACAmECw6USw1XPLOWyS+kuJaRZVBQAAAKApgk0nDrTX6pm7NQAAAEDMINh0glbPAAAAQOwj2HSiVUc0Wj0DAAAAMYdg04nAHDbcsQEAAABiV0SCTVFRkf7rv/5LAwYMUHJysqZMmaL169dH4qsiLtg8IKNlq2eesQEAAABihTPcJywtLdWZZ56pefPm6a233lJWVpZ27dqljIyMcH9VxNV6fTpS4ZHEHRsAAAAgloU92Dz22GPKy8vT888/H9w2fPjwcH9NVBxseL4mJcGhjGSXVFclVR8zd9IVDQAAAIgZYR+K9uabb2r69Om64oorlJWVpalTp+q3v/1tu8d7PB6Vl5c3e8WKpnPY2Gy2xjls3OlSUoZ1hQEAAABoJuzBZvfu3fr1r3+t0aNH6z//+Y9uvvlm3X777XrppZfaPH7JkiVKT08PvvLyYudOSGNHtJbP1zAMDQAAAIglNsMwjHCeMCEhQdOnT9fq1auD226//XatXbtWa9asaXW8x+ORx+MJvi8vL1deXp7KysqUlpYWztJCVuv16UBptQxDGj04Vfr4t9K/viuNuUj62iuW1gYAAAD0duXl5UpPT+9SNgj7MzY5OTkaP358s23jxo3Ta6+91ubxbrdbbrc73GWERaLLoZOyUhs3lDUMReOODQAAABBTwj4U7cwzz9S2bduabdu+fbuGDesFE1rS6hkAAACISWEPNnfddZc+/PBDPfLII9q5c6deeeUVPffcc7r11lvD/VXRxzM2AAAAQEwKe7A57bTT9MYbb+iPf/yjJkyYoB//+Md68sknde2114b7q6KPYAMAAADEpLA/YyNJF198sS6++OJInNo63hqp6oi5TrABAAAAYkrY79j0WoE5bBJSpcQMS0sBAAAA0BzBpquaDkOz2aytBQAAAEAzBJuuKuP5GgAAACBWEWy6isYBAAAAQMwi2HQVc9gAAAAAMYtg01XcsQEAAABiFsGmqwJd0Qg2AAAAQMwh2HSFt1aqLDbXM4ZZWwsAAACAVgg2XVF2wFy6UqSk/tbWAgAAAKAVgk1XnNhrLpnDBgAAAIhJBJuuKOP5GgAAACCWEWy6glbPAAAAQEwj2HQFrZ4BAACAmEaw6QpaPQMAAAAxjWDTFdyxAQAAAGIawaYz9R6p4pC5nk6wAQAAAGIRwaYzZQckGZIzSUoZaHU1AAAAANpAsOlM01bPzGEDAAAAxCSCTWd4vgYAAACIeQSbzjCHDQAAABDzCDad4Y4NAAAAEPMINp1hDhsAAAAg5hFsOhO8YzPM2joAAAAAtItg0xGfV6o4aK6n84wNAAAAEKsINh0pL5IMv+RMlPplWV0NAAAAgHYQbDoSGIaWnsccNgAAAEAMI9h0hFbPAAAAQFxwWl1ATDvpXOlrr0quZKsrAQAAANABgk1HUrOlMQVWVwEAAACgEwxFAwAAABD3CDYAAAAA4h7BBgAAAEDcI9gAAAAAiHsEGwAAAABxj2ADAAAAIO4RbAAAAADEPYINAAAAgLhHsAEAAAAQ9wg2AAAAAOIewQYAAABA3CPYAAAAAIh7BBsAAAAAcY9gAwAAACDuEWwAAAAAxD2CDQAAAIC4R7ABAAAAEPecVhfQkmEYkqTy8nKLKwEAAABgpUAmCGSEjsRcsKmoqJAk5eXlWVwJAAAAgFhQUVGh9PT0Do+xGV2JP1Hk9/t18OBBpaamymazWV2OysvLlZeXp/379ystLc3qchBHuHbQE1w/6AmuH/QE1w96ItzXj2EYqqioUG5uruz2jp+iibk7Nna7XUOHDrW6jFbS0tL4jxvdwrWDnuD6QU9w/aAnuH7QE+G8fjq7UxNA8wAAAAAAcY9gAwAAACDuEWw64Xa79dBDD8ntdltdCuIM1w56gusHPcH1g57g+kFPWHn9xFzzAAAAAAAIFXdsAAAAAMQ9gg0AAACAuEewAQAAABD3CDYAAAAA4h7BBgAAAEDcI9h04Fe/+pVGjBihxMRETZs2Te+9957VJSEGrVq1SvPnz1dubq5sNpuWLl3abL9hGFq8eLFyc3OVlJSkuXPnavPmzdYUi5iyZMkSnXbaaUpNTVVWVpa+8pWvaNu2bc2O4fpBe379619r0qRJwdm9Z86cqbfeeiu4n2sHoViyZIlsNpvuvPPO4DauIbRn8eLFstlszV7Z2dnB/VZdOwSbdvzpT3/SnXfeqQceeECffPKJzjrrLBUUFGjfvn1Wl4YYU1VVpcmTJ+uZZ55pc//jjz+uJ554Qs8884zWrl2r7OxsffnLX1ZFRUWUK0WsWblypW699VZ9+OGHevvtt1VfX6/zzjtPVVVVwWO4ftCeoUOH6tFHH9W6deu0bt06felLX9KCBQuCPzxw7aCr1q5dq+eee06TJk1qtp1rCB055ZRTdOjQoeBr06ZNwX2WXTsG2nT66acbN998c7NtY8eONe677z6LKkI8kGS88cYbwfd+v9/Izs42Hn300eC22tpaIz093Xj22WctqBCxrKSkxJBkrFy50jAMrh+Ern///sbvfvc7rh10WUVFhTF69Gjj7bffNubMmWPccccdhmHw9w869tBDDxmTJ09uc5+V1w53bNpQV1en9evX67zzzmu2/bzzztPq1astqgrxqLCwUMXFxc2uJbfbrTlz5nAtoZWysjJJUmZmpiSuH3Sdz+fTq6++qqqqKs2cOZNrB11266236qKLLtK5557bbDvXEDqzY8cO5ebmasSIEbr66qu1e/duSdZeO86Inj1OHT16VD6fT4MHD262ffDgwSouLraoKsSjwPXS1rW0d+9eK0pCjDIMQ4sWLdLs2bM1YcIESVw/6NymTZs0c+ZM1dbWql+/fnrjjTc0fvz44A8PXDvoyKuvvqoNGzZo7dq1rfbx9w86MmPGDL300ks6+eSTdfjwYf3kJz/RrFmztHnzZkuvHYJNB2w2W7P3hmG02gZ0BdcSOnPbbbfps88+0/vvv99qH9cP2jNmzBht3LhRJ06c0GuvvaaFCxdq5cqVwf1cO2jP/v37dccdd2jZsmVKTExs9ziuIbSloKAguD5x4kTNnDlTo0aN0osvvqgzzjhDkjXXDkPR2jBw4EA5HI5Wd2dKSkpapU+gI4EOIVxL6Mh3vvMdvfnmm1q+fLmGDh0a3M71g84kJCTopJNO0vTp07VkyRJNnjxZv/jFL7h20Kn169erpKRE06ZNk9PplNPp1MqVK/XUU0/J6XQGrxOuIXRFSkqKJk6cqB07dlj69w/Bpg0JCQmaNm2a3n777Wbb3377bc2aNcuiqhCPRowYoezs7GbXUl1dnVauXMm1BBmGodtuu02vv/663n33XY0YMaLZfq4fhMowDHk8Hq4ddOqcc87Rpk2btHHjxuBr+vTpuvbaa7Vx40aNHDmSawhd5vF4tHXrVuXk5Fj69w9D0dqxaNEiXXfddZo+fbpmzpyp5557Tvv27dPNN99sdWmIMZWVldq5c2fwfWFhoTZu3KjMzEzl5+frzjvv1COPPKLRo0dr9OjReuSRR5ScnKxrrrnGwqoRC2699Va98sor+tvf/qbU1NTgv26lp6crKSkpOKcE1w/a8r3vfU8FBQXKy8tTRUWFXn31Va1YsUL//ve/uXbQqdTU1ODzfAEpKSkaMGBAcDvXENrz3e9+V/Pnz1d+fr5KSkr0k5/8ROXl5Vq4cKG1f/9EtOdanPvlL39pDBs2zEhISDBOPfXUYAtWoKnly5cbklq9Fi5caBiG2fbwoYceMrKzsw23222cffbZxqZNm6wtGjGhretGkvH8888Hj+H6QXtuuOGG4P+jBg0aZJxzzjnGsmXLgvu5dhCqpu2eDYNrCO276qqrjJycHMPlchm5ubnGZZddZmzevDm436prx2YYhhHZ6AQAAAAAkcUzNgAAAADiHsEGAAAAQNwj2AAAAACIewQbAAAAAHGPYAMAAAAg7hFsAAAAAMQ9gg0AAACAuEewAQAAABD3CDYAAAAA4h7BBgAAAEDcI9gAAAAAiHv/PxTklcK+7Y19AAAAAElFTkSuQmCC",
      "text/plain": [
       "<Figure size 1000x600 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Transition probabilities\n",
    "alpha = 0.01\n",
    "beta = 0.2\n",
    "\n",
    "# Discount parameter\n",
    "gamma = 0.7\n",
    "\n",
    "# Expected rewards\n",
    "r_search = 6.0\n",
    "r_wait = 5.0\n",
    "\n",
    "V = np.zeros(nb_states)\n",
    "Q = np.zeros((nb_states, nb_actions))\n",
    "\n",
    "pi = np.array([a['search'], a['search']], dtype=int)\n",
    "\n",
    "V_high_history = []\n",
    "V_low_history = []\n",
    "\n",
    "for k in range(50):\n",
    "    \n",
    "    # Policy evaluation\n",
    "    Q[s['high'], a['search']] = alpha * (r_search + gamma * V[s['high']]) + (1 - alpha) * (r_search + gamma*V[s['low']])\n",
    "    Q[s['high'], a['wait']] = r_wait + gamma * V[s['high']]\n",
    "\n",
    "    Q[s['low'], a['search']] = beta * (r_search + gamma * V[s['low']]) + (1 - beta) * (-3 + gamma*V[s['high']])\n",
    "    Q[s['low'], a['wait']] = r_wait + gamma * V[s['low']]\n",
    "    Q[s['low'], a['recharge']] = gamma * V[s['high']]\n",
    "\n",
    "    V[s['high']] = Q[s['high'], :2].max()\n",
    "    V[s['low']] = Q[s['low'], :].max()\n",
    "\n",
    "    V_high_history.append(V[s['high']])\n",
    "    V_low_history.append(V[s['low']])\n",
    "        \n",
    "    # Compute the greedy policy\n",
    "    pi_old = pi.copy()\n",
    "    pi[s['high']] = Q[s['high'], :2].argmax()\n",
    "    pi[s['low']] = Q[s['low'], :].argmax()\n",
    "    \n",
    "print('Greedy policy after iteration', k)\n",
    "print('pi(high)=', pi[s['high']])\n",
    "print('pi(low)=', pi[s['low']])\n",
    "print(\"V=\", V)\n",
    "print(\"Q=\", Q)\n",
    "    \n",
    "plt.figure(figsize=(10, 6))  \n",
    "plt.plot(V_high_history, label=\"high\")\n",
    "plt.plot(V_low_history, label=\"low\")\n",
    "plt.legend()\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**A:** The agent now decides to wait in the low state and accumulate quite a lot of rewards (r=5, compared to 6 while searching). But it is still worth searching in the high state: even if we transition immediately into low with 99% probability, one still gets 6 instead of 5, so the return is higher than when waiting in high."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Q:** Find a set of parameters where it would be optimal to search while in the *low* state."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Greedy policy after iteration 49\n",
      "pi(high)= 0\n",
      "pi(low)= 0\n",
      "V= [28.03236199 25.7375694 ]\n",
      "Q= [[28.03236199 24.62265325  0.        ]\n",
      " [25.7375694  23.01629844 19.62265325]]\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAzYAAAH5CAYAAABTbqsJAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjYuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/av/WaAAAACXBIWXMAAA9hAAAPYQGoP6dpAABKiElEQVR4nO3deXxU9b3/8fdsmSwkIQlkgwQCIsgiqChCFWKtVGq9WrzdbKu2vW1t0VvL7c+6tJX2V8F6W2/bH63W3lvUW6m2da11oyqg4AIIgmyCLIlAWLLvmcyc3x9nZsiemWRmzkzyej4e85gz55yZ+QCnMe+e7/fztRmGYQgAAAAAEpjd6gIAAAAAYLAINgAAAAASHsEGAAAAQMIj2AAAAABIeAQbAAAAAAmPYAMAAAAg4RFsAAAAACQ8p9UFdOXz+XT06FGlp6fLZrNZXQ4AAAAAixiGofr6ehUWFspu7/ueTNwFm6NHj6qoqMjqMgAAAADEifLyco0dO7bPc+Iu2KSnp0syi8/IyLC4GgAAAABWqaurU1FRUTAj9CXugk1g+FlGRgbBBgAAAEBIU1RoHgAAAAAg4RFsAAAAACQ8gg0AAACAhBd3c2xC5fV65fF4rC4jobhcLjkcDqvLAAAAACIu4YKNYRiqqKhQTU2N1aUkpJEjRyo/P581ggAAADCkJFywCYSa3Nxcpaam8gt6iAzDUFNTk06cOCFJKigosLgiAAAAIHISKth4vd5gqMnJybG6nISTkpIiSTpx4oRyc3MZlgYAAIAhI6GaBwTm1KSmplpcSeIK/N0xPwkAAABDSUIFmwCGnw0cf3cAAAAYihIy2AAAAABARwSbGCktLdUtt9zS63Gbzaann3465M9bu3atbDYb3eEAAAAAJVjzgKHs2LFjysrKsroMAAAAICERbOJEfn6+1SUAAAAACYuhaDHk8/l06623Kjs7W/n5+Vq2bFnwWNehaBs3btSsWbOUnJys2bNn6+mnn5bNZtO2bds6feaWLVs0e/Zspaamat68edq7d29s/jAAAABAHEn4OzaGYajZ47Xku1NcjrC6jD388MNaunSp3n77bb355pu64YYb9LGPfUyXXXZZp/Pq6+t15ZVX6lOf+pRWr16tw4cP9zo/584779Qvf/lLjR49WjfeeKO+9rWvacOGDYP5YwEAAAAJJ+GDTbPHq6k/fsmS7971008qNSn0v8Kzzz5bd911lyRp0qRJWrlypV555ZVuwebRRx+VzWbTH/7wByUnJ2vq1Kk6cuSIvvGNb3T7zLvvvlsLFiyQJN1222264oor1NLSouTk5EH8yQAAAIDEwlC0GDr77LM7vS4oKNCJEye6nbd3716dffbZncLJBRdc0O9nFhQUSFKPnwkAAAAMZQl/xybF5dCun37Ssu8Oh8vl6vTaZrPJ5/N1O88wjG5D3AzD6PczA+/p6TMB9M4wDHl9htoDD6/P/2zI4/X5j/nk9Ulen3mu1/8en2Ge5/O/7njc5/88n2HIMBQ83zBkHjcM+YzT39912zzXPN9nSIbM/Qq8T4H3mO8zJPl8p48FfmwEjhmBz/Rvy3+O0eH8wDFD/hcK7DMCLzucZwSPnz634/ee3tf59ent4Bu7HA/U3fWsjj8KO/5U7O1nZNf3dKwnlHNDPTaYz+37MwdhwN85qG/FEDDQ6xVDzx1XnKVzixOna2/CBxubzRbWcLBEMGXKFD366KNqbW2V2+2WJG3evNniqoDI8/kMtbR71eLxqdnjVYvHq+Y2r1rbvWpu86nF41VLu7mvpd2nljb/OR6v2tp9avP65PH61Nruk8drqK3d3G9um8c7nmceOx1a2n0+/zP/FQcAoKvaZo/VJYRlaCWCIeLaa6/VnXfeqW9+85u67bbbVFZWpl/84heSFFazAiBaDMNQfWu7aho9qmpqU32LRw0t7WpoNR+Nre2q9z+b+71qaPWosdUbPKehpd2yxh+hctptcjpsctrtcthtctptcvgfdpstuM9ut8lhM59Pv5acdrvsdgXPt9lsstskR4ftwOfY/NuBfXa7TTYFtiXp9LHAudLp9wT32SSbTu+zyeZ/Nn9+dN93+udKj8d0+pjU4dzg8dP7g+couLPD8cCxjt/V+VjH450/qNNmp5+Dnfd3/vfr9XN7OLc3XX/m9ve2vj63aw2hGsyP/UT6Lwb/eQPiz7TCDKtLCAvBJg5lZGTo73//u7797W9r1qxZmjFjhn784x/r2muvpSkAoqKxtV3H61pU1dim6iaPqpvaVO3frmlq87/27/fvi/RdjiSHXckuu5JdDqUkOZTsdCg5yaFkpz34OiXJoWSXXW6nQ26XXW6HXS6HXUnO089JTruSuu5z2JXktCnJ4ZDTYZPLH1acnZ5tcjrs5rM9EDb4TQsAgERBsImRtWvXdtvXcd2aruPD582bp/feey/4+tFHH5XL5VJxcbEkqbS0tNt7Zs2a1ec4cww/hmGousmjitoWVdQ1q6K2VRW1zaqoa9Gx2hb//hbVt7QP6POTXXZlpSYpM8WlEW6n0txOjUh2akSS+Zzmdiq94363QyPcLqW5HUp3u5TqdijF5VCyyyGHnRABAAAGjmATpx555BFNmDBBY8aM0Xvvvacf/OAH+tznPqeUlBSrS0OcqW/x6MDJRh041aADJxtVVtXUKbS0tYfWTGKE26nstCRlpbqUlZakrNQkjUx1KSs1yf/aFdyX7T+eHGYDDQAAgGgh2MSpiooK/fjHP1ZFRYUKCgr02c9+VnfffbfVZcEiXp+hI9XN+tAfXg6cbNCHJ83tE/Wt/b5/1Igk5WUkqyAzuctzivIz3crLSFZ6sqvfzwEAAIhXBJs4deutt+rWW2+1ugzEmGEYKq9q1tbyau07fjq8HKxs7PPOy6gRbk0YnaaJo0eoZFSq8jNTVJCZrPyMZOVmuOV2cmcFAAAMbQQbwEINre3a/lGNtpbVaGtZtbaW1aiysa3Hc5Mcdo0flaqJo0dowug0TRg1QhNzR6hkVJoyU7jbAgAAhjeCDRAjPp+hA6ca9a4/wGwtq9YHx+vVtbmYy2HT1MJMTS3I0ET/XZiJo0doTFYKE+wBAAB6QbABoqTF49XbB6v07uFqbS2v0bayatX10H1szMgUzSoeqXOKRuqc4ixNK8xgUj4AAECYCDZABNU0temV3Sf08q4Krf/gVLcFKJNddp09ZqTOKQ48spSXwdpEAAAAg0WwAQbpSE2z1uys0Es7j+udQ1XydhhbVpiZrDkTcnSuP8RMzk+Xy2G3sFoAAIChiWADhMkwDH1wvEEv7azQy7sq9P6Ruk7Hp+Sna+G0fC2cmqdphRmsXg8AABADBJsYKS0t1axZs/SrX/3K6lIwAF6foXfLqvXyzgq9vOu4Dlc2BY/ZbNL547K1cFqeLpuap3E5aRZWCgAAMDwRbIA+HDrVqIc2HtJz24/qVMPpNsxJTrsuPmOUPjktXx8/K1ejRrgtrBIAAAAEG6ALwzC0+XC1/vv1A3p513EZ/ikzGclOXXpWnhZOzdP8M0crzc3/fAAAAOIFs5gtUF1dreuuu05ZWVlKTU3VokWLtG/fPknmL9WjR4/WE088ETx/1qxZys3NDb5+88035XK51NDQEPPah7J2r0/PbT+qq3+3UZ994E29tNMMNaWTR+uhr56vLT+6TP/1+VlaNKOAUAMAABBnEv+3M8OQPE39nxcNrlRzgkWYbrjhBu3bt0/PPvusMjIy9IMf/ECf+tSntGvXLrlcLs2fP19r167VNddco+rqau3atUtpaWnatWuXpk6dqrVr1+q8887TiBEjovCHGn7qWzx6fFO5Vm04pCM1zZLMoWaLzxmjr19Uokl56RZXCAAAgP4kfrDxNEnLC6357juOSknhTRQPBJoNGzZo3rx5kqRHH31URUVFevrpp/XZz35WpaWlevDBByVJ69ev18yZM1VcXKy1a9cGg01paWmk/zTDzpGaZj204aAee6dc9a3mwpnZaUn6yoXj9JW545g3AwAAkEASP9gkmN27d8vpdGrOnDnBfTk5OZo8ebJ2794tyeyg9t3vflenTp3SunXrVFpaquLiYq1bt07f/OY3tXHjRt1yyy0W/QkS3/aPavSH1w/q+R3HgmvOTBydpn+7eII+c84YJbscFlcIAACAcCV+sHGlmndOrPruMBmG0ev+wHon06dPV05OjtatW6d169bppz/9qYqKinT33Xdr06ZNam5u1kUXXTSo0ocbwzD0yu4TenD9Ab1zqCq4f97EHP3bxSUqPTNXdjvrzQAAACSqxA82NlvYw8GsNHXqVLW3t+vtt98ODkWrrKzUBx98oLPOOkuSZLPZNH/+fD3zzDN6//33dfHFFys9PV0ej0cPPPCAzj33XKWnM+8jVOVVTbrz6fe1/oOTkiSn3aYrZxbq6xeVaPqYTIurAwAAQCQkfrBJMJMmTdJVV12lb3zjG/r973+v9PR03XbbbRozZoyuuuqq4HmlpaX63ve+p3POOUcZGRmSpPnz5+vRRx/V0qVLrSo/oXh9hh7aeEi/eGmvmj1eJTnt+uq88frqx0qUn5lsdXkAAACIINo9W2DVqlU677zz9OlPf1pz586VYRh6/vnn5XK5gudccskl8nq9nZoELFiwQF6vVwsWLLCg6sSy+1idFv9ug/7vc7vU7PHqgpJsvfDdi3X7p84i1AAAAAxBNqO3SR8WqaurU2Zmpmpra4N3KgJaWlp08OBBlZSUKDmZX04HYqj/HbZ4vPp/r+7T79cdULvPUHqyU3d86ix9fnYRc2gAAAASTF/ZoCuGomHIeOtApe54cocOnGqUJC2anq+f/Ms05WYMvQAHAACAzgg2SHi1zR7d88Ju/fmdcklSbrpbP71qui6fnm9xZQAAAIgVgg0S2ovvH9OPntmpk/WtkqRr5xTrB5dPUWaKq593AgAAYCgh2CAhVdS26MfPvK+Xdx2XJE0YnaZ7Fp+tC0qyLa4MAAAAViDYIKH4fIZWv1Omn7+wR/Wt7XLabfp26UQtueQMJbscVpcHAAAAiyRksPH5fFaXkLAS+e+uxePVzX/eqjX+uzSzikbqnmtmaEp+3x0yAAAAMPQlVLBJSkqS3W7X0aNHNXr0aCUlJclmo4VvKAzDUFtbm06ePCm73a6kpCSrSwpLbbNH//bwJm06VK0kp123L5qi6+aOl4MWzgAAAFCCBRu73a6SkhIdO3ZMR48etbqchJSamqri4mLZ7YmzNuuJuhZd98d3tKeiXunJTv33dbM1Z0KO1WUBAAAgjiRUsJHMuzbFxcVqb2+X1+u1upyE4nA45HQ6E+ou18FTjfrK/7ytj6qbNTrdrYe/eoGmFjL0DAAAAJ0lXLCRJJvNJpfLJZeLlr5D2ftHanXDqnd0qqFN43NS9cjX5qg4J9XqsgAAABCHEjLYYOjb+OEpffORLWpobde0wgw99NULNDrdbXVZAAAAiFMEG8SdF98/pn//8za1eX2aOyFHD153ntKTuTsHAACA3hFsEFdWv12mHz69Qz5Dunxavn71hVmsTwMAAIB+EWwQFwzD0MpX9+uXaz6QJH3xgmL97OrptHMGAABASAg2sJzPZ+inz+3SQxsPSZJu/vgZWnrZmQnVvQ0AAADWItjAUm3tPn3/r+/p2ffMdYnuunKqvvqxEourAgAAQKIh2MAyja3tuvFPW/T6vlNy2m365edm6qpZY6wuCwAAAAmIYANLVDW26asPbdJ75TVKcTl0/5fPVenkXKvLAgAAQIIi2CDm6ls8+sKDb+qD4w0amerSqhvO1znFWVaXBQAAgARGsEFMGYahW/+2XR8cb1BehluP/tscnZGbbnVZAAAASHAEG8TUHzcc0gvvV8jlsOmBL59HqAEwcIbR87a5I7RjvZ7Tw3k9nhOqPt7b5+cO5jsHaFB/zlhLpFqBBORMkRyJExcSp1IkvC2Hq7Ti+d2SpB9eMZXhZ4gOw5B8XsnbKnnbpPY2/7ZHavfv83kln8fc52s3H8Ftj+Rt77DtOX2+zysZXv/rDtv97vP5Hx22fV6z1lCOKbDd0z71fp4M/y+pXZ572tfrszrvU+Cpp+M9bAf+Tbq97nqsw/G+9gMAYudLf5MmXWZ1FSEj2CAmKhtateTRrWr3GbpyZqGumzvO6pIQS4Zhhoq2Bqm1TmptkFrrpbZGydMktbeYz56WDq+bzUePx5p6CCwe83V7q/hFGACA4Ydgg6jz+gzd8vg2VdS1aOLoNK1YPIPFNxONYZhhoqlKaq6Smir929Xmc0utGVja/IElGFzqT7/2eSwq3iY53ZIjyf9wSXaXeWvd7pLsTv+2/7XDJdkdPRwLPBySzdHhObDP7n+f8/TxTufa/Q+HZLOdft3tmN08bneYtQeP+beD+2zmo9Nre4dzbN2fg8fU+zk9PUtd9qnLtno4t+s5IRzrerzT6372ddvfRbdj4Zwb4vsG+94BfWa/bx7EewfydfxshxTz6w7R43BZXUFYCDaIul+/sk+v7zvlb+t8nka4uezigtcj1VdI9cekuiPmduMpf3Dxh5dAcGmqNO+GRIIrTXKPkNzpUlKa5EqVXCnmOF5XiuRKNvc5/c89vXammGHF6TZ/6Dr8wcWZ5N92nQ4zdie/bAEAMAzwGyaiau3eE/p/r+6TJK1YPENn5tEsICbaGqW6o6cf9UelumMdto9KDScU9pAtR5KUki2lZkupOVJKlrmdPNIMKu50KckfWtwjJHdG59dJI/x3IgAAACKLYIOoOVLTrO89vk2GIX1pTrGuPmeM1SUNLW1NUuV+qXKfdGq/dOoDc7v6kDk0LBR2l5ReIGUUShkFUuooM7CkZncIMB22k0Zw9wMAAMQlgg2ioq3dpyWPvqvqJo9mjMnUjz491eqSEpNhmHdXTn1ghphT+05v15b3/d6kEf7AUiilF54OLxljToeZ1FGS3R6bPwsAAEAUEWwQFcuf361t5TXKSHbqd186V8kuhh/1y+uRjr8vfbTZfJzcbd6J8TT2/p6ULGnUmVLOJGnUGeZ29kQztCRnxK52AAAAixFsEHHPbT+qhzYekiT91+dnqSg71dqC4lXdMemjTf7HZunoVqm9uft5dqeUVSKNmiTl+MPLqElmmEnLiX3dAAAAcYhgg4j68GSDfvC37ZKk75RO1KVn5VlcUZzwtEgV208HmfJNUt1H3c9LzpTGnm8+8meYISZrfMK1WwQAAIg1gg0ipqmtXd/+0xY1tnl14YRsLb3sTKtLsk57m3RovbTvn9JH70jHtndfx8Vml3KnSWNnm0Gm6AJzGBlzXgAAAMJGsEFEGIahHz71vj443qDR6W795ovnyOkYZr+gt9RJ+9dIe/4h7VtjLljZUeooM7yMnS2NvUAqPMdsgQwAAIBBCyvYrFixQk8++aT27NmjlJQUzZs3Tz//+c81efLk4Dk33HCDHn744U7vmzNnjt56663IVIy49Od3yvXk1iNy2G1a+cVzlJuebHVJsVF/XNr7vBlmDq6TvG2nj43IkyYvksZdJBWdL40cR6tkAACAKAkr2Kxbt05LlizR+eefr/b2dt15551auHChdu3apbS0tOB5l19+uVatWhV8nZSUFLmKEXd2fFSrZc/ulCT9n09O1pwJQ3xCe+WH0p7npN3PmfNlOi5ymXOGNOXT5mPMeQwrAwAAiJGwgs2LL77Y6fWqVauUm5urLVu2aP78+cH9brdb+fn5kakQca22yaPvrN6iNq9PnzgrT9+aP8HqkiLPMKSj75p3Zfb8Qzq5p/PxMedJU64ww8zoyT1/BgAAAKJqUHNsamvN1c2zs7M77V+7dq1yc3M1cuRILViwQHfffbdyc3N7/IzW1la1trYGX9fV1fV4HuKPz2foP/66TeVVzSrKTtEvPzdTtqE01KqtUdq2Wnrrfqnqw9P77U5p/MX+MHOFuWYMAAAALGUzDMPo/7TuDMPQVVddperqar3++uvB/Y8//rhGjBihcePG6eDBg/rRj36k9vZ2bdmyRW63u9vnLFu2TD/5yU+67a+trVVGBgsMxrPVb5fpjqd2KMlp15PfnqfpYzKtLiky6iukdx6UNv9Raq4297nSpEmXmXdlJl0mpYy0tEQAAIDhoK6uTpmZmSFlgwEHmyVLlugf//iH3njjDY0dO7bX844dO6Zx48bpscce0+LFi7sd7+mOTVFREcEmzjW1tWvBf67VyfpW/ejTU/X1i0qsLmnwKt6X3vqdtOOvp5sAZI2XLvyONOtLdDADAACIsXCCzYCGot1888169tlntX79+j5DjSQVFBRo3Lhx2rdvX4/H3W53j3dyEN/++MZBnaxvVXF2qr5y4Tiryxk4w5A+fEXauFI68Nrp/UUXSnOXmEPN7A7r6gMAAEBIwgo2hmHo5ptv1lNPPaW1a9eqpKT//5e+srJS5eXlKigoGHCRiC9VjW16YN0BSdL3PzlZSc4E7PzlaTHvzLz5W+nkbnOfzS6d9S/SvJvNtWYAAACQMMIKNkuWLNHq1av1zDPPKD09XRUVFZKkzMxMpaSkqKGhQcuWLdM111yjgoICHTp0SHfccYdGjRqlz3zmM1H5AyD2fvvafjW0tmtaYYY+PSPBAmtjpbT5f8w5NI0nzX1JI6Rzr5Pm3ChlJfDdJwAAgGEsrGBz//33S5JKS0s77V+1apVuuOEGORwO7dixQ4888ohqampUUFCgSy65RI8//rjS09MjVjSs81F1k/73zcOSpB9cPkV2e4J0QWs8Jb22XNr2qNTeYu7LGGOGmXOvoxkAAABAggt7KFpfUlJS9NJLLw2qIMS3/1qzT21en+ZNzNHFk0ZZXU7/DEPa/rj04u1Sc5W5r2CmNPdmadrVksNlaXkAAACIjEGtY4PhZU9FnZ7c+pEk825N3K9ZU31Ieu570oevmq/zpkuXrzDXoIn32gEAABAWgg1C9p8v7pVhSFfMKNDMopFWl9M7b7v09gPSa3dLnibJ4ZZKbzObAnCHBgAAYEgi2CAk7xys0it7Tshht+k/Fp5pdTm9q9ghPXuzdHSr+Xr8xdKVv5ZyJlpbFwAAAKKKYIN+GYahe14wWyJ/4fwiTRgdhwtVepqldT+XNvxGMrySO1P65M+kc77CsDMAAIBhgGCDfq3ZdVzvltUoxeXQdy+dZHU53R1cL/39u1KVubaOpl4lLbpXSs+3ti4AAADEDMEGffL6DP3nS3slSV+7aLxyM5ItrqiD5mppzY+ldx8xX6cXSFf8UppyhbV1AQAAIOYINujTE+9+pH0nGjQy1aVvLYiTeSqGIe16Rnr+/0iNJ8x9s78ufeIuKTnT2toAAABgCYINetXi8eq/1nwgSbrpkjOUkRwHHcXqK6Tnlkp7/2G+HnWmdOVvpHFzra0LAAAAliLYoFePvHlIx2pbVJiZrC9fOM7qcqSTe6X/XSzVfSTZXdLFS6WL/0Nyuq2uDAAAABYj2KBHtc0e/fa1DyVJ37vsTCW7HNYW9NFm6dF/NefV5EySPveIlDfV2poAAAAQNwg26NED6z5UbbNHZ+aN0OJzx1pbzP5/So9/xVxss/Bc6Ut/k9JyrK0JAAAAcYVgg26O17Vo1YaDkqRbPzlFDruF68Ds+Jv01I2SzyNNuET6/J8kdxyuowMAAABL2a0uAPHnV//cpxaPT7PHZenSs3KtK+Tt30tP/JsZaqZfI137F0INAAAAesQdG3Ty4ckG/WVzuSTptkVTZLNZcLfGMKTXlkvr7zVfX/BN6fKfS3ZyOAAAAHpGsEEnv3hpr7w+Q584K0+zx2fHvgCfV/rHf0hbVpmvL/mhNP/7khUBCwAAAAmDYIOgbeU1euH9Ctlt0q2XT459Ae2t5tCz3c9Kskmfvk+a/bXY1wEAAICEQ7CBJMkwDN3zwm5J0uJzx+rMvPTYFtBSJz12rXTodcmRJC3+gzTt6tjWAAAAgIRFsIEkad0HJ/XWgSolOe363mVnxvbLG05If7pGqtguJY2QvrBamrAgtjUAAAAgoRFsIJ/P0M9f3CtJun7uOI0ZmRK7L68+JP3vZ6SqA1LqKOnLf5MKz4nd9wMAAGBIINhAf99+VLuP1Snd7dR3Ss+I3RdXvC/9abHUcFwaWSx95WkpZ2Lsvh8AAABDBsFmmDMMQ795ZZ8k6cbSicpKS4rNFx/eKK3+gtRaK+VOk778hJRREJvvBgAAwJBDsBnmtpbX6MOTjUpxOXT9vPGx+dITu805NZ4mqXiu9MXHpJSRsfluAAAADEkEm2Hub1s+kiQtmpGvEe4YXA6t9dLjXzFDzfiLpWv/IiWlRv97AQAAMKSxlPsw1uLx6u/vHZUk/et5Y6P/hYYhPXuzVLlPSi+UPvsQoQYAAAARQbAZxl7edVz1Le0aMzJFF5bkRP8L3/mDtPMpye40Q03aqOh/JwAAAIYFgs0w9tfN5ZKka84bK7vdFt0v+2iz9NId5vZl/1cqnhPd7wMAAMCwQrAZpo7VNuuN/ackSdecOya6X9ZUJf3lesnnkaZeJV347eh+HwAAAIYdgs0w9eS7R2QY0gUl2RqXkxa9L/L5pCe/IdV9JGVPlP5lpWSL8t0hAAAADDsEm2HIMAw94e+GFvWmAa//Qtr/T8mZIn3+f6XkjOh+HwAAAIYlgs0w9G5ZjQ6cMteu+dSMKC6K+eFr0mvLze1P3yflTYvedwEAAGBYI9gMQzFZu6b2iPTE1yUZ0rnXSbOujc73AAAAACLYDDstHq+ei/baNV6P9LevSk2VUv4MadG90fkeAAAAwI9gM8y8tLNC9a1RXrvmn8uk8rcld6b0uUckV0p0vgcAAADwI9gMM4FhaFFbu2bXs9KbK83tz9wvZU+I/HcAAAAAXRBshpGOa9f867lRGIZW+aH0zBJze96/S1OuiPx3AAAAAD0g2AwjgbVr5pRkqzgnNbIf7mmW/nKd1FonFc+TLv1xZD8fAAAA6APBZpiI+to1z39fOv6+lDZa+tc/Sg5X5L8DAAAA6AXBZpgIrF2TmhSFtWve/V9p658km90MNRlRXBsHAAAA6AHBZpgIrl0zvUBpkVy75th2826NJF1yp1QyP3KfDQAAAISIYDMMRG3tmpZac15Ne4s06ZPSRUsj99kAAABAGAg2w0Bg7ZqxWSmaU5IduQ9+bYVUfVDKLJY+84Bk53ICAACANfhNdBgIrl1zbgTXrqk+LG3+H3P7X34tpUYwMAEAAABhItgMcUdrTq9dc00k1655bbnkbZMmlEoTPx65zwUAAAAGgGAzxD21NQpr11S8L21/3Nz+xLLIfCYAAAAwCASbIcwwjOAwtIg2DXjlJ5IMadpiqfCcyH0uAAAAMEAEmyHs3bJqHYz02jWH3pD2vSzZndLHfxiZzwQAAAAGiWAzhEV87RrDkNbcZW6fd4OUM3HwnwkAAABEAMFmiGpu8+q5945JiuAwtD3PSUc2S65Uaf6tkflMAAAAIAIINkPUy7sivHaNt1165afm9twlUnre4D8TAAAAiBCCzRAV8bVr3lstnfpASsmW5v374D8PAAAAiCCCzRDUce2aiAxD8zRLr60wt+f/Hyk5Y/CfCQAAAEQQwWYICqxdc+GEbBVlR2Dtmrd/L9UflTKLpfO/PvjPAwAAACKMYDPEdF67pmjwH9hcLb1xn7l9yR2S0z34zwQAAAAijGAzxHRcu2bR9PzBf+Ab/yW11Eq5U6WzPzf4zwMAAACigGAzxPx1s3m35lMzIrB2Te0RcxiaJH1imWR3DO7zAAAAgCgh2AwhzW1ePbc9gmvXrLtHam+RiudJkxYO/vMAAACAKCHYDCEv7axQQ2u7irJTdMH4Qa5dc3KvtPVP5vZlP5FsEWgZDQAAAEQJwWYIiejaNa/8VDJ80pRPS0UXRKA6AAAAIHoINkPEsdpmbfjQXLvmmnMHOQytfJO05znJZpcu/XEEqgMAAACii2AzRKz/4KQMQzqneOTg1q4xDOmfd5nbs66VRk+OTIEAAABAFBFshogN+yslSRefMWpwH7RvjXR4g+RwS6W3R6AyAAAAIPoINkOAYRja6B+GNm8wwcbnk175ibk951tSZgQ6qwEAAAAxQLAZAvYer9ephjaluBw6p3jkwD9ox1+l4+9LyZnSRd+LWH0AAABAtBFshoDAMLTzS7Lldg5wEc32VunVn5nbH7tFSh1ku2gAAAAghgg2Q8CG/eYwtIvOyBn4h2z+o1RbJqUXSHNujFBlAAAAQGwQbBKcx+vT2wfMOzbzJg5wfk1LnbT+P83t0tukpEF0VQMAAAAsQLBJcNs/qlFjm1dZqS5NLcgY2Ie8uVJqqpRyJkmzvhzZAgEAAIAYINgkuDf2nb5bY7fbwv+A9lbpnQfN7Y//UHI4I1gdAAAAEBsEmwS3IdjmeYDza/a+IDVXS+mF0llXRrAyAAAAIHYINgmsqa1dW8uqJUkXDXT9mm2rzeeZX5DsA+yoBgAAAFiMYJPA3jlYJY/X0JiRKSrOHsCE//oKaf8/ze1Z10a2OAAAACCGCDYJbOOH5vyaj52RI5ttAPNrtv9FMrzS2AukUZMiXB0AAAAQOwSbBPbGPnN+zccGMgzNME4PQ+NuDQAAABIcwSZBVTW2adexOkkDXL/m6Fbp5G7JmSxNXxzh6gAAAIDYItgkqI3+bmhT8tM1Ot0d/gcE7tacdaWUnBnBygAAAIDYI9gkqA37T69fE7b2VmnHX81thqEBAABgCCDYJKjAHZuPDWT9mr3PSy01UsYYqWRBZAsDAAAALECwSUDlVU06XNkkh92mORMGEGxYuwYAAABDDMEmAQXu1swqGqkRbmd4b+64ds1MhqEBAABgaCDYJKA39gfWrxnA/Jrtj0uGTyqaI406I8KVAQAAANYg2CQYwzD0ZmB+zcQwh6Gxdg0AAACGKIJNgtl7vF6nGtqU4nLonOKs8N589F3p5B5z7Zppn4lOgQAAAIAFCDYJ5o195t2aC0qyleQM85+PtWsAAAAwRBFsEszGDwPza8IchuZpkXb8zdye9aUIVwUAAABYi2CTQDxen94+MMDGAR+84F+7ZqxUMj/yxQEAAAAWItgkkPfKa9TY5lV2WpLOys8I781bHzWfWbsGAAAAQxDBJoFs8Ld5njshR3a7LfQ31h2TPnzF3KYbGgAAAIYggk0C2bDf3+Y53GFowbVrLpRyJkahMgAAAMBaBJsE0djarq3l1ZLCbBzA2jUAAAAYBgg2CeKdQ1XyeA2NzUpRcXZq6G888q50aq/kTJGmXR21+gAAAAArEWwSxMbAMLSJo2SzhTG/Zpu/aQBr1wAAAGAICyvYrFixQueff77S09OVm5urq6++Wnv37u10jmEYWrZsmQoLC5WSkqLS0lLt3LkzokUPR4HGAfPCGYbmaZHeD6xdwzA0AAAADF1hBZt169ZpyZIleuutt7RmzRq1t7dr4cKFamxsDJ5z77336r777tPKlSu1adMm5efn67LLLlN9fX3Eix8uKhtatetYnSRp3sQwGgfsfV5qqfWvXbMgStUBAAAA1nOGc/KLL77Y6fWqVauUm5urLVu2aP78+TIMQ7/61a905513avHixZKkhx9+WHl5eVq9erW+9a1vRa7yYeRN/6KcU/LTNTrdHfobg00DvijZGXUIAACAoWtQv+3W1tZKkrKzsyVJBw8eVEVFhRYuXBg8x+12a8GCBdq4cWOPn9Ha2qq6urpOD3Q2oDbPdUdPr10z84tRqAoAAACIHwMONoZhaOnSpbrooos0ffp0SVJFRYUkKS8vr9O5eXl5wWNdrVixQpmZmcFHUVHRQEsasgLza8Jq8xxYu6Z4LmvXAAAAYMgbcLC56aabtH37dv35z3/udqxr1y7DMHrt5HX77bertrY2+CgvLx9oSUNSeVWTyqqa5LTbdEFJiMGGtWsAAAAwzIQ1xybg5ptv1rPPPqv169dr7Nixwf35+fmSzDs3BQUFwf0nTpzodhcnwO12y+0OY97IMBMYhjaraKRGuEP85zqyRTr1gbl2zdSro1ccAAAAECfCumNjGIZuuukmPfnkk3r11VdVUlLS6XhJSYny8/O1Zs2a4L62tjatW7dO8+bNi0zFw8yGDwNtnsOYXxNYu2bqv0jJGVGoCgAAAIgvYd2xWbJkiVavXq1nnnlG6enpwXkzmZmZSklJkc1m0y233KLly5dr0qRJmjRpkpYvX67U1FRdey1DosLl8xnBhTkvCjXYeFqkHU+Y2wxDAwAAwDARVrC5//77JUmlpaWd9q9atUo33HCDJOnWW29Vc3OzvvOd76i6ulpz5szRyy+/rPT09IgUPJzsPV6vysY2pbgcmlU0MsQ3/UNq9a9dM35+VOsDAAAA4kVYwcYwjH7PsdlsWrZsmZYtWzbQmuAXmF9zQUm2kpwhjhpk7RoAAAAMQ/zmG8c2hDsMre6o9OGr5jbD0AAAADCMEGzilMfr09sHqyRJ80Jdvya4ds08KXtCFKsDAAAA4gvBJk5tK69RU5tX2WlJOis/hM5mhiFt9XdD424NAAAAhhmCTZwKDEObOzFHdnvPi5t2cuw9qXKff+2aq6JcHQAAABBfCDZxauN+c/2aj00McX7NgdfM54mXsHYNAAAAhh2CTRxqbG3Xu2XVksJoHHBgnflcsiBKVQEAAADxi2ATh945VKV2n6GxWSkqzknt/w3trVLZW+b2BIINAAAAhh+CTRzasC/MNs/l70jtzdKIPGn0lChWBgAAAMQngk0c2vChOb9mXsjD0NaazyULJFsIjQYAAACAIYZgE2cqG1q1+1idJGnexBDXrznon1/DMDQAAAAMUwSbOLPRf7dmSn66Ro1w9/+GljrpyLvmNo0DAAAAMEwRbOLM2wf9w9BCbfN8eINkeKXsCdLIoihWBgAAAMQvgk2c2XXUHIY2sygztDcE5tdMKI1KPQAAAEAiINjEEZ/P0J6KeknS1IIQF9lk/RoAAACAYBNPyqub1NTmVZLTrpJRaf2/of64dHK3JJtUMj/q9QEAAADximATRwLd0M7MGyGnI4R/moPrzef8GVJqdhQrAwAAAOIbwSaO7D5mDkM7Kz/UYWhrzWfm1wAAAGCYI9jEkcAdmymhzK8xDNavAQAAAPwINnFkd4UZbM4qSO//5KoDUm25ZHdJxXOjXBkAAAAQ3wg2caK+xaPyqmZJIQ5FCwxDK7pASgqh0QAAAAAwhBFs4sRef5vn/IxkZaUl9f+G4DC00ugVBQAAACQIgk2c2O0PNiENQ/P5pIOvm9usXwMAAAAQbOJFWI0Dju+QmqukpBHSmHOjXBkAAAAQ/wg2cSIQbM4KJdgE5teM+5jkcEWvKAAAACBBEGzigM9nBOfYTA1lKNoB5tcAAAAAHRFs4kBZVZOa2rxKcto1PqefDmftbVLZm+Y269cAAAAAkgg2cSEwDG1yXrqcjn7+ST7aJHmapLTRUu7UGFQHAAAAxD+CTRwIqyNaYH5NyXzJZoteUQAAAEACIdjEgWBHtFAW5mT9GgAAAKAbgk0c2FMRYke01nrpyBZzm/VrAAAAgCCCjcXqWzwqr2qWFMJQtMMbJV+7lDVeyhoX/eIAAACABEGwsdge//yagsxkjUxN6vvk4Pwa7tYAAAAAHRFsLLYnrIU5mV8DAAAA9IRgY7Fdx8w7NlPy+xmG1nBSOrHT3C6ZH+WqAAAAgMRCsLFYyI0DAt3Q8mZIaaOiXBUAAACQWAg2FvL5DO0NrmHTT7AJzK+ZwPwaAAAAoCuCjYUOVzWpqc0rt9Ou8TmpfZ/M+jUAAABArwg2Fgo0Dpicny6no49/iqqDUk2ZZHdKxXNjVB0AAACQOAg2FtrtDzb9Ng4I3K0Ze77kHhHlqgAAAIDEQ7CxUKAjWsjza1i/BgAAAOgRwcZCIXVE8/mkg+vNbebXAAAAAD0i2FikrsWjj6qbJUln5fcRbE7slJoqJVeaNOa8GFUHAAAAJBaCjUUCbZ4LM5OVmerq/cQD/vk14+ZJzqQYVAYAAAAkHoKNRQKNA1i/BgAAABg8go1Fgh3RCvroiNbeJh3eaG4zvwYAAADoFcHGIrtD6Yh2ZIvkaZRSc6TcaTGqDAAAAEg8BBsLeH1GcI7NlL4aBwTWrymZL9n5pwIAAAB6w2/LFiiralKzx6tkl10lo9J6P5H1awAAAICQEGwsEJhfMzkvXQ67reeTWhukjzaZ2zQOAAAAAPpEsLFAsHFAX8PQyt6UfO3SyGIpqyRGlQEAAACJiWBjgdONA/roiNZxGJqtl7s6AAAAACQRbCwR0ho2gYU5afMMAAAA9ItgE2O1zR4dqWmW1MdQtMZT0vEd5nbJ/BhVBgAAACQugk2MBdo8jxmZosxUV88nHVxvPudOk0bkxqgyAAAAIHERbGLsdOOAPubXBNavoRsaAAAAEBKCTYztqQhlfs1a85n1awAAAICQEGxibFewI1ovwab6sFR9SLI5pHHzYlcYAAAAkMAINjHk9Rna679jM6W3Vs+BYWhjZ0vJfdzVAQAAABBEsImhw5WNavH4lOyya3xOWs8nlb1lPo+/OHaFAQAAAAmOYBNDgYU5J+dnyGHvZdHNiu3mc+E5MaoKAAAASHwEmxgKNg7orSNae5t0cq+5nT89RlUBAAAAiY9gE0OBVs+9Ng449YHkbZPcGdLIcTGsDAAAAEhsBJsYCgxF63UNm+Pvm8950yRbL0PVAAAAAHRDsImR2maPjtQ0S5Km9HbHpmKH+Zw/I0ZVAQAAAEMDwSZG9viHoY0ZmaLMFFfPJwXv2DC/BgAAAAgHwSZG9lQEFubsZRiaYUgV/mBD4wAAAAAgLASbGOm3cUDDcanplGSzS7lTY1gZAAAAkPgINjHSb7AJzK/JOUNypcSoKgAAAGBoINjEgNdnaO/xfjqiBYIN82sAAACAsBFsYuBQZaNaPD6luBwal5PW80mBxgF0RAMAAADCRrCJgcAwtDPz0+Ww97I+TQXBBgAAABgogk0M7PEvzDm1t45onmapcp+5zVA0AAAAIGwEmxjot3HAiV2S4ZNSc6T0/BhWBgAAAAwNBJsYCKxhMyW/t45oHRbmtPUyVA0AAABArwg2UVbb5NGRmmZJ0pTehqLROAAAAAAYFIJNlO2uMIehjc1KUUayq+eTaBwAAAAADArBJsr2+OfX9DoMzTCk4zvNbRoHAAAAAANCsImy3f11RKspk1prJbtLGnVmDCsDAAAAhg6CTZTt8Q9Fm9JbR7SKHebz6CmSMylGVQEAAABDC8Emirw+Q3uPm3dsem31HGwcwDA0AAAAYKAINlF08FSjWjw+pbgcGped2vNJgTs2zK8BAAAABoxgE0WBYWiT89Nlt/eyPg2tngEAAIBBI9hE0W5/R7Reh6G11EnVh8xtgg0AAAAwYASbKOq3I1qgzXN6oZSaHaOqAAAAgKGHYBNFwTVsaBwAAAAARBXBJkpqmtp0tLZFkjnHpkc0DgAAAAAigmATJXsqzGFoY7NSlJHs6vkkGgcAAAAAEUGwiZJ+Gwf4vNLxXeY2wQYAAAAYFIJNlPQbbKoOSO3NkjNFyp4Qw8oAAACAoYdgEyWBoWhn9Tq/Zrv5nDdVsjtiVBUAAAAwNBFsosDnM/TBcTPY9NoRrcI/v4bGAQAAAMCghR1s1q9fryuvvFKFhYWy2Wx6+umnOx2/4YYbZLPZOj0uvPDCSNWbEE7Ut6rF45PTblNRVkrPJ9E4AAAAAIiYsINNY2OjZs6cqZUrV/Z6zuWXX65jx44FH88///ygikw0hysbJUljslLkdPTyV1xBsAEAAAAixRnuGxYtWqRFixb1eY7b7VZ+fv6Ai0p0h6uaJEnF2ak9n9BYKdUfNbfzpsWoKgAAAGDoisocm7Vr1yo3N1dnnnmmvvGNb+jEiRO9ntva2qq6urpOj0RXVtlPsDnuX5gza7zk7qW5AAAAAICQRTzYLFq0SI8++qheffVV/fKXv9SmTZv08Y9/XK2trT2ev2LFCmVmZgYfRUVFkS4p5sr8d2zG5fQSbGgcAAAAAERU2EPR+vP5z38+uD19+nTNnj1b48aN0z/+8Q8tXry42/m33367li5dGnxdV1eX8OHm9FC0tJ5PoHEAAAAAEFERDzZdFRQUaNy4cdq3b1+Px91ut9xud7TLiKkyf/MA7tgAAAAAsRH1dWwqKytVXl6ugoKCaH9VXKhr8ai6ySNJKuppjk17m3Ryj7nNHRsAAAAgIsK+Y9PQ0KD9+/cHXx88eFDbtm1Tdna2srOztWzZMl1zzTUqKCjQoUOHdMcdd2jUqFH6zGc+E9HC41WgccCoEUka4e7hr/fUB5LPI7kzpZHFMa4OAAAAGJrCDjabN2/WJZdcEnwdmB9z/fXX6/7779eOHTv0yCOPqKamRgUFBbrkkkv0+OOPKz19eHT/Kuuv1XOFvyNa3jTJZotRVQAAAMDQFnawKS0tlWEYvR5/6aWXBlVQojtcGeiI1l/jAObXAAAAAJES9Tk2w01Zldk4oP87NgQbAAAAIFIINhF2uK/FOQ2DOzYAAABAFBBsIqzPxTnrK6SmSslml3KnxrgyAAAAYOgi2ERQW7tPR2uaJUnFPQWbwDC0nEmSKyWGlQEAAABDG8Emgo7UNMtnSKlJDo0e0cOio8f9wYZhaAAAAEBEEWwi6HDl6cYBtp5aOVf459fQOAAAAACIKIJNBAXm1xT11hEt2DhgRowqAgAAAIYHgk0ElQXWsOkp2Hiapcr95jbBBgAAAIgogk0EHe6rI9qJXZLhk1JHSSPyYlwZAAAAMLQRbCIocMemOCet+8GKDo0Depp/AwAAAGDACDYRYhhGcI5Nj4tz0jgAAAAAiBqCTYScrG9Vs8cru00aM7KHNWpoHAAAAABEDcEmQgJ3awpHpijJ2eWv1TCk4zvNbe7YAAAAABFHsImQw5V9NA6oOSy11kmOJGnUmTGuDAAAABj6CDYRcjg4v6aPxgGjJ0vOpBhWBQAAAAwPBJsIKatslNRf4wDm1wAAAADRQLCJkLK+1rAJNg5gfg0AAAAQDQSbCOm71bN/KBqNAwAAAICoINhEQENru041tEnq4Y5NS53ZPECi1TMAAAAQJQSbCCjzd0TLTktSerKr88FAm+eMMVJqdowrAwAAAIYHgk0ElFWZjQOKGIYGAAAAWIJgEwHBxgE9BZvj/mBD4wAAAAAgagg2EdDn4pzBVs8EGwAAACBaCDYR0GtHNJ9XOrHb3KZxAAAAABA1BJsICNyx6RZsKj+U2pslZ4qUPcGCygAAAIDhgWAzSB6vT0dqmiVJ43LSOh+s2G4+502T7I4YVwYAAAAMHwSbQTpW0yKvz5DbaVduurvzweP++TU0DgAAAACiimAzSIf9rZ6Ls1Nlt9s6H6RxAAAAABATBJtB6rMjWvCODY0DAAAAgGgi2AxSoCNat8U5Gyul+mPmdt60GFcFAAAADC8Em0E6XGkOReu2OGdgYc6sEsmdHuOqAAAAgOGFYDNIZVW9dUTzBxsaBwAAAABRR7AZBMMwVOa/Y1PcdY5NsHEA82sAAACAaCPYDEJlY5sa27yy2aSxWSmdD9LqGQAAAIgZgs0gBDqiFWQky+3ssACnzyud3Gtu0zgAAAAAiDqCzSCU+zuidRuGVn9M8nkku1PKLLKgMgAAAGB4IdgMQnANm+wujQOqD5vPmWMlu0MAAAAAootgMwiHq3ppHFBTZj6PLI5xRQAAAMDwRLAZhDL/HZvirmvY1Pjv2IwcF+OKAAAAgOGJYDMIh/1zbMZ1vWMTGIqWRbABAAAAYoFgM0DNbV6drG+V1MMcm+BQNIINAAAAEAsEmwEq89+tyUxxKTPV1fkgQ9EAAACAmCLYDNDhSrNxQLdhaF6PVHfE3KZ5AAAAABATBJsBCtyxKeraOKD2I8nwSQ63NCLPgsoAAACA4YdgM0Cn17DprSNasWTnrxcAAACIBX7zHqCy3jqisYYNAAAAEHMEmwEKBJvirh3RaPUMAAAAxBzBZgC8PkMfVfd2x4aOaAAAAECsEWwG4GhNszxeQ0kOu/IykjsfZCgaAAAAEHMEmwEIDEMbm50ih93W+SBD0QAAAICYI9gMQLBxQNeOaJ4WqaHC3GYoGgAAABAzBJsBCLZ6zunSOKC23Hx2pUmpOTGuCgAAABi+CDYDUFbVKKmHxTk7DkOzdRmiBgAAACBqCDYDENLinAAAAABihmATJsMwVFZJq2cAAAAgnhBswlTT5FF9a7ukHoaiBVo90xENAAAAiCmCTZgO+zui5WckK9nl6HywmqFoAAAAgBUINmE6XGk2DijuerdGYigaAAAAYBGCTZgC82uKu86vaW2QmirNbe7YAAAAADFFsAlTr4tzBubXJGdKKSNjWxQAAAAwzBFswhSYY9Ptjk0g2DAMDQAAAIg5gk2YTrd6Tut8gDVsAAAAAMsQbMLQ4vGqoq5FUg/NAwId0bLGx7YoAAAAAASbcJT7h6Glu53KSnV1PsgdGwAAAMAyBJswlHWYX2Oz2TofpNUzAAAAYBmCTRgOB+fX9LCGTbW/eUAWwQYAAACINYJNGAJ3bIq6zq9prpFaa83tzKLYFgUAAACAYBOOw5WNkqRx2b10REsdJblHxLgqAAAAAASbMATWsOk2FK2GYWgAAACAlQg2IfL5DH1U1Sypj1bPdEQDAAAALEGwCVFFXYvavD457TYVjkzpfJCOaAAAAIClCDYhCnREG5uVIoe9a6tn/1A07tgAAAAAliDYhKisymwcUJyT1v1gYCgac2wAAAAASxBsQhRo9Tyu6/waw+hwx2Z8bIsCAAAAIIlgE7JeF+dsqpQ85t0cZY6NcVUAAAAAJIJNyAJ3bHrtiJZeILmSY1wVAAAAAIlgE7LAHZvibmvY0BENAAAAsBrBJgS1TR7VNnsk9XDHpoY1bAAAAACrEWxCEBiGNjrdrdQkZ+eDgcYBdEQDAAAALEOwCcFhf6vnbh3RpNNzbLhjAwAAAFiGYBOC4PyanoINc2wAAAAAyxFsQlDWW+MAn0+qKTe3GYoGAAAAWIZgE4LgULSuwabhuORtlWx2KWOMBZUBAAAAkAg2ISmvapYkFWendT4QGIaWMVZyuGJcFQAAAIAAgk0/Wtu9OlprBptud2wCHdFoHAAAAABYimDTj4+qm2UYUmqSQzlpSZ0PBjqiMb8GAAAAsBTBph9lHTqi2Wy2zgfpiAYAAADEBYJNPw5X9tI4QOoQbBiKBgAAAFiJYNOPsqrA/Jq07gcZigYAAADEBYJNP8r8rZ67Lc7pbZfqjpjbDEUDAAAALEWw6cfhDnNsOqk/KvnaJbtLSs+3oDIAAAAAAWEHm/Xr1+vKK69UYWGhbDabnn766U7HDcPQsmXLVFhYqJSUFJWWlmrnzp2RqjemfD5DZVVmsOm91XORZHfEuDIAAAAAHYUdbBobGzVz5kytXLmyx+P33nuv7rvvPq1cuVKbNm1Sfn6+LrvsMtXX1w+62Fg72dCq1nafHHabCkemdD5YTeMAAAAAIF44w33DokWLtGjRoh6PGYahX/3qV7rzzju1ePFiSdLDDz+svLw8rV69Wt/61re6vae1tVWtra3B13V1deGWFDWBYWhjRqbI5eiSAWn1DAAAAMSNiM6xOXjwoCoqKrRw4cLgPrfbrQULFmjjxo09vmfFihXKzMwMPoqKiiJZ0qD03erZPxSNjmgAAACA5SIabCoqKiRJeXl5nfbn5eUFj3V1++23q7a2NvgoLy+PZEmDcuGEHP3nv56t6+aO736wmjs2AAAAQLwIeyhaKGw2W6fXhmF02xfgdrvldrujUcagFWWnqqhrN7SAYPMAgg0AAABgtYjescnPN9sed707c+LEiW53cRJae9vpNWwYigYAAABYLqLBpqSkRPn5+VqzZk1wX1tbm9atW6d58+ZF8qusVVsuyZCcKVLaaKurAQAAAIa9sIeiNTQ0aP/+/cHXBw8e1LZt25Sdna3i4mLdcsstWr58uSZNmqRJkyZp+fLlSk1N1bXXXhvRwi0VHIZWLPUyxA4AAABA7IQdbDZv3qxLLrkk+Hrp0qWSpOuvv14PPfSQbr31VjU3N+s73/mOqqurNWfOHL388stKT0+PXNVWq2ENGwAAACCehB1sSktLZRhGr8dtNpuWLVumZcuWDaau+BboiMb8GgAAACAuRHSOzbBBRzQAAAAgrhBsBoKhaAAAAEBcIdgMROCODUPRAAAAgLhAsAmXp1lqOG5uMxQNAAAAiAsEm3AF7tYkpUspWdbWAgAAAEASwSZ8HYehsYYNAAAAEBcINuGqPmQ+0zgAAAAAiBsEm3DR6hkAAACIOwSbcNWwOCcAAAAQbwg24apmDRsAAAAg3hBswsVQNAAAACDuEGzC0VovNVeZ29yxAQAAAOIGwSYcgbs1KVlScoa1tQAAAAAIItiEIzi/hmFoAAAAQDwh2ISjhsYBAAAAQDwi2IQjMBSNVs8AAABAXCHYhIOhaAAAAEBcItiEo4ZgAwAAAMQjgk2oDIOhaAAAAECcItiEqrlaaq0ztzOLrK0FAAAAQCcEm1AF7tak5UpJqdbWAgAAAKATgk2oAvNrGIYGAAAAxB2CTaiqWcMGAAAAiFcEm1AFhqLREQ0AAACIOwSbUNVwxwYAAACIVwSbUNHqGQAAAIhbBJtQdFzDhqFoAAAAQNwh2ISi8aTkaZJkkzLHWl0NAAAAgC4INqEI3K3JKJScbmtrAQAAANANwSYU1YfMZ4ahAQAAAHGJYBOK4PwaOqIBAAAA8YhgE4pAq2c6ogEAAABxiWATimrWsAEAAADiGcEmFLR6BgAAAOIawaY/Pp9UW25uMxQNAAAAiEsEm/7UH5O8bZLNIaUXWl0NAAAAgB4QbPoTGIaWOVZyOK2tBQAAAECPCDb9oSMaAAAAEPcINv1hDRsAAAAg7hFs+hNs9Tze0jIAAAAA9I5g058a1rABAAAA4h3Bpj/MsQEAAADiHsGmL952qfaIuc3inAAAAEDcItj0pe6IZHglh1sakWd1NQAAAAB6QbDpS3B+TZFk568KAAAAiFf8tt6XahoHAAAAAInAaXUBcW3SZdIXH5NcqVZXAgAAAKAPBJu+pOdLkxdZXQUAAACAfjAUDQAAAEDCI9gAAAAASHgEGwAAAAAJj2ADAAAAIOERbAAAAAAkPIINAAAAgIRHsAEAAACQ8Ag2AAAAABIewQYAAABAwiPYAAAAAEh4BBsAAAAACY9gAwAAACDhEWwAAAAAJDyCDQAAAICER7ABAAAAkPAINgAAAAASHsEGAAAAQMJzWl1AV4ZhSJLq6uosrgQAAACAlQKZIJAR+hJ3waa+vl6SVFRUZHElAAAAAOJBfX29MjMz+zzHZoQSf2LI5/Pp6NGjSk9Pl81ms7oc1dXVqaioSOXl5crIyLC6HCQQrh0MBtcPBoPrB4PB9YPBiPT1YxiG6uvrVVhYKLu971k0cXfHxm63a+zYsVaX0U1GRgb/48aAcO1gMLh+MBhcPxgMrh8MRiSvn/7u1ATQPAAAAABAwiPYAAAAAEh4BJt+uN1u3XXXXXK73VaXggTDtYPB4PrBYHD9YDC4fjAYVl4/cdc8AAAAAADCxR0bAAAAAAmPYAMAAAAg4RFsAAAAACQ8gg0AAACAhEewAQAAAJDwCDZ9+N3vfqeSkhIlJyfrvPPO0+uvv251SYhD69ev15VXXqnCwkLZbDY9/fTTnY4bhqFly5apsLBQKSkpKi0t1c6dO60pFnFlxYoVOv/885Wenq7c3FxdffXV2rt3b6dzuH7Qm/vvv19nn312cHXvuXPn6oUXXgge59pBOFasWCGbzaZbbrkluI9rCL1ZtmyZbDZbp0d+fn7wuFXXDsGmF48//rhuueUW3Xnnndq6dasuvvhiLVq0SGVlZVaXhjjT2NiomTNnauXKlT0ev/fee3Xfffdp5cqV2rRpk/Lz83XZZZepvr4+xpUi3qxbt05LlizRW2+9pTVr1qi9vV0LFy5UY2Nj8ByuH/Rm7Nixuueee7R582Zt3rxZH//4x3XVVVcFf3ng2kGoNm3apAcffFBnn312p/1cQ+jLtGnTdOzYseBjx44dwWOWXTsGenTBBRcYN954Y6d9U6ZMMW677TaLKkIikGQ89dRTwdc+n8/Iz8837rnnnuC+lpYWIzMz03jggQcsqBDx7MSJE4YkY926dYZhcP0gfFlZWcZ///d/c+0gZPX19cakSZOMNWvWGAsWLDC++93vGobBzx/07a677jJmzpzZ4zErrx3u2PSgra1NW7Zs0cKFCzvtX7hwoTZu3GhRVUhEBw8eVEVFRadrye12a8GCBVxL6Ka2tlaSlJ2dLYnrB6Hzer167LHH1NjYqLlz53LtIGRLlizRFVdcoU984hOd9nMNoT/79u1TYWGhSkpK9IUvfEEHDhyQZO2144zqpyeoU6dOyev1Ki8vr9P+vLw8VVRUWFQVElHgeunpWjp8+LAVJSFOGYahpUuX6qKLLtL06dMlcf2gfzt27NDcuXPV0tKiESNG6KmnntLUqVODvzxw7aAvjz32mN59911t2rSp2zF+/qAvc+bM0SOPPKIzzzxTx48f189+9jPNmzdPO3futPTaIdj0wWazdXptGEa3fUAouJbQn5tuuknbt2/XG2+80e0Y1w96M3nyZG3btk01NTV64okndP3112vdunXB41w76E15ebm++93v6uWXX1ZycnKv53ENoSeLFi0Kbs+YMUNz587VxIkT9fDDD+vCCy+UZM21w1C0HowaNUoOh6Pb3ZkTJ050S59AXwIdQriW0Jebb75Zzz77rF577TWNHTs2uJ/rB/1JSkrSGWecodmzZ2vFihWaOXOmfv3rX3PtoF9btmzRiRMndN5558npdMrpdGrdunX6zW9+I6fTGbxOuIYQirS0NM2YMUP79u2z9OcPwaYHSUlJOu+887RmzZpO+9esWaN58+ZZVBUSUUlJifLz8ztdS21tbVq3bh3XEmQYhm666SY9+eSTevXVV1VSUtLpONcPwmUYhlpbW7l20K9LL71UO3bs0LZt24KP2bNn60tf+pK2bdumCRMmcA0hZK2trdq9e7cKCgos/fnDULReLF26VF/5ylc0e/ZszZ07Vw8++KDKysp04403Wl0a4kxDQ4P2798ffH3w4EFt27ZN2dnZKi4u1i233KLly5dr0qRJmjRpkpYvX67U1FRde+21FlaNeLBkyRKtXr1azzzzjNLT04P/71ZmZqZSUlKCa0pw/aAnd9xxhxYtWqSioiLV19frscce09q1a/Xiiy9y7aBf6enpwfl8AWlpacrJyQnu5xpCb77//e/ryiuvVHFxsU6cOKGf/exnqqur0/XXX2/tz5+o9lxLcL/97W+NcePGGUlJSca5554bbMEKdPTaa68Zkro9rr/+esMwzLaHd911l5Gfn2+43W5j/vz5xo4dO6wtGnGhp+tGkrFq1argOVw/6M3Xvva14H+jRo8ebVx66aXGyy+/HDzOtYNwdWz3bBhcQ+jd5z//eaOgoMBwuVxGYWGhsXjxYmPnzp3B41ZdOzbDMIzoRicAAAAAiC7m2AAAAABIeAQbAAAAAAmPYAMAAAAg4RFsAAAAACQ8gg0AAACAhEewAQAAAJDwCDYAAAAAEh7BBgAAAEDCI9gAAAAASHgEGwAAAAAJj2ADAAAAIOH9f+llz0ddUZimAAAAAElFTkSuQmCC",
      "text/plain": [
       "<Figure size 1000x600 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Transition probabilities\n",
    "alpha = 0.01\n",
    "beta = 0.8\n",
    "\n",
    "# Discount parameter\n",
    "gamma = 0.7\n",
    "\n",
    "# Expected rewards\n",
    "r_search = 10.0\n",
    "r_wait = 5.0\n",
    "\n",
    "V = np.zeros(nb_states)\n",
    "Q = np.zeros((nb_states, nb_actions))\n",
    "\n",
    "pi = np.array([a['search'], a['search']], dtype=int)\n",
    "\n",
    "V_high_history = []\n",
    "V_low_history = []\n",
    "\n",
    "for k in range(50):\n",
    "    \n",
    "    # Policy evaluation\n",
    "    Q[s['high'], a['search']] = alpha * (r_search + gamma * V[s['high']]) + (1 - alpha) * (r_search + gamma*V[s['low']])\n",
    "    Q[s['high'], a['wait']] = r_wait + gamma * V[s['high']]\n",
    "\n",
    "    Q[s['low'], a['search']] = beta * (r_search + gamma * V[s['low']]) + (1 - beta) * (-3 + gamma*V[s['high']])\n",
    "    Q[s['low'], a['wait']] = r_wait + gamma * V[s['low']]\n",
    "    Q[s['low'], a['recharge']] = gamma * V[s['high']]\n",
    "\n",
    "    V[s['high']] = Q[s['high'], :2].max()\n",
    "    V[s['low']] = Q[s['low'], :].max()\n",
    "\n",
    "    V_high_history.append(V[s['high']])\n",
    "    V_low_history.append(V[s['low']])\n",
    "        \n",
    "    # Compute the greedy policy\n",
    "    pi_old = pi.copy()\n",
    "    pi[s['high']] = Q[s['high'], :2].argmax()\n",
    "    pi[s['low']] = Q[s['low'], :].argmax()\n",
    "    \n",
    "print('Greedy policy after iteration', k)\n",
    "print('pi(high)=', pi[s['high']])\n",
    "print('pi(low)=', pi[s['low']])\n",
    "print(\"V=\", V)\n",
    "print(\"Q=\", Q)\n",
    "    \n",
    "plt.figure(figsize=(10, 6))  \n",
    "plt.plot(V_high_history, label=\"high\")\n",
    "plt.plot(V_low_history, label=\"low\")\n",
    "plt.legend()\n",
    "plt.show()"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3.9.13 ('deeprl')",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.13"
  },
  "vscode": {
   "interpreter": {
    "hash": "932956c8e5d2f79d68ff59e849758b6e4ddbf01f7f22c7d8bb3532c38341d908"
   }
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}