{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "# Dynamic programming" ] }, { "cell_type": "code", "execution_count": 1, "metadata": {}, "outputs": [], "source": [ "import numpy as np\n", "import matplotlib.pyplot as plt" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The goal of this exercise is to find the optimal policy for the recycling robot.\n", " \n", "In this problem, a recycling robot has to search for empty cans to collect (each can defines a \"reward\" given to the robot). It can also decide to stay where it is to save its battery and wait that somebody brings it a can (which gives less cans in average than actively searching for them). \n", "\n", "The robot has two battery levels, *high* and *low*. \n", "\n", "* In the *high* level, the robot can either search or wait. \n", "\n", "* In the *low* state, three actions are possible: search, wait and recharge. \n", "\n", "State-action transitions are probabilistic, i.e. they bring the robot in different states based on different probabilities $\\alpha$ and $\\beta$.\n", "\n", "This problem defines a finite MDP, with two states *high* and *low* corresponding to the battery level. The actions *search* and *wait* are possible in the *high* and *low* states, while the action *recharge* is only possible in the *low* state.\n", "\n", "$$\n", "\\begin{aligned}\n", " \\mathcal{S} &=& \\{ \\text{high}, \\text{low} \\} \\\\\n", " \\mathcal{A}(\\text{high} ) &=& \\{ \\text{search}, \\text{wait} \\} \\\\\n", " \\mathcal{A}(\\text{low} ) &=& \\{ \\text{search}, \\text{wait}, \\text{recharge} \\}\n", "\\end{aligned}\n", "$$\n", "\n", "The action *search* brings on average a reward of $\\mathcal{R}^\\text{search}$, the action *wait* a reward of $\\mathcal{R}^\\text{wait}$, the action *recharge* brings no reward, but allows to get in the *high* state.\n", "\n", "Note that if the robot decides to search in the *low* state, there is a probability $1 - \\beta$ that it totally empties its battery, requiring human intervention. This is punished with a negative reward of -3.\n", "\n", "The transition and reward probabilities of each transition is defined in the following table, completely defining a MDP.\n", "\n", "| s | s' | a | p(s' / s, a) | r(s, a, s') |\n", "|:-------------:|:-------------:|:------------:|:-----------------:|:-----------------------------:|\n", "| high | high | search | $\\alpha$ | $\\mathcal{R}^\\text{search}$ |\n", "| high | low | search | $1 - \\alpha$ | $\\mathcal{R}^\\text{search}$ |\n", "| low | high | search | $1 - \\beta$ | $-3$ |\n", "| low | low | search | $\\beta$ | $\\mathcal{R}^\\text{search}$ |\n", "| high | high | wait | $1$ | $\\mathcal{R}^\\text{wait}$ |\n", "| high | low | wait | $0$ | $\\mathcal{R}^\\text{wait}$ |\n", "| low | high | wait | $0$ | $\\mathcal{R}^\\text{wait}$ |\n", "| low | low | wait | $1$ | $\\mathcal{R}^\\text{wait}$ |\n", "| low | high | recharge | $1$ | $0$ |\n", "| low | low | recharge | $0$ | $0$ |\n", "\n", "The goal of this exercise is to find the optimal policy $\\pi^*$ of the robot, i.e to find for each state the action that should be performed systematically in order to gather the maximum of reward on the long term. \n", "\n", "We will apply here two **dynamic programming** methods, policy iteration and value iteration, to solve the Bellman equations.\n", "\n", "The Bellman equation for the state function is:\n", "\n", "$$V^{\\pi} (s) = \\sum_{a \\in \\mathcal{A}(s)} \\pi(s, a) \\, \\sum_{s' \\in \\mathcal{S}} p(s' | s, a) \\, [ r(s, a, s') + \\gamma \\, V^{\\pi} (s') ]$$\n", "\n", "**Q:** On paper, adapt the Bellman equation to the problem. First, for every state $s$ and possible action $a$, find the optimal value of the action with the form:\n", "\n", "$$Q^{\\pi} (s, a) = f( V^\\pi (\\text{high}), V^\\pi (\\text{low}), \\alpha, \\beta, \\gamma, \\mathcal{R}^{\\text{search}}, \\mathcal{R}^{\\text{wait}} )$$\n", "\n", "Deduce the Bellman equation for the two states $V^\\pi (\\text{high})$ and $V^\\pi (\\text{low})$." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "**A:**\n", "\n", "$$Q^\\pi(\\text{high}, \\text{search}) = \\alpha \\, (\\mathcal{R }^\\text{search} + \\gamma \\, V^\\pi(\\text{high} )) + (1- \\alpha)\\, (\\mathcal{R}^\\text{search} + \\gamma \\, V^\\pi(\\text{low}))$$\n", "\n", "$$Q^\\pi(\\text{high}, \\text{wait}) = \\mathcal{R}^\\text{wait} + \\gamma \\, V^\\pi(\\text{high})$$\n", "\n", "$$Q^\\pi(\\text{low}, \\text{search}) = \\beta * (\\mathcal{R }^\\text{search} + \\gamma \\, V^\\pi(\\text{low})) + (1- \\beta) \\, (-3 + \\gamma \\, V^\\pi(\\text{high}))$$\n", "\n", "$$Q^\\pi(\\text{low}, \\text{wait}) = \\mathcal{R}^\\text{wait} + \\gamma \\, V^\\pi(\\text{low})$$\n", "\n", "$$Q^\\pi(\\text{low}, \\text{recharge}) = \\gamma \\, V^\\pi(\\text{high})$$\n", "\n", "$$V^\\pi(\\text{high}) = \\pi(\\text{high}, \\text{search}) \\, Q^\\pi(\\text{high}, \\text{search}) + \\pi(\\text{high}, \\text{wait}) \\, Q^\\pi(\\text{high}, \\text{wait})$$\n", "\n", "$$V^\\pi(\\text{low}) = \\pi(\\text{low}, \\text{search}) \\, Q^\\pi(\\text{low}, \\text{search}) + \\pi(\\text{low}, \\text{wait}) \\, Q^\\pi(\\text{low}, \\text{wait}) + \\pi(\\text{low}, \\text{recharge}) \\, Q^\\pi(\\text{low}, \\text{recharge})$$\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Policy Iteration\n", "\n", "Now that we have the Bellman equations for the two states high and low, we can solve them using **iterative policy evaluation** for a fixed policy $\\pi$. \n", "\n", "### Iterative policy evaluation\n", "\n", "Let's start by setting the parameters of the MDP. In the rest of the exercise, you will modify these parameters to investigate how it changes the optimal policy." ] }, { "cell_type": "code", "execution_count": 2, "metadata": {}, "outputs": [], "source": [ "# Transition probabilities\n", "alpha = 0.3\n", "beta = 0.2\n", "\n", "# Discount parameter\n", "gamma = 0.7\n", "\n", "# Expected rewards\n", "r_search = 6.0\n", "r_wait = 2.0" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "There are many ways to represent states and actions in a MDP. The suggestion for this exercise is to use dictionaries here the keys are the actions' name and the vaues are indices:" ] }, { "cell_type": "code", "execution_count": 3, "metadata": {}, "outputs": [], "source": [ "nb_states = 2\n", "nb_actions = 3\n", "\n", "s = {'high': 0, 'low': 1}\n", "a = {'search': 0, 'wait': 1, 'recharge': 2}" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Using dictionaries, you can access numpy arrays with `s['high']` or `a['recharge']` instead of 0 and 2, what will make the code readable.\n", "\n", "The next step is to initialize numpy arrays where we will store the V and Q values. `V` will have only two elements for high and low, while `Q` will be a 2x3 matrix with one element for each state-action pair. Notice that (high, recharge) is not a possible action, so this element will not be be updated." ] }, { "cell_type": "code", "execution_count": 4, "metadata": {}, "outputs": [], "source": [ "V = np.zeros(nb_states)\n", "Q = np.zeros((nb_states, nb_actions))" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "You can then access the individual values with `V[s['high']]` or `Q[s['low'], a['wait']]`.\n", "\n", "We can now evaluate a policy $\\pi$. In dynamic programming, the policies are deterministic, as we want to estimate the optimal policy.\n", "\n", "To implement the policy, we just need to assign the index of an action to each state, i.e. $\\pi(s)$. The following cell creates an initial policy $\\pi$ where the agent **searches** in both states high and low. We here make sure that the array contains integers (0, 1 or 2), but that is not even necessary." ] }, { "cell_type": "code", "execution_count": 5, "metadata": {}, "outputs": [], "source": [ "pi = np.array([a['search'], a['search']], dtype=int)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "**Q:** Evaluate this policy using iterative policy evaluation.\n", "\n", "We would normally only need to update the V-value of the two states using:\n", "\n", "$$\n", " V (s) \\leftarrow \\sum_{a \\in \\mathcal{A}(s)} \\pi(s, a) \\, \\sum_{s' \\in \\mathcal{S}} p(s' | s, a) \\, [ r(s, a, s') + \\gamma \\, V (s') ] \\quad \\forall s \\in \\mathcal{S}\n", "$$\n", "\n", "The code will be more readable if you first update the Q-values of the 5 state-action pairs:\n", "\n", "$$\n", " Q (s, a) \\leftarrow \\sum_{s' \\in \\mathcal{S}} p(s' | s, a) \\, [ r(s, a, s') + \\gamma \\, V (s') ] \\quad \\forall s \\in \\mathcal{S}\n", "$$\n", "\n", "and only then update the two V-values:\n", "\n", "$$\n", " V (s) \\leftarrow \\sum_{a \\in \\mathcal{A}(s)} \\pi(s, a) \\, Q(s, a)\n", "$$\n", "\n", "These updates should normally be applied until the V-values converge. For simplicity, we could decide to simply apply 50 updates or so, and hope that it is enough.\n", "\n", "Record the V-value of the two states after each update and plot them. " ] }, { "cell_type": "code", "execution_count": 6, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "[[11.28888873 9.90222206 0. ]\n", " [ 5.9555554 6.16888873 7.90222206]]\n" ] }, { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAzYAAAH5CAYAAABTbqsJAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjkuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8hTgPZAAAACXBIWXMAAA9hAAAPYQGoP6dpAAA/kklEQVR4nO3deXxU9b3/8ffMJJksJIEEyAIJBEEQZJFVRAUVtagU1LpUe4van13EBbl1QUWrLaJtbdHWuvXWpYprL7hcN0oFVFZZFBUim+whbNn3mfP74yRDAgESMpPvnMzr+eh5zPcsM+eTeIzz7jnnc1yWZVkCAAAAAAdzmy4AAAAAAFqKYAMAAADA8Qg2AAAAAByPYAMAAADA8Qg2AAAAAByPYAMAAADA8Qg2AAAAABwvynQBh/P7/dq1a5cSExPlcrlMlwMAAADAEMuyVFxcrMzMTLndxz4nE3bBZteuXcrKyjJdBgAAAIAwsX37dnXt2vWY24RdsElMTJRkF5+UlGS4GgAAAACmFBUVKSsrK5ARjiXsgk3d5WdJSUkEGwAAAABNukWF5gEAAAAAHI9gAwAAAMDxCDYAAAAAHC/s7rFpKp/Pp+rqatNlOEp0dLQ8Ho/pMgAAAICgc1ywsSxLeXl5KigoMF2KI7Vv317p6ek8IwgAAABtiuOCTV2o6dy5s+Lj4/mC3kSWZamsrEz5+fmSpIyMDMMVAQAAAMHjqGDj8/kCoSY1NdV0OY4TFxcnScrPz1fnzp25LA0AAABthqOaB9TdUxMfH2+4Eueq+91xfxIAAADaEkcFmzpcfnbi+N0BAACgLXJksAEAAACA+gg2rWTMmDGaMmXKUde7XC7NnTu3yZ+3YMECuVwuusMBAAAAcljzgLZs9+7d6tChg+kyAAAAAEci2ISJ9PR00yUAAAAAjsWlaK3I7/frzjvvVEpKitLT0/Wb3/wmsO7wS9EWL16sQYMGKTY2VkOHDtXcuXPlcrm0Zs2aBp+5cuVKDR06VPHx8TrjjDOUm5vbOj8MAAAAEEYcf8bGsiyVV/uM7Dsu2tOsLmMvvviipk6dqmXLlmnJkiW67rrrNGrUKJ1//vkNtisqKtL48eN10UUXafbs2dq6detR78+599579dhjj6lTp0765S9/qRtuuEGff/55S34sAAAAwHEcH2zKq33qe/9HRvb97UMXKj6m6b/CAQMG6IEHHpAk9erVS3/96181f/78I4LN7Nmz5XK59Nxzzyk2NlZ9+/bVzp07deONNx7xmTNmzNDo0aMlSXfffbcuvvhiVVRUKDY2tgU/GQAAAOAsXIrWigYMGNBgPiMjQ/n5+Udsl5ubqwEDBjQIJ8OHDz/uZ2ZkZEhSo58JAAAAtGWOP2MTF+3Rtw9daGzfzREdHd1g3uVyye/3t6iG+p9Zd1lcSz8TCDeWZcnnt1Tjt1Tt86vGZ499fks+y5LPZ6nG75ffspfX+KzA2FdvqvFb8teO/VbdpAbzPr/ssb92XWBsz1uWJctSYN5vWZIU2N5vWbLqtlW9bez/ye+3ZEmyatdbdZ9Zu8xfb6zA+nrb1nuvrNrfjxp+hlXv92YFfom17wl8dr3POex99bepv9SyGi6xrHrrGn3vkesOf19jGlttHfEpjeznBN/XVCf4tiC8+UR218o7RMic6PEKBMOfrxqkrJR402U0meODjcvlatblYE7Qu3dvvfzyy6qsrJTX65UkrVixwnBViFQ+v6WyqhqVV/lUXu1TWZU9lVf57OXVPlVW+1Xp86uy2qcqn1+V1f56r77D5v2qrPGpqsZvTz5LNT5/g9BS7fPb49qQUu338x93AABaWWWNmfvYT1TbSgRtxDXXXKN7771XP//5z3X33Xdr27Zt+uMf/yhJzWpWgMhV11SjqLxGxRXVKqqoUVFFtYor7PniihoVlTecL66oUVl1Tb3QYgeZqprwPQMY5XbJ43Ypyu2Su/bV43bL45ai3G55atcHtnHZY7fbJY9Lcrvssdsle7mr3jZ162vnXYF5+9VVO3YdNm+PD7261HA71S6z5w+NXbL//XYdsezQv/eNrtOhdar7jPrzqrf8sGV1C1z1Zg//vMPX2/MNP6j+Pg5/z2G7anTedcTWR25/NMf6m3isjzjW5zdWT1PedzxO+uvNf2oASFJakrPu2SbYhKGkpCS9++67+tWvfqVBgwapf//+uv/++3XNNdfQFCBC+f2WCsqrtb+kUvtKqrS/tFL7S6rs+VL7dX9JlfaXVulgWZWKK2rk8wf3FIfLJcVHexQXE6X4GI/ioj2Kq/ca43HLG+2u9+o5bN4tb7RH3nrzMVFuRXnciva4FO1xK8ptv0Z73IryuBTttl+jPC7FeOxt67bxuPnmBQAADiHYtJIFCxYcsaz+c2sOv+b8jDPO0JdffhmYf+WVVxQdHa3s7GxJ0pgxY454z6BBg4577TrCT3mVTzsLyrTjYLl2FpRrx8Fy7Soo197iurBSqQOlVTqRnOJxu5QYG6Wk2GglxkbVTtGB+aTYKCXF2eN23mjFez2Kj/YoPibKDi0xntow45E3ys0ZQwAAELYINmHqpZdeUo8ePdSlSxd9+eWXuuuuu3TllVcqLi7OdGlopsLyau08WK4dB8u0s6BcO+sFmJ0F5TpQWtXkz2ofH63UhBiltvOqY7sYpSZ4ldrOnk9NiFFqQow6JMQoKTZaSXFRzX7WEgAAgFMRbMJUXl6e7r//fuXl5SkjI0NXXHGFZsyYYbosHMP+kkrl5hVrfV6xcvOKlbunWJv2lqi4oua47030RqlLhzh17RCnLu3j1KVDnNKSYg8Fl9rAEu2hQzsAAEBjCDZh6s4779Sdd95pugw0oqyqRt/tKdF3dSFmT5Fy84q1r+ToZ15SEmIOhZba4NK1Q3xgnBwXfdT3AgAA4PgINsAxHCyt0vLvD+jrnYVan1es7/YUa9uBskZbD7tcUnZKvHqnJap3uj2dnJaorh3i2lxLcgAAgHDDty2gngOlVVq+Zb+Wbj6gpZv3a31ecaPbdWznVe/0duqdlqQ+tSGmV1o7AgwAAIAhfAtDRNtXUqnlW+wQs2zzAeXuOTLI9OrcToOzO6hPRmLgbExqO6+BagEAAHA0BBtElL3Fh4LM0s37tSG/5IhteqclakSPFJ3eI1XDc1LUkRADAAAQ9gg2aPO27CvV3NU79f7a3Y0GmT7piTq9R6pO75GiYd1TOBsDAADgQAQbtEl7iyv13le7NHfNLn25vSCw3OWS+qQn6fQeKRqRk6oROSnqkBBjrlAAAAAEBcGmlYwZM0aDBg3SrFmzTJfSZpVV1ejjb/Zozuqd+mzjPvn8dusyj9uls3p11MRBXTSmdye1jyfIAAAAtDUEGzhajc+vTzfu09urd+qjb/aovNoXWDcwq70uHZSpSwZmcp8MAABAG0ewgeNYlqUvdxRq7uqdeu+rXQ0ejNktNV4TB3XRxNO6KKdjgsEqAQAA0JrcpguIRAcPHtRPf/pTdejQQfHx8Ro3bpw2bNggyf7S3qlTJ7311luB7QcNGqSMjIzA/GeffSav16uysrJWr92k8iqfnlu0Wec+tlATn/xcLyz+XvtKqpSaEKNJI7tpzk1naMGvx+j2808m1AAAAEQY55+xsSyp2tAX/Oh4+270Zrruuuu0YcMGvfPOO0pKStJdd92liy66SN9++62io6N19tlna8GCBfrRj36kgwcPat26dYqLi9P69evVp08fLVy4UMOGDVN8fHwIfqjwU17l0yvLturphZsCZ2dio926oG+6Lj2ti87s1VHRHjI6AABAJHN+sKkukx7ONLPve3ZJMc07M1AXaD7//HOdccYZkqRXXnlFWVlZmjt3rq644gqNGTNGzzzzjCRp0aJFOu2005Senq4FCxaoT58+WrBggUaPHh30HyfcVFT79MqybXp64SbtLa6UJHXtEKfJ5/TU+IGZaud1/uELAACA4OCbYStbt26doqKiNGLEiMCy1NRU9e7dW+vWrZMkjR49Wrfddpv27t2rhQsXasyYMYFg87Of/UyLFy/WnXfeaepHCLmKap9eXb5NTy3YpPzaQNOlfZxuObenLh/SlbMzAAAAOILzg010vH3mxNS+Q6B///5KSUnRwoULtXDhQs2YMUPp6el69NFHtWLFClVXVwfO9rQlFdU+vbZ8m55auEl7ig4FmpvP7anLB3dVTBSBBgAAAI1zfrBxuZp9OZhJp5xyimpqarRs2bJAONm/f79yc3PVt29fSZLL5dJZZ52lt99+W998843OPPNMxcfHq7KyUs8884yGDh2qhATn/MzHU1nj0+srtutvn2xSXlGFJCkzOVaTz+2pK4ZkEWgAAABwXM4PNg7Tq1cvTZgwQTfeeKOeeeYZJSYm6u6771aXLl00YcKEwHZjxozRf//3f2vo0KFq166dJOnss8/WK6+8ojvuuMNU+UFVWePTG1/s0N8+2ajdhXagyUiO1U3n9NSVQ7vKG+UxXCEAAACcgmBjwPPPP6/bbrtNl1xyiaqqqnT22Wfr/fffV3R0dGCb0aNHy+fzacyYMYFlY8aM0dtvv91gmRP5/Zbe+GK7npi/QbtqA01akleTz+mpq4ZlEWgAAADQbC7LsizTRdRXVFSk5ORkFRYWKikpqcG6iooKbdmyRTk5OYqNjTVUobOZ/h0eKK3Sr9/8Uv9Zny9J6pzo1U1jTtLVw7MVG02gAQAAwCHHygaH44wNWs2yzft122trlFdUoZgot+64oLf+a2Q3Ag0AAABajGCDkPP5Lf3tk43687+/k9+SenRK0JPXDNYpGcdO3QAAAEBTEWwQUvnFFbr99TX6fON+SdLlg7vqoQn9lMDDNQEAABBEfLtEyHy6Ya9uf32N9pVUKS7ao99OPFU/GtLVdFkAAABogwg2CLoan19/mvednlq4SZYl9UlP1F+vGayenduZLg0AAABtlCODjd/vN12CY4X6d7eroFy3vrpaX2w9KEm6dkS2pl/SlwYBAAAACClHBZuYmBi53W7t2rVLnTp1UkxMjFwul+myHMGyLFVVVWnv3r1yu92KiYkJ+j7mfbtHd7z1pQrKqpXojdIjlw/QxQMygr4fAAAA4HCOCjZut1s5OTnavXu3du3aZbocR4qPj1d2drbcbnfQPrOqxq9HPlivf3y+RZI0oGuy/vrjwcpOjQ/aPgAAAIBjcVSwkeyzNtnZ2aqpqZHP5zNdjqN4PB5FRUUF9SzX1v2luuXV1fpqR6Ek6Wdn5uiuH/RRTFTwghMAAABwPI4LNpLkcrkUHR2t6Oho06VEtE9y83XL7NUqqaxR+/ho/fFHAzW2b5rpsgAAABCBHBlsYN78dXv0q5dXqcrn19BuHfTEj09TZvs402UBAAAgQhFs0Gzz1+3RL19eqWqfpYv6p+vxq09TtIdLzwAAAGAO30bRLP/+9lCoubh/BqEGAAAAYYFvpGiyed/u0a9eORRqZl09iFADAACAsMClaGiSed/u0U11oWZAhh6/apCiCDUAAAAIE83+Zrpo0SKNHz9emZmZcrlcmjt3boP1lmXp/vvvV0ZGhuLi4jR27Fht2LAhWPXCgI+/yQuEmksINQAAAAhDzf52WlpaqoEDB+rJJ59sdP3vf/97PfHEE3r66ae1bNkyJSQk6MILL1RFRUWLi0Xr+/ibPE2evUrVPkvjB2ZqFqEGAAAAYajZl6KNGzdO48aNa3SdZVmaNWuW7rvvPk2YMEGS9NJLLyktLU1z587V1Vdf3bJq0ao++iZPk19ZpRq/HWr+fOVAQg0AAADCUlC/pW7ZskV5eXkaO3ZsYFlycrJGjBihJUuWNPqeyspKFRUVNZhg3odfHwo1PyTUAAAAIMwF9ZtqXl6eJCktreHT59PS0gLrDjdz5kwlJycHpqysrGCWhBPw4de7dfNsO9RMGJSpPxFqAAAAEOaMf1udNm2aCgsLA9P27dtNlxTRPli7WzfPXh0INY9dQagBAABA+Atqu+f09HRJ0p49e5SRkRFYvmfPHg0aNKjR93i9Xnm93mCWgRP0wdrduvnV1fL5LU0clKnHrhwkj9tluiwAAADguIL6f8Xn5OQoPT1d8+fPDywrKirSsmXLNHLkyGDuCkH2fr1Qc+lpXQg1AAAAcJRmn7EpKSnRxo0bA/NbtmzRmjVrlJKSouzsbE2ZMkW/+93v1KtXL+Xk5Gj69OnKzMzUxIkTg1k3guj9tbt1S71Q88crBhJqAAAA4CjNDjZffPGFzjnnnMD81KlTJUmTJk3SCy+8oDvvvFOlpaX6+c9/roKCAp155pn68MMPFRsbG7yqETRb95fq9tfXyOe3dNlpXfQHQg0AAAAcyGVZlmW6iPqKioqUnJyswsJCJSUlmS6nTbMsSz/9x3J9umGfRvZI1cv/bwShBgAAAGGjOdmAdlcR7J0vd+nTDfsUE+XWjEtPJdQAAADAsQg2EaqwrFq/fe9bSdLN5/RUj07tDFcEAAAAnDiCTYR69KP12ldSpZM6JegXo3uYLgcAAABoEYJNBFq59YBmL9smSXr40v7yRnkMVwQAAAC0DMEmwlT7/Lrnf7+WJF0xpKtG9Eg1XBEAAADQcgSbCPPcp5uVu6dYKQkxuueiU0yXAwAAAAQFwSaCbNtfpsf/vUGSdO9Fp6hDQozhigAAAIDgINhECMuydN/bX6uyxq+RPVJ12eAupksCAAAAgoZgEyHe/Wq3Fn23VzEe+5k1LhfPrAEAAEDbQbCJAIVl1XroXfuZNZN5Zg0AAADaIIJNBLCfWVOpHp0S9MsxPLMGAAAAbQ/Bpo3jmTUAAACIBASbNuzwZ9aczjNrAAAA0EYRbNownlkDAACASEGwaaN4Zg0AAAAiCcGmDeKZNQAAAIg0BJs26D2eWQMAAIAIQ7BpYwrLq/Ugz6wBAABAhCHYtDG//5Bn1gAAACDyEGzakJVbD+oVnlkDAACACESwaSPsZ9aslcQzawAAABB5CDZtxEtLtvLMGgAAAEQsgk0bUFBWpSfm28+suePC3jyzBgAAABGHYNMGPD5/gwrLq9UnPVFXDs0yXQ4AAADQ6gg2Drd5b4n+uWSrJOnei0+Rx80zawAAABB5CDYO98gH61Xjt3RO7046q1cn0+UAAAAARhBsHGzJpv36+Ns98rhdNAwAAABARCPYOJTfb2nG+99Kkn48PEu90hINVwQAAACYQ7BxqDmrd+rrnUVK9EZpytiTTZcDAAAAGEWwcaCyqhr94aNcSdLkc3uqYzuv4YoAAAAAswg2DvTcoi3KK6pQ1w5xuu6M7qbLAQAAAIwj2DjMnqIKPb1wkyTprh/0UWy0x3BFAAAAgHkEG4d57ONclVf7dFp2e10yIMN0OQAAAEBYINg4yDe7CvXmyh2SpOmX9JXLxcM4AQAAAIlg4xiWZWnG/62TZUnjB2ZqcHYH0yUBAAAAYYNg4xD/WZ+vxZv2KybKrTsv7G26HAAAACCsEGwcoNrn14z310mSbhiVo6yUeMMVAQAAAOGFYOMAs5dt0+a9pUpNiNFN55xkuhwAAAAg7BBswlxhebVm/fs7SdKU809WUmy04YoAAACA8EOwCXNPfrJRB8uq1bNzO/14WJbpcgAAAICwRLAJY1v3l+qFz7+XJN178SmK8vCPCwAAAGgM35TD2KMfrleVz6+zenXUmJM7mS4HAAAACFsEmzC14vsDen9tntwu+2wND+MEAAAAjo5gE4b8fku/+z+7vfNVw7LUJz3JcEUAAABAeCPYhKF3v9qlL7cXKCHGo9vPP9l0OQAAAEDYI9iEmYpqnx79YL0k6aZzeqpzYqzhigAAAIDwR7AJM//4fIt2FVYoMzlWPzszx3Q5AAAAgCMQbMLMO2t2SZJuG9tLsdEew9UAAAAAzkCwCSMHSqu0Pq9YknTeKWmGqwEAAACcg2ATRpZt3i9JOjmtnTq28xquBgAAAHAOgk0YWVobbE7vkWq4EgAAAMBZCDZhZOnmA5KkkQQbAAAAoFkINmFif0mlcvfY99cMz0kxXA0AAADgLASbMLF8i322pndaolK5vwYAAABoFoJNmDh0fw1nawAAAIDmItiEiSU0DgAAAABOWNCDjc/n0/Tp05WTk6O4uDiddNJJ+u1vfyvLsoK9qzZjX0mlvttTIkkaQbABAAAAmi0q2B/46KOP6qmnntKLL76ofv366YsvvtD111+v5ORk3XrrrcHeXZtQd39Nn/REpSTEGK4GAAAAcJ6gB5vFixdrwoQJuvjiiyVJ3bt316uvvqrly5cHe1dtBs+vAQAAAFom6JeinXHGGZo/f76+++47SdKXX36pzz77TOPGjWt0+8rKShUVFTWYIg2NAwAAAICWCfoZm7vvvltFRUXq06ePPB6PfD6fZsyYoWuvvbbR7WfOnKkHH3ww2GU4Rv37a4bncMYGAAAAOBFBP2Pzxhtv6JVXXtHs2bO1atUqvfjii/rjH/+oF198sdHtp02bpsLCwsC0ffv2YJcU1pZt5v4aAAAAoKWCfsbmjjvu0N13362rr75aktS/f39t3bpVM2fO1KRJk47Y3uv1yuuN3AdScn8NAAAA0HJBP2NTVlYmt7vhx3o8Hvn9/mDvqk0g2AAAAAAtF/QzNuPHj9eMGTOUnZ2tfv36afXq1frTn/6kG264Idi7crx9JZXakF/7/JocGgcAAAAAJyroweYvf/mLpk+frptuukn5+fnKzMzUL37xC91///3B3pXj1b+/pgP31wAAAAAnLOjBJjExUbNmzdKsWbOC/dFtzpLN+yRxGRoAAADQUkG/xwZNt7T2jM3Ikwg2AAAAQEsQbAzZW1ypjfklcrm4vwYAAABoKYKNIcu22N3Q+qQnqX0899cAAAAALUGwMeRQm2fO1gAAAAAtRbAxZMkmnl8DAAAABAvBxoD84gpt2lvK/TUAAABAkBBsDKh7fs0p3F8DAAAABAXBxoBD99dwGRoAAAAQDAQbA2gcAAAAAAQXwaaV5Rcdur9mOPfXAAAAAEFBsGllS7dwfw0AAAAQbASbVlZ3GdrIk7i/BgAAAAgWgk0ro3EAAAAAEHwEm1aUX1ShzXX313Tn/hoAAAAgWAg2raju/pq+GUlKjo82XA0AAADQdhBsWtGSTVyGBgAAAIQCwaYVLeP+GgAAACAkCDatZE9RhTbv4/k1AAAAQCgQbFpJXTe0fplJSo7j/hoAAAAgmAg2rWTpZrtxwOk5XIYGAAAABBvBppXw/BoAAAAgdAg2rSCvsEJb9pXK7ZKGcX8NAAAAEHQEm1awbEvd/TXJ3F8DAAAAhADBphUcugyNszUAAABAKBBsWkGgcQD31wAAAAAhQbAJsfr31wztzhkbAAAAIBQINiF26Pk13F8DAAAAhArBJsTqgs3Ik7gMDQAAAAgVgk2I0TgAAAAACD2CTQjtLizX9/vLuL8GAAAACDGCTQgtq+2GdmqXZCXFcn8NAAAAECoEmxBasqnuMjTurwEAAABCiWATQku3cH8NAAAA0BoINiGyq6BcW2vvrxnG/TUAAABASBFsQmRZ7dma/l2Slcj9NQAAAEBIEWxCZOkmu3EA99cAAAAAoUewCZFD99cQbAAAAIBQI9iEwOcb92nr/jJ53C4N7d7BdDkAAABAm0ewCbKKap/umbNWknTtiGzurwEAAABaAcEmyGb9e4O27i9TelKs7riwt+lyAAAAgIhAsAmib3YV6rlPN0uSfjvxVM7WAAAAAK2EYBMkPr+laf+7Vj6/pYv6p+v8vmmmSwIAAAAiBsEmSF5Y/L2+2lGoxNgo/WZ8P9PlAAAAABGFYBMEOw6W6bGPcyVJ08ados5JsYYrAgAAACILwaaFLMvSfXO/VlmVT8O7p+jqYVmmSwIAAAAiDsGmhd75cpcW5O5VjMethy/rL7fbZbokAAAAIOIQbFrgYGmVHnr3W0nSzef2VM/O7QxXBAAAAEQmgk0LzHh/nfaXVunktHb65eiTTJcDAAAARCyCzQn6fOM+vbVyh1wuaeZlAxQTxa8SAAAAMIVv4yegotqne+aslST9ZEQ3DenWwXBFAAAAQGQj2JyAx+dv0Nb9ZUpPitWdP+htuhwAAAAg4hFsmunbXUV6dtFmSdJDE/opMTbacEUAAAAACDbN4PNbmva/X8nntzTu1HRd0C/ddEkAAAAARLBplhcWf68vdxQqMTZKD/6wn+lyAAAAANQi2DTRjoNleuzjXEnStHGnqHNSrOGKAAAAANQh2DSBZVm6b+7XKqvyaXj3FF09LMt0SQAAAADqCUmw2blzp37yk58oNTVVcXFx6t+/v7744otQ7KpVvPPlLi3I3asYj1sPX9ZfbrfLdEkAAAAA6okK9gcePHhQo0aN0jnnnKMPPvhAnTp10oYNG9ShgzOf9XKwtEoPvfutJGnyOT3Vs3M7wxUBAAAAOFzQg82jjz6qrKwsPf/884FlOTk5wd5Nq5nx/jrtL61Sr87t9KsxJ5kuBwAAAEAjgn4p2jvvvKOhQ4fqiiuuUOfOnXXaaafpueeeO+r2lZWVKioqajCFi8Ub9+mtlTvkckmPXN5fMVHckgQAAACEo6Cfsdm8ebOeeuopTZ06Vffcc49WrFihW2+9VTExMZo0adIR28+cOVMPPvhgsMsIio+/3SNJ+smIbhrSLcVwNQBCzrIky29Pfl/t2FdvbEmqt01gOnxZ7XyDbeu/t25sNRw3tv64r6odq+G6up/nqGM18hl1YzWy7dGW6ejbNbrusPWNfdbh72/qdi36rKY40fe1ZJ8nvMNW3h/CUqsfd2hzTr1MinPO7SQuywruUR8TE6OhQ4dq8eLFgWW33nqrVqxYoSVLlhyxfWVlpSorKwPzRUVFysrKUmFhoZKSkoJZWrNZlqWPvsnTqJ4dlRgbbbQWIGxZluSrlnyVUk3dVCH5quzXmqrD5mu38VXa63xVkr9a8tXUvtZO/vqvNUdu56+xA4ffVzuuORRCAutqX6162wRCSiPhhS+DAAAcMnm51Km30RKKioqUnJzcpGwQ9DM2GRkZ6tu3b4Nlp5xyiv71r381ur3X65XX6w12GUHhcrn0g1MzTJcBBI9lSdVlUkWRVFEoVRbZ48pC+7W6TKoqk6pLa1/LGi6rLj9sfbk9tvymf7JW5pJc7tqp/rh2kqvecle9ZbXz9dfLJbnU+Pomvdav6bB1Uu24/no1vi4wf/i6xrbXYfPHeM8R2zay/oh1jby/Odsdc/sQvu94jvm5ABCGYpzVNCvowWbUqFHKzc1tsOy7775Tt27dgr0rIHJZllRRIJXul0r31pv2SeUHasNKbXgJBJja8GL5QlubO1qKipWiYuxXT8xR5r322BNtT+7DXgPjqNrXmHrjaMkdJbk9ta9RkstTb9nhy+utq5uvCyGBseew+UbWBYIKX1ABAAg3QQ82t99+u8444ww9/PDDuvLKK7V8+XI9++yzevbZZ4O9K6DtqSyWCrZLhTukkj1S2T47rBweXkr32ZdjnSiXR4pNkrxJUmyyPXkTpZgEKTr+0Gt0XL1xvBQTX2993KHl0XG1QcUruWmyAQAAWl/Q77GRpPfee0/Tpk3Thg0blJOTo6lTp+rGG29s0nubcx0d4Ch1Z1kKttnhpWCbVFj7WjcuP9i8z/QmSQkdpYROUnxHexyfUi+sJB02rp2PjuesAwAACHvNyQYhCTYtQbCB45UdkPK/lfLXSfu+OxRcCrZLVcXHf39sspScLSWm24EloeOh8BKYrw0y0bGh/3kAAAAMMdo8AIgYVWXS3vV2gMn/1p72fCuV5B37fQmdpOQsqX221D7LDjHta+eTs+yzKgAAAGgWgg1wPL4a6cBmKf8bO7jUhZgDW3TU9sDts6XO/aTOfaT23WqDSzcpuat9PwoAAACCimADHM5XI+3+Uvr+U+n7z6RtS6Sqksa3jU+VOveV0vrZr5372mHGm9i6NQMAAEQ4gg3QlCATHS91PqV26me/pvWT2nU2UzMAAAAaINgg8jQlyMQmS93OlLrXTmn97GeaAAAAICwRbBAZDmyW1r0rbfn0KEGmvdRtFEEGAADAoQg2aLtK9krfzJHWviHtWNFwXWz7QyGm+5n25WU8WBIAAMCxCDZoWyqLpfX/J619U9r0iWT57OUut5RztnTyDwgyAAAAbRDBBs5XUyVtmm+HmfXvSzXlh9ZlDpYGXCn1u9R+4CUAAADaJIINnMnvl7YvtcPMN3Ok8oOH1qWcZIeZ/ldIqSeZqxEAAACthmADZ9m/SVr1kvT1v6TC7YeWt0uTTr3cDjOZp0kul7kaAQAA0OoINnCGg1ulRb+X1rx66L4Zb5J0yg+l/j+y75+hixkAAEDEItggvBXtkhb90T5L46+2l/UcK532X9LJF0rRcWbrAwAAQFgg2CA8leyVPvuztOLvkq/SXtZjjHTOfVLWMKOlAQAAIPwQbBBeyg5Ii5+Qlj0jVZfZy7JHSufeZ7dpBgAAABpBsEF4qCiUlvxNWvo3qbLIXpY52A40J51LMwAAAAAcE8EGZlWWSMuflT5/XKoosJel9ZfOvdd+mCaBBgAAAE1AsIEZ1eXSF/+QPv2TVLbPXtbxZOmce6RTJkhut9n6AAAA4CgEG7S+zQukuZOloh32fIccacw0u20zLZsBAABwAgg2aD01ldJ/fist/os9n9RVGn2nNOgayRNttjYAAAA4GsEGrWNvrvSvn0l5a+35oTdIF/xOikkwWxcAAADaBIINQsuy7GfRfHyfVFMhxadKP/yr1Oci05UBAACgDSHYIHRK9kpvT5Y2fGTPn3SeNPFvUmK62boAAADQ5hBsEBrffSy9fZNUulfyeKXzH5KG/5xuZwAAAAgJgg2Cq7pc+ni6tOI5e75zX+nyv0tp/czWBQAAgDaNYIPgyVsr/ev/SXvX2/MjfiWN/Y0UHWu0LAAAALR9BBu0nN8vLf2bNP9ByVcltUuz76XpOdZ0ZQAAAIgQBBu0TNEuae6v7IduSlLvi6Qf/kVK6Gi0LAAAAEQWgg1O3JZF0hs/lcoPSlFx0g8eloZcL7lcpisDAABAhCHY4MTsWCnNvlqqLpUyBkqX/V3qdLLpqgAAABChCDZovr250is/skNNzmjpmjdoEAAAAACjeKgImqdgu/TPS6XyA1LmYOnqVwg1AAAAMI5gg6Yr3WeHmqKdUseTpWvfkryJpqsCAAAACDZoospi6eXLpf0bpKSu0n/NkRJSTVcFAAAASCLYoCmqK6TXrpF2r5HiU+1Qk9zVdFUAAABAAMEGx+b3Sf/7/+zWzjHt7MvP6H4GAACAMEOwwdFZlvTeFGndu5InRrp6ttRlsOmqAAAAgCMQbHB08x+UVr0kudzS5f8j9RhtuiIAAACgUQQbNO7zJ6TP/myPxz8u9f2h2XoAAACAYyDY4EirX5bmTbfHYx+UBv/UbD0AAADAcRBs0NC696R3brHHZ9wqnTnFaDkAAABAUxBscMiWT6W3bpAsv3TaT6TzHzJdEQAAANAkBBvYdq2RXv2x5KuU+lwiXfK45HKZrgoAAABoEoINpH0bpZcvl6qKpe5n2R3QPFGmqwIAAACajGAT6coLpJcvk8r2SRkD7WfVRMeargoAAABoFoJNJLMsu1FAwVapfTfp2n9JsUmmqwIAAACajWATyVb8XVr3juSOlq54XmrXyXRFAAAAwAkh2ESq3V9JH91jj89/SOoyxGw9AAAAQAsQbCJRZbH05nWSr0rqfZF0+q9MVwQAAAC0CMEm0liW9N7t0oFNUlJXacKTtHUGAACA4xFsIs3ql6W1b0ouj/Sj/5HiU0xXBAAAALQYwSaS5K+T3r/DHp97n5R9utl6AAAAgCAh2ESKqjL7vpqacumkc6VRU0xXBAAAAAQNwSZSfHCntHe91C5NuvRZyc0/egAAALQdfLuNBF+9Ka3+pySXdNlzPK8GAAAAbQ7Bpq3bv0l6b4o9Hn2n1GO00XIAAACAUCDYtGXVFdKbk6SqEqnbmdLou0xXBAAAAIREyIPNI488IpfLpSlTpoR6VzjcvOlS3lopPlW6/DnJ7TFdEQAAABASIQ02K1as0DPPPKMBAwaEcjdozLfvSMuftceXPiMlZZqtBwAAAAihkAWbkpISXXvttXruuefUoUOHo25XWVmpoqKiBhNa6OD30ts32+NRt0m9zjdaDgAAABBqIQs2kydP1sUXX6yxY8cec7uZM2cqOTk5MGVlZYWqpMhQUyW9dYNUWSh1HSadO910RQAAAEDIhSTYvPbaa1q1apVmzpx53G2nTZumwsLCwLR9+/ZQlBQ5/vOQtHOlFJss/egfkifadEUAAABAyEUF+wO3b9+u2267TfPmzVNsbOxxt/d6vfJ6vcEuIzJ995G0+C/2eMLfpPbZZusBAAAAWknQg83KlSuVn5+vwYMHB5b5fD4tWrRIf/3rX1VZWSmPh+5cQVe8R5rzS3s8/BfSKZeYrQcAAABoRUEPNuedd57Wrl3bYNn111+vPn366K677iLUhMqK56TyA1Jaf+mC35quBgAAAGhVQQ82iYmJOvXUUxssS0hIUGpq6hHLESS+GmnVP+3xWVOlKC7tAwAAQGQJ+QM60Qq++1AqyZPiO0p9uAQNAAAAkSfoZ2was2DBgtbYTeRa+YL9etq1UlSM0VIAAAAAEzhj43QF26SN/7bHgyeZrQUAAAAwhGDjdKv+KcmScs6WUk8yXQ0AAABgBMHGyXw10urapgFDrjdbCwAAAGAQwcbJNnwkFe+maQAAAAAiHsHGyeqaBgy6hqYBAAAAiGgEG6cq2C5tmGePh1xntBQAAADANIKNU616STQNAAAAAGwEGydq0DTgOqOlAAAAAOGAYONEGz6ubRqQStMAAAAAQAQbZ1r5vP066Fopymu2FgAAACAMEGycpn7TgMGTzNYCAAAAhAmCjdOs/qckS+p+ltSxp+lqAAAAgLBAsHESX420qrZpwNDrzdYCAAAAhBGCjZNs+Fgq3kXTAAAAAOAwBBsnWfmC/TroGpoGAAAAAPUQbJyiYLu0sa5pwHVGSwEAAADCDcHGKVb/U7L8NA0AAAAAGkGwcYL6TQOGXGe0FAAAACAcEWycYOO8Q00DThlvuhoAAAAg7BBsnICmAQAAAMAxEWzCXeEOu82zRNMAAAAA4CgINuFuFU0DAAAAgOMh2IQzX43dDU2iaQAAAABwDASbcLbx31LRTikuhaYBAAAAwDEQbMLZyuftV5oGAAAAAMdEsAlX9ZsGDLnebC0AAABAmCPYhKvVL9M0AAAAAGgigk048tVIq16yxzQNAAAAAI6LYBOOaBoAAAAANAvBJhytfMF+pWkAAAAA0CQEm3BTuFPa8JE95jI0AAAAoEkINuFm9T/tpgHdzpQ69jJdDQAAAOAIBJtwUlEkLXvaHg+lxTMAAADQVASbcLL0b1L5QSm1l9R3oulqAAAAAMcg2ISLsgPS4r/a43PukTxRZusBAAAAHIRgEy4+nyVVFUtp/TlbAwAAADQTwSYcFOdJy561x+feJ7n5xwIAAAA0B9+gw8Gnj0k15VLXYdLJF5quBgAAAHAcgo1pBdukL563x+dOl1wus/UAAAAADkSwMW3ho5K/Wso5W+ox2nQ1AAAAgCMRbEzat1Fa86o9Pvd+s7UAAAAADkawMWnBw5Llk07+gZQ1zHQ1AAAAgGMRbEzJ+1r6+l/2+Jx7zdYCAAAAOBzBxpRPZtiv/S6VMgaYrQUAAABwOIKNCTu+kHLfl1xuacw9pqsBAAAAHI9gY8J/fmu/Dvyx1Olks7UAAAAAbQDBprVt+VTavEByR0uj7zJdDQAAANAmEGxak2UdOlszZJLUoZvZegAAAIA2gmDTmjbMk7Yvk6JipbN+bboaAAAAoM0g2LQWv//Q2ZrhN0pJGWbrAQAAANoQgk1rWfeOlPeVFNNOGnW76WoAAACANoVg0xr8PumTh+3x6TdJCalm6wEAAADaGIJNa/jqDWlfrhTbXjrjZtPVAAAAAG0OwSbUaqqkBTPt8ajbpNhks/UAAAAAbRDBJtRW/1Mq2ColdJZG/MJ0NQAAAECbRLAJpepyadEf7PFZ/y3FJJitBwAAAGijgh5sZs6cqWHDhikxMVGdO3fWxIkTlZubG+zdOMOK/5GKd0tJXaWh15uuBgAAAGizgh5sFi5cqMmTJ2vp0qWaN2+eqqurdcEFF6i0tDTYuwpvlcXSZ3+yx6PvlKK8ZusBAAAA2rCoYH/ghx9+2GD+hRdeUOfOnbVy5UqdffbZwd5d+Fr6tFS2X0rpIQ26xnQ1AAAAQJsW9GBzuMLCQklSSkpKo+srKytVWVkZmC8qKgp1SaFXXSEt/os9HnOP5Ik2Ww8AAADQxoW0eYDf79eUKVM0atQonXrqqY1uM3PmTCUnJwemrKysUJbUOg5skioL7dbOp15uuhoAAACgzQtpsJk8ebK+/vprvfbaa0fdZtq0aSosLAxM27dvD2VJrePg9/ZrSg/JTeM5AAAAINRCdinazTffrPfee0+LFi1S165dj7qd1+uV19vGbqyvCzYdcoyWAQAAAESKoAcby7J0yy23aM6cOVqwYIFyciLwy/2BLfZrh+5GywAAAAAiRdCDzeTJkzV79my9/fbbSkxMVF5eniQpOTlZcXFxwd5deAqcselusgoAAAAgYgT9BpCnnnpKhYWFGjNmjDIyMgLT66+/HuxdhS+CDQAAANCqQnIpWkTz+6SCrfY4JQIvwwMAAAAMoGVXsBXvlnxVkjtKSupiuhoAAAAgIhBsgq3uMrT22ZLbY7QUAAAAIFIQbIIt0BGNy9AAAACA1kKwCTYaBwAAAACtjmATbAQbAAAAoNURbIKNYAMAAAC0OoJNsB2svceGVs8AAABAqyHYBFNFkVS23x6372a2FgAAACCCEGyCqe7BnPGpUmyS2VoAAACACEKwCSZaPQMAAABGEGyCicYBAAAAgBEEm2Ai2AAAAABGEGyCqa4jGsEGAAAAaFUEm2CqO2NDq2cAAACgVRFsgsXvkwq22WPO2AAAAACtimATLEU7JX+N5ImREjNMVwMAAABEFIJNsNS1em7fTXJ7zNYCAAAARBiCTbDQEQ0AAAAwhmATLAQbAAAAwBiCTbDQ6hkAAAAwhmATLLR6BgAAAIwh2AQLl6IBAAAAxhBsgqG8QCo/aI/bdzNaCgAAABCJCDbBUHe2JqGz5G1ntBQAAAAgEhFsgoHL0AAAAACjCDbBQLABAAAAjCLYBAOtngEAAACjCDbBQKtnAAAAwCiCTTBwKRoAAABgFMGmpXzVUsF2e0ywAQAAAIwg2LRU4Q7J8klRsVK7dNPVAAAAABGJYNNSdZehte8mufl1AgAAACbwTbyluL8GAAAAMI5g01K0egYAAACMI9i0FK2eAQAAAOMINi3FpWgAAACAcQSblrAs6cD39phgAwAAABhDsGmJ8oNSZaE9bt/NbC0AAABABCPYtETdZWjt0qWYeKOlAAAAAJGMYNMSdEQDAAAAwgLBpiVoHAAAAACEBYJNS9DqGQAAAAgLBJuW4IwNAAAAEBYINi1Bq2cAAAAgLBBsTlRNlVS0wx534FI0AAAAwCSCzYkq3C5ZfikqTmrX2XQ1AAAAQEQj2Jyo+q2eXS6jpQAAAACRjmBzouiIBgAAAIQNgs2JoiMaAAAAEDYINieKYAMAAACEDYLNiaLVMwAAABA2CDYnwrLqnbHhHhsAAADANILNiSg7IFUV2+P22WZrAQAAAECwOSF1rZ4TM6XoWLO1AAAAACDYnBBaPQMAAABhhWBzIuo/nBMAAACAcSELNk8++aS6d++u2NhYjRgxQsuXLw/VrlofHdEAAACAsBKSYPP6669r6tSpeuCBB7Rq1SoNHDhQF154ofLz80Oxu9bHM2wAAACAsBKSYPOnP/1JN954o66//nr17dtXTz/9tOLj4/WPf/wjFLtrfbR6BgAAAMJK0INNVVWVVq5cqbFjxx7aidutsWPHasmSJUdsX1lZqaKiogZTWKuplIp22mPO2AAAAABhIejBZt++ffL5fEpLS2uwPC0tTXl5eUdsP3PmTCUnJwemrKysYJcUXAXbJFlSdIKU0NF0NQAAAAAUBl3Rpk2bpsLCwsC0fft20yUdW/1Wzy6X0VIAAAAA2KKC/YEdO3aUx+PRnj17Gizfs2eP0tPTj9je6/XK6/UGu4zQoXEAAAAAEHaCfsYmJiZGQ4YM0fz58wPL/H6/5s+fr5EjRwZ7d63vAM+wAQAAAMJN0M/YSNLUqVM1adIkDR06VMOHD9esWbNUWlqq66+/PhS7a12csQEAAADCTkiCzVVXXaW9e/fq/vvvV15engYNGqQPP/zwiIYCjkSrZwAAACDsuCzLskwXUV9RUZGSk5NVWFiopKQk0+U0ZFnSw12k6lLp5pVSx56mKwIAAADarOZkA+Nd0RyldK8dauSS2od5W2oAAAAgghBsmqPuMrTkrlKUgzq5AQAAAG0cwaY5aBwAAAAAhCWCTXMEWj13M1sHAAAAgAYINs3BGRsAAAAgLBFsmoNWzwAAAEBYItg0x8G6S9EINgAAAEA4Idg0VXW5VLzbHnMpGgAAABBWCDZNVbDNfvUmSfEpZmsBAAAA0ADBpqkC99d0k1wuo6UAAAAAaIhg01SBVs/djZYBAAAA4EgEm6ai1TMAAAAQtgg2TUWrZwAAACBsEWya6iCXogEAAADhimDTFJbFpWgAAABAGCPYNEXJHqmmQnK5pfbZpqsBAAAAcBiCTVPUdURL7ip5os3WAgAAAOAIBJum4DI0AAAAIKwRbJqCYAMAAACENYJNU9DqGQAAAAhrBJumoNUzAAAAENYINk3BpWgAAABAWCPYHE9Vmd3uWZJSuBQNAAAACEcEm+OpO1sTmyzFdTBaCgAAAIDGEWyOh8vQAAAAgLBHsDkegg0AAAAQ9gg2xxPoiMb9NQAAAEC4ItgcD2dsAAAAgLBHsDkegg0AAAAQ9gg2x+L3Swe32mNaPQMAAABhi2BzLMW7JV+l5PJISV1NVwMAAADgKAg2x1J3GVr7LMkTZbQUAAAAAEdHsDmWwP01XIYGAAAAhDOCzbEEWj13N1oGAAAAgGPj+qpjOe2/pIyBUmKm6UoAAAAAHAPB5lg6dLMnAAAAAGGNS9EAAAAAOB7BBgAAAIDjEWwAAAAAOB7BBgAAAIDjEWwAAAAAOB7BBgAAAIDjEWwAAAAAOB7BBgAAAIDjEWwAAAAAOB7BBgAAAIDjEWwAAAAAOB7BBgAAAIDjEWwAAAAAOB7BBgAAAIDjEWwAAAAAOB7BBgAAAIDjEWwAAAAAOF6U6QIOZ1mWJKmoqMhwJQAAAABMqssEdRnhWMIu2BQXF0uSsrKyDFcCAAAAIBwUFxcrOTn5mNu4rKbEn1bk9/u1a9cuJSYmyuVymS5HRUVFysrK0vbt25WUlGS6HDgIxw5aguMHLcHxg5bg+MGJCsWxY1mWiouLlZmZKbf72HfRhN0ZG7fbra5du5ou4whJSUn8y40TwrGDluD4QUtw/KAlOH5wooJ97BzvTE0dmgcAAAAAcDyCDQAAAADHI9gch9fr1QMPPCCv12u6FDgMxw5aguMHLcHxg5bg+MGJMn3shF3zAAAAAABoLs7YAAAAAHA8gg0AAAAAxyPYAAAAAHA8gg0AAAAAxyPYAAAAAHA8gs0xPPnkk+revbtiY2M1YsQILV++3HRJCEOLFi3S+PHjlZmZKZfLpblz5zZYb1mW7r//fmVkZCguLk5jx47Vhg0bzBSLsDJz5kwNGzZMiYmJ6ty5syZOnKjc3NwG21RUVGjy5MlKTU1Vu3btdPnll2vPnj2GKkY4eeqppzRgwIDAE75HjhypDz74ILCeYwfN8cgjj8jlcmnKlCmBZRxDOJrf/OY3crlcDaY+ffoE1ps6dgg2R/H6669r6tSpeuCBB7Rq1SoNHDhQF154ofLz802XhjBTWlqqgQMH6sknn2x0/e9//3s98cQTevrpp7Vs2TIlJCTowgsvVEVFRStXinCzcOFCTZ48WUuXLtW8efNUXV2tCy64QKWlpYFtbr/9dr377rt68803tXDhQu3atUuXXXaZwaoRLrp27apHHnlEK1eu1BdffKFzzz1XEyZM0DfffCOJYwdNt2LFCj3zzDMaMGBAg+UcQziWfv36affu3YHps88+C6wzduxYaNTw4cOtyZMnB+Z9Pp+VmZlpzZw502BVCHeSrDlz5gTm/X6/lZ6ebv3hD38ILCsoKLC8Xq/16quvGqgQ4Sw/P9+SZC1cuNCyLPtYiY6Ott58883ANuvWrbMkWUuWLDFVJsJYhw4drL///e8cO2iy4uJiq1evXta8efOs0aNHW7fddptlWfz9wbE98MAD1sCBAxtdZ/LY4YxNI6qqqrRy5UqNHTs2sMztdmvs2LFasmSJwcrgNFu2bFFeXl6DYyk5OVkjRozgWMIRCgsLJUkpKSmSpJUrV6q6urrB8dOnTx9lZ2dz/KABn8+n1157TaWlpRo5ciTHDpps8uTJuvjiixscKxJ/f3B8GzZsUGZmpnr06KFrr71W27Ztk2T22IkK6ac71L59++Tz+ZSWltZgeVpamtavX2+oKjhRXl6eJDV6LNWtAyTJ7/drypQpGjVqlE499VRJ9vETExOj9u3bN9iW4wd11q5dq5EjR6qiokLt2rXTnDlz1LdvX61Zs4ZjB8f12muvadWqVVqxYsUR6/j7g2MZMWKEXnjhBfXu3Vu7d+/Wgw8+qLPOOktff/210WOHYAMAYWDy5Mn6+uuvG1yjDBxP7969tWbNGhUWFuqtt97SpEmTtHDhQtNlwQG2b9+u2267TfPmzVNsbKzpcuAw48aNC4wHDBigESNGqFu3bnrjjTcUFxdnrC4uRWtEx44d5fF4jujesGfPHqWnpxuqCk5Ud7xwLOFYbr75Zr333nv65JNP1LVr18Dy9PR0VVVVqaCgoMH2HD+oExMTo549e2rIkCGaOXOmBg4cqMcff5xjB8e1cuVK5efna/DgwYqKilJUVJQWLlyoJ554QlFRUUpLS+MYQpO1b99eJ598sjZu3Gj07w/BphExMTEaMmSI5s+fH1jm9/s1f/58jRw50mBlcJqcnBylp6c3OJaKioq0bNkyjiXIsizdfPPNmjNnjv7zn/8oJyenwfohQ4YoOjq6wfGTm5urbdu2cfygUX6/X5WVlRw7OK7zzjtPa9eu1Zo1awLT0KFDde211wbGHENoqpKSEm3atEkZGRlG//5wKdpRTJ06VZMmTdLQoUM1fPhwzZo1S6Wlpbr++utNl4YwU1JSoo0bNwbmt2zZojVr1iglJUXZ2dmaMmWKfve736lXr17KycnR9OnTlZmZqYkTJ5orGmFh8uTJmj17tt5++20lJiYGrj1OTk5WXFyckpOT9bOf/UxTp05VSkqKkpKSdMstt2jkyJE6/fTTDVcP06ZNm6Zx48YpOztbxcXFmj17thYsWKCPPvqIYwfHlZiYGLifr05CQoJSU1MDyzmGcDS//vWvNX78eHXr1k27du3SAw88II/Hox//+Mdm//6EtOeaw/3lL3+xsrOzrZiYGGv48OHW0qVLTZeEMPTJJ59Yko6YJk2aZFmW3fJ5+vTpVlpamuX1eq3zzjvPys3NNVs0wkJjx40k6/nnnw9sU15ebt10001Whw4drPj4eOvSSy+1du/eba5ohI0bbrjB6tatmxUTE2N16tTJOu+886yPP/44sJ5jB81Vv92zZXEM4eiuuuoqKyMjw4qJibG6dOliXXXVVdbGjRsD600dOy7LsqzQRicAAAAACC3usQEAAADgeAQbAAAAAI5HsAEAAADgeAQbAAAAAI5HsAEAAADgeAQbAAAAAI5HsAEAAADgeAQbAAAAAI5HsAEAAADgeAQbAAAAAI5HsAEAAADgeP8fFRH/Fp2t1ToAAAAASUVORK5CYII=", "text/plain": [ "
" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "V = np.zeros(nb_states)\n", "Q = np.zeros((nb_states, nb_actions))\n", "\n", "V_high_history = []\n", "V_low_history = []\n", "\n", "for k in range(50):\n", " \n", " Q[s['high'], a['search']] = alpha * (r_search + gamma * V[s['high']]) + (1 - alpha) * (r_search + gamma*V[s['low']])\n", " Q[s['high'], a['wait']] = r_wait + gamma * V[s['high']]\n", " \n", " Q[s['low'], a['search']] = beta * (r_search + gamma * V[s['low']]) + (1 - beta) * (-3 + gamma*V[s['high']])\n", " Q[s['low'], a['wait']] = r_wait + gamma * V[s['low']]\n", " Q[s['low'], a['recharge']] = gamma * V[s['high']]\n", " \n", " V[s['high']] = Q[s['high'], pi[s['high']]]\n", " V[s['low']] = Q[s['low'], pi[s['low']]]\n", " \n", " V_high_history.append(V[s['high']])\n", " V_low_history.append(V[s['low']])\n", " \n", "print(Q)\n", " \n", "plt.figure(figsize=(10, 6))\n", "plt.plot(V_high_history, label=\"high\")\n", "plt.plot(V_low_history, label=\"low\")\n", "plt.legend()\n", "plt.show()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "**Q:** Do the V-values converge? How fast? What do the final values represent? Change the value of $\\gamma$ and conclude on its importance (do not forget to reset the V and Q arrays to 0!). " ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "**A:** The V-values converge quite fast (~15 iterations) to their true value. The high state has a higher value than the low state, as there is no risk in that state to receive the punishment of -3.\n", "\n", "The final value is the expected return in that state, that is:\n", "\n", "$$R_t = \\sum_{k=0}^\\infty \\gamma^k \\, r_{t+k+1}$$\n", "\n", "$\\gamma$ completely changes the scale of the return. Small values of $\\gamma$ lead to small returns (only a couple of rewards count in the sum), while high values lead to high returns (there are a lot of rewards to be summed, especially because the task is continuing)." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "**Q:** Print the Q-values at the end of the policy evaluation. What would the greedy policy with respect to these Q-values? \n", "\n", "**Q:** Change the initial policy to this policy and evaluate it. What happens? Compare the final value of the states under both policies. Which one is the best?" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "**A:** The greedy policy w.r.t. the Q-values is searching in high, recharging in low, as the Q-values are maximal for these actions.\n", "\n", "If we evaluate this policy, we observe that:\n", "\n", "* the value of both states is higher: this is a better policy, as we collect more return on average.\n", "* the greedy policy does not change after the evaluation: we have found the optimal policy already!" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Policy iteration\n", "\n", "Improving the policy is now straightforward. We just to look at the Q-values in each state, and change the policy so that it takes the action with the maximal Q-value. If this does not change the policy (we still take the same actions), we have found the optimal policy, we can stop.\n", "\n", "**Q:** Implement policy iteration.\n", "\n", "Do not forget to reset the V and Q arrays at the beginning of the cell, as well as the original policy.\n", "\n", "Use an infinite loop that you will quit when the policy has not changed between two iterations. Something like:\n", "\n", "```python\n", "while True:\n", " # 1 - Policy evaluation\n", " for k in range(50):\n", " # Update the values\n", " \n", " # 2 - Policy improvement\n", " \n", " if pi != pi_old:\n", " break\n", "```\n", "\n", "**Beware:** if you simply assign the policy to another array and modify the policy:\n", "\n", "```python\n", "pi_old = pi\n", "pi[s['high']] = a['search']\n", "```\n", "\n", "`pi_old` will also change! You need to `.copy()` the policy." ] }, { "cell_type": "code", "execution_count": 7, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Greedy policy after iteration 1\n", "pi(high)= 0\n", "pi(low)= 2\n", "-\n", "Greedy policy after iteration 2\n", "pi(high)= 0\n", "pi(low)= 2\n", "-\n" ] }, { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAzYAAAH6CAYAAADV+tmnAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjkuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8hTgPZAAAACXBIWXMAAA9hAAAPYQGoP6dpAABIXklEQVR4nO3deXxU9b3/8fdsmSxkIYFskEhAZN8UpYBVqFiL1GptbVW0qL22VqxS+qtKrVtbRbvaVq9ae29tK25tFZW6XIoCRdlXEQyoCBEIYUsm6ySZOb8/TjIkECATZuacmbyej57HWWfmk3ia5M35ns9xGIZhCAAAAADimNPqAgAAAADgVBFsAAAAAMQ9gg0AAACAuEewAQAAABD3CDYAAAAA4h7BBgAAAEDcI9gAAAAAiHsEGwAAAABxz211AUcLBoPas2eP0tPT5XA4rC4HAAAAgEUMw1B1dbUKCwvldJ74moztgs2ePXtUVFRkdRkAAAAAbKKsrEx9+/Y94TFhB5ulS5fql7/8pdauXau9e/fq5Zdf1mWXXdbhsTfddJOefPJJ/fa3v9WsWbM69f7p6emSzOIzMjLCLQ8AAABAgvD5fCoqKgplhBMJO9jU1tZq1KhRuuGGG3T55Zcf97iXX35ZK1asUGFhYVjv3zr8LCMjg2ADAAAAoFO3qIQdbKZOnaqpU6ee8Jjdu3fr+9//vt566y1NmzYt3I8AAAAAgLBE/B6bYDCoa6+9Vj/60Y80bNiwkx7v9/vl9/tD6z6fL9IlAQAAAEhwEW/3/PDDD8vtduvWW2/t1PFz585VZmZmaKJxAAAAAIBwRfSKzdq1a/W73/1O69at63Sr5jlz5mj27Nmh9dYbhE4mEAioqampy7V2Rx6PRy6Xy+oyAAAAgIiLaLD5z3/+o4qKChUXF4e2BQIB/fCHP9QjjzyiTz/99JjXeL1eeb3eTn+GYRgqLy9XZWVlBCrufrKyspSfn88zggAAAJBQIhpsrr32Wk2ZMqXdtosuukjXXnutrr/++oh8Rmuoyc3NVWpqKn+gd5JhGKqrq1NFRYUkqaCgwOKKAAAAgMgJO9jU1NToo48+Cq3v2LFDGzZsUHZ2toqLi5WTk9PueI/Ho/z8fA0aNOiUiw0EAqFQc/Tn4ORSUlIkSRUVFcrNzWVYGgAAABJG2MFmzZo1mjx5cmi99f6YGTNm6Omnn45YYR1pvacmNTU1qp+TyFq/d01NTQQbAAAAJIywg82kSZNkGEanj+/ovppTxfCzruN7BwAAgEQU8XbPAAAAABBrBBsAAAAAcY9gEyOTJk3SrFmzjrvf4XBo/vz5nX6/xYsXy+Fw0PYaAAAAUITbPaPr9u7dq549e1pdBgAAABCXCDY2kZ+fb3UJAAAAQNyK+6FohmGorrHZkimc7nCSFAwGdfvttys7O1v5+fm67777QvuOHor23nvvafTo0UpOTtbYsWM1f/58ORwObdiwod17rl27VmPHjlVqaqomTJig0tLSU/huAgAAAPEp7q/Y1DcFNPSetyz57C0/vUipSZ3/Fv7lL3/R7NmztXLlSi1fvlzXXXedJk6cqAsvvLDdcT6fT5dccokuvvhiPfvss9q5c+dx78+566679Otf/1q9e/fWTTfdpBtuuEHvvvvuqXxZAAAAQNyJ+2ATT0aOHKl7771XkjRw4EA9+uijWrRo0THB5tlnn5XD4dBTTz2l5ORkDR06VLt379aNN954zHs+8MADOv/88yVJd955p6ZNm6aGhgYlJydH/wsCAAAAbCLug02Kx6UtP73Iss8Ox8iRI9utFxQUqKKi4pjjSktLNXLkyHbh5JxzzjnpexYUFEiSKioqVFxcHFZtAIDuJxA05G8OyN8UlL85aC43B1vWA2psDqoxEFRzwFBToP1yc9BQIGi0zFvWA0Zoe8AwFAwaChqGAkG1zM31oGEOJW9dDhqGjKPnkmRIhgwFg+bcMCRDktGy88i6OTS8dV/rQPGjh4y3rra8u44eUd523VDHrz2y/wTCG6l+3M9MRGGO4ofFfvvN0SrKTrW6jE6L+2DjcDjCGg5mJY/H027d4XAoGAxG7D0dDocknfJ7AgDszzAM+eqbdaDWr4M1jaqsa5SvoVlV9U2qqm+Sr3VqaFKNv1n1jQHVhaZm1TWaIQYAjsffHLC6hLDERyLoZgYNGqRnnnlGfr9fXq9XkrR69WqLqwIAxIphGDpU26idh+q062Cddh6s02eH63Sgxq8DNY06UGOGmcZA5IKJx+WQ1+2S1+2U1+1UUsvkcbVOjnbLLqdDbpdTbmfLstMhl/PIutPhkMspOR0OOZ0OuVrmTofkcjjkcJj/IOd0mNucbbY5JDlbl9tsczgkh1q2tVlv+Z+kI//I17pfbectR7Wutz0+tN52uf2uYzjaHH2yY4/3GeEI5zOASMjLiK9bGwg2NnT11Vfrrrvu0ne+8x3deeed2rVrl371q19JOvYHMAAgfjUHgtpxoFal+6pVWl6t7ftqtPNQncoO1anG39yp90j3upXTI0mZqUnKTPEoM8WjjGS3OU/xKCPZox7JbqV6XEr1upSa5FZqkqtlcivF41KS2ymXk98vAOIbwcaGMjIy9Nprr+l73/ueRo8erREjRuiee+7R1VdfTVMAAIhTzYGgNpRVas3Owyotr9aH5dX6uKLmuFddHA4pPyNZxdmpOi0nVUU9U5Wb4VWvHuaU0yNJvXp4lRzm/Z4AkKgINjGyePHiY7a1fW7N0Tc4TpgwQRs3bgytz5s3Tx6PJ9QUYNKkSce8ZvTo0WE/WwcAED0Havxaum2/3indr6Xb9quqvumYY1KTXDojL12D89N1Rl66+vVKVXF2mvr2TCG0AEAYCDY29de//lX9+/dXnz59tHHjRt1xxx36xje+oZSUFKtLAwAch2EY+mCPTwu37NPibfu16bPKdl2gMlM8mjAgR0MLMjQoP12D8zPUt2eKnAwDA4BTRrCxqfLyct1zzz0qLy9XQUGBrrjiCj3wwANWlwUAOIphGNqy16d/bdqrf72/VzsP1rXbP7QgQ5MH99bkQbkaXZQlt8tpUaUAkNgINjZ1++236/bbb7e6DABABwzD0Ifl1aEws+NAbWhfssep88/orQsG5+n8Qb3jrqsQAMQrgg0AAGHYUFapn8x/X5t3+0LbvG6nJg/K1bSRBfrC4Fylefn1CgCxxk9eAAA6ocbfrF//X6mefu9TGYaU5HZq8qDemjayUF8YnKsehBkAsBQ/hQEAOIm3P9ynn7y8WXuqGiRJl4/po7umDVFOD6/FlQEAWhFsAAA4jorqBt3/2hb9a9NeSVJRdooeuGyEzjujt8WVAQCORrABAOAowaChF9eU6cHXt8rX0CyX06H/OrdEt00ZqNQkfnUCgB3x0xkAgDY2lFXq3lc/0MaySknS8D4ZeujykRreJ9PawgAAJ0SwiZFJkyZp9OjReuSRR6wuBQDQgQM1fv3izQ/14prPJElpSS794MIzdN2Efjx7BgDiAMEGANCtNQeC+uvynfrtv7epuqFZktkc4M6pg5XLM2gAIG4QbAAA3dbyjw/qvlc/UOm+aknSsMIM/fTSYTrrtGyLKwMAhCv+r60bhtRYa81kGF0q+fDhw/rWt76lnj17KjU1VVOnTtX27dtbvhxDvXv31j/+8Y/Q8aNHj1ZBQUFofdmyZfJ6vaqrqzu17x0AdGPvfFihq/+0QqX7qtUz1aMHvjpcr95yLqEGAOJU/F+xaaqTHiy05rN/vEdKSgv7Zdddd522b9+uV199VRkZGbrjjjt08cUXa8uWLfJ4PDrvvPO0ePFiff3rX9fhw4e1detWpaSk6MMPP9TgwYO1ZMkSnX322UpNTY3CFwUAia+qrkl3/HOTDEP68sgC/fyy4cpKTbK6LADAKYj/KzZxpjXQ/OlPf9LnP/95jRo1SvPmzdPu3bs1f/58SWajgcWLF0uSli5dqjFjxrTbtnjxYp1//vnWfAEAkADuf+0DVVT71b93mn51xShCDQAkgPi/YuNJNa+cWPXZYdq6davcbrfGjRsX2paTk6NBgwZp69atkqTzzz9ft912m/bv368lS5Zo0qRJys/P1+LFi/Xtb39b7733nm6//faIfRkA0J0s3LJPL63fLadD+tUVo5TscVldEgAgAuI/2DgcXRoOZmcjRoxQdna2lixZoiVLluiBBx5Qfn6+Hn74Ya1evVpNTU2aMGGC1WUCQNw5XNuoH7/8viTpxvP668zinhZXBACIFIaixdiQIUPU3NyslStXhrYdPHhQpaWlGjp0qCTJ4XDo85//vF555RV98MEHOvfcczVy5Ej5/X49+eSTGjt2rNLSEivMAUAs3PfaB9pf7dfpuT30gylnWF0OACCCCDYxNnDgQF166aW68cYbtWzZMm3cuFHXXHON+vTpo0svvTR03KRJk/Tcc89p9OjR6tGjh5xOp8477zzNmzeP+2sAoAve3LxXr2zYI5fToV8zBA0AEg7BxgJ//vOfddZZZ+nLX/6yxo8fL8Mw9Prrr8vj8YSOOf/88xUIBDRp0qTQtkmTJh2zDQBwcodqG/WT+ZslSd89r79GFWVZWxAAIOIchtHFh7FEic/nU2ZmpqqqqpSRkdFuX0NDg3bs2KGSkhIlJ/M06K7gewigO7rl2XVasGmvzsjrode+f668bq7WAEA8OFE2OBpXbAAACe1fm/Zqwaa9LUPQRhNqACBBEWwAAAlrb1W97n7FHII2c9IAjeibaXFFAIBoIdgAABLS2p2H9ZVH39Wh2kYNzk/XLV8YaHVJAIAoiv/n2AAAcJQXVu/S3fM/UGMgqEF56XrqW2OV5Obf8gAgkcVlsAkGg1aXELf43gFIZE2BoH62YIv+unynJGnq8Hz96opRSvPG5a87AEAY4uonfVJSkpxOp/bs2aPevXsrKSlJDofD6rLigmEYamxs1P79++V0OpWUlGR1SQAQUQdr/Lp53jqt3HFIkvTDC8/QLV84nd8TANBNxFWwcTqdKikp0d69e7Vnzx6ry4lLqampKi4ultPJkAwAiWPz7ip9929rtbuyXj28bj3yzdGaMjTP6rIAADEUV8FGMq/aFBcXq7m5WYFAwOpy4orL5ZLb7eZfLwEklLU7D2v6n1aooSmokl5peupbZ+n03HSrywIAxFjcBRtJcjgc8ng88ng8VpcCALDY35Z/qoamoCYMyNHj15ylzBR+NwBAd8R4JABA3DIMQ+99fFCSdMsXTifUAEA3RrABAMStj/fXqKLaL6/bqTOLe1pdDgDAQmEHm6VLl+qSSy5RYWGhHA6H5s+fH9rX1NSkO+64QyNGjFBaWpoKCwv1rW99ixv9AQBR0Xq1Zmy/nkr2uCyuBgBgpbCDTW1trUaNGqXHHnvsmH11dXVat26d7r77bq1bt04vvfSSSktL9ZWvfCUixQIA0Na7Hx2QJE0Y0MviSgAAVgu7ecDUqVM1derUDvdlZmZq4cKF7bY9+uijOuecc7Rr1y4VFxd3rUoAAI4SCBpa8Yn5zJoJA3IsrgYAYLWod0WrqqqSw+FQVlZWh/v9fr/8fn9o3efzRbskAEAC2LLHp6r6JqV73RrRJ9PqcgAAFotq84CGhgbdcccduuqqq5SRkdHhMXPnzlVmZmZoKioqimZJAIAE8d7H5jC0cf2z5XbRCwcAuruo/SZoamrSN77xDRmGoccff/y4x82ZM0dVVVWhqaysLFolAQASyLstjQO4vwYAIEVpKFprqNm5c6fefvvt416tkSSv1yuv1xuNMgAACaqxOajVO1rurzmd+2sAAFEINq2hZvv27XrnnXeUk8MvHABAZG0oq1R9U0A5aUkalJdudTkAABsIO9jU1NToo48+Cq3v2LFDGzZsUHZ2tgoKCvT1r39d69at04IFCxQIBFReXi5Jys7OVlJSUuQqBwB0W63314wfkCOHw2FxNQAAOwg72KxZs0aTJ08Orc+ePVuSNGPGDN1333169dVXJUmjR49u97p33nlHkyZN6nqlAAC0eO8j8/6aiadzfw0AwBR2sJk0aZIMwzju/hPtAwDgVNU1Nmt92WFJPL8GAHAE/TEBAHFl9aeH1RQw1CcrRcXZqVaXAwCwCYINACCutN5fM4H7awAAbRBsAABxhftrAAAdIdgAAOJGVV2TNu+pkmR2RAMAoBXBBgAQN5Z/clCGIZ2e20N5GclWlwMAsBGCDQAgbixvc38NAABtEWwAAHHj3Y/N+2smDOD+GgBAewQbAEBcqPA16KOKGjkc0uf6Z1tdDgDAZgg2AIC48F7L1ZrhhZnKSk2yuBoAgN0QbAAAceE97q8BAJwAwQYAYHuGYejdlufXTOD5NQCADhBsAAC2V3aoXrsr6+VxOXR2v55WlwMAsCG31QUAVmkOBNUUMNQYCKo5EFQgaKgpaCgQMNQcDKo5aKg5YCgQNBQwzHmwdd6yLWhIQcNcb102DEOBYMuyzH9pNkL7jmyXIRkyt5nHtV9X6PVmvUbr69pua/P1GEbbtaOPMzrcfvR7HO/1He474Su77kSfie6rtLxakjSmqKdSk/jVBQA4Fr8dYGvNgaCq6ptUWd+kyrpG+eqbVe1vVm3LVBOaB1TX2KyGpoAamoLmvDkof1NADU0B+ZuDamwOqjFgzpsCQQX5AxqIOxNO5/4aAEDHCDaIuWDQ0KG6Ru3zNbRMfpVXNaii2lw+UONXZV2TDtc1qrqhOWZ1OR2S2+mU2+WQy+mQx+WUy+mQ2+mQ02FuczkdcjrUMneE5k6H5Gyz7GidyyGn05w7WrY7JDkckrPNslr3t6wfOd6szaGWnQrNQu8ltT3uyL5WjmMWjuU4amebl5/oZSfk6OoLLXD01w97Sk9267oJ/awuAwBgUwQbRIVhGDpU26gdB2o7nPzNwbDeLyPZrazUJGWmeJTmdamH1620lqmH1620JLfSvC55PS4lu51K9rhappZlt0tJbqc8LoeS3E4luZzyuJxKcptBxuN0yunkj1sAAIB4RbBBRByo8WvTZ5XaUFalTZ9VatNnVTpU23jc4x0OKSfNq/xMr/LSk5WXmWzOM7zqne5VVmqSslI96pmapIxkt9wu+lwAAADg+Ag26JKP99fonQ8rtL6sUhvLKvXZ4fpjjnE4pMLMFJX0Sjsy9U5T/15pKsxKkYewAgAAgAgh2KBTmgNBrdl5WP/esk+LPqzQjgO1xxxzem4PjeqbpVFFmRrVN0uD8tOV7HFZUC0AAAC6G4INjquhKaB/b92nhVv2aXHpflXVN4X2eVwOjSvJ0fgBORpTlKXhfTOVkeyxsFoAAAB0ZwQbHKO8qkHPrNip51bt0sE298n0TPVo8qBcXTAkT+ed0UvpBBkAAADYBMEGkswuZmt3HtbT732qNzeXq7nlIS+Fmcm6ZHShpgzJ05nFPeWicxgAAABsiGDTzTUFgnplwx49/d4Obd7tC20/pyRb10/opwuH5tGRDAAAALZHsOnGVn96SD95ebNK91VLkrxupy4dXagZE/ppWGGmxdUBAAAAnUew6YYO1zbqoTc+1AtryiSZ987ceF5/XXl2sbLTkiyuDgAAAAgfwaYbMQxD/1j7mR58fasO15kdzq48u0h3fGmwehJoAAAAEMcINt3E9n3Vumv+Zq3acUiSNCgvXQ98dbjG9su2uDIAAADg1BFsEpxhGPqfZTv08JsfqilgKMXj0qwpA3XDuSXy0BQAAAAACYJgk8CaA0Hd99oHembFLknSlCG5uu8rw9S3Z6rFlQEAAACRRbBJULX+Zt3y7Dq9U7pfDof0k2lD9e1zS6wuCwAAAIgKgk0C2udr0A1Pr9YHe3zyup363ZWj9aXhBVaXBQAAAEQNwSbBlJZX6/o/r9KeqgblpCXpTzPGakxxT6vLAgAAAKKKYJNAlm0/oO89s1bV/mb1752mp687R8U53E8DAACAxEewSRCvbNitH764Uc1BQ+eUZOuP156lrFSeTQMAAIDugWCTADaUVepHf9+k5qChr4wq1C+vGCmv22V1WQAAAEDMEGzi3MEav25+Zq0aA0F9cWieHvnmaDmdDqvLAgAAAGKKJzTGsUDQ0K3Pr9eeqgb175WmX31jFKEGAAAA3RLBJo79+v9K9e5HB5XicemJa89SRrLH6pIAAAAASxBs4tRbH5Trvxd/LEl6+OsjdUZeusUVAQAAANYh2MShT/bX6P+9uFGSdMPEEn1lVKHFFQEAAADWItjEmVp/s25qeVbN2f16as7Fg60uCQAAALAcwSaOGIahO/65Sdv21ah3ulePXX2mPC7+EwIAAAD8VRxH/vzup1qwaa/cTof+e/qZys1ItrokAAAAwBYINnHis8N1evjNDyVJd00borP7ZVtcEQAAAGAfBJs48eDrW+VvDupz/bN13YR+VpcDAAAA2ArBJg689/EBvf5+uZwO6d5Lhsnh4CGcAAAAQFsEG5trDgT109e2SJKmjztNQwoyLK4IAAAAsJ+wg83SpUt1ySWXqLCwUA6HQ/Pnz2+33zAM3XPPPSooKFBKSoqmTJmi7du3R6rebue5Vbv0YXm1MlM8mn3hGVaXAwAAANhS2MGmtrZWo0aN0mOPPdbh/l/84hf6/e9/ryeeeEIrV65UWlqaLrroIjU0NJxysd1NZV2jfr1wmyTph188Qz3TkiyuCAAAALAnd7gvmDp1qqZOndrhPsMw9Mgjj+gnP/mJLr30UknSX//6V+Xl5Wn+/Pm68sorT63abua3C7epsq5Jg/PTdfU5xVaXAwAAANhWRO+x2bFjh8rLyzVlypTQtszMTI0bN07Lly/v8DV+v18+n6/dBKm0vFrPrNwlSbrny0Pl5kGcAAAAwHFF9K/l8vJySVJeXl677Xl5eaF9R5s7d64yMzNDU1FRUSRLikuGYej+1z5QIGjoS8PyNeH0XlaXBAAAANia5ZcB5syZo6qqqtBUVlZmdUmWe+uDcr338UEluZ26a9oQq8sBAAAAbC+iwSY/P1+StG/fvnbb9+3bF9p3NK/Xq4yMjHZTd9bQFNDP/7VVkvTd8/qrKDvV4ooAAAAA+4tosCkpKVF+fr4WLVoU2ubz+bRy5UqNHz8+kh+VsJ5a+ok+O1yvgsxkfW/SAKvLAQAAAOJC2F3Rampq9NFHH4XWd+zYoQ0bNig7O1vFxcWaNWuWfv7zn2vgwIEqKSnR3XffrcLCQl122WWRrDsh1fib9fiSjyVJd04drNSksP/zAAAAAN1S2H85r1mzRpMnTw6tz549W5I0Y8YMPf3007r99ttVW1ur73znO6qsrNS5556rN998U8nJyZGrOkG9tblcdY0BlfRK01dGFVpdDgAAABA3HIZhGFYX0ZbP51NmZqaqqqq63f021/xppZZ9dECzLzxDt14w0OpyAAAAAEuFkw0s74oGU3lVg979+IAk6atj+lhcDQAAABBfCDY28erG3TIM6ex+PemEBgAAAISJYGMTL63bLUm6jKs1AAAAQNgINjawda9PH5ZXK8nl1JdH0DQAAAAACBfBxgZeXm9erfnC4FxlpnosrgYAAACIPwQbiwWChl7ZwDA0AAAA4FQQbCy2/OOD2ufzKzPFo8mDe1tdDgAAABCXCDYWe2n9Z5KkL48skNftsrgaAAAAID65rS6gO6trbNabm8sl8ewaAACOYRgtU1BSy/Jx52q/rfX1Mtq/39Hv335D5/Z1uP8Ex3aWvZ6ZfgoS5euA0npLrvi5/5tgY6GFW/aprjGg4uxUnXVaT6vLAQAkoma/5K+RGmukpjqpsdZcbqyVGuuk5nrzmOaGlql1uVEK+KVAU8vUaM6DLcvBgBRsPmoKtF82guYUDEhGoM0248i+oye1BhkAlpu5Suo9yOoqOo1gY6G2z65xOBwWVwMAsL2mBqm2QqreJ9W0nSqkhkqpwSc1VLWfAn6rq05gbX53h/V7vIu/8/lbATghgo1FKqob9J/t+yUxDA0A0KLukFS5U6oskyp3SVUt88oyc7mhsuvv7U6RktKkpFQpqYe57Ek1J3eS5E6W3N4jc5e3Ze6RXEmS03Nk2eWRnC5zm9PdMrnaLztcktMpOZwtyy3bHE5zWWrZ13ZymHM5jqzL0bK9zfJx51IoNLRuDy0fvU9HvU5H7QMQbwg2Fnl1wx4FDWl0UZZKeqVZXQ4AINZqKqQ961umDea8pvzkr3MlST3ypB65Uo/8lnmelNJTSs7sYMowg4yTBjUAEhvBxiLzW55dc/mZXK0BgITX7DeDy67l0mdrzGXf7o6P7ZEnZRZJWUVSVnHL8mlSZl8pPd8MMFxVAIBjEGwssH1ftTbv9sntdOjLIwutLgcAEGn1h6WyVWaQ2bVC2r2ug3tdHFKvM6TCMS3TaCl/hDlEDAAQNoKNBV5ab/4r3aRBvZWdlmRxNQCAiAgGpA//Ja143Aw0R7e8TestFY+XisZJfc40Q4w33ZJSASAREWxiLBg09EpLsPnqmL4WVwMAOGX+amn9PGnFf5s3/rfKHiCdNt4MM8Xjpez+DCEDgCgi2MTYul2HtaeqQelety4Ykmt1OQCArqosk1Y9Ka39q+SvMrel9JTGflsae4OUyT2UABBLBJsYe/19s+PNBUNyleyhQw0AxBXDkHa+J63+k7TlFfOhk5KUM1D63PekUVeZ7ZQBADFHsIkhwzD01gdmsJk6osDiagAAndbgkza9IK3+H2n/1iPbS86Txt8inX6h+cwWAIBlCDYxtOmzKu2urFdqkkvnn9Hb6nIAACdTvlla8z/SphelxhpzmydVGnGFdPZ/SQUjra0PABBCsImh1zfvlSRNHsQwNACwtbpD0j+ulz5ZfGRbr0HS2d+WRl1pPvgSAGArBJsYMQxDb25uHYaWb3E1AIDjaqiSnrncfIim0y0N/rJ5dabfuXQ1AwAbI9jEyNa91dp5sE5et1OTB9ENDQBsqbFWmvcNM9Sk5kjX/UvKHWJ1VQCATuBOxxh5o2UY2nln9FaalzwJALbTVC89d6VUtsIcanbtfEINAMQRgk2MvNEyDO1ihqEBgP00N0ovfkvasVRK6iFd8xKNAQAgzhBsYuCjimp9VFEjj8uhLwzOs7ocAEBbgWbpn9+Wtv+f5E6Rrn5R6jvW6qoAAGEi2MTAGy0P5Zx4ei9lpngsrgYAEBIMSq/cLG19VXIlSVfOk/pNtLoqAEAXEGxi4PXWYWjDeSgnANhGoElaMMt88KbTLV3xF+n0C6yuCgDQRdzFHmU7D9Zq616fXE6HLhzKMDQAsJxhSNsXSv93l3Rgm+RwSpf/URp8sdWVAQBOAcEmylqbBnyuf7Z6piVZXA0AdHP7tpiB5uO3zfXUHOniX0nDL7e2LgDAKSPYRNkb75ttnqcyDA0ArFOzX1r8oLT2ackISk6P9LmbpM//Pykly+rqAAARQLCJot2V9dr4WZUcDumLwxiGBgAxFwxKyx+Vlv5S8vvMbUMukS78qZTd39raAAARRbCJojdbhqGdfVq2ctOTLa4GALqh9X+TFt5tLheMki56UOp3rrU1AQCigmATRW9ubhmGxkM5AcAa6/5izifeJl1wn+SkGSgAJCp+wkdJha9Ba3YeliR9aTjBBgBibv82afdayeGSxt9CqAGABMdP+Sh564NyGYY0uihLBZkpVpcDAN3PxmfN+cALpR651tYCAIg6gk2UtLZ5vphhaAAQe8GAtPEFc3nUVdbWAgCICYJNFByqbdSKTw5Kos0zAFhixxKpeo+UnCUNmmp1NQCAGCDYRMGGssMKGtLA3B4qyk61uhwA6H42PGfOh39NcnutrQUAEBMEmyjYcaBOkjQwr4fFlQBAN9Tgk7a+Zi6PvtraWgAAMUOwiYJPD9RKkvrlpFlcCQB0Q1tekZrrpZyBUp+zrK4GABAjBJso+PRgS7DpRbABgJjb2DIMbfRVksNhbS0AgJgh2ETBjpYrNiUEGwCIrcOfSjvfleSQRl5pdTUAgBgi2ERYQ1NAuyvrJTEUDQBibuPz5rz/+VJmH2trAQDEFMEmwsoO1ckwpHSvW716JFldDgB0H4ZxZBjaKJoGAEB3E/FgEwgEdPfdd6ukpEQpKSkaMGCAfvazn8kwjEh/lC21DkPr1ytNDsZ2A0Ds7FpuDkVL6iEN+bLV1QAAYswd6Td8+OGH9fjjj+svf/mLhg0bpjVr1uj6669XZmambr311kh/nO3QOAAALLLhWXM+9DIpiZ/BANDdRDzYvPfee7r00ks1bdo0SVK/fv303HPPadWqVZH+KFtqfYZNSQ4P5gSAmGmskz6Yby6PvsrSUgAA1oj4ULQJEyZo0aJF2rZtmyRp48aNWrZsmaZOndrh8X6/Xz6fr90Uz3YcqJHEFRsAiKkP/yU1VktZxVLxBKurAQBYIOJXbO688075fD4NHjxYLpdLgUBADzzwgKZPn97h8XPnztX9998f6TIs82nLFRuCDQDE0MaWYWijrpKc9MUBgO4o4sHmxRdf1Lx58/Tss89q2LBh2rBhg2bNmqXCwkLNmDHjmOPnzJmj2bNnh9Z9Pp+KiooiXVZM1DcGVO5rkCT1J9gAQOQYhlSxVQr4j93X4JM+WWwuj+LZNQDQXUU82PzoRz/SnXfeqSuvNH+5jBgxQjt37tTcuXM7DDZer1derzfSZViitXFAVqpHWam0egaAiFn8kLTkoRMfUzxeyu4fm3oAALYT8WBTV1cn51HDAFwul4LBYKQ/ynY+bW31zIM5ASCyylaY85RsydNBcxZPsnT+HbGtCQBgKxEPNpdccokeeOABFRcXa9iwYVq/fr1+85vf6IYbboj0R9nOJy3BpoRhaAAQWYd3mvNv/k3qd661tQAAbCniweYPf/iD7r77bt18882qqKhQYWGhvvvd7+qee+6J9EfZDldsACAKggGp6jNzOes0a2sBANhWxINNenq6HnnkET3yyCORfmvba73HpqQ3wQYAIqZ6rxRskpxuKaPQ6moAADZFT8wIOvJwToINAERM6zC0zL6S02VtLQAA2yLYREh1Q5MO1JhtSPv16uDGVgBA11TuMucMQwMAnADBJkJaH8zZq0eS0pM9FlcDAAmksuWKTVaxtXUAAGyNYBMhOw7SOAAAoqJ1KFpPrtgAAI6PYBMhn9LqGQCiIzQUrZ+lZQAA7I1gEyGhVs8EGwCILIaiAQA6gWATIa1D0bhiAwARFGiSfLvNZYaiAQBOgGATITt4OCcARF7VZ5IRlFxeKS3X6moAADZGsImAyrpGVdY1SaLVMwBEVOj+mmLJya8sAMDx8VsiAlqv1uRleJWa5La4GgBIINxfAwDoJIJNBHzK/TUAEB20egYAdBLBJgJ2tDyck2ADABEWGopGsAEAnBjBJgJoHAAAUcJQNABAJxFsIoBn2ABAlLResWEoGgDgJAg2p8gwjFCwYSgaAERQU4NUvddcZigaAOAkCDan6GBto6r9zXI4pOJsWj0DQMRUlZlzT5qUmmNtLQAA2yPYnKLW+2sKM1OU7HFZXA0AJJDKNh3RHA5rawEA2B7B5hTtYBgaAETHYRoHAAA6j2Bzio40DmAYGgBEFK2eAQBhINicotaHc9LqGQAijFbPAIAwEGxOUevDOfv3JtgAQETR6hkAEAaCzSlo2+qZKzYAEGGhe2wINgCAkyPYnIJ9Pr/qmwJyOR0qotUzAESOv0aqO2AuMxQNANAJBJtT0NoRrW/PFHlcfCsBIGJan2GTnCmlZFlaCgAgPvDX+CmgcQAARAmtngEAYSLYnIJPeYYNAEQHrZ4BAGEi2JyCTwg2ABAdra2ee/aztAwAQPwg2JyCIw/nJNgAQEQd/tScMxQNANBJBJsuCgYN7TxkPsOmhHtsACCyGIoGAAgTwaaL9lTVq7E5KI/LocKsZKvLAYDEUknzAABAeAg2XfTpAfNqTVF2qty0egaAyKmvlBqqzGWCDQCgk/iLvIuWbt8vSRpSkGFxJQCQYFqHoaX2krw9rK0FABA3CDZd0BwI6uX1uyVJXxlVaHE1AJBgQvfXcLUGANB5BJsuePfjg9pf7VfPVI8mD8q1uhwASCyhVs80DgAAdB7Bpgv+ufYzSebVmiQ330IAiKjDNA4AAISPv8rDVN3QpLc+KJckXX5mX4urAYAERKtnAEAXEGzC9Mb75fI3B3V6bg+N7JtpdTkAkHgYigYA6AKCTZj+sc4chnb5mX3kcDgsrgYAEoxhcMUGANAlBJswlB2q06odh+RwSF8d08fqcgAg8dQdkhprzOXMImtrAQDEFYJNGF5aZ7Z4njiglwoyUyyuBgASUOWn5rxHvuRJtrQUAEB8Idh0kmEYemn9kWFoAIAoaB2Gxv01AIAwEWw6ad2uw9p5sE6pSS59aXi+1eUAQGIKtXom2AAAwkOw6aR/rDWHoU0dXqDUJLfF1QBAggo1DuAZNgCA8BBsOqGhKaAFm/ZIkr7GMDQAiB5aPQMAuohg0wmLtlaouqFZhZnJ+lz/HKvLAYDExRUbAEAXEWw64Z8tz6756pl95HTy7BoAiAqeYQMAOAUEm5PYX+3Xkm37JUmXn9nX4moAIIHV7JOaGySHU8rk5y0AIDxRCTa7d+/WNddco5ycHKWkpGjEiBFas2ZNND4q6l7ZsFuBoKHRRVka0LuH1eUAQOJqvVqT0UdyeaytBQAQdyLe3uvw4cOaOHGiJk+erDfeeEO9e/fW9u3b1bNnz0h/VEy0PpSTpgEAEGW0egYAnIKIB5uHH35YRUVF+vOf/xzaVlJSctzj/X6//H5/aN3n80W6pC7butenLXt98rgcumRUodXlAEBia+2IRuMAAEAXRDzYvPrqq7rooot0xRVXaMmSJerTp49uvvlm3XjjjR0eP3fuXN1///2RLiMiXmppGnDB4DxlpSZZXA0sYRhSMCAZgaPmwSPzY6aA+TojeGSuE6zLkAy1zI3289Ya2i23PVYd72tb/9HbOvoa22/o5PemyzsTwzHfN5yyspXmnFbPAIAucBhGZH87JycnS5Jmz56tK664QqtXr9Ztt92mJ554QjNmzDjm+I6u2BQVFamqqkoZGRmRLC1sW/f69Pc1n2nSoN4674zeltbS7RmG1FQnNVRJ/hqpsbplXnNkvbFWamqQmuvbzFumZr8U8EuBJinQaE7NLfNAkxRsapk3H5kCTWZIARBblz0hjb7K6ioAADbg8/mUmZnZqWwQ8WCTlJSksWPH6r333gttu/XWW7V69WotX778pK8Pp3jEuUCz2QWpulyq3tMy3yvVVEj1h49MdYfMecB/8veMNYfzBJPDnMvRwfpRy3JIDh213jKXOlhue6w62NdmW2j16Fbljg4XO/giT7DrVNqf0zodHeiRJ13+pJQSn/dlAgAiK5xsEPGhaAUFBRo6dGi7bUOGDNE///nPSH8U4oG/Wjr4sXToY3N+8CNzqvrMDDDhDllyuqWkHpI33ZwnpUneHi3LPSRPsuRJldzJkiel/dztNTstubySK8lcdnslp0dyuc25092y3Gbd6TInx9Fz5yn+YQ8AAIBIiXiwmThxokpLS9tt27Ztm047jTHTCc+3R9q99si0f5tUU37i1zhcUnp+y1RgTj3ypNSe5r/YpmRLqdlHlpPSCBMAAAA4RsSDzQ9+8ANNmDBBDz74oL7xjW9o1apV+uMf/6g//vGPkf4oWKm5USpbIX22Wtq9zgwy1Xs7Pja1l5RzupQzoGU63ex6lF4opfUyr34AAAAApyDi99hI0oIFCzRnzhxt375dJSUlmj179nG7oh2Ne2xsrPagtP3/pG1vSh+/LfmPas3tcEl5Q6XCM6U+Z0l5w80gk5JlSbkAAACIb5Y2DzhVBBub2b9NKv2XVPqm9NmqllbFLdJypX7nSn3HmkEmf6SUlGpdrQAAAEgoljYPQAIINEsfviatfFLadVQnu7wR0qAvSWdMlQrHSE6nNTUCAAAAbRBscETdIWnt09LqP0m+3eY2p1vqP0k640vmlFVkZYUAAABAhwg2kPZ9IK18Qtr0otTcYG5L7SWNvcGcMgqsrQ8AAAA4CYJNd1ZdLr11l7T5H0e25Y+UPvc9adjl5jNhAAAAgDhAsOmOAs3Smv+R3v652dnM4ZSGfEUad5NU/DmeEwMAAIC4Q7Dpbj5bIy34gVS+yVwvPFP68m/MRgAAAABAnCLYdBd1h6RFPzWbA8iQkjOlC+6VzrqOB2QCAAAg7hFsuoPSN6RXbpHqDpjro66WLvyp1KO3tXUBAAAAEUKwSXSrnpLeuN18sGbvwdK030j9JlpdFQAAABBRBJtEZRjSovulZb8118+cIV38K8mdZG1dAAAAQBQQbBJRc6P06velTc+b65Pvks77Ed3OAAAAkLAINommwSe9eK30yWLJ4ZK+8ntpzDVWVwUAAABEFcEmkfj2SvOukPa9L3nSpG/8RRp4odVVAQAAAFFHsEkU+7dJz1wuVZVJab2lq1+U+pxpdVUAAABATBBsEkFNhfS3yyTfbil7gHTNP6XsEqurAgAAAGKGYBPvmv3SC9eaoSZnoHTDW1JajtVVAQAAADHltLoAnALDkF7/f1LZCsmbKV31PKEGAAAA3RLBJp6tekpa91fJ4ZS+/r9Sr9OtrggAAACwBMEmXn2yRHrzTnN5yv3SwCnW1gMAAABYiGATjw7tkP4+QzIC0shvShO+b3VFAAAAgKUINvHGXy09f7VUf1gqPFO65HeSw2F1VQAAAIClCDbxJBiUXr5Jqtgi9ciXrpwneVKsrgoAAACwHMEmniz9pfThAsmVJH3zGSmj0OqKAAAAAFsg2MSLQ59IS39hLn/5EanobEvLAQAAAOyEYBMv3nlQCjZLAy6Qxky3uhoAAADAVgg28WDvJun9v5vLU+61thYAAADAhgg28WDR/eZ8+NelglHW1gIAAADYEMHG7nb8R/ro35LTLU3+sdXVAAAAALZEsLEzw5D+fZ+5fNZ1Us4AK6sBAAAAbItgY2cfLpB2r5E8qdJ5t1tdDQAAAGBbBBu7CjRLi35qLn/uZik9z9p6AAAAABsj2NjVxmelA9uklGxp4q1WVwMAAADYGsHGjprqpcUPmcuf/6GUnGltPQAAAIDNEWzsaNVTkm+3lNFXOvu/rK4GAAAAsD2Cjd3UV0r/+bW5PHmO5Em2tBwAAAAgHhBs7Obd30kNlVLvwdKoq6yuBgAAAIgLBBs7qT8srXzCXL7gHsnpsrYeAAAAIE4QbOxk3d+kpjopb7g06GKrqwEAAADiBsHGLoIBs2mAJI27SXI4rK0HAAAAiCMEG7sofUOq2mU+t2bE162uBgAAAIgrBBu7aL235qzrJE+KpaUAAAAA8YZgYwflm6VP/yM5XNLZ37a6GgAAACDuEGzsYNWT5nzIJVJmX2trAQAAAOIQwcZqdYekTS+ay+NusrYWAAAAIE4RbKy27i9Sc4OUP1Iq/pzV1QAAAABxiWBjpUCztOpP5jItngEAAIAui3qweeihh+RwODRr1qxof1T8Kf2X5PtMSu0lDf+a1dUAAAAAcSuqwWb16tV68sknNXLkyGh+TPxa2dI0YOz1kifZ2loAAACAOBa1YFNTU6Pp06frqaeeUs+ePaP1MfFr7yZp57uS0y2NvcHqagAAAIC4FrVgM3PmTE2bNk1Tpkw54XF+v18+n6/d1C20tngeeqmUUWhtLQAAAECcc0fjTZ9//nmtW7dOq1evPumxc+fO1f333x+NMuyr9qC06e/mMi2eAQAAgFMW8Ss2ZWVluu222zRv3jwlJ5/8vpE5c+aoqqoqNJWVlUW6JPtZ97QU8EuFY6S+Z1tdDQAAABD3In7FZu3ataqoqNCZZ54Z2hYIBLR06VI9+uij8vv9crlcoX1er1derzfSZdhXoFla/T/mMi2eAQAAgIiIeLC54IIL9P7777fbdv3112vw4MG644472oWabmnvBsm3W0rOkoZ91epqAAAAgIQQ8WCTnp6u4cOHt9uWlpamnJycY7Z3S7tWmPPi8ZK7G12pAgAAAKIo6g/oxFHKWoPNOGvrAAAAABJIVLqiHW3x4sWx+Bj7Mwxp10pzuXi8tbUAAAAACYQrNrF06BOptkJyJUkFo62uBgAAAEgYBJtYKmu5WlM4RvKcvBU2AAAAgM4h2MRSa+OAIu6vAQAAACKJYBNLbTuiAQAAAIgYgk2s1B2SDpSay1yxAQAAACKKYBMrZavMec5AKS3H2loAAACABEOwiRWeXwMAAABEDcEmVri/BgAAAIgagk0sNPul3evM5aLPWVsLAAAAkIAINrGwd6MU8EupvaScAVZXAwAAACQcgk0s7FpuzovGSQ6HtbUAAAAACYhgEwu7VprzYoahAQAAANFAsIk2w5DKCDYAAABANBFsou3gx1LdAcnllQpGWV0NAAAAkJAINtHWen9NnzMlt9faWgAAAIAERbCJttCDORmGBgAAAEQLwSbaWhsH8PwaAAAAIGoINtFUe0A6uN1cLjrH2loAAACABEawiabWbmi9Bkmp2dbWAgAAACQwgk007eL+GgAAACAWCDbRxPNrAAAAgJgg2ERLU4O0Z725XDTO2loAAACABEewiZY966VAo5SWK2X3t7oaAAAAIKERbKIl9PyacZLDYW0tAAAAQIIj2EQLz68BAAAAYoZgEw3BYJsrNgQbAAAAINoINtFwYJtUf1hyp0j5I62uBgAAAEh4BJto+GihOS8eJ7mTrK0FAAAA6AYINtFQ+qY5P2OqtXUAAAAA3QTBJtLqD0u7lpvLg75kbS0AAABAN0GwibTt/5aMgNR7iNSzn9XVAAAAAN0CwSbStr1hzrlaAwAAAMQMwSaSAk3mFRuJ+2sAAACAGCLYRNKu5ZK/SkrNkfqOtboaAAAAoNsg2ERSaze0gRdJTpe1tQAAAADdCMEmUgyD+2sAAAAAixBsIuXAdunQJ5IrSRrwBaurAQAAALoVgk2ktF6t6Xeu5E23thYAAACgmyHYRErr/TV0QwMAAABijmATCXWHpLIV5jL31wAAAAAxR7CJhO0LJSMo5Q6TsoqtrgYAAADodgg2kUA3NAAAAMBSBJtT1dwofbTIXOb+GgAAAMASBJtTtes9ye+T0npLfc6yuhoAAACgWyLYnKrWbmgDL5KcfDsBAAAAK/CX+KkwjDb31zAMDQAAALAKweZU7C+VDn8qubzSgMlWVwMAAAB0WxEPNnPnztXZZ5+t9PR05ebm6rLLLlNpaWmkP8YeSl835yXnSUlp1tYCAAAAdGMRDzZLlizRzJkztWLFCi1cuFBNTU364he/qNra2kh/lPW2tdxfQ5tnAAAAwFLuSL/hm2++2W796aefVm5urtauXavzzjvvmOP9fr/8fn9o3efzRbqk6Kg9IJWtMpfPINgAAAAAVor6PTZVVVWSpOzs7A73z507V5mZmaGpqKgo2iVFxqfLJBlS3nAps6/V1QAAAADdWlSDTTAY1KxZszRx4kQNHz68w2PmzJmjqqqq0FRWVhbNkiKnZp85zznd2joAAAAARH4oWlszZ87U5s2btWzZsuMe4/V65fV6o1lGdNQdNOepHV+JAgAAABA7UQs2t9xyixYsWKClS5eqb98EHKoVCjY51tYBAAAAIPLBxjAMff/739fLL7+sxYsXq6SkJNIfYQ8EGwAAAMA2Ih5sZs6cqWeffVavvPKK0tPTVV5eLknKzMxUSkpKpD/OOgQbAAAAwDYi3jzg8ccfV1VVlSZNmqSCgoLQ9MILL0T6o6xVd9icc48NAAAAYLmoDEXrFrhiAwAAANhG1J9jk5AMg2ADAAAA2AjBpisaa6WA31wm2AAAAACWI9h0RevVGpdX8qRaWwsAAAAAgk2XtB2G5nBYWwsAAAAAgk2X1B0y5wxDAwAAAGyBYNMVoSs2tHoGAAAA7IBg0xX1XLEBAAAA7IRg0xW0egYAAABshWDTFQQbAAAAwFYINl1BsAEAAABshWDTFaGuaDQPAAAAAOyAYNMVdEUDAAAAbIVg0xUMRQMAAABshWATLsPgAZ0AAACAzRBswuWvloJN5nIKQ9EAAAAAOyDYhKt1GJonVUpKtbYWAAAAAJIINuFjGBoAAABgOwSbcNERDQAAALAdgk246IgGAAAA2A7BJlytwYbGAQAAAIBtEGzCxRUbAAAAwHYINuGqp3kAAAAAYDcEm3DRPAAAAACwHYJNuGj3DAAAANgOwSZc3GMDAAAA2A7BJlwEGwAAAMB2CDbhCAYZigYAAADYEMEmHP4qyQiYyzQPAAAAAGyDYBOO1qs1ST0kt9faWgAAAACEEGzCERqGxtUaAAAAwE4INuGgcQAAAABgSwSbcBBsAAAAAFsi2ISDYAMAAADYEsEmHAQbAAAAwJYINuEIBRuaBwAAAAB2QrAJR2tXtBSCDQAAAGAnBJtw1Le2e2YoGgAAAGAnBJtwcI8NAAAAYEsEm3AQbAAAAABbIth0VjAg1R82lwk2AAAAgK0QbDqroUoyguYyXdEAAAAAWyHYdFbrMDRvpuTyWFsLAAAAgHYINp3FM2wAAAAA2yLYdFYdrZ4BAAAAuyLYdBZXbAAAAADbIth0Fq2eAQAAANuKWrB57LHH1K9fPyUnJ2vcuHFatWpVtD4qNgg2AAAAgG1FJdi88MILmj17tu69916tW7dOo0aN0kUXXaSKiopofFxshO6xYSgaAAAAYDdRCTa/+c1vdOONN+r666/X0KFD9cQTTyg1NVX/+7//G42Piw2u2AAAAAC2FfFg09jYqLVr12rKlClHPsTp1JQpU7R8+fJjjvf7/fL5fO0mWyLYAAAAALYV8WBz4MABBQIB5eXltduel5en8vLyY46fO3euMjMzQ1NRUVGkS4oMgg0AAABgW5Z3RZszZ46qqqpCU1lZmdUldaye59gAAAAAduWO9Bv26tVLLpdL+/bta7d93759ys/PP+Z4r9crr9cb6TIiK9As1Veayyk0DwAAAADsJuJXbJKSknTWWWdp0aJFoW3BYFCLFi3S+PHjI/1xsdFQKckwl1N6WlkJAAAAgA5E/IqNJM2ePVszZszQ2LFjdc455+iRRx5RbW2trr/++mh8XPS13l+TnCW5ovItAwAAAHAKovJX+je/+U3t379f99xzj8rLyzV69Gi9+eabxzQUiBs0DgAAAABsLWqXH2655Rbdcsst0Xr72CLYAAAAALZmeVe0uECwAQAAAGyNYNMZdbR6BgAAAOyMYNMZoSs2tHoGAAAA7Ihg0xmhKzYEGwAAAMCOCDadwT02AAAAgK0RbDqDYAMAAADYGsGmMwg2AAAAgK0RbDqDrmgAAACArRFsTibQJPmrzGWCDQAAAGBLBJuTqT9szh1OKTnT2loAAAAAdIhgczKt99ek9JScLmtrAQAAANAhgs3J0DgAAAAAsD2CzcmErtjwcE4AAADArgg2J8MVGwAAAMD2CDYnEwo2XLEBAAAA7IpgczI8wwYAAACwPYLNyRBsAAAAANsj2JwM99gAAAAAtkewORmCDQAAAGB7BJuTIdgAAAAAtkewOZnQPTZ0RQMAAADsimBzIs1+qbHaXCbYAAAAALZFsDmR1qs1DpfkzbS2FgAAAADHRbA5kbYP53TyrQIAAADsir/WT6SeZ9gAAAAA8YBgcyJ0RAMAAADiAsHmRNoORQMAAABgWwSbE6ljKBoAAAAQD9xWF2BrI74u5Q6RMgqtrgQAAADACRBsTiS7vzkBAAAAsDWGogEAAACIewQbAAAAAHGPYAMAAAAg7hFsAAAAAMQ9gg0AAACAuEewAQAAABD3CDYAAAAA4h7BBgAAAEDcI9gAAAAAiHsEGwAAAABxj2ADAAAAIO4RbAAAAADEPYINAAAAgLhHsAEAAAAQ9wg2AAAAAOIewQYAAABA3CPYAAAAAIh7bqsLOJphGJIkn89ncSUAAAAArNSaCVozwonYLthUV1dLkoqKiiyuBAAAAIAdVFdXKzMz84THOIzOxJ8YCgaD2rNnj9LT0+VwOKwuRz6fT0VFRSorK1NGRobV5SBOcN6gKzhv0FWcO+gKzht0RazPG8MwVF1drcLCQjmdJ76LxnZXbJxOp/r27Wt1GcfIyMjg//QIG+cNuoLzBl3FuYOu4LxBV8TyvDnZlZpWNA8AAAAAEPcINgAAAADiHsHmJLxer+699155vV6rS0Ec4bxBV3DeoKs4d9AVnDfoCjufN7ZrHgAAAAAA4eKKDQAAAIC4R7ABAAAAEPcINgAAAADiHsEGAAAAQNwj2AAAAACIewSbE3jsscfUr18/JScna9y4cVq1apXVJcFG5s6dq7PPPlvp6enKzc3VZZddptLS0nbHNDQ0aObMmcrJyVGPHj30ta99Tfv27bOoYtjRQw89JIfDoVmzZoW2cd7geHbv3q1rrrlGOTk5SklJ0YgRI7RmzZrQfsMwdM8996igoEApKSmaMmWKtm/fbmHFsFogENDdd9+tkpISpaSkaMCAAfrZz36mtk1xOW8gSUuXLtUll1yiwsJCORwOzZ8/v93+zpwnhw4d0vTp05WRkaGsrCx9+9vfVk1NTcy+BoLNcbzwwguaPXu27r33Xq1bt06jRo3SRRddpIqKCqtLg00sWbJEM2fO1IoVK7Rw4UI1NTXpi1/8ompra0PH/OAHP9Brr72mv//971qyZIn27Nmjyy+/3MKqYSerV6/Wk08+qZEjR7bbznmDjhw+fFgTJ06Ux+PRG2+8oS1btujXv/61evbsGTrmF7/4hX7/+9/riSee0MqVK5WWlqaLLrpIDQ0NFlYOKz388MN6/PHH9eijj2rr1q16+OGH9Ytf/EJ/+MMfQsdw3kCSamtrNWrUKD322GMd7u/MeTJ9+nR98MEHWrhwoRYsWKClS5fqO9/5Tqy+BMlAh8455xxj5syZofVAIGAUFhYac+fOtbAq2FlFRYUhyViyZIlhGIZRWVlpeDwe4+9//3vomK1btxqSjOXLl1tVJmyiurraGDhwoLFw4ULj/PPPN2677TbDMDhvcHx33HGHce655x53fzAYNPLz841f/vKXoW2VlZWG1+s1nnvuuViUCBuaNm2accMNN7TbdvnllxvTp083DIPzBh2TZLz88suh9c6cJ1u2bDEkGatXrw4d88YbbxgOh8PYvXt3TOrmik0HGhsbtXbtWk2ZMiW0zel0asqUKVq+fLmFlcHOqqqqJEnZ2dmSpLVr16qpqandeTR48GAVFxdzHkEzZ87UtGnT2p0fEucNju/VV1/V2LFjdcUVVyg3N1djxozRU089Fdq/Y8cOlZeXtzt3MjMzNW7cOM6dbmzChAlatGiRtm3bJknauHGjli1bpqlTp0rivEHndOY8Wb58ubKysjR27NjQMVOmTJHT6dTKlStjUqc7Jp8SZw4cOKBAIKC8vLx22/Py8vThhx9aVBXsLBgMatasWZo4caKGDx8uSSovL1dSUpKysrLaHZuXl6fy8nILqoRdPP/881q3bp1Wr159zD7OGxzPJ598oscff1yzZ8/Wj3/8Y61evVq33nqrkpKSNGPGjND50dHvLs6d7uvOO++Uz+fT4MGD5XK5FAgE9MADD2j69OmSxHmDTunMeVJeXq7c3Nx2+91ut7Kzs2N2LhFsgAiYOXOmNm/erGXLllldCmyurKxMt912mxYuXKjk5GSry0EcCQaDGjt2rB588EFJ0pgxY7R582Y98cQTmjFjhsXVwa5efPFFzZs3T88++6yGDRumDRs2aNasWSosLOS8QcJhKFoHevXqJZfLdUwXon379ik/P9+iqmBXt9xyixYsWKB33nlHffv2DW3Pz89XY2OjKisr2x3PedS9rV27VhUVFTrzzDPldrvldru1ZMkS/f73v5fb7VZeXh7nDTpUUFCgoUOHtts2ZMgQ7dq1S5JC5we/u9DWj370I91555268sorNWLECF177bX6wQ9+oLlz50rivEHndOY8yc/PP6bJVnNzsw4dOhSzc4lg04GkpCSdddZZWrRoUWhbMBjUokWLNH78eAsrg50YhqFbbrlFL7/8st5++22VlJS023/WWWfJ4/G0O49KS0u1a9cuzqNu7IILLtD777+vDRs2hKaxY8dq+vTpoWXOG3Rk4sSJx7SU37Ztm0477TRJUklJifLz89udOz6fTytXruTc6cbq6urkdLb/c8/lcikYDErivEHndOY8GT9+vCorK7V27drQMW+//baCwaDGjRsXm0Jj0qIgDj3//POG1+s1nn76aWPLli3Gd77zHSMrK8soLy+3ujTYxPe+9z0jMzPTWLx4sbF3797QVFdXFzrmpptuMoqLi423337bWLNmjTF+/Hhj/PjxFlYNO2rbFc0wOG/QsVWrVhlut9t44IEHjO3btxvz5s0zUlNTjWeeeSZ0zEMPPWRkZWUZr7zyirFp0ybj0ksvNUpKSoz6+noLK4eVZsyYYfTp08dYsGCBsWPHDuOll14yevXqZdx+++2hYzhvYBhmt87169cb69evNyQZv/nNb4z169cbO3fuNAyjc+fJl770JWPMmDHGypUrjWXLlhkDBw40rrrqqph9DQSbE/jDH/5gFBcXG0lJScY555xjrFixwuqSYCOSOpz+/Oc/h46pr683br75ZqNnz55Gamqq8dWvftXYu3evdUXDlo4ONpw3OJ7XXnvNGD58uOH1eo3Bgwcbf/zjH9vtDwaDxt13323k5eUZXq/XuOCCC4zS0lKLqoUd+Hw+47bbbjOKi4uN5ORko3///sZdd91l+P3+0DGcNzAMw3jnnXc6/LtmxowZhmF07jw5ePCgcdVVVxk9evQwMjIyjOuvv96orq6O2dfgMIw2j54FAAAAgDjEPTYAAAAA4h7BBgAAAEDcI9gAAAAAiHsEGwAAAABxj2ADAAAAIO4RbAAAAADEPYINAAAAgLhHsAEAAAAQ9wg2AAAAAOIewQYAAABA3CPYAAAAAIh7/x+J4pqHQW04ygAAAABJRU5ErkJggg==", "text/plain": [ "
" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "V = np.zeros(nb_states)\n", "Q = np.zeros((nb_states, nb_actions))\n", "\n", "pi = np.array([a['search'], a['search']], dtype=int)\n", "\n", "V_high_history = []\n", "V_low_history = []\n", "\n", "t = 1\n", "while True:\n", " # Policy evaluation\n", " for k in range(50):\n", "\n", " Q[s['high'], a['search']] = alpha * (r_search + gamma * V[s['high']]) + (1 - alpha) * (r_search + gamma*V[s['low']])\n", " Q[s['high'], a['wait']] = r_wait + gamma * V[s['high']]\n", "\n", " Q[s['low'], a['search']] = beta * (r_search + gamma * V[s['low']]) + (1 - beta) * (-3 + gamma*V[s['high']])\n", " Q[s['low'], a['wait']] = r_wait + gamma * V[s['low']]\n", " Q[s['low'], a['recharge']] = gamma * V[s['high']]\n", "\n", " V[s['high']] = Q[s['high'], pi[s['high']]]\n", " V[s['low']] = Q[s['low'], pi[s['low']]]\n", "\n", " V_high_history.append(V[s['high']])\n", " V_low_history.append(V[s['low']])\n", " \n", " # Policy improvement\n", " pi_old = pi.copy()\n", " pi[s['high']] = Q[s['high'], :2].argmax()\n", " pi[s['low']] = Q[s['low'], :].argmax()\n", " \n", " print('Greedy policy after iteration', t)\n", " print('pi(high)=', pi[s['high']])\n", " print('pi(low)=', pi[s['low']])\n", " print('-')\n", " \n", " # Exit if the policy does not change\n", " if pi[s['high']] == pi_old[s['high']] and pi[s['low']] == pi_old[s['low']]:\n", " break\n", " t += 1\n", " \n", "plt.figure(figsize=(10, 6)) \n", "plt.plot(V_high_history, label=\"high\")\n", "plt.plot(V_low_history, label=\"low\")\n", "plt.legend()\n", "plt.show()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Value iteration\n", "\n", "In value iteration, we merge the policy evaluation and improvement in a single update rule: \n", "\n", "$$\n", " V (s) \\leftarrow \\max_{a \\in \\mathcal{A}(s)} \\sum_{s' \\in \\mathcal{S}} p(s' | s, a) \\, [ r(s, a, s') + \\gamma \\, V (s') ]\n", "$$\n", "\n", "The value of state takes the value of its greedy action. The policy is therefore implicitly greedy w.r.t the Q-values.\n", "\n", "The algorithm becomes:\n", "\n", "* while not converged:\n", "\n", " * for all states $s$:\n", " \n", " * Update the value estimates with:\n", " \n", " $$\n", " V (s) \\leftarrow \\max_{a \\in \\mathcal{A}(s)} \\sum_{s' \\in \\mathcal{S}} p(s' | s, a) \\, [ r(s, a, s') + \\gamma \\, V (s') ]\n", " $$\n", " \n", "**Q:** Modify your previous code to implement value iteration. Use a fixed number of iterations (e.g. 50) as in policy evaluation. Visualize the evolution of the V-values and print the greedy policy after each iteration. Conclude." ] }, { "cell_type": "code", "execution_count": 8, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Greedy policy after iteration 49\n", "pi(high)= 0\n", "pi(low)= 2\n", "V= [13.4228186 9.39597296]\n", "Q= [[13.4228186 11.39597296 0. ]\n", " [ 7.63221457 8.57718102 9.39597296]]\n" ] }, { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAzYAAAH5CAYAAABTbqsJAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjkuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8hTgPZAAAACXBIWXMAAA9hAAAPYQGoP6dpAABGHElEQVR4nO3deXxU9b3/8ffMZDJZSIYkkA0SCDvIpiCIuGDFWtywi1vpdetqsUpt3W6v2s3S9naxVq9ae3/W3orWWgW11ZZaARdABEHWAIKAhCxsmSxkMpk5vz8mMyQQIMvMfGcmr+fjMY9z5nvOzPmAx5i33+XYLMuyBAAAAAAJzG66AAAAAADoKYINAAAAgIRHsAEAAACQ8Ag2AAAAABIewQYAAABAwiPYAAAAAEh4BBsAAAAACS/FdAHHCgQCqqioUFZWlmw2m+lyAAAAABhiWZbq6upUXFwsu/3kfTJxF2wqKipUUlJiugwAAAAAcWLPnj0aOHDgSc+Ju2CTlZUlKVh8dna24WoAAAAAmOLxeFRSUhLOCCcTd8EmNPwsOzubYAMAAACgU1NUWDwAAAAAQMIj2AAAAABIeAQbAAAAAAkv7ubYdJbf75fP5zNdRkJxOp1yOBymywAAAAAiLuGCjWVZqqys1OHDh02XkpD69u2rwsJCnhEEAACApJJwwSYUavLz85WRkcEv6J1kWZYaGxtVXV0tSSoqKjJcEQAAABA5CRVs/H5/ONTk5eWZLifhpKenS5Kqq6uVn5/PsDQAAAAkjYRaPCA0pyYjI8NwJYkr9HfH/CQAAAAkk4QKNiEMP+s+/u4AAACQjBIy2AAAAABAWwSbGJkxY4bmzZt3wuM2m00LFy7s9PctWbJENpuN1eEAAAAAJdjiAcls3759ysnJMV0GAAAAkJAINnGisLDQdAkAAABAwmIoWgwFAgHdddddys3NVWFhob7//e+Hjx07FO3dd9/VxIkTlZaWpsmTJ2vhwoWy2Wxau3Ztu+9cvXq1Jk+erIyMDJ199tkqLy+PzR8GAAAAiCMJ32NjWZaO+PxGrp3udHRplbGnn35ad9xxh1auXKnly5frxhtv1PTp03XRRRe1O8/j8ejyyy/XJZdcogULFmjXrl0nnJ/zve99T7/85S/Vv39/feMb39DNN9+sd955pyd/LAAAACDhJHywOeLza8z9/zBy7U0/vFgZqZ3/Kxw/frweeOABSdLw4cP1yCOP6I033jgu2CxYsEA2m01PPvmk0tLSNGbMGO3du1df/epXj/vOBx98UOeff74k6Z577tGll16qpqYmpaWl9eBPBgAAACQWhqLF0Pjx49u9LyoqUnV19XHnlZeXa/z48e3CyZQpU075nUVFRZLU4XcCAAAAySzhe2zSnQ5t+uHFxq7dFU6ns917m82mQCDQoxrafmdoWFxPvxNIdP6AJZ8/IH/AUovfki8QCG5b2/yWpUDAUkvACr4/pi187Ji2gCX5LUuWZSlgWfIHpIB19Figtf249637lqXWz0pWuM2SJYXPO3qO1XqOZCm4L6nd+ZYlWVLrMUuBwNFzW09v3T/aYLX5DquDtnCDWj/X9jyrg7Y212nf0tE5bY4d97ljP93+/I4c/9njz+/oK072tR19R2c+dyrd/mgPrtn9Sxq4KKKiJ/csIEm/vmaiSnIzTJfRaQkfbGw2W5eGgyWCkSNH6k9/+pO8Xq9cLpckadWqVYarAk7N5w/oiM+vpmZ/cOsLvj/S7FeTzx/eDx4LvppbAvL6A2puafPyd7B/TFsoqLR0EGL4jzkAAD3nbTEzj727kisRJIkvfvGL+t73vqevfe1ruueee7R792794he/kKQuLVYAdIZlWWpo9utQQ7PqmlpU721Rvdd3dL91Wxfe+sLtda3bUFhpCcRvokix25TisMlhs8lhtynFYZfdZpPDLqXY7bKHtrbQ++Axh90uh01y2G2y22ytn7HJZlN4324L/rvpsNlktyt8nr31HLvdJptC+5J09JitzdYWam89X6FjUvh4cBtsCLW3O6f1Z0Tb81u/6ri24Hltv7/950Ofa9tga3es43Pbfb/aH2z7E8zW5jvVrr3j/Xbfd4LjHeno5+bJPnay7zz2+l3R3R/fifRTn/9EAcmlIDux5mwTbOJQdna2XnnlFd1yyy2aOHGixo0bp/vvv19f/OIXWRQAp9Tk8+two08HGrw61ODTwcZmHWpo1sGGZh1qPLo9UB/cHmrwqdkf2eGLNpuU4XQorfWVnupQujP4Skt1KN1pV7rTIVeKQ6kp9qMvR3DrOua903H0HFfrforDrhS7TU6HXSkOm5z24DalNbQc28b/FAAAILkRbGJkyZIlx7W1fW7NsWPJzz77bK1bty78/plnnpHT6VRpaakkacaMGcd9ZuLEiacck47E1uTzq+LwEVXWNqmitkn7Dh8Jbmtb2w4fkaeppVvf7UqxKzvdqSxXivqkpaiPq/WVltKmzXn0fZvzMl0pbYJLMJAQJAAAQCwRbOLUH//4Rw0ZMkQDBgzQunXrdPfdd+vqq69Wenq66dIQRUea/fqopl4f1dTrk0NH2gSWYHg51Ojr1Pek2G3KyUxVbkaqcjKdys1MVU5Gartt6BU6Lz21a4thAAAAxBOCTZyqrKzU/fffr8rKShUVFemqq67Sgw8+aLosRMjhxmZtr64/+qqp17aqeu09fOSUn81IdajInabivukqzE5TUd90FbuPbvOz0pSdnkKPCQAA6FUINnHqrrvu0l133WW6DPSAZVmqrvNqW1W9tlfXhcPLRzX12l/ffMLP5WQ4NSy/j0pyM1TsTldR3zQVudNU5E5XsTud0AIAANABgg0QIT5/QBsrPHr/44NaveuQVu86pOo67wnPL3anaWh+Hw0LvfoHt3l9XDGsGgAAIDkQbIBuOtzYrDW7D+n9jw/p/V2H9OEnh9Xka7+6mMNu06DcjKMBpjW8DM3voz4u/vUDAACIFH6zAjrBsizt3N+g93cd0ppdwSCzvbr+uPP6Zjg1qTRHZwzK0eRBOZpQ0ldpTiblAwAARBvBBjiBBm+L/r2lWq9vqNSKHQd0oOH4eTFD+mVq0qAcTR6co0mDcjSkXx/Z7cx/AQAAiDWCDdCGp8mnf2+u1t/X79PSrTXythwdWpaaYteEge7W3phcTRqUo9zMVIPVAgAAIIRgg16vttGnxZur9Nr6fXpr2341+4+GmcF5GbpkXJEuHJ2vsQPccqUwrAwAACAeEWxiZMaMGZo4caIeeugh06VA0sGGZi3eVKm/r6/UO9v3qyVghY8N7Z+pS8cVada4Io0qzGJpZQAAgARAsEGvUVPn1T83Veq19ZVavuOA/G3CzKjCLM0aW6RLxhVqeEGWwSoBAADQHQQbJL1NFR49vvQj/W39vnZh5rTibF0yrkifGVuoof37GKwQAAAAPWU3XUBvdOjQIV1//fXKyclRRkaGZs2apW3btkkKLivcv39/vfDCC+HzJ06cqKKiovD7t99+Wy6XS42NjTGvPVFYlqWVOw7oxqfe0yUPv6WX11XIH7A0fqBb98wapaV3ztDfbjtXcy8YRqgBAABIAonfY2NZks/QL/jODKkb8y9uvPFGbdu2TS+//LKys7N1991365JLLtGmTZvkdDp13nnnacmSJfrCF76gQ4cOafPmzUpPT9eWLVs0atQoLV26VGeeeaYyMjKi8IdKbIGApX9trtJjSz/SB7sPS5LsNumScUX6xvlDNXaA22yBAAAAiIrEDza+RuknxWau/Z8VUmpmlz4SCjTvvPOOzj77bEnSM888o5KSEi1cuFBXXXWVZsyYoSeeeEKStGzZMp1++ukqLCzUkiVLNGrUKC1ZskTnn39+xP84iay5JaBFa/fqiWU7wg/OTE2x66pJA/W184ZoUF7X/jkBAAAgsSR+sEkwmzdvVkpKiqZOnRpuy8vL08iRI7V582ZJ0vnnn6/bb79dNTU1Wrp0qWbMmBEONl/+8pf17rvv6q677jL1R4grDd4WPbdqj37/1g7tq22SJGW5UvSlaYN00/TBys9KM1whAAAAYiHxg40zI9hzYuraUTBu3Djl5uZq6dKlWrp0qR588EEVFhbqZz/7mVatWiWfzxfu7emtDjY06+l3P9bTyz/W4UafJKl/lktfPqdMX5xaquw0p+EKAQAAEEuJH2xsti4PBzNp9OjRamlp0cqVK8Ph5MCBAyovL9eYMWMkSTabTeeee64WLVqkjRs36pxzzlFGRoa8Xq+eeOIJTZ48WZmZifNnjqTK2iY9vvQj/XnVHh3x+SVJg/Iy9PXzhupzZwxQmpMHaAIAAPRGiR9sEszw4cM1e/ZsffWrX9UTTzyhrKws3XPPPRowYIBmz54dPm/GjBn6zne+o8mTJ6tPn+CqXeedd56eeeYZ3XnnnabKN8ayLD3//h79+NXNqvO2SAou13zLjKGaNbZIDjsP0QQAAOjNCDYGPPXUU7r99tt12WWXqbm5Weedd57+/ve/y+k8Onzq/PPPl9/v14wZM8JtM2bM0KJFi9q19Qb7ao/onr+u19KtNZKkCSV99d1Pj9A5w/rJ1o1V6QAAAJB8bJZlWac+LXY8Ho/cbrdqa2uVnZ3d7lhTU5N27typsrIypaUxKbw7Eunv0LIs/WX1J/rRq5tU19Si1BS7vnPRCH3l3CH00AAAAPQCJ8sGx6LHBnGpsrZJ9774od4sP9pL88urxmtYfpbhygAAABCPCDaIK5Zl6a9r9uqHr2yUp6lFqQ67vn3RCH313DKlOOymywMAAECcItggblR5mvSfL67XG1uqJUkTBrr1i6smaHgBvTQAAAA4OYINjLMsSwvX7tX3X96k2iM+pTrsun3mcH39vCH00gAAAKBTCDYwqrquSf/54gb9a3OVJGncgGAvzchCemkAAADQeQkZbAKBgOkSEla8/N1ZlqWX11XogZc36nCjT06HTbdfOFxfP3+onPTSAAAAoIsSKtikpqbKbreroqJC/fv3V2pqKs8x6STLstTc3KyamhrZ7XalpqYaq6XB26LvPL9Or2+slBR80OYvrpqg0UUnX8IPAAAAOJGECjZ2u11lZWXat2+fKioqTJeTkDIyMlRaWiq73UyvSO0Rn2566j2t2X1YKXabbrtwuG6ZQS8NAAAAeiahgo0U7LUpLS1VS0uL/H6/6XISisPhUEpKirFeroMNzfqP/12pjRUeZael6A83T9EZpTlGagEAAEBySbhgI0k2m01Op1NOp9N0Keikak+T5vx+pbZV1ysvM1X/9+WpGlPM0DMAAABERkIGGySWTw41as7vV2rXgUYVZLv0zFfO0rD8PqbLAgAAQBIh2CCqdu5v0JwnV6iitkkDc9K14CtnqTQvw3RZAAAASDIEG0TN1qo6zfn9StXUeTWkf6ae+cpUFbnTTZcFAACAJESwQVSs/6RW1/+/lTrU6NOowiz935enqn+Wy3RZAAAASFIEG0Tc+x8f1E1PrVKdt0UTBrr19M1T1DfD3HNzAAAAkPwINoiod7bv11eefl9HfH5NGZyr/71xsrLSWL0OAAAA0UWwQcT8e0uVvvGnNWpuCejc4f30u/+YrPRUh+myAAAA0AsQbBARf/twn25/7gO1BCxdNKZAj3zxdLlSCDUAAACIDYINeuyvqz/RnS+sU8CSrphQrF9ePUFOh910WQAAAOhFCDbokf9bsUv3LdwgSbpmcol+8rlxcththqsCAABAb0OwQbc9997ucKi58ezBuv+yMbITagAAAGAAwQbdsnmfR/e/vFGS9I3zh+ruz4yUzUaoAQAAgBlMhECXNTa36FvPfqDmloAuGNmfUAMAAADjCDbosh++sknbq+uVn+XSL66aQKgBAACAcV0ONsuWLdPll1+u4uJi2Ww2LVy4MHzM5/Pp7rvv1rhx45SZmani4mJdf/31qqioiGTNMOiVdRV6btUe2WzSQ9dMVF4fl+mSAAAAgK4Hm4aGBk2YMEGPPvrocccaGxu1Zs0a3XfffVqzZo1efPFFlZeX64orrohIsTBrz8FG/eeL6yVJc2cM09nD+hmuCAAAAAjq8uIBs2bN0qxZszo85na7tXjx4nZtjzzyiKZMmaLdu3ertLS0e1XCOJ8/oG89+4HqvC2aNChH82YON10SAAAAEBb1VdFqa2tls9nUt2/fDo97vV55vd7we4/HE+2S0A2//OdWrd1zWNlpKfrNtROVwgM4AQAAEEei+ttpU1OT7r77bl133XXKzs7u8Jz58+fL7XaHXyUlJdEsCd3w1rYaPb70I0nSzz4/XgNzMgxXBAAAALQXtWDj8/l09dVXy7IsPfbYYyc8795771VtbW34tWfPnmiVhG6oqfPq239eJ0maM7VUs8YVGa4IAAAAOF5UhqKFQs2uXbv073//+4S9NZLkcrnkcrGyVjwKBCzd8fxa7a/3amRBlu67bIzpkgAAAIAORTzYhELNtm3b9OabbyovLy/Sl0CMPPnWDr21bb/SnHb99ounK83pMF0SAAAA0KEuB5v6+npt3749/H7nzp1au3atcnNzVVRUpC984Qtas2aNXn31Vfn9flVWVkqScnNzlZqaGrnKEVVr9xzWf/+jXJJ0/2WnaURBluGKAAAAgBOzWZZldeUDS5Ys0QUXXHBc+w033KDvf//7Kisr6/Bzb775pmbMmHHK7/d4PHK73aqtrT3pEDZET12TT5c+/LZ2H2zUpeOK9MgXT5fNZjNdFgAAAHqZrmSDLvfYzJgxQyfLQl3MSYgzlmXpP1/aoN0HGzWgb7p+8rlxhBoAAADEPR5Ggnb+8v4nemVdhRx2mx6+7nS5052mSwIAAABOiWCDsO3VdXrg5Y2SpDsuGqFJg3IMVwQAAAB0DsEGkqQmn1+3LvhAR3x+TR+Wp1vOH2q6JAAAAKDTCDaQJM3/+2ZtqaxTXmaqfn31RNntzKsBAABA4iDYQP/cWKmnl++SJP3i6gnKz04zXBEAAADQNQSbXq7e26J7X1wvSfrquWW6YGS+4YoAAACAriPY9HJ/eGenDjQ0q6xfpu68eJTpcgAAAIBuIdj0YrVHfPrdsh2SpHkzhys1hdsBAAAAiYnfZHux/31rhzxNLRpR0EeXjy82XQ4AAADQbQSbXupgQ7P+9+2dkoLPrGEVNAAAACQygk0v9cSyj9TQ7Ndpxdm6+LRC0+UAAAAAPUKw6YWq65r09LsfS5K+8+kRstnorQEAAEBiI9j0Qv/z5kdq8gV0emlflncGAABAUiDY9DIVh49owcrdkqTvXDSS3hoAAAAkBYJNL/PIm9vV7A9oalmupg/LM10OAAAAEBEEm15kz8FGPb9qjyTpO5+mtwYAAADJg2DTi/zmjW1qCVg6d3g/TSnLNV0OAAAAEDEEm15iR029XlzziaRgbw0AAACQTAg2vcRD/9qmgCXNHJ2viSV9TZcDAAAARBTBphcor6zTKx9WSJK+fdEIw9UAAAAAkUew6QV+vXirLEu6ZFyhTit2my4HAAAAiDiCTZLbsLdWr2+slM0mfXsmvTUAAABITgSbJPerxVslSbMnFGt4QZbhagAAAIDoINgksdW7DunfW6rlsNt0O701AAAASGIEmyT2q8XlkqTPnzFAZf0yDVcDAAAARA/BJkkt/+iA3tl+QE6HTd/61HDT5QAAAABRRbBJQpZlhXtrrj2zVCW5GYYrAgAAAKKLYJOE3tq2X6s+PqTUFLvmXjDMdDkAAABA1BFskoxlWfrlP4O9Nf9x1iAVutMMVwQAAABEH8EmybyxuVrrPqlVutOhW2YMNV0OAAAAEBMEmyQSCFj6Zetza26cPlj9+rgMVwQAAADEBsEmiby2oVKb93nUx5Wir507xHQ5AAAAQMwQbJKEP2Dp1/8K9tZ8+Zwy5WSmGq4IAAAAiB2CTZJYvKlS26vr5U536svnlpkuBwAAAIgpgk2SeHHNXknStVNKlJ3mNFwNAAAAEFsEmyRwuLFZS8prJEmfO32g4WoAAACA2CPYJIG/r69Usz+gUYVZGlmYZbocAAAAIOYINklg4QfBYWifPX2A4UoAAAAAMwg2Ce6TQ4167+ODstmkKyYWmy4HAAAAMIJgk+AWra2QJJ1Vlqcid7rhagAAAAAzCDYJzLKs8DC0K0+ntwYAAAC9F8EmgW3a59G26nqlptj1mbFFpssBAAAAjCHYJLDQMLQLR+XLnc6zawAAANB7EWwSlD9gadHa0DA0VkMDAABA70awSVArdxxQlccrd7pTM0b2N10OAAAAYBTBJkG91LpowCXjiuRKcRiuBgAAADCLYJOAmnx+vb6hUhIP5QQAAAAkgk1CemNzteq8LRrQN12TB+WYLgcAAAAwjmCTgELD0GZPLJbdbjNcDQAAAGAewSbBHGpo1tKt1ZJYDQ0AAAAIIdgkmL+t3yef39KYomyNKMgyXQ4AAAAQFwg2Cebos2uKDVcCAAAAxA+CTQLZc7BRqz4+JJtNumICw9AAAACAEIJNAnl5XYUk6eyheSp0pxmuBgAAAIgfBJsEYVlWm9XQ6K0BAAAA2iLYJIiNFR5tr66XK8Wuz4wtNF0OAAAAEFcINgliYWtvzczRBcpOcxquBgAAAIgvBJsE4A9Y4fk1PLsGAAAAOB7BJgEs/+iAquu86pvh1Pkj+psuBwAAAIg7BJsEEFo04NJxRUpN4R8ZAAAAcCx+S45zR5r9+sfGSknSZxmGBgAAAHSIYBPn/rW5SvXeFg3MSdekQTmmywEAAADiUpeDzbJly3T55ZeruLhYNptNCxcubHfcsizdf//9KioqUnp6umbOnKlt27ZFqt5eZ9Ha4DC0KycOkM1mM1wNAAAAEJ+6HGwaGho0YcIEPfroox0e//nPf66HH35Yjz/+uFauXKnMzExdfPHFampq6nGxvc3BhmYtKa+RJF15erHhagAAAID4ldLVD8yaNUuzZs3q8JhlWXrooYf0X//1X5o9e7Yk6Y9//KMKCgq0cOFCXXvttT2rtpf52/p9aglYGjsgW8Pys0yXAwAAAMStiM6x2blzpyorKzVz5sxwm9vt1tSpU7V8+fIOP+P1euXxeNq9EBR6KOeVE1k0AAAAADiZiAabysrg6l0FBQXt2gsKCsLHjjV//ny53e7wq6SkJJIlJazdBxq1etch2W3S5RMYhgYAAACcjPFV0e69917V1taGX3v27DFdUlwILRpw9tB+KshOM1wNAAAAEN8iGmwKCwslSVVVVe3aq6qqwseO5XK5lJ2d3e7V21mWpZdCq6Hx7BoAAADglCIabMrKylRYWKg33ngj3ObxeLRy5UpNmzYtkpdKahv2erSjpkFpTrsuPq3g1B8AAAAAerkur4pWX1+v7du3h9/v3LlTa9euVW5urkpLSzVv3jz9+Mc/1vDhw1VWVqb77rtPxcXFuvLKKyNZd1J7qXXRgJmjC5SV5jRcDQAAABD/uhxs3n//fV1wwQXh93fccYck6YYbbtAf/vAH3XXXXWpoaNDXvvY1HT58WOecc45ef/11paUxT6QzWvwBvfJhhSTpswxDAwAAADrFZlmWZbqItjwej9xut2pra3vlfJtlW2t0/f97TzkZTr33vZlyOoyv7wAAAAAY0ZVswG/NcebldcHemsvGFxNqAAAAgE7iN+c4YlmW3tpWI0n6zNiOV5EDAAAAcDyCTRzZub9BVR6vUh12TRqUY7ocAAAAIGEQbOLI8h0HJEmnl/ZVmtNhuBoAAAAgcRBs4siKHQclSdOG5hmuBAAAAEgsBJs4YVmWln8U7LE5awjBBgAAAOgKgk2c+KimXvvrvXKl2HV6aV/T5QAAAAAJhWATJ5a3DkObNChHrhTm1wAAAABdQbCJEysYhgYAAAB0G8EmDliWpRWtK6KxcAAAAADQdQSbOLCtul4HGpqV5rRr/EC36XIAAACAhEOwiQOh1dAmD8plfg0AAADQDQSbOBAKNgxDAwAAALqHYGNYIGBp5c7QwgG5hqsBAAAAElOK6QJ6u/KqOh1q9Ckj1aHxA/uaLgdIPoGAFPBJfl9wG/AHX5ZfCrS07gfa7IfaA8ecEzovENxagTZt/jZtbV7HtsuSLKv1vXVMe+CYdqtNu3X0s22/o8M2HXO+jj9PVgfnHXN++Hjb9yc7dsz7k55zorYTfb6DYx0eP8m5nf7cyb6ym9fryTW7LdbX66GY//0A6JTLfiW5B5quotMINoaFVkObPDhXTgcdaEgCliW1NEm+I5Kv8QTbNi+/V2rxSv7m4KuluX3bsdvQfqDlaFjxt7QJLy3tj4V+2QcAAF3T3GC6gi4h2Bi2/COGoSEOBAKS19P6qmvzOvZ9ndTUwXm+xvbhJRHYUySbQ7I7WvftwW34vUOy29ufZ3NINlvrvr311Wbf3kGbzR48X2rTZju6L9sx7bY27cfsn7TNdnybbK3XbdumDs5TB5+R2u0cd94JjrV7f5JzTtR2os93eKwDHR7voO2k33OSY6e6/kl187M9umZ3mbgmTsjIPQBI6lNguoIuIdgYFJxfc1CSNI0HcyJSLCsYLhr2S40HpMaDrduTvQ4Gh1VFmsMlOdMkZ4bkTG+zbd1PcQXPSUlt3bokR+oxW5fkcHbQliLZncFjdmcwPIT2Q8fsKa1tKe3Ps7P6IAAAyYZgY9DmSo9qj/iUmerQuAE8vwadYFnBwOLZG3zV7pU8n0ieiqP79dXBoWDdkZImubLavLKP2XZ0LEtKzew4vBAgAABAjBBsDAoNQzuzLFcpzK+BFBwSdniXtH+bVLu7NaxUtIaY1gDj93buuxwuKbOflJErZeR14pUb7A0BAABIQAQbg0ILBzAMrRdqaZYOfiTVbJFqtkr7y4PbA9s619vSp0DKHiBlFwdXK8keILkHSNkDpawCKaNfsBeFcdkAAKCXINgY4m87v4YHcyYvb520f2v78LK/XDq488RzWhypUt4wKWdwm8Ay4Oh+VnFwTgoAAADCCDaGbKrwqK6pRVmuFI0pyjZdDiLBsqQD26XdK6Q9K4Ov/VtPfH5qltR/hNRvZJvtSKnvoODkdwAAAHQavz0ZsnzHfknSFObXJC5fk1TxgbRnhbTnvWCQaTxw/HmZ+cHA0m9E+21WEUPFAAAAIoRgY8iKHQxDSzj1NcEQs7s1yOxbG3xYZFspaVLxGVLJFKn0LGngmcEJ/AAAAIgqgo0BLf6A3mudX3MWCwfEL79P2rlU2vyKtHOZdHDH8edk5kulU6WSqVLJWVLRBOa/AAAAGECwMWBDhUf13hZlp6VoNPNr4kuLV9qxRNq0SNryN6npcJuDNil/dLA3puSsYKDJKWM4GQAAQBwg2BgQWuZ56pA8Oez8Umyc74i0/Q1p88tS+WuS13P0WGa+NPpyacRngoEmva+xMgEAAHBiBBsDQg/mZBiaQc0N0rbFwZ6Zrf+QfA1Hj2UVSaOvkMbMDs6TsTvM1QkAAIBOIdjEmM8f0KqPWxcOINjElrcuGGI2LQqGmpYjR4+5S4JBZvQVwQn/dlaqAwAASCQEmxhbv7dWjc1+9c1walRhlulyeoeDO6V3fyutfUZqaTranjM4GGbGzA6uZMZcGQAAgIRFsImx0DC0qWW5sjO/Jroq10tvPyRtfFGyAsG2vGFHw0zheMIMAABAkiDYxFho4QCGoUWJZUm73pHe/rW0/V9H24ddJJ0zTxo0nTADAACQhAg2MdTcEtD7Hx+SJJ3FgzkjKxCQtr4WDDSfrAq22ezSaZ8LBprCcUbLAwAAQHQRbGLow08O64jPr9zMVI3IZ35NRPh90vq/BIec7S8Ptjlc0ulfks7+lpRbZrQ8AAAAxAbBJoZCw9DOGsL8mh5rbpDW/FF69xHJ80mwzZUtnfkV6axbpD75ZusDAABATBFsYmj5Dp5f02ONB6X3fietfFw6EhzWpz4F0lnflCbfJKW5zdYHAAAAIwg2MeJt8Yfn17BwQDdtWiS9eofUuD/4PqdMmn67NOE6yZlmtjYAAAAYRbCJkXV7auVtCahfn1QNy+9jupzE0nBAeu1OacNfg+/7jZRm3BNcstnuMFsbAAAA4gLBJkbCz68Zkicbyw133uZXpFe/LTXUBFc5mz4vGGpSXKYrAwAAQBwh2MTI8h3B4VMMQ+ukxoPS3++UNrwQfN9/lHTl/0gDJpmtCwAAAHGJYBMDTT6/1uw+LEmaxvNrTm3zq629NNX00gAAAKBTCDYx8MHuw2puCah/lktD+mWaLid+NR6UXrsr+FwaKTiX5srHpIH00gAAAODkCDYxEFrmeRrza05sy9+kV+Yd7aU5+zZpxr2sdgYAAIBOIdjEQOjBnAxD60DjQen1e6QP/xx8329kcC7NwMlm6wIAAEBCIdhEWZPPr7Wt82t4MOcxyl+TXrldqq+ilwYAAAA9QrCJstW7DqnZH1BhdpoG52WYLic++JqCiwOsWxB8329E61waemkAAADQPQSbKGs7DI35NZKaG6Rnr5N2Lg320ky7Vbrge/TSAAAAoEcINlEWejDnWUNyDVcSB5pqpWeulvaskFL7SNc9K5WdZ7oqAAAAJAGCTRQ1Nrdo3SeHJUnThvQzW4xpjQelP31OqvhASnNLX3qRoWcAAACIGIJNFK3edUg+v6UBfdNVkptuuhxz6qulP86WqjdJGXnSfyyUisabrgoAAABJhGATRaFhaFOH5Pbe+TW1e6U/XiEd2C71KZSuXyTljzJdFQAAAJIMwSaKVrR5MGevdHBnMNQc3i25S4KhJm+o6aoAAACQhAg2UdLgbdGHn9RK6qXPr6nZGhx+Vlch5Q6Rrn9Z6ltiuioAAAAkKYJNlKz6+KBaApYG5qSrJLeXPb+mcoP0f1dKDTVS/1HBnpqsQtNVAQAAIIkRbKJkxY6DknrhMLS9q6X/+5zUdFgqHB9cKCCzl/0dAAAAIOYINlGyfEfo+TW96Jf6XculZ66SmuukgWdKc16Q0vuargoAAAC9AMEmChq8LdqwNzi/ZtrQXhJsPnpTevY6qeWINPjc4MM3XVmmqwIAAEAvQbCJgi2VdfIHLBVku1Tctxc8v6b8den56yW/Vxo2U7rmT5KzF/y5AQAAEDfspgtIRuWVdZKkkYXZhiuJgY0vSX+eEww1oy6Trl1AqAEAAEDM0WMTBVurWoNNQR/DlUTZpkXSCzdLVkAad5V05eOSg1sKAAAAsUePTRRsqfRISvIem4M7pYVzg6Hm9C9Jn32CUAMAAABjCDYRZllWeCjaqMIknTzf0hzsqWmuk0qnSZf9RrI7TFcFAACAXiziwcbv9+u+++5TWVmZ0tPTNXToUP3oRz+SZVmRvlRcqqn36lCjT3abNCw/SYei/fuHUsUaKa2v9Pnf01MDAAAA4yL+G+nPfvYzPfbYY3r66ad12mmn6f3339dNN90kt9ut2267LdKXizuh3prBeZlKcyZhL8a2xdK7vw3uX/k/knug2XoAAAAARSHYvPvuu5o9e7YuvfRSSdLgwYP17LPP6r333ov0peLS0RXRknAYmmef9NLXg/tTvi6NutRsPQAAAECriA9FO/vss/XGG29o69atkqR169bp7bff1qxZszo83+v1yuPxtHslslCwGVGQZMEm4Jde/KrUeEAqHCdd9EPTFQEAAABhEe+xueeee+TxeDRq1Cg5HA75/X49+OCDmjNnTofnz58/Xz/4wQ8iXYYx5VVJunDAW7+SPn5LcmZKX3hKcqaZrggAAAAIi3iPzfPPP69nnnlGCxYs0Jo1a/T000/rF7/4hZ5++ukOz7/33ntVW1sbfu3ZsyfSJcWMP2AdfYZNMgWbXe9KS34S3L/0l1K/4WbrAQAAAI4R8R6bO++8U/fcc4+uvfZaSdK4ceO0a9cuzZ8/XzfccMNx57tcLrlcrkiXYcSeg41q8gXkSrFrUF6m6XIio/Gg9NevBJ9XM/5aaeJ1pisCAAAAjhPxHpvGxkbZ7e2/1uFwKBAIRPpScWdL6/ya4QV95LDbDFcTAZYlLbpV8uyVcodKl/7CdEUAAABAhyLeY3P55ZfrwQcfVGlpqU477TR98MEH+tWvfqWbb7450peKO+EV0QqyDVcSIe89KZX/TXKkSlc9JbmSaHgdAAAAkkrEg81vf/tb3XffffrmN7+p6upqFRcX6+tf/7ruv//+SF8q7pRXBVd0S4qFA/Z9KP3ze8H9T/9YKppgth4AAADgJCIebLKysvTQQw/poYceivRXx73wUs+JHmy89dILN0n+ZmnkJdKUr5muCAAAADipiM+x6a2afH59fKBRUhL02Pz9TunAdil7gDT7UcmWBPOFAAAAkNQINhGyvbpe/oClvhlO5Wcl8Cpv656T1i2QbHbp87+XMnJNVwQAAACcEsEmQkLPrxlRkCVbovZw7N8uvXpHcH/GvdKgs83WAwAAAHQSwSZCQvNrEnYYWos3OK/G1yANPlc69zumKwIAAAA6jWATIaFn2IxM1GCz+AGp8kMpI0/63JOS3WG6IgAAAKDTCDYREhqKNrIgAYPNlr9LKx8L7l/5mJRdZLYeAAAAoIsINhFQ2+jTvtomSQm41LO3Tnr5W8H9abdKIy42Ww8AAADQDQSbCChv7a0Z0Ddd2WlOw9V00conpMb9Uu4Q6cIHTFcDAAAAdAvBJgLKwyui9TFcSRc11Urv/ja4P+NeKSXVbD0AAABANxFsIqC80iNJGlmYbbiSLlrxmNR0WOo/Shr7edPVAAAAAN1GsImAhFzqufGgtPzR4P6Me1gFDQAAAAmNYNNDlmWFg01CLfW8/BHJ65EKxkqjZ5uuBgAAAOgRgk0PVXqa5GlqkcNu05D+mabL6ZyG/dKKx4P7M+6V7NwGAAAASGz8RttDoQdzDumXKVdKggzneuchydcgFU2URl1quhoAAACgxwg2PZRww9DqqqT3fh/cv+B7ks1mth4AAAAgAgg2PbQ1FGwKEiTYvP1rqeWINGCyNPwi09UAAAAAEUGw6aEtidRjU7tXev//Bfc/RW8NAAAAkgfBpgda/AFtr6mXJI1KhGfYvPVLye+VSs+WhlxguhoAAAAgYgg2PfDxgUY1twSUkerQwJx00+Wc3OHd0po/BvfprQEAAECSIdj0QGjhgOEFWbLb4zwoLPtvKeCTys6TBp9juhoAAAAgogg2PVBe6ZEkjYr3hQMO7pA+eCa4f8F/ma0FAAAAiAKCTQ+UVwV7bEbE+8IBS38uWX5p2EypdKrpagAAAICII9j0QGgo2qh4DjY1W6UP/xzcv+A/zdYCAAAARAnBppsam1u062CjpDhf6nnpzyQrII2YJQ2YZLoaAAAAICoINt20vbpeliXlZaaqXx+X6XI6VrVJ2vDX4D69NQAAAEhiBJtuSogHcy6ZL8mSRl8hFY03XQ0AAAAQNQSbbiqP92Cz70Np88uSbPTWAAAAIOkRbLppa1WcLxzw5k+C27Gfl/JHm60FAAAAiDKCTTeFhqKNiMdn2OxdLW19TbLZpRn3mK4GAAAAiDqCTTccbGhWTZ1XUpwGm1BvzfhrpH7DzdYCAAAAxADBphu2VHokSaW5Gcp0pRiu5hi7V0rb/yXZHNL5d5muBgAAAIgJgk03bI3nYWhv/ji4PX2OlDvEbC0AAABAjBBsuqE8XhcO2PmWtHOZZHdK591puhoAAAAgZgg23RCXz7CxrKNza864XupbarYeAAAAIIYINl1kWVZ4KFpcBZtd70i735UcLum875quBgAAAIgpgk0XfXLoiBqa/XI6bCrrl2m6nKM+eCa4nXCtlF1sthYAAAAgxgg2XVTe2lsztH8fOR1x8tfX3Chtfjm4P+E6s7UAAAAABsTJb+aJI7RwQFwNQyv/u9RcH5xXU3qW6WoAAACAmCPYdFF5PM6v+fDPwe34aySbzWwtAAAAgAEEmy4KBZu4Weq5vkba/kZwf/w1ZmsBAAAADCHYdEFzS0Af1dRLkkYWZhuuptXGFyXLLxWfIfUbbroaAAAAwAiCTRfs3N+gloClLFeKit1ppssJWvdccEtvDQAAAHoxgk0XbKn0SJJGFGbJFg9zWfZvkyrWSDaHNPbzpqsBAAAAjCHYdMHWeFsR7cPng9thF0p9+putBQAAADCIYNMF4RXRCuIg2FhW+9XQAAAAgF6MYNMFW+Jpqec9K6XDu6TUPtLIS0xXAwAAABhFsOmkem+LPjl0RFKc9NiEFg0YfYWUmmG2FgAAAMAwgk0nhebX5Ge5lJOZaraYFq+08aXg/gSGoQEAAAAEm04qj6dhaNsWS02HpawiafC5pqsBAAAAjCPYdFIo2IyKh2DzYeswtHFfkOwOs7UAAAAAcYBg00mhYDPC9PyaI4ekrf8I7o+/1mwtAAAAQJwg2HSCZVkqrwr12GSbLWbTIsnfLOWfJhWONVsLAAAAECcINp1QU+/VwYZm2W3S8II+ZotZF3p2zdVm6wAAAADiCMGmE7ZW1kuSBudlKs1pcE7LoV3S7ncl2aRxV5mrAwAAAIgzBJtO2FLpkRQH82vWPx/clp0ruQeYrQUAAACIIwSbToiLpZ4tS/qwNdiM59k1AAAAQFsEm07YWhUHSz3vWyvt3yqlpEmjrzBXBwAAABCHCDanEAhY2loVnGMzwmSwCS0aMPISKc3wymwAAABAnCHYnMLug4064vPLlWLX4LxMM0X4W6QNLwT3J/DsGgAAAOBYBJtTCD2/ZnhBHznsNjNF7FgiNdRIGXnS0E+ZqQEAAACIYwSbUwgtHGB0RbQPnwtux35ecjjN1QEAAADEKYLNKYSCjbGFA7x10uZXg/vjGYYGAAAAdCQqwWbv3r360pe+pLy8PKWnp2vcuHF6//33o3GpqAsNRRtZaGjC/pa/SS1HpNyh0oAzzNQAAAAAxLmUSH/hoUOHNH36dF1wwQV67bXX1L9/f23btk05OTmRvlTUeVv82rm/QZI00tRQtHWtw9DGXyPZDM3xAQAAAOJcxIPNz372M5WUlOipp54Kt5WVlUX6MjGxvbpe/oAld7pTBdmu2Bfg2SftXBrcH3917K8PAAAAJIiID0V7+eWXNXnyZF111VXKz8/X6aefrieffPKE53u9Xnk8nnaveBGaXzOyMEs2E70lG16QrIBUMlXKTcxwCAAAAMRCxIPNjh079Nhjj2n48OH6xz/+oVtuuUW33Xabnn766Q7Pnz9/vtxud/hVUlIS6ZK6LTy/xtQwtA9bH8o5/hoz1wcAAAAShM2yLCuSX5iamqrJkyfr3XffDbfddtttWrVqlZYvX37c+V6vV16vN/ze4/GopKREtbW1ys42NGG/VW2jT5v2eZSbmaqRsV4VrWqT9Ng0ye6UvrtVysiN7fUBAAAAwzwej9xud6eyQcTn2BQVFWnMmDHt2kaPHq2//vWvHZ7vcrnkchmYv9IJ7gynpg3NM3PxUG/N8E8TagAAAIBTiPhQtOnTp6u8vLxd29atWzVo0KBIXyp5BQLS+r8E9ycwDA0AAAA4lYgHm29/+9tasWKFfvKTn2j79u1asGCBfve732nu3LmRvlTy2vWO5NkrudzS8ItNVwMAAADEvYgHmzPPPFMvvfSSnn32WY0dO1Y/+tGP9NBDD2nOnDmRvlTy+rD12TWnzZacaWZrAQAAABJAxOfYSNJll12myy67LBpfnfx8R6RNLwf3x19rthYAAAAgQUS8xwY9VP6a5PVI7hKpdJrpagAAAICEQLCJNxtfDG7HXy3Z+ccDAAAAdAa/OceTQEDa+VZwf+QlZmsBAAAAEgjBJp5UbZCaDkupfaSiiaarAQAAABIGwSae7HonuC2dJjmisq4DAAAAkJQINvHk47eD28HTzdYBAAAAJBiCTbwIBNoEm3PN1gIAAAAkGIJNvKje2GZ+zQTT1QAAAAAJhWATL0K9NaVnSQ6n2VoAAACABEOwiRfhYWjnmK0DAAAASEAEm3gQCBxdEY35NQAAAECXEWziQfUm6cghyZnJ/BoAAACgGwg28YD5NQAAAECPEGziwcdvBbfMrwEAAAC6hWBjGvNrAAAAgB4j2JjWdn5N8UTT1QAAAAAJiWBjWqi3hvk1AAAAQLcRbEwLz6+ZbrYOAAAAIIERbEwKBKSPmV8DAAAA9BTBxqSazdKRg5IzQyo+3XQ1AAAAQMIi2JjE82sAAACAiCDYmMTzawAAAICIINiYwvwaAAAAIGIINqbUbGF+DQAAABAhBBtTQvNrSqYyvwYAAADoIYKNKcyvAQAAACKGYGNCICDtYn4NAAAAECkEGxNqtkiNB5hfAwAAAEQIwcaEtvNrUlLN1gIAAAAkAYKNCbtagw3zawAAAICIINjEmmUd7bEh2AAAAAARQbCJtdD8mpR0qfgM09UAAAAASYFgE2uh3ppS5tcAAAAAkUKwiTWeXwMAAABEHMEmltrNr+H5NQAAAECkEGxiifk1AAAAQFQQbGIp/PyaKcyvAQAAACKIYBNLDEMDAAAAooJgEys8vwYAAACIGoJNrNSUS437g/NrBjC/BgAAAIgkgk2shJZ5LpkipbjM1gIAAAAkGYJNrDC/BgAAAIgagk0sWJa0653g/uDpZmsBAAAAkhDBJhb2b5UaaqSUNGnAJNPVAAAAAEmHYBMLzK8BAAAAoopgEwvMrwEAAACiimATbTy/BgAAAIg6gk20Mb8GAAAAiDqCTbSFemsGnsn8GgAAACBKCDbRxvwaAAAAIOoINtHE/BoAAAAgJgg20bR/m9RQzfwaAAAAIMoINtEUen7NwDMlZ5rZWgAAAIAkRrCJJubXAAAAADFBsImWdvNrpputBQAAAEhyBJtoObA9OL/G4ZIGTDZdDQAAAJDUCDbREppfUzKF+TUAAABAlBFsooVlngEAAICYIdhEA8+vAQAAAGKKYBMNh3dL9VWS3cn8GgAAACAGCDbRULUxuO0/ivk1AAAAQAwQbKIhFGwKxpitAwAAAOglCDbRUB0KNqeZrQMAAADoJaIebH7605/KZrNp3rx50b5U/Aj12OQTbAAAAIBYiGqwWbVqlZ544gmNHz8+mpeJL76m4MM5JXpsAAAAgBiJWrCpr6/XnDlz9OSTTyonJydal4k/NVskKyCl50hZhaarAQAAAHqFqAWbuXPn6tJLL9XMmTNPep7X65XH42n3SmjVm4LbgrGSzWa2FgAAAKCXSInGlz733HNas2aNVq1adcpz58+frx/84AfRKMOMKhYOAAAAAGIt4j02e/bs0e23365nnnlGaWmnfobLvffeq9ra2vBrz549kS4ptsILB7DUMwAAABArEe+xWb16taqrq3XGGWeE2/x+v5YtW6ZHHnlEXq9XDocjfMzlcsnlckW6DHPCPTZjzdYBAAAA9CIRDzYXXnih1q9f367tpptu0qhRo3T33Xe3CzVJp75GaqiWZJPyR5muBgAAAOg1Ih5ssrKyNHZs+96KzMxM5eXlHdeedEIP5swtk1IzzdYCAAAA9CJRf0Bnr1LVuiIa82sAAACAmIrKqmjHWrJkSSwuYx7zawAAAAAj6LGJpNBQtAJ6bAAAAIBYIthESsAvVW8O7tNjAwAAAMQUwSZSDu6UWpqklHQpZ7DpagAAAIBehWATKVUbgtv80ZI9iZe0BgAAAOIQwSZSqltXRGN+DQAAABBzBJtIYUU0AAAAwBiCTaSEgg3PsAEAAABijmATCd566dDO4H7BaWZrAQAAAHohgk0k1GwJbvsUSJn9zNYCAAAA9EIEm0gIrYhGbw0AAABgBMEmEqpaV0Rjfg0AAABgBMEmElgRDQAAADCKYNNTliVVh4INPTYAAACACQSbnqrbJx05JNkcUr+RpqsBAAAAeiWCTU+F5tfkDZOcaWZrAQAAAHopgk1PsSIaAAAAYBzBpqeqW3tsmF8DAAAAGEOw6SlWRAMAAACMI9j0hN8n1ZQH93mGDQAAAGAMwaYn9m+TAj4pNUvqW2q6GgAAAKDXItj0RNv5NTab2VoAAACAXoxg0xOsiAYAAADEBYJNT4SeYcP8GgAAAMAogk1PsCIaAAAAEBcINt115LDk+SS4nz/aaCkAAABAb0ew6a7QwgHuEim9r9FSAAAAgN6OYNNdoWFozK8BAAAAjCPYdFd4fg0rogEAAACmEWy6K/wMG4INAAAAYBrBpjss6+hSzwQbAAAAwDiCTXcc3i0110l2p5Q3zHQ1AAAAQK9HsOmO0Pya/qMkh9NsLQAAAAAINt1SHVo4gBXRAAAAgHhAsOkOVkQDAAAA4grBpjtCCwfkE2wAAACAeECw6Spfk3Rge3CfHhsAAAAgLhBsump/uWT5pfQcKavQdDUAAAAARLDpuvD8mrGSzWa2FgAAAACSCDZdFwo2+ayIBgAAAMQLgk1XsSIaAAAAEHcINl1V3boiGsEGAAAAiBsEm65o2C/VV0mySf1Hma4GAAAAQCuCTVeEhqHlDJZcfYyWAgAAAOAogk1XML8GAAAAiEsEm66oJtgAAAAA8Yhg0xX02AAAAABxiWDTWQG/VL0luJ9PsAEAAADiCcGmsw7ulFqOSCnpUm6Z6WoAAAAAtEGw6azQ/Jr8UZLdYbYWAAAAAO0QbDqL+TUAAABA3CLYdFYo2DC/BgAAAIg7BJvOoscGAAAAiFsEm87w1kuHPg7uE2wAAACAuEOw6YyaLZIsqU+BlNnPdDUAAAAAjkGw6Yzw/JoxZusAAAAA0CGCTWcwvwYAAACIawSbzqjeFNwSbAAAAIC4RLA5FcuSqjYE9wk2AAAAQFwi2JxKXaV05JBkc0j9RpquBgAAAEAHCDanEppfkzdMcqaZrQUAAABAhwg2p1IdWjiAFdEAAACAeEWwORVWRAMAAADiXsSDzfz583XmmWcqKytL+fn5uvLKK1VeXh7py8ROVeuKaPkEGwAAACBeRTzYLF26VHPnztWKFSu0ePFi+Xw+ffrTn1ZDQ0OkLxV9fp9UsyW4T48NAAAAELdSIv2Fr7/+erv3f/jDH5Sfn6/Vq1frvPPOi/TlouvAdingk1KzpL6lpqsBAAAAcAIRDzbHqq2tlSTl5uZ2eNzr9crr9YbfezyeaJfUeVVtFg6w2czWAgAAAOCEorp4QCAQ0Lx58zR9+nSNHTu2w3Pmz58vt9sdfpWUlESzpK4JBZt8VkQDAAAA4pnNsiwrWl9+yy236LXXXtPbb7+tgQMHdnhORz02JSUlqq2tVXZ2drRK65zaT6S9a6TsAdLASWZrAQAAAHoZj8cjt9vdqWwQtaFot956q1599VUtW7bshKFGklwul1wuV7TK6Bn3wOALAAAAQFyLeLCxLEvf+ta39NJLL2nJkiUqKyuL9CUAAAAAoJ2IB5u5c+dqwYIFWrRokbKyslRZWSlJcrvdSk9Pj/TlAAAAACDyc2xsJ1g97KmnntKNN954ys93ZRwdAAAAgORldI5NFNciAAAAAIAORXW5ZwAAAACIBYINAAAAgIRHsAEAAACQ8Ag2AAAAABIewQYAAABAwiPYAAAAAEh4BBsAAAAACY9gAwAAACDhEWwAAAAAJDyCDQAAAICER7ABAAAAkPAINgAAAAASHsEGAAAAQMJLMV3AsSzLkiR5PB7DlQAAAAAwKZQJQhnhZOIu2NTV1UmSSkpKDFcCAAAAIB7U1dXJ7Xaf9Byb1Zn4E0OBQEAVFRXKysqSzWYzXY48Ho9KSkq0Z88eZWdnmy4HCYR7Bz3B/YOe4P5BT3D/oLuice9YlqW6ujoVFxfLbj/5LJq467Gx2+0aOHCg6TKOk52dzb/c6BbuHfQE9w96gvsHPcH9g+6K9L1zqp6aEBYPAAAAAJDwCDYAAAAAEh7B5hRcLpceeOABuVwu06UgwXDvoCe4f9AT3D/oCe4fdJfpeyfuFg8AAAAAgK6ixwYAAABAwiPYAAAAAEh4BBsAAAAACY9gAwAAACDhEWwAAAAAJDyCzUk8+uijGjx4sNLS0jR16lS99957pktCHFq2bJkuv/xyFRcXy2azaeHChe2OW5al+++/X0VFRUpPT9fMmTO1bds2M8UirsyfP19nnnmmsrKylJ+fryuvvFLl5eXtzmlqatLcuXOVl5enPn366POf/7yqqqoMVYx48thjj2n8+PHhJ3xPmzZNr732Wvg49w664qc//alsNpvmzZsXbuMewol8//vfl81ma/caNWpU+Lipe4dgcwJ//vOfdccdd+iBBx7QmjVrNGHCBF188cWqrq42XRriTENDgyZMmKBHH320w+M///nP9fDDD+vxxx/XypUrlZmZqYsvvlhNTU0xrhTxZunSpZo7d65WrFihxYsXy+fz6dOf/rQaGhrC53z729/WK6+8or/85S9aunSpKioq9LnPfc5g1YgXAwcO1E9/+lOtXr1a77//vj71qU9p9uzZ2rhxoyTuHXTeqlWr9MQTT2j8+PHt2rmHcDKnnXaa9u3bF369/fbb4WPG7h0LHZoyZYo1d+7c8Hu/328VFxdb8+fPN1gV4p0k66WXXgq/DwQCVmFhofXf//3f4bbDhw9bLpfLevbZZw1UiHhWXV1tSbKWLl1qWVbwXnE6ndZf/vKX8DmbN2+2JFnLly83VSbiWE5OjvX73/+eewedVldXZw0fPtxavHixdf7551u33367ZVn8/MHJPfDAA9aECRM6PGby3qHHpgPNzc1avXq1Zs6cGW6z2+2aOXOmli9fbrAyJJqdO3eqsrKy3b3kdrs1depU7iUcp7a2VpKUm5srSVq9erV8Pl+7+2fUqFEqLS3l/kE7fr9fzz33nBoaGjRt2jTuHXTa3Llzdemll7a7VyR+/uDUtm3bpuLiYg0ZMkRz5szR7t27JZm9d1Ki+u0Jav/+/fL7/SooKGjXXlBQoC1bthiqComosrJSkjq8l0LHAEkKBAKaN2+epk+frrFjx0oK3j+pqanq27dvu3O5fxCyfv16TZs2TU1NTerTp49eeukljRkzRmvXruXewSk999xzWrNmjVatWnXcMX7+4GSmTp2qP/zhDxo5cqT27dunH/zgBzr33HO1YcMGo/cOwQYA4sDcuXO1YcOGdmOUgVMZOXKk1q5dq9raWr3wwgu64YYbtHTpUtNlIQHs2bNHt99+uxYvXqy0tDTT5SDBzJo1K7w/fvx4TZ06VYMGDdLzzz+v9PR0Y3UxFK0D/fr1k8PhOG71hqqqKhUWFhqqCokodL9wL+Fkbr31Vr366qt68803NXDgwHB7YWGhmpubdfjw4Xbnc/8gJDU1VcOGDdOkSZM0f/58TZgwQb/5zW+4d3BKq1evVnV1tc444wylpKQoJSVFS5cu1cMPP6yUlBQVFBRwD6HT+vbtqxEjRmj79u1Gf/4QbDqQmpqqSZMm6Y033gi3BQIBvfHGG5o2bZrBypBoysrKVFhY2O5e8ng8WrlyJfcSZFmWbr31Vr300kv697//rbKysnbHJ02aJKfT2e7+KS8v1+7du7l/0KFAICCv18u9g1O68MILtX79eq1duzb8mjx5subMmRPe5x5CZ9XX1+ujjz5SUVGR0Z8/DEU7gTvuuEM33HCDJk+erClTpuihhx5SQ0ODbrrpJtOlIc7U19dr+/bt4fc7d+7U2rVrlZubq9LSUs2bN08//vGPNXz4cJWVlem+++5TcXGxrrzySnNFIy7MnTtXCxYs0KJFi5SVlRUee+x2u5Weni63260vf/nLuuOOO5Sbm6vs7Gx961vf0rRp03TWWWcZrh6m3XvvvZo1a5ZKS0tVV1enBQsWaMmSJfrHP/7BvYNTysrKCs/nC8nMzFReXl64nXsIJ/Ld735Xl19+uQYNGqSKigo98MADcjgcuu6668z+/InqmmsJ7re//a1VWlpqpaamWlOmTLFWrFhhuiTEoTfffNOSdNzrhhtusCwruOTzfffdZxUUFFgul8u68MILrfLycrNFIy50dN9Isp566qnwOUeOHLG++c1vWjk5OVZGRob12c9+1tq3b5+5ohE3br75ZmvQoEFWamqq1b9/f+vCCy+0/vnPf4aPc++gq9ou92xZ3EM4sWuuucYqKiqyUlNTrQEDBljXXHONtX379vBxU/eOzbIsK7rRCQAAAACiizk2AAAAABIewQYAAABAwiPYAAAAAEh4BBsAAAAACY9gAwAAACDhEWwAAAAAJDyCDQAAAICER7ABAAAAkPAINgAAAAASHsEGAAAAQMIj2AAAAABIeP8fYWL5luPXE/sAAAAASUVORK5CYII=", "text/plain": [ "
" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "V = np.zeros(nb_states)\n", "Q = np.zeros((nb_states, nb_actions))\n", "\n", "pi = np.array([a['search'], a['search']], dtype=int)\n", "\n", "V_high_history = []\n", "V_low_history = []\n", "\n", "for k in range(50):\n", " \n", " # Policy evaluation\n", " Q[s['high'], a['search']] = alpha * (r_search + gamma * V[s['high']]) + (1 - alpha) * (r_search + gamma*V[s['low']])\n", " Q[s['high'], a['wait']] = r_wait + gamma * V[s['high']]\n", "\n", " Q[s['low'], a['search']] = beta * (r_search + gamma * V[s['low']]) + (1 - beta) * (-3 + gamma*V[s['high']])\n", " Q[s['low'], a['wait']] = r_wait + gamma * V[s['low']]\n", " Q[s['low'], a['recharge']] = gamma * V[s['high']]\n", "\n", " V[s['high']] = Q[s['high'], :2].max()\n", " V[s['low']] = Q[s['low'], :].max()\n", "\n", " V_high_history.append(V[s['high']])\n", " V_low_history.append(V[s['low']])\n", " \n", " # Compute the greedy policy\n", " pi_old = pi.copy()\n", " pi[s['high']] = Q[s['high'], :2].argmax()\n", " pi[s['low']] = Q[s['low'], :].argmax()\n", " \n", "print('Greedy policy after iteration', k)\n", "print('pi(high)=', pi[s['high']])\n", "print('pi(low)=', pi[s['low']])\n", "print(\"V=\", V)\n", "print(\"Q=\", Q)\n", " \n", "plt.figure(figsize=(10, 6)) \n", "plt.plot(V_high_history, label=\"high\")\n", "plt.plot(V_low_history, label=\"low\")\n", "plt.legend()\n", "plt.show()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "**Q:** Change the value of the discount factor $\\gamma =0.3$ so that the agent becomes short-sighted: it only takes into account the immediate rewards, but forgets about the long-term. Does it change the strategy? Explain why." ] }, { "cell_type": "code", "execution_count": 9, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Greedy policy after iteration 49\n", "pi(high)= 0\n", "pi(low)= 1\n", "V= [7.25274725 2.85714286]\n", "Q= [[7.25274725 4.17582418 0. ]\n", " [0.71208791 2.85714286 2.17582418]]\n" ] }, { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAy0AAAH5CAYAAACMINEWAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjkuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8hTgPZAAAACXBIWXMAAA9hAAAPYQGoP6dpAAArOklEQVR4nO3df5RV9X3v/9eZn4AwIxAQkMGgQRQNmopaNIkk/qqxuSbttVmp+UajN/32lqQSv1lJvW00uW2KadquNG1uYtJ8TbsSYprcan58a4yagCvxR5Ascv1BEKyNKCBqZQYQBpjZ3z8GRkB+zAAze8M8Hmuddc6cs2fOm8mOa55r788+taIoigAAAFRUXdkDAAAA7I9oAQAAKk20AAAAlSZaAACAShMtAABApYkWAACg0kQLAABQaQ2D/Ybd3d1ZvXp1Ro0alVqtNthvDwAAVERRFNmwYUMmTZqUurp9H08Z9GhZvXp12traBvttAQCAilq1alUmT568z9cHPVpGjRqVpGewlpaWwX57AACgIjo6OtLW1tbbCPsy6NGy85SwlpYW0QIAABxw2YiF+AAAQKWJFgAAoNJECwAAUGmiBQAAqDTRAgAAVJpoAQAAKk20AAAAlSZaAACAShMtAABApYkWAACg0kQLAABQaaIFAACoNNECAABUmmgBAAAqTbQAAACVJloAAIBKEy0AAEClNZQ9AANnU+f2vLChMy+/sjVbt3dna1d3z/2Ox53bd//6NY+7utPdXaS7KNJdpOe++9XHxc7ndrxe7Lrdjq932vmwSNH79a7Pvfp4P/bzYrH/7xwQxeC/JQDAYfHf3nJifuv0CWWP0Wei5QizeWtXXtjQmRc2duaFDZ15cWPPbefjnvuteWFDZzZv6yp7XAAAKuiKN3WWPUK/iJYKK4oiK9ZtzN2Prc09y57PU+s2ZtPW/oXI8Mb6jDmmKc2NdWmqr0tzQ12adt7qdz6uT2N9ree1+l1fr09DfS21WlJXq6Vux31tl8d1tez4etfXe56rJanVeuao1ZKeZ159rufxjud23Ta7bNC73Wv/bXs+tbdtAAB4rRkTW8seoV9ES8V0dxdZ+uz63P342vzo8efz9IubXrNNc0Ndxo1qzutG9tzGjWrOuJFNvc/ten9Ms/+JAQA4svmLtgK2dXXnoX9/KXc/vjb3PPF8nu949XBdU31dzn/D2Fx62oScPXVMxo9qzsjmht4jFAAAcLQTLSXZvLUri558IXc/vjb3LXs+HVu29742srkhc6aPy6WnTcic6eMyalhjiZMCAEC5RMsg6uou8v1frs6/Pbom9694IVu2dfe+NvaYplw847hcetqEnPeGsWluqC9xUgAAqA7RMkiKosif/O//k28vebb3ucmjh+fS0ybk0tMm5KwTRqe+zilfAACwJ9EySL606N/z7SXPpq6W/Pc5J+Udb5yYGRNbrE0BAIADEC2D4IePrclnfvirJMlNvz0j15w/teSJAADgyFFX9gBHu0efbc+8by1Nkrx/9gmCBQAA+km0DKA17Ztz3T8tzpZt3XnryeNy02/PKHskAAA44oiWAbKpc3uu+9ojWbehM9PGj8w//P6b0lDv1w0AAP3lr+gB0NVd5Prbl+aJNR0Ze0xT/t9rzk6Lz1oBAICDIloGwGd++Kvcu+z5NDXU5cvvn5W2MSPKHgkAAI5YouUwu/3nz+TL9/97kuSz/3VmzjphdMkTAQDAkU20HEY/W/li/uzOx5Ik1184LVeceXzJEwEAwJFPtBwmK9dtzH//+pJs7y7yX86YlHkXTSt7JAAAOCr0K1pe//rXp1arveY2d+7cgZrviPDypq257p8Wp2PL9vzGlGPzV/91pk+6BwCAw6ShPxsvXrw4XV1dvV8/9thjufjii3PllVce9sGOFJ3bu/J/f31Jfv3SK5k8eni+/P5ZGdZYX/ZYAABw1OhXtIwbN263r2+55ZacdNJJueCCC/b5PZ2dnens7Oz9uqOjo58jVldRFPkf//pYfv70f2Zkc0O+evXZed3I5rLHAgCAo8pBr2nZunVrvv71r+faa6/d76lQ8+fPT2tra++tra3tYN+ycv7Xwqfyv3/xbOpqyT/8/psyfcKoskcCAICjzkFHy5133pn169fnmmuu2e92N954Y9rb23tvq1atOti3rJS7Hl2Tz969PEnyyf9yWuZMH1/yRAAAcHTq1+lhu/rqV7+ayy67LJMmTdrvds3NzWluPrpOmfo/z67PR/5laZLkmvNen/fPfn2p8wAAwNHsoKLl17/+de69997867/+6+Ge54jwiTsfy5Zt3ZkzfVz+7PJTyx4HAACOagd1ethtt92W8ePH5/LLLz/c81Telm1deWx1z8UEPv3uN6ah3kfdAADAQOr3X9zd3d257bbbcvXVV6eh4aDPLjtirXh+Y7q6i4we0ZhJrcPKHgcAAI56/Y6We++9N88880yuvfbagZin8p5Y054kmTGpxQdIAgDAIOj3oZJLLrkkRVEMxCxHhCd2nBo2Y2JLyZMAAMDQYEFGPz2xZke0TBItAAAwGERLP3R3F1m2ZkOSZMbE1pKnAQCAoUG09MOql1/Jxs7taWqoy4njjil7HAAAGBJESz/sXM8y/bhRaXSpYwAAGBT+8u6HZWsswgcAgMEmWvrBInwAABh8oqUfei93LFoAAGDQiJY+ennT1qxu35IkOWXCqJKnAQCAoUO09NHO9SwnjB2RUcMaS54GAACGDtHSR09YhA8AAKUQLX3Uu55FtAAAwKASLX3kymEAAFAO0dIHW7Z1ZeW6jUlECwAADDbR0gcr123M9u4io0c0ZkLLsLLHAQCAIUW09MGun89Sq9VKngYAAIYW0dIHO9eznDrBqWEAADDYREsf7HqkBQAAGFyi5QC6uwtXDgMAgBKJlgN49uXN2di5PU31dTlp3MiyxwEAgCFHtBzAE2vakyQnTxiZxnq/LgAAGGz+Cj+A3vUsE50aBgAAZRAtB9C7nkW0AABAKUTLAbx65bDWkicBAIChSbTsx8ubtmZ1+5YkySkTR5U8DQAADE2iZT+W7Tg1bMqYEWkZ1ljyNAAAMDSJlv2wngUAAMonWvbDh0oCAED5RMt+uNwxAACUT7TsQ+f2rqxctzGJIy0AAFAm0bIPK57fmO3dRY4d0ZiJrcPKHgcAAIYs0bIPuy7Cr9VqJU8DAABDl2jZB+tZAACgGkTLPrhyGAAAVINo2YuiKLJstWgBAIAqEC178ezLm7Ohc3ua6uty0riRZY8DAABDmmjZi8d3HGU5ecLINNb7FQEAQJn8Rb4XO9eznDrBqWEAAFA20bIXT1jPAgAAlSFa9mLZGpc7BgCAqhAte1j/ytY8t35zkuRUR1oAAKB0omUPO9eztI0ZnpZhjSVPAwAAiJY99K5ncWoYAABUgmjZwxO961laS54EAABIRMtruHIYAABUi2jZRef2rqxctzGJaAEAgKoQLbtY8fzGbO8u0jq8MZNah5U9DgAAENGym10/n6VWq5U8DQAAkIiW3fQuwndqGAAAVIZo2YXLHQMAQPWIlh2KonCkBQAAKki07PDsy5uzYcv2NNXX5aRxI8seBwAA2EG07LDzKMu040amqcGvBQAAqsJf5ztYzwIAANUkWnawngUAAKpJtOzgSAsAAFSTaEnS/sq2PLd+c5LkVEdaAACgUkRLXj01rG3M8LQMayx5GgAAYFeiJbusZ3FqGAAAVI5oyavrWU4VLQAAUDmiJY60AABAlQ35aNm6vTsr121I4nLHAABQRUM+Wlas25BtXUVahjXk+GOHlz0OAACwhyEfLb2fzzKpJbVareRpAACAPYmW3vUsrSVPAgAA7I1o2eVICwAAUD1DOlqKonDlMAAAqLghHS3Pvrw5G7ZsT2N9LW8YP7LscQAAgL0Y0tGybMdRlmnjR6WpYUj/KgAAoLKG9F/qvaeGWc8CAACV1e9oee655/K+970vY8eOzfDhw/PGN74xjzzyyEDMNuB6F+FbzwIAAJXV0J+NX3755Zx//vl529velrvuuivjxo3LihUrMnr06IGab0A50gIAANXXr2j5zGc+k7a2ttx22229z02dOnW/39PZ2ZnOzs7erzs6Ovo54sBo37wtz768OUlyqiMtAABQWf06Pex73/teZs2alSuvvDLjx4/Pm970pnzlK1/Z7/fMnz8/ra2tvbe2trZDGvhw2bkIf/Lo4Wkd3ljyNAAAwL7UiqIo+rrxsGHDkiQ33HBDrrzyyixevDjXX399vvSlL+Xqq6/e6/fs7UhLW1tb2tvb09JS3hGO59Zvzl2PrkldrZZr37z/o0UAAMDh19HRkdbW1gO2Qb+ipampKbNmzcoDDzzQ+9wf//EfZ/HixXnwwQcP62AAAMDRra9t0K/TwyZOnJgZM2bs9typp56aZ5555uCmBAAAOIB+Rcv555+f5cuX7/bck08+mRNOOOGwDgUAALBTv6LlIx/5SB566KH85V/+ZVauXJkFCxbky1/+cubOnTtQ8wEAAENcv6Ll7LPPzh133JFvfvObOf300/Pnf/7n+dznPperrrpqoOYDAACGuH4txD8cLMQHAACSAVqIDwAAMNhECwAAUGmiBQAAqDTRAgAAVJpoAQAAKk20AAAAlSZaAACAShMtAABApYkWAACg0kQLAABQaaIFAACoNNECAABUmmgBAAAqTbQAAACVJloAAIBKEy0AAECliRYAAKDSRAsAAFBpogUAAKg00QIAAFSaaAEAACpNtAAAAJUmWgAAgEoTLQAAQKWJFgAAoNJECwAAUGmiBQAAqDTRAgAAVJpoAQAAKk20AAAAlSZaAACAShMtAABApYkWAACg0kQLAABQaaIFAACoNNECAABUmmgBAAAqTbQAAACVJloAAIBKEy0AAECliRYAAKDSRAsAAFBpogUAAKg00QIAAFSaaAEAACpNtAAAAJUmWgAAgEoTLQAAQKWJFgAAoNJECwAAUGmiBQAAqDTRAgAAVJpoAQAAKk20AAAAlSZaAACAShMtAABApYkWAACg0kQLAABQaaIFAACoNNECAABUmmgBAAAqTbQAAACVJloAAIBKEy0AAECliRYAAKDSRAsAAFBpogUAAKg00QIAAFSaaAEAACqtX9HyyU9+MrVabbfbKaecMlCzAQAApKG/33Daaafl3nvvffUHNPT7RwAAAPRZv4ujoaEhEyZM6PP2nZ2d6ezs7P26o6Ojv28JAAAMYf1e07JixYpMmjQpJ554Yq666qo888wz+91+/vz5aW1t7b21tbUd9LAAAMDQUyuKoujrxnfddVc2btyY6dOnZ82aNfnUpz6V5557Lo899lhGjRq11+/Z25GWtra2tLe3p6Wl5dD/BQAAwBGpo6Mjra2tB2yDfkXLntavX58TTjghf/u3f5vrrrvusA4GAAAc3fraBod0yeNjjz02J598clauXHkoPwYAAGCfDilaNm7cmKeeeioTJ048XPMAAADspl/R8tGPfjSLFi3Kf/zHf+SBBx7Iu9/97tTX1+e9733vQM0HAAAMcf265PGzzz6b9773vXnppZcybty4vPnNb85DDz2UcePGDdR8AADAENevaLn99tsHag4AAIC9OqQ1LQAAAANNtAAAAJUmWgAAgEoTLQAAQKWJFgAAoNJECwAAUGmiBQAAqDTRAgAAVJpoAQAAKk20AAAAlSZaAACAShMtAABApYkWAACg0kQLAABQaaIFAACoNNECAABUmmgBAAAqTbQAAACVJloAAIBKEy0AAECliRYAAKDSRAsAAFBpogUAAKg00QIAAFSaaAEAACpNtAAAAJUmWgAAgEoTLQAAQKWJFgAAoNJECwAAUGmiBQAAqDTRAgAAVJpoAQAAKk20AAAAlSZaAACAShMtAABApYkWAACg0kQLAABQaaIFAACoNNECAABUmmgBAAAqTbQAAACVJloAAIBKEy0AAECliRYAAKDSRAsAAFBpogUAAKg00QIAAFSaaAEAACpNtAAAAJUmWgAAgEoTLQAAQKWJFgAAoNJECwAAUGmiBQAAqDTRAgAAVJpoAQAAKk20AAAAlSZaAACAShMtAABApYkWAACg0kQLAABQaaIFAACoNNECAABUmmgBAAAqTbQAAACVJloAAIBKEy0AAECliRYAAKDSDilabrnlltRqtcybN+8wjQMAALC7g46WxYsX59Zbb83MmTMP5zwAAAC7Oaho2bhxY6666qp85StfyejRo/e7bWdnZzo6Ona7AQAA9NVBRcvcuXNz+eWX56KLLjrgtvPnz09ra2vvra2t7WDeEgAAGKL6HS233357fvGLX2T+/Pl92v7GG29Me3t7723VqlX9HhIAABi6Gvqz8apVq3L99dfnnnvuybBhw/r0Pc3NzWlubj6o4QAAAGpFURR93fjOO+/Mu9/97tTX1/c+19XVlVqtlrq6unR2du722t50dHSktbU17e3taWlpOfjJAQCAI1pf26BfR1ouvPDCPProo7s994EPfCCnnHJKPv7xjx8wWAAAAPqrX9EyatSonH766bs9d8wxx2Ts2LGveR4AAOBwOKQPlwQAABho/TrSsjcLFy48DGMAAADsnSMtAABApYkWAACg0kQLAABQaaIFAACoNNECAABUmmgBAAAqTbQAAACVJloAAIBKEy0AAECliRYAAKDSRAsAAFBpogUAAKg00QIAAFSaaAEAACpNtAAAAJUmWgAAgEoTLQAAQKWJFgAAoNJECwAAUGmiBQAAqDTRAgAAVJpoAQAAKk20AAAAlSZaAACAShMtAABApYkWAACg0kQLAABQaaIFAACoNNECAABUmmgBAAAqTbQAAACVJloAAIBKEy0AAECliRYAAKDSRAsAAFBpogUAAKg00QIAAFSaaAEAACpNtAAAAJUmWgAAgEoTLQAAQKWJFgAAoNJECwAAUGmiBQAAqDTRAgAAVJpoAQAAKk20AAAAlSZaAACAShMtAABApYkWAACg0kQLAABQaaIFAACoNNECAABUmmgBAAAqTbQAAACVJloAAIBKEy0AAECliRYAAKDSRAsAAFBpogUAAKg00QIAAFSaaAEAACpNtAAAAJUmWgAAgEoTLQAAQKWJFgAAoNJECwAAUGmiBQAAqDTRAgAAVFq/ouWLX/xiZs6cmZaWlrS0tGT27Nm56667Bmo2AACA/kXL5MmTc8stt2TJkiV55JFH8va3vz1XXHFFHn/88YGaDwAAGOJqRVEUh/IDxowZk89+9rO57rrr9vp6Z2dnOjs7e7/u6OhIW1tb2tvb09LScihvDQAAHME6OjrS2tp6wDY46DUtXV1duf3227Np06bMnj17n9vNnz8/ra2tvbe2traDfUsAAGAI6veRlkcffTSzZ8/Oli1bMnLkyCxYsCDveMc79rm9Iy0AAMDe9PVIS0N/f/D06dOzdOnStLe35zvf+U6uvvrqLFq0KDNmzNjr9s3NzWlubu7v2wAAACQ5DGtaLrroopx00km59dZb+7R9X2sKAAA4ug34mpaduru7dzv9CwAA4HDq1+lhN954Yy677LJMmTIlGzZsyIIFC7Jw4cLcfffdAzUfAAAwxPUrWtatW5f3v//9WbNmTVpbWzNz5szcfffdufjiiwdqPgAAYIjrV7R89atfHag5AAAA9uqQ17QAAAAMJNECAABUWr8/p4WjVFEkXduSrs5k+9ae+66tSXdXz2tF945b1y6Pd96KvTy34/meH777453v1/t4z+dfM9ze5z1sDufPAgA4AoyfkYw+oewp+ky0HA06Nyab1iUbX9hxvy7Z9MKO+3XJKy/viJEdIbLXe5etBgAYMi7/m+Ts/1b2FH0mWo4Er/xnsvLe5MUVe4+Tba8c/vesa0jqm3rua7WkVrfHrX6Xx3t7vZaktuM+PY+TvXy9n9d2VdvLc3tut9dtAAB4jWPGlT1Bv4iWKiqK5MUnk+V3JU/+MFn1cM/pVvvTOKJn5xs5PjlmfDJy3I778cmIMUnD8KShqSdE6pt3PG5OGpp7ntvzvq5+cP6tAABwAKKlKrq2Jb9+oCdSlt+VvPz07q8fd3oy+exk5HG7B8kx43qeax5ZztwAADDAREuZXvnPZMU9yZN3JSvvSzo7Xn2tvil5/VuS6ZclJ1+aHDulvDkBAKBEomWwvbgyWf7/Jct/mKx6aPfTvka8ridQTv6t5KS3Jc2jypsTAAAqQrQMpvv/Ovnxn+/+3PjTkum/lZx8WXL8b1hLAgAAexAtg2XRZ5Of/EXP4xPnJNMv7zmqcgRdHxsAAMogWgbDrsFy4c3JW24odx4AADiC1JU9wFFPsAAAwCERLQPpfsECAACHSrQMlPs/m/xYsAAAwKESLQNBsAAAwGEjWg633YLlJsECAACHSLQcTq8Jlv+n3HkAAOAoIFoOl/v/WrAAAMAAEC2Hw66fdC9YAADgsBIth0qwAADAgBIth2LXYHn7JwQLAAAMANFysPYMlrd+tNx5AADgKCVaDsZDXxQsAAAwSERLf3VtTxbe0vP4bX8mWAAAYICJlv5a9VCyZX0yYqwPjgQAgEEgWvrryR/23E+7JKmrL3cWAAAYAkRLfy3fES0nX1ruHAAAMESIlv546ankpRVJXWNy0oVlTwMAAEOCaOmP5Xf13L/+/GRYS7mzAADAECFa+mPnepaTf6vcOQAAYAgRLX21eX3yzIM9j0ULAAAMGtHSVyvvTbq3J+NOScZMLXsaAAAYMkRLXzk1DAAASiFa+qJre7Linp7HogUAAAaVaOmLVQ8nW9Ynw8ckbeeUPQ0AAAwpoqUvntxxqeNplyR19eXOAgAAQ4xo6YvlO9azTHdqGAAADDbRciAvPZW8tCKpa0hOenvZ0wAAwJDTUPYAlbfzqmEnnJ8May13FgAAStPV1ZVt27aVPcYRpbGxMfX1h768QrQcyPId61mmX1buHAAAlKIoiqxduzbr168ve5Qj0rHHHpsJEyakVqsd9M8QLfuzeX3yzIM9j0++tNRRAAAox85gGT9+fEaMGHFIf3wPJUVR5JVXXsm6deuSJBMnTjzonyVa9uep+5Lu7cnrpidjTix7GgAABllXV1dvsIwdO7bscY44w4cPT5KsW7cu48ePP+hTxSzE3x9XDQMAGNJ2rmEZMWJEyZMcuXb+7g5lPZBo2Zeu7cmKH/U8Ptl6FgCAocwpYQfvcPzuRMu+PPvzZMv6ZPjoZPLZZU8DAABDlmjZl51XDZt2SVJv6Q8AAJRFtOzLzs9nOdl6FgAAjjxz5szJvHnz9vl6rVbLnXfe2eeft3DhwtRqtVIu/ewQwt689FTy4pNJXUPyhgvLngYAAA67NWvWZPTo0WWP0SeiZW+evLvn/oTzkmGt5c4CAAADYMKECWWP0GdOD9ubJ3esZ3HVMAAA9lAURV7Zur2UW1EU/Zq1u7s7H/vYxzJmzJhMmDAhn/zkJ3tf2/P0sAceeCBnnnlmhg0bllmzZuXOO+9MrVbL0qVLd/uZS5YsyaxZszJixIicd955Wb58+SH8NvvGkZY9bWlPfv1Az2OfzwIAwB42b+vKjJvuLuW9n/ifl2ZEU9//hP+nf/qn3HDDDXn44Yfz4IMP5pprrsn555+fiy++eLftOjo68s53vjPveMc7smDBgvz617/e53qYP/3TP83f/M3fZNy4cfnDP/zDXHvttfnZz352KP+sAxIte1p5b9K9PXndycmYE8ueBgAADtrMmTNz8803J0mmTZuWf/iHf8h99933mmhZsGBBarVavvKVr2TYsGGZMWNGnnvuuXzwgx98zc/89Kc/nQsuuCBJ8id/8ie5/PLLs2XLlgwbNmzA/h2iZU8717O4ahgAAHsxvLE+T/zPS0t77/6YOXPmbl9PnDgx69ate812y5cvz8yZM3cLj3POOeeAP3PixIlJknXr1mXKlCn9mq0/RMuuurYnK37U83i69SwAALxWrVbr1ylaZWpsbNzt61qtlu7u7sP2M3d+2v2h/swDsRB/V8/+PNn8cjLs2GTy3ssSAACONtOnT8+jjz6azs7O3ucWL15c4kS7Ey27Wr7jqmHTLknqj4x6BgCAQ/X7v//76e7uzh/8wR9k2bJlufvuu/PXf/3XSV49mlIm0bKrnetZXDUMAIAhpKWlJd///vezdOnSnHnmmfnTP/3T3HTTTUkyoAvs+8rhhJ3+89+TF5cndQ3JSReWPQ0AAByShQsXvua5XT+XZc/PfDnvvPPyy1/+svfrb3zjG2lsbOxdYD9nzpzXfM+ZZ57Z78+OORiiZaflP+y5nzI7GX5sqaMAAMBg++d//ueceOKJOf744/PLX/4yH//4x/N7v/d7GT58eNmjiZZeT+6IFlcNAwBgCFq7dm1uuummrF27NhMnTsyVV16ZT3/602WPlUS09NjSnvx6x6d4+nwWAACGoI997GP52Mc+VvYYe2UhfpKsvC/p3p687uRk7EllTwMAAOxCtCSvnhp2cjmfbAoAAOybaOnuSlb8qOfxydazAABA1YiWVT9PNr+cDDs2aTu37GkAAIA9iJYn7+q5n3ZxUu+6BAAAUDWiZefns7hqGAAAR5E5c+Zk3rx5ZY9xWPQrWubPn5+zzz47o0aNyvjx4/Oud70ry5cvH6jZBt5//nvy4vKkriF5w0VlTwMAAOxFv6Jl0aJFmTt3bh566KHcc8892bZtWy655JJs2rRpoOYbWE/e3XM/ZXYy/NhSRwEAAPauX9Hywx/+MNdcc01OO+20nHHGGfna176WZ555JkuWLBmo+QbW8h3rWZwaBgDAUezll1/O+9///owePTojRozIZZddlhUrViRJiqLIuHHj8p3vfKd3+zPPPDMTJ07s/fqnP/1pmpub88orrwz67Mkhrmlpb29PkowZM2af23R2dqajo2O3WyVsaU9+/bOex9Nd6hgAgD4qimTrpnJuRXFQI19zzTV55JFH8r3vfS8PPvhgiqLIO97xjmzbti21Wi1vfetbs3DhwiQ9gbNs2bJs3rw5v/rVr5L0nHF19tlnZ8SIEYfrt9gvB325rO7u7sybNy/nn39+Tj/99H1uN3/+/HzqU5862LcZOE/9OOnenoydlow9qexpAAA4Umx7JfnLSeW89/9YnTQd069vWbFiRb73ve/lZz/7Wc4777wkyTe+8Y20tbXlzjvvzJVXXpk5c+bk1ltvTZLcf//9edOb3pQJEyZk4cKFOeWUU7Jw4cJccMEFh/2f01cHfaRl7ty5eeyxx3L77bfvd7sbb7wx7e3tvbdVq1Yd7FseXtMvT/6vO5OLKxhUAABwmCxbtiwNDQ0599xXP5Nw7NixmT59epYtW5YkueCCC/LEE0/khRdeyKJFizJnzpzMmTMnCxcuzLZt2/LAAw9kzpw5Jf0LDvJIy4c+9KH84Ac/yP3335/Jkyfvd9vm5uY0Nzcf1HADqqEpOeltZU8BAMCRpnFEzxGPst57ALzxjW/MmDFjsmjRoixatCif/vSnM2HChHzmM5/J4sWLs23btt6jNGXoV7QURZEPf/jDueOOO7Jw4cJMnTp1oOYCAIBqqtX6fYpWmU499dRs3749Dz/8cG94vPTSS1m+fHlmzJiRJKnVannLW96S7373u3n88cfz5je/OSNGjEhnZ2duvfXWzJo1K8ccU96/uV+nh82dOzdf//rXs2DBgowaNSpr167N2rVrs3nz5oGaDwAAOATTpk3LFVdckQ9+8IP56U9/ml/+8pd53/vel+OPPz5XXHFF73Zz5szJN7/5zZx55pkZOXJk6urq8ta3vjXf+MY3Sl3PkvQzWr74xS+mvb09c+bMycSJE3tv3/rWtwZqPgAA4BDddtttOeuss/Lbv/3bmT17doqiyL/927+lsbGxd5sLLrggXV1du61dmTNnzmueK0OtKA7yumkHqaOjI62trWlvb09LS8tgvjUAAPTLli1b8vTTT2fq1KkZNmxY2eMckfb3O+xrGxzS57QAAAAMNNECAABUmmgBAAAqTbQAAACVJloAAIBKEy0AAHAA3d3dZY9wxDocv7uGwzAHAAAclZqamlJXV5fVq1dn3LhxaWpqSq1WK3usI0JRFNm6dWteeOGF1NXVpamp6aB/lmgBAIB9qKury9SpU7NmzZqsXr267HGOSCNGjMiUKVNSV3fwJ3mJFgAA2I+mpqZMmTIl27dvT1dXV9njHFHq6+vT0NBwyEenRAsAABxArVZLY2NjGhsbyx5lSLIQHwAAqDTRAgAAVJpoAQAAKm3Q17QURZEk6ejoGOy3BgAAKmRnE+xshH0Z9GjZsGFDkqStrW2w3xoAAKigDRs2pLW1dZ+v14oDZc1h1t3dndWrV2fUqFGlfzBPR0dH2trasmrVqrS0tJQ6C0ce+w8Hy77DobD/cCjsPxyKgdh/iqLIhg0bMmnSpP1+jsugH2mpq6vL5MmTB/tt96ulpcX/cTlo9h8Oln2HQ2H/4VDYfzgUh3v/2d8Rlp0sxAcAACpNtAAAAJU2pKOlubk5N998c5qbm8sehSOQ/YeDZd/hUNh/OBT2Hw5FmfvPoC/EBwAA6I8hfaQFAACoPtECAABUmmgBAAAqTbQAAACVJloAAIBKG7LR8oUvfCGvf/3rM2zYsJx77rn5+c9/XvZIVND999+fd77znZk0aVJqtVruvPPO3V4viiI33XRTJk6cmOHDh+eiiy7KihUryhmWypk/f37OPvvsjBo1KuPHj8+73vWuLF++fLdttmzZkrlz52bs2LEZOXJkfvd3fzfPP/98SRNTFV/84hczc+bM3k+dnj17du66667e1+039Mctt9ySWq2WefPm9T5nH2JfPvnJT6ZWq+12O+WUU3pfL2vfGZLR8q1vfSs33HBDbr755vziF7/IGWeckUsvvTTr1q0rezQqZtOmTTnjjDPyhS98Ya+v/9Vf/VU+//nP50tf+lIefvjhHHPMMbn00kuzZcuWQZ6UKlq0aFHmzp2bhx56KPfcc0+2bduWSy65JJs2berd5iMf+Ui+//3v59vf/nYWLVqU1atX53d+53dKnJoqmDx5cm655ZYsWbIkjzzySN7+9rfniiuuyOOPP57EfkPfLV68OLfeemtmzpy52/P2IfbntNNOy5o1a3pvP/3pT3tfK23fKYagc845p5g7d27v111dXcWkSZOK+fPnlzgVVZekuOOOO3q/7u7uLiZMmFB89rOf7X1u/fr1RXNzc/HNb36zhAmpunXr1hVJikWLFhVF0bO/NDY2Ft/+9rd7t1m2bFmRpHjwwQfLGpOKGj16dPGP//iP9hv6bMOGDcW0adOKe+65p7jggguK66+/vigK/+1h/26++ebijDPO2OtrZe47Q+5Iy9atW7NkyZJcdNFFvc/V1dXloosuyoMPPljiZBxpnn766axdu3a3fam1tTXnnnuufYm9am9vT5KMGTMmSbJkyZJs27Ztt33olFNOyZQpU+xD9Orq6srtt9+eTZs2Zfbs2fYb+mzu3Lm5/PLLd9tXEv/t4cBWrFiRSZMm5cQTT8xVV12VZ555Jkm5+07DgP70CnrxxRfT1dWV4447brfnjzvuuPzqV78qaSqORGvXrk2Sve5LO1+Dnbq7uzNv3rycf/75Of3005P07ENNTU059thjd9vWPkSSPProo5k9e3a2bNmSkSNH5o477siMGTOydOlS+w0HdPvtt+cXv/hFFi9e/JrX/LeH/Tn33HPzta99LdOnT8+aNWvyqU99Km95y1vy2GOPlbrvDLloASjD3Llz89hjj+12XjDsz/Tp07N06dK0t7fnO9/5Tq6++uosWrSo7LE4AqxatSrXX3997rnnngwbNqzscTjCXHbZZb2PZ86cmXPPPTcnnHBC/uVf/iXDhw8vba4hd3rY6173utTX17/mKgfPP/98JkyYUNJUHIl27i/2JQ7kQx/6UH7wgx/kJz/5SSZPntz7/IQJE7J169asX79+t+3tQyRJU1NT3vCGN+Sss87K/Pnzc8YZZ+Tv/u7v7Dcc0JIlS7Ju3br8xm/8RhoaGtLQ0JBFixbl85//fBoaGnLcccfZh+izY489NieffHJWrlxZ6n9/hly0NDU15ayzzsp9993X+1x3d3fuu+++zJ49u8TJONJMnTo1EyZM2G1f6ujoyMMPP2xfIknPJbE/9KEP5Y477siPf/zjTJ06dbfXzzrrrDQ2Nu62Dy1fvjzPPPOMfYjX6O7uTmdnp/2GA7rwwgvz6KOPZunSpb23WbNm5aqrrup9bB+irzZu3JinnnoqEydOLPW/P0Py9LAbbrghV199dWbNmpVzzjknn/vc57Jp06Z84AMfKHs0Kmbjxo1ZuXJl79dPP/10li5dmjFjxmTKlCmZN29e/uIv/iLTpk3L1KlT84lPfCKTJk3Ku971rvKGpjLmzp2bBQsW5Lvf/W5GjRrVe75va2trhg8fntbW1lx33XW54YYbMmbMmLS0tOTDH/5wZs+end/8zd8seXrKdOONN+ayyy7LlClTsmHDhixYsCALFy7M3Xffbb/hgEaNGtW7dm6nY445JmPHju193j7Evnz0ox/NO9/5zpxwwglZvXp1br755tTX1+e9731vuf/9GdBrk1XY3//93xdTpkwpmpqainPOOad46KGHyh6JCvrJT35SJHnN7eqrry6Koueyx5/4xCeK4447rmhubi4uvPDCYvny5eUOTWXsbd9JUtx2222922zevLn4oz/6o2L06NHFiBEjine/+93FmjVryhuaSrj22muLE044oWhqairGjRtXXHjhhcWPfvSj3tftN/TXrpc8Lgr7EPv2nve8p5g4cWLR1NRUHH/88cV73vOeYuXKlb2vl7Xv1IqiKAY2iwAAAA7ekFvTAgAAHFlECwAAUGmiBQAAqDTRAgAAVJpoAQAAKk20AAAAlSZaAACAShMtAABApYkWAACg0kQLAABQaaIFAACotP8fIGCAH5FypFoAAAAASUVORK5CYII=", "text/plain": [ "
" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "# Transition probabilities\n", "alpha = 0.3\n", "beta = 0.2\n", "\n", "# Discount parameter\n", "gamma = 0.3\n", "\n", "# Expected rewards\n", "r_search = 6.0\n", "r_wait = 2.0\n", "\n", "V = np.zeros(nb_states)\n", "Q = np.zeros((nb_states, nb_actions))\n", "\n", "pi = np.array([a['search'], a['search']], dtype=int)\n", "\n", "V_high_history = []\n", "V_low_history = []\n", "\n", "for k in range(50):\n", " \n", " # Policy evaluation\n", " Q[s['high'], a['search']] = alpha * (r_search + gamma * V[s['high']]) + (1 - alpha) * (r_search + gamma*V[s['low']])\n", " Q[s['high'], a['wait']] = r_wait + gamma * V[s['high']]\n", "\n", " Q[s['low'], a['search']] = beta * (r_search + gamma * V[s['low']]) + (1 - beta) * (-3 + gamma*V[s['high']])\n", " Q[s['low'], a['wait']] = r_wait + gamma * V[s['low']]\n", " Q[s['low'], a['recharge']] = gamma * V[s['high']]\n", "\n", " V[s['high']] = Q[s['high'], :2].max()\n", " V[s['low']] = Q[s['low'], :].max()\n", "\n", " V_high_history.append(V[s['high']])\n", " V_low_history.append(V[s['low']])\n", " \n", " # Compute the greedy policy\n", " pi_old = pi.copy()\n", " pi[s['high']] = Q[s['high'], :2].argmax()\n", " pi[s['low']] = Q[s['low'], :].argmax()\n", " \n", "print('Greedy policy after iteration', k)\n", "print('pi(high)=', pi[s['high']])\n", "print('pi(low)=', pi[s['low']])\n", "print(\"V=\", V)\n", "print(\"Q=\", Q)\n", " \n", "plt.figure(figsize=(10, 6)) \n", "plt.plot(V_high_history, label=\"high\")\n", "plt.plot(V_low_history, label=\"low\")\n", "plt.legend()\n", "plt.show()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "**A:** The agent now decides to wait in the low state (r=2) instead of recharging (r=0) and then be in the high state (r=6). The agent is so greedy that it cannot stand not getting reward for one step, although he will collect much more reard later. " ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "**Q:** Change $\\gamma$ to 0.99 (far-sighted agent). What does it change and why? " ] }, { "cell_type": "code", "execution_count": 10, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Greedy policy after iteration 49\n", "pi(high)= 0\n", "pi(low)= 2\n", "V= [141.37219244 137.82818773]\n", "Q= [[141.37219244 139.82818773 0. ]\n", " [135.92647479 136.31962304 137.82818773]]\n" ] }, { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAz8AAAH5CAYAAACve4DDAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjkuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8hTgPZAAAACXBIWXMAAA9hAAAPYQGoP6dpAAB9WElEQVR4nOzdd3yUVdrG8d9MeiCFBNIgQKih994lSC+CbZe1rxVQBKSooNhQbIhd11fdVdcuICJI7713Qq9JgJDeZ573jwejrO4yk8yQdn0/H/44JzOHezRCLs9z7mMxDMNARERERESknLOWdAEiIiIiIiLXgsKPiIiIiIhUCAo/IiIiIiJSISj8iIiIiIhIhaDwIyIiIiIiFYLCj4iIiIiIVAgKPyIiIiIiUiF4lnQBRWG32zl79iwBAQFYLJaSLkdEREREREqIYRikp6cTFRWF1fq/93bKZPg5e/Ys0dHRJV2GiIiIiIiUEqdOnaJGjRr/8zVlMvwEBAQA5gcMDAws4WpERERERKSkpKWlER0dXZgR/pcyGX5+fdQtMDBQ4UdERERERBw6DqOGByIiIiIiUiEo/IiIiIiISIWg8CMiIiIiIhVCmTzz4yibzUZ+fn5Jl1GmeHt7X7VFoIiIiIhIWVQuw49hGCQkJJCSklLSpZQ5VquVmJgYvL29S7oUERERERGXKpfh59fgExYWhr+/vy5CddCvl8eeO3eOmjVr6p+biIiIiJQr5S782Gy2wuATGhpa0uWUOdWqVePs2bMUFBTg5eVV0uWIiIiIiLhMuTvc8esZH39//xKupGz69XE3m81WwpWIiIiIiLhWuQs/v9IjW0Wjf24iIiIiUl6V2/AjIiIiIiLyewo/pUjPnj0ZO3bsf/26xWJhzpw5Dq+3YsUKLBaLut6JiIiIiFAOGx6UZ+fOnaNKlSolXYaIiIiISJmk8FOGRERElHQJIiIiIiJllh57K2XsdjsTJ04kJCSEiIgInn766cKv/edjb+vWraNly5b4+vrStm1b5syZg8ViYceOHVesuXXrVtq2bYu/vz+dO3fm4MGD1+bDiIiIiIiUIhVi58cwDLLzr33rZj8vD6e7p3366aeMGzeOjRs3sn79eu688066dOlCnz59rnhdWloagwcPZsCAAXzxxRecOHHiv54XeuKJJ3j11VepVq0aDzzwAHfffTdr164t6scSERERESmTKkT4yc630Xjaomv+++57pi/+3s79I27evDlPPfUUAPXr1+ett95i6dKlfwg/X3zxBRaLhQ8//BBfX18aN27MmTNnuPfee/+w5vPPP0+PHj0AmDx5MgMHDiQnJwdfX98ifjIRERERkbJHj72VMs2bN79iHBkZSVJS0h9ed/DgQZo3b35FgGnfvv1V14yMjAT40zVFRERERBxVYLOXdAlOqxA7P35eHux7pm+J/L7O8vLyumJssViw24v3jfX7NX99DK+4a4qIiIhIxWMYBuuPXuQfq4/h7+3BW39tXdIlOaVChB+LxeL042elXcOGDfnss8/Izc3Fx8cHgM2bN5dwVSIiIiJSHuXb7CzYfY4PVx9lz5k0ALw8LFzMyCW0sk8JV+c4px97W7VqFYMHDyYqKuqql24+8MADWCwWZs2adcV8cnIyI0eOJDAwkODgYO655x4yMjKcLaVC++tf/4rdbue+++5j//79LFq0iFdeeQXA6SYLIiIiIiJ/Jj0nnw9XHaXHzOU88uUO9pxJw8fTyt861uSXR3uUqeADRdj5yczMpEWLFtx9990MHz78v77uhx9+YMOGDURFRf3hayNHjuTcuXMsXryY/Px87rrrLu677z6++OILZ8upsAIDA/nxxx958MEHadmyJc2aNWPatGn89a9/VSMDERERESmWsynZfLz2GF9uOkV6bgEAVSt7c3un2vytYy1CKnmXcIVF43T46d+/P/379/+frzlz5gxjxoxh0aJFDBw48Iqv7d+/n4ULF7J582batm0LwJtvvsmAAQN45ZVX/jQs5ebmkpubWzhOS0tztuwyYcWKFX+Y+/3OmmEYV3ytc+fO7Ny5s3D8+eef4+XlRc2aNQHo2bPnH97TsmXLP8yJiIiIiADsOZPKh6uP8tOucxTYzZ8Z64VV5u9dYxjWqjq+RTjTXpq4/CCM3W7ntttu47HHHqNJkyZ/+Pr69esJDg4uDD4AcXFxWK1WNm7cyA033PCH98yYMYPp06e7utQy75///Cd16tShevXq7Ny5k0mTJnHzzTfj5+dX0qWJiIiISBlhtxusPHSeD1YdZf3Ri4XzneqEcm/3GHo2CMNqLR/HKlwefl566SU8PT15+OGH//TrCQkJhIWFXVmEpychISEkJCT86XumTJnCuHHjCsdpaWlER0e7rugyKiEhgWnTppGQkEBkZCQ33XQTzz//fEmXJSIiIiJlQE6+jbk7zvDh6mMcTjLP33tYLQxqHsm93erQtHpQCVfoei4NP1u3buWNN95g27ZtLj107+PjU9jRTH4zceJEJk6cWNJliIiIiEgZkpKVx2cbTvDJuhNcyDCPllT28eQv7aO5s0sM1YPL71NELg0/q1evJikpqfDMCYDNZmP8+PHMmjWL48ePExER8YcLNgsKCkhOTiYiIsKV5YiIiIiIyGWnkrP4aM0xvtp8iux8GwCRQb7c3SWGW9pHE+jrdZUVyj6Xhp/bbruNuLi4K+b69u3Lbbfdxl133QVAp06dSElJYevWrbRp0waAZcuWYbfb6dChgyvLERERERGp8HadTuGDVUdZsPscl3sY0CgykPu712Fg80i8PJy+/abMcjr8ZGRkcPjw4cLxsWPH2LFjByEhIdSsWZPQ0NArXu/l5UVERAQNGzYEoFGjRvTr1497772X9957j/z8fEaPHs2tt976p53eRERERETEOXa7wYpDSXyw6igbjiYXznerX5X7uteha72qxTumkpUM6QkQ3tgF1V47ToefLVu20KtXr8Lxr40I7rjjDj755BOH1vj8888ZPXo0vXv3xmq1MmLECGbPnu1sKSIiIiIi8ju5BTbm7jjLh6uOEn+5iYGn1cKQFlH8vVsdGkcFFu83SD4GG96B7Z9BaD24fxW48Ky/uzkdfv7s7pj/5fjx43+YCwkJ0YWmIiIiIiIukpqVz+ebTvDJ2uMkpV/ZxOCuLjFEFbeJwanNsG42HJgPht2cM+yQdREqVS1m9deOy1tdi4iIiIjItXEmJZuPVh/jq80nycwzmxhEBPpyV5fa/KVDzeI1MbDb4OACWPcmnNr423zd3tB5DNTpWaZ2fUDhp1Tp2bMnLVu2ZNasWSVdioiIiIiUYvvPpfHBqqPM23kW2+UuBrERAdzbrQ6DW0Th7VmMJgZ5WbDjc1j/Nlw6Zs5ZvaD5LdBpVJk75/N7Cj8iIiIiImWAYRisP3qR91ceZeWh84XzneuGcl/3OvRoUK14TQzSE2HTB7DlI8i+ZM75BkO7e6D9fRBQ9q+lUfgRERERESnFCmx2Fu5N4P2VR9l9JhUAqwX6N4vk/u51aF4juHi/QdIBWP8W7PoKbHnmXHAt6DQaWo0E70rFW78UqThNvcuYS5cucfvtt1OlShX8/f3p378/8fHxgJn6q1Wrxrffflv4+pYtWxIZGVk4XrNmDT4+PmRlZV3z2kVERESk+LLzbPxr/XGue3Ulo7/Yzu4zqfh6Wbm9Uy1WTOjF239tXfTgYxhwbBV8fhO80wG2/8sMPjXawc3/hIe3Q4f7ylXwgYqy82MYkF8CIcDLv8iHwO68807i4+OZN28egYGBTJo0iQEDBrBv3z68vLzo3r07K1as4MYbb+TSpUvs378fPz8/Dhw4QGxsLCtXrqRdu3b4+/u7+EOJiIiIiDtdyszjn+tP8On64yRnmjsxVfy9uL1TbW7vVIvQyj5FX9xWAPvmmJ3bzu28PGmBRoOg0xio2aHY9ZdmFSP85GfBCyVwgerjZ4uUln8NPWvXrqVz586AeTdSdHQ0c+bM4aabbqJnz568//77AKxatYpWrVoRERHBihUriI2NZcWKFfTo0cOlH0dERERE3OdUchYfrTnGV5tPkZ1vdm6LDvHj3m51uKlNNH7eHkVfPDfD3N1Z/w6knjTnPP3Mx9o6PgShdV3wCUq/ihF+ypj9+/fj6elJhw6/Je/Q0FAaNmzI/v37AejRowePPPII58+fZ+XKlfTs2bMw/Nxzzz2sW7eOiRMnltRHEBEREREH7T2byvsrj/LT7nOFnduaVg/k/u516d80Ak+PYpxUSU+ETe/D5o8gJ8Wc869qNjBo93eoFFr8D1CGVIzw4+Vv7sKUxO/rJs2aNSMkJISVK1eycuVKnn/+eSIiInjppZfYvHkz+fn5hbtGIiIiIlK6GIbB+iMXeW/VUVb9rnNbt/pVeaBHXTrXDS1e57bzB837eX7fxCCkLnQeDS3+Al7FvPS0jKoY4cdiKVOHtRo1akRBQQEbN24sDDAXL17k4MGDNG5s9lW3WCx069aNuXPnsnfvXrp27Yq/vz+5ubm8//77tG3blkqVys5nFhEREakIbHaDRXsTeH/lEXae/q1z28DmUdzfvQ5NqwcVfXHDgBPrzPM8hxb+Nh/dATo/DA37g7UYj86VAxUj/JQx9evXZ+jQodx77728//77BAQEMHnyZKpXr87QoUMLX9ezZ0/Gjx9P27ZtqVy5MgDdu3fn888/57HHHiup8kVERETkP+Tk2/h+2xk+XH2UYxcyAfDxtHJz22ju7VaHmqHFeGLIboP988ydnjNbL09aIHagGXrKeRMDZyj8lFIff/wxjzzyCIMGDSIvL4/u3buzYMECvLy8Cl/To0cPbDYbPXv2LJzr2bMnc+fOvWJOREREREpGWk4+n204wcdrj3M+PReAID8vbu9Uizs616ZqcTq35WXBjs/NO3ouHTfnPH2h5V+h4yioWq/4H6CcsRiGYZR0Ec5KS0sjKCiI1NRUAgMDr/haTk4Ox44dIyYmBl9f3xKqsOzSPz8RERGR4ktMy+H/1hzj840nycgtACAqyJd7utXh1nbRVPIpxh5EVjJs+tBsZJB10ZzzC4H290K7e6FyNRd8grLjf2WD/6SdHxERERERFzl6PoP3Vx7lh+1nyLPZAWgQXpn7u9dlSMsovIrTue3ScVj/Nmz/7Lc7LINrQecx0HIkeOt+x6tR+BERERERKaZdp1N4d8URFu5N4NfnqtrVrsIDPerSq2EYVmsxOred3WE2Mdj7AxhmoCKyJXR5GBoNBQ/9SO8o/ZMSERERESkCwzBYe/gi7648zNrDFwvn4xqF8WDPurSpFVKcxeHIMlj7Bhxb+dt83d7Q5RGI6W52NBanKPyIiIiIiDjBZjf4ZW8C7648wq7L7ao9rBaGtojigZ51aRAeUIzF82HvHDP0JO425ywe0OxG8/G2iGbF/wAVmMKPiIiIiIgDcgtszNl+hvdXHuXo5XbVvl5Wbm1Xk793i6FGlWKcucnLhG3/Ms/0pJ4057wqQZs7oOODEFzTBZ9Aym34sdvtJV1CmVQGm/+JiIiIuFVGbgH/3niSf6w5SmKa2a460NeTOzvX5o7OtQktTrvqzItm17ZNH0D2JXOuUjXocD+0vQf8i/HonPxBuQs/3t7eWK1Wzp49S7Vq1fD29sai5yEdYhgG58+fx2KxXHGfkIiIiEhFdDEjl0/XHefT9SdIzc4HIDzQh793rcNfOtSkcnHaVV86Yd7Ps+1fUJBtzoXUgU6jzXt6vPxc8AnkP5W78GO1WomJieHcuXOcPXu2pMspcywWCzVq1MDDw6OkSxEREREpEWdSsvlw1VG+3HySnHzzaaKYqpV4oEcdhrWqjo9nMX5OSthtnufZ8z0YNnMusiV0HQuNhoBVP4O5U7kLP2Du/tSsWZOCggJsNltJl1OmeHl5KfiIiIhIhXQ4KYP3Vh5hzvYzFNjNowDNqgfxYM+69G0SgUdR21UbBhxfDWtmwZGlv83XvQ66jFXntmuoXIYfoPDRLT2+JSIiIiL/y+7Tqbyz4vAVd/R0qhPKQ73q0rVe1aIfobDb4MB8M/Sc3WbOWazQ5AazXXVkC5fUL44rt+FHREREROS/MQyDDUeTeWfFYVbHXyic79M4nId61qVVzSpFX7wgF3b+G9a9CRcPm3OevtDqb+aZnpCYYlYvRaXwIyIiIiIVht1usOxAEm+vOMz2kymAeUfPkBZRPNCjLg0jinFHT04abPk/2PAOZCSac77B0P5eaH8/VK5W7PqleBR+RERERKTcK7DZ+Wn3Od5ZfoSDiekAeHtaubltDe7vXpfokGLc0ZORBBvehc0fQa556SmB1aHTKGh9O/gUI1CJSyn8iIiIiEi5lZNv49utp/lg1VFOJmcBUNnHk791rMXdXWsTFuBb9MUvHTcfbdv+GRTkmHNVG5rneZrdBJ7exf8A4lIKPyIiIiJS7mTkFvDFxhN8uPoY59PNi0lDKnlzV+fa3N6pNkH+xWiKlbAH1s66sl119bbQbRw06A9Wa/E/gLiFwo+IiIiIlBspWXl8su44H689XngxaWSQL/d1r8Mt7aLx9y7Gj78n1sOa1yD+l9/m6vaGro9C7a5qV10GKPyIiIiISJmXlJ7DR2uO8dn6E2TmmbsxMVUr8WCPugxrVR1vzyLuxhgGHFoEa16HUxvMOYsVGg81Q4/aVZcpCj8iIiIiUmadvpTFB6uO8tXmU+QW2AGIjQhgVK96DGgWWfSLSW0FsPd7M/Qk7TPnPLyh5V+h88MQWtdFn0CuJYUfERERESlzjpzP4N0VR5iz/QwFdvNm0lY1gxndqx7XxYYV/WLS/GyzgcG62ZBy0pzzDoB2d0PHhyAgwkWfQEqCwo+IiIiIlBl7z6byzvIjLNhzDsPMPHSpF8qonvXoVDe06KEnJw22fATr34HMJHPOvyp0fBDa/R38gl1Sv5QshR8RERERKfW2nkjm7eVHWHYgqXAurlE4D/WqS+uaVYq+cOYF846eTR/+dkdPULTZrrrV38DLr5iVS2mi8CMiIiIipZJhGKw/cpE3lx1m/dGLAFgtMLB5FA/1rEujyMCiL55yCta/BVs/hYJsc65qQ7OJQbMbwaMYrbCl1FL4EREREZFSxTAMlh9M4q1lh9l2MgUALw8Lw1vV4IGedYmpWqnoi1+IhzWzYNeXYC8w56Jam3f0NByoO3rKOYUfERERESkV7HaDRXsTeGv5YfaeTQPAx9PKre2iua9HXaoHF+MRtLM7zDt69s0DLh8WiukOXcdBnZ66o6eCUPgRERERkRJVYLMzf9c53l5+mPikDAD8vT24rWMt7ukWQ1iAb9EWNgw4sRZWvwpHlv0233CgudNTo60LqpeyROFHREREREpEXoGdH7af5p0VRzhxMQuAAF9P7upcm7u6xFClknfRFjYMiF8Mq1+BUxvNOYuHeZany1gIb+yaDyBljsKPiIiIiFxTOfk2vtp8ivdXHuFsag4AIZW8uadrDLd1qkWgbxGbDdhtsH+eudOTsNuc8/Axu7Z1eRiq1HbNB5AyS+FHRERERK6JzNwCPt94gg9WHeNCRi4AYQE+3Ne9Dn/tUBN/7yL+aGrLh11fw5rX4WK8OeddGdreDZ1G6WJSKaTwIyIiIiJulZaTzz/XHeejNce4lJUPQPVgPx7oWZeb2tTA18ujaAvnZ8P2z2DtG5B6ypzzDYYOD0CH+8E/xDUfQMoNhR8RERERcYuUrDz+b+1xPll7jLQcs610TNVKPNSzLsNaVcfLo4htpXPTYfNHsP5tyLx86WmlMOg82tzt8Qlw0SeQ8kbhR0RERERc6mJGLv9Yc4x/rT9BRq4ZeuqHVWb0dfUY1DwKD2sR20pnJcPG98xfOanmXFA0dHnEPNfjVYxW2FIhKPyIiIiIiEskpeXwwaqjfL7xJNn5NgAaRQYy5rp69GsSgbWooSc9Ada/BZv/D/IzzbnQ+ma76mY3gUcRGyRIhaPwIyIiIiLFci41m/dWHOHfm0+RV2AHoHmNIMZcV5+4RmFYinqBaMpJ8zzPtn+BzWyQQEQz6DYeGg0BaxHPCkmFpfAjIiIiIkVyKjmLd1ce4dstp8mzmaGnTa0qjLmuHj0aVCt66Ll4BNa8Bju/BLv52Bw12kP3x6B+HyjqulLhKfyIiIiIiFOOX8jk7eWH+WH7GQrsBgAdYkJ4pHd9OtUNLXroSdpv3tGz5zswzDBFTA8z9NTuqtAjxabwIyIiIiIOOZyUwdvLDzN3xxkuZx661a/KmOvq0z6mGG2lz+6A1a/A/h9/m6vfF7pPgOj2xapZ5Pec7i+4atUqBg8eTFRUFBaLhTlz5hR+LT8/n0mTJtGsWTMqVapEVFQUt99+O2fPnr1ijeTkZEaOHElgYCDBwcHcc889ZGRkFPvDiIiIiIjrxSem8/C/t9Pn9ZX8sN0MPtfFhvH9Q5351z0dih58Tm6Ez26ED3pcDj4W8yzP/atg5NcKPuJyTu/8ZGZm0qJFC+6++26GDx9+xdeysrLYtm0bU6dOpUWLFly6dIlHHnmEIUOGsGXLlsLXjRw5knPnzrF48WLy8/O56667uO+++/jiiy+K/4lERERExCUOJKTx5tLDLNhzDuPyTk+fxuE8fF19mtUIKtqihgHHVsGql+H4anPOYjW7tnUdB2Gxrile5E9YDOPXb+UivNli4YcffmDYsGH/9TWbN2+mffv2nDhxgpo1a7J//34aN27M5s2badu2LQALFy5kwIABnD59mqioqKv+vmlpaQQFBZGamkpgYGBRyxcRERGRP7H3bCpvLj3Mwr0JhXP9mkQwpnc9mkQVI/TELzYfbzu10ZyzekHLv0CXsRBat/iFS4XkTDZw+5mf1NRULBYLwcHBAKxfv57g4ODC4AMQFxeH1Wpl48aN3HDDDX9YIzc3l9zc3MJxWlqau8sWERERqXB2n05l9rJ4Fu9LBMz+AgOaRjKmdz1iI4r4P5ztdji4wNzpObfDnPPwgTZ3QOeHITjaNcWLOMCt4ScnJ4dJkybxl7/8pTCFJSQkEBYWdmURnp6EhISQkJDwZ8swY8YMpk+f7s5SRURERCqsHadSeHNpPEsPJAFm6BnUPIox19WjQXhA0Ra122DfXFj1CiTtNee8KkG7u6HTGAgId1H1Io5zW/jJz8/n5ptvxjAM3n333WKtNWXKFMaNG1c4TktLIzpa/5dAREREpDi2nbzEG0viWXnoPABWCwxtWZ1RvepRL6xy0Ra1FZitqle/AhcOmXPeAdDhfuj4EFQKdVH1Is5zS/j5NficOHGCZcuWXfHsXUREBElJSVe8vqCggOTkZCIiIv50PR8fH3x8fNxRqoiIiEiFs/VEMrOWxLM6/gIAHlYLw1pWZ1SvutSpVtTQk29eSrrmNUg+as75BpmBp8P94FfFRdWLFJ3Lw8+vwSc+Pp7ly5cTGnpluu/UqRMpKSls3bqVNm3aALBs2TLsdjsdOnRwdTkiIiIictmW48m8sfS30ONptTC8dXUe6lmP2lUrFW3RglzY/hmsmQWpJ805/1DoNAra3Qu+ak4lpYfT4ScjI4PDhw8Xjo8dO8aOHTsICQkhMjKSG2+8kW3btjF//nxsNlvhOZ6QkBC8vb1p1KgR/fr149577+W9994jPz+f0aNHc+uttzrU6U1EREREnLPluLnTs+bwb6HnxjY1GNWrHtEh/kVbND8btv3TDD3pl+90rBQGXR6GtneDdxHDlIgbOd3qesWKFfTq1esP83fccQdPP/00MTExf/q+5cuX07NnT8C85HT06NH8+OOPWK1WRowYwezZs6lc2bFtVrW6FhEREbm6zceTmbXkEGsPXwTM0HNT2xo81LMYoScvE7b8H6ydDZmXjzIEREGXR8wObl5+LqpexDHOZINi3fNTUhR+RERERP67TceSeWOpi0NPbgZs/hDWvQlZ5roE1YSuY6HV38BT57OlZJSqe35ERERE5NrYePQibyyNZ92R34eeaB7qWbfooScnDTZ9AOvfhuxkc65Kbeg2AVrcCh5erile5BpQ+BEREREp4zYevcisJfGsP2qGHi+P30JPjSpFDD3ZKb+FnpwUcy6kLnR/DJrdBB76MVLKHn3XioiIiJRR7gk9l2DDe7DhXchNNeeqNjBDT5PhCj1Spum7V0RERKSM2Xw8mdcXHyp8vM3Lw8LNbaN5qFc9qgcXseFAVrK5y7PxfchLN+eqxV4OPTeA1cNF1YuUHIUfERERkTJi64lkXl/8W8tql4SezAuw/i3Y9CHkZZhzYU2gx0RoNASsVhdVL1LyFH5ERERESrltJy/x+uJDV1xOelPbaEb1KsbjbRnnYd1s2PwR5GeacxHNoMckaDhQoUfKJYUfERERkVJqx6kUXl98iJWHzgMuupw0IwnWvmGGnoJscy6y5eXQ0x8sFtcUL1IKKfyIiIiIlDI7T6Uwa8khlh80Q4+H1cKI1tUZc11914aeqNbQczLUv16hRyoEhR8RERGRUmL36VReX3KIZQeSADP03NCqOmOuq0et0EpFWzQ98bfH234NPdXbQM8pUC9OoUcqFIUfERERkRK250wqs5YcYsl+M/RYLXBDqxqMua4etasWI/SsfQO2fAQFOeZc9baXQ09vhR6pkBR+RERERErIgYQ0Xl98iEV7EwEz9AxrWZ0xvesTU+TQk3A59PzflaGn1xSoq9AjFZvCj4iIiMg1Fp+Yzqyl8fy06xxg5pEhLaJ4uHd96larXLRF/yz01GhnnulR6BEBFH5ERERErpmj5zN4Y2k883aexTDMuYHNIxnbuz71wwOKtmh6AqyZBVs//l3oaX859Fyn0CPyOwo/IiIiIm524mIms5ce5oftp7FfDj39mkQwtk99YiMCi7bofws9vaZAnV4KPSJ/QuFHRERExE1OX8rirWWH+WbraWyXU09cozDGxjWgafWgoi2akWSGnt83MojuYO70KPSI/E8KPyIiIiIudi41m7eWHebrLafIt5mhp2fDajwa14AW0cFFWzTjPKyddWXLau30iDhF4UdERETERZLScnhnxRG+2HiSPJsdgG71qzI2rgFtalUp2qKZF34LPflZ5py6t4kUicKPiIiISDFdyMjlvRVH+NeGE+QWmKGnQ0wI4/o0oEOd0KItmnkR1r0Bmz78LfREtYZej+tyUpEiUvgRERERKaKUrDw+XH2Uj9ceJyvPBkCbWlUY36cBnetVLdqiWcmwbjZs/ADyM825qFbm5aT1r1foESkGhR8RERERJ6Xn5PN/a47zj9VHSc8tAKB5jSDGX9+Q7vWrYilKQMlKhvVvwcb3IS/DnItsYYaeBv0UekRcQOFHRERExEFZeQV8uu4E7686QkpWPgCxEQGMv74hcY3CihZ6slNg/duw4V3ISzfnIpqboadhf4UeERdS+BERERG5ipx8G59vPMm7Kw5zISMPgLrVKvFonwYMaBqJ1VqEgJKTBhvfg3VvQW6qORfezGxZHTtQoUfEDRR+RERERP6LvAI7X205xVvL4klMywWgZog/Y+PqM7RldTyKEnpyM2DT+7DuTci+ZM6FNb4cegaD1erCTyAiv6fwIyIiIvIfCmx2vt92hjeWxnMmxbxTJyrIl4d712dEmxp4eRQhoORlweZ/mG2rsy6ac1UbmKGn8Q0KPSLXgMKPiIiIyGV2u8GPu84ya0k8xy6YndaqBfgwulc9bm0fjY+nh/OL5mfDlo9hzeuQmWTOhdQ1Q0/TEWAtwpoiUiQKPyIiIlLhGYbBL/sSee2XQxxMNJsOhFTy5sEedflbx1r4eRchoBTkwrZ/wupXIf2cORdcC3pMgua3gId+DBO51vRfnYiIiFRYhmGwKv4Cr/5ykF2nzaYDAb6e3N+9Dnd2iaGyTxF+VCrIgx2fw6pXIO20ORdYA3o8Bi1HgoeXCz+BiDhD4UdEREQqpE3Hknll0UE2HU8GwN/bg7u7xHBvtzoE+RchoNgKYOe/YdVMSDlpzgVEQrfx0Pp28PRxYfUiUhQKPyIiIlKh7Dqdwiu/HGLVofMAeHtaua1jLR7sWZeqlYsQUOw22PMdrHgRko+Yc5XCzNDT5k7w8nVd8SJSLAo/IiIiUiEcTEjntcUHWbQ3EQBPq4Wb20Uz5rp6RAb5Ob+g3Q7758GKGXD+gDnnHwpdxkK7v4O3v+uKFxGXUPgRERGRcu34hUxeX3KIeTvPYhjm3aE3tKzOI3H1qRVayfkFDQMOLYTlz0PCbnPONwg6Pwwd7gefANd+ABFxGYUfERERKZfOpmQze2k832w9jc1uADCgWQSPxjWgfngRAophwJFlZug5s9Wc8w6ATg9Bx4fAL9h1xYuIWyj8iIiISLlyISOXd5Yf4bMNJ8iz2QHo1bAa469vSNPqQUVb9PhaWPYcnFxnjr38of190OUR8A9xUeUi4m4KPyIiIlIupOXk8+Gqo3y05hhZeTYAOsSE8FjfhrStXcSAcmozLH8Ojq4wxx4+0O4e6PooVA5zTeEics0o/IiIiEiZlp1n49P1x3l3xRFSs/MBaF4jiMf6NqRrvapYLBbnFz27A5a/APGLzLHVy2xX3W08BFV3XfEick0p/IiIiEiZlFdg56vNJ5m97DDn03MBqBdWmQnXN6Bvk4iihZ6kA+aZnv3zzLHFA1r+BbpPhCq1XFi9iJQEhR8REREpU2x2g7k7zvD6kkOcSs4GoEYVPx6Na8CwVtXxsBYh9CQfhRUvwa6vAAOwQLMboecUCK3r0vpFpOQo/IiIiEiZYBgGi/Ym8uovB4lPygCgamUfHu5dj1vb1cTb0+r8oqlnYNVM2P4Z2AvMudhB0OsJCG/swupFpDRQ+BEREZFSzTAM1h6+yMuLDrDzdCoAQX5ePNCjLnd0roW/dxF+nMk4D2teg80fgc18ZI56cWboqd7ahdWLSGmi8CMiIiKl1vaTl5i58CDrj14EwN/bg7u7xHBv9zoE+Xk5v2D2JVj3Jmx4D/IzzblaXeC6J6FWZxdWLiKlkcKPiIiIlDqHEtN5ZdFBftmXCIC3h5WRHWvyUM96VAvwcX7B3HQz8Kx7E3LN3SOiWsF1U6HudVCU5ggiUuYo/IiIiEipcSo5i1lL4vlh+2nsBlgtMKJ1DR6Jq0+NKv7OL5ifbT7atuY1yDJ3jwhrbD7eFjtQoUekglH4ERERkRJ3ISOXt5Yd5vONJ8i3GQD0axLB+OsbUD88wPkFbfmw/V+wciaknzPnQupCr8ehyQ1g9XBh9SJSVij8iIiISIlJz8nnw9XH+Mfqo2Tl2QDoXDeUif1iaRkd7PyCdhvs+c68q+fScXMusAb0nAQt/goe+tFHpCLTnwAiIiJyzeXk2/jX+hO8s+Iwl7LyAWheI4iJfWPpWr+q8wsaBhxcAMueg6R95lylatBtArS9CzyLcE5IRModhR8RERG5Zgpsdr7bdppZS+I5l5oDQJ1qlXjs+ob0axqBpShncI6uhKXPwJkt5tg3CLo8Ah0eAO9KLqxeRMo6hR8RERFxO8Mw+HlPAq/8cpCj580W05FBvjwa14Dhravj6VGEC0pPbzFDz7GV5tjL3ww8XR4GvyourF5EyguFHxEREXGrtYcv8NLCA+y6fEFpFX8vRvWqx9861sLXqwiNBxL3mWd6Dsw3x1Yv89G2bhMgINyFlYtIeaPwIyIiIm6x50wqLy08wOr4C4B5Qenfu9Xh3m4xBPgW4YLS5GOwYgbs+howwGKFFn+BHpOgSi3XFi8i5ZLTe8yrVq1i8ODBREVFYbFYmDNnzhVfNwyDadOmERkZiZ+fH3FxccTHx1/xmuTkZEaOHElgYCDBwcHcc889ZGRkFOuDiIiISOlw/EImo7/YxqA317A6/gJeHhbu7FybVRN7Ma5PA+eDT9o5mD8O3moLu74CDGg0BB7aAMPeUfAREYc5vfOTmZlJixYtuPvuuxk+fPgfvj5z5kxmz57Np59+SkxMDFOnTqVv377s27cPX19fAEaOHMm5c+dYvHgx+fn53HXXXdx333188cUXxf9EIiIiUiKS0nOYvTSeLzedosBuYLHA0BZRjOvTkJqhRbigNCsZ1s6CjR9AQbY5V7c3XPckVG/t0tpFpGKwGIZhFPnNFgs//PADw4YNA8xdn6ioKMaPH8+ECRMASE1NJTw8nE8++YRbb72V/fv307hxYzZv3kzbtm0BWLhwIQMGDOD06dNERUVd9fdNS0sjKCiI1NRUAgMDi1q+iIiIuEBaTj4frDzKR2uOkZ1v3tXTs2E1JvaNpXFUEf6ezsuEDe/C2tmQa54TIroD9J4Gtbu6sHIRKQ+cyQYuPfNz7NgxEhISiIuLK5wLCgqiQ4cOrF+/nltvvZX169cTHBxcGHwA4uLisFqtbNy4kRtuuOEP6+bm5pKbm1s4TktLc2XZIiIiUgQ5+TY+23CCt5YfJuXyXT0to4OZ3D+WjnVCnV+wIA+2fQorZ0JmkjkX1sQMPQ36QlHaYIuI/I5Lw09CQgIA4eFXdloJDw8v/FpCQgJhYWFXFuHpSUhISOFr/tOMGTOYPn26K0sVERGRIrLZDfOunsWHOHv5rp661SrxWN9Y+jYJd/6uHrsd9nxrXlCacsKcC65lPt7WdARYi9ARTkTkT5SJbm9Tpkxh3LhxheO0tDSio6NLsCIREZGKxzAMluxPYubCA8QnmY2KIgJ9ebRPfUa0ruH8XT2GAYcWmXf1JO015yqFQY+J0PoO8PR28ScQkYrOpeEnIiICgMTERCIjIwvnExMTadmyZeFrkpKSrnhfQUEBycnJhe//Tz4+Pvj4+LiyVBEREXHC1hPJzFhwgC0nLgEQ5OfFQz3rckfn2kW7q+fEOlgyHU5tMMc+QeblpB0fBO9KLqxcROQ3Lg0/MTExREREsHTp0sKwk5aWxsaNG3nwwQcB6NSpEykpKWzdupU2bdoAsGzZMux2Ox06dHBlOSIiIlJMh5PSmbnwIL/sSwTAx9PKXV1ieLBnXYL8inBXz7ldsOxZiP/FHHv6Qof7octY8A9xXeEiIn/C6fCTkZHB4cOHC8fHjh1jx44dhISEULNmTcaOHctzzz1H/fr1C1tdR0VFFXaEa9SoEf369ePee+/lvffeIz8/n9GjR3Prrbc61OlNRERE3C8xLYdZSw7x1eZT2A2wWuDmttGMjWtARJCv8wsmH4Vlz5tnewAsHtD6dvMRt0D9/S8i14bT4WfLli306tWrcPzrWZw77riDTz75hIkTJ5KZmcl9991HSkoKXbt2ZeHChYV3/AB8/vnnjB49mt69e2O1WhkxYgSzZ892wccRERGR4kjLyef9lUf4aM0xcvLtAPRpHM6kfg2pFxbg/ILpibBqJmz9BOwF5lzTEdDrCQit67rCRUQcUKx7fkqK7vkRERFxrdwCG/9af2Xb6ja1qjClfyxtaxfhcbScVFj3Jqx/G/KzzLl6cWbb6sgWLqxcRCq6ErvnR0RERMoWu91g7s4zvLLoEGdSsgGoF1aZSf1iiWsU5nzb6vwc2PIRrHoFspPNueptIG46xHRzcfUiIs5R+BEREamADMNgVfwFXvz5APvPmZeHhwf6MK5Pg6K1rbbbYOeXsGIGpJ4y50Lrmzs9jQbrglIRKRUUfkRERCqY3adTeXHhftYevghAgK8nD/asy12dY/DzdrJttWHAwZ/Nu3rO7zfnAqKg52RoORI89KOGiJQe+hNJRESkgjiVnMXLiw4yb+dZALw9rNzRuRYP9axHlUpFuFD0xHpY8vRvd/X4BkHXcWbrai8/1xUuIuIiCj8iIiLl3KXMPN5cdph/bThOvs3AYoEbWlZn3PUNqFHF3/kFE/fB0ulwaKE59vSFDg9A17HgV8WltYuIuJLCj4iISDmVk2/j47XHeWfFYdJzzDbT3epXZXL/WJpEBTm/YMopWP4C7Pw3YJh39bT6m/mIm+7qEZEyQOFHRESknLHZDb7fdprXFh/iXGoOAI0iA5nSP5buDao5v2BWMqx5DTZ+ALZcc67xULhuKlSt78LKRUTcS+FHRESknDAMg5WHzvPizwc4kJAOQPVgPyb0bcDQFtWxWp1tW50NG9+D1a9Dbqo5V6sr9HkGarRxcfUiIu6n8CMiIlIO7D6dyoyf97PuiNnBLdDXk9HX1eP2TrXx9XKyg5vdBju+MB9xSzebIxDWBPpMNy8qVdtqESmjFH5ERETKsFPJWbzyy0Hm7vitg9udXWrzUM+6BPs72cHNMMwmBkuehvMHzLmgaOj1BDS/GaxOhigRkVJG4UdERKQMSsnK461lh/nn+hPk2ewA3NCqOuP6NCA6pAgd3E5tgsVPwcl15tivCnSbAO3+Dl6+LqxcRKTkKPyIiIiUIbkFNv657gRvLosn7XIHty71QpnSvxFNqxehg9uFeHOn58B8c+zpCx0fhC5jwS/YVWWLiJQKCj8iIiJlgN1u8OOus7y86CCnL2UDEBsRwOT+sfRoUA2Ls+dw0s7Byhdh27/AsIHFerlt9RS1rRaRckvhR0REpJTbcPQiMxbsZ+dps+NaeKAP469vyIjWNfBwtoNbThqsmw3r34b8LHOu4UCIewqqNXRx5SIipYvCj4iISCl1OCmDF38+wJL9iQBU8vbgwZ51uadrHfy8nWw+UJAHWz8xd3uyzI5wRHcw21bX7OjawkVESimFHxERkVLmfHous5Yc4svNp7DZDTysFv7SPppHejegWoCPc4sZBuybA0ufgeSj5lxofYh7GmIHqm21iFQoCj8iIiKlRHaejX+sPsp7K4+QmWcDoE/jcCb1i6VeWGXnFzy+FhZPgzNbzHGlMOg1BVrdDh76EUBEKh79ySciIlLCbHaD77ae5tXFB0lMywWgRY0gHh/QiA51Qp1f8PxBs4PbwQXm2KsSdHkYOo0GnyKEKBGRckLhR0REpAStOnSeFxbs50BCOgA1qvgxsV8sg5pFYnW2mUHaOVgxA7b/Cww7WDygzZ3QYxIEhLu+eBGRMkbhR0REpAQcTEjn+QX7WXXoPABBfl6Mua4et3WqhY+nk80Mfu3gtu4tKDDbYBM7yDzXU7W+awsXESnDFH5ERESuoaT0HF775RBfbzmF3QAvDwu3d6rNmOvqEezv7dxitnyzg9uKFyHrgjmnDm4iIv+Vwo+IiMg1kJVXwD9WH+O9lUfIutzMYECzCCb1i6VWaCXnFjMM2D/PPNdT2MGt3uUOboPUwU1E5L9Q+BEREXEjm93g+22neeWX35oZtKoZzJMDG9GmVojzC57aBL88Cac2muNK1aDnZGh9B3h4ubByEZHyR+FHRETETdbEX+D5BfvZfy4NMJsZTOoXy6DmkVic3Z25eASWTod9c82xpx90HmN2cfMJcHHlIiLlk8KPiIiIix1KTGfGgv0sP2g2Mwjw9WTMdfW4o3Nt55sZZCXDypmw+R9gzwcs0Gok9HoCAqNcX7yISDmm8CMiIuIi59NzeX3JIb7cdBK7AZ5WC3/rWIuHe9cnpJKTzQzyc2Dje7D6NchNNefqxZnNDMKbuL54EZEKQOFHRESkmHLybXy05hjvLD9M5uVmBn2bhDOpXyx1qjl5qajdDnu+haXPQOopcy6iGfR5Fur2cnHlIiIVi8KPiIhIERmGwbydZ5m58CBnUsz7dZrXCOKJAY3oUCfU+QWPrTKbGZzbaY4Dq8N1U6H5LWC1urByEZGKSeFHRESkCLaeSObZ+fvZcSoFgMggXyb1i2VIiyisViebGSQdgMXTIH6ROfYOgG6PQseHwMvPtYWLiFRgCj8iIiJOOJWcxYsLD/DTrnMA+Ht78FDPutzTtQ5+3k42M8hIguUvwLZPwbCD1RPa3g09JkGlqm6oXkSkYlP4ERERcUBaTj5vLz/Mx2uOk2ezY7HALW2jGXd9A8ICfJ1bLC8LNrwNa2ZBXoY5FzsI4qZD1Xour11EREwKPyIiIv9Dgc3Ovzef4vXFh0jOzAOga72qPD6gEY2jAp1bzG6HXV/Bsmch7Yw5F9Ua+j4PtTq7uHIREflPCj8iIiJ/wjAMVhw6z/M/7edwkrk7U7daJZ4c2JieDas5f0npsVWw6AlI2GWOg2pC3FPQZLiaGYiIXCMKPyIiIv/hQEIaz/+0n9XxFwCo4u/FuD4NuLV9Tbw8nAwq5w+ZzQwO/WyOfQKh23jo8AB4Ofm4nIiIFIvCj4iIyGUXMnJ59ZdDfLXZvKTU28PKXV1q81CvegT5eTm3WMZ5WPkibPkYDBtYPKDdPWpmICJSghR+RESkwsstsPHx2uO8tewwGbkFAAxoFsGkfrHUCq3k3GL52bDhXVj9GuSlm3MNB0Kf6VC1vosrFxERZyj8iIhIhWUYBgv3JDDj5wOcTM4CoFn1IKYOakz7mBDnFrPbYc+3sPQZSD1lzkW2gOufh5huLq5cRESKQuFHREQqpD1nUnlm/j42HUsGIDzQh4l9Y7mhVXXnLyk9sQ4WPQ5nt5vjwBrQexo0u0nNDEREShGFHxERqVAS03J4edFBvtt2GsMAXy8r93WvywM96uDv7eRfi8lHYfFTsH+eOfauDF0fhU6jwMvP9cWLiEixKPyIiEiFkJNv48NVR3l35RGy8mwADGsZxcR+sUQFOxlUslNg9Suw8X2w5YHFCq1vh15PQOUw1xcvIiIuofAjIiLlmmEYzNt5lpd+PsDZ1BwAWtUMZtqgxrSqWcW5xWz5sPUTWP4CZJuPy1Gnl3lJaXgT1xYuIiIup/AjIiLl1raTl3h2/j62n0wBICrIl8kDGjG4eaRzl5QaBsT/Ar88CRcOmXNVG5qhp14cOHvhqYiIlAiFHxERKXfOpmQzc+EB5uw4C4C/twcP9qjLvd3r4Ovl4dxiCXvglyfg6Apz7B8KvR6H1neCh/4aFREpS/SntoiIlBvZeTbeX3WE91YeISffjsUCN7auwYS+DQkP9HVusfREWP4cbPsXYICHN3R8ELqNB98gt9QvIiLupfAjIiJl3p+d62lXuwrTBjWhWQ0ng0p+Nqx/C1a/DvmZ5lyTGyDuaahS26V1i4jItaXwIyIiZdrOUyk8M38fW09cAqB6sB9TBsQysFkRzvXs+c5sXZ122pyr3hb6vgA1O7ihchERudYUfkREpExKTMvhpYUH+H7bGQD8vDx4qGcRz/Wc2gyLpsDpzeY4KNrc6Wk6Qs0MRETKEYUfEREpU3Lybfxj9VHeWfHbfT3DW1dnUr9Y58/1pJ6GJU/D7m/MsVcl6PYodBqtS0pFRMohhR8RESkTDMNgwe4EXliwnzMp2QC0rhnMtMFNaBkd7NxiuRmw9g1YNxsKcgALtBwJvadCQITLaxcRkdLB6uoFbTYbU6dOJSYmBj8/P+rWrcuzzz6LYRiFrzEMg2nTphEZGYmfnx9xcXHEx8e7uhQRESkn9pxJ5Zb3NzDqi22cSckmMsiXN25tyXcPdnYu+NjtsOMLeLMNrJppBp9aXeC+FTDsbQUfEZFyzuU7Py+99BLvvvsun376KU2aNGHLli3cddddBAUF8fDDDwMwc+ZMZs+ezaeffkpMTAxTp06lb9++7Nu3D19fJx9ZEBGRcut8ei6vLDrI11tPYRjg62Xl/u51ub9HHfy9nfwr7MQ6WDgFzu0wx1VqQ59nodFgnesREakgLMbvt2RcYNCgQYSHh/PRRx8Vzo0YMQI/Pz8+++wzDMMgKiqK8ePHM2HCBABSU1MJDw/nk08+4dZbb73q75GWlkZQUBCpqakEBga6snwRESkF8grsfLLuGLOXHiYjtwCAIS2imNw/lqhgJ8/iXDoOi6fBvrnm2DsAejwGHR4ATx/XFi4iItecM9nA5Ts/nTt35oMPPuDQoUM0aNCAnTt3smbNGl577TUAjh07RkJCAnFxcYXvCQoKokOHDqxfv/5Pw09ubi65ubmF47S0NFeXLSIipYBhGCw/mMSz8/dz7IJ5x07zGkE8NbgxbWqFOLdYThqsfhU2vAO2PLBYofUd0OsJqFzNDdWLiEhp5/LwM3nyZNLS0oiNjcXDwwObzcbzzz/PyJEjAUhISAAgPDz8iveFh4cXfu0/zZgxg+nTp7u6VBERKUUOJ2Xw7Px9rDx0HoCqlX2Y2K8hN7augdXqxGNpdhts/wyWPQuZ5lrU6Wne1xPexPWFi4hImeHy8PP111/z+eef88UXX9CkSRN27NjB2LFjiYqK4o477ijSmlOmTGHcuHGF47S0NKKjo11VsoiIlKDU7HxmL43n03XHKbAbeHlYuLtrDKN71SPA18u5xY6vhYWTIGG3OQ6tB9c/Dw366lyPiIi4Pvw89thjTJ48ufDxtWbNmnHixAlmzJjBHXfcQUSE2UknMTGRyMjIwvclJibSsmXLP13Tx8cHHx89ly0iUp7Y7AZfbznFK4sOcjEzD4DesWE8OagxMVUrObfYf57r8QmCnpOg3b3g6e3awkVEpMxyefjJysrCar2yg7aHhwd2ux2AmJgYIiIiWLp0aWHYSUtLY+PGjTz44IOuLkdEREqhTceSmf7jXvaeNc9w1q1WiamDGtOzYZhzC+Wmw+rXYP3bYMs1z/W0udM811OpqusLFxGRMs3l4Wfw4ME8//zz1KxZkyZNmrB9+3Zee+017r77bgAsFgtjx47lueeeo379+oWtrqOiohg2bJiryxERkVLkTEo2MxbsZ/6ucwAE+HoyNq4Bt3eqhZeHE1fP2e2w89+wdDpkJJpzMd2h7wyIaOqGykVEpDxwefh58803mTp1Kg899BBJSUlERUVx//33M23atMLXTJw4kczMTO677z5SUlLo2rUrCxcu1B0/IiLlVHaejfdXHeG9lUfIybdjscCt7Woy4foGhFZ28rHmkxtg4WQ4u90cV4mB65+D2IE61yMiIv+Ty+/5uRZ0z4+ISNlgGAY/70ng+Z/2cyYlG4D2tUN4akhjmkQFObdYyilY8hTs+c4c674eERGhhO/5ERERATiQkMb0eftYf/QiAFFBvjw+sBEDm0VicWaHJi8T1r4Ba2dDQTZggda3wXVTobKTZ4RERKRCU/gRERGXSsnK4/XFh/hs40lsdgMfTysP9KjLAz3q4uft4fhChgG7vzW7uKWfNedqdYF+MyCyhXuKFxGRck3hR0REXMJmN/hy80leWXSQS1n5APRvGsHjAxoRHeLv3GJntpnnek5tNMfBNaHPs9B4qM71iIhIkSn8iIhIsW0+nsxTc/ey75zZurpBeGWeGtyELvWcbDednghLn4EdnwMGePlDt3HQaQx4qSmOiIgUj8KPiIgUWUJqDjN+3s/cHeZjaQG+nozr04C/dXSydXVBHmx8D1bOhLx0c675LRD3NARGub5wERGpkBR+RETEaTn5Nj5ac4y3lx8mK892uXV1NBOub+hc62rDgPhfYOEUSD5izkW1gv4zIbq9e4oXEZEKS+FHREQcZhgGS/cn8exP+zhxMQuA1jWDmT6kKc1qONm6+vwhWDQFDi8xx5XCzJ2eFn8BqxO7RiIiIg5S+BEREYccOZ/BMz/uY+Wh8wCEBfjw+IBGDG0Z5Vzr6uwU8/G2Te+DvQCsXtDpIeg2AXx1d5uIiLiPwo+IiPxPGbkFvLksnv9bc4x8m4GXh4W/d6vDqF71qOzjxF8jdhts/xcsfRayLphzDfpD3+chtK57ihcREfkdhR8REflThmEwd8dZXliwn6T0XAB6NazGtMFNiKlaybnFTqyDnydBwi5zXLWBeV9PvTgXVy0iIvLfKfyIiMgf7DubxtPz9rLpeDIAtUL9mTaoMb0bhTu3UOoZ85LSPd+aY58g6DkZ2t8LHl4urlpEROR/U/gREZFCKVl5vLb4EJ9tOIHdAF8vK6N71ePv3erg6+Xh+EL5ObD+LVj9KuRnARZocwdcNxUqOXn3j4iIiIso/IiICDa7wddbTvHyooMkZ+YBMLBZJI8PbET1YD/HFzIMOPiz2cXt0nFzLroj9H8Jolq6vG4RERFnKPyIiFRw209e4ql5e9l1OhWA+mGVmT6kCZ3rOblDc/4QLJwMR5aa44BI6PMsNLsRnOkGJyIi4iYKPyIiFdT59FxmLjzAN1tPAxDg48nYPg24vVMtvDycuGcnJ9VsXb3xPbN1tYc3dBoN3caDT2U3VS8iIuI8hR8RkQom32bnX+tP8PriQ6TnFgBwY5saTOoXS7UAH8cXstth5xew5GnINO/+UetqEREpzRR+REQqkA1HL/LU3L0cTEwHoFn1IKYPbULrmlWcW+j0Fvh5IpzZao5D60G/F6F+HxdXLCIi4joKPyIiFUBiWg7P/7SfeTvPAlDF34uJ/WK5uW00HlYnzuOkJ8LS6bDjc3PsHQA9JkKHB8DT2w2Vi4iIuI7Cj4hIOZZvs/Px2mO8sSSezDwbFguM7FCTCdc3JNjfibBiy4eN78OKFyHP3DWi5Ujo/RQEOHn3j4iISAlR+BERKafWHb7AtHl7OZyUAUCrmsE8O7QpTasHObfQ0RWwYCJcOGiOo1rDgJehRlvXFiwiIuJmCj8iIuXMudRsnvtpPz/tOgdAaCVvJvWP5cbWNbA684hbyklY9ATsn2eO/atC3FPQ8m9gdaIbnIiISCmh8CMiUk7kFdj5aM0x3lwWT1aeDasFbutYi3F9GhLk7+X4Qvk5sG42rH4NCrLB4gHt74WeU8Av2G31i4iIuJvCj4hIObA6/jxPzdvL0fOZALStVYXpQ5vQJMqJR9wMAw7+bF5UmnLCnKvVFQbMhPAmbqhaRETk2lL4EREpw86kZPPc/H38vCcBgKqVfZjSP5bhratjsTjxiNuFw7BwEhxeYo4DouD6Z6HpCHBmHRERkVJM4UdEpAzKLbDxj9XHeGvZYbLzbXhYLdzeqRaP9mlAoK8Tj7jlZsCql2H922DPB6sXdB4N3SaAT2X3fQAREZESoPAjIlLGrI4/z1Nz93L0gvmIW/vaIUwf2oRGkYGOL2IYsOc7+GUqpJt3/1CvD/R/CULruqFqERGRkqfwIyJSRpxLzea5+fv5abfZxa1agA+PD4hlWEsnH3FL3AsLHoMTa81xldrQ70Vo0E+PuImISLmm8CMiUsrlFVy+qHSp2cXNw2rhjk61GdunvnOPuGWnwIoZsOlDMGzg6QfdxkPnMeDl67b6RURESguFHxGRUmzdkQtMm/vbRaVta1XhmaFNaRzlxCNudjvs+hIWT4PM8+ZcoyHQ9wUIjnZD1SIiIqWTwo+ISCmUmJbD8z/tZ95O8zxOaCVvpgxoxPBW1Z27qPTcLlgwAU5tNMeh9c3W1XWvc0PVIiIipZvCj4hIKZJvs/PpuuPMWhJPRm4BVgv8rWMtxjt7UWn2JVj2PGz5CAw7eFWCHhOh40Pg6e2+DyAiIlKKKfyIiJQSm44lM23uHg4kpAPQMjqY54Y1pWl1Jy4qtdthx+ew5GnIumDONRkO1z8HQdVdX7SIiEgZovAjIlLCzqfnMuPn/Xy/7QwAVfy9mNw/lpvaRDv3iNvZHeYjbqc3m+OqDWHAy1Cnh+uLFhERKYMUfkRESojNbvD5xhO8vOgg6TkFWCxwa7uaTOzbkCqVnHg0LSsZlj0LWz4GDPCuDD0nQ4cHwMOJR+VERETKOYUfEZESsONUCk/O2c2eM2kANKsexLPDmtIyOtjxRex22P5PWDIdspPNuWY3QZ9nITDS9UWLiIiUcQo/IiLXUGpWPjMXHeCLTScxDAjw9WRiv1j+2r4mHs484nZmm/mI25mt5jissfmIW+2u7ilcRESkHFD4ERG5BgzD4LttZ5ixYD8XM/MAGN66OlP6N6JagI/jC/3nI24+gdBzCrS/V4+4iYiIXIXCj4iImx1MSGfqnD1sOm4+mlY/rDLPDmtKxzqhji9S2MXtKci6aM41vwX6PAMBEW6oWkREpPxR+BERcZPM3ALeWBrPR2uOYbMb+Hl5MDauPnd3jcHLw+r4Qud2wU/j4fQmc1ytEQx8FWp3cU/hIiIi5ZTCj4iIixmGwcI9CTwzfx/nUnMA6NckgqmDG1M92M/xhbJTYPkLsPlD86JSdXETEREpFoUfEREXOnExk2lz97Ly0HkAaob4M31IE3rFhjm+iGHArq/gl6mQmWTONRkOfZ+HwCg3VC0iIlIxKPyIiLhATr6N91ce5e0Vh8krsOPtYeWBnnV5qGddfL08HF8ocS/8NAFOrjPHVRtcvqi0p1vqFhERqUgUfkREiml1/HmmztnD8YtZAHSrX5VnhjYlpmolxxfJSYMVL8LG98CwgZc/9JgIHUeBpxMXnoqIiMh/pfAjIlJESWk5PPvTfn7ceRaA8EAfpg5qzMBmkVgsDt7ZYxiw5ztY9ARkJJhzjYZA3xcgONpNlYuIiFRMCj8iIk6y2Q0+33iClxceJD23AKsF7uwcw6N96hPg60QjgvOHYMF4OLbKHIfUhQEzoV6cewoXERGp4BR+REScsPt0Kk/M2c2u06kAtIgO5vlhTWlaPcjxRfKyYPUrsHY22PPB0xe6TYAuD4OnExeeioiIiFMUfkREHJCWk8+riw7yrw0nsBsQ4OvJpH6x/KV9TTysDj7iBnDwZ1gwEVJPmuP6fc3dniq13VK3iIiI/EbhR0TkfzAMg/m7zvHM/H2cT88FYFjLKB4f2IiwAF/HF7p0AhZOhoMLzHFQNPR/CRoOAEfPB4mIiEixKPyIiPwXxy9kMnXuHlbHXwCgTtVKPDusKV3qVXV8kYI8WP8mrHwZCrLB6gmdx0D3x8DbiW5wIiIiUmwKPyIi/yG3wMZ7K353Z4+nldG96nF/jzr4eDpxZ8/RlbBgAlw4ZI5rd4MBr0BYrHsKFxERkf/J6o5Fz5w5w9/+9jdCQ0Px8/OjWbNmbNmypfDrhmEwbdo0IiMj8fPzIy4ujvj4eHeUIiLilLWHL9B/1mpeX3KIvAI73epX5Zex3Xm4d33Hg096Inx3L/xziBl8KlWDGz6AO35U8BERESlBLt/5uXTpEl26dKFXr178/PPPVKtWjfj4eKpUqVL4mpkzZzJ79mw+/fRTYmJimDp1Kn379mXfvn34+jrxDL2IiItcyMjlufn7mLPDvLOnWoAP0wY1ZlBzJ+7ssdtg80ew7FnITQMs0O7vcN2T4BfsttpFRETEMRbDMAxXLjh58mTWrl3L6tWr//TrhmEQFRXF+PHjmTBhAgCpqamEh4fzySefcOutt/7hPbm5ueTm5haO09LSiI6OJjU1lcDAQFeWLyIVjN1u8OXmU7z4837ScgqwWOD2jrUY37chgc7c2XN6K8wfCwm7zHFUaxj0GkS1ckvdIiIiYkpLSyMoKMihbODyx97mzZtH27ZtuemmmwgLC6NVq1Z8+OGHhV8/duwYCQkJxMX9dolfUFAQHTp0YP369X+65owZMwgKCir8FR2tW89FpPgOJKRx0/vrefyH3aTlFNC0eiBzR3Vh+tCmjgef7BSYPw7+0dsMPr5BMPA1+PsSBR8REZFSxuWPvR09epR3332XcePG8fjjj7N582YefvhhvL29ueOOO0hISAAgPDz8iveFh4cXfu0/TZkyhXHjxhWOf935EREpiuw8G28sjecfq49SYDeo5O3BuOsbckenWnh6OPj/hAwDdn8Dix6HzPPmXPNb4frnoHI19xUvIiIiReby8GO322nbti0vvPACAK1atWLPnj2899573HHHHUVa08fHBx8f3XouIsW3/EASU+fu4fSlbAD6Ngnn6SFNiAzyc3yRC/Hw0zg4tsocV21g7vbEdHNDxSIiIuIqLg8/kZGRNG7c+Iq5Ro0a8d133wEQEREBQGJiIpGRkYWvSUxMpGXLlq4uR0QEgMS0HKb/uJcFu80d5qggX6YPbUqfxuFXeefv5GfD6ldh7RtgywNPX/O+ns4Pg6e3myoXERERV3F5+OnSpQsHDx68Yu7QoUPUqlULgJiYGCIiIli6dGlh2ElLS2Pjxo08+OCDri5HRCo4m93gsw0neHnRQTJyC/CwWri7S23GxjWgko8TfwTGL4EF4+HScXNcrw8MeBlCYtxSt4iIiLiey8PPo48+SufOnXnhhRe4+eab2bRpEx988AEffPABABaLhbFjx/Lcc89Rv379wlbXUVFRDBs2zNXliEgFtudMKo//sJtdp1MBaBEdzAs3NKVJVJDji6Sdg4WTYd8ccxwQBf1fhEZDwNEW2CIiIlIquDz8tGvXjh9++IEpU6bwzDPPEBMTw6xZsxg5cmThayZOnEhmZib33XcfKSkpdO3alYULF+qOHxFxiYzcAl775RCfrDuG3YAAH08m9mvIXzvUwsPqYGCxFcDmf8Cy5yAvHSxW6PAg9JoCPgHu/QAiIiLiFi6/5+dacKaXt4hULIv3JTJt7h7OpeYAMKh5JNMGNSYs0In/ufKfd/ZUb2ve2RPZwvUFi4iISLE4kw1cvvMjIlISElJzeHreXhbuNRsaRIf48ezQpvRsGOb4ItkpsOxZ2PwRYJh39sQ9Da3vBKvLr0UTERGRa0zhR0TKtD9raHBvtzo80rs+ft4eji1iGLD3e1g4BTISzbnmt8L1z0JlJ8KTiIiIlGoKPyJSZu0/l8aU73ez41QKAC2jg5kxvBmNIp14HDb5GPw0Ho4sNceh9cw7e+r0cH3BIiIiUqIUfkSkzMnOszFr6SH+sfoYNrtB5csNDUY609CgIA/WzYZVL0NBDnh4Q7fx0PVR8NSlyiIiIuWRwo+IlCkrD53nyTm7OZWcDUC/JhE8PaQJEUFONDQ4sd5saHD+gDmO6Q4DX4eq9VxfsIiIiJQaCj8iUiacT8/l2fn7mLfzLACRQb48M7QpfRqHO75IVjIsngbb/2WO/UOh7wvQ/Bbd2SMiIlIBKPyISKlmtxt8veUUM34+QGp2PlYL3Nk5hnHXN6Cyj4N/hBkG7PwSfnkCsi6ac61vh7jp4B/ivuJFRESkVFH4EZFS63BSOo9/v4dNx5MBaBIVyIzhzWheI9jxRS4chp8ehWOrzHG1WBg0C2p1cnm9IiIiUrop/IhIqZNbYOPdFUd4Z/kR8mx2/Lw8GH99A+7sXBtPDwfv2ynIhTWvw+pXwZYHnr7QYyJ0GgOe3u79ACIiIlIqKfyISKmy6VgyU77fxZHzmQD0aliNZ4c1pUYVf8cXObYK5j8KFw+b47q9YeCrEBLjhopFRESkrFD4EZFSITU7nxd/PsC/N50EoGplH54e0piBzSKxONqMIPMi/PIk7PzCHFcOh34zoMlwNTQQERERhR8RKVmGYfDzngSemreX8+m5ANzaLpop/RsR5O/l6CJmQ4NFj0N2MmCBtndB76fAL9httYuIiEjZovAjIiXmbEo20+buZcn+RADqVK3EC8Ob0bFOqOOLXDxiPuJ2bKU5DmsMg9+A6PZuqFhERETKMoUfEbnmbHaDf60/zsuLDpKZZ8PLw8KDPeryUK96+Hp5OLZIQR6sfQNWvQy23MsNDSZB5zHg4eCOkYiIiFQoCj8ick0dSEhj8ne72XEqBYA2taowY3gzGoQHOL7IifUwfyycP2CO6/SCQa9BSB2X1ysiIiLlh8KPiFwTOfk2Zi+N54NVRymwG1T28WRS/1hGtq+J1epgM4LsS7Dkadj6iTn2rwr9XoRmN6qhgYiIiFyVwo+IuN26wxd4/IfdHL+YBUDfJuFMH9KUiCBfxxYwDNj7Pfw8GTKTzLlWt0GfZ8A/xE1Vi4iISHmj8CMibpOalc/zC/bx9ZbTAIQH+jB9SFP6NY1wfJFLx+Gn8XB4iTmu2gAGzYLaXVxer4iIiJRvCj8i4nKGYbBgt9m++kKG2b76bx1rMrFfLIG+DjYjsBXAhndg+QtQkA0e3tBtAnQdC54+7iteREREyi2FHxFxqYTUHKbO3cPifWb76rrVKvHSiOa0re3E42lnt8O8MZCw2xzX7gaDXoeq9d1QsYiIiFQUCj8i4hJ2u8EXm07y0s8HSM8tMNtX96zHqF518fF0sH11Xqa507PhHTDs4FcFrn8OWo5UQwMREREpNoUfESm2I+czmPLdbjYdTwagZXQwL41oTsMIJ9pXxy8xLytNPWmOm95odnKrXM0NFYuIiEhFpPAjIkWWV2Dng1VHmL30MHk2O/7eHjzWtyG3d6qNh6PtqzPOw6IpsPsbcxxU07yzp34f9xUuIiIiFZLCj4gUyY5TKUz+bhcHEtIB6NGgGs/f0JQaVfwdW8AwYMfnsOgJyEkBixU6PgQ9p4BPZfcVLiIiIhWWwo+IOCUrr4BXfznEx2uPYTegir8XTw1uwtCWUVgcPZdz8QjMHwvHVpnjiGYweDZUb+22ukVEREQUfkTEYSsPneeJH3Zz+lI2ADe0qs6TAxsRWtnB1tO2fFg3G1bOhIIc8PSDXlPMHR8PB1tgi4iIiBSRwo+IXNWlzDye/Wkf3287A0D1YD+ev6EpPRuGOb7I6S0w72FI2muO6/Q021eH1HF9wSIiIiJ/QuFHRP4rwzD4afc5np63lwsZeVgscGfn2ky4viGVfBz84yM3HZY9BxvfBwzwC4F+M6D5LWpfLSIiIteUwo+I/KmE1ByenLOHJfvNy0rrh1XmpRub07pmFccXOfQL/DQOUk+Z4+a3Qt/noVJVN1QsIiIi8r8p/IjIFex2gy83n2LGgv2Fl5U+1LMeDzlzWWnGeVg4GfZ8a46Da8KgWVCvt9vqFhEREbkahR8RKXT8QiaTv9/FhqPmZaUtooOZ6cxlpYYBO7807+3JvvRb++pej4N3JTdWLiIiInJ1Cj8iQoHNzkdrjvHa4kPkFtjx8/JgQt+G3NnZictKLx2HH8fC0eXmOLwZDFH7ahERESk9FH5EKrh9Z9OY9N0udp9JBaBrvarMGN6M6BAHLyu1FcDG92D585CfBR4+0HMydB6j9tUiIiJSqij8iFRQOfk23lp2mPdWHqHAbhDo68mTgxpzU5sajl9WmrAb5o2Bs9vNce1uMPgNCK3rvsJFREREikjhR6QC2nI8mUnf7eLI+UwA+jeNYPqQJoQF+jq2QH62eVHp2jfAsIFPEPR9DlrdpvbVIiIiUmop/IhUIBm5Bby88AD/3HACw4BqAT48O7QJ/ZpGOr7I8TXmZaXJR8xx46HQfyYERLinaBEREREXUfgRqSBWx59n8ne7OZOSDcDNbWvwxIDGBPk7eC4nOwUWT4Ntn5rjgEgY8Ao0GuSegkVERERcTOFHpJxLzc7n+Z/28fWW0wDUqOLHi8Ob07W+ExeN7p8PP42HjARz3PZuiHsafINcX7CIiIiImyj8iJRji/cl8uSc3SSm5WKxwB2davNY34ZU8nHwP/2MJFjwGOybY45D68GQN6FWZ7fVLCIiIuIuCj8i5VByZh7Tf9zL3B1nAahTtRIv3dicdrVDHFvg18tKF06GnBSweECXR6DHJPBysCmCiIiISCmj8CNSjhiGwU+7z/HU3L1czMzDaoF7u9fh0bgG+Hp5OLZIykmY/ygcXmKOI5rB0LchsoX7ChcRERG5BhR+RMqJpLQcps7dw6K9iQA0CK/Myze2oEV0sGML2O2w5SNY8jTkZeiyUhERESl3FH5EyjjDMPh+2xmemb+P1Ox8PK0WHupVj1G96uLj6eBuz4V4mDsaTm0wxzU7mWd7qtZ3X+EiIiIi15jCj0gZdiYlm8e/383KQ+cBaFo9kJkjWtA4KtCxBWz5sG42rHgJbLngXdns4tb2HrBa3Ve4iIiISAlQ+BEpg+x2g39vPsmMBQfIyC3A29PK2Lj63NetDp4eDoaWczth7ihI2G2O68XBoNchuKb7ChcREREpQQo/ImXMyYtZTPpuF+uPXgSgdc1gZt7YgnphlR1bID8bVr4Ea2eDYQO/KtDvRWh+C1gsbqxcREREpGQp/IiUEXa7wafrjzNz4UGy8234ell5rG8sd3aujYfVwdByYj3MGw0XD5vjJjdA/5lQOcx9hYuIiIiUEgo/ImXA0fMZTPpuF5uPXwKgQ0wIM29sTq3QSo4tkJsBS6fDpg8BAypHwMBXodEg9xUtIiIiUsoo/IiUYja7wUdrjvLqL4fILbDj7+3BlP6xjOxQC6ujuz2Hl8KPYyH1pDlu9Te4/jnzcTcRERGRCsTt7ZxefPFFLBYLY8eOLZzLyclh1KhRhIaGUrlyZUaMGEFiYqK7SxEpU+IT0xnx7jpeWHCA3AI73epXZdHY7tzWqbZjwSf7EswZBZ8NN4NPcE247QfzwlIFHxEREamA3Lrzs3nzZt5//32aN29+xfyjjz7KTz/9xDfffENQUBCjR49m+PDhrF271p3liJQJBTY77686yhtL4smz2Qnw8eSJgY24pV00FkcbEuyfDz+Ng4xEwAId7ofrpoKPg00RRERERMoht4WfjIwMRo4cyYcffshzzz1XOJ+amspHH33EF198wXXXXQfAxx9/TKNGjdiwYQMdO3Z0V0kipd7+c2k89u1O9pxJA6BXw2q8MLwZkUF+ji2QcR5+fgz2/mCOQ+vD0Legpv67EhEREXHbY2+jRo1i4MCBxMXFXTG/detW8vPzr5iPjY2lZs2arF+//k/Xys3NJS0t7YpfIuVJXoGdWUsOMeStNew5k0agryev3tSC/7uznWPBxzBg19fwdnsz+Fg8oOs4eGCNgo+IiIjIZW7Z+fnyyy/Ztm0bmzdv/sPXEhIS8Pb2Jjg4+Ir58PBwEhIS/nS9GTNmMH36dHeUKlLi9pxJZcI3OzmQkA5An8bhPD+sKWGBvo4tkHoG5j8K8YvMcXgzc7cnqqV7ChYREREpo1wefk6dOsUjjzzC4sWL8fV18Ie3q5gyZQrjxo0rHKelpREdHe2StUVKSm6BjTeXHubdlUew2Q2q+HsxfWhTBjePdOxsj2HA1k9g8TTITQMPb+gxEbqMBQ8vd5cvIiIiUua4PPxs3bqVpKQkWrduXThns9lYtWoVb731FosWLSIvL4+UlJQrdn8SExOJiIj40zV9fHzw8fFxdakiJWbPmVTGf72Tg4nmbs/AZpFMH9qEqpUd/D5PPgbzxsDx1ea4RjsY8haExbqpYhEREZGyz+Xhp3fv3uzevfuKubvuuovY2FgmTZpEdHQ0Xl5eLF26lBEjRgBw8OBBTp48SadOnVxdjkipkldg561l8by9wtztCa3kzXPDmtK/WaRjC9jtsPlDWPI05GeBl7/Zxa3D/WD1cGvtIiIiImWdy8NPQEAATZs2vWKuUqVKhIaGFs7fc889jBs3jpCQEAIDAxkzZgydOnVSpzcp1/aeNXd7fj3bM7B5JM8MaUKoo7s9F4+Yuz0nLreEr90NhrwJITFuqlhERESkfHHrPT//zeuvv47VamXEiBHk5ubSt29f3nnnnZIoRcTt8grsvL38MG8vP0yB3SCkkjfPDm3KwOZO7PZseh+WTIeCbPCqBNc/A23uBqvb7ykWERERKTcshmEYJV2Es9LS0ggKCiI1NZXAwMCSLkfkv9p3No0J3+xk3zmzPXv/phE8O6yp42d7LhyGuaPg1AZzHNPdPNtTpZabKhYREREpW5zJBiWy8yNS3uXb7Ly74gizl8ZTYDcI9vfiGWc6udltsOFdWPYsFOSAd2W4/jlocyc48n4RERER+QOFHxEXO5Bg7vbsOWPu9lzfOJznbmhKWICDrd8vxMOch+D0JnNcpxcMmQ3BNd1UsYiIiEjFoPAj4iIFNjvvrTzCG0vjybcZBPl58czQJgxpEeX4bs/6t2H585d3ewKg7/PQ+nbt9oiIiIi4gMKPiAscSkxnwjc72XU6FYC4RuG8cENTwgId3O05f9A823N6szmuFweD34CgGm6qWERERKTiUfgRKYYCm50PVh9l1uJ48mx2An09mT60CcNaVndst8dWAOvfhOUzwJYLPkHQ7wVoOVK7PSIiIiIupvAjUkSHk9IZ/80udp5KAeC62DBmDG9GuKO7PUn7zbM9Z7eZ4/rXw6BZEFTdLfWKiIiIVHQKPyJOstkN/rH6KK8uPkRegZ0AX0+eGtyEEa2d2O1Z9waseBFseeZuT/8XocVftNsjIiIi4kYKPyJOOHI+gwnf7GT7yRQAejasxovDmxMR5OBuT+Jec7fn3A5z3KCfudsT6OCFpyIiIiJSZAo/Ig6w2Q3+b80xXvnlILkFdgJ8PJk6qDE3ta3h4G5PPqyZBStfAns++AZD/5nQ/Gbt9oiIiIhcIwo/Ildx7EImE77ZydYTlwDoVr8qL41oTlSwn2MLJOyBOQ9Cwi5z3HAADHodAiLcVLGIiIiI/BmFH5H/wm43+GTdcWYuOkBOvp3KPp48ObARt7SLdny3Z/VrsOplc7fHrwr0fxma3ajdHhEREZESoPAj8ieOX8hk4re72HQ8GYAu9UJ5aURzalTxd2yBc7tg7kOQsNscxw6Cga9BQLibKhYRERGRq1H4Efkdu93gn+uP89LCg2Tn2/D39uDxAY0Y2aGmY7s9BXmw+lVY/QrYC8AvBAa8DE1HaLdHREREpIQp/Ihcdio5iwnf7GTjMXO3p1OdUGbe2JzoEEd3e3aandwS95jjRoPN3Z7KYW6qWEREREScofAjFZ5hGHy5+RTPzt9HVp4NPy8PHh8Qy8gOtbBaHdztWfWyueNj2MA/FAa8Ak1u0G6PiIiISCmi8CMVWlJaDpO+28Xyg+cBaF87hFduakHNUAd3exJ2ww8P/Lbb03goDHgVKldzU8UiIiIiUlQKP1Jhzd91lifn7CElKx9vDyuP9W3I3V1j8HBkt8dWAGtnwYoXL3dyC4GBr0LT4W6vW0RERESKRuFHKpyUrDymzd3LvJ1nAWgSFcjrt7SkQXiAYwucPwRzHoAzW81xw4EweJbO9oiIiIiUcgo/UqGsPHSeid/uJDEtFw+rhVE96zL6uvp4e1qv/ma7HTa+C0ufgYIc8AmCATOh+S062yMiIiJSBij8SIWQmVvACwv28/nGkwDUqVaJ125uScvoYMcWuHTc7OR2Yq05rnsdDHkLgqq7pV4RERERcT2FHyn3thxPZvw3OzlxMQuAu7rUZmLfWPy8Pa7+ZsOArZ/AoicgPxO8KkHf56DNXdrtERERESljFH6k3MotsPH64ng+WHUEuwFRQb68clMLOter6tgCqWdg3hg4stQc1+oCQ9+GkBj3FS0iIiIibqPwI+XSvrNpjPt6BwcS0gEY0boGTw1pTKCv19XfbBiw6ytYMBFyU8HTF3o/BR0eAKsDZ4NEREREpFRS+JFypcBm5/1VR5m15BD5NoPQSt68MLwZfZtEOLZARhLMfxQOzDfH1dvAsPegWgP3FS0iIiIi14TCj5QbR89nMP6bnWw/mQLA9Y3DeWF4M6pW9nFsgX1zzeCTdRGsXtBzMnQZCx76z0RERESkPNBPdVLm2e0Gn64/zksLD5CTbyfAx5OnhjRhROvqWBxpSpCVDAsmwJ7vzHF4M7jhXYho5t7CRUREROSaUviRMu1UchaPfbuTDUeTAehWvyovjWhOVLCfYwscWAA/PgKZSWDxgK6PQo9J4OntxqpFREREpCQo/EiZZBgGX285xbPz95ORW4CflwePD2zE3zrUdGy3J/sS/DwZdn1pjqvFwrB3zDM+IiIiIlIuKfxImZOYlsPk73ax/OB5ANrVrsIrN7WgVmglxxY49Av8+DCknwOLFTo/DD2ngJevG6sWERERkZKm8CNlhmEY/LjrHFPn7CE1Ox9vTyuPXd+Qu7vG4GF1YLcnJxUWPQ7bPzPHofVg2LsQ3d69hYuIiIhIqaDwI2VCcmYeU+fs4afd5wBoVj2IV29uQYPwAMcWOLIM5o6BtNOABTo+BL2ngpeDZ4NEREREpMxT+JFSb/G+RKZ8v4sLGXl4Wi2Mua4+D/Wqi5eHAxeO5qbDL1Nh68fmuEqMebanVmf3Fi0iIiIipY7Cj5RaaTn5PPPjPr7dehqABuGVee3mljStHuTYAsdWwdxRkHLSHLe/D+KeBm8HzwaJiIiISLmi8COl0rrDF5jwzU7OpuZgscB93eswrk8DfDw9rv7mvExY8jRs+sAcB9eEoW9DTHe31iwiIiIipZvCj5QquQU2Xll0kA9XHwOgdqg/r97cgja1Qhxb4NRm+OE+SD5qjtvcBdc/Cz4Ong0SERERkXJL4UdKjUOJ6Tz87+0cSEgHYGSHmjwxsBH+3g58m9ryYeVMWP0KGHYIrA5D3oR6vd1ctYiIiIiUFQo/UuLsdoNP1x9nxs8HyCuwE1rJm5dGNCeucbhjC1yIh+/vg7PbzHGzm2HAy+AX7LaaRURERKTsUfiREpWUlsOEb3ex6pB5YWmvhtWYeWMLqgX4XP3NhgFbPoJFT0JBNvgGwaDXoekIN1ctIiIiImWRwo+UmIV7Epjy/S4uZeXj42nlyYGN+FvHWlgsDlxYmp5odnI7vNgcx/QwLywNqu7eokVERESkzFL4kWsuM7eAZ37cx1dbTgHQJCqQN25tSb0wB5sS7P8R5j0M2cng4QN9pkP7+8HqwL0/IiIiIlJhKfzINbX95CXGfrWDExezsFjg/u51GdenAd6eDl5Y+vNk2PGZOY5oBsM/hLBG7i1aRERERMoFhR+5Jgpsdt5efoTZy+Kx2Q2ignx57ZaWdKwT6tgCJzeYTQ1STgAW6PII9HocPB04GyQiIiIigsKPXAMnL2Yx9qvtbDuZAsCQFlE8O6wpQX5eV39zQR6sfBHWvG62sA6qCTe8B7W7uLdoERERESl3FH7EbQzD4Jutp5k+by+ZeTYCfDx57oamDG3pYFOC8wfh+3vh3E5z3OKv0P9Fs6ubiIiIiIiTFH7ELc6n5zLl+90s2Z8IQPuYEF67uQU1qvhf/c12O2x6H5Y8DQU54FcFBr8BjYe6t2gRERERKdcUfsTlFu45x+M/7CE5Mw9vDyuP9mnAfd3r4GF1oIX1peMwZxScWGOO6/aGoW9DYKRbaxYRERGR8k/hR1wmNTuf6fP28v32MwDERgTw+i0taRQZePU3GwZs+xQWPQF5GeBVCa5/FtreDY7c+yMiIiIichUKP+ISa+Iv8Ni3OzmXmoPVAg/2rMsjvR1sYZ12FuaNgcNLzHHNzjDsbQip496iRURERKRCUfiRYsnOs/Hiz/v5dP0JAGqH+vPqzS1pU6vK1d9sGLDra/j5MchJNS8s7T0NOj4IVg83Vy4iIiIiFY0D/1veOTNmzKBdu3YEBAQQFhbGsGHDOHjw4BWvycnJYdSoUYSGhlK5cmVGjBhBYmKiq0sRN9t+8hIDZ68uDD63dazFgke6ORZ8Ms7D17fBD/eZwSeqFTywGjqPVvAREREREbdwefhZuXIlo0aNYsOGDSxevJj8/Hyuv/56MjMzC1/z6KOP8uOPP/LNN9+wcuVKzp49y/Dhw11dirhJXoGdVxYdZMS76zh6IZOIQF/+eXd7nh3WFH9vBzYT982DdzrC/h/B6gm9noR7lkC1hu4vXkREREQqLIthGIY7f4Pz588TFhbGypUr6d69O6mpqVSrVo0vvviCG2+8EYADBw7QqFEj1q9fT8eOHa+6ZlpaGkFBQaSmphIY6MBhenGZgwnpPPrVDvadSwNgWMsopg9pSpC/AxeWZl+CBRNh99fmOKwJ3PAuRLZwY8UiIiIiUp45kw3cfuYnNTUVgJCQEAC2bt1Kfn4+cXFxha+JjY2lZs2a/zX85ObmkpubWzhOS0tzc9Xyn2x2g3+sPsqrvxwiz2anir8Xzw1rxsDmDragjl8C80ZD+jmwWKHLWOg5GTx93Fq3iIiIiMiv3Bp+7HY7Y8eOpUuXLjRt2hSAhIQEvL29CQ4OvuK14eHhJCQk/Ok6M2bMYPr06e4sVf6HMynZPPrlDjYdTwagd2wYM0Y0IyzA9+pvzk2HX56ErZ+Y49B6MOw9iG7nvoJFRERERP6EW8PPqFGj2LNnD2vWrCnWOlOmTGHcuHGF47S0NKKjo4tbnjhg4Z5zTPx2F2k5BVTy9mDa4Mbc3DYaiyN375zeCt/dbV5cCtDhQbObm7e/W2sWEREREfkzbgs/o0ePZv78+axatYoaNWoUzkdERJCXl0dKSsoVuz+JiYlERET86Vo+Pj74+OjxqGspJ9/Gs/P38fnGkwC0iA7mzVtbUTPUgeBit8PaWbD8ebAXQFA0DHsHYrq7t2gRERERkf/B5eHHMAzGjBnDDz/8wIoVK4iJibni623atMHLy4ulS5cyYsQIAA4ePMjJkyfp1KmTq8uRIjiUmM6YL7ZzMDEdgPt71GHC9Q3x8nDkwtJz8MP9cGylOW48DAa/AX7BbqtXRERERMQRLg8/o0aN4osvvmDu3LkEBAQUnuMJCgrCz8+PoKAg7rnnHsaNG0dISAiBgYGMGTOGTp06OdTpTdzHMAy+3HyK6T/uJSffTtXK3rx2c0u6N6jm2AIHf4Y5D0F2Mnj5Q/+XoNVt4MgjciIiIiIibubyVtf/7SzIxx9/zJ133gmYl5yOHz+ef//73+Tm5tK3b1/eeeed//rY239Sq2vXS83O5/Hvd/PT7nMAdKtflddubkm1AAceN8zPgcXTYNP75jiiGYz4P6jWwI0Vi4iIiIg4lw3cfs+POyj8uNbWE5d45MvtnL6UjafVwoS+DbmvWx2sVgd2bJIOwHf3QOIec9xxFMQ9pRbWIiIiInJNlKp7fqT0stsN3l15hNcWH8JmN4gO8WP2ra1oVbPK1d9sGLD1Y1j4OBRkg39VuOE9qN/H/YWLiIiIiBSBwk8FlZSWw6Nf72Dt4YsADG4RxfM3NCXQ1+vqb85Khh8fhv0/muM6veCG9yEg3I0Vi4iIiIgUj8JPBbTiYBLjv97Jxcw8/Lw8mD6kCTe1reHY3T3H18L390LaGbB6mff2dBoNVgc6wYmIiIiIlCCFnwokr8DOy4sO8OHqYwDERgTw1l9bUS8s4OpvthXAqpmw6mUw7BBSF278CKJaublqERERERHXUPipIA4kpDHuq53sO5cGwO2davH4gEb4enlc/c3JR+GHB+HUBnPcciT0nwk+ld1YsYiIiIiIayn8lHM2u8EHq47y2uKD5NsMqvh78eKI5vRt4kBbcbsdtnxktrHOzwKfQBj0OjS70f2Fi4iIiIi4mMJPOXb8Qibjv9nJ1hOXAIhrFMYLw5sRFuB79TennIK5o+DYSnNcuxsMfRuq1HJjxSIiIiIi7qPwUw4ZhsFnG07wwoIDZOfbqOzjybTBjbmpjQNNDQwDtn8GC6dAXjp4+kHc09D+PjU1EBEREZEyTeGnnDmXms3Eb3exOv4CAB3rhPDKTS2oUcX/6m9OO2e2sI7/xRzXaA/D3oWq9dxYsYiIiIjItaHwU04YhsEP28/w1Ly9pOcU4ONpZVK/WO7sXBur1YHdnt3fwoIJkJMCHt5w3ZOXW1g70BBBRERERKQMUPgpBy5m5PLED3tYuDcBgBbRwbx6UwvqhTnQjS3jPPz06G8Xlka2hBveg7BG7itYRERERKQEKPyUcb/sTeDxH3ZzISMPT6uFR3rX58GedfH0cOB8zr65MH8cZF0Aqyf0mARdHwUPL/cXLiIiIiJyjSn8lFFpOflMn7eP77adBqBheACv3tyCptWDrv7mrGT4eSLs/sYchzWBG96FyBZurFhEREREpGQp/JRB6w5fYMI3OzmbmoPFAvd1r8O4Pg3w8XTgfM6hRTDvYchIAIvV3OnpMQk8fdxfuIiIiIhICVL4KUPsdoO3lh/m9SWHMAyoGeLPqze3oF3tkKu/uSAXFj0Om/9hjkPrm2d7arR1b9EiIiIiIqWEwk8ZcSkzj7Ff7WDlofMA3Ny2Bk8NbkIlHwf+FV46Dt/cCWe3m+OOo6D3VPDyc1u9IiIiIiKljcJPGbDzVAoPfb6NMynZ+HhaeW5YU25qG+3Ymw/+DD/cDzmp4FcFhn8I9fu4t2ARERERkVJI4acUMwyDzzae5Nkf95Fns1M71J93RrahcVTg1d9sK4Blz8DaN8xx9bZw0ycQ7GBoEhEREREpZxR+SqmsvAIe/343c3acBaBvk3BevqkFgb4OtKFOOwff3g0n15njDg9Cn2fA09uNFYuIiIiIlG4KP6XQ4aQMHvxsK/FJGXhYLUzuF8vfu8VgsViu/uajK+C7v0PmefAOgKFvQZNh7i5ZRERERKTUU/gpZebvOsukb3eRmWejWoAPb/2lFR3qhF79jXY7rH4Flr8AGBDeFG7+J4TWdXvNIiIiIiJlgcJPKZFXYOeFBfv5ZN1xADrWCWH2X1oRFuB79TdnXoTv74UjS81xq9tgwMvq5iYiIiIi8jsKP6XAudRsRn2+jW0nUwB4sGddxvdpgKeH9epvPrXJbGOddgY8/WDQa9Dyr26tV0RERESkLFL4KWFr4i/w8JfbSc7MI8DXk9dubkmfxuFXf6NhwIZ3YPE0sBdAaD3zMbfwJu4vWkRERESkDFL4KSE2u8E7yw/z2pJDGAY0iQrk3ZFtqBnqf/U356TC3FGw/0dz3GQ4DJkNPgHuLVpEREREpAxT+CkBR85nMPHbXWw9cQmAW9tF8/SQJvh6eVz9zUdXwLwxkHISrF7Qbwa0+zs40glORERERKQCU/i5hmx2g/9bc4xXfjlIboGdyj6ePD2kCTe2qXH1N+ekmo+4bf3EHAfXhBs/gRpt3FmyiIiIiEi5ofBzjRxOyuCxb3ey/XJTg271q/LiiOZUD3agI1v8YvjxEbOpAUC7eyHuKT3mJiIiIiLiBIUfN7PZDf6x+iivLj5EXoGdAB9PnhzUiJvbRl/90tKsZFj0OOz8tzmuEmNeWlq7q/sLFxEREREpZxR+3OhwUjoTvtnFjlMpAPRoUI0Zw5sR5chuz/758NM4yEgELNBpFPR6ArwdaIggIiIiIiJ/oPDjBgU2Ox+uPsbrSy7v9vh6MnVQY25qU+Pquz2ZF2DBY7D3e3NctQEMfRui27u/cBERERGRckzhx8UOJabz2Dc72Xk6FYBeDavxwvBmRAZdZbfHMGDPd/DzRMi6CBYP6PII9JgEXr7XoHIRERERkfJN4cdFCmx23l91lDeWxJNnM3d7nhrchBGtq199tyc9AX4aDwfmm+PwpubZnqhW7i9cRERERKSCUPhxgQMJaTz2zS52nzF3e3rHhvH8Dc2ICLrKjo1hmM0MFk42W1lbPaH7Y9B1HHh6X4PKRUREREQqDoWfYrDZDd5ZfpjZy+LJtxkE+pr39tzQyoHdnswL8MMDcHixOY5saZ7tiWjq9rpFRERERCoihZ9isADrjlwk32YQ1yicF25oSligA+dzEvbAv/8CqSfBwwd6TobOD4OH/nWIiIiIiLiLftouBqvVwswbm7P1xCWGtoy6+m4PmC2sv78P8jMhpA7c+gWENXJ/sSIiIiIiFZzCTzFFh/gTHeLA3TuGAatfgWXPmeOYHnDTJ+Af4tb6RERERETEpPBzLeRnw9xRZitrgPb3Qd8XwMOrZOsSEREREalAFH7cLe0sfPlXOLvd7OY24GVoe3dJVyUiIiIiUuEo/LjT6a1m8MlIAL8QuPmfENOtpKsSEREREamQFH7cZdc35qNutlyo1gj+8m8IiSnpqkREREREKiyFH1ez22HZM7DmdXPcoD8M/wB8A0u2LhERERGRCk7hx5Vy08021gcXmOOuj8J1U8HqUbJ1iYiIiIiIwo/LXDpuXlyatM+8uHTIm9DilpKuSkRERERELlP4cYXja+Cr2yA7GSqHmxeX1mhb0lWJiIiIiMjvKPwU15aPYcEEsBdAZEsz+ARVL+mqRERERETkPyj8FEdBHmz5PzP4NBkOQ98Gb/+SrkpERERERP6Ewk9xeHqbOz17f4DOY8BiKemKRERERETkv1D4Ka7gaOjycElXISIiIiIiV2Etyd/87bffpnbt2vj6+tKhQwc2bdpUkuWIiIiIiEg5VmLh56uvvmLcuHE89dRTbNu2jRYtWtC3b1+SkpJKqiQRERERESnHLIZhGCXxG3fo0IF27drx1ltvAWC324mOjmbMmDFMnjz5itfm5uaSm5tbOE5LSyM6OprU1FQCAwOvad0iIiIiIlJ6pKWlERQU5FA2KJGdn7y8PLZu3UpcXNxvhVitxMXFsX79+j+8fsaMGQQFBRX+io6OvpblioiIiIhIOVAi4efChQvYbDbCw8OvmA8PDychIeEPr58yZQqpqamFv06dOnWtShURERERkXKiTHR78/HxwcfHp6TLEBERERGRMqxEdn6qVq2Kh4cHiYmJV8wnJiYSERFREiWJiIiIiEg5VyLhx9vbmzZt2rB06dLCObvdztKlS+nUqVNJlCQiIiIiIuVciT32Nm7cOO644w7atm1L+/btmTVrFpmZmdx1110lVZKIiIiIiJRjJRZ+brnlFs6fP8+0adNISEigZcuWLFy48A9NEERERERERFyhxO75KQ5nenmLiIiIiEj5Verv+REREREREbnWFH5ERERERKRCUPgREREREZEKQeFHREREREQqBIUfERERERGpEBR+RERERESkQiixe36K49fu3GlpaSVciYiIiIiIlKRfM4EjN/iUyfCTnp4OQHR0dAlXIiIiIiIipUF6ejpBQUH/8zVl8pJTu93O2bNnCQgIwGKxlGgtaWlpREdHc+rUKV24Kk7T948Uh75/pDj0/SNFpe8dKQ53fP8YhkF6ejpRUVFYrf/7VE+Z3PmxWq3UqFGjpMu4QmBgoP4AkCLT948Uh75/pDj0/SNFpe8dKQ5Xf/9cbcfnV2p4ICIiIiIiFYLCj4iIiIiIVAgKP8Xk4+PDU089hY+PT0mXImWQvn+kOPT9I8Wh7x8pKn3vSHGU9PdPmWx4ICIiIiIi4izt/IiIiIiISIWg8CMiIiIiIhWCwo+IiIiIiFQICj8iIiIiIlIhKPyIiIiIiEiFoPBTTG+//Ta1a9fG19eXDh06sGnTppIuSUqhVatWMXjwYKKiorBYLMyZM+eKrxuGwbRp04iMjMTPz4+4uDji4+NLplgpVWbMmEG7du0ICAggLCyMYcOGcfDgwStek5OTw6hRowgNDaVy5cqMGDGCxMTEEqpYSpN3332X5s2bF96k3qlTJ37++efCr+t7Rxz14osvYrFYGDt2bOGcvn/kf3n66aexWCxX/IqNjS38ekl9/yj8FMNXX33FuHHjeOqpp9i2bRstWrSgb9++JCUllXRpUsr8fzv3F9JUG8cB/PvmPGZm01VsLtlYZEaFgyZbIyJoUkhEf24kvBjVXcfQpJsuRILAqJsygoKgrkwyGFE3NcwOBCq2GKiUtBAMnEkXmo5M2X7vRXh4h6LmS++Z774fOHDO8zyM38WXB37snCeRSMDtduPu3buLzt+4cQOtra24d+8eent7UVBQgKNHj2JmZuY/rpQyjaZpUFUVPT09CIfDmJubw5EjR5BIJPQ1ly5dwvPnz9HR0QFN0zA6OorTp08bWDVlitLSUly/fh2RSATv3r3D4cOHceLECQwODgJgdmhl+vr6cP/+fVRUVKSNMz+0nD179iAej+vX27dv9TnD8iO0al6vV1RV1Z+TyaTY7XZpaWkxsCrKdAAkFArpz6lUSmw2m9y8eVMfm5iYkLy8PHn8+LEBFVImGx8fFwCiaZqI/MpKbm6udHR06Gs+fPggAKS7u9uoMimDFRcXy4MHD5gdWpGpqSkpKyuTcDgshw4dkvr6ehHh3kPLa25uFrfbveickfnhPz+rNDs7i0gkgqqqKn1s3bp1qKqqQnd3t4GV0VozPDyMsbGxtCyZzWb4fD5miRaYnJwEAFgsFgBAJBLB3NxcWn527doFh8PB/FCaZDKJ9vZ2JBIJ+P1+ZodWRFVVHDt2LC0nAPceWplPnz7Bbrdj+/btqK2txcjICABj82P6o7/+P/bt2zckk0lYrda0cavVio8fPxpUFa1FY2NjALBolubniAAglUqhoaEBBw4cwN69ewH8yo+iKCgqKkpby/zQvP7+fvj9fszMzGDjxo0IhULYvXs3otEos0NLam9vx/v379HX17dgjnsPLcfn8+HRo0coLy9HPB7H1atXcfDgQQwMDBiaHzY/RERrhKqqGBgYSHtnmmg55eXliEajmJycxNOnTxEMBqFpmtFlUYb78uUL6uvrEQ6HsX79eqPLoTWourpav6+oqIDP54PT6cSTJ0+Qn59vWF187W2VtmzZgpycnAWnUnz9+hU2m82gqmgtms8Ls0RLqaurw4sXL9DV1YXS0lJ93GazYXZ2FhMTE2nrmR+apygKduzYAY/Hg5aWFrjdbty+fZvZoSVFIhGMj49j3759MJlMMJlM0DQNra2tMJlMsFqtzA/9lqKiIuzcuROxWMzQ/YfNzyopigKPx4POzk59LJVKobOzE36/38DKaK1xuVyw2WxpWfr+/Tt6e3uZJYKIoK6uDqFQCK9fv4bL5Uqb93g8yM3NTcvP0NAQRkZGmB9aVCqVws+fP5kdWlIgEEB/fz+i0ah+VVZWora2Vr9nfuh3TE9P4/PnzygpKTF0/+Frb/9CY2MjgsEgKisr4fV6cevWLSQSCZw9e9bo0ijDTE9PIxaL6c/Dw8OIRqOwWCxwOBxoaGjAtWvXUFZWBpfLhaamJtjtdpw8edK4oikjqKqKtrY2PHv2DIWFhfq70GazGfn5+TCbzTh//jwaGxthsViwadMmXLx4EX6/H/v37ze4ejLalStXUF1dDYfDgampKbS1teHNmzd4+fIls0NLKiws1L8tnFdQUIDNmzfr48wPLeXy5cs4fvw4nE4nRkdH0dzcjJycHJw5c8bY/eePniWXBe7cuSMOh0MURRGv1ys9PT1Gl0QZqKurSwAsuILBoIj8Ou66qalJrFar5OXlSSAQkKGhIWOLpoywWG4AyMOHD/U1P378kAsXLkhxcbFs2LBBTp06JfF43LiiKWOcO3dOnE6nKIoiW7dulUAgIK9evdLnmR36Hf886lqE+aGl1dTUSElJiSiKItu2bZOamhqJxWL6vFH5+UtE5M+2V0RERERERMbjNz9ERERERJQV2PwQEREREVFWYPNDRERERERZgc0PERERERFlBTY/RERERESUFdj8EBERERFRVmDzQ0REREREWYHNDxERERERZQU2P0RERERElBXY/BARERERUVZg80NERERERFnhb80PxAKsbd+BAAAAAElFTkSuQmCC", "text/plain": [ "
" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "# Transition probabilities\n", "alpha = 0.3\n", "beta = 0.2\n", "\n", "# Discount parameter\n", "gamma = 0.99\n", "\n", "# Expected rewards\n", "r_search = 6.0\n", "r_wait = 2.0\n", "\n", "V = np.zeros(nb_states)\n", "Q = np.zeros((nb_states, nb_actions))\n", "\n", "pi = np.array([a['search'], a['search']], dtype=int)\n", "\n", "V_high_history = []\n", "V_low_history = []\n", "\n", "for k in range(50):\n", " \n", " # Policy evaluation\n", " Q[s['high'], a['search']] = alpha * (r_search + gamma * V[s['high']]) + (1 - alpha) * (r_search + gamma*V[s['low']])\n", " Q[s['high'], a['wait']] = r_wait + gamma * V[s['high']]\n", "\n", " Q[s['low'], a['search']] = beta * (r_search + gamma * V[s['low']]) + (1 - beta) * (-3 + gamma*V[s['high']])\n", " Q[s['low'], a['wait']] = r_wait + gamma * V[s['low']]\n", " Q[s['low'], a['recharge']] = gamma * V[s['high']]\n", "\n", " V[s['high']] = Q[s['high'], :2].max()\n", " V[s['low']] = Q[s['low'], :].max()\n", "\n", " V_high_history.append(V[s['high']])\n", " V_low_history.append(V[s['low']])\n", " \n", " # Compute the greedy policy\n", " pi_old = pi.copy()\n", " pi[s['high']] = Q[s['high'], :2].argmax()\n", " pi[s['low']] = Q[s['low'], :].argmax()\n", " \n", "print('Greedy policy after iteration', k)\n", "print('pi(high)=', pi[s['high']])\n", "print('pi(low)=', pi[s['low']])\n", "print(\"V=\", V)\n", "print(\"Q=\", Q)\n", " \n", "plt.figure(figsize=(10, 6)) \n", "plt.plot(V_high_history, label=\"high\")\n", "plt.plot(V_low_history, label=\"low\")\n", "plt.legend()\n", "plt.show()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "**A:** The optimal policy stays the same (search in high, recharge in low) but the V values grow very high. The difference between the values of the high and low state is comparatively very small: the high state is always only one action away from the low state, it is nothing with such a high gamma." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "**Q:** Change the parameters to:\n", "\n", "$$\\alpha = 0.01 \\quad \\beta = 0.2 \\quad \\gamma = 0.7 \\quad \\mathcal{R}^{\\text{search}} = 6 \\quad \\mathcal{R}^{\\text{wait}} = 5$$\n", "\n", "Find the optimal policy. What is the optimal action to be taken in the state *high*, although the probability to stay in this state is very small? Why?" ] }, { "cell_type": "code", "execution_count": 11, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Greedy policy after iteration 49\n", "pi(high)= 0\n", "pi(low)= 1\n", "V= [17.67371571 16.66666637]\n", "Q= [[17.67371571 17.37160091 0. ]\n", " [11.030614 16.66666637 12.37160091]]\n" ] }, { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAzYAAAH5CAYAAABTbqsJAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjkuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8hTgPZAAAACXBIWXMAAA9hAAAPYQGoP6dpAABOjklEQVR4nO3dd5xU5d3///f07bsssA12YUWkKE0pYgMid8yqiJrY4m2IescUEjUkthhLisGSeBONP/VO7lh+sURNIGpi1FsFVIoUUUFYAalbWNr23dnZmfP9Y3Zme5+ZM7P7ej4e85gz55w58wFG3Dfnuj6XxTAMQwAAAAAQw6xmFwAAAAAA/UWwAQAAABDzCDYAAAAAYh7BBgAAAEDMI9gAAAAAiHkEGwAAAAAxj2ADAAAAIObZzS6gLZ/Pp+LiYiUnJ8tisZhdDgAAAACTGIahqqoq5eTkyGrt+p5M1AWb4uJi5ebmml0GAAAAgChx4MABjRw5sstzoi7YJCcnS/IXn5KSYnI1AAAAAMxSWVmp3NzcYEboStQFm8Dws5SUFIINAAAAgB5NUaF5AAAAAICYR7ABAAAAEPMINgAAAABiXtTNsekpr9crj8djdhkxxeFwyGazmV0GAAAAEHK9DjarV6/WQw89pE2bNqmkpETLly/XxRdfHDxeXV2t22+/XStWrNDRo0eVn5+vG2+8Ud/73vdCUrBhGCotLVV5eXlIrjfYpKWlKSsrizWCAAAAMKD0OtjU1NRoypQpuu6663TppZe2O75kyRK9++67+stf/qLRo0frrbfe0g9+8APl5OTooosu6nfBgVCTkZGhhIQEfkDvIcMwVFtbq7KyMklSdna2yRUBAAAAodPrYFNQUKCCgoJOj69Zs0aLFi3S3LlzJUk33HCDnnzySX300UcdBhu32y232x18XVlZ2em1vV5vMNQMHTq0t6UPevHx8ZKksrIyZWRkMCwNAAAAA0bImwecccYZevXVV1VUVCTDMPTee+/piy++0Fe/+tUOz1+6dKlSU1ODj9zc3E6vHZhTk5CQEOqyB43A7x3zkwAAADCQhDzYPProo5o4caJGjhwpp9Opr33ta3rsscd0zjnndHj+HXfcoYqKiuDjwIED3X4Gw8/6jt87AAAADEQh74r26KOPat26dXr11Vc1atQorV69WosXL1ZOTo7mz5/f7nyXyyWXyxXqMgAAAAAMIiG9Y1NXV6ef/exnevjhh7VgwQJNnjxZP/zhD3XFFVfot7/9bSg/KubMnTtXN998c6fHLRaLVqxY0ePrrVy5UhaLhe5wAAAAgEJ8x8bj8cjj8chqbZ2XbDabfD5fKD9qwCkpKdGQIUPMLgMAAACISb0ONtXV1dq1a1fw9Z49e7Rlyxalp6crLy9Pc+bM0S233KL4+HiNGjVKq1at0rPPPquHH344pIUPNFlZWWaXAAAAAMSsXg9F27hxo6ZNm6Zp06ZJ8q9bM23aNN19992SpBdffFEzZszQ1VdfrYkTJ+r+++/XfffdF7IFOmOZz+fTrbfeqvT0dGVlZenee+8NHms7FG3NmjWaOnWq4uLiNH36dK1YsUIWi0Vbtmxpdc1NmzZp+vTpSkhI0BlnnKHCwsLI/GIAAACAKNLrOzZz586VYRidHs/KytJTTz3Vr6J6wzAM1Xm8Efu8luIdtl51GXvmmWe0ZMkSrV+/XmvXrtW3v/1tnXnmmfqP//iPVudVVlZqwYIFOv/88/X8889r3759nc7PufPOO/W73/1Ow4cP1/e+9z1dd911+vDDD/vzywIAAABiTsi7okVanceriXe/acpnf/7L85Tg7Plv4eTJk3XPPfdIksaOHas//OEPeuedd9oFm+eff14Wi0V//OMfFRcXp4kTJ6qoqEjf+c532l3zvvvu05w5cyRJt99+uy644ALV19crLi6uH78yAAAAILaEfB0bdG7y5MmtXmdnZ6usrKzdeYWFhZo8eXKrcDJz5sxur5mdnS1JHV4TAAAAGMhi/o5NvMOmz395nmmf3RsOh6PVa4vF0u9ucS2vGRgWRwc6oG8Mw5DHa6jR5/M/e33y+gx5fM3bXp8hr2E0b7d8dLLfZ0hew5BhGPIZhrw+ydf0uvW2/1xf03mBbcPw12YYat6nlvv85xpqPtdnyL/dYp//Pc37fIak4PEW57Y4z/8bo+DnNb0Mnhf4fQsOUG7xvsB1mz6lxbaCx5tfNZ/beq9aDX9uvb/j89u+p/2fcwf72l2h/XldXLLTa/T0vZ1fsx/69ea+fFyEPxBh09fvKwaen10wQafmxU7X3pgPNhaLpVfDwWLBuHHj9Je//EVutzu4eOmGDRtMrgoIH6/PP1eu3uNVXUPTc2C70dfBPq/cHp8avD41NLZ4eNs8d7Ld6GsKLE3hJRBcfPzPHACAoIo6j9kl9MrASgQDxDe/+U3deeeduuGGG3T77bdr//79wQVOe9OsAAi3eo9Xx2sbVFnXqGq3R1X1jap2N6q66bnda3ejqus9rfbVNwWUaGW1SHabVXarRTarJfhstfi3rU2vbVaLbJYW203nNG8r+NpiscjW9Npischm9W9bLf7rWYPHAvubX1uaXlvU9NzmPIvFIoua91vU4r2S1HS85b7AdRXYVutjUvO+gObPCRxvfY3AvsALS/B97a/X8m+1lsfbHmy+hqXdvpbv7ei1RZZOj3Wl7d+53b2tq+u2raGn+vPXfiz9H4P/vQHR5+ScFLNL6BWCTRRKSUnRa6+9pu9///uaOnWqJk2apLvvvlvf/OY3aQqAsPF4fSqv9eh4bYOO1TToeE2DjtX6n48GX3v8zzUNOl7boNqG0HckjHfYFO+0Kc5uVZzT5n/tsCmu6RE45nJY5bTZ5LRb5bRb5bJb5bBZ5LRZ5bQ373fa/McCrx1NIcVus8hu9b/HbrPKEQgutqZ9VmswuAAAgOhHsImQlStXttvXct2atuPBzzjjDH3yySfB188995wcDofy8vIkddx2e+rUqV2OK8fg5fH6dKiyXqUV9SquqFdpRZ2Ky+tVUlEX3Hek2t2ncdV2q0Wp8Q4lx9mVFGdXksuuJFfTa1fzvuDrpn3JLocSXDYlOJuDi8tu5a4kAADoE4JNlHr22Wd1wgknaMSIEfrkk09022236fLLL1d8fLzZpSEKNXp92nesVrvLqrXvaK2KWwSWkvI6He5haLFYpLR4h4YkOpWe4NSQRKeGJjpbvU5PdGhIglPpTfuTXXbCCAAAMB3BJkqVlpbq7rvvVmlpqbKzs3XZZZfpvvvuM7ssmKy2oVFfHq7RrrJq7T5crV1l/sfeozXyeLtOLg6bRVmpccpOjVd2q2f/dlZqnNITnbIx9AoAAMQggk2UuvXWW3XrrbeaXQZMUlHrUeGhqmBwCYSYovK6Tt8T77BpTEai8oclKSctTjlNYSXwPDTRyXwRAAAwYBFsAJMZhqHdh2u0ed9xbdx3TJv2HdfuwzWdnp+e6NSJw5M0JiNJJzY9xgxPVE5qPMEFAAAMWgQbIMLqGrz69GC5Nu47rs37jmvT/uMqr23fJ35EWnwwuPjDi/85PdFpQtUAAADRjWADhNmhynpt3Htcm/Yd16Z9x7StuFKNbVaCdNmtmpKbptNGDdFpeUN06qghBBgAAIBeINgAIdbo9emjPcf0xtZSvbujrMN5MRnJLk0fPUSnjUrXaaOGaGJ2ipx2qwnVAgAADAwEGyAEPF6f1uw+qjc+K9Fbnx/SsZqG4DGrRZqQneK/G9P0GJEWT4tkAACAECLYAH1U7/Hqg51H9MbWUr39eakq6xuDx9ISHPrqxEx97ZQszcwfqiQX/6kBAACEEz9tRcjcuXM1depULVu2zOxS0A91DV6tLCwLDjOrdjeHmWFJLp13cqYKTsnWrBPS5bAxtAwAACBSCDZAN2rcjXpnR5ne+KxEKwsPq87jDR7LSonT107JUsEpWZo+Op3FLQEAAExCsAE6UVZVr6c+3Ku/rNunqhbDzEYOidf5k7L1tVOyNHVkGmvHAAAARAHGypjg+PHj+ta3vqUhQ4YoISFBBQUF2rlzpyT/Yo3Dhw/XK6+8Ejx/6tSpys7ODr7+4IMP5HK5VFtbG/HaB4M9R2p0x98/01kPvKfHV+5WVX2jRg1N0OJ5Y/T6j87S+7fO08/On6BT84YQagAAAKJE7N+xMQzJY9IP+I4EqQ+drb797W9r586devXVV5WSkqLbbrtN559/vj7//HM5HA6dc845Wrlypb7xjW/o+PHj2r59u+Lj47Vjxw6NHz9eq1at0owZM5SQkBCGX9Tg9enBcj2xarfe2Foqo2mZmVPz0vS9OWM0f0ImIQYAACCKxX6w8dRKv8kx57N/Viw5E3v1lkCg+fDDD3XGGWdIkp577jnl5uZqxYoVuuyyyzR37lw9+eSTkqTVq1dr2rRpysrK0sqVKzV+/HitXLlSc+bMCfkvZzAyDEPv7zyiJ1bt1prdR4P7vzI+Q9+bM0YzRg+hLTMAAEAMiP1gE2O2b98uu92uWbNmBfcNHTpU48aN0/bt2yVJc+bM0U033aTDhw9r1apVmjt3bjDYXH/99VqzZo1uvfVWs34JA0Kj16c3tpbqiVW7ta24UpJks1q0cEqObphzgsZnpZhcIQAAAHoj9oONI8F/58Sszw6DSZMmKT09XatWrdKqVat03333KSsrSw888IA2bNggj8cTvNuD3qn3ePXyxgP64/t7tP+YfwhjvMOmK2fm6vqz8jVyCMP7AAAAYlHsBxuLpdfDwcw0YcIENTY2av369cFwcvToURUWFmrixImSJIvForPPPlv/+Mc/tG3bNp111llKSEiQ2+3Wk08+qenTpysxMXZ+zdGg3uPV/36wR3/+YI+O1jRIkoYkOLTojNFaNHu0hiQ6Ta4QAAAA/RH7wSbGjB07VgsXLtR3vvMdPfnkk0pOTtbtt9+uESNGaOHChcHz5s6dq5/85CeaPn26kpKSJEnnnHOOnnvuOd1yyy1mlR+TNu8/rlte/kS7D9dIkkakxes7Z+fr8hm5SnDynwAAAMBAQLtnEzz11FM67bTTdOGFF2r27NkyDEP/+te/5HA4gufMmTNHXq9Xc+fODe6bO3duu33oXL3Hq9/8a7u+8fga7T5co+HJLj18+RStvGWuvn1mPqEGAABgALEYRqCxbXSorKxUamqqKioqlJLSegJ3fX299uzZo/z8fMXFxZlUYWwbLL+Hm/Yd0y0vf6ovj/jv0lx66gjdfeFEpSUw5AwAACBWdJUN2uKfrDGg1DV49bu3CvW/H+6RYUiZKS795pJJOndCptmlAQAAIIwINhgwNu49plte+VR7mu7SfOO0kbrrgolKTXB0804AAADEOoINYl5dg1cPvVmop9Y036W5/9LJmjc+w+zSAAAAECEEG8S0j/Yc062vfKK9R/1r0lw+faTuvGCiUuO5SwMAADCYEGwQk2obGvXgvwv1zNq9MgwpOzVOSy+dpLnjuEsDAAAwGMVksPH5fGaXELMGwu/d+i+P6ta/fap9TXdprpieqzsvnKCUOO7SAAAADFYxFWycTqesVquKi4s1fPhwOZ1OWSwWs8uKCYZhqKGhQYcPH5bVapXTGXttjw3D0LL/26nfv7NTkv8uzf1fn6w5Jw03uTIAAACYLaaCjdVqVX5+vkpKSlRcXGx2OTEpISFBeXl5slpja21Wn8/QL1//XE+v2StJunJGrn52AXdpEMMMQzJ8ks/rfzZ8kuFtsc9o3q8W2+32G53sb9oXOCd4bgf7un1Wx8cUeOroPLU4p6PXbY91cX63+zr5/W173S7f18G+dud1s+xbl8vC9XHJOFOWmjPhM6NrST0AASeeK6WONLuKHoupYCP579rk5eWpsbFRXq/X7HJiis1mk91uj7m7XF6foTv+/qle2nhQkvSrhSfrmtmjzS0KkWcYktcjeWqlxvqmR4Pkdbd4dkveBv8jsN3q2e2/htcj+Rr9D69H8nkkb9NrX4vjbc8zvE2vAwEksB3Y72tzjrfFuW0CDAAA0e7qVwg24WaxWORwOORw8K/1A11Do08/fmmL/vlpiawW6aFvTNHXT4ud/8AGLa9HcldJ7sqm55aPlvuq/UHFU9ccWALbwecW+wZlILBIFqtkCTxbO9hnab2vy+0Wzx3t6/RZbfY1vZY6Pq/l/uB2d8daHO/xfnWxv6v3dHCNDq9r6cE53Vy3R+8N0z849fkfsmLrH8AiLsb+gRDos8RhZlfQKzEZbDA41Hu8WvzcZr2zo0wOm0WPXDlNBZOyzS5rcPHUS7VH2zyOtd9XX946vDTWh7cui1Wyx0k2p2R3STaXZHf6Xwf3tT3W4hyrQ7LaJJvDv21zSFa7/xHYDhyz2iWbvfm41S5ZbJLV2mLb1rRtbbHdtN9ibT4WOG6x+o8HgklwX2B/4NzYGjIKAICZCDaISjXuRn3n2Y1as/uoXHarnrzmNFo5h4ph+MNIxUGpskiqLPZvV5VINUdahxdPTf8+y5EguZLbPFKat52JkiNRcsS3eSS0frbHtd5nc/AvpgAAoBWCDaJORa1H3376I328v1yJTpv+99szdPoJQ80uKzYYhj+UVB5sDizB8FLUvO119/yaVruUMLTFI73N66FSXJoUl9I6wDiT/Xc6AAAAIoCfOhBVjlS79a3//Uifl1QqNd6hZ66bqam5aWaXFX18Xun4XunIF9LhwhbPOyV3Rc+ukZQppeRIKSOaHjlSUkb7AONK4e4IAACIegQbRI2Sijr955/Wa/fhGg1Lcukv/zVT47NSzC7LXJ466eiu5vBy5Avp8Bf+fV3ddUkc3hxYUke02c6RknP8800AAAAGCIINosL+o7X65p/W6eDxOuWkxukv/zVLJwxPMrusyPLUScUfSwfWSwc2SGXbpOP71OmaEvY4aehYadhYafg4adhJ/uf0MZIjLqKlAwAAmI1gA9PtKqvS1X9ar0OVbo0amqDn/muWRg5JMLus8Ksq9YeY/ev9zyWf+NdQaSsurSm4jJWGjWsOMWl5/m5aAAAAINjAXFuLKvStP3+kYzUNOikzSX+5fpYyUgbg3QafVzq0reluTNOjfH/78xIzpLxZUu7pUvYUf4hJHM4cFwAAgG4QbGCaTfuO6dtPbVBVfaMmj0zVM9fO1JDEATLvwzCk4s3SF29JB9ZJBzdKDdVtTrJImadIuTOl3Fn+QJM2ihADAADQBwQbmOLDXUf0X89sVJ3Hqxmjh+jP356h5DiH2WX1j88nHfxI+vxVafurUsWB1sedydLI6VLe6f4wM2K6v0UyAAAA+o1gg4grLK3S9c9sUL3Hp7PHDtOT15ymBGeMfhW9jdL+NU1h5jWpurT5mCNRGjtfyj/HP7QsYwJzYgAAAMIkRn+aRKyqa/Dqh89vVr3Hp7NOHKY/LZoulz3Gftj3eqQ9q/xhZsfrUu3R5mOuFGlcgTThIunEcyVHvHl1AgAADCIEG0TUva9u086yamUku7TsyqmxE2o89dKX7/nDTOE/pfoWi2DGD5HGXyBNWCidMEeyu8yrEwAAYJAi2CBi/rGlSH/deEAWi7TsiqkalhQDAeBwobTmUWnbCqmhqnl/YoY04UL/nZnRZ0m2GJ8fBAAAEOMINoiIvUdqdOfyrZKkH807UWecOMzkirpx4CPpg2X+uzMByTnSxIv8YSbvdObLAAAARBGCDcLO3ejVj174WNXuRs0cna4bzx1rdkkdMwxp51v+QLN/TdNOi3+Y2ezF/gYAVquZFQIAAKATBBuE3QNvFOqzogqlJTj0+6umym6LsnDg9Uhb/yZ9+Hup7HP/PqtDmnKFdMZN0vCTzK0PAAAA3er1T5irV6/WggULlJOTI4vFohUrVrQ7Z/v27brooouUmpqqxMREzZgxQ/v3d7DKOga8//v8kP784R5J0m+/MUXZqVHUJayhRlr3uPTINGn5d/2hxpksnfEj6eZPpYWPEWoAAABiRK/v2NTU1GjKlCm67rrrdOmll7Y7vnv3bp111lm6/vrr9Ytf/EIpKSnatm2b4uLiQlIwYkdJRZ1++sonkqTrzszX/ImZJlfUpOao9NH/SB89KdUd9+9LzJBO/540/XopPs3U8gAAANB7vQ42BQUFKigo6PT4nXfeqfPPP18PPvhgcN+YMWP6Vh1iVqPXp5te2KLyWo8mjUjVbQXjzC5JKt8vrfmDtPlZqbHOv29IvnTmjdKUb0oOwjcAAECsCulkB5/Pp3/+85866aSTdN555ykjI0OzZs3qcLhagNvtVmVlZasHYt8j7+zUR3uPKcll16NXTTN3vZrGBum9pf4hZx896Q812VOly56WfrRJmn4doQYAACDGhTTYlJWVqbq6Wvfff7++9rWv6a233tIll1yiSy+9VKtWrerwPUuXLlVqamrwkZubG8qSYII1u47o0fd2SZLuu+QUjR6WaF4xJZ9If5wnrbpf8jVK+edI3/qHdMNK6eRLaNkMAAAwQIS0K5rP55MkLVy4UD/+8Y8lSVOnTtWaNWv0xBNPaM6cOe3ec8cdd2jJkiXB15WVlYSbGHak2q2b/rpFhiFdMT1XC6eOMKeQxgZp9UPSBw/7A018unTBb6WTL5UsFnNqAgAAQNiENNgMGzZMdrtdEydObLV/woQJ+uCDDzp8j8vlkssVAyvQo1s+n6ElL32iw1Vujc1I0r0XnWxOIcVbpBU/kMq2+V9PXCid/zspabg59QAAACDsQhpsnE6nZsyYocLCwlb7v/jiC40aNSqUH4Uo9Mf3v9TqLw7LZbfqD988VfHOCA/zanT779K8/7BkeKWEodIFv/MPOQMAAMCA1utgU11drV27dgVf79mzR1u2bFF6erry8vJ0yy236IorrtA555yjefPm6d///rdee+01rVy5MpR1I8ps3n9cD73pD7T3XnSyxmUlR7aAos3SPxY3L7B58iXS+b+VEodFtg4AAACYwmIYhtGbN6xcuVLz5s1rt3/RokV6+umnJUl//vOftXTpUh08eFDjxo3TL37xCy1cuLBH16+srFRqaqoqKiqUkpLSm9Jgkoo6jy545H0dPF6nCydn69GrpskSqXksjW5p1QPSB8ua7tIMa7pLc3FkPh8AAABh05ts0OtgE24Em9hiGIZ+8NxmvbG1VHnpCXr9xrOUEueIzIcXbfbPpTm83f/6lK9LBQ9JiUMj8/kAAAAIq95kg5DOscHg85f1+/XG1lI5bBY9etW0yIQaT72/ffOHj/jv0iQOly54WJp4Ufg/GwAAAFGJYIM+++JQlX71un9Oy21fG68puWnh/9CST6W/f0c6vMP/etJlUsGDUkJ6+D8bAAAAUYtggz5b+q/tamj0ad644br+rPzwf+C+NdJzl0sNVVJihnThf0sTLgz/5wIAACDqEWzQJ5v2HdN7hYdls1p0z4KTw98sYNc70otXS4110qizpCv+f+7SAAAAIIhggz753VtfSJIuO22kRg9LDO+HbX9deuVaydsgnfgf/lDjiA/vZwIAACCmWM0uALFnza4jWrP7qJw2q3507tjwftinL0kvfcsfaiYulK58nlADAACAdgg26BXDMPS7t/13a66amasRaWEMGRv/LP39Bn/nsynflL7+Z8nuDN/nAQAAIGYRbNArKwsPa9O+44pzWLV43onh+6A1j0qv/1iSIc34jrTwMcnGyEkAAAB0jJ8U0WOGYei3bxVKkhbNHq2MlLhwfIi08n7/OjWSdNaPpXPvkcLdnAAAAAAxjWCDHntzW6m2FVcq0WnTd+eMCf0HGIb01s+ltX/wvz73bunsn4T+cwAAADDgEGzQI16foYeb5tZcf1a+0hNDPNfF55X+uUTa9LT/dcGD0qzvhvYzAAAAMGARbNAjr39arC8OVSslzq7rzz4htBf3eqQV35c+e1myWKUFj0inXhPazwAAAMCARrBBtxq9Pv13092a784Zo9R4Rwgv7pZevlYq/KdktUuX/lE65dLQXR8AAACDAsEG3frb5oPae7RWQxOd+vYZo0N34YYa6cWrpS/fk2wu/8KbJ50XuusDAABg0CDYoEvuRq8eeWeXJOn7c8co0RWir0x9hfTc5dKBdZIjUfrmi1L+OaG5NgAAAAYdgg269NcNB1RUXqfMFJf+8/RRobmop0569mKpeLMUlypd/Tcpd0Zorg0AAIBBiWCDTtU1ePXou/67NT/8yljFOWyhufC/7/CHmoSh0jUrpOzJobkuAAAABi2r2QUgev1l3T4drnJrRFq8rpieG5qLblsubXpKkkX6+v8SagAAABASBBt0qNrdqMdX7ZYk3TR/rJz2EHxVju2RXr3Rv332EmnMvP5fEwAAABDBBp146oM9OlbToBOGJerSaSP6f8HGBumV6yR3pZR7ujT3Z/2/JgAAANCEYIN2Kmo9+p/3v5Qk3fwfJ8luC8HX5N1fNjULSJO+/ifJxvQuAAAAhA7BBu388f0vVVXfqHGZybpwUnb/L7jzbWnNo/7thY9JaSGarwMAAAA0IdiglaPVbv35wz2SpCVfPUlWq6V/F6wslpZ/178987vShAv7WSEAAADQHsEGrTyxardqG7yaNCJVX52Y2b+L+bzS32+Qao9KWZOlr/4qNEUCAAAAbRBsEHSosl7Prt0nSfrJV0+SxdLPuzWrH5L2vi85EqVvPCXZXSGoEgAAAGiPYIOgP7y7S+5Gn6aPGqI5Jw3v38X2fiCtesC/feF/S8NO7H+BAAAAQCcINpAkHThWqxc37Jck/fS8cf27W1NzRPrbf0mGT5p6tTTlihBVCQAAAHSMYANJ0qPv7pTHa+isE4fp9BOG9v1ChiGt+L5UVSINO0k6/6HQFQkAAAB0gmADfXm4Wn/bXCTJ3wmtX9Y+Ju18S7K5/PNqnIkhqBAAAADoGsEGeuSdnfL6DJ07PkOn5g3p+4WKNkn/d69/+2u/kbJOCUl9AAAAQHcINoNcRa1H//qsVJJ00/yxfb9QfYX08rWSzyNNuEiafn2IKgQAAAC6R7AZ5P75WYkavD6Nz0rW5JFpfbuIYUiv3SSV75PS8qSLHpX62yoaAAAA6AWCzSC3/OODkqRLpo3o+0U2PS1tWy5Z7dLX/yzFp4WkNgAAAKCnCDaD2IFjtdqw97gsFmnh1D4Gm0OfS/++3b/9lbuk3BmhKxAAAADoIYLNILbiY38ntDPGDFVWalzvL9BQK71yrdRYL405VzrjxhBXCAAAAPQMwWaQMgxDy7f4g80l00b27SLv/046vENKypIueVKy8nUCAACAOfhJdJD69GCFvjxcoziHVV87Jav3F6g5Iq173L99/oNS0vDQFggAAAD0AsFmkFreNAztqxOzlOSy9/4CH/y35KmRsqf62zsDAAAAJiLYDEIer0+vfVIsqY/d0CpLpA1/8m9/5S5aOwMAAMB0BJtB6P2dh3W0pkFDE506e+ywPlzgd/6GAbmnSyeeG/oCAQAAgF4i2AxCyz/2361ZMCVHdlsvvwLl+/3r1kjSV37O3RoAAABEBYLNIFNV79Fb20olSZee2odhaKsekHweKX+OlH92iKsDAAAA+oZgM8j8e2up3I0+nTA8UZNGpPbuzUd2SVte8G9/5eehLw4AAADoI4LNIBPohnbptBGy9HYY2ar7JcMrjT1Pyp0ZhuoAAACAviHYDCIlFXVa++VRSdLCqb0chnboc+mzV/zbX7kzxJUBAAAA/UOwGUT+saVYhiHNHJ2u3PSE3r155W8kGdLEhVL2lLDUBwAAAPQVwWYQWdE0DO2S3jYNKP5Y2v6aJIs092ehLwwAAADoJ4LNIPF5caV2lFbJabPq/FOye/fmd+/zP0++XMoYH/riAAAAgH4i2AwSK7b479acOyFDqQmOnr9x/zpp19uSxSbNuS1M1QEAAAD9Q7AZBLw+Q/9oCjYXT+vlMLR3f+1/nna1NHRMiCsDAAAAQoNgMwis3X1UhyrdSktwaN64jJ6/8ctV0t73JZtTOufW8BUIAAAA9BPBZhD4+8cHJUkXTMqW097DP3LDaL5bc9q1UlpumKoDAAAA+o9gM8DVNjTqza2lkqRLe9MNbefb0sGPJHu8dPaSMFUHAAAAhAbBZoB7+/NDqmnwKi89QafmDenZmwxDevdX/u2Z35GSs8JXIAAAABACBJsB7u+bm5sGWCyWnr1p+6tS6aeSM0k68+bwFQcAAACESK+DzerVq7VgwQLl5OTIYrFoxYoVnZ77ve99TxaLRcuWLetHieirw1Vuvb/zsCTpkp52Q/N5pfd+498+/QdS4tAwVQcAAACETq+DTU1NjaZMmaLHHnusy/OWL1+udevWKScnp8/FoX9e/aRYPkOampum/GGJPXvT1r9Jh3dIcWnS7MVhrQ8AAAAIFXtv31BQUKCCgoIuzykqKtKPfvQjvfnmm7rgggv6XBz6Z8XH/mFoPb5b4/VIK5f6t8+8UYpPC09hAAAAQIj1Oth0x+fz6ZprrtEtt9yik08+udvz3W633G538HVlZWWoSxqUdpVV6bOiCtmtFi2Y0sO7Zp+8IB37UkoYJs38bngLBAAAAEIo5M0DHnjgAdntdt144409On/p0qVKTU0NPnJzWS8lFJY33a2ZO2640hOd3b+h0S2tetC/ffYSyZUUxuoAAACA0AppsNm0aZN+//vf6+mnn+5xB6477rhDFRUVwceBAwdCWdKg5PMZWvFxsSR/N7Qe2fSMVHFASs6Wpl8XxuoAAACA0AtpsHn//fdVVlamvLw82e122e127du3Tz/5yU80evToDt/jcrmUkpLS6oH+2bD3mIrK65Tssmv+hMzu39BQK73/W//2OT+VHPHhLRAAAAAIsZDOsbnmmms0f/78VvvOO+88XXPNNbr22mtD+VHoQmAYWsGkLMU5bN2/YcOfpOpDUlqeNO1bYa4OAAAACL1eB5vq6mrt2rUr+HrPnj3asmWL0tPTlZeXp6FDW6974nA4lJWVpXHjxvW/WnSr3uPVPz8rkSRdMm1k92/weaV1/59/e85tkr0H83EAAACAKNPrYLNx40bNmzcv+HrJkiWSpEWLFunpp58OWWHom3d3lKmqvlE5qXGalZ/e/Rv2rJaqSvzr1ky6LOz1AQAAAOHQ62Azd+5cGYbR4/P37t3b249APwSGoS2cNkJWaw8aOHz6V//zyZdIdlcYKwMAAADCJ+TtnmGe4zUNWllYJqmHi3I21EjbX/NvT7kyjJUBAAAA4UWwGUBe/6xEHq+hk3NSdFJmcvdv2PEvqaFaShsl5c4Kf4EAAABAmBBsBpDlmw9K6uHdGql5GNrkK6QerjsEAAAARCOCzQBx4FitNu8vl9UiXTQlp/s3VJdJu9/1b0++IrzFAQAAAGFGsBkg3t95RJI0fVS6MlLiun/D1r9JhlcacZo07MQwVwcAAACEF8FmgFj75VFJ0uwxQ7s5s8knL/qfJ9M0AAAAALGPYDMAGIahdU3B5vQTehBsDhdKJVskq1065dLwFgcAAABEAMFmANh9uEaHq9xy2q2alpfW/RsCTQNOnC8lDgtrbQAAAEAkEGwGgMAwtNPyhijOYev6ZJ9P+vRl//bky8NcGQAAABAZBJsBYN3uXsyv2b9WqtgvOZOlceeHuTIAAAAgMgg2Ma7X82sCw9AmLpQc8WGsDAAAAIgcgk2M21lWraM1DYpzWDUlN7Xrkz310rYV/u0prF0DAACAgYNgE+PWNg1Dmz4qXS57N/Nrdr4puSuklBHSqLMiUB0AAAAQGQSbGLe2N/NrPmkahjbpMsnKHz0AAAAGDn66jWE+n6H1ewLza9K7Prn2mLTzLf/2ZIahAQAAYGAh2MSwwkNVOl7rUYLTpskj07o+edvfJZ9HypokZU6MSH0AAABApBBsYlhwfs3odDls3fxRfvqS/3nylWGuCgAAAIg8gk0MCyzMObu7Ns/HvpQOrJcsVmnSNyJQGQAAABBZBJsY5fMZ+mjPMUk9mF/z6cv+5xPmSslZ4S0MAAAAMAHBJkZ9XlKpijqPklx2TRrRxfo1hiF9+qJ/m6YBAAAAGKAINjFqXdMwtBmjh8je1fyaok3+oWiOBGn8hRGqDgAAAIgsgk2M6vH6NZ803a0Zf6HkSgpzVQAAAIA5CDYxyNtqfk0XwaaxQdr6N//2FIahAQAAYOAi2MSgbcUVqnI3KjnOrpNzuphfs/sdqe6YlJgh5c+NVHkAAABAxBFsYlBgGNqs/HTZrJbOTwwMQ5t0mWSzR6AyAAAAwBwEmxgUWL+my2Fo9RVS4Rv+7cmXR6AqAAAAwDwEmxjj8fq0oWl+TZeNAz7/h+R1S8PHS9lTIlQdAAAAYA6CTYzZWlShmgavUuMdmpCV0vmJn77kf558uWTpYrgaAAAAMAAQbGJMYBjarPx0WTubX1N+QNr7vn97EsPQAAAAMPARbGJMj9av+exl//Oos6S03AhUBQAAAJiLYBNDGhp92rj3uKQugo1hSJ/+1b/N2jUAAAAYJAg2MeSzonLVebwakuDQSRnJHZ9U+ql0eIdkc0kTF0a2QAAAAMAkBJsYEhiGdvoJQzufX/NJ092acQVSXBeLdwIAAAADCMEmhgQaB3Q6DM3bKG19xb895coIVQUAAACYj2ATI9yN3ub5NZ0tzLlnpVR9SIpPl8acG7niAAAAAJMRbGLEJwcq5G70aViSUydmJHV8UmDtmlO+LtmdkSsOAAAAMBnBJkYE5tfMOmGoLB0tuOmulra/5t+eTDc0AAAADC4Emxix9ssjkroahrZa8tRKaaOkkdMjWBkAAABgPoJNDKj3eLV5f7mkLhoH7H3f/zxmntTRHR0AAABgACPYxICP95erodGn4ckunTAsseOTAsFm9NmRKwwAAACIEgSbGBBs89zZ/JraY1LpVv82wQYAAACDEMEmBqzb3c36Nfs+lGRIw8ZJyZmRKwwAAACIEgSbKFfX4NXHB7pbvyYwDO2sCFUFAAAARBeCTZTbvP+4PF5DWSlxGjU0oeOT9n7gf85nGBoAAAAGJ4JNlFvbYhhah/Nrao5IZdv828yvAQAAwCBFsIlyLRsHdChwtyZjopQ4LEJVAQAAANGFYBPFatyN+uRAuaQerF/D/BoAAAAMYgSbKLZp33E1+gyNSIvXyCHxHZ8UuGPDMDQAAAAMYgSbKBYYhnZ6Z+vXVJdJh3dIsnDHBgAAAIMawSaKre1u/ZrAMLTMU6SE9AhVBQAAAEQfgk2UqnY36rOiCknS6Sd0ElpYvwYAAACQRLCJWhv2HJPXZyg3PV4jh7B+DQAAANAVgk2UWtddm+fKEunoTkkWadQZkSsMAAAAiEIEmygVXL+m0/k1TXdrsidL8UMiVBUAAAAQnQg2Uaiy3qOtwfk1nQWb1f5n2jwDAAAABJto9NGXx+QzpNFDE5Sdyvo1AAAAQHd6HWxWr16tBQsWKCcnRxaLRStWrAge83g8uu222zRp0iQlJiYqJydH3/rWt1RcXBzKmge8dd0NQ6soko59KVms0qjZEawMAAAAiE69DjY1NTWaMmWKHnvssXbHamtrtXnzZt11113avHmz/v73v6uwsFAXXXRRSIodLFouzNmhwPo12VOluNTIFAUAAABEMXtv31BQUKCCgoIOj6Wmpurtt99ute8Pf/iDZs6cqf379ysvL6/de9xut9xud/B1ZWVlb0saUMprG/R5if/3oNOOaIH1a2jzDAAAAEiKwBybiooKWSwWpaWldXh86dKlSk1NDT5yc3PDXVJUW7/nmAxDGjM8URkpcR2fFLhjw/waAAAAQFKYg019fb1uu+02XXXVVUpJSenwnDvuuEMVFRXBx4EDB8JZUtT7aM8xSV0MQyvfL5Xvkyw2Ke/0CFYGAAAARK9eD0XrKY/Ho8svv1yGYejxxx/v9DyXyyWXyxWuMmLO58X+YWhTctM6PiEwDG3EqZIrOTJFAQAAAFEuLMEmEGr27dund999t9O7NWjNMAwVHqqSJE3I6uT3jGFoAAAAQDshDzaBULNz50699957Gjq0kyFVaOdwtVvHahpktUhjM5Pan2AYzXdsRp8V2eIAAACAKNbrYFNdXa1du3YFX+/Zs0dbtmxRenq6srOz9Y1vfEObN2/W66+/Lq/Xq9LSUklSenq6nE5n6CofgHaU+O/WjB6aqDiHrf0Jx/dKlQclq4P5NQAAAEALvQ42Gzdu1Lx584KvlyxZIklatGiR7r33Xr366quSpKlTp7Z633vvvae5c+f2vdJBoLDUH2zGZ3cydyYwDG3EaZIzMUJVAQAAANGv18Fm7ty5Mgyj0+NdHUPXtpf6GweMy+xkfg3r1wAAAAAdCvs6Nui5wB2bcVkd3LExjBaNA5hfAwAAALREsIkSjV6fdpZVS5ImdDQU7diXUlWJZHNKubMiXB0AAAAQ3Qg2UWLv0Ro1NPqU4LQpd0hC+xP2rPY/j5whOeIjWxwAAAAQ5Qg2UWJH0zC0sZnJslot7U9g/RoAAACgUwSbKBGYXzOhs/k1rF8DAAAAdIpgEyW2l3TROODITqmmTLK5/EPRAAAAALRCsIkShYeaWj13FGz2Ns2vyZ0pOeIiWBUAAAAQGwg2UaDa3agDx+okSeOzOljDJrh+zTkRrAoAAACIHQSbKBCYX5OR7FJ6orP1QcOQ9n7g36ZxAAAAANAhgk0U6HJhzsM7pNojkj1eGnFqhCsDAAAAYgPBJgoUlvrn10zI7mIYWt4sye6KYFUAAABA7CDYRIHtgTs2mV00DmAYGgAAANApgo3JDMPofCiaz9c8v4bGAQAAAECnCDYmO1TpVkWdRzarRSdmJLU+WPa5VHdcciRKOdPMKRAAAACIAQQbk21vml+TPyxRcQ5b64N7A/NrTpdsjghXBgAAAMQOgo3JuuyIFly/hvk1AAAAQFcINiYLBJsJ7ebXeKV9gfVrmF8DAAAAdIVgY7LtJf6haOOy2rR6PrRVqq+QnMlS9hQTKgMAAABiB8HGRB6vT7sPV0uSxre9YxMYhjZqtmSzR7gyAAAAILYQbEy050iNPF5DSS67RqTFtz4YaBzA+jUAAABAtwg2JgoMQzspM0lWq6X5gLdR2rfGv03jAAAAAKBbBBsTNXdEazO/pvQTyV0puVKlrMkmVAYAAADEFoKNiYId0bLbzK/Z29QNbdQZkrXN2jYAAAAA2iHYmGhH4I5NZieNAxiGBgAAAPQIwcYklfUeFZXXSZLGtxyK5vVI+9f6t2kcAAAAAPQIwcYkXzTdrclOjVNqgqP5QPEWqaFaikuTMk8xpTYAAAAg1hBsTLI92Dig7fyaQJvnsyQrfzwAAABAT/CTs0kKS/2tntsFm+LN/ue80yNcEQAAABC7CDYmCXZEa9vq+dDn/meGoQEAAAA9RrAxgWEYzR3RWt6xaaiRjn3p38482YTKAAAAgNhEsDFBcUW9quobZbdaNGZ4UvOBwzskGVLCMCkpw7T6AAAAgFhDsDFBYH7NmOFJctpb/BEEh6FxtwYAAADoDYKNCbaXdNIR7dA2/zPBBgAAAOgVgo0JCjtr9VxGsAEAAAD6gmBjgmBHtOwWwcYwmu/YZEw0oSoAAAAgdhFsIqyh0afdh6slSeNatnquLpNqj0qySMPHm1McAAAAEKMINhG2+3C1Gn2GkuPsykmNaz4QGIY2dIzkTDCnOAAAACBGEWwiLDAMbXxWsiwWS/OBQEc0hqEBAAAAvUawibDtTa2e6YgGAAAAhA7BJsKaO6KltD5ARzQAAACgzwg2ERbsiNbyjo23UTpc6N9mKBoAAADQawSbCKqo9aikol6SdFLLYHPsS6mxXnIkSEPyTaoOAAAAiF0Emwja0TS/ZkRavFLiHM0HAsPQMiZIVv5IAAAAgN7ip+gIKjzU3BGtFTqiAQAAAP1CsImg7SWBxgF0RAMAAABCiWATQYWdtXqmIxoAAADQLwSbCPH5DH1xqFqSNCG7Ratnd7V0fK9/O4NgAwAAAPQFwSZCisrrVO1ulMNmUf6wxOYDZdv9z0mZUuJQc4oDAAAAYhzBJkJ2NK1fM2Z4khy2Fr/tDEMDAAAA+o1gEyGB+TWthqFJdEQDAAAAQoBgEyHbS+mIBgAAAIQLwSZCCjsKNobBUDQAAAAgBAg2EVDv8WrPkRpJ0oSsFkPRqkqluuOSxSYNG2dSdQAAAEDsI9hEwK6yanl9hlLjHcpMcTUfCAxDGzpGcsSZUxwAAAAwABBsIqDlMDSLxdJ8gGFoAAAAQEj0OtisXr1aCxYsUE5OjiwWi1asWNHquGEYuvvuu5Wdna34+HjNnz9fO3fuDFW9MWlHoCNau8YBgY5oBBsAAACgP3odbGpqajRlyhQ99thjHR5/8MEH9cgjj+iJJ57Q+vXrlZiYqPPOO0/19fX9LjZW7QjesWnb6jlwx4ZWzwAAAEB/2Hv7hoKCAhUUFHR4zDAMLVu2TD//+c+1cOFCSdKzzz6rzMxMrVixQldeeWX/qo1RHXZE83qkI4X+bYaiAQAAAP0S0jk2e/bsUWlpqebPnx/cl5qaqlmzZmnt2rUdvsftdquysrLVYyA5VtOgsiq3pDbB5uguydsgOZOk1DyTqgMAAAAGhpAGm9LSUklSZmZmq/2ZmZnBY20tXbpUqampwUdubm4oSzJdYH5Nbnq8klwtbpAFhqFlTJCs9HAAAAAA+sP0n6jvuOMOVVRUBB8HDhwwu6SQCg5Dy2wzv6asqXEAw9AAAACAfgtpsMnKypIkHTp0qNX+Q4cOBY+15XK5lJKS0uoxkOwo8QebCdltO6IF7tgQbAAAAID+Cmmwyc/PV1ZWlt55553gvsrKSq1fv16zZ88O5UfFjB2HOmgcIDW3eqYjGgAAANBvve6KVl1drV27dgVf79mzR1u2bFF6erry8vJ0880369e//rXGjh2r/Px83XXXXcrJydHFF18cyrpjgs9n6IumoWjjWwab+gqpYr9/O4NgAwAAAPRXr4PNxo0bNW/evODrJUuWSJIWLVqkp59+Wrfeeqtqamp0ww03qLy8XGeddZb+/e9/Ky4uLnRVx4j9x2pV5/HKabdq9NDE5gNl2/3PyTlSQro5xQEAAAADSK+Dzdy5c2UYRqfHLRaLfvnLX+qXv/xlvwobCAILc47NSJLd1mLUHwtzAgAAACFlele0gazDhTklOqIBAAAAIUawCaPAGjYTstp0eqMjGgAAABBSBJsw6vCOjWHQEQ0AAAAIMYJNmNQ1eLX3aI2kNh3RKoskd4VktUvDTjKpOgAAAGBgIdiEyc6yKvkMKT3RqeHJruYDgWFoQ8dKdlfHbwYAAADQKwSbMAl0RBuXmSyLxdJ8gI5oAAAAQMgRbMKEjmgAAABA5BBswiTYES27TbChIxoAAAAQcgSbMGm+Y9Oi1XNjg3TkC/82Q9EAAACAkCHYhEG1u1FHqhskSScMT2w+cHSn5GuUXClSaq5J1QEAAAADD8EmDIqO10mSUuLsSolzNB8IDkObKLVsKAAAAACgXwg2YVBUXitJGjkkofUBOqIBAAAAYUGwCYODTXdsRg6Jb32AjmgAAABAWBBswqA52HRyx4aOaAAAAEBIEWzCIDDHZkTLOzZ1x6XKIv92xgQTqgIAAAAGLoJNGBw8Hphj0yLYlG33P6fmSvFpkS8KAAAAGMAINmHQ4Ryblh3RAAAAAIQUwSbE6hq8OlrjX8NmZFqLOTZ0RAMAAADChmATYoFWz8kuu1Li7c0Hgh3RTjGhKgAAAGBgI9iE2IEWjQMsgUU4DUM61BRsGIoGAAAAhBzBJsSKOppfU75faqiSrA5p2FiTKgMAAAAGLoJNiHW4hk1gGNrwcZLNYUJVAAAAwMBGsAmxDls9H9rqf2YYGgAAABAWBJsQKypvmmOT1jLYBBoHEGwAAACAcCDYhFiXQ9HoiAYAAACEBcEmhOo9Xh2ucktqMRSt0S0d2enfZigaAAAAEBYEmxAqbhqGluC0KS2hqUnA4ULJ8EpxqVJKjonVAQAAAAMXwSaEDrZo9Rxcw6blMLTAPgAAAAAhRbAJoQ7n19ARDQAAAAg7gk0IFZX7Wz3TEQ0AAACILIJNCLUcihZERzQAAAAg7Ag2IdRuKFrtMamqxL+dMcGkqgAAAICBj2ATQkVNwWZE4I7NoW3+57Q8yZVsUlUAAADAwEewCRF3o1eHquoltRiKxjA0AAAAICIINiFSUl4vw5DiHFYNTXT6d9IRDQAAAIgIgk2IFDUtzjkircUaNnREAwAAACKCYBMiB4/7Wz0HGwf4fFLZdv82Q9EAAACAsCLYhEi7Vs/leyVPjWRzSeljzCsMAAAAGAQINiHSviNa0zC04SdJNrtJVQEAAACDA8EmRNqtYUNHNAAAACBiCDYh0jzHJnDHho5oAAAAQKQQbELA4/WptLJpDZu0NkPR6IgGAAAAhB3BJgRKK+rlMySn3aphSS7J2ygd+9J/cPgEc4sDAAAABgGCTQgcCAxDS4uX1WqRKoskwyvZnFJytsnVAQAAAAMfwSYE2nVEK9/vf07Nlaz8FgMAAADhxk/dIdBuDZuKA/7ntFyTKgIAAAAGF4JNCLRr9Ry4Y5OWZ1JFAAAAwOBCsAmBonL/HJsRaW2GohFsAAAAgIgg2IRAu6FowWAzyqSKAAAAgMGFYNNPjV6fSiqa1rBpOxQtlTk2AAAAQCQQbPrpUJVbXp8hh82ijOSmNWwqi/wHGYoGAAAARATBpp8OHvPPr8kJrGFTVSL5GiWrQ0rOMrk6AAAAYHAg2PRTp/NrUkdIVptJVQEAAACDC8Gmn4rKmxbnTGu7hg3D0AAAAIBIIdj008Hj/qForGEDAAAAmCfkwcbr9equu+5Sfn6+4uPjNWbMGP3qV7+SYRih/qio0H4o2j7/M62eAQAAgIixh/qCDzzwgB5//HE988wzOvnkk7Vx40Zde+21Sk1N1Y033hjqjzNdu6FotHoGAAAAIi7kwWbNmjVauHChLrjgAknS6NGj9cILL+ijjz4K9UeZzuszVNwUbEamB4aiMccGAAAAiLSQD0U744wz9M477+iLL76QJH3yySf64IMPVFBQ0OH5brdblZWVrR6xoqyqXh6vIbvVosxkl+TzShUH/QcJNgAAAEDEhPyOze23367KykqNHz9eNptNXq9X9913n66++uoOz1+6dKl+8YtfhLqMiChqml+TnRYnu80qVRRJPo9ktUvJ2SZXBwAAAAweIb9j89JLL+m5557T888/r82bN+uZZ57Rb3/7Wz3zzDMdnn/HHXeooqIi+Dhw4ECoSwqbQOOAdq2eU3IkW8gzIwAAAIBOhPyn71tuuUW33367rrzySknSpEmTtG/fPi1dulSLFi1qd77L5ZLL5Qp1GRHReatnOqIBAAAAkRTyOza1tbWyWltf1mazyefzhfqjTBfoiNa+1TPzawAAAIBICvkdmwULFui+++5TXl6eTj75ZH388cd6+OGHdd1114X6o0zXbigai3MCAAAApgh5sHn00Ud111136Qc/+IHKysqUk5Oj7373u7r77rtD/VGma16cs02rZ9awAQAAACIq5MEmOTlZy5Yt07Jly0J96aji8xkdDEXjjg0AAABghpDPsRksjlS71dDok9UiZaXGST5fc1c0gg0AAAAQUQSbPjoQWMMmNV4Om1WqPiR5GySLTUoZYXJ1AAAAwOBCsOmjwDC0EUNYwwYAAAAwG8Gmj4Jr2NARDQAAADAdwaaPmjuisYYNAAAAYDaCTR8V0eoZAAAAiBoEmz4KDEUbQatnAAAAwHQEmz4wDNawAQAAAKIJwaYPjtY0qN7jk8Xib/csw2ANGwAAAMBEBJs+CDQOyEyOk9NulWoOS431kiysYQMAAACYgGDTB8FWz22HoaXkSHanSVUBAAAAgxfBpg+KaPUMAAAARBWCTR8EhqLREQ0AAACIDgSbPmgeisYaNgAAAEA0INj0Aa2eAQAAgOhCsOklwzCah6KlEWwAAACAaECw6aXjtR7VNnglSTlprGEDAAAARAOCTS8FOqJlJLsU57BJtUclj3/OjVJHmlgZAAAAMHgRbHop0DhgRNtWz8nZkt1lUlUAAADA4Eaw6aWDwTVsAh3RmF8DAAAAmI1g00vtO6LR6hkAAAAwG8Gml4JD0eiIBgAAAEQNgk0vNQ9FI9gAAAAA0YJg0wuGYQS7ojHHBgAAAIgeBJteqKxrVJW7UVLTUDTWsAEAAACiAsGmFw40za8ZluRUvNMm1R2XGqr9B1nDBgAAADANwaYXAh3RRgSHoTWtYZOUKTniTaoKAAAAAMGmF4KNA+iIBgAAAEQVgk0vBFo9s4YNAAAAEF0INr1QRKtnAAAAICoRbHohMBRtBMEGAAAAiCoEm15oHorW1Dwg2Op5lEkVAQAAAJAINj1WWe9RZX2bNWyCd2yYYwMAAACYiWDTQ4H5NUMSHEp02aX6csld6T9I8wAAAADAVASbHgq2eg6uYdN0tyZxuORMMKkqAAAAABLBpseKOmv1TOMAAAAAwHQEmx4KdkRruzgnw9AAAAAA0xFseugga9gAAAAAUYtg00NF5Z3MsSHYAAAAAKYj2PRQYA2b4OKcFQQbAAAAIFoQbHqgxt2o47UeSS2CDXdsAAAAgKhBsOmBwDC01HiHUuIcUn2F/yHRPAAAAACIAgSbHjjYWavnhKGSK8mkqgAAAAAEEGx6gFbPAAAAQHQj2PRA0XE6ogEAAADRjGDTA6xhAwAAAEQ3gk0PdN7qeZRJFQEAAABoiWDTA82Lc7a9Y8McGwAAACAaEGy6Udfg1ZHqBknMsQEAAACiFcGmG0Xl/mFoyS67UuMdkrtKqjvuP0hXNAAAACAqEGy6EWz13HYNm/ghUlyKSVUBAAAAaIlg042DnbV65m4NAAAAEDUINt2g1TMAAAAQ/Qg23WjXEY1WzwAAAEDUIdh0I7CGDXdsAAAAgOgVlmBTVFSk//zP/9TQoUMVHx+vSZMmaePGjeH4qLALNg9Ia9vqmTk2AAAAQLSwh/qCx48f15lnnql58+bpjTfe0PDhw7Vz504NGTIk1B8VdvUerw5XuSVxxwYAAACIZiEPNg888IByc3P11FNPBffl5+eH+mMiorhpfk2i06a0BIfUUCPVHvUfpCsaAAAAEDVCPhTt1Vdf1fTp03XZZZcpIyND06ZN0x//+MdOz3e73aqsrGz1iBYt17CxWCzNa9i4UqX4NPMKAwAAANBKyIPNl19+qccff1xjx47Vm2++qe9///u68cYb9cwzz3R4/tKlS5Wamhp85OZGz52Q5o5obefXMAwNAAAAiCYWwzCMUF7Q6XRq+vTpWrNmTXDfjTfeqA0bNmjt2rXtzne73XK73cHXlZWVys3NVUVFhVJSUkJZWq/Ve7w6eLxWhiGNzUyWPvqj9K+fSuMukK563tTaAAAAgIGusrJSqampPcoGIZ9jk52drYkTJ7baN2HCBP3tb3/r8HyXyyWXyxXqMkIizmHTiRnJzTsqmoaicccGAAAAiCohH4p25plnqrCwsNW+L774QqNGDYAFLWn1DAAAAESlkAebH//4x1q3bp1+85vfaNeuXXr++ef1P//zP1q8eHGoPyrymGMDAAAARKWQB5sZM2Zo+fLleuGFF3TKKafoV7/6lZYtW6arr7461B8VeQQbAAAAICqFfI6NJF144YW68MILw3Fp83jqpJrD/m2CDQAAABBVQn7HZsAKrGHjTJbi0kwtBQAAAEBrBJueajkMzWIxtxYAAAAArRBseqqC+TUAAABAtCLY9BSNAwAAAICoRbDpKdawAQAAAKIWwaanuGMDAAAARC2CTU8FuqIRbAAAAICoQ7DpCU+9VF3q304bZW4tAAAAANoh2PRExUH/syNRih9ibi0AAAAA2iHY9ET5Pv8za9gAAAAAUYlg0xMVzK8BAAAAohnBpido9QwAAABENYJNT9DqGQAAAIhqBJueoNUzAAAAENUINj3BHRsAAAAgqhFsutPolqpK/NupBBsAAAAgGhFsulNxUJIh2eOlxGFmVwMAAACgAwSb7rRs9cwaNgAAAEBUIth0h/k1AAAAQNQj2HSHNWwAAACAqEew6Q53bAAAAICoR7DpDmvYAAAAAFGPYNOd4B2bUebWAQAAAKBTBJuueD1SVbF/O5U5NgAAAEC0Ith0pbJIMnySPU5KyjC7GgAAAACdINh0JTAMLTWXNWwAAACAKEaw6QqtngEAAICYYDe7gKh24nzpqhclR4LZlQAAAADoAsGmK8lZ0rgCs6sAAAAA0A2GogEAAACIeQQbAAAAADGPYAMAAAAg5hFsAAAAAMQ8gg0AAACAmEewAQAAABDzCDYAAAAAYh7BBgAAAEDMI9gAAAAAiHkEGwAAAAAxj2ADAAAAIOYRbAAAAADEPIINAAAAgJhHsAEAAAAQ8wg2AAAAAGIewQYAAABAzCPYAAAAAIh5drMLaMswDElSZWWlyZUAAAAAMFMgEwQyQleiLthUVVVJknJzc02uBAAAAEA0qKqqUmpqapfnWIyexJ8I8vl8Ki4uVnJysiwWi9nlqLKyUrm5uTpw4IBSUlLMLgcxhO8O+oPvD/qD7w/6g+8P+iPU3x/DMFRVVaWcnBxZrV3Poom6OzZWq1UjR440u4x2UlJS+I8bfcJ3B/3B9wf9wfcH/cH3B/0Ryu9Pd3dqAmgeAAAAACDmEWwAAAAAxDyCTTdcLpfuueceuVwus0tBjOG7g/7g+4P+4PuD/uD7g/4w8/sTdc0DAAAAAKC3uGMDAAAAIOYRbAAAAADEPIINAAAAgJhHsAEAAAAQ8wg2AAAAAGIewaYLjz32mEaPHq24uDjNmjVLH330kdklIQqtXr1aCxYsUE5OjiwWi1asWNHquGEYuvvuu5Wdna34+HjNnz9fO3fuNKdYRJ2lS5dqxowZSk5OVkZGhi6++GIVFha2Oqe+vl6LFy/W0KFDlZSUpK9//es6dOiQSRUjWjz++OOaPHlycHXv2bNn64033gge53uD3rj//vtlsVh08803B/fxHUJn7r33XlksllaP8ePHB4+b9d0h2HTir3/9q5YsWaJ77rlHmzdv1pQpU3TeeeeprKzM7NIQZWpqajRlyhQ99thjHR5/8MEH9cgjj+iJJ57Q+vXrlZiYqPPOO0/19fURrhTRaNWqVVq8eLHWrVunt99+Wx6PR1/96ldVU1MTPOfHP/6xXnvtNb388statWqViouLdemll5pYNaLByJEjdf/992vTpk3auHGjvvKVr2jhwoXatm2bJL436LkNGzboySef1OTJk1vt5zuErpx88skqKSkJPj744IPgMdO+OwY6NHPmTGPx4sXB116v18jJyTGWLl1qYlWIdpKM5cuXB1/7fD4jKyvLeOihh4L7ysvLDZfLZbzwwgsmVIhoV1ZWZkgyVq1aZRiG//vicDiMl19+OXjO9u3bDUnG2rVrzSoTUWrIkCHGn/70J7436LGqqipj7Nixxttvv23MmTPHuOmmmwzD4O8edO2ee+4xpkyZ0uExM7873LHpQENDgzZt2qT58+cH91mtVs2fP19r1641sTLEmj179qi0tLTVdyk1NVWzZs3iu4QOVVRUSJLS09MlSZs2bZLH42n1HRo/frzy8vL4DiHI6/XqxRdfVE1NjWbPns33Bj22ePFiXXDBBa2+KxJ/96B7O3fuVE5Ojk444QRdffXV2r9/vyRzvzv2sF49Rh05ckRer1eZmZmt9mdmZmrHjh0mVYVYVFpaKkkdfpcCx4AAn8+nm2++WWeeeaZOOeUUSf7vkNPpVFpaWqtz+Q5Bkj777DPNnj1b9fX1SkpK0vLlyzVx4kRt2bKF7w269eKLL2rz5s3asGFDu2P83YOuzJo1S08//bTGjRunkpIS/eIXv9DZZ5+trVu3mvrdIdgAQJRYvHixtm7d2mqcMtCVcePGacuWLaqoqNArr7yiRYsWadWqVWaXhRhw4MAB3XTTTXr77bcVFxdndjmIMQUFBcHtyZMna9asWRo1apReeuklxcfHm1YXQ9E6MGzYMNlstnbdGw4dOqSsrCyTqkIsCnxf+C6hOz/84Q/1+uuv67333tPIkSOD+7OystTQ0KDy8vJW5/MdgiQ5nU6deOKJOu2007R06VJNmTJFv//97/neoFubNm1SWVmZTj31VNntdtntdq1atUqPPPKI7Ha7MjMz+Q6hx9LS0nTSSSdp165dpv79Q7DpgNPp1GmnnaZ33nknuM/n8+mdd97R7NmzTawMsSY/P19ZWVmtvkuVlZVav3493yVI8rcD/+EPf6jly5fr3XffVX5+fqvjp512mhwOR6vvUGFhofbv3893CO34fD653W6+N+jWueeeq88++0xbtmwJPqZPn66rr746uM13CD1VXV2t3bt3Kzs729S/fxiK1oklS5Zo0aJFmj59umbOnKlly5appqZG1157rdmlIcpUV1dr165dwdd79uzRli1blJ6erry8PN1888369a9/rbFjxyo/P1933XWXcnJydPHFF5tXNKLG4sWL9fzzz+sf//iHkpOTg+OPU1NTFR8fr9TUVF1//fVasmSJ0tPTlZKSoh/96EeaPXu2Tj/9dJOrh5nuuOMOFRQUKC8vT1VVVXr++ee1cuVKvfnmm3xv0K3k5OTgXL6AxMREDR06NLif7xA689Of/lQLFizQqFGjVFxcrHvuuUc2m01XXXWVuX//hLXnWox79NFHjby8PMPpdBozZ8401q1bZ3ZJiELvvfeeIandY9GiRYZh+Fs+33XXXUZmZqbhcrmMc8891ygsLDS3aESNjr47koynnnoqeE5dXZ3xgx/8wBgyZIiRkJBgXHLJJUZJSYl5RSMqXHfddcaoUaMMp9NpDB8+3Dj33HONt956K3ic7w16q2W7Z8PgO4TOXXHFFUZ2drbhdDqNESNGGFdccYWxa9eu4HGzvjsWwzCM8EYnAAAAAAgv5tgAAAAAiHkEGwAAAAAxj2ADAAAAIOYRbAAAAADEPIINAAAAgJhHsAEAAAAQ8wg2AAAAAGIewQYAAABAzCPYAAAAAIh5BBsAAAAAMY9gAwAAACDm/T+iCX41ZeGR6wAAAABJRU5ErkJggg==", "text/plain": [ "
" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "# Transition probabilities\n", "alpha = 0.01\n", "beta = 0.2\n", "\n", "# Discount parameter\n", "gamma = 0.7\n", "\n", "# Expected rewards\n", "r_search = 6.0\n", "r_wait = 5.0\n", "\n", "V = np.zeros(nb_states)\n", "Q = np.zeros((nb_states, nb_actions))\n", "\n", "pi = np.array([a['search'], a['search']], dtype=int)\n", "\n", "V_high_history = []\n", "V_low_history = []\n", "\n", "for k in range(50):\n", " \n", " # Policy evaluation\n", " Q[s['high'], a['search']] = alpha * (r_search + gamma * V[s['high']]) + (1 - alpha) * (r_search + gamma*V[s['low']])\n", " Q[s['high'], a['wait']] = r_wait + gamma * V[s['high']]\n", "\n", " Q[s['low'], a['search']] = beta * (r_search + gamma * V[s['low']]) + (1 - beta) * (-3 + gamma*V[s['high']])\n", " Q[s['low'], a['wait']] = r_wait + gamma * V[s['low']]\n", " Q[s['low'], a['recharge']] = gamma * V[s['high']]\n", "\n", " V[s['high']] = Q[s['high'], :2].max()\n", " V[s['low']] = Q[s['low'], :].max()\n", "\n", " V_high_history.append(V[s['high']])\n", " V_low_history.append(V[s['low']])\n", " \n", " # Compute the greedy policy\n", " pi_old = pi.copy()\n", " pi[s['high']] = Q[s['high'], :2].argmax()\n", " pi[s['low']] = Q[s['low'], :].argmax()\n", " \n", "print('Greedy policy after iteration', k)\n", "print('pi(high)=', pi[s['high']])\n", "print('pi(low)=', pi[s['low']])\n", "print(\"V=\", V)\n", "print(\"Q=\", Q)\n", " \n", "plt.figure(figsize=(10, 6)) \n", "plt.plot(V_high_history, label=\"high\")\n", "plt.plot(V_low_history, label=\"low\")\n", "plt.legend()\n", "plt.show()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "**A:** The agent now decides to wait in the low state and accumulate quite a lot of rewards (r=5, compared to 6 while searching). But it is still worth searching in the high state: even if we transition immediately into low with 99% probability, one still gets 6 instead of 5, so the return is higher than when waiting in high." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "**Q:** Find a set of parameters where it would be optimal to search while in the *low* state." ] }, { "cell_type": "code", "execution_count": 12, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Greedy policy after iteration 49\n", "pi(high)= 0\n", "pi(low)= 0\n", "V= [28.03236199 25.7375694 ]\n", "Q= [[28.03236199 24.62265325 0. ]\n", " [25.7375694 23.01629844 19.62265325]]\n" ] }, { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAzYAAAH5CAYAAABTbqsJAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjkuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8hTgPZAAAACXBIWXMAAA9hAAAPYQGoP6dpAABLG0lEQVR4nO3deXxcZd338e8smcm+ts3SJN3pAl2wpaVlaZFCKVpBEEXwFhT1Fgs3FRXEBXB7it763IgicKsPiFJQkEVQdmgQaAu0lK0LbVqatmmSNm32TDKZOc8fJzNZmmUmmZkzk3zer9e85sw5Z2Z+TQ81X6/r/C6bYRiGAAAAACCB2a0uAAAAAACGi2ADAAAAIOERbAAAAAAkPIINAAAAgIRHsAEAAACQ8Ag2AAAAABIewQYAAABAwnNaXUBvfr9flZWVysjIkM1ms7ocAAAAABYxDEONjY0qKiqS3T7wmEzcBZvKykqVlJRYXQYAAACAOLF//34VFxcPeE7cBZuMjAxJZvGZmZkWVwMAAADAKg0NDSopKQlmhIHEXbAJTD/LzMwk2AAAAAAI6RYVmgcAAAAASHgEGwAAAAAJj2ADAAAAIOHF3T02ofL5fPJ6vVaXkVCSkpLkcDisLgMAAACIuIQLNoZhqKqqSnV1dVaXkpCys7NVUFDAGkEAAAAYURIu2ARCzbhx45Samsov6CEyDEMtLS2qqamRJBUWFlpcEQAAABA5CRVsfD5fMNTk5eVZXU7CSUlJkSTV1NRo3LhxTEsDAADAiJFQzQMC99SkpqZaXEniCvzsuD8JAAAAI0lCBZsApp8NHT87AAAAjEQJGWwAAAAAoDuCTYwsW7ZMa9as6fe4zWbT448/HvLnrV+/Xjabje5wAAAAgBKsecBIdujQIeXk5FhdBgAAAJCQCDZxoqCgwOoSAAAAgITFVLQY8vv9uuGGG5Sbm6uCggLdeuutwWO9p6K9/vrrmjdvnpKTk7VgwQI9/vjjstls2rp1a4/P3Lx5sxYsWKDU1FQtWbJEO3fujM0fBgAAAIgjCT9iYxiGWr0+S747JckRVpexP/3pT7r++uu1adMmbdiwQVdeeaVOO+00nXPOOT3Oa2ho0KpVq3T++edr3bp12rdvX7/353z/+9/Xr371K40dO1Zf//rX9eUvf1mvvfbacP5YAAAAQMJJ+GDT6vVp1s3PWvLd2368Qqmu0H+Ec+bM0S233CJJmjZtmn7729/qxRdfPC7YrFu3TjabTb///e+VnJysWbNm6eDBg/rqV7963Gf+7Gc/09KlSyVJ3/3ud/WJT3xCHo9HycnJw/iTAQAAAImFqWgxNGfOnB6vCwsLVVNTc9x5O3fu1Jw5c3qEk4ULFw76mYWFhZLU52cCAAAAI1nCj9ikJDm07ccrLPvucCQlJfV4bbPZ5Pf7h1VD988MTIsb7mcCo41hGOrwG/L5DXl9/s5nQx1+vzp8gWP+4Dl+v+QzzH0+v8x9nZ/h7zyno3Nf4JjfMN8X3DbU9ew3+t42DBmGIcOQ/IZkyNyv7sdlPit4fte5htH15wuc13lq57HO190/W12vFXjd7TO6jvfc13Vu9+/t2tfzdde2uj6hx/FA3b3P6rar2zt7nnv832+v1wr93JCPDfCZg713wPcN7W3DeOPgfxaMfEO9XjHy/M/n5qkkN9XqMkKW8MHGZrOFNR0sEUyfPl1/+ctf1NbWJrfbLUl68803La4KiDyf35DH65PH61Or1yeP13/c69bO110Pc197h19en1/tHZ2Pzm2vr2u73Wccd16H3y+vryvEBMIKAADoqa3DmvvYh2pkJYIR4rLLLtP3v/99fe1rX9N3v/tdVVRU6Je//KUkhdWsAIgWv99Qo6dDR1vadaylXY2eDjV5OtTc1qGmzkdzW4caO5+bPD33B7Y93vgeXUxy2OSw25Rkt8vhsMlpt8luM/cFHzab7PaexwKvzWMy99lsstlsctgU3Lbbuh8z99ttkj2wT5377OZ/+4HXNlvXs03me2zdPtfcr+Bxm63r/T32BV53/rsS+Ayp5/sDx8zn3sd7vj9wTN3O63ms69zex7of7/Hmnps9/h3sub/n31+/n9vHuf3p/W/uQG8b7DN71xCqof6zn2j/a8H/vAHxJz8zse7ZJtjEoczMTD355JO6+uqrNW/ePM2ePVs333yzLrvsMpoCIOIMw1BjW4dqGjw62uzVsZZ2HWtu17EWr+pa2nW02/axlq7tSA9yuJ12JSc5lJLkUHKSuZ3cuZ3SuZ2S5JC7c5/b6ZDLYZPLaZfLaVeSo+vZ7bTL5ei5z+U09zsdNjnt9q7Q4rAHw4uz2z677fhfagEAQPwi2MTI+vXrj9vXfd2a3vPDlyxZonfeeSf4+oEHHlBSUpJKS0slScuWLTvuPfPmzRtwnjlGH7/fUG1zu6rqPapq8KiqvlVVDR4dqveouvO5qt6jlvahDTWnuhzKSXUpMyVJ6W6H0t1Opbmdykh2Ks3lVHqyU+luZ3B/99fpbqdSXQ6lupxyO+2y2wkRAABg6Ag2cer+++/X5MmTNX78eL3zzju68cYb9dnPflYpKSlWl4Y4U9fSrvLDzSo/3KQ9h5u1/1iLGWTqPapp9MjrCy3sZiQ7NSbdrezUJOWkupSdmqTcVJdy0lzBfTmpLuWkdR13O8NroAEAABAtBJs4VVVVpZtvvllVVVUqLCzUJZdcop/97GdWlwWLeH1+7T/aovLDzdrTGWDKDzdpz5FmHW1uH/C9Nps0Nt2twqxk5Wcmm89Zyd1ep6ggM1kpLkIKAABIXASbOHXDDTfohhtusLoMxJjfb2jPkSZt3V+vXTWNKq9p1p4jTaqobVHHADe1FGYla/LYNE0Zm67S3FQVZacEQ8zYDLeSHCxZBQAARjaCDWChupZ2bd1fp7cr6rSl4pi27q9To6ejz3NTkhyaNCZNU8ala/KYtGCQmTQmTWlu/lMGAACjG78NATHS4fPrw+ombak4prcr6vT2/mPac7j5uPOSk+yaPT5LswozNXlsejDAFGQmc4M9AABAPwg2QJQ0eLzaWF6rt/fX6e2KY3r3QH2f3ccmjUnTySXZOrk0WyeX5mh6QQZTxwAAAMJEsAEiqLrBo+e3Veu5bdXaUH7kuI5k6W6n5gVDTLbmleQoN81lUbUAAAAjB8EGGKbyw0167oNqPftBlbbur+txbPKYNJ0yMVcfm2COxkwZmy4H08kAAAAijmADhMnvN/TOgTo9t61az31QpfJe98nMK8nWihMLdM6sfE0dl25RlQAAAKMLwSZGli1bpnnz5un222+3uhQMQXuHXxv31Oq5bVV6flu1qhvagseSHDYtnjJG587K1zmz8pWfmWxhpQAAAKMTwQYYwAeV9br3tY/07AdVPdowp7udWjZ9rM49sUDLpo9VZnKShVUCAACAYAP04vcbKvvwsH7/7z16vbw2uH9MulvnzMrXuSfma8mUPLmdDgurBAAAQHf0lLXAsWPH9MUvflE5OTlKTU3VypUrtWvXLkmSYRgaO3asHnnkkeD58+bNU2FhYfD1q6++KrfbrZaWlpjXPpJ5vD49+EaFzr39FX3pvjf1enmtHHabVs0t0sNfX6w3vne21l40W2dNH0eoAQAAiDOJP2JjGJLXol/wk1IlW/gdrq688krt2rVL//jHP5SZmakbb7xR559/vrZt26akpCSdeeaZWr9+vT7zmc/o2LFj2r59u1JSUrRjxw7NmDFDZWVlOuWUU5SamhqFP9Toc6SpTX/ZuE9/3rBPtc3tksypZp9fWKIrlkxUcQ4/ZwAAgHiX+MHG2yL9nyJrvvt7lZIrLay3BALNa6+9piVLlkiSHnjgAZWUlOjxxx/XJZdcomXLlumee+6RJL3yyis6+eSTVVBQoPXr12vGjBlav369li5dGvE/zmizu6ZRf3x1r/6+5aDaO/ySpPHZKfrSaRP1uVNKlMF9MwAAAAkj8YNNgtm+fbucTqcWLVoU3JeXl6fp06dr+/btkqSlS5fquuuu0+HDh1VWVqZly5YFg81VV12l119/XTfccINVf4SEZhiGXi+v1R/+vUcv7zwc3D+3OEtfOWOyVp5UIKeDGZoAAACJJvGDTVKqOXJi1XdHwezZs5Wbm6uysjKVlZXpZz/7mQoKCvTzn/9cb775prxeb3C0B6Hx+Q39452D+v0re7XtUIMkcxbhOTPz9dUzJ2vBhBzZhjCtEAAAAPEh8YONzRb2dDArzZw5Ux0dHdq0aVMwnNTW1mrnzp2aNWuWJMlms+mMM87QE088oQ8++ECnn366UlNT1dbWpnvuuUcLFixQWlri/Jmttv1Qg77793f1zoF6SVJKkkOXLCjWl06bpElj+DkCAACMBIkfbBLMtGnTdMEFF+irX/2q7rnnHmVkZOi73/2uxo8frwsuuCB43rJly/Stb31LCxYsUHq6uXr9mWeeqQceeEDf+c53rCo/oXi8Pv32pd26u6xcHX5DGclOfX3pFF2+qFTZqS6rywMAAEAEcTOBBe69917Nnz9fn/zkJ7V48WIZhqF//etfSkrqull96dKl8vl8WrZsWXDfsmXLjtuHvm3cU6vzf/1v/fbl3erwGzrvxAK9cP1SrT5rKqEGAABgBLIZhmFYXUR3DQ0NysrKUn19vTIzM3sc83g82rt3ryZNmqTk5GSLKkxsI/1nWN/q1W1Pb9eDb+yXJI3LcOvHF5yk804qsLgyAAAAhGugbNAbU9EwYjzz/iHd/MQHqmlskyRdtqhUN543Q1kptG0GAAAY6Qg2SHjVDR7d/MT7evaDaknS5DFpWnvRbC2anGdxZQAAAIgVgg0Slt9v6ME3K3Tbv3aosa1DTrtNVy+botVnTVVyksPq8gAAABBDBBskpPLDTbrp7+/pjY+OSpLmlWTrtotna0bBwHMvAQAAMDIRbJBQ2jv8uqesXL95abfafX6luhz6zorp+uLiiXLYWWATAABgtErIYOP3+60uIWEl8s/uWHO7vvynN/V2RZ0kadn0sfrphSepOCfV2sIAAABguYQKNi6XS3a7XZWVlRo7dqxcLpdsNv5f+lAYhqH29nYdPnxYdrtdLldireVSWdeqL/6/N7S7pklZKUn68QUn6lNzi/j7BwAAgKQECzZ2u12TJk3SoUOHVFlZaXU5CSk1NVWlpaWy2xNnbdbdNY36jz++oUP1HhVmJev+Ly/UtPwMq8sCAABAHEmoYCOZozalpaXq6OiQz+ezupyE4nA45HQ6E2qU4+2KY/rSfW+qrsWryWPT9OerFml8dorVZQEAACDOJFywkSSbzaakpCQlJbHw4khW9uFhff3Pm9Xq9WluSbbuvfIU5aYl1hQ6AAAAxEZCBhuMfE9sPahvP/yOvD5DZ0wbo7u/MF9pbi5XAAAA9I3fFBF37nttr3701DYZhrRqbpF+dclcuZyJc08QAAAAYo9gg7hhGIb+7/Mf6jcv7ZYkXbF4gm5ZdaLsrE8DAACAQRBsEBd8fkM/ePx9PfhGhSTpW+ecoGs+PjWhGh0AAADAOgQbWM7j9WnNQ1v1zAdVstmkn154ki5fNMHqsgAAAJBACDawVKPHq6/dv1kb9tTK5bDr9kvn6fzZhVaXBQAAgARDsIFlDje26cp739AHlQ1Kczn0+y8u0JKpY6wuCwAAAAmIYANL7D/aov/44yZ9VNuivDSX7vvSQs0uzrK6LAAAACQogg1irrKuVRff9bpqGttUnJOiP1+1SJPGpFldFgAAABIYwQYx1d7h1+p1W1TT2KZp49L1l68sUn5mstVlAQAAIMERbBBTa5/errcr6pSZ7NT/u/IUQg2AoTOMvrfNHaEd6/ecPs7r85xQDfDeAT93ON85RMP6c8ZaItUKJKCkVMnusLqKkBFsEDP/fPeQ7n3tI0nSrz47TyW5qdYWhJHJMCR/h9TRJvnaux4d7ZKvTfJ5zeOBZ79X8nV0bfs7Ol/3Pi/w8EmGv/PZ18++zv2GT/L7zWNG5znB8/xmrYMe80vqth3YH9xn9LGvc7+Mns997evx7O/6GQb3q+c5Cjz13tf7XPV9rPvn9zjW7fhA+wEAsbP6DWnsdKurCBnBBjFRfrhJNzzyjiTp60un6JxZ+RZXhJgyDKnDI7U1dj3am6T2ZsnbInk95nOHp9vrVqmj1Xz2th5/zNd2fGAJhBl+EQYAYNQh2CDqWtt9+sZftqi53adFk3L17XNPsLokhMswzCDSclRqqZVaj0otx7q2PQ2dYaVRamvqCi7dg4zhs6h4m+R0Sw635HRJ9iTJ4TSf7U7JkWQOs9uTOredXY/ux4L77ZLN0bnt6Nx29NwOHrd329e5bbNLNpv5HNzf/Zi98312s/bgcVsf+9RrX/dzbL2e7d221c85gff3Oi71Olc9t7vv635uv8d6vT9wbMDzB9l33P5ejjsWzrmReN8g7x3wbUN831C/bziGXCtGFq6DEcPhsrqCsBBsEFWGYej7j7+nndWNGpvh1m8uO1lOh33wNyL6OtqkxkNSQ6X5aKzqFlpqzeAS3D5qTs2KBFeG5E6X3BmSK82cv+tMlpJSzIcz2dyXlNztWO/Xyeazw2U+nO6ubYfLDDCOwD7+mQMAYDTgf/ERVQ+9uV+Pbjkou036zedP1rgMmgVEnWFIbQ1SwyGp4WBneOm+fdB83XIk/M92JkspuVJqnpSa07mdKyVnd4aVTMnVGVrc6Z0hpluQSUozRyMAAAAijGCDqHn/YL1u+ccHkqTvrJihUyfnWVzRCONpkGp3SUd2dz5/aG7X7TOngYXC4ZYyi8xHRoGUNrYrrKTmdtvOM7ddNHwAAADxiWCDqKhv8erqBzarvcOv5TPH6T/PnGx1SYnJ75Pq95uB5ciHnQGm89FUNfB7k7OkzPFSRmG38FJo7svsfE7JYU48AAAYEQg2iDjDMPSth9/R/qOtKs5J0a8umSe7nV+eB+X1SIfekQ68KR3cLB3eKdXuNjt+9SdtnDTmBGnMVClvmrmdO8kMMa602NUOAABgMYINIu5/X9mjF7ZXy+Ww667L5ysrNcnqkuKPYZhTxg68ZQaZA29Kh97t+wZ9h0vKm2o+xkzrCjB5U6SU7JiXDgAAEI8INoioTXtq9Ytnd0qSbv3UiZpdnGVxRXGirUmqfLsrxBx4U2o+fPx5aWOl4oVS8XypYI4ZZrJLE2rVXwAAACsQbBAxNY0eXfPg2/L5DV108nh9fmGJ1SVZp71ZKn9J2v2iOSpT80HXqu4B9iSpcI5UfErXI7uUe14AAACGgGCDiOjw+XXdg1t1uLFNJ+Sn66efPkm20fYLenOt9OHT0o5/mqGmw9PzeGaxVLzADDAlC80RmSTaXwMAAERCWMFm7dq1evTRR7Vjxw6lpKRoyZIl+vnPf67p06cHz1m2bJnKysp6vO8///M/dffdd0emYsSl//v8h9qwp1ZpLofu+sJ8pbpGSWY+9pG0419mmKl4veeoTHapNP0T0oTFZpjJLLKsTAAAgJEurN8+y8rKtHr1ap1yyinq6OjQ9773PZ177rnatm2b0tK6OjB99atf1Y9//OPg69RU1r4YyV7cXq3frS+XJN128RxNGZtucUVRZBhS9fvS9qfMMFP9Xs/jBXOkGZ+UZnxCyj+RaWUAAAAxElaweeaZZ3q8vu+++zRu3Dht3rxZZ555ZnB/amqqCgoKIlMh4tr+oy365l+3SpKuXDJRq+aOwFEJX4e0f6MZZHY8JdVVdB2z2aUJp5lBZvr5Us4E6+oEAAAYxYY1X6i+vl6SlJub22P/Aw88oL/85S8qKCjQqlWr9MMf/rDfUZu2tja1tXWt09HQ0DCckhBDbR0+feOBLWrwdGheSba+d/5Mq0uKrJaj0ub7pDf+V2o81LXfmSxNOVua+Ulp2gopLc+yEgEAAGAacrDx+/1as2aNTjvtNJ100knB/ZdddpkmTJigoqIivfvuu7rxxhu1c+dOPfroo31+ztq1a/WjH/1oqGXAQr9+YZfeO1ivnNQk3Xn5x+Ry2q0uKTKO7pE23iW9/RfJ22LuS8mRTlhpjsxMOYvFLwEAAOKMzTAMYyhvvPrqq/X000/r1VdfVXFxcb/nvfTSSzr77LO1e/duTZky5bjjfY3YlJSUqL6+XpmZmUMpDTFwqL5Vy/57vdo6/LrnP+ZrxYkJPvXQMKT9m6TXf2NOOVPnfxb5J0mLr5FOulhyuiwtEQAAYLRpaGhQVlZWSNlgSCM211xzjZ566im98sorA4YaSVq0aJEk9Rts3G633G73UMqAhX79wi61dfi1cGKuzp2Vb3U5Q+frkLb/Q9pwp3Twra79U8+RllwjTVpKAwAAAIAEEFawMQxD1157rR577DGtX79ekyZNGvQ9W7dulSQVFhYOqUDEn901jfrbW/slSTeunJGY69V4GqS3/yxtvFuq72wG4HBLcz8nnbpaGjfD2voAAAAQlrCCzerVq7Vu3To98cQTysjIUFVVlSQpKytLKSkpKi8v17p163T++ecrLy9P7777rr75zW/qzDPP1Jw5c6LyB0Ds/fezO+U3pHNn5Wv+hByrywlP3X5p093Slvults5GFal50ilfMR/p46ytDwAAAEMSVrC56667JJmLcHZ377336sorr5TL5dILL7yg22+/Xc3NzSopKdHFF1+sH/zgBxErGNbaUnFMz35QLbtNuuG86YO/IV4c2ye99BPp/Uclw2fuy5smLV4tzb1USkqxtj4AAAAMS9hT0QZSUlKisrKyYRWE+GUYhm57eock6TPzizV1XIbFFYXA7zNHaF76aVeHs4lnSEuuNe+jsY+QTm4AAACj3LDWscHosn7nYb2x96jcTrvWLD/B6nIGV/We9I9rpcq3zdcTTpdW/EwqmmdpWQAAAIg8gg1C4vcb+vkz5mjNlUsmqig7jqdueVulsl9Ir/3anHbmzpLO/Yl08n8wQgMAADBCEWwQkifeOagdVY3KTHbq6mXHt+2OG3v/LT15nXS03Hw96wJp5S+kjARfZwcAAAADIthgUG0dPv3y2Q8lSVcvm6rs1DhcqLL1mPT8zWa3M0nKKJTO/6U085PW1gUAAICYINhgUA9srNDBulblZ7p15ZKJVpfTk2FI256Q/vUdqbnG3Lfgy9LyW6XkLEtLAwAAQOwQbDCgRo9Xv315tyRpzfITlOJyWFxRNw2V0j+/Le38p/l6zAnSql9LE5ZYWxcAAABijmCDAf3+33t1tLldk8em6ZL5xVaXY/L7pc33Si/cai6yaU+STv+mdMa3pKRkq6sDAACABQg26Nfhxjb94d97JEk3rJgupyMOOood2S394xqpYoP5evwC6VO/kfJnWVsXAAAALEWwQb9+89IutbT7NK8kWytOjIOuYvs2SA9+TvLUS6506eybpVO+ItnjaHocAAAALEGwQZ/21TZr3aYKSdKN582QzWaztqCdT0sPXyl1eKSSRdLFf5SyS6ytCQAAAHGDYIM+/fK5D9XhN7T0hLFaPCXP2mK2rpOeuMZcbHPaCumS+yRXqrU1AQAAIK7EwU0TiDfvH6zXk+9UymYzR2ss9dod0uNXm6Fm7uelSx8g1AAAAOA4jNjgOD9/Zock6YK5RZpVlGlNEYZhLrj5+h3m6yXXSst/LNnJ4gAAADgewQY9vLb7iP6964iSHDZ969zp1hTh65Ce/C9p6wPm63N+LJ12nTW1AAAAICEQbBBkGEZwtObyRRNUkmvBlC9vq/Twl6QPn5ZsDulTd0gnfyH2dQAAACChEGwQ9K/3qvTugXqluRy65uNTY19Aa5304KXmGjXOZOkz90ozzo99HQAAAEg4BBtIkrw+v3753E5J0lfPnKwx6e7YFtBwSPrLxVLNB5I7S7rsIWnCktjWAAAAgIRFsIEk6a9v7tfeI83KS3PpK2dMju2X15ZLf75QqquQ0vOlLzwqFZwU2xoAAACQ0Ag2UEt7h3794i5J0n+dPU3p7hheFpVbzZGaliNS7mTpPx6TcibG7vsBAAAwIhBsoHtf+0iHG9tUmpuqzy8sjd0X7ymTHrpcam+UCuZIX/i7lD4udt8PAACAEYNgM8p5vD7dU1YuSfrWuSfI5YzROjHbnpD+/hXJ1y5NPEO6dJ2UbNGaOQAAAEh4BJtR7rlt1WrwdGh8dopWzSmKzZfuWS89fKVk+KWZn5Iu+r2UlByb7wYAAMCIRLAZ5R7ZfECSdPH8Ytnttuh/YUOl9MhVZqiZ/Vnp03dLdkf0vxcAAAAjWozmHSEeVdV79Oquw5Kkiz82Pvpf6POai2+2HJHyZ5uLbxJqAAAAEAEEm1Hs0bcPyG9ICyflakJeWvS/8IVbpf0bJXem9Nk/SUkp0f9OAAAAjAoEm1HKMAw98pY5De0z84uj/4Xb/iFt+K25feHvpLwp0f9OAAAAjBoEm1FqS0Wd9hxpVkqSQ+fPLozul9WWS0+sNreXXCvNXBXd7wMAAMCoQ7AZpQJNA1bOLojugpzeVulvV0htDVLpYunsW6L3XQAAABi1CDajkMfr01PvVEqKwTS0f31bqn5PShsrfeZeyZEU3e8DAADAqESwGYWe/aBKjW3m2jWnTsqL3hdt+bP09l8km126+A9SZpSnvAEAAGDUItiMQjFZu6bqPXO0RpLO+p40eVl0vgcAAAAQwWbUOVTfqld3H5EUxbVrPPXS374odXikqedIp38rOt8DAAAAdCLYjDKPbjkoI5pr1xiG2QHt6B4pq0S66H8lO5cZAAAAoovfOEcRwzD0981RXrtm4++k7U9K9iRzEc7U3Oh8DwAAANANwWYUifraNRUbpedvNrfPWyuNnx/57wAAAAD6QLAZRQJNA86fXRj5tWuaDksPf0nyd0gnfUY65SuR/XwAAABgAASbUSKqa9f4fdKjX5EaK6Ux06VVv5ZsUeq2BgAAAPSBYDNKBNauKc5J0aJJEb7vZf1t0p71UlKq9Nn7JXd6ZD8fAAAAGATBZpQIrl3zsQivXbPreemVX5jbq+6Qxs2I3GcDAAAAISLYjAI9166J4DS0uv3So181txdcJc25JHKfDQAAAISBYDMKBNauWTQpV6V5qZH74KdvlFqPSUUnm13QAAAAAIsQbEY4wzCC09Ai2jSgYpO085+SzS59+h7J6Y7cZwMAAABhItiMcFsqjmnvkWaluiK4do1hSC/cam6f/AVp7PTIfC4AAAAwRASbES4wWrPypEKlRWrtmg+flSpel5zJ0rKbIvOZAAAAwDAQbEaw1nafnnrnkKQITkPz+6QXf2RuL/q6lFkUmc8FAAAAhoFgM4I9ty0Ka9e8+zepZpuUnCWdviYynwkAAAAME8FmBIv42jVej/Tyz8ztM74lpeQM/zMBAACACCDYjFCVdVFYu+atP0r1+6XM8dLCr0XmMwEAAIAIINiMUI+9HeG1azz10iu/NLeXfVdKShn+ZwIAAAARQrAZgaKyds3rv5Faj0pjTpDmXhaZzwQAAAAihGAzAkV87ZrGKmnDneb22bdIjgi1jQYAAAAihGAzAgVGa86fHaG1a8p+IXlbpOKF0oxPDP/zAAAAgAgj2IwwEV+7prZc2vInc3v5rZItAt3VAAAAgAgj2IwwgbVrSnJTtHBiBNaueeknkr9DmnauNPG04X8eAAAAEAUEmxHm4bciuHZN5dvSB49Jspn31gAAAABximAzghysa9Vr5RFcu+aFW83nOZ+TCk4a/ucBAAAAUUKwGUEe23JAhiGdOjlXJbnDXLum/CVpz3rJ4ZLO+l5E6gMAAACihWAzQvRcu6ZkeB/m93eN1pzyFSlnwvA+DwAAAIgygs0IsaWiTh/VtijV5dDKkwqG92HbHpMOvSO5MqQzvh2ZAgEAAIAoItiMEOt31kiSzp6ZP7y1a3xe6cWfmNun/ZeUlheB6gAAAIDoItiMEK/tNpsGnDF1zPA+aPN90rG9Uto46dRvDL8wAAAAIAYINiNAo8erdw7US5KWTB3GCEtbk1T2C3N76Q2SOz0C1QEAAADRR7AZATbtOSqf39DEvFQV5wyjG9rGu6TmGilnkjT/yojVBwAAAEQbwWYECKxds2Q409Caj0iv/drc/vgPJEdSBCoDAAAAYoNgMwIE7q85fTjB5t+/ktobpcK50okXRagyAAAAIDYINgmuptGjD6ubZLNJiycP8f6augrpzT+Y28tvlexcFgAAAEgs/Aab4DaU10qSZhVmKifNNbQPefn/SL52adJSacrHI1gdAAAAEBsEmwT36q5hTkNrrJbe/Zu5vfzWyBQFAAAAxBjBJoEZhhG8v2bIjQPe/atk+KTihdL4j0WwOgAAACB2CDYJ7KPaFlXWe+Ry2HXKxJzwP8AwpK3rzO15l0W2OAAAACCGCDYJLDBac3JptlJdzvA/oPJt6fB2yZksnfjpCFcHAAAAxA7BJoG93rl+zWlDnYYWGK2Z8UkpJTsyRQEAAAAWINgkKL/f0OudHdGGFGw62qT3Hja3mYYGAACABEewSVDbDjWorsWrdLdTc4uzwv+AnU9Lnjopc7w0eVmkywMAAABiimCToF7tvL/m1Mm5cjqG8NcYmIY291LJ7ohgZQAAAEDsEWwSVLDN85QhTENrrJJ2v2Buz2UaGgAAABIfwSYBtXX49OZHRyUN8f6awNo1JYukMVMjXB0AAAAQewSbBLRlX508Xr/GpLt1Qn56eG9m7RoAAACMQASbBNTV5jlPNpstvDdXbpEO72DtGgAAAIwoBJsEFGgcMKRpaIHRmpmrpOQhdFMDAAAA4hDBJsE0erx690C9pCEEG69Heu8Rc5tpaAAAABhBCDYJZtOeo/L5DU3MS9X47JTw3vxht7VrJi2NSn0AAACAFQg2CSYi09BYuwYAAAAjDMEmwXQ1Dggz2HRfu2be5RGuCgAAALAWwSaB1DR49GF1k2w2afHkvPDe/O5fJcMvlZwq5U2JToEAAACARQg2CeT18lpJ0olFmcpJc4X+RsOQ3n7A3KZpAAAAAEYggk0CeS1wf82UMKehHdwiHdkpOVOkEy+MfGEAAACAxQg2CcIwjK5gE+79NVs7R2tYuwYAAAAjFMEmQXxU26LKeo9cDrtOmZgb+hu9Hul91q4BAADAyEawSRCBNs8fm5CtFFcYrZp3/kvy1EuZxdKkM6NUHQAAAGAtgk2CeH2o99ewdg0AAABGgbCCzdq1a3XKKacoIyND48aN04UXXqidO3f2OMfj8Wj16tXKy8tTenq6Lr74YlVXV0e06NHG5ze0YY/ZEW1JOPfXNBySyl80t5mGBgAAgBEsrGBTVlam1atXa+PGjXr++efl9Xp17rnnqrm5OXjON7/5TT355JN6+OGHVVZWpsrKSl100UURL3w02VbZoLoWr9LdTs0tDuPm/8DaNaWLWbsGAAAAI5oznJOfeeaZHq/vu+8+jRs3Tps3b9aZZ56p+vp6/fGPf9S6dev08Y9/XJJ07733aubMmdq4caNOPfXUyFU+irxWbk5DO3VyrpyOELOoYXRNQ2O0BgAAACPcsO6xqa+vlyTl5ppdujZv3iyv16vly5cHz5kxY4ZKS0u1YcOGPj+jra1NDQ0NPR7oaUhtng9u7lq7ZtaF0SkMAAAAiBNDDjZ+v19r1qzRaaedppNOOkmSVFVVJZfLpezs7B7n5ufnq6qqqs/PWbt2rbKysoKPkpKSoZY0IrV1+PTmR0clhRlsAmvXzPqUlJwZhcoAAACA+DHkYLN69Wq9//77euihh4ZVwE033aT6+vrgY//+/cP6vJFmy746ebx+jc1wa9q49NDe5PVI7/3d3GYaGgAAAEaBsO6xCbjmmmv01FNP6ZVXXlFxcXFwf0FBgdrb21VXV9dj1Ka6uloFBQV9fpbb7Zbb7R5KGaNCcBralDzZbLbQ3rTzn1Jb59o1E1m7BgAAACNfWCM2hmHommuu0WOPPaaXXnpJkyZN6nF8/vz5SkpK0osvvhjct3PnTlVUVGjx4sWRqXiUCTQOCKvNc7BpwOclO0sVAQAAYOQLa8Rm9erVWrdunZ544gllZGQE75vJyspSSkqKsrKydNVVV+n6669Xbm6uMjMzde2112rx4sV0RBuCBo9X7+yvkxTG/TUNlVL5S+b23M9HpzAAAAAgzoQVbO666y5J0rJly3rsv/fee3XllVdKkv7nf/5HdrtdF198sdra2rRixQr97ne/i0ixo82mPUflN6RJY9I0PjsltDexdg0AAABGobCCjWEYg56TnJysO++8U3feeeeQi4IpcH/Nkil5ob2BtWsAAAAwSnEDRhwLBJvTQ52GdnCzdORDKSlVOvHTUawMAAAAiC8EmzhV0+DRrpom2WzS4lBHbAJr18z8lOTOiF5xAAAAQJwh2MSpQDe0k4qylJ3qGvwN3lbWrgEAAMCoRbCJU6/trpUkLZka4mjNrufNtWuySqSJZ0SxMgAAACD+EGzikGEYej24MGeI99fsedl8nvEJ1q4BAADAqMNvwHFo75FmVdZ75HLYdcrE3NDetKfMfJ60NHqFAQAAAHGKYBOHXis3p6F9bEK2UlyOwd9Qf0A6Wi7Z7NLE06JcHQAAABB/CDZx6LVdYbZ5DozWjJ8vJWdFqSoAAAAgfhFs4ozPb2jDnkDjgFCDzXrzmWloAAAAGKUINnFmW2WD6lu9ynA7NWd8CKMvhiHt7RyxmUywAQAAwOhEsIkzr3Z2Q1s0OU9ORwh/PYd3Sk3VkjNZKl4Y5eoAAACA+ESwiTOb9nZOQ5sS4vo1gdGa0lOlpOQoVQUAAADEN4JNnNlW2SBJmluSHdobAvfXTF4WjXIAAACAhECwiSO1TW2qaWyTJM0oyBj8Db4O6aNXzW0aBwAAAGAUI9jEkR1VjZKkCXmpSnM7B3/Doa1SW4PZ4rlwbnSLAwAAAOIYwSaObD9kTkObWZAZ2hsC09AmniHZQ1jIEwAAABihCDZxZPshc8RmZmGYwYb7awAAADDKEWziSGDEZkZhCPfXeFul/W+Y2wQbAAAAjHIEmzjh9fm1u6ZJkjQrlBGbio2Sr03KKJLypka5OgAAACC+EWzixJ7DzWr3+ZXudqo4JyWEN6w3nycvlWy2qNYGAAAAxDuCTZwITkMryJAtlKASWJiTaWgAAAAAwSZebK/q7IgWyjS01mNS5VZzm/VrAAAAAIJNvAh0RAupccBHr0oypDEnSJmF0S0MAAAASAAEmzgRXMMmlBGbwP01jNYAAAAAkgg2ceFIU5sON7bJZpOm54cwYrOH+2sAAACA7gg2cWBH5zS0CbmpSnM7Bz65oVKq3SXZ7NLE02NQHQAAABD/CDZxILxpaJ2jNYXzpJTsqNUEAAAAJBKCTRwIqyNa9/VrAAAAAEgi2MSFYEe0gkHurzEM1q8BAAAA+kCwsZjX59fuGjPYDDpic2SX1HhIcrilkkUxqA4AAABIDAQbi5UfbpLXZyjD7VRxTsrAJwdGa0oXSUmDnAsAAACMIgQbiwUaB8wozJDNZhv4ZNavAQAAAPpEsLFYoNXzoNPQ/D7po3+b25PPinJVAAAAQGIh2FhsW2DEpmCQYHNoq+Spl9xZUtG8qNcFAAAAJBKCjcV2VAVGbAbpiBZYv2bi6ZLdEeWqAAAAgMRCsLHQkaY2HW5sk80mTR+s1TPr1wAAAAD9IthYKNA4YGJemlJdzv5P9Hqk/ZvMbdavAQAAAI5DsLFQV+OAQUZr9m+SOjxSeoE05oQYVAYAAAAkFoKNhbaH2jggsH7N5KXSYC2hAQAAgFGIYGOhQEe0QVs9s34NAAAAMCCCjUXaO/wqP9wkaZCpaK11UuXb5jaNAwAAAIA+EWwsUn64SV6foYxkp8Znp/R/4r7XJMMv5U2VsopjVyAAAACQQAg2FtlR1TkNrSBTtoHumwmsX8M0NAAAAKBfBBuLbA+1Ixrr1wAAAACDIthYJNgRbaDGAQ2HpCM7JdmkiWfEpjAAAAAgARFsLNI1YjNAsNn7ivlcOFdKzY1BVQAAAEBiIthY4HBjm440tclmk07IT+//xO7r1wAAAADoF8HGAoHGAZPy0pTqcvZ9kmGwfg0AAAAQIoKNBbaHsjBnbbnUcFByuKTSxTGqDAAAAEhMBBsLBO6vmVEwQEe0vevN55JFkis1+kUBAAAACYxgY4GQRmxYvwYAAAAIGcEmxto7/Co/3CRJmlnUT7Dx+7o6otE4AAAAABgUwSbGdtc0yeszlJnsVFFWct8nVb0reeokV4ZU9LGY1gcAAAAkIoJNjAU6os0ozJTNZuv7pMA0tImnS45+uqYBAAAACCLYxFjw/poBGwewfg0AAAAQDoJNjO2oMjui9ds4oKNN2rfB3KZxAAAAABASgk2MDdoRbf8bUkerlDZOGjczhpUBAAAAiYtgE0M1jR4daWqX3SadkN/PVLTu09D6uwcHAAAAQA8Emxja0bkw58QxaUpxOfo+qWKj+TzxjBhVBQAAACQ+gk0MDToNzTDMVs+SVHRyjKoCAAAAEh/BJoaCjQP664hWf0Dy1Et2pzR2egwrAwAAABIbwSaGBh2xqXrPfB4zXXK6Y1QVAAAAkPgINjHS1uHT7pomSebinH2qft98LjgpRlUBAAAAIwPBJkbKa5rV4TeUmexUUVZy3ycFRmwKZseuMAAAAGAEINjESPdpaLb+2jgHRmzyGbEBAAAAwkGwiZEdVYPcX9PWJB3da24zYgMAAACEhWATI9s717CZWdhPR7SabZIMKb1AShsTu8IAAACAEYBgEwOGYYTQEa1z/RoaBwAAAABhI9jEwOGmNtU2t8tuk07I72fEpor7awAAAIChItjEQGAa2qQxaUpOcvR9UrDVM/fXAAAAAOEi2MRAYBpav+vX+P1S9TZzm2ADAAAAhI1gEwM7OoPNrP6CzbG9krdZciZLuVNiWBkAAAAwMhBsYmDQjmiBxgHjZkoOZ4yqAgAAAEYOgk2UtXX4VH64SZI0o6C/jmg0DgAAAACGg2ATZbtrmtThN5SVkqTCrOS+T6JxAAAAADAsBJso6z4NzWaz9X1SFcEGAAAAGA6CTZQFGgf0Ow2t5ajUcMDczj8xRlUBAAAAIwvBJsq2Vw3SEa36A/M5u1RKzopRVQAAAMDIQrCJIsMwglPRZvTbEe098zmfaWgAAADAUBFsouhwY5uONrfLbpNOyO8n2AQbB9ARDQAAABgqgk0Ubeu8v2by2HQlJzn6Pik4YkOwAQAAAIaKYBNFO6o6p6EV9DNa4/NKh3eY23REAwAAAIaMYBNF2ztHbGb21zjgyC7J1y65MqTsCTGsDAAAABhZCDZRFAg2/XZEC05DO1Gy81cBAAAADBW/TUdJW4dP5YebJQ3QEa26M9jQOAAAAAAYFoJNlOyqbpLPbyg7NUkFmcl9n1TV2RGNxgEAAADAsBBsoqR74wCbzdb3ScFWz3NiVBUAAAAwMhFsomTQxgGN1VLzYclml8bNjGFlAAAAwMhDsImSQYNN4P6a3CmSKzVGVQEAAAAjE8EmCgzD6Ao2BYN0RKNxAAAAADBsBJsoONLUrmMtXtlt0rT89L5PonEAAAAAEDFhB5tXXnlFq1atUlFRkWw2mx5//PEex6+88krZbLYej/POOy9S9SaEfbVmm+ei7BQlJzn6PinYOGB2jKoCAAAARq6wg01zc7Pmzp2rO++8s99zzjvvPB06dCj4ePDBB4dVZKLZV9siSZqQ18+9M16PdGSXuU2wAQAAAIbNGe4bVq5cqZUrVw54jtvtVkFBwZCLSnT7jprBpjQ3re8TDm+XDJ+UkitlFMawMgAAAGBkiso9NuvXr9e4ceM0ffp0XX311aqtre333La2NjU0NPR4JLqKzqlopbn9jNh0bxzQ3xo3AAAAAEIW8WBz3nnn6f7779eLL76on//85yorK9PKlSvl8/n6PH/t2rXKysoKPkpKSiJdUsxVHB1kKlqwcQDT0AAAAIBICHsq2mAuvfTS4Pbs2bM1Z84cTZkyRevXr9fZZ5993Pk33XSTrr/++uDrhoaGhA83FcGpaP0Em2DjADqiAQAAAJEQ9XbPkydP1pgxY7R79+4+j7vdbmVmZvZ4JLKmtg4daWqXJJX2NWJjGLR6BgAAACIs6sHmwIEDqq2tVWHh6LhJvqKzI1pOapIyk5OOP6F+v9RWL9mTpLEzYlwdAAAAMDKFPRWtqampx+jL3r17tXXrVuXm5io3N1c/+tGPdPHFF6ugoEDl5eW64YYbNHXqVK1YsSKihcer4DS0vH46ogVGa8ZOl5yuGFUFAAAAjGxhB5u33npLZ511VvB14P6YK664QnfddZfeffdd/elPf1JdXZ2Kiop07rnn6ic/+Yncbnfkqo5jFUfNjmgTBuuIxjQ0AAAAIGLCDjbLli2TYRj9Hn/22WeHVVCiG3RxzupurZ4BAAAARETU77EZbQbtiEbjAAAAACDiCDYRFhix6TPYtDVKx/aa2wWsYQMAAABECsEmgjp8fh2sa5UkTeireUD1NvM5o1BKGxPDygAAAICRjWATQZV1Hvn8htxOu8Zl9NEsoepd85lpaAAAAEBEEWwiaF9nR7TS3FTZ7bbjT6juvL+GxgEAAABARBFsImjQjmg0DgAAAACigmATQYGOaCV9NQ7w+6SazntsaBwAAAAARBTBJoIqAiM2fQWbo3slb4vkTJHypsa4MgAAAGBkI9hE0L6jgalofXVE61yYc9xMye6IYVUAAADAyEewiRDDMFRR29k8oK97bKo6gw2NAwAAAICII9hESG1zu5rbfbLZpOKclONPCDYO4P4aAAAAINIINhES6IhWmJkst7OPqWa0egYAAACihmATIfs776/pcxpay1Gp4aC5nX9iDKsCAAAARgeCTYQE17DJ7atxQOdoTfYEKTkrhlUBAAAAowPBJkL2HQ2lcQD31wAAAADRQLCJkMAaNqV9rWETbBzA/TUAAABANBBsIqQiuIZNH8GmmlbPAAAAQDQRbCKgtd2nmsY2SX3cY+PzSod3mtuM2AAAAABRQbCJgMBoTVZKkrJSk3oePPKh5GuXXBlm8wAAAAAAEUewiYB9tWbjgD6noVV1W7/Gzo8bAAAAiAZ+046AwIhNSZ+NA941n5mGBgAAAEQNwSYCgo0D+go21d1GbAAAAABEBcEmAoKLc/aeimYY3Vo9s4YNAAAAEC0EmwgIjNiU9u6I1lQttRyRbHZp3EwLKgMAAABGB4LNMPn8hg4c6ww2vUdsAqM1uVMkVx/T1AAAAABEBMFmmCrrWuX1GXI57CrITO55MNA4oIBpaAAAAEA0EWyGaX/nNLTi3BQ57LaeB2kcAAAAAMQEwWaY9g3UEY3GAQAAAEBMEGyGqasjWq/GAd5WqXaXuc2IDQAAABBVBJthqjjaLKmPxTlrtkuGX0rJlTIKLagMAAAAGD0INsMUHLHpHWyC99fMlmy97r0BAAAAEFEEm2EwDEMV/S3OWfWe+UxHNAAAACDqCDbDUNfiVWNbh6Q+pqIFGwdwfw0AAAAQbQSbYQh0RCvITFZykqPrgGFI1R+Y2zQOAAAAAKKOYDMM+2rNxgGlvUdrGiqltnrJ5pDGTLegMgAAAGB0IdgMQ2BxztLe99fU7TOfs4olpyvGVQEAAACjD8FmGPrtiHasM9jkTIhxRQAAAMDoRLAZhn39jthUmM/ZpTGuCAAAABidCDbDEGj1fNw9NoGpaNkTY1sQAAAAMEoRbIbI4/WpqsEjSZqQl9bzIFPRAAAAgJgi2AzRgWPmaE2G26mc1KSeB5mKBgAAAMQUwWaIAo0DSvNSZbPZug74vFLDAXM7mxEbAAAAIBYINkMU7IjWu3FAw0HJ8EsOt5Seb0FlAAAAwOhDsBmiis6OaCX9tXrOLpHs/HgBAACAWOA37yHaV9ssSZqQ26txQLAjGtPQAAAAgFgh2AxRYMTmuKloNA4AAAAAYo5gMwR+v6H9x1ol9bGGDa2eAQAAgJgj2AxBVYNH7R1+Oe02FWWn9DzIVDQAAAAg5gg2QxDoiFackyKH3dbzYHAqGsEGAAAAiBWCzRBUHDUbB5Tm9Woc4PVIjYfMbaaiAQAAADFDsBmCYOOA3vfX1HcuzJmUKqXmxbgqAAAAYPQi2AxBv4tz1n1kPmdPkGy9pqgBAAAAiBqCzRAMujgn09AAAACAmCLYDEH/IzasYQMAAABYgWATpvoWr+pbvZL6WMOGVs8AAACAJQg2YQpMQxub4Vaqy9nzYGDEhqloAAAAQEwRbMK0r7PV83Ed0aSue2yYigYAAADEFMEmTIH7a46bhtbWJLUcMbeZigYAAADEFMEmTBWBYNO7cUD9fvPZnSWlZMe2KAAAAGCUI9iEKbg4Z+9gE2z1zDQ0AAAAINYINmEKBJvS3LSeB4KtnpmGBgAAAMQawSYMbR0+Vda3SuprDRtaPQMAAABWIdiE4cCxVhmGlOpyKC/N1fPgsY/MZ1o9AwAAADFHsAlDRbeOaDabrefB4FQ07rEBAAAAYo1gE4Z+GwdITEUDAAAALESwCUNgDZsJeb0aB7TWSZ56c5sRGwAAACDmCDZhqDjaLEkq6b04Z2AaWmqe5E6PcVUAAAAACDZhCI7YHBdsmIYGAAAAWIlgEyK/3+j/HpvAiA0d0QAAAABLEGxCdLipTW0dfjnsNhVlp/Q8eCwwYsP9NQAAAIAVCDYhCkxDG5+doiRHrx8bU9EAAAAASxFsQrSv1mwcUNr7/hqp2xo2BBsAAADACgSbEAXuryntfX+NYXRNReMeGwAAAMASBJsQBRsH9B6xaTkqec3RHGWVxLgqAAAAABLBJmRdi3P27oj2kfmcXiAlJce2KAAAAACSCDYhC05Fy03reYBpaAAAAIDlCDYhaPR4dbS5XVIf99jQOAAAAACwHMEmBIFpaHlpLqW7nT0P1rGGDQAAAGA1gk0I9vfXEU3qGrFhKhoAAABgGYJNCPb11xFN6rrHhhEbAAAAwDIEmxAEpqIdtzin3889NgAAAEAcINiEoOKouU5NaV6vjmjNNZKvTbLZpaxiCyoDAAAAIBFsQtLvGjaBaWiZ4yVHUoyrAgAAABBAsBmE1+dXZV2rpD7usQl2RGMaGgAAAGAlgs0gDh5rld+QkpPsGpvh7nmQVs8AAABAXCDYDCLQEa00N1U2m63nwcBUNFo9AwAAAJYi2AyiorazcUBu2vEH6YgGAAAAxAWCzSD6bRwgMRUNAAAAiBMEm0FUHO0n2Ph9Uv0Bc5upaAAAAIClCDaDqDjaz+KcDZWSv0OyJ0kZhRZUBgAAACCAYDMAwzD6DzaBaWhZxZLdEePKAAAAAHQXdrB55ZVXtGrVKhUVFclms+nxxx/vcdwwDN18880qLCxUSkqKli9frl27dkWq3pg63NSmlnaf7DapOKd3sOlsHMA0NAAAAMByYQeb5uZmzZ07V3feeWefx3/xi1/ojjvu0N13361NmzYpLS1NK1askMfjGXaxsba/c7SmMCtFLmevH9UxGgcAAAAA8cIZ7htWrlyplStX9nnMMAzdfvvt+sEPfqALLrhAknT//fcrPz9fjz/+uC699NLj3tPW1qa2trbg64aGhnBLiprQOqIxYgMAAABYLaL32Ozdu1dVVVVavnx5cF9WVpYWLVqkDRs29PmetWvXKisrK/goKSmJZEnDMnCwCUxFmxi7ggAAAAD0KewRm4FUVVVJkvLz83vsz8/PDx7r7aabbtL1118ffN3Q0BA34eYTcwpVmJWsCXl9LM7JVDQAAAAgbkQ02AyF2+2W2+22uow+nZCfoRPyM44/0NEuNVaa20xFAwAAACwX0aloBQUFkqTq6uoe+6urq4PHRoSGA5Lhl5zJUvo4q6sBAAAARr2IBptJkyapoKBAL774YnBfQ0ODNm3apMWLF0fyq6zVfRqazWZtLQAAAADCn4rW1NSk3bt3B1/v3btXW7duVW5urkpLS7VmzRr99Kc/1bRp0zRp0iT98Ic/VFFRkS688MJI1m2tQOMApqEBAAAAcSHsYPPWW2/prLPOCr4O3Ph/xRVX6L777tMNN9yg5uZmfe1rX1NdXZ1OP/10PfPMM0pOTo5c1Varo3EAAAAAEE9shmEYVhfRXUNDg7KyslRfX6/MzEyry+nbI1dJ7z8infNj6bTrrK4GAAAAGJHCyQYRvcdm1GAqGgAAABBXCDZDwVQ0AAAAIK4QbMLlbZWaOttZ50y0tBQAAAAAJoJNuOr2m8+udCklx9paAAAAAEgi2IQvOA1tAmvYAAAAAHGCYBOuQLDJoXEAAAAAEC8INuE6RuMAAAAAIN4QbMJFq2cAAAAg7hBswsVUNAAAACDuEGzCxVQ0AAAAIO4QbMLR1ii1HjW3CTYAAABA3CDYhCNwf01ytpScZWkpAAAAALoQbMIRCDbcXwMAAADEFYJNOI51W5wTAAAAQNwg2ISjjsYBAAAAQDwi2IQjOBVtoqVlAAAAAOiJYBMOpqIBAAAAcYlgEyrDYCoaAAAAEKcINqHy1EltDeY2wQYAAACIKwSbUAWmoaWNlVyp1tYCAAAAoAeCTagCjQO4vwYAAACIOwSbUAXur2FxTgAAACDuEGxCdYzGAQAAAEC8ItiEiqloAAAAQNwi2ISKVs8AAABA3CLYhMIwukZsciZaWgoAAACA4xFsQtF8RPK2SLJJWcVWVwMAAACgF4JNKALT0DIKJafb2loAAAAAHIdgEwpaPQMAAABxjWATimCrZ4INAAAAEI8INqEItnqmIxoAAAAQjwg2oWAqGgAAABDXCDahOMYaNgAAAEA8I9gMxu+X6veb29xjAwAAAMQlgs1gmqokX7tkc0iZ462uBgAAAEAfCDaDCUxDyxovOZzW1gIAAACgTwSbwQQ7ojENDQAAAIhXBJvB0BENAAAAiHsEm8HUsTgnAAAAEO8INoM5RrABAAAA4h3BZjB1rGEDAAAAxDuCzUB8HVL9QXObe2wAAACAuEWwGUjDQcnwSQ6XlF5gdTUAAAAA+kGwGUig1XNWiWTnRwUAAADEK35bHwitngEAAICEQLAZyDEaBwAAAACJwGl1AXHt5MulwjlSRpHVlQAAAAAYAMFmIDkTzQcAAACAuMZUNAAAAAAJj2ADAAAAIOERbAAAAAAkPIINAAAAgIRHsAEAAACQ8Ag2AAAAABIewQYAAABAwiPYAAAAAEh4BBsAAAAACY9gAwAAACDhEWwAAAAAJDyCDQAAAICER7ABAAAAkPAINgAAAAASHsEGAAAAQMIj2AAAAABIeAQbAAAAAAnPaXUBvRmGIUlqaGiwuBIAAAAAVgpkgkBGGEjcBZvGxkZJUklJicWVAAAAAIgHjY2NysrKGvAcmxFK/Ikhv9+vyspKZWRkyGazWV2OGhoaVFJSov379yszM9PqcpBAuHYwHFw/GA6uHwwH1w+GKhrXjmEYamxsVFFRkez2ge+iibsRG7vdruLiYqvLOE5mZib/cWNIuHYwHFw/GA6uHwwH1w+GKtLXzmAjNQE0DwAAAACQ8Ag2AAAAABIewWYQbrdbt9xyi9xut9WlIMFw7WA4uH4wHFw/GA6uHwyV1ddO3DUPAAAAAIBwMWIDAAAAIOERbAAAAAAkPIINAAAAgIRHsAEAAACQ8Ag2AAAAABIewWYAd955pyZOnKjk5GQtWrRIb7zxhtUlIQ698sorWrVqlYqKimSz2fT444/3OG4Yhm6++WYVFhYqJSVFy5cv165du6wpFnFl7dq1OuWUU5SRkaFx48bpwgsv1M6dO3uc4/F4tHr1auXl5Sk9PV0XX3yxqqurLaoY8eSuu+7SnDlzgit8L168WE8//XTwONcOwnHbbbfJZrNpzZo1wX1cQ+jPrbfeKpvN1uMxY8aM4HGrrh2CTT/++te/6vrrr9ctt9yiLVu2aO7cuVqxYoVqamqsLg1xprm5WXPnztWdd97Z5/Ff/OIXuuOOO3T33Xdr06ZNSktL04oVK+TxeGJcKeJNWVmZVq9erY0bN+r555+X1+vVueeeq+bm5uA53/zmN/Xkk0/q4YcfVllZmSorK3XRRRdZWDXiRXFxsW677TZt3rxZb731lj7+8Y/rggsu0AcffCCJawehe/PNN3XPPfdozpw5PfZzDWEgJ554og4dOhR8vPrqq8Fjll07Bvq0cOFCY/Xq1cHXPp/PKCoqMtauXWthVYh3kozHHnss+Nrv9xsFBQXGf//3fwf31dXVGW6323jwwQctqBDxrKamxpBklJWVGYZhXitJSUnGww8/HDxn+/bthiRjw4YNVpWJOJaTk2P84Q9/4NpByBobG41p06YZzz//vLF06VLjuuuuMwyDf38wsFtuucWYO3dun8esvHYYselDe3u7Nm/erOXLlwf32e12LV++XBs2bLCwMiSavXv3qqqqqse1lJWVpUWLFnEt4Tj19fWSpNzcXEnS5s2b5fV6e1w/M2bMUGlpKdcPevD5fHrooYfU3NysxYsXc+0gZKtXr9YnPvGJHteKxL8/GNyuXbtUVFSkyZMn6/LLL1dFRYUka68dZ1Q/PUEdOXJEPp9P+fn5Pfbn5+drx44dFlWFRFRVVSVJfV5LgWOAJPn9fq1Zs0annXaaTjrpJEnm9eNyuZSdnd3jXK4fBLz33ntavHixPB6P0tPT9dhjj2nWrFnaunUr1w4G9dBDD2nLli168803jzvGvz8YyKJFi3Tfffdp+vTpOnTokH70ox/pjDPO0Pvvv2/ptUOwAYA4sHr1ar3//vs95igDg5k+fbq2bt2q+vp6PfLII7riiitUVlZmdVlIAPv379d1112n559/XsnJyVaXgwSzcuXK4PacOXO0aNEiTZgwQX/729+UkpJiWV1MRevDmDFj5HA4juveUF1drYKCAouqQiIKXC9cSxjINddco6eeekovv/yyiouLg/sLCgrU3t6uurq6Hudz/SDA5XJp6tSpmj9/vtauXau5c+fq17/+NdcOBrV582bV1NToYx/7mJxOp5xOp8rKynTHHXfI6XQqPz+fawghy87O1gknnKDdu3db+u8PwaYPLpdL8+fP14svvhjc5/f79eKLL2rx4sUWVoZEM2nSJBUUFPS4lhoaGrRp0yauJcgwDF1zzTV67LHH9NJLL2nSpEk9js+fP19JSUk9rp+dO3eqoqKC6wd98vv9amtr49rBoM4++2y999572rp1a/CxYMECXX755cFtriGEqqmpSeXl5SosLLT03x+movXj+uuv1xVXXKEFCxZo4cKFuv3229Xc3KwvfelLVpeGONPU1KTdu3cHX+/du1dbt25Vbm6uSktLtWbNGv30pz/VtGnTNGnSJP3whz9UUVGRLrzwQuuKRlxYvXq11q1bpyeeeEIZGRnBucdZWVlKSUlRVlaWrrrqKl1//fXKzc1VZmamrr32Wi1evFinnnqqxdXDajfddJNWrlyp0tJSNTY2at26dVq/fr2effZZrh0MKiMjI3g/X0BaWpry8vKC+7mG0J9vf/vbWrVqlSZMmKDKykrdcsstcjgc+vznP2/tvz9R7bmW4H7zm98YpaWlhsvlMhYuXGhs3LjR6pIQh15++WVD0nGPK664wjAMs+XzD3/4QyM/P99wu93G2WefbezcudPaohEX+rpuJBn33ntv8JzW1lbjG9/4hpGTk2OkpqYan/70p41Dhw5ZVzTixpe//GVjwoQJhsvlMsaOHWucffbZxnPPPRc8zrWDcHVv92wYXEPo3+c+9zmjsLDQcLlcxvjx443Pfe5zxu7du4PHrbp2bIZhGNGNTgAAAAAQXdxjAwAAACDhEWwAAAAAJDyCDQAAAICER7ABAAAAkPAINgAAAAASHsEGAAAAQMIj2AAAAABIeAQbAAAAAAmPYAMAAAAg4RFsAAAAACQ8gg0AAACAhPf/Ae/FpmtyKi1YAAAAAElFTkSuQmCC", "text/plain": [ "
" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "# Transition probabilities\n", "alpha = 0.01\n", "beta = 0.8\n", "\n", "# Discount parameter\n", "gamma = 0.7\n", "\n", "# Expected rewards\n", "r_search = 10.0\n", "r_wait = 5.0\n", "\n", "V = np.zeros(nb_states)\n", "Q = np.zeros((nb_states, nb_actions))\n", "\n", "pi = np.array([a['search'], a['search']], dtype=int)\n", "\n", "V_high_history = []\n", "V_low_history = []\n", "\n", "for k in range(50):\n", " \n", " # Policy evaluation\n", " Q[s['high'], a['search']] = alpha * (r_search + gamma * V[s['high']]) + (1 - alpha) * (r_search + gamma*V[s['low']])\n", " Q[s['high'], a['wait']] = r_wait + gamma * V[s['high']]\n", "\n", " Q[s['low'], a['search']] = beta * (r_search + gamma * V[s['low']]) + (1 - beta) * (-3 + gamma*V[s['high']])\n", " Q[s['low'], a['wait']] = r_wait + gamma * V[s['low']]\n", " Q[s['low'], a['recharge']] = gamma * V[s['high']]\n", "\n", " V[s['high']] = Q[s['high'], :2].max()\n", " V[s['low']] = Q[s['low'], :].max()\n", "\n", " V_high_history.append(V[s['high']])\n", " V_low_history.append(V[s['low']])\n", " \n", " # Compute the greedy policy\n", " pi_old = pi.copy()\n", " pi[s['high']] = Q[s['high'], :2].argmax()\n", " pi[s['low']] = Q[s['low'], :].argmax()\n", " \n", "print('Greedy policy after iteration', k)\n", "print('pi(high)=', pi[s['high']])\n", "print('pi(low)=', pi[s['low']])\n", "print(\"V=\", V)\n", "print(\"Q=\", Q)\n", " \n", "plt.figure(figsize=(10, 6)) \n", "plt.plot(V_high_history, label=\"high\")\n", "plt.plot(V_low_history, label=\"low\")\n", "plt.legend()\n", "plt.show()" ] } ], "metadata": { "kernelspec": { "display_name": ".venv", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.12.7" } }, "nbformat": 4, "nbformat_minor": 4 }