{ "cells": [ { "cell_type": "markdown", "metadata": { "id": "-qm_04eLVkW2" }, "source": [ "# Run this Notebook\n", "\n", "[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/DeepRLCourse/Homework-1-Questions/blob/main/HW1_Notebook.ipynb) \n", "[![Open in Kaggle](https://kaggle.com/static/images/open-in-kaggle.svg)](https://kaggle.com/kernels/welcome?src=https://github.com/DeepRLCourse/Homework-1-Questions/blob/main/HW1_Notebook.ipynb)\n", "\n", "# HW3: REINFORCE Vs. GA\n", "> - Full Name: **[Full Name]**\n", "> - Student ID: **[Stundet ID]**\n", "\n", "\n", "This notebook implements a Grid World environment where an agent learns to navigate from a start position to a goal while avoiding penalties. It compares two learning approaches:\n", "\n", "1. REINFORCE Algorithm (Policy Gradient)\n", "2. Genetic Algorithm\n", "\n", "Follow the instructions in each section to complete the homework.\n", "\n", "**Grading Breakdown:**\n", "\n", "- Practical Implementation: 60 points\n", "- Conceptual Understanding: 40 points" ] }, { "cell_type": "markdown", "metadata": { "id": "UIV1aBVOVkW3" }, "source": [ "# Setup" ] }, { "cell_type": "markdown", "metadata": { "id": "R3E7ds6pVkW3" }, "source": [ "All required packages are pre-installed if using Google Colab." ] }, { "cell_type": "markdown", "metadata": { "id": "1zIoHwp7VkW3" }, "source": [ "Import the following libraries." ] }, { "cell_type": "code", "execution_count": 24, "metadata": { "colab": { "base_uri": "https://localhost:8080/" }, "id": "lPTr6YAZVkW3", "outputId": "7fa3e5cd-b0df-4aaf-b787-91f80ea9b263" }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Using device: cuda\n" ] } ], "source": [ "import numpy as np\n", "import random\n", "import torch\n", "import torch.nn as nn\n", "import torch.optim as optim\n", "import matplotlib.pyplot as plt\n", "\n", "device = 'cuda' if torch.cuda.is_available() else 'cpu'\n", "print(f\"Using device: {device}\")" ] }, { "cell_type": "markdown", "metadata": { "id": "npDihcn-VkW4" }, "source": [ "# Environment (10 Points)" ] }, { "cell_type": "markdown", "metadata": { "id": "4VIbPrUNVkW4" }, "source": [ "### GridWorld Class Definition (10 Points)" ] }, { "cell_type": "code", "execution_count": 10, "metadata": { "id": "gzZ21DR3VkW4" }, "outputs": [], "source": [ "# Grid World Parameters\n", "GRID_SIZE = 7\n", "\n", "# TODO: Specify the penalty locations on the Grid World. The number of penalty locations must be 8. {(x_1, y_1): penalty_reward, .... , (x_8, y_8): penalty_reward}\n", "PENALTIES = {(2,3): -10, (4,4): -10, (1,6): -10, (5,2): -10, (5,5): -10, (4,1): -10, (2,2): -10, (1,3): -10}\n", "\n", "# TODO: Specify the goal location.\n", "GOAL = (6,6)\n", "\n", "# TODO: Properly assign values to the following rewards.\n", "GOAL_REWARD = 100\n", "STEP_PENALTY = -1\n", "BOUNDARY_PENALTY = -5\n", "\n", "ACTIONS = ['up', 'down', 'left', 'right']\n", "ACTION_IDX = {a: i for i, a in enumerate(ACTIONS)}" ] }, { "cell_type": "markdown", "metadata": { "id": "PdM2t2xbVkW4" }, "source": [ "### GridWorld Environmnet Definition" ] }, { "cell_type": "code", "execution_count": 11, "metadata": { "id": "jQCWrZErVkW4" }, "outputs": [], "source": [ "class GridWorld:\n", " def step(self, state, action):\n", " x, y = state\n", " new_x, new_y = x, y\n", "\n", " if action == 'right':\n", " new_x = min(x + 1, GRID_SIZE - 1)\n", " elif action == 'left':\n", " new_x = max(x - 1, 0)\n", " elif action == 'up':\n", " new_y = min(y + 1, GRID_SIZE - 1)\n", " elif action == 'down':\n", " new_y = max(y - 1, 0)\n", "\n", " reward = STEP_PENALTY\n", " if (new_x, new_y) == (x, y):\n", " reward += BOUNDARY_PENALTY\n", " if (new_x, new_y) in PENALTIES:\n", " reward += PENALTIES[(new_x, new_y)]\n", " if (new_x, new_y) == GOAL:\n", " reward += GOAL_REWARD\n", "\n", " return (new_x, new_y), reward" ] }, { "cell_type": "markdown", "metadata": { "id": "nurQzzl_VkW5" }, "source": [ "### Initialize the Grid World" ] }, { "cell_type": "code", "execution_count": 12, "metadata": { "id": "fU3GAx06VkW5" }, "outputs": [], "source": [ "grid_world = GridWorld()" ] }, { "cell_type": "markdown", "metadata": { "id": "Djg0CI67VkW5" }, "source": [ "### Plot Empty Grid World" ] }, { "cell_type": "code", "execution_count": 13, "metadata": { "colab": { "base_uri": "https://localhost:8080/", "height": 468 }, "id": "TxqhAyerVkW5", "outputId": "6f9410db-5981-4ce6-d5d8-1d7f6b4d8e8c" }, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAaoAAAHDCAYAAAB1dF5kAAAAOnRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjEwLjAsIGh0dHBzOi8vbWF0cGxvdGxpYi5vcmcvlHJYcgAAAAlwSFlzAAAPYQAAD2EBqD+naQAANfhJREFUeJzt3Xtc1HW+x/H3AMMAAmpewRAVNe9shrGiqWvgpTJtt8umbShdTh6wi8ezu+7jsZtWq+0+2rKyLGvVTi7VbietbVeTLHXVTNAs23VNDVfXa3oSBBRG+J0/xplAQBmcGb4Mr+fjwWP4fX+3z5cZ5j2/3+87MzbLsiwBAGCokKYuAACAiyGoAABGI6gAAEYjqAAARiOoAABGI6gAAEYjqAAARiOoAABGI6gAAEYjqIB6TJ06Vd26dbvkcvv375fNZtOyZcv8XtPlGjVqlEaNGnXJ5datWyebzaZ169b5vSbgUggqBJ3CwkLl5OSod+/eioqKUlRUlPr166fs7Gx98cUXTVLT1q1bZbPZ9Mwzz9SaN3HiRNlsNi1durTWvBEjRqhLly6BKBEwFkGFoPL+++9rwIABev3115Wenq5nnnlGzz77rMaPH6+//vWv+t73vqd//etfDdrWK6+8ot27d/ukrsGDBysqKkobN26sNW/z5s0KCwvTpk2barRXVFQoPz9fw4YN80kNQHMV1tQFAL6yb98+/fjHP1ZiYqLWrl2ruLi4GvN/85vf6MUXX1RIyMVfn5WWlqpVq1ay2+0+qy0sLEypqam1wmj37t06ceKEJk+eXCvEtm3bprNnz2r48OGXvf+ysjJFRUVd9naApsARFYLGb3/7W5WWlmrp0qW1QkpyhcWDDz6ohIQET9vUqVMVHR2tffv26YYbblBMTIymTJnimXfhNapTp05p6tSpat26tdq0aaPMzEydOnWqQfUNHz5cx44d0969ez1tmzZtUmxsrO6//35PaFWf517P7cUXX1T//v3lcDgUHx+v7OzsWvsfNWqUBgwYoG3btmnEiBGKiorSL37xi3rr+ve//61JkyapVatW6tixox555BGVl5c3qE9AIHBEhaDx/vvvq2fPnkpNTfVqvXPnzmns2LEaPny4nnrqqXqPPCzL0sSJE7Vx40Y98MAD6tu3r1asWKHMzMwG7ccdOBs3blTPnj0lucLo+9//vlJTU2W327V582bdfPPNnnkxMTFKTk6WJM2ZM0dz585Venq6pk+frt27d2vRokXKz8/Xpk2bahwBnjx5UuPHj9ePf/xj3XXXXerUqVOdNZ05c0bXX3+9Dhw4oAcffFDx8fF6/fXX9dFHHzXsjwcEggUEgaKiIkuSNWnSpFrzvv32W+ubb77x/JSVlXnmZWZmWpKsn//857XWy8zMtBITEz3TK1eutCRZv/3tbz1t586ds6677jpLkrV06dKL1lhcXGyFhoZa99xzj6ftqquusubOnWtZlmVde+211n//93975nXo0MHKyMiwLMuyjh8/boWHh1tjxoyxKisrPcssXLjQkmQtWbLE0zZy5EhLkvXSSy/VqmHkyJHWyJEjPdMLFiywJFl//OMfPW2lpaVWz549LUnWxx9/fNE+AYHAqT8EheLiYklSdHR0rXmjRo1Shw4dPD8vvPBCrWWmT59+yX389a9/VVhYWI1lQ0NDNWPGjAbVGBMTo0GDBnmuRZ04cUK7d+9WWlqaJGnYsGGe031fffWVvvnmG89R2IcffqiKigo9/PDDNa6x3XfffYqNjdVf/vKXGvtyOByaNm1ag/oUFxenW2+91dMWFRWl+++/v0F9AgKBoEJQiImJkSSVlJTUmvfyyy8rLy9Py5cvr3PdsLAwXXnllZfcx7/+9S/FxcXVCsOrrrqqwXUOHz7ccy1q8+bNCg0N1fe//31JUlpamrZt26by8vJa16fcIxUv3Fd4eLh69OhRayRjly5dFB4e3qA+9ezZUzabrdF9AvyNoEJQaN26teLi4vTll1/Wmpeamqr09PR6h3k7HI5LjgT0FXfwbNq0SZs2bdLAgQM9wZeWlqby8nLl5+dr48aNCgsL84SYtyIjI31WM9DUCCoEjRtvvFF79+7V1q1b/bL9xMREHTlypNZRmzfvtao+oGLTpk01wjM+Pl6JiYmeELv66qs9AzsSExPr3FdFRYUKCws98xvTp3379smyrEb3CfA3ggpB46c//amioqKUlZWlY8eO1Zp/4ZOxt2644QadO3dOixYt8rRVVlbq+eefb/A24uPj1b17d61du1YFBQWe61NuaWlpWrlypXbv3l1jWHp6errCw8P13HPP1ejH73//exUVFenGG29sdJ8OHz6st99+29NWVlamxYsXN2p7gD8wPB1Bo1evXsrNzdWdd96pq666SlOmTFFycrIsy1JhYaFyc3MVEhLSoOtRdZkwYYKGDRumn//859q/f7/69eund955R0VFRV5tZ/jw4Xr99dclqdbpyLS0NL3xxhue5dw6dOig2bNna+7cuRo3bpxuvvlm7d69Wy+++KKGDBmiu+66q1F9uu+++7Rw4ULdfffd2rZtm+Li4vT666/z5mCYpWkHHQK+t3fvXmv69OlWz549rYiICCsyMtLq06eP9cADD1g7duyosWxmZqbVqlWrOrdz4fB0y7KskydPWj/5yU+s2NhYq3Xr1tZPfvIT67PPPmvQ8HS3l19+2ZJkdenSpda87du3W5IsSdaxY8dqzV+4cKHVp08fy263W506dbKmT59uffvttzWWGTlypNW/f/86933h8HTLsqx//etf1s0332xFRUVZ7du3tx566CFr9erVDE+HMWyWdZnnQwAA8COuUQEAjEZQAQCMRlABAIxGUAEAjEZQAQCMRlABAIwW8Df8VlVV6fDhw4qJian1QZgAgJbBsiydPn1a8fHxl/yszYAH1eHDh2t8wyoAoOU6ePDgJT8tJuBB5f46hoMHDyo2NrZR23A6nVqzZo3GjBlT41tNgxF9DT4tpZ8SfQ1GvupncXGxEhISPJlwMQEPKvfpvtjY2MsKqqioKMXGxgb1A0Kir8GopfRToq/ByNf9bMglIAZTAACMRlABAIxGUAEAjEZQAQCMRlABAIxGUAEAjEZQAQCMRlABAIxGUAEAjEZQma5wec1bAGhhvA6qQ4cO6a677lK7du0UGRmpgQMHqqCgwB+1Yd8SqSDH9XtBjmsaAJpCebn08ceSZbmmLcs1XV7u9117FVTffvuthg0bJrvdrlWrVukf//iHfve736lt27b+qq/l2rdE+vReSecfFLJc04QVgEArL5cmTZJGj5Zmz3a1/fznrulJk/weVl59KO1vfvMbJSQkaOnSpZ627t27+7yoFq9WSLmdDytJSsoKdFUAWiJ3SK1Z45petEgaMUJ66SXX9Jo1rvkrV0oOh19K8OqI6r333lNKSopuu+02dezYUVdffbVeeeUVvxTWYtUbUm4cWQEIoM2bpdWrpaqquudXVbnmf/KJ30rw6ojq66+/1qJFizRz5kz94he/UH5+vh588EGFh4crMzOzznXKy8tVXu2wsLi4WJLro+KdTmejinav19j1jVW4XCqYISnC0+RUZI1bj09nSJU2qftdASzQv4L2fr1AS+mnRF+DwrBh0iOPeI6gnJGRNW4lSdOnS2lpkhd99+bvZLMsq76X7rWEh4crJSVFmzdv9rQ9+OCDys/P1yf1pOmcOXM0d+7cWu25ubmKiopqcKEAgOBRVlamyZMnq6io6JLfTejVEVVcXJz69etXo61v37763//933rXmT17tmbOnOmZdn+r45gxYy7rixPz8vKUkZERXF9QVrj8/Ci/7147OBWpvFZLlFGaJbvOVFvYJqUsDLojqqC8Xy/QUvop0degYFmugRPVjqjylixRRlaW7GfOPydNny7Nny814EsQ3dxn1xrCq6AaNmyYdu/eXaPtq6++UmJiYr3rOBwOOeq4wGa32y/7zvTFNozSe5oUatV5jcquM9WCyialviolTQt4iYEQdPdrPVpKPyX62qx9/LH0zDO1mu1nznwXVE8/LU2YII0a1eDNevM38mowxSOPPKItW7Zo3rx52rt3r3Jzc7V48WJlZ2d7sxlcTFKWK4RU3ysTd0gx6g9AAKSlSePGSSH1xEVIiGv+0KF+K8GroBoyZIhWrFihN954QwMGDNDjjz+uBQsWaMqUKf6qr2WqN6wIKQAB5nC4hp6PGeOanj7ddfvAA67bMWP8OjRd8vLUnyTddNNNuummm/xRC6pzh9GnM843EFIAmog7rD75xHWEtWqV9OST0s03u46k/BhSUiOCCgGUlOUagr5LroETQXpNCkAz4HC4rkG5h5XbbF5dk7ocfCit6dyj+oJodB8AeIOgAgAYjaACABiNoAIAGI2gAgAYjaACABiNoAIAGI2gAgAYjaACABiNoAIAGI2gAgAYjaACABiNoAIAGI2gAgAYjaACABiNoAIAGI2gAgAYjaACABiNoAIAGI2gAgAYjaACABiNoAIAGI2gAgAYjaACABiNoAIAGI2gAgAYjaACABiNoAIAGI2gAgAYjaACABiNoAIAGI2gAgAYjaACABiNoAIAGI2gAgAYjaACABiNoAIAGI2gAgAYjaACABiNoAIAGI2gAgAYjaACABiNoAIAGI2gAgAYjaACABiNoAIAXyhcXvMWPkNQAcDl2rdEKshx/V6Q45qGz3gVVHPmzJHNZqvx06dPH3/VBgDm27dE+vReSdb5Bss1TVj5TJi3K/Tv318ffvjhdxsI83oTABAcaoWU2/mwkqSkrEBXFXS8TpmwsDB17tzZH7UAQPNRb0i5EVa+4nVQ7dmzR/Hx8YqIiNDQoUM1f/58de3atd7ly8vLVV5e7pkuLi6WJDmdTjmdzkaULM96jV2/OaGvwael9FMK4r4WLpcKZkiK8DQ5FVnj1uPTGVKlTep+VwAL9B9f3aferG+zLKu+lwO1rFq1SiUlJbrqqqt05MgRzZ07V4cOHdKXX36pmJiYOteZM2eO5s6dW6s9NzdXUVFRDS4UABA8ysrKNHnyZBUVFSk2Nvaiy3oVVBc6deqUEhMT9fTTT+uee+6pc5m6jqgSEhJ04sSJSxZXH6fTqby8PGVkZMhutzdqG80FfQ0+LaWfUhD3tXD5+VF+3z19OhWpvFZLlFGaJbvOVFvYJqUsDKojKl/cp8XFxWrfvn2DguqyRkK0adNGvXv31t69e+tdxuFwyOFw1Gq32+2X/cD1xTaaC/oafFpKP6Ug7GvvaVKoVec1KrvOVAsqm5T6qpQ0LeAl+tvl3qferHtZ76MqKSnRvn37FBcXdzmbAYDmJynLFUKy1bOAO6QYSHG5vAqqWbNmaf369dq/f782b96sW265RaGhobrzzjv9VR8AmKvesCKkfMmrU3///ve/deedd+rkyZPq0KGDhg8fri1btqhDhw7+qg8AzOYOo09nnG8gpHzNq6B68803/VUHADRfSVmuIei75Bo4EYTXpJoSn/UHAL7gHtUXJKP7TEJQAQCMRlABAIxGUAEAjEZQAQCMRlABAIxGUAEAjEZQAQCMRlABAIxGUAEAjEZQAQCMRlABAIxGUAEAjEZQAQCMRlABAIxGUAEAjEZQAQCMRlABAIxGUAEAjEZQAQCMRlABAIxGUAEAjEZQAQCMRlABAIxGUAEAjEZQAQCMRlABAIxGUAEAjEZQAQCMRlABAIxGUAEAjEZQAQCMRlABAIxGUAEAjEZQAQCMRlABAIxGUAEAjEZQAQCMRlABAIxGUAEAjEZQAQCMRlABAIxGUAEAjEZQAQCMRlABAIxGUAHwn8LlNW+BRiCoAPjHviVSQY7r94Ic1zTQCJcVVE8++aRsNpsefvhhH5UDICjsWyJ9eq8k63yD5ZomrNAIjQ6q/Px8vfzyyxo0aJAv6wHQ3NUKKTfCCo3TqKAqKSnRlClT9Morr6ht27a+rglAc1VvSLkRVvBeWGNWys7O1o033qj09HQ98cQTF122vLxc5eXlnuni4mJJktPplNPpbMzuPes1dv3mhL4Gn6DtZ+FyqWCGpAhPk1ORNW49Pp0hVdqk7ncFsED/Ctr79QK+6qc369ssy6rvpU+d3nzzTf36179Wfn6+IiIiNGrUKH3ve9/TggUL6lx+zpw5mjt3bq323NxcRUVFebNrAECQKCsr0+TJk1VUVKTY2NiLLutVUB08eFApKSnKy8vzXJu6VFDVdUSVkJCgEydOXLK4+jidTuXl5SkjI0N2u71R22gu6GvwCdp+Fi4/P8rvu6cUpyKV12qJMkqzZNeZagvbpJSFQXdEFZT36wV81c/i4mK1b9++QUHl1am/bdu26fjx4xo8eLCnrbKyUhs2bNDChQtVXl6u0NDQGus4HA45HI5a27Lb7Zd9Z/piG80FfQ0+QdfP3tOkUKvOa1R2nakWVDYp9VUpaVrASwyEoLtf63G5/fRmXa+C6vrrr9fOnTtrtE2bNk19+vTRz372s1ohBaCFScpy3dY7oMIdUlmBrArNnFdBFRMTowEDBtRoa9Wqldq1a1erHUALVSOsqiOk0Dh8MgUA30vKcoWSbOcbCCk0XqOGp1e3bt06H5QBIOgkZbmGoO+Sa+BEkF6Tgv9xRAXAf9yj+oJodB8Cj6ACABiNoAIAGI2gAgAYjaACABiNoAIAGI2gAgAYjaACABiNoAIAGI2gAgAYjaACABiNoAIAGI2gAgAYjaACABiNoAIAGI2gAgAYjaACABiNoAIAGI2gAgAYjaACABiNoAIAGI2gAgAYjaACABiNoAIAGI2gAgAYjaACABiNoAIAGI2gAgAYjaACABiNoAIAGI2gAgAYjaACABiNoAIAGI2gAgAYjaACABiNoAIAGI2gAgAYjaACABiNoAIAGI2gAgAYjaACABiNoAIAGI2gAgAYjaACABiNoAIAGI2gAgAYjaAyXeHymrfBrCX1FUCDeRVUixYt0qBBgxQbG6vY2FgNHTpUq1at8ldt2LdEKshx/V6Q45oOVi2prwC84lVQXXnllXryySe1bds2FRQUaPTo0Zo4caL+/ve/+6u+lmvfEunTeyVZ5xss13QwPoG3pL4C8FqYNwtPmDChxvSvf/1rLVq0SFu2bFH//v19WliLVuuJ2+38E7gkJWUFuir/aEl9BdAoXgVVdZWVlfrTn/6k0tJSDR061Jc1tWz1PnG7BdETeEvqK4BG8zqodu7cqaFDh+rs2bOKjo7WihUr1K9fv3qXLy8vV3l5uWe6uLhYkuR0OuV0OhtRsjzrNXZ9YxUulwpmSIrwNDkVWePW49MZUqVN6n5XAAv0oZbU1wsE7eO3DvQ1+Piqn96sb7Msq76Xs3WqqKjQgQMHVFRUpLfffluvvvqq1q9fX29YzZkzR3Pnzq3Vnpubq6ioKG92DQAIEmVlZZo8ebKKiooUGxt70WW9DqoLpaenKykpSS+//HKd8+s6okpISNCJEycuWVx9nE6n8vLylJGRIbvd3qhtGKlw+fmRb9/dJU5FKq/VEmWUZsmuM9UWtkkpC5vvUUZL6usFgvbxWwf6Gnx81c/i4mK1b9++QUHV6GtUblVVVTWC6EIOh0MOh6NWu91uv+w70xfbMErvaVKoVed1G7vOVHvytkmpr0pJ0wJeos+0pL7WI+gevxdBX4PP5fbTm3W9CqrZs2dr/Pjx6tq1q06fPq3c3FytW7dOH3zwgddFoh7uQQP1DjJwP3EHweCCltRXAI3mVVAdP35cd999t44cOaLWrVtr0KBB+uCDD5SRkeGv+lqmGk/g1QXhE3dL6iuARvEqqH7/+9/7qw5cyPMEPuN8QxA/cbekvgLw2mVfo4IfJWW5hmXvkmswQRBep/FoSX0F4BU+lNZ07pFuQTLi7aJaUl8BNBhBBQAwGkEFADAaQQUAMBpBBQAwGkEFADAaQQUAMBpBBQAwGkEFADAaQQUAMBpBBQAwGkEFADAaQQUAMBpBBQAwGkEFADAaQQUAMBpBBQAwGkEFADAaQQUAMBpBBQAwGkEFADAaQQUAMBpBBQAwGkEFADAaQQUAMBpBBQAwGkEFADAaQQUAMBpBBQAwGkEFADAaQQUAMBpBBQAwGkEFADAaQQUAMBpBBQAwGkEFADAaQQUAMBpBBQAwGkEFADAaQQUAMBpBBQAwGkEFADAaQQUAMBpBBQAwGkEFADAaQQVzFC6veQs0Jzx+/Yagghn2LZEKcly/F+S4poHmgsevX3kVVPPnz9eQIUMUExOjjh07atKkSdq9e7e/akNLsW+J9Om9kqzzDZZrmn92NAc8fv3Oq6Bav369srOztWXLFuXl5cnpdGrMmDEqLS31V30IdrX+yd34Z0czwOM3IMK8WXj16tU1ppctW6aOHTtq27ZtGjFihE8LQwtQ7z+52/l/dklKygpUVUDD8PgNGK+C6kJFRUWSpCuuuKLeZcrLy1VeXu6ZLi4uliQ5nU45nc5G7de9XmPXb06Ctq+Fy6WCGZIiPE1ORda49fh0hlRpk7rfFcAC/Sdo79M6BG1fefxe9n3qzfo2y7LqezlwUVVVVbr55pt16tQpbdy4sd7l5syZo7lz59Zqz83NVVRUVGN2DQBo5srKyjR58mQVFRUpNjb2oss2OqimT5+uVatWaePGjbryyivrXa6uI6qEhASdOHHiksXVx+l0Ki8vTxkZGbLb7Y3aRnMRtH0tXH5+lNR3Dz+nIpXXaokySrNk15lqC9uklIVB9Yo0KO/TOgRtX3n8XvZ9WlxcrPbt2zcoqBp16i8nJ0fvv/++NmzYcNGQkiSHwyGHw1Gr3W63X/YD1xfbaC6Crq+9p0mhVp3n+O06U+0f3SalviolTQt4if4WdPfpRQRdX3n8XvZ96s26XgWVZVmaMWOGVqxYoXXr1ql79+5eFwd4uC8w13tB2v1PzoVoGIjHb8B4FVTZ2dnKzc3Vu+++q5iYGB09elSS1Lp1a0VGRl5ibaAONf7Zq+OfHM0Aj9+A8Op9VIsWLVJRUZFGjRqluLg4z89bb73lr/rQEiRluf6pZTvfwD85mhEev37n9ak/wC+SslxDeHfJdeE5CM/pI4jx+PUrPusP5nCPigqS0VFoYXj8+g1BBQAwGkEFADAaQQUAMBpBBQAwGkEFADAaQQUAMBpBBQAwGkEFADAaQQUAMBpBBQAwGkEFADAaQQUAMBpBBQAwGkEFADAaQQUAMBpBBQAwGkEFADAaQQUAMBpBBQAwGkEFADAaQQUAMBpBBQAwGkEFADAaQQUAMBpBBQAwGkEFADAaQQUAMBpBBQAwGkEFADAaQQUAMBpBBQAwGkEFADAaQQUAMBpBBQAwGkEFADAaQQUAMBpBBQAwGkEFADAaQQUAMBpBBQAwGkEFADAaQQUAMBpBBQAwGkEFADAaQQXAfwqX17wFGoGgAuAf+5ZIBTmu3wtyXNNAI3gdVBs2bNCECRMUHx8vm82mlStX+qEsAM3aviXSp/dKss43WK5pwgqN4HVQlZaWKjk5WS+88II/6gHQ3NUKKTfCCo0T5u0K48eP1/jx4/1RC4Dmrt6QcjsfVpKUlBWoqtDMeR1U3iovL1d5eblnuri4WJLkdDrldDobtU33eo1dvzmhr8EnaPtZuFwqmCEpwtPkVGSNW49PZ0iVNqn7XQEs0L+C9n69gK/66c36Nsuy6nvpc+mVbTatWLFCkyZNqneZOXPmaO7cubXac3NzFRUV1dhdAwCasbKyMk2ePFlFRUWKjY296LJ+D6q6jqgSEhJ04sSJSxZXH6fTqby8PGVkZMhutzdqG80FfQ0+QdvPwuXnR/l995TiVKTyWi1RRmmW7DpTbWGblLLQb0dUlZWVOnfunC7j6c1r586d0+bNm5WWlqawML+frGoyDemnzWZTWFiYQkND691OcXGx2rdv36Cg8vtf0+FwyOFw1Gq32+2X/U/qi200F/Q1+ARdP3tPk0KtOq9R2XWmWlDZpNRXpaRpPi/BsiwdPXpUp06d8vm2G7Lvzp0768iRI7LZbAHff6B40882bdqoc+fOdS7nzWM/eGMfQOC5B0jUO6DCHVL+GUjhDqmOHTsqKioqoIFRVVWlkpISRUdHKyQkeN+i2pB+WpalsrIyHT9+XJIUFxd3Wfv0OqhKSkq0d+9ez3RhYaF27NihK664Ql27dr2sYgAEgRphVZ1/Q6qystITUu3atfPLPi6mqqpKFRUVioiICPqgakg/IyNdA2iOHz+ujh07XvQ04KV4HVQFBQX6wQ9+4JmeOXOmJCkzM1PLli1rdCEAgognrGacb/BvSEnfjSJjkJY53PeF0+kMbFCNGjUqoBcoATRTSVmuIei75Bo44YdrUnUJ5utDzY2v7ovgPT4F0PTco/qC6P1SCDyCCgBgNIIKAJrYN998o+nTp6tr165yOBzq3Lmzxo4dq02bNkmSTz8AfP/+/bLZbNqxY4dPthcIDE8HAEl7Tu7R6YrT9c6PCY9Rr3a9/LLvH/3oR6qoqNBrr72mHj166NixY1q7dq1Onjzp0/1UVFT4dHuBQlABaPH2nNyj3gt7X3K5r3K+8nlYnTp1Sn/729+0bt06jRw5UpKUmJioa6+9VpLUrVs3SdItt9zimbd//37t27dPM2fO1JYtW1RaWqq+fftq/vz5Sk9P92y7W7duuueee7Rnzx6tXLlSP/zhD/Xaa69Jkq6++mpJ0siRI7Vu3Tqf9snXOPUHoMW72JFUY5bzRnR0tKKjo7Vy5coaHzfnlp+fL0launSpjhw54pkuKSnRDTfcoLVr1+qzzz7TuHHjNGHCBB04cKDG+k899ZSSk5P12Wef6Ze//KW2bt0qSfrwww915MgRvfPOOz7vk68RVADQhMLCwrRs2TK99tpratOmjYYNG6Zf/OIX+uKLLyRJHTp0kPTdxxG5p5OTk/Uf//EfGjBggHr16qXHH39cSUlJeu+992psf/To0fqv//ovJSUlKSkpybN+u3bt1LlzZ11xxRUB7G3jEFQA0MR+9KMf6fDhw3rvvfc0btw4rVu3ToMHD77ohyiUlJRo1qxZ6tu3r9q0aaPo6Gjt2rWr1hFVSkqKn6v3P4IKAAwQERGhjIwM/fKXv9TmzZs1depUPfroo/UuP2vWLK1YsULz5s3T3/72N+3YsUMDBw6sNWCiVatW/i7d7wgqADBQv379VFpaKsn1SeOVlZU15m/atElTp07VLbfcooEDB6pz587av3//JbcbHh4uSbW2ZzKCCgCa0MmTJzV69GgtX75cX3zxhQoLC/WnP/1Jv/3tbzVx4kRJrtF7a9eu1dGjR/Xtt99Kknr16qV33nlHO3bs0Oeff67Jkyerqqrqkvvr2LGjIiMjtXr1ah07dkxFRUV+7Z8vEFQA0ISio6OVmpqqZ555RiNGjNCAAQP0y1/+Uvfdd58WLlwoSfrd736nvLw8JSQkeIaVP/3002rbtq3S0tI0YcIEjR07VoMHD77k/sLCwvTcc8/p5ZdfVnx8vCcMTcb7qAC0eDHhMT5dzhsOh0Pz58/X/Pnz611mwoQJmjBhQo22bt266aOPPqrRlp2dXWO6vlOB9957r+6998KvYTEXQQWgxevVrpe+yvmqyT6ZAhdHUAGARAgZjGtUAACjEVQAAKMRVAAAoxFUAACjEVQAAKMRVAAAoxFUAACjEVQA0IJ069ZNCxYsaOoyvEJQAYABjh49qoceekg9e/ZURESEOnXqpGHDhmnRokUqKytr6vKaFJ9MAQBu5eXS5s3SqFGSzSZZlrRunZSWJjkcftvt119/rWHDhqlNmzaaN2+eBg4cKIfDoZ07d2rx4sXq0qWLbr75Zr/t33QcUQGA5AqpSZOk0aOlRx6Rqqqkhx92TU+a5JrvJ//5n/+psLAwFRQU6Pbbb1ffvn3Vo0cPTZw4UX/5y188H0h74MABTZw4UdHR0YqNjdXtt9+uY8eOebazb98+TZw4UZ06dVJ0dLSGDBmiDz/80G91BwpBBQDukFqzxjX97LPS4MHSc8+5ptes8VtYnTx5UmvWrFF2dna938Zrs9lUVVWliRMn6v/+7/+0fv165eXl6euvv9Ydd9zhWa6kpEQ33HCD1q5dq88++0zjxo3ThAkTan09fXPDqT8A2LxZWr26Ztvnn3/3e1WVa/4nn7hOC/rQ3r17ZVmWrrrqqhrt7du319mzZyW5vr4jPT1dO3fuVGFhoRISEiRJ//M//6P+/fsrPz9fQ4YMUXJyspKTkz3bePzxx7VixQq99957ysnJ8WndgcQRFQCMGiU9+ODFl3noIWnkyICUI0lbt27Vjh071L9/f5WXl2vXrl1KSEjwhJTk+rr6Nm3aaNeuXZJcR1SzZs1S37591aZNG0VHR2vXrl0cUQFAs2ezSc88I61fX/NIyi05WXr6addyPtazZ0/ZbDbt3r27RnuPHj0kSZGRkQ3e1qxZs5SXl6ennnpKPXv2VGRkpG699VZVVFT4tOZA44gKACzLNYCirpCSXO0zZ7qW87F27dopIyNDCxcuVGlpab3L9e3bVwcPHtTBgwc9bf/4xz906tQp9evXT5K0adMmTZ06VbfccosGDhyozp071/stv80JQQUA69Z9N3CiPs8+6zri8oMXX3xR586dU0pKit566y3t2rVLu3fv1vLly/XPf/5ToaGhSk9P18CBAzVlyhRt375dW7du1d13362RI0cqJSVFktSrVy+988472rFjhz7//HNNnjxZVVVVfqk5kAgqAEhLk8aNk0KqPSVWG5SgkBDX/KFD/bL7pKQkffbZZ0pPT9fs2bOVnJyslJQUPf/885o1a5Yef/xx2Ww2vfvuu2rbtq1GjBih9PR09ejRQ2+99ZZnO08//bTatm2rtLQ0TZgwQWPHjtXgwYP9UnMgcY0KABwOaeVK1xD01atdAyeeftp1OvC556QxY1zz/fim37i4OD3//PN6/vnn612ma9euevfdd+ud361bN3300Uc12rKzs2tMN8dTgQQVAEjfhdUnn7hG99ls0oIF0i23uI6k/BhSuDiCCgDcHI6a75Oy2Xz+vil4j2tUAACjEVQAAKMRVAAAoxFUAACjEVQAAKMRVAAAoxFUAACjEVQAUJcqZ1NXEFA2m00rV65s6jLqRFABwIX2LZH+GO26DYCpU6fKZrPJZrMpPDxcPXv21GOPPaZz584FZP8X2r9/v2w2m3bs2NEk+78Qn0wBANXtWyJ9eq8k6/ytpKQsv+923LhxWrp0qcrLy/XXv/5V2dnZstvtmj17tt/3bTqOqADArXpISfKEVQCOrBwOhzp37qzExERNnz5d6enpeu+991ReXq5Zs2apS5cuatWqlVJTU7Vu3TrPesuWLVObNm30wQcfqG/fvoqOjta4ceN05MgRzzL5+fnKyMhQ+/bt1bp1a40cOVLbt2+vt5bu3btLkq6++mrZbDaNGjVKGzZskN1u19GjR2ss+/DDD+u6667z7R/jAo0KqhdeeEHdunVTRESEUlNTtXXrVl/XVcuek3u0/ch2bT+yXZ8fc3252efHPve07Tm5x+81AAhitULKLXBhVV1kZKQqKiqUk5OjTz75RG+++aa++OIL3XbbbRo3bpz27PnuOa+srExPPfWUXn/9dW3YsEEHDhzQrFmzPPNPnz6tzMxMbdy4UVu2bFGvXr10ww036PTp03Xu2/2c/uGHH+rIkSN65513NGLECPXo0UPLly/3LOd0OvWHP/xBWVn+PeL0+tTfW2+9pZkzZ+qll15SamqqFixYoLFjx2r37t3q2LGjP2rUnpN71Hthb890ZEik3hj0hkYsHaEzVWc87V/lfKVe7Xr5pQYAQazekHIL3GlAy7K0du1affDBB7rzzju1dOlSHThwQPHx8ZJcXze/evVqLV26VPPmzZPkCoyXXnpJSUlJkqScnBw99thjnm2OHj26xj4WL16sNm3aaP369brppptq1dChQwdJrm8f7ty5s6f9nnvu0bJly3T//fdLkv785z/r7Nmzuv322334F6jN6yOqp59+Wvfdd5+mTZumfv366aWXXlJUVJSWLPHfq43TFXWnfmOXAwCPS4aUm3+PrN5//31FR0crIiJC48eP1x133KFbb71VlZWV6t27t6Kjoz0/69ev1759+zzrRkVFeUJKcn231fHjxz3Tx44d03333adevXqpdevWio2NVUlJiQ4cOOBVjVOnTtXevXuVn58vyXXa8fbbb1erVq0us/cX59URVUVFhbZt21bj4l5ISIjS09P1ySef+Lw4APCrKqeUP12XDik3y7V8959IIXaflvKDH/xAixYtUnh4uOLj4xUWFqa33npLoaGh2rZtm0JDQ2ssHx0d7fndbq9Zi81mk2V916fMzEydPHlSzz77rBITE+VwODR06FBVVFR4VWPHjh1100036Q9/+IMGDBigVatW1bhe5i9eBdWJEydUWVmpTp061Wjv1KmT/vnPf9a5Tnl5ucrLyz3TxcXFklyHqk5nw96nUFVZpciQSM+0+/fqbe7lGrrN5sLdn2DrV11aSl9bSj+lwPbV6XTKsixVVVWpqqqqgWuFSikvyLb1fkmWbBdZ0vW0b5OV8oJrvQv24Q4Gdw3esCxLUVFR6tGjh6etqqpKycnJqqys1NGjR+scsFC9r9X3eWHbpk2btHDhQo0bN06SdPDgQZ04caJWre7thYW5osHpdNbqS1ZWlqZMmaLu3bsrKSlJQ4cOrbe/VVVVsixLTqezVtB685jw+/D0+fPna+7cubXa16xZo6ioqAZv541Bb9RqWzKg5iH4oW2HdEiHvC+yGcjLy2vqEgKmpfS1pfRTCkxfw8LC1LlzZ5WUlHh3pND+VoUPPKvInQ/Kqies3CF1ZuBzqmh/q3T+BXdd6hugcDFOp1Pnzp3zvJB369y5s2677TbdfffdeuKJJzRo0CCdOHFC69evV//+/TV27FidPXtWlmXVWPfMGde1e3dbjx499Nprr6lPnz46ffq0fvWrXykyMlJnz56ttV5xcbEiIiIUGRmpd999V61bt5bD4VDr1q0lSWlpaYqJidG8efM0e/bsWjVXV1FRoTNnzmjDhg213hNWVlbW4L+PV0HVvn17hYaG6tixYzXajx07VuOCW3WzZ8/WzJkzPdPFxcVKSEjQmDFjFBsb26D9fn7sc41YOsIzHRkSqSUDlijry6wagyk2TNug5E7J3nTJeE6nU3l5ecrIyKh1eB9sWkpfW0o/pcD29ezZszp48KDnOo9X+v+nrMgI2bbeXyusPEdS1y5WRI8s1bdly7J0+vRpxcTEyGa72LFZbXa7XWFhYXU+J77++uv69a9/rV/96lc6dOiQ2rdvr9TUVP3oRz9SbGysIiIiZLPZaqwbGek62+RuW7JkiR544AGNGjVKCQkJeuKJJ/TTn/5UERERtdZzTy9YsEBPPPGE5s+fr+uuu04fffSRp5+TJ0/2jFe42PP42bNnFRkZqREjRtS6Ty4WcBfyKqjCw8N1zTXXaO3atZo0aZIk16Hd2rVrlZOTU+c6DodDDoejVrvdbm/wAzckNKRGILmdqTpToz0kNCRo//G9+Xs1dy2lry2ln1Jg+lpZWSmbzaaQkBCFhDTinTc975VsIbUGVthkk1Jfle0So/3cp7/cNXjjtddeq3eew+HQY489VmMUX3VZWVm1hof/8Ic/rHGN6pprrvEMgHC7cKRe9eUl6f777/eM7quuqqpKR44c0fjx49WlS5d665ZcYxhsNlud9783jwevT/3NnDlTmZmZSklJ0bXXXqsFCxaotLRU06ZN83ZTAGAWdxh5wsoVUoH4ZIrmoKioSJ9//rnefvvtgH4uoNdBdccdd+ibb77Rr371Kx09elTf+973tHr16loDLHwpJjzGp8sBQL3coZQ/XRqyiJCqZuLEidq6daumTZumjIyMgO23UYMpcnJy6j3V5w+92vXSVzlfed4nVVVZpUPbDmnDtA0KCXUdYseEx/BmXwC+kZTllyHozd26detUVVXl1fUlX2g2H0pbPYScTqcO6ZCSOyW3mHP8AAKMkDIGH0oLADAaQQUgqHj7Zlv4j6/ui2Zz6g8ALiY8PFwhISE6fPiwOnTooPDwcK/fz3Q5qqqqVFFRobNnzzZueHwz0ZB+WpaliooKffPNNwoJCVF4ePhl7ZOgAhAUQkJC1L17dx05ckSHDx8O+P4ty9KZM2cUGRkZ0IAMNG/6GRUVpa5du152cBNUAIJGeHi4unbtqnPnzqmysjKg+3Y6ndqwYYNGjBgR1IO8GtrP0NBQhYWF+SS0CSoAQaW+T0Lwt9DQUJ07d04RERFBHVRN0c/gPZEKAAgKBBUAwGgEFQDAaAG/RuX+hN7L+QgOp9OpsrIyFRcXB/W5YIm+BqOW0k+JvgYjX/XTnQEXfmp7XQIeVO4vFUtISAj0rgEAhjl9+rTnSxnrY7MaEmc+VFVVpcOHDzfqy8Xc3F++ePDgwQZ/+WJzRV+DT0vpp0Rfg5Gv+un+osn4+PhLvs8q4EdUISEhuvLKK32yrdjY2KB+QFRHX4NPS+mnRF+DkS/6eakjKTcGUwAAjEZQAQCM1iyDyuFw6NFHH5XD4WjqUvyOvgafltJPib4Go6boZ8AHUwAA4I1meUQFAGg5CCoAgNEIKgCA0QgqAIDRmmVQvfDCC+rWrZsiIiKUmpqqrVu3NnVJPrdhwwZNmDBB8fHxstlsWrlyZVOX5Bfz58/XkCFDFBMTo44dO2rSpEnavXt3U5flF4sWLdKgQYM8b5QcOnSoVq1a1dRl+d2TTz4pm82mhx9+uKlL8bk5c+bIZrPV+OnTp09Tl+U3hw4d0l133aV27dopMjJSAwcOVEFBgd/32+yC6q233tLMmTP16KOPavv27UpOTtbYsWN1/Pjxpi7Np0pLS5WcnKwXXnihqUvxq/Xr1ys7O1tbtmxRXl6enE6nxowZo9LS0qYuzeeuvPJKPfnkk9q2bZsKCgo0evRoTZw4UX//+9+bujS/yc/P18svv6xBgwY1dSl+079/fx05csTzs3HjxqYuyS++/fZbDRs2THa7XatWrdI//vEP/e53v1Pbtm39v3Ormbn22mut7Oxsz3RlZaUVHx9vzZ8/vwmr8i9J1ooVK5q6jIA4fvy4Jclav359U5cSEG3btrVeffXVpi7DL06fPm316tXLysvLs0aOHGk99NBDTV2Szz366KNWcnJyU5cRED/72c+s4cOHN8m+m9URVUVFhbZt26b09HRPW0hIiNLT0/XJJ580YWXwlaKiIknSFVdc0cSV+FdlZaXefPNNlZaWaujQoU1djl9kZ2frxhtvrPH/Goz27Nmj+Ph49ejRQ1OmTNGBAweauiS/eO+995SSkqLbbrtNHTt21NVXX61XXnklIPtuVkF14sQJVVZWqlOnTjXaO3XqpKNHjzZRVfCVqqoqPfzwwxo2bJgGDBjQ1OX4xc6dOxUdHS2Hw6EHHnhAK1asUL9+/Zq6LJ978803tX37ds2fP7+pS/Gr1NRULVu2TKtXr9aiRYtUWFio6667zvN1RsHk66+/1qJFi9SrVy998MEHmj59uh588EG99tprft93wD89HahPdna2vvzyy6A9xy9JV111lXbs2KGioiK9/fbbyszM1Pr164MqrA4ePKiHHnpIeXl5ioiIaOpy/Gr8+PGe3wcNGqTU1FQlJibqj3/8o+65554mrMz3qqqqlJKSonnz5kmSrr76an355Zd66aWXlJmZ6dd9N6sjqvbt2ys0NFTHjh2r0X7s2DF17ty5iaqCL+Tk5Oj999/Xxx9/7LOvgTFReHi4evbsqWuuuUbz589XcnKynn322aYuy6e2bdum48ePa/DgwQoLC1NYWJjWr1+v5557TmFhYaqsrGzqEv2mTZs26t27t/bu3dvUpfhcXFxcrRdUffv2DcipzmYVVOHh4brmmmu0du1aT1tVVZXWrl0btOf5g51lWcrJydGKFSv00UcfqXv37k1dUkBVVVWpvLy8qcvwqeuvv147d+7Ujh07PD8pKSmaMmWKduzYodDQ0KYu0W9KSkq0b98+xcXFNXUpPjds2LBabx356quvlJiY6Pd9N7tTfzNnzlRmZqZSUlJ07bXXasGCBSotLdW0adOaujSfKikpqfGqrLCwUDt27NAVV1yhrl27NmFlvpWdna3c3Fy9++67iomJ8VxrbN26tSIjI5u4Ot+aPXu2xo8fr65du+r06dPKzc3VunXr9MEHHzR1aT4VExNT6xpjq1at1K5du6C79jhr1ixNmDBBiYmJOnz4sB599FGFhobqzjvvbOrSfO6RRx5RWlqa5s2bp9tvv11bt27V4sWLtXjxYv/vvEnGGl6m559/3uratasVHh5uXXvttdaWLVuauiSf+/jjjy1JtX4yMzObujSfqquPkqylS5c2dWk+l5WVZSUmJlrh4eFWhw4drOuvv95as2ZNU5cVEME6PP2OO+6w4uLirPDwcKtLly7WHXfcYe3du7epy/KbP//5z9aAAQMsh8Nh9enTx1q8eHFA9svXfAAAjNasrlEBAFoeggoAYDSCCgBgNIIKAGA0ggoAYDSCCgBgNIIKAGA0ggoAYDSCCgBgNIIKAGA0ggoAYDSCCgBgtP8HrtIY8ChQKfoAAAAASUVORK5CYII=", "text/plain": [ "
" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "plt.figure(figsize=(5, 5))\n", "plt.scatter(0, 0, c='green', marker='s', label='Start')\n", "plt.scatter(GOAL[0], GOAL[1], c='red', marker='X', label='Goal')\n", "for i, (px, py) in enumerate(PENALTIES):\n", " if i == 0:\n", " plt.scatter(px, py, c='orange', marker='D', label='Penalty')\n", " else:\n", " plt.scatter(px, py, c='orange', marker='D')\n", "plt.xticks(range(GRID_SIZE))\n", "plt.yticks(range(GRID_SIZE))\n", "plt.title(\"Grid World\")\n", "plt.legend()\n", "plt.grid(True)\n", "plt.show()" ] }, { "cell_type": "markdown", "metadata": { "id": "fLm0Q_NUVkW5" }, "source": [ "# REINFORCE Algorithm (30 Points)" ] }, { "cell_type": "markdown", "metadata": { "id": "gC1g8VL_VkW5" }, "source": [ "The REINFORCE algorithm updates policy parameters by using the log-probability of actions multiplied by the discounted return.\n", "\n", "This algorithm optimizes a **stochastic policy** $( \\pi_{\\theta}(a_t \\mid s_t) )$ by updating its parameters in the direction that increases expected rewards. The update rule is based on the **policy gradient theorem**: \n", "\n", "$$[\n", "\\theta \\leftarrow \\theta + \\alpha \\sum_{t=0}^{T} \\nabla_{\\theta} \\log \\pi_{\\theta}(a_t \\mid s_t) G_t\n", "]$$\n", "\n", "where: \n", "\n", "- $( \\theta )$ are the policy parameters (weights of the neural network). \n", "- $( \\alpha )$ is the learning rate. \n", "- $( G_t )$ is the **discounted return** from timestep $( t )$: \n", "\n", "$$[\n", "G_t = \\sum_{k=0}^{T-t} \\gamma^k R_{t+k}\n", "]$$\n", "\n", "- $( \\nabla_{\\theta} \\log \\pi_{\\theta}(a_t \\mid s_t) )$ is the gradient of the log-probability of the selected action, used to adjust the policy in the correct direction." ] }, { "cell_type": "markdown", "metadata": { "id": "mpvODJhGVkW5" }, "source": [ "### Policy Network Definition (10 points)" ] }, { "cell_type": "code", "execution_count": 14, "metadata": { "id": "NDniXDH8VkW5" }, "outputs": [], "source": [ "class PolicyNetwork(nn.Module):\n", " def __init__(self, input_dim=2, output_dim=4, hidden_dim=128):\n", " super().__init__()\n", " self.fc1 = nn.Linear(input_dim, hidden_dim)\n", " self.fc2 = nn.Linear(hidden_dim, output_dim)\n", "\n", " def forward(self, state):\n", " x = torch.relu(self.fc1(state))\n", " return torch.log_softmax(self.fc2(x), dim=-1)\n" ] }, { "cell_type": "markdown", "metadata": { "id": "z3s9oBcxVkW5" }, "source": [ "### REINFORCE Agent Implementation (20 Points)" ] }, { "cell_type": "code", "execution_count": 15, "metadata": { "id": "SFgBI88IVkW6" }, "outputs": [], "source": [ "class ReinforceAgent:\n", " def __init__(self, lr=0.005, gamma=0.99):\n", " self.policy_net = PolicyNetwork()\n", " self.optimizer = optim.Adam(self.policy_net.parameters(), lr=lr)\n", " self.gamma = gamma\n", "\n", " def train(self, episodes=8000, epsilon=0.99, epsilon_decay=0.999, min_epsilon=0.1):\n", " for episode in range(episodes):\n", " state = (0, 0)\n", " trajectory, rewards = [], []\n", " STOP = 0\n", "\n", " while state != GOAL and STOP <= 100:\n", " state_tensor = torch.tensor(state, dtype=torch.float32)\n", " log_action_probs = self.policy_net(state_tensor).detach().numpy()\n", " action_probs = np.exp(log_action_probs)\n", "\n", " action = np.random.choice(ACTIONS) if random.random() < epsilon else ACTIONS[np.argmax(action_probs)]\n", " new_state, reward = grid_world.step(state, action)\n", "\n", " trajectory.append((state, action, reward))\n", " rewards.append(reward)\n", " state = new_state\n", " STOP += 1\n", "\n", " returns, G = [], 0\n", " for r in reversed(rewards):\n", " G = r + self.gamma * G\n", " returns.insert(0, G)\n", "\n", " returns = torch.tensor(returns, dtype=torch.float32)\n", " baseline = returns.mean()\n", " returns -= baseline\n", "\n", " loss = 0\n", " for (state, action, _), G in zip(trajectory, returns):\n", " state_tensor = torch.tensor(state, dtype=torch.float32)\n", " log_action_probs = self.policy_net(state_tensor)\n", " loss -= log_action_probs[ACTION_IDX[action]] * G\n", "\n", " self.optimizer.zero_grad()\n", " loss.backward()\n", " self.optimizer.step()\n", "\n", " epsilon = max(min_epsilon, epsilon * epsilon_decay)\n", "\n", " if episode % 50 == 0:\n", " print(f\"Episode {episode}: Total Reward = {sum(rewards)}, Epsilon = {epsilon:.4f}\")\n", "\n", " def get_optimal_trajectory(self):\n", " state = (0, 0)\n", " trajectory = [state]\n", " rewards = 0\n", " STOP = 0\n", " while state != GOAL and STOP <= 100:\n", " state_tensor = torch.tensor(state, dtype=torch.float32)\n", " log_action_probs = self.policy_net(state_tensor).detach().numpy()\n", " action_probs = np.exp(log_action_probs)\n", " # action = np.random.choice(ACTIONS, p=action_probs / action_probs.sum())\n", " action = ACTIONS[np.argmax(action_probs)]\n", " state, reward = grid_world.step(state, action)\n", " rewards += reward\n", " trajectory.append(state)\n", " STOP += 1\n", " return trajectory, rewards" ] }, { "cell_type": "markdown", "metadata": { "id": "Kq55_sVbVkW6" }, "source": [ "# Genetic Algorithm" ] }, { "cell_type": "markdown", "metadata": { "id": "3_TK-j01VkW6" }, "source": [ "\n", "Genetic Algorithms (GAs) are optimization algorithms inspired by **natural selection**. They evolve a population of candidate solutions over multiple generations to find an optimal or near-optimal solution.\n", "\n", "#### **1. Population ($P$)**\n", "A **population** consists of multiple candidate solutions (individuals). Each individual represents a **policy or solution** encoded as a chromosome.\n", "\n", "$$[\n", "P_t = \\{ X_1^t, X_2^t, ..., X_N^t \\}\n", "]$$\n", "\n", "where:\n", "- $( P_t )$ is the population at generation $( t )$,\n", "- $( X_i^t )$ is the $( i )$-th individual in the population,\n", "- $( N )$ is the population size.\n", "\n", "#### **2. Fitness Function ($F$)**\n", "Each individual is evaluated using a **fitness function**, which measures how good a solution is.\n", "\n", "$$[\n", "F(X_i) = \\text{reward or performance score of } X_i\n", "]$$\n", "\n", "#### **3. Selection**\n", "The best individuals are selected based on their fitness scores to produce the next generation. Common methods include:\n", "- **Roulette Wheel Selection** (Probability proportional to fitness)\n", "- **Tournament Selection** (Select the best out of a subset)\n", "\n", "#### **4. Crossover (Recombination)**\n", "Two parents **combine genetic information** to create offspring. A common method is **single-point crossover**, where a random crossover point is chosen.\n", "\n", "$$\n", "\\begin{aligned}\n", "\\text{Parent 1} &= (A_1, A_2, | A_3, A_4, A_5) \\\\\n", "\\text{Parent 2} &= (B_1, B_2, | B_3, B_4, B_5) \\\\\n", "\\text{Offspring 1} &= (A_1, A_2, | B_3, B_4, B_5) \\\\\n", "\\text{Offspring 2} &= (B_1, B_2, | A_3, A_4, A_5)\n", "\\end{aligned}\n", "$$\n", "\n", "#### **5. Mutation**\n", "Mutation introduces small **random changes** in individuals to maintain diversity and avoid local optima. If $( X_i )$ is an individual, a mutation function $( M )$ alters some genes:\n", "\n", "$$[\n", "X_i' = M(X_i)\n", "]$$\n", "\n", "For example, if an individual’s policy is `['up', 'right', 'down']`, mutation might randomly change `right` to `left`.\n", "\n", "#### **6. Generations & Evolution**\n", "The new population is formed after selection, crossover, and mutation. The process repeats for **multiple generations** until a stopping criterion is met (e.g., max generations or convergence).\n", "\n", "$$[\n", "P_{t+1} = \\text{next\\_generation}(P_t)\n", "]$$\n", "\n" ] }, { "cell_type": "markdown", "metadata": { "id": "BhuDXH6gVkW6" }, "source": [ "### Genetic Algorithm Implementation" ] }, { "cell_type": "code", "execution_count": 16, "metadata": { "id": "dTb0hWMKVkW6" }, "outputs": [], "source": [ "class GeneticAlgorithm:\n", " def __init__(self, population_size, mutation_rate, crossover_rate, policy_network, generations=100, device='cpu'):\n", " self.device = device\n", " self.population_size = population_size\n", " self.mutation_rate = mutation_rate\n", " self.crossover_rate = crossover_rate\n", " self.generations = generations\n", " self.policy_network = policy_network.to(self.device) # Move the model to GPU\n", " self.population = [self._initialize_individual() for _ in range(population_size)]\n", "\n", " def _initialize_individual(self):\n", " # Initialize an individual by setting random weights for the policy network\n", " individual = {}\n", " for name, param in self.policy_network.named_parameters():\n", " individual[name] = torch.randn_like(param).to(self.device) # Move individual weights to GPU\n", " return individual\n", "\n", " def _evaluate_individual(self, individual):\n", " # Simulate the evaluation (for example, run a few episodes in GridWorld environment)\n", " self._apply_individual_weights(individual)\n", " state = (0, 0)\n", " STOP = 0\n", " trajectory, rewards = [], []\n", " while state != GOAL and STOP <= 150:\n", " state_tensor = torch.tensor(state, dtype=torch.float32).to(self.device) # Move state to GPU\n", " log_action_probs = self.policy_network(state_tensor)\n", " action_probs = torch.exp(log_action_probs)\n", " action_idx = torch.argmax(action_probs).item() # Get action with highest probability\n", " action = ACTIONS[action_idx] # Map action index back to action\n", " new_state, reward = grid_world.step(state, action)\n", " trajectory.append((state, action, reward))\n", " rewards.append(reward)\n", " state = new_state\n", " STOP += 1\n", " return sum(rewards)\n", "\n", " def _apply_individual_weights(self, individual):\n", " # Apply the weights of an individual to the policy network\n", " for name, param in individual.items():\n", " self.policy_network.state_dict()[name].copy_(param)\n", "\n", " def _select_parents(self):\n", " # Select two individuals based on fitness (higher reward is better)\n", " scores = [(ind, self._evaluate_individual(ind)) for ind in self.population]\n", " sorted_scores = sorted(scores, key=lambda x: x[1], reverse=True)\n", " parent1 = sorted_scores[0][0]\n", " parent2 = sorted_scores[1][0]\n", " return parent1, parent2\n", "\n", " def _crossover(self, parent1, parent2):\n", " # Perform crossover to combine the genetic material of two parents\n", " child = {}\n", " for name in parent1.keys():\n", " if random.random() < self.crossover_rate:\n", " child[name] = parent1[name]\n", " else:\n", " child[name] = parent2[name]\n", " return child\n", "\n", " def _mutate(self, individual):\n", " # Perform mutation (randomly modify the weights)\n", " for name, param in individual.items():\n", " if random.random() < self.mutation_rate:\n", " individual[name] = torch.randn_like(param).to(self.device) # Ensure mutated weight is on GPU\n", " return individual\n", "\n", " def run(self):\n", " for generation in range(self.generations):\n", " new_population = []\n", " for _ in range(self.population_size // 2):\n", " parent1, parent2 = self._select_parents()\n", " child1 = self._crossover(parent1, parent2)\n", " child2 = self._crossover(parent2, parent1)\n", " new_population.append(self._mutate(child1))\n", " new_population.append(self._mutate(child2))\n", " self.population = new_population\n", " print(f\"Generation {generation + 1} completed\")\n", "\n", " def get_optimal_trajectory(self):\n", " state = (0, 0)\n", " trajectory = [state]\n", " rewards = 0\n", " STOP = 0\n", " while state != GOAL and STOP <= 150:\n", " state_tensor = torch.tensor(state, dtype=torch.float32).to(self.device) # Move state to GPU\n", " log_action_probs = self.policy_network(state_tensor)\n", " action_probs = torch.exp(log_action_probs)\n", " action_idx = torch.argmax(action_probs).item() # Get action with highest probability\n", " action = ACTIONS[action_idx] # Map action index back to action\n", " state, reward = grid_world.step(state, action)\n", " rewards += reward\n", " trajectory.append(state)\n", " STOP += 1\n", " return trajectory, rewards" ] }, { "cell_type": "markdown", "metadata": { "id": "y2_rrmBCVkW6" }, "source": [ "# Running & Comparing Agents (20 Points)" ] }, { "cell_type": "code", "execution_count": 21, "metadata": { "colab": { "base_uri": "https://localhost:8080/" }, "id": "ISBNYDieVkW6", "outputId": "f45571eb-d591-4b70-9fcd-f71046539468" }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Episode 0: Total Reward = -386, Epsilon = 0.9995\n", "Episode 50: Total Reward = -296, Epsilon = 0.9748\n", "Episode 100: Total Reward = -10, Epsilon = 0.9507\n", "Episode 150: Total Reward = -281, Epsilon = 0.9273\n", "Episode 200: Total Reward = -48, Epsilon = 0.9044\n", "Episode 250: Total Reward = -80, Epsilon = 0.8820\n", "Episode 300: Total Reward = -22, Epsilon = 0.8602\n", "Episode 350: Total Reward = -366, Epsilon = 0.8390\n", "Episode 400: Total Reward = -8, Epsilon = 0.8183\n", "Episode 450: Total Reward = -240, Epsilon = 0.7981\n", "Episode 500: Total Reward = -376, Epsilon = 0.7784\n", "Episode 550: Total Reward = 40, Epsilon = 0.7591\n", "Episode 600: Total Reward = -232, Epsilon = 0.7404\n", "Episode 650: Total Reward = -28, Epsilon = 0.7221\n", "Episode 700: Total Reward = 0, Epsilon = 0.7043\n", "Episode 750: Total Reward = -74, Epsilon = 0.6869\n", "Episode 800: Total Reward = -386, Epsilon = 0.6699\n", "Episode 850: Total Reward = -356, Epsilon = 0.6534\n", "Episode 900: Total Reward = 42, Epsilon = 0.6372\n", "Episode 950: Total Reward = -396, Epsilon = 0.6215\n", "Episode 1000: Total Reward = 32, Epsilon = 0.6062\n", "Episode 1050: Total Reward = -210, Epsilon = 0.5912\n", "Episode 1100: Total Reward = -351, Epsilon = 0.5766\n", "Episode 1150: Total Reward = -366, Epsilon = 0.5623\n", "Episode 1200: Total Reward = -132, Epsilon = 0.5485\n", "Episode 1250: Total Reward = -336, Epsilon = 0.5349\n", "Episode 1300: Total Reward = 26, Epsilon = 0.5217\n", "Episode 1350: Total Reward = -288, Epsilon = 0.5088\n", "Episode 1400: Total Reward = -68, Epsilon = 0.4963\n", "Episode 1450: Total Reward = 48, Epsilon = 0.4840\n", "Episode 1500: Total Reward = -184, Epsilon = 0.4720\n", "Episode 1550: Total Reward = -436, Epsilon = 0.4604\n", "Episode 1600: Total Reward = 50, Epsilon = 0.4490\n", "Episode 1650: Total Reward = -446, Epsilon = 0.4379\n", "Episode 1700: Total Reward = -64, Epsilon = 0.4271\n", "Episode 1750: Total Reward = 30, Epsilon = 0.4166\n", "Episode 1800: Total Reward = -162, Epsilon = 0.4063\n", "Episode 1850: Total Reward = -226, Epsilon = 0.3962\n", "Episode 1900: Total Reward = -304, Epsilon = 0.3865\n", "Episode 1950: Total Reward = -148, Epsilon = 0.3769\n", "Episode 2000: Total Reward = -426, Epsilon = 0.3676\n", "Episode 2050: Total Reward = -344, Epsilon = 0.3585\n", "Episode 2100: Total Reward = -130, Epsilon = 0.3497\n", "Episode 2150: Total Reward = -466, Epsilon = 0.3410\n", "Episode 2200: Total Reward = -376, Epsilon = 0.3326\n", "Episode 2250: Total Reward = -436, Epsilon = 0.3244\n", "Episode 2300: Total Reward = -461, Epsilon = 0.3164\n", "Episode 2350: Total Reward = -376, Epsilon = 0.3086\n", "Episode 2400: Total Reward = -461, Epsilon = 0.3010\n", "Episode 2450: Total Reward = -142, Epsilon = 0.2935\n", "Episode 2500: Total Reward = -276, Epsilon = 0.2863\n", "Episode 2550: Total Reward = -240, Epsilon = 0.2792\n", "Episode 2600: Total Reward = -496, Epsilon = 0.2723\n", "Episode 2650: Total Reward = -42, Epsilon = 0.2656\n", "Episode 2700: Total Reward = -66, Epsilon = 0.2590\n", "Episode 2750: Total Reward = -196, Epsilon = 0.2526\n", "Episode 2800: Total Reward = -461, Epsilon = 0.2464\n", "Episode 2850: Total Reward = -4, Epsilon = 0.2403\n", "Episode 2900: Total Reward = -376, Epsilon = 0.2344\n", "Episode 2950: Total Reward = -12, Epsilon = 0.2286\n", "Episode 3000: Total Reward = -136, Epsilon = 0.2229\n", "Episode 3050: Total Reward = -94, Epsilon = 0.2174\n", "Episode 3100: Total Reward = -344, Epsilon = 0.2121\n", "Episode 3150: Total Reward = -210, Epsilon = 0.2068\n", "Episode 3200: Total Reward = -371, Epsilon = 0.2017\n", "Episode 3250: Total Reward = -104, Epsilon = 0.1967\n", "Episode 3300: Total Reward = 54, Epsilon = 0.1919\n", "Episode 3350: Total Reward = -271, Epsilon = 0.1871\n", "Episode 3400: Total Reward = -74, Epsilon = 0.1825\n", "Episode 3450: Total Reward = -521, Epsilon = 0.1780\n", "Episode 3500: Total Reward = -18, Epsilon = 0.1736\n", "Episode 3550: Total Reward = 4, Epsilon = 0.1693\n", "Episode 3600: Total Reward = -148, Epsilon = 0.1651\n", "Episode 3650: Total Reward = -382, Epsilon = 0.1611\n", "Episode 3700: Total Reward = -511, Epsilon = 0.1571\n", "Episode 3750: Total Reward = -22, Epsilon = 0.1532\n", "Episode 3800: Total Reward = -486, Epsilon = 0.1494\n", "Episode 3850: Total Reward = -526, Epsilon = 0.1457\n", "Episode 3900: Total Reward = -496, Epsilon = 0.1421\n", "Episode 3950: Total Reward = -546, Epsilon = 0.1386\n", "Episode 4000: Total Reward = -110, Epsilon = 0.1352\n", "Episode 4050: Total Reward = -110, Epsilon = 0.1319\n", "Episode 4100: Total Reward = 34, Epsilon = 0.1286\n", "Episode 4150: Total Reward = 88, Epsilon = 0.1254\n", "Episode 4200: Total Reward = 26, Epsilon = 0.1223\n", "Episode 4250: Total Reward = 82, Epsilon = 0.1193\n", "Episode 4300: Total Reward = -28, Epsilon = 0.1164\n", "Episode 4350: Total Reward = -24, Epsilon = 0.1135\n", "Episode 4400: Total Reward = 40, Epsilon = 0.1107\n", "Episode 4450: Total Reward = -180, Epsilon = 0.1080\n", "Episode 4500: Total Reward = -541, Epsilon = 0.1053\n", "Episode 4550: Total Reward = -356, Epsilon = 0.1027\n", "Episode 4600: Total Reward = 82, Epsilon = 0.1002\n", "Episode 4650: Total Reward = 86, Epsilon = 0.1000\n", "Episode 4700: Total Reward = 68, Epsilon = 0.1000\n", "Episode 4750: Total Reward = 58, Epsilon = 0.1000\n", "Episode 4800: Total Reward = 88, Epsilon = 0.1000\n", "Episode 4850: Total Reward = 86, Epsilon = 0.1000\n", "Episode 4900: Total Reward = 46, Epsilon = 0.1000\n", "Episode 4950: Total Reward = 88, Epsilon = 0.1000\n", "Episode 5000: Total Reward = 82, Epsilon = 0.1000\n", "Episode 5050: Total Reward = 88, Epsilon = 0.1000\n", "Episode 5100: Total Reward = 66, Epsilon = 0.1000\n", "Episode 5150: Total Reward = 88, Epsilon = 0.1000\n", "Episode 5200: Total Reward = 88, Epsilon = 0.1000\n", "Episode 5250: Total Reward = 80, Epsilon = 0.1000\n", "Episode 5300: Total Reward = 58, Epsilon = 0.1000\n", "Episode 5350: Total Reward = 80, Epsilon = 0.1000\n", "Episode 5400: Total Reward = 86, Epsilon = 0.1000\n", "Episode 5450: Total Reward = 88, Epsilon = 0.1000\n", "Episode 5500: Total Reward = 66, Epsilon = 0.1000\n", "Episode 5550: Total Reward = 88, Epsilon = 0.1000\n", "Episode 5600: Total Reward = 88, Epsilon = 0.1000\n", "Episode 5650: Total Reward = 82, Epsilon = 0.1000\n", "Episode 5700: Total Reward = 46, Epsilon = 0.1000\n", "Episode 5750: Total Reward = 82, Epsilon = 0.1000\n", "Episode 5800: Total Reward = 58, Epsilon = 0.1000\n", "Episode 5850: Total Reward = -2, Epsilon = 0.1000\n", "Episode 5900: Total Reward = 76, Epsilon = 0.1000\n", "Episode 5950: Total Reward = 76, Epsilon = 0.1000\n" ] } ], "source": [ "# Train REINFORCE agent\n", "\n", "LEARNING_RATE = 0.001\n", "GAMMA = 0.99\n", "\n", "reinforce_agent = ReinforceAgent(lr = LEARNING_RATE, gamma = GAMMA)\n", "reinforce_agent.train(episodes=6000, epsilon=1.0, epsilon_decay=0.9995, min_epsilon=0.1)" ] }, { "cell_type": "code", "execution_count": 35, "metadata": { "colab": { "base_uri": "https://localhost:8080/" }, "id": "_u-6R1O0VkW7", "outputId": "fb09e8bd-d6ac-4be6-a307-3e2d75219a79" }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Generation 1 completed\n", "Generation 2 completed\n", "Generation 3 completed\n", "Generation 4 completed\n", "Generation 5 completed\n", "Generation 6 completed\n", "Generation 7 completed\n", "Generation 8 completed\n", "Generation 9 completed\n", "Generation 10 completed\n", "Generation 11 completed\n", "Generation 12 completed\n", "Generation 13 completed\n", "Generation 14 completed\n", "Generation 15 completed\n", "Generation 16 completed\n", "Generation 17 completed\n", "Generation 18 completed\n", "Generation 19 completed\n", "Generation 20 completed\n", "Generation 21 completed\n", "Generation 22 completed\n", "Generation 23 completed\n", "Generation 24 completed\n", "Generation 25 completed\n", "Generation 26 completed\n", "Generation 27 completed\n", "Generation 28 completed\n", "Generation 29 completed\n", "Generation 30 completed\n", "Generation 31 completed\n", "Generation 32 completed\n", "Generation 33 completed\n", "Generation 34 completed\n", "Generation 35 completed\n", "Generation 36 completed\n", "Generation 37 completed\n", "Generation 38 completed\n", "Generation 39 completed\n", "Generation 40 completed\n", "Generation 41 completed\n", "Generation 42 completed\n", "Generation 43 completed\n", "Generation 44 completed\n", "Generation 45 completed\n", "Generation 46 completed\n", "Generation 47 completed\n", "Generation 48 completed\n", "Generation 49 completed\n", "Generation 50 completed\n", "Generation 51 completed\n", "Generation 52 completed\n", "Generation 53 completed\n", "Generation 54 completed\n", "Generation 55 completed\n", "Generation 56 completed\n", "Generation 57 completed\n", "Generation 58 completed\n", "Generation 59 completed\n", "Generation 60 completed\n", "Generation 61 completed\n", "Generation 62 completed\n", "Generation 63 completed\n", "Generation 64 completed\n", "Generation 65 completed\n", "Generation 66 completed\n", "Generation 67 completed\n", "Generation 68 completed\n", "Generation 69 completed\n", "Generation 70 completed\n", "Generation 71 completed\n", "Generation 72 completed\n", "Generation 73 completed\n", "Generation 74 completed\n", "Generation 75 completed\n", "Generation 76 completed\n", "Generation 77 completed\n", "Generation 78 completed\n", "Generation 79 completed\n", "Generation 80 completed\n", "Generation 81 completed\n", "Generation 82 completed\n", "Generation 83 completed\n", "Generation 84 completed\n", "Generation 85 completed\n", "Generation 86 completed\n", "Generation 87 completed\n", "Generation 88 completed\n", "Generation 89 completed\n", "Generation 90 completed\n", "Generation 91 completed\n", "Generation 92 completed\n", "Generation 93 completed\n", "Generation 94 completed\n", "Generation 95 completed\n", "Generation 96 completed\n", "Generation 97 completed\n", "Generation 98 completed\n", "Generation 99 completed\n", "Generation 100 completed\n", "Generation 101 completed\n", "Generation 102 completed\n", "Generation 103 completed\n", "Generation 104 completed\n", "Generation 105 completed\n", "Generation 106 completed\n", "Generation 107 completed\n", "Generation 108 completed\n", "Generation 109 completed\n", "Generation 110 completed\n", "Generation 111 completed\n", "Generation 112 completed\n", "Generation 113 completed\n", "Generation 114 completed\n", "Generation 115 completed\n", "Generation 116 completed\n", "Generation 117 completed\n", "Generation 118 completed\n", "Generation 119 completed\n", "Generation 120 completed\n", "Generation 121 completed\n", "Generation 122 completed\n", "Generation 123 completed\n", "Generation 124 completed\n", "Generation 125 completed\n", "Generation 126 completed\n", "Generation 127 completed\n", "Generation 128 completed\n", "Generation 129 completed\n", "Generation 130 completed\n", "Generation 131 completed\n", "Generation 132 completed\n", "Generation 133 completed\n", "Generation 134 completed\n", "Generation 135 completed\n", "Generation 136 completed\n", "Generation 137 completed\n", "Generation 138 completed\n", "Generation 139 completed\n", "Generation 140 completed\n", "Generation 141 completed\n", "Generation 142 completed\n", "Generation 143 completed\n", "Generation 144 completed\n", "Generation 145 completed\n", "Generation 146 completed\n", "Generation 147 completed\n", "Generation 148 completed\n", "Generation 149 completed\n", "Generation 150 completed\n", "Generation 151 completed\n", "Generation 152 completed\n", "Generation 153 completed\n", "Generation 154 completed\n", "Generation 155 completed\n", "Generation 156 completed\n", "Generation 157 completed\n", "Generation 158 completed\n", "Generation 159 completed\n", "Generation 160 completed\n", "Generation 161 completed\n", "Generation 162 completed\n", "Generation 163 completed\n", "Generation 164 completed\n", "Generation 165 completed\n", "Generation 166 completed\n", "Generation 167 completed\n", "Generation 168 completed\n", "Generation 169 completed\n", "Generation 170 completed\n", "Generation 171 completed\n", "Generation 172 completed\n", "Generation 173 completed\n", "Generation 174 completed\n", "Generation 175 completed\n", "Generation 176 completed\n", "Generation 177 completed\n", "Generation 178 completed\n", "Generation 179 completed\n", "Generation 180 completed\n", "Generation 181 completed\n", "Generation 182 completed\n", "Generation 183 completed\n", "Generation 184 completed\n", "Generation 185 completed\n", "Generation 186 completed\n", "Generation 187 completed\n", "Generation 188 completed\n", "Generation 189 completed\n", "Generation 190 completed\n", "Generation 191 completed\n", "Generation 192 completed\n", "Generation 193 completed\n", "Generation 194 completed\n", "Generation 195 completed\n", "Generation 196 completed\n", "Generation 197 completed\n", "Generation 198 completed\n", "Generation 199 completed\n", "Generation 200 completed\n", "Generation 201 completed\n", "Generation 202 completed\n", "Generation 203 completed\n", "Generation 204 completed\n", "Generation 205 completed\n", "Generation 206 completed\n", "Generation 207 completed\n", "Generation 208 completed\n", "Generation 209 completed\n", "Generation 210 completed\n", "Generation 211 completed\n", "Generation 212 completed\n", "Generation 213 completed\n", "Generation 214 completed\n", "Generation 215 completed\n", "Generation 216 completed\n", "Generation 217 completed\n", "Generation 218 completed\n", "Generation 219 completed\n", "Generation 220 completed\n", "Generation 221 completed\n", "Generation 222 completed\n", "Generation 223 completed\n", "Generation 224 completed\n", "Generation 225 completed\n", "Generation 226 completed\n", "Generation 227 completed\n", "Generation 228 completed\n", "Generation 229 completed\n", "Generation 230 completed\n", "Generation 231 completed\n", "Generation 232 completed\n", "Generation 233 completed\n", "Generation 234 completed\n", "Generation 235 completed\n", "Generation 236 completed\n", "Generation 237 completed\n", "Generation 238 completed\n", "Generation 239 completed\n", "Generation 240 completed\n", "Generation 241 completed\n", "Generation 242 completed\n", "Generation 243 completed\n", "Generation 244 completed\n", "Generation 245 completed\n", "Generation 246 completed\n", "Generation 247 completed\n", "Generation 248 completed\n", "Generation 249 completed\n", "Generation 250 completed\n", "Generation 251 completed\n", "Generation 252 completed\n", "Generation 253 completed\n", "Generation 254 completed\n", "Generation 255 completed\n", "Generation 256 completed\n", "Generation 257 completed\n", "Generation 258 completed\n", "Generation 259 completed\n", "Generation 260 completed\n", "Generation 261 completed\n", "Generation 262 completed\n", "Generation 263 completed\n", "Generation 264 completed\n", "Generation 265 completed\n", "Generation 266 completed\n", "Generation 267 completed\n", "Generation 268 completed\n", "Generation 269 completed\n", "Generation 270 completed\n", "Generation 271 completed\n", "Generation 272 completed\n", "Generation 273 completed\n", "Generation 274 completed\n", "Generation 275 completed\n", "Generation 276 completed\n", "Generation 277 completed\n", "Generation 278 completed\n", "Generation 279 completed\n", "Generation 280 completed\n", "Generation 281 completed\n", "Generation 282 completed\n", "Generation 283 completed\n", "Generation 284 completed\n", "Generation 285 completed\n", "Generation 286 completed\n", "Generation 287 completed\n", "Generation 288 completed\n", "Generation 289 completed\n", "Generation 290 completed\n", "Generation 291 completed\n", "Generation 292 completed\n", "Generation 293 completed\n", "Generation 294 completed\n", "Generation 295 completed\n", "Generation 296 completed\n", "Generation 297 completed\n", "Generation 298 completed\n", "Generation 299 completed\n", "Generation 300 completed\n", "Generation 301 completed\n", "Generation 302 completed\n", "Generation 303 completed\n", "Generation 304 completed\n", "Generation 305 completed\n", "Generation 306 completed\n", "Generation 307 completed\n", "Generation 308 completed\n", "Generation 309 completed\n", "Generation 310 completed\n", "Generation 311 completed\n", "Generation 312 completed\n", "Generation 313 completed\n", "Generation 314 completed\n", "Generation 315 completed\n", "Generation 316 completed\n", "Generation 317 completed\n", "Generation 318 completed\n", "Generation 319 completed\n", "Generation 320 completed\n", "Generation 321 completed\n", "Generation 322 completed\n", "Generation 323 completed\n", "Generation 324 completed\n", "Generation 325 completed\n", "Generation 326 completed\n", "Generation 327 completed\n", "Generation 328 completed\n", "Generation 329 completed\n", "Generation 330 completed\n", "Generation 331 completed\n", "Generation 332 completed\n", "Generation 333 completed\n", "Generation 334 completed\n", "Generation 335 completed\n", "Generation 336 completed\n", "Generation 337 completed\n", "Generation 338 completed\n", "Generation 339 completed\n", "Generation 340 completed\n", "Generation 341 completed\n", "Generation 342 completed\n", "Generation 343 completed\n", "Generation 344 completed\n", "Generation 345 completed\n", "Generation 346 completed\n", "Generation 347 completed\n", "Generation 348 completed\n", "Generation 349 completed\n", "Generation 350 completed\n", "Generation 351 completed\n", "Generation 352 completed\n", "Generation 353 completed\n", "Generation 354 completed\n", "Generation 355 completed\n", "Generation 356 completed\n", "Generation 357 completed\n", "Generation 358 completed\n", "Generation 359 completed\n", "Generation 360 completed\n", "Generation 361 completed\n", "Generation 362 completed\n", "Generation 363 completed\n", "Generation 364 completed\n", "Generation 365 completed\n", "Generation 366 completed\n", "Generation 367 completed\n", "Generation 368 completed\n", "Generation 369 completed\n", "Generation 370 completed\n", "Generation 371 completed\n", "Generation 372 completed\n", "Generation 373 completed\n", "Generation 374 completed\n", "Generation 375 completed\n", "Generation 376 completed\n", "Generation 377 completed\n", "Generation 378 completed\n", "Generation 379 completed\n", "Generation 380 completed\n", "Generation 381 completed\n", "Generation 382 completed\n", "Generation 383 completed\n", "Generation 384 completed\n", "Generation 385 completed\n", "Generation 386 completed\n", "Generation 387 completed\n", "Generation 388 completed\n", "Generation 389 completed\n", "Generation 390 completed\n", "Generation 391 completed\n", "Generation 392 completed\n", "Generation 393 completed\n", "Generation 394 completed\n", "Generation 395 completed\n", "Generation 396 completed\n", "Generation 397 completed\n", "Generation 398 completed\n", "Generation 399 completed\n", "Generation 400 completed\n" ] } ], "source": [ "# Train Genetic Algorithm agent\n", "\n", "POPULATION_SIZE = 50\n", "GENERATIONS = 400\n", "MUTATION_RATE = 0.3\n", "CROSSOVER_RATE = 0.7\n", "\n", "policy_network = PolicyNetwork()\n", "genetic_agent = GeneticAlgorithm(generations=GENERATIONS, population_size=POPULATION_SIZE, mutation_rate=MUTATION_RATE, crossover_rate=CROSSOVER_RATE, policy_network=policy_network, device=device)\n", "optimal_genetic_policy = genetic_agent.run()" ] }, { "cell_type": "markdown", "metadata": { "id": "SLk5xtH9VkW7" }, "source": [ "# Visualizing Results (50 Points)" ] }, { "cell_type": "markdown", "metadata": { "id": "Es2e9ubnVkW7" }, "source": [ "Plots the agent’s trajectory in the Grid World." ] }, { "cell_type": "code", "execution_count": 26, "metadata": { "id": "IEbA7yzOVkW7" }, "outputs": [], "source": [ "def visualize_trajectory(trajectory, title):\n", " plt.figure(figsize=(5, 5))\n", " x_vals, y_vals = zip(*trajectory)\n", " plt.plot(x_vals, y_vals, marker='o', color='blue', linestyle='-', alpha=0.7)\n", " plt.scatter(0, 0, c='green', marker='s', label='Start')\n", " plt.scatter(GOAL[0], GOAL[1], c='red', marker='X', label='Goal')\n", " for i, (px, py) in enumerate(PENALTIES):\n", " if i == 0:\n", " plt.scatter(px, py, c='orange', marker='D', label='Penalty')\n", " else:\n", " plt.scatter(px, py, c='orange', marker='D')\n", " plt.xticks(range(GRID_SIZE))\n", " plt.yticks(range(GRID_SIZE))\n", " plt.title(title)\n", " plt.legend()\n", " plt.grid(True)\n", " plt.show()" ] }, { "cell_type": "markdown", "metadata": { "id": "g45-MsSlVkW7" }, "source": [ "### Results" ] }, { "cell_type": "code", "execution_count": 36, "metadata": { "colab": { "base_uri": "https://localhost:8080/", "height": 1000 }, "id": "s7q8Cm2AVkW7", "outputId": "78cecc78-84a4-4b5d-e1b7-8d46dbca6097" }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Final Reward (REINFORCE): 88\n", "\n" ] }, { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAaoAAAHDCAYAAAB1dF5kAAAAOnRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjEwLjAsIGh0dHBzOi8vbWF0cGxvdGxpYi5vcmcvlHJYcgAAAAlwSFlzAAAPYQAAD2EBqD+naQAAR7dJREFUeJzt3Xl8VNXB//HPnWSykxCWsMgSCKAsYRNZgiwqm1IKtiovgbKpWEoERVoen6ePgvYpatGKQlG0gJVStLagVUEiCFSRnWgMiBCC8FMgYctKkiFzf3+MmRKSQCZkkpuZ7/v1yiu5Z86995zZvnPPPbljmKZpIiIiYlG22m6AiIjI1SioRETE0hRUIiJiaQoqERGxNAWViIhYmoJKREQsTUElIiKWpqASERFLU1CJiIilKahEfMjkyZOJjY31m30fO3YMwzBYuXKlu2zevHkYhlGj7RDvUlD5iJUrV2IYhvsnMDCQG264gcmTJ/P999+XqT948OBS9S//uemmm8psd8+ePe6ykjeCJk2akJ+fX2bbsbGx/OQnPylVVtG+mjZtWmb9zz//nLvvvpsmTZoQHBxMbGwsDz/8MMePHy9Tt6QtJT92u53Y2FhmzpzJhQsXyr2v1q5dy5133kmjRo0ICgqiefPm3HfffWzevNldZ8uWLRW22TAM1qxZU+62r/TBBx8wYsQIGjZsSEhICB06dGDOnDmcPXu2UuuX54cffmDevHkkJydXeRu16crnXoMGDbjllltYvnw5TqeztpsnFhRY2w2Q6vX000/Tpk0bCgoK2LFjBytXruSzzz7j66+/JiQkpFTdFi1asGDBgjLbiIqKqtS+MjIyWLp0KY8//nil6g8dOpSJEyeWKgsNDS21/MorrzBr1izatm3LI488QrNmzTh48CBvvPEGb7/9Nh999BEJCQlltr106VIiIiLIy8tj06ZNvPLKK+zbt4/PPvvMXcc0TaZOncrKlSvp0aMHs2fPpmnTppw8eZK1a9dyxx138Pnnn5fa/syZM7nlllvK7K9fv37X7O+cOXN44YUX6NatG3PnzqVBgwbs27ePxYsXs2bNGjZt2sSNN954ze1c6YcffmD+/PnExsbSvXv3Ure9/vrrdeLN/vLnXmZmJn/5y1944IEH+Pbbb3n22Weva9u//e1v+a//+q/qaKZYhSk+YcWKFSZg7t69u1T53LlzTcB8++23S5UPGjTI7Ny5c5W2+9RTT5mA2b17d7NJkyZmfn5+qXVat25tjhw5slQZYM6YMeOq+/rss89Mm81mDhgwwMzLyyt125EjR8wmTZqYzZo1M8+dO1emLZmZmaXqjx071gTMnTt3usv+8Ic/mID56KOPmk6ns8z+//KXv7jrf/rppyZg/v3vf79qmyuyevVqEzDHjh1rXrp0qdRtO3fuNMPCwsz4+HjT4XB4vO3du3ebgLlixYoqtc1bJk2aZLZu3fqa9cp77uXl5ZktWrQww8PDzaKiokrvMz093ZL3hVQvDf35uAEDBgCQlpZW7dt+8sknOX36NEuXLq2W7T3zzDMYhsGbb75JWFhYqdvi4uJ4/vnnOXnyJK+99to1t3Vlvy9evMiCBQu46aabWLhwYbnnMH7xi1/Qu3fvaugJzJ8/n+joaJYtW0ZAQECp23r37s3cuXNJSUnh3XffdZcPHjyYLl26sHfvXhISEggNDaVNmza8+uqr7jpbtmxxH+FNmTLFPXxWco7myvNEJedwFi5cyJIlS2jbti1hYWEMGzaMEydOYJomzzzzDC1atCA0NJTRo0dz7ty5Uu197733GDlyJM2bNyc4OJi4uDieeeYZiouLq+W+AggLC6Nv377k5eWRmZkJwNGjR7n33ntp0KCB+/YPP/zwmtuq6BzVqlWr6N27N2FhYURHRzNw4EA2btwIwKRJk2jUqBEOh6PMesOGDavSka9UHwWVjzt27BgA0dHRZW4rLi7mzJkzZX7y8vIqte0BAwZw++238/zzz3Px4sVr1i8oKCizr8LCQgDy8/PZtGkTAwYMoE2bNuWuP3bsWIKDg/nggw+uua8r+/3ZZ59x7tw5xo0bVyY4riYnJ6fc+8i8yrfjHD58mEOHDjF69GgiIyPLrVMyBHplX86fP89dd93FzTffzPPPP0+LFi2YPn06y5cvB6Bjx448/fTTAEybNo233nqLt956i4EDB161H3/961/505/+xCOPPMLjjz/O1q1bue+++/jtb3/Lhg0bmDt3LtOmTeNf//oXc+bMKbXuypUriYiIYPbs2SxatIibb76ZJ598stqH144ePUpAQAD169fn9OnTJCQk8PHHH/OrX/2K//u//6OgoICf/vSnrF271uNtz58/n1/84hfY7Xaefvpp5s+fT8uWLd3nJX/xi19w9uxZPv7441LrnTp1is2bNzNhwoRq6aNUUW0f0kn1KBmi++STT8zMzEzzxIkT5rvvvms2btzYDA4ONk+cOFGq/qBBg0yg3J+HH364zHbLG/rLzMw0t27dagLmiy++6L69oqG/8n5KhmySk5NNwJw1a9ZV+9m1a1ezQYMGZdpy6NAhMzMz0zx27Ji5fPlyMzQ01GzcuLF7CHHRokUmYK5du7ZS92fJ0F9FPydPnqxw3XXr1pmA+cc//vGq+4iMjDR79uzpXi55TF544QV3WWFhodm9e3czJibGPSR2taG/K4ffSobGGjdubF64cMFd/sQTT5iA2a1bt1LDj/fff78ZFBRkFhQUuMuuHNo1TdN8+OGHzbCwsFL1PBn6u+mmm8zMzEwzMzPTPHjwoDlz5kwTMEeNGmWapmk++uijJmD++9//dq+Xk5NjtmnTxoyNjTWLi4tL9e/y+6LkOVHi8OHDps1mM++++273eiVKhoCLi4vNFi1amGPHji11+4svvmgahmEePXr0mv0S79FkCh8zZMiQUsuxsbGsWrWKFi1alKkbGxvL66+/Xqa8vLoVGThwILfddhvPP/88v/zlL8tMjrjc6NGjSUxMLFXWuXNnwHXkAlCvXr2r7q9evXpkZ2eXKb9yaCY+Pp4VK1a4hxBL1rnW9q/05JNPuocRL9egQYMK17mevgQGBvLwww+7l4OCgnj44YeZPn06e/fupW/fvp403+3ee+8tNUmmT58+AEyYMIHAwMBS5X/729/4/vvvadu2LVB6wktOTg6FhYUMGDCA1157jW+++YZu3bp53J5vvvmGxo0bu5cNw2DkyJHuI8ePPvqI3r17c+utt7rrREREMG3aNJ544gkOHDhAly5dKrWvdevW4XQ6efLJJ7HZSg8ilQwR2mw2xo8fz8svv0xOTo77sfvrX/9KQkJChUf5UjMUVD5myZIldOjQgaysLJYvX862bdsIDg4ut254eHiZYKuKefPmMWjQIF599VUee+yxCuu1aNGiwv2VvDGUvMlX5PI3kcv94x//IDIykszMTF5++WXS09NLvcGWDMFda/tXio+P9/g+8qQvMTExpcqaN29OeHh4qbIOHToAruHMqgZVq1atSi2XhFbLli3LLT9//ry7LDU1ld/+9rds3ry5TLBmZWVVqT0lH5IMwyAkJIT27duXui++++47d5hermPHju7bKxtUaWlp2Gw2OnXqdNV6EydO5LnnnmPt2rVMnDiRQ4cOsXfv3lLnCKV2KKh8TO/evenVqxcAY8aM4dZbb2XcuHEcOnSIiIgIr+xz4MCBDB482H1UVRXt2rUjMDCQr776qsI6hYWFHDp0yN2/K9vQqFEjAEaNGkV8fDzjx49n79692Gw29/+GpaSkMGbMmCq1sbJK3kyv1pfvvvuO7Ozsa755VpeKzstVVG7+eA7uwoULDBo0iMjISJ5++mni4uIICQlh3759zJ07t8pT4avrQ1J16tSpEzfffDOrVq1i4sSJrFq1iqCgIO67777abprf02QKHxYQEMCCBQv44YcfWLx4sVf3NW/ePE6dOlWpGXnlCQ8P57bbbmPbtm1899135dZ55513KCwsLPPPxFeKiIjgqaeeIjk5mXfeeQeAW2+9lejoaP72t79V62y18nTo0IEOHTqwbt26Co+q/vKXvwCU6csPP/xQZjLLt99+C+CezVeTV13YsmULZ8+eZeXKlcyaNYuf/OQnDBkypNzJOdWpdevWHDp0qEz5N9984769suLi4nA6nRw4cOCadSdOnMjmzZs5efIkq1evZuTIkV7vq1ybgsrHDR48mN69e/PSSy9RUFDgtf0MGjSIwYMH89xzz1V5P7/97W8xTZPJkyeXmUWYnp7Ob37zG5o1a1bqHE5Fxo8fT4sWLXjuuecA1/TnuXPncvDgQebOnVvurL1Vq1axa9euKrX9Sk8++STnz5/nl7/8ZZlg3Lt3L8899xxdunTh5z//eanbLl26VCrsi4qKeO2112jcuDE333wzgHtosKIrb1SnkiOuy++voqIi/vSnP3l1v3fddRe7du3iiy++cJfl5eWxbNkyYmNjPToSHTNmDDabjaeffrrMEeCVz4P7778fwzCYNWsWR48e1Ww/i9DQnx/49a9/zb333svKlStLDc1lZWWxatWqctepygv0qaee4rbbbqtyOwcOHMjChQuZPXs2Xbt2ZfLkyTRr1oxvvvnGfcWFjz76qFKfcO12O7NmzeLXv/41GzZsYMSIEfz6178mNTWVF154gU8//ZR77rmHpk2bcurUKdatW8euXbvYvn17qe38+9//Ljd4u3btSteuXSvc//jx49m9ezeLFi3iwIEDjB8/nujoaPbt28fy5ctp2LAh7777Lna7vdR6zZs357nnnuPYsWN06NCBt99+m+TkZJYtW+auGxcXR/369Xn11VepV68e4eHh9OnTxysn/BMSEoiOjmbSpEnMnDkTwzB46623rjo9vzr813/9F3/729+48847mTlzJg0aNODNN98kPT2df/zjH2UmRVxNu3bt+J//+R+eeeYZBgwYwM9+9jOCg4PZvXs3zZs3L3V1lsaNGzNixAj+/ve/U79+fUaOHOmN7omnanPKoVSfiq5MYZquqbdxcXFmXFyc+yoJV5uefvnT4lrT069Ust2qXJmixLZt28zRo0ebjRo1Mu12u9mqVSvzoYceMo8dO1am7tXakpWVZUZFRZmDBg0qVf7uu++aw4YNMxs0aGAGBgaazZo1M8eOHWtu2bLFXeda09OfeuqpSvVl3bp15tChQ83o6GgzODjYbNeunfn4449XeN917tzZ3LNnj9mvXz8zJCTEbN26tbl48eIydd977z2zU6dOZmBgYKnp2RVNT//DH/5Qav2KrrxR3uP9+eefm3379jVDQ0PN5s2bm7/5zW/Mjz/+2ATMTz/91F3veq5MUZ60tDTznnvuMevXr2+GhISYvXv3Nj/44INSdSozPb3E8uXLzR49epjBwcFmdHS0OWjQIDMpKalMvXfeeccEzGnTpl2zjVIzDNP08kcjEamUwYMHc+bMGb7++uvabopfe++99xgzZgzbtm0r918TpObpHJWIyGVef/112rZtW+p/uKR26RyViAiwZs0avvrqKz788EMWLVqk77SyEAWViAiuGX8RERE88MAD/OpXv6rt5shldI5KREQsTeeoRETE0hRUIiJiaTV+jsrpdPLDDz9Qr149nawUEfFTpmmSk5ND8+bNr/kP3DUeVD/88EOZKzaLiIh/OnHixDW/WqjGg6rkKxBOnDhR4befXovD4WDjxo0MGzaszCVofI366nv8pZ+gvvqi6upndnY2LVu2rNR3xNV4UJUM90VGRl5XUIWFhREZGenTTwhQX32Rv/QT1FdfVN39rMwpIE2mEBERS1NQiYiIpSmoRETE0nQJJalznE4nRUVFtd2MKnM4HAQGBlJQUOD1bxv2JrvdXuFX2YtUJwWV1ClFRUWkp6eX+abWusQ0TZo2bcqJEyfq/P8S1q9fn6ZNm9b5foi1KaikzjBNk5MnTxIQEEDLli09+pZXK3E6neTm5hIREVFn+2CaJvn5+WRkZADQrFmzWm6R+DIFldQZly5dIj8/n+bNmxMWFlbbzamykqHLkJCQOhtUAKGhoQBkZGQQExOjYUDxmrr7KhG/U3I+JygoqJZbIiVKPjA4HI5abon4MgWV1Dk6H2IdeiykJiiorC59VenfIiJ+xuOg+v7775kwYQINGzYkNDSU+Ph49uzZ4422Sdpy2JPo+ntPomtZRKQ2FBbi3PQpKSmQmtqQlBRwbvoUCgu9vmuPgur8+fP0798fu93O+vXrOXDgAC+88ALR0dHeap//SlsOOx8ESr6A2XQtK6zqpMzMTKZPn06rVq0IDQ3lxhtvZMSIEXz++eeAawht3bp11bKvY8eOYRgGycnJ1bI9EQoL2T7oCSYMOcnUERksXdqNqSNOM2HISbYPesLrYeXRrL/nnnuOli1bsmLFCndZmzZtqr1Rfq9MSJX4MawA4qbWdKvqvMNnD5NTlFPh7fWC6tG+YXuv7PvnP/85RUVFvPnmm8TGxnL06FF27tzJ2bNnq3U/dfkfocWifgypOTvv5Rz1aXLmCAE9biRifxr76cGcnW1YOOgJErYugOBgrzTBo6B6//33GT58OPfeey9bt27lhhtu4Fe/+hUPPfSQVxrnlyoMqRIKq6o4fPYwHRZ3uGa9bxO/rfawunDhAv/+97/ZsmULgwYNwul0Eh0dzW233YbNZiM2NhaAu+++G4DWrVtz7Ngx0tLSmD17Njt27CAvL4+OHTuyYMEChgwZ4t52bGwsDzzwAIcPH2bdunX87Gc/48033wSgR48eAAwaNIgtW7ZUa5/Efzg/287inb04R33acYRiw04uEGHkUc88QhrtWLLzZvp+/gW22wd7pQ0eBdXRo0dZunQps2fP5r//+7/ZvXs3M2fOJCgoiEmTJpW7TmFhIYWXHRZmZ2cDrumsVZ3SWrKez02JTV8Fex4BQtxFDkJL/Xbb+QgUG9BmQg020Luu9bg6HA5M08TpdHp8ZYqsgqxK16vuq16EhYURERHB2rVr6d27t3t6fUlfdu7cSdOmTfnzn//MiBEjCAgIwOl0kp2dzYgRI3jmmWcIDg7mrbfeYtSoURw8eJBWrVq5t79w4UL+93//l//93/8FYPr06fTt25eNGzfSuXNngoKCvHYlD6fTiWmaOByOcv+Pymdfq+Xw1b6mRN/KgcanaXImjWLDzl6zJ0VpkTQOsBMAxJgZpDYaTHL9GOI96Lsn95NhmmZFH93LCAoKolevXmzfvt1dNnPmTHbv3s0XX3xR7jrz5s1j/vz5ZcpXr15dp/9pU2peYGAgTZs2pWXLlh7/L9WXGV8y+G+Dr1lvy/1b6BbTrYotrNj777/PrFmzKCgooGvXrvTv35+f/exndOnSBYDo6GhWrVrFyJEjr7qdfv36MWXKFKZNmwZA165d6dq1K6tW/WdW6PHjx+nWrRvbtm0jPj6+2vtyuaKiIk6cOMGpU6e4dOmSV/cltSM1tSFLl3YjJiYPgLS0+gDExV3AZgOnEzIywpk+/Us6d678UHZ+fj7jxo0jKyvrmt9N6NERVbNmzejUqVOpso4dO/KPf/yjwnWeeOIJZs+e7V4u+VbHYcOGXdcXJyYlJTF06FDf+oKy9FU/zvL7z2cHB6EkhS9naN5U7Fy8rLIBvRb73BHV1R7XgoICTpw4QUREBCEhIeVsoWLheeGVqxceXuXn5dVMmDCBe+65h3//+9/s2LGDDz/8kJdffplly5YxefJkwHWlh8v3nZuby/z58/noo484efIkly5d4uLFi2RmZrrr2Ww2+vbtW2q9iIgIr/blcgUFBYSGhjJw4MByHxOffa2Ww1f72rIlvPPSaSL2pxFmXOSE2ZOiyEgap6QQUOwg1wynuFE7Rozogyefi0pG1yrDo6Dq378/hw4dKlX27bff0rp16wrXCQ4OJricE2x2u/26H8zq2IaldJgCAWa556jsXLwsqAzo8wbETanxJtaEih7X4uJiDMPAZrN5fOmhytavyrYrKywsjOHDhzN06FBmzZrF448/zvz585k6dWq5+/7Nb35DUlISCxcupF27doSGhnLPPffgcDhK1bvymoElf3uzL5fvyzCMa74Wfe61ehW+1tfu5z+lU+ZJ9tODNuZRjB/fm4ziYgxnMRnE0DNzC90v3IDNPrjS2/XkPvLoWfzYY4+xY8cOfv/733PkyBFWr17NsmXLmDFjhiebkauJm+oKISr6j/+SkNJEirquU6dO5OW5hlPsdnuZr/z4/PPPmTx5MnfffTfx8fE0bdqUY8eOXXO7JcOidfkrRMQ6bLcmkNhnD9FcII04LhGAaUKuGU4a7YjmPDP67MXWv5/32uBJ5VtuuYW1a9fyt7/9jS5duvDMM8/w0ksvMX78eG+1zz9VGFYKqbro7Nmz3H777axatYqvvvqK9PR01q1bxx/+8AdGjx4NuGbvbdq0iVOnTnH+/HkA2rdvzz//+U+Sk5P58ssvGTduXKUmRcTExBAaGsqGDRs4ffo0WVmVm0giUq7gYBK2LmBhn7/TnWQuBYVTVBRAVsO29GQfC/u869Wp6VCFK1P85Cc/ISUlhYKCAg4ePKip6d5SJqwUUnVVREQEffr04Y9//CMDBw6ka9eu/P73v+fBBx9k8eLFALzwwgskJSXRsmVL97TyF198kejoaBISEhg1ahTDhw+nZ8+e19xfYGAgL7/8Mq+99hrNmzd3h6FIlf0YVivWN6dr72Bat85m2fsxvLXpBq+HFOhrPqwtbqprCvpBXBMnfPScVE2oF1SvWut5Ijg4mAULFrBgwQIA99TzyMhI9zmkUaNGMWrUqFLrxcbGsnnz5lJlVw6zVzQU+OCDD/Lggw9WUw9EgOBgbIMHEvmKk4ICB527gK3e4BrZtYLK6tpMgIMf+dTsvtrQvmF7vk38ttauTCEiVaegEr+hEBKpm/Q1HyIiYmkKKhERsTQFlYiIWJqCSkRELE1BJSIilqagEhERS1NQiYiIpSmoRHxMbGwsL730Um03Q6TaKKhEasipU6eYNWsWHTp0oGnTpjRr1oz+/fuzdOlS8vPza7t5IpalK1OIfykshO3bYfBgMAwwTdiyBRISvHphzaNHj9K/f3/q16/P7373O9q0aUPDhg1JTU1l2bJl3HDDDfz0pz/12v5F6jIdUYn/KCyEMWPg9tvhscdc36H96KOu5TFjXLd7ya9+9SsCAwPZs2cP9913HzfeeCNt27Zl9OjRfPjhh+4L0h4/fpzRo0cTERFBZGQk9913H6dPn3ZvJy0tjdGjR9OkSRMiIiK45ZZb+OSTT7zWbhErUFCJfygJqY0bXcuLFkHPnvDyy67ljRu9FlZnz55l48aNzJgxg/Dw8HLrGIaB0+lk9OjRnDt3jq1bt5KUlMTRo0cZO3asu15ubi533XUXmzZtYv/+/YwYMYJRo0Zx/Pjxam+3iFVo6E/8w/btsGFD6bIvv/zP306n6/YvvnANC1ajI0eOYJomN954Y6nymJgYCgoKANfXdwwZMoSUlBTS09Np2bIlAH/5y1/o3Lkzu3fv5pZbbqFbt25069bNvY1nnnmGtWvX8v7775OYmFit7RaxCh1RiX8YPBhmzrx6nVmzYNCgGmkOwI4dO0hOTqZz584UFhZy8OBBWrZs6Q4pcH1dff369Tl48CDgOqKaM2cOHTt2pH79+kRERHDw4EEdUYlP0xGV+AfDgD/+EbZuLX0kVaJbN3jxRVe9atauXTsMw+DQoUOlytu2bYvNZiM0NLTS25ozZw5JSUksXLiQdu3aERoayj333ENRUVF1N1vEMnREJf7BNF0TKMoLKXCVz57tqlfNGjZsyNChQ1m8eDF5eXkV1uvYsSMnTpzgxIkT7rIDBw5w4cIFOnXqBMDnn3/O5MmTufvuu4mPj6dp06YVfsuviK9QUIl/2LLlPxMnKrJokeuIywv+9Kc/cenSJXr16sXbb7/NoUOHOHToEKtWreKbb74hICCAIUOGEB8fz/jx49m3bx+7du1i4sSJDBo0iF69egHQvn17/vnPf5KcnMyXX37JuHHjcDqdXmmziFUoqMQ/JCTAiBFgu+wpf9mkBGw21+39+nll93Fxcezfv58hQ4bwP//zPwwYMIDevXvzyiuvMGfOHJ555hkMw+C9994jOjqagQMHMmTIENq2bcvbb7/t3s6LL75IdHQ0CQkJjBo1iuHDh9OzZ0+vtFnEKnSOSvxDcDCsW+eagr5hg2vixIsvuoYDX34Zhg1z3e7Ff/pt1qwZr7zyCosWLSI7O5vIyEhsttKfFVu1asV7771X4TZiY2PZvHlzqbIZM2aUWtZQoPgaBZX4j5Kw+uIL1+w+w4CXXoK773YdSXkxpESk6hRU4l+Cg0v/n5RhVPv/TYlI9dI5KhERsTQFlYiIWJqCSkRELE1BJSIilqagEhERS1NQiYiIpSmoRETE0hRU4r+cjtpuQY0xDIN169bVdjNEqkRBJf4pbTm8E+H67WWTJ0/GMAwMwyAoKIgOHTrw/PPPc+nSJa/vuzzHjh3DMAySk5NrZf8inlJQif9JWw47HwRnket3DYTViBEjOHnyJIcPH+axxx7j2WefZeHChV7fr4gvUFCJfykJKUq+d8qskbAKDg6madOmtG7dmunTpzN48GD+9a9/UVhYyJw5c7jhhhsIDw+nT58+bNmyxb3eypUrqV+/Ph9//DEdO3YkIiLCHXoldu/ezdChQ2nUqBFRUVEMGjSIffv2VdiWNm3aANCjRw8Mw2Dw4MFs27YNu93OqVOnStV99NFHGTBgQPXeGSIeUlCJ/ygTUiVqJqwuFxISQlFREYmJiXzxxResWbOGr776invvvZcRI0Zw+PBhd938/HwWLlzIW2+9xbZt2zh+/Dhz5sxx356Tk8OkSZP47LPP2LFjB+3bt+euu+4iJyen3H3v2rULgE8++YSTJ0/yz3/+k4EDB9K2bVveeustdz2Hw8Ff//pXpk6d6qV7QaRyFFTiHyoMqRI1E1amafLJJ5+wefNm4uPjWbFiBX//+98ZMGAAcXFxzJkzh1tvvZUVK1a413E4HLz66qv06tWLnj17kpiYyKZNm9y333777UyYMIGbbrqJjh07smzZMvLz89lawZdANm7cGHB983DTpk1p0KABAA888ECp/f7rX/+ioKCA++67zxt3hUilKajE910zpEp4L6w++OADIiIiCAkJYeTIkdx99938/Oc/p7i4mA4dOhAREeH+2bp1K2lpae51w8LCiIuLcy83a9aMjIwM9/Lp06d56KGHaN++PVFRUURGRpKbm8vx48c9auPkyZM5cuQIO3bsAFzDjvfddx/h4eHX2XuR66Ov+RDf5nTA7ulcO6RKmK76bX4BNnu1NeO2225j6dKlBAUF0bRpU/Lz81m/fj0BAQHs3buXgICAUvUjIiLcf9vtpdthGAam+Z/+TJo0ibNnz7Jo0SJat25NcHAw/fr1o6ioyKM2xsTEMGrUKFasWEGbNm1Yv359qfNlIrVFQSW+zWaHW5ZW8ogKwHDVr8aQAggPD6ddu3YAOJ1OwDWZobi4mIyMjOuasPD555/zpz/9ibvuuguAEydOcObMmQrrBwUFAVBcXFzmtgcffJD777+fFi1aEBcXR//+/avcLpHqoqE/8X1xU6HPG4BxjYqGq15czUwe6NChA+PHj2fixIn885//JD09nV27drFgwQI+/PDDSm+nffv2vPXWWxw8eJCdO3cyfvx4QkNDK6wfExNDaGgoGzZs4PTp02RlZblvGz58OJGRkfzud79jypQp19U/keqioBL/cM2wqtmQKrFixQomTpzI448/zo033siYMWPYvXs3rVq1qvQ2/vznP3P+/Hl69uzJL37xC2bOnElMTEyF9QMDA3n55Zd57bXXaN68OaNHj3bfZrPZmDx5MsXFxUycOPG6+iZSXTT0J/6jJITKDAN6N6RWrlxZ4W12u5358+czf/78cm+fPHkykydPLlU2ZsyYUueoevTowe7du0vVueeee0otX14fXEN8Dz74YLn7/P7777nrrrto1qxZhe0WqUkKKvEvZcKqdo6krCgrK4uUlBRWr17N+++/X9vNEXFTUIn/KQml3dNdEycUUgCMHj2aXbt28ctf/pKhQ4fWdnNE3BRU4p/iplb7FPS6TlPRxao0mUL8l0JKqlP6qtK/pdooqKTOuXJigNQePRY/SlsOexJdf+9JrNHrRtYkpxOysyEnx05qqmu5JngUVPPmzXN/r07Jz0033eSttomUUnL1Bk+vuCDek5+fD5S9eoZfqaUr8te07dthyhT46iuD776LZNq0ACZMcJV7m8fnqDp37swnn3zynw0E6jSX1IzAwEDCwsLIzMzEbrdjs9XNAQGn00lRUREFBQV1tg+maZKfn09GRgb169cvcwkov3GtK/KDT0zW2b4d5syBs2fB9ZZfTFRUAPv3u8oXLoSEBO/t3+OUCQwMpGnTpt5oi8hVGYZBs2bNSE9P57vvvqvt5lSZaZpcvHiR0NBQDONaV8uwtvr16/vv+0Flr8gPdTqsnE5YvBjOnYO4OLhwAYqLISICoqIgLQ2WLIG+fcFbn7s8DqrDhw/TvHlzQkJC6NevHwsWLLjqf9EXFhZSWFjoXs7OzgZcX13gcDiq0GTc61V1/bpEfS3NMAxiY2NxOBx19vzIpUuX2L59OwkJCXV2RMIwDAIDAwkICODSpUsV1vPZ52/6KtjzCBDiLnIQWuq3285HoNiANhNqsIHVJyUFDhwIoEkTME0wTdeHK9M0MU0nMTGQmgrJycXEx1d+u548JwzTg1f7+vXryc3N5cYbb+TkyZPMnz+f77//nq+//pp69eqVu868efPK/a/71atXExYWVumGiohIzUtNbcjSpd2IickDIC2tPgBxcRew2VxHXBkZ4Uyf/iWdO5+t9Hbz8/MZN24cWVlZREZGXrWuR0F1pQsXLtC6dWtefPFFHnjggXLrlHdE1bJlS86cOXPNxlXE4XCQlJTE0KFDff4krvrqe/yln+DDfU1f9eMsv/+8fToIJSl8OUPzpmLn4mWVDei1uE4fUU2dGkBUFISFwd69BkVFhfTtaycgwCA3F7KyYPlyz46osrOzadSoUaWC6rrGHerXr0+HDh04cuRIhXWCg4MJDg4uU26326/7iVsd26gr1Fff4y/9BB/sa4cpEGCWe47KzsXLgqrkEl1190r03btDp06wfz+0aQOG4eqva+a3jYwM6NkTune3eXSOypPnw3Wd+srNzSUtLU0XrxQR/2PRK/JXN5sNEhMhOto1ceLSJde5qtxc13J0NMyY4b2JFOBhUM2ZM4etW7dy7Ngxtm/fzt13301AQAD333+/t9onImJdFYaVb4RUiYQE1xT07t1dQVVUFEBWlutIyttT08HDob//9//+H/fffz9nz56lcePG3HrrrezYsYPGjRt7q30iItbmviL/Iz8W+FZIlUhIcAXV8OEmmZnZLFsWSe/eng33VZVHQbVmzRpvtUNEpO6Km+qagn4Q18SJOnxO6mpsNoiMhIICB507e3e4r9R+a2Y3IiI+rmRWXx2d3WdlCioREbE0BZWIiFiagkpERCxNQSUiIpamoBIREUtTUImIiKUpqERExNIUVCIiYmkKKhERsTQFlYiIWJqCSkRELE1BJSIilqagEhERS1NQiYiIpSmoRETE0hRUIiJiaQoqERGxNAWViIhYmoJKREQsTUElIiKWpqASERFLU1CJiIilKahERMTSFFQiImJpCioREbE0BZWIiFiagkpERCxNQSUiIpamoBIREUtTUImIiKUpqERExNIUVCIiYmkKKhERsTQFlYiIWJqCSkRELE1BJSIilqagEhERS1NQiYiIpSmoRETE0hRUIiJiaQoqERGxNAWViIhYmoJKREQsTUElIiKWpqASEe9JX1X6t0gVKKhExDvSlsOeRNffexJdy1KnOZ2QnQ05OXZSU13LNeG6gurZZ5/FMAweffTRamqOiPiEtOWw80HA/LHAdC0rrOqs7dthyhT46iuD776LZNq0ACZMcJV7W5WDavfu3bz22mt07dq1OtsjInVdmZAqobCqq7ZvhzlzIDkZAgMhKKiYqCjYv99V7u2wqlJQ5ebmMn78eF5//XWio6Oru00iUldVGFIlFFZ1jdMJixfDuXMQF+cKKsOAiAjX8vnzsGSJd4cBA6uy0owZMxg5ciRDhgzhd7/73VXrFhYWUlhY6F7Ozs4GwOFw4HA4qrJ793pVXb8uUV99j8/2M30V7HkECHEXOQgt9dtt5yNQbECbCTXYQO/y1cc1JQUOHAigSRMwTTBNAwDTNDFNJzExkJoKycnFxMdXfrue3E8eB9WaNWvYt28fu3fvrlT9BQsWMH/+/DLlGzduJCwszNPdl5KUlHRd69cl6qvv8b1+NoDw1eXekhRezhHUQeDgR95tUi3wtcc1NbUhZ892IyAgj9xcKCqqD0BmZiY2m+tI6uzZcDZs+JITJ85Werv5+fmVrutRUJ04cYJZs2aRlJRESEjItVcAnnjiCWbPnu1ezs7OpmXLlgwbNozIyEhPdu/mcDhISkpi6NCh2O32Km2jrlBffY/P9jN91Y+z/P4z7OcglKTw5QzNm4qdi5dVNqDXYp87ovLFx7VlS3jnnQAiIsIJC4MTJwyKigpp3LgxAQEGublQXAwjRvTx6IiqZHStMjwKqr1795KRkUHPnj3dZcXFxWzbto3FixdTWFhIQEBAqXWCg4MJDg4usy273X7dD2Z1bKOuUF99j8/1s8MUCDDLPUdl5+JlQWVAnzcgbkqNN7Em+Nrj2r07dOrkmjjRpg0YhuuxNQwDw7CRkQE9e0L37jZsHsx68OQ+8iio7rjjDlJSUkqVTZkyhZtuuom5c+eWCSkR8TNxU12/K5xQURJSU2uyVXIdbDZITHTN7ktLg0uXXOeqcnPhzBmIjoYZM/AopDzlUVDVq1ePLl26lCoLDw+nYcOGZcpFxE+VCqvLKaTqqoQEWLgQFi2CDRugqCiArCzXkdSMGa7bvalKs/5ERK7KHVaP/FigkKrrEhJcw4DDh5tkZmazbFkkvXt7NtxXVdcdVFu2bKmGZoiIz4mb6pqCfhDXxAkfPSflT2w2iIyEggIHnTt7d7iv1H5rZjci4pdKZvX50Ow+qXkKKhERsTQFlYiIWJqCSkRELE1BJSIilqagEhERS1NQiYiIpSmoRETE0hRUIiJiaQoqERGxNAWViIhYmoJKREQsTUElIiKWpqASERFLU1CJiIilKahERMTSFFQiImJpCioREbE0BZWIiFiagkpERCxNQSUiIpamoBIREUtTUImIiKUpqERExNIUVCIiYmkKKhERsTQFlYiIWJqCSkRELE1BJSIilqagEhERS1NQiYiIpSmoRETE0hRUIiJiaQoqERGxNAWViIhYmoJKREQsTUElIiKWpqASERFLU1CJiIilKahERMTSFFQiImJpCioREbE0BZWIiFiagkpERCxNQSUiIpamoBIREUtTUFld+qrSv32ZP/VVRCrNo6BaunQpXbt2JTIyksjISPr168f69eu91TZJWw57El1/70l0Lfsqf+qrSB3ldEJ2NuTk2ElNdS3XBI+CqkWLFjz77LPs3buXPXv2cPvttzN69GhSU1O91T7/lbYcdj4ImD8WmK5lX3wD96e+itRR27fDlCnw1VcG330XybRpAUyY4Cr3No+CatSoUdx11120b9+eDh068H//939ERESwY8cOb7XPP5V54y7hg2/g/tRXkTpq+3aYMweSkyEwEIKCiomKgv37XeXeDqsqn6MqLi5mzZo15OXl0a9fv+psk3+r8I27hA+9gftTX0XqKKcTFi+Gc+cgLs4VVIYBERGu5fPnYckS7w4DBnq6QkpKCv369aOgoICIiAjWrl1Lp06dKqxfWFhIYWGhezk7OxsAh8OBw+GoQpNxr1fV9S0rfRXseQQIcRc5CC31223nI1BsQJsJNdjAauRPfb2Czz5/y6G+1n0pKXDgQABNmoBpgmkaAJimiWk6iYmB1FRITi4mPr7y2/XkfjJM06zo42y5ioqKOH78OFlZWbz77ru88cYbbN26tcKwmjdvHvPnzy9Tvnr1asLCwjzZtYiI1LDU1IYsXdqNmJg8ANLS6gMQF3cBm811JJWREc706V/SufPZSm83Pz+fcePGkZWVRWRk5FXrehxUVxoyZAhxcXG89tpr5d5e3hFVy5YtOXPmzDUbVxGHw0FSUhJDhw7FbrdXaRuWlL7qx5lv/3lIHISSFL6coXlTsXPxssoG9Fpcd48y/KmvV/DZ52851Ne6LyUFpk4NICoKwsJg716DoqJC+va1ExBgkJsLWVmwfLlnR1TZ2dk0atSoUkHl8dDflZxOZ6kgulJwcDDBwcFlyu12+3U/mNWxDUvpMAUCzHLP29i5eNmbtwF93oC4KTXexGrjT32tgM89f69Cfa27uneHTp1cEyfatAHDcL1eDcPAMGxkZEDPntC9uw2bB7MePLmPPAqqJ554gjvvvJNWrVqRk5PD6tWr2bJlCx9//LEnm5GriZvq+l3hJIOSN+6pNdkq7/CnvorUUTYbJCa6ZvelpcGlS65zVbm5cOYMREfDjBl4FFIet8GTyhkZGUycOJEbb7yRO+64g927d/Pxxx8zdOhQb7XPP8VNdb1BY1xxgw++cftTX0XqqIQEWLjQdXR16RIUFQWQleU6klq40HW7N3l0RPXnP//ZW+2QK7mPNh75scCH37j9qa8idVRCgiuohg83yczMZtmySHr39my4r6qu+xyVeFHcVNe07IO4JhP44HkaN3/qq0gdZbNBZCQUFDjo3Nm7w32l9lszu5EqK5np5iMz3q7Kn/oqIpWmoBIREUtTUImIiKUpqERExNIUVCIiYmkKKhERsTQFlYiIWJqCSkRELE1BJSIilqagEhERS1NQiYiIpSmoRETE0hRUIiJiaQoqERGxNAWViIhYmoJKREQsTUElIiKWpqASERFLU1CJiIilKahERMTSFFQiImJpCioREbE0BZWIiFiagkpERCxNQSUiIpamoBIREUtTUImIiKUpqERExNIUVCIiYmkKKhERsTQFlYiIWJqCSkRELE1BJSIilqagEhERS1NQiYiIpSmoRETE0hRUIiJiaQoqERGxNAWViIhYmoJKREQsTUElIiKWpqASERFLU1CJiIilKahERMTSFFQiImJpCiqxjvRVpX+L1CV6/nqNgkqsIW057El0/b0n0bUsUlf4yfPX6YTsbMjJsZOa6lquCR4F1YIFC7jllluoV68eMTExjBkzhkOHDnmrbeIv0pbDzgcB88cC07Xsoy928TF+8vzdvh2mTIGvvjL47rtIpk0LYMIEV7m3eRRUW7duZcaMGezYsYOkpCQcDgfDhg0jLy/PW+0TX1fmRV7CN1/s4mP85Pm7fTvMmQPJyRAYCEFBxURFwf79rnJvh1WgJ5U3bNhQannlypXExMSwd+9eBg4cWK0NEz9Q4Yu8xI8vdoC4qTXVKpHK8ZPnr9MJixfDuXMQFwcXLkBxMUREQFQUpKXBkiXQty/YvHQyyaOgulJWVhYADRo0qLBOYWEhhYWF7uXs7GwAHA4HDoejSvstWa+q69clPtvX9FWw5xEgxF3kILTUb7edj0CxAW0m1GADvcdnH9Ny+Gxf/ej5m5ICBw4E0KQJmCaYpgGAaZqYppOYGEhNheTkYuLjK79dT54ThmmaFX0cuCqn08lPf/pTLly4wGeffVZhvXnz5jF//vwy5atXryYsLKwquxYRkRqSmtqQpUu7ERPjOsWTllYfgLi4C9hsriOujIxwpk//ks6dz1Z6u/n5+YwbN46srCwiIyOvWrfKQTV9+nTWr1/PZ599RosWLSqsV94RVcuWLTlz5sw1G1cRh8NBUlISQ4cOxW63V2kbdYXP9jV91Y+zpP7z9HMQSlL4cobmTcXOxcsqG9BrcZ39RHoln31My+GzffWj529KCkydGkBUFISFwd69BkVFhfTtaycgwCA3F7KyYPlyz46osrOzadSoUaWCqkpDf4mJiXzwwQds27btqiEFEBwcTHBwcJlyu91+3U/c6thGXeFzfe0wBQLMcsf47Vy87IVuQJ83IG5KjTfR23zuMb0Kn+urHz1/u3eHTp1cEyfatAHDcPXXMAwMw0ZGBvTsCd272zw6R+XJ88GjU1+maZKYmMjatWvZvHkzbdq08WR1kdLiprpexBgVVCh5kdfdE9Hiw/zk+WuzQWIiREe7Jk5cuuQ6V5Wb61qOjoYZM7w3kQI8DKoZM2awatUqVq9eTb169Th16hSnTp3i4sWL115ZpDwVvth940UuPs5Pnr8JCbBwoevo6tIlKCoKICvLdSS1cKHrdm/yaOhv6dKlAAwePLhU+YoVK5g8eXJ1tUn8TcmLeecjPxb41otcfJyfPH8TElxBNXy4SWZmNsuWRdK7t2fDfVXlUVBVcd6FyLXFTXVN4T2I68RzHR7TFz/kJ89fmw0iI6GgwEHnzt4d7iu135rZjUgllMyKqqOzo8TP6fnrNQoqERGxNAWViIhYmoJKREQsTUElIiKWpqASERFLU1CJiIilKahERMTSFFQiImJpCioREbE0BZWIiFiagkpERCxNQSUiIpamoBIREUtTUImIiKUpqERExNIUVCIiYmkKKhERsTQFlYiIWJqCSkRELE1BJSIilqagEhERS1NQiYiIpSmoRETE0hRUIiJiaQoqERGxNAWViIhYmoJKREQsTUElIiKWpqASERFLU1CJiIilKahERMTSFFQiImJpCioREbE0BZWIiFiagkpERCxNQSUiIpamoBIREUtTUImIiKUpqERExNIUVCIiYmkKKhERsTQFlYiIWJqCSkRELE1BJSIilqagEhHvSV9V+rdIFSioRMQ70pbDnkTX33sSXctSpzmdkJ0NOTl2UlNdyzXB46Datm0bo0aNonnz5hiGwbp167zQLBGp09KWw84HAfPHAtO1rLCqs7ZvhylT4KuvDL77LpJp0wKYMMFV7m0eB1VeXh7dunVjyZIl3miPiNR1ZUKqhMKqrtq+HebMgeRkCAyEoKBioqJg/35XubfDKtDTFe68807uvPNOb7RFROq6CkOqxI9hBRA3taZaJdfB6YTFi+HcOYiLgwsXoLgYIiIgKgrS0mDJEujbF2xeOpnkcVB5qrCwkMLCQvdydnY2AA6HA4fDUaVtlqxX1fXrEvXV9/hsP9NXwZ5HgBB3kYPQUr/ddj4CxQa0mVCDDfQuX31cU1LgwIEAmjQB0wTTNAAwTRPTdBITA6mpkJxcTHx85bfryf1kmKZZ0Uefa69sGKxdu5YxY8ZUWGfevHnMnz+/TPnq1asJCwur6q5FRKQGpKY2ZOnSbsTE5AGQllYfgLi4C9hsriOujIxwpk//ks6dz1Z6u/n5+YwbN46srCwiIyOvWtfrQVXeEVXLli05c+bMNRtXEYfDQVJSEkOHDsVut1dpG3WF+up7fLaf6at+nOX3n7cUB6EkhS9naN5U7Fy8rLIBvRb73BGVLz6uKSkwdWoAUVEQFgZ79xoUFRXSt6+dgACD3FzIyoLlyz07osrOzqZRo0aVCiqvD/0FBwcTHBxcptxut1/3g1kd26gr1Fff43P97DAFAsxyz1HZuXhZUBnQ5w2Im1LjTawJvva4du8OnTq5Jk60aQOG4XpsDcPAMGxkZEDPntC9u82jc1Se3Ef6PyoRqT5xU10hhFFBhZKQ0kSKusJmg8REiI52TZy4dMl1rio317UcHQ0zZnhvIgVUIahyc3NJTk4mOTkZgPT0dJKTkzl+/Hh1t01E6qIKw0ohVVclJMDCha6jq0uXoKgogKws15HUwoWu273J46G/PXv2cNttt7mXZ8+eDcCkSZNYuXJltTVMROqwkjDa+ciPBQqpui4hwRVUw4ebZGZms2xZJL17ezbcV1UeB9XgwYO5jvkXIuIv4qa6pqAfxDVxwkfPSfkTmw0iI6GgwEHnzt4d7iu135rZjYj4pZJZfT40u09qnoJKREQsTUElIiKWpqASERFLU1CJiIilKahERMTSFFQiImJpCioREbE0BZWIiFiagkpERCxNQSUiIpamoBIREUtTUImIiKUpqERExNIUVCIiYmkKKhERsTQFlYiIWJqCSkRELE1BJSIilqagEhERS1NQiYiIpSmoRETE0hRUIiJiaQoqERGxNAWViIhYmoJKREQsTUElIiKWpqASERFLU1CJiIilKahERMTSFFQiImJpCioREbE0BZWIiFiagkpERCxNQSUiIpamoBIREUtTUImIiKUpqERExNIUVCIiYmkKKhERsTQFlYiIWJqCSkRELE1BJSIilqagEhERS1NQiYiIpSmoRETE0hRUIiJiaVUKqiVLlhAbG0tISAh9+vRh165d1d2uMg6fPcy+k/vY8/0+/rHlW1JTG/KPLd+y5/t97Du5j8NnD3u9DTXN6YSUFEhNbUhKimvZV/lLX/2ln6C++iKnE7KzISfHTmpqDfbT9NCaNWvMoKAgc/ny5WZqaqr50EMPmfXr1zdPnz5dqfWzsrJMwMzKyqr0Pr89863JPEym9jPp8lfTaLrfbNEi2zSa7jfp8ldX+TzMb89862l3LOvzz03z/vtNs1u3YrNFi2yzW7di8/77XeW+xl/66i/9NE311Rf7+vnnpnnffaYZGek0Q0KKzPj46+unJ1ng8RHViy++yEMPPcSUKVPo1KkTr776KmFhYSxfvrz6U/RHOUU5cLwfbFwIJ3tASBYxMXkQkuVa3rgQjvdz1fMB27fDnDmwbx9ERUFMTB5RUbB/v6t8+/babmH18Ze++ks/QX31xb6W9DM5GQIDISiouEb7GehJ5aKiIvbu3csTTzzhLrPZbAwZMoQvvvii2htXwukEdiXCxWhocAQDOwCG/SJm/aNwIQ52PsLFfIOCAq81o0Y4nbBoEZw9C3FxYJqQmwthYdCmDaSlwcsvQ/fuYKvjZxj9pa/+0k9QX32xr5f3MzYWLlyA4mKIiHCFc1oaLFkCfft6r58eBdWZM2coLi6mSZMmpcqbNGnCN998U+46hYWFFBYWupezs7MBcDgcOByOSu33yIFgjDOdoF4GBnbMH3qSFhiJeaknYIIzAI7cxSMTDJo1qNuDw9nZ8NVXBoGBrieEaRoUFdXnxAkDwzC5dAnWr4fhw00iI2u7tdfHX/rqL/0E9dUX+1q2n65y0zQxTScxMZCaCsnJxcTHV367lX3/Bw+DqioWLFjA/Pnzy5Rv3LiRsLCwSm3jaGpDbgjsRkyDPADSAl2PeuSPv00TiooCyD+fTcaljGpqee3IybFTVBQJFFNc/J/yoiJX2Jf0NTMzm4KCyj/QVuQvffWXfoL6Cr7X1/L6GRJyibNnLwCuI66zZ8PZsOFLTpw4W+nt5ufnV7quR0HVqFEjAgICOH36dKny06dP07Rp03LXeeKJJ5g9e7Z7OTs7m5YtWzJs2DAiK/kx42LYt3z/Shrfn8vCCMojoImd+Ih4UnJTKDaLMYvCoSCK/34hgLsHdvCkS5aTmgrTpgUQFRVARITrU0tmZiaNGzfGMAxycyErC5Yti6Rz59pu7fXxl776Sz9BffXFvpbXz7NnL5TqZ3ExjBjRx6MjqpLRtcrwKKiCgoK4+eab2bRpE2PGjAHA6XSyadMmEhMTy10nODiY4ODgMuV2ux273V6p/bbrVIjZ6Bs42QOzwREwXGOhxThwUgy5jaD5Pjp170i9epXbplX17g1durhOUkZFgWk6sdkgIMDAMGycOQM9e0Lv3rY6Pe4N/tNXf+knqK++2Nfy+glgGK5+ZmS4+tm9u2f9rOz7P1Th/6hmz57N66+/zptvvsnBgweZPn06eXl5TJkyxdNNVZrNBvReDKHn4Vw7zKJwnE5cR1Ln2rnKb1lSp58MJWw2SEyE6GjXScrcXNehdW6uazk6GmbMqNsnZ0v4S1/9pZ+gvvpiX63QT483PXbsWBYuXMiTTz5J9+7dSU5OZsOGDWUmWFSnekH1oNUXMGwONNsPBVFkZLiG+2i+z1Xe6gtXPR+QkAALF0KPHq6hg4yMcLKyXJ9aFi503e4r/KWv/tJPUF99sa+13U/DNEvmcNSM7OxsoqKiyMrKqvQ5KnBdmSKnKAen0zUL8Oju87S9JZp2nQqx2Vxh1r5hey+2vOY5nZCc7GDDhp2MGNGH7t3tdf7TWUX8pa/+0k9QX32xr9XZT0+ywOuz/qrL5SHULcbBR/kfcdfgPh6Nc9Y1NhvEx8OJE2eJj6/7QwhX4y999Zd+gvrqi2qrnz56d4qIiK9QUImIiKUpqERExNIUVCIiYmkKKhERsTQFlYiIWJqCSkRELE1BJSIilqagEhERS6vxK1OUXLHJk0u8X8nhcJCfn092drZPX5kC1Fdf5C/9BPXVF1VXP0syoDJX8avxoMrJyQGgZcuWNb1rERGxmJycHKKioq5ap8YvSut0Ovnhhx+oV68ehmFUaRslX7544sQJjy5sWxepr77HX/oJ6qsvqq5+mqZJTk4OzZs3x3aNiwbW+BGVzWajRYsW1bKtyMhIn35CXE599T3+0k9QX31RdfTzWkdSJTSZQkRELE1BJSIillYngyo4OJinnnqK4ODg2m6K16mvvsdf+gnqqy+qjX7W+GQKERERT9TJIyoREfEfCioREbE0BZWIiFiagkpERCytTgbVkiVLiI2NJSQkhD59+rBr167ablK127ZtG6NGjaJ58+YYhsG6detqu0lesWDBAm655Rbq1atHTEwMY8aM4dChQ7XdLK9YunQpXbt2df+jZL9+/Vi/fn1tN8vrnn32WQzD4NFHH63tplS7efPmYRhGqZ+bbrqptpvlNd9//z0TJkygYcOGhIaGEh8fz549e7y+3zoXVG+//TazZ8/mqaeeYt++fXTr1o3hw4eTkZFR202rVnl5eXTr1o0lS5bUdlO8auvWrcyYMYMdO3aQlJSEw+Fg2LBh5OXl1XbTql2LFi149tln2bt3L3v27OH2229n9OjRpKam1nbTvGb37t289tprdO3atbab4jWdO3fm5MmT7p/PPvustpvkFefPn6d///7Y7XbWr1/PgQMHeOGFF4iOjvb+zs06pnfv3uaMGTPcy8XFxWbz5s3NBQsW1GKrvAsw165dW9vNqBEZGRkmYG7durW2m1IjoqOjzTfeeKO2m+EVOTk5Zvv27c2kpCRz0KBB5qxZs2q7SdXuqaeeMrt161bbzagRc+fONW+99dZa2XedOqIqKipi7969DBkyxF1ms9kYMmQIX3zxRS22TKpLVlYWAA0aNKjllnhXcXExa9asIS8vj379+tV2c7xixowZjBw5stTr1RcdPnyY5s2b07ZtW8aPH8/x48dru0le8f7779OrVy/uvfdeYmJi6NGjB6+//nqN7LtOBdWZM2coLi6mSZMmpcqbNGnCqVOnaqlVUl2cTiePPvoo/fv3p0uXLrXdHK9ISUkhIiKC4OBgfvnLX7J27Vo6depU282qdmvWrGHfvn0sWLCgtpviVX369GHlypVs2LCBpUuXkp6ezoABA9xfZ+RLjh49ytKlS2nfvj0ff/wx06dPZ+bMmbz55pte33eNXz1dpCIzZszg66+/9tkxfoAbb7yR5ORksrKyePfdd5k0aRJbt271qbA6ceIEs2bNIikpiZCQkNpujlfdeeed7r+7du1Knz59aN26Ne+88w4PPPBALbas+jmdTnr16sXvf/97AHr06MHXX3/Nq6++yqRJk7y67zp1RNWoUSMCAgI4ffp0qfLTp0/TtGnTWmqVVIfExEQ++OADPv3002r7GhgrCgoKol27dtx8880sWLCAbt26sWjRotpuVrXau3cvGRkZ9OzZk8DAQAIDA9m6dSsvv/wygYGBFBcX13YTvaZ+/fp06NCBI0eO1HZTql2zZs3KfKDq2LFjjQx11qmgCgoK4uabb2bTpk3uMqfTyaZNm3x2nN/XmaZJYmIia9euZfPmzbRp06a2m1SjnE4nhYWFtd2ManXHHXeQkpJCcnKy+6dXr16MHz+e5ORkAgICaruJXpObm0taWhrNmjWr7aZUu/79+5f515Fvv/2W1q1be33fdW7ob/bs2UyaNIlevXrRu3dvXnrpJfLy8pgyZUptN61a5ebmlvpUlp6eTnJyMg0aNKBVq1a12LLqNWPGDFavXs17771HvXr13Ocao6KiCA0NreXWVa8nnniCO++8k1atWpGTk8Pq1avZsmULH3/8cW03rVrVq1evzDnG8PBwGjZs6HPnHufMmcOoUaNo3bo1P/zwA0899RQBAQHcf//9td20avfYY4+RkJDA73//e+677z527drFsmXLWLZsmfd3XitzDa/TK6+8YrZq1coMCgoye/fube7YsaO2m1TtPv30UxMo8zNp0qTablq1Kq+PgLlixYrablq1mzp1qtm6dWszKCjIbNy4sXnHHXeYGzdurO1m1QhfnZ4+duxYs1mzZmZQUJB5ww03mGPHjjWPHDlS283ymn/9619mly5dzODgYPOmm24yly1bViP71dd8iIiIpdWpc1QiIuJ/FFQiImJpCioREbE0BZWIiFiagkpERCxNQSUiIpamoBIREUtTUImIiKUpqERExNIUVCIiYmkKKhERsTQFlYiIWNr/BwhnieT6cAXEAAAAAElFTkSuQmCC", "text/plain": [ "
" ] }, "metadata": {}, "output_type": "display_data" }, { "name": "stdout", "output_type": "stream", "text": [ "\n", "--------------------------------------------------------------------------------------\n", "\n", "Final Reward (Genetic Algorithm): 88\n", "\n" ] }, { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAaoAAAHDCAYAAAB1dF5kAAAAOnRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjEwLjAsIGh0dHBzOi8vbWF0cGxvdGxpYi5vcmcvlHJYcgAAAAlwSFlzAAAPYQAAD2EBqD+naQAATm1JREFUeJzt3XlcVPX+P/DXmWHYGUAFQUNAxNwQJJLEMm9JoGnqL7NccqGyDMzy2v197Xcrvbf01rXS1LhaV/Pr0p5m3TTJNSV3UUItIVyuG4uyb8PM5/fHcSZGQBmcYQ7D6/l4+MDzmbN83sPMvOZ8zocZSQghQEREpFAqe3eAiIjoZhhURESkaAwqIiJSNAYVEREpGoOKiIgUjUFFRESKxqAiIiJFY1AREZGiMaiIiEjRGFR0S5IkYe7cuS1+3ClTpiAkJKTFj2s0d+5cSJJk0boFBQU27pWyfPzxx5AkCWfOnGkzxw4JCcGUKVNMyzt37oQkSdi5c2eL9qMtYVDZWG5uLlJSUtC9e3e4u7vD3d0dvXr1QnJyMo4fP27v7pl8//33LRpGRUVFcHV1hSRJOHnyZIsd93bNnz8fGzdutHc3GpWVlYWJEyeic+fOcHFxQadOnTBhwgRkZWXd1n6VXvfNGN9EGP8Zn4N//etfUVJSYu/uURMwqGzou+++Q58+fbBmzRoMGTIE7733HhYvXoyhQ4fi+++/R1RUFM6ePWvvbgKQg2revHkN3lZZWYm//vWvVj3eF198AUmSEBAQgHXr1ll139by17/+FZWVlWZtSn7B/vrrrxEdHY1t27Zh6tSp+OCDD/DUU09hx44diI6OxoYNG5q978bqfvLJJ1FZWYng4ODb6HnLSE1NxZo1a/Duu++iR48eePPNN5GYmIjb/bjTQYMGobKyEoMGDbJST+lGTvbugKPKycnBE088geDgYGzbtg2BgYFmt7/11lv44IMPoFIp/72Cq6ur1fe5du1aDBs2DMHBwVi/fj3eeOMNqx+jucrLy+Hh4QEnJyc4ObWOp0hOTg6efPJJdO3aFbt374afn5/ptpkzZ+K+++7Dk08+iePHj6Nr165WO65arYZarbba/mxpzJgx6NChAwDgueeew6OPPoqvv/4a+/btw4ABA5q9X5VKZZPnCP1B+a+SrdTbb7+N8vJyrFq1ql5IAYCTkxNeeOEFBAUFmbWfOnUKY8aMQbt27eDq6oqYmBhs2rTJbB3j2PzevXsxa9Ys+Pn5wcPDA6NHj0Z+fn69Y23evBn33XcfPDw84OXlhYcffthsKGjKlClYtmwZAJgNkRg1dI3qwoULeOqpp9CpUye4uLggNDQU06dPR01NzS3vm3PnzuGnn37CE088gSeeeAK5ublIT0+/5XYAUFhYiCeffBJarRY+Pj6YPHkyjh07BkmS8PHHH5utu337dlPdPj4+GDlyZL1hRuOw0IkTJzB+/Hj4+vri3nvvNbut7v1QXl6O1atXm+6jutcqAHlIc8qUKfDx8YG3tzemTp2KiooKs3UkSUJKSgq++OIL9OrVC25ubhgwYAAyMzMBAMuXL0e3bt3g6uqKwYMHN+kazD//+U9UVFRgxYoVZiEFAB06dMDy5ctRXl6Ot99+u17tp06dwtixY6HVatG+fXvMnDkTVVVVTaq7oetEISEhGD58OHbu3ImYmBi4ubkhIiLCdA3n66+/RkREBFxdXXHXXXfh6NGjZv09fvw4pkyZgq5du8LV1RUBAQFISkpCYWHhLe8HSzzwwAMA5OF5QH6D8uc//xlBQUFwcXHBnXfeiYULF97yjKuxa1T79+/HsGHD4OvrCw8PD/Tt2xeLFy8GAKxatQqSJNWrHZDPXtVqNS5cuGCFKh1D63i72Ap999136NatG2JjY5u8TVZWFgYOHIjOnTvjf/7nf+Dh4YHPP/8co0aNwldffYXRo0ebrT9jxgz4+vri9ddfx5kzZ7Bo0SKkpKTgs88+M62zZs0aTJ48GQkJCXjrrbdQUVGB1NRU3HvvvTh69ChCQkLw7LPP4uLFi0hLS8OaNWtu2c+LFy+if//+KCoqwrRp09CjRw9cuHABX375JSoqKuDs7HzT7T/55BN4eHhg+PDhcHNzQ1hYGNatW4e4uLibbmcwGDBixAgcOHAA06dPR48ePfDNN99g8uTJ9db98ccfMXToUHTt2hVz585FZWUllixZgoEDB+LIkSP1Jmk89thjCA8Px/z58xt9YVqzZg2efvpp9O/fH9OmTQMAhIWFma0zduxYhIaGYsGCBThy5Ag++ugj+Pv746233jJb76effsKmTZuQnJwMAFiwYAGGDx+Ov/zlL/jggw/w/PPP49q1a3j77beRlJSE7du33/S++fbbbxESEoL77ruvwdsHDRqEkJAQ/Oc//6l329ixYxESEoIFCxZg3759eP/993Ht2jX87//+b5PrvlF2djbGjx+PZ599FhMnTsTChQsxYsQI/Otf/8Irr7yC559/3lT32LFj8euvv5pGF9LS0vD7779j6tSpCAgIQFZWFlasWIGsrCzs27evyRNcbiUnJwcA0L59ewgh8Mgjj2DHjh146qmnEBUVhR9++AEvv/wyLly4gPfee8+ifaelpWH48OEIDAzEzJkzERAQgJMnT+K7777DzJkzMWbMGCQnJ2PdunXo16+f2bbr1q3D4MGD0blzZ6vU6RAEWV1xcbEAIEaNGlXvtmvXron8/HzTv4qKCtNtDz74oIiIiBBVVVWmNoPBIOLi4kR4eLipbdWqVQKAGDJkiDAYDKb2l156SajValFUVCSEEKK0tFT4+PiIZ555xqwPly9fFt7e3mbtycnJorGHAwDx+uuvm5YnTZokVCqVOHjwYL116/anMREREWLChAmm5VdeeUV06NBB6HQ6s/UmT54sgoODTctfffWVACAWLVpkatPr9eKBBx4QAMSqVatM7VFRUcLf318UFhaa2o4dOyZUKpWYNGmSqe31118XAMS4cePq9dN4W10eHh5i8uTJja6blJRk1j569GjRvn17szYAwsXFReTm5prali9fLgCIgIAAUVJSYmqfM2eOAGC27o2KiooEADFy5MhG1xFCiEceeUQAMO3f2OdHHnnEbL3nn39eABDHjh27Zd3Gx2Ld/gUHBwsAIj093dT2ww8/CADCzc1NnD17tl7dO3bsMLXVfU4YffLJJwKA2L17902P3RBjnb/++qvIz88Xubm5Yvny5cLFxUV07NhRlJeXi40bNwoA4o033jDbdsyYMUKSJJGdnW1WX937YseOHWY11NbWitDQUBEcHCyuXbtmtr+6z49x48aJTp06Cb1eb2o7cuRIvccyCcGhPxswziTy9PSsd9vgwYPh5+dn+mcccrt69Sq2b9+OsWPHorS0FAUFBSgoKEBhYSESEhJw+vTpekMB06ZNM3t3ed9990Gv15smaKSlpaGoqAjjxo0z7a+goABqtRqxsbHYsWOHxbUZDAZs3LgRI0aMQExMTL3bb/Vu9/jx48jMzMS4ceNMbcb+/fDDDzfddsuWLdBoNHjmmWdMbSqVynRWYnTp0iVkZGRgypQpaNeunam9b9++iI+Px/fff19v388999xNj91UN+7nvvvuQ2FhYb3ZZQ8++KDZWZ3xzPvRRx+Fl5dXvfbff/+90WOWlpYCgNl2DTHefmNfbrz/ZsyYAQAN3k9N1atXL7PrPsY6HnjgAXTp0qVee9363NzcTP+vqqpCQUEB7rnnHgDAkSNHmt2nO++8E35+fggNDcWzzz6Lbt264T//+Q/c3d3x/fffQ61W44UXXjDb5s9//jOEENi8eXOTj3P06FHk5ubixRdfhI+Pj9ltdZ8fkyZNwsWLF82eh+vWrYObmxseffTR5hXpoDj0ZwPGF4SysrJ6ty1fvhylpaW4cuUKJk6caGrPzs6GEAKvvvoqXn311Qb3m5eXZzYcUPcJDwC+vr4AgGvXrgEATp8+DeCPsfgbabXappZkkp+fj5KSEvTp08fibQF5EoWHhwe6du2K7OxsAPJkjZCQEKxbtw4PP/xwo9uePXsWgYGBcHd3N2vv1q1bvfUA+YXpRj179sQPP/xgmjBhFBoa2qx6bnSz30nd+/vG9by9vQGg3jVLY7vxd9oQ4+PNGFiNaSzQwsPDzZbDwsKgUqlu6++Tbqe+q1evYt68efj000+Rl5dntn5xcXGz+/TVV19Bq9VCo9HgjjvuMBu+PHv2LDp16lTvvunZs6fp9qYyDine6jkSHx+PwMBArFu3Dg8++CAMBgM++eQTjBw58pZvOtoaBpUNeHt7IzAwEL/88ku924zvIG98ETAYDACA2bNnIyEhocH93viC3NhsK3H9Gotxn2vWrEFAQEC99Vp6RpsQAp988gnKy8vRq1everfn5eWhrKyswTNRW6v7Lv523Op3cqv1mrp9XcbH263+Lu/48ePo3LnzLd+gWOMa0O3UN3bsWKSnp+Pll19GVFQUPD09YTAYkJiYaHpMN8egQYNMs/6UQK1WY/z48fjwww/xwQcfYO/evbh48aLZG1iSMahs5OGHH8ZHH32EAwcOoH///rdc3zhlWKPRYMiQIVbpg/Edo7+//y332dQXJz8/P2i12gZD+FZ27dqF//73v/jb3/5meqdqdO3aNUybNg0bN25s9IkaHByMHTt2oKKiwuysynhmVnc9APj111/r7ePUqVPo0KGD2dmUJax1Id/ahg8fjg8//BB79uwxzVqs66effsKZM2fw7LPP1rvt9OnTZmeU2dnZMBgMZkOTLVX3tWvXsG3bNsybNw+vvfaaWR9tKTg4GD/++CNKS0vNzmZOnTplur2pjM+7X3755ZbPu0mTJuGdd97Bt99+i82bN8PPz6/RN6ptGa9R2chf/vIXuLu7IykpCVeuXKl3+43vkP39/TF48GAsX74cly5dqrd+Q9PObyUhIQFarRbz58+HTqe76T6NL9xFRUU33adKpcKoUaPw7bff4tChQ/Vuv9k7f+Ow38svv4wxY8aY/XvmmWcQHh5+0z/+TUhIgE6nw4cffmhqMxgMput8RoGBgYiKisLq1avN6vnll1+wdetWDBs27KY13oyHh8ct7yN7ePnll+Hm5oZnn3223jTuq1ev4rnnnoO7uztefvnletveeP8tWbIEADB06FBTW0vVbTzjuvFxtGjRIpsed9iwYdDr9Vi6dKlZ+3vvvQdJkszui1uJjo5GaGgoFi1aVO8+u7Guvn37om/fvvjoo4/w1Vdf4Yknnmg1f7vXkniP2Eh4eDjWr1+PcePG4c4778SECRMQGRkJIQRyc3Oxfv16qFQq3HHHHaZtli1bhnvvvRcRERF45pln0LVrV1y5cgU///wz/vvf/+LYsWMW9UGr1SI1NRVPPvkkoqOj8cQTT8DPzw/nzp3Df/7zHwwcOND0xLzrrrsAAC+88AISEhKgVqvxxBNPNLjf+fPnY+vWrbj//vsxbdo09OzZE5cuXcIXX3yBPXv21LuADADV1dX46quvEB8f3+gfRz7yyCNYvHgx8vLy4O/vX+/2UaNGoX///vjzn/+M7Oxs9OjRA5s2bcLVq1cBmL/r/+c//4mhQ4diwIABeOqpp0zT0729vW/ro6Luuusu/Pjjj3j33XfRqVMnhIaGWvQnCLYSHh6O1atXY8KECYiIiMBTTz2F0NBQnDlzBv/+979RUFCATz75pMFp5bm5uXjkkUeQmJiIn3/+GWvXrsX48eMRGRlpWqel6tZqtRg0aBDefvtt6HQ6dO7cGVu3bjX9rZOtjBgxAn/605/w//7f/8OZM2cQGRmJrVu34ptvvsGLL754y+n4dalUKqSmpmLEiBGIiorC1KlTERgYiFOnTiErK6vepKFJkyZh9uzZAMBhv8bYZ7Jh25GdnS2mT58uunXrJlxdXYWbm5vo0aOHeO6550RGRka99XNycsSkSZNEQECA0Gg0onPnzmL48OHiyy+/NK1jnJZ74/TwG6fJ1m1PSEgQ3t7ewtXVVYSFhYkpU6aIQ4cOmdapra0VM2bMEH5+fkKSJLNp2bhheroQQpw9e1ZMmjRJ+Pn5CRcXF9G1a1eRnJwsqqurG7wfjFPL//3vfzd6X+3cuVMAEIsXLxZC1J+eLoQQ+fn5Yvz48cLLy0t4e3uLKVOmiL179woA4tNPPzVb98cffxQDBw4Ubm5uQqvVihEjRogTJ06YrWOcupyfn1+vPw1NTz916pQYNGiQcHNzEwBM05Qb209DU6gBiOTkZLP1cnNzBQDxz3/+06zd+Dv94osvGr7TbnD8+HExbtw4ERgYKDQajQgICBDjxo0TmZmZjdZ34sQJMWbMGOHl5SV8fX1FSkqKqKysbFLdjU1Pf/jhh+sdr6l1//e//xWjR48WPj4+wtvbWzz22GPi4sWL9R6Hlk5Pb+h3XFdpaal46aWXRKdOnYRGoxHh4eHin//8Z70/ubjV9HSjPXv2iPj4eOHl5SU8PDxE3759xZIlS+od99KlS0KtVovu3bvftH9tmSTEbX7QFZGdbdy4EaNHj8aePXswcOBAe3en1Zg7dy7mzZuH/Px8RU0yaGsKCgoQGBiI1157rdEZv20dr1FRq3Ljh8Tq9XosWbIEWq0W0dHRduoVUfN9/PHH0Ov1ePLJJ+3dFcXiNSpqVWbMmIHKykoMGDAA1dXV+Prrr5Geno758+dbbYo5UUvYvn07Tpw4gTfffBOjRo2y63evKR2DilqVBx54AO+88w6+++47VFVVoVu3bliyZAlSUlLs3TUii/ztb39Deno6Bg4caJppSQ3jNSoiIlI0XqMiIiJFY1AREZGitfg1KoPBgIsXL8LLy0uxH0dDRES2JYRAaWkpOnXqdMtvOm/xoLp48WK9T1AmIqK26fz582af0NOQFg8q4wc+nj9/vllfMwEAOp0OW7duxUMPPQSNRmPN7ikOa3U8baVOgLU6ImvVWVJSgqCgoCZ9pUmLB5VxuE+r1d5WULm7u5u+W8aRsVbH01bqBFirI7J2nU25BMTJFEREpGgMKiIiUjQGFRERKRqDioiIFI1BRUREisagIiIiRWNQERGRojGoiIhI0RhURESkaAwqpctda/6TiKiNsTioLly4gIkTJ6J9+/Zwc3NDREQEDh06ZIu+Uc5K4ND1b649lCIvExHZQ3U1DNt2IDMTyMpqj8xMwLBtB1BdbfNDW/RZf9euXcPAgQPxpz/9CZs3b4afnx9Onz4NX19fW/Wv7cpZCex/GoDr9QZxfRlAWJK9ekVEbVF1NdLvn4Ol+2Nwwi8PhS6R+HzRFfTKv4SU2DmI27UAcHGx2eEtCqq33noLQUFBWLVqlaktNDTU6p1q80whJW64gWFFRC3sekjN3v8YrsIHHQuyoe53JzyP5uAo+mH2/lAsvN+2YWXR0N+mTZsQExODxx57DP7+/ujXrx8+/PBDm3SszWo0pIyuhxWHAYmoBRj2pGPp/hhchQ+6IRvuUiUAwFMqRxiycQ0+WLb/Lhj2/myzPlh0RvX7778jNTUVs2bNwiuvvIKDBw/ihRdegLOzMyZPntzgNtXV1aiuM4ZZUlICQP6oeJ1O16xOG7dr7vaKlbsWODQDfwz3ATq4mf002T8D0EtA6MQW7KBtOezv9QZtpU6AtTqCTN97ccLvCjoW5EAvaXBYRKMmRws/tQZqAP4iD1kdBiPDxx8RFtRuyf0kCSEae+tej7OzM2JiYpCenm5qe+GFF3Dw4EH8/HPDaTp37lzMmzevXvv69evh7u7e5I4SEVHLy8pqj9TUSPj7lwMAcnJ8AABhYUVQqQCDAcjL88D06cfQu3dhk/dbUVGB8ePHo7i4+JbfTWjRGVVgYCB69epl1tazZ0989dVXjW4zZ84czJo1y7Rs/FbHhx566La+ODEtLQ3x8fGO9QVluWuvz/L7472DDm5I81iJ+PIkaFBZZ2UJiFnqcGdUDvl7vUFbqRNgrY4gKAj4fNEVeB7NgbtUifMiGjVaLfwyM6HW61AmPKDv0A2JibGIiGj6fo2ja01hUVANHDgQv/76q1nbb7/9huDg4Ea3cXFxgUsDF9g0Gs1t/zKtsQ9F6T4VUIsGr1FpUFknqCQg9iMgbGqLd7ElONzvtRFtpU6AtbZmUdd2oFf+JRxFP4SK3yFdf22S9HpIBj3y4I/o/J2IKuoMlWZwk/dryX1k0WSKl156Cfv27cP8+fORnZ2N9evXY8WKFUhOTrZkN3QzYUlyCKGxr2c2hhRn/RGR7anujUNK7CH4ogg5CEMt1BACKBMeyEE3+OIakmMPQzVwgO36YMnKd999NzZs2IBPPvkEffr0wd///ncsWrQIEyZMsFX/2qZGw4ohRUQtzMUFcbsWYGHsF4hCBmqdPVBTo0Zx+66IxhEsjP1SWX9HBQDDhw/H8OHDbdEXqssYRvtnXG9gSBGRnVwPq6gd+5Hwpgvy869hxSp/9K/sDNXAMTYNKaAZQUUtKCxJnoJ+EvLECQe9JkVErYCLC1SDB0G7xICqKh169wFUXoNb5ND8UFqlM87qc6DZfURElmBQERGRojGoiIhI0RhURESkaAwqIiJSNAYVEREpGoOKiIgUjUFFRESKxqAiIiJFY1AREZGiMaiIiEjRGFRERKRoDCoiIlI0BhURESkag4qIiBSNQUVERIrGoCIiIkVjUBERkaIxqIiISNEYVEREpGgMKiIiUjQGFRERKRqDioiIFI1BRUREisagIiIiRWNQERGRojGoiIhI0RhURESkaAwqIiJSNAYVEREpGoOKiIgUjUFFRESKxqAiIiJFY1AREZGiMaiIiEjRGFRERKRoDCoiIlI0BhURESkag4qIiBSNQUVERIrGoCIiIkVjUBERkaIxqIiISNEYVEREpGgMKiIiUjQGFRGRNeSuNf9JVsOgIiK6XTkrgUMp8v8PpcjLDshgAEpKgNJSDbKy5OWWYFFQzZ07F5Ikmf3r0aOHrfpGRKR8OSuB/U8DENcbhLzsYGGVng5MnQocPy7h7Fktpk1TY+JEud3WnCzdoHfv3vjxxx//2IGTxbsgInIM9ULK6HpYAUBYUkv3yurS04HZs4HCQkB+ydfD21uNo0fl9oULgbg42x3f4pRxcnJCQECALfpCRNR6NBpSRo4RVgYDsHQpcPUqEBYGFBUBej3g6Ql4ewM5OcCyZcA99wAqG11MsjioTp8+jU6dOsHV1RUDBgzAggUL0KVLl0bXr66uRnV1tWm5pKQEAKDT6aDT6ZrRZZi2a+72rQlrdTxtpU7AgWvNXQscmgHA1dSkg5vZT5P9MwC9BIRObMEOWk9mJnDihBodOwJCAEJIAAAhBIQwwN8fyMoCMjL0iIho+n4teUxIQojG3g7Us3nzZpSVleHOO+/EpUuXMG/ePFy4cAG//PILvLy8Gtxm7ty5mDdvXr329evXw93dvckdJSKilpeV1R6pqZHw9y8HAOTk+AAAwsKKoFLJZ1x5eR6YPv0YevcubPJ+KyoqMH78eBQXF0Or1d50XYuC6kZFRUUIDg7Gu+++i6eeeqrBdRo6owoKCkJBQcEtO9cYnU6HtLQ0xMfHQ6PRNGsfrQVrdTxtpU7AgWvNXXt9lt8fL586uCHNYyXiy5OgQWWdlSUgZmmrPqNKSlLD2xtwdwcOH5ZQU1ONe+7RQK2WUFYGFBcDK1dadkZVUlKCDh06NCmobmsmhI+PD7p3747s7OxG13FxcYGLi0u9do1Gc9sPXGvso7VgrY6nrdQJOGCt3acCatHgNSoNKusElQTEfgSETW3xLlpLVBTQqxdw9CgQGgpIklyvPPNbhbw8IDoaiIpSWXSNypLHw21d+iorK0NOTg4CAwNvZzdERK1PWJIcQpAaWcEYUq13IgUgT5BISQF8feWJE7W18rWqsjJ52dcXSE623UQKwMKgmj17Nnbt2oUzZ84gPT0do0ePhlqtxrhx42zVPyIi5Wo0rBwjpIzi4uQp6FFRclDV1KhRXCyfSdl6ajpg4dDff//7X4wbNw6FhYXw8/PDvffei3379sHPz89W/SMiUjZjGO2fcb3BsULKKC5ODqqEBIH8/BKsWKFF//6WDfc1l0VB9emnn9qqH0RErVdYkjwF/STkiROt+JrUzahUgFYLVFXp0Lu3bYf7zI7bMochInJwxll9rXR2n5IxqIiISNEYVEREpGgMKiIiUjQGFRERKRqDioiIFI1BRUREisagIiIiRWNQERGRojGoiIhI0RhURESkaAwqIiJSNAYVEREpGoOKiIgUjUFFRESKxqAiIiJFY1AREZGiMaiIiEjRGFRERKRoDCoiIlI0BhURESkag4qIiBSNQUVERIrGoCIiIkVjUBERkaIxqIiISNEYVEREpGgMKiIiUjQGFRERKRqDioiIFI1BRUREisagIiIiRWNQERGRojGoiIhI0RhURESkaAwqIiJSNAYVEREpGoOKiIgUjUFFRESKxqAiIiJFY1AREZGiMaiIiEjRGFRERKRoDCoiIlI0BhURESkag4qIbCd3rflPomZgUBGRbeSsBA6lyP8/lCIvOyiDAcjMBLKy2iMzU152RAYDUFIClJZqkJXVcnXeVlD94x//gCRJePHFF63UHSJyCDkrgf1PAxDXG4S87IBhlZ4OTJwIJCWpkZoaiaQkNSZOlNsdSXo6MHUqcPy4hLNntZg2reXqbHZQHTx4EMuXL0ffvn2t2R8iau3qhZSR44VVejowezZw5Ajg7Q34+5fD2xs4elRud5SwMtaZkQE4OQHOzvoWrdOpORuVlZVhwoQJ+PDDD/HGG29Yu09E1Fo1GlJG18MKAMKSWqpXNmEwAEuXAlevAt26AUIAZWWAuzsQGgrk5ADvvw9ERQGqVnyRxWAAFi8GCguBkBCgqAjQ6wFPTzmcc3KAZcuAe+6xXZ3NCqrk5GQ8/PDDGDJkyC2Dqrq6GtXV1ablkpISAIBOp4NOp2vO4U3bNXf71oS1Oh6HrTN3LXBoBgBXU5MObmY/TfbPAPQSEDqxBTtoXZmZwIkTanTsKIfUyZPA1as+OH9egiQJ1NYCmzcDCQkCWq29e9t8JSXycJ+TkxxS4vp7ECEEhDDA3x/IygIyMvSIiGj6fi15/FscVJ9++imOHDmCgwcPNmn9BQsWYN68efXat27dCnd3d0sPbyYtLe22tm9NWKvjcbw62wEe6xu8Jc2jgeG+kwBOfm/bLtlQVlZ7FBZGQq0uR1mZHFIAUFMjvzEXAqipUSM/vwRVVa33TUlpqQY1NVoAeuj1cpuray0KC4sAyGdchYUe2LLlGM6fL2zyfisqKpq8rkVBdf78ecycORNpaWlwdXW99QYA5syZg1mzZpmWS0pKEBQUhIceegjaZr7N0Ol0SEtLQ3x8PDQaTbP20VqwVsfjsHXmrr0+y++PYT8d3JDmsRLx5UnQoLLOyhIQs7RVn1EFBQGff66Gp6cH3N2B8+cl1NRU4+67NXByklBWBhQXAytWaNG7t71723xZWcC0aWp4e6vh6SmfSRUWFsHPzw+SJNep1wOJibEWnVEZR9eawqKgOnz4MPLy8hAdHW1q0+v12L17N5YuXYrq6mqo1WqzbVxcXODi4lJvXxqN5rafpNbYR2vBWh2Pw9XZfSqgFg1eo9Kgsk5QSUDsR0DY1BbvojVFRQG9eskTCkJDAUmSa3ZykqBWq1BQAERHA/37q1r1Nar+/YE+feQ6vb0BIeQ56ZIkQZJUyMuT64yKsqxOSx77Ft19Dz74IDIzM5GRkWH6FxMTgwkTJiAjI6NeSBFRGxOWJIcQpEZWMIZU655IAcgTB1JSAF9feUJBbe0fEypycuT25OTWPZECqF9nWZk83NeSdVp0RuXl5YU+ffqYtXl4eKB9+/b12omojTKGkHF2n4njhJRRXBywcKE8K27LFvmaVHGxfIaRnCzf7giMdS5dCpw4IV+T0utbrs5mzfojIropU1jNuN7geCFlFBcnDwMmJAjk55dgxQptqx/ua0hcnDwFPSNDjy1bjiExMdbi4b7muu2g2rlzpxW6QUQOJyxJnoJ+EvLEiVZ+TepmVCpAqwWqqnTo3bv1D/c1RqUCIiKA8+cLERHRcnU66N1JRIpgnNXXimf3kf0xqIiISNEYVEREpGgMKiIiUjQGFRERKRqDioiIFI1BRUREisagIiIiRWNQERGRojGoiIhI0RhURESkaAwqIiJSNAYVEREpGoOKiIgUjUFFRESKxqAiIiJFY1AREZGiMaiIiEjRGFRERKRoDCoiIlI0BhURESkag4qIiBSNQUVERIrGoCIiIkVjUBERkaIxqIiISNEYVEREpGgMKiIiUjQGFRERKRqDioiIFI1BRUREisagIiIiRWNQERGRojGoiIhI0RhURESkaAwqIiJSNAYVEREpGoOKiIgUjUFFRESKxqAiIiJFY1AREZGiMaiIiEjRGFRERKRoDCoiIlI0BhURESkag4qIiBSNQaV0uWvNfzqytlQrETWZRUGVmpqKvn37QqvVQqvVYsCAAdi8ebOt+kY5K4FDKfL/D6XIy46qLdXaRhgMQGYmkJXVHpmZ8rKjMhiAkhKgtFSDrCzHrtUenCxZ+Y477sA//vEPhIeHQwiB1atXY+TIkTh69Ch69+5tqz62TTkrgf1PA3C93iCuLwMIS7JXr2yjLdXaRqSnA0uXAidOqFFYGInPP1ejVy8gJQWIi7N376wrPR1YvBg4flxCTY0W06ap0aePY9ZqLxadUY0YMQLDhg1DeHg4unfvjjfffBOenp7Yt2+frfrXNpleuMUNN1x/AXeks422VGsbkZ4OzJ4NHDkCeHsD/v7l8PYGjh6V29PT7d1D6zHWmpEBODkBzs56h63Vniw6o6pLr9fjiy++QHl5OQYMGGDNPrVtjb5wGznQ2UZbqrWNMBjkM6mrV4Fu3QAhgLIywN0dCA0FcnKA998HoqIAVSu/Qm4wyGdShYVASAhQVATo9YCnpxzQOTnAsmXAPfe0/lrtzeKgyszMxIABA1BVVQVPT09s2LABvXr1anT96upqVFdXm5ZLSkoAADqdDjqdrhldhmm75m6vWLlrgUMz8McQGKCDm9lPk/0zAL0EhE5swQ5aUVuq9QYO+/iFfE3qxAk1OnaUQ+rkSeDqVR+cPy9BkgRqa4HNm4GEBAGt1t69vT0lJfJwn5OTHFLi+vstIQSEMMDfH8jKAjIy9IiIsGtXrcpaj19LtpeEEI29nW1QTU0Nzp07h+LiYnz55Zf46KOPsGvXrkbDau7cuZg3b1699vXr18Pd3d2SQxORwmVltUdqaiT8/csBADk5Pma3CwHU1KgRHFwCL6/WHdSlpRqcPauFs7MekiS3ubrWIiioDIB8xpWX54Hp04+hd+9CO/ZUmSoqKjB+/HgUFxdDe4t3LRYH1Y2GDBmCsLAwLF++vMHbGzqjCgoKQkFBwS071xidToe0tDTEx8dDo9E0ax+KlLv2+sy3P34lOrghzWMl4suToEFlnZUlIGZp6z3LaEu13sBhH7+Qz6iSktTw9paH+w4fllBTU42779bAyUlCWRlQXAysWKFHa59/lZUFTJsm1+rpKZ9JFRbmw8/PD5L0R60rVzreGZU1Hr8lJSXo0KFDk4Kq2deojAwGg1kQ3cjFxQUuLi712jUazW0/Sa2xD0XpPhVQiwav22hQWefFWwJiPwLCprZ4F62mLdXaCId7/EK+9tSrlzyZIDQUkCT5d+vkJEGtVqGgAIiOBvr3V7X66zb9+wN9+si1ensDQshz0iVJgiSpkJcn1xoV1fprbcjtPn4t2daiu2/OnDnYvXs3zpw5g8zMTMyZMwc7d+7EhAkTLO4kNSIsSX5hhtTICsYXbgeYXNCWam0jVCp5WravrzyZoLb2jwkVOTlye3KyY0wuuLHWsjJ5uM8Ra7U3i+7CvLw8TJo0CXfeeScefPBBHDx4ED/88APi4+Nt1b+2qdEXcAd84W5LtbYRcXHAwoXy2VVtrXxNqrhYPrtYuNCx/rbIWGu/fvIwX16eh8PWak8WDf39+9//tlU/6EbGF+j9M643OPALd1uqtY2Ii5ODKiFBID+/BCtWaB1iuK8hcXHyFPSMDD22bDmGxMRYhx3us5fbvkZFNhSWJE/LPgl5MoEDXqcxaUu1thEqFaDVAlVVOvTu7dhDYCoVEBEBnD9fiIgIx67VHnh3Kp1xppuDzHi7qbZUKxE1GYOKiIgUjUFFRESKxqAiIiJFY1AREZGiMaiIiEjRGFRERKRoDCoiIlI0BhURESkag4qIiBSNQUVERIrGoCIiIkVjUBERkaIxqIiISNEYVEREpGgMKiIiUjQGFRERKRqDioiIFI1BRUREisagIiIiRWNQERGRojGoiIhI0RhURESkaAwqIiJSNAYVEREpGoOKiIgUjUFFRESKxqAiIiJFY1AREZGiMaiIiEjRGFRERKRoDCoiIlI0BhURESkag4qIiBSNQUVERIrGoCIiIkVjUBERkaIxqIiISNEYVEREpGgMKiIiUjQGFRERKRqDioiIFI1BRUREisagIiIiRWNQERGRojGoSDly15r/JGpN+Pi1GQYVKUPOSuBQivz/QynysgMyGIDMTCArqz0yM+VlR2UwACUlQGmpBllZjl1rW3n82otFQbVgwQLcfffd8PLygr+/P0aNGoVff/3VVn2jtiJnJbD/aQDieoOQlx3syZ6eDkycCCQlqZGaGomkJDUmTpTbHU16OjB1KnD8uISzZ7WYNs1xa20rj197siiodu3aheTkZOzbtw9paWnQ6XR46KGHUF5ebqv+kaOr9yQ3cqwne3o6MHs2cOQI4O0N+PuXw9sbOHpUbnekF3BjrRkZgJMT4Oysd9ha28rj196cLFl5y5YtZssff/wx/P39cfjwYQwaNMiqHaM2oNEnudH1JzsAhCW1VK+szmAAli4Frl4FunUDhADKygB3dyA0FMjJAd5/H4iKAlStfDDeYAAWLwYKC4GQEKCoCNDrAU9POaBzcoBly4B77mn9tbaVx68SWBRUNyouLgYAtGvXrtF1qqurUV1dbVouKSkBAOh0Ouh0umYd17hdc7dvTRy21ty1wKEZAFxNTTq4mf002T8D0EtA6MQW7KD1ZGYCJ06o0bGjHFInTwJXr/rg/HkJkiRQWwts3gwkJAhotfbu7e0pKZGH+5yc5JAS11/DhRAQwgB/fyArC8jI0CMiwq5dvT1t6PF7I2u9JlmyvSSEaOztwE0ZDAY88sgjKCoqwp49expdb+7cuZg3b1699vXr18Pd3b05hyZqVbKy2iM1NRL+/vIQeU6Oj9ntQgA1NWoEB5fAy6t1vyEpLdXg7FktnJ31kCS5zdW1FkFBZQDkM668PA9Mn34MvXsX2rGnZG8VFRUYP348iouLob3FO7RmB9X06dOxefNm7NmzB3fccUej6zV0RhUUFISCgoJbdq4xOp0OaWlpiI+Ph0ajadY+WguHrTV37fVZUn88/HRwQ5rHSsSXJ0GDyjorS0DM0lb7jjQzU55A4e0tD/cdPiyhpqYad9+tgZOThLIyoLgYWLFCj9697d3b25OVBUybJtfq6SmfSRUW5sPPzw+S9EetK1c6whlV23j83shar0klJSXo0KFDk4KqWUN/KSkp+O6777B79+6bhhQAuLi4wMXFpV67RqO57Rdea+yjtXC4WrtPBdSiwTF+DSrrPNElIPYjIGxqi3fRWqKigF695MkEoaGAJMn1OjlJUKtVKCgAoqOB/v1Vrf66Tf/+QJ8+cq3e3oAQ8px0SZIgSSrk5cm1RkW18lrb0OO3Mbf7mmTJthY9VIQQSElJwYYNG7B9+3aEhoZa3Dkik7Ak+UkMqZEVjE/y1n0hWqUCUlIAX195MkFt7R8TKnJy5PbkZAeYXID6tZaVycN9jlhrW3n8KoFFD5fk5GSsXbsW69evh5eXFy5fvozLly+jsrLy1hsTNaTRJ7tjPcnj4oCFC+Wzq9pa+ZpUcbF8drFwoXy7ozDW2q+fPMyXl+fhsLW2lcevvVk09JeamgoAGDx4sFn7qlWrMGXKFGv1idoa45N5/4zrDY75JI+Lk4MqIUEgP78EK1ZoHWK4ryFxcfIU9IwMPbZsOYbExNjWP9zXmDby+LUni4KqmfMuiG4tLEmewnsS8oVnBxzTB+QhL60WqKrSoXdvBxkCa4RKBUREAOfPFyIiwrFrbSuPX3tx5IcOtTbGWVEOMjuK2hg+fm2GQUVERIrGoCIiIkVjUBERkaIxqIiISNEYVEREpGgMKiIiUjQGFRERKRqDioiIFI1BRUREisagIiIiRWNQERGRojGoiIhI0RhURESkaAwqIiJSNAYVEREpGoOKiIgUjUFFRESKxqAiIiJFY1AREZGiMaiIiEjRGFRERKRoDCoiIlI0BhURESkag4qIiBSNQUVERIrGoCIiIkVjUBERkaIxqIiISNEYVEREpGgMKiIiUjQGFRERKRqDioiIFI1BRUREisagIiIiRWNQERGRojGoiIhI0RhURESkaAwqIiJSNAYVEREpGoOKiIgUjUFFRESKxqAiIiJFY1AREZGiMaiIiEjRGFREZDu5a81/EjUDg4qoBRkMQEkJUFqqQVaWvOywclYCh1Lk/x9KkZeJmsHioNq9ezdGjBiBTp06QZIkbNy40QbdInI86enA1KnA8eMSzp7VYto0NSZOlNsdTs5KYP/TAMT1BiEvM6yoGSwOqvLyckRGRmLZsmW26A+RQ0pPB2bPBjIyACcnwNlZD29v4OhRud2hwqpeSBkxrKh5nCzdYOjQoRg6dKgt+kLkkAwGYOlS4OpVICwMKCoC9HrA0xPw9gZycoBly4B77gFUrX0wvtGQMroeVgAQltRSvaJWzuKgslR1dTWqq6tNyyUlJQAAnU4HnU7XrH0at2vu9q0Ja239MjOBEyfU6NgREAIQQgIACCEghAH+/kBWFpCRoUdEhJ07ezty1wKHZgBwNTXp4Gb202T/DEAvAaETW7CDtuWoj98bWatOS7aXhBCNvfW59caShA0bNmDUqFGNrjN37lzMmzevXvv69evh7u7e3EMTtRpZWe2RmhoJf/9yAEBOjg8AICysCCqVfMaVl+eB6dOPoXfvQjv2lKjlVFRUYPz48SguLoZWq73pujYPqobOqIKCglBQUHDLzjVGp9MhLS0N8fHx0Gg0zdpHa8FaW7/MTCApSQ1vb8DdHTh8WEJNTTXuuUcDtVpCWRlQXAysXOkIZ1QpqDvsp4Mb0jxWIr48CRpU1llZAmKW2uyMSq/Xo7a2Frfx8max2tpapKenIy4uDk5ONh+sspum1ClJEpycnKBWqxvdT0lJCTp06NCkoLL5veni4gIXF5d67RqN5rZfjKyxj9aCtbZeUVFAr17yxInQUECS5BdPSZIgSSrk5QHR0UBUlKp1X6PqPhVQiwavUWlQWSeoJCD2IyBsqtW7IITA5cuXUVRUZPV9N+XYAQEBuHTpEiRJavHjtxRL6vTx8UFAQECD61nyHHfc2CdSCJUKSEmRZ/fl5AC1tfK1qrIyoKAA8PUFkpMdYCIF8McEiUYnVBhDyjYTKYwh5e/vD3d39xYNDIPBgLKyMnh6ekLlEL/MhjWlTiEEKioqkJeXBwAIDAy8rWNaHFRlZWXIzs42Lefm5iIjIwPt2rVDly5dbqszRI4qLg5YuBBYvBjYsgWoqVGjuFg+k0pOlm93GGZhVZdtQ0qv15tCqn379jY5xs0YDAbU1NTA1dXV4YOqKXW6uckTaPLy8uDv73/TYcBbsTioDh06hD/96U+m5VmzZgEAJk+ejI8//rjZHSFydHFx8jBgQoJAfn4JVqzQon//Vj7c1xhTWM243mDbkAL+mEXGSVrKYfxd6HS6lg2qwYMHt+gFSiJHolIBWi1QVaVD794OMtzXmLAkeQr6ScgTJ2xwTaohjnx9qLWx1u/CkZ8mRGRvxll9DvT3UtTyGFRERKRoDCoiIjvLz8/H9OnT0aVLF7i4uCAgIAAJCQnYu3cvAFj1A8DPnDkDSZKQkZFhlf21BE5PJyICcLrwNEprShu93cvZC+Htw21y7EcffRQ1NTVYvXo1unbtiitXrmDbtm0oLLTuJ5XU1NRYdX8thUFFRG3e6cLT6L60+y3X+y3lN6uHVVFREX766Sfs3LkT999/PwAgODgY/fv3BwCEhIQAAEaPHm267cyZM8jJycGsWbOwb98+lJeXo2fPnliwYAGGDBli2ndISAieeuopnD59Ghs3bsT/+T//B6tXrwYA9OvXDwBw//33Y+fOnVatydo49EdEbd7NzqSas54lPD094enpiY0bN5p93JzRwYMHAQCrVq3CpUuXTMtlZWUYNmwYtm3bhqNHjyIxMREjRozAuXPnzLZfuHAhIiMjcfToUbz66qs4cOAAAODHH3/EpUuX8PXXX1u9JmtjUBER2ZGTkxM+/vhjrF69Gj4+Phg4cCBeeeUVHD9+HADg5+cH4I+PIzIuR0ZG4tlnn0WfPn0QHh6Ov//97wgLC8OmTZvM9v/AAw/gz3/+M8LCwhAWFmbavn379ggICEC7du1asNrmYVAREdnZo48+iosXL2LTpk1ITEzEzp07ER0dfdMPUSgrK8Ps2bPRs2dP+Pj4wNPTEydPnqx3RhUTE2Pj3tseg4qISAFcXV0RHx+PV199Fenp6ZgyZQpef/31RtefPXs2NmzYgPnz5+Onn35CRkYGIiIi6k2Y8PDwsHXXbY5BRUSkQL169UJ5ufwdZhqNBnq93uz2vXv3YsqUKRg9ejQiIiIQEBCAM2fO3HK/zs7OAFBvf0rGoCIisqPCwkI88MADWLt2LY4fP47c3Fx88cUXePvttzFy5EgA8uy9bdu24fLly7h27RoAIDw8HF9//TUyMjJw7NgxjB8/HgaD4ZbH8/f3h5ubG7Zs2YIrV66guLjYpvVZA4OKiMiOPD09ERsbi/feew+DBg1Cnz598Oqrr+KZZ57B0qVLAQDvvPMO0tLSEBQUZJpW/u6778LX1xdxcXEYMWIEEhISEB0dfcvjOTk54f3338fy5cvRqVMnUxgqGf+OiojaPC9nL6uuZwkXFxcsWLAACxYsaHSdESNGYMSIEWZtISEh2L59u1lbcnKy2XJjQ4FPP/00nn76xq9hUS4GFRG1eeHtw/Fbym92+2QKujkGFRERwBBSMF6jIiIiRWNQERGRojGoiIhI0RhURESkaAwqIiJSNAYVEREpGoOKiIgUjUFFRNSGhISEYNGiRfbuhkUYVERECnD58mXMnDkT3bp1g6urKzp27IiBAwciNTUVFRUV9u6eXfGTKYiIjKqrgfR0YPBgQJIAIYCdO4G4OMDFxWaH/f333zFw4ED4+Phg/vz5iIiIgIuLCzIzM7FixQp07twZjzzyiM2Or3Q8oyIiAuSQGjUKeOAB4KWXAIMBePFFeXnUKPl2G3n++efh5OSEQ4cOYezYsejZsye6du2KkSNH4j//+Y/pA2nPnTuHkSNHwtPTE1qtFmPHjsWVK1dM+8nJycHIkSPRsWNHeHp64u6778aPP/5os363FAYVEZExpLZulZcXLwaio4H335eXt261WVgVFhZi69atSE5ObvTbeCVJgsFgwMiRI3H16lXs2rULaWlp+P333/H444+b1isrK8OwYcOwbds2HD16FImJiRgxYkS9r6dvbTj0R0SUng5s2WLeduzYH/83GOTbf/5ZHha0ouzsbAghcOedd5q1d+jQAVVVVQDkr+8YMmQIMjMzkZubi6CgIADA//7v/6J37944ePAg7r77bkRGRiIyMtK0j7///e/YsGEDNm3ahJSUFKv2uyXxjIqIaPBg4IUXbr7OzJnA/fe3SHcA4MCBA8jIyEDv3r1RXV2NkydPIigoyBRSgPx19T4+Pjh58iQA+Yxq9uzZ6NmzJ3x8fODp6YmTJ0/yjIqIqNWTJOC994Bdu8zPpIwiI4F335XXs7Ju3bpBkiT8+uuvZu1du3YFALi5uTV5X7Nnz0ZaWhoWLlyIbt26wc3NDWPGjEFNTY1V+9zSeEZFRCSEPIGioZAC5PZZs+T1rKx9+/aIj4/H0qVLUV5e3uh6PXv2xPnz53H+/HlT24kTJ1BUVIRevXoBAPbu3YspU6Zg9OjRiIiIQEBAQKPf8tuaMKiIiHbu/GPiRGMWL5bPuGzggw8+QG1tLWJiYvDZZ5/h5MmT+PXXX7F27VqcOnUKarUaQ4YMQUREBCZMmIAjR47gwIEDmDRpEu6//37ExMQAAMLDw/H1118jIyMDx44dw/jx42EwGGzS55bEoCIiiosDEhMBVZ2XxDqTEqBSybcPGGCTw4eFheHo0aMYMmQI5syZg8jISMTExGDJkiWYPXs2/v73v0OSJHzzzTfw9fXFoEGDMGTIEHTt2hWfffaZaT/vvvsufH19ERcXhxEjRiAhIQHR0dE26XNL4jUqIiIXF2DjRnkK+pYt8sSJd9+VhwPffx946CH5dhv+0W9gYCCWLFmCJUuWNLpOly5d8M033zR6e0hICLZv327WlpycbLbcGocCGVRERMAfYfXzz/LsPkkCFi0CRo+Wz6RsGFJ0cwwqIiIjFxfzv5OSJKv/3RRZjteoiIhI0RhURESkaAwqIiJSNAYVEREpGoOKiIgUjUFFRESKxqAiIiJFY1ARETXEoLN3D1qUJEnYuHGjvbvRIAYVEdGNclYCn3vKP1vAlClTIEkSJEmCs7MzunXrhr/97W+ora1tkePf6MyZM5AkCRkZGXY5/o34yRRERHXlrAT2Pw1AXP8JICzJ5odNTEzEqlWrUF1dje+//x7JycnQaDSYM2eOzY+tdDyjIiIyqhtSAExh1QJnVi4uLggICEBwcDCmT5+OIUOGYNOmTaiursbs2bPRuXNneHh4IDY2Fjt37jRt9/HHH8PHxwc//PADevbsCU9PTyQmJuLSpUumdQ4ePIj4+Hh06NAB3t7euP/++3HkyJFG+xIaGgoA6NevHyRJwuDBg7F7925oNBpcvnzZbN0XX3wR9913n3XvjBs0K6iWLVuGkJAQuLq6IjY2FgcOHLB2v+o5XXgaRy4dwaELR/DVzt+QldUeX+38DYcuHMGRS0dwuvC0zfvQ0gwGIDMTyMpqj8xMedlRtZVaDQagpAQoLdUgK8tx62yV6oWUUcuFVV1ubm6oqalBSkoKfv75Z3z66ac4fvw4HnvsMSQmJuL06T9e8yoqKrBw4UKsWbMGu3fvxrlz5zB79mzT7aWlpZg8eTL27NmDffv2ITw8HMOGDUNpaWmDxza+pv/444+4dOkSvv76awwaNAhdu3bF2rVrTevpdDqsW7cOSUk2PuMUFvr000+Fs7OzWLlypcjKyhLPPPOM8PHxEVeuXGnS9sXFxQKAKC4ubvIxfyv4TWAuBJIGCPRZJ6SAo+KOO0qEFHBUoM86uX0uxG8Fv1lajmLt3SvEuHFCREbqxR13lIjISL0YN05udzRtpda9e4UYO1YIrdYgXF1rRESEY9ZZV01Njdi4caOoqamx+bEqKyvFiRMnRGVlpeUbZ/9biHWSEOtwk3+SvF4j9Hq9uHbtmtDr9RYffvLkyWLkyJFCCCEMBoNIS0sTLi4uYsqUKUKtVosLFy6Yrf/ggw+KOXPmCCGEWLVqlQAgsrOzTbcvW7ZMdOzY8aZ99fLyEt9++62pDYDYsGGDEEKI3NxcAUAcPXrUbLu33npL9OzZ01TnV199JTw9PUVZWVmDx7nZ78SSLLD4jOrdd9/FM888g6lTp6JXr17417/+BXd3d6xcabt3G6U1pcC5AcDWhcClfoBrMfz9ywHXYnl560Lg3AB5PQeQng7Mng0cOQJ4ewP+/uXw9gaOHpXb09Pt3UPraSu1GuvMyACcnABnZ71D1tkqNXomdSPbnll999138PT0hKurK4YOHYrHH38cY8aMgV6vR/fu3eHp6Wn6t2vXLuTk5Ji2dXd3R1hYmGk5MDAQeXl5puUrV67gmWeeQXh4OLy9vaHValFWVoZz585Z1McpU6YgOzsbBw8eBCAPO44dOxYeHh63Wf3NWTSZoqamBocPHza7uKdSqTBkyBD8/PPPVu+ckcEA4EAKUOkLtMuGBA0AQNJUQvj8DhSFAftnoLJCQlWVzbrRIgwG+RuvCwuBsDBACKCsDHB3B0JDgZwc+XvcoqLMv4y0NWortdatMyQEKCoC9HrA01MO55wcYNky4J57WnedrZJBBxycjluHlJGQ1w99ElBprNqVP/3pT0hNTYWzszM6deoEJycnfPbZZ1Cr1Th8+DDUarXZ+p6enqb/azTmfZEkCfJJkmzy5MkoLCzE4sWLERwcDBcXFwwYMAA1NTUW9dHf3x/Dhw/HunXr0KdPH2zevNnsepmtWBRUBQUF0Ov16Nixo1l7x44dcerUqQa3qa6uRnV1tWm5pKQEgDy2qdM17e8Usk+4QCroBXjlQYIG4mI0cpy0ELXRAARgUAPZwzBjooTAdq170L+kBDh+XIKTk/yCJoSEmhofnD8vQZIEamuBzZuBhAQBrdbevb09baXW+nXK7UIICGGAvz+QlQVkZOgREWHXrlqd8Tne1Of67R5LCAGDwQBDky/+qYGYZZAOTAMgIN1kTfnXJkHELJO3u+EYxmAw9sESQgi4u7uja9eupjaDwYDIyEjo9Xpcvny5wQkLdWute8wb2/bu3YulS5ciMTERAHD+/HkUFBTU66txf05OcjTodLp6tSQlJWHChAkIDQ1FWFgYBgwY0Gi9BoMBQgjodLp6QWvJY8Lm09MXLFiAefPm1WvfunUr3N3dm7SP37Pao7NTJPzblQMAcpzkVy3t9Z9CADU1alRcK0FebV6j+2kNSks1qKnRAtBDr/+jvaZGDntjrfn5Jaiqat1/kNhWam2oTlfXWhQWFgGQX+8KCz2wZcsxnD9faLd+2lJaWprNj+Hk5ISAgACUlZVZdqbQYQycI6rglvkCRCNhZQypyoj3UdNhjPzuoxGNTVC4GZ1Oh9raWtMbeaOAgAA89thjmDRpEt544w307dsXBQUF2LVrF3r37o2EhARUVVVBCGG2bWVlJYA/Tgy6du2K1atXo0ePHigtLcVrr70GNzc3VFVV1duupKQErq6ucHNzwzfffANvb2+4uLjA29sbABAXFwcvLy/Mnz8fc+bMqdfnumpqalBZWYndu3fX+5uwioqKJt8/FgVVhw4doFarceXKFbP2K1euICAgoMFt5syZg1mzZpmWS0pKEBQUhIceegjaJr5NrnT/DReW5ODC1WJIzuVQd9QgwjMCmWWZ0As9RI0HUOWNV95RY/Sg7paUpDhZWcC0aWp4e6vh6Sm/08rPz4efnx8kSUJZGVBcDKxYoUXv3vbu7e1pK7U2VGdhYZFZnXo9kJgY65BnVGlpaYiPj683PGVtVVVVOH/+vOk6j0V6Pw/h5grpwLR6YWU6k+q/Aq5dk9DYnoUQKC0thZeXFyTpZudm9Wk0Gjg5OTX4mrhmzRq8+eabeO2113DhwgV06NABsbGxePTRR6HVauHq6gpJksy2dXNzAwBT28qVK/Hcc89h8ODBCAoKwhtvvIG//OUvcHV1rbedcXnRokV44403sGDBAtx3333Yvn27qc7x48eb5ivc7HW8qqoKbm5uGDRoUL3fyc0C7kaSqDuQ2QSxsbHo378/lixZAkA+tevSpQtSUlLwP//zP7fcvqSkBN7e3iguLm5yUB26cAR3J56SJ060y4ZKUiNaG40jJUdgEHrgajeg0xEc3NwTMZ2jLSlHcQwGYOJE+SK7fN3GgLy8PPj7+0OSVMjJAaKjgTVrWv/1jLZSa1upsyE6nQ7ff/89hg0b1iJBlZubi9DQUMuDyqjBiRUSEPvRLf/o12AwoKSkBFqtFipH+0XWYTAYMHnyZBQVFeHbb7+96bo3+51YkgUW35uzZs3Chx9+iNWrV+PkyZOYPn06ysvLMXXqVEt31WQqFYD+SwG3a8DVbhA1HjAYIJ9JXe0mt9+9zCGe5CoVkJIC+PrKF9nLyuQXurIyednXF0hOdowXtLZSa1up0yGEJcmhZDqnalpItRXFxcXYs2cPvvzyS6SkpLTYcS1+ajz++ONYuHAhXnvtNURFRSEjIwNbtmypN8HCmrycvYAuPwMPzQYCjwJV3sjLk4f70OmI3N7lZ3k9BxAXByxcCPTrJw995eV5oLhYfte9cKF8u6NoK7W2lTodgjGsVM4MqRuMHDkSiYmJmDp1KuLj41vsuBYP/d2u5gz9AfInU5TWlMJgkGcB/n7wGrre7YtuvaqhUslhFt4+3IY9b3kGA5CRocOWLfuRmBiLqCiNw77rbiu1tpU6jVrd0F9dBp1FU9Db0tBfU+u01tBfq/lQ2rohFOmvw/cV32PY4FibP/jtSaUCIiKA8+cLERHh2ENDbaXWtlKnQ7Dy30lR8/FpQkREisagIiKHYukf25LtWOt30WqG/oiIbsbZ2RkqlQoXL16En58fnJ2dLf57ptthMBhQU1ODqqoqh79Gdas6hRCoqalBfn4+VCoVnJ2db+uYDCoicggqlQqhoaG4dOkSLl682OLHF0KgsrISbm5uLRqQLc2SOt3d3dGlS5fbDm4GFRE5DGdnZ3Tp0gW1tbXQ1/1crhag0+mwe/duDBo0yKEneTW1TrVaDScnJ6uENoOKiByKJEnQaDQtHhZqtRq1tbVwdXV16KCyR52OO5BKREQOgUFFRESKxqAiIiJFa/FrVMZPbLLkI95vpNPpUFFRgZKSEoceCwZYqyNqK3UCrNURWatOYwY05VP8WjyojF8qFhQU1NKHJiIihSktLTV9KWNjWvxDaQ0GAy5evNisLxczMn754vnz5y36YNvWiLU6nrZSJ8BaHZG16jR+0WSnTp1u+XdWLX5GpVKpcMcdd1hlX1qt1qEfEHWxVsfTVuoEWKsjskadtzqTMuJkCiIiUjQGFRERKVqrDCoXFxe8/vrrcHFxsXdXbI61Op62UifAWh2RPeps8ckURERElmiVZ1RERNR2MKiIiEjRGFRERKRoDCoiIlK0VhlUy5YtQ0hICFxdXREbG4sDBw7Yu0tWt3v3bowYMQKdOnWCJEnYuHGjvbtkEwsWLMDdd98NLy8v+Pv7Y9SoUfj111/t3S2bSE1NRd++fU1/KDlgwABs3rzZ3t2yuX/84x+QJAkvvviivbtidXPnzoUkSWb/evToYe9u2cyFCxcwceJEtG/fHm5uboiIiMChQ4dsftxWF1SfffYZZs2ahddffx1HjhxBZGQkEhISkJeXZ++uWVV5eTkiIyOxbNkye3fFpnbt2oXk5GTs27cPaWlp0Ol0eOihh1BeXm7vrlndHXfcgX/84x84fPgwDh06hAceeAAjR45EVlaWvbtmMwcPHsTy5cvRt29fe3fFZnr37o1Lly6Z/u3Zs8feXbKJa9euYeDAgdBoNNi8eTNOnDiBd955B76+vrY/uGhl+vfvL5KTk03Ler1edOrUSSxYsMCOvbItAGLDhg327kaLyMvLEwDErl277N2VFuHr6ys++ugje3fDJkpLS0V4eLhIS0sT999/v5g5c6a9u2R1r7/+uoiMjLR3N1rE//2//1fce++9djl2qzqjqqmpweHDhzFkyBBTm0qlwpAhQ/Dzzz/bsWdkLcXFxQCAdu3a2bkntqXX6/Hpp5+ivLwcAwYMsHd3bCI5ORkPP/yw2fPVEZ0+fRqdOnVC165dMWHCBJw7d87eXbKJTZs2ISYmBo899hj8/f3Rr18/fPjhhy1y7FYVVAUFBdDr9ejYsaNZe8eOHXH58mU79YqsxWAw4MUXX8TAgQPRp08fe3fHJjIzM+Hp6QkXFxc899xz2LBhA3r16mXvblndp59+iiNHjmDBggX27opNxcbG4uOPP8aWLVuQmpqK3Nxc3HfffaavM3Ikv//+O1JTUxEeHo4ffvgB06dPxwsvvIDVq1fb/Ngt/unpRI1JTk7GL7/84rBj/ABw5513IiMjA8XFxfjyyy8xefJk7Nq1y6HC6vz585g5cybS0tLg6upq7+7Y1NChQ03/79u3L2JjYxEcHIzPP/8cTz31lB17Zn0GgwExMTGYP38+AKBfv3745Zdf8K9//QuTJ0+26bFb1RlVhw4doFarceXKFbP2K1euICAgwE69ImtISUnBd999hx07dljta2CUyNnZGd26dcNdd92FBQsWIDIyEosXL7Z3t6zq8OHDyMvLQ3R0NJycnODk5IRdu3bh/fffh5OTE/R6vb27aDM+Pj7o3r07srOz7d0VqwsMDKz3hqpnz54tMtTZqoLK2dkZd911F7Zt22ZqMxgM2LZtm8OO8zs6IQRSUlKwYcMGbN++HaGhofbuUosyGAyorq62dzes6sEHH0RmZiYyMjJM/2JiYjBhwgRkZGRArVbbu4s2U1ZWhpycHAQGBtq7K1Y3cODAen868ttvvyE4ONjmx251Q3+zZs3C5MmTERMTg/79+2PRokUoLy/H1KlT7d01qyorKzN7V5abm4uMjAy0a9cOXbp0sWPPrCs5ORnr16/HN998Ay8vL9O1Rm9vb7i5udm5d9Y1Z84cDB06FF26dEFpaSnWr1+PnTt34ocffrB316zKy8ur3jVGDw8PtG/f3uGuPc6ePRsjRoxAcHAwLl68iNdffx1qtRrjxo2zd9es7qWXXkJcXBzmz5+PsWPH4sCBA1ixYgVWrFhh+4PbZa7hbVqyZIno0qWLcHZ2Fv379xf79u2zd5esbseOHQJAvX+TJ0+2d9esqqEaAYhVq1bZu2tWl5SUJIKDg4Wzs7Pw8/MTDz74oNi6dau9u9UiHHV6+uOPPy4CAwOFs7Oz6Ny5s3j88cdFdna2vbtlM99++63o06ePcHFxET169BArVqxokePyaz6IiEjRWtU1KiIiansYVEREpGgMKiIiUjQGFRERKRqDioiIFI1BRUREisagIiIiRWNQERGRojGoiIhI0RhURESkaAwqIiJSNAYVEREp2v8HSl68MWJh5XkAAAAASUVORK5CYII=", "text/plain": [ "
" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "# Results\n", "\n", "optimal_reinforce_trajectory, final_reward_reinforce = reinforce_agent.get_optimal_trajectory()\n", "print(f\"Final Reward (REINFORCE): {final_reward_reinforce}\\n\")\n", "visualize_trajectory(optimal_reinforce_trajectory, \"REINFORCE Optimal Policy\")\n", "\n", "print(\"\\n--------------------------------------------------------------------------------------\\n\")\n", "\n", "optimal_genetic_trajectory, final_reward_genetic = genetic_agent.get_optimal_trajectory()\n", "print(f\"Final Reward (Genetic Algorithm): {final_reward_genetic}\\n\")\n", "visualize_trajectory(optimal_genetic_trajectory, \"Genetic Algorithm Optimal Policy\")" ] }, { "cell_type": "markdown", "metadata": { "id": "qT86bSuuVkW7" }, "source": [ "$\\bullet$ Based on the implementation and results from comparing policy search using Genetic Algorithm (GA) and the REINFORCE algorithm: \n", "\n", "**Question 1:** (10 points)\n", "\n", "How do these two methods differ in terms of their effectiveness for solving reinforcement learning tasks? \n", "\n", "**Question 2:** (15 points)\n", "\n", "Discuss the key differences in their **performance**, **convergence rates**, and **stability**. \n", "\n", "**Question 3:** (15 points)\n", "\n", "Additionally, explore how each method handles exploration and exploitation, and suggest situations where one might be preferred over the other. \n", "\n", "
\n", "............\n", "
\n", "............\n", "
\n", "............" ] } ], "metadata": { "accelerator": "GPU", "colab": { "gpuType": "T4", "provenance": [] }, "kernelspec": { "display_name": "Python 3", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.11.8" } }, "nbformat": 4, "nbformat_minor": 0 }