{ "cells": [ { "cell_type": "markdown", "metadata": { "id": "ctpoSTL9oGq7" }, "source": [ "# Gym environments" ] }, { "cell_type": "markdown", "metadata": { "id": "PsWSoHfooGrJ" }, "source": [ "## Installing gym\n", "\n", "In this course, we will mostly address RL environments available in the **OpenAI Gym** framework:\n", "\n", "\n", "\n", "It provides a multitude of RL problems, from simple text-based problems with a few dozens of states (Gridworld, Taxi) to continuous control problems (Cartpole, Pendulum) to Atari games (Breakout, Space Invaders) to complex robotics simulators (Mujoco):\n", "\n", "\n", "\n", "However, `gym` is not maintained by OpenAI anymore since September 2022. We will use instead the `gymnasium` library maintained by the Farama foundation, which will keep on maintaining and improving the library.\n", "\n", "\n", "\n", "You can install gymnasium and its dependencies using:\n", "\n", "```bash\n", "pip install -U gymnasium pygame moviepy swig\n", "pip install gymnasium[classic_control]\n", "pip install gymnasium[box2d]\n", "```\n", "\n", "For this exercise and the following, we will focus on simple environments whose installation is straightforward: toy text, classic control and box2d. More complex environments based on Atari games or the Mujoco physics simulator are described in the last (optional) section of this notebook, as they require additional dependencies. \n", "\n", "On colab, `gym` cannot open graphical windows for visualizing the environments, as it is not possible in the browser. We will see a workaround allowing to produce videos. Running that cell in colab should allow you to run the simplest environments:" ] }, { "cell_type": "code", "execution_count": 1, "metadata": { "id": "YxLOmLKIoGrN" }, "outputs": [], "source": [ "try:\n", " import google.colab\n", " IN_COLAB = True\n", "except:\n", " IN_COLAB = False\n", "\n", "if IN_COLAB:\n", " !pip install -U gymnasium pygame moviepy swig\n", " !pip install gymnasium[box2d]\n" ] }, { "cell_type": "code", "execution_count": 2, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "gym version: 0.29.1\n" ] } ], "source": [ "import numpy as np\n", "import matplotlib.pyplot as plt\n", "import os\n", "\n", "import gymnasium as gym\n", "print(\"gym version:\", gym.__version__)\n", "\n", "from moviepy.editor import ImageSequenceClip, ipython_display\n", "\n", "class GymRecorder(object):\n", " \"\"\"\n", " Simple wrapper over moviepy to generate a .gif with the frames of a gym environment.\n", " \n", " The environment must have the render_mode `rgb_array_list`.\n", " \"\"\"\n", " def __init__(self, env):\n", " self.env = env\n", " self._frames = []\n", "\n", " def record(self, frames):\n", " \"To be called at the end of an episode.\"\n", " for frame in frames:\n", " self._frames.append(np.array(frame))\n", "\n", " def make_video(self, filename):\n", " \"Generates the gif video.\"\n", " directory = os.path.dirname(os.path.abspath(filename))\n", " if not os.path.exists(directory):\n", " os.mkdir(directory)\n", " self.clip = ImageSequenceClip(list(self._frames), fps=self.env.metadata[\"render_fps\"])\n", " self.clip.write_gif(filename, fps=self.env.metadata[\"render_fps\"], loop=0)\n", " del self._frames\n", " self._frames = []" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Interacting with an environment\n", "\n", "A gym environment is created using:\n", "\n", "```python\n", "env = gym.make('CartPole-v1', render_mode=\"human\")\n", "```\n", "\n", "where 'CartPole-v1' should be replaced by the environment you want to interact with. The following cell lists the environments available to you (including the different versions)." ] }, { "cell_type": "code", "execution_count": 3, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "CartPole-v0\n", "CartPole-v1\n", "MountainCar-v0\n", "MountainCarContinuous-v0\n", "Pendulum-v1\n", "Acrobot-v1\n", "phys2d/CartPole-v0\n", "phys2d/CartPole-v1\n", "phys2d/Pendulum-v0\n", "LunarLander-v2\n", "LunarLanderContinuous-v2\n", "BipedalWalker-v3\n", "BipedalWalkerHardcore-v3\n", "CarRacing-v2\n", "Blackjack-v1\n", "FrozenLake-v1\n", "FrozenLake8x8-v1\n", "CliffWalking-v0\n", "Taxi-v3\n", "tabular/Blackjack-v0\n", "tabular/CliffWalking-v0\n", "Reacher-v2\n", "Reacher-v4\n", "Pusher-v2\n", "Pusher-v4\n", "InvertedPendulum-v2\n", "InvertedPendulum-v4\n", "InvertedDoublePendulum-v2\n", "InvertedDoublePendulum-v4\n", "HalfCheetah-v2\n", "HalfCheetah-v3\n", "HalfCheetah-v4\n", "Hopper-v2\n", "Hopper-v3\n", "Hopper-v4\n", "Swimmer-v2\n", "Swimmer-v3\n", "Swimmer-v4\n", "Walker2d-v2\n", "Walker2d-v3\n", "Walker2d-v4\n", "Ant-v2\n", "Ant-v3\n", "Ant-v4\n", "Humanoid-v2\n", "Humanoid-v3\n", "Humanoid-v4\n", "HumanoidStandup-v2\n", "HumanoidStandup-v4\n", "GymV26Environment-v0\n", "GymV21Environment-v0\n", "Adventure-v0\n", "AdventureDeterministic-v0\n", "AdventureNoFrameskip-v0\n", "Adventure-v4\n", "AdventureDeterministic-v4\n", "AdventureNoFrameskip-v4\n", "Adventure-ram-v0\n", "Adventure-ramDeterministic-v0\n", "Adventure-ramNoFrameskip-v0\n", "Adventure-ram-v4\n", "Adventure-ramDeterministic-v4\n", "Adventure-ramNoFrameskip-v4\n", "AirRaid-v0\n", "AirRaidDeterministic-v0\n", "AirRaidNoFrameskip-v0\n", "AirRaid-v4\n", "AirRaidDeterministic-v4\n", "AirRaidNoFrameskip-v4\n", "AirRaid-ram-v0\n", "AirRaid-ramDeterministic-v0\n", "AirRaid-ramNoFrameskip-v0\n", "AirRaid-ram-v4\n", "AirRaid-ramDeterministic-v4\n", "AirRaid-ramNoFrameskip-v4\n", "Alien-v0\n", "AlienDeterministic-v0\n", "AlienNoFrameskip-v0\n", "Alien-v4\n", "AlienDeterministic-v4\n", "AlienNoFrameskip-v4\n", "Alien-ram-v0\n", "Alien-ramDeterministic-v0\n", "Alien-ramNoFrameskip-v0\n", "Alien-ram-v4\n", "Alien-ramDeterministic-v4\n", "Alien-ramNoFrameskip-v4\n", "Amidar-v0\n", "AmidarDeterministic-v0\n", "AmidarNoFrameskip-v0\n", "Amidar-v4\n", "AmidarDeterministic-v4\n", "AmidarNoFrameskip-v4\n", "Amidar-ram-v0\n", "Amidar-ramDeterministic-v0\n", "Amidar-ramNoFrameskip-v0\n", "Amidar-ram-v4\n", "Amidar-ramDeterministic-v4\n", "Amidar-ramNoFrameskip-v4\n", "Assault-v0\n", "AssaultDeterministic-v0\n", "AssaultNoFrameskip-v0\n", "Assault-v4\n", "AssaultDeterministic-v4\n", "AssaultNoFrameskip-v4\n", "Assault-ram-v0\n", "Assault-ramDeterministic-v0\n", "Assault-ramNoFrameskip-v0\n", "Assault-ram-v4\n", "Assault-ramDeterministic-v4\n", "Assault-ramNoFrameskip-v4\n", "Asterix-v0\n", "AsterixDeterministic-v0\n", "AsterixNoFrameskip-v0\n", "Asterix-v4\n", "AsterixDeterministic-v4\n", "AsterixNoFrameskip-v4\n", "Asterix-ram-v0\n", "Asterix-ramDeterministic-v0\n", "Asterix-ramNoFrameskip-v0\n", "Asterix-ram-v4\n", "Asterix-ramDeterministic-v4\n", "Asterix-ramNoFrameskip-v4\n", "Asteroids-v0\n", "AsteroidsDeterministic-v0\n", "AsteroidsNoFrameskip-v0\n", "Asteroids-v4\n", "AsteroidsDeterministic-v4\n", "AsteroidsNoFrameskip-v4\n", "Asteroids-ram-v0\n", "Asteroids-ramDeterministic-v0\n", "Asteroids-ramNoFrameskip-v0\n", "Asteroids-ram-v4\n", "Asteroids-ramDeterministic-v4\n", "Asteroids-ramNoFrameskip-v4\n", "Atlantis-v0\n", "AtlantisDeterministic-v0\n", "AtlantisNoFrameskip-v0\n", "Atlantis-v4\n", "AtlantisDeterministic-v4\n", "AtlantisNoFrameskip-v4\n", "Atlantis-ram-v0\n", "Atlantis-ramDeterministic-v0\n", "Atlantis-ramNoFrameskip-v0\n", "Atlantis-ram-v4\n", "Atlantis-ramDeterministic-v4\n", "Atlantis-ramNoFrameskip-v4\n", "BankHeist-v0\n", "BankHeistDeterministic-v0\n", "BankHeistNoFrameskip-v0\n", "BankHeist-v4\n", "BankHeistDeterministic-v4\n", "BankHeistNoFrameskip-v4\n", "BankHeist-ram-v0\n", "BankHeist-ramDeterministic-v0\n", "BankHeist-ramNoFrameskip-v0\n", "BankHeist-ram-v4\n", "BankHeist-ramDeterministic-v4\n", "BankHeist-ramNoFrameskip-v4\n", "BattleZone-v0\n", "BattleZoneDeterministic-v0\n", "BattleZoneNoFrameskip-v0\n", "BattleZone-v4\n", "BattleZoneDeterministic-v4\n", "BattleZoneNoFrameskip-v4\n", "BattleZone-ram-v0\n", "BattleZone-ramDeterministic-v0\n", "BattleZone-ramNoFrameskip-v0\n", "BattleZone-ram-v4\n", "BattleZone-ramDeterministic-v4\n", "BattleZone-ramNoFrameskip-v4\n", "BeamRider-v0\n", "BeamRiderDeterministic-v0\n", "BeamRiderNoFrameskip-v0\n", "BeamRider-v4\n", "BeamRiderDeterministic-v4\n", "BeamRiderNoFrameskip-v4\n", "BeamRider-ram-v0\n", "BeamRider-ramDeterministic-v0\n", "BeamRider-ramNoFrameskip-v0\n", "BeamRider-ram-v4\n", "BeamRider-ramDeterministic-v4\n", "BeamRider-ramNoFrameskip-v4\n", "Berzerk-v0\n", "BerzerkDeterministic-v0\n", "BerzerkNoFrameskip-v0\n", "Berzerk-v4\n", "BerzerkDeterministic-v4\n", "BerzerkNoFrameskip-v4\n", "Berzerk-ram-v0\n", "Berzerk-ramDeterministic-v0\n", "Berzerk-ramNoFrameskip-v0\n", "Berzerk-ram-v4\n", "Berzerk-ramDeterministic-v4\n", "Berzerk-ramNoFrameskip-v4\n", "Bowling-v0\n", "BowlingDeterministic-v0\n", "BowlingNoFrameskip-v0\n", "Bowling-v4\n", "BowlingDeterministic-v4\n", "BowlingNoFrameskip-v4\n", "Bowling-ram-v0\n", "Bowling-ramDeterministic-v0\n", "Bowling-ramNoFrameskip-v0\n", "Bowling-ram-v4\n", "Bowling-ramDeterministic-v4\n", "Bowling-ramNoFrameskip-v4\n", "Boxing-v0\n", "BoxingDeterministic-v0\n", "BoxingNoFrameskip-v0\n", "Boxing-v4\n", "BoxingDeterministic-v4\n", "BoxingNoFrameskip-v4\n", "Boxing-ram-v0\n", "Boxing-ramDeterministic-v0\n", "Boxing-ramNoFrameskip-v0\n", "Boxing-ram-v4\n", "Boxing-ramDeterministic-v4\n", "Boxing-ramNoFrameskip-v4\n", "Breakout-v0\n", "BreakoutDeterministic-v0\n", "BreakoutNoFrameskip-v0\n", "Breakout-v4\n", "BreakoutDeterministic-v4\n", "BreakoutNoFrameskip-v4\n", "Breakout-ram-v0\n", "Breakout-ramDeterministic-v0\n", "Breakout-ramNoFrameskip-v0\n", "Breakout-ram-v4\n", "Breakout-ramDeterministic-v4\n", "Breakout-ramNoFrameskip-v4\n", "Carnival-v0\n", "CarnivalDeterministic-v0\n", "CarnivalNoFrameskip-v0\n", "Carnival-v4\n", "CarnivalDeterministic-v4\n", "CarnivalNoFrameskip-v4\n", "Carnival-ram-v0\n", "Carnival-ramDeterministic-v0\n", "Carnival-ramNoFrameskip-v0\n", "Carnival-ram-v4\n", "Carnival-ramDeterministic-v4\n", "Carnival-ramNoFrameskip-v4\n", "Centipede-v0\n", "CentipedeDeterministic-v0\n", "CentipedeNoFrameskip-v0\n", "Centipede-v4\n", "CentipedeDeterministic-v4\n", "CentipedeNoFrameskip-v4\n", "Centipede-ram-v0\n", "Centipede-ramDeterministic-v0\n", "Centipede-ramNoFrameskip-v0\n", "Centipede-ram-v4\n", "Centipede-ramDeterministic-v4\n", "Centipede-ramNoFrameskip-v4\n", "ChopperCommand-v0\n", "ChopperCommandDeterministic-v0\n", "ChopperCommandNoFrameskip-v0\n", "ChopperCommand-v4\n", "ChopperCommandDeterministic-v4\n", "ChopperCommandNoFrameskip-v4\n", "ChopperCommand-ram-v0\n", "ChopperCommand-ramDeterministic-v0\n", "ChopperCommand-ramNoFrameskip-v0\n", "ChopperCommand-ram-v4\n", "ChopperCommand-ramDeterministic-v4\n", "ChopperCommand-ramNoFrameskip-v4\n", "CrazyClimber-v0\n", "CrazyClimberDeterministic-v0\n", "CrazyClimberNoFrameskip-v0\n", "CrazyClimber-v4\n", "CrazyClimberDeterministic-v4\n", "CrazyClimberNoFrameskip-v4\n", "CrazyClimber-ram-v0\n", "CrazyClimber-ramDeterministic-v0\n", "CrazyClimber-ramNoFrameskip-v0\n", "CrazyClimber-ram-v4\n", "CrazyClimber-ramDeterministic-v4\n", "CrazyClimber-ramNoFrameskip-v4\n", "Defender-v0\n", "DefenderDeterministic-v0\n", "DefenderNoFrameskip-v0\n", "Defender-v4\n", "DefenderDeterministic-v4\n", "DefenderNoFrameskip-v4\n", "Defender-ram-v0\n", "Defender-ramDeterministic-v0\n", "Defender-ramNoFrameskip-v0\n", "Defender-ram-v4\n", "Defender-ramDeterministic-v4\n", "Defender-ramNoFrameskip-v4\n", "DemonAttack-v0\n", "DemonAttackDeterministic-v0\n", "DemonAttackNoFrameskip-v0\n", "DemonAttack-v4\n", "DemonAttackDeterministic-v4\n", "DemonAttackNoFrameskip-v4\n", "DemonAttack-ram-v0\n", "DemonAttack-ramDeterministic-v0\n", "DemonAttack-ramNoFrameskip-v0\n", "DemonAttack-ram-v4\n", "DemonAttack-ramDeterministic-v4\n", "DemonAttack-ramNoFrameskip-v4\n", "DoubleDunk-v0\n", "DoubleDunkDeterministic-v0\n", "DoubleDunkNoFrameskip-v0\n", "DoubleDunk-v4\n", "DoubleDunkDeterministic-v4\n", "DoubleDunkNoFrameskip-v4\n", "DoubleDunk-ram-v0\n", "DoubleDunk-ramDeterministic-v0\n", "DoubleDunk-ramNoFrameskip-v0\n", "DoubleDunk-ram-v4\n", "DoubleDunk-ramDeterministic-v4\n", "DoubleDunk-ramNoFrameskip-v4\n", "ElevatorAction-v0\n", "ElevatorActionDeterministic-v0\n", "ElevatorActionNoFrameskip-v0\n", "ElevatorAction-v4\n", "ElevatorActionDeterministic-v4\n", "ElevatorActionNoFrameskip-v4\n", "ElevatorAction-ram-v0\n", "ElevatorAction-ramDeterministic-v0\n", "ElevatorAction-ramNoFrameskip-v0\n", "ElevatorAction-ram-v4\n", "ElevatorAction-ramDeterministic-v4\n", "ElevatorAction-ramNoFrameskip-v4\n", "Enduro-v0\n", "EnduroDeterministic-v0\n", "EnduroNoFrameskip-v0\n", "Enduro-v4\n", "EnduroDeterministic-v4\n", "EnduroNoFrameskip-v4\n", "Enduro-ram-v0\n", "Enduro-ramDeterministic-v0\n", "Enduro-ramNoFrameskip-v0\n", "Enduro-ram-v4\n", "Enduro-ramDeterministic-v4\n", "Enduro-ramNoFrameskip-v4\n", "FishingDerby-v0\n", "FishingDerbyDeterministic-v0\n", "FishingDerbyNoFrameskip-v0\n", "FishingDerby-v4\n", "FishingDerbyDeterministic-v4\n", "FishingDerbyNoFrameskip-v4\n", "FishingDerby-ram-v0\n", "FishingDerby-ramDeterministic-v0\n", "FishingDerby-ramNoFrameskip-v0\n", "FishingDerby-ram-v4\n", "FishingDerby-ramDeterministic-v4\n", "FishingDerby-ramNoFrameskip-v4\n", "Freeway-v0\n", "FreewayDeterministic-v0\n", "FreewayNoFrameskip-v0\n", "Freeway-v4\n", "FreewayDeterministic-v4\n", "FreewayNoFrameskip-v4\n", "Freeway-ram-v0\n", "Freeway-ramDeterministic-v0\n", "Freeway-ramNoFrameskip-v0\n", "Freeway-ram-v4\n", "Freeway-ramDeterministic-v4\n", "Freeway-ramNoFrameskip-v4\n", "Frostbite-v0\n", "FrostbiteDeterministic-v0\n", "FrostbiteNoFrameskip-v0\n", "Frostbite-v4\n", "FrostbiteDeterministic-v4\n", "FrostbiteNoFrameskip-v4\n", "Frostbite-ram-v0\n", "Frostbite-ramDeterministic-v0\n", "Frostbite-ramNoFrameskip-v0\n", "Frostbite-ram-v4\n", "Frostbite-ramDeterministic-v4\n", "Frostbite-ramNoFrameskip-v4\n", "Gopher-v0\n", "GopherDeterministic-v0\n", "GopherNoFrameskip-v0\n", "Gopher-v4\n", "GopherDeterministic-v4\n", "GopherNoFrameskip-v4\n", "Gopher-ram-v0\n", "Gopher-ramDeterministic-v0\n", "Gopher-ramNoFrameskip-v0\n", "Gopher-ram-v4\n", "Gopher-ramDeterministic-v4\n", "Gopher-ramNoFrameskip-v4\n", "Gravitar-v0\n", "GravitarDeterministic-v0\n", "GravitarNoFrameskip-v0\n", "Gravitar-v4\n", "GravitarDeterministic-v4\n", "GravitarNoFrameskip-v4\n", "Gravitar-ram-v0\n", "Gravitar-ramDeterministic-v0\n", "Gravitar-ramNoFrameskip-v0\n", "Gravitar-ram-v4\n", "Gravitar-ramDeterministic-v4\n", "Gravitar-ramNoFrameskip-v4\n", "Hero-v0\n", "HeroDeterministic-v0\n", "HeroNoFrameskip-v0\n", "Hero-v4\n", "HeroDeterministic-v4\n", "HeroNoFrameskip-v4\n", "Hero-ram-v0\n", "Hero-ramDeterministic-v0\n", "Hero-ramNoFrameskip-v0\n", "Hero-ram-v4\n", "Hero-ramDeterministic-v4\n", "Hero-ramNoFrameskip-v4\n", "IceHockey-v0\n", "IceHockeyDeterministic-v0\n", "IceHockeyNoFrameskip-v0\n", "IceHockey-v4\n", "IceHockeyDeterministic-v4\n", "IceHockeyNoFrameskip-v4\n", "IceHockey-ram-v0\n", "IceHockey-ramDeterministic-v0\n", "IceHockey-ramNoFrameskip-v0\n", "IceHockey-ram-v4\n", "IceHockey-ramDeterministic-v4\n", "IceHockey-ramNoFrameskip-v4\n", "Jamesbond-v0\n", "JamesbondDeterministic-v0\n", "JamesbondNoFrameskip-v0\n", "Jamesbond-v4\n", "JamesbondDeterministic-v4\n", "JamesbondNoFrameskip-v4\n", "Jamesbond-ram-v0\n", "Jamesbond-ramDeterministic-v0\n", "Jamesbond-ramNoFrameskip-v0\n", "Jamesbond-ram-v4\n", "Jamesbond-ramDeterministic-v4\n", "Jamesbond-ramNoFrameskip-v4\n", "JourneyEscape-v0\n", "JourneyEscapeDeterministic-v0\n", "JourneyEscapeNoFrameskip-v0\n", "JourneyEscape-v4\n", "JourneyEscapeDeterministic-v4\n", "JourneyEscapeNoFrameskip-v4\n", "JourneyEscape-ram-v0\n", "JourneyEscape-ramDeterministic-v0\n", "JourneyEscape-ramNoFrameskip-v0\n", "JourneyEscape-ram-v4\n", "JourneyEscape-ramDeterministic-v4\n", "JourneyEscape-ramNoFrameskip-v4\n", "Kangaroo-v0\n", "KangarooDeterministic-v0\n", "KangarooNoFrameskip-v0\n", "Kangaroo-v4\n", "KangarooDeterministic-v4\n", "KangarooNoFrameskip-v4\n", "Kangaroo-ram-v0\n", "Kangaroo-ramDeterministic-v0\n", "Kangaroo-ramNoFrameskip-v0\n", "Kangaroo-ram-v4\n", "Kangaroo-ramDeterministic-v4\n", "Kangaroo-ramNoFrameskip-v4\n", "Krull-v0\n", "KrullDeterministic-v0\n", "KrullNoFrameskip-v0\n", "Krull-v4\n", "KrullDeterministic-v4\n", "KrullNoFrameskip-v4\n", "Krull-ram-v0\n", "Krull-ramDeterministic-v0\n", "Krull-ramNoFrameskip-v0\n", "Krull-ram-v4\n", "Krull-ramDeterministic-v4\n", "Krull-ramNoFrameskip-v4\n", "KungFuMaster-v0\n", "KungFuMasterDeterministic-v0\n", "KungFuMasterNoFrameskip-v0\n", "KungFuMaster-v4\n", "KungFuMasterDeterministic-v4\n", "KungFuMasterNoFrameskip-v4\n", "KungFuMaster-ram-v0\n", "KungFuMaster-ramDeterministic-v0\n", "KungFuMaster-ramNoFrameskip-v0\n", "KungFuMaster-ram-v4\n", "KungFuMaster-ramDeterministic-v4\n", "KungFuMaster-ramNoFrameskip-v4\n", "MontezumaRevenge-v0\n", "MontezumaRevengeDeterministic-v0\n", "MontezumaRevengeNoFrameskip-v0\n", "MontezumaRevenge-v4\n", "MontezumaRevengeDeterministic-v4\n", "MontezumaRevengeNoFrameskip-v4\n", "MontezumaRevenge-ram-v0\n", "MontezumaRevenge-ramDeterministic-v0\n", "MontezumaRevenge-ramNoFrameskip-v0\n", "MontezumaRevenge-ram-v4\n", "MontezumaRevenge-ramDeterministic-v4\n", "MontezumaRevenge-ramNoFrameskip-v4\n", "MsPacman-v0\n", "MsPacmanDeterministic-v0\n", "MsPacmanNoFrameskip-v0\n", "MsPacman-v4\n", "MsPacmanDeterministic-v4\n", "MsPacmanNoFrameskip-v4\n", "MsPacman-ram-v0\n", "MsPacman-ramDeterministic-v0\n", "MsPacman-ramNoFrameskip-v0\n", "MsPacman-ram-v4\n", "MsPacman-ramDeterministic-v4\n", "MsPacman-ramNoFrameskip-v4\n", "NameThisGame-v0\n", "NameThisGameDeterministic-v0\n", "NameThisGameNoFrameskip-v0\n", "NameThisGame-v4\n", "NameThisGameDeterministic-v4\n", "NameThisGameNoFrameskip-v4\n", "NameThisGame-ram-v0\n", "NameThisGame-ramDeterministic-v0\n", "NameThisGame-ramNoFrameskip-v0\n", "NameThisGame-ram-v4\n", "NameThisGame-ramDeterministic-v4\n", "NameThisGame-ramNoFrameskip-v4\n", "Phoenix-v0\n", "PhoenixDeterministic-v0\n", "PhoenixNoFrameskip-v0\n", "Phoenix-v4\n", "PhoenixDeterministic-v4\n", "PhoenixNoFrameskip-v4\n", "Phoenix-ram-v0\n", "Phoenix-ramDeterministic-v0\n", "Phoenix-ramNoFrameskip-v0\n", "Phoenix-ram-v4\n", "Phoenix-ramDeterministic-v4\n", "Phoenix-ramNoFrameskip-v4\n", "Pitfall-v0\n", "PitfallDeterministic-v0\n", "PitfallNoFrameskip-v0\n", "Pitfall-v4\n", "PitfallDeterministic-v4\n", "PitfallNoFrameskip-v4\n", "Pitfall-ram-v0\n", "Pitfall-ramDeterministic-v0\n", "Pitfall-ramNoFrameskip-v0\n", "Pitfall-ram-v4\n", "Pitfall-ramDeterministic-v4\n", "Pitfall-ramNoFrameskip-v4\n", "Pong-v0\n", "PongDeterministic-v0\n", "PongNoFrameskip-v0\n", "Pong-v4\n", "PongDeterministic-v4\n", "PongNoFrameskip-v4\n", "Pong-ram-v0\n", "Pong-ramDeterministic-v0\n", "Pong-ramNoFrameskip-v0\n", "Pong-ram-v4\n", "Pong-ramDeterministic-v4\n", "Pong-ramNoFrameskip-v4\n", "Pooyan-v0\n", "PooyanDeterministic-v0\n", "PooyanNoFrameskip-v0\n", "Pooyan-v4\n", "PooyanDeterministic-v4\n", "PooyanNoFrameskip-v4\n", "Pooyan-ram-v0\n", "Pooyan-ramDeterministic-v0\n", "Pooyan-ramNoFrameskip-v0\n", "Pooyan-ram-v4\n", "Pooyan-ramDeterministic-v4\n", "Pooyan-ramNoFrameskip-v4\n", "PrivateEye-v0\n", "PrivateEyeDeterministic-v0\n", "PrivateEyeNoFrameskip-v0\n", "PrivateEye-v4\n", "PrivateEyeDeterministic-v4\n", "PrivateEyeNoFrameskip-v4\n", "PrivateEye-ram-v0\n", "PrivateEye-ramDeterministic-v0\n", "PrivateEye-ramNoFrameskip-v0\n", "PrivateEye-ram-v4\n", "PrivateEye-ramDeterministic-v4\n", "PrivateEye-ramNoFrameskip-v4\n", "Qbert-v0\n", "QbertDeterministic-v0\n", "QbertNoFrameskip-v0\n", "Qbert-v4\n", "QbertDeterministic-v4\n", "QbertNoFrameskip-v4\n", "Qbert-ram-v0\n", "Qbert-ramDeterministic-v0\n", "Qbert-ramNoFrameskip-v0\n", "Qbert-ram-v4\n", "Qbert-ramDeterministic-v4\n", "Qbert-ramNoFrameskip-v4\n", "Riverraid-v0\n", "RiverraidDeterministic-v0\n", "RiverraidNoFrameskip-v0\n", "Riverraid-v4\n", "RiverraidDeterministic-v4\n", "RiverraidNoFrameskip-v4\n", "Riverraid-ram-v0\n", "Riverraid-ramDeterministic-v0\n", "Riverraid-ramNoFrameskip-v0\n", "Riverraid-ram-v4\n", "Riverraid-ramDeterministic-v4\n", "Riverraid-ramNoFrameskip-v4\n", "RoadRunner-v0\n", "RoadRunnerDeterministic-v0\n", "RoadRunnerNoFrameskip-v0\n", "RoadRunner-v4\n", "RoadRunnerDeterministic-v4\n", "RoadRunnerNoFrameskip-v4\n", "RoadRunner-ram-v0\n", "RoadRunner-ramDeterministic-v0\n", "RoadRunner-ramNoFrameskip-v0\n", "RoadRunner-ram-v4\n", "RoadRunner-ramDeterministic-v4\n", "RoadRunner-ramNoFrameskip-v4\n", "Robotank-v0\n", "RobotankDeterministic-v0\n", "RobotankNoFrameskip-v0\n", "Robotank-v4\n", "RobotankDeterministic-v4\n", "RobotankNoFrameskip-v4\n", "Robotank-ram-v0\n", "Robotank-ramDeterministic-v0\n", "Robotank-ramNoFrameskip-v0\n", "Robotank-ram-v4\n", "Robotank-ramDeterministic-v4\n", "Robotank-ramNoFrameskip-v4\n", "Seaquest-v0\n", "SeaquestDeterministic-v0\n", "SeaquestNoFrameskip-v0\n", "Seaquest-v4\n", "SeaquestDeterministic-v4\n", "SeaquestNoFrameskip-v4\n", "Seaquest-ram-v0\n", "Seaquest-ramDeterministic-v0\n", "Seaquest-ramNoFrameskip-v0\n", "Seaquest-ram-v4\n", "Seaquest-ramDeterministic-v4\n", "Seaquest-ramNoFrameskip-v4\n", "Skiing-v0\n", "SkiingDeterministic-v0\n", "SkiingNoFrameskip-v0\n", "Skiing-v4\n", "SkiingDeterministic-v4\n", "SkiingNoFrameskip-v4\n", "Skiing-ram-v0\n", "Skiing-ramDeterministic-v0\n", "Skiing-ramNoFrameskip-v0\n", "Skiing-ram-v4\n", "Skiing-ramDeterministic-v4\n", "Skiing-ramNoFrameskip-v4\n", "Solaris-v0\n", "SolarisDeterministic-v0\n", "SolarisNoFrameskip-v0\n", "Solaris-v4\n", "SolarisDeterministic-v4\n", "SolarisNoFrameskip-v4\n", "Solaris-ram-v0\n", "Solaris-ramDeterministic-v0\n", "Solaris-ramNoFrameskip-v0\n", "Solaris-ram-v4\n", "Solaris-ramDeterministic-v4\n", "Solaris-ramNoFrameskip-v4\n", "SpaceInvaders-v0\n", "SpaceInvadersDeterministic-v0\n", "SpaceInvadersNoFrameskip-v0\n", "SpaceInvaders-v4\n", "SpaceInvadersDeterministic-v4\n", "SpaceInvadersNoFrameskip-v4\n", "SpaceInvaders-ram-v0\n", "SpaceInvaders-ramDeterministic-v0\n", "SpaceInvaders-ramNoFrameskip-v0\n", "SpaceInvaders-ram-v4\n", "SpaceInvaders-ramDeterministic-v4\n", "SpaceInvaders-ramNoFrameskip-v4\n", "StarGunner-v0\n", "StarGunnerDeterministic-v0\n", "StarGunnerNoFrameskip-v0\n", "StarGunner-v4\n", "StarGunnerDeterministic-v4\n", "StarGunnerNoFrameskip-v4\n", "StarGunner-ram-v0\n", "StarGunner-ramDeterministic-v0\n", "StarGunner-ramNoFrameskip-v0\n", "StarGunner-ram-v4\n", "StarGunner-ramDeterministic-v4\n", "StarGunner-ramNoFrameskip-v4\n", "Tennis-v0\n", "TennisDeterministic-v0\n", "TennisNoFrameskip-v0\n", "Tennis-v4\n", "TennisDeterministic-v4\n", "TennisNoFrameskip-v4\n", "Tennis-ram-v0\n", "Tennis-ramDeterministic-v0\n", "Tennis-ramNoFrameskip-v0\n", "Tennis-ram-v4\n", "Tennis-ramDeterministic-v4\n", "Tennis-ramNoFrameskip-v4\n", "TimePilot-v0\n", "TimePilotDeterministic-v0\n", "TimePilotNoFrameskip-v0\n", "TimePilot-v4\n", "TimePilotDeterministic-v4\n", "TimePilotNoFrameskip-v4\n", "TimePilot-ram-v0\n", "TimePilot-ramDeterministic-v0\n", "TimePilot-ramNoFrameskip-v0\n", "TimePilot-ram-v4\n", "TimePilot-ramDeterministic-v4\n", "TimePilot-ramNoFrameskip-v4\n", "Tutankham-v0\n", "TutankhamDeterministic-v0\n", "TutankhamNoFrameskip-v0\n", "Tutankham-v4\n", "TutankhamDeterministic-v4\n", "TutankhamNoFrameskip-v4\n", "Tutankham-ram-v0\n", "Tutankham-ramDeterministic-v0\n", "Tutankham-ramNoFrameskip-v0\n", "Tutankham-ram-v4\n", "Tutankham-ramDeterministic-v4\n", "Tutankham-ramNoFrameskip-v4\n", "UpNDown-v0\n", "UpNDownDeterministic-v0\n", "UpNDownNoFrameskip-v0\n", "UpNDown-v4\n", "UpNDownDeterministic-v4\n", "UpNDownNoFrameskip-v4\n", "UpNDown-ram-v0\n", "UpNDown-ramDeterministic-v0\n", "UpNDown-ramNoFrameskip-v0\n", "UpNDown-ram-v4\n", "UpNDown-ramDeterministic-v4\n", "UpNDown-ramNoFrameskip-v4\n", "Venture-v0\n", "VentureDeterministic-v0\n", "VentureNoFrameskip-v0\n", "Venture-v4\n", "VentureDeterministic-v4\n", "VentureNoFrameskip-v4\n", "Venture-ram-v0\n", "Venture-ramDeterministic-v0\n", "Venture-ramNoFrameskip-v0\n", "Venture-ram-v4\n", "Venture-ramDeterministic-v4\n", "Venture-ramNoFrameskip-v4\n", "VideoPinball-v0\n", "VideoPinballDeterministic-v0\n", "VideoPinballNoFrameskip-v0\n", "VideoPinball-v4\n", "VideoPinballDeterministic-v4\n", "VideoPinballNoFrameskip-v4\n", "VideoPinball-ram-v0\n", "VideoPinball-ramDeterministic-v0\n", "VideoPinball-ramNoFrameskip-v0\n", "VideoPinball-ram-v4\n", "VideoPinball-ramDeterministic-v4\n", "VideoPinball-ramNoFrameskip-v4\n", "WizardOfWor-v0\n", "WizardOfWorDeterministic-v0\n", "WizardOfWorNoFrameskip-v0\n", "WizardOfWor-v4\n", "WizardOfWorDeterministic-v4\n", "WizardOfWorNoFrameskip-v4\n", "WizardOfWor-ram-v0\n", "WizardOfWor-ramDeterministic-v0\n", "WizardOfWor-ramNoFrameskip-v0\n", "WizardOfWor-ram-v4\n", "WizardOfWor-ramDeterministic-v4\n", "WizardOfWor-ramNoFrameskip-v4\n", "YarsRevenge-v0\n", "YarsRevengeDeterministic-v0\n", "YarsRevengeNoFrameskip-v0\n", "YarsRevenge-v4\n", "YarsRevengeDeterministic-v4\n", "YarsRevengeNoFrameskip-v4\n", "YarsRevenge-ram-v0\n", "YarsRevenge-ramDeterministic-v0\n", "YarsRevenge-ramNoFrameskip-v0\n", "YarsRevenge-ram-v4\n", "YarsRevenge-ramDeterministic-v4\n", "YarsRevenge-ramNoFrameskip-v4\n", "Zaxxon-v0\n", "ZaxxonDeterministic-v0\n", "ZaxxonNoFrameskip-v0\n", "Zaxxon-v4\n", "ZaxxonDeterministic-v4\n", "ZaxxonNoFrameskip-v4\n", "Zaxxon-ram-v0\n", "Zaxxon-ramDeterministic-v0\n", "Zaxxon-ramNoFrameskip-v0\n", "Zaxxon-ram-v4\n", "Zaxxon-ramDeterministic-v4\n", "Zaxxon-ramNoFrameskip-v4\n", "ALE/Adventure-v5\n", "ALE/Adventure-ram-v5\n", "ALE/AirRaid-v5\n", "ALE/AirRaid-ram-v5\n", "ALE/Alien-v5\n", "ALE/Alien-ram-v5\n", "ALE/Amidar-v5\n", "ALE/Amidar-ram-v5\n", "ALE/Assault-v5\n", "ALE/Assault-ram-v5\n", "ALE/Asterix-v5\n", "ALE/Asterix-ram-v5\n", "ALE/Asteroids-v5\n", "ALE/Asteroids-ram-v5\n", "ALE/Atlantis-v5\n", "ALE/Atlantis-ram-v5\n", "ALE/Atlantis2-v5\n", "ALE/Atlantis2-ram-v5\n", "ALE/Backgammon-v5\n", "ALE/Backgammon-ram-v5\n", "ALE/BankHeist-v5\n", "ALE/BankHeist-ram-v5\n", "ALE/BasicMath-v5\n", "ALE/BasicMath-ram-v5\n", "ALE/BattleZone-v5\n", "ALE/BattleZone-ram-v5\n", "ALE/BeamRider-v5\n", "ALE/BeamRider-ram-v5\n", "ALE/Berzerk-v5\n", "ALE/Berzerk-ram-v5\n", "ALE/Blackjack-v5\n", "ALE/Blackjack-ram-v5\n", "ALE/Bowling-v5\n", "ALE/Bowling-ram-v5\n", "ALE/Boxing-v5\n", "ALE/Boxing-ram-v5\n", "ALE/Breakout-v5\n", "ALE/Breakout-ram-v5\n", "ALE/Carnival-v5\n", "ALE/Carnival-ram-v5\n", "ALE/Casino-v5\n", "ALE/Casino-ram-v5\n", "ALE/Centipede-v5\n", "ALE/Centipede-ram-v5\n", "ALE/ChopperCommand-v5\n", "ALE/ChopperCommand-ram-v5\n", "ALE/CrazyClimber-v5\n", "ALE/CrazyClimber-ram-v5\n", "ALE/Crossbow-v5\n", "ALE/Crossbow-ram-v5\n", "ALE/Darkchambers-v5\n", "ALE/Darkchambers-ram-v5\n", "ALE/Defender-v5\n", "ALE/Defender-ram-v5\n", "ALE/DemonAttack-v5\n", "ALE/DemonAttack-ram-v5\n", "ALE/DonkeyKong-v5\n", "ALE/DonkeyKong-ram-v5\n", "ALE/DoubleDunk-v5\n", "ALE/DoubleDunk-ram-v5\n", "ALE/Earthworld-v5\n", "ALE/Earthworld-ram-v5\n", "ALE/ElevatorAction-v5\n", "ALE/ElevatorAction-ram-v5\n", "ALE/Enduro-v5\n", "ALE/Enduro-ram-v5\n", "ALE/Entombed-v5\n", "ALE/Entombed-ram-v5\n", "ALE/Et-v5\n", "ALE/Et-ram-v5\n", "ALE/FishingDerby-v5\n", "ALE/FishingDerby-ram-v5\n", "ALE/FlagCapture-v5\n", "ALE/FlagCapture-ram-v5\n", "ALE/Freeway-v5\n", "ALE/Freeway-ram-v5\n", "ALE/Frogger-v5\n", "ALE/Frogger-ram-v5\n", "ALE/Frostbite-v5\n", "ALE/Frostbite-ram-v5\n", "ALE/Galaxian-v5\n", "ALE/Galaxian-ram-v5\n", "ALE/Gopher-v5\n", "ALE/Gopher-ram-v5\n", "ALE/Gravitar-v5\n", "ALE/Gravitar-ram-v5\n", "ALE/Hangman-v5\n", "ALE/Hangman-ram-v5\n", "ALE/HauntedHouse-v5\n", "ALE/HauntedHouse-ram-v5\n", "ALE/Hero-v5\n", "ALE/Hero-ram-v5\n", "ALE/HumanCannonball-v5\n", "ALE/HumanCannonball-ram-v5\n", "ALE/IceHockey-v5\n", "ALE/IceHockey-ram-v5\n", "ALE/Jamesbond-v5\n", "ALE/Jamesbond-ram-v5\n", "ALE/JourneyEscape-v5\n", "ALE/JourneyEscape-ram-v5\n", "ALE/Kaboom-v5\n", "ALE/Kaboom-ram-v5\n", "ALE/Kangaroo-v5\n", "ALE/Kangaroo-ram-v5\n", "ALE/KeystoneKapers-v5\n", "ALE/KeystoneKapers-ram-v5\n", "ALE/KingKong-v5\n", "ALE/KingKong-ram-v5\n", "ALE/Klax-v5\n", "ALE/Klax-ram-v5\n", "ALE/Koolaid-v5\n", "ALE/Koolaid-ram-v5\n", "ALE/Krull-v5\n", "ALE/Krull-ram-v5\n", "ALE/KungFuMaster-v5\n", "ALE/KungFuMaster-ram-v5\n", "ALE/LaserGates-v5\n", "ALE/LaserGates-ram-v5\n", "ALE/LostLuggage-v5\n", "ALE/LostLuggage-ram-v5\n", "ALE/MarioBros-v5\n", "ALE/MarioBros-ram-v5\n", "ALE/MiniatureGolf-v5\n", "ALE/MiniatureGolf-ram-v5\n", "ALE/MontezumaRevenge-v5\n", "ALE/MontezumaRevenge-ram-v5\n", "ALE/MrDo-v5\n", "ALE/MrDo-ram-v5\n", "ALE/MsPacman-v5\n", "ALE/MsPacman-ram-v5\n", "ALE/NameThisGame-v5\n", "ALE/NameThisGame-ram-v5\n", "ALE/Othello-v5\n", "ALE/Othello-ram-v5\n", "ALE/Pacman-v5\n", "ALE/Pacman-ram-v5\n", "ALE/Phoenix-v5\n", "ALE/Phoenix-ram-v5\n", "ALE/Pitfall-v5\n", "ALE/Pitfall-ram-v5\n", "ALE/Pitfall2-v5\n", "ALE/Pitfall2-ram-v5\n", "ALE/Pong-v5\n", "ALE/Pong-ram-v5\n", "ALE/Pooyan-v5\n", "ALE/Pooyan-ram-v5\n", "ALE/PrivateEye-v5\n", "ALE/PrivateEye-ram-v5\n", "ALE/Qbert-v5\n", "ALE/Qbert-ram-v5\n", "ALE/Riverraid-v5\n", "ALE/Riverraid-ram-v5\n", "ALE/RoadRunner-v5\n", "ALE/RoadRunner-ram-v5\n", "ALE/Robotank-v5\n", "ALE/Robotank-ram-v5\n", "ALE/Seaquest-v5\n", "ALE/Seaquest-ram-v5\n", "ALE/SirLancelot-v5\n", "ALE/SirLancelot-ram-v5\n", "ALE/Skiing-v5\n", "ALE/Skiing-ram-v5\n", "ALE/Solaris-v5\n", "ALE/Solaris-ram-v5\n", "ALE/SpaceInvaders-v5\n", "ALE/SpaceInvaders-ram-v5\n", "ALE/SpaceWar-v5\n", "ALE/SpaceWar-ram-v5\n", "ALE/StarGunner-v5\n", "ALE/StarGunner-ram-v5\n", "ALE/Superman-v5\n", "ALE/Superman-ram-v5\n", "ALE/Surround-v5\n", "ALE/Surround-ram-v5\n", "ALE/Tennis-v5\n", "ALE/Tennis-ram-v5\n", "ALE/Tetris-v5\n", "ALE/Tetris-ram-v5\n", "ALE/TicTacToe3D-v5\n", "ALE/TicTacToe3D-ram-v5\n", "ALE/TimePilot-v5\n", "ALE/TimePilot-ram-v5\n", "ALE/Trondead-v5\n", "ALE/Trondead-ram-v5\n", "ALE/Turmoil-v5\n", "ALE/Turmoil-ram-v5\n", "ALE/Tutankham-v5\n", "ALE/Tutankham-ram-v5\n", "ALE/UpNDown-v5\n", "ALE/UpNDown-ram-v5\n", "ALE/Venture-v5\n", "ALE/Venture-ram-v5\n", "ALE/VideoCheckers-v5\n", "ALE/VideoCheckers-ram-v5\n", "ALE/VideoChess-v5\n", "ALE/VideoChess-ram-v5\n", "ALE/VideoCube-v5\n", "ALE/VideoCube-ram-v5\n", "ALE/VideoPinball-v5\n", "ALE/VideoPinball-ram-v5\n", "ALE/WizardOfWor-v5\n", "ALE/WizardOfWor-ram-v5\n", "ALE/WordZapper-v5\n", "ALE/WordZapper-ram-v5\n", "ALE/YarsRevenge-v5\n", "ALE/YarsRevenge-ram-v5\n", "ALE/Zaxxon-v5\n", "ALE/Zaxxon-ram-v5\n" ] } ], "source": [ "for env in gym.envs.registry.items():\n", " print(env[0])" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The `render_mode` argument defines how you will see the environment:\n", "\n", "* `None` (default): allows to train a DRL algorithm without wasting computational resources rendering it.\n", "* `rgb_array_list`: allows to get numpy arrays corresponding to each frame. Will be useful when generating videos.\n", "* `ansi`: string representation of each state. Only available for the \"Toy text\" environments.\n", "* `human`: graphical window displaying the environment live." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The main interest of gym(nasium) is that all problems have a common interface defined by the class `gym.Env`. There are only three methods that have to be used when interacting with an environment:\n", "\n", "* `state, info = env.reset()` restarts the environment and returns an initial state $s_0$.\n", "\n", "* `state, reward, terminal, truncated, info = env.step(action)` takes an action $a_t$ and returns:\n", " * the new state $s_{t+1}$, \n", " * the reward $r_{t+1}$, \n", " * two boolean flags indicating whether the current state is terminal (won/lost) or truncated (timeout),\n", " * a dictionary containing additional info for debugging (you can ignore it most of the time).\n", "\n", "* `env.render()` displays the current state of the MDP. When the render mode is set to `rgb_array_list` or `human`, it does not even have to called explicitly (since gym 0.25)." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "With this interface, we can interact with the environment in a standardized way:\n", "\n", "* We first create the environment.\n", "* For a fixed number of episodes:\n", " * We pick an initial state with `reset()`.\n", " * Until the episode is terminated:\n", " * We select an action using our RL algorithm or randomly.\n", " * We take that action (`step()`), observe the new state and the reward.\n", " * We go into the new state.\n", "\n", "The following cell shows how to interact with the CartPole environment using a random policy. Note that it will only work on your computer, not in colab." ] }, { "cell_type": "code", "execution_count": 4, "metadata": {}, "outputs": [], "source": [ "env = gym.make('CartPole-v1', render_mode=\"human\")\n", "\n", "for episode in range(10):\n", " state, info = env.reset()\n", " done = False\n", " while not done:\n", " # Select an action randomly\n", " action = env.action_space.sample()\n", " \n", " # Sample a single transition\n", " next_state, reward, terminal, truncated, info = env.step(action)\n", " \n", " # Go in the next state\n", " state = next_state\n", "\n", " # End of the episode\n", " done = terminal or truncated\n", "\n", "env.close()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "On colab (or whenever you want to record videos of the episodes instead of watching them live), you need to create the environment with the rendering mode `rgb_array_list`. \n", "\n", "You then create a `GymRecorder` object (defined in the first cell of this notebook). \n", "\n", "```python\n", "recorder = GymRecorder(env)\n", "```\n", "\n", "At the end of each episode, you tell the recorder to record all frames generated during the episode. The frames returned by `env.render()` are (width, height, 3) numpy arrays which are accumulated by the environment during the episode and flushed when `env.reset()` is called.\n", "\n", "```python\n", "recorder.record(env.render())\n", "```\n", "\n", "You can then generate a gif at the end of the simulation with:\n", "\n", "```python\n", "recorder.make_video('videos/CartPole-v1.gif')\n", "```\n", "\n", "Finally, you can render the gif in the notebook by calling **at the very last line of the cell**:\n", "\n", "```python\n", "ipython_display('videos/CartPole-v1.gif')\n", "```" ] }, { "cell_type": "code", "execution_count": 5, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "MoviePy - Building file videos/CartPole-v1.gif with imageio.\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ " \r" ] }, { "data": { "text/html": [ "
" ], "text/plain": [ "" ] }, "execution_count": 5, "metadata": {}, "output_type": "execute_result" } ], "source": [ "env = gym.make('CartPole-v1', render_mode=\"rgb_array_list\")\n", "recorder = GymRecorder(env)\n", "\n", "for episode in range(10):\n", " state, info = env.reset()\n", "\n", " done = False\n", " while not done:\n", " # Select an action randomly\n", " action = env.action_space.sample()\n", " \n", " # Sample a single transition\n", " next_state, reward, terminal, truncated, info = env.step(action)\n", " \n", " # Go in the next state\n", " state = next_state\n", "\n", " # End of the episode\n", " done = terminal or truncated\n", "\n", " # Record at the end of the episode \n", " recorder.record(env.render())\n", "\n", "recorder.make_video('videos/CartPole-v1.gif')\n", "ipython_display('videos/CartPole-v1.gif', autoplay=1, loop=0)" ] }, { "cell_type": "markdown", "metadata": { "id": "WENOr5atoGr1" }, "source": [ "Each environment defines its state space (`env.observation_space`) and action space (`env.action_space`). \n", "\n", "State and action spaces can either be :\n", "\n", "* discrete (`gym.spaces.Discrete(nb_states)`), with states being an integer between 0 and `nb_states` -1.\n", "\n", "* feature-based (`gym.spaces.Box(low=0, high=255, shape=(SCREEN_HEIGHT, SCREEN_WIDTH, 3))`) for pixel frames.\n", "\n", "* continuous. Example for two joints of a robotic arm limited between -180 and 180 degrees:\n", "\n", "```python\n", "gym.spaces.Box(-180.0, 180.0, (2, ))\n", "```\n", "\n", "You can sample a state or action randomly from these spaces:" ] }, { "cell_type": "code", "execution_count": 6, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "[107.88474 76.81408]\n" ] } ], "source": [ "action_space = gym.spaces.Box(-180.0, 180.0, (2, ))\n", "action = action_space.sample()\n", "print(action)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "\n", "\n", "Sampling the action space is particularly useful for exploration. We use it here to perform random (but valid) actions:\n", "\n", "```python\n", "action = env.action_space.sample()\n", "```\n", "\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "**Q:** Create a method `random_interaction(env, number_episodes, recorder=None)` that takes as arguments:\n", "\n", "* The environment.\n", "* The number of episodes to be performed.\n", "* An optional `GymRecorder` object that may record the frames of the environment if it is not None (`if renderer is not None:`). Otherwise, do not nothing.\n", "\n", "The method should return the list of undiscounted returns ($\\gamma=1$, i.e. just the sum of rewards obtained during each episode) for all episodes." ] }, { "cell_type": "code", "execution_count": 7, "metadata": {}, "outputs": [], "source": [ "def random_interaction(env, number_episodes, recorder=None):\n", "\n", " returns = []\n", "\n", " # Sample episodes\n", " for episode in range(number_episodes):\n", "\n", " # Sample the initial state\n", " state, info = env.reset()\n", "\n", " return_episode = 0.0\n", " done = False\n", " while not done:\n", "\n", " # Select an action randomly\n", " action = env.action_space.sample()\n", " \n", " # Sample a single transition\n", " next_state, reward, terminal, truncated, info = env.step(action)\n", "\n", " # Update the return\n", " return_episode += reward\n", " \n", " # Go in the next state\n", " state = next_state\n", "\n", " # End of the episode\n", " done = terminal or truncated\n", "\n", " # Record at the end of the episode\n", " if recorder is not None:\n", " recorder.record(env.render())\n", "\n", " returns.append(return_episode)\n", "\n", " return returns\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "**Q:** Use that method to visualize all the available simple environments for a few episodes:\n", "\n", "* CartPole-v1\n", "* MountainCar-v0\n", "* Pendulum-v1\n", "* Acrobot-v1\n", "* LunarLander-v2\n", "* BipedalWalker-v3\n", "* CarRacing-v2\n", "* Blackjack-v1\n", "* FrozenLake-v1\n", "* CliffWalking-v0\n", "* Taxi-v3\n", "\n", "If you do many episodes (CarRacing or Taxi have very long episodes with a random policy), plot the obtained returns to see how they vary. \n", "\n", "If you managed to install the mujoco and atari dependencies, feel free to visualize them too. " ] }, { "cell_type": "code", "execution_count": 8, "metadata": {}, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAA0kAAAISCAYAAAAZR4TAAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjguMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8g+/7EAAAACXBIWXMAAA9hAAAPYQGoP6dpAAB+lklEQVR4nO3deXzcZbn///dnZpLJvu/N0r1pmy40qaWAFdrKoiJIxe9RPIIofvEUf1BAoXpAELWIsrhgXb6IxwMctEo9ggLSFspaaVPapvveJM3apNn3mfn9kflME2hLm2bmM8vr+XjMg2ZmMnMVwmTec9/XdRsej8cjAAAAAIAkyWZ1AQAAAAAQTAhJAAAAADAEIQkAAAAAhiAkAQAAAMAQhCQAAAAAGIKQBAAAAABDEJIAAAAAYAhCEgAAAAAMQUgCAAAAgCEISQAAAAAwhKUh6b777pNhGMMuxcXFvtsvvvjiD9x+8803W1gxAAAAgHDnsLqA6dOna82aNb6vHY7hJd1000363ve+5/s6Li4uYLUBAAAAiDyWhySHw6GcnJxT3h4XF3fa2wEAAABgNFkekvbt26e8vDzFxMRo/vz5WrFihQoLC323P/3003rqqaeUk5OjK6+8Uvfcc89pV5N6e3vV29vr+9rtdqu5uVnp6ekyDMOvfxcAAAAAwcvj8ai9vV15eXmy2U7deWR4PB5PAOsa5sUXX1RHR4emTJmi2tpa3X///Tp69Ki2b9+uxMRE/eY3v1FRUZHy8vK0bds23XXXXfrIRz6i55577pSPed999+n+++8P4N8CAAAAQCipqqpSfn7+KW+3NCS9X0tLi4qKivTII4/oK1/5ygduX7dunRYtWqT9+/drwoQJJ32M968ktba2qrCwUFVVVUpKSvJb7QAAAACCW1tbmwoKCtTS0qLk5ORT3s/y7XZDpaSkaPLkydq/f/9Jb583b54knTYkOZ1OOZ3OD1yflJRESAIAAADwoW04QXVOUkdHhw4cOKDc3NyT3r5lyxZJOuXtAAAAAHCuLF1JuvPOO3XllVeqqKhINTU1+u53vyu73a7Pf/7zOnDggJ555hl94hOfUHp6urZt26Zly5ZpwYIFmjlzppVlAwAAAAhjloak6upqff7zn1dTU5MyMzN10UUXacOGDcrMzFRPT4/WrFmjxx57TJ2dnSooKNCSJUv0n//5n1aWDAAAACDMBdXgBn9oa2tTcnKyWltb6UkCAAAAItiZZoOg6kkCAAAAAKsRkgAAAABgCEISAAAAAAxBSAIAAACAIQhJAAAAADAEIQkAAAAAhiAkAQAAAMAQhCQAAAAAGIKQBAAAAABDEJIAAAAAYAhCEgAAAAAMQUgCAAAAgCEISQh6Pf0uba1qkcfjsboUAAAARABCEoLeo2v26qrH39Kq8mqrSwEAAEAEICQh6L2x95gk6W9baiyuBAAAAJGAkISg1tPv0t76dknSvw41qb2n3+KKAAAAEO4ISQhqu2rbNOAe7EXqd3n05r5jFlcEAACAcEdIQlDbfrR12NdrdjVYVAkAAAAiBSEJQW1b9WBImjs2VZL02p4GudxMuQMAAID/EJIQ1Cq8K0lfvnCcEmMcaurs09bqFmuLAgAAQFgjJCFodfe5tK+hQ5J0XmGKPjY5U5K0dle9lWUBAAAgzBGSELR21rbJ5fYoI8GpnKQYLZ6aLUlaS18SAAAA/IiQhKBlDm2YMSZJhmHoY5MzZTOk3XXtOtrSbXF1AAAACFeEJAQtc2jDjPwUSVJqfLRKiwYHOKxjyx0AAAD8hJCEoHViJSnZd90ic8vdbrbcAQAAwD8ISQhKXX0D2tfQLkmamT8kJBVnSZLePtCkrr4BS2oDAABAeCMkISjtqm2T2yNlJTqVnRTju35iVoIK0mLVN+DWm/uOWVghAAAAwhUhCUHJ1480ZKudJBmGoUXFg1vu1rHlDgAAAH5ASEJQqvANbUj+wG2Lpg5uuVu7u0FutyegdQEAACD8EZIQlCpOMrTBNG9cuuKj7Wps79X2mtZAlwYAAIAwR0hC0OnsHdD+xg5JJw9J0Q6bFkzOlMTBsgAAABh9hCQEnZ21bfJ4pOwkp7KGDG0YamGxueWO85IAAAAwughJCDonhjaknPI+lxRnyTCk7UfbVNfaE6DKAAAAEAkISQg6FdUtkoafj/R+GQlOzS5IkSS9uoctdwAAABg9hCQEndMNbRjKPFh27S623AEAAGD0EJIQVDp6B3TwWKckqeTDQtLUwfOS3tx/TD39Lr/XBgAAgMhASEJQ2XG0VR6PlJsco8xE52nvW5yTqLzkGPX0u/XOgaYAVQgAAIBwR0hCUDnTrXaSZBiGFnoPll3DljsAAACMEkISgooZkk43tGEoc8vdut0N8ng8fqsLAAAAkYOQhKBS4R3//WH9SKb549MVG2VXbWuPdtW2+7M0AAAARAhCEoJGW0+/b2jDmWy3k6SYKLsunJghiSl3AAAAGB2WhqT77rtPhmEMuxQXF/tu7+np0dKlS5Wenq6EhAQtWbJE9fW8EQ5XO462SZLGpMQqPeH0QxuGWuztS1q7m/OSAAAAcO4sX0maPn26amtrfZc333zTd9uyZcv0/PPPa9WqVVq/fr1qamp0zTXXWFgt/KniaIukM19FMl3iPS9pa3WLGtt7R7ssAAAARBiH5QU4HMrJyfnA9a2trXriiSf0zDPPaOHChZKkJ598UlOnTtWGDRt0/vnnB7pU+FmFdyVpxhkObTBlJ8VoxphkVRxt1at7GvS5sgJ/lAcAAIAIYflK0r59+5SXl6fx48fruuuuU2VlpSSpvLxc/f39Wrx4se++xcXFKiws1DvvvHPKx+vt7VVbW9uwC0JDRXWLpLNfSZKkRd4td+t2seUOQGg52Nih63/3rrZWtVhdCgDAy9KQNG/ePP3+97/XSy+9pJUrV+rQoUP66Ec/qvb2dtXV1Sk6OlopKSnDvic7O1t1dXWnfMwVK1YoOTnZdykoYFUhFLR29+twU5ekEYak4sFR4G/sa1TvgGtUawMAf3rizUNav7dRK187YHUpAAAvS7fbXXHFFb4/z5w5U/PmzVNRUZH+9Kc/KTY2dkSPuXz5ct1+++2+r9va2ghKIWCH93yk/NRYpcZHn/X3T89LUlaiUw3tvfrXwWYtmJw52iUCgF9sOnx88J9HmuXxeGQYhsUVAQAs3243VEpKiiZPnqz9+/crJydHfX19amlpGXaf+vr6k/YwmZxOp5KSkoZdEPy2neUhsu9nsxknttwx5Q5AiGjt6tee+sEz3o519PlW1AEA1gqqkNTR0aEDBw4oNzdXpaWlioqK0tq1a32379mzR5WVlZo/f76FVcIfKo6e3SGyJ7PQu+Vuza56eTyeUakLAPxpc+XxYV9vOtxsUSUAgKEsDUl33nmn1q9fr8OHD+vtt9/WZz7zGdntdn3+859XcnKyvvKVr+j222/Xq6++qvLycn35y1/W/PnzmWwXhiqqvStJY1JG/BgXTkxXtMOm6uPd2tfQMUqVAYD/bPSGInOHnbn1DgBgLUtDUnV1tT7/+c9rypQp+tznPqf09HRt2LBBmZmD/SSPPvqoPvWpT2nJkiVasGCBcnJy9Nxzz1lZMvygtatflc0jH9pgiot26MIJ6ZKktUy5AxACzFB06bTBlfCNR1hJAoBgYOnghmefffa0t8fExOjxxx/X448/HqCKYAVzq11hWpyS46LO6bEWTs3Wq3satXZXvb5+8YTRKA8A/KJ3wKWt3qMPvrZggl7eUa+DjZ1q6uhVeoLT2uIAIMIFVU8SItO2oy2Szv4Q2ZNZWDw4vGFz5XE1d/ad8+MBgL9sP9qm3gG30uOjNacwRZOyEiRJ5UfYcgcAViMkwXLbvStJ57LVzjQmJVZTc5Pk9kjr97LlDkDwMoc0lBalyjAMlY1NHbyekAQAliMkwXLbfEMbzj0kSdIi72rSGvqSAAQxMwzNHZsmSSorGvwnE+4AwHqEJFjqeGefqo93S5Kmj1JIWug9L+n1PY3qd7lH5TEBYDR5PB5fGDJXkMywVHG0VT39LstqAwAQkmAxc2jD2PQ4Jcee29AG0+z8FKXHR6u9d8A3XhcAgsmBxk4d7+qX02HT9LzBD4gK0mKVmehUv8ujrVUt1hYIABGOkARLmSFpRn7KqD2mzWboEu+WO0aBAwhG5d5R37MLUhTtGPxVbBiG5tKXBABBgZAES5mHyM4YkzSqj7vYu+Vu3W5CEoDgs9F7PpK51c5USl8SAAQFQhIs5VtJGpMyqo970aRMRdkNHTrWqYONHaP62ABwrswx32XePiSTuZJUfuS43G5PwOsCAAwiJMEyTR29OtpiDm0Y3ZWkBKdD549Pl8SWOwDBpbG9V4eOdcowpDmFw1eSpuUmKS7arraeAe1r4AMeALAKIQmWMVeRxmfEKylmdIY2DGWOAl+7u37UHxsARsrsR5qSnfiBgTUOu02zC1IkicEzAGAhQhIs4ztENn90Rn+/38LibEmDe/9bu/v98hwAcLY2naIfyWRuwStneAMAWIaQBMts8w1t8E9IKkyP06SsBLncHq3f2+iX5wCAs7XxfYfIvp/Zl8RKEgBYh5AEy5wY2uCfkCRJi6YOriat28WWOwDW6+5zaYf3ta+06OQrSecVpspmSNXHu1XX2hPI8gAAXoQkWKKxvVe1rT0yDGm6X0PSYF/Sa3sbNeBy++15AOBMbKlq0YDbo9zkGI1JiT3pfRKcDk3NHRxms+kIq0kAYAVCEiyxfcjQhgSnw2/Pc15BilLiotTS1a/NlS1+ex4AOBPm+UdlY9NkGMYp72duxTP7lwAAgUVIgiXMrXYz81P8+jwOu02XTGHKHYDgYPYjlZ1iq53J3IpHXxIAWIOQBEuYQxtK/LjVzrTQOwp8HeclAbCQy+3Re0dOP9nOZN6+q7ZNHb0Dfq8NADAcIQmWqDjaIkma6afx30MtmJwph83QvoYOVTZ1+f35AOBk9tS1q713QAlOh4pzTn+Adm5yrPJTY+X2SO9VsuUOAAKNkISAa2jrUX1brwxj8HR5f0uOjfLt72fLHQCrmEMYzitMkd126n4kU5lvyx0hCQACjZCEgDP7kSZmJijej0MbhjKn3K1lyx0Ai5hDGE51PtL7nThUlr4kAAg0QhICLhDnI72f2Zf0r0NNau/pD9jzAoDJN9nuQ4Y2mMww9V5lC0cYAECAEZIQcBXeoQ0zAtCPZBqfmaDxGfHqd3n05r5jAXteAJCkoy3dqmntkd1maHZhyhl9z6SsBCXFONTV59Ku2nb/FggAGIaQhIA7Mf47cCFJOrGatIYtdwACzFxFKslLUlz0mW0zttkMRoEDgEUISQio+rYeNbT3ymZI03IDHJK8fUmv7WmQy+0J6HMDiGxmP1Jp0Zn1I5nMvqRN9CUBQEARkhBQ5vlIk7ISFRttD+hzzx2bpsQYh5o6+7S1uiWgzw0gsm06Yg5tOLN+JJPZv7Tp8HF5PHy4AwCBQkhCQJlb7QJxiOz7Rdlt+tjkTEnS2l2MAgcQGG09/dpd1yZJKj3LkDSrIEVRdkMN7b2qau72R3kAgJMgJCGgKrwrOIHuRzIxChxAoG0+clwej1SUHqesxJiz+t6YKLvvQyX6kgAgcAhJCBiPx6OKo4OfpgZyst1QF0/Oks2Qdte162gLn8oC8L9y71a7srPsRzLN9fUlcagsAAQKIQkBU9fWo2MdvbLbDE3LTbKkhtT4aN+0qHVsuQMQAOYK0Nn2I5lO9CWxkgQAgUJIQsCcGNqQoJiowA5tGGphcbYkae1uttwB8K9+l1tbqlokSWUjDEnmBzv7GjrU0tU3WqUBAE6DkISA2e4d2jDDgqENQy329iW9faBJXX0DltYCILztqGlTT79bqXFRmpCZMKLHSE9wanxmvKQTW/cAAP5FSELAmCtJVg1tME3MSlBBWqz6Btx6c98xS2sBEN7MLXKlRWkyDGPEjzPX28+08TAhCQACgZCEgPB4PL6VJCvGfw9lGIYWebfcrWPLHQA/MvuRRrrVzmSODqcvCQACg5CEgKhp7VFTZ58cNkNTLRraMJQ5Cnzd7ga53RzQCGD0eTwe3/a4kQ5tMJkT7rZVt6qn33XOtQEATo+QhIAwz0eanJ1o6dAG00fGpSk+2q6G9l5tr2m1uhwAYehwU5eOdfQp2mE75xX0selxykiIVp/L7VuVBwD4DyEJAVERJEMbTE6HXQsmZ0riYFkA/mFujZuVnyyn49w+HDIMwzfljr4kAPA/QhICwhzaYNUhsiezsPjEljsAGG2bvGGmbOzIDpF9P3PLXfkR+pIAwN8ISfC7oUMbgmUlSZIuKc6SYQyuctW39VhdDoAws9EbZszDYM+VuZK06chxeikBwM8ISfC76uPdOt7Vryi7oeLcRKvL8clIcGp2QYokVpMAjK6mjl4dbOyUdCLcnKvpecmKibKppatfB491jMpjAgBOjpAEvzNXkabkJJ7zvvzRtsi75Y6+JACjyZxqNzk7QSlx0aPymNEOm++DHfqSAMC/CEnwu21BuNXOtNB7XtKb+xsZqwtg1GzyhqTSotHpRzKV+Q6VpS8JAPyJkAS/qzCHNoxJsbaQk5iam6i85Bj19Lv1zoEmq8sBECbMyXbnej7S+5mH0porVQAA/wiakPTggw/KMAzddtttvusuvvhiGYYx7HLzzTdbVyTOmsfjCbrx30MZhqGF3oNl1+yqt7gaAOGgp9/le92bO0qT7UxzilJlGNKRpi41MHAGAPwmKELSxo0b9etf/1ozZ878wG033XSTamtrfZeHHnrIggoxUlXN3Wrt7le03abJOQlWl3NSi7xb7tbtbpDHw8QoAOdma1WL+l0eZSU6lZ8aO6qPnRQTpSnZgwNwNrGaBAB+Y3lI6ujo0HXXXaff/va3Sk394LaEuLg45eTk+C5JSUkWVImRqgjioQ2m+RPSFRtlV21rj3bVtltdDoAQZ4aXuWPTZBjGqD++uTq1ieENAOA3loekpUuX6pOf/KQWL1580tuffvppZWRkqKSkRMuXL1dXV9dpH6+3t1dtbW3DLrDOtqMtkoLrENn3i4my68KJGZKktWy5A3COzH6kslHuRzKZj7uJQ2UBwG8cVj75s88+q82bN2vjxo0nvf0LX/iCioqKlJeXp23btumuu+7Snj179Nxzz53yMVesWKH777/fXyXjLJlDG2YGYT/SUIumZmnNrnqt3d2gbyyaZHU5AEKU2+3xDVUoG+XJdqYy70rSjpo2dfYOKN5p6a9yAAhLlr2yVlVV6dZbb9Urr7yimJiYk97na1/7mu/PM2bMUG5urhYtWqQDBw5owoQJJ/2e5cuX6/bbb/d93dbWpoKCgtEtHmdk6NCGkiAPSQu95yVtrW5RY3uvMhOdFlcEIBTta+hQW8+A4qLtmuqnw7PHpMQqLzlGNa092lrVogu8K+EAgNFj2Xa78vJyNTQ0aM6cOXI4HHI4HFq/fr1+9rOfyeFwyOX64Jk18+bNkyTt37//lI/rdDqVlJQ07AJrHGnqUnvPgKIdNk3O9s+bhdGSnRSjGWOS5fFIr+7hYFkAI2OeX3ReYYocdv/9ijVXkzhUFgD8w7KQtGjRIlVUVGjLli2+S1lZma677jpt2bJFdvsHm/y3bNkiScrNzQ1wtRgJcxVpak6ioh2Wt799KHM1ad0uQhKAkfH3VjsTfUkA4F+WbbdLTExUSUnJsOvi4+OVnp6ukpISHThwQM8884w+8YlPKD09Xdu2bdOyZcu0YMGCk44KR/DxnY8UxEMbhlo8NVs/XbtPb+xrVO+AK2in8QEIXht9h8j6OSR5Q9jmI8c14HL7ddUKACJR0L6qRkdHa82aNbr00ktVXFysO+64Q0uWLNHzzz9vdWk4QyeGNqRYW8gZmp6XpKxEpzr7XPrXQT6dBXB26lp7VH28WzZDml2Y4tfnmpKTqESnQ519Lu2u4+gCABhtQTUS57XXXvP9uaCgQOvXr7euGJwTt9uj7SEytMFksxlaWJylZzdWad3uBi2YnGl1SQBCiLn1bVpekhL8PHHObjN0XlGqXt/bqPIjx0PmdRYAQkXQriQhtB1u6lR774CcDpsmZSdYXc4ZWzQ1W5K0dne9PB6PxdUACCXm4a7+7kcyzS0a7Esyt/gBAEYPIQl+4RvakJukqBDaK3/hxHRFO2yqau7WvoYOq8sBEEI2+vkQ2fcrNYc3HD7OhzoAMMpC590rQoqvHylEhjaY4qIdumBCuiRpLVPuAJyhjt4B7aptkxS4laTZBSly2AzVtfXoaEt3QJ4TACIFIQl+ESqHyJ6MueVu3e56iysBECreqzwut0cqSItVTvLJD0gfbXHRDk33vsZu4rwkABhVhCSMuqFDG0JtJUk6cV5S+ZHjOt7ZZ3E1AELBxgD3I5nK6EsCAL8gJGHUHTzWqc4+l2KibJqYGTpDG0xjUmJVnJMot0d6bS9b7gB8uPIjge1HMs31Pp95iC0AYHQQkjDqzFWkablJIXvA4WJzyh19SQA+RL/LrfcqWyQFfiWp1Pt8e+rb1drVH9DnBoBwFprvYBHUtvmGNqRYW8g5WDh1cMvd+r2N6ne5La4GQDDbVdumrj6XkmIcmpQV2NXzzESnxqbHyeORNleymgQAo4WQhFEXaofInszs/BSlx0ervWeAvf4ATst3PtLYNNlsRsCfv2zs4GqSeZgtAODcEZIwqlxuj7bXhO7QBpPNZugS7wCHdWy5A3AaZjgpLQpsP5LJ7EvayIQ7ABg1hCSMqkPHOtTV51JslF0TQnBow1CLvCFp7W5CEoCT83g8vpWkuWMD249kMvuStla1qG+A7cEAMBoISRhVZj/S9Lwk2S3YdjKaPjo5U1F2Q4eOdepgY4fV5QAIQlXN3Wpo71W03WbZ6vmEzHilxkWpd8DtW8kHAJwbQhJGlRmSZoTwVjtTgtOh88enS2LKHYCTM3sWS8YkKSbKbkkNhmGc6EuihxIARgUhCaPKHNowI4SHNgy10Lflrt7iSgAEo01HrN1qZzpxqCx9SQAwGghJGDUut0c7atokhfbQhqEWFQ+el7Tx8HG1dnMGCYDhzJWbMqtDkvf5y48cl8fjsbQWAAgHhCSMmgONHerudyku2q5xGaE9tMFUmB6nSVkJcrk9Wr+30epyAASRlq4+7WsY7Fe0arKdqWRMkqIdNjV39ungsU5LawGAcEBIwqgx+5FK8pJDfmjDUObBsut2seUOwAnl3q12EzLjlRYfbWktTodds70HeJez5Q4AzhkhCaOmorpFUngMbRhq8dTBLXev7W3UgIvxugAGbbR49Pf7lfnOS2J4AwCcK0ISRk1FmA1tMJ1XkKKUuCi1dPVrc2WL1eUACBJmP5LVW+1MZkgyh0kAAEaOkIRRMeBya2ft4NCGcFtJcthtunhypiSm3AEY1NPv8m0xDpaVpNLCwToOHevUsY5ei6sBgNBGSMKo2N/YoZ5+txKcDo1Lj7e6nFG3yLvlbh3nJQHQ4HEHfS63MhKiVZQeZ3U5kqTkuChNyU6UJG2iLwkAzgkhCaPC/ER1el6SbGE0tMG0YHKm7DZD+xo6VNnUZXU5ACxm9iOVFaXJMILnNa/U3HJHXxIAnBNCEkaFeYhsuJyP9H7JsVGa633zwZY7AOVHzPORgqMfyTSXviQAGBWEJIwK3/jvMBvaMJQ55W7dbrbcAZHM7fb4QojVh8i+X1nRYD3bj7aqu89lcTUAELoISThn/UOGNsz0ntMRjhYWD56XtOFgk9p7+i2uBoBVDh7rUEtXv2KibJqel2R1OcPkp8YqO8mpAbdHW6parC4HAEIWIQnnbF99h/oG3Ep0OlSUFhwNzP4wPjNB4zLi1e/y6M19x6wuB4BFzH6k8wpSFWUPrl+jhmH4VrfMLYEAgLMXXK/uCEkVR1skDW61C8ehDUMt8q4mrWXLHRCxzMNag60fyTS3yDxUlr4kABgpQhLOWUWYD20YauHUwZD06u4Gudwei6sBYIXyIO1HMpl1bT5ynNcpABghQhLOWUUEDG0wzR2bpsQYh5o6+7S1usXqcgAEWENbj440dclmSHMKU6wu56SKcxIVH21Xe++A9ta3W10OAIQkQhLOSd+AW7tqB38JR8JKUpTdpo9NzpTEwbJAJDKn2k3JSVJiTJTF1Zycw27TnCLOSwKAc0FIwjnZW9+uPpdbSTEOFYbx0IahFnm33K3ZxXlJQKTZ5O3zmRuk/Uim0iLOSwKAc0FIwjkx+5Fm5CcH1anz/nTx5CzZDGl3XbuOtnRbXQ6AANrkO0Q2OPuRTHO99W1ieAMAjAghCefEDEmR0I9kSo2P9n1Ky8GyQOTo7B3QjprBM+HKioJ7JWl2QYrsNkNHW7pVw4c5AHDWCEk4J+bQhpljUqwtJMAWFmdLktay5Q6IGFurWuRyezQmJVZ5KbFWl3Na8U6HpuUOHnTLljsAOHuEJIxY74BLu+sGP1WNhKENQy329iW9faBJXX0DFlcDIBDMc4dKg3wVyWSe48TwBgA4e4QkjNjeug71uzxKjo1Sfmpwf6o62iZmJaggLVZ9A269ue+Y1eUACACzHynYhzaYyooG+5I4VBYAzh4hCSO27WiLpMFVpEgZ2mAyDEOLvFvu6EsCwt+Ay63NQX6I7PuZK0l76trU1tNvcTUAEFoISRix7RE4tGEocxT4ut0NcnOqPRDWdte1q7PPpUSnQ5OzE60u54xkJ8WoMC1Obo/0XmWL1eUAQEghJGHEtvmGNkRmSPrIuDTFR9vV0N6r7TWtVpcDwI/KvatIc4pSZbeFzsp5GYfKAsCIEJIwIj39Lu2tb5c0eEZSJHI67PropExJ0tpdbLkDwtnGw6HVj2Qq47wkABgRQhJGZE9du/pdHqXGRWlMkI/C9aehW+4AhCePx+MLGaVFodGPZDJD3XtVx9XvcltcDQCEjqAJSQ8++KAMw9Btt93mu66np0dLly5Venq6EhIStGTJEtXXcy5NMNjm7UeakZ8ScUMbhrp4SpYMY/BQ3fq2HqvLAeAHR1u6VdfWI4fN0OyCFKvLOSsTMhOUHBulnn637yBcAMCHC4qQtHHjRv3617/WzJkzh12/bNkyPf/881q1apXWr1+vmpoaXXPNNRZViaG2e/uRZoxJsrgSa2UmOjUrP0USq0lAuDJXkUrGJCs22m5xNWfHZjPoSwKAEbA8JHV0dOi6667Tb3/7W6Wmntjr3draqieeeEKPPPKIFi5cqNLSUj355JN6++23tWHDBgsrhjRkJWlMirWFBAHzYFn6koDwZPYjlYXIIbLvR18SAJw9y0PS0qVL9clPflKLFy8edn15ebn6+/uHXV9cXKzCwkK98847p3y83t5etbW1DbtgdPX0u7Qvwoc2DLXQe17Sm/sb1dPvsrgaAKOtPMTOR3o/87ykTUea5fFwXAEAnAlLQ9Kzzz6rzZs3a8WKFR+4ra6uTtHR0UpJSRl2fXZ2turq6k75mCtWrFBycrLvUlBQMNplR7xdtW0acHuUHh+tvOQYq8ux3NTcROUlx6in3613DjRZXQ6AUdTa1a893g+FykJssp1pxphkRdttOtbRpyNNXVaXAwAhwbKQVFVVpVtvvVVPP/20YmJG74328uXL1dra6rtUVVWN2mNj0Hbf0IbkiB7aYDIMQwvNLXe7GSwChJPNlcfl8UjjMuKVkeC0upwRiYmy+1b9N9KXBABnxLKQVF5eroaGBs2ZM0cOh0MOh0Pr16/Xz372MzkcDmVnZ6uvr08tLS3Dvq++vl45OTmnfFyn06mkpKRhF4yubb6hDWy1My3ybrlbt6uB7SxAGNl0JLT7kUzmKpi5dRAAcHqWhaRFixapoqJCW7Zs8V3Kysp03XXX+f4cFRWltWvX+r5nz549qqys1Pz5860qGxocdy0RkoaaPyFdMVE21bT2aFdtu9XlABglGw+b/UihHZLmes93YiUJAM6Mw6onTkxMVElJybDr4uPjlZ6e7rv+K1/5im6//XalpaUpKSlJ3/jGNzR//nydf/75VpQMSd19Lu1r6JDE0IahYqLsumhiptbsqte63fWalscKJhDq+gbc2lrVIil0hzaYSr0rYQcaO9Xc2ae0+GiLKwKA4Gb5dLvTefTRR/WpT31KS5Ys0YIFC5STk6PnnnvO6rIi2s7aNrncHmUkOJWTxNCGoRZ5+5LWMAocCAvba1rVO+BWWny0xmfEW13OOUmNj9bErARJbLkDgDNh2UrSybz22mvDvo6JidHjjz+uxx9/3JqC8AHm0IaZDG34gIXFgyFpa3WLGtt7lZkYmk3eAAaZh6+WFqWGxevd3LGp2t/QoU2Hm/XxadlWlwMAQS2oV5IQfMyhDSX0I31AdlKMZoxJlscjvbaH1SQg1JmHr84N8X4kUyl9SQBwxghJOCsVR1skSTMJSSdlriatZcsdENI8Ho82hfghsu9nhr2Ko60cfA0AH4KQhDPW1Teg/QxtOC2zL+mNfY3qHeBNCBCqDh4bHHDgdNhUkhcer3eFaXHKTHSq3+Xx7QoAAJwcIQlnbGdNm9weKSvRqWyGNpxUSV6yshKd6uxz6d1DbGkBQlW5d6vdrIIURTvC41elYRi+857YcgcApxcer/wICM5H+nA2m8GWOyAMmCEiXPqRTObWQSbcAcDpEZJwxiq82zPYand6vpC0u14ej8fiagCMhK8fqSg8+pFMZujbdLhZbjevTwBwKoQknLGKIeO/cWoXTcpQtMOmquZu38G7AELHsY5eHTrWKcOQ5hSG10rS1NwkxUbZ1dYzwOsTAJwGIQlnpLN3QPsbB3+hMv779OKiHbpgQrokttwBocgc/T0lO1HJcVEWVzO6ouw2nVeYIknadIS+JAA4FUISzsiOmjZ5PFJOUoyyEhna8GEWebfcrdtdb3ElAM7W0ENkw5HZl2SGQQDABxGScEbMrXasIp2ZhVMHT7MvP3Jcxzv7LK4GwNkw+5Hmhsn5SO9nTrhjJQkATo2QhDNSUd0iiX6kMzUmJVbFOYlye6TX9rLlDggV3X0ubfd+KBSuK0nnFabIZkhVzd2qa+2xuhwACEqEJJwR3/hvQtIZW+xdTaIvCQgdW6paNOD2KCcpRvmpsVaX4xeJMVEqzkmSxGoSAJwKIQkfqr2nXwePdUrijKSzsXDqYF/S+r2N6ne5La4GwJko94aGsrGpMgzD4mr858QocPqSAOBkCEn4UObQhrzkGGUkOK0uJ2TMyk9Reny02nsGON0eCBEbD5vnI4XnVjuTb3gDK0kAcFKEJHyo7QxtGBG7zdAl5pQ7ttwBQc/l9mhzpTckhenQBlOZdyVpZ02bOnoHLK4GAIIPIQkfals1h8iOlDkKfO1uQhIQ7PbWt6u9Z0AJToeKcxKtLsevcpNjNSYlVm6PtKWyxepyACDoEJLwoVhJGrmLJmUoym7o0LFOHWzkdHsgmJnnI51XmCKHPfx/PZp9SWwHBoAPCv/fAjgnbQxtOCeJMVE6f3y6JGkdq0lAUDPPRyorCu+tdqZS+pIA4JQISTitHUfbJA2e+5PO0IYRWejdcrdmV73FlQA4HXPSm7nCEu7Mv+d7lS0aYAInAAxDSMJpVRxtkcQq0rlYVDx4XtLGw8fV2t1vcTUATqampVtHW7pltxmaXZhidTkBMTkrUYkxDnX1ubSrtt3qcgAgqBCScFrm0AYOkR25wvQ4TcpKkMvt0et7G60uB8BJmFvtpuclKS7aYXE1gWGzGSotoi8JAE6GkITTMoc2sJJ0bsyDZdey5Q4ISubQhkjpRzLN9fYllR/hUFkAGIqQhFNq7e7X4aYuSYSkc2VuuXttbyN7/4Eg5DtENkL6kUxlQ1aSPB6PxdUAQPAgJOGUdnhXkQrSYpUaH21xNaFtTmGKUuKi1NLVr/eqWqwuB8AQbT392lM3OKTGDA2RYlZBiqLshhrae1XV3G11OQAQNAhJOKVtbLUbNQ67TRdPzpTElDsg2LxX2SK3RypMi1NWUozV5QRUTJTddwYeo8AB4ARCEk6pwhzaMCbF2kLCxMKpg1vu1u3ivCQgmPj6kSJsq53pxJY7+pIAwERIwilVsJI0qj42OVN2m6F9DR2q9PZ6AbDeifORImtog6nMN7yBlSQAMBGScFItXX2qbGZow2hKjo3yHd64djdb7oBg0O9y670q79CGCOtHMpl/7731HWrp6rO4GgAIDoQknNT2o4NNzIVpcUqOi7K4mvBhTrlbt5std0Aw2FnTpp5+t1LiojQhM8HqciyRnuDU+Ix4SYwCBwATIQknte1oiyQOkR1ti7znJW042KT2nn6LqwGw0Xc+UqpsNsPiaqxj9mNtIiQBgCRCEk7BPER2JlvtRtX4zASNy4hXv8ujN/cds7ocIOKZ/UilEXaI7PuZfUnmEAsAiHSEJJzUtmqGNvjLwuLB1aS1bLkDLOXxeHwrJ3MjdLKdyexL2lrdqt4Bl8XVAID1CEn4gOOdfao+Pnio4HRC0qgzt9y9urtBLjcn3ANWOdLUpWMdvYp22CJ+a/G4jHilx0erb8Dt20kAAJGMkIQPMEd/j02PU3IsQxtG29yxaUp0OtTU2aet1S1WlwNELLMfaeaYZDkddoursZZhGL6+JM5LAgBCEk7Cdz5Sfoq1hYSpKLtNC6ZkSuJgWcBK5iS3sgg9H+n9yoroSwIAEyEJH1BRzdAGf1vs3XK3ZhfnJQFWMVeSIr0fyWSuJJUfOS43W4EBRDhCEj7AXEkqIST5zcWTs2QzpN117Tra0m11OUDEae7s04HGTklSaYQeIvt+0/OSFRNl0/Gufh081mF1OQBgKUIShmnq6PW9aS8Zk2RxNeErNT7a98aMg2WBwDO32k3KSlBKXLTF1QSHaIdNs7zbrOlLAhDpCEkYxlxFGp8Rr8QYhjb408LibEnSWrbcAQFn9t2UsdVumLm+85IISQAiGyEJw5j9SJE+DjcQzFHgbx9oUlffgMXVAJHF7Ecqi/BDZN/PDI2bjjC8AUBkIyRhGN9kO/qR/G5SVoIK0mLVN+DWW/ubrC4HiBg9/S7fa91cJtsNM6coVYYxeIZUQ3uP1eUAgGUISRiGkBQ4hmFoEVvugIDbVt2qfpdHmYlOFaTFWl1OUEmKidKU7ERJUjlb7gBEMEtD0sqVKzVz5kwlJSUpKSlJ8+fP14svvui7/eKLL5ZhGMMuN998s4UVh7fG9l7VtvbIMKTphKSAWFg8uOVu3e4GRu4CATJ09LdhGBZXE3w4VBYALA5J+fn5evDBB1VeXq5NmzZp4cKFuuqqq7Rjxw7ffW666SbV1tb6Lg899JCFFYe37UOGNiQ4HRZXExnmjU9TfLRdDe292lHTZnU5QETwHSJLP9JJmVsQy+lLAhDBLA1JV155pT7xiU9o0qRJmjx5sn7wgx8oISFBGzZs8N0nLi5OOTk5vktSEmOp/WWbeYisdwQs/M/psOujkzIlcbAsEAhut4fJdh+izBuStte0MVQGQMQKmp4kl8ulZ599Vp2dnZo/f77v+qeffloZGRkqKSnR8uXL1dXVddrH6e3tVVtb27ALzgyHyFpj4dQTW+4A+Nf+xg619QwoLtquabl86HYyY1JilZscI5fboy2VLVaXAwCWsHxPVUVFhebPn6+enh4lJCRo9erVmjZtmiTpC1/4goqKipSXl6dt27bprrvu0p49e/Tcc8+d8vFWrFih+++/P1Dlh5WKoy2SpJmM/w6oS6ZkyTAGQ2p9W4+yk2KsLgkIW2Y/0nmFKXLYg+ZzwqBTNjZNz2+t0aYjx3XBxAyrywGAgLP8N8SUKVO0ZcsW/etf/9LXv/51XX/99dq5c6ck6Wtf+5ouu+wyzZgxQ9ddd53+8Ic/aPXq1Tpw4MApH2/58uVqbW31XaqqqgL1VwlpDW09qm/rlc0Qn64GWGai03fKPatJgH+Zh6SW0o90WnN9wxvoSwIQmSwPSdHR0Zo4caJKS0u1YsUKzZo1Sz/96U9Pet958+ZJkvbv33/Kx3M6nb5peeYFH87cajchM0HxDG0IuEXeKXdrdxGSAH8yD0mdSz/SaZUWDf77ea+yRS4mbwKIQCN6N9zZ2akHH3xQa9euVUNDg9xu97DbDx48OOKC3G63ent7T3rbli1bJEm5ubkjfnycnDm0YQZb7SyxaGq2Hn5lr97af0w9/S7FRNmtLgkIO3WtPapq7pbNkM4rJCSdTnFOkhKcDnX0Dmh3XZum5/G7AUBkGVFI+upXv6r169fr3//935WbmzvicyaWL1+uK664QoWFhWpvb9czzzyj1157TS+//LIOHDigZ555Rp/4xCeUnp6ubdu2admyZVqwYIFmzpw5oufDqW3nEFlLTc1NVG5yjGpbe/TOgSZd4l1ZAjB6zFWkqblJHHPwIew2Q3OKUvX63kZtOnyckAQg4ozot8SLL76ov//977rwwgvP6ckbGhr0pS99SbW1tUpOTtbMmTP18ssv6+Mf/7iqqqq0Zs0aPfbYY+rs7FRBQYGWLFmi//zP/zyn58TJbTtqjv/mF6EVDMPQwuIsPf2vSq3dXU9IAvzA7EcyzwHC6ZV5Q9LGw826/oKxVpcDAAE1opCUmpqqtLRz/yXzxBNPnPK2goICrV+//pyfAx+uvq1Hje3m0AZCklUWT83W0/+q1LpdDfJc5RnxCi2AkzNXksx+G5yeeY7UpsPH5fHwmgQgsoxocMMDDzyge++990PPLEJoMPuRJmUlKjaaXhirzJ+Qrpgom2pae7Srtt3qcoCw0tE7oJ01g+fmcYjsmZldkCKHzVBdW4+OtnRbXQ4ABNSIVpIefvhhHThwQNnZ2Ro7dqyioqKG3b558+ZRKQ6BYU62Y2iDtWKi7LpoYobW7GrQut31mpbHZEZgtGypbJHbI+Wnxio3OdbqckJCXLRD0/OStLW6VZsOH1d+apzVJQFAwIwoJF199dWjXAasVFHdIomhDcFg0dRsrdnVoDW7GnTLwklWlwOEDfO8nzK22p2VsrFpgyHpSLOuPm+M1eUAQMCcdUgaGBiQYRi68cYblZ+f74+aEEAej4eVpCByyZTBgQ1bq1vU2N6rzESnxRUB4cHsRypjaMNZKStK1RNvHvINvQCASHHWPUkOh0M//vGPNTAw4I96EGB1bT061tEnu83QtFy2d1ktJzlGJWOS5PFIr+3hYFlgNAy43HqvskUSk+3OVqm3f2tPfbtau/strgYAAmdEgxsWLlzI5LkwcWJoQwIHmAaJRcXZkqS1uwhJwGjYVduurj6XkmIcmpSVYHU5ISUrMUZj0+Pk8UibK1lNAhA5RtSTdMUVV+juu+9WRUWFSktLFR8fP+z2T3/606NSHPxvO+cjBZ1FU7P007X79Ma+RvUOuOR0EF6BczF09LfNxhjrs1ValKbDTV3adLjZtyUYAMLdiELSf/zHf0iSHnnkkQ/cZhiGXC7XuVWFgDFXkhjaEDxK8pKVmehUY3uv3j3UrI9OyrS6JCCkmf009CONzNyxqfrL5mr6kgBElBFtt3O73ae8EJBCx/ChDSnWFgMfm83QouLBT2vZcgecG4/Hw2S7c2SGyy1VLeobcFtcDQAExohCEsJDTWuPmjv75LAZKs5JtLocDLHQDEm76+XxeCyuBghd1ce71dDeqyi7oVkFKVaXE5ImZMYrNS5KvQNuba9ptbocAAiIEW23+973vnfa2++9994RFYPAMs9HmpydyNCGIHPRpAxFO2yqau7W/oYOTcomxAIjYa4ilYxJ5nVuhAzDUGlRmtbsqlf54eOaU8iKHIDwN6KQtHr16mFf9/f369ChQ3I4HJowYQIhKURUMLQhaMVFO3TBhHS9tqdRa3Y1EJKAEdro7aNh9Pe5mTs2VWt21Wvj4WbdtGC81eUAgN+NKCS99957H7iura1NN9xwgz7zmc+cc1EIDHNoQwlDG4LSouIsvbanUet21+vrF0+wuhwgJJUfoR9pNJR5z0sqP3JcHo9HhsGUQADhbdR6kpKSknT//ffrnnvuGa2HhB95PB7Gfwe5hVMHz0sqP3Jcxzv7LK4GCD0tXX3aW98haXD8N0auZEyyoh02NXX26dCxTqvLAQC/G9XBDa2trWptpakzFFQf79bxrn5F2Q1NYWhDUBqTEqvinES5PdJre5lyB5yt8iODW+3GZ8YrPcFpcTWhzemwa7Z3CiqjwAFEghFtt/vZz3427GuPx6Pa2lr993//t6644opRKQz+ZfYjTclJ5LDSILZoapZ217Vr7a4Gfea8fKvLAULKJm9ImltEP9JoKB2bqncPN2vj4WZ9bm6B1eUAgF+NKCQ9+uijw7622WzKzMzU9ddfr+XLl49KYfAv3/lI9CMFtUVTs/X4qwe0fm+j+l1uRdmZ2g+cqU3eyXalY9lqNxrmjk3VSp1YoQOAcDaikHTo0KHRrgMBVlFthqQUawvBac3KT1F6fLSaOvu08XCzLpiQYXVJQEjoHXBpq/d1jsl2o6O0cPDf48FjnTrW0asMtjACCGMj+lj6xhtvVHt7+weu7+zs1I033njORcG/PB4P479DhN1m6OIpgwfLrttFXxJwprYfbVXfgFsZCdEamx5ndTlhITkuSpOzEySxmgQg/I0oJP3Xf/2Xuru7P3B9d3e3/vCHP5xzUfCvquZutXb3K9pu02TO3wl6i6d6Q9JuQhJwpszzkUqLUhlXPYrKvKty5lZGAAhXZxWS2tra1NraKo/Ho/b2drW1tfkux48f1z/+8Q9lZWX5q1aMkm1HWyRJxbmJinbQ4xLsLpqUoSi7oYPHOnWwscPqcoCQsIlDZP3CPG9qIxPuAIS5s+pJSklJkWEYMgxDkydP/sDthmHo/vvvH7Xi4B/mVjsOkQ0NiTFRmjcuXW/uP6Z1uxs0PjPB6pKAoOZ2e04cIktIGlVm6NxR06ruPpdio5mOCiA8nVVIevXVV+XxeLRw4UL95S9/UVraiV8+0dHRKioqUl5e3qgXidFlDm2YSUgKGYumZunN/ce0Zle9vvrR8VaXAwS1g8c6dLyrXzFRNk3PS7K6nLCSnxqr7CSn6tt6tbW6ReePT7e6JADwi7MKSR/72MckDU63KywsZJ93CBo6tGEGQxtCxsLiLN3//E5tPHxcrd39So6NsrokIGiZW+1mF6QwNn+UGYahsqI0/b2iVpsONxOSAIStEf32KCoq0ptvvqkvfvGLuuCCC3T06FFJ0n//93/rzTffHNUCMbqONHWpvWdA0Q6GNoSSovR4TcxKkMvt0et7G60uBwhqZr9MGYfI+kWZ99ypTUy4AxDGRhSS/vKXv+iyyy5TbGysNm/erN7eXklSa2urfvjDH45qgRhd5irS1NwkPmENMYu8U+7W7qq3uBIguG3y9SNxiKw/mH1J5UeOy+X2WFwNAPjHiN4lf//739evfvUr/fa3v1VU1IltPxdeeKE2b948asVh9Pm22o1hn36oWVScLUl6bW+jBlxui6sBglNDe4+ONHXJMKQ5RYQkfyjOSVRctF3tPQPaW//BMxMBIByMKCTt2bNHCxYs+MD1ycnJamlpOdea4EfbqlskSTPHpFhaB87enMIUJcdGqaWrX+9VtVhdDhCUyr1b7aZkJyopht49f3DYbZpTyJY7AOFtRCEpJydH+/fv/8D1b775psaPZ/JWsHK7PdpxtE0S479DkcNu0yVTMiVJa9hyB5zURs5HCghfXxKHygIIUyMKSTfddJNuvfVW/etf/5JhGKqpqdHTTz+tO+64Q1//+tdHu0aMksNNnWrvHZDTYdOkbM7aCUULpw5uuVu3q8HiSoDgVE4/UkCYQzE2cagsgDB1ViPATXfffbfcbrcWLVqkrq4uLViwQE6nU9/85jf11a9+dbRrxCgx+5Gm5TG0IVR9bFKm7DZD+xo6VNnUpcL0OKtLAoJGV9+AttcMrpZziKx/zS5Mkd1m6GhLt2paupWXEmt1SQAwqkb0TtkwDH3nO99Rc3Oztm/frg0bNqixsVHJyckaN27caNeIUWIeIjuDrXYhKzkuSnO9n5Cv3c2WO2CoLZUtcrk9ykuO0RjetPtVgtOhabmDA4DoSwIQjs4qJPX29mr58uUqKyvThRdeqH/84x+aNm2aduzYoSlTpuinP/2pli1b5q9acY62HSUkhQNzyt263Wy5A4Yy36yzihQYpUX0JQEIX2cVku69916tXLlSY8eO1aFDh3Tttdfqa1/7mh599FE9/PDDOnTokO666y5/1YpzMDi0wRuS8glJoWyh97ykDQeb1NE7YHE1QPDYeJh+pEAyh2PQlwQgHJ1VT9KqVav0hz/8QZ/+9Ke1fft2zZw5UwMDA9q6dasMw/BXjRgFB491qrPPpZgomyZmMrQhlE3ITNC4jHgdOtapN/Y26ooZuVaXBFjO5fbovcoWSSeGCsC/zDC6u65N7T39SmTkOoAwclYrSdXV1SotLZUklZSUyOl0atmyZQSkELDdu4o0PS9ZDoY2hLyFxYOrSWvZcgdIGnyj3tE7oESnQ1NyEq0uJyJkJ8WoIC1Wbo98ARUAwsVZvVt2uVyKjo72fe1wOJSQwKpEKNjG0Iawssgbkl7d3SC322NxNYD1zC1f5xWlym7jg7tAmesbBU5fEoDwclbb7Twej2644QY5nU5JUk9Pj26++WbFx8cPu99zzz03ehViVGxnaENYmTsuTYlOh5o6+7SlukVzCunBQGQzhzbMLeL/hUAqHZuq59476jvEFwDCxVmFpOuvv37Y11/84hdHtRj4h8vt0fYahjaEkyi7TQumZOrv22q1blcDIQkRzePxaOOhwZWMUoY2BJQ5vGFLVYv6XW7O4AMQNs4qJD355JP+qgN+dLCxQ119LsVG2TWBoQ1hY/HULP19W63W7m7QnZdNsbocwDJHW7pV19Yjh83Q7IIUq8uJKBMzE5QcG6XW7n7trGnTLP79AwgTfOQTASp8QxuS2KsfRj42OUs2Q9pV26ajLd1WlwNYpty71W76mGTFRZ/VZ384Rzab4TsvaSN9SQDCCCEpAviGNrDVLqykxUf7ttlxsCwime98JPqRLGGOAjfDKgCEA0tD0sqVKzVz5kwlJSUpKSlJ8+fP14svvui7vaenR0uXLlV6eroSEhK0ZMkS1dfXW1hxaDKHNswkJIWdRVOzJUnrdvH/BSKXOdluLv1IljD7kjYePi6Ph2mbAMKDpSEpPz9fDz74oMrLy7Vp0yYtXLhQV111lXbs2CFJWrZsmZ5//nmtWrVK69evV01Nja655horSw45Ay63dtS0SWKyXThaNHVwFPhbB5rU1TdgcTVA4LV292tPfbskqZRDZC0xY0yyou02Hevo1ZGmLqvLAYBRYenm7SuvvHLY1z/4wQ+0cuVKbdiwQfn5+XriiSf0zDPPaOHChZIGB0dMnTpVGzZs0Pnnn3/Sx+zt7VVvb6/v67a2Nv/9BULAgcZOdfe7FB9t17gMhjaEm0lZCcpPjVX18W69tb9JH5+WbXVJQEBtrjwuj0camx6nzESn1eVEpJgou2bkJ6v8yHFtOnJcYzPiP/ybACDIBU1Pksvl0rPPPqvOzk7Nnz9f5eXl6u/v1+LFi333KS4uVmFhod55551TPs6KFSuUnJzsuxQUFASi/KB1YmhDMkMbwpBhGFrs3XK3li13iEDmIaZlY1lFspLZl8ShsgDCheUhqaKiQgkJCXI6nbr55pu1evVqTZs2TXV1dYqOjlZKSsqw+2dnZ6uuru6Uj7d8+XK1trb6LlVVVX7+GwS3iuoWSQxtCGcLiwe33K3b3SC3m34ARBb6kYJDWZHZl0RIAhAeLJ+VOmXKFG3ZskWtra3685//rOuvv17r168f8eM5nU45nWy5MFUwtCHszRufpvhouxrae7Wjpo1AjIjRN+DWlqoWSfQjWc0cA36gsVPNnX1Ki4+2uCIAODeWryRFR0dr4sSJKi0t1YoVKzRr1iz99Kc/VU5Ojvr6+tTS0jLs/vX19crJybGm2BAz4HJrZ+1gT1YJQxvCltNh10cnZUqS1rDlDhFkR02regfcSo2L0oRM+mCslBYfrYlZg32vjAIHEA4sD0nv53a71dvbq9LSUkVFRWnt2rW+2/bs2aPKykrNnz/fwgpDx/7GDvX0u5XgdGhcOm8gwtnCqSe23AGRwtxqVzY2TYZBz6XVzHOqNh1hyx2A0Gfpdrvly5friiuuUGFhodrb2/XMM8/otdde08svv6zk5GR95Stf0e233660tDQlJSXpG9/4hubPn3/KyXYYzjxEdnpekmwMbQhrl0zJkmEMbq+sb+tRdlKM1SUBfschssGlbGyant1Y5QuvABDKLA1JDQ0N+tKXvqTa2lolJydr5syZevnll/Xxj39ckvToo4/KZrNpyZIl6u3t1WWXXaZf/vKXVpYcUiqq6UeKFJmJTs3KT9GWqhat292gz3+k0OqSAL/yeDy+bV1MtgsOZlitqG5VT79LMVF2iysCgJGzNCQ98cQTp709JiZGjz/+uB5//PEAVRRezKEN9CNFhkXFWdpS1aK1uwhJCH+HjnWqqbNP0Q6bSsYkWV0OJBWlxykjwaljHb2qONqquYRXACEs6HqSMDr6hwxtmJmfYm0xCAizL+mt/cfU0++yuBrAv8wtXbPzU+R0sGIRDAzD8I1iZxQ4gFBHSApT++o71DfgVmKMQ0VpcVaXgwCYlpuk3OQYdfe79M6BJqvLCUkDLrcqm7r01v5jOtLUaXU5OA1zOEAZ5yMFFXMUOH1JAEKd5eckwT8qjrZIkkrykhnaECEMw9DC4iw9/a9Krd1dr0u8h8xiuLaeflU2damyecjF+/XRlm65vAfyxkfb9eKtC1SYzocMwejEZDtCUjAxt9iVHzkut9vD7x8AIYuQFKa2MbQhIi2aOhiS1u1qkOcqT0SORXa5Papt7VZlc5eqmrt0xBuAqpq7dKS5Sy1d/af9/miHTTEOm9p6BnTnqq169mvn80YvyBzr6NXBY4MrfaWF9L0Ek2l5SYqNsqu1u1/7Gzs0OTvR6pIAYEQISWFqO0MbItIFEzIUE2VTTWuPdtW2a1peeDa0d/QO+FZ/BsNPpyqbu1XV3KXq413qd3lO+/0ZCU4VpsWqMC1Ohenxg/9Mi1NRepwyE5w62tKtyx57Xe8ebtbv3jqkr350fID+ZjgT5lS7KdmJSo6LsrgaDBVlt2l2QYreOdikjYebCUkAQhYhKQz1Dbi1q7ZdEitJkSYmyq6LJmZoza4GrdtdH7Ihye32qL69R5VNg6s/Vd5tcUeaBv/c1Nl32u+PshsqSI1TgTf4FKad+HNBapzinad/6StIi9N3PjlV31m9XT9+eY8unpKliVkJo/lXxDnY5B0KUMpWu6A0d2yq3jnYpPLDx3XdvCKrywGAESEkhaG99e3qc7mVFONQIUMbIs7C4myt2dWgtbsbdMvCSVaXc0rdfa739QV1+v5cdbxbfQPu035/alzUkFWgWBWlxasgLU6F6XHKSYqR/Ry3yH3hI4V6aXud3th3THes2qq/3DxfDjuzboLBRm8/0lxCUlAyz63aeIQJdwBCFyEpDJnnI83IT47InpRIt9A7sGFLVYuOdfQqI8FpSR0ej0eN7b2+FaChfUGVzV1qbO897fc7bIbGpMb6tsL5LumDq0JJMf7dZmUYhh767Exd+ujr2lrVol+/flBLL5no1+fEh+vuc2lHzeBrXFkR/UjB6LzCFNkMqaq5W/VtPcpOirG6JAA4a4SkMOQLSWNSrC0ElshJjlHJmCRtP9qmV3c36NqyAr89V0+/S9XHT0yIG7o1rrK5Sz39p18NSopxqMi7GjR0a1xhWpxyk2MsX7nJTY7VfVdO1x2rtuqxNXu1sDhLU3NDcwtjuNha3aJ+l0fZSU7lp8ZaXQ5OIjEmSsU5SdpZ26ZNh4/rkzNzrS4JAM4aISkMVVSbIYl+pEi1sDhb24+2ad05hiSPx6Omzr5hY7KH/rmuree0328zpLyUWN9QhAJzQELaYDAKhab7a+aM0Yvb67RmV73u+NNW/XXphYp2sO3OKubQhrKxaayUB7G5Y1O1s7ZNGw83E5IAhCRCUpjpHXBpd12bJIY2RLLFU7P0s7X79PreRvUOuOR02E95374Bt281aOjIbPPrzj7XaZ8rwekYthVu6Na4MamxigrxPh7DMPTDa0pU/mizdta26Rfr9un2S6dYXVbE2ugd2jC3iH6kYFY6Nk3/9c4R36G/ABBqCElhZm9dh/pdHqXERbEVJYKV5CUrM9GpxvZe/etgs2aMSR7sDTK3wzUNjs2uau5WTWu3PKeZmG0YUm5SzEkmxQ2uBqXGRYX9J/pZiTF64OoS3fLMe3r8tQNaPC1bM/NTrC4r4rjdnmErSQhe5lCNnTVt6ugdUMKHTJQEgGDDq1aY2Xa0RdLgVrtwf+OKU7PZDC2ckqU/bqrSl3+/US736c8Nio2yD98ON+TP+amxp12JihSfmpmnl7bX6YVttbr9T1v1wjcuUkwU/14CaW9Du9p7BhQfbVdxDufvBLPc5FiNSYnV0ZZubals0UWTMqwuCQDOCiEpzGw/Sj8SBn16dp7+uKnKF5Cyk5zebXDesdnpsb4/ZyREE6rPwANXlWjDwWbtb+jQo6/s1fJPTLW6pIhijv4+rzDV8qEe+HBlY1N1dEu3Nh1pJiQBCDmEpDCzjaEN8LpwYob+uWyBDA0ejsqqx7lLjY/Wimtm6KY/bNJv3jioj0/LZttXAJmHyJZxPlJIKBubpv/dUqNN3nALAKGEj+LCSE+/S3vq2iUNnpEETM5O1KTsRALSKPr4tGwtmZMvj0e6c9VWdfUNWF1SxNjkO0SWYBoKyrzDNTZXHteA6/THAQBAsCEkhZE9de0acHuUGhelMSkMbQD85d4rpyk3OUaHm7r0oxd3W11ORKhp6dbRlm7ZbYZmF6RYXQ7OwOTsRCXGONTV59Ju7wd4ABAqCElhZJvZj5SfQn8J4EfJsVH60ZKZkqT/eueI3t5/zOKKwt8m71S7ablJimdSWkiw2wyVeleTzNHtABAqCElhZLu3H2km/UiA3y2YnKnr5hVKkr75521q7+m3uKLwVk4/Ukgyt9zRlwQg1BCSwoi5klRCSAIC4tufmKqCtMExxz/4+y6rywlr5mS7siL6kUKJOdhk05FmeU53IBsABBlCUpjo6XdpX/3gnu+ZDG0AAiLe6dCPPztLhiE9u7FKr+5usLqksNTe06/ddW2SWEkKNbPyUxRlN1Tf1qvq491WlwMAZ4yQFCZ21bZpwO1Reny0cpNjrC4HiBjnj0/Xly8YJ0m66y/b1NLVZ3FF4ee9yha5PVJhWpyyk3h9CyWx0XZNzxv84I6+JAChhJAUJip8QxuSGdoABNi3Lp+i8Znxamjv1X1/22F1OWHHdz5SEatIoWiud/XPHL4BAKGAkBQmKhjaAFgmJsquh6+dJZsh/XVLjV7aXmt1SWHFfHPNwb2hydeXxEoSgBBCSAoTFQxtACx1XmGqbv7YBEnSd1Zv17GOXosrCg/9Lrfeq2yRdGJFAqHFHAO+t76D7agAQgYhKQx097m0r6FDkjQzP8XaYoAIduviSSrOSVRTZ5/+c/V2pnmNgp01berudyk5NkoTMhOsLgcjkJHg1PiMeEnS5kq23AEIDYSkMLCztk0ut0cZCU5lJzmtLgeIWE6HXT+5dpYcNkMv7ajT37bWWF1SyPNttStKlc1Gv2WoMqcSbuS8JAAhgpAUBiqqWyQNjv5maANgrZIxyfrGwkmSpHv+ul31bT0WVxTazD6WUrbahTTzfKtyQhKAEEFICgMVRwfPD6EfCQgO/3HJBM0Yk6y2ngHd/ZdtbLsbIY/H41t5mMvQhpBmriRtqW5R74DL4moA4MMRksJAxdEWSUy2A4JFlN2mhz83S9F2m17d06g/baqyuqSQVNncpWMdvYq22zSD17eQNi4jXunx0eobcGu7d9AQAAQzQlKI6+ob0H7v0IYZ+byJAILF5OxE3XHpZEnSAy/sUvXxLosrCj3mKtKM/GTFRNktrgbnwjAM35S7TWy5AxACCEkhbmdNm9weKSvRyUn0QJD56kfHq7QoVR29A/rWn7fJ7Wbb3dnwHSJLP1JYMLdMMrwBQCggJIU483ykmawiAUHHbjP0k2tnKSbKprcPNOmpfx2xuqSQYk62m1tEP1I4MIdvlB9ppk8PQNAjJIW4imoOkQWC2biMeC2/YqokacU/duvwsU6LKwoNzZ19vq3E5jYthLaSvGQ5HTYd7+rXgUb+PwAQ3AhJIW4bK0lA0Pv384s0f3y6uvtdunPVVrnYdvehyr2rSBOzEpQaH21xNRgN0Q6bZhekSDqxlRIAghUhKYR19g7oQOPgJ62sJAHBy2Yz9NBnZyrB6dCmI8f1xJsHrS4p6G06Mvgmei79SGGFQ2UBhApCUgjbUdMmj0fKSYpRViJDG4BgVpAWp//85OC2u5/8c6/21bdbXFFwMyegldKPFFbKvMMbyo+wkgQguBGSQpg5tIHR30Bo+D9zC3TxlEz1Dbh1x6qtGnC5rS4pKPX0u3z9lqwkhZc5hakyDOlwU5ca2nusLgcATomQFMIqqlskiUMWgRBhGIZ+tGSmkmIc2lbdqpWvHbC6pKBUcbRVfS63MhOdKkyLs7ocjKLk2ChNyU6UJJWz5Q5AECMkhbBtrCQBISc7KUbfu6pEkvTTtfu0o6bV4oqCz0bzfKSiVBmGYXE1GG1mX5I54h0AghEhKUS19/TrkHeUMCtJQGi5anaeLpuerQG3R3f8aat6B1xWlxRUzBUGs38F4cU8VJYJdwCCmaUhacWKFZo7d64SExOVlZWlq6++Wnv27Bl2n4svvliGYQy73HzzzRZVHDzMoQ15yTHKSHBaXQ6As2AYhn7wmRlKi4/W7rp2/WztPqtLChput8e3wlDG+UhhyTz3antNm7r6BiyuBgBOztKQtH79ei1dulQbNmzQK6+8ov7+fl166aXq7Bx+yNxNN92k2tpa3+Whhx6yqOLgsZ2tdkBIy0hw6gdXD267W/naAb1XydYjSdrf2KHW7n7FRtk1LS/J6nLgB2NSYpWbHCOX26MtVS1WlwMAJ2VpSHrppZd0ww03aPr06Zo1a5Z+//vfq7KyUuXl5cPuFxcXp5ycHN8lKYlfnNu8k5/YageEritm5Oqq2Xlye6Q7Vm1VTz/b7szR3+cVpijKzo7wcGQYhm81aRPDGwAEqaD6DdTaOvjGPy1t+D70p59+WhkZGSopKdHy5cvV1dV1ysfo7e1VW1vbsEs4OrGSlGJtIQDOyf2fnq6sRKcONnbqJy/v+fBvCHObhgxtQPjy9SUxvAFAkAqakOR2u3XbbbfpwgsvVElJie/6L3zhC3rqqaf06quvavny5frv//5vffGLXzzl46xYsULJycm+S0FBQSDKD6i2nn4dZGgDEBZS4qL14JIZkqQn3jqkdw9FdjP7Ru8howxtCG/mhLvNR47L5fZYXA0AfFDQhKSlS5dq+/btevbZZ4dd/7WvfU2XXXaZZsyYoeuuu05/+MMftHr1ah04cPLzRZYvX67W1lbfpaqqKhDlB5S5ijQmJVZp8dEWVwPgXC0sztbnyvLl8Uh3rtqqzt7IbGavb+tRVXO3bMbgdjuEr+KcJCU4HeroHdDuuvDc8QEgtAVFSLrlllv0wgsv6NVXX1V+fv5p7ztv3jxJ0v79+096u9PpVFJS0rBLuDFD0kyGNgBh455PTdOYlFhVNndpxYu7rC7HEmZ/SnFOkhJjoiyuBv5ktxm+IFzOljsAQcjSkOTxeHTLLbdo9erVWrduncaNG/eh37NlyxZJUm5urp+rC17m0IYSttoBYSMxJkoPfXamJOmpDZV6Y1+jxRUFnnmI7Nyx9CNFArMvaSPDGwAEIUtD0tKlS/XUU0/pmWeeUWJiourq6lRXV6fu7m5J0oEDB/TAAw+ovLxchw8f1t/+9jd96Utf0oIFCzRz5kwrS7cUK0lAeLpwYoa+NL9IkvStP29TW0+/xRUFlrmiQD9SZCjzTbiL7D48AMHJ0pC0cuVKtba26uKLL1Zubq7v8sc//lGSFB0drTVr1ujSSy9VcXGx7rjjDi1ZskTPP/+8lWVbqrWrX4ebBqf7leQRkoBwc/cVxSpKj1Nta48eeH6n1eUETGfvgHbWDvamlLGSFBFmF6bIbjNU29qjoy3dVpcDAMM4rHxyj+f0E20KCgq0fv36AFUTGrbXDK4iFaTFKpWhDUDYiYt26CfXztLnfv2OVpVX6/KSHC2amm11WX63papFLrfHe9BorNXlIADioh0qyUvS1upWbTrcrDGzx1hdEgD4BMXgBpy5iqMcIguEu7lj0/TViwZ7NO9+rkLHO/ssrsj/6EeKTKVFZl8SW+4ABBdCUoipqDZDUoq1hQDwqzsunaKJWQlqbO/VvX/bYXU5fmdOtiulHymimKF4E8MbAAQZQlKIqWBoAxARYqLsevjaWbLbDD2/tUZ/31ZrdUl+M+By673KwTfJrCRFllLvf+899e1q7Y6sQSUAghshKYS0dPWpspmhDUCkmFWQov+4eIIk6T//WqHG9l6LK/KP3XXt6uxzKTHGoclZiVaXgwDKSoxRUXqcPB5pcyWrSQCCByEphGw/Ojj5qSg9TslxHLQIRIJvLJykqblJOt7Vr++srvjQgTehyOxHKS1Klc1mWFwNAq3M25dUzpY7AEGEkBRCth1tkcQhskAkiXbY9PC1sxRlN/TPnfVa/d5Rq0sadZuOmFvt6EeKROYWS4Y3AAgmhKQQYg5tmElIAiLKtLwk3bpokiTpu3/bodrW8DlTxuPx+A4TLS2iHykSmediba1uUd+A2+JqAGAQISmE+MZ/M7QBiDg3f2yCZuUnq71nQHf9JXy23VUf71Z9W6+i7IZm5adYXQ4sMCEzQalxUerpd2uH9yxAALAaISlEHO/sU/XxwU+P2W4HRB6H3aaHPzdL0Q6bXt/bqGc3Vlld0qjYdGRwFalkTLJio+0WVwMrGIbhW0VkFDiAYEFIChHmKtK4jHglxTC0AYhEE7MS9a3LpkiSvv/CTlV5p12Gso3eN8VlbLWLaGXefjQzNAOA1QhJIcIMSawiAZHtyxeO09yxqersc+mbf94qtzu0t92Z/UhlDG2IaEMPlQ2XraQAQhshKURsq26RxNAGINLZbYZ+cu0sxUbZteFgs/7rncNWlzRirV392lvfIYmVpEhXMiZZ0Q6bmjr7dOhYp9XlAAAhKVSYZySxkgSgKD1e3/5EsSTpRy/t1sHGDosrGpnyysFVpPEZ8UpPcFpcDazkdNg1yzuUyBwJDwBWIiSFgKaOXh1tMYc2JFlcDYBgcN28Il00MUM9/W7dsWqrXCG47c5s0jdHQCOy+fqSOC8JQBAgJIUAsx9pfGa8EhnaAECSzWboR5+dqUSnQ+9Vtug3rx+0uqSz5gtJRfQj4cSWSybcAQgGhKQQYB4iO4OtdgCGGJMSq3uunCZJevSVvdpT125xRWeud8ClLd5eS1aSIJ04TPjgsU41dfRaXA2ASEdICgG+Q2QJSQDe59rSfC0qzlKfy63b/7RF/S631SWdke1H29Q34FZ6fLTGZcRbXQ6CQEpctCZnJ0iiLwmA9QhJIYCQBOBUDMPQimtmKDk2Sjtq2vT4q/utLumMmH0npUWpMgzD4moQLEqL6EsCEBwISUGusb1Xta09MgxpOiEJwElkJcXoe1dNlyT9Yt1+bfd+sBLMzENk53I+EobwnZfEShIAixGSgpz5ZmdCZoISnA6LqwEQrD49K0+fmJGjAbdHt/9pi3oHXFaXdEoej0flR8xDZOlHwglmaN5+tFXdfcH7Mwwg/BGSgtw2hjYAOAOGYeiBq0qUHh+tvfUdevSVfVaXdEoHGjt1vKtfTodN0/N4bcMJ+amxykp0qt/l0VbvYA8AsAIhKcjRjwTgTKUnOPXDa2ZIkn7z+gGVB+mWJbPfZHZBiqId/BrCCYZh+FaTgvXnF0Bk4LdTkKs42iJJmpFPSALw4S6bnqPPnDdGbo9056qtQbllyew3oR8JJ2OOAt/I8AYAFiIkBbGGth7Vt/XKZkjTcpOsLgdAiLjvyunKTnLq0LFOPfTybqvL+QDfZDv6kXASQ1eS3G6PxdUAiFSEpCBmbrWbmJWgeIY2ADhDyXFR+tGSmZKkJ986rHcONFlc0QmN7b063NQlw5DmFBKS8EFTcxMVF21Xe8+A9jaEzgHJAMILISmImUMbSuhHAnCWLp6Spc9/pECS9M0/b1VH74DFFQ0yp9pNyU5UcmyUxdUgGDnsNp1XmCLpxKh4AAg0QlIQM8d/zyQkARiB73xymsakxKr6eLd+8PddVpcj6cSbXkZ/43TKvIfKltOXBMAihKQgts2cbMfQBgAjkOB06MfXDm67+593K7V+b6PFFZ3oR2JoA07H/PlgJQmAVQhJQaq+rUeN7ebQBkISgJG5YEKGbrhgrCTprj9vU2t3v2W1dPUNaEdNm6QTE8yAk5ldmCKbIR1t6VZta7fV5QCIQISkIGX2I03KSlRstN3iagCEsrsuL9a4jHjVtfXo/ud3WFbHlqoWDbg9yk2O0ZiUWMvqQPBLcDo0LW9wqusmVpMAWICQFKQqvCeNs9UOwLmKjbbrJ9fOlM2Qntt8VP/cUWdJHeW+fqQ0GYZhSQ0IHWZf0ib6kgBYgJAUpMzx3zMJSQBGQWlRmm5aMF6S9O3VFWru7At4DRu9h8iWsdUOZ8Ac7kFfEgArEJKCkMfj8YUkxn8DGC3LFk/WpKwEHevo0z1/3R7Q53a5Pdp8hMl2OHPmStLuuja191jXSwcgMhGSglBdW4+OdfTJbjM0LTfJ6nIAhImYKLse+dxs2W2G/l5Rq+e31gTsuffUtaujd0AJToeKc3hdw4fLSY5RQVqs3B7pvcoWq8sBEGEISUHoxNCGBMVEMbQBwOiZkZ+spZdMlCTd87/b1dDeE5Dn3eQ9RPa8whTZbfQj4czQlwTAKoSkIFRRTT8SAP+55ZKJmp6XpJaufi3/S4U8Ho/fn9PsK+F8JJwNc2vmpiP0JQEILEJSEKrwHSKbYm0hAMJStMOmhz83S1F2Q2t3N+jP5dV+f85y70oA/Ug4G2aofq+yRf0ut8XVAIgkhKQgM3RowwyGNgDwk+KcJC37+GRJ0vee36maFv8d2Hm0pVs1rT2y2wzNLkjx2/Mg/EzMTFBSjEPd/S7tqm2zuhwAEYSQFGRqWnvU3Nknh81QcU6i1eUACGNf++h4zS5IUXvvgO76yza/bbsz+0lK8pIUF+3wy3MgPNlshsq8q0mMAgcQSISkIGMeIjs5O5GhDQD8ymEf3HbndNj0xr5jevpflX55nk1DDpEFzlap91wthjcACCRCUpDhEFkAgTQhM0F3XV4sSfrhP3apsqlr1J9jo9mPxCGyGAGzL2nTkeMBGTICAJLFIWnFihWaO3euEhMTlZWVpauvvlp79uwZdp+enh4tXbpU6enpSkhI0JIlS1RfX29Rxf5njv/mEFkAgXLDBWM1b1yauvpcunPVVrndo/dGtLW7X3vq2yVJpQxtwAjMzE9WtN2mxvZeVTaPfogHgJOxNCStX79eS5cu1YYNG/TKK6+ov79fl156qTo7O333WbZsmZ5//nmtWrVK69evV01Nja655hoLq/afoUMbWEkCECg2m6GfXDtLcdF2vXu4Wb9769CoPfZ7lcfl8UhF6XHKSowZtcdF5IiJsqtkzOABxPQlAQgUS0PSSy+9pBtuuEHTp0/XrFmz9Pvf/16VlZUqLy+XJLW2tuqJJ57QI488ooULF6q0tFRPPvmk3n77bW3YsMHK0v2i+ni3Wrr6FWU3NIWhDQACqCAtTt/55FRJ0o9f3qP9DR2j8ri+fqQi+pEwcuaWu/Ij9CUBCIyg6klqbR1cRUlL874Ylperv79fixcv9t2nuLhYhYWFeuedd076GL29vWpraxt2CRXmKtKUnEQ5HQxtABBYX/hIoT46KUO9A27dsWqrBkbhXJpN3je1c9lqh3PAhDsAgRY0Icntduu2227ThRdeqJKSEklSXV2doqOjlZKSMuy+2dnZqqurO+njrFixQsnJyb5LQUGBv0sfNSfOR0qxthAAEckwDD302ZlKjHFoa1WLfv36wXN6vL4Bt7ZUtUjiEFmcG3PC3f6GDh3v7LO4GgCRIGhC0tKlS7V9+3Y9++yz5/Q4y5cvV2trq+9SVVU1ShX6X0U1h8gCsFZucqzuu3K6JOmxNXvP6QDPHTWt6ul3KzUuShMyE0arRESgtPhoTciMlySVH2E1CYD/BUVIuuWWW/TCCy/o1VdfVX5+vu/6nJwc9fX1qaWlZdj96+vrlZOTc9LHcjqdSkpKGnYJBQxtABAsrpkzRounZqvf5dEdf9qqvoGRbbsz38yWFqXJMIzRLBERyOxL2khfEoAAsDQkeTwe3XLLLVq9erXWrVuncePGDbu9tLRUUVFRWrt2re+6PXv2qLKyUvPnzw90uX5V1dyt1u5+RdttmpzN0AYA1jEMQz+8pkSpcVHaWdumX6zbN6LH8Z2PxFY7jIITh8qykgTA/ywNSUuXLtVTTz2lZ555RomJiaqrq1NdXZ26u7slScnJyfrKV76i22+/Xa+++qrKy8v15S9/WfPnz9f5559vZemjbtvRFklScW6ioh1BscAHIIJlJcbogasH+0Mff+2AtlW3nNX3ezwe35tZhjZgNJgrSRXVrerpd1lcDYBwZ+m78ZUrV6q1tVUXX3yxcnNzfZc//vGPvvs8+uij+tSnPqUlS5ZowYIFysnJ0XPPPWdh1f5xYmgDW+0ABIdPzczTp2bmyuX26PY/bT2rN6aHm7rU1NmnaIeNw7ExKorS45SR4FSfy+37nQkA/mL5druTXW644QbffWJiYvT444+rublZnZ2deu65507ZjxTKGNoAIBg9cFWJMhKc2t/QoUdf2XvG32dutZuVn8yRBhgVhmGozLvlzvz5AgB/YV9XEBg6tGEGQxsABJHU+GituGaGJOk3bxzUpjN8c7rJ14/EIbIYPWZ/Wzl9SQD8jJAUBI40dam9Z0DRDoY2AAg+H5+WrSVz8uXxSHeu2qquvoEP/Z5NR+hHwugzQ/emI8fldnssrgZAOCMkBYFt3lWkqblJirLznwRA8Ln3ymnKTY7R4aYu/ejF3ae9b1NHrw42dkqS5hQSkjB6puclKSbKptbufh1o7LC6HABhjHfkQWC7eT4S/UgAglRybJR+tGSmJOm/3jmit/YfO+V9zVWkydkJSomLDkh9iAxRdpvOKzD7kthyB8B/CElBwByty9AGAMFsweRMXTevUJL0rT9vU3tP/0nvN/QQWWC0mX1JZ9ofBwAjQUiymNvt0Y6jbZIY2gAg+H37E1NVkBaroy3d+v4Lu056H3PyGP1I8IehfUkA4C+EJIsdbupUe++AnA6bJmUlWF0OAJxWvNOhH392lgxD+uOmKr26u2HY7T39Lt8W4rlMtoMfzClMkc2QKpu7VN/WY3U5AMIUIcli5ujvaXlJcjC0AUAIOH98ur58wThJ0l1/2aaWrj7fbVurWtTv8igr0an81FirSkQYS4yJ0pScJEnSJvqSAPgJ78otZh4iy9AGAKHkW5dP0fjMeDW09+q+v+3wXX9i9HeaDMOwqjyEOXMr56Yj9CUB8A9CksXM8d8lhCQAISQmyq6Hr50lmyH9dUuNXtpeK2noIbL0I8F/fH1JrCQB8BNCkoUGhzZ4V5LyU6wtBgDO0nmFqbr5YxMkSd9ZvV0N7T2+laQyJtvBj8qKBkP4zto2dfZ++OHGAHC2CEkWOnisU519LsVE2TQhM97qcgDgrN26eJKKcxLV1NmnG3+/Ue09A4qLtmtqbqLVpSGM5aXEakxKrFxuj7ZUtVhdDoAwREiykDkBanpeMkMbAIQkp8Oun1w7Sw6boe3e4wzmFKbymga/M7d0buS8JAB+wG8xC23zDm3gEFkAoaxkTLK+sXCS7+vSIvqR4H/mlrt1uxvYcgdg1BGSLFRxtEUSIQlA6PuPSyZodkGKDENaWJxldTmIAAsmZyrKbmhbdasu/+nreudAk9UlAQgjhCSLuNwe7agZ3JoyM5+QBCC0RdltevZr52vt7R/TrIIUq8tBBChKj9d/ffkjGpMSq6rmbn3+txt03992qKuPVSUA546QZJGDjR3q6nMpLtqu8ZkJVpcDAOcsJorXMwTWBRMz9NJtH9XnP1IoSfr924d1xU/f0LuH6FMCcG4ISRap8A1tSJLdxoGLAACMRGJMlFZcM0N/uPEjykuO0ZGmLv2f37yj7z2/U919LqvLAxCiCEkWMYc2cIgsAADnbsHkTL20bIH+T1mBPB7pd28d0id+9obKj7CqBODsEZIsUuE7RJaQBADAaEiKidKPPjtTT355rnKSYnToWKc++6t39IO/71RPP6tKAM4cIckCAy63dnqHNswYk2JtMQAAhJlLpmTp5WUL9NnSfHk80m/fGFxV2lx53OrSAIQIQpIFDjR2qrvfpfhou8ZnxFtdDgAAYSc5Nko/uXaWfndDmbISnTrY2KnPrnxbK17cxaoSgA9FSLKAb2jDmGTZGNoAAIDfLCzO1ivLPqZrzhsjt0f69fqD+tTP39TWqharSwMQxAhJFqiobpHEIbIAAARCclyUHvk/s/XbL5UpI8Gp/Q0d+swv39JDL+1W7wCrSoA/udwe/bm8Wtu9iwShgpBkAYY2AAAQeB+flq1Xli3QVbPz5PZIv3ztgK78+ZuqqA6tN29AKPB4PFq7q16f+OkbunPVVv3wH7vk8XisLuuMEZICbMDl1g7v0AbGfwMAEFip8dH66b+dp199sVQZCdHaW9+hq3/5lh7+5x71DbitLg8IC+VHmvW5X7+jr/zXJu2pb1dSjEMLJmfKHToZSQ6rC4g0+xo61DvgVoLToXHpDG0AAMAKl5fk6CPj0nTv/27XC9tq9fN1+/XKznr95NpZfIgJjNC++nY99PIevbKzXpLkdNh0w4Vj9R8fm6jkuCiLqzs7hKQAM7falYxJYmgDAAAWSouP1i++MEdXlNTqnv/drt117br68be09JKJWnrJREU72HADnImalm49tmav/lxeLbdHshnS58oKdOviScpNjrW6vBEhJAWYue+ZoQ0AAASHT87M1bzxabrnr9v14vY6/XTtPt+q0rS8JKvLA4JWS1efVr52QE++fdi3XfWy6dn65mVTNDEr0eLqzg0hKcDMlaQZ+SnWFgIAAHwyEpz65XVz9MK2Wt37v9u1s7ZNVz3+pr6xcJK+fvEERdlZVQJM3X0uPfn2If3qtQNq6xmQJH1kXJruurxYpUWpFlc3OghJAdTvcmtn7eDQBlaSAAAILoZh6MpZeTp/fLq+s7pC/9xZr0de2at/7qzTw9fO1pSc0P5kHDhXAy63VpVX67E1e1Xf1itJKs5J1F2XF+viKZkyjPBpJSEkBdDe+nb1DbiVGONQUVqc1eUAAICTyEx06tf/Xqq/ba3Rvf+7Q9uPtulTP39Dty2erP+7YLwcrCohwng8Hr28o04PvbxHBxs7JUljUmJ1x6WTddXsMbKHYZ89ISmAzEO0ZoxJZmgDAABBzDAMXTV7jOaPT9e3V1doza4G/fjlPfrnjjr95NpZmpTNqhIiwzsHmvSjl3ZrS1WLJCk1LkrfWDhJ151fKKfDbm1xfkRICqBtDG0AACCkZCXF6LdfKtPq947qvr/t0NbqVn3yZ29q2ccn66aPjmNVCWFrZ02bHnp5t17b0yhJio2y66aPjtNNC8YrMSa0xnmPBCEpgHwrSfmEJAAAQoVhGLpmTr4unJihu/+yTa/uadSPXtqtl72rShOzEqwuERg1Vc1deuSVvfrrlqPyeCSHzdDnP1KobyyaqKzEGKvLCxhCUoD0Dbi1q7ZdEitJAACEouykGP3uhrn6c3m1vvf8Tm2patEnfvaGvnnpFN140biw7MtA5Gjq6NXP1+3X0/86on6XR5L0qZm5uvPSKRqbEW9xdYFneDwej9VF+FNbW5uSk5PV2tqqpCTrzjpwuz06eKxDO2ra9OlZeWE1/QMAgEhT29qtu/9SofV7B7cilRal6sefnanxmawqIbR09g7o/71xSL9946A6egfHeV80MUN3XV4clrufzjQbEJIAAABGwOPx6E+bqvTAC7vU0Tsgp8Omb142RV++kFUlBL++Abf+591K/XzdPh3r6JM0uNvprsuLddGkDIur8x9CkhchCQAA+NPRlm7d/ZdtemPfMUnS3LGp+vFnZ0XkFiUEP7fbo+e31ejhf+5VZXOXJGlsepzuvGyKPlGSG/YTmAlJXoQkAADgbx6PR//zbpV+8Ped6uxzKSbKprsvL9aX5o8N+zedCA0ej0dv7DumH720Wztq2iRJGQlO3bp4kv5tboGiImRSIyHJi5AEAAACpaq5S3f9ZZvePtAkSZo3Lk0//uwsFaZziDyss7WqRT96abfv5zLB6dD/XTBeN140TvHOyJrjRkjyIiQBAIBAcrs9evrdSq34xy519bkUF23X3VcU64vzilhVQkAdOtapn7y8R3+vqJUkRdtt+uL5Rbpl4USlxUdbXJ01zjQbWLqu9vrrr+vKK69UXt7gtLe//vWvw26/4YYbZBjGsMvll19uTbEAAABnwGYz9O/nF+mlWxdo3rg0dfW5dO//7tB1/+9fqvL2gAD+1NDWo++srtDiR9br7xW1MgzpmjljtPaOj+neK6dFbEA6G5aur3V2dmrWrFm68cYbdc0115z0PpdffrmefPJJ39dOpzNQ5QEAAIxYYXqc/uem8/XfG47owRd3652DTbr8sdf17U9O1Rc+UshxIBh1bT39+vX6A/rdm4fV3e+SJC0sztI3L5uiqbnsqDobloakK664QldcccVp7+N0OpWTkxOgigAAAEaPzWbo+gvG6uIpmfrmqm1693CzvrN6u16sqNODS2YoP5VeJZy7nn6XntpwRL94db9auvolSecVpujuy4s1b3y6xdWFpqDv1HrttdeUlZWl1NRULVy4UN///veVnn7q/9i9vb3q7e31fd3W1haIMgEAAE6pKD1ez37tfP3+7cN66OXdenP/MV3+2Bv6z09O1f+ZW8CqEkbE5fZo9XtH9egre3W0pVuSNDErQd+8bIounZbNz9U5CJrBDYZhaPXq1br66qt91z377LOKi4vTuHHjdODAAX37299WQkKC3nnnHdnt9pM+zn333af777//A9czuAEAAASDg40d+uaft6n8yHFJ0oLJmfrRkhnKTY61uDKECo/Ho3W7G/TQS3u0p75dkpSTFKNlH5+kJXPy5YiQcd4jEXLT7U4Wkt7v4MGDmjBhgtasWaNFixad9D4nW0kqKCggJAEAgKDhcnv05FuH9OOX96h3wK1Ep0P3XDlN15bm8+k/Tqv8SLMefHG3Nh4eDNlJMQ4tvWSirr9grGKiTr6IgBPONCQF/Xa7ocaPH6+MjAzt37//lCHJ6XQy3AEAAAQ1u83QVz86XpcUZ+nOVVv1XmWLvvXnbXqxolYrrpmpnOQYq0tEkNlb366HXtqjNbvqJUlOh01fvnCcvv6xCUqOi7K4uvATUiGpurpaTU1Nys3NtboUAACAczYhM0F/vvkC/b83DurhV/bq1T2N+vij63XfldN1zZwxrCpBNS3devSVvfrL5mq5PZLNkD5XVqBbF09ii6YfWRqSOjo6tH//ft/Xhw4d0pYtW5SWlqa0tDTdf//9WrJkiXJycnTgwAF961vf0sSJE3XZZZdZWDUAAMDosdsM/d+PTdBC76rS1upW3bFqq17cXqsffmaGspJYVYpELV19+uVrB/T7tw+rb8AtSbpsera+edkUTcxKtLi68GdpT9Jrr72mSy655APXX3/99Vq5cqWuvvpqvffee2ppaVFeXp4uvfRSPfDAA8rOzj7j5zjTfYcAAABWG3C59Zs3DuqxV/apz+VWcmyU7v/0dF01O49VpQjR3efSk28f0srXDqi9Z0CSNG9cmu66olhzClMtri70hdzgBn8hJAEAgFCzp65dd67aqoqjrZKkS6dl6wefmaHMRPquw9WAy60/barWT9fuVX3b4BCy4pxE3XVFsS6enElIHiWEJC9CEgAACEX9Lrd+vf6Afrp2n/pdHqXGRen+q0p05cxc3jCHEY/Ho5e21+nH/9yjg42dkqQxKbG687LJumrWGNls/LceTYQkL0ISAAAIZbtq23THn7ZqZ22bJOmKkhw9cHWJMhJYVQp1bx84ph+9tEdbq1okSWnx0brlkom67vxCOR2M8/YHQpIXIQkAAIS6fpdbv3z1gH6+bp8G3B6lxUfrgatK9MmZTPwNRTtqWvXQS3u0fm+jJCku2q6vXjRONy0Yr8QYxnn7EyHJi5AEAADCxY6aVt3xp63aXdcuSfrkzFw9cFWJ0uKjLa4MZ6KquUsP/3OP/rqlRpLksBn6wrxCfWPhJPrNAoSQ5EVIAgAA4aRvwK1frNunx187IJfbo4yEaH3/6hJdXsKqUrA61tGrX6zbr6f/dUT9rsG33lfOytMdH5+ssRnxFlcXWQhJXoQkAAAQjiqqW3Xnqq3aUz+4qvTpWXm6/9PTlcqqUtDo6B3Q/3vjoH77+kF19rkkSR+dlKG7Li9WyZhki6uLTIQkL0ISAAAIV70DLv187X6tXG+uKjn1w8+U6NLpOVaXFtH6Btz6n3cr9bO1+9TU2SdJmjEmWXdfUawLJ2ZYXF1kIyR5EZIAAEC421rVojtXbdW+hg5J0mfOG6PvXjlNKXGsKgWS2+3R89tq9PA/96qyuUuSNDY9TndeNkWfKMllnHcQICR5EZIAAEAk6Ol36bE1+/Sb1w/I7ZGyEp1acc0MLZqabXVpYc/j8ej1fcf0oxd3+0a1ZyQ4ddviSfo/cwsUZbdZXCFMhCQvQhIAAIgk71Ue152rtuqA92DSJXPyde+V05Qcy2hpf9ha1aIHX9ytdw42SZISnA7d/LHxuvGicYqLdlhcHd6PkORFSAIAAJGmp9+lR1/Zq9+8cVAej5Sd5NSDS2bqkilZVpcWNg42dugn/9yjf1TUSZKi7Tb9+/wiLb1kIiPZgxghyYuQBAAAIlX5kWbduWqbDh0bXFX6XFm+/vNT05TEgaUj1tDWo8fW7tMfN1bJ5fbIMAZ7wG7/+GTlp8ZZXR4+BCHJi5AEAAAiWXefSz/55x797q1D8nik3OQYLVs8WUmxbAU7W1urW/XkW4fU0++WJC0qztI3L5+i4hzeY4YKQpIXIQkAAEDaeLhZ31y1VYebuqwuJeTNKUzR3VdM1UfGpVldCs7SmWYDPkIAAACIAHPHpunFWxfoZ+v2aeOhZqvLCUlxToe+OK9QH5+WLcNgnHc4IyQBAABEiNhou+66vNjqMoCgx9B2AAAAABiCkAQAAAAAQxCSAAAAAGAIQhIAAAAADEFIAgAAAIAhCEkAAAAAMAQhCQAAAACGICQBAAAAwBCEJAAAAAAYgpAEAAAAAEMQkgAAAABgCEISAAAAAAxBSAIAAACAIQhJAAAAADAEIQkAAAAAhiAkAQAAAMAQhCQAAAAAGIKQBAAAAABDOKwuwN88Ho8kqa2tzeJKAAAAAFjJzARmRjiVsA9J7e3tkqSCggKLKwEAAAAQDNrb25WcnHzK2w3Ph8WoEOd2u1VTU6PExEQZhmFpLW1tbSooKFBVVZWSkpIsrQWRgZ85BBI/bwg0fuYQSPy8hQePx6P29nbl5eXJZjt151HYryTZbDbl5+dbXcYwSUlJ/M+FgOJnDoHEzxsCjZ85BBI/b6HvdCtIJgY3AAAAAMAQhCQAAAAAGIKQFEBOp1Pf/e535XQ6rS4FEYKfOQQSP28INH7mEEj8vEWWsB/cAAAAAABng5UkAAAAABiCkAQAAAAAQxCSAAAAAGAIQhIAAAAADEFICqDHH39cY8eOVUxMjObNm6d3333X6pIQhlasWKG5c+cqMTFRWVlZuvrqq7Vnzx6ry0IEefDBB2UYhm677TarS0GYOnr0qL74xS8qPT1dsbGxmjFjhjZt2mR1WQhTLpdL99xzj8aNG6fY2FhNmDBBDzzwgJh9Ft4ISQHyxz/+Ubfffru++93vavPmzZo1a5Yuu+wyNTQ0WF0awsz69eu1dOlSbdiwQa+88or6+/t16aWXqrOz0+rSEAE2btyoX//615o5c6bVpSBMHT9+XBdeeKGioqL04osvaufOnXr44YeVmppqdWkIUz/60Y+0cuVK/eIXv9CuXbv0ox/9SA899JB+/vOfW10a/IgR4AEyb948zZ07V7/4xS8kSW63WwUFBfrGN76hu+++2+LqEM4aGxuVlZWl9evXa8GCBVaXgzDW0dGhOXPm6Je//KW+//3va/bs2XrsscesLgth5u6779Zbb72lN954w+pSECE+9alPKTs7W0888YTvuiVLlig2NlZPPfWUhZXBn1hJCoC+vj6Vl5dr8eLFvutsNpsWL16sd955x8LKEAlaW1slSWlpaRZXgnC3dOlSffKTnxz2WgeMtr/97W8qKyvTtddeq6ysLJ133nn67W9/a3VZCGMXXHCB1q5dq71790qStm7dqjfffFNXXHGFxZXBnxxWFxAJjh07JpfLpezs7GHXZ2dna/fu3RZVhUjgdrt122236cILL1RJSYnV5SCMPfvss9q8ebM2btxodSkIcwcPHtTKlSt1++2369vf/rY2btyo/+//+/8UHR2t66+/3uryEIbuvvtutbW1qbi4WHa7XS6XSz/4wQ903XXXWV0a/IiQBISxpUuXavv27XrzzTetLgVhrKqqSrfeeqteeeUVxcTEWF0Owpzb7VZZWZl++MMfSpLOO+88bd++Xb/61a8ISfCLP/3pT3r66af1zDPPaPr06dqyZYtuu+025eXl8TMXxghJAZCRkSG73a76+vph19fX1ysnJ8eiqhDubrnlFr3wwgt6/fXXlZ+fb3U5CGPl5eVqaGjQnDlzfNe5XC69/vrr+sUvfqHe3l7Z7XYLK0Q4yc3N1bRp04ZdN3XqVP3lL3+xqCKEu29+85u6++679W//9m+SpBkzZujIkSNasWIFISmM0ZMUANHR0SotLdXatWt917ndbq1du1bz58+3sDKEI4/Ho1tuuUWrV6/WunXrNG7cOKtLQphbtGiRKioqtGXLFt+lrKxM1113nbZs2UJAwqi68MILP3Cswd69e1VUVGRRRQh3XV1dstmGv2W22+1yu90WVYRAYCUpQG6//XZdf/31Kisr00c+8hE99thj6uzs1Je//GWrS0OYWbp0qZ555hn97//+rxITE1VXVydJSk5OVmxsrMXVIRwlJiZ+oOctPj5e6enp9MJh1C1btkwXXHCBfvjDH+pzn/uc3n33Xf3mN7/Rb37zG6tLQ5i68sor9YMf/ECFhYWaPn263nvvPT3yyCO68cYbrS4NfsQI8AD6xS9+oR//+Meqq6vT7Nmz9bOf/Uzz5s2zuiyEGcMwTnr9k08+qRtuuCGwxSBiXXzxxYwAh9+88MILWr58ufbt26dx48bp9ttv10033WR1WQhT7e3tuueee7R69Wo1NDQoLy9Pn//853XvvfcqOjra6vLgJ4QkAAAAABiCniQAAAAAGIKQBAAAAABDEJIAAAAAYAhCEgAAAAAMQUgCAAAAgCEISQAAAAAwBCEJAAAAAIYgJAEAAADAEIQkAEDIOXz4sAzD0JYtW/z2HDfccIOuvvpqvz0+ACB4EZIAAAF3ww03yDCMD1wuv/zyM/r+goIC1dbWqqSkxM+VAgAikcPqAgAAkenyyy/Xk08+Oew6p9N5Rt9rt9uVk5Pjj7IAAGAlCQBgDafTqZycnGGX1NRUSZJhGFq5cqWuuOIKxcbGavz48frzn//s+973b7c7fvy4rrvuOmVmZio2NlaTJk0aFsAqKiq0cOFCxcbGKj09XV/72tfU0dHhu93lcun2229XSkqK0tPT9a1vfUsej2dYvW63WytWrNC4ceMUGxurWbNmDavpw2oAAIQOQhIAICjdc889WrJkibZu3arrrrtO//Zv/6Zdu3ad8r47d+7Uiy++qF27dmnlypXKyMiQJHV2duqyyy5TamqqNm7cqFWrVmnNmjW65ZZbfN//8MMP6/e//71+97vf6c0331Rzc7NWr1497DlWrFihP/zhD/rVr36lHTt2aNmyZfriF7+o9evXf2gNAIDQYnje/1EZAAB+dsMNN+ipp55STEzMsOu//e1v69vf/rYMw9DNN9+slStX+m47//zzNWfOHP3yl7/U4cOHNW7cOL333nuaPXu2Pv3pTysjI0O/+93vPvBcv/3tb3XXXXepqqpK8fHxkqR//OMfuvLKK1VTU6Ps7Gzl5eVp2bJl+uY3vylJGhgY0Lhx41RaWqq//vWv6u3tVVpamtasWaP58+f7HvurX/2qurq69Mwzz5y2BgBAaKEnCQBgiUsuuWRYCJKktLQ035+HhhHz61NNs/v617+uJUuWaPPmzbr00kt19dVX64ILLpAk7dq1S7NmzfIFJEm68MIL5Xa7tWfPHsXExKi2tlbz5s3z3e5wOFRWVubbcrd//351dXXp4x//+LDn7evr03nnnfehNQAAQgshCQBgifj4eE2cOHFUHuuKK67QkSNH9I9//EOvvPKKFi1apKVLl+onP/nJqDy+2b/097//XWPGjBl2mzlswt81AAACh54kAEBQ2rBhwwe+njp16invn5mZqeuvv15PPfWUHnvsMf3mN7+RJE2dOlVbt25VZ2en775vvfWWbDabpkyZouTkZOXm5upf//qX7/aBgQGVl5f7vp42bZqcTqcqKys1ceLEYZeCgoIPrQEAEFpYSQIAWKK3t1d1dXXDrnM4HL5hB6tWrVJZWZkuuugiPf3003r33Xf1xBNPnPSx7r33XpWWlmr69Onq7e3VCy+84AtU1113nb773e/q+uuv13333afGxkZ94xvf0L//+78rOztbknTrrbfqwQcf1KRJk1RcXKxHHnlELS0tvsdPTEzUnXfeqWXLlsntduuiiy5Sa2ur3nrrLSUlJen6668/bQ0AgNBCSAIAWOKll15Sbm7usOumTJmi3bt3S5Luv/9+Pfvss/qP//gP5ebm6n/+5380bdq0kz5WdHS0li9frsOHDys2NlYf/ehH9eyzz0qS4uLi9PLLL+vWW2/V3LlzFRcXpyVLluiRRx7xff8dd9yh2tpaXX/99bLZbLrxxhv1mc98Rq2trb77PPDAA8rMzNSKFSt08OBBpaSkaM6cOfr2t7/9oTUAAEIL0+0AAEHHMAytXr1aV199tdWlAAAiED1JAAAAADAEIQkAAAAAhqAnCQAQdNgJDgCwEitJAAAAADAEIQkAAAAAhiAkAQAAAMAQhCQAAAAAGIKQBAAAAABDEJIAAAAAYAhCEgAAAAAMQUgCAAAAgCH+fytoRJkQDZelAAAAAElFTkSuQmCC", "text/plain": [ "
" ] }, "metadata": {}, "output_type": "display_data" }, { "name": "stdout", "output_type": "stream", "text": [ "MoviePy - Building file videos/CartPole-v1.gif with imageio.\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ " \r" ] }, { "data": { "text/html": [ "
" ], "text/plain": [ "" ] }, "execution_count": 8, "metadata": {}, "output_type": "execute_result" } ], "source": [ "envname = 'CartPole-v1'\n", "env = gym.make(envname, render_mode=\"rgb_array_list\")\n", "recorder = GymRecorder(env)\n", "\n", "returns = random_interaction(env, 10, recorder)\n", "\n", "plt.figure(figsize=(10, 6))\n", "plt.plot(returns)\n", "plt.xlabel(\"Episodes\")\n", "plt.ylabel(\"Return\")\n", "plt.show()\n", "\n", "video = \"videos/\" + envname + \".gif\"\n", "recorder.make_video(video)\n", "ipython_display(video)" ] }, { "cell_type": "code", "execution_count": 9, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "MoviePy - Building file videos/CarRacing-v2.gif with imageio.\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ " \r" ] }, { "data": { "text/html": [ "
" ], "text/plain": [ "" ] }, "execution_count": 9, "metadata": {}, "output_type": "execute_result" } ], "source": [ "envname = 'CarRacing-v2'\n", "env = gym.make(envname, render_mode=\"rgb_array_list\")\n", "recorder = GymRecorder(env)\n", "\n", "returns = random_interaction(env, 1, recorder)\n", "\n", "video = \"videos/\" + envname + \".gif\"\n", "recorder.make_video(video)\n", "ipython_display(video)" ] }, { "cell_type": "code", "execution_count": 10, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "MoviePy - Building file videos/Taxi-v3.gif with imageio.\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ " \r" ] }, { "data": { "text/html": [ "
" ], "text/plain": [ "" ] }, "execution_count": 10, "metadata": {}, "output_type": "execute_result" } ], "source": [ "envname = 'Taxi-v3'\n", "env = gym.make(envname, render_mode=\"rgb_array_list\")\n", "recorder = GymRecorder(env)\n", "\n", "returns = random_interaction(env, 1, recorder)\n", "\n", "video = \"videos/\" + envname + \".gif\"\n", "recorder.make_video(video)\n", "ipython_display(video)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Creating your own environment\n", "\n", "### Random environment\n", "\n", "You can create your own environment using the gym interface:\n", "\n", "\n", "\n", "Here is an example of a dummy environment with discrete states and actions, where the transition probabilities and rewards are completely random:" ] }, { "cell_type": "code", "execution_count": 11, "metadata": { "id": "B5u7Z8tjoGr3" }, "outputs": [], "source": [ "class RandomEnv(gym.Env):\n", " \"Random discrete environment that does nothing.\"\n", " \n", " metadata = {\"render_modes\": [\"ansi\"], \"render_fps\": 1}\n", "\n", " def __init__(self, nb_states, nb_actions, max_episode_steps=10, render_mode=\"ansi\"):\n", "\n", " self.nb_states = nb_states\n", " self.nb_actions = nb_actions\n", " self.max_episode_steps = max_episode_steps\n", " self.render_mode = render_mode\n", "\n", " # State space, can be discrete or continuous.\n", " self.observation_space = gym.spaces.Discrete(nb_states)\n", " \n", " # Action space, can be discrete or continuous.\n", " self.action_space = gym.spaces.Discrete(nb_actions) \n", "\n", " # Reset\n", " self.reset()\n", "\n", "\n", " def reset(self, seed=None, options=None):\n", "\n", " # Re-initialize time\n", " self.current_step = 0\n", " \n", " # Sample one state randomly \n", " self.state = self.observation_space.sample()\n", " \n", " return self.state, info\n", "\n", " def step(self, action):\n", "\n", " # Random transition to another state\n", " self.state = self.observation_space.sample() \n", " \n", " # Random reward\n", " reward = np.random.uniform(0, 1, 1)[0] \n", " \n", " # Terminate the episode after 10 steps\n", " terminal = False \n", " truncated = False\n", "\n", " self.current_step +=1\n", " if self.current_step % self.max_episode_steps == 0:\n", " truncated = True \n", "\n", " info = {} # No info\n", "\n", " return self.state, reward, terminal, truncated, info\n", "\n", "\n", " def render(self):\n", " if self.render_mode == \"ansi\":\n", " description = \"Step \" + str(self.current_step) + \": state \" + str(self.state)\n", " return description\n", " return None\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The different methods should be quite self-explanatory.\n", "\n", "`metadata` defines which render modes are available for this environment (here only the text mode \"ansi\").\n", "\n", "The constructor accepts the size of the state and action spaces as arguments, the duration of the episode and the render mode. \n", "\n", "`reset()` samples an initial state randomly.\n", "\n", "`step()` ignores the action, samples a new state and a reward, and truncates an episode after `max_episode_steps`.\n", "\n", "`render()` returns a string with the current state.\n", "\n", "**Q:** Interact with the random environment for a couple of episodes.\n", "\n", "As the mode is `ansi` (text-based), you will need to print the string returned by `render()` after each step:\n", "\n", "```python\n", "while not done:\n", "\n", " action = env.action_space.sample()\n", " \n", " next_state, reward, terminal, truncated, info = env.step(action)\n", "\n", " print(env.render())\n", "```" ] }, { "cell_type": "code", "execution_count": 12, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Episode 0\n", "Step 0: state 9\n", "Step 1: state 8\n", "Step 2: state 3\n", "Step 3: state 0\n", "Step 4: state 5\n", "Step 5: state 4\n", "Step 6: state 3\n", "Step 7: state 0\n", "Step 8: state 6\n", "Step 9: state 8\n", "Step 10: state 4\n", "Return of the episode: 5.007583939302424\n", "----------\n", "Episode 1\n", "Step 0: state 6\n", "Step 1: state 4\n", "Step 2: state 0\n", "Step 3: state 4\n", "Step 4: state 4\n", "Step 5: state 5\n", "Step 6: state 2\n", "Step 7: state 6\n", "Step 8: state 2\n", "Step 9: state 2\n", "Step 10: state 2\n", "Return of the episode: 3.3300428387017016\n", "----------\n" ] } ], "source": [ "# Create the environment\n", "env = RandomEnv(nb_states=10, nb_actions=4)\n", "\n", "# Sample episodes\n", "for episode in range(2):\n", "\n", " print(\"Episode\", episode)\n", "\n", " # Sample the initial state\n", " state, info = env.reset()\n", "\n", " # Render the initial state\n", " print(env.render())\n", "\n", " # Episode\n", " return_episode = 0.0\n", " done = False\n", " while not done:\n", " # Select an action randomly\n", " action = env.action_space.sample()\n", " \n", " # Sample a single transition\n", " next_state, reward, terminal, truncated, info = env.step(action)\n", " \n", " # Go in the next state\n", " state = next_state\n", "\n", " # Update return\n", " return_episode += reward\n", "\n", " # Render the current state\n", " print(env.render())\n", "\n", " # End of the episode\n", " done = terminal or truncated\n", " \n", " print(\"Return of the episode:\", return_episode)\n", " print('-'*10)\n" ] }, { "cell_type": "markdown", "metadata": { "id": "m7JjS7uwoGse" }, "source": [ "### Recycling robot\n", "\n", "**Q:** Create a `RecyclingRobot` gym-like environment using last week's exercise.\n", "\n", "The parameters `alpha`, `beta`, `r_wait` and `r_search` should be passed to the constructor of the environment and saved as attributes.\n", "\n", "The state space is discrete, with two states `high` and `low` which will have indices 0 and 1. The three discrete actions `search`, `wait` and `recharge` have indices 0, 1, and 2.\n", "\n", "The initial state of the MDP (`reset()`) should be the high state.\n", "\n", "The `step()` should generate transitions according to the dynamics of the MDP. Depending on the current state and the chosen action, make a transition to another state. For the actions `search` and `wait`, sample the reward from the normal distribution with mean `r_search` (resp. `r_wait`) and variance 0.5. \n", "\n", "If the random agent selects `recharge` in `high`, do nothing (next state is high, reward is 0).\n", "\n", "Rendering is just printing the current state. There is nothing to close, so you do not even need to redefine the function.\n", "\n", "Although the recycling robot is a continuing task, limit the number of steps per episode to 10, as in the the previous random environment.\n", "\n", "Interact randomly with the MDP for several episodes and observe the returns. " ] }, { "cell_type": "code", "execution_count": 13, "metadata": { "id": "HRhkICMBoGsf" }, "outputs": [], "source": [ "class RecyclingRobot(gym.Env):\n", " \"Recycling robot environment.\"\n", "\n", " metadata = {\"render_modes\": [\"ansi\"], \"render_fps\": 1}\n", "\n", " def __init__(self, alpha, beta, r_search, r_wait, max_episode_steps=10, render_mode=\"ansi\"):\n", " \n", " # Store parameters\n", " self.alpha = alpha\n", " self.beta = beta\n", " self.r_search = r_search\n", " self.r_wait = r_wait\n", " self.max_episode_steps = max_episode_steps\n", " self.render_mode = render_mode\n", " \n", " # State space, can be discrete or continuous.\n", " self.observation_space = gym.spaces.Discrete(2) \n", " self.states = ['high', 'low']\n", " \n", " # Action space, can be discrete or continuous.\n", " self.action_space = gym.spaces.Discrete(3) \n", " self.actions = ['search', 'wait', 'recharge'] \n", "\n", " # Reset\n", " self.reset()\n", " \n", " def reset(self, seed=None, options=None):\n", " \n", " # Re-initialize time\n", " self.current_step = 0\n", "\n", " # Start in the high state\n", " self.state = 0\n", " \n", " return self.state, {}\n", " \n", " def step(self, action):\n", " \n", " if self.state == 0: # high\n", " if action == 0: # search\n", " p = np.random.rand()\n", " if p < self.alpha:\n", " self.state = 0 # high\n", " else:\n", " self.state = 1 # low\n", " self.reward = float(np.random.normal(self.r_search, 0.5, 1))\n", " elif action == 1: # wait\n", " self.state = 0 # high\n", " self.reward = float(np.random.normal(self.r_wait, 0.5, 1))\n", " elif action == 2: # recharge\n", " self.state = 0 # high\n", " self.reward = 0.0\n", " \n", " elif self.state == 1: # low\n", " if action == 0: # search\n", " p = np.random.rand()\n", " if p < self.beta:\n", " self.state = 1 # low\n", " self.reward = float(np.random.normal(self.r_search, 0.5, 1))\n", " else:\n", " self.state = 0 # high\n", " self.reward = -3.0\n", " elif action == 1: # wait\n", " self.state = 1 # low\n", " self.reward = float(np.random.normal(self.r_wait, 0.5, 1))\n", " elif action == 2: # recharge\n", " self.state = 0 # high\n", " self.reward = 0.0\n", " \n", " terminal = False\n", " truncated = False\n", " self.current_step +=1\n", " if self.current_step % self.max_episode_steps == 0:\n", " truncated = True \n", "\n", " info = {} # No info\n", "\n", " return self.state, self.reward, terminal, truncated, info\n", "\n", " def render(self):\n", " \n", " if self.render_mode == \"ansi\":\n", " description = \"Step \" + str(self.current_step) + \": state \" + self.states[self.state]\n", " return description\n", " \n", " return None" ] }, { "cell_type": "code", "execution_count": 14, "metadata": { "id": "KmYJr3nVoGsl" }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Episode: 0\n", "Step 0: state high\n", "high + search -> low : 5.824371578099047\n", "Step 1: state low\n", "low + wait -> low : 2.9450113775797497\n", "Step 2: state low\n", "low + recharge -> high : 0.0\n", "Step 3: state high\n", "high + wait -> high : 1.19425009588391\n", "Step 4: state high\n", "high + wait -> high : 1.1899989446304127\n", "Step 5: state high\n", "high + wait -> high : 1.9428395745820943\n", "Step 6: state high\n", "high + recharge -> high : 0.0\n", "Step 7: state high\n", "high + wait -> high : 2.3322524158366855\n", "Step 8: state high\n", "high + search -> low : 5.892699540496995\n", "Step 9: state low\n", "low + wait -> low : 1.4083503027532602\n", "Step 10: state low\n", "Return of the episode: 22.729773829862154\n", "----------\n", "Episode: 1\n", "Step 0: state high\n", "high + search -> low : 5.437227719900913\n", "Step 1: state low\n", "low + wait -> low : 1.845777038847942\n", "Step 2: state low\n", "low + wait -> low : 2.489897166433866\n", "Step 3: state low\n", "low + wait -> low : 2.4018187411534972\n", "Step 4: state low\n", "low + recharge -> high : 0.0\n", "Step 5: state high\n", "high + wait -> high : 2.402843421805779\n", "Step 6: state high\n", "high + wait -> high : 1.8639466909583808\n", "Step 7: state high\n", "high + recharge -> high : 0.0\n", "Step 8: state high\n", "high + recharge -> high : 0.0\n", "Step 9: state high\n", "high + search -> low : 5.871762172601576\n", "Step 10: state low\n", "Return of the episode: 22.313272951701954\n", "----------\n", "Episode: 2\n", "Step 0: state high\n", "high + search -> low : 5.653305608734461\n", "Step 1: state low\n", "low + wait -> low : 1.260637353066509\n", "Step 2: state low\n", "low + recharge -> high : 0.0\n", "Step 3: state high\n", "high + recharge -> high : 0.0\n", "Step 4: state high\n", "high + recharge -> high : 0.0\n", "Step 5: state high\n", "high + search -> low : 6.179745460703129\n", "Step 6: state low\n", "low + search -> high : -3.0\n", "Step 7: state high\n", "high + wait -> high : 1.7318126590102751\n", "Step 8: state high\n", "high + search -> low : 6.189826308355821\n", "Step 9: state low\n", "low + recharge -> high : 0.0\n", "Step 10: state high\n", "Return of the episode: 18.015327389870194\n", "----------\n", "Episode: 3\n", "Step 0: state high\n", "high + wait -> high : 0.3116973832342118\n", "Step 1: state high\n", "high + wait -> high : 1.5890528296212245\n", "Step 2: state high\n", "high + recharge -> high : 0.0\n", "Step 3: state high\n", "high + search -> low : 6.1770715710684865\n", "Step 4: state low\n", "low + wait -> low : 1.9950025916418077\n", "Step 5: state low\n", "low + wait -> low : 2.288732539726356\n", "Step 6: state low\n", "low + search -> high : -3.0\n", "Step 7: state high\n", "high + recharge -> high : 0.0\n", "Step 8: state high\n", "high + search -> low : 6.114749510159829\n", "Step 9: state low\n", "low + recharge -> high : 0.0\n", "Step 10: state high\n", "Return of the episode: 15.476306425451915\n", "----------\n", "Episode: 4\n", "Step 0: state high\n", "high + search -> high : 5.677653252270961\n", "Step 1: state high\n", "high + search -> low : 6.270961762981069\n", "Step 2: state low\n", "low + search -> high : -3.0\n", "Step 3: state high\n", "high + recharge -> high : 0.0\n", "Step 4: state high\n", "high + recharge -> high : 0.0\n", "Step 5: state high\n", "high + recharge -> high : 0.0\n", "Step 6: state high\n", "high + wait -> high : 2.167628510196825\n", "Step 7: state high\n", "high + recharge -> high : 0.0\n", "Step 8: state high\n", "high + recharge -> high : 0.0\n", "Step 9: state high\n", "high + search -> high : 6.00958550093353\n", "Step 10: state high\n", "Return of the episode: 17.125829026382384\n", "----------\n", "Episode: 5\n", "Step 0: state high\n", "high + recharge -> high : 0.0\n", "Step 1: state high\n", "high + wait -> high : 1.0072443461744696\n", "Step 2: state high\n", "high + wait -> high : 1.1770716510613748\n", "Step 3: state high\n", "high + recharge -> high : 0.0\n", "Step 4: state high\n", "high + search -> high : 6.498953899017299\n", "Step 5: state high\n", "high + search -> high : 6.084695387366945\n", "Step 6: state high\n", "high + wait -> high : 2.523957175613759\n", "Step 7: state high\n", "high + recharge -> high : 0.0\n", "Step 8: state high\n", "high + wait -> high : 2.2205143495052986\n", "Step 9: state high\n", "high + wait -> high : 1.565489159288247\n", "Step 10: state high\n", "Return of the episode: 21.077925968027394\n", "----------\n", "Episode: 6\n", "Step 0: state high\n", "high + wait -> high : 2.909574962101888\n", "Step 1: state high\n", "high + recharge -> high : 0.0\n", "Step 2: state high\n", "high + wait -> high : 1.5723174882068984\n", "Step 3: state high\n", "high + recharge -> high : 0.0\n", "Step 4: state high\n", "high + wait -> high : 1.4266076247041248\n", "Step 5: state high\n", "high + wait -> high : 2.5804443044435845\n", "Step 6: state high\n", "high + recharge -> high : 0.0\n", "Step 7: state high\n", "high + recharge -> high : 0.0\n", "Step 8: state high\n", "high + wait -> high : 0.9842375744951053\n", "Step 9: state high\n", "high + recharge -> high : 0.0\n", "Step 10: state high\n", "Return of the episode: 9.4731819539516\n", "----------\n", "Episode: 7\n", "Step 0: state high\n", "high + wait -> high : 1.3162358550374915\n", "Step 1: state high\n", "high + search -> high : 5.6548000306144\n", "Step 2: state high\n", "high + wait -> high : 2.263322213479556\n", "Step 3: state high\n", "high + wait -> high : 1.3881496262912796\n", "Step 4: state high\n", "high + search -> high : 6.636954022870793\n", "Step 5: state high\n", "high + search -> high : 6.610354453744501\n", "Step 6: state high\n", "high + search -> high : 4.750619854167663\n", "Step 7: state high\n", "high + recharge -> high : 0.0\n", "Step 8: state high\n", "high + search -> high : 5.5069516395763545\n", "Step 9: state high\n", "high + search -> high : 5.805846615859272\n", "Step 10: state high\n", "Return of the episode: 39.93323431164131\n", "----------\n", "Episode: 8\n", "Step 0: state high\n", "high + search -> high : 5.695885508830509\n", "Step 1: state high\n", "high + recharge -> high : 0.0\n", "Step 2: state high\n", "high + wait -> high : 2.3396804906667272\n", "Step 3: state high\n", "high + recharge -> high : 0.0\n", "Step 4: state high\n", "high + wait -> high : 2.0632140879809655\n", "Step 5: state high\n", "high + wait -> high : 2.207471429939306\n", "Step 6: state high\n", "high + wait -> high : 1.1333787694607529\n", "Step 7: state high\n", "high + recharge -> high : 0.0\n", "Step 8: state high\n", "high + search -> low : 6.595384487237137\n", "Step 9: state low\n", "low + wait -> low : 1.1887366998910425\n", "Step 10: state low\n", "Return of the episode: 21.223751474006438\n", "----------\n", "Episode: 9\n", "Step 0: state high\n", "high + search -> low : 6.317901393973506\n", "Step 1: state low\n", "low + search -> high : -3.0\n", "Step 2: state high\n", "high + recharge -> high : 0.0\n", "Step 3: state high\n", "high + search -> low : 6.372994582022667\n", "Step 4: state low\n", "low + wait -> low : 2.0416319726753738\n", "Step 5: state low\n", "low + wait -> low : 2.6438194473841232\n", "Step 6: state low\n", "low + search -> high : -3.0\n", "Step 7: state high\n", "high + search -> high : 5.970627774061589\n", "Step 8: state high\n", "high + recharge -> high : 0.0\n", "Step 9: state high\n", "high + recharge -> high : 0.0\n", "Step 10: state high\n", "Return of the episode: 17.34697517011726\n", "----------\n" ] } ], "source": [ "# Create the environment\n", "env = RecyclingRobot(alpha=0.3, beta=0.2, r_search=6, r_wait=2)\n", "\n", "# Sample episodes\n", "for episode in range(10):\n", "\n", " print(\"Episode:\", episode)\n", "\n", " # Sample the initial state\n", " state, info = env.reset()\n", " print(env.render())\n", "\n", " return_episode = 0.0\n", " done = False\n", " while not done:\n", "\n", " # Select an action randomly\n", " action = env.action_space.sample()\n", " \n", " # Sample a single transition\n", " next_state, reward, terminal, truncated, info = env.step(action)\n", " \n", " print(env.states[state], \"+\", env.actions[action], \"->\", env.states[next_state], \":\", reward)\n", " \n", " # Go in the next state\n", " state = next_state\n", "\n", " # Update return\n", " return_episode += reward\n", "\n", " # Render the current state\n", " print(env.render())\n", "\n", " # End of the episode\n", " done = terminal or truncated\n", " \n", " print(\"Return of the episode:\", return_episode)\n", " print('-'*10)" ] }, { "cell_type": "markdown", "metadata": { "id": "udFjupHmoGso" }, "source": [ "### Random agent\n", "\n", "To be complete, let's implement the random agent as a class. The class should look like:\n", "\n", "```python\n", "class RandomAgent:\n", " \"\"\"\n", " Random agent exploring uniformly the environment.\n", " \"\"\"\n", " \n", " def __init__(self, env):\n", " self.env = env\n", " \n", " def act(self, state):\n", " \"Returns a random action by sampling the action space.\"\n", " action = # TODO\n", " return action\n", " \n", " def update(self, state, action, reward, next_state):\n", " \"Updates the agent using the transition (s, a, r, s').\"\n", " pass\n", " \n", " def train(self, nb_episodes, render=False):\n", " \"Runs the agent on the environment for nb_episodes. Returns the list of obtained returns.\"\n", " \n", " # List of returns\n", " returns = []\n", "\n", " # TODO\n", " \n", " return returns\n", "```\n", "\n", "The environment is passed to the constructor. `act(state)` should sample a random action. `update(state, action, reward, next_state)` does nothing for the random agent (`pass` is a Python command doing nothing), but we will implement it in the next exercises. \n", "\n", "`train(nb_episodes, render)` implements the interaction loop between the agent and the environment for a fixed number of episodes. It should return the list of obtained returns. `render` defines whether you print the state at each step or not.\n", "\n", "**Q:** Implement the random agent and have it interact with the environment for a fixed number of episodes." ] }, { "cell_type": "code", "execution_count": 15, "metadata": { "id": "or3R2eIpoGsp" }, "outputs": [], "source": [ "class RandomAgent:\n", " \"\"\"\n", " Random agent exploring uniformly the environment.\n", " \"\"\"\n", " \n", " def __init__(self, env):\n", " self.env = env\n", " \n", " def act(self, state):\n", " \"Returns a random action by sampling the action space.\"\n", " return self.env.action_space.sample()\n", " \n", " def update(self, state, action, reward, next_state):\n", " \"Updates the agent using the transition (s, a, r, s').\"\n", " pass\n", " \n", " def train(self, nb_episodes, render=False):\n", " \"Runs the agent on the environment for nb_episodes. Returns the list of obtained rewards.\"\n", " # List of returns\n", " returns = []\n", "\n", " for episode in range(nb_episodes):\n", " if render:\n", " print(\"Episode:\", episode)\n", "\n", " # Sample the initial state\n", " state, info = self.env.reset()\n", " if render:\n", " print(self.env.render())\n", "\n", " return_episode = 0.0\n", " done = False\n", " while not done:\n", "\n", " # Select an action randomly\n", " action = self.act(state)\n", " \n", " # Sample a single transition\n", " next_state, reward, terminal, truncated, info = self.env.step(action)\n", " \n", " # Go in the next state\n", " state = next_state\n", "\n", " # Update return\n", " return_episode += reward\n", "\n", " # Render the current state\n", " if render:\n", " print(env.render())\n", "\n", " # End of the episode\n", " done = terminal or truncated\n", " \n", " returns.append(return_episode)\n", "\n", " return returns" ] }, { "cell_type": "code", "execution_count": 16, "metadata": { "id": "5eN9ChPaoGst" }, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAA0oAAAINCAYAAAAA8I+NAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjguMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8g+/7EAAAACXBIWXMAAA9hAAAPYQGoP6dpAACEuElEQVR4nOzdd3iUZdo28POZmp6Q3hsBkhBa6L2XICCKKMWCXResu/rJ7uruvquya19XRUVXVECKiAJSpBepAQIJJXQI6YVM+mTK8/0xmSgISEIy95TzdxxzHC/JZOZk3zDONff1XJcky7IMIiIiIiIiaqQQHYCIiIiIiMjesFAiIiIiIiK6CgslIiIiIiKiq7BQIiIiIiIiugoLJSIiIiIioquwUCIiIiIiIroKCyUiIiIiIqKrsFAiIiIiIiK6ikp0gNZmNpuRl5cHb29vSJIkOg4REREREQkiyzIqKysRHh4OheLGZ0ZOXyjl5eUhKipKdAwiIiIiIrITOTk5iIyMvOF9nL5Q8vb2BmD5H8PHx0dwGiIiIiIiEqWiogJRUVGNNcKNOH2hZG238/HxYaFEREREREQ3dUkOhzkQERERERFdhYUSERERERHRVVgoERERERERXYWFEhERERER0VVYKBEREREREV2FhRIREREREdFVhBZKc+fORefOnRtHd/ft2xdr164FAJSVleGpp55Chw4d4O7ujujoaDz99NPQ6XQiIxMRERERkQsQukcpMjIS//rXv9CuXTvIsowvv/wSt99+Ow4dOgRZlpGXl4e33noLycnJuHDhAp544gnk5eXh22+/FRmbiIiIiIicnCTLsiw6xK/5+/vjzTffxMMPP/yb7y1btgz33nsvqquroVLdXI1XUVEBX19f6HQ6LpwlIiIiInJhTakNhJ4o/ZrJZMKyZctQXV2Nvn37XvM+1r/QjYokvV4PvV7f+OeKiooWz0pERERERM5N+DCHzMxMeHl5QavV4oknnsCKFSuQnJz8m/uVlJTgn//8Jx577LEbPt6cOXPg6+vbeIuKimqt6ERERERE5KSEt97V19fj4sWL0Ol0+Pbbb/HZZ59h27ZtVxRLFRUVGDlyJPz9/bFy5Uqo1errPt61TpSioqLYekdERERE5OKa0nonvFC62ogRI9C2bVt88sknAIDKykqMHj0aHh4eWL16Ndzc3Jr0eLxGiYiIiIiIgKbVBsJb765mNpsbT4QqKiowatQoaDQarFy5sslFEhERERERUXMIHeYwe/ZspKWlITo6GpWVlVi0aBG2bt2K9evXNxZJNTU1WLBgASoqKhoHMwQFBUGpVIqMTkRERERETkxooVRUVIT7778f+fn58PX1RefOnbF+/XqMHDkSW7duxd69ewEACQkJV/zcuXPnEBsbKyAxERERERG5Aru7Rqml8RolImqqwoo6VOuNiA/yEh2FiIiIWpBDX6NERCSS0WTGXR/vQtp/duDS5RrRcYiIiEgQFkpERL+y+2wpcspqoTeasSYzX3QcIiIiEoSFEhHRr6zMyGv8v9dkFghMQkRERCKxUCIiaqA3mrDu6C/FUUZOOXLLawUmIiIiIlFYKBERNdiWXYzKOiNCfdzQM7YNAGBdFk+ViIiIXBELJSKiBisPW9ruxnUOw22dwgAAa3mdEhERkUtioUREBKBab8TG44UAgAldwzEmxVIopV+4jAJdnchoREREJAALJSIiABuPF6LOYEZMgAc6Rfgi1NcN3WMs7Xfrj7L9joiIyNWwUCIiArCqoe1uQpdwSJIEAEhLCQUAjgknIiJyQSyUiMjlldfUY9vJYgCWQskqreE6pX3ny1BcqReSjYiIiMRgoURELm/90QIYTDISQ73RLsS78esRfu7oEuUHWWb7HRERkathoURELs867W78r06TrMY2tN+tzWL7HRERkSthoURELq2osg67z5QCuLLtziqtYfrdnrNlKK1i+x0REZGrYKFERC7txyP5MMtAt2g/RPl7/Ob70QEeSInwgcksY8OxQgEJiYiISAQWSkTk0qzT7sZ3/u1pkpX1VGlNFq9TIiIichUslIjIZeWU1eDgxXIoJGBc57Dr3s86JnzX6RKU19TbKh4REREJxEKJiFzWqiOW06Q+8QEI9nG77v3ig7yQGOoNI9vviIiIXAYLJSJyWasOWybZXWva3dXGNuxUWsf2OyIiIpfAQomIXNLpokocz6+AWik1ttbdyNhOlvvsOFWCijpDa8cjIiIiwVgoEZFLWplhabsb1C4Ifh6a371/QrA32gV7od5kxubjRa0dj4iIiARjoURELkeW5cYlsxO6/n7bnVVaQ/vdmkwunyUiInJ2LJSIyOVk5VbgfGkN3NQKjEgKuemfs7bfbT1ZjCq9sbXiERERkR1goURELmfl4VwAwPCkEHhqVTf9cx1CvBEX6Il6oxlbTrD9joiIyJmxUCIil2I2y1h9xNI6N+Empt39miT9MvhhbRbb74iIiJwZCyUicinpFy4jX1cHb60Kg9sHNfnnrWPCt5woRk092++IiIicFQslInIp1ra70SmhcFMrm/zzHcN9EOXvjlqDCduyi1s6HhEREdkJFkpE5DIMJjPWZFoWxja17c5KkiSMTWmYfsfls0RERE6LhRIRuYyfT5egrLoeAZ4a9Gsb0OzHsY4J33y8EHUGU0vFIyIiIjvCQomIXMaqw5YBDGM7hUGlbP7LX5dIX4T7uqG63oTtJ9l+R0RE5IxYKBGRS6gzmPDT0Ya2uyYsmb0WSZIaT5XWsv2OiIjIKbFQIiKXsDW7CJV6I8J93dA9us0tP551+ezGY4XQG9l+R0RE5GxYKBGRS7C23Y3rEg6FQrrlx+sW1QYhPlpU6o34+XTJLT8eERER2RcWSkTk9Kr0Rmw8Xgig+dPurqZQSEizTr/LZPsdERGRs2GhREROb8OxAuiNZsQHeqJjuE+LPW5aiqX97qejBag3mlvscYmIiEg8FkpE5PRWZuQBAMZ3CYck3XrbnVWPWH8EemlRUWfE7rOlLfa4REREJB4LJSJyaper67HjlOUaovEt1HZnpVRIGJMSAgBYm5nfoo9NREREYrFQIiKntjarAEazjOQwHyQEe7X4449tuE5p/dECGE1svyMiInIWLJSIyKmtPJwL4NZ3J11Przh/+HtqcLnGgL3nylrlOYiIiMj2WCgRkdMqrKhrLF7GdQ5rledQKRUY3dHSfreG7XdEREROg4USETmt1UfyIctA95g2iGzj0WrPk/ar9juTWW615yEiIiLbYaFERE5r5WHLtLuW2p10PX3bBsDXXY2SqnrsP8/2OyIiImfAQomInNKF0moczimHQgLGdmqdtjsrtVKBUcmcfkdERORMWCgRkVNafcRSsPRrG4ggb22rP5+1GFubVQAz2++IiIgcHgslInJK1iWzrd12Z9UvIQDebioUVepx8OJlmzwnERERtR4WSkTkdLILKpFdWAm1UsLolFCbPKdWpcTIJOv0uwKbPCcRERG1HhZKROR0VjUMcRjcPhi+7mqbPW9aQ/vduqx8yDLb74iIiBwZCyUiciqyLP8y7a6Vlsxez8B2gfDUKJGnq8PhSzqbPjcRERG1LKGF0ty5c9G5c2f4+PjAx8cHffv2xdq1axu/X1dXh5kzZyIgIABeXl6YNGkSCgsLBSYmInt3+JIOF8tq4K5WYkRSsE2f202txPAkTr8jIiJyBkILpcjISPzrX//CgQMHkJ6ejmHDhuH222/H0aNHAQDPPfccVq1ahWXLlmHbtm3Iy8vDnXfeKTIyEdk56xCHkckh8NCobP78YztZrolaw/Y7IiIih2b7dxG/Mn78+Cv+/Nprr2Hu3LnYs2cPIiMj8fnnn2PRokUYNmwYAOCLL75AUlIS9uzZgz59+oiITER2zGSWsfqIpVAab6Npd1cb3D4Y7molcspqcTSvAikRvkJyEBER0a2xm2uUTCYTFi9ejOrqavTt2xcHDhyAwWDAiBEjGu+TmJiI6Oho7N69+7qPo9frUVFRccWNiFzDvnNlKKrUw8dNhUHtA4VkcNcoMTQxCACwhu13REREDkt4oZSZmQkvLy9otVo88cQTWLFiBZKTk1FQUACNRgM/P78r7h8SEoKCguuP3p0zZw58fX0bb1FRUa38NyAie2Ed4pCWEgatSiksR1qKZfrdmky23xERETkq4YVShw4dkJGRgb179+LJJ5/EAw88gGPHjjX78WbPng2dTtd4y8nJacG0RGSvDCYz1mZZTnBEtd1ZDU0MhlalwPnSGpwoqBSahYiIiJpH6DVKAKDRaJCQkAAA6N69O/bv34///Oc/uOeee1BfX4/y8vIrTpUKCwsRGnr9BZJarRZarba1YxORndl5qgTlNQYEemnRt22A0CxeWhUGtw/CT8cKsTYzH0lhPkLzEBERUdMJP1G6mtlshl6vR/fu3aFWq7Fp06bG72VnZ+PixYvo27evwIREZI+sbXfjOodBqZAEpwHGNiyfXZN1/VZhIiIisl9CT5Rmz56NtLQ0REdHo7KyEosWLcLWrVuxfv16+Pr64uGHH8bzzz8Pf39/+Pj44KmnnkLfvn058Y6IrlBbb8JPRy0Fiei2O6thScHQKBU4XVSFU4WVaBfiLToSERERNYHQQqmoqAj3338/8vPz4evri86dO2P9+vUYOXIkAODdd9+FQqHApEmToNfrMXr0aHz00UciIxORHdqSXYTqehMi/NyRGu0nOg4AwMdNjYHtArHpRBHWZBbgGRZKREREDkVoofT555/f8Ptubm748MMP8eGHH9ooERE5IuuS2fFdwiFJ4tvurNI6hWHTiSKszcrHMyPaiY5DRERETWB31ygRETVFRZ0Bm7OLAAAT7KTtzmpkUghUCgknCipxprhKdBwiIiJqAhZKROTQNhwtRL3RjLZBnkgKs6/2Nl8PNfonWBbfruNQByIiIofCQomIHJp12t2ELhF21XZnNbaTZZ3Bmsx8wUmIiIioKVgoEZHDKq3SY+fpEgDAhK721XZnNTI5FEqFhKN5FbhQWi06DhEREd0kFkpE5LDWZBXAZJbRKcIXcYGeouNck7+nBn3jLQtw17L9joiIyGGwUCIih7XqsHXaXZjgJDeW1tB+t5btd0RERA6DhRIROaR8XS32ny8DAIzrbJ9td1ajkkOhkIDDl3S4dLlGdBwiIiK6CSyUiMghrT6cD1kGesX6I9zPXXScGwry1qJXnD8ATr8jIiJyFCyUiMghrTriGG13VmM7WXJy+h0REZFjYKFERA7nXEk1jlzSQamQGgsQeze6YygkCTh4sRz5ulrRcYiIiOh3sFAiIodjHeLQPyEQAV5awWluToiPG3rEtAHA9jsiIiJHwEKJiByKLMu/WjJr30McrpaWYjn9WpvJQomIiMjesVAiIodyoqASp4uqoFEpMKpjiOg4TTImxTImfP+FMhRV1AlOQ0RERDfCQomIHIr1NGlohyD4uKkFp2macD93dIv2gywD64/yVImIiMiesVAiIochy3Lj9UkTukQITtM8Y1Os0+9YKBEREdkzFkpE5DAO5ZTj0uVaeGqUGJYYLDpOs1jb7/aeK0VJlV5wGiIiIroeFkpE5DBWZlhOk0Ymh8BdoxScpnmi/D3QOdIXZhn46Wih6DhERER0HSyUiMghmMwyfmxY1jqhq2NNu7ta4/S7LC6fJSIislcslIjIIew5W4riSj38PNQYkBAkOs4tSWtov9t1phSXq+sFpyEiIqJrYaFERA7BOsQhLSUUGpVjv3TFBnoiOcwHJrOMDcfZfkdERGSPHPvdBhG5hHqjGWuzLFPixjvYktnrsZ4qrc1k+x0REZE9YqFERHZv+8li6GoNCPbWondcgOg4LSKtk+U6pZ2nS6CrNQhOQ0RERFdjoUREds+6ZPa2zmFQKiTBaVpGQrAX2od4wWCSsYntd0RERHaHhRIR2bWaeiM2HLMUEhOcpO3OKo3LZ4mIiOwWCyUismubjheh1mBCtL8Hukb5iY7TosY2tN9tP1WMyjq23xEREdkTFkpEZNesbXfju4RBkpyj7c6qfYgX4oM8UW80Y/OJItFxiIiI6FdYKBGR3dLVGrAtuxiA80y7+zVJkjDWunyW7XdERER2hYUSEdmt9UcLUG8yo32IFxJDfUTHaRVpnSxjwrdkF6FabxSchoiIiKxYKBGR3bIumXW2IQ6/lhzmg5gAD+iNZmxtOD0jIiIi8VgoEZFdKq7U4+fTJQCAcZ2dt1CSJOmX6XdZXD5LRERkL1goEZFdWpuVD7MMdIn0RWygp+g4rWqstf3uRBFq602C0xARERHAQomI7NTKDOu0O+c9TbLqFOGLCD931NSbsO0k2++IiIjsAQslIrI7ueW1SL9wGZLk3G13VpIkNZ4qrWX7HRERkV1goUREdmd1wxCHXrH+CPV1E5zGNtIals9uOl6EOgPb74iIiERjoUREdse6ZHZCV+c/TbLqGumHMF83VOmN2HmqRHQcIiIil8dCiYjsypniKhzNq4BK8cs0OFegUEgYk2Jpv+P0OyIiIvFYKBGRXbEOcRjQLhD+nhrBaWxrbEP73YZjhag3mgWnISIicm0slIjIbsiyjFVHnH/J7PV0j26DYG8tKuuM+PkM2++IiIhEYqFERHbjaF4FzhZXQ6tSYFTHUNFxbO7X7XdrM9l+R0REJBILJSKyG6sahjgMTwqGl1YlOI0Y1uuyfjpWCIOJ7XdERESisFAiIrtgNstYfcRyijLeBXYnXU+vOH8EeGpQXmPAnrOlouMQERG5LBZKRGQXDl68jNzyWnhpVRiaGCw6jjBKhYTR1ul3mQWC0xAREbkuFkpEZBesu5NGdQyBm1opOI1YY63td0cLYGT7HRERkRAslIhIOKPJjDUNwwvGu+C0u6v1jvdHGw81Sqvrse98meg4RERELomFEhEJt/tsKUqq6tHGQ40BCYGi4winViowKtk6/Y7td0RERCKwUCIi4axLZsd2CoNayZclAEjrZCmU1h0tgMksC05DRETkeviOhIiE0htNWHfUcmriiktmr6df20D4uKlQXKnHgQuXRcchIiJyOUILpTlz5qBnz57w9vZGcHAwJk6ciOzs7CvuU1BQgPvuuw+hoaHw9PREamoqli9fLigxEbW0bdnFqKwzItTHDT1j/UXHsRsalQIjk63T77h8loiIyNaEFkrbtm3DzJkzsWfPHmzYsAEGgwGjRo1CdXV1433uv/9+ZGdnY+XKlcjMzMSdd96Ju+++G4cOHRKYnIhainXa3bjOYVAoJMFp7Etaw5jw9UcLYGb7HRERkU0JLZTWrVuHGTNmoGPHjujSpQvmz5+Pixcv4sCBA4332bVrF5566in06tUL8fHx+Otf/wo/P78r7kNEjqlab8TG44UAgAld2XZ3tQHtAuGlVSFfV4eMS+Wi4xAREbkUu7pGSafTAQD8/X9pv+nXrx+WLFmCsrIymM1mLF68GHV1dRgyZMg1H0Ov16OiouKKGxHZp43HC1FnMCMmwAOdInxFx7E7bmolhidZlu+uZfsdERGRTdlNoWQ2m/Hss8+if//+SElJafz60qVLYTAYEBAQAK1Wi8cffxwrVqxAQkLCNR9nzpw58PX1bbxFRUXZ6q9ARE20qqHtbkKXcEgS2+6uJa1h+eyazALIMtvviIiIbMVuCqWZM2ciKysLixcvvuLrL7/8MsrLy7Fx40akp6fj+eefx913343MzMxrPs7s2bOh0+kabzk5ObaIT0RNVF5Tj20niwFw2t2NDOkQBA+NErnltcjM1YmOQ0RE5DJUogMAwKxZs7B69Wps374dkZGRjV8/c+YMPvjgA2RlZaFjx44AgC5dumDHjh348MMP8fHHH//msbRaLbRarc2yE1HzrMsqgMEkIzHUG+1CvEXHsVtuaiWGJgbjxyP5WJNZgM6RfqIjERERuQShJ0qyLGPWrFlYsWIFNm/ejLi4uCu+X1NTAwBQKK6MqVQqYTabbZaTiFreqiOWtrvxPE36XWMb2u/WZuWz/Y6IiMhGhBZKM2fOxIIFC7Bo0SJ4e3ujoKAABQUFqK2tBQAkJiYiISEBjz/+OPbt24czZ87g7bffxoYNGzBx4kSR0YnoFhRV1mH3mVIAbLu7GUM6BMFNrcCF0hocy+eAGiIiIlsQWijNnTsXOp0OQ4YMQVhYWONtyZIlAAC1Wo01a9YgKCgI48ePR+fOnfHVV1/hyy+/xNixY0VGJ6Jb8OORfJhloFu0H6L8PUTHsXueWhWGtLdOvysQnIaIiMg1CL1G6WZaSNq1a4fly5fbIA0R2Yp1yez4zjxNullpnUKx7mgB1mTm44+j2nNKIBERUSuzm6l3ROQacspqcOhiORQSMK5zmOg4DmNYYjA0KgXOllTjZGGV6DhEREROj4USEdmUdYhDn/gABPu4CU7jOLzd1BjULggAsIbLZ4mIiFodCyUisqmVGZx211xjO4UCsEy/IyIiotbFQomIbOZUYSVOFFRCrZSQlhIqOo7DGZ4UArVSwsnCKpwuqhQdh4iIyKmxUCIim1nVMMRhULsg+HloBKdxPL7uagxICATA6XdEREStjYUSEdmELMuN0+4mdGXbXXOldbIMwFiTxUKJiIioNbFQIiKbyMzV4XxpDdzUCoxIChEdx2GNSg6BSiHheH4FzpVUi45DRA4oI6ccX+8+D7P599e0ELkyFkpEZBPWtrvhSSHw1Apd4ebQ/Dw06Ns2AACHOhBR0/14JB93f7wbL/9wFN8dyhUdh8iusVAiolZnNstYfcTypn4Cp93dsrEN7Xe8TomImuLrPRcw65uDqDeZAQCf7TgLWeapEtH1sFAiola3/3wZ8nV18NaqMLh9kOg4Dm9UcggUkqWdMaesRnQcIrJzsizjvY0n8fL3WZBlYHL3SHholDhRUIkdp0pExyOyWyyUiKjVWZfMjk4JhZtaKTiN4wvw0qJPPNvviOj3mcwyXvnhKN7beAoA8Mzwdnjjrs64p2cUAGDejrMi4xHZNRZKRNSqDCYz1jS0iLHtruU0Tr9j+x0RXYfeaMLTiw/h6z0XIEnA/93eEc+NbA9JkvBQ/zgoJGDHqRIcz68QHZXILrFQIqJW9fPpEpRV1yPAU4N+DUMI6NaN7hgCSbJMr8otrxUdh4jsTJXeiIfm78ePR/KhVkp4f0o33N83tvH7Uf4eSEuxfODy2Y5zglIS2TcWSkTUqqy7k8Z2CoNKyZeclhLs7Yaesf4AgHXcqUREv1JapcfUT/fg59Ol8NAo8b8ZPTH+Gif6jwyMAwCsPJyLwoo6W8cksnt810JErabOYMJPRwsBcMlsaxibEgoAWJvJ65SIyCKnrAZ3fbwbmbk6+Htq8M2jfTCw3bWH6HSLboOesW1gMMmYv+u8bYMSOQAWSkTUarZmF6FKb0S4rxu6R7cRHcfpjGlom0m/cBkFOn4aTOTqThRUYNLcXThXUo0IP3cse6IvukT53fBnHh0YDwBYuOcCqvVGG6QkchwslIio1Vjb7sZ1CYdCIQlO43xCfd3QPcZSgK4/yvY7IleWfr4Md3+8G0WVerQP8cLyJ/uhbZDX7/7ciKQQxAV6oqLOiKXpOTZISuQ4WCgRUauo0hux6XgRAE67a01pDe13a9h+R+SyNh0vxPTP9qKizojuMW2w9PG+CPV1u6mfVSgkPDzAcq3S5zvPwdiwjJaIWCgRUSvZcKwAeqMZ8YGe6BjuIzqO07KOCd93vgzFlXrBaYjI1r49cAmPfX0AeqMZwxKDseDh3vDz0DTpMSalRqKNhxqXLtdifcN1pUTEQomIWsnKDEvb3fgu4ZAktt21lgg/d3SJ9IUss/2OyNV8su0M/rTsMExmGZNSI/HJfd3hrmn6Um93jRL3NYwO/3THWciy3MJJiRwTCyUianGXq+ux41QJAFxzJC21LOupEseEE7kGWZbx+prjmLP2BADgsUHxeGtyZ6hvYQXD/X1joFEpcDinHOkXLrdUVCKHxkKJiFrc2qwCGM0yksN8kBD8+xcT062xXqe0+2wpyqrrBachotZkMJnxp2VH8On2swCA2WmJ+PPYpFs+uQ/00mJSagQAYF7DYxO5OhZKRNTiVh7OBcDdSbYSE2C5DsxklrHhGE+ViJxVbb0JT3x9AMsPXoJSIeHNuzrj8cFtW+zxHx5gGRW+4XghzpVUt9jjEjkqFkpE1KIKdHXYe64MADCuc5jgNK5jbEP73ZpMFkpEzkhXY8B9n+/FphNF0KoU+OTe7pjcI6pFnyMh2AvDEoMhy8DnO3mqRMRCiYha1I+Z+ZBloHtMG0S28RAdx2VY2+9+Pl0CXY1BcBoiakkFujrc/clupF+4DB83FRY80hsjkkNa5bmsC2iXpV9iKy+5PBZKRNSirEtmuTvJtuKDvJAY6g2jWcaG4xzvS+QszhRXYdLcXcgurESwtxZLn+iLnrH+rfZ8feL9kRLhA73RjAV7LrTa8xA5AhZKRNRiLpRW43BOORTSL61gZDtpKZb/zddy+SyRUzhyqRyTP96N3PJaxAV6YvmT/ZAY2rp76SRJajxV+mr3edQZTK36fET2jIUSEbWYVQ2nSf3aBiLIWys4jesZ28nSfrfjVAkq6th+R+TIdp4qwdRP96Csuh6dInyx7Im+iPK3TTvz2E5hCPd1Q0lVPb4/lGuT5ySyRyyUiKjFrDpsOclg250Y7UK8kRDshXqTGZuPF4mOQ0TNtPpIHh6cvw/V9Sb0TwjAN4/1QaCX7T58UisVeGhAHABg3o6zMJu5gJZcEwslImoR2QWVyC6shFopYXTDYAGyvbEN/9uvYfsdkUP6evd5PPXNIRhMMm7rFIb/zegJL63K5jnu6RkFb60KZ4qrsfUkP3gh18RCiYhahHV30uD2wfB1VwtO47rSGq4N23qyGFV6o+A0RHSzZFnGuxtO4uUfjkKWgfv6xOD9qd2gVSmF5PF2U2Nq72gAwLzt54RkIBKNhRIR3TJZln9pu+OSWaESQ70RF+iJeqMZW07wU2AiR2Ayy3j5hyz8Z9MpAMCzI9rh/27vCKVCEpprRr9YqBQSdp8tRVauTmgWIhFYKBHRLTt8SYeLZTVwVysxIilYdByXJklS406ltVlsvyOyd3qjCU99cxAL9lyEJAH/nJiCZ0e0hySJLZIAINzPvXFx+LwdXEBLroeFEhHdspUZlml3I5ND4KGxfS89Xck6mn3LiWLU1LP9jsheVemNePCL/ViTWQC1UsIHU1NxX58Y0bGu8EjDqPDVR/KRW14rOA2RbbFQIqJbYjLLWH3EUiiN57Q7u9Ax3AdR/u6oNZiwLbtYdBwiuoaSKj2mfLobu86UwlOjxPwHe+G2zva3fy4lwhd94wNgMsuY/zOvVSLXwkKJiG7JvnNlKKrUw8dNhUHtA0XHIVja78Y2LJ9dk1UgOA0RXS2nrAZ3zd2FrNwKBHhqsPixvuifYL+vn48NspwqfbMvhzvayKWwUCKiW7KyYclsWkqYsOlM9FvW6XebjxeizmASnIaIrI7nV2DS3F04X1qDCD93LHuiLzpF+oqOdUOD2wchIdgLVXojluzLER2HyGZYKBFRs9UbzY0DA9h2Z1+6RPoi3NcN1fUmbD/J9jsie7DvXBnu/mQ3iir16BDije/+0A/xQV6iY/0uhULCowMtC2j/9/M5GExmwYmIbIOFEhE128+nS1BeY0CglxZ92waIjkO/IklS46nSWrbfEQm34Vgh7vt8LyrrjOgR0wZLH++LEB830bFu2u1dIxDopUG+ro4LrcllsFAiomaztt2N6xwmfN8H/dbYTpYx4RuPFUJvZPsdkShL03PwxIID0BvNGJ4YjK8f7g1fD8dazO2mVuKBvrEAgE+3n4Usy2IDEdkACyUiapbaehN+Omo5qWDbnX3qFtUGIT5aVOqN+Pl0ieg4RC5HlmV8vO0MXvz2CExmGXd1j8Qn93WHu8Yxr+e8t08M3NQKHM2rwO6zpaLjELU6FkpE1CybTxShut6ECD93pEb7iY5D16BQSEizTr/LZPsdkS2ZzTJeX3Mc/1p7AgDw+OB4vHlXZ6iUjvvWq42nBpO7RwEAPtvBUeHk/Bz3XysRCbXq8C+7k+xhgzxdW1qKpf3up6MFqDfyAmwiWzCYzPjTt4cxr6GY+PPYRMxOS3KK18qHB8RBkiwflp0uqhQdh6hVsVAioiarqDNgc3YRAGAC2+7sWo9YfwR6aVFRZ2SrDJEN1Nab8PjXB/DdwVwoFRLentwFjw1qKzpWi4kN9MTIpBAAPFUi58dCiYia7Kejhag3mtE2yBNJYd6i49ANKBUSxqRY3tSs5aQqolZVXlOPez/fi80niuCmVmDe/d0xqXuk6FgtzrqA9ruDuSiu1AtOQ9R6WCgRUZNZ2+4mdIlwilYSZze24Tql9UcLYOT+E6JWka+rxd2f7MaBC5fh46bCwkd6Y1hiiOhYraJ7TBt0jfJDvcmMr3efFx2HqNWwUCKiJimt0mNnwwS1CV3ZducIesX5w99Tg8s1Buw9VyY6DpHTOVNchbvm7sbJwiqE+Gix7Il+6B7jLzpWq5EkqfFU6es9F1Bbz/UD5JyEFkpz5sxBz5494e3tjeDgYEycOBHZ2dm/ud/u3bsxbNgweHp6wsfHB4MGDUJtba2AxES0JqsAJrOMThG+iAv0FB2HboJKqcCoZMsn21wUSdSyDueUY/LHu5FbXov4QE8sf7IfOoQ6f0vy6I6hiPJ3x+UaA749eEl0HKJWIbRQ2rZtG2bOnIk9e/Zgw4YNMBgMGDVqFKqrqxvvs3v3bowZMwajRo3Cvn37sH//fsyaNQsKBQ/DiERYlWGddhcmOAk1RVqnX9rvTGYuiiRqCTtOFWPqvD0oq65H50hfLHuiLyLbeIiOZRNKhYSH+8cBAD7fcZavK+SUVCKffN26dVf8ef78+QgODsaBAwcwaNAgAMBzzz2Hp59+Gi+99FLj/Tp06GDTnERkka+rxb7zltatcZ3ZdudI+rUNgK+7GiVV9Ug/X4be8QGiIxE5tJWH8/DHpRkwmGQMSAjEx/d1h5dW6Nsqm5vcIwrvbDiJ86U12Hi8EKM7hoqORNSi7OpYRqfTAQD8/S19vUVFRdi7dy+Cg4PRr18/hISEYPDgwdi5c+d1H0Ov16OiouKKGxG1jNWHLW1bvWL9Ee7nLjgNNYVaqcDIhva7tVlcPkt0K77cdR7PLD4Eg0nGuM5h+HxGD5crkgDAU6vCvX1iAACf7TgrOA1Ry7ObQslsNuPZZ59F//79kZKSAgA4e9byj+7vf/87Hn30Uaxbtw6pqakYPnw4Tp06dc3HmTNnDnx9fRtvUVFRNvs7EDm7lYfZdufIxnayfNq7NisfZrbJEDWZLMt456ds/G3lUcgy8EDfGLw/pRu0KqXoaMI80C8WaqWE/ecv49DFy6LjELUouymUZs6ciaysLCxevLjxa2azZYzt448/jgcffBDdunXDu+++iw4dOuB///vfNR9n9uzZ0Ol0jbecnByb5CdydudKqpGZq4NSIWFsJxZKjqh/QiC8tSoUVuhxKIdvaIiawmSW8Zfvs/D+5tMAgOdHtsffJ3SEQuHaKxJCfNxwe9cIAFxAS87HLgqlWbNmYfXq1diyZQsiI39ZzBYWZnkzlpycfMX9k5KScPHixWs+llarhY+PzxU3Irp11t1J/RMCEeClFZyGmkOrUmJE4/Q7tt8R3aw6gwmzFh3Eor0XIUnAqxNT8PTwdtwj1+CRgZahDmuz8pFTViM4DVHLEVooybKMWbNmYcWKFdi8eTPi4uKu+H5sbCzCw8N/MzL85MmTiImJsWVUIpcmy3Jj292ELhzi4MjSUhra7zLzIctsvyP6PZV1Bjz4xX6szSqARqnAh9NSG6/LIYvEUB8MbBcIswx8vpOnSuQ8hBZKM2fOxIIFC7Bo0SJ4e3ujoKAABQUFjTuSJEnCCy+8gPfffx/ffvstTp8+jZdffhknTpzAww8/LDI6kUs5nl+J00VV0KgUGNXROTfNu4pB7YPgqVEiT1eHw5d0ouMQ2bXiSj2mfLoHu8+WwlOjxPwHe7L1+DqsC2iXpudAV2MQnIaoZQgtlObOnQudTochQ4YgLCys8bZkyZLG+zz77LOYPXs2nnvuOXTp0gWbNm3Chg0b0LZtW4HJiVzLqiOW06ShHYLg46YWnIZuhZtaiWFJDdPvuHyW6Loultbgro934WheBQI8NVj8WF/0SwgUHctuDUgIRGKoN2rqTVi474LoOEQtQnjr3bVuM2bMuOJ+L730EnJyclBdXY1du3ZhwIABYgITuSBZlhuvT5rQJUJwGmoJYxva79Zksf2O6FqO51dg0se7cKG0BpFt3PHtk/3QKdJXdCy7JkkSHh1oOVWa//N51BvNghMR3Tq7GOZARPbr4MVyXLpcC0+NEsMSg0XHoRYwpEMw3NVK5JTV4mged80R/dres6W4+5PdKK7UIzHUG8uf7Ie4QE/RsRzC+C7hCPHRoqhS33hdK5EjY6FERDdkPU0amRwCd43r7gpxJu4aJYYmBgEA1rD9jqjRT0cLcN//9qGyzohesf5Y8nhfhPi4iY7lMDQqBWb0swzm+mzHWZ5Yk8NjoURE12Uyy/ix4Y30hK6cdudM0lIsF6Sv4fQ7IgDA0v05eGLBAdQbzRiRFIKvHu4FX3dek9lU03pFw0OjxImCSuw4VSI6DtEtYaFERNe152wpiiv18PNQY0BCkOg41IKGJgZDq1LgfGkNThRUio5DJIwsy5i79QxeXH4EZhmY3D0SH9+bCjc1T9Cbw9dDjXt6RgEA5u04KzgN0a1hoURE17Uyw9J2l5YSCo2KLxfOxEurwuD2luKX0+/IVZnNMl798Tj+ve4EAOCJwW3xxl2doVLy9e5WPNQ/DgoJ2HGqBMfzeR0kOS6+EhDRNdUbzVibZXkDPZ5LZp2SdR/MmqwCwUmIbM9gMuOPyw43Lkj9621JeCktEZIkCU7m+KL8PZDW8Pry2Q4uoCXHxUKJiK5p+8liVNQZEeytRe+4ANFxqBUMSwqGRqnA6aIqnCpk+x25jpp6Ix79Kh0rDuVCpZDwzt1d8EjDaGtqGdZR4SsP56Kwok5wGqLmYaFERNdkHe16W+cwKBX8hNUZ+bipMbCdZYHmmkyeKpFrKK+px/TP9mJrdjHc1ArMu78H7kyNFB3L6XSN8kPP2DYwmGTM33VedByiZmGhRES/UVNvxIZjhQCACWy7c2rW9hhrmyWRM8vX1WLyx7tx6GI5fN3VWPhIHwzlfrhWYz1VWrjnAqr1RsFpiJqOhRIR/cam40WoNZgQ7e+BrlF+ouNQKxqZFAKVQsKJgkqcKa4SHYeo1ZwuqsKkj3bhVFEVQn3csOyJvuge00Z0LKc2IikEcYGeqKgzYml6jug4RE3GQomIfsPadje+SxgvbHZyvh5q9E+wtN+t41AHclKHLl7G5I93IU9Xh/ggTyz/Qz+0D/EWHcvpKRQSHh5gWUD7+c5zMJrMghMRNQ0LJSK6gq7WgG3ZxQA47c5VjO0UCsCyfJbI2Ww7WYxp8/bico0BXSJ98e0T/RDh5y46lsuYlBqJNh5qXLpci/VHC0XHIWoSFkpEdIX1RwtQbzKjfYgXEkN9RMchGxiZHAqlQsLRvApcKK0WHYeoxfyQkYtHvtyPWoMJA9sFYtGjfeDvqREdy6W4a5S4r28sAODTHWchy7LYQERNwEKJiK6wqqHtjkMcXIe/pwZ94y0j4Ney/Y6cxBc/n8MzizNgMMkY3yUcnz/QE55alehYLun+vjHQqBQ4nFOO9AuXRcchumkslIioUXGlHj+fLgEAjOvMQsmVpDW0361l+x05OFmW8fZP2fjHqmMAgAf6xuA/93SFRsW3PKIEemkxKTUCADBv+1nBaYhuHl81iKjRmsx8mGWgS6QvYgM9RcchGxqVHAqFBBy+pMOlyzWi4xA1i8ks488rsvDfzacBAH8c2R5/n9ARCu6CE+7hAZZR4RuOF+JcCVt8yTGwUCKiRqsap93xNMnVBHlr0SvOHwCn35FjqjOYMHPhQXyz7yIUEvD6HZ3w1PB2nNxpJxKCvTA8MRiyDHy+k6dK5BhYKBERACC3vBbpFy5Dkth256rSUizLZzn9jhxNRZ0BM77Yh3VHC6BRKvDR9FRM6x0tOhZd5ZGGBbTL0i+hrLpecBqi39esQqm6uhovv/wy+vXrh4SEBMTHx19xIyLHYz1N6hXrj1BfN8FpSIQxKZbrlA5eLEe+rlZwGqKbU1ypx5RP9mDP2TJ4aVWY/1BPjGko+sm+9In3R0qED/RGMxbsuSA6DtHvatb4l0ceeQTbtm3Dfffdh7AwLqQkcgaN0+668jTJVYX4uKFHTBukX7iMdVkFeLB/nOhIRDd0sbQG9/1vLy6U1iDQS4P5D/ZCSoSv6Fh0HZIk4dGB8XhmcQa+2n0ejw2Kh5taKToW0XU1q1Bau3YtfvzxR/Tv37+l8xCRAGeKq3A0rwIqhdTYfkWuKa1TGNIvXMbaTBZKZN90tQZM/mQXCiv0iPJ3x9cP9eYQGgcwtlMY/r32BPJ0dfj+UC6m9GKLJNmvZrXetWnTBv7+/i2dhYgEWZlhOU0a0C6QyxhdnLX9bv+FMhRV1glOQ3R9H2w+hcIKPWIDPLD8iX4skhyEWqnAQwMsH8LM23EWZjMX0JL9alah9M9//hOvvPIKamo4QpbI0cmyzCWz1CjCzx1do/wgy8D6o4Wi4xBd04XSany5y3KNy98ndESwD6+rdCT39IyCt1aFM8XV2HqySHQcoutqVqH09ttvY/369QgJCUGnTp2Qmpp6xY2IHMfRvAqcLamGVqXAqI6houOQHRjL5bNk5/697gTqTWYMbBeIIR2CRcehJvJ2U2Nqw1TCedvPCU5DdH3NukZp4sSJLRyDiESxniYNTwqGl7ZZLwnkZNJSwvD6mhPYc7YUpVV6BHhpRUciarT/fBnWZBZAIQF/uS1JdBxqphn9YvG/neew+2wpsnJ1HMJBdqnJ74qMRiMkScJDDz2EyMjI1shERDZiNv/Sdjeeu5OoQZS/BzpF+CIzV4efjhViKi+2JjthNst4dfUxAMA9PaORGOojOBE1V7ifO8Z1DsP3GXmYt+Ms/jOlm+hIRL/R5NY7lUqFN998E0ajsTXyEJENHbx4GXm6OnhpVRiayPYV+kVaQ/sdl8+SPVl1JA+HL+ngqVHi+ZHtRcehW2RdQLv6SD5yy7m7jexPs65RGjZsGLZt29bSWYjIxlY2nCaN6hjCXRZ0BeuY+F1nSnG5ul5wGiKgzmDCv9eeAAD8YWgCgrzZEuroUiJ80a9tAExmGfN/5rVKZH+adUFCWloaXnrpJWRmZqJ79+7w9LxyJOeECRNaJBwRtR6jydx4WjCe0+7oKnGBnkgK88Hx/ApsOF6Iu3tEiY5ELu7zneeQp6tDhJ87Hh7AHV/O4tGB8dh1phTf7MvBU8PbwcdNLToSUaNmFUp/+MMfAADvvPPOb74nSRJMJtOtpSKiVrfrTClKqurRxkONAQmBouOQHRqbEorj+RVYm5nPQomEKqqsw0dbTgMAXhzTgSfgTmRw+yAkBHvhdFEVluzLwaOD4kVHImrUrNY7s9l83RuLJCLHYB3iMLZTGNTKZr0UkJNL62Rpv9t5ugS6WoPgNOTK3t1wCtX1JnSJ8uPgGSejUEh4dKDlhPB/P5+DwWQWnIjoF3x3ROSC9EYT1h0tAMAls3R9CcFeaB/iBYNJxqbjXD5LYpwoqMCS/RcBAC/flgSFQhKciFra7V0jEOilQb6ujgNkyK40q/Xu//7v/274/VdeeaVZYYjINrZmF6OyzohQHzf0jPUXHYfsWFpKGE4WnsKazALcmcqVEGRbsizjtR+PwyxbFiH34OuVU3JTK/FA31i8veEkPt1+FhO6hEOSWBCTeM0qlFasWHHFnw0GA86dOweVSoW2bduyUCKyc9a2u3Gdw/jpLN3Q2E5h+M+mU9h+qhiVdQZ480JrsqGtJ4ux41QJNEoF/t+YRNFxqBXd2ycGH249jaN5Fdh9thT92vLaWRKvWYXSoUOHfvO1iooKzJgxA3fccccthyKi1lOtN2JjQxvVhK5su6Mbax/ihfggT5wtrsbmE0W4vWuE6EjkIowmM1778TgAYEb/WMQEeP7OT5Aja+OpweTuUfh6zwV8tuMcCyWyCy12jZKPjw/+8Y9/4OWXX26phySiVrDxeCHqDGbEBHigU4Sv6Dhk5yRJwtiGnUprMwsEpyFXsnh/Dk4XVaGNhxozhyaIjkM28PCAOEgSsPlEEU4XVYqOQ9Sywxx0Oh10Ol1LPiQRtbCVGZa2O/aA081K6xQKANiSXYRqvVFwGnIFFXUGvLvhJADg2RHt4evOlk9XEBvoiVHJIQCAz3ZwAS2J16zWu/fff/+KP8uyjPz8fHz99ddIS0trkWBE1PLKa+qx/VQxAE67o5uXHOaDmAAPXCitwdbsYtzWOUx0JHJyH205g9LqesQHeWJa72jRcciGHh0Yj/VHC/HdwVz8cVQHBHlrRUciF9asQundd9+94s8KhQJBQUF44IEHMHv27BYJRkQtb11WAQwmGYmh3mgX4i06DjkISZKQlhKGj7edwZqsfBZK1Kpyymrwv52W04S/jE3injcX0z2mDbpG+SEjpxxf7z6P50d1EB2JXFizCqVz53gcSuSIVjZMuxvP0yRqorGdQvHxtjPYcqIItfUmuGuUoiORk3pjfTbqTWb0TwjAsMRg0XHIxiRJwmOD4vGHhQfx9Z4LeHJIAl9vSJhmfUzz0EMPobLytxfZVVdX46GHHrrlUETU8ooq6rD7bCkAtt1R03WK8EWEnztq6k3YdrJYdBxyUgcuXMaqw3mQJOAvY5N5HaWLGt0xFFH+7rhcY8C3By+JjkMurFmF0pdffona2trffL22thZfffXVLYciopb3Y2Y+ZBnoFu2HKH8P0XHIwUiShLENQx3WZuULTkPOSJZlvPrjMQDA5O6RSA73EZyIRFEqJDzcPw4A8PmOszCZZcGJyFU1qVCqqKiATqeDLMuorKxERUVF4+3y5ctYs2YNgoN5TE5kjxrb7jrzNImaJ62T5dqkTceLUGcwCU5Dzmb1kXwculgOD40Sf+R1KS5vco8o+LipcL60pnH3H5GtNalQ8vPzg7+/PyRJQvv27dGmTZvGW2BgIB566CHMnDmztbISUTPllNXg0MVyKCRgHC/Ep2bqGumHMF83VOmN2HmqRHQcciJ1BhP+ve4EAOCJwW0R4uMmOBGJ5qlV4d4+MQCAedvPCk5DrqpJwxy2bNkCWZYxbNgwLF++HP7+/o3f02g0iImJQXg4P60msjerjlhOk/rEByCYb0ComRQKCWNSQvHFz+exJisfIxr2nRDdqvm7zuPS5VqE+rjh0YHxouOQnXigXyzm7TiL9AuXcejiZXSLbiM6ErmYJhVKgwcPBmCZehcdHc2LLIkchHXJLKfd0a1KSwnDFz+fx4Zjhag3mqFRcXQz3ZqSKj0+3HwaAPDC6A6ccEaNQnzccHvXCHx74BI+23EOH05noUS21az/wsXExGDnzp2499570a9fP+Tm5gIAvv76a+zcubNFAxLRrTlVWIkTBZVQKyWkpYSKjkMOrntMGwR5a1FZZ8TPZ9h+R7fuvY0nUak3IiXCB3d0ixAdh+zMIwMtQx3WZuUjp6xGcBpyNc0qlJYvX47Ro0fD3d0dBw8ehF6vBwDodDq8/vrrN/04c+bMQc+ePeHt7Y3g4GBMnDgR2dnZ17yvLMtIS0uDJEn4/vvvmxNbKKPJjC9+PofbP9gJXa1BdBxyIasahjgMahcEPw+N4DTk6JQKCWM6Nky/y+T0O7o1pworsWjvRQDAX29LhkLBThW6UmKoDwa1D4JZBj7fyT2eZFvNKpReffVVfPzxx5g3bx7UanXj1/v374+DBw/e9ONs27YNM2fOxJ49e7BhwwYYDAaMGjUK1dXVv7nve++959CtfkqFhG/2XcThSzqszMgVHYdchCzLjdPuJnRl2x21jLSGMeE/HSuEwWQWnIYc2etrjsMsA6M7hqBPfIDoOGSnHm04VVqangNdDT9sJttpVqGUnZ2NQYMG/ebrvr6+KC8vv+nHWbduHWbMmIGOHTuiS5cumD9/Pi5evIgDBw5ccb+MjAy8/fbb+N///tecuHZBkiRM6RkNAPhmXw5kmTsBqPVl5upwvrQGbmoFRiTxwntqGb1i/RHgqUF5jQF7GpYYEzXV9pPF2JJdDJVCwktpSaLjkB0bkBCIxFBv1NSbsHDfBdFxyIU0q1AKDQ3F6dOnf/P1nTt3Ij6++dNqdDodAFwxTa+mpgbTpk3Dhx9+iNDQ37++Qq/XX7HfqaKiotl5WtqdqRHQqBQ4ll+BzFyd6DjkAqxDHIYnhcBT26TZLUTXpVIqMKqh/W5NZoHgNOSITGYZr/14HABwf99YxAV6Ck5E9kySpMZpiPN/Po96I0+yyTaaVSg9+uijeOaZZ7B3715IkoS8vDwsXLgQf/zjH/Hkk082K4jZbMazzz6L/v37IyUlpfHrzz33HPr164fbb7/9ph5nzpw58PX1bbxFRUU1K09r8PPQNF5M/82+HMFpyNmZzTJWH7FcQzKB0+6ohY21tt8dLYDJzBNyapql6TnILqyEr7saTw9PEB2HHMD4LuEI8dGiqFLf2FJO1NqaVSi99NJLmDZtGoYPH46qqioMGjQIjzzyCJ588kk88sgjzQoyc+ZMZGVlYfHixY1fW7lyJTZv3oz33nvvph9n9uzZ0Ol0jbecHPsqSKztdyszclGtNwpOQ85sz9lSFFTUwVurwuD2QaLjkJPpEx8APw81Sqvrse9cmeg45ECq9Ea8/dNJAMAzw9txyAzdFI1KgRn9LNcqfbbjLC9hIJtoVqEkSRL+8pe/oKysDFlZWdizZw+Ki4vh6+uLuLi4Jj/erFmzsHr1amzZsgWRkZGNX9+8eTPOnDkDPz8/qFQqqFSW1qFJkyZhyJAh13wsrVYLHx+fK272pE+8P+ICPVFdb8LqI/xEhFrPwn2WSVITuobDTc29JNSy1EoFRjUsnF2bxel3dPM+3noGJVV6xAZ44N4+MaLjkAOZ1isaHholThRUYscprieg1tekQkmv12P27Nno0aMH+vfvjzVr1iA5ORlHjx5Fhw4d8J///AfPPffcTT+eLMuYNWsWVqxYgc2bN/+myHrppZdw5MgRZGRkNN4A4N1338UXX3zRlOh2Q5Ik3NPT0g7I9jtqLSVVevx01HLtyLTe0YLTkLNK6xQGAFibVQAz2+/oJuSW12LejrMAgNljk7iwmJrE10Pd+B7K+ntE1JqadHX3K6+8gk8++QQjRozArl27MHnyZDz44IPYs2cP3n77bUyePBlK5c1/cj1z5kwsWrQIP/zwA7y9vVFQYHlj5+vrC3d3d4SGhl5zgEN0dHSzTq7sxaTUSLy1PhsZOeU4nl+BpDD7OvUix7cs/RIMJhldovzQMdxXdBxyUv3bBsLbTYXiSj0OXLyMnrH+v/9D5NLeXHcCeqMZveP8G08kiZriof5x+HLXeew4VcL3UNTqmvRRzrJly/DVV1/h22+/xU8//QSTyQSj0YjDhw9jypQpTSqSAGDu3LnQ6XQYMmQIwsLCGm9Llixp0uM4miBvLUY2/AdicUN7FFFLMZtlfNPwezW9F0+TqPVoVIrG17I1XD5Lv+NwTjm+z8iDJAEvj0t26N2IJE6Uv0fjafZnO7iAllpXkwqlS5cuoXv37gCAlJQUaLVaPPfcc81+sZNl+Zq3GTNm3PBnJk6c2KznsydTGt7ArjiUizqDSXAaciY7T5fgYlkNvN1UGNclTHQccnJjUyy/Y+vYfkc3IMsyXv3xGADgjm4RSIngSTc1n3VU+MrDuSisqBOchpxZkwolk8kEjeaX6TQqlQpeXl4tHsoVDEwIRISfOyrqjPwkllrUor2W06Q7u0XAQ8PdSdS6BrQLhJdWhXxdHTIulYuOQ3ZqXVYB9p+/DDe1Ai+M7iA6Djm4rlF+6BXrD4NJxvxd50XHISfWpHdR1tMerVYLAKirq8MTTzwBT88rF8V99913LZfQSSkUlqEO72w4icX7cnBnauTv/xDR7yiqqMOG44UAgGm9OU2KWp+bWonhScH4ISMPazPzkRrdRnQksjN6owlz1p4AADw2qC3CfN0FJyJn8MjAOOw7X4aFey5g1tAELlWnVtGkE6UHHngAwcHBjctc7733XoSHh1+x4NXXl8fpN2tyj0goJGDf+TKcLqoSHYecwNL0HJjMMrrHtEGHUG/RcchFpDW0363JLOBuE/qNr3dfwMWyGgR7a/H4oHjRcchJjEgKQVygJyrqjFiazinC1DqaVH476khuexXm645hicHYeLwIS/ZfxF9uSxYdiRyYySw3jpyfzpHgZENDOgTBQ6NEbnktMnN16BzpJzoS2Ymy6nr8Z9MpAMCfRnXgp/7UYhQKCQ8PiMNfv8/C5zvP4b4+MVApOW6eWhZ/owSb0tPyhnb5wVzojRzqQM23/WQxcstr4euuxthOHOJAtuOmVmJoYjAAy6kSkdX7m06hss6IpDAfTOrOFnNqWZNSI9HGQ41Ll2ux/mih6DjkhFgoCTakQxBCfLQoq67HhmP8R07Nt7BhiMOk1Ei4qZs2qp/oVlmn363Nymf7HQEAThdV4es9FwAAf70tCUoFx4FTy3LXKHFf31gAwKc7zvK1h1ocCyXBVEoF7u5h2TK9eB97bKl58nW12HzCOsQhSnAackVDOgTBTa3AhdIaHM2rEB2H7MC/1h6HySxjRFIw+icEio5DTur+vjHQqBQ4nFOO9AuXRcchJ8NCyQ7c3SMKktSw/6a0RnQcckCL9+XALAO94/yREMwhDmR7nloVhjW03/1lRSb3w7m4XadLsPF4EVQKCbPHJomOQ04s0EuLSakRAIB5288KTkPOhoWSHYjy98CAhk/blqRfFJyGHI3RZMaS/ZbTyGkc4kAC/b8xifDzUOPwJR3+3/IjbINxUSazjFd/PA7AMlimbRD3LVLreniAZZrihuOFOFdSLTgNORMWSnZiai/LG9xl6ZdgNJkFpyFHsiW7GAUVdfD31GBMSqjoOOTCYgI88dG0VCgVEn7IyMPcbWdERyIBlh+8hGP5FfB2U+GZEe1FxyEXkBDsheGJwZBl4POdPFWilsNCyU6MSApBgKcGRZV6bD5RJDoOOZCFey0XS9/VPRJaFYc4kFj9EgLx9/GWVQdvrs/GRg6pcSnVeiPeWp8NAHh6WDv4e2oEJyJX8chAy6nSsvRLKKuuF5yGnAULJTuhUSlwV8Po1MX7OdSBbk5OWQ22nSwG8MupJJFo9/WNxfTe0ZBl4JnFh5BdUCk6EtnIp9vPoqhSj2h/D9zfL0Z0HHIhfeL90SnCF3qjGQsapi0S3SoWSnbknp6WaWVbs4uQr6sVnIYcwZL9OZBloH9CAOICPUXHIWr09wkd0SfeH9X1Jjz85X5+wusCCnR1+GS7pd3ypbREnnCTTUmShEcGxgEAvtp9ngNlqEWwULIj8UFe6B3nD7MMLN1/SXQcsnMGkxlL0huGOPTiJ7dkX9RKBeZO745ofw9culyLJxccQL2R1186szfXZ6POYEaPmDZI4/WSJMDYTmEI93VDSVU9vj+UKzoOOQEWSnbG2j61ND0HJjMnRtH1bTxWiOJKPQK9NBiZHCI6DtFvtPHU4LMHesBLq8Lec2X4+6qjnITnpLJydVh+0PIB31/HJUOSuFyWbE+tVOChAZZTpXk7zsLM91F0i1go2ZkxKaHwdVcjt7wWO04Vi45DdmzRPsso+bt7REGj4j9lsk/tQ7zxnyldIUnAor0X8TWvHXA6sizj1R+PAQAmdg1H1yg/sYHIpd3TMwreWhXOFFdj60kOx6Jbw3dXdsZNrcQd3SyL0xbv41AHurYLpdXYcaoEksQhDmT/hieF4MXRiQCAf6w6hp9PlwhORC1pw7FC7DlbBq1KgRfGJIqOQy7O202NqQ07BT/lAlq6RSyU7JD1je/G44UoqqwTnIbs0TcNRfTAdkGI8vcQnIbo9z0xOB53dIuAySzjDwsP4jyXQjqFeqMZc9aeAAA8MjAOEX7ughMRATP6xUKlkLDnbBmycnWi45ADY6FkhzqEeqNbtB+MZhnfHuBQB7pSvdGMZY1DHHiaRI5BkiTMubMTukT5QVdrwCNfpaOiziA6Ft2iBXsu4FxJNQK9NHhySILoOEQAgHA/d4zrHAbAcq0SUXOxULJTU3ta3gAv2Z/DixHpCuuPFqC0uh4hPloMTwoWHYfoprmplZh3X3eE+GhxuqgKz3xziENrHFh5TT3+s+kUAOCPozrAS6sSnIjoF9YFtKuP5CO3nCtXqHlYKNmpcV3C4KVV4UJpDfacLRUdh+zIor2WIQ739IiCWsl/wuRYgn3cMO/+HtCqFNiSXYw31p0QHYma6b+bT0NXa0CHEG/c3SNKdByiK6RE+KJf2wCYzDLm/3xOdBxyUHyXZac8NCpM6BoOAPhmP4c6kMWZ4irsPlsKhQTcw7Y7clCdI/3wxl2dAQCfbD+L5WwxdjjnSqrx1e7zAIC/3JYEpYLjwMn+PNpwqvTNvhy2+lKzsFCyY9brT9ZnFXCrPQEAvmk4TRrSIZgXTZNDu71rBGYObQsAmP1dJg5evCw4ETXFv9Yeh8EkY0iHIAxqHyQ6DtE1DW4fhHbBXqjSG7GEk4SpGVgo2bGUCF+kRPig3mTGdwf5iaurqzOY8G3D78H03jxNIsf3x5EdMDI5BPUmMx776gDydbyOwBHsOVuK9UcLoVRI+MvYJNFxiK5LoZDwyEDLAtr//XwOBpNZcCJyNCyU7NyUhqEOi/fncKO9i1uXVYDyGgPCfd0wpAOHOJDjUygkvHtPV3QI8UZJlR6PfpWO2nqT6Fh0A2azjNd+PA4AmNorCu1CvAUnIrqx27tGINBLg3xdHdZk5ouOQw6GhZKdu71rONzVSpwuqsKBC2xNcWWNQxx6RvN6AHIaXloVPnugB/w9NcjKrcAL3x7mh0J27PuMXGTm6uClVeHZEe1FxyH6XW5qJR7oGwvAsoCWry/UFCyU7Jy3m7pxF8A37K91WScLK7HvfBmUCgn39OR0KXIuUf4e+Gh6KlQKCauP5OODzadFR6JrqK034Y112QCAmUMTEOilFZyI6Obc2ycGbmoFjuZVYDcnCVMTsFByAFMahjr8mJkHXS2ntrgi62nS8MRghPq6CU5D1PL6xAfg/25PAQC8veEk1mUVCE5EV5u34ywKKuoQ4eeOB/vHio5DdNPaeGowubvlQ8Z527mAlm4eCyUHkBrth/YhXqgzmLEyI1d0HLKxOoOpcZjHNA5xICc2rXc0HugbAwB4fmkGjudXCE5EVkUVdfh42xkAwEtpiXBTKwUnImqahwfEQZKALdnFOF1UKToOOQgWSg5AkqTGoQ7f7ONQB1ez+kg+KuqMiGzjjkHtOIaXnNvL45LRPyEANfUmPPJlOkqq9KIjEYC3fzqJmnoTukX7NbaDEzmS2EBPjEoOAQB8toMLaOnmsFByEHemRkCjUuBYfgUyc3Wi45ANLdx7AQAwtVc0FBziQE5OpVTgw2mpiA3wQG55Lf6w4CDqjRzpK9KxvAosPWC5RvavtyVDkvg6RI7JuoD2u4O5KK7khzD0+1goOQg/Dw3SUkIBcKiDKzmeX4FDF8uhUkiY3CNSdBwim/Dz0OCzB3rAW6vCvvNleOWHLJ6kCyLLMl798RhkGRjXOQzdY9qIjkTUbN1j2qBbtB/qTWZ8vfu86DjkAFgoORBr+93KjFxU642C05AtWIc4jOoYgmBvDnEg15EQ7I33p3aDJFn2yM3fdV50JJe0+UQRdp0phUalwP8bkyg6DtEtkSSp8VTp6z0XuLeNfhcLJQfSJ94fcYGeqK43YfWRPNFxqJVV641YccgyvGNarxjBaYhsb2hiMGanWd6c/3P1Mew4VSw4kWsxmMx4fY1lueyD/WMR5e8hOBHRrRvdMRRR/u64XGPAtw2Dkoiuh4WSA5GkX3bosP3O+a06nIcqvRExAR7o1zZAdBwiIR4dGI9JqZEwy8DMhQdxtrhKdCSX8c2+izhTXA1/Tw1mDk0QHYeoRSgVEh7uHwcA+HzHWZjMbOul62Oh5GAmpUZCpZCQkVOOEwUcnevMFu2ztN1N4xAHcmGSJOH1O1OQGu2HijojHvkqnfvkbEBXa8C7G04CAJ4b2R4+bmrBiYhazuQeUfBxU+F8aQ02Hi8UHYfsGAslBxPkrcXIhvGWi3mq5LSycnU4ckkHjVKBu7pziAO5Nq1KiY/v644wXzecLa7G098c4qfArezDLadxucaAhGAvTG3oZCByFp5aFe7tY2lp5wJauhEWSg5oSi/LUIfvDl5CnYEXIjqjhQ1DHEanhCLASys4DZF4wd5umHd/D7ipFdh2shhzGq6doZZ3sbQG838+DwD4y9gkqJR8q0DO54F+sVArJaRfuIyDFy+LjkN2iq9+DmhgQiAi/NxRUWfEmsx80XGohVXWGfBDhnWIQ7TgNET2IyXCF29P7goA+GznOSxL56l6a/j3uhOoN5kxsF0ghnTgkmtyTiE+bri9awQA4LMdPFWia2Oh5IAUil+GOrD9zvn8kJGHmnoT4oM80SfeX3QcIrtyW+cwPD28HQDgLyuycOBCmeBEziX9fBl+zMyHQgL+clsSl8uSU3tkoGWow7qsAuSU1QhOQ/aIhZKDmtwjEgoJ2He+DKeLOAXKWciy3Lg7aVqvaL5JIbqGZ4e3Q1pKKOpNZjz+9QHklteKjuQUzGYZ//zR0tJ4T88oJIb6CE5E1LoSQ30wqH0QzDLw+c5zouOQHWKh5KDCfN0xtEMwAGDJ/ouC01BLOXxJh2P5FdCoOMSB6HoUCglv390FSWE+KKmqx6NfpqOmnku4b9WqI3k4nFMOT40Sz41sLzoOkU082nCqtDQ9B7oaTtSkK7FQcmBTG65fWX4wF3ojhzo4g4V7LgAAbusUBj8PjeA0RPbLQ6PCvPu7I8BTg2P5FfjTssMwcxJes9UZTHhjXTYA4MkhbRHs7SY4EZFtDEgIRGKoN2rqTVi474LoOGRnWCg5sCEdghDio0VZdT02HOMeAEenqzVg1ZE8AMD03hziQPR7Itt44JP7ukOtlLAmswDvbz4lOpLD+nznOeSW1yLc1w2PDIwXHYfIZiRJwqMNv/Pzfz6PeqNZcCKyJyyUHJhKqcDdPTjUwVl8fygXdQYz2od4oXtMG9FxiBxCj1h/vDaxEwDgvY2nsJaTQJusuFKPj7acBgC8OCYRbmql4EREtjW+SzhCfLQoqtRj5eE80XHIjrBQcnB394iCJAE7T5fgYikntjgqDnEgar67e0bhof6W6wyeX3oYR/N0ghM5lnc3nkR1vQldIn0xoUu46DhENqdRKTCjn+U15LMdZyHLbOMlCxZKDi7K3wMDEgIBAEvSOdTBUR24cBnZhZVwUytwRyqHOBA11Z/HJmJQ+yDUGkx49Mt0FFfqRUdyCNkFlVi8z/Lfjr+OS4ZCwQ9pyDVN6xUND40SJwoqseNUieg4ZCeEFkpz5sxBz5494e3tjeDgYEycOBHZ2dmN3y8rK8NTTz2FDh06wN3dHdHR0Xj66aeh0/HTwl+zDnVYln4JRhN7ax2R9TRpfOdw+LqrBachcjwqpQL/ndoN8YGeyNPV4YkFBzjk5ia8tuY4zDKQlhKKnrHc20auy9dD3bijch4X0FIDoYXStm3bMHPmTOzZswcbNmyAwWDAqFGjUF1dDQDIy8tDXl4e3nrrLWRlZWH+/PlYt24dHn74YZGx7c6IpBAEeGpQVKnH5hNFouNQE5XX1GN1w3UV0zjEgajZfN3VmPdAD3i7qXDgwmX8dUUWW2huYGt2EbafLIZaKeGltETRcYiEe6h/HBQSsONUCY7nV4iOQ3ZAaKG0bt06zJgxAx07dkSXLl0wf/58XLx4EQcOHAAApKSkYPny5Rg/fjzatm2LYcOG4bXXXsOqVatgNHJnhtWvd+4s3s+hDo5m+cFc1BvNSArzQdcoP9FxiBxa2yAvfDgtFQoJWHbgEpdIXofRZMZrDctlZ/SLRUyAp+BEROJF+XsgrVMYAOCzHXztIDu7RsnaUufvf/3jf51OBx8fH6hUqmt+X6/Xo6Ki4oqbK7AeF2/NLkK+jlvqHYUsy1i417K3YVpvDnEgagmD2gfhL7clAwBeX3Mc204WC05kf5ak5+BUURX8PNSYNbSd6DhEdsM6Knzl4VwUVtQJTkOi2U2hZDab8eyzz6J///5ISUm55n1KSkrwz3/+E4899th1H2fOnDnw9fVtvEVFRbVWZLsSH+SF3nH+MMvA0v2XRMehm7T3XBnOFlfDQ6PExK6cNkXUUh7qH4u7e0TCLAOzFh3EmeIq0ZHsRmWdAe/8dBIA8OzwdvD14HWRRFZdo/zQK9YfBpOM+bvOi45DgtlNoTRz5kxkZWVh8eLF1/x+RUUFbrvtNiQnJ+Pvf//7dR9n9uzZ0Ol0jbecHNdpRbMOdViangMTN9Q7BOsQh9u7hsPbjW9WiFqKJEn458QU9Ihpg8o6Ix75Mh26GoPoWHbho61nUFpdj/hAT0zvEyM6DpHdeWSgZVT4wj0XUK3npR6uzC4KpVmzZmH16tXYsmULIiN/Oxq5srISY8aMgbe3N1asWAG1+vpvKLVaLXx8fK64uYoxKaHwdVcjt7wWO06x1cTelVbpsS6rAAAwrRffrBC1NK1KiY/v644IP3ecK6nGrG8Ouvxk0Jyymsbrtv48NglqpV28DSCyKyOSQhAX6ImKOiOWprvOB+70W0JfIWVZxqxZs7BixQps3rwZcXFxv7lPRUUFRo0aBY1Gg5UrV8LNzU1AUsfgplbijm4RAIDF+/gP2959e+AS6k1mdIrwRadIX9FxiJxSoJcWn97fHe5qJXacKsFra46LjiTUm+uzUW80o298AIYnBYuOQ2SXFAoJDw+wvCf9fOc5l/+AxZUJLZRmzpyJBQsWYNGiRfD29kZBQQEKCgpQW2sZRmAtkqqrq/H555+joqKi8T4mE/djXIu1/W7j8UIuXLRjZrOMbxqWPE7nSHCiVtUx3Bfv3tMFAPDFz+exZL9rLuc+ePEyVh7OgyQBf7kticNjiG5gUmok2niocelyLdYfLRQdhwQRWijNnTsXOp0OQ4YMQVhYWONtyZIlAICDBw9i7969yMzMREJCwhX3caVrj5qiQ6g3ukX7wWiW8e0BDnWwV7vPluJ8aQ28tCqM78IhDkStbUxKGJ4b0R4A8Nfvs7D/fJngRLYlyzJeXX0MAHBXaiRSIniKTXQj7hol7usbCwD4dMdZ7mRzUcJb7651mzFjBgBgyJAh171PbGysyOh2bWpPywnFkv0X+Q/bTlmHOEzsFg5P7bVH3RNRy3p6eAJu6xwGg0nGE18fwKXLNaIj2cyPmfk4eLEc7mol/jS6g+g4RA7h/r4x0KgUOJxTjvQLl0XHIQF4FacTGtclDF5aFc6X1mD32VLRcegqRZV1WH+UQxyIbE2SJLx1VxekRPigtLoej3yZ7hITreoMJvx73QkAwOOD4xHiw2t9iW5GoJcWk1It135/uv2s4DQkAgslJ+ShUWFCw06ebzjUwe4sS78Eo1lGt2g/JIe7zlRGInvgrlHi0/t6INBLixMFlXh+aQbMTr5O4ctd55FTVosQHy0eGxQvOg6RQ3l4gOXfzMbjhThXUi04DdkaCyUnZW2/W59VgLLqesFpyMpslrG44ULyab04xIFIhHA/d3x6f3dolAqsP1qI9zaeFB2p1ZRW6fHB5tMAgBdGJ8JDw1ZfoqZICPbC8MRgyDLw+U6eKrkaFkpOqlOkLzqG+6DeZMZ3BznUwV7sOF2CnLJaeLupMK4zhzgQiZIa3QZz7uwEAHh/82msPpInOFHreG/jKVTqjUiJ8MGdDesjiKhpHhloOVValn6JHz67GBZKTsw6Knzx/hwOdbATC/dcAGAZO+quUQpOQ+TaJnWPbGxF+9Oyw8jK1QlO1LJOF1ViUcMagr+MTYZCwXHgRM3RJ94fnSJ8oTeasaDhv+PkGlgoObHbu4bDXa3E6aIqHOC0FuEKK+qw6UQRAGAadycR2YX/NyYRQzsEoc5gxqNfpaOook50pBbz+poTMJlljEwOQd+2AaLjEDksSZLwyEDLAtqvdp9HnYG7PF0FCyUn5u2mxrjOYQA41MEeLNmfA5NZRs/YNmgf4i06DhEBUCok/GdqNyQEeyFfV4fHvj7gFG+CdpwqxuYTRVApJMxOSxQdh8jhje0Uhgg/d5RU1eOv32ehpEovOhLZAAslJzelof3ux8w86GoNgtO4LpNZxuKGFhieJhHZFx83NT67vwd83dXIyCnHn7/LdOh2ZZNZxms/HgcA3Nc3BvFBXoITETk+tVKBp4cnAAC+PXAJg97Ygnc2nERlHd9bOTMWSk4uNdoP7UO8UGcwY2VGrug4LmtrdhHydHXw81AjLSVMdBwiukpsoCc+mp4KpULCd4dyHXpnyrL0HJwoqISvuxrPDG8nOg6R07inZzQWPtIbXSJ9UVNvwvubTmHQG1vw2Y6zTnESTb/FQsnJSZKEKQ2jwr/Zx6EOoizaazlNuis1Em5qDnEgskf9EwLxyrhkAMC/1p3A5hOFghM1XZXeiLc3WMadPzUsAX4eGsGJiJxL/4RAfD+zPz6+NxVtgzxxucaAV388jmFvbcXS/TkwmsyiI1ILYqHkAu5MjYBGpcCx/ApkOtlUJ0eQV16LLdmWIQ5T2XZHZNfu7xuDqb2iIcvA099k4FRhpehITfLJtjMortQjNsAD9/eNFR2HyClJkoQxKWFY/+wgvDGpM8J83ZCnq8OLy49g9HvbsS4rnx9MOwkWSi7Az0ODtJRQABzqIMLi/Tkwy5bxom15rQCRXZMkCf+Y0BG94vxRpTfika/ScdlB9qbkldc2tgy+lJYEjYr/iSdqTSqlAnf3jMKWPw3BX29LQhsPNc4UV+OJBQcx8cOfset0ieiIdIv4KuoirO13KzNyUa03Ck7jOowmM5bstw5xiBGchohuhkalwMf3dkdkG3dcKK3BzEUHYXCAdpo312dDbzSjV5w/RncMER2HyGW4qZV4ZGA8tr04FE8PS4CHRonDl3SY9tle3Pf5Xhy5VC46IjUTCyUX0SfeH3GBnqiuNzntBnp7tOlEEQor9Ajw1PCNC5ED8ffU4LMHesBTo8SuM6X45+pjoiPd0JFL5VhxyDKw56+3JUGSuFyWyNZ83NR4flQHbH9xKGb0i4VaKWHHqRJM+OBn/GHhAZwprhIdkZqIhZKLkCQJ9/SMAsD2O1tqHOLQIxJaFYc4EDmSxFAfvHtPV0gS8NXuC1i494LoSNckyzJeXW0ZB35ntwh0jvQTG4jIxQV6afH3CR2x+Y9DcGdqBCQJWJNZgFHvbsdLy48gX1crOiLdJBZKLmRSaiRUCgkZOeU4UVAhOo7TyymrwfZTxQCAqT05xIHIEY3qGIo/jeoAAPjbD0ex+0yp4ES/tf5oAfadL4ObWoE/je4gOg4RNYjy98A7d3fFumcGYWRyiGWn4v4cDH5zK1778ZjDXP/oylgouZAgby1GJlvavxbzVKnVfbPvImQZGJAQiNhAT9FxiKiZ/jCkLSZ0CYfRLOMPCw/gYmmN6EiN9EYT5qw9AQB4bGA8wv3cBScioqt1CPXGvPt7YPmT/dArzh/1RjPm7TiHQW9swX83neK143aMhZKLmdLLcrLx3cFLXI7WigwmM5amXwIATOdIcCKHJkkS3rirMzpH+uJyjQGPfpWOKjt5Y/P17gu4UFqDIG8tHh/cVnQcIrqB7jFtsOSxPpj/YE8kh/mgsmHv2eA3t+DLXedRb7T/oTGuhoWSixmYEIgIP3dU1BmxNitfdBynteFYIUqq9Ajy1mJEMoc4EDk6N7USn97XA8HeWmQXVuLZxRkwm8XuSblcXY/3N50CAPxpVHt4alVC8xDR75MkCUM6BGP1UwPw/tRuiAnwQElVPf628iiGv7MVKw5dgknwawv9goWSi1EoONTBFqxDHO7uEQm1kv/MiJxBqK8bPr2/BzQqBTYeL8RbP2ULzfOfTadQUWdEYqg37uoeJTQLETWNQiFhQpdwbHx+MF6dmIIgby1yymrx3JLDuO39Hdh0vJBLa+0A38G5oMk9IqGQgH3nyjiqshWcL6nGztMlkKRf9lcRkXPoGuWHNyZ1BgB8tPUMfsjIFZLjTHEVFuyxTOH7623JUCo4DpzIEamVCtzbJwbbXxiK/zcmET5uKpwoqMTDX6bjro93Y9+5MtERXRoLJRcU5uuOoR2CAQBL9vNUqaV9s89ymjS4fRCi/D0EpyGiljaxWwSeHGK5HujFb4/gcE65zTPMWXMCRrOMYYnBGNAu0ObPT0Qty12jxJND2mLHi8Pw5JC2cFMrcODCZdz9yW48+MU+HMvjtGIRWCi5KOtQh28PXILeyKEOLUVvNGHZAcsQh2m9eJpE5KxeGNUBI5KCoTea8ehX6SisqLPZc+86U4KNxwuhVEj489hEmz0vEbU+Xw81/t+YRGx7YSim946GSiFhS3Yxxr6/A88sPoQLpdWiI7oUFkouamiHIIT4aFFWXY8NxwpFx3Ea648Woqy6HqE+bhiWGCw6DhG1EoVCwntTuqF9iBeKKvV47Kt0m0wSNZl/WS47vXc0EoK9W/05icj2Qnzc8NodnbDx+cGY0CUcAPBDRh6Gv70Nf/0+E0U2/HDGlbFQclEqpQJ397Bc/MudSi1nYcM1A3f3jIKKQxyInJqXVoXP7u+JNh5qHL6kw/9bfqTVL77+7uAlHMuvgLebCs8Mb9eqz0VE4sUGeuL9qd2w+qkBGNIhCEazjAV7LmLQm1vwxroT0NUaREd0anwn58Lu7hEFSQJ2ni6xqwWKjup0URX2niuDQgKm9OQEKiJXEB3ggY+md4dKIeGHjDzM3Xam1Z6rpt6IN9dbJu09NSwBAV7aVnsuIrIvKRG+mP9gLyx+rA9So/1QZzDjo61nMOiNLfh42xnU1vMyitbAQsmFRfl7YECC5SLgJekXBadxfNYhDsMSgxHu5y44DRHZSt+2Afj7hI4AgDfXZ2NjK7Uzf7r9LIoq9Yjyd8cD/WJb5TmIyL71iQ/A8if7Yd79PdA+xAu6WgP+tfYEhry1BYv2XoTBxKW1LYmFkoub2jBwYFn6JRj5j6vZ6gwmLD/YMMShN4c4ELmae/vE4L4+MZBl4JnFh5BdUNmij1+gq8Mn284CAF4akwStStmij09EjkOSJIxMDsHaZwbh7cldEOHnjsIKPf68IhMj39mGVYfzhC/EdhYslFzciKQQBHhqUFSpx+YTRaLjOKw1mfkorzEgws8dg9tziAORK3plfDL6xgegut6ER77aj7Lq+hZ77Ld+ykatwYTuMW0wtlNoiz0uETkupULCpO6R2Pynwfj7+GQEeGpwvrQGT31zCOM/2IltJ4u5tPYWsVBycRqVAnd1jwQALOZOpWZbtNfSdjelZxQXPxK5KLVSgY+mpyImwAM5ZbX4w8IDLdIGk5Wrazyx/uttSZAkvsYQ0S+0KiVm9I/DtheH4vmR7eGlVeFoXgUe+N8+TJ23BwcvXhYd0WGxUCLc0zB4YGt2EfJ1tYLTOJ6ThZVIv3AZSoWEuznEgciltfHU4LP7e8BLq8Kes2X4+8qjt/R4sizjtR+PQ5aBCV3C0S26TQslJSJn46VV4enh7bD9xaF4ZEAcNCoF9pwtw50f7cKjX6XjZGHLtgS7AhZKhPggL/SO84dZBpbuvyQ6jsOxniaNSApGiI+b4DREJFq7EG+8P7UrJAlYuPcivt59vtmPtfF4EXafLYVGpcCLYzq0XEgiclr+nhr8dVwytv5pCO7pEQWFBGw4VojR723HH5cexqXLnHR8s1goEYBfhjosTc+BiRcA3rTa+l8PcYgRnIaI7MWwxBD8vzGJAIC/rzqGXadLmvwY9UYzXl9jWS77yIA4RLbxaNGMROTcwv3c8e+7OuOn5wYhLSUUsgwsP3gJw97ahn+sOoqSKr3oiHaPhRIBAMakhMLXXY3c8lrsOFUsOo7DWHUkD5V1RkT5u2Ngw6h1IiIAeHxQPO7sFgGTWcYfFh3EhdLqJv38wr0XcK6kGoFeGjw5pG0rpSQiZ5cQ7I2593bH9zP7o39CAOpNZnzx83kMfmML3t1wEpV1XFp7PSyUCADgplbijm4RAIDF+zjU4WZZ2+6m9oqGgkMciOhXJEnC63d2QtcoP5TXGPDwl+k3/YZEV2PAfzadAgA8N7I9vN3UrRmViFxA1yg/LHykDxY83BudI31RXW/CfzadwqA3tuCzHWdRZ+DS2quxUKJG1va7jccLUVzJ49jfczRPh4yccqgUEiZ35xAHIvotN7USn97XHaE+bjhdVIVnFmfcVHvzfzefQnmNAe1DvHBPD76+EFHLGdAuED/M7I+501MRH+SJyzUGvPrjcQx7ayuWpudwr+avsFCiRh1CvdEt2g9Gs4xvD3Cow++xniaN7hiKIG+t4DREZK+Cfdzw6f3doVUpsPlEEd5Yf+KG9z9fUo0vGwZA/OW2ZKiU/E81EbUsSZKQ1ikMPz07CP+e1Alhvm7I09XhxW+PYMx/dmBdVgF3MIGFEl1lak/LqdKS/Rf5D+QGqvVG/JCRBwCY3jtacBoisnedI/3w1uQuAIBPtp3Fdwev/2HUv9aegMEkY3D7IAxuH2SriETkglRKBe7pGY0tfxqCv4xNgp+HGqeLqvDEggOY+NEu7DrT9EE0zoSFEl1hXJcweGlVOF9ag91nS0XHsVsrD+ehSm9EXKAn+rYNEB2HiBzA+C7hmDU0AQDw0neZOHSNJZB7z5Zi3dECKCTgL7cl2ToiEbkoN7USjw6Kx/YXh+KpYQnw0ChxOKcc0+btxX2f70XmJZ3oiEKwUKIreGhUmNA1HACHOtzIL0McoiBJHOJARDfn+ZHtMSo5BPVGMx77+sAVS77NZhmvNYwDn9IrGu1DvEXFJCIX5eOmxh9HdcC2F4ZiRr9YqJUSdpwqwfgPdmLmwoM4U1wlOqJNsVCi37C2363LKsDl6nrBaezPkUvlyMzVQaNU4C4OcSCiJlAoJLx7T1ckhnqjuFKPx746gNp6y6SpHw7n4sglHby0Kjw3or3gpETkyoK8tfj7hI7Y/MchuLNbBCQJ+DEzH6Pe3Y7Z3x254kMeZ8ZCiX6jU6QvOob7oN5kxneHckXHsTvW06S0TqHw99QITkNEjsZTq8K8+3vA31ODzFwdXlx+BDX1RryxLhsA8IehbTkghojsQpS/B965pyvWPjMQI5KCYTLL+GZfDga/uRWvrznu9B+os1Cia5rSMCp88T4Odfi1yjoDVh62DHGY1otDHIioeaL8PTB3eipUCgmrDufhrrm7ka+rQ4SfOx7qHyc6HhHRFRJDffDZAz2x/Mm+6BXrj3qjGZ9uP4tBb2zBB5tPoabeKDpiq2ChRNd0e9dwuKuVOFVUhQMXfnvBsav6PiMPNfUmJAR7oVecv+g4ROTAescH4NWJKQCAY/kVAIAXx3SAm1opMhYR0XV1j/HHksf74IsHeyIpzAeVeiPe+ukkBr2xFV/tPo96o3PtYGKhRNfk46bGuM5hAIBvONQBACDLMhbuuQDAspyXQxyI6FZN6RWNGf1iAQDdov0woUu42EBERL9DkiQM7RCMH58agP9M6Ypofw+UVOnxyg9HMfydrfj+UC7MN7FY2xFIspP3VVVUVMDX1xc6nQ4+Pj6i4ziUAxcuY9LcXXBTK7D3zyPg664WHUmogxcv486PdkGrUmDvn4fDz4PXJxHRrTOZZWw+UYRu0X4I9OK1SUTkWOqNZixJz8H7m06huFIPAEgM9cYLoztgWGKw3X2w3JTagCdKdF2p0X5oH+KFOoMZKzM41ME6xOG2zmEskoioxSgVEkYmh7BIIiKHpFEpcF+fGGx7YQheGN0B3m4qnCioxMNfpmPyx7ux/3yZ6IjNJrRQmjNnDnr27Alvb28EBwdj4sSJyM7OvuI+dXV1mDlzJgICAuDl5YVJkyahsLBQUGLXIkkSpjSMCv9mX45LD3XQ1Riw+ohliMP03hziQERERPRrHhoVZg5NwI4Xh+KJwW2hVSmQfuEyJn+8Gw/N349jeRWiIzaZ0EJp27ZtmDlzJvbs2YMNGzbAYDBg1KhRqK6ubrzPc889h1WrVmHZsmXYtm0b8vLycOeddwpM7VruTI2ARqXAsfwKZOa65lZmAPju0CXUGczoEOKN1Og2ouMQERER2SU/Dw1eSkvEtheGYlrvaCgVEjafKMLkj3ehss4gOl6T2NU1SsXFxQgODsa2bdswaNAg6HQ6BAUFYdGiRbjrrrsAACdOnEBSUhJ2796NPn36/O5j8hqlW/fM4kP4ISMPU3tFY86dnUTHsTlZljHq3e04VVSF/7u9I+7vGys6EhEREZFDOFdSjbd/ykZcoCf+OKqD6DiOe42STmc5sfD3t4xdPnDgAAwGA0aMGNF4n8TERERHR2P37t3XfAy9Xo+KioorbnRrrO13KzNyUa13zjn5N5J+4TJOFVXBXa3ExG4RouMQEREROYy4QE98MC0Vz49sLzpKk9lNoWQ2m/Hss8+if//+SEmx7JUoKCiARqOBn5/fFfcNCQlBQUHBNR9nzpw58PX1bbxFRUW1dnSn1yfeH3GBnqiuNzVep+NKrEMcxncJg4+ba0/+IyIiImoOe5t+dzPsplCaOXMmsrKysHjx4lt6nNmzZ0On0zXecnK4A+hWSZKEe3paCk5X26l0uboeP2bmAwCm9Y4RnIaIiIiIbMUuCqVZs2Zh9erV2LJlCyIjIxu/Hhoaivr6epSXl19x/8LCQoSGhl7zsbRaLXx8fK640a2blBoJlUJCRk45ThS4Tjvj8oOXUG80o2O4D7pE+oqOQ0REREQ2IrRQkmUZs2bNwooVK7B582bExcVd8f3u3btDrVZj06ZNjV/Lzs7GxYsX0bdvX1vHdWlB3lqMTA4BACx2kVMlWZaxaJ+l7W5a72iHPDImIiIiouYRWijNnDkTCxYswKJFi+Dt7Y2CggIUFBSgtrYWAODr64uHH34Yzz//PLZs2YIDBw7gwQcfRN++fW9q4h21rCm9LEMdvjt4CXUGk+A0rW/P2TKcLa6Gp0aJ27tyiAMRERGRKxFaKM2dOxc6nQ5DhgxBWFhY423JkiWN93n33Xcxbtw4TJo0CYMGDUJoaCi+++47gald18CEQET4uaOizoi1Wfmi47S6hXsvAAAmdI2Al1YlOA0RERER2ZLQd383s8LJzc0NH374IT788EMbJKIbUSgsQx3e2XAS3+zLwR3dIn//hxxUSZUe649aJitO7x0tOA0RERER2ZpdDHMgxzG5RyQUErDvXBnOFFeJjtNqvj1wCQaTjC6RvkiJ4BAHIiIiIlfDQomaJMzXHUM7BAMAlux3zqEOZrOMb341xIGIiIiIXA8LJWoy61CH5Qcso7Odzc9nSnChtAbeWhXGdwkXHYeIiIiIBGChRE02tEMQQny0KK2ux4ZjhaLjtLhFey2nSXekRsBDwyEORERERK6IhRI1mUqpwOTuUQCAxfsvCk7Tsooq6xqLP7bdEREREbkuFkrULPf0tBRKO06VIKesRnCalrMs/RKMZhmp0X5IDPURHYeIiIiIBGGhRM0S5e+Bge0CATjPqZLJLDe23U3rHSM4DRERERGJxEKJmm1qw1CHZemXYDQ5/lCH7aeKkVteCx83FcZ1DhMdh4iIiIgEYqFEzTYiKQQBnhoUVeqx+USR6Di3zHqaNKl7JNzUSsFpiIiIiEgkFkrUbBqVAnd1jwQALHbwnUr5utrGYm86hzgQERERuTwWSnRLrEMdtmYXIV9XKzhN8y3ZnwOTWUavWH8kBHuLjkNEREREgrFQolsSH+SF3nH+MMvA0v2XRMdpFqPJjCUNJ2LT+/A0iYiIiIhYKFELsA51WJpuOZVxNFuzi5Gvq0MbDzXGpISKjkNEREREdoCFEt2yMSmh8HVXI7e8FjtOFYuO02SL9lmGONzVPRJaFYc4EBERERELJWoBbmol7ugWAQBYvM+xhjpculyDLdmWIQ7WkzEiIiIiIhZK1CKsRcbG44UortQLTnPzluzPgSwD/doGID7IS3QcIiIiIrITLJSoRXQI9Ua3aD8YzTK+PeAYQx0MvxriMI0jwYmIiIjoV1goUYuZ2tNSbCzZfxGybP9DHTYdL0JRpR6BXhqMSuYQByIiIiL6BQslajHjuoTBS6vC+dIa7D5bKjrO71q49wIA4K7uUdCo+E+BiIiIiH7Bd4fUYjw0KkzoGg7A/oc6XCytwY5TJQCAaRziQERERERXYaFELcrafrcuqwCXq+sFp7m+b/ZbRoIPbBeI6AAPwWmIiIiIyN6wUKIW1SnSFx3DfVBvMuO7Q7mi41xTvdGMZemWE6/pHOJARERERNfAQola3JSGVrbF++xzqMNPxwpQUlWPIG8thieFiI5DRERERHaIhRK1uNu7hsNdrcSpoiocvHhZdJzfWLTX0nY3pWcU1Er+EyAiIiKi3+K7RGpxPm5q3NY5DADwjZ0NdThXUo1dZ0ohScA9PaNExyEiIiIiO8VCiVrF1F6WImT1kTxU1BkEp/nFN/ssp0lD2gchsg2HOBARERHRtbFQolaRGt0G7UO8UGcw44eMPNFxAAB1BlPjEIdpvWMEpyEiIiIie8ZCiVqFJEmY0jAq/Ju99jHUYf3RAlyuMSDM1w1DOwSJjkNEREREdoyFErWaO1MjoFEpcCy/Apm5OtFxsLBhiMM9PaOg4hAHIiIiIroBvlukVuPnoUFaSigA8UMdThdVYt+5Mig4xIGIiIiIbgILJWpV1va7lRm5qNYbheWwniYNSwxBmK+7sBxERERE5BhYKFGr6hPvj7hAT1TXm7D6iJihDnUGE5YfuAQAmN4nWkgGIiIiInIsLJSoVUmS1NjqJqr97scj+aioMyLCzx2D2nGIAxERERH9PhZK1OompUZCpZCQkVOOEwUVNn/+RQ27k6b2ioJSIdn8+YmIiIjI8bBQolYX5K3FyOQQAMBiG58qnSiowIELl6FSSLi7B4c4EBEREdHNYaFENjGll+XaoO8OXkKdwWSz513UMMRhZHIIgn3cbPa8REREROTYWCiRTQxMCESEnzsq6oxYm5Vvk+esqTdixcFcAMC03hziQEREREQ3j4US2YRCYfuhDqsP56NSb0S0vwf6tw20yXMSERERkXNgoUQ2M7lHJBQSsO9cGc4UV7X68y3cewEAMLVXNBQc4kBERERETcBCiWwmzNcdQzsEAwCW7G/dU6WsXB0OX9JBrZQwuUdkqz4XERERETkfFkpkU9ahDssPXEK90dxqz2MdCT66YygCvbSt9jxERERE5JxYKJFNDe0QhBAfLUqr67HhWGGrPEeV3ogfDnGIAxERERE1HwslsimVUoHJ3S1DHRbvv9gqz/FDRi6q602ID/RE3/iAVnkOIiIiInJuLJTI5qzT73acKkFOWU2LPrYsy427k6b1joYkcYgDERERETUdCyWyuSh/DwxsZxnX3dJDHY5c0uFoXgU0KgUmpXKIAxERERE1DwslEmJKT8u1Q8sO5MBoarmhDtbTpLEpoWjjqWmxxyUiIiIi18JCiYQYmRyCAE8NCiv02JJd3CKPWVFnwMrDeQCAab1jWuQxiYiIiMg1CS2Utm/fjvHjxyM8PBySJOH777+/4vtVVVWYNWsWIiMj4e7ujuTkZHz88cdiwlKL0qgUuKu7pTVu8b6WGerw/aFc1BpMaBfshZ6xbVrkMYmIiIjINQktlKqrq9GlSxd8+OGH1/z+888/j3Xr1mHBggU4fvw4nn32WcyaNQsrV660cVJqDdahDluyi5Cvq72lx+IQByIiIiJqSUILpbS0NLz66qu44447rvn9Xbt24YEHHsCQIUMQGxuLxx57DF26dMG+fftsnJRaQ3yQF3rH+cMsA0v3X7qlxzp4sRwnCiqhVSlwZzcOcSAiIiKiW2PX1yj169cPK1euRG5uLmRZxpYtW3Dy5EmMGjXquj+j1+tRUVFxxY3s19RelqEOS9NzYDLLzX6chXsvAADGdQ6Hr4e6RbIRERERkeuy60Lpv//9L5KTkxEZGQmNRoMxY8bgww8/xKBBg677M3PmzIGvr2/jLSoqyoaJqanGpITC112N3PJa7DjVvKEOuhoDfjySDwCY3ie6JeMRERERkYuy+0Jpz549WLlyJQ4cOIC3334bM2fOxMaNG6/7M7Nnz4ZOp2u85eS07J4ealluaiXu6BYBAFi8r3n/v1p+8BL0RjMSQ73RLcqvBdMRERERkatSiQ5wPbW1tfjzn/+MFStW4LbbbgMAdO7cGRkZGXjrrbcwYsSIa/6cVquFVqu1ZVS6RVN7RWP+rvPYeLwQxZV6BHnf/P//ZFnGooapedM5xIGIiIiIWojdnigZDAYYDAYoFFdGVCqVMJtbbkEpidch1Bvdov1gNMv49kDThjrsO1eG00VVcFcrcXvDyRQRERER0a0SeqJUVVWF06dPN/753LlzyMjIgL+/P6KjozF48GC88MILcHd3R0xMDLZt24avvvoK77zzjsDU1Bqm9ozGoYvlWLL/Ip4YHH/TJ0PW06Tbu4bDx41DHIiIiIioZQg9UUpPT0e3bt3QrVs3AJa9Sd26dcMrr7wCAFi8eDF69uyJ6dOnIzk5Gf/617/w2muv4YknnhAZm1rBuC5h8NKqcL60BrvPlt7Uz5RV12NtZgEAy+4kIiIiIqKWIvREaciQIZDl64+EDg0NxRdffGHDRCSKh0aFCV3DsWjvRSzel4N+bQN/92e+PZCDepMZKRE+6Bzp1/ohiYiIiMhl2O01SuR6pva0nAqtyyrA5er6G95XlmV80zAlb1qvmFbPRkRERESuhYUS2Y1Okb7oGO6DepMZ3x3KveF9d58pxbmSanhpLSdRREREREQtiYUS2ZUpvSynSov3XbxhW+bCXw1x8NLa7ZR7IiIiInJQLJTIrtzeNRzuaiVOFVXh4MXL17xPcaUe67M4xIGIiIiIWg8LJbIrPm5q3NY5DAAar0G62rIDOTCaZXSJ8kPHcF9bxiMiIiIiF8FCiezO1F5RAIDVR/JQUWe44ntms4zFDQXUdJ4mEREREVErYaFEdic1ug3aBXuhzmDGDxl5V3xv5+kSXCyrgbebCuM7c4gDEREREbUOFkpkdyRJumKow68t3HsBAHBntwi4a5Q2z0ZEREREroGFEtmlO7tFQKNS4GheBTIv6QAAhRV12Hi8CAAwrTd3JxERERFR62GhRHapjacGaSmhAIBv9ltOlZbuz4HJLKNHTBt0CPUWGY+IiIiInBwLJbJbU3pa2u9WZuShss6AxfstQxw4EpyIiIiIWhsLJbJbfeL9ERfoiSq9ES9+ewS55bXwdVdjbKcw0dGIiIiIyMmxUCK7JUkS7ulpGRW+tmHB7KTUSLipOcSBiIiIiFoXCyWya5NSI6FSSI1/ZtsdEREREdkCCyWya0HeWoxMDgEA9I7zR0Kwl+BEREREROQKVKIDEP2eF8ckAgBmDUsQnISIiIiIXAULJbJ7cYGemHtvd9ExiIiIiMiFsPWOiIiIiIjoKiyUiIiIiIiIrsJCiYiIiIiI6CoslIiIiIiIiK7CQomIiIiIiOgqLJSIiIiIiIiuwkKJiIiIiIjoKiyUiIiIiIiIrsJCiYiIiIiI6CoslIiIiIiIiK7CQomIiIiIiOgqLJSIiIiIiIiuwkKJiIiIiIjoKiyUiIiIiIiIrsJCiYiIiIiI6CoslIiIiIiIiK7CQomIiIiIiOgqLJSIiIiIiIiuohIdoLXJsgwAqKioEJyEiIiIiIhEstYE1hrhRpy+UKqsrAQAREVFCU5CRERERET2oLKyEr6+vje8jyTfTDnlwMxmM/Ly8uDt7Q1JkoRmqaioQFRUFHJycuDj4yM0C7kG/s6RLfH3jWyNv3Nka/ydc3yyLKOyshLh4eFQKG58FZLTnygpFApERkaKjnEFHx8f/uMim+LvHNkSf9/I1vg7R7bG3znH9nsnSVYc5kBERERERHQVFkpERERERERXYaFkQ1qtFn/729+g1WpFRyEXwd85siX+vpGt8XeObI2/c67F6Yc5EBERERERNRVPlIiIiIiIiK7CQomIiIiIiOgqLJSIiIiIiIiuwkKJiIiIiIjoKiyUbOjDDz9EbGws3Nzc0Lt3b+zbt090JHJCc+bMQc+ePeHt7Y3g4GBMnDgR2dnZomORC/nXv/4FSZLw7LPPio5CTiw3Nxf33nsvAgIC4O7ujk6dOiE9PV10LHJCJpMJL7/8MuLi4uDu7o62bdvin//8JzgPzfmxULKRJUuW4Pnnn8ff/vY3HDx4EF26dMHo0aNRVFQkOho5mW3btmHmzJnYs2cPNmzYAIPBgFGjRqG6ulp0NHIB+/fvxyeffILOnTuLjkJO7PLly+jfvz/UajXWrl2LY8eO4e2330abNm1ERyMn9O9//xtz587FBx98gOPHj+Pf//433njjDfz3v/8VHY1aGceD20jv3r3Rs2dPfPDBBwAAs9mMqKgoPPXUU3jppZcEpyNnVlxcjODgYGzbtg2DBg0SHYecWFVVFVJTU/HRRx/h1VdfRdeuXfHee++JjkVO6KWXXsLPP/+MHTt2iI5CLmDcuHEICQnB559/3vi1SZMmwd3dHQsWLBCYjFobT5RsoL6+HgcOHMCIESMav6ZQKDBixAjs3r1bYDJyBTqdDgDg7+8vOAk5u5kzZ+K222674rWOqDWsXLkSPXr0wOTJkxEcHIxu3bph3rx5omORk+rXrx82bdqEkydPAgAOHz6MnTt3Ii0tTXAyam0q0QFcQUlJCUwmE0JCQq74ekhICE6cOCEoFbkCs9mMZ599Fv3790dKSoroOOTEFi9ejIMHD2L//v2io5ALOHv2LObOnYvnn38ef/7zn7F//348/fTT0Gg0eOCBB0THIyfz0ksvoaKiAomJiVAqlTCZTHjttdcwffp00dGolbFQInJiM2fORFZWFnbu3Ck6CjmxnJwcPPPMM9iwYQPc3NxExyEXYDab0aNHD7z++usAgG7duiErKwsff/wxCyVqcUuXLsXChQuxaNEidOzYERkZGXj22WcRHh7O3zcnx0LJBgIDA6FUKlFYWHjF1wsLCxEaGiooFTm7WbNmYfXq1di+fTsiIyNFxyEnduDAARQVFSE1NbXxayaTCdu3b8cHH3wAvV4PpVIpMCE5m7CwMCQnJ1/xtaSkJCxfvlxQInJmL7zwAl566SVMmTIFANCpUydcuHABc+bMYaHk5HiNkg1oNBp0794dmzZtavya2WzGpk2b0LdvX4HJyBnJsoxZs2ZhxYoV2Lx5M+Li4kRHIic3fPhwZGZmIiMjo/HWo0cPTJ8+HRkZGSySqMX179//N2sPTp48iZiYGEGJyJnV1NRAobjyLbNSqYTZbBaUiGyFJ0o28vzzz+OBBx5Ajx490KtXL7z33nuorq7Ggw8+KDoaOZmZM2di0aJF+OGHH+Dt7Y2CggIAgK+vL9zd3QWnI2fk7e39m2vgPD09ERAQwGvjqFU899xz6NevH15//XXcfffd2LdvHz799FN8+umnoqORExo/fjxee+01REdHo2PHjjh06BDeeecdPPTQQ6KjUSvjeHAb+uCDD/Dmm2+ioKAAXbt2xfvvv4/evXuLjkVORpKka379iy++wIwZM2wbhlzWkCFDOB6cWtXq1asxe/ZsnDp1CnFxcXj++efx6KOPio5FTqiyshIvv/wyVqxYgaKiIoSHh2Pq1Kl45ZVXoNFoRMejVsRCiYiIiIiI6Cq8RomIiIiIiOgqLJSIiIiIiIiuwkKJiIiIiIjoKiyUiIiIiIiIrsJCiYiIiIiI6CoslIiIiIiIiK7CQomIiIiIiOgqLJSIiIiIiIiuwkKJiIjsXnFxMZ588klER0dDq9UiNDQUo0ePxs8//wwAkCQJ33//vdiQRETkVFSiAxAREf2eSZMmob6+Hl9++SXi4+NRWFiITZs2obS0VHQ0IiJyUjxRIiIiu1ZeXo4dO3bg3//+N4YOHYqYmBj06tULs2fPxoQJExAbGwsAuOOOOyBJUuOfAeCHH35Aamoq3NzcEB8fj3/84x8wGo2N35ckCXPnzkVaWhrc3d0RHx+Pb7/9tvH79fX1mDVrFsLCwuDm5oaYmBjMmTPHVn91IiISiIUSERHZNS8vL3h5eeH777+HXq//zff3798PAPjiiy+Qn5/f+OcdO3bg/vvvxzPPPINjx47hk08+wfz58/Haa69d8fMvv/wyJk2ahMOHD2P69OmYMmUKjh8/DgB4//33sXLlSixduhTZ2dlYuHDhFYUYERE5L0mWZVl0CCIiohtZvnw5Hn30UdTW1iI1NRWDBw/GlClT0LlzZwCWk6EVK1Zg4sSJjT8zYsQIDB8+HLNnz2782oIFC/Diiy8iLy+v8eeeeOIJzJ07t/E+ffr0QWpqKj766CM8/fTTOHr0KDZu3AhJkmzzlyUiIrvAEyUiIrJ7kyZNQl5eHlauXIkxY8Zg69atSE1Nxfz586/7M4cPH8b//d//NZ5IeXl54dFHH0V+fj5qamoa79e3b98rfq5v376NJ0ozZsxARkYGOnTogKeffho//fRTq/z9iIjI/rBQIiIih+Dm5oaRI0fi5Zdfxq5duzBjxgz87W9/u+79q6qq8I9//AMZGRmNt8zMTJw6dQpubm439Zypqak4d+4c/vnPf6K2thZ333037rrrrpb6KxERkR1joURERA4pOTkZ1dXVAAC1Wg2TyXTF91NTU5GdnY2EhITf3BSKX/7zt2fPnit+bs+ePUhKSmr8s4+PD+655x7MmzcPS5YswfLly1FWVtaKfzMiIrIHHA9ORER2rbS0FJMnT8ZDDz2Ezp07w9vbG+np6XjjjTdw++23AwBiY2OxadMm9O/fH1qtFm3atMErr7yCcePGITo6GnfddRcUCgUOHz6MrKwsvPrqq42Pv2zZMvTo0QMDBgzA/2/njnFMjcI4Dv9DdCRKX0QvOoVYgo3QKJU6OhIsQJRCohWLsAWR2Ifm1vPdmfbeyczzLOAUp/vlPec9Ho+53+85HA5Jku12m6Io0u/3U6lUcrlc0mq10mw2/8dVAPAPCSUAvrV6vZ7hcJjdbpfn85n3+51Op5PJZJL5fJ4k2Ww2mc1m2e/3abfbeb1eGY1GuV6vWS6XWa1WqdVq6Xa7GY/HH85fLBY5n8+ZTqcpiiKn0ym9Xi9J0mg0sl6v83g8Uq1WMxgMcrvdPkykAPiZbL0D4Nf6bFseACT+KAEAAPxFKAEAAJT4owTAr+X1OQBfMVECAAAoEUoAAAAlQgkAAKBEKAEAAJQIJQAAgBKhBAAAUCKUAAAASoQSAABAiVACAAAo+QPx2a9JMTghVwAAAABJRU5ErkJggg==", "text/plain": [ "
" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "# Create the environment\n", "env = RecyclingRobot(alpha=0.3, beta=0.2, r_search=6, r_wait=2)\n", "\n", "# Creating the random agent\n", "agent = RandomAgent(env)\n", "\n", "# Train the agent for 10 episodes\n", "returns = agent.train(10)\n", "\n", "\n", "# Plot the rewards\n", "plt.figure(figsize=(10, 6))\n", "plt.plot(returns)\n", "plt.xlabel(\"Steps\")\n", "plt.ylabel(\"Return\")\n", "plt.show()" ] }, { "cell_type": "markdown", "metadata": { "id": "HM17OH2joGsw" }, "source": [ "That's it! We now \"only\" need to define classes for all the sampling-based RL algorithms (MC, TD, deep RL) and we can interact with any environment with a single line!" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Mujoco and Atari environments\n", "\n", "Note: both mujoco and atari environments will not work on colab. \n", "\n", "You may have to install non-Python packages on your computer, such as openGL. A lot of debugging in sight...\n", "\n", "The environments should work under Linux and MacOS, but I am not sure about windows. \n", "\n", "### Mujoco\n", "\n", "To install the mujoco environments of gymnasium, this should work:\n", "\n", "```bash\n", "pip install mujoco\n", "pip install gymnasium[mujoco]\n", "```\n", "\n", "Interaction should work as usual. See all environments here: " ] }, { "cell_type": "code", "execution_count": 17, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "MoviePy - Building file videos/Walker2d-v4.gif with imageio.\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ " \r" ] }, { "data": { "text/html": [ "
" ], "text/plain": [ "" ] }, "execution_count": 17, "metadata": {}, "output_type": "execute_result" } ], "source": [ "envname = 'Walker2d-v4'\n", "env = gym.make(envname, render_mode=\"rgb_array_list\")\n", "recorder = GymRecorder(env)\n", "\n", "returns = random_interaction(env, 10, recorder)\n", "\n", "video = \"videos/\" + envname + \".gif\"\n", "recorder.make_video(video)\n", "ipython_display(video)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Atari\n", "\n", "The atari games are available as binary ROM files, which have to be downloaded separately. The AutoROM package can do that for you: \n", "\n", "```bash\n", "pip install autorom\n", "AutoROM --accept-license\n", "```\n", "\n", "You can then install the atari submodules of gym (in particular ale_py):\n", "\n", "```bash\n", "pip install gymnasium[atari]\n", "```\n", "\n", "Check out the list of Atari games here: " ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "env = gym.make('ALE/Breakout-v5', render_mode='human')\n", "\n", "returns = random_interaction(env, 1, None)\n", "\n", "env.close()" ] } ], "metadata": { "colab": { "name": "7-Gym-solution.ipynb", "provenance": [] }, "kernelspec": { "display_name": "ANNarchy", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.11.6" } }, "nbformat": 4, "nbformat_minor": 4 }