{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "## Configurations for Colab" ] }, { "cell_type": "code", "execution_count": 1, "metadata": {}, "outputs": [], "source": [ "import sys\n", "IN_COLAB = \"google.colab\" in sys.modules\n", "\n", "if IN_COLAB:\n", " !apt install python-opengl\n", " !apt install ffmpeg\n", " !apt install xvfb\n", " !pip install PyVirtualDisplay==3.0\n", " !pip install gymnasium==0.28.1\n", " from pyvirtualdisplay import Display\n", " \n", " # Start virtual display\n", " dis = Display(visible=0, size=(400, 400))\n", " dis.start()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "# 03. Prioritized Experience Replay (PER)\n", "\n", "[T. Schaul et al., \"Prioritized Experience Replay.\" arXiv preprint arXiv:1511.05952, 2015.](https://arxiv.org/pdf/1511.05952.pdf)\n", "\n", "Using a replay memory leads to design choices at two levels: which experiences to store, and which experiences to replay (and how to do so). This paper addresses only the latter: making the most effective use of the replay memory for learning, assuming that its contents are outside of our control.\n", "\n", "The central component of prioritized replay is the criterion by which the importance of each transition is measured. A reasonable approach is to use the magnitude of a transition’s TD error $\\delta$, which indicates how ‘surprising’\n", "or unexpected the transition is. This algorithm stores the last encountered TD error along with each transition in the replay memory. The transition with the largest absolute TD error is replayed from the memory. A Q-learning update\n", "is applied to this transition, which updates the weights in proportion to the TD error. One thing to note that new transitions arrive without a known TD-error, so it puts them at maximal priority in order to guarantee that all experience is seen at least once. (see *store* method)\n", "\n", "We might use 2 ideas to deal with TD-error: 1. greedy TD-error prioritization, 2. stochastic prioritization. However, greedy TD-error prioritization has a severe drawback. Greedy prioritization focuses on a small subset of the experience: errors shrink slowly, especially when using function approximation, meaning that the initially high error transitions get replayed frequently. This lack of diversity that makes the system prone to over-fitting. To overcome this issue, we will use a stochastic sampling method that interpolates between pure greedy prioritization and uniform random sampling.\n", "\n", "$$\n", "P(i) = \\frac{p_i^{\\alpha}}{\\sum_k p_k^{\\alpha}}\n", "$$\n", "\n", "where $p_i > 0$ is the priority of transition $i$. The exponent $\\alpha$ determines how much prioritization is used, with $\\alpha = 0$ corresponding to the uniform case. In practice, we use additional term $\\epsilon$ in order to guarantee all transactions can be possibly sampled: $p_i = |\\delta_i| + \\epsilon$, where $\\epsilon$ is a small positive constant.\n", "\n", "One more. Let's recall one of the main ideas of DQN. To remove correlation of observations, it uses uniformly random sampling from the replay buffer. Prioritized replay introduces bias because it doesn't sample experiences uniformly at random due to the sampling proportion correspoding to TD-error. We can correct this bias by using importance-sampling (IS) weights\n", "\n", "$$\n", "w_i = \\big( \\frac{1}{N} \\cdot \\frac{1}{P(i)} \\big)^\\beta\n", "$$\n", "\n", "that fully compensates for the non-uniform probabilities $P(i)$ if $\\beta = 1$. These weights can be folded into the Q-learning update by using $w_i\\delta_i$ instead of $\\delta_i$. In typical reinforcement learning scenarios, the unbiased nature of the updates is most important near convergence at the end of training, We therefore exploit the flexibility of annealing the amount of importance-sampling correction over time, by defining a schedule on the exponent $\\beta$ that reaches 1 only at the end of learning. " ] }, { "cell_type": "code", "execution_count": 2, "metadata": {}, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "/Users/jinwoo.park/miniforge3/envs/rainbow-is-all-you-need/lib/python3.8/site-packages/tqdm/auto.py:21: TqdmWarning: IProgress not found. Please update jupyter and ipywidgets. See https://ipywidgets.readthedocs.io/en/stable/user_install.html\n", " from .autonotebook import tqdm as notebook_tqdm\n" ] } ], "source": [ "import os\n", "import random\n", "from typing import Dict, List, Tuple\n", "\n", "import gymnasium as gym\n", "import matplotlib.pyplot as plt\n", "import numpy as np\n", "import torch\n", "import torch.nn as nn\n", "import torch.nn.functional as F\n", "import torch.optim as optim\n", "from IPython.display import clear_output\n", "\n", "if IN_COLAB and not os.path.exists(\"segment_tree.py\"):\n", " # download segment tree module\n", " !wget https://raw.githubusercontent.com/curt-park/rainbow-is-all-you-need/master/segment_tree.py\n", " \n", "from segment_tree import MinSegmentTree, SumSegmentTree" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Replay buffer\n", "\n", "Please see *01.dqn.ipynb* for detailed description." ] }, { "cell_type": "code", "execution_count": 3, "metadata": {}, "outputs": [], "source": [ "class ReplayBuffer:\n", " \"\"\"A simple numpy replay buffer.\"\"\"\n", "\n", " def __init__(self, obs_dim: int, size: int, batch_size: int = 32):\n", " self.obs_buf = np.zeros([size, obs_dim], dtype=np.float32)\n", " self.next_obs_buf = np.zeros([size, obs_dim], dtype=np.float32)\n", " self.acts_buf = np.zeros([size], dtype=np.float32)\n", " self.rews_buf = np.zeros([size], dtype=np.float32)\n", " self.done_buf = np.zeros(size, dtype=np.float32)\n", " self.max_size, self.batch_size = size, batch_size\n", " self.ptr, self.size, = 0, 0\n", "\n", " def store(\n", " self,\n", " obs: np.ndarray,\n", " act: np.ndarray, \n", " rew: float, \n", " next_obs: np.ndarray, \n", " done: bool,\n", " ):\n", " self.obs_buf[self.ptr] = obs\n", " self.next_obs_buf[self.ptr] = next_obs\n", " self.acts_buf[self.ptr] = act\n", " self.rews_buf[self.ptr] = rew\n", " self.done_buf[self.ptr] = done\n", " self.ptr = (self.ptr + 1) % self.max_size\n", " self.size = min(self.size + 1, self.max_size)\n", "\n", " def sample_batch(self) -> Dict[str, np.ndarray]:\n", " idxs = np.random.choice(self.size, size=self.batch_size, replace=False)\n", " return dict(obs=self.obs_buf[idxs],\n", " next_obs=self.next_obs_buf[idxs],\n", " acts=self.acts_buf[idxs],\n", " rews=self.rews_buf[idxs],\n", " done=self.done_buf[idxs])\n", "\n", " def __len__(self) -> int:\n", " return self.size" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Prioritized replay Buffer\n", "\n", "The key concept of PER's implementation is *Segment Tree*. It efficiently stores and samples transitions while managing the priorities of them. We recommend you understand how it works before you move on. Here are references for you:\n", "\n", "- In Korean: https://mrsyee.github.io/rl/2019/01/25/PER-sumtree/\n", "- In English: https://www.geeksforgeeks.org/segment-tree-set-1-sum-of-given-range/" ] }, { "cell_type": "code", "execution_count": 4, "metadata": {}, "outputs": [], "source": [ "class PrioritizedReplayBuffer(ReplayBuffer):\n", " \"\"\"Prioritized Replay buffer.\n", " \n", " Attributes:\n", " max_priority (float): max priority\n", " tree_ptr (int): next index of tree\n", " alpha (float): alpha parameter for prioritized replay buffer\n", " sum_tree (SumSegmentTree): sum tree for prior\n", " min_tree (MinSegmentTree): min tree for min prior to get max weight\n", " \n", " \"\"\"\n", " \n", " def __init__(\n", " self, \n", " obs_dim: int,\n", " size: int, \n", " batch_size: int = 32, \n", " alpha: float = 0.6\n", " ):\n", " \"\"\"Initialization.\"\"\"\n", " assert alpha >= 0\n", " \n", " super(PrioritizedReplayBuffer, self).__init__(obs_dim, size, batch_size)\n", " self.max_priority, self.tree_ptr = 1.0, 0\n", " self.alpha = alpha\n", " \n", " # capacity must be positive and a power of 2.\n", " tree_capacity = 1\n", " while tree_capacity < self.max_size:\n", " tree_capacity *= 2\n", "\n", " self.sum_tree = SumSegmentTree(tree_capacity)\n", " self.min_tree = MinSegmentTree(tree_capacity)\n", " \n", " def store(\n", " self, \n", " obs: np.ndarray, \n", " act: int, \n", " rew: float, \n", " next_obs: np.ndarray, \n", " done: bool\n", " ):\n", " \"\"\"Store experience and priority.\"\"\"\n", " super().store(obs, act, rew, next_obs, done)\n", " \n", " self.sum_tree[self.tree_ptr] = self.max_priority ** self.alpha\n", " self.min_tree[self.tree_ptr] = self.max_priority ** self.alpha\n", " self.tree_ptr = (self.tree_ptr + 1) % self.max_size\n", "\n", " def sample_batch(self, beta: float = 0.4) -> Dict[str, np.ndarray]:\n", " \"\"\"Sample a batch of experiences.\"\"\"\n", " assert len(self) >= self.batch_size\n", " assert beta > 0\n", " \n", " indices = self._sample_proportional()\n", " \n", " obs = self.obs_buf[indices]\n", " next_obs = self.next_obs_buf[indices]\n", " acts = self.acts_buf[indices]\n", " rews = self.rews_buf[indices]\n", " done = self.done_buf[indices]\n", " weights = np.array([self._calculate_weight(i, beta) for i in indices])\n", " \n", " return dict(\n", " obs=obs,\n", " next_obs=next_obs,\n", " acts=acts,\n", " rews=rews,\n", " done=done,\n", " weights=weights,\n", " indices=indices,\n", " )\n", " \n", " def update_priorities(self, indices: List[int], priorities: np.ndarray):\n", " \"\"\"Update priorities of sampled transitions.\"\"\"\n", " assert len(indices) == len(priorities)\n", "\n", " for idx, priority in zip(indices, priorities):\n", " assert priority > 0\n", " assert 0 <= idx < len(self)\n", "\n", " self.sum_tree[idx] = priority ** self.alpha\n", " self.min_tree[idx] = priority ** self.alpha\n", "\n", " self.max_priority = max(self.max_priority, priority)\n", " \n", " def _sample_proportional(self) -> List[int]:\n", " \"\"\"Sample indices based on proportions.\"\"\"\n", " indices = []\n", " p_total = self.sum_tree.sum(0, len(self) - 1)\n", " segment = p_total / self.batch_size\n", " \n", " for i in range(self.batch_size):\n", " a = segment * i\n", " b = segment * (i + 1)\n", " upperbound = random.uniform(a, b)\n", " idx = self.sum_tree.retrieve(upperbound)\n", " indices.append(idx)\n", " \n", " return indices\n", " \n", " def _calculate_weight(self, idx: int, beta: float):\n", " \"\"\"Calculate the weight of the experience at idx.\"\"\"\n", " # get max weight\n", " p_min = self.min_tree.min() / self.sum_tree.sum()\n", " max_weight = (p_min * len(self)) ** (-beta)\n", " \n", " # calculate weights\n", " p_sample = self.sum_tree[idx] / self.sum_tree.sum()\n", " weight = (p_sample * len(self)) ** (-beta)\n", " weight = weight / max_weight\n", " \n", " return weight" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Network\n", "\n", "We are going to use a simple network architecture with three fully connected layers and two non-linearity functions (ReLU)." ] }, { "cell_type": "code", "execution_count": 5, "metadata": {}, "outputs": [], "source": [ "class Network(nn.Module):\n", " def __init__(self, in_dim: int, out_dim: int):\n", " \"\"\"Initialization.\"\"\"\n", " super(Network, self).__init__()\n", "\n", " self.layers = nn.Sequential(\n", " nn.Linear(in_dim, 128), \n", " nn.ReLU(),\n", " nn.Linear(128, 128), \n", " nn.ReLU(), \n", " nn.Linear(128, out_dim)\n", " )\n", "\n", " def forward(self, x: torch.Tensor) -> torch.Tensor:\n", " \"\"\"Forward method implementation.\"\"\"\n", " return self.layers(x)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## DQN + PER Agent\n", "\n", "Here is a summary of DQNAgent class.\n", "\n", "| Method | Note |\n", "| --- | --- |\n", "|select_action | select an action from the input state. |\n", "|step | take an action and return the response of the env. |\n", "|compute_dqn_loss | return dqn loss. |\n", "|update_model | update the model by gradient descent. |\n", "|target_hard_update| hard update from the local model to the target model.|\n", "|train | train the agent during num_frames. |\n", "|test | test the agent (1 episode). |\n", "|plot | plot the training progresses. |\n", "\n", "\n", "All differences from pure DQN are noted with comments - PER.\n", "\n", "#### __init__\n", "\n", "Here, we use PrioritizedReplayBuffer, instead of ReplayBuffer, and use hold 2 more parameters beta and priority epsilon which are used to calculate weights and new priorities respectively.\n", "\n", "#### compute_dqn_loss & update_model\n", "\n", "It returns every loss per each sample for importance sampling before average. After updating the nework, it is necessary to update priorities of all sampled experiences.\n", "\n", "#### train\n", "\n", "beta linearly increases to 1 at every training step." ] }, { "cell_type": "code", "execution_count": 6, "metadata": {}, "outputs": [], "source": [ "class DQNAgent:\n", " \"\"\"DQN Agent interacting with environment.\n", " \n", " Attribute:\n", " env (gym.Env): openAI Gym environment\n", " memory (ReplayBuffer): replay memory to store transitions\n", " batch_size (int): batch size for sampling\n", " epsilon (float): parameter for epsilon greedy policy\n", " epsilon_decay (float): step size to decrease epsilon\n", " max_epsilon (float): max value of epsilon\n", " min_epsilon (float): min value of epsilon\n", " target_update (int): period for target model's hard update\n", " gamma (float): discount factor\n", " dqn (Network): model to train and select actions\n", " dqn_target (Network): target model to update\n", " optimizer (torch.optim): optimizer for training dqn\n", " transition (list): transition information including \n", " state, action, reward, next_state, done\n", " beta (float): determines how much importance sampling is used\n", " prior_eps (float): guarantees every transition can be sampled\n", " \"\"\"\n", "\n", " def __init__(\n", " self, \n", " env: gym.Env,\n", " memory_size: int,\n", " batch_size: int,\n", " target_update: int,\n", " epsilon_decay: float,\n", " seed: int,\n", " max_epsilon: float = 1.0,\n", " min_epsilon: float = 0.1,\n", " gamma: float = 0.99,\n", " # PER parameters\n", " alpha: float = 0.2,\n", " beta: float = 0.6,\n", " prior_eps: float = 1e-6,\n", " ):\n", " \"\"\"Initialization.\n", " \n", " Args:\n", " env (gym.Env): openAI Gym environment\n", " memory_size (int): length of memory\n", " batch_size (int): batch size for sampling\n", " target_update (int): period for target model's hard update\n", " epsilon_decay (float): step size to decrease epsilon\n", " lr (float): learning rate\n", " max_epsilon (float): max value of epsilon\n", " min_epsilon (float): min value of epsilon\n", " gamma (float): discount factor\n", " alpha (float): determines how much prioritization is used\n", " beta (float): determines how much importance sampling is used\n", " prior_eps (float): guarantees every transition can be sampled\n", " \"\"\"\n", " obs_dim = env.observation_space.shape[0]\n", " action_dim = env.action_space.n\n", " \n", " self.env = env\n", " \n", " self.batch_size = batch_size\n", " self.epsilon = max_epsilon\n", " self.epsilon_decay = epsilon_decay\n", " self.seed = seed\n", " self.max_epsilon = max_epsilon\n", " self.min_epsilon = min_epsilon\n", " self.target_update = target_update\n", " self.gamma = gamma\n", " \n", " # device: cpu / gpu\n", " self.device = torch.device(\n", " \"cuda\" if torch.cuda.is_available() else \"cpu\"\n", " )\n", " print(self.device)\n", " \n", " # PER\n", " # In DQN, We used \"ReplayBuffer(obs_dim, memory_size, batch_size)\"\n", " self.beta = beta\n", " self.prior_eps = prior_eps\n", " self.memory = PrioritizedReplayBuffer(\n", " obs_dim, memory_size, batch_size, alpha\n", " )\n", "\n", " # networks: dqn, dqn_target\n", " self.dqn = Network(obs_dim, action_dim).to(self.device)\n", " self.dqn_target = Network(obs_dim, action_dim).to(self.device)\n", " self.dqn_target.load_state_dict(self.dqn.state_dict())\n", " self.dqn_target.eval()\n", " \n", " # optimizer\n", " self.optimizer = optim.Adam(self.dqn.parameters())\n", "\n", " # transition to store in memory\n", " self.transition = list()\n", " \n", " # mode: train / test\n", " self.is_test = False\n", "\n", " def select_action(self, state: np.ndarray) -> np.ndarray:\n", " \"\"\"Select an action from the input state.\"\"\"\n", " # epsilon greedy policy\n", " if self.epsilon > np.random.random():\n", " selected_action = self.env.action_space.sample()\n", " else:\n", " selected_action = self.dqn(\n", " torch.FloatTensor(state).to(self.device)\n", " ).argmax()\n", " selected_action = selected_action.detach().cpu().numpy()\n", " \n", " if not self.is_test:\n", " self.transition = [state, selected_action]\n", " \n", " return selected_action\n", "\n", " def step(self, action: np.ndarray) -> Tuple[np.ndarray, np.float64, bool]:\n", " \"\"\"Take an action and return the response of the env.\"\"\"\n", " next_state, reward, terminated, truncated, _ = self.env.step(action)\n", " done = terminated or truncated\n", "\n", " if not self.is_test:\n", " self.transition += [reward, next_state, done]\n", " self.memory.store(*self.transition)\n", " \n", " return next_state, reward, done\n", "\n", " def update_model(self) -> torch.Tensor:\n", " \"\"\"Update the model by gradient descent.\"\"\"\n", " # PER needs beta to calculate weights\n", " samples = self.memory.sample_batch(self.beta)\n", " weights = torch.FloatTensor(\n", " samples[\"weights\"].reshape(-1, 1)\n", " ).to(self.device)\n", " indices = samples[\"indices\"]\n", "\n", " # PER: importance sampling before average\n", " elementwise_loss = self._compute_dqn_loss(samples)\n", " loss = torch.mean(elementwise_loss * weights)\n", "\n", " self.optimizer.zero_grad()\n", " loss.backward()\n", " self.optimizer.step()\n", " \n", " # PER: update priorities\n", " loss_for_prior = elementwise_loss.detach().cpu().numpy()\n", " new_priorities = loss_for_prior + self.prior_eps\n", " self.memory.update_priorities(indices, new_priorities)\n", "\n", " return loss.item()\n", " \n", " def train(self, num_frames: int, plotting_interval: int = 200):\n", " \"\"\"Train the agent.\"\"\"\n", " self.is_test = False\n", " \n", " state, _ = self.env.reset(seed=self.seed)\n", " update_cnt = 0\n", " epsilons = []\n", " losses = []\n", " scores = []\n", " score = 0\n", "\n", " for frame_idx in range(1, num_frames + 1):\n", " action = self.select_action(state)\n", " next_state, reward, done = self.step(action)\n", "\n", " state = next_state\n", " score += reward\n", " \n", " # PER: increase beta\n", " fraction = min(frame_idx / num_frames, 1.0)\n", " self.beta = self.beta + fraction * (1.0 - self.beta)\n", "\n", " # if episode ends\n", " if done:\n", " state, _ = self.env.reset(seed=self.seed)\n", " scores.append(score)\n", " score = 0\n", "\n", " # if training is ready\n", " if len(self.memory) >= self.batch_size:\n", " loss = self.update_model()\n", " losses.append(loss)\n", " update_cnt += 1\n", " \n", " # linearly decrease epsilon\n", " self.epsilon = max(\n", " self.min_epsilon, self.epsilon - (\n", " self.max_epsilon - self.min_epsilon\n", " ) * self.epsilon_decay\n", " )\n", " epsilons.append(self.epsilon)\n", " \n", " # if hard update is needed\n", " if update_cnt % self.target_update == 0:\n", " self._target_hard_update()\n", "\n", " # plotting\n", " if frame_idx % plotting_interval == 0:\n", " self._plot(frame_idx, scores, losses, epsilons)\n", " \n", " self.env.close()\n", " \n", " def test(self, video_folder: str) -> None:\n", " \"\"\"Test the agent.\"\"\"\n", " self.is_test = True\n", " \n", " # for recording a video\n", " naive_env = self.env\n", " self.env = gym.wrappers.RecordVideo(self.env, video_folder=video_folder)\n", " \n", " state, _ = self.env.reset(seed=self.seed)\n", " done = False\n", " score = 0\n", " \n", " while not done:\n", " action = self.select_action(state)\n", " next_state, reward, done = self.step(action)\n", "\n", " state = next_state\n", " score += reward\n", " \n", " print(\"score: \", score)\n", " self.env.close()\n", " \n", " # reset\n", " self.env = naive_env\n", "\n", " def _compute_dqn_loss(self, samples: Dict[str, np.ndarray]) -> torch.Tensor:\n", " \"\"\"Return dqn loss.\"\"\"\n", " device = self.device # for shortening the following lines\n", " state = torch.FloatTensor(samples[\"obs\"]).to(device)\n", " next_state = torch.FloatTensor(samples[\"next_obs\"]).to(device)\n", " action = torch.LongTensor(samples[\"acts\"].reshape(-1, 1)).to(device)\n", " reward = torch.FloatTensor(samples[\"rews\"].reshape(-1, 1)).to(device)\n", " done = torch.FloatTensor(samples[\"done\"].reshape(-1, 1)).to(device)\n", "\n", " # G_t = r + gamma * v(s_{t+1}) if state != Terminal\n", " # = r otherwise\n", " curr_q_value = self.dqn(state).gather(1, action)\n", " next_q_value = self.dqn_target(\n", " next_state\n", " ).max(dim=1, keepdim=True)[0].detach()\n", " mask = 1 - done\n", " target = (reward + self.gamma * next_q_value * mask).to(self.device)\n", "\n", " # calculate element-wise dqn loss\n", " elementwise_loss = F.smooth_l1_loss(curr_q_value, target, reduction=\"none\")\n", "\n", " return elementwise_loss\n", " \n", " def _target_hard_update(self):\n", " \"\"\"Hard update: target <- local.\"\"\"\n", " self.dqn_target.load_state_dict(self.dqn.state_dict())\n", " \n", " def _plot(\n", " self, \n", " frame_idx: int, \n", " scores: List[float], \n", " losses: List[float], \n", " epsilons: List[float],\n", " ):\n", " \"\"\"Plot the training progresses.\"\"\"\n", " clear_output(True)\n", " plt.figure(figsize=(20, 5))\n", " plt.subplot(131)\n", " plt.title('frame %s. score: %s' % (frame_idx, np.mean(scores[-10:])))\n", " plt.plot(scores)\n", " plt.subplot(132)\n", " plt.title('loss')\n", " plt.plot(losses)\n", " plt.subplot(133)\n", " plt.title('epsilons')\n", " plt.plot(epsilons)\n", " plt.show()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Environment\n", "\n", "You can see the [code](https://github.com/Farama-Foundation/Gymnasium/blob/main/gymnasium/envs/classic_control/cartpole.py) and [configurations](https://github.com/Farama-Foundation/Gymnasium/blob/main/gymnasium/envs/classic_control/cartpole.py#L91) of CartPole-v1 from Farama Gymnasium's repository." ] }, { "cell_type": "code", "execution_count": 7, "metadata": {}, "outputs": [], "source": [ "# environment\n", "env = gym.make(\"CartPole-v1\", max_episode_steps=200, render_mode=\"rgb_array\")" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Set random seed" ] }, { "cell_type": "code", "execution_count": 8, "metadata": {}, "outputs": [], "source": [ "seed = 777\n", "\n", "def seed_torch(seed):\n", " torch.manual_seed(seed)\n", " if torch.backends.cudnn.enabled:\n", " torch.cuda.manual_seed(seed)\n", " torch.backends.cudnn.benchmark = False\n", " torch.backends.cudnn.deterministic = True\n", "\n", "np.random.seed(seed)\n", "random.seed(seed)\n", "seed_torch(seed)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Initialize" ] }, { "cell_type": "code", "execution_count": 9, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "cpu\n" ] } ], "source": [ "# parameters\n", "num_frames = 10000\n", "memory_size = 1000\n", "batch_size = 32\n", "target_update = 150\n", "epsilon_decay = 1 / 2000\n", "\n", "# train\n", "agent = DQNAgent(env, memory_size, batch_size, target_update, epsilon_decay, seed)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Train" ] }, { "cell_type": "code", "execution_count": 10, "metadata": {}, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAABIYAAAE/CAYAAAAzEsgaAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/YYfK9AAAACXBIWXMAAAsTAAALEwEAmpwYAACMDElEQVR4nO3dd3xb9bk/8M8jyXvLcZzEGVZCQkiAkBCcUqCUDV0UbtsftKWLltKW9nbc24b2dtzb2wVdtwVKWd1AKaOlZYQ9A3EW2cuJndgZ3lOSrfX9/XHOkWVZsrXn5/165RX76Ej6HsfR0Xn0DFFKgYiIiIiIiIiI8o8p3QsgIiIiIiIiIqL0YGCIiIiIiIiIiChPMTBERERERERERJSnGBgiIiIiIiIiIspTDAwREREREREREeUpBoaIiIiIiIiIiPIUA0M5TEROFpGtIjIsIl9K93qIiIgymYi0icjF6V4HERFlHxH5pojco3/dKCJKRCzpXhdRJBgYym1fB/CSUqpCKfWrdC8mmIjcJSL7RMQnIp8IcftXROSEiAyKyH0iUhRwm1VEHhMRu4gcFpEPB933IhHZKyIOEXlRRBYE3CYi8hMR6dX/3CIiktSDTRERKRSRh/WLGyUi7wy6/csickhEhkTkmIj8wjhhich8ERkJ+qNE5Gthnus/RWSnHnhsFZH/TP4REhERERFlHqXUD5VSn073OohiwcBQblsAYFe4G0XEnMK1hLINwOcBbAm+QUQuA7AWwEUAGgEsBPDfAbvcDsAFoB7ARwD8RkSW6/edAeBRAN8GYAWwCcBfA+57A4D3A1gB4HQA7wHw2YQdVRwS9KnCawA+CuBEiNv+CWCVUqoSwKnQfgZfAgCl1BGlVLnxB8BpAHwAHgm3XAAfA1AD4HIAN4nINQlYPxEREREREaUIA0M5SkReAHABgNv0zI8lIvJ7EfmNiDwpInYAF4jIu/VysyERaReR7wU8hpEC+Un9tn4RuVFEzhKR7SIyICK3BT3vp0Rkj77vusBMnWBKqduVUs8DGA1x88cB3KuU2qWU6gfwfQCf0J+jDMC/Afi2UmpEKfUagMcBXKff92oAu5RSf1NKjQL4HoAVIrI04LF/ppTqUEodBfAz47Ej+LmeJCIv61lMPSLy14DblovIsyLSJyKdIvJNfXuRiPxSz9A5pn9dpN/2ThHpEJFviMgJAL8TEZOIrBWRg3pG00MiYo1kfUopl1Lql/rPxBvi9oNKqQFjydACPyeFebiPAXhFKdUW5rluUUptUUp5lFL7APwDwDmRrJOIKJNN87o9Q0T+pZ8D+0TkVREx6bd9Q0SO6pmU+0TkovQeCRERhSMic0TkERHp1rPfv6Rv/56egf9X/fV8i4isCLhfyNd6/X5/nuK5HtfPGy0i8pmA276nv9//o/6Yu0Rk9XTPR5RIDAzlKKXUhQBeBXCTngGyX7/pwwB+AKACWmaJHVoAoBrAuwF8TkTeH/RwawAsBvD/APwSwLcAXAxgOYAPicj5AKDf75vQAjN1+vM/EOMhLIeWUWTYBqBeRGoBLAHgDTgm4/bloe6rlLIDOBju9qD7Tuf7AJ6BliUzF8CvAUBEKgA8B+BpAHOgBVue1+/zLQBvA3AGtAydJgD/FfCYs6BlNi2Als30JWgZTefrj9UPLUMK+nNtl6DSuWiIyIdFZAhAj76e34bZ9WMA/hDhYwqA8zBFhhoRURaZ6nX7awA6oJ3n6qGd95SInAzgJgBnKaUqAFwGoC2lqyYioojoAf1/QrsOaIBWpfBl0aoWAOBKAH+D9h79fgB/F5GCOF7rH4B27pgD4AMAfhgU4HkfgAehXZM9DuA2fZ08t1BKMDCUf/6hlHpdKeVTSo0qpV5SSu3Qv98O7UXr/KD7fF/f9xlogaQHlFJderbNqwBW6vt9FsCPlFJ7lFIeAD8EcMZUWUNTKAcwGPC98XVFiNuM2yvC3He62wcBlOvBjem4oQVw5ug/k9f07e8BcEIp9TN9+7BSaoN+20cA/I/+M+uGVhJ3XcBj+gB8Vyk1ppRyQvs5fkvPaBqDlvH0AdHLzJRSpyul7o9grSEppe7XS8mWALgTQGfwPiJyHrQLnocjfNjvQXs9+V2s6yIiyiBTvW67AcwGsEAp5VZKvaqUUtCyNIsALBORAqVUm1LqYFpWT0RE0zkLQJ1S6n/0jPtDAO4GYLRF2KyUelgp5QbwcwDF0D4wiPq1XkTmATgXwDf064S3ANyDidcDrymlnlRKeQH8CdqHEojl+YhiwcBQ/mkP/EZE1ojWnLlbRAYB3AhgRtB9AgMHzhDfl+tfLwDwf3p6/QCAPmjlSg0xrHMEQGXA98bXwyFuM24fDnPf6W6vBDCiv7GfztehHVOznub5KX37PGhZSaHMAXA44PvD+jZDt17yZlgA4LGAn+MeaCeF+gjWFzGl1AFoGT53hLj54wAeUUqNTPc4InITtOyid+uBLCKibDfV6/atAFoAPCNaM/+1AKCUagHwZWiB8i4ReVBEAl/riYgocywAMMd4v62/5/4mxt9v+6+ZlFI+6Nk+Mb7WzwHQp5QaDth2GBOvkQJ7gzoAFIuIhecWShUGhvJPcPDjfmjpivOUUlXQMkhindDVDuCzSqnqgD8lSqn1MTzWLoxHyqF/3amU6gWwH4BFRBYH3b4r1H1F60m0KNztQfedklLqhFLqM0qpOdAye+4QkZOgHfuiMHc7Bu3kY5ivb/M/bND+7QCuCPo5FusZWolmQdC6RaQEwAcRQRmZHhhbC+AipVRHEtZHRJQOYV+39YzQrymlFgJ4L4CvGuUAekbmufp9FYCfpHbZREQUoXYArUHvtyuUUu/Sb59n7KiXnc3F+Hkg2tf6YwCseusJw3wAEb2357mFUoGBIaqAFsEeFZEmaD2IYnUngJtlfDpYlYh8MNzOoo1WL4YWiCoQkWKjgSeAPwK4XkSWiUgNtN4Ovwf8PYMeBfA/IlImIudAqwP+k37fxwCcKiL/pj/+dwBsV0rtDXjsr4pIgx5x/5rx2NMRkQ+KyFz9235oL85eAP8CMEu0cfBFIlIhImv0/R4A8F8iUifaxLTvAAjZmE53J4AfGCV4+v2ujGR9+v5F+nEDQKH+cxX9tk+LyEz962UAbsZ4LyTDVQAGALw4zfN8BFq54CV6+i0RUa4I+7otIu8RbRCBABiCdg7wisjJInKhaE2qR6Fl1E4aAkBERBmhGcCQ3ti5RETMInKqiJyl336miFytt3L4MoAxAG/G8lqvlGoHsB7Aj/T35acDuB7AX6ZbJM8tlCoMDNHnoQVYhqG98X0o1gdSSj0GLYL9oN7ceCeAK6a4yzPQXtzeDuAu/et36I/1NIBboAUnDut/vhu07hIAXdDewH9OKbVLv283tKllP4AWvFmD8XphQGu2/E8AO/Q1PoGABsx6idhHwqz5LAAbRGQEWqbVvyulWvXU0EugfXp8AsABaFPhAOB/AWwCsF1/zi36tnD+T3/sZ/R/lzf1Y4hkfQCwD9rPsgHAOv1r45PvcwDsEG0q3ZP6n28G3f/jAP4YXFonIufpx234XwC1ADaKNvluRETunGJdRETZYqrX7cXQhg2MAHgDwB1KqZeg9YD4MbTG/icAzMTk11ciIsoAei+f90IbMtAK7bX7HgBV+i7/gDZ4px9aL6Cr9X5Dsb7WXwugEVr20GPQ+os+G8H9eG6hlJDI2qoQERERERER5TYR+R6Ak5RSH033WohShRlDRERERERERER5ioEhIiIiIiIiIqI8xVIyIiIiIiIiIqI8xYwhIiIiIiIiIqI8xcAQEREREREREVGesqR7AQAwY8YM1djYmO5lEBFlpM2bN/coperSvY504nmCiCg0niM0PE8QEYUWyXkiIwJDjY2N2LRpU7qXQUSUkUTkcLrXkG48TxARhcZzhIbnCSKi0CI5T7CUjIiIiIiIiIgoTzEwRERERERERESUpxgYIiIiIiIiIiLKU9MGhkRknoi8KCJ7RGSXiPy7vt0qIs+KyAH975qA+9wsIi0isk9ELkvmARARERERERERUWwiyRjyAPiaUuoUAG8D8AURWQZgLYDnlVKLATyvfw/9tmsALAdwOYA7RMScjMUTEREREREREVHspg0MKaWOK6W26F8PA9gDoAHAlQD+oO/2BwDv17++EsCDSqkxpVQrgBYATQleNxERERERERERxSmqHkMi0ghgJYANAOqVUscBLXgEYKa+WwOA9oC7dejbgh/rBhHZJCKburu7Y1g6ERERERFlExG5T0S6RGRnmNtFRH6lt6XYLiKrUr1GIqJ8E3FgSETKATwC4MtKqaGpdg2xTU3aoNRdSqnVSqnVdXV1kS6DiIiIiIiy1++htZsI5woAi/U/NwD4TQrWRESU1yIKDIlIAbSg0F+UUo/qmztFZLZ++2wAXfr2DgDzAu4+F8CxxCyXiIiIiIiylVLqFQB9U+xyJYA/Ks2bAKqNaw4iIkqOSKaSCYB7AexRSv084KbHAXxc//rjAP4RsP0aESkSERu0aH9z4pZMFJ/ekTFs7xhI9zKIKAXeONiLf7x1NN3LICKiyEXUliJRXtnfjXW7TiTr4YmIskIkGUPnALgOwIUi8pb+510AfgzgEhE5AOAS/XsopXYBeAjAbgBPA/iCUsqblNUTxeC3rxzCNXe9CZ9vUoUjEeWYhzd34Jan96V7GUREFLmI2lIAielZeverh/CLZ/fHdF8iolxhmW4HpdRrCP0CDQAXhbnPDwD8II51ESXN8cFROFxenBgaxZzqknQvh4iIiIjGRdyWQil1F4C7AGD16tUxfeLX1GjFz5/bjwGHC9WlhbE8BBFR1otqKhlRLuizjwEADvc60rwSIiIiIgryOICP6dPJ3gZg0JiEnAxNNiuUAja19SfrKYiIMh4DQ5R3ekdcAIAjffY0r4SIkk3C5bsSEVFaiMgDAN4AcLKIdIjI9SJyo4jcqO/yJIBDAFoA3A3g88lcz4p51Sg0m9DcNlU/bCKi3DZtKRlRrunRA0PMGCIiIiJKLaXUtdPcrgB8IUXLQXGBGWfMq8aGVgaGiCh/MWOI8orPp9Dv0ANDfQwMEeUD7RqDiIgotCabFTuPDsI+5kn3UoiI0oKBIcorg043vPo0siPMGCLKeawkIyKi6TTZrPD6FLYcYZ8hIspPDAxRXum1a9lCVSUFaOu1M5OAiIiIKM+tWlADs0nQzHIyIspTDAxRXukd0SaSrZpfjeFRDwYc7jSviIiSjeFfIiKaSnmRBac2VGHDIQaGiCg/MTBEecXIGFo1vwYA+wwR5TpOJSMiokissVnxVvsARt3edC+FiCjlGBiivGIEhlYagaFejqwnIiIiyndNjVa4vD5sax9I91KIiFKOgSHKK0Yp2Yp5VQDYgJooH7CVGBERTeesRitEwD5DRJSXGBiivNJnd6GqpAAVxQWoryxiKRlRjhPOJSMioghUlRbg5PoKNLcxMERE+YeBIcorvSMu1JYXAgAWWMuYMUSUBxTbTxMRUQTW2KzYfLgfbq8v3UshIkopBoYoZ7V0DeMj97yJ4dHxyWM9I2OoLdMCQ/NrS3G4b7zH0M+e2Yc7XmpJ+TqJKHnYfJqIiCLVZKuFw+XFrmND6V4KEVFKMTBEOevVAz14vaUXO44O+rf12V2oLSsCACywlqJzaAxOlxfHBpy446WD+Ne24+laLhERERGl0Vk2bThJc2tvmldCRJRaDAxRzjra7wQAtPWMl4v12sdLyebXlgIAjvQ58Mc3DsPrU+gaHk39Qokoqdh8moiIIjGzohgLZ5SxATUR5R0GhihnHR3QA0P6SHqvT6Hf4fKXki2oLQMA7D0xhAeaj0BECxyxrpwod7CUjIiIotFks6K5tQ8+Hz9VIKL8wcAQ5SwjMHSoWwsM9TtcUAqoLR8vJQOAX7/QgkGnG1etbIBSWh8iIiIiIso/TTYrhkY92Nc5nO6lEBGlDANDlLP8pWR6xlDviAsAYNUzhqpLC1BRbEFL1whWzK3CFafOBgB0DTEwRJRL+JkvERFFas3CWgDAhkPsM0RE+YOBIcpJDpcHvXYXCs0mHOl1wOtT6LVrAR+jx5CIoFEvJ/vUuTbUV2qZRJ1D7DNElDtYS0ZERJFrqC5BQ3UJmtvYZ4iI8gcDQ5STjullZGcuqIHL68OxAac/Y2iGXkoGACfPqsCcqmJccepszKwoBgB0DTNjiIiIiChfrdH7DClOLyCiPMHAEOWkDr2M7NzFMwAArT129NknlpIBwHffuwyPf/FcFFpMmFFeCBGgixlDRDmF7+uJiCgaTTYrekZcONRjT/dSiIhSgoEhyklG4+lzTxoPDPWOjEEEqCkdDwxVFBf4M4gsZhNqy4qYMUSUQziVjIiIotVkswIAx9YTUd5gYIhy0tF+JywmwfI5lSgrNKO1x44euws1pYUwm8JfKdZXFrHHEBEREVEes80ow4zyIgaGiChvWNK9AKJkODrgxOzqYljMJjTOKENrjx0lBWbUBpSRhTKzghlDRLmHtWRERBQ5EfH3GSIiygfTZgyJyH0i0iUiOwO2/VVE3tL/tInIW/r2RhFxBtx2ZxLXThRWR78TDdUlALRPfdp67ei1j/knkoVTX1mMTo6rJ4qLiFwuIvtEpEVE1oa4/Z0iMhhwrvhO0taSrAcmIqKc1mSz4uiAEx39jnQvhYgo6SLJGPo9gNsA/NHYoJT6f8bXIvIzAIMB+x9USp2RoPURxeRovxPn6P2FbDPK8NTOE/AphdMbqqe838yKIvTax+Dx+mAxs9KSKFoiYgZwO4BLAHQA2Cgijyuldgft+qpS6j0pXyAREVEEAvsMza0pTfNqiIiSa9orX6XUKwBC5lGKiAD4EIAHErwuopi5PD50Do+ioWY8Y8jrU2jvc06bMTSzshhKAT36aHsiiloTgBal1CGllAvAgwCuTOeCOJWMiIiidXJ9BSqLLSwnI6K8EG9KxHkAOpVSBwK22URkq4i8LCLnhbujiNwgIptEZFN3d3ecyyAad2JwFEoBc/VSssYZZf7brBH0GAKArmE2oCaKUQOA9oDvO/Rtwc4WkW0i8pSILE/WYjiVjIiIYmEyCZrYZ4iI8kS8gaFrMTFb6DiA+UqplQC+CuB+EakMdUel1F1KqdVKqdV1dXVxLoNoXMeAVgvuzxiqHQ8M1eqj6cOprywGAPYZIopdqFBMcM7OFgALlFIrAPwawN9DPhA/QCAiojRaY6vFoR47ujixlohyXMyBIRGxALgawF+NbUqpMaVUr/71ZgAHASyJd5FE0Tja7wQAzNUDQzVlhaguLQAAzJgmY8gIDDFjiChmHQDmBXw/F8CxwB2UUkNKqRH96ycBFIjIjOAHStQHCKwkIyKiWPj7DLUxa4iIcls8GUMXA9irlOowNohInd54FCKyEMBiAIfiWyJRdI4OOCECzK4q8W9r1LOGpislm1FeCBFmDBHFYSOAxSJiE5FCANcAeDxwBxGZpfeog4g0QTsX9SZjMcK5ZEREFKPlcypRWmhmORkR5bxIxtU/AOANACeLSIeIXK/fdA0mN51+B4DtIrINwMMAblRK8ZWUUqqj34mZFUUotIz/ei/U+wxNV0pmMZtQW1aEbmYMEcVEKeUBcBOAdQD2AHhIKbVLRG4UkRv13T4AYKd+rvgVgGuUYotoIiLKLBazCWcuqGFgiIhy3rTj6pVS14bZ/okQ2x4B8Ej8yyKK3dF+JxqqSyZsWzSzHABQN01gCNAaUDNjiCh2ennYk0Hb7gz4+jYAt6VwPal6KiIiyjFrbFb89Jn9GHC4UF06deY5EVG2irf5NFHGOTrgRENN6YRtH12zAPd9YjWq9F5DU6mvLGKPIaIcwalkREQUjyZbLQBgY1t/mldCRJQ8DAxRTvH5FI4PTs4YqiotwIVL6yN6jJkVxcwYIsohzBciIqJYnT63CoUWE5pbk9IKj4goIzAwRDmla3gMbq/yTySLRX1lEXpHxuDx+hK4MiJKByYMERFRPIoLzDhjXjX7DBFRTmNgiHKCUgrNrX34xiPbAQALakunuUd4dZXF8Cmg1+5K1PKIiIiIKEutsVmx89gQRsY86V4KEVFSMDBEWW/M48WH796AD/32DWzvGMBXL1mCcxbNiPnx6iu0BtVdLCcjygnsPU1ERPFoslnh9SlsOcw+Q0SUm6adSkaU6XYeHcIbh3rxhQsW4aYLFqOk0BzX482sLAYAdA6N4jRUJWKJRJQmwu7TREQUp1Xza2A2CZpb+/COJXXpXg4RUcIxY4iyXluPHQDwb6vmxh0UArQeQ4DWr4iIiIiI8ltZkQWnNVRhAxtQE1GOYmCIsl5rjx1mk2CeNfa+QoFmlBdBBBxZT5QjFGvJiIgoTmtsVmxrH8So25vupRARJRwDQ5T1WnvtmFdTggJzYn6dC8wm1JYV4vgAA0NEREREpPUZcnl9eKt9IN1LISJKOAaGKOu19dhhm1GW0MdcOb8Gz+7p5KdCFLPndneim+WIREREOWH1AitEwLH1RJSTGBiirKaUQmuPHY0JDgx98u2N6LO78PhbxxL6uJQfxjxe3PCnTfjvf+5K91IIAAvJiIgoXlWlBVg6q5KBISLKSQwMUVbrHh6Dw+VNeMbQ2YtqsXRWBe57vZX9SShqg043fAp4aucJHBtwpns5eY1DyYiIKFHW2KzYfLgfbq8v3UshIkooBoYoqx3SJ5IlOjAkIvjUuTbsPTGM9Qc5gYKiM+hwAwC8PoU/vNGW3sUQERFRQjTZrHC6vdh5dDDdSyEiSigGhiirGaPqG2sTGxgCgPetmIMZ5YW477XWhD825bZBpxYYmlFehAc2HIHD5UnzivIck/6IiCgBzmq0AmCfISLKPQwMUVZr7bGj0GLCnOqShD92cYEZH1mzAM/v7cKh7pGEPz7lrgE9Y+jz71yEoVEPHtlyNM0ryl8C1pIREVFi1FUUYWFdGQNDRJRzGBiirNbaY8cCaynMpuRc/H30bQtQaDbh/g1HkvL4lJuMjKGLTpmJFXOr8LvXW+HzMW2FiIgo262xWdHc1gcvz+tElEMYGKKs1tab+IlkgeoqinDOSbV4Zncnm1BTxAb0wFB1SSE+da4Nh7rteHl/d5pXlb/4P5eIKLOIyOUisk9EWkRkbYjbq0TknyKyTUR2icgn07HOUJpsVgyPerDvxHC6l0JElDAMDFHW8vkU2nodWJjEwBAAXLp8Fo70ObA34A2A2+vDfa+1+jNDiAINOt0QASqKLbji1Nm4bHk9SgvN6V5WXuJUMiKizCIiZgC3A7gCwDIA14rIsqDdvgBgt1JqBYB3AviZiBSmdKFhrLHVAgA2tHI4CRHlDgaGKGsdG3TC5fElNWMIAC4+pR4iwDO7Ov3bHn/rGP7nX7vxx/VtSX1uyk6DDhcqiwtgMgkKLSb89rrVWLOwNt3LIiIiygRNAFqUUoeUUi4ADwK4MmgfBaBCRARAOYA+ABkxyWFOdQnm1pSwzxAR5RQGhihrtSZpVH2wuooinDm/Bs/sPgEAUErhXn1S2aNbj7LEjCYZdLpRXVqQ7mWQjv9HiYgySgOA9oDvO/RtgW4DcAqAYwB2APh3pZQvNcubXpPNiubWPp5fiChnMDBEWastRYEhALh0eT12HRtCR78DG1r7sPv4EJoarWjtseOt9oGkPz9llwGnG1UlDAxlAlaSERFlnFAvzcERlssAvAVgDoAzANwmIpWTHkjkBhHZJCKburtT18tvjc2KXrsLB7vtKXtOIqJkYmCIstahHjtKC82YWVGU9Oe6ZNksAMCzuztx32utqCktwO0fWYUiiwmPchQ5BRlkYIiIiCicDgDzAr6fCy0zKNAnATyqNC0AWgEsDX4gpdRdSqnVSqnVdXV1SVtwsCa9zxDLyYgoVzAwRFmrrceOxtoySAq6y9pmlGFJfTn+/OZhPLunEx9eMx91FUW4dPks/HP7Mbg8GZPdTBlg0MHAUCZhoj8RUUbZCGCxiNj0htLXAHg8aJ8jAC4CABGpB3AygEMpXeUUGmtLUVdRhGY2oCaiHMHAEGWttl5HSsrIDJcum4WD3XaYRfCxsxsBAFevasCAw40X93WlbB2U+dhjKHNwKhkRUWZRSnkA3ARgHYA9AB5SSu0SkRtF5EZ9t+8DeLuI7ADwPIBvKKV60rPiyUQETTYrNrDPEBHliGkDQyJyn4h0icjOgG3fE5GjIvKW/uddAbfdLCItIrJPRC5L1sIpv7m9Phzpc6BxRmnKnvPS5fUAgPecPhv1lcUAgPNOmoEZ5UV4dEtHytZBmU0pxR5DGYbv2YmIMotS6kml1BKl1CKl1A/0bXcqpe7Uvz6mlLpUKXWaUupUpdSf07viydbYrDg+OIqOfme6l0JEFLdIMoZ+D+DyENt/oZQ6Q//zJACIyDJo6aDL9fvcISLmRC2WyHB8YBRen8ICa+oyhk5rqML33rsMX798vMTdYjbhyjPm4IW9Xei3u1K2FspcI2MeeH0K1SWF6V4KASkpNSUiovzTZLMCYJ8hIsoN0waGlFKvAIj0Fe9KAA8qpcaUUq0AWgA0xbE+opDa+x0AgLnWkpQ9p4jgE+fYMKd64nNetbIBbq/Cv3YcT9laKHMNOt0AwIwhIiKiHLZkZgWqSgoYGCKinBBPj6GbRGS7XmpWo29rANAesE+Hvm2SdI2XpNzQ3qcFhubVpK6ULJzlcypxcn0FHmM5GQEYcOiBIfYYyhiK7aeJiCjBTCbBWY1WNLcxMERE2S/WwNBvACwCcAaA4wB+pm8PlbMf8h15usZLUm5o73fAbBLMripO91IgIrhqVQO2HBlAa4893cuhNBtixlBGYSEZERElyxqbFa09dnQNjaZ7KUREcYkpMKSU6lRKeZVSPgB3Y7xcrAPAvIBd5wI4Ft8SiSZr73NiTnUxLObMGKz3/jMaIAJmDQVYt+sEnsrD8roBPTDEqWRERES5bc1Crc/QBpaTEVGWi+mqWkRmB3x7FQBjYtnjAK4RkSIRsQFYDKA5viUSTdbe78iIMjLDrKpinHvSDDy69Sh8PpatAMAtT+/Fj57am7DHG3V7sfPoYMIeL1nYYyjzcCoZERElw7LZlSgrNLPPEBFlvUjG1T8A4A0AJ4tIh4hcD+AWEdkhItsBXADgKwCglNoF4CEAuwE8DeALSilv0lZPeau9z5lRgSFAa0Ld0e/EpsP96V5K2o2MeXCox44jfQ70jIwl5DF/8dx+vP/21zGo9/DJVEaPIU4lyxCsJSMioiSxmE04s9HKwBARZb1IppJdq5SarZQqUErNVUrdq5S6Til1mlLqdKXU+5RSxwP2/4FSapFS6mSl1FPJXT7lI6fLi56RMcxL4USySFy2fBZKC814bCvLyXYfG/Jnabx1ZCDux/P6FB7bchQen8Lu40NxP14yDTrdKLSYUFyQGWWORERElDxrbFbs6xxGv92V7qUQEcWMVy6UdTr0UfXzrJmVMVRWZMHlp87Cv7Yfh9OV34lyO/SSL5MAW9vjz6B6vaUHXcNa5tHeE5keGHKhqqQAIkxVyRSsJCMiomRpsml9hjZyOhkRZTEGhijrtOuBobkZVkoGAB9umo/hUQ9+t7413UtJq51HB1FfWYTlc6qwNQEZQ49u6UBlsQU1pQXYkwUZQ9XsL5QxhLVkRESURKfPrUKhxcRyMiLKagwMUdZp73MCQMaVkgHA6kYrLj5lJn7z4kH05XFK8Y6jgzitoQor51djW/sAvHE05LaPebBuVyfes2IOls+pwp7jwwlcaeINONx533haRC4XkX0i0iIia6fY7ywR8YrIB1K5PiIiokQpspixcl41mpkxRERZjIEhyjrtfQ4UF5hQV16U7qWE9I3Ll8Lu8uBXzx9I91LSwj7mwcHuEZyqB4bsLi/2d8YezHl65wk43V5cvbIBS2dVYF/nMDxeXwJXnFiDTndej6oXETOA2wFcAWAZgGtFZFmY/X4CYF3SF8VaMiIiSqI1Nit2Hh3EyJgn3UshIooJA0OUddr7HZhbU5qxPVwW11fg/501H39+8zDaeuzpXk7K7T6uNZ4+raEKK+fVAMCkcjKlFN442IsvPrAV1927AdfduwGf/sMmf/+oQI9u7cB8aynOXFCDU2ZXwuXxoa03c3+uAw43KvM7Y6gJQItS6pBSygXgQQBXhtjviwAeAdCVzMVk6MsEERHlkCZbLXwK2MzJtESUpRgYoqyjjarPvDKyQF+5eDEKzCbcum5fupeScjs6tMbTpzVUYUFtKWpKC7D1yPgbpef3dOJ9t72Oa+9+E6+39GBkzIPeERee29OJTW0T31AdH3Ri/cFeXL2qASKCU2ZXAgB2Z3A5mdZjKK9H1TcAaA/4vkPf5iciDQCuAnBnCtdFRESUFKsWVMNiEmw41JvupRARxYSBIco67f2OjJtIFmxmZTE+dW4jnthxHMcGnOleTkrtPDqImRVFmFlZDBHByvk12No+AABYf7AHn/7jJoyMefDDq07D+rUX4rHPn4PfXncmAMAdVCK24VAflAIuWz4LALBoZhksJsnYBtRurw8jY5587zEUKkcnuJjrlwC+oZSacnyfiNwgIptEZFN3d3fMC1KsJSMioiQqLbTg1IYqNqAmoqzFwBBllUGHG8OjHszLwIlkwa5aORcA8NyezjSvJLWMxtOGVfOr0dI1grYeO77y17dgm1GGf33xXHx4zXwUF5gBAGaTFksIblLt8miBIqM0q8hixkkzyzM2MDTkdANAXvcYgpYhNC/g+7kAjgXtsxrAgyLSBuADAO4QkfcHP5BS6i6l1Gql1Oq6urqYFsNKMiIiSoU1C63Y1jGAUfeUn3kQEWUkBoYoqxij6jNxIlmwk2aWY2FdGZ7ZlT+BIYdrvPG0YeV8rc/QR+/dgH67G7++diXKiiwT7mcxa5fv7qDAkNunBYYspvHL+1NmV2JvhpaSDeqBoTzPGNoIYLGI2ESkEMA1AB4P3EEpZVNKNSqlGgE8DODzSqm/p3ylRERECbLGZoXbqyb1VSQiygYMDFFWae/TAkNzsyBjCNBKoN481ItBhzvdS0mJ3ceG4NMbTxtOn1sFEaCj34mb37UUy+dUTbqfxaS9FHmDSsmMDKKJgaEKnBgaRb/dlYxDmGDQ4caL+7qgVGSlSANGYCiPM4aUUh4AN0GbNrYHwENKqV0icqOI3JieNaXjWYmIKJ+cucAKEbCcjIiyEgNDlFXGM4ayIzB06bJ6eHwKL+zLj6yhHUf1xtNzx4M/FcUFOKvRiitOnYVPvL0x5P2MjCFPcMaQVw8MmcdfqpbO0hpQp6Kc7NZn9uKTv9uI7z2+Cz7f9NEFZgxplFJPKqWWKKUWKaV+oG+7Uyk1qdm0UuoTSqmHk7UWTiUjIqJUqCopwCmzKtHcxgbURJR9GBiirNLe50RlsSVrLrxXzK3GzIqivCkn294xiLqKItRXFk/Y/uBn3oY7PrIKEuYq3cgICg4MebyhS8kAYHeSA0M+n8IzuzpRU1qAP7xxGP/+17f8PY/CMTLDqrPk95OIiIgSp8lmxebD/dO+XyAiyjQMDFFWyYaJZIFMJsEly+rx8v7uvGhG2Nzah9ULaiZtN5kkbFAIGC8l8wSVkhmBIiOjCADqKoowo7wIe08Mo6VrBN98bAd+/uz+RCx/gm0dA+gaHsN33rsMa69Yin9uO4Yb/7x5UoPsQIP+5tN5Pa4+47CSjIiIUmGNzYpRtw87jw2meylERFFhYIiySnufIysmkgW6bPksOFxevN7Sk+6lJFVHvwNHB5xYY7NGfd/wGUNGj6GJL1WnzK7AP7cdw8U/fxn3bziC2144gK7h0RhXHtozuzthMQkuPLkeN56/CP/9vuV4YW8XfvNSS9j7DOgZQ5XFlrD7UGoJ55IREVGKnKW/B2KfISLKNgwMUVbpGhrDrKri6XfMIG9bWIuKIgvW7TqR7qXErW+Khs8b27Q3QU222qgf12QSmGQ8EGTw+nwQGR9nb3jnyTNRWVKAL120GA999mz4FPD4W8ET0ePzzK4TeNvCWn8j6Y+dvQDvWzEHv3juADYfDv2Gb9DpRkWRZUJPJEq/SJuHExERxWNGeREW1ZUxMEREWYdXL5Q1lFJwuL0oKzKneylRKbSYcMHSmXh2dyfc3uysOff6FL7zj51Y9f1n8ZF73sTL+7snXWxvONSHymILTp5VEdNzWMwm/3h6g9unUGCa/DJ1/bk2bPzWxfjqJUvQZLNixdwqPLrlaEzPG0pL1wgOdttx6fJ6/zYRwQ+uOhUN1SX40gNvhZw0N+B0oZL9hTIKm08TEVEqNdlqsbGtb8rScyKiTMPAEGUNl9cHr0+htDD7ynTet2IO+h1uvLK/e8L2n67bh9tfDF+alAnGPF586cGt+OMbh3H58llo6RrBx+9rxlV3rJ/QN6m5tQ9NNuuk7J5IWUwCr3dy8+lIHu+qlQ3YfXwIe08kpiH1M7u17K6LT6mfsL2iuAC/unYlOodG8d//3DXpfkNON6rzeFQ9ERFRvltjs2J41JOS6alERInCwBBlDadLC0KUFGRXxhAAnH9yHaxlhROyWtr7HLjjpRb8/Nn9aOkaSePqwht1e3H97zfhie3HcfMVS3HndWfi1a9fiO+8Zxneah/AP97SjqdreBSHeuxoiqG/kMFiksk9hnxqQuPpcN67Yg4sJsFjAT/fp3Ycx74TwzGt5ZldnTh9bhXmVJdMuu2MedX40Fnz8PSuE5OaZQ843FkzMS+f8DNbIiJKlSb2GSKiLMTAEGUNp56dUlKYfYGhArMJ71sxB8/u6fRPrvrD+jaICIotJtzy9N6w9zX2T4f//uduvNbSg1s/cDo+e/4iAFpp3CfPacQpsytx72utUEr53/ysiaG/kMFiNsETVErm8aoJo+rDqS0vwjtPrsPf3zoKj9eHW9ftxef+sgW/euHApH2PDzqx69ggdh0bxOFe+6TbO4dG8Vb7AC5dVj/pNsPbF9XC4fJi17GJnwYOMGMo47CSjIiIUmlOdQnmWUsYGCKirMLAEGUNh54xVJqFgSEAuHpVA1weH57ccRwjYx78dWM73nXabHzunYvwzO5Of/PmQBsO9WL1/z6LP77RlvL1PrH9OB5oPoLPnr8QH1w9b8JtIoLrz7Vhf+cIXm/pRXNrH0oLzVg+pzLm57OYZFLzaS1jKLKXqatXzUXn0Bg+fPcG3P7iQRSaTejod07YZ9Dhxvm3vIR3/+o1vPtXr+H8W1/Cw5s7JuzzyBbt+0uXzwr7XE2N2qeBG1p7/du8PoWuoVHUcFQ9ERFRXmtqrEVzWx+HHxBR1mBgiLKGUUpWnIWlZABwWkMVFtWV4bEtR/G3Te0YHvPg+nNtuP7chaivLMIPn9wz4Q1Ev92Ff3/wLbi9Cr99+dCksqVkau9zYO2j23HGvGr8x6Unh9znvStmY0Z5Ie597RCaW/tw5oKauKZxhSwl8/pQEGHPoguXzkRFsQXNbX344oUn4aqVDTgaFBhq7bXD5fXhSxeehN9edyZWzK3Crev2wuHyANB+5r956SAuXDoTS+rDN9GeWVkM24yJU0e2HOnH0KgHb1sYe9YUJQfflxMRUSqtsVnRZ3fhYHdmtgogIgrGwBBlDaOULFszhkQEV6+ai+a2Ptzx0kGsml+NM+ZVo6TQjK9dcjK2HhnAfa+3wetTUErhPx/ejl77GL544Uk4OuDEs7s7E7KO11t68I+3joadkNbe58AXH9gKKODX165EQZhgT5HFjI++bQFe3NeNvSeG4w6IWMymScEvj0/BHEGPIUALGN76gdPxf9ecga9dejLm1pSgZ2RsQoPs9j4HAOBdp8/GZctn4b/eswydQ2O499VWAMCvX2iBfcyDtVcsnfb51tisaG7tg08PZj2z6wQKzSa88+S6iNZLKcKxZERElGJGn6ENLCcjoizBwBBlDWeWl5IBwPtXNgAAuofH8Klzbf7t/3bmXKxeUIPv/2s3LvjpS/jaQ9vw3J5OrL3iFHz54iWYZy3Bfa+3+vdXSvmDHNH63uO78O8PvoV33PIi7nrlIDYf7sfmw/149UA3vvjAVrzzpy9h59FB3PKB0zHPWjrlY31kzQIU6oGjeBpPA+GbT4caVx/O5afOxpVnaD/jhhqtcfSxgfGsofZ+7Wc2r0Y7rrMarbh0WT3ufPkgthzpx5/ebMOHVs+bMlvI0GSzYmjUg70nhqGUwrpdnXj7SbWoKGaPISIiony2oLYUMyuK2GeIiLJG9s39przlyPJSMgBoqC7BuSfNQGuPHZcH9LAxmwR//ezZeHb3Cfz2lUN4dOtRXLh0Jj51TiNEBJ94uw3f/9dubO8YwLLZlVj76A48vLkDnz1/IdZevhQSRVZEv8ON1QtqYDELfvjkxKbX5UUWXH+uDZ88pxGzqyZP5ApWV1GEq1Y24Mkdx3H63KrIfxAhWMwhegxFOK4+lAZ9olhHvxML68oBAO19TljLClFWNP7S940rluLSX7yCj9y9ARaTCV+5ZElEjz8+daQXJhNwpM+BG/UG3URERJS/RARNNis2HNL6DEXzPo2IKB2mDQyJyH0A3gOgSyl1qr7tVgDvBeACcBDAJ5VSAyLSCGAPgH363d9USt2YjIVT/nG6tT4wpYXZHc/89bUr4fL6JvXjMZsEl5+qlTjtPTEM24wy/xuJD62ei188ux+/eekgXB4fnt/bhVXzq/Hblw+hb8SFH119WkT9fZRSGHK6cWZjDW6+4hTsPTGEzqExAIBJgBXzqlEZZcbL9963HF+44CQUWeIL2FlMIaaSRdF8OpiRMXQ0IGOoo9+BeTUTA16L6spxbdM8/PnNI/jShSehvrI4osefW1OKhuoSNLf1YWjUAxHg4mUzY1orJQ/fihMRUTqssVnxr+3H0dHvnDYDm4go3SK5wv49gNsA/DFg27MAblZKeUTkJwBuBvAN/baDSqkzErlIIgBwurSgQUkWZwwBQE3Z1FOrRASnzJ443auiuAAfXD0Xv3u9DSLA/77/VHxkzXz88rkD+L/nD6BnZAzffe9yNM4om/KxR90+uLw+VJVowZ+lsyqxNPzwrYiUFJoxvzb+NzwWc5jm0xH2GAo2q7IYZpNMaEDd3ufA8obJmU3/eelSLLCW4SNvmx/Vc6yxWfHKgW4c7nVg1fwazKyILKhEREREua3JpvVe3NDax8AQEWW8aT+KV0q9AqAvaNszSimP/u2bAOYmYW2UZ0bGPOgaGg17uzE5qiSLewzF4/pzbVgxtwq3f3gVPvq2BRARfOWSJfj+lcvxeksvLvjZS7jxT5ux5Uh/2McYdLoBwB8YyiThxtXHWkpmMZswq7LYnzHk9SkcHXD6+wsFqiotwGfesTDqbLQmmxU9Iy7sOjaES5fVx7ROSg2ODCYiolRaPLMc1aUFaG7tTfdSiIimlYianE8B+GvA9zYR2QpgCMB/KaVeDXUnEbkBwA0AMH9+dJ/SU2669em9eHrXCbz2jQtDTsIazfKpZPGaW1OKf9x07qTt153diMtOnYU/rG/Dn988gqd3ncBZjTX4zHkLcfEp9TAFBFYyOzAUopTMG13z6WANNSX+jKHOoVG4vQrzrNP3TorUmoBJbJcujzP1ipKCbR2IiCgdTCbBWY1WTiYjoqwQ11QyEfkWAA+Av+ibjgOYr5RaCeCrAO4XkcpQ91VK3aWUWq2UWl1Xx/HOBHQNj6FzaAyvHugOebvD5YXFJGHHp+ezmRXF+M/LlmL92gvx3fcuw7GBUdzwp8343F82T9gvowNDoZpP+2JvPg0Ac6tL/BlDxhS3UBlDsWqsLUVdRREWzyyHbZoyPiIiIsova2xWHO514MRg+Ix4IqJMEPMVtoh8HFpT6o8oPUdfKTWmlOrVv94MrTF1ZCN+KO+NjGmlYo9sORrydofLm7dlZJEqK7Lgk+fY8PJ/vhPvWzEHr7dMTF/O5MCQOcy4ekuMPYYALWPo+KATbq8P7XrmUCLr/EUEt3zgdPzgqtMS9piUHKwkIyKiVFuj9xlqbmPWEBFltpgCQyJyObRm0+9TSjkCtteJiFn/eiGAxQAOJWKhlPuMcfTP7u70BzACjbq9Wd94OlUsZhMWzyzHyJgHYx6vf3smB4YKzGFKyeLIEGuoLoFPAScGR9He54AIMKc6sQ2iLzh5pn90PWUe4VwyIiJKk1NmV6C8yMI+Q0SU8aa94hKRBwC8AeBkEekQkeuhTSmrAPCsiLwlInfqu78DwHYR2QbgYQA3KqUYIqeI2Mc8mFNVDJfHh6d2HJ90u8Plzdv+QrEwpp8NOMaDbJkcGArVfNrtja+ULHBkfXu/A7Mqi1Fk4e8QERERJZ/FbMKZC2rQzD5DRJThpm0+rZS6NsTme8Ps+wiAR+JdFOUnh8uLJpsV248O4tGtR3FN08Sm5E63F8XMGIqYVQ8M9dldqK/UsmSMwFBFcQYGhkKMq/f6VMzj6gEtYwgAjvY70dEXeiIZ5QdWkhERUTo02ay4dd0+9Nld/vdmRESZhl18KWM4XB6UFVlw9coGNLf2+ZsFG5zMGIpKTan25qPf7vJvG3K6UVFsiSsLJ1ksJhM83qBSMp+COY6pZHOqxzOGjvQ5MDeBE8koO3AqGRERpdMavdx8I/sMEVEGY2CIMsbImBYYev/KBgDA37dObELtcHlQWjhtkhvp/BlDjvHA0KDTnZFlZIBeSjap+bQPBXEEsYoLzKirKEJrjx2dw6PMGMpjit2niYgoDU6bW4Uii4nlZESU0RgYoozg9SmMun0oLTRjbk0pVs6vxkv7J46td7p9LCWLghEYCswYyujAUKhx9d74ppIBWjlZc2sflErsRDLKDkwYIiKidCqymLFyfjUDQ0SU0RgYoozgcGmj6suLtIyg2VXFGAjIdAEAp8vDUrIoVJdqAaA++8Tm05kaGDKbTJMyhtze+ErJAK0B9dEBfVR9DUvJiIiIKLWabLXYdWwQw6OTp+4SEWUCBoYoIxij6o1SsYqiAgyPeibs4+S4+qgUmE2oLLagP0tKyQrMMmlcvdfni6v5NADMrR4PBjFjKH+xkIyIiNJljc0KnwI2H+5P91KIiEJiYIgygn1MCwKVFWmBn4piy6TAkMPlRQkzhqJiLStEX5aUkplNAm+IUrJ4G2UbI+sLzOKfzkb5g82niYgyj4hcLiL7RKRFRNaG2eedIvKWiOwSkZdTvcZEWjm/GhaTYAPLyYgoQ7GTL2UE+9jEjKHKkgI43V64vT4UmLX45aibU8miVVNWmEUZQya4fZOnkhn//rGaqweGGqpLMnIaGxERUT4RETOA2wFcAqADwEYReVwptTtgn2oAdwC4XCl1RERmpmWxCVJaaMFpc6vYZ4iIMhYzhigj2PUeQ2WF4xlDAPxZQ26vD26vYilZlKyl4xlDo24vXB4fKjM0MGQxhWg+7fPBEm/GULVWPsYysvzGoWRERBmjCUCLUuqQUsoF4EEAVwbt82EAjyqljgCAUqorxWtMuDW2WmzvGIBTb59ARJRJGBiijGA0ny7Tm09XFGvBC6NJn9GDiKVk0akpK/RPJRt0aj/LTM0YMsbVG2PFlVJwe1X8gSE9Y4iBofwkrCUjIso0DQDaA77v0LcFWgKgRkReEpHNIvKxlK0uSdbYrHB7Fba2s88QEWUeBoYoIxilZIE9hoDxjKFRNwNDsbCWFaLPkSWBIb1kzKtPJjMGlFniLCUrL7LgU+fY8L4Vc+J6HCIiIkqIUBH74LxOC4AzAbwbwGUAvi0iSyY9kMgNIrJJRDZ1d3cnfqUJdGZjDUTAcjIiykjsMUQZwcgY8k8l0wNDQ0EZQ+wxFJ2a0kKMun1wurwZHxgy+v94fAoWs1Y+GLg9Ht9577K4H4Oym+JcMiKiTNEBYF7A93MBHAuxT49Syg7ALiKvAFgBYH/gTkqpuwDcBQCrV6/O6Bf6yuICLJtdycAQEWUkZgxRRhgxMoaM5tP+UjItYGTUY7PHUHSsZdrPsc/hwqAjswNDxlh6j54qZGQOxTuunoiIiDLKRgCLRcQmIoUArgHweNA+/wBwnohYRKQUwBoAe1K8zoRrslmx5Ug/XB7f9DsTEaUQA0OUERz6uHqjVMwIDA3pWS5Ot3E7k9yiUVNaCADoG3FlQcaQXkqmN6A2GlFbTHyZyjbTjSEWkStFZLs+hniTiJybjnUSEVHqKaU8AG4CsA5asOchpdQuEblRRG7U99kD4GkA2wE0A7hHKbUzXWtOlDU2K0bdPuw4OpjupRARTcCrbMoIdpcXhWYTCi1aECC4xxBLyWJjLdMDQ47MDwwZmUHGyHrjbwszhrJKJGOIATwP4HGllBKR0wE8BGBpMtfFqWRERJlDKfUkgCeDtt0Z9P2tAG5N5bqS7axGKwCtz9CZC2rSvBoionH8KJ4ygsPl8TeeBoDyoMAQS8liU6MHhvrt44GhzB1XP7H5tPE3M4ayzrRjiJVSI0r5QzVlmNx0NGE4lIyIiDJFbXkRTppZjubW3nQvhYhoAl5xUUawj3n9jacBoMBsQkmB2T+u3smpZDGxGqVkemCoosiSkGbOyWCMpTeaTht/xzuunlIukjHEEJGrRGQvgCcAfCpFayMiIkqrJpsVm9r6/R+AERFlAgaGKCPYxyZmDAFaORkzhuJTWVIAkwD9DheGnO6MzRYCxkvGjN5C/owhlpJlm0jGEEMp9ZhSaimA9wP4fsgHyqIxxERERJFYY7NieMyDPceH0r0UIiI/BoYoI9hdngkZQ4AW1Bge47j6eJhNgurSQn/GUKb2FwImjqsHALfRfNrMl6ksE8kYYj+l1CsAFonIjBC33aWUWq2UWl1XVxfTYiRknIqIiCg9mmxan6ENHFtPRBmEV1yUERwub8iMoSGnnjHEUrKY1ZQWoF9vPl1dmrmBoQI9AOTRm04bf7OULOtMO4ZYRE4S0br/iMgqAIUA2HCBiIhy3uyqEsy3lrLPEBFlFE4lo4xgH/Ogtqx0wraK4gIMOlwAtFIykwCFzB6JmrVsPGPopJnl6V5OWP6MoUnj6hkYyiZKKY+IGGOIzQDuM8YQ67ffCeDfAHxMRNwAnAD+X0Az6iStK5mPTkREFLkmmxXP7+mEUgrCKQlElAEYGKKMoGUMTfx1rCi2oKPP4b+9tNDCk2cMakoLcbjXkfGlZMa4eqOUzMMeQ1lrujHESqmfAPhJKtbClwwiIso0TTYrHt7cgZauESyur0j3coiIWEpGmcE+5pnUP6iy2IKh0fFSsmI2no5JbXkh+hzZ0GPIGFfvm/A3x9UTERFRLlnDPkNElGF4xUUZwe7yhMgYKhgfV++aHDiiyNSUFqJ3ZAxjHl9GTyUr8I+rD24+zZQPip+aPBiNiIgoLeZbS1FfWYRmBoaIKEMwMERp5/UpjLp9ITOGxjw+jHm8cLq9DAzFyFpWCL0qK6MzhozpY5N7DPFlimLHsCIREWUaEUGTrRbNrX1Icos9IqKITHvFJSL3iUiXiOwM2GYVkWdF5ID+d03AbTeLSIuI7BORy5K1cModDpdWLlYeImMIAIZHPXC4WEoWq5rSQv/XmRwYGh9XHzSVjBlDlAB8301ERJmkyWbFiaFRtPc5070UIqKIMoZ+D+DyoG1rATyvlFoM4Hn9e4jIMmijiZfr97lDRHg1T1NyuLRR9KWFk5tPA1pgyOlixlCsrGXZERjyN5/mVDJKIDafJiKiTDTeZ4hj64ko/aYNDCmlXgEQXAB7JYA/6F//AcD7A7Y/qJQaU0q1AmgB0JSYpVKuGhnTMobKiiYGfsYzhtxwur0oYcZQTGqyJDA0njEUNJWMpWRERESUY06qK0dNaQH7DBFRRoj1iqteKXUcAPS/Z+rbGwC0B+zXoW8jCssxFlnGUAkzhmJizZJSsgKjx1BQKVkBS8koAVhJRkREmcRkEpzVaOVkMiLKCIn+KD7UFVzI9+MicoOIbBKRTd3d3QleBmUTu95jqKwwOGPICAwxYygeNWXjwaBMDgwZGUNe38RSMjNLySgOwvbTRESUoZpsVhzpc+D4IPsMEVF6xRoY6hSR2QCg/92lb+8AMC9gv7kAjoV6AKXUXUqp1Uqp1XV1dTEug3KB0Xw6eFx9pV5KNqQ3n2aPodiUF1n8WTeZPa5eezkyxtQbpWRGJhERERFRLlljqwUAlpMRUdrFesX1OICP619/HMA/ArZfIyJFImIDsBhAc3xLpFxn10vJgnsM+QNDTrdeSmaZdF+anoigprQQFUWWjM6+MaaPeY1SMq/2dyavmbIHxwETEVGmWTanEuVFFgaGiCjtpr3SFpEHALwTwAwR6QDwXQA/BvCQiFwP4AiADwKAUmqXiDwEYDcAD4AvKKW8SVo75Qi73nw6uMdQuV5KNuh0w+X1sZQsDtayQgybPelexpSM6WPBGUMcV0/x4FQyIiLKVGaTYHVjDQNDRJR20waGlFLXhrnpojD7/wDAD+JZFOUXuz6uviwoMGQ2CcoKzegaGgMAlpLFwVpWmPGZNxaj+bR3YsZQAaeSERERUY5qsllxy7596B0ZQ215UbqXQ0R5irU5lHYOI2OoaHLgp6K4AJ3DowCAYgaGYva1S0/GmDuzk/fCjas3M2OIEoCFZERElInW2KwAgI1t/bj81FlpXg0R5St+FE9pZ3d5UWgxhWwyXFFsQaeRMcRSspiduaAGbz9pRrqXMSWjQXZwYIgZQ0RERJSrTmuoRpHFxHIyIkorXnFR2jlcnkmj6g2VJQXoGtIyhlhKltsmj6tn82kiIiLKbYUWE1bNr0FzW2+6l0JEeYyBIUq7kTHPpMbThopiC3rtLgAsJct14+Pq9R5D/nH1DAxR/DiUjIiIMlWTzYrdx4YwNOpO91KIKE8xMERp5xjzThpVb6jQR9YDLCXLdSaTQCQwY0jBbBIIx0pRHPj7Q0REmW6NzQqfAjYf7k/3UogoTzEwRGlnd02dMWQoYcZQziswmfzj6t0+H8vIiIiIKOetnF8Di0mw4RD7DBFRejAwRGnncHlRXjR9YIg9hnKfxSzw+rRSMq9XoYCBIUoUlpIREVGGKik04/S5VWhuZZ8hIkoPBoYo7exjnrBBn8qAUrJilpLlPLNJ/BlDHp9ixhDFjb9BRESUDZpstdjeMQiny5vupRBRHmJgiNLO7vKgLEzGUOWEjKHQ+1DuKDCb4PEZzad9KDDzJYqIiIhy35qFVnh8CluPsM8QEaUer7oo7Rxj3rAZQxOaT7OULOeZTTKh+bSFE8koQVRALdnDmztgH/OkcTVEREQTnbmgBiYBNrSyzxARpR4DQ5R2U2UMGT2GRIAiC39dc11BQCmZ26tgMfHfnOITPJRs8+F+/MfftuHbf9+ZngURERGFUFlcgGVzKtHMwBARpQGvuiitvD6FUbcPZWGnkmkZQyUFZo6dzgNm83jGkNfnY8YQJZzDpWUKdQ2PpXklREREEzU11mLLkX64PL50L4WI8gwDQ5RWxkVaWVG4UjItYFTCxtN5QRtXr70ZcrP5NCWQ4lQyIiLKcE02K8Y8Puw4OpDupRBRnmFgiNLKPqZNXgjXWNofGGJ/obxgCcwY8ioUsJSM4sTQIhERZYuzGmsAsM8QEaUer7oorezTZAxVlmilZGw8nR/MJlPAuHqWklHiMGGIiIgyXW15ERbPLGefISJKOQaGKK0c02QMlRdaIMJSsnxRYBb/uHqt+TQDQxSfcL3JFENFRESUgZpsVmxq6/dnUBMRpQIDQ5RW02UMmUyC8kILS8nyROC4eq9PwWLmSxQllrC4jIiIMliTzYqRMQ92HxtK91KIKI/wqovSyt98OkzGEKD1GWLGUH6Y0Hza62PzaUoYxe7TRESUBZpsVgDAhtbeNK+EiPIJA0OUViN6KVm4jCEAWN5QhZNnVaZqSZRGwRlDBewxRHEKU0lGRESUkWZXlWC+tZR9hogopcKnaRClgGNMyxgK12MIAO7+2OpULYfSzGIWON1aYMjtUyjjVDIiIiLKM2tsVjy3pxM+n4KJ2dNElAK86qKQntvdiVue3pv057G79IyhKQJDlD8sARlDHq+PzacpYYILyVhZRkREmarJZkW/w42W7pF0L4WI8gQDQxTS07tO4Pfr25L+PP6MoSlKySh/WMzjPYa05tMMDFF8gn+DWFpGRESZbo2tFgCwgeVkRJQiDAxRSE63Fw6X198cOll67S6UF1lQwOlTBG1cvZEx5Pb6YGEpGREREeWZedYSzKosZp8hIkoZXnVRSKN6iVfviCupz7O/cxgnzSxP6nNQ9jCbTPBMGFfP9I5sJCKXi8g+EWkRkbUhbv+IiGzX/6wXkRXJXpNROsYSMiIiynQigiabFc2tvZyqSUQpEXNgSEROFpG3Av4MiciXReR7InI0YPu7ErlgSg2HHhjqHhlL6vPs7xzGknoGhkhTYJKAcfWKGUNZSETMAG4HcAWAZQCuFZFlQbu1AjhfKXU6gO8DuCuJC4pmMxERUUZoslnROTSGI32OdC+FiPJAzFddSql9SqkzlFJnADgTgAPAY/rNvzBuU0o9mYB1Uoo53cnPGOqzu9Az4sKS+oqkPQdll8Bx9R4fm09nqSYALUqpQ0opF4AHAVwZuINSar1Sql//9k0Ac1O8RmYOERFRRltjswJgnyEiSo1EfRx/EYCDSqnDCXo8SjOnnjHUk8SMof2dwwCAxQwMkU5rPs1SsizXAKA94PsOfVs41wN4KtQNInKDiGwSkU3d3d1xLUrpc8mYKURERNngpJnlsJYVYsMhBoaIKPkSFRi6BsADAd/fpPeOuE9EahL0HJRC4xlDyQsMHdADQywlI4M2rj6wlIxX8Vko1D9ayPwcEbkAWmDoG6FuV0rdpZRarZRaXVdXl7DFEBFRek3Xiy5gv7NExCsiH0jl+jKBiOCsxho0t/WmeylElAfiDgyJSCGA9wH4m77pNwAWATgDwHEAPwtzv4R9EkyJZwSGepJYSra/cwQVxRbMqixO2nNQdrGYBZ4JGUPsMZSFOgDMC/h+LoBjwTuJyOkA7gFwpVKK73qJiPJEhL3ojP1+AmBdaleYOZpstWjvc+LYgDPdSyGiHJeIq64rAGxRSnUCgFKqUynlVUr5ANwNrd/EJIn4JJiSJxWlZPs6h7GkvgLC2g7SWUzin0rm9vpYSpadNgJYLCI2/YODawA8HriDiMwH8CiA65RS+1OyKvYUIiLKFNP2otN9EcAjALpSubhMYvQZ2tjGcjIiSq5EBIauRUAZmYjMDrjtKgA7E/AclEJKqYCMoeQEhpRSOMCJZBTEYjbBo5eSeXwsJctGSikPgJugfcK7B8BDSqldInKjiNyo7/YdALUA7tCnV25K1nrCxZ3ZfJqIKG2m7UUnIg3QriPuTOG6Ms4psytRUWRhA2oiSjpLPHcWkVIAlwD4bMDmW0TkDGifz7YF3UZZwOX1+SdDJWsqWc+IC/0ONxbPZONpGlegZwwppbRSMo6rz0r6NMong7bdGfD1pwF8OtXrAthziIgoA0TSi+6XAL6hlPJOlVkuIjcAuAEA5s+fn6j1ZQyzSbC6sQbNDAwRUZLFFRhSSjmgfeobuO26uFZEaTfq8vm/7rUnJzA03niagSEaZzaZoJQWnATAjCFKGCYIERFljEh60a0G8KAeFJoB4F0i4lFK/T1wJ6XUXQDuAoDVq1fn5Et9k60WL+7bi56RMcwoL0r3cogoR/HjeJrEKCOrryxCv8MFj9c3zT2iZ4yqXzKLpWQ0zugpZAQn2Xya4iXMESIiyjTT9qJTStmUUo1KqUYADwP4fHBQKF806X2GNrHPEBElEa+6aBKHywMAmFdTCqWAPkfis4b2dY6gurQAdfzkgwIYGUKjHi04WcDm00RERDklwl50pDutoQrFBSb2GSKipIqrlIxyk5ExNM9aik2H+9Ez7MLMisSOlD/QOYwlMzmRjCYyMoRG9d9BM0vJKEGCm00rFpcREaXNdL3ogrZ/IhVrylSFFhNWzWefISJKLmYM0STGRfncmhIAQK89sZPJlFLY3zmMxZxIRkH8GUNulpJRYjD2TERE2a7JZsXu40MYGnWneylElKN41UWTOFx6xlBNKYDEj6zvGh7D0KiHjadpEn+PIT04yebTlCjBGULsPURERNmiyWaFUuwzRETJw8AQTeLUA0NzrXrGUIJH1u/nRDIKo0AfT+9kYIgShL9BRESU7VbOq0GBWdhniIiShoEhmmR8KlkxCs0mdCc4Y6itxw4AWFRXltDHpexnNk3MGCpgKRkRERHluZJCM06fW80+Q0SUNLzqokmMjKHSQjNqywsTnjE0PKZNPassKUjo41L2Gy8l03oMsfk0JQqbTxMRUTZrslmxo2PQPz2YiCiRGBiiSYyMoZICM2aUFyW8x5BjzAuTAEUW/vrRRBa9lGyM4+opQSY1n+avFBERZaE1Nis8PoWtRwbSvRQiykG8MqdJjObTxQXxZQwppbDlSD98vomfzNtdHpQVWjiqniYJbj5tNvEliogoFoe6R9C49gm81T6Q7qUQUQKcuaAGJgH7DBFRUvCqiyYZdY9n9MwoL0JvjBlDL+3vxtV3rMebh3onbHe6vCgtMidiqZRjjGbTRjmjhRlDlCAsHKN889K+bgDA37ceTfNKiCgRKooLsHxOFZpbe6ffmYgoSgwM0SQOlxclBWaICGrLC9Ez4oIKbtARgUc2dwAAeuwTM47sLi9KCy0JWSvlFovebHrUo/UYKmDGEMWJY+mJiChXNNms2HpkwF9yT0SUKLzqokmcbi9K9MDNjLIiuLw+f8PoSA063XhmdycAwBF0X8eYB6WFzBiiySym4FIyXtRTcsQQ6ybKKplUrf3Svi40rn0Cxwed6V4KUVZrslkx5vFhR8dgupdCRDmGgSGaZNTlRUmh9qsxo6IQANAzHF052VM7jsOlZ32MBAeGXF6UMWOIQhgPDOkZQywlowQxsh6ZQUT5JpaM30S7f8MRAMC2dl7MEsXjrEYrAPYZIqLEY2CIJjFKyQCgtqwIANBrj64B9aNbj6KxttT/eBMf38MeQxSSv5SMGUOUKPwVojyV7F/9UbcX3VF+aJRMBzqHJw27IMo11rJCLKkvRzMDQ0SUYAwM0SQTSsnKtcBQNBlD7X0ONLf24YOr56HQYoI9KGNI6zHEwBBNZmQMjY+r50sUJZZKUBtqpRSe2H6cfR4ob336D5tw1g+eS/cyAAA7jw7ikl+8gt+8fDDdSyFKuiabFZva+uDx+tK9FCLKIbzqokmcbi9KCvRSsnK9lCyKjCFjAsr7VzagvMgCuytUjyGWktFkxhQyTiWjRAuupom3/8orB3rwhfu34OfP7I/vgYiy1GstPf6vf/7sfvz5zcPT3ufGP2/GzqOJLyc7NqD1Ltp6ZCDhj02UaZpstbC7vNh9fCjdSyGiHMLAEE3iDJgaZi2LrseQUgqPbj2Kty20oqG6BKWFZjjGgkrJ3F6UMWOIQrCYjFIyn/49A0MUn3C/QfG2XRlwaMHyY4Oj8T0QUZKlorjqV88fwH/9fWdEa3j1QE/Y/WLFAjLKJ016nyGWkxFRIjEwRJNoGUNa4MZiNqGmtAC99sgCQ28e6kNrjx3/tmouAKCs0DK5+fTYeKkaUSAjQ2hUL8+xcFw9JRibT1M8jvQ6cPuLLRnR0Hk6kkljyYgoYWZVFWNBbSkbUBNRQvGqiyZxurwoLhjP6JlRXoSe4chKye57vRXWskK8d8UcAEBZkXlC82m31weX18eMIQqJ4+op22RDgCBTDThc+L/nDmRVw+BP/L4Zt67bhxNDmZsp1tpjR5/dhV+/cABA/NlxueCn6/bhf/+1O93LIEqYpkYrNrb1ZdXrJxFlNgaGaBKne2Jz6NryQvSMTJ8xdLjXjuf2dOIja+b7A0tlRRMzhowgUWkRM4ZosvGpZMa4er5EUXySlTXBbIz4/dffd+IXz+3HKwe6072UiLi9PhzqtgMAMvla7IKfvoR33PIieka0D3QS1XA9UZKxHiP4Fe6/5W0vtuCe11oT/rxE6bJmYS0GHG4c6BpJ91KIKEfwqosmcbq8KAkIDNWUFmJo1D3t/X6/vg0Wk+C6ty3wbysrtMDhCgwMaV9zKhmFUhCUMcTm00S5y/igwOPNjMCFfcyD9S3h+9/84Ik9KVyNlo3mjnHqUHAJdz5pXPsEfv7MvnQvgyip1tiMPkO9aV4JEeUKBoZoAp9PTegxBOhZP6NTv8kcGnXjoY3teO/pczCzsti/vbTIDHtA82njawaGKBSzf1w9m09TYiWrnCYzQhrZLVOSr7720DZ8+J4NOD7oDHl7qhu9/vipvVj8rafg8mTvSOqu4VHc8+qhlJez/eqFltQ+IVGKza0pweyqYvYZIqKEYWCIJjAuyAMzhsqLLBie5tPHhza2w+7y4pPn2CZsDx5Xb4whL2PzaQphvJTMO+F7oliFnUqWpMelxNp3YhiNa5/AnhSMZd7fOQwAEz7MCJTqIKAx/t0VJmuoa3gUL+3rSuWSonbT/Vvxv0/sQUvXsH9bMoNEsWZYEWUbEUGTzYrm1j72uiOihIjrqktE2kRkh4i8JSKb9G1WEXlWRA7of9ckZqmUCkapV2DGUEWxBfYxT9gTj1IKf3ijDU2NVpw2t2rCbaWF2n0NdpaS0RSCm08zY4gSxehrEml2yv7O4SmbevJtePwiuZh5audx/e8TET3mh377Bp7Yfjy2Bfl/N+L7123rsePbf98Jb5yNiIx7P7K5A41rn8DDmzuwvWPAf/sHfvMGPvG7jdM/TgTL2HVsMOzvu8vjQ+PaJ/DQpvYIVj3RkFMrQ/fE8LN4eucJNK59Av326Ydf/OnNNgDAS/uyo18VUSI02azoGh7D4V5HupdCRDkgER/HX6CUOkMptVr/fi2A55VSiwE8r39PWcKpX5AHZgyVFVngU+O3BRse86C9z4mLl82cdFt5kRlur/Knwvt7DLH5NIVg9BRyMjBECRJLmdLOo4O49Bev4PYXWY6SCoksJWtu7cMX7t+C9j5HRAGFCevQ/472w/f1B3twYnB8StkX7t+CP715GC/s7ZoyONQzMoZzfvzChGwaQAuML7z5CX8Ppv97Xpsu9h9/24b33fa61gvpYA+O9EV/MTjm8U76uWzvGMC7f/Uabgvz+z7g0Pa/dV30fXtCNYWONGB272uHACCi5rqvt7DPCuWf8T5DLCcjovglo07jSgB/0L/+A4D3J+E5KEmMTI3AjKFyPYgTrs9Qrz75ZEZ50aTbSvWSMSNryOEvJWPGEE1mMU2cSsZx9ZQOxwa0HjPbArIzgqX7N/PFvV14bndn1PfLpKbEkYQHYq2QOO+WF3HuT16I6j7GpLlwTxkuw+nDd2/AJb94OWA/7e/P/HETfvxU+IbVtz69D0cHnLjn1YnTsnpGxiZMPesLCuR85a9v4cN3bwj7uJPWHfD1+379OlZ+/1k8trXDv+33r7cB0AKiUz5OBP8W29oHgp578p1+/uz+6R8owucDgJsf3R7ZjkQ5ZlFdOaxlhXiTDaiJKAHiDQwpAM+IyGYRuUHfVq+UOg4A+t+T00goYxmBm+BSMgBh+wz12bVR9taywkm3GUElo4TMMcZx9RSe2ST+T5YtJuFIcEqYaAIM2fB798nfb8Sn/7gpqvu8uLcLp353XVZ+uhzLv4jdFTrLNZwWPTPleED2T6SGAz44Cfz1eWV/6Cln7X0O/DVMadZ0v6s7ggI4DpcH593yQkT/rvv0Pkpf+es2/7ZHtx7VnjfcnaIosTs6MN642+HyYH/niP4Qsf+fmu6/4wPN0Ze4EeUCEUFTozUrX9OJKPPEGxg6Rym1CsAVAL4gIu+I9I4icoOIbBKRTd3drAnPFEZz6NLCyRlD9jCBoZ6pMoaKtMcxAk7+HkMFzBii0IzyMY6qp0SIJ8YTUTApyoyW7R0DeONgej7dXX9QC1K81d6f8Mfe0TGIl/ePn8t7RsZw4U9fQluPPeHP9fNn9uHJHTH2EYpAJA2dp+qPFPg753R78cLeTv99Ovq18q9BvfeO4XuP78Kv9ZKx6QQHrnYfG0J7n3PK7KR4GEGdaLO3vvOPXeOPkYSX8z+9eRiNa5+Ycp+ndhzHszFk1hFliyabFR39zglBWSKiWMQVGFJKHdP/7gLwGIAmAJ0iMhsA9L9DvsNSSt2llFqtlFpdV1cXzzKm1Nzah93Hkj/NJFcYvV2Kg3oMAeFLyYw091AZQ/77BpWSGQEjomBGOZnxN1GyDY260TU0frEdyTVsrBe677vtdVx795tT7vP1h7fhs3+KLhsoEskcXPPe217Dx+9r9n//5I7jONRjx72vtYa9j7//TJTZJL96oQWf/8uWmNYZLaXUpFIuQDvvdQ5Nn1l0pM+BT/1+E/YcH8JvXzmEc3/yIg50Dk/a7/fr2/CzCEusghm/i/H+84b7/Yjmdz3wMQL7LpmmeZAnth+PuifUbS9MH0j73F+24DMhMuuODzoxPOoOcQ+i7NKk9xnayKwhIopTzFdeIlImIhXG1wAuBbATwOMAPq7v9nEA/4h3kfH49t934qfPRN8wMV9NlTEUrpSsdyR8KZkxlt4oIXO4PLCYBIUcQ05hGJlCzBiiRJrqovn8W15E0w+fj+o+yfTQpg6s25W8LId4ynqSIsOWAwDdw2P4+H3N+O9/7saq7z+LxrVPYO+J8YDO+257HWtC/M4AoX++w6Mef6ZYR9An+/Fm09x0/1YA4QM74bbfN0XQLuTjRLRP9P9rjg868YX7t0wK9k31SOsP9qBzaCzq5zKc/aMX8O5fvTZh2+bD/Tjcm/gMN6JkOmV2JSqKLdjAwBARxSmeq/N6AK+JyDYAzQCeUEo9DeDHAC4RkQMALtG/T5t+h8sfuKDpOUM0nzZ6DE1VSlZRZEFxiPKwMj0zyMgYso95UVJozooeHpQe/lIyZgxRAkQSBOl3TMwciCpDIosG16dypaGCEWf/6PmkZEIlmlLAWT94Di/v78bv17dFff9Un96M0rJo/31/+8rBsLcppXDm95/FXzceCZjWNv0zBO4S+HNonaKk0JhaGk0pTDTNt8MJnur2b79Zj/NvfSnuxyVKJbNJcFajFc1sQE1EcYq5A7BS6hCAFSG29wK4KJ5FJdLQqBuFFl5gRipU8+ngcrBgfXYXrOWTs4WAgIwho/m0y+PfRhSKxWyUkjF4SJRIoUaHp9LxwdEJ/XGS0MIpY0USVAG0Xk3VpQWxPkm4GwAAT++MvC+Tx6fQa3fhm4/txMWn1E+571BASVak/16Na5/AaQ1V2HF0EKFe6l/a14WeJHyop5TC/c1HJmzrHh6bNmDp8fpw6zP78LnzF6G6NPT7HaJ0abJZ8cJe7f9MqH6fRESRyOmIyZjHi1G3L2R/AArNP64+VClZuHH19jHUhigjA8aDSoHj6tlfiKbC5tPZT0QuF5F9ItIiImtD3L5URN4QkTER+Y9UrGnShXkEV7CRXMwHZiQdHXBi74noetqNebz44ZN7cq7fSTTBp6l27TSyYUL8WwyNuuHx+vzfT9XPKBqRBnHCieeV6723vTb9TmGEW/UDze3Ye2IIN/458r5MxjF4fdP/LL4QY78nY7paqKf4xO824nCvw7+WxrVPTNtoOhJ3vHQQ33ps54Rtf9lwGFuODEx5v6d2nsBvXz6E/30iOQ2+ieLBPkNElAg5HRgyAhkOl9cf8KCpOUNkDBVZTCgwS9hSst4RF6xloT+hMErJjLHBDpd3Qv8iomD+HkPMGMpKImIGcDu0aZXLAFwrIsuCdusD8CUAP03+erS/jWvPaBpLR9tT5Zwfv4DLf/lqNMvD3zZ14K5XDuGXz0U2kSoekZS9KaVwqHsk/udKUKqPMdL9zUN9cLq82Hx4fKLar547gJO+9ZR/vd//1+6EPGfcSw8RFQt+zAFH+ECgERCJ1vaOQX92brBQv5den5oQBHv1QDeODjjx/ttfx8U/f9m/3Sj9VgB8PoU/v3kYY57x91SHusfLxOINqoV6jFf2J25y7a3rYus5afQecgcEIokyxalzqlBSYGafISKKS04HhoYCxsH2MmsoIg63F4Vmk7+cB9DeFJYVWcKWkvXaXZgRppSspMAMkfGMIfuYB6UsJaMp+KeSsUF5tmoC0KKUOqSUcgF4EMCVgTsopbqUUhsBpCxNJprr1Uj6EiWqgbOR8eJJwQXneCnZ+Nq3HOnHTj1zAwAe3tyBC3/2Mta39CTkOafKHAoVRHB7fWE/hLjsl6/g336z3v/9UztPAAD2HJ885SudIvnNuOmB8SybB5rbJ9z20Xtj759zz6uRZ031jLjwzcd2+L8f8/hwzo9fwFvtA2gLCE6t+v6zALTfn8e3HcN//X0nfvV88gKZwb8WI2PjQahUTJld9f1n8evnD0z4QPGnz8Q2MY4oFQotJqxaUI1mBoaIKA45feU1FFD6FO0Y1HzldHlRXDD516K8yBJyXL3Pp9Bvd6E2TGBIRFBWaIFdf2PndHtRxowhmsJ482lmDGWpBgCBV7od+ra0GA+CRJ/JEHiB+sUHtuL8W19MzKJCPVec9998uA+Na5/A5sPTXxgE/s+6+o71eM+vx8uXtnUMAABaEpA1FKnAQNVH79mA5d9dN2kfBTWpWXA2CX41mypjKB7RNssODkpNZdDpxl82HAYwcf2BAT6lgAOdWqDu1QPRBRfDZeoFZrpFU2oXabA1OMjbZ3fhZ8/ux12vHIr4uYjSramxFntODGHQmVtlyUSUOrkdGGLGUNScLu+E/kKG8iJLyHH1Q6NueHwqbCkZoJWTMWOIIuVvPs0eQ9kq1D9cTHEPEblBRDaJyKbu7tjKScYnKsVwpwD/3HYsZIlPvJUz/jKdOB/n5f3aRfgr+6e/GI+k/0/gLn/deARP7RhvXDw86sZP1+2b8sK7XQ/i2MfCl3EbZTljAZkZmVAKEf+/6eRtDpcXrx5IXElUOMnuqbixrX/StmMBDcW//Ne3cMkvXsHLcZR/TVWOFknPI0Okwc2X9neF3O5kCwLKIk02K5QCNrWl/zWUiLJTbgeGApp59tk5sj4STrc3ZOCmotgSMr2/Z0R7ExqulAzQJpPZXQHNp5kxRFPguPqs1wFgXsD3cwEci+WBlFJ3KaVWK6VW19XVxbSYSLIQ4nncVBh1e/GjJ/eE7R8DBATAArZtauub0JMn2nHjhm88sgOfC2gwfMvT+3Dbiy14fFv4f9Z79EbQL+7TLrpD9Yl585B2AXPHS+HHphs8UQQEQlFKYdexwQnbjg86Q57X4v3dcIQIhn3y9xv9TZY/+buNcT1+Nvj4fc0Je6xYyzY/ek9ka9g6TePpQInsd0SUSCvnV6PALCwnI6KY5fSV15Bz/A1f7wgzhiLhdHtRXDA5cBOux5Dx6aQ1zFQy476OgObTxqQyolDYfDrrbQSwWERsIlII4BoAj6drMcZFZaj+OtOZKjwQSVbJyJgHV97+Olq64uuB8+c3D+O3rxzCb6YIoEiI1KgP3PnGhJ480TTgnorReyWSRrxKKfh8Ch+bIlAQyXCIUBfvRqDrxNAofvTU1NOiHtzYjnf/6jV/oAoAzv7RC3j/7a+HeNxpl+MXPCmrce0T2NeZWT2PsknX0Kh/WIXhvtdjmzbnS0Aj7DP+55kJ/+9YpkOZqrjAjBVzqzMi65KIslNuB4YCMob6HQwMRcIZJqMnXI+h3hEtE6t2ilKy0kKzP6jkcHlClqoRGTiuPrsppTwAbgKwDsAeAA8ppXaJyI0iciMAiMgsEekA8FUA/yUiHSJSmYz1jGcMRX6ROB5jie/C8tX93djWPoCfrgvfuDaSOJXbq63D5QkfiPEHwCJY11TBsYjKzIyfaYQ/ngQNKAvr+//ajd++PHU/mL3HtabF973WioGA9wMHukbQNTQ64YL/LxuOJGehNK2mHz6PK375SrqX4TfgcOMnT+/1f+9TwNJvP8XpZJSRmmxW7Dw6GLaBPxHRVHI6dWPI6YbFJKguLUx63X2ucLq9E0bVGyqKQ2cMGb2bpiolKy+yoHN4FC6PD26vYvNpmpJRQlbAqWRZSyn1JIAng7bdGfD1CWglZkkXnEgTSbAnkqyiRJeSTRW4iiZYM+VzRBChiWSfRAWh4hXJY+87MQyHy4Nh/YONVw/04Iz/eXbCPk0/fD4p66PYBPYsCuWNg70RPU6y3veNun2459VWXHnGHNjHPGjvd6CyuAAr5lXzvEVp1WSz4o6XDmLrkQGcu3hGupdDRFkmtwNDo25UlhSgtqyQpWQRcrq8qCmdHOQpKwwTGNJ/rjVTlJKVFllg7/H6+2Ow+TRNxcgUMrOUjBIgXHZLosbNR2LKoE9UjxPBPlPsZKwjolhNBFlFEWcMJaCkJ9bHvSyDsk9ySTLifaGau4dy7d1vJv7Jo/STp/dOyCQCgM++YyFuftcpaVoREbC60QqTAM2tvQwMEVHUcvqjjSGnB5XFFljLmDEUKa35dIhSsmKtT1DwRJBe+xiqSgqm/JSsXJ9KZvQZKitixhCFx+bTlEj+qV9BYZVIgjWJyrBJhfHm0+EX1K+PGH9mV2fYfSI5nAc3tk/7XP7HiyNDa7oJVNNllhBN54ntx8PeFu3/7b0n2FuK0qu8yIJTG6rYZ4iIYpLTV15GxhADQ5FzukKXkpXrDaPtQVNxeu0u1E6RLQRoGUJaYEi7bwkzhmgKZj0gxObTlAjBQZ5ElTZF8yipyE4ymaYfe39Ab4r8WksEI+0jeM5E9hjy+hSODjgnbFvDEi9Ksi/cvyXsbdvaB6J6rKleWgYcrpBZ10SJ1tRoxdb2AYx5pm/qT0QUKKev0IecblQZgaEENZ/eeXQQG1r7cP25toQ8XqYJ1xzaCAyNjHpQWVzg3947MobaKfoLAfpUMrcXI/oIX/YYoqkUGFPJ2HyaEsCfMZTGzJ7IMmvC3xZNM+x4DzOan1OoEm2vT01bBvqF+7egPOADgj3Hh7Dom09O2q9HH26QLZJVMkfp8cahyHoZReKM/3kWFUUW7PjvyxL2mEShNNmsuOe1VmzvGMRZjdZ0L4eIskjOZAz1211o7bFP2DakBzGsZYUYcLjhScAUib9tasf3/7U7ovG6qXSoe8Q/ISweo25f6MBQsR4YCvrEq8/umnIiGaAFgpQan2DGHkM0FYuZzacpcSIpsZp0nygmmcXbNDqSnaJJckplbOIXz02etrb+4ORspPa+ib1jnth+HH/d1J60dVHy/flNTm4LNt1/02FmDFEKGMGgZpaTEVGUcubK66fP7MN1926YsG3I6UZlicWf0WL0V4iHMYWro985zZ6p9bH7mvGDJ/fE9Rgerw8ur2/KUrLhoJH1vSMuWCPIGAKA7mEjMMSMIQrPKCFj82lKhGgbJQPxl34Zr3XRPGe8jaWN8dlbDvdH/qQheH3a48Ra9nLdvc24+o7X/d/3O9y48Gcvx7WmbGH00aP8lMwJfESRqikrxMn1FewzRERRy5nAUEe/E8cGnBOygoZG3f6MISAxo0uN1Pn2/simZ6SCfcyDjn4ndh8biutxnHoWVKjATYWeMWQPuFjw+hT6HC7MmKbHkNFs2rhYYvNpmooRGCpgKRklwHj2z0SJHt0eyJ2A7NRoHegaAQA0t8V3MfDY1qMAgJ8/MzkbKFJbjgzEtYZs9f7bX59+J8pZL+ztmraccGTMw15DlHRNNis2t/UlpFKCiPJHzgSGekbG4FPjGT1jHi9G3T6t+XRpAgNDdi240dGXOYEho4TuULc9rpOAERgqDpExZGT9BL6hGXC4oBT8gbdwjNKxbpaSUQQ4rp4SyQjcGBdskXyoH1XpVohcH+P+qUwgMCfoydxe7XhcvKCImhGco/z1xzcOw+XxYfexIfzH37ZNek926nfX4dTvrkvT6jKLiFwuIvtEpEVE1oa4/SMisl3/s15EVqRjndmoyWaF3eXFrjg/MCai/JIzV+hGNkrX0BjqK4v9JU+VxRZ/qVMiAkPGY7RnUClZW68WGHJ5fWjrdeCkmeUxPY5TT4OfqpRsJKCUzAjC1ZZP3WPIuG/XEEvJaHoW/1SynIlbUzqFyxiK92GniMMEZxFF0lh6yucygltT7BNJHDXe/kM/eGI37n61Nb4HidB0o+qJMtF3H9+F7z6+y//9NWfNwxnzqtO3oAwlImYAtwO4BEAHgI0i8rhSanfAbq0AzldK9YvIFQDuArAm9avNPk228T5DK/j7R0QRyokrL59P+YMUnUOjALT+QgD84+oBoM8eX3Nmn0+NB4YyKWOoe7zp9n59HHEspiwlK9ImkQVmDBllddOPq9dLyZgxRBEws5SMEih4XL0RAN8cQS+eWAMpsSTvRPJcJwZHp3jOqZ9059HBuDNa7n0tNUEhYOKHEETZakNrH972oxfC3t7e58BTO46ncEUZowlAi1LqkFLKBeBBAFcG7qCUWq+UMl6o3wQwN8VrzFr1lcVorC1lnyEiikpOBIYGnG7/p4udw9ob50EjMFRcgBq9lKw3zoyhAacbxoeYmdRjqLXXDmtZIUQmBoaGRt1Y/b/P4umdJyJ6HOOCqThEYMjoCzQhMKQH2iLNGOoZGUOBWVBoyYlfO0qSAn8pGX9PKH7jARPtxTu4gX7I+0Tx+BH1IZoqu0i/rbUnfNCmVc8KfWKKC8jpglHv+fVrU++QYV4+0J3uJRDF7dZ1+9AzxcTY8255EZ/7y5YUrihjNAAIHE3YoW8L53oATyV1RTmmyWbFxrY++Jh9SUQRyokrL6OMDBgvVxoySslKLCgwm1BVUhB3KZkxbr26tADtfRlUStZjx8n1FZhvLcWBzvGLi+ZDfegZceGFvZ0h7+d0efHHN9rwn3/bBofL4w8MlYYoJbOYTSguMIXOGJpmKllpQClZqDI1okBGQIgZQ5QIwRlD0ZSyTj2uPvzvZ/Ath3snf5Bg9Dwy1vXmocmf7I7vM/0be9MUkaFQZVkPhRkXnymDlb70wNZ0L4GIkifUK03IFzoRuQBaYOgbYW6/QUQ2icim7m4GlA1rbLUYdLqxvyv2SgIiyi85UdMT+GlM13BQKVmxVgJlLSuMOzDUowdCVsytxsv7u/1Tz9KttceOy0+djbIiy4SMIWM6zdagCTEerw+3vdiCP6xvQ79D+zktqC3FKbMrAQAlYS6cyosKgjKGXBCBPyMrnHK9dMzp9qK6tDi6g6O8U8Dm05RAwVPJZlVpr0Fza0qmuJN+nxCXKe19Dsyzlvq/n6pJc5f+ocXeE5PfmPsUYJapgz5en/I3Y5/OVHs9vu3opG1ff3g7vv7wdv/3SimISNx9iIhoesOj7rhK/3NAB4B5Ad/PBXAseCcROR3APQCuUEr1hnogpdRd0PoPYfXq1XwF0wX2GVo6qzLNqyGibJBTGUMlBeaAjKHxHkNAYgJDxv2NRm6Z0GdowOFCv8MN24xSLKkvR2uPHS6PdqGy4ZB2Dj3QNeIvrQOAf20/jl8+dwCr5tfgbzeejUuW1ePOlw+hQ2+oHS6rp7zIPLH59MgYakoLp72ALw0YT8/G0zQdiz9jKCdenijNxqeSad8bmTVTvRaFGkFvMJr9G17aF+ITav3u3/nHrsm36SLJAjISfabrH6TtE/62dTtDZ42Geq5oHermJC6iaLV0jUz60C7PbASwWERsIlII4BoAjwfuICLzATwK4Dql1P40rDGrza0pwZyqYvYZIqKI5VTG0CmzK/w9hoacxlSy8cBQvIEco6fOSn9gyInlc6riesx4GaPqbTPKYR/zwONTaOu1Y051CXYeG8IZ86rxVvsAtncM4LzFdQCAV/Z3w1pWiLs/thomk8BaVohLf/EKbnuxBcAUGUPFlgkZQ31217Sj6gHtAr/QYoLL4/OPvScKx8iQsDBjiBLAnzGkB2KMX6tkTr2aKrBkUEF/h95H6Y837kDnsP+1OlBgKdm29gEsrCvzf//0run7zPmUgnmadYda64U/e3naxyaiifK97YtSyiMiNwFYB8AM4D6l1C4RuVG//U4A3wFQC+AOPTjuUUqtTteas42IoMlmxesHe/0ZoUREU4n5I3kRmSciL4rIHhHZJSL/rm//nogcFZG39D/vStxyQ+seGUOh2YSTZpZPyBgqMAuKC7RDrC0rjLv5dO+IVjp12lwtGNSRAQ2ojU+vtYyhCgBaA+oth/vh9Snc8I6FEBkvJ1NK4ZUDPTj3pBkw6VdIi+rKcW3TvAmZV6GUF1mCMoZc004kM5TpwSb2GKLpGAEhlpJRIkxsPR0YKJr+vrFeu0Xy/nvM48P3Ht+Ffvt4NueWI/1oXPvE+POHWMAlv3gF/3hrUsUFigNeWz/zx014169ejWrNoZ5r97Ghafchoug9uqVjQnB6e8dA+haTJkqpJ5VSS5RSi5RSP9C33akHhaCU+rRSqkYpdYb+h0GhKDXZatE9PIa2EH3uiIiCxZO+4QHwNaXUFhGpALBZRJ7Vb/uFUuqn8S8vMt3DY5hRXohZlcXoGRmD16cw5NT6/xgRcmtZIfrtrrii5r32MVSXFKC2rBDlRZaMKCVr7XHAJMA8aymU0j4N339iGF6lYDYJzl9Sh8Uzy7H1iDbxc++JYfSMjOG8xTMmPM6/X7QEj245CofLG3acfHlRAY4NjDfd7rGP4ZQI65bLiizod7iZMUTTGh9Xz1Iyip/xeh8c1PBNEeWY6hSRqODIP7cdw+/Xt03Y9sr+iWVpj2zpQFOjdcrHGXV74fWpCY/VNRx+ClI4PqUmZVFFG1wiosj8ZcORCd+/77bX0fqjdzGrgxJqvM9QL2wzyqbZm4jyXcxXXkqp40qpLfrXwwD2YOpRk0nTM+JCXUUR6iqL4VNa75uhUY+/vxCgBYY8PuWfVhaL3hEXasuLICKYW1OC9v7pJ5ONur1RjYp0T9HINJTWHjsaakpQZDGjuMCMxtoy7O8cQXNrH05tqEJZkQWr5tdga/uAli2kX3gYZWWGuooifPWSJZhbU4KiMOPky4vM/lIyn0/hxOAo6iqmHlVvKNODTewxRNMxAkLMGKJECC4lM3jjTBmacgR9BOvyRHBe+NZjO3HJL16Z8rmWfvtpLP/uugiecWpLv/00Fn3zybgfh4hi8+DG0JMCiWK1qK4MtWWF2BBi6iURUbCEfCQvIo0AVgLYoG+6SUS2i8h9IlKTiOeYSs/wGGaUF6FeD1J0Do3pGUPj2SlGL5x4GlD32sdLp+ZZS6fNGPL6FC786Uv4ybq9ET3+qwe6sfw76/DXjUem31nX1mNHY+34pwCL68ux4+ggtrUPYo3+ScHK+dUYcLjR2mPHqwd6sKS+3D+ZJ9Cnz1uIV/7zAn+JWbDAHkPt/Q44XF4snVUR0TrL9AbUDAzRdMYzhhgYovgFl5IZfFPE4Mfvk9jaqfUHe8a/CRGY+uVzB0Le78TgaELXQUSZ5+ZHd6R7CZRjjD5DbEBNRJGIOzAkIuUAHgHwZaXUEIDfAFgE4AwAxwH8LMz9bhCRTSKyqbs7xFSXKHSPaIGhmZVasKNreFQbJR+UMQQAffboU+wNvSNjqC3XA0M1pejod045WWbfiWEcGxzFn9847J+SNpU3D/XC5fXhG4/swG9eOjjt1BqlFNp67FgYkB66pL4CRweccHl9AYEhLTa3/mAvmtv68I6gbKFA4YJCwMRx9XuOa70njBH30zFKyMKVqREZCvzNp1lKRvELV0o21eur8ToYKqnH7fXhj2+0TXn/l/d340O/fWPS9g/fvcH/dTTNr5/b0xXxvkRERIYmmxVHB5wZ0ReViDJbXFdeIlIALSj0F6XUowCglOpUSnmVUj4AdwNoCnVfpdRdSqnVSqnVdXXhAxXT8foU+uxaKVl9ZXDG0HhgqLZMu613JPaMoT67y/8486wlcLq96Jni8ZpbtXHxdpcXD0WQIry/cwS2GWW48ow5+MnTe/HjpydnGu09MYTXW7RPnXtGXBge86BxRmDGkJbBIwKsXqAFhk6qK0dFkQX3vtYKl8eH85bE9vMuLzLD5fFhzOPF7uPDMAn8Da+nY5SSlRUxY4imZgSELMwYogTwl5IFZf9MFZcx4uObD/dPuu2+11vxnX/swgPN4V/Tv/rQNjQHfUIb2FQaAL73z91TrJqIiCh+Rp+hjW3MGiKiqcUzlUwA3Atgj1Lq5wHbZwfsdhWAnbEvb3r9Dhe8PoUZ5YWYUV4EESNjyIPKkoBSMj3Tp98RW2DI4/Wh3+H2Zx7NqykFoJVUhbOhtQ8N1SVoarTid6+3wRPQP8jhmtzr6EDnME6ZXYFffOgMfODMufjty4fQMzIxw+lHT+7FJ37XjG3tAwETycYDQyfrgZqlsypRVaoFxkwmwYp51WjtsaPQYpq2mWk45XrWj33Miz3Hh9A4oyzsaPtgpf5SMmYM0dQszBiiBPKXhQUFgqbqMRTcADbwdfj1Fi3gPxBwLvmvv++IqgSYiIgoFZbOqkRFsWXShxVERMHiufI6B8B1AC4MGk1/i4jsEJHtAC4A8JVELDQc4w17XUUxCswm1JYVhswYspZqAZ1YR9b36RcBM8rHewwBCNtnSCmF5tY+rFloxafObcTRASee29MJAPjd66049bvr8OK+8fIAp8uLw30OLJ5ZAZNJ8P4ztD7e+zuHJzzu/s5huL0KX3xgK7Z3DAKYGBiyzShDcYEJb19UO+F+K+dXAwCaGq0RB3OCles/T/uYB3tPDEVcRgaMB5XYY4imYwSE2HyaEmE8Y2iiqUrBzAGBoX0nhrH6f5+btM/eE+OvzX9+8wi+8Qj7gxARUWYxmwRNjewzRETTizl9Qyn1GkIPX0npWJOe4YkBm7qKYnT0OzDm8U3oMVRSaEZJgRmdMTbxNJpWW/VSsrk1JQCAjjCTyQ5229Frd2GNzYpLls3C3JoS3PtaK3YeHcJtL7YAAF7e140LTp6p7z8CpYCT9WbOS+rLAWij59++SBstP+h04/jgKC5ZVo/n93Ti1nV7YTEJGqpL/M9baDHh7184Z8I2YDwwFDymPhrletbP8cFRtPc5cc1Z8yO+r5EpVMaMIZoGm09TYhk9hiIvJQsMSh7sHgm5z5gnugmSRERE6dBks+L5vV3oHh6LeJowEeWfrK/V6B7RAj0z9Be6+soitHRpb+QDp5IBwDknzcCjW4+iP4asIaM3kdF8uqzIgtqywrAZQxv0/kJNtlqYTYJPvL0RG9v6cduLLbjmrHlYvUAbIW8wMoOMgFBdRRGqSgqwv2v8oqSlS9vnmrPm4csXL8Go24f5taWwmCf+M2ppowUTtr190Qzc8I6F+Lcz50Z97IbyIu0xNx3WPnU4ZXZk/YW0+2pBpVizlSh/+JtPm7P+5YkyQLiMoamaPwdWkkUy1Z6IiChTsc8QEUUi66+8jIwhIwI+s6IIx/WsoMCMIQD4+uUnwz7m8WfsRMMoQTMykwBgrrU0bI+h5tY+zKwoQmOtVnL2obPmYemsCnzposX40dWn4czGGuw+NohRtxeA1ni6wCxYoI+eFxEsqS/HgYBSsv2dWpBoSX0FvnDBSbj4lHqcH2Ej6eICM775rlMwozz2TwrK9UDbpjatIevSWZGXkpWy+TRFyMjWsLCUjBLA/1ukB3iMQI8vRMTHyCoyBUSGjoQJ/hMREWWDUxuqUFJgZp8hIppS9geGRsZQaDGhQu9hU6+PrAcwoccQoAVUPrR6Hv74RhuO9Eb3Zr9X72VklJIBgK22FLuPDU1qJK2UwoZDfWiyWf1NTCuLC/D0l9+Br16yBCKCVfNr4PYq7DqmjX0/0DmMhTPKURCQJbG4vgL7O0f8Fyv7O4dRUmBGQ3UJzCbBPR9fje++d3lUxxEPo0/QprY+VJUUYHZV8TT3mHxfNp+m6dTo/cCqggK7RLHwj6sPzhkKkQnUFyKb9CchpkMSERFliwKzCWcuqGGfISKaUtYHhrqHx1BXXuR/8z8zoHY2cCqZ4SuXLIHFZMKtz+yL6nn67C6YBKgOuFj96NsWoN/hxj2vtk7Yt73PiRNDo1hjCz/9a+W8agDA1iNa9s2+zmEs1svIDCfXV2DQ6UbXsBaUOtA5gsX15TClKZPCCO4MjXpwyuyKSZN7plJWxB5DFJnT51bhqX8/D6c2VKV7KZQDjJdLX1BLoFAZQ1f836sAgFsYDCKiNIj2Q0uiSDXZrNh7YgiDDne6l0JEGSr7A0MjY/7+QgAwc4qMIUDLKPrMeTb8c9sxbAvo8TOdnhEXrGWFE4IyqxutuGx5PX778kF0D4+PMw7sLxTOzMpiNFSXYGv7AOxjHnT0O7GkfmLPHiNQZPQf2t85jMUzI+/rk2jlAT2boplIBgDvWDIDX754cVR9iSg/iUjUv19E4YheTBYcCAo1rr5reAzX3PUGXtzXnZK1EREF2n18KN1LoBzVZLNCKfYZIqLwsj8wpGcMGSZmDIUuRbnh/EWoLLbg9+vbJt12uNce8j69I2OoLZvcn+frly/FqMeHXz1/wL9tQ2sfakoLsHhm+aT9A62cX423jgz4m2UHB4aM7/d3jmDA4ULX8Ji/OXU6lBaY/U1Zo71wryguwJcvXsKGwkSUUgNOrTzs/wJeo4HwU8nePMQ3zUSUHoF9JYkS6Yx51Sg0m9DMwBARhZH1V+k9Iy7UVYw3hJ6qx5ChvMiC96yYg6d3nsDI2Hh/oJf3d+P8W1/CI5s7Jt2nz+7yTyQLtKiuHB9umo/7m4/gn9uO4XN/3oxHtnTg7SfNmLbka+X8GhwdcOK1lh4AmBT0mVFeBGtZIQ50Do83np6Vvowbk0n8pWCnRNF4mogoXTqHtGxOo5+bwcVx80SUYdpYSkZJUlxgxop5VewzRERhZXVgyOtT6LOPTZi0ZXxdYBYUF4Q/vKtXNsDp9uLpnSf82+5+5RAA4KfP7PNPCzP02rVSslC+dNFiFFtM+OIDW/F6Sw8+d/4i/O+Vp067/pXzqwEAf9vUjkKLyT+RLNDimeXY1zkcMM4+vaVY5UUWmE0yqR8SEVEmKingJEQiyg7Bw0yIEqnJZsXOo4Owj/H3jIgmy+rAUJ/dBZ8aH1UPAIUWE2rLClFZXDBlc+QzF9RgvrUUj23VsoP2nRjGay09uGjpTBwfHMXvXm+bsH/vyFjYUe91FUX49YdX4n+uXI71N1+Er1++FDVhgkiBls+pRKHZhLZeBxbVlfvHdAdaUl+Bls4R7O8cRnmRBXOimASWDOXFFiycUYZiXmwRURa4eNnMdC+BiCgi3nA1rkQJ0GSrhdensEUffENEFCirA0M9+gj54IBNXUVR2P5CBhHBVSsbsP5gL44POvG711tRXGDCTz+4AhctnYk7Xmzxjy52eXwYGvWEzRgCgAuX1uNjZzf6J3dFoshixrI5WklWuN5BS+rLMTzmwSv7u3HSzPKoJoElw0VLZ+KqVQ1pXQMRUaQKp+hr1jk0msKVEBERpc+ZC2pgNgmaWU5GRCFkdWDImAQWmDEEAItmlqOhumTa+1+9qgFKAfe91opHtx7F1avmoqasEN+4YinsLg9+/YLWrNQIEIXqMRQvo5wsXInYYn17W68jrY2nDTe/6xR8/p0npXsZREQRmSqYvuaHz+Pzf9mcwtUQEYXHfCFKpvIiC06dU8k+Q0QUUlYHhsJlDP346tNw+0dWTXv/BbVlOHNBDe5+tRUujw+ffHsjAC1I86HV8/DnNw9jfUsPeu3a89RGUB4WrZXza/zPGUrg9nT3FyIiioSIXC4i+0SkRUTWhrhdRORX+u3bRWT6F+w4fOLtjagoDp3N+eSOEyG3ExGlmo+lZJRkTTYr3mofmNRLlYgoJwJDwRlDFcUFqJqmlMxwtV4W9Y4ldf7sHAD4z8tOhm1GGT7xu424f8MRAEBtmB5D8bhseT2+/Z5lOH9JXcjbrWWF/sAXA0NElOlExAzgdgBXAFgG4FoRWRa02xUAFut/bgDwm2Suae+JIQyPerDz6GDE9zmrsQa3fOB0LJ45nqnZ9uN345HPvR1fvPCksB8U/OL/rQAAnDK7Er/75Fl45T8vmPa5Pnb2gojXRUS5y8PAECVZk60WLo8P2zsiPx8SUX6IvCFOBjKJYL61FGWFsTdCfs/pc/CPt47hyxcvnrC9trwID332bFz/h034ixEYSkLGUJHFjOvPtU25z5L6cvSMjDEwRETZoAlAi1LqEACIyIMArgSwO2CfKwH8USmlALwpItUiMlspdTwZC5pXU4o30Yf3/Po1vPv02WH32/v9y1FcYIbH64NF7030wTPnwnbzk/59zlxQgzMX1OD8JXX4wJ1v4OYrluJQtx2XnzoLFyydiQGHVnr8kTXzccHJWuPra86ahwc3tod93kV1k8uE376oFusP9sZ0vJQ7qkoKMOh0p3sZlCIffzuDxJRcZzVqlQp/29TO1xaiLHT2otqoehpHI6sDQ58+byE+fd7CuB6jqqQAD3327JC3VZcW4s/Xr8FN92/Bqwd6MLMyPRPBVs2vQVuPHfWVic9YIiJKsAYAgVGQDgBrItinAcCEwJCI3AAtowjz58+PeUG3fnAFFICHN3fgie3aU3z2/IX43PmLcMb/PIvq0gI89vlz/NMWLQENq0UEO//7MliCpkaubrTizZsvwqygSZHVpYVo+/G7J2z79nuWYemsCnz87Y3o6HeioboEt6zbh6tWNuCFvV346NsW4B1L6jCnuhib2voxs6IIi+srsOVIP/7joW34+uUn45ldnbju7AW46o71sM0ow9UrG3D9eTYs+846AEBZoRnPfvV8bDrcjy89sNX/3N961yn4wZN7Yv7Z0fTmWUvQ3ueM+3E+c54Nd7/aOmHb+rUXYvl3tX/j2rJC9Oo9DwO9b8UcPL7t2JSP/dVLluDnz+6fcp/q0gIMOKK/UPyfK5fjO//YBQD46NvmwzajHN//lxYH/ua7lqJraAwt3SN4aV83AC1D+8oVc/C1v20DoGXX7Tk+hCKLCWMen/9xf/6hFfjqQ9umfO4bz1+EO18+GNV67/7Yanzmj5smbPvtdWfis3+aut/Yq1+/AOfd8mLI226+Yil+9NTeqNYRihFMJkqW6tJCnD63Cn/b3IG/be5I93KIKErPffV8nDQzOX2HRfvANr1Wr16tNm3aNP2OaeLzKfTYxzCzIj2BIZfHB6fLi6rSyMrjiCi3iMhmpdTqdK8jEiLyQQCXKaU+rX9/HYAmpdQXA/Z5AsCPlFKv6d8/D+DrSqmwV2aJOE8MOt041D2C8iJLRkx5zAVKqYh/jl6fggAwmSbvH83jGAIzuzKFz6dCHh9lrlh+9zJNNp0jkinTrycyxaDDjfZ+R7qXQUQxOGlmuf+DzGhEcp7I6oyhVDGZJG1BIQAotJhQaMmsN79ERGF0AJgX8P1cAMHpDJHsk3BVJQX+hv+UGNFcUJunCJjEcmGeaUEhIHTQizJbtgeFiKJVVVqAqtKqdC+DiDJM5r2rIiKibLYRwGIRsYlIIYBrADwetM/jAD6mTyd7G4DBZPUXIiIiIiKiqTFjiIiIEkYp5RGRmwCsA2AGcJ9SapeI3KjffieAJwG8C0ALAAeAT6ZrvURERERE+Y6BISIiSiil1JPQgj+B2+4M+FoB+EKq10VERERERJOxlIyIiIiIiIiIKE8xMERERERERERElKcYGCIiIiIiIiIiylMMDBERERERERER5SkGhoiIiIiIiIiI8hQDQ0REREREREREeYqBISIiIiIiIiKiPCVKqXSvASLSDeBwjHefAaAngcvJVDzO3MLjzC3JPs4FSqm6JD5+xuN5Iio83tzG481dsR5r3p8jAJ4nosTjzW35dLz5dKxAEs8TGREYioeIbFJKrU73OpKNx5lbeJy5JV+OM1vl278Pjze38XhzVz4da6bJt589jze35dPx5tOxAsk9XpaSERERERERERHlKQaGiIiIiIiIiIjyVC4Ehu5K9wJShMeZW3icuSVfjjNb5du/D483t/F4c1c+HWumybefPY83t+XT8ebTsQJJPN6s7zFERERERERERESxyYWMISIiIiIiIiIiikFWB4ZE5HIR2SciLSKyNt3rSRQRmSciL4rIHhHZJSL/rm+3isizInJA/7sm3WuNl4iYRWSriPxL/z4Xj7FaRB4Wkb36v+nZOXqcX9F/X3eKyAMiUpwrxyki94lIl4jsDNgW9thE5Gb9dWmfiFyWnlUTkBvniVjOCeF+B0XkTBHZod/2KxGRdBxTJKI5P2T78UZ7nsjm4432XJGNx5qoc0a4YxSRIhH5q759g4g0pvQAc0gunCOA/DxP5NM5AuB5IteONSPPE0qprPwDwAzgIICFAAoBbAOwLN3rStCxzQawSv+6AsB+AMsA3AJgrb59LYCfpHutCTjWrwK4H8C/9O9z8Rj/AODT+teFAKpz7TgBNABoBVCif/8QgE/kynECeAeAVQB2BmwLeWz6/9VtAIoA2PTXKXO6jyEf/+TKeSLac8JUv4MAmgGcDUAAPAXginQf3xTHHdH5IReON5rzRDYfb7Tnimw91kSdM8IdI4DPA7hT//oaAH9N9zFn4x/kyDlCP5a8O08gj84R+lp5nsihY0UGnifS/kOJ44d5NoB1Ad/fDODmdK8rScf6DwCXANgHYLa+bTaAfeleW5zHNRfA8wAuDHhRz7VjrNRf3CRoe64dZwOAdgBWABYA/wJwaS4dJ4DGoBfvkMcW/FoEYB2As9O9/nz8k6vnienOCeF+B/V99gZsvxbAb9N9PGGOMeLzQ7Yfb7TniWw+3mjPFVl+rHGdM6Y6xsDziv5z7An+/eGfiP6NcvIcoR9LTp8n8ukcoa+N54ncPNaMOk9kcymZ8Utj6NC35RQ97WslgA0A6pVSxwFA/3tmGpeWCL8E8HUAvoBtuXaMCwF0A/idnu56j4iUIceOUyl1FMBPARwBcBzAoFLqGeTYcQYJd2x58dqUJXLu3yLCc0K4427Qvw7enol+icjPD9l+vNGeJ7L2eGM4V2TtsYaQyGP030cp5QEwCKA2aSvPXTl3jgDy5jzxS+TPOQLgeYLniRScJ7I5MBSqRlClfBVJJCLlAB4B8GWl1FC615NIIvIeAF1Kqc3pXkuSWaClCf5GKbUSgB1aamBO0Wtgr4SW3jgHQJmIfDS9q0qbnH9tyiI59W8RxTkh3HFnxc8jhvNDVh8voj9PZO3xxnCuyNpjjUIsx5hLx59OOfdzzIfzRB6eIwCeJ3iemCzh54lsDgx1AJgX8P1cAMfStJaEE5ECaC/sf1FKPapv7hSR2frtswF0pWt9CXAOgPeJSBuABwFcKCJ/Rm4dI6D9nnYopTbo3z8M7YU9147zYgCtSqlupZQbwKMA3o7cO85A4Y4tp1+bskzO/FtEeU4Id9wd+tfB2zNNtOeHbD/eaM8T2Xy80Z4rsvlYgyXyGP33ERELgCoAfUlbee7KmXMEkFfniXw7RwA8T/A8kYLzRDYHhjYCWCwiNhEphNZU6fE0rykh9G7i9wLYo5T6ecBNjwP4uP71x6HVD2clpdTNSqm5SqlGaP92LyilPoocOkYAUEqdANAuIifrmy4CsBs5dpzQ0j3fJiKl+u/vRQD2IPeOM1C4Y3scwDX6NAAbgMXQGsNR6uXEeSKGc0LI30E9LXlYRN6mP+bHkIH/J2M4P2T78UZ7nsjm4432XJHNxxoskccY+FgfgPZ/JFs/CU+nnDhHAPl1nsi3cwTA8wR4nkjNeSIRjZPS9QfAu6B13T8I4FvpXk8Cj+tcaKle2wG8pf95F7S6wOcBHND/tqZ7rQk63ndivHFczh0jgDMAbNL/Pf8OoCZHj/O/AewFsBPAn6B1zs+J4wTwALQ6Zze0CPz1Ux0bgG/pr0v7kEETEPLxTy6cJ2I5J4T7HQSwWv8/ehDAbcjwhrWRnh+y/XijPU9k8/FGe67IxmNN1Dkj3DECKAbwNwAt0D54WJjuY87WP7lwjtCPIy/PE/lyjtDXyvNEDh1rJp4njDsSEREREREREVGeyeZSMiIiIiIiIiIiigMDQ0REREREREREeYqBISIiIiIiIiKiPMXAEBERERERERFRnmJgiIiIiIiIiIgoTzEwRERERERERESUpxgYIiIiIiIiIiLKUwwMERERERERERHlqf8PUHjfqhtO0dYAAAAASUVORK5CYII=\n", "text/plain": [ "
" ] }, "metadata": { "needs_background": "light" }, "output_type": "display_data" } ], "source": [ "agent.train(num_frames)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Test\n", "\n", "Run the trained agent (1 episode)." ] }, { "cell_type": "code", "execution_count": 11, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Moviepy - Building video /Users/jinwoo.park/Repositories/rainbow-is-all-you-need/videos/per/rl-video-episode-0.mp4.\n", "Moviepy - Writing video /Users/jinwoo.park/Repositories/rainbow-is-all-you-need/videos/per/rl-video-episode-0.mp4\n", "\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ " " ] }, { "name": "stdout", "output_type": "stream", "text": [ "Moviepy - Done !\n", "Moviepy - video ready /Users/jinwoo.park/Repositories/rainbow-is-all-you-need/videos/per/rl-video-episode-0.mp4\n", "score: 102.0\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "\r" ] } ], "source": [ "video_folder=\"videos/per\"\n", "agent.test(video_folder=video_folder)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Render" ] }, { "cell_type": "code", "execution_count": 12, "metadata": {}, "outputs": [ { "data": { "text/html": [ "\n", " \n", " " ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" }, { "name": "stdout", "output_type": "stream", "text": [ "Played: videos/per/rl-video-episode-0.mp4\n" ] } ], "source": [ "import base64\n", "import glob\n", "import io\n", "import os\n", "\n", "from IPython.display import HTML, display\n", "\n", "\n", "def ipython_show_video(path: str) -> None:\n", " \"\"\"Show a video at `path` within IPython Notebook.\"\"\"\n", " if not os.path.isfile(path):\n", " raise NameError(\"Cannot access: {}\".format(path))\n", "\n", " video = io.open(path, \"r+b\").read()\n", " encoded = base64.b64encode(video)\n", "\n", " display(HTML(\n", " data=\"\"\"\n", " \n", " \"\"\".format(encoded.decode(\"ascii\"))\n", " ))\n", "\n", "\n", "def show_latest_video(video_folder: str) -> str:\n", " \"\"\"Show the most recently recorded video from video folder.\"\"\"\n", " list_of_files = glob.glob(os.path.join(video_folder, \"*.mp4\"))\n", " latest_file = max(list_of_files, key=os.path.getctime)\n", " ipython_show_video(latest_file)\n", " return latest_file\n", "\n", "\n", "latest_file = show_latest_video(video_folder=video_folder)\n", "print(\"Played:\", latest_file)" ] } ], "metadata": { "kernelspec": { "display_name": "rainbow-is-all-you-need", "language": "python", "name": "rainbow-is-all-you-need" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.8.12" } }, "nbformat": 4, "nbformat_minor": 4 }