{
"metadata": {
"name": "",
"signature": "sha256:63283541f64bcfc0e839e859a18aaf406ea69bf352d52c6ff78c60b4bb942089"
},
"nbformat": 3,
"nbformat_minor": 0,
"worksheets": [
{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Sebastian Raschka \n",
"last updated: 04/14/2014 \n",
"\n",
"[Link to this IPython Notebook on GitHub](https://github.com/rasbt/pattern_classification/blob/master/stat_pattern_class/supervised/parametric/parameter_estimation/maximum_likelihood_estimate.ipynb)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"
\n",
"I am really looking forward to your comments and suggestions to improve and extend this tutorial! Just send me a quick note \n",
"via Twitter: [@rasbt](https://twitter.com/rasbt) \n",
"or Email: [bluewoodtree@gmail.com](mailto:bluewoodtree@gmail.com)\n",
"
"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"#Maximum Likelihood Estimation for Statistical Pattern Classification"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Sections\n",
"\n",
"- [Introduction](#introduction)\n",
"- [1) A simple case where the parameters are known - no MLE required](#one)\n",
" - [Generating some sample data](#sample_data)\n",
" - [Plotting the sample data](#plotting_data)\n",
" - [Defining the objective function and decision rule](#objective_function)\n",
" - [Implementing the discriminant function](#discriminant_functions)\n",
" - [Implementing the decision rule (classifier)](#decision_rule)\n",
" - [Classifying our sample data](#classifying)\n",
" - [Drawing the confusion matrix and calculating the empirical error](#confusion_matrix)\n",
"- [2) Assuming that the parameters are unknown - using MLE](#two)\n",
" - [About the Maximum Likelihood Estimate (MLE)](#about_mle)\n",
" - [MLE of the mean vector $\\pmb \\mu$](#mle_mu)\n",
" - [MLE of the covariance matrix $\\pmb \\Sigma$](#mle_cov)\n",
" - [Classification using our estimated parameters](#classifying_mle)\n",
" - [Conclusion](#conclusion)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"
\n",
"
\n",
"\n",
"# Introduction"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Popular applications for Maximum Likelihood Estimates are the typical statistical pattern classification tasks, and in the past, I posted some [examples](https://github.com/rasbt/pattern_classification#param) using Bayes' classifiers for which the **probabilistic models and parameters were known**. In those cases, the design of the classifier was rather easy, however, in real applications, we are rarely given this information; this is where the Maximum Likelihood Estimate comes into play."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"However, the Maximum Likelihood Estimate still **requires partial knowledge** about the problem: We have to assume that the **model of the class conditional densities is known** (e.g., that the data follows typical Gaussian distribution). In contrast, non-parametric approaches like the Parzen-window technqiue do not require prior information about the distribution of the data (I will discuss this technique in more detail in a future article, the IPython notebook is already in preparation). "
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"**To summarize the problem:** Using MLE, we want to estimate the values of the parameters of a given distribution for the class-conditional densities, for example, the *mean* and *variance* assuming that the class-conditional densities are *normal* distributed (Gaussian) with $p(\\pmb x \\; | \\; \\omega_i) \\sim N(\\mu, \\sigma^2)$."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"To illustrate the problem with an example, we will first take a look at a case where we already know the parameters, and then we will use the same dataset and estimate the parameters. This will give us some idea about the performance of the classifier using the estimated parameters. "
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"
\n",
"
\n",
"\n",
"# 1) A simple case where the parameters are known - no MLE required"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Imagine that we want to classify data consisting of two-dimensional patterns, $\\pmb{x} = [x_1, x_2]^t$ that could belong to 1 out of 3 classes $\\omega_1,\\omega_2,\\omega_3$. \n",
"\n",
"Let's assume the following information about the model and the parameters are known:\n",
"\n",
"####model: continuous univariate normal (Gaussian) model for the class-conditional densities\n",
"\n",
"\n",
"$ p(\\pmb x | \\omega_j) \\sim N(\\pmb \\mu|\\Sigma) $\n",
"\n",
"$ p(\\pmb x | \\omega_j) \\sim \\frac{1}{(2\\pi)^{d/2} \\; |\\Sigma|^{1/2}} exp \\bigg[ -\\frac{1}{2}(\\pmb x - \\pmb \\mu)^t \\Sigma^{-1}(\\pmb x - \\pmb \\mu) \\bigg]$\n",
"\n",
"$p([x_1, x_2]^t |\\omega_1) \u223c N([0,0]^t,3I), \\\\\n",
"p([x_1, x_2]^t |\\omega_2) \u223c N([9,0]^t,3I), \\\\\n",
"p([x_1, x_2]^t |\\omega_3) \u223c N([6,6]^t,4I),$\n",
"\n",
"#### Means of the sample distributions for 2-dimensional features:\n",
"\n",
"$ \\pmb{\\mu}_{\\,1} = \\bigg[ \n",
"\\begin{array}{c}\n",
"0 \\\\\n",
"0 \\\\\n",
"\\end{array} \\bigg] $,\n",
"$ \\; \\pmb{\\mu}_{\\,2} = \\bigg[ \n",
"\\begin{array}{c}\n",
"9 \\\\\n",
"0 \\\\\n",
"\\end{array} \\bigg] $,\n",
"$ \\; \\pmb{\\mu}_{\\,3} = \\bigg[ \n",
"\\begin{array}{c}\n",
"6 \\\\\n",
"6 \\\\\n",
"\\end{array} \\bigg] $\n",
"\n",
"\n",
"#### Covariance matrices for the statistically independend and identically distributed ('i.i.d') features: \n",
"\n",
"$ \\Sigma_i = \\bigg[ \n",
"\\begin{array}{cc}\n",
"\\sigma_{11}^2 & \\sigma_{12}^2\\\\\n",
"\\sigma_{21}^2 & \\sigma_{22}^2 \\\\\n",
"\\end{array} \\bigg] \\\\ \n",
"\\Sigma_1 = \\bigg[ \n",
"\\begin{array}{cc}\n",
"3 & 0\\\\\n",
"0 & 3 \\\\\n",
"\\end{array} \\bigg] \\\\\n",
"\\Sigma_2 = \\bigg[ \n",
"\\begin{array}{cc}\n",
"3 & 0\\\\\n",
"0 & 3 \\\\\n",
"\\end{array} \\bigg] \\\\\n",
"\\Sigma_3 = \\bigg[ \n",
"\\begin{array}{cc}\n",
"4 & 0\\\\\n",
"0 & 4 \\\\\n",
"\\end{array} \\bigg] \\\\$\n",
"\n",
"#### Equal prior probabilities\n",
"$P(\\omega_1\\; |\\; \\pmb x) \\; = \\; P(\\omega_2\\; |\\; \\pmb x) \\; = \\; P(\\omega_3\\; |\\; \\pmb x) \\; = \\frac{1}{3}$"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"
\n",
"
\n",
"\n",
"## Generating some sample data\n",
"Given those information, let us draw some random data from a Gaussian distribution."
]
},
{
"cell_type": "code",
"collapsed": false,
"input": [
"import numpy as np\n",
"\n",
"np.random.seed(123456)\n",
"\n",
"# Generate 100 random patterns for class1\n",
"mu_vec1 = np.array([[0],[0]])\n",
"cov_mat1 = np.array([[3,0],[0,3]])\n",
"x1_samples = np.random.multivariate_normal(mu_vec1.ravel(), cov_mat1, 100)\n",
"\n",
"# Generate 100 random patterns for class2\n",
"mu_vec2 = np.array([[9],[0]])\n",
"cov_mat2 = np.array([[3,0],[0,3]])\n",
"x2_samples = np.random.multivariate_normal(mu_vec2.ravel(), cov_mat2, 100)\n",
"\n",
"# Generate 100 random patterns for class3\n",
"mu_vec3 = np.array([[6],[6]])\n",
"cov_mat3 = np.array([[4,0],[0,4]])\n",
"x3_samples = np.random.multivariate_normal(mu_vec3.ravel(), cov_mat3, 100)"
],
"language": "python",
"metadata": {},
"outputs": [],
"prompt_number": 35
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"\n",
"
\n",
"
\n",
"\n",
"## Plotting the sample data\n",
"To get an intuitive idea of how our data looks like, let us visualize it in a simple scatter plot."
]
},
{
"cell_type": "code",
"collapsed": false,
"input": [
"%pylab inline\n",
"\n",
"import numpy as np\n",
"from matplotlib import pyplot as plt\n",
"\n",
"f, ax = plt.subplots(figsize=(7, 7))\n",
"ax.scatter(x1_samples[:,0], x1_samples[:,1], marker='o', color='green', s=40, alpha=0.5, label='$\\omega_1$')\n",
"ax.scatter(x2_samples[:,0], x2_samples[:,1], marker='s', color='blue', s=40, alpha=0.5, label='$\\omega_2$')\n",
"ax.scatter(x3_samples[:,0], x3_samples[:,1], marker='^', color='red', s=40, alpha=0.5, label='$\\omega_2$')\n",
"plt.legend(loc='upper right') \n",
"plt.title('Training Dataset', size=20)\n",
"plt.ylabel('$x_2$', size=20)\n",
"plt.xlabel('$x_1$', size=20)\n",
"plt.show()"
],
"language": "python",
"metadata": {},
"outputs": [
{
"output_type": "stream",
"stream": "stdout",
"text": [
"Populating the interactive namespace from numpy and matplotlib\n"
]
},
{
"output_type": "stream",
"stream": "stderr",
"text": [
"WARNING: pylab import has clobbered these variables: ['f']\n",
"`%matplotlib` prevents importing * from pylab and numpy\n"
]
},
{
"metadata": {},
"output_type": "display_data",
"png": "iVBORw0KGgoAAAANSUhEUgAAAcMAAAHPCAYAAAAvRLGlAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzsnXd8VGX2/z93JmWSSQ9JJoUUIGAoSwcBlQACK4guZX+7\n6q6Cfd1dxbrWBdRVEHR13a+7rutSdFmVXhKBUEKRIgQILXQmpPfeJpm5vz+ONzOTzIRJMi0z5/16\n3VfIc9u5E71nPs9ziiCKogiGYRiGcWNkjjaAYRiGYRwNO0OGYRjG7WFnyDAMw7g97AwZhmEYt4ed\nIcMwDOP2sDNkGIZh3B52hgwDQCaTYdKkSd2+TnJyMmQy/t+KYXoa/H8t4xTIZLJObatXr7a6DYIg\nWOUa1riONUlPT2/3+SmVSkRFRWHixIl45ZVXcPr0aavdb/78+ZDJZLh586bVrmkrepKtjG3xcLQB\nDAMAixYtMnIioiji448/RlVVFRYuXIigoCCj44cPH27V+1+8eBG+vr7dvs6aNWvQ0NBgBYusT3x8\nPObPnw8A0Gg0KCkpQUZGBlasWIEVK1bgwQcfxOeffw6lUtnteznbF4KO6Em2MraDnSHjFCxatKjd\n2MqVK1FdXY2FCxciNjbWpvfv37+/Va7Tu3dvq1zHFsTHx+PPf/5zu/HMzEw8/PDDWLt2LcrLy5Ga\nmtrte4miiJ5S3Kon2crYDp4mZXoc0rpcc3Mz3n77bQwYMAAKhQILFiwAAFRXV2P58uWYPHkyYmJi\n4O3tjfDwcNx///04evSoyWuaWjNcvHgxZDIZ9u/fj/Xr12PMmDFQKpUIDQ3FAw88gPz8fLO2GSJN\nUy5ZsgSnT5/GzJkzERQUBKVSieTkZBw5csSkTQUFBViwYAHCw8Ph6+uL4cOHY82aNUbXswZDhw7F\n7t27ERYWhh07dmDLli1G+zdv3ozf/OY36N+/P/z8/ODn54dRo0bh008/bedEZDIZ1qxZAwBISEho\nnZZNSEhoPSYjIwPPPfcchg4ditDQUPj4+KB///546aWXUFlZ2c4+jUaDv/3tbxgxYgRCQkKgVCqR\nkJCAX/ziF9izZ0+74y9evIj58+ejd+/e8Pb2hkqlwkMPPYTLly932lbGfWBlyPRY5syZgxMnTmDG\njBmYM2cOwsPDAQAXLlzAm2++iYkTJ2LWrFkIDg5GdnY2tm7diu+//x7btm3D9OnT213P3HTZZ599\nhq1bt+L+++/HpEmTcPToUXz77bfIzMzE6dOn4eXlZdF1Tpw4gQ8++ADjx4/Hk08+iezsbGzYsAFT\npkzB6dOnjdRpcXExxo0bh5s3b2LixIkYP348CgoK8Mwzz2Dq1Kkd3qcrhIWF4amnnsK7776L//73\nv7j//vtb97322muQy+UYN24coqOjUVVVhT179uC5557D8ePHWx0KQAp/8+bNyMzMNJreNpzm/uKL\nL7B582YkJydj2rRp0Ol0OHHiBD766CN8//33OHbsGPz8/FqPnz9/Pr755hsMGTIEjzzyCHx8fJCX\nl4cffvgBO3fuxJQpU1qP3bFjB+bMmQOtVotZs2ahX79+yMnJwcaNG5GSkoJ9+/a1TrFbYivjRogM\n46TExcWJMplMzM7ONhqfOHGiKAiCOHToULGsrKzdeVVVVSbHc3NzxaioKDEpKandPkEQxEmTJhmN\nLVq0SBQEQQwMDBTPnTtntO/BBx8UBUEQv/vuu3a2yWQyo7F9+/aJgiCIgiCIq1evNtr3+eefi4Ig\niM8884zR+KOPPioKgiC++uqrRuOZmZmit7e3KAiCuGTJknbPYQrp/m2fry179uwRBUEQ4+Pjjcav\nX7/e7lidTic+8sgjoiAI4rFjx4z2SeNt/24S2dnZok6nazf+5ZdfioIgiMuWLWsdq6ysFAVBEEeP\nHm3yHMO/c3l5uRgUFCSGhYWJWVlZRsedO3dO9PPzE0eMGNEpWxn3gadJmR7LO++8g5CQkHbjAQEB\nJsejo6Mxd+5cXLx4Ebm5uRbf59lnn8WgQYOMxp544gkAwPHjxy2+zh133IGHH37YaOzRRx+FXC43\nuo5Go8H//vc/BAUF4c033zQ6/mc/+1m7a1iLqKgoAEBJSYnRuKlpQ0EQ8OyzzwIAdu3a1an7xMbG\nmlS1CxYsgL+/v9H1pOO8vb1NnmP4d16zZg2qqqqwZMkS3HbbbUbHDRo0CI8//jhOnTqFrKysTtnL\nuAc8Tcr0SARBwJgxY8zu/+GHH/DJJ5/gyJEjKCkpgUajMdqfl5eHmJgYi+41atSodmPSuRUVFRbb\nbOo6Hh4eiIiIMLrOpUuX0NjY2LpG2ZYJEybg3//+t8X3tRTxp/W/tk6nrKwMy5cvR2pqKq5fv476\n+nqj/Xl5eZ26T3NzMz7//HN88803uHDhAqqrq6HT6UxeLyAgALNmzcK2bdswbNgwzJ07F3feeSfG\njBnTLvpXWns9ffo0Fi9e3O6+0pphVlYWkpKSOmUz4/qwM2R6LBERESbHN23ahHnz5sHX1xdTp05F\n3759oVQqIZPJsG/fPuzfvx9NTU0W38fUGpKHB/2vo9Vqu3Ud6VqG16mqqgJg/vnMjXcXKSAoLCys\ndayyshKjR4+GWq3G2LFjMX/+fISEhMDDwwMVFRX45JNPOvVZAsCvfvUrbN68GX379sXs2bOhUqng\n7e3dmk7T9nrffvstli1bhrVr17ZGHSsUCsybNw8rVqxoXSsuKysDQGuS5hAEAXV1dZ2yl3EP2Bky\nLsdbb70FhUKBEydOYMCAAUb78vLysH//fgdZZhkBAQEAgKKiIpP7zY13l3379gEAxo4d2zr273//\nG2q1GosXL26XlnHkyBF88sknnbrHiRMnsHnzZkydOhXff/+9UeStKIpYtmxZu3MUCgUWLVqERYsW\nITc3FwcOHMCqVavw9ddfQ61W48CBAwCAwMBAAMCZM2cwePDgTtnFMLxmyLgcV69excCBA9s5Qp1O\nh0OHDjnIKstJSkqCQqHAmTNnUFtb226/LZ6huLgYn3/+OQRBwEMPPdQ6fvXqVQDA3Llz251j7kuF\nXC4HYFo1S9e777772qWgHDt2DI2NjR3aGRMTgwcffBA7d+5E3759cejQodYp5nHjxgFAq3O0hI5s\nZdwLdoaMy5GQkIDLly+joKCgdUwURSxevBhZWVlOX3HE09MTv/71r1FZWYl3333XaF9mZqZRKoM1\nyMzMxNSpU1FWVoYZM2bg3nvvbd0nBc9IqlHi1KlTeP/9901eLzQ0FACQnZ3dbp+56xUXF+P3v/99\nu+NLS0tx9uzZduO1tbWora2Fp6dna2rLggULEBQUhCVLlpgMbNLpdEhPT7fYVsa94GlSxqkRzVQG\nMTcOAM8//zyefvppDB8+HHPmzIGnpyd++OEHZGVltQZj2JKObLOUpUuXYu/evfjggw9w7NgxjBs3\nDgUFBVi3bh1mzpyJzZs3d7og+I0bN1oDS5qbm1FaWoqMjAycPHkSgiDgt7/9Lf75z38anfPwww9j\n+fLlWLhwIfbt24d+/frhypUrSElJwdy5c/HNN9+0u8/dd9+NFStW4IknnsCcOXPg7++P4OBg/P73\nv8fo0aMxYcIEbNy4ERMmTMCECRNQVFSEHTt24LbbbkNUVJTR55ebm4sRI0ZgyJAhGDJkCHr37o3q\n6mps374dRUVFeO6551qDjEJCQrB+/XrMnj0bt99+O6ZMmYKBAwdCEATk5OTgyJEjqKioMAoA6shW\nxs1wXFYHw3RMfHy8yTzD5OTkdrl8bVm1apU4bNgwUalUimFhYeKcOXPEc+fOiYsXLxZlMpm4f/9+\no+NN5eGZO1YURfHGjRuiIAjiggULbmmblOdnLi8wPj5eTEhIaDeel5cnPvLII2JYWJjo4+MjDh8+\nXFyzZo24fv16URAE8ZNPPunwM5BIT08XBUEQZTJZa76jj4+PGBUVJU6cOFF85ZVXxMzMTLPnX7hw\nQbzvvvvE8PBwUalUiqNGjRK//PJLUa1Wm/wMRFEUP/roIzEpKak1J9Lw+crLy8VnnnlGjI+PFxUK\nhdivXz/xjTfeEOvr69t9FpWVleLbb78tTp48WYyOjha9vb3FqKgocdKkSeI333xj0l61Wi3+4Q9/\nEBMTE0WFQiEGBgaKSUlJ4sMPPyxu2bKlU7Yy7oMgis5RlO/RRx9FSkoKwsPDW6dFXn75ZWzfvh1e\nXl7o27cvVq5c2bpIzjDuyhtvvIH3338fO3fubK1GwzBM93CaNcMFCxZgx44dRmPTpk3D+fPnkZmZ\nif79+5tdo2AYV8RU7dOzZ8/ib3/7G0JDQzFx4kQHWMUwronTrBneeeedUKvVRmOG33rHjh2LDRs2\n2NkqhnEco0aNQmJiIgYNGgSlUtm6VgdQLl3bmqgMw3Qdp3GGt+I///kPHnjgAUebwTB24+mnn8bm\nzZvxzTffoKamBsHBwbjnnnvw0ksv4a677nK0eQzjUjjNmiEAqNVqzJo1q10o9V/+8hecPHmSlSHD\nMAxjE5xeGa5atQqpqakm+5YBQL9+/XDt2jU7W8UwDMM4M3379m0t8mAJThNAY4odO3Zg+fLl2LJl\nCxQKhcljrl271tqpmjfLt0WLFjnchp648efGnx1/bj1j66xIchpn+MADD2D8+PG4dOkSevfujf/8\n5z/44x//iNraWkydOhXDhw/HM88842gzGYZhGBfEaaZJ//e//7Ube/TRRx1gCcMwDONuOI0yZOxL\ncnKyo03okfDn1nX4s+sa/LnZB6eKJu0KgiCghz8CwzAMY2U66xucZpqUYRiGsS4hISGtLa5cleDg\nYJSXl3f7OqwMGYZhXBR3eD+ae8bOPjuvGTIMwzBuDztDhmEYxu1hZ8gwDMO4PewMGYZhGLeHnSHD\nMAzj9nBqBcMwDGOETtQhIz8De67vQY2mBiMiR2BywmQE+wQ72jSbwakVDMMwLoq592OtphYHsw/i\nZMFJKD2VSE5IxtCIoRAEAQDw7flvsf3SdgT7BMNL7oXyhnIEK4Lx5l1vGjnEmqYaFNcVI8A7AGHK\nMLs9lyHWSq1gZ8gwDOOimHo/1mpq8d7B95Bfk49gRTA0Wg1qmmowa8As/HLQL1FcV4w/pf0JMQEx\nkMvkrefdrLqJe/rdg18O+iV0og4bL2zEjms7IIoidKIOwyOH47Hhj0HppbTYvqqqKhw6dAgzZ840\nGh8zZgy2bNmCyMjILj1jR+Pm4DVDhmEYNyJdnY78mnzEB8UjUBGIMGUY4oLikHolFUW1RbhRcQMA\njBwhAIT6hCKjIAMAsOf6Hmy5tAUqPxV6B/ZGbGAsTheexsrTKztly549ezBjxgwAQEZGRuv47Nmz\nIZPZ1z2xM2QYhnEjjucfR6hPqNGYXCaHCBHXKq5B4WG6d6xGq4G/lz9EUUTKlRRE+kfCQ0ZhJ4Ig\nICYgBhn5GSitL7XYFkEQWqdmly1b1joeEhIChUKBjRs34v333+/sI3YJdoYMwzBuhI/cB826ZpP7\nvOReuK3XbVB6KVHVWNU6rhN1KG8ox5SEKdCKWlQ2VsLHw8foXJkgg0yQobqp2mJbTp06BQBIS0uD\nv78/AGDdunWIiopCYGAgRo4cCY1G09lH7BLsDBmGYdyISQmTUNFQAZ2oax2r1dRC4aFAUq8keHt4\nY+HtC9Gsa0Z2ZTbUlWrcrLqJ6f2mY0zMGMgFOeIC41DVVGV03RZdCwRBQLgy3GJbZDIZ4uPjkZaW\nhsbGRvTp0wf5+fmYNWuW1Z7XUjiAhmEYxkUx9X7UiTp8lfkV9qn3AT/t8vbwxh/H/BGDIwa3HtfY\n0oiLpRfR0NyA+KB4RPrrg1nOFJ7Bh0c+RJAiCEGKIDS0NKCgpgCzk2ZjTtIcq9mfnZ2NVatWYdGi\nRZ16xo7GzV6HnSHDMIxrYu79KIoicqtzcb3iOrw9vDE4fDD8vPw6de2zRWex/sJ6ZFdlI8QnBPcm\n3ovkhGTIBOtNOKrVaqxevZqdoSWwM2QYhjGNPd6PWp0WMkHWGghjLWpra/H5559j//79eO+99zB4\n8GCTx7Ez/Al2hgzDMKZxh/cj5xkyDMMwjJVgZ8gwDMO4PewMGYZhGLeHnSHDMAzj9rAzZBiGYdwe\ndoYMwzCM28POkGEYhnF72BkyDMMwbg87Q4ZhGMbtYWfIMIzr4OLVVhjbwc6QYRjXYe1aYN8+R1vB\n9EA8HG0AwzCMVSguBtLSAKUSGDcOUJju2M7cmldfBQoL24+rVMDSpfa3xx6wM2QYxjVITQU8PIC6\nOuDQIeDuux1tkdNyK2dXWAjEx7ffr1Z3fL7hNSyhqqoKhw4dwsyZM43Gx4wZgy1btiAyMtLMmdaH\nnSHDMD2f4mLgwAEgOhrQaIBNm4A77mB1aIZbObuunt+ZawDAnj17MHv2bABARkYGRo4cCQCYPXs2\nZDL7ruLxmiHDMD2f1FRAJgPkcsDHR68OGadGEITWPojLli1rHQ8JCUFNTQ02bdqEJUuW4OTJkza3\nxSmc4aOPPoqIiAgMGTKkday8vBxTp05F//79MW3aNFRWVjrQQoZhnBZJFRpOqUVEkDpsbHScXcwt\nOXXqFAAgLS0N/v7+AIB169YhMjIS27ZtQ3R0NF544QWsWLHC5rY4hTNcsGABduzYYTS2dOlSTJ06\nFZcvX8aUKVOw1FVXbRmG6R4HDpASLCgAcnNpKyujLSPD0dYxHSCTyRAfH4+0tDQ0NjaiT58+yM/P\nx3333Yfnn38eY8aMQU5ODhISEmxui1OsGd55551Qt5lo3rp1K/bv3w8AeOSRR5CcnMwOkWEYWhO8\neRPo149+nzwZGDbM9LEqlf3sciFUKtNrf9b+OBcvXozFixd3eMymTZvwxhtvWPfGJnAKZ2iKoqIi\nREREAAAiIiJQVFTkYIsYhnEKfviB8gk/+AAIDgZCQmhjLOZWzs5ZdMfWrVvx7LPPIi8vD4mJiTa9\nl9M6Q0MMF1kZhnFjmpqAjRtpWnTXLuBXv3K0RT2S7jo7c85U2mcNNm3ahPfeew+ffvopkpOTba4O\nndYZRkREoLCwECqVCgUFBQgPDzd7rKHMTk5ORnJysu0NZBjG/hw+DNTUAAkJlGA/bRqpQ8au2EM5\nzp49uzXtwhLS09ORnp7e5fsJougcxfzUajVmzZqFs2fPAgBeeeUVhIaG4k9/+hOWLl2KyspKk2uG\ngiDASR6BYRhb0tQEvPQSpU74+gI5OeQMWR2axR3ej+aesbPP7hTRpA888ADGjx+PS5cuoXfv3li5\nciVeffVVpKWloX///ti7dy9effVVR5vJMIwjkVShry/9rlKROqyocKxdjEvgNMqwq7jDNx+GcXua\nmoAXX6R/S84QAPLzgVmzWB2awR3ej9ZShk67ZsgwDNNKeTkQGkppFYb07k37GKabsDJkGIZxUdzh\n/ehSa4YMwzAM40jYGTIMwzBuDztDhmEYxu1hZ8gwDMO4PewMGYZhGLeHnSHDMAxjmqIioBslznoS\n7AwZhnEMOp2jLWCOHSOHZ44NG4AvvwRKS80f0zb3s4fCzpBhGPvT0AC88w5VkHEE5vLP8vKAn/qo\nujxVVcA//0kOzxS5ucCPPwJeXsD335s+pr4e+POfgRs3umhCFVJSUtqNjxkzBgUFBV26ZldhZ8gw\njP05dIi60G/fbv97nzwJfPZZe4coisB335EScod6p7t3089jx6joeVu2biVHGBkJ7N1rWh3u3w9c\nvAhs3mz+C0YH7NmzBzNmzAAAZGRktI7Pnj0bMpl93RM7Q4Zh7EtDA708Bwyg4tv2VIdaLfDNN+SM\nr1833qdWA6dPAx4ewM6d9rPJEVRVkdqLjAQUCmDbNuP9kipUqejzkMvbq8P6enKYAwYAmZnmGxx2\ngGGv2mXLlrWOh4SEoKamBps2bcKSJUtw8uTJTl+7s7AzZBjGvhw6RM15lUpSHvZUhydP0hpZUBCw\naZNezYgiOWhfX3IQu3e7tjrcvZvWbL28gIiI9upQUoWSOlOp2qvD/fuBxkZqqaVQdEkdnjp1CgCQ\nlpYGf39/AMC6desQGRmJbdu2ITo6Gi+88AJWrFjRrce1BHaGDMPYD0kVRkTQ7yqV/dShVgusW0cF\nv8PDgbNn9epQUoXh4YCnJ73UXVUdSqpQakkvkxmrQ2ndVKulf+fl0ReIqipgxw46RlKF0t8xIqJL\n6lAmkyE+Ph5paWlobGxEnz59kJ+fj/vuuw/PP/88xowZg5ycHCQkJFjn2TuAu1YwDGM/JFUYFka/\ny2R6dfjkk7a9t6QKpRerry+pwxde0KvCn6bsoFKRepo+HQgOtq1d9mb3bur04dHm9b9/P7XD8vam\nllimVJ7kQCVVKDlDQdCrw4UL9Z/jLVi8eDEWL17c4TGbNm3CG2+8YdH1ugM7Q4Zh7IOktrRa4yk5\nnY7Wp+bNA0JCbHNvQ1UoIanDAwdInQYE0FqZRFkZsGuX6/VKDA8HZs9uPy4I9OUkPByYM6fja6Sn\n09/N8O8oikBWFlBSQtewAlu3bsWzzz6LvLw8JCYmWuWa5uAWTgzD2I+CAmrU2xaZDIiJ0a9RWZuz\nZ4GlS2l9y5DaWmDkSMDcizY6Ghg92jY22QGbvR+rqkznF8pk9IXGQmXYEZs2bcJ7772HoKAgJCcn\nm1WH1mrhxM6QYRjXp7GR1r5MERCgn7Z1Mdzh/cjO8Cfc4Y/NMAzTFdzh/cjNfRmGcV1++MFx1WkY\nt4SdIcMwzkVlJfCvfwHfftulqiYM0xXYGTIM41ykpVHFk9Onu1TVpFMcPEj3YdwedoYMwzgPlZWU\nfhEZSZGfXax5aRH19cDatcCaNUBzs23uwfQY2BkyDOM8pKWR8/P0pIRuW6pDKXG8vBw4ftw292B6\nDOwMGYZxDiRVKFU5EQTbqUPDcmKhocD69awO3Rx2hgzDOAe7d1PVl5ISiiTNz6fKMUeOANnZ1r2X\npAoVCsDf32XVYXBwcGtnCFfdgq1ULo/LsTEM4xzExAAPPmh6n5eX9e7Ttsg0oFeHo0fTFK2zUVVF\nTruTFXrKy8ttZJDrwc6QYRjn4PbbabM1Bw9SWbiGBuPxykpqOGwPGzqDRgO89x5w333AhAmOtsZl\nYWfIMIx70b8/daowRUyMfW2xhGPHKIho3TpgzBjnVK4uAJdjYxiGcVY0GuCVV6jdUkkJtblidWgR\nXI6NYRjX5+BBCrZxdY4do+lbPz8qJs5RrzaDnSHDMD2L4mLg88/1ndldFY0G2LAB6NWLfvfzo6jX\nH390rF0uCjtDhmF6Fjt20LrZ/v3kGF0VQ1UowerQZrAzZBim51BcDOzbR4EucjmQmupoi7pOZSWl\nTJjj+HHqJp+drd/Kyqgh8Y0b9rPTTXD6aNL3338fX3/9NWQyGYYMGYKVK1fC29vb0WYxDOMIduwg\nJyiXU/3SAweAGTOA8HBHW9Y5RBH4z38ob3DhQtPHPPcc0NJieh+/A62OUytDtVqNL774AidPnsTZ\ns2eh1WrxzTffONoshmEcgaQKIyPpd7mcnElbdajTUeSlM3PjBpCZCZw6Zb72qlxOTs/Uxlgdp3aG\nAQEB8PT0RH19PVpaWlBfX4/o6GhHm8UwjCPYsQOoriZHV1hImyBQcW/DtcOTJ4F33qHpRGdEFKne\nqq+vvvYq43Ccepo0JCQEL774ImJjY+Hj44Pp06fj7rvvdrRZDMM4gpgYYM6c9uOCoP+3Vgt89x1V\nmElPB+69127mWYykCuPj6feMDBpLSHCoWe6OUzvDa9eu4eOPP4ZarUZgYCB++ctf4r///S8eeugh\nR5vGMIy9mTz51secOkUqsV8/Sr1ITjaOxnQ0hqpQEGhKNysL+PvfgQ8/dLR1bo1TO8MTJ05g/Pjx\nCA0NBQDMmTMHhw8fbucMFy9e3Prv5ORkJCcn29FKhmGcAkkVBgfTulpzs/Opwxs3yGH37k32FRQA\ndXU0BfzMM0Dfvo62sMeSnp6O9PT0Lp/v1OXYMjMz8dBDD+H48eNQKBSYP38+xowZg9///vetx3A5\nNoZhAAAnTpDCkqYfm5ooSf3DD51HHe7eDWzaRP8WReDoUXLijY3A888DTz/tWPtciM76BqdWhkOH\nDsXDDz+MUaNGQSaTYcSIEXjyyScdbRbDOB9aLb1cPZz6f2nLqK+nlk2deRZDVSi9AL28qIpLV9Sh\nKAJffglMn04qzlrcfTdtAK0V1tXRWmFVFUWVarUURcrYHaeOJgWAV155BefPn8fZs2exevVqeHLF\ndoZpz3ffAf/9r22uLYrAtWvW7zZv7l6ffgqkpHTuvMJCih6tqDBOUhdFClbpLFlZwPff2y7SU6ul\nLhQhIfR7YCBNmZ4+bZv7MbfEBb5GMoybU1ZG6QWiCNxzj/UT0G/cAN5/H/jTn4DEROteuy2XLwPn\nzpHznTyZGtpaQnQ08Nln1rFBFKkmaGQkTb1mZwNxcda5tsTp0+T8DCNIQ0LIQQ4bxurQATi9MmQY\n5hbs3EmRibYoTyZFP1ZVARs36tWhLVSi5IQCAqjyyr591r+HJWRlAVevAqGhlAe4dav175GeTpGk\nOTn6rboaKCqiezN2h5Uhw/RkysooKCMqihzi/v3WLU8m5cQlJQEXLtCL2teXAlWeftq6iunyZeDS\nJQqAaW4Gtm8HJk2yXB1aA0OHLAj0OdpCHT71FNDQYHrfT9HzjH1hZcgwPRlJFXp46Gt2WksdGubE\nyWSAUkmOYvlyWqNcv94695HutWEDRX0KAgW/OEIdSqpQWsuTyWyjDqX+hKY2Gb+WHQF/6gzTU5FU\noVSrE9AXr7ZGayNJFUoqMyyM2gqlppKzSkszX1ezs0iqUOrdBwAqFanDmhrr3MMStm8nxZabq5++\nbGqiFIi8PPvZwdgdniZlmJ7K8eOUOlBQYDze1EQNYLuTbC6pQg8Puoc0duUK5e4lJpJz2LABePFF\n/XnFxZTe0Nmo7+PH6fo5OcbjOh1Nz44d2/Vn6QyzZwNTp5reJ6lFW7B7NxARAQwZYrt7MB3i1En3\nlsBJ90yPp6WFpjcNa2xagkZDQRem8PfvXneDpiZg6VLquSdRXk4pD97eQJ8+pNhUKuCf/6R1vqYm\n4PXXKTefQ93LAAAgAElEQVRv2rTO3a+lRe9026JQuPbUYXU1JdyHhgLvvecauaJOgEsl3TOMyyOK\nwD/+QQEqnS1C7+VlPK1oTby9gUWL9L+LIvDCC6SOoqJozNfXWB0ePUoqddMm4I47aL+leHi4rxPY\ns4c+3+JiKtU2erSjLXJLXPjrFsP0ANRqmtLctMl8dKEzcP06TeVJL+3iYlKNtbXArl20vrhhA+X7\nNTYChw452uLOcfaseWVqS6qraQ1WpdLnGZpr6MvYFDf9KsYwToC0LqdUUgmyQ4fMr1c5Go2G2ieZ\nchhJSeQsq6tpulQu75o6dBSFhcCKFcCCBdTlwp7s2UPVaLy8aFOrWR06CHaGDOMo1GqK1oyLIzW1\neTM5EB+frl1PFCknbsQI61cwSUoCliwxva+pCXj5ZYo2BWiNr6iInHtn1w4dQUoKqbENG4Bx4+zX\nSd5QFUpI6nD4cPedNnYQPE3KMI5AUoUKBQXO+Pjo1WFXuXIF+OQT+9e3PHqUpkwVClI5Wi2tZW7a\nRM/kbBgGVRQW0mcuBQQdOWI/O9LTgZISID+fvhip1RSkdPVq1+qpMt2Cv3owjCMwVIUS4eFdV4dS\n0rog2L++ZVaWXg0a4uVF+Xr9+3f92lVVpJCUStP78/IoLeMXv7D8mt9+C8TGAuPHkyqUChaEh9tX\nHQ4ZQoraFNHRtr8/YwQ7Q8Z9KCoiJ2SvnLWOOHSI1t9yc43HNRrgzJnO23jlCnDxIikctZrU4ciR\nVjO3Q6zVgy81laYJb7+dfhdF4F//oo4O5lq3bdgAHD5Mz2pJq6XiYmqkGxgIxMTQ3yEmhvb5+tL+\nI0fss3aYkGBcqJtxKOwMGffhu+8oOOG22+hl6EjmzDG/ntbZ5O62pcycrfvB6dOkvgYPNn+MVAhc\nqaQ1Ty8v6lxx7hw906xZxpV2AKoXmpFBdUS3bgUMmn4DoM+lvt5YVaamki1VVVRfVaOhnoISCgWV\nmbPn2iHjFPCaIeMe3LxJwSWiSBF8jkappIojprbOVm+RVKGUc+hMvfE0GmDlSmDVqo5TBnbvpmoz\nVVWkzESR1hyVSnKM27a1P2fLFppOVqloqrRt9Zrz54HFi/UpK8XFVKouMpKCfTIyqJJOSIh+i4mh\nqWt7loBjnAJWhox7sGULfesPDaWmrVOmOF4dWgNJFUqOREIudw51ePSo3q4TJ/RToIZUVdHfRKWi\nbhUbN5KzOneOUjVEkaZCDdWhpArj4qg6jbe3sTrU6UjhXbumT1lJTaVj5XJysr17U1eMiRPt8lEw\nzg0rQ8b1kVRhRASpDK3WOdShNdBqqRbokCFUGUbabruNVE5jo+Ns02j0ji00lJyTKXUoqUIvL3JS\nlZXU7V6ppClSmay9OpRUoVSmLSLCWB1euECFABITKSgpO1uvCiXCwsg+RyTbM04HK0PG9ZFUofTi\nVKlcRx16eBgHsIgiKaS77iIn6UiklIv4ePpdrW6vDg1VoYSXF6UdzJ5NThKgSM8ffiB12NJCas/H\nx7hrRmUldZ146ilaD/Tyor97cTHwxRdULSc/n64p/bdQV0fryB0FLOl0wNdf070d/ZkyNoOdIePa\n3LxJL86gIMopkygvJ3U4Z47jbLMFV68CX31FgSMPPGDfezc1kQMSBGNVKCGpw1Gj9AnlBw7Q30Jy\negCpu8ZGKpEWEaEf9/KiVk9DhwJ/+INpG4KCqM1UWpo+GCciglIwXnuNrr1vH/D443qHeKs0hsxM\nWr/08gJ+/evOfy5Mj4CdIePayGT0jd5U9XprdYO3NxoNUFqqL5gtIYrkgEJCaOpx+nTbth0yRKcD\nPvwQmDCB1uCOHaOpydBQoKJCf1xZmbE6HDHCvDOKjTVfiPyuu8zb8etf02eRm0tOd8YM2ldcTFOp\nubmkLocOtey51q0jG6XPlNWhS8LOkHFtYmKA3/zG0VZYl/R0ShZfutQ4Of/qVVori4+nF/7OnfZR\nh6JI9z17lnI5x42jLxoLFpg+3tDBRUdbN8H84kVScg0NtJ6an0+BNuHh9EWhsZG+RKxbR+ust2oN\nlZlJqjIhgVTlzp2sDl0UdoYM05Oor6cpu8pK48LekiqUgk4iI22jDrVaUnx9+ujH/vMf6rwRFqZP\njZg4ERgwwHr3tZSLF4GBA+mLgb8/fU7h4VSI+69/pTXCoCBaazx7tmN1KKlCSQmqVKwOXRiOJmWY\nnsShQ6RuYmMpSlLKoZNUobRGJ63J7dpl3fsfPw68/z5N0wKkljZupOCWwED7RmiKIjnimzfp96oq\nSp+oqaFUCymA5sYNCirKyyNHKAj0c90647XKtkiqMCiIfvf0pHvu3Nk1Wxmnhp0hw/QUJFUYEUHT\no3V1+sLemzZRtGRODjmHmzcpZy8lxXjNrju0tFAATHU1lTQDKFK3sJD2FRaSMpXUoa25fJmc34YN\n+mIKBQXkwGQy+qIgl5Pj/vvf9RV6AHJwOTmkDs2xcSN95tnZ+q2piZ7dMKfTEjZvBvbu7fqzMjaH\np0kZpqcgqUIpwjIiQl/Ye/x4WgNri0xmvVZAJ06QY0lMpBf7kCGkPFtayLlcuEA/Kyv1Ba+9vKxz\nb4CcV2Eh9fqTig2Eh5OCU6tJuQUGGucfyuVkQ1mZPkpVcogtLRR1am6qdPZs03magtC5Um0VFZTy\n4eVFgUM9ocejG8LOkGF6AvX1+kjR5mYa8/AglWaPpsCSKgwNpfvKZMBHH9GLXhRJMdXUkDqrqKD9\nWVmWRWxagihSrt+NG7QmmJsLXLpEwUJFRfSlYOFCWqtsW0qtpYUiSX/4gZz1lCn6fea6YQAU6WoN\npKnqxsae0+PRDWFnyDA9gYICUjyNjcZqJSCASo7Z2hlKqlBKoPf1pSnKmTNpXQ4gh3P0KCnVfv06\nLszdWaT6qzIZ5QmeOaOf9oyI0KvDhAT6TNqiVAL/+Aepx3nz6Fx7UFFB6lOlouCjTZvo82F16HSw\nM2SYnkDfvhQN6QgkVahU6h3x1as0fvo0OT6AVFtTE73or1+n36V93cGwK4e/PylEuVx/bUEghyyp\nQ2ka1JCdO8mRNjeTM501q/t2WcKuXWS/pydtRUWsDp0UDqBhGGdHFEkVOSoisaKCnI1CQc6kuZmi\nWSdNojWwl18GnniCpnCnTaOpVKWSVJA1bDbsyqFQkBK+edM4WKipSZ9Q35bSUpq+ValIRW7fTsFG\ntsZQFUpERNDnUl9v23tfvkxpIIzFsDJkGGfnyhXggw+onFhiov3uu3cvvcgHDgTefbfjY9eupdw7\n6cUvdZ24dq176rBtr0YAGD6cAmIee8y46EBpqelC4JIq9PCgzV7q8OBBWtNt+4WgtpYct626Zeh0\nVJIvL49K30mpIUyHsDNkGGdGSqavraWfr7xiehrQ2lRXk4MLCyNH2FEbqPJyfXskKecPoECWzZuB\nl17quh0XL1IFmZAQWjcF6PlFkaJWJ0ygMVEkO0+coC8N0mckqULD0nWSOpw0qf3aYUODsYPtDuPH\nm//y0rZRsSFpabTe2tExHXHuHP0d5HJ69rlzu3YdN4OdIcM4M1euUFRmUhKlLly9ah91uHcvKYz8\nfApO6Siy0tOTSt6ZmhI1FcxiKVotJdXffnv7OqyAcRHwrCz6bACaIpSq3xw+TFOShkXaAcrR/PFH\nYPJk/VhNDfD228DvfmdcYaer9OplvraqOYqKqBHy+PH63oydQaqaExREjl7qzsLq8JawM2QYZ0VS\nhdIUoZ+ffdRhdTUl66tU5EjWraMUCXPq0N8f+PnPrW/H6dMUhOPnRzVWzT2zNJUaEKD/t6QOJ040\nnX8JGDtTgKZOL1+mQgLmAnFsTUoKrYv++CNN48bGdu58SRXGx+sVNKtDi3D6AJrKykrMmzcPSUlJ\nGDhwII4ePepokxjGPkiqUFIXvXrp1aEt2buXVJmXFykKSR3aE62WnHDv3hSZeuGC+WMlVRgSQsE7\nly6RUwMoCT8hwfRmOEVaU0OOaOBAcsKGfRLtRVERrTNGRZFDNGxmbAmGqlBy5FLvzspK69vrYji9\nM3zuuecwY8YMZGVl4cyZM0hKSnK0SQxjeyRV6OFB6QwNDfTTw4PGbRVZaqgKJYKD6SWr1drmnqY4\nfZrWCAMDSfGtX2/6mQ1VoSDQ5u+vL9FmKfv2UT1Vb29KDdm82f7Ruykp9PeVy2ld88cfjddgb8WF\nC8D58+T41Gra8vKAkhIuBWcBTj1NWlVVhYMHD2L16tUAAA8PDwT29M7kDGMJTU30TV9ShVotbb16\n0c+mJn2yuzVJT6fk+baFtrOzKdF9+HDr37MtkioMCSFVGBioV4eDBumPE0VykufOUasuKV3C25tq\njhquHXaEpAqlLwDh4Xp1mJBg9ccziaQKY2Lod5lMrw4tXTvs1Qt47jnT+wy/3DAmcWpneOPGDYSF\nhWHBggXIzMzEyJEj8cknn8CXqzcwro5CAbz+uv73tWvp5WwYKWkLkpKAF14wvc9eL1RJFapUND0b\nEkJOcP16msaUnv/SJeBf/yI1GxZmHKwyYAClX1iCoSoE6PqSOrTX2mFqKik6w/6KOh2wfz9w3300\nXXwrVCp2et3AqZ1hS0sLTp48ib///e8YPXo0Fi5ciKVLl+Ltt992tGkMYz/KyiiBuqWFHMBtt9nu\nXomJ9s1lNIX0rD/+SNPDOTnkEGtqSCH27aufHvX0pA4S4eHAW2913nGJIikynY7uI6HTUVpHWVnn\nI0K7wuDB9AymsFaqB9MhTu0MY2JiEBMTg9GjRwMA5s2bh6VLl7Y7bvHixa3/Tk5ORnJysp0sZBg7\nsHMnveQDAsgBvP66YyIdO+LMGXJcY8d2/1qPP05BO+++S06ipoYc0rPPAtHRdMylS7RVV5OaOn68\na18UBAFYvFhf/LztPnsty/z0jmO6Tnp6OtLT07t8vlM7Q5VKhd69e+Py5cvo378/du/ejUGGawY/\nYegMGcalkFRhVBQFVly+bHt12Fmamyk3TqOhFIzurmWGhlLiuZ8f/TskhKaIa2vpM5BUodQpIyCA\nplW/+65r6rCjzhVMj6GtEFqyZEmnznf6aNJPP/0UDz30EIYOHYozZ87gdcN1FIZxdSRV6OHR9UhJ\nW3PiBFWhqaujJPfu0ramp6EqFkW9KszLoylEqWaqpA4Zpgs4tTIEgKFDh+L48eOONoNh7I+hKpQI\nDXUuddjcTJGfUp/DjRupekp31GFaGqUDtE3lyMujnMJNm8j5lpXRNKbUsLc76pBxe5zeGTKM23Lp\nEikhU50YzpyxjzNsbKRSZlIfw7ZIqlDaX1xM6tCwzFlnGTyYojhNERBA06ZaLUV/SoW5pSLcFy/a\nNyWCcRkEUXSm+ZbOIwgCevgjMIxpbvXftT3Uz7ZtwI4dwPLl7RvSNjcDf/qTfvoWoCCaujpgxYr2\n6rCxkRz80KHdt6u4mO5jipgYijJl3JrO+gZWhgzjrDh6qq+2Vt/7b/9+4J57jPdnZFCqQ3AwqUOJ\nigrgyBHqCmHIgQPUmHfFCvNpBJbS3fMZpg3sDBmGMY2UjN67N7B1KxW9NlSHkZHAM8+YPrdtknhD\ng77Zb0oKsGCB7exmmC7AzpBhmPZIqlClorW5oqL26jAujjZLOHiQHGJ8PCnEmTNZ3TFOhdOnVjAM\n4wDaliiLiCB1WF/f+WtJqjAigvIEPTxIHTKME8HOkGEYY2prqadfYCA5soYGmt6sriZ12FkkVSiV\nFVOpSB0WF1vXbobpBuwMGYYxpqREPz1qSExM5x1YQwPlHioUVIi6spKqxjQ2sjpknApeM2QYxpiE\nBKoLag1qa6mDRNuWUJGRNGXaEVotcOwYMG6c4yNrGZeH8wwZhnFOMjKADz8E3nyTWje5C01NQGmp\nvig50yU66xt4mpRhGOdDavDr4UE/3ekLb1oa8MEHNJXM2A12hgzDOB9Sg9+EBErsz8pqf4wrOou6\nOkppKSmxTtFzxmLYGTIM41xIqjAkRN+xoq06rK8H/vxnKlreFqleaU9k3z6aJo2JocAjV3T4Tgo7\nQ4ZxRfbsoXWnnoikCqXGuiEh7dXhoUPkCDduNHaSzc3AO++YVpLOjqQKVSqq9FNby+rQjrAzZBhX\no6AA+Pe/KUn+VrRtk+RoJFUoCFTvtLycap3qdHp1WF9PSfwDBpDTu3JFf/7x48DZs8D69T1vnVFS\nhVJKS3g4q0M7ws6QYVyNbdtIWRw8SGXUzJGbCyxeTLmAzoJGA/TpAwwbRj+lbeRImjpsaSFV2NhI\nSfx+fnp12NxMTjAhAbh6ldo59RTq6ujLS0gIfQYaDQUPVVayOrQTnGfIMK5EQQG9PGNj6d8pKcCj\nj5o+dutWIDOTnOa0afa10xw+PsCTT5rfL6nCiAj6vVcvvTosLdX3VtRoyDG++WbPyFHMz6c2WM3N\ntEmEhAA3bjjOLjeCnSHDuBLbtgFeXoBMRmtPBw9SUWzJeUjk5FBC+223AZs3A3feqS+X1lnq64HV\nq4FHHmnf89DaSKpQeh5BIHW4bh11vg8NpfGQEL06TEqyrU3WIDGRcioZh8HTpAzjKkiqUKWi3zsq\nir1tG5VI8/UlZ3bwYNfve+AAkJpKjsrWpKfTuuLNm/qtoQH48Uea9i0rI6UoRaGuX0/rjQxzC7gC\nDcO4Cl98AXz/vV4dAeQ4GhqAjz/Wq6mcHJo+jIsjBdnQQPVCP/yw8+qwvh548UUK+mhqomvYUh3W\n1rYv7QZQw+Dr12naV6sFxowhm8rKqJzbn/5kO5sYp4Q73TOMu3LbbcaOUEIQSCFKSKpQ9tPEkI8P\nBdp0Ze3wwAH9tGV2NqlDW64/+vmZHn/lFWDnTvq3IAB33QXcey+wdCmlaqjVtJbIMGZgZ8gwrsKd\nd976mLIy4NQpUk+5ufpxnQ7YtQuYOtU44CQvj7YxY9pfq76eWj1JijMigoJb7rjD+uqwrAy4dAkY\nP970fm9vcsxxceT4jx2j1IuqKspX3LwZWLjQujYxLgU7Q4ZxJ0JCgCVLTOfgeXkZO0JRBNaupeT2\ngQPbqzJDVQiQ2iwqso063LKF8vAGDDCtfg8dovSEsDC97X/9K6nBwEBWh8wt4QAahnEnBAGIiqKO\nCG03yZFIXLsGnDtHuX3p6cb7mppoulWjMQ5m0WgoZcPUul5XkaZwPTyAHTva729oIOVnGDErCMD5\n86QYBYEc9ebN1rOJcTlYGTIM0x5RpClPpZKU1bZtQHKyXh16eABPPGG6DqiHx617FXaGlBS6nkpF\nZeZ+/nNjdXjiBDliX1+yTxSp/RNA4wMGkKM0pw41GlLFjFvDzpBhmPZIqjA+npRVczOpw3vvpf1y\nOVWJsTWSKoyJoXvKZKQOH3qI9osiMGgQ2dnYCDz7LEWVFhaSGhQEip4FKDF/7Vrg9df11y8ooOnU\nN97Q10Jl3BJ2hozTI4oi8mvycb3iOhQeCgwMGwill9LRZrkuhqpQWkOMiGivDu2BpAolpWmoDs+e\nBYqLgZ/9DKiupv0tLcDddwP9+xtfp7SUegRWVtLzSc+VkkJ5iXv2AHPm2O+5GKeDnSHj1OhEHb4+\n8zX23tjbOubj4YM/jvkjBoa7Ufdze3L9OiWxBweTcpIoKTFWh7amqAjYvZucb2GhfryqitYlT5yg\niNZTp6iUmdQI+N1321edWbmSHHpRkb4qTUEBBd4MGkRFA6ZMYXXoxrAzZJya43nHkXYtDQnBCZAJ\nFO9V01SDT3/8FB9O/xC+njYu/+UIGhq6XhrNGnh6ArNnm97Xq5f97BBF4J57TFeQKS6m6NHaWpr+\nnDKF1N6NG5R4P3y4/tiiIop8jYmhDhgbNtC0aEoKrRUqFHQPVoduDTtDxqnZd2Mfgn2CWx0hAPh7\n+6OsoQxZJVkYGTXSgdbZgPx8quLyxhuUBuEIYmOBBx90zL0NUamA3/62/XhjI1W9CQ8n51dWRl8g\nfH1Jza5bBwwdqi8qkJqqD+oJDSXnefAgqcLevfX3YnXo1nBqBePU1DXXwVPmaXKfRmvF8H1nYft2\nellL1VRcmbIy6rvY2Z6KUk5hfT1dQy4Hzpyh9UCAgn8yM+nfkiqU6rUKAk2pfvQRKWBpLdLLS68O\nGbeEnSHj1IyOHo3yhnKjMa1OCwEC+gT3cZBVNiI/nwptDx5Ma2Xl5bc+pyfz/ffk/E+ftvycxkZ9\nC6fmZiAykhRiWRkpuqgo6n1YX0/H79pFTjI3l8rFZWfT53rmDJ2Tk6PftFpg/37T6SKMy8PTpIxT\nkxyfjB9yfoC6Uo0QnxBotBpUNVbh3gH3IsIv4tYX6Els304KRcp527kTeOABx9p0KyoraR0uIaFz\n55WWAnv30jTlunWUpmFJbmJGBp0rdX+PjaWfNTV0jbZrnWPGUHNgQ0QR+M1vaLztVLSnJ6nOgICe\n0QeRsRrctYJxemqaarA/ez8y8jPg7+WPSQmTMEw1DIIrvazy8yn/LTaW1rpaWijacflyx60dWsKX\nX5LKWr68c4nrX39NkakxMbTu94c/UO3Re+8FgoLMn1dVZRzhakivXt0P8KmtBd56C3jsMVLoTI+F\nu1YwLoe/tz/u7X8v7u1vp5B+RyCpQinoQ+oy4azqsKUF+N//aD1OpwOOHqVOEZYgqcKoKPo9JAT4\nxz9o+tLLC/h//8/8uYGB1g9wEUVygv7+5KBv3qQ+iAMH6v8ejMvTI/7SWq0Ww4cPx6xZsxxtCsNY\nn9paWjdrbm6/hnXkCI07GydPUo/EigoKTtmwwfJ6pDt2kJORHH5AAOUMenqS86+osJ3dpjhzhoqX\nl5RQYYH+/UmtXrhgXzsYh9IjlOEnn3yCgQMHoqamxtGmMIz18fMDli0zHVXp4UFOwploaQFWraJ1\nuqIiSmMoKbFMHVZUUHCQTkcKDCBFWF1N1/L3B9LSOlaH1kSnozXLGzeAzz+nLx7e3qQ+WR26FU7/\nV87NzUVqaioef/xxXhtkXBd/f1ora7vZs/SZpZw8SUrWz4+6V+TnU8cLS9Shry+tD/7xj8Dvfgc8\n9RRNk95+O1WCiYwkdSilSdias2dJhcfG0rSvVAA8OJjVoZvh9M7w+eefx/LlyyHjb2cM43gkVVhR\nQet7crl+ireoiNRhR3h7A6NGAaNH06ZQkCKWOkt4etIaXlqa7Z9FUoVBQWR7UxP9BCiSVFKHpirg\nMC6HU3uY7du3Izw8HMOHD2dVyDDOwMmTpARVKnJk/v7kzGpqqN5nVZX+2F27gLy8jq+3fj2lMmRn\nU3sltZrUZUqKvvi2rZBUYVAQ2alQAMePU23W7Gx6psJCvYNkXBqnXjM8fPgwtm7ditTUVDQ2NqK6\nuhoPP/ww1qxZY3Tc4sWLW/+dnJyM5ORk+xrKMO6AVktKyteXEt0l6upIRb3xhj4oprgYWL2a1N/C\nheaved99wLRp7ccFgZSnKNJ9Paz8qjJUhYIATJxIY9nZwK9/TWXZJLy9rXtvxiakp6cjvW0T6k7Q\nY/IM9+/fjxUrVmDbtm1G45xnyDB2ormZ6nc2NbXf5+1NRbWlXMNVq6hsWksLsHhx+4a6lpKeTl0p\nFi60bhJ8Tg7wzjv0TIbX1WqBvn2BP//ZevdiHIJL5xm6VJI1w/Q0PD2B+++/9XHFxVTWLCaGokw3\nb+5YHZqjsZGmUSsqaOqyb9/OX8McvXtTaogprK1CmR6BU68ZGjJx4kRs3brV0WYwDHMrDLtERERQ\ngI1a3fnrHD5MOZiBgVSP1NozQL6+prfOVNJhXAb+CsQwjHWQAk4kVQjQFKRCYV4diiIFzLRdl2ts\nBDZupLVJHx8KdrG2OnRiXn3VuJ+xISoVsHSpfe1xB9gZMgxjHb76isqzNTQYr8OJIvDDD1REOy7O\n+JxDh+ic114zTm6XVKFUa1SpJHX44otuUUC7sND8MmtXRDZza9gZMow7U11N5dC6S04OcOwYObC5\nc6nuqFRSTnqrK5XG52g0lKhfVETJ7VJhbENVKBEW5nbqkLEvPWbNkGEYK5OXRx0aiou7f61t22g6\nNDaW+gROmQJcvkzb5MnA9OntO0ocPUqVZsLCKM1BSm4/d47yFQ37DebmUmTqgQPdt5VhTMDKkGHc\nlW3baM4tNRWYP7/r15FUYVwcTWGePUvXLiqi30+cAMaNMz5HoyH1FxZGQStqtV4dDh1KLaFM0cny\ndLz2xlgKO0OGcUfy8kiZDR5MuXwzZhhPS3YGSRVKa34+PsAnnwDDh9PY+vVUgs2w4LikCqUp1MBA\nUocDB9JxKlV3nq4VXntjLIWdIcO4I9u2UQqBpyelQXRVHebkkGPr3ZumMQFKys/JAX72M3JyarWx\nOjRUhRLBwcbqsIdjTpFaqkZVKvPO2krfE5g2sDNkGHdDUoWxsfR7ZCSlQ3RFHV67RtOcpaX0uyhS\nxRilktb8QkKoE4ShOrx4kVShXE7tmyR0OloTdAFnaE6RWqpGefrW/rAzZBh3Q1KF0rSmXE5bV9Rh\ncjJtEidOkGOUVF9BAf0sKdGrw8GDgY8+Mn09H5/O3b+HsHs3BdrW1hp/xLxu6Tx02Rnm5+fj8OHD\nSExMxNChQwEA2dnZKCgowODBg+HnjH3YmB5DfXM9LpRcQFNLE/qG9IXKj+eGrEJdHXD+PE1VZmfr\nxyVF98AD3StMrVQC8+aZ3ufvTz9lMlKM1karpcjYyEjrX7ub1NZSTXDAWDGq1d2fUmWsQ5ec4YED\nB3DPPfegoaEBAPDiiy9i+fLlUKlUOHnyJCZMmACtqa7dDGMB54vP4+8//h0NLQ2tYz/v93P8atCv\nuD5td1EqKVLTVI8+ubz7HRqSkmhzBMePA19/Dbz3nnVyJ+1Ed6dUGevQpTzDd999F6tXr0ZlZSXO\nnTuHwsJCvPrqq/D29sa4ceO4iwTTZeo0dfj0x0+h9FIiPige8UHx6B3QG6mXU3Gq8JSjzXMNFArT\nNTl7cquilhZalywuBvbubR2WAlFMbRyIwhjSJWU4fvx4zPtpKmTgwIH46quv8OWXX2LlypWYMWOG\nVXfY8CoAACAASURBVA1k3IvzJefR2NxoNC0ql8kRqAjEvhv7MCJyhAOtY5wWaa2yXz9qDDx5MhAQ\n4LBpxrbRoLW19JNXj5yXLjnDgJ+mIK5fv44+ffoAAB577DGkpKQgJSXFetYxbodGqzE57in3RK2m\n1s7W2J5mbTMAej6mi0iqMDSUAoO0WlKHv/iFw0xq64Tnz+96S0fGPnRpmnTChAl47bXX0K9fPxw9\nerR1fObMmejbty8HzzBdJiEoARAAnWi8plXeUI6RUSMdZJX1Kakrwf/9+H94evvTeHr70/jH8X+g\nrL7M0Wb1TCRVKAXoqFSkDqurHWuXAeama3mq1nnocqf7+vp6XL16FT/72c/a7TNUjLaGO927Hl9l\nfoVd13ZB4aGAVqdFfUs94oPi8cadb8Df29/R5nWbWk0t/rzvz6hpqkGkfyREUURhbSGCfYKxJHkJ\nfDxdM73AJrS0UDhmS4veGQLAzZvUJcOB6tBSzKlGtRpYtcq+trgSVu90f/36dWzZsgXz589HcHBw\n67ivr69JRwjAbo6QcU3mDZyHrJIs7Lq+C1qdFqG+oZjQewK85K7RdPV43nFUNFQgLuindkYCEB0Q\nDXWlGhkFGbgj9g7HGtiTKC4mR9jQQJuEQkFFwq2ArVMfzFWbYdVoX27pDBctWoS1a9eioKAAH3zw\nAQBykMuXL8f8+fMxduxYmxvJuBcbsjYgryYPMxJnQCbIoBN1OJF/AqG+oXhwyIOONq/bXCu/ZlL9\nKTwUUFeq2Rl2hqgo4OOPbXoLW6c+cC6hc3DLNcPo6GgcPHgQzz77bOtYnz598H//93/YtWsX9uzZ\nY1MDGfeitK4U6y+sh07UoVZTC1EUIRNkiAmIwb4b+9DY0mjxteo0ddh3Yx8+O/4Z1p1fh7zqPBta\nbjkqfxUam9s/R5O2CRHKCAdYxDDMLZ1hUFAQZDIZYmJijE+UyfDWW29hy5YtNjOOcS8uFF/AS2kv\n4XzxeZwtPos91/cgsygToijCU+4JrahFfXO9RdeqaKjAkv1LsDpzNc4Xn8fOazvx1r63kJGfYeOn\nALQ6LfJr8lFSV2JyzWJczDh4yj1R2VjZOlbeUA6FhwKjo0fb3L5u0dJCDXmbmhxtCcNYlVtOkz71\n1FO4/fbbERISgrvvvhuTJk3C+PHjoVAoAAAajelQeIbpDA3NDfj78b8jWBGMYJ9geMo84SHzwPWK\n6+jl0wshviHw9/JHgLdllUW2Xd6G0vpSxAfFt47VN9fjy1NfYnD4YHh72CbBPLMwE6tOr0JVUxVE\nUURiaCIeH/E4wpX6AtihvqF4afxL+OLkF7hZeRMQgHBlOJ4c+SSCFEE2sctqnDxJVV5CQoBJkxxt\njU2R1goPHQJOn9aP+/kBd9/tOLsY23BLZ/j4449j3LhxqKurw8qVK/GXv/wFXl5eGDp0KLy9vTlY\nhrEKWaVZaGhuQLgyHIPCBiGjIAO+nr5QyBXIKs1C35C+eGLkE/CQWZYaezjncLt6pr6eviitL4W6\nUo0BvQZY/RmyK7Px8dGPEeITgtjAWIiiiJuVN7Hi8Aq8O/ldowCgxNBELL17KQpqCiAIAlR+KsiE\nLmU62Y+WFuC77yiyY+NGYPz4nl215hZIa4WnT1NdUU9dE2LrLyKjdqijTbMaXBdVzy3fLPHx8fjw\nww9bf7906RL27t2LtLQ0XL16FZ999plNDWTcA8Nk+9jAWHjIPHCx9CIqNBUI8gjCs2OfxchIy/MM\n5TK52bBqWzmdvTf2wlPu2Zr+IQgCVP4qqCvVuFByAcNUw9rZER0QbRNbbMLJk5TPFx9P0SOHD/d8\ndVhURL0UvcxHKvv5UcepsTUHcW/5GpwP/ABqtapb0Z5dcULmzrnVeR3BdVH13NIZti24PWDAAAwY\nMAC/+93vcPHiRbz99ttY6m5fIRir0yeYZhi0Oi3kMjmiA6IRHRCNGxU3MHfgXIyKGtXh+S26FsgF\neWsh74lxE5F6JdVomrS6qRr+Xv5GY9YktzoXfl7tC06Ioojy+nITZ/QgJFUodZsID7ebOrSFEwBA\n657LlgHTpgE//7nZw+6+G/BoacS0XZvhrdThCf8UPLbqsS7elOiKEzJ3zq3OM6TtZylNAfPUrwXO\n8Le//S3+8Ic/YNmyZVAqla3j586dw/nz56EzVf2eYTpJuDIcMxJnYOulrQhUBMJL7oXyhnKo/FVI\njk82e965onNYn7Ue6ko1ghRBmJk4E1P6TMHMxJnIKsnC9Yrr8JJ7oUXXAi8PL7xw+ws2K33WL6Qf\n9tzY025dUybIoPLv4UljJ09CV1KMkjAlKkrz4ePhg8gqDbw6UocFBUBaGvDb3wLd6DZiDSfQSlER\nEBhIeYhHjpCNW7YAd91FxcrN0Dv7EDw1tagMSkD//ENA4UynTQTsSHW2/SylKeDKyvbHuxu3dIYj\nR45EcHAwXnnlFbz88suI/+mTXLNmDVasWIH5nW0GyjBmmDdwHvqF9MM+9T7UaGowOX4yJsZPNFt1\n5lzROSw/vBxBiiDEBcahoaUBazLXoKqxCvMGzcPrd76Oc8XncKX8CkJ9QjEicgSCfYJNXssaTE6Y\njHR1OkrqStDLtxe0IkWVJgQnYECo9dco7YZWi+b//ReXc06hNK8BAmQARBQ1i0j8r4DgCRNMTzNu\n2ULOcMwY4Lbb7GdvSgowbBgQ3WYKurkZ+PBD4PbbgZkzSdn27k2Nhw8cMKsOPVoakXRpM+qUERBl\ncrTIPOgej3VPHdoKnvrsGhZFI0h5hYYsWbIEY8eORbJhl2uG6QaCIGB45HAMjxxu0fEbLm5AkCKo\n1cH5evoiLigO31/7HtP7TYe/t3+nrtddIvwi8Nqdr2Ht2bW4UnYFcpkcd8TegV8O/CXkMrldbLAV\nRwcH4bhvOML9IiBpvLrmemTI6/CErgUeaOMM8/KAo0eBiAhKxXj99W6pQ4vJz6do12vXAIPcaABU\nwzQvD/j+e3Le1dU07RsRYVIdSpVhBuYfQlN5Lcr8egEA/HupaH5xpvOqQ6bzdLnTvY+PD+bOnWtN\nWxjGYnSiDjcqbiAuMM5o3EPmAYhAUV2RQ+qYxgfF4/U7X0dDcwPkMrlrlJCTy/FdZDmUcaNR7aEw\n2nWz6iam1uejn6Kf8TnbtpHDCQujsmiXLtlHHW7fTgtgGRlAdjYQ99N/H83NwLp15LxKS4HPPgMG\nD6Z9CgVNn7ZRh0uXAmhsBF7YCPRRAD5V+vsUNTm1OrQUKTiottZYObqjj++yM2QYRyJAQIhPCOqb\n66H00q9li6IIrai1OB/RVtiz2HZTSxMulV1CY0sjEoISEKYMs/o9mnXNkAum1W2LrsV4QFKFsbGk\nBv39u6YOd+6kQB1YqOzz8ynCNTaWapZu2aJXhydOAOXlNH9YWEgOeuBACgwCgF69gM2b268d1tYC\niYlA23xqlQqQd13tO0s9UilohouCszNkeiiCIODexHux8vRKxAXFwUPmAVEUkVOVg+Gq4UZJ7rei\nTlOHi6UX0aJrQb+Qfgj1DbWh5dblavlVfHz0Y9Rp6lrHZibOxNyBc1sja63BuJhxOJB9AL0De7eO\nNTQ3wEvu1T46V1KFsp9SWEJDO68Oq6ooejUoCDLdEFj0qtq+XX/f8HC9OoyKIlUYGqq/tlwOnDkD\nJCToz/fyAnJzgf799WO9egHPP2+ZzZ2gKxGw5hyotK8713BHJdgWdoZMjyU5IRmVjZVIuUINpXWi\nDsNUw/DYCMunrjILM/HZ8c+g0Wogguqgzkmag5mJM63qTKyFVqdFVmkWcqtzofRU4uszX7eulUr7\nt17eij7BfTAiaoTV7jtrwCycKT4DdaUa/l7+aGxpRIuuBU+NegoKw6nTwkKabvT0JEckUVcHbN1q\nuTPcswcQRaCsDKPE4/hRPc7kYa0vcUNVCJBD9PEhdTh6NCnFuDiaLh06lFRhWRnwwQcUXdoDsNSB\nduTwOAvOPF3uZ+gscD9DplZTi6LaIgR4B3RqirCysRIv73oZQYqg1qnWFl0LblbdxGt3vIaksCRb\nmdwl6jR1+OuRv+JqxVUIEFBaX4prFdcwve90BCr0L/Sy+jLEBsbi5QkvW/X+NU01OJJzBBdKL6CX\nby/cFXcXYgNjjQ+qrwfOnSNHVltLuXySIvPzAwYNuvWNqqqAF18kdSfVQF22DPDo4Lv7F1+QMjRo\nMwdRpLZO48cDV660P8fDA3jiCf3aYSfh6i3OjdX7GTKMs+Pn5Qe/kPbJ7rfibNFZNOuajdYcPWQe\n8PX0xcGbB53OGW69tBXXKq4hLjAOgiBAEARcKb+CE/knMDlhcquS9fbwRrXG+l3e/b39Ma3fNEzr\nNw1YvRrwqgfaiipfX0qlAIB//IPWD99+Wz9lagl79gA6HU1benmRzDl+HBhnWh0CoHv27Wt637Bh\n+mIBVsTRKQw2K0bgprAzZNyW+uZ6CGg/Feol90J1k/WdSXcQRRHp6nRE+Ue1Or0gRRAUHgrUaGpQ\n3VTdqg7L6sswa8As2xmTnQ3s2AFcvw4sWmTa0eXmAseOkVM7e5amJi2hqgpITTVexAoNBdavp+lO\nc+rQ0uu7EFYtRsA4vzPMycnBww8/jOLiYgiCgCeffNKotyLDdJW+IX2hgw6iKBqtD1Y3VWNEpPXW\n26yBCBEancYoXzHAOwDxQfE4V3wOlY2V8JR7oqSuBCE+IZicMNl2xmzZgqNZAWj5UY3vT19AXrB+\nmrFVkUhBND4+FAgzZMit1aEoAn/9K1WFaVvZqqoKOHWKHCLD2ACnd4aenp7461//imHDhqG2thYj\nR47E1KlTkZTkXFNYTM+jb3Bf3B59Ow7nHEaobyg8ZB4orS9FTEAMbo+53dHmGSETZBgZORJni84i\n0j+ydbxfSD/IBBniguLQ0NyAGYkzcHefu23XCio7G8jIQKEYh8helZhauQ77hw2E+FPxc7UaelUo\npVao1ZapQ7WaUiDGjgWmT2+/Py6u/RjDWAmnd4YqlQqqn6ZM/Pz8kJSUhPz8fHaGTLcRBAFPjHwC\nQyKGYJ96H5pamjBv4DwkxyfD19N8nUpHMW/gPFwqu4SblTcRoAhAvaYezbpmvH7n67csZG41/j97\n5x0eV3nm7ftMb9KojHqXLMtyb9gGY8cGG4zpCRBISJYAoS0kLBs2ZfPlg3wJJMACSSDZZTeE3YQQ\nOgaMKQbM4t5t3C2s3ttIml7O+f6YaKzRjGRJVhnb731dXNhHc855z4DmN8/zPs/vWbMmFO1JKtyG\nZJLsVaS1HqIlvU8RSv/WiqSkU0eHihLq8+sthSwtHZN9PoFgIOJeDPtSVVXFnj17WLhw4UQvRXCW\noFFpuDD/Qi7Mv3Cil3JKMi2ZPLzsYT6v/pwjbUfItIRMzHvbKsacv0eF4QhNkvAarEw9+CqtaaHo\nMNlZB5s2QVZWqJITQg4vlZWDR4dVVSHX6MLCUGT5wQdw003j8VQjZig9e6Li9MzhjBFDh8PBdddd\nx29+8xssluFXDgrGBlmR438obT/8QT+V9kpkRaYwqTCyTy7OSTGmcPWUq7maq8f/5h9+GGqdqKsj\nxQGWv2+zmpwtpLYfo802hUR3ExTnhCK9vmXtubkhkYslhr1RodEYSqtmZYUMvi+9dEyiw9GqwhzK\n62IVuaxfH7I27b8GIZATyxkhhn6/n6997WvcfPPNXHPNNVE/f+ihh8J/XrZsmTAPHwd2N+7mzcNv\nUttdS7o5navKrmJx3uK4bFTvy9G2ozy741kcPgcQqhy9bc5tnJcjCjNOyaWXhtsmPmoLGbv00p2Y\nC0C1bT78epgp275RIYQqRiVpzKLDia7CdDhCLZf91zDce4+GI83ZxIYNG9iwYcOIz497MVQUhdtu\nu42pU6dy//33x3xNXzEUjD3b6rbxzPZnsJlsFFgLcPgc/MfO/8Dpc3LppBiFD3FCl6eLp7Y+hUlr\nCjeLu/1u/rDzD2QlZJH79w90wQDk54cdXvxTYVvfyKYh9K8RfQivWQM9PaGexF6CwVCLxRhFh2cD\npxtFnm0p3P6B0MMPPzys8+NeDDdt2sRf/vIXZs6cyZw5IcPeRx99lFWDTKYWjB2yIvPqoVdJN6eH\np0Ik6BPQqXW8eeRNlhUuQ68Z28nnI2VX4y68AS+ZlpOf2EatEbWkZmP1Rm6cceO4rMPhc3Co9RC+\noI9JKZMi1nOmMKoflrNnxw7V1OrTMsMWDM5EmwbEG3EvhhdeeCFy/54jwYTh9DnpcHdE2XDpNXr8\nQT/t7nayE7IHOHtisXvsMecK6jV62txt47KG/U37eXbHs2EvVIDVpau5fur1cZ9iHjPEtoYgDoh7\nMRTEFwaNAb1ajy/oi5jVF5SDKCgk6MZ/huBQKU4uxi/7o5rsHT4HU9Omjvn9e7w9PLvjWRL1iWEL\nuKAc5N2j71KWWsaszHPPReVsJ9a+nsNxbu7pxTtCDAXDQqvWcsmkS3jz8JsUWAtQq9TIikxtdy1L\n8pdMyEDdoTI9fTqlyaUc7zhOVkIWKklFk6OJdHM6C3PGvl3nQMsBvEFvhBeqWqUmQZ/AhqoNQgzH\nifEsPImVTu7dq+u/BiGQE4sQQ8GwuXLylfR4e9hQtQEJCRmZRbmL+ObMb57y3E53J8fajyFJElNs\nU045hDcoB9nfvJ8tdVuA0Fy9mRkzY6Y7T4VGpeGBCx5g3fF1fFb9GQE5wEVFF3F56eURAjUcTnSe\nYM3RNRxrP0aaKY3VpatZmLMwZsrTE/DEvIZWrcXpd8b8mWD0mejikIm+vyA2QgwFw0aj0vDtWd/m\nqrKraHO1kWRIwmaynfK8j098zItfvIisyOHrfGfOd1ictzjm62VF5r92/xcbazeG069b6rZwYd6F\nfHfed0fU32jSmvja1K/xtalfG/a5/anoqOCRzx/BoDGQYkgJpUG3P0v79HYun3x51OsnpUwConsz\n7R47qyetPu31CATDQQz6jUSIoWDEJBmShuyBWW2v5s/7/0x2QnZ4r9ET8PDH3X+kJLkkoqIyKAfZ\nUb+DVw+9yieVnzAjfQYpxhQ0Kg1ppjQ21W7iwvwLmZY+hNl4Y8irh17FpDWFvwhY1VZMWhNvHXmL\nZYXLoqLN3MRclhcuZ33lepINyWhUGjrcHeQm5rI4P/YXAsHIOdtaB0Yb8R5EIsRQMC5srd+KRqWJ\nKLoxaAygwM76nVxRdgXQJxqs2Uizs5mAHGBf8z7qeuq4IO8CNCoNBo2BPU17JlQMZUXmaNtRCqyR\nVmhatRZf0Meao2sIyAEyzZksyF1AkiEJSZL41qxvUW4r57Pqz3AFXKwsXsmSgiWYdWa6vd009DRg\n1prJTcw9d6tLRwnROiAYDkIMBeNCj6cHrUobdVytUofdYCCUetxcu5ni5GI8AQ+NPY2oJBV13XXU\ndtVSlFyErMho1dHXGk8kJBL1iXgCHoxaY/i4w+tgV+Mu3AE3KcYUvEEvbx59k3+54F8oSi5CJalY\nkLuABbkLwufIiswbh99g7bG14b+XpJRwz3n3kGIUDecCwXggxFAwLszMmMnnNZ9HtDUoioIv6GNq\n+sm2hqNtR1Gr1Cgo2D126rrrUFDwBDw09DQwO2M2mQmZnJc9sfZpkiSxatIqXvripbDIKYrC5trN\nGDVGym3l4efscHfwn7v/k19e9MuY0d7m2s2h6tykAjQqDYqiUNNVw7Pbn+WnS386ZhGimJR+5iFS\nv2OHEEPBuDA7azZltjKOtB3BZrKhKArt7nZmZ85mWtrJdKdBY0BRFGq7aml2NmPUGOnwdIQn0h9s\nO4hBa4gZZY43l5RcQquzlQ3VoaragBzAHXCzonhFhIAlG5Kp6a6hydEUMYuwl3XH15FmTkOjCv06\nSpJEliWLE50nqOmqGbOpFBPt0SkYPiL1O3YIMRSMCzq1jgfOf4DPqz9nU+0mJEkKmXvnL45ok5id\nOZuXDrzE0bajaFVaZGRSjak4/U7SzeloVVpSDCmsq1jHHfPumMAnClXD/sPsf2B16WqaHE3oNXoe\n+fyRiLQphMQNZYCLAG3uNtJMaVHnqCQVPb6esVi6QCDohxBDwbhh0BhYWbKSlSUrB3xNmjmN2+fe\nznff+S4evyfsdJObmEuqMZVubzdmnZlDrYcGvVdtVy1rj6/laPtR0k3prJq0itmZs8ck5ZhmTiPN\nHBKzuVlzOdB8gOzEk5Z0ne5OMiwZZFgyYp5fbivnaNvRiJ/3OvrEq7XdmYBoHRAMByGGgrjjgrwL\nuG/Bfbx88GWkLol0czo6tQ5ZkVFQ0Kl1g/Y1VnZW8sjnj6CSVCQbk2l0NPLU1qf45oxvjvlUjRun\n38iv7L+iyl6FUWPEG/SiU+v43sLvDdgXeXXZ1fyi+Re0OFuwmWy4/W6aHE2sLl0tCmhOg5HsoYl9\n1HMXIYaCuORr5V/jQMsBOlwduHwuAuoAnqCH0pRSXH4Xq0oGnlry+uHX0aq1pJvTgVBEatFZeP3w\n6ywpWIJJaxqzdaeb0/n58p+zo34HlfZKsixZLMxdOKioFSUX8a9L/5U3Dr/B4dbDJBuTuWX2LSwr\nXDZm6zwXGEmxidhHPXcRYiiIS9LMaTy07CFeP/Q6/7P/f/D4PRRaC0kyJPHV8q8yL3tezPNkReZg\ny0HyrHkRx3VqHUE5SENPQ9gJZqyw6CwsL1rOcpYP+Zzi5GJ+cMEPxnBV0YzEo7O/yXk8Ew/FJqNd\n/SlSv2OHEENB3GIz2bhz/p3cMe8Oartrcfld5CbmYtFZBjxHQsKis+AL+kJN/X9HURRkRcaoOVnc\n4va7OdF5AkmSKEkuGbU5jA09Dfxv1f/S4GigNKWUC/MvJNmYPKxruPwuDrcexhv0UpxcPCYzD4f6\nYewJeFh3fB0fV36MO+BmftZ8ri25nMxjDbBwYWgqvSAmoy3IIk07dggxFMQ9kiRFzU8c7LWXlFzC\nK4deoTipOBzFNDmaKEkpCRekbK3byp/2/Am/7AclNOT37vl3Mz1j+mmt9UDzAZ7a9hQSEiatiQMt\nB/jwyw/58ZIfD7kY5lDLIX63/Xe4A+7wsVWTVvH1aV8f96hMURSe3f4s+5r3kZ2QTbIhmT1Ne3B/\n+hH/uEuF/tHHoLh4XNc0GgwUse3aNXCadDwR/YTjjxBDQVxQ113HJ5WfUNddR0lKCcsKlg1YfXkq\nVk1aRUNPA1vrtiIhoaCQa83lrvl3IUkSdd11PLfrOdJMaeE2CIfPwW+2/4Zfr/j1iItWAnKAP+75\nI0n6pPAoqxRjCo09jbx68FW+v+j7p7yG0+fkd9t/h0lrCj9/UA7y6sFXaXG0kJ2QzbT0aZTZykZk\nVD5cKjoq2N+yn6KkorAQ5xkymPH5JpokGwVvvQX/9E/jFh2OpMBl/frQDMFeev9ssYT+WbHi5M82\nbhy9tZ4O8ZDiPdcQYiiYcA40H+DJrU+iltRYdBZOdJzgkxOf8OMlP6YwqXDY19Oqtdw5/06uLLuS\nxp5GEvWJlKSUhMVjU+0mVJIqoh/QorPQ7mpnV8OuQVs/BqOxp5Eub1dUFJthyWBv096ogcixONR6\nCHfAHfFFoLarlkOth6jprmFWxizeOfYOC3MXcse8O8KN+mNFQ08DElJERJp9uA6rB6qyJQr27YPK\nynGLDkdS4OJwQFIMP/mkJLDbI48ZjeM361AQXwgxFEwoQTnIn/b+iSRDUni2odVgpdXZyov7X+Qn\nS34y4tRgdkJ2zNRkh6sDvTp6f1Cj0tDp6RzRvSDks6ooSlSRiazIqFXqIUVy3qA3okHf7Xezr3kf\nCboETFoTuYm5KIrClrotzMmcw/l550ecrygKuxt38/axt2nobqAgqYBrp1w7YlPzBH1C2P0HQO0P\nMvWzw9gTNCTrzaA2wThHh0Olt9ikb1QIoWiw/7Fe5s2DF14Y65WdJN7TtecSQgwFE0qrq5VOT2dU\nNGUz2ajoqMDld0WNQur2drOzYScNPQ3kJeYxP3v+sIbzTk2byrb6baRx0vVFURR8so/JqZNPeb7T\n56S+px6jxhgxXSLLkkVuYi5trrZwEz6EIsbF+YuHFMUVJRWBdHLmYaurFRkZn+xjkiVUBStJEsmG\nZDbWbIwSw8+qPuP5Pc+TYkoh05JJi7OFX2/6Nfcvup+5WXOH9P70ZVraNKwGK22uNmwmG9mH69A6\nnDiTVMxLKgZjCoxzdDhUelOmt9wSLSxvvTV69xlsf+9U1Z8DRbrxkq49lxBiKJhQtCotCrGjqaAc\nZHPtZjo9nRRYC5iVOYtWZyuPbXoMh8+BXqPHG/Cy5ugafrj4h1gNVo63HyeoBClJLgnv2/VnQc4C\nPvjyA6rt1WRYMlAUhSZHE5NTJ0f4pPZHURTer3ifNw6/gazIyIpMpiWTb8z4BuVp5WhUGu6cfyeP\nb3qcKnsVaklNUA6Sl5THdVOvG9L7kZOYw8VFF/PhiQ9JNiTj9Dlx+pxkWDIiPEp7vVD74gv6ePXQ\nq2QlZIVTwL1zIP924G/Mzpw97H1GvUbPDy74Ac9sf4a61hMs/nAnHjnIAuN0Uj0SeDrB74/b6HA8\nGGx/bzyjTMHpIcRQMGIURaG2u5YuTxdZCVlDmnbfnxRjCmWpZVR2VkaYWB9rP0aLs4UXv3gRnVqH\nL+gjwxzaR5MVOUIYGnsa+bfN/4bD78AT8ACgklTcPONmlhUti7qnUWvkRxf+iPcr3mdTzSbUKjVf\nLf8qK0tWDjoaak/jHl764iVyrbnIiszuht1srtvMe8ffY0nBEm6eeTOLchfx6IpH2de0jzZXG3nW\nPKalTRvWyKlvzvwmZallfFL5CWpJTZurjTmZc8KtH4qi0Onp5KvlX404r93VjifgiYhKARL1idR0\n1eD0OQf8gjAYuYm5PHLxI9TWHkRf/QYpkgld3+cpLgarddjXnUgsltB+ocMRGbkdORKKJGMxHpWc\nvcU+zc2R0Wv/Qh/B6CPEUDAiujxdPLvjWY61H0MtqZEVma8UfoWbZ948rKIOSZK4dc6tPLH52y7V\nxwAAIABJREFUCarsVUDow77R0ciU1CkRHp8VHRWc6DgRVeCSqE9kzdE1rJq0inRryHXGG/Dywr4X\nyE/Kpzg5On2XqE/khmk3cMO0G4a81nVfriPJmIRWpeWz6s/o9naTac6ky9uFJ+DhDzv+QJIhiSm2\nKVHpy+HQf+bh2mNreeXgK3R5u9CqtTh8DqalTWNR7qKI88w6MwoKQTkYYX7uD/rRqrSn1UepklQU\n5M+AH8wY8TVGg5EYBcSiV1j6R299U6qxqlCbmsZWFHuLfYx/r+1qaIBAIBR8NzWFjt9yi2ixGAuE\nGApGxHO7nuNExwkKrAVIkoSsyHxc+TFppjQun3z5sK6Vbk7nFxf9goMtB2l3t6NRaXhh7wtR447S\nTelsr9tOUA6iUZ/8X7fZ0UxQCUbsG+o1enQqHRtrNsYUw5HQ5mrDpDXR4e7A7rGTZDhZoqiSVFh0\nFtYdX8cU25QBr6EoCjvqd/DBiQ+wu+1MT5/OZaWXDdpUv7p0NWW2MrbUbsEVcDE3cy6zM2dHRZuJ\n+kQW5S5iS+0W8q354f8udd11rJ68+pSVrGcCo+na0vuzgYhVhVpYOD7tDdnZcM01oeiwt+r1mmtO\n/ly0WIw+QgwFw6bF2cKh1kPhD1wIiUFOQg7vV7zP6tLVw64A1al1zMmaA4TK+VWSKuoaZp0Zk85E\nu7s9ovWg2dlMiiEl6sNer9HT6Y6uDq3rruOL5i8AmJ4+PaIIppegHAwX8ORZ87CZbJTbytnZsDO0\nv8nJAcUACboENCoN9T31gz7nm0fe5M3Db5JiTMGoNbKpdhPbG7bzs6U/iznrEELR86SUSWEbOYfP\nQaW9EpPWRE5CTsTab555My6/i71Ne8MR++L8xVw75dpB13U2E88RVH+h7tsDKRhfhBgKho3D54gp\nVnq1niZfE0EliEYa+f9amZZMUk2pUdFXi7OFq8quotXVSo29BhkZh89BsjGZRH1iVBFOj7eHmRkz\nw39XFIW3j77Nm0feDIvZywdf5pop13B12dXhcxt6Gnh669O0OlvD0dWqSatYNWkVOxt24g64kRUZ\nX8CHw++gwFpAgj6BJkdTxP36c6LjBP+z73/IScghyZCEJEnkJubS0N3Au8fe5bvzvjvo+6IoCmuP\nr+WtI2+F7eVKUkq4e/7dpJpSATBpTdy/6H4aexrp9HSSZkqL2kOMJ851p5X+zxir8lUwPggxFAyb\nTEsmKkkV1UTe4e5gUsqk024EV0kqvjv3uzyx+Ql6vD0YtUacfieJukTuOe8evEEvv/jsFxxtPYpF\nbyHdlI5Za6bSXkm6OT3UkuBsJTsxm4W5C8PXrbJX8eaRN8lNzA2vMSAHeOvIW8xIn0FJSglBOcjT\nW5/G6XOGi3SCcpC1x9aSb83nX5f8K68fep2j7Ufp9HQyM2MmpSmldLo78QV9rC5dHfU8siLzysFX\n+NuBv3Gw9SDV9moS9YksyFmAWWfGZraxr3nfKd+X7fXbefnAy+Rb89GqtaECpq5afrf9d/zfr/zf\niC8CWQlZA0aa8cTZ4LQizLPPDoQYCoaNSWvi2vJreemLl7CZbJh1Zjrdnbj8Lu6deu+o3GNy6mR+\nedEv2VS7iYaeBoqTizk/93wS9Yk8tukxFBRWFK9AkiT8QT8nOk8wP3s+ra5W/LKfK8uuZEXxiohx\nTTsbdqKRNBFirVFp0Kq07KjfQUlKCRUdFbQ6WyOqVdUqNTaTjQ8qPuDh5Q/zwAUPcNf8u3jjyBt8\nVvUZ9Y568hPzuXv+3WHHHEVRqOmqYV/zPr5o/oIdDTvIMmdRZa/CarDS4+the/12lhUuwxvwYtWf\nuhrzvYr3SDWlhvcKJUki05JJtb2aSnvlqO2Nnsv0FbZYzfqxGM0Idihp0/XrQ18iYlW9nisR9Vgg\nxFAwIi6bdBmpxlTWHl9Li7OFSSmTuGbKNaM6HinNnMY1U66JOFbfXc+RtiMR+5VatZashCza3e38\nasXAnwR+2Y9KFd1np5JUIcNuQtMiYu13GjQGurxd4b+bdCZunnkzN0y7AX/Qj0lrCp+nKApvHnmT\nt4++jQoVOxt3hio8JTVmrRmnz4lFa6HL2xUqxvHauW3Obad8P9pd7WGXnl4kSUJCosfbc8rzBaem\nr5DESuFWVY1txDeUtKnDERLHMz2ijjeEGApGhCRJLMxdGJGGHA+6vF2xi2u0ZpocAzg4/53ZmbN5\nv+L9iL1FRVFwB9zh4p08a17M9oQ2V1vMdgmdWhdVuFNpr+Tto2+H07H7W/Zj0Buo6KxgXuY8jrQf\nocvbhdPnpKarhuumXseS/CWnfPYyWxkHWw5GVJ7KioyCMuSJGIKhEw8RVqwUrMMhUrBjgRBDwRlF\npiUzplh1ejoHjErdfjc7GnZwqPUQRo2RQ62HwkUlPb4eLsi9gHJbORCygbu05FLWHluLzWTDoDHQ\n5mpDo9bE3A+Mxa6GXRHp2DRzGq3OVtSSmi5vFyuKV9DiaKHB0cBjKx+jzFY2pOteNfkq9jXto8XZ\ngs1kwxPw0NjTyMXFF8d1kcxocS4W2wwUqTocJ5vyB2rIPxffr9NBiKHgjCLFmMLywuV8+OWHZCdk\nY9AY6PR04vA5uLY8un2g093JTz7+CY2ORtJMaQTkAH7Zj1VvJc+ax6LcRczKmBUhrDdMu4G8xDw+\n/PJDurxdnJ93PqtLV5NpycTld+H2u0kyJEWc05egHIyIXKekTqHF0YIn4MEv+3H4HDj9Tr4181tD\nFkKAgqSCUAHP4dc52HIQq8HKzTNv5uLii4fxDsYXwyk+ibdim5GMkzodep9/797I/sf+kzf6v74/\nIpUaGyGGgjOOm2bcRKoplbXH1nK47TAmrYlLiy8l3Zwe8bomRxN3vXMXB1sPYtFZqOmqodxWzqTk\nSXR7u7l97u0xG9FVkorF+YtZnL84fMzld/H8nufZXLsZWZFJMiRx4/QbWZCzIOr82Vmzea/ivbDZ\nttVgZUn+ErbUbcGkNWHUGLlj3h0R1x8qRclF/OCCH0S1kZypxFWEEgyCSjVkf9WRjJMajFOJq2Bs\nEWIoOOPQqDQsL1zO3sa9dPu60av1/G/t/7K1fivfX/h9pqZPJSAHeHLLk1R0VpBpyUStCplmH2g5\nQII+AV/QR2Vn5ZAiM0VReG7Xc+xt2kteYh5qlRqHz8Ez25/hh4t/GDUeaXLqZJYVLmND1YZQYQ0S\nTp+T2+fezm1zbxuVobxngxDGHf/935CeDldcMSG3H6q49vqq9tLXX7Wvt+rGjaEosu95wt90YIQY\nCs5IPvzyQ462H6U4qTgsDD3eHn6/8/c8eemT4RYJk8aE8vcBgWqVGr1Gz5cdX5JnzRuyoDT0NLCv\naV/Yeg5Cw4C9AS9vH3s7SgxVkopbZt/Cednnsa1+G4qisDB3IdPTp4/LdHrB4MSKwKyuRq7b/b/o\nLHoWLlsW1xYw/QWtr79q3+rToaZTBSHiXgzff/997r//foLBILfffjs//OEPJ3pJgjhgQ9UGMi2Z\nEYKWoE+gs6uTLzu+xOFzgAT5SflUdFSEnWy0Ki2dnk7K08pDswOHQJurLWYFa6I+kbquupjnqCQV\nMzJmMCNjYo2tR5MOdwc763eGi5VmZMyIS7/TUxWOxIrA5u5eiylRi7PbB59+CldeOe7rG8lA3/49\nh32jwYaGaG9VwcDEtRgGg0Huvfde1q9fT05ODueddx5XXXUV5eXlE700wQQTVIIoikJADkQ53siK\nTHZCNoqiUJpSSquzFbvbjlqlpsfXQ1ZCFnfNu2vIY5VsJluohaHfPl2rqzVU3XqW7N8NxqGWQzy9\n7WkCwQAatYZ1FesoSgrtXw5nsPJ4MNzCEUtPI3k1m+hOzKXHH4B334Xly8csOly7FtQxaq9qa4d/\nrf49h32jwYaG2OlUsf8Ym7gWw+3btzNp0iQK//5f+sYbb2TNmjVCDM9x7B47Lr+L9yvex6Q1kWZK\nC0cpWpWW4uRiDBoDF+RdwOfVnzM/ez6d7k6qu0LDfJ9Y+QRT06cO+X7ZCdnMypzF3qa95Cbm4gl4\n2Fa3jcaeRmZkzuDhzx7m0pJL2dW4iy21W7B77GQnZHNJySUsL1o+ojmP8YQ/6Offd/07CbqEiHmI\nlfZK3jv+HtdPu35c1tG/8nTXLnC7T4416mXjRqioGPr+2OTja5FVGhSVmoBaDb6xjQ7dbsjNjT4e\nDA5+3nB7DnsnX/Qihg0PTlyLYX19PXl5eeG/5+bmsm3btglckWCi8Qf9PLH5CbwBLzaTDafPSaOj\nkfqeemZlzuJ7C74XnvL+ndnfId+azwcVH6BWqbm2/FqunXItOYk54esF5ACegAeT1jTgfp4kSdw5\n707+duBvrD2+lo01G/EH/eQm5mLVWanqrOK2t29jcupkquxVBJUgX3Z+yeHWw3xS+Qk/XvJj8q35\nY/q+tLna2F6/nRZnC5NTJzM3ay4GjWFUrl1pr8Thc0Q9Q5Yli8+qPxs3MRyqqfXevdFWagPRGxXu\nbc/F2xDSwXeDmZg3vcvf/rac5DzLgBWvozVbsRe1evDrxVqHMPYePeJaDM/21JNg+BxoOUBddx0l\nKSXkWfOo766n1dWKy+fiosKLIlxitGpteNpEL0E5yCeVn/De8ff4ouULXD4XWZYsCpIKuH7a9RGt\nEgE5QEVHBb6gj8KkQpYXLeeVg69g0VrISskiIAfY3bQbnVpHQA5wtO0oVoMVg8aAoih0ebtw+V38\nZf9f+MmSn4zZe3Ko5RBPbXuKYDCITqNjQ+UGshOz+eHiH2I1nP4EelmRYx7vnegRz/Qd0OtwhMSj\nb+SYV7cZTdCL1dWITgc+CbK1oMHNIu0uPmv6yoDXHk5bSN99wubmSLHW6aC4GGy2yMit7zl99wUH\n62HsW2nat8q09zzBwMS1GObk5FDbJ5FeW1tLboz8wkMPPRT+87Jly1i2bNk4rE4wEdR116GWQhsu\nOrWOouQiipKL6HR30uM7tT/ni/tf5KMTH9HmaqOhuwG1So2r00WSMYlntj/DA4seYHbWbKrsVfxm\n62+we+wh/09JQiNpCMpBEg2JqFVq1Co1WrWWI21HsBltdPu6w3MWez1DtWotx9uP4/Q5h723drTt\nKG8dfYvKzkqyLFlcOflK5mTNifiSGJAD/Meu/yBRlxiRwqy2V7Pm6Bq+Pevbw7pnLIqSijBoDLj8\nrgjj86aeprhv+O8/oLewMCSETU0hoWjQrGJL+fls7wGzGUwmWLo09Fq3MQVi10cNm777mFotGPoE\n7R7Pqc/py2A9jH1Tw+daWnTDhg1s2LBhxOfHtRjOnz+f48ePU1VVRXZ2Ni+//DIvvfRS1Ov6iqHg\n7CbNlEZQid5ccfqdEenPWLQ4W/i06lMyLZl80fIFycZkVJKKbm83jY5GSpJLeOPIG0xJm8KTW55E\nQgpPr/AH/bxy6BXKU8sjoiGVpAoJqt+FTq2LKqbRqrTIijygW81AfNH8Bf+25d+w6CzYTDY6PZ08\ntfUpbp1zK8uLlodfV22vpsfXE5XCzE7IZmPNRr4181unnWHRa/TcNuc2nt3xLCpU6DV6nH4nmZZM\nLp98+WldeyywWE6KXd8IrLceZsWKvkJhBswR6cbRtDzvje76Vnm6XOD1QsLfv7v4fKFozmgc/vVH\nO1V7JtM/EHr44YeHdX5ci6FGo+GZZ57h0ksvJRgMctttt4nimXOcWZmzSDYk0+JsIc2UhiRJdHu7\nAViav3TQc+u665AkCU/Ag4QU3iM0aU20OluZmzmX6q5qDrQcoNvbHR7HBKGUq0Vnwe6xY9QY6fH2\nYNFZUBQFvUqPJElkWbJw+p1YdBYcPgcmrQlvwMt5OecNa/9OURReOvASyYbkcJozyZCEhMRvt/0W\nh89Bma1sVCeEnIp52fP4xUW/YHPNZtrd7ZTZyliQsyAiUowX+ordaOypnY7HZywLtfT0yMIXhwNm\nzx6eeI23Fdy5QFyLIcBll13GZZddNtHLEMQJRq2RBxc/yH/t/i+q7FVAyK/0gUUPnHKYrUVnQUHB\nqDWioISjuIAcwKgx4vA5yDBn4Pa7Y55fmlzKkfYjXFx0MQdbD9LsaMYT8FCUXMS3Zn2L9SfWs795\nP3aPHbPOTElyCZkJmdw4/cZhPaM74Kahp4EC68mZiq3OVrbVb6PL08Wf9/0Zo9bI3Ky53D73diw6\nCz3enog0aWNPI8sKl43qvnt2QjbXTbtu1K53uoxXVDTaHp/FxaFIsLfScyTpzNG2ghOcAWIoEEAo\nWqroqKCiowKT1sR9C+7DL/sJyAEyLZlDcnYpSS4h05yJ3WMnJyGH+p56LDoLLr+LspQyWpwt3DX/\nLnITc8P37CsmZp2ZFcUr6HR3kpeYR7o5nbzEPP75/H/GZrZx/dTr+bLzS6o6q1Cr1OQm5jI9ffqQ\n+xl70al1GDQG/LI/XJyzvX47GkmDRWehOKUYvVrPzsadTK2dyp3z7uTprU/T6elEr9bjDrjJSsji\nqilXDe9NhlD+7tNP4ZJLQj6dccxYDtXte3ygCExwdiHEUBD3+IN+ntv1HNsbtqNChYKCVqXlvgX3\nMTNz5pCvo1apuX/R/Tyz/RkcPgdd3i5anC3kJ+Zj1Bq5tvxaLsi7AICFOQvZXLuZDEsGWpWWVlcr\nVoOVny79KRJSWEhVqMKWa9MzpjMrYxazM2cjKzKbajbx889+jt1jZ2bGTK6YfMUpo1cIea+uKFrB\n28fepjCpkA53B76gD5WkIishK5xyzTBn8EnlJzy64lEeufgRttVto9XVSmlqKfOy5oVbTIbF1q3w\nxz9CTg7MOPPdc4Y6FWMgYf3Rj0L7fe+9B4HAyeMeT6gVYtcumDcv+tr9rzeYn+hAUawQ6PFFiKFg\nQNx+N4daD+EJeChIKiAnIWdC2l021W5ia91WipNP+pA6fU6e3fEsT616alj7VhmWDB5e/jDV9mpc\nfheJ+lBlaKoxFb1GH37d7XNvZ1LKJD6u/Bin38n5ueczxTaFhp4GipOLmZY2jXePvcvrh18PRaUK\nvHb4NVaXrub6qdfzysFXWHNkDRqVBrVKzcaajexu3M3PvvKzIQniVVOuosPTwZbaLXR6OnH5XRQl\nFzE7c3b4NWpJjS/oA0IzE68oO02Daa8XXn8drFZ49VWYNi3uosPh7t+dbvTY1BQSMocDEhNPHne7\nQ9WnanV0ujKWgK1YEdnmMRQGW3tfkwHB6CDEUBCT4+3HeXrr0zj9TlAACZYVLONbs7417MrI02VD\n1QZsJltUyrLN1caRtiPMzZo7rOupJBVFyYP7kmrVWlaWrGRlyUoONB/gDzv/wMaajaEWC5WGy0ou\n462jb5FnzQvbwQXlIGuPrSU/MZ9XDr5Ctb06XPkqIZFuTufdY+/y3XnfPeUadWodd8y7g6vLrqbS\nXsnTW5+mKKkowgu01dXKZZNGcT9961bo7oaCgtAn+sGDI4oOR2OorDfgxRPwkKBPiEiBn2kz+vpG\nd73C2nu89znide3nGkIMBVF4A15+s+036DX68AR1WZH5uPJjSlNLRzSH73TwBX0DCnBADsQ8Plp0\nuDv47fbfkqhPDL8Xbr+b3+34HTaTLcIXtXcqxtrjaznUeohkQ3I42pQVmfruejZUbRiSGPaSYckg\nw5LB7XNv58/7/4xBbcCgMdDl7SLdnM4lJZeMzoP2RoVpaaF5fqcRHZ6OYHkDXt44/AafVH1CUA6S\nYkzhxuk3Mj97/rDWMFwGM8+2WkPtD30JBkPN8qeir/gPVtna//69VnMQarnom4rNzBQtFWOBEENB\nFEfajuD0ObElnfTUVEkqUk2pfFL5ybiL4aLcRbx++HUsupPGyf6gH0mSxry9YHfjbvxBP3q1nsae\nRhQUUowpSEi0OlshLfL1KklFm6uNgByISLv29iO2u9pHtI4VxSsosBawoXoDdredyzMu54L8CyLe\nk9OiNyrs/bROTj6t6HCk/Gnvn9hSu4XcxFy0ai093h5+u+23MedGjiYDCfjGjfC1r0U37+/ff7Iq\ndCzuv3fvSf9Suz3yZ+daM/14IcRQEIU36I15XKvShtKm48xFRRexvX47lZ2VWA1WfEEfbr+br0//\nOinGlDG9t91tx+6xs795f0TKM9mYjMvvCk+zh1D1qcvvYmn+Uj6v+TzCdUZWZHxBX7hSdSSUppZS\nmlp6+g/VH58PXnsNZDkyPPH5QtHh9OlDnv5+OjQ7mtlat5XCpMJwSjxBn4Bf9vPmkTfHVAxHi/4j\nlfoykhFNgvFDiKEgiuLkYpCIGo/U7m5n9aTV474ei87CT5b8hK11W9nbtJcEfQJL85cyOXXymN87\n3ZLOgZYD2Ey2cKQXlINU26tZUbyCKnsVZq0ZSZLo8fVwfu75LC9azocnPqSuu44uT9fJa5nTR9bu\nMNYEg7BkCfj90T8zGkFRRiSG69fDsWMnqzD9/lCk1Zv267+H2OZqQy2po4q0rHorNV01w77/aGA0\nhiKxYBDq+lizqdWhv/f+vJempsj9wL5s3DjGixWcFkIMBVHYTDauKL2Ct468RZIhCZ1aR4e7gxRT\nCitKhjgXZ5QxaU1cVHQRFxVdNK739fg9JOoTcQVc4QG/Dp+DBH0CszNnszh/MVvrtiLLMgtzFzIz\nYyYqScWK4hV8VvUZWpUWRVLwBXzkWnNZVrhsXNc/JIxGuOGGUb+swxHabuytwvR4Qqm/3rRf/z2v\nFGNKeE5lX0Hs9naTkzC41d5YMW/e8FKSg+0L9hfOXkSrRHwgxFAQk6+Wf5Xi5GI+rvyYHl8PF+Zf\nyEVFF43KFIQzCXfAzfT06fiCPqq6qpBlmdKUUlKMKfhlPzMzZjIzYya+oI9qezVV9ioKkgq4dc6t\nlKWW8WnVp7j8LhbmLOTi4otJ1Cee+qZnOL3FHQ5HZOHJqQpOshKymJM5Jzw3Uq1S4/a76XB3cMvs\nWyKuHeuegzEaFa6ny2DCKlolJh4hhoKYSJLEnKw5zMmaM9FLmVB6U7GTUydTZisLH6+0VzIzI9Tw\nv7txN3/c/UfcgVD5X5Ihibvn381XCr/C0oKlHG0/yp7GPXxQ8QFzs+YyKWXSmPRryorM0bajNDub\nSTIkMTVtakQrxnjRKy633BLpyTkUvjvvu/z1i7+ypXYLEMoI3DH/jnB/5UiF61QVriMV2dGi//37\npmX7R5SiWnRsEGIoEAxCma2MOVlz2NWwizRzGmpJTYurhXxrPgtzFlLfXc8z258h1Zgabr3o8nTx\n5JYneeTiR3jn2DusP7EevVqPgsJ7x9/jstLL+Pq0r4+qIDp9Tp7e+jTHO46jKAoqSYXNZOMHF/wg\nPFbqTMCkNXH73Nu5cfqNOH1OUowpw7azGwkTbWw90fcXCDEUCAZFJam457x72FizkU+rPiUQDHBd\n+XUsL1qOUWtk07FNqCRVxKxCq8GK3W7nrSNvsaFqA4VJheGK06AcZF3FOs7LPo+SlJJRW+frh1+n\noqOCAmtBWGSbHE08t+u5kIXcGTYo26KzjF7byATR33Gmd7gwiMkS8YgQQ4HgFOjUugGLd1qdrTHH\nM2lUGnY37MagMUQ4qKhVarSSlj2Ne0ZNDANygM+rPycnMdIuL8OcwQn7CVqcLRMSHWZmRldhwsm0\n31im+/rvEfbOE7RYIgfgjjYDOc70/ky4zsQvQgwFgtNgim0KOxp2YDPZIo77g37yrHnUd9dHnySB\ngjJqawjKQQJyALUU6dIjSRISUti/dLz51a8mLvqJ1cSelDR6TfIDMVTHmfEmHgqI4p34cuEVCM4w\nFuYuxGayUdtVS0AOhKpO7VUUJRdxddnVuANuZEUOvz4oB/EH/aNamKTX6JmSNoVWV2vEcafPSYIu\nYUjG4Gc7vVMjeqdF9P5zrhSj9H456P+PaOk4iYgMBYLToNcQYM2RNWyp24JaUrNq0iqumHxFqDey\n8CI+qfoEg8aAoij4gj5Wla6iJHn09gsBbpx2I49sfITarlqSDEk4fA58QR/3Lrg3wjjhXKU3NSqs\nzAQDIX5LBILTJMWYwnfmfId/mPUPVHRWsLVuK3/94q/MzZzLN2Z+g4W5C9nVuAuVpGJ+9nxKU0pH\nvaClIKmAny/7OZ9UfsLxjuOUp5VzcdHFp5zOIRAIQggxFAhGibeOvsWaI2vQa/SopdAMw9mZs7lv\nwX2Up5VHvLbd1c6RtiNIksQU25RR8VjNsGRw04ybTvs6glMzlD24sZosMR77f+fiHqMQQ0Hc4g14\n2dmwk/3N+7EarJyfe35cRjqyIlPZWcmaI2si5hsqisLepr3sbNjJ+Xnnh1//0Zcf8dKBl5BlGaRQ\n+8a3Z36bZUXLJugJBMNlKGOqxko0xmOm45k2N3I0EGIoiEtcfhePb3qcE50nsOgs+II+PvjyA26Z\ndQvLi5ZP9PKAkAh+UvkJ7xx9hyNtR6jrqUOj0pCbmBuq5JQkrHorW+q2hMWwyl7Fi1+8SHZCdtgd\nxhvw8t/7/ptJqZNOa6qF4CQT7SgTb4j349QIMRTEJRuqNnCi80REJOgL+vjL/r8wN2tuXHikvnf8\nPV4+8DJZCVlkWjKp665jR8MOFBTyrflAqIWib8vDtvptaCRNhE2aXqNHJanY1bBLiOEocbam8kaK\neD9OjRBDQVyyqXZT2N6sF51ah6zIHGs/xnk5503QykJ4Ah7eOfoOedY8dGod6eZ0dGoderWew62H\nyUvMQ0Ghy9PF4ryTw5BdPhcadfSvnVqlnpBZkWcz5+K+l2DkCDEUjAtBOcinlZ+yrmIdXd4upqVN\n49ryaylMKoz5erWkjujP60tfRxcIucD0+HrIMGdE2KKNFU6fkxf3v8jGmo2YdWbyrfmUppQyI2MG\n+5v34/Q5qbRXIiGxpGBJRE/hrMxZfFr1acSYIkVR8Aa8zEgfv4ny5wLn4r6XYOQIMRSMCy9+8SIf\nffkRmZZMsixZHO84zi//95f8n6/8n3BKsS9fKfgKL+x7gQRdQlg03H43WrU2PD2ix9tNNDQOAAAe\nt0lEQVTD83ueZ2/TXlSSCpWk4sqyK7ly8pXDal1QFIXDbYfZULUBu8fO7MzZLMlfQoI+Ieq1vqCP\nJzY/wbH2Y6hVajQqDRUdFTQ7mllasJQEXQJV9iquKL2CWZmzKE0tjRDvGekzmJkxk31N+0g1paIo\nCp3uTuZmz2Vq2tThvq3jhoiyIjnb9+DO9ueLhRBDwZjT6mzl08pPKUouCgtDujmdJkcT7xx9h39c\n8I9R5ywpWMK+5n3sa9qHVq0lKAcBuKTkEjZUbiDRkMjn1Z/zZeeX5FvzkSQJf9DPqwdfJdmQzJKC\nJUNe37qKdbx84GVMWhN6jZ5XDr7ChqoN/GTJT0gyRM4f2t+0n8rOSkpTS3H5XZywn8Cqt9Lt7aa2\nuxZFUbjnvHu4fPLlMe+lVWv53sLvsbl2M5tqQibf1029jkW5i1Cr1DHPiQdElBXJRH4BGA+hOhe/\n4AgxFIw5DT0NSJIUld5MMaZwuO1wzHN0ah3fX/h9Drcd5kjrETQqDdsbtvN+xftoVBocPgf7mvex\nsnhlOArUqrWkm9N599i7QxZDu8fOa4deIzcxNzwqKMmQRLW9mo++/Ijrp10PhCpHq+xVfHzi47Cv\n6PT06SgoVHdV4w16OdF5gvsX3s+qSasGvadOrWNZ4bL4nHoviHvORaEaD4QYCsacBH1CTGNqt99N\nqjF1wPPUKjXT06czPX06rx18jYbuBgqTCpEkiTZXGypJxc6GnawsWRkWWrPOTF13XcSe3GCc6DyB\noihRM/PSzGlsq9/G9dOup8XZwu+2/Y667jrquuuotFfi9DuZljaN2ZmzKbeVc6LzBFdPuZrrpl03\nzHdHIBgfRKp7cIQYCsacwqRCbEYb2+u2o9foSTGmkGZOo83Vxg3TbhjSNT6t/pSshKywwFl0FvRq\nPe6AG7vHHnZw6XR3UpJcMuQ9Q61KG/O1ATmASWtCVmR+u+23tLvaKUgqCK/7YMtBEnQJFCQVEFSC\nmHVmlheOTf+jN+CluqsalaSiMKlQeI0Og/4zBSH09x/96NwTAJHqHhzxWyUYcyo6KmhxtdDoaMTl\ndyErMiatiR8u/iGLchcN6Rr+oD9iT82gMVCcXMze5r24/W4Ug4LdY6fH18M95fcMeW2TUydj0pro\n9naTqE8EQgU1bc42bplzC5WdldR311OQVACEJrEvyl3Eltot7G7cjYSEXqPnH8/7xzGZDrGzYSfP\n73keT8ADgFVv5Z7z7qE0tXTU73W2kZkZmmNosUQfF9MaBP0RYigYU4JykH/f+e+kGFK4quwqur3d\n+GU/XZ4uvEHvkCO4RbmL2Fy7OaIpvTi5GE/Ag06to7qrmsKkQu6Yd0eUD+hg6DV67ltwH09vfZoO\ndwcogASL8haxtGAph1oPRa0xw5LBqkmrqOmq4cHFD1KcXIxeox/yPYdKXXcdv9/xe2wmG+nmdAC6\nPF08ueVJfr3y12HxFsTmV78S0ZBg6AgxFIwptd212D128q35eANearpqqO6qJigHqe6q5uqyq0kx\nndqk+uqyqznQcoDqrmoSdAm4/W4CcoCffeVnLMhZQFAJopbUI5oGUWYr4/FLHudAywFcPhcFSQUU\nJxcjSVJYfINyMByZuv1u9jTuQafRsaFqAypJFW73GE02Vm9ELakxaU3hY1aDFbvdzu7G3eNagHO2\nldrv2hUavtsfsX927iLEUDCmKEqocCYoB9lct5kuTxcJugRkRabJ0cSjGx/loWUPDdos7/a7+bzm\nc9x+N23ONvxBP0sKlrCqZFU4famRTu9/ZYvOEjNlm2JM4ZKSS1h7bC1p5jRkWWZ95Xr8QT9LC5ay\nv3k/W+u3csvMW5ieMR21Sk2yIXlURjS1uFowaAxRx9UqdSiKHUfONoFwu8+OiFEUxYweQgwFY0qe\nNY9EfSInOk7Q5e4iyRjq23N4HEyxhaazb6/fPqD5dkAO8OSWJznWfoxMSybT06fT5GiisrMSm8k2\nLs9ww7QbyE7I5oOKD9hUu4lEfSKLcheF05RNPU088OEDTM+YjkbSUJpayq1zbiXTMrKwqcnRRKe7\nkxxLDnsa95Bqiqy49cv+UR8OLBg9BhIoGH2REmng0SOuxfDBBx/k3XffRafTUVJSwp/+9Ces1ok3\naD6XURSFSnslHe4O0kxp4Yb3gdCoNNwx9w7uW3cf7oAbrU+LX/Zj1popt5Xj8Dk40naEqWlTaXI0\nYTVYKbAWhK95sOUgx9qPhVsqIDTItqqzalARHU1UkoqlBUtZWrCUO9+5E5vJFm7FcPqc7GjYgTfg\nJUGbQJo5jdquWh7b9Bi/vOiXGLXGId/H6XPyn7v/k31N+1BJKrwBL23uNgCyE7KRFZlGRyMlySVM\nS582Js96tpGZCa+/HooE+9LWFqo0XbFi9O85kEDBxIrU2ZbqHm3iWgwvueQSfv3rX6NSqfjRj37E\no48+yq9E7D9h9Hh7+N3233G8/TiSJCErMjPSZ3D3eXdH7Gv1Z2r6VO4+727+sOMPmHVmUowp5CTk\noFVraXG2sK95H9vrt4evOSl5EvcuvJckQxJfdn6JVh3d/mDRWzjYevC0xdAT8HCg5QDtrnayE7Ip\nTysftHXBqDUSkANhMazpqkFWZHQaHRq1BkmSyLBkUGWvYm/T3og5hqfiv/f9N/ub94e/YMiKjL/N\nj0VvodXVikalYXXpai4vvVy0VwxArKjM7Q594PcVvrfeim65ONsRH52DE9e/UStXrgz/eeHChbz+\n+usTuBrBn/f/mS87TtqfKYrCgZYDvHrwVf5h9j8Meu6K4hWsP7Eek9YUTi86fU7quutI1CcyLX0a\nKkmFooQcXZ7b9RwPXvAgVr01bMXWF4/fg814emnSJkcTj296nHZ3OxISCgoFSQX88/n/HF6jrMhU\ndFTQ5ekiKyGLiwov4rXDr1GUVIQkSXR7u5EVGbPOHGHdplFpaHQ0Dnktdo+dnQ07w7MQIRSRFicX\nY/fY+cPlf4hru7b+TNReVqyobO/e+BM+sdcXf8S1GPbl+eef56abbproZZyzdHu7oz6sJUkiJzGH\nz2s+5+vTvx6z2KOXJEMSD5z/AL/f8XtqumoAMGqMpBhTIjxLJUkiOyGbw62HaXG2MDdrLq8cfIUe\nb0/YONvtdxNUgizOXzzg/U6Foij8567/xOV3RUzOqOmq4bVDr3HrnFtpd7Xz9LanqeuqC0dqczLn\nMCN9BgdbD6IoCr6gD7/sZ0HOggi7uYAcICchJ/z3dlc77oCbDHNGlNsNgMPnQCLask6n1uEJePAG\nvZhUA0ff8Ua872VZLKE19l/PeKUM4/39OReZcDFcuXIlTTG+Ij3yyCNceeWVAPzyl79Ep9PxjW98\nY7yXJ/g7bn9o06X/h7VaUhNUgviCvkHFEEIN7k9c8gTV9mpkRSYnMYd71t6DXh3Zo9frY+r0O8mw\nZPD9hd/n9zt/T0dXB5IioVVruXP+neRZ80b8PK2uVk7YT5CfGDkxIzshm821m7l5xs08t+s5Whwt\n4YpVRVHY3bibq8uu5trya6nvricoB/nbwb/hC/pQFAUFhSZHE+nmdGZlzqLT3ckf9/yRg60HkZAw\naU3cNOOmiBmHQHgf0hvwRvQsdnu7yUrIwqgZ+t6j4NSsWBESnhdemOiVCOKFCRfDjz76aNCfv/DC\nC7z33nt8/PHHA77moYceCv952bJlLFu2bJRWJ+jFZrKRqE/E6XNGtEF0e7vJNGeSoIsedxQLjUpD\nScrJSsjSlFKaHE0RFZO+oA+1Sk2WJeToMjV9Kk9e+iQnOk8gKzJFSUXDKkyJhT/oR0KK2ovsFfcG\nRwPH2o9FjJfqjVrXV67n2vJrmZQyCYBJqZP4y/6/hPdSZ2fO5hszvoFOrePRjY/S0N1AfmIotez2\nu3lu13OkGFIizAEMGgPXTrmWv+z/C2nmNCw6C3aPnS5PF/80+59GpVVDcPYhimJOsmHDBjZs2DDi\n8ydcDAfj/fff5/HHH+ezzz7DYBg46ugrhoKxQa1S840Z3+DZ7c9i8VtI0CfQ5enCE/DwwPkPjPjD\n+oZpN/DoxkcJOAKkGFNw+p20u9r55sxvRgieTq1jim3KaD0OmZZMrHprRPoVQhFjua0cRVFQSaqo\n59KpdeE0bW9vY741nx9f+GMcPgcqSRX+snCs/Rg19ppwZAmhAhyL1sK6inVRTjmXlFyC1WDlnaPv\nUN9dT1FyEXfNvyuu5xyeCYx3SnQggRqLe8baX+zdj+xvKnC270f2D4QefvjhYZ0f12J433334fP5\nwoU0559/Pr///e8neFXnLgtyFpC4JJH3jr9HbXct5WnlXF56eUSkN1xKU0v56dKf8vbRtznefpx0\nczo3z7iZednzRnHl0ahVam6dcytPbX2Kbm83Zp2ZHm8POrWOm6bfRJo5DY1KE5W2bHO1UWYri6rm\nlCQpahiw3WOPeW+LzkJjT3RxjSRJLMpdNGS/VkE0sYRo0iS48MLxE4Kh3GfXrlBhT3+C0bViQ6Jv\nQU5fP1aL5WQVrdiPHJy4FsPjx49P9BIE/ZhimzKqERqEPEbvX3T/qF5zKMzImMHPl/+cTys/paGn\ngQvzLmRZ0bJwM/+N02/k+T3Pk2RIwqwz0+nuJKgEhzxpI8OcgYISNU7K7rEzJ2vOmDxTvDBR6bsz\nJfJxuyE3N/p4Xd3Irte3IGfvXkj6e2GzPfb3MUEM4loMBfGN2+9mf/N+mp3NZFuymZExY0wMq8eS\n3MRcvjXrWzF/tqwwJIzrjq+j2dnMvOx5rJq0KmIfcTDyrfnMypjF3ua95CbkolFp6PR0EpADXDbp\nstF8jLjjTBGlicJojC1UxiFuhfdvzdi4MSSC/Sd0CIaOEEPBiGhyNPHYpsfocHeEi04yLZk8eMGD\nUfZhZyqSJDEjYwYzMmaM+Py7z7ubNw+/yadVn+IP+ilMLuTeBfdG7CMKxod46u2bN+/0Wiv6t2b0\nRoMiEhw5QgwFw0ZRFJ7f83xUj159dz0vHXiJexfcO3GLizMMGgM3zbiJ66ddjz/ox6AxjKjYKCAH\nAITzzGkw3r19g4lvLNavj1340nuOiLbHFvGbJRg2nZ5Ojrcfj0oXZiVksathFy6/a1B7tnMRjUoT\nFrLeAccW3alzWp3uTl479Bpb67aioDA/ez7XTb0uPN9QEL8MV3wdjlCa83QF22I5GSE6HCfPPRfb\nLYaDEEPBsAnIAYgR3Eh/PxjLPk0QqkT96xd/ZU/jHpBgcspkbp5584DmAZ6Ah8c2P0aLo4XshGwA\n9jTuoaKjgoeXPRxVvdr/3E01m9hcuxmNSsPSgqUszF0oIsuzlF4BdDhg9uyTx0VEOXTEb4Zg2KSZ\n0sg0Z2L32Ek2JoePt7naKE0tHVLEc67hCXh4bNNj2D128qx5SEjUdtfyq02/4hfLfxHxPvayr2kf\njT2NEanonMQcqu3VbK/fzsXFF8e8lzfg5YnNT3C8/TgpxhRkRebfd/47+5r3cdf8u6JchATjT6xq\nW4dj5NFb3/YJ4aozMoQYCoaNJEncOudWHt/8OI4uB2adGafPiU6t45szvincUmKwr2kfLc6WCGFL\nN6dT01XDxpqNXFl2ZdQ5lfZKdGpd1HGT1kRFR8WAYrircRfH249TlFwUPpZkSGJb3TYuLrqYMlvZ\n6T+QYMisXx9pFO5whPoe+0dtt9wy8OgnwdgjxFAwIkpTS/l/y/8fn1d/Tm13LUXJRSzJX3LWVJKO\nNvU99WhV0QbdZq2ZSntlzHPSzen4g/6o456AhwxLxoD32tO4JyqFKkkSGpWGI21HhBiOMw7Hyb6/\nXgoLT69wZ7h9nPFUSRuvCDEUjJgMSwbXTbtuopdxRpBhzsAvRwuby+8iLzH2nuH87Pm8dug1Otwd\npBhTgJAXrFqljjL67otZZ8YX9EUdlxX5tD1dB2M8J7yPhPE2Aui9X//xUaPRCzjc91JMyTg1QgwF\ngnFgTtYckg3JNDuaw5Wgdo8djUrDhfkXxjwnUZ/Igxc8yH/s+g9q7DUgQbIhmQfOf4A0c9qA97og\n7wI+rQz1NfaOi/IEPKgkFXMyx875Jl4nvPcy3mLce7+hpj/H09NUEI0QQ4FgHDBpTfzL4n/h+T3P\nU9FZAUC2JZv7Ft43qLAVJRfxyMWP0NjTiKzIZCdkn3LIb2lKKTdMu4HXDr+GIisghVo7bp97+6D3\nEkwsEx05n+sIMRQIxomshCx+suQndLg7kBUZm8k2pGIjlaQiJzHnlK/rRZIkLp98OQtyFnCs/Rhq\nlZpyWzlWg/V0li8QnNUIMRQIxhFJksatyCjNnCYiwThAzBw8MxBiKBAIBGNIPKQ/hSCfGiGGAoFg\nVBAFIPFLPAhyvCMpiqJM9CJOB0mSOMMfQSAQCASjzHC1QfgyCQQCgeCcR4ihQCAQCM55xJ6hQCAQ\nCIZEvLsMnQ5CDAUCgUAwJOLdZeh0EGlSgUAgEJzzCDEUCAQCwTmPSJMKBALBBCLGK8UHQgwFAoFg\nAhHjleIDIYYCQRwRlIMcaj3EgZYDmHVm5mfPJzshe6KXJRCc9QgxFAjiBH/Qz7M7nmVP4x7+f3v3\nGxNHnfhx/LMUFqStUvoHOLa6lEJpEAja9NALFQ/X8EBpmxjTJmf6zycSn0j9U59oiYetrZjc1cY0\nWj3iRW1MWsGkErhQYs+Lh21pK8Gz1AXLn4JpC7WtCAXmHlx++5NCa0HlO7PzfiWTdGYX+sk3w3yy\nM9+diY6M1vDosA58dUAbcjdoxR0rTMcDwvqWe5QhYBP/6viXjnUfU8qclNCjnQaHB1V5vFJZC7I0\n55Y5hhPC7cL5GiazSQGb+OeZfyo+Nn7MMw6jI6M1Yo3oP+f+YzAZEP74ZIiwMDg8qLpgnerb6jU4\nMqjlv1uuh9IfmrZnB/4aLFnyaOKH/VriZvThiscr2QNPrYDjjVqj+svnf1FTT5MSZyUqMiJSvZd7\ndWv0rdpasNUxT3ivb6vX347/TSlx/3+adGhkSD2Xe7QzsNNRxQ6YxlMr4DrfXPhGJ3pPKCUuRbFR\nsfLO8GrhbQt1cfCiDn972HS8m/aHhX9Q5vxMtfW3qfdyrzq/71T3pW6tvXMtRQj8xjhNCsf79uK3\nkjTmWpsk3RZ9m05+d1IPLXnIRKxJi46MVuk9pTrRc0InvjuhWd5Z+n3y7+WP85uOBoQ9yhCON9s7\ne8LtA8MDmhc7b5rTTN7Vkas61HZI/2j7hwauDmjZ75Zp5ZKVjsgOhAtOk8LxshKyNMs7S/0/9oe2\nDQ4PamB4QAX+AnPBboJlWdpzdI/+/uXfZVmWZnln6fCZw/rzp39W30Cf6XiAa9i+DCsqKhQREaEL\nFy6YjgKbio2KVek9pfLIozMXz+jMxTM698M5rc9Zr/S56abj3VBbf5uOdB9RSlyKZnpnKjoyWr5b\nffr+x+/V0N5gOh7gGrY+TdrR0aG6ujrdcccdpqPA5hbNWaSdD+5UsC+oqyNX5Y/za6Z3pulYP6vj\nYocsyxp3vTPuljh9+d2XWr10taFkgLvY+pNhaWmpduzYYToGHCIyIlLpc9OVuSDTEUUoSTO9M8cV\noST9OPyj4mPiDSQC3Mm2ZVhVVSWfz6fs7GzTUYDfTOb8TM32zh5zfXBoZEiXhy7rj4v+aDAZ4C5G\nT5MGAgH1TPAgr/Lycm3btk21tbWhbXyxHuHolqhbVHpPqf7677/qzMUzkiSPPPpT9p+0dN5Sw+kA\n97DlHWiam5tVWFio2NhYSVJnZ6eSk5PV2NioBQsWjHmvx+PRiy++GFovKChQQUHBdMYFfrHh0WF9\nc+EbDY0MyR/n1+zoib8uAmBiDQ0NamhoCK2XlZVN6kOULcvwWikpKTp69Kji48dfQ+F2bADCyZYt\n/3vg77USE8P7qRG/tsl2g61nk/6fiSYYAEA46umR/P7x26/3HEH8OhxRhsFg0HQEAEAYs+1sUgAA\npgtlCABwPcoQAOB6lCEAwPUcMYEGANwiMXHimaOJidMexVUc8T3DG+F7hgCAa022GzhNCgBwPcoQ\nAOB6lCEAwPUoQwCA61GGAADXowwBAK5HGQIAXI8yBAC4HmUIAHA9yhAA4HqUIQDA9ShDAIDrUYYA\nANejDAEArkcZAgBcjzIEALgeZQgAcD3KEADgepQhAMD1KEMAgOtRhgAA16MMAQCuRxkCAFyPMgQA\nuB5lCABwPcoQAOB6lCEAwPUoQwCA61GGAADXs3UZ7tq1S0uXLtWdd96p5557znQcAECYsm0ZHjp0\nSNXV1Tp58qSam5v19NNPm44UVhoaGkxHcCTGbeoYu6lh3KaHbcvwjTfe0PPPP6+oqChJ0vz58w0n\nCi/8gU0N4zZ1jN3UMG7Tw7Zl2Nraqk8//VR5eXkqKCjQkSNHTEcCAISpSJP/eSAQUE9Pz7jt5eXl\nGh4eVl9fnz7//HN98cUXevTRRxUMBg2kBACEPcumioqKrIaGhtB6amqqde7cuXHvS01NtSSxsLCw\nsLCEltTU1El1jtFPhjeyatUq1dfX67777tOpU6c0NDSkuXPnjnvf6dOnDaQDAIQTj2VZlukQE7l6\n9ao2btyo48ePy+v1qqKiQgUFBaZjAQDCkG3LEACA6WLb2aQ/58MPP1RmZqZmzJihY8eOjXlt27Zt\nSktLU0ZGhmpraw0ltL+tW7fK5/MpNzdXubm5qqmpMR3J1mpqapSRkaG0tDS98sorpuM4ht/vV3Z2\ntnJzc7V8+XLTcWxr48aNSkhIUFZWVmjbhQsXFAgElJ6ergcffFD9/f0GE9rTROM2lWObY8swKytL\nBw4c0IoVK8Zsb2lp0b59+9TS0qKamhqVlJRodHTUUEp783g8Ki0tVVNTk5qamlRUVGQ6km2NjIzo\nySefVE1NjVpaWvT+++/rq6++Mh3LETwejxoaGtTU1KTGxkbTcWxrw4YN4w7a27dvVyAQ0KlTp1RY\nWKjt27cbSmdfE43bVI5tji3DjIwMpaenj9teVVWltWvXKioqSn6/X4sXL+YP8AY4S35zGhsbtXjx\nYvn9fkVFRWnNmjWqqqoyHcsx2M9+Xn5+vubMmTNmW3V1tdatWydJWrdunT766CMT0WxtonGTJr/P\nObYMr6e7u1s+ny+07vP51NXVZTCRve3atUs5OTnatGkTp2BuoKurSwsXLgyts1/dPI/HowceeEDL\nli3Tm2++aTqOo/T29iohIUGSlJCQoN7eXsOJnGOyxzZbl2EgEFBWVta45eOPP57U7/F4PL9RQvu7\n3hhWV1friSeeUFtbm44fP66kpCRt3rzZdFzbcvM+9Et99tlnampq0ieffKLdu3fr8OHDpiM5ksfj\nYT+8SVM5ttn2e4aSVFdXN+mfSU5OVkdHR2i9s7NTycnJv2YsR7nZMXz88cf18MMP/8ZpnOva/aqj\no2PMGQhcX1JSkqT/3V949erVamxsVH5+vuFUzpCQkKCenh4lJibq7NmzWrBggelIjvDTcbrZY5ut\nPxnerJ+eGy4uLtYHH3ygoaEhtbW1qbW1lRls13H27NnQvw8cODBmNhbGWrZsmVpbW9Xe3q6hoSHt\n27dPxcXFpmPZ3g8//KBLly5Jkq5cuaLa2lr2s0koLi5WZWWlJKmyslKrVq0ynMgZpnRsm/L90gzb\nv3+/5fP5rJiYGCshIcEqKioKvVZeXm6lpqZaS5YssWpqagymtLfHHnvMysrKsrKzs62VK1daPT09\npiPZ2sGDB6309HQrNTXVevnll03HcYRgMGjl5ORYOTk5VmZmJuN2A2vWrLGSkpKsqKgoy+fzWW+/\n/bZ1/vx5q7Cw0EpLS7MCgYDV19dnOqbtXDtue/fundKxjS/dAwBcLyxOkwIA8EtQhgAA16MMAQCu\nRxkCAFyPMgQAuB5lCABwPcoQAOB6lCEAwPVsfW9SAOMdPXpU7777rmbMmKH29na99dZb2rNnj/r7\n+9XV1aWysjItWrTIdEzAUShDwEGCwaDeeecdvf7665Kk9evXKy8vT5WVlRodHVV+fr7uuusuPfXU\nU4aTAs5CGQIOUlFRoR07doTWr1y5ovj4eOXl5amzs1ObN2/W+vXrzQUEHIp7kwIO0t7eLr/fH1r3\n+XzasGGDXnrpJXOhgDDABBrAQX5ahF9//bW6u7t1//33mwsEhAnKEHCo+vp6eb1e3XvvvaFtwWBw\nzHsuXbqkRx55ZMyDiQGMRxkCDjEwMKBnn31Wzc3NkqS6ujrl5OQoJiZGkjQ6OqqdO3eG3r937169\n9tpr2r9/v7gaAtwYE2gAhzh48KBeffVV3X333YqMjNTp06cVFxcXer28vHzM5JlNmzZJksrKyqY7\nKuA4TKABHOL8+fN65plnNG/ePEVEROiFF15QSUmJYmJi5PV6tXLlShUWFo77uYiICLW3t+v22283\nkBpwBsoQCHOUIfDzuGYIAHA9yhAA4HqUIeACXA0BbowyBMLUe++9p5KSEnk8Hm3ZskW7d+82HQmw\nLSbQAABcj0+GAADXowwBAK5HGQIAXI8yBAC4HmUIAHA9yhAA4HqUIQDA9ShDAIDrUYYAANf7L6s6\n0nX+T+nzAAAAAElFTkSuQmCC\n",
"text": [
""
]
}
],
"prompt_number": 36
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"
\n",
"
\n",
"\n",
"\n",
"## Defining the objective function and decision rule\n",
"\n",
"Here, our **objective function** is to maximize the discriminant function $g_i(\\pmb x)$, which we define as the posterior probability to perform a **minimum-error classification** (Bayes classifier). \n",
"\n",
"$ g_1(\\pmb x) = P(\\omega_1 | \\; \\pmb{x}), \\quad g_2(\\pmb{x}) = P(\\omega_2 | \\; \\pmb{x}), \\quad g_3(\\pmb{x}) = P(\\omega_2 | \\; \\pmb{x})$\n",
"\n",
"So that our decision rule is to choose the class $\\omega_i$ for which $g_i(\\pmb x)$ is max., where \n",
" $ \\quad g_i(\\pmb{x}) = \\pmb{x}^{\\,t} \\bigg( - \\frac{1}{2} \\Sigma_i^{-1} \\bigg) \\pmb{x} + \\bigg( \\Sigma_i^{-1} \\pmb{\\mu}_{\\,i}\\bigg)^t \\pmb x + \\bigg( -\\frac{1}{2} \\pmb{\\mu}_{\\,i}^{\\,t} \\Sigma_{i}^{-1} \\pmb{\\mu}_{\\,i} -\\frac{1}{2} ln(|\\Sigma_i|)\\bigg) $"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"
\n",
"
\n",
"\n",
"## Implementing the discriminant function\n",
"Now, let us implement the discriminant function for $g_i(\\pmb x)$ in Python code:"
]
},
{
"cell_type": "code",
"collapsed": false,
"input": [
"def discriminant_function(x_vec, cov_mat, mu_vec):\n",
" \"\"\"\n",
" Calculates the value of the discriminant function for a dx1 dimensional\n",
" sample given the covariance matrix and mean vector.\n",
" \n",
" Keyword arguments:\n",
" x_vec: A dx1 dimensional numpy array representing the sample.\n",
" cov_mat: numpy array of the covariance matrix.\n",
" mu_vec: dx1 dimensional numpy array of the sample mean.\n",
" \n",
" Returns a float value as result of the discriminant function.\n",
" \n",
" \"\"\"\n",
" W_i = (-1/2) * np.linalg.inv(cov_mat)\n",
" assert(W_i.shape[0] > 1 and W_i.shape[1] > 1), 'W_i must be a matrix'\n",
" \n",
" w_i = np.linalg.inv(cov_mat).dot(mu_vec)\n",
" assert(w_i.shape[0] > 1 and w_i.shape[1] == 1), 'w_i must be a column vector'\n",
" \n",
" omega_i_p1 = (((-1/2) * (mu_vec).T).dot(np.linalg.inv(cov_mat))).dot(mu_vec)\n",
" omega_i_p2 = (-1/2) * np.log(np.linalg.det(cov_mat))\n",
" omega_i = omega_i_p1 - omega_i_p2\n",
" assert(omega_i.shape == (1, 1)), 'omega_i must be a scalar'\n",
" \n",
" g = ((x_vec.T).dot(W_i)).dot(x_vec) + (w_i.T).dot(x_vec) + omega_i\n",
" return float(g)"
],
"language": "python",
"metadata": {},
"outputs": [],
"prompt_number": 39
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"
\n",
"
\n",
"\n",
"## Implementing the decision rule (classifier)\n",
"Next, we need to implement the code that returns the max. $g_i(\\pmb x)$ with the corresponding class label:"
]
},
{
"cell_type": "code",
"collapsed": false,
"input": [
"import operator\n",
"\n",
"def classify_data(x_vec, g, mu_vecs, cov_mats):\n",
" \"\"\"\n",
" Classifies an input sample into 1 out of 3 classes determined by\n",
" maximizing the discriminant function g_i().\n",
" \n",
" Keyword arguments:\n",
" x_vec: A dx1 dimensional numpy array representing the sample.\n",
" g: The discriminant function.\n",
" mu_vecs: A list of mean vectors as input for g.\n",
" cov_mats: A list of covariance matrices as input for g.\n",
" \n",
" Returns a tuple (g_i()_value, class label).\n",
" \n",
" \"\"\"\n",
" assert(len(mu_vecs) == len(cov_mats)), 'Number of mu_vecs and cov_mats must be equal.'\n",
" \n",
" g_vals = []\n",
" for m,c in zip(mu_vecs, cov_mats): \n",
" g_vals.append(g(x_vec, mu_vec=m, cov_mat=c))\n",
" \n",
" max_index, max_value = max(enumerate(g_vals), key=operator.itemgetter(1))\n",
" return (max_value, max_index + 1)"
],
"language": "python",
"metadata": {},
"outputs": [],
"prompt_number": 40
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"
\n",
"
\n",
"\n",
"## Classifying the sample data\n",
"Using the discriminant function and classifier that we just implemented above, let us classify our sample data. (I have to apologize for the long code below, but I thought it makes it a little more clear of what exactly is going on)"
]
},
{
"cell_type": "code",
"collapsed": false,
"input": [
"class1_as_1 = 0\n",
"class1_as_2 = 0\n",
"class1_as_3 = 0\n",
"for row in x1_samples:\n",
" g = classify_data(\n",
" row, \n",
" discriminant_function,\n",
" [mu_vec1, mu_vec2, mu_vec3],\n",
" [cov_mat1, cov_mat2, cov_mat3]\n",
" )\n",
" if g[1] == 2:\n",
" class1_as_2 += 1\n",
" elif g[1] == 3:\n",
" class1_as_3 += 1\n",
" else:\n",
" class1_as_1 += 1\n",
"\n",
"class2_as_1 = 0\n",
"class2_as_2 = 0\n",
"class2_as_3 = 0\n",
"for row in x2_samples:\n",
" g = classify_data(\n",
" row, \n",
" discriminant_function,\n",
" [mu_vec1, mu_vec2, mu_vec3],\n",
" [cov_mat1, cov_mat2, cov_mat3]\n",
" )\n",
" if g[1] == 2:\n",
" class2_as_2 += 1\n",
" elif g[1] == 3:\n",
" class2_as_3 += 1\n",
" else:\n",
" class2_as_1 += 1\n",
"\n",
"class3_as_1 = 0\n",
"class3_as_2 = 0\n",
"class3_as_3 = 0\n",
"for row in x3_samples:\n",
" g = classify_data(\n",
" row, \n",
" discriminant_function,\n",
" [mu_vec1, mu_vec2, mu_vec3],\n",
" [cov_mat1, cov_mat2, cov_mat3]\n",
" )\n",
" if g[1] == 2:\n",
" class3_as_2 += 1\n",
" elif g[1] == 3:\n",
" class3_as_3 += 1\n",
" else:\n",
" class3_as_1 += 1"
],
"language": "python",
"metadata": {},
"outputs": [],
"prompt_number": 81
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"
\n",
"
\n",
"\n",
"## Drawing the confusion matrix and calculating the empirical error\n",
"Now, that we classified our data, let us plot the confusion matrix to see what the empirical error looks like."
]
},
{
"cell_type": "code",
"collapsed": false,
"input": [
"import prettytable\n",
"\n",
"confusion_mat = prettytable.PrettyTable([\"sample dataset\", \"w1 (predicted)\", \"w2 (predicted)\", \"w3 (predicted)\"])\n",
"confusion_mat.add_row([\"w1 (actual)\",class1_as_1, class1_as_2, class1_as_3])\n",
"confusion_mat.add_row([\"w2 (actual)\",class2_as_1, class2_as_2, class2_as_3])\n",
"confusion_mat.add_row([\"w3 (actual)\",class3_as_1, class3_as_2, class3_as_3])\n",
"print(confusion_mat)\n",
"misclass = x1_samples.shape[0]*3 - class1_as_1 - class2_as_2 - class3_as_3\n",
"bayes_err = misclass / (len(x1_samples)*3)\n",
"print('Empirical Error: {:.2f} ({:.2f}%)'.format(bayes_err, bayes_err * 100))"
],
"language": "python",
"metadata": {},
"outputs": [
{
"output_type": "stream",
"stream": "stdout",
"text": [
"+----------------+----------------+----------------+----------------+\n",
"| sample dataset | w1 (predicted) | w2 (predicted) | w3 (predicted) |\n",
"+----------------+----------------+----------------+----------------+\n",
"| w1 (actual) | 98 | 1 | 1 |\n",
"| w2 (actual) | 2 | 93 | 5 |\n",
"| w3 (actual) | 1 | 2 | 97 |\n",
"+----------------+----------------+----------------+----------------+\n",
"Empirical Error: 0.04 (4.00%)\n"
]
}
],
"prompt_number": 85
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"
\n",
"
\n",
"\n",
"# 2) Assuming that the parameters are unknown - using MLE"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"
\n",
"
\n",
"\n",
"### About the Maximum Likelihood Estimate (MLE)\n",
"\n",
"In contrast to the first section, let us assume that we only know the number of parameters for the class conditional densities $p (\\; \\pmb x \\; | \\; \\omega_i)$, and we want to use a Maximum Likelihood Estimation (MLE) to estimate the quantities of these parameters from the training data (*here:* our random sample data).\n",
"\n",
"\n",
"Given the information about the form of the model - the data is normal distributed - the 2 parameters to be estimated are $\\pmb \\mu_i$ and $\\pmb \\Sigma_i$, which are summarized by the \n",
"parameter vector $\\pmb \\theta_i = \\bigg[ \\begin{array}{c}\n",
"\\ \\theta_{i1} \\\\\n",
"\\ \\theta_{i2} \\\\\n",
"\\end{array} \\bigg]=\n",
"\\bigg[ \\begin{array}{c}\n",
"\\pmb \\mu_i \\\\\n",
"\\pmb \\Sigma_i \\\\\n",
"\\end{array} \\bigg]$ "
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"For the Maximum Likelihood Estimate (MLE), we assume that we have a set of samples $D = \\left\\{ \\pmb x_1, \\pmb x_2,..., \\pmb x_n \\right\\} $ that are *i.i.d.* (independent and identically distributed, drawn with probability $p(\\pmb x \\; | \\; \\omega_i, \\; \\pmb \\theta_i) $). \n",
"Thus, we can **work with each class separately** and omit the class labels, so that we write the probability density as $p(\\pmb x \\; | \\; \\pmb \\theta)$ "
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"
\n",
"
\n",
"### Likelihood of $ \\pmb \\theta $"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Thus, the probability of observing $D = \\left\\{ \\pmb x_1, \\pmb x_2,..., \\pmb x_n \\right\\} $ is: \n",
"
\n",
"
\n",
"$p(D\\; | \\; \\pmb \\theta\\;) = p(\\pmb x_1 \\; | \\; \\pmb \\theta\\;)\\; \\cdot \\; p(\\pmb x_2 \\; | \\;\\pmb \\theta\\;) \\; \\cdot \\;... \\; p(\\pmb x_n \\; | \\; \\pmb \\theta\\;) = \\prod_{k=1}^{n} \\; p(\\pmb x_k \\pmb \\; | \\; \\pmb \\theta \\;)$ \n",
"
\n",
"Where $p(D\\; | \\; \\pmb \\theta\\;)$ is also called the ***likelihood of $\\pmb\\ \\theta$***."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"We are given the information that $p([x_1,x_2]^t) \\;\u223c \\; N(\\pmb \\mu,\\pmb \\Sigma) $ (remember that we dropped the class labels, since we are working with every class separately)."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"And the mutlivariate normal density is given as:\n",
"$\\quad \\quad p(\\pmb x) = \\frac{1}{(2\\pi)^{d/2} \\; |\\Sigma|^{1/2}} exp \\bigg[ -\\frac{1}{2}(\\pmb x - \\pmb \\mu)^t \\Sigma^{-1}(\\pmb x - \\pmb \\mu) \\bigg]$\n",
"\n",
"So that \n",
"$p(D\\; | \\; \\pmb \\theta\\;) = \\prod_{k=1}^{n} \\; p(\\pmb x_k \\pmb \\; | \\; \\pmb \\theta \\;) = \\prod_{k=1}^{n} \\; \\frac{1}{(2\\pi)^{d/2} \\; |\\Sigma|^{1/2}} exp \\bigg[ -\\frac{1}{2}(\\pmb x - \\pmb \\mu)^t \\Sigma^{-1}(\\pmb x - \\pmb \\mu) \\bigg]$"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"and the log of the multivariate density"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"$ l(\\pmb\\theta) = \\sum\\limits_{k=1}^{n} - \\frac{1}{2}(\\pmb x - \\pmb \\mu)^t \\pmb \\Sigma^{-1} \\; (\\pmb x - \\pmb \\mu) - \\frac{d}{2} \\; ln \\; 2\\pi - \\frac{1}{2} \\;ln \\; |\\pmb\\Sigma|$"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"
\n",
"
\n",
"#### Maximum Likelihood Estimate (MLE)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"In order to obtain the MLE $\\boldsymbol{\\hat{\\theta}}$, we maximize $l (\\pmb \\theta)$, which can be done via differentiation:\n",
"\n",
"with \n",
"$\\nabla_{\\pmb \\theta} \\equiv \\begin{bmatrix} \n",
"\\frac{\\partial \\; }{\\partial \\; \\theta_1} \\\\ \n",
"\\frac{\\partial \\; }{\\partial \\; \\theta_2}\n",
"\\end{bmatrix} = \\begin{bmatrix} \n",
"\\frac{\\partial \\; }{\\partial \\; \\pmb \\mu} \\\\ \n",
"\\frac{\\partial \\; }{\\partial \\; \\pmb \\sigma}\n",
"\\end{bmatrix}$"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"$\\nabla_{\\pmb \\theta} l = \\sum\\limits_{k=1}^n \\nabla_{\\pmb \\theta} \\;ln\\; p(\\pmb x| \\pmb \\theta) = 0 $"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"
\n",
"
\n",
"\n",
"## MLE of the mean vector $\\pmb \\mu$"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"After doing the differentiation, we find that the MLE of the parameter $\\pmb\\mu$ is given by the equation: \n",
"${\\hat{\\pmb\\mu}} = \\frac{1}{n} \\sum\\limits_{k=1}^{n} \\pmb x_k$\n",
"\n",
"As you can see, this is simply the mean of our dataset, so we can implement the code very easily and compare the estimate to the actual values for $\\pmb \\mu$."
]
},
{
"cell_type": "code",
"collapsed": false,
"input": [
"import prettytable\n",
"\n",
"mu_est1 = np.array([[sum(x1_samples[:,0])/len(x1_samples[:,0])],[sum(x1_samples[:,1])/len(x1_samples[:,1])]])\n",
"mu_est2 = np.array([[sum(x2_samples[:,0])/len(x2_samples[:,0])],[sum(x2_samples[:,1])/len(x2_samples[:,1])]])\n",
"mu_est3 = np.array([[sum(x3_samples[:,0])/len(x3_samples[:,0])],[sum(x3_samples[:,1])/len(x3_samples[:,1])]])\n",
"\n",
"mu_mle = prettytable.PrettyTable([\"\", \"mu_1\", \"mu_2\", \"mu_3\"])\n",
"mu_mle.add_row([\"MLE\",mu_est1, mu_est2, mu_est3])\n",
"mu_mle.add_row([\"actual\",mu_vec1, mu_vec2, mu_vec3])\n",
"\n",
"print(mu_mle)"
],
"language": "python",
"metadata": {},
"outputs": [
{
"output_type": "stream",
"stream": "stdout",
"text": [
"+--------+-----------------+-----------------+-----------------+\n",
"| | mu_1 | mu_2 | mu_3 |\n",
"+--------+-----------------+-----------------+-----------------+\n",
"| MLE | [[-0.17370434] | [[ 8.65908903] | [[ 5.77749337] |\n",
"| | [ 0.01919151]] | [ 0.02617762]] | [ 5.67218058]] |\n",
"| actual | [[0] | [[9] | [[6] |\n",
"| | [0]] | [0]] | [6]] |\n",
"+--------+-----------------+-----------------+-----------------+\n"
]
}
],
"prompt_number": 77
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"
\n",
"
\n",
"\n",
"## MLE of the covariance matrix $\\pmb \\Sigma$\n",
"\n",
"Analog to $\\pmb \\mu$ we can find the equation for the $\\pmb\\Sigma$ via differentiation - okay the equations are a little bit more involved, but the approach is the same - so that we come to this equation: "
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"${\\hat{\\pmb\\Sigma}} = \\frac{1}{n} \\sum\\limits_{k=1}^{n} (\\pmb x_k - \\hat{\\mu})(\\pmb x_k - \\hat{\\mu})^t$\n",
"\n",
"which we will also implement in Python code, and then compare to the acutal values of ${\\pmb\\Sigma}$."
]
},
{
"cell_type": "code",
"collapsed": false,
"input": [
"import prettytable\n",
"\n",
"def mle_est_cov(x_samples, mu_est):\n",
" \"\"\"\n",
" Calculates the Maximum Likelihood Estimate for the covariance matrix.\n",
" \n",
" Keyword Arguments:\n",
" x_samples: np.array of the samples for 1 class, n x d dimensional \n",
" mu_est: np.array of the mean MLE, d x 1 dimensional\n",
" \n",
" Returns the MLE for the covariance matrix as d x d numpy array.\n",
" \n",
" \"\"\"\n",
" cov_est = np.zeros((2,2))\n",
" for x_vec in x_samples:\n",
" x_vec = x_vec.reshape(2,1)\n",
" assert(x_vec.shape == mu_est.shape), 'mean and x vector hmust be of equal shape'\n",
" cov_est += (x_vec - mu_est).dot((x_vec - mu_est).T)\n",
" return cov_est / len(x_samples)\n",
"\n",
"cov_est1 = mle_est_cov(x1_samples, mu_est1)\n",
"cov_est2 = mle_est_cov(x2_samples, mu_est2)\n",
"cov_est3 = mle_est_cov(x3_samples, mu_est3)\n",
"\n",
"cov_mle = prettytable.PrettyTable([\"\", \"covariance_matrix_1\", \"covariance_matrix_2\", \"covariance_matrix_3\"])\n",
"cov_mle.add_row([\"MLE\", cov_est1, cov_est2, cov_est3])\n",
"cov_mle.add_row(['','','',''])\n",
"cov_mle.add_row([\"actual\", cov_mat1, cov_mat2, cov_mat3])\n",
"\n",
"print(cov_mle)\n"
],
"language": "python",
"metadata": {},
"outputs": [
{
"output_type": "stream",
"stream": "stdout",
"text": [
"+--------+-----------------------------+-----------------------------+-----------------------------+\n",
"| | covariance_matrix_1 | covariance_matrix_2 | covariance_matrix_3 |\n",
"+--------+-----------------------------+-----------------------------+-----------------------------+\n",
"| MLE | [[ 3.988021 -0.19957158] | [[ 3.79760541 -0.04062998] | [[ 4.35960544 0.51274876] |\n",
"| | [-0.19957158 2.69991303]] | [-0.04062998 3.05143476]] | [ 0.51274876 4.44341942]] |\n",
"| | | | |\n",
"| actual | [[3 0] | [[3 0] | [[4 0] |\n",
"| | [0 3]] | [0 3]] | [0 4]] |\n",
"+--------+-----------------------------+-----------------------------+-----------------------------+\n"
]
}
],
"prompt_number": 76
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"
\n",
"
\n",
"\n",
"##Classification using our estimated parameters\n",
"\n",
"Using the estimated parameters $\\pmb \\mu_i$ and $\\pmb \\Sigma_i$, which we obtained via MLE, we calculate the error on the sample dataset again. "
]
},
{
"cell_type": "code",
"collapsed": false,
"input": [
"class1_as_1 = 0\n",
"class1_as_2 = 0\n",
"class1_as_3 = 0\n",
"for row in x1_samples:\n",
" g = classify_data(\n",
" row, \n",
" discriminant_function,\n",
" [mu_est1, mu_est2, mu_est3],\n",
" [cov_est1, cov_est2, cov_est3]\n",
" )\n",
" if g[1] == 2:\n",
" class1_as_2 += 1\n",
" elif g[1] == 3:\n",
" class1_as_3 += 1\n",
" else:\n",
" class1_as_1 += 1\n",
"\n",
"class2_as_1 = 0\n",
"class2_as_2 = 0\n",
"class2_as_3 = 0\n",
"for row in x2_samples:\n",
" g = classify_data(\n",
" row, \n",
" discriminant_function,\n",
" [mu_est1, mu_est2, mu_est3],\n",
" [cov_est1, cov_est2, cov_est3]\n",
" )\n",
" if g[1] == 2:\n",
" class2_as_2 += 1\n",
" elif g[1] == 3:\n",
" class2_as_3 += 1\n",
" else:\n",
" class2_as_1 += 1\n",
"\n",
"class3_as_1 = 0\n",
"class3_as_2 = 0\n",
"class3_as_3 = 0\n",
"for row in x3_samples:\n",
" g = classify_data(\n",
" row, \n",
" discriminant_function,\n",
" [mu_est1, mu_est2, mu_est3],\n",
" [cov_est1, cov_est2, cov_est3]\n",
" )\n",
" if g[1] == 2:\n",
" class3_as_2 += 1\n",
" elif g[1] == 3:\n",
" class3_as_3 += 1\n",
" else:\n",
" class3_as_1 += 1"
],
"language": "python",
"metadata": {},
"outputs": [],
"prompt_number": 87
},
{
"cell_type": "code",
"collapsed": false,
"input": [
"import prettytable\n",
"\n",
"confusion_mat = prettytable.PrettyTable([\"sample dataset\", \"w1 (predicted)\", \"w2 (predicted)\", \"w3 (predicted)\"])\n",
"confusion_mat.add_row([\"w1 (actual)\",class1_as_1, class1_as_2, class1_as_3])\n",
"confusion_mat.add_row([\"w2 (actual)\",class2_as_1, class2_as_2, class2_as_3])\n",
"confusion_mat.add_row([\"w3 (actual)\",class3_as_1, class3_as_2, class3_as_3])\n",
"print(confusion_mat)\n",
"misclass = x1_samples.shape[0]*3 - class1_as_1 - class2_as_2 - class3_as_3\n",
"bayes_err = misclass / (len(x1_samples)*3)\n",
"print('Empirical Error: {:.2f} ({:.2f}%)'.format(bayes_err, bayes_err * 100))"
],
"language": "python",
"metadata": {},
"outputs": [
{
"output_type": "stream",
"stream": "stdout",
"text": [
"+----------------+----------------+----------------+----------------+\n",
"| sample dataset | w1 (predicted) | w2 (predicted) | w3 (predicted) |\n",
"+----------------+----------------+----------------+----------------+\n",
"| w1 (actual) | 96 | 1 | 3 |\n",
"| w2 (actual) | 2 | 94 | 4 |\n",
"| w3 (actual) | 1 | 3 | 96 |\n",
"+----------------+----------------+----------------+----------------+\n",
"Empirical Error: 0.05 (4.67%)\n"
]
}
],
"prompt_number": 89
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"
\n",
"
\n",
"\n",
"## Conclusion"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"I would claim that the results look pretty good! The error rate on our random dataset increased by just 0.67% (from 4.00% to 4.67%) when we estimated $\\pmb \\mu$ and $\\pmb \\Sigma$ using MLE. \n",
"In a real application of course, we would have an separate training dataset to derive and estimate the parameters, and a test data set for calculating the error rate. However, I ommitted the usage of to separate datasets here for the sake of brevity."
]
}
],
"metadata": {}
}
]
}