{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "[Sebastian Raschka](http://www.sebastianraschka.com) \n", "- Link to the containing GitHub Repository: [https://github.com/rasbt/pattern_classification](https://github.com/rasbt/pattern_classification) \n", "- Link to this IPython Notebook on GitHub: [linear_discriminant_analysis.ipynb](https://github.com/rasbt/pattern_classification/blob/master/dimensionality_reduction/projection/linear_discriminant_analysis.ipynb)" ] }, { "cell_type": "code", "execution_count": 1, "metadata": { "collapsed": false }, "outputs": [], "source": [ "%load_ext watermark" ] }, { "cell_type": "code", "execution_count": 2, "metadata": { "collapsed": false }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "last updated: 2017-01-06 \n", "\n", "CPython 3.5.2\n", "IPython 5.1.0\n", "\n", "pandas 0.19.1\n", "scikit-learn 0.18.1\n", "numpy 1.11.3\n", "matplotlib 2.0.0rc2\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "/Users/Sebastian/miniconda3/lib/python3.5/site-packages/watermark/watermark.py:143: DeprecationWarning: Importing scikit-learn as `scikit-learn` has been depracated and will not be supported anymore in v1.7.0. Please use the package name `sklearn` instead.\n", " DeprecationWarning)\n" ] } ], "source": [ "%watermark -v -d -u -p pandas,scikit-learn,numpy,matplotlib " ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "[More information](http://nbviewer.ipython.org/github/rasbt/python_reference/blob/master/ipython_magic/watermark.ipynb) about the `watermark` magic command extension." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "
\n", "I would be happy to hear your comments and suggestions. \n", "Please feel free to drop me a note via\n", "[twitter](https://twitter.com/rasbt), [email](mailto:bluewoodtree@gmail.com), or [google+](https://plus.google.com/+SebastianRaschka).\n", "
" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "# Linear Discriminant Analysis bit by bit" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "
\n", "
" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Sections" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "- [Introduction](#Introduction)\n", " - [Principal Component Analysis vs. Linear Discriminant Analysis](#Principal-Component-Analysis-vs.-Linear-Discriminant-Analysis)\n", " - [What is a \"good\" feature subspace?](#What-is-a-\"good\"-feature-subspace?)\n", " - [Summarizing the LDA approach in 5 steps](#Summarizing-the-LDA-approach-in-6-steps)\n", "- [Preparing the sample data set](#Preparing-the-sample-data-set)\n", " - [About the Iris dataset](#About-the-Iris-dataset)\n", " - [Reading in the dataset](#Reading-in-the-dataset)\n", " - [Histograms and feature selection](#Histograms-and-feature-selection)\n", " - [Normality assumptions](#normality-assumptions)\n", "- [LDA in 5 steps](#LDA-in-5-steps)\n", " - [Step 1: Computing the d-dimensional mean vectors](#Step-1:-Computing-the-d-dimensional-mean-vectors)\n", " - [Step 2: Computing the Scatter Matrices](#Step-2:-Computing-the-Scatter-Matrices)\n", " - [Step 3: Solving the generalized eigenvalue problem for the matrix $S_{W}^{-1}S_B$](#Step-3:-Solving-the-generalized-eigenvalue-problem-for-the-matrix-$S_{W}^{-1}S_B$)\n", " - [Step 4: Selecting linear discriminants for the new feature subspace](#Step-4:-Selecting-linear-discriminants-for-the-new-feature-subspace)\n", " - [Step 5: Transforming the samples onto the new subspace](#Step-5:-Transforming-the-samples-onto-the-new-subspace)\n", "- [A comparison of PCA and LDA](#A-comparison-of-PCA-and-LDA)\n", "- [LDA via scikit-learn](#LDA-via-scikit-learn)\n", "- [A Note About Standardization](#A-Note-About-Standardization)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "
\n", "
" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "# Introduction" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "[[back to top](#Sections)]" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Linear Discriminant Analysis (LDA) is most commonly used as dimensionality reduction technique in the pre-processing step for pattern-classification and machine learning applications. \n", "The goal is to project a dataset onto a lower-dimensional space with good class-separability in order avoid overfitting (\"curse of dimensionality\") and also reduce computational costs.\n", "\n", "Ronald A. Fisher formulated the *Linear Discriminant* in 1936 ([The Use of Multiple Measurements in Taxonomic Problems](http://onlinelibrary.wiley.com/doi/10.1111/j.1469-1809.1936.tb02137.x/abstract)), and it also has some practical uses as classifier. The original Linear discriminant was described for a 2-class problem, and it was then later generalized as \"multi-class Linear Discriminant Analysis\" or \"Multiple Discriminant Analysis\" by C. R. Rao in 1948 ([The utilization of multiple measurements in problems of biological classification](http://www.jstor.org/stable/2983775))\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "**The general LDA approach is very similar to a Principal Component Analysis (for more information about the PCA, see the previous article [Implementing a Principal Component Analysis (PCA) in Python step by step](http://sebastianraschka.com/Articles/2014_pca_step_by_step.html)), but in addition to finding the component axes that maximize the variance of our data (PCA), we are additionally interested in the axes that maximize the separation between multiple classes (LDA).**" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "So, in a nutshell, often the goal of an LDA is to project a feature space (a dataset n-dimensional samples) onto a smaller subspace $k$ (where $k \\leq n-1$) while maintaining the class-discriminatory information. \n", "In general, dimensionality reduction does not only help reducing computational costs for a given classification task, but it can also be helpful to avoid overfitting by minimizing the error in parameter estimation (\"curse of dimensionality\")." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "
\n", "
" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Principal Component Analysis vs. Linear Discriminant Analysis" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "[[back to top](#Sections)]" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Both Linear Discriminant Analysis (LDA) and Principal Component Analysis (PCA) are linear transformation techniques that are commonly used for dimensionality reduction. PCA can be described as an \"unsupervised\" algorithm, since it \"ignores\" class labels and its goal is to find the directions (the so-called principal components) that maximize the variance in a dataset.\n", "In contrast to PCA, LDA is \"supervised\" and computes the directions (\"linear discriminants\") that will represent the axes that that maximize the separation between multiple classes.\n", "\n", "Although it might sound intuitive that LDA is superior to PCA for a multi-class classification task where the class labels are known, this might not always the case. \n", "For example, comparisons between classification accuracies for image recognition after using PCA or LDA show that PCA tends to outperform LDA if the number of samples per class is relatively small ([PCA vs. LDA](http://ieeexplore.ieee.org/xpl/articleDetails.jsp?arnumber=908974), A.M. Martinez et al., 2001).\n", "In practice, it is also not uncommon to use both LDA and PCA in combination: E.g., PCA for dimensionality reduction followed by an LDA.\n", "
\n", "
\n", "\n", "\n", "\n", "![](../../Images/lda_1.png)\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "
\n", "
" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## What is a \"good\" feature subspace?" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "[[back to top](#Sections)]" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Let's assume that our goal is to reduce the dimensions of a $d$-dimensional dataset by projecting it onto a $(k)$-dimensional subspace (where $k\\;<\\;d$). \n", "So, how do we know what size we should choose for $k$ ($k$ = the number of dimensions of the new feature subspace), and how do we know if we have a feature space that represents our data \"well\"? \n", "\n", "Later, we will compute eigenvectors (the components) from our data set and collect them in a so-called scatter-matrices (i.e., the in-between-class scatter matrix and within-class scatter matrix). \n", "Each of these eigenvectors is associated with an eigenvalue, which tells us about the \"length\" or \"magnitude\" of the eigenvectors. \n", "\n", "If we would observe that all eigenvalues have a similar magnitude, then this may be a good indicator that our data is already projected on a \"good\" feature space. \n", "\n", "And in the other scenario, if some of the eigenvalues are much much larger than others, we might be interested in keeping only those eigenvectors with the highest eigenvalues, since they contain more information about our data distribution. Vice versa, eigenvalues that are close to 0 are less informative and we might consider dropping those for constructing the new feature subspace." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "
\n", "
" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Summarizing the LDA approach in 5 steps" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "[[back to top](#Sections)]" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Listed below are the 5 general steps for performing a linear discriminant analysis; we will explore them in more detail in the following sections.\n", "\n", "1. Compute the $d$-dimensional mean vectors for the different classes from the dataset.\n", "2. Compute the scatter matrices (in-between-class and within-class scatter matrix).\n", "3. Compute the eigenvectors ($\\pmb e_1, \\; \\pmb e_2, \\; ..., \\; \\pmb e_d$) and corresponding eigenvalues ($\\pmb \\lambda_1, \\; \\pmb \\lambda_2, \\; ..., \\; \\pmb \\lambda_d$) for the scatter matrices.\n", "4. Sort the eigenvectors by decreasing eigenvalues and choose $k$ eigenvectors with the largest eigenvalues to form a $d \\times k$ dimensional matrix $\\pmb W\\;$ (where every column represents an eigenvector).\n", "5. Use this $d \\times k$ eigenvector matrix to transform the samples onto the new subspace. This can be summarized by the matrix multiplication $\\pmb Y = \\pmb X \\times \\pmb W$ (where $\\pmb X$ is a $n \\times d$-dimensional matrix representing the $n$ samples, and $\\pmb y$ are the transformed $n \\times k$-dimensional samples in the new subspace)." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "\n", "
\n", "
" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "# Preparing the sample data set" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "[[back to top](#Sections)]" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "\n", "
\n", "
" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## About the Iris dataset" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "[[back to top](#Sections)]" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "For the following tutorial, we will be working with the famous \"Iris\" dataset that has been deposited on the UCI machine learning repository \n", "(https://archive.ics.uci.edu/ml/datasets/Iris).\n", "\n", "\n", "\n", "\n", "**Reference:**\n", "Bache, K. & Lichman, M. (2013). UCI Machine Learning Repository. Irvine, CA: University of California, School of Information and Computer Science." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The iris dataset contains measurements for 150 iris flowers from three different species.\n", "\n", "The three classes in the Iris dataset:\n", "\n", "1. Iris-setosa (n=50)\n", "2. Iris-versicolor (n=50)\n", "3. Iris-virginica (n=50)\n", "\n", "The four features of the Iris dataset:\n", "\n", "1.\tsepal length in cm\n", "2.\tsepal width in cm\n", "3.\tpetal length in cm\n", "4.\tpetal width in cm\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "![](../../Images/iris_petal_sepal.png)" ] }, { "cell_type": "code", "execution_count": 3, "metadata": { "collapsed": false }, "outputs": [], "source": [ "feature_dict = {i:label for i,label in zip(\n", " range(4),\n", " ('sepal length in cm', \n", " 'sepal width in cm', \n", " 'petal length in cm', \n", " 'petal width in cm', ))}" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "\n", "
\n", "
" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Reading in the dataset" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "[[back to top](#Sections)]" ] }, { "cell_type": "code", "execution_count": 4, "metadata": { "collapsed": false }, "outputs": [ { "data": { "text/html": [ "
\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
sepal length in cmsepal width in cmpetal length in cmpetal width in cmclass label
1456.73.05.22.3Iris-virginica
1466.32.55.01.9Iris-virginica
1476.53.05.22.0Iris-virginica
1486.23.45.42.3Iris-virginica
1495.93.05.11.8Iris-virginica
\n", "
" ], "text/plain": [ " sepal length in cm sepal width in cm petal length in cm \\\n", "145 6.7 3.0 5.2 \n", "146 6.3 2.5 5.0 \n", "147 6.5 3.0 5.2 \n", "148 6.2 3.4 5.4 \n", "149 5.9 3.0 5.1 \n", "\n", " petal width in cm class label \n", "145 2.3 Iris-virginica \n", "146 1.9 Iris-virginica \n", "147 2.0 Iris-virginica \n", "148 2.3 Iris-virginica \n", "149 1.8 Iris-virginica " ] }, "execution_count": 4, "metadata": {}, "output_type": "execute_result" } ], "source": [ "import pandas as pd\n", "\n", "df = pd.io.parsers.read_csv(\n", " filepath_or_buffer='https://archive.ics.uci.edu/ml/machine-learning-databases/iris/iris.data', \n", " header=None, \n", " sep=',', \n", " )\n", "df.columns = [l for i,l in sorted(feature_dict.items())] + ['class label']\n", "df.dropna(how=\"all\", inplace=True) # to drop the empty line at file-end\n", "\n", "df.tail()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "$\\pmb X = \\begin{bmatrix} x_{1_{\\text{sepal length}}} & x_{1_{\\text{sepal width}}} & x_{1_{\\text{petal length}}} & x_{1_{\\text{petal width}}}\\\\ \n", "x_{2_{\\text{sepal length}}} & x_{2_{\\text{sepal width}}} & x_{2_{\\text{petal length}}} & x_{2_{\\text{petal width}}}\\\\ \n", "... \\\\\n", "x_{150_{\\text{sepal length}}} & x_{150_{\\text{sepal width}}} & x_{150_{\\text{petal length}}} & x_{150_{\\text{petal width}}}\\\\ \n", "\\end{bmatrix}, \\;\\;\n", "\\pmb y = \\begin{bmatrix} \\omega_{\\text{setosa}}\\\\ \n", "\\omega_{\\text{setosa}}\\\\ \n", "... \\\\\n", "\\omega_{\\text{virginica}}\\end{bmatrix}$" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "\n", "
\n", "
" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Since it is more convenient to work with numerical values, we will use the `LabelEncode` from the `scikit-learn` library to convert the class labels into numbers: `1, 2, and 3`." ] }, { "cell_type": "code", "execution_count": 5, "metadata": { "collapsed": false }, "outputs": [], "source": [ "from sklearn.preprocessing import LabelEncoder\n", "\n", "X = df[[0,1,2,3]].values \n", "y = df['class label'].values \n", "\n", "enc = LabelEncoder()\n", "label_encoder = enc.fit(y)\n", "y = label_encoder.transform(y) + 1\n", "\n", "label_dict = {1: 'Setosa', 2: 'Versicolor', 3:'Virginica'}" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "$\\pmb y = \\begin{bmatrix}{\\text{setosa}}\\\\ \n", "{\\text{setosa}}\\\\ \n", "... \\\\\n", "{\\text{virginica}}\\end{bmatrix} \\quad \\Rightarrow\n", "\\begin{bmatrix} {\\text{1}}\\\\ \n", "{\\text{1}}\\\\ \n", "... \\\\\n", "{\\text{3}}\\end{bmatrix}$\n", "\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "\n", "
\n", "
" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Histograms and feature selection" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "[[back to top](#Sections)]" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Just to get a rough idea how the samples of our three classes $\\omega_1$, $\\omega_2$ and $\\omega_3$ are distributed, let us visualize the distributions of the four different features in 1-dimensional histograms." ] }, { "cell_type": "code", "execution_count": 6, "metadata": { "collapsed": false }, "outputs": [], "source": [ "%matplotlib inline" ] }, { "cell_type": "code", "execution_count": 7, "metadata": { "collapsed": false }, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAA1gAAAGoCAYAAABbkkSYAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzs3X18VPWZ///XJYQaiCAs0AoiEbFZXdSISBEqGW7E0FU0\n1t64yI0tRbGulbXetKINlchKC9ZafvitsURaV5fWkoCFLKLEVlzsGkSFpSAKsgqCciMCioRcvz/m\nkOZmEobMZGaSvJ+PRx7MnJvP5zqHZK65zvmcc8zdERERERERkdidlOwAREREREREWgoVWCIiIiIi\nInGiAktERERERCROVGCJiIiIiIjEiQosERERERGROFGBJSIiIiIiEicqsKRVMrOxZra8EeuVmdmk\neuadYWYHzKxN7BGKiEhroHwk0vKowJIWycy2mtnI+ua7+5PuPiqefbr7NnfPcPejx4ltopm9FM++\nk8XMZpnZ94LX75pZp2rzhpnZSjP72My2Ji1IEZEkUj5KjOPkozvMbJ2ZfWJmW8zsjuRFKq2BCixp\ndcysbbJjaCoWlsi/64uAcjPrBnzu7h9Xm3cQ+A2gRCYiEoHyUVw1lI8MGA90BnKBW8zs2wmMTVoZ\nFVjS4gVH6FaZ2UNmtgfIr37ULkgCD5nZruBsyxtm1q+BJnsH7X1iZsvNrGvQTqaZ+bGEGfTxTrUj\nZmPN7BzgUeCSYPjGvmDZTma2wMw+DI68TTuWmMysjZnNNrOPgnZuqdVPmZkVmNkq4BDQx8xuMLMN\nQd/vmNmN1fZHyMzeM7M7g23eYWZXm9nXzGyTme0xsx9HsV8N+CdgHTAAeK36fHf/q7v/Fngnqv8o\nEZEWTvkoaflolruvcfcKd98IlABDovk/E2kMFVjSWnyF8Bf97kBBrXmjgKHAl4FTgW8Buxto61+A\nG4K22gE/rL2AmXUAfgmMdvdTgMHAWnffANwE/HcwfOPUYJVHgE5AHyCH8JG2G4J53wNGA9lAf+Dq\nCDGNAyYDpwDvAruAK4COQTsPmVn/ast/CTgZ6AncBzwGXE/4COClwH1m1ifSxpvZ2UEi/hjoGvS1\nCLjSzPaZ2bjIu01ERFA+Smo+CoqxS4H1kdoUiQcVWNJabHf3R4KjV5/WmneEcCL4R8DcfYO772ig\nrfnuviloZyHhRBNJJdDPzNLdfYe7R/wwt/BFyN8CfuTun7j7VmA24SQF8E3gYXd/z933Av8eoZki\nd18fbN8Rd/+Tu7/tYS8CywknlOrbXODuR4CnCSemh4P+1xNOPOdHitfd3woS8S+B2wkPudgE9HX3\nU4OzViIiEpnyUXLzUT7h77/zI+8qkdipwJLW4v/qm+HuLwC/AuYCO83s12bWsYG2Pqj2+hCQEaHN\ng4ST1E3ADjP7k5n9Yz3tdSV85PHdatPeJXw0D6BHrfgjbUuNaWY22sxWB8Mr9gFfC/o5Zne1i5+P\nJfid1eZ/Gmm7grZfDtr8EfBTYD9wDrDezP4QeRNFRCSgfJSkfGRmtxA+I/fP7n44Upsi8aACS1oL\nb3Cm+y/d/SLCY7i/TBxuzODu/+XulwGnAX8jPOwhUiwfET6C17vatDOA94PXO4DTq83rFam7Yy/M\n7AvAM8DPgS8GR/eWEr7IN2buPhjIAt5y907ANODB4GjhtfHoQ0SkBVM+SkI+MrPvAHcDI9z9vXj0\nL1IfFVjS6pnZxWb2FTNLI3znu8+ABm9tG0WbXzSzMcHY98PAgWpt7gRON7N2AMGRu4VAgZmdYma9\ngX8DfhcsvxD4gZn1NLNTgbuO03074AvAh0CFmY0mPK4/nqpfRNwfeLX2AmZ2kpmdDKSF39rJx7ZZ\nRETqUj5qlGjy0VjgAeAyd9eNl6TJqcASCV94+xiwl/BQiN2Ej7bF4iTC48G3A3sIXyh8czDvBcJj\nyj8ws4+Caf9KOJm+A7wE/AfhW5wTxLYceINwElkKVFBP0nX3T4BbCSfCvYQvgl4c4/bUdhGwJnjd\nHyiPsMxQwkM7lhI+Avop4e0QEZHIlI9OXDT5aAbwD8D/BHdMPGBmj8Y5DpEq5t7gmWoRSTHBEcBH\n3b33cRcWERFpIspHIpHpDJZIijOz9OCZIG3NrCfwE8K3oRUREUkY5SOR6OgMlkiKM7P2wIuEb9v7\nKfAn4Afuvj+pgYmISKuifCQSneZSYDWLIEVEpMnE5a5jcaB8JCLSekWVizREUEREREREJE5UYImI\niIiIiMSJCiwREREREZE4aZvsAEREmoNDhw6xb9++ZIfRKqSlpdGlSxfatGmT7FBERFLK0aNH2bNn\nD0eOHEl2KC1eLLlIN7kQEYnCzp076dy5M+3atUt2KC2au3PgwAE+++wzunXrVn2WbnIhIq3ehx9+\nyMknn0xGRgZmqfKx2PLEmos0RFBEJApHjx4lLS0t2WG0eGZGRkaGjs6KiERw5MgRFVcJEGsuUoEl\nIhIlJbTE0H4WEamfPiMTI5b9rAJLRCSFhUIhKioqYm7nySefZNCgQVx66aVMnjy53uWKi4vZs2dP\nzP2JiEjLonwUPd3kQkTkBOXnJ3f9xnj44Yd5+eWXadu2LXv37q13ueLiYvr160eXLl0SGJ2IiDSG\n8lFq0hksEZEUUVlZyaRJk8jJyWH06NE15pWWlpKTk8OAAQNYsGABAHPnzmXQoEEMGzaMNWvWsGjR\nIgYOHMjw4cNZunRpjfUPHjzIK6+8QmVlJZ07dwZg8+bNjBo1ipycHGbMmMG2bdsoLS1l7Nix/Oxn\nP2Pbtm0MHz6cIUOG8OCDD0bsc+3ateTk5DBo0CAeeOCBBOwlERFpaspHsdEZLBGRFFFSUkL37t0p\nLCyksrKyxryhQ4eSm5tLRUUFoVCI8ePHU1JSwsqVK0lPT8fdmTNnDgsXLiQzM5Pad4gtKiqioKCA\nN998kzvvvJMbb7yRe+65h8cff5xevXpx3XXXcdJJJ5Gbm8u0adPo27cv3//+95k+fTqXXnopl19+\nOePGjavT52effUZZWRlmxrBhw5g6dSrp6emJ3G0iIhJnykexUYElIpIiNm3axODBgwE46aSaAwzK\ny8uZPn06R44cYf369QBMnz6dKVOm0K5dO+6//36mTZvGjBkzqKio4J577uHss8+uWv/iiy+muLiY\nQ4cOMWzYMMaOHcvGjRsZN24cAPv27eP999+v0efbb79N//79AbjwwgvZsmVLnT53797N7bffzqFD\nh9i4cSO7du2id+/eTbaPRESk6SkfxUZDBEVEUkRWVharV68GqHPEcNasWRQWFrJixQo6deoEQHZ2\nNkVFRYRCIYqKiujduzeFhYVMnjyZOXPm1Fj/rbfeAqB9+/ZVR/SysrJ46qmnKCsro7y8nIsvvpi0\ntDSOHj0KQJ8+fSgvLwfgtddeIzMzs06f8+bN46677uLFF1+kb9++dY5UiohI86N8FBudwRIRSRFj\nxoxhyZIlDB06lIyMjBrj1vPy8rjqqqvIzs6uGrN+0003sWXLFg4fPsz8+fPJz89n9erVHDhwgNmz\nZ9doe+rUqVUXE1977bVkZGRQUFDAd77zHQ4fPkxaWhrPPPMMl19+OTfffDPf+MY3uOuuu5gwYQKf\nf/45V155JT179mTChAk1+ty2bRu33HIL5557rh7CLCLSQigfxcaaydHGZhGkiLRc27dvp0ePHskO\no9WIsL9T5cEvykcikjTKRYnV2FykIYIiIiIiIiJxogJLREREREQkTlRgiYiIiIiIxIkKLBERERER\nkThRgSUiIiIiIhInKrBERFJYKBSioqIipjZKS0u54447qt7v3r2byy67rFFt3XbbbVXPJYnGV7/6\n1Ub1IyIiqUX5KHp6DpaIyInKz0/u+idoxIgR3HvvvVXvFy9ezJgxY467XmVlJSedVPM43C9+8Yu4\nxxepHxERiYLyUVzFKx+pwJK4iOXvM8F/2yIpq7KyksmTJ/PWW2/Rvn17li1bVjWvtLSUmTNncvDg\nQW699VbGjx/P3Llz+e1vf0t6ejqzZ8/m3XffZebMmWRkZPDDH/6Qr33tawCkpaWRlZXFunXr6Nev\nH8XFxfzqV7/C3bn55pvZuHEj6enp/O53v+P1119nzpw5uDuTJk3iscce4+DBg3Tr1o2FCxcSCoVY\nsWIFH330ETfccAOffvopl1xyCTNnzuTBBx9k8eLFfOELX6CoqIgzzjijKv4VK1Ywbdo0AGbMmMHI\nkSMJhUIMHDiQ7du387vf/S6xO1skjvLL8hu/bqjx64o0FeWj2KjAEhFJESUlJXTv3p3CwkIqKytr\nzBs6dCi5ublUVFQQCoUYP348JSUlrFy5kvT0dNydOXPmsHDhQjIzM6n9EPlrrrmG4uJizjzzTPbu\n3UuvXr1YsmQJZ5xxBvPmzWPZsmU8+uijXHLJJXz++eeUlpayefNmunbtyrPPPlunvZkzZzJ16lRG\njRpFZWUlH3zwAS+88AKrVq3ipZdeYubMmcybN69q+fz8fJYvXw5Abm4uI0eOBCAvL49LLrmkKXan\niIg0kvJRbDQmQ0QkRWzatInBgwcD1BmiUF5ezsiRIxkxYgTr168HYPr06UyZMoXJkyeza9cupk2b\nxowZM5g4cSKbN2+usX5ubi7Lly9n2bJl5ObmArBhwwaefvppQqEQBQUF7NmzB4D+/fsD0LdvX847\n7zzGjh3LQw891GCsW7du5fzzzwdgwIABdfo3Mzp27EjHjh1p06ZN1fSLLrqo8TtMRESahPJRbFRg\niYikiKysLFavXg1Q54jhrFmzKCwsZMWKFXTq1AmA7OxsioqKCIVCFBUV0bt3bwoLC5k8eTJz5syp\nsX779u3p2rUrjzzyCHl5eVX9jR8/nrKyMl566SUeeOAB4O/J9PDhw0ydOpUnn3yS0tJSdu7cWW+s\nmZmZvP766wC8+uqrnHXWWTX6r6ysZP/+/ezfv7/GRcm69kpEJPUoH8VGQwRFRFLEmDFjWLJkCUOH\nDiUjI4OlS5dWzcvLy+Oqq64iOzubzp07A3DTTTexZcsWDh8+zPz588nPz2f16tUcOHCA2bNn12k/\nLy+PgoICzjnnnKr+br31VoYPHw6E78jUsWPHquXfffddvvvd71JRUUGfPn3o3r171by7776bCRMm\nMGPGDAYPHswDDzzAsGHDGDx4MO3ateOJJ56o0fd9993HqFGjcHd++tOfxm+niYhI3CkfxcZqj2OM\nW8NmvwGuAHa5e79gWj7wPeDDYLEfu/vSyC3U0DRBStzoJhfS0m3fvp0ePXokO4xWI8L+tsa2pXwk\nTU03uZBEUS5KrMbmoqYcm1EE5EaY/pC7Zwc/0SQzERGRWBShfCQiIgnSZAWWu/8Z2NNU7YuIiERD\n+UhERBIpGVcX32Jmb5jZb8yscxL6FxERAeUjERFpAokusOYBZwHZwA6g7lVvIiIiTU/5SEREmkRC\nCyx33+nuR929EngMGJjI/kVERED5SEREmk5CCywzO63a2zxgXSL7FxFpbkKhEBUVFTG1UVpayh13\n3FH1fvfu3Vx22WWsXbuWxx9/PKo2ioqKKC8vjzjvRNpJFcpHIiInRvkoek32HCwzewoIAV3N7D3g\nJ0DIzLIJ3+Z2K3BjU/UvItJUYrklMyT+tswjRozg3nvvrXq/ePFixowZQ3Z2NtnZ2TWWraysjPiw\nxYkTJ9bbfqR2UonykYi0VMpHNaVKPmrKuwhe5+6nuXuau5/u7o+7+zh3P8/dz3f3Me6+o6n6FxFp\nbiorK5k0aRI5OTmMHj26xrzS0lJycnIYMGAACxYsAGDu3LkMGjSIYcOGsWbNGhYtWsTAgQMZPnx4\njYdCpqWlkZWVxbp14ZM0xcXFXH311ZSVlTFt2jQALrjgAq6//npmzZrFK6+8Qv/+/bnuuuvo378/\nAPn5+axYsYKysjJGjx7NlVdeyZAhQzhw4ECNdubNm1cV08aNGyPGnWjKRyIiJ0b5KDZNdgZLRERO\nTElJCd27d6ewsJDKysoa84YOHUpubi4VFRWEQiHGjx9PSUkJK1euJD09HXdnzpw5LFy4kMzMTGo/\nRP6aa66huLiYM888k71799KrVy/efvvtqvnvvfceL7/8Mh06dOCKK65g8eLFdO7cmd69e9eJs127\ndpSUlFBQUMDzzz9Pp06dANi1axe///3vWbVqFW3atKGyspJevXrViVtERFKb8lFsknGbdhERiWDT\npk0MHjwYoM6wiPLyckaOHMmIESNYv349ANOnT2fKlClMnjyZXbt2MW3aNGbMmMHEiRPZvHlzjfVz\nc3NZvnw5y5YtIze37jN3s7Ky6NChAwD79+/n9NNPp0OHDpx99tl1lu3Xrx8APXv2ZN++fVXTt2zZ\nQv/+/WnTpk3VNkSKW0REUpvyUWxUYImIpIisrCxWr14NUOeI4axZsygsLGTFihVVR+iys7MpKioi\nFApRVFRE7969KSwsZPLkycyZM6fG+u3bt6dr16488sgj5OXl1em7egLt2LEj27dv59ChQ3USI4CZ\nVb2ufmSyT58+vPbaa1WxV1ZWRoxbRERSm/JRbDREUEQkRYwZM4YlS5YwdOhQMjIyaoxbz8vL46qr\nriI7O5vOncPPxL3pppvYsmULhw8fZv78+eTn57N69WoOHDjA7Nl1H+uUl5dHQUEB55xzToNx3Hvv\nvVx55ZX07duXXr16RR1/t27d+PrXv87gwYNJT0/n0UcfjRi3iIikNuWj2FjtcZEpqlkE2Zrl5ydn\nXZFE2b59Oz169Eh2GAlRUVFB27ZtOXjwIKNGjWLVqlUJjyHC/rb6lk0w5SOpI5Y7uSX6Lm7SvLWm\nXATJz0eNzUUaIigiIjWsWrWKnJwcLr300hrPKxEREUmk5pqPNERQRERqyMnJ4cUXX0x2GCIi0so1\n13ykM1giIlFqJkOqmz3tZxGR+ukzMjFi2c8qsEREotCmTRuOHDmS7DBaPHfnwIEDpKWlJTsUEZGU\nk5aWxoEDB1RkNbFYc5GGCIqIROGUU07ho48+SnYYrUJaWhpdunRJdhgiIimnS5cu7Nmzh08++STZ\nobR4seQiFVgiIlFo37497du3T3YYIiLSirVp04Zu3bolOww5Dg0RFBERERERiRMVWCIiIiIiInGi\nAktERERERCROVGCJiIiIiIjEiQosERERERGROFGBJSIiIiIiEicqsEREREREROJEBZaIiIiIiEic\nqMASERERERGJExVYIiIiIiIicaICS0REREREJE5UYImIiIiIiMSJCiwREREREZE4UYElIiIiIiIS\nJyqwRERERERE4kQFloiIiIiISJyowBIREREREYmTtskOQERERKQ521pU1viVQ/GKQkRShc5giYiI\niIiIxElUBZaZPR/NNBERERERkdaswSGCZnYy0B7oamadAQtmdQR6NHFsIiIiIiIizcrxrsG6EbiN\ncDFVzt8LrP3A3CaMS0REREREpNlpcIiguz/s7mcCP3T3Pu5+ZvBzgbv/qqF1zew3ZrbLzNZVm9bF\nzJ4zs7eCfzvHaTtEREQiUj4SEZFEiuoaLHd/xMwGm9m/mNn4Yz/HWa0IyK017W7geXc/G3g+eC8i\nItKUilA+EhGRBIn2Jhe/BX4OfBW4OPgZ0NA67v5nYE+tyVcBTwSvnwCuPpFgRURETpTykYiIJFK0\nz8EaAJzr7h5jf1909x0A7r7DzLrH2J5I4+TnJ2ddEUkVykciItIkon0O1jrgS00ZiIiIiIiISHMX\n7RmsrsD/mtlfgcPHJrr7mBPsb6eZnRYcLTwN2HWC64uIiMSD8pGIiDSJaAus/Dj1txiYAPx78G9J\nnNoVERE5EcpHIiLSJKIqsNz9xRNt2MyeAkKEH1L8HvATwolsoZl9F9gGfONE2xURETkRykciIpJI\nURVYZvYJcOwGF+2ANOCgu3esbx13v66eWSNOKEIREZEYKB+JiEgiRXsG65Tq783samBQk0QkIiIi\nIiLSTEV7F8Ea3L0YuDzOsYiIiIiIiDRr0Q4RvKba25MIPxfrsyaJSERERESaTGMf56jHQIpEJ9q7\nCF5Z7XUFsBW4Ku7RiIiIiIiINGPRXoN1Q1MHIiIiIiIi0txFdQ2WmZ1uZovMbJeZ7TSzZ8zs9KYO\nTkREREREpDmJ9iYX8wk/lLEH0BNYEkwTERERERGRQLQFVjd3n+/uFcFPEdCtCeMSERERERFpdqIt\nsD4ys+vNrE3wcz2wuykDExERERERaW6iLbC+A3wT+ADYAVwL6MYXIiIiIiIi1UR7m/b7gQnuvhfA\nzLoAPydceIm0Lsl4gEgj180PxdBlqHF9iogkU35ZfuPW02fecSUhjYk0S9GewTr/WHEF4O57gAub\nJiQREREREZHmKdoC6yQz63zsTXAGK9qzXyIiIiIiIq1CtEXSbOBlM/sD4ISvxyposqhERERERESa\noagKLHdfYGavAsMBA65x9/9t0shERERERESamaiH+QUFlYoqERERERGRekR7DZaIiIiIiIgchwos\nERERERGROFGBJSIiIiIiEie61XoLlIzn4IrEQ2MfEAp6SKhIymluT6VVEhSRONEZLBERERERkThR\ngSUiIiIiIhInKrBERERERETiRAWWiIiIiIhInKjAEhERERERiRMVWCIiIiIiInGiAktERERERCRO\n9BwsERERibt8ymJYNwZlje03FEuvIiJVdAZLREREREQkTlRgiYiIiIiIxIkKLBERERERkThRgSUi\nIiIiIhInSbnJhZltBT4BjgIV7j4gGXGIiEjrpnwkIiLxlsy7CA5z94+S2L+IiAgoH4mISBxpiKCI\niIiIiEicJOsMlgPLzcyB/+fuv05SHCkrPz/ZEaS+WPZRDKs2L41+Hgzkx7CX8kONX1ckwZSPUlEr\nSoKhRm5rWSvaRyLNTbIKrCHuvt3MugPPmdnf3P3PSYpFRERaL+UjERGJq6QMEXT37cG/u4BFwMBk\nxCEiIq2b8pGIiMRbwgssM+tgZqccew2MAtYlOg4REWndlI9ERKQpJGOI4BeBRWZ2rP//cPfSJMQh\nIiKtm/KRiIjEXcILLHd/B7gg0f2KiIhUp3wkIiJNQbdpFxERERERiRMVWCIiIiIiInGSrNu0SwpK\n1iM1mtujPPIpa+R6ie8zWfLL8hu3YgzP7SLU+FUbq9HbSXKeFdbYePVcM0m05vaZJ02rsd8Tmtv3\nC2k5dAZLREREREQkTlRgiYiIiIiIxIkKLBERERERkThRgSUiIiIiIhInKrBERERERETiRAWWiIiI\niIhInKjAEhERERERiRM9B6uJtZZnMIRieB5QWTN7xs7WrcmOoBmI5XlWjaRnPIk0oLUko2Yoc2tZ\no9aL5Tl8sT2ZsZE9Jr5LkaTRGSwREREREZE4UYElIiIiIiISJyqwRERERERE4kQFloiIiIiISJyo\nwBIREREREYkTFVgiIiIiIiJxogJLREREREQkTlRgiYiIiIiIxIkeNCytUkzPwc1s5HrN7SmLSXhY\nsIgcR3P7HGlG8ilLdggnJLaP6PxGrRVKwgOKAcoa2W9+Ev5e9CcqoDNYIiIiIiIicaMCS0RERERE\nJE5UYImIiIiIiMSJCiwREREREZE4UYElIiIiIiISJyqwRERERERE4kQFloiIiIiISJyYuyc7hmgk\nNcjW8kyDrVtDjV53YgzrJkNRZlnC+8zMTHiX0tRCoWRHkPLyQ/nxasri1VCMYs9HMSSVxj6rKVQW\nanSfjf01b27Pldq6NdkRpL6tmaFGrxvLM7Qa+xysZD23q7Fi+b7Z2HVby3fcOIoqF+kMloiIiIiI\nSJyowBIREREREYkTFVgiIiIiIiJxogJLREREREQkTpJSYJlZrpltNLPNZnZ3MmIQERFRPhIRkXhL\neIFlZm2AucBo4FzgOjM7N9FxiIhI66Z8JCIiTSEZZ7AGApvd/R13/xx4GrgqCXGIiEjrpnwkIiJx\nl/DnYJnZtUCuu08K3o8DvuLutyQ0EBERadWUj0REpCkk4wxWpAd0NYunHYuISIuifCQiInGXjALr\nPaBXtfenA9uTEIeIiLRuykciIhJ3ySiw/gc428zONLN2wLeBxUmIQ0REWjflIxERibu2ie7Q3SvM\n7Bbgv4A2wG/cfX2i4xARkdZN+UhERJpCwm9yISIiIiIi0lIl5UHDIiIiIiIiLZEKLBERERERkThp\nkQWWmbUxs9fM7NkI875gZv9pZpvN7BUzy0x8hHViaijeiWb2oZmtDX4mJSPGavFsNbM3g1hejTDf\nzOyXwf59w8z6JyPOavEcL96QmX1cbf/el4w4q8Vzqpn9wcz+ZmYbzOySWvNTZv9GEWvK7Fszy6oW\nx1oz229mt9VaJpX2bTTxpsz+DeKZambrzWydmT1lZifXmp9yn71Nycx6mdnK4G9jvZn9IMIyKfM7\n11hRbmdK/a42lpmdbGZ/NbPXg22dHmGZZv97HuV2ptR3k1hYM/vOGIvjbGuL+D+N4ntfQj53E36T\niwT5AbAB6Bhh3neBve7e18y+DTwIfCuRwUXQULwA/5liD74c5u4f1TNvNHB28PMVYF7wbzI1FC/A\nX9z9ioRF07CHgVJ3v9bCdzVrX2t+Ku3f48UKKbJv3X0jkA3hBAO8DyyqtVjK7Nso44UU2b9m1hO4\nFTjX3T81s4WE78hXVG2xVPzsbUoVwO3uvsbMTgHKzew5d//fasukzO9cDKLZTkiR39UYHQaGu/sB\nM0sDXjKzZe6+utoyLeH3PJrthNT7btJYze07Yyya2/fNxkr699QWdwbLzE4H/hkorGeRq4Angtd/\nAEaYWaSHTSZEFPE2N1cBCzxsNXCqmZ2W7KCaAzPrCAwFHgdw98/dfV+txVJi/0YZa6oaAbzt7u/W\nmp4S+zaC+uJNNW2BdDNrS7jYrv08qZT67G1q7r7D3dcErz8h/KWmZ63FUvV3LmpRbmeLEPw/HQje\npgU/te8U1ux/z6PczhahuX1njEUL/L7ZWAn53G1xBRbwC+BOoLKe+T2B/4PwLXqBj4F/SExoER0v\nXoCvB6cx/2BmvRpYLhEcWG5m5WY2OcL8qv0beI/kJtvjxQtwSTAUYpmZ/VMig6ulD/AhMD84hV9o\nZh1qLZOQEHpLAAAgAElEQVQq+zeaWCF19m113waeijA9VfZtbfXFCymyf939feDnwDZgB/Cxuy+v\ntViqffYmTDCs6ELglVqzUvV3rlEa2E5Ikd/VWAVDrNYCu4Dn3L3e/9Pm/HsexXZCan03aazm9p0x\nFs3t+2ZjpcT31BZVYJnZFcAudy9vaLEI05JyZCbKeJcAme5+PrCCvx9JSZYh7t6f8CnW75vZ0Frz\nU2b/Bo4X7xqgt7tfADwCFCc6wGraAv2Bee5+IXAQuLvWMqmyf6OJNZX2LQDBUMYxwO8jzY4wLalH\nbY8Tb8rsXzPrTPio4JlAD6CDmV1fe7EIq7bIo+LVmVkG8Axwm7vvrz07wirNcp8cZztT5nc1Vu5+\n1N2zgdOBgWbWr9YiLeL/NIrtTLXvJiesuX1njEUz/b7ZWCnxPbVFFVjAEGCMmW0FngaGm9nvai3z\nHtALIBjK0gnYk8ggqzluvO6+290PB28fAy5KbIg1ufv24N9dhK8JGVhrkar9GzidukOFEuZ48br7\n/mNDIdx9KZBmZl0THmjYe8B71Y4U/oFwEVN7mVTYv8eNNcX27TGjgTXuvjPCvFTZt9XVG2+K7d+R\nwBZ3/9DdjwB/BAbXWiaVPnsTIrh+5RngSXf/Y4RFUvF37oQdbztT7Hc1LoIh0WVAbq1ZLer3vL7t\nTLXvJo3U3L4zxqLZfd9srFT5ntqiCix3/5G7n+7umYSH1bzg7rWPoi4GJgSvrw2WScrRiGjirTUu\ndAzh8e1JYWYdgouYCYaDjQLW1VpsMTA+uEvLIMJDhXYkOFQgunjN7EvHxlOb2UDCfxO7Ex0rgLt/\nAPyfmWUFk0YAtS8UT4n9G02sqbRvq7mO+ofbpcS+raXeeFNs/24DBplZ+yCmEdT9rEqZz95ECPbD\n48AGd59Tz2Kp+Dt3QqLZzhT7XW00M+tmZqcGr9MJH1j4W63Fmv3veTTbmUrfTRqruX1njEVz+77Z\nWKn0PbWl3kWwBjP7KfCquy8mnAh+a2abCR+F+HZSg4ugVry3mtkYwndq2gNMTGJoXwQWBXmyLfAf\n7l5qZjcBuPujwFLga8Bm4BBwQ5JihejivRaYYmYVwKfAt5P84fmvwJPB0LB3gBtSeP8eL9aU2rdm\n1h64DLix2rRU3bfRxJsy+9fdXzGzPxAeClYBvAb8url99sbZEGAc8KaFr2UB+DFwBqTm71wjRbOd\nKfO7GqPTgCcsfGfPk4CF7v5sC/w9j2Y7U+m7SVy1wP/PerXA/9OU+Z5qzfMzTkREREREJPW0qCGC\nIiIiIiIiyaQCS0REREREJE5UYImIiIiIiMSJCiwREREREZE4UYElIiIiIiISJyqwROLIzEJm9my0\n0+PQ39Vmdm6192VmNuA46/QIbqktIiItWGNzT0N5onqeMbMfV5ueaWa1nzkUaf2bzGz8icYk0pyo\nwBJp3q4Gzj3uUtW4+3Z3v7aJ4hERkWbuBPLEj4+/SJ22H3X3BY0IS6TZUIElrUrwlO8/mdnrZrbO\nzL4VTL/IzF40s3Iz+69jTzQPjtQ9bGZrg+UHBtMHmtnLZvZa8G/WCcbwGzP7n2D9q4LpE83sj2ZW\namZvmdmsaut818w2mdlfzewxM/uVmQ0m/LT1nwXxnRUs/o1guU1mdmmE/quOMjbUZ611Lg628/Wg\n7VOCdYvN7Dkz22pmt5jZvwXbtNrMukS7T0REWqNk5SQzW2pm5wevXzOz+4LX95vZpFp5It3Mnjaz\nDWa2CEgPpv87kB7E8mTQdJsgR603s+Vmlh6h73wz+2G17XmwoZwVLHenmb0Z7Kd/r7buQ2b2ahDb\nxUE+e8vMZpzgf4VIXKnAktYmF9ju7he4ez+g1MzSgEeAa939IuA3QEG1ddq7ezZwczAP4G/AUHe/\nELgPeOAEYrgHeMHdLwaGES6QOgTzsoFvAecB3zKzXmbWA7gXGAQMAf4RwN1fBhYDd7h7tru/HbTR\n1t0HArcBP4kinjp9Vp9pZu2A/wR+4O4XACOBT4PZ/YBrgIsJ77NDwT75b0BDQEREGpasnPRn4FIz\n6whUEM4tAF8F/lJr2SmEP9vPIZxTLgJw97uBT4P8MzZY9mxgrrv/E7AP+HoU+6DBnGVmowmP1vhK\nkIOqHwj83N0HAI8CJcD3CeeliWb2D1H0LdIk2iY7AJEEexP4uZk9CDzr7n8xs36EP5CfMzOANsCO\naus8BeDufzazjmZ2KnAK8ISZnQ04kHYCMYwCxhw7ggecDJwRvH7e3T8GMLP/BXoDXYEX3X1PMP33\nwJcbaP+Pwb/lQGYU8UTq8/+qzc8Cdrj7/wC4+/5gWYCV7v4J8ImZfQwsCdZ5Ezg/ir5FRFqzZOWk\nvwC3AluAPwGXmVl7INPdN5pZZrVlhwK/DPp8w8zeaKDdLe6+NngdbQ46Xs4aCcx390NBDHuqzVsc\n/PsmsN7ddwCY2TtAL2B3FP2LxJ0KLGlV3H2TmV0EfA2YYWbPA4sIfzBfUt9qEd7fT7i4yAsSUdkJ\nhGHA1919Y42JZl8BDlebdJTw36idQNtUa+PY+tEuX986Rt19EGndymrvK6PsW0Sk1UpiTvofYADw\nDvAc4QN53yNc5ETTZ31q55M6QwQbWKe+nBVNDqqef469Vw6SpNEQQWlVguF2h9z9d8DPgP7ARqCb\nmV0SLJNmZv9UbbVjY+K/CnwcnO3pBLwfzJ94gmH8F/CvFhyaNLMLj7P8X4EcM+tsZm2pOeTiE8JH\nLpvS34AeZnYxQHD9lRKXiEiMkpWT3P1zwiMVvgmsJnxG64fUHR4I4eGEY4M++1FzdMKRYEhjU1oO\nfCc4w4au75XmQAWWtDbnAX81s7WEx3rPCBLNtcCDZvY6sBYYXG2dz8zsNcJjvL8bTJsFzAymn2ix\ncT/h4RtvBBcR39/Qwu7+PuHx9H8FVgFbgY+D2U8DdwQXKZ8VuYXYBPvnW8Ajwf55jvCwRhERiU0y\nc9JfgJ3B0Lu/AKcTucCaB2SY2Qbgp9Q8y/VrwrnsyQjrxYW7lxIeCvhqsJ9+eJxVRJLO3KM96yvS\n+phZGfBDd381yXFkuPuB4MzRIuA37r4omTGJiEhipUpOEpGG6QyWSPOQHxy5W0f4ouTiJMcjIiIi\nIhHoDJaIiIiIiEic6AyWiIiIiIhInKjAEhERERERiRMVWCIiIiIiInGiAktERERERCROVGCJiIiI\niIjEiQosERERERGROFGBJSIiIiIiEicqsEREREREROJEBZaIiIiIiEicqMASERERERGJExVY0iqZ\n2VgzW96I9crMbFI9884wswNm1ib2CEVEpDVQPhJpeVRgSYtkZlvNbGR98939SXcfFc8+3X2bu2e4\n+9HjxDbRzF6KZ9/JYmazzOx7wet3zaxTtXm3mdk7ZrbfzLab2UNm1jZ50YqIJJ7yUWI0lI+qLdPO\nzP5mZu8lPkJpTVRgSavTkr/kW1gi/64vAsrNrBvwubt/XG3eEqC/u3cE+gEXALcmMDYRkZSmfBRX\nDeWjY+4AdiUwJmmlVGBJixccoVsVnEHZA+RXP2oXJIGHzGyXmX1sZm+YWb8GmuwdtPeJmS03s65B\nO5lm5scSZtDHO8FyW4JhIOcAjwKXBMM39gXLdjKzBWb2YXDkbdqxxGRmbcxstpl9FLRzS61+ysys\nwMxWAYeAPmZ2g5ltCPp+x8xurLY/Qmb2npndGWzzDjO72sy+ZmabzGyPmf04iv1qwD8B64ABwGvV\n57v72+6+79jiQCXQ93jtioi0VMpHyclHwTJnAtcDM4/XnkisVGBJa/EV4B2gO1BQa94oYCjwZeBU\n4FvA7gba+hfghqCtdsAPay9gZh2AXwKj3f0UYDCw1t03ADcB/x0M3zg1WOURoBPQB8gBxgd9AHwP\nGA1kA/2BqyPENA6YDJwCvEv4CN0VQMegnYfMrH+15b8EnAz0BO4DHiOceC4CLgXuM7M+kTbezM4O\nEvHHQNegr0XAlWa2z8zGVVv2X8xsP/AR4TNY/y9SmyIirYjyURLyUbBdPwY+jdSWSDypwJLWYru7\nP+LuFe5e+8P1COFE8I+AufsGd9/RQFvz3X1T0M5Cwokmkkqgn5mlu/sOd18faSELX4T8LeBH7v6J\nu28FZhNOUgDfBB529/fcfS/w7xGaKXL39cH2HXH3PwVnkNzdXwSWE05U1be5wN2PAE8TTkwPB/2v\nB9YD50eK193fChLxL4Hbgc7AJqCvu5/q7r+ttux/BEMEv0z4SOnOevaViEhroXyU4HxkZnlAW3df\nVM/+EYkrFVjSWvxffTPc/QXgV8BcYKeZ/drMOjbQ1gfVXh8CMiK0eZBwkroJ2GFmfzKzf6ynva6E\njzy+W23au4SP5gH0qBV/pG2pMc3MRpvZ6mB4xT7ga0E/x+yudvHzsQRfvfj5NNJ2BW2/HLT5I+Cn\nwH7gHGC9mf0h0jru/hbhJPn/RZovItKKKB8lMB8FZ/BmAf8aqQ2RpqACS1oLb3Cm+y/d/SLCY7i/\nTPhC2Ng6dP8vd78MOA34G+FhD5Fi+YjwEbze1aadAbwfvN4BnF5tXq9I3R17YWZfAJ4Bfg58MTi6\nt5TwdVAxc/fBQBbwlrt3AqYBDwZHC69tYNW2wFnxiEFEpBlTPkpsPjobyAT+YmYfAH8ETjOzD8ws\nMx5xiNSmAktaPTO72My+YmZpwEHgM6DBW9tG0eYXzWxMcOTsMHCgWps7gdPNrB1AcORuIVBgZqeY\nWW/g34DfBcsvBH5gZj3N7FTgruN03w74AvAhUGFmowmP64+n6hcR9wderb2AmU0ys+7B63MJH2F8\nPs5xiIi0GMpHjXK8fLSOcCGYHfxMIrzd2TRwNlEkFiqwRMIX3j4G7CU8FGI34aNtsTiJ8Hjw7cAe\nwhcK3xzMe4HwcLkPzOyjYNq/Ek6m7wAvAf8B/CaY9xjhMetvEE4iS4EK6km67v4J4duhLwy26V+A\nxTFuT20XAWuC1/2B8gjLDAHeNLODQcxLCV9gLCIikSkfnbgG81FwLdgHx34I74PK4H1MxatIfcy9\nwTPVIpJigiOAj7p77+MuLCIi0kSUj0Qi0xkskRRnZunBM0HamllP4CeEb0MrIiKSMMpHItHRGSyR\nFGdm7YEXCd+291PgT8AP3H1/UgMTEZFWRflIJDrNpcBqFkGKiEiTictdx+JA+UhEpPWKKhdpiKCI\niIiIiEicqMASERERERGJExVYIiIiIiIicdI22QGIiDQHhw4dYt++fckOo1VIS0ujS5cutGnTJi7t\nmdnJwJ8JP/C0LfAHd/+JmZ0JPA10IfwcnXHu/nlcOhURaQJHjx5lz549HDlyJNmhtHix5CLd5EJE\nJAo7d+6kc+fOtGvXLtmhtGjuzoEDB/jss8/o1q1b9VmNvsmFmRnQwd0PmFka4Yen/gD4N+CP7v60\nmT0KvO7u844XYmPjEBGJ1YcffsjJJ59MRkYG4Y82aQqx5iINERQRicLRo0dJS0tLdhgtnpmRkZER\n16OzHnYgeJsW/DgwHPhDMP0J4Oq4dSoi0gSOHDmi4ioBYs1FKrBERKKkhJYYTbGfzayNma0FdgHP\nAW8D+9y9IljkPaBn3DsWEYkz5aLEiGU/q8ASEUlhoVCIioqK4y94HE8++SSDBg3i0ksvZfLkyfUu\nV1xczJ49e2LuL9W4+1F3zwZOBwYC50RaLLFRiYg0H8pH0dNNLkRETlB+fnLXb4yHH36Yl19+mbZt\n27J37956lysuLqZfv3506dIlgdEljrvvM7MyYBBwqpm1Dc5inQ5sT2pwIiInSPkoNekMlohIiqis\nrGTSpEnk5OQwevToGvNKS0vJyclhwIABLFiwAIC5c+cyaNAghg0bxpo1a1i0aBEDBw5k+PDhLF26\ntMb6Bw8e5JVXXqGyspLOnTsDsHnzZkaNGkVOTg4zZsxg27ZtlJaWMnbsWH72s5+xbds2hg8fzpAh\nQ3jwwQcj9rl27VpycnIYNGgQDzzwQAL20okzs25mdmrwOh0YCWwAVgLXBotNAEqSE6GISGpRPoqN\nzmCJiKSIkpISunfvTmFhIZWVlTXmDR06lNzcXCoqKgiFQowfP56SkhJWrlxJeno67s6cOXNYuHAh\nmZmZ1L5DbFFREQUFBbz55pvceeed3Hjjjdxzzz08/vjj9OrVi+uuu46TTjqJ3Nxcpk2bRt++ffn+\n97/P9OnTufTSS7n88ssZN25cnT4/++wzysrKMDOGDRvG1KlTSU9PT+Rui8ZpwBNm1obwgcWF7v6s\nmf0v8LSZzQBeAx5PZpAiIqlC+Sg2KrBERFLEpk2bGDx4MAAnnVRzgEF5eTnTp0/nyJEjrF+/HoDp\n06czZcoU2rVrx/3338+0adOYMWMGFRUV3HPPPZx99tlV61988cUUFxdz6NAhhg0bxtixY9m4cSPj\nxo0DYN++fbz//vs1+nz77bfp378/ABdeeCFbtmyp0+fu3bu5/fbbOXToEBs3bmTXrl307t27yfZR\nY7j7G8CFEaa/Q/h6LBERqUb5KDYaIigikiKysrJYvXo1QJ0jhrNmzaKwsJAVK1bQqVMnALKzsykq\nKiIUClFUVETv3r0pLCxk8uTJzJkzp8b6b731FgDt27evOqKXlZXFU089RVlZGeXl5Vx88cWkpaVx\n9OhRAPr06UN5eTkAr732GpmZmXX6nDdvHnfddRcvvvgiffv2rXOkUkREmh/lo9joDJaIyAlqqouC\nx4wZw5IlSxg6dCgZGRk1xq3n5eVx1VVXkZ2dXTVm/aabbmLLli0cPnyY+fPnk5+fz+rVqzlw4ACz\nZ8+u0fbUqVOrLia+9tprycjIoKCggO985zscPnyYtLQ0nnnmGS6//HJuvvlmvvGNb3DXXXcxYcIE\nPv/8c6688kp69uzJhAkTavS5bds2brnlFs4991w9hFlEJMGUj1IzH1lTVXdm1gtYAHwJqAR+7e4P\nm1k+8D3gw2DRH7v70sitVNEhURFJqu3bt9OjR49kh9FqRNjfqfLgF+UjEUka5aLEamwuasozWBXA\n7e6+xsxOAcrN7Llg3kPu/vMm7FtERERERCThmqzAcvcdwI7g9SdmtgHo2VT9iYiItHSNHQ6UjGfd\niIi0Vgm5BsvMMgnfwekVYAhwi5mNB14lfJar/qeMNXNKhiIiIiIirUeT30XQzDKAZ4Db3H0/MA84\nC8gmfIZrdgOri4iIiIiINBtNWmCZWRrh4upJd/8jgLvvdPej7l4JPIaeQSIiIiIiIi1EkxVYZmbA\n48AGd59Tbfpp1RbLA9Y1VQwiIs1dKBSioqIipjZKS0u54447qt7v3r2byy67rFFt3XbbbVXPJYnG\nV7/61Ub1IyIiqUX5KHpNeQ3WEGAc8KaZrQ2m/Ri4zsyyCd/qditwYxPGICISf7FeJJngiyxHjBjB\nvffeW/V+8eLFjBkz5rjrVVZWctJJNY/D/eIXv4h7fJH6ERGRKCgfxVW88lGTZTR3f8ndzd3Pd/fs\n4Gepu49z9/OC6WOCuw2KiLR6lZWVTJo0iZycHEaPHl1jXmlpKTk5OQwYMIAFCxYAMHfuXAYNGsSw\nYcNYs2YNixYtYuDAgQwfPrzGQyHT0tLIyspi3brwgIHi4mKuvvpq3J0pU6YwfPhw/vmf/5m9e/dS\nVlbGmDFjuPLKK1myZAlXXHEFw4YN45vf/Cbw9yOYH3zwAaNHjyYUCvGjH/0IgAcffJAhQ4YwfPhw\ntm3bViP+FStWMGjQIAYNGsSKFSuq2rrzzjsZP3580+xQERFpFOWj2CTkLoIiInJ8JSUldO/encLC\nQiorK2vMGzp0KLm5uVRUVBAKhRg/fjwlJSWsXLmS9PR03J05c+awcOFCMjMzqf0Q+WuuuYbi4mLO\nPPNM9u7dS69evViyZAlnnHEG8+bNY9myZTz66KNccsklfP7555SWlrJ582a6du3Ks88+W6e9mTNn\nMnXqVEaNGkVlZSUffPABL7zwAqtWreKll15i5syZzJs3r2r5/Px8li9fDkBubi4jR44EIC8vj0su\nuaQpdqeIiDSS8lFsNCZDRCRFbNq0icGDBwPUGaJQXl7OyJEjGTFiBOvXrwdg+vTpTJkyhcmTJ7Nr\n1y6mTZvGjBkzmDhxIps3b66xfm5uLsuXL2fZsmXk5uYCsGHDBp5++mlCoRAFBQXs2bMHgP79+wPQ\nt29fzjvvPMaOHctDDz3UYKxbt27l/PPPB2DAgAF1+jczOnbsSMeOHWnTpk3V9IsuuqjxO0xERJqE\n8lFsVGCJiKSIrKwsVq9eDVDniOGsWbMoLCxkxYoVdOrUCYDs7GyKiooIhUIUFRXRu3dvCgsLmTx5\nMnPmzKmxfvv27enatSuPPPIIeXl5Vf2NHz+esrIyXnrpJR544AHg78n08OHDTJ06lSeffJLS0lJ2\n7txZb6yZmZm8/vrrALz66qucddZZNfqvrKxk//797N+/v8ZFybr2SkQk9SgfxUZDBEVETlQTXRQ8\nZswYlixZwtChQ8nIyKgxbj0vL4+rrrqK7OxsOnfuDMBNN93Eli1bOHz4MPPnzyc/P5/Vq1dz4MAB\nZs+u+4jBvLw8CgoKOOecc6r6u/XWWxk+fDgQviNTx44dq5Z/9913+e53v0tFRQV9+vShe/fuVfPu\nvvtuJkyYwIwZMxg8eDAPPPAAw4YNY/DgwbRr144nnniiRt/33Xcfo0aNwt356U9/Gr+dJiLSmikf\npWQ+strjGFNUswgyksb+3if4pi4ichzbt2+nR48eyQ6j1Yiwv62xbZlZL2AB8CWgEvi1uz9sZvnA\n94APg0V/7O5LI7dSJan5SDlFpHVTLkqsxuYincESEZGWrgK43d3XmNkpQLmZPRfMe8jdf57E2ERE\npIVRgSUiIi1a8DiQHcHrT8xsA9AzuVGJiEhLpauLRUSk1TCzTOBC4JVg0i1m9oaZ/cbMOictMBER\naTFUYImISKtgZhnAM8Bt7r4fmAecBWQTPsNV90psERGRE6QCS0REWjwzSyNcXD3p7n8EcPed7n7U\n3SuBx4CByYxRRERaBhVYIiIpLBQKUVFREVMbpaWl3HHHHVXvd+/ezWWXXcbatWt5/PHHo2qjqKiI\n8vLyiPNOpJ1kMDMDHgc2uPucatNPq7ZYHrAu0bGJiDQXykfR000uREROUH5Zfmzrh2Jb/0SNGDGC\ne++9t+r94sWLGTNmDNnZ2WRnZ9dYtrKyMuLDFidOnFhv+5HaSTFDgHHAm2a2Npj2Y+A6M8smfOv1\nrcCNyQlPRKRxlI9qSpV8pDNYIiIporKykkmTJpGTk8Po0aNrzCstLSUnJ4cBAwawYMECAObOncug\nQYMYNmwYa9asYdGiRQwcOJDhw4fXeChkWloaWVlZrFsXPkFTXFzM1VdfTVlZGdOmTQPgggsu4Prr\nr2fWrFm88sor9O/fn+uuu47+/fsDkJ+fz4oVKygrK2P06NFceeWVDBkyhAMHDtRoZ968eVUxbdy4\nMWLciebuL7m7ufv57p4d/Cx193Hufl4wfUxwt0ERkVZP+Sg2OoMlIpIiSkpK6N69O4WFhVRWVtaY\nN3ToUHJzc6moqCAUCjF+/HhKSkpYuXIl6enpuDtz5sxh4cKFZGZmUvsh8tdccw3FxcWceeaZ7N27\nl169evH2229XzX/vvfd4+eWX6dChA1dccQWLFy+mc+fO9O7du06c7dq1o6SkhIKCAp5//nk6deoE\nwK5du/j973/PqlWraNOmDZWVlfTq1atO3CIiktqUj2KjM1giIili06ZNDB48GKDOsIjy8nJGjhzJ\niBEjWL9+PQDTp09nypQpTJ48mV27djFt2jRmzJjBxIkT2bx5c431c3NzWb58OcuWLSM3N7dO31lZ\nWXTo0AGA/fv3c/rpp9OhQwfOPvvsOsv269cPgJ49e7Jv376q6Vu2bKF///60adOmahsixS0iIqlN\n+Sg2KrBERFJEVlYWq1evBqhzxHDWrP+/vfsPtqSs7zz+/jjgwogTICABQdDIGtGNoxKMIDCKGnSV\nkUAU1h+gpoxGV6nSMqxV6l2NKU1FXYM/WOXHoCFGFyUOrlGRFRCy/gBEAUcXgmwkjEwUBPwRZge+\n+8fpi4fhztwzc09333vO+1V16vTp093Pt2/PnKe/3U8/z19yxhln8JWvfOW+K3QrV65kzZo1rFq1\nijVr1rD//vtzxhln8KpXvYr3ve9991t/+fLl7LHHHpx22mkce+yxDyh7uAJdsWIFt9xyC7/85S8f\nUDECDPqMGBi+MvmoRz2Kb3/72/fFfu+9984ZtyRpcbM+WhibCErSNmrroeBjjjmGCy64gCOOOIJd\ndtnlfu3Wjz32WFavXs3KlSvZbbfBeLivfvWr+eEPf8jdd9/N2WefzczMDF//+tf5+c9/znvf+8Ah\nnY499lje9a538djHPnarcbz1rW/l+c9/Po9+9KPZb7/9Ro5/zz335LjjjuPQQw9l55135vTTT58z\nbknSeFgfza3v+iibt4tcpJZEkHOZmel2PUntuOWWW9hnn336DqMTmzZtYocdduAXv/gFz372s7n8\n8ss7j2GOv3e2tGzHeq2PrFOk6TZNdRH0Xx9tb13UWhPBJPsl+WqSdUmuS/KGZv7uSS5Mcn3z7iVN\nSVpELr/8co488kgOP/zw+41XIklSl5ZqfdRmE8FNwBur6qokDwWuTHIhcDJwUVW9O8mpwKnAn7UY\nhyRpGxx55JFccsklfYchSZpyS7U+au0OVlWtr6qrmum7gHXAw4HVwDnNYucAL2grBkkap80f9NX4\nVRUbN27sOwxJWrQ2btz4gK7PNV4LrYs66eQiyQHAE4FvAHvNDuZYVeuTPKyLGCRpIXbZZRd+/OMf\n9x3GVFi2bBm77rpr32FI0qKz6667cvvtt3PPPff0HcrEW0hd1HqClWQX4DPAKVV153B3ipK0VKxY\nsYIVK1b0HYYkaYotX76c5cuX9x2G5tHqOFhJdmSQXJ1bVZ9tZt+aZO/m+72BDW3GIEmSJEldabMX\nwZ8426IAABQOSURBVABnAuuqaniEsbXASc30ScDn2opBkiRJkrrUZhPBw4CXAtckubqZ9xbg3cCn\nk7wS+Gfgj1qMQZIkSZI601qCVVWXseXBuI5qq1xJkiRJ6ksnvQgudTMzfUcgSZIkaSlotZMLSZIk\nSZomJliSpImWZL8kX02yLsl1Sd7QzN89yYVJrm/ed+s7VknS0meCJUmadJuAN1bVY4HfB16b5CDg\nVOCiqjoQuKj5LEnSgphgSZImWlWtr6qrmum7gHXAw4HVwDnNYucAL+gnQknSJDHBkiRNjSQHAE8E\nvgHsVVXrYZCEAQ/rLzJJ0qQwwZIkTYUkuwCfAU6pqjv7jkeSNJlMsCRJEy/JjgySq3Or6rPN7FuT\n7N18vzewoa/4JEmTwwRLkjTRkgQ4E1hXVe8b+motcFIzfRLwua5jkyRNHgcaliRNusOAlwLXJLm6\nmfcW4N3Ap5O8Evhn4I96ik+SNEFMsCRJE62qLgOyha+P6jIWSdLks4mgJEmSJI2JCZYkSZIkjYkJ\nliRJkiSNiQmWJEmSJI3JSAlWkotGmSdJkiRJ02yrvQgm2QlYDuyRZDd+3QvTCmCflmOTJEmSpCVl\nvm7a/wQ4hUEydSW/TrDuBD7UYlySJEmStORstYlgVX2gqh4JvKmqHlVVj2xeT6iqD25t3SRnJdmQ\n5NqheTNJ/iXJ1c3ruWPaD0mSJEnq3UgDDVfVaUkOBQ4YXqeqPr6V1dYAHwQ2X+b9VfVX2xamJEmS\nJC1+IyVYST4B/DZwNXBPM7t4YPJ0n6q6NMkBC4xPkiRJkpaMkRIs4GDgoKqqMZT5uiQvA64A3lhV\nt49hm5IkSZLUu1ETrGuB3wLWL7C8jwDvZHD3653Ae4FXLHCbktowM9PPupIkSUvYqAnWHsD3knwT\nuHt2ZlUdsy2FVdWts9NJPgZ8flvWlyRJkqTFbNQEa2YchSXZu6pm74Idy+DOmCRJkiRNhFF7Ebxk\nWzec5JPAKgaDFN8MvB1YlWQlgyaCNzEYZ0uSJEmSJsKovQjexSApAngwsCPwi6pasaV1qurEOWaf\nuc0RSpIkSdISsdWBhmdV1UOrakXz2gk4Dvhwu6FJkrRwDnwvSerSSAnW5qrq74E/GHMskiS1YQ1w\n9Bzz319VK5vXFzqOSZI0oUZtIviHQx8fxGBcrH9rJSJJksbIge8lSV0atRfB5w9Nb2LQQcXqsUcj\nSVJ3HPhekjR2o/Yi+PK2A5EkqUMOfC9JasVIz2Al2TfJ+c1Dwrcm+UySfdsOTpKkNlTVrVV1T1Xd\nC3wMOKTvmCRJk2HUTi7OBtYC+wAPBy5o5kmStOQk2XvoowPfS5LGZtRnsPasquGEak2SU9oISJKk\ncXLge0lSl0ZNsH6S5CXAJ5vPJwI/bSckSZLGx4HvJUldGrWJ4CuAFwI/BtYDxwN2fCFJkiRJQ0a9\ng/VO4KTZLmyT7A78Ffa4JEmSJEn3GTXB+t3h8UGq6rYkT2wpJkmbm5npdr2ezFw8s/3rrtq+dfso\nU5IkTa5Rmwg+KMlusx+aO1ijJmeSJEmSNBVGTZLeC/xjkvMY9Lj0QuBdrUUlSZKkLVtIC4U+WkUs\nsRYV0kKMlGBV1ceTXAE8Awjwh1X1vVYjkyRJkqQlZuRmfk1CZVIlSZIkSVsw6jNYkiRJkqR5mGBJ\nkiRJ0pjYE6AkSZI0ZHuH8HD4DkGLd7CSnJVkQ5Jrh+btnuTCJNc377ttbRuSJEmStJS02URwDXD0\nZvNOBS6qqgOBi5rPkiRJkjQRWkuwqupS4LbNZq8GzmmmzwFe0Fb5kiRJktS1rju52Kuq1gM07w/r\nuHxJkiRJao2dXEiSJGnRssMJLTVd38G6NcneAM37ho7LlyRNGTtdkiR1qesEay1wUjN9EvC5jsuX\nJE2fNdjpkiSpI2120/5J4H8Dj0lyc5JXAu8GnpXkeuBZzWdJklpjp0uSpC619gxWVZ24ha+OaqtM\nSZJGdL9Ol5LY6ZIkaSzs5EKSJGkhZmb6WVfSotT1M1iSJC0GdrokSWqFCZYkaRrZ6ZIkqRUmWJKk\niWanS5KkLvkMliRpotnpkiSpS97BkiRJkqQxMcGSJEmSpDExwZIkSZKkMTHBkiRJkqQxsZMLSZIk\nqWczF89s/7qrtn9djZ93sCRJkiRpTEywJEmSJGlMTLAkSZIkaUxMsCRJkiRpTOzkQpKkCTczMx1l\nakQeHKlV3sGSJEmSpDHxDpY0ybxKKUmS1CnvYEmSJEnSmPRyByvJTcBdwD3Apqo6uI84JEmSJGmc\n+mwi+PSq+kmP5UuS1Dlb7krSZLOJoCRJkiSNSV93sAr4cpIC/ntVfbSnOCS1YObimb5D6EQf+zmz\nqvsyJ5lN1iVJ49ZXgnVYVd2S5GHAhUm+X1WX9hSLJGm62WRdkjQ2vTQRrKpbmvcNwPnAIX3EIUmS\nJEnj1PkdrCQPAR5UVXc1088G3tF1HJIkYZN19c1eT1ozLc3Vtfj00URwL+D8JLPl/21VfbGHOCRJ\nssm6JGmsOk+wqupG4AldlytJ0uaGm6wnmW2yboIlSdpufY6D1SnvwGts/Mc0v4sv3r71Vq3a7iJt\nCqJtZZN1SVIbpibBkiRpMzZZlySNnQmWJGkq2WRdktSGXrpplyRJkqRJZIIlSZIkSWNiE0FJmhLb\n2xHIzKrtW0+SpGnkHSxJkiRJGhPvYEmSJGleM1y8gHWnQ19DhvRRrq0btsw7WJIkSZI0JiZYkiRJ\nkjQmJliSJEmSNCY+g7VIzcz0s+7U8I+0OF188favu2rVuKJY1Ppq3y9JkkZjgiVJksbOC4WSppVN\nBCVJkiRpTEywJEmSJGlMTLAkSZIkaUx8BktLl430W7WQASU1WRbSsYYDUUqSpo0JliRJ0hTp4wLa\nzMyqzsuclt5ltfjYRFCSJEmSxqSXBCvJ0Ul+kOSGJKf2EYMkSdZHkqRx6zzBSrIM+BDwHOAg4MQk\nB3UdhyRpulkfSZLa0McdrEOAG6rqxqraCPwdsLqHOCRJ0836SJI0dqmqbgtMjgeOrqo/bj6/FHhK\nVb2u00AkSVPN+kiS1IY+7mBljnndZnmSJFkfSZJa0EeCdTOw39DnfYFbeohDkjTdrI8kSWPXR4L1\nLeDAJI9M8mDgBGBtD3FIkqab9ZEkaew6H2i4qjYleR3wJWAZcFZVXdd1HJKk6WZ9JElqQ+edXEiS\nJEnSpOploGFJkiRJmkQmWJIkSZI0JhOfYCU5K8mGJNf2HUubkuyX5KtJ1iW5Lskb+o6pLUl2SvLN\nJN9p9vW/9h1Tm5IsS/LtJJ/vO5Y2JbkpyTVJrk5yRd/xtCXJrknOS/L95v/rU/uOqQ1JHtMcy9nX\nnUlO6TuuLiU5OskPktyQ5NQ5vv93ST7VfP+NJAd0H+XCjbCfJyf516F/C3/cR5wLNd/5RAb+uvk7\nfDfJk7qOcRxG2M9VSe4YOp5v6zrGcRnl3GkSjuuI+7nkj+so54dd/e523slFD9YAHwQ+3nMcbdsE\nvLGqrkryUODKJBdW1ff6DqwFdwPPqKqfJ9kRuCzJP1TV1/sOrCVvANYBK/oOpANPr6qf9B1Eyz4A\nfLGqjm96rlved0BtqKofACthcJEA+Bfg/F6D6lCzzx8CnsWgO/hvJVm72W/yK4Hbq+rRSU4A3gO8\nqPtot9+I+wnwqQkYwHkNWz+feA5wYPN6CvCR5n2pWcP8501fq6rndRNOq0Y5d5qE4zrqOeJSP66j\nnB928rs78XewqupS4La+42hbVa2vqqua6bsYnJA/vN+o2lEDP28+7ti8JrK3liT7Av8ROKPvWLRw\nSVYARwBnAlTVxqr6Wb9RdeIo4J+q6v/2HUiHDgFuqKobq2oj8HfA6s2WWQ2c00yfBxyVZK7Bjxez\nUfZzIoxwPrEa+HhTR30d2DXJ3t1ENz7Tct4EI587LfnjOi3niCOeH3byuzvxCdY0am53PhH4Rr+R\ntKdpNnc1sAG4sKomdV//G/Bm4N6+A+lAAV9OcmWSV/UdTEseBfwrcHbT7POMJA/pO6gOnAB8su8g\nOvZw4EdDn2/mgSc09y1TVZuAO4Df7CS68RllPwGOa5pXnZdkvzm+nwSj/i0mwVObZlj/kORxfQcz\nDls5d5qo4zrPOeKSP64jnB928rtrgjVhkuwCfAY4paru7DuetlTVPVW1EtgXOCTJ4/uOadySPA/Y\nUFVX9h1LRw6rqicxaI7x2iRH9B1QC3YAngR8pKqeCPwCeMAzK5OkaQZ5DPA/+o6lY3NdEd38Suoo\nyyx2o+zDBcABVfW7wFf49dXjSTMJx3MUVwH7V9UTgNOAv+85ngWb59xpYo7rPPs5Ecd1hPPDTo6n\nCdYEadqbfgY4t6o+23c8XWiaV10MHN1zKG04DDgmyU0Mmt08I8nf9BtSe6rqluZ9A4NndQ7pN6JW\n3AzcPHRF7TwGCdckew5wVVXd2ncgHbsZGL5Tsy9wy5aWSbID8BssvaZZ8+5nVf20qu5uPn4MeHJH\nsXVtlGO+5FXVnbPNsKrqC8COSfboOaztNsK500Qc1/n2c9KO61bODzv53TXBmhBN+9EzgXVV9b6+\n42lTkj2T7NpM7ww8E/h+v1GNX1X9l6rat6oOYNDE6n9V1Ut6DqsVSR7SPHhL02Tu2cDE9fxZVT8G\nfpTkMc2so4BJ7Ihm2IlMX/NAgG8BByZ5ZHMX7wRg7WbLrAVOaqaPZ/B/fKldGZ93Pzd7XuUYBs9/\nTKK1wMuaXud+H7ijqtb3HdS4Jfmt2WdWkhzC4Fzyp/1GtX1GPHda8sd1lP2chOM64vlhJ7+7E9+L\nYJJPAquAPZLcDLy9qs7sN6pWHAa8FLimaXsK8JbmKsSk2Rs4p+m96kHAp6tqorswnwJ7Aec3v+07\nAH9bVV/sN6TW/Gfg3OZk9Ebg5T3H05okyxn0LvcnfcfStaralOR1wJeAZcBZVXVdkncAV1TVWgYn\nPJ9IcgODK6gn9Bfx9hlxP1+f5BgGPZndBpzcW8ALMNf5BIOH6Kmq04EvAM8FbgB+yRL9vz3Cfh4P\nvCbJJuBXwAlL8MLArDnPnYBHwEQd11H2cxKO65znh3387mbp/e0kSZIkaXGyiaAkSZIkjYkJliRJ\nkiSNiQmWJEmSJI2JCZYkSZIkjYkJliRJkiSNiQmWNI8kJyfZZ4Tl1iQ5ftT5Y4jrLUPTBySZd9yo\nJK9O8rJxxyJJ6s5C66UR1puzrhiua5KsTPLcoe9mkrxphG1/YXasImlSmWBJ8zsZmLci68Fb5l/k\n/qrq9Kr6eBvBSJI6czIt1ksj1hUrGYwPta3bfm5V/Wz7IpOWBhMsTZXm6tv3k5ybZF2S85rBUEny\n5CSXJLkyyZeS7N1c+TuYwcCwVyfZOcnbknwrybVJPjo78vmI5T+gjGb+xUnek+SbSf5PksOb+cuT\nfDrJ95Kcn+QbSQ5O8m5g5yamc5vNL0vysSTXJflyM4r55uXfd4VxS2XOsc6bk1yT5DtNubPrvj/J\nFc3f8feSfDbJ9Un+fPQjIknTret6KcnDklzZTD8hSSV5RPP5n5p6Z7iueHLz+/8d4LXNvAcD7wBe\n1MTwombzBzX1w41JXr+F8m9Kskez3+tGqLf2auq/7zSvQ4f+Zmua+uvcJM9McnlTDx2yvcdDGgcT\nLE2jxwAfrqrHAncCf5pkR+A04PiqejJwFvCuqjoPuAJ4cVWtrKpfAR+sqt+rqscDOwPPG6XQLZUx\ntMgOVXUIcArw9mbenwK3V9VBwFuBJwNU1anAr5qYXtwseyDwoap6HPAz4LgRwpqrzOGYnwO8AHhK\nVT0B+MuhrzdW1cHA6cDnGFS8jwdOTvKbI5QtSRrorF6qqg3ATklWAIc32zo8yf7Ahqr65WarnA28\nvqkDZrexEXgb8Kkmhk81X/0O8AfAIcDbm33YmlHqrb8GLmnKfxJwXTP/0cB7mzJ/B/hPwNOAN7Ed\nLTykcdqh7wCkHvyoqi5vpv8GeD3wRQbJwYXNhb9lwPotrP/0JG8GlgO7M/ixv2CEch8zTxmfbd6v\nBA5opp8GfACgqq5N8t2tbP+HVXX1HNvYmrnKHPZM4OzZCreqbhv6bm3zfg1wXVWtB0hyI7Af8NMR\nypckdV8v/SNwGHAE8BfA0UCArw0vlOQ3gF2r6pJm1ieA52xlu/+zqu4G7k6yAdgLuHkry49Sbz0D\neBlAVd0D3JFkt2bda5o4rwMuqqpKcs0WtiN1xgRL06jm+BwGScJTt7Zikp2ADwMHV9WPkswAO41Y\n7nxl3N2838Ov/2+O3PxwaP3ZbTygqcWIZQ4LD/x7bb7uvZuVfe8WtiVJmlvX9dLXGNy92p9BC4Q/\na8r8/OabnyO2rdm8HpqvLtieemuudYfrIesg9c4mgppGj0gyW2GdCFwG/ADYc3Z+kh2TPK5Z5i7g\noc30bKX1kyS7ANvSO9PWytiSy4AXNssfBPyHoe/+3wjNLxbqy8Arhp4H2L3l8iRpGnVdL10KvAS4\nvqruBW5j0GHF5cMLNZ1R3JHkac2sFw99PRxDmy4CXgOQZFnTtFFa1EywNI1+ALw2yToGTSk+0rQn\nPx54T/Mg79XAoc3ya4DTk1zN4ArZx4BrgS8B3xq10HnK2JIPM6hgvwf8OYNmH3c0330U+G5+3cnF\n2FXVFxk0Bbyi2f95u+CVJG2zTuulqrqpmby0eb8M+FlV3T7H4i8HPtSUNdyq4qsMOrUY7uSiDW9g\n0ATyGgbNCOe7MCn1LlXbcudXWtqSHAB8vnkQeNFLsgzYsar+LclvM7iS9++bileStMQttXpJ0vxs\noyotbsuBrzZNAQO8xuRKkiRp8fIOliRJkiSNic9gSZIkSdKYmGBJkiRJ0piYYEmSJEnSmJhgSZIk\nSdKYmGBJkiRJ0pj8f9CbeGldo0keAAAAAElFTkSuQmCC\n", "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "from matplotlib import pyplot as plt\n", "import numpy as np\n", "import math\n", "\n", "fig, axes = plt.subplots(nrows=2, ncols=2, figsize=(12,6))\n", "\n", "for ax,cnt in zip(axes.ravel(), range(4)): \n", " \n", " # set bin sizes\n", " min_b = math.floor(np.min(X[:,cnt]))\n", " max_b = math.ceil(np.max(X[:,cnt]))\n", " bins = np.linspace(min_b, max_b, 25)\n", " \n", " # plottling the histograms\n", " for lab,col in zip(range(1,4), ('blue', 'red', 'green')):\n", " ax.hist(X[y==lab, cnt],\n", " color=col, \n", " label='class %s' %label_dict[lab], \n", " bins=bins,\n", " alpha=0.5,)\n", " ylims = ax.get_ylim()\n", " \n", " # plot annotation\n", " leg = ax.legend(loc='upper right', fancybox=True, fontsize=8)\n", " leg.get_frame().set_alpha(0.5)\n", " ax.set_ylim([0, max(ylims)+2])\n", " ax.set_xlabel(feature_dict[cnt])\n", " ax.set_title('Iris histogram #%s' %str(cnt+1))\n", " \n", " # hide axis ticks\n", " ax.tick_params(axis=\"both\", which=\"both\", bottom=\"off\", top=\"off\", \n", " labelbottom=\"on\", left=\"off\", right=\"off\", labelleft=\"on\")\n", "\n", " # remove axis spines\n", " ax.spines[\"top\"].set_visible(False) \n", " ax.spines[\"right\"].set_visible(False) \n", " ax.spines[\"bottom\"].set_visible(False) \n", " ax.spines[\"left\"].set_visible(False) \n", " \n", "axes[0][0].set_ylabel('count')\n", "axes[1][0].set_ylabel('count')\n", " \n", "fig.tight_layout() \n", " \n", "plt.show()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "From just looking at these simple graphical representations of the features, we can already tell that the petal lengths and widths are likely better suited as potential features two separate between the three flower classes. In practice, instead of reducing the dimensionality via a projection (here: LDA), a good alternative would be a feature selection technique. For low-dimensional datasets like Iris, a glance at those histograms would already be very informative. Another simple, but very useful technique would be to use feature selection algorithms, which I have described in more detail in another article: [Feature Selection Algorithms in Python](http://sebastianraschka.com/Articles/2014_sequential_sel_algos.html)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "\n", "
\n", "
" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Normality assumptions" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "[[back to top](#Sections)]" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "It should be mentioned that LDA assumes normal distributed data, features that are statistically independent, and identical covariance matrices for every class. However, this only applies for LDA as classifier and LDA for dimensionality reduction can also work reasonably well if those assumptions are violated. And even for classification tasks LDA seems can be quite robust to the distribution of the data: \n", "\n", "> \"linear discriminant analysis frequently achieves good performances in\n", "> the tasks of face and object recognition, even though the assumptions\n", "> of common covariance matrix among groups and normality are often\n", "> violated (Duda, et al., 2001)\" (Tao Li, et al., 2006).\n", "\n", "
\n", "\n", " \n", "\n", "Tao Li, Shenghuo Zhu, and Mitsunori Ogihara. “[Using Discriminant Analysis for Multi-Class Classification: An Experimental Investigation](http://link.springer.com/article/10.1007%2Fs10115-006-0013-y).” Knowledge and Information Systems 10, no. 4 (2006): 453–72.) \n", "\n", "Duda, Richard O, Peter E Hart, and David G Stork. 2001. Pattern Classification. New York: Wiley." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "\n", "
\n", "
" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "# LDA in 5 steps" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "[[back to top](#Sections)]" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "After we went through several preparation steps, our data is finally ready for the actual LDA. In practice, LDA for dimensionality reduction would be just another preprocessing step for a typical machine learning or pattern classification task." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "\n", "
\n", "
" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Step 1: Computing the d-dimensional mean vectors" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "[[back to top](#Sections)]" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "In this first step, we will start off with a simple computation of the mean vectors $\\pmb m_i$, $(i = 1,2,3)$ of the 3 different flower classes:\n", " \n", "$\\pmb m_i = \\begin{bmatrix} \n", "\\mu_{\\omega_i (\\text{sepal length)}}\\\\ \n", "\\mu_{\\omega_i (\\text{sepal width})}\\\\ \n", "\\mu_{\\omega_i (\\text{petal length)}}\\\\\n", "\\mu_{\\omega_i (\\text{petal width})}\\\\\n", "\\end{bmatrix} \\; , \\quad \\text{with} \\quad i = 1,2,3$" ] }, { "cell_type": "code", "execution_count": 8, "metadata": { "collapsed": false }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Mean Vector class 1: [ 5.006 3.418 1.464 0.244]\n", "\n", "Mean Vector class 2: [ 5.936 2.77 4.26 1.326]\n", "\n", "Mean Vector class 3: [ 6.588 2.974 5.552 2.026]\n", "\n" ] } ], "source": [ "np.set_printoptions(precision=4)\n", "\n", "mean_vectors = []\n", "for cl in range(1,4):\n", " mean_vectors.append(np.mean(X[y==cl], axis=0))\n", " print('Mean Vector class %s: %s\\n' %(cl, mean_vectors[cl-1]))" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "\n", "
\n", "
" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Step 2: Computing the Scatter Matrices" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "[[back to top](#Sections)]" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Now, we will compute the two *4x4*-dimensional matrices: The within-class and the between-class scatter matrix." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "
\n", "
" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### 2.1 Within-class scatter matrix $S_W$" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "[[back to top](#Sections)]" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The **within-class scatter** matrix $S_W$ is computed by the following equation: \n", "\n", "$S_W = \\sum\\limits_{i=1}^{c} S_i$\n", "\n", "where \n", "$S_i = \\sum\\limits_{\\pmb x \\in D_i}^n (\\pmb x - \\pmb m_i)\\;(\\pmb x - \\pmb m_i)^T$ \n", "(scatter matrix for every class) \n", "\n", "and $\\pmb m_i$ is the mean vector \n", "$\\pmb m_i = \\frac{1}{n_i} \\sum\\limits_{\\pmb x \\in D_i}^n \\; \\pmb x_k$" ] }, { "cell_type": "code", "execution_count": 9, "metadata": { "collapsed": false }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "within-class Scatter Matrix:\n", " [[ 38.9562 13.683 24.614 5.6556]\n", " [ 13.683 17.035 8.12 4.9132]\n", " [ 24.614 8.12 27.22 6.2536]\n", " [ 5.6556 4.9132 6.2536 6.1756]]\n" ] } ], "source": [ "S_W = np.zeros((4,4))\n", "for cl,mv in zip(range(1,4), mean_vectors):\n", " class_sc_mat = np.zeros((4,4)) # scatter matrix for every class\n", " for row in X[y == cl]:\n", " row, mv = row.reshape(4,1), mv.reshape(4,1) # make column vectors\n", " class_sc_mat += (row-mv).dot((row-mv).T)\n", " S_W += class_sc_mat # sum class scatter matrices\n", "print('within-class Scatter Matrix:\\n', S_W)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "
" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### 2.1 b" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Alternatively, we could also compute the class-covariance matrices by adding the scaling factor $\\frac{1}{N-1}$ to the within-class scatter matrix, so that our equation becomes\n", "\n", "$\\Sigma_i = \\frac{1}{N_{i}-1} \\sum\\limits_{\\pmb x \\in D_i}^n (\\pmb x - \\pmb m_i)\\;(\\pmb x - \\pmb m_i)^T$.\n", "\n", "and $S_W = \\sum\\limits_{i=1}^{c} (N_{i}-1) \\Sigma_i$\n", "\n", "where $N_{i}$ is the sample size of the respective class (here: 50), and in this particular case, we can drop the term ($N_{i}-1)$ \n", "since all classes have the same sample size.\n", "\n", "However, the resulting eigenspaces will be identical (identical eigenvectors, only the eigenvalues are scaled differently by a constant factor)." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "
\n", "
" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### 2.2 Between-class scatter matrix $S_B$" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "[[back to top](#Sections)]" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The **between-class scatter** matrix $S_B$ is computed by the following equation: \n", "\n", "$S_B = \\sum\\limits_{i=1}^{c} N_{i} (\\pmb m_i - \\pmb m) (\\pmb m_i - \\pmb m)^T$\n", "\n", "where \n", " $\\pmb m$ is the overall mean, and $\\pmb m_{i}$ and $N_{i}$ are the sample mean and sizes of the respective classes." ] }, { "cell_type": "code", "execution_count": 10, "metadata": { "collapsed": false }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "between-class Scatter Matrix:\n", " [[ 63.2121 -19.534 165.1647 71.3631]\n", " [ -19.534 10.9776 -56.0552 -22.4924]\n", " [ 165.1647 -56.0552 436.6437 186.9081]\n", " [ 71.3631 -22.4924 186.9081 80.6041]]\n" ] } ], "source": [ "overall_mean = np.mean(X, axis=0)\n", "\n", "S_B = np.zeros((4,4))\n", "for i,mean_vec in enumerate(mean_vectors): \n", " n = X[y==i+1,:].shape[0]\n", " mean_vec = mean_vec.reshape(4,1) # make column vector\n", " overall_mean = overall_mean.reshape(4,1) # make column vector\n", " S_B += n * (mean_vec - overall_mean).dot((mean_vec - overall_mean).T)\n", " \n", "print('between-class Scatter Matrix:\\n', S_B)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "
\n", "
" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Step 3: Solving the generalized eigenvalue problem for the matrix $S_{W}^{-1}S_B$" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "[[back to top](#Sections)]" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Next, we will solve the generalized eigenvalue problem for the matrix $S_{W}^{-1}S_B$ to obtain the linear discriminants." ] }, { "cell_type": "code", "execution_count": 11, "metadata": { "collapsed": false }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "\n", "Eigenvector 1: \n", "[[-0.2049]\n", " [-0.3871]\n", " [ 0.5465]\n", " [ 0.7138]]\n", "Eigenvalue 1: 3.23e+01\n", "\n", "Eigenvector 2: \n", "[[ 0.009 ]\n", " [ 0.589 ]\n", " [-0.2543]\n", " [ 0.767 ]]\n", "Eigenvalue 2: 2.78e-01\n", "\n", "Eigenvector 3: \n", "[[ 0.138 ]\n", " [-0.326 ]\n", " [-0.3796]\n", " [ 0.6671]]\n", "Eigenvalue 3: -2.17e-16\n", "\n", "Eigenvector 4: \n", "[[ 0.138 ]\n", " [-0.326 ]\n", " [-0.3796]\n", " [ 0.6671]]\n", "Eigenvalue 4: -2.17e-16\n" ] } ], "source": [ "eig_vals, eig_vecs = np.linalg.eig(np.linalg.inv(S_W).dot(S_B))\n", "\n", "for i in range(len(eig_vals)):\n", " eigvec_sc = eig_vecs[:,i].reshape(4,1) \n", " print('\\nEigenvector {}: \\n{}'.format(i+1, eigvec_sc.real))\n", " print('Eigenvalue {:}: {:.2e}'.format(i+1, eig_vals[i].real))" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "
\n", "
" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "After this decomposition of our square matrix into eigenvectors and eigenvalues, let us briefly recapitulate how we can interpret those results. As we remember from our first linear algebra class in high school or college, both eigenvectors and eigenvalues are providing us with information about the distortion of a linear transformation: The eigenvectors are basically the direction of this distortion, and the eigenvalues are the scaling factor for the eigenvectors that describing the magnitude of the distortion. \n", "\n", "If we are performing the LDA for dimensionality reduction, the eigenvectors are important since they will form the new axes of our new feature subspace; the associated eigenvalues are of particular interest since they will tell us how \"informative\" the new \"axes\" are. \n", "\n", "Let us briefly double-check our calculation and talk more about the eigenvalues in the next section." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "
\n", "
" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Checking the eigenvector-eigenvalue calculation" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "[[back to top](#Sections)]" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "\n", "\n", "A quick check that the eigenvector-eigenvalue calculation is correct and satisfy the equation:\n", "\n", "$\\pmb A\\pmb{v} = \\lambda\\pmb{v}$ \n", "\n", "
\n", "where \n", "$\\pmb A = S_{W}^{-1}S_B\\\\\n", "\\pmb{v} = \\; \\text{Eigenvector}\\\\\n", "\\lambda = \\; \\text{Eigenvalue}$" ] }, { "cell_type": "code", "execution_count": 12, "metadata": { "collapsed": false }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "ok\n" ] } ], "source": [ "for i in range(len(eig_vals)):\n", " eigv = eig_vecs[:,i].reshape(4,1) \n", " np.testing.assert_array_almost_equal(np.linalg.inv(S_W).dot(S_B).dot(eigv), \n", " eig_vals[i] * eigv, \n", " decimal=6, err_msg='', verbose=True)\n", "print('ok')" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "
\n", "
" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Step 4: Selecting linear discriminants for the new feature subspace" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "[[back to top](#Sections)]" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "
\n", "
" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### 4.1. Sorting the eigenvectors by decreasing eigenvalues" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "[[back to top](#Sections)]" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "\n", "\n", "Remember from the introduction that we are not only interested in merely projecting the data into a subspace that improves the class separability, but also reduces the dimensionality of our feature space, (where the eigenvectors will form the axes of this new feature subspace). \n", "\n", "However, the eigenvectors only define the directions of the new axis, since they have all the same unit length 1." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "So, in order to decide which eigenvector(s) we want to drop for our lower-dimensional subspace, we have to take a look at the corresponding eigenvalues of the eigenvectors. Roughly speaking, the eigenvectors with the lowest eigenvalues bear the least information about the distribution of the data, and those are the ones we want to drop. \n", "The common approach is to rank the eigenvectors from highest to lowest corresponding eigenvalue and choose the top $k$ eigenvectors." ] }, { "cell_type": "code", "execution_count": 13, "metadata": { "collapsed": false }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Eigenvalues in decreasing order:\n", "\n", "32.2719577997\n", "0.27756686384\n", "4.05933275259e-15\n", "4.05933275259e-15\n" ] } ], "source": [ "# Make a list of (eigenvalue, eigenvector) tuples\n", "eig_pairs = [(np.abs(eig_vals[i]), eig_vecs[:,i]) for i in range(len(eig_vals))]\n", "\n", "# Sort the (eigenvalue, eigenvector) tuples from high to low\n", "eig_pairs = sorted(eig_pairs, key=lambda k: k[0], reverse=True)\n", "\n", "# Visually confirm that the list is correctly sorted by decreasing eigenvalues\n", "\n", "print('Eigenvalues in decreasing order:\\n')\n", "for i in eig_pairs:\n", " print(i[0])" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "
\n", "
" ] }, { "cell_type": "code", "execution_count": 14, "metadata": { "collapsed": false }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Variance explained:\n", "\n", "eigenvalue 1: 99.15%\n", "eigenvalue 2: 0.85%\n", "eigenvalue 3: 0.00%\n", "eigenvalue 4: 0.00%\n" ] } ], "source": [ "print('Variance explained:\\n')\n", "eigv_sum = sum(eig_vals)\n", "for i,j in enumerate(eig_pairs):\n", " print('eigenvalue {0:}: {1:.2%}'.format(i+1, (j[0]/eigv_sum).real))" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "If we take a look at the eigenvalues, we can already see that 2 eigenvalues are close to 0. The reason why these are close to 0 is not that they are not informative but it's a floating-point imprecision. In fact, these two last eigenvalues should be exactly zero: In LDA, the number of linear discriminants is at most $c−1$ where $c$ is the number of class labels, since the in-between scatter matrix $S_B$ is the sum of $c$ matrices with rank 1 or less. Note that in the rare case of perfect collinearity (all aligned sample points fall on a straight line), the covariance matrix would have rank one, which would result in only one eigenvector with a nonzero eigenvalue." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "
\n", "
" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The first eigenpair is by far the most informative one, and we won't loose much information if we would form a 1D-feature spaced based on this eigenpair." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "
\n", "
" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### 4.2. Choosing *k* eigenvectors with the largest eigenvalues" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "[[back to top](#Sections)]" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "After sorting the eigenpairs by decreasing eigenvalues, it is now time to construct our $d \\times k$-dimensional eigenvector matrix $\\pmb W$ (here $4 \\times 2$: based on the 2 most informative eigenpairs) and thereby reducing the initial 4-dimensional feature space into a 2-dimensional feature subspace." ] }, { "cell_type": "code", "execution_count": 15, "metadata": { "collapsed": false }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Matrix W:\n", " [[-0.2049 0.009 ]\n", " [-0.3871 0.589 ]\n", " [ 0.5465 -0.2543]\n", " [ 0.7138 0.767 ]]\n" ] } ], "source": [ "W = np.hstack((eig_pairs[0][1].reshape(4,1), eig_pairs[1][1].reshape(4,1)))\n", "print('Matrix W:\\n', W.real)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "
\n", "
" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "# Step 5: Transforming the samples onto the new subspace" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "[[back to top](#Sections)]" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "In the last step, we use the $4 \\times 2$-dimensional matrix $\\pmb W$ that we just computed to transform our samples onto the new subspace via the equation \n", "\n", "$\\pmb Y = \\pmb X \\times \\pmb W $.\n", "\n", "(where $\\pmb X$ is a $n \\times d$-dimensional matrix representing the $n$ samples, and $\\pmb Y$ are the transformed $n \\times k$-dimensional samples in the new subspace)." ] }, { "cell_type": "code", "execution_count": 16, "metadata": { "collapsed": false }, "outputs": [], "source": [ "X_lda = X.dot(W)\n", "assert X_lda.shape == (150,2), \"The matrix is not 150x2 dimensional.\"" ] }, { "cell_type": "code", "execution_count": 17, "metadata": { "collapsed": false }, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYUAAAEUCAYAAADEGSquAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJztvXl8lNW9+P/+ZI+GhCBIWATqgqAQWVxAvToiLvW6FfW2\n1qW43P74Wm29V79XqrVStb3W0tb2arWtVepyq/0qKsXWrSZFatzAgAuIiDGGJSxZByYJSc7vj/NM\nMpnMTGYms2Y+79drXsk8z3nO8zkzz5zPOeezHDHGoCiKoigAWckWQFEURUkdVCkoiqIoPahSUBRF\nUXpQpaAoiqL0oEpBURRF6UGVgqIoitKDKoU4IiKXicgrKSDHRyLiSpd6UwkRcYlIXQzru1tEdovI\nDhGZICJuEcmOVf2DkKtPO1P5uxWRGhGZ7/x/q4g8HOP6/0VEPony2pT5TqMlrZWC78Phd9wlIt3O\nl+MWkToR+bOIHBegrIjIFhH5OFb392KMedIYc2ak9cYaY8zRxpjKwdQhIstE5O5Y1xtrBvpOwrje\niMjhsZTJp+5DgJuAo4wxZcaYWmNMkTGmK4q6ForI6gHKLBWRT0WkVUQ2isiV4dafit9tIIwxPzHG\nXBvjOt8wxhwZ5bVRf6eREq9nNa2VwgBsM8YUAcOAOcBG4A0ROd2v3CnAwcChgZRGtIhITqzqSoX7\nKDFhIrDHGLNzoILOYGWwv8+9wHlACfAt4FcicuIg64wbqfAsp4IMSccYk7YvoAaYH+C4C6gLcPx+\n4D2/Y48ATwLLgfujvT+wEPgn8EugAbjbObbaOS/OuZ1AM7AemBak3krgv4F3nLIvACOcc5MAA1wD\n1AKrnOPnAx8BTc71U4PImQUsBj4D9gB/9tbtnD8ZeNOp50unDd8G9gMdgBv4S4B684H7gG3O6z4g\n3/f7wI6SdwLbgatCfK5jgRXO57gZ+Hefc0scmR8DWp02H+ucexzoBjyOnP810Gfjd99Vzme717n+\n6wPJ7rR7qfNd1AMPAYUB6p7vyNXt1L3M57vM8fnef4x9jjzA4c7nv8Vp6+fAZcBUoA3ocupqCvN5\nXQHcFOScC5/fjN93G/Qz9/m+ngV2OTJ+1+fc8UCV89lvx/4G83zOG+A7wKfA50FkuwL4Avu83hZA\ntiec/wuAJ5xyTcC7wGjn3AjgUeyz2Qg87/ds3gLswD5DgT6L/4v9ze4F/gCMBv7mfB6vAaV+v0/f\n7/Qu5zttBV4BRvrU/f+c+zZjn7+jfc4tAx4AXnSufRs4LMSzOhJY6bS9AXgDyIq4X42mM06VF5Er\nhXnYH+WBzvsDgBbgHOAiYLffA7sYWBnO/bE/3k7gBiAHKKSvUjgLWAMMxyqIqcCYIPVWAluBacCB\n2B+c98H3PnSPOecKgcnOw3EGkAv8F7YzzQsg543AW8B4bIf2W+BPzrkJzsN3qVPPQcAMnwf07hDt\nv9Op92BgFFax3OXzfXQ6ZXKdz3sfzg8pQPv/AfwG+yOfge1sTvfpBNqcOrKxyvOtYM/EQJ9NgHsb\n4HC/Zymo7FjltwLb6QwD/gL8d5C6XfTtbLzfpW8HUgscjX2GSrDP55HO+TE4nQY+z1aYv5VCbKd8\ndpiy+X63QT9z7CBjDfBDIA84FKvEznLOz8bO1HOc9m4AbvT7vF91Pr9AyvQobKd3CvZ5/YXzfQRS\nCv+f8/kf4Mg5Gyh2zr0IPA2UOt/jqX7f70+d+guDfBZvYRXBOOzgYC0w07nmdeCOEN/pZ9jnsNB5\nf49P3VdjnxvvoKra59wybOd+vPP5PQk8FeJZ/W/soCTXef0LIBH3q5FekEovIlcKU5wPcpzz/nJs\nh5PjfClNwNeiuT/2R1rrd34hvUphHrAJ+wMJqb0DPDhHYUfp2T4P3aE+528H/uzzPgurVFwB5NyA\n08E678dgZwE5wPeB54LItIzQSuEz4Byfc2cBNT7fh8f7Q3GO7QTmBLjPIdgR8DC/h32Z8/8S4DW/\nz8YT7JkY6LMJcP9ASiGg7Fjlvhdn9Oacm0vwEW+f55LAHcidPucPxD6TF+HXYRK5Uvgj8BJBOokA\nsvl+t0E/c+AE+j/33wceDXKfG32fMaf980LI/UP6doQHYn8LgZTC1djBSLlfHWOwg8F+gxCn3R1A\nwQCfxWU+758FHvR5fwO9M49A3+kPfMpeB7wUpK3DnWtLfH5zD/ucPwfYGOJZvRO7qnB4oPrDfQ1l\nm0IgxmE/yCbn/bewHUanMaYdu4T0rUHU/2WwE8aY17FT5weAehH5nYgUh1nXF1jNPzLI+bFOGe+9\nup3z4wLUOxF4TkSaRKQJqyS6sKOgQ7CdezT0kcH5f6zP+z3GmE6f9/uAoiD1NBhjWv3q8m3LDr96\nCkKsBUfy2QQjmOyjsKPSNT6f50vO8Wjp+V6NMXuxywKLgO0i8qKITIm0QhH5GXbW+W/G6T2iINhn\nPhEY622/8xncin2eEJHJIrLS8bZqAX5C3+cYQvxusN+f/2eyJ0jZx4GXgadEZJuI3CsiudjnusEY\n0xjkul3GmLYQMoBdGvTiCfA+0LPsxf+zKwIQkWwRuUdEPnM+mxqnzMiBrg3Cz7Cz4Fcc55nFIcoG\nJdOUwteAtcaYvSIyHjt6v9x5YHcAFwPniIj/QxsuIX9wxphfG2NmY5cHJmPXKYNxiM//E7Cj+d1B\n7rUN++MErJHSuX5rgHq/BL5qjBnu8yowxmx1zh0WTPwQsvaTwZF52wDXBKtnhIgM86srUFsC4S9n\nJJ9NpOzGdghH+3yWJcY6OERLH/mNMS8bY87AjnY3Ar8PVC4YIvIj4KvAmcaYlkHIFYwvsTMj3+dp\nmDHmHOf8g1i5jzDGFGMVhvjVEaot2/H5LYjIAdhlzX4YY/YbY35kjDkKOBE4F7jSkXGEiAwPco9o\nFeVg+SZwAdbeVIKdZUD/zycsjDGtxpibjDGHYh0M/jOAY82ADAWlkCsiBT6vPiNGx4tjnIjcAVyL\nfSjBGq82AUdi161nYDvqOuyaekwRkeNE5ARn5LKXXkNhMC4XkaOcH8GdwDMmuJvbn4F/FZHTnfpv\nAtqxU2l/HgJ+LCITHblGicgFzrkngfki8m8ikiMiB4nIDOdcPXa9OBh/An7g1DcSO+1/IkT5gBhj\nvnTk/m/n+yzHGtWfDLMKfzkj+WwCXR9K1m5sJ/1LETkYwHnWzgpT1pCIyGgROV9EDnRkdtP7zNQD\n40UkL8T138d2PGcYY4KNrgfLO0CLiNwiIoXO6HeajyffMKxdxO3Mcv5PhPU/A5wrIic7bb2TIP2W\niJwmItPFxgi0YAdSXcaY7Vij8G9EpFREckXklMibGnOGYb/XPdgZ508ivL7Psyoi54rI4c7ApwX7\nrETsGjsUlMJfsaM172uJc3ysiLixP6R3genYdWRvMNm3gN8YY3b4vrCd5regJzDmbzGSsxjbgTTS\n60mxNET5x7FrijuwBtfvBitojPkEax/5H+zo9TzgPGNMR4Div8IaRl8RkVasAe0Ep55a7LrlTVgD\nVzVwjHPdH4CjnCWC5wPUezfwHtZD4wOsIe7uAOXC4VLsqGkb8BzWiPdqmNf+N1Y5NYnIzRF+NmCf\nnz861/9bGPe7BTtlf8tZAngNO9CIBVnY72Ib9vs4FbsmDda4+RGwQ0R2B76cn2BnWZ9Kb8zOrUHK\nRoUzUDkPO6j6HPsZP4wd+QLcjFVMrdjn/+kI6/8I6530v9hZQyN24BaIMqwSacEui/6D3oHJFVgl\nsRFrE7oxEjnixGPYvmAr8DH2txgJS+j7rB6Bff7cWI+v35goYk0k+iVGJV6ISCXWeBaTSE0RqQUu\nN8asikV9iqIMXYbCTEEJgYiMwho+a5IsiqIoaYAqhSGMs677KfA/ztKQoihKSNJRKZhIXlVVVRGV\nT4WXMeZUY8zvB1vPm2+++Y7jDfN/k92mZLzS8bvXtmvb49j2sEhHpRAR7e3tyRYhaWRy2yGz269t\nz0xi0fYhrxQURVGU8FGloCiKovSgSkFRFEXpIW65w8VuKPIYNqCkG/idMeZXAcq5sNkBc4HdxphT\n4yWToiipRVdXFw0NDezfvz9mdRYXF7NtWzQZVtKf4uJidu3axYgRI8jOjm7zt3huKNGJzd2+1slj\ns0ZEXjXG9Oxw5uQi+Q02nW+tN1WAoiiZQUNDAwUFBYwcORKbnWHwdHR0MHbs2IELDkHa29spKCig\noaGBUaOiy8sYt+UjY8x2Y8xa5/9WbNi5f2bKbwLLvT70JowdqRRFGTrs37+foqKimCmETEdEKCoq\nGtTMKyFpLkRkEnanoGm+mRpFxLtsdDQ2OdSvjDGPhaqrqqrKROJ25Xa7KSoaTNLK9CWT2w6Z3f50\naXtxcTEjRoyIaZ0dHR3k5QXNEzik8ba9oaGBlpa+SXFdLld4mncwmzGE88Lm/14DLAhw7n5sEqgD\nsTnEPwUmD1BnRFRUVER6iTHGmJYWY376U2NaW6O6PCWItu1DhUxuf7q0fevWrTGv8/PPP495nemC\nt+1BPtfkb7LjpCp+FnjSGLM8QJE67C5Ee40xu7GziWMClEs4FRXwxhv2r6IoQ5sf//jHHH300ZSX\nlzNjxgzefvvtoGWXLVs2pA3Z8fQ+Emy65Q3GmF8EKfYCcL+zB0IeNoXzL+MlU7i0tsKLL8LkybBy\nJZx2GqTBTFxRlCioqqpi5cqV/G3V39jSuoUvt3/Jl3zJJPckRheN7ld+2bJlTJs2bcgas+M5UzgJ\nm8N8nohUO69zRGSRiCwCMMZswG5fuB67WcfDxpgP4yhTWFRUQEcHDBtm/+psQVFSh9ZWuPdecLtj\nU9/27dsZNnwYa3auwbPfw/iy8Rw44kAef+lx5p48l9mzZ3PWWWexfft2nnnmGd577z0uu+wyZsyY\ngcfj4e9//zszZ85k+vTpXH311T2pJhYvXsxRRx1FeXk5N998MwB/+ctfOOGEE5g5cybz58+nvr4+\nlGhJIZ7eR6uNMWKMKTfGzHBefzXGPGSMecin3M+MMUcZY6YZY+6Llzzh4p0llJXZ92VldrYQqwdQ\nUZTBEeul3TPPPJMtX2zh+nOu56E7H2Ld2+vIJZdHfvIIt/zqFtasWcPVV1/NbbfdxsUXX8yxxx7L\nk08+SXV1NSLCwoULefrpp/nggw/o7OzkwQcfpKGhgeeee46PPvqI9evX84Mf/ACAk08+mbfeeov3\n33+fb3zjG9x7772xaUQM0YhmP7yzhPx8+z4/X2cLipIq+C/txmKwVlRUxL3/715u/snNDB8xnDtv\nuJO//O9f+OLTL/iPy/+DGTNmcPfdd1NX13/Dt08++YSvfOUrTJ48GYBvfetbrFq1iuLiYgoKCrj2\n2mtZvnw5BxxwAAB1dXWcddZZTJ8+nZ/97Gd89NFHg29AjFGl4Me6dWAM1NT0voyB6uokC6YoStyW\ndkccOIIpx07hqv+4iu/96HusemkVEw6fwKN/e5Tq6mo++OADXnnllX7XmSAu/Tk5ObzzzjtcdNFF\nrFy5krPPPhuAG264geuvv54PPviA3/72t7S1tcWmATEknhHNacnttydbAkVRAhFsaXewjiCffPIJ\n+Z58GvMbAfj0408Zd+g41ryxhr2f7YVJNshu06ZNHH300QwbNozW1lYApkyZQk1NDZs3b+bwww/n\n8ccf59RTT8XtdrNv3z7OOeccTjrpJA499FAAmpubGTfOxvD+8Y9/jF7oOKIzBUVR0oJ4Le263W5u\nuu4mbvnaLVx/3vV89sln/PtN/84TTz3BT3/0U4455hhmzJjBm2++CcDChQtZtGgRM2bMwBjDo48+\nyiWXXML06dPJyspi0aJFtLa2cu6551JeXs7JJ5/ML35hHTCXLFnCJZdcwuzZsxk5cuTgBI8TOlNQ\nFCUt8F3a9aW6Gs47L/p6Z8+e3dPh+3P6qtP7Hbvooou46KKLesucfjrvv/9+nzJjxozhnXfe6Xft\nBRdcwAUXXBC9sAlAlYKiKGmBLu0mBl0+UhRFUXrQmYKiKBlHvbueDbs30NzWTElBCVNHTg0YvZyJ\nqFJQFCWjqHfX8+aXb1KQU0BxfjGe/R7e/PJNjjzoSHbu25nxikKXjxRFySg27N5AQU4BhbmFiAiF\nuYV0m27+UfsPPPs9fRRFvTv10lDEG1UKiqJkFM1tzRTkFPQ51tLRQnd3dx9FUZBTwIbdG5IkZfJQ\npaAoypCh3l1Pa0crL2x8gcqayoAj/ZKCEto6bSTxjd+4kXf+8Q77OvZxQK5NRfHMI8/wy9t/SUFO\nAc1tzVHJ8cMf/pDXXnst4usqKys599xzo7pnrFCloCjKkMBrK+g23SGXgKaOnEpbZxue/R7mnTeP\nV1e8SpZkUZxfDMDrf3md0887nbbONkoKSoLezxhDd3d3wHN33nkn8+fPj13jgtDZ2RnzOlUpKIqS\nHixeDAsX9n8tXgz02gqyJCvkEtDootGceMiJFOYWMnPeTN6tfJe5ZXPJkixqPq9hT/0eDp9xOE/9\n9iluvOhGysvLueOOOwCoqanhyCOP5Morr2TatGl8+eWXLFy4kGnTpjF9+nR++Uu7HczChQt55pln\nAHj33Xc58cQTOeaYYzj++ONpbW2lra2Nq666iunTpzNz5kwqAoRlNzQ0cOGFF1JeXs6cOXNYv349\nYKOir7jiCk466SSuuOKKmH/M6n2kKEp6sGMHTJrU/7gT4tzc1mxH+z571gdbAhpdNNp6Fk2Cp+Y8\nxRdrv+DE00/k9mW3c+JXT+Sjtz6ie08377/3PsYYzj//fFatWsWECRP49NNP+eMf/8icOXNYs2YN\nW7du5cMP7TYwTU1Nfe7T0dHB17/+dZ5++mmOO+44WlpaKCws5Fe/+hUAH3zwARs3buTMM89k06ZN\nfa694447mDlzJs8//zyvv/46V155JdVOZs6PP/6Y1atXU1hYGN1nGQKdKSiKMiTwtRV4GWgJCODS\nSy/lqaeeYnTRaN55+R1u/87t1K+v542KN5g5cyazZs1i48aNfPrppwBMnDiROXPmAHDooYeyZcsW\nbrjhBl566SWKi4v71P3JJ58wZswYjjvuOACKi4vJyclh9erVPaP8KVOmMHHixH5KwbfMvHnz2LNn\nD83NVsGdf/75cVEIoEpBUZQhgtdW0G26Mcbg2e+hrbONqSOnhrzuwgsv5O9//ztr167F4/Ewa9Ys\njDF8//vfp7q6murqajZv3sw111wDwIEHHthzbWlpKevWrcPlcvHQQw9x7bXX9qnbGIPdmZh+xwci\nUBlvXb4yxJq4KQUROUREKkRkg4h8JCLfC1H2OBHpEpGL4yWPoihDG6+tIEuyaGlvoTC3kBMPOXHA\nALSioiJcLhdXX301l156KQBnnXUWjzzyCG5nF5+tW7eyc+fOftfu3r2b7u5uLrroIu6++27Wrl3b\n5/yUKVPYtm0b7777LgCtra10dnZyyimn8OSTTwKwadMmamtrOfLII/tc61umsrKSkSNH9puJxIN4\n2hQ6gZuMMWtFZBiwRkReNcZ87FtIRLKBnwIvx1EWRVEygNFFo/HkebhgUmSZSC+99FIWLFjAU089\nBdgtOjds2MDcuXMBqzieeOIJsrOz+1y3detWrrrqKjo6O2jrbOObN36TyppKPPs9AOTl5fH0009z\nww034PF4KCws5LXXXuO6665j0aJFTJ8+nZycHJYtW0a+Nye4w5IlS7jqqqsoLy/ngAMOSNj+CxLO\nNCYmNxJ5AbjfGPOq3/Ebsaah44CVxphnBqgqIoErKytxuVyRXDJkyOS2Q2a3P13avm3bNsaOHRte\n4cWLrbHZn7IyuOeenrc1NTVMCmSQjhO+aTMKcgpo62yjrbMtrFlKrPG2Pcjn2n8dKwAJUQoiMglY\nBUwzxrT4HB8H/C8wD/gDYSiFqqoq097eHva93W43RYPZlimNyeS2Q2a3P13aXlxczIgRI2JaZ0dH\nB3l5eTGtMxStHa10m26ypHc13vt+WN6whMkBvW1vaGigpaWlzzmXyxWWUoi7S6qIFAHPAjf6KgSH\n+4BbjDFdgYwxgfBO58IlXUZM8SCT2w6Z3f50aXtEM4UwSfRM4YWNL1CcX9zHoGyMoaW9JeJlrMHi\nbXteXh6zZs2Kqo64KgURycUqhCeNMcsDFDkWeMr5MEcC54hIpzHm+XjKpSiKEitKCkrw7PdQmNvr\nIhqOK2yqEjelILan/wOwwRjzi0BljDFf8Sm/DLt8pApBUZSI9jzwLXto1qHUu+sTtp4/deRU3vzS\nbufpa1OYNSa6kXqyiWecwknAFcA8Eal2XueIyCIRWRTH+yqKkuZ4jbfhpLL2L9ttuhOa9to3bUYk\nrrCpStxmCsaY1YRp7XbKL4yXLIqipBe+ex4APX837N7Qr7P1L5slWT05jxLVMfekzRgCaESzoigp\nR6A9D4LlMYqkrD8ul4uXX+4bInXfffdx9dVXc/HFkcfSXnvttXz88cchyzz00EM89thjEdedKFQp\nDILWVrj3XnCCHhVFiRGR5DGKNucR9OY98uWpp57iqquu6sly6stAqaoffvhhjjrqqJBlFi1axJVX\nXjmgbMlClQLRd+4VFfDGG/avoiixw3fPA988RqbbsKRyCVe/cDVLKpewfsf6fmW7TXdYOY8ALr74\nYlauXIk39qmmpoZt27Yxfvx4pk2bBsCyZcs4//zzmTdvHqeffjrd3d1cd911TJkyhTPOOINzzjmn\nR4G4XC7ee+89wEZB33bbbRxzzDHMmTOH+npr41iyZAlLly4FYPPmzcyfP59jjjmGWbNm8dlnn+F2\nuzn99NOZNWsW06dP54UXXoj55xsKVQpE17m3tsKLL8LkybBypc4WFCWWBDLeDs8fzqPrHqXR08j4\n4vE0ehpZWrWUend9n7JZkhW2ofeggw7i+OOP56WXXgLsLOHrX/96vyR2a9eu5ZlnnuEf//gHy5cv\np6amho8//pjHH3+cqqqqgHXv3buXOXPmsG7dOk455RR+//vf9ytz2WWX8Z3vfId169bx5ptvMmbM\nGAoKCnjuuedYu3YtFRUV3HTTTWEl0IsVGb+fgn/nftppEE4gaEUFdHTAsGHQ0GDfn3de/OVVlEzB\n33i7pHIJpQWllBaWAvT8Xb5xOUtcS3rK1tTUhG30rXfXM2P+DH7+u59TckwJT/zvEzy2rP96/xln\nnNETeb169WouueQSsrKyKCsr47TTTgtYd15eXs/WmrNnz+bVV/tk+KG1tZWtW7fyta99DYCCAmsX\n2b9/P7feeiurVq0iKyuLrVu3Ul9fT1lZWVhtGiwZP1Pw7dw7OsKbLXgVifc7KivT2YKixJva5tp+\ndoKSghJqm2ujqs/ryjr7tNl88PYHfFD9AY2tjYybPK5fWd9U1eGO2nNzc3tmHNnZ2f3sEcHqefLJ\nJ9m1axdr1qyhurqa0aNH09bWFrBsPMhopRBt5+5VJN6khvn54SsURVGiY0LJhH4eRc1tzUwomRBV\nfe9ue5fd+3azs3MnR8w6gl9+/5e4znX1277Tn5NPPplnn32W7u5u6uvrqaysjOr+xcXFjB8/nuef\nt/G67e3t7Nu3j+bmZg4++GByc3OpqKjgiy++iKr+aMlopRBt575uHRhjdwH0vowBZ6c8RVHiwIIp\nC2hsa6TR00i36abR00hjWyMLpiyIuK56dz1bGrcAkJ+dz7FnHsvnGz/n5HNOHtCV9aKLLmL8+PEc\nddRRXH755cyaNYuSkuhSWjz++OP8+te/pry8nBNPPJEdO3Zw2WWX8d577zF9+nQee+wxpkyZElXd\n0ZKw1NkxJGaps++6Cz77rP/xww6D22+PRrTUIl2SosWLTG5/urQ90oR463esZ/nG5dQ21zKhZAKn\nTjgVyZI+qTA8uz0DJsSrrKnk0z12e83c7FwA9nfZzZ2POOgIXJNcIa/3ZqHds2cPxx9/PP/85z8T\ntuYfilikzs5oQ/NQ6PgVJZMoLyunvKwc6LuPgW8qjGn50/pd559HaUfrDsqKyqhrrYMuyMmyXaGn\n0xOWK+u5555LU1MTHR0d3H777SmhEGJFRimF1lZ48EG47rrwPIwURUldgqXC8A9kC6Q8Gtsbyc3O\n5ZDiQ9i9bzdtnW1kSzaHlh4aludStHaEdCCjbAoabKYoqUe0S9jB0lt0dXf1OearPESEwtxCDj7g\nYHbu3Um2ZDOxZCITSyYy8oCRHDf2uKjbkSoM1iSQMUpBg80UJfXIzc3F7XZH1ZEFS2+RndV3H+VA\nymNE4QhKC0p7At66TBe52bm8VfcWlTWVCcuwGmuMMbjdbnJzc6OuI2OWjzTYTFFSjxEjRtDQ0EBr\na2vE147sGslHez6iPbud3Kxc9nfvp72rnbbcNrZt29ZTLndfLs2tzeRl927R2dHVQXF2MZPzJtPQ\n1cBHuz4iPzuf3KxcmlqbqNxRydGjjmZEYWy3Co03jY2NFBUVDWqL04xQCsHiEcKNXlYUJT5kZ2cz\natSoqK4dy1i6D+zu4420YMoCGjY2MHZ2r+fN7qzdLK1aSmlBKSUFJTS3NdPY1sjNc29mbNlYflf5\nOxo9jT0R0gCNnkY2tG1giWvJYJuYUDZt2hT15+klI5RCqHiEZMwW1OCtKOHj74a6YMqCHg8kX28k\nL5UbK/u8Ly8r5+a5N/ep45qZ1/RcV9tcy/ji8X2uGUykdLqTEUrBN9jMl+rq5CgFr8F76lRdwlKU\nUKzfsb5nlO+bBO/muTf3UwahCKQ8vEwomcCm3ZvY5t7W47I6tmgsk0dOjlUz0op47tF8CPAYUAZ0\nA78zxvzKr8xlwC3OWzfwf4wx62ItSyrFI0SbgE9RMpHlG5cHTYIXiVIIxbRR03hs3WMU5xVTnF9M\nk6eJ2uZaFkyNPFJ6KBBP76NO4CZjzFRgDvAdEfHffeJz4FRjTDlwF/C7OMqTEkSTgE9RMpVYJ8EL\nxIe7PmTu+LkMLxxOa0crwwuHM3f8XD7c9WHM7pFOxHOP5u3Aduf/VhHZAIwDPvYp86bPJW8BfRf2\nhhhq8FaUyJhQMqGfEXgwSfACUdtcy2EjDuOIg47oOdZtujPWppCQOAURmQTMBN4OUewa4G+JkCdZ\naHZVRbFDMagdAAAgAElEQVR2Av/d04IRyyR4wYh19tV0J+4J8USkCPgH8GNjzPIgZU4DfgOcbIzZ\nE6q+qqoq4906Lxy8iatSge3bIZDo+fkwZkzs75dKbU8Gmdz+VG27p9NDvbue7KxssiWbLtNFV3cX\no4tGU5hTGPSaprYmOro6yMvOY3jB8KBlIfK2RyNTqhKq7S6XK6yEeHFVCiKSC6wEXjbG/CJImXLg\nOeCrxphNYVQbsyypQ51MbjtkdvtTte1LKpcEjAkoLSyNOibA32X1hM4T+Or8rw6qDl+313RigO89\nuVlSxW459AdgQwiFMAFYDlwRpkJIGOkUS5BOsiqZTaxjAgK5rNa317N+x/qYuaxmGvG0KZwEXAHM\nE5Fq53WOiCwSkUVOmR8CBwG/cc6/F0d5IiIRyfNaW+Heewefh0kT/SnpQqzX731dVrMki9LCUrKz\nslm+MeBKtRIG8fQ+Ws0A0xVjzLXAtfGSIVoSFUsQiyA2jXtQArJ1Kyxc2P94WRncc0/CxfGyYMoC\nllYtBeiTcuKamddEVV+gmUe2ZGes51AsyJgsqZEQy1iCYLOBWGVt1bgHJSD798OkSf1fO3YkVSxv\nyonSwlLqWuooLSyNODrZl0Azjy7TlbGeQ7EgI9JcREKsYwmCzQZikbVV4x6UdCTS9ftQRuBAM4+u\n7q6YuqxmGjpT8COWsQTBZgPBOvNIZwsa96AMdbyG5EZPY5/cR97YhkAzj9FFo9VoPAh0puBHLJPn\nBZsNxCpra6ol+lOUWBNO7iP/mcdQ3iozEahS8CNWyfNCLe1E05kHcjtNpUR/ihIPNK114lGlECdC\nzQai6cw13bYSEbm5/Ucd0DtKiRWLFwc2XsfIyykRuY+UvqhSiBPr1lkl8Je/wJQp9jcK0S3tqNup\nEjHjxsGyZfG/z44d1qvJn0AKKQpi7cKqDIwqhThx++2wYgX8/vfwjW8MbnSv+0srmcpAu6YpsUeV\nQpyI1ehe3U6VTEdTUCQWdUmNE7EKKlO3U0VREokqhSgYKGdRrOIQoK+nkvdljLVNKIqixBpdPoqC\ngTyBYhWHAOp2qqQ4ZWWJ8XKKM0MldXYsUKUQIeHYCjSoTMkYkphcL1YESr+9tGrpoHIypTOqFCIk\nHE8gHd0rSvREMmoPVDZSvFHTHV0drPpiFc1tzeRl5/Hgew/y4LkPDrY5aYfaFCIgUltBrPZLUJSY\nsXixTant/1q8OKlieRko11E4ZT2dnojuWdtcS1tnG29++Sae/R6K84sxxvDqlldD7h89VFGlEAGR\negLp5jdKyuENNkuxlNpeAm2aU1pQGnDTnGBlm9qaIrrnhJIJVO+opiCngMLcQkQEEeGgwoMycrMe\nVQoREIknUKz2S1CUTKK2uZaSgpI+x4LlOgpWtqOrI6J7LpiygD2ePRgMxhg8+z20dbUxo2xGRuZY\nUptCBERiK/C3Pbz4Inzxhe6jrCihiCTXUbCyedl5Ed2zvKyc+YfO5/3t79PS3kJJQQkzx8wkPzuf\nMYVjom9MmhK3mYKIHCIiFSKyQUQ+EpHvBSgjIvJrEdksIutFZFa85Ik3ra1w111w992wfXt/28Mj\nj1hFoUtJihKcBVMW0NjWSKOnkW7TTaOnkca2xoAG5GBlhxcMj/i+1x17HUeOPJJTJp7CKRNPIT87\nP+h9hzrxXD7qBG4yxkwF5gDfEZGj/Mp8FTjCeX0bSFtTf0WFzXX0wgvwwAN9bQ9gl5ry83UpSVFC\nEWq7zvU71rOkcglXv3A1SyqXAAQsW5hTGNP7ZhpxWz4yxmwHtjv/t4rIBmAc8LFPsQuAx4wxBnhL\nRIaLyBjn2rShtRWeew7a2+37v/7VBrZ54xRqa6GrC1pa7NLR9dfD/ffrMpKSBBIQbDbYQLBAuY5C\nxRIscS3pU7ZyY2VUcmuOJYvY/jjONxGZBKwCphljWnyOrwTuMcasdt7/HbjFGPNesLqqqqpMu7f3\nDQO3201RnHvf5maor7cdvzGQkwOjR0NJCXR3Q10dZGdDVhZ4PLB3L4wfD6W9S6F0d8OuXTBqlC0X\nCxLR9lQmk9ufrLZ7Oj3Uu+vJzsomW7LpMl10dXcxumh0VCN4L9vd2+ns7iQnq3cc630/pqjvur9+\n74Hb7nK5JJw64m5oFpEi4FngRl+F4D0d4JKQWmru3LkR3b+yshKXyxXRNZHQ2grf/S68/37vyN/t\nhlmz4Ne/htdft0tLEyfamcQrr9i/48bBs8/2XrNiBTzxBHz727GLeo5321OdTG5/stq+pHIJjfl9\njb+NnkZKc0r7jegj4eoXrmZ88XiypHfE1G26qWup4xHXI33KRtP2oZLmIhbfe1xdUkUkF6sQnjTG\nBHL4rQMO8Xk/HtgWT5liTUWFXR4SsTOEHEfNbtlil4nefbfXjfXtt+2sIicHdu7sNTqr+6oyVIjE\npTQSJpRMoLmtuc+xWO3AFknAXCYQT+8jAf4AbDDG/CJIsRXAlY4X0hygOd3sCevWWZfT1la7TFRX\nZzv12lqoqoLjj7cbYP3P/8ARR8All8CFF8LZZ/cqgFil2VaUZOPbede766msqeSZj59hS+OWQXWy\nkXglRUokAXOZQDxnCicBVwDzRKTaeZ0jIotEZJFT5q/AFmAz8HvgujjKExduv90uHW3e3Pt6/304\n4ww499z+Hb9/NPSLL4afOkPTZihB8U9fUVOTlPQV3s770z2f8s/af9LkaSInK4exRWMHNfqOp3dQ\nvGY36Uo8vY9WE9hm4FvGAN+JlwzJIlDSvGCZU194wSqIcNJsD5SyG6ziePBBGySnZBD+eyXn59v3\nMdorOVy8nfd3X/ounaaTgw84mCkjp1BWVEajp5HlG5dH3ZEPxjsolM0gkoC5TEAjmgeBbwfsNRgH\nS5r3858HdkG96y747LOB02yHu72nr+IYNiwmzVSUiCgvK+fQ0kM5ZeIpfQzDyRp9D5Qae8GUBSyt\nWtojY3NbM41tjVwz85qEy5oKqFIYBIFG7pFusBNu6oxwUnb7K46vfz36tinKYEil0bevzQDo+eud\ntXhnN74ziWtmXpOW3kexQJVClAQbucdigx3/GUiw2Yf/bMFfcbS2xqSpihIxqTT6rm2uZXzx+D7H\n/GctGrjWiyqFKAk2cvcd+QdaXgq3bt8ZSDizj0CKo7nZGqUzNI5HSSLhjr4TER+QSrOWdEBTZ0dB\nuJvtRLOfQqCYhXBSdgdSHN3d6t6aMZSV2WjIJ56wrz177N81a5K2gU55WTlLXEt45IJHWOJaEjR1\nRbzjA+LpzjoU0ZlCFEQych/IMBysbt8ZSDh2h0DLVpMm6b7QGcM99/T1QDroILj8cvt/gj2QwmWg\ntf5YEWjW8i8T/oXlG5dz39v3pXUEczxQpRAF4dgNwjEM+xOu7SAQgRRHZaV1VVeUVCSctf5Y4Wsz\nGMgbKdNRpRAFA43c/Tv30lK7z8Jxx4VORhmp55KipDPJWutP1AwlXVGbQhzw79y3b4dt2+w+C6GI\nZLtPRUl3krXWrxHModGZQhzw7dz377e2vuJi+Nvf4JZbgi8FRbLdp6IkjcWLrf3Cn7Iya9twGMiz\nKJiHEthsq/HySFJvpNCEVAoikg1ci81e+pIx5p8+535gjLk7zvKlJb6d+4oVkJdnU2d/8YUuBSlx\nxHcDHd8UFzHcQAfon1LDi4+RLdx1e//4gESs96dSDEUqMtBM4bfAAcA7wK9F5B/GmP90zi0AVCmE\nwLsjW00NjBkTmeFYUSLGZ5ROZaVNz5skol23T8R6v0Ywh2YgpXC8MaYcQETuB34jIsuBSxkg2Z3S\nu9fC1q3w+edw5JFqOM5owlx2GfQ1sbh2kETrWZQojySNYA7OQEohz/uPMaYT+LaI/BB4HdCx7gC8\n8w58+SXk5lq7QlaW/V9jBzKUMJZdYnJNLK4dJNGu2+t6f/IZSCm8JyJnG2Ne8h4wxtwpItuAB+Mr\nWvpz/PF2luC1JyxYYJWBd1+ESNNfKAqvvWbD3N3uvkEogxn9e2cUa9bYTcS9FBbC7NlR1R3tur2u\n9yefkC6pxpjLfRWCz/GHjTG58RMr/QmVCqOiwr6uv143zFEixO2G4cPtaGLSpN5XoGWicPHOKLKz\nYfz43ld2duC6vQZt/5ePQTvaTXHiuZmOEh4DuqSKyEHAN4EpzqENwJ+MMXviKVi6E2qntddft+9f\nfx3++lf4t39LrqyKEhFhzhqiXbf3XuM1BHu3xVTFkBgGckmdirUfvAy8jzUuHwfcKiLzjDEbQ1z7\nCHAusNMYMy3A+RLgCWCCI8dSY8yj0TYk1Qi101pWlrU1FBfDH/4A55yjy0iK4iWQW+qtr9/KIcWH\n0N7VTmt7K583fU5zezPjho3j+uOu5+KjL0622EOGgWYKdwHfM8b82fegiFwE/Bi4KMS1y4D7gceC\nnP8O8LEx5jwRGQV8IiJPGmM6wpI8xQkUiNbaCjffbBNYdnXZVYDPP+87W4g23baSBvjGEfgfD/ca\n73pjOA+H/7Vem0FhYV97xJo1gQ3SScLfLbW9q53PGj5j195dHFJ8CK9ueZUsyeIrw79Ck6eJ/3rt\nvwBUMcSIgZTCdGNMv0/aGPOsiPwk1IXGmFUiMilUEWCYiAjWk6kB6BxAnrSmogL27oXNm6GpySbL\nKyjoO1sIZx9mJU2JxhDsf83CheF34OFeu3p15HLFEX+31I27NzIsfxgdXR28t/09CnIKyMnKoaGt\ngUnDJwFw/7v39ygFT6cnrhHRQx0xxgQ/KbLWGDMr0nM+ZSYBK4MsHw0DVmBtFcOArxtjXhxI4Kqq\nKtPe3j5QsR7cbjdFSR5yd3fDrl12duB2Q1ub/T8nx9ryRGDsWKsk6urs8lJ3t7X1ZQ0iO1UqtD2Z\nDMn2b9wInQHGTjk5MGVKz9uAba+p6TVy+dLYCAccAPv22QfPS1aWPZ6bC+PGxUR8T6eHprYmOro6\nyMvOY3jBcApzCvuU2e7eTmd3JzlZdsza5GlCRMiSLDz7PXbfZwFjDAU5BRgM+7v2M+3gaXg6Pexz\n72Nf1j6yJZsu00VXdxeji0b3u89QJNQz73K5wootG2imcLCI/GeA4wKMCucGITgLqAbmAYcBr4rI\nG8aYllAXzZ07N6KbVFZW4nK5opUxJqxYYfc7+fa34eOP4c9/tr+z/fthxgzYsgXOPtu6sFZU9Hdh\njZZUaHsyGVLt97qNrl7dd+moqAjmz7cd/qJFPYcDtt13puB1bQX79+ST7f9xDGzztRWUHOi4m7Y2\n9vMu6imXW0pJQQkvb3+ZlvYWXJNc/O3Lv+HZ7yEnK4fc7FwmDZ9Ek6eJ4YXDud51PUsql3BY1mF8\nVvxZT32NnkZKc0pZ4loSkzbEe6e4wRCLZ36gcejvsaN4/1cR8PCg7gxXAcuNZTPwOb0eTkMG/812\npk+Hk06CSy6xfydPtikwpk8Pbzc3JUPxuo0WFVljlPcV7QPidW31d28djGvrAPjaCrIki9LCUkoL\nSnu8i7z4u6XOHDOTw0ccTl52HseOOZa2zjb27d/HiIIRNHmaaOlo4frjrgfs0lO2ZPepL1YR0Yna\nKS7ZhJwpGGN+FOyciNw4yHvXAqcDb4jIaOBIYMsg60w5fDfbqa+HRx+FmTPtudJSu4Pi/PnwyCMw\napRVEKB7KShDj0hSWARKlLd843JaO1o5YewJfLjrQzY3bmZE4QhuOO6GHnvChJIJdLV09akrVhHR\nmbIPw2BSZ/8ncF+wkyLyJ8AFjBSROuAOIBfAGPMQ1rNpmYh8gF2OusUYs3sQ8qQc/gFsbW3W22iW\nY4nZvt12/KtW2fedndaN9ZNPbJ4kTYmhxBRfbyTfGUaC7C6DSWHhVRLe0frUg6f2RDyv27mO9TvW\nU15WzoIpC1j71loaPY0xj4hO5E5xyWQwSiGk0cIYc+kA57cBZw7i/imPfwBbQ4M1ML/9tv19rllj\nFUFdnc2cetBBMG+etT984xuqDJQY42sriMSLKUbEIoXFQKP18rJythZtpTSnNOYZUDMlL9NglEJw\ntyUF6B/ANmGCfR12GBxzjPU62rDBeh59/rn1RHr0UTj6aE2xrQShqMj6M3txu/ulmEhVYpGyOpzR\nemFOYUyMyv5kSl6mgSKaWwnc+Qsw9P27BkmondTuustu0dncbH/nbrdVENnZ1uawZo2mwFB88C79\nHH54/+PRpNBes8a+Zs/uX18cGWzKat/Rer27ng27N7Bz704OPvDgniWkeJEp+zAMZGgelihBMo0b\nb7TLRiecYJeXWlrsdp1nnGF/+01NA6fA0OjnDCJcN1GvAnC5eqOWV6+2nf38+b3lvDuzJXEjnnDw\ndwGdNmoaKzatYPe+3XxQ/wFZkkVOVg5ji8b27NAWTzJhH4ZBhEYpg8Hf3rB9u41b+OIL2LTJeiJ5\nU2CEquONN+xfRQF6XVfz83vdTL1T0TQjkAvoik0rOH/y+Wxt3Uqn6WR44XBOPOREJo+cHNC9VYmc\nwdgUlEHgb2/Y6KQWXLvWBpIWFdng0hdeCLyE5B//oPYHZagRzKj84a4PObT0UE6ZeIqNbnbosS2U\nJEXcIYMqhSQRKmHeiBF2oNfebj2W3O7+Hb5v/ENDg8YzKEOPUEblTPEESgaqFFKIYHsw+Hb4ra1w\n3312ack/+llnC0OEcPdWDlRu9WqbcfHII0Pf47XX7LW+2VID3SOJhOr4Q3kCNWxsSJbIQwJVCilE\nsD0YfAPYKipsLqXiYuveChr9POQId2/lQOWqq/vbD4qKbFnf63fssArA//oE7N/sS6hcQqE6/lCe\nQJUbKxPahqGGKoUUIpQLK1gX1h//2LqtfvklfPqpjXr2otHPQ5DXXrOeB52d1hPBm+a60PEI9+/U\nvQqgvb23gz/8cJvwLsnBa/4E2kzH60Hk9fIJ5QKaCZ5AyUCVQhrxwANWMZx8st1vYbBZVJU0wO22\nngfFxTZPSkeHfTU0WCXx/PO2nDdbqjdj6qRJKe9uGk4uIe34E48qhTRh2zZYvtwmzNu0ybqhqx0h\nA+nosDsztbZapeC1KXR09Hok+Ae4pSiZkkso3dA4hTThgQdsH3DAATZ/0rZtvXYEJQPp7rbriAUF\n9pWXN7hU2klgQskEmtua+xxTD6LkozOFNKC1FV55xf7uP/nEuqyuWQPHHad2hCGJfzbTDmfb8ry8\n3v/BLiu1tdn/OzpsGLzbHV6qirIym7fd4+l7vLDQejUlwAMpU3IJpRuqFNKAigqYNs3+/tesgfJy\nOzhUm8IQxd8gXF1tZwHQG+UINkjFuwVnUxNceKFVJvfcA5WVA98jXC+nOJEpuYTSDVUKacA779h+\nobXVup/qLCGDKCuz64V1dfb93r12VmCMnTmkOWpITj1UKaQBxx8PL79s924+/nh49VW44ALNoJoR\n3HNP4IA1717N3jTa6m0QlFTfVznVUKWQongzoF5xBTz3nHU737PHxiaEk0FVGaJ4FUSgaOY02Vch\nkQwUC5HqJEOhxU0piMgjwLnATmPMtCBlXNgtPXOB3caYU+MlTyoSKvW1NwNqSwvU1lqbYnc3vP8+\nHHxwbwZVnS1kKCmSiiLVSed9lZOl0OI5U1gG3A88FuikiAwHfgOcbYypFZGD4yhLSuLt+KdO7Wsb\n8GZAnTTJOoiUOqlfmpuhsdG6pYbKoKokgHDzEyVLFpdrYC8iXy8n/+NDhHSOhUiWQoubUjDGrBKR\nSSGKfBNYboypdcrvjJcsqUio1NfexHgtLTY2YeJEqyBeeQVKSux2nZMmBc+gqiSAJHvu9CGQLPn5\ngZWWLxkw20jnbKrJUmhiTPy2WnaUwspAy0ci4l02OhoYBvzKGBNwVuFLVVWVaW9vD1sGt9tNUQr2\nmt5RvzdFdmmp7fC7u62jSXa2dTTp7LTHcnOtgsjKgpwc643oe10gUrXtiSKu7a+p6U1n60t7e+Jz\nCgWQxZ2fT1FLS9LzG4XC0+mhqa2Jjq4O8rLzGF4wnMKcwe/y6/u9ezo91Lvryc7KJluy6TJddHV3\nMbpodEzuFU+2u7fT2d1JTlbv2N37fkzRmIDXhHrmXS6XhHPfZBqac4DZwOnY/Z6rROQtY8ymUBfN\nnTs3optUVlbicrmilTEuBNs34ec/h9dftzOFiRN7y3/xhVUIOQG+rcMOC55ILxXbnkji2v5gCeUG\ns8VltEtS/rK89hqV3/wmrl/8wibKCreeROC0cX1uA0tLPqK0O4+S7jyai3JpnH1UTNbL/b/3dPU+\n6rEp5Jb2Bvftb+Tm44N/RrF45pOpFOqwxuW9wF4RWQUcA4RUCkOBYPsmPP647eBPOaX/KsSRRw6c\nRVVJc2K1JOV22xFEUVHf+mpqkm8Lcdq4nBpKGUkpdrRe2tQEznaase6w0zUWIlnBfclUCi8A94tI\nDpAHnAD8MonyJIxg+yY88ICdReTmBh9shvJYUpQBiZMtJNLReC3NjKe4z7F0MQAnkmQotHi6pP4J\ncAEjRaQOuANrQ8AY85AxZoOIvASsB7qBh40xH8ZLnlQi0Ih/0yYbmFZSYgPVPvvMLg35E8xjSUkw\nqeS54y+L222NUf6jhjVrbK6j6uq+xweZWTUa18kJlNCIp2emAPE1AMd6CSldl6TCIZ7eR5eGUeZn\nwM/iJUM6cdttNpvBAQfY3+2tt8LTT/ct4++xdOyxdslJZw1JINlr8774y7JwofVAmD+/73GPxz4o\n3jxKXrxR0VESjevkAqawlCoASiigOas9bsnwYu3vn+4BcQOhqbNTgE2bbOqKYcPs+2HDemcLvnht\nEcOG2b8PPGBnDZo+W0kmtc21lBT0dYEbaCmonDJuZi6lFFJHC6Xd+XHrVH2VVpZkUVpYSqljv0iF\n+lINTXORAnhnCTk51lV12DD73ne24J0leFcnSkttYNv8+brZzpAhVktSZWV9t+P0UhgfF8yIYgF8\n2lgOlDOp93icRtmx9vdP54C4cFClkAKsWWP/7tplZwBtbVZBvPtubxl/j6Xt262b6u7dNo12RYXa\nGNKeWC1JeVNn+3srLFwImzf3Xy4Kdw+GIES0L0ISlt1iHcCWzgFx4aBKIQXYsqU3dqGoyP5Gf/7z\nviN/X4+l/futIikshPp6OOEEnS0oYeJvZ4DePRiiJNX3RYj1Zj5DfXMgVQopgq+9oKGh/8jf12Np\nxQqbSt83wM27NafOFjIcbxyCy2VnBl7KyuLqMZXKsQCxVlqprgQHiyqFFMDfXlBWFnrkHyzOQTfd\nUXriEPLz+weuRRtpPQSItdJKZSU4WFQppADBIpyDjfw1sjnDSHYUspJRqFJIAXTkr4QklTKyKkMe\nVQopgI78FUVJFTR4TVEURelBZwqKMhhSbb3f62E0aVLf5aUhtJuaEl9UKSjKYEi19X6vIgoUvKYo\nYaBKQVFSnVTKyKoMeVQpKEqqkw5up6m2jObHUE51HWtUKSjKUCcRHXaqLaP5MNRTXccaVQqKMtRJ\n4Q47EUSz30Mmo0pBUQaDrvenPEM91XWsUaWgKIMhBdbLldAM9VTXsSZuwWsi8oiI7BSRkPsui8hx\nItIlIhfHSxZFUTKXBVMW0NjWSKOnkW7TTaOnkca2RhZMWZBs0VKSeEY0LwPODlVARLKBnwIvx1EO\nRVHijXcZzf+VAsto3lTXpYWl1LXUUVpYqkbmEMRt+cgYs0pEJg1Q7AbgWeC4eMmhKEkn2e6aGzfC\n6tWwZw90dvYeLyiwey7EQo4UX0YbyqmuY40YY+JXuVUKK40x0wKcGwf8LzAP+INT7pmB6qyqqjLt\n7e1hy+B2uynK0O3IMrntkELtr6npzYvuS3t7YK8ggK1b7RZ7/uTmwrhxA96yT9u9929stPu8euns\ntJt9h5IjDUmZ7z0JhGq7y+WScOpIpqH5PuAWY0yXSFiyAjB37tyIblJZWYnL5YpMsiFCJrcdUqj9\nCxcGdwldtizwTGL1ajuC998+M8zNcvq03Xv/55+H4cN7CzU1wYUXDrkNeFLme08CsWh7MpXCscBT\njkIYCZwjIp3GmOeTKJOiJJ5AcQTV1Xaz7lQk2cthSlxJmlIwxnzF+7+ILMMuH6lCUJRUJ8OD4YY6\ncVMKIvInwAWMFJE64A4gF8AY81C87qsoiqJETzy9jy6NoOzCeMmhKEkn2qjnbdusHcAXt9su30Sy\nTOO9f1cX1NX1Hi8sTBm3USV10IhmRYk399wTeB1+xw57PBBFReDx9D9eVhZ4PX+g+ytKmKhSUJRE\nEGodPtBM4vDDobnZegcFukZR4oQqBUVJNsFG8gsXJlSMsNEkgEMaVQqKokSGLkcNaeKZ+0hRFEVJ\nM3SmkIG0tsKDD8J111l7ppIkXnvN2hr8l4m8QWDxWKbRwDNlAFQpZCAVFfDGGzB1Kpx3XrKlyRAC\ndfA7dtjj/gZob7l4dNIaeKYMgCqFDKO1FV58ESZPhpUr4bTTdLaQEAJ18MFyIilKElGbQoZRUQEd\nHTBsmP1bUZFsiRRFSSVUKWQQ3lmCd0m6rMzOFlI175qiKIlHlUIG4Z0leFP75+frbEFRlL6oTSGD\nWLcOjOlvU6yuVoNzUkhGEJgGnikDoEohg7j99mRLoPQhGS6gaeR2un7HepZvXE5tcy0TSiawYMoC\n3VIzAahSUBQl6fgrgGmjprFi0wpKC0oZXzyeRk8jS6uWcvPcm1UxxBm1KSiKklTW71jP0qqlNHoa\nexTAXavuoqu7i9LCUrIki9LCUkoLSlm+cXmyxR3yqFJQFCWpLN+4nNKC0j4KYH/3fupa6vqUKyko\noba5NklSZg6qFBRFSSq1zbWUFJT0OTbqwFHs2rerz7HmtmYmlExIpGgZSdyUgog8IiI7ReTDIOcv\nE5H1zutNETkmXrIoipK6TCiZQHNbc59j44rGkZudS6OnkW7TTaOnkca2RhZMWZAkKTOHeM4UlgFn\nhzj/OXCqMaYcuAv4XRxlURQlRVkwZQGNbY19FEBOdg63/8vtlBaWUtdSR2lhqRqZE0Q892heJSKT\nQhNR2CoAAAV2SURBVJx/0+ftW8D4eMmiKErqUl5Wzs1zb+7jfXTNzGsoLyvnYi5OtngZhxhj4le5\nVQorjTHTBih3MzDFGHPtQHVWVVWZ9vb2sGVwu90UZWjGt0xuO2R2+7Xt2nZ/XC6XhFNH0uMUROQ0\n4Brg5HDKz507N6L6KysrcblckQs2BMjktkNmt1/b7kq2GEkhFm1PqlIQkXLgYeCrxpg9yZRFURRF\nSaJLqohMAJYDVxhjNiVLDkVRFKWXuM0URORPgAsYKSJ1wB1ALoAx5iHgh8BBwG9EBKDTGHNsvORR\nFEVRBiae3keXDnD+WmBAw7KiKIqSODSiWVEURelBlYKiKIrSgyoFRVEUpQdVCoqiKEoPSQ9eUxQl\nCIsXw4svgsfT93hhIfzrv6bVLmpK+qBKQVFSlR07IDsbxvulBWtqsucUJQ7o8pGiKIrSgyoFRVEU\npQdVCoqiKEoPqhQURVGUHtTQrCipSlkZdHVBXd8N7CkstOcUJQ6oUlCUVOWee9TtVEk4unykKIqi\n9KBKQVEURelBlYKiKIrSgyoFRVEUpQdVCoqiKEoPqhQURVGUHlQpKIqiKD2oUlAURVF6EGNMsmVQ\nFEVRUgSdKSiKoig9qFJQFEVRelCloCiKovSgSkFRFEXpQZWCoiiK0oMqBUVRFKUHVQqKoihKDxmh\nFETkZyKyUUTWi8hzIjI82TIlChG5REQ+EpFuETk22fIkAhE5W0Q+EZHNIrI42fIkEhF5RER2isiH\nyZYl0YjIISJSISIbnGf+e8mWKVGISIGIvCMi65y2/yjaujJCKQCvAtOMMeXAJuD7SZYnkXwILABW\nJVuQRCAi2cADwFeBo4BLReSo5EqVUJYBZydbiCTRCdxkjJkKzAG+k0HffTswzxhzDDADOFtE5kRT\nUUYoBWPMK8aYTuftW8D4ZMqTSIwxG4wxnyRbjgRyPLDZGLPFGNMBPAVckGSZEoYxZhXQkGw5koEx\nZrsxZq3zfyuwARiXXKkSg7G4nbe5ziuqdBUZoRT8uBr4W7KFUOLGOOBLn/d1ZEjHoPQiIpOAmcDb\nyZUkcYhItohUAzuBV40xUbU9J7ZiJQ8ReQ0oC3DqNmPMC06Z27BTzCcTKVu8CaftGYQEOKYJvjII\nESkCngVuNMa0JFueRGGM6QJmODbT50RkmjEmYtvSkFEKxpj5oc6LyLeAc4HTzRDLAjhQ2zOMOuAQ\nn/fjgW1JkkVJMCKSi1UITxpjlidbnmRgjGkSkUqsbSlipZARy0cicjZwC3C+MWZfsuVR4sq7wBEi\n8hURyQO+AaxIskxKAhARAf4AbDDG/CLZ8iQSERnl9aoUkUJgPrAxmroyQikA9wPDgFdFpFpEHkq2\nQIlCRL4mInXAXOBFEXk52TLFE8eh4HrgZayh8c/GmI+SK1XiEJE/AVXAkSJSJyLXJFumBHIScAUw\nz/mdV4vIOckWKkGMASpEZD12YPSqMWZlNBXpfgqKoihKD5kyU1AURVHCQJWCoiiK0oMqBUVRFKUH\nVQqKoihKD6oUFEVRlB5UKShKmIiIO8CxJSKy1XF//FRElvsmYROR651srUZERiZWYkWJHFUKijJ4\nfmmMmWGMOQJ4GnhdREY55/6JDST6ImnSKUoEqFJQlBhijHkaeAX4pvP+fWNMTVKFUpQIUKWgKLFn\nLTAl2UIoSjSoUlCU2BMoU6uipAWqFBQl9szE5l1SlLRDlYKixBARuQg4E/hTsmVRlGhQpaAo4XOA\nk3nU+/pP5/h/eF1Sgcuxe+XuAhCR7zpZascD60Xk4STJrihhoVlSFUVRlB50pqAoiqL0oEpBURRF\n6UGVgqIoitKDKgVFURSlB1UKiqIoSg+qFBRFUZQeVCkoiqIoPfz/CVTKuZlUPwcAAAAASUVORK5C\nYII=\n", "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "from matplotlib import pyplot as plt\n", "\n", "def plot_step_lda():\n", " \n", " ax = plt.subplot(111)\n", " for label,marker,color in zip(\n", " range(1,4),('^', 's', 'o'),('blue', 'red', 'green')):\n", "\n", " plt.scatter(x=X_lda[:,0].real[y == label],\n", " y=X_lda[:,1].real[y == label],\n", " marker=marker,\n", " color=color,\n", " alpha=0.5,\n", " label=label_dict[label]\n", " )\n", "\n", " plt.xlabel('LD1')\n", " plt.ylabel('LD2')\n", "\n", " leg = plt.legend(loc='upper right', fancybox=True)\n", " leg.get_frame().set_alpha(0.5)\n", " plt.title('LDA: Iris projection onto the first 2 linear discriminants')\n", " \n", " # hide axis ticks\n", " plt.tick_params(axis=\"both\", which=\"both\", bottom=\"off\", top=\"off\", \n", " labelbottom=\"on\", left=\"off\", right=\"off\", labelleft=\"on\")\n", "\n", " # remove axis spines\n", " ax.spines[\"top\"].set_visible(False) \n", " ax.spines[\"right\"].set_visible(False) \n", " ax.spines[\"bottom\"].set_visible(False) \n", " ax.spines[\"left\"].set_visible(False) \n", " \n", " plt.grid()\n", " plt.tight_layout\n", " plt.show()\n", " \n", "plot_step_lda()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The scatter plot above represents our new feature subspace that we constructed via LDA. We can see that the first linear discriminant \"LD1\" separates the classes quite nicely. However, the second discriminant, \"LD2\", does not add much valuable information, which we've already concluded when we looked at the ranked eigenvalues is step 4." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "
\n", "
" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "# A comparison of PCA and LDA" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "[[back to top](#Sections)]" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "In order to compare the feature subspace that we obtained via the Linear Discriminant Analysis, we will use the `PCA` class from the `scikit-learn` machine-learning library. The documentation can be found here: \n", "[http://scikit-learn.org/stable/modules/generated/sklearn.decomposition.PCA.html](http://scikit-learn.org/stable/modules/generated/sklearn.decomposition.PCA.html). \n", "\n", "For our convenience, we can directly specify to how many components we want to retain in our input dataset via the `n_components` parameter. " ] }, { "cell_type": "markdown", "metadata": {}, "source": [ " n_components : int, None or string\n", " \n", " Number of components to keep. if n_components is not set all components are kept:\n", " n_components == min(n_samples, n_features)\n", " if n_components == ‘mle’, Minka’s MLE is used to guess the dimension if 0 < n_components < 1, \n", " select the number of components such that the amount of variance that needs to be explained \n", " is greater than the percentage specified by n_components" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "
\n", "
" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "But before we skip to the results of the respective linear transformations, let us quickly recapitulate the purposes of PCA and LDA: PCA finds the axes with maximum variance for the whole data set where LDA tries to find the axes for best class seperability. In practice, often a LDA is done followed by a PCA for dimensionality reduction." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "
\n", "
" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "\n", "![](https://raw.githubusercontent.com/rasbt/pattern_classification/master/Images/lda_1.png)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "
\n", "
" ] }, { "cell_type": "code", "execution_count": 18, "metadata": { "collapsed": false }, "outputs": [], "source": [ "from sklearn.decomposition import PCA as sklearnPCA\n", "\n", "sklearn_pca = sklearnPCA(n_components=2)\n", "X_pca = sklearn_pca.fit_transform(X)\n", "\n", "def plot_pca():\n", "\n", " ax = plt.subplot(111)\n", " \n", " for label,marker,color in zip(\n", " range(1,4),('^', 's', 'o'),('blue', 'red', 'green')):\n", "\n", " plt.scatter(x=X_pca[:,0][y == label],\n", " y=X_pca[:,1][y == label],\n", " marker=marker,\n", " color=color,\n", " alpha=0.5,\n", " label=label_dict[label]\n", " )\n", "\n", " plt.xlabel('PC1')\n", " plt.ylabel('PC2')\n", "\n", " leg = plt.legend(loc='upper right', fancybox=True)\n", " leg.get_frame().set_alpha(0.5)\n", " plt.title('PCA: Iris projection onto the first 2 principal components')\n", "\n", " # hide axis ticks\n", " plt.tick_params(axis=\"both\", which=\"both\", bottom=\"off\", top=\"off\", \n", " labelbottom=\"on\", left=\"off\", right=\"off\", labelleft=\"on\")\n", "\n", " # remove axis spines\n", " ax.spines[\"top\"].set_visible(False) \n", " ax.spines[\"right\"].set_visible(False) \n", " ax.spines[\"bottom\"].set_visible(False) \n", " ax.spines[\"left\"].set_visible(False) \n", " \n", " plt.tight_layout\n", " plt.grid()\n", " \n", " plt.show()" ] }, { "cell_type": "code", "execution_count": 19, "metadata": { "collapsed": false }, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAY0AAAEUCAYAAADXzmpaAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJztnXl8VOW5+L9PNhIIWViUJULcQSFsVaFaGxWXUgsKWPVa\nKy6311K9t7+W23JdKq311uult7bVqtfWYqt1uYiKaKuiiYpGURSCICJiDFsQzTqYjeT9/fGeSWaG\n2ZLMzJkJz/fzmU9ytvc8c+Y95znP+yyvGGNQFEVRlGhIc1sARVEUJXVQpaEoiqJEjSoNRVEUJWpU\naSiKoihRo0pDURRFiRpVGoqiKErUqNJIECLydxG5wmUZviYiH6ZKu8mGiCwTkV/GqK3DReRVEWkS\nkV+LyA0i8sdYtO0GIuIRkaP62EbMroGIGBE5JhZtKf70G6UhIlUi0ux03r0i8mcRyfXZfq7PTbpP\nRF4RkdkBbZQ6ne0nPTx3qYjsDLePMeYbxpgHe/atYosx5jVjzPF9bSfwhoxVu7Ekmt8kwvELRGRN\nLGUK4HvA50CeMebHxpj/NMZc05uGRKRcREIeKyLHicjTTr+vFZHnRSSmv5cxJtcYs72PbfT6GijR\nEamvREO/URoO3zLG5AJTgZOAmwBEZD7wf8BfgCLgcOBnwLcCjr8CqHX+xgSxJOQ6i0hGIs6jxISx\nwGYTRXZtDH7XAmAlcDy2768Fnu5jm4D2uUMSY0y/+ABVwEyf5f8GVgECVAP/HuH4gUATcAnQBnyl\nB+cuBXb6LJcDtwGvA83AMc66a5ztxwCvAA3Yt83HQrRbDBjsW+luYA/wY5/tS4DlwENAI3ANMAC4\n09l/t/P/gBByjgKeAPYBnwD/6rMtHbgB+Ni5LuuAI4BXHZn2Ax7g4iDtjne+bz2wCZjts20ZcDfw\nrNPuW8DRYa7tbKeNeqfN8QG/+SKg0rmWjwHZwCDnunc6Mnqc7xry2gScczzQAnQ4x9ZHIzswDngR\n++LxIfDtEN9pGdCO7WceYKbzWz4U8Ltfje27rzrf6yHgC+davI1VALc5crY4bd0VRX8d4rQ/NIx8\n9zrfpQnbV8f6bDfAD4CPgE981h0T5XU60ec67QVu8OnPgdcgVN8/GahwrsUe4C4gK0DGY8J8/z87\n7dYBT/ls+2dgmyPbSmBUQJsLne/dBNwKHO3I0Qg87pUB557A3kOfY/vqZT5t5WNfYvcBn2JfcNOc\nbQuANcBSR75PgG8EHPsn53vvAn4JpEc6NlhfwT4ffwN8hr2HKoEJYftPrB7abn/wURrYh9sm50cd\n5/zYR0Y4/nLnR0gHngF+F7C9EvinEMeWcrDSqMbeHBlAJv5K4xHgRqyllw2cFqLdYkf2R7APwolO\nJ/N+zyXYh88FTls5wC+AN4HDgOHAG8CtgXI6+6/DWlxZwFHAduBcZ/u/Axuxb6cCTMJ5yBBwQwa0\nm4m96W5w2j0Te4Md7/NAqcXe9BnAw8CjIb7/cVjldLbT7k+ctr03ZhX2rXkU9kHwAXBtsN/EWRfy\n2gQ59wJgTcC6kLI7v88O4Epn21Tsw+LEEO0vA37ps7yEgx+Yf3HazQH+BdsvB2L76DTs0Bb49K0o\n75ULgD1hti9zfrPTsYr2t77XwpHtReea5wT2iQjXaTCOAsD2/cHAKWGuQai+Pw2Y7rRf7Pz2PwyQ\nMZTSeBb7glGI7Vdfd9af6fxmU53v/Xvg1YA2VwJ52Hu7FXgJe+/kA5uBK3z63wHgf5y2vo7ty977\n4C9Ya2+wI/9W4GqfvteOVWDpwPexCk6c7U8B9znX5TDsPfAvUR7r11eAc7HPgQLsfT4eGBm2//T1\nYZ0sH+wDxIN98/gU+AP2ZjvV+bGzIxy/GrjT+f9Sp4NmRnnuUg5WGr8I2Kfrx3I6zP8CRRHaLXZk\nH+ez7g7gTz432asBx3wMzAroFFWBcgKnANUBx/4H8Gfn/w+BOSHkCqc0vgbU4Lw1OeseAZY4/y8D\n/uizbRawJcR5bgYe91lOw75Zlfr85t8JuDb3BvtNIl2bIOdeQHClEVR2rMX1WsD+9wG3hGh/GZGV\nxlE+26/CKrmSIG119a0o+mqRcw0vDbPPMnwUOZCLfUM9wuf3PzNUn4hwnS4F3gtx3mDXIGjfD3Ls\nD4EnQ/VRn/UjsRZoYZBtfwLuCPje7UCxT5un+mxfB/zUZ/nXdD9DSrFKY5DP9sexfTodq3BO8Nn2\nL0C5T9/b5rNtoHPuEVjrshVHWftc07JIxwbrK1hFuRWrgNMCr0mwT3/zaVxgjCkwxow1xiw0xjRj\nzXmwnSUoInIEcAb2jQjsG0A28M0+yLIjzLafYLX6WhHZJCJX9aCtT7Fv1qHOM8rZJ9T+XsYCo0Sk\n3vvBWgeHO9uPwD5ke8ooYIcxpjNAhtE+yzU+/3+JvTlDtdX1XZw2d/SyrYPaI/S1CUeo840FTgm4\nnpdhb/Te4vvb/hV4HnhURHaLyB0iktmTxkRkOPAC8AdjzCPRntsY48FaDuH6XSChrlNP+1XQvu84\n91eJSI2INAL/CQyLor0jgFpjTF2QbYH9zYN9fvj2t70+/zcHWfbtf3XGmP1B5B+GtcID+2LQfm2M\n+dL5NxfbzzKBPT797D6sxRHp2IMwxryMHaa6G9grIv8rInnB9vXS35RGMD7Edrx5Yfa5HHstnhGR\nGuwwTTbw3T6c14TcYEyNMeafjTGjsG8Yf4gQHniEz/9jsOZmqPPsxnasUPt72YEdjy7w+Qw2xszy\n2X50GJlCsRs4IsD5Pwb7dtubtrq+i4gI9lpE01aw6x/ttQl1fDh2AK8EXM9cY8z3e9hOUBmMMe3G\nmJ8bY04AvgqcT3f/jCiriBRiFcZKY8xtUZy7q885UYhDCN/voqWn/SpU378H2AIca4zJw77wSJTn\nHyIiBUG2Bfa3QcBQetd3AQqdNrx45f8ca8EE9sVozrMDa2kM8+lnecaYE6OU6aDfzRjzO2PMNOyQ\n23HYoemQ9HulYawN9iPgZhG5UkTyRCRNRE4Tkf91dvsu8HNgss9nHvBNERkaa5lE5CIRKXIW67A/\nZEeYQ24WkYEiciJ2zPyxMPs+AtwkIsNFZBjWZ/FQkP3WAo0i8lMRyRGRdBGZICInOdv/CNwqIsc6\nEWAlPtdiL3YcNxhvYcdufyIimSJSio1SezSMzKF4HPsbnOW8Vf8Ye8O8EcWxe4GhIpLvsy7aa+M9\nvkhEsqKUdRVwnIhc7nzvTBE5SUTGR3l8WETkDBGZKCLpWKdrO919JtzvgfPm+DzwujFmcZSnnOXc\nI1lY3+BbxphI1kU0rAJGiMgPRWSAiAwWkVPC7B+q7w/GXgePiIzDjt1HxBizB/g79kWt0PmdTnc2\n/w24UkQmi8gArPXyljGmqsffspufi0iWiHwNq+j/zxjTge3btznffyz2GRWqLwbK/wLwa59n2dEi\n8vUo5fHrK04fPcW5v/bTHQASkn6vNACMMcuxY85XYTX9XmzEwdMiMh07fnq3YwF4PyuxTtdLAZxh\npMtiJNJJwFsi4sE61v7NGPNJmP1fcWR5CVhqjHkhzL6/BN7BOu43Au866/xwOu63sAryE+zbzx+x\nDj2wDrzHsR20ETvem+NsWwI86JjH3w5otw0b8fQNp80/AN81xmwJI3NQjDEfAt/BOiQ/d+T9lnOO\nSMduwSqJ7Y6co4jy2ji8jA2mqBGRz6M4XxNwDjb6bjd2iOC/sE7QWDACGynXiHX6vkL3Q+a3wHwR\nqROR3wU59kJsn7tSbB6T9zMmzPn+BtyCHZaahh1q6zPOdTob+1vWYCORzghzSKi+vwj4J6zD/n7C\nv0gFcjlW6W7BRg390JHtJazP4Qmss/5o7O/ZW2qwL4W7sUPf1/rcB9djH9LbsdFOfwMeiLLd72KH\ntzY77S8nzPB7AIF9JQ97/eqwQ2RfYCOvQuL1qCtJiIgUYx/omcaYAzFo70ysg7JPmbtK/0ZElmGD\nCG5yUYZiYtj3E41jYT9kjCmKtG+qcUhYGkoXE7A3oqIoSq/QbM5DBBH5LXbY6Aq3ZVEUJXXpj5aG\nScSnoqIi7ucwxnzixEa391VWY8y/GmOKjTGvJOoaJet1VVnDf4wxVxhjbnRT1lj1fbeuqzGmzBgz\n2m35etgHosJVpSEiD4jIZyLyfojtpSLSICLrnc/PEi1jKFpbW90WIWpU1vigssYHlTU+xEpWt4en\nlmETS/4SZp/XjDHnJ0YcRVEUJRyuWhrGmFexIX2KoihKCuB6yK0TWrfKGDMhyLZSbMz0Tmys8yJj\nzKZw7VVUVJhEmIwej4fc3HAVK5IHlTU+qKzxQWWND5FkLS0tjSajPumVRh7QaYzxiMgs4LfGmGMj\nNJmQL1ReXk5paWkiTtVnVNb4oLLGh0BZOzo6qK2tpb293T2hQlBTU8OIEX0pL5Y4vLJmZmYyZMgQ\n0tPTA3eJSmm47dMIizGm0ef/50TkDyIyzBgTMUNXUZT+QW1tLdnZ2QwbNgxbfix5aGtrY9Sonta8\ndIe2tjZGjhyJx+OhtraW4cOH96qdpA65FZERTpE6RORkrLxfhD9KUZT+RHt7O7m5uUmnMFIRESE3\nN7dPVpurloaIPIKtOz9M7HzOt2DL/mKMuReYD3xfRA5gyw5fYtweT1MUJeGowogdfb2WrioNY8yl\nEbbfhQ3JVRRFUZKApB6eUhRFSQZuu+02TjzxREpKSpg8eTJvvfVWyH2XLVvG7t122o+9nr2UV5Xz\n9JanKa8qZ69nb8jjUoWkdoQriqK4TUVFBatWreLdd99lwIABfP7557S1ha7Ov2zZMiZMmEB6Xjpv\n7HiD7Ixs8gbk0dzezBs73uCrR3yVw3MPD3l8sqOWRhLT1AR33AEej9uSKEpqEct7Z8+ePQwbNowB\nA+zUKMOGDWPUqFGsW7eOiy++mGnTpnHuueeyZ88eli9fzjvvvMNll13GqSefihwQNq/dzPfO/x4/\n+NYPuOfme9iwawMAixcv5oQTTqCkpIRFixYB8Mwzz3DKKacwZcoUZs6cyd69yWeZqNJIYsrK4LXX\n7F9FUaInlvfOOeecw44dOzjuuONYuHAhr7zyCu3t7Vx//fX84Q9/YN26dVx11VXceOONzJ8/n698\n5Ss8/PDD/HrFr8nJzOG//v2/+Nnvf8YD/3gAOuGxZY9RW1vLk08+yaZNm6isrOSmm+zUJaeddhpv\nvvkm7733Hpdccgl33HFH379AjNHhqSSlqQmefRaOOw5WrYIzzoAUSTxVFFeJ9b2Tm5vLunXreO21\n1ygrK+Piiy/mpptu4v333+fyyy8nKyuLjo4ORo70nzwvPzufTZWbGFk0kiOOslOdnzHnDP7+yN/J\ny8sjOzuba665hm9+85ucf74tr7dz504uvvhi9uzZQ1tbG0ceeWTvBY8TamkkKWVl0NYGgwfbv2pt\nKEp0xOPeSU9Pp7S0lJ///OfcddddPPHEE5x44ok899xzrF+/no0bN/LCC/6zMI8fNp7WA610mA6M\nMTS3N9PW0cbgrMFkZGSwdu1a5s2bx6pVqzjvvPMAuP7667nuuuvYuHEj9913Hy0tLX0XPsao0khC\nvG9K3uoEI0bYNyb1bShKeOJx73z44Yd89NFHXcvr169n/Pjx7Nu3j3fffRewCYibNtmyeIMHD6ap\nqYnDcw9n3unz2Ld7Hx9u/ZCczBw2v7SZc846B4/HQ0NDA7NmzeI3v/kNGzZYP0dDQwOjR48G4MEH\nH+y90HFElUYS4n1TcvxuDBig1oaiREM87h2Px8MVV1zR5bTevHkzv/jFL1i+fDm33347kyZNYvLk\nybzxxhsALFiwgGuvvZbJkydz2KDD+Ntf/sZ9i+/j+m9dz6ABg7j22mtpamri/PPPp6SkhNNOO43/\n+Z//AWDJkiVcdNFFTJs2jWHDhvX1csQF9WkkIRs2gDFQVeW/fv16+Na3XBFJUVKCeNw706ZN61II\nvgwbNozHH3+c4uJiv/Xz5s1j3rx5XctnnXUW7733nt8+I0eOZO3atQe1OWfOHObMmdM7QROEKo0k\n5Oab3ZZAUVITvXfijw5PKYqiKFGjSkNRFEWJGlUaiqIoStSo0lAURVGiRpVGgtF6UoqipDKqNBKM\n1pNSlNSitLSU559/3m/dnXfeycKFC/vU7s9+9jNWr17dtRxtGfXy8vKusiNuoEojgQTWxFFrQ1GS\nn0svvZRHH33Ub92jjz7KpZeGnUMOAGMMnZ2dQbf94he/YObMmYBVGG/seIPm9ma/MuqxmH/jwIED\nfW7DF1UaCUTrSSlKnFm8GBYsOPizeHGvm5w/fz6rVq2itbUVgKqqKnbv3s1pp53Gfffdx0knnURJ\nSQm33HJL1/bjjz+e7373u0yYMIEdO3awYMECJkyYwMSJE/nNb34D2Mzx5cuXA/DkS09y42U3ct3s\n61h4wUJMqyGtI40FVy5g4sSJTJkyhbIgD4za2louuOACSkpKmD59OpWVlYDNLL/88ss59dRTufzy\ny3v93YOhyX0JIlRNHK1eqygxpKYGAjK0gYNTxHvA0KFDOfnkk/nHP/7BnDlzePTRR7n44ot58cUX\nqaqqYu3atRhjmD17Nq+++ipjxozho48+4sEHH2T69OmsW7eOXbt28f777wNQX1/v135bWxtLrlvC\nLb+/hfGTx7O/aT8DsgfwzJ+foa2jjY0bN7JlyxbOOecctm7d6nfsLbfcwpQpU3jqqad4+eWX+e53\nv8v69esB2Lx5M2vWrCEnJ6fX3z0YamkkCK0npSipi+8QlXdo6oUXXuC1115jypQpTJ06lS1btnQV\nNhw7dizTp08H4KijjmL79u1cf/31/OMf/yAvL8+v7Q8//JBhhw+j+MRiAAYNHkR6Rjob1m7gm/O/\nCcC4ceMYO3bsQUpjzZo1XZbEmWeeyRdffEFDQwMAs2fPjrnCAFUaCcO3Jk5VFXz0EWzcCGGmGlYU\nJUm44IILeOmll3j33Xdpbm5m6tSpGGNYuHAh69evZ/369Wzbto2rr74agEGDBnUdW1hYyIYNGygt\nLeXee+/lmmuu8WvbGMOgzEG0HGihub25q4x6R2cHY/PHhpXLGHPQOhE5SIZYokojQdx8Myxb1v25\n+GIYORJOOcVlwRRFiUhubi6lpaVcddVVXQ7wc889l8cffxyPE9Gya9cuPvvss4OO/fzzz+ns7GTe\nvHn88pe/7Cqn7mXcuHHs27uPnM9yyMnMYW/tXrIki1kzZ/HciucA2Lp1K9XV1Rx//PF+x55++uk8\n/PDDgI2qGjZs2EGWTKxRn4YL6Kx8ipJ6XHrppcydO7drmOqcc87h9ddfZ8aMGYBVLA899BDp6el+\nx+3atYsrr7yyK4rqV7/6ld/2rKwsHnvsMa6//nqam5vJyclh9erVnPr/TuXaa69l4sSJZGRksGzZ\nsq55yr0sWbKEK6+8kpKSEgYOHJiQOThUabiAbxRVba1d1pLnihIDRowI7vT2RqD0gQsvvPCg4aAr\nr7ySn//85wft63V6A0yaNOkg6wJg2bJlXf+fdNJJvPnmm2H38VJaWkppaSkAQ4YM4emnnz5onyVL\nloT4Fn1HlUaC0SgqRYkjt9/utgT9HvVpJBiNolIUJZVRpZFgAqOoqqrsshNarSiKktS4OjwlIg8A\n5wOfGWMmBNkuwG+BWcCXwAJjzMGDgymEziymKEoq47alsQw4L8z2bwDHOp/vAfckQCZFURQlBK4q\nDWPMq0BtmF3mAH8xljeBAhEZmRjpFEVRlEAkWEZhQgUQKQZWhRieWgXcboxZ4yy/BPzUGPNOqPYq\nKiqMt7BYPPF4POSmSLiTyhofVNb4EChrXl4eQ4YMcU2eSy65hO9///t8/etf71r3wAMPsGXLFhob\nG7n33nt71N5Pf/pTrrnmGo499tiQ+zz88MNkZ2czb968XssdSFtbG1lZWYAtdNjY2Oi3vbS0VKJq\nyBjj6gcoBt4Pse1Z4DSf5ZeAaRHaTAhlZWWJOlWfUVnjg8oaHwJl3bVrlzuCONx7771mwYIFfutO\nOeUU8+qrr5pPPvnkoP3b29sTJFnP8JU1xDWN6pnttk8jEjuBI3yWi4DdLsmiKEoKUFlTyZLyJVz1\n9FUsKV9CZU1ln9oLVRq9qKiIc889F7BJeLNnz+bMM8/krLPOorOzk4ULFzJu3DjOPvtsZs2a1VUG\nvbS0lHfesYMlubm53HjjjUyaNInp06ezd6+dP2PJkiUsXboUgG3btjFz5kwmTZrE1KlT+fjjj/F4\nPJx11llMnTqViRMnBk3wixfJrjRWAt8Vy3SgwRizx22hFEVJTiprKllasZS65jqK8oqoa65jacXS\nPikO39LoQFdpdG9hQC/vvvsuy5cv55VXXmHFihVUVVWxefNm/vrXv1JRURG07f379zN9+nQ2bNjA\n6aefzv3333/QPpdddhk/+MEP2LBhA2+88QYjR44kOzubJ598knfffZeysjJ+/OMfBy1eGA9cVRoi\n8ghQARwvIjtF5GoRuVZErnV2eQ7YDmwD7gf6Nr+ioij9mhVbVlCYXUhhTiFpkkZhTiGF2YWs2LKi\nT+0GK40eyNlnn93le1mzZg0XXXQRaWlpjBgxgjPOOCNou1lZWV1Tt06bNo2qgBIoTU1N7Nq1iwsv\nvBCA7OxsBg4ciDGGG264gZKSEmbOnMmuXbu6rJR442qehjEm7HyJxqrOHyRIHEVRUpzqhmqK8or8\n1uVn51PdUN2ndi+44AJ+9KMf+ZVGD3zA+5Yij/atPzMzs8tiSU9PP2hq1lDtPPzww+zbt49169aR\nmZlJcXExLS0tPfhGvSfZh6eUMDQ1wR136FzjiuJlTP4YGloa/NY1tDQwJn9Mn9oNVho9HKeddhpP\nPPEEnZ2d7N27l/Ly8l6dNy8vj6KiIp566ikAWltb+fLLL2loaOCwww4jMzOTsrIyPv3001613xtU\naaQwZWXw2mtat0pRvMwdN5e6ljrqmuvoNJ3UNddR11LH3HFz+9z2pZdeyoYNG7jkkksi7jtv3jyK\nioo44YQT+M53vsPUqVPJz8/v1Xn/+te/8rvf/Y6SkhK++tWvUlNTw2WXXcY777zDxIkT+ctf/sK4\nceN61XZvcD1PIw4k5AuVl5d3lSd2g6YmWLTIVsb1eODXvw5dJddtWXuCyhofUlnW3bt3M2rUqKiP\nr6ypZMWWFVQ3VDMmfwxzx82lZERJHCS1kVTFweYkpzvf5IsvvuDkk0/m9ddfZ0QMSrT3Fl9ZQ1zT\nqPI0tDR6iqJzcihKcEpGlMRNSfSE888/n/r6etra2rj55ptdVRixRJVGgmlqgnvugYULez9/hs7J\noSjJT2/9GMmO+jQSTFmZ/Vx3Xe8d2Donh3Ko0Q+H0V2jr9dSlUYC8VoIAwbAyy/Dc8/1rh2dk0M5\nlMjMzMTj8ajiiAHGGDweD5mZmb1uQ4enEkhZGezfDzt2QF4e/OlPMGtWz4eUdE4O5VBiyJAh1NbW\n0tTU5JoMtc21VDdUs79tP4OyBjEmfwxDcqxc3iKAyY5X1szMzD4VgFSlkSC8VkZLC3R0QEEBfPKJ\ntTa+/W23pVOU5CU9PZ3hw4e7dv7KmkqWVi6lMLuQ/Ox8GuoaqPu0jkUzFtHY2MjUqVNdk60nbN26\nNSay6vBUgvBaGZ98Yi2LjAzIzrbWhibnKUryEq/SJKmKKo0EsWED7N4NDQ1WSdTXQ1oafPZZzxzY\nmgWuKImluqGa/Gz/xLxYlCZJVXR4KkF4/RAff3zwtvXro8+x8GaBjx+veRmKkgjG5I+hrrmOwpzC\nrnWxKE2SqqjSSCB9dWB7/SLHHad5GYqSKOaOm8vSCju3RX52Pg0tDdS11HH1lKup3RJutur+iQ5P\npRC+WeCal6EoiaFkRAmLZiyiMKeQnY07KcwpZNGMRUmRde4GammkCJoFrsSdxYuhpubg9SNGwO23\nJ16eJCJcaZJE1rpKBtTSSBE0C1yJOzU1UFx88CeYIlEAaD7QHPOZApMdVRopgmaBK0ryUd9Sf8iF\n4+rwVJIQqZChZoErSvLR1tFG/qBDKxxXLY0kQSdUUpTUIys9Ky4zBSYzqjSSgMBQWk3cU5TUoCC7\nIG4zBSYrqjRcIDCrW0NplaRgxAh/p5n3008mD4oHORk5h1w4rvo0XMA3q3vaNLjtNjjtNLtNQ2kV\n1zjEw2p7S7LMFJgo1NJIMIFDUb/5ja1JtXu33a6htIqiJDNqaSQY36GovXvhpZfs3Brr1tkCht65\nUXpSj0pRlOg51JLxYo0qjQQSmNXd0gLp6TBnDuzZA3PnqqJQlHhSWVPJ0go7N4ZvMp4bfohUVV46\nPJVAfLO6W1vt3BpZWfav15ehkVOKEj+SZW4Mr/JKxUxyVRoJxDer+4037F+wVRrUl6Eo8SdZ5sZI\nFuXVG3R4KoH4ZnVfdpmdW+Oww6yV8cwzMG6c+jKUJKYfFDRMlrkxqhuqKcor8luXKpnkrloaInKe\niHwoIttEZHGQ7QtEZJ+IrHc+17ghZ6xparLhtBdfDMccA7Nnw6hRcMklWi5ESWL6QUHDuePmJkUy\n3pj8MSmbSe6a0hCRdOBu4BvACcClInJCkF0fM8ZMdj5/TKiQccI3gmr/fvjznzUbXFESQbLMjZEs\nyqs3uDk8dTKwzRizHUBEHgXmAJtdlCnuBIug+uQTmDoVamutQtHhKaXXhBpCOvvsxMuSYKKNRkqG\nZDyv8vKV9+opV7suVzSIMcadE4vMB84zxlzjLF8OnGKMuc5nnwXAr4B9wFbg/xljdoRrt6KiwrS2\ntsZNbi8ej4fcXqRsNzRAXZ11fHd2QmOjdY7n5NhIqo4OKCqyORtuy+oGKmsfqarqnnTFB09WFrlD\nh8albVpb7TBVjOjNdW0+0Mxez17S09JJl3Q6TAcdnR0cnns4ORk5MZMtkFj2geYDzdS31NPW0UZW\nehYF2QUxlT2SrKWlpRJNO25aGsEEDNRgzwCPGGNaReRa4EHgzHCNzpgxI0bihae8vJzS0tIeH3fr\nrdYBDrBjh7UyWlqsb+PMM+HTT6PP14hUTr2vsrqBytpHFiwI+gAvLy6mdN68uLRNVRUsW9a3tn3o\nzXVdUr7ApEiKAAAgAElEQVSEugH+Du665joKMwpZUrokZrIFEqs+4Js/kj/ImYe8qS6mQ2exktVN\npbETOMJnuQjY7buDMeYLn8X7gf9KgFxxxdfRfeut1srYutVmgntDcKONoPKtYaVDWkrc8RY0DLbe\nZVI5Ggn8Q3CBrr8rtqxIuiErN5XG28CxInIksAu4BPgn3x1EZKQxZo+zOBv4ILEixpcf/hB27oQp\nU6wD/Ne/7rYYIlkRgTWstMCh0it6EkYb47DaWGZEJ0sobW9JJaXnWvSUMeYAcB3wPFYZPG6M2SQi\nvxCR2c5u/yoim0RkA/CvwAJ3pI0P4UqiB07KpOXUlbjgUhhtrDOiUzkaCVIrBNfVPA1jzHPGmOOM\nMUcbY25z1v3MGLPS+f8/jDEnGmMmGWPOMMZscVPeWBIYReVbRiTYpEy+SiTcscohTqg5MbyVMJOE\nWGdEJ0sobW9JJaWnGeEu4bUUAF5/HU46qdtiMKbbiqittQri5Ze7lciXX3bXsAL/EiTq2zjECTWE\nVF6eUDEiEXY4xqny0dPhK28orfe4O9+6s0/DXoksKJhKIbiqNFzCW4dq7VrrCO/shCOOgLfegn37\n/K2IBx6A4cNh5EirRJ55BjIyDvZJagkSJVWI5IPobTXaWFWxDdbOjS/fyOi80bR1tHUpkViSDPkj\n0aBKwyVuvtkOMy1aBJMndzvCX34ZnnzSPxy+qspaHWCVSG2tv9NcUVKNuePmsrRiKWAtjIaWBupa\n6rh6ytXUbqntdTRRNMdFY0EEttPW0ca22m18tv8zzj3m3C5ldOngS2N4VVIDVRpxxhsFdfnl8Ne/\n+kdD+TqzvdngvpVwAaqrbcJfba1d1qEoJSi9LSboUhhtuOGY8i3lUUUTBXv4RzouWksksJ0PPv+A\nwVmDaeto6/LBANS31Mf82iQ7qjTijNeB3dhoFcJhh8Hjj8PvfhfcmR1oQfgmA/re2zoUpfjhjYJa\nvdo/IsLjsdtClRFxsTptuOGY3g5fDcwcSENLQ8jjorVgAs/f0NJAZnqmX1n1/Ox82hrbYnEpUgpV\nGnHEG+VUXAxPPAEzZ8Jvfwu7dsENN8CgQZGd2Vr1VukRHg8UFPivKy6G9nZXxOkt4YavIPTDv62j\njY/rPmbfzn20drQyIH0AwwcN5z+n/CcQfT5E4Pmz0rNobG1k6sipXfs0tDSQlZ4Vh2+f3KjSiCPe\n4afGRnvPVlfD9u3WqV1WZhPy1JmtJIwUmg8jUjRRqId/ZU0l4q1Q5BQlEp+KRdEmAQaef+rIqexo\n3MGA9AF0ms4uJVYwOEBBHwKo0ogTTU3Wob1tm/1/2DB4800QseXQ8/Lg+OPt8JOiJATvEFYgwXwa\nSUBvhq/qW+uZdPgkpo2a1rW+rrmuK/9jr2cvL25/kaE5Q5k8YjLZGdl+Fky48wf6ULxO+0MNVRpx\noqys27LIyoLsbKivt/4KY2zI7PLl8IMfJEXpHkXpOS5aLqGGrwoGFASdznX9nvVsr9tOYXYhZx15\nFutr1vPSJy8x86iZUYfjBlNi5VvKY/adUgVVGnFi7VpbxbatzVoatbU2F6O11SbntrZaxXH33Wpt\nKDHAGwUVWBagN3HZ0SoDFy2XUMNXK7asCGmBjC0Y27V+5OCRXfulQm5EKBKZgOhFlUacOPlk6/Ae\nO9aWO1+3zmZye+nstPNqvPqqezIq/QjvwzzYA7+qCo49Nvq2+qAMKqlhRcF6qp++KiFZ1MHa7okF\nkowFAaMlVomMPUWVRhwIVhtqxgz/cNqVK+H+++F733NPTqUfEmxYaPFiG4mxZo2NtPCSm2tD+mJE\nJTUspYLCtLaEPsR86akFkowFAaPFrXLqqjTigDdqKlQ4rZY1VxJKTQ1MmmTfXnyHr2pqrPUQI6fa\nCrZQSDaFnQI+CXCxeIj1ZBimJxZIMAd4quBWOXVVGnEgMKvbizecNlgmeGCYbbSz8imHKL1xQgda\nFTGeca+aBorIA7qnW47FQywWwzCpVBAwWtyaQ0SVRhwIl5AXqqx5oLWhs/IpYUmG8NmAEiRjCgx1\naZ9RmNs9F3mkh1gwCyKQcMMw3r99sUBSlUgJkPFClUaCKSuzeRrbt9ty6MEywXX4SnGVaOtRBVg0\ncx2LgOxC8n0S4EI9xEJZEIFFAEMNw/iG0brlQ3ETt6wnVRoJZsMG2L3bvxw6+GeCRzN8pRxiBA5H\neZ3aMXZmA73OsejpQyyUBRFYBDBcIp9vGG0yz6sdL9ywnlRp+JAIP0K4ecHBKpTbboPTTrPLoYav\nlEOMwOGo9ettjan6KKqsjhhhE4MCrYc4ZJX25CEWyoIILAI4YfgEbn31Vto72xk+aDijc0eTkZ7R\nL8NoUwFVGj4kwo8QyYq4+26rOHbvtmVGtBS60mduv93O3BdDp3csCGVB+BYBrKypZOXWlUw4bAI7\nG3eyb/8+6lvquflrN/P+vvf7XRhtKqBKwyGefgTfOTXCOcGbmuCFF2xdqnXrIC2te2pnLWSo+JGb\na60Mj8ffgkihmjSDMwdzzzv30HqglYIBBZww/ASGDRrmVwTQdwjr2KE2QbGuuY73973vmiP4UCei\n0hCRPGC4MebjgPUlxpjKuEmWYOLpR/CdUyNc/kZZGUyY0J1FPneuvwwahqt04fVjxDhsNlEs37Sc\nu9+5m4IBBTSnN9PY2kjFzgpu+tpN5LTndO0XLhehP4bRpgJhlYaIfBu4E/hMRDKBBcaYt53Ny4Cp\noY5NJaINg+1L28cdB//4hx1yCpa/UVoaWQYNw1VSkWBhtXe9fRd5WXkU5HRbFfXN9Tzz0TNMK+6u\nUBspF6G/hdGmApEsjRuAacaYPSJyMvBXEbnBGLMCfIrUpziRMrijIZQV4GvBTJx4sPXgZeVKzSJX\nwhCLaVldqEobKqx2e912ivOL/fbNG5DHrqZdfut0CCr5iKQ00o0xewCMMWtF5AxglYgU0TXFSeoT\nKYM7GoJZAT2xYGKRRa70Y2LxUHchITBUWG2apNHY2uhnaTS2NjJ68Gi/43UIKvmIpDSaRORorz/D\nsThKgaeAE+MtXKLo65SqoayAnlgwscgiV5RkI5RP4sj8I/mo7iP2ePbQ3tFOa4ctPTLr2Fk0H2j2\n21+HoJKLtAjbv0/AMJQxpgk4D7gqXkKlGr5WgFcpgL/14P0Y419otCftB1M+ipLMjMkfQ0NLg9+6\nhpYGxg0fx1EFR2GMYX/7fgCGDhxKZlomez17qazpNzE2/Y5IlsZ+4HBgW8D6k4E34yJRihHOCghl\nPTQ1wR13RB8FFYvhMyUFSaE5vUMRyicxMHMgU0ZOIS0tjeb2ZnIyc2hub2a3ZzfpQ9IPqazuVCOS\n0rgT+I8g6xudbX16ZInIecBvgXTgj8aY2wO2DwD+AkwDvgAuNsZU9eWcsaY3TvSeRkH1dfhMSVGS\noShhHwnlk7jzrTs5bNBhNLQ0kDcgD4DsjGwaWhpIl3TN6k5iIimNw40xGwNXGmM2ikhxX04sIunA\n3cDZwE7gbRFZaYzZ7LPb1UCdMeYYEbkE+C/g4r6cN9b01ArQKCjFNWIRgdULgvkkvKG0+dn5XZZG\ny4EW8rPz6TAdMc/qdmNa1P5KJKVREGZbTpht0XAysM0Ysx1ARB4F5gC+SmMOsMT5fzlwl4iIMSZp\nIrd6agX0NApKE/qUmJFEQ1reYavRg0ezce9GWg+00kknRxceTUdnR9AS6b3FrWlR+ysS7vkrIo8A\nLxtj7g9YfzVwjjGm12/9IjIfOM8Yc42zfDlwijHmOp993nf22eksf+zs83modisqKkxra2uozTHD\n4/GQ28OneGenLVaYnm5LhHR2QkcHFBXZ5WA0NMC+fTB8OOTnB98nHrK6hcrqQ1VV97inL62tBw9b\n7dplp3QNJDMTRo9OyuvafKCZ+pZ6vmz7kg7TQbqkMzBrIIPMIPIG5/WqrbaONrLSsyjILiAnw77X\n7vHs4UDnATLSut+Rvcsjc0f26Tsk43UNRSRZS0tLo8q9i2Rp/BB4UkQuA9Y5674CZAEXRnOCMAQT\nMFCDRbOPHzNmzOi1QD2hvLyc0tLSHh2zcqW1LMaO7V4XrFyIl6YmWLTIWhhvv31wRdx4yuoWKqsP\nCxaE9mkElg6JsG9/vq6+lkT+IMfZ3lTXZUlc9fRVFOUVkSbdb2adppOdjTt5oPSBhMrqJrGSNazS\nMMbsBb7qJPVNcFY/a4x5uc9ntn6MI3yWi4DdIfbZKSIZQD5QG4Nzu0JP/R+a0NfP6Gk0lEs+iFQj\n3Mx+JSNKXJsWtb8SqfZUNnAtcAywEfiTMeZAjM79NnCsiBwJ7AIuAf4pYJ+VwBVABTAfO1SWNP6M\nYITzQfTE/6EJff2QnkZDxcMH0Q/CeAMJV9QQtBRJrImU3PcgdjhqI/ANYGmsTuwon+uA54EPgMeN\nMZtE5BciMtvZ7U/AUBHZBvwIWByr88cLbzhtXxPvNKFPiQtexRX4CaZIUoRQCYS+RQ0XzVhEYU4h\nOxt3UphTqE7wPhDJp3GCMWYigIj8CVgby5MbY54DngtY9zOf/1uAi2J5zngSKZx261aYPdvuc/TR\n4dvShL5DjH5oASSKaCwJLUUSOyIpja5wDGPMAZF+U9g2LkTyQdx4I3zyCdxwAzz2WPi2NKHvEKOv\niXyB/o9166C5GXJyrJO8tNT+Xbcu+HkcUjGfQYsaJpZISmOSiDQ6/wuQ4ywLYIwxPYuL66c0NcGd\nd1qFEMoHsXUrvPiiDZt9/nn4+GN/a0PzMZQ+EWiNBEZTDRhgl9esCdlEKuczqCWROCJFT6UnSpBU\npqzMhtPm58MYJyAjsJzIjTfanIyBA+0LYKC1oRMsHQKEi4ZKAp9CpCikVLRClNijc4T3Ea8fIysL\nqqvho4+65/UG64M4/nhrZQwebNcNHuxvbWhpkUOEcL6JBQt63l44P0gocnK6FFdlZi0rBlVTneFh\nTNEI1u+pP0gJeKOQUtkKUWKLKo0+4vVjnH126ES9Cy+ElhbIcwbzsrKsovBaG5qPofSK3vhBpk2D\nZct8lMCRFGXnU9fSwOZtf2db7TYy0zPJz85n/LDxZKVnMSZ/TEQrpK/4WjFnp59NZU2lKqMkJVLI\nrRKGULkUHo//fm+8YSs87NsHX3wBn39ulcRbb0XfhtLP8Q5dBX56ksi3ejU89ZT1W6xZY/9/6im7\nPgBfJZAmabR1tNHe0U7N/hoy0zP5su1LyqvK2V63nbnj5lLdUE1+tn8dG99ciL7gVWB1zXUU5RVx\noPMASyuW6pwaSYpaGn0gmrLoTU1wwQV2uMnjsaVAXn7ZOr0PPxyee67v85Mr/YBYhNV6PFDg1Bj1\ndjiwFklrq58SCkyI++DzDxg+cDhZGVkMzBzYVbI8OyObFVtW8O6ed9n02SamjpzK4bmHA7HLqg60\nYjLSMijMLNQ5NZIUVRp9IFguRXs7/P73B0/56h16evZZqzSysuy2L7+0znHNx+jnJDoPY+bM7v+r\nquwwlk+9qsDSGg0tDWSmZzIydySlxaUA7Gnaw0ufvMSowaM4ZfQpvPrpq5RXlXP62NPJzsjuUVZ1\nOCd6uIxu73Hr96ynvrWeggEFTB45WZ3wLqJKow8Ey6VYuRLuv98qh61bDw7DfeAB+zK4c6dVKo2N\ndp06vvs5STahUmBCXFZ6Fo2tjUwdObVrn/U16xmaM7RLsXy9+Ou8t+c91u5ay5xxc6LOhYjkRA9V\nGyorPYulFUs50HGA7fXbSSON2i9rGZQ1SJ3wLqI+jRjiGwX1wAPw5JPWOQ7w+uvW4vjkE9ixw5ZF\nLyy0y889F75dRQlKoB/E44H6+qjeQAJLa0wdOZWjhxzNgPQBdJpO6prr+KL5CyaPmNx9utwRnHvM\nuUwZOYUlpUuifmAH+k8KcwopzLbDT2AVWF1LHXXNdXSaTg50HqCupQ5BKMwuZLdnNzkZORTkFJCT\nmcOupl1+x1fWVLKkfAlXPX0VS8qXqC8kzqilEUO8Q1FZWfYeFrH38YEDsH077N1r/6+qgqOOgowM\nu+8DD8CsWWptJAWpVM4jUkJfBAIT4gKHkGYeNZMB6f7zefTGjxGpoGBgRvf49PEsOnkRd751J8MH\nDQ86JayGAruHKo0Y4RsFVVVllUFWFhxzjJ1I6dvfhldegSOOsPPleH2U6elWmajjO0mI1TBSoPJZ\ns8Y6qnJz/f0NsaSPpdSDKZFYVIeNpjS577nLy8v9hq2CTQnrPT7eocDKwajSiBFeKwOsLyM/H/bv\nt0NRX3wBU6fCxIl2nyOPPPh4dXz3MwKVz/r11plVXx+/c4azhMrLe9yc1wL4wzt/YOWHKxGE6UXT\ne9xOb0uTe48blTuK9/e9b6eENZ0cM+SYruPvfOvOsFaMEntUacQIbyTVW29Zy6Kjw36qqmDoUOu7\nKC62EVS9nYFPSWFyc63C+PRTeOih7vXegoLJOPzl8GX7l3x97Ne7Hvg9Hf7pbUFB3+O+bP+yK3rq\n2KHHdkVP6QRLiUeVRozwRlLdeqstDwLWyti/3ybz7dply4loDkY/JdJwlHdI6qGH4DvfOfh4l6Ko\nIhGr4Z/eFhSMdJxOsJR4VGnEGN8w3FtvtdbH1q22HpX3uaBDUf2QwOGo556zjqu2Nv/0/s8/T7ho\nPSJA+VUPW0PRgUGQO7hL8SXT8I+WRU88qjR6QVOTdV57POGHmX74Q5uPMWVKdza4DkslObGal/vA\nge5iY94sbbBjlslMgPIbQxV1NFNY3634km34R8uiJxZVGr2grMwqjkjDTFqIMAWJlV8hI8NWqWxr\ng82brRIBW87jqafs/9FEUrkcAjyXcSylAtLayDedOvyjqNLoKd7Q2q99LXwZ81CFCLXseRIRzwfy\nqFH+0VJea6Oysvv/aCKpwoUAJ0ChlDCCRcxgReebVDfu1OEfRZVGT/FaD+np4Z3a0RQzVFzGjdIe\nGRndysLj6T5XT4e/IGHylzCCkvrJMOeBrnU6IdOhiyqNHtAT6yFYMUNQJ3i/JdAX4nV++1abBWuB\nXHCB/b+qyq+IYKqgWdiHNqo0ekBPrIdgxQyVfkzgcND06TZSyuOxURPe6RwHDky8bD0hikAAzcI+\ntFGl0QN8rYfiYg2hVcIwblz30NFTT/XMj+Fl8eLufA9fcnNtfZp44Kv8fH0mNTVdU9JWH/EuRWde\n4HdYMoXhKvFFlUYP8LUeystTcmRBcQNvNjj0zI9RU2P3CZzGsaYGTjstuBM8loTwmYypX09dS4Nm\nYR+iqNJQDl1ilZMRicAJkXrythEsJLeqyloEixcnRv4A5u4fw9KWOkCzsA9FVGkohy5JWuspalyS\nv6R9CItm/EizsA9RVGn0ks5OuOMOWLhQ8y6UGOP1JQT6M+JZVr2HaBb2oYsqjV7S1ASvvQbjx6sT\nXAlCX4a+vL4Ebzl1L/Esq64oUeKK0hCRIcBjQDFQBXzbGFMXZL8OYKOzWG2MmR1PuZqa4J57IlsP\nTU22/Plxx2mWtxKCYENHXgvCiULqIlQGt68DHbqd6HH2WfjJFQOfiSYC9i/csjQWAy8ZY24XkcXO\n8k+D7NdsjJkcZH1cKCuLznooK7PDU1pTSukRPc3gDhyKSnQyYAx8JpoI2P9Ic+m8c4AHnf8fBC4I\ns29C8GZ7e62HwCjHwP28uVrerPBQ+yvKoYxvImCapFGYU0hhdiErtqxwWzSll4gxJvEnFak3xhT4\nLNcZYwqD7HcAWA8cAG43xjwVqe2KigrT2traY5kaGqCuzmZ5t7ZCYaGdsjXUfnl5Hlpb7ZhUuP2T\nAY/HQ26KjJ/1K1l37YL2dt8DIC3NFi7zlk0H24F8LZCqqu6yA74E7hfNOb2nzssjd8iQ8Mf2kuYD\nzdS31NPW0UZWehYF2QXkZOQA8GnDp2SmZSIiXfsbY2jvbGds/tig7fWrPpBERJK1tLRUQm70IW7D\nUyKyGgg2+HljD5oZY4zZLSJHAS+LyEZjzMfhDpgxY0ZPxASs9bBoEQwZ0q00Qk3L6p2Zr7S0nPLy\n0q71Rx+dvKVDysvLKS0tdVuMqOhXsi5Y4P+Q92aG19d315+Cg4edwlWvDfSHRDqnV9biYkrnzg1/\nbC/wHX7KH+TkbDTVdQ0/LSlfctB0rN7lK0qvCNpmv+oDSUSsZI2b0jDGhIwNFJG9IjLSGLNHREYC\nn4VoY7fzd7uIlANTgLBKozf0pqZUYEZ4U5OG4CoxIgnzR5ZvWs5db9/FrqZdjB48mutOuo75J86P\nWIdKp2Ptf7jlCF8JXAHc7vx9OnAHESkEvjTGtIrIMOBU4I54CBOLirTROtGVfsLixba+VLSRUNAd\nDeVbSsR7TBKzfNNyfrL6J+Rl5TFy0Ejqm+v5yeqfAFDdUE1RXpHf/r51qHQ61v6HW0rjduBxEbka\nqAYuAhCRrwDXGmOuAcYD94lIJ9Zhf7sxZnM8hOnrsFKgE11DcA8Bampg0qSDh4LCzWXhjYZKopLo\n0YTD3vX2XeRl5VGQY92Q3r93vX0XpcWlBw0/Bdah0kTA/oUr0VPGmC+MMWcZY451/tY6699xFAbG\nmDeMMRONMZOcv39yQ9Zo8J3W1TuspSjJjtcfUddc5xcOW1lT6bffrqZd5A3I81uXNyCPXU27mDtu\nLnUtddQ119FpOqlrrqOupY6542LvP1GSA80I7yM6rasSkkQVRIzmnMcee9CqaOfFGD14NPXN9V0W\nBkBjayOjB4/W4adDEFUafUSndVVC4oZDO9Q5y8sPWhXJH+HlupOu6/Jh5A3Io7G1kca2Rm46/SZA\nh58ONdxK7us3+DrRvR9jDp43R1GSjTH5Y2hoafBbF2xejPknzueOmXdQkFPAnv17KMgp4I6ZdzD/\nxPmJFFdJEtTS6CPJmpuhxJkRI2xCT+BQUE+HnsLlZMTZUulJOOz8E+erklAAVRqK4k+0D/Hbbz84\nWaenBQmh5/WoYoj6I5TeoEpDUXwJ9hBfvdrObRGoTM4+O/KxkBAF0FvUH6H0FFUaihIJj8eGwgUq\nhCA1nhSlv6NKQ1GiZfVq/3LG11xjh6IS4H9QlGRBlYaiRIvH4z+TXlqatT6SePhJUWKNKg0l+XAx\noijhuJEAqCh9QJWGkny46VAO9hD3eOz6SDNt9UYBJKES1OlZlXCo0lAUX8LN7b1mjf/69PTIx6YY\nvvNjZKZl8veP/s5DlQ8x86iZLPzKQlUeiioNRYmIVxkETnCUlwd79rghUdzw1qNq62jjzZ1vkp2R\nzZCcIby35z2d21sBtIyIokSPd/jJ+/FmhPcj/0N1QzX52fl88PkHZGdkk5OZQ05GDm0dbTq3twKo\npaEo0RM4/BSYEQ4p78Qfkz+GuuY6GloausqhtxxoIT87P2gxQ+XQQ5WGknykckRRCmaF++KtR5WV\nnkVzezMiQktHC1NGTglazFA59FCloSQfKfBG3l/x1qO65517eHH7iwzNGcr00dMZkD5A5/ZWAFUa\niqIEUDKihHvOv8cv9HZkzkgtZqgAqjQURQmBFjNUgqHRU4qiKErUqKWhKLEklZ34ihIFqjQUJZao\nE1/p5+jwVAxpaoI77ohcokhRFCVVUaURQ8rK4LXX7F9FUZT+iCqNGNHUBM8+C8cdB6tWqbWhKEr/\nRJVGjCgrg7Y2GDzY/lVrQ1GU/ogqjRjgtTK8ATIjRqi1oShK/8SV6CkRuQhYAowHTjbGvBNiv/OA\n3wLpwB+NMUkZmuK1MgYMsMsDBnRbG9/6lruy9StSvBigovQH3Aq5fR+YC9wXagcRSQfuBs4GdgJv\ni8hKY8zmxIgYPRs2gDEHh+evX69KI6Z4iwGuXu1vxnk8dpsqD0WJO64oDWPMBwAiEm63k4Ftxpjt\nzr6PAnOApFMaN9/stgSHGB4PFBT4rysuTplKsoqSyogxxr2Ti5QDi4INT4nIfOA8Y8w1zvLlwCnG\nmOvCtVlRUWFaW1vjIa4fHo+H3NzcuJ8nFvQbWauq7NhfXR1k+LzvHDgAhYV2UqRgZcnjRL+5rkmG\nyhofIslaWloa9i3eS9wsDRFZDQSrnXCjMebpaJoIsi6ihpsxY0YUTfed8vJySktLE3KuvtJvZPVO\nt/rUU/6WRn09XHCBVSqBkyLFkX5zXZMMlTU+xErWuCkNY8zMPjaxEzjCZ7kI2N3HNhVFUZQ+kMy1\np94GjhWRI4FdwCXAP7krkuIq3mKAgbHMKTI8oCj9AbdCbi8Efg8MB54VkfXGmHNFZBQ2tHaWMeaA\niFwHPI8NuX3AGLPJDXmVJMEbGRUs9LaqSivJKkoCcCt66kngySDrdwOzfJafA55LoGhKKqBhtYri\nGpoRriiKokSNKg1FURQlalRpKIqiKFGjSkNRFEWJGlUaiqIoStQkc56GohxSVNZUsmLLCqobqhmT\nP4a54+ZSMqLEbbEUxQ+1NBQlCaisqWRpxVLqmusoyiuirrmOpRVLqaypdFs0RfFDlYaiJAErtqyg\nMLuQwpxC0iSNwpxCCrMLWbFlhduiKYofqjQUJQmobqgmPzvfb11+dj7VDdUuSaQowVGloShJwJj8\nMTS0NPita2hpYEz+GJckUpTgqCNc6Z+k2NSwc8fNZWnFUsBaGA0tDdS11HH1lKtdlkxR/FGlofRP\nvFPDBpKks/uVjChh0YxFftFTV0+5WqOnlKRDlYaiJAklI0pUSShJj/o0FEVRlKhRpaEoiqJEjSoN\nRVEUJWrUp6H0T7xTwwZbryhKr1GlofRPkjCsVlH6Azo8pSiKokSNKg1FURQlalRpKIqiKFGjSkNR\nFEWJGlUaiqIoStSo0lAURVGiRpWGoiiKEjWqNBRFUZSoEWOM2zIoiqIoKYJaGoqiKErUqNJQFEVR\nokaVhqIoihI1qjQURVGUqFGloSiKokSNKg1FURQlalRpKIqiKFGjSqMPiMitIlIpIutF5AURGeW2\nTMEQkf8WkS2OrE+KSIHbMoVCRC4SkU0i0ikiX3FbnmCIyHki8qGIbBORxW7LEw4ReUBEPhOR992W\nJbQWaDMAAAQRSURBVBwicoSIlInIB87v/29uyxQKEckWkbUissGR9eduyxQJEUkXkfdEZFVf21Kl\n0Tf+2xhTYoyZDKwCfua2QCF4EZhgjCkBtgL/4bI84XgfmAu86rYgwRCRdOBu4BvACcClInKCu1KF\nZRlwnttCRMEB4MfGmPHAdOAHSXxdW4EzjTGTgMnAeSIy3WWZIvFvwAexaEiVRh8wxjT6LA4CkjK9\n3hjzgjHmgLP4JlDkpjzhMMZ8YIz50G05wnAysM0Ys90Y0wY8CsxxWaaQGGNeBWrdliMSxpg9xph3\nnf+bsA+40e5KFRxj8TiLmc4nKe99ABEpAr4J/DEW7anS6CMicpuI7AAuI3ktDV+uAv7uthApzGhg\nh8/yTpL04ZaqiEgxMAV4y11JQuMM96wHPgNeNMYkrazAncBPgM5YNKZKIwIislpE3g/ymQNgjLnR\nGHME8DBwXbLK6exzI3YY4GG35HTkiChrEiNB1iXtW2aqISK5wBPADwMs+aTCGNPhDEsXASeLyAS3\nZQqGiJwPfGaMWRerNjNi1VB/xRgzM8pd/wY8C9wSR3FCEklOEbkCOB84y7hcpbIH1zQZ2Qkc4bNc\nBOx2SZZ+hYhkYhXGw8aYFW7LEw3GmHoRKcf6jZIx2OBUYLaIzAKygTwRecgY853eNqiWRh8QkWN9\nFmcDW9ySJRwich7wU2C2MeZLt+VJcd4GjhWRI0UkC7gEWOmyTCmPiAjwJ+ADY8z/uC1POERkuDcC\nUURygJkk6b1vjPkPY0yRMaYY21df7ovCAFUafeV2Z1ilEjgHG6GQjNwFDAZedMKD73VboFCIyIUi\nshOYATwrIs+7LZMvTkDBdcDzWGft48aYTe5KFRoReQSoAI4XkZ0icrXbMoXgVOBy4Eynj6533o6T\nkZFAmXPfv431afQ5lDVV0Pk0FEVRlKhRS0NRFEWJGlUaiqIoStSo0lAURVGiRpWGoiiKEjWqNBRF\nUZSoUaWhKDFCRDqcUNH3ReT/RGSgs36EiDwqIh+LyDoReU5EjnO2/UNE6mNRfVRREoEqDUWJHc3G\nmMnGmAlAG3Ctk7T2JFBujDnaGDMNW2X4cOeY/8bmJyhKSqBKQ1Hiw2vAMcAZQLsxpiuh0hizwRjz\nmvP/S0CTOyIqSs9RpaEoMUZEMrDzbWwEJgAxKxanKG6jSkNRYkeOUy77HaAaW0tJUfoVWuVWUWJH\ns1MuuwsR2QTMd0keRYk5amkoSnx5GRggIv/sXSEiJSLyNRdlUpReo0pDUeKIM3fJhcDZTsjtJuBX\nQA2AiLwG/B9wllOF9lz3pFWUyGiVW0VRFCVq1NJQFEVRokaVhqIoihI1qjQURVGUqFGloSiKokSN\nKg1FURQlalRpKIqiKFGjSkNRFEWJmv8PAHmUZRZkj6kAAAAASUVORK5CYII=\n", "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYUAAAEUCAYAAADEGSquAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJztvXl8lNW9+P/+ZI+GhCBIWATqgqAQWVxAvToiLvW6FfW2\n1qW43P74Wm29V79XqrVStb3W0tb2arWtVepyq/0qKsXWrSZFatzAgAuIiDGGJSxZByYJSc7vj/NM\nMpnMTGYms2Y+79drXsk8z3nO8zkzz5zPOeezHDHGoCiKoigAWckWQFEURUkdVCkoiqIoPahSUBRF\nUXpQpaAoiqL0oEpBURRF6UGVgqIoitKDKoU4IiKXicgrKSDHRyLiSpd6UwkRcYlIXQzru1tEdovI\nDhGZICJuEcmOVf2DkKtPO1P5uxWRGhGZ7/x/q4g8HOP6/0VEPony2pT5TqMlrZWC78Phd9wlIt3O\nl+MWkToR+bOIHBegrIjIFhH5OFb392KMedIYc2ak9cYaY8zRxpjKwdQhIstE5O5Y1xtrBvpOwrje\niMjhsZTJp+5DgJuAo4wxZcaYWmNMkTGmK4q6ForI6gHKLBWRT0WkVUQ2isiV4dafit9tIIwxPzHG\nXBvjOt8wxhwZ5bVRf6eREq9nNa2VwgBsM8YUAcOAOcBG4A0ROd2v3CnAwcChgZRGtIhITqzqSoX7\nKDFhIrDHGLNzoILOYGWwv8+9wHlACfAt4FcicuIg64wbqfAsp4IMSccYk7YvoAaYH+C4C6gLcPx+\n4D2/Y48ATwLLgfujvT+wEPgn8EugAbjbObbaOS/OuZ1AM7AemBak3krgv4F3nLIvACOcc5MAA1wD\n1AKrnOPnAx8BTc71U4PImQUsBj4D9gB/9tbtnD8ZeNOp50unDd8G9gMdgBv4S4B684H7gG3O6z4g\n3/f7wI6SdwLbgatCfK5jgRXO57gZ+Hefc0scmR8DWp02H+ucexzoBjyOnP810Gfjd99Vzme717n+\n6wPJ7rR7qfNd1AMPAYUB6p7vyNXt1L3M57vM8fnef4x9jjzA4c7nv8Vp6+fAZcBUoA3ocupqCvN5\nXQHcFOScC5/fjN93G/Qz9/m+ngV2OTJ+1+fc8UCV89lvx/4G83zOG+A7wKfA50FkuwL4Avu83hZA\ntiec/wuAJ5xyTcC7wGjn3AjgUeyz2Qg87/ds3gLswD5DgT6L/4v9ze4F/gCMBv7mfB6vAaV+v0/f\n7/Qu5zttBV4BRvrU/f+c+zZjn7+jfc4tAx4AXnSufRs4LMSzOhJY6bS9AXgDyIq4X42mM06VF5Er\nhXnYH+WBzvsDgBbgHOAiYLffA7sYWBnO/bE/3k7gBiAHKKSvUjgLWAMMxyqIqcCYIPVWAluBacCB\n2B+c98H3PnSPOecKgcnOw3EGkAv8F7YzzQsg543AW8B4bIf2W+BPzrkJzsN3qVPPQcAMnwf07hDt\nv9Op92BgFFax3OXzfXQ6ZXKdz3sfzg8pQPv/AfwG+yOfge1sTvfpBNqcOrKxyvOtYM/EQJ9NgHsb\n4HC/Zymo7FjltwLb6QwD/gL8d5C6XfTtbLzfpW8HUgscjX2GSrDP55HO+TE4nQY+z1aYv5VCbKd8\ndpiy+X63QT9z7CBjDfBDIA84FKvEznLOz8bO1HOc9m4AbvT7vF91Pr9AyvQobKd3CvZ5/YXzfQRS\nCv+f8/kf4Mg5Gyh2zr0IPA2UOt/jqX7f70+d+guDfBZvYRXBOOzgYC0w07nmdeCOEN/pZ9jnsNB5\nf49P3VdjnxvvoKra59wybOd+vPP5PQk8FeJZ/W/soCTXef0LIBH3q5FekEovIlcKU5wPcpzz/nJs\nh5PjfClNwNeiuT/2R1rrd34hvUphHrAJ+wMJqb0DPDhHYUfp2T4P3aE+528H/uzzPgurVFwB5NyA\n08E678dgZwE5wPeB54LItIzQSuEz4Byfc2cBNT7fh8f7Q3GO7QTmBLjPIdgR8DC/h32Z8/8S4DW/\nz8YT7JkY6LMJcP9ASiGg7Fjlvhdn9Oacm0vwEW+f55LAHcidPucPxD6TF+HXYRK5Uvgj8BJBOokA\nsvl+t0E/c+AE+j/33wceDXKfG32fMaf980LI/UP6doQHYn8LgZTC1djBSLlfHWOwg8F+gxCn3R1A\nwQCfxWU+758FHvR5fwO9M49A3+kPfMpeB7wUpK3DnWtLfH5zD/ucPwfYGOJZvRO7qnB4oPrDfQ1l\nm0IgxmE/yCbn/bewHUanMaYdu4T0rUHU/2WwE8aY17FT5weAehH5nYgUh1nXF1jNPzLI+bFOGe+9\nup3z4wLUOxF4TkSaRKQJqyS6sKOgQ7CdezT0kcH5f6zP+z3GmE6f9/uAoiD1NBhjWv3q8m3LDr96\nCkKsBUfy2QQjmOyjsKPSNT6f50vO8Wjp+V6NMXuxywKLgO0i8qKITIm0QhH5GXbW+W/G6T2iINhn\nPhEY622/8xncin2eEJHJIrLS8bZqAX5C3+cYQvxusN+f/2eyJ0jZx4GXgadEZJuI3CsiudjnusEY\n0xjkul3GmLYQMoBdGvTiCfA+0LPsxf+zKwIQkWwRuUdEPnM+mxqnzMiBrg3Cz7Cz4Fcc55nFIcoG\nJdOUwteAtcaYvSIyHjt6v9x5YHcAFwPniIj/QxsuIX9wxphfG2NmY5cHJmPXKYNxiM//E7Cj+d1B\n7rUN++MErJHSuX5rgHq/BL5qjBnu8yowxmx1zh0WTPwQsvaTwZF52wDXBKtnhIgM86srUFsC4S9n\nJJ9NpOzGdghH+3yWJcY6OERLH/mNMS8bY87AjnY3Ar8PVC4YIvIj4KvAmcaYlkHIFYwvsTMj3+dp\nmDHmHOf8g1i5jzDGFGMVhvjVEaot2/H5LYjIAdhlzX4YY/YbY35kjDkKOBE4F7jSkXGEiAwPco9o\nFeVg+SZwAdbeVIKdZUD/zycsjDGtxpibjDGHYh0M/jOAY82ADAWlkCsiBT6vPiNGx4tjnIjcAVyL\nfSjBGq82AUdi161nYDvqOuyaekwRkeNE5ARn5LKXXkNhMC4XkaOcH8GdwDMmuJvbn4F/FZHTnfpv\nAtqxU2l/HgJ+LCITHblGicgFzrkngfki8m8ikiMiB4nIDOdcPXa9OBh/An7g1DcSO+1/IkT5gBhj\nvnTk/m/n+yzHGtWfDLMKfzkj+WwCXR9K1m5sJ/1LETkYwHnWzgpT1pCIyGgROV9EDnRkdtP7zNQD\n40UkL8T138d2PGcYY4KNrgfLO0CLiNwiIoXO6HeajyffMKxdxO3Mcv5PhPU/A5wrIic7bb2TIP2W\niJwmItPFxgi0YAdSXcaY7Vij8G9EpFREckXklMibGnOGYb/XPdgZ508ivL7Psyoi54rI4c7ApwX7\nrETsGjsUlMJfsaM172uJc3ysiLixP6R3genYdWRvMNm3gN8YY3b4vrCd5regJzDmbzGSsxjbgTTS\n60mxNET5x7FrijuwBtfvBitojPkEax/5H+zo9TzgPGNMR4Div8IaRl8RkVasAe0Ep55a7LrlTVgD\nVzVwjHPdH4CjnCWC5wPUezfwHtZD4wOsIe7uAOXC4VLsqGkb8BzWiPdqmNf+N1Y5NYnIzRF+NmCf\nnz861/9bGPe7BTtlf8tZAngNO9CIBVnY72Ib9vs4FbsmDda4+RGwQ0R2B76cn2BnWZ9Kb8zOrUHK\nRoUzUDkPO6j6HPsZP4wd+QLcjFVMrdjn/+kI6/8I6530v9hZQyN24BaIMqwSacEui/6D3oHJFVgl\nsRFrE7oxEjnixGPYvmAr8DH2txgJS+j7rB6Bff7cWI+v35goYk0k+iVGJV6ISCXWeBaTSE0RqQUu\nN8asikV9iqIMXYbCTEEJgYiMwho+a5IsiqIoaYAqhSGMs677KfA/ztKQoihKSNJRKZhIXlVVVRGV\nT4WXMeZUY8zvB1vPm2+++Y7jDfN/k92mZLzS8bvXtmvb49j2sEhHpRAR7e3tyRYhaWRy2yGz269t\nz0xi0fYhrxQURVGU8FGloCiKovSgSkFRFEXpIW65w8VuKPIYNqCkG/idMeZXAcq5sNkBc4HdxphT\n4yWToiipRVdXFw0NDezfvz9mdRYXF7NtWzQZVtKf4uJidu3axYgRI8jOjm7zt3huKNGJzd2+1slj\ns0ZEXjXG9Oxw5uQi+Q02nW+tN1WAoiiZQUNDAwUFBYwcORKbnWHwdHR0MHbs2IELDkHa29spKCig\noaGBUaOiy8sYt+UjY8x2Y8xa5/9WbNi5f2bKbwLLvT70JowdqRRFGTrs37+foqKimCmETEdEKCoq\nGtTMKyFpLkRkEnanoGm+mRpFxLtsdDQ2OdSvjDGPhaqrqqrKROJ25Xa7KSoaTNLK9CWT2w6Z3f50\naXtxcTEjRoyIaZ0dHR3k5QXNEzik8ba9oaGBlpa+SXFdLld4mncwmzGE88Lm/14DLAhw7n5sEqgD\nsTnEPwUmD1BnRFRUVER6iTHGmJYWY376U2NaW6O6PCWItu1DhUxuf7q0fevWrTGv8/PPP495nemC\nt+1BPtfkb7LjpCp+FnjSGLM8QJE67C5Ee40xu7GziWMClEs4FRXwxhv2r6IoQ5sf//jHHH300ZSX\nlzNjxgzefvvtoGWXLVs2pA3Z8fQ+Emy65Q3GmF8EKfYCcL+zB0IeNoXzL+MlU7i0tsKLL8LkybBy\nJZx2GqTBTFxRlCioqqpi5cqV/G3V39jSuoUvt3/Jl3zJJPckRheN7ld+2bJlTJs2bcgas+M5UzgJ\nm8N8nohUO69zRGSRiCwCMMZswG5fuB67WcfDxpgP4yhTWFRUQEcHDBtm/+psQVFSh9ZWuPdecLtj\nU9/27dsZNnwYa3auwbPfw/iy8Rw44kAef+lx5p48l9mzZ3PWWWexfft2nnnmGd577z0uu+wyZsyY\ngcfj4e9//zszZ85k+vTpXH311T2pJhYvXsxRRx1FeXk5N998MwB/+ctfOOGEE5g5cybz58+nvr4+\nlGhJIZ7eR6uNMWKMKTfGzHBefzXGPGSMecin3M+MMUcZY6YZY+6Llzzh4p0llJXZ92VldrYQqwdQ\nUZTBEeul3TPPPJMtX2zh+nOu56E7H2Ld2+vIJZdHfvIIt/zqFtasWcPVV1/NbbfdxsUXX8yxxx7L\nk08+SXV1NSLCwoULefrpp/nggw/o7OzkwQcfpKGhgeeee46PPvqI9evX84Mf/ACAk08+mbfeeov3\n33+fb3zjG9x7772xaUQM0YhmP7yzhPx8+z4/X2cLipIq+C/txmKwVlRUxL3/715u/snNDB8xnDtv\nuJO//O9f+OLTL/iPy/+DGTNmcPfdd1NX13/Dt08++YSvfOUrTJ48GYBvfetbrFq1iuLiYgoKCrj2\n2mtZvnw5BxxwAAB1dXWcddZZTJ8+nZ/97Gd89NFHg29AjFGl4Me6dWAM1NT0voyB6uokC6YoStyW\ndkccOIIpx07hqv+4iu/96HusemkVEw6fwKN/e5Tq6mo++OADXnnllX7XmSAu/Tk5ObzzzjtcdNFF\nrFy5krPPPhuAG264geuvv54PPviA3/72t7S1tcWmATEknhHNacnttydbAkVRAhFsaXewjiCffPIJ\n+Z58GvMbAfj0408Zd+g41ryxhr2f7YVJNshu06ZNHH300QwbNozW1lYApkyZQk1NDZs3b+bwww/n\n8ccf59RTT8XtdrNv3z7OOeccTjrpJA499FAAmpubGTfOxvD+8Y9/jF7oOKIzBUVR0oJ4Le263W5u\nuu4mbvnaLVx/3vV89sln/PtN/84TTz3BT3/0U4455hhmzJjBm2++CcDChQtZtGgRM2bMwBjDo48+\nyiWXXML06dPJyspi0aJFtLa2cu6551JeXs7JJ5/ML35hHTCXLFnCJZdcwuzZsxk5cuTgBI8TOlNQ\nFCUt8F3a9aW6Gs47L/p6Z8+e3dPh+3P6qtP7Hbvooou46KKLesucfjrvv/9+nzJjxozhnXfe6Xft\nBRdcwAUXXBC9sAlAlYKiKGmBLu0mBl0+UhRFUXrQmYKiKBlHvbueDbs30NzWTElBCVNHTg0YvZyJ\nqFJQFCWjqHfX8+aXb1KQU0BxfjGe/R7e/PJNjjzoSHbu25nxikKXjxRFySg27N5AQU4BhbmFiAiF\nuYV0m27+UfsPPPs9fRRFvTv10lDEG1UKiqJkFM1tzRTkFPQ51tLRQnd3dx9FUZBTwIbdG5IkZfJQ\npaAoypCh3l1Pa0crL2x8gcqayoAj/ZKCEto6bSTxjd+4kXf+8Q77OvZxQK5NRfHMI8/wy9t/SUFO\nAc1tzVHJ8cMf/pDXXnst4usqKys599xzo7pnrFCloCjKkMBrK+g23SGXgKaOnEpbZxue/R7mnTeP\nV1e8SpZkUZxfDMDrf3md0887nbbONkoKSoLezxhDd3d3wHN33nkn8+fPj13jgtDZ2RnzOlUpKIqS\nHixeDAsX9n8tXgz02gqyJCvkEtDootGceMiJFOYWMnPeTN6tfJe5ZXPJkixqPq9hT/0eDp9xOE/9\n9iluvOhGysvLueOOOwCoqanhyCOP5Morr2TatGl8+eWXLFy4kGnTpjF9+nR++Uu7HczChQt55pln\nAHj33Xc58cQTOeaYYzj++ONpbW2lra2Nq666iunTpzNz5kwqAoRlNzQ0cOGFF1JeXs6cOXNYv349\nYKOir7jiCk466SSuuOKKmH/M6n2kKEp6sGMHTJrU/7gT4tzc1mxH+z571gdbAhpdNNp6Fk2Cp+Y8\nxRdrv+DE00/k9mW3c+JXT+Sjtz6ie08377/3PsYYzj//fFatWsWECRP49NNP+eMf/8icOXNYs2YN\nW7du5cMP7TYwTU1Nfe7T0dHB17/+dZ5++mmOO+44WlpaKCws5Fe/+hUAH3zwARs3buTMM89k06ZN\nfa694447mDlzJs8//zyvv/46V155JdVOZs6PP/6Y1atXU1hYGN1nGQKdKSiKMiTwtRV4GWgJCODS\nSy/lqaeeYnTRaN55+R1u/87t1K+v542KN5g5cyazZs1i48aNfPrppwBMnDiROXPmAHDooYeyZcsW\nbrjhBl566SWKi4v71P3JJ58wZswYjjvuOACKi4vJyclh9erVPaP8KVOmMHHixH5KwbfMvHnz2LNn\nD83NVsGdf/75cVEIoEpBUZQhgtdW0G26Mcbg2e+hrbONqSOnhrzuwgsv5O9//ztr167F4/Ewa9Ys\njDF8//vfp7q6murqajZv3sw111wDwIEHHthzbWlpKevWrcPlcvHQQw9x7bXX9qnbGIPdmZh+xwci\nUBlvXb4yxJq4KQUROUREKkRkg4h8JCLfC1H2OBHpEpGL4yWPoihDG6+tIEuyaGlvoTC3kBMPOXHA\nALSioiJcLhdXX301l156KQBnnXUWjzzyCG5nF5+tW7eyc+fOftfu3r2b7u5uLrroIu6++27Wrl3b\n5/yUKVPYtm0b7777LgCtra10dnZyyimn8OSTTwKwadMmamtrOfLII/tc61umsrKSkSNH9puJxIN4\n2hQ6gZuMMWtFZBiwRkReNcZ87FtIRLKBnwIvx1EWRVEygNFFo/HkebhgUmSZSC+99FIWLFjAU089\nBdgtOjds2MDcuXMBqzieeOIJsrOz+1y3detWrrrqKjo6O2jrbOObN36TyppKPPs9AOTl5fH0009z\nww034PF4KCws5LXXXuO6665j0aJFTJ8+nZycHJYtW0a+Nye4w5IlS7jqqqsoLy/ngAMOSNj+CxLO\nNCYmNxJ5AbjfGPOq3/Ebsaah44CVxphnBqgqIoErKytxuVyRXDJkyOS2Q2a3P13avm3bNsaOHRte\n4cWLrbHZn7IyuOeenrc1NTVMCmSQjhO+aTMKcgpo62yjrbMtrFlKrPG2Pcjn2n8dKwAJUQoiMglY\nBUwzxrT4HB8H/C8wD/gDYSiFqqoq097eHva93W43RYPZlimNyeS2Q2a3P13aXlxczIgRI2JaZ0dH\nB3l5eTGtMxStHa10m26ypHc13vt+WN6whMkBvW1vaGigpaWlzzmXyxWWUoi7S6qIFAHPAjf6KgSH\n+4BbjDFdgYwxgfBO58IlXUZM8SCT2w6Z3f50aXtEM4UwSfRM4YWNL1CcX9zHoGyMoaW9JeJlrMHi\nbXteXh6zZs2Kqo64KgURycUqhCeNMcsDFDkWeMr5MEcC54hIpzHm+XjKpSiKEitKCkrw7PdQmNvr\nIhqOK2yqEjelILan/wOwwRjzi0BljDFf8Sm/DLt8pApBUZSI9jzwLXto1qHUu+sTtp4/deRU3vzS\nbufpa1OYNSa6kXqyiWecwknAFcA8Eal2XueIyCIRWRTH+yqKkuZ4jbfhpLL2L9ttuhOa9to3bUYk\nrrCpStxmCsaY1YRp7XbKL4yXLIqipBe+ex4APX837N7Qr7P1L5slWT05jxLVMfekzRgCaESzoigp\nR6A9D4LlMYqkrD8ul4uXX+4bInXfffdx9dVXc/HFkcfSXnvttXz88cchyzz00EM89thjEdedKFQp\nDILWVrj3XnCCHhVFiRGR5DGKNucR9OY98uWpp57iqquu6sly6stAqaoffvhhjjrqqJBlFi1axJVX\nXjmgbMlClQLRd+4VFfDGG/avoiixw3fPA988RqbbsKRyCVe/cDVLKpewfsf6fmW7TXdYOY8ALr74\nYlauXIk39qmmpoZt27Yxfvx4pk2bBsCyZcs4//zzmTdvHqeffjrd3d1cd911TJkyhTPOOINzzjmn\nR4G4XC7ee+89wEZB33bbbRxzzDHMmTOH+npr41iyZAlLly4FYPPmzcyfP59jjjmGWbNm8dlnn+F2\nuzn99NOZNWsW06dP54UXXoj55xsKVQpE17m3tsKLL8LkybBypc4WFCWWBDLeDs8fzqPrHqXR08j4\n4vE0ehpZWrWUend9n7JZkhW2ofeggw7i+OOP56WXXgLsLOHrX/96vyR2a9eu5ZlnnuEf//gHy5cv\np6amho8//pjHH3+cqqqqgHXv3buXOXPmsG7dOk455RR+//vf9ytz2WWX8Z3vfId169bx5ptvMmbM\nGAoKCnjuuedYu3YtFRUV3HTTTWEl0IsVGb+fgn/nftppEE4gaEUFdHTAsGHQ0GDfn3de/OVVlEzB\n33i7pHIJpQWllBaWAvT8Xb5xOUtcS3rK1tTUhG30rXfXM2P+DH7+u59TckwJT/zvEzy2rP96/xln\nnNETeb169WouueQSsrKyKCsr47TTTgtYd15eXs/WmrNnz+bVV/tk+KG1tZWtW7fyta99DYCCAmsX\n2b9/P7feeiurVq0iKyuLrVu3Ul9fT1lZWVhtGiwZP1Pw7dw7OsKbLXgVifc7KivT2YKixJva5tp+\ndoKSghJqm2ujqs/ryjr7tNl88PYHfFD9AY2tjYybPK5fWd9U1eGO2nNzc3tmHNnZ2f3sEcHqefLJ\nJ9m1axdr1qyhurqa0aNH09bWFrBsPMhopRBt5+5VJN6khvn54SsURVGiY0LJhH4eRc1tzUwomRBV\nfe9ue5fd+3azs3MnR8w6gl9+/5e4znX1277Tn5NPPplnn32W7u5u6uvrqaysjOr+xcXFjB8/nuef\nt/G67e3t7Nu3j+bmZg4++GByc3OpqKjgiy++iKr+aMlopRBt575uHRhjdwH0vowBZ6c8RVHiwIIp\nC2hsa6TR00i36abR00hjWyMLpiyIuK56dz1bGrcAkJ+dz7FnHsvnGz/n5HNOHtCV9aKLLmL8+PEc\nddRRXH755cyaNYuSkuhSWjz++OP8+te/pry8nBNPPJEdO3Zw2WWX8d577zF9+nQee+wxpkyZElXd\n0ZKw1NkxJGaps++6Cz77rP/xww6D22+PRrTUIl2SosWLTG5/urQ90oR463esZ/nG5dQ21zKhZAKn\nTjgVyZI+qTA8uz0DJsSrrKnk0z12e83c7FwA9nfZzZ2POOgIXJNcIa/3ZqHds2cPxx9/PP/85z8T\ntuYfilikzs5oQ/NQ6PgVJZMoLyunvKwc6LuPgW8qjGn50/pd559HaUfrDsqKyqhrrYMuyMmyXaGn\n0xOWK+u5555LU1MTHR0d3H777SmhEGJFRimF1lZ48EG47rrwPIwURUldgqXC8A9kC6Q8Gtsbyc3O\n5ZDiQ9i9bzdtnW1kSzaHlh4aludStHaEdCCjbAoabKYoqUe0S9jB0lt0dXf1OearPESEwtxCDj7g\nYHbu3Um2ZDOxZCITSyYy8oCRHDf2uKjbkSoM1iSQMUpBg80UJfXIzc3F7XZH1ZEFS2+RndV3H+VA\nymNE4QhKC0p7At66TBe52bm8VfcWlTWVCcuwGmuMMbjdbnJzc6OuI2OWjzTYTFFSjxEjRtDQ0EBr\na2vE147sGslHez6iPbud3Kxc9nfvp72rnbbcNrZt29ZTLndfLs2tzeRl927R2dHVQXF2MZPzJtPQ\n1cBHuz4iPzuf3KxcmlqbqNxRydGjjmZEYWy3Co03jY2NFBUVDWqL04xQCsHiEcKNXlYUJT5kZ2cz\natSoqK4dy1i6D+zu4420YMoCGjY2MHZ2r+fN7qzdLK1aSmlBKSUFJTS3NdPY1sjNc29mbNlYflf5\nOxo9jT0R0gCNnkY2tG1giWvJYJuYUDZt2hT15+klI5RCqHiEZMwW1OCtKOHj74a6YMqCHg8kX28k\nL5UbK/u8Ly8r5+a5N/ep45qZ1/RcV9tcy/ji8X2uGUykdLqTEUrBN9jMl+rq5CgFr8F76lRdwlKU\nUKzfsb5nlO+bBO/muTf3UwahCKQ8vEwomcCm3ZvY5t7W47I6tmgsk0dOjlUz0op47tF8CPAYUAZ0\nA78zxvzKr8xlwC3OWzfwf4wx62ItSyrFI0SbgE9RMpHlG5cHTYIXiVIIxbRR03hs3WMU5xVTnF9M\nk6eJ2uZaFkyNPFJ6KBBP76NO4CZjzFRgDvAdEfHffeJz4FRjTDlwF/C7OMqTEkSTgE9RMpVYJ8EL\nxIe7PmTu+LkMLxxOa0crwwuHM3f8XD7c9WHM7pFOxHOP5u3Aduf/VhHZAIwDPvYp86bPJW8BfRf2\nhhhq8FaUyJhQMqGfEXgwSfACUdtcy2EjDuOIg47oOdZtujPWppCQOAURmQTMBN4OUewa4G+JkCdZ\naHZVRbFDMagdAAAgAElEQVR2Av/d04IRyyR4wYh19tV0J+4J8USkCPgH8GNjzPIgZU4DfgOcbIzZ\nE6q+qqoq4906Lxy8iatSge3bIZDo+fkwZkzs75dKbU8Gmdz+VG27p9NDvbue7KxssiWbLtNFV3cX\no4tGU5hTGPSaprYmOro6yMvOY3jB8KBlIfK2RyNTqhKq7S6XK6yEeHFVCiKSC6wEXjbG/CJImXLg\nOeCrxphNYVQbsyypQ51MbjtkdvtTte1LKpcEjAkoLSyNOibA32X1hM4T+Or8rw6qDl+313RigO89\nuVlSxW459AdgQwiFMAFYDlwRpkJIGOkUS5BOsiqZTaxjAgK5rNa317N+x/qYuaxmGvG0KZwEXAHM\nE5Fq53WOiCwSkUVOmR8CBwG/cc6/F0d5IiIRyfNaW+Heewefh0kT/SnpQqzX731dVrMki9LCUrKz\nslm+MeBKtRIG8fQ+Ws0A0xVjzLXAtfGSIVoSFUsQiyA2jXtQArJ1Kyxc2P94WRncc0/CxfGyYMoC\nllYtBeiTcuKamddEVV+gmUe2ZGes51AsyJgsqZEQy1iCYLOBWGVt1bgHJSD798OkSf1fO3YkVSxv\nyonSwlLqWuooLSyNODrZl0Azjy7TlbGeQ7EgI9JcREKsYwmCzQZikbVV4x6UdCTS9ftQRuBAM4+u\n7q6YuqxmGjpT8COWsQTBZgPBOvNIZwsa96AMdbyG5EZPY5/cR97YhkAzj9FFo9VoPAh0puBHLJPn\nBZsNxCpra6ol+lOUWBNO7iP/mcdQ3iozEahS8CNWyfNCLe1E05kHcjtNpUR/ihIPNK114lGlECdC\nzQai6cw13bYSEbm5/Ucd0DtKiRWLFwc2XsfIyykRuY+UvqhSiBPr1lkl8Je/wJQp9jcK0S3tqNup\nEjHjxsGyZfG/z44d1qvJn0AKKQpi7cKqDIwqhThx++2wYgX8/vfwjW8MbnSv+0srmcpAu6YpsUeV\nQpyI1ehe3U6VTEdTUCQWdUmNE7EKKlO3U0VREokqhSgYKGdRrOIQoK+nkvdljLVNKIqixBpdPoqC\ngTyBYhWHAOp2qqQ4ZWWJ8XKKM0MldXYsUKUQIeHYCjSoTMkYkphcL1YESr+9tGrpoHIypTOqFCIk\nHE8gHd0rSvREMmoPVDZSvFHTHV0drPpiFc1tzeRl5/Hgew/y4LkPDrY5aYfaFCIgUltBrPZLUJSY\nsXixTant/1q8OKlieRko11E4ZT2dnojuWdtcS1tnG29++Sae/R6K84sxxvDqlldD7h89VFGlEAGR\negLp5jdKyuENNkuxlNpeAm2aU1pQGnDTnGBlm9qaIrrnhJIJVO+opiCngMLcQkQEEeGgwoMycrMe\nVQoREIknUKz2S1CUTKK2uZaSgpI+x4LlOgpWtqOrI6J7LpiygD2ePRgMxhg8+z20dbUxo2xGRuZY\nUptCBERiK/C3Pbz4Inzxhe6jrCihiCTXUbCyedl5Ed2zvKyc+YfO5/3t79PS3kJJQQkzx8wkPzuf\nMYVjom9MmhK3mYKIHCIiFSKyQUQ+EpHvBSgjIvJrEdksIutFZFa85Ik3ra1w111w992wfXt/28Mj\nj1hFoUtJihKcBVMW0NjWSKOnkW7TTaOnkca2xoAG5GBlhxcMj/i+1x17HUeOPJJTJp7CKRNPIT87\nP+h9hzrxXD7qBG4yxkwF5gDfEZGj/Mp8FTjCeX0bSFtTf0WFzXX0wgvwwAN9bQ9gl5ry83UpSVFC\nEWq7zvU71rOkcglXv3A1SyqXAAQsW5hTGNP7ZhpxWz4yxmwHtjv/t4rIBmAc8LFPsQuAx4wxBnhL\nRIaLyBjn2rShtRWeew7a2+37v/7VBrZ54xRqa6GrC1pa7NLR9dfD/ffrMpKSBBIQbDbYQLBAuY5C\nxRIscS3pU7ZyY2VUcmuOJYvY/jjONxGZBKwCphljWnyOrwTuMcasdt7/HbjFGPNesLqqqqpMu7f3\nDQO3201RnHvf5maor7cdvzGQkwOjR0NJCXR3Q10dZGdDVhZ4PLB3L4wfD6W9S6F0d8OuXTBqlC0X\nCxLR9lQmk9ufrLZ7Oj3Uu+vJzsomW7LpMl10dXcxumh0VCN4L9vd2+ns7iQnq3cc630/pqjvur9+\n74Hb7nK5JJw64m5oFpEi4FngRl+F4D0d4JKQWmru3LkR3b+yshKXyxXRNZHQ2grf/S68/37vyN/t\nhlmz4Ne/htdft0tLEyfamcQrr9i/48bBs8/2XrNiBTzxBHz727GLeo5321OdTG5/stq+pHIJjfl9\njb+NnkZKc0r7jegj4eoXrmZ88XiypHfE1G26qWup4xHXI33KRtP2oZLmIhbfe1xdUkUkF6sQnjTG\nBHL4rQMO8Xk/HtgWT5liTUWFXR4SsTOEHEfNbtlil4nefbfXjfXtt+2sIicHdu7sNTqr+6oyVIjE\npTQSJpRMoLmtuc+xWO3AFknAXCYQT+8jAf4AbDDG/CJIsRXAlY4X0hygOd3sCevWWZfT1la7TFRX\nZzv12lqoqoLjj7cbYP3P/8ARR8All8CFF8LZZ/cqgFil2VaUZOPbede766msqeSZj59hS+OWQXWy\nkXglRUokAXOZQDxnCicBVwDzRKTaeZ0jIotEZJFT5q/AFmAz8HvgujjKExduv90uHW3e3Pt6/304\n4ww499z+Hb9/NPSLL4afOkPTZihB8U9fUVOTlPQV3s770z2f8s/af9LkaSInK4exRWMHNfqOp3dQ\nvGY36Uo8vY9WE9hm4FvGAN+JlwzJIlDSvGCZU194wSqIcNJsD5SyG6ziePBBGySnZBD+eyXn59v3\nMdorOVy8nfd3X/ounaaTgw84mCkjp1BWVEajp5HlG5dH3ZEPxjsolM0gkoC5TEAjmgeBbwfsNRgH\nS5r3858HdkG96y747LOB02yHu72nr+IYNiwmzVSUiCgvK+fQ0kM5ZeIpfQzDyRp9D5Qae8GUBSyt\nWtojY3NbM41tjVwz85qEy5oKqFIYBIFG7pFusBNu6oxwUnb7K46vfz36tinKYEil0bevzQDo+eud\ntXhnN74ziWtmXpOW3kexQJVClAQbucdigx3/GUiw2Yf/bMFfcbS2xqSpihIxqTT6rm2uZXzx+D7H\n/GctGrjWiyqFKAk2cvcd+QdaXgq3bt8ZSDizj0CKo7nZGqUzNI5HSSLhjr4TER+QSrOWdEBTZ0dB\nuJvtRLOfQqCYhXBSdgdSHN3d6t6aMZSV2WjIJ56wrz177N81a5K2gU55WTlLXEt45IJHWOJaEjR1\nRbzjA+LpzjoU0ZlCFEQych/IMBysbt8ZSDh2h0DLVpMm6b7QGcM99/T1QDroILj8cvt/gj2QwmWg\ntf5YEWjW8i8T/oXlG5dz39v3pXUEczxQpRAF4dgNwjEM+xOu7SAQgRRHZaV1VVeUVCSctf5Y4Wsz\nGMgbKdNRpRAFA43c/Tv30lK7z8Jxx4VORhmp55KipDPJWutP1AwlXVGbQhzw79y3b4dt2+w+C6GI\nZLtPRUl3krXWrxHModGZQhzw7dz377e2vuJi+Nvf4JZbgi8FRbLdp6IkjcWLrf3Cn7Iya9twGMiz\nKJiHEthsq/HySFJvpNCEVAoikg1ci81e+pIx5p8+535gjLk7zvKlJb6d+4oVkJdnU2d/8YUuBSlx\nxHcDHd8UFzHcQAfon1LDi4+RLdx1e//4gESs96dSDEUqMtBM4bfAAcA7wK9F5B/GmP90zi0AVCmE\nwLsjW00NjBkTmeFYUSLGZ5ROZaVNz5skol23T8R6v0Ywh2YgpXC8MaYcQETuB34jIsuBSxkg2Z3S\nu9fC1q3w+edw5JFqOM5owlx2GfQ1sbh2kETrWZQojySNYA7OQEohz/uPMaYT+LaI/BB4HdCx7gC8\n8w58+SXk5lq7QlaW/V9jBzKUMJZdYnJNLK4dJNGu2+t6f/IZSCm8JyJnG2Ne8h4wxtwpItuAB+Mr\nWvpz/PF2luC1JyxYYJWBd1+ESNNfKAqvvWbD3N3uvkEogxn9e2cUa9bYTcS9FBbC7NlR1R3tur2u\n9yefkC6pxpjLfRWCz/GHjTG58RMr/QmVCqOiwr6uv143zFEixO2G4cPtaGLSpN5XoGWicPHOKLKz\nYfz43ld2duC6vQZt/5ePQTvaTXHiuZmOEh4DuqSKyEHAN4EpzqENwJ+MMXviKVi6E2qntddft+9f\nfx3++lf4t39LrqyKEhFhzhqiXbf3XuM1BHu3xVTFkBgGckmdirUfvAy8jzUuHwfcKiLzjDEbQ1z7\nCHAusNMYMy3A+RLgCWCCI8dSY8yj0TYk1Qi101pWlrU1FBfDH/4A55yjy0iK4iWQW+qtr9/KIcWH\n0N7VTmt7K583fU5zezPjho3j+uOu5+KjL0622EOGgWYKdwHfM8b82fegiFwE/Bi4KMS1y4D7gceC\nnP8O8LEx5jwRGQV8IiJPGmM6wpI8xQkUiNbaCjffbBNYdnXZVYDPP+87W4g23baSBvjGEfgfD/ca\n73pjOA+H/7Vem0FhYV97xJo1gQ3SScLfLbW9q53PGj5j195dHFJ8CK9ueZUsyeIrw79Ck6eJ/3rt\nvwBUMcSIgZTCdGNMv0/aGPOsiPwk1IXGmFUiMilUEWCYiAjWk6kB6BxAnrSmogL27oXNm6GpySbL\nKyjoO1sIZx9mJU2JxhDsf83CheF34OFeu3p15HLFEX+31I27NzIsfxgdXR28t/09CnIKyMnKoaGt\ngUnDJwFw/7v39ygFT6cnrhHRQx0xxgQ/KbLWGDMr0nM+ZSYBK4MsHw0DVmBtFcOArxtjXhxI4Kqq\nKtPe3j5QsR7cbjdFSR5yd3fDrl12duB2Q1ub/T8nx9ryRGDsWKsk6urs8lJ3t7X1ZQ0iO1UqtD2Z\nDMn2b9wInQHGTjk5MGVKz9uAba+p6TVy+dLYCAccAPv22QfPS1aWPZ6bC+PGxUR8T6eHprYmOro6\nyMvOY3jBcApzCvuU2e7eTmd3JzlZdsza5GlCRMiSLDz7PXbfZwFjDAU5BRgM+7v2M+3gaXg6Pexz\n72Nf1j6yJZsu00VXdxeji0b3u89QJNQz73K5wootG2imcLCI/GeA4wKMCucGITgLqAbmAYcBr4rI\nG8aYllAXzZ07N6KbVFZW4nK5opUxJqxYYfc7+fa34eOP4c9/tr+z/fthxgzYsgXOPtu6sFZU9Hdh\njZZUaHsyGVLt97qNrl7dd+moqAjmz7cd/qJFPYcDtt13puB1bQX79+ST7f9xDGzztRWUHOi4m7Y2\n9vMu6imXW0pJQQkvb3+ZlvYWXJNc/O3Lv+HZ7yEnK4fc7FwmDZ9Ek6eJ4YXDud51PUsql3BY1mF8\nVvxZT32NnkZKc0pZ4loSkzbEe6e4wRCLZ36gcejvsaN4/1cR8PCg7gxXAcuNZTPwOb0eTkMG/812\npk+Hk06CSy6xfydPtikwpk8Pbzc3JUPxuo0WFVljlPcV7QPidW31d28djGvrAPjaCrIki9LCUkoL\nSnu8i7z4u6XOHDOTw0ccTl52HseOOZa2zjb27d/HiIIRNHmaaOlo4frjrgfs0lO2ZPepL1YR0Yna\nKS7ZhJwpGGN+FOyciNw4yHvXAqcDb4jIaOBIYMsg60w5fDfbqa+HRx+FmTPtudJSu4Pi/PnwyCMw\napRVEKB7KShDj0hSWARKlLd843JaO1o5YewJfLjrQzY3bmZE4QhuOO6GHnvChJIJdLV09akrVhHR\nmbIPw2BSZ/8ncF+wkyLyJ8AFjBSROuAOIBfAGPMQ1rNpmYh8gF2OusUYs3sQ8qQc/gFsbW3W22iW\nY4nZvt12/KtW2fedndaN9ZNPbJ4kTYmhxBRfbyTfGUaC7C6DSWHhVRLe0frUg6f2RDyv27mO9TvW\nU15WzoIpC1j71loaPY0xj4hO5E5xyWQwSiGk0cIYc+kA57cBZw7i/imPfwBbQ4M1ML/9tv19rllj\nFUFdnc2cetBBMG+etT984xuqDJQY42sriMSLKUbEIoXFQKP18rJythZtpTSnNOYZUDMlL9NglEJw\ntyUF6B/ANmGCfR12GBxzjPU62rDBeh59/rn1RHr0UTj6aE2xrQShqMj6M3txu/ulmEhVYpGyOpzR\nemFOYUyMyv5kSl6mgSKaWwnc+Qsw9P27BkmondTuustu0dncbH/nbrdVENnZ1uawZo2mwFB88C79\nHH54/+PRpNBes8a+Zs/uX18cGWzKat/Rer27ng27N7Bz704OPvDgniWkeJEp+zAMZGgelihBMo0b\nb7TLRiecYJeXWlrsdp1nnGF/+01NA6fA0OjnDCJcN1GvAnC5eqOWV6+2nf38+b3lvDuzJXEjnnDw\ndwGdNmoaKzatYPe+3XxQ/wFZkkVOVg5ji8b27NAWTzJhH4ZBhEYpg8Hf3rB9u41b+OIL2LTJeiJ5\nU2CEquONN+xfRQF6XVfz83vdTL1T0TQjkAvoik0rOH/y+Wxt3Uqn6WR44XBOPOREJo+cHNC9VYmc\nwdgUlEHgb2/Y6KQWXLvWBpIWFdng0hdeCLyE5B//oPYHZagRzKj84a4PObT0UE6ZeIqNbnbosS2U\nJEXcIYMqhSQRKmHeiBF2oNfebj2W3O7+Hb5v/ENDg8YzKEOPUEblTPEESgaqFFKIYHsw+Hb4ra1w\n3312ack/+llnC0OEcPdWDlRu9WqbcfHII0Pf47XX7LW+2VID3SOJhOr4Q3kCNWxsSJbIQwJVCilE\nsD0YfAPYKipsLqXiYuveChr9POQId2/lQOWqq/vbD4qKbFnf63fssArA//oE7N/sS6hcQqE6/lCe\nQJUbKxPahqGGKoUUIpQLK1gX1h//2LqtfvklfPqpjXr2otHPQ5DXXrOeB52d1hPBm+a60PEI9+/U\nvQqgvb23gz/8cJvwLsnBa/4E2kzH60Hk9fIJ5QKaCZ5AyUCVQhrxwANWMZx8st1vYbBZVJU0wO22\nngfFxTZPSkeHfTU0WCXx/PO2nDdbqjdj6qRJKe9uGk4uIe34E48qhTRh2zZYvtwmzNu0ybqhqx0h\nA+nosDsztbZapeC1KXR09Hok+Ae4pSiZkkso3dA4hTThgQdsH3DAATZ/0rZtvXYEJQPp7rbriAUF\n9pWXN7hU2klgQskEmtua+xxTD6LkozOFNKC1FV55xf7uP/nEuqyuWQPHHad2hCGJfzbTDmfb8ry8\n3v/BLiu1tdn/OzpsGLzbHV6qirIym7fd4+l7vLDQejUlwAMpU3IJpRuqFNKAigqYNs3+/tesgfJy\nOzhUm8IQxd8gXF1tZwHQG+UINkjFuwVnUxNceKFVJvfcA5WVA98jXC+nOJEpuYTSDVUKacA779h+\nobXVup/qLCGDKCuz64V1dfb93r12VmCMnTmkOWpITj1UKaQBxx8PL79s924+/nh49VW44ALNoJoR\n3HNP4IA1717N3jTa6m0QlFTfVznVUKWQongzoF5xBTz3nHU737PHxiaEk0FVGaJ4FUSgaOY02Vch\nkQwUC5HqJEOhxU0piMgjwLnATmPMtCBlXNgtPXOB3caYU+MlTyoSKvW1NwNqSwvU1lqbYnc3vP8+\nHHxwbwZVnS1kKCmSiiLVSed9lZOl0OI5U1gG3A88FuikiAwHfgOcbYypFZGD4yhLSuLt+KdO7Wsb\n8GZAnTTJOoiUOqlfmpuhsdG6pYbKoKokgHDzEyVLFpdrYC8iXy8n/+NDhHSOhUiWQoubUjDGrBKR\nSSGKfBNYboypdcrvjJcsqUio1NfexHgtLTY2YeJEqyBeeQVKSux2nZMmBc+gqiSAJHvu9CGQLPn5\ngZWWLxkw20jnbKrJUmhiTPy2WnaUwspAy0ci4l02OhoYBvzKGBNwVuFLVVWVaW9vD1sGt9tNUQr2\nmt5RvzdFdmmp7fC7u62jSXa2dTTp7LTHcnOtgsjKgpwc643oe10gUrXtiSKu7a+p6U1n60t7e+Jz\nCgWQxZ2fT1FLS9LzG4XC0+mhqa2Jjq4O8rLzGF4wnMKcwe/y6/u9ezo91Lvryc7KJluy6TJddHV3\nMbpodEzuFU+2u7fT2d1JTlbv2N37fkzRmIDXhHrmXS6XhHPfZBqac4DZwOnY/Z6rROQtY8ymUBfN\nnTs3optUVlbicrmilTEuBNs34ec/h9dftzOFiRN7y3/xhVUIOQG+rcMOC55ILxXbnkji2v5gCeUG\ns8VltEtS/rK89hqV3/wmrl/8wibKCreeROC0cX1uA0tLPqK0O4+S7jyai3JpnH1UTNbL/b/3dPU+\n6rEp5Jb2Bvftb+Tm44N/RrF45pOpFOqwxuW9wF4RWQUcA4RUCkOBYPsmPP647eBPOaX/KsSRRw6c\nRVVJc2K1JOV22xFEUVHf+mpqkm8Lcdq4nBpKGUkpdrRe2tQEznaase6w0zUWIlnBfclUCi8A94tI\nDpAHnAD8MonyJIxg+yY88ICdReTmBh9shvJYUpQBiZMtJNLReC3NjKe4z7F0MQAnkmQotHi6pP4J\ncAEjRaQOuANrQ8AY85AxZoOIvASsB7qBh40xH8ZLnlQi0Ih/0yYbmFZSYgPVPvvMLg35E8xjSUkw\nqeS54y+L222NUf6jhjVrbK6j6uq+xweZWTUa18kJlNCIp2emAPE1AMd6CSldl6TCIZ7eR5eGUeZn\nwM/iJUM6cdttNpvBAQfY3+2tt8LTT/ct4++xdOyxdslJZw1JINlr8774y7JwofVAmD+/73GPxz4o\n3jxKXrxR0VESjevkAqawlCoASiigOas9bsnwYu3vn+4BcQOhqbNTgE2bbOqKYcPs+2HDemcLvnht\nEcOG2b8PPGBnDZo+W0kmtc21lBT0dYEbaCmonDJuZi6lFFJHC6Xd+XHrVH2VVpZkUVpYSqljv0iF\n+lINTXORAnhnCTk51lV12DD73ne24J0leFcnSkttYNv8+brZzpAhVktSZWV9t+P0UhgfF8yIYgF8\n2lgOlDOp93icRtmx9vdP54C4cFClkAKsWWP/7tplZwBtbVZBvPtubxl/j6Xt262b6u7dNo12RYXa\nGNKeWC1JeVNn+3srLFwImzf3Xy4Kdw+GIES0L0ISlt1iHcCWzgFx4aBKIQXYsqU3dqGoyP5Gf/7z\nviN/X4+l/futIikshPp6OOEEnS0oYeJvZ4DePRiiJNX3RYj1Zj5DfXMgVQopgq+9oKGh/8jf12Np\nxQqbSt83wM27NafOFjIcbxyCy2VnBl7KyuLqMZXKsQCxVlqprgQHiyqFFMDfXlBWFnrkHyzOQTfd\nUXriEPLz+weuRRtpPQSItdJKZSU4WFQppADBIpyDjfw1sjnDSHYUspJRqFJIAXTkr4QklTKyKkMe\nVQopgI78FUVJFTR4TVEURelBZwqKMhhSbb3f62E0aVLf5aUhtJuaEl9UKSjKYEi19X6vIgoUvKYo\nYaBKQVFSnVTKyKoMeVQpKEqqkw5up6m2jObHUE51HWtUKSjKUCcRHXaqLaP5MNRTXccaVQqKMtRJ\n4Q47EUSz30Mmo0pBUQaDrvenPEM91XWsUaWgKIMhBdbLldAM9VTXsSZuwWsi8oiI7BSRkPsui8hx\nItIlIhfHSxZFUTKXBVMW0NjWSKOnkW7TTaOnkca2RhZMWZBs0VKSeEY0LwPODlVARLKBnwIvx1EO\nRVHijXcZzf+VAsto3lTXpYWl1LXUUVpYqkbmEMRt+cgYs0pEJg1Q7AbgWeC4eMmhKEkn2e6aGzfC\n6tWwZw90dvYeLyiwey7EQo4UX0YbyqmuY40YY+JXuVUKK40x0wKcGwf8LzAP+INT7pmB6qyqqjLt\n7e1hy+B2uynK0O3IMrntkELtr6npzYvuS3t7YK8ggK1b7RZ7/uTmwrhxA96yT9u9929stPu8euns\ntJt9h5IjDUmZ7z0JhGq7y+WScOpIpqH5PuAWY0yXSFiyAjB37tyIblJZWYnL5YpMsiFCJrcdUqj9\nCxcGdwldtizwTGL1ajuC998+M8zNcvq03Xv/55+H4cN7CzU1wYUXDrkNeFLme08CsWh7MpXCscBT\njkIYCZwjIp3GmOeTKJOiJJ5AcQTV1Xaz7lQk2cthSlxJmlIwxnzF+7+ILMMuH6lCUJRUJ8OD4YY6\ncVMKIvInwAWMFJE64A4gF8AY81C87qsoiqJETzy9jy6NoOzCeMmhKEkn2qjnbdusHcAXt9su30Sy\nTOO9f1cX1NX1Hi8sTBm3USV10IhmRYk399wTeB1+xw57PBBFReDx9D9eVhZ4PX+g+ytKmKhSUJRE\nEGodPtBM4vDDobnZegcFukZR4oQqBUVJNsFG8gsXJlSMsNEkgEMaVQqKokSGLkcNaeKZ+0hRFEVJ\nM3SmkIG0tsKDD8J111l7ppIkXnvN2hr8l4m8QWDxWKbRwDNlAFQpZCAVFfDGGzB1Kpx3XrKlyRAC\ndfA7dtjj/gZob7l4dNIaeKYMgCqFDKO1FV58ESZPhpUr4bTTdLaQEAJ18MFyIilKElGbQoZRUQEd\nHTBsmP1bUZFsiRRFSSVUKWQQ3lmCd0m6rMzOFlI175qiKIlHlUIG4Z0leFP75+frbEFRlL6oTSGD\nWLcOjOlvU6yuVoNzUkhGEJgGnikDoEohg7j99mRLoPQhGS6gaeR2un7HepZvXE5tcy0TSiawYMoC\n3VIzAahSUBQl6fgrgGmjprFi0wpKC0oZXzyeRk8jS6uWcvPcm1UxxBm1KSiKklTW71jP0qqlNHoa\nexTAXavuoqu7i9LCUrIki9LCUkoLSlm+cXmyxR3yqFJQFCWpLN+4nNKC0j4KYH/3fupa6vqUKyko\noba5NklSZg6qFBRFSSq1zbWUFJT0OTbqwFHs2rerz7HmtmYmlExIpGgZSdyUgog8IiI7ReTDIOcv\nE5H1zutNETkmXrIoipK6TCiZQHNbc59j44rGkZudS6OnkW7TTaOnkca2RhZMWZAkKTOHeM4UlgFn\nhzj/OXCqMaYcuAv4XRxlURQlRVkwZQGNbY19FEBOdg63/8vtlBaWUtdSR2lhqRqZE0Q892heJSKT\nQhNR2CoAAAV2SURBVJx/0+ftW8D4eMmiKErqUl5Wzs1zb+7jfXTNzGsoLyvnYi5OtngZhxhj4le5\nVQorjTHTBih3MzDFGHPtQHVWVVWZ9vb2sGVwu90UZWjGt0xuO2R2+7Xt2nZ/XC6XhFNH0uMUROQ0\n4Brg5HDKz507N6L6KysrcblckQs2BMjktkNmt1/b7kq2GEkhFm1PqlIQkXLgYeCrxpg9yZRFURRF\nSaJLqohMAJYDVxhjNiVLDkVRFKWXuM0URORPgAsYKSJ1wB1ALoAx5iHgh8BBwG9EBKDTGHNsvORR\nFEVRBiae3keXDnD+WmBAw7KiKIqSODSiWVEURelBlYKiKIrSgyoFRVEUpQdVCoqiKEoPSQ9eUxQl\nCIsXw4svgsfT93hhIfzrv6bVLmpK+qBKQVFSlR07IDsbxvulBWtqsucUJQ7o8pGiKIrSgyoFRVEU\npQdVCoqiKEoPqhQURVGUHtTQrCipSlkZdHVBXd8N7CkstOcUJQ6oUlCUVOWee9TtVEk4unykKIqi\n9KBKQVEURelBlYKiKIrSgyoFRVEUpQdVCoqiKEoPqhQURVGUHlQpKIqiKD2oUlAURVF6EGNMsmVQ\nFEVRUgSdKSiKoig9qFJQFEVRelCloCiKovSgSkFRFEXpQZWCoiiK0oMqBUVRFKUHVQqKoihKDxmh\nFETkZyKyUUTWi8hzIjI82TIlChG5REQ+EpFuETk22fIkAhE5W0Q+EZHNIrI42fIkEhF5RER2isiH\nyZYl0YjIISJSISIbnGf+e8mWKVGISIGIvCMi65y2/yjaujJCKQCvAtOMMeXAJuD7SZYnkXwILABW\nJVuQRCAi2cADwFeBo4BLReSo5EqVUJYBZydbiCTRCdxkjJkKzAG+k0HffTswzxhzDDADOFtE5kRT\nUUYoBWPMK8aYTuftW8D4ZMqTSIwxG4wxnyRbjgRyPLDZGLPFGNMBPAVckGSZEoYxZhXQkGw5koEx\nZrsxZq3zfyuwARiXXKkSg7G4nbe5ziuqdBUZoRT8uBr4W7KFUOLGOOBLn/d1ZEjHoPQiIpOAmcDb\nyZUkcYhItohUAzuBV40xUbU9J7ZiJQ8ReQ0oC3DqNmPMC06Z27BTzCcTKVu8CaftGYQEOKYJvjII\nESkCngVuNMa0JFueRGGM6QJmODbT50RkmjEmYtvSkFEKxpj5oc6LyLeAc4HTzRDLAjhQ2zOMOuAQ\nn/fjgW1JkkVJMCKSi1UITxpjlidbnmRgjGkSkUqsbSlipZARy0cicjZwC3C+MWZfsuVR4sq7wBEi\n8hURyQO+AaxIskxKAhARAf4AbDDG/CLZ8iQSERnl9aoUkUJgPrAxmroyQikA9wPDgFdFpFpEHkq2\nQIlCRL4mInXAXOBFEXk52TLFE8eh4HrgZayh8c/GmI+SK1XiEJE/AVXAkSJSJyLXJFumBHIScAUw\nz/mdV4vIOckWKkGMASpEZD12YPSqMWZlNBXpfgqKoihKD5kyU1AURVHCQJWCoiiK0oMqBUVRFKUH\nVQqKoihKD6oUFEVRlB5UKShKmIiIO8CxJSKy1XF//FRElvsmYROR651srUZERiZWYkWJHFUKijJ4\nfmmMmWGMOQJ4GnhdREY55/6JDST6ImnSKUoEqFJQlBhijHkaeAX4pvP+fWNMTVKFUpQIUKWgKLFn\nLTAl2UIoSjSoUlCU2BMoU6uipAWqFBQl9szE5l1SlLRDlYKixBARuQg4E/hTsmVRlGhQpaAo4XOA\nk3nU+/pP5/h/eF1Sgcuxe+XuAhCR7zpZascD60Xk4STJrihhoVlSFUVRlB50pqAoiqL0oEpBURRF\n6UGVgqIoitKDKgVFURSlB1UKiqIoSg+qFBRFUZQeVCkoiqIoPfz/CVTKuZlUPwcAAAAASUVORK5C\nYII=\n", "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "plot_pca()\n", "plot_step_lda()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "
\n", "
" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The two plots above nicely confirm what we have discussed before: Where the PCA accounts for the most variance in the whole dataset, the LDA gives us the axes that account for the most variance between the individual classes." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "
\n", "
" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "# LDA via scikit-learn" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "[[back to top](#Sections)]" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Now, after we have seen how an Linear Discriminant Analysis works using a step-by-step approach, there is also a more convenient way to achive the same via the `LDA` class implemented in the [`scikit-learn`](http://scikit-learn.org/stable/) machine learning library.\n" ] }, { "cell_type": "code", "execution_count": 20, "metadata": { "collapsed": false }, "outputs": [], "source": [ "from sklearn.discriminant_analysis import LinearDiscriminantAnalysis as LDA\n", "\n", "# LDA\n", "sklearn_lda = LDA(n_components=2)\n", "X_lda_sklearn = sklearn_lda.fit_transform(X, y)" ] }, { "cell_type": "code", "execution_count": 21, "metadata": { "collapsed": false }, "outputs": [], "source": [ "def plot_scikit_lda(X, title):\n", " \n", " ax = plt.subplot(111)\n", " for label,marker,color in zip(\n", " range(1,4),('^', 's', 'o'),('blue', 'red', 'green')):\n", " \n", " plt.scatter(x=X[:,0][y == label],\n", " y=X[:,1][y == label] * -1, # flip the figure\n", " marker=marker,\n", " color=color,\n", " alpha=0.5,\n", " label=label_dict[label])\n", " \n", " plt.xlabel('LD1')\n", " plt.ylabel('LD2')\n", "\n", " leg = plt.legend(loc='upper right', fancybox=True)\n", " leg.get_frame().set_alpha(0.5)\n", " plt.title(title)\n", " \n", " # hide axis ticks\n", " plt.tick_params(axis=\"both\", which=\"both\", bottom=\"off\", top=\"off\", \n", " labelbottom=\"on\", left=\"off\", right=\"off\", labelleft=\"on\")\n", "\n", " # remove axis spines\n", " ax.spines[\"top\"].set_visible(False) \n", " ax.spines[\"right\"].set_visible(False) \n", " ax.spines[\"bottom\"].set_visible(False) \n", " ax.spines[\"left\"].set_visible(False) \n", " \n", " plt.grid()\n", " plt.tight_layout\n", " plt.show()" ] }, { "cell_type": "code", "execution_count": 22, "metadata": { "collapsed": false }, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYUAAAEUCAYAAADEGSquAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJztvXl8lNW9+P/+ZI+GhCBIWATqgqAQWVxAvToiLvW6FfW2\n1qW43P74Wm29V79XqrVStb3W0tb2arWtVepyq/0qKsXWrSZFatzAgAuIiDGGJSxZByYJSc7vj/NM\nMpnMTGYms2Y+79drXsk8z3nO8zkzz5zPOeezHDHGoCiKoigAWckWQFEURUkdVCkoiqIoPahSUBRF\nUXpQpaAoiqL0oEpBURRF6UGVgqIoitKDKoU4IiKXicgrKSDHRyLiSpd6UwkRcYlIXQzru1tEdovI\nDhGZICJuEcmOVf2DkKtPO1P5uxWRGhGZ7/x/q4g8HOP6/0VEPony2pT5TqMlrZWC78Phd9wlIt3O\nl+MWkToR+bOIHBegrIjIFhH5OFb392KMedIYc2ak9cYaY8zRxpjKwdQhIstE5O5Y1xtrBvpOwrje\niMjhsZTJp+5DgJuAo4wxZcaYWmNMkTGmK4q6ForI6gHKLBWRT0WkVUQ2isiV4dafit9tIIwxPzHG\nXBvjOt8wxhwZ5bVRf6eREq9nNa2VwgBsM8YUAcOAOcBG4A0ROd2v3CnAwcChgZRGtIhITqzqSoX7\nKDFhIrDHGLNzoILOYGWwv8+9wHlACfAt4FcicuIg64wbqfAsp4IMSccYk7YvoAaYH+C4C6gLcPx+\n4D2/Y48ATwLLgfujvT+wEPgn8EugAbjbObbaOS/OuZ1AM7AemBak3krgv4F3nLIvACOcc5MAA1wD\n1AKrnOPnAx8BTc71U4PImQUsBj4D9gB/9tbtnD8ZeNOp50unDd8G9gMdgBv4S4B684H7gG3O6z4g\n3/f7wI6SdwLbgatCfK5jgRXO57gZ+Hefc0scmR8DWp02H+ucexzoBjyOnP810Gfjd99Vzme717n+\n6wPJ7rR7qfNd1AMPAYUB6p7vyNXt1L3M57vM8fnef4x9jjzA4c7nv8Vp6+fAZcBUoA3ocupqCvN5\nXQHcFOScC5/fjN93G/Qz9/m+ngV2OTJ+1+fc8UCV89lvx/4G83zOG+A7wKfA50FkuwL4Avu83hZA\ntiec/wuAJ5xyTcC7wGjn3AjgUeyz2Qg87/ds3gLswD5DgT6L/4v9ze4F/gCMBv7mfB6vAaV+v0/f\n7/Qu5zttBV4BRvrU/f+c+zZjn7+jfc4tAx4AXnSufRs4LMSzOhJY6bS9AXgDyIq4X42mM06VF5Er\nhXnYH+WBzvsDgBbgHOAiYLffA7sYWBnO/bE/3k7gBiAHKKSvUjgLWAMMxyqIqcCYIPVWAluBacCB\n2B+c98H3PnSPOecKgcnOw3EGkAv8F7YzzQsg543AW8B4bIf2W+BPzrkJzsN3qVPPQcAMnwf07hDt\nv9Op92BgFFax3OXzfXQ6ZXKdz3sfzg8pQPv/AfwG+yOfge1sTvfpBNqcOrKxyvOtYM/EQJ9NgHsb\n4HC/Zymo7FjltwLb6QwD/gL8d5C6XfTtbLzfpW8HUgscjX2GSrDP55HO+TE4nQY+z1aYv5VCbKd8\ndpiy+X63QT9z7CBjDfBDIA84FKvEznLOz8bO1HOc9m4AbvT7vF91Pr9AyvQobKd3CvZ5/YXzfQRS\nCv+f8/kf4Mg5Gyh2zr0IPA2UOt/jqX7f70+d+guDfBZvYRXBOOzgYC0w07nmdeCOEN/pZ9jnsNB5\nf49P3VdjnxvvoKra59wybOd+vPP5PQk8FeJZ/W/soCTXef0LIBH3q5FekEovIlcKU5wPcpzz/nJs\nh5PjfClNwNeiuT/2R1rrd34hvUphHrAJ+wMJqb0DPDhHYUfp2T4P3aE+528H/uzzPgurVFwB5NyA\n08E678dgZwE5wPeB54LItIzQSuEz4Byfc2cBNT7fh8f7Q3GO7QTmBLjPIdgR8DC/h32Z8/8S4DW/\nz8YT7JkY6LMJcP9ASiGg7Fjlvhdn9Oacm0vwEW+f55LAHcidPucPxD6TF+HXYRK5Uvgj8BJBOokA\nsvl+t0E/c+AE+j/33wceDXKfG32fMaf980LI/UP6doQHYn8LgZTC1djBSLlfHWOwg8F+gxCn3R1A\nwQCfxWU+758FHvR5fwO9M49A3+kPfMpeB7wUpK3DnWtLfH5zD/ucPwfYGOJZvRO7qnB4oPrDfQ1l\nm0IgxmE/yCbn/bewHUanMaYdu4T0rUHU/2WwE8aY17FT5weAehH5nYgUh1nXF1jNPzLI+bFOGe+9\nup3z4wLUOxF4TkSaRKQJqyS6sKOgQ7CdezT0kcH5f6zP+z3GmE6f9/uAoiD1NBhjWv3q8m3LDr96\nCkKsBUfy2QQjmOyjsKPSNT6f50vO8Wjp+V6NMXuxywKLgO0i8qKITIm0QhH5GXbW+W/G6T2iINhn\nPhEY622/8xncin2eEJHJIrLS8bZqAX5C3+cYQvxusN+f/2eyJ0jZx4GXgadEZJuI3CsiudjnusEY\n0xjkul3GmLYQMoBdGvTiCfA+0LPsxf+zKwIQkWwRuUdEPnM+mxqnzMiBrg3Cz7Cz4Fcc55nFIcoG\nJdOUwteAtcaYvSIyHjt6v9x5YHcAFwPniIj/QxsuIX9wxphfG2NmY5cHJmPXKYNxiM//E7Cj+d1B\n7rUN++MErJHSuX5rgHq/BL5qjBnu8yowxmx1zh0WTPwQsvaTwZF52wDXBKtnhIgM86srUFsC4S9n\nJJ9NpOzGdghH+3yWJcY6OERLH/mNMS8bY87AjnY3Ar8PVC4YIvIj4KvAmcaYlkHIFYwvsTMj3+dp\nmDHmHOf8g1i5jzDGFGMVhvjVEaot2/H5LYjIAdhlzX4YY/YbY35kjDkKOBE4F7jSkXGEiAwPco9o\nFeVg+SZwAdbeVIKdZUD/zycsjDGtxpibjDGHYh0M/jOAY82ADAWlkCsiBT6vPiNGx4tjnIjcAVyL\nfSjBGq82AUdi161nYDvqOuyaekwRkeNE5ARn5LKXXkNhMC4XkaOcH8GdwDMmuJvbn4F/FZHTnfpv\nAtqxU2l/HgJ+LCITHblGicgFzrkngfki8m8ikiMiB4nIDOdcPXa9OBh/An7g1DcSO+1/IkT5gBhj\nvnTk/m/n+yzHGtWfDLMKfzkj+WwCXR9K1m5sJ/1LETkYwHnWzgpT1pCIyGgROV9EDnRkdtP7zNQD\n40UkL8T138d2PGcYY4KNrgfLO0CLiNwiIoXO6HeajyffMKxdxO3Mcv5PhPU/A5wrIic7bb2TIP2W\niJwmItPFxgi0YAdSXcaY7Vij8G9EpFREckXklMibGnOGYb/XPdgZ508ivL7Psyoi54rI4c7ApwX7\nrETsGjsUlMJfsaM172uJc3ysiLixP6R3genYdWRvMNm3gN8YY3b4vrCd5regJzDmbzGSsxjbgTTS\n60mxNET5x7FrijuwBtfvBitojPkEax/5H+zo9TzgPGNMR4Div8IaRl8RkVasAe0Ep55a7LrlTVgD\nVzVwjHPdH4CjnCWC5wPUezfwHtZD4wOsIe7uAOXC4VLsqGkb8BzWiPdqmNf+N1Y5NYnIzRF+NmCf\nnz861/9bGPe7BTtlf8tZAngNO9CIBVnY72Ib9vs4FbsmDda4+RGwQ0R2B76cn2BnWZ9Kb8zOrUHK\nRoUzUDkPO6j6HPsZP4wd+QLcjFVMrdjn/+kI6/8I6530v9hZQyN24BaIMqwSacEui/6D3oHJFVgl\nsRFrE7oxEjnixGPYvmAr8DH2txgJS+j7rB6Bff7cWI+v35goYk0k+iVGJV6ISCXWeBaTSE0RqQUu\nN8asikV9iqIMXYbCTEEJgYiMwho+a5IsiqIoaYAqhSGMs677KfA/ztKQoihKSNJRKZhIXlVVVRGV\nT4WXMeZUY8zvB1vPm2+++Y7jDfN/k92mZLzS8bvXtmvb49j2sEhHpRAR7e3tyRYhaWRy2yGz269t\nz0xi0fYhrxQURVGU8FGloCiKovSgSkFRFEXpIW65w8VuKPIYNqCkG/idMeZXAcq5sNkBc4HdxphT\n4yWToiipRVdXFw0NDezfvz9mdRYXF7NtWzQZVtKf4uJidu3axYgRI8jOjm7zt3huKNGJzd2+1slj\ns0ZEXjXG9Oxw5uQi+Q02nW+tN1WAoiiZQUNDAwUFBYwcORKbnWHwdHR0MHbs2IELDkHa29spKCig\noaGBUaOiy8sYt+UjY8x2Y8xa5/9WbNi5f2bKbwLLvT70JowdqRRFGTrs37+foqKimCmETEdEKCoq\nGtTMKyFpLkRkEnanoGm+mRpFxLtsdDQ2OdSvjDGPhaqrqqrKROJ25Xa7KSoaTNLK9CWT2w6Z3f50\naXtxcTEjRoyIaZ0dHR3k5QXNEzik8ba9oaGBlpa+SXFdLld4mncwmzGE88Lm/14DLAhw7n5sEqgD\nsTnEPwUmD1BnRFRUVER6iTHGmJYWY376U2NaW6O6PCWItu1DhUxuf7q0fevWrTGv8/PPP495nemC\nt+1BPtfkb7LjpCp+FnjSGLM8QJE67C5Ee40xu7GziWMClEs4FRXwxhv2r6IoQ5sf//jHHH300ZSX\nlzNjxgzefvvtoGWXLVs2pA3Z8fQ+Emy65Q3GmF8EKfYCcL+zB0IeNoXzL+MlU7i0tsKLL8LkybBy\nJZx2GqTBTFxRlCioqqpi5cqV/G3V39jSuoUvt3/Jl3zJJPckRheN7ld+2bJlTJs2bcgas+M5UzgJ\nm8N8nohUO69zRGSRiCwCMMZswG5fuB67WcfDxpgP4yhTWFRUQEcHDBtm/+psQVFSh9ZWuPdecLtj\nU9/27dsZNnwYa3auwbPfw/iy8Rw44kAef+lx5p48l9mzZ3PWWWexfft2nnnmGd577z0uu+wyZsyY\ngcfj4e9//zszZ85k+vTpXH311T2pJhYvXsxRRx1FeXk5N998MwB/+ctfOOGEE5g5cybz58+nvr4+\nlGhJIZ7eR6uNMWKMKTfGzHBefzXGPGSMecin3M+MMUcZY6YZY+6Llzzh4p0llJXZ92VldrYQqwdQ\nUZTBEeul3TPPPJMtX2zh+nOu56E7H2Ld2+vIJZdHfvIIt/zqFtasWcPVV1/NbbfdxsUXX8yxxx7L\nk08+SXV1NSLCwoULefrpp/nggw/o7OzkwQcfpKGhgeeee46PPvqI9evX84Mf/ACAk08+mbfeeov3\n33+fb3zjG9x7772xaUQM0YhmP7yzhPx8+z4/X2cLipIq+C/txmKwVlRUxL3/715u/snNDB8xnDtv\nuJO//O9f+OLTL/iPy/+DGTNmcPfdd1NX13/Dt08++YSvfOUrTJ48GYBvfetbrFq1iuLiYgoKCrj2\n2mtZvnw5BxxwAAB1dXWcddZZTJ8+nZ/97Gd89NFHg29AjFGl4Me6dWAM1NT0voyB6uokC6YoStyW\ndkccOIIpx07hqv+4iu/96HusemkVEw6fwKN/e5Tq6mo++OADXnnllX7XmSAu/Tk5ObzzzjtcdNFF\nrFy5krPPPhuAG264geuvv54PPviA3/72t7S1tcWmATEknhHNacnttydbAkVRAhFsaXewjiCffPIJ\n+Z58GvMbAfj0408Zd+g41ryxhr2f7YVJNshu06ZNHH300QwbNozW1lYApkyZQk1NDZs3b+bwww/n\n8ccf59RTT8XtdrNv3z7OOeccTjrpJA499FAAmpubGTfOxvD+8Y9/jF7oOKIzBUVR0oJ4Le263W5u\nuu4mbvnaLVx/3vV89sln/PtN/84TTz3BT3/0U4455hhmzJjBm2++CcDChQtZtGgRM2bMwBjDo48+\nyiWXXML06dPJyspi0aJFtLa2cu6551JeXs7JJ5/ML35hHTCXLFnCJZdcwuzZsxk5cuTgBI8TOlNQ\nFCUt8F3a9aW6Gs47L/p6Z8+e3dPh+3P6qtP7Hbvooou46KKLesucfjrvv/9+nzJjxozhnXfe6Xft\nBRdcwAUXXBC9sAlAlYKiKGmBLu0mBl0+UhRFUXrQmYKiKBlHvbueDbs30NzWTElBCVNHTg0YvZyJ\nqFJQFCWjqHfX8+aXb1KQU0BxfjGe/R7e/PJNjjzoSHbu25nxikKXjxRFySg27N5AQU4BhbmFiAiF\nuYV0m27+UfsPPPs9fRRFvTv10lDEG1UKiqJkFM1tzRTkFPQ51tLRQnd3dx9FUZBTwIbdG5IkZfJQ\npaAoypCh3l1Pa0crL2x8gcqayoAj/ZKCEto6bSTxjd+4kXf+8Q77OvZxQK5NRfHMI8/wy9t/SUFO\nAc1tzVHJ8cMf/pDXXnst4usqKys599xzo7pnrFCloCjKkMBrK+g23SGXgKaOnEpbZxue/R7mnTeP\nV1e8SpZkUZxfDMDrf3md0887nbbONkoKSoLezxhDd3d3wHN33nkn8+fPj13jgtDZ2RnzOlUpKIqS\nHixeDAsX9n8tXgz02gqyJCvkEtDootGceMiJFOYWMnPeTN6tfJe5ZXPJkixqPq9hT/0eDp9xOE/9\n9iluvOhGysvLueOOOwCoqanhyCOP5Morr2TatGl8+eWXLFy4kGnTpjF9+nR++Uu7HczChQt55pln\nAHj33Xc58cQTOeaYYzj++ONpbW2lra2Nq666iunTpzNz5kwqAoRlNzQ0cOGFF1JeXs6cOXNYv349\nYKOir7jiCk466SSuuOKKmH/M6n2kKEp6sGMHTJrU/7gT4tzc1mxH+z571gdbAhpdNNp6Fk2Cp+Y8\nxRdrv+DE00/k9mW3c+JXT+Sjtz6ie08377/3PsYYzj//fFatWsWECRP49NNP+eMf/8icOXNYs2YN\nW7du5cMP7TYwTU1Nfe7T0dHB17/+dZ5++mmOO+44WlpaKCws5Fe/+hUAH3zwARs3buTMM89k06ZN\nfa694447mDlzJs8//zyvv/46V155JdVOZs6PP/6Y1atXU1hYGN1nGQKdKSiKMiTwtRV4GWgJCODS\nSy/lqaeeYnTRaN55+R1u/87t1K+v542KN5g5cyazZs1i48aNfPrppwBMnDiROXPmAHDooYeyZcsW\nbrjhBl566SWKi4v71P3JJ58wZswYjjvuOACKi4vJyclh9erVPaP8KVOmMHHixH5KwbfMvHnz2LNn\nD83NVsGdf/75cVEIoEpBUZQhgtdW0G26Mcbg2e+hrbONqSOnhrzuwgsv5O9//ztr167F4/Ewa9Ys\njDF8//vfp7q6murqajZv3sw111wDwIEHHthzbWlpKevWrcPlcvHQQw9x7bXX9qnbGIPdmZh+xwci\nUBlvXb4yxJq4KQUROUREKkRkg4h8JCLfC1H2OBHpEpGL4yWPoihDG6+tIEuyaGlvoTC3kBMPOXHA\nALSioiJcLhdXX301l156KQBnnXUWjzzyCG5nF5+tW7eyc+fOftfu3r2b7u5uLrroIu6++27Wrl3b\n5/yUKVPYtm0b7777LgCtra10dnZyyimn8OSTTwKwadMmamtrOfLII/tc61umsrKSkSNH9puJxIN4\n2hQ6gZuMMWtFZBiwRkReNcZ87FtIRLKBnwIvx1EWRVEygNFFo/HkebhgUmSZSC+99FIWLFjAU089\nBdgtOjds2MDcuXMBqzieeOIJsrOz+1y3detWrrrqKjo6O2jrbOObN36TyppKPPs9AOTl5fH0009z\nww034PF4KCws5LXXXuO6665j0aJFTJ8+nZycHJYtW0a+Nye4w5IlS7jqqqsoLy/ngAMOSNj+CxLO\nNCYmNxJ5AbjfGPOq3/Ebsaah44CVxphnBqgqIoErKytxuVyRXDJkyOS2Q2a3P13avm3bNsaOHRte\n4cWLrbHZn7IyuOeenrc1NTVMCmSQjhO+aTMKcgpo62yjrbMtrFlKrPG2Pcjn2n8dKwAJUQoiMglY\nBUwzxrT4HB8H/C8wD/gDYSiFqqoq097eHva93W43RYPZlimNyeS2Q2a3P13aXlxczIgRI2JaZ0dH\nB3l5eTGtMxStHa10m26ypHc13vt+WN6whMkBvW1vaGigpaWlzzmXyxWWUoi7S6qIFAHPAjf6KgSH\n+4BbjDFdgYwxgfBO58IlXUZM8SCT2w6Z3f50aXtEM4UwSfRM4YWNL1CcX9zHoGyMoaW9JeJlrMHi\nbXteXh6zZs2Kqo64KgURycUqhCeNMcsDFDkWeMr5MEcC54hIpzHm+XjKpSiKEitKCkrw7PdQmNvr\nIhqOK2yqEjelILan/wOwwRjzi0BljDFf8Sm/DLt8pApBUZSI9jzwLXto1qHUu+sTtp4/deRU3vzS\nbufpa1OYNSa6kXqyiWecwknAFcA8Eal2XueIyCIRWRTH+yqKkuZ4jbfhpLL2L9ttuhOa9to3bUYk\nrrCpStxmCsaY1YRp7XbKL4yXLIqipBe+ex4APX837N7Qr7P1L5slWT05jxLVMfekzRgCaESzoigp\nR6A9D4LlMYqkrD8ul4uXX+4bInXfffdx9dVXc/HFkcfSXnvttXz88cchyzz00EM89thjEdedKFQp\nDILWVrj3XnCCHhVFiRGR5DGKNucR9OY98uWpp57iqquu6sly6stAqaoffvhhjjrqqJBlFi1axJVX\nXjmgbMlClQLRd+4VFfDGG/avoiixw3fPA988RqbbsKRyCVe/cDVLKpewfsf6fmW7TXdYOY8ALr74\nYlauXIk39qmmpoZt27Yxfvx4pk2bBsCyZcs4//zzmTdvHqeffjrd3d1cd911TJkyhTPOOINzzjmn\nR4G4XC7ee+89wEZB33bbbRxzzDHMmTOH+npr41iyZAlLly4FYPPmzcyfP59jjjmGWbNm8dlnn+F2\nuzn99NOZNWsW06dP54UXXoj55xsKVQpE17m3tsKLL8LkybBypc4WFCWWBDLeDs8fzqPrHqXR08j4\n4vE0ehpZWrWUend9n7JZkhW2ofeggw7i+OOP56WXXgLsLOHrX/96vyR2a9eu5ZlnnuEf//gHy5cv\np6amho8//pjHH3+cqqqqgHXv3buXOXPmsG7dOk455RR+//vf9ytz2WWX8Z3vfId169bx5ptvMmbM\nGAoKCnjuuedYu3YtFRUV3HTTTWEl0IsVGb+fgn/nftppEE4gaEUFdHTAsGHQ0GDfn3de/OVVlEzB\n33i7pHIJpQWllBaWAvT8Xb5xOUtcS3rK1tTUhG30rXfXM2P+DH7+u59TckwJT/zvEzy2rP96/xln\nnNETeb169WouueQSsrKyKCsr47TTTgtYd15eXs/WmrNnz+bVV/tk+KG1tZWtW7fyta99DYCCAmsX\n2b9/P7feeiurVq0iKyuLrVu3Ul9fT1lZWVhtGiwZP1Pw7dw7OsKbLXgVifc7KivT2YKixJva5tp+\ndoKSghJqm2ujqs/ryjr7tNl88PYHfFD9AY2tjYybPK5fWd9U1eGO2nNzc3tmHNnZ2f3sEcHqefLJ\nJ9m1axdr1qyhurqa0aNH09bWFrBsPMhopRBt5+5VJN6khvn54SsURVGiY0LJhH4eRc1tzUwomRBV\nfe9ue5fd+3azs3MnR8w6gl9+/5e4znX1277Tn5NPPplnn32W7u5u6uvrqaysjOr+xcXFjB8/nuef\nt/G67e3t7Nu3j+bmZg4++GByc3OpqKjgiy++iKr+aMlopRBt575uHRhjdwH0vowBZ6c8RVHiwIIp\nC2hsa6TR00i36abR00hjWyMLpiyIuK56dz1bGrcAkJ+dz7FnHsvnGz/n5HNOHtCV9aKLLmL8+PEc\nddRRXH755cyaNYuSkuhSWjz++OP8+te/pry8nBNPPJEdO3Zw2WWX8d577zF9+nQee+wxpkyZElXd\n0ZKw1NkxJGaps++6Cz77rP/xww6D22+PRrTUIl2SosWLTG5/urQ90oR463esZ/nG5dQ21zKhZAKn\nTjgVyZI+qTA8uz0DJsSrrKnk0z12e83c7FwA9nfZzZ2POOgIXJNcIa/3ZqHds2cPxx9/PP/85z8T\ntuYfilikzs5oQ/NQ6PgVJZMoLyunvKwc6LuPgW8qjGn50/pd559HaUfrDsqKyqhrrYMuyMmyXaGn\n0xOWK+u5555LU1MTHR0d3H777SmhEGJFRimF1lZ48EG47rrwPIwURUldgqXC8A9kC6Q8Gtsbyc3O\n5ZDiQ9i9bzdtnW1kSzaHlh4aludStHaEdCCjbAoabKYoqUe0S9jB0lt0dXf1OearPESEwtxCDj7g\nYHbu3Um2ZDOxZCITSyYy8oCRHDf2uKjbkSoM1iSQMUpBg80UJfXIzc3F7XZH1ZEFS2+RndV3H+VA\nymNE4QhKC0p7At66TBe52bm8VfcWlTWVCcuwGmuMMbjdbnJzc6OuI2OWjzTYTFFSjxEjRtDQ0EBr\na2vE147sGslHez6iPbud3Kxc9nfvp72rnbbcNrZt29ZTLndfLs2tzeRl927R2dHVQXF2MZPzJtPQ\n1cBHuz4iPzuf3KxcmlqbqNxRydGjjmZEYWy3Co03jY2NFBUVDWqL04xQCsHiEcKNXlYUJT5kZ2cz\natSoqK4dy1i6D+zu4420YMoCGjY2MHZ2r+fN7qzdLK1aSmlBKSUFJTS3NdPY1sjNc29mbNlYflf5\nOxo9jT0R0gCNnkY2tG1giWvJYJuYUDZt2hT15+klI5RCqHiEZMwW1OCtKOHj74a6YMqCHg8kX28k\nL5UbK/u8Ly8r5+a5N/ep45qZ1/RcV9tcy/ji8X2uGUykdLqTEUrBN9jMl+rq5CgFr8F76lRdwlKU\nUKzfsb5nlO+bBO/muTf3UwahCKQ8vEwomcCm3ZvY5t7W47I6tmgsk0dOjlUz0op47tF8CPAYUAZ0\nA78zxvzKr8xlwC3OWzfwf4wx62ItSyrFI0SbgE9RMpHlG5cHTYIXiVIIxbRR03hs3WMU5xVTnF9M\nk6eJ2uZaFkyNPFJ6KBBP76NO4CZjzFRgDvAdEfHffeJz4FRjTDlwF/C7OMqTEkSTgE9RMpVYJ8EL\nxIe7PmTu+LkMLxxOa0crwwuHM3f8XD7c9WHM7pFOxHOP5u3Aduf/VhHZAIwDPvYp86bPJW8BfRf2\nhhhq8FaUyJhQMqGfEXgwSfACUdtcy2EjDuOIg47oOdZtujPWppCQOAURmQTMBN4OUewa4G+JkCdZ\naHZVRbFDMagdAAAgAElEQVR2Av/d04IRyyR4wYh19tV0J+4J8USkCPgH8GNjzPIgZU4DfgOcbIzZ\nE6q+qqoq4906Lxy8iatSge3bIZDo+fkwZkzs75dKbU8Gmdz+VG27p9NDvbue7KxssiWbLtNFV3cX\no4tGU5hTGPSaprYmOro6yMvOY3jB8KBlIfK2RyNTqhKq7S6XK6yEeHFVCiKSC6wEXjbG/CJImXLg\nOeCrxphNYVQbsyypQ51MbjtkdvtTte1LKpcEjAkoLSyNOibA32X1hM4T+Or8rw6qDl+313RigO89\nuVlSxW459AdgQwiFMAFYDlwRpkJIGOkUS5BOsiqZTaxjAgK5rNa317N+x/qYuaxmGvG0KZwEXAHM\nE5Fq53WOiCwSkUVOmR8CBwG/cc6/F0d5IiIRyfNaW+Heewefh0kT/SnpQqzX731dVrMki9LCUrKz\nslm+MeBKtRIG8fQ+Ws0A0xVjzLXAtfGSIVoSFUsQiyA2jXtQArJ1Kyxc2P94WRncc0/CxfGyYMoC\nllYtBeiTcuKamddEVV+gmUe2ZGes51AsyJgsqZEQy1iCYLOBWGVt1bgHJSD798OkSf1fO3YkVSxv\nyonSwlLqWuooLSyNODrZl0Azjy7TlbGeQ7EgI9JcREKsYwmCzQZikbVV4x6UdCTS9ftQRuBAM4+u\n7q6YuqxmGjpT8COWsQTBZgPBOvNIZwsa96AMdbyG5EZPY5/cR97YhkAzj9FFo9VoPAh0puBHLJPn\nBZsNxCpra6ol+lOUWBNO7iP/mcdQ3iozEahS8CNWyfNCLe1E05kHcjtNpUR/ihIPNK114lGlECdC\nzQai6cw13bYSEbm5/Ucd0DtKiRWLFwc2XsfIyykRuY+UvqhSiBPr1lkl8Je/wJQp9jcK0S3tqNup\nEjHjxsGyZfG/z44d1qvJn0AKKQpi7cKqDIwqhThx++2wYgX8/vfwjW8MbnSv+0srmcpAu6YpsUeV\nQpyI1ehe3U6VTEdTUCQWdUmNE7EKKlO3U0VREokqhSgYKGdRrOIQoK+nkvdljLVNKIqixBpdPoqC\ngTyBYhWHAOp2qqQ4ZWWJ8XKKM0MldXYsUKUQIeHYCjSoTMkYkphcL1YESr+9tGrpoHIypTOqFCIk\nHE8gHd0rSvREMmoPVDZSvFHTHV0drPpiFc1tzeRl5/Hgew/y4LkPDrY5aYfaFCIgUltBrPZLUJSY\nsXixTant/1q8OKlieRko11E4ZT2dnojuWdtcS1tnG29++Sae/R6K84sxxvDqlldD7h89VFGlEAGR\negLp5jdKyuENNkuxlNpeAm2aU1pQGnDTnGBlm9qaIrrnhJIJVO+opiCngMLcQkQEEeGgwoMycrMe\nVQoREIknUKz2S1CUTKK2uZaSgpI+x4LlOgpWtqOrI6J7LpiygD2ePRgMxhg8+z20dbUxo2xGRuZY\nUptCBERiK/C3Pbz4Inzxhe6jrCihiCTXUbCyedl5Ed2zvKyc+YfO5/3t79PS3kJJQQkzx8wkPzuf\nMYVjom9MmhK3mYKIHCIiFSKyQUQ+EpHvBSgjIvJrEdksIutFZFa85Ik3ra1w111w992wfXt/28Mj\nj1hFoUtJihKcBVMW0NjWSKOnkW7TTaOnkca2xoAG5GBlhxcMj/i+1x17HUeOPJJTJp7CKRNPIT87\nP+h9hzrxXD7qBG4yxkwF5gDfEZGj/Mp8FTjCeX0bSFtTf0WFzXX0wgvwwAN9bQ9gl5ry83UpSVFC\nEWq7zvU71rOkcglXv3A1SyqXAAQsW5hTGNP7ZhpxWz4yxmwHtjv/t4rIBmAc8LFPsQuAx4wxBnhL\nRIaLyBjn2rShtRWeew7a2+37v/7VBrZ54xRqa6GrC1pa7NLR9dfD/ffrMpKSBBIQbDbYQLBAuY5C\nxRIscS3pU7ZyY2VUcmuOJYvY/jjONxGZBKwCphljWnyOrwTuMcasdt7/HbjFGPNesLqqqqpMu7f3\nDQO3201RnHvf5maor7cdvzGQkwOjR0NJCXR3Q10dZGdDVhZ4PLB3L4wfD6W9S6F0d8OuXTBqlC0X\nCxLR9lQmk9ufrLZ7Oj3Uu+vJzsomW7LpMl10dXcxumh0VCN4L9vd2+ns7iQnq3cc630/pqjvur9+\n74Hb7nK5JJw64m5oFpEi4FngRl+F4D0d4JKQWmru3LkR3b+yshKXyxXRNZHQ2grf/S68/37vyN/t\nhlmz4Ne/htdft0tLEyfamcQrr9i/48bBs8/2XrNiBTzxBHz727GLeo5321OdTG5/stq+pHIJjfl9\njb+NnkZKc0r7jegj4eoXrmZ88XiypHfE1G26qWup4xHXI33KRtP2oZLmIhbfe1xdUkUkF6sQnjTG\nBHL4rQMO8Xk/HtgWT5liTUWFXR4SsTOEHEfNbtlil4nefbfXjfXtt+2sIicHdu7sNTqr+6oyVIjE\npTQSJpRMoLmtuc+xWO3AFknAXCYQT+8jAf4AbDDG/CJIsRXAlY4X0hygOd3sCevWWZfT1la7TFRX\nZzv12lqoqoLjj7cbYP3P/8ARR8All8CFF8LZZ/cqgFil2VaUZOPbede766msqeSZj59hS+OWQXWy\nkXglRUokAXOZQDxnCicBVwDzRKTaeZ0jIotEZJFT5q/AFmAz8HvgujjKExduv90uHW3e3Pt6/304\n4ww499z+Hb9/NPSLL4afOkPTZihB8U9fUVOTlPQV3s770z2f8s/af9LkaSInK4exRWMHNfqOp3dQ\nvGY36Uo8vY9WE9hm4FvGAN+JlwzJIlDSvGCZU194wSqIcNJsD5SyG6ziePBBGySnZBD+eyXn59v3\nMdorOVy8nfd3X/ounaaTgw84mCkjp1BWVEajp5HlG5dH3ZEPxjsolM0gkoC5TEAjmgeBbwfsNRgH\nS5r3858HdkG96y747LOB02yHu72nr+IYNiwmzVSUiCgvK+fQ0kM5ZeIpfQzDyRp9D5Qae8GUBSyt\nWtojY3NbM41tjVwz85qEy5oKqFIYBIFG7pFusBNu6oxwUnb7K46vfz36tinKYEil0bevzQDo+eud\ntXhnN74ziWtmXpOW3kexQJVClAQbucdigx3/GUiw2Yf/bMFfcbS2xqSpihIxqTT6rm2uZXzx+D7H\n/GctGrjWiyqFKAk2cvcd+QdaXgq3bt8ZSDizj0CKo7nZGqUzNI5HSSLhjr4TER+QSrOWdEBTZ0dB\nuJvtRLOfQqCYhXBSdgdSHN3d6t6aMZSV2WjIJ56wrz177N81a5K2gU55WTlLXEt45IJHWOJaEjR1\nRbzjA+LpzjoU0ZlCFEQych/IMBysbt8ZSDh2h0DLVpMm6b7QGcM99/T1QDroILj8cvt/gj2QwmWg\ntf5YEWjW8i8T/oXlG5dz39v3pXUEczxQpRAF4dgNwjEM+xOu7SAQgRRHZaV1VVeUVCSctf5Y4Wsz\nGMgbKdNRpRAFA43c/Tv30lK7z8Jxx4VORhmp55KipDPJWutP1AwlXVGbQhzw79y3b4dt2+w+C6GI\nZLtPRUl3krXWrxHModGZQhzw7dz377e2vuJi+Nvf4JZbgi8FRbLdp6IkjcWLrf3Cn7Iya9twGMiz\nKJiHEthsq/HySFJvpNCEVAoikg1ci81e+pIx5p8+535gjLk7zvKlJb6d+4oVkJdnU2d/8YUuBSlx\nxHcDHd8UFzHcQAfon1LDi4+RLdx1e//4gESs96dSDEUqMtBM4bfAAcA7wK9F5B/GmP90zi0AVCmE\nwLsjW00NjBkTmeFYUSLGZ5ROZaVNz5skol23T8R6v0Ywh2YgpXC8MaYcQETuB34jIsuBSxkg2Z3S\nu9fC1q3w+edw5JFqOM5owlx2GfQ1sbh2kETrWZQojySNYA7OQEohz/uPMaYT+LaI/BB4HdCx7gC8\n8w58+SXk5lq7QlaW/V9jBzKUMJZdYnJNLK4dJNGu2+t6f/IZSCm8JyJnG2Ne8h4wxtwpItuAB+Mr\nWvpz/PF2luC1JyxYYJWBd1+ESNNfKAqvvWbD3N3uvkEogxn9e2cUa9bYTcS9FBbC7NlR1R3tur2u\n9yefkC6pxpjLfRWCz/GHjTG58RMr/QmVCqOiwr6uv143zFEixO2G4cPtaGLSpN5XoGWicPHOKLKz\nYfz43ld2duC6vQZt/5ePQTvaTXHiuZmOEh4DuqSKyEHAN4EpzqENwJ+MMXviKVi6E2qntddft+9f\nfx3++lf4t39LrqyKEhFhzhqiXbf3XuM1BHu3xVTFkBgGckmdirUfvAy8jzUuHwfcKiLzjDEbQ1z7\nCHAusNMYMy3A+RLgCWCCI8dSY8yj0TYk1Qi101pWlrU1FBfDH/4A55yjy0iK4iWQW+qtr9/KIcWH\n0N7VTmt7K583fU5zezPjho3j+uOu5+KjL0622EOGgWYKdwHfM8b82fegiFwE/Bi4KMS1y4D7gceC\nnP8O8LEx5jwRGQV8IiJPGmM6wpI8xQkUiNbaCjffbBNYdnXZVYDPP+87W4g23baSBvjGEfgfD/ca\n73pjOA+H/7Vem0FhYV97xJo1gQ3SScLfLbW9q53PGj5j195dHFJ8CK9ueZUsyeIrw79Ck6eJ/3rt\nvwBUMcSIgZTCdGNMv0/aGPOsiPwk1IXGmFUiMilUEWCYiAjWk6kB6BxAnrSmogL27oXNm6GpySbL\nKyjoO1sIZx9mJU2JxhDsf83CheF34OFeu3p15HLFEX+31I27NzIsfxgdXR28t/09CnIKyMnKoaGt\ngUnDJwFw/7v39ygFT6cnrhHRQx0xxgQ/KbLWGDMr0nM+ZSYBK4MsHw0DVmBtFcOArxtjXhxI4Kqq\nKtPe3j5QsR7cbjdFSR5yd3fDrl12duB2Q1ub/T8nx9ryRGDsWKsk6urs8lJ3t7X1ZQ0iO1UqtD2Z\nDMn2b9wInQHGTjk5MGVKz9uAba+p6TVy+dLYCAccAPv22QfPS1aWPZ6bC+PGxUR8T6eHprYmOro6\nyMvOY3jBcApzCvuU2e7eTmd3JzlZdsza5GlCRMiSLDz7PXbfZwFjDAU5BRgM+7v2M+3gaXg6Pexz\n72Nf1j6yJZsu00VXdxeji0b3u89QJNQz73K5wootG2imcLCI/GeA4wKMCucGITgLqAbmAYcBr4rI\nG8aYllAXzZ07N6KbVFZW4nK5opUxJqxYYfc7+fa34eOP4c9/tr+z/fthxgzYsgXOPtu6sFZU9Hdh\njZZUaHsyGVLt97qNrl7dd+moqAjmz7cd/qJFPYcDtt13puB1bQX79+ST7f9xDGzztRWUHOi4m7Y2\n9vMu6imXW0pJQQkvb3+ZlvYWXJNc/O3Lv+HZ7yEnK4fc7FwmDZ9Ek6eJ4YXDud51PUsql3BY1mF8\nVvxZT32NnkZKc0pZ4loSkzbEe6e4wRCLZ36gcejvsaN4/1cR8PCg7gxXAcuNZTPwOb0eTkMG/812\npk+Hk06CSy6xfydPtikwpk8Pbzc3JUPxuo0WFVljlPcV7QPidW31d28djGvrAPjaCrIki9LCUkoL\nSnu8i7z4u6XOHDOTw0ccTl52HseOOZa2zjb27d/HiIIRNHmaaOlo4frjrgfs0lO2ZPepL1YR0Yna\nKS7ZhJwpGGN+FOyciNw4yHvXAqcDb4jIaOBIYMsg60w5fDfbqa+HRx+FmTPtudJSu4Pi/PnwyCMw\napRVEKB7KShDj0hSWARKlLd843JaO1o5YewJfLjrQzY3bmZE4QhuOO6GHnvChJIJdLV09akrVhHR\nmbIPw2BSZ/8ncF+wkyLyJ8AFjBSROuAOIBfAGPMQ1rNpmYh8gF2OusUYs3sQ8qQc/gFsbW3W22iW\nY4nZvt12/KtW2fedndaN9ZNPbJ4kTYmhxBRfbyTfGUaC7C6DSWHhVRLe0frUg6f2RDyv27mO9TvW\nU15WzoIpC1j71loaPY0xj4hO5E5xyWQwSiGk0cIYc+kA57cBZw7i/imPfwBbQ4M1ML/9tv19rllj\nFUFdnc2cetBBMG+etT984xuqDJQY42sriMSLKUbEIoXFQKP18rJythZtpTSnNOYZUDMlL9NglEJw\ntyUF6B/ANmGCfR12GBxzjPU62rDBeh59/rn1RHr0UTj6aE2xrQShqMj6M3txu/ulmEhVYpGyOpzR\nemFOYUyMyv5kSl6mgSKaWwnc+Qsw9P27BkmondTuustu0dncbH/nbrdVENnZ1uawZo2mwFB88C79\nHH54/+PRpNBes8a+Zs/uX18cGWzKat/Rer27ng27N7Bz704OPvDgniWkeJEp+zAMZGgelihBMo0b\nb7TLRiecYJeXWlrsdp1nnGF/+01NA6fA0OjnDCJcN1GvAnC5eqOWV6+2nf38+b3lvDuzJXEjnnDw\ndwGdNmoaKzatYPe+3XxQ/wFZkkVOVg5ji8b27NAWTzJhH4ZBhEYpg8Hf3rB9u41b+OIL2LTJeiJ5\nU2CEquONN+xfRQF6XVfz83vdTL1T0TQjkAvoik0rOH/y+Wxt3Uqn6WR44XBOPOREJo+cHNC9VYmc\nwdgUlEHgb2/Y6KQWXLvWBpIWFdng0hdeCLyE5B//oPYHZagRzKj84a4PObT0UE6ZeIqNbnbosS2U\nJEXcIYMqhSQRKmHeiBF2oNfebj2W3O7+Hb5v/ENDg8YzKEOPUEblTPEESgaqFFKIYHsw+Hb4ra1w\n3312ack/+llnC0OEcPdWDlRu9WqbcfHII0Pf47XX7LW+2VID3SOJhOr4Q3kCNWxsSJbIQwJVCilE\nsD0YfAPYKipsLqXiYuveChr9POQId2/lQOWqq/vbD4qKbFnf63fssArA//oE7N/sS6hcQqE6/lCe\nQJUbKxPahqGGKoUUIpQLK1gX1h//2LqtfvklfPqpjXr2otHPQ5DXXrOeB52d1hPBm+a60PEI9+/U\nvQqgvb23gz/8cJvwLsnBa/4E2kzH60Hk9fIJ5QKaCZ5AyUCVQhrxwANWMZx8st1vYbBZVJU0wO22\nngfFxTZPSkeHfTU0WCXx/PO2nDdbqjdj6qRJKe9uGk4uIe34E48qhTRh2zZYvtwmzNu0ybqhqx0h\nA+nosDsztbZapeC1KXR09Hok+Ae4pSiZkkso3dA4hTThgQdsH3DAATZ/0rZtvXYEJQPp7rbriAUF\n9pWXN7hU2klgQskEmtua+xxTD6LkozOFNKC1FV55xf7uP/nEuqyuWQPHHad2hCGJfzbTDmfb8ry8\n3v/BLiu1tdn/OzpsGLzbHV6qirIym7fd4+l7vLDQejUlwAMpU3IJpRuqFNKAigqYNs3+/tesgfJy\nOzhUm8IQxd8gXF1tZwHQG+UINkjFuwVnUxNceKFVJvfcA5WVA98jXC+nOJEpuYTSDVUKacA779h+\nobXVup/qLCGDKCuz64V1dfb93r12VmCMnTmkOWpITj1UKaQBxx8PL79s924+/nh49VW44ALNoJoR\n3HNP4IA1717N3jTa6m0QlFTfVznVUKWQongzoF5xBTz3nHU737PHxiaEk0FVGaJ4FUSgaOY02Vch\nkQwUC5HqJEOhxU0piMgjwLnATmPMtCBlXNgtPXOB3caYU+MlTyoSKvW1NwNqSwvU1lqbYnc3vP8+\nHHxwbwZVnS1kKCmSiiLVSed9lZOl0OI5U1gG3A88FuikiAwHfgOcbYypFZGD4yhLSuLt+KdO7Wsb\n8GZAnTTJOoiUOqlfmpuhsdG6pYbKoKokgHDzEyVLFpdrYC8iXy8n/+NDhHSOhUiWQoubUjDGrBKR\nSSGKfBNYboypdcrvjJcsqUio1NfexHgtLTY2YeJEqyBeeQVKSux2nZMmBc+gqiSAJHvu9CGQLPn5\ngZWWLxkw20jnbKrJUmhiTPy2WnaUwspAy0ci4l02OhoYBvzKGBNwVuFLVVWVaW9vD1sGt9tNUQr2\nmt5RvzdFdmmp7fC7u62jSXa2dTTp7LTHcnOtgsjKgpwc643oe10gUrXtiSKu7a+p6U1n60t7e+Jz\nCgWQxZ2fT1FLS9LzG4XC0+mhqa2Jjq4O8rLzGF4wnMKcwe/y6/u9ezo91Lvryc7KJluy6TJddHV3\nMbpodEzuFU+2u7fT2d1JTlbv2N37fkzRmIDXhHrmXS6XhHPfZBqac4DZwOnY/Z6rROQtY8ymUBfN\nnTs3optUVlbicrmilTEuBNs34ec/h9dftzOFiRN7y3/xhVUIOQG+rcMOC55ILxXbnkji2v5gCeUG\ns8VltEtS/rK89hqV3/wmrl/8wibKCreeROC0cX1uA0tLPqK0O4+S7jyai3JpnH1UTNbL/b/3dPU+\n6rEp5Jb2Bvftb+Tm44N/RrF45pOpFOqwxuW9wF4RWQUcA4RUCkOBYPsmPP647eBPOaX/KsSRRw6c\nRVVJc2K1JOV22xFEUVHf+mpqkm8Lcdq4nBpKGUkpdrRe2tQEznaase6w0zUWIlnBfclUCi8A94tI\nDpAHnAD8MonyJIxg+yY88ICdReTmBh9shvJYUpQBiZMtJNLReC3NjKe4z7F0MQAnkmQotHi6pP4J\ncAEjRaQOuANrQ8AY85AxZoOIvASsB7qBh40xH8ZLnlQi0Ih/0yYbmFZSYgPVPvvMLg35E8xjSUkw\nqeS54y+L222NUf6jhjVrbK6j6uq+xweZWTUa18kJlNCIp2emAPE1AMd6CSldl6TCIZ7eR5eGUeZn\nwM/iJUM6cdttNpvBAQfY3+2tt8LTT/ct4++xdOyxdslJZw1JINlr8774y7JwofVAmD+/73GPxz4o\n3jxKXrxR0VESjevkAqawlCoASiigOas9bsnwYu3vn+4BcQOhqbNTgE2bbOqKYcPs+2HDemcLvnht\nEcOG2b8PPGBnDZo+W0kmtc21lBT0dYEbaCmonDJuZi6lFFJHC6Xd+XHrVH2VVpZkUVpYSqljv0iF\n+lINTXORAnhnCTk51lV12DD73ne24J0leFcnSkttYNv8+brZzpAhVktSZWV9t+P0UhgfF8yIYgF8\n2lgOlDOp93icRtmx9vdP54C4cFClkAKsWWP/7tplZwBtbVZBvPtubxl/j6Xt262b6u7dNo12RYXa\nGNKeWC1JeVNn+3srLFwImzf3Xy4Kdw+GIES0L0ISlt1iHcCWzgFx4aBKIQXYsqU3dqGoyP5Gf/7z\nviN/X4+l/futIikshPp6OOEEnS0oYeJvZ4DePRiiJNX3RYj1Zj5DfXMgVQopgq+9oKGh/8jf12Np\nxQqbSt83wM27NafOFjIcbxyCy2VnBl7KyuLqMZXKsQCxVlqprgQHiyqFFMDfXlBWFnrkHyzOQTfd\nUXriEPLz+weuRRtpPQSItdJKZSU4WFQppADBIpyDjfw1sjnDSHYUspJRqFJIAXTkr4QklTKyKkMe\nVQopgI78FUVJFTR4TVEURelBZwqKMhhSbb3f62E0aVLf5aUhtJuaEl9UKSjKYEi19X6vIgoUvKYo\nYaBKQVFSnVTKyKoMeVQpKEqqkw5up6m2jObHUE51HWtUKSjKUCcRHXaqLaP5MNRTXccaVQqKMtRJ\n4Q47EUSz30Mmo0pBUQaDrvenPEM91XWsUaWgKIMhBdbLldAM9VTXsSZuwWsi8oiI7BSRkPsui8hx\nItIlIhfHSxZFUTKXBVMW0NjWSKOnkW7TTaOnkca2RhZMWZBs0VKSeEY0LwPODlVARLKBnwIvx1EO\nRVHijXcZzf+VAsto3lTXpYWl1LXUUVpYqkbmEMRt+cgYs0pEJg1Q7AbgWeC4eMmhKEkn2e6aGzfC\n6tWwZw90dvYeLyiwey7EQo4UX0YbyqmuY40YY+JXuVUKK40x0wKcGwf8LzAP+INT7pmB6qyqqjLt\n7e1hy+B2uynK0O3IMrntkELtr6npzYvuS3t7YK8ggK1b7RZ7/uTmwrhxA96yT9u9929stPu8euns\ntJt9h5IjDUmZ7z0JhGq7y+WScOpIpqH5PuAWY0yXSFiyAjB37tyIblJZWYnL5YpMsiFCJrcdUqj9\nCxcGdwldtizwTGL1ajuC998+M8zNcvq03Xv/55+H4cN7CzU1wYUXDrkNeFLme08CsWh7MpXCscBT\njkIYCZwjIp3GmOeTKJOiJJ5AcQTV1Xaz7lQk2cthSlxJmlIwxnzF+7+ILMMuH6lCUJRUJ8OD4YY6\ncVMKIvInwAWMFJE64A4gF8AY81C87qsoiqJETzy9jy6NoOzCeMmhKEkn2qjnbdusHcAXt9su30Sy\nTOO9f1cX1NX1Hi8sTBm3USV10IhmRYk399wTeB1+xw57PBBFReDx9D9eVhZ4PX+g+ytKmKhSUJRE\nEGodPtBM4vDDobnZegcFukZR4oQqBUVJNsFG8gsXJlSMsNEkgEMaVQqKokSGLkcNaeKZ+0hRFEVJ\nM3SmkIG0tsKDD8J111l7ppIkXnvN2hr8l4m8QWDxWKbRwDNlAFQpZCAVFfDGGzB1Kpx3XrKlyRAC\ndfA7dtjj/gZob7l4dNIaeKYMgCqFDKO1FV58ESZPhpUr4bTTdLaQEAJ18MFyIilKElGbQoZRUQEd\nHTBsmP1bUZFsiRRFSSVUKWQQ3lmCd0m6rMzOFlI175qiKIlHlUIG4Z0leFP75+frbEFRlL6oTSGD\nWLcOjOlvU6yuVoNzUkhGEJgGnikDoEohg7j99mRLoPQhGS6gaeR2un7HepZvXE5tcy0TSiawYMoC\n3VIzAahSUBQl6fgrgGmjprFi0wpKC0oZXzyeRk8jS6uWcvPcm1UxxBm1KSiKklTW71jP0qqlNHoa\nexTAXavuoqu7i9LCUrIki9LCUkoLSlm+cXmyxR3yqFJQFCWpLN+4nNKC0j4KYH/3fupa6vqUKyko\noba5NklSZg6qFBRFSSq1zbWUFJT0OTbqwFHs2rerz7HmtmYmlExIpGgZSdyUgog8IiI7ReTDIOcv\nE5H1zutNETkmXrIoipK6TCiZQHNbc59j44rGkZudS6OnkW7TTaOnkca2RhZMWZAkKTOHeM4UlgFn\nhzj/OXCqMaYcuAv4XRxlURQlRVkwZQGNbY19FEBOdg63/8vtlBaWUtdSR2lhqRqZE0Q892heJSKT\nQhNR2CoAAAV2SURBVJx/0+ftW8D4eMmiKErqUl5Wzs1zb+7jfXTNzGsoLyvnYi5OtngZhxhj4le5\nVQorjTHTBih3MzDFGHPtQHVWVVWZ9vb2sGVwu90UZWjGt0xuO2R2+7Xt2nZ/XC6XhFNH0uMUROQ0\n4Brg5HDKz507N6L6KysrcblckQs2BMjktkNmt1/b7kq2GEkhFm1PqlIQkXLgYeCrxpg9yZRFURRF\nSaJLqohMAJYDVxhjNiVLDkVRFKWXuM0URORPgAsYKSJ1wB1ALoAx5iHgh8BBwG9EBKDTGHNsvORR\nFEVRBiae3keXDnD+WmBAw7KiKIqSODSiWVEURelBlYKiKIrSgyoFRVEUpQdVCoqiKEoPSQ9eUxQl\nCIsXw4svgsfT93hhIfzrv6bVLmpK+qBKQVFSlR07IDsbxvulBWtqsucUJQ7o8pGiKIrSgyoFRVEU\npQdVCoqiKEoPqhQURVGUHtTQrCipSlkZdHVBXd8N7CkstOcUJQ6oUlCUVOWee9TtVEk4unykKIqi\n9KBKQVEURelBlYKiKIrSgyoFRVEUpQdVCoqiKEoPqhQURVGUHlQpKIqiKD2oUlAURVF6EGNMsmVQ\nFEVRUgSdKSiKoig9qFJQFEVRelCloCiKovSgSkFRFEXpQZWCoiiK0oMqBUVRFKUHVQqKoihKDxmh\nFETkZyKyUUTWi8hzIjI82TIlChG5REQ+EpFuETk22fIkAhE5W0Q+EZHNIrI42fIkEhF5RER2isiH\nyZYl0YjIISJSISIbnGf+e8mWKVGISIGIvCMi65y2/yjaujJCKQCvAtOMMeXAJuD7SZYnkXwILABW\nJVuQRCAi2cADwFeBo4BLReSo5EqVUJYBZydbiCTRCdxkjJkKzAG+k0HffTswzxhzDDADOFtE5kRT\nUUYoBWPMK8aYTuftW8D4ZMqTSIwxG4wxnyRbjgRyPLDZGLPFGNMBPAVckGSZEoYxZhXQkGw5koEx\nZrsxZq3zfyuwARiXXKkSg7G4nbe5ziuqdBUZoRT8uBr4W7KFUOLGOOBLn/d1ZEjHoPQiIpOAmcDb\nyZUkcYhItohUAzuBV40xUbU9J7ZiJQ8ReQ0oC3DqNmPMC06Z27BTzCcTKVu8CaftGYQEOKYJvjII\nESkCngVuNMa0JFueRGGM6QJmODbT50RkmjEmYtvSkFEKxpj5oc6LyLeAc4HTzRDLAjhQ2zOMOuAQ\nn/fjgW1JkkVJMCKSi1UITxpjlidbnmRgjGkSkUqsbSlipZARy0cicjZwC3C+MWZfsuVR4sq7wBEi\n8hURyQO+AaxIskxKAhARAf4AbDDG/CLZ8iQSERnl9aoUkUJgPrAxmroyQikA9wPDgFdFpFpEHkq2\nQIlCRL4mInXAXOBFEXk52TLFE8eh4HrgZayh8c/GmI+SK1XiEJE/AVXAkSJSJyLXJFumBHIScAUw\nz/mdV4vIOckWKkGMASpEZD12YPSqMWZlNBXpfgqKoihKD5kyU1AURVHCQJWCoiiK0oMqBUVRFKUH\nVQqKoihKD6oUFEVRlB5UKShKmIiIO8CxJSKy1XF//FRElvsmYROR651srUZERiZWYkWJHFUKijJ4\nfmmMmWGMOQJ4GnhdREY55/6JDST6ImnSKUoEqFJQlBhijHkaeAX4pvP+fWNMTVKFUpQIUKWgKLFn\nLTAl2UIoSjSoUlCU2BMoU6uipAWqFBQl9szE5l1SlLRDlYKixBARuQg4E/hTsmVRlGhQpaAo4XOA\nk3nU+/pP5/h/eF1Sgcuxe+XuAhCR7zpZascD60Xk4STJrihhoVlSFUVRlB50pqAoiqL0oEpBURRF\n6UGVgqIoitKDKgVFURSlB1UKiqIoSg+qFBRFUZQeVCkoiqIoPfz/CVTKuZlUPwcAAAAASUVORK5C\nYII=\n", "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYcAAAEUCAYAAADA7PqTAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJztvXl8VeW1+P1dmQMZCKKEQQSqCApYUCm2Vo+z16ogYNVa\nFdG31/pqBy9vLw7YtNVfW7Wt7WurHa7FW73VqqiII2iichsnEKJWJiVCIIwJIcHMeX5/PGeHkzMk\nZ94nJ+v7+ezPydnjOvucPGuvZ01ijEFRFEVRfMlwWwBFURQl9VDloCiKogSgykFRFEUJQJWDoiiK\nEoAqB0VRFCUAVQ6KoihKAKoclLggIt8VkV0i0iQih8V4rmoROTtesiUTEXlJRK5J0rXGeO93pvd9\nhYhcHw+5RMQjIjXxklXpf6hyUJzBuFlEGkVkv4j8U0RuEJGwfh8ikg38GjjXGFNgjNkXR9nKROTR\nPvYJqky8A1yXdwBtEpEaEfmHiJwcZF8Rkc9E5F+xyGuM+TdjzCOxnCOCa2313u/OSOQSkfkisirx\nEir9GVUOisNFxphC4CjgF8B/Av8V5rHDgTzg4wTJFgs7jDEFQCEwE1gPvCUiZ/ntdxpwBDA+mPJQ\n4oOIZLktgxIeqhyUHhhjGowxy4DLgGtEZDKAiOSKyH0istU7ffSQiOSLyARgg/fw/SLyunf/34rI\nNhE5ICKrReTrzjVEZImI3OXzPugUhoicD9wGXOZ98l8Xw+cyxpgaY8ydwF+AX/rtcg3wHPCi9++g\niMgiEXnKb91vReR33r+7p3ZE5Esi8rqI7BORvSLymIgMCXFeEZHfiMhuEWkQkSqfe58vIr8Skc+9\n21Z5140VERNswBWREd5zLPSVS0QmAQ8Bp3jv6f5w7p+IjBSRp0Vkj4hsEZHv+WybISKVXquzVkQe\nEJEcn+1GRP5fEdkEbPJZd4OIbBKRehH5vYhIOLIoyUGVgxIUY8y7QA3gDOq/BCYAXwaOBkYBdxpj\nNgLHe/cZYow50/v3e959hwL/AzwpInkRyvAy8H+AJ7zTJyfE8JF8WQpMF5HBACIyCJgHPOZdLvcd\n3Pz4O3CBiBR5j80Evon9jP4I8HNgJDAJOBIoC3Hec7HWywRgCFY5O9Nz9wEnAl/F3s8fAV2hPpyI\njAXeAB4wxtznu80Y8wlwA1DpvadBlZXf+TKA54F12O/9LOAHInKed5dO4IfAMOAU7/Yb/U4zG/gK\ncJzPuguBk4ETsPfwPJSUQZWD0hs7gKHeJ7r/B/ihMabOGNOIHbQvD3WgMeZRY8w+Y0yHMeZXQC5w\nbFKk7psd2IHbGRjnAK3Aq8ByIAv4RrADjTGfA2uwgx3AmcAXxpi3g+y72RizwhjTaozZg/XLnB5C\npnbs1NdEQIwxnxhjar0D8wLg+8aY7caYTmPMP40xrSHOcxxQAfzYGPOn0LcgIk4GDjfG/NQY02aM\n+Qz4M97v3xiz2hjztve7rgb+GORz/tz722n2WfcLY8x+Y8xWoBz7MKGkCDr/p/TGKKAOOBwYBKz2\nsfwFyAx1oIj8B3A99qnZAEXYJ8tUYBRWJmdK5RrgH8aYDqBDRJZ61z0T4vj/Aa4A/hv4FsGtBkTk\nCOB3WOurEPswVh9sX2PM6yLyAPB7YIyIPAMsxPpy8oBPw/xsVwKbgaf62tFHzjFAtyPe66Px5Shg\npN8UVCbwlvf4CVjFdxL2d5IFrPY7x7Ygl97p8/cXgP91FRdRy0EJitcpOwpYBewFmoHjjTFDvEtx\nkEHEOfbrWIf2N4ES79RFA1ahABzEDiIOpb2IkoiywZcAa4wxB0VkNPbp/9sislNEdmKnmC4QkVDK\n7EnA4z32EkIoB+yUkgGmGmOKgG9z6B4EYIz5nTHmROw03QTg/8Pe+xbgS2F+tjLvMf/jnfIKeim/\n6zpRTwUhvtNtwBaf736IMabQGHOBd/uDWEf/Md7PeVuQz6nln/sZqhyUHohIkYhcCDwOPGqM+dAY\n04WdRviN92kYERnlM+fsTyHQAewBskTkTqzl4LAWO/gOFZFS4Ae9iLQLGCt9h9Vmi0iez9LDKvY6\nfEeJyI+xFs1t3k1XARuxU15f9i4TsP6WK4JdyDtFVAH8FTtofhJCpkKgCeuoH4Ud7IMiIieLyFfE\nhgUfxCqETu+9fxj4tdcpnCkip4hIbohTtQOXAoOBv4W4b7uA0b34Vfx5FzggIv/pdYRnishkORTV\nVQgcAJpEZCLw3TDPq6QwqhwUh+dFpBH7lHg7dprgWp/t/4mdrnhbRA4AKwntQ3gFeAk76H6OHeh8\npxX+hnVuVmPn+Z/oRa4nva/7RGRNL/u9iLVunKXMu36kiDRhB+n3gCmAxxjzqnf7NcAfjDE7fRds\nRE9vSWP/A5xNaKsB4CfAdKzV9ALWER6KIqwCrsfes31YRzTY6aUPvfLXYYMDQv7vGmPasH6UI4CH\ngyiI17FhxztFZG8vMjnn6wQuwirOLVjL5C9AsY983wIavZ+ht+9T6SeINvtRFEVR/FHLQVEURQlA\nlYOiKIoSQH9TDibSpbKyMuJjkrmksnypLJvKl76yqXwJkS1i+ptyiJjW1lC5QqlBKsuXyrKByhcL\nqSwbqHyxEC/Z0l45KIqiKJGjykFRFEUJQJWDoiiKEoDWVlIUxRU6Ozupq6ujvb09YFtRURE7duxw\nQarwSFX5srOz43YuVQ6KorhCXV0deXl5DBs2DP9WDm1tbYwcOdIlyfomFeUzxtDU1ER+fn5czqfT\nSoqiuEJ7ezsFBQUBikGJDhGhoKAgbtaDKgdFUVxDFUN8ief9VOWgKIqiBKDKQVFSkUWLYMoUOPro\nnsuUKXabEjfuvvtujj/+eKZOncqXv/xl3nnnnZD7LlmyJCUd0YlAHdKKkors3AmZmTB6dM/1+/fb\nbUpcqKysZPny5axZs4bc3Fz27t1LW1tbyP2XLFnC5MmTGTYsVZoaJg61HBRF6Tc0NsI990BTU3zO\nV1tby7Bhw8jNtb2Thg0bxsiRI1m9ejWnn346J554Iueddx61tbU89dRTvP/++1x55ZVccMEFNDc3\n89prrzFt2jSmTJnCggULuktXLFq0iOOOO46pU6eycOFCAJ5//nm+8pWvMG3aNM4++2x27doVnw+R\nIFQ5KIrSbygvh7fesq/x4Nxzz2Xbtm1MmDCBG2+8kTfeeIP29nZuvvlmnnrqKVavXs2CBQu4/fbb\nmTdvHieddBKPPfYYL774IiLC/PnzeeKJJ/jwww/p6OjgwQcfpK6ujmeeeYaPP/6Yqqoq7rjjDgBO\nPfVU3n77bT744AMuv/xy7rnnnvh8iASh00qKovQLGhvhhRdgwgRYvhzOOAMKgnYxD5+CggJWr17N\nW2+9RXl5OZdddhl33HEHH330Eeeccw5gk/VGjBgRcOyGDRsYN24cEyZMAOCaa67h97//PTfddBN5\neXlcf/31fOMb3+DCCy8EoKamhssuu4za2lra2toYN25cbMInGLUcFEXpF5SXQ1sbFBba13hZD5mZ\nmXg8Hn7yk5/wwAMP8PTTT3P88cezdu1a1q5dy4cffsirr74acFyoLppZWVm8++67zJ07l+XLl3P+\n+ecDcPPNN3PTTTfx4Ycf8sc//pGWlpb4fIAEocpBUVKR0lLo7ISamp5LZ6fdNsBwrAbno5eWWush\nVt/Dhg0b2LRpU/f7tWvXMmnSJPbs2UNlZSVgk/U+/vhjAAoLC2lsbARg4sSJVFdXs3nzZgD+9re/\ncfrpp9PU1ERDQwMXXHABv/nNb1i3bh0ADQ0NjBo1CoBHHnkkNsGTgE4rKUoq8otf2EUBDlkNXr8x\nubmHrIeLLor+vE1NTdx8883s37+frKwsjj76aP70pz/xne98h+9973s0NDTQ0dHBD37wA44//njm\nz5/PDTfcQGZmJqtXr+avf/0rl156KR0dHZx88snccMMN1NXVMWvWLFpaWjDG8Otf/xqAsrIyLr30\nUkpKSjjzzDPZsmVLHO5M4lDloChKyrNuHRgD1dU9169dG5tyOPHEE/nnP/8ZsH7YsGG8+eabAevn\nzp3L3Llzqa6uJj8/n7POOosPPvigxz4jRozg3XffDTh21qxZzJo1K3phk4wqB0VRUp7Fi92WYOCh\nPgdFURQlAFUOiqIoSgCqHBRFUZQAVDkoiqIoAahyUBRFUQJQ5aAoyoDE4/Hwyiuv9Fh3//33c+ON\nN8Z03jvvvJOVK1dGfFxFRUV3qY1UQJWDoigDkiuuuILHH3+8x7rHH3+cK664os9jjTF0dXUF3fbT\nn/6Us88+Oy4y9kZHR0dCz6/KQVEU19jVtIuK6gqeW/8cFdUV7GoKUcZ60SKYPz9wiaHx0bx581i+\nfHl3me3q6mp27NjBqaeeyr333svJJ5/M1KlT+fGPf9y9/dhjj+Xqq6/mvPPOY9u2bcyfP5/Jkycz\nZcoUfvOb3wAwf/58nnrqKQDee+89vvrVr3LCCScwY8YMGhsbaWlp4dprr2XKlClMmzaN8iBFourq\n6pg9ezZTp05l5syZVFVVATbL+qqrruJrX/saV111VdSfPRw0CU5RFFeoa65jU8Mm8rLyKMotorm9\nmX9u+yfHHnYsnW2dPLf+OYrzipk0bBLDd+6EsWMDT+KfMh0Bhx12GDNmzODll19m1qxZPP7441x2\n2WWsWLGCTZs28e6772KM4eKLL+bNN99kzJgxbNq0iUceeYSf/vSn7N27l+3bt/PRRx8BsH///h7n\nb2tr47LLLuOJJ57g5JNP5sCBA+Tn5/Pb3/4WgA8//JD169dz7rnnsnHjxh7H/vjHP2batGk8++yz\nvP7661x99dWsXbsWgH/961+sWrWK/Pz8qD97OKjloCiKK2xt2EpeVh752fmICPnZ+XSZLt7Y+gZd\npquHwmhub06IDL5TS86U0quvvsqrr77KtGnTmD59OuvXr+8uznfUUUcxc+ZMAMaPH89nn33GzTff\nzMsvv0xRUVGPc2/YsIERI0Zw8sknA1BUVERWVharVq3qfuqfOHEiRx11VIBy8N3nzDPPZN++fTQ0\nNABw8cUXJ1wxgCoHRVFc4mDbQfKy8nqsO9B2gK6uLjIko1th5GXl0dDakBAZZs+ezWuvvcaaNWto\nbm5m+vTpGGO49dZbu0t2b968meuuuw6AwYMHdx9bUlLCunXr8Hg8PPTQQ1x//fU9zm2MQUQCrhmq\n1Hdf+zjn8pUhkahyUBTFFQbnDKalo2dPgy/avmBQ9qAe6/Ky8mjrDN3XORYKCgrweDwsWLCg2xF9\n3nnn8fDDD9PkrQe+fft2du/eHXDs3r176erqYu7cudx1112sWbOmx/aJEyeyY8cO3nvvPQAaGxvp\n6OjgtNNO47HHHgNg48aNbN26lWOPPbbHsb77VFRUMGzYsADLJNG45nMQkSOB/wZKgS7gT8aY37ol\nj6IoyWVM8Rg2tdnpmrysPFo6WsiQDIpyew6CLR0t5GTmJEyOK664gjlz5nRPL5177rl88sknnHLK\nKYBVII8++iiZmZk9jtu+fTvXXnttd9TSz3/+8x7bc3JyeOKJJ7j55ptpbm4mPz+flStXcuONN3LD\nDTcwZcoUsrKyWLJkSXcPa4eysjKuvfZapk6dyqBBg1zp/yDhmDgJubDICGCEMWaNiBQCq4HZxph/\n9XJYxMJWVFTg8XiilDLxpLJ8qSwbqHyxkAqy7dixg8yiTD7Z+wkNLQ0U5xVzxKAj2LBvA+MzxlOX\nU0dLRwstHS2c8+fXKKgL0tmntNSVvhfV1dWMDeYgTwHWrFnD9OnT/VcHzm/1gWuWgzGmFqj1/t0o\nIp8Ao4DelIOiKGnE8ILhDC8Y3mPdYYMOY/eO3RxoPUBxXjHTR0yn4NeXuCThwMU1y6GHECJjgTeB\nycaYA6H2q6ysNE5Mcrg0NTVREGsX8gSSyvKlsmzQj+Xbvh3a2wPXZ2eDt41kQvC5blNhIQXedpcJ\nv24IioqKGDp0aNBtbW1t5OQkbiopVlJZvt27d/PFF1/0WOfxeCK2HFxXDiJSALwB3G2MWdrH7jqt\nFCONjfDgg3DjjdDXuKr3LjZCyjd/fuiY/SVLEieQz3Urjj0Wz4YNybluCHbs2MHIkSODbkvlaRtI\nbfniNa3karSSiGQDTwOPhaEYlDhQXg5vvWVfFUVRQuGachAbtPtfwCfGmF+7JcdAorERXngBJkyA\n5cuhKYh/T1EUBdy1HL4GXAWcKSJrvcsFLsqT9pSXQ1sbFBbaV7UeFEUJhZvRSquIYh5MiQ7Haigt\nte9LS631cMYZffselBhYtAh27uy5btUq2LwZklC5UwmNx+Ph1ltv5bzzzuted//991NVVcWBAwe6\ni+eFy/XXX88tt9zCcccdF3Kfhx56iEGDBnH11VdHLXey0MJ7AwTHanBybXJzD1kPF13krmxpTbCC\ncZs32/X+ReMczZ0oSksPXXPs2EN/J/q6KYpTV8lXOTz++OPce++9fP3rXw/Yv6Ojg6ys0EPmX/7y\nlz6vecMNN0QnrAuochggrFsHxgSOR2vXqnJIOmef3XuEUDBrA2JP+PI9tqLClQilWKjaWcXS9UvZ\n2rCVMcVjmDNxDlNLp0Z9vnnz5nHHHXfQ2tpKbm5ud8nu0aNHM3nyZD766COWLFnC0qVLaWpqorOz\nk/Lycm666SZeeeUVxo8fT3Z2NgsWLGDevHl4PB7uu+8+TjrpJAoKCvj+97/P8uXLyc/P57nnnmP4\n8OGUlZVRUFDAwoUL2bx5MzfccAN79uwhMzOTJ598kuHDhzNr1izq6+tpb2/nrrvuYtasWXG8i+Gj\nymGAsHix2xIoYZOA8tT9naqdVdxXeR8leSWMLhpNfXM991Xex8JTFkatIEKV7PYvlrdmzRqqqqoY\nOnQoTz31FNXV1axYsYJBgwYxadIkFixYEHDugwcPMnPmTO6++25+9KMf8ec//5k77rijxz5XXnkl\nixYt4pJLLqGlpYWuri5ycnJ45plnKCoqYu/evcycOZOLL744aAG/RKPKQVGU8EiURRMGS9cvpSSv\nhJL8EoDu16Xrl8ZkPThTS45yePjhhwP2Oeecc7qT9VatWsWll15KRkYGpaWlnHHGGUHPm5OT093y\n88QTT2TFihU9tjc2NrJ9+3YuucRmfufl2eq07e3t3Hbbbbz55ptkZGSwfft2du3aRakLU3+qHBTF\nTfqTw9pFi2Zrw1ZGF43usa44r5itDVtjOu/s2bO55ZZbepTsrvb7PL4lssNNGs7Ozu5+2s/MzAxo\n6RnqPI899hh79uxh9erVZGdnM3bsWFpaWoLum2hUOShKIvF1Avuvh+AD7osvQl1dYCJKZ2ciJOwX\njCkeQ31zfbfFANDQ0sCY4jExnTdYye7eOPXUU3nkkUc4/fTT2bVrFxUVFXzrW9+K+LpFRUWMHj2a\nZ599ltmzZ9Pa2kpnZycNDQ0cccQRZGdnU15ezueffx7Nx4oLqhwUJZFEM93S0QE5OTBkSM/1NTWR\nn8uxTFavhmafbmq33GId0i5VNY2UORPncF/lfYC1GBpaGqhvqee6adfFfG7/kt29MXfuXF577TXO\nOeccxo8fz/Tp0ykuLo7qun/729/493//d+68806ys7N58sknufLKK7nooouYMmUKJ510EhMnTozq\n3PFAlYOiJIrt2209I3/6GpCzsmycsV9PYqJpDelYJmvXwuie0zI9wllTnKmlU1l4ysIe0UrXTbsu\nJn+DwyWXXNJjmmfs2LHdfaHnz5/PfJ/vMCMjg/vuu4+9e/dSWFjIjBkzmDJlCmBraTk0+Vh98+bN\nY968eYDt0+BwzDHH8PrrrwfIU1lZGfNnigeqHBQlUbS3RzdH7xSjmz07suPSnKmlU+OiDGLlwgsv\n7O4Mt3jxYlecxclAlYOiKOHRl/9kgFBRUZHSVVnjhSoHRXGTYANuU1NqDrgJ8E0YY1yJ4U9X4tmC\nQZWDorhJsAHXcSInu7xGksnOzu5uiKQKInaMMTQ1NdEerJFUFKhyUJRUI15P6IsW2SilVatg3z74\n9FO7PjMTvvlNq3xcVDhDhw6lrq6ORqcjnQ91dXUp22kNUle+7Oxsmn2j0mJAlYOiJIrsbHfn6Hfu\nhLlzA9dXV8Nxx9l2gLESQ9Z0ZmYmhx9+eNBtGzduDNbNLGVIdfnigSoHRUkUo0b1u+J2EdMP6kDF\nu2DfQMHVNqGKoiiJxCnYV99c36NgX9XOKrdFS3nUclCUaHGxEJ0SHokq2DcQUOWgKNHSD6ZUBjqJ\nKtg3EFDloCjpiiatJaxg30BAlYOipCu9TW351AGKiRRXQIks2JfuqHJQlFQm1f0aqSBDLySyYF+6\no8pBUVIZ9WvETKoU7OtvqHJQlGhJtSmVVLcylH6FKgdFiZZUG3DVylDiiCbBKYqiKAGoclAURVEC\n0GmlFKCxER580NZBKyhwWxolpUg1v4YyYHBVOYjIw8CFwG5jzGQ3ZXGT8nJ46y2YNAkuushtaZSU\nItX8GsqAwe1ppSXA+S7L4CpdXfDCCzBhAixfbpuA9UVjI9xzT3j7KgMIx8rwX9TKiCtVO6uobapl\nwXMLKKsoS9sifq5aDsaYN0VkrJsyuE1jI7S1QWEh1NVZK6Iv60EtDSUovVkZ/mGuHg/Mnz8gwlzj\nWbLbqfJ6TuY5Paq8LjxlYdrlUkg8e45GJYBVDsvDmVaqrKw0ra2tEZ3faUOYinR1QX19E01NBezf\nDyXe8i+jR0NGRuC+e/bA0KFQW2u3d3UF3zdepPK9A5UvIqqrobUVOjsBaDr8cAr27LE/opIS23si\nSpo7mtnfsp+2zjZyMnMYkjeE/Kz8mMSN171r7mhmV9MuMjMyyZRMOk0nnV2dDC8YHpWMtU21dHR1\nMNgMpjXTjkUdXR1kZWQxomBEzPLGg2D3zuPxRNyHtV85pE855ZSIj6moqMDj8cRfmDiwbBk0NlZw\n770e1q+HiRNhxgyYMyfQIli2DB59FE44AbZvh6OOgs8/D75vvEjlewdpJl88Eth6O8fOnbB2LQwZ\nYmX71rfw/OMfsH8/fPnLUTclcp6kS/JKKB7srV3UWB/zk3S8vtuyijLqc3sW3qtvrqckq4QyT1nE\n51vw3AJGF41m0sFJbCjYAECX6aLmQA0Pex6OWd54EK9716+UQ7qxbh2MHAmbNtmOkps2wbHH2v9h\n3wG/sdH6JcaOhaefhgsusOtLS62f4owzNMqp3xOPBDYXkuAi6pfgQgZ3vEt2O1VefUnXKq9uO6QH\nNIsXgwjk5tr/j9xcqyQWL+65X3m59Uvs3Qvt7bBjh12fm2vXl5cnX3ZFATv4FucV91gXcvD1Kq+q\nsXmUja1mwdh1lI2tpmrfvxIm35jiMTS0NPRYF8tgPmfiHOpb6uno6qDLdFHfXE99Sz1zJs6Jh7gp\nhavKQUT+DlQCx4pIjYgMqDq6GzfCgQPWGQ329ZVX4NNPD+3jWA3OzEB+Pqxeba2M6mowxloaygBk\n0SLrVHaWVavg2Wdh5cqkiRDp4FvFTu6jknqaGU0R9TRzX/HHCYv4cQbz+ub6uAzmTpXXrIwsag7U\nUJJfkpbOaHA/WukKN6/vFk7SW2UlnHYa5OTY9Tk5dtttt8ETT9h1jtWQmwunn27XJdrXoPRCKhW3\n859GcnwK+/cH7lta2jP2uaPD7hfjfGSk/RKWsp4S8ijBOoNLyIeunIS17UxEye6ppVOpK6hLGR9D\nolCfgws4oajvvmuVw759h7YZAytW2P/jggLrlzAmcNrY3y+hJIn+WtzOUVyOYhs0yDqiIaY8iEgH\n3600MJqiHuuKu3IS2razP5bsjmf4bbSockgyzjTRhAnWGX3CCXadw7Jl8Oc/H8p3cPwPWmIjzYlH\nmYyCAmsNNDX1PJdzDl/LpqIi6gglfyIZfMdQTD3N3ZYDQENGW1o6dKPFNwLMzVwKVQ5JxpkmcpLe\nfBWDr+Lwj0LSxLc0Jx5TUmefbV+rq+M28MeV0lLmbK/jvuJt0JVDcVcODRlt1A/O5LoEOHRT4ek7\nGiKKAEsgqhySiK9zGexrQ8OhKSR/xfHCC9a/cNVVoZWGovQbfvELpgIL/Qbt6xIwaKfK03c0xDv8\nNlpUOSQRX+cy2NeuLrve4wlUHA8/bLOf9+yxEUrnnht+iQ1lANBPK7YmwweQKk/f0eDkUvgm7rmR\nS6HKIYkEcy6PHWudy8b0VBwAW7bAMcfA3/9uI5m2bLH7q/XgIqk0IMcyFbV9uw1/9SdNai2lytN3\nNEQaAZYoVDkkEf/kNrB+wfnz4Wc/66k4tm2zVkV1tZ2OKi21eRHjxh1KfFPrwQXcHjjjFUrb3t4/\no67CJFWevqMhEeG30aDKIUXwVRyNjbBwoa2ztHy5nVpysqPfeQeOPFJDWQcs/TWUNsmkytN3tKRC\n+K0qhxTE8U2AnU46/ng4eBC+9CU47DD41a90SintcSyE1auhudkmw3R02MqqBQWQlWVjoQsKDkUp\nKd2kytN3f0aVQwri+CbeecdGMzl1lQoL7aJTSgMAx0JYu9bWZW9qguJi2LULioqgpSV0NnQ/IdGh\npqnw9N2f0cJ7KcjixTZM/Ywz4NRT4Ygj7NTSjh1WUWgtpTTGqZfk1EnauhXWr++ZEJMGOKGm9c31\nPUJN07WrWn9ELYcUZvHiQ/6HggKrFGbNgm9+023JlIThazEMGWLf5+VZE9Khvt4+Jaxfb1+ffdZa\nFosW9e2UdqarZsywDUIc8vPhxBOTFnXVn0NNBwqqHFKc8nLrb6ipsQri4YdtPwf1OQxQMjIOOaR8\nccr29oWjfEpK4NvfPrQ+yVnV/TnUdKCg00opjJNR3dJyqJvjli3w4otuS6a4RmGh9T0UF9vWgWPG\nwOzZ/c4pHe8+C0r8UcshhXGshi1bYPBgG6CSkwN//CNs2AA//KFaEGmLU0Svq8s2/ejstBFLmZn2\n7/z8uJTcdotwQk37a22kdEGVQwrhX3l13TqorbXTzd6+8GRmWmVx4ICtuKxRS2mKvyXgO+0zf37w\nXIfVq/tN1nNfoabNHc0hayMBqjSSgCqHFMK/8qqTGOfbGa69Hd57z1oQWkYjDYmlPEdzc6DSWLnS\nRj45/oj46zw1AAAfgUlEQVRVq6yz+7vfjVXSmOkt1HR/y/6gDusH33+Qg+0H+2VBvf6GKocUIVS5\nbv+SG8uWWcVw1FG2YqvmPKQZ4Tzhh1Ig+fmB65ySv47S2LzZrmtvD97zIUVo62yjeHBgb+rnNzzP\naUedplFOSUCVQ4rg+Bc++wyGDw8+6Acr+a3WwwAklAIJNqXkjzNdVVCQmj0fvORk5tDQ0hBQG8lg\nKM4LVBoa5RR/NFrJRRobbcJrbe2hqKTPP7evy5f3bPkLwUt+O0X4FCWdGJI3hPqWeuqb6+kyXdQ3\n11PfUs/M0TM1yilJqHJwkfJyqyB+//tDUUmHHWZfDx4MHPR9S347izGaMa2kH/lZ+Sw8ZSEl+SXU\nHKihJL+EiydcjCC8sOkFXtr0ErWNtd1KY04COskNdHRaySWcKaKvfx1eecX2e29osNZ+U5O1Jvwr\nrwYr+a0o3QTzRTQ1BfoTVq6Eyy8PnIZKsagmX4e1b2e3s8adxdqda3lty2ucPf7spDij/cNqv9Lx\nlYReLxVQ5eASjo+huRnGj7eK4dJL7VRRa6vt+PbDH7otpZIShNvDIdjA7hzrqzR27oTs7MDIphQu\n++1fbmNE4Yjufg3JUAz+YbW7WndRtbMqrZ3gqhwSiH/egu96x8fQ2mob++zbB9On2+2+vgSNRFK6\nS16sXNnTEdXUZLf19sQfbP38+baya21tIqQNSqwJbW6W2whWByqzPTPtI6RUOSQQ/7wFsIrh5put\nxbBliw0eqa62tdX+93/t//vEifbBThv6KD1oarLF+HwZOza8J35f62PVKlt479lnk9IPItiTd6S5\nCeF0dktURnUwxZQpmWkfIaUO6QThn7fgPPCVl0NlJXzyiZ1K6uqy/oZx42DUKNu/5fLLbZSh+hiU\nuOFYH2PHHmoWNGRIYEhcAvB98s6QDErySyjJK2Hp+qVhn2POxDlBo5ccR3QiS4AHqwPVaTrTPkKq\nV8tBRDKB64HRwMvGmP/12XaHMeauBMvXb3HCTgsLrf+gvBw8Hqswzj3XzhDMmmWL6V16qf3fFYER\nIzR3QUkvIp0SqtpZRW1TLQueW0BOZg6C0NrZyuDswfxr97/4cM+HtHW2Mb5kPBv3bWRq6dSElgAP\nVgeqs6sz7SOk+rIc/gicDuwDficiv/bZFvOdEZHzRWSDiGwWkUWxni9VCJWs9uKLh7q6tbfb5j1g\nfQxbt9och8JCzV1QEkxBgW05un+/tRycuOgEZUlHUoHVsQA6ujrIzsjmjeo3qKiuIDsjm8/3f07l\n9kqKc4uZfPhkOjo7+NHKH/HUx0+xtWFrwpLjnDpQvmG1wwuGp7W/Afr2OcwwxkwFEJEHgD+IyFLg\nCkBiubDXKvk9cA5QA7wnIsuMMf+K5bypQLBktYMH4U9/stNIbW1Wgbz3nrUiNm2yTmnNfFaC4oSo\n+k8BRfvjOPtsa7LOnn2ooJ/jk0hAeGs4FVgdHAsgqzOLDXs3UJRbBMCGfRuo3l9NfmY+rZ2tZGRk\nMCTf+l8eeO8BPGM9ffokYsG/DlRFRUVczpvK9KUccpw/jDEdwHdE5E7gdSDWYWsGsNkY8xmAiDwO\nzAL6vXLwTVZzqK21DuisLPv/VlxsS/EPHQqXXXaoXhJotJLihzM4z5xpzU6HpibbzW3YsOjOu3Ll\nIYWwatUhZePrpI5DeGtfFVh96Z6COmgHd0c5NLQ00NjayODswbR0tHTvX5RbxPbG7REpICU8xBgT\neqPIo8CjxpiX/dZfDzxojMmO+sIi84DzjTHXe99fBXzFGHNTqGMqKytNa2trRNdpamqiIAUev3fs\nsL4HETullJ1tFcjIkU20thYQ7GPl5lofhFukyr0LxYCTr7r6kDnqS2tr8BLevmzfbn94jmyFhRTU\n1tofYlGRbT2a5X1W7OiwlkW4544jtU21dHR1MNgMZm/HXrpMFwAZkkFrZytdXV1kZGSQm2nvQ0dX\nB1kZWUw4bALNHc3sb9lPW2cbOZk5DMkbQn5WkGKEcSCVf3vBZPN4PBHP9PRqORhjvh1i/V+Av0R6\nMT+CCRtaUwGnnHJKxBepqKjA4/FEfFy8WbYMVqywuQ3vvQcnn2zDV488soIrrnBfvmCkyr0LRb+S\nL9xENl/8j3Ge7v3DT4O1+OzjehUVFXief94O/LW1NqzVCZPdv99OOYU6dwJxfA7nZJ7D6pzVvPn5\nmwCcdtRpbGvYxptb32RU4SiGDx7OgdYDHGg7wD1n34PneE/SZITU/u3FS7Y+8xxE5DDgW8BE76pP\ngL8bY/bFeO0a4Eif96OBHTGeM6VwkuCuuso6qEtK4I037CzAxo02eqmh4VBVZSWNcUJJ/elt2sb/\nmLVr7QC+f39irpcCOFNQH7//Me2d7Zw+9vTuaKUZo2dw1rizeH7T82xv3M6owlHccdodzDt+ntti\npyV9hbJOwvoXXgE+wD7tnwzcJiJnGmPWx3Dt94BjRGQcsB24HKuE0gYnCe7AAetDcLo9FhZaa33H\nDpgyRX0LSj8iGgsoQqaWTqWuoI6HPQ8H3f4fX/uPuFxH6Z2+LIefAd83xvzDd6WIzAXuBuZGe2Fj\nTIeI3IRVPJnAw8aYj6M9X6rhmwT3yiv2df16ayU4QSfr18N552kmtJICOD2r4VB4KwSGt4ZpkUSb\nrewcN65hHGUVZdoC1EX6Ug5TjDEBNpsx5mkR+T+xXtwY8yLwYqznSUV8k+AmT4Y5c4IrgIqK8Hq0\nKEr3AO47eEP0+Qm+VVyPPrrn+hisgGjLZfgeNyFjgrYAdZm+lMPBKLcNaHyT4Fpb7f/fM89o3oIS\nI77hpfFwEieoPHe02cq+x0mTaAtQl+lLORwhIrcEWS/A4QmQJy3wTYLbsMFGEWZmqm9hQBOq73Nv\nT/3RHBOPY2Mk2gqqblZeVQLpSzn8GSgMsS3WUNa0xUmC27QJVq+2oeRbt8K776pyGLBE85Qey5O9\ni017wqmgGs/jIiFRlVvTkb7yHH4SapuI/CD+4qQHixfbqaWbbrKVkY8+2tZNmjHDbcmUtCQJEUQ9\nztmHRRJutrL/QD358Mks27gMAGNMd+XVeGU5x6N0+EAiln4OtwD3x0uQdOOFF+w0kqMQtF6SkjCS\nmdMQhrIJp1xGsIF62cZlXDzhYj7a8xHtDe2U5JeELLMRDYms3JqOxKIcYiq8l840NsJf/2qrEmzZ\nYsNYw6mXFKpznKL0N/wL1fkTaqD+aM9HlHnKqKio4BrPNXGVSX0akRFLs59eS10MZMrLYfduW6qm\noQHeecc+xBljcxp6O+6tt7Rct5L+JLLEdigiKR2u9J0h3UhwJSBAYipa9QN6e8J3wljPP99aC62t\ntuDer37VuzXg3zlOp5+UdCYZzmd/tHJrZPRqORhjCo0xRUGWQmPMgO0/3dsTfrBeDuE07/FNmoum\n2U9jI9xzT1K6PipKzPTV9jOeVO2soqyijPvfuZ9B2YNo62zrbtqjzujQDNgBPlr6esIP1ssBei+R\nEapznHPucHwRjsKaNEnDZQcUixbZeOlVq3quz8+Hb3zDHZnCIJIeD7Hg7/h2rAVVCn2jyiFCgvWG\n9h2MFy+O/pzBrI2LLup74NcpqQHMzp0wN0iJs+pq13Idws0l6MtpHQ80Qil6YnFIDzhCPeH7TuVE\nM73ja204i+O89h/4g5031ikpRYkXzpN6fXN9j1yCqp1VrsjjhuM7XVDLIQL6esJ39ol0eqc3a2PZ\nst4tlb6mpBQlmaTak3o4jm/Nmg6OWg4R0NsTPvT+lN+bRRFqWziWSrQOcEWJN1U7q3h2/bO88fkb\nVFRXsLPJZm27+aTel+M71SydVEIthwjoy5/Qmz+iN4si1LZwLJVoHOCKEm+cQTY3MxdjDM3tzVTW\nVHLK6FPIzcyNOUQ12qf7vhzfqWbppBKqHOJEb9M7xoR2GHd1hd4WzsAfjQNcSSNcrL7qizPITh8x\nnX9u+yd5WXnkZubyQe0HHDvs2JhyCWKtidSb41uzpkOjyiFGnDDTI46wEYXnnmvX+z7lGxPaomhs\nDL1NB36lT1ysvuqLM8hmSAZfPfKrfLL3E/Y370dEYg4bTeTTvRvJeP0FVQ4x4tsnetcuWyrjyCMP\nbX/zTbuceqp9729RNDSoM1np//gOssMLhjO8YHj3+1gH8EQ+3WvWdGjUIR0DzlTS2LG2ltIll9jy\n3A88YBt1LVkCOTmwY4ddoKdFUV5up5XUmaz0dxKZ8ZzImkiOT6Ikv0Szpv1QyyEGHIfxgQPQ3g57\n9kBe3qGpocZGeOUVW5119WrIyLCNf+BQhNORR6ozWen/JDLjOdFP98lIxuuPqHKIEsdqKCmBN96A\nYcNg40bweGDpUjvAT5gAU6bAUUfZZj9z5gQO+hUV8WkHrCi94t8QyOOB+fPj2hAoUYNsskptKD1R\n5RAlvlZDV5e1GFpa7PTR7t020ujww2H6dLu/+hMUV/FvCJSba98noiFQDDghq+MaxlFWUdYdshpM\n8cQreU2T4IKjPococcJM16+3VkRNjU1O+/hj2LYNmpttox/xtkRSf4Ki9I5vQlp2RnavCWnxSl6r\n2lnFba/fxkubXmJN7Rpe2vQSt71+mybBocohahYvttNBH3wAmzcfWu66C772NWspdHUdavQTTrMf\nRRnI+Iasiggl+SWU5JWwdP3SXvfNkIxe9+2NP7z/Bz6t+xSA4lxbg+nTuk/5w/t/iP0D9XN0WilC\nwmn0U1pq/QzhNvpRFCWykNV4hbe+XfM2hbmF5Gfb3mX52fkYDG/XvB2h9OmHWg4RkohGP4qiRBay\nGq/wVkECe10a7/oBjiqHCOirfHZfhfkUxTWcMhvO0tpqX5NcZqM3fHMljDG95krEK69i5uiZNLY1\n0tze3F0TqrGtkZmjZ8brY/VbXJlWEpFLgTJgEjDDGPO+G3JESriNfsLp3KYoScU/XDUFY6h9Q1bb\nG9opyS8JGbIar/DW7570XWoO1LD74G4aWhrIzcrl6KFH892Tvhuvj9Vvccvn8BEwB/ijS9ePmEj6\nJmjLTkWJDidktaKigms814S1b6zXu/vMuzWUNQiuKAdjzCcAIv1nXi+c8tmgLTuVAYR/Yp1DHBPr\nkoFmSAdHfQ5hEq4/IREtO6NpPaooCcdJrPNfgikMpd8hxvi76uN0YpGVQDBv1+3GmOe8+1QAC8P1\nOVRWVprW1taI5GhqaqIgSY/uXV02GS4z09ZR6uqCzk4YPdq+j1a+hgZbt+nww6G4uNdd40oy7100\nqHzRExfZqqsPmdK+tLb2zMaOglS+d5Da8gWTzePxRDxNkzDlENbFI1QOBAad9UlFRQUejyfSw6Ji\n2TJ45hmb4+AQqqZSuPI1NsLChXZqqqkpuTkTybx30aDyRU9cZJs/P7gSqK6O2dmdyvcOUlu+ELJF\nrBw0CS6OJKJlZ18RUoqSrmjNI3dxK5T1EuD/Bw4HXhCRtcaY89yQJZ74dm6LRzhrJBFSitJNGjiK\nmzuaY2oNqsSOW9FKzwDPuHHtZBGPcNZwI6QUpQf+FVgd4l2BNYH9q/e37E9Ya1AlPHRaKQH0Fc7q\na1X0RiKmqRQlbiTQCmnrbKN4cM/oi3i1BlXCQ5VDEGKdEurLT+BrVRQWhj6P7zRVomRVlFQkJzOH\nhpaGbosB4tcaVAkPzXMIQm/F9foilJ/AyVHwtyq6utyTVVFSlSF5Q8KqnVS1s4qyijIWPLeAsooy\n7cMQR1Q5+NFXcb2+6Ksyq3+SXGOje7IqSqqSn5XPwlMWUpJfQs2BGkrySwKc0fFq+KMER6eV/Ig1\ndLQ3P4HHE2hVNDTYQT0R01fKACWBjuJYiDQ0ta+yFr4Nf0Cd1vFGlYMP8Qgd7c1PsGxZoFXR1RXd\noK5hrkpIUjBc1XnKj2doarwa/ijB0WklHxLdrCdYfSaIrt+DNhZS+hPxauvpS7wa/ijBUcvBh0SH\njgazKioqbBWCSNEwV6U/kYin/DkT53Bf5X3d52poaaC+pZ7rpl0Xk6yKRZWDD+GEjqYK/UlWRRlT\nPIb65vq4hqbGq+GPlukIjiqHJKI5CcpAJZqn/HAG7Vh7MSTCF5IuqM8hiWhOgjJQcZ7yewtN9cWp\nrZToMNVE+ELSBbUckoR2iFMGOpE85SertpJGPIVGlUOS0JwERQmfZNVWys3M5ZXNr9jr5RUzadgk\ncjJzNOIJnVZKCn2V1FAUpSdObSVf4h2mWrWzim0HtnGg9QDZmdl80fYFFdUVfFb/WUCZjoHIgFAO\nbvdg1pwERYmMcGsrxcLS9Uv5UsmXOH3s6QzKHkR7VztFuUWMKho14J3RMECmleLRWyEWestJ+PrX\nky+PoqQ6Tm2lRIapOv6GDMmgtMCa9V2mi5oDNXH/PP2RtFcOXV3uO4J7y0moqEiaGIrSr+jLgd1X\nqGtfYaqJyL1IJ9J+WqmxsWcVVJ3KUZT+TzgVWfsKU50zcU7Cp676M2mtHBobbdVTdQQrSnoRTn7C\n1oatFOcFRjytrV1LWUUZ979zP4OzB9Pa2RpW7sVAI62nlcrL7bSS9mBWlPQinPyEYNNGn9Z9ypb9\nWzhqyFGMLhrdnamtSiGQtLYc1q2zr75VUI2JrgqqoiipQzgVWYNNG3205yOOP/x4SvJL2H1wN+t2\nreP9He/zvZe/p02C/Ehry2HxYuvwXbLEbUkURYkn4dRqClaYb1zxOI4+7Gh2Nu2ksqaSvMw8huUP\nY/fB3VpTyY+0Vg59oYXwFKV/Em5FVv+Ip7KKMuqb61m/dz15mXnkZ+fT3N7MEYOP6PZZqHKwDGjl\nEG3+gyoVRXGfaCqyOhbH7oO7GZY/jOb2Zlo6Wpg+YrrWVPIjrX0OveFfCC+SCCatrqoo/RPH4jhi\n8BHsbd5LfnY+Xz3yqwwvGK45Dn4MWOXgWwgvkvyHWJSKoijuM7V0Kr87/3ecNPIkThh+AocPPlxz\nHIIwIJVDLIXwolUqiqKkDpH2lxiIDEifQ2+F8HrzPYRSKtqbQVHiS3NHM2UVZWG17oy2zWesXeTS\nHVcsBxG5V0TWi0iViDwjIkOSeX3fQniR5D9odVVFSTxVO6vY1bQrrC5w4ZTRUKLDLcthBXCrMaZD\nRH4J3Ar8Z7Iu3lshvN7orbqqZlwrSnxYun4pX8r4Ulhd4HzLaPS1rxIZrigHY8yrPm/fBua5IUek\nRKtUFEUJn60NW5mQMaHHulBhptrmM3GIMcZdAUSeB54wxjza176VlZWmtbU1ovM3NTVRkMIOgVSW\nL5VlA5UvFlJZttqmWnI7c2nNPPS/3tHVQVZGFiMKRgTs62zra994ksr3L5hsHo9HIj1PwpSDiKwE\nSoNsut0Y85x3n9uBk4A5JjxBIha2oqICj8cT6WFJI5XlS2XZQOWLhVSWrWpnFWveXsMHuR/0KI0R\nLJrIt2dDX/vGE9/7F61DPBmy+RCxckiYQ9oYc7YxZnKQxVEM1wAXAleGqRgURRkATC2dyvCC4WGF\nmbodkprODnFXfA4icj7WAX26MeYLN2RQFCV1yc/Kp8xTFta+boakprND3K0kuAeAQmCFiKwVkYdc\nkkNRFCVqQjUUSgeHuFvRSke7cV1FUZR4ks59qAdk+QxFUZR4kM59qFU5KIqiRInbDvFEMiBrKymK\nkl64GU6arjWa1HJQFKVfk87hpG6iykFRlH6NbzhphmRQkl/S3fJTiR6dVlIUpV8Tj/pKqZblnAqo\nclAUpV8TSThpMCUAdJfg8J2WShfHcrTotJKiKP2acMNJQ/kmHnz/QZ2WCoIqB0VR+jXhhpOG8k28\nXfN22mY5x4JOKymK0u8JJ5w0lG/CYGhoaUjLLOdYUMtBUZQBwZjiMTS0NPRY19DSwMzRM9M2yzkW\nVDkoijIgCOWbuPGkG9M2yzkWdFpJUZQBgeOb8I1Wum7add1KYKArA39UOSiKMmBI11IXiUCnlRRF\nUZQAVDkoiqIoAei0kqIoAx4tnxGIWg6KogxotKprcFQ5+NHYCPfcA01NbkuiKEoy0KquwVHl4Ed5\nObz1ln1VFCX92dqwVctnBEGVgw+NjfDCCzBhAixfrtaDogwEQmVOa/kMpZvycmhrg8JC+6rWg6Kk\nP+FWdR1oqHLw4lgNpaX2fWmpWg+KMhAIt6rrQENDWb04VkNurn2fm3vIerjoIndlUxQlsWjmdCBq\nOXhZtw6MgerqQ4sxsHaty4IpiqK4gFoOXhYvdlsCRVGU1EEtB0VRFCUAV5SDiPxMRKpEZK2IvCoi\nI92QQ1EURQmOW5bDvcaYqcaYLwPLgTtdkkNRFEUJgivKwRhzwOftYMC4IYeiKIoSHDHGnXFZRO4G\nrgYagDOMMXv6OqaystK0trZGdJ2mpiYKCgqiEzIJpLJ8qSwbqHyxkMqygcoXC8Fk83g8EvGJjDEJ\nWYCVwEdBlll++90K/CTM80ZMeXl5NIcljVSWL5VlM0bli4VUls0YlS8WQsgW8RjumuXgICJHAS8Y\nYya7KoiiKIrSjVvRSsf4vL0YWO+GHIqiKEpwXLEcRORp4FigC/gcuMEYsz3pgiiKoihBcX1aSVEU\nRUk9NENaURRFCUCVg6IoihKAKgdFURQlAFUOiqIoSgBpoRxE5FIR+VhEukTkJL9tt4rIZhHZICLn\nhTh+nIi8IyKbROQJEclJoKxPeAsOrhWRahEJ2jHCu+1D737vJ0oev2uWich2H/kuCLHf+d77uVlE\nFiVDNu917xWR9d6ijc+IyJAQ+yXt3vV1L0Qk1/udb/b+xsYmUh6/ax8pIuUi8on3/+P7QfbxiEiD\nz3ee1DpnfX1XYvmd9/5Vicj0JMl1rM89WSsiB0TkB377JPXeicjDIrJbRD7yWTdURFZ4x64VIlIS\n4thrvPtsEpFrwrpgNJlzqbYAk7ChsRXAST7rjwPWAbnAOOBTIDPI8f8ALvf+/RDw3STJ/SvgzhDb\nqoFhSb6PZcDCPvbJ9N7H8UCO9/4elyT5zgWyvH//Evilm/cunHsB3Ag85P37cuCJJH6fI4Dp3r8L\ngY1B5PMAy5P5O4vkuwIuAF4CBJgJvOOCjJnATuAoN+8dcBowHfjIZ909wCLv34uC/U8AQ4HPvK8l\n3r9L+rpeWlgOxphPjDEbgmyaBTxujGk1xmwBNgMzfHcQEQHOBJ7yrnoEmJ1IeX2u+03g74m+VpyZ\nAWw2xnxmjGkDHsfe54RjjHnVGNPhffs2MDoZ1+2FcO7FLOxvCuxv7Czvd59wjDG1xpg13r8bgU+A\nUcm4dhyZBfy3sbwNDBGREUmW4SzgU2PM50m+bg+MMW8CdX6rfX9focau84AVxpg6Y0w9sAI4v6/r\npYVy6IVRwDaf9zUE/nMcBuz3GXSC7ZMIvg7sMsZsCrHdAK+KyGoR+U4S5HG4yWu+PxzCRA3nniaD\nBdgnymAk696Fcy+69/H+xhqwv7mk4p3Omga8E2TzKSKyTkReEpHjkypY399VKvzeLif0Q5yb9w5g\nuDGmFuzDAHBEkH2iuof9pk2oiKwESoNsut0Y81yow4Ks88/6C2efiAhT1ivo3Wr4mjFmh4gcAawQ\nkfXeJ4eY6E024EHgZ9jP/zPstNcC/1MEOTZumZTh3DsRuR3oAB4LcZqE3Ltg4gZZl/DfV6SISAHw\nNPAD07NcPsAa7HRJk9fH9CxwjP85Ekhf35Wr98/rf7wYWyDUH7fvXbhEdQ/7jXIwxpwdxWE1wJE+\n70cDO/z22Ys1VbO8T3bB9omIvmQVkSxgDnBiL+fY4X3dLSLPYKcwYh7gwr2PIvJnbCMmf8K5p1ET\nxr27BrgQOMt4J1SDnCMh9y4I4dwLZ58a7/deTODUQMIQkWysYnjMGLPUf7uvsjDGvCgifxCRYcaY\nvcmQL4zvKqG/tzD4N2CNMWaX/wa3752XXSIywhhT651u2x1knxqsf8RhNNY/2yvpPq20DLjcGzEy\nDqvV3/XdwTvAlAPzvKuuAUJZIvHibGC9MaYm2EYRGSwihc7fWEfsR8H2jSd+c7mXhLjme8AxYiO8\ncrAm97JEy+aV73zgP4GLjTFfhNgnmfcunHuxDPubAvsbez2UUos3Xt/GfwGfGGN+HWKfUscHIiIz\nsGPCviTJF853tQy42hu1NBNocKZRkkRIC9/Ne+eD7+8r1Nj1CnCuiJR4p4rP9a7rnWR52hO5YAey\nGqAV2AW84rPtdmxEyQbg33zWvwiM9P49Hqs0NgNPArkJlncJttig77qRwIs+8qzzLh9jp1SScR//\nBnwIVHl/dCP8ZfO+vwAb+fJpsmTzXnczdu50rXd5yF++ZN+7YPcC+ClWgQHkeX9Tm72/sfFJvF+n\nYqcPqnzu2QXADc7vD7jJe5/WYZ38X02ifEG/Kz/5BPi99/5+iE80YhLkG4Qd7It91rl277BKqhZo\n945312H9V68Bm7yvQ737ngT8xefYBd7f4Gbg2nCup4X3FEVRlADSfVpJURRFiQJVDoqiKEoAqhwU\nRVGUAFQ5KIqiKAGoclAURVECUOWgKGEgIk1B1vlWsd0kIktF5Dif7Td5q4kaERmWXIkVJTZUOShK\nbPzGGPNlY8wxwBPA6yJyuHfb/2ITHl0t2KYo0aDKQVHihDHmCeBV4Fve9x8YY6pdFUpRokSVg6LE\nlzXARLeFUJRYUeWgKPElKb0aFCXRqHJQlPgyDdtUR1H6NaocFCVOiMhcbMXL/tbdT1ECUOWgKOEx\nSERqfJZbvOt/6ISyAt8GzjTG7AEQke+JSA22fn6ViPzFJdkVJWK0KquiKIoSgFoOiqIoSgCqHBRF\nUZQAVDkoiqIoAahyUBRFUQJQ5aAoiqIEoMpBURRFCUCVg6IoihLA/wX+dNuczQYTbwAAAABJRU5E\nrkJggg==\n", "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "plot_step_lda()\n", "plot_scikit_lda(X_lda_sklearn, title='Default LDA via scikit-learn')" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "
\n", "
" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "# A Note About Standardization" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "[[back to top](#Sections)]" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "To follow up on a question that I received recently, I wanted to clarify that feature scaling such as [standardization] does **not** change the overall results of an LDA and thus may be optional. Yes, the scatter matrices will be different depending on whether the features were scaled or not. In addition, the eigenvectors will be different as well. However, the important part is that the eigenvalues will be exactly the same as well as the final projects -- the only difference you'll notice is the scaling of the component axes. This can be shown mathematically (I will insert the formulaes some time in future), and below is a practical, visual example for demonstration." ] }, { "cell_type": "code", "execution_count": 23, "metadata": { "collapsed": false }, "outputs": [ { "data": { "text/html": [ "
\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
01234
1456.73.05.22.32
1466.32.55.01.92
1476.53.05.22.02
1486.23.45.42.32
1495.93.05.11.82
\n", "
" ], "text/plain": [ " 0 1 2 3 4\n", "145 6.7 3.0 5.2 2.3 2\n", "146 6.3 2.5 5.0 1.9 2\n", "147 6.5 3.0 5.2 2.0 2\n", "148 6.2 3.4 5.4 2.3 2\n", "149 5.9 3.0 5.1 1.8 2" ] }, "execution_count": 23, "metadata": {}, "output_type": "execute_result" } ], "source": [ "%matplotlib inline\n", "import pandas as pd\n", "import matplotlib.pyplot as plt\n", "\n", "import pandas as pd\n", "\n", "df = pd.read_csv('https://archive.ics.uci.edu/ml/machine-learning-databases/iris/iris.data', header=None)\n", "df[4] = df[4].map({'Iris-setosa':0, 'Iris-versicolor':1, 'Iris-virginica':2})\n", "df.tail()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "After loading the dataset, we are going to standardize the columns in `X`. Standardization implies mean centering and scaling to unit variance:\n", "\n", "$$x_{std} = \\frac{x - \\mu_x}{\\sigma_X}$$\n", "\n", "After standardization, the columns will have zero mean ( $\\mu_{x_{std}}=0$ ) and a standard deviation of 1 ($\\sigma_{x_{std}}=1$)." ] }, { "cell_type": "code", "execution_count": 24, "metadata": { "collapsed": true }, "outputs": [], "source": [ "y, X = df.iloc[:, 4].values, df.iloc[:, 0:4].values\n", "X_cent = X - X.mean(axis=0) \n", "X_std = X_cent / X.std(axis=0)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Below, I simply copied the individual steps of an LDA, which we discussed previously, into Python functions for convenience." ] }, { "cell_type": "code", "execution_count": 25, "metadata": { "collapsed": false }, "outputs": [], "source": [ "import numpy as np\n", "\n", "def comp_mean_vectors(X, y):\n", " class_labels = np.unique(y)\n", " n_classes = class_labels.shape[0]\n", " mean_vectors = []\n", " for cl in class_labels:\n", " mean_vectors.append(np.mean(X[y==cl], axis=0))\n", " return mean_vectors\n", " \n", "def scatter_within(X, y):\n", " class_labels = np.unique(y)\n", " n_classes = class_labels.shape[0]\n", " n_features = X.shape[1]\n", " mean_vectors = comp_mean_vectors(X, y)\n", " S_W = np.zeros((n_features, n_features))\n", " for cl, mv in zip(class_labels, mean_vectors):\n", " class_sc_mat = np.zeros((n_features, n_features)) \n", " for row in X[y == cl]:\n", " row, mv = row.reshape(n_features, 1), mv.reshape(n_features, 1) \n", " class_sc_mat += (row-mv).dot((row-mv).T)\n", " S_W += class_sc_mat \n", " return S_W\n", "\n", "def scatter_between(X, y):\n", " overall_mean = np.mean(X, axis=0)\n", " n_features = X.shape[1]\n", " mean_vectors = comp_mean_vectors(X, y) \n", " S_B = np.zeros((n_features, n_features))\n", " for i, mean_vec in enumerate(mean_vectors): \n", " n = X[y==i+1,:].shape[0]\n", " mean_vec = mean_vec.reshape(n_features, 1) \n", " overall_mean = overall_mean.reshape(n_features, 1) \n", " S_B += n * (mean_vec - overall_mean).dot((mean_vec - overall_mean).T)\n", " return S_B\n", "\n", "def get_components(eig_vals, eig_vecs, n_comp=2):\n", " n_features = X.shape[1]\n", " eig_pairs = [(np.abs(eig_vals[i]), eig_vecs[:,i]) for i in range(len(eig_vals))]\n", " eig_pairs = sorted(eig_pairs, key=lambda k: k[0], reverse=True)\n", " W = np.hstack([eig_pairs[i][1].reshape(4, 1) for i in range(0, n_comp)])\n", " return W" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "First, we are going to print the eigenvalues, eigenvectors, transformation matrix of the un-scaled data:" ] }, { "cell_type": "code", "execution_count": 26, "metadata": { "collapsed": false }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "EigVals: [ 2.0905e+01 1.4283e-01 -5.2637e-16 -2.4119e-15]\n", "\n", "EigVecs: [[-0.2067 -0.0018 -0.5059 0.574 ]\n", " [-0.4159 0.5626 0.4445 0.0663]\n", " [ 0.5616 -0.2232 0.4866 0.1359]\n", " [ 0.6848 0.796 -0.5565 -0.8048]]\n", "\n", "W: [[-0.2067 -0.0018]\n", " [-0.4159 0.5626]\n", " [ 0.5616 -0.2232]\n", " [ 0.6848 0.796 ]]\n" ] } ], "source": [ "S_W, S_B = scatter_within(X, y), scatter_between(X, y)\n", "eig_vals, eig_vecs = np.linalg.eig(np.linalg.inv(S_W).dot(S_B))\n", "W = get_components(eig_vals, eig_vecs, n_comp=2)\n", "print('EigVals: %s\\n\\nEigVecs: %s' % (eig_vals, eig_vecs))\n", "print('\\nW: %s' % W)" ] }, { "cell_type": "code", "execution_count": 27, "metadata": { "collapsed": false }, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXcAAAD8CAYAAACMwORRAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3XGQHOV55/HvI7EyUiGtjVCdfRK76xCDIQjiYx3jg5QJ\nStVhiM1B2ZXjNpzsM7UliaRQHVdBPt2Fo1T6I+U6l5wKMlGMBLa2sC+gWICxOY7AUbGNj8UFCKLE\noWClLFoKIXySbBGvtPvcHz2DZme7Z7pnuqdnun+fqi1pp3tm3i3QM+8+7/M+r7k7IiJSLAvyHoCI\niKRPwV1EpIAU3EVECkjBXUSkgBTcRUQKSMFdRKSAFNxFRApIwV1EpIAU3EVECuiMvN74nHPO8aGh\nobzeXkSkJz3//PNvu/uKZvflFtyHhoYYHx/P6+1FRHqSmR2Ic5/SMiIiBaTgLiJSQAruIiIFpOAu\nIlJACu4iIgWk4C4iUkAK7iIiBaTgLiJSQAruIlIIY/vGGNo2xIK7FjC0bYixfWN5DylXue1QFRFJ\ny9i+MUYfGeXEyRMAHDh6gNFHRgEYWT2S59Byo5m7iPS8zU9ufi+wV504eYLNT27OaUT5U3AXkZ53\n8OjBRI+XQdPgbmbnmtlTZrbfzF4xs9si7rvKzF6o3PN/0h+qiEi4gf6BRI+XQZyZ+yngdne/ELgc\nuNXMLqq9wczeD2wHPuvuvwF8PvWRiohE2LpmK0v6lsx5bEnfErau2ZrTiPLXNLi7+5S7/7Ty9+PA\nfmBl3W3/Htjj7gcr972V9kBFRKKMrB5hx2d2MNg/iGEM9g+y4zM7SruYCgmrZcxsCPgY8JO6S+cD\nfWb2NLAU+Jq7fzOF8YmIxDKyeqTUwbxe7OBuZmcBDwEb3f1YyOtcBqwBFgM/NrNn3f1nda8xCowC\nDAyUNxcmIpK1WNUyZtZHENjH3H1PyC2TwA/c/Zfu/jbwDHBp/U3uvsPdh919eMWKpqdEiYhIi+JU\nyxhwL7Df3b8acdte4LfN7AwzWwJ8giA3LyKSurDdqNqhOlectMwVwM3APjN7ofLYfwEGANz9Hnff\nb2Y/AF4CZoFvuPvLWQxYRMotbDfqF7/7RcyM6Znp9x4r+w5Vc/dc3nh4eNh1QLaIJDW0bYgDR2Od\nEc1g/yATGyeyHVCHmdnz7j7c7D7tUBWRnpJk16l2qIqIdIE4efMku061Q1VEJGfVXPqBowdw/L28\neX2AD9uN2regj0ULF815TDtURUS6QNzOjmG7UXf9213svH6ndqjW0IKqiHSFBXctwJkfjwxj9s7Z\nHEbUnbSgKiI9Jc/OjkWskVdwF5GukFdnx7i5/l6j4C4iXSGvzo5FPcVJZ6iKSNfIo7NjUU9x0sxd\nREqtqKc4KbiLSKkV9RSnQgf3qSk47zx48828RyIi3aqopzgVOue+ZQtMTAR/3n133qMRkW5VxFOc\nCjtzn5qCXbtgdjb4U7N3ke5WxFrzPBU2uG/ZEgR2gJmZ4HsR6U5FrTXPUyGDe3XWPh307Wd6WrN3\nkTzEnY0XtdY8T4UM7rWz9irN3kU6K8lsvKi15nmKc4bquWb2lJntN7NXzOy2Bvd+3MxmzOxz6Q4z\nmYcfPj1rr5qehr178xmPSBklmY0XtdY8T3Fm7qeA2939QuBy4FYzu6j+JjNbCPwp8Hi6Q0xuchLc\n539NTuY9MpHySDIbL2qteZ6aBnd3n3L3n1b+fhzYD6wMufWPgIeAt1IdoYj0pCSz8VZrzVVhEy1R\nnbuZDQEfA35S9/hK4AbgauDjKY1NRHrY1jVbGX1kdE5qptFsPGmt+YbvbeCe8Xve6wFfzelXX6vs\nYi+omtlZBDPzje5+rO7yNuAOd59p8hqjZjZuZuOHDx9OPloR6RlZ7vwc2zc2J7BXqcLmtFgnMZlZ\nH/Ao8Li7fzXk+uuAVb49BzgBjLr7d6NeUycxiUirhrYNceDogdBrRT+5Ke5JTE3TMmZmwL3A/rDA\nDuDuH665/z7g0UaBXUSkHY1KJFVhE4iTlrkCuBm42sxeqHxda2brzGxdxuMTkQKLuyBaf9/Zi88O\nvc8wVdhUNJ25u/vfcjrl0pS7f6GdAYlIOVQ3OVUXXKMWRMPuW7RwEX0L+jg5e/K9+wxj3fA6LaZW\nFHKHqoh0v7ibnMLum56ZZtn7lr23WLt88XLOXnw294zfo5LICgV3EclF3E1OUfe98+47TGyc4Fs3\nfot3T73LkXePqOlYDQV3EclF3E1Oze5T07FwCu4ikou4LQea3aemY+EU3EUkF3E3OTW7T03HwsXa\nxJQFbWISkTRs+N4Gvj7+9XmPrx9ez/brtucwomzF3cSkmbuIZCrr5l6P/eNjiR4vi0IfkC0i+Ypb\ny94O5dzDaeYuIpnpRCWLcu7hFNxFJDOdmFXroI9wCu4ikplOzKqzbC3cyxTcRSQzcWfV7S66jqwe\nYWLjBLN3zjKxcaL0gR0U3EUkoSSBOM6surroeuDoAbUPSJHq3EUktvrqFwhm4u2kQaIO3hjsH2Ri\n40SrQy0s1bmLSOqyqH5RKWM2FNyBqSk47zx48828RyLS3bIIxCplzEbT4G5m55rZU2a238xeMbPb\nQu4ZMbOXKl8/MrNLsxluNrZsgYmJ4E8RiZZFIFYpYzbizNxPAbe7+4XA5cCtZnZR3T2vA59y90uA\nLcCOdIcZTysz8Kkp2LULZmeDPzV7F4nWaiButAirUsZsxDlmbwqYqvz9uJntB1YCf1dzz49qnvIs\nsCrlccZSOwO/++74z5mtHJQ+M5PsuSJlUw24m5/czMGjBxnoH2Drmq0NA3GcFgQjq0cUzFOWqFrG\nzIaAZ4CL3f1YxD3/Gfiou9/S6LXSrpaZmoJf+zX453+GxYvhtdfggx+M/5yquM8VkXhUDZOu1Ktl\nzOws4CFgY4PA/jvAl4A7Iq6Pmtm4mY0fPnw47lvHEjYDT/KcqrjPFZF4VA2Tj1jB3cz6CAL7mLvv\nibjnEuAbwPXufiTsHnff4e7D7j68YsWKVsc8TzVvPj0dfD89HS9//vDDp59TNT0Ne/emNjSR9ixb\nBmbzv5Yty3tksakaJh9xqmUMuBfY7+5fjbhnANgD3OzuP0t3iM21OgOfnAT3+V+Tk9mNVSSR48eT\nPd6FVA2Tjzgz9yuAm4GrzeyFyte1ZrbOzNZV7vkTYDmwvXK9o1tPNQMXaUPGvx2oGiYfaj8g0s3M\noq+l9W+3E+/RAWP7xhJV8fSquAuqOolJRHpeJ0586jU92X5A7QJEpFZUz5vbvj9vQ31p9GRw75Z2\nAfqQkcwtXZrs8R6UxgHaUWWVR949UtrWwT0X3LupXUC3fMhIgR07Fl7SdSx0q0nPSauXe6OyyjTP\na+0lPRfcW9mslIVu+pARaUsGvx3EnY2n1UK4UVllWTdL9VRwb3WzUha65UNGpG0p/3aQZDae1u7V\nkdUjLF+8PPRaWTdL9VRw75Z2Ad30ISPSbZLMxtPcvfq1T39Nm6Vq9FRw75bNSll9yGiBVoogyWw8\nzd2r2iw1lzYxtWDVKnjjjfmPr1zZXuuCDRvgL/4C1q1T22HpXUm7QJZl81Fa4m5iUnDvEq20LJaS\nWrYsvLfM0qVdUUXT6BBtSNYLPun7luFDQgdkd1Aa6RQt0EpsXd5MLCo9AqRS9hgmrZLKItHMPQXt\nplN0aIgk0qO9YLI8tKNMB4Jo5l6R9SJlGvXu3VIFJJKlLA/t0IEg8xU+uKe1izTqQyKNdEq3VAGJ\nVKXREqBelod26ECQ+Qod3NPcRRr2IZFWvbsODZFuklX+ut2yx0YfODoQZL5CB/e0FimjPiRaSaeo\nll3alnEzsbRaAtRrpw692QeOatznK+yCapqLlBs2wL33BjPzRYvglluChdNW6t1Vyy7dbsFdC3DC\n48LuG3fnEjDLtGDaTGoLqmZ2rpk9ZWb7zewVM5vXINkCf2Zmr5rZS2b2r1odeFrSWqRslHpJmk5R\nszHpBY3y1HmVF2rBNLk4aZlTwO3ufiFwOXCrmV1Ud8+ngY9UvkaBr6c6yhaktUiZZiWLatmlF4Tl\nr6vSSM+0QgumyTUN7u4+5e4/rfz9OLAfWFl32/XANz3wLPB+M/tQ6qNNIK1FyrQ+JNRsTHpFNX8d\nJY/ZctgHTt+CPn4x/YtUK3qKJNGCqpkNAR8DflJ3aSXwTzXfTzL/A6AnTU7C+vWwYEGQL2/1Q0K1\n7NLUsmXBBqX6r2XLOj6UkdUjDPYPhl7LY7Zcv2C6fPFyzIwj7x7RjtQIsYO7mZ0FPARsdPf6BhZh\nW+bmrciY2aiZjZvZ+OHDh5ONNCdp5clVyy5NdVlbgSTlhVnUxdcbWT3CxMYJZu+c5axFZzE9M/cf\nVF4po24VK7ibWR9BYB9z9z0ht0wC59Z8vwo4VH+Tu+9w92F3H16xYkUr4+24tPLkqmWXXhO3vDCP\nvi5aYG2uaSmkmRlwP/COu2+MuOc64A+Ba4FPAH/m7r/V6HV7obeMer5IR6lnTFe/Z7dIs7fMFcDN\nwNVm9kLl61ozW2dm6yr3PAa8BrwK/CWwodWBd1qjTUXKk4s0l8csWjtSmzuj2Q3u/reE59Rr73Hg\n1rQG1Um1bQXqNxU1ypNrA5JIYKB/IHQWneXCazU1VIb+7a0q7A7VOOIekDE1BVdeCT/8odIxkqGo\nQzigaw7iCBN1OMfaS9fy2D8+puCbMrX8jSHuYmlanSVFGmoUvHM8iKNZJUzYwuvaS9dy/4v36/CM\nHJV25h53sVTH30lHddmiaqMj8xrNwsu84Jk1zdybiLtYWj+737RJXR2lIGJsmmq1Q6RKFfNX2uAe\nZ1NRWMuA3bvh9deVopECiLFpqtUgrV4w+SttcI/aVPTcc8HM/MUX4aMfDWbrtWZmgvvUF0bKoNUg\nrVLF/JU2uEepLp6OjATrWydPht83MwPnnx8e4HUgh7Qs44M4kmo1SOvwjPyVdkE1TKNFVvf51wC+\n8IVgFl9LB3JIqqJKJNstj4y5eDu2b6zlevLa5569+GwA3nn3HQb6B/j1s3+dpyeeZsZnWGgLGb1s\nlO3XbW/95ymJuAuqCu41ak9cqqqevOQ+/xrAwoVBiqdaQaPqGkld0gqauB8GGVfmhFXaNLN+eD3b\nr9ve1gdKnHH18uYnVcskVL94WjU9DTt3BjPx+mswv8ImTu183LSN0jsF0ol2vtX3iNtdMoezWJvZ\n8fyOTBuR5dHkLC8K7hVhpZFV09PBtQ0b4NAhOPPMuderi6txD+SIuylKm6e6VCuBulHATSvYJ93o\ndOxYeFVBSjthWyl7nPGZzA7ohuwO/+5GCu4VYaWRVdWgv2sXfPnL0fXxcWrn4/aH13mrXSzLvus5\n7kSNK27v9lbKHhfawkxr5MtUf6/gXhFVGrl+fZB3hyBQP/hgdH18nNr5JC0PdN6qdJskaY1GZ7FG\nGb1sNNMa+TLV3yu4E53bDkuzzM4Gj4cdulH/AXHoULC4Wl03jpu20XmrMkejHHhtSqcDkqQ1wo7G\nW754eTDskEazaz68hu3Xbc+0Rr5M9fcK7kTnttvt517/uq20PGjlfaVgwnLjrUhhoTRpWqP2aLy3\n//ht3v7jtxnsH8Tnn8LJq++8+t5zsqqRL1P9felLIRuVLq5aBW+8Mf85K1c2Px4v7HWHh+O9Xjvv\nKx3QSglho3a+cV8j7hjqpdguOI2GYAvuWhAa3A1j9s6IqgZ5T2qlkGa208zeMrOXI673m9kjZvai\nmb1iZl9sZcB5aZTbrk2zVFMsU1PxAmzY68Y9R1XnrXa5VkoI62ffWVu6NPXqF0gnrVGmvHee4qRl\n7gOuaXD9VuDv3P1S4Crgf5jZovaHlr0kue0kZYnKmRdcOyWE1TLKrGVUdZNGWiPsA8IwDhw90LD6\nRpJpGtzd/RngnUa3AEsrB2mfVbn3VDrDy1bc3HbSskTlzCVSD5Q6NlObR5/YOBErsNeWT25+cjNr\nL13LYP8gEAT2apqmyJuKOi2NBdU/By4EDgH7gNvcvScSZ3FKF2FusD51Ci64oHGAj/u6Ii3LqZFY\nK8LKJ+9/8X62rtkaurha1E1FnRZrQdXMhoBH3f3ikGufA64A/hNwHvAEcKm7z/sd1cxGgVGAgYGB\nyw4cmL8w023CmokBrF0L992Xy5Ckl8VJySTJyXfZyU1hGi3CHjx6UIurCXWyt8wXgT0eeBV4Hfho\n2I3uvsPdh919eMWKFSm8dfai2hLs3q0cukgcjcontbianTSC+0FgDYCZ/QvgAuC1FF63K0S1JVAO\nXTLRLN1S39em1dfpoEYBvEybijotTinkA8CPgQvMbNLMvmRm68xsXeWWLcC/NrN9wJPAHe7+dnZD\n7qxqWeKhQ/C+9829pgoYSaxRGWV9xU1Yg7JGC7IZNP9KQ6MAXqZNRZ1W+k1McVUP4KhN0VR7vetA\nDslE0pLJHHPszXqk93oP9W6iwzpS9qEPhc/StWtUMtMjwT3sUI4lfUs0A6+R5oebDutI2Q03nO4O\nuWhRMJOvHqg9NASDg0rRSM6yOgSkiTL1SG9FXgeEKLjH0GjH6ZYtcOAAHDyoBVZpUdThH+3o4Gap\nMvVIb0VeH34K7jFE7TjdtCk4gq9q507N3qUFPb5rtVPljHEPCek2eX34KbjHELXj9MEH4eTJuY+F\ntS7QOagllsXZqbVNwbpAJ8oZe/ns07xq+RXcYwjr0njoUNCKoHZGPzs7f/auc1BLLs7ZqY1keMZp\nWjpRztjLef28avlVLdOisNJIgAULYN26oDyyUa94KYl2c+fN/n32QPuBNPR6D/g8qmXOaOnVS2xq\nCq68Ek6cCG9LMDsLe/YEwT2sp7tq4iVVS5dG/3ZglupBHXka6B8I7U/TK20KRlaPdLwsVGmZhKpp\nlhtvPJ2eOfPM4NrChcG/pxtvVE93SUGcFgLV3vJRenyxtkptCpJTcE8grK97/ezcPbj25S+rp7u0\nqQAz7rSoTUFyyrknsGED3HtvMAtftAhuugm+85357YD7+oLrv/zl/NfQjtaSSXJ2ar2Ctf7tRr3Y\nFkE7VFMWlmbZvTs8737yZPD42rXBAmt1N6vOQS2hatqki7o0pq1X6897ubwyDgX3mKI2MoW1A4ag\nTLIa/JVrl8hzV3vZsmWMXWKMfvsP5gbI7/xBZgEyzQ+SXi6vjEPBPaaovu4rVwZf9U6eDII/wLvv\nBjn4RrTZqaQatQDO43WSOH6czWvgxKK5D5/oI5MAmfZMu+htExTcYwrbyFRNszz33OmKmSjf+lbj\nwK3NTiVVP6OvBuPaTU5xdrRG/WaQ8aLswf6IxzMIkGnPtIt+CpSCewqijuKrNTMTPXsPq8KRkmq0\no7VDkqQ+Bo5GPJ5BgEx7pl308koF9xREpWzq/dVfhT8ettlJJA9JUx9bn4Qldf/vL5kmkwCZ9ky7\n6OWVTUshzWwn8HvAW+5+ccQ9VwHbgD7gbXf/VLM37sVSyCRqWw9U1bcgmJqCT34ymKn/6lfR90nG\nosoV89jdGbddQW36Juxai+Me2jYUuhN0sH+QiY0Tcx+sjHVsNWxeE6RoBo4GAX/kpfQXi3UoSCDN\nUsj7gGsavNH7ge3AZ939N4DPxx1kkUVV11Rn5VNTcMEFQS/42s6S9fdJB2SRCsmiG2T92DIYd6LU\nR+UDZmQfTGyD2buCP0cmslnELfpMO22xNjGZ2RDwaNjM3cw2AP/S3f9rkjcu+sx91Sp44435j1c3\nMa1dC9/8ZvTztdmpg7LYANTqa7bbaCzOezSQaOYuuejkJqbzgQ+Y2dNm9ryZ/YcUXrPnNaqumZqC\nsZoUZu2xfdrsVHI5b3Yq+iJjmaQR3M8ALgOuA/4N8N/M7PywG81s1MzGzWz88OHDKbx1b9q06XQN\nPKipmNSoL2nsMKU+iiONtMwm4Ex3/++V7+8FfuDuEbUhgaKnZaJMTcG5584N7hDM3m+5RS2Bc9FN\naZkkr9NIr+9+lUidTMvsBX7bzM4wsyXAJ4D9KbxuIW3ZMj+wQzB737u38+MR8tndGVcrY+iGcUvu\nmh7WYWYPAFcB55jZJHAnQckj7n6Pu+83sx8ALwGzwDfc/eXshtzbHn44/HEtoOYoi3LHqEM0kgbe\nqLF1U/mmdKWmwd3db4pxz1eAr6QyooKrr5RRTXtBpRFgFcClDdqh2mH1lTJxmopJSXVBK4Ju1Kst\nhjtNwb1Dql0fb7ttfs69WVMxEQkUvQd7mhTcO2TLFnj9dXjwwfnXGjUVk4LJeudqwRW9B3uaFNxT\nFNWTvdr1sVHpclRTMSkYpVraUvQe7GlScE9RVE/2OC2BZ2eVmhFppug92NOk4J6SqJ7s9WevAixc\nGGxaqqVmYTJPXvX3XZw6UnuE+BTcUxLVkz3u2avaxCTz5HS6UjenjtQeIb5Y7QeyUKT2A2G92xct\nCgL28uVw5Mj856xcGRzPd+WV8MMfqs69NLJodZC2XhhjiXWy/UDphc3OqzPz2dno7pD1OXodkl0C\n7aZaujhlIt1FwT0FjY7Z+/nP4W/+Zv7jYTl6HZLdBbIOnu2mWro4ZSLdRcE9BfW92z/wgbnXP/e5\n+c+pz9Fv2qRDsruCgqcUhIJ7yp54Ipit16qfvddX0ExPw+7dp3euqnJGctXNXTIlNgX3lP3+74c/\nXjt7j6qgqZ6lqsM7JFd5VelIqhTcU1Y/aw97vFGOvkqzdxFph4J7yqqTnPXrT29Uqp6RWlWfo1+5\ncv7rqO5dElPVjNRQcM9AWE69UZql0WHa0mG9nm9OuvCr0srCahrczWynmb1lZg1PVzKzj5vZjJmF\n1IaUS1ROXWmWHlC2fLOqgworzsz9PuCaRjeY2ULgT4HHUxhTzwvLqSvNUhCtznQ1Q5YOi3PM3jNm\nNtTktj8CHgI+nsKYep7SKQUWZ6YbdTxektcTaVPbOXczWwncANzT/nBECkABW7pAGguq24A73H2m\n2Y1mNmpm42Y2fvjw4RTeWqRkmi3s9srCr2SuaVomhmHg2xZ0kjsHuNbMTrn7d+tvdPcdwA4IukKm\n8N4i5ZL2wu7SpeG/aehDoue1Hdzd/cPVv5vZfcCjYYFdRLpQUauApHlwN7MHgKuAc8xsErgT6ANw\nd+XZpVw005UeEada5qa4L+buX2hrNCLd7tix8GqY48eDx48di/4AEOkg7VAVSapZOWTYRqisqH5e\nIii4i/Qy7TCVCAruIp3Q6z1rpOcouIt0Qtl61tQZ2zfG0LYhFty1gKFtQ4ztG8t7SIWXRp27iFQF\n+z1OW7q0NAG8amzfGJuf3MzBowcZ6B/g2o9cy/0v3s+JkycAOHD0AKOPjAIwsnokz6EWmnmWiz0N\nDA8P+/j4eC7vLdKWJL1jINsF1foPk069b4SxfWOMPjL6XiAHMAxn/lgG+weZ2DjRwdEVg5k97+7D\nze5TWkYkqU5WwzTTZbn8zU9unhPYgdDADnDw6MFODKm0lJYR6WVdlvJJErAH+gcyHIlo5i4iqYkK\n2Mbc9NGSviVsXbO1E0MqLQV3EUnN1jVbWdK3ZM5jS/qWsG54HYP9gxjGYP8gOz6zQ4upGVNaRiQN\n6jkDnK5+qa2W2bpmqwJ5DlQtIyLSQ1QtIyJSYgruBTE1BeedB2++mfdIRKQbKLgXxJYtMDER/Cki\nouBeAFNTsGsXzM4Gf2r2LiIK7gWwZUsQ2AFmZjR7F5EYwd3MdprZW2b2csT1ETN7qfL1IzO7NP1h\nSpTqrH16Ovh+elqzdxGJN3O/D7imwfXXgU+5+yXAFmBHCuOSmGpn7VWavYtI0+Du7s8A7zS4/iN3\n/3nl22eBVSmNTWJ4+OHTs/aq6WnYuzef8YhId0h7h+qXgO+n/JrSwORk3iMQkW6UWnA3s98hCO5X\nNrhnFBgFGBhQRzgRkaykUi1jZpcA3wCud/cjUfe5+w53H3b34RUrVqTx1iIiEqLt4G5mA8Ae4GZ3\n/1n7QxIRkXY1TcuY2QPAVcA5ZjYJ3An0Abj7PcCfAMuB7RYc+XUqTlMbERHJTtPg7u43Nbl+C3BL\naiMSEZG2aYeqSCctWxYcal3/tWxZ3iOTglFwF+mksAM9Gj0u0iIFdxGRAlJwFxEpIAV3EZECUnAX\nESkgBXcRkQJScBfppKVLkz0u0qK0u0KKSCPHjuU9AikJzdxFRApIwV1EpIAU3EVECkjBXUSkgBTc\nRUQKSMFdRKSAFNxFRArI3D2fNzY7DBxI6eXOAd5O6bV6TVl/9rL+3FDen10/d2DQ3ZseQp1bcE+T\nmY2X9Wi/sv7sZf25obw/u37uZJSWEREpIAV3EZECKkpw35H3AHJU1p+9rD83lPdn18+dQCFy7iIi\nMldRZu4iIlKjMMHdzL5iZn9vZi+Z2V+b2fvzHlMnmNnnzewVM5s1s1JUEpjZNWb2D2b2qpltyns8\nnWJmO83sLTN7Oe+xdJKZnWtmT5nZ/sr/67flPaZOMLMzzez/mtmLlZ/7riTPL0xwB54ALnb3S4Cf\nAV/OeTyd8jJwI/BM3gPpBDNbCNwNfBq4CLjJzC7Kd1Qdcx9wTd6DyMEp4HZ3vxC4HLi1JP/NfwVc\n7e6XAr8JXGNml8d9cmGCu7v/L3c/Vfn2WWBVnuPpFHff7+7/kPc4Oui3gFfd/TV3nwa+DVyf85g6\nwt2fAd7Jexyd5u5T7v7Tyt+PA/uBlfmOKnse+EXl277KV+xF0sIE9zr/Efh+3oOQTKwE/qnm+0lK\n8A9dAmY2BHwM+Em+I+kMM1toZi8AbwFPuHvsn7unjtkzs/8NfDDk0mZ331u5ZzPBr3FjnRxbluL8\n3CViIY+p5KsEzOws4CFgo7uX4rxCd58BfrOyhvjXZnaxu8dac+mp4O7uv9voupmtBX4PWOMFqvFs\n9nOXzCRwbs33q4BDOY1FOsTM+ggC+5i778l7PJ3m7v/PzJ4mWHOJFdwLk5Yxs2uAO4DPuvuJvMcj\nmXkO+IiZfdjMFgH/Dng45zFJhszMgHuB/e7+1bzH0ylmtqJa9Wdmi4HfBf4+7vMLE9yBPweWAk+Y\n2Qtmdk/eA+oEM7vBzCaBTwLfM7PH8x5TliqL5n8IPE6wsPY/3f2VfEfVGWb2APBj4AIzmzSzL+U9\npg65Ari6O789AAAAV0lEQVQZuLryb/sFM7s270F1wIeAp8zsJYJJzRPu/mjcJ2uHqohIARVp5i4i\nIhUK7iIiBaTgLiJSQAruIiIFpOAuIlJACu4iIgWk4C4iUkAK7iIiBfT/ASYTzIqDD9y/AAAAAElF\nTkSuQmCC\n", "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "X_lda = X.dot(W)\n", "for label,marker,color in zip(\n", " np.unique(y),('^', 's', 'o'),('blue', 'red', 'green')):\n", " plt.scatter(X_lda[y==label, 0], X_lda[y==label, 1], \n", " color=color, marker=marker)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Next, we are repeating this process for the standarized flower dataset:" ] }, { "cell_type": "code", "execution_count": 28, "metadata": { "collapsed": false }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "EigVals: [ 2.0905e+01 1.4283e-01 -1.3976e-15 1.5997e-17]\n", "\n", "EigVecs: [[-0.1492 -0.0019 0.3845 -0.6214]\n", " [-0.1572 0.3193 -0.181 -0.0241]\n", " [ 0.8635 -0.5155 -0.8081 -0.2471]\n", " [ 0.4554 0.7952 0.4079 0.7431]]\n", "\n", "W: [[-0.1492 -0.0019]\n", " [-0.1572 0.3193]\n", " [ 0.8635 -0.5155]\n", " [ 0.4554 0.7952]]\n" ] } ], "source": [ "S_W, S_B = scatter_within(X_std, y), scatter_between(X_std, y)\n", "eig_vals, eig_vecs = np.linalg.eig(np.linalg.inv(S_W).dot(S_B))\n", "W_std = get_components(eig_vals, eig_vecs, n_comp=2)\n", "print('EigVals: %s\\n\\nEigVecs: %s' % (eig_vals, eig_vecs))\n", "print('\\nW: %s' % W_std)" ] }, { "cell_type": "code", "execution_count": 30, "metadata": { "collapsed": false }, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYYAAAD8CAYAAABzTgP2AAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3X+Q3HWd5/HnOzEDSZGEJERhEzIDLHKy4uE5h2tJ7blG\nJOJKgGLXYNTgSk0Bcrus52mo8Y6SMXXsenXE24PVnCRGMyUqpxJOPQ6inFWWeBnuwPDjgGyYxCHN\nMhAMUgkMybzvj/528u2e77d/fb/d3293vx5VXTP97e+3+8MU+b778+P9/pi7IyIiUjIr6waIiEi+\nKDCIiEgZBQYRESmjwCAiImUUGEREpIwCg4iIlFFgEBGRMgoMIiJSRoFBRETKvCnrBjTjlFNO8YGB\ngaybISLSUR5++OEX3X1prfM6MjAMDAwwNjaWdTNERDqKme2t5zwNJYmISBkFBhERKaPAICIiZRQY\nRESkjAKDiIiUUWAQEZEyCgwiIlJGgUFERMooMIhITxvdNcrAxgFmfWkWAxsHGN01mnWTMteRmc8i\nImkY3TXK0L1DHHrjEAB7D+5l6N4hANaetzbLpmVKPQYR6VnDO4aPBYWSQ28cYnjHcEYtygcFBhHp\nWfsO7mvoeK9QYBCRnrVi4YqGjvcKBQYR6VkbVm5g3px5ZcfmzZnHhpUbMmpRPqQSGMxss5m9YGaP\nxbxuZvafzWy3mf3GzP5F6LV1ZvZM8FiXRntEROqx9ry1bPrIJvoX9mMY/Qv72fSRTT098Qxg7p78\nTcz+BHgV+Ja7vz3i9UuAfw1cArwb+Kq7v9vMFgNjwCDgwMPAu9z95WqfNzg46NqPQUSkMWb2sLsP\n1jovlR6Du/8COFDllNUUg4a7+0PAyWZ2GnAxcL+7HwiCwf3AqjTaJCIizWnXHMMy4Leh5xPBsbjj\nIiKSkXYFBos45lWOz3wDsyEzGzOzscnJyVQbJyK9ISrLWZnPM7Ur83kCOD30fDmwPzj+vorjD0a9\ngbtvAjZBcY6hFY0Uke4VleX8qR99CjNj6ujUsWPKfG5fj2E78MlgddIfAwfdvQDcB3zQzBaZ2SLg\ng8ExEZFURWU5vzH9xrGgUKLM55R6DGb2HYrf/E8xswngZmAOgLt/DfgJxRVJu4FDwKeC1w6Y2Qiw\nM3irW9y92iS2iEhTGslm7vXM51QCg7tfVeN1Bz4T89pmYHMa7RCR3jS6a5ThHcPsO7iPFQtXsGHl\nhhlDQSsWrmDvwb11vZ8yn0VEOlhp7mDvwb04fmyeoHISOSrLec6sOfTN7is7psxnBQYR6XD1VkiN\nynLectkWNq/erMznCqlkPrebMp9FpGTWl2bhEavcDWP65ukMWpRfbc18FhHJStYVUrsxD0KBQUQ6\nWpYVUuud3+g0Cgwi0tGyrJDarTvAac9nEel4a89bm8mEcbfuAKceg4hIk7Ke32gVBQYRkSZ16w5w\nCgwxCgU46yx4/vmsWyIiedWtO8BpjiHGyAiMjxd/3n571q0RkbzKan6jldRjiFAowJYtMD1d/Kle\ng0h+dWMeQdYUGCKMjBSDAsDRo8XnItI+9d7suzWPIGsqiVGhUIAzz4TXXjt+bO5c2LMHTj21JR8p\nIiGVG+pAcUI3aux+YONAZMXU/oX9jN843uqmdhyVxGhSuLdQol6DSPs0kjTWrXkEWVNgqLB9O0yV\nb+jE1BTcc0827RHpNY3c7Ls1jyBrqQQGM1tlZk+Z2W4zWx/x+m1m9kjweNrMfhd67Wjote1ptCeJ\niQlwn/mYmMi6ZSK9oZGbfbfmEWQtcWAws9nA7cCHgHOBq8zs3PA57v437n6+u58P/D3wg9DLh0uv\nufulSdsjIp2tkZt9kjwCrWaKl0YewwXAbnffA2BmdwGrgSdizr+K4p7QIiIzlG7qtbbqDJ/faB7B\n9T++nq+Nfe3YPg6l1Uzhz+9liVclmdmVwCp3vyZ4/gng3e5+Q8S5/cBDwHJ3PxocOwI8AhwBbnX3\nH8V8zhAwBLBixYp37d1b396tIiJho7tG+cQPPhG5uU+3r2Zq56okizgWF23WAHeXgkJgRdDQjwEb\nzeysqAvdfZO7D7r74NKlS5O1WER61vCO4cigAFrNVJJGYJgATg89Xw7sjzl3DfCd8AF33x/83AM8\nCLwzhTaJiESqdvPXaqaiNALDTuBsMzvDzPoo3vxnrC4ys3OARcCvQscWmdkJwe+nAO8lfm5CROSY\nRrKjw+ctnrs48jzDtJopkHjy2d2PmNkNwH3AbGCzuz9uZrcAY+5eChJXAXd5+aTG24Cvm9k0xSB1\nq7srMIhIVZXZ0XGTx1Hn9c3uY86sObwx/cax8wzj2sFrNfEcUEkMEek49ZbCiDtvydwlnNR3EvsO\n7jvWgzhw+EDNFVCdTiUxRKRr1ZsdHXfegcMHGL9xnG9f8W0OHznMS4dfUhG+EAUGEek49WZH1zqv\nkbpMvUSBQUQ6Tr3Z0bXOUxG+aAoMItJx6i2FUes8FeGLpslnEelZ1//4ev5h7B9mHL9u8Dru+PAd\nGbSotTT5LCIdr9WF7n7yzE8aOt4r0iiiJyKSunpzFZLQHEM09RhEJJfasWJIcwzRFBhEJJfa8W1e\nG/1EU2AQkVxqx7f5JBv9dDMFBhHJpXq+zacxOb32vLWM3zjO9M3TjN843vNBARQYRKSNGrmR1/o2\nX5qc3ntwr8pZpEx5DCLSFpWrjKDYA2h26KbeQnpynPIYRCRX0l5lpKWmraPAkFChAGedBc8/n3VL\nRPIt7Ru5lpq2TiqBwcxWmdlTZrbbzNZHvH61mU2a2SPB45rQa+vM7JngsS6N9rTTyAiMjxd/iki8\ntG/kWmraOokDg5nNBm4HPgScC1xlZudGnPpddz8/eHwjuHYxcDPwbuAC4GYzW5S0TY1q9lt/oQBb\ntsD0dPGneg0i8Zq5kVebrNZS09ZJo8dwAbDb3fe4+xRwF7C6zmsvBu539wPu/jJwP7AqhTY1pNlv\n/SMjxaAAcPSoeg0i1TR6I69n1ZGWmrZG4lVJZnYlsMrdrwmefwJ4t7vfEDrnauA/AJPA08DfuPtv\nzexzwInu/uXgvH8HHHb3/1jtM9NclVQowJlnwmuvwdy5sGcPnHpqY9eVNHK9iFSnVUfpa+eqJIs4\nVhlt7gUG3P0dwAPA1gauLZ5oNmRmY2Y2Njk52XRjKzX7rT98XYl6DSLp0aqj7KQRGCaA00PPlwP7\nwye4+0vu/nrw9L8C76r32tB7bHL3QXcfXLp0aQrNPj5HMDVVfD41Vf9cwfbtx68rmZqCe+5JpWki\nySxYAGYzHwsWZN2yumnVUXbSCAw7gbPN7Awz6wPWANvDJ5jZaaGnlwJPBr/fB3zQzBYFk84fDI61\nRZJv/RMT4D7zMTHRmraKNOT3v2/seA5p1VF2EgcGdz8C3EDxhv4k8D13f9zMbjGzS4PT/srMHjez\nR4G/Aq4Orj0AjFAMLjuBW4JjbaFv/SIJtbBnolVH2VFJDJFuZFHTd4E0/82363NabHTXKMM7htl3\ncB8rFq5gw8oNXRmA6p181g5uItLT2rFTXKfpuZIYKmEhImHt2Cmu0/RcYMhLCQsFKGmp+fMbO96h\n0tiPISpXotrxXtBTgSFPJSzyEqCkS73ySvSyuVdeybplqUlrP4bZNruh472gpwJDXkpY5ClAiSTS\ngp5Jvb2AtIaAjvrRho73gp4JDEmS2dKWlwAlkljKPZNGegFpZUb3L+xv6Hgv6JnAkJcSFnkKUCJ5\n00gvIK3MaCXSzdQzgSEvyWytClCazJZu0EgvIK0buhLpZlKCW5stXw7PPTfz+LJlycppXH89fP3r\ncO21cPvtzb+PSJYarajaK4lpaak3wU2BoQs0WzpcetCCBdH1kubPz8WKpcpkMyj2AjZ9ZBNAy4JA\nrwSYdpbdlgTSGALSZLbULefF9eKGdYBUlqZGSWvZazdRjyFjSYeAtGGQNKRDaxu1ctOeXtoQSD2G\nFLR6QjeNfIa8rLYSaaVWbtqjDYFmUmCoIq3s5LgAk8YQUF5WW4mUpFGmolIrN+3RhkAzKTDESDM7\nOSrApJXPoA2DJE9aNV6fdGlqtWClPIaZUgkMZrbKzJ4ys91mtj7i9c+a2RNm9hsz22Fm/aHXjprZ\nI8Fje+W1WUlrQjcuwDQzBKRcBUmsxcX1WlWpNEmuQa1gpTyGmRJPPpvZbOBp4CKKezjvBK5y9ydC\n5/wp8Gt3P2Rm1wHvc/ePBq+96u4nNfKZrZ58TnNC9/rr4c47iz2Cvj645priJHMz+QzKVZC8m/Wl\nWTjR95RtV2zL5GbbS5PLtbRz8vkCYLe773H3KeAuYHX4BHf/ubuXvkY8BCxP4XNbJq0J3WrDRY0O\nAanwnnSCauPyWS0B1eRy49IIDMuA34aeTwTH4nwa+Gno+YlmNmZmD5nZZSm0J7G0JnTTXDGkXAXp\nBFHj9SVZbX6jyeXGpREYohZGR/YlzezjwCDwldDhFUHX5mPARjM7K+baoSCAjE1OTiZtc1VpTeim\nFWBUeE86RWm8Pk4W39KjgtWcWXN4derVVFdOdZM0AsMEcHro+XJgf+VJZvYBYBi41N1fLx139/3B\nzz3Ag8A7oz7E3Te5+6C7Dy5dujSFZrfexARcdx3MmlWcH2g2wChXQWpasKCYvFb5WLCg7U1Ze97a\n2JLVWXxLr5xcXjJ3CWbGS4dfUqZzjDQCw07gbDM7w8z6gDVA2eoiM3sn8HWKQeGF0PFFZnZC8Psp\nwHuBJ+gSac0LKFdBaspZqYtGloC2Iu+h0trz1jJ+4zjTN09zUt9JTB0t/wfV63s8V0ocGNz9CHAD\ncB/wJPA9d3/czG4xs0uD074CnAR8v2JZ6tuAMTN7FPg5cGt4NVOnS2teQLkK0mnqXQKaRZ0iTUbX\nplpJLaIaRtJWqoGU68/MC9VKaoNqCWeaFxCpLYtv78p0rk2BIYFqtZQ0LyBSWxZLSZXpXJuGkppU\n7+Y4hQJceCH88pcaQpIWituAB3KzCU+UqI15+mb3Mb9vPgcOH+jqTXOyoKGkFqt3YjmtCq0iVVW7\n8We0Mqme1UZRS0ndXUtJM6YeQxPqnVjWlpvSVjmagK62RWe1b/+9PDHcDuoxtFC9E8uVvYr161Ud\nVbpAHcl0zVZZ1VLSfFBgaEI9E8tRZSy2bYNnn9WwknS4OpLpmr3Bq65RPigwNCEu4WznTujvh4EB\nuOmm6F6Fu+ocSfdr9gavpaT5oMCQopER2LcP9u6Fu++e2asoOXoU3vrWmcFBG/FIIi3ehKcRzd7g\ntZQ0HzT5nJJCAc44A14PygOeeGJx2OjUU6MnqwGuvrrYeyjRRjySurhlrEmWsNY5yT26a5ThHcPs\nO7ivqWWn4esXz10McGwJ6x8u/kMeHH+Qo36U2TaboXcNcceH72juv6eH1Dv5rMCQktJNvTR8NGvW\n8Rt8eBe3sNmzi8NSlcFDK5gkNY2uVKonkLRh9VPUqqZarhu8jjs+fEfigFSrXa1673bQqqQ2KhRg\n8+byOYXp6WIw6O+HH/4welgpvJKp3ryIeoabNCTVZdpRUrv0GfVUaW3DkFXUqqZaNj28qaVF+bIo\n+JcVBYYUjIzAG2/MPP7668U5hyuuKH6R2r+/OMQUtmULPPpo/Rvx1JMwp6S6HGvmJl/tZp1WoGgk\nCe6VV6JXX6SYXd3M8tSjfrTpZbL1aOV7540CQwq2b5+5AimsdJOPy39Yu7a+vIh69nfQ3tA518p9\nEzLKcG5EvXsvNLM8dbbNbmkeRC/lWCgwpCBq+ep110FfX/H10k0+Lv/hySfrK7hXz3CT9oaWvGpk\nKKba3tFxht411NI8iF7KsVBgaIG4PZrHxqJ74KX8hvBj/3444YTj3/jr2fdZe0PLDNXG/cPDUG3Q\nyFBMVA2lJXOXFJsdsc38yjNWcseH72hpHkQv5VikEhjMbJWZPWVmu81sfcTrJ5jZd4PXf21mA6HX\nbgqOP2VmF6fRnnaKmuhNYy+GynmCet5Te0DIDJXzAc1KYWK50aGY8HacL37+RV78/Iv0L+zHmfnf\nsfvA7mPXtCoPopdyLBIvVzWz2cDTwEXABMU9oK8Kb9FpZtcD73D3a81sDXC5u3/UzM4FvgNcAPwB\n8ADwVnc/Wu0z87RcNSr3YPlyeO65mecuW1bfdpxRS1cHB2u/Z9LPlTZoZqlntZLa9b5HPZ8fJcWS\n3WkUyJv1pVmRgcEwpm+uMtEnQHuXq14A7Hb3Pe4+BdwFrK44ZzWwNfj9bmClmVlw/C53f93dnwV2\nB+/XEeImeivnHPbvL97o641lUfME9ez7rL2hO0AzSz3T+tZfr/nzW7LSKI2hmF4a589SGoFhGfDb\n0POJ4FjkOe5+BDgILKnzWgDMbMjMxsxsbHJyMoVmJ9eKPRk0T9Dlkiz1LC11bbUWrW5KYygmblL6\n1alXuzKfICtpBIao/1Mrv9bEnVPPtcWD7pvcfdDdB5cuXdpgE9NX7w280eWjmieQWB2wHLWW8LzB\n+I3jdQWF8BLX4R3DrPvn645NRJe8dPilrk02y0IagWECOD30fDmwP+4cM3sTsBA4UOe1udTMngxH\njsA551QPDtorWloqg4J6SUQtcd366NbIc7s12SwLaQSGncDZZnaGmfUBa4DtFedsB9YFv18J/MyL\ns97bgTXBqqUzgLOB/51Cm1qumT0Z3nijOGKwfsa6reM0TyAtFR7K6gBxS1xfOvxS5PndmGyWhcSB\nIZgzuAG4D3gS+J67P25mt5jZpcFpdwJLzGw38FlgfXDt48D3gCeA/wF8ptaKpLyo5wYe1auA4oY9\nmjMQqa3RG70modPxpjTexN1/Avyk4ti/D/3+GvDnMdduALovQ4ToXgUcH3JSaW1JVbVhokaWvOZo\nuGnFwhWRS1yXzF3C4SOHZ+wp3Y3JZllQ5nMLlXoVpSzmMK00koZVW+oaXtkUVaivWlBoYTG8pOKW\nuH71Q1/tmWSzLKTSY5DqoqqvqtcgDav3ht1hq5eq7XFQ+lnrdUmXNuppg9NOi+4dKCNZWqLRXIcM\n7wFRG/LMmzNP3/4DaW8MpI16cuTyy49XWu3rK5bRcIedO2FgoLiZj4aVJDOt3ACohl7a46BRWW4M\npMDQYtUS4UZGYO/e4mY+SmCTpkTNJyTVxqGoXtrjoFFZBk0FhhaLS4Rbv764HWjJ5s3qNUgTOmw+\noVK7ah/Vu0FQnmQZNBUYWiwuEe7uu8snpKemynsN2re5x7Vin+dwcbyczC22Y4+DTt2rOcuCgQoM\nLRaVCLd/f7E8RrgnMT1d3mvQvs09rt59nqvJ8TLUknbscdCp8xhZbgykVUkZKO3hUDnENGtWcV+H\nL35x5n4Mp56aTVslI2nMFSTZm6ED7wtxOnkPh6xWJSmPoY0KBbjwQjh0KLpUxvQ0/OAHxX+TleW8\nle8gqZs/P75nsmBBLnsYzYjLnu6E8hlrz1ubybJdDSW1UWl46IorisNJJ55YPD53LqxbV+wxXHyx\n9mOQFNRT1qLajb/DJ7XDemmv5rQoMLRJ5b4MN91U3ivYtq34fNu24vMw7ccgDeuSb/tp6KW9mtOi\nOYY2uf56uPPOYg9gzpxiEKgMANUoS7rHNFL0Lkq9/657ZJ6hFdIe/28HZT7nSNS+DNWCQnhoqZQl\nraDQY8L7JuSo2mnaOjG/ADp3CWy9FBjaIG5fhjhHjhwfWtL8gsTuE92pghyN0XcYQ3d9vG031zSD\nUKcuga1XosBgZovN7H4zeyb4uSjinPPN7Fdm9riZ/cbMPhp67Ztm9qyZPRI8zk/SnryK25dh2bLi\no1K4R3H4cHE+oholw/WoamW42/kejQqGyIZXwqG+8pdadXNN+xt+t5fySNpjWA/scPezgR3B80qH\ngE+6+x8Bq4CNZnZy6PV/6+7nB49HErYnl6rt9rZz5/HVSXG+/e3qN30lw/WouOGmcBJcrUzpuN5I\nGyav9y2MOd6Cm2va3/CzzEpuh6SBYTVQ2pl7K3BZ5Qnu/rS7PxP8vh94AVia8HO7Rj3DTEePxvca\nKlc7qdfQo6plSrdRI8M1Kw7GHG/BzTXtb/jdvgQ2aWB4i7sXAIKfb652spldAPQB/xg6vCEYYrrN\nzE6IuRQzGzKzMTMbm5ycTNjs/IgbZqr0/e9HHw8HFi1rlSw1OlyzYQfMq/h/v1U317S/4Xf7Etia\ny1XN7AEgqiDDMLDV3U8Onfuyu8+YZwheOw14EFjn7g+Fjj1PMVhsAv7R3W+p1ehOXK7aiELheEmM\nksrSGIUCvOc9xR7C66/HnyctFLekdP789ucR1FtCozTk1IJ2D2wciMww7l/Yz/iN48cPhNo6el5x\nrmHfwmIPYsNfbmvJzVUbAhWlVhLD3T9Q5UP+ycxOc/dCcJN/Iea8BcCPgS+WgkLw3oXg19fNbAvw\nuVrt6QVxpbpLpTEKBTjnnOK/7Vmz4s+TFmvV8E0rA061tiVsd93DNaFSHGt3FR/Hjt/Wmpt0rS1C\npVyiBDcz+wrwkrvfambrgcXu/vmKc/qAnwL3uvvGitdKQcWA24DX3D1qArtMt/cYli+H556bebyU\n5LZuHXzrW/HXKxmuTVqVHNbM+6ZRdK/a+9eh7h6DZKZdCW63AheZ2TPARcFzzGzQzL4RnPMXwJ8A\nV0csSx01s13ALuAU4MsJ29MVqq1iKhRgNDRkG94qNHye9JgcJMF1+4RsL1FJjA4T1VvQvEJG8tRj\naOQ9qkl4P+jEMhG9pN4egwJDBykU4PTTZ5bT6OuDa67RvELbKTBIh1GtpC40MhJdY2lqCu65p/3t\n6XlZZA3XK89tk9zTRj0d5Ic/jD6uyeaMtGpJatwGOo3c1Ku1rdqqJxHUY+gol19eHCGYPbv4fO7c\n4vCSgkKXSaNMRVCobsajtDNbRmUwpDMoMHSIUukL98YK7EmPykmJjLzp1DLf7abAkHOlyqnhHd/C\nahXYE5Gibt9DIU0KDDk3MgLPPlvcnyGqplK1AnvSZaoND0lN3b6HQpoUGHIgbj+FqOGjKHEF9qTL\naHgokW7fQyFNCgw5ELefQr07v01PazhJpJZu30MhTQoMGYvbT6Fyn2g4vhd0X8WuVyq3LTNkkceQ\n86EuleyonwJDxuL2U4irsHr33TPnGpTgJjNksSQ150Nd3b6HQppUEiNDcfsubNkCa9ZEX1NKZisU\n4MIL4Ze/VI2kntGqEhxpyXv7RCUxOkFcr+CTnyz+vmhRdIXV0rWleYm4yWvpMkmGh3I+zCP5osCQ\noahtPaemjh97+WX42c9mXlc5L3HTTdGT19JG7bjxJhkeyvkwj+SLAkOGovZdWFSxMeqVV868rnJe\nYtu2mZPX0ma68UoXSRQYzGyxmd1vZs8EP+P2ez4a2qRne+j4GWb26+D67wa7vfWs++8v9hLCKnsN\nlauVpqaO5zhodZJkShVdu0bSHsN6YIe7nw3sCJ5HOezu5wePS0PH/xa4Lbj+ZeDTCdvT0T760ejj\n4V5DtdyGqSn1GiRDKs7XNZIGhtXA1uD3rcBl9V4Y7PP8fuDuZq7vRpW9hajjUfMSYeo1iEhSSQPD\nW9y9ABD8fHPMeSea2ZiZPWRmpZv/EuB37n4keD4BLEvYno7mDtdddzyBLbyfc0l4XmJZxF9LOQ0S\nqdpwjlYoSYWagcHMHjCzxyIeqxv4nBXB2tmPARvN7CwgatFz7GJnMxsKgsvY5ORkAx/dOaLmD6oN\nDUVNXoeXtEob5X18PcnKpThaAtu1agYGd/+Au7894nEP8E9mdhpA8POFmPfYH/zcAzwIvBN4ETjZ\nzEq7yC0H9ldpxyZ3H3T3waVLlzbwn9g54vIaNDTUAXpxfF0rsbpW0qGk7cC64Pd1wIxBDDNbZGYn\nBL+fArwXeMKLKdc/B66sdn0victr0NBQl2jmG7a+lUsGkgaGW4GLzOwZ4KLgOWY2aGbfCM55GzBm\nZo9SDAS3uvsTwWtfAD5rZrspzjncmbA9HU1DQ12u1jfsqCCgb+WSAdVKEmmXWrWEqr0ed01an9/M\ne6o2UsdRrSQRKVdrIjwvE+WSOQUGkU7V6DxD3AR5sxPleV+JJU17U+1TRCSXsp5n6OYVVz1OPQaR\ndtE3bOkQCgwi7fLKK9FB4Pe/Lw4LKUBITigwiLRTteWncXMAraIcCYmhwCDSq5QjITEUGETyTnMT\n0mYKDCJ514t1mEJGd40ysHGAWV+axcDGAUZ3jWbdpK6n5aoieVGZSTx/fs/c/EtGd40yvGOYfQf3\nsWLhCi45+xK2PrqVQ28cAmDvwb0M3TsEwNrz1mbZ1K6mkhgi7bRgQWNj+K3895mzkhaju0YZunfo\nWBAAMAyPqMbfv7Cf8RvH29i67qCSGCJ5FDUslJWczV0M7xguCwpAZFAA2HdwXzua1LM0lCTSq3I2\nTNXIzX7FwhUtbImoxyAiuRB3s7eKzR7nzZnHhpUb2tGknqXAICK5sGHlBubNmVd2bN6ceVw7eC39\nC/sxjP6F/Wz6yCZNPLdYoqEkM1sMfBcYAMaBv3D3lyvO+VPgttChfwascfcfmdk3gX8FHAxeu9rd\nH0nSJpGOM39+9IR0j+UplG724VVJG1ZuUBDIQKJVSWb2d8ABd7/VzNYDi9z9C1XOXwzsBpa7+6Eg\nMPx3d7+7kc/VqiQRkca1a1XSamBr8PtW4LIa518J/NTdD9U4T0REMpI0MLzF3QsAwc831zh/DfCd\nimMbzOw3ZnabmZ2QsD3SoEIBzjoLnn8+65aISF7UDAxm9oCZPRbxWN3IB5nZacB5wH2hwzdRnHP4\nl8BioNow1JCZjZnZ2OTkZCMfLVWMjMD4ePGniAgkn2N4CnifuxeCG/+D7n5OzLl/DfyRuw/FvP4+\n4HPu/me1PldzDOkoFODMM+G112DuXNizB049NetWiUirtGuOYTuwLvh9HXBPlXOvomIYKQgmmJlR\nnJ94LGF7pAEjIzA9Xfz96FH1GkSkKGlguBW4yMyeAS4KnmNmg2b2jdJJZjYAnA78r4rrR81sF7AL\nOAX4csL2SJ0KBdiyBaamis+nporPNdcgIonyGNz9JWBlxPEx4JrQ83FgWcR570/y+dK8cG+hpNRr\nuP32bNqwg6sJAAAECElEQVQkIvmgzOcetX378d5CydQU3FNtMFBEeoKK6PWoiYmsWyAieaUeg4iI\nlFFgEBGRMgoMIiJSRoFBRETKKDCIiEgZBQaRTrFgAZjNfCxYkHXLpMsoMIh0ggULojfzgfjjIk1S\nYBDpBLr5SxspMIiISBkFBhERKaPAICIiZRQYRDrd/PlZt0C6jAKDSCeIu/nPnw+vvNLetkjXU3VV\nkU6gm7+0UaIeg5n9uZk9bmbTZha7j6iZrTKzp8xst5mtDx0/w8x+bWbPmNl3zawvSXtERCS5pENJ\njwFXAL+IO8HMZgO3Ax8CzgWuMrNzg5f/FrjN3c8GXgY+nbA9IiKSUKLA4O5PuvtTNU67ANjt7nvc\nfQq4C1htZga8H7g7OG8rcFmS9oiISHLtmHxeBvw29HwiOLYE+J27H6k4HsnMhsxszMzGJicnW9ZY\nEZFeV3Py2cweAE6NeGnY3evZIdgijnmV45HcfROwCWBwcDD2PBERSaZmYHD3DyT8jAng9NDz5cB+\n4EXgZDN7U9BrKB0XEZEMtWMoaSdwdrACqQ9YA2x3dwd+DlwZnLcOqKcHIiIiLWTF+3OTF5tdDvw9\nsBT4HfCIu19sZn8AfMPdLwnOuwTYCMwGNrv7huD4mRQnoxcD/xf4uLu/XsfnTgJ7m254c06h2MuR\naPr7xNPfpjr9feKl/bfpd/eltU5KFBh6iZmNuXtsrkav098nnv421envEy+rv41KYoiISBkFBhER\nKaPAUL9NWTcg5/T3iae/TXX6+8TL5G+jOQYRESmjHoOIiJRRYGiAmX3FzP6fmf3GzH5oZidn3aa8\nqLfSbq+Jqyzc68xss5m9YGaPZd2WvDGz083s52b2ZPBv6q/b3QYFhsbcD7zd3d8BPA3clHF78qRm\npd1eU6OycK/7JrAq60bk1BHg37j724A/Bj7T7v9vFBga4O7/M1T07yGKZTyEuivt9prIysIZtykX\n3P0XwIGs25FH7l5w9/8T/P574EmqFBhtBQWG5v0l8NOsGyG5FldZWKQuZjYAvBP4dTs/V1t7Vqin\nmqyZDVPs7o22s21ZS6HSbq9pqIKwSJiZnQT8N+BGd2/r3q4KDBVqVZM1s3XAnwErvcfW+qZQabfX\nxFUWFqnKzOZQDAqj7v6Ddn++hpIaYGargC8Al7r7oazbI7kXWVk44zZJzgW7W94JPOnu/ymLNigw\nNOa/APOB+83sETP7WtYNygszu9zMJoD3AD82s/uyblPWgoUKNwD3UZxA/J67P55tq/LBzL4D/Ao4\nx8wmzEz7vR/3XuATwPuD+8wjQYXqtlHms4iIlFGPQUREyigwiIhIGQUGEREpo8AgIiJlFBhERKSM\nAoOIiJRRYBARkTIKDCIiUub/A3CPxETOq8YoAAAAAElFTkSuQmCC\n", "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "X_std_lda = X_std.dot(W_std)\n", "X_std_lda[:, 1] = X_std_lda[:, 1]\n", "for label,marker,color in zip(\n", " np.unique(y),('^', 's', 'o'),('blue', 'red', 'green')):\n", " plt.scatter(X_std_lda[y==label, 0], X_std_lda[y==label, 1], \n", " color=color, marker=marker)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "As we can see, the eigenvalues are excactly the same whether we scaled our data or not (note that since $W$ has a rank of 2, the two lowest eigenvalues in this 4-dimensional dataset should effectively be 0). Furthermore, we see that the projections look identical except for the different scaling of the component axes and that it is mirrored in this case." ] } ], "metadata": { "kernelspec": { "display_name": "Python 3", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.5.2" } }, "nbformat": 4, "nbformat_minor": 0 }