{ "cells": [ { "cell_type": "code", "execution_count": 1, "metadata": {}, "outputs": [], "source": [ "%load_ext watermark" ] }, { "cell_type": "code", "execution_count": 2, "metadata": {}, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "/Users/sebastian/miniconda3/lib/python3.6/site-packages/watermark/watermark.py:155: DeprecationWarning: Importing scikit-learn as `scikit-learn` has been depracated and will not be supported anymore in v1.7.0. Please use the package name `sklearn` instead.\n", " DeprecationWarning)\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Sebastian Raschka 2018-09-13 \n", "\n", "CPython 3.6.6\n", "IPython 6.5.0\n", "\n", "scikit-learn 0.19.1\n", "matplotlib 2.2.2\n", "numpy 1.15.0\n", "pandas 0.23.3\n" ] } ], "source": [ "%watermark -v -d -a 'Sebastian Raschka' -p scikit-learn,matplotlib,numpy,pandas" ] }, { "cell_type": "code", "execution_count": 3, "metadata": {}, "outputs": [], "source": [ "%matplotlib inline" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "
\n", "
" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "# Principal Component Analysis in 3 Simple Steps" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Principal Component Analysis (PCA) is a simple yet popular and useful linear transformation technique that is used in numerous applications, such as stock market predictions, the analysis of gene expression data, and many more. In this tutorial, we will see that PCA is not just a \"black box\", and we are going to unravel its internals in 3 basic steps." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "This article just got a complete overhaul, the original version is still available at [principal_component_analysis_old.ipynb](http://nbviewer.ipython.org/github/rasbt/pattern_classification/blob/master/dimensionality_reduction/projection/principal_component_analysis.ipynb)." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "
\n", "
" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "
" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Sections" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "- [Introduction](#Introduction)\n", " - [PCA Vs. LDA](#PCA-Vs.-LDA)\n", " - [PCA and Dimensionality Reduction](#PCA-and-Dimensionality-Reduction)\n", " - [A Summary of the PCA Approach](#A-Summary-of-the-PCA-Approach)\n", "- [Preparing the Iris Dataset](#Preparing-the-Iris-Dataset)\n", " - [About Iris](#About-Iris)\n", " - [Loading the Dataset](#Loading-the-Dataset)\n", " - [Exploratory Visualization](#Exploratory-Visualization)\n", " - [Standardizing](#Standardizing)\n", "- [1 - Eigendecomposition - Computing Eigenvectors and Eigenvalues](#1---Eigendecomposition---Computing-Eigenvectors-and-Eigenvalues)\n", " - [Covariance Matrix](#Covariance-Matrix)\n", " - [Correlation Matrix](#Correlation-Matrix)\n", " - [Singular Vector Decomposition](#Singular-Vector-Decomposition)\n", "- [2 - Selecting Principal Components](#2---Selecting-Principal-Components)\n", " - [Sorting Eigenpairs](#Sorting-Eigenpairs)\n", " - [Explained Variance](#Explained-Variance)\n", " - [Projection Matrix](#Projection-Matrix)\n", "- [3 - Projection Onto the New Feature Space](#3---Selecting-Principal-Components)\n", "- [Shortcut - PCA in scikit-learn](#Shortcut---PCA-in-scikit-learn)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "
\n", "
" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "
" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Introduction" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "[[back to top](#Sections)]" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The sheer size of data in the modern age is not only a challenge for computer hardware but also a main bottleneck for the performance of many machine learning algorithms. The main goal of a PCA analysis is to identify patterns in data; PCA aims to detect the correlation between variables. If a strong correlation between variables exists, the attempt to reduce the dimensionality only makes sense. In a nutshell, this is what PCA is all about: Finding the directions of maximum variance in high-dimensional data and project it onto a smaller dimensional subspace while retaining most of the information." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "
\n", "
" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### PCA Vs. LDA" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "[[back to top](#Sections)]" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Both Linear Discriminant Analysis (LDA) and PCA are linear transformation methods. PCA yields the directions (principal components) that maximize the variance of the data, whereas LDA also aims to find the directions that maximize the separation (or discrimination) between different classes, which can be useful in pattern classification problem (PCA \"ignores\" class labels). \n", "***In other words, PCA projects the entire dataset onto a different feature (sub)space, and LDA tries to determine a suitable feature (sub)space in order to distinguish between patterns that belong to different classes.*** " ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "
\n", "
" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### PCA and Dimensionality Reduction" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "[[back to top](#Sections)]" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Often, the desired goal is to reduce the dimensions of a $d$-dimensional dataset by projecting it onto a $(k)$-dimensional subspace (where $k\\;<\\;d$) in order to increase the computational efficiency while retaining most of the information. An important question is \"what is the size of $k$ that represents the data 'well'?\"\n", "\n", "Later, we will compute eigenvectors (the principal components) of a dataset and collect them in a projection matrix. Each of those eigenvectors is associated with an eigenvalue which can be interpreted as the \"length\" or \"magnitude\" of the corresponding eigenvector. If some eigenvalues have a significantly larger magnitude than others that the reduction of the dataset via PCA onto a smaller dimensional subspace by dropping the \"less informative\" eigenpairs is reasonable.\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "
\n", "
" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### A Summary of the PCA Approach" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "[[back to top](#Sections)]" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "- Standardize the data.\n", "- Obtain the Eigenvectors and Eigenvalues from the covariance matrix or correlation matrix, or perform Singular Vector Decomposition.\n", "- Sort eigenvalues in descending order and choose the $k$ eigenvectors that correspond to the $k$ largest eigenvalues where $k$ is the number of dimensions of the new feature subspace ($k \\le d$)/.\n", "- Construct the projection matrix $\\mathbf{W}$ from the selected $k$ eigenvectors.\n", "- Transform the original dataset $\\mathbf{X}$ via $\\mathbf{W}$ to obtain a $k$-dimensional feature subspace $\\mathbf{Y}$." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "
\n", "
" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Preparing the Iris Dataset" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "[[back to top](#Sections)]" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "
\n", "
" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### About Iris" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "[[back to top](#Sections)]" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "For the following tutorial, we will be working with the famous \"Iris\" dataset that has been deposited on the UCI machine learning repository \n", "([https://archive.ics.uci.edu/ml/datasets/Iris](https://archive.ics.uci.edu/ml/datasets/Iris)).\n", "\n", "The iris dataset contains measurements for 150 iris flowers from three different species.\n", "\n", "The three classes in the Iris dataset are:\n", "\n", "1. Iris-setosa (n=50)\n", "2. Iris-versicolor (n=50)\n", "3. Iris-virginica (n=50)\n", "\n", "And the four features of in Iris dataset are:\n", "\n", "1. sepal length in cm\n", "2. sepal width in cm\n", "3. petal length in cm\n", "4. petal width in cm\n", "\n", "\"Iris\"" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "
\n", "
" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Loading the Dataset" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "[[back to top](#Sections)]" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "In order to load the Iris data directly from the UCI repository, we are going to use the superb [pandas](http://pandas.pydata.org) library. If you haven't used pandas yet, I want encourage you to check out the [pandas tutorials](http://pandas.pydata.org/pandas-docs/stable/tutorials.html). If I had to name one Python library that makes working with data a wonderfully simple task, this would definitely be pandas!" ] }, { "cell_type": "code", "execution_count": 4, "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
sepal_lensepal_widpetal_lenpetal_widclass
1456.73.05.22.3Iris-virginica
1466.32.55.01.9Iris-virginica
1476.53.05.22.0Iris-virginica
1486.23.45.42.3Iris-virginica
1495.93.05.11.8Iris-virginica
\n", "
" ], "text/plain": [ " sepal_len sepal_wid petal_len petal_wid class\n", "145 6.7 3.0 5.2 2.3 Iris-virginica\n", "146 6.3 2.5 5.0 1.9 Iris-virginica\n", "147 6.5 3.0 5.2 2.0 Iris-virginica\n", "148 6.2 3.4 5.4 2.3 Iris-virginica\n", "149 5.9 3.0 5.1 1.8 Iris-virginica" ] }, "execution_count": 4, "metadata": {}, "output_type": "execute_result" } ], "source": [ "import pandas as pd\n", "\n", "df = pd.read_csv(\n", " filepath_or_buffer='https://archive.ics.uci.edu/ml/machine-learning-databases/iris/iris.data', \n", " header=None, \n", " sep=',')\n", "\n", "df.columns=['sepal_len', 'sepal_wid', 'petal_len', 'petal_wid', 'class']\n", "df.dropna(how=\"all\", inplace=True) # drops the empty line at file-end\n", "\n", "df.tail()" ] }, { "cell_type": "code", "execution_count": 5, "metadata": {}, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "/Users/sebastian/miniconda3/lib/python3.6/site-packages/ipykernel_launcher.py:3: DeprecationWarning: \n", ".ix is deprecated. Please use\n", ".loc for label based indexing or\n", ".iloc for positional indexing\n", "\n", "See the documentation here:\n", "http://pandas.pydata.org/pandas-docs/stable/indexing.html#ix-indexer-is-deprecated\n", " This is separate from the ipykernel package so we can avoid doing imports until\n", "/Users/sebastian/miniconda3/lib/python3.6/site-packages/pandas/core/indexing.py:890: DeprecationWarning: \n", ".ix is deprecated. Please use\n", ".loc for label based indexing or\n", ".iloc for positional indexing\n", "\n", "See the documentation here:\n", "http://pandas.pydata.org/pandas-docs/stable/indexing.html#ix-indexer-is-deprecated\n", " retval = getattr(retval, self.name)._getitem_axis(key, axis=i)\n", "/Users/sebastian/miniconda3/lib/python3.6/site-packages/ipykernel_launcher.py:4: DeprecationWarning: \n", ".ix is deprecated. Please use\n", ".loc for label based indexing or\n", ".iloc for positional indexing\n", "\n", "See the documentation here:\n", "http://pandas.pydata.org/pandas-docs/stable/indexing.html#ix-indexer-is-deprecated\n", " after removing the cwd from sys.path.\n" ] } ], "source": [ "# split data table into data X and class labels y\n", "\n", "X = df.ix[:,0:4].values\n", "y = df.ix[:,4].values" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Our iris dataset is now stored in form of a $150 \\times 4$ matrix where the columns are the different features, and every row represents a separate flower sample.\n", "Each sample row $\\mathbf{x}$ can be pictured as a 4-dimensional vector \n", "\n", "\n", "$\\mathbf{x^T} = \\begin{pmatrix} x_1 \\\\ x_2 \\\\ x_3 \\\\ x_4 \\end{pmatrix} \n", "= \\begin{pmatrix} \\text{sepal length} \\\\ \\text{sepal width} \\\\\\text{petal length} \\\\ \\text{petal width} \\end{pmatrix}$" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "
\n", "
" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Exploratory Visualization" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "[[back to top](#Sections)]" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "To get a feeling for how the 3 different flower classes are distributes along the 4 different features, let us visualize them via histograms." ] }, { "cell_type": "code", "execution_count": 6, "metadata": {}, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAjAAAAGoCAYAAABYLN/cAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAAIABJREFUeJzs3X18DOf+P/5XbkQiN9IIqs3GNyhn8fBTnJbTuumHNhTlIEhOo0S1lDqctkQaqZ4gouipPEQkLdqg4aieao/qo6Vu2pISBLEoRTehkRCNjSS7kvn94WRlZW9mJ7M3w+v5T7Mz11zXe2bH7Lszc12XhyAIAoiIiIgUxNPVARARERHZiwkMERERKQ4TGCIiIlIcJjBERESkOExgiIiISHG8HVVxXl6eo6omIhfq2bOnq0Pg9YXoPmXP9cVhCQzg2AudRqOBWq12WP1sm22z7YZtu1PiIPb64spj2FhKjV2pcQPKjV2pcQN3Y7f3+sJHSERERKQ4TGCIiIhIcZjAEBERkeIwgSEiIiLFEZXA5OfnIzY2FsCdl21iYmIQGxuLyZMno7S01KEBEhEREd3LZgKTlZWFxMREVFdXAwAWLVqE+fPnIzs7G88++yyysrIcHiQRERFRfTYTmPDwcKSlpRk/r1ixwthVq6amBk2bNnVcdERERERm2BwHJjIyEoWFhcbPrVq1AgAcOXIEGzZswMaNGy1uq9FoZAjRvKqqKofWf1BbYXGdQW/AQe1hAEBvlb/DYjDH0fvNttm2O7RNyna47DCKtcWSth2gGiBvMHTfkjSQ3Y4dO7B69WpkZmYiJCTEYjlHDqrj6EF7igTL//i0hVqowlQAALW6tcNiMMfdBjZj2w9W2+40kB0RPdjsTmC++OILbN68GdnZ2QgODnZETERERERW2ZXA1NTUYNGiRWjTpg1ef/11AMCf//xnzJw50yHBEREREZkjKoEJCwvDli1bAAA///yzQwMiIiIisoUD2REREZHiMIEhIiIixWECQ0QOVX8k74KCAvTt2xexsbGIjY3Fjh07TMpWVVXh9ddfR0xMDKZMmYLr16+7ImQiUgBJ3aiJiMTIysrC9u3b4efnBwA4deoUJk2ahLi4OLPlP/30U3Ts2BGvv/46/vvf/yI9PR2JiYnODJmIFIJ3YIjIYe4dyfvkyZPYs2cP/va3vyEhIQE6nc6kfF5eHvr27QsA6NevHw4cOODUeIlIOXgHhogc5t6RvLt164aoqCh07doVq1evxqpVqzB37lzjep1Oh8DAQACAv78/bt68abFusaMEK3lEYaXGbjAYoNVqJW2r0cmzv9ZGU7em/kjrdZw94roUSj1XAOmxM4EhIqd59tlnERQUZPw7OTnZZH1AQAAqKu788FRUVBjLmiN2hGJXjmbcWEqN/XDZYahUKknbqlXy7K+10dStqT/Seh1nj7guhVLPFeBu7PaO9M1HSETkNJMnT8bx48cBAAcOHECXLl1M1vfo0QN79+4FAOzbtw89e/Z0eoxEpAxMYIjIaRYsWIDFixcjNjYWR44cwWuvvQYAiIuLg16vR3R0NH755RdER0dj8+bNmDFjhosjJiJ3xUdIRORQ9Ufy7tKlC3JychqUWbt2rfHvlStXOi02IlIu3oEhIiIixWECQ0RERIrDBIaIiIgUhwkMERERKQ4TGCIiIlIcJjBERESkOExgiIiISHGYwBAREZHiMIEhIiIixWECQ0RERIojKoHJz89HbGwsAODSpUuIjo5GTEwM3nnnHdTW1jo0QCIiIqJ72UxgsrKykJiYiOrqagBASkoKZs2ahU2bNkEQBOzatcvhQRIRERHVZ3Myx/DwcKSlpWHOnDkAgIKCAjzxxBMAgH79+uHHH3/Es88+a3ZbjUYjY6imqqqqHFq/trDC4jqD3gBtoRYAoPG47rAYzKnb74Ci/XZvq3u0ryxtu4K9bR8uOyxb2139uipmv++XtomIbLGZwERGRqKwsND4WRAEeHh4AAD8/f1x8+ZNi9uq1WoZQjRPo9E4tP4iodjiOm2hFqowFQBArW7tsBjMMe6350X7N+7UuOPl6GMuZ9vFWsvfn718db6K2W9Ht52Xl+eSWIiI7mX3S7yennc3qaioQFBQkKwBEREREdlidwLTuXNn5ObmAgD27duHXr16yR4UERERkTV2JzBz585FWloaxo0bB4PBgMjISEfERURERGSRzXdgACAsLAxbtmwBAERERGDDhg0ODYqIiIjIGlEJDBGRVPn5+Vi2bBmys7Oh0WiQnJwMLy8v+Pj4IDU1FaGhoSblR44cicDAQAB3/ucpJSXFFWETkZtjAkNEDpOVlYXt27fDz88PALBo0SLMnz8farUaOTk5yMrKwrx584zl68abys7Odkm8RKQcnEqAiBymbhypOitWrDB2za6pqUHTpk1Nyp8+fRqVlZWIi4vDhAkTcOzYMafGS0TKwTswROQw944j1apVKwDAkSNHsGHDBmzcuNGkvK+vLyZPnoyoqChcvHgRU6ZMwc6dO+Ht3fBSJXaQPSUPyKfU2A0GA7RaraRtNTp59tfaYKTW1B+otI6zByyVQqnnCiA9diYwRORUO3bswOrVq5GZmYmQkBCTdREREWjbti08PDwQERGB4OBglJSUoE2bNg3qETvAnysHA2wspcZ+uOwwVCqVpG3VKnn219pgpNbUH6i0jrMHLJVCqecKcDd2ewfK5CMkInKaL774Ahs2bEB2drbZH7itW7diyZIlAIDi4mLodDq0bNnS2WESkQIwgSEip6ipqcGiRYtQUVGB119/HbGxsVi5ciUAYM6cObh8+TLGjBmDmzdvIjo6GrNnz8bixYvNPj4iIuKVgYgcqv44Uj///LPZMkuXLjX+vXz5cqfERUTKxgSG3Moe7R6zy7VlWlknaCR6EFj69yTGANUA2eIgcgQ+QiIiIiLFYQJDREREisMEhoiIiBSHCQwREREpDhMYIiIiUhwmMERERKQ4TGCIiIhIcZjAEBERkeIwgSEiIiLFYQJDREREiiNpKgGDwYD4+HgUFRXB09MTycnJaN++vdyxERGRHIry7N9GxFQC350ynd4j9PJuAMC14mMwXLsgqpnyFt0AAN1VwfbFRw88SXdg9u7di9u3byMnJwfTp0/Hv/71L7njIiIiIrJIUgITERGBmpoa1NbWQqfTcbp7IiIicipJmUezZs1QVFSEIUOGoKysDBkZGWbLaTSaRgVnTVVVlUPr1xZWWFxn0BugLdQCANb977/W9Fb5yxZX3X4HFNlu9166WvuPV0DRfuPfPgYDtPU+18mtNL1VXNmis93t2GIwGKDV2r/Pcmju19yh55o1jj7P3bVtIiJbJCUw69evx9NPP4033ngDV65cwUsvvYQvv/wSTZs2NSmnVqtlCdIcjUbj0PqLhGKL67SFWqjCVKLrUqtbyxESgHr77XnR/o07SThe9drRarVQqRru9/mym6YLHhV/bMSy1LYz+Op8HXquWePo89zetvPyJLxLQUTkAJISmKCgIDRp0gQA0Lx5c9y+fRs1NTWyBkZERERkiaQEZuLEiUhISEBMTAwMBgNmz56NZs2ayR0bERERkVmSEhh/f3988MEHcsdCRPeh/Px8LFu2DNnZ2bh06RLi4+Ph4eGBxx57DO+88w48Pe/2JaiqqsJbb72Fa9euwd/fH6mpqQgJCXFh9ETkrjiQHRE5TFZWFhITE1FdXQ0ASElJwaxZs7Bp0yYIgoBdu3aZlP/000/RsWNHbNq0CSNHjkR6erorwiYiBWACQ0QOEx4ejrS0NOPngoICPPHEEwCAfv364aeffjIpn5eXh759+xrXHzhwwHnBEpGicAAXInKYyMhIFBYWGj8LggAPDw8Adx5F37xp2oNNp9MhMDDQ4vr6xHbxVnJ38MbGri27M/SA37VSu7cV026D4SaulwAAamtrUV7+h6h2SoU7sWlx57vW6OT5rqwNhWFN/WEy6mg8rssRkkM9iOc5Exgicpr677tUVFQgKCjIZH1AQAAqKiosrq9PbPdyV3ZFb6zGxl6s/d9wEJ5X7d5WTLv3DjcR6tnyTnPFlxAU1FxcQy1CAQCq/00loFbJ811ZGwrDGnPDZMg5FIaj3A/nub3DNPAREhE5TefOnZGbmwsA2LdvH3r16mWyvkePHti7d69xfc+ePZ0eIxEpAxMYInKauXPnIi0tDePGjYPBYEBkZCQAIC4uDnq9HtHR0fjll18QHR2NzZs3Y8aMGS6OmIjcFR8hEZFDhYWFYcuWLQDuzKO2YcOGBmXWrl1r/HvlypVOi42IlIt3YIiIiEhxmMAQERGR4jCBISIiIsVhAkNERESKwwSGiIiIFIcJDBERESkOExgiIiJSHCYwREREpDhMYIiIiEhxmMAQERGR4jCBISIiIsVhAkNERESKwwSGiIiIFEfybNRr1qzB7t27YTAYEB0djaioKDnjIiIiIrJIUgKTm5uLo0eP4tNPP0VlZSXWrl0rd1xEREREFklKYH744Qd07NgR06dPh06nw5w5c+SOi4iIiMgiSQlMWVkZLl++jIyMDBQWFmLatGnYuXMnPDw8TMppNBpZgjSnqqrKbP0HtRU2t+2t8rdZRltouR6D3gBtodZmHXU0Htetrg8o2i+6Lh+DAVo7ytenq7X/+wgourufeoMBWm3D/S6tLDX5XFkr/tiIZbDQtjM092vu0HPZGkvn+eGyw7K10euhXna1TeRIe7R7JG87QDVAtjjI/UlKYIKDg9GuXTv4+PigXbt2aNq0Ka5fv44WLVqYlFOr1bIEaY5GozFbf5FQbHNbtbq1zTLW6tEWaqEKU9msQ3R7nhdF16XVaqFSiW/bRCcJ30e92Cy1fb7spumCRyXGZ0Wj9ruRfHW+Dj2XrbF0nhdrbZ/nYqlV5vfNXNt5eXmytUtE1BiSEpiePXvik08+waRJk3D16lVUVlYiODhY7tiI6D60bds2fP755wCA6upqaDQa/PjjjwgKCgIALFy4EEeOHIG//507penp6QgMDHRZvETkniQlMM888wwOHTqEMWPGQBAEJCUlwcvLS+7YiOg+NGrUKIwaNQoA8O6772L06NHG5AUACgoK8OGHHyIkJMRVIRKRAkjuRs0Xd4moMU6cOIFz587hnXfeMS6rra3FpUuXkJSUhNLSUowZMwZjxoxxYZRE5K4kJzBERI2xZs0aTJ8+3WTZrVu38OKLL2LSpEmoqanBhAkT0LVrV/zpT39qsL3YF4yV/DJyY2PXlt158d3vWqmNkg2JabdBZ4frJQDuJKLl5X+IaqdUuBObFjdtlLRNo7sbs7WOGNaY66RhqyOGO3gQz3MmMETkdOXl5fj111/Ru3dvk+V+fn6YMGEC/Pz8AAC9e/fG6dOnzSYwYl+stvQitBI0Nnbjy96eV+3eVky793Z2CPVseae54ksICmourqEWoQAAlarx71HWfyFdTIcOc8x10hDT8cPV7ofz3N5OApxKgIic7tChQ/jLX/7SYPnFixcRExODmpoaGAwGHDlyBF26dHFBhETk7ngHhoic7sKFCwgLCzN+XrduHcLDwzFw4EAMHz4cY8eORZMmTTBixAg89thjLoyUiNwVExgicrqXX37Z5POkSZOMf0+ZMgVTpkxxdkhEpDBMYIiIqKEzX9ssEnr5hmzNHdM2vq7bN+Ub4JHcH9+BISIiIsVhAkNERESKwwSGiIiIFOeBfAfmu1Pu95xU7PPfc5UXGk6eKNbBU9K2+5/SylJxbRdJmPDv0Z72b+Okdg6XHZZt8kTOlktEJA/egSEiIiLFYQJDREREisMEhoiIiBSHCQwREREpDhMYIiIiUpwHshcSEZHsRIxc20CnIfLHQW5P7p6wgzq7/2zZjsA7MERERKQ4TGCIiIhIcZjAEBERkeIwgSEiIiLFaVQCc+3aNfTv3x/nz5+XKx4iIiIimyQnMAaDAUlJSfD19ZUzHiIiIiKbJHejTk1Nxfjx45GZmWmxjEajEVXXQW2FzTK9Vf4mn6uqqszWry20XZcYqus/WVzX5nYNcN2rYdshfzFbfl2h1kZbv4iOy+B7G6UlpaLLy8lQ47i2K2utHyODwQCt1rSM3zX7Y7HVjti2pdLoxP2bqGPxPC+TJx7AckyW2iYicgeSEpht27YhJCQEffv2tZrAqNVqUfUVCbb7xKvVpv3cNRqN2frF1CVGqGdLi+tKS0sQGmpm/SMq2du6143KKwhtGSqpncYqLSl1XNuPWj92Wq0WKtU9ZTyvyt6O6LYlUqvE/ZuoY+k8l2t2bMByTObazsuTMAO4GSNHjkRgYCAAICwsDCkpKcZ1W7ZsQU5ODry9vTFt2jQ888wzsrRJRPcXSQnMZ599Bg8PDxw4cAAajQZz587F6tWr0bKl+B9iInowVVdXAwCys7MbrCspKUF2djY+++wzVFdXIyYmBk899RR8fHycHSYRuTlJCczGjRuNf8fGxmLBggVMXohIlNOnT6OyshJxcXG4ffs2/vGPf6B79+4AgOPHj+Pxxx+Hj48PfHx8EB4ejtOnT6Nbt24ujpqI3A2nEiAip/L19cXkyZMRFRWFixcvYsqUKdi5cye8vb2h0+mMj5YAwN/fHzqdzmw9Yt/Pcda7PAFF9r+XpKu1HldjY697V0rK+2Jane39KS2tMru8trYW5eV/iGqnVJDvvToxMdti0Bugvee9RVvvMbqaxuO6ot9Zkxp7oxMYc7eBiYgsiYiIQNu2beHh4YGIiAgEBwejpKQEbdq0QUBAACoq7r6IX1FRYZLQ1Cf2HTtL7xHJzvOi/dt0sh5XY2M3visl4X0x1UO23/u6hhtml3sWX0JQUHNxDbWQ7706VXDj31XTFmqhCpPnnTdnUatbO+88d4C62O19x44D2RGRU23duhVLliwBABQXF0On0xkfQXfr1g15eXmorq7GzZs3cf78eXTs2NGV4RKRm+IjJCJyqjFjxmDevHmIjo6Gh4cHFi9ejOzsbISHh2PgwIGIjY1FTEwMBEHA7Nmz0bRpU1eHTERuiAkMETmVj48Pli9fbrKsR48exr/Hjh2LsWPHOjssIlIYPkIiIiIixWECQ0RERIrDBIaIiIgUhwkMERERKQ4TGCIiIlIc9kK6D/1aYntG7nYt/W2Wcaoi6wMY+V0rlTZ5o53tmNeq8e3SA29P2amGCw+aWVZPaUkpiv8IxYCHOt9d2GmIzJERKRPvwBAREZHiMIEhIiIixWECQ0RERIrDBIaIiIgUhwkMERERKQ4TGCIiIlIcJjBERESkOExgiIiISHGYwBAREZHiMIEhIiIixZE0lYDBYEBCQgKKioqg1+sxbdo0DBw4UO7YiIjoXme+Fl/W3PQFbibo2nG7tylv0c0BkZDSSEpgtm/fjuDgYLz33nsoKyvDX//6VyYwRERE5DSSEpjBgwcjMjLS+NnLy0u2gIjo/mbrDu66deuwdetWhISEAADeffddtGvXzlXhEpGbkpTA+PvfmclYp9Nh5syZmDVrltlyGo1GVH3aQtuzJ2s8rpt8rqqqMlu/mLpEuV5icdXt2zUoLW24Xlurlb2te/1mqEJh+WVp7dRTikq7tzHU3EZpSWmj25bCpW0HPQStVuJ3ew+NTty/iToWz/MyeeIBLMdkqe3GsnUHt6CgAKmpqejatavsbRPR/UNSAgMAV65cwfTp0xETE4Phw4ebLaNWq0XVVSQU2yyjVrc2+azRaMzWL6YuMUI9W1pcV1pagtBQM+sfUcne1r08iy8hKKi5pHZM2mzpb/c2pSWlCG0Z2ui2pXBl25W1TaBSSftu76VWifs3UcfSeV6slec8ByzHZK7tvLy8Rrdn6w5uQUEBMjMzUVJSggEDBuDVV19tdJtEdP+RlMCUlpYiLi4OSUlJ6NOnj9wxEdF9zNYd3KFDhyImJgYBAQGYMWMGvv/+ezzzzDMN6hF7d8hRd5LuFVBk/a5YaaX9dxDr7jxqddLuuElps46YNktLq8wur62tRXn5H5LbttmuYH6/pB6n+gx6A7SF8t3hdAaNx3WnneeOIDV2SQlMRkYGysvLkZ6ejvT0dABAVlYWfH19pVRHRA8YS3dwBUHASy+9hMDAQABA//79cerUKbMJjNg7vJbuYsnO86LV1efLbtpdZd2dR9VD0u4ASmmzjpg2r+GG2eVy3Sm2qIX5u7Gq4MbfKdUWaqEKk+eOq7Oo1a2dd547QF3s9t7hlZTAJCYmIjExUcqmRPSAs3YHV6fTYdiwYdixYweaNWuG3NxcjB492kWREpE7k/wODBGRFObu4EZFRaGyshLjxo3D7NmzMWHCBPj4+KBPnz7o37+/iyMmInfEBIaInMrWHdyRI0di5MiRToyIiJSICQwR0b3sGe3WCfa4YERdMW3+WinTsBV2sjR67282RvW1NoJv12DpHVJO3jggedvGtFvnu1PF0BZWyNYLV06DOre2XUgizoVEREREisMEhoiIiBSHCQwREREpDhMYIiIiUhwmMERERKQ4TGCIiIhIcRTTjfq7U6bdwxzdZexI5S8W15XX/IGgSjNDaJ83v00Pv8fkCosUbo92j13ltWVaWSduJCK6X/AODBERESkOExgiIiJSHMU8QiIikiKgaL/NmaLFEjU6bYl8o9OWl+sR2lK26mSN7X5TN5puaVUp/rhR6OJoSAzegSEiIiLFYQJDREREisMEhoiIiBSHCQwREREpDhMYIiIiUhwmMERERKQ4TGCIiIhIcSQlMLW1tUhKSsK4ceMQGxuLS5cuyR0XEd2nbF0/tmzZglGjRmHs2LH4/vvvXRQlEbk7SQnMd999B71ej82bN+ONN97AkiVL5I6LiO5T1q4fJSUlyM7ORk5ODj766COsWLECer3ehdESkbuSlMDk5eWhb9++AIDu3bvj5MmTsgZFRPcva9eP48eP4/HHH4ePjw8CAwMRHh6O06dPuypUInJjkqYS0Ol0CAgIMH728vLC7du34e1tWl1eXp6o+h6SEMNDLTyASscN9/zQI0Mtr3zEvrpqGheKib4PdZSxNvsE2rnf90vbnQHgqovabtrZ4W3nXbX871Tsv2F7WLt+6HQ6BAYGGtf5+/tDp9M1LraHeuKq+SrsFtiklc0y/5+c56rM572ssVnjwn+vjfaoyrntyfQ75ujfRKny8sTFJOVaIymBCQgIQEXF3Tk1amtrGyQvPXv2lFI1Ed3nrF0/7l1XUVFhktDU4fWFiCQ9QurRowf27dsHADh27Bg6dnTdXQEiUhZr149u3bohLy8P1dXVuHnzJs6fP8/rCxGZ5SEIgmDvRrW1tViwYAHOnj0LQRCwePFitG/f3hHxEdF9xtz1Y9++fQgPD8fAgQOxZcsWbN68GYIg4NVXX0VkZKSrQyYiNyQpgXG2a9euYdSoUVi7dq1JorRu3Tps3boVISEhAIB3330X7dq1k63dkSNHGm9fh4WFISUlxbhuy5YtyMnJgbe3N6ZNm4ZnnnlGtnZttb1w4UIcOXIE/v7+AID09HSzt9mlWrNmDXbv3g2DwYDo6GhERUUZ1+3evRurVq2Ct7c3Ro8ejbFjx8rWrq22Hf19b9u2DZ9//jkAoLq6GhqNBj/++COCgoIAOPY7t9W2I79zg8GA+Ph4FBUVwdPTE8nJySb/zhz9ncvBYDAgISEBRUVF0Ov1mDZtGgYOHGhc7877YCt2R5/3jVFTU4PExERcuHABXl5eSElJQXh4uHG9ux53W3G78zGvY+l30V2PeR1Zf88FN6fX64XXXntNeO6554Rz586ZrHvjjTeEEydOOKTdqqoqYcSIEWbXXb16VRg2bJhQXV0tlJeXG/92RtuCIAjjx48Xrl27Jlt79R08eFB49dVXhZqaGkGn0wkrV640rtPr9cKgQYOEGzduCNXV1cKoUaOEq1evOqVtQXDs932vBQsWCDk5OcbPjv7OrbUtCI79zr/99lth5syZgiAIwg8//CDMmDHDuM7R37lctm7dKixcuFAQBEG4fv260L9/f+M6d98Ha7ELgnPPe3t9++23Qnx8vCAId/79Tp061bjOnY+7tbgFwb2PuSBY/l1052MuCPL/nrv9SLypqakYP348WrVq+PZ/QUEBMjMzER0djTVr1sja7unTp1FZWYm4uDhMmDABx44dM65zdFdPa23X1tbi0qVLSEpKwvjx47F161bZ2gWAH374AR07dsT06dMxdepUDBgwwLju/PnzCA8PR/PmzeHj44OePXvi8OHDTmkbcOz3Xd+JEydw7tw5jBs3zrjMWd17zbXt6O88IiICNTU1qK2thU6nM3kh39HfuVwGDx6Mv//978bPXl5exr/dfR+sxQ4477yXYtCgQUhOTgYAXL58GaGhocZ17nzcrcUNuPcxByz/LrrzMQfk/z2X1AvJWbZt24aQkBD07dsXmZmZDdYPHToUMTExCAgIwIwZM/D999/Ldlvf19cXkydPRlRUFC5evIgpU6Zg586ddnf1lLvtW7du4cUXX8SkSZNQU1ODCRMmoGvXrvjTn/4kS9tlZWW4fPkyMjIyUFhYiGnTpmHnzp3w8PBw+H5baxtw7Pdd35o1azB9+nSTZY7ed2ttO/o7b9asGYqKijBkyBCUlZUhIyPDuM5Z+91YdY/WdDodZs6ciVmzZhnXufs+WIsdcN55L5W3tzfmzp2Lb7/9FitXrjQud/fjbiluwL2PubXfRXc+5o74PXfrOzCfffYZfvrpJ8TGxkKj0WDu3LkoKSkBAAiCgJdeegkhISHw8fFB//79cerUKdnajoiIwAsvvAAPDw9EREQgODjY2LbYrp6OaNvPzw8TJkyAn58fAgIC0Lt3b1nvBAQHB+Ppp5+Gj48P2rVrh6ZNm+L69esAHL/f1tp29Pddp7y8HL/++it69+5tstzR+26tbUd/5+vXr8fTTz+Nb775Bl988QXi4+NRXV0NwDn7LZcrV65gwoQJGDFiBIYPH25croR9sBS7s877xkpNTcU333yD+fPn49atWwCUcdzNxe3ux9za76I7H3NH/J67dQKzceNGbNiwAdnZ2VCr1UhNTUXLli0B3Mk0hw0bhoqKCgiCgNzcXHTt2lW2trdu3Woc4ry4uBg6nc7YtqO7elrZUFfZAAAgAElEQVRr++LFi4iJiUFNTQ0MBgOOHDmCLl26yNZ2z549sX//fgiCgOLiYlRWViI4OBgA0L59e1y6dAk3btyAXq/H4cOH8fjjjzulbUd/33UOHTqEv/zlLw2WO6N7r6W2Hf2dBwUFGS9yzZs3x+3bt1FTc2f4RUd/53IpLS1FXFwc3nrrLYwZM8Zknbvvg7XYnXXeS/Wf//zHeLvfz88PHh4exkdg7nzcrcXt7sfc2u+iOx9zR/yeK6IXEgDExsZiwYIFOHXqFG7duoVx48bhP//5D7Kzs+Hj44M+ffpg5syZsrWn1+sxb948XL58GR4eHnjzzTeRn5/vlK6ettrOysrCzp070aRJE4wYMQLR0dGytQ0AS5cuRW5uLgRBwOzZs3Hjxg3jMa97w10QBIwePRp/+9vfnNa2I7/vOh9++CG8vb0xceJEAHfejHdW915rbTvyO6+oqEBCQgJKSkpgMBgwYcIEAHDady6HhQsX4uuvvzbptRAVFYXKykq33wdbsTvjvJfq1q1bmDdvHkpLS3H79m1MmTIFlZWVbn/u2IrbnY95feZ+F931mNcn1++5YhIYIiIiojpu/QiJiIiIyBwmMERERKQ4TGCIiIhIcZjAEBERkeIwgSEiIiLFYQJDiI+Px759+2wuk+ry5cvYvXs3gDvd586fP2+xbFpaGiIjI/Hll19Kauvtt99Gr169rLZBRM5n7zVl37592Lx5c4PlY8eORWFhIW7cuGG8Ttiqe9u2bRgwYADWrVtnf+AA3n//fTz11FOyXRNJHm49lQDdHw4ePIhff/0V//d//yeq/MSJE01GI7XHokWL8Ntvv0nalojcR79+/ayuP3PmDHbv3i36WjFs2DBMmjRJUiyzZ89GcXGxpG3JcZjAKMiFCxcwb948eHt7w8vLC0uXLkXr1q2xfPlyHDp0CIIgYOLEiRgyZAhiY2MRERGBCxcuQBAEvP/++wgJCUFSUhJ+//13lJWVoV+/fg3mXbmXwWDAO++8g0uXLqG2thazZs3Ck08+ieHDh+OJJ57AmTNn4OHhgfT0dAQEBODdd9/FyZMnERoaiqKiIqxatQqZmZmoqqoyjgi5atUqlJaWorKyEitWrIBKpTLbdn5+PhYtWgRBENC6dWssW7YMU6ZMQadOnfDLL7+gWbNm6NWrF3744QeUl5dj7dq1aN68uezHnehB4qzrTFlZGSZOnIgvvvgCR48exauvvooDBw6gpKQEb7/9NoYOHYpff/0Vb775Jt5//33s378fDz/8MMrKygAAGRkZOH36tPEuzebNm/Hhhx9Cp9NhwYIF6Natm9n9u3jxIhITE2EwGODr64v3338fS5cuhbe3Ny5fvgy9Xo/nn38e33//Pa5cuYL09HSEh4c77oCTZHyEpCA//fQTunTpgnXr1mHq1Kn4448/sHfvXhQWFiInJweffPIJMjIyUF5eDgDo0aMHsrOzMWTIEKxZswZXrlxB9+7d8dFHH+HTTz/Fp59+arPNf//733jooYewceNGpKen45///CeAO6O3Dh06FBs2bECrVq2wb98+7Nq1Czdu3MDWrVuxePFiXLlyBZ6ennjllVcwbNgwDBw4EADQv39/fPLJJ+jXrx927txpse358+cjJSUF//73v9GnTx/jY6Fu3brh448/hl6vh6+vL9atW4cOHTrg0KFDjT3ERA88Z11nHnroIQQHB+PKlSvG5KSgoAC7du3CoEGDjOXOnj2LQ4cOYevWrVi6dKlxrp+pU6eid+/expnbu3Tpgk8++QQvvvgitm3bZnH/UlNT8corr2Dz5s0YN26ccc6dRx99FGvXrkW7du1QWFiIrKwsPPfcc8bH3+R+eAdGQcaMGYOsrCy8/PLLCAwMxOzZs3H27FkUFBQgNjYWAHD79m1cvnwZAIyTAvbo0QO7d+9GcHAwTpw4gYMHDyIgIAB6vd5mm2fPnkVeXh6OHz9urL/u/4A6d+4MAGjTpg2qq6tRVFSE7t27AwBCQkJMhkavr26Oi9DQUJSWllps+9q1a2jfvj0AmAyHXTcPUFBQEDp06GD8u24CQiKSzpnXmWeffRZ79+7F0aNHMWXKFPz44484evQoFi1ahP379wMAzp07h65du8LT0xMBAQEW5yCruy6EhoaiqqrKYpsXLlww3g1+/vnnAQBfffWV8XoWFBRkvHYFBQWJuk6Sa/AOjILs2rULPXv2xMcff4zBgwfjww8/RLt27fDkk08iOzsbH3/8MYYMGYKwsDAAwMmTJwEAR44cQYcOHbBt2zYEBgZi+fLliIuLQ1VVFWzNJNGuXTsMHToU2dnZyMrKwuDBg42PaTw8PEzKPvbYYzh27BgA4I8//sDFixcBAJ6enqitrbV7f1u1amWsIzMzE99++63ddRCRfZx5nRk0aBC++uorBAQEoF+/fvjuu++g1+uNk/wBQEREBI4fP47a2lrcunUL586dA9DwunLv9ciS9u3b48SJEwCA7du3Izs7267tyX3wDoyCdO3aFW+99RbS0tLg6emJefPmoXPnzvj5558RExODW7duYdCgQQgICAAAfP7551i/fj38/PywdOlSlJaW4h//+Afy8vLg5+eHtm3b4urVq1bbHD9+PBITE/Hiiy9Cp9MhJiYGnp7m894BAwZg3759GD9+PEJDQ+Hr64smTZqgY8eOWL16td0zKL/77rtISEiAp6cnWrZsiYkTJ+KTTz6xqw4iso8zrzMPP/wwqqur0bt3bzRv3hze3t4YMGCASRm1Wo3BgwdjzJgxaNWqFVq0aAEACA8Px9mzZ7F+/Xq79m/OnDlISkrC6tWr4evri/feew8FBQV2HydyPU7meJ+qm+2z7hGMM5w/fx6nT5/G0KFDUVZWhmHDhuH777+Hj4+P6DrS0tIQGhraqNmWXbHvRA8ipfxb27Ztm/GFYKni4+Px/PPP2+wdRc7DR0gkmzZt2uCrr77C2LFj8fLLL+PNN9+0K3mps379+kaNA6PRaCRtS0T3r6+++qpR48DUvZND7oN3YIiIiEhxeAeGiIiIFIcJDBERESkOExgiIiJSHCYwREREpDhMYIiIiEhxmMAQERGR4jCBISIiIsVhAkNERESKwwSGiIiIFMdhkznm5eU5qmoicqGePXu6OgReX4juU/ZcXxw6G7XYQDQaDdRqtSNDkYRx2Ydx2UeJcblT4tCzZ0+3PYa2KDVugLG7glLjBuyL3d7rCx8hERERkeIwgSEiIiLFYQJDREREisMEhoiIiBRHVAKTn5+P2NhYk2Vffvklxo0b55CgiIiIiKyx2QspKysL27dvh5+fn3GZRqPB1q1bIQiCQ4MjIiJSou9OFTdq+0GdW8sUyf3LZgITHh6OtLQ0zJkzBwBQVlaGZcuWISEhAfPnz7e6rUajERVEVVWV6LKWHNRWmHzurfJvVH2APHE5AuOyD+Oyj7vG5Ujmfmz4A0IPksuXL+ORRx5xdRh2sZnAREZGorCwEABQU1ODt99+GwkJCWjatKnNysX2/Zajj3uRYHoBUqsbf/Fx1773jMs+jMs+ShkHhuhBlZubi/z8fLzyyisAgIyMDIwfPx7BwcGS60xKSsKHH34oV4hOYddAdgUFBbh06RIWLFiA6upqnDt3DosWLcLbb7/tqPiIiIjIjBdffBGdO3dGeXk5DAYDkpOT0aRJE5SUlCA1NRXe3nd+4n/77TekpaXBz88PnTp1wogRI7B8+XJ4eXlBr9dj4sSJuHjxIr788kuEhIRg27Zt8PHxQe/evfH444+bbDtkyBCkpKTA398fxcXFWLlyJZo0aeKS/berF1K3bt3w3//+F9nZ2VixYgU6dOjA5IWIiMgFunfvjoSEBONnrVaLhx9+GH/729/g4eFhXH7jxg3odDr07t0bTz31FLZv344bN27A19cXFRUVuHXrFtq2bYvhw4fj448/RmpqKlJSUrBt27YG23p6emLMmDF46qmncPXqVVy9etUVuw6A3aiJiIgUKSgoyPj37du3MWPGDHTt2hWZmZkoKCjA7NmzMXv2bLRo0QJvvPEGACA5ORmCIOCpp57Cm2++ieeffx6tW9995aK2ttYk+bl329zcXHz99ddo3bo1HnnkEZd25hH1CCksLAxbtmyxuYyIiIic/xK4t7c3cnJy0LRpUwQGBkKlUuH9998HAJw/fx7Lly/Ho48+iscffxwvvPACEhIScPLkSVRVVWHAgAFQqVT46KOP8NJLLyEhIQH+/v6IiopCVVWVybbBwcG4cuUKduzYgd9//x03btxAWFiYU/fVuM8uaZWIiIgkefLJJ/Hkk08aPy9ZsgQAsHjxYrPl27dvj9WrV5ssS0tLM/m8YMEC4999+/Y1WXfvtvXbdiU+QiIiIiLFYQJDREREisMEhoiIiBSH78AQERHJ7czXjdu+0xB54riP8Q4MERGRguTm5iIzM9P4OSMjAzdu3HBIW6dOncL27dttlissLERSUpJDYrCEd2CIiIgUSOxIvOvWrcNjjz2Gp59+GjNmzEBKSgpWrFhhHIk3MTEREyZMQLt27TBhwgR89NFHxpF3O3TogN9//x1nzpzB+vXrUVNTg6eeegodO3ZEZmYmAgMD0bZtW0RGRgIAfv/9dyxZsgQhISFo2rQpXnjhBURGRqJXr16YOXOmyZgzjcU7MERERAokdiTeESNG4L///S+0Wi0eeeQRfPnllyYj8Z49exa1tbVYvHgx9Hq9yci7dTIzM5GQkIDU1FR06NABmZmZeOedd/DPf/4TR44cQUXFnQmVs7Oz8corryApKQkGgwG//fYbmjdvjkWLFsmavABMYIiIiBRJ7Ei8AQEBAIBNmzZh9OjRZkfiDQwMBNBw5N06BoPBmBRdunTJZMReDw8P44i8tbW18PT0NFleV7fc+AiJiFzGYDAgISEBRUVF0Ov1mDZtGh5++GFMnToV/+///T8AQHR0NJ5//nnXBkpkLye/hGttJF4AeOGFF/DBBx9g7ty5eOSRRxqMxFvn3pF367z88sv45z//CeDOQHevvPIKkpOTERISgl69ehmTpBdffBErVqxAq1at0KxZM7Rt29Zx++ywmomIbNi+fTuCg4Px3nvvoaysDH/9618xffp0TJo0CXFxca4Oj8gt2TsSLwD06dMHffr0AQAEBgY2GIn3o48+AmB+1N66tpYuXWqyfNmyZSaf6xKc5cuXG5dpNBpj3XJjAkNELjN48GDjy38A4OXlhZMnT+LChQvYtWsX2rZti4SEBOP/3RER1WECQ0Qu4+/vDwDQ6XSYOXMmZs2aBb1ej6ioKHTt2hWrV6/GqlWrMHfu3AbbajQaVFVVQaPRNDoObWFFw/o9rje6XkvkitsVGLvzKTVuwLGxM4EhIpe6cuUKpk+fjpiYGAwfPhzl5eXGlxOfffZZkxcJ61Or1dBoNFCr1Y2OoUgoNlO/42YTlituV2DszqfUuAH7Ys/Ly7OrbiYwROQypaWliIuLQ1JSkvH5/OTJkzF//nx069YNBw4cQJcuXVwcJZH99mj3NGr7AaoBssRxPxOVwOTn52PZsmXIzs6GRqNBcnIyvLy84OPjg9TUVISGhjo6TiK6D2VkZKC8vBzp6elIT08HAMTHx2Px4sVo0qQJQkNDLd6BIaIHm80EJisrC9u3b4efnx8AYNGiRZg/fz7UajVycnKQlZWFefPmOTxQIrr/JCYmIjExscHynJwcF0RDpAy5ubnIz8/HK6+8AuDO/wiMHz8ewcHBorY/deoUzp07hxdeeMFimb1798LHx8d4Z9SebZ3FZgITHh6OtLQ0zJkzBwCM/bsBoKamBk2bNnVshERERNSA1KkExo0bh99//x1paWm4cOECevXqBb1ej19//RV6vR4+Pj7o3r07mjZtivj4eDz88MPQ6XSIiIiwOLWASqVCTk4OvL290bx5c7Mv3svNZgITGRmJwsJC4+e65OXIkSPYsGEDNm7caHFbsW8eS31L+aC2Yc8BY9sy9CBw1ze/GZd4h8sOw2Aw4HDZYYtlej3Uy4kR3eWOxwtw37iIyFT37t3x5ptvIj4+HsCdqQT+8pe/4LnnnmswlcB7772Htm3b4pFHHoGPj49x3ciRI/Hkk09izpw5+OCDD3D06FF8/vnnJu2MGjUKYWFhePnll9GhQwcAd6YWWLBgAQICAnDq1CkEBARgzJgx+P3337F27Von7L3El3h37NiB1atXIzMzEyEhIRbLiX3zWOob1uZ6Dtxtu/E9CNz1zW/GJV6xthharRYqlcpiGbXKNTG74/ECrMdlby8BInIcc1MJ6PV6ZGZm4rXXXsO6desAAKmpqQDuTiVQf+bqoKAgGAwG41QAXl5eDdrx8/ODp6enSVJ079QCeXl5eOyxx9CrVy+nPZmxO4H54osvsHnzZmRnZ4t+3kZERPQgcXYvInumEsjNzTXZNiAgAF26dMHChQtRVlYmqmPOvVMLtGnTBkePHsWZM2dQXV2Nmpoas8mQnOxKYGpqarBo0SK0adMGr7/+OgDgz3/+M2bOnOmQ4IiIiMhUY6cSuHd7AKisrESTJk3g6+uL0aNHo2PHjgCAoUOHGsvUTQlgaWoBZxOVwISFhWHLli0AgJ9//tmhAREREZFzzZo1y9Uh2M3T1QEQERER2YsJDBERESkOExgiIiJSHCYwREREpDhMYIiIiEhxmMAQERGR4jCBISIiIsVhAkNERESKwwSGiIiIFIcJDBERESkOExgiIiJSHCYwREREpDhMYIiIiEhxmMAQERGR4jCBISIiIsVhAkNERESK4y2mUH5+PpYtW4bs7GxcunQJ8fHx8PDwwGOPPYZ33nkHnp7Mg4jIfgaDAQkJCSgqKoJer8e0adPQoUMHXmOIyCabV4WsrCwkJiaiuroaAJCSkoJZs2Zh06ZNEAQBu3btcniQRHR/2r59O4KDg7Fp0yZkZWUhOTmZ1xgiEsVmAhMeHo60tDTj54KCAjzxxBMAgH79+uGnn35yXHREdF8bPHgw/v73vxs/e3l58RpDRKLYfIQUGRmJwsJC42dBEODh4QEA8Pf3x82bNy1uq9FoRAVRVVUlumx92sIKy217XLe7vntJjcvR7ve4Aor2W12ve7Sv6Lq0ZVoYDAZotVqLZTQ61xzL+/17FMPf3x8AoNPpMHPmTMyaNQupqamirjEajUa2WM1dS+S4hljirt+9GIzd+ZQaN+DY2EW9A1Nf/WfRFRUVCAoKslhWrVaLqlOj0YguW1+RUGyl7dZ213cvqXE52n0fl+dF6+s7iW+jWFsMrVYLlUplsYxa5ZpjqcTvMS8vT/b2rly5gunTpyMmJgbDhw/He++9Z1xn7RqjVqtlO4bmriVyXEMscdfvXgzG7nxKjRuwL3Z7ry92vxnXuXNn5ObmAgD27duHXr162VsFEREAoLS0FHFxcXjrrbcwZswYALzGEJE4dicwc+fORVpaGsaNGweDwYDIyEhHxEVED4CMjAyUl5cjPT0dsbGxiI2NxaxZs3iNISKbRD1CCgsLw5YtWwAAERER2LBhg0ODIqIHQ2JiIhITExss5zWGiGzh4ApERESkOExgiIiISHGYwBAREZHiMIEhIiIixWECQ0RERIrDBIaIiIgUhwkMERERKY7dUwkQPYj2aPfYLDNANcDhcRAR0R28A0NERESKwwSGiIiIFIcJDBERESkOExgiIiJSHCYwREREpDhMYIiIiEhxmMAQERGR4jCBISIiIsWRNJCdwWBAfHw8ioqK4OnpieTkZLRv317u2IiIiIjMknQHZu/evbh9+zZycnIwffp0/Otf/5I7LiIiIiKLJN2BiYiIQE1NDWpra6HT6eDtbb4ajUYjqr6qqirRZevTFlZYXKfxuG53ffeSGpejuTqugKL9Zpf7GAzQFu2H7tG+jaxfa3W9rlb8vmvLtDAYDNBqLdep0dmuT1tmPSYAyNZm2yzT66Fexr9d/T1a4q5xERHVJymBadasGYqKijBkyBCUlZUhIyPDbDm1Wi2qPo1GI7psfUVCscV1anVru+u7l9S4HM3lcXleNLtYq9VCpVIBnRoZm4X6jeyov1hbfDcuC9Qq2/UVay2fa/ao35bLv0cLrMWVl5fn5GiIiMyT9Ahp/fr1ePrpp/HNN9/giy++QHx8PKqrq+WOjYiIiMgsSXdggoKC0KRJEwBA8+bNcfv2bdTU1MgaGBEREZElku7ATJw4EQUFBYiJicFLL72E2bNno1mzZnLHRkQPgPz8fMTGxgIACgoK0LdvX8TGxiI2NhY7duxwcXRE5K4k3YHx9/fHBx98IHcsRPSAycrKwvbt2+Hn5wcAOHXqFCZNmoS4uDgXR0ZE7k5SAkNEJIfw8HCkpaVhzpw5AICTJ0/iwoUL2LVrF9q2bYuEhAQEBASY3Vaj0cjWY8pcj0Y5ejJaouSeXozd+ZQaN+DY2JnAEJHLREZGorCw0Pi5W7duiIqKQteuXbF69WqsWrUKc+fONbutWq2WrSeXuR6NcvRktMRde6CJwdidT6lxA/bFbm8vR04lQERu49lnn0XXrl2Nf586dcrFERGRu2ICQ0RuY/LkyTh+/DgA4MCBA+jSpYuLIyIid8VHSETkNhYsWIDk5GQ0adIEoaGhSE5OdnVIROSmmMAQkUuFhYVhy5YtAIAuXbogJyfHxRERkRIoKoH57pT44dzrlx3U2XEv45EZZ762vr7TEPeun4iI3B7fgSEiIiLFYQJDREREisMEhoiIiBSHCQwREREpDhMYIiIiUhxF9UIiIqL7FHsXkp14B4aIiIgUhwkMERERKY7kR0hr1qzB7t27YTAYEB0djaioKDnjIiIiIrJIUgKTm5uLo0eP4tNPP0VlZSXWrl0rd1xEREREFklKYH744Qd07NgR06dPh06nw5w5c+SOi4iIiMgiSQlMWVkZLl++jIyMDBQWFmLatGnYuXMnPDw8TMppNBpR9VVVVYkqqy2skBIuNB7XJW0nNi5nc3VcAUVas8v1BgO0WvPrTGgzZY7IlK727rHRlmlhsBGXRifi3CsTsV8i1G/L1d+jJe4aFxFRfZISmODgYLRr1w4+Pj5o164dmjZtiuvXr6NFixYm5dRqtaj6NBqNqLJFgvjJHE3jkDaZo9i4nM3lcXleNLtYq9VCpVI5NxZzOt09NsXaYptxqVW2j2WxVtq5Z60tl3+PFliLKy8vz8nREBGZJ6kXUs+ePbF//34IgoDi4mJUVlYiODhY7tiIiIiIzJJ0B+aZZ57BoUOHMGbMGAiCgKSkJHh5eckdGxEREZFZkrtR88VdIiIichUOZEdERESKw7mQiIiIJNqj3SOq3ADVAIfG8SDiHRgiIiJSHCYwREREpDhMYIjIpfLz8xEbGwsAuHTpEqKjoxETE4N33nkHtbW1Lo6OiNwVExgicpmsrCwkJiaiuroaAJCSkoJZs2Zh06ZNEAQBu3btcnGEROSumMAQkcuEh4cjLS3N+LmgoABPPPEEAKBfv3746aefXBUaEbk59kIixdpTdsr8Cq2fcwOxQ/0eC9oyrcUpCh6UHguRkZEoLCw0fhYEwTinmr+/P27evGlxW41GI9u8TebmWZM6h5oYSp5vylGxW5pjrU79Oc7sr3s/AMDHYID2f38b6320r+R6AfHzpImZc80Sni/mMYEhIrfh6Xn3pnBFRQWCgoIsllWr1bLNJ2VunjWpc6iJ4a7zYInhsNgtzLFm1KkRbf6vbrPzojWmXoifJ03MnGuWPCjni71zrfEREhG5jc6dOyM3NxcAsG/fPvTq1cvFERGRu2ICQ0RuY+7cuUhLS8O4ceNgMBgQGRnp6pCIyE3xERIRuVRYWBi2bNkCAIiIiMCGDRtcHBERKQHvwBAREZHiMIEhIiIixWECQ0RERIrDBIaIiIgUp1EJzLVr19C/f3+cP39erniIiIiIbJKcwBgMBiQlJcHX11fOeIiIiIhskpzApKamYvz48WjVqpWc8RARERHZJGkcmG3btiEkJAR9+/ZFZmamxXJi5z8QO1eCuflKxFhXeHeuit4qf9Hbuev8E66Oy9KcJXqDAVqtuHlB5FBaWWp+Rck3Jh+9a26j9NrdeZMqW3Q2WZ+tzZY9NjEMVo5XY+ZNaSxXn19E5Br152qzxR3ma5OUwHz22Wfw8PDAgQMHoNFoMHfuXKxevRotW7Y0KSd2/gOxcyWYm6/EXvbMb+Ku80+4PC4Lc5aYnWfEgc6XWZ7or77SklKEtgy9u+BR58VojbXj1Zh5UxrL2vll71wlRESOIimB2bhxo/Hv2NhYLFiwoEHyQkREROQo7EZNREREitPouZCys13z/gARERE9uHgHhoiIiBSHs1ETEZnx3amGnQYGdbbdCUDqdkoRULTf4ov86DTEqbEYnfnaNe2SS/EODBERESkOExgiIiJSHCYwREREpDhMYIiIiEhxmMAQERGR4rAX0oPI1hv7rupJQLITM7eJO8xp4kj3e68gur+Y+zerLdOiWCt9Kh1H/BsXO29Sazju3xrvwBAREZHiMIEhIiIixeEjJCJyOyNHjkRgYCAAICwsDCkpKS6OiIjcDRMYInIr1dXVADjPGhFZx0dIRORWTp8+jcrKSsTFxWHChAk4duyYq0MiIjfEOzBE5FZ8fX0xefJkREVF4eLFi5gyZQp27twJb2/Ty5VGo0FVVRU0Go3V+rSFFQ2WaTyu2yxjzr3bSW1PTNzuysdggFarNbtOV2t9nwKK9ktu11rdAUXm47mX3kzstmK2RVsmru1srfQ7igYrx1wMjU7cPordF3s092vusHOdCQwRuZWIiAi0bdsWHh4eiIiIQHBwMEpKStCmTRuTcmq1GhqNBmq12mp9RULD7qdqdWubZcy5dzup7YmJ211pi/ZDpVKZX9nJxj5ZmgRSDGt1i6xXq9U2jN1WzDY0pnuzWGbjtoNaJW4fHbEvvjpf0ed6Xl6eXXXzERIRuTyw/WkAABEdSURBVJWtW7diyZIlAIDi4mLodDq0bNnSxVERkbuRdAfGYDAgISEBRUVF0Ov1mDZtGgYOHCh3bET0ABozZgzmzZuH6OhoeHh4YPHixQ0eHxERSboqbN++HcHBwXjvvfdQVlaGv/71r0xgiEgWPj4+WL58uavDICI3JymBGTx4MCIjI42fvby8ZAuIiIiIyBZJCYy/vz8AQKfTYebMmZg1a5bZcmLfPLb2Rv5BrbjeAWLd2xugfv29Vf6i43IlW3E15k1/AIA2U9Jm5t7wd6TSylJR5Qw1t1FacrdsZa3zYrTGWs8CMT0Wej3Uy2YZMb0K7u2h4K7nPRFRfZIfLF+5cgXTp09HTEwMhg8fbraM2DePrb2RL7Z3gFjWeh8opaeAzbga86Z/IzT2TXl7nS+7KapcaUkpQluG3l3wqPNitMYZPQvE9Cq4tx5r55e9vQSIiBxFUgJTWlqKuLg4JCUloU+fPnLHRERERGSVpG7UGRkZKC8vR3p6OmJjYxEbG4uqqiq5YyMiIiIyS9IdmMTERCQmJsodCxEREZEoHMiOiIiIFIejQxERkTzOfO2SZveUnRJVrj0CGy60FXOnIRIici97tHtcHYJD8A4MERERKQ4TGCIiIlIcJjBERESkOExgiIiISHHc4iXeg9oK2UfcvZ8FFO132Wi7ziL2pbwH2f36Yh4RkRi8A0NERESKwwSGiIiIFIcJDBERESkOExgiIiJSHCYwREREpDhu0QuJiOhBIrbn5aDOrZ0QzYMjt/ICzpfdtG+jg43oEfloT+nbkk28A0NERESKwwSGiIiIFIcJDBERESmOpHdgamtrsWDBApw5cwY+Pj5YuHAh2rZtK3dsRPQA4vWFiMSQdAfmu+++g16vx+bNm/HGG29gyZIlcsdFRA8oXl+ISAwPQRAEezdKSUlBt27dMHToUABA3759sX//fpMyeXl58kRIRG6lZ0/H9qzg9YXowWXP9UXSIySdToeAgADjZy8vL9y+fRve3nerc/RFjojuT7y+EJEYkh4hBQQEoKKiwvi5trbW5OJCRCQVry9EJIakBKZHjx7Yt28fAODYsWPo2LGjrEER0YOL1xciEkPSOzB1vQTOnj0LQRCwePFitG/f3hHxEdEDhtcXIhJDUgIjp/z8fCxbtgzZ2dmuDMOEwWBAQkICioqKoNfrMW3aNAwcONDVYaGmpgaJiYm4cOECvLy8kJKSgvDwcFeHBQC4du0aRo0ahbVr17rVj83IkSMRGBgIAAgLC0NKSoqLIwLWrFmD3bt3w2AwIDo6GlFRUa4OCdu2bcPnn38OAKiuroZGo8GPP/6IoKAgF0dmu1v1li1bkJOTA29vb0ybNg3PPPOMC6M1ZSv2hQsX4siRI/D39wcApKenG89Xd2Dp+rx7926sWrUK3t7eGD16NMaOHeuiCC2zFPu6deuwdetWhISEAADeffddtGvXzhUhNmDrt8ddj7utuB12zAUXyszMFIYNGyZERUW5MowGtm7dKixcuFAQBEG4fv260L9/f9cG9D/ffvutEB8fLwiCIBw8eFCYOnWqiyO6Q6/XC6+99prw3HPPCefOnXN1OEZVVVXCiBEjXB2GiYMHDwqvvvqqUFNTI+h0OmHlypWuDqmBBQsWCDk5Oa4Ow+ibb74R5s6dKwiCIBw9etTkvL969aowbNgwobq6WigvLzf+7S6sxS4IgjB+/Hjh2rVrrgjNJkvXZ71eLwwaNEi4ceOGUF1dLYwaNUq4evWqi6I0z9pvyxtvvCGcOHHCBVHZZu23x52Pu63fTEcdc5eOxBseHo60tDRXhmDW4MGD8fe//9342cvLy4XR3DVo0CAkJycDAC5fvozQ0FAXR3RHamoqxo8fj1atWrk6FBOnT59GZWUl4uLiMGHCBBw7dszVIeGHH35Ax44dMX36dEydOhUDBgxwdUgmTpw4gXPnzmHcuHGuDsUoLy8Pffv2BQB0794dJ0+eNK47fvw4Hn/8cfj4+CAwMBDh4eE4ffq0q0JtwFrstbW1uHTpEpKSkjB+/Hhs3brVVWGaZen6fP78eYSHh6N58+bw8fFBz549cfjwYRdEaJm135aCggJkZmYiOjoaa9ascXJk1ln77XHn427rN9NRx9ylr/ZHRkaisLDQlSGYVXc7V6fTYebMmZj1/7d3/zFR138cwJ8fkAME8eKHUVO2Q21BgD9rqBkeP7SpSK2jDDj8FVLDIhM7hCmY0kJcLGLIYVtbWq0pWGtNy5w5g6zDJDZpKRDuUiYwITg47uB4ff/wy6fI4/gRt89dvh7/efe5z/vJe/P9ed/7c5/36/XXJU70l2nTpkGj0eDs2bMoKSmROg6qqqrg6+uLlStXoqKiQuo4I3h4eGDbtm1ITExES0sL0tLScObMGUmfaOns7MStW7dQXl6OP/74A6+88grOnDkDQRAky/R3Wq0WGRkZUscYwdZj1QaDYcQtFy8vLxgMBiliWmUre19fH1JSUrBlyxZYLBakpqYiLCwMjz76qISJ/zLa+OzofQ7YvrasW7cOSUlJ8Pb2xo4dO3D+/HmHue1o69rjyP0+1jXTXn3OtZBG0draitTUVCQkJCA+Pl7qOCMUFhbi66+/xt69e9HX1ydplsrKStTU1ECtVuPXX3+FRqNBe3u7pJmGKRQKbNiwAYIgQKFQQC6XS55NLpfjySefhEwmQ3BwMNzd3XHnzh1JMw3r7u5Gc3MzIiMjpY4ygq3Hqv/5Xm9vr0P9hsRWdk9PT6SmpsLT0xPe3t6IjIx0qNWj0Th6n9tCRNi0aRN8fX0hk8kQFRWFhoYGqWONMNq1x9H7fbTc9uxznsBY0dHRga1bt2L37t1QqVRSxxF9/vnn4vKbp6cnBEGQ/PbWxx9/jOPHj+PYsWMICQlBYWEhAgICJM007OTJk+I29Ldv34bBYJA825IlS3Dx4kUQEW7fvg2j0Qi5XC5ppmE6nQ7Lly+XOsY9bD1WHRERgcuXL8NkMqGnpwdNTU0O9di1rewtLS1ISkqCxWLBwMAAfv75Zzz22GNSRR23uXPn4saNG+jq6oLZbEZtbS0WLVokdaxxMRgMWL9+PXp7e0FE+PHHHxEWFiZ1LJGta48j97ut3Pbsc94dyory8nJ0d3ejrKwMZWVlAICjR4/Cw8ND0lyrV6/Gnj17kJycjMHBQeTk5MDd3V3STI5MpVJhz549ePHFFyEIAt5++23JN0RTKpXQ6XRQqVQgIuzbt0/ySeiw33//HbNnz5Y6xj3i4uJQXV2NjRs3io9Vf/jhhwgKCkJMTAzUajWSkpJARNi5c6dD/Z8YK3t8fDyef/55uLm5ISEhAfPnz5c68qi+/PJL9PX14YUXXkB2dja2bdsGIsJzzz2HBx98UOp4Nv09+86dO5GamgqZTIZly5YhKipK6ngia9eexMREGI1Gh+73sXLbq88lf4yaMcYYY2yi+BYSY4wxxpwOT2AYY4wx5nR4AsMYY4wxp8MTGMYYY4w5HZ7AMMYYY8zp8ATmP8xkMuHEiRM2j4mOjobJZBrztcnS6XTi5lwrVqyweaxarYZKpUJjY+OE2+nt7YVarR6zDcbY1JrsOGNLRUUF6uvr72knOjoaAPDbb79Bp9ON69zZ2dmIj4/HDz/8MO72/y49PR3h4eFTNiayqcMTmP+w9vb2MQcWe6usrERbW9u4jy8sLMS8efMm3I6Xl5dDVTRn7H5hj3Fm+/btiIiIGPX9b775ZkJfdHbv3o1ly5ZNKotWq5V8A0xmHW9k52Sqqqpw7tw5GAwGdHZ2IiMjA2vWrMFPP/2E4uJiuLq6Ys6cOXjrrbdQXl6OxsZGlJaWQqVSIT8/HyaTCV1dXcjIyEBsbKzNtlpbW7F3716YTCa4u7vjwIEDsFgs2LVrFwIDA6HX6xEeHo79+/fjzp07yMrKgtlshkKhwKVLl1BcXIyLFy/i6tWrmDdvHsxmM3bt2oVbt25BLpejpKQEbm5uVts+ceIEPv30UwwNDSEmJgavvvoq4uLisGjRIty4cQORkZHo6elBfX09FAoFioqK7NHdjN2X7D3OfPvtt6ipqcG+ffug1WpRV1eHI0eO4IsvvkBraytaWlqwdu1aLFmyBFlZWeju7kZQUBCAu7tqnzp1Cm5ubuLOxfn5+WLto9LSUsycOdPq33X+/HmUlpYCAEJDQ7F//34kJCRg6dKluHbtGhQKBfz8/FBbWwuZTIaKiopRxyjmAKa8vjWzq8rKStq8eTNZLBZqb2+nVatWkdlsptWrV1NHRwcRERUXF9Nnn31Ger1eLCdfXV1Nly5dIiKiy5cv0+bNm4mISKlUUn9//4g2hl/LzMyk7777joiIampq6I033iC9Xk9PPPEE9fT00ODgIK1atYra2tqooKCAjh8/TkRE33//PSmVSiIi0mg0dOHCBSIiCg0NJb1eT0REKSkp9Msvv4xoNyUlhRobG6mjo4Pi4uLIaDSSxWKhgoICMhgMFBISQjdv3iSz2UwLFy6k69ev09DQECmVSvrzzz+JiGj58uVT2+GM3YfsPc4YjUZav349ERG99NJL9Mwzz9DAwABlZmZSY2OjOG4cO3aM3n33XSIiqqurE8eVkpIS+uSTT8Rz63Q6Iro73nz11Vcj/pbhcw0MDJBSqRTzv//++3Tz5k1SKpVUW1tLRERr1qwRx7zk5GRqaGiwmp85Bl6BcUKPP/44XFxc4O/vDx8fH7S1taGtrU2sANrf33/Pb0ECAgJw5MgRnDx5EoIgYHBwcMx2rl27Bq1Wiw8++ABEJH4TCQoKEivsBgQEwGQyoampCc8++ywAYOnSpVbPN3PmTHGren9/fxiNRqvH6fV6zJ8/XyzdkJOTA+BuIcSHH34YADB9+nTxVtOMGTP4/jRjU8ye44yHhwcUCgXq6+sxbdo0LFy4EDqdDq2trZg7d6543PXr17Fy5UoAwIIFC0YtBTJcW8ff3x/9/f1Wj+ns7ISPjw/8/PwAADt27BDfG17J8fHxEdv38fHhccXB8QTGCV29ehXA3QJaBoMBgYGBCAwMRFlZGWbMmIFz585h+vTpcHFxwdDQEADgvffeQ2JiIqKiolBZWYlTp06N2U5wcDC2bt2KxYsXo6mpSfzRnCAI9xz7yCOP4MqVKwgJCUFdXZ34uiAIoP9Xq7D2OWuCgoLQ3NwMs9kMmUyG1157Dbm5ueP+PGPs37P3OBMbG4uioiLExMRgzpw5KC4uvqeYaHBwMOrq6hAbG4uGhgZxQiQIgtjm8L/H4ufnh+7ubnR1dUEul+PgwYPYsGHDuD/PHA9PYJxQR0cHNm3ahJ6eHuTl5cHV1RW5ubnYvn07iAheXl44dOgQvL29MTAwgKKiIjz99NMoKCiAVqvFQw89hM7OzjHb0Wg04v3s/v5+5ObmjnpsWloa3nzzTZw+fRqzZs0SvyktWLAAhw8fnlCRQF9fX6SlpSElJQWCIECpVDpM0TLG7hf2HmeUSiVycnKQl5eHwMBAZGZmIj8/f8QxycnJYkHW4OBgcRU4LCwMhw4dGrFaMxYXFxfk5eUhPT0dLi4uCA0NRXh4+KT6hjkGLuboZKqqqtDc3IysrCypo4xw4cIFPPDAA4iIiEBNTQ3Ky8vx0UcfTegcarUa+fn5ExqU/mnFihWorq6e9OcZY447zkxGdnY21q5di6eeemrS54iOjsbp06cdqtI54xUYNkVmz56NnJwcuLq6YmhoyOZqjS0ajQbvvPPOhB+l7u3txcsvvzypNhlj/21FRUVwc3Ob1KPU6enpaG9vt0Mq9m/xCgxjjDHGnA5vZMcYY4wxp8MTGMYYY4w5HZ7AMMYYY8zp8ASGMcYYY06HJzCMMcYYczr/AySVLHrD0RBEAAAAAElFTkSuQmCC\n", "text/plain": [ "
" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "from matplotlib import pyplot as plt\n", "import numpy as np\n", "import math\n", "\n", "label_dict = {1: 'Iris-Setosa',\n", " 2: 'Iris-Versicolor',\n", " 3: 'Iris-Virgnica'}\n", "\n", "feature_dict = {0: 'sepal length [cm]',\n", " 1: 'sepal width [cm]',\n", " 2: 'petal length [cm]',\n", " 3: 'petal width [cm]'}\n", "\n", "with plt.style.context('seaborn-whitegrid'):\n", " plt.figure(figsize=(8, 6))\n", " for cnt in range(4):\n", " plt.subplot(2, 2, cnt+1)\n", " for lab in ('Iris-setosa', 'Iris-versicolor', 'Iris-virginica'):\n", " plt.hist(X[y==lab, cnt],\n", " label=lab,\n", " bins=10,\n", " alpha=0.3,)\n", " plt.xlabel(feature_dict[cnt])\n", " plt.legend(loc='upper right', fancybox=True, fontsize=8)\n", "\n", " plt.tight_layout()\n", " plt.show()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "
\n", "
" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Standardizing" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "[[back to top](#Sections)]" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Whether to standardize the data prior to a PCA on the covariance matrix depends on the measurement scales of the original features. Since PCA yields a feature subspace that maximizes the variance along the axes, it makes sense to standardize the data, especially, if it was measured on different scales. Although, all features in the Iris dataset were measured in centimeters, let us continue with the transformation of the data onto unit scale (mean=0 and variance=1), which is a requirement for the optimal performance of many machine learning algorithms." ] }, { "cell_type": "code", "execution_count": 7, "metadata": {}, "outputs": [], "source": [ "from sklearn.preprocessing import StandardScaler\n", "X_std = StandardScaler().fit_transform(X)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "
\n", "
" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## 1 - Eigendecomposition - Computing Eigenvectors and Eigenvalues" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "[[back to top](#Sections)]" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The eigenvectors and eigenvalues of a covariance (or correlation) matrix represent the \"core\" of a PCA: The eigenvectors (principal components) determine the directions of the new feature space, and the eigenvalues determine their magnitude. In other words, the eigenvalues explain the variance of the data along the new feature axes." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "
\n", "
" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Covariance Matrix" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "[[back to top](#Sections)]" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The classic approach to PCA is to perform the eigendecomposition on the covariance matrix $\\Sigma$, which is a $d \\times d$ matrix where each element represents the covariance between two features. The covariance between two features is calculated as follows:\n", "\n", "$\\sigma_{jk} = \\frac{1}{n-1}\\sum_{i=1}^{N}\\left( x_{ij}-\\bar{x}_j \\right) \\left( x_{ik}-\\bar{x}_k \\right).$\n", "\n", "We can summarize the calculation of the covariance matrix via the following matrix equation: \n", "$\\Sigma = \\frac{1}{n-1} \\left( (\\mathbf{X} - \\mathbf{\\bar{x}})^T\\;(\\mathbf{X} - \\mathbf{\\bar{x}}) \\right)$ \n", "where $\\mathbf{\\bar{x}}$ is the mean vector \n", "$\\mathbf{\\bar{x}} = \\sum\\limits_{i=1}^n x_{i}.$ \n", "The mean vector is a $d$-dimensional vector where each value in this vector represents the sample mean of a feature column in the dataset." ] }, { "cell_type": "code", "execution_count": 8, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Covariance matrix \n", "[[ 1.00671141 -0.11010327 0.87760486 0.82344326]\n", " [-0.11010327 1.00671141 -0.42333835 -0.358937 ]\n", " [ 0.87760486 -0.42333835 1.00671141 0.96921855]\n", " [ 0.82344326 -0.358937 0.96921855 1.00671141]]\n" ] } ], "source": [ "import numpy as np\n", "mean_vec = np.mean(X_std, axis=0)\n", "cov_mat = (X_std - mean_vec).T.dot((X_std - mean_vec)) / (X_std.shape[0]-1)\n", "print('Covariance matrix \\n%s' %cov_mat)" ] }, { "cell_type": "code", "execution_count": 9, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Covariance matrix \n", "[[ 1.00671141 -0.11010327 0.87760486 0.82344326]\n", " [-0.11010327 1.00671141 -0.42333835 -0.358937 ]\n", " [ 0.87760486 -0.42333835 1.00671141 0.96921855]\n", " [ 0.82344326 -0.358937 0.96921855 1.00671141]]\n" ] } ], "source": [ "import numpy as np\n", "mean_vec = np.mean(X_std, axis=0)\n", "cov_mat = (X_std - mean_vec).T.dot((X_std - mean_vec)) / (X_std.shape[0]-1)\n", "print('Covariance matrix \\n%s' %cov_mat)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The more verbose way above was simply used for demonstration purposes, equivalently, we could have used the numpy `cov` function:" ] }, { "cell_type": "code", "execution_count": 10, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "NumPy covariance matrix: \n", "[[ 1.00671141 -0.11010327 0.87760486 0.82344326]\n", " [-0.11010327 1.00671141 -0.42333835 -0.358937 ]\n", " [ 0.87760486 -0.42333835 1.00671141 0.96921855]\n", " [ 0.82344326 -0.358937 0.96921855 1.00671141]]\n" ] } ], "source": [ "print('NumPy covariance matrix: \\n%s' %np.cov(X_std.T))" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "
\n", "
" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Next, we perform an eigendecomposition on the covariance matrix:" ] }, { "cell_type": "code", "execution_count": 11, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Eigenvectors \n", "[[ 0.52237162 -0.37231836 -0.72101681 0.26199559]\n", " [-0.26335492 -0.92555649 0.24203288 -0.12413481]\n", " [ 0.58125401 -0.02109478 0.14089226 -0.80115427]\n", " [ 0.56561105 -0.06541577 0.6338014 0.52354627]]\n", "\n", "Eigenvalues \n", "[2.93035378 0.92740362 0.14834223 0.02074601]\n" ] } ], "source": [ "cov_mat = np.cov(X_std.T)\n", "\n", "eig_vals, eig_vecs = np.linalg.eig(cov_mat)\n", "\n", "print('Eigenvectors \\n%s' %eig_vecs)\n", "print('\\nEigenvalues \\n%s' %eig_vals)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "
\n", "
" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Correlation Matrix" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "[[back to top](#Sections)]" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Especially, in the field of \"Finance,\" the correlation matrix typically used instead of the covariance matrix. However, the eigendecomposition of the covariance matrix (if the input data was standardized) yields the same results as a eigendecomposition on the correlation matrix, since the correlation matrix can be understood as the normalized covariance matrix." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "
\n", "
" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Eigendecomposition of the standardized data based on the correlation matrix:" ] }, { "cell_type": "code", "execution_count": 12, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Eigenvectors \n", "[[ 0.52237162 -0.37231836 -0.72101681 0.26199559]\n", " [-0.26335492 -0.92555649 0.24203288 -0.12413481]\n", " [ 0.58125401 -0.02109478 0.14089226 -0.80115427]\n", " [ 0.56561105 -0.06541577 0.6338014 0.52354627]]\n", "\n", "Eigenvalues \n", "[2.91081808 0.92122093 0.14735328 0.02060771]\n" ] } ], "source": [ "cor_mat1 = np.corrcoef(X_std.T)\n", "\n", "eig_vals, eig_vecs = np.linalg.eig(cor_mat1)\n", "\n", "print('Eigenvectors \\n%s' %eig_vecs)\n", "print('\\nEigenvalues \\n%s' %eig_vals)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "
\n", "
" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Eigendecomposition of the raw data based on the correlation matrix:" ] }, { "cell_type": "code", "execution_count": 13, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Eigenvectors \n", "[[ 0.52237162 -0.37231836 -0.72101681 0.26199559]\n", " [-0.26335492 -0.92555649 0.24203288 -0.12413481]\n", " [ 0.58125401 -0.02109478 0.14089226 -0.80115427]\n", " [ 0.56561105 -0.06541577 0.6338014 0.52354627]]\n", "\n", "Eigenvalues \n", "[2.91081808 0.92122093 0.14735328 0.02060771]\n" ] } ], "source": [ "cor_mat2 = np.corrcoef(X.T)\n", "\n", "eig_vals, eig_vecs = np.linalg.eig(cor_mat2)\n", "\n", "print('Eigenvectors \\n%s' %eig_vecs)\n", "print('\\nEigenvalues \\n%s' %eig_vals)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "
\n", "
" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "We can clearly see that all three approaches yield the same eigenvectors and eigenvalue pairs:\n", " \n", "- Eigendecomposition of the covariance matrix after standardizing the data.\n", "- Eigendecomposition of the correlation matrix.\n", "- Eigendecomposition of the correlation matrix after standardizing the data." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "
\n", "
" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Singular Vector Decomposition" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "[[back to top](#Sections)]" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "While the eigendecomposition of the covariance or correlation matrix may be more intuitiuve, most PCA implementations perform a Singular Vector Decomposition (SVD) to improve the computational efficiency. So, let us perform an SVD to confirm that the result are indeed the same:" ] }, { "cell_type": "code", "execution_count": 14, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Vectors U:\n", " [[-0.52237162 -0.37231836 0.72101681 0.26199559]\n", " [ 0.26335492 -0.92555649 -0.24203288 -0.12413481]\n", " [-0.58125401 -0.02109478 -0.14089226 -0.80115427]\n", " [-0.56561105 -0.06541577 -0.6338014 0.52354627]]\n" ] } ], "source": [ "u,s,v = np.linalg.svd(X_std.T)\n", "print('Vectors U:\\n', u)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "
\n", "
" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## 2 - Selecting Principal Components" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "[[back to top](#Sections)]" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "
\n", "
" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Sorting Eigenpairs" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "[[back to top](#Sections)]" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The typical goal of a PCA is to reduce the dimensionality of the original feature space by projecting it onto a smaller subspace, where the eigenvectors will form the axes. However, the eigenvectors only define the directions of the new axis, since they have all the same unit length 1, which can confirmed by the following two lines of code:" ] }, { "cell_type": "code", "execution_count": 15, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Everything ok!\n" ] } ], "source": [ "for ev in eig_vecs.T:\n", " np.testing.assert_array_almost_equal(1.0, np.linalg.norm(ev))\n", "print('Everything ok!')" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "
\n", "
" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "In order to decide which eigenvector(s) can dropped without losing too much information\n", "for the construction of lower-dimensional subspace, we need to inspect the corresponding eigenvalues: The eigenvectors with the lowest eigenvalues bear the least information about the distribution of the data; those are the ones can be dropped. \n", "In order to do so, the common approach is to rank the eigenvalues from highest to lowest in order choose the top $k$ eigenvectors." ] }, { "cell_type": "code", "execution_count": 16, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Eigenvalues in descending order:\n", "2.91081808375\n", "0.921220930707\n", "0.147353278305\n", "0.0206077072356\n" ] } ], "source": [ "# Make a list of (eigenvalue, eigenvector) tuples\n", "eig_pairs = [(np.abs(eig_vals[i]), eig_vecs[:,i]) for i in range(len(eig_vals))]\n", "\n", "# Sort the (eigenvalue, eigenvector) tuples from high to low\n", "eig_pairs.sort(key=lambda x: x[0], reverse=True)\n", "\n", "# Visually confirm that the list is correctly sorted by decreasing eigenvalues\n", "print('Eigenvalues in descending order:')\n", "for i in eig_pairs:\n", " print(i[0])" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "
\n", "
" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Explained Variance" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "[[back to top](#Sections)]" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "After sorting the eigenpairs, the next question is \"how many principal components are we going to choose for our new feature subspace?\" A useful measure is the so-called \"explained variance,\" which can be calculated from the eigenvalues. The explained variance tells us how much information (variance) can be attributed to each of the principal components." ] }, { "cell_type": "code", "execution_count": 17, "metadata": { "collapsed": true }, "outputs": [], "source": [ "tot = sum(eig_vals)\n", "var_exp = [(i / tot)*100 for i in sorted(eig_vals, reverse=True)]\n", "cum_var_exp = np.cumsum(var_exp)" ] }, { "cell_type": "code", "execution_count": 23, "metadata": {}, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAagAAAEYCAYAAAAJeGK1AAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3XlUVPX/P/DnsBlruITihkoWaaGyW4ILkiig5AoormWS\nC4oLiqi4kKZSVqSiqQT6C000F0iT3FM/omWYpCGiFCC4kWyyzf394fF+GWEYRBku8nyc4xHm3rnz\nvG848+Leuff9kgmCIICIiEhiNOo7ABERUVVYoIiISJJYoIiISJJYoIiISJJYoIiISJJYoIiISJLq\ntED98ccf8PX1BQCkp6fDx8cHY8aMwdKlS8V1du3ahWHDhsHLywvHjx+vyzhERNSA1FmB+vbbbxEc\nHIzS0lIAwMqVKxEQEIDt27dDLpcjISEBd+/eRXR0NHbu3Ilvv/0WYWFh4vpERNS41VmBMjMzwzff\nfCN+f+XKFdjY2AAAnJyccObMGSQlJcHa2hpaWlowMDBAhw4dcO3atbqKREREDUidFSgXFxdoamqK\n31ecsEJfXx/5+fkoKCiAoaGh+Lienh7y8vLqKhIRETUgWup6IQ2N/6uFBQUFMDIygoGBAfLz8ys9\nXpWLFy/WeUYiIqo71tbWz7S+2gpUly5dkJiYCFtbW5w8eRIODg5455138MUXX6CkpATFxcW4ceMG\nOnfurHQbz7pzUpCZmYnWrVvXd4xnxtzq1bSpHLm5vKhWXYyN5XjwoOGNd0P9/QZqd5ChtgIVGBiI\nRYsWobS0FObm5nB1dYVMJoOvry98fHwgCAICAgKgo6OjrkhEkpGbq4GGOG1zQ33DzMy8DaDh5W5s\n6rRAtWnTBjExMQCADh06IDo6utI6I0aMwIgRI+oyBhERNUAN7xiXiIgaBRYoIiKSJBYoIiKSJBYo\nIiKSJBYoIiKSJBYoIiKSJLXdB0UNS9eurZCbW98paqNh3ttibCwH/14kUsQCRVXijaPqxRtHiSrj\nn2xERCRJLFBERCRJLFBERCRJLFBERCRJLFBERCRJLFBERCRJLFBERCRJLFBERCRJLFBERCRJLFBE\nRCRJLFBERCRJLFBERCRJLFBERCRJLFBERCRJLFBERCRJLFBERCRJLFBERCRJLFBERCRJLFBERCRJ\nLFBERCRJLFBERCRJLFBERCRJLFBERCRJLFBERCRJLFBERCRJLFBERCRJLFBERCRJLFBERCRJLFBE\nRCRJLFBERCRJLFBERCRJWup8sbKyMgQGBiIjIwNaWlpYvnw5NDU1MX/+fGhoaKBz585YsmSJOiMR\nEZFEqbVAnThxAnK5HDExMThz5gy++OILlJaWIiAgADY2NliyZAkSEhLQv39/dcYiIiIJUuspvg4d\nOqC8vByCICAvLw9aWlpITk6GjY0NAMDJyQlnz55VZyQiIpIotR5B6evr499//4Wrqytyc3OxceNG\nXLhwQWF5Xl6eOiMREZFEqbVARUZGwtHREbNmzUJ2djZ8fX1RWloqLi8oKICRkZHS52dmZqoj5guV\nl5fXIHMDrRtk7oY63sytXszdMKi1QL366qvQ0nr8koaGhigrK0OXLl1w/vx52NnZ4eTJk3BwcFD6\n/NatW6sr6guTmZnZIHMDHG91Ym71Ym71y8rKeubnqLVAjRs3DkFBQRg9ejTKysowZ84cdO3aFcHB\nwSgtLYW5uTlcXV3VGYmIiCRKrQVKT08P69atq/R4dHS0OmMQEVEDwBt1iYhIkligiIhIkligiIhI\nkligiIhIkligiIhIkligiIhIkligiIhIkligiIhIkligiIhIkligiIhIkligiIhIkligiIhIklig\niIhIktQ6m3ldW7x4HdLTc+s7hoK9e6fg4cP6TvHsjI3l4N8vRFSfXqoClZ6eiw4dQuo7hoIPP8yD\noWF9p1B082YIIiNDql0nM/M2gIbZGI2IXg4q/0S+ffs2ZsyYATc3N0ydOhX//vuvOnIREVEjp7JA\nBQcHY8iQIfj+++/xwQcfYOHCherIRUREjZzKAlVcXAxnZ2cYGRmhf//+KC8vV0cuIiJq5FQWqPLy\ncly7dg0AxP+JiIjqmsqLJIKDgxEUFIScnBy0bNkSy5cvV0cuIiJq5FQWqC5duiA2NlYdWYiIiERK\nC9SMGTPw1VdfoVevXpWWnT59uk5DERERKS1QX331FQDghx9+gKmpqfh4ampq3aciIqJGT2mB+vvv\nv5GdnY21a9di3rx5EAQBcrkcYWFh2LdvnzozEhFRI6S0QD18+BDx8fG4d+8eDh48CACQyWTw8fFR\nWzgiImq8lBYoGxsb2NjY4MqVK+jatas6MxEREam+iu/27dv4/PPPUVpaCkEQkJubiwMHDqgjGxER\nNWIqb9Rdt24dpk2bBlNTU3zwwQd488031ZGLiIgaOZUFysTEBD169AAADB06FNnZ2XUeioiISGWB\n0tbWRmJiIsrKynDq1Ck8ePBAHbmIiKiRU1mgli5dirKyMvj5+WHXrl3w8/NTRy4iImrkVF4ksXr1\naoSFhQEAvv766zoPREREBNTgCKqkpARXr15FcXExSkpKUFJSoo5cRETUyKk8grp58yY++eQTyGQy\nCIIAmUyGX375RR3ZiIioEVNZoHjPExER1QeVp/iIiIjqAwsUERFJUo0K1M2bN3HixAncvn0bgiDU\ndSYiIiLVn0Ft374dR44cwX///QdPT0+kp6dj8eLF6shGRESNmMojqLi4OGzbtg2GhoYYP348/vjj\nj+d6wU2bNsHLywvDhg1DbGws0tPT4ePjgzFjxmDp0qXPtW0iInp5qCxQTy4tl8lkAAAdHZ1av9j5\n8+fx+++/IyYmBtHR0cjKysLKlSsREBCA7du3Qy6XIyEhodbbJyKil4fKAuXu7o7Ro0cjPT0dH330\nEfr371/rFzt9+jTeeOMNfPLJJ/Dz80OfPn2QnJwMGxsbAICTkxPOnj1b6+0TEdHLQ+VnUGPGjIGD\ngwNSUlLQqVOn52q38eDBA2RmZiIiIgL//PMP/Pz8IJfLxeX6+vrIy8ur9faJiOjlobJA7dq1C2lp\naQgMDMTEiRMxePBgeHp61urFjI2NYW5uDi0tLXTs2BFNmjRRaN9RUFAAIyMjpc/PzMysdvuFhYWS\nK3DFxcX1HaGSwsJClWOZl5ench0pYm71Ym71aqi5a0tlgfr+++/xww8/AAAiIiIwZsyYWhcoa2tr\nREdHY/z48cjOzkZRUREcHBxw/vx52NnZ4eTJk3BwcFD6/NatW1e7fT09PRgaGtYqW12SWiY9PT2V\nY5mZmalyHSlibvVibvVqqLkBICsr65mfo7JAaWhoQEvr8Wra2trixRK10adPH1y4cAHDhw+HIAgI\nCQlBmzZtEBwcjNLSUpibm8PV1bXW2yciopeHygLl7OwMHx8fWFpa4sqVK+jXr99zveCcOXMqPRYd\nHf1c2yQiopePygL1ySefoG/fvkhLS4OnpycsLCzUkYuIiBo5lZeZZ2Vl4fTp07hx4wYSEhIQHh6u\njlxERNTIqSxQ/v7+yM/PR4sWLcR/REREdU3lKT59fX3MmjVLHVmIiIhEKo+gOnfujLi4ONy4cQNp\naWlIS0tTRy4iegH27t0LHx+fWj//o48+wo8//vgCE9Wtfv361Xg2mh49euDff/994Rmed8yfRURE\nBBYtWqSW16oPKo+g/vrrL/z111/i9zKZDFFRUXUaiohenJreGhIeHo709HSsXr1afGzz5s11Fave\n/f7773W27ee5HedZfPzxx2p5nfqiskA9fQl4SUlJnYUhIqKaKS8vh6amZn3HqFMqT/HFxMRgwIAB\ncHZ2Rr9+/eDh4aGOXEQN3u3btzF9+nT07NkTDg4OWLFiBYDHRypz584V18vIyEC/fv3EeSl9fX2x\nbt06eHl5oUePHvDz80Nubi7mzJkDa2trjBgxQpzuJiMjAxYWFgpzWvr6+mL37t1VZgoNDUWfPn1g\nbW2NYcOG4cKFCwCAU6dOYePGjYiPj0ePHj3E2WKebKukpAS2tra4fv26uK379+/D1dUV9+/fBwAc\nO3YMnp6esLW1hbe3N65du6Z0bFJTUzFx4kTY29tj4MCB+OmnnwAApaWl8PT0xPbt2wEAcrkc3t7e\nWL9+vTh2M2bMwKxZs2BlZYWhQ4fi6tWrVb5GUlISvLy8YGtrC0dHRyxfvhxlZWXicgsLC/zzzz8A\ngAULFmDZsmX4+OOPYWVlhVGjRonLqssLALm5uZgyZQqsra0xcuRIpKenK93vjz76CDt27FB4bMiQ\nIWIXB2U/nyf7HhISgrlz58LGxgZ79+6t9Lvk7++PXr16wdbWFr6+vgo/L1X7mJKSIu5jr169sGnT\nJgCPO1ps2rQJLi4ucHBwwKxZs/Dw4UOl+/giqSxQO3bsQHR0NJycnLBy5UqYm5urIxdRgyaXy/Hx\nxx+jbdu2OH78OE6ePIlBgwaJy58+BfT09z/99BPWrl2LU6dOIT09HV5eXhg+fDgSExPRqVMnhds9\nnuV0kqWlJfbv34/ExER4eHhg5syZKCkpgaOjI6ZMmYJBgwbh999/r/S5k46ODt5//30cPHhQIWP3\n7t3RrFkzJCcnY+HChVi+fDnOnz+PUaNGwc/PD6WlpZUyFBUVYdKkSRg8eDDOnTuHL774AsuWLUNq\naiq0tbWxZs0afP3110hNTUVERAQEQYCfn5/4/KNHj2LQoEFITEyEu7s7pk6divLy8kqvo6mpiaCg\nIJw/fx47d+7EuXPn8P/+3/9TOm7x8fGYPn06Lly4gHbt2mHdunVK8y5duhSpqakAgKVLl0JXVxdn\nzpxBaGgoYmNjlY6/m5ubwhhev34dWVlZ6N27d7U/nyfOnDmDgQMH4sKFC+LBQsX96N27N44cOYIz\nZ86gS5culSZGULaPBQUFmDBhApycnHDq1Cn8/PPP4rRzUVFROHr0KHbs2IFTp07ByMhIbb37VBYo\nExMTmJiYoKCgAPb29pKbjJWoon79XoNMhjr79/bbNcuRlJSEO3fuYO7cuWjSpAl0dHRgZWVV4/0Y\nOnQo2rZtCwMDAzg5OaF9+/ZwcHCAhoYGXF1dFT4XfhYeHh4wMjKChoYGxo8fj5KSkhpf+OTu7o64\nuDjx+4MHD8LZ2RnA40mlvby88M4770Amk8HT0xM6OjpVNjg9duwY2rZtC09PT8hkMlhYWMDFxQWH\nDh0C8PjCLD8/P0ydOhWRkZFYs2aNwpvw22+/DRcXF2hqamLChAkoLi7GpUuXKr1O165dYWlpCZlM\nhtatW2PkyJFITEwUlwuCoLC+i4sL3n77bWhoaMDDw0Mc46ryvv/++zh06BDkcjmOHDkCf39/NGnS\nBJ07d8YHH3ygdAxdXFxw9epVcV66AwcOwMXFBdra2gBU/3y6dOkizubTpEmTStsfOnQodHV1oa2t\njalTp+Lq1avIz8+v0T6amJhg/Pjx0NHRgZ6eHiwtLQEAO3fuxMyZM2FiYiJu9/DhwwpH7XVF5WdQ\nhoaGSEhIgEwmQ0xMDHJzc+s8FFFtHT16RxKTaWZlZaF169bQ0FD5N2CVmjdvLn7dpEkThe9feeUV\nFBYW1mq7W7ZsQWxsLO7cuQPg8V/ODx48qNFz7e3t8ejRIyQlJaF58+a4evWqeNoyMzMT+/btE0/N\nCYKAsrIy5OTkVNpOZmYmLl26BDs7O3Hd8vJyDBkyRFxnyJAh+PzzzzFgwAC0a9dO4fmtWrUSv5bJ\nZGjVqlWVr3Pz5k2sWrUKf/75Jx49eoTy8nJ07dpV6f5VvMdTV1dXHGNleT09PXH//n2UlZUpZKru\n909fXx+9e/dGXFwcPvzwQ8TFxYljCKj++ZiYmCjdtlwux+eff47Dhw/jwYMHYqPZBw8ewMDAoNp9\nvH37dqVxfiIzMxPTpk0Tf5cFQYCWlhbu3r1bbZ4XQWWBWrFiBdLT0xEQEIBt27YhODi4TgMRvQxM\nTU2RlZUFuVxeqUjp6uri0aNH4vdP3oxqQ1dXF8Dj01D6+voAgLt371a57oULF7BlyxZERUXh9ddf\nBwDY2dlVOpJQRkNDAwMHDsTBgwfRokUL9O3bV3z9Vq1aYcqUKTW6qszU1BT29vbYsmWL0nWWLl2K\nvn374vTp0/jtt98Ujj5v374tfi0IAm7fvo2WLVtW2kZISAi6dOmCL774Arq6uvjuu+/w888/12hf\na5pXLpdDS0sLWVlZ6NixIwDVbYHc3d0RHh4OGxsblJSUiKfSnvfns3//fhw7dgzfffcdWrdujby8\nPNja2tZ4H+Pj45Uu+/TTT9GjR48abetFUvrn3eXLlwEAly5dwv3795GSkoJevXpVeU6ZiBRZWlri\ntddew9q1a1FUVISSkhL89ttvAIC33noLiYmJyMrKQl5envhhdG00a9YMLVu2xP79+yGXy7F7926F\nD74rKigogJaWFoyNjVFSUoLw8HAUFBSIy1u0aIGMjIxq3xDd3d0RHx+PAwcOwN3dXXx85MiRiImJ\nQVJSEoDHPcdOnDhR5ZFenz59kJaWhn379qGsrAylpaW4fPmy+JnOjz/+iOTkZKxatQoLFy5EYGAg\nioqKxOf/+eefSEhIQHl5OSIjI9GkSRN069atyv01MDCArq4uUlNT8f3336sYzaopy3vjxg1oaGjg\n/fffR3h4OB49eoTr16+rvG/MyckJmZmZ+OqrrxQ+l1T181GlsLAQOjo6MDIyQmFhIcLCwmr8+WSf\nPn1w584dREVFoaSkBAUFBeLPctSoUfj888/Fwnv//n388ssvNc71PJQWqCc3u8XFxVX6R0TV09DQ\nwMaNG3Hr1i306dMHvXv3Fq/8evfddzFo0CAMHjwYw4cPR9++fRWe+6z30CxfvhzffvstHBwckJqa\nqvQvXUdHR/Tq1Uu8KldXV1fh1JSrqysEQYC9vT2GDh1aZRZLS0vo6enhzp07cHJyEh9/++23sXz5\ncixbtgx2dnYYMGAA9u7dW2UOfX19bN26FfHx8XB0dISjoyPCwsJQWlqKrKwsrFq1CqtXr4auri7c\n3d3xzjvvYOXKleLznZ2dER8fD1tbWxw4cADh4eHi5dYV8wYGBuLAgQOwsrLCkiVL4ObmppCjpuOs\nLO+TixeCg4NRUFCAXr16ISgoCMOGDat2ezo6OnBxccHZs2cViryqn48qnp6eMDU1hZOTE9zd3Z/p\niEdfXx/btm3D0aNHxQz/+9//AADjxo2Ds7MzJk6cCGtra3h5eYnFq67JBBXHj6tWrcL8+fPVEqY6\nFy9ehLW1dbXrjB8fgg4dQtQTqIby8vIk17Dw5s0QREaGVLtOQ22Mxtzqpe7cVd1MXBscb/WryXv4\n01R+gnv9+nW1XfNORET0hMqLJFJTU2Fvb49mzZqJh8SnT5+u82BERNS4qSxQx44dU0cOIiKVpk2b\nVt8RSI1UFqhLly5hz5494tV7OTk51V4eSkRE9CKo/AwqJCQEdnZ2yM/PR+vWrWFsbKyOXERE1Mip\nLFBNmzaFu7s7DAwMMH36dGRnZ6sjFxERNXIqC5SGhgZSUlJQVFSEGzdu4L///lNHLiIiauRUFqj5\n8+cjJSUFvr6+mDNnjsqb0IiIiF4ElQXqzJkzeO+992BtbY09e/Zg/PjxaohF1PC5u7srzJ79LBYs\nWIAvv/wSwOM52gYOHFij51XXor2q3lG1VTGfupw/f15sS6HKgQMHMGnSpDrJUV2/rRfteX6HXgYq\nr+IrLy/HhAkT0LFjR4wcORL29vbqyEVUK2vWbMGDB5V7A70o7dsbY9mymTVat2Lfn+dhY2Oj0CCv\nOqpatKurFXldqWl+Dw+Pl6K56ov6HWqoVBaoiRMnYuLEiUhKSsKWLVuwePFiHD58WB3ZiJ5ZRkYe\nunR5vmlwqnPzZkidbZvoicbQzr0mVJ7ie/ToEfbt24cvvvgC//33H6ZPn66OXEQNXr9+/cRJl8PD\nwzFz5kwEBgbCysoKHh4euHLlirhuSkoKhg4dCmtra8yaNQvFxcXisoqntjZv3owZM2YovM6KFSsQ\nGhoKQPH0k1wux2effQYHBwe4uLjg+PHjSvM9yVjT9uGq7N69G4MGDYK9vT0+/PBDcSbs33//HQ4O\nDuLVwFevXoWdnZ3YlK9fv37YtGkT3NzcYG9vj6CgIIWOshU9aUNuZWUFd3d3sW06AOzduxc+Pj7i\n9xYWFoiJicGAAQNgZ2dX6fSksrwA8Ouvv2LgwIGwtbXF8uXLle5zTk4OunXrpjA1XHJyMhwcHFBe\nXo5//vkH48aNg729PXr27Ik5c+YoNBPs168fNm/ejMGDB6NHjx4oLy9X+BklJSVh2rRp1bawr7iP\ny5YtU8i3a9cuDBo0SByvJ80Kc3JyMGPGDPTs2RP9+/dHdHS00n1UN5UFavDgwbh+/TpCQkIQGRmp\nMPsuEdXcsWPH4O7ujosXL6Jv377iG0hpaSkWLVoET09PnD9/Hq6urpX6Fj05teXm5oZTp06JbSzk\ncjkOHTpU5emsnTt34sSJE9i3bx9iY2NrdObjWdqHK5OQkIDNmzfjm2++wdmzZ2FjY4OAgAAAQI8e\nPeDl5YXAwEAUFxdj7ty5mDlzpthLCXj8+dHWrVtx5MgRpKWlYcOGDVW+jpmZGb7//nv89ttvmDp1\nKubOnavQC+vp04HHjx9HbGws9u3bh+PHj4tTtlWX9/79+5g+fToCAgJw7tw5tGvXTmyb8jQTExP0\n6NFDYZwPHjwIV1dXaGpqQhAETJkyBb/++ivi4+ORnZ2Nr7/+WmEb8fHx2Lx5My5cuFDpCEpTUxNT\np05V2sL+6X386aefxH386aef8M0332DNmjX47bffsGHDBhgbG4uZ3nrrLZw+fRqRkZGIiorCr7/+\nWuU+qpvKAhUfH4/Zs2fDzMxMHXmIXlrW1tZwdHSETCbDkCFDcO3aNQCPZ2spLy/H2LFjoampiQED\nBuBtJb3lW7dujS5duuDIkSMAHrfFqdieu6JDhw5h3LhxaNmyJYyMjGrUTLAiVe3Dldm5cycmT56M\njh07QkNDA5MnT1Zocz5t2jQ8fPgQw4cPh6mpqcKRDvD4KPBJ5ilTpiht8TNgwACxQ+zAgQNhZmZW\nbRuIjz/+GAYGBjA1NUX37t3FI4jq8p48eRJvvPGG2GJ+/PjxCl1pn+bm5qbwuVF8fLz4R3379u3R\ns2dPaGlpoWnTphg3blylCyDGjh2Lli1bQkdHp9K2u3btirfeektpC/un99He3l7cx927d+PDDz8U\nOwq3a9cOpqamuHz5MnJzc+Hn5wdNTU20bdsWI0aMkExbJZWfQWlpqVyFiGqg4hvbK6+8guLiYsjl\ncty5c6fSm16bNm2UbsfNzQ1xcXEYMmQIDh48qPSsRk5OTo1bkT+tJu3DlcnMzERoaCg+++wzAI+7\n3spkMmRnZ8PU1BRaWloYOnQoQkNDERQUVOn5FbvjtmnTpsp27sDjxoaRkZHIyMgA8LircHXt658e\n/4ot3ZXlfXoMgccdZpUZMGAAQkNDcffuXbGhoY2NDQDg3r17CA0NxYULF1BYWIjy8vJKM/NU1//p\n5s2bCAkJwfXr15W2sK+upXv79u0rbTMjIwPZ2dkK7ezlcnmNO/HWNVYfonr22muvVWrTnpmZWeUb\nCvC4seDq1auRnZ2NhIQE7Ny5U+l2K7ZHf7oVuZ6entLW88/bPtzPz09p4czOzkZ4eDiGDh2KlStX\nIjY2Ftra2uLyipkzMjJgYmJSaRuZmZlYtGgRoqKixMZ8np6eNW6PXlGrVq2U5r1582al7rFPjgSr\nYmRkhPfeew9xcXFITU1VaJL4+eefQyaTIS4uDoaGhkhISMCKFStqnDMkJATt27fHhg0bnrmFfatW\nrZCenl7pcVNTU7Rt21ayF74pPcWXmJio9B8RPb8nb6bdu3eHpqYmoqOjUVZWhp9//hmXL19W+rxm\nzZrB1tYWCxYsQLt27dCpU6cq1xs4cCCio6ORnZ2N//77r9Il6BYWFoiLi0NZWRkuX76s8Cb1PO3D\nvby8EBERIV5UkZeXh0OHDonLFyxYgJEjRyI0NBQmJiZYt26dwvN37NiB7Oxs5ObmIiIiQqEt+hNF\nRUXQ0NBA06ZNIZfLERsbi5SUlBrle5q3t7fSvL1798b169fFFvPfffcd7t27V+323NzcsG/fPvz8\n888KRa+goAD6+vrQ19dHdnb2M0+6/eT5tWlhP2LECGzdulW8MCc9PR1ZWVmwtLSEvr4+Nm/ejOLi\nYpSXlyMlJaXa3z91UnoE9WTn09PTUVpainfeeQfJycnQ19eX1FUeRBW1aWNYp5eCt29f88mSVb2h\nP1mura2NZcuW4csvv8S6devQu3dvvP/++9U+193dHfPnz8e8efOUvubIkSNx69YtDB48GIaGhpg4\ncaLYxht4fJXe7NmzYWdnBzs7O3h4eIhTmXl6euL06dNwcnKCsbEx/P39lR6pPa1///4oLCzErFmz\nkJWVBUNDQ7z77rtwdXVFVFQU7t+/L16J+Omnn8LT0xP9+vUTu626u7tj4sSJuHPnDpydneHn51fp\nNczNzTFhwgSMGjUKGhoa8PT0hJWVldJM1f0sqsvbtGlTfPnll1i+fDkWLFiAIUOGqGyl7uzsjODg\nYLRp0wZvvvmm+Pi0adMwb9482NjYwMzMDEOGDEFkZGS1GZ9uYb9gwQLs3LkTXbp0gZubG86dO1ej\nfXR1dcV///2H2bNnIycnB23atMGaNWtgamqKiIgIrFq1Cs7OzigtLUXHjh3h7+9f7T6qi8qW75Mn\nT8b69euhpaWF8vJyTJ48uV7abbDl+4vDlu/Sw9yP9evXD6GhoejZs+cL22ZVON7qVyct3yuely4v\nL8f9+/efPRkREdEzUnmRxPDhw+Hm5oY33ngDKSkp+Oijj9SRi4gaoYY+FRO9WCoL1OjRo+Hq6or0\n9HSYmZmhWbNm6shFRI3Q01fMUeOmskClpKRgyZIlePjwIQYPHozOnTujb9++6shGRESNmMrPoFas\nWIGVK1eiadOmGD58eKWpOYiIiOqCygIFPJ7zSiaToVmzZtDX13/uF7137x769OmDtLQ0pKenw8fH\nB2PGjMGPvI6kAAAS30lEQVTSpUufe9tERPRyUFmgXn31VcTExKCoqAhxcXEwMjJ6rhcsKyvDkiVL\n8MorrwAAVq5ciYCAAGzfvh1yuVxhRmIiImq8VBaoTz/9FP/++y+aNm2KP//8U5zWv7Y+++wzeHt7\nw8TEBIIgIDk5WZyrysnJSWH6fyIiarxUXiRhYGCACRMmiP1pCgsLK01wWFN79uxB8+bN8d5772Hj\nxo0AoNB+Wl9fH3l5ebXaNhERvVxUFqiQkBCcPHlSPOKRyWSIiYmp1Yvt2bMHMpkMv/76K65du4bA\nwECF2YcLCgqqPYX49GSXTyssLJRcgavYeE4qCgsLVY5lXl6eynWkiLnVi7nVq6Hmri2VBSopKQkJ\nCQnQ0KjR9RTV2r59u/j12LFjsXTpUqxevRqJiYmwtbXFyZMn4eDgoPT5qqb40NPTk9y0QgAkl0lP\nT0/lWDbUKVWYW72YW70aam6g+lnglVFZoMzMzFBcXAxdXd1ahVIlMDAQixYtQmlpKczNzeHq6lon\nr0NERA2LygKVlZWFvn37ih11n+cUX0VRUVHi15wdnYiInqayQIWFhakjBxERkQKlBeqHH37AiBEj\nEBMTU2kCx4CAgDoPRkREjZvSAtWqVSsAUNqtk4iIqC4pLVCOjo4AAA8PD1y+fBllZWUQBAE5OTlq\nC0dERI2Xys+gpk2bhtLSUuTk5KC8vBwmJiZwd3dXRzYiImrEVN7c9ODBA2zZsgWWlpbYs2ePJG88\nJSKil4/KAvVkUteioiK88sor7HhJRERqobJAvf/++wgPD4eFhQVGjhwJHR0ddeQiIqJGrkYt35/o\n3bs3OnToUJd5iIiIAFRToAICApSezuPNu0REVNeUFigvLy915iAiIlKgtEDZ2dkBeNyefcOGDbh5\n8yY6d+6MKVOmqC0cERE1Xiovkpg5cybMzc0xZ84ctG3bFvPmzVNHLiIiauRUXiQBAN7e3gAACwsL\nHDp0qE4DERERATU4gurUqRP279+P7OxsHD16FMbGxkhLS0NaWpo68hERUSOl8gjqxo0buHHjBn74\n4QfxscWLF0Mmkyn0dCIiInqRVBaotWvXomXLluL3V65cQdeuXes0FBERkcpTfJMmTcLp06cBAFu3\nbsXChQvrPBQREZHKAhUZGYmtW7fC09MTmZmZ2LVrlzpyERFRI6eyQF27dg137txBt27d8Ndff+H2\n7dvqyEVERI2cys+gvv76a0RERKB169a4dOkSpk6digMHDqgjGxERNWIqC9SOHTugqakJAOjevTu+\n//77Og9FRESk9BTfzJkzAQCamprYunWr+Pgnn3xS96mIiKjRU1qg7t27J359/Phx8WtBEOo0EBER\nEVCDiyQAxaLEjrpERKQOSgtUxULEokREROqm9CKJ69evY/bs2RAEQeHr1NRUdeYjIqJGSmmBWrdu\nnfh1xeaFbGRIRETqoLJhIRERUX2o0UUSRERE6sYCRUREksQCRUREksQCRUREksQCRUREksQCRURE\nksQCRUREkqSy3QaRMosXr0N6em59x1BQWFgIPT29+o6hoH17YyxbNrO+YxA1OCxQVGvp6bno0CGk\nvmMoyMvLg6GhYX3HUHDzZkh9RyBqkHiKj4iIJEmtR1BlZWUICgpCRkYGSktLMWXKFLz++uuYP38+\nNDQ00LlzZyxZskSdkYiISKLUWqD279+Ppk2bYvXq1Xj48CGGDBkCCwsLBAQEwMbGBkuWLEFCQgL6\n9++vzlhERCRBaj3FN3DgQPj7+wMAysvLoampieTkZNjY2AAAnJyccPbsWXVGIiIiiVJrgdLV1YWe\nnh7y8/Ph7++PWbNmKXTr1dfXR15enjojERGRRKn9Kr6srCxMmzYNY8aMgZubG9asWSMuKygogJGR\nkdLnZmZmVrvtwsJCyRW44uLi+o5QSWFhocqxzMvL43i/IC9qvKWIudWroeauLbUWqLt372LSpElY\nvHgxHBwcAABvvfUWEhMTYWtri5MnT4qPV6V169bVbl9PT09ylxgDkFwmPT09lWOZmZnJ8X5BXtR4\nSxFzq1dDzQ08Pjh5VmotUBEREXj48CHWr1+Pb775BjKZDAsXLsSKFStQWloKc3NzuLq6qjMSERFJ\nlFoL1MKFC7Fw4cJKj0dHR6szBhERNQC8UZeIiCSJBYqIiCSJBYqIiCSJBYqIiCSJBYqIiCSJBYqI\niCSJBYqIiCSJBYqIiCSJBYqIiCSJBYqIiCSJBYqIiCSJBYqIiCSJBYqIiCSJBYqIiCSJBYqIiCSJ\nBYqIiCSJBYqIiCSJBYqIiCSJBYqIiCSJBYqIiCSJBYqIiCSJBYqIiCSJBYqIiCSJBYqIiCSJBYqI\niCSJBYqIiCSJBYqIiCSJBYqIiCSJBYqIiCSJBYqIiCSJBYqIiCSJBYqIiCSJBYqIiCSJBYqIiCRJ\nq74DEFHNLV68DunpufUdQ1RYWAg9Pb36jqGgfXtjLFs2s75j0AvAAkXUgKSn56JDh5D6jiHKy8uD\noaFhfcdQcPNmSH1HoBeEp/iIiEiSWKCIiEiSWKCIiEiSJPEZlCAICAkJwbVr16Cjo4PQ0FC0a9eu\nvmMREVE9ksQRVEJCAkpKShATE4PZs2dj5cqV9R2JiIjqmSQK1MWLF+Ho6AgA6NatG/788896TkRE\nRPVNEqf48vPzFS5V1dLSglwuh4aGJOonETVSvO9Mtbq870wmCIJQJ1t+BqtWrUL37t3h6uoKAOjT\npw+OHz+usM7FixfrIRkREb0o1tbWz7S+JI6grKyscOzYMbi6uuLSpUt44403Kq3zrDtGREQNmySO\noCpexQcAK1euRMeOHes5FRER1SdJFCgiIqKn8SoEIiKSJEl8BlWV4uJizJ07F/fu3YOBgQFWrVqF\npk2bKqwTGhqK3377Dfr6+gCA9evXw8DAoD7iqrzZ+OjRo1i/fj20tLQwbNgwjBgxol5yPk1V7sjI\nSOzevRvNmjUDACxbtgwdOnSop7SK/vjjD6xduxbR0dEKj0t1rJ9QllvKY11WVoagoCBkZGSgtLQU\nU6ZMQb9+/cTlUh1zVbmlOuZyuRzBwcFIS0uDhoYGli5ditdff11cLtXxVpX7mcdbkKht27YJX3/9\ntSAIghAXFyesWLGi0jre3t7CgwcP1B2tSj///LMwf/58QRAE4dKlS4Kfn5+4rLS0VHBxcRHy8vKE\nkpISYdiwYcK9e/fqK6qC6nILgiDMmTNHuHLlSn1Eq9bmzZsFd3d3YdSoUQqPS3msBUF5bkGQ7lgL\ngiDExsYKn376qSAIgpCbmyv06dNHXCblMa8utyBId8yPHDkiBAUFCYIgCP/73/8azPtJdbkF4dnH\nW7Kn+C5evAgnJycAgJOTE86ePauwXBAE3Lp1C4sXL4a3tzdiY2PrI6aoupuNU1NTYWZmBgMDA2hr\na8Pa2hqJiYn1FVWBqpukr1y5goiICPj4+GDTpk31EbFKZmZm+Oabbyo9LuWxBpTnBqQ71gAwcOBA\n+Pv7A3j8V7KW1v+dfJHymFeXG5DumPfv3x/Lly8HAGRkZODVV18Vl0l5vKvLDTz7eEviFN/u3bvx\n3XffKTzWokUL8XSdvr4+8vPzFZYXFhbC19cXEyZMQFlZGcaOHYt33nmnykvU1aG6m42fXqavr4+8\nvLz6iFmJqpuk3dzcMHr0aBgYGGDq1Kk4ceIEevfuXV9xRS4uLsjIyKj0uJTHGlCeG5DuWAOArq4u\ngMfj6+/vj1mzZonLpDzm1eUGpD3mGhoamD9/PhISEvDVV1+Jj0t5vAHluYFnH29JHEENHz4cBw4c\nUPhnYGCAgoICAEBBQUGlpmi6urrw9fVFkyZNoK+vDwcHB1y9erU+4gOAQl4ACm/yBgYGCgW2oKAA\nRkZGas9YlepyA8C4ceNgbGwMLS0t9O7dG8nJyfURs8akPNaqSH2ss7KyMG7cOHzwwQcYNGiQ+LjU\nx1xZbkD6Y75q1SocPnwYwcHBePToEQDpjzdQdW7g2cdbEgWqKlZWVjhx4gQA4MSJE7CxsVFYnpaW\nBm9vbwiCgNLSUly8eBFdu3atj6gAFPM+fbOxubk5bt26hYcPH6KkpASJiYno3r17fUVVUF3u/Px8\nuLu7o6ioCIIg4Ny5c/U6xlURnrpLQspjXdHTuaU+1nfv3sWkSZMwd+5cfPDBBwrLpDzm1eWW8pjv\n27dPPAXWpEkTaGhoiH84Snm8q8tdm/GW7H1Qjx49QmBgIO7cuQMdHR2EhYWhefPmiIyMhJmZGfr2\n7YutW7ciPj4e2tra8PT0xKhRo+otr1DFzcZXrlxBUVERRowYgePHjyM8PByCIGD48OHw9vaut6wV\nqcq9f/9+REVFoUmTJujZsyemTZtWz4n/T0ZGBmbPno2YmBgcPHhQ8mP9hLLcUh7r0NBQ/PTTT+jU\nqRMEQYBMJsPIkSMlP+aqckt1zIuKirBgwQLcvXsXZWVlmDx5MgoLCyU/3qpyP+t4S7ZAERFR4ybZ\nU3xERNS4sUAREZEksUAREZEksUAREZEksUAREZEksUAREZEksUDRS+H8+fN49913MXbsWIwdOxZe\nXl7Yvn17pfVOnTqFH3744Zm2vXfvXhw7duyZnpORkVGv9+XVhR07dtR3BGpkJDEXH9GL0LNnT4SF\nhQEASkpK4OrqCk9PT4UWLE8mxn0WT89AUFMymaxWz5OqDRs2YPTo0fUdgxoRFih6aVS85zw/Px9a\nWlrQ1NSEr68vmjdvjocPH2LQoEG4desWvLy8MHv2bJiamuLWrVuwtLRESEgI7t+/j/nz5+Phw4cA\ngM8++wwHDhxAixYt0KlTJ2zcuBEymQz37t3DiBEjMHr0aCQmJop39RcWFiIsLKzSrNlPrF+/Hr/8\n8gvkcjm8vb0xcuRIcUYULS0t2NraYvbs2QgPD8etW7fw4MED5ObmYvTo0Th8+DBu3bqFzz77DM2b\nN4e/vz9MTExw+/ZtODo6YtasWcjIyEBQUBDkcjkAIDg4GG+++SYGDBgAKysrpKWloXnz5ggPD0d5\neTmWLFmC9PR0yOVyzJw5E7a2thg8eDDs7Oxw7do1yGQyrF+/Htu3b0dubi6WLVuGsWPHYsGCBdDS\n0oIgCAgLC0PLli3r/gdMjQ4LFL00zp07h7Fjx0Imk0FbWxuLFi0SZ7P28PCAs7Mz9u7dKx7Z3Lx5\nE9u2bUOTJk3Qv39/3Lt3Dxs3boSzszNGjRqFS5cu4fLlywD+72goJycHP/74I8rLy+Hh4YGBAwci\nJSUFa9euxWuvvYaIiAgcOnQI7u7ulfL99ddfOH36NGJjY1FWVoawsDD8/fffOHz4MHbt2gUNDQ3M\nmDEDx48fB/B4QuQ1a9Zg06ZNOHnyJDZu3Ig9e/YgLi4OY8eORWZmJrZt2wZ9fX34+PggOTkZGzdu\nxPjx49G3b19cvXoVQUFBiI2NxT///IOoqCi0bNkSPj4+uHz5Mq5cuYJmzZohNDQUubm5GDNmDA4e\nPIj8/Hx4eHggODgYc+bMwcmTJzFlyhRs374dixcvxo4dO9CtWzfMnTsXiYmJyMvLY4GiOsECRS+N\niqf4nlZV104zMzOxgJmYmKC4uBhpaWkYPnw4AKB79+7o3r07wsPDxef06NEDWlpa0NLSQufOnfHP\nP/+gZcuWWL58OfT19ZGdnQ0rK6sqM6SlpcHS0hLA47YmgYGBOHToELp16yZOqGllZYWUlBQAQJcu\nXQAARkZGMDc3F78uLi4GAFhYWIiz/FtaWiItLQ1paWnixMoWFhbIzs4GADRt2lQsIqampiguLsbf\nf/+Nixcv4o8//oAgCCgvL8eDBw8AAG+99Za4bklJicJ+jBgxAps2bcKkSZNgZGRUqYUF0YvCiySo\nUajYQqQqT04Pvv7660hKSgIAJCYmYu3atQrrJScnQxAEFBUV4fr16zAzM8OiRYuwatUqrFy5EiYm\nJpW2+USnTp1w5coVAEBpaSkmTpyIjh07IikpCXK5HIIg4MKFC+jYsSMA1Z9hXb9+HcXFxSgvL0dS\nUhJef/11dOrUSWxe99dff6FFixaVtvUkl7m5Odzd3REVFYVvv/0Wrq6uMDY2rvY1ASAhIQE2NjaI\njIzEgAEDsHnzZpXPIaoNHkHRS0/ZG33Fx598PXnyZAQFBWH//v3Q0NBAaGgofvzxR3G9srIyfPjh\nh8jNzcUnn3wCY2NjDBkyBD4+PtDT00OLFi2Qk5NT5etaWFjA0dERXl5eEAQB3t7eePPNN+Hq6io+\nZmNjg/79+9eot5m2tjb8/f1x9+5duLq64s0338S8efOwaNEibN26FWVlZfj000+V7vfIkSOxaNEi\n+Pr6oqCgAN7e3pDJZFWOC/C4oM2bNw/Tp09HYGAgNmzYALlcjqCgIJVZiWqDs5kT1dD58+exc+dO\npacR1aliyw6ilxVP8RERkSTxCIqIiCSJR1BERCRJLFBERCRJLFBERCRJLFBERCRJLFBERCRJ/x9E\n5RfWMHDEKAAAAABJRU5ErkJggg==\n", "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "with plt.style.context('seaborn-whitegrid'):\n", " plt.figure(figsize=(6, 4))\n", "\n", " plt.bar(range(4), var_exp, alpha=0.5, align='center',\n", " label='individual explained variance')\n", " plt.step(range(4), cum_var_exp, where='mid',\n", " label='cumulative explained variance')\n", " plt.ylabel('Explained variance ratio')\n", " plt.xlabel('Principal components')\n", " plt.legend(loc='best')\n", " plt.tight_layout()\n", " plt.savefig('/Users/Sebastian/Desktop/pca2.pdf')" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The plot above clearly shows that most of the variance (72.77% of the variance to be precise) can be explained by the first principal component alone. The second principal component still bears some information (23.03%) while the third and fourth principal components can safely be dropped without losing to much information. Together, the first two principal components contain 95.8% of the information." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "
\n", "
" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Projection Matrix" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "[[back to top](#Sections)]" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "It's about time to get to the really interesting part: The construction of the projection matrix that will be used to transform the Iris data onto the new feature subspace. Although, the name \"projection matrix\" has a nice ring to it, it is basically just a matrix of our concatenated top *k* eigenvectors.\n", "\n", "Here, we are reducing the 4-dimensional feature space to a 2-dimensional feature subspace, by choosing the \"top 2\" eigenvectors with the highest eigenvalues to construct our $d \\times k$-dimensional eigenvector matrix $\\mathbf{W}$." ] }, { "cell_type": "code", "execution_count": 19, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Matrix W:\n", " [[ 0.52237162 -0.37231836]\n", " [-0.26335492 -0.92555649]\n", " [ 0.58125401 -0.02109478]\n", " [ 0.56561105 -0.06541577]]\n" ] } ], "source": [ "matrix_w = np.hstack((eig_pairs[0][1].reshape(4,1), \n", " eig_pairs[1][1].reshape(4,1)))\n", "\n", "print('Matrix W:\\n', matrix_w)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "
\n", "
" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## 3 - Projection Onto the New Feature Space" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "[[back to top](#Sections)]" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "In this last step we will use the $4 \\times 2$-dimensional projection matrix $\\mathbf{W}$ to transform our samples onto the new subspace via the equation \n", "$\\mathbf{Y} = \\mathbf{X} \\times \\mathbf{W}$, where $\\mathbf{Y}$ is a $150\\times 2$ matrix of our transformed samples." ] }, { "cell_type": "code", "execution_count": 20, "metadata": {}, "outputs": [], "source": [ "Y = X_std.dot(matrix_w)" ] }, { "cell_type": "code", "execution_count": 24, "metadata": {}, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAagAAAEYCAYAAAAJeGK1AAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzs3XtYlGX6wPEvhxkYBVQIFVaFPJT2cw20NEvXUGi1zbQj\nYJLptElqmaes7GAWauWhg5onUitFy9WsNNs8tK1thSmErdWmCSmgIB7RgRng/f2BTBxmmBmYI9yf\n6/Iqhpn3vR1f5uZ53ue5by9FURSEEEIIN+Pt6gCEEEIIUyRBCSGEcEuSoIQQQrglSVBCCCHckiQo\nIYQQbkkSlBBCCLfk6+wTVlRU8Oyzz3Ls2DG8vb158cUX6dq1q7PDEEII4eacPoLas2cPXl5epKWl\nMXnyZBYtWuTsEIQQQngAp4+gYmNjGTx4MAC5ubm0atXK2SEIIYTwAE5PUADe3t489dRT7Nq1izff\nfNMVIQghhHBzXq4sdVRUVMR9993Hjh078Pf3Nz5+4MABV4UkhBDCSfr06VPv950+gtq2bRunTp3i\nkUcewc/PD29vb7y9694KsxS4u8rLyyM8PNzVYTSIxO46nhy/xO46nhy/NQMRpyeo2267jaeffprR\no0dTVlbGrFmzUKvVzg5DCCGEm3N6gtJoNLz++uvOPq0QQggPIxt1hRBCuCVJUEIIIdySJCghhBBu\nSRKUEEIItyQJSggnKSwsZP/+/RQWFro6FCE8giQoIZxgU1oa3SMiSI6Lo3tEBJvS0lwdkhBuTxKU\nEA5WWFjIBK2WvTodB86fZ69OxwStVkZSQlggCUoIB8vOziZSrabXla97AREqFdnZ2S6MSgj3JwlK\nCAeLjIwkW68n68rXWUCOwUBkZKQLoxLC/UmCEsLBQkNDWZaaSoxGQ++gIGI0GpalphIaGurq0IRw\nay5ptyFEcxOfmMjg2NjK6b7ISElOQlhBEpQQThIaGiqJSQgbyBSfEEIItyQJSgghhFuSBCWEEMIt\nSYIS9ZLyPEIIV5EEJcxKS9tERER34uKSiYjoTlraJleHJIRoRiRBCZMKCwvRaieg0+3l/PkD6HR7\n0WonyEhKCOE0kqCESdnZ2ajVkVCtQI9KFSHleYQQTiMJSpgUGRmJXp8N1Qr0GAw5Up5HCOE0kqCE\nSaGhoaSmLkOjiSEoqDcaTQypqctko6kQwmmkkoQwKzExntjYwVKeRwjhEpKgRL2kPI8QwlVkik8I\nIYRbkgQlhBDCLUmCEkII4Zaceg+qrKyMZ555htzcXAwGA8nJyQwePNiZIQhhN4WFhbKARAgHcuoI\n6uOPP6ZNmzasX7+eVatW8dJLLznz9ELYzaa0NLpHRJAcF0f3iAg2paW5OiQhmhynjqCGDRvG0KFD\nAaioqMDXVxYRCs9TWFjIBK2WvTodvXQ6soAYrZbBsbEykhLCjpyaITQaDQDFxcVMnjyZKVOmmH1u\nXl6es8Kyq4sXL0rsLuDM2DMzM4nw9a1WBAo6+fqyf/9+oqKiGnRMee9dw5NjB8+P3xKnD2Hy8/OZ\nNGkSo0eP5vbbbzf7vPDwcCdGZT95eXkSuws4M3aVSkVOWRlZVCanLOD3sjJuvPHGBo+g5L13DU+O\nHTw7/vz8fIvPceo9qNOnT6PVapkxYwZ33XWXM08thN2EhoayLDWVGI2G3kFBxGg0LEtNlek9IezM\nqSOoFStWcOHCBZYtW8bSpUvx8vJi9erVqNVqZ4YhhE1MrdaLT0xkcGysrOITwoGcmqBmzZrFrFmz\nnHlKIRplU1oaE7RaItVqsvV6lqWmEp+YCEgZKCEcTTbqCmFG9dV6B86fZ69OxwSt1mObNhYWFrJ/\n/36PjV80P5KghDAjOzubSLW6xmq9CJXKrZs2mktCsm9LeCJJUEKYERkZSbZeX61lI+QYDG7btNFc\nEnKnkaCM4oQtJEEJYYYnrdarLwm5y0gwLS2NiK4RxN0fR0TXCNI2yihO1E9KOQhRD09ZrWdMQjod\nUDMJVR8JVu3bcvZIsLCwEG2yFt0oHbr2OjgJ2vFaYodI9Q1hnoyghLAgNDS0UZtwnaG+6Uh3GAlm\nZGTg3cob2l95oD2ogt37fp5wPRlBCdEEGJOQVkuESkWOwVAjCblyJLgpLY3kceO4XF4CJ6lMUifB\ncMZ97+cJ9yAJSggXKywsJDMzE5VKZUwctrbyKCwspHPXruw7cIDi4mKTr3PFvq2qe2P/Kinhv8C4\n1VASCJoyDakr3fN+nnAfMsUnhJUcsQKtauXdswkJxpV3ti4Jr774oM9NfThy9IjbfPBXX6CRCPxe\nBtfoW/LR5o9ITEh0dXjCzUmCEsIKjthHVH3l3cGLF40r75LHjbN6SXj1xQfnHzqPbpQO7Xj32Uxc\n+95YPnC6ooLo6GhXhiU8hCQoISxw1D4iU8u/O3h7087Hx+ol4dnZ2aiD1W67+MAdFmgIzyX3oISw\noL4l3I35oDW1/PtERQUVimL1kvDIyEj0Z/RuvfjAU5bqC/cjCUoICxy1j6j6yrtOvr78XlbGstRU\nALOr8UwdI3VFKtrxWlTBKgxnDKSucL8RihTWFQ0hCUoICywt4W6MqtHF/v37a+y1smXEkZiQSOwQ\n145QbF11KIQ1JEEJYQVHTlOFhoYSFRVV45i2jjhcOULZtnUrz86YYbIliRCNIQlKCCu50zSVu4xY\nCgsLmTV9Ol+WlNBLpyOLyunJwbFSwkg0nqziE8KNmdp75U6tM7Kzs4lUqVxeiFY0TZKghHBTphKR\nO7XOgCsLSAwGj2lJIjyLJCgh3JC5RJSRkeEWrTOqhIaGkrJwoexzEg4h96CEcEPm9l4BLm+dUduI\nkSO597773OKemGhaZAQlhBsy1z4jOjraLSszWNuSRDrqCltIghLCDdVXIig+MZGfc3JYsWsXP+fk\neMySbumoK2wlU3xCuKmmVCJIOuqKhpARVBMhUydNk6mpM3daZm4tdy9qK9yTJKgmIC1tExER3YmL\nSyYiojtpaZtcHZKwQkN+qXC3ZebWqlHUFtyyqK1wPy5JUD/88ANJSUmuOHWTU1hYiFY7AZ1uL+fP\nH0Cn24tWO8HtP7Cau9qjoDffeMOqfzNTLTo8YWNsVVFbzQYNQWuD0GzQuGVRW+FenJ6gVq9ezbPP\nPovBYHD2qZuk7Oxs1OpIqPaRpVJFuP0HVnNmahT01quvck2nThan62qv7vsSOFpaSkBAgKPDbrTE\nhERyjuSw64Nd5BzJkY66wiKnJ6iIiAiWLl3q7NM2WZGRkej12VBtQbLBkCNTJ27M1CjoWmBpSYnF\n6brqq/s6+/tzOxDh7c2APn084l6UtcvRhQAXJKi4uDh8fHycfVqPYet9idDQUFJTl6HRxBAU1BuN\nJobU1GXyAeDGTO5xAuKwbrouPjGRfQcOcFZR+BbI0uk84l6ULOQRtnLbZeZ5eXmuDqFBLl682ODY\nt27dxvTps1CpIjEYslm4MIWRI0dYfN2gQQP57rsvOX78OB07diQkJKRBMTQmdlfztNhfXrCAW6dN\nI7ikhDPA20A+lVUi/P39Lf5djh07xtVqNb1KS4HKUVgnX1/2799PVFSUo8OvwZr3ftvWrcyaPp1I\nlYpsg4GUhQsZMXKk1ecoKiqqcX3bi6ddN7V5evwWKS5w4sQJ5f777zf7/e+//96J0dhXbm5ug15X\nUFCgaDTBCvyggKLAD4pGE6wUFBSYfX56errZ7zdEQ2N3B54Ye0FBgZLy0ktKa39/JSowUAnWaJSN\nGzZY/dpgjUb5ofJiUX4AJVijsev1YC1L731jY92wYYOiCdIorSJbKZogjbIhzbr3yBqeeN1U58nx\nW/M5b3aKLzMzk7vvvpvExES+//574+MTJ060S2L08vKyy3GaClsWO8iy8qYhNDSUZ559lv/9/jsp\nGzfaXBVi2jPPMMjf361KHpnSmJWH1Tf4nn/oPLpROrTj3XsqU9iP2QQ1f/58Fi5cyJw5c0hJSWHf\nvn0AXLhwodEn/dOf/sTGjRsbfZymxNrFDrKsvOkx1VG3PlVL1P+xYAHeXl7cO2OGW5c8CggI4EhJ\nCV9e+dqWAreywbd5M5ugVCoVV199Nd26dWPlypW88sor/PLLLzLycRBrFzvIsvLmpfbCAlNL1BfO\nneviKM1LS0ujz019MFzlTYwvRGr8bRrtyQbf5s3sIomWLVvy7rvvkpCQQGhoKAsWLOCJJ55Ar9c7\nM75mJTExntjYwfXWXqs50qpsuCDLypumTWlpTNBqiVSrydbrWZaaSpuQEDp6e5ucLnO36b3q03O0\nB07CyfcVMtIP0qNHD6uOUbXBVzteiypYheGMQTb4NiNmE9SCBQtYs2YNer0etVrNtddey1tvvcWi\nRYucGV+zExoaWu8PX9VIS6uNQaWKwGDIkWXlTVD1kVIvnY4sYMCYMfj4+KAvKXGrflDmVE3P6dpX\n9rSiPfhd5UdxcbFNx0lMSCR2iP2L5hYWFpKZmYlKpZKfHzdlNkEFBATw2GOP1Xisa9euLFu2zOFB\nifpZM9KqUlhYaPZ59X1PuFbthoVhQLnBwD6DgZ+AW4Fg4KwbL47Q6/VcPnUZjgFX06jpOUu/uNkq\nLS0NbbIW3za+lJ0tI3VFqlS2cENSLNZDWbMjv77VfrIS0L3V3sz7BRBO5agpHvgF8GnZkrSPPnLL\nxRGPPfYYA2IGYNAYYD2o3lC5Tf296lOPF8delJWBbsxtN+qKxqm+2k+nq5wM0mpjiI0dDGDye999\n9yXh4eEujVtUMpY00mqJUKk4ptejVFSQpdfTi8pNvacrKoiOjnZ1qEZVI3K9Xs+SFUtAi/Hek2G1\ngb1f7uWWW25xdZgmpx6rVga6OnmKmsyOoMrLy9Hr9UyaNAmDwYBer6e0tJQHH3zQmfGJBqpvtd+K\nFavQ6YLrfO/48eNSjsaNVO+c+7/ff2fF2rVu1+q9SvXq7H+NiYEgaiwNpxUcOXLEhRH+QVYGeg6z\nI6h//OMfLF++nNOnTzN06FAURcHb25sbbrjBmfEJG1X9FhsQEGBytV9AQAApKa8BXnW+l5X1I/fe\nm4RaXblSMDV1GYmJ8a77y4ga916qOuxmZGQA1Bk9OfueYtX5Ll++XGNBx0fAXeepTABXRlBcgL59\n+zo8JmtUXxno29qXsnNlbjH1KEywVGriww8/tEdVC5s0x1JH9rBhw0ZFowlWWrXqrWg0wcqkSY8r\nGk2wEhQUrWg0wcqGDRuV9PR0pVWr3gpsVCBYgWgFWihPPfW04u/fxupSS+7Gk0u+KIr18W/csEEJ\n1miU3q1a1SiNZO5xR6lefkjdUq1EajTKlQtHUUAJUfkq+KIQgoIKZdJjkxwaT0MUFBQo27dv95hr\n3BRPvu6t+Zy3mKDy8vKUlStXKm+99Zbxj6NJgrKduVp+hw8frlGzr+bzChRYr/j7t1Y+//xzJTAw\nuvpnjBIUFK2kp6e75O9jK0/+QVUU6+I3V9Pu8OHDTq3LV1BQoGiCNArJKMym8r++KHtrnX/fvn3K\n2rVrlcOHDzskDntoDteNu7Lmc97iIonJkyfTv39/wsLCnDGgEw1Udc+pctEDVN1XKi4u5sYbbwT+\nmJJZvHg+U6ZU30e1nOjoaAyGbGQDsPuqvfS8apNuenq6yccdddPf1CIDTTsNI05X0MXPjxyDgWWp\nqdxyyy0WF0XIVgdRH4sJqmXLlkyZMsUZsQgTrP0BtlRhIi1tE1rtBOP9pcWL59O7d1SN4y5cmML0\n6bIB2F1VX3pefZNu3759TT7uqF8uaiwyqLrHdBG+zciguLjY6mRTtRdJHaxGf0Yve5FEXZaGWCkp\nKcqnn36qHD16VPntt9+U3377zS7Du/rIFF+l2veUNmzYaNXzq99zUhRF+eSTTxQfH40CW+u9v5Sb\nm+uQNh7O4IlTHdXfa1vvQUUHBZm8B1X7cUfZkFZ5DyooMkjxD/S3uQWGqWlCTZCmxlS0qevQ3ten\nJ1431Xly/Ha5BzV69Ogaf5KSkuwSXH0kQdneH6r666r/AMfFDVNAo0C3K/993Oz9JU++2D0t9tqL\nGpYtXWr1a6398K7vw9weH/RVx8jKyrL5tenp6UqryFaVyenKn6DIICU9Pd1s/ydH9IXytOumNk+O\n3y4JSlEU5cKFC8pPP/2kFBcXNzooa0iCUqqttjO/aMHSh8y+ffuuJKU/klzl11vNjqA8lSfFbmqx\nQxt/f5t/+ahPfav67L3iryHvvbkR1OHDh216vLEjKU+6bkzx5Pgb1bCwyueff05SUhIzZsxg7dq1\nUovPSSz1h7KmVNE///lPoAPUqH39J3x8Rsn9JRdqSAO/6hthu0dEsCktzexzTbXkmKCtLOVT3/ec\nqWovkmaDhqC1QcYySMXFxSb7P6Wnp0tfqGbIYoJas2YNH3zwAa1bt2bChAns2rXLGXE1e/X1hzLV\ntHDs2GR++ukn4+sLCwvp2LEjcILqSQ5y+eijD2QDrgvVrrNnaVGDrUmlvgTYmO62DVFfZZLEhERy\njuSw64Nd5BzJITEh0WyVh759+0r1h2bI4io+Hx8f1Go1Xl5eeHl5odFonBGXwHzVclNLyktLQ4mO\nvok1a1YCGFfseXlVoCg3AX8Ccrnttlu54447XPL3EZVq19nLMRhIWbDA7IjW3PJyc8vIza32q/ow\nd/SKv6qVpwcPHmTKk1PqXaVXu0q5uf5PPXr0kL5QzZGlOcCFCxcqU6ZMUW677TblueeeU+bNm2eX\n+cf6yD2o+plaQFFZFWKv4u/fus73/PyClGnTpin79u1zeeyO4omxW7uKr6CgQGmlVte4Z9VKra73\n/kt9q/rsveKveuxVCxkCIwIVVCjENuyekazis44nx2+XjbpTp07lq6++4rrrrqNLly7ExMQ4I2+K\nelRN/40dO4jS0lCgCFgG3IqPT1tAQ/X7Tn5+XYiPjzdu2BXuofroIS8vDzC/761cUbgViASyr3xd\nn6q6faaOVf17AQEBFBcXU1hY2OjRyE8//cTYR8ZSOrr0j/1R64BobK4Ybq7/k737Qgn3ZvEeVHFx\nMcXFxVx11VWcP3+ejz76yBlxCQsSE+PJyPgPfn6ngH9Q2SUoi/LyAioqjvPHfacvKS09SkBAgMti\nFdYxtxAiOzuba1q04BdgBZW9oLppNMb7Rubu89TXMyw0NJTfjhxhQJ8+xvOtWrGiwZXsN6WlcVN0\nNAZ1ac0q5kHAOZx6z6j2+yEV+j2YpSFWUlKSMm3aNGXBggXKggULlIULF9pleFcfmeKznqnNuVWP\n+ftfrYBG0Wj+bNVGX0+eLvDk2BVFUbKysoxLzwtAWQ9K6ytLz83V4CsoKLB5yXjVFJmp+n0aUP4c\nGGjztF9V7HtB0fhSs0afCiWgY4Dd9i1ZUnuv1KTHJtW7d8rTrxtPjt9uG3WdTRKUbUzNyx8+fFjx\n82tt00ZfT77YPTl2RVGU7du3K71btVI2ghIMSm9QWoCS8tJLiqKYvm9UX+IypXoya+3np1xdqwJ5\nL1DSG1Bstip2BZQNV5KUdzCKX4CfsnzFcqdVJqmzt2pMZYKsb++Up183nhy/Xe5BXXvttfzwww/0\n6NHD+JharXboqE7YxtS8fHFxMf7+nSktrduwUObw3U/Hjh35rbSUR4Ev+WOFXczcufx9/HiT95T2\n799v9eq+6kvVe+l0ZAE3XTnXrVfOdYLKe1yh9RzHXOxVKwMTgbAyGHHJj28zMmp8bjhanSK2auo0\nTpTOuZ7FYoJKT09nz549xq+9vLzYvXu3Q4MSjWepeKxwLyEhIcyYNYs1zz1nco9S1S8h1T9YLS0n\nr87UUvUuGg0jKirorFbzy8WLaK8819al5yEhIXWWza9MTTWZnBxZvbxOEVs9cIEaRW1l75SHsXY4\ndvr0aaWsrKxRQzpryRSffZgrHmuOO8VuK0+OXVH+KNRra18nc1N/tafV6usllfLSS0qgSqV0vTKt\nGKBS2XQPquq9t7QE3Nr7ZY1ZSl69iG31e1BVX8s9KPdhl3tQ3377rTJ48GBlxIgRSkxMjMW9NPWp\nqKhQnn/+eSU+Pl5JSkpSfv/9d5PPkwRlP7b8sLtb7Lbw5NgV5Y/4G7JHqfq/sTU1+BpzH8vUORvT\nbLH2eexRENaWorlN5brxRHZJUAkJCcrJkycVRVGUkydPKvfee2+DA/rnP/+pPPXUU4qiKEpmZqby\n6KOPmnyeJCjXkNhdp3r8DR1BWJMEah87PT3duMCh6k90UJDZTsoFBQXKSy+/pGgC/0giS5dZrsRu\nzXksteBwhKZ03VjLXVrq2KVYrI+PD+3atQOgXbt2+Pn5NXg68cCBAwwcOBCA66+/nh9//LHBxxLW\nkT0gnqe+/Uv1sabOXtWxAfbv309AQIDVdQHT0tLo1LkTz734HLoHdJx/6Dy6UTqmzZxm8fqypv5g\n1SIHKQjrOGlpaUR0jSDu/jgiukaQttF80WF3YDFBBQQE8N577/Hzzz/z3nvv0apVqwafrLi4mMDA\nQOPXvr6+VFRUNPh4zZW1Sceaiuei6bC2CG31DcED+vRhtFZLjEZD76AgYjQalqXWrXFXWFiINllL\nyW0lcBU1kohva1927NhR7/VorD9Yz3nMFYqVRQ32UfVvqBv1xy8X2vHOr2RvC4ur+F577TWWLVvG\n66+/TufOnZk7d26DTxYQEMClS5eMX1dUVODtbTpHVpV+8TQXL150aOxbt25j+vRZqFSRGAzZLFyY\nwsiRI+o8r6ioiHHjHqWk5MsrRWWzGDfuVnr2vI6QkJAGxV5UVMTx48fp2LGj2WO4iqPfd0ezV/wv\nL1jArdOm1ShCazAYjMcuKiri0XHj+LKkxLjc/NbVq9m6cyeXLl0y/tvWjiUzMxOf1j7QBfiMGivj\nivOLmfjsRMofK2fhKwsZOWKkydgGDhrEl999V+Maqn2eBfMXMG3mNFRtVBjOGljwSs347a05XTeZ\nmZn4tvGt88vF/v37iYqKclyQjWHNXOE///lPZcWKFcqePXsaNef4+eefG+9BZWRkKH//+99NPk/u\nQZlmS5ddaxoe2hK7re3nna053kswp757DLbec6p+TL8Av8r7Q/ei4I9CGxR8G14QtiHx21tzum5c\ncY+vPna5BzVr1ix27NiBn58fH330UaNGUHFxcajVahISEpg/fz5PP/10g4/VHFW12aheCLZq8y3U\nnPqz1PDQFqb6T2m1E9x6aqA5q+8elq29qKof842Fb8BqCNgN/ga49ywQAAy48iQ73TNq6D04UT9z\nTSLd+X22OMX3v//9jw8//BCAMWPGcP/99zf4ZF5eXrz44osNfn1zV9/m27S0TcYeUHp9NlptEmVl\neqA/EIZafZrU1BUNuhhN9Z+SqhSeyVQvKlP3nEwZ/8h4vBWYPnkyXTUqdpWVodZXoD+pl42wHiIx\nIZHYIaar3LsjiwmqU6dOxjnjoqIiwsLCnBGXMKGqzYZWG4NKFYHBkENq6jIA4winMol8yZIltwPf\nAmHAF3h7TyQ2dnCDzitVKZqW+lpxWPL38eMZeffdxtfu2r2LcY+MQx2iliaCHsKTWpZYTFCZmZkM\nGzaM8PBwTp06hVqtZsCAyjH9vn37HB6gqMlUl939+/fXGuG0BDryx1TgKNTqBQ0e8ZhLjJ5ykYu6\nbP2Qql2iqOq1iQmJ9Py/npSUlNj1N3JHlkRyJE+N211ZTFBSd8/9mKrJVnOEcwk4TvURj053hLNn\nz9ZpTFf9B6o+5trPi6ZvU1oaE7RaItVqsvV6lqWmEp/4R+v2kJAQwsPD7Xa+tLQ0tMlaY6v4xa8t\npnd07wZfd85KGrXjNtXiXtjI0iqK3bt3KxMnTlQefvhh4x9Hk1V8tqtdd8/b21+BNgpEKxCggEYJ\nDKxZk6/2yrylS5e5JHZ7aE6rsZzJmuoU9l6BWGOlWWxly4zAiMAGlT6yVDrJXrG7aoWcu1431rBL\nu41XXnmFOXPmNGqDrnC86iOcs2fPcv/9T3P+/E4gg8omCHu5eLFyNDV27CA6depQ675VFtOm3cp9\n990royNhZKoKui2tOKxRfYRTo2XGJeBrQAsX21+Ek6AdryV2SKxV566+MVXXXmfz621Rp9WHtPaw\nC4sJqlu3bvTr188ZsYhGqpr6KywsvDLllw+0oeb9qF6UloYSEzMUX99QTC1Zlx8oUcWWlh4NUXv6\ncP7ixX9UkyinUf2cjEkjUAe5QGvHJY06rT5kRaNdWExQQ4YMIT4+ns6dOxsfmzdvnkODEo1TfVGD\nj084xcVHocZHTBEGwycYDLdTvWWdrMwTphZDNHRZujXnqt1EMWbKFBa/vpgpM6bg08qH4jPFDf7Q\nj4yM5PKpy/AW0Bo4C5e5bJdr3NT7lLoiFe14Lapgld1XNDbbxReW5gDvuusuZfv27cpXX31l/ONo\ncg/KPqp25C9fvvJK+/duCgQrsFEBRdFoeip+fkHG+1bV70G5S8Vja7nT+94Q7hB/fa06HNGyIj09\nXflzYKCSDkpBraoWxmt3xfJ6+znVp6CgQFG3UNe4L4QvyvIVyxsVe333tRzxc1Pf+dzhumkou7Tb\nMFeOyJEkQTWOqR+Sw4cPK35+QQrsrVEm6fDhw3X6+pgqa+TuCcsd3vfGcHX8De0LpSgNj3358uUK\nvigBbVA0vijzzZyzoddeenq6EhgRWJmcqv60Q/Fr6Wc8lq2xO3sxhKXzufq6aQy7LJLw9/dHq9Vy\n3XXX4eXlBcDUqVMdPrJr6hw1ZK9dUSI1dRmJifH06NGDNWtWotXeU2MvU/W23Hl5efz000+MHfsI\npaXb0OluBbIYM2YAvr6qOscUTYczFkNUV1hYyJQnp8DDUHxl+u6p1bBy8eI652voxtLIyEj0RTXv\nC3EBVFc1/O/l7MUQzX3xhcUEFRMT44w4mhVzSaSxqtfMq1qZp9XGEBXVi+LiYmJjB5OT87PZxLh1\n6zamTXuG0tL2wD3AMmAwBkM5BsO+GseMjR3cLH5AmgtLiyEs/UJl6y9cpj54A8MDierd225/p6r6\ngcmTkiEYuADcAuXp5Q2+D+XsxRDNfvGFpSGWwWBQ3n//fWX27NnK2rVrldLSUrsM7+rTlKf4bKlI\nbitTFczVLMOeAAAgAElEQVShi+Lr21IJDPyz4ucXpCxfvtJsXP7+bWrEVXm/6m0FutpUFd0VPHmq\nQ1HcI35z7ebruzelKIqybMmSer9vijOnyl5b8Jqi8lcpLcNbNugeTu0pxg1pGxp8X6wh6jufO1w3\nDWWXe1BPP/20Mm/ePOWLL75QUlJSlBkzZtgluPo05QTVkDYY1jKV/ECjwHNXks31CmhMJqn09HQl\nMDC6VnLrpqjVAYpa3cohCdWePPkHVVHcJ/7aH8aW7k0VFBQobfz9G3Tvyhkf9FULDAIjAhW/AL8a\nCyQUxfL7bm6BgrPvyZo7n7tcNw1hlwQ1atSoGl/Hx8c3PCIrNeUE5cgRlKIoyksvpSjQ4koFidYK\ndLqSnP44n59fa5M3omuPoPz8WiuHDx+uU6XC3XpBKYpn/6AqinvEb+pD0FL/qPT0dCU6MNCq/lKm\nju/ID3prRmlZWVlmz+9u/ZNMcYfrpqHs0g+qtLQU3ZUbpyUlJZSXlzt82rEpq9qjpNHEEBTUG40m\nxq6FV8eP/zv+/mpgOvAf4Cy1N+qq1ZF1evaEhoaycGFKjbjWrFlOjx49SEyMJyfnZ3btWkFOzs+y\nQKIJqt4GvntEBJvS0gDL/aMiIyPJNhhsajNf/fiO7P1UdZ/L1EZfqKyd1/eWvsTdH0dE1wjSNqbZ\n9HrhBJYy2LZt25S4uDhlwoQJym233aZ8+umndsme9WnKI6gq9vrN0dRxqo94fH1bXpnmszxiy83N\ndfvl5OZ48m+SiuLa+C1N45m7N1Vl2dKl9X6/MUvYTcVq7fVZUFCg+Af4K9yNwoyaIyBrRkcygnIs\nu0zxKYqinD17VsnKylLOnDnT6KCs0RwSlD3U14a9+g9y1UbdwMCoep/nyRe7J8euKK6N35o28JY2\n6jamzby1ScdS4VdTz1e3UCsEVxac9fX3VV56+SXj+VpFtqqxRyoosu7UpLMXRNjKk6/7RiWoixcv\nKlOnTlUuXryoKIqifPzxx8oTTzxh/NqRJEFZZuu9rPpGWlLN3PXceQRliTX3Xc0d39qkY+toxmRV\ndN8/qqJXVaiw5njuPKvgydd9o+5BvfDCC/z5z3+mZcuWAAwbNoyePXsye/ZsZ80+inpUtWE3VezV\nlNpz/dX3TJ0/fwCdbi/Tps2isLDQGeELN2Kst6fR0DsoiBiNxm719uo7PmCsNn7+ofPoRunQjtea\nvAZtvR9U4/lVVdEfhotjL6IbpWPKjCksfm0x/uv9CVobhGaDxmztPEfeJxP1M7tRNy8vj4ULF/7x\nRF9ftFot8fFyg9wdNLYNe1WC+6MLr1Qzb86q2sBnZGQAEB0d7ZDj1+kEbWWVBFs3rNZ4vpmq6L2j\ne5P+dbrduwE3VLMtCFsPsyMoX1/TuUulUjksGGG9xq4GrJngQKqZiz27dpE4ciRP339/jZV29lJ7\nJFIjiUC9SaeqWrhmg8biiKf28wN2BsAZTJ4nJCTELUZHaWlpRHSNMLuisNkyN/f3zDPPKF988UWN\nx3bt2qU88cQTjZ98tEDuQVmvMfPjtfc3yT0o17F3V1pbrwlXFItVFNsXIdj6d7NUFd0drps698vG\nVBa0PXz4sMXXukP8DdWoYrEzZ85k6tSpLF26lA4dOpCfn09wcDCvvvqqM/OnsKChhTShZhfeyMhI\nDAaDnaMTzla7AeCy1FTiExMtvs6exWJtmapKTEgkdkis1c9v6PV+9113c/ddd5s8j6un1mrUJfwR\n2A6lLUqJ7hvNmlVrSEyw/O/XZFnKYLm5uUpGRoZy8uRJu2RNa8gIyjUkdtexR/yNGQXZawRlqW6f\nM1mzQnDJ0iWKJlCjtOzYUtEEumYZuXEENQYFDTbtu/Lk694ulSTCw8OJioqiXbt2zsiXQogGMo6C\nrnxdfRRkiT1W8lXvkHvg/Hn26nRM0JpeledohYWFFlcIFhYWMmXGFHQP6LikvYTuAR0PaR9yerxV\n98v8NvtBC6RyRTUWE5QjfPHFF0ybNs0Vp26WCgsL2b9/vywhb+IslSWyJD4xkZ9zclixaxc/5+RY\nNTVYXWMSpL1Zsyw9IyMDg7+hxnP0/nrjSkZnSkxIJGN/Bn6lflYtGmkunJ6gUlJSWLx4sbNP22yl\npW0iIqI7cXHJRER0Jy1tk6tDEg5ij1FQY/b8NDZB2lN9KwSrfmE7d+4cXKTGc7jo9FCNevTowZpV\na6xeqdgcmF0kER8fb+ygW0VRFLy8vNi4cWODT9i7d2/i4uLYtEk+KB2tsLCQceOSKSlZik4XB+RL\ns8EmztR+I2cxJkitlgiVihyDweoE2ZiFCqZeWzVtph2vRRWswnDGQOqKVHbt2oU2WYs6WI2+SI8X\nXihrFWgNnAOVr8rue8BsYeuikabObIJatGhRow68efNm1q1bV+OxefPmMWzYMNLT0xt1bGGdFStW\nUVKiBxYCjwHLZDNuM9CYlZ2N1ZAEmZaW9kfSOKMndUWq1SvX6ntt7Q97gIiuEehG6SpXzB0Dn40+\n+Hj7oPJSUe5bzjur3nH5z4Yr//3cjZeiKEp9T8jJyWHnzp3GJcgFBQXMmTOnUSdNT09n06ZNNSpV\nVHfgwAHCwsIadQ5XuXjxIoGBga4Og6KiIvr2HURJyZf80cT7Vvz8Kti//9+EhITUeY27xN4Qnhw7\neHb8jYm9qKiIvrf0peSBEmOFCP/1/qR/nV7nGi0qKuL48eN07NiRkJAQm14LkJmZScKEBC6OvWhc\nzk0LUOvUPDHpCUaPHm3yde7Mk6+b/Px8+vTpU+9zzI6gqkybNo24uDgOHjxI27ZtuXz5st0CrE94\neLhTzmNveXl5bhF7bm4ufn5XU1JS/ZZ1MGVlefz442GTPZ3cJfaG8OTYwbPjb0zsubm5+IX4UdK+\npPKB9qAOUVNSUlLjmKZGSl27dLXqtVVUKhVlZ8vgGJXJaUzla/Qn9by1/C2mT5/ucSMXT75u8vPz\nLT7H4iKJFi1aMH78eNq1a8f8+fM5ffq0XYIT9lF1w/enn36qsVLPVCkjOEt5+Q602gmyok+4BWvK\nHZlbMh4QEGB1qSSQ5dyeyGKC8vLyorCwkEuXLnH58mW7jKD69u1rdnpPWK9qhd6gQVquu64PgwbF\nG1fqVdXq8/MbBFwDxADLgP/D2/sqlyylFaI2a2rsmVsyXlxcbFN9PpDl3B7H0k7e9PR0Zf369cqu\nXbuU/v37K/Pnz2/8FmILpJKEZab6QUGwAntr9IU6fPiw4ucXpMBeBTYq0EaBrnUaFzozdkfw5NgV\nxbPjt1cVDHM19iz1gmpI7cGqGoCBnQLdshGhtTz5umlULb4qN954I126dOH48ePs2LGD1q1bOyNv\nCgtMtcuACKBljZV6PXr0YM2alYwbd9eVFX3fAL3Q6bJkyblwG/WtXDO3ZLz6knJbr+GqFX779+93\ni2rmwjSLCWr9+vWsW7eObt26ceTIESZMmMCIESOcEZuoh6l+UJADXKrTNiMxMZ6QkDbcffeTXLok\n/Z+E53HE/qDQ0FCioqLk+ndjFhPUhx9+yCeffIKfnx86nY7Ro0dLgnIDVfeYtNoYIByd7ij+/u3w\n8rrHZF+o6OhoKiqO09AGh0K4muwPan4sJqiQkBB8fHwA8Pf3lyk+N1K9XUZAQADFxcVmf7usntBU\nqggMhhybGhwKIYSzWUxQiqIwcuRIoqOjOXz4MGVlZcZCr7ISz/Vs+a2ydv8nSU5CCHdmMUElJycb\n/3/48OEODUY4nkyTCCE8hdkEtXfvXmJiYvjtt9/qFI2Nj69bhUAIIYSwJ7MJ6ty5cwBSOaIZc3Ur\nbCFE82a2ksRdd90FVE7rRUZGMmnSJEpKShg5cqTTghOuI32khBCuZrHU0cyZM+nQoQMAgwYNYtas\nWQ4PStiXrR11CwsL0WonoNPt5fz5A+h0e6V+nxDC6azqqBsVFQVUVpWoqKhwaEDCvuobCZlLXFVV\nKqDupl4hhHAWiwkqKCiITZs28csvv/Dhhx/SsmVLZ8Ql7KC+kVB9ictUJXTZ1CuEcDaLCWr+/Pkc\nOXKE1157jaNHjzJ37lxnxCXswNxIKCMjw2TiKioqAv7Y1KvRxBAU1BuNJkY29QohnM7iPqjg4GCS\nk5MpLS0FoKSkxOFBCfswVa/PYMgBqFNoVqWK4Pjx4/z5z38GZFOvEML1LCao2bNn89VXX9G2bVsU\nRcHLy4uNGzc6IzbRSObKG0VHR5tMXB07dqzzeklMQghXsZigsrKy2LVrF97eVq2nEG7G3EjIVOIK\nCQlxcbRCCPEHiwkqIiKC0tJSNBqNM+IRDmBqJGQqceXl5bkoQiGEqMtigsrPzycmJoaIiAgAmeJr\nQmQKTwjhziwmKKlYLoQQwhXMJqgPP/yQ++67j40bN9YpFjt16lSHByaEEKJ5M5ug2rdvD1Teg6pq\nWCicQ4q0CiFEPQlq4MCBAOzYsYN33nnHaQE1d2lpm9BqJ6BWV+5hSk1dRmKitDcRQjQ/VpU62r17\nN0ePHuXYsWMcO3bMGXE1S44q0mprsVghhHAHFhdJFBUVsXbtWuPXXl5evPvuu46MqdmqKk1Uu8JD\ndnZ2g6f6ZEQmhP3I9Ltz1ZugiouLWblypeyBchJzpYkaWqS1+oisMullodXGEBs7WH64hLBRWloa\n2mQt6mA1+jN6UlekkpiQ6OqwmjSzU3zvv/8+d955JyNGjODf//63XU5WXFxMcnIySUlJJCQkkJmZ\naZfjNhX2LtIqbTOEsI/CwkK0yVp0o3Scf+g8ulE6tOO1Mm3uYGZHUJ9++ik7d+6kuLiYJ5980rho\nojHWrFnDzTffzIMPPsixY8eYNm0aW7ZsafRxmxJ7Fmm194hMiOYqOzsbdbAaXXtd5QPtQRWsatT0\nu7DMbIJSq9Wo1WqCg4MxGAx2OdnYsWNRq9UAlJWV4efnZ5fjNjX2qvBgrlis/EAJYZvIyEj0Z/Rw\nEmgPnATDGYP8sudgFhdJACiKYvOBN2/ezLp162o8Nm/ePHr27ElhYSFPPvmktI93AmmbIUTjhYaG\nkroiFe14LapgFYYzBlJXpMrPk4N5KWayz80330z//v1RFIVvv/2W/v37G7/XmPJHv/zyC9OnT2fm\nzJkMGDDA5HMOHDhAWFhYg8/hShcvXiQwMNDVYTSIxO46nhx/c4q9qKiI48eP07FjR7eo/u/J731+\nfj59+vSp9zlmE1R6errZF/Xt27dBAR05coTHHnuM119/nWuvvdbs8w4cOGAxcHeVl5dHeHi4q8No\nEInddTw5fonddTw5fms+581O8TU0CdVn0aJF6PV6UlJSUBSFoKAgli5davfzCCGE8HxW3YOyl2XL\nljnzdE2abBgUQjR10ibXA6WlbSIiojtxcclERHQnLW2Tq0MSQgi7kwTlYRxVr08IIdyNJCgPI9Uh\n3NMnn3yCVqt1dRhCNCmSoDxMzeoQINUhnGfw4MF88803Jr83fPhwUlNTnRbLkiVLePLJJ512PiFc\nQRKUh7F3vT7ReOXl5a4OQYgmSRKUB0pMjCcn52d27VpBTs7P0j4DyMjIICpqIO3bd2PUqIe5ePGi\nw861detWEhMTmTdvHv369WPJkiVs3bqVUaNGGZ8zd+5cbr75Zvr06cOdd97JkSNHTB7r7NmzJCcn\nM3z4cPr168fo0aON3ysoKODxxx+nf//+xMbG8t577wHw73//m+XLl7Njxw6io6MZOXKk8fmPPvoo\n/fr1469//Ssffvih8VhZWVncc8899OnThwEDBvDKK68Yvzd58mQGDBjAjTfeSFJSktlYhXA2py4z\nF/Zjr3p9TUFOTg7x8Q9x6dLrQB+2bHmZM2ceYufOfzjsnFlZWdxxxx188803lJWVsX37dry8vADY\nt28fBw4c4J///CcBAQH89ttvBAUFmTzOmjVraN++Pdu2bSMsLMxY4V9RFJKTk4mLi2Px4sXk5+cz\nduxYOnfuzMCBA0lOTub333/n1VdfNR5rypQpdO/enTfffJOjR48yduxYOnXqRL9+/Zg7dy5jxozh\nzjvvRKfT8euvvxpfN2jQIObPn4+vry8LFixg+vTpfPTRRw5774SwloyghMfbvXs3inIb8ADQndLS\n1eza9Yndihyb0q5dOx544AG8vb2NBZCr+Pr6cunSJY4ePYqiKHTu3JmrrrrK5HF8fX0pLCwkPz8f\nHx8f4876Q4cOce7cOR599FF8fHzo0KED9913H9u3bzd5nJMnT5KZmcn06dNRqVR0796d++67z5ho\nfH19+f333zl79iwajYZevXoZX3v33Xej0WhQqVRMnDiRn3/+meLiYnu8TUI0ioyghMdr2bIlXl4n\nAQXwAk7h46PC19dxl3f79u3Nfu+mm25i9OjRvPjii+Tn5xMXF8fMmTO5cOECt99+O1DZmfrgwYNo\ntVrjggdfX1/uu+8+HnnkEXJzczl16pSxoouiKFRUVHDjjTeaPGdBQQGtWrWq0Vw0PDyc//73v0Dl\nlOMbb7zBsGHD6NixIxMnTuTWW2+loqKCRYsW8fnnn3P27Fm8vLzw8vLi7NmzBAQE2OvtEqJBJEEJ\nj3fnnXfSvv1cTpxIpLS0Dy1arOLZZ2cbp9wcwdKxR48ezejRozlz5gyTJ08mNTWVxx9/nIyMjBrP\na9myJTNnziQpKYnLly/z4IMP0qtXL8LCwujQoQOff/65VfG0bduW8+fPc/nyZVq0aAFUFuNs27Yt\nAJ06dTIWef788895/PHHSU9PZ+fOnezdu5d169YRHh7OxYsXzSZBIZxNpviEx9NoNHz22WZefLE3\nEyfms379qzz99AyXxXPo0CGysrIoKyvD398fPz8/vL1N/6h9+eWX/P7770BlsvLx8cHb25tevXrR\nsmVLVq1aRWlpKeXl5fz6668cOnQIgKuuuorc3FxjK5z27dsTHR1trHf5888/s3nzZkaMGAHAxx9/\nzJkzZwAIDAzEy8sLb29vLl++jFqtJigoiMuXL7Nw4UKHJnYhbCEjKNEkVI5EHLsvyNoP7uLiYubN\nm8eJEyfw8/NjwIABZjfxZmdnM2fOHM6cOUPr1q154IEHjNN6K1asYP78+QwZMgSDwcDVV1/N5MmT\nARg6dCgff/wx/fr1o0OHDmzZsoWFCxfywgsvMHDgQFq1asXkyZO56aabgMqVf/Pnz6ekpIQ//elP\nLF68GLVazciRI9m3bx9/+ctfaN26NZMnT2bTJimd1VBSI9O+zLbbcCVpt+EaErvreHL8EnultLQ0\ntMla1MFq9Gf0pK5IJTEh0S7HNseT33trPudlik8IIRqpsLAQbbIW3Sgd5x86j26UDu14rdTIbCRJ\nUEIIYaPCwkL2799vTEDZ2dmog9VQtbizPaiCVVIjs5EkQQkhhA3S0tKI6BpB3P1xRHSNIG1jWmWN\nzDN6OHnlSSfBcMYgNTIbSRZJCCGElapP5ena6+AkaMdryTmSQ+qKVLTjtaiCVRjOGEhdkSoLJRpJ\nElQTJauJhLC/qqk8XXtd5QPVpvISExKJHRIrP3d2JFN8TZB03BXCMSxN5YWGhnLjjTdKcrITSVBN\njHTcFcJxQkNDSV2RimaDhqC1QWg2aGQqz4Fkiq+Jqeq4q9PV7bgrP0RCNJ5M5TmPjKCaGOm46xru\n3vLdHvGlp6czaNAgO0Xk2WQqzzkkQTUx0nHXcdyp5but7BWf1OkTziRTfE1QYmI8sbGDZQrCScrL\ny/Hx8XF1GCiK4rYJpKKiwmzBXCHMkSumiWpuUxAZGRkMjIqiW/v2PDxqlMe0fL/99tv517/+Zfy6\nvLyc/v3789NPPwGQmZlJQkICN954IyNHjiQ9Pd343KSkJBYvXkxiYiJRUVGcOHGCLVu2EBsbS+/e\nvYmNjeXTTz81xlw9vl9//ZVx48bRr18/BgwYwMqVKwHQ6/WkpKQwcOBA/vKXvzB37lyzjR+PHj1K\nUlISw4cPZ/jw4ezZs8f4vaeffprZs2fzyCOPEB0dzXfffWfr2yyEjKCE58vJyeGh+Hhev3SJPsDL\nW7bw0Jkz/GPnToed014t3++44w4++eQTpk6dClRWHQ8ODqZHjx6cOnWK8ePHs2DBAgYOHMg333zD\nY489xs6dO2nTpg1QeW9p1apVXH311Vy+fJmUlBS2bNlCREQEp0+f5vz588ZzVcV36dIlxo4dy8MP\nP8zy5cspKyszJtC3336bQ4cO8fHHHwPw6KOP8vbbb/P444/XiLusrIxHH32Ue++9l7lz55Kbm8uE\nCRPYsmWL8X7n9u3bWbVqFVFRUej1eju986I5ceoISqfTMWHCBEaPHs24ceMoKChw5ulFE7V7925u\nU5QrDd9hdWkpn+za5REt3++44w727Nlj/AD/9NNP+dvf/gZU9nC69dZbGThwIAD9+/enZ8+eNUZc\nd911F126dMHb2xsfHx98fHz43//+R2lpKVdddRVdunSpc84vv/yStm3b8tBDD6FWq2nRooWxBfyn\nn37KxIkTadOmDW3atGHSpEls27atzjEyMzO5fPkyjzzyCD4+Ptx0003ExMQYR2wAQ4YMISoqCqDO\neySENZyaoD744AN69uzJ+++/z/Dhw1m1apUzTy+aqJYtW3LSy4uqvjGnAJWPj1u0fL/55pt5/vnn\nuXTpEvn5+URHRxMdHU3v3r2Byk63Xbt25T//+Q8lJSXs2bOH4cOHA5WtFD777DP69u1L3759ufHG\nGzl48CCnT582GYdGo2Hx4sWkpaUxYMAAkpOT+e233+rEl5+fT8eOHU3GXlBQUKN9Q3h4uMlfJAsK\nCggLC6vxWO3n1vceCWENp07xjRkzxtgBNC8vj1atWjnz9KKJuvPOO5nbvj2JJ07Qp7SUVS1aMPvZ\nZz2i5TtU3ofavXs3rVq1olu3bsbkERYWxsiRI5kzZ47Vcdxyyy3ccsst6PV6Fi9ezPPPP8/7779f\n4zlhYWHs2LHD5PHatWtHbm6uceSVl5dnbBtfXdu2bcnPz6/xWF5eHldffbXZ2ISwlcMS1ObNm1m3\nbl2Nx+bNm0fPnj0ZM2YMv/76K++8847Z1+fl5TkqNIe6ePGixO4Caz74gC3/+AdH8vJ4auBAhg4d\nave/S3l5OUVFRZw9exa9Xl/j+NUf++WXX6ioqOCaa64xTt1dunTJbDw33HADixYt4sKFCwwaNMj4\nvH79+rF27Vr69OlDnz59KCsr4/Dhw3To0IGrrroKvV7PuXPnjM8/e/Yshw8fpk+fPqjVasrLyzEY\nDOTl5dWIr3v37pw6dYq33nqLO++8k7KyMrKzs+nRowd/+ctfeOONN4yLa15//XUGDx5MXl4ep0+f\npry83Ji0VCoVCxcuZNiwYWRmZrJ3717i4+PJy8vj8uXLHnE9eUKM9fH0+C1xWIK69957uffee01+\nb926dfz222+MHz+eL774wuRzPLVLpCd3uPTk2AFeTklx6PF9fX0JCQnBYDCgVqtrvFdt2rQxPpaT\nk1On5fsTTzyBRqMxedzw8HD+7//+j0OHDrF8+XJCQkKMjy9fvpzXXnuNuXPn4uPjQ69evZg9ezbt\n27fHz8+P1q1bG+NQqVTMnz+fV199FS8vL7p3787cuXMJDw+vER9U/gy+/PLLvPfee6jVasaMGcOQ\nIUN48sknee211/j73/+Ol5cXw4YNY/r06ajVak6cOIGPj4/xGKtXr2b27NmsX7+e8PBwXnvtNWO7\n+hYtWhAYGOj215OnX/OeHH/tEbgpTm35vnLlStq1a8eIESM4efIkDz30EDtNrLSSlu+uIbG7jifH\nL7G7jifHb83nvFPvQd1zzz3MnDmTzZs3oygK8+bNc+bphRBCeBCnJqiQkBBWr17tzFMKIYTwUFJJ\nQgghhFuSBCWEEMItSYISQgjhliRBCSGEcEuSoIQQQrglSVBCCCHckiQoIezAES3f8/Pz6d27Nw3d\nS2/L6xt7LiEcQRKUEFZydsv3sLAwDh482OCiq7a8vrHnEsIRJEEJ0Ujl5eUuOa+MdkRTJwlKNAkZ\nGRlE9Y2ifUR7Ro1pGi3fc3Nz6d69OxUVFYDpFu8nTpxg9OjR9OnTh3HjxjFnzhxmzJgBYPL1b7zx\nBomJifTu3RutVsu5c+dMPvf8+fM8/fTTDBw4kH79+jFp0iQALly4QHJyMv3796dfv34kJyfX6E8l\nhD1JghIeLycnh/gH4vmh4w+cGn6KLYe2cN+o+xx6zqysLDp16sQ333zDo48+CmCy5fuBAwd4/fXX\nad26tcnjVLV8r1K95Xv1Y1b55JNPePnllzl48CBhYWFMnz6d66+/nu+++46JEyeybdu2Gq+p/frt\n27fzyiuv8O2336LX62u0vKn+3BkzZlBaWspnn33Gf/7zHx566CEAKioquOeee/jXv/7F3r178ff3\n580337T17RPCKpKghMfbvXs3ShcFegGhUHp7Kbt2en7Ld1Oqt3gvLCzkxx9/5PHHH8fX15c+ffow\nePDgeuO+++676dSpE2q1mmHDhvHTTz/VeU5BQQH79u1jzpw5BAQE4OPjww033ABA69atiYuLM7aK\nHz9+PD/88EO95xSioZxaLFYIR2jZsiVexV6gAF7AJfDxdY+W7/n5+cTFxTFz5kwuXLjA7bffDlSO\nVg4ePFij5Xv79u3Zs2cPkydPtuq8BQUFtGrVCj8/P+NjYWFhnDx50uzrqydKjUbD5cuX6zzn5MmT\ntGrVioCAgDrfKykpYe7cuezbt48LFy6gKAqXL19GURRZYCHsTkZQwuPdeeedtPdpj99HfvA1tNjY\ngtmzZ7u85fuWLVvYvn07x44dIzU1lbCwMDIyMsjIyODgwYPG51a1fN+9e3eNlu+WzhsaGsr58+cp\nLS01PmZNEzhLwsLCOH/+PMXFxXW+984775Cdnc3mzZv5/vvvWb9+PSALNoRjSIISHk+j0fDZts94\nMfFFJvaYyPq31/P0zKddFs+hQ4fIysqirKwMf39//Pz88PY2/6P2t7/9je+//560tDTuuOOOGt+r\n7zcL62sAAAviSURBVIM/PDycnj178tZbb2EwGMjIyGDv3r1Wv762queGhobyl7/8hdmzZ3PhwgUM\nBgPff/89UNm63t/fn4CAAM6dO8dbb71l9fGFsJUkKNEktGzZkpkzZ7LkjSWMHDnSIeewdkRWXFzM\ns88+S9++fRkyZAht2rSpdxNvaGgo1113HZmZmcYpQFPnNHX+1157jYyMDG666SbefPNNbr/99hr3\nxCy93ty5Xn31VXx9fRk2bBgDBgzg3XffBWDMmDHodDr69etHQkICgwYNqveYQjSGU1u+W0tavruG\nxO469op/ypQpdOnSxbgs3Bk8+b335NjBs+O35nNeRlBCeLBDhw5x/PhxFEXhq6++Ys+ePcTGxro6\nLCHsQlbxCeHBTp8+zWOPPcb58+dp164dL774It27d3d1WELYhSQoITxYTEwMMTExrg5DCIeQKT4h\nhBBuSRKUEEIItyQJSgghhFuSBCWEEMItSYISQgjhllySoI4ePcoNN9xgrOAshBBC1Ob0BFVcXMyr\nr75aowKzEEIIUZvTE9Tzzz/P1KlT8ff3d/aphRBCeBCHbdTdvHkz69atq/FYeHg4f/vb37j22mul\nPL8QQoh6ObVY7F//+lfatWuHoij88MMPXH/99bz33nt1nnfgwAFnhSSEEMJFLBWLdVk188GDB/P5\n55+jUqlccXohhBBuzmXLzL28vGSaTwghhFlu2Q9KCCGEkI26Qggh3JLbJihP3cyr0+mYMGECo0eP\nZty4cRQUFLg6JKsVFxeTnJxMUlISCQkJZGZmujokm33xxRdMmzbN1WFYRVEUXnjhBRISEnjwwQc5\nfvy4q0Oy2Q8//EBSUpKrw7BZWVkZTz75JA888AD3338/e/bscXVIVquoqOCZZ54hMTGRBx54gCNH\njrg6JJsVFRVx6623cuzYsXqf55YJypM3837wwQf07NmT999/n+HDh7Nq1SpXh2S1NWvWcPPNN/Pe\ne+8xb9485syZ4+qQbJKSksLixYtdHYbVdu3ahV6vZ+PGjUybNo158+a5OiSbrF69mmeffRaDweDq\nUGz28ccf06ZNG9avX8+qVat46aWXXB2S1fbs2YOXlxdpaWlMnjyZRYsWuTokm5SVlfHCCy9YtRfW\nLROUJ2/mHTNmDI8++igAeXl5tGrVysURWW/s2LEkJCQAlReRp/2C0Lt3b2bPnu3qMKx24MABBg4c\nCMD111/Pjz/+6OKIbBMREcHSpUtdHUaDDBs2jMmTJwOVIxJfX8/p3RobG2tMqLm5uR71GQPwyiuv\nkJiYSNu2bS0+16X/Kp6+mddU/PPmzaNnz56MGTOGX3/9lXfeecdF0dWvvtgLCwt58sknmTVrloui\nq5+52IcNG0Z6erqLorJdcXExgYGBxq99fX2pqKjA29stf2+sIy4ujtzcXFeH0SAajQao/DeYPHky\nU6ZMcXFEtvH29uapp55i165dvPnmm64Ox2pbtmwhJCSEW265heXLl1t8vtut4rN2M68n+O233xg/\nfjxffPGFq0Ox2i+//ML06dOZOXMmAwYMcHU4NktPT2fTpk0sXLjQ1aFYNH/+fKKiohg6dCgAt956\nK19++aVrg7JRbm4u06ZNY+PGja4OxWb5+flMmjSJ0aNHc9ddd7k6nAYpKirivvvuY8eOHR4x4zR6\n9Gi8vLwA+Pnnn7n66qt5++23CQkJMfl8txvXfv7558b/Hzx4sNuOQMxZuXIl7dq1Y8SIEbRo0QIf\nHx9Xh2S1I0eO8MQTT/D6669z7bXXujqcJq93797s3buXoUOHkpmZyTXXXOPqkBrEzX7Htcrp06fR\narU8//zz3HTTTa4Oxybbtm3j1KlTPPLII/j5+eHt7e0xo+7333/f+P9JSUnMmTPHbHICN0xQ1Xni\nZt577rmHmTNnsnnzZhRF8agb34sWLUKv15OSkoKiKAQFBXnsPQZPEBcXx9dff2287+dJ10p1Vb8R\ne5IVK1Zw4cIFli1bxtKlS/Hy8mL16tWo1WpXh2bRbbfdxtNPP83o0aMpKytj1qxZHhF3bdZcN243\nxSeEEEKAm67iE0IIISRBCSGEcEuSoIQQQrglSVBCCCHckiQoIYQQbkkSlBBCCLckCUp4rPT0dG6+\n+WYefPBBHnzwQRISEmpsBKzy73//mw8//NCmY2/dupW9e/fa9Jrc3Fzi4+PrPH7hwgVmzZpFUlIS\niYmJTJs2jeLiYpuO7WoffPAB5eXlJr/nSRXkhWdx6426QljSv39/Y1kjvV7P0KFDGTlyJAEBAcbn\nVBVktUVDS9+Y2nw4depUEhISiI2NBWDt2rW88MILHlGOqcry5csZOXJkncooKSkpfP311/To0cNF\nkYmmTBKU8GjV95kXFxfj6+uLj48PSUlJhISEcOHCBW6//XZycnJISEhg2rRphIWFkZOTQ69evZg9\nezZnzpzhqaee4sKFC0BlteVPPvmEq666is6dO7N8+XK8vLyMdc8eeOAB9u/fz5IlS1AUhcuXL7Nw\n4UKTFbHz8vIoKioyJieorHh/6dIloLLtw7vvvoufnx8RERHMmTOHTz75hL1791JSUsLp06dJSkpi\n9+7d/Prrr8ycOZPBgwczZMgQoqKi+P3337nmmmtISUnh4sWLzJgxg+LiYsrLy3niiSfo168fd955\nJ3379uWXX37By8uLZcuWERAQwKJFizhw4ADl5eWMHTuWv/71ryQlJdGjRw9+/fVXLl26xBtvvMHX\nX3/N6dOnmTp1KkuWLKnx9+vduzdxcXFs2rTJEf+8opmTBCU82rfffsuDDz6Il5cXKpWK5557zlip\nevjw4QwZMoStW7caRzbZ2dmsWbMGPz8/YmNjKSoqYvny5QwZMoT4+HgyMzM5dOgQ8MdoqKCggI8+\n+ojy8nKGDx/OsGHD+PXXX1mwYAGhoaGsWLGCnTt3cscdd9SJr6CggA4dOtR4zMvLi4CAAM6dO8eS\nJUvYtm0bGo2G+fPns2nTJlq0aMGlS5dITU1lx44drFu3jk2bNvHdd9/x3nvvMXjwYE6dOsUTTzxB\nx44dmTJlCl988QUHDx7klltuISkpiVOnTjFq1Ch2795NcXExw4cP59lnn2X69Ol89dVXBAQEcOLE\nCdavX49er+f+++/n5ptvBipbfzzzzDMsXryYTz/9lL///e+8/fbbJntteVoFeeFZJEEJj1Z9iq+2\nyMjIOo9FREQYE1jbtm0pLS3l2LFj3HvvvQBERUURFRVVY6QQHR2Nr68vvr6+dOvWjePHj9OuXTte\neuklWrZsyalTp+jdu7fJGMLCwsjPz6/xWFlZGZ999hmRkZF069bNGM8NN9zA119/Ta9evbjuuusA\nCAwMpHPnzgC0atWK0tJSoLItTceOHY0xHzt2jGPHjjFixAgA2rVrR2BgIEVFRQDGKbiwsDD0ej3/\n+9//+O9//8uDDz6IoiiUl5cbW2dUf+7p06eBypGqVEUTziaLJESTZanCc9UHbteuXcnKygJg//79\nLFiwoMbzDh8+jKIo6HQ6jhw5QkREBM899xzz589n3rx5NRqv1f4Qb9euHcHBwezevdv42Lp169iz\nZw8dOnTgyJEjlJSUAJWLPqqSqqVCmqdOnTImn4MHD9KtWzc6d+7M/v37jd+/cOECrVu3Nvn6Ll26\n0K9fP959913effddhg4dakx4ps7t7e0tCUo4nYygRJNk7gO++uNV///II4/wzDPP8PHHH+Pt7U1K\nSgofffSR8XllZWU8/PDDnDt3jgkTJvx/e3eI4yAQhXH8SyapoAgEFs7QkEoMCR4PQWJRKBwdxyVI\nuAMCX9OTkHAB0tTsBbZbs5sd8f9dYN4b8+U9MaMgCFQUhcqylOd5CsNQ+76/PXccRw3DoGma9Hq9\nFEWRrLXyfV9t26quaxljFMexuq7Tsiwf+zudTrrdbtq2TZfLRVmWKUkS9X2vdV31fD5lrZUx5tue\nsyzT4/FQVVU6jkN5nut8Pr+9t+v1qqZpNM/zx9qA38Jr5sAPXP0AMU1T3e/3/y4D+FOs+AAATmKC\nAgA4iQkKAOAkAgoA4CQCCgDgJAIKAOAkAgoA4KQvD2UwMXkQkMgAAAAASUVORK5CYII=\n", "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "with plt.style.context('seaborn-whitegrid'):\n", " plt.figure(figsize=(6, 4))\n", " for lab, col in zip(('Iris-setosa', 'Iris-versicolor', 'Iris-virginica'), \n", " ('blue', 'red', 'green')):\n", " plt.scatter(Y[y==lab, 0],\n", " Y[y==lab, 1],\n", " label=lab,\n", " c=col)\n", " plt.xlabel('Principal Component 1')\n", " plt.ylabel('Principal Component 2')\n", " plt.legend(loc='lower center')\n", " plt.tight_layout()\n", "\n", " plt.savefig('/Users/Sebastian/Desktop/pca1.pdf')" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "
\n", "
\n", "" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "
\n", "
\n", " " ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Shortcut - PCA in scikit-learn" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "[[back to top](#Sections)]" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "For educational purposes, we went a long way to apply the PCA to the Iris dataset. But luckily, there is already implementation in scikit-learn. " ] }, { "cell_type": "code", "execution_count": 100, "metadata": {}, "outputs": [], "source": [ "from sklearn.decomposition import PCA as sklearnPCA\n", "sklearn_pca = sklearnPCA(n_components=2)\n", "Y_sklearn = sklearn_pca.fit_transform(X_std)" ] }, { "cell_type": "code", "execution_count": 101, "metadata": {}, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAakAAAEbCAYAAABgLnslAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzs3Xtc03X7P/DXgA2mHASdiKSQWuGdKeABzSMIViqKB1JK\nS1kJmof4qqXm3Z2p3WYa5RlrdWf+XHbQklvLwuNtBzFE0ZQ79RZUDjJBicNgA96/P5C5wcYObPt8\nBtfz8bgfd26fbZdz7OL9+Vzv6xIwxhgIIYQQHnLiOgBCCCHEEEpShBBCeIuSFCGEEN6iJEUIIYS3\nKEkRQgjhLUpShBBCeMuFyxdXqVR4/vnnoVarUVtbi6eeegrz58/nMiRCCCE8IuB6n5RSqYRYLEZt\nbS3i4uKwcuVK9O3bl8uQCCGE8ATnp/vEYjGA+lVVTU0Nx9EQQgjhE86TVF1dHWJiYjB06FAMHTqU\nVlGEEEI0OE9STk5O+Pbbb3Hy5EmcP38eV69e5TokQgghPMFp4YQ2d3d3hIWF4T//+Q969eqlc19G\nRgZHURFCCLGV/v37Gz2G0yRVUlICoVAIDw8PVFVV4ZdffsGcOXP0HmvKX4aP8vPz0bVrV67DsIij\nxu6ocQMUO1codvszdfHBaZJSKBRYtmwZ6urqUFdXh7Fjx2LkyJFchkQIIYRHOE1Sjz32GPbv389l\nCIQQQniM88IJQgghxBBKUoQQQniLkhQhhBDeoiRFCCGEtyhJEWJnCoUCZ86cgUKh4DoUQniPkhQh\ndrRXLkdQQAASo6IQFBCAvXI51yERwmuUpAixE4VCgXlSKY4plcgoLcUxpRLzpFJaURHSDEpShNhJ\nTk4OAkUiNLRQ7gsgQChETk4Oh1ERwm+UpAixk8DAQOSoVMi6/+csALlqNQIDAzmMihB+oyRFiJ1I\nJBJsk8kQLhYj1NMT4WIxtslkkEgkXIdGCG/xpgs6IW3BtLg4RERG1p/6CwykBEWIEZSkCLEziURC\nyYkQE9HpPkIIIbxFSYoQQghvUZIihBDCW5SkiFmopQ8hxJ4oSRGTyeV7ERAQhKioRAQEBEEu38t1\nSISQVo6SFDGJQqGAVDoPSuUxlJZmQKk8Bql0Hq2oCCE2RUmKmCQnJwciUSCg1dRHKAyglj6EEJui\nJEVMEhgYCJUqB9Bq6qNW51JLH0KITVGSIiaRSCSQybZBLA6Hp2coxOJwyGTbaFMqIcSmqOMEMVlc\n3DRERkZQSx9CiN1QkiJmoZY+hBB7otN9hBBCeIuSFCGEEN7i9HRfYWEhXnvtNRQXF8PJyQmxsbF4\n4YUXuAyJEEIIj3CapJydnbF8+XL07t0bFRUVmDx5MoYOHYqePXtyGRYhZlMoFFRQQogNcHq6TyKR\noHfv3gCA9u3bo2fPnigqKuIyJELMtlcuR1BAABKjohAUEIC9cjnXIRHSavDmmtStW7eQnZ2Nvn37\nGj+YEJ5QKBSYJ5XimFKJjNJSHFMqMU8qpXZRhFgJL0rQKyoqsHDhQqxYsQLt27fXe0x+fr6do7KO\nsrIyit3O7Bn3uXPnEODiotUsCuju4oIzZ84gODjY7Odz1PccoNi54sixm4LzJFVTU4OFCxdi4sSJ\niIyMNHhc165d7RiV9eTn51PsdmbPuIVCIXJrapCF+gSVBeBGTQ0GDhxo0bUpR33PAYqdK44ae0FB\ngUnHcX66b8WKFejVqxdefPFFrkMhxGwSiQTbZDKEi8UI9fREuFiMbTIZFU8QYiWcrqQyMjKQmpqK\nRx99FDExMRAIBEhKSsKIESO4DIsQg/RV8U2Li0NEZCRV9xFiA5wmqf79++Py5ctchkCIyfbK5Zgn\nlSJQJEKOSoVtMhmmxcUBoHZRhNgK56f7CHEEramKT6FQ4MyZMw4ZO2l7KEkRYoKcnBwEikQ6VXwB\nQiFvhz4aSkS0p4s4GkpShJggMDAQOSqV1shHIFet5uXQR0OJiE+rQVrNEVNRkiLEBI5SxddcIuLL\nalAulyOgVwCino1CQK8AyL+g1RwxjPN9UoQ4Ckeo4tMkIqUSgG4i0l4NNuzpsvdqUKFQQJoohfI5\nJZRdlEAhIE2QInJ0JC/fT8I9WkkRYgaJRGLxRl17aO60JB9Wg5mZmXDycgK63L+hCyD04e+1PcI9\nWkkR0opoEpFUigChELlqtU4i4nI1uFcuR2J8PCprq4BC1CeqQkBdws9re4QfKEkRwiPFxcXIy8vT\nSSDmjAFRKBTo0asXTmVkoLy8XO9juNjT1XCt7ERVFf4AEP8xUOUBiGvEkO3k37U9wh90uo8QC9ii\nOm2vXI6RgwbpVOWZUzKuXZDQf3B/XL12lTdf/tpFG3EAbtQAj6ra49uvv0Xc9DiuwyM8RkmKEDPZ\nYq9Rw0rjeFWVTlVeYny8SSXj2gUJpbNKoXxOCWkCfzYbN75WVgDgTl0dQkJCuAyLOABKUoSYwVZ7\njfSVhz/k5ARfZ2eTSsZzcnIg8hHxtiCBD0UbxDHRNSlCzNBciXdLvnD1lYffqqtDHWMmlYwHBgZC\nVaLidUGCI5TwE/6hJEWIGWy116hhpTEqPh6BIpGmKg+AwUq9xo+XpcggTZBC6COEukQNWQr/VirU\niJeYi5IUIWYwVuLdEtPi4vC3Pn1QVVWls9IwdfURNz0OkaO5XamYU4lIiCkoSRFiJlueturYsWOT\nKavmrD64XKl8t38/Vi5dqneUCSGWoiRFiAX4ctqKLysXhUKBN5YswfGqKvRVKpGF+tOUEZHU7oi0\nDFX3EeIA9O3L4tPYjZycHAQKhZw3ryWtDyUpQnhOXzLi09gN4H5BiVrtEKNMiGOhJEUIjxlKRpmZ\nmbwYu9FAIpFg7caNtA+KWB1dkyKExwztywLA+diNxibGxGBqbCwvrpGR1oNWUoTwmKHRGyEhIbzs\n4GDqKBOazEtMRUmKEB5rrp3QtLg4ZOfmIiUtDdm5uQ5T7k2TeYk56HQfITzXmtoJ0WReYi5aSbVS\ndDqlddF3Go1PJeim4nsjXMI/RpOUWq1ucltJSYnVAlixYgWefPJJREdHW+052zq5fC8CAoIQFZWI\ngIAgyOV7uQ6JNMOSXyj4VoJuKp1GuAAvG+ESfjGYpH777TeMGDECw4YNQ3x8PG7duqW5TyqVWi2A\nyZMnQ3a/kSZpOYVCAal0HpTKYygtzYBSeQxS6Tzef3m1VY1XQ5s+/NCkfyt9oz0cYfNsQyNc8R4x\nPP/lCfEeMS8b4RL+MJik3nvvPchkMpw+fRrPPvss4uPjce7cOQAAY8xqAQwYMACenp5We762Licn\nByJRIKD19SUUBvD+y6st0rca2rx+PR7t3t3oqTt9VX/XVSrcvXuX97+QxE2PQ+7VXKR9mYbcq7k0\nmZc0y2CSUqvVeOSRRwAATz/9NLZu3YrXX38daWlpEAgEdguQmCcwMBAqVQ6g9fWlVufS6RQe0rca\negzA1qoqo6fuGlf9DRMKwerqsPzZZx3i+pSppeqEGExSLi4uOj8kjzzyCD777DNs3ryZfiu3I3Ov\nV0gkEshk2yAWh8PTMxRicThksm30ZcBDevdAAYiCaafuGkrQ1331FYQuLjipUjnE9Skq6iHmMFiC\nvmTJEhQXF+t8uXXp0gW7d+/G7t277RKctvz8fLu/pjWUlZVZHPv+/d9hyZI3IBQGQq3OwcaNaxET\nM9Ho40aOHI7Tp4/j5s2b6NatGzp27GhRDC2JnUuOFPeaDRswavFi+FRVoQTAdgAFqO8m4ebmZtLf\no6amBgEuLjorsu4uLjhz5gyCg4NtF3wjprzv3+3fjzeWLEGgUIgctRprN27ExJgYk1+juLhY53Nt\nLY70mWnMkWM3CeOBmzdvsvHjxxu8//fff7djNNaVl5dn0eOKioqYWOzDgPMMYAw4z8RiH1ZUVKT3\n2PT0dL33tYSlsXPN0eIuKipia1evZh3c3FiwhwfzEYvZF3v2mPV4H7GYna//oLDzAPMRi63+eTDG\n2Pve0jj37NnDxJ5i5hXoxcSeYrZHbvp7ZIyjfWa0OWrspn6vc75PavHixZg+fTquX7+OUaNG4Ztv\nvuE6JF4wtQCCys0dn0QiwYqVK/HnjRtY+8UXZnWPaJgntS45mXctkhprSUWi9ibg0lmlUD6nhDSB\nv6c0ifVw3nFi48aNXIfAS7oFEPUtRBsXQGiXmyuV9cdIpeGIjIzg3RcUMU4ikSA4ONjkf7u9cjnm\nSaWaSbjrkpMRHBrK264U7u7uuFpVheMARsG8prgNm4CVXeob7WpvAubj35VYj9GV1Pfff2/SbcS6\nTCmAoHLztqNxsYG+8vVlSUm8TVByuRz9B/eHupMTwl2AQLGbWSs+2gTcdhldSe3cuRPPPPOM0duI\n9cXFTUNkZITBnm2mrLaI42u8Ytomk8G7Y0d0c3LSe+qMb0lK+1QdugAoBAp3M2Smn0Xv3r1Neo6G\nTcDSBCmEPkKoS9S0CbiNMJikTpw4gZMnT+L27dtYs2aN5vby8nI4OzvbJThS/8Np6AexYbUllYZD\nKAyAWp1L5eatjPaKqa9SiSwAw158Ec7OzlBVVfFqnpQh+k7VuXZyRXl5uVnPEzc9DpGjrd9oV6FQ\n4Ny5cxAKhfSzw0MGk5Svry/69OmDo0eP4vHHH9fc3r59eyxfvtwuwRHjjK22tDVcZNd3XHP3Ee40\nHnroB6BWrcYptRqXUX9txwfAXZ4WSwCASqVC5e1K4DqAh9GiU3XN/dJmCblcDmmiFC7eLqi5WwNZ\niow6YPCNsfI/lUrV4lLDlmqLJejWtmfPF0ws9mFeXqFMLPZhe/Z8YfQ+vsRuLkeNm7GmsTcu2/5/\nAOt1/78ZwIoA9mj79uzw4cMcRfyAvvd9/vz5DEIw+IDBBUzoLbR6+bilioqKmNhTzJAIhrfAkAgm\n9rR/6X5LOern3dTvdaPXpLKysrBlyxbk5+ejpqYGjDEIBAIcOXLEHjmUWEFzVYAAmr2PcEvT/kgq\nRYBQiOsqFVhdHbJUKvRF/cbfO3V1CAkJ4TpUjYZVuUqlwpaULYAUmmtR6o/VOHb8GIYOHcp1mFQx\n6CCMJqk33ngDy5cvR58+feDkxPm2KmKBhirA+iQEaFcBHj78E5RKH+irEPT396fTgDzQeOjh0bQ0\nTdLKVat5dZpPu8jjv5WVgCd0ZkfBC7h69SovkpROxWBDEqWKQd4xmqQ8PDwwcuRIe8RCrKwhwbi7\nu+utAnR3d8fate8BEDS5LzAwEF9++TWWLl0Jkai+ilAm24a4uGnc/YXaMO1rMQ1JKzMzEwCarKLs\n/YtFw+tVVlbqFHl8C2BSKXSSAP4CBg0aZPOYTKFdMejSwQU192qoYpCPjJ0PfO+999i6devY2bNn\n2cWLFzX/sye6JmW+xteZ5s9fyMRiH+bpGaK57pSens68vEIZ8AUDfBgQwoB2bPXqtayoqIi5uXmb\n1JaJbxz1HD1jpsf+xZ49zEcsZqFeXjptlAzdbivarYpE7UUsUCzWXC9jAOsodGFwAUNHMAjB5i+Y\nb9N4LFFUVMQOHjzoEJ9tfRz18261a1Lnz58HAFy8eFFzm0AgwK5du2yXOUmL6LsGJZOFIyPjFMrL\nyzW/YSsUivsrrN4AsgH8BDe3V5CQ8DJycnIgFAaiqqrpaUD6TZNb+srSw6VS9A0O1nt7RGSkTf7N\ntPc/KbsogUIg52PodJRgLkKc+uk4rl69ikGDBpm8L8qezO30QezLaJL6/PPP7REHsSJD16DKy8sx\ncOBAnWNXrFiMtWtHQiR6+P4+qx2aH1a1Oge0UZh/GpelN2zkTU9P13u7rX6x0Fd4IPYVY+KdOvR0\nddVcLxs6dGiz16DouidpjtFKiDt37mDFihV46aWXANRf9Pzqq69sHhhpytQ5PKYMPmxoTLthwzcQ\nCJywdOlU5OZma645SSQSbNy4luZS8ZDeOVRqNQYNGqT3dlv9YqGvVRHKgN8yM5GSlmZSo1y5XI6A\nXgGIejYKAb0CIP+C38MaCQeMnQ+USqXs4MGDLDo6mjHGmFqtbnashi3QNanm9zk1d7z2NagGqamp\nzNlZzID9zV5vysvLs9kYEFtytHP02u+xudekQjw99V6Tany7reyR11+T8gz0ZG4ebmbtfzJln5Kh\nz5+1P5eO9pnR5qixm/q9bjRJTZ48mTHG2MSJEzW3TZgwwcKwLNPWk5Q5s6UaP67xD3JU1DMMEDPg\nkfv/v5ABjHl6hrD09HSrx84FR4q7caHDtq1bTX6sqV/gxr7QW/qF3/D4rKwssx6Xnp7OvAK96hPU\n/f95BnpqPoeG5kfZYq6UI31mGnPU2K2WpGbMmMFKSkpYTEwMY4yxzMxM9vzzz7csOjO19ST1oArv\nQeGUvqRi7Mvm1KlT9xPTg2RX/+f9BldSjshR4tY3BNDbzc2kZGFqYjFW7WfNakBz3/fmVlKG7rt0\n6ZJNukQ4ymdGH0eN3WpDD5ctW4a5c+fixo0bmD59Ol5//XWsXLnSHmciyX3mXGNqbvjhjz/+COAh\nQKd3tj+cnZ+j600csHQI4F65HEEBAUiMikJQQAD2yvVfx9E3zmOeVNrsuA/t+22tYZ+SeI8Ynv/y\nhHiPWLNPqaEoQ3sjsNCnvjhE3+00nqYVMyWTqdVq9ueff7L//ve/nPTya+srKcaav8Zk7HRgw2/d\nqampeldSqampNo3d3hwlbktWUuaMYE9PT2ehXl46+5ZCPB+cTjN2v7mae9+bW/npu49WUqZz1Nit\ntk8KqO/fl5eXh9raWly6dAkAEBMTY9PkSXQ11+1cX8m5UumNlJSP0LNnT0il8zRdI5544jFcuDAY\ngD+APIwZMwrjx4/n4q/U5jXuy5erVmPthg3NrmhzcnIQ4OJi0hwp7SpAfeM8jN3fUg2l5WfPnkXS\na0kQ+YigKlE16TSur7O5oflRvXv3prlSbYzRJLV06VLcvHkTQUFBmjlSAoGAkhQHDI0p0Df8ECjB\nmjXvwsnJRWdT79Wr4UhN/RJnzpzBmDFjeNFDrS1r3JdPrVY3e/y5s2eRXVZmUmLRlwS1+/wZu78l\ntEdglOWXASMB5bD6Db/SBCkiRxvfYGxofpSt5koRfjKapC5evIhDhw5BIBDYIx5iAYlEghUrFuPv\nfx8C4DEAuQC2w8XlTQBiNG4e6+vri1WrVnEVLmlE+5eP/Px8APo3uCoUCixLSsJbAMJRf3XxCoAP\nk5MNflE3ToKNjzPWB9ASly9fxuw5s1E9o/pBz77PAITA7E7jhn4xs/ZcKcJfRgsnHnnkEbtdSCWW\nS0h4GW5uIgBLUN/iqDdqa4tQV3cT2gUXKtV13L17l/5NecxQYURDocVrqP8X/hhAT3d3BIeGAjC8\n2VsikWDgwIEGv9SPpqUhLiYGy599FkEBAfgoJcWkTeOGYh8cEgK1qFq3+7kngHuwa6dx7ffD1I3w\nhIeMXbSaMWMGGzBgAIuPj2cJCQma/9kTFU6YRl9xhfZtQqEHE4m8TN4Q7KgXZB01bsYYy8rK0hRG\nFN0fctjhfjFFc0UT5paSNxQrXLp0qclzigH2hIeH2SXpDbEfA5jYBTrFDRCCuXdzt9vAQ+29VEKx\nkInaiZrdV+XInxlHjd1q+6ROnz6t93/2REnKdIYqpQ4fPmz2hmBH/fA7atyMMXbw4EEW6uXFvgCY\nD8BCAdYOYGtXr2aM6e8oYU7Fn/ZzhHp5sQ6uruzhRp3L+wIs3YTnMRQ7A9ie+4nKyQfM1d2V7UjZ\nYbfOJTqVgUvB4Aaj1YCO/Jlx1NitVt03aNAg3LlzBxcuXAAA9O3bFx07drT5Co9YxlCllLe3t8HB\nh3Runz+6deuG/1VXYy7qu4k3FEeEv/MOXk5I0HuN6cyZMyY3ltXXQX0wdDuX3wIQCEDSzPMYir2h\nWjAOgF8NMLHCFb9lZtq1+7lO49s8AN7Qu6+KPveOweg1qUOHDiE2NhY//PADvv/+e81/E8diyoZg\nwr2OHTti6RtvoCNgcJNv42tMhhrO6vu31beBuKdYjImurgjx8MBgAM+Z8DyGYt8mkyFcLEaopyem\niMXY+emnehOULa8R6TS+7QDgLnSa4NL0XQdjbKkVHR3N7ty5o/lzcXGxptmsNZw4cYI99dRTbMyY\nMSwlJUXvMXS6zzqa2xCsD59iN4ejxs3Yg6a+5py+Y8xwY1l9ffz0PfelS5fY2tWrmYdQyHrdP8Xo\nLhSadU2q4X031rLJlOtnLe0nqN34tuGalGegJ12T4hGrXZNq3PG8trbWal3Qa2trWWRkJLt16xZT\nqVRswoQJ7OrVq02OoyRlPeb88PMtdlM5atyMPYjdkm7mjf9tjU3vbcl1LX2vacr7bsrrWKuBrHZs\nxj73reEzYw4+TDewWpJat24di4+PZ9988w375ptvmFQqZevXr29xgIzVN6uVSqWaP6ekpOhdTVGS\n4oajxu6ocTOmG3tLvkiMJQPt6r709HR2+PBhs1okFRUVsdVrVjOxx4NksnWb8Q7uxloxmTK+wxZa\ny2fGFLboIm8JqxVOvP766/jxxx+RkZEBAJg2bRqioqKscqrx9u3b8PPz0/zZ19dXU6BBbIOmoDqO\nlmxYNTS9t6FgQCKR4GhaGuZJpQgUiZCjUkFVU2NSJwu5XI74l+NRpaoCpNCMjl/8+mLETo1tNmZj\nrZj0TfulQgfrUSgUkCZKoXxOqfl3M7UDCFeMFk4A9bvQBw4ciLCwMAQHB9s6JmImUy9Cm9IpnbQO\nxoop9HVAdxYIMNLNDaGenggXi/W2SFIoFJj10ixUPVUFdIJO1ZxLBxccOnSo2c+hphXT/eKKxq+j\nb9ovFTpYj6Hu8nzuIm90JfXVV19h69atGDx4MBhjWLNmDebNm4epU6e2+MV9fX01bWCA+pVV586d\n9R6rfZwjKSsrs2ns+/d/hyVL3oBQGAi1OgcbN65FTMzEJscVFxcjPn4uqqqOa/r4xcePQp8+fzO4\npaC52IuLi3Hz5k1069aNd1sSbP2e25I1Y1+zYQNGLV6s07xWrVYjPz8f586da9Ko9mFXV7yWkgIv\nLy/Nv2vjWE6cOAGVmwroCeB71CeT+62PygvK8crKV1C7oBYb392ImIn6+3sOHzkSx0+f1vn8aL/O\nhnUbsPj1xRB6C6G+q8aGdx/EbStt5TPj5uaG6uJqnX83VbEKbm5u/P37GzsfOGbMGFZSUqL5c0lJ\nCRszZozF5yG11dTUaAonqqurqXDCTOZM7DV1cKIpsZs7yt7e2tL1BWOam95rSaHE4cOHmUB4f3Ps\n1PsbZb3B4AKGSOtdR7L3hf229JnRrnxsFdekvL290b59e82f27dvD29vb6skSGdnZ/z9739HfHw8\nGGOYOnUqevbsaZXnbgv0jeho2KDbcH/DtSd9ndIt2SelUCgglc7T6awulYYjMjKCt+e027LmGrRa\n0gE9JCQE7SBE7cdqiDyAajXA1E4Q+olRMayi/iArXEeiBrK242hd5I0mqe7du+PZZ5/F6NGjIRAI\ncOTIETz22GP49NNPAQCzZ89uUQAjRozAiBEjWvQcbZWhxHP27DmMHPm0ZoaUTLYNAFBTowIwBIAf\nRKI7kMlSzP6ANpcY+f5hJ7qMdUjXRyKRQPbZZ0iMj0dnlTOKhLVY/8EHWPRaks4pJLqOxG+O9EuA\nSUmqe/fumj+PHj0aAFBRUWG7qIhJJBIJZLJtkErDIRQGQK3ORXLyOiQlLdNZ6cTHj4RA4AS1+mcA\nfgB+gpPTK4iMjDD7Na21IiP8YMmXlb7k5u7lifg58RB1FNEgQmJVRpPU/Pnz7REHsVDjib36VjrO\nzp2hO1fqOYhEG5qsfkwpT9eXGGWybfSF1EZof0YGDhyouT1uehz6PN4HVVVVVj2F5IhbJhwxZj4z\nmqQuXLiAHTt2ID8/HzU1NZrbU1NTbRoYMV3j34Ybr3Rqa4tQW1unc5tSeVUzV0oikUAu36szZl4m\n24aRI4frfb3mRtmT1muvXK6zr2qbTIZpcQ/GwHfs2BFdu3a12us1TPcV+YhQfacabyx7AwlzEiz+\nvNkjeWjHrCpRQZYiQ9z0OOMPJIYZq6wYM2YMS0tLYzdu3GC3bt3S/M+eqLrPPI179O3YsZMJhe4M\n8GZACAPcGSBmHh4P7tdXJZiVlWX32K2hLVVq2Ysp1YDWjF2n80RDFaEPmNjDsmo0Y10WrBE7dcsw\nj9Wq+3x8fDTXoYhj0HcKsF27R1Fa+gOATNQPUjiGsrL6VdWiRSMhEnVD4zHzN2/exBNPPMHVX4Pw\niLEOFtagvdLRdJ7wUNaPnp8FoAugLFSa3SHBXl0WqFuGbRhNUgsXLsQbb7yBIUOGQCQSaW4fM2aM\nTQMjLaP/FGAB6ofr6Cak6uqOqKm5isbFEN26dbNv0IS3jLUzaqnGpxLXJSfXd564hvpxGy2YB6WT\n8PLqn88WyUOnWwZVOVqN0ST1zTff4H//+x9qamrg5PSgixIlKcehXezg7NwV5eXXAJ2vmxI4OblA\nKBwJkehhTTEE3zpJEPtpfP3G0n1Vpr5W40GM4UlJSP4gGa8ufhVV1VUt+uIPDAxE5e1KYDMALwCl\ngFKgbHHy0PceyVJkkCZIIfQRWr3Ksc0WZBg7H2it7hItQdekrKNhF/+yZcsZ0O7+9SkfBnzBPD1D\n2OHDh3V2+VurI7e98ek9NxcfYm9u3lNznwNLY09PT2dPeHiwdIAVNeqMrt1t3dIOCUVFRczFzUXn\nWpGLm0uLrqc1d43LFj8rzb0eHz4zlrDaqI5ly5axK1eutDiglqAkZTl9PzBFRUXMza0DA/4fA4oM\ntlNqiL1xG6QdO3byOmFx/Z63BNexW9ouiTHLY9+xYweDC5i7N5jYBWydntdsyRf/4cOHGXzuJ6iG\n//mAHT582KLY7V0gYez1uP7MWMpqhRPnzp1DTEwM/P39da5JUQm69dhqGa+vrDwubhokEgk++WQH\npNJ5ze51Ki4uxsWLFxEfn4iqqhP3916tR2LiInh4BKGmJlfznKR1sEeBhDaFQoGk15KAl4Dy+6fz\nln0M7ExO1nm9FndIKIPOKUOUWf5U9i6QaOsFGUaT1Mcff2yPONosQ4mkpZrrsQcAvXr1QEbGKZSX\nl+tNjnJCdtCcAAAgAElEQVT5XsTHz4Wzc3dUVakAXEZ9t4p3AfymqQykvn2ti7ECCWO/UJn7C5e+\nL2CPrh4IDg212t8pJCQEQhch1P9S1xdh3AOELkKEhIRY9Hz2LpBo8wUZpiy3Ll++zD7//HP2+eef\ns8uXL7doiWeJ1nq6z5wu5ubS1/Uc6MliY6cxsdiHeXg8wVxdPdmOHTtNiqt+j9VhBvQzq5M6Fxz1\n9Adj/Ijd0Oj65q5VMcbYti1bmr1fH3udOtuRsoOJ2omYuLOYubm7mb1PqvHpRnt3Em/u9fjwmbGE\n1a5J/etf/2Ljxo1jH3zwAfvggw/Y+PHj2a5du1ocoDlaa5KyZHyGqfQnmg4McGPAu/cLJvoxQNwk\nURlKcGJxDwaIbZJUrclRf2gZ40/sjb+UTRlH7+3mZtG1LFt/4TcUHXgEeDBXd1e2I2VHk2Oae98N\nFS3Yu5jI0Ovx5TNjLqslqfHjx7OKigrNnysqKtj48eMtj8wCrTVJ2XIlxRhjq1evbVTFt5YBPe7/\n94PXdHXt0OQitb64Dh8+rOlO0dDNgm+zpBhz3B9axvgRu74vw/T0dBbq5aX9W4umAq/h/hAPD4P3\nG3t+W33hm7JSKyoqYgcPHtT72lx1kTAHHz4zljD1e92k8fHOzs56/5u0TMP+JbE4HJ6eoRCLw63a\nrDUh4WW4uYkALAGQDeBJ1O9m1N3MKxIF6oyPbojLzW2UTlxjxoxBQsLLyM3NRlpaCnJzs6loopXZ\nK5cjKCAAiVFRCAoIwF65HIDxcfSBgYHIUasN3m/s+SUSCQYOHGj1a5vGxqXL5XIE9ArA9HnTEdAr\nAPIv5GY9ntiBsSz2ySefsOjoaLZp0ya2adMmNmHCBPbpp5+2NImapbWupBpY67dIfc/TuI+fVPqS\nyafssrKyeF1qboij/mbJGLexGzulZ+haVYNtW7c2e39Lytv1xWrKZ1OzEnoRDC+D4cUHKyFTV1m0\nkrINq53uY4yxixcvss8++4x99tln7I8//mhRYJZo7UnKGpob6d74B3rHjp3M1bUD8/AI1jm28XGO\n+uF31LgZ4zZ2Y6f0GDO+mbe5+409v6mJx1iz2MbmL5jPIARDx/ox97HPxmpeyyvQS2f/lGdg01OU\nfBm3boijft5bnKTOnz/Pjh8/3uT248ePswsXLlgemQUoSTXPkmtbTaqV9CQ5R/3wO2rcjPF7JWWM\nKRVyhp7f1MRj7sqmuW7qO1J2mPxcfO644qif9xZfk9qwYQN69erV5PZevXph/fr1Nj0FSczTMOiw\ncRfz5s6ba18D0N5TVVqaAaXyGKTSeSguLrZD9IQvNP35xGKEenoiXCy2Wn++5p4fgKZLeemsUiif\nq+90rlAomjyHudeINMd7ADiI+m7qCwHl80okLU1C8nvJEO8Rw+MTD4j3iA322rPVNTNinMHNvBUV\nFfD3929yu7+/P+7evWvToIh5WjrSXd80XxrV0TY1jIbPzMwEAIs3vBp7fu0Nv2fOnDG5o4K5G1s1\nxxvoph4aEorcq7k4c+YML5JQm20i2wyDK6m//vrL4IOqqqpsEgyxTEurBHWTHECjOtq2o2lpiIuJ\nwfJnn9WpwLOWxqsSncQDNJt4GjqNi/eI4fkvz2ZXP9rHu/3oBtyB3teQSCQIDg7mPCk0VBpGPRul\nt9KwzTJ0HvDvf/87e//991ldXZ3mtrq6OvbBBx+wlStXtvyEpBnompRpWnLevHEVIF2T4oa1p9ua\n+3ngosEsY+YXJ5j7dzPWTZ3rz0yTa20vgrm2d2WXLl0y+liuY7dUixvMLlu2DCtXrkRUVBR69+4N\nAMjOzkafPn2wZs0auyVRYrqWNOFsPM1XIpEgPz/fyhESe2k8RHCbTIZpcXFGH2fNBrPmnLqKmx6H\nyNGRJh9vyWf9qTFPYcrkKQb7VXJ5qk2nh+FFAAeB6nbVCBkUgk8/+hRx043/27VaxrLYjRs32JEj\nR9iRI0fYjRs3Wpw9LUErKW44auyOGjdj1om9Jasha62kjPX5sydTKge3bN3CxB5i1r5beyb2sH+Z\nuc5+LjHM2pflqJ93q3Wc6NatGyIiIhAREWHVaxQ//PADxo8fj969e+OPP/6w2vMS0tZpVkP3/6y9\nGjLGGhV+2pN2M0pLcUypxDyp/mo9W1MoFEYrBxUKBZKWJkH5vBIV0goon1dilnSWXeNtuHbm+rUr\n0A7U4UKLSW2RbOHRRx/Fli1bMHDgQK5CaJMUCgXOnDnDyRcGsQ9jLYyMmRYXh+zcXKSkpSE7N9ek\n04TaWpIkrc2UkvXMzEyo3dQ6x6jcVJoKR3uJmx6HzDOZcK12NamIpK3gLEn16NEDgYGBYIxxFUKb\nI5fvRUBAEKKiEhEQEAS5fC/XIREbsMZqqCX7glqaJK2pucrBhl/Y7t2792Ao4v1jWjIUsSV69+6N\nTz/61OTqxbbAYOHEvXv3mn1ghw4drB4MsR2FQnF/wu5WKJVRAApoYGErpm8/kr1okqRUigChELlq\ntclJsiXFC/oe23AaTZoghdBHCHWJGrIUGdLS0iBNlELkI4KqWAUBBGD/YlYZithS5haRtHYGk9Tk\nyZMhEAj0rnQEAgGOHDli9Mlnz56NO3fuNLk9KSkJERERZoZKWiIl5aP7E3Y3AlgAYBucnB5CZmYm\nxowZw3F0xBZaPHK9BSxJknK5/EHiKFFBliIzuaqtucc2/tIHgIBeAVA+p6yvprsOOH/hDGcnZwgF\nQtS61OKTjz7hNDlw+W/HNwLG8fm2mTNnYtmyZXj88ccNHpORkQE/Pz87RmU9ZWVl8PDw4DSG4uJi\nDBo0ElVVx/FgKPgoAFVwdRXh/ff/iZiYiU0ex4fYLeGocQNtN/bi4mIMGjoIVc9XaTpJuP0/N6T/\nnI6OHTs2OfbmzZvo1q0bOnbsaNZjAeDcuXOYPm86ymaXacq90Q4QKUV4df6rmDFjht7H8ZWjfmYK\nCgrQv39/o8cZXElpKy0tRW5uLqqrqzW3WbPgwZQ82bVrV6u9nj3l5+dzHnteXh5cXR9GVZX2pWwf\nAPGorh6PJUvCERs7tclvbnyI3RKOGjfQdmPPy8uDa0dXVHW5382mCyDqKEJVVZXOc+pbMfXq2cuk\nxzYQCoWouVsDXEd9gnqx/jGqQhU279iMJUuWONQqxlE/MwUFBSYdZzRJffXVV9i1axcKCwsRFBSE\n8+fPIzg4GLt27WpRgGlpaVi9ejXu3r2LxMREBAUF4eOPP27Rc5IH5+Xd3d01mxb19fYD7gJ4GYBE\n04zWkX4wSetiSk8+7XJyZRclUAhIE6TI+C3DrH5+DdepZr80G9XtqvVW/tHPAn8YTVK7du3C119/\njWeffRaff/45rl27huTk5Ba/cGRkJCIjI1v8POQBuXwvpNJ5APyhVF6FWNwFQClksm2QybZBKg2H\ni0t3lJX9F8Bb9x+1ByrV9TZd4kq4Z6jAQTtZ6HRlADRJpby83OhjG4ubHofgfsEIGRSC6sJqk5Ib\n4YbRJCUSieDq6goAUKlU6NmzJ65fv27zwIh5tMdtNKyWlMpwAN9AKp2C3Nxs5OZmIycnB2fPnsOC\nBYuhVq8C0BV1dQxpaUdpFDzhlLGqtuZWWwMHDjS7Iq6h3FuaIIVLBxfU3Ktp8+XefGQ0SXXp0gV/\n/fUXIiMjMXv2bHh6ejrk+c/WTt+4DSAAQHvN6byGfS+BgYFISloGtfoUgL5QqbKoHJ3wQnNVbcZW\nW5ZUxDUkRr6M6iBNGU1SW7duBQAsWLAAYWFhKCsrw/Dhw20eGDGP/utOuQAqmsyWMjQ/is7FE76z\nxR4ivozqIPqZVN33xx9/ICMjAwKBAKGhoRCJRLaOi5ipYaaUVBoOoCuUymtwc/OFQDClyWyplg5J\nJIRLtIeobTGapLZs2YLDhw8jKioKALB8+XI8/fTTmDdvns2DI+bRHrehXd3X+AdaO6EJhQFQq3PN\nGpJICCH2YjRJpaam4sCBA5riiTlz5mDixImUpHjK1N8y9c2PIoQQvjGapDp37ozq6mqdCj9fX1+b\nB0Zsj06bEEL4zmiS8vDwwLhx4zB06FAIBAL8/PPP6Nu3r2Y678qVK20eJCGEkLbJaJKKiorSXI8C\ngEGDBtk0IMJPXI7WJoS0XUaT1KRJk+wRB+Gxhk4WIlF9VaBMto02/hJC7MJgklq0aBE+/PBDREdH\n670/NTXVZkER2zF3RaTdyaJ+XxVt/CWE2I/BJPXGG28AAHbs2GG3YIhtNbciMpS8aOMvIYRLBsfH\nd+7cGQBQV1eHTp06wd/fH/7+/ujYsSONfHdA2iui0tIMKJXHIJXOg0KhaHasvO7GX4A2/hJC7Mlg\nkmqwaNEiCASCBw9wcsKiRYtsGhSxvoYVUX2HCaBhRZSZmWkweQEPNv6KxeHw9AyFWBxOG38JIXZj\ntHCitrZWpw2SSCSCWq22aVDE+gy1QgJg8HSev78/ANr4SwjhjtGVlI+PD44cOaL5c1paGry9vW0a\nFLE+QyuikJAQk07nSSQS6hJNCLE7oyupVatWYcmSJVi9ejUYY/Dz88O7775rj9iIlRlaERnq45ef\nn89xxISQts5okurevTu+/PJLVFRUAADat29v86CI7ehrhUSn8wghfGU0SalUKhw+fBh5eXmoqanR\n3D5//nybBkbsi/r4EUL4yGiSmjt3Ljw8PPD444/THClCCCF2ZTRJ3b59GzKZzB6xEEIIITqMVveF\nhITgv//9rz1iIQYoFAqcOXNGs3eJEELaCqMrqYyMDOzfvx/+/v46p/uod599UHNXQkhbZjRJffTR\nR/aIg+hhq+auNHaDEOIoDJ7uKy8vB1Bfcq7vf8T2DLUyysnJsfg5m+vTRwgxjk6/25fBldTixYuR\nkpKCyZMnQyAQ6DSVFQgEOl0oLLF+/XocO3YMIpEI3bt3xz//+U+4u7u36DlbG0OtjCxt7kpjNwhp\nGblcDmmiFCIfEVQlKshSZIibHsd1WK2awSSVkpICxhh2796Nrl27Wv2Fhw0bhiVLlsDJyQkbNmxA\nSkoKFi9ebPXXcWQNrYz0dYOwBI3dIMRyCoUC0kQplM8poeyiBAoBaYIUkaMj6efHhpqt7hMIBEhI\nSLDJCz/55JNwcqp/+eDgYBQWFtrkdRxdXNw05OZmIy0tBbm52S0qmqCxG4RYLicnByIfEdDl/g1d\nAKGPsEWn34lxRkvQ//a3vyErK8vYYS3y9ddfY8SIETZ9DUdmreauNHaDEMsFBgZCVaICGn6fLgTU\nJWr6Jc/GjFb3nT9/HgcOHIC/vz/EYrHmdlNK0GfPno07d+40uT0pKQkREREAgO3bt0MoFBocU0+s\ni/r0EWIZiUQCWYoM0gQphD5CqEvUkKXI6GfIxgTMyJjdvLw8vbc3zBpqiX379uHLL7/Erl27mm25\nlJGRAT8/vxa/HhfKysrg4eHBdRgWcdTYHTVugGLnijmxFxcX4+bNm+jWrRs6duxo48iMc9T3vaCg\nAP379zd6nMGVVHV1NeRyOW7cuIFHH30UU6dOhYuL0YWXyU6ePAmZTIbdu3eb1BPQFsUb9pCfn0+x\n25mjxg1Q7FwxJ/auXbviiSeesHFEpnPU972goMCk4wxmnddffx0uLi4YMGAATp48iatXr2LlypVW\nC3DNmjVQq9WIj48HAPTr1w9vvfWW1Z6fEEKI4zOYpK5du6a57jR16lTExsZa9YV//PFHqz5fW0Yd\nJAghrZXB6j7tU3vWPM1HrIs6SBBCWjOD2Sc7OxuhoaEAAMYYqqurERoaCsYYBAIBzp49a7cgiX7U\nQYJ/UlNT8e2339J4G0KsxGCSunz5sj3jIBagDhLciIiIwNq1azFkyJAm90VHR9t1O8WWLVtw48YN\nrF+/3m6vSYg9Gd3MS/iLOkjwS21tLdchENLqUJJyYNRBglv79+9HXFwc/vnPfyIsLAxbtmzB/v37\n8dxzz2mOeeedd/Dkk0+if//+mDBhAq5evar3ue7evYvExERER0cjLCwMM2bM0NxXVFSEhQsXYsiQ\nIYiMjMTnn38OAPjPf/6DHTt24NChQwgJCUFMTIzm+Llz5yIsLAxPPfUUvvrqK81zZWVlYcqUKejf\nvz+GDRuGd999V3PfokWLMGzYMAwcOBAzZ840GCsh9kQVEQ6OOkjoKi8vx/z5S/HLL+nw9fXFzp0b\nERYWZrPXy8rKwvjx4/Hrr7+ipqYGBw8ehEAgAACcOnUKGRkZ+PHHH+Hu7o7//e9/8PT01Ps8n376\nKbp06YLvvvsOfn5+OHfuHID668GJiYmIiopCcnIyCgoKMHv2bPTo0QPDhw9HYmJik9N9SUlJCAoK\nwqZNm3Dt2jXMnj0b3bt3R1hYGN555x28+OKLmDBhApRKJa5cuaJ53MiRI7Fu3Tq4uLhgw4YNWLJk\nCb799lubvXeEmIJWUq2AtXr7tQaxsbNw6BDD7dsHkJU1D5GR0TZtAOrr64vnn38eTk5OTTalu7i4\noKKiAteuXQNjDD169ECnTp30Po+LiwsUCgUKCgrg7Oys2Yl/4cIF3Lt3D3PnzoWzszMeeughxMbG\n4uDBg3qfp7CwEOfOncOSJUsgFAoRFBSE2NhYTbJxcXHBjRs3cPfuXYjFYvTt21fz2MmTJ0MsFkMo\nFOKVV15Bdna2Zq4cIVyhJEVajZqaGvz00wFUV8sA9AbwHOrqnkZaWprNXrNLly4G7xs8eDBmzJiB\nVatW4cknn8Sbb76JiooKFBQUICQkBCEhIZoKWqlUiu7du+O1115DVFQUdu7cCaC+Ldnt27cxaNAg\nDBo0CAMHDkRKSgpKSkr0vmZRURG8vLx0+mx27doVRUVFAOpPP16/fh3PPPMMYmNjcfz4cQBAXV0d\nNmzYgKioKAwYMACjR4+GQCDA3bt3rfE2EWIxOt1HWg1nZ2e4uLiitvY2gEAADE5OBTadJN1was+Q\nGTNmYMaMGSgpKcGiRYsgk8mwcOFCZGZm6hzXvn17vP7665g5cyYqKyvxwgsvoG/fvvDz88NDDz2E\nw4cPmxRP586dUVpaisrKSrRr1w5AffuZzp07AwC6d++OjRs3AgAOHz6MhQsXIj09HT/88AOOHTuG\nzz77DF27dkVZWRkGDhxo7ttBiNXRSoq0GgKBAKtWvQWxOBLAe3B1fQ5duxZj4sSJnMRz4cIFZGVl\noaamBm5ubnB1ddXMUGvs+PHjuHHjBoD6hOXs7AwnJyf07dsX7du3x0cffYTq6mrU1tbiypUruHDh\nAgCgU6dOyMvL00zO7tKlC0JCQvD+++9DpVIhOzsbX3/9teY9OHDggGYV5uHhAYFAACcnJ1RWVkIk\nEsHT0xOVlZXYuHGj0QRMiD3QSoq0Kq+/vhidO/sgI+MCHnooGK+8slOzorAWU7+8y8vL8c9//hO3\nbt2Cq6srhg0bBqlUqvfYnJwcvP322ygpKUGHDh3w/PPPY9CgQQDqp2SvW7cOo0ePhlqtxsMPP4xF\nixYBAJ5++mkcOHAAYWFheOihh7Bv3z5s3LgR//jHPzB8+HB4eXlh0aJFGDx4MID6isB169ahqqoK\n/v7+SE5OhkgkQkxMDE6dOoURI0agQ4cOWLRoEfbupe4lhHtGR3XwQUZGhkkt3fnIUTsUA44bu6PG\nDVDsXLFm7Pbupemo77up3+t0uo8QQqxELpcjoFcAop6NQkCvAMi/kHMdksOjJEUIIVagUCggTZRC\n+ZwSpbNKoXxOCWmCFAqFguvQHBolKUIIsYBCocCZM2c0SSgnJwciHxHQsCuhCyD0Edp0n15bQEmK\nEELMpO+0XmBgIFQlKqDw/kGFgLpETb00W4iq+wghxAzap/WUXZRAISBNkCL3ai5kKTJIE6QQ+gih\nLlFDliKjTjAtREmqjaDpvYRYR8NpPWUXZf0NWqf14qbHIXJ0JP2sWRGd7msDaHovIdZj7LQe9dK0\nLkpSrZz29N7S0gwolccglc6jiiNCLCSRSCBLkUG8RwzPf3lCvEdMp/VsiJJUK9cwvRdoOr2XWF9q\naqrBrhJ8YI340tPTMXLkSCtF5Jjipsch92ou0r5MQ+7VXMRNj+M6pFaLklQrR9N7rS8iIgK//vqr\n3vuio6Mhk8nsHJHprBUf9fWj03r2QkmqlaPpvfbDl/HxfO50VldXx3UIxMFQkmoD4uKmITc3G2lp\nKcjNzUZc3DSuQ2oVrDk+fuzYsThx4oTmz7W1tRgyZAguX74MADh37hymT5+OgQMHIiYmBunp6Zpj\nZ86cieTkZMTFxSE4OBi3bt3Cvn37EBkZidDQUERGRuLf//63Jmbt+K5cuYL4+HiEhYVh2LBhmjlW\nKpUKa9euxfDhwzFixAi88847UKvVemO/du0aZs6ciejoaERHR+Po0aOa+5YvX4633noLc+bMQUhI\nCE6fPm3u20zaOCpBbyMkEkmbWD2Vl5dj6fz5SP/lF/j6+mLjzp0OMT5+/PjxSE1Nxf/93/8BqO9W\n7uPjg969e+P27dtISEjAhg0bMHz4cPz6669YsGABfvjhB3h7ewOov9b00Ucf4eGHH0ZlZSXWrl2L\nffv2ISAgAHfu3EFpaanmtRriq6iowOzZs/HSSy9hx44dqKmp0STR7du348KFCzhw4AAAYO7cudi+\nfTsWLlyoE3dNTQ3mzp2LqVOn4p133kFeXh7mzZuHffv2aU4pHzx4EB999BGCg4OhUqms9M6TtoKz\nldSHH36ICRMmICYmBlIp9bci1jErNhbs0CEcuH0b87KyEB0Z6RDj48ePH4+jR49qvsT//e9/Y9y4\ncQDqZ0CNGjUKw4cPBwAMGTIEffr00Vl5TZo0CT179oSTkxOcnZ3h7OyMP//8E9XV1ejUqRN69uzZ\n5DWPHz+Ozp07Y9asWRCJRGjXrp1mnPy///1vvPLKK/D29oa3tzfmz5+P7777rslznDt3DpWVlZgz\nZw6cnZ0xePBghIeHa1ZuADB69GgEBwcDQJP3iBBjOEtSL730Eg4cOIBvv/0Wo0aNwpYtW7gKhbQS\nNTU1OPDTT5BVV98fHg88XVfnEOPju3fvjl69euGXX35BVVUVjh49iujoaAD1oxi+//57nRHyZ8+e\nxZ07d/TGIRaLkZycDLlcjmHDhiExMRH/+9//msRXUFCAbt266Y29qKhIZ/yD9gj6xsf5+fnp3Nb4\n2ObeI0KM4ex0n/ZIb6VSaXBiKSGmcnZ2hquLC27X1t4fHg8UODk5xPh4oP661JEjR+Dl5YVHHnlE\nk0D8/PwQExODt99+2+Q4hg4diqFDh0KlUiE5ORlvvvkmdu/erXOMn58fDh06pPf5fH19kZeXp1mB\n5efna0bQa+vcuTMKCgp0bsvPz8fDDz9sMDZCzMFpZkhOTsaoUaOQmpra5Fw3IeYSCAR4a9UqRIrF\neA/Ac66uKO7a1SHGxwPAuHHj8Pvvv0Mul2P8+PGa2ydMmICjR4/i1KlTqKurQ3V1NdLT03H79m29\nz1NcXIwjR45AqVTCxcUF7dq105soRo0aBYVCgV27dkGlUqGiogJZWfVbFcaOHYvt27ejpKQEJSUl\n2LZtm973sV+/fhCLxfjoo49QW1uL06dP4/jx4zrxE9ISNl1JzZ49W+eURIOkpCREREQgKSkJSUlJ\n2LlzJ3bv3o0FCxYYfK78/HxbhmozZWVlFLsdxc2cCbGHB86fOYMefn5YNWsW7t27h3v37lntNerq\n6lBcXIy7d+9CpVLpvEfat+Xm5mLr1q0oLCyESCTCwIEDMXbs2Gbf08ceewznzp3D8uXLdY57++23\nsWnTJiQlJcHZ2RlBQUFISkpCbW0t1Go17t27pzm+pKQEKSkpeO211yAQCNCrVy+8+uqryM/PbxLz\nu+++i82bN2PTpk0QiUSYMmUKOnXqhJiYGBQVFWH8+PEQCAQYNWoUJk6ciPz8fNy5cwe1tbWa53j7\n7beRnJyM7du3QyKRYNmyZRCJRMjPz0dlZaVDfI4cIUZDHDl2U/BifHxBQQHmzJmD1NRUvffT+Hhu\nOGrsjho3QLFzhWK3P96Pj8/NzdX8d1paGnr06MFVKIQQQniKs8KJjRs34vr163ByckLXrl2xatUq\nrkIhhBDCU5wlqU2bNnH10oQQQhwE1X0TQgjhLUpShBBCeIuSFCGEEN6iJEUIIYS3KEkRQgjhLUpS\nhFiRLcbHFxQUIDQ01OJhhuY8vqWvRYi1UZIixEz2Hh/v5+eHs2fPWtyo1ZzHt/S1CLE2SlKEWAlX\n4+Np1UNaM0pShFjIXuPj8/LyEBQUhLq6OgD6x8XfunULM2bMQP/+/REfH4+3334bS5cuBQC9j//w\nww8RFxeH0NBQSKVSTQPexseWlpZi+fLlGD58OMLCwjB//nwAwF9//YXExEQMGTIEYWFhSExM1NtM\nmpCWoiRFWpXy8nLMT5qPLgFd0G9gP5w+fdqmr5eVlYXu3bvj119/xdy5cwFA7/j4jIwMfPDBB+jQ\noYPe52kYH99Ae3y89nM2SE1NxZo1a3D27Fn4+flhyZIl6Nev/u/7yiuv4LvvvtN5TOPHHzx4EO++\n+y5+++03qFQqfPLJJ3qPXbp0Kaqrq/H999/jl19+waxZswDUd4KfMmUKTpw4gWPHjsHNzY26yBCb\noCRFWpXY52Nx6M9DuB19G1ndsxD5jOOPj9dHe1y8QqHAxYsXsXDhQri4uKB///6IiIhoNu7Jkyej\ne/fuEIlEeOaZZ3D58uUmxxQVFeHUqVN4++234e7uDmdnZwwYMAAA0KFDB0RFRWnGzickJOD8+fPN\nviYhlqAkRVqNmpoa/HToJ1SPqwYkAPoCdT0df3y8sdctKiqCl5cXXF1dNbc1HunemHayFIvFqKys\nbHJMYWEhvLy84O7u3uS+qqoqvPnmm4iIiMCAAQMwY8YMlJeX0/UxYnWcNZglxNqcnZ3hInJBbXkt\n4A2AAU7ljj8+3tjrSiQSlJaWorq6WpOoCgoKWlyh5+fnh9LSUpSXlzdJVJ988glycnLw9ddfw8fH\nB2BXJrgAAA2sSURBVNnZ2Zg0aRIYY1QZSKyKVlKk1RAIBFj11iqI5WLgZ8D1O1d0dXH88fFA8xV8\nXbt2RZ8+fbB582ao1WpkZmbi2LFjJj++sYZjJRIJRowYgbfeegt//fUX1Go1fv/9dwBARUUF3Nzc\n4O7ujnv37mHz5s0mPz8h5qAkRVqV15e+jq3rtuKV3q/grWlv4fdffke7du2s+hqmrhTKy8uxcuVK\nDBo0CKNHj4a3t3ezG30lEgn+9re/4dy5cxg7dqzB19T3+u+99x4yMzMxePBgbNq0CWPHjtW5Rmbs\n8YZea/369XBxccEzzzyDYcOGYdeuXQCAF198EUqlEmFhYZg+fTpGjhzZ7HMSYilejI83hsbHc8NR\nY3fUuAHrxZ6UlISePXtqSsbtgd53bjhq7LwfH08IsZ4LFy7g5s2bYIzh5MmTOHr0KCIjI7kOi5AW\no8IJQlqBO3fuYMGCBSgtLYWvry9WrVqFoKAgrsMipMUoSRHSCoSHhyM8PJzrMAixOjrdRwghhLco\nSRFCCOEtSlKEEEJ4i5IUIYQQ3qIkRQghhLc4T1KffPIJgoKCNPNsCCGEkAacJqnCwkL8/PPPDrlb\nmhBCiO1xmqTeeecdvPbaa1yGQAghhMc4S1JHjhyBn58fHnvsMa5CIIQQwnM27Tgxe/Zs3Llzp8nt\nr776KlJSUnRGVjtAn1tCCCF2xkkX9D///BOzZ8+Gm5sbGGO4ffs2fH198dVXX6Fjx45Njs/IyLB3\niIQQQmzMlC7ovBjVERERgf3798PLy4vrUAghhPAI5yXoQP2QNR7kSkIIITzDi5UUIYQQog8vVlLm\ncLTNvx9++CEmTJiAmJgYSKVSKBQKrkMy2fr16/HMM89g4sSJWLBgAcrLy7kOyWQ//PADxo8fj969\ne+OPP/7gOhyTnDx5Ek8//TSeeuop7Ny5k+twTLZixQo8+eSTiI6O5joUsxQWFuKFF17AuHHjEB0d\njV27dnEdkslUKhViY2MRExOD6OhobNmyheuQzFZXV4dJkyYhMTGx+QOZAykoKGDx8fEsPDyc3b17\nl+twTFJeXq757127drE333yTw2jM8/PPP7Pa2lrGGGPvvfce27BhA8cRme7atWvs+vXrbObMmezi\nxYtch2NUbW0ti4yMZLdu3WIqlYpNmDCBXb16leuwTHLmzBl26dIlNn78eK5DMUtRURG7dOkSY6z+\n53TMmDEO854zxlhlZSVjjLGamhoWGxvLzp8/z3FE5vn000/Z4sWLWUJCQrPHOdRKyhE3/7Zv317z\n30qlEk5OjvOWP/nkk5p4g4ODUVhYyHFEpuvRowcCAwMd5lpnVlYWAgIC4O/vD6FQiHHjxuHIkSNc\nh2WSAQMGwNPTk+swzCaRSNC7d28A9T+nPXv2RFFREcdRmU4sFgOoX1XV1NRwHI15CgsLceLECcTG\nxho91mEm8zry5t/k5GR899138PDwcKhTCtq+/vprjBs3juswWq3bt2/Dz89P82dfX19cuHCBw4ja\nllu3biE7Oxt9+/blOhST1dXVYfLkybhx4waef/55h4q9YcFRVlZm9FheJSlH3fxrKO6kpCREREQg\nKSkJSUlJ2LlzJ3bv3o0FCxZwEKV+xmIHgO3bt0MoFPLumoMpsRNiTEVFBRYuXIgVK1bonPngOycn\nJ3z77bcoLy/HvHnzcPXqVfTq1YvrsIw6fvw4OnXqhN69e+P06dNGj+dVkvr000/13v7nn38iLy8P\nEydO1Gz+nTJlisHNv/ZmKO7GoqOjMWfOHF4lKWOx79u3DydOnODlCtDU990R+Pr6Ij8/X/Pn27dv\no3PnzhxG1DbU1NRg4cKFmDhxIiIjI7kOxyLu7u4ICwvDf/7zH4dIUmfPnsXRo0dx4sQJVFdXo6Ki\nAq+99hrWr1+v93iHuEDy6KOP4ueff8aRI0dw9OhR+Pr6Yv/+/bxIUMbk5uZq/jstLQ09evTgMBrz\nnDx5EjKZDNu3b4dIJOI6HIvxadVtyBNPPIEbN24gLy8PKpUKBw8exOjRo7kOy2SO8B7rs2LFCvTq\n1Qsvvvgi16GYpaSkRHOqrKqqCr/88ovDfLf83//9H44fP44jR47g/fffR1hYmMEEBfBsJWUqR9r8\nu3HjRly/fh1OTk7o2rUrVq1axXVIJluzZg3UajXi4+MBAP369cNbb73FbVAmSktLw+rVq3H37l0k\nJiYiKCgIH3/8MddhGeTs7Iy///3viI+PB2MMU6dORc+ePbkOyySLFy/G6dOnce/ePYwaNQoLFizA\nlClTuA7LqIyMDKSmpuLRRx9FTEwMBAIBkpKSMGLECK5DM0qhUGDZsmWoq6tDXV0dxo4di5EjR3Id\nlk3QZl5CCCG85RCn+wghhLRNlKQIIYTwFiUpQgghvEVJihBCCG9RkiKEEMJblKQIIYTwFiUp4nB6\n9+6NSZMmITo6Gq+++iqqq6v1HpeQkGDReJGioiIsWrTI4vgiIiL0jpKprKzEm2++iaioKEyZMgUv\nvPACsrKyLH4dPsjOzsaJEyf03nfv3j288MILCAkJwZo1a+wcGWktHHIzL2nbxGIx9u/fDwBYsmQJ\n5HI5Zs2apXMMYwwpKSkWPX/nzp3x4YcfWhyfQCDQe/vKlSvRrVs3/PTTTwCAvLw8XL161eLX4YPL\nly/j4sWLejeSurq64tVXX8WVK1dw5coVDqIjrQElKeLQBgwYoOntKJVK0a9fP1y6dAkpKSmYMWMG\n9u3bh4qKCrz88svo378/MjMz4evrq2n1dOPGDfzjH/9ASUkJnJ2d8eGHH8LJyQmJiYlITU3F/v37\n8dNPP6GsrAxFRUWIjo7G/PnzAQCvvPIKCgsLoVKp8MILL2jGDujbH3/z5k1kZWVh48aNmtv8/f3h\n7+8PoL4P4b59+wAAU6dOxYsvvoi8vDy89NJLCA4OxtmzZ/HEE09gypQp2LRpE0pKSrBhwwY88cQT\n2LJlC27cuIEbN27g7t27eOmllzSxvPvuuzh16hQEAgESExMxduxYpKenY/PmzfD29saVK1fQp08f\nvPfeewCAP/74A+vWrUNlZSW8vb2xbt06dOrUCTNnzkS/fv1w+vRplJWVYe3atejbty82bdqE6upq\nnD17FnPmzMEzzzyj+fuJxWKEhobqtAYjxGy2HGpFiC0EBwczxhhTq9Vs7ty5TC6Xs1u3brGgoCCd\nwW8RERHs7t277NatW+zxxx9n2dnZjDHGFi1axA4cOMAYYyw2NpalpaUxxhirrq5mVVVV7NatW5oB\nfvv27WPDhg1jpaWlrKqqio0fP14zRLG0tJQxxjS337t3jzHG9A7lPHLkCJs/f77ev8/FixdZdHQ0\nq6qqYhUVFWzcuHHs8uXLmrivXLnCGGNs0qRJbMWKFYwxxtLS0ti8efMYY4xt3ryZTZw4kVVXV7OS\nkhI2cuRIVlRUxA4fPszi4+MZY4zduXOHjRo1iikUCnb69Gk2YMAAdvv2bVZXV8emTZvGMjIymFqt\nZtOmTWMlJSWMMcYOHjzIli9fzhhjbMaMGWzdunWMMcaOHz/OZs2apXl/Vq9e3ey/lynHEGIIraSI\nw6mursakSZMAAP3798fUqVNx+/Zt+Pv768zUYVorGn9/f80ssscffxx5eXmoqKhAUVGRppGroSa6\nQ4cO1Qz1i4qKQkZGBh5//HF89tlnSEtLA1A/xC03N9eimT4ZGRmIioqCq6ur5jV+//13hIeHw9/f\nX9PZ+pFHHsGQIUMA1Ddd1u6aPnr0aIhEIohEIgwePBjnz59HRkaGZgZYx44dMWjQIFy4cAHt27dH\n3759NV3Wg4KCkJeXBw8PD1y5ckXTP7Curk6nE/uYMWMAAH369NF5bUJsiZIUcThubm6aa1LaGiaV\n6qOdgJydnTXFFsyE1pWNrzEJBAKkp6fjt99+w1dffQWRSISZM2caLOAA6hNMdnY2GGMGr1kZi9vJ\nyUnzZycnp//f3t26qBKFYQB/RBRhQBAxGAyCIviFyIB/gckwDIhxwCpmk5gMGh1Mgt1mM43BYNKg\nwWg0WBxQFD/Q2bCs7L26Xi/sZUfu80vDcGbOOXPCyzu88P7SjfXzOw3DuNsB+vNebTbb9dpqteJ8\nPsMwDASDQXQ6nYdr+X1uon+J1X30cp4JLM8QBAFer/eaDR2PR+z3+5txw+EQ6/Ua+/0emqYhmUxi\ns9nA6XTCbrdjPp9jOp0+nMvn8yEajUJV1eu9xWKBwWAAURShaRoOhwN2ux00TYMoin+1l36/j+Px\nCF3XMRqNEIvFIIoier0eLpcLVqsVxuPxw0zP7/dD13VMJhMA772Wvirs+DgDQRCeqqD8rjOj/w8z\nKXo5z2Yiz4yr1+uoVCpQVRU2mw2NRuPmuXg8jmKxiOVyCUmSEIlErhlHJpOB3+9HIpH447zVahW1\nWg3pdBoOhwMulwulUgnhcBiyLCObzQIAcrnc9Rfcs0KhEBRFga7rKBQK8Hg8SKfTmEwmkCQJFosF\npVIJbrcb8/n87nf62H+1WsVms8HlcoGiKAgEAnezSQBIpVJotVqQZfmmcAJ4L8ffbrc4nU7o9/to\nt9sv04KEzIGtOoge6Ha7mM1mKJfLP72ULzWbTQiCgHw+/9NLIfp2/N1HRESmxUyKiIhMi5kUERGZ\nFoMUERGZFoMUERGZFoMUERGZFoMUERGZFoMUERGZ1hsJLAzXWFxlxAAAAABJRU5ErkJggg==\n", "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "with plt.style.context('seaborn-whitegrid'):\n", " plt.figure(figsize=(6, 4))\n", " for lab, col in zip(('Iris-setosa', 'Iris-versicolor', 'Iris-virginica'), \n", " ('blue', 'red', 'green')):\n", " plt.scatter(Y_sklearn[y==lab, 0],\n", " Y_sklearn[y==lab, 1],\n", " label=lab,\n", " c=col)\n", " plt.xlabel('Principal Component 1')\n", " plt.ylabel('Principal Component 2')\n", " plt.legend(loc='lower center')\n", " plt.tight_layout()\n", " plt.show()" ] } ], "metadata": { "kernelspec": { "display_name": "Python 3", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.6.6" } }, "nbformat": 4, "nbformat_minor": 1 }