{
"cells": [
{
"cell_type": "code",
"execution_count": 1,
"metadata": {
"collapsed": false
},
"outputs": [],
"source": [
"%load_ext watermark"
]
},
{
"cell_type": "code",
"execution_count": 2,
"metadata": {
"collapsed": false
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Sebastian Raschka 2016-06-25 \n",
"\n",
"CPython 3.5.1\n",
"IPython 4.2.0\n",
"\n",
"scikit-learn 0.17.1\n",
"matplotlib 1.5.1\n",
"numpy 1.11.0\n",
"pandas 0.18.1\n"
]
}
],
"source": [
"%watermark -v -d -a 'Sebastian Raschka' -p scikit-learn,matplotlib,numpy,pandas"
]
},
{
"cell_type": "code",
"execution_count": 3,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"%matplotlib inline"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"
\n",
"
"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Principal Component Analysis in 3 Simple Steps"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Principal Component Analysis (PCA) is a simple yet popular and useful linear transformation technique that is used in numerous applications, such as stock market predictions, the analysis of gene expression data, and many more. In this tutorial, we will see that PCA is not just a \"black box\", and we are going to unravel its internals in 3 basic steps."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"This article just got a complete overhaul, the original version is still available at [principal_component_analysis_old.ipynb](http://nbviewer.ipython.org/github/rasbt/pattern_classification/blob/master/dimensionality_reduction/projection/principal_component_analysis.ipynb)."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"
\n",
"
"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"
"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Sections"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"- [Introduction](#Introduction)\n",
" - [PCA Vs. LDA](#PCA-Vs.-LDA)\n",
" - [PCA and Dimensionality Reduction](#PCA-and-Dimensionality-Reduction)\n",
" - [A Summary of the PCA Approach](#A-Summary-of-the-PCA-Approach)\n",
"- [Preparing the Iris Dataset](#Preparing-the-Iris-Dataset)\n",
" - [About Iris](#About-Iris)\n",
" - [Loading the Dataset](#Loading-the-Dataset)\n",
" - [Exploratory Visualization](#Exploratory-Visualization)\n",
" - [Standardizing](#Standardizing)\n",
"- [1 - Eigendecomposition - Computing Eigenvectors and Eigenvalues](#1---Eigendecomposition---Computing-Eigenvectors-and-Eigenvalues)\n",
" - [Covariance Matrix](#Covariance-Matrix)\n",
" - [Correlation Matrix](#Correlation-Matrix)\n",
" - [Singular Vector Decomposition](#Singular-Vector-Decomposition)\n",
"- [2 - Selecting Principal Components](#2---Selecting-Principal-Components)\n",
" - [Sorting Eigenpairs](#Sorting-Eigenpairs)\n",
" - [Explained Variance](#Explained-Variance)\n",
" - [Projection Matrix](#Projection-Matrix)\n",
"- [3 - Projection Onto the New Feature Space](#3---Selecting-Principal-Components)\n",
"- [Shortcut - PCA in scikit-learn](#Shortcut---PCA-in-scikit-learn)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"
\n",
"
"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"
"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Introduction"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"[[back to top](#Sections)]"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"The sheer size of data in the modern age is not only a challenge for computer hardware but also a main bottleneck for the performance of many machine learning algorithms. The main goal of a PCA analysis is to identify patterns in data; PCA aims to detect the correlation between variables. If a strong correlation between variables exists, the attempt to reduce the dimensionality only makes sense. In a nutshell, this is what PCA is all about: Finding the directions of maximum variance in high-dimensional data and project it onto a smaller dimensional subspace while retaining most of the information."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"
\n",
"
"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### PCA Vs. LDA"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"[[back to top](#Sections)]"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Both Linear Discriminant Analysis (LDA) and PCA are linear transformation methods. PCA yields the directions (principal components) that maximize the variance of the data, whereas LDA also aims to find the directions that maximize the separation (or discrimination) between different classes, which can be useful in pattern classification problem (PCA \"ignores\" class labels). \n",
"***In other words, PCA projects the entire dataset onto a different feature (sub)space, and LDA tries to determine a suitable feature (sub)space in order to distinguish between patterns that belong to different classes.*** "
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"
\n",
"
"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### PCA and Dimensionality Reduction"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"[[back to top](#Sections)]"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Often, the desired goal is to reduce the dimensions of a $d$-dimensional dataset by projecting it onto a $(k)$-dimensional subspace (where $k\\;<\\;d$) in order to increase the computational efficiency while retaining most of the information. An important question is \"what is the size of $k$ that represents the data 'well'?\"\n",
"\n",
"Later, we will compute eigenvectors (the principal components) of a dataset and collect them in a projection matrix. Each of those eigenvectors is associated with an eigenvalue which can be interpreted as the \"length\" or \"magnitude\" of the corresponding eigenvector. If some eigenvalues have a significantly larger magnitude than others that the reduction of the dataset via PCA onto a smaller dimensional subspace by dropping the \"less informative\" eigenpairs is reasonable.\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"
\n",
"
"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### A Summary of the PCA Approach"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"[[back to top](#Sections)]"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"- Standardize the data.\n",
"- Obtain the Eigenvectors and Eigenvalues from the covariance matrix or correlation matrix, or perform Singular Vector Decomposition.\n",
"- Sort eigenvalues in descending order and choose the $k$ eigenvectors that correspond to the $k$ largest eigenvalues where $k$ is the number of dimensions of the new feature subspace ($k \\le d$)/.\n",
"- Construct the projection matrix $\\mathbf{W}$ from the selected $k$ eigenvectors.\n",
"- Transform the original dataset $\\mathbf{X}$ via $\\mathbf{W}$ to obtain a $k$-dimensional feature subspace $\\mathbf{Y}$."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"
\n",
"
"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Preparing the Iris Dataset"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"[[back to top](#Sections)]"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"
\n",
"
"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### About Iris"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"[[back to top](#Sections)]"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"For the following tutorial, we will be working with the famous \"Iris\" dataset that has been deposited on the UCI machine learning repository \n",
"([https://archive.ics.uci.edu/ml/datasets/Iris](https://archive.ics.uci.edu/ml/datasets/Iris)).\n",
"\n",
"The iris dataset contains measurements for 150 iris flowers from three different species.\n",
"\n",
"The three classes in the Iris dataset are:\n",
"\n",
"1. Iris-setosa (n=50)\n",
"2. Iris-versicolor (n=50)\n",
"3. Iris-virginica (n=50)\n",
"\n",
"And the four features of in Iris dataset are:\n",
"\n",
"1. sepal length in cm\n",
"2. sepal width in cm\n",
"3. petal length in cm\n",
"4. petal width in cm\n",
"\n",
"![\"Iris\"](\"http://sebastianraschka.com/Images_old/2014_python_lda/iris_petal_sepal.png\")
"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"
\n",
"
"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Loading the Dataset"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"[[back to top](#Sections)]"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"In order to load the Iris data directly from the UCI repository, we are going to use the superb [pandas](http://pandas.pydata.org) library. If you haven't used pandas yet, I want encourage you to check out the [pandas tutorials](http://pandas.pydata.org/pandas-docs/stable/tutorials.html). If I had to name one Python library that makes working with data a wonderfully simple task, this would definitely be pandas!"
]
},
{
"cell_type": "code",
"execution_count": 4,
"metadata": {
"collapsed": false
},
"outputs": [
{
"data": {
"text/html": [
"\n",
"
\n",
" \n",
" \n",
" | \n",
" sepal_len | \n",
" sepal_wid | \n",
" petal_len | \n",
" petal_wid | \n",
" class | \n",
"
\n",
" \n",
" \n",
" \n",
" | 145 | \n",
" 6.7 | \n",
" 3.0 | \n",
" 5.2 | \n",
" 2.3 | \n",
" Iris-virginica | \n",
"
\n",
" \n",
" | 146 | \n",
" 6.3 | \n",
" 2.5 | \n",
" 5.0 | \n",
" 1.9 | \n",
" Iris-virginica | \n",
"
\n",
" \n",
" | 147 | \n",
" 6.5 | \n",
" 3.0 | \n",
" 5.2 | \n",
" 2.0 | \n",
" Iris-virginica | \n",
"
\n",
" \n",
" | 148 | \n",
" 6.2 | \n",
" 3.4 | \n",
" 5.4 | \n",
" 2.3 | \n",
" Iris-virginica | \n",
"
\n",
" \n",
" | 149 | \n",
" 5.9 | \n",
" 3.0 | \n",
" 5.1 | \n",
" 1.8 | \n",
" Iris-virginica | \n",
"
\n",
" \n",
"
\n",
"
\n",
"
"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Exploratory Visualization"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"[[back to top](#Sections)]"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"To get a feeling for how the 3 different flower classes are distributes along the 4 different features, let us visualize them via histograms."
]
},
{
"cell_type": "code",
"execution_count": 6,
"metadata": {
"collapsed": false
},
"outputs": [
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAjgAAAGoCAYAAABL+58oAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzs3XtAVHX+P/4n12FmAEcgScRE0SzDT1ui6yfXshaMQttN\nu6wXzNbdNs39WVrhhcRbmtnFT6Uu5KorfoqtjdYQt/1+yNI085ZioGampsPgJIMjwzAMM8z8/mCZ\nQGaGmcOZq8/HP8yc87683jPD4cWZc97vEKvVagURERFREAn1dQBEREREYmOCQ0REREGHCQ4REREF\nHSY4REREFHSY4BAREVHQYYJDREREQcelBKeiogI5OTkAgJMnT2LKlCmYNm0a/vCHP6Curs6jARLR\n9aP9saaurg6zZs1CTk4OJk+ejIsXL/o4OiIKJOFdFdi4cSO2b98OuVwOAFi5ciUWL16MwYMH4+9/\n/zsKCwsxf/58jwdKRMHt2mPNmjVr8NBDDyErKwsHDhzA2bNn0bdvXx9HSUSBosszOP369cO6dets\nz998800MHjwYAGA2myGRSDwXHRFdN6491nzzzTe4dOkSnnzySezYsQO//OUvfRgdEQWaLhOczMxM\nhIWF2Z4nJCQAaD34vPfee5g+fbrHgiOi68e1x5rq6mooFAps3rwZN954IwoLC30YHREFmi6/orJn\n586dKCgoQGFhIXr27Gm3zJEjR7oVGBEFhmHDhnmkXYVCgXvvvRcAcN9992Ht2rV2y/FYQ3R9cPdY\n43aCs337dnzwwQcoKipCbGysqMGISaVSISkpyaN9bNhQiuTk8Z22azQaxMfHAwCUylLMnNm5jKd4\nY9zs27/692Xfnkwuhg0bht27d+Ohhx7CoUOHMHDgQKdlA52vP8NiKVq1CjlpaYLrlyqVGD9zpogR\nCRMs70ewjEPIscatBMdisWDlypVISkrCM888g5CQEIwYMQKzZ892u2MiImdyc3ORl5eH999/HzEx\nMXj99dd9HRIRBRCXEpw+ffqguLgYAHDgwAGPBkRE16/2x5qkpCRs2rTJxxERUaDiRH9EREQUdJjg\nEBERUdBhgkNERERBhwkOERERBR0mOERERBR0mOAQERFR0BE0kzEREZG/Kykph1ptEFRXq9VCoVB0\n2p6YKMWECRndDY28gAkOEREFJbXaYHe2eVdIpT/PSN+eUlna3bDIS/gVFREREQUdJjhEREQUdJjg\nEBERUdBhgkNERERBhwkOERERBR0mOETkNyoqKpCTk9NhW2lpKX73u9/5KCIiClS8TZyI/MLGjRux\nfft2yOVy27YTJ07go48+8mFURBSoeAaHiPxCv379sG7dOtvzK1euYO3atVi0aJEPoyKiQMUEh4j8\nQmZmJsLCwgAAFosFeXl5mD9/PqRSKaxWq4+jI6JAw6+oiMjvVFVV4cKFC1iyZAmMRiN++OEHrFq1\nCgsWLLBbXqVSeTlC8el0uqAYR5PRCI1GI7i+VqsV7XXQarWQSoXFYjAY7I5DzPi8IVg+V0IwwSEi\nv2K1WjF06FCUlrZOiV9dXY158+Y5TG4AICkpyVvheYxKpQqKcURJJHaXOHCVwmAQ7XVQKBSCY9Fo\n7C/VYDAoAup9CpbPVU1Njdt1+BUVEfmVkJAQX4dAREHApQSn/a2bFy5cwOTJkzF16lQsXbrUo8ER\n0fWlT58+KC4u7nIbEVFXukxwNm7ciLy8PJhMJgDAqlWrMHfuXGzbtg0WiwXl5eUeD5KIiIjIHV0m\nONfeullVVYX09HQAwN133439+/d7LjoiIiIiAbpMcNrfugmgw+2acrkcOp3OM5ERERERCeT2XVSh\noT/nRHq9HrGxsQ7L+vLWNG/cGufoFsT2txd6+5bCtnGXfVaG2vpat+omxCYg+9fZ3e7bF4T0/WVZ\nGZpr3XuNHGmRyzH2kUdEactd1/NtoEREjrid4AwZMgSHDh3C8OHDsWfPHowcOdJhWV/emuaNW+Mc\n3YLY/vZCb99S2DZuc6gZaRlpbtVVHlZ2K1Zf3o4opG+Z2YzH09x7jRwpqqwMqLGLRcitm0RE3uB2\ngpObm4uXXnoJJpMJqampyMrK8kRcRERERIK5lOC0v00zJSUFRUVFHg2KiIiIqDs40R8REREFHSY4\nREREFHSY4BAREVHQYYJDREREQYcJDhEREQUdJjhEREQUdJjgEJHfqKioQE5ODgDg5MmTmDJlCqZN\nm4Y//OEPqKur83F0RBRImOAQkV/YuHEj8vLyYDKZAAArV67E4sWLsXXrVmRmZqKwsNDHERJRIGGC\nQ0R+oV+/fli3bp3t+ZtvvonBgwcDAMxmMyQSia9CI6IAxASHiPxCZmYmwsLCbM8TEhIAAN988w3e\ne+89TJ8+3UeREVEgcnstKiJfKi8pgUGt7rRdq9VCoVC41dbJo0cxPjlZrNDIA3bu3ImCggIUFhai\nZ8+eDssFw2rqYq0K/2VZGZprawXXj0xIwOjsbMH1m4xGaDQawfW1Wq1o76dWq4VUKiwWg8Fgdxxi\nxucNYn2uAhETHAooBrXablKikUrtruzuTMWePWKFRR6wfft2fPDBBygqKkJsbKzTsr5aTV1MYq0K\nLzOb8XhamuD6pUplt+KIkkjc/l1sT2EwiPZ+KhQKwbFoNBq7dQ0GRUB93sT6XPlaTU2N23WY4BCR\n37FYLFi5ciWSkpLwzDPPICQkBCNGjMDs2bN9HRoRBQgmOETkN/r06YPi4mIAwIEDB3wcDREFMl5k\nTEREREGHCQ4REREFHSY4REREFHSY4BAREVHQYYJDREREQYcJDhEREQUdQbeJm81m5Obmorq6GuHh\n4Vi+fDn69+8vdmxEREREgghKcHbv3g2LxYLi4mJ89dVXePPNN/HWW2+JHRsREQn01aGvoNVr3a53\n1NCC8V2UKSkph1pt6FjvxFfQGbW4euwoLn3h+izh0gg5BvS9FQpFJO666xdux0vkiKAEJyUlBS0t\nLbBardDpdIiIiBA7LiIi6gatXouEgQlu16s/eKbLMmq1AcnJHdOg4+eVGDjiLpyvb0LvJNfXeKv/\nsRYJCSNQW3vQ7ViJnBGU4MjlciiVSmRlZUGr1aKgoEDsuIiIiIgEE5TgbNmyBaNHj8Zzzz0HtVqN\nadOmobS0FJGRkR3K+XIFU2+soOpopdr2q9Du3XsAWq3z08QJCZHIzh4tSkxt49ZqtZDWSd2qK2SV\n3LLPylBb37pysdFohEQi6VTmwtFKhDY02J5HR0UjbbCwxQDPVFbirszMTtsdrfzrjKGxsVurHrfX\nZDT67PN+Pa8WTETkiKAEp0ePHggPb60aExMDs9kMi8XSqZwvVzD1xgqqjlaqbb8KrcUShbS0HKft\nKJWlosXaNm6FQoH4OPdW0TUo3F/F1xxqRlpGa7KiqdPY7bPl8gWMTRxge157phYPClzteMXx412+\n5q6SymTdWvW4vaiaGp993n25WrCQFX6JiLxBUILzxBNPYOHChZgyZQrMZjPmzZuHqKgosWMjIiIi\nEkRQgiOTybB27VqxYyGi61xFRQVee+01FBUV4cKFC5g/fz5CQ0MxaNAg5Ofn+zo8IgognOiPiPzC\nxo0bkZeXB5PJBABYtWoV5s6di23btsFisaC8vNzHERJRIGGCQ0R+oV+/fli3bp3teVVVFdLT0wEA\nd999N/bv3++r0IgoADHBISK/kJmZibCwMNtzq9VqeyyXy6HT6XwRFhEFKEHX4BAReVpo6M//f+n1\nesTGxjosGwy3yYt1u79Wq4VGKoVer0dkQ+dpG7piMDR2GYe9KTL0+gZIdPUwt5hhMDS63F9TUxPq\ndfXQ6/XQaDSCpqtwJ05XOZp6Qsz4vOF6nkaCCQ4R+aUhQ4bg0KFDGD58OPbs2YORI0c6LOvLKSnE\nItbt/m3TV8jlcsRGx7hdXyqVdRmHvSky5PJoxMTEQhMWDqlU5nJ/pqhGxMbEotkoR3x8PBQG96er\ncCdOVzmaesJgUATU582X00iISciUFExwiMgv5ebm4qWXXoLJZEJqaiqysrJ8HRIRBRAmOETkN/r0\n6YPi4mIArWveFRUV+TgiIgpUvMiYiIiIgg4THCIiIgo6THCIiIgo6DDBISIioqDDBIeIiIiCDhMc\nIiIiCjpMcIiIiCjoMMEhIiKioMMEh4iIiIIOExwiIiIKOkxwiIiIKOgwwSEiIqKgwwSHiIiIgo7g\n1cQLCwuxa9cumEwmTJ48GRMnThQzLiIimM1m5Obmorq6GuHh4Vi+fDn69+/v67CIKAAIOoNz8OBB\nHD16FMXFxSgqKkJNTY3YcRERYffu3bBYLCguLsasWbPw5ptv+jokIgoQgs7g7N27FzfffDNmzZoF\nvV6PF198Uey4iIiQkpKClpYWWK1W6HQ6RERE+DokIgoQghKcK1euQKVSoaCgABcvXsTMmTPx6aef\nih0bEV3n5HI5lEolsrKyoNVqUVBQ4OuQiChACEpwFAoFUlNTER4ejv79+0MikaCurg5xcXEdyqlU\nKlGCFEKn03m8f61WC6lU02m7wWCARtO6vbHx58fO2nEWa9lnZaitr3UpJqPRCIlEgspTlcgckOlS\nHVfjcFRHWicF8J9x13Ueq16vR32DpMPzrl4TRwyNjXbrtn/Nu9uWEE1Go88+7974rPvKli1bMHr0\naDz33HNQq9WYNm0aSktLERkZ2aFcMIxfrPdRq9VCI5VCr9cjst3vnasMhsYu47B37NPrGyDR1cPc\nYobB0Ohyf01NTajX1duOCwf27oVWq3U77jaRCQkYnZ3tME5XOTqmCDlO+lIwHx+6IijBGTZsGIqK\nijB9+nSo1Wo0NTWhZ8+encolJSV1O0ChVCqVx/tXKBSIj4/vtF2j0di2y2RSu2XaMxgUTmM1h5qR\nlpHmUkyaOg3i4+Jx/PRxxMc577dTHAqD26+ZQqGw9dPW97Xkcjlio2Nsz5vlxi5fE0ekMlmXr3l3\n2xIiqqbGZ593b3zWHfH09Xc9evRAeHjrYSomJgZmsxkWi6VTOV8ea8Qi1vvYdly69vfOVVKprMs4\n7B375PJoxMTEQhMWDqlU5nJ/pqhGxMbEotkoR3x8PKIsFuSkuXa8s6dUqbTF7+gY7QpHx5Sujtf+\nxpfHBzEJOdYISnDGjBmDw4cP45FHHoHVakV+fj5CQkKENEVE5NATTzyBhQsXYsqUKTCbzZg3bx6i\noqJ8HRYRBQDBt4k///zzYsZBRNSJTCbD2rVrfR0GEQUgTvRHREREQYcJDhEREQUdJjhEREQUdJjg\nEBERUdBhgkNERERBhwkOERERBR0mOERERBR0BM+DQ0REwedi9UVs+N8NTsvsOXocivPKDtvOXjiK\nhLRkwf2ePfsjdu4Evv++Gjt3HhTczm5tLZQoBQAcPXoSycnjBbdFgY0JDhER2TSZmpCc7jxRUVxW\nISG+Y5mqH/Z0q1+DwYKEhBGQyc4gIWGE4HYUUNqSmj17KroVEwU2fkVFREREQSeozuCUlJRDrTYA\naF3xVaFQdCqTmCjFhAkZLrfjjLdPf361/xi02manZeoqv8XQ5N5oPP49Kt7f6Vb73371LX4o//m/\nsEs/1eLGXglO61ysvoih3w8F0LpKuFwu71Sm5sRZIPHnds6eP4udX7gXm0KuwF3D73KrzleHvoJW\n73hV4u/Pf98pDiH9AMB3lZUo3eD8tL4rpImJyJgwodvtAEB5SQkMarUobYkZFxGRNwRVgqNWG2wJ\nh1RqfyVYpbLUrXac8fbpT622GQnxzk/dqn/6CmOHDYUkWoZ7Ep0nJ9eSSEJxz4iBtuebd1TjsXbP\n7dm9s9rWT32DxO7qxZsPVXV4bmg2IGGge7HVnql1qzwAaPVap/3ITsk67RfSDwBYGxowPln49Qdt\nSpXKrgu5yKBWixITIG5cRETewK+oiIiIKOgE1RkcIgo+hYWF2LVrF0wmEyZPnoyJEyf6OiQiCgBM\ncIjIbx08eBBHjx5FcXExGhsbsWnTJl+HREQBggkOEfmtvXv34uabb8asWbOg1+vx4osv+jokIgoQ\nTHCIyG9duXIFKpUKBQUFuHjxImbOnIlPP/3U12ERUQBggkNEfkuhUCA1NRXh4eHo378/JBIJ6urq\nEBcX16GcSqXyWkxln5Whtt79u+0SYhOQ/etsh/t1Op0o49BqtdBIpdDr9YhskLhd39RsgqZO47SM\nXq+HJLK+wzaj0Qidrh7mFjMMhkaX+2tqakK9rh5GY/N/fhpRr6vvuqLD2Bqg0bTG39hosD12l8Fg\nv65WqxX981ZW9iVqa51PAeKuhIRIZGePFu1zFYiY4BCR3xo2bBiKioowffp0qNVqNDU1oWfPnp3K\nJSUleS0mc6gZaRlpbtdTHlY6jVOlUokyDoVCgfj4eMjlcrvTNnQlIjIC8XGdp9hoTy6XIyYmtsM2\niUSCmJhYaMLCIZXKXO7PFNWI2JhYSCSR//kpQew1bbtDbqy3TREik0ntThfiCo3G/lQjBoNC9M+b\n2SxDWtrjorapVJYiKSlJtM+Vr9XU1LhdhwkOEfmtMWPG4PDhw3jkkUdgtVqRn5+PkJAQX4dFRAGg\nWwmORqPBxIkTsXnzZvTv31+smIiIbJ5//nlfh0BEAUjwRH9msxn5+fmIiooSMx4iIiKibhOc4Kxe\nvRqTJk1Cr169xIyHiIiIqNsEfUVVUlKC+Ph4jBo1Cn/5y1/EjqkDi8WC6upqWK1Wp+WkUqlH47iW\nTlcHvV6Ly5cvdNp35coVWCx6AOhQJjQ0HPHxgX+xFxERkb8TnOCEhIRg3759OHXqFHJzc7Fhw4ZO\nV5yLcWvapUuXUFZ2DpGRzhdnDA29ALPZCqm09ba+7tzip9Vqbe04sr/iHzhvrsTuHz7stK/Z1IzI\nukgAwPmWKlsZi64Fd936GKTSjnc27N17AFqt41Wvvz5xAD3Of48ff1Thjl/c4jSulhYz6ht0rbda\nNuiclr3WtXWMzV230b5Os9EIezd3XtuOkNj0ej00Gg0MjY1231d773dXt8naG19bP+4ym82Cb0dt\nT8gtqI5uA227XVgMnrg1lojIkwQlONu2bbM9zsnJwbJly+zeTifWrWm9elmRlDTMaRmlcges1hZb\nHN25xa/tNktnZNFSxN7YEzfdcnOnfTpdve0WynM/HbWVqftehZ49e0Iu79GhvMUShbS0HId9XdA2\nIOGmZJw///dOt2ZeKywsHLHRMa23Wrp5i+i1dSSRXbfRvk49YLf8te0Iia1ZbkR8fDykMpnd98be\n+93VbbL2xtfWj7vCw8MF347ansJgcPv3xtFtoK58jrsbl5BbN4mIvKHbq4nzlk0iIiLyN92eB2fr\n1q1ixEFEREQkGk70R0RE5KKjRyuxYYPYbZ5EcvJ4cRslJjhERESu0ulaRE9G9uypELU9atXta3CI\niIiI/A0THCIiIgo6THCIiIgo6DDBISIioqDDBIeI/J5Go8GYMWNw7tw5X4dCRAGCCQ4R+TWz2Yz8\n/HxERUX5OhQiCiBMcIjIr61evRqTJk1Cr169fB0KEQUQJjhE5LdKSkoQHx+PUaNGwWq1+jocIgog\nnOjvOnTyu7PQN5g7bVepNDh85LTteW3tFW+G5dTZ82ex84ud+P7899j5xc5O+/V6PeRyecc6P55F\nwkDnq9A76sddKjUXnfSEkpIShISEYN++fTh16hRyc3OxYcOGTouIenOlc61WC2md41XaT39xGNa6\n+k7bTx08hcrSTx3WM5nNiAgPhyU6GjfdkWbbnhCbgOxfZ7sVn0YqhV6vR2SDxOV6tjiaTdDUaZyW\n0ev1kER2HKPRaIROVw9zixkGQ6PL/TU1NaFeVw+jsfk/P42o13V+/Vyl1zdAo2mNv7HRYHvsLoPB\nft3utOmIJ9rUarVQqVTQ6XRe/f3wJ0xwrkP6BjNiYzuvgi6R1HTY3myq9GZYThmaDUgYmADZKZnd\npCWyofPK4FWnqwT34y7jF0a361DXtm3bZnuck5ODZcuW2V0h3d0V2LtDoVAgPs7xKu0yYwvGDujX\nafvuU+dxzz3/5bBefYMOsdEx+H/qWqRl/JzgKA8r3Rpf2yrycrm80++EKyIiI5yODwDkcjliYmI7\nbJNIJIiJiYUmLBxSqczl/kxRjYiNiYVEEvmfnxLEXtO2O+TGettnRCaT2v28uEKj0dit2502HfFE\nmwaDAklJSVCpVF79/fCUmhr3/4nkV1REFBBCQkJ8HQIRBRCewSGigLB161Zfh0BEAYRncIiIiCjo\nMMEhIiKioMMEh4iIiIIOExwiIiIKOkxwiIiIKOgwwSEiIqKgI+g2cbPZjIULF6K6uhomkwlPP/00\n7rvvPrFjIyIiIhJEUILzySefoGfPnnj11Vdx9epV/Pa3v2WCQ0TkAUePH8UGbHC5/A8H9yDsjELQ\nUiW+cOmnsziMnVBpT+PwiZ1QXf4eh090vVyKPFKBWwfe5YUIKVAJSnAeeOABZGVlAQAsFgvCwzlf\nIBGRJ+iadEhOT3a5vOZ7BRISEwQtVeILzS0GxPZLgEQna/1Z3fqzK/U/1nohOgpkgjITqbR1obmG\nhgbMmTMHzz33nKhBEREREXWH4FMvNTU1mD17NqZOnYoHH3zQbhkxVjC9dOkStFotJBLnK61evaqF\n1WqBTNZaztFKsHv3HoBWq3XaVmXlGWRmOj/12dDQAKOxGTo7q94am5uB/2xvX6ZB34C6ujo0NXVc\nyburlWT1+gZI/rParr3+2qutrcPer47j4gU19n513G6Zyz/VYcCAzv8RmsymDqsAm81m1DfonPZn\nNBptZZqNRtiLzths7NBO+zquaqtzbVtt7PXdVT/22hISGwC0WFpEWQ34wN69XX4+r9VkNCJK0nnV\n6DOVlbgrM7PbMQE/r0xMRBQoBCU4tbW1mDFjBhYvXoyRI0c6LCfWCqYKhbXLlVYNBgWs1hZbOUcr\nwVosUUhLy3Ha1vHjK7rsLzo6GpKrkZ1W1AUA6Opt2yWSn8uY5A2Ii4uDXN6jQ/GuVpKVy6MR85/V\ndu32105LSyh63fBf+El+Bb1usL9y8U/qL+2u9hsRHtFhe3h4eJerEUskP6/iXQ/YLS+J7LjSd/s6\nrmqrc21bbez13VU/9toSEhsAhIWGibIacJTFgpy0tK4LtuPos77i+HHRVihWGAx2f5+FrPBLROQN\ngm4TLygoQH19PdavX4+cnBxMmzYNzc3NYsdGREREJIigMziLFi3CokWLxI6FiKgDTklBRELx9ici\n8luckoKIhGKCQ0R+i1NSEJFQPFoQkd/ilBREJBQTHCLya56akqLsszLU1rs/WVzlqUpkDnB8+71e\nr0d9Q+fb9ruagqBtqoMzR09Cr9fbtl89fAJfbfzQ5fh+Ov0j6n91h+ApD66qLnfZn/q7ajTIlB22\ntXx3EufLSqD98SwMPeNc7q9tegqTyQyDoRFmU8fpKhy5WH0KTU1NnbbvPncC35z4PwCAUnkKm1Xf\ndNgvjYjGTUlDHbYbEnsDbk4f73Cqka6m9Th9uBTW+stdxt/e1RN78dWHazr0311tU6IYjU2QSKK6\n3R4AXLjwA266KVWUttokJEQiO3u0qG22YYJDRH7Lk1NSmEPNSMtw75Z8ADh++jji45xN6yC3P11C\nF1MQtE11IDVb8PCAfrbtcafO4552z7uy+cT51ikVBE55ILOiQ//2HL5iRGxsx7m0Ki/8gLSkZJR8\nf8ruNBSOtE1PERERDqlUhvCICJfqW0Nb0Gtw5/m8emp+wEPDbmuNSVKHtLTbOuyv/7EW6f1ucdju\n/6tVIj4+3uH0C11N6yFrMWCsk/btiTtfgXv+U6et/+5qmxLF0TiEOH58BR580Pk0K+5SKktd+v0V\nMiUFVxMnIr/FKSmISCiewSEiv8UpKYhIKJ7BISIioqDDBIeIiIiCDr+iEshoNKC50QCdtq7TPn2D\nDmhpXVCzfRm97iquXLkEo7HjHQImk9HzARMREV1HmOAIVP99BW5SKxEtPdhpX3iTAVFRrfN33PTj\nRURHtpYJ+UmLsydPwmT5+dbGFqsF59TnsHOv46vIz144ioS0zncLEBERkX1McAQKtZjRLyoKt9iZ\n78FgaLTd5miWyWxldAbAZGhCXOpAW9lGYxOO4pLTBKbqhz0iR09ERBTceA0OERERBR2ewSGigFfy\naYnbderr6zttu1p3FZcOn3BaT3/mIr755//horLz9XcAYFTXoq9W32l7be1VnPzuR4ftNhkaMXiw\n6xP6EZFzTHCIKOCZbjC5Vf6y6nKH5RDaNFxtwI3qWiTFKRzWNbRYcKtOD9mVcMjlPTq2W38VVU0x\nsLTc2KmexXrG7vY2uoYaNBk7Lz3QXSe/Owt9g9nl8jpdIw4fOe20zCV1LWJjb+5uaH7p4tmjwM4N\n0OsbIJdHd9rf+P0BVOzc4LB+zdmjQAKvmfQHTHCIKODJol1fGgAAIiIjHO8LD0e0tPNaUm2iIsIh\ni4pEtDQEMfLYDvt0RhPCjc2IiOhcPyw03O72n/eHuRC5+/QNZreSkbCwr7ssf+GCurth+a1Qgw5j\nE5JRL6lHbExsp/0SWQ/c4ySB2VzFayb9Ba/BISIioqDDBIeIiIiCDhMcIiIiCjpMcIiIiCjoCLrI\n2Gq1YsmSJfjuu+8QGRmJl19+GX379hU7NiK6zvFYQ0RCCTqDU15ejubmZhQXF2PevHlYtWqV2HER\nEfFYQ0SCCUpwjhw5gtGjRwMAbr/9dlRWVooaFBERwGMNEQkn6CuqhoYGxMTE/NxIeDgsFgtCQz1z\nSU9zcxOuXPG/eRd0jY346XLnuAxGI6SS1vkuruoabGWMjU1oMZoRpm+wlTWa3JugjOh64uqx5krt\nFbfaNTQaHO5rMpqgqW9wuF9nbMaVhkbU6cwwmFo67tM3oNnUhKamzpMIms3Ndre3aWnhsYBITCFW\nq9XqbqVXXnkFv/jFL5CVlQUAGDNmDL744osOZY4cOSJKgETk34YNG+axtnmsIaI27h5rBJ3BufPO\nO/H5558jKysLx44dw803d5710pMHPSK6PvBYQ0RCCTqD0/7OBgBYtWoV+vfvL3pwRHR947GGiIQS\nlOAQERG11qUqAAAgAElEQVQR+TNO9EdERERBR7TVxDUaDSZOnIjNmzd3OIW8a9curF+/HuHh4Zg4\ncSIeffRRsbrssu8tW7bgH//4B+Li4gAAy5YtQ0pKimj9TpgwAdHR0QCA5ORkrFy50rbP0+N21ren\nxw0AhYWF2LVrF0wmEyZPnoyJEyfa9nl67M769uTYP/74Y5SUlCAkJARGoxGnTp3Cvn37bO+DJ8fd\nVd+eHLfZbEZubi6qq6sRHh6O5cuXe/133Gw2Y+HChaiurobJZMLTTz+N++67z6sxiKGrcXjjd1cM\nFosFeXl5OHfuHEJDQ7F06VIMHDjQtj9Q3o+uxhEo70cbX/4dFpNof9OtIjCZTNZnnnnGev/991vP\nnj3bYXtmZqZVp9NZm5ubrRMnTrRqNBoxuuyyb6vVan3++eetVVVVovbXxmg0Wh9++GGHMXly3M76\ntlo9O26r1Wo9cOCA9emnn7ZarVarXq+3vv3227Z9nh67s76tVs+Pvc3SpUutH3zwge25Nz7rjvq2\nWj077vLycuuzzz5rtVqt1n379ln//Oc/2/Z5a9wfffSRdeXKlVar1WrVarXWMWPGeD0GMTgbh9Xq\nvc9vd/3f//2fdeHChVartfV3cubMmbZ9gfR+OBuH1Ro474fV6tu/w2IS82+6KF9RrV69GpMmTUKv\nXr06bP/hhx/Qr18/REdHIyIiAsOGDcOhQ4fE6LLLvgGgqqoKBQUFmDx5MgoLC0Xt99SpU2hsbMSM\nGTMwffp0VFRU2PZ5etzO+gY8O24A2Lt3L26++WbMmjULM2fOxL333mvb5+mxO+sb8PzYAeDbb7/F\nmTNnOvwX5I3PuqO+Ac+OOyUlBS0tLbBardDpdIiIiLDt89a4H3jgAcyZMwdA63/d4eE/n3z2Vgxi\ncDYOwDufXzFkZGRg+fLlAIDq6mr06NHDti+Q3g9n4wAC5/0AfPt3WExi/k3vdoJTUlKC+Ph4jBo1\nCtZrrle+dpIuuVwOnU7X3S5d6hsAsrOzsXTpUmzduhVHjhzB7t27Res7KioKM2bMwF//+lcsWbIE\nzz//PCwWCwDPj9tZ34Bnxw0AV65cQWVlJd566y0sWbIE8+bNs+3z9Nid9Q14fuxA61dks2fP7rDN\n0+N21jfg2XHL5XIolUpkZWVh8eLFyMnJse3z1rilUilkMhkaGhowZ84cPPfcc16PQQzOxgF45/Mr\nltDQUMyfPx8vv/wyxo8fb9seSO8H4HgcQOC8H778Oywmsf+mi5Lg7Nu3Dzk5OTh16hRyc3Oh0WgA\nANHR0Who+HlGUL1ej9jY2O526VLfAPDEE09AoVAgPDwc99xzD06cOCFa3ykpKXjooYdsjxUKBS5f\nvgzA8+N21jfg2XEDgEKhwOjRoxEeHo7+/ftDIpGgrq4OgOfH7qxvwPNj1+l0OH/+PEaMGNFhu6fH\n7axvwLPj3rJlC0aPHo1///vf+OSTT5Cbm4vm5mYA3hl3m5qaGjzxxBN4+OGH8eCDD9q2ezMGMTga\nB+D5z6/YXnnlFfz73/9GXl4empqaAATe+wHYHwcQOO+HL/8Oi0nsv+ndTnC2bduGoqIiFBUV4ZZb\nbsHq1asRHx8PAEhNTcWPP/6I+vp6NDc349ChQ/jFL37R3S5d6ruhoQHjxo2DwWCA1WrF119/jdtu\nu020vj/66CO88sorAAC1Wg29Xo8bbrgBgOfH7axvT48baJ1Y7csvv7T139TUhJ49ewLw/Nid9e2N\nsR86dAgjR47stN3T43bWt6fH3aNHD9vFzDExMTCbzbYzht4YNwDU1tZixowZeOGFF/Dwww932Oet\nGMTgbBze+PyKZfv27bavCCQSCUJDQ23LZwTS++FsHIH0fvjy77CYxP6bLuo8ONOmTcPSpUtRVVUF\ng8GARx99FF988QXeeecdWK1WPPLII5g0aZJY3XXZ9yeffIKtW7dCIpHgv//7v+2e2hfKZDJhwYIF\nUKlUCA0NxfPPPw+lUumVcXfVtyfH3ea1117D119/DavVirlz5+LKlStee8+d9e3psf/1r39FREQE\npk2bBgDYsWOH18btrG9PjruxsRELFy7E5cuXYTabMW3aNFitVq/+jr/88sv417/+hQEDBsBqtSIk\nJASPPfaY148z3dXVOLzxuysGg8GABQsWoLa2FmazGU899RQaGxsD7v3oahyB8n6058u/w2IS4286\nJ/ojIiKioMOJ/oiIiCjoMMEhIiKioMMEh4iIiIIOExwiIiIKOkxwiIiIKOgwwSEiIqKgwwSHbBYs\nWIC9e/d2uU2ompoafP755wCAnJwcnDt3zmHZd955B/fffz/KysoE9ZWXl4fhw4c77YOIfMvd48uX\nX36JDz/8sNP2xx9/HCqVClevXsWOHTtcavvjjz/Gvffeiy1btrgdNwCsXbsWv/rVr0Q7PpL4wrsu\nQiSOr7/+GufOneu0QKYjv//975GdnS2orxUrVuDChQuC6hKRfxo9erTd7SEhIQBaFyLetWsXxo0b\n51J748ePx/Tp0wXF8uyzz0KtVguqS97BBCcAnT9/HgsWLEB4eDisVitef/11JCYm4o033sCRI0fQ\n0tKCJ598Evfffz9ycnIwYMAAnD17FkDrfx09e/bE4sWLcenSJVy+fBn33XefbYVjR8xmM/Lz83Hh\nwgVYLBY8++yzGD58OB566CGMGDEC3333HUJCQrB+/XpER0fbZqCMj4+HUqnE+vXrUVhYCKPRiDvu\nuANA61ma2tpaNDU14fXXX0dycrLdvisqKrBq1SpYrVYkJiZizZo1+MMf/oBbbrkF33//PWQyGdLT\n07F3717odDps2rQJMTExdhdrIyL3eeuYo9VqMX36dPzzn//EsWPH8NRTT+HgwYNQq9VYuHAhxo0b\nh7Nnz2LevHl48803sXfvXtx44424cuUKAKCgoADfffed7SxPcXEx3n33XTQ0NGDJkiUYOnSo3fH9\n+OOPyMvLg8lkglQqxeuvv441a9YgPDwcKpUKzc3NePDBB/H555+jpqYG69evR9++fT30apNY+BVV\nANq3bx9uv/12bNmyBbNnz4ZOp8OePXtQXV2N//3f/8XWrVuxYcMG24qxw4YNQ1FRER544AFs2LAB\nly5dwi9+8Qts3LgRH374Id5///0u+/zwww8RFxeHoqIirFu3DkuXLgXQuj7I+PHjUVRUhF69emHP\nnj347LPPcPXqVXzwwQd4+eWXoVarERYWhqeeegrjxo2zncG599578be//c22kKMj+fn5WLVqFf7+\n97/jnnvuwQ8//AAAttegubkZUqkUmzZtQmpqKg4ePNjdl5iI2vHWMUehUKBnz55Qq9X48ssvkZSU\nhG+//RafffYZxo4dC6D1bE1lZSWOHDmCjz76CKtXr4ZerwcAPP300xg5ciQeffRRAEBaWhr+9re/\nYerUqfj4448djm/16tV4+umnUVxcjGnTpuHkyZMAgOTkZPz1r3/FgAEDUF1djcLCQowdO9b2VTv5\nN57BCUCPPvooCgsLMWPGDMTGxuLZZ5/F6dOnUVlZaVsnqKWlBdXV1QCAX/7ylwCAO++8E7t27UJs\nbCyOHz+OAwcOQC6Xw2Qyddnn6dOnceTIEVRUVNjab/uv6dZbbwUA9O7dG83NzVAqlbbF3OLi4tC/\nf3+7bQ4ZMgQAkJCQgNraWod919bW2tqYOHFip/qxsbEYOHCg7bHRaOxyPETkOm8eczIyMvDFF1/g\n6NGjeOqpp7Bv3z4cO3YMK1euxO7duwG0nlFKS0sD0Lpa9qBBg+y21bYYY0JCAgwGg8M+z507h9tv\nvx0AbP+A7dixo8MxJjU11faYx5jAwDM4Aai8vBzp6enYsmUL7r//fmzcuBGpqan45S9/ia1bt2Lr\n1q3IysqynUKtqqoCABw5cgSDBg3Cxx9/jB49emDNmjV48skn0dTU1GWfqampGDduHLZu3YqNGzci\nKysLCoXCbtnBgwfj2LFjAICrV6/i/PnzAFr/82pbhbrtuSt69eplu57m3XffRXl5uVv1iah7vHnM\nycjIwI4dOxAdHY3Ro0ejvLwczc3NiIuLs5UZOHAgjh8/DqB1MdgzZ84AAEJDQwUdYwYOHIhvv/0W\nAFBaWopt27a5VZ/8E8/gBKChQ4ciNzcXGzZsgMViwcKFC3HrrbfiwIEDmDJlCgwGAzIyMiCXywG0\n3i2wefNmyGQyvPrqq7h8+TLmzZuHY8eOISIiAikpKfjpp5+c9vnYY4/hpZdeQk5ODvR6PSZNmoSQ\nkJAOB4C2x/fccw92796NSZMmISEhAVKpFOHh4Rg8eDAKCgowZMgQtw4cS5cuxYIFCxAaGopevXph\n+vTp2Lp1a6d+r31MROLw5jEnMTERzc3NuOuuuxATE4Pw8HCMGTOmQ5lbbrkFo0ePxsSJE3HDDTcg\nISEBANC3b1+cPn26w/HBFS+88AIWL16M9evXQyaTYc2aNbYkDeBxJVBxNfEgl5OTg2XLljn8msgT\nzp49i1OnTuHBBx+EVqvFuHHj8PnnnyMiIsLlNt555x0kJCTgd7/7neA4fDF2outdoPzeffzxx7YL\nloVasGABsrOz8atf/UrEyEgs/IoqyPniP4/evXtjx44dePzxx/HHP/4RL7zwglvJTZstW7Z0ax6c\n7777TlBdIhIukM52lJWVdWsenC+//FLcgEhUPINDREREQYdncIiIiCjoMMEhIiKioMMEh4iIiIIO\nExwiIiIKOkxwiIiIKOgwwSEiIqKgwwSHiIiIgg4THCIiIgo6THCIiIgo6DDBISIioqDjUoJTUVGB\nnJycDttKS0u7tRAiEVGbthWqJ02ahClTpuDMmTO4cOECJk+ejKlTp2Lp0qW+DpGIAkx4VwU2btyI\n7du3Qy6X27adOHECH330kUcDI6Lrx65duxASEoL3338fBw8exBtvvAGr1Yq5c+ciPT0d+fn5KC8v\nR0ZGhq9DJaIA0eUZnH79+mHdunW251euXMHatWuxaNEijwZGRNePjIwMLF++HACgUqnQo0cPnDhx\nAunp6QCAu+++G/v37/dliEQUYLpMcDIzMxEWFgag9TRyXl4e5s+fD6lUCi5ETkRiCQ0Nxfz587Fi\nxQqMGzeuw/FFLpdDp9P5MDoiCjRdfkXVXlVVFS5cuIAlS5bAaDTihx9+wKpVq7BgwYJOZY8cOSJa\nkETkv4YNGyZaW6+88go0Gg0eeeQRGI1G23a9Xo/Y2Fi7dXisIbo+uHuscTnBsVqtGDp0KEpLSwEA\n1dXVmDdvnt3kRmgwnqRSqZCUlCS4/oYNpUhOHg8AUCpLMXPmeJ/GIyZ/igVgPM74UyyAeMnF9u3b\noVar8dRTT0EikSA0NBRpaWk4ePAgRowYgT179mDkyJEO6/vTscYRf3vv7AmEGIHAiDMQYgQCJ04h\nxxqXE5yQkBC3GycicsXYsWOxYMECTJ06FWazGXl5eRgwYADy8vJgMpmQmpqKrKwsX4dJRAHEpQSn\nT58+KC4u7nIbEZEQUqkUa9eu7bS9qKjIB9EQUTDgRH9EREQUdJjgEBERUdBhgkNERERBx63bxImI\niKh7SkrKoVYbBNdPTJRiwgTvzepdU1OD3r17e60/sTDBISIi8iK12mCbdkQIpbLU4b6DBw/i2LFj\neOqppwAABQUFmDRpksN5pFyxePFivPvuu4Lr+woTHCIioiAzefJk/Nd//ReuXr0Kk8mEFStWICIi\nArW1tVi9ejVCQ1uvUFGpVFi7di2kUiluueUWjB8/Hm+88QZCQ0PR3NyMJ598EufPn0dZWRl69uyJ\nkpISREZGYuTIkbjjjjvw9ttv2+ref//9eO211xAbG4uLFy/irbfesq2E4Au8BoeIiCjIpKenY/78\n+QBaJ+pVKpXo3bs3pkyZ0qGcTqeDXq/HyJEjcdddd6G0tBRXr16FTCaDwWBAY2MjUlJSkJ2djb/9\n7W9Ys2YNVq5ciZKSEly9erVD3bCwMEyYMAHp6en46aef8NNPP/li6DY8g0NE171rr4nw9jUORGKL\niYmxPW5pacGsWbNgMplQUFCAZ555Bps2bUJISAhycnIwd+5cnD59GsuWLcOvf/1rjBo1ChMmTMCu\nXbuQmJhoa6f9+nAhISGIj4/vUPexxx7DwYMH8fDDD6N3794+X6+SCQ4RXfeuvSbC2TUORN2VmCjt\n1mcsMVHqdP+1Kw+Eh4fj73//O6RSKRQKBfr27Ys33ngDAPD111+jsLAQKSkpuPPOO/HQQw9h4cKF\nOHnyJPR6Pe6991707dsXmzdvxhNPPIEFCxYgOjoajz76KAwGA15//XVb3bi4OFRXV6OsrAxqtRpa\nrdany0AwwSEiIvIiT54dHDFiBEaMGGF7vmrVKgDAyy+/bLf8TTfdhPXr13fY9tZbb3V4vnjxYtvj\nUaNGddh3bd3hw4e7H7SH8BocIiIiCjpMcIiIiCjoMMEhIiKioMNrcIiIiLyoZEcJ1FfVgusn9kjE\nhHETRIwoODHBISIi8iL1VTWS05MF11ceVooYTfDiV1RERERB4uDBgygsLLQ9LygoQH19vUf6Onny\nJEpLu77dvbq6Gvn5+R6JwRmXzuBUVFTgtddeQ1FREU6ePIkVK1YgLCwMkZGRePXVVxEXF+fpOImI\niMhFri7V8I9//APDhg3DqFGj8Oc//xmrVq2yLdVgMpmwaNEiTJs2DQMGDMC0adOwadMm29IMqamp\nuHTpEk6fPo3NmzfDarVi1KhRGDRoEAoLCxETE4N+/frh/vvvR0hICNRqNV555RXEx8dDIpHghRde\nQEZGBkaOHIk5c+bghhtuEPU16PIMzsaNG5GXlweTyQQAWLlyJRYvXoytW7ciMzOzQ6ZIREREvufq\nUg1jx45FWVkZLl68iKSkpA5LNTQ2NuL7778H0Pq332QydViaoa3twsJCLFq0CK+88gpSU1NRWFiI\npUuXYunSpfjmm2/Q2NgIq9WKoqIi/OlPf0JeXh6am5tx5swZJCQkYMWKFaInN4ALCU6/fv2wbt06\n2/M333wTgwcPBgCYzWZIJBLRgyIiIiLh7C3VcNttt6GgoAAnTpzA3LlzMW/ePMhkMoSEhOC9997D\nxIkTbWdh5s6diwceeACJiYmIjo4GANvSDACwbNky24zJZrPZ1ld1dXWHOEJDQ2GxWDrFFxISAqvV\namvbE7r8iiozM7NDwAkJCQCAb775Bu+99x62bdvmseCIiIiCTWKPxG5dKJzYI9HpfneWalCpVBg/\nfjz+53/+B7m5uUhKSuqwVMOYMWNs7TU1NXVYmqGtrxkzZmD58uUIDQ3Fr371K/zxj3/EsmXLEB8f\nj/T0dERHRyMkJARTpkzB66+/jl69ekEul2PQoEGdYhVTiNWF1bCqq6sxb948FBcXAwB27tyJgoIC\nrF+/Hn369LFb58iRI+jdu7e40XaDTqfrkNG6oqzsS9TWNgMAKivPIDPz/wMA1NTsxBNP/Nrr8XiK\nP8UC+Fc8X5aVob66GlFOzlRGJiRgdHa2V+Lxp9cGAGpqajBs2LBut2M2m7Fw4UJUV1fDZDLh6aef\nRu/evfGnP/0JKSkpAIBJkybhgQce6FT3yJEj3Y5hw4bSTmtRzZw53kkN96lUKp+uy+OKQIgRCIw4\nAyFGIHDiFPJ77vZt4tu3b8cHH3yAoqIixMbGOi3rTy+akDfRbJYhLe1xAMDx4ysQHx8PADAYFN0e\nmz99qPwpFsC/4pGZzchISbG99/aUKpVei9efXhugNcERwyeffIKePXvi1VdfxdWrV/Hb3/4Wzzzz\nDH7/+99j+vTpovRBRNcXtxIci8WClStXIikpCc888wxCQkIwYsQIzJ4921PxEdF14IEHHkBWVhaA\n1uNMeHg4qqqqcPbsWZSXl6Nfv35YtGgRZDKZjyMlokDhUoLTp08f29dTBw4c8GhARHT9kUqlAICG\nhgbMmTMHzz77LJqbm/Hoo49iyJAh+Mtf/oK3334bubm5Po6UiAIFZzImIr9QU1OD2bNnY+rUqcjO\nzu5wvVFmZiZWrFjhsK5KpepW31qtFlKppsPz7rZ5LZ1OJ3qbYguEGIHAiNNZjF+WlaG5tlZw22Je\n9xcIr6VQTHCIyOdqa2sxY8YMLF68GCNHjgQAzJgxAy+99BKGDh2K/fv347bbbnNYv7vXJSkUig7X\nWYlxnd21/O36KXsCIUYgMOJ0FqPMbMbjaWmC23Z23d/Bgwdx7NgxPPXUUwBaZzKeNGmSw2tmr43z\n5MmTOHPmDMaPd3yR/Z49exAZGWn7XXWnrlBCrvdjgkNEPtc2nfz69euxbt06hISEYMGCBVi5ciUi\nIiJwww03YNmyZb4OkyhgCJ3J+PHHH0dNTQ3eeecdnDlzBiNHjkRTUxPOnj0Lk8mEyMhI3HHHHYiM\njMSCBQtw4403Qq/XIyUlBQMHDrQ7s3FycjI+/vhjhIWFISoqymtfNTPBISKfW7RoERYtWtRp+/vv\nv++DaIgCX3p6OubOnYsFCxbYZjK+6667cP/993coN3bsWGzduhU33XQTkpKSOkze+8gjj2DEiBF4\n8cUXsXbtWhw7dgz//Oc/O9SfMGEC+vbtixkzZmDQoEG2mY2XLFmC6OhonDx5EnK5HL/5zW+gUqmw\nefNmr4wfYIJDREQUdOzNZGwymVBQUIBnnnkGmzZtQkhICObMmdNhJmOtVmurFxsba1umCWidlfja\nqfPa7mxsP2Ff+5mNlUolDhw4gMGDB+POO+9EZGSk6GN1hAkOERFREOnOTMYHDx7sUF8ul2PIkCFY\nsWIFrly5gl69ejnt097MxklJSfjmm29w8uRJNDc3w2q1enQGY9u4Pd4DERER2UgTE1GqFL5UgzTR\n8VINI0aMwIgRI2zPV61aBQB4+eWXHdYZOXKk7YLha+sDgMFgQGRkJKKiovDQQw/h1ltvBQA8+OCD\ntjLvvvsuAGD48OEAgNWrV7szJI9ggkNERORFGRMm+DoEt8yZM8fXIQjS5WriRERERIGGCQ4REREF\nHSY4REREFHSY4BAREVHQYYJDREREQYcJDhEREQUdJjhEREQUdJjgEBERUdBxKcGpqKhATk4OAODC\nhQuYPHkypk6diqVLl3o0OCIiIiIhukxwNm7ciLy8PNuCW6tWrcLcuXOxbds2WCwWlJeXezxIIiIi\nInd0meD069cP69atsz2vqqpCeno6AODuu+/G/v37PRcdERERkQBdJjiZmZkICwuzPW+/VLpcLodO\np/NMZEREREQCub3YZmjozzmRXq9HbGysw7IqlUpYVB6g0+ncjker1UIq1QAAGhsN0Gg0tu3dHZuQ\neDzFn2IBxImn7LMy1NbX2t2XEJuA7F9nu9SOVquFAbC9947KeOv187f3iojIX7md4AwZMgSHDh3C\n8OHDsWfPHtsS6/YkJSV1KzgxqVQqt+NRKBSIj48HAMhkUttjg0HR7bEJicdT/CkWQJx4zKFmpGWk\n2d2nPKx0uX2FQgGp9Of33m4Zg8Frr5+/vVc1NTW+DoGIyC63E5zc3Fy89NJLMJlMSE1NRVZWlifi\nIiIiIhLMpQSnT58+KC4uBgCkpKSgqKjIo0ERERERdYfbZ3CIiMRmNpuxcOFCVFdXw2Qy4emnn8bA\ngQMxf/58hIaGYtCgQcjPz/d1mEQUQJjgEJHPffLJJ+jZsydeffVV1NfX4ze/+Q1uueUWzJ07F+np\n6cjPz0d5eTkyMjJ8HSoRBQgu1UBEPvfAAw9gzpw5AICWlhaEhYXhxIkTnHOLiARjgkNEPieVSiGT\nydDQ0IA5c+bgueee45xbRNQt/IqKiPxCTU0NZs+ejalTpyI7Oxtr1qyx7fP0nFvt57xqey72fEOB\nMIdRIMQIBEacgRAjEDhxCsEEh4h8rra2FjNmzMDixYttc2vdeuutXptzq/2cV4A4c11dy9/mMLIn\nEGIEAiPOQIgRCJw4hcy5xQSHiHyuoKAA9fX1WL9+PdatW4eQkBAsWrQIK1as4JxbRCQIExwi8rlF\nixZh0aJFnbZzzi0iEooJDpGHlZeUwKBWOy0jTUxExoQJXoqIiCj4McEh8jCDWo3xyclOy5QqlV6K\nhojo+sDbxImIiCjoMMEhIiKioMMEh4iIiIIOExwiIiIKOkxwiIiIKOgwwSEiIqKgI+g2cbPZjNzc\nXFRXVyM8PBzLly9H//79xY6NiIiISBBBZ3B2794Ni8WC4uJizJo1C2+++abYcREREREJJijBSUlJ\nQUtLC6xWK3Q6HSIiIsSOi4iIiEgwQV9RyeVyKJVKZGVlQavVoqCgQOy4iIiIiAQTlOBs2bIFo0eP\nxnPPPQe1Wo1p06ahtLQUkZGRHcqpVCpRghSDTqdzOx6tVgupVAMAaGw0QKPR2LZ3d2xC4vEUX8VS\n9lkZautrO203Go2QSCRIiE1A9q+zBbWt1WohrZM63OfqeLVaLQyA7b0X0p5Wq4VGaj+WNgf27oVW\nq3W4PzIhAaOzs/3qc0NE5M8EJTg9evRAeHhr1ZiYGJjNZlgslk7lkpKSuhediFQqldvxKBQKxMfH\nAwBkMqntscGg6PbYhMTjKb6KxRxqRlpGWqftmjoN4uPioTysFByXQqFAfFy83X0GhcHldhUKBaTS\nn997u2UMzttr/zlyJMpiQU5a59eiTamy9bXwp88NANTU1Pg6BCIiuwQlOE888QQWLlyIKVOmwGw2\nY968eYiKihI7NiIiIiJBBCU4MpkMa9euFTsWIiIiIlFwoj8iIiIKOkxwiIiIKOgwwSEiIqKgwwSH\niIiIgg4THCLyGxUVFcjJyQEAnDx5EnfffTemTZuGadOm4V//+pePoyOiQCLoLioiIrFt3LgR27dv\nh1wuBwBUVlbi97//PaZPn+7bwIgoIPEMDhH5hX79+mHdunW251VVVfjiiy8wdepULFq0CI2NjT6M\njogCDRMcIvILmZmZCAsLsz2//fbb8eKLL2Lbtm3o27cv3n77bR9GR0SBhl9RXaOkpBxqtQEAcPTo\nSadfd+sAAB37SURBVCQnj+9U5ujRSmzY8PPzxEQpJkzI8FaI142jx49iAzbY3ZfYIxETxk0Qvd3u\ntk3iycjIQExMDIDW5GfFihUOy3Z3fa726861PRd7za9AWEcsEGIEAiPOQIgRCJw4hWCCcw212mBL\navbsqbBbRqdr6ZD4KJWlXonteqNr0iE5PdnuPuVhpUfa7W7bJJ4ZM2bgpZdewtChQ7F//37cdttt\nDst2d32ua9cLE2O9uWv52zpi9gRCjEBgxBkIMQKBE6eQde+Y4BCRX1qyZAmWL1+OiIgI3HDDDVi2\nbJmvQyKiAMIEh4j8Rp8+fVBcXAwAGDJkCN5//30fR0REgYoXGRMREVHQYYJDREREQYcJDhEREQUd\nJjhEREQUdARfZFxYWIhdu3bBZDJh8uTJmDhxophxEREREQkmKME5ePAgjh49iuLiYjQ2NmLTpk1i\nx0VEREQkmKAEZ+/evbj55psxa9Ys6PV6vPjii2LHRURERCSYoATnypUrUKlUKCgowMWLFzFz5kx8\n+umnYsdGREQBrGRHCdRX1Xb3cUkU8jRBCY5CoUBqairCw8PRv39/SCQS1NXVIS4urkM5f1rfwtX1\nNtqvSdPYaIBG4/xxWx13x+pP63/4KhatVgtpnbTTdoPBAE2dBo2NjdDUaezUBPbu3wutVuuw7cpT\nlcgckGl3n7N22+Jqez20Wi0MQIf321l5R/s10s7jbM/Q2OhSH/70uSHqivqq2iPLrRC5QlCCM2zY\nMBQVFWH69OlQq9VoampCz549O5Xzp/UtXF1vo/2aNDKZtMvHgLB1a/xp/Q9fxaJQKBAfF99pu6ZO\ng/i4eMhkMrv7AcASZkFaRprDto+fPu6wrrN2AcCgMNheD4VCAam04/vdaRwGg9PX79p1juyRymQu\n9eFPnxtA2PowRETeICjBGTNmDA4fPoxHHnkEVqsV+fn5CAkJETs2IiIiIkEE3yb+/PPPixkHERER\nkWg40R8REREFHSY4REREFHSY4BAREVHQYYJDREREQYcJDhEREQUdJjhEREQUdATfJk5ERBToyktK\nYFDbX06ijTQxERkTuKxEoGGCQ0RE1y2DWo3xyfaXk2hTquSyEoGICQ5dtyr//RVa6jqvZ6X9UYvS\n+tbHJ48exV133eXlyDqrPHoU2LABWq0WCoXCbplg+C+zoqICr732GoqKinDhwgXMnz8foaGhGDRo\nEPLz830dHhEFECY4dN1qqdNibGJCp+21OuDB//xHV7Fnj7fDsqtFp8P45GRonKyLFej/ZW7cuBHb\nt2+HXC4HAKxatQpz585Feno68vPzUV5ejoyMDB9HSUSBghcZE5Ff6NevH9atW2d7XlVVhfT0dADA\n3Xffjf379/sqNCIKQExwiMgvZGZmIiwszPbcarXaHsvlcuh0Ol+ERUQBil9REZFfCg39+f8vvV6P\n2NhYh2VVKlW3+tJqtZBKNR2ed7fNa+l0OtHbFJvYMWq1WkjrpA73udtX2WdlqK2vhdFohEQi6bAv\nITYB2b/OFhSjRmo/xu7EGgjvNxA4cQrBBIeI/NKQIUNw6NAhDB8+HHv27MHIkSMdlk1KSupWXwqF\nosO1TQaDotttXkulUoneptjEjlGhUCA+zv41YwaFwe2+zKFmpGWkQVOn6dSu8rBSUOzXvvd2yxjc\njzUQ3m8gcOKsqalxuw4THCLyS7m5uXjppZdgMpmQmpqKrKwsX4dERAGECQ4R+Y0+ffqguLgYAJCS\nkoKioiIfR0REgapbFxlrNBqMGTMG586dEyseIiIiom4TnOCYzWbk5+cjKipKzHiIiIiIuk1wgrN6\n9WpMmjQJvXr1EjMeIiIiom4TlOCUlJQgPj4eo0aN6jBXBREREZE/EHSRcUlJCUJCQrBv3z6cOnUK\nubm52LBhQ6db7fzp3npX7/VvPx9GY+P/3979R0VV5n8Af8/wG0ccRCULE7T16xqZq7a2FquQuP7K\nXfFXa7B55OBqedJSS8RWMRfT1WNbRw1qT53s7NdTiWezNnXVVsXKHzWBaMaK4lccBBkcGWaGmUGe\n7x8uIzi/YBi4l+H9+mtmnufe5zMPc5774T73PtcMnc79awAoKDgJvf7OM4369AnG1KkJPounM0gV\ni6s1MsxmM3Q1OphMJuhqdE62hNsyT+VNZUajEbV1IQ7l5y+ch9FovPO65DwOhANBQUEAgIjwCIwe\nPrpF/ZMFBfa/vzMXi4sxNjnZZTkAmE2mFr8pV+Vms9llvY5Yu4WIqKvyKsH56KOP7K/T0tKwfv16\np+sIyOne+tbe6998TYTw8DCPrwGgsTEU8fFpAIDy8n2takdOaw9IFYurNTKa1rgIDw93uYaGuzJP\n5U1lPXr0QISqp0O5UAoMfHQgAKDX1TL0Gxptr1d9sdrhtx7a2Ii0+HiXsWwoKvK4zkZYeLjbOk3l\nOp3OZT1v1upoL2/WpiAi6gztflSDQqHwRRxEREREPtPudXA+/PBDX8RBRERErXQoPx/mykq3dcKi\nozEhJaWTIpIfLvRHRETUxZgrK/F0TIzbOvvKyzspGnni08SJiIjI7zDBISIiIr/DBIeIiIj8Dq/B\nISIiGAwG7PlyD8JUjmtTBSoC8fRTT3tc7oBITpjgEBHdQ6Mpxs6dd99HR4chJWWCy/r5+YdQWWlu\ndX05qq+vhzHAiAGPDHAoq7hYAZPJ5NMER1OkwU7sdPg8ulc0UqZ13zt/yHeY4BAR3cNguI2YmKft\n78vL97mtX1lpblN9uVIoFAgICHD8XOn79c4M9QbEjHa8C6j8TPe+84d8h9fgEBERkd/hGRxql/zP\n81F5y/ViUzzd3LV4Wjysuy8cRkRdBxMcapfKW5VOTzM34enmrsXT4mFdceGwqqoq2Gw2+/vQ0FBe\nLEvUDTDBISK/ZTKZkJ9/BsB99s8CA7VYsGAKlErO0JPv3Hv2U6/XQ61Wt6hTcvkyhsTFedyXr86U\nFms0aHG1vBOmwEDMzchod1tyxASHiPxcCB54YKT93bVrfAI6+d69Zz91YWEOZwo3HDuGpxMSPO7L\nV2dKbxsMHh/nsKu42CdtyRETHCKStZSUFKhUKgBATEwMcnJyJI6IiLoCJjhEJFtWqxUA8OGHH0oc\nCRF1NZyEJiLZunDhAkwmE9LT0zF//nwUFhZKHRIRdRE8g0NEshUaGor09HTMnj0bZWVlyMjIwIED\nB3iBMBF55FWC09DQgNWrV+PatWuw2WxYtGgRkpKSfB0bEXVzsbGxGDhwoP21Wq3GjRs3EB0d3aKe\nVqt1ur3ZbMatWzcRGqqzf6bX66HValskSXq9HmFhd+uYTGbodI7buHLv9s7qGwwGt/uQmk6nQ319\nPXQ1Oocy/S09rl+/7nSVYwD44vAXqK6tdvi8+EIxkgclO93GZDI5b8tNX+v1eoTVhMFsNjts6+lv\n5Iper4cuzPH5W82dLCiAXq93W+dicTHGJt/9rmZzy98QAJhNJofPXMXk6bu0Ju7WtFdvscj6d9ke\nXiU4n332GSIjI7F582bcunULv/vd75jgEJHP7dmzByUlJVi7di0qKythNBrRt29fh3r333+/0+1N\nJhN69SprcTdLfb0a999/f4sER61Wt6gTHt7yDhizWe2yDWfbO6uv1Wrd7kNqQUFBd9YI6u24RlC9\nrh733Xefy/gblA2InxDv8HlRSZHT/QFAeHi40zKz2uyyHbVajajeUdDV6By2dbedO/f+7ZwJbWxE\nWrzj92tuQ1FRi/3odDqH/YaFh7dqDSa12fN3aU3crWkvtKJC1r/LJhUVbb/70asEZ/LkyZg0aRIA\noLGxEYGBnOkiIt+bNWsWMjMzMW/ePCiVSuTk5HB6iohaxavMJOy/p8Xq6uqwdOlSvPTSSz4NiogI\nuHNWYcuWLVKHQURdkNenXioqKrBkyRKkpqZiypQpTuvIaV7P3fz3F18cR3X1ndtRi4svIjl5LICW\n8/CuXt/7vvncafP99ukTjKlT7y7wJKf5eE+xuJpfB9zPsQNAwTeu565dbds0v+5qjh5wPX/fmvKm\nMqPRiNq6EIdyi8WC2jrDnddWC6wWC2r/W2Y0Gts8r96aefDW7sPZvH4TT9cJBPfpg4SpU93G4Wle\n39vrHIiIOptXCU51dTXS09Pxpz/9CY8//rjLenKa13M3/93QEI74+LkAgKKiDfY5y+bz8K5e3/u+\n+dx78/2Wl+9r0b6c5uM9xeJqfh1wP8cOAI0BjW3etml+3dUcPeB6/r415U1lPXr0QISqp0N5SEiI\n/fOQ4BAEN3tv7WFp87x6a+bBW7sPZ/P6TTxdJ7CvvLzd8/r3Xhvgzbw4EVFn8GoyOzc3F7W1tdix\nYwfS0tLwhz/8wb4gFxEREZHUvDqDk5WVhaysLF/HQkREROQTvB2BiIiI/A4THCIiIvI7XMCGiIhk\nQ3ejGj+eO+e0rOKqFqLXbRhMdegd2RsKhaKTo6OuhAkOERF59OXhL3E76LbTMs1ZDWJGx7S7DSEE\n9IXnERjWz2n5gJKLiLTUoKr2Fiw/G4zQ8NC7MRRpsBM7HbaJ7hWNlGkp7Y6tMxVrNMBOx+/S3I8a\nDZ6OaX+f+zMmOERE5FF1XTUeSX7EadmxU8d82tbPXCxn8J9INfr0640LxjqHMkO9wWmSVX6m3Kex\ndYbbBoPH5KXwmG/73B/xGhwiIiLyO0xwiIiIyO90iykqg8GA48c1iIz8PwDAgw/2xdChgyWOSh4a\nGxtxWnMaYf9xvjx/oDIQFoulk6NqP7PBhIsFGqdlxivXcbFAA32lDoju08mRERFRZ+gWCU5NTQ1K\nSoIQGzsI9fVGHD78GWJjB9nLNZofERPztIQRSsdiseD81fOIGe58vlev1cNkMnVyVO1322BE/yta\nqFXhDmV6kxk/q9Lh4vVqYPj/SBCdNHjhIhF1J90iwQGA4OBQqNX9YDTegl5/u0VCc+xYoYSRSU+h\nVEAdpXZaZtQZOzka31GFhaB3zx6On4cEQ93DMfHxd7xwkYi6E16DQ0RERH6HCQ4RERH5HSY4RERE\n5HeY4BAREZHf6TYXGRMRSemLL46joeHuxe2XL5cgLm6I/X10dBhSUiZIEVqX9OPhk1DW3b0JwlT0\nHxT+7z9b1Ll2tRI1V6tQesj5xfO6Gzo0Gk0IGOm4QrO6hxpjHxvr26CpUzHBISLqBNXVVsTHz7W/\nP3ZsAxIS7t7NWV6+T4qwuqzGmluYNOA++/sQVTjG3bOu1funz2F0iBLjfvmQ030c/ec1XApXos9D\njuthVV+s9m3A1Om8SnCEEFi3bh1++uknBAcH489//jMGDBjg69iIqJvjWENE3vLqGpxDhw7BarVi\n9+7dWL58OTZu3OjruIiIONYQkde8SnC+++47JCQkAAAeffRRFBcX+zQoIiKAYw0Rec+rKaq6ujr0\n7Nnz7k4CA9HY2AilUp43ZSkUClitVdBqT6GhwYaAAKkjkg+FQgHlbSW0F7ROy20mGxQKRSdH1X5C\nqUBZnQkVVptDWYnRjIgaPfhDkL/2jjUKhQJKpQVa7Sn7Z0FBPg/TLygUCtw233Y6FnTmONAYFIhT\nWufjUfHNW4isUqAeCgR2vWGJOplCCCHautEbb7yBESNGYNKkSQCA8ePH49///neLOt99951PAiQi\neRs1alSH7ZtjDRE1aetY49UZnJEjR+Krr77CpEmT8MMPP2DIkCEOdTpy0COi7oFjDRF5y6szOM3v\nbACAjRs3Ii4uzufBEVH3xrGGiLzlVYJDREREJGfyvCqYiIiIqB18vpKxHBfmKiwsxJYtW7Br1y5J\n42hoaMDq1atx7do12Gw2LFq0CElJSZLF09jYiDVr1uDy5ctQKpXIzs7GQw85X/Gzs+h0OsycORPv\nv/++5FMRKSkpUKlUAICYmBjk5ORIGk9eXh6OHDkCm82GefPmYebMmZLFsnfvXuTn50OhUMBiseDC\nhQs4ceKEvb98zdO4cuTIEezYsQOBgYGYOXMmZs+e3SFxtDfODz74AJ9++il69+4NAFi/fj1iY2Ml\nidXVuCiXvmziKk659KWncV0O/ekpRrn0padjUpv7UvjYwYMHxapVq4QQQvzwww9i8eLFvm6iTd59\n910xbdo0MXfuXEnjEEKIPXv2iJycHCGEEHq9XowfP17SeP71r3+J1atXCyGEOHnypOR/K5vNJl54\n4QXxm9/8Rly6dEnSWCwWi5gxY4akMTR38uRJsWjRIiGEEEajUbz99tsSR3RXdna2+Pjjjzu0DXfj\nis1mE8nJycJgMAir1SpmzpwpdDpdh8bjTZxCCLFixQpx7tw5KUJrwdW4KKe+FML9+C2XvnQ3rsul\nPz0de+TSl+6OSd70pc+nqOS2MNfAgQOxfft2SWNoMnnyZCxduhTAnUw1MFDaR4FNmDABr7/+OgDg\n2rVr6NWrl6TxbNq0Cb///e/Rr18/SeMAgAsXLsBkMiE9PR3z589HYWGhpPEUFBRgyJAheP7557F4\n8WIkJiZKGk+Ts2fP4uLFix3+X6m7caW0tBQDBw6ESqVCUFAQRo0ahdOnT3doPN7ECQDnzp1Dbm4u\n5s2bh7y8PClCBOB6XJRTXwLux2+59KW7cV0u/enp2COXvnR3TPKmL31+hJXbIoDJycm4du2aJG3f\nKywsDMCdPlq6dCleeukliSMClEolVq1ahUOHDuGtt96SLI78/HxERUXhiSeewDvvvCNZHE1CQ0OR\nnp6O2bNno6ysDBkZGThw4IBkv+ObN29Cq9UiNzcXV69exeLFi7F//35JYmkuLy8PS5Ys6fB23I0r\n95b16NEDBoOhw2NyxtP4N3XqVDz77LNQqVR44YUXcPToUYwbN67T43Q1LsqpLwH347dc+tLduC6X\n/vR07JFLXwKuj0ne9KXPR2uVSgWj8e4j7OW8wrEUKioq8Nxzz2HGjBmYMmWK1OEAuLOY2oEDB7Bm\nzRrU19dLEkN+fj5OnDiBtLQ0XLhwAa+++ip0Op0ksQBAbGwspk+fbn+tVqtx48YNyeJRq9VISEhA\nYGAg4uLiEBISgpqaGsniAQCDwYCysjL88pe/7PC23I0rKpUKdXV19jKj0YiIiIgOj8kZT+Pfc889\nB7VajcDAQIwbNw7nz5+XIkyX5NSXnsipL12N63LqT3fHHjn1JeD8mORNX/o88xg5ciSOHj0KAC4X\n5pKCkMHd8NXV1UhPT8fKlSsxY8YMqcPBP/7xD/vpyJCQECiVSsmS0Y8++gi7du3Crl27MHToUGza\ntAlRUVGSxAIAe/bswRtvvAEAqKyshNFoRN++fSWLZ9SoUTh+/Lg9nvr6ekRGRkoWDwCcPn0ajz/+\neKe05W5cGTx4MK5cuYLa2lpYrVacPn0aI0aM6JS42hJnXV0dpk2bBrPZDCEEvv32Wzz88MOSxNnk\n3nFRTn3Z3L1xyqkv3Y3rculPdzHKqS/dHZO86UufT1ElJyfjxIkTeOaZZwBANk//lcPzlHJzc1Fb\nW4sdO3Zg+/btUCgUeO+99xAcHCxJPBMnTkRmZiZSU1PR0NCArKwsyWJpTg5/q1mzZiEzMxPz5s2D\nUqlETk6OpGcix48fjzNnzmDWrFkQQmDt2rWS99Ply5c77Q5JZ+PK559/DrPZjNmzZyMzMxMLFiyA\nEAKzZ8+W7DouT3G+/PLLSEtLQ0hICH71q1/h17/+tSRxNmn6DcmxL5tzFqdc+tLZuD5nzhxZ9aen\nGOXSl/cek1avXo2DBw963Zdc6I+IiIj8Di+OISIiIr/DBIeIiIj8DhMcIiIi8jtMcIiIiMjvMMEh\nIiIiv8MEh4iIiPwOE5xuwGq14pNPPnFbJykpCVar1eNn3jpz5gxKSkoAAE8++aTbumlpaZgzZw5K\nS0vb3I7JZEJaWprHNoio43g75riTl5eHs2fPOrTT9FTskpISnDlzplX7zszMxG9/+1ucPHmy1e03\nt2jRIgwfPtxn4yN1DCY43UBVVRU+/fRTt3WcLRrny4Xk9uzZg6qqqlbX37x5MwYPHtzmdsLDw7Fr\n1642b0dEvuPtmOPOwoUL8cgjj7T4TAhh38/Bgwft/xS1Zt8rV67EmDFj2hRDk3feeUfSlc2pdaR9\nnDV5be/evTh06BCMRiP0ej2ef/55TJw4EadOncKbb76JgIAAPPjgg8jOzkZubi5KS0uxY8cOzJw5\nE2vXroXNZkNVVRWWLVuGp556yu2jLK5fv47XXnsNFosFoaGheP3119HQ0IDly5ejf//+uHLlCoYP\nH45169bh5s2bWLFiBaxWK+Li4vDtt99i27ZtOH78OM6fP4/BgwfDarVixYoV0Gq1iIyMxFtvvYWA\ngIAWbTbF88knn2D37t0QQiApKQlLlixBcnIyRo0ahbKyMowZMwZ1dXUoKipCXFwcNm/e3KH9TtRd\ndfSYc/jwYXz99dd47bXXkJeXB41Gg507d2Lfvn3QarUoKyvD1KlTMXLkSKxYsQIGg8G+knZVVRXy\n8/MRHByMn//85xBCYN26dbh69SoUCgW2b9/e4kGNzX311Vf2J5YPGzYM2dnZmD59Oh577DH89NNP\nGDRoEKKionDmzBmEhIQgLy8PAQEBsnj8D3kgqEvKz88XCxYsEEIIUV1dLRITE4XNZhMTJ04UOp1O\nCCHEm2++KT7++GNRXl4u5s6dK4QQ4uuvvxanTp0SQgjx/fff2/eRmJgoLBZLizaSkpKExWIRy5Yt\nE8eOHbNvv3z5clFeXi7GjBkjTCaTuH37tkhMTBTV1dUiJydH/P3vfxdCCHHixAmRlJQkhBBi1apV\noqCgQAghxMMPPyy0Wq0QQojU1FRRVFTUot3U1FRx6dIlodPpxMSJE+1xbd26VRiNRjFs2DBx/fp1\nYbPZxC9+8QtRWlpqj9dgMAghhHjiiSd80s9EdEdHjzn19fVi+vTpQgghMjIyREpKimhoaBDLli0T\nly5dEqtWrRLHjx8Xf/vb38S2bduEEEIUFhbax5i3335b7N69277v77//XghxZ+z58ssvW3yXpn01\nNDSIxMREUVNTI4QQ4r333hNarVYkJiYKjUYjhBBi0qRJ9vEvNTVV/Pjjj07jJ/nhGZwu7LHHHgMA\nREVFoVevXqiqqsKNGzewbNkyAIDFYsHYsWNbbNO3b1/s3LnTfvrYZrN5bKekpAS5ubl49913IYRA\nUFAQAGDgwIEICwsDAPTr1w8WiwWlpaX2h7mNHj26xX7Ef//jUavV6N+/vz0eV08wv3r1KoYMGWJ/\nPtbLL78MAIiMjER0dDSAO1NSgwYNAgBERETAYrFApVJ5/E5E1HYdOeaEhIQgNjYWZ8+eRWBgIEaM\nGIHTp0+joqICcXFx9nplZWUYP348AGD48OEIDHR+GGt6YGSfPn1cjjE3b96EWq22P7g2PT0dwJ0p\nrmHDhgG4M640TZdHRETwupsuhAlOF3bu3DkAd54UW1dXh/79+6N///7YsWMHVCoVjhw5gh49ekCp\nVKKxsREA8Ne//hVz5sxBQkIC8vPzsXfvXpf7b0pIBg8ejAULFmDEiBG4dOmS/UI+Z3WHDBkCjUaD\noUOHQqPR2MsVCoU9htYaMGAALl26BJvNhqCgILz44ovIysryGC8RdYyOHnMmTJiAzZs3Izk5GQMG\nDMC2bdscbhh46KGHoNFokJSUhPPnz6OhoQGAd2NMVFQUamtrUVtbi4iICGzYsAHTp0/nWOInmOB0\nYTdu3MD8+fNRV1eHdevWQaFQYPXq1Vi4cCEaGxvRs2dPbNq0CSqVCjabDVu3bsXkyZOxadMm5OXl\noV+/ftDr9QDcX2S8cuVKrFu3DlarFRaLxZ5kNN+m6XVGRgZeeeUV7N+/H3379rX/d/Xoo49i69at\neOCBB5y24Uzv3r2RkZGB1NRUKBQKJCUl2c/cOCP107WJ/F1HjzmJiYnIyspCdnY2oqOj8eKLLyI7\nO7tFnWeeeQavvPIKnn32WcTFxdnP8MbHx+Mvf/kLBg0a5HRsckahUGDt2rVYuHAhAgICMGzYMAwf\nPtzl9q3dL8kDnybeRe3duxeXL1+2T9vIxdGjRxEVFYX4+Hh88803yM3NxQcffNCmfaSlpWH9+vUt\nTku31ZNPPomCggKvtyeiluQ65ngjMzMTU6ZMQUJCgtf7SEpKwv79++0JFskPz+CQT8XExCArKwsB\nAQFobGzEmjVrvNrPq6++io0bN7b5VnGTyYQ//vGP/O+KiNzasmULgoODvbpVfNGiRaipqemAqMiX\neAaHiIiI/A4X+iMiIiK/wwSHiIiI/A4THCIiIvI7THCIiIjI7zDBISIiIr/z/1cXl5ndNP+WAAAA\nAElFTkSuQmCC\n",
"text/plain": [
""
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"from matplotlib import pyplot as plt\n",
"import numpy as np\n",
"import math\n",
"\n",
"label_dict = {1: 'Iris-Setosa',\n",
" 2: 'Iris-Versicolor',\n",
" 3: 'Iris-Virgnica'}\n",
"\n",
"feature_dict = {0: 'sepal length [cm]',\n",
" 1: 'sepal width [cm]',\n",
" 2: 'petal length [cm]',\n",
" 3: 'petal width [cm]'}\n",
"\n",
"with plt.style.context('seaborn-whitegrid'):\n",
" plt.figure(figsize=(8, 6))\n",
" for cnt in range(4):\n",
" plt.subplot(2, 2, cnt+1)\n",
" for lab in ('Iris-setosa', 'Iris-versicolor', 'Iris-virginica'):\n",
" plt.hist(X[y==lab, cnt],\n",
" label=lab,\n",
" bins=10,\n",
" alpha=0.3,)\n",
" plt.xlabel(feature_dict[cnt])\n",
" plt.legend(loc='upper right', fancybox=True, fontsize=8)\n",
"\n",
" plt.tight_layout()\n",
" plt.show()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"
\n",
"
"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Standardizing"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"[[back to top](#Sections)]"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Whether to standardize the data prior to a PCA on the covariance matrix depends on the measurement scales of the original features. Since PCA yields a feature subspace that maximizes the variance along the axes, it makes sense to standardize the data, especially, if it was measured on different scales. Although, all features in the Iris dataset were measured in centimeters, let us continue with the transformation of the data onto unit scale (mean=0 and variance=1), which is a requirement for the optimal performance of many machine learning algorithms."
]
},
{
"cell_type": "code",
"execution_count": 8,
"metadata": {
"collapsed": false
},
"outputs": [],
"source": [
"from sklearn.preprocessing import StandardScaler\n",
"X_std = StandardScaler().fit_transform(X)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"
\n",
"
"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## 1 - Eigendecomposition - Computing Eigenvectors and Eigenvalues"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"[[back to top](#Sections)]"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"The eigenvectors and eigenvalues of a covariance (or correlation) matrix represent the \"core\" of a PCA: The eigenvectors (principal components) determine the directions of the new feature space, and the eigenvalues determine their magnitude. In other words, the eigenvalues explain the variance of the data along the new feature axes."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"
\n",
"
"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Covariance Matrix"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"[[back to top](#Sections)]"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"The classic approach to PCA is to perform the eigendecomposition on the covariance matrix $\\Sigma$, which is a $d \\times d$ matrix where each element represents the covariance between two features. The covariance between two features is calculated as follows:\n",
"\n",
"$\\sigma_{jk} = \\frac{1}{n-1}\\sum_{i=1}^{N}\\left( x_{ij}-\\bar{x}_j \\right) \\left( x_{ik}-\\bar{x}_k \\right).$\n",
"\n",
"We can summarize the calculation of the covariance matrix via the following matrix equation: \n",
"$\\Sigma = \\frac{1}{n-1} \\left( (\\mathbf{X} - \\mathbf{\\bar{x}})^T\\;(\\mathbf{X} - \\mathbf{\\bar{x}}) \\right)$ \n",
"where $\\mathbf{\\bar{x}}$ is the mean vector \n",
"$\\mathbf{\\bar{x}} = \\sum\\limits_{k=1}^n x_{i}.$ \n",
"The mean vector is a $d$-dimensional vector where each value in this vector represents the sample mean of a feature column in the dataset."
]
},
{
"cell_type": "code",
"execution_count": 10,
"metadata": {
"collapsed": false
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Covariance matrix \n",
"[[ 1.00671141 -0.11010327 0.87760486 0.82344326]\n",
" [-0.11010327 1.00671141 -0.42333835 -0.358937 ]\n",
" [ 0.87760486 -0.42333835 1.00671141 0.96921855]\n",
" [ 0.82344326 -0.358937 0.96921855 1.00671141]]\n"
]
}
],
"source": [
"import numpy as np\n",
"mean_vec = np.mean(X_std, axis=0)\n",
"cov_mat = (X_std - mean_vec).T.dot((X_std - mean_vec)) / (X_std.shape[0]-1)\n",
"print('Covariance matrix \\n%s' %cov_mat)"
]
},
{
"cell_type": "code",
"execution_count": 11,
"metadata": {
"collapsed": false
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Covariance matrix \n",
"[[ 1.00671141 -0.11010327 0.87760486 0.82344326]\n",
" [-0.11010327 1.00671141 -0.42333835 -0.358937 ]\n",
" [ 0.87760486 -0.42333835 1.00671141 0.96921855]\n",
" [ 0.82344326 -0.358937 0.96921855 1.00671141]]\n"
]
}
],
"source": [
"import numpy as np\n",
"mean_vec = np.mean(X_std, axis=0)\n",
"cov_mat = (X_std - mean_vec).T.dot((X_std - mean_vec)) / (X_std.shape[0]-1)\n",
"print('Covariance matrix \\n%s' %cov_mat)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"The more verbose way above was simply used for demonstration purposes, equivalently, we could have used the numpy `cov` function:"
]
},
{
"cell_type": "code",
"execution_count": 12,
"metadata": {
"collapsed": false
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"NumPy covariance matrix: \n",
"[[ 1.00671141 -0.11010327 0.87760486 0.82344326]\n",
" [-0.11010327 1.00671141 -0.42333835 -0.358937 ]\n",
" [ 0.87760486 -0.42333835 1.00671141 0.96921855]\n",
" [ 0.82344326 -0.358937 0.96921855 1.00671141]]\n"
]
}
],
"source": [
"print('NumPy covariance matrix: \\n%s' %np.cov(X_std.T))"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"
\n",
"
"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Next, we perform an eigendecomposition on the covariance matrix:"
]
},
{
"cell_type": "code",
"execution_count": 13,
"metadata": {
"collapsed": false
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Eigenvectors \n",
"[[ 0.52237162 -0.37231836 -0.72101681 0.26199559]\n",
" [-0.26335492 -0.92555649 0.24203288 -0.12413481]\n",
" [ 0.58125401 -0.02109478 0.14089226 -0.80115427]\n",
" [ 0.56561105 -0.06541577 0.6338014 0.52354627]]\n",
"\n",
"Eigenvalues \n",
"[ 2.93035378 0.92740362 0.14834223 0.02074601]\n"
]
}
],
"source": [
"cov_mat = np.cov(X_std.T)\n",
"\n",
"eig_vals, eig_vecs = np.linalg.eig(cov_mat)\n",
"\n",
"print('Eigenvectors \\n%s' %eig_vecs)\n",
"print('\\nEigenvalues \\n%s' %eig_vals)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"
\n",
"
"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Correlation Matrix"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"[[back to top](#Sections)]"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Especially, in the field of \"Finance,\" the correlation matrix typically used instead of the covariance matrix. However, the eigendecomposition of the covariance matrix (if the input data was standardized) yields the same results as a eigendecomposition on the correlation matrix, since the correlation matrix can be understood as the normalized covariance matrix."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"
\n",
"
"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Eigendecomposition of the standardized data based on the correlation matrix:"
]
},
{
"cell_type": "code",
"execution_count": 14,
"metadata": {
"collapsed": false
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Eigenvectors \n",
"[[ 0.52237162 -0.37231836 -0.72101681 0.26199559]\n",
" [-0.26335492 -0.92555649 0.24203288 -0.12413481]\n",
" [ 0.58125401 -0.02109478 0.14089226 -0.80115427]\n",
" [ 0.56561105 -0.06541577 0.6338014 0.52354627]]\n",
"\n",
"Eigenvalues \n",
"[ 2.91081808 0.92122093 0.14735328 0.02060771]\n"
]
}
],
"source": [
"cor_mat1 = np.corrcoef(X_std.T)\n",
"\n",
"eig_vals, eig_vecs = np.linalg.eig(cor_mat1)\n",
"\n",
"print('Eigenvectors \\n%s' %eig_vecs)\n",
"print('\\nEigenvalues \\n%s' %eig_vals)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"
\n",
"
"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Eigendecomposition of the raw data based on the correlation matrix:"
]
},
{
"cell_type": "code",
"execution_count": 15,
"metadata": {
"collapsed": false
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Eigenvectors \n",
"[[ 0.52237162 -0.37231836 -0.72101681 0.26199559]\n",
" [-0.26335492 -0.92555649 0.24203288 -0.12413481]\n",
" [ 0.58125401 -0.02109478 0.14089226 -0.80115427]\n",
" [ 0.56561105 -0.06541577 0.6338014 0.52354627]]\n",
"\n",
"Eigenvalues \n",
"[ 2.91081808 0.92122093 0.14735328 0.02060771]\n"
]
}
],
"source": [
"cor_mat2 = np.corrcoef(X.T)\n",
"\n",
"eig_vals, eig_vecs = np.linalg.eig(cor_mat2)\n",
"\n",
"print('Eigenvectors \\n%s' %eig_vecs)\n",
"print('\\nEigenvalues \\n%s' %eig_vals)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"
\n",
"
"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"We can clearly see that all three approaches yield the same eigenvectors and eigenvalue pairs:\n",
" \n",
"- Eigendecomposition of the covariance matrix after standardizing the data.\n",
"- Eigendecomposition of the correlation matrix.\n",
"- Eigendecomposition of the correlation matrix after standardizing the data."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"
\n",
"
"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Singular Vector Decomposition"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"[[back to top](#Sections)]"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"While the eigendecomposition of the covariance or correlation matrix may be more intuitiuve, most PCA implementations perform a Singular Vector Decomposition (SVD) to improve the computational efficiency. So, let us perform an SVD to confirm that the result are indeed the same:"
]
},
{
"cell_type": "code",
"execution_count": 23,
"metadata": {
"collapsed": false
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Vectors U:\n",
" [[-0.52237162 -0.37231836 0.72101681 0.26199559]\n",
" [ 0.26335492 -0.92555649 -0.24203288 -0.12413481]\n",
" [-0.58125401 -0.02109478 -0.14089226 -0.80115427]\n",
" [-0.56561105 -0.06541577 -0.6338014 0.52354627]]\n"
]
}
],
"source": [
"u,s,v = np.linalg.svd(X_std.T)\n",
"print('Vectors U:\\n', u)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": []
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"
\n",
"
"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## 2 - Selecting Principal Components"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"[[back to top](#Sections)]"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"
\n",
"
"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Sorting Eigenpairs"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"[[back to top](#Sections)]"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"The typical goal of a PCA is to reduce the dimensionality of the original feature space by projecting it onto a smaller subspace, where the eigenvectors will form the axes. However, the eigenvectors only define the directions of the new axis, since they have all the same unit length 1, which can confirmed by the following two lines of code:"
]
},
{
"cell_type": "code",
"execution_count": 65,
"metadata": {
"collapsed": false
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Everything ok!\n"
]
}
],
"source": [
"for ev in eig_vecs:\n",
" np.testing.assert_array_almost_equal(1.0, np.linalg.norm(ev))\n",
"print('Everything ok!')"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"
\n",
"
"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"In order to decide which eigenvector(s) can dropped without losing too much information\n",
"for the construction of lower-dimensional subspace, we need to inspect the corresponding eigenvalues: The eigenvectors with the lowest eigenvalues bear the least information about the distribution of the data; those are the ones can be dropped. \n",
"In order to do so, the common approach is to rank the eigenvalues from highest to lowest in order choose the top $k$ eigenvectors."
]
},
{
"cell_type": "code",
"execution_count": 66,
"metadata": {
"collapsed": false
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Eigenvalues in descending order:\n",
"2.91081808375\n",
"0.921220930707\n",
"0.147353278305\n",
"0.0206077072356\n"
]
}
],
"source": [
"# Make a list of (eigenvalue, eigenvector) tuples\n",
"eig_pairs = [(np.abs(eig_vals[i]), eig_vecs[:,i]) for i in range(len(eig_vals))]\n",
"\n",
"# Sort the (eigenvalue, eigenvector) tuples from high to low\n",
"eig_pairs.sort(key=lambda x: x[0], reverse=True)\n",
"\n",
"# Visually confirm that the list is correctly sorted by decreasing eigenvalues\n",
"print('Eigenvalues in descending order:')\n",
"for i in eig_pairs:\n",
" print(i[0])"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"
\n",
"
"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Explained Variance"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"[[back to top](#Sections)]"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"After sorting the eigenpairs, the next question is \"how many principal components are we going to choose for our new feature subspace?\" A useful measure is the so-called \"explained variance,\" which can be calculated from the eigenvalues. The explained variance tells us how much information (variance) can be attributed to each of the principal components."
]
},
{
"cell_type": "code",
"execution_count": 67,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"tot = sum(eig_vals)\n",
"var_exp = [(i / tot)*100 for i in sorted(eig_vals, reverse=True)]\n",
"cum_var_exp = np.cumsum(var_exp)"
]
},
{
"cell_type": "code",
"execution_count": 72,
"metadata": {
"collapsed": false
},
"outputs": [
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAakAAAEbCAYAAABgLnslAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3XtYFHXfP/D3AqKA8iCiyMGsNNM8laiIRwQROclBMe2+\ny9QUTVQwtcBDpnlIy7DoKeDW1OpWSyVFUBLPlng2NbOEW2+Sw4IiiKAcduf3Bz/mcYVlV3B3Z+P9\nuq6uS2ZnZ977hfgwM9+Zj0wQBAFEREQSZGLoAEREROqwSBERkWSxSBERkWSxSBERkWSxSBERkWSx\nSBERkWTptEhFR0dj4MCBCAgIEJetWbMGPj4+CAwMxKxZs3D//n3xtbi4OIwcORI+Pj44ceKELqMR\nEZER0GmRCgkJwYYNG1SWDR48GMnJydi9ezc6duyIuLg4AEBGRgb27duHlJQUJCQk4IMPPgBv4SIi\natp0WqT69u0La2trlWUDBw6EiUn1bl9++WXk5eUBAA4dOgRfX1+YmZnB2dkZHTt2xKVLl3QZj4iI\nJM6g16R27NiBYcOGAQDkcjkcHBzE1+zt7SGXyw0VjYiIJMBgRerLL79Es2bN4O/vb6gIREQkcWaG\n2OmuXbtw9OhRbNmyRVxmb2+P3Nxc8eu8vDzY29vX+f5z587pPCMRET09Li4uDXqfzovU45Mfjh07\nhg0bNuDbb7+Fubm5uNzDwwPz5s3Dm2++CblcjqysLPTq1Uvtdhv6gQ0pJycHjo6Oho7xxIwxtzFm\nBphbn4wxM2CcuRtzYKHTIvXOO+/g1KlTKCoqgru7O2bNmoW4uDhUVlZi8uTJAIDevXtj6dKl6Ny5\nM3x8fODn5wczMzO8//77kMlkuoxHREQSp9Mi9cknn9RaNmbMGLXrh4WFISwsTJeRiIjIiPCJE0RE\nJFksUkREJFksUkREJFksUkREJFksUkREJFksUkREJFksUkREJFksUkREJFksUkREJFksUkREJFks\nUkREJFksUkREJFksUkREJFksUkREJFksUkREJFksUkREJFksUkREJFksUkREJFksUkREJFksUkRE\nJFksUkREJFksUkREJFksUkREJFksUkREJFksUkREJFksUkREJFksUkREJFksUkREJFksUkREJFks\nUkREJFk6LVLR0dEYOHAgAgICxGXFxcWYPHkyvL29MWXKFJSUlIivxcXFYeTIkfDx8cGJEyd0GY2I\niIyATotUSEgINmzYoLIsPj4ebm5uSE1NhaurK+Li4gAAGRkZ2LdvH1JSUpCQkIAPPvgAgiDoMh4R\nEUmcTotU3759YW1trbLs4MGDCA4OBgAEBwcjLS0NAHDo0CH4+vrCzMwMzs7O6NixIy5duqTLeERE\nJHF6vyZVWFgIOzs7AEDbtm1RWFgIAJDL5XBwcBDXs7e3h1wu13c8IiKSEINPnJDJZIaOQEREEmWm\n7x22adMGt2/fhp2dHQoKCmBrawug+sgpNzdXXC8vLw/29vZqt5OTk6PzrE9bSUkJc+uJMWYGmFuf\nXnrJHsXFhk7x5F54oQ2OHDGusW4MnRepxyc/eHh4YNeuXZg2bRoSExPh6ekpLp83bx7efPNNyOVy\nZGVloVevXmq36+joqNPcupCTk8PcemKMmQGgdWsliooMfoKjSbCxUcIY52bl5Nwxup/tRw9AnpRO\ni9Q777yDU6dOoaioCO7u7pg1axamTZuGOXPmYOfOnXByckJMTAwAoHPnzvDx8YGfnx/MzMzw/vvv\n81QgNTlFRSZG+ovT+P4oyMnJA2BcmZsinRapTz75pM7lmzZtqnN5WFgYwsLCdJiIiIiMCc8rEBGR\nZLFIERGRZLFIERGRZLFIERGRZOn9PikyPt27t0dRkaFTPCnjnLVlY6ME/3Yk+j8sUqSRMU6LNsYp\n0QCnRRM9jn+yERGRZLFIERGRZLFIERGRZLFIERGRZLFIERGRZLFIERGRZLFIERGRZLFIERGRZLFI\nERGRZGksUnl5eZg5cyYGDBgANzc3zJo1C3l5efrIRkRETZzGIhUVFQUPDw+cOHECx48fx/DhwxEV\nFaWPbERE1MRpLFKFhYUYM2YMzMzMYGZmhpCQEBQWFuojGxERNXEai5SNjQ12794NhUIBhUKB3bt3\nw8bGRh/ZiIioidNYpFauXIl9+/Zh0KBBGDx4MFJTU7Fq1Sp9ZCMioiZOY6sOJycnfPXVV/rIQkRE\npEJtkUpISMDUqVOxfPlyyGSyWq8vWrRIp8GIiIjUFqlOnToBAHr06KG3MERERI9SW6Q8PDwAAC1a\ntICPj4/Ka/v27dNtKiIiImgxcSI+Pl6rZURERE+b2iOpo0eP4tixY5DL5fjwww/F5ffv34epqale\nwhERUdOmtkjZ29ujR48eOHToELp37y4ut7Ky4hMniIhIL9QWqa5du6Jr167w9/dHs2bN9JmJiIgI\ngBb3SWVnZ2PdunXIyMhAeXm5uPzgwYM6DUZERKTVA2YnTJgAU1NTbNmyBUFBQRg9erQ+shERUROn\nsUiVl5fDzc0NQPXTJ2bNmoWjR4/qPBgREZHG033m5uZQKpXo2LEjvv32W9jb26O0tLTRO960aRN2\n7NgBmUyGLl26YNWqVXjw4AEiIyORnZ0NZ2dnxMTEoFWrVo3eFxERGSeNR1LR0dF48OABFi1ahN9+\n+w179uzBRx991KidyuVyfPPNN9i1axeSkpKgUCiQnJyM+Ph4uLm5ITU1Fa6uroiLi2vUfoiIyLjV\nW6QUCgX27dsHKysrtG/fHqtWrcLnn3+Ol19+udE7ViqVePDgAaqqqvDw4UPY29vj4MGDCA4OBgAE\nBwcjLS2t0fshIiLjVe/pPlNTU5w7d+6p79Te3h6TJk2Cu7s7LCwsMGjQIAwcOBB37tyBnZ0dAKBt\n27ZsrkhE1MRpvCbVrVs3TJ8+HaNGjYKlpaW4fOTIkQ3e6b1793Dw4EEcPnwYrVq1wpw5c7Bnz55a\nT1uv6+nrNXJychq8f0MpKSkxytyAo9HlNtaxZm79McbMgPHmbiiNRaqiogKtW7fGqVOnVJY3pkj9\n8ssv6NChg9jhd8SIEbhw4QLatGmD27dvw87ODgUFBbC1tVW7DUdHxwbv31BycnKMMjdgfONtrGPN\n3PpjjJkB48ydm5vb4PdqLFK66MLr6OiIX3/9FeXl5TA3N0d6ejp69uwJS0tL7Nq1C9OmTUNiYiI8\nPT2f+r6JiMh4aCxSutCrVy94e3sjKCgIZmZmeOmllzBu3DiUlpYiIiICO3fuhJOTE2JiYgwRj4iI\nJMIgRQoAwsPDER4errLMxsYGmzZtMkwgIiKSHI33SRERERmKxiJ1+/ZtREdH46233gIAZGRk4Icf\nftB5MCIiIo1F6r333sPgwYORn58PAHj22WexZcsWnQcjIiLSWKTu3r0LX19fmJhUr2pmZib+m4iI\nSJc0VhtLS0vcvXtXvLH24sWLfOgrERHphcbZfe+99x5mzJiBrKwsjB8/Hnfv3sX69ev1kY2IiJo4\njUWqe/fu+Pbbb3Hjxg0IgoDnnnuO7eSJiEgvNJ7u++6771BWVoYXXngBXbp0QVlZGb777jt9ZCMi\noiZOY5H6/vvvYW1tLX79P//zP5yCTkREeqHxdJ9SqYQgCOLECYVCgcrKSp0Ha6wlS2KQlVVk6Bgq\nEhOn4949Q6d4cjY2SvC+byIyBI1FavDgwYiIiMD48eMBANu2bcOQIUN0HqyxsrKK8OyzSw0dQ8Vb\nb5VAShMjb95cik2blmpcLycnD4BxPXWZiP4eNBap+fPnY9u2bdi6dSsAYODAgQgNDdV5MCIiIo1F\nysTEBK+99hpee+01feQhIiISaSxS586dQ2xsLHJyclBVVSVenzp48KA+8hERUROmsUgtXLgQUVFR\n6NGjBx+HREREeqWxSLVq1QrDhg3TRxYiIiIVGouUq6srPvroI4wcORLm5ubi8u7du+s0GBERkcYi\n9euvvwIArly5Ii6TyWRs10FERDqnsUh98803+shBRERUi1YzIY4cOYKEhATExsaK/xGR9CUmJjbq\n9pGpU6fixx9/fIqJdMvDwwMnT57Ual1fX1/cunXrqWdo7Jg/ibi4OCxevFgv+zIUjUdSS5YswcOH\nD3Hq1CmEhoYiNTUVPXv21Ec2InoKah5ppklsbCyysrKwZs0acVlCQoKuYhlcSkoKHB118yQVbce8\nscLCwvSyH0PSeCR14cIFrFmzBtbW1ggPD8e2bdtw8+ZNPUQjIiJ1FAqFoSPohcYi1aJFCwCAhYUF\n5HI5mjVrhoKCAp0HIzJ2eXl5mDVrFtzc3DBgwAB8+OGHAKqPWObPny+ul52dja5du0KpVAIAXn/9\ndcTExGD8+PF45ZVXMGPGDBQVFWHevHlwcXFBaGgocnJy6nxvzft37NhRZ6YVK1bA3d0dLi4uGDNm\nDM6ePQsAOH78OL766iukpKTglVdeQVBQkMq2Kioq0K9fP2RkZIjbKiwsRO/evVFYWAgAOHz4MIKC\ngtCvXz9MmDABf/zxh9qxyczMxOTJk+Hq6gofHx/s27cPAFBZWYmgoCB8++23AKofcD1hwgT87//+\nrzh2s2fPRmRkJPr06YOQkBBcu3atzn1cunQJ48ePR79+/TBkyBAsX74cVVVV4useHh7466+/AABR\nUVFYtmwZwsLC0KdPH7z66qvia/XlBYCioiJMnz4dLi4uGDduHLKystR+7qlTp9ZqdRQYGIi0tLR6\nvz+PfvaVK1eib9++SExMrPWzNGfOHAwePBj9+vXD66+/rvL90vQZr1+/Ln7GwYMHIz4+HgAgCALi\n4+Ph5eWFAQMGIDIyEvf0+KRsjUXK3d0d9+7dw5QpUxASEgIPDw/4+fnpIxuR0VIqlQgLC4OzszOO\nHDmCY8eOwdfXV3z98dNBj3+9b98+fPzxxzh+/LjYFXvs2LE4c+YMnn/+eZXrwk9yaqlXr17Ys2cP\nzpw5g4CAAERERKCiogJDhgzB9OnT4evriwsXLtS6DmVubo6RI0di7969Khn79+8PW1tbXL9+HQsX\nLsTy5ctx+vRpvPrqq5gxY0adHRMePHiAKVOmYPTo0UhPT8enn36KZcuWITMzE82aNcPatWvx+eef\nIzMzE3FxcRAEATNmzBDff+jQIfj6+uLMmTPw9/fHzJkz6zyqMDU1RXR0NE6fPo3t27cjPT0d//73\nv9WOW0pKCmbNmoWzZ8+iQ4cOiImJUZv3gw8+QGZmJgDggw8+gIWFBX755ResWLECO3fuVDv+fn5+\nKmOYkZGB3Nxc8V5Udd+fRz+7u7s7zp49i4CAgFqfY9iwYThw4AB++eUXvPTSS5g3b55Wn7G0tBST\nJk3C0KFDcfz4cfz0008YMGAAAGDLli04dOgQvvvuOxw/fhzW1tb44IMP1H7Gp01jkZo5cyasra3h\n7e2Nw4cPY9++fYiIiNBHNqIn1qMHIJMBTk6OkMnw1P/r0UO7HJcuXUJBQQHmz5+P5s2bw9zcHH36\n9NH6c4SEhMDZ2RktW7bE0KFD8cwzz2DAgAEwMTHBqFGj8PvvvzdofAICAmBtbQ0TExO8+eabqKio\nwI0bN7R6r7+/P5KTk8Wv9+7dK/6i3Lt3L8aPH4+ePXtCJpMhKCgI5ubm4i0sjzp8+DCcnZ0RFBQE\nmUyGrl27wsvLC/v37wcAvPDCC5gxYwZmzpyJTZs2Ye3atSq/iHv06AEvLy+Ymppi0qRJKC8vx8WL\nF2vtp3v37ujVqxdkMhkcHR0xbtw4nDlzRnxdEASV9b28vMQn6wQEBIhjXFfekSNHYv/+/VAqlThw\n4ADmzJmD5s2b44UXXkBwcLDaMfTy8sK1a9eQm5sLAEhKSoKXl5fY7VzT9+eVV17BwIEDAQDNmzev\ntf2QkBBYWFigWbNmmDlzJq5du4b79+9r9RnbtWuHN998E+bm5rC0tESvXr0AANu3b0dERATatWsn\nbjc1NVXl6F2X1E6cOHnyJNzc3PDTTz/V+frIkSN1FoqooWpu58vJydHZRXFt5ObmwtHRscGPEmvT\npo347+bNm6t83aJFC5SVlTVouxs2bMDOnTvFU/alpaW4e/euVu91dXXFw4cPcenSJbRp0wbXrl3D\niBEjAAByuRxpaWniaTpBEFBVVYX8/Pxa28nJycHFixfRv39/cV2FQoHAwEBxncDAQKxbtw7e3t7o\n0KGDyvvbt28v/lsmk6F9+/Z17ufmzZtYvXo1rly5gocPH0KhUNT7EAI7Ozvx3xYWFuIYq8sbFBSE\nwsJCVFVVqWSq7+fOysoKw4YNQ3JyMt566y0kJyeLp4EBzd+fR/fzOKVSiXXr1iE1NRV3796FTCaD\nTCbD3bt30bJly3o/Y15eXq1xrpGTk4Pw8HDxZ1kQBJiZmeH27dto166d2jxPi9oidebMGbi5ueHw\n4cN1vs4iRaSeg4MDcnNzoVQqaxUqCwsLPHz4UPy6Mdd4LSwsAFSfkrKysgIA3L59u851z549iw0b\nNmDLli3o3LkzAKB///61jijUMTExgY+PD/bu3Qs7OzsMHz4clpaWAIB27dph+vTpWs02c3BwgKur\nKzZs2KB2nQ8++ADDhw/HiRMncP78eZWj0Ly8PPHfgiAgLy8P9vb2tbaxdOlSvPTSS/j0009hYWGB\nzZs3q/2ju6F5lUolzMzMkJubi+eeew4AxOuF6vj7+yM2NhZ9+/ZFRUWFeFpNm+9Pfad29+zZg8OH\nD2Pz5s1wdHRESUkJ+vXrp/VnTElJUfvaypUr8corr2i1radN7Z95s2fPhlKpxJAhQ7Bq1apa/xGR\ner169ULbtm3x8ccf48GDB6ioqMD58+cBAN26dcOZM2eQm5uLkpIS8QJ1Q9ja2sLe3h579uyBUqnE\njh07VC6GP6q0tBRmZmawsbFBRUUFYmNjUVpaKr5uZ2eH7OzseouWv78/UlJSkJSUBH9/f3G5n58f\ntm3bhkuXLgEAysrKcPTo0TqP+Nzd3XHjxg3s3r0bVVVVqKysxOXLl8VrPD/++COuXr2K1atXY+HC\nhXj33Xfx4MED8f1XrlxBWloaFAoFNm3ahObNm6N37951ft6WLVvCwsICmZmZYk+8J6Uu73/+8x+Y\nmJhg5MiRiI2NxcOHD5GRkaHxvrKhQ4ciJycHn332mcp1Sk3fH03Kyspgbm4Oa2trlJWV4ZNPPtH6\neqW7uzsKCgqwZcsWVFRUoLS0VPxevvrqq1i3bp1YfAsLC/XaBaPecxEmJib417/+pa8sRH8bJiYm\n+Oqrr/Df//4X7u7uGDZsmDgjbODAgfD19cXo0aMxduxYDB8+XOW9T3qPzfLly/Gvf/0LAwYMQGZm\nptq/eIcMGYLBgwfD29sbnp6esLCwUDl9NGrUKAiCAFdXV4SEhNSZpVevXrC0tERBQQGGDh0qLn/x\nxRexfPlyLFu2DP3794e3tzcSExPrzGFlZYWNGzciJSUFQ4YMwZAhQ/DJJ5+gsrISubm5WL16Ndas\nWQMLCwv4+/ujZ8+eKn8Ye3p6IiUlBf369UNSUhJiY2NhampaK++7776LpKQk9OnTB++//36tCV/a\njrO6vDUTGhYtWoTS0lIMHjwY0dHRGDNmTL3bMzc3h5eXF06ePKlS6DV9fzQJCgqCg4MDhg4dCn9/\n/yc68rGyssLXX3+NQ4cOiRlOnToFAJg4cSI8PT0xefJkuLi4YPz48WIB0weZoOFY/+OPP0br1q3h\n6+srnloAABsbG52HU+fcuXNwcXGpd50331wqufbxJSUlaCWh/vHat4837PWdhjDGzABza1LXDccN\nxbHWH21+Z6uj8YkTNecpH53b/zSaHpaUlGDhwoW4fv06TExMsHLlSjz77LOIjIxEdnY2nJ2dERMT\nI6lf6kREpF8ai9ShQ4d0suMVK1Zg2LBh+Oyzz1BVVYUHDx7gq6++gpubG6ZOnYr4+HjExcXVmudP\nRERNh8YiBQB//vknMjIyVG4qq7kjvSHu37+Ps2fPYvXq1dUhzMzQqlUrHDx4UJzCGhwcjNdff51F\niohE4eHhho5AeqaxSMXGxuLUqVPIzMzEsGHDcOzYMbi4uDSqSN26dQutW7dGVFQUrl27hh49eiA6\nOhp37twR5/G3bdtWfNwKERE1TRqLVGpqKnbv3o2goCCsWrUKt2/fVnlWVENUVVXh6tWrWLJkCXr2\n7ImVK1ciPj5e46NiHqXpXoSysjKUlJQ0KufTVl5ebugIKsrKyjSOI1B9/VCb9aTEGDMDzK1PxpgZ\nMN7cDaWxSDVv3hwmJiYwMzPD/fv30aZNG/GRHg3Vvn17tG/fXmz5MXLkSCQkJKBNmza4ffs27Ozs\nUFBQAFtbW7Xb0DS7xdLSUpKTLqSUydLSUqtZQsY4m8gYMwPMrU/GmBkwztyNqRkan9nSo0cP3Lt3\nD6GhoQgJCUFwcHCj7zy2s7ODg4OD+Eyq9PR0dO7cGR4eHti1axeA6sZhnp6ejdoPEREZN41HUkuX\nLgUATJgwAUOGDMH9+/fRtWvXRu940aJFmDdvHqqqqtChQwesWrUKCoUCERER2LlzJ5ycnMQn9BIR\nUdOk8Uhq+vTpSEpKQllZGZydnZ9KgQKArl27YufOndi9ezdiY2PRqlUr2NjYYNOmTUhNTcXGjRth\nbW39VPZFZAj+/v4qT91+ElFRUVi/fj2A6me6+fj4aPW++tq919V7qqEezacvp0+fFltaaJKUlIQp\nU6boJEd9/bqetsb8DP1daDySmjx5MlJSUrBu3Tr07NkTvr6+GD58eJ2PiScytCVLYpCVVYSysjLx\n4adP0zPP2GDZMu1a1TzaN6gx+vbtq9Jkrz6a2r3rq625rmibPyAgQGwjYsye1s+QMdNYpPr374/+\n/ftDoVAgPT0d33//PaKjo8WHZRJJSVZWEZ59dqnOHkF18+bSp75NoscpFArxeYRNnVbNbh4+fIjU\n1FRs27YNly9frrepFxFV8/DwwMmTJwFU328YERGBd999F3369EFAQAB+++03cd2rV68iJCQE/v7+\niIyMVLld4dHTXAkJCZg9e7bKfj788EOsWLECgOqpKKVSiY8++ggDBgyAl5cXjhw5ojZfTUZtW5Fr\nsmPHDvj6+sLV1RVvvfWWOGX6woULGDBgAORyOQDg2rVr6N+/vziJysPDA/Hx8fDz84Orqyuio6NV\nHiLwqJqW5n369IG/v7/Ygh2onnj12muviV937doV27Ztg7e3N/r3749ly5ZplRcAfv75Z/j4+KBf\nv35Yvny52s+cn5+P3r17q7RWv3r1KgYMGACFQoG//voLEydOhKurK9zc3DBv3jyVhoQeHh5ISEjA\n6NGj8corr0ChUKh8jy5duoTx48cjICAAQ4YMwfLly1FVVaX1Z/z+++/h6+srjldNw8P8/HzMnj0b\nbm5uGDFiBL755hu1n9EQNBapOXPmwNfXF+np6fjHP/6BtLQ0LF68WB/ZiP5WDh8+DH9/f5w7dw7D\nhw8Xf4lUVlYiPDwcQUFB2L17N0aNGlWr71HNaS4/Pz8cP35cbIGhVCqxf//+Ok9tbd++HUePHsXu\n3buxc+dOpKamasz4JK3I1UlLS0NCQgK++OILnDx5En379sXcuXMBVHeWHT9+PN59912Ul5dj/vz5\niIiIEHsxAdXXkzZu3IgDBw7gxo0b+PLLL+vcT8eOHbF161acP38eM2fOxPz581V6aT1+avDIkSPi\ndfB9+/aJ13rqy1tYWIhZs2Zh7ty5SE9PR4cOHdSeRWrXrh1eeeUVlXHeu3cvRo0aBVNTUwiCgOnT\np+Pnn39GSkoK5HI5Pv/8c5VtpKSkICEhAWfPnq11JGVqaoro6Gjs2bMH27dvR3p6Ov7973/X+xlP\nnDgBANi3bx+++OILrF27FufPn8eXX34JGxsbMVO3bt1w4sQJbNq0CVu2bMHPP/9c52c0BI1FauzY\nsThw4ACWLVsmtq8moifn4uKCIUOGQCaTITAwEH/88QcA4OLFi6iqqsIbb7wBU1NTeHt7o4eaPvWO\njo546aWXcODAAQDVHbQfbfX9qP3792PixImwt7eHtbW1Vg0JH6WpFbk627dvx7Rp0/Dcc8/BxMQE\n06ZNU2mZHh4ejnv37mHs2LFwcHBQOeIBqo8GazJPnz5dpWX9o7y9vcUn1Pj4+KBjx471tpAICwtD\ny5YtxSaGNUeG9eU9duwYunTpIrarf/PNN1W62z7Oz89P5TpSSkqK2I7jmWeegZubG8zMzNC6dWtM\nnDix1qSIN954A/b29jA3N6+17e7du6NXr16QyWRwdHTEuHHjar3/8c9Yc7S0Y8cOvPXWW2Jn4g4d\nOsDBwQGXL19GUVERZsyYAVNTUzg7OyM0NFTtmBuCxmtSQ4YM0UcOor+9R3+5tWjRAuXl5VAqlSgo\nKKjVWdbJyUntdvz8/JCcnIzAwEDs3btXpSfRo/Lz87Vua/44bVqRq5OTk4MVK1bgo48+AlDdPVcm\nk0Eul8PBwQFmZmYICQnBihUrEB0dXev9j46Fk5NTna3hgermiJs2bUJ2djaA6u7Ej7Zaf9zjrdNr\nGinWl/fxMQSqO9Wq4+3tjRUrVuD27dtiU8S+ffsCAO7cuYMVK1bg7NmzKCsrg0KhqNXyqL7+UTdv\n3sTq1avx66+/orKyEgqFQiw66j7jo+3hn3nmmVrbzM7OhlwuR//+/cXPrlQqte7oqw9aPWCWiHSn\nbdu24jWaGjk5OXX+UgGqmxOuWbMGcrkcaWlp2L59u9rtPtpq/fFH6VhaWqptY9/YVuQzZsxQWzzl\ncjliY2MREhKCVatWYefOnWjWrJn4+qOZs7Oz0a5du1rbyMnJweLFi7Flyxbx4QJBQUH1dhVWp337\n9mrz3rx5s1ZbovqenmBtbY1BgwYhOTkZmZmZKo0W161bB5lMhuTkZLRq1QppaWn48MMPtc65dOlS\nvPTSS1iwYAGef/55bN68udZpYXXat2+PrKysWssdHBzg7Oys1algQ+G5OyIDqfmF+vLLL8PMzAzf\nfPMNFAoFfvrpJ1y+fFnt+2xtbdGvXz9ERUWhQ4cOeP755+tcz8fHB9988w3kcjmKi4trTU/v2rUr\nkpOTUVVMEdvJAAAYFUlEQVRVhcuXL6v8ompMK/Lx48cjLi5OPJ1WUlKC/fv3i69HRUVh3LhxWLFi\nBdq1a1frpv3vvvsOcrkcRUVFiIuLU2mxXuPBgwcwMTFB69atoVQqsXPnTly/fl2rfI+bMGGC2rzD\nhg1DRkaG2K5+8+bNuHPnTr3b8/Pzw+7du/HTTz+pFL7S0lJYWVnBysoKcrkcGzZseKKcpaWlaNmy\nJVq0aIHMzExs3bpV6/eGhoZi48aN4mSdrKws5ObmolevXrCyskJCQgLKy8uhUChw/fr1en/+9E3t\nkdSjM4/q8vhhJpEUPPOMDW7eXKrT+6S0pemXes3rzZo1w+eff45Fixbh008/hbu7O0aOHFnve/39\n/fHee+9hwYIFavc5btw4/Pe//8Xo0aPRqlUrTJ48WWwJDlRPinrnnXfE20wCAgJQXFwMoPqo5MSJ\nExg6dChsbGwwZ84ctUdsjxsxYgTKysoQGRmJ3NxctGrVCgMHDsSoUaOwZcsWFBYWijMUV65ciaCg\nIHh4eIidW/39/TF58mQUFBTA09MTM2bMqLWPTp06YdKkSXj11VdhYmKCoKAg9OnTR22m+r4X9eVt\n3bo11q9fj+XLlyMqKgqBgYEaHwvn6emJRYsWwcnJCS+++KK4PDw8HAsWLEDfvn3RsWNHBAYGYtOm\nTfVmfHTZu+++i8WLFyMhIQHdu3eHn58f0tPTtfqMo0aNQnFxMd555x3k5+fDyckJa9euhYODA+Li\n4rB69Wp4enqisrISzz33HObMmVPvZ9Qnte3jX3/9dQBARUUFrly5Ig72H3/8gR49emj9A6sLbB//\ndLB9vPQ09dweHh5YsWIF3NzcnkKq+jX1sdYnnbSPr5krHx4ejl27dolF6s8//0RsbGyDdkZERPQk\nNF6TunHjhsoha5cuXZCZmanTUETUNBn7Y5vo6dM4u+/FF1/EwoULMXr0aADVN9o9WrSIiJ6Wx2fS\nEWksUqtWrcLWrVuxZcsWAEC/fv0wYcIEnQcjIiLSqjPv+PHjMXToULVTXYmIiHRB4zWpgwcPIjAw\nEG+99RYA4Pfff8f06dN1HoyIiEhjkfriiy+wY8cOsQFht27dxMeQEBER6ZLGImVmZiape3uIiKjp\n0HhNqnPnzkhKSoJCocDNmzfxzTffaLzjmoiI6GnQeCS1ePFiZGRkwNzcHHPnzkXLli2xcOFCfWQj\nIqImTuORlIWFBSIjIxEZGamPPERERCKNRerGjRvYuHEjsrOzVVoV19w3RUREpCsai9ScOXMwfvx4\nhIaGsisvERHplcYiZWZmVqu9MxERkT5oPDQaPnw4vvvuO+Tn56OoqEj8j4iISNc0HkklJiYCgEoX\nSZlMxgdBEhGRzmksUocOHdJHDiIiolrUFqmTJ0/Czc0NP/30U52va2pvTURE1Fhqi9SZM2fg5uaG\nw4cP1/k6ixQREema2iI1e/ZsANX9pIiIiAxB4zUpADhy5AiuX7+O8vJycVl4eHijd65UKjFmzBjY\n29vjq6++QnFxMSIjI5GdnQ1nZ2fExMTw4bZERE2YxinoS5YsQUpKCr799lsAQGpqKnJycp7Kzrds\n2YJOnTqJX8fHx8PNzQ2pqalwdXVFXFzcU9kPEREZJ41F6sKFC1izZg2sra0RHh6Obdu24ebNm43e\ncV5eHo4ePYrQ0FBx2cGDBxEcHAwACA4ORlpaWqP3Q0RExktjkWrRogWA6gfNyuVyNGvWDAUFBY3e\n8cqVK7FgwQLIZDJx2Z07d2BnZwcAaNu2LQoLCxu9HyIiMl4ar0m5u7vj3r17mDJlCkJCQiCTyTB2\n7NhG7fTIkSOws7NDt27dcOrUKbXrPVrAiIio6dFYpGbOnAkA8Pb2xvDhw1FeXt7oyQznz5/HoUOH\ncPToUZSXl6O0tBTz58+HnZ0dbt++DTs7OxQUFMDW1lbtNjRdFysrK0NJSUmjcj5tj048kYKysjKt\nri+WlJQ8teuQ+mKMmQHm1idjzAwYb+6GUluk1N3EW6Mx90nNnTsXc+fOBQCcPn0aGzduxNq1a7Fm\nzRrs2rUL06ZNQ2JiIjw9PdVuw9HRsd59WFpaSnJmoJQyWVpaahxHoPoPAm3WkxJjzAwwtz4ZY2bA\nOHPn5uY2+L1qi5S6m3hr6OJm3mnTpiEiIgI7d+6Ek5MTYmJinvo+iIjIeKgtUvq6ibd///7o378/\nAMDGxgabNm3Sy36JiEj6NF6Tunv3Lr744gucO3cOMpkMffr0wcyZM9G6dWt95CMioiZM4xT0uXPn\nonXr1vjss8+wfv162NraIjIyUh/ZiIioidN4JFVQUCDO8AOAt99+G/v27dNpKCIiIkCLI6lBgwYh\nOTkZSqUSSqUSKSkpGDx4sD6yERFRE6fxSOr777/H5s2bsWDBAgCAQqGAhYUFtm3bBplMhvPnz+s8\nJBERNU0ai9SFCxf0kYOIiKgWjaf7fvjhB5WvFQoFYmNjdRaIiIiohsYilZ6ejqlTpyI/Px9//vkn\nxo0bh9LSUn1kIyKiJk7j6b5PPvkEKSkpCAgIgKWlJT7++GO4uLjoIxsRETVxGo+kbt68iS1btsDb\n2xuOjo7YvXs3Hjx4oI9sRETUxGk8kpo+fTref/99uLm5QRAEfP311xg7diySk5P1kY+IiJowjUVq\nx44daNmyJYDq/k6TJ0/G8OHDdR6MiIhI7em+hIQEAEDLli1rPWEiMTFRt6mIiIhQT5FKSUkR/x0f\nH6/y2vHjx3WXiIiI6P9TW6QEQajz33V9TUREpAtqi5RMJqvz33V9TUREpAtqJ05cu3YNffr0gSAI\nKC8vR58+fQBUH0VVVFToLSARETVdaovU77//rs8cREREtWi8mZeIiMhQWKSIiEiyWKSIiEiyWKSI\niEiyWKSIiEiyWKSIiEiyWKSIiEiyWKSIiEiyWKSIiEiyWKSIiEiyWKSIiEiyNHbmJdLGkiUxyMoq\nMnQMUVlZGSwtLQ0dQ8Uzz9hg2bIIQ8cgMioGKVJ5eXlYsGAB7ty5AxMTE4SGhuKNN95AcXExIiMj\nkZ2dDWdnZ8TExKBVq1aGiEhPKCurCM8+u9TQMUQlJSWS+9m5eXOpoSMQGR2DnO4zNTVFVFQUkpOT\nsW3bNnz33XfIzMxEfHw83NzckJqaCldXV8TFxRkiHhERSYRBilTbtm3RrVs3AICVlRU6deoEuVyO\ngwcPIjg4GAAQHByMtLQ0Q8QjIiKJMPjEiVu3buHatWvo3bs37ty5Azs7OwDVhaywsNDA6YiIyJAM\nWqRKS0sxe/ZsREdHw8rKim3qiYhIhcFm91VVVWH27NkIDAzEiBEjAABt2rTB7du3YWdnh4KCAtja\n2qp9f05OTr3bLysrQ0lJyVPN3Fjl5eWGjqCirKxM4zgC1ZMQjG28pTbWgHbjrc1YS5Ex5jbGzIDx\n5m4ogxWp6OhodO7cGRMnThSXeXh4YNeuXZg2bRoSExPh6emp9v2Ojo71bt/S0lJys7sASCqTpaWl\nxnEEqv8gMMbxlloebcZbm7GWImPMbYyZAePMnZub2+D3GuR037lz55CUlIT09HQEBQUhODgYx44d\nw9SpU/HLL7/A29sb6enpmDZtmiHiERGRRBjkSMrFxQW///57na9t2rRJv2GIiEiyDD67j4iISB0W\nKSIikiwWKSIikiwWKSIikiwWKSIikiwWKSIikiwWKSIikiwWKSIikiwWKSIikiwWKSIikiwWKSIi\nkiwWKSIikiwWKSIikiwWKSIikiwWKSIikiwWKSIikiwWKSIikiwWKSIikiwWKSIikiwWKSIikiwW\nKSIikiwWKSIikiwWKSIikiwWKSIikiwWKSIikiwWKSIikiwWKSIikiwWKSIikiwzQwcgoiezZEkM\nsrKKDB1DRVlZGSwtLQ0dQ/TMMzZYtizC0DHoKWCRIjIyWVlFePbZpYaOoaKkpAStWrUydAzRzZtL\nDR2BnhJJnu47duwYRo0aBW9vb8THxxs6DhERGYjkipRSqcTy5cuxYcMG7N27F8nJycjMzDR0LCIi\nMgDJFalLly6hY8eOcHJyQrNmzeDn54eDBw8aOhYRERmA5IqUXC6Hg4OD+LW9vT3y8/MNmIiIiAyF\nEyeIiNTgTErNdD2TUiYIgqCzrTfAxYsX8fnnn2PDhg0AIE6cmDZtmrjOuXPnDJKNiIgaxsXFpUHv\nk9yRVM+ePZGVlYXs7Gy0bdsWycnJWLdunco6Df2wRERkXCRXpExNTbF48WJMnjwZgiBg7Nix6NSp\nk6FjERGRAUjudB8REVENyR1J1aW4uBiRkZHIzs6Gs7MzYmJi6ry73cPDAy1btoSJiQnMzMywY8cO\nA6Stvhl55cqVEAQBY8aMUbmeVuPDDz/EsWPHYGFhgdWrV6Nbt24GSPp/NGU+ffo03n77bXTo0AEA\n4OXlhbffftsQUUXR0dE4cuQI2rRpg6SkpDrXkdo4A5pzS3Gs8/LysGDBAty5cwcmJiYIDQ3FG2+8\nUWs9qY23NrmlON4VFRX4xz/+gcrKSigUCnh7eyM8PLzWelIab20yN2isBSOwZs0aIT4+XhAEQYiL\nixPWrl1b53oeHh5CUVGRPqPVolAohBEjRgi3bt0SKioqhNGjRwsZGRkq6xw5ckSYOnWqIAiCcPHi\nRSE0NNQQUUXaZD516pQQFhZmoIR1O3PmjHD16lXB39+/ztelNs41NOWW4ljn5+cLV69eFQRBEO7f\nvy+MHDlS8j/XgqBdbimOtyAIQllZmSAIglBVVSWEhoYKv/76q8rrUhxvTZkbMtaSu0+qLgcPHkRw\ncDAAIDg4GGlpaXWuJwgClEqlPqPVos3NyAcPHkRQUBAAoHfv3igpKcHt27cNEReA8d5A3bdvX1hb\nW6t9XWrjXENTbilq27at+Fe6lZUVOnXqVOv+RSmOtza5pcrCwgJA9RFKVVVVrdelON6aMjeEURSp\nwsJC2NnZAaj+oSssLKxzPZlMhsmTJ2PMmDH4/vvv9RlRpM3NyPn5+Wjfvr3KOnK5XG8ZH6ftDdQX\nLlxAYGAgpk2bhoyMDH1GbBCpjfOTkPJY37p1C9euXUOvXr1Ulkt9vNXlBqQ53kqlEkFBQRg0aBAG\nDRpkFOOtKTPw5GMtmWtSkyZNqvOvgIiI2jeJyWSyOrexdetWtGvXDoWFhZg0aRKef/559O3b96ln\nbYq6d++OI0eOwMLCAkePHsXMmTORmppq6Fh/S1Ie69LSUsyePRvR0dGwsrIydByt1ZdbquNtYmKC\nH3/8Effv38fbb7+NjIwMdO7c2dCx6qUpc0PGWjJHUl9//TWSkpJq/efp6Yk2bdqIBaygoAC2trZ1\nbqNdu3YAAFtbW3h5eeHy5ct6y1/D3t4eOTk54tdyuVzMVaNdu3bIy8sTv87Ly4O9vb3eMj5Om8xW\nVlbiofywYcNQWVmJoiJp3Yn/OKmNs7akOtZVVVWYPXs2AgMDMWLEiFqvS3W8NeWW6njXaNmyJVxd\nXXH8+HGV5VIdb0B95oaMtWSKVH08PDywa9cuAEBiYiI8PT1rrfPgwQOUlpYCqH5syIkTJ/DCCy/o\nNSegejNyRUUFkpOTa+X19PTEjz/+CKD6CRvW1tbi6UxD0Cbzo0e5ly5dAgDY2NjoNWddhHruoJDa\nOD+qvtxSHevo6Gh07twZEydOrPN1qY63ptxSHO/CwkKUlJQAAB4+fIhffvkFzz//vMo6UhtvbTI3\nZKwlc7qvPlOnTkVERAR27twJJycnxMTEAKg+J7t48WLExcXh9u3bCA8Ph0wmg0KhQEBAAAYPHqz3\nrOpuRt62bRtkMhleffVVDBs2DEePHoWXlxcsLCywatUqved80sypqanYunUrzMzM0KJFC3z66acG\nzQwA77zzDk6dOoWioiK4u7tj1qxZqKyslOw419CUW4pjfe7cOSQlJaFLly4ICgqCTCZDZGQkcnJy\nJD3e2uSW4ngXFBTgvffeg1KphFKphK+vL4YNGybp3yPaZG7IWPNmXiIikiyjON1HRERNE4sUERFJ\nFosUERFJFosUERFJFosUERFJFosUERFJFosUGa1u3bohODgYAQEBiIiIQHl5eZ3rhYWF4f79+0+8\n/fz8fMyZM6fB+Tw8PCT15AJdSUxMREFBgaFj0N8UixQZLQsLCyQmJiIpKQlmZmbYunVrrXUEQUBc\nXBxatmz5xNtv164d1q9f3+B86p4x+Xeza9cugz/YlP6+jOKJE0Sa9O3bF3/++Seys7MxZcoU9O7d\nG1evXkVcXBz++c9/YteuXSgtLcXUqVPh4uKCCxcuwN7eHl9++SXMzc2RlZWF999/H4WFhTA1NcX6\n9ethYmKC6dOnIykpCYmJiThw4ABKSkqQn5+PgIAAsaHbzJkzkZeXh4qKCrzxxhsIDQ0FoP7RR8eO\nHUNMTAyUSiVat26Nr7/+GsXFxYiOjsZff/0FS0tLLFu2DF26dEFsbCxu3bqFv/76C7m5uYiKisLF\nixdx7NgxtG/fHl999RVMTU3h4eEBHx8fsQHeJ598gg4dOiA7OxvR0dEoKiqCra0tVq1ahfbt2yMq\nKgpWVla4cuUK7ty5g/nz52PkyJEAgA0bNmDfvn2orKyEl5cXwsPDkZ2dXefYHT58GFeuXMH8+fPR\nokULbN++HZ999hkOHz4MMzMzDBo0CAsWLNDPDwH9PTW2yRWRobz88suCIAhCZWWlMGPGDGHr1q3C\nrVu3hK5du6o0W/Pw8BDu3r0r3Lp1S+jevbtw7do1QRAEYc6cOcKePXsEQRCE0NBQIS0tTRAEQSgv\nLxcePnwo3Lp1S2xMuGvXLmHw4MFCcXGx8PDhQ8Hf31+4cuWKIAiCUFxcLAiCIC6vabw5fPhw4e7d\nuyqZ79y5IwwbNkzIzs5Wee/y5cuF2NhYQRAE4eTJk0JgYKAgCILw+eefC6+99pqgUCiE33//Xejd\nu7dw/PhxQRAEYebMmWLm4cOHC3FxcYIgCEJiYqLYWC4sLEz48ccfBUEQhB07dghvv/22IAiC8N57\n7wlz5swRBEEQMjIyBC8vL0EQBOHEiRPC4sWLBUEQBKVSKYSFhQlnzpypd+z++c9/Cr/99psgCIJw\n9+5dwdvbW/y8JSUlGr+PRPXh6T4yWuXl5QgODkZoaCgcHR0xduxYAICTk5NKHxvhkSMaJycnvPji\niwCq2wZkZ2ejtLQU+fn54kN1zc3N0bx581r7GzRoEKytrdG8eXN4eXnh3LlzAIDNmzcjMDAQ48aN\nQ15eHv773/+qzfzrr7+iX79+cHR0BACx+eG5c+cQGBgIABgwYACKi4vFByYPHToUJiYmePHFF6FU\nKsVnUnbp0gXZ2dnitn19fQEA/v7+uHjxIoDqB4/6+/sDAAIDA3H+/Hlx/Zongnfq1Al37twBAJw4\ncQI///wzgoODERwcjBs3boifp66xe3yMW7VqhRYtWmDhwoU4cOBAneNI9CR4uo+MVosWLZCYmFhr\neU0rgLqYm5uL/zY1NRUnWwhaPMLy8WtMMpkMp0+fRnp6On744QeYm5vj9ddfVzuBo0Zd+6rv+lVN\nZplMBjOz//tf1sTEBAqFos5taHM97NGxeDRTWFgYxo0bp7Judna22rF7lKmpKX744QecPHkS+/fv\nx7fffovNmzdrzEKkDo+kyGhpU1i0YWVlBQcHB6SlpQGobn398OHDWuv9/PPPuHfvHh4+fIi0tDT0\n6dMHJSUlsLa2hrm5OTIzM/Hrr7/Wu6/evXvj3Llz4lFIcXExAMDFxQV79uwBAJw6dQqtW7eus6lg\nfZ85JSUFAJCcnIyXX34ZANCnTx/s3bsXALBnzx61TUBrtjt48GDs3LkTZWVlAKp7i6nrhF3DyspK\nnD1ZVlaGkpISDB06FFFRUfjjjz/qfS+RJjySIqOl7ew5bdb76KOPsGTJEnz22Wdo1qwZ1q9fX+t9\nvXr1Qnh4OORyOQIDA9G9e3e88MIL2LZtG/z8/PDcc8+JxUHdfm1tbbFs2TKEh4dDEAS0adMGGzZs\nQHh4OKKjozF69GhYWlrio48+euLPcu/ePYwePRrNmzfHunXrAACLFi1CVFQUNm7cKE6cqG+7gwYN\nwn/+8x+8+uqrAKoL0Nq1a2Fiov7v2ZCQELz//vuwsLBAQkICZsyYIR5lRUVFqX0fkTbYqoNIC4mJ\nifjtt9+waNEiQ0epU01jUEM36yN62ni6j+hvoKnck0VND4+kiIhIsngkRUREksUiRUREksUiRURE\nksUiRUREksUiRUREksUiRUREkvX/ABoSRNjr/939AAAAAElFTkSuQmCC\n",
"text/plain": [
""
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"with plt.style.context('seaborn-whitegrid'):\n",
" plt.figure(figsize=(6, 4))\n",
"\n",
" plt.bar(range(4), var_exp, alpha=0.5, align='center',\n",
" label='individual explained variance')\n",
" plt.step(range(4), cum_var_exp, where='mid',\n",
" label='cumulative explained variance')\n",
" plt.ylabel('Explained variance ratio')\n",
" plt.xlabel('Principal components')\n",
" plt.legend(loc='best')\n",
" plt.tight_layout()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"The plot above clearly shows that most of the variance (72.77% of the variance to be precise) can be explained by the first principal component alone. The second principal component still bears some information (23.03%) while the third and fourth principal components can safely be dropped without losing to much information. Together, the first two principal components contain 95.8% of the information."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"
\n",
"
"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Projection Matrix"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"[[back to top](#Sections)]"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"It's about time to get to the really interesting part: The construction of the projection matrix that will be used to transform the Iris data onto the new feature subspace. Although, the name \"projection matrix\" has a nice ring to it, it is basically just a matrix of our concatenated top *k* eigenvectors.\n",
"\n",
"Here, we are reducing the 4-dimensional feature space to a 2-dimensional feature subspace, by choosing the \"top 2\" eigenvectors with the highest eigenvalues to construct our $d \\times k$-dimensional eigenvector matrix $\\mathbf{W}$."
]
},
{
"cell_type": "code",
"execution_count": 73,
"metadata": {
"collapsed": false
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Matrix W:\n",
" [[ 0.52237162 -0.37231836]\n",
" [-0.26335492 -0.92555649]\n",
" [ 0.58125401 -0.02109478]\n",
" [ 0.56561105 -0.06541577]]\n"
]
}
],
"source": [
"matrix_w = np.hstack((eig_pairs[0][1].reshape(4,1), \n",
" eig_pairs[1][1].reshape(4,1)))\n",
"\n",
"print('Matrix W:\\n', matrix_w)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"
\n",
"
"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## 3 - Projection Onto the New Feature Space"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"[[back to top](#Sections)]"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"In this last step we will use the $4 \\times 2$-dimensional projection matrix $\\mathbf{W}$ to transform our samples onto the new subspace via the equation \n",
"$\\mathbf{Y} = \\mathbf{X} \\times \\mathbf{W}$, where $\\mathbf{Y}$ is a $150\\times 2$ matrix of our transformed samples."
]
},
{
"cell_type": "code",
"execution_count": 78,
"metadata": {
"collapsed": false
},
"outputs": [],
"source": [
"Y = X_std.dot(matrix_w)"
]
},
{
"cell_type": "code",
"execution_count": 99,
"metadata": {
"collapsed": false
},
"outputs": [
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAakAAAEbCAYAAABgLnslAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzs3Xtc03X7P/DXgA2mHASdiKSQWuGdKeABzSMIViqKB1JK\nS1kJmof4qqXm3Z2p3WYa5RlrdWf+XHbQklvLwuNtBzFE0ZQ79RZUDjJBicNgA96/P5C5wcYObPt8\nBtfz8bgfd26fbZdz7OL9+Vzv6xIwxhgIIYQQHnLiOgBCCCHEEEpShBBCeIuSFCGEEN6iJEUIIYS3\nKEkRQgjhLUpShBBCeMuFyxdXqVR4/vnnoVarUVtbi6eeegrz58/nMiRCCCE8IuB6n5RSqYRYLEZt\nbS3i4uKwcuVK9O3bl8uQCCGE8ATnp/vEYjGA+lVVTU0Nx9EQQgjhE86TVF1dHWJiYjB06FAMHTqU\nVlGEEEI0OE9STk5O+Pbbb3Hy5EmcP38eV69e5TokQgghPMFp4YQ2d3d3hIWF4T//+Q969eqlc19G\nRgZHURFCCLGV/v37Gz2G0yRVUlICoVAIDw8PVFVV4ZdffsGcOXP0HmvKX4aP8vPz0bVrV67DsIij\nxu6ocQMUO1codvszdfHBaZJSKBRYtmwZ6urqUFdXh7Fjx2LkyJFchkQIIYRHOE1Sjz32GPbv389l\nCIQQQniM88IJQgghxBBKUoQQQniLkhQhhBDeoiRFCCGEtyhJEWJnCoUCZ86cgUKh4DoUQniPkhQh\ndrRXLkdQQAASo6IQFBCAvXI51yERwmuUpAixE4VCgXlSKY4plcgoLcUxpRLzpFJaURHSDEpShNhJ\nTk4OAkUiNLRQ7gsgQChETk4Oh1ERwm+UpAixk8DAQOSoVMi6/+csALlqNQIDAzmMihB+oyRFiJ1I\nJBJsk8kQLhYj1NMT4WIxtslkkEgkXIdGCG/xpgs6IW3BtLg4RERG1p/6CwykBEWIEZSkCLEziURC\nyYkQE9HpPkIIIbxFSYoQQghvUZIihBDCW5SkiFmopQ8hxJ4oSRGTyeV7ERAQhKioRAQEBEEu38t1\nSISQVo6SFDGJQqGAVDoPSuUxlJZmQKk8Bql0Hq2oCCE2RUmKmCQnJwciUSCg1dRHKAyglj6EEJui\nJEVMEhgYCJUqB9Bq6qNW51JLH0KITVGSIiaRSCSQybZBLA6Hp2coxOJwyGTbaFMqIcSmqOMEMVlc\n3DRERkZQSx9CiN1QkiJmoZY+hBB7otN9hBBCeIuSFCGEEN7i9HRfYWEhXnvtNRQXF8PJyQmxsbF4\n4YUXuAyJEEIIj3CapJydnbF8+XL07t0bFRUVmDx5MoYOHYqePXtyGRYhZlMoFFRQQogNcHq6TyKR\noHfv3gCA9u3bo2fPnigqKuIyJELMtlcuR1BAABKjohAUEIC9cjnXIRHSavDmmtStW7eQnZ2Nvn37\nGj+YEJ5QKBSYJ5XimFKJjNJSHFMqMU8qpXZRhFgJL0rQKyoqsHDhQqxYsQLt27fXe0x+fr6do7KO\nsrIyit3O7Bn3uXPnEODiotUsCuju4oIzZ84gODjY7Odz1PccoNi54sixm4LzJFVTU4OFCxdi4sSJ\niIyMNHhc165d7RiV9eTn51PsdmbPuIVCIXJrapCF+gSVBeBGTQ0GDhxo0bUpR33PAYqdK44ae0FB\ngUnHcX66b8WKFejVqxdefPFFrkMhxGwSiQTbZDKEi8UI9fREuFiMbTIZFU8QYiWcrqQyMjKQmpqK\nRx99FDExMRAIBEhKSsKIESO4DIsQg/RV8U2Li0NEZCRV9xFiA5wmqf79++Py5ctchkCIyfbK5Zgn\nlSJQJEKOSoVtMhmmxcUBoHZRhNgK56f7CHEEramKT6FQ4MyZMw4ZO2l7KEkRYoKcnBwEikQ6VXwB\nQiFvhz4aSkS0p4s4GkpShJggMDAQOSqV1shHIFet5uXQR0OJiE+rQVrNEVNRkiLEBI5SxddcIuLL\nalAulyOgVwCino1CQK8AyL+g1RwxjPN9UoQ4Ckeo4tMkIqUSgG4i0l4NNuzpsvdqUKFQQJoohfI5\nJZRdlEAhIE2QInJ0JC/fT8I9WkkRYgaJRGLxRl17aO60JB9Wg5mZmXDycgK63L+hCyD04e+1PcI9\nWkkR0opoEpFUigChELlqtU4i4nI1uFcuR2J8PCprq4BC1CeqQkBdws9re4QfKEkRwiPFxcXIy8vT\nSSDmjAFRKBTo0asXTmVkoLy8XO9juNjT1XCt7ERVFf4AEP8xUOUBiGvEkO3k37U9wh90uo8QC9ii\nOm2vXI6RgwbpVOWZUzKuXZDQf3B/XL12lTdf/tpFG3EAbtQAj6ra49uvv0Xc9DiuwyM8RkmKEDPZ\nYq9Rw0rjeFWVTlVeYny8SSXj2gUJpbNKoXxOCWkCfzYbN75WVgDgTl0dQkJCuAyLOABKUoSYwVZ7\njfSVhz/k5ARfZ2eTSsZzcnIg8hHxtiCBD0UbxDHRNSlCzNBciXdLvnD1lYffqqtDHWMmlYwHBgZC\nVaLidUGCI5TwE/6hJEWIGWy116hhpTEqPh6BIpGmKg+AwUq9xo+XpcggTZBC6COEukQNWQr/VirU\niJeYi5IUIWYwVuLdEtPi4vC3Pn1QVVWls9IwdfURNz0OkaO5XamYU4lIiCkoSRFiJlueturYsWOT\nKavmrD64XKl8t38/Vi5dqneUCSGWoiRFiAX4ctqKLysXhUKBN5YswfGqKvRVKpGF+tOUEZHU7oi0\nDFX3EeIA9O3L4tPYjZycHAQKhZw3ryWtDyUpQnhOXzLi09gN4H5BiVrtEKNMiGOhJEUIjxlKRpmZ\nmbwYu9FAIpFg7caNtA+KWB1dkyKExwztywLA+diNxibGxGBqbCwvrpGR1oNWUoTwmKHRGyEhIbzs\n4GDqKBOazEtMRUmKEB5rrp3QtLg4ZOfmIiUtDdm5uQ5T7k2TeYk56HQfITzXmtoJ0WReYi5aSbVS\ndDqlddF3Go1PJeim4nsjXMI/RpOUWq1ucltJSYnVAlixYgWefPJJREdHW+052zq5fC8CAoIQFZWI\ngIAgyOV7uQ6JNMOSXyj4VoJuKp1GuAAvG+ESfjGYpH777TeMGDECw4YNQ3x8PG7duqW5TyqVWi2A\nyZMnQ3a/kSZpOYVCAal0HpTKYygtzYBSeQxS6Tzef3m1VY1XQ5s+/NCkfyt9oz0cYfNsQyNc8R4x\nPP/lCfEeMS8b4RL+MJik3nvvPchkMpw+fRrPPvss4uPjce7cOQAAY8xqAQwYMACenp5We762Licn\nByJRIKD19SUUBvD+y6st0rca2rx+PR7t3t3oqTt9VX/XVSrcvXuX97+QxE2PQ+7VXKR9mYbcq7k0\nmZc0y2CSUqvVeOSRRwAATz/9NLZu3YrXX38daWlpEAgEdguQmCcwMBAqVQ6g9fWlVufS6RQe0rca\negzA1qoqo6fuGlf9DRMKwerqsPzZZx3i+pSppeqEGExSLi4uOj8kjzzyCD777DNs3ryZfiu3I3Ov\nV0gkEshk2yAWh8PTMxRicThksm30ZcBDevdAAYiCaafuGkrQ1331FYQuLjipUjnE9Skq6iHmMFiC\nvmTJEhQXF+t8uXXp0gW7d+/G7t277RKctvz8fLu/pjWUlZVZHPv+/d9hyZI3IBQGQq3OwcaNaxET\nM9Ho40aOHI7Tp4/j5s2b6NatGzp27GhRDC2JnUuOFPeaDRswavFi+FRVoQTAdgAFqO8m4ebmZtLf\no6amBgEuLjorsu4uLjhz5gyCg4NtF3wjprzv3+3fjzeWLEGgUIgctRprN27ExJgYk1+juLhY53Nt\nLY70mWnMkWM3CeOBmzdvsvHjxxu8//fff7djNNaVl5dn0eOKioqYWOzDgPMMYAw4z8RiH1ZUVKT3\n2PT0dL33tYSlsXPN0eIuKipia1evZh3c3FiwhwfzEYvZF3v2mPV4H7GYna//oLDzAPMRi63+eTDG\n2Pve0jj37NnDxJ5i5hXoxcSeYrZHbvp7ZIyjfWa0OWrspn6vc75PavHixZg+fTquX7+OUaNG4Ztv\nvuE6JF4wtQCCys0dn0QiwYqVK/HnjRtY+8UXZnWPaJgntS45mXctkhprSUWi9ibg0lmlUD6nhDSB\nv6c0ifVw3nFi48aNXIfAS7oFEPUtRBsXQGiXmyuV9cdIpeGIjIzg3RcUMU4ikSA4ONjkf7u9cjnm\nSaWaSbjrkpMRHBrK264U7u7uuFpVheMARsG8prgNm4CVXeob7WpvAubj35VYj9GV1Pfff2/SbcS6\nTCmAoHLztqNxsYG+8vVlSUm8TVByuRz9B/eHupMTwl2AQLGbWSs+2gTcdhldSe3cuRPPPPOM0duI\n9cXFTUNkZITBnm2mrLaI42u8Ytomk8G7Y0d0c3LSe+qMb0lK+1QdugAoBAp3M2Smn0Xv3r1Neo6G\nTcDSBCmEPkKoS9S0CbiNMJikTpw4gZMnT+L27dtYs2aN5vby8nI4OzvbJThS/8Np6AexYbUllYZD\nKAyAWp1L5eatjPaKqa9SiSwAw158Ec7OzlBVVfFqnpQh+k7VuXZyRXl5uVnPEzc9DpGjrd9oV6FQ\n4Ny5cxAKhfSzw0MGk5Svry/69OmDo0eP4vHHH9fc3r59eyxfvtwuwRHjjK22tDVcZNd3XHP3Ee40\nHnroB6BWrcYptRqXUX9txwfAXZ4WSwCASqVC5e1K4DqAh9GiU3XN/dJmCblcDmmiFC7eLqi5WwNZ\niow6YPCNsfI/lUrV4lLDlmqLJejWtmfPF0ws9mFeXqFMLPZhe/Z8YfQ+vsRuLkeNm7GmsTcu2/5/\nAOt1/78ZwIoA9mj79uzw4cMcRfyAvvd9/vz5DEIw+IDBBUzoLbR6+bilioqKmNhTzJAIhrfAkAgm\n9rR/6X5LOern3dTvdaPXpLKysrBlyxbk5+ejpqYGjDEIBAIcOXLEHjmUWEFzVYAAmr2PcEvT/kgq\nRYBQiOsqFVhdHbJUKvRF/cbfO3V1CAkJ4TpUjYZVuUqlwpaULYAUmmtR6o/VOHb8GIYOHcp1mFQx\n6CCMJqk33ngDy5cvR58+feDkxPm2KmKBhirA+iQEaFcBHj78E5RKH+irEPT396fTgDzQeOjh0bQ0\nTdLKVat5dZpPu8jjv5WVgCd0ZkfBC7h69SovkpROxWBDEqWKQd4xmqQ8PDwwcuRIe8RCrKwhwbi7\nu+utAnR3d8fate8BEDS5LzAwEF9++TWWLl0Jkai+ilAm24a4uGnc/YXaMO1rMQ1JKzMzEwCarKLs\n/YtFw+tVVlbqFHl8C2BSKXSSAP4CBg0aZPOYTKFdMejSwQU192qoYpCPjJ0PfO+999i6devY2bNn\n2cWLFzX/sye6JmW+xteZ5s9fyMRiH+bpGaK57pSens68vEIZ8AUDfBgQwoB2bPXqtayoqIi5uXmb\n1JaJbxz1HD1jpsf+xZ49zEcsZqFeXjptlAzdbivarYpE7UUsUCzWXC9jAOsodGFwAUNHMAjB5i+Y\nb9N4LFFUVMQOHjzoEJ9tfRz18261a1Lnz58HAFy8eFFzm0AgwK5du2yXOUmL6LsGJZOFIyPjFMrL\nyzW/YSsUivsrrN4AsgH8BDe3V5CQ8DJycnIgFAaiqqrpaUD6TZNb+srSw6VS9A0O1nt7RGSkTf7N\ntPc/KbsogUIg52PodJRgLkKc+uk4rl69ikGDBpm8L8qezO30QezLaJL6/PPP7REHsSJD16DKy8sx\ncOBAnWNXrFiMtWtHQiR6+P4+qx2aH1a1Oge0UZh/GpelN2zkTU9P13u7rX6x0Fd4IPYVY+KdOvR0\nddVcLxs6dGiz16DouidpjtFKiDt37mDFihV46aWXANRf9Pzqq69sHhhpytQ5PKYMPmxoTLthwzcQ\nCJywdOlU5OZma645SSQSbNy4luZS8ZDeOVRqNQYNGqT3dlv9YqGvVRHKgN8yM5GSlmZSo1y5XI6A\nXgGIejYKAb0CIP+C38MaCQeMnQ+USqXs4MGDLDo6mjHGmFqtbnashi3QNanm9zk1d7z2NagGqamp\nzNlZzID9zV5vysvLs9kYEFtytHP02u+xudekQjw99V6Tany7reyR11+T8gz0ZG4ebmbtfzJln5Kh\nz5+1P5eO9pnR5qixm/q9bjRJTZ48mTHG2MSJEzW3TZgwwcKwLNPWk5Q5s6UaP67xD3JU1DMMEDPg\nkfv/v5ABjHl6hrD09HSrx84FR4q7caHDtq1bTX6sqV/gxr7QW/qF3/D4rKwssx6Xnp7OvAK96hPU\n/f95BnpqPoeG5kfZYq6UI31mGnPU2K2WpGbMmMFKSkpYTEwMY4yxzMxM9vzzz7csOjO19ST1oArv\nQeGUvqRi7Mvm1KlT9xPTg2RX/+f9BldSjshR4tY3BNDbzc2kZGFqYjFW7WfNakBz3/fmVlKG7rt0\n6ZJNukQ4ymdGH0eN3WpDD5ctW4a5c+fixo0bmD59Ol5//XWsXLnSHmciyX3mXGNqbvjhjz/+COAh\nQKd3tj+cnZ+j600csHQI4F65HEEBAUiMikJQQAD2yvVfx9E3zmOeVNrsuA/t+22tYZ+SeI8Ynv/y\nhHiPWLNPqaEoQ3sjsNCnvjhE3+00nqYVMyWTqdVq9ueff7L//ve/nPTya+srKcaav8Zk7HRgw2/d\nqampeldSqampNo3d3hwlbktWUuaMYE9PT2ehXl46+5ZCPB+cTjN2v7mae9+bW/npu49WUqZz1Nit\ntk8KqO/fl5eXh9raWly6dAkAEBMTY9PkSXQ11+1cX8m5UumNlJSP0LNnT0il8zRdI5544jFcuDAY\ngD+APIwZMwrjx4/n4q/U5jXuy5erVmPthg3NrmhzcnIQ4OJi0hwp7SpAfeM8jN3fUg2l5WfPnkXS\na0kQ+YigKlE16TSur7O5oflRvXv3prlSbYzRJLV06VLcvHkTQUFBmjlSAoGAkhQHDI0p0Df8ECjB\nmjXvwsnJRWdT79Wr4UhN/RJnzpzBmDFjeNFDrS1r3JdPrVY3e/y5s2eRXVZmUmLRlwS1+/wZu78l\ntEdglOWXASMB5bD6Db/SBCkiRxvfYGxofpSt5koRfjKapC5evIhDhw5BIBDYIx5iAYlEghUrFuPv\nfx8C4DEAuQC2w8XlTQBiNG4e6+vri1WrVnEVLmlE+5eP/Px8APo3uCoUCixLSsJbAMJRf3XxCoAP\nk5MNflE3ToKNjzPWB9ASly9fxuw5s1E9o/pBz77PAITA7E7jhn4xs/ZcKcJfRgsnHnnkEbtdSCWW\nS0h4GW5uIgBLUN/iqDdqa4tQV3cT2gUXKtV13L17l/5NecxQYURDocVrqP8X/hhAT3d3BIeGAjC8\n2VsikWDgwIEGv9SPpqUhLiYGy599FkEBAfgoJcWkTeOGYh8cEgK1qFq3+7kngHuwa6dx7ffD1I3w\nhIeMXbSaMWMGGzBgAIuPj2cJCQma/9kTFU6YRl9xhfZtQqEHE4m8TN4Q7KgXZB01bsYYy8rK0hRG\nFN0fctjhfjFFc0UT5paSNxQrXLp0qclzigH2hIeH2SXpDbEfA5jYBTrFDRCCuXdzt9vAQ+29VEKx\nkInaiZrdV+XInxlHjd1q+6ROnz6t93/2REnKdIYqpQ4fPmz2hmBH/fA7atyMMXbw4EEW6uXFvgCY\nD8BCAdYOYGtXr2aM6e8oYU7Fn/ZzhHp5sQ6uruzhRp3L+wIs3YTnMRQ7A9ie+4nKyQfM1d2V7UjZ\nYbfOJTqVgUvB4Aaj1YCO/Jlx1NitVt03aNAg3LlzBxcuXAAA9O3bFx07drT5Co9YxlCllLe3t8HB\nh3Runz+6deuG/1VXYy7qu4k3FEeEv/MOXk5I0HuN6cyZMyY3ltXXQX0wdDuX3wIQCEDSzPMYir2h\nWjAOgF8NMLHCFb9lZtq1+7lO49s8AN7Qu6+KPveOweg1qUOHDiE2NhY//PADvv/+e81/E8diyoZg\nwr2OHTti6RtvoCNgcJNv42tMhhrO6vu31beBuKdYjImurgjx8MBgAM+Z8DyGYt8mkyFcLEaopyem\niMXY+emnehOULa8R6TS+7QDgLnSa4NL0XQdjbKkVHR3N7ty5o/lzcXGxptmsNZw4cYI99dRTbMyY\nMSwlJUXvMXS6zzqa2xCsD59iN4ejxs3Yg6a+5py+Y8xwY1l9ffz0PfelS5fY2tWrmYdQyHrdP8Xo\nLhSadU2q4X031rLJlOtnLe0nqN34tuGalGegJ12T4hGrXZNq3PG8trbWal3Qa2trWWRkJLt16xZT\nqVRswoQJ7OrVq02OoyRlPeb88PMtdlM5atyMPYjdkm7mjf9tjU3vbcl1LX2vacr7bsrrWKuBrHZs\nxj73reEzYw4+TDewWpJat24di4+PZ9988w375ptvmFQqZevXr29xgIzVN6uVSqWaP6ekpOhdTVGS\n4oajxu6ocTOmG3tLvkiMJQPt6r709HR2+PBhs1okFRUVsdVrVjOxx4NksnWb8Q7uxloxmTK+wxZa\ny2fGFLboIm8JqxVOvP766/jxxx+RkZEBAJg2bRqioqKscqrx9u3b8PPz0/zZ19dXU6BBbIOmoDqO\nlmxYNTS9t6FgQCKR4GhaGuZJpQgUiZCjUkFVU2NSJwu5XI74l+NRpaoCpNCMjl/8+mLETo1tNmZj\nrZj0TfulQgfrUSgUkCZKoXxOqfl3M7UDCFeMFk4A9bvQBw4ciLCwMAQHB9s6JmImUy9Cm9IpnbQO\nxoop9HVAdxYIMNLNDaGenggXi/W2SFIoFJj10ixUPVUFdIJO1ZxLBxccOnSo2c+hphXT/eKKxq+j\nb9ovFTpYj6Hu8nzuIm90JfXVV19h69atGDx4MBhjWLNmDebNm4epU6e2+MV9fX01bWCA+pVV586d\n9R6rfZwjKSsrs2ns+/d/hyVL3oBQGAi1OgcbN65FTMzEJscVFxcjPn4uqqqOa/r4xcePQp8+fzO4\npaC52IuLi3Hz5k1069aNd1sSbP2e25I1Y1+zYQNGLV6s07xWrVYjPz8f586da9Ko9mFXV7yWkgIv\nLy/Nv2vjWE6cOAGVmwroCeB71CeT+62PygvK8crKV1C7oBYb392ImIn6+3sOHzkSx0+f1vn8aL/O\nhnUbsPj1xRB6C6G+q8aGdx/EbStt5TPj5uaG6uJqnX83VbEKbm5u/P37GzsfOGbMGFZSUqL5c0lJ\nCRszZozF5yG11dTUaAonqqurqXDCTOZM7DV1cKIpsZs7yt7e2tL1BWOam95rSaHE4cOHmUB4f3Ps\n1PsbZb3B4AKGSOtdR7L3hf229JnRrnxsFdekvL290b59e82f27dvD29vb6skSGdnZ/z9739HfHw8\nGGOYOnUqevbsaZXnbgv0jeho2KDbcH/DtSd9ndIt2SelUCgglc7T6awulYYjMjKCt+e027LmGrRa\n0gE9JCQE7SBE7cdqiDyAajXA1E4Q+olRMayi/iArXEeiBrK242hd5I0mqe7du+PZZ5/F6NGjIRAI\ncOTIETz22GP49NNPAQCzZ89uUQAjRozAiBEjWvQcbZWhxHP27DmMHPm0ZoaUTLYNAFBTowIwBIAf\nRKI7kMlSzP6ANpcY+f5hJ7qMdUjXRyKRQPbZZ0iMj0dnlTOKhLVY/8EHWPRaks4pJLqOxG+O9EuA\nSUmqe/fumj+PHj0aAFBRUWG7qIhJJBIJZLJtkErDIRQGQK3ORXLyOiQlLdNZ6cTHj4RA4AS1+mcA\nfgB+gpPTK4iMjDD7Na21IiP8YMmXlb7k5u7lifg58RB1FNEgQmJVRpPU/Pnz7REHsVDjib36VjrO\nzp2hO1fqOYhEG5qsfkwpT9eXGGWybfSF1EZof0YGDhyouT1uehz6PN4HVVVVVj2F5IhbJhwxZj4z\nmqQuXLiAHTt2ID8/HzU1NZrbU1NTbRoYMV3j34Ybr3Rqa4tQW1unc5tSeVUzV0oikUAu36szZl4m\n24aRI4frfb3mRtmT1muvXK6zr2qbTIZpcQ/GwHfs2BFdu3a12us1TPcV+YhQfacabyx7AwlzEiz+\nvNkjeWjHrCpRQZYiQ9z0OOMPJIYZq6wYM2YMS0tLYzdu3GC3bt3S/M+eqLrPPI179O3YsZMJhe4M\n8GZACAPcGSBmHh4P7tdXJZiVlWX32K2hLVVq2Ysp1YDWjF2n80RDFaEPmNjDsmo0Y10WrBE7dcsw\nj9Wq+3x8fDTXoYhj0HcKsF27R1Fa+gOATNQPUjiGsrL6VdWiRSMhEnVD4zHzN2/exBNPPMHVX4Pw\niLEOFtagvdLRdJ7wUNaPnp8FoAugLFSa3SHBXl0WqFuGbRhNUgsXLsQbb7yBIUOGQCQSaW4fM2aM\nTQMjLaP/FGAB6ofr6Cak6uqOqKm5isbFEN26dbNv0IS3jLUzaqnGpxLXJSfXd564hvpxGy2YB6WT\n8PLqn88WyUOnWwZVOVqN0ST1zTff4H//+x9qamrg5PSgixIlKcehXezg7NwV5eXXAJ2vmxI4OblA\nKBwJkehhTTEE3zpJEPtpfP3G0n1Vpr5W40GM4UlJSP4gGa8ufhVV1VUt+uIPDAxE5e1KYDMALwCl\ngFKgbHHy0PceyVJkkCZIIfQRWr3Ksc0WZBg7H2it7hItQdekrKNhF/+yZcsZ0O7+9SkfBnzBPD1D\n2OHDh3V2+VurI7e98ek9NxcfYm9u3lNznwNLY09PT2dPeHiwdIAVNeqMrt1t3dIOCUVFRczFzUXn\nWpGLm0uLrqc1d43LFj8rzb0eHz4zlrDaqI5ly5axK1eutDiglqAkZTl9PzBFRUXMza0DA/4fA4oM\ntlNqiL1xG6QdO3byOmFx/Z63BNexW9ouiTHLY9+xYweDC5i7N5jYBWydntdsyRf/4cOHGXzuJ6iG\n//mAHT582KLY7V0gYez1uP7MWMpqhRPnzp1DTEwM/P39da5JUQm69dhqGa+vrDwubhokEgk++WQH\npNJ5ze51Ki4uxsWLFxEfn4iqqhP3916tR2LiInh4BKGmJlfznKR1sEeBhDaFQoGk15KAl4Dy+6fz\nln0M7ExO1nm9FndIKIPOKUOUWf5U9i6QaOsFGUaT1Mcff2yPONosQ4mkpZrrsQcAvXr1QEbGKZSX\nl+tNjnJCdtCcAAAgAElEQVT5XsTHz4Wzc3dUVakAXEZ9t4p3AfymqQykvn2ti7ECCWO/UJn7C5e+\nL2CPrh4IDg212t8pJCQEQhch1P9S1xdh3AOELkKEhIRY9Hz2LpBo8wUZpiy3Ll++zD7//HP2+eef\ns8uXL7doiWeJ1nq6z5wu5ubS1/Uc6MliY6cxsdiHeXg8wVxdPdmOHTtNiqt+j9VhBvQzq5M6Fxz1\n9Adj/Ijd0Oj65q5VMcbYti1bmr1fH3udOtuRsoOJ2omYuLOYubm7mb1PqvHpRnt3Em/u9fjwmbGE\n1a5J/etf/2Ljxo1jH3zwAfvggw/Y+PHj2a5du1ocoDlaa5KyZHyGqfQnmg4McGPAu/cLJvoxQNwk\nURlKcGJxDwaIbZJUrclRf2gZ40/sjb+UTRlH7+3mZtG1LFt/4TcUHXgEeDBXd1e2I2VHk2Oae98N\nFS3Yu5jI0Ovx5TNjLqslqfHjx7OKigrNnysqKtj48eMtj8wCrTVJ2XIlxRhjq1evbVTFt5YBPe7/\n94PXdHXt0OQitb64Dh8+rOlO0dDNgm+zpBhz3B9axvgRu74vw/T0dBbq5aX9W4umAq/h/hAPD4P3\nG3t+W33hm7JSKyoqYgcPHtT72lx1kTAHHz4zljD1e92k8fHOzs56/5u0TMP+JbE4HJ6eoRCLw63a\nrDUh4WW4uYkALAGQDeBJ1O9m1N3MKxIF6oyPbojLzW2UTlxjxoxBQsLLyM3NRlpaCnJzs6loopXZ\nK5cjKCAAiVFRCAoIwF65HIDxcfSBgYHIUasN3m/s+SUSCQYOHGj1a5vGxqXL5XIE9ArA9HnTEdAr\nAPIv5GY9ntiBsSz2ySefsOjoaLZp0ya2adMmNmHCBPbpp5+2NImapbWupBpY67dIfc/TuI+fVPqS\nyafssrKyeF1qboij/mbJGLexGzulZ+haVYNtW7c2e39Lytv1xWrKZ1OzEnoRDC+D4cUHKyFTV1m0\nkrINq53uY4yxixcvss8++4x99tln7I8//mhRYJZo7UnKGpob6d74B3rHjp3M1bUD8/AI1jm28XGO\n+uF31LgZ4zZ2Y6f0GDO+mbe5+409v6mJx1iz2MbmL5jPIARDx/ox97HPxmpeyyvQS2f/lGdg01OU\nfBm3boijft5bnKTOnz/Pjh8/3uT248ePswsXLlgemQUoSTXPkmtbTaqV9CQ5R/3wO2rcjPF7JWWM\nKRVyhp7f1MRj7sqmuW7qO1J2mPxcfO644qif9xZfk9qwYQN69erV5PZevXph/fr1Nj0FSczTMOiw\ncRfz5s6ba18D0N5TVVqaAaXyGKTSeSguLrZD9IQvNP35xGKEenoiXCy2Wn++5p4fgKZLeemsUiif\nq+90rlAomjyHudeINMd7ADiI+m7qCwHl80okLU1C8nvJEO8Rw+MTD4j3iA322rPVNTNinMHNvBUV\nFfD3929yu7+/P+7evWvToIh5WjrSXd80XxrV0TY1jIbPzMwEAIs3vBp7fu0Nv2fOnDG5o4K5G1s1\nxxvoph4aEorcq7k4c+YML5JQm20i2wyDK6m//vrL4IOqqqpsEgyxTEurBHWTHECjOtq2o2lpiIuJ\nwfJnn9WpwLOWxqsSncQDNJt4GjqNi/eI4fkvz2ZXP9rHu/3oBtyB3teQSCQIDg7mPCk0VBpGPRul\nt9KwzTJ0HvDvf/87e//991ldXZ3mtrq6OvbBBx+wlStXtvyEpBnompRpWnLevHEVIF2T4oa1p9ua\n+3ngosEsY+YXJ5j7dzPWTZ3rz0yTa20vgrm2d2WXLl0y+liuY7dUixvMLlu2DCtXrkRUVBR69+4N\nAMjOzkafPn2wZs0auyVRYrqWNOFsPM1XIpEgPz/fyhESe2k8RHCbTIZpcXFGH2fNBrPmnLqKmx6H\nyNGRJh9vyWf9qTFPYcrkKQb7VXJ5qk2nh+FFAAeB6nbVCBkUgk8/+hRx043/27VaxrLYjRs32JEj\nR9iRI0fYjRs3Wpw9LUErKW44auyOGjdj1om9Jasha62kjPX5sydTKge3bN3CxB5i1r5beyb2sH+Z\nuc5+LjHM2pflqJ93q3Wc6NatGyIiIhAREWHVaxQ//PADxo8fj969e+OPP/6w2vMS0tZpVkP3/6y9\nGjLGGhV+2pN2M0pLcUypxDyp/mo9W1MoFEYrBxUKBZKWJkH5vBIV0goon1dilnSWXeNtuHbm+rUr\n0A7U4UKLSW2RbOHRRx/Fli1bMHDgQK5CaJMUCgXOnDnDyRcGsQ9jLYyMmRYXh+zcXKSkpSE7N9ek\n04TaWpIkrc2UkvXMzEyo3dQ6x6jcVJoKR3uJmx6HzDOZcK12NamIpK3gLEn16NEDgYGBYIxxFUKb\nI5fvRUBAEKKiEhEQEAS5fC/XIREbsMZqqCX7glqaJK2pucrBhl/Y7t2792Ao4v1jWjIUsSV69+6N\nTz/61OTqxbbAYOHEvXv3mn1ghw4drB4MsR2FQnF/wu5WKJVRAApoYGErpm8/kr1okqRUigChELlq\ntclJsiXFC/oe23AaTZoghdBHCHWJGrIUGdLS0iBNlELkI4KqWAUBBGD/YlYZithS5haRtHYGk9Tk\nyZMhEAj0rnQEAgGOHDli9Mlnz56NO3fuNLk9KSkJERERZoZKWiIl5aP7E3Y3AlgAYBucnB5CZmYm\nxowZw3F0xBZaPHK9BSxJknK5/EHiKFFBliIzuaqtucc2/tIHgIBeAVA+p6yvprsOOH/hDGcnZwgF\nQtS61OKTjz7hNDlw+W/HNwLG8fm2mTNnYtmyZXj88ccNHpORkQE/Pz87RmU9ZWVl8PDw4DSG4uJi\nDBo0ElVVx/FgKPgoAFVwdRXh/ff/iZiYiU0ex4fYLeGocQNtN/bi4mIMGjoIVc9XaTpJuP0/N6T/\nnI6OHTs2OfbmzZvo1q0bOnbsaNZjAeDcuXOYPm86ymaXacq90Q4QKUV4df6rmDFjht7H8ZWjfmYK\nCgrQv39/o8cZXElpKy0tRW5uLqqrqzW3WbPgwZQ82bVrV6u9nj3l5+dzHnteXh5cXR9GVZX2pWwf\nAPGorh6PJUvCERs7tclvbnyI3RKOGjfQdmPPy8uDa0dXVHW5382mCyDqKEJVVZXOc+pbMfXq2cuk\nxzYQCoWouVsDXEd9gnqx/jGqQhU279iMJUuWONQqxlE/MwUFBSYdZzRJffXVV9i1axcKCwsRFBSE\n8+fPIzg4GLt27WpRgGlpaVi9ejXu3r2LxMREBAUF4eOPP27Rc5IH5+Xd3d01mxb19fYD7gJ4GYBE\n04zWkX4wSetiSk8+7XJyZRclUAhIE6TI+C3DrH5+DdepZr80G9XtqvVW/tHPAn8YTVK7du3C119/\njWeffRaff/45rl27huTk5Ba/cGRkJCIjI1v8POQBuXwvpNJ5APyhVF6FWNwFQClksm2QybZBKg2H\ni0t3lJX9F8Bb9x+1ByrV9TZd4kq4Z6jAQTtZ6HRlADRJpby83OhjG4ubHofgfsEIGRSC6sJqk5Ib\n4YbRJCUSieDq6goAUKlU6NmzJ65fv27zwIh5tMdtNKyWlMpwAN9AKp2C3Nxs5OZmIycnB2fPnsOC\nBYuhVq8C0BV1dQxpaUdpFDzhlLGqtuZWWwMHDjS7Iq6h3FuaIIVLBxfU3Ktp8+XefGQ0SXXp0gV/\n/fUXIiMjMXv2bHh6ejrk+c/WTt+4DSAAQHvN6byGfS+BgYFISloGtfoUgL5QqbKoHJ3wQnNVbcZW\nW5ZUxDUkRr6M6iBNGU1SW7duBQAsWLAAYWFhKCsrw/Dhw20eGDGP/utOuQAqmsyWMjQ/is7FE76z\nxR4ivozqIPqZVN33xx9/ICMjAwKBAKGhoRCJRLaOi5ipYaaUVBoOoCuUymtwc/OFQDClyWyplg5J\nJIRLtIeobTGapLZs2YLDhw8jKioKALB8+XI8/fTTmDdvns2DI+bRHrehXd3X+AdaO6EJhQFQq3PN\nGpJICCH2YjRJpaam4sCBA5riiTlz5mDixImUpHjK1N8y9c2PIoQQvjGapDp37ozq6mqdCj9fX1+b\nB0Zsj06bEEL4zmiS8vDwwLhx4zB06FAIBAL8/PPP6Nu3r2Y678qVK20eJCGEkLbJaJKKiorSXI8C\ngEGDBtk0IMJPXI7WJoS0XUaT1KRJk+wRB+Gxhk4WIlF9VaBMto02/hJC7MJgklq0aBE+/PBDREdH\n670/NTXVZkER2zF3RaTdyaJ+XxVt/CWE2I/BJPXGG28AAHbs2GG3YIhtNbciMpS8aOMvIYRLBsfH\nd+7cGQBQV1eHTp06wd/fH/7+/ujYsSONfHdA2iui0tIMKJXHIJXOg0KhaHasvO7GX4A2/hJC7Mlg\nkmqwaNEiCASCBw9wcsKiRYtsGhSxvoYVUX2HCaBhRZSZmWkweQEPNv6KxeHw9AyFWBxOG38JIXZj\ntHCitrZWpw2SSCSCWq22aVDE+gy1QgJg8HSev78/ANr4SwjhjtGVlI+PD44cOaL5c1paGry9vW0a\nFLE+QyuikJAQk07nSSQS6hJNCLE7oyupVatWYcmSJVi9ejUYY/Dz88O7775rj9iIlRlaERnq45ef\nn89xxISQts5okurevTu+/PJLVFRUAADat29v86CI7ehrhUSn8wghfGU0SalUKhw+fBh5eXmoqanR\n3D5//nybBkbsi/r4EUL4yGiSmjt3Ljw8PPD444/THClCCCF2ZTRJ3b59GzKZzB6xEEIIITqMVveF\nhITgv//9rz1iIQYoFAqcOXNGs3eJEELaCqMrqYyMDOzfvx/+/v46p/uod599UHNXQkhbZjRJffTR\nR/aIg+hhq+auNHaDEOIoDJ7uKy8vB1Bfcq7vf8T2DLUyysnJsfg5m+vTRwgxjk6/25fBldTixYuR\nkpKCyZMnQyAQ6DSVFQgEOl0oLLF+/XocO3YMIpEI3bt3xz//+U+4u7u36DlbG0OtjCxt7kpjNwhp\nGblcDmmiFCIfEVQlKshSZIibHsd1WK2awSSVkpICxhh2796Nrl27Wv2Fhw0bhiVLlsDJyQkbNmxA\nSkoKFi9ebPXXcWQNrYz0dYOwBI3dIMRyCoUC0kQplM8poeyiBAoBaYIUkaMj6efHhpqt7hMIBEhI\nSLDJCz/55JNwcqp/+eDgYBQWFtrkdRxdXNw05OZmIy0tBbm52S0qmqCxG4RYLicnByIfEdDl/g1d\nAKGPsEWn34lxRkvQ//a3vyErK8vYYS3y9ddfY8SIETZ9DUdmreauNHaDEMsFBgZCVaICGn6fLgTU\nJWr6Jc/GjFb3nT9/HgcOHIC/vz/EYrHmdlNK0GfPno07d+40uT0pKQkREREAgO3bt0MoFBocU0+s\ni/r0EWIZiUQCWYoM0gQphD5CqEvUkKXI6GfIxgTMyJjdvLw8vbc3zBpqiX379uHLL7/Erl27mm25\nlJGRAT8/vxa/HhfKysrg4eHBdRgWcdTYHTVugGLnijmxFxcX4+bNm+jWrRs6duxo48iMc9T3vaCg\nAP379zd6nMGVVHV1NeRyOW7cuIFHH30UU6dOhYuL0YWXyU6ePAmZTIbdu3eb1BPQFsUb9pCfn0+x\n25mjxg1Q7FwxJ/auXbviiSeesHFEpnPU972goMCk4wxmnddffx0uLi4YMGAATp48iatXr2LlypVW\nC3DNmjVQq9WIj48HAPTr1w9vvfWW1Z6fEEKI4zOYpK5du6a57jR16lTExsZa9YV//PFHqz5fW0Yd\nJAghrZXB6j7tU3vWPM1HrIs6SBBCWjOD2Sc7OxuhoaEAAMYYqqurERoaCsYYBAIBzp49a7cgiX7U\nQYJ/UlNT8e2339J4G0KsxGCSunz5sj3jIBagDhLciIiIwNq1azFkyJAm90VHR9t1O8WWLVtw48YN\nrF+/3m6vSYg9Gd3MS/iLOkjwS21tLdchENLqUJJyYNRBglv79+9HXFwc/vnPfyIsLAxbtmzB/v37\n8dxzz2mOeeedd/Dkk0+if//+mDBhAq5evar3ue7evYvExERER0cjLCwMM2bM0NxXVFSEhQsXYsiQ\nIYiMjMTnn38OAPjPf/6DHTt24NChQwgJCUFMTIzm+Llz5yIsLAxPPfUUvvrqK81zZWVlYcqUKejf\nvz+GDRuGd999V3PfokWLMGzYMAwcOBAzZ840GCsh9kQVEQ6OOkjoKi8vx/z5S/HLL+nw9fXFzp0b\nERYWZrPXy8rKwvjx4/Hrr7+ipqYGBw8ehEAgAACcOnUKGRkZ+PHHH+Hu7o7//e9/8PT01Ps8n376\nKbp06YLvvvsOfn5+OHfuHID668GJiYmIiopCcnIyCgoKMHv2bPTo0QPDhw9HYmJik9N9SUlJCAoK\nwqZNm3Dt2jXMnj0b3bt3R1hYGN555x28+OKLmDBhApRKJa5cuaJ53MiRI7Fu3Tq4uLhgw4YNWLJk\nCb799lubvXeEmIJWUq2AtXr7tQaxsbNw6BDD7dsHkJU1D5GR0TZtAOrr64vnn38eTk5OTTalu7i4\noKKiAteuXQNjDD169ECnTp30Po+LiwsUCgUKCgrg7Oys2Yl/4cIF3Lt3D3PnzoWzszMeeughxMbG\n4uDBg3qfp7CwEOfOncOSJUsgFAoRFBSE2NhYTbJxcXHBjRs3cPfuXYjFYvTt21fz2MmTJ0MsFkMo\nFOKVV15Bdna2Zq4cIVyhJEVajZqaGvz00wFUV8sA9AbwHOrqnkZaWprNXrNLly4G7xs8eDBmzJiB\nVatW4cknn8Sbb76JiooKFBQUICQkBCEhIZoKWqlUiu7du+O1115DVFQUdu7cCaC+Ldnt27cxaNAg\nDBo0CAMHDkRKSgpKSkr0vmZRURG8vLx0+mx27doVRUVFAOpPP16/fh3PPPMMYmNjcfz4cQBAXV0d\nNmzYgKioKAwYMACjR4+GQCDA3bt3rfE2EWIxOt1HWg1nZ2e4uLiitvY2gEAADE5OBTadJN1was+Q\nGTNmYMaMGSgpKcGiRYsgk8mwcOFCZGZm6hzXvn17vP7665g5cyYqKyvxwgsvoG/fvvDz88NDDz2E\nw4cPmxRP586dUVpaisrKSrRr1w5AffuZzp07AwC6d++OjRs3AgAOHz6MhQsXIj09HT/88AOOHTuG\nzz77DF27dkVZWRkGDhxo7ttBiNXRSoq0GgKBAKtWvQWxOBLAe3B1fQ5duxZj4sSJnMRz4cIFZGVl\noaamBm5ubnB1ddXMUGvs+PHjuHHjBoD6hOXs7AwnJyf07dsX7du3x0cffYTq6mrU1tbiypUruHDh\nAgCgU6dOyMvL00zO7tKlC0JCQvD+++9DpVIhOzsbX3/9teY9OHDggGYV5uHhAYFAACcnJ1RWVkIk\nEsHT0xOVlZXYuHGj0QRMiD3QSoq0Kq+/vhidO/sgI+MCHnooGK+8slOzorAWU7+8y8vL8c9//hO3\nbt2Cq6srhg0bBqlUqvfYnJwcvP322ygpKUGHDh3w/PPPY9CgQQDqp2SvW7cOo0ePhlqtxsMPP4xF\nixYBAJ5++mkcOHAAYWFheOihh7Bv3z5s3LgR//jHPzB8+HB4eXlh0aJFGDx4MID6isB169ahqqoK\n/v7+SE5OhkgkQkxMDE6dOoURI0agQ4cOWLRoEfbupe4lhHtGR3XwQUZGhkkt3fnIUTsUA44bu6PG\nDVDsXLFm7Pbupemo77up3+t0uo8QQqxELpcjoFcAop6NQkCvAMi/kHMdksOjJEUIIVagUCggTZRC\n+ZwSpbNKoXxOCWmCFAqFguvQHBolKUIIsYBCocCZM2c0SSgnJwciHxHQsCuhCyD0Edp0n15bQEmK\nEELMpO+0XmBgIFQlKqDw/kGFgLpETb00W4iq+wghxAzap/WUXZRAISBNkCL3ai5kKTJIE6QQ+gih\nLlFDliKjTjAtREmqjaDpvYRYR8NpPWUXZf0NWqf14qbHIXJ0JP2sWRGd7msDaHovIdZj7LQe9dK0\nLkpSrZz29N7S0gwolccglc6jiiNCLCSRSCBLkUG8RwzPf3lCvEdMp/VsiJJUK9cwvRdoOr2XWF9q\naqrBrhJ8YI340tPTMXLkSCtF5Jjipsch92ou0r5MQ+7VXMRNj+M6pFaLklQrR9N7rS8iIgK//vqr\n3vuio6Mhk8nsHJHprBUf9fWj03r2QkmqlaPpvfbDl/HxfO50VldXx3UIxMFQkmoD4uKmITc3G2lp\nKcjNzUZc3DSuQ2oVrDk+fuzYsThx4oTmz7W1tRgyZAguX74MADh37hymT5+OgQMHIiYmBunp6Zpj\nZ86cieTkZMTFxSE4OBi3bt3Cvn37EBkZidDQUERGRuLf//63Jmbt+K5cuYL4+HiEhYVh2LBhmjlW\nKpUKa9euxfDhwzFixAi88847UKvVemO/du0aZs6ciejoaERHR+Po0aOa+5YvX4633noLc+bMQUhI\nCE6fPm3u20zaOCpBbyMkEkmbWD2Vl5dj6fz5SP/lF/j6+mLjzp0OMT5+/PjxSE1Nxf/93/8BqO9W\n7uPjg969e+P27dtISEjAhg0bMHz4cPz6669YsGABfvjhB3h7ewOov9b00Ucf4eGHH0ZlZSXWrl2L\nffv2ISAgAHfu3EFpaanmtRriq6iowOzZs/HSSy9hx44dqKmp0STR7du348KFCzhw4AAAYO7cudi+\nfTsWLlyoE3dNTQ3mzp2LqVOn4p133kFeXh7mzZuHffv2aU4pHzx4EB999BGCg4OhUqms9M6TtoKz\nldSHH36ICRMmICYmBlIp9bci1jErNhbs0CEcuH0b87KyEB0Z6RDj48ePH4+jR49qvsT//e9/Y9y4\ncQDqZ0CNGjUKw4cPBwAMGTIEffr00Vl5TZo0CT179oSTkxOcnZ3h7OyMP//8E9XV1ejUqRN69uzZ\n5DWPHz+Ozp07Y9asWRCJRGjXrp1mnPy///1vvPLKK/D29oa3tzfmz5+P7777rslznDt3DpWVlZgz\nZw6cnZ0xePBghIeHa1ZuADB69GgEBwcDQJP3iBBjOEtSL730Eg4cOIBvv/0Wo0aNwpYtW7gKhbQS\nNTU1OPDTT5BVV98fHg88XVfnEOPju3fvjl69euGXX35BVVUVjh49iujoaAD1oxi+//57nRHyZ8+e\nxZ07d/TGIRaLkZycDLlcjmHDhiExMRH/+9//msRXUFCAbt266Y29qKhIZ/yD9gj6xsf5+fnp3Nb4\n2ObeI0KM4ex0n/ZIb6VSaXBiKSGmcnZ2hquLC27X1t4fHg8UODk5xPh4oP661JEjR+Dl5YVHHnlE\nk0D8/PwQExODt99+2+Q4hg4diqFDh0KlUiE5ORlvvvkmdu/erXOMn58fDh06pPf5fH19kZeXp1mB\n5efna0bQa+vcuTMKCgp0bsvPz8fDDz9sMDZCzMFpZkhOTsaoUaOQmpra5Fw3IeYSCAR4a9UqRIrF\neA/Ac66uKO7a1SHGxwPAuHHj8Pvvv0Mul2P8+PGa2ydMmICjR4/i1KlTqKurQ3V1NdLT03H79m29\nz1NcXIwjR45AqVTCxcUF7dq105soRo0aBYVCgV27dkGlUqGiogJZWfVbFcaOHYvt27ejpKQEJSUl\n2LZtm973sV+/fhCLxfjoo49QW1uL06dP4/jx4zrxE9ISNl1JzZ49W+eURIOkpCREREQgKSkJSUlJ\n2LlzJ3bv3o0FCxYYfK78/HxbhmozZWVlFLsdxc2cCbGHB86fOYMefn5YNWsW7t27h3v37lntNerq\n6lBcXIy7d+9CpVLpvEfat+Xm5mLr1q0oLCyESCTCwIEDMXbs2Gbf08ceewznzp3D8uXLdY57++23\nsWnTJiQlJcHZ2RlBQUFISkpCbW0t1Go17t27pzm+pKQEKSkpeO211yAQCNCrVy+8+uqryM/PbxLz\nu+++i82bN2PTpk0QiUSYMmUKOnXqhJiYGBQVFWH8+PEQCAQYNWoUJk6ciPz8fNy5cwe1tbWa53j7\n7beRnJyM7du3QyKRYNmyZRCJRMjPz0dlZaVDfI4cIUZDHDl2U/BifHxBQQHmzJmD1NRUvffT+Hhu\nOGrsjho3QLFzhWK3P96Pj8/NzdX8d1paGnr06MFVKIQQQniKs8KJjRs34vr163ByckLXrl2xatUq\nrkIhhBDCU5wlqU2bNnH10oQQQhwE1X0TQgjhLUpShBBCeIuSFCGEEN6iJEUIIYS3KEkRQgjhLUpS\nhFiRLcbHFxQUIDQ01OJhhuY8vqWvRYi1UZIixEz2Hh/v5+eHs2fPWtyo1ZzHt/S1CLE2SlKEWAlX\n4+Np1UNaM0pShFjIXuPj8/LyEBQUhLq6OgD6x8XfunULM2bMQP/+/REfH4+3334bS5cuBQC9j//w\nww8RFxeH0NBQSKVSTQPexseWlpZi+fLlGD58OMLCwjB//nwAwF9//YXExEQMGTIEYWFhSExM1NtM\nmpCWoiRFWpXy8nLMT5qPLgFd0G9gP5w+fdqmr5eVlYXu3bvj119/xdy5cwFA7/j4jIwMfPDBB+jQ\noYPe52kYH99Ae3y89nM2SE1NxZo1a3D27Fn4+flhyZIl6Nev/u/7yiuv4LvvvtN5TOPHHzx4EO++\n+y5+++03qFQqfPLJJ3qPXbp0Kaqrq/H999/jl19+waxZswDUd4KfMmUKTpw4gWPHjsHNzY26yBCb\noCRFWpXY52Nx6M9DuB19G1ndsxD5jOOPj9dHe1y8QqHAxYsXsXDhQri4uKB///6IiIhoNu7Jkyej\ne/fuEIlEeOaZZ3D58uUmxxQVFeHUqVN4++234e7uDmdnZwwYMAAA0KFDB0RFRWnGzickJOD8+fPN\nviYhlqAkRVqNmpoa/HToJ1SPqwYkAPoCdT0df3y8sdctKiqCl5cXXF1dNbc1HunemHayFIvFqKys\nbHJMYWEhvLy84O7u3uS+qqoqvPnmm4iIiMCAAQMwY8YMlJeX0/UxYnWcNZglxNqcnZ3hInJBbXkt\n4A2AAU7ljj8+3tjrSiQSlJaWorq6WpOoCgoKWlyh5+fnh9LSUpSXlzdJVJ988glycnLw9ddfw8fH\nB2BXJrgAAA2sSURBVNnZ2Zg0aRIYY1QZSKyKVlKk1RAIBFj11iqI5WLgZ8D1O1d0dXH88fFA8xV8\nXbt2RZ8+fbB582ao1WpkZmbi2LFjJj++sYZjJRIJRowYgbfeegt//fUX1Go1fv/9dwBARUUF3Nzc\n4O7ujnv37mHz5s0mPz8h5qAkRVqV15e+jq3rtuKV3q/grWlv4fdffke7du2s+hqmrhTKy8uxcuVK\nDBo0CKNHj4a3t3ezG30lEgn+9re/4dy5cxg7dqzB19T3+u+99x4yMzMxePBgbNq0CWPHjtW5Rmbs\n8YZea/369XBxccEzzzyDYcOGYdeuXQCAF198EUqlEmFhYZg+fTpGjhzZ7HMSYilejI83hsbHc8NR\nY3fUuAHrxZ6UlISePXtqSsbtgd53bjhq7LwfH08IsZ4LFy7g5s2bYIzh5MmTOHr0KCIjI7kOi5AW\no8IJQlqBO3fuYMGCBSgtLYWvry9WrVqFoKAgrsMipMUoSRHSCoSHhyM8PJzrMAixOjrdRwghhLco\nSRFCCOEtSlKEEEJ4i5IUIYQQ3qIkRQghhLc4T1KffPIJgoKCNPNsCCGEkAacJqnCwkL8/PPPDrlb\nmhBCiO1xmqTeeecdvPbaa1yGQAghhMc4S1JHjhyBn58fHnvsMa5CIIQQwnM27Tgxe/Zs3Llzp8nt\nr776KlJSUnRGVjtAn1tCCCF2xkkX9D///BOzZ8+Gm5sbGGO4ffs2fH198dVXX6Fjx45Njs/IyLB3\niIQQQmzMlC7ovBjVERERgf3798PLy4vrUAghhPAI5yXoQP2QNR7kSkIIITzDi5UUIYQQog8vVlLm\ncLTNvx9++CEmTJiAmJgYSKVSKBQKrkMy2fr16/HMM89g4sSJWLBgAcrLy7kOyWQ//PADxo8fj969\ne+OPP/7gOhyTnDx5Ek8//TSeeuop7Ny5k+twTLZixQo8+eSTiI6O5joUsxQWFuKFF17AuHHjEB0d\njV27dnEdkslUKhViY2MRExOD6OhobNmyheuQzFZXV4dJkyYhMTGx+QOZAykoKGDx8fEsPDyc3b17\nl+twTFJeXq757127drE333yTw2jM8/PPP7Pa2lrGGGPvvfce27BhA8cRme7atWvs+vXrbObMmezi\nxYtch2NUbW0ti4yMZLdu3WIqlYpNmDCBXb16leuwTHLmzBl26dIlNn78eK5DMUtRURG7dOkSY6z+\n53TMmDEO854zxlhlZSVjjLGamhoWGxvLzp8/z3FE5vn000/Z4sWLWUJCQrPHOdRKyhE3/7Zv317z\n30qlEk5OjvOWP/nkk5p4g4ODUVhYyHFEpuvRowcCAwMd5lpnVlYWAgIC4O/vD6FQiHHjxuHIkSNc\nh2WSAQMGwNPTk+swzCaRSNC7d28A9T+nPXv2RFFREcdRmU4sFgOoX1XV1NRwHI15CgsLceLECcTG\nxho91mEm8zry5t/k5GR899138PDwcKhTCtq+/vprjBs3juswWq3bt2/Dz89P82dfX19cuHCBw4ja\nllu3biE7Oxt9+/blOhST1dXVYfLkybhx4waef/55h4q9YcFRVlZm9FheJSlH3fxrKO6kpCREREQg\nKSkJSUlJ2LlzJ3bv3o0FCxZwEKV+xmIHgO3bt0MoFPLumoMpsRNiTEVFBRYuXIgVK1bonPngOycn\nJ3z77bcoLy/HvHnzcPXqVfTq1YvrsIw6fvw4OnXqhN69e+P06dNGj+dVkvr000/13v7nn38iLy8P\nEydO1Gz+nTJlisHNv/ZmKO7GoqOjMWfOHF4lKWOx79u3DydOnODlCtDU990R+Pr6Ij8/X/Pn27dv\no3PnzhxG1DbU1NRg4cKFmDhxIiIjI7kOxyLu7u4ICwvDf/7zH4dIUmfPnsXRo0dx4sQJVFdXo6Ki\nAq+99hrWr1+v93iHuEDy6KOP4ueff8aRI0dw9OhR+Pr6Yv/+/bxIUMbk5uZq/jstLQ09evTgMBrz\nnDx5EjKZDNu3b4dIJOI6HIvxadVtyBNPPIEbN24gLy8PKpUKBw8exOjRo7kOy2SO8B7rs2LFCvTq\n1Qsvvvgi16GYpaSkRHOqrKqqCr/88ovDfLf83//9H44fP44jR47g/fffR1hYmMEEBfBsJWUqR9r8\nu3HjRly/fh1OTk7o2rUrVq1axXVIJluzZg3UajXi4+MBAP369cNbb73FbVAmSktLw+rVq3H37l0k\nJiYiKCgIH3/8MddhGeTs7Iy///3viI+PB2MMU6dORc+ePbkOyySLFy/G6dOnce/ePYwaNQoLFizA\nlClTuA7LqIyMDKSmpuLRRx9FTEwMBAIBkpKSMGLECK5DM0qhUGDZsmWoq6tDXV0dxo4di5EjR3Id\nlk3QZl5CCCG85RCn+wghhLRNlKQIIYTwFiUpQgghvEVJihBCCG9RkiKEEMJblKQIIYTwFiUp4nB6\n9+6NSZMmITo6Gq+++iqqq6v1HpeQkGDReJGioiIsWrTI4vgiIiL0jpKprKzEm2++iaioKEyZMgUv\nvPACsrKyLH4dPsjOzsaJEyf03nfv3j288MILCAkJwZo1a+wcGWktHHIzL2nbxGIx9u/fDwBYsmQJ\n5HI5Zs2apXMMYwwpKSkWPX/nzp3x4YcfWhyfQCDQe/vKlSvRrVs3/PTTTwCAvLw8XL161eLX4YPL\nly/j4sWLejeSurq64tVXX8WVK1dw5coVDqIjrQElKeLQBgwYoOntKJVK0a9fP1y6dAkpKSmYMWMG\n9u3bh4qKCrz88svo378/MjMz4evrq2n1dOPGDfzjH/9ASUkJnJ2d8eGHH8LJyQmJiYlITU3F/v37\n8dNPP6GsrAxFRUWIjo7G/PnzAQCvvPIKCgsLoVKp8MILL2jGDujbH3/z5k1kZWVh48aNmtv8/f3h\n7+8PoL4P4b59+wAAU6dOxYsvvoi8vDy89NJLCA4OxtmzZ/HEE09gypQp2LRpE0pKSrBhwwY88cQT\n2LJlC27cuIEbN27g7t27eOmllzSxvPvuuzh16hQEAgESExMxduxYpKenY/PmzfD29saVK1fQp08f\nvPfeewCAP/74A+vWrUNlZSW8vb2xbt06dOrUCTNnzkS/fv1w+vRplJWVYe3atejbty82bdqE6upq\nnD17FnPmzMEzzzyj+fuJxWKEhobqtAYjxGy2HGpFiC0EBwczxhhTq9Vs7ty5TC6Xs1u3brGgoCCd\nwW8RERHs7t277NatW+zxxx9n2dnZjDHGFi1axA4cOMAYYyw2NpalpaUxxhirrq5mVVVV7NatW5oB\nfvv27WPDhg1jpaWlrKqqio0fP14zRLG0tJQxxjS337t3jzHG9A7lPHLkCJs/f77ev8/FixdZdHQ0\nq6qqYhUVFWzcuHHs8uXLmrivXLnCGGNs0qRJbMWKFYwxxtLS0ti8efMYY4xt3ryZTZw4kVVXV7OS\nkhI2cuRIVlRUxA4fPszi4+MZY4zduXOHjRo1iikUCnb69Gk2YMAAdvv2bVZXV8emTZvGMjIymFqt\nZtOmTWMlJSWMMcYOHjzIli9fzhhjbMaMGWzdunWMMcaOHz/OZs2apXl/Vq9e3ey/lynHEGIIraSI\nw6mursakSZMAAP3798fUqVNx+/Zt+Pv768zUYVorGn9/f80ssscffxx5eXmoqKhAUVGRppGroSa6\nQ4cO1Qz1i4qKQkZGBh5//HF89tlnSEtLA1A/xC03N9eimT4ZGRmIioqCq6ur5jV+//13hIeHw9/f\nX9PZ+pFHHsGQIUMA1Ddd1u6aPnr0aIhEIohEIgwePBjnz59HRkaGZgZYx44dMWjQIFy4cAHt27dH\n3759NV3Wg4KCkJeXBw8PD1y5ckXTP7Curk6nE/uYMWMAAH369NF5bUJsiZIUcThubm6aa1LaGiaV\n6qOdgJydnTXFFsyE1pWNrzEJBAKkp6fjt99+w1dffQWRSISZM2caLOAA6hNMdnY2GGMGr1kZi9vJ\nyUnzZycnp//f3t26qBKFYQB/RBRhQBAxGAyCIviFyIB/gckwDIhxwCpmk5gMGh1Mgt1mM43BYNKg\nwWg0WBxQFD/Q2bCs7L26Xi/sZUfu80vDcGbOOXPCyzu88P7SjfXzOw3DuNsB+vNebTbb9dpqteJ8\nPsMwDASDQXQ6nYdr+X1uon+J1X30cp4JLM8QBAFer/eaDR2PR+z3+5txw+EQ6/Ua+/0emqYhmUxi\ns9nA6XTCbrdjPp9jOp0+nMvn8yEajUJV1eu9xWKBwWAAURShaRoOhwN2ux00TYMoin+1l36/j+Px\nCF3XMRqNEIvFIIoier0eLpcLVqsVxuPxw0zP7/dD13VMJhMA772Wvirs+DgDQRCeqqD8rjOj/w8z\nKXo5z2Yiz4yr1+uoVCpQVRU2mw2NRuPmuXg8jmKxiOVyCUmSEIlErhlHJpOB3+9HIpH447zVahW1\nWg3pdBoOhwMulwulUgnhcBiyLCObzQIAcrnc9Rfcs0KhEBRFga7rKBQK8Hg8SKfTmEwmkCQJFosF\npVIJbrcb8/n87nf62H+1WsVms8HlcoGiKAgEAnezSQBIpVJotVqQZfmmcAJ4L8ffbrc4nU7o9/to\nt9sv04KEzIGtOoge6Ha7mM1mKJfLP72ULzWbTQiCgHw+/9NLIfp2/N1HRESmxUyKiIhMi5kUERGZ\nFoMUERGZFoMUERGZFoMUERGZFoMUERGZFoMUERGZ1hsJLAzXWFxlxAAAAABJRU5ErkJggg==\n",
"text/plain": [
""
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"with plt.style.context('seaborn-whitegrid'):\n",
" plt.figure(figsize=(6, 4))\n",
" for lab, col in zip(('Iris-setosa', 'Iris-versicolor', 'Iris-virginica'), \n",
" ('blue', 'red', 'green')):\n",
" plt.scatter(Y[y==lab, 0],\n",
" Y[y==lab, 1],\n",
" label=lab,\n",
" c=col)\n",
" plt.xlabel('Principal Component 1')\n",
" plt.ylabel('Principal Component 2')\n",
" plt.legend(loc='lower center')\n",
" plt.tight_layout()\n",
" plt.show()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"
\n",
"
\n",
""
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"
\n",
"
\n",
" "
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Shortcut - PCA in scikit-learn"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"[[back to top](#Sections)]"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"For educational purposes, we went a long way to apply the PCA to the Iris dataset. But luckily, there is already implementation in scikit-learn. "
]
},
{
"cell_type": "code",
"execution_count": 100,
"metadata": {
"collapsed": false
},
"outputs": [],
"source": [
"from sklearn.decomposition import PCA as sklearnPCA\n",
"sklearn_pca = sklearnPCA(n_components=2)\n",
"Y_sklearn = sklearn_pca.fit_transform(X_std)"
]
},
{
"cell_type": "code",
"execution_count": 101,
"metadata": {
"collapsed": false
},
"outputs": [
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAakAAAEbCAYAAABgLnslAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzs3Xtc03X7P/DXgA2mHASdiKSQWuGdKeABzSMIViqKB1JK\nS1kJmof4qqXm3Z2p3WYa5RlrdWf+XHbQklvLwuNtBzFE0ZQ79RZUDjJBicNgA96/P5C5wcYObPt8\nBtfz8bgfd26fbZdz7OL9+Vzv6xIwxhgIIYQQHnLiOgBCCCHEEEpShBBCeIuSFCGEEN6iJEUIIYS3\nKEkRQgjhLUpShBBCeMuFyxdXqVR4/vnnoVarUVtbi6eeegrz58/nMiRCCCE8IuB6n5RSqYRYLEZt\nbS3i4uKwcuVK9O3bl8uQCCGE8ATnp/vEYjGA+lVVTU0Nx9EQQgjhE86TVF1dHWJiYjB06FAMHTqU\nVlGEEEI0OE9STk5O+Pbbb3Hy5EmcP38eV69e5TokQgghPMFp4YQ2d3d3hIWF4T//+Q969eqlc19G\nRgZHURFCCLGV/v37Gz2G0yRVUlICoVAIDw8PVFVV4ZdffsGcOXP0HmvKX4aP8vPz0bVrV67DsIij\nxu6ocQMUO1codvszdfHBaZJSKBRYtmwZ6urqUFdXh7Fjx2LkyJFchkQIIYRHOE1Sjz32GPbv389l\nCIQQQniM88IJQgghxBBKUoQQQniLkhQhhBDeoiRFCCGEtyhJEWJnCoUCZ86cgUKh4DoUQniPkhQh\ndrRXLkdQQAASo6IQFBCAvXI51yERwmuUpAixE4VCgXlSKY4plcgoLcUxpRLzpFJaURHSDEpShNhJ\nTk4OAkUiNLRQ7gsgQChETk4Oh1ERwm+UpAixk8DAQOSoVMi6/+csALlqNQIDAzmMihB+oyRFiJ1I\nJBJsk8kQLhYj1NMT4WIxtslkkEgkXIdGCG/xpgs6IW3BtLg4RERG1p/6CwykBEWIEZSkCLEziURC\nyYkQE9HpPkIIIbxFSYoQQghvUZIihBDCW5SkiFmopQ8hxJ4oSRGTyeV7ERAQhKioRAQEBEEu38t1\nSISQVo6SFDGJQqGAVDoPSuUxlJZmQKk8Bql0Hq2oCCE2RUmKmCQnJwciUSCg1dRHKAyglj6EEJui\nJEVMEhgYCJUqB9Bq6qNW51JLH0KITVGSIiaRSCSQybZBLA6Hp2coxOJwyGTbaFMqIcSmqOMEMVlc\n3DRERkZQSx9CiN1QkiJmoZY+hBB7otN9hBBCeIuSFCGEEN7i9HRfYWEhXnvtNRQXF8PJyQmxsbF4\n4YUXuAyJEEIIj3CapJydnbF8+XL07t0bFRUVmDx5MoYOHYqePXtyGRYhZlMoFFRQQogNcHq6TyKR\noHfv3gCA9u3bo2fPnigqKuIyJELMtlcuR1BAABKjohAUEIC9cjnXIRHSavDmmtStW7eQnZ2Nvn37\nGj+YEJ5QKBSYJ5XimFKJjNJSHFMqMU8qpXZRhFgJL0rQKyoqsHDhQqxYsQLt27fXe0x+fr6do7KO\nsrIyit3O7Bn3uXPnEODiotUsCuju4oIzZ84gODjY7Odz1PccoNi54sixm4LzJFVTU4OFCxdi4sSJ\niIyMNHhc165d7RiV9eTn51PsdmbPuIVCIXJrapCF+gSVBeBGTQ0GDhxo0bUpR33PAYqdK44ae0FB\ngUnHcX66b8WKFejVqxdefPFFrkMhxGwSiQTbZDKEi8UI9fREuFiMbTIZFU8QYiWcrqQyMjKQmpqK\nRx99FDExMRAIBEhKSsKIESO4DIsQg/RV8U2Li0NEZCRV9xFiA5wmqf79++Py5ctchkCIyfbK5Zgn\nlSJQJEKOSoVtMhmmxcUBoHZRhNgK56f7CHEEramKT6FQ4MyZMw4ZO2l7KEkRYoKcnBwEikQ6VXwB\nQiFvhz4aSkS0p4s4GkpShJggMDAQOSqV1shHIFet5uXQR0OJiE+rQVrNEVNRkiLEBI5SxddcIuLL\nalAulyOgVwCino1CQK8AyL+g1RwxjPN9UoQ4Ckeo4tMkIqUSgG4i0l4NNuzpsvdqUKFQQJoohfI5\nJZRdlEAhIE2QInJ0JC/fT8I9WkkRYgaJRGLxRl17aO60JB9Wg5mZmXDycgK63L+hCyD04e+1PcI9\nWkkR0opoEpFUigChELlqtU4i4nI1uFcuR2J8PCprq4BC1CeqQkBdws9re4QfKEkRwiPFxcXIy8vT\nSSDmjAFRKBTo0asXTmVkoLy8XO9juNjT1XCt7ERVFf4AEP8xUOUBiGvEkO3k37U9wh90uo8QC9ii\nOm2vXI6RgwbpVOWZUzKuXZDQf3B/XL12lTdf/tpFG3EAbtQAj6ra49uvv0Xc9DiuwyM8RkmKEDPZ\nYq9Rw0rjeFWVTlVeYny8SSXj2gUJpbNKoXxOCWkCfzYbN75WVgDgTl0dQkJCuAyLOABKUoSYwVZ7\njfSVhz/k5ARfZ2eTSsZzcnIg8hHxtiCBD0UbxDHRNSlCzNBciXdLvnD1lYffqqtDHWMmlYwHBgZC\nVaLidUGCI5TwE/6hJEWIGWy116hhpTEqPh6BIpGmKg+AwUq9xo+XpcggTZBC6COEukQNWQr/VirU\niJeYi5IUIWYwVuLdEtPi4vC3Pn1QVVWls9IwdfURNz0OkaO5XamYU4lIiCkoSRFiJlueturYsWOT\nKavmrD64XKl8t38/Vi5dqneUCSGWoiRFiAX4ctqKLysXhUKBN5YswfGqKvRVKpGF+tOUEZHU7oi0\nDFX3EeIA9O3L4tPYjZycHAQKhZw3ryWtDyUpQnhOXzLi09gN4H5BiVrtEKNMiGOhJEUIjxlKRpmZ\nmbwYu9FAIpFg7caNtA+KWB1dkyKExwztywLA+diNxibGxGBqbCwvrpGR1oNWUoTwmKHRGyEhIbzs\n4GDqKBOazEtMRUmKEB5rrp3QtLg4ZOfmIiUtDdm5uQ5T7k2TeYk56HQfITzXmtoJ0WReYi5aSbVS\ndDqlddF3Go1PJeim4nsjXMI/RpOUWq1ucltJSYnVAlixYgWefPJJREdHW+052zq5fC8CAoIQFZWI\ngIAgyOV7uQ6JNMOSXyj4VoJuKp1GuAAvG+ESfjGYpH777TeMGDECw4YNQ3x8PG7duqW5TyqVWi2A\nyZMnQ3a/kSZpOYVCAal0HpTKYygtzYBSeQxS6Tzef3m1VY1XQ5s+/NCkfyt9oz0cYfNsQyNc8R4x\nPP/lCfEeMS8b4RL+MJik3nvvPchkMpw+fRrPPvss4uPjce7cOQAAY8xqAQwYMACenp5We762Licn\nByJRIKD19SUUBvD+y6st0rca2rx+PR7t3t3oqTt9VX/XVSrcvXuX97+QxE2PQ+7VXKR9mYbcq7k0\nmZc0y2CSUqvVeOSRRwAATz/9NLZu3YrXX38daWlpEAgEdguQmCcwMBAqVQ6g9fWlVufS6RQe0rca\negzA1qoqo6fuGlf9DRMKwerqsPzZZx3i+pSppeqEGExSLi4uOj8kjzzyCD777DNs3ryZfiu3I3Ov\nV0gkEshk2yAWh8PTMxRicThksm30ZcBDevdAAYiCaafuGkrQ1331FYQuLjipUjnE9Skq6iHmMFiC\nvmTJEhQXF+t8uXXp0gW7d+/G7t277RKctvz8fLu/pjWUlZVZHPv+/d9hyZI3IBQGQq3OwcaNaxET\nM9Ho40aOHI7Tp4/j5s2b6NatGzp27GhRDC2JnUuOFPeaDRswavFi+FRVoQTAdgAFqO8m4ebmZtLf\no6amBgEuLjorsu4uLjhz5gyCg4NtF3wjprzv3+3fjzeWLEGgUIgctRprN27ExJgYk1+juLhY53Nt\nLY70mWnMkWM3CeOBmzdvsvHjxxu8//fff7djNNaVl5dn0eOKioqYWOzDgPMMYAw4z8RiH1ZUVKT3\n2PT0dL33tYSlsXPN0eIuKipia1evZh3c3FiwhwfzEYvZF3v2mPV4H7GYna//oLDzAPMRi63+eTDG\n2Pve0jj37NnDxJ5i5hXoxcSeYrZHbvp7ZIyjfWa0OWrspn6vc75PavHixZg+fTquX7+OUaNG4Ztv\nvuE6JF4wtQCCys0dn0QiwYqVK/HnjRtY+8UXZnWPaJgntS45mXctkhprSUWi9ibg0lmlUD6nhDSB\nv6c0ifVw3nFi48aNXIfAS7oFEPUtRBsXQGiXmyuV9cdIpeGIjIzg3RcUMU4ikSA4ONjkf7u9cjnm\nSaWaSbjrkpMRHBrK264U7u7uuFpVheMARsG8prgNm4CVXeob7WpvAubj35VYj9GV1Pfff2/SbcS6\nTCmAoHLztqNxsYG+8vVlSUm8TVByuRz9B/eHupMTwl2AQLGbWSs+2gTcdhldSe3cuRPPPPOM0duI\n9cXFTUNkZITBnm2mrLaI42u8Ytomk8G7Y0d0c3LSe+qMb0lK+1QdugAoBAp3M2Smn0Xv3r1Neo6G\nTcDSBCmEPkKoS9S0CbiNMJikTpw4gZMnT+L27dtYs2aN5vby8nI4OzvbJThS/8Np6AexYbUllYZD\nKAyAWp1L5eatjPaKqa9SiSwAw158Ec7OzlBVVfFqnpQh+k7VuXZyRXl5uVnPEzc9DpGjrd9oV6FQ\n4Ny5cxAKhfSzw0MGk5Svry/69OmDo0eP4vHHH9fc3r59eyxfvtwuwRHjjK22tDVcZNd3XHP3Ee40\nHnroB6BWrcYptRqXUX9txwfAXZ4WSwCASqVC5e1K4DqAh9GiU3XN/dJmCblcDmmiFC7eLqi5WwNZ\niow6YPCNsfI/lUrV4lLDlmqLJejWtmfPF0ws9mFeXqFMLPZhe/Z8YfQ+vsRuLkeNm7GmsTcu2/5/\nAOt1/78ZwIoA9mj79uzw4cMcRfyAvvd9/vz5DEIw+IDBBUzoLbR6+bilioqKmNhTzJAIhrfAkAgm\n9rR/6X5LOern3dTvdaPXpLKysrBlyxbk5+ejpqYGjDEIBAIcOXLEHjmUWEFzVYAAmr2PcEvT/kgq\nRYBQiOsqFVhdHbJUKvRF/cbfO3V1CAkJ4TpUjYZVuUqlwpaULYAUmmtR6o/VOHb8GIYOHcp1mFQx\n6CCMJqk33ngDy5cvR58+feDkxPm2KmKBhirA+iQEaFcBHj78E5RKH+irEPT396fTgDzQeOjh0bQ0\nTdLKVat5dZpPu8jjv5WVgCd0ZkfBC7h69SovkpROxWBDEqWKQd4xmqQ8PDwwcuRIe8RCrKwhwbi7\nu+utAnR3d8fate8BEDS5LzAwEF9++TWWLl0Jkai+ilAm24a4uGnc/YXaMO1rMQ1JKzMzEwCarKLs\n/YtFw+tVVlbqFHl8C2BSKXSSAP4CBg0aZPOYTKFdMejSwQU192qoYpCPjJ0PfO+999i6devY2bNn\n2cWLFzX/sye6JmW+xteZ5s9fyMRiH+bpGaK57pSens68vEIZ8AUDfBgQwoB2bPXqtayoqIi5uXmb\n1JaJbxz1HD1jpsf+xZ49zEcsZqFeXjptlAzdbivarYpE7UUsUCzWXC9jAOsodGFwAUNHMAjB5i+Y\nb9N4LFFUVMQOHjzoEJ9tfRz18261a1Lnz58HAFy8eFFzm0AgwK5du2yXOUmL6LsGJZOFIyPjFMrL\nyzW/YSsUivsrrN4AsgH8BDe3V5CQ8DJycnIgFAaiqqrpaUD6TZNb+srSw6VS9A0O1nt7RGSkTf7N\ntPc/KbsogUIg52PodJRgLkKc+uk4rl69ikGDBpm8L8qezO30QezLaJL6/PPP7REHsSJD16DKy8sx\ncOBAnWNXrFiMtWtHQiR6+P4+qx2aH1a1Oge0UZh/GpelN2zkTU9P13u7rX6x0Fd4IPYVY+KdOvR0\nddVcLxs6dGiz16DouidpjtFKiDt37mDFihV46aWXANRf9Pzqq69sHhhpytQ5PKYMPmxoTLthwzcQ\nCJywdOlU5OZma645SSQSbNy4luZS8ZDeOVRqNQYNGqT3dlv9YqGvVRHKgN8yM5GSlmZSo1y5XI6A\nXgGIejYKAb0CIP+C38MaCQeMnQ+USqXs4MGDLDo6mjHGmFqtbnashi3QNanm9zk1d7z2NagGqamp\nzNlZzID9zV5vysvLs9kYEFtytHP02u+xudekQjw99V6Tany7reyR11+T8gz0ZG4ebmbtfzJln5Kh\nz5+1P5eO9pnR5qixm/q9bjRJTZ48mTHG2MSJEzW3TZgwwcKwLNPWk5Q5s6UaP67xD3JU1DMMEDPg\nkfv/v5ABjHl6hrD09HSrx84FR4q7caHDtq1bTX6sqV/gxr7QW/qF3/D4rKwssx6Xnp7OvAK96hPU\n/f95BnpqPoeG5kfZYq6UI31mGnPU2K2WpGbMmMFKSkpYTEwMY4yxzMxM9vzzz7csOjO19ST1oArv\nQeGUvqRi7Mvm1KlT9xPTg2RX/+f9BldSjshR4tY3BNDbzc2kZGFqYjFW7WfNakBz3/fmVlKG7rt0\n6ZJNukQ4ymdGH0eN3WpDD5ctW4a5c+fixo0bmD59Ol5//XWsXLnSHmciyX3mXGNqbvjhjz/+COAh\nQKd3tj+cnZ+j600csHQI4F65HEEBAUiMikJQQAD2yvVfx9E3zmOeVNrsuA/t+22tYZ+SeI8Ynv/y\nhHiPWLNPqaEoQ3sjsNCnvjhE3+00nqYVMyWTqdVq9ueff7L//ve/nPTya+srKcaav8Zk7HRgw2/d\nqampeldSqampNo3d3hwlbktWUuaMYE9PT2ehXl46+5ZCPB+cTjN2v7mae9+bW/npu49WUqZz1Nit\ntk8KqO/fl5eXh9raWly6dAkAEBMTY9PkSXQ11+1cX8m5UumNlJSP0LNnT0il8zRdI5544jFcuDAY\ngD+APIwZMwrjx4/n4q/U5jXuy5erVmPthg3NrmhzcnIQ4OJi0hwp7SpAfeM8jN3fUg2l5WfPnkXS\na0kQ+YigKlE16TSur7O5oflRvXv3prlSbYzRJLV06VLcvHkTQUFBmjlSAoGAkhQHDI0p0Df8ECjB\nmjXvwsnJRWdT79Wr4UhN/RJnzpzBmDFjeNFDrS1r3JdPrVY3e/y5s2eRXVZmUmLRlwS1+/wZu78l\ntEdglOWXASMB5bD6Db/SBCkiRxvfYGxofpSt5koRfjKapC5evIhDhw5BIBDYIx5iAYlEghUrFuPv\nfx8C4DEAuQC2w8XlTQBiNG4e6+vri1WrVnEVLmlE+5eP/Px8APo3uCoUCixLSsJbAMJRf3XxCoAP\nk5MNflE3ToKNjzPWB9ASly9fxuw5s1E9o/pBz77PAITA7E7jhn4xs/ZcKcJfRgsnHnnkEbtdSCWW\nS0h4GW5uIgBLUN/iqDdqa4tQV3cT2gUXKtV13L17l/5NecxQYURDocVrqP8X/hhAT3d3BIeGAjC8\n2VsikWDgwIEGv9SPpqUhLiYGy599FkEBAfgoJcWkTeOGYh8cEgK1qFq3+7kngHuwa6dx7ffD1I3w\nhIeMXbSaMWMGGzBgAIuPj2cJCQma/9kTFU6YRl9xhfZtQqEHE4m8TN4Q7KgXZB01bsYYy8rK0hRG\nFN0fctjhfjFFc0UT5paSNxQrXLp0qclzigH2hIeH2SXpDbEfA5jYBTrFDRCCuXdzt9vAQ+29VEKx\nkInaiZrdV+XInxlHjd1q+6ROnz6t93/2REnKdIYqpQ4fPmz2hmBH/fA7atyMMXbw4EEW6uXFvgCY\nD8BCAdYOYGtXr2aM6e8oYU7Fn/ZzhHp5sQ6uruzhRp3L+wIs3YTnMRQ7A9ie+4nKyQfM1d2V7UjZ\nYbfOJTqVgUvB4Aaj1YCO/Jlx1NitVt03aNAg3LlzBxcuXAAA9O3bFx07drT5Co9YxlCllLe3t8HB\nh3Runz+6deuG/1VXYy7qu4k3FEeEv/MOXk5I0HuN6cyZMyY3ltXXQX0wdDuX3wIQCEDSzPMYir2h\nWjAOgF8NMLHCFb9lZtq1+7lO49s8AN7Qu6+KPveOweg1qUOHDiE2NhY//PADvv/+e81/E8diyoZg\nwr2OHTti6RtvoCNgcJNv42tMhhrO6vu31beBuKdYjImurgjx8MBgAM+Z8DyGYt8mkyFcLEaopyem\niMXY+emnehOULa8R6TS+7QDgLnSa4NL0XQdjbKkVHR3N7ty5o/lzcXGxptmsNZw4cYI99dRTbMyY\nMSwlJUXvMXS6zzqa2xCsD59iN4ejxs3Yg6a+5py+Y8xwY1l9ffz0PfelS5fY2tWrmYdQyHrdP8Xo\nLhSadU2q4X031rLJlOtnLe0nqN34tuGalGegJ12T4hGrXZNq3PG8trbWal3Qa2trWWRkJLt16xZT\nqVRswoQJ7OrVq02OoyRlPeb88PMtdlM5atyMPYjdkm7mjf9tjU3vbcl1LX2vacr7bsrrWKuBrHZs\nxj73reEzYw4+TDewWpJat24di4+PZ9988w375ptvmFQqZevXr29xgIzVN6uVSqWaP6ekpOhdTVGS\n4oajxu6ocTOmG3tLvkiMJQPt6r709HR2+PBhs1okFRUVsdVrVjOxx4NksnWb8Q7uxloxmTK+wxZa\ny2fGFLboIm8JqxVOvP766/jxxx+RkZEBAJg2bRqioqKscqrx9u3b8PPz0/zZ19dXU6BBbIOmoDqO\nlmxYNTS9t6FgQCKR4GhaGuZJpQgUiZCjUkFVU2NSJwu5XI74l+NRpaoCpNCMjl/8+mLETo1tNmZj\nrZj0TfulQgfrUSgUkCZKoXxOqfl3M7UDCFeMFk4A9bvQBw4ciLCwMAQHB9s6JmImUy9Cm9IpnbQO\nxoop9HVAdxYIMNLNDaGenggXi/W2SFIoFJj10ixUPVUFdIJO1ZxLBxccOnSo2c+hphXT/eKKxq+j\nb9ovFTpYj6Hu8nzuIm90JfXVV19h69atGDx4MBhjWLNmDebNm4epU6e2+MV9fX01bWCA+pVV586d\n9R6rfZwjKSsrs2ns+/d/hyVL3oBQGAi1OgcbN65FTMzEJscVFxcjPn4uqqqOa/r4xcePQp8+fzO4\npaC52IuLi3Hz5k1069aNd1sSbP2e25I1Y1+zYQNGLV6s07xWrVYjPz8f586da9Ko9mFXV7yWkgIv\nLy/Nv2vjWE6cOAGVmwroCeB71CeT+62PygvK8crKV1C7oBYb392ImIn6+3sOHzkSx0+f1vn8aL/O\nhnUbsPj1xRB6C6G+q8aGdx/EbStt5TPj5uaG6uJqnX83VbEKbm5u/P37GzsfOGbMGFZSUqL5c0lJ\nCRszZozF5yG11dTUaAonqqurqXDCTOZM7DV1cKIpsZs7yt7e2tL1BWOam95rSaHE4cOHmUB4f3Ps\n1PsbZb3B4AKGSOtdR7L3hf229JnRrnxsFdekvL290b59e82f27dvD29vb6skSGdnZ/z9739HfHw8\nGGOYOnUqevbsaZXnbgv0jeho2KDbcH/DtSd9ndIt2SelUCgglc7T6awulYYjMjKCt+e027LmGrRa\n0gE9JCQE7SBE7cdqiDyAajXA1E4Q+olRMayi/iArXEeiBrK242hd5I0mqe7du+PZZ5/F6NGjIRAI\ncOTIETz22GP49NNPAQCzZ89uUQAjRozAiBEjWvQcbZWhxHP27DmMHPm0ZoaUTLYNAFBTowIwBIAf\nRKI7kMlSzP6ANpcY+f5hJ7qMdUjXRyKRQPbZZ0iMj0dnlTOKhLVY/8EHWPRaks4pJLqOxG+O9EuA\nSUmqe/fumj+PHj0aAFBRUWG7qIhJJBIJZLJtkErDIRQGQK3ORXLyOiQlLdNZ6cTHj4RA4AS1+mcA\nfgB+gpPTK4iMjDD7Na21IiP8YMmXlb7k5u7lifg58RB1FNEgQmJVRpPU/Pnz7REHsVDjib36VjrO\nzp2hO1fqOYhEG5qsfkwpT9eXGGWybfSF1EZof0YGDhyouT1uehz6PN4HVVVVVj2F5IhbJhwxZj4z\nmqQuXLiAHTt2ID8/HzU1NZrbU1NTbRoYMV3j34Ybr3Rqa4tQW1unc5tSeVUzV0oikUAu36szZl4m\n24aRI4frfb3mRtmT1muvXK6zr2qbTIZpcQ/GwHfs2BFdu3a12us1TPcV+YhQfacabyx7AwlzEiz+\nvNkjeWjHrCpRQZYiQ9z0OOMPJIYZq6wYM2YMS0tLYzdu3GC3bt3S/M+eqLrPPI179O3YsZMJhe4M\n8GZACAPcGSBmHh4P7tdXJZiVlWX32K2hLVVq2Ysp1YDWjF2n80RDFaEPmNjDsmo0Y10WrBE7dcsw\nj9Wq+3x8fDTXoYhj0HcKsF27R1Fa+gOATNQPUjiGsrL6VdWiRSMhEnVD4zHzN2/exBNPPMHVX4Pw\niLEOFtagvdLRdJ7wUNaPnp8FoAugLFSa3SHBXl0WqFuGbRhNUgsXLsQbb7yBIUOGQCQSaW4fM2aM\nTQMjLaP/FGAB6ofr6Cak6uqOqKm5isbFEN26dbNv0IS3jLUzaqnGpxLXJSfXd564hvpxGy2YB6WT\n8PLqn88WyUOnWwZVOVqN0ST1zTff4H//+x9qamrg5PSgixIlKcehXezg7NwV5eXXAJ2vmxI4OblA\nKBwJkehhTTEE3zpJEPtpfP3G0n1Vpr5W40GM4UlJSP4gGa8ufhVV1VUt+uIPDAxE5e1KYDMALwCl\ngFKgbHHy0PceyVJkkCZIIfQRWr3Ksc0WZBg7H2it7hItQdekrKNhF/+yZcsZ0O7+9SkfBnzBPD1D\n2OHDh3V2+VurI7e98ek9NxcfYm9u3lNznwNLY09PT2dPeHiwdIAVNeqMrt1t3dIOCUVFRczFzUXn\nWpGLm0uLrqc1d43LFj8rzb0eHz4zlrDaqI5ly5axK1eutDiglqAkZTl9PzBFRUXMza0DA/4fA4oM\ntlNqiL1xG6QdO3byOmFx/Z63BNexW9ouiTHLY9+xYweDC5i7N5jYBWydntdsyRf/4cOHGXzuJ6iG\n//mAHT582KLY7V0gYez1uP7MWMpqhRPnzp1DTEwM/P39da5JUQm69dhqGa+vrDwubhokEgk++WQH\npNJ5ze51Ki4uxsWLFxEfn4iqqhP3916tR2LiInh4BKGmJlfznKR1sEeBhDaFQoGk15KAl4Dy+6fz\nln0M7ExO1nm9FndIKIPOKUOUWf5U9i6QaOsFGUaT1Mcff2yPONosQ4mkpZrrsQcAvXr1QEbGKZSX\nl+tNjnJCdtCcAAAgAElEQVT5XsTHz4Wzc3dUVakAXEZ9t4p3AfymqQykvn2ti7ECCWO/UJn7C5e+\nL2CPrh4IDg212t8pJCQEQhch1P9S1xdh3AOELkKEhIRY9Hz2LpBo8wUZpiy3Ll++zD7//HP2+eef\ns8uXL7doiWeJ1nq6z5wu5ubS1/Uc6MliY6cxsdiHeXg8wVxdPdmOHTtNiqt+j9VhBvQzq5M6Fxz1\n9Adj/Ijd0Oj65q5VMcbYti1bmr1fH3udOtuRsoOJ2omYuLOYubm7mb1PqvHpRnt3Em/u9fjwmbGE\n1a5J/etf/2Ljxo1jH3zwAfvggw/Y+PHj2a5du1ocoDlaa5KyZHyGqfQnmg4McGPAu/cLJvoxQNwk\nURlKcGJxDwaIbZJUrclRf2gZ40/sjb+UTRlH7+3mZtG1LFt/4TcUHXgEeDBXd1e2I2VHk2Oae98N\nFS3Yu5jI0Ovx5TNjLqslqfHjx7OKigrNnysqKtj48eMtj8wCrTVJ2XIlxRhjq1evbVTFt5YBPe7/\n94PXdHXt0OQitb64Dh8+rOlO0dDNgm+zpBhz3B9axvgRu74vw/T0dBbq5aX9W4umAq/h/hAPD4P3\nG3t+W33hm7JSKyoqYgcPHtT72lx1kTAHHz4zljD1e92k8fHOzs56/5u0TMP+JbE4HJ6eoRCLw63a\nrDUh4WW4uYkALAGQDeBJ1O9m1N3MKxIF6oyPbojLzW2UTlxjxoxBQsLLyM3NRlpaCnJzs6loopXZ\nK5cjKCAAiVFRCAoIwF65HIDxcfSBgYHIUasN3m/s+SUSCQYOHGj1a5vGxqXL5XIE9ArA9HnTEdAr\nAPIv5GY9ntiBsSz2ySefsOjoaLZp0ya2adMmNmHCBPbpp5+2NImapbWupBpY67dIfc/TuI+fVPqS\nyafssrKyeF1qboij/mbJGLexGzulZ+haVYNtW7c2e39Lytv1xWrKZ1OzEnoRDC+D4cUHKyFTV1m0\nkrINq53uY4yxixcvss8++4x99tln7I8//mhRYJZo7UnKGpob6d74B3rHjp3M1bUD8/AI1jm28XGO\n+uF31LgZ4zZ2Y6f0GDO+mbe5+409v6mJx1iz2MbmL5jPIARDx/ox97HPxmpeyyvQS2f/lGdg01OU\nfBm3boijft5bnKTOnz/Pjh8/3uT248ePswsXLlgemQUoSTXPkmtbTaqV9CQ5R/3wO2rcjPF7JWWM\nKRVyhp7f1MRj7sqmuW7qO1J2mPxcfO644qif9xZfk9qwYQN69erV5PZevXph/fr1Nj0FSczTMOiw\ncRfz5s6ba18D0N5TVVqaAaXyGKTSeSguLrZD9IQvNP35xGKEenoiXCy2Wn++5p4fgKZLeemsUiif\nq+90rlAomjyHudeINMd7ADiI+m7qCwHl80okLU1C8nvJEO8Rw+MTD4j3iA322rPVNTNinMHNvBUV\nFfD3929yu7+/P+7evWvToIh5WjrSXd80XxrV0TY1jIbPzMwEAIs3vBp7fu0Nv2fOnDG5o4K5G1s1\nxxvoph4aEorcq7k4c+YML5JQm20i2wyDK6m//vrL4IOqqqpsEgyxTEurBHWTHECjOtq2o2lpiIuJ\nwfJnn9WpwLOWxqsSncQDNJt4GjqNi/eI4fkvz2ZXP9rHu/3oBtyB3teQSCQIDg7mPCk0VBpGPRul\nt9KwzTJ0HvDvf/87e//991ldXZ3mtrq6OvbBBx+wlStXtvyEpBnompRpWnLevHEVIF2T4oa1p9ua\n+3ngosEsY+YXJ5j7dzPWTZ3rz0yTa20vgrm2d2WXLl0y+liuY7dUixvMLlu2DCtXrkRUVBR69+4N\nAMjOzkafPn2wZs0auyVRYrqWNOFsPM1XIpEgPz/fyhESe2k8RHCbTIZpcXFGH2fNBrPmnLqKmx6H\nyNGRJh9vyWf9qTFPYcrkKQb7VXJ5qk2nh+FFAAeB6nbVCBkUgk8/+hRx043/27VaxrLYjRs32JEj\nR9iRI0fYjRs3Wpw9LUErKW44auyOGjdj1om9Jasha62kjPX5sydTKge3bN3CxB5i1r5beyb2sH+Z\nuc5+LjHM2pflqJ93q3Wc6NatGyIiIhAREWHVaxQ//PADxo8fj969e+OPP/6w2vMS0tZpVkP3/6y9\nGjLGGhV+2pN2M0pLcUypxDyp/mo9W1MoFEYrBxUKBZKWJkH5vBIV0goon1dilnSWXeNtuHbm+rUr\n0A7U4UKLSW2RbOHRRx/Fli1bMHDgQK5CaJMUCgXOnDnDyRcGsQ9jLYyMmRYXh+zcXKSkpSE7N9ek\n04TaWpIkrc2UkvXMzEyo3dQ6x6jcVJoKR3uJmx6HzDOZcK12NamIpK3gLEn16NEDgYGBYIxxFUKb\nI5fvRUBAEKKiEhEQEAS5fC/XIREbsMZqqCX7glqaJK2pucrBhl/Y7t2792Ao4v1jWjIUsSV69+6N\nTz/61OTqxbbAYOHEvXv3mn1ghw4drB4MsR2FQnF/wu5WKJVRAApoYGErpm8/kr1okqRUigChELlq\ntclJsiXFC/oe23AaTZoghdBHCHWJGrIUGdLS0iBNlELkI4KqWAUBBGD/YlYZithS5haRtHYGk9Tk\nyZMhEAj0rnQEAgGOHDli9Mlnz56NO3fuNLk9KSkJERERZoZKWiIl5aP7E3Y3AlgAYBucnB5CZmYm\nxowZw3F0xBZaPHK9BSxJknK5/EHiKFFBliIzuaqtucc2/tIHgIBeAVA+p6yvprsOOH/hDGcnZwgF\nQtS61OKTjz7hNDlw+W/HNwLG8fm2mTNnYtmyZXj88ccNHpORkQE/Pz87RmU9ZWVl8PDw4DSG4uJi\nDBo0ElVVx/FgKPgoAFVwdRXh/ff/iZiYiU0ex4fYLeGocQNtN/bi4mIMGjoIVc9XaTpJuP0/N6T/\nnI6OHTs2OfbmzZvo1q0bOnbsaNZjAeDcuXOYPm86ymaXacq90Q4QKUV4df6rmDFjht7H8ZWjfmYK\nCgrQv39/o8cZXElpKy0tRW5uLqqrqzW3WbPgwZQ82bVrV6u9nj3l5+dzHnteXh5cXR9GVZX2pWwf\nAPGorh6PJUvCERs7tclvbnyI3RKOGjfQdmPPy8uDa0dXVHW5382mCyDqKEJVVZXOc+pbMfXq2cuk\nxzYQCoWouVsDXEd9gnqx/jGqQhU279iMJUuWONQqxlE/MwUFBSYdZzRJffXVV9i1axcKCwsRFBSE\n8+fPIzg4GLt27WpRgGlpaVi9ejXu3r2LxMREBAUF4eOPP27Rc5IH5+Xd3d01mxb19fYD7gJ4GYBE\n04zWkX4wSetiSk8+7XJyZRclUAhIE6TI+C3DrH5+DdepZr80G9XtqvVW/tHPAn8YTVK7du3C119/\njWeffRaff/45rl27huTk5Ba/cGRkJCIjI1v8POQBuXwvpNJ5APyhVF6FWNwFQClksm2QybZBKg2H\ni0t3lJX9F8Bb9x+1ByrV9TZd4kq4Z6jAQTtZ6HRlADRJpby83OhjG4ubHofgfsEIGRSC6sJqk5Ib\n4YbRJCUSieDq6goAUKlU6NmzJ65fv27zwIh5tMdtNKyWlMpwAN9AKp2C3Nxs5OZmIycnB2fPnsOC\nBYuhVq8C0BV1dQxpaUdpFDzhlLGqtuZWWwMHDjS7Iq6h3FuaIIVLBxfU3Ktp8+XefGQ0SXXp0gV/\n/fUXIiMjMXv2bHh6ejrk+c/WTt+4DSAAQHvN6byGfS+BgYFISloGtfoUgL5QqbKoHJ3wQnNVbcZW\nW5ZUxDUkRr6M6iBNGU1SW7duBQAsWLAAYWFhKCsrw/Dhw20eGDGP/utOuQAqmsyWMjQ/is7FE76z\nxR4ivozqIPqZVN33xx9/ICMjAwKBAKGhoRCJRLaOi5ipYaaUVBoOoCuUymtwc/OFQDClyWyplg5J\nJIRLtIeobTGapLZs2YLDhw8jKioKALB8+XI8/fTTmDdvns2DI+bRHrehXd3X+AdaO6EJhQFQq3PN\nGpJICCH2YjRJpaam4sCBA5riiTlz5mDixImUpHjK1N8y9c2PIoQQvjGapDp37ozq6mqdCj9fX1+b\nB0Zsj06bEEL4zmiS8vDwwLhx4zB06FAIBAL8/PPP6Nu3r2Y678qVK20eJCGEkLbJaJKKiorSXI8C\ngEGDBtk0IMJPXI7WJoS0XUaT1KRJk+wRB+Gxhk4WIlF9VaBMto02/hJC7MJgklq0aBE+/PBDREdH\n670/NTXVZkER2zF3RaTdyaJ+XxVt/CWE2I/BJPXGG28AAHbs2GG3YIhtNbciMpS8aOMvIYRLBsfH\nd+7cGQBQV1eHTp06wd/fH/7+/ujYsSONfHdA2iui0tIMKJXHIJXOg0KhaHasvO7GX4A2/hJC7Mlg\nkmqwaNEiCASCBw9wcsKiRYtsGhSxvoYVUX2HCaBhRZSZmWkweQEPNv6KxeHw9AyFWBxOG38JIXZj\ntHCitrZWpw2SSCSCWq22aVDE+gy1QgJg8HSev78/ANr4SwjhjtGVlI+PD44cOaL5c1paGry9vW0a\nFLE+QyuikJAQk07nSSQS6hJNCLE7oyupVatWYcmSJVi9ejUYY/Dz88O7775rj9iIlRlaERnq45ef\nn89xxISQts5okurevTu+/PJLVFRUAADat29v86CI7ehrhUSn8wghfGU0SalUKhw+fBh5eXmoqanR\n3D5//nybBkbsi/r4EUL4yGiSmjt3Ljw8PPD444/THClCCCF2ZTRJ3b59GzKZzB6xEEIIITqMVveF\nhITgv//9rz1iIQYoFAqcOXNGs3eJEELaCqMrqYyMDOzfvx/+/v46p/uod599UHNXQkhbZjRJffTR\nR/aIg+hhq+auNHaDEOIoDJ7uKy8vB1Bfcq7vf8T2DLUyysnJsfg5m+vTRwgxjk6/25fBldTixYuR\nkpKCyZMnQyAQ6DSVFQgEOl0oLLF+/XocO3YMIpEI3bt3xz//+U+4u7u36DlbG0OtjCxt7kpjNwhp\nGblcDmmiFCIfEVQlKshSZIibHsd1WK2awSSVkpICxhh2796Nrl27Wv2Fhw0bhiVLlsDJyQkbNmxA\nSkoKFi9ebPXXcWQNrYz0dYOwBI3dIMRyCoUC0kQplM8poeyiBAoBaYIUkaMj6efHhpqt7hMIBEhI\nSLDJCz/55JNwcqp/+eDgYBQWFtrkdRxdXNw05OZmIy0tBbm52S0qmqCxG4RYLicnByIfEdDl/g1d\nAKGPsEWn34lxRkvQ//a3vyErK8vYYS3y9ddfY8SIETZ9DUdmreauNHaDEMsFBgZCVaICGn6fLgTU\nJWr6Jc/GjFb3nT9/HgcOHIC/vz/EYrHmdlNK0GfPno07d+40uT0pKQkREREAgO3bt0MoFBocU0+s\ni/r0EWIZiUQCWYoM0gQphD5CqEvUkKXI6GfIxgTMyJjdvLw8vbc3zBpqiX379uHLL7/Erl27mm25\nlJGRAT8/vxa/HhfKysrg4eHBdRgWcdTYHTVugGLnijmxFxcX4+bNm+jWrRs6duxo48iMc9T3vaCg\nAP379zd6nMGVVHV1NeRyOW7cuIFHH30UU6dOhYuL0YWXyU6ePAmZTIbdu3eb1BPQFsUb9pCfn0+x\n25mjxg1Q7FwxJ/auXbviiSeesHFEpnPU972goMCk4wxmnddffx0uLi4YMGAATp48iatXr2LlypVW\nC3DNmjVQq9WIj48HAPTr1w9vvfWW1Z6fEEKI4zOYpK5du6a57jR16lTExsZa9YV//PFHqz5fW0Yd\nJAghrZXB6j7tU3vWPM1HrIs6SBBCWjOD2Sc7OxuhoaEAAMYYqqurERoaCsYYBAIBzp49a7cgiX7U\nQYJ/UlNT8e2339J4G0KsxGCSunz5sj3jIBagDhLciIiIwNq1azFkyJAm90VHR9t1O8WWLVtw48YN\nrF+/3m6vSYg9Gd3MS/iLOkjwS21tLdchENLqUJJyYNRBglv79+9HXFwc/vnPfyIsLAxbtmzB/v37\n8dxzz2mOeeedd/Dkk0+if//+mDBhAq5evar3ue7evYvExERER0cjLCwMM2bM0NxXVFSEhQsXYsiQ\nIYiMjMTnn38OAPjPf/6DHTt24NChQwgJCUFMTIzm+Llz5yIsLAxPPfUUvvrqK81zZWVlYcqUKejf\nvz+GDRuGd999V3PfokWLMGzYMAwcOBAzZ840GCsh9kQVEQ6OOkjoKi8vx/z5S/HLL+nw9fXFzp0b\nERYWZrPXy8rKwvjx4/Hrr7+ipqYGBw8ehEAgAACcOnUKGRkZ+PHHH+Hu7o7//e9/8PT01Ps8n376\nKbp06YLvvvsOfn5+OHfuHID668GJiYmIiopCcnIyCgoKMHv2bPTo0QPDhw9HYmJik9N9SUlJCAoK\nwqZNm3Dt2jXMnj0b3bt3R1hYGN555x28+OKLmDBhApRKJa5cuaJ53MiRI7Fu3Tq4uLhgw4YNWLJk\nCb799lubvXeEmIJWUq2AtXr7tQaxsbNw6BDD7dsHkJU1D5GR0TZtAOrr64vnn38eTk5OTTalu7i4\noKKiAteuXQNjDD169ECnTp30Po+LiwsUCgUKCgrg7Oys2Yl/4cIF3Lt3D3PnzoWzszMeeughxMbG\n4uDBg3qfp7CwEOfOncOSJUsgFAoRFBSE2NhYTbJxcXHBjRs3cPfuXYjFYvTt21fz2MmTJ0MsFkMo\nFOKVV15Bdna2Zq4cIVyhJEVajZqaGvz00wFUV8sA9AbwHOrqnkZaWprNXrNLly4G7xs8eDBmzJiB\nVatW4cknn8Sbb76JiooKFBQUICQkBCEhIZoKWqlUiu7du+O1115DVFQUdu7cCaC+Ldnt27cxaNAg\nDBo0CAMHDkRKSgpKSkr0vmZRURG8vLx0+mx27doVRUVFAOpPP16/fh3PPPMMYmNjcfz4cQBAXV0d\nNmzYgKioKAwYMACjR4+GQCDA3bt3rfE2EWIxOt1HWg1nZ2e4uLiitvY2gEAADE5OBTadJN1was+Q\nGTNmYMaMGSgpKcGiRYsgk8mwcOFCZGZm6hzXvn17vP7665g5cyYqKyvxwgsvoG/fvvDz88NDDz2E\nw4cPmxRP586dUVpaisrKSrRr1w5AffuZzp07AwC6d++OjRs3AgAOHz6MhQsXIj09HT/88AOOHTuG\nzz77DF27dkVZWRkGDhxo7ttBiNXRSoq0GgKBAKtWvQWxOBLAe3B1fQ5duxZj4sSJnMRz4cIFZGVl\noaamBm5ubnB1ddXMUGvs+PHjuHHjBoD6hOXs7AwnJyf07dsX7du3x0cffYTq6mrU1tbiypUruHDh\nAgCgU6dOyMvL00zO7tKlC0JCQvD+++9DpVIhOzsbX3/9teY9OHDggGYV5uHhAYFAACcnJ1RWVkIk\nEsHT0xOVlZXYuHGj0QRMiD3QSoq0Kq+/vhidO/sgI+MCHnooGK+8slOzorAWU7+8y8vL8c9//hO3\nbt2Cq6srhg0bBqlUqvfYnJwcvP322ygpKUGHDh3w/PPPY9CgQQDqp2SvW7cOo0ePhlqtxsMPP4xF\nixYBAJ5++mkcOHAAYWFheOihh7Bv3z5s3LgR//jHPzB8+HB4eXlh0aJFGDx4MID6isB169ahqqoK\n/v7+SE5OhkgkQkxMDE6dOoURI0agQ4cOWLRoEfbupe4lhHtGR3XwQUZGhkkt3fnIUTsUA44bu6PG\nDVDsXLFm7Pbupemo77up3+t0uo8QQqxELpcjoFcAop6NQkCvAMi/kHMdksOjJEUIIVagUCggTZRC\n+ZwSpbNKoXxOCWmCFAqFguvQHBolKUIIsYBCocCZM2c0SSgnJwciHxHQsCuhCyD0Edp0n15bQEmK\nEELMpO+0XmBgIFQlKqDw/kGFgLpETb00W4iq+wghxAzap/WUXZRAISBNkCL3ai5kKTJIE6QQ+gih\nLlFDliKjTjAtREmqjaDpvYRYR8NpPWUXZf0NWqf14qbHIXJ0JP2sWRGd7msDaHovIdZj7LQe9dK0\nLkpSrZz29N7S0gwolccglc6jiiNCLCSRSCBLkUG8RwzPf3lCvEdMp/VsiJJUK9cwvRdoOr2XWF9q\naqrBrhJ8YI340tPTMXLkSCtF5Jjipsch92ou0r5MQ+7VXMRNj+M6pFaLklQrR9N7rS8iIgK//vqr\n3vuio6Mhk8nsHJHprBUf9fWj03r2QkmqlaPpvfbDl/HxfO50VldXx3UIxMFQkmoD4uKmITc3G2lp\nKcjNzUZc3DSuQ2oVrDk+fuzYsThx4oTmz7W1tRgyZAguX74MADh37hymT5+OgQMHIiYmBunp6Zpj\nZ86cieTkZMTFxSE4OBi3bt3Cvn37EBkZidDQUERGRuLf//63Jmbt+K5cuYL4+HiEhYVh2LBhmjlW\nKpUKa9euxfDhwzFixAi88847UKvVemO/du0aZs6ciejoaERHR+Po0aOa+5YvX4633noLc+bMQUhI\nCE6fPm3u20zaOCpBbyMkEkmbWD2Vl5dj6fz5SP/lF/j6+mLjzp0OMT5+/PjxSE1Nxf/93/8BqO9W\n7uPjg969e+P27dtISEjAhg0bMHz4cPz6669YsGABfvjhB3h7ewOov9b00Ucf4eGHH0ZlZSXWrl2L\nffv2ISAgAHfu3EFpaanmtRriq6iowOzZs/HSSy9hx44dqKmp0STR7du348KFCzhw4AAAYO7cudi+\nfTsWLlyoE3dNTQ3mzp2LqVOn4p133kFeXh7mzZuHffv2aU4pHzx4EB999BGCg4OhUqms9M6TtoKz\nldSHH36ICRMmICYmBlIp9bci1jErNhbs0CEcuH0b87KyEB0Z6RDj48ePH4+jR49qvsT//e9/Y9y4\ncQDqZ0CNGjUKw4cPBwAMGTIEffr00Vl5TZo0CT179oSTkxOcnZ3h7OyMP//8E9XV1ejUqRN69uzZ\n5DWPHz+Ozp07Y9asWRCJRGjXrp1mnPy///1vvPLKK/D29oa3tzfmz5+P7777rslznDt3DpWVlZgz\nZw6cnZ0xePBghIeHa1ZuADB69GgEBwcDQJP3iBBjOEtSL730Eg4cOIBvv/0Wo0aNwpYtW7gKhbQS\nNTU1OPDTT5BVV98fHg88XVfnEOPju3fvjl69euGXX35BVVUVjh49iujoaAD1oxi+//57nRHyZ8+e\nxZ07d/TGIRaLkZycDLlcjmHDhiExMRH/+9//msRXUFCAbt266Y29qKhIZ/yD9gj6xsf5+fnp3Nb4\n2ObeI0KM4ex0n/ZIb6VSaXBiKSGmcnZ2hquLC27X1t4fHg8UODk5xPh4oP661JEjR+Dl5YVHHnlE\nk0D8/PwQExODt99+2+Q4hg4diqFDh0KlUiE5ORlvvvkmdu/erXOMn58fDh06pPf5fH19kZeXp1mB\n5efna0bQa+vcuTMKCgp0bsvPz8fDDz9sMDZCzMFpZkhOTsaoUaOQmpra5Fw3IeYSCAR4a9UqRIrF\neA/Ac66uKO7a1SHGxwPAuHHj8Pvvv0Mul2P8+PGa2ydMmICjR4/i1KlTqKurQ3V1NdLT03H79m29\nz1NcXIwjR45AqVTCxcUF7dq105soRo0aBYVCgV27dkGlUqGiogJZWfVbFcaOHYvt27ejpKQEJSUl\n2LZtm973sV+/fhCLxfjoo49QW1uL06dP4/jx4zrxE9ISNl1JzZ49W+eURIOkpCREREQgKSkJSUlJ\n2LlzJ3bv3o0FCxYYfK78/HxbhmozZWVlFLsdxc2cCbGHB86fOYMefn5YNWsW7t27h3v37lntNerq\n6lBcXIy7d+9CpVLpvEfat+Xm5mLr1q0oLCyESCTCwIEDMXbs2Gbf08ceewznzp3D8uXLdY57++23\nsWnTJiQlJcHZ2RlBQUFISkpCbW0t1Go17t27pzm+pKQEKSkpeO211yAQCNCrVy+8+uqryM/PbxLz\nu+++i82bN2PTpk0QiUSYMmUKOnXqhJiYGBQVFWH8+PEQCAQYNWoUJk6ciPz8fNy5cwe1tbWa53j7\n7beRnJyM7du3QyKRYNmyZRCJRMjPz0dlZaVDfI4cIUZDHDl2U/BifHxBQQHmzJmD1NRUvffT+Hhu\nOGrsjho3QLFzhWK3P96Pj8/NzdX8d1paGnr06MFVKIQQQniKs8KJjRs34vr163ByckLXrl2xatUq\nrkIhhBDCU5wlqU2bNnH10oQQQhwE1X0TQgjhLUpShBBCeIuSFCGEEN6iJEUIIYS3KEkRQgjhLUpS\nhFiRLcbHFxQUIDQ01OJhhuY8vqWvRYi1UZIixEz2Hh/v5+eHs2fPWtyo1ZzHt/S1CLE2SlKEWAlX\n4+Np1UNaM0pShFjIXuPj8/LyEBQUhLq6OgD6x8XfunULM2bMQP/+/REfH4+3334bS5cuBQC9j//w\nww8RFxeH0NBQSKVSTQPexseWlpZi+fLlGD58OMLCwjB//nwAwF9//YXExEQMGTIEYWFhSExM1NtM\nmpCWoiRFWpXy8nLMT5qPLgFd0G9gP5w+fdqmr5eVlYXu3bvj119/xdy5cwFA7/j4jIwMfPDBB+jQ\noYPe52kYH99Ae3y89nM2SE1NxZo1a3D27Fn4+flhyZIl6Nev/u/7yiuv4LvvvtN5TOPHHzx4EO++\n+y5+++03qFQqfPLJJ3qPXbp0Kaqrq/H999/jl19+waxZswDUd4KfMmUKTpw4gWPHjsHNzY26yBCb\noCRFWpXY52Nx6M9DuB19G1ndsxD5jOOPj9dHe1y8QqHAxYsXsXDhQri4uKB///6IiIhoNu7Jkyej\ne/fuEIlEeOaZZ3D58uUmxxQVFeHUqVN4++234e7uDmdnZwwYMAAA0KFDB0RFRWnGzickJOD8+fPN\nviYhlqAkRVqNmpoa/HToJ1SPqwYkAPoCdT0df3y8sdctKiqCl5cXXF1dNbc1HunemHayFIvFqKys\nbHJMYWEhvLy84O7u3uS+qqoqvPnmm4iIiMCAAQMwY8YMlJeX0/UxYnWcNZglxNqcnZ3hInJBbXkt\n4A2AAU7ljj8+3tjrSiQSlJaWorq6WpOoCgoKWlyh5+fnh9LSUpSXlzdJVJ988glycnLw9ddfw8fH\nB2BXJrgAAA2sSURBVNnZ2Zg0aRIYY1QZSKyKVlKk1RAIBFj11iqI5WLgZ8D1O1d0dXH88fFA8xV8\nXbt2RZ8+fbB582ao1WpkZmbi2LFjJj++sYZjJRIJRowYgbfeegt//fUX1Go1fv/9dwBARUUF3Nzc\n4O7ujnv37mHz5s0mPz8h5qAkRVqV15e+jq3rtuKV3q/grWlv4fdffke7du2s+hqmrhTKy8uxcuVK\nDBo0CKNHj4a3t3ezG30lEgn+9re/4dy5cxg7dqzB19T3+u+99x4yMzMxePBgbNq0CWPHjtW5Rmbs\n8YZea/369XBxccEzzzyDYcOGYdeuXQCAF198EUqlEmFhYZg+fTpGjhzZ7HMSYilejI83hsbHc8NR\nY3fUuAHrxZ6UlISePXtqSsbtgd53bjhq7LwfH08IsZ4LFy7g5s2bYIzh5MmTOHr0KCIjI7kOi5AW\no8IJQlqBO3fuYMGCBSgtLYWvry9WrVqFoKAgrsMipMUoSRHSCoSHhyM8PJzrMAixOjrdRwghhLco\nSRFCCOEtSlKEEEJ4i5IUIYQQ3qIkRQghhLc4T1KffPIJgoKCNPNsCCGEkAacJqnCwkL8/PPPDrlb\nmhBCiO1xmqTeeecdvPbaa1yGQAghhMc4S1JHjhyBn58fHnvsMa5CIIQQwnM27Tgxe/Zs3Llzp8nt\nr776KlJSUnRGVjtAn1tCCCF2xkkX9D///BOzZ8+Gm5sbGGO4ffs2fH198dVXX6Fjx45Njs/IyLB3\niIQQQmzMlC7ovBjVERERgf3798PLy4vrUAghhPAI5yXoQP2QNR7kSkIIITzDi5UUIYQQog8vVlLm\ncLTNvx9++CEmTJiAmJgYSKVSKBQKrkMy2fr16/HMM89g4sSJWLBgAcrLy7kOyWQ//PADxo8fj969\ne+OPP/7gOhyTnDx5Ek8//TSeeuop7Ny5k+twTLZixQo8+eSTiI6O5joUsxQWFuKFF17AuHHjEB0d\njV27dnEdkslUKhViY2MRExOD6OhobNmyheuQzFZXV4dJkyYhMTGx+QOZAykoKGDx8fEsPDyc3b17\nl+twTFJeXq757127drE333yTw2jM8/PPP7Pa2lrGGGPvvfce27BhA8cRme7atWvs+vXrbObMmezi\nxYtch2NUbW0ti4yMZLdu3WIqlYpNmDCBXb16leuwTHLmzBl26dIlNn78eK5DMUtRURG7dOkSY6z+\n53TMmDEO854zxlhlZSVjjLGamhoWGxvLzp8/z3FE5vn000/Z4sWLWUJCQrPHOdRKyhE3/7Zv317z\n30qlEk5OjvOWP/nkk5p4g4ODUVhYyHFEpuvRowcCAwMd5lpnVlYWAgIC4O/vD6FQiHHjxuHIkSNc\nh2WSAQMGwNPTk+swzCaRSNC7d28A9T+nPXv2RFFREcdRmU4sFgOoX1XV1NRwHI15CgsLceLECcTG\nxho91mEm8zry5t/k5GR899138PDwcKhTCtq+/vprjBs3juswWq3bt2/Dz89P82dfX19cuHCBw4ja\nllu3biE7Oxt9+/blOhST1dXVYfLkybhx4waef/55h4q9YcFRVlZm9FheJSlH3fxrKO6kpCREREQg\nKSkJSUlJ2LlzJ3bv3o0FCxZwEKV+xmIHgO3bt0MoFPLumoMpsRNiTEVFBRYuXIgVK1bonPngOycn\nJ3z77bcoLy/HvHnzcPXqVfTq1YvrsIw6fvw4OnXqhN69e+P06dNGj+dVkvr000/13v7nn38iLy8P\nEydO1Gz+nTJlisHNv/ZmKO7GoqOjMWfOHF4lKWOx79u3DydOnODlCtDU990R+Pr6Ij8/X/Pn27dv\no3PnzhxG1DbU1NRg4cKFmDhxIiIjI7kOxyLu7u4ICwvDf/7zH4dIUmfPnsXRo0dx4sQJVFdXo6Ki\nAq+99hrWr1+v93iHuEDy6KOP4ueff8aRI0dw9OhR+Pr6Yv/+/bxIUMbk5uZq/jstLQ09evTgMBrz\nnDx5EjKZDNu3b4dIJOI6HIvxadVtyBNPPIEbN24gLy8PKpUKBw8exOjRo7kOy2SO8B7rs2LFCvTq\n1Qsvvvgi16GYpaSkRHOqrKqqCr/88ovDfLf83//9H44fP44jR47g/fffR1hYmMEEBfBsJWUqR9r8\nu3HjRly/fh1OTk7o2rUrVq1axXVIJluzZg3UajXi4+MBAP369cNbb73FbVAmSktLw+rVq3H37l0k\nJiYiKCgIH3/8MddhGeTs7Iy///3viI+PB2MMU6dORc+ePbkOyySLFy/G6dOnce/ePYwaNQoLFizA\nlClTuA7LqIyMDKSmpuLRRx9FTEwMBAIBkpKSMGLECK5DM0qhUGDZsmWoq6tDXV0dxo4di5EjR3Id\nlk3QZl5CCCG85RCn+wghhLRNlKQIIYTwFiUpQgghvEVJihBCCG9RkiKEEMJblKQIIYTwFiUp4nB6\n9+6NSZMmITo6Gq+++iqqq6v1HpeQkGDReJGioiIsWrTI4vgiIiL0jpKprKzEm2++iaioKEyZMgUv\nvPACsrKyLH4dPsjOzsaJEyf03nfv3j288MILCAkJwZo1a+wcGWktHHIzL2nbxGIx9u/fDwBYsmQJ\n5HI5Zs2apXMMYwwpKSkWPX/nzp3x4YcfWhyfQCDQe/vKlSvRrVs3/PTTTwCAvLw8XL161eLX4YPL\nly/j4sWLejeSurq64tVXX8WVK1dw5coVDqIjrQElKeLQBgwYoOntKJVK0a9fP1y6dAkpKSmYMWMG\n9u3bh4qKCrz88svo378/MjMz4evrq2n1dOPGDfzjH/9ASUkJnJ2d8eGHH8LJyQmJiYlITU3F/v37\n8dNPP6GsrAxFRUWIjo7G/PnzAQCvvPIKCgsLoVKp8MILL2jGDujbH3/z5k1kZWVh48aNmtv8/f3h\n7+8PoL4P4b59+wAAU6dOxYsvvoi8vDy89NJLCA4OxtmzZ/HEE09gypQp2LRpE0pKSrBhwwY88cQT\n2LJlC27cuIEbN27g7t27eOmllzSxvPvuuzh16hQEAgESExMxduxYpKenY/PmzfD29saVK1fQp08f\nvPfeewCAP/74A+vWrUNlZSW8vb2xbt06dOrUCTNnzkS/fv1w+vRplJWVYe3atejbty82bdqE6upq\nnD17FnPmzMEzzzyj+fuJxWKEhobqtAYjxGy2HGpFiC0EBwczxhhTq9Vs7ty5TC6Xs1u3brGgoCCd\nwW8RERHs7t277NatW+zxxx9n2dnZjDHGFi1axA4cOMAYYyw2NpalpaUxxhirrq5mVVVV7NatW5oB\nfvv27WPDhg1jpaWlrKqqio0fP14zRLG0tJQxxjS337t3jzHG9A7lPHLkCJs/f77ev8/FixdZdHQ0\nq6qqYhUVFWzcuHHs8uXLmrivXLnCGGNs0qRJbMWKFYwxxtLS0ti8efMYY4xt3ryZTZw4kVVXV7OS\nkhI2cuRIVlRUxA4fPszi4+MZY4zduXOHjRo1iikUCnb69Gk2YMAAdvv2bVZXV8emTZvGMjIymFqt\nZtOmTWMlJSWMMcYOHjzIli9fzhhjbMaMGWzdunWMMcaOHz/OZs2apXl/Vq9e3ey/lynHEGIIraSI\nw6mursakSZMAAP3798fUqVNx+/Zt+Pv768zUYVorGn9/f80ssscffxx5eXmoqKhAUVGRppGroSa6\nQ4cO1Qz1i4qKQkZGBh5//HF89tlnSEtLA1A/xC03N9eimT4ZGRmIioqCq6ur5jV+//13hIeHw9/f\nX9PZ+pFHHsGQIUMA1Ddd1u6aPnr0aIhEIohEIgwePBjnz59HRkaGZgZYx44dMWjQIFy4cAHt27dH\n3759NV3Wg4KCkJeXBw8PD1y5ckXTP7Curk6nE/uYMWMAAH369NF5bUJsiZIUcThubm6aa1LaGiaV\n6qOdgJydnTXFFsyE1pWNrzEJBAKkp6fjt99+w1dffQWRSISZM2caLOAA6hNMdnY2GGMGr1kZi9vJ\nyUnzZycnp//f3t26qBKFYQB/RBRhQBAxGAyCIviFyIB/gckwDIhxwCpmk5gMGh1Mgt1mM43BYNKg\nwWg0WBxQFD/Q2bCs7L26Xi/sZUfu80vDcGbOOXPCyzu88P7SjfXzOw3DuNsB+vNebTbb9dpqteJ8\nPsMwDASDQXQ6nYdr+X1uon+J1X30cp4JLM8QBAFer/eaDR2PR+z3+5txw+EQ6/Ua+/0emqYhmUxi\ns9nA6XTCbrdjPp9jOp0+nMvn8yEajUJV1eu9xWKBwWAAURShaRoOhwN2ux00TYMoin+1l36/j+Px\nCF3XMRqNEIvFIIoier0eLpcLVqsVxuPxw0zP7/dD13VMJhMA772Wvirs+DgDQRCeqqD8rjOj/w8z\nKXo5z2Yiz4yr1+uoVCpQVRU2mw2NRuPmuXg8jmKxiOVyCUmSEIlErhlHJpOB3+9HIpH447zVahW1\nWg3pdBoOhwMulwulUgnhcBiyLCObzQIAcrnc9Rfcs0KhEBRFga7rKBQK8Hg8SKfTmEwmkCQJFosF\npVIJbrcb8/n87nf62H+1WsVms8HlcoGiKAgEAnezSQBIpVJotVqQZfmmcAJ4L8ffbrc4nU7o9/to\nt9sv04KEzIGtOoge6Ha7mM1mKJfLP72ULzWbTQiCgHw+/9NLIfp2/N1HRESmxUyKiIhMi5kUERGZ\nFoMUERGZFoMUERGZFoMUERGZFoMUERGZFoMUERGZ1hsJLAzXWFxlxAAAAABJRU5ErkJggg==\n",
"text/plain": [
""
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"with plt.style.context('seaborn-whitegrid'):\n",
" plt.figure(figsize=(6, 4))\n",
" for lab, col in zip(('Iris-setosa', 'Iris-versicolor', 'Iris-virginica'), \n",
" ('blue', 'red', 'green')):\n",
" plt.scatter(Y_sklearn[y==lab, 0],\n",
" Y_sklearn[y==lab, 1],\n",
" label=lab,\n",
" c=col)\n",
" plt.xlabel('Principal Component 1')\n",
" plt.ylabel('Principal Component 2')\n",
" plt.legend(loc='lower center')\n",
" plt.tight_layout()\n",
" plt.show()"
]
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.5.1"
}
},
"nbformat": 4,
"nbformat_minor": 0
}