{
 "cells": [
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "%matplotlib inline"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# PrincipalComponentAnalysis: Principal component analysis (PCA) for dimensionality reduction"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Implementation of Principal Component Analysis for dimensionality reduction"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "> from mlxtend.feature_extraction import PrincipalComponentAnalysis"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Overview"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The sheer size of data in the modern age is not only a challenge for computer hardware but also a main bottleneck for the performance of many machine learning algorithms. The main goal of a PCA analysis is to identify patterns in data; PCA aims to detect the correlation between variables. If a strong correlation between variables exists, the attempt to reduce the dimensionality only makes sense. In a nutshell, this is what PCA is all about: Finding the directions of maximum variance in high-dimensional data and project it onto a smaller dimensional subspace while retaining most of the information."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### PCA and Dimensionality Reduction\n",
    "\n",
    "Often, the desired goal is to reduce the dimensions of a $d$-dimensional dataset by projecting it onto a $(k)$-dimensional subspace (where $k\\;<\\;d$) in order to increase the computational efficiency while retaining most of the information. An important question is \"what is the size of $k$ that represents the data 'well'?\"\n",
    "\n",
    "Later, we will compute eigenvectors (the principal components) of a dataset and collect them in a projection matrix. Each of those eigenvectors is associated with an eigenvalue which can be interpreted as the \"length\" or \"magnitude\" of the corresponding eigenvector. If some eigenvalues have a significantly larger magnitude than others that the reduction of the dataset via PCA onto a smaller dimensional subspace by dropping the \"less informative\" eigenpairs is reasonable."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### A Summary of the PCA Approach\n",
    "\n",
    "-  Standardize the data.\n",
    "-  Obtain the Eigenvectors and Eigenvalues from the covariance matrix or correlation matrix, or perform Singular Vector Decomposition.\n",
    "-  Sort eigenvalues in descending order and choose the $k$ eigenvectors that correspond to the $k$ largest eigenvalues where $k$ is the number of dimensions of the new feature subspace ($k \\le d$).\n",
    "-  Construct the projection matrix $\\mathbf{W}$ from the selected $k$ eigenvectors.\n",
    "-  Transform the original dataset $\\mathbf{X}$ via $\\mathbf{W}$ to obtain a $k$-dimensional feature subspace $\\mathbf{Y}$.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### References\n",
    "\n",
    "- Pearson, Karl. \"LIII. [On lines and planes of closest fit to systems of points in space.](https://www.tandfonline.com/doi/abs/10.1080/14786440109462720?journalCode=tphm17)\" The London, Edinburgh, and Dublin Philosophical Magazine and Journal of Science 2.11 (1901): 559-572."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Example 1 - PCA on Iris"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "from mlxtend.data import iris_data\n",
    "from mlxtend.preprocessing import standardize\n",
    "from mlxtend.feature_extraction import PrincipalComponentAnalysis\n",
    "\n",
    "X, y = iris_data()\n",
    "X = standardize(X)\n",
    "\n",
    "pca = PrincipalComponentAnalysis(n_components=2)\n",
    "pca.fit(X)\n",
    "X_pca = pca.transform(X)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAagAAAEYCAYAAAAJeGK1AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAgAElEQVR4nO3de3wU5b0/8M8mJIQQLnKRoDSJVarUV9AGRT0WbwEFJAe13GQJcFApqCWEHKQmiBhZQJETKG2AcCpyWV4JeVUpBNRDUvvjYE9BoyK1EQUh4RIgoRoIMeQ2vz+WXbLJzM7s7szszOTzfr3ywkwmu8+Y3f3O8zzf5/vYBEEQQEREZDBhoW4AERGRGAYoIiIyJAYoIiIyJAYoIiIyJAYoIiIyJAYoIiIypE56PllzczMWLlyI48ePIzw8HMuWLUNcXJyeTSAiIpPQtQf10UcfAQDy8/MxZ84cLFu2TM+nJyIiE9G1BzV8+HA89NBDAIAzZ86gT58+ej49ERGZiK4BCgA6deqEBQsWYO/evfjd737n9bPS0lK9m0NERAYwZMiQdsdsoSp1VFVVhQkTJmD37t2Ijo4G4ApQYo0MpbKyMgwaNCjUzdAUr9H8rH59AK/RKsSuUeqzX9c5qB07dmD9+vUAgC5dusBmsyE8PFzPJhARkUnoOsT36KOP4uWXX4bdbkdTUxMyMzPRuXNnPZtAREQmoWuAio6OxurVq/V8SiIiMiku1CUiIkNigCIiIkNigCIiIkNigCIiIkNigCIiIkNigCKyMqcTSEgAwsJc/zqdoW4RkWK6lzoiIp04ncDMmUBdnev78nLX9wBgt4euXUQKsQdFZFVZWdeCk1tdnes4kQkwQBFZVUWFf8eJDIYBisiqpDYD5SahZBIMUERW5XAAV3cK8IiOdh0nMgEGKCKrstuBvDwgPh6w2Vz/5uUxQYJMg1l8RFZmtzMgkWmxB0VERIbEAEVERIbEAEVERIbEAEVERIbEAEVERIbEAEVEpAPW7fUf08yJiDTGur2BYQ+KiEhjrNsbGAYoIiKNsW5vYBigiIg0xrq9gWGAIiLSGOv2BoYBiohIY6zbGxhm8RER6YB1e/3HHhQRERkSAxQRERkSAxQRERkSAxQRERkSAxQRERkSAxQRtcfKpmQAuqWZNzY2IjMzE6dPn0ZDQwNmz56N5ORkvZ6eiJRiZVMyCN16UDt37kTPnj2xbds2bNiwAa+//rpeT01E/mBlUzIImyAIgh5PdPnyZQiCgJiYGHz//fcYN24cSkpKvM4pLS1FdNt6ICFWX1+PqKioUDdDU7xG81Pz+m67/XbYRD4WBJsNX3/1lSrPEQir/w2BjnuNdXV1GDJkSLtzdRvi69q1KwCgtrYWc+bMwdy5c0XPGzRokF5NUqSsrMxwbVIbr9H8VL2+uDjXsF4btri4kP4/tPrfEOi411haWip6rq5JEpWVlZg6dSrGjh2LlJQUPZ+aiJRiZVMyCN0CVHV1NWbMmIH58+dj3Lhxej0tEflLrrIpM/xIJ7oN8a1btw4XL15Ebm4ucnNzAQAbNmyw/HgrkSlJVTZlhh/pSLcAtXDhQixcuFCvpyMiLfjK8GOAIpVxoS5RsDrSkBf3LicdMUARBcM95FVeDgjCtSGvtkHKKkGMe5eTjhigiIKhZFGr0iBmBszwIx0xQJHhOQ87kbAqAWGvhSFhVQKchw30wa5kyMtKlRm4dznpiFu+k6E5Dzsxc9dM1DW6PuDLa8oxc5cra8yeaIAPRYlFrV5DXlabt+He5aQT9qDI0LJKsjzBya2usQ5ZJQbpfSgZ8uK8DVFAGKDI0CpqxHsZFT+UGyPZQMmQF+dtiALCAEWGFtdDvJcRVwPjJBvY7cCJE0BLi+vftsNfnLchCggDFBmaI9mB6Ajv3kd0A+BwF8I3S7KBXBAL1NX09dtuv90YPUoiFTFAkaHZE+3IS8lDfI942AQg/gcgbxdgP9zqJBMlGwSckSi2jqpV+rrN7OnrRCIYoMjw7Il2nJh7Ai3vxOPEqjbBCTBNsoE7I7G8phwCBE9GomyQklpHlZamX/q6VRYak6kwQJF5mDzZIOCMRKl1VBcuiJ+vdo/SSguNyVQYoMg8TJ5sIJmRKHH82gl+Bhy1e5RWWmhMpsIAReaiVbKBDiQzEiWOXztB4ue9e+vTo7TaQmMyDQYoIp2IZiRGRMORLBNQpIY2V6/29CgFLXuUXGhMIcIARdbUZlK/e1FRqFvknZEIG+J7xCMvJU++ZJOvoc2rPcqvv/pKux6lyef+yLxYi4+sR2TX1/6LFgE33BDyIUF7oj2wGoKhrH/nft6sLNewXlycKziZaHiVzIk9KLIekUn9sPp6TuoHw8Rzf2ReDFBkPWab1OcaIyJRDFBkPWaa1JdbY8TgRR0Y56DIehwO7zkoAC1RUQgz4qS+3BqjNnNpmOnaC4tDbNQRsAdF1iOS9VaZnW3MD3Vfw5FcIEsdHAMUWVObSf2LY8aEukXifA1Hmm0ujUhlDFBEoeRrjZGZ5tKINMAARYYX8BYVZuBrES4XyFIHJxmgvvjiCzz11FN4+umn8emnn3qOv/DCC7o0jAgIYosKM5FaY2Ty4rhEwZIMUMuXL8fKlSuRnZ0Nh8OB/fv3AwAuXryoW+OIAt6iwkx8pZJzgSx1YJIBKiIiAjfddBMGDhyIvLw8vPHGGzhy5AhsNpue7aMOLuAtKsyCey2Ryqy0dE4yQHXt2hWbN29GQ0MD+vbti7feegtz587F6dOn9WwfdXABb1FhFkwlJxVZ7X5HMkC99dZbqKmpQUNDAwDg1ltvxZo1a3Drrbfq1jiigLeoMAs1U8mtdOtMAbHa/Y5kJYmYmBj85je/8Tp2yy23IDc3V/NGEbm5K39nlWShoqYCcT3i4Eh2BFYR3Iji4ly3uWLHlXA6gaws3FZe7kqkEATXcVad6JCstnRO9zTzQ4cOITU1Ve+nJROzJ9pxYu4JtLzaghNzT1gjOLl7O+7A0prSVPJW4zk24FpwcjPzrTMFxGpL53QNUBs2bMDChQtx5coVPZ+WyFhaTxQArsDiDlL+pJKLjee0ZdZbZwqI1ZbOSQao5uZmNDQ04MUXX0RjYyMaGhpw5coVTJ06NeAni4uLw5o1awL+fergrDLHIhZYBMEVnPxJJVcSfMx660wBUWvpnFHeapJzUH/605+wbt06VFdXY+TIkRAEAWFhYbjrrrsCfrLHHnsMp06d8nlOWVlZwI+vhfr6esO1SW1muMbuRUXov2iRa+NBACgvR8uzz6LyzBlFdfaMdI23VVRAbLGGUFGBr/1o482xsYisrJT8eUtUFCpfeAEXDXLdwTLS31AralxjUhLw/vvex/x5yKKi7li0qD/q6139l/Jy4NlnW3DmTCXGjAl+Haxf1yjIKCwslDvFLydPnhTGjx8v+rNPP/1U1edSwz//+c9QN0FzWl7j1i+3CvE58YJtsU2Iz4kXtn65NbAHio8XBFc/w/srPl7Rrxvq7xjktXhs3SoI0dHej2GzXXusrQH+vzYoQ/0NNWKEa1Tr5SlF7BqlPvtl56Duv/9+bNiwAb///e89X0RKqFqmyErpSWpNFLQazxHc4zlbtrg+T1h1gtpQOmxnpLeabIBKS0tDbW0t+vTp4/kiUkLVMkVWSk9qPVEAAOHh1zLu/B3sv1oK6euvvmJQIkn+LOA10ltNdkfdrl27Ij09XbUnHDBgALZv367a45FxqVqmqNUuuc5EICsZqOgBxEXUwnHYab7Uc3cg4Y65pANfC3jbvtRENqQOWSagbA9q4MCB2L17N7777jscP34cx48f16NdZAGqlim62utwPtQbM1OA8p6AYAPKmy6Yt7q5kmX/RkmnIlPzZ9jOSEX0ZXtQZWVlXhkXNpsNmzdv1rRRZA2OZAdm7prpNcwXVJkiux1ZVVmoq7ngddg9bGi6XpTUp0Z5uSsYjR4NbNrEHhYFzd+CJXa7MV5isgFqy5YtuHTpEk6fPo2f/OQn6Nq1qx7tIgvQokyRpaqbS31qAK7j69ZJV4cwwqcHmYaRhu38ITvE9+GHHyI1NRXz58/HO++8w1p8FuLeqfb27bdrtlOt2mWKLFXdXCybr7W2wcnNjJmLpDp/Rn+NNGznD9kAtXHjRmzfvh09e/bE888/j+LiYj3aRRoz6061YtXNAeDkxZN4fvfzIWhRENpm8yllxsxF8otc8AlkWw0z7n0pG6DCwsIQGRkJm80Gm82GLl266NEu0phZd6q1J9qRl5KHrhHeQ80tQgvWfrrWnEHqxAnpIBVoIVkyLSXBx2rbakiRDVB33XUX5s2bh3PnzmHRokVITEzUo12kMTPP5dgT7ahvqhf9WV5pnjkz36QW786aZb5xGQqKVPBJS7v2vZEW02pJNkDNmzcPTzzxBMaPH4+HH34Yv/3tb/VoF2nM7HM5zUKz9HGR28/uRUU6t9BPUpMEubnmG5chRaTuo6SCzIUL184x0mJaLckGqNraWk8liZqaGuzYsUOPdpHGzL5TbbgtXPx4C0RvP/vm5GjfKDm+enZXNx5ERYXrU8bhYDCyMKlhvKKi7j6DjHsIz2rbakiRDVDPP/88/vKXv+DYsWM4duwYvvvuOz3aRRpzz+XE94iHDTbE94hHXkqeadYSzRwyU/z4J+2POROBgRMrEfZamGbZirJ8TSwEMuNNpiY1jJeT09dnkHH3rsQ63NOmuR7XTCPbsuQqz06ZMiX48rUKsZp5aJj1GmcXzRbCXwsXsBhC+Gvhwuyi2e1KMW9NhBCdCQGLr31FO6IDr6oeKF8lolUoH23Wv6E/rHSN7qLzbb9sthZBEAShd2//XhJihe2jo41Z0F7Vaua33norDh06hIaGBs8XkRHkPp6LpkVNEF4V0LSoCbmP57Yb+8hKBuoivX8vJNmKvma1O8qMN3lIDePFxjYCAFav9m8Iz6pZfbIB6uDBg0hPT8fIkSMxcuRIjBo1So92UQfmXkAc0JBcm7GPih7ip+merehrVrujzHiTh9QcUnp6FYD2Q3i9ewNdugCpqeLDd1a9x5ENUDt37kRJSQkKCwuxd+9elJSU6NEu6qBUWUDcakViXE/x9UW6Zyv6mtXuKDPe5CGVtNl6x1r3y3jLFuDHH11ZfFJTlMHc4xh5VYZsgDpw4ACGDx+OZ555BiNGjMDHH3+sR7vIwnz1kNReQGyYbEVftWbUqkNj5E8aakdpZQep4bspU1wvl+HDA7/HMXx+jtyE1qRJk4SzZ88KgiAIZ8+eFcaNGxfkFJk0JkmEhhbXKLXV+9YvtwrRjmjJpAXbYpvXz9xftsW2oNrS/83+wW8773nAra7ZaptNemt1Jeeo6NSbb5pnljxAHfW9KJVQ0forOTmwl5zW27uL8SdJQraaeXh4OPr16wcA6NevHzp37qx50CRzcw/TuXtC7mE6wHcPyZ5oR1yPOJTXtK/wHcyQnD3RjqROSRg0aFDAj+HhvuX0tQWGknN8PX4A66H65uQo35GOTMVX0Xu3khKguNj/P7XR565kh/hiYmKwZcsWfP3119iyZQt69JCYdSa6ylcQkiuxZJghOSlK0qUCTakKYrwl4uxZ8R8Y5ZOGAiZX9N4XuVFfo+fnyAaoFStW4MyZM1i1ahUqKyuxdOlSPdpFJuYrCMmVWDL8AmIlt5yB3pYGkSvcGBsr/gOjfNJQwAIteq/kfsfo+TmyAapbt25ISkpCUlIS7r77bvagSJavIKSkh6RkD6mgUtGDoeSWM9Db0iDGW6rS0439SUNBcSdUJCeL/zwysn0vKS1N/n7H6PtEyQaorKws7NmzB507d8aOHTvYgyJZvoKQGj0ksVT01HdTYXvNpn2wUnLLGehtaRDjLRfHjDH2Jw2pori4fZCy2YCGBu9e0vPPu9LSxbS93zHyPlGySRLffPMNCgsLAQDTpk3DhAkTNG8UmZvcVu/2RHtQQ3Zic1wCXLvPtk7I0GRY0P3u9ZXIoOQcMaNHA2vXih9X2jYjfbqQJlrvGZuQ0D6Boq7OdW8ixUyjvrIBKi4uDidPnsRPfvITXLhwAf3799ejXWRywQYhX+SqQLTOCtSEkkAQSLDYs8e/49ThSY3+NovvRgPAXKO+skN8X3zxBUaNGoVHH30UjzzyCP72t7/hl7/8JX75y1/q0T6idpSknFf8UK7+YlWtF8IqnYPigly6Sqo3FC6+Gw169zZXJ1s2QJWUlOAf//gH/ud//geHDh3CJ598gv3792P//v16tI9CqG0iwvO7nw9NYkIbYnNcbcXVQJ1l8e5gYLO5CqGpseReKsAomYOSSM0y/IaMpCr3S6i83PXSbC062vUSEZsGXb1atyaqQjZA/eUvf8GLL76I5557zvNF1ieWiLD207XB1chTSetECwCwwfsdGt0AONwlI4Mp6dw6GACugNCaex9uf3ozvnJ/5ZIrnE7Xpj9G3ZCRdCH2snQHqdYbMVshZ0Z2DuqNN95AdnY208s7GLFEhLY0n+vxofUcl/Ow05WQ8UM54mpcwcl+uNXJgS5WFVuX1NaFC9fSpZRUjPC11unEiWvntE2ucH8qSUwuSC7UJcsRewkJgisIuV9CgDVyZmQD1MCBA3HPPffo0RYyEKXbUei+bYUIT7ASS2kCAk9bCiSwyZUXkptnkvpUkQmWjbGxiJT8KVmJ0csTqUl2iC85ORkTJ07Eyy+/7Pki61Na+06XbSuUJgWovSxei8Cm9iJeAIiOdi3UpQ7B6OWJ1CQboLZs2YJp06Zh9OjRni+yPiWJCMHWyFNUDcKf+nRqL4v3VQQtPt6VEiWmbVJD6+A6erS6i3jDw4G8PNdCXeoQjF6eSFVypdGfe+654OurK8TtNkJD6hrbbpkxu2i26BYagZDbdsNDpf0AAv47zp7dfr8D9zYWW7f63uJC6uezZ/u/L4LMc3Xk16mVKL1GnXdzUZWq221ERUXhmWeewc9//nPYrqaKzJs3T/PAGQzPpLlIFQNSTsvFtnLbbnioNOBeVF6EUR+O8v81sWePePaeXFKD+7hYQsSePd6z2UoEWp2CLMkKCRBKyAaohx9+WLUna2lpweLFi3HkyBFERkZiyZIliPe3RK8MX3sRMUgZh9y2Gx5Sm+H4MeDuPOzEok8Xob65HoDrNTHl3SlIez8Nq0et9v26CDSpQcnv+qujfCoRXSU7B5WSkoK6ujp8+eWXuHjxIh5//PGAn6y4uBgNDQ0oKChARkYGli9fHvBjSVF7y3DShty2Gx4qDLhnlWR5glNrF368gJnvzfC9lksqEIaFySdtdKTZbNJVRykmYhOEtuMX3jIzM9G9e3fcddddOHjwIH744Qe8+eabAT3ZsmXLMHjwYE+QGzZsGP73f//X8/PS0lJEB7oz11W3b7/dUzi0NRts+GrCV34/Xn19PaKiooJqk9GF4hqLyou8ejUAEBUehey7sjEm3nvCv3tREfrm5CDi7Fk0xsaiKj1dUVJAUXkRcg7noLKu0ud5N9p6Yu/4v4n+rHtREfovWoSw+mvtFACvpcEtUVGozM5u1yax35U6N1h8nVqDkmssKuqORYv6o77+Wv8iKqoF2dmVGDPmos/fy8npi7NnIxAb24j09Cqf52tF7Brr6uowZMiQ9ifLTWhNnjzZ6/uJEycGNDEmCIKQmZkp/PWvf/V8/+CDDwqNjY2e79VIkojPifeaeHd/xefEB/R4nJjVTtskjGCSLsQeu20ShtSX7VWZt0HrGenwcP+SNnSazebr1BqUXGMgeUNy+Tx68idJQnaI78qVK/jxxx89ka/ZV5lcGTExMbh8+bLn+5aWFnTqJDsN5hfDbxlOHko2JgyUkkoYbnE1Mie03jCnpUX8HKl5JSNvtkOa02IoLpCpzSA2aw4p2QA1depUjB07Fi+88ALGjh2L6dOnB/xkSUlJ2LdvHwBXlfSf/exnAT+WFMNvGU66kKxw0Wb0N7oBcHwhsZ5JDOeVSCF/lvD5I5CXoFmrT8h2X/793/8dDzzwAE6ePIkBAwbguuuuC/jJRowYgY8//hiTJk2CIAia7c6rZXo0mUNcjziU17TP/uv9IxDTAFT0cPWcHP8bAXu6HyWeHQ7Xp0zr21HLrpKkYPjqtQTTkRZ7CdpsrjXgTqf4SgQVkmFDQrIHVVtbi4yMDNTW1qJnz54oLy9HdnY2amtrA3+ysDBkZ2cjPz8fBQUFuPnmmwN+LCJfpIZ6VyfMxon34tGSbcOJ9+JhT9/o36eF2tUqyLK06rXY7a6i9q232RAE4I9/BP7jPwIrlG9Ukj2oV199FYmJiejatSsAYOTIkTh37hwWL16Mt956S7cGEgWi7bbzsdGxWDFyhev47GAfnOuRSJ6WvRax9eMNDe3PU7qm3Kgke1CVlZWYPn26p3pEp06d8Mwzz+DkyZO6Na4jU1SnzqT0urbWSRglY0raD/t2lMUkFBJa9lr86YW1XlNutnwdyQAVFib+o4iICM0aQy5imwWGanNAtbiDku01G1LfTQ39tWk1g010VaCjwUrum3r1Ut4Oo88z+SIZoOLj41FcXOx1rKSkBH379tW8UR2dUaphqNXTaR1wAbRbSB2SSh9mzbslU/G311JU1F3yvql14Pr+e2XPb4Z5Jl8k56AWLFiAefPm4Q9/+AMGDBiAyspK9OrVK+AqEqSc4jp1GlKzpqGSNUkVNRXSKUhaMGveLVlaTk5f0fumtDTgxx+v3VP5qv8TH2+ueSZfJANU9+7d8d///d84c+YMzp8/j/79+6Nfv356tq3DkkqR1mVzwKsUVxtXQElgjevUyzt3Vsn26cEwa94tWdrZs+JTKBcuKPv9ttu+m53sQt0bbrgBd955J4OTjoxQDUPNXpxcYI2OiIajGPoOuZk175YsLTa2MeDfteLLVzZAkTc9MtCMUA2jVxfxWdhAenFiAdd2tdyq59r+37/Ef1mrITclM9jM8iOdpadXtbtvar3eqa3wcGsvx2OA8oOe2XVydeq0DJTOw07UXGlfoC4iLCKgXpxYwN3y1BYIrwrXru3q0JozEUiYC4S96vrX+aAf6Up+N8zHDDaz/Egjvu57xoy52O6+ydd806ZN5kob95fkHNTEiRM9a6DcBEGAzWZDfn6+5g0zIjXnZYKh9aaMae+noamlqd3xzp06B/z4suWnHA44c/4DMx9rRF2k61B5T2Dmw5eAw079S1dpVaeGOpS2eT+jR7uCiq+p1rbrwBMSxKdLe/e2/ktRMkD913/9l57tMAUjZNcB2gfKCz+Kz8jWNgRe5kqW3Y6sE2moa/J+7jqhQfcbAADM8qOguTvhrYPRunXte0Ry9z1S5R9XtyohqWcCrJ4kh/huvPFG3HjjjWhqakJRURHee+89vPfee1i/fr2e7TMUxbvAaswogVJtFU3i81AhuS5WLacgiXXCpYbrystdPaWiou7tfiY3Xfr880Bqqr6j0XpNz8rOQS1YsAAA8Nlnn+HUqVP44YcftGmJCRghuw7QPlD27iK+/YTUcbWodl1K3z2+zmOWHwXJ3852eTmwaFF/0Zdr6+lSh8MV/MLCgD59gLVrpXtlWtBzelY2QEVFReHXv/41+vXrh+XLl6O6ulr9VpiEEbLrAOlAOXrgaFUSJ1aPWo2IsPbrMSbcPiGgx1NKlRsAiXdP96IiRed53mWsWk5Bkups+8rKq68PkwwsTqcrIE2Zcu1l62t9lFaj0XoWYZENUIIgoKqqCpcvX0ZdXR1qauS2H7U2LXeB9acNbQPltDumYdOhTapkGNoT7Xg26VlPKrjbpkObNK2Zp8oNgMS7p29OjqLzvN5lZqyuSYYh1QmfNct1vyNFLLC476eULtgFtBuN1nN6VjZAvfjii9i7dy/Gjh2L5ORkPPDAA+q3gvzWNlDu+XaPqvX79ny7JyQ184K+AZB4l0ScPavoPK/jXAdFQZDqhOfmuu53pIKUWGARu5/yxWbTbjRaz+lZ2QB19913Y+TIkejTpw/ef/99z5wUGYu/iROt11ElFyW36xkZLhFDabCQeJc0xsYqOs9znOugSAW+OuFiPayoqBbRwOJP78Rmc/XStOrw6zk9KxugnE4nJk2ahLy8PEycOBF//vOf1W8FBc2fBIO2C44r6yrbDQcaJWMRgH/BQuLdU5Werug8z7uM1c5JY2I9rOzsStHA4qt3EhnpWhPlfowtW1y9ND3brdX0rGyAKiwsxK5du/CHP/wBO3bswObNm9VvBQXNnwQDJdt5OJId7RIl/KkkoWqlC3+ChcS75+KYMYrO87zLuA6KdNC2hwWIDxSI3U8BrsD09ttAdbV0QRQtRqn1mp6VXKjr1rt3b4SHhwNwZfT17NlTm5ZQUNpucR7XIw6OZIfoHI7c8J3zsBNp76ehscW7cGXbyiJSAq104TzsFG+/v8FCbEv2sjJl57mx2jnpzOl0pZnX17u+F6sy4c9iXLGFwlpuEKAFmyD4qvQEzJgxA+fPn8cvfvEL/POf/0RTUxNuueUWAMDKlStVbUxpaSmGDBmi6mMGq6ysDIMGDQp1M1SVsCpBdDuP+B7xcCQ7vIKL2Dkn5p4I+PGlfrdtUANcPcC8lDzYU7LEg4Ufewv4/Xds++4GXLewBk01t+LrtC2rX6NUSaNAt9BQ+/HUIvZ3lPrsl+1BzZo1y/PfKSkpKjSPQk0sCLmHA+U2F1SSJBFIgoXP8k1StV60XDQbyC0rURDUHlW2wii15BzURx99BAD47rvvcPz4ca+voUOHYujQobo1ktTVdr1R/+j+nvVGcgFISZJEIAkWPoNaKBbNWrW4GRmW2unbVqjWJRmg3CWNqqurUVVV5fVF0vTYL0oNrdcblYwp8cwN+QoiSqs6BFIRQjao6blolinmFAIOhyvNvLVgBgqsUK1LMkA9+eSTAFzDegkJCXjxxRdRX1+PJ554QrfGmY2e+0VpRSy4AK46fEqrOgRSEcIodQ4BMMWcQsJud6WZqzVQYIVqXYqKxfbt2xcA8OCDDyKLb1JRzsNOTHtvmqrVHEJBLLhsfWorql+q9quqg78VIYxS5xCAuoP3rEZBfhgz5qKqAwVmr9alaEfde+65B4CrqkRLS4vM2R2Pu+fULDSL/txo22C0HYYsKsS/LnwAABT/SURBVPcupNo6uLgTJ/QYsgyqzJGagUCtwXsOFRIFRTZAde/eHQUFBThy5AgKCwvRtWtXPdplKnKZbyGpviDBediJGX+e4TUMmfVJlmjgMc2QpdqBQK3Bew4VEgVFNkAtX74cR48exYoVK3Ds2DEsXbpUj3aZiq8eUsjmUSSkvZ+GhuYGr2ONLY1Iez+t3blKKk4YgtqBQK3Beyvk+RKFkOw6qF69emHWrFm4cuUKAKDevcyZPOJ6xIkuTA23hYduHkWC1HbuYscNVzBWihaBwFeVCaVYjYI00JFWQMgGqMWLF2Pfvn24/vrrIQgCbDYb8vPz9WibaUgtfDVacPKXVOA10pAlAOMGglAsMCZLs0L5In/IDvF9+eWXKC4uRn5+PgoKCoIOTnv37kVGRkZQj2E0hspAkyG1bXvXiK7t1m8pSf02xLovoy74sEKeLxlKR5vWlA1Q8fHxnuG9YC1ZsgQrV660ZCagEXbaVUJsO3cbbGhsaWyXDAHAZ+BVM4kiqEBn5EBg9jxfMhSpUevycmsmh8oO8VVWVuLhhx9G/NXtH4MZ4ktKSsLw4cNRUFAQ0O9T8FpXPS+vKUe4LRzNQnO7xAl3MoSvYOuzfp4fATrQ6ufeF6bCnBGRwUmNZgPWHOqTrWZ++vTpdsduvPFGnw9aWFiITZs2eR1bunQpBg8ejAMHDiA/Px85OTntfq+0tBTRYpuehFB9fT2ioqJC3QzVFZUXYdGni1DfLJ30YoMNX034SvLnt2+/vd228K1/r6i8CDmHc3C27ixio2ORnpiOMfFj2p2fXJSMyrrKdsf7R/dHyZgShVfkm1X/jm5Wvz6A11hU1B0ORz/U1IQDEN/6pn//BpSUHNOwhcETu8a6ujr/qpkXFhZi/PjxyM/Pb7cP0Lx583w2YPz48Rg/frw/bfYwWjl9q5b4H/XhKJ/BCXAlQ/i6dl9JFJ81fYbFny329Ioq6yqx+LPFuOHGG9r1is5uPyv6+Gfrzqr2/96qf0c3q18f0LGv0ekEFi9uP//U1tmzkYb/fyS13YYYyTmo2NhYAK45qJtuusnri8xPLlVcyfotX0kU/qyhMtT28kQGJJYcISbUiatqkwxQw4YNAwDs2bMHTz75pNcXmZ+vD3+lWYi+shf9WUNlqEKxRAakZEmfO3HVXfXLZgM6dXL9a9YykLJJEt26dUNJSQkSEhIQFuaKZ8H0ou655x5PbT8KHbXWbtkT7aLn+7OGyp/t6ok6GqfTVWKyWaTUZ3i4K0HUvWAX8F4n5f4ds66Xkg1Q//rXv/DOO+94vrfZbNi8ebOWbSIdaB0UfO3aK9UeBiQib+6FuWLBKTq6/WqKhATpoUD3einLBKja2lrk5eWhS5cuerWHdOQOClpMPrNXRBQ8qbmn8HDxpX5yQ4FmKwMpGaC2bt2Kt99+G506dcIrr7zimZMiUoq9IqLgSAWUlhbxnpCvdVLun5uJZJJEUVERPvjgA+Tn57db00RERNrzd2sysapfbkao/uUvyQAVGRmJyMhI9OrVC42NjXq2iYiI4H+ZydZVvwDXUCBgrOpf/pBNkgAAmWITRESkAXdA8Wd7DStV/ZIMUEePHkVGRgYEQfD8t9vKlSt1aRwRUUdnpYDjL8kAtWrVKs9/T5o0SZfGEBERuUkGqKFDh+rZDiIiIi+y+0EREZE63GWIwsLMW35IT4qSJIiIKDgdbbt2NbAHRUSkg462XbsaGKCIiHQgVRVC7LivocCONEzIIT4iIh1IlSFqWxXC11Ag0LGGCdmDIiLSgdKqEL6GAjvaMCEDFBGRDlqXIbLZpMsP+RoK9GeY0Ao4xEdEpBMlVSHkhgKVDBNaBXtQREQG4mso0N/isWbHHhQRkYEoKRDrT/FYM2OAIiIyGF9DgR2peCyH+IiIyJAYoIiIyJAYoIiIyJAYoIiIyJAYoIiIyJAYoIiICIDxCtEyzZyIiAy5XxV7UEREJqJVL8eIhWgZoEzOediJhFUJCHstDAmrEuA8bOHNYYg6OHcvp7wcEIRrvRw1gpQRC9EyQJmY87ATM3fNRHlNOQQIKK8px8xdMxmkOoCWlhYsWrQIEydORGpqKiorK0PdJNKBlr0cqYKzoSxEywBlYlklWahr9H611jXWIavEopvDmJjawzLFxcVoaGhAQUEBMjIysHHjRjWaSQanZS/HiIVoGaBMrKJG/FUpdZxCQ4thmdLSUgwbNgwAcOedd+Lo0aMqtZaMTMtejtL9qvSkW4C6dOkSZs2ahSlTpmDixIn4/PPP9Xpqy4rrIf6qlDpOoaHFsExtbS1iYmI834eFhaGpqSnwByRT0LqXY7cDJ04ALS2uf0NdlFa3ALVx40bce++92Lp1K5YtW4bs7Gy9ntqyHMkOREd4v1qjI6LhSLbo5jAmpcWwTExMDC5fvuz5XhAEdOrEVSNWZ8RejpZ0e0VPnz4dkZGRAIDm5mZ07txZ9LyysjK9mqRIfX294drkltQpCYuTFiPncA7O1p1FbHQs0hPTkdQpya82G/ka1RLKa4yNvRmVlZEixxtQVnYsoMe8/vrrsXPnTtx00004cuQIBgwYwL+hBSi5xqQk4P33vY+Z6X+LP39HmyAIgtoNKCwsxKZNm7yOLV26FIMHD0ZVVRWee+45ZGZmYujQoV7nlJaWYsiQIWo3JyhlZWUYNGhQqJuhKV6jttougARcwzLB3Pm2tLRg8eLF+OabbyAIAp599lmMGDFCnQYbFF+n1iB2jVKf/Zr0oMaPH4/x48e3O37kyBHMmzcPL730UrvgRGRVSnZI9VdYWJjXMLnVexbUMek2xHf06FGkpaVh1apVuO222/R6WiJD6Ei7oBKpRbcAtXLlSjQ0NMBxNd0kJiYGa9eu1evpiYjIZHQLUAxGRESh43SqO8ysB+alEhFZnBErlSvBShJERBZnxErlSjBAERFZnBErlSvBAEVkYocOHUJqamqom0EGZ8RK5UowQBHpQYNd5jZs2ICFCxfiypUrQT8WWZsRK5UrwQBFpDWNdpmLi4vDmjVrVGokWZlZa/gxQBFpTaMZ6scee4wFYkkxo1UqV4IBikhrZp2hJgoxBigirZl1hpooxBigiLRm1hlqohBjgCLSmoYz1AMGDMD27dtVaCSR8XCGlUgPLGdO5Df2oIiIyJAYoAzKediJhFUJCHstDAmrEuA8HPzCTiIiM+EQnwE5Dzsxc9dM1DW61s6U15Rj5i5X6WF7IoeJiKhjYA/KgLJKsjzBya2usQ5ZJQYvPUxEpCIGKAOqqBFfwOk+zuE/IuoIOMRnQHE94lBeUy56nMN/BACNjY3IzMzE6dOn0dDQgJSUFAwaNCjUzSJSFXtQBuRIdiA6wnthZ3RENBzJDg7/mZTavd6dO3eiZ8+e2LZtGzZs2IC8vDyVWkpkHAxQBmRPtCMvJQ/xPeJhgw3xPeKRl5IHe6JddviPjMfd6y2vKYcAwdPrDSZIjRw5EmlpaZ7vw8PD1WgqkaEwQBmUPdGOE3NPoOXVFpyYe8IzfBfXQ7x+m9RxCj0ter1du3ZFTEwMamtrMWfOHEyePDnYZpIJabDNmKEwQJmMr+E/Miater2VlZWYOnUqxo4diwcffDCoxyLz0WibMUNhgDIZX8N/ZExa9Hqrq6sxY8YMzJ8/H+PGjQv4cci8NNpmzFCYxWdC9kQ7A5KJOJIdXpmXQPC93nXr1uHixYvIzc1Fbm4uLl++jG3btiEqKkqNJpMJdIRtxhigiDTmvpnIKslCRU0F4nrEwZHsCOomY+HChVi4cKHn+7KyMganDiYuzjWsJ3bcKhigiHTAXi+pzeFwzTm1Huaz2jZjnIMiIjIhDbcZMwz2oIiITMrq24yxB0VERIbEAEVERIbEAEVERIak2xxUXV0dMjIyUFNTgy5dumDFihXo1auXXk9PREQmo1sPavv27bj99tuxbds2PP7448jNzdXrqYmIyIR060FNnz4dzc3NAIAzZ86gT58+ej01ERGZkE0QBEHtBy0sLMSmTZu8ji1duhSDBw/G1KlT8c0332Djxo3tNlgrLS1VuylERGQCQ4YMaXdMkwAl59ixY/j1r3+N4uJivZ+aiIhMQrc5qPXr12PHjh0AgOjoaG6wRkREPunWg6qursaCBQvQ0NCA5uZmZGRkiHbpiIiIgBAN8REREcnhQl0ZdXV1mD17NiZPnoxnnnkG//rXv0LdJNVdunQJs2bNwpQpUzBx4kR8/vnnoW6SZvbu3YuMjIxQN0M1LS0tWLRoESZOnIjU1FSUi+2/YAGHDh1CampqqJuhicbGRsyfPx+TJ0/GuHHjUFJSEuomqa65uRkvv/wyJk2aBLvdjgqFm1YxQMnoCOu3Nm7ciHvvvRdbt27FsmXLkJ2dHeomaWLJkiVYuXIlWlpaQt0U1RQXF6OhoQEFBQXIyMjA8uXLQ90k1W3YsAELFy7ElStXQt0UTezcuRM9e/bEtm3bsGHDBrz++uuhbpLqPvroIwBAfn4+5syZg2XLlin6PVYzl9ER1m9Nnz4dkZGRAFx3Op07dw5xi7SRlJSE4cOHo6CgINRNUU1paSmGDRsGALjzzjvxj3/8I8QtUl9cXBzWrFmDl156KdRN0cTIkSPx2GOPeb63YgLZ8OHD8dBDDwHw73OUAaoVpeu3zMzXNVZVVWH+/PnIzMwMUevUIXWNo0ePxoEDB0LUKm3U1tYiJibG8314eDiamprQqZN13tqPPfYYTp06FepmaKZr164AXH/LOXPmYO7cuSFukTY6deqEBQsWYO/evfjd736n7JcEUuzo0aNCcnJyqJuhia+//loYPXq08Ne//jXUTdHU3//+d2Hu3LmhboZqli5dKuzevdvz/bBhw0LYGu2cPHlSGD9+fKiboZkzZ84ITz75pFBYWBjqpmju/PnzwkMPPSRcvnxZ9lzOQcnoCOu3jh49irS0NKxcuRIPPvhgqJtDfkhKSsK+ffsAAF988QV+9rOfhbhF5K/q6mrMmDED8+fPx7hx40LdHE3s2LED69evBwB06dIFNptN0WepdcYBNPKrX/0KCxYswJ/+9Cc0Nzdj6dKloW6S6lauXImGhgY4HA4AQExMDNauXRviVpESI0aMwMcff4xJkyZBEARLvj6tbt26dbh48SJyc3M9SVgbNmxAVFRUiFumnkcffRQvv/wy7HY7mpqakJmZqWium+ugiIjIkDjER0REhsQARUREhsQARUREhsQARUREhsQARUREhsQARaZ24MAB3HfffUhNTUVqaiomTJiALVu2tDtv3759fpc4evfdd/0u3Hnq1ClMmDCh3fGamhpkZmbCbrdj0qRJSE9Px6VLl/x67FArKChAY2Oj6M+sVoSXjIHroMj07r33XuTk5AAAGhoaMHLkSIwdOxbdu3f3nPPAAw/4/bhPPfWUam2cN28eJk2ahBEjRgAA3nnnHSxatMjTbjNYv349nnjiiXbHlyxZgv3792PQoEEhaBVZGQMUWUptbS3CwsIQHh6O1NRUXHfddbh48SIef/xxlJeXY9KkScjIyEBsbCxOnjyJxMREvPbaa7hw4QJ++9vf4tKlSxAEAW+88QZ27dqFPn364Kc//SnWrVuHsLAwVFVVYeLEibDb7Th48CB+//vfAwDq6+vxxhtvICIiol2bTp8+jerqak9wAoDU1FT86le/AuCqZr1p0yZERkYiISEB2dnZ2LVrFz766CPU19ejqqoKU6dORUlJCb799lu89NJLGD58OJKTk3HHHXegoqICAwcOhMPhQG1tLebPn4/a2lo0NzcjLS0N9913H1JSUjB06FAcOXIENpsNubm56NatG1auXIlPPvkEgiBg+vTpGDVqFFJTU3Hbbbfh22+/RW1tLVavXo2//e1vqKqqQnp6eruK/lYswkvGwABFpvf3v/8dqampsNlsiIiIwCuvvOIpwJmSkoIRI0bg3Xff9Zx/4sQJ/PGPf0SXLl0wfPhwVFVVYf369XjkkUfw9NNP4//+7//w5Zdfej3HuXPnsGPHDrS0tCAlJQUjR47Et99+ixUrVqBfv35Yt24dPvjgA6SkpLRr3/nz5zFgwACvY+Hh4ejWrRu+//57rFmzBu+99x5iYmKwdOlSFBQUIDo6GpcvX8bbb7+N3bt345133sH27dtx4MABbN68GcOHD8e5c+eQlpaG+Ph4pKWlobi4GJ9//jn+7d/+DdOmTcO5c+fw9NNPo7i4GJcvX8bjjz+OV155BRkZGdi3bx9iYmJw6tQp5Ofn48qVK5gwYQLuv/9+AMDgwYORlZWFnJwc7N69GzNnzsTatWtFe3xWLMJLxsAARabXeoivrZtuuqndsbi4OE8F8L59++LKlSs4fvy4pw7afffdBwBYs2aN53d+8YtfeLYkGThwICoqKtCvXz84HA5ER0fj3LlzSEpKEm3DDTfcgLNnz3oda2xsxAcffID4+Hjccsstnvbcfffd2L9/P+644w7PkFm3bt1w8803w2azoUePHp59kfr374/4+HhP+44fP45jx455gmS/fv0QExPj2WTz5z//uef3rly5gjNnzuCrr77ybATY1NSEM2fOeJ0bGxuL6upq0esi0hqTJMjSbDabomM333wzDh8+DAD45JNPsGLFCq+fl5WVobm5GT/++COOHj2K+Ph4LFy4EEuXLsXy5ctx/fXXQ6pqWL9+/XDdddehuLjYc2zz5s0oLi7GgAEDcOzYMdTV1QEADh486AmqYu1s7dy5c6iqqgIAfPbZZ7jllltw880349NPP/X8/OLFi+jZs6fo4/30pz/FPffcgy1btmDTpk0YNWpUu55eazabzVKbPZLxsQdFBGDWrFnIzMzEzp07Abj2j3JXsQdcvYvnnnsOP/zwA2bPno1evXph7NixmDBhArp3744+ffrg/Pnzko//5ptvIjs7G2+//TYaGxsRFxeHJUuWoFu3bvjNb36DqVOnIiwsDHFxcfjP//xP7N69W7bNkZGReP3111FZWYk77rgDjzzyCIYMGYLMzEx8+OGHqK+vR3Z2tuTeUI888ggOHjyIyZMno66uDsOHD/faW6qtu+66CzNnzsTmzZtlgyeRGlgslkjGgQMHkJ+fb7iMu/vvvx8ff/xxqJtBpBkO8RERkSGxB0VERIbEHhQRERkSAxQRERkSAxQRERkSAxQRERkSAxQRERnS/wcMreQE4ac56QAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "import matplotlib.pyplot as plt\n",
    "\n",
    "with plt.style.context('seaborn-whitegrid'):\n",
    "    plt.figure(figsize=(6, 4))\n",
    "    for lab, col in zip((0, 1, 2),\n",
    "                        ('blue', 'red', 'green')):\n",
    "        plt.scatter(X_pca[y==lab, 0],\n",
    "                    X_pca[y==lab, 1],\n",
    "                    label=lab,\n",
    "                    c=col)\n",
    "    plt.xlabel('Principal Component 1')\n",
    "    plt.ylabel('Principal Component 2')\n",
    "    plt.legend(loc='lower center')\n",
    "    plt.tight_layout()\n",
    "    plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Example 2 - Plotting the Variance Explained Ratio"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "from mlxtend.data import iris_data\n",
    "from mlxtend.preprocessing import standardize\n",
    "\n",
    "X, y = iris_data()\n",
    "X = standardize(X)\n",
    "\n",
    "pca = PrincipalComponentAnalysis(n_components=None)\n",
    "pca.fit(X)\n",
    "X_pca = pca.transform(X)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([2.91081808, 0.92122093, 0.14735328, 0.02060771])"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "pca.e_vals_"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([0.72770452, 0.23030523, 0.03683832, 0.00515193])"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "pca.e_vals_normalized_"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "\n",
    "tot = sum(pca.e_vals_)\n",
    "var_exp = [(i / tot)*100 for i in sorted(pca.e_vals_, reverse=True)]\n",
    "cum_var_exp = np.cumsum(pca.e_vals_normalized_*100)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAakAAAEZCAYAAAAt5touAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAgAElEQVR4nO3de1zP9/8//turg4pEITm998aYxBxySjllCfXqKOKjj5lpGDmNVU779DaNYWNsZuYQhklJKqccModmOb2R8zKZSpJKKr16fv/o1/On9Xom1Ov1rG7Xy2WXi9ez5+H+fNS69XgeHg+FIAgCiIiIZEhH2wUQERFJYUgREZFsMaSIiEi2GFJERCRbDCkiIpItPW0XUBEJCQnaLoGIiKqYtbV1mWXVIqQA9cVXd4mJibC0tNR2GTUe21kz2M6aUVPbWaozwst9REQkWwwpIiKSLYYUERHJFkOKiIhkiyFFRESyxZAiIiLZYkgREZFsVXpIBQQEwMbGBs7OzuKyzMxMjB8/HkOGDMH48ePx9OlTAIAgCFi8eDEcHBygVCpx9erVyi6HiIiqsUoPKQ8PD2zYsKHUsvXr18PGxgaHDh2CjY0N1q9fDwCIi4tDUlISDh06hP/85z/44osvKrscIiKqxip9xImePXsiOTm51LLY2Fhs3boVAODm5gYfHx/MmTMHsbGxcHNzg0KhQNeuXZGVlYW0tDSYm5tXdllERKX8Ev8XIi4+0HYZr82uhS5q4IATkjQyLNLjx4/F4DE3N0dGRgYAIDU1FRYWFuJ6FhYWSE1NVRtSiYmJmihVo/Ly8mrkeclNdWvn6JtZOH43R9tlvLaioiLoHPhb22VU2H9T8wAAnZsaarmS1/OiwLBa/Ty/La2O3adu5nqFQqF23Zo4VlVNHYNLbqpbO38RdwZJTwvRsZmJtkt5Lbm5uahbt662y6iw3q3rwrVrC4zp/S9tl/JaqtvPc0VJjd2nkZBq1KiReBkvLS0NZmZmAIp7TikpKeJ6KSkpvNRHBKBjMxPs+sRG22W8lpr6y5O0SyOPoNvb22Pv3r0AgL1792Lw4MGllguCgIsXL6J+/foMKSIiElV6T2rWrFn4/fff8eTJE/Tv3x/Tpk2Dr68vZsyYgdDQUDRr1gyrVq0CAAwYMAAnTpyAg4MDjIyMsGTJksouh4iIqrFKD6mVK1eqXb5ly5YyyxQKBRYtWlTZJRARUQ3BESeIiEi2GFJERCRbDCkiIpItrb4nRdVLdX1DPzc3F3XjMrVdRoVde5hV7d6RIqoq7ElRhUVcfIBrD7O0XUaN17GZCVy7ttB2GUSywJ4UvRa+ZEpEmsSeFBERyRZDioiIZIshRUREssWQIiIi2WJIERGRbDGkiIhIthhSREQkWwwpIiKSLYYUERHJFkOKiIhkiyFFRESyxZAiIiLZYkgREZFsMaSIiEi2GFJERCRbDCkiIpIthhQREckWQ4qIiGSLIUVERLLFkCIiItliSBERkWwxpIiISLYYUkREJFsMKSIiki2GFBERyRZDioiIZIshRUREssWQIiIi2WJIERGRbDGkiIhIthhSREQkW3qaPNjmzZuxe/duKBQKtG/fHsHBwUhLS8OsWbPw9OlTdOzYEcuWLUOdOnU0WRYREcmUxnpSqampCAkJwZ49e7B//36oVCpERUVh+fLl+PDDD3Ho0CGYmJggNDRUUyUREZHMafRyn0qlQl5eHgoLC5GXl4cmTZrg7NmzcHR0BAC4u7sjNjZWkyUREZGMaexyX9OmTfHRRx9h0KBBMDAwgK2tLaysrGBiYgI9veIyLCwskJqaqnb7xMRETZWqMXl5edXqvHJzcwFUv+9FdWvn6ortrBm1rZ01FlJPnz5FbGwsYmNjUb9+fUyfPh1xcXFl1lMoFGq3t7S0rOoSNS4xMbFanVfduEwA1e97Ud3aubpiO2tGTW3nhIQEtcs1FlKnT59Gy5YtYWZmBgAYMmQILly4gKysLBQWFkJPTw8pKSkwNzfXVElERCRzGrsn1bx5c1y6dAnPnz+HIAg4c+YM3n33XfTu3RsHDx4EAISHh8Pe3l5TJRERkcxJhlRKSgo+/fRT9OnTB3379sW0adOQkpLyxgfq0qULHB0d4e7uDqVSiaKiIowaNQpz5szBpk2b4ODggMzMTHh5eb3xMYiIqGaRvNwXEBAAZ2dnrFq1CgCwb98+BAQEYNOmTW98MD8/P/j5+ZVa1qpVKz52TkREakn2pDIyMuDp6Qk9PT3o6enBw8MDGRkZmqyNiIhqOcmQMjU1RUREBFQqFVQqFSIiItCwYUNN1kZERLWcZEgtWbIEMTExsLW1hZ2dHQ4ePIglS5ZosjYiIqrlJO9JNW/eHOvWrdNkLURERKWUCamffvoJEydOxH/+8x+1L9bOnz9fI4URERGVCam2bdsCADp16qTxYoiIiF5WJqRKXqY1NDTEsGHDSn0tJiZGM1URERGhnAcn1q9fX6FlREREVaVMT+rEiROIi4tDamoqFi9eLC7PycmBrq6uRosjIqLarUxINW3aFJ06dcLRo0dhZWUlLq9Xrx4CAgI0WlxNtichGZvj/hZHFq8Orj3MQsdmJtoug4hqkTIh1aFDB3To0AHOzs7Q19fXRk1V4pvDN7VdQimJD7OQmvMC+vnPtV2KqKWpUblf79jMBK5dW2ioGiKict6TevDgAVauXInbt28jPz9fXM6ZcyuHZTMTNNHPR+PGTbRdimimQ3ttl0BEVIrkgxMBAQEYPXo0dHV1ERISAjc3N7i6umqyNiIiquUkQyo/Px82NjYAgBYtWmDatGk4e/asxgojIiKSvNxXp04dFBUV4Z133sG2bdvQtGlTPH78WJO1ERFRLSfZkwoMDMTz588xf/58XL16Ffv27cPSpUs1WRsREdVyantSKpUKMTEx+Pzzz1GvXj0EBwdrui4iIiL1PSldXV1cvXoVgiBouh4iIiKR5D2pjh07YvLkyRg6dCjq1q0rLh8yZIhGCiMiIpIMqadPn8LU1BTx8fGlljOkiIhIUyRDivehiIhI2ySf7iMiItI2hhQREckWQ4qIiGRLMqTS09MRGBiIjz/+GABw+/Zt7N69W2OFERERSYaUv78/7OzskJaWBgD497//jZCQEI0VRkREJBlST548wfDhw6GjU7yKnp6e+G8iIiJNkEydunXr4smTJ1AoFACAixcvon79+horjIiISPI9KX9/f0yePBl//fUXvL298eTJE6xatUqTtRERUS0nGVJWVlbYtm0b/vzzTwiCgNatW9eo6eSJiEj+JC/3bd++Hbm5uWjXrh3at2+P3NxcbN++XZO1ERFRLScZUr/++itMTEzEzw0aNOAj6EREpFGSIVVUVFRqqg6VSoUXL15opCgiIiKgnHtSdnZ2mD59OkaPHg0A2LlzJ/r166exwoiIiCRDas6cOdi5cyd27NgBQRBga2sLLy8vTdZGRES1nGRI6ejoYMyYMRgzZowm6yEiIhJJhlRCQgLWrFmDv//+G4WFhRAEAQqFArGxsW98sKysLMyfPx83b96EQqHAkiVL0Lp1a8ycORMPHjxAixYt8O2336JBgwZvfAwiIqo5JENq3rx5CAgIQKdOnSptOKQvv/wS/fr1w+rVq1FQUIC8vDysW7cONjY28PX1xfr167F+/XrMmTOnUo5HRETVm2T61K9fHwMGDECjRo1gamoq/vemcnJycO7cOYwYMQIAUKdOHZiYmCA2NhZubm4AADc3Nxw5cuSNj0FERDWLZE+qd+/eWLp0KYYMGYI6deqIy62srN7oQPfv34eZmRkCAgJw/fp1WFlZYd68eXj8+DHMzc0BAObm5sjIyHij/RMRUc0jGVKXLl0CAFy5ckVcplAo3ni6jsLCQly7dg0LFixAly5dsHjxYqxfv77C2ycmJr7RcUukp8sv/AoLC5Ge/kjbZYgSE1XaLqFK5OXlvfXPD70a21kzals7S4bU1q1bK/VAFhYWsLCwQJcuXQAAQ4cOxfr169GoUSOkpaXB3NwcaWlpMDMzU7u9paXlWx2/cfLNt9q+KqSnP0Ljxk20XYbI0rK9tkuoEomJiW/980OvxnbWjJrazgkJCWqXS4YUABw/fhy3bt1Cfn6+uGzq1KlvVECTJk1gYWGBu3fvok2bNjhz5gzatm2Ltm3bYu/evfD19cXevXsxePDgN9o/ERHVPJIhtXDhQuTl5SE+Ph5eXl44ePAgOnfu/FYHW7BgAT777DO8ePECrVq1QnBwMIqKijBjxgyEhoaiWbNmnA6EiIhEkiF14cIFREZGQqlUYurUqRg/fjymTZv2VgeztLREWFhYmeVbtmx5q/0SEVHNJPkIuqGhIQDAyMgIqamp0NfXR3JyssYKI6KaJzk5Gc7Ozq9cJzIyUvz83//+F4sXL67q0l5Lt27dXrmOt7d3pRyrIm32piqrxqok2ZMaOHAgsrKyMGHCBHh4eEChUIjvOBERVZUHDx5g//79UCqVAIDOnTu/9a0Gbdi5c6e2S5CkUqmgq6sr6xpLSPakPv30U5iYmMDR0RHHjh1DTEwMZsyYocnaiEgD9u7dC6VSCRcXF3G0F39/fxw4cEBcp6TnEB8fj7Fjx2L69OlwdHTE8uXLsW/fPowYMQJ+fn7466+/yt3+ZcnJyRgzZgzc3d3h7u6O8+fPAwBWrFiBP/74A66urti8eTPi4+PxySefoKioCPb29sjKyhL34eDggPT0dGRkZGDatGnw9PSEp6en2ifFVCoVli5dCk9PTyiVSvEX9OHDh/Hhhx9CEASkpaXB0dERjx49QlhYGCZPnowJEybA0dERa9asKbPPZ8+eYdy4cXB3d4dSqSw1GMHLbebj4wM/Pz8MHToUs2fPFqdBunLlCsaOHQsPDw9MmDABaWlp4nIXFxeMGjVKcrLZGTNm4MSJE+Jnf39/HDx4ULJdS+qYPXu2+AdASY1S55GcnIxhw4Zh/vz5cHJywkcffYS8vDwAwL179/Dhhx/CxcUF7u7u4vd+w4YNYhuvXr1abe2vo0xP6syZM7CxscGhQ4fUbjBkyJC3PigRlbUnIRm//nG/Uvc5skcreFq3lPz6rVu38MMPP2DHjh0wMzNDZmbmK/d5/fp1REdHo2HDhhg8eDC8vLwQGhqKZcuWYevWrZg3b16FamvUqBE2bdoEAwMDJCUlYdasWQgLC8Ps2bOxceNG/PjjjwCKf7kCxYNe29vb4/Dhw/D09MSlS5fQokULNG7cGLNnz8a4cePQo0cP/P3335gwYQJiYmJKHS80NBT169fHnj17UFBQAG9vb9ja2sLBwQEHDx7E9u3bcfLkSUybNg1NmhS/GvLf//4XkZGRMDIywogRIzBgwIBSvToDAwOsXbsWxsbGyMjIwKhRozB48GAoFIpSx7527RqioqJgbm6O0aNHIyEhQXxf9Pvvv4eZmRmio6PxzTffIDg4GAEBAViwYAF69eqFpUuXqm0/JycnREdHY8CAASgoKMCZM2fwxRdfQBAEte368vm0atWq1L6kzgMoDqOVK1di8eLFmD59Og4ePAhXV1d89tln8PX1hYODA/Lz81FUVITffvsN9+7dQ2hoKARBwOTJk3Hu3Dn07NmzQj8T6pQJqXPnzsHGxgbHjh1TuwFDiqjmOHv2LIYOHSq+n9iwYcNXbtO5c2dxlJh//etfsLW1BQC88847pf6yf5XCwkIEBQXh+vXr0NHRQVJS0iu3GT58ONauXQtPT09ERUVh+PDhAIDTp0/j9u3b4no5OTnIycmBsbGxuOzUqVO4ceMGDh48CADIzs7GvXv30KpVKyxYsADOzs7o2rVrqfs/ffv2FYeDc3BwQEJCQqmQEgQBK1euxLlz56Cjo4PU1FSkp6eLIVfi/fffh4WFBQCgQ4cOePDgAUxMTHDz5k2MHz8eQPFEs02aNEF2djays7PRq1cvAICrqytOnjxZpi369++PxYsXo6CgAHFxcejRowcMDQ2RnZ0t2a6dO3cuE1DlnQcAtGzZUnwvy8rKCg8ePEBOTg5SU1Ph4OAAoDjkStr41KlT4lB3ubm5SEpKqtyQ8vPzQ1FREfr16yf+ABBR1fO0bllur6cqvDz79st0dXVRVFQkrvPyrNwvD5Omo6MjflYoFFCpVK/cvsTmzZvRuHFjREREoKioCO+///4r6+3WrRv++usvZGRk4MiRI5g8eTKA4l/wu3btEh/4kjrX+fPnq528NTU1FTo6OkhPT0dRUZE4qPY/e0T//BwZGYmMjAyEhYVBX18f9vb2pd4rLfFym+nq6kKlUkEQBLRr1w67du0qtW5WVlaZ46hjYGCAXr164eTJk4iJiYGTkxOA8tu1bt26avdV3nn8s3Z151dCEAT4+vpW6gMZau9J6ejoSF4HJaKaw8bGBgcOHMCTJ08AQLzc16JFC1y9ehUAEBsbqzZkylOR7bOzs9GkSRPo6OggIiJCDLh69erh2bNnaverUCjwwQcfIDg4GG3bthV7OXZ2dti2bZu4nrphg+zs7LBjxw6xlj///BO5ubkoLCxEQEAAVqxYgbZt22LTpk3iNqdOnUJmZiby8vJw5MgRdO/evcw5NGrUCPr6+jh79iwePHhQ4TZq3bo1MjIycOHCBQDAixcvcOvWLZiYmMDY2Bh//PEHAJR60vGfnJycEBYWhj/++AN2dnZiTeratTyvex7GxsawsLAQ710VFBTg+fPnsLOzw549e8TvX2pqKh4/fvzqxiiH5IMTffv2xc8//4yHDx8iMzNT/I+Iao527dph0qRJ8PHxgYuLC7766isAwMiRI8VZCy5duiT5F7iUimw/ZswYhIeHY+TIkUhKShLXee+996CrqwsXFxds3ry5zHbDhw/Hvn37Sl3pmTdvHq5cuQKlUonhw4djx44dZbbz8vLCu+++Cw8PDzg7O2PhwoVQqVRYt24devTogR49esDf3x+7d+/GnTt3AADW1taYO3cuXF1d4ejoWOYpQ6VSiStXrsDDwwORkZFo06ZNhduoTp06WL16NZYvXw4XFxe4ubmJgRUcHIygoCCMGjWq3N6hra0t/vjjD/Tt21fs8Ui1a3ne5DyWLVuGkJAQKJVKeHt7Iz09HXZ2dnB2doa3tzeUSiX8/Pwk/+CoKIUg0d+3t7cvu/JbTnr4phISEmBtbf1W+/jmMMfue5WZDhy7j95cTWvnsLAwXLlyBQsXLtR2KaXUtHYuIfV7XvI9qaNHj1ZpQURERK9S7gCzN2/exO3bt1FQUCAuK3lqg4ioJvPw8ICHh4e2y6j1JENqzZo1iI+Px507dzBgwADExcXB2tqaIUVERBoj+eDEwYMHsWXLFjRu3BjBwcGIiIgo1aMiIiKqapIhZWBgAB0dHejp6SEnJweNGjXC/fuV+zY8ERFReSQv93Xq1AlZWVnw8vKCh4cH6tatW6GX7YiIiCqLZEh98cUXAIDRo0ejX79+yMnJQYcOHTRVF1GtVNmvSlTktQJvb+/XGg07Pj5eHFsvNjYWd+7cga+vr+T6q1atQs+ePdG3b1/J/bwJe3t7hIaGikM6VTZ/f38MHDgQQ4cOlVxH6tzehI+PD+bOnVvpI75XZo3aIBlSkydPxvDhwzF48GC0bKnZoVqISHPeZrqGwYMHiwORSpk+ffob71/u5H5uKpVK9jW+iuQ9qfHjxyMhIQFOTk7w8/PDgQMHyh2ziYiqp4pMKREXF4ehQ4di9OjROHz4sLhtWFgYgoKCkJ2djYkTJ4rj9T1//hwDBgzAixcvSk3bIbWf7777Dj///LP42dnZWZxkdcqUKfDw8ICTk1OZce7U+e233zBq1Ci4u7uLIx5kZ2fD0dERd+/eBQDMmjULv/76q3j+X331Fdzd3TFu3DhkZGSU2eeaNWvg6ekJZ2dnLFiwQGyXl8/N3t4eq1evFqe7KBm1Ijc3FwEBAfD09ISbm5s4lFBeXh5mzpwJpVKJGTNmiFNgvOzEiROlQiY+Pl6cAHLRokViu7w8JYa9vT3WrFmD0aNH48CBA6VqlDoPHx8ffP311xgxYgQcHR3FIZlKpjdRKpVQKpXYunUrAOkpRqqCZEj16tULX3zxBY4cOYJRo0YhJiYGNjY2VVYIEWnftWvXEBgYiOjoaCQnJyMhIQH5+flYsGAB1q1bh19++QWPHj0qs139+vXRunVr/P777wCAY8eOwc7ODvr6+uI6FdmPOkuWLEFYWBj27NmDrVu3iuMMqpORkYEffvgBmzZtQnh4ODp16oRNmzahfv36WLhwIQICAhAVFYWnT59i5MiRAIpDpGPHjggPD0fPnj3Vzhs1duxY7NmzB/v370deXp7kLBGmpqYIDw+Ht7c3Nm7cCABYt24d+vTpgz179iAkJARff/01cnNzsWPHDhgaGiIyMhKTJk0Sxzp8ma2tLS5duoTc3FwAQHR0tDhG38yZMxEWFoZ9+/bh3LlzuH79uridgYEBduzYIQ46W5HzUKlUCA0NRWBgoNgGu3btQnJyMsLDwxEZGQmlUokXL15g8eLFWL16NcLCwuDp6YlvvvlG8nvytsp9mTcvLw9Hjx5FTEwMrl69Cnd39yorhIi0T92UEvXq1UPLli3x73//GwDg4uIi9kJeZmtri+joaPTp0wdRUVEYM2ZMqa/fvXu3Qvv5p61bt4q9rocPH+LevXviwLL/dOnSJdy+fRujR48GUDxoa9euXcX6Dhw4gKCgIERERIjb6OjoiOMAurq6YurUqWX2Gx8fjw0bNiAvLw+ZmZlo166d2qHjSqYy6tSpk1jzb7/9hqNHj4qhlZ+fj4cPH+LcuXPw8fEBUNzW7733Xpn96enpoV+/fjh27BgcHR1x4sQJrFy5EgAQExODX3/9FYWFhXj06BHu3LkjPjcgNYNFeedRMu1GyXQcQPH8gt7e3tDTK46Khg0b4ubNm2qnGKkqkiE1Y8YMXL58GXZ2dhgzZgx69+4tDl9PRDWTuiklgLJTVKjTq1cvzJ49G5mZmbh69Sr69OlTZh2p/bw8tQcA8dZCfHw8Tp8+jV27dsHIyAg+Pj6vnCrC1tZW/EX+sqKiIty5cwcGBgbIzMwUw/hVNebn5+P//u//sGfPHjRr1gzfffedZA0lPUcdHZ1So4+vXr1a7aCtFWnX4cOHY/v27WjQoAE6d+4MIyMj3L9/Hxs3bkRoaCgaNGgAf3//UjUZGRmV2c+rzqPke/9y7YIglKlRaoqRqiKZOh4eHjh8+DCCgoJgY2PDgCKqpdq0aYPk5GRxevCoqCi16xkZGaFz58748ssvMXDgQOjq6lZ4Py1atMC1a9cAAFevXhXvR2VnZ6NBgwYwMjLCnTt3cPHixXJr7dq1K86fP4979+4BKL439ueffwIonmepbdu2WLlyJQIDA8UpO4qKisSJECMjI8sMclryi9zU1BTPnj0T162okmlESu7/lJxnz549xWk4bt68iRs3bqjdvlevXrh27Rp+/fVXDBs2DEDxdO9GRkaoX78+0tPTERcX98o63uQ8bG1tsXPnThQWFgIonspFaoqRqiLZk+rfv3+VHZSI1JPjSPQGBgYICgqCr68vTE1NYW1tLflLafjw4Zg+fbp4g72i+3F0dERERARcXV3RuXNn8ZJg//79sXPnTiiVSrRu3Vq8dCfFzMwMwcHBmDVrljhCzowZMwAAu3fvxu7du2FsbIyePXvihx9+gJ+fH+rWrYtbt27Bw8MDxsbG+Pbbb0vt08TEBF5eXlAqlWjRosVrPyI+ZcoULFmyBC4uLhAEAS1atMCPP/6I0aNHIyAgAEqlEpaWlpLvoerq6mLgwIEIDw/H0qVLkZSUhA4dOqBjx45wcnJCq1atysxzpc6bnIeXlxeSkpLg4uICPT09jBw5EmPHjsXq1auxePFiZGdnQ6VSYdy4cWjXrt1rtUtFSU7VISecqkMz5PgLsjLU1KkN5Ka6tnO3bt3EXkF1UF3b+VWkfs/zGh4REclWmct96h6DfJmVlVWVFUNEpGnVqRdVG5UJqZLpowsKCnDlyhXxscgbN27g/fffVzstMxERUVUoE1IlNzxnzpyJoKAgMaRu3rwpPudPRESkCZL3pO7evVvq5bL27dsjMTFRI0UREREB5TyC3rZtW8ybNw8uLi5QKBTYt28f2rZtq8naiIiolpMMqeDgYOzYsQMhISEAil88KxlqhIiISBMkQ8rAwADe3t7o37+/2uE8iIiIqprkPanY2Fi4urri448/BlD8AtmkSZM0VhgREZFkSK1duxahoaEwMTEBAFhaWooj4xIREWmCZEjp6uqifv36mqyFiIioFMl7Uu3atUNkZCRUKhWSkpKwdetWcQZPIiIiTZDsSS1YsAC3b99GnTp1MGvWLBgbG2PevHmarI2IiGo5yZ6UkZERZs6ciZkzZ1bqAVUqFTw9PdG0aVP8+OOPuH//PmbNmoWnT5+iY8eOWLZsWamJ14iIqPaSDKk///wTGzduxIMHD8QJrwCI7029qZCQELRt2xY5OTkAgOXLl+PDDz+Ek5MTFi5ciNDQ0DLTThMRUe0kGVLTp0+Ht7c3vLy8Km1W3pSUFBw/fhyTJk3C5s2bIQgCzp49ixUrVgAA3N3dsWbNGoYUEREBKCek9PT0Kj0slixZgjlz5uDZs2cAgCdPnsDExAR6esVlWFhYIDU1tVKPSURE1ZdkSA0aNAjbt2+Hg4NDqXtEDRs2fKMDHTt2DGZmZujUqRPi4+Ml11MoFGqXv+3gtunpGW+1fVUoLCxEevojbZchSkxUabuEKpGXl8fBkTWA7awZta2dJUMqPDwcAPDzzz+LyxQKBWJjY9/oQOfPn8fRo0cRFxeH/Px85OTk4Msvv0RWVhYKCwuhp6eHlJQUmJubq93+badLbpzM6eNfxdKS08fTm2M7a0ZNbeeEhAS1yyVD6ujRo5VawOzZszF79mwAQHx8PDZu3IgVK1bAz88PBw8ehJOTE8LDw2Fvb1+pxyUiouqrTEidOXMGNjY2OHTokNoNhgwZUqkFzJkzBzNnzsS3334LS0tLeHl5Ver+iYio+ioTUufOnYONjQ2OHTumdoPKCKnevXujd+/eAIBWrVohNDT0rfdJREQ1T5mQ8vPzA1A8nxQRESsxE8EAAA8/SURBVJE2Sd6TAoDjx4/j1q1byM/PF5dNnTq1yosiIiICyhm7b+HChYiOjsa2bdsAAAcPHsTff/+tscKIiIgkQ+rChQtYtmwZTExMMHXqVOzcuRMpKSmarI2IiGo5yZAyNDQEUDzQbGpqKvT19ZGcnKyxwoiIiCTvSQ0cOBBZWVmYMGECPDw8oFAoMGLECE3WRkREtZxkSH366acAAEdHRwwaNAj5+fmcqZeIiDSqTEhJvcRborJf5iUiIpJSJqSkXuItwZAiIiJNKRNSfImXiIjkQvKe1JMnT7B27VokJCRAoVCge/fu+PTTT2FqaqrJ+oiIqBaTfAR91qxZMDU1xerVq7Fq1SqYmZlh5syZmqyNiIhqOcme1NOnT8Un/ABgypQpOHLkiEaKIiIiAsrpSfXu3RtRUVEoKipCUVERoqOjMXDgQA2WRkREtZ1kT2rnzp14/vw55s6dCwBQqVQwMjLCpk2boFAocP78eY0VSUREtZNkSF24cEGTdRAREZUheblv9+7dpT6rVCqsWbOmygsiIiIqIRlSZ8+excSJE5GWloYbN25g5MiRePbsmSZrIyKiWk7yct+KFSsQHR0NpVIJIyMjrFixAtbW1pqsjYiIajnJnlRSUhJCQkLg6OiIFi1aICIiAs+fP9dkbUREVMtJ9qQmTZqERYsWwcbGBoIgYNOmTRgxYgSioqI0WR8REdVikiEVGhoKY2NjAIBCocBHH30Ee3t7jRVGRERU5nLfTz/9BAAwNjZGTExMqa+FhYVppioiIiKoCano6Gjx3+vXry/1tZMnT1Z9RURERP+fMiElCILaf6v7TEREVJXKhJRCoVD7b3WfiYiIqlKZByeuX7+O7t27QxAE5Ofno3v37gCKe1EFBQUaL5CIiGqvMiGVmJiojTqIiIjKkHyZl4iISNsYUkREJFsMKSIiki2GFBERyRZDioiIZEty7D6iN/XN4ZvaLqGU9PQMNE6WT00zHdpruwSiaoM9KSIiki2GFBERyRZDioiIZEtjIfXw4UP4+Phg2LBhcHJywpYtWwAAmZmZGD9+PIYMGYLx48fj6dOnmiqJiIhkTmMhpaurC39/f8TExGDXrl345ZdfcPv2baxfvx42NjY4dOgQbGxsykwPQkREtZfGQsrc3BxWVlYAiidUbNOmDVJTUxEbGws3NzcAgJubG44cOaKpkoiISOa08gh6cnIyEhMT0aVLFzx+/Bjm5uYAioMsIyND7TZvO/Bterr6/WpTYWEh0tMfabsMUWKiqlL2I7e2rqntLDd5eXkcoFoDals7azyknj17Bj8/PwQGBsLY2LjC21laWr7VceX0nkyJ9PRHaNy4ibbLEFlaVs77O3Jr65raznKTmJj41v+f0qvV1HZOSEhQu1yjT/e9ePECfn5+UCqVGDJkCACgUaNGSEtLAwCkpaXBzMxMkyUREZGMaSykBEHAvHnz0KZNG4wfP15cbm9vj7179wIA9u7di8GDB2uqJCIikjmNXe5LSEhAREQE2rdvD1dXVwDArFmz4OvrixkzZiA0NBTNmjXDqlWrNFUSERHJnMZCqkePHrhx44bar5W8M0VERPQyjjhBRESyxZAiIiLZYkgREZFsMaSIiEi2GFJERCRbDCkiIpIthhQREckWQ4qIiGSLIUVERLLFkCIiItliSBERkWwxpIiISLYYUkREJFsMKSIiki2GFBERyRZDioiIZIshRUREssWQIiIi2WJIERGRbDGkiIhIthhSREQkWwwpIiKSLYYUERHJFkOKiIhkiyFFRESyxZAiIiLZYkgREZFsMaSIiEi2GFJERCRbDCkiIpIthhQREckWQ4qIiGRLT9sFENGb+ebwTW2XUEp6egYaJ8unppkO7bVdAlUC9qSIiEi2GFJERCRbDCkiIpIthhQREcmWLEIqLi4Ojo6OcHBwwPr167VdDhERyYTWQ0qlUiEoKAgbNmxAVFQU9u/fj9u3b2u7LCIikgGtP4J++fJlvPPOO2jVqhUAwMnJCbGxsXj33Xe1XBkRER/1f5WqftRf6yGVmpoKCwsL8XPTpk1x+fLlMuslJCS81XH6m73V5lXDzBBAtrarEL1tG5eQXVuznTWD7awZNbSdpWg9pARBKLNMoVCU+mxtba2pcoiISEa0fk/KwsICKSkp4ufU1FSYm5trsSIiIpILrYdU586dkZSUhPv376OgoABRUVGwt7fXdllERCQDWr/cp6enh4ULF+Ljjz+GSqWCp6cn2rVrp+2yiIhIBhSCuptCVKUCAgJw/PhxNGrUCPv379d2OTXSw4cPMXfuXKSnp0NHRwcjR47EuHHjtF1WjZSfn4//+Z//QUFBAVQqFRwdHeHn56ftsmqskj/mmzZtih9//FHb5VQ5rV/uq408PDywYcMGbZdRo+nq6sLf3x8xMTHYtWsXfvnlF75/V0Xq1KmDLVu2YN++fdi7dy9OnjyJixcvarusGiskJARt27bVdhkaw5DSgp49e6JBgwbaLqNGMzc3h5WVFQDA2NgYbdq0QWpqqparqpkUCgXq1asHACgsLERhYWGZJ3SpcqSkpOD48eMYMWKEtkvRGIYU1XjJyclITExEly5dtF1KjaVSqeDq6oq+ffuib9++bOsqsmTJEsyZMwc6OrXnV3ftOVOqlZ49ewY/Pz8EBgbC2NhY2+XUWLq6uoiIiMCJEydw+fJl3LwpnxERaopjx47BzMwMnTp10nYpGqX1p/uIqsqLFy/g5+cHpVKJIUOGaLucWsHExAS9e/fGyZMn0b49Z8atTOfPn8fRo0cRFxeH/Px85OTk4LPPPsPy5cu1XVqVYk+KaiRBEDBv3jy0adMG48eP13Y5NVpGRgaysrIAAHl5eTh9+jTatGmj5apqntmzZyMuLg5Hjx7FypUr0adPnxofUAB7Uloxa9Ys/P7773jy5An69++PadOmwcvLS9tl1SgJCQmIiIhA+/bt4erqCqC43QcMGKDlymqetLQ0+Pv7Q6VSQRAEDB06FIMGDdJ2WVRD8D0pIiKSLV7uIyIi2WJIERGRbDGkiIhIthhSREQkWwwpIiKSLYYUVXuWlpZwdXWFs7Mz/Pz88Pz5c7XrTZw4UXyf53Wkpqa+1aje9vb2yMjIeOPtq4uwsDCOj0iVjiFF1Z6hoSEiIiKwf/9+6OvrY+fOnaW+LggCioqK8NNPP8HExOS199+0aVOsXr26ssqtscLDw5GWlqbtMqiG4cu8VKP06NEDN27cQHJyMiZOnIjevXvj4sWLWLt2LXx8fBAaGorc3FxMnDgR1tbWuHDhApo2bYrvv/8ehoaGuHfvHhYtWoSMjAzo6upi1apV0NHRwaRJk7B//36EhYXh8OHDKCgoQHJyMpRKJaZOnQoAmDJlClJSUpCfn4///d//xahRo8qtNS4uDt988w1UKhVMTU2xZcsWZGZmIjAwEPfv34eRkRGCgoLQoUMHfPfdd0hOTsajR4+QlJQEf39/XLx4ESdPnoS5uTnWrVsHfX192NvbY9iwYYiPjwcArFixAu+88w4ePHiAwMBAZGRkwMzMDMHBwWjevDn8/f1hbGyMK1eu4NGjR5gzZw6GDh0KANiwYQNiYmJQUFAABwcH+Pn5ie36z7Y7fvw4rly5gs8++wyGhobYtWsX1qxZg6NHj0JXVxd2dnb4/PPPq/abTzWTQFTNde3aVRAEQXjx4oUwadIkYfv27cL9+/eF9957T7hw4YK43qBBg4THjx8L9+/fFywtLYVr164JgiAIfn5+wt69ewVBEIQRI0YIhw4dEgRBEPLy8oTc3Fzh/v37gpOTkyAIgrBnzx7B1tZWyMjIEJ4/fy44OTkJly9fFgRBEJ48eSIIgiAuz8jIKHXclz1+/Fjo37+/8Ndff5XaNigoSPjuu+8EQRCE06dPCy4uLoIgCMLq1asFb29voaCgQEhMTBTef/994fjx44IgCMKUKVOEw4cPi8f6/vvvBUEQhPDwcMHX11cQBEH45JNPhLCwMEEQBGH37t3C5MmTBUEQhM8//1yYNm2aoFKphFu3bgkffPCBIAiCcPLkSWH+/PlCUVGRoFKpBF9fX+H3338vt+3Gjh1bqi2GDBkiFBUVCYIgCE+fPn2N7yjR/4+X+6jay8vLg6urKzw9PdG8eXNxrp3mzZuja9euardp2bIlLC0tAQBWVlZ48OABcnJykJqaCgcHBwCAgYEBjIyMymzbt29fmJqawtDQEA4ODkhISAAAbN26FS4uLhg5ciQePnyIe/fuSdZ88eJF9OjRA61atQIANGzYEEDxcE4lwzjZ2NggMzMT2dnZAID+/ftDX18f7du3h0qlQv/+/QEA7du3R3JysrhvZ2dnAICTk5M4+eCFCxfE5a6urmLNAPDBBx9AR0cH7777LtLT0wEAp06dwqlTp+Dm5gZ3d3fcvXsXSUlJkm33T8bGxjAwMMC8efNw6NAhGBoaSrYFUXl4uY+qvZJ7Uv9Ut25dyW3q1Kkj/ltXVxf5+fkVPt4/J/RTKBSIj4/H6dOnsWvXLhgZGcHHx6fcfQqCoHZiQEHNKGUl65XUrKOjA319fXG5jo4OVCpVhev/5zm83BYv1+Hr6wtvb+9Sy5OTkyvUdnp6eggNDcWZM2cQFRWFbdu2ISQk5LVqJAL44ASRyNjYGBYWFjhy5AgAoKCgQO2TgqdOnUJmZiby8vJw5MgRdO/eHdnZ2WjQoAGMjIxw586dV06f3q1bN5w7dw73798HAGRmZgIonrV53759AID4+HiYmpq+9jxYMTExAIDo6Gh069ZNPF5UVBQAIDIyEtbW1uXuw87ODnv27MGzZ88AFD/h+Pjx43K3qVevnrj+s2fPkJ2djQEDBiAwMBDXr19/rXMgKsGeFNFLli1bhoULF2LVqlXQ19fHqlWryvR4rK2tMXfuXNy7dw9KpRKdO3fGe++9h507d0KpVKJ169aSlxlLmJmZISgoCNOmTUNRUREaNWqETZs2YerUqQgICIBSqYSRkRG++uqr1z6HgoICeHl5oaioCCtXrgQAzJ8/H4GBgfj555/FByfKY2dnhzt37og9qbp16+Lrr78ud0ZYd3d3LFq0CIaGhvjpp58wZcoUsZcVEBDw2udBBHAUdKLXEhYWhitXrmDhwoXaLkUte3t7hIaGwszMTNulEFUKXu4jIiLZYk+KiIhkiz0pIiKSLYYUERHJFkOKiIhkiyFFRESy9f8AUWYDxvUZRWoAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "with plt.style.context('seaborn-whitegrid'):\n",
    "    fig, ax = plt.subplots(figsize=(6, 4))\n",
    "    plt.bar(range(4), var_exp, alpha=0.5, align='center',\n",
    "            label='individual explained variance')\n",
    "    plt.step(range(4), cum_var_exp, where='mid',\n",
    "             label='cumulative explained variance')\n",
    "    plt.ylabel('Explained variance ratio')\n",
    "    plt.xlabel('Principal components')\n",
    "    plt.xticks(range(4))\n",
    "    ax.set_xticklabels(np.arange(1, X.shape[1] + 1))\n",
    "    plt.legend(loc='best')\n",
    "    plt.tight_layout()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Example 3 - PCA via SVD"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "While the eigendecomposition of the covariance or correlation matrix may be more intuitiuve, most PCA implementations perform a Singular Vector Decomposition (SVD) to improve the computational efficiency. Another advantage of using SVD is that the results tend to be more numerically stable, since we can decompose the input matrix directly without the additional covariance-matrix step."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [],
   "source": [
    "from mlxtend.data import iris_data\n",
    "from mlxtend.preprocessing import standardize\n",
    "from mlxtend.feature_extraction import PrincipalComponentAnalysis\n",
    "\n",
    "X, y = iris_data()\n",
    "X = standardize(X)\n",
    "\n",
    "pca = PrincipalComponentAnalysis(n_components=2,\n",
    "                                 solver='svd')\n",
    "pca.fit(X)\n",
    "X_pca = pca.transform(X)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAagAAAEYCAYAAAAJeGK1AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAgAElEQVR4nO3de3wU5b0/8M8mJIQQLnKRoDSJVarUV9AGRT0WbwEFJAe13GQJcFApqCWEHKQmiBhZQJETKG2AcCpyWV4JeVUpBNRDUvvjYE9BoyK1EQUh4RIgoRoIMeQ2vz+WXbLJzM7s7szszOTzfr3ywkwmu8+Y3f3O8zzf5/vYBEEQQEREZDBhoW4AERGRGAYoIiIyJAYoIiIyJAYoIiIyJAYoIiIyJAYoIiIypE56PllzczMWLlyI48ePIzw8HMuWLUNcXJyeTSAiIpPQtQf10UcfAQDy8/MxZ84cLFu2TM+nJyIiE9G1BzV8+HA89NBDAIAzZ86gT58+ej49ERGZiK4BCgA6deqEBQsWYO/evfjd737n9bPS0lK9m0NERAYwZMiQdsdsoSp1VFVVhQkTJmD37t2Ijo4G4ApQYo0MpbKyMgwaNCjUzdAUr9H8rH59AK/RKsSuUeqzX9c5qB07dmD9+vUAgC5dusBmsyE8PFzPJhARkUnoOsT36KOP4uWXX4bdbkdTUxMyMzPRuXNnPZtAREQmoWuAio6OxurVq/V8SiIiMiku1CUiIkNigCIiIkNigCIiIkNigCIiIkNigCIiIkNigCKyMqcTSEgAwsJc/zqdoW4RkWK6lzoiIp04ncDMmUBdnev78nLX9wBgt4euXUQKsQdFZFVZWdeCk1tdnes4kQkwQBFZVUWFf8eJDIYBisiqpDYD5SahZBIMUERW5XAAV3cK8IiOdh0nMgEGKCKrstuBvDwgPh6w2Vz/5uUxQYJMg1l8RFZmtzMgkWmxB0VERIbEAEVERIbEAEVERIbEAEVERIbEAEVERIbEAEVEpAPW7fUf08yJiDTGur2BYQ+KiEhjrNsbGAYoIiKNsW5vYBigiIg0xrq9gWGAIiLSGOv2BoYBiohIY6zbGxhm8RER6YB1e/3HHhQRERkSAxQRERkSAxQRERkSAxQRERkSAxQRERkSAxQRtcfKpmQAuqWZNzY2IjMzE6dPn0ZDQwNmz56N5ORkvZ6eiJRiZVMyCN16UDt37kTPnj2xbds2bNiwAa+//rpeT01E/mBlUzIImyAIgh5PdPnyZQiCgJiYGHz//fcYN24cSkpKvM4pLS1FdNt6ICFWX1+PqKioUDdDU7xG81Pz+m67/XbYRD4WBJsNX3/1lSrPEQir/w2BjnuNdXV1GDJkSLtzdRvi69q1KwCgtrYWc+bMwdy5c0XPGzRokF5NUqSsrMxwbVIbr9H8VL2+uDjXsF4btri4kP4/tPrfEOi411haWip6rq5JEpWVlZg6dSrGjh2LlJQUPZ+aiJRiZVMyCN0CVHV1NWbMmIH58+dj3Lhxej0tEflLrrIpM/xIJ7oN8a1btw4XL15Ebm4ucnNzAQAbNmyw/HgrkSlJVTZlhh/pSLcAtXDhQixcuFCvpyMiLfjK8GOAIpVxoS5RsDrSkBf3LicdMUARBcM95FVeDgjCtSGvtkHKKkGMe5eTjhigiIKhZFGr0iBmBszwIx0xQJHhOQ87kbAqAWGvhSFhVQKchw30wa5kyMtKlRm4dznpiFu+k6E5Dzsxc9dM1DW6PuDLa8oxc5cra8yeaIAPRYlFrV5DXlabt+He5aQT9qDI0LJKsjzBya2usQ5ZJQbpfSgZ8uK8DVFAGKDI0CpqxHsZFT+UGyPZQMmQF+dtiALCAEWGFtdDvJcRVwPjJBvY7cCJE0BLi+vftsNfnLchCggDFBmaI9mB6Ajv3kd0A+BwF8I3S7KBXBAL1NX09dtuv90YPUoiFTFAkaHZE+3IS8lDfI942AQg/gcgbxdgP9zqJBMlGwSckSi2jqpV+rrN7OnrRCIYoMjw7Il2nJh7Ai3vxOPEqjbBCTBNsoE7I7G8phwCBE9GomyQklpHlZamX/q6VRYak6kwQJF5mDzZIOCMRKl1VBcuiJ+vdo/SSguNyVQYoMg8TJ5sIJmRKHH82gl+Bhy1e5RWWmhMpsIAReaiVbKBDiQzEiWOXztB4ue9e+vTo7TaQmMyDQYoIp2IZiRGRMORLBNQpIY2V6/29CgFLXuUXGhMIcIARdbUZlK/e1FRqFvknZEIG+J7xCMvJU++ZJOvoc2rPcqvv/pKux6lyef+yLxYi4+sR2TX1/6LFgE33BDyIUF7oj2wGoKhrH/nft6sLNewXlycKziZaHiVzIk9KLIekUn9sPp6TuoHw8Rzf2ReDFBkPWab1OcaIyJRDFBkPWaa1JdbY8TgRR0Y56DIehwO7zkoAC1RUQgz4qS+3BqjNnNpmOnaC4tDbNQRsAdF1iOS9VaZnW3MD3Vfw5FcIEsdHAMUWVObSf2LY8aEukXifA1Hmm0ujUhlDFBEoeRrjZGZ5tKINMAARYYX8BYVZuBrES4XyFIHJxmgvvjiCzz11FN4+umn8emnn3qOv/DCC7o0jAgIYosKM5FaY2Ty4rhEwZIMUMuXL8fKlSuRnZ0Nh8OB/fv3AwAuXryoW+OIAt6iwkx8pZJzgSx1YJIBKiIiAjfddBMGDhyIvLw8vPHGGzhy5AhsNpue7aMOLuAtKsyCey2Ryqy0dE4yQHXt2hWbN29GQ0MD+vbti7feegtz587F6dOn9WwfdXABb1FhFkwlJxVZ7X5HMkC99dZbqKmpQUNDAwDg1ltvxZo1a3Drrbfq1jiigLeoMAs1U8mtdOtMAbHa/Y5kJYmYmBj85je/8Tp2yy23IDc3V/NGEbm5K39nlWShoqYCcT3i4Eh2BFYR3Iji4ly3uWLHlXA6gaws3FZe7kqkEATXcVad6JCstnRO9zTzQ4cOITU1Ve+nJROzJ9pxYu4JtLzaghNzT1gjOLl7O+7A0prSVPJW4zk24FpwcjPzrTMFxGpL53QNUBs2bMDChQtx5coVPZ+WyFhaTxQArsDiDlL+pJKLjee0ZdZbZwqI1ZbOSQao5uZmNDQ04MUXX0RjYyMaGhpw5coVTJ06NeAni4uLw5o1awL+fergrDLHIhZYBMEVnPxJJVcSfMx660wBUWvpnFHeapJzUH/605+wbt06VFdXY+TIkRAEAWFhYbjrrrsCfrLHHnsMp06d8nlOWVlZwI+vhfr6esO1SW1muMbuRUXov2iRa+NBACgvR8uzz6LyzBlFdfaMdI23VVRAbLGGUFGBr/1o482xsYisrJT8eUtUFCpfeAEXDXLdwTLS31AralxjUhLw/vvex/x5yKKi7li0qD/q6139l/Jy4NlnW3DmTCXGjAl+Haxf1yjIKCwslDvFLydPnhTGjx8v+rNPP/1U1edSwz//+c9QN0FzWl7j1i+3CvE58YJtsU2Iz4kXtn65NbAHio8XBFc/w/srPl7Rrxvq7xjktXhs3SoI0dHej2GzXXusrQH+vzYoQ/0NNWKEa1Tr5SlF7BqlPvtl56Duv/9+bNiwAb///e89X0RKqFqmyErpSWpNFLQazxHc4zlbtrg+T1h1gtpQOmxnpLeabIBKS0tDbW0t+vTp4/kiUkLVMkVWSk9qPVEAAOHh1zLu/B3sv1oK6euvvmJQIkn+LOA10ltNdkfdrl27Ij09XbUnHDBgALZv367a45FxqVqmqNUuuc5EICsZqOgBxEXUwnHYab7Uc3cg4Y65pANfC3jbvtRENqQOWSagbA9q4MCB2L17N7777jscP34cx48f16NdZAGqlim62utwPtQbM1OA8p6AYAPKmy6Yt7q5kmX/RkmnIlPzZ9jOSEX0ZXtQZWVlXhkXNpsNmzdv1rRRZA2OZAdm7prpNcwXVJkiux1ZVVmoq7ngddg9bGi6XpTUp0Z5uSsYjR4NbNrEHhYFzd+CJXa7MV5isgFqy5YtuHTpEk6fPo2f/OQn6Nq1qx7tIgvQokyRpaqbS31qAK7j69ZJV4cwwqcHmYaRhu38ITvE9+GHHyI1NRXz58/HO++8w1p8FuLeqfb27bdrtlOt2mWKLFXdXCybr7W2wcnNjJmLpDp/Rn+NNGznD9kAtXHjRmzfvh09e/bE888/j+LiYj3aRRoz6061YtXNAeDkxZN4fvfzIWhRENpm8yllxsxF8otc8AlkWw0z7n0pG6DCwsIQGRkJm80Gm82GLl266NEu0phZd6q1J9qRl5KHrhHeQ80tQgvWfrrWnEHqxAnpIBVoIVkyLSXBx2rbakiRDVB33XUX5s2bh3PnzmHRokVITEzUo12kMTPP5dgT7ahvqhf9WV5pnjkz36QW786aZb5xGQqKVPBJS7v2vZEW02pJNkDNmzcPTzzxBMaPH4+HH34Yv/3tb/VoF2nM7HM5zUKz9HGR28/uRUU6t9BPUpMEubnmG5chRaTuo6SCzIUL184x0mJaLckGqNraWk8liZqaGuzYsUOPdpHGzL5TbbgtXPx4C0RvP/vm5GjfKDm+enZXNx5ERYXrU8bhYDCyMKlhvKKi7j6DjHsIz2rbakiRDVDPP/88/vKXv+DYsWM4duwYvvvuOz3aRRpzz+XE94iHDTbE94hHXkqeadYSzRwyU/z4J+2POROBgRMrEfZamGbZirJ8TSwEMuNNpiY1jJeT09dnkHH3rsQ63NOmuR7XTCPbsuQqz06ZMiX48rUKsZp5aJj1GmcXzRbCXwsXsBhC+Gvhwuyi2e1KMW9NhBCdCQGLr31FO6IDr6oeKF8lolUoH23Wv6E/rHSN7qLzbb9sthZBEAShd2//XhJihe2jo41Z0F7Vaua33norDh06hIaGBs8XkRHkPp6LpkVNEF4V0LSoCbmP57Yb+8hKBuoivX8vJNmKvma1O8qMN3lIDePFxjYCAFav9m8Iz6pZfbIB6uDBg0hPT8fIkSMxcuRIjBo1So92UQfmXkAc0JBcm7GPih7ip+merehrVrujzHiTh9QcUnp6FYD2Q3i9ewNdugCpqeLDd1a9x5ENUDt37kRJSQkKCwuxd+9elJSU6NEu6qBUWUDcakViXE/x9UW6Zyv6mtXuKDPe5CGVtNl6x1r3y3jLFuDHH11ZfFJTlMHc4xh5VYZsgDpw4ACGDx+OZ555BiNGjMDHH3+sR7vIwnz1kNReQGyYbEVftWbUqkNj5E8aakdpZQep4bspU1wvl+HDA7/HMXx+jtyE1qRJk4SzZ88KgiAIZ8+eFcaNGxfkFJk0JkmEhhbXKLXV+9YvtwrRjmjJpAXbYpvXz9xftsW2oNrS/83+wW8773nAra7ZaptNemt1Jeeo6NSbb5pnljxAHfW9KJVQ0forOTmwl5zW27uL8SdJQraaeXh4OPr16wcA6NevHzp37qx50CRzcw/TuXtC7mE6wHcPyZ5oR1yPOJTXtK/wHcyQnD3RjqROSRg0aFDAj+HhvuX0tQWGknN8PX4A66H65uQo35GOTMVX0Xu3khKguNj/P7XR565kh/hiYmKwZcsWfP3119iyZQt69JCYdSa6ylcQkiuxZJghOSlK0qUCTakKYrwl4uxZ8R8Y5ZOGAiZX9N4XuVFfo+fnyAaoFStW4MyZM1i1ahUqKyuxdOlSPdpFJuYrCMmVWDL8AmIlt5yB3pYGkSvcGBsr/gOjfNJQwAIteq/kfsfo+TmyAapbt25ISkpCUlIS7r77bvagSJavIKSkh6RkD6mgUtGDoeSWM9Db0iDGW6rS0439SUNBcSdUJCeL/zwysn0vKS1N/n7H6PtEyQaorKws7NmzB507d8aOHTvYgyJZvoKQGj0ksVT01HdTYXvNpn2wUnLLGehtaRDjLRfHjDH2Jw2pori4fZCy2YCGBu9e0vPPu9LSxbS93zHyPlGySRLffPMNCgsLAQDTpk3DhAkTNG8UmZvcVu/2RHtQQ3Zic1wCXLvPtk7I0GRY0P3u9ZXIoOQcMaNHA2vXih9X2jYjfbqQJlrvGZuQ0D6Boq7OdW8ixUyjvrIBKi4uDidPnsRPfvITXLhwAf3799ejXWRywQYhX+SqQLTOCtSEkkAQSLDYs8e/49ThSY3+NovvRgPAXKO+skN8X3zxBUaNGoVHH30UjzzyCP72t7/hl7/8JX75y1/q0T6idpSknFf8UK7+YlWtF8IqnYPigly6Sqo3FC6+Gw169zZXJ1s2QJWUlOAf//gH/ud//geHDh3CJ598gv3792P//v16tI9CqG0iwvO7nw9NYkIbYnNcbcXVQJ1l8e5gYLO5CqGpseReKsAomYOSSM0y/IaMpCr3S6i83PXSbC062vUSEZsGXb1atyaqQjZA/eUvf8GLL76I5557zvNF1ieWiLD207XB1chTSetECwCwwfsdGt0AONwlI4Mp6dw6GACugNCaex9uf3ozvnJ/5ZIrnE7Xpj9G3ZCRdCH2snQHqdYbMVshZ0Z2DuqNN95AdnY208s7GLFEhLY0n+vxofUcl/Ow05WQ8UM54mpcwcl+uNXJgS5WFVuX1NaFC9fSpZRUjPC11unEiWvntE2ucH8qSUwuSC7UJcsRewkJgisIuV9CgDVyZmQD1MCBA3HPPffo0RYyEKXbUei+bYUIT7ASS2kCAk9bCiSwyZUXkptnkvpUkQmWjbGxiJT8KVmJ0csTqUl2iC85ORkTJ07Eyy+/7Pki61Na+06XbSuUJgWovSxei8Cm9iJeAIiOdi3UpQ7B6OWJ1CQboLZs2YJp06Zh9OjRni+yPiWJCMHWyFNUDcKf+nRqL4v3VQQtPt6VEiWmbVJD6+A6erS6i3jDw4G8PNdCXeoQjF6eSFVypdGfe+654OurK8TtNkJD6hrbbpkxu2i26BYagZDbdsNDpf0AAv47zp7dfr8D9zYWW7f63uJC6uezZ/u/L4LMc3Xk16mVKL1GnXdzUZWq221ERUXhmWeewc9//nPYrqaKzJs3T/PAGQzPpLlIFQNSTsvFtnLbbnioNOBeVF6EUR+O8v81sWePePaeXFKD+7hYQsSePd6z2UoEWp2CLMkKCRBKyAaohx9+WLUna2lpweLFi3HkyBFERkZiyZIliPe3RK8MX3sRMUgZh9y2Gx5Sm+H4MeDuPOzEok8Xob65HoDrNTHl3SlIez8Nq0et9v26CDSpQcnv+qujfCoRXSU7B5WSkoK6ujp8+eWXuHjxIh5//PGAn6y4uBgNDQ0oKChARkYGli9fHvBjSVF7y3DShty2Gx4qDLhnlWR5glNrF368gJnvzfC9lksqEIaFySdtdKTZbNJVRykmYhOEtuMX3jIzM9G9e3fcddddOHjwIH744Qe8+eabAT3ZsmXLMHjwYE+QGzZsGP73f//X8/PS0lJEB7oz11W3b7/dUzi0NRts+GrCV34/Xn19PaKiooJqk9GF4hqLyou8ejUAEBUehey7sjEm3nvCv3tREfrm5CDi7Fk0xsaiKj1dUVJAUXkRcg7noLKu0ud5N9p6Yu/4v4n+rHtREfovWoSw+mvtFACvpcEtUVGozM5u1yax35U6N1h8nVqDkmssKuqORYv6o77+Wv8iKqoF2dmVGDPmos/fy8npi7NnIxAb24j09Cqf52tF7Brr6uowZMiQ9ifLTWhNnjzZ6/uJEycGNDEmCIKQmZkp/PWvf/V8/+CDDwqNjY2e79VIkojPifeaeHd/xefEB/R4nJjVTtskjGCSLsQeu20ShtSX7VWZt0HrGenwcP+SNnSazebr1BqUXGMgeUNy+Tx68idJQnaI78qVK/jxxx89ka/ZV5lcGTExMbh8+bLn+5aWFnTqJDsN5hfDbxlOHko2JgyUkkoYbnE1Mie03jCnpUX8HKl5JSNvtkOa02IoLpCpzSA2aw4p2QA1depUjB07Fi+88ALGjh2L6dOnB/xkSUlJ2LdvHwBXlfSf/exnAT+WFMNvGU66kKxw0Wb0N7oBcHwhsZ5JDOeVSCF/lvD5I5CXoFmrT8h2X/793/8dDzzwAE6ePIkBAwbguuuuC/jJRowYgY8//hiTJk2CIAia7c6rZXo0mUNcjziU17TP/uv9IxDTAFT0cPWcHP8bAXu6HyWeHQ7Xp0zr21HLrpKkYPjqtQTTkRZ7CdpsrjXgTqf4SgQVkmFDQrIHVVtbi4yMDNTW1qJnz54oLy9HdnY2amtrA3+ysDBkZ2cjPz8fBQUFuPnmmwN+LCJfpIZ6VyfMxon34tGSbcOJ9+JhT9/o36eF2tUqyLK06rXY7a6i9q232RAE4I9/BP7jPwIrlG9Ukj2oV199FYmJiejatSsAYOTIkTh37hwWL16Mt956S7cGEgWi7bbzsdGxWDFyhev47GAfnOuRSJ6WvRax9eMNDe3PU7qm3Kgke1CVlZWYPn26p3pEp06d8Mwzz+DkyZO6Na4jU1SnzqT0urbWSRglY0raD/t2lMUkFBJa9lr86YW1XlNutnwdyQAVFib+o4iICM0aQy5imwWGanNAtbiDku01G1LfTQ39tWk1g010VaCjwUrum3r1Ut4Oo88z+SIZoOLj41FcXOx1rKSkBH379tW8UR2dUaphqNXTaR1wAbRbSB2SSh9mzbslU/G311JU1F3yvql14Pr+e2XPb4Z5Jl8k56AWLFiAefPm4Q9/+AMGDBiAyspK9OrVK+AqEqSc4jp1GlKzpqGSNUkVNRXSKUhaMGveLVlaTk5f0fumtDTgxx+v3VP5qv8TH2+ueSZfJANU9+7d8d///d84c+YMzp8/j/79+6Nfv356tq3DkkqR1mVzwKsUVxtXQElgjevUyzt3Vsn26cEwa94tWdrZs+JTKBcuKPv9ttu+m53sQt0bbrgBd955J4OTjoxQDUPNXpxcYI2OiIajGPoOuZk175YsLTa2MeDfteLLVzZAkTc9MtCMUA2jVxfxWdhAenFiAdd2tdyq59r+37/Ef1mrITclM9jM8iOdpadXtbtvar3eqa3wcGsvx2OA8oOe2XVydeq0DJTOw07UXGlfoC4iLCKgXpxYwN3y1BYIrwrXru3q0JozEUiYC4S96vrX+aAf6Up+N8zHDDaz/Egjvu57xoy52O6+ydd806ZN5kob95fkHNTEiRM9a6DcBEGAzWZDfn6+5g0zIjXnZYKh9aaMae+noamlqd3xzp06B/z4suWnHA44c/4DMx9rRF2k61B5T2Dmw5eAw079S1dpVaeGOpS2eT+jR7uCiq+p1rbrwBMSxKdLe/e2/ktRMkD913/9l57tMAUjZNcB2gfKCz+Kz8jWNgRe5kqW3Y6sE2moa/J+7jqhQfcbAADM8qOguTvhrYPRunXte0Ry9z1S5R9XtyohqWcCrJ4kh/huvPFG3HjjjWhqakJRURHee+89vPfee1i/fr2e7TMUxbvAaswogVJtFU3i81AhuS5WLacgiXXCpYbrystdPaWiou7tfiY3Xfr880Bqqr6j0XpNz8rOQS1YsAAA8Nlnn+HUqVP44YcftGmJCRghuw7QPlD27iK+/YTUcbWodl1K3z2+zmOWHwXJ3852eTmwaFF/0Zdr6+lSh8MV/MLCgD59gLVrpXtlWtBzelY2QEVFReHXv/41+vXrh+XLl6O6ulr9VpiEEbLrAOlAOXrgaFUSJ1aPWo2IsPbrMSbcPiGgx1NKlRsAiXdP96IiRed53mWsWk5Bkups+8rKq68PkwwsTqcrIE2Zcu1l62t9lFaj0XoWYZENUIIgoKqqCpcvX0ZdXR1qauS2H7U2LXeB9acNbQPltDumYdOhTapkGNoT7Xg26VlPKrjbpkObNK2Zp8oNgMS7p29OjqLzvN5lZqyuSYYh1QmfNct1vyNFLLC476eULtgFtBuN1nN6VjZAvfjii9i7dy/Gjh2L5ORkPPDAA+q3gvzWNlDu+XaPqvX79ny7JyQ184K+AZB4l0ScPavoPK/jXAdFQZDqhOfmuu53pIKUWGARu5/yxWbTbjRaz+lZ2QB19913Y+TIkejTpw/ef/99z5wUGYu/iROt11ElFyW36xkZLhFDabCQeJc0xsYqOs9znOugSAW+OuFiPayoqBbRwOJP78Rmc/XStOrw6zk9KxugnE4nJk2ahLy8PEycOBF//vOf1W8FBc2fBIO2C44r6yrbDQcaJWMRgH/BQuLdU5Werug8z7uM1c5JY2I9rOzsStHA4qt3EhnpWhPlfowtW1y9ND3brdX0rGyAKiwsxK5du/CHP/wBO3bswObNm9VvBQXNnwQDJdt5OJId7RIl/KkkoWqlC3+ChcS75+KYMYrO87zLuA6KdNC2hwWIDxSI3U8BrsD09ttAdbV0QRQtRqn1mp6VXKjr1rt3b4SHhwNwZfT17NlTm5ZQUNpucR7XIw6OZIfoHI7c8J3zsBNp76ehscW7cGXbyiJSAq104TzsFG+/v8FCbEv2sjJl57mx2jnpzOl0pZnX17u+F6sy4c9iXLGFwlpuEKAFmyD4qvQEzJgxA+fPn8cvfvEL/POf/0RTUxNuueUWAMDKlStVbUxpaSmGDBmi6mMGq6ysDIMGDQp1M1SVsCpBdDuP+B7xcCQ7vIKL2Dkn5p4I+PGlfrdtUANcPcC8lDzYU7LEg4Ufewv4/Xds++4GXLewBk01t+LrtC2rX6NUSaNAt9BQ+/HUIvZ3lPrsl+1BzZo1y/PfKSkpKjSPQk0sCLmHA+U2F1SSJBFIgoXP8k1StV60XDQbyC0rURDUHlW2wii15BzURx99BAD47rvvcPz4ca+voUOHYujQobo1ktTVdr1R/+j+nvVGcgFISZJEIAkWPoNaKBbNWrW4GRmW2unbVqjWJRmg3CWNqqurUVVV5fVF0vTYL0oNrdcblYwp8cwN+QoiSqs6BFIRQjao6blolinmFAIOhyvNvLVgBgqsUK1LMkA9+eSTAFzDegkJCXjxxRdRX1+PJ554QrfGmY2e+0VpRSy4AK46fEqrOgRSEcIodQ4BMMWcQsJud6WZqzVQYIVqXYqKxfbt2xcA8OCDDyKLb1JRzsNOTHtvmqrVHEJBLLhsfWorql+q9quqg78VIYxS5xCAuoP3rEZBfhgz5qKqAwVmr9alaEfde+65B4CrqkRLS4vM2R2Pu+fULDSL/txo22C0HYYsKsS/LnwAABT/SURBVPcupNo6uLgTJ/QYsgyqzJGagUCtwXsOFRIFRTZAde/eHQUFBThy5AgKCwvRtWtXPdplKnKZbyGpviDBediJGX+e4TUMmfVJlmjgMc2QpdqBQK3Bew4VEgVFNkAtX74cR48exYoVK3Ds2DEsXbpUj3aZiq8eUsjmUSSkvZ+GhuYGr2ONLY1Iez+t3blKKk4YgtqBQK3Beyvk+RKFkOw6qF69emHWrFm4cuUKAKDevcyZPOJ6xIkuTA23hYduHkWC1HbuYscNVzBWihaBwFeVCaVYjYI00JFWQMgGqMWLF2Pfvn24/vrrIQgCbDYb8vPz9WibaUgtfDVacPKXVOA10pAlAOMGglAsMCZLs0L5In/IDvF9+eWXKC4uRn5+PgoKCoIOTnv37kVGRkZQj2E0hspAkyG1bXvXiK7t1m8pSf02xLovoy74sEKeLxlKR5vWlA1Q8fHxnuG9YC1ZsgQrV660ZCagEXbaVUJsO3cbbGhsaWyXDAHAZ+BVM4kiqEBn5EBg9jxfMhSpUevycmsmh8oO8VVWVuLhhx9G/NXtH4MZ4ktKSsLw4cNRUFAQ0O9T8FpXPS+vKUe4LRzNQnO7xAl3MoSvYOuzfp4fATrQ6ufeF6bCnBGRwUmNZgPWHOqTrWZ++vTpdsduvPFGnw9aWFiITZs2eR1bunQpBg8ejAMHDiA/Px85OTntfq+0tBTRYpuehFB9fT2ioqJC3QzVFZUXYdGni1DfLJ30YoMNX034SvLnt2+/vd228K1/r6i8CDmHc3C27ixio2ORnpiOMfFj2p2fXJSMyrrKdsf7R/dHyZgShVfkm1X/jm5Wvz6A11hU1B0ORz/U1IQDEN/6pn//BpSUHNOwhcETu8a6ujr/qpkXFhZi/PjxyM/Pb7cP0Lx583w2YPz48Rg/frw/bfYwWjl9q5b4H/XhKJ/BCXAlQ/i6dl9JFJ81fYbFny329Ioq6yqx+LPFuOHGG9r1is5uPyv6+Gfrzqr2/96qf0c3q18f0LGv0ekEFi9uP//U1tmzkYb/fyS13YYYyTmo2NhYAK45qJtuusnri8xPLlVcyfotX0kU/qyhMtT28kQGJJYcISbUiatqkwxQw4YNAwDs2bMHTz75pNcXmZ+vD3+lWYi+shf9WUNlqEKxRAakZEmfO3HVXfXLZgM6dXL9a9YykLJJEt26dUNJSQkSEhIQFuaKZ8H0ou655x5PbT8KHbXWbtkT7aLn+7OGyp/t6ok6GqfTVWKyWaTUZ3i4K0HUvWAX8F4n5f4ds66Xkg1Q//rXv/DOO+94vrfZbNi8ebOWbSIdaB0UfO3aK9UeBiQib+6FuWLBKTq6/WqKhATpoUD3einLBKja2lrk5eWhS5cuerWHdOQOClpMPrNXRBQ8qbmn8HDxpX5yQ4FmKwMpGaC2bt2Kt99+G506dcIrr7zimZMiUoq9IqLgSAWUlhbxnpCvdVLun5uJZJJEUVERPvjgA+Tn57db00RERNrzd2sysapfbkao/uUvyQAVGRmJyMhI9OrVC42NjXq2iYiI4H+ZydZVvwDXUCBgrOpf/pBNkgAAmWITRESkAXdA8Wd7DStV/ZIMUEePHkVGRgYEQfD8t9vKlSt1aRwRUUdnpYDjL8kAtWrVKs9/T5o0SZfGEBERuUkGqKFDh+rZDiIiIi+y+0EREZE63GWIwsLMW35IT4qSJIiIKDgdbbt2NbAHRUSkg462XbsaGKCIiHQgVRVC7LivocCONEzIIT4iIh1IlSFqWxXC11Ag0LGGCdmDIiLSgdKqEL6GAjvaMCEDFBGRDlqXIbLZpMsP+RoK9GeY0Ao4xEdEpBMlVSHkhgKVDBNaBXtQREQG4mso0N/isWbHHhQRkYEoKRDrT/FYM2OAIiIyGF9DgR2peCyH+IiIyJAYoIiIyJAYoIiIyJAYoIiIyJAYoIiIyJAYoIiICIDxCtEyzZyIiAy5XxV7UEREJqJVL8eIhWgZoEzOediJhFUJCHstDAmrEuA8bOHNYYg6OHcvp7wcEIRrvRw1gpQRC9EyQJmY87ATM3fNRHlNOQQIKK8px8xdMxmkOoCWlhYsWrQIEydORGpqKiorK0PdJNKBlr0cqYKzoSxEywBlYlklWahr9H611jXWIavEopvDmJjawzLFxcVoaGhAQUEBMjIysHHjRjWaSQanZS/HiIVoGaBMrKJG/FUpdZxCQ4thmdLSUgwbNgwAcOedd+Lo0aMqtZaMTMtejtL9qvSkW4C6dOkSZs2ahSlTpmDixIn4/PPP9Xpqy4rrIf6qlDpOoaHFsExtbS1iYmI834eFhaGpqSnwByRT0LqXY7cDJ04ALS2uf0NdlFa3ALVx40bce++92Lp1K5YtW4bs7Gy9ntqyHMkOREd4v1qjI6LhSLbo5jAmpcWwTExMDC5fvuz5XhAEdOrEVSNWZ8RejpZ0e0VPnz4dkZGRAIDm5mZ07txZ9LyysjK9mqRIfX294drkltQpCYuTFiPncA7O1p1FbHQs0hPTkdQpya82G/ka1RLKa4yNvRmVlZEixxtQVnYsoMe8/vrrsXPnTtx00004cuQIBgwYwL+hBSi5xqQk4P33vY+Z6X+LP39HmyAIgtoNKCwsxKZNm7yOLV26FIMHD0ZVVRWee+45ZGZmYujQoV7nlJaWYsiQIWo3JyhlZWUYNGhQqJuhKV6jttougARcwzLB3Pm2tLRg8eLF+OabbyAIAp599lmMGDFCnQYbFF+n1iB2jVKf/Zr0oMaPH4/x48e3O37kyBHMmzcPL730UrvgRGRVSnZI9VdYWJjXMLnVexbUMek2xHf06FGkpaVh1apVuO222/R6WiJD6Ei7oBKpRbcAtXLlSjQ0NMBxNd0kJiYGa9eu1evpiYjIZHQLUAxGRESh43SqO8ysB+alEhFZnBErlSvBShJERBZnxErlSjBAERFZnBErlSvBAEVkYocOHUJqamqom0EGZ8RK5UowQBHpQYNd5jZs2ICFCxfiypUrQT8WWZsRK5UrwQBFpDWNdpmLi4vDmjVrVGokWZlZa/gxQBFpTaMZ6scee4wFYkkxo1UqV4IBikhrZp2hJgoxBigirZl1hpooxBigiLRm1hlqohBjgCLSmoYz1AMGDMD27dtVaCSR8XCGlUgPLGdO5Df2oIiIyJAYoAzKediJhFUJCHstDAmrEuA8HPzCTiIiM+EQnwE5Dzsxc9dM1DW61s6U15Rj5i5X6WF7IoeJiKhjYA/KgLJKsjzBya2usQ5ZJQYvPUxEpCIGKAOqqBFfwOk+zuE/IuoIOMRnQHE94lBeUy56nMN/BACNjY3IzMzE6dOn0dDQgJSUFAwaNCjUzSJSFXtQBuRIdiA6wnthZ3RENBzJDg7/mZTavd6dO3eiZ8+e2LZtGzZs2IC8vDyVWkpkHAxQBmRPtCMvJQ/xPeJhgw3xPeKRl5IHe6JddviPjMfd6y2vKYcAwdPrDSZIjRw5EmlpaZ7vw8PD1WgqkaEwQBmUPdGOE3NPoOXVFpyYe8IzfBfXQ7x+m9RxCj0ter1du3ZFTEwMamtrMWfOHEyePDnYZpIJabDNmKEwQJmMr+E/Miater2VlZWYOnUqxo4diwcffDCoxyLz0WibMUNhgDIZX8N/ZExa9Hqrq6sxY8YMzJ8/H+PGjQv4cci8NNpmzFCYxWdC9kQ7A5KJOJIdXpmXQPC93nXr1uHixYvIzc1Fbm4uLl++jG3btiEqKkqNJpMJdIRtxhigiDTmvpnIKslCRU0F4nrEwZHsCOomY+HChVi4cKHn+7KyMganDiYuzjWsJ3bcKhigiHTAXi+pzeFwzTm1Huaz2jZjnIMiIjIhDbcZMwz2oIiITMrq24yxB0VERIbEAEVERIbEAEVERIak2xxUXV0dMjIyUFNTgy5dumDFihXo1auXXk9PREQmo1sPavv27bj99tuxbds2PP7448jNzdXrqYmIyIR060FNnz4dzc3NAIAzZ86gT58+ej01ERGZkE0QBEHtBy0sLMSmTZu8ji1duhSDBw/G1KlT8c0332Djxo3tNlgrLS1VuylERGQCQ4YMaXdMkwAl59ixY/j1r3+N4uJivZ+aiIhMQrc5qPXr12PHjh0AgOjoaG6wRkREPunWg6qursaCBQvQ0NCA5uZmZGRkiHbpiIiIgBAN8REREcnhQl0ZdXV1mD17NiZPnoxnnnkG//rXv0LdJNVdunQJs2bNwpQpUzBx4kR8/vnnoW6SZvbu3YuMjIxQN0M1LS0tWLRoESZOnIjU1FSUi+2/YAGHDh1CampqqJuhicbGRsyfPx+TJ0/GuHHjUFJSEuomqa65uRkvv/wyJk2aBLvdjgqFm1YxQMnoCOu3Nm7ciHvvvRdbt27FsmXLkJ2dHeomaWLJkiVYuXIlWlpaQt0U1RQXF6OhoQEFBQXIyMjA8uXLQ90k1W3YsAELFy7ElStXQt0UTezcuRM9e/bEtm3bsGHDBrz++uuhbpLqPvroIwBAfn4+5syZg2XLlin6PVYzl9ER1m9Nnz4dkZGRAFx3Op07dw5xi7SRlJSE4cOHo6CgINRNUU1paSmGDRsGALjzzjvxj3/8I8QtUl9cXBzWrFmDl156KdRN0cTIkSPx2GOPeb63YgLZ8OHD8dBDDwHw73OUAaoVpeu3zMzXNVZVVWH+/PnIzMwMUevUIXWNo0ePxoEDB0LUKm3U1tYiJibG8314eDiamprQqZN13tqPPfYYTp06FepmaKZr164AXH/LOXPmYO7cuSFukTY6deqEBQsWYO/evfjd736n7JcEUuzo0aNCcnJyqJuhia+//loYPXq08Ne//jXUTdHU3//+d2Hu3LmhboZqli5dKuzevdvz/bBhw0LYGu2cPHlSGD9+fKiboZkzZ84ITz75pFBYWBjqpmju/PnzwkMPPSRcvnxZ9lzOQcnoCOu3jh49irS0NKxcuRIPPvhgqJtDfkhKSsK+ffsAAF988QV+9rOfhbhF5K/q6mrMmDED8+fPx7hx40LdHE3s2LED69evBwB06dIFNptN0WepdcYBNPKrX/0KCxYswJ/+9Cc0Nzdj6dKloW6S6lauXImGhgY4HA4AQExMDNauXRviVpESI0aMwMcff4xJkyZBEARLvj6tbt26dbh48SJyc3M9SVgbNmxAVFRUiFumnkcffRQvv/wy7HY7mpqakJmZqWium+ugiIjIkDjER0REhsQARUREhsQARUREhsQARUREhsQARUREhsQARaZ24MAB3HfffUhNTUVqaiomTJiALVu2tDtv3759fpc4evfdd/0u3Hnq1ClMmDCh3fGamhpkZmbCbrdj0qRJSE9Px6VLl/x67FArKChAY2Oj6M+sVoSXjIHroMj07r33XuTk5AAAGhoaMHLkSIwdOxbdu3f3nPPAAw/4/bhPPfWUam2cN28eJk2ahBEjRgAA3nnnHSxatMjTbjNYv349nnjiiXbHlyxZgv3792PQoEEhaBVZGQMUWUptbS3CwsIQHh6O1NRUXHfddbh48SIef/xxlJeXY9KkScjIyEBsbCxOnjyJxMREvPbaa7hw4QJ++9vf4tKlSxAEAW+88QZ27dqFPn364Kc//SnWrVuHsLAwVFVVYeLEibDb7Th48CB+//vfAwDq6+vxxhtvICIiol2bTp8+jerqak9wAoDU1FT86le/AuCqZr1p0yZERkYiISEB2dnZ2LVrFz766CPU19ejqqoKU6dORUlJCb799lu89NJLGD58OJKTk3HHHXegoqICAwcOhMPhQG1tLebPn4/a2lo0NzcjLS0N9913H1JSUjB06FAcOXIENpsNubm56NatG1auXIlPPvkEgiBg+vTpGDVqFFJTU3Hbbbfh22+/RW1tLVavXo2//e1vqKqqQnp6eruK/lYswkvGwABFpvf3v/8dqampsNlsiIiIwCuvvOIpwJmSkoIRI0bg3Xff9Zx/4sQJ/PGPf0SXLl0wfPhwVFVVYf369XjkkUfw9NNP4//+7//w5Zdfej3HuXPnsGPHDrS0tCAlJQUjR47Et99+ixUrVqBfv35Yt24dPvjgA6SkpLRr3/nz5zFgwACvY+Hh4ejWrRu+//57rFmzBu+99x5iYmKwdOlSFBQUIDo6GpcvX8bbb7+N3bt345133sH27dtx4MABbN68GcOHD8e5c+eQlpaG+Ph4pKWlobi4GJ9//jn+7d/+DdOmTcO5c+fw9NNPo7i4GJcvX8bjjz+OV155BRkZGdi3bx9iYmJw6tQp5Ofn48qVK5gwYQLuv/9+AMDgwYORlZWFnJwc7N69GzNnzsTatWtFe3xWLMJLxsAARabXeoivrZtuuqndsbi4OE8F8L59++LKlSs4fvy4pw7afffdBwBYs2aN53d+8YtfeLYkGThwICoqKtCvXz84HA5ER0fj3LlzSEpKEm3DDTfcgLNnz3oda2xsxAcffID4+Hjccsstnvbcfffd2L9/P+644w7PkFm3bt1w8803w2azoUePHp59kfr374/4+HhP+44fP45jx455gmS/fv0QExPj2WTz5z//uef3rly5gjNnzuCrr77ybATY1NSEM2fOeJ0bGxuL6upq0esi0hqTJMjSbDabomM333wzDh8+DAD45JNPsGLFCq+fl5WVobm5GT/++COOHj2K+Ph4LFy4EEuXLsXy5ctx/fXXQ6pqWL9+/XDdddehuLjYc2zz5s0oLi7GgAEDcOzYMdTV1QEADh486AmqYu1s7dy5c6iqqgIAfPbZZ7jllltw880349NPP/X8/OLFi+jZs6fo4/30pz/FPffcgy1btmDTpk0YNWpUu55eazabzVKbPZLxsQdFBGDWrFnIzMzEzp07Abj2j3JXsQdcvYvnnnsOP/zwA2bPno1evXph7NixmDBhArp3744+ffrg/Pnzko//5ptvIjs7G2+//TYaGxsRFxeHJUuWoFu3bvjNb36DqVOnIiwsDHFxcfjP//xP7N69W7bNkZGReP3111FZWYk77rgDjzzyCIYMGYLMzEx8+OGHqK+vR3Z2tuTeUI888ggOHjyIyZMno66uDsOHD/faW6qtu+66CzNnzsTmzZtlgyeRGlgslkjGgQMHkJ+fb7iMu/vvvx8ff/xxqJtBpBkO8RERkSGxB0VERIbEHhQRERkSAxQRERkSAxQRERkSAxQRERkSAxQRERnS/wcMreQE4ac56QAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "import matplotlib.pyplot as plt\n",
    "\n",
    "with plt.style.context('seaborn-whitegrid'):\n",
    "    plt.figure(figsize=(6, 4))\n",
    "    for lab, col in zip((0, 1, 2),\n",
    "                        ('blue', 'red', 'green')):\n",
    "        plt.scatter(X_pca[y==lab, 0],\n",
    "                    X_pca[y==lab, 1],\n",
    "                    label=lab,\n",
    "                    c=col)\n",
    "    plt.xlabel('Principal Component 1')\n",
    "    plt.ylabel('Principal Component 2')\n",
    "    plt.legend(loc='lower center')\n",
    "    plt.tight_layout()\n",
    "    plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "If we compare this PCA projection to the previous plot in example 1, we notice that they are mirror images of each other. Note that this is not due to an error in any of those two implementations, but the reason for this difference is that, depending on the eigensolver, eigenvectors can have either negative or positive signs.\n",
    "\n",
    "For instance, if $v$ is an eigenvector of a matrix $\\Sigma$, we have\n",
    "\n",
    "$$\\Sigma v = \\lambda v,$$\n",
    "\n",
    "where $\\lambda$ is our eigenvalue\n",
    "\n",
    "then $-v$ is also an eigenvector that has the same eigenvalue, since\n",
    "\n",
    "$$\\Sigma(-v) = -\\Sigma v = -\\lambda v = \\lambda(-v).$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Example 4 - Factor Loadings"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "After evoking the `fit` method, the factor loadings are available via the `loadings_` attribute. In simple terms, the loadings are the unstandardized values of the eigenvectors. Or in other words, we can interpret the loadings as the covariances (or correlation in case we standardized the input features) between the input features and the principal components (or eigenvectors), which have been scaled to unit length.\n",
    "\n",
    "By having the loadings scaled, they become comparable by magnitude and we can assess how much variance in a component is attributed to the input features (as the components are  just a weighted linear combination of the input features)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [],
   "source": [
    "from mlxtend.data import iris_data\n",
    "from mlxtend.preprocessing import standardize\n",
    "from mlxtend.feature_extraction import PrincipalComponentAnalysis\n",
    "import matplotlib.pyplot as plt\n",
    "\n",
    "X, y = iris_data()\n",
    "X = standardize(X)\n",
    "\n",
    "pca = PrincipalComponentAnalysis(n_components=2,\n",
    "                                 solver='eigen')\n",
    "pca.fit(X);"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAjgAAADQCAYAAAAK/RswAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAgAElEQVR4nO3debwcVZ3+8c9DZN93kC0gCC6ASljc+KGIgjigIoswowiKioiMOoqojOIyARRXRCOggAsKokQBGQFRB0EIyq5IDCiRxbCD7PD8/jh1tb3ce9NJd93q5Xm/Xv263dWVW99Kd7459T11zpFtIiIiIgbJIk0HEBEREdFtaeBERETEwEkDJyIiIgZOGjgRERExcNLAiYiIiIGTBk5EREQMnDRwImJoSDpR0t8kXTPO+5L0RUmzJV0l6QWTHWNEdEcaOBExTL4J7DjB+zsBG1WPA4DjJiGmiKhBGjgRMTRs/xK4a4JddgVOdnEJsIKkNScnuojopqc1HUC3rbLKKp46dWrTYUQMpcsvv/wO26s2HUcH1gJubnk9t9p26+gdJR1AqfKw9NJLb7HJJptMSoAR8a/GyzsD18CZOnUqs2bNajqMiKEk6c9Nx9AhjbFtzPVsbM8AZgBMmzbNyTsRzRgv76SLKiLin+YC67S8Xhu4paFYIqIDjTZwMqIhInrMTOBNVe7ZBrjX9lO6pyKi9zXdRfVN4MvAyeO83zqiYWvKiIatJyWyiBg4kr4LbAesImku8N/AogC2vwqcDbwamA08CLylmUgjolONNnBs/1LS1Al2+ceIBuASSStIWjNXVBGxMGy/cT7vG3jXJIUTETVquoIzP22NaGgdzbDuuutOWnAxOaYeelbTIbTtpuk7Nx1CRETQ+zcZtzWiwfYM29NsT1t11X4eoRoRERHd0OsNnIxoiIiIiAW2UF1Uknaw/bNuBzOGmcBBkk6l3FycEQ0xMNL1NjZJywGr2v7TqO2b2b5q0gKJiL62sBWcE7px8GpEw8XAxpLmStpf0jskvaPa5WxgDmVEw9eBA7tx3IjoTZL2AP4A/EDStZK2bHn7m81EFRH9aNwKjqSZ470FrNyNg2dEQ0SMchiwhe1bJW0FnCLpMNtnMPY9eRERY5qoi+qlwL8DD4zaLmCr2iKKiGE2ZaQb2valkl4G/ETS2oyzZEJExFgmauBcAjxo+xej35B0fX0hRcQQu1/SM0buv6kqOdsBPwKe02hkEdFXxm3g2N5pgve2rSeciBhy72RUV5Tt+yXtCOzRTEgR0Y8mugdnQ2B12xeN2v5S4JbRIxz6Ub+MYsnkcTFE/g6sThlY0GobSlU5IqItE42i+jxw/xjbH6rei4jotuSdiOiKiRo4U8eac8L2LGBqbRFFxDBL3omIrpiogbPEBO8t2e1AIiJI3omILpmogXOZpLeN3ihpf+Dy+kKKiCGWvBMRXTHRMPFDgB9K2od/JpZpwGLA6+oOLCKGUvJORHTFRMPEbwdeVE209dxq81m2L5iUyCJi6CTvRES3TDRMfAngHcCGwNXACbYfn6zAImL4JO9ERLdMdA/OSZTS8NXATsBnJiWiiBhmyTsR0RUT3YPzbNubAkg6Abh0ckKKiCGWvBMRXTFRBeexkScpEUfEJEneiYiumKiCs7mk+6rnApasXguw7eVqjy4ihk3yTkR0xUSjqKZMZiAREck7EdEtE3VRRURERPSlNHAiIiJi4KSBExFDRdKOkq6XNFvSoWO8v6+keZKuqB5vbSLOiOjMRDcZ/4Ok1YEtq5eX2v5bfSFFRNSTdyRNAY4FdgDmUta+mmn7ulG7fs/2QZ0eLyKaM98KjqQ9KHNR7A7sAfxG0hvqDiwihleNeWcrYLbtObYfBU4Fdu3C742IHtNOBefDwJYjV0+SVgXOA06vM7CIGGp15Z21gJtbXs8Fth5jv90kbQv8EfhP2zePsU9E9LB27sFZZFRp+M42/1xExMKqK+9ojG0e9frHwFTbm1EaVSeN+YukAyTNkjRr3rx5XQgtIrqpnQrOTyWdC3y3er0ncE59IUVE1JZ35gLrtLxeG7ildQfbd7a8/Dpw5Fi/yPYMYAbAtGnTRjeSIqJh823g2P4vSa8HXkK5+plh+4e1RxYRQ6vGvHMZsJGk9YG/AnsBe7fuIGlN27dWL3cBft+F40bEJJtvA0fSkbY/CJwxxraIiK6rK+/YflzSQcC5wBTgRNvXSjoCmGV7JnCwpF2Ax4G7gH07OWZENKOdPu0dxti2U7cDiYhoUVvesX227WfafobtT1XbDq8aN9j+kO3n2N7c9sts/6Ebx42IyTVuBUfSO4EDgQ0kXdXy1rLARXUHFhHDJ3knIrploi6q71Bu6vsfoHW2z/tt31VrVBExrJJ3IqIrJlpN/F7gXuCN1eyfq1f7LyNpGdt/maQYI2JIJO9ERLe0c5PxQcDHgNuBJ6vNBjarL6yIGGbJOxHRqXbmwTkE2HjU3BAREXVK3omIjrQziupmSsk4ImKyJO9EREfaqeDMAS6UdBbwyMhG28d0enBJOwJfoMxHcbzt6aPe3xc4mjIhF8CXbR/f6XEjoufVlnciYji008D5S/VYrHp0RXUD4bGU+S7mApdJmmn7ulG7fs/2Qd06bkT0hVryTkQMj3aWavg4gKRly0s/0KVjbwXMtj2n+v2nArsCoxs4ETFkasw7ETEk5nsPjqTnSvodcA1wraTLJT2nC8dei9LPPmJutW203SRdJel0SeuM8X5W9Y0YMDXmnYgYEu3cZDwDeK/t9WyvB7yPssJupzTGttEr8v4YmGp7M+A84KSxfpHtGban2Z626qqrdiG0iGhYXXknIoZEOw2cpW3/fOSF7QuBpbtw7LlAa0VmbeCW1h1s32l75AbDrwNbdOG4EdH76so7ETEk2mngzJH0UUlTq8dHgBu7cOzLgI0krS9pMWAvYGbrDpLWbHm5C/D7Lhw3InpfXXknIoZEOw2c/YBVgTOqxyrAWzo9sO3HgYOAcykNl+/bvlbSEZJ2qXY7WNK1kq4EDgb27fS4EdEXask7ETE82hlFdTelcdF1ts8Gzh617fCW5x8CPlTHsSOid9WZdyJiOLRTwYmIiIjoK2ngRERExMBJAyciIiIGznzvwZH0xTE23wvMsn1m90OKiGGXvBMRnWqngrME8DzghuqxGbASsL+kz9cYW0QMr+SdiOhIO4ttbgi8vBrWjaTjgP+lLJJ5dY2xRcTwSt6JiI60U8FZi3+dQXRp4Om2nwAeGfuPRER0JHknIjrSTgXnKOAKSRdS1o/aFvi0pKUp60NFRHRb8k5EdKSdif5OkHQ2sBUl0Rxme2TNqP+qM7iIGE7JOxHRqXaHiS8CzAPuAjaUtG19IUVEAMk7EdGBdoaJHwnsCVwLPFltNvDLGuOKiCFWZ96RtCPwBWAKcLzt6aPeXxw4GdgCuBPY0/ZNnR43IiZXO/fgvBbY2HZu7IuIyVJL3pE0BTiWMhprLnCZpJm2r2vZbX/gbtsbStoLGGlsRUQfaaeLag6waN2BRES0qCvvbAXMtj3H9qPAqcCuo/bZFTipen46sL0k1RBLRNSonQrOg5TRDOfTMjzTdlb6jYi61JV31gJubnk9F9h6vH1sPy7pXmBl4I7WnSQdABwAsO6667Z18KmHnrVQQTfhpuk7t73vIJ7XIJ4T9M95Lcg5jaedBs7M6hERMVnqyjtjVWK8EPtgewYwA2DatGlPeT/6Wzf+g41mtTNM/KT57RMR0U015p25wDotr9cGbhlnn7mSngYsTxnJFRF9ZNwGjqTv295D0tWMffWyWa2RRcTQmYS8cxmwkaT1gb8CewF7j9pnJvBm4GLgDcAFtlOhiegzE1Vw3lP9fM1kBBIRQc15p7qn5iDgXMow8RNtXyvpCMpK5TOBE4BTJM2mVG72qiOWiCYMU9fbuA0c27dWP/88eeFExDCbjLxj+2zg7FHbDm95/jCwe13Hj4jJMVEX1f2MUSIeYXu5WiKKiKGVvBMR3TJRBWdZgKp0extwCmV0wT7AspMSXUQMleSdiOiWdib6e5Xtr9i+3/Z9to8Ddqs7sIgYask7EdGRdho4T0jaR9IUSYtI2gd4ou7AImKoJe9EREfaaeDsDewB3F49duepwyojIropeSciOtLORH838dS1WiIiapO8ExGdmm8DR9ISlNV1nwMsMbLd9n41xhURQyx5JyI61U4X1SnAGsCrgF9Qpja/v86gImLoJe9EREfaaeBsaPujwN+r9WF2BjatN6yIGHLJOxHRkXYaOI9VP++R9FzKwnNTa4soIiJ5JyI6NN97cIAZklYEPkpZhG6Z6nlERF2SdyKiI+2Mojq+evoLYIN6w4mISN6JiM7Nt4tK0vKSPidpVvX4jKTlJyO4iBhOyTsR0al27sE5EbiPMunWHpSRDN/oxsEl7SjpekmzJR06xvuLS/pe9f5vJE3txnEjoufVlnciYji0cw/OM2y3rgHzcUlXdHpgSVOAY4EdgLnAZZJm2r6uZbf9gbttbyhpL+BIYM9Ojx0RPa+WvBMRw6OdCs5Dkl4y8kLSi4GHunDsrYDZtufYfhQ4lafOXLorcFL1/HRge0nqwrEjorfVlXciYki0U8F5J3BS1f8t4C5g3y4cey3g5pbXc4Gtx9vH9uOS7gVWBu5o3UnSAcABAOuuu27bAdw0fecFDrrXTT30rKZDaFu7f/+D+DnB4J5Xl9SVdyJiSLQziuoKYHNJy1Wv7+vSsceqxHgh9sH2DGAGwLRp057yfkT0lxrzTkQMiXEbOJLeO852AGwf0+Gx5wLrtLxeG7hlnH3mSnoaZbKvuzo8bkT0qEnIO41K1S5i8kxUwVm25mNfBmwkaX3gr8BewN6j9pkJvBm4GHgDcIHtVGgiBlfdeScihsS4DRzbH6/zwNU9NQcB5wJTgBNtXyvpCGCW7ZnACcApkmZTKjd71RlTRDSr7rwTEcOjnZuMa2P7bODsUdsOb3n+MLD7ZMcVERER/a2dYeIRERERfWXCBo6kRSTtMVnBREQk70REN0zYwLH9JHDQJMUSEVFb3pG0kqSfSbqh+rniOPs9IemK6jGz23FExORop4vqZ5LeL2mdKkGsJGml2iOLiGFWR945FDjf9kbA+dXrsTxk+3nVY5cOjxkRDWnnJuP9qp/vatlmYIPuhxMRAdSTd3YFtquenwRcCHywg98XET2snZmM15+MQCIiRtSUd1a3fWv1+2+VtNo4+y0haRbwODDd9o/G2mlhl4iJiMkx3waOpEUp68JsW226EPia7cdqjCsihtjC5h1J5wFrjPHWhxfg8OvavkXSBsAFkq62/afRO2WJmIje1k4X1XHAosBXqtf/UW17a11BRcTQW6i8Y/sV470n6XZJa1bVmzWBv43zO26pfs6RdCHwfOApDZyI6G3tNHC2tL15y+sLJF1ZV0AREdSTd0aWfple/Txz9A7VyKoHbT8iaRXgxcBRHR43IhrQziiqJyQ9Y+RFVbZ9or6QIiJqyTvTgR0k3QDsUL1G0jRJx1f7PAuYVTWmfk65B+e6Do8bEQ1op4LzX8DPJc0BBKzHP0c4RETUoet5x/adwPZjbJ9F1fVl+9fApp0cJyJ6QzsNnP8DNgI2piSaP9QaUURE8k5EdKidLqqLbT9i+yrbV9p+BLi47sAiYqgl70RER8at4EhaA1gLWFLS8ylXUQDLAUtNQmwRMWSSdyKiWybqonoVsC+wNnBMy/b7gMNqjCkihlfyTkR0xbgNHNsnASdJ2s32DyYxpogYUsk7EdEt7dyDs4WkFUZeSFpR0idrjCkiInknIjrSTgNnJ9v3jLywfTfw6vpCiohI3omIzrTTwJkiafGRF5KWBBafYP+IiE4l70RER9qZB+dbwPmSvgGYMtnWSbVGFRHDLnknIjoy3waO7aMkXU2ZAVTAJ2yfW3tkETG0knciolPtVHCwfQ5wTs2xRET8Q/JORHRivvfgSNpG0mWSHpD0qKQnJN03GcFFxHBK3omITrVzk/GXgTcCNwBLUhal+1KdQUXE0EveiYiOtNtFNVvSFNtPAN+Q9Oua44qIIZe8ExGdaKeB86CkxYArJB0F3AosXW9YETHkkncioiPtdFH9R7XfQcDfgXWA3eoMKiKGXvJORHRkotXE17X9F9t/rjY9DHx8csKKiGGUvBMR3TJRBedHI08kZdG7iJgMyTsR0RUTNXDU8nyDugOJiCB5JyK6ZKIGjsd5HhFRl+SdiOiKiUZRbV5NrCVgyZZJtgTY9nK1RxcRwyZ5pw/dNH3npkOIeIpxGzi2p0xmINEdSTTRz5J3IqJb2hkm3nWSVpL0M0k3VD9XHGe/JyRdUT1mTnacERER0Z8aaeAAhwLn294IOL96PZaHbD+veuwyeeFFREREP2uqgbMrcFL1/CTgtQ3FERFDQtLukq6V9KSkaRPst6Ok6yXNljTexVdE9LimGjir274VoPq52jj7LSFplqRLJI3bCJJ0QLXfrHnz5tURb0T0v2uA1wO/HG8HSVOAY4GdgGcDb5T07MkJLyK6qa3FNheGpPOANcZ468ML8GvWtX2LpA2ACyRdbftPo3eyPQOYATBt2rQMLY2Ip7D9ewBJE+22FTDb9pxq31MpFefrag8wIrqqtgaO7VeM956k2yWtaftWSWsCfxvnd9xS/Zwj6ULg+cBTGjitLr/88jsk/XmifWq2CnBHg8evwyCeEwzmeTV9Tus1eOxuWAu4ueX1XGDrsXaUdABwQPXyAUnX1xzbRJr+3OswiOcEg3leTZ/TmHmntgbOfMwE3gxMr36eOXqHamTVg7YfkbQK8GLgqPn9YturdjnWBSJplu1x+/f70SCeEwzmeQ3iOS2IiSrHtp+SZ8b6FWNsG7Mq3Fo5btogfu6DeE4wmOfVq+fUVANnOvB9SfsDfwF2B6hu/HuH7bcCzwK+JulJyr1C022nTBwR45qoctymuZSVy0esDdzS4e+MiAY00sCxfSew/RjbZwFvrZ7/Gth0kkOLiOF2GbCRpPWBvwJ7AXs3G1JELIymRlENsp4oWXfZIJ4TDOZ5DeI5dYWk10maC7wQOEvSudX2p0s6G8D248BBwLnA74Hv2762qZgXwCB+7oN4TjCY59WT5yQ7g44iIiJisKSCExEREQMnDZyIiIgYOGngRERExMBJAyciIiIGTho4UStJTc21VCtJizYdQzdVazAhKTkh+tqg5hxI3lng359RVL1B0rLAY7YflrQTZU2c+2x/ruHQFoqkFW3fXT1/BWW6+wuB60a29yNJa9i+rXr+auB1wE+BK8ZaJ63XSVoesO37JL0c2JLyGf244dCiZsk5/SN5Z+Hkaq0HSFoa+Bawm6StgWOAB4CXSvpRo8EtBEmLA6dJOkTSRsBngI0okzi+VdJ4q8f3tOoq47OSTpH0HOBDwDzKytNvkdRXE1NW/8F9ENhb0iuBrwKLASdKemejwUWtknP6R/JOB8dKBac3SNoNeCfwR+Bi26dIWgz4DjDF9usaDXABSXoJ8ElgCvA+25dK2gXYFrgdONn27U3GuDCqxWGPBjYHDrV9lqQXUpLNIsDptq9oMsYFUS2XsjGwOjDT9g+q//C+CnzN9lcbDTBqk5zTP5J3Fk4qOA2StISktaqXMylXUZsDz5e0tO1HKVPFLy7pnKbibJekpSWNLFb4F+BAYCrwRgDbMykl46nA/v3SnyxpWUkrVy9XpFxB3Qm8D8D2xcBPKFchb5S0TCOBtqn63q1bvfxf4CbKeb1S0kq2f0NZJfv9kt7dUJhRg+Sc/sg5kLzTlWOmgtMcSS8FNgOWBt5UPX8FcBjwNeBM2w9WN81tZvu3jQXbBkk7Am8BTgTeD+wDbAgcC5xi+5hqv12AG2z/vqlYF4SkLYAvAadQEue+lHL+N4Dbbe9X7bcNcJftPzYUalskbUn5ni1C+c/tPcBLqsc1wGm276muqBaz/avGgo2uSs7pj5wDyTtdyTu282joASwKnAb8HTiwZftrgPMoX+ilmo6zjfNYDVi9ev4L4DHgpS3vbwVcAhzWdKwdnONXgCeBN1WvBawB/BA4ten42jyH9YDnVs+/C9wNHNLy/puAzwHvBlZsOt48avkOJOf00SN5p7NHuqiatQzlqulUYE1JL5G0mO2fUK5A9gdWaDLA+anKwx8HvixpbUrZ+zzgqJGhf7YvBQ4GXitpg34ZijxS+q5uivs/4LPA0ZI2dXEb8C5gEUmbNxhqu3agLDL5TOD7wOnAetUoBmyfDPyBUs5ftqkgo1bJOT0uead7+uZDHxQtX96Ngc8DS9jen3Jl9QZgQ0nbAvcAu9m+pbFg50OSqn9w76RcZRwAfMv2TpT+8Muq/Z4DrAu83PYc2082FnSbRs6tKpceB8y2/V/AUcAFklaTtAnw75SrqyubjHd+qvM5nvKf23eAK4G3A7cBe0narLo34y7gS7b/0ly00U3JOf2RcyB5p9t5Jw2cSVZ9eV8DHAlsAhwgaWfgo8AjwCHAGcAytv/WXKTtq/qAoSSbkyWtbHtP4GZJl1Na7Q/YfqCxIBdQ9Tm9EvhP4PnANyW9wPZnKVdUF1PO6/e2H24w1PmStEjL+awGPE7ppngWpY//BuB/gCuAv9m+qalYo/uSc/pH8k6XY6j6wGKSVK3Vs4HdgYeAVwEvAr4NXABsACxp+5qR1nxjwY5D0qK2H6ueP5vyD2534A5Kmfsh4N0ukzjtBNzqPhrCCCBpfUo/95ttXynpU5R+/UNtX16Vhh+2fX2jgU5A0lK2H6yebwj8jHKz4t2U4aVvBva0/UdJzwMWdxnJEAMkOad/JO90Vyo4k28pyg1xf7V9MyXxPI0y9O/Vtv9k+xoorfnmwhxbNWzxRElLVpsM3AjcZnseZRTDpsAZktayfU4/JhrKP8bZwKMAtj9MKaOeJGlD21f2eJJZBXibpJH7KR4ELrV9SRX3scBVwMzqCvGKNG4GVnJO/0je6aI0cGrW0v+9AoDtGyhlxo+qTC1+C+UGuZuAbSUt1VSs7bB9J6W0vXZ1JXUDcD/wAknLVldZx1BuVOzpc2nV8jktXV2B3ENJNtvon3NRHAfcB8xoKMwFMQU4E1iyuplvHrCxpP8GqD6ni4Df0kefU8xfck7/SN6p18AuStYrqj7InYGDJN1PmazpNEqZ+AeSTqHM33Ao8A7KxEcPNhVvm26lxHogsB3wPcqcBpdIupty4+KBVWLtC9XntCul7/tmSd+k3LPwReC5kv5OGQ3wVuAjktaproZ7TtX3fXv1/HBgbUqi2QX4oaRVgV8Bb6OUwq9rLNjouuSc/pG8U69UcGomaRolmXya0jL/MOVmq69QhjduQLkj/nZgJarSZK9SWeflTZSripMo/cUXU6ZIfxzYBjjaZZhm36jO6x2UK8EzKXf9T6X0F18JLEm5oXEVSjn8oUYCbYPtJyW9WNKBlGR5I7AfZVr0HSmjT7YCPpHGzeBJzukfyTv1B5VHfRMcrU2Z2Oj4lm2fAL4JvLBl2/aUSame13TM45yHWp6/gpJgVqheH06Zq+EZ1eunjf4zvf6g3NV/HvDZlm2voqzRs1fLtu0o5fFNm465jc9pA+C66vNagVLi/wyw1Xh/Jo/+fyTnNB/7Apxj8k7Nj1Rw6vUYpRX+XJWpwrH9UeBvwH+qLBkPZfrtvdyjN8bZZV4GSftRRl1cBvygeu8I4HxK6XsZyg2AuPoW96qRvm8Al+nbf0fp039mVWo9l1I2Pkr/XLvnKuBVtq+e/Ijnr/qcdpT0VuBmyvDf7Sizn54IPEyZe2KV1j/TRKxRm+ScHpa8888/MxmxZZh4F40MsZT0ImBNYC5wLaW8ujll1dSzqn2f6R5fO6SVpG8DuwI/ovwD/DRlBeITq/c3sD2nwRDb1vI5vYxSDr7T9kxJR1PKqZ+kTLD1pKRVbN9RJZ+enyysGlZ6IPBlyk18fwfOtX1RNQR1iu3ZTcYY3ZOc0x85B5J3msg7qeB0iaQp1Zf3lcAJlCmnLwZeRlnx9XeUFV9Hrqr6ItFIWq96uj9wNOXLewSl735vlenE6adEU31Or6Gcz+PAxyQd5DJj6D2Uyac2rPa9o/rZ00lG0guqq/PPUOYIOY8yiODtwNclPd32jWncDI7knP7JOZC800R8GUXVoZGWtu0nqmF9BwC7UfofrwEus32bpNMoy9rf2GC4batKqUsCX5D0B+B6SsK8iDLN9ouBL1CuRHqydNqqSoqL2b5T0qKUScJ2BrallFDPBLB9sKQv0WfDTYE9KKMt3ky5aXQn2++R9GfK93EtoGen4I/2Jef0R86B5B0azjvpouqApMUp02fb9rurbR8AVgb+H7CP7T9V/cgXAX/s5X5iSUu4mv67pZz6dGAa5abEHYA5wMdtXyZpOdv3NRhyW6p++qMpyfKHtudVwzHvpkyH/jbbN0h6HXCz7VnNRTt/kha3/cgY2w8C1qcklLcDH7D9I0kr2b5rsuOM7kvO6Y+cA8k7vZB30kXVmceBU4ClJB1ZbVuUMrPmm6pEsxllxtDVejzRLE+ZLfTlLdumuEwK9mPb76GsV7MF5QprWUo/67/cONeLXNaj+T/K9PSvruI9h3LVcXSVZF4CTKdMVNWzVOaSOKy652Jk2xQA21+mTL8/B1ieMqPoUk0nmeiq5Bx6P+dA8k4v5J10US2k6mrjCUmzKEnnEEkfsf1JSc+i9K8+BmwGfMj2rxoNeP4WoYxUOFjSY1W8T7QmEtsfkXQRcIvt+1u293ISXcT2k7a/LekR4LXVWz+nTHT2xSrBvhJ4n3t/uYJHKTeT7irpcduXVt/DKbafsP1b4LeSrgdWdrUmTPS/5Jz+yDmQvNMreSddVAtonJLqFOB5wHuBq2wfKWkrStn4dtu/Hdm3wdDnq+rP34PSR3xka4JUn9zNP6K1nDryj7B6/gbKyIz/pczu+lzK1dOjtn/Xy5/TyGcgaUXgv4EngO+5muCs5fv4j8+ql88n2pOc0z+Sd3or76SBswCqkupxlEm0Lqi2jU44B1MSzAcaDLVto7+IKjfCvZ0y8+S/JJx+UZVTD6IMU/x1ta012Yzc6HcR8INeKKXOT8v3bHHbj0hajjLh2SLAqRDdQRMAAAdUSURBVO7DWVxj/pJz+kfyTu/JPTgLprWk+lL4x9A/VV/iKyirpT5d0iYNxtmWli/vTpKmS/oIsFLVp/pT4H2Stms2yoXSWk7dCmCknFo9P41yfi+jjNroaS2f0w6U0va+lPWDDqd0Vewu6YVNxhi1Sc7pH8k7PSYVnAU0Xkl11FXV0u6fO/1fA3ycsiDfeyhzaexj+2ZJ76X0Eb8RuKcXSo7z00Y5tfWKag3btzUYbttU5jr5HPBB4DBgNmVtoSuAoyizuR5u+97GgoxaJOf0vuSd3sw7qeC0QfqXm97uBI6ntMQ/ONZVVR8lmmUoa4bsDSxNSTS/o6z8urbtY4C32767HxJN9ff/ZFVOvZtypWHKNOFjXVH1bJJp/c5Vn9O/Aa+nLLa3LGXG2ncBzwY+ABzXq0kmFlxyTn/kHEje6eW8kwrOfLSWVCnzTDwAfN327Srj/18BfN72hU3G2a4x+r9XpswQ+h1gT+BWSsJ5FNjSY8x70ItGlVPfQJnR9efAPOBjlHVRzrB9cXNRtkfSUsCzbF8uaXvK+i53UMra36HcrLgopS//58AHbd/TVLzRXck5/ZFzIHmn1/NOKjjzUX15X0NZJ+QC4IXAaZLWqfqNfwkcKmnF1tZvr6rO52WS/kPSm6qrwwcpC/TNo0xAdRqwbz8lmuq8Xgl8HvgxZXbXT1CGzB4OLE65olp+/N/SM5YADpB0MuXKfeXqhsRlKf8x3Fc9v5byH11PJ5lYMMk5/SN5p7elgTMfg1JSHUmEVcn0a8A6lHL3sZQv7kqUadDPAH7lMq9BTxu0cuqIKqmcSbnn4lzbF1dXin8AfkPp/z4bONFlReIYIMk5vS15p3/yTrqoxjCoJVVJW1JWGf6F7dNVpn2/kDK75qeBTSirvl7ZXJTtGcRyaku5ezHK92094MOUsvfxtudV+20MLNIvSSbmLzmn93MOJO/QZ3knMxmPYaSkCqxNaQSeLGkJnlpS/WG/JJrKVpTW+R2Sfmr7AZUJqE4EFrd9TbPhLZCRcuqSwEuBvW3/UWWo7Eg5dSp9Uk4ddd/FvpSFBn9FuRI8AXhQ0p2U+UJe7ZZZXaP/Jef0jeSdPpIuqhaDVlJtOZ8NVNYGORZ4P/ASYFpVXl0bWJXSV9w3Bq2cWiWZ7YFPAV8Eng683/bNwCHAsyhDhb/Yb0kmxpec01+Sd/pLuqhGGcCS6k6Um97OAV5AKaG+izLPxI2U+Rq+b3tmY0EugEEup0raB/gDsALlu7aH7T+rzB76ALCM7ftGd2dEf0vO6X3JO/2Zd9JF9VQDU1JVWYDvU5Thi68HVgGWsP0FSXdTyo5H9UuiGbRy6hgJY3HKlfotlPjvrEZovJDyOd0Hvb/QYCyw5JwelrzTv3ln6LuoBryk+hhwMrARpcy4d5U8X2T7ZMpNcR+Q9MLWkQG9atDKqdX5vFjSfpJeBPwImAncWiWZl1O6Jy61/VCjwUbXJOf0T86B5J1Gg+3Q0FdwWlrmnwDOkTRSUl0HmM4/S6ofc48vjtZypbEJ8DBwN/Buyo1xG9p+SNK2lD7+t9g+VtKTwF/7qHW+BvA2Sjl1U0pigTI080D6oJza8jltQ1lI8bfAdpRJwT4P7C/pPMpojPfbPqexYKPrknP6LudA8k5/sj3UD0rr+7fABpSrqN9QvqxQ+sUvAnZtOs4FOJ9/owwn3bJ6vTNlYrCDKWXj3/XZ+WjU6/2AP1P6vleutr2Ssv7Lkk3HuwDntRVwPrB19Xp94AjKui5QFrVbYay/gzz6+5Gc0/uP5J3ByDtD30XFAJVUJT2PclW4h+3LJK0B3Aa8A9gG2Bo4zPaZqjQYblvsgS2nLk+5etq+en0zcAkl4eAyids91fOevCKMhZac0+OSdwYj7wxdF9WAl1QfoQxTfLmkPSlf4ieB6bb3Htmpl8uoIwa9nGr7Z5JeD3xW0o22vyvpAeDZklYD5vX6ZxTtSc7pj5wDyTsMWN4ZymHikv6NUpY7oLrq2JmyHPzplDvJP0zp/z6zwTAXWHVz4r6U4ZifBX5PSTj32j6lwdAWisrcIP9DuQL8jaT1gbcAj9s+QtKKlAuNe/olgY5WfRe/TRlS+yDwA9s/aTaq6LbknP6RvDM4hq6LalBLqgC2H3BZjO9lts+grGPzdkoC7UcDX061/WPg3yndFVfb/km/fe9iYsk5fSd5Z0AMXRcVA1ZSHccTkrYAjgU+Yvv8pgNaGMNSTrU9U9LDwImSbqr+o4jBkZzTR5J3BsfQdVENYkl1LJKWBlazfWOfJ86hKadK2gH4k+05TccS3ZOc05+Sd/rf0DVwRkhazPajkqYB3wAO6eerjkEnaRfgY8C3bB8zUkrt9yQawyM5p/8k7/S3YeyiGjEwJdVhMAzl1Bh4yTl9Jnmnvw1tBQcGr6Q6DAa5nBqDLzmnPyXv9KehbuBERETEYBq6YeIREREx+NLAiYiIiIGTBk5EREQMnDRwIiIiYuCkgRMREREDJw2ciIiIGDj/H2vtn+75YAr0AAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 576x216 with 2 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "xlabels = ['sepal length', 'sepal width', 'petal length', 'petal width']\n",
    "\n",
    "fig, ax = plt.subplots(1, 2, figsize=(8, 3))\n",
    "\n",
    "ax[0].bar(range(4), pca.loadings_[:, 0], align='center')\n",
    "ax[1].bar(range(4), pca.loadings_[:, 1], align='center')\n",
    "\n",
    "ax[0].set_ylabel('Factor loading onto PC1')\n",
    "ax[1].set_ylabel('Factor loading onto PC2')\n",
    "\n",
    "ax[0].set_xticks(range(4))\n",
    "ax[1].set_xticks(range(4))\n",
    "ax[0].set_xticklabels(xlabels, rotation=45)\n",
    "ax[1].set_xticklabels(xlabels, rotation=45)\n",
    "plt.ylim([-1, 1])\n",
    "plt.tight_layout()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "For instance, we may say that most of the variance in the first component is attributed to the petal features (although the loading of sepal length on PC1 is also not much less in magnitude). In contrast, the remaining variance captured by PC2 is mostly due to the sepal width. Note that we know from Example 2 that PC1 explains most of the variance, and based on the information from the loading plots, we may say that petal features combined with sepal length may explain most of the spread in the data."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Example 5 - Feature Extraction Pipeline"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.pipeline import make_pipeline\n",
    "from sklearn.neighbors import KNeighborsClassifier\n",
    "from sklearn.preprocessing import StandardScaler\n",
    "from sklearn.model_selection import train_test_split\n",
    "from mlxtend.data import wine_data\n",
    "\n",
    "X, y = wine_data()\n",
    "X_train, X_test, y_train, y_test = train_test_split(X, y, random_state=123, test_size=0.3, stratify=y)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Transf. training accyracy: 96.77%\n",
      "Transf. test accyracy: 96.30%\n"
     ]
    }
   ],
   "source": [
    "pipe_pca = make_pipeline(StandardScaler(),\n",
    "                         PrincipalComponentAnalysis(n_components=3),\n",
    "                         KNeighborsClassifier(n_neighbors=5))\n",
    "\n",
    "pipe_pca.fit(X_train, y_train)\n",
    "\n",
    "\n",
    "print('Transf. training accyracy: %.2f%%' % (pipe_pca.score(X_train, y_train)*100))\n",
    "print('Transf. test accyracy: %.2f%%' % (pipe_pca.score(X_test, y_test)*100))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Example 6 - Whitening"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Certain algorithms require the data to be whitened. This means that the features have unit variance and the off-diagonals are all zero (i.e., the features are uncorrelated). PCA already ensures that the features are uncorrelated, hence, we only need to apply a simple scaling to whiten the transformed data.\n",
    "\n",
    "For instance, for a given transformed feature $X'_i$, we divide it by the square-root of the corresponding eigenvalue $\\lambda_i$:\n",
    "\n",
    "$$X'_{\\text{whitened}} = \\frac{X'_i}{\\sqrt{\\lambda_i}}.$$\n",
    "\n",
    "The whitening via the `PrincipalComponentAnalysis` can be achieved by setting `whitening=True` during initialization. Let's demonstrate that with an example."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.preprocessing import StandardScaler\n",
    "from sklearn.model_selection import train_test_split\n",
    "from mlxtend.data import wine_data\n",
    "\n",
    "X, y = wine_data()\n",
    "X_train, X_test, y_train, y_test = train_test_split(X, y, random_state=123, test_size=0.3, stratify=y)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Regular PCA"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAagAAAEYCAYAAAAJeGK1AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAgAElEQVR4nO3de3RU5bk/8O8kEEIIl3IxQT0ZFLHSHrBNFPQgWiHYcElBCyQyDrDkiIEqMcmJKGCkkRAoYqAoYNKjcgkngaOlGKwuktIflfagRFEr1EoKQSRAwJYQ4iQh2b8/xgy57D17Lvs+389aLMieZOaZMe5nv+9+3ue1CYIggIiIyGDC9A6AiIhIDBMUEREZEhMUEREZEhMUEREZEhMUEREZUje9A2ivsrJS7xCIiEgHCQkJXY4ZKkEB4kF2duzYMQwfPlyDaILHWNXBWNXBWNXBWL2TGpxwio+IiAyJCYqIiAyJCYqIiAyJCYqIiAyJCYqIiAyJCYqIiAyJCYqIiAyJCYqIiAyJCYqso7gYGDIECAtz/11crHdERBQEw3WSIApIcTEwfz7Q0OD+urra/TUAOBz6xUVEAeMIiqxh6dJryalNQ4P7OBGZEhMUWcOpU/4dJyLDY4Iia4iL8+84ERkeExRZQ14eEBXV8VhUlPs4EZkSExRZg8MBFBYCdjtgs7n/LixkgQSRibGKj6zD4WBCIrIQjqCIiMiQmKCIiMiQmKCIiMiQmKCIiMiQmKCIiMiQNK3ia2lpwbJly3DixAmEh4cjPz8fcVxISUREIjQdQe3fvx8AUFJSgkWLFiE/P1/LlyciIhPRdASVmJiIn/zkJwCAM2fOYODAgVq+PBERmYhNEARB6xddvHgx9u3bh1//+te45557PMcrKysR1bldjQiXy4XIyEg1Q1QMY1UHY1UHY1UHY/WuoaEBCQkJXR8QdHL+/HnhJz/5iXDlyhXPscOHD/v0s0ePHlUrLMUxVnUwVnUwVnUwVu+kzv2a3oPavXs3Xn31VQBAz549YbPZEB4ermUIRERkEpomqAceeABHjx6Fw+HAvHnzsGTJEvTo0UPLEIiMhdvUE0nStEgiKioK69ev1/IliYyL29QTecWFukR64Tb1RF4xQRHpRY9t6jmlSCbCBEWkl2C3qfc32bRNKVZXA4JwbUqRSYoMigmKSC/BbFMfSLLhlCKZDBMUkV6C2aY+kGSjx5QiURC45TuRngLdpj6QZBMX5x5piR0nMiCOoIjMKJD7V8FMKRLpgAmKyIwCSTbBTCkS6YBTfERmUlzsvs906hTQvz/QsyfwzTfukVNennyyCXRKkUgHTFBEZtG588TFi+5R07ZtTDpkSZziI+PiotJriouBOXNYJk4hhSMoMib2qbum7bNoaRF/nGXiZFEcQZExcVHpNWKfRXssEyeLYoIiY+Ki0mu8vWeWiZOFMUGRMQXbp85KpN5zeDjLxMnSmKDImIy+qFTLAg6pz2LLFiYnsjQmKDImfxeVapkwtO4KzgW2FKKYoMi4HA7g5EmgtdX9t7fkpGXC0KOAw9fPok1bwrbZgG7d3H+Heqk+mQ4TFJmf1gnD6AUc7RM2cK08nfs/kckwQZE5dJrC61NWdu0xrROG0Qs4vJWlh2qpPpkSExQZn8gU3uCcnGsjAa0Thl4FHL7eZ5NLzEYZ6RHJYIIidShZtCAyIghzua6NBLROGHoULYjdZ3M6gYULu36vXGI2ykiPSAYTFClP6aIFuSk8PRKGv0ULwRKbthMEYPPmrp+rWMJuY6RSfSIZTFCkPKWLFnyZwtM6YahJbPQplaQFoevn2j5hA+4FvQDL08l0mKBIeUoXLYiMCFojI9UbCejZRV1q9Nm/v/TPiH2ubQlbEICrV91/mz1xU8hhgiLlKV20IDKFV5Obq87JVus1VZ1JjT4B93sXo8c9JW6FQhpggiLlqVG00GkKr27KlKBClKR3F3WpUWbb5oSd6XFPSe8kTiGDCYqUZ+bWPHovwpUaDdlswJUrHY8NGKDP56p3EqeQoVmCam5uRnZ2NmbNmoXp06ejoqJCq5cmPZi1aEHvRbhio0+bzT1S6Sw6Wp/PVe8kTiFDswS1Z88e9OvXDzt27EBRURFeeOEFrV6a2uO9A+/07qIuNvoUS06AfglB7yROIcMmCFK//cq6cuUKBEFAdHQ0/vnPf4qOoiorKxEltX6jHZfLhcjISLVCVZSRYu1TVobBOTnuRa7faY2MRE1uLuqmTDFUrHLUjLVPWRkGFRSg+9mzaI6NRW1GRlD3vORilXu9oePHI6KmpsvPNQ0ejCqFZyJ8+Vzlfo+0wt9XdegRa0NDAxISEro+IGjs8uXLwiOPPCLs2bOny2OHDx/26TmOHj2qdFiqMVSsdrsguK/HO/6x2wVBMFisMiwT6/btghAV1fG/R1SU+7i37wEEYcCAjt+ndqyd47bbBcFmc/+tcBy+sMzvgMHoEavUuV/TIomamhrMnj0bU6dORXJyspYvTQDvHRiRLwUHbdN+AwZ0/L6LF/WrnvNnKxROKVOANEtQFy5cwKOPPors7GxMnz5dq5el9njvwHh8vWhwONxFEZ2pVT2nRGJhOToFSbMEtXnzZtTV1WHjxo1wOp1wOp1wtZvDJg3oXQBAXflz0aDVCFipxMJydApSN61eaNmyZVi2bJlWL0di2qZhli51n9Ti4tzJySwl4FaUl+c++bc/kUtdNMTFXduEsPNxJXlLLP78rnBKmYLEhbpWIzc1Y9b1SVblz6JmrUbASiUWTilTkJigrIRz/ubk60WDVh06lEosnFKmIDFBWQnn/K1PixGwUonF34TKij/qRLN7UKQBzvmTEpS8V+lw+PZzbaP/tgusttF/+3go5HAEZSWc8yelGGHHYI7+Q55kgjpy5AgeeughPPzwwzh8+LDn+C9+8QtNAqMAcM6fzIqjfxIhmaBWrVqFtWvXIjc3F3l5eXj//fcBAHV1dZoFR34y8zYXFNo4+icRkgmqe/fuuOmmmzBs2DAUFhZi9erV+OKLL2CT2tWTjMGqZeSdbqD3KSvTOyJSEkf/JEIyQfXq1Qtbt25FU1MTBg0ahBdffBFPPfUUvv76ay3jIxItnx+ck8MqLyvh6J9ESCaoF198EZcuXUJTUxMA4Pvf/z42bNiA73//+5oFRwRA9AZ6mMvFG+hWIzb6Z+l5SJNMUNHR0XjyyScR3a5B5S233IKNGzdqEhiRRyA30HliMz8uPA95LDO3IqucnNveh9SemlI30HliswaWnoc8JiirETs5O53ueX0zJav270NEa2Sk9A10ntiMyd8LJ5aehzzJBNXS0oKmpiY88cQTaG5uRlNTExobGzF79mwt4yN/iZ2c20Ygvo4kjDACE3sfbex21OTmSt9A54nNeAIZ1bL0PORJJqg333wTSUlJOHDgAJKSkpCUlITJkyfj+uuv1zI+8pfcSVhuJGGU6TGp92GzASdPom7KFOmf5YnNeAIZ1bL0PORJ9uKbOXMmZs6cif/93//lDrhmIrVnUHvekphSewEFK5i9j/zZY4m0IfU76e13lfuXhTzZe1BjxoxBUVERXn75Zc8fMjCxq87OvJ3kjTI9FszVs95rariouKvwcP+Ot7HqwnPyiWyCSk9PR319PQYOHOj5QwbW/uQMuE/Q7cmd5I0yPRZsktHrxMZFxeJaWvw7TgQfttvo1asXMjIytIiFlNJ+i4PiYv+mSIw0PebrVg1G4m1Rsdnei5LsdvHpvLYLKSIRsiOoYcOGYe/evfjHP/6BEydO4MSJE1rERUrxdySh9/SY2RllitRoWPBAAZAdQR07dgzHjh3zfG2z2bB161ZVgyKdmXHkYhTBFHdYGQseKACyCWrbtm24fPkyvv76a/zbv/0bevXqpUVcROYkMkXaGhmJMI4UeOFDfpOd4nvvvffgdDqRnZ2NN954g734iMS0Ve45nUDPnsCAAZ4pUq+LiukaIywQJ0ORTVCvv/46du7ciX79+mHhwoUoLy/XIi4i8+hcuXfxIvDtt8C2bfKLisnNKAvEyVBkE1RYWBgiIiJgs9lgs9nQs2dPLeIiMg+j9P4z8wjEKJ8hGYrsPag77rgDmZmZOHfuHHJycjBixAgt4iIyDyNU7rWNQNpO8m0jEMAc04tG+AzJcGRHUJmZmZg2bRpmzJiB+++/H88884wWcRGZhxEWN5t9BKL1Z2jm0WYIkU1Q9fX1nk4Sly5dwu7du4N6wU8++QROpzOo5yAyFCOs8TH7CETLz5D3u0xDdopv4cKFuO666zB48GAA7nVQgSoqKsKePXt4H4usxQhrfMy+/krLz9AoDZFJlmyCEgQBL774oiIvFhcXhw0bNuDpp59W5PmIDEPvNT5GalEVKK0+Q7OPNkOITRCk9tN2W7FiBZKTkzF8+HDPsYiIiIBf8PTp08jMzMTOnTu7PFZZWYkouU7cAFwuFyIjIwOOQUuMVR2Mtas+ZWUYVFCA7mfPojk2FrUZGX6XuIfC5zp0/HhE1NR0Od40eDCqKiqUCK2LUPhcg9HQ0ICEhIQux2VHUB988AH+8Ic/eL622WyoUOk/IoAOiVDKsWPHfPo+I2Cs6mCsIoYPB7KzAQARAG747o8/QuJzXbNGdLQZsWaNau89JD7XIFRWVooel01Qe/bsgSAI+Oabb9CvXz+Ey+3fQkTkbxd9LRnhniH5RLaK79ChQ0hMTMS8efMwYcIEHDx4UIu4iEhvgZZim6FKjhshmoLsCGrdunXYsWMHYmJicO7cOTzxxBMYM2ZMwC944403it5/IiIDCWbhL6vkSCGyI6jw8HDExMQAAGJiYtCjRw/VgyIinXlLMnIjK1bJkUJkR1DR0dHYtm0b7rzzTnz44Yfo27evFnERkZ6kkknbSMrbyMrsa7LIMGRHUGvWrMGZM2ewbt061NTUYOXKlVrERVbCtjLmI5VMwsPlWyoZobMGWYJsgurduzfi4+MRHx+PO++8kyMo8o8ZbphTV1JJpqVF/Pvbj7gcDqCwELDbPXtiobCQ95/Ib7IJaunSpXjnnXfQo0cP7N69myMo8o/Zm5iGKqkkY7eLf3/nERer5EgBsgnq73//OwoKCjBnzhysX78eR44c0SIusgreMA+OntOjYklG7ek7TgdTO7IJKi4uDl999RUA4OLFi56msUQ+McJWFGZlxOlRNafvjPh+SVeyCerIkSOYOHEiHnjgAYwbNw5//vOfcc899+Cee+7RIj4yO94wD5xRp0fVmr4z6vsl3ciWmavZd49CANvKBC7UpkdD7f2SLNkE9Yc//AFvvfUWGhsbPceKiopUDYqMz69Wa3pvRWFWobaeKNTeL8mSTVCrV69Gbm4uy8vJI5guOOQHK+zx5I9Qe78kSzZBDRs2DKNHj9YiFjIJtlrTSKhNj4ba+yVZsglq/PjxSElJwc033+w5lp+fr2pQZGy8VaChUJseDbX3S17JJqht27bhP//zP9G7d28t4iET4K0CItKCbIIaOHAgJk2apEUsZEBixRC8VUBEWpBNUJGRkZg3bx5+8IMfwGazAQAyMzNVD4z0J1UMUVjo/sNbBUSkJtkEdf/992sRBxmQt2IItlcjIrXJdpJITk5GQ0MDPv30U9TV1WHy5MlaxEUGwGIIspriz4oxZN0QhP0yDEPWDUHxZ2yjZGSyCSonJwdfffUVxowZg6+//hrLli3TIi4yALbRIysp/qwY89+ej+pL1RAgoPpSNea/PZ9JysBkE1R1dTWeeeYZJCYmYsmSJTjFy+eQwTZ6ZCVLK5aiobnjnHVDcwOWVrDXn1HJJqjGxkZ8++23AACXy4UWqQ3LyHK47xxZyalL4hfXUsdJf7JFErNnz8bUqVMxbNgwHD9+HIsWLdIiLjIIrpskq4jrG4fqS10X8MX15Zy1UcmOoH72s59h586dSEtLQ0lJCYskTIL7vhF1lDc+D1HdO85ZR3WPQt54zlkblWSCqq+vR1ZWFurr69GvXz9UV1cjNzcX9fX1WsZHAeC+b0RdOUY4UJhcCHtfO2ywwd7XjsLkQjhGcIrAqCQT1PPPP48RI0agV69eAICkpCT8+7//O5YvX65VbBQg7vtGHhxKd+AY4cDJp06i9flWnHzqpG7JieXuvpFMUDU1NZg7d66ne0S3bt0wb948z/bvZFymWL/EE6f6OJQ2JJa7+04yQYWFiT/UvXt31YIhZUitUwoLM0g+4IlTGxxK+07DCyaWu/tOMkHZ7XaUl5d3OFZRUYFBgwapHhQFR2z9EgC0tBgkH0icOE/PWcocpSRTDKUNQOMLJpa7+04yQS1evBglJSV48MEH8eSTT2L69OkoLS3F888/H/CLtba2IicnBykpKXA6nagW27Mh1Khw5dZ5/VJ4eNfv0fVCWuIEeX3LKQ6klMRWIL7ReKQpVdbOcveuJNdB9enTB7/5zW9w5swZnD9/HoMHD0ZMTExQL1ZeXo6mpiaUlpbiyJEjWLVqFTZt2hTUc5qainunt1+/JDFbq9+FtMSGUqcQx515lcR9UXyj8Ugzb3we5r89v8M0H8vdxdkEQRC0erH8/HyMHDnSs5Zq7Nix+NOf/uR5vLKyElFic1OduFwuREZGqhankrzFOnT8eETU1HQ53jR4MKoqKhSLYfz4oaipiehyfPDgJlRUVPkUq5L6lJVhcE4Owlwuz7EriMJjKMT/wAGbTcDnn//N63NY5XdAbX3KyjCooADdz55Fc2wsajMyUDdliuT3h+LnqsX/h51jLasuQ8FnBTjbcBaxUbHIGJGBKXbp/y5a0uN3oKGhAQkJCV0fEDS0ZMkS4Y9//KPn6/vuu09obm72fH348GGfnufo0aOKx6YWr7HabILgnvXu+MdmUzSG7dsFISqq40tERbmP+xyr0rZvF74KtwstsAknYBcexnZPbHa7/I9b5nfAYEIyVl//BwlCSH6ufpA698t2klBSdHQ0rly54vm6tbUV3brJdluyLo3uEfjSU6+42D3S0qzKz+HA/9tyEr2jWnETTuJ/4A7GZnPP/uleaUihg00nDUsyO6SkpHjWQLURBAE2mw0lJSUBvVh8fDz279+PSZMm4ciRI7j11lsDeh7L0PAegbeeetduhbmnARW8FSYbE+C+51Rd7T43tE04axUDEQA2nTQoyQT10ksvKf5iEyZMwMGDB5GamgpBELBy5UrFX8NU2p+hddw73VsRk9qhtJ0XhgzpWjfBggmi0CaZoG644QYA7v2g3n33XTQ3NwMAzp8/j9zc3IBeLCwsLOCftSwDXLkZYbmM1GsFuxKhuFj3/E9EAZK9B7V48WIAwEcffYTTp0/jX//6l+pBkf+CWU5lhOUyUq9lswV+L4oNK4jMTTZBRUZG4vHHH0dMTAxWrVqFCxcuaBEX+SHYE7ERds7Ny3Mno84EIfD1kuz0oxwrNDe1wnsINbIJShAE1NbW4sqVK2hoaMClS5e0iMswzNDTNNgTcVsR0+DBTboVMTkc1wokOgt0qtEIU5dWYIXmpmq/ByY/dcgmqCeeeAL79u3D1KlTMX78eNx7771axGUIZpkiUuJE7HAAFRVVaG0FTp7U5z6N3S5+PNCpRiNMXVqBFZqbqvkerJDAjUo2Qd15551ISkrCwIED8fvf/95zTyoUmGWKyConYqWnGo0wdWkFVmhuqtR7EBspWSGBG5VsgiouLkZqaioKCwuRkpKC3/3ud1rEZQhmmSKyyolY6fWSvi5QNvoUrt6s0NxUifcgNVKqviReamqmBG5Usglq165dePvtt/HKK69g9+7d2Lp1qxZxGYJZRiZWWgjvcLinGJWaavT2fGaZwtVb3vg8RHXveAVktuamSrwHqZFSuE1kuwCYK4EblWyCGjBgAMK/268hMjIS/fr1Uz0oozDTyETpE3soMMsUrt4cIxwoTC6Eva8dNthg72tHYXKhbtulB0KJ9yA1ImoRWkyfwI1KthGeIAiYNm0afvzjH+Po0aO4evUqsrKyAABr165VPUA9GaTRA6nELFO4RuAY4TBVQhIT7HuI6xsnOp1n72tH3vg8LK1YilOXTiGubxzyxueZ/vMyAtkElZaW5vl3cnKyqsEYkQEaPZBKJLalMtwULgWmrYBBqaThbR8nKyRwI5Kc4tu/fz8A4B//+AdOnDjR4c+oUaMwatQozYIkUoOZpnDJP2qUfgczTVhWXcZ1UgGQHEG1tTRi5wiyKk7hWpe30u9gRjqBjJSKPyvG0g+XornV3c+0+lI1Hv3do57nI2mSI6gHH3wQgHtab8iQIXjiiSfgcrkwbdo0zYIjUhuLS6zJSGu30n+f7klObZpampD++3TNYzEbn5rFDho0CABw3333YSlLnIjI4Iy0duvitxf9Ok7X+LSj7ujRowG4u0q0traqGhARUbCCWffEvnrGIZug+vTpg9LSUnzxxRfYtWsXevXqpUVcZBDstEBmFGhBgxrFFQN6DvDrOF0jm6BWrVqF48ePY82aNaiqquIuuCEk2E4LTG6kJ8cIB04+dRKtz7fi5FMnfSpIUKOv3vqJ69HN1rEerXtYd6yfuD7g5wwVsuug+vfvj7S0NDQ2NgIAXC6X6kFZRXExkJ09FGfPmrNCTKrTwpw57n97ey9tya3t59uSm9zPEelJjeIKxwgHznx9Bq/87RUu5PWT7Ahq+fLlmDFjBjIzM5GRkYHMzEwt4jK9thN0TU2E7n3eAh3JSHVUaGmRfy9KtBHiCIy0plZxxRT7FL9Hc+TDCOrTTz9FeXk5wsJ8qqeg73g7QWs5gghmJCPVaQGQfy/BthHiCIz04K1bBGlPNuvY7XbP9F6o8+eKPtATtNKjhmBGMmKdFtrz9l6C7QTPRq6kBys0xrUS2RFUTU0N7r//fti/2+7UZrOhpKRE9cCMxt8r+kD6vKkxaghmJNP2mnPmuKf1OvP2XvLyOr4XwL82QmzkSlrr3Ltv20PbmJh0JjuCWrt2Ld5880289NJLeOmllyzfwVyKv1f0gfR5U2PUEOxIxuEAtmzx/70Eu0eVWfbiImvQctt2M6+z0jp2yQS1a9cuAEBJSQlKS0s7/AlF/l7Rt52gBw9u8vkErcaoQYmGqIEmm2DaCLGRK2lJq23btUyEStMjdskEFRsbC8B9D+qmm27q8CcUBXJF73AAFRVVPp+g1Rg1KLXbrtY966y0SzAZn1a9+7RKhGrQI3bJe1Bjx44FALzzzjt47bXXVAvALIK9p6Lna5h1Tyuzxk3mI7UZodK9+4zUxNZfesQuew+qd+/eqKioQFVVlWc/qFCkxRU9Rw1E+gimd58/jNTE1l96xC6boL755hu88cYbWL58OXJycvD8888H9YL79u3zbBlvNlpMc3H7ByLtaVVebuYmtlol8fa8lpnX19ejsLAQPXv2VOTFVqxYgffffx/Dhw9X5PmsrriYm+kRaUWLbdvbnt/frejbChTa7gG1FSi0f061BRp7MCRHUNu3b8fPfvYzTJ06FX/6058UebH4+HgsX75ckeeyumAbtbY9B1sFERmLmk1s1R5lBRJ7MGyCIAhiD6SmpmLr1q2or6/H008/jd/85jc+P+muXbuwZcuWDsdWrlyJkSNH4tChQygpKUFBQUGXn6usrESUt9YF33G5XIiMjPQ5Hj0FGuv48UNRUxPR5Xjfvlfxl798KfvzZWV9kJMzGC7XtWuQyMhW5ObWYMqUOkVj1QNjVQdjVUewsf5w5w8hoOup2gYbPp/5OQCgrLoMOYdz4Gq51tA7MjwSuXfkYop9imaxBqKhoQEJCQldjktO8UVERCAiIgL9+/dHc3Oz1LeJmjFjBmbMmOF/lIBP03/Hjh0zzTRhoLGePSt+/NKlbvjoo+GyU30TJwKdG8+7XGF45ZUbkJ19g6Kx6oGxqoOxqiPYWL1VGbY978T3JnZITgDganHhlb+9guykbM1iDURlZaXocZ86wEoMskhF3tY++dJZgq2CiKzDlwIFM5ewS5EcQR0/fhxZWVkQBMHz7zah2u5IS3l5wCOPiD/mS5IJpBcgERmTLwUKWq3l0pJkglq3bp3n36mpqYq94OjRozF69GjFns+qHA4gPR24eLHrY74kGS0WFhORduSqDK24VYhkgho1apSWcYQcX0rI168HHn0UaGq6diwiwrck43AABw+6F/q2tADh4e6u5CxTJ7ImPcrA1RayuxDqWYLtTwl559t/TU3ukZVcvMXF7i7kbdtktLS4v2apOZF1aV0GrraQTFALFwJOZ3BrjILh67YaS5cCYgWUFy9qs+U6kZno3WlBDVZ8T/4IuQRVXAxs3tx1ZKLlydvXCjtvxRBy8bKKj0LJwr0L4XzLacptLKSYeWsOpYRcglq6tGtyaqPVydvXbTXkiiHU3HKdyCyKPyvG5sObuyxkNcs2FlLMvDWHUkIuQQVyUlear5vxiX1fe3JbrnPDPwoFSyuWinZZAMy9BsiK65r8FXIJSuqkbrNpd/L2dVuNtu8bMKDrc6i95TqRWXg7YZt5DZAv21tY/R5VyCUosZGFzQakpbn/rVVln6/bajgcwIULwPbt2m65TmQWUidyG2ymXgMk1z0iFO5RhVyCEhtZbNsGjBkTfPdwX3UucV+4UD4xMtmQFSkxAhA7kdtgQ9odaaYus5bboyoU7lF53Q/KqsS2Eh8yRLosW8lk0LYGqu21qquBTZuuPd6WGNviJLIqpfY4MsoC1eLPihWPwVv3iFC4RxWSCUqMVmXZYuuTOlMjMRIZjbcRgL8ndi02G/RGjw0Frdh7r7OQm+KTolVZtq8Jj+uVyOqsNALQY7pNjy3YtcYE9R2tyrJ9TXhcr0RW50uVmlnokWzl7lFZARPUd7Qqy5Zb2wSot16JW8CTkVhpBKBXsrVa773OmKDa0aJSTiwRLligfmL0p0EtkRasNAKwUrI1EhZJ6ECsilBt3prHshiD9KJ3cYNSjFJJaDUcQZmcr9N2bB5LpByx9VtWn27TA0dQJia2pkpqDRW3gCdShlRJ+cFTB/HOl+9wBKUgjqBMzJ89n9g8lkgZUiXlmw9vtnTbIT0wQZmYP9N2bB5LpAyp0nGrbfdhBExQOgum9NvfxcXs50cUPH9Kx8246NhImKB0FGzpN6ftiEpjwOQAABDhSURBVJTja+Naqea0Ysy46NhImKB05M89JDGctiNShj9bV4it30q7I43roFTABKUjqXtI1dW+T/l1nrYD2C0iFLS2tiInJwcpKSlwOp2oFivRJJ/520uvc0n5xskbLbPo2EiYoHTkrcQ7kCk/doswLqXbTJWXl6OpqQmlpaXIysrCqlWrlAgzZCnRS4/roJTHBKUjX/ry+TPlF+yUIamjrKyP4hcOlZWVGDt2LADgRz/6Ef76178qFG1oMmvjWm75TqrpfA9JSrBbdLBbhL4KCgYpfuFQX1+P6Ohoz9fh4eG4evVq4E8Y4szYS49bvivo8uXLSEtLwyOPPIKUlBR8/PHHWr20obW/h2S3i39PsFt0sFuEvs6e7S56PJgLh+joaFy5csXzdWtrK7p1Y2OYQJmxcW0obPmuWYJ6/fXXcdddd2H79u3Iz89Hbm6uVi9tGsGWjbPs3JhiY5tFjwdz4RAfH48DBw4AAI4cOYJbb7018CcjAOa7h2SlDR+laJag5s6di9TUVABAS0sLevToodVLm0awZeMsOzemjIxaxS8cJkyYgIiICKSmpiI/Px/PPvtscEGS6Zj1vpk/bIIgCPLf5p9du3Zhy5YtHY6tXLkSI0eORG1tLR577DEsWbIEo0aN6vA9lZWViJKrGgDgcrkQGRmpaMxqYazqMFus5eXXoaBgEM6e7Y7Y2GZkZNRiypQ6vUPrwmyfayjHWlZdhpzDOXC1uDzHIsMjkXtHLqbYpwT8vHp8rg0NDUhISOj6gKChv/3tb8KkSZOEP/7xj6KPHz582KfnOXr0qJJhqYqxqoOxqoOx+mf7p9sFe4FdsC23CfYCu7D90+2i36dWrL6+vj/0+Fylzv2a3VU9fvw40tPTsW7dOtx2221avSwRkaTiz4oD3mRQatsNAJrdv7LKho9SNLsHtXbtWjQ1NSEvLw9OpxMLFizQ6qVVp/QiTCJSX7Bl2qFQRac3zUZQmzZt0uqlNOXPpoFEZBzeEowvo5LqS+LtpaSOk/+4UDdI7N5AZE7BlmmH28L9Ok7+Y4IKErs3EJlTsGXaLUKLX8fJf0xQQTJi9wbeEwsdn3zyCZxOp95hmFKw7Y3sfcVbv0gdJ/8xQQXJaN0b2NHcoFS4aigqKsKyZcvQ2NgY9HOFomDbG5mxf5/ZMEEFyWjdG3hPzHj6lJWpctUQFxeHDRs2KBRlaPKnvVHnzuEATNe/z2yYoBTQedNAPav3eE/MeAYVFKhy1fDTn/6UDWID5O82FVIl6QBM1b/PbJigLMaI98RCXfezZ8Uf4FWDLgJZ/6Tlmier7/HkDyYoizHaPTECmmNjxR/gVYNm2p/05/x2jt/JRqvO4aGwx5M/mKAsxmj3xAiozcjgVYOOOp/0pcrAvSUbrTqHsztFR0xQFmSke2IE1E2ZotpVw4033oidO3cqEKV1iZ30xXhLNlpV7IXCHk/+YIIi0gKvGnTjy8ldLtloteNuKOzx5A+WABGRpcX1jRPtjxduC0er0OpzF3MtOofnjc/r0CEdCO21VSE3gmKXBaLQIjU9t+XBLYYrD9dqpGYWITWCYudxotDTdnIPdN8nrVl9jyd/hFSC8tZlgQmKyLp40jenkJriY5cFIiLzCKkExS4LZBXNzc3Izs7GrFmzMH36dFRUVOgdkmGxM4N5hVSCYpcF0ovSJ8k9e/agX79+2LFjB4qKivDCCy8oFKm1sDODuYVUgmKXBdJDWXWZ4ifJpKQkpKene74OD+curmLYmcHcLJWgfCkh53pJ0lrBZwWKnyR79eqF6Oho1NfXY9GiRXjqqaeCDdOS2JnB3CyToLhRHxnV2QbxbubBniRramowe/ZsTJ06FcnJyUE9l1WxM4O5WSZBcaM+MqrYKPFu5sGcJC9cuIBHH30U2dnZmD59esDPY3Xc9dbcLJOgWEJORpUxIkPxk+TmzZtRV1eHjRs3wul0wul0wuVyBRuq5bAzg7lZZqFuXJx7Wk/sOJGeptin4Pobrle0k8GyZcuwbNkyBaO0Li7SNS/LJKi8vI5tjACWkJNx8CRJ5D/LTPGxhJyIyFosM4IC3MmICYmIyBosM4IiIiJr0WwE1dDQgKysLFy6dAk9e/bEmjVr0L9/f61enoiITEazEdTOnTvxwx/+EDt27MDkyZOxceNGrV6aiIhMyCYIgqDVi7W0tCA8PBwvv/wyunXrhrS0tA6PV1ZWIqpzN1cRLpcLkZGRaoWpKMaqDsaqDsaqDsbqXUNDAxISErocV2WKb9euXdiyZUuHYytXrsTIkSMxe/Zs/P3vf8frr78u+rPDhw+Xff5jx4759H1GwFjVwVjVwVjVwVi9q6ysFD2uSoKaMWMGZsyYIfrY1q1bUVVVhccffxzl5eVqvDwREVmAZlN8r776KmJiYjBt2jTU1NRg7ty5eO+99zp8j1QWJSIiaxOb4tMsQV24cAGLFy9GU1MTWlpakJWVJRoQERERoHGRBBERka+4UJeIiAyJCYqIiAzJtAmqqqoKCQkJaGxs1DsUrxoaGrBgwQLMmjUL8+bNwzfffKN3SJIuX76MtLQ0PPLII0hJScHHH3+sd0iy9u3bh6ysLL3DENXa2oqcnBykpKTA6XSiWmw/GAP55JNP4HQ69Q5DVnNzM7KzszFr1ixMnz4dFRUVeockqaWlBc8++yxSU1PhcDhwygQb1F28eBH33Xcfqqqq9A7FnAmqvr4eq1evRkREhN6hyDJTB43XX38dd911F7Zv3478/Hzk5ubqHZJXK1aswNq1a9Ha2qp3KKLKy8vR1NSE0tJSZGVlYdWqVXqHJKmoqAjLli0z/AUfAOzZswf9+vXDjh07UFRUhBdeeEHvkCTt378fAFBSUoJFixYhPz9f54i8a25uRk5OjmEWFZsuQQmCgOeeew6ZmZno2bOn3uHImjt3LhYsWAAAOHPmDAYOHKhzRNLmzp2L1NRUAO4rvx49eugckXfx8fFYvny53mFIqqysxNixYwEAP/rRj/DXv/5V54ikxcXFYcOGDXqH4ZOkpCSkp6d7vg4PD9cxGu8SExM9CdTo//8DwOrVq5GamorrrrtO71AAGHy7DbGOFNdffz0mTZqE2267TaeopAXTQUNr3mKtra1FdnY2lixZolN0HUnFOmnSJBw6dEinqOTV19cjOjra83V4eDiuXr2Kbt2M97/dT3/6U5w+fVrvMHzSq1cvAO7Pd9GiRXjqqad0jsi7bt26YfHixdi3bx9+/etf6x2OpLfeegv9+/fH2LFjUVhYqHc4AExYZj5hwgTExsYCAI4cOYKRI0eiuLhY56h8Y4YOGl988QUyMzPx9NNP47777tM7HFmHDh1CSUkJCgoK9A6li/z8fNx+++2YNGkSAODee+/FgQMHdI5K2unTp5GZmYmdO3fqHYqsmpoa/OIXv/DchzKD2tpazJw5E3v37vWp56jWHA4HbDYbbDYbjh07hiFDhmDTpk0YNGiQbjEZ71JOxr59+zz/HjduHF577TUdo5HXvoNGVFSUoacjjh8/jvT0dKxbt86QI1SziY+Px/79+zFp0iQcOXIEt956q94hWcKFCxfw6KOPIicnB3fffbfe4Xi1e/dunDt3Do8//jh69uwJm81m2HNA+wt9p9OJ5cuX65qcABMmKLP5+c9/jsWLF+PNN99ES0sLVq5cqXdIktauXYumpibk5eUBAKKjo7Fp0yadozKvCRMm4ODBg0hNTYUgCIb+b28mmzdvRl1dHTZu3OgpOioqKjLMjf32HnjgATz77LNwOBy4evUqlixZYvh7u0Ziuik+IiIKDaar4iMiotDABEVERIbEBEVERIbEBEVERIbEBEVERIbEBEWmdujQIdx9991wOp1wOp2YOXMmtm3b1uX7Dhw4gNLSUr+e+6233vK7Eenp06cxc+bMLscvXbqEJUuWwOFwIDU1FRkZGbh8+bJfz6230tJSNDc3iz5m5Ka9ZF5cB0Wmd9ddd3k6STQ1NSEpKQlTp05Fnz59PN9z7733+v28Dz30kGIxZmZmIjU1FRMmTAAAvPHGG8jJyTFkBwwpr776KqZNm9bl+IoVK/D+++9j+PDhOkRFVsYERZZSX1+PsLAwhIeHw+l04nvf+x7q6uowefJkVFdXIzU1FVlZWYiNjcVXX32FESNG4Je//CUuXryIZ555BpcvX4YgCFi9ejXefvttDBw4EDfffDM2b96MsLAw1NbWIiUlBQ6HAx988AFefvllAIDL5cLq1avRvXv3LjF9/fXXuHDhgic5Ae6V+j//+c8BuLtzb9myBRERERgyZAhyc3Px9ttvY//+/XC5XKitrcXs2bNRUVGBL7/8Ek8//TQSExMxfvx43H777Th16hSGDRuGvLw81NfXIzs7G/X19WhpaUF6ejruvvtuJCcnY9SoUfjiiy9gs9mwceNG9O7dG2vXrsWHH34IQRAwd+5cTJw4EU6nE7fddhu+/PJL1NfXY/369fjzn/+M2tpaZGRkdOnIHx8fj8TERL9HqERymKDI9P7v//4PTqcTNpsN3bt3x3PPPedpKJqcnIwJEybgrbfe8nz/yZMn8d///d/o2bMnEhMTUVtbi1dffRXjxo3Dww8/jL/85S/49NNPO7zGuXPnsHv3brS2tiI5ORlJSUn48ssvsWbNGsTExGDz5s149913kZyc3CW+8+fP48Ybb+xwLDw8HL1798Y///lPbNiwAb/97W8RHR2NlStXorS0FFFRUbhy5Qpee+017N27F2+88QZ27tyJQ4cOYevWrUhMTMS5c+eQnp4Ou92O9PR0lJeX4+OPP8Z//Md/YM6cOTh37hwefvhhlJeX48qVK5g8eTKee+45ZGVl4cCBA4iOjsbp06dRUlKCxsZGzJw5E2PGjAEAjBw5EkuXLkVBQQH27t2L+fPnY9OmTaIjPqM37SXzYoIi02s/xdfZTTfd1OVYXFycp8v4oEGD0NjYiBMnTniajrb1d2u//cSPf/xjz/5jw4YNw6lTpxATE4O8vDxERUXh3LlziI+PF43h+uuvx9mzZzsca25uxrvvvgu73Y5bbrnFE8+dd96J999/H7fffrtnyqx3794YOnQobDYb+vbt69mzafDgwbDb7Z74Tpw4gaqqKk+SjImJQXR0tGeTzB/84Aeen2tsbMSZM2fw+eefezYpvHr1Ks6cOdPhe2NjY3HhwgXR90WkNhZJkKXZbDafjg0dOhSfffYZAODDDz/EmjVrOjx+7NgxtLS04Ntvv8Xx48dht9uxbNkyrFy5EqtWrcJ1110Hqa5hMTEx+N73vtehi/3WrVtRXl6OG2+8EVVVVWhoaAAAfPDBB56kKhZne+fOnUNtbS0A4KOPPsItt9yCoUOH4vDhw57H6+rq0K9fP9Hnu/nmmzF69Ghs27YNW7ZswcSJE7uM9Nqz2WyG3RySrIkjKCIAaWlpWLJkCfbs2QPAvd/U7t27PY9fvXoVjz32GP71r39hwYIF6N+/P6ZOnYqZM2eiT58+GDhwIM6fPy/5/L/61a+Qm5uL1157Dc3NzYiLi8OKFSvQu3dvPPnkk5g9ezbCwsIQFxeH//qv/8LevXtlY46IiMALL7yAmpoa3H777Rg3bhwSEhKwZMkSvPfee3C5XMjNzZXcf2rcuHH44IMPMGvWLDQ0NCAxMbHD/lWd3XHHHZg/fz62bt0qmzyJlMBmsUQyjLrn1JgxY3Dw4EG9wyBSDaf4iIjIkDiCIiIiQ+IIioiIDIkJioiIDIkJioiIDIkJioiIDIkJioiIDOn/AzArGIRr70tSAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "sc = StandardScaler()\n",
    "\n",
    "pca1 = PrincipalComponentAnalysis(n_components=2)\n",
    "\n",
    "X_train_scaled = sc.fit_transform(X_train)\n",
    "X_train_transf = pca1.fit(X_train_scaled).transform(X_train_scaled)\n",
    "\n",
    "\n",
    "with plt.style.context('seaborn-whitegrid'):\n",
    "    plt.figure(figsize=(6, 4))\n",
    "    for lab, col in zip((0, 1, 2),\n",
    "                        ('blue', 'red', 'green')):\n",
    "        plt.scatter(X_train_transf[y_train==lab, 0],\n",
    "                    X_train_transf[y_train==lab, 1],\n",
    "                    label=lab,\n",
    "                    c=col)\n",
    "    plt.xlabel('Principal Component 1')\n",
    "    plt.ylabel('Principal Component 2')\n",
    "    plt.legend(loc='lower center')\n",
    "    plt.tight_layout()\n",
    "    plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Covariance matrix:\n",
      "\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "array([[4.9, 0. ],\n",
       "       [0. , 2.5]])"
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "np.set_printoptions(precision=1, suppress=True)\n",
    "\n",
    "print('Covariance matrix:\\n')\n",
    "np.cov(X_train_transf.T)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As we can see, the features are uncorrelated after transformation but don't have unit variance."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### PCA with Whitening"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAagAAAEYCAYAAAAJeGK1AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAgAElEQVR4nO3de1hU5b4H8O+ATHLVQAO1PVikSV5KrKzTUU+iT5iwNS9cG0zTMs0UPaRH3MQmQcsM3e7ItDREStRTbsUuT2BPHqtjidtLih0jAS+gSFtxwGFGZp0/xhkZmDVrLmvWZeb3eR4eYa2ZWb9Z4PrN+673/b0KhmEYEEIIIRLjI3YAhBBCiDWUoAghhEgSJShCCCGSRAmKEEKIJFGCIoQQIkndxA7AEZWVlWKHQAghxA1GjBjRZZusEhRg/U1IQVVVFaKjo8UOwylyjV2ucQPyjZ3iFp5cY3ckbrbGB3XxEUIIkSRKUIQQQiSJEhQhhBBJogRFCCFEkihBEUIIkSRKUIQQQiSJEhQhhBBJogRFCCFEkihBEc9WUgL07w/4+Bj/LSkROyJCiJ1kV0mCELuVlAAvvQS0thp/rq01/gwAaWnixUUIsQu1oIjnysq6k5xMWluN2wkhkkcJiniuujrHthNCJIUSFPFcKpVj2wkhkkIJiniuvDwgIMByW0CAcTshRPIoQRHPlZYGbNoEREYCCoXx302baIAEITJBo/iIZ0tLo4REiExRC4oQQogkCdaC0uv1WL58OS5evAidTodXXnkFsbGx5v0HDhzAe++9h27dumHq1KlITEwUKjRCCCESJFiC2rt3L3r27Ik1a9bgX//6F5577jlzgtLr9Vi1ahV2794Nf39/pKSk4Omnn0bv3r2FCo8QQojECNbFFxcXh4ULF5p/9vX1NX9fXV0NlUqFHj16QKlUYsSIEThy5IhQoRFCCJEgwVpQgYGBAACNRoPXXnsNixYtMu/TaDQIDg62eKxGo7H6OlVVVe4N1ElarVaysXGRa+xyjRuQb+wUt/DkGjsfcQs6iq++vh7z589HamoqEhISzNuDgoLQ0tJi/rmlpcUiYXUUHR3t9jidUVVVJdnYuMg1drnGDcg3dopbeHKN3ZG4KysrrW4XrIvv6tWrmDVrFjIzMzFt2jSLfVFRUaitrcW1a9eg0+lw5MgRDB8+XKjQCCGESJBgLaiNGzeiubkZhYWFKCwsBABMnz4dN2/eRFJSEpYtW4YXX3wRDMNg6tSpCA8PFyo0QgghEiRYglqxYgVWrFjBun/s2LEYO3asUOEQQgiROJqoSwghRJIoQRFCCJEkSlCESAktUU+IGRWLJUQqaIl6QixQC4oQqaAl6gmxQAmKEKlwdYl6Z7oHqUuRSBh18REiFSqVsVvP2nYuznQPUpcikThqQREiFa4sUe9M9yB1KRKJowRFiFS4skS9M92DrnYpEuJm1MVHiJQ4u0S9M92DrnQpEiIAakER4gmc6R50pUuREAFQgiLEEzjTPehKlyIhAqAuPkLkrKTEOKihrs7YNZeX51iCcbZLkRABUIIiRK5omDjxcNTFR+Tn9uTSQYMHe+/k0pISYMYMGiZOPBq1oIi8dGg1KADvbDWYzkF7u/X9NEyceAhqQRF5ocml1s9BRzRMnHgISlBEXmhyqe33SsPEiQehBEXkha114E2tBrb36utLw8SJR6EEReRFypNLhaoMznYOioooORGPIniCOn78ONRqdZftW7duxcSJE6FWq6FWq/H7778LHRqRgw6TSxl7JpcKlTRMAxdqawGGuTN4wx3Howm2xEsIOopv8+bN2Lt3L/z9/bvsO3XqFN566y0MGTJEyJCIHN2eXHqmqgrR0dHsjxNynpCtwRvuSByOTrA1TeitrTV2Bba3GxOboxN7CRGQgmEYRqiDff3113jwwQfx+uuvY+fOnRb7JkyYgAEDBqCxsRH/8R//gZdffrnL8ysrKxHQuWtDIrRaLbp37y52GE6Ra+xccUfFxkJZX99lu65PH1RXVPAay6DBg6Gw8l+JUShw5tSpLtuFPOchZWXok50NH622yz5D9+6oz81Fc3y8Xa/lqX8rUibX2B2Ju7W1FSNGjOi6gxHY+fPnmenTp3fZvmHDBqapqYlpa2tj5syZwxw4cKDLY44cOSJEiE45ffq02CE4Ta6xnz59mmG2b2eYyEiGUSiM/27ffucBCgXDGDvcLL8UCv6DiYy0fqzISPbYhcIWG0eM1sj6b0Wm5Bq7I3GzXdslMUiCYRjMmDEDoaGhUCqVGDNmDE6fPi12WETiQsrKbN/3EXLEnxiDN0pKgF69jPehFArj99bueXENwfemIfpEViSRoDQaDeLj49HS0gKGYXD48GG6F+WpeBy00LugwPakXSGThtADF0pKgJkzgaamO9uamoBZs7qeU66E7E1D9ImsiJqg9u3bh9LSUgQHByMjIwPp6elITU3FAw88gDFjxogZGnEHnke6+TU0WN9hahEInTTS0oCaGsBgMP7rzsEHWVmAXt91u07XtaqGtURtIpUh+oRYw09vozDoHpR7CBa7g/dpuLT16cPr6wnJoXNu7T4b2/01tntsptcAGMbX98556njPju+4JUSucTOMfGP3mHtQxEvwXKaoMSNDuC48oeZTWTuutVZnaCj7c6x12ZladwwD3Lpl/NfdrTxCXEQJigiH50ELzfHxwnThCTkJtzO2+VUA4OfX9fFKpThddmIlcOLRKEER4bhj0IIQ933ErKDO1rpsagJCQiy3hYUBW7YI3yoSM4ETj0YJighHriV6xKygzta6VCgsR/AFBADr14tzLmkJFOImlKCIsIQc6cYXMSuoW2t1KhTGlkpHYiYEWgKFuAklKHIH3UewTuhJuB1/D1lZxqXdO7Y62aqTiZUQaAkU4iaUoIgR3UdgJ2TXpLXfQ1GRMRmaWp2RkdafK1ZCkPISKETWKEERI7qPYJtQXZP2/B7YJt5qNOJ8oJDrvUUieZSgiBHdR5AGe34PpoQQFmb5mKYm8Vq99iRw6kImDqIERYzoPoI02Pt7SEsDgoK6Ps5drd7byWXQ4MHOJRfqQiZOoARFjOg+gjQ48nsQqtXbIbkonE0u1IVMnEAJihjRfQRpcOT3IFSrl4/kQl3IxAmUoLwJ1z0AOc5R8kT2/h6EavXykVyoC5k4gRKUt6B7AJ5HqFYvH8mFupCJEyhBeQu6B+CZhGj18pFcHE2mNOKPAOgmdgBEIHQPgDjLlESyssDU1UGhUhmTk6PJMC3NvueYWvumD1Sm1n7HWIhXoBaUt6B7AMQVt1tqZ06dEma1YGrtE9hIUMeOHcOUKVOQkpKCI0eOmLfPnz9fkMAIz+geAJELau2T21gT1OrVq7F27Vrk5uYiLy8Phw4dAgA0NzcLFhzhEQ0jJ3JBrX1yG2uC8vPzw3333YcBAwZg06ZNeOutt/Drr79CoVC4dMDjx49DrVZ32X7gwAFMnToVSUlJ2Llzp0vHICw8cRg53Uz3PNTaJ7exJqjAwEBs27YNOp0OvXv3xjvvvINFixbh4sWLTh9s8+bNWLFiBdra2iy26/V6rFq1Clu2bEFxcTFKS0vR2Njo9HGIdwgpK6Oh856IWvvkNtYE9c477+D69evQ6XQAgAcffBAbNmzAgw8+6PTBVCoVNmzY0GV7dXU1VCoVevToAaVSiREjRljc9yLEmt4FBXQz3VNZa+1Ta9nrsA4zDwoKwoIFCyy2PfDAAygsLHT6YM888wwuXLjQZbtGo0FwcLD558DAQGg0GquvUVVV5fTx3Umr1Uo2Ni5yjX1QQ4PV7UxdHc6wvJ+QsjL0LiiAX0MD9BERaMzIQHN8vDvDtEqu51ysuEPKytAnOxs+Wq1xQ20tDLNno/7SJbt+f3I934B8Y+cjbknMgwoKCkJLS4v555aWFouE1VF0dLRQYTmkqqpKsrFZKCkxtjDq6ow3nfPyUBUTI4/YTW6/B4ZlZVmFSmX9/ZSUADk55laXsr4e/XJy0K9vX8G7j2Tz99KJaHFPmACYktNtPlot+r33HvplZnI+Xa7nG5Bv7I7EXVlZaXW7JOZBRUVFoba2FteuXYNOp8ORI0cwfPhwscPyPNbKHanVGPTQQ/LpMulYWdvafls302l+jXQ42l1HQ8+9EmuCam9vh06nw6uvvgq9Xg+dToe2tjakp6fzdvB9+/ahtLQUfn5+WLZsGV588UUkJydj6tSpCA8P5+045DZrF2iGMV7o7R1gIPZ9AGvvwYTrZjpd5KTBmbqQNPTcK7F28f33f/83Nm7ciKtXryIuLg4Mw8DHxwePPvqoSwe89957zcPIExISzNvHjh2LsWPHuvTahAPXhdjUmrBVH03sEjRs70GhMN5Mt0WlMsZsbTsRjq2WrK3K7R3/9gAaeu4FWBNUYmIiEhMTsXv3bkybNk3ImIi7sF2gO7KVxJy5sPDNlSRDFzlpYPsbtPW32aEeYMf7pzT03LNx3oN66qmnsHnzZvz97383fxGZsjYBsjNbF3opdJG5MolTzPk1nbpGQ8rK3H9MqfL1dWy7iSdONCc2cSaohQsXQqPRoFevXuYvIlMdL9CA8SLdEdeFXgr3ATq8B8aZJCPGRc7KPZc+2dnyGJTiDu3tjm0nXoszQQUGBiIjIwPJycnmLyJjpgs0wwDFxY5d6KVSgkbIytp8sNI16qPVeu/oQdMHJHu3E6/FmaAGDBiA/fv34/fff8e5c+dw7tw5IeIiQnD0Qk8laJwjha5RKZHKBx0ieZwTdauqqixmAysUCmzbts2tQREJs3fROXIHjR60RAMeiJ04E1RxcTFu3LiBixcv4k9/+hMCAwOFiIsQz2Fl9KChe3f4eHOLgeuDjpWKJ5TAvA9ngvr666/x/vvvo729HXFxcVAoFJg3b54QsREibx0vsqGhgL8/8McfgEqF+vnz0Y8uuNZJYb4dkQTOe1Bbt27Fzp070bNnT8ybNw/l5eVCxEWIvHUeudfUBNy8aRyYUlMjSoFa2aCSVOQ2zgTl4+MDpVIJhUIBhUIBf39/IeIiRN6kcJEVuyyVs2hQCbmNM0E9+uijWLx4MS5fvozs7GwMHTpUiLgIkTexL7LO1LuTCinMtyOSwJmgFi9ejMmTJ2P69Ol4+umnsWzZMiHiIkTexL7ISqEF5ywhh6HLtZXpJTgTlEajMVeSuH79Ovbs2SNEXITIm9hzfcRuwblCqPl2cm5legnOUXzz5s3DPffcgz59+gAwzoMihHAQe66P3OdeCTHfTgrFj4lNnAmKYRi88847QsRCiGcRc1IzVW7nJudWppfg7OJ78MEHcfz4ceh0OvMXIUTiqCwVN7HvExJOnC2on376CQcOHDD/rFAoUFFR4dagCCE8oLJUtlErU/I4W1B79+5FRUUFdu3ahW+++YaSEyHe6PZot0GDBzs+2k2qI+WolSl5nC2ow4cPY/ny5QgODkZzczPefPNNPPXUU0LERgiRgg6lhxSAY6WHpF62iFqZksbZglq3bh0++eQT7NmzB59++inWrVsnRFyEEKlwZU6VnOdjEdFxJihfX1+Eh4cDAMLDw3HXXXc5fTCDwYDs7GwkJSVBrVajttMw2JUrV2LKlClQq9VQq9W4ceOG08cihPDE1mg3ru47GilHXMDZxRcUFITi4mI89thj+Pnnn9GjRw+nD1ZeXg6dTofS0lIcO3YMq1evxvvvv2/ef+rUKXz44YcIDQ11+hiEEJ6xzakKDeXuvpP7fCwiKs4W1Jo1a3Dp0iWsW7cO9fX1yM/Pd/pglZWVGDVqFADgkUcewS+//GLeZzAYUFtbi+zsbCQnJ2P37t1OH4fIhFRvnhNLbFUxAO7uO7ErahBZ42xBBQcHIyYmBnfffTcGDBjgUgtKo9EgKCjI/LOvry9u3bqFbt26obW1Fc8//zxmzpyJ9vZ2pKenY8iQIRg0aJDFa3Rc3VdKtFqtZGPjIkbsIWVl6JOdDR+t1rihthaG2bNRf+mS3UtR0DkXSEwMQnJy0LugAH4NDdBHRKAxIwN9ly6FtboyTF0dzpjeG8tzm2NiAAHfv6zOdydyjZ2PuBUMwzC2HpCVlYXW1lY88sgjOHr0KMLDw7F8+XKnDrZq1So8/PDDePbZZwEAo0ePxsGDBwEA7e3tuHnzpjmBvf322xg4cCAmT55sfn5lZSVGjBjh1LHdraqqCtHR0WKH4RRRYu/f33rXT2QkUFNj10vQOReeRdw8/A6FItfzDcg3dkfiZru2c3bx/d///R8KCgowY8YMrF+/HseOHXM80ttiYmLMCenYsWMYOHCgeV9NTQ1SU1PR3t4OvV6Po0ePYvDgwU4fi0gc3Tx3nlS6Rt3dfSeV90lEw9nFp1KpcP78efzpT39CU1OTuWisM8aPH4/vv/8eycnJYBgG+fn52Lp1K1QqFWJjY5GQkIDExET4+flh0qRJGDBggNPHIhJHN8+dI6V5Re4siCul90lEw5mgjh07hgkTJqBv3764fPkylEol/v3f/x0AcOjQIYcO5uPjg9zcXIttUVFR5u/nzJmDOXPmOPSaRKaozIxzpFaB210TXaX2PokoOBMUlTYibiH2chRy5S1do97yPolNnAnqwIED+Oyzz9DW1mbetnnzZrcGReSrpMSBnENlZhznLV2j3vI+iU2cCeqtt95Cbm6uS8PLiXeg2wYC8JauUW95n8QmzgQ1YMAAjBw5UohYiMzRbQMBeEvXqLe8T2ITZ4KKjY1FUlIS7r//fvO2VatWuTUoIk9020Ag3tI16i3vk7DiTFDFxcWYPXs2goODhYiHyIS1e01024AQwifOBNWrVy9z5QdCAPZ7TTNmAEVFdNuAEMIPzgTVvXt3vPjii3jooYegUBgrby1evNjtgRHpYrvX9MUXxgVJ6bYBIYQPnAnq6aefFiIOIiO27jXRbQNCCF84a/ElJCSgtbUVJ06cQHNzMyZOnChEXETC2O4p0b0mIhclJ0vQf11/+PzVB/3X9UfJSarzJ0WcCSo7Oxvnz5/HU089hYsXL2LFihVCxEUkjJb4IXJWcrIEL+17CbXXa8GAQe31Wry07yVKUhLEmaBqa2uxbNkyjBs3DsuXL0cdjRn2emlpxntNkZGAQmH8d9Mm6toj8pBVkYVWveVN1FZ9K7IqslieQcTCeQ+qra0NN2/ehL+/P7RaLdrb24WIi0gc3WsiclV33fqHbLbtRDycLaj09HRMmjQJ8+fPx6RJk/DCCy8IEBZxF1pih3g7VQ/rN0vZthPxcLag/vznP2P06NE4f/487r33Xtx9991CxEXcgG3+Uk5OCGS4YCchTsmLzcNL+16y6OYL8AtAXizdRJUa1haURqPBkiVLoNFo0LNnT9TW1iI3NxcajUbI+AiP2OYvFRT0FicgQkSQNjQNmxI2IbJHJBRQILJHJDYlbELaUOqzlhrWBPXGG29g6NChCAwMBADExcVhyJAhyMnJESo2wjO28S0NDX7CBkLEQ328AIxJqmZRDQxvGFCzqEa05ETD3W1jTVD19fV44YUXzNUjunXrhhdffBHnz58XLDjCL7Z5ShERemEDsYUuoO5j6uOtrQUY5k4fL51jUdBwd26sCcrHx/ouPz/6tC1XbPOXxozRSCMn0AXUvWyth0LuEOhDEg1358aaoCIjI1FeXm6xraKiAr17O3+/wmAwIDs7G0lJSVCr1ajtVPp6586dmDJlChITE/Htt986fRxinbX5SzNmAHv29JRGTmC5gGoW0n9YXtB6KNwE/JBEw925sSaopUuXYseOHXjuueewYMECTJs2DaWlpXjjjTecPlh5eTl0Oh1KS0uxZMkSrF692ryvsbERxcXF2LFjBz766CO8++670Ol0Th/LY/D8aS4tDaipAQwG479ffAFotZZ/BqJ9qGa5UAY01VEjig9Uo4qbgK1MGu7OjXWYeUhICD788ENcunQJV65cQZ8+fRAeHu7SwSorKzFq1CgAwCOPPIJffvnFvO/EiRMYPnw4lEollEolVCoVzpw5g2HDhrl0TFkTYA11SX2oZllQqg4qWpWXD7SMOjcB/0PQcHdunPOg+vbti759+/JyMI1Gg6CgIPPPvr6+uHXrFrp16waNRmOxKGJgYKDVIe1VVVW8xMI3rVbLe2xRmZlQWvk0p8vMRHVMDC/HiIiIQn290sp2Haqqqnk5hr1C5s9Hz9dzEIg777kFAViOPNTVMaiqOmPxeHecc6GIEntMDEJyctC7oAB+DQ3QR0SgMSMDzTExgJ2xyPWc2xt3VEQElPX1XbbrIiJQzfP7jukWg5yYHBScLEBDawMiAiKQMTQDMd1iLGL19HNuEyOg/Px8Zv/+/eafR40aZf6+vLyceeONN8w/z5s3jzlx4oTF848cOeL2GJ11+vRp/l9UoWAYY0+45ZdCwdshtm9nmO7d2y1ePiDAuL3jYyIjjYeNjLTcx7cFYduZc4hk2qFgziGSScF2BjAetzO3nHOByDV2j497+3bjfwBb/yEE5vHnnGG/tnOWOuJTTEwMDh48CAA4duwYBg4caN43bNgwVFZWoq2tDTdu3EB1dbXFfq8kwD2DtDQgN7eetfCr0APrRq5Pw+CAGvjCgPtQg0+RBqUS0GgkMMqQeD6qhCwprF18SUlJ5jlQJgzDQKFQYMeOHU4dbPz48fj++++RnJwMhmGQn5+PrVu3QqVSITY2Fmq1GqmpqWAYBhkZGbjrrrucOo7HEOieQXx8MzIz+1ndZ+uesTv+z5pe07Qqb2go0NwMNDUZt3e8DcdTLychlqgSsmSwJqh3332X94P5+PggNzfXYltUVJT5+8TERCQmJvJ+XNnqfLUWYQ11MQZRdLw+9O9/JzmZmBLkl1+6LwZCiPhYu/j69euHfv364datWygrK8Pnn3+Ozz//HB988IGQ8ZHO48IF/mQn9shkdyVIKlhBiPRx3oNaunQpAODo0aO4cOECrl275vagCL9cuRiLvXpuaKhj2+1BBStcRzXkiBA4E1T37t3x8ssvIzw8HKtXr8bVq1eFiEs2TBf/wYMHSfKTuKsXY0+8Z0wVf1wj9xpylFzlgzNBMQyDxsZGtLS0oLW1FdevXxciLlmwvPgrJPlJnI+LsZi9jH/84dh2e0hqcrIMybmGnDuTKyU+/nEmqFdffRXffPMNJk2ahNjYWIwePVqIuGRBDp/E5X4xdsc9MLHvq8mdnGvIuSu5yr1VKVWcCeqxxx5DXFwcevXqhS+//NJ8T4rI4+Iv94uxO+6BiX1fTe7kXEOOr+TaubW08MuFsm1VShlngiopKUFycjI2bdqEpKQk/OMf/xAiLlmQw8Vf7hdjd9wD43pNGuFnW15sHgL8LP+o5FJDjo/kaq211HSzyepj5dCqlDLOBLVr1y7s27cP7733Hvbs2YNt27YJEZcsyOHi7wmDHNxxD4ztNWmEHzc5L5nOR3K11k3IRg6tSinjLBYbFhYGX19fAMYRfT179nR7UHJhOY+WgUqlEHoerV1oYrz9hK6cIVdpQ9NkkZA6M8WcVZGFuut1UPVQIS82z6H3Ym+rSC6tSinjTFAMw2Dy5MkYPnw4Tp8+jVu3bmHJkiUAgLVr17o9QKkzXfyrqs4gOjpa7HCIi+RwX5G4xtXkquqhQu31rsvChPmHIUgZ5HTiI11xJqi5c+eav09ISHBrMISIjWVJKkndVySOK6stw4SvJ/CSPNjWcVo/YT0lJJ6x3oMyLbn++++/49y5cxZfjz/+OB5//HHBgiREKHK4r0gcU3KyBNlHsnkbAu7KPTiaK+UY1haUqaQRVY4g3kQC9XkJz7IqsqBt11psMw0Bd7bF40w34bz987DxyEYwYADAnChNr0e6Ym1BPffccwCM3Xr9+/fHq6++Cq1Wi8mTJwsWHCFiELk+L+GZFCYWl5wssUhOJjRXyja7isX27t0bADBmzBhkSalMAiGEcJDCxOKsiqwuycmE5kqxs2tF3ZEjRwIwVpUwGAxuDYgQQviUF5uH7r7dLbbZMwScz/tFtpIQzZVix5mgQkJCUFpail9//RW7du1CYGCgEHERiaNqC0Qu0oamIffRXIcGNfBdW48tCSmgoLlSNnAmqNWrV+O3337DmjVrUF1djfz8fCHiIhLmarUFSm5EaPGR8ahZVAPDGwbULKrhHJTAd1FZaxUsFFBg7qNzaYCEDZwJKjQ0FHPnzkVubi7S09Oh1Wq5nkLsJIULdUkJEBsb5VAMbNUWZszgfj6VEiJywPfACmtD04unFKNwYqErYXo8zom6OTk5OHjwIO655x4wDAOFQoEdO3YIEZtHM12oTRd604UaEG7U2J0YlA7FwFZVob2d+/l8lBIqKTENAx9Ew8CJW7BVi3DlfpFcy0OJibMFdeLECZSXl2PHjh0oLS11OjlptVosWLAAqampmDNnDv6wsuLc3LlzkZycDLVajdmzZzt1HDE50iJyZi0pvltczq5nZauqAtfzXS0lJIdFIon8ybliuyfhTFCRkZFoa2tz+UCffvopBg4ciE8++QSTJ09GYWHXpm1dXR0+/fRTFBcX48MPP3T5mEIqKwtxqOvK0Qu1O7rGnE0W1qot2Pt8V5cokcMikUT+5Fyx3ZNwJqj6+no8/fTTSEpKQlJSEpKTk506UGVlJUaNGgUAGD16NH788UeL/VevXkVzczPmzp2LlJQUc6kluSgo6O3QhdPRC7U7LszOJgvTEh63i9w79HxXSwlRMVfibqbh5erP1ACA4inFdg2sIPzjvAflTMXyXbt2oaioyGJbWFgYgoODAQCBgYG4ceOGxX69Xo9Zs2YhPT0d169fR0pKCoYNG4awsDCLx1VVVTkcjxAaGgZZ3V5Xx6Cq6kyX7fPnhyA7uw+02jufEbp3N2D+/HpUVTVbeZ1BABR2v749HI2ho5gYYNUqx58fEwPk5ISgoKA3Ghr8EBGhR0ZGI2JimmHPrzYiIgr19Uor23WoqqrmfgGJ0Gq1kv1btsXT4y6rLUP2kWxzaaTa67WY/Y/ZuHTxEuIj43mNqay2DAUnC9DQ2oCIgAhkDM2wegwpnXN7YwZ4ipthsXPnToZhGOadd95h1q5da/HljPnz5zPHjx9nGIZhmpubmYkTJ1rs1+l0TLaCnPcAABrtSURBVEtLi/nn1157jfn5558tHnPkyBGnji2EPn3aGGPnm+VXZCT7c7ZvN+5XKIz/bt/O/tjIyK6vzfX69ti+3Ri7PTGwPd/e98CH7dsZJiDA8hwEBLj/uHw7ffq02CE4xdPjjiyIZJCDLl+RBZG8xrP9xHYmIC/A4hgBeQHM9hNd/5Clcs4diZlhHIub7drO2sUXEREBwHgP6r777rP4ckZMTAy+++47AMDBgwcxYsQIi/0//PADFi1aBABoaWnB2bNncf/99zt1LDFkZDQ63HXlSM03d1XZTksDKiqqna47J3TdOssVghlZrhBMpEuoun18z7MSghgxs3bxme4XffHFF9iyZYvLB0pJScHSpUuRkpICPz8/c9fh22+/jbi4OIwZMwaHDh1CYmIifHx8sHjxYoSGhrp8XKHExzejb99+bquCTVW276BFIom7uGN4uTVSKGDrKDFi5rwHFRwcjIqKCvTv3x8+PsYGlzOtKH9/f/ztb3/rsv311183fy/3QrTuXlqdlm4nxL3YFiPke3i5UImQT2LEzDmK748//sDHH3+MnJwcZGdn44033nBbMIQQIiahhpe7Ms9KrEUPxZgbZrMFpdFosGnTJvj7+7stAOKcO9UUvLu7jxC+CVHxwfT6WRVZDi1Dbypia2rhCbnoobMxu4K1BbV9+3b8+c9/xqRJk/A///M/bguAOI6PSbsdq1LExkZRJQZCBJY2NM2hAraAfQMV3NnCciZmV7AmqLKyMnz11VfYsWNHlzlNRFxsk3YXLrTv+Z0TXH29ksoFESIDXAMV+F4mRGysCUqpVEKpVCI0NBR6vV7ImAgHtqoJTU2uVSOX+RgVQjwe1+rAchy+botdK+oyjPWliok4bJUSsifJULkgQuSJa6CCHIev28I6SOK3337DkiVLwDCM+XsTZ8ofEf7k5QHPP299nz1JRqUydu9Z204IkS6ugQpyHL5uC2uCWrdunfl7ZwvEEvdISzPeb2pq6rrPniSTl2e5FhXAT1UKQoj72RplKNQ8LqGwdvE9/vjjrF/EvexZ92n9ekDZtWYqNBru+1BpacbVb03VyH18GMyYQcPUCZE7T1smxK57UN6mpATo1QtQKIxfvXoJN8LNkSHk1m4NNjVxDzkvKQGKiowr4AKAwaBAURGN4iPEEwg9FNydKEF1UlICzJpl2X3W1ATMnCnMBdzeEXZZWQDb4EquEXk0io94MrEqLbiTJ74ne1CC6iQrC9Dpum7X64W5gNs7wo5rMISt/TSKj3iqkpMlmLlnpsU8oJl7ZqKstkzs0JxWVlvmUXObHEEJqhNnLux8sneVW67BELb2u7rsOiFStfDLhdAbLLsW9AY98o7Kc5AAABScLPCouU2OoATViTMXdj7Zu+6TtcfZerwzxyBEbppuWhnaCuC6/rrAkfCnobXB6na5zm1yBCWoTvLyrI+O8/MT5gJuuSAfWBfk6/g44M6IPHsW8Ot8jD59dLToHyESFREQYXW7aW6TJ9+fogTVSVoasGULEBZ2Z1tYGLB1q/F7ruHffMVgzyq1pscxDHDrlvFfe1e17XiMiopqSk7EI4T5h1nd3lPZU+BI+JMxNIO1eoSn1d7rjBKUFWlpwNWrxgs+wxi/B1yvIG6PznOg5s0TJikSIia+WgHrJ6yH0teyC0Tpq8Ty4cv5CJOTO1oz8ZHxrHObPK32XmecK+oSI1tDs/lqfZjmQJmOU1sLvP/+nf2mpAhQdxzxHHyuccRWCiimWwy/QVvhzrWa2KpHeFrtvc6oBWUnIYZmW0uCndF8JeJp+G4FiDVRVYzWDFd1c7mjBGUnIYZm25vsaL4S8SSe0goQ432IsQy7kARPUN98841FZfSOdu7ciSlTpiAxMRHffvutwJHZJsTQbHuTHc1XIp7EU1oBYrwPT6u915mgCWrlypVYu3YtDAZDl32NjY0oLi7Gjh078NFHH+Hdd9+FzlpJB5HYO/zbFbbmNpm4Y75SWVkIDcQgovGUVoBY78OTau91pmAEXI3wiy++QGhoKEpLS1FQUGCxr6KiAt999x1yc3MBAPPnz8fLL7+MYcOGmR9TWVmJAK4ruEi0Wi26d+/u8uuUlYWgoKA3Ghr8EBGhx5gxGnz3XZD554yMRsTHN/MQ8Z3jZWdHQKv1NW/r3t2A3Nx6Xo/jDnydczHINXZ3xV1WW4aCkwVoaG1AREAEMoZmID4ynrfXF+p8u+N9eMPfSmtrK0aMGNFlu1tG8e3atQtFRUUW2/Lz8/Hss8/i8OHDVp+j0WgQHBxs/jkwMBAajabL46Kjo/kNlidVVVW8xBYdDWRmmn5SAgjtsFcJoN/tL35MmABotZbbtFofvPdeP2Rm8nccd+DrnItBrrG7K+7o6GhkxmVyP9BJQp1vd7wPb/hbqaystLrdLQlq+vTpmD59ukPPCQoKQktLi/nnlpYWi4RFHFdSYhzxV1dnvG+Vl9e1S5IKxxLinJKTJawr2xJ+SGYe1LBhw7Bu3Tq0tbVBp9OhuroaAwcOFDss2bI2p8raHCpa/p0Qx9ma8wSwL8lOHCN6gtq6dStUKhViY2OhVquRmpoKhmGQkZGBu+66S+zwZMveicV5ecDs2QZotXfGy1DhWEJsY5vztPDLhbh566ZbJut6I8ET1MiRIzFy5EjzzzNnzjR/n5iYiMTERKFD8kj2dt2lpQGXLtXjvff62ewKJITcwTa3yVo1ddNkXUpQjqOJuhLWuS6fI8O/HZlYHB/fbFdxWkK8gT319Byd2yS3ScdSQQlKokz3kJwtTktrPhHiOHurg7PNeWKrpi63ScdSQQlKotjuIc2YYV+LSoiJxYR4Gnvr6bFVcFg/Yb1HTDqWCtEHSRDr2O4htbcb/7WnsnlamuU+U5ch3WvyPAaDATk5Ofj111+hVCqxcuVKRJpWsyR2c6SeHluFcYBG8fGFWlASZc8wb0cqm7vaZUj4xXd5qfLycuh0OpSWlmLJkiVYvXo1H2F6HT7q6Xly6SGhUYKSKHvq8gH2T6i1NeycCKukBMjO7sPrh4XKykqMGjUKAPDII4/gl19+4Sla7yLHuoC05DsRXOd7SL6+1h9n74RaqhghHVlZsJh3Brj+YUGj0SAoKMj8s6+vL27duuX8C3opuVUHpyXfiWjS0mAe/l1U5NqoPCHWsyL2cceHhc6lwgwGA7p1o1vMzpBTF52nL/lOCUomXB2VR8POpcMdHxZiYmJw8OBBAMCxY8eoTJiX8JTFHtlQgpKRji0qRyfU0rBz6cjLMy5p0pGrHxbGjx8PpVKJ5ORkrFq1Cv/1X//lYpREDjxlsUc21AfgRToPOyficEd5KR8fH/NaasR75MXmWRStBaQ/qMMR1IIiRARUXsqziDWSTm6DOhxFLShCCIHz6zvZWnpDiERha8Kw3FELys1cKfhKCBGGK8O1F3650KNH0omJEpQbUfUGQuTB2eHaJSdLrC6xAXjOSDoxUYJyI6reQIg8ODtc21YC85SRdGKiBOVGUqzeQF2OhHTl7HBtWwnMU0bSiYkSlBtJrXoDW5djWVmIOAER3h0/fhxqtVrsMGTH2Rp8bAkszD/MYwcuCIkSlBtJrXoDW5djQUFvcQLyYiFlZbw3ZTdv3owVK1agra3N5dfyNs4O12ZLbOsnrHdnuF5D8GHm33zzDb766iusXbu2y76VK1fi6NGjCAwMBAAUFhYiODhY6BB5Y5rbkpUljTWY2LoWGxr8hA3E25WUoE92NqDVGn+2Z3EvO6hUKmzYsAGvv/46D0F6H3uHa3cejj7j4Rn44uwXtP6TGwiaoFauXIlDhw4hOjra6v5Tp07hww8/RGhoqJBhuZWUqjeoVMZrYWcREXoASsHj8VpZWfAxJScT0+gZF/5YnnnmGVy4cMHF4LyLo3OfrM15KjpexPvk2I5xRQREYM2tNV6Z9ATt4ouJiUFOTo7VfQaDAbW1tcjOzkZycjJ2794tZGhega3LMSOjUZyAvJUUR894IWfmPglRPbxzXPWt9R61hIYj3NKC2rVrF4qKiiy25efn49lnn8Xhw4etPqe1tRXPP/88Zs6cifb2dqSnp2PIkCEYNGiQxeOqqqrcEbLLtFqtZGMziYkBcnJCUFDQGw0NfoiI0CMjoxHjxl1BVVWz2OE5TA7n3JqoiAgo6+u7bNdFRKDaxfdz+fJl3Lx5023nRa7nvGPcZbVlKDhZgPrWrr+DVn0rMr/KREy3GKuvY2s4Ol/nJfOrTKtJ0FZcUsTH34pbEtT06dMxffp0h57j7++P9PR0+Pv7AwCeeOIJnDlzpkuCYuseFFtVVZVkY+soOhrIzDT9pATQD1VVzbKIvTO5nPMu1qyBYfZsy26+gAAo16xx+f0EBwfD39/fbedFrufcFHfJyRLkHM3pkgA6amhtYH2Pqh4q1F7v2k+u6qHi7bw07GxwOC4pcuRvpbKy0up2yYziq6mpQWpqKtrb26HX63H06FEMHjxY7LAI4V9aGupzc92y9sm9996LnTt38hCkZ7LWRdeZrblPQiwJ7+lLaDhC9AS1detWVFRUICoqCgkJCUhMTIRarcakSZMwYMAAscMjxC2a4+OdX9yLOI2rMgRXshGiergQSVAuBB9mPnLkSIwcOdL888yZM83fz5kzB3PmzBE6JEKIl2DrogOAyB6Rdg0Rd3f1cNNrW4zii6NRfARUCogQT8bWOtk+ZTtqFtVIJgmkDU1DzaIaGN4woCK+QjJxCY0SVAdUfZwQz+bpC/x5GlqwsANb1cfpFgEhnsGTF/jzNNSC6oDmTxLi2cRamp04h1pQHbCVAhKr+jgh9tLr9Vi+fDkuXrwInU6HV155BbGxsWKHJSliL81OHEctqA6kVn2ceK6y2jJeP8nv3bsXPXv2xCeffILNmzfjzTff5ClSzyFEmSLCL2pBdSC16uPEM5WcLEH2kWxo242VJPj4JB8XF4dnnnnG/LOvr6/rgXoYZ1fNJeLxqhaUPUPI09Jo/iRxr6yKLHNyMnH1k3xgYCCCgoKg0Wjw2muvYdGiRa6G6XGoQoP8eE2CoiHkRCrc9Um+vr4e6enpmDRpEhISElx6LU9EFRrkx2sSlK0h5IQIyR2f5K9evYpZs2YhMzMT06ZNc/p1PBnNgZIfr7kHRUPIiVTkxeZh9j9mW3TzufpJfuPGjWhubkZhYSEKCwsBGJeA7969u8vxehKaAyUvXpOgaAg5kYq0oWm4dPES3jvzHm/LhK9YsQIrVqzgMUpCxOc1CSovz3jPqWM3Hw0hJ2KJj4xHZlwm9wMJ8WJecw8qLc245I4bluAhhBDiBl7TggKMyYgSEiGEyIPXtKAIIYTICyUoQgghkkQJihBCiCRRgiKEECJJlKAIIYRIEiUoQgghkqRgGIYROwh7VVZWih0CIYQQNxgxYkSXbbJKUIQQQrwHdfERQgiRJEpQhBBCJIkSFCGEEEmiBOWib775BkuWLLG6b+XKlZgyZQrUajXUajVu3LghcHTsbMW9c+dOTJkyBYmJifj2228FjoydVqvFggULkJqaijlz5uCPP/7o8pi5c+ciOTkZarUas2fPFiHKOwwGA7Kzs5GUlAS1Wo3aTuu9SPU8A9yxS/lvGwCOHz8OtVrdZfuBAwcwdepUJCUlYefOnSJEZhtb3Fu3bsXEiRPN5/v3338XITrr9Ho9MjMzkZqaimnTpqGiosJiv0vnnCFOe/PNN5lnnnmGWbRokdX9ycnJTFNTk8BRcbMV95UrV5j4+Himra2NaW5uNn8vBVu2bGH+9re/MQzDMGVlZcybb77Z5TETJkxgDAaD0KFZ9fXXXzNLly5lGIZh/vnPfzJz584175PyeWYY27EzjHT/thmGYTZt2sTEx8cz06dPt9iu0+mYcePGMdeuXWPa2tqYKVOmMFeuXBEpyq7Y4mYYhlmyZAlz8uRJEaLitnv3bmblypUMwzDMH3/8wYwZM8a8z9VzTi0oF8TExCAnJ8fqPoPBgNraWmRnZyM5ORm7d+8WNjgbbMV94sQJDB8+HEqlEsHBwVCpVDhz5oywAbKorKzEqFGjAACjR4/Gjz/+aLH/6tWraG5uxty5c5GSkiJ6q6RjvI888gh++eUX8z4pn2fAduxS/tsGAJVKhQ0bNnTZXl1dDZVKhR49ekCpVGLEiBE4cuSICBFaxxY3AJw6dQqbNm1CSkoKPvjgA4Ejsy0uLg4LFy40/+zr62v+3tVz7lXLbThr165dKCoqstiWn5+PZ599FocPH7b6nNbWVjz//POYOXMm2tvbkZ6ejiFDhmDQoEFChAzAubg1Gg2Cg4PNPwcGBkKj0bg1TmusxR4WFmaOLTAwsEu3kl6vx6xZs5Ceno7r168jJSUFw4YNQ1hYmGBxd6TRaBAUFGT+2dfXF7du3UK3bt0kc57Z2IpdCn/btjzzzDO4cOFCl+1SP+dscQPAxIkTkZqaiqCgILz66qv49ttv8fTTTwscoXWBgYEAjOf3tddew6JFi8z7XD3nlKDsMH36dEyfPt2h5/j7+yM9PR3+/v4AgCeeeAJnzpwR9D+xM3EHBQWhpaXF/HNLS4vFH5hQrMX+6quvmmNraWlBSEiIxf5evXohOTkZ3bp1Q1hYGKKjo3Hu3DnRElTnc2kwGNCtWzer+8Q6z2xsxS6Fv21nSP2cs2EYBjNmzDDHOmbMGJw+fVoyCQoA6uvrMX/+fKSmpiIhIcG83dVzTl18blJTU4PU1FS0t7dDr9fj6NGjGDx4sNhhcRo2bBgqKyvR1taGGzduoLq6GgMHDhQ7LADGrsnvvvsOAHDw4MEuM89/+OEH86e3lpYWnD17Fvfff7/gcZrExMTg4MGDAIBjx45ZnEcpn2fAduxy/duOiopCbW0trl27Bp1OhyNHjmD48OFih8VJo9EgPj4eLS0tYBgGhw8fxpAhQ8QOy+zq1auYNWsWMjMzMW3aNIt9rp5zakHxbOvWrVCpVIiNjUVCQgISExPh5+eHSZMmYcCAAWKHx6pj3Gq1GqmpqWAYBhkZGbjrrrvEDg8AkJKSgqVLlyIlJQV+fn5Yu3YtAODtt99GXFwcxowZg0OHDiExMRE+Pj5YvHgxQkNDRYt3/Pjx+P7775GcnAyGYZCfny+L8wxwxy6nv+19+/ahtbUVSUlJWLZsGV588UUwDIOpU6ciPDxc7PBYdYw7IyMD6enpUCqVePLJJzFmzBixwzPbuHEjmpubUVhYiMLCQgDGHpCbN2+6fM6p1BEhhBBJoi4+QgghkkQJihBCiCRRgiKEECJJlKAIIYRIEiUoQgghkkQJiniUw4cP48knnzQX1UxMTERxcXGXxx08eBClpaUOvfZnn33WpRAmlwsXLiAxMbHL9uvXr2P58uVIS0tDcnIyMjIyJFdwlUtpaSn0er3VfbaKERNiL5oHRTzOE088gYKCAgCATqdDXFwcJk2aZFF5YvTo0Q6/7pQpU3iLcfHixUhOTsb48eMBAB9//DGys7PNccvBBx98gMmTJ3fZvnLlShw6dAjR0dEiREU8CSUo4tE0Gg18fHzg6+sLtVqNu+++G83NzZg4cSJqa2uRnJyMJUuWICIiAufPn8fQoUPx17/+FU1NTVi2bBlu3LgBhmHw1ltvYd++fejVqxfuv/9+bNy4ET4+PmhsbERSUhLS0tLw008/4e9//zsA49Igb731Fvz8/LrEdPHiRVy9etWcnABArVZj6tSpAIC9e/eiqKgISqUS/fv3R25uLvbt24dvv/0WWq0WjY2NSE9PR0VFBc6ePYvXX38d48aNQ2xsLB5++GHU1dVhwIAByMvLg0ajQWZmJjQaDdrb27Fw4UI8+eSTSEhIwOOPP45ff/0VCoUChYWFCA4Oxtq1a/Hzzz+DYRi88MILmDBhAtRqNQYNGoSzZ89Co9Fg/fr1+OGHH9DY2IiMjAzz5EyTmJgYjBs3zuEWKiGdUYIiHud///d/oVaroVAo4Ofnh7/85S/mgpYJCQkYP348PvvsM/Pja2pq8NFHH8Hf3x/jxo1DY2MjPvjgA4wdOxYpKSn48ccfceLECYtjXL58GXv27IHBYEBCQgLi4uJw9uxZrFmzBuHh4di4cSO++uori7pkJleuXMG9995rsc3X1xfBwcH417/+hQ0bNuDzzz9HUFAQ8vPzUVpaioCAALS0tGDLli3Yv38/Pv74Y+zcuROHDx/Gtm3bMG7cOFy+fBkLFy5EZGQkFi5ciPLycvzzn//Ev/3bv2HGjBm4fPkyUlJSUF5ejpaWFkycOBF/+ctfsGTJEhw8eBBBQUG4cOECduzYgba2NiQmJuKpp54CYCzNlJWVhYKCAuzfvx8vvfQS3n//fastPlvFiAlxBCUo4nE6dvF1dt9993XZplKpzJW7e/fujba2Npw7d85cV+zJJ58EAIulEExLZQDAgAEDUFdXh/DwcOTl5SEgIACXL19GTEyM1Rj69u2LhoYGi216vR5fffUVIiMj8cADD5jjeeyxx3Do0CE8/PDD5i6z4OBgREVFQaFQoEePHmhrawMA9OnTB5GRkeb4zp07h+rqanOSDA8PR1BQkHmhx4ceesj8vLa2Nly6dAmnTp0yL5h369YtXLp0yeKxERERuHr1qtX3RQjfaJAE8SoKhcKubVFRUTh58iQA4Oeff8aaNWss9ldVVaG9vR03b97Eb7/9hsjISKxYsQL5+flYvXo17rnnHrBVEQsPD8fdd9+N8vJy87Zt27ahvLwc9957L6qrq9Ha2goA+Omnn8xJ1VqcHV2+fBmNjY0AgKNHj+KBBx5AVFSUef2dy5cvo7m5GT179rT6evfffz9GjhyJ4uJiFBUVYcKECV1aeh0pFAoYDAabMRHiCmpBEWLF3LlzsXz5cuzduxeAcR2tPXv2mPffunULc+bMwbVr1/DKK68gNDQUkyZNQmJiIkJCQtCrVy9cuXKF9fXffvtt5ObmYsuWLdDr9VCpVFi5ciWCg4OxYMECpKenw8fHByqVCv/5n/+J/fv3c8asVCrx5ptvor6+Hg8//DDGjh2LESNGYPny5fj666+h1WqRm5trXjajs7Fjx+Knn35CamoqWltbMW7cOIs1oTp79NFH8dJLL2Hbtm2cyZMQZ1CxWEIcdPjwYezYsUNyI+6eeuopfP/992KHQQhvqIuPEEKIJFELihBCiCRRC4oQQogkUYIihBAiSZSgCCGESBIlKEIIIZJECYoQQogk/T+ORK2qexYH3gAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "sc = StandardScaler()\n",
    "\n",
    "pca1 = PrincipalComponentAnalysis(n_components=2, whitening=True)\n",
    "\n",
    "X_train_scaled = sc.fit_transform(X_train)\n",
    "X_train_transf = pca1.fit(X_train_scaled).transform(X_train_scaled)\n",
    "\n",
    "\n",
    "with plt.style.context('seaborn-whitegrid'):\n",
    "    plt.figure(figsize=(6, 4))\n",
    "    for lab, col in zip((0, 1, 2),\n",
    "                        ('blue', 'red', 'green')):\n",
    "        plt.scatter(X_train_transf[y_train==lab, 0],\n",
    "                    X_train_transf[y_train==lab, 1],\n",
    "                    label=lab,\n",
    "                    c=col)\n",
    "    plt.xlabel('Principal Component 1')\n",
    "    plt.ylabel('Principal Component 2')\n",
    "    plt.legend(loc='lower center')\n",
    "    plt.tight_layout()\n",
    "    plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Covariance matrix:\n",
      "\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "array([[1., 0.],\n",
       "       [0., 1.]])"
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "np.set_printoptions(precision=1, suppress=True)\n",
    "\n",
    "print('Covariance matrix:\\n')\n",
    "np.cov(X_train_transf.T)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As we can see above, the whitening achieves that all features now have unit variance. I.e., the covariance matrix of the transformed features becomes the identity matrix."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## API"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "## PrincipalComponentAnalysis\n",
      "\n",
      "*PrincipalComponentAnalysis(n_components=None, solver='svd', whitening=False)*\n",
      "\n",
      "Principal Component Analysis Class\n",
      "\n",
      "**Parameters**\n",
      "\n",
      "- `n_components` : int (default: None)\n",
      "\n",
      "    The number of principal components for transformation.\n",
      "    Keeps the original dimensions of the dataset if `None`.\n",
      "\n",
      "- `solver` : str (default: 'svd')\n",
      "\n",
      "    Method for performing the matrix decomposition.\n",
      "    {'eigen', 'svd'}\n",
      "\n",
      "- `whitening` : bool (default: False)\n",
      "\n",
      "    Performs whitening such that the covariance matrix of\n",
      "    the transformed data will be the identity matrix.\n",
      "\n",
      "**Attributes**\n",
      "\n",
      "- `w_` : array-like, shape=[n_features, n_components]\n",
      "\n",
      "    Projection matrix\n",
      "\n",
      "- `e_vals_` : array-like, shape=[n_features]\n",
      "\n",
      "    Eigenvalues in sorted order.\n",
      "\n",
      "- `e_vecs_` : array-like, shape=[n_features]\n",
      "\n",
      "    Eigenvectors in sorted order.\n",
      "\n",
      "- `e_vals_normalized_` :  array-like, shape=[n_features]\n",
      "\n",
      "    Normalized eigen values such that they sum up to 1.\n",
      "    This is equal to what's often referred to as\n",
      "    \"explained variance ratios.\"\n",
      "\n",
      "- `loadings_` : array_like, shape=[n_features, n_features]\n",
      "\n",
      "    The factor loadings of the original variables onto\n",
      "    the principal components. The columns are the principal\n",
      "    components, and the rows are the features loadings.\n",
      "    For instance, the first column contains the loadings onto\n",
      "    the first principal component. Note that the signs may\n",
      "    be flipped depending on whether you use the 'eigen' or\n",
      "    'svd' solver; this does not affect the interpretation\n",
      "    of the loadings though.\n",
      "\n",
      "**Examples**\n",
      "\n",
      "For usage examples, please see\n",
      "    [https://rasbt.github.io/mlxtend/user_guide/feature_extraction/PrincipalComponentAnalysis/](https://rasbt.github.io/mlxtend/user_guide/feature_extraction/PrincipalComponentAnalysis/)\n",
      "\n",
      "### Methods\n",
      "\n",
      "<hr>\n",
      "\n",
      "*fit(X, y=None)*\n",
      "\n",
      "Learn model from training data.\n",
      "\n",
      "**Parameters**\n",
      "\n",
      "- `X` : {array-like, sparse matrix}, shape = [n_samples, n_features]\n",
      "\n",
      "    Training vectors, where n_samples is the number of samples and\n",
      "    n_features is the number of features.\n",
      "\n",
      "**Returns**\n",
      "\n",
      "- `self` : object\n",
      "\n",
      "\n",
      "<hr>\n",
      "\n",
      "*get_params(deep=True)*\n",
      "\n",
      "Get parameters for this estimator.\n",
      "\n",
      "**Parameters**\n",
      "\n",
      "- `deep` : boolean, optional\n",
      "\n",
      "    If True, will return the parameters for this estimator and\n",
      "    contained subobjects that are estimators.\n",
      "\n",
      "**Returns**\n",
      "\n",
      "- `params` : mapping of string to any\n",
      "\n",
      "    Parameter names mapped to their values.'\n",
      "\n",
      "    adapted from\n",
      "    https://github.com/scikit-learn/scikit-learn/blob/master/sklearn/base.py\n",
      "    Author: Gael Varoquaux <gael.varoquaux@normalesup.org>\n",
      "    License: BSD 3 clause\n",
      "\n",
      "<hr>\n",
      "\n",
      "*set_params(**params)*\n",
      "\n",
      "Set the parameters of this estimator.\n",
      "The method works on simple estimators as well as on nested objects\n",
      "(such as pipelines). The latter have parameters of the form\n",
      "``<component>__<parameter>`` so that it's possible to update each\n",
      "component of a nested object.\n",
      "\n",
      "**Returns**\n",
      "\n",
      "self\n",
      "\n",
      "adapted from\n",
      "https://github.com/scikit-learn/scikit-learn/blob/master/sklearn/base.py\n",
      "Author: Gael Varoquaux <gael.varoquaux@normalesup.org>\n",
      "License: BSD 3 clause\n",
      "\n",
      "<hr>\n",
      "\n",
      "*transform(X)*\n",
      "\n",
      "Apply the linear transformation on X.\n",
      "\n",
      "**Parameters**\n",
      "\n",
      "- `X` : {array-like, sparse matrix}, shape = [n_samples, n_features]\n",
      "\n",
      "    Training vectors, where n_samples is the number of samples and\n",
      "    n_features is the number of features.\n",
      "\n",
      "**Returns**\n",
      "\n",
      "- `X_projected` : np.ndarray, shape = [n_samples, n_components]\n",
      "\n",
      "    Projected training vectors.\n",
      "\n",
      "\n"
     ]
    }
   ],
   "source": [
    "with open('../../api_modules/mlxtend.feature_extraction/PrincipalComponentAnalysis.md', 'r') as f:\n",
    "    s = f.read()\n",
    "print(s)"
   ]
  }
 ],
 "metadata": {
  "anaconda-cloud": {},
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.7"
  },
  "toc": {
   "nav_menu": {},
   "number_sections": true,
   "sideBar": true,
   "skip_h1_title": false,
   "title_cell": "Table of Contents",
   "title_sidebar": "Contents",
   "toc_cell": false,
   "toc_position": {},
   "toc_section_display": true,
   "toc_window_display": false
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}