{
 "metadata": {
  "name": "",
  "signature": "sha256:85f2a6a4928ac54278684064ce0a61a587b66d3c8dc844846878042448ec0721"
 },
 "nbformat": 3,
 "nbformat_minor": 0,
 "worksheets": [
  {
   "cells": [
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "# - compatibility with Python 3\n",
      "from __future__ import print_function  # print('me') instead of print 'me'\n",
      "from __future__ import division  # 1/2 == 0.5, not 0"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [],
     "prompt_number": 1
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "# - show figures inside the notebook\n",
      "%matplotlib inline"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [],
     "prompt_number": 2
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "# - import common modules\n",
      "import numpy as np  # the Python array package\n",
      "import matplotlib.pyplot as plt  # the Python plotting package"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [],
     "prompt_number": 3
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "# - import numpy.linalg with a shorter name\n",
      "import numpy.linalg as npl"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [],
     "prompt_number": 4
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "## Introducing principal component analysis"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "This page was largely inspired by these two excellent tutorials:\n",
      "\n",
      "* http://randomanalyses.blogspot.com/2012/01/principal-components-analysis.html\n",
      "* https://liorpachter.wordpress.com/2014/05/26/what-is-principal-component-analysis\n",
      "\n",
      "Let's say I have some data in a 2D array $\\mathbf{X}$.\n",
      "\n",
      "I have taken two different measures - or *variables* - and 50 samples.  So, I have two variables for each of the 50 samples.\n",
      "\n",
      "Each column is one sample (I have 50 columns).  Each row is one variable (I have two rows)."
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "# Make some random, but predictable data\n",
      "np.random.seed(1966)\n",
      "X = np.random.multivariate_normal([0, 0], [[3, 1.5], [1.5, 1]], size=50).T\n",
      "X.shape"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 5,
       "text": [
        "(2, 50)"
       ]
      }
     ],
     "prompt_number": 5
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "To make things simpler, I will subtract the mean across samples from each variable:"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "# Subtract mean across samples (mean of each variable)\n",
      "x_mean = X.mean(axis=1)\n",
      "X[0] = X[0] - x_mean[0]\n",
      "X[1] = X[1] - x_mean[1]"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [],
     "prompt_number": 6
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "The values for the two variables (rows) in $\\mathbf{X}$ are somewhat correlated:"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "plt.scatter(X[0], X[1])\n",
      "plt.axis('equal')"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 7,
       "text": [
        "(-5.0, 4.0, -3.0, 3.0)"
       ]
      },
      {
       "metadata": {},
       "output_type": "display_data",
       "png": "iVBORw0KGgoAAAANSUhEUgAAAXAAAAEACAYAAACqOy3+AAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAEvlJREFUeJzt3X2MXOV1x/HfWYPFVqihrlsbAhGoJU1DWrAVUdSgMhEd\nr4taUxtFbSRak1S1+hIaWRvqGqvxGohcSjdESdRGVgh2GkEqhYCgkB2vCBNipXJbYhsT24VUSgtB\nkBKiqBGuSNjTP+bui5fdmTtz357nzvcjjTSzM3Pv2Vnt2WfPc57nmrsLABCfkaoDAAAMhgQOAJEi\ngQNApEjgABApEjgARIoEDgCRypTAzewcMztsZkfN7ISZ7c0rMABAd5a1D9zMfsrdXzOzsyQdkvQR\ndz+US3QAgGVlLqG4+2vJ3ZWSVkh6NesxAQC9ZU7gZjZiZkclvSzpCXc/kT0sAEAveYzAZ9z9CkkX\nSvoNM2tkjgoA0NNZeR3I3X9oZo9Kerek9uzXzYzNVgBgAO5u3Z7P2oWy2szOS+6PSmpKOrJEEEHd\ndu/eXXkMMcQUalzEREzDEFcaWUfg50s6YGYj6vwx+Ed3fzzjMQEAKWRK4O5+XNL6nGIBAPRhKFdi\nNhqNqkN4kxBjksKMi5jSIab0Qo2rl8wLeXqewMyLPgcA1I2ZyYucxAQAVIcEDgCRIoEDQKRI4AAQ\nKRI4AESKBA4AkSKBA0CkSOAAECkSOABEigQOAJEigQNApEjgABApEjgARIoEDgCRIoEDQKRI4AAQ\nKRI4AESKBA4AkSKBA0CkSOAAECkSOABEigQOAJEigQNApEjgABApEjgARIoEDgB9arVa2rDhBm3Y\ncINarVZlcZi7D/5ms4skfV7Sz0tySfvc/ZOLXuNZzgEAIWm1Wtq8eatOn75TkjQ6ukMPPnhAY2Nj\nuZ7HzOTu1vU1GRP4Wklr3f2omZ0r6SlJv+vuJxe8hgQOoDY2bLhB09ObJG1NvnJAzebDOnjwgVzP\nkyaBZyqhuPtL7n40uf8jSSclXZDlmACAdM7K60BmdrGkdZIO53VMAAjN+Pg2HTq0VadPdx6Pju7Q\n+PiBSmLJVEKZO0infNKWdIe7P7ToOUooAGql1WppcnKfpE5Cz7v+LZVQA09Ocrakf5b0FXf/xBLP\n++7du+ceNxoNNRqNTOcEgLppt9tqt9tzj/fs2VP4JKZJOiDp++6+fZnXMAIHgD6V0YVytaQnJT2t\nThuhJO1096kFryGBA0CfSimhpAiCBA4AfSq8jRAAUB0SOABEigQOAJEigQNApEjgABApEjgARIoE\nDgCRIoEDQKRI4AAQKRI4gNKEcimyumApPYBSlHUpsrpgLxQAwSjrUmR1wV4oANCnmMo8uV1SDQC6\nCelSZMtZXOY5dGhr0GUeSigASlPGpciyCKnMk6aEwggcQGnGxsaCS9oxowaOKMRUl0S8xse3aXR0\nhzpXijyQlHm2VR3WsiihIHi0n6FMoZR5aCNELYRUlwTKQhshANQYk5gIXgztZ0AVKKEgCqHUJavG\n5zA8qIEDNcJk7nAhgQM1wmTucGESEwBqjElMIBJM5mIxSihARJjEHB7UwAEgUtTAAaDGMidwM/uc\nmb1sZsfzCAgAkE4eI/B7JW3M4TgAgD5kTuDu/nVJP8ghFgAlYGve+qCNEBgisV0yDN2VksAnJibm\n7jcaDTUajTJOC2CRycl9SfLurOY8fbrztaITOO2PvbXbbbXb7b7eU3oCBzBcGPWns3hwu2fPnp7v\noYQCDJEqVnNWNeofBnm0Ed4v6RuS3m5mz5vZB7KHBaAIY2NjevDBziZYzebDjIQjx0pMAIViG9zB\nsJQeQGm6TVQyidk/EjhQM1kTYVGJlFF2/tIkcLl7obfOKQBkNTU15aOja1za79J+Hx1d41NTU6W9\nv9txV636heS4ntz2e7O5JfOxh1mSO7vmV7pQgEhk7eYoohtkfuR9ycDHwOBI4AAGNv9HYa3mL/XG\nxSbKQgIHBlT2xFzWHu5ie8DHJB2QNKFVq/5H991H/bsUvWosWW+iBo4aKqqenOa8zeYWbza3DHS+\nrO9f6nhVfA7DQClq4HShAAPgCvHzaBEsRpouFEooQAqLkxTmjY2NkbQrQgIHelhqM6Zdu27WoUM7\nuEI8KsU1MYEezmy/6yTyr33tm+wp0gUXjSgHI3BgQJQOlsb2seUhgQM9VLEFa8zYPrY8lFCAHvLe\ngjXk8kLIsWEJvfoMs95EHzgwJ+S+6bxiC/l7jInoA8cwiKkPOeT+8Txji+lnEir6wFF7TJjlo9Vq\n6amnjknalMvxmOAtBwkcUYttwizECdH5P4I3SvrI3NdDiA3dMYkJlGjxhOiuXTdrcnJfpZOG838E\n/07SFyR9RqtW3d7zPxkmPAPQq0ie9SYmMVGgkCfMum0cNTU15evWXeMjIz/r0nilsTebW/q+GEPI\nn3tdKMUkJgkc0ct7h708ztUtwS1+Tlrj0lRlV7GZmprylSvPc+kql67ylSvP6/k5DpL00Z80CZwa\nOKJX1oRZPxOm3Wrzi5/r2Ke8JhAHc7akP0nu31JhHOgHCRxIqdgJ0xcrmzScnNyn11+/S7Pf1+uv\n9/6+QpyMHUZMYqLWip5oe+WV7y/59fHxbRod3aHOVWoOJAlu25LPjYxs17p1K6Jqf8x7dSoG1KvG\nkvUmauCoSN4TbZ1a8c8tqF2v7lov7jWJWVbdvhcmJMMkVmJimBWx6nH9+qt15Mgbki6QtE3SS8Gs\npMyClZPhSbMSkxIK0IfVq9eoM9n3gDoX8s1PlX3VY2NjOnjwAR08+EDP5E3/d0B6DdGz3kQJBUso\no4RQRGmgqHJDLGWMWOKsA5XRBy5po6RTkp6TtGOJ50v5ZhGPO+64w0dGfqaUJFDEH4oijpmmrzqE\nujn93+VJk8AztRGa2QpJn5b0m5K+K+nfzOxhdz+Z5bior1arpY9+9G7NzNytMvYvKaJHvIqNmti0\nC0vJ2gd+paRvu/t3JMnMvijpekkkcCxpcnKfZmYurTqM4PTqqw5l0y76v8OSNYG/VdLzCx6/IOnX\nMh4TtfceSTvmHo2MbNf4+P3VhROA2b7q+U6QMEfXscQ5LLIm8FT9gRMTE3P3G42GGo1GxtMiVvMj\nuBslfUYjI8/pttvGSQLqXpoJaeTLXt/FaLfbarfbfb0nUx+4mV0lacLdNyaPd0qacfc7F7zGs5wD\n9UPP8WD43IZLmj7wrAn8LEn/IelaSS9K+ldJ7184iUkCB4D+FX5JNXf/iZl9SFJL0gpJ99CBAgDl\nYCk9AASIpfRACVhajqowAgcyWLzAZnR0BwtskIvCJzFTBkECR20VseMhIFFCATKjPIKQcUk1YBlp\n9h8JaYENhg8jcGAZZ+4/0knkswtpZs0uLV+37l6tWnW73vGOX6wkVgwnEjiQg1OnTunVV/9aR478\nsTZv3kq5BaWghAIsI215JJSdAjF8SODAMth5D6GjjRDRCW1TJ3rBUQT6wFE7oSbL0P6oIH4kcNQO\nC2cwLFjIg2ixgAbojQSOgRWVZGfLJNPTmzQ9vemMtrzx8W0aHd0h6YCkA0lnyLbczl02/lAhk16X\nrc9665wCdTM1NeWjo2tc2u/Sfh8dXeNTU1O5HLvZ3JIc15Pbfm82t5xx7mZzizebW3I7ZxWK/AwR\nvyR3ds2vtBFiIFX2Ptflmoz0jyMrEjiCw/4iQDokcAykyCQ7LAto+EOFrGgjxMDofc6OzxDLoQ8c\nQVouaZHMgHkkcASl1Wpp5869OnbsGc3M3CTpV+ZWUkoKcoUlUBUSOIKxeAm8NNvL/ZKazYcliRWW\nwAJpEjiTmCjF4pa5jn2SNlUUERA/Ejgq9OIZnRd0ZAD9IYGjFItb5kZGtuvyy9+pvXvn69zD0DoI\n5IkaOEpDlwmQHpOYABAptpMFAsCOgyjKwAnczN5nZt8yszfMbH2eQQF10W1rXCCrLCPw45I2S3oy\np1iA2jmzfbLTBz87DwBkNXAXirufkjp1GgBA+aiBIxh1rBXX7QpCCEvXEbiZTUtau8RTt7r7I2lP\nMjExMXe/0Wio0WikfSuGxOKl9ocOba3FXijDsjUusmu322q32329J3MboZk9IWnc3b+5zPO0EaIn\nrjYPnKnMNkIK4RWqY+kBQG8DT2Ka2WZJn5S0WtKjZnbE3X8rt8iQSl1KD1ydBugfKzEjV6fSA0vt\ngXlsJ4s5MSTHulxtHigLCTxyaUoPdSmzADgTJZQa6DW6rlOZBRgWlFCGBKUHYDixEnMIhLgakNZH\nIDtKKEMipEnMxTV5rkAPvBkXdECQqMkDvXFBBwCoMSYxUTpWXQL5oISCSoRUkwdCRA0cACJFDRx9\nobUPiAsjcEiitQ8IDSUUpEZrHxAWSigAUGO0EUISrX1AjCihYA6tfUA4qIEDQKSogaNwtB4C1WEE\njoHReggUhxIKCkXrIVAcSigAUGO0EWJgtB4C1aKEgkxoPQSKQQ0cACJFDRwAaowEDgCRIoEDQKQG\nTuBmdpeZnTSzY2b2ZTN7S56BAQC6yzICPyjpMne/XNKzknbmExIAII2BE7i7T7v7TPLwsKQL8wkJ\nAJBGXjXwD0p6LKdjFY4NmADUQdc+cDOblrR2iadudfdHktfskrTe3W9Y5hhB9YGzAROAGKTpA++6\nlN7dmz1OcJOk6yRd2+11ExMTc/cbjYYajUa3lxdqcnJfkrw7GzCdPt35GgkcQJXa7bba7XZf7xl4\nJaaZbZQ0Kekad3+ly+uCGoGzgx6AGGQegffwKUkrJU2bmST9i7v/WYbjlYINmADUxVDuhcIGTNnw\n+QHFYzMr5I5JYKAcJHDkjjkEoBzsRggANcYVedAXJoGBcFBCQd+YxASKRw0cACJFDRwAaowEDgCR\nIoEDQKRI4AAQKRI4AESKBA4AkSKBA0CkSOAAECkSOABEigQOAJEigQNApEjgABApEjgARIoEDgCR\nIoEDQKRI4AAQKRI4AESKBA4AkSKBA0CkSOAAECkSOABEigQOAJEaOIGb2e1mdszMjprZ42Z2UZ6B\nAQC6yzIC/1t3v9zdr5D0kKTdOcVUuHa7XXUIbxJiTFKYcRFTOsSUXqhx9TJwAnf3/13w8FxJr2QP\npxwh/rBCjEkKMy5iSoeY0gs1rl7OyvJmM/uYpD+Q9Jqkq3KJCACQStcRuJlNm9nxJW6/I0nuvsvd\n3yZpv6S7S4gXAJAwd89+ELO3SXrM3d+1xHPZTwAAQ8jdrdvzA5dQzOxSd38ueXi9pCODBAAAGMzA\nI3Az+5KkX5L0hqT/lPSn7v69HGMDAHSRSwkFAFC+UlZimtmEmb1gZkeS28YyzpuGmY2b2YyZrQog\nluAWR5nZXWZ2Monry2b2lgBiep+ZfcvM3jCz9RXHstHMTpnZc2a2o8pYZpnZ58zsZTM7XnUss8zs\nIjN7Ivm5PWNmfxFATOeY2eHk9+2Eme2tOqZZZrYiyZWPdHtdWUvpXdLH3X1dcpsq6bxdJQmyKem/\nqo4lEeLiqIOSLnP3yyU9K2lnxfFI0nFJmyU9WWUQZrZC0qclbZT0TknvN7NfrjKmxL3qxBSSH0va\n7u6XqdNy/OdVf1bu/n+S3pv8vv2qpPea2dVVxrTAhyWdUCd3LqvMvVBCnMz8uKS/rDqIWSEujnL3\naXefSR4elnRhlfFIkrufcvdnq45D0pWSvu3u33H3H0v6ojoT+pVy969L+kHVcSzk7i+5+9Hk/o8k\nnZR0QbVRSe7+WnJ3paQVkl6tMBxJkpldKOk6SZ9Vj7xZZgK/Ofk3/B4zO6/E8y7JzK6X9IK7P111\nLAuZ2cfM7L8lbZX0N1XHs8gHJT1WdRABeauk5xc8fiH5Grows4slrVNnQFApMxsxs6OSXpb0hLuf\nqDomddbU3CJpptcLM63EXMjMpiWtXeKpXZL+QdJtyePbJU1K+qO8zj1gTDslbVj48qLj6RHTre7+\niLvvkrTLzP5KnR/kB6qOKXnNLkmvu/t9RceTNqYA0AHQJzM7V9KXJH04GYlXKvnv8opkbqdlZg13\nb1cVj5n9tqTvufsRM2v0en1uCdzdm2leZ2aflVTKL+ByMZnZuyRdIumYmUmdssBTZnZl0a2QaT8n\nSfeppNFur5jM7CZ1/qW7tox4pL4+pyp9V9LCieaL1BmFYwlmdrakByR9wd0fqjqehdz9h2b2qKR3\nS2pXGMqvS9pkZtdJOkfST5vZ5939D5d6cVldKOcveLhZnUmoyrj7M+6+xt0vcfdL1PmlW191H7uZ\nXbrg4bKLo8qUdAzdIun6ZNInNFXOrfy7pEvN7GIzWynp9yQ9XGE8wbLOSOkeSSfc/RNVxyNJZrZ6\ntpxrZqPqNDRU+jvn7re6+0VJXvp9SV9dLnlL5dXA7zSzp83smKRrJG0v6bxphfKv8N5kr5mjkhqS\nxiuOR5I+pc6E6nTS1vT3VQdkZpvN7Hl1uhkeNbOvVBGHu/9E0ocktdTpGPgndz9ZRSwLmdn9kr4h\n6e1m9ryZFV6GS+E9km5Up9MjlHbi8yV9Nfl9OyzpEXd/vOKYFuuam1jIAwCR4pJqABApEjgARIoE\nDgCRIoEDQKRI4AAQKRI4AESKBA4AkSKBA0Ck/h+oGee65al33gAAAABJRU5ErkJggg==\n",
       "text": [
        "<matplotlib.figure.Figure at 0x105fc15d0>"
       ]
      }
     ],
     "prompt_number": 7
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "We want to explain the variation in these data."
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "The variation we want to explain is given by the sum of squares of the data values.  "
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "squares = X ** 2\n",
      "squares.sum()"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 8,
       "text": [
        "155.66928985835315"
       ]
      }
     ],
     "prompt_number": 8
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "The sums of squares of the data can be thought of as the squared lengths of the 50 2D vectors in the columns of $\\mathbf{X}$.\n",
      "\n",
      "We can think of each sample as being a point on a 2D coordinate system, where the first variable is the position on the x axis, and the second is the position on the y axis.  In fact, this is how we just plotted the values in the scatter plot.  We can also think of each column as a 2D *vector*.  Call $\\vec{v_j}$ the vector contained in column $j$ of matrix $\\mathbf{X}$, where $j \\in 1..50$."
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "The sum of squares across the variables, is also the squared distance of the point (column) from the origin (0, 0).  That is the same as saying that the sum of squares is the squared *length* of $\\vec{v_j}$. This can be written as $\\|\\vec{v_j}\\|^2$"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "Take the first column / point / vector as an example ($\\vec{v_1}$):"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "v1 = X[:, 0]\n",
      "v1"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 9,
       "text": [
        "array([ 3.3783217 ,  2.06815777])"
       ]
      }
     ],
     "prompt_number": 9
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "# Show first vector as sum of x and y axis vectors\n",
      "x, y = v1\n",
      "# Make subplots for vectors and text\n",
      "fig, (vec_ax, txt_ax) = plt.subplots(1, 2, figsize=(14, 4))\n",
      "# Plot 0, 0\n",
      "vec_ax.plot(0, 0, 'ro')\n",
      "# Show vectors as arrows\n",
      "vec_ax.arrow(0, 0, x, 0, color='r', length_includes_head=True, width=0.01)\n",
      "vec_ax.arrow(0, 0, x, y, color='k', length_includes_head=True, width=0.01)\n",
      "vec_ax.arrow(x, 0, 0, y, color='b', length_includes_head=True, width=0.01)\n",
      "# Label origin\n",
      "vec_ax.annotate('$(0, 0)$', (-0.3, -0.3), fontsize=16)\n",
      "# Label vectors\n",
      "vec_ax.annotate(r'$\\vec{{v_1}} = ({x:.2f}, {y:.2f})$'.format(x=x, y=y),\n",
      "                (x / 2 - 2.2, y / 2), fontsize=16)\n",
      "vec_ax.annotate(r'$\\vec{{x}} = ({x:.2f}, 0)$'.format(x=x),\n",
      "                (x / 2 - 0.5, -0.3), fontsize=16)\n",
      "vec_ax.annotate(r'$\\vec{{y}} = (0, {y:.2f})$'.format(y=y),\n",
      "                (x + 0.1, y / 2 - 0.1), fontsize=16)\n",
      "# Make sure axes are correct lengths\n",
      "vec_ax.axis((-1, 5.5, -1, 3))\n",
      "vec_ax.set_aspect('equal', adjustable='box')\n",
      "vec_ax.set_title(r'x- and y- axis components of $\\vec{v_1}$')\n",
      "# Text about lengths\n",
      "txt_ax.axis('off')\n",
      "txt_ax.annotate(r'$\\|\\vec{v_1}\\|^2 = \\|\\vec{x}\\|^2 + \\|\\vec{y}\\|^2$ =' +\n",
      "                '${x:.2f}^2 + {y:.2f}^2$'.format(x=x, y=y),\n",
      "                (0, 0.5), fontsize=16)"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 10,
       "text": [
        "<matplotlib.text.Annotation at 0x1064a4410>"
       ]
      },
      {
       "metadata": {},
       "output_type": "display_data",
       "png": "iVBORw0KGgoAAAANSUhEUgAAAz0AAAEKCAYAAAA4iDtJAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzs3Xl4VOX5//H3DQFZ/aIoIIiICIosgsomAilWxA1wBVwK\nbqWiIG1VrFZZ/SmoSHHFCmrdsCoiVlGsNCyRTUHAslQ2RTZBRAIBEsL9+2MmYxImySSTZJLh87qu\nczHnzLPcZ47C3PM85znm7oiIiIiIiMSrcrEOQEREREREpDgp6RERERERkbimpEdEREREROKakh4R\nEREREYlrSnpERERERCSuKekREREREZG4pqRHRERERETimpIeKXZm9oqZjYp1HFmZ2Tdm1jnWcQiY\n2Rlm9rWZ7TGzuyKs81czq1LcsYmIiEh8SIh1AHJU8OBWarh781jHUJaY2UbgFnefVQzN3wd87u6t\n8uj/KqB+lkMXAIuAmcUQj4iIiMQZJT1SUizWAUhUnOK7hg2AL/Ls3H1q5mszqwSsdPfPiikeERER\niTOa3hanzKyRmf1kZq2D+3XNbEekU7rM7H4zWxuccvRfM+uV5b2NZvZnM1tmZrvNbIqZHZPl/dZm\ntiRYdwpQKZc+7jWzd3Mcm2Bm44sgxjzPP3gOXYOvh5rZD8F2VmceD9NffTObamY/mtlOM3s6y3tN\nzSzJzH4OTp27IsfndY+ZLTezFDObZGa1zWyGmf1iZp+ZWY0c5e8PntMuM5uc+fnm1U+WumGvTfAz\neC8Y/3ozGxRJXTN7DTgF+DAY/z0F/Nzy+mxmAYnAM8F2Ts/jkgPg7geU8IiIiEiBuLu2ON2A24D/\nApWBT4GxBah7DVAn+Po6YC9QO7i/AVgA1AGOA1YCA4LvVQS+A+4GygNXA2nAyDB91Am2+3/B/QRg\nO9A6ihjrRHL+wXPoCpwBfJ+lnVOA08L0VR5YBjwZbO8YoGPwvQrAWuD+4Dn8BtgDNA6+v5HASMaJ\nQN3gOS4Bzg628znwcJa+NgLLgXrBz3ceMCrYdrh+muQ4ryOuDYFRmq+AvwbrNgTWAd3yq5v188pS\nNtLPLbfPJmvM/yEwdS7m/89o06ZNmzZt2uJz00hPHHP3lwh84VwE1AYeLEDdd919W/D1P4FvgbZZ\nikxw923u/jPwIZB5P0Z7IMHd/+buGe7+HrA4lz62AXOBa4OHugM73H1pUcQY4fkfIpB4NDOzCu7+\nvbuvD1OuLXAScK+773f3g+6enOWcq7r7Y+5+yN3/A/wLuD4zFOBpd9/h7luC5zzf3Ze5+0HgfaB1\n1lMDnnH3zcHP9xGgbx799M0Ra7hr0xY4wd1HB+tuAF4C+kRQN5yMCD+3SGMOO3XOzE42s15m9s/g\n/h8zR5pEREREIqWkJ/69BDQj8KU7HcDMbghOU0oxs4/CVTKz35nZ0uCUpJ+B5sAJWYpsy/J6P1At\n+LousDlHc9+R+/0grwI3Bl/fCLwWZYw18zv/rNx9HTAEGA5sN7O3zOykMN3VB75z98Nh3qsLbMpx\n7Lvg8Uzbs7zen2P/AL9+fpmytvd9sK3c+qmX41jWa5MabPsUoG7mZxX8vP4C1Mqj7n6gKmG4+1oi\n+9wijTm3hS6aAEsJjD4BTAHCXQMRERGRXCnpiWNmVg0YT+CL/wgzOw7A3d9w9+rB7bIw9RoALwJ3\nAse7+3HAN0R2I/tWjvxC24Dcv9R+ALQ0s+bAZcAbRRVjbuefk7u/5e6dssQ5JkyxTcApZlY+zHtb\ngPpmlvXzacCRyV+2U8jjPQgkKVlfb8mjnx/yaQsC8W9w9+OybMe6++UR1IUw1y/Czy2amPHAanG3\nAq8EDx0PfGlmV5nZXyKMXURERI5ySnri29+ARe7+e+Aj4IUI61Ul8CV2J1DOzG4mMIoSifnAITMb\nbGYVLLDUcJvcCrv7fuA94E1gobtH9GU4whjzPX8za2JmXYM3+x8kMOqSEaa/hQQSusfMrIqZVTKz\n87O8lwrcFzznROByAqMShWHAQDOrZ2bHE5iWN6WQ/WQmG4uAFDO7z8wqm1l5M2tuZudFEAsERqYa\nhQ5G/rktiDDmvJLAcwmcOwTu95pD4P6kivnELiIiIgIo6YlbZtYT6AbcETz0J+AcM8t5L8UR3H0l\ngRv25xOY7tScwM30uVYJbrh7GnAV0B/4icACA+/l0+WrwT5eyy+2SGMswPkfAzwK7CCQ1JxAYNpX\nzv4OA1cApxOYbrYpeG6Z53wFcEmwnWeAm9z9f3mdQo7XOfffJPAMmnUE7lUaHZyeV5h+PBj/5QTu\n0VkfrP8icGx+dYOvHwX+Gpwa9yci/9wijTmv5zhNAXoFE+g8l7YWERERCcfcS9UzI+UoZGb1gdUE\nVofbG+t4Ys3MNgC3evE8CDQuBKc39nf3EbGORUREREq/qEZ6glN8FprZ12a20swezaXcBDP7Nvj8\nj9bhysjRyczKAX8G3lLCIwWgh92KiIhIxBKiqezuB8zsN+6eamYJwDwzu8Dds04zuhQ43d0bm1k7\n4HkCy9jKUc7MqhK4V2QDgeWqRfIVXKDiauBcM2vu7t/EOiYREREp3YpsepuZVQFmA/2C91tkHn8B\n+I+7vx3cXw10cfft4VsSEREREREpOlEvZGBm5czsawK/2P8na8ITVI/sz+n4ATg52n5FREREREQi\nEXXS4+6H3b0VgUSmc3BJ2pxyzr/X6gkiIiIiIlIiorqnJyt3/8XMPgLOA5KyvLWZwNPsM51MmIc2\nmpkSIRGRUsDdtVCEiIjElWhXbzvBzGoEX1cGLgKW5ig2HfhdsEx7YHdu9/O4e9xtw4YNi3kMOjed\nW7yfVzyfW0mfl4iISDyKdqTnJODV4LLD5YDX3P1zMxsA4O4T3f1jM7vUzNYC+4Cbo+xTREREREQk\nYtEuWb0COCfM8Yk59u+Kph8REREREZHCinohA8lbYmJirEMoNjq3sidezwvi99zi9bxERERKUpE9\npydaZualJRYRkaOVmeFayEBEROKMRnpERERERCSuKekREREREZG4pqRHRERERETimpIeERERERGJ\na0p6REREREQkrinpERERERGRuKakR0RERERE4pqSHhERERERiWtKekREREREJK4p6RERERERkbim\npEdEREREROKakh4REREREYlrSnpERERERCSuKekREREREZG4pqRHRERERETimpIeERERERGJa0p6\nREREREQkrkWV9JhZfTP7j5n918y+MbPBYcokmtkvZrY0uP01mj5FREREREQKIiHK+unAH939azOr\nBnxlZp+5+6oc5Wa7e48o+xIRERERESmwqEZ63H2bu38dfL0XWAXUDVPUoulHRERERESksIrsnh4z\nOxVoDSzM8ZYD55vZMjP72MzOKqo+RURERERE8hPt9DYAglPb3gXuDo74ZLUEqO/uqWZ2CTANaBKu\nneHDh4deJyYmkpiYWBThiYhILpKSkkhKSop1GCIiIsXK3D26BswqAP8CZrj7+AjKbwDOdfddOY57\ntLGIiEh0zAx315RkERGJK9Gu3mbAJGBlbgmPmdUOlsPM2hJItHaFKysiIiIiIlLUop3e1hG4EVhu\nZkuDxx4ATgFw94nANcAdZnYISAX6RNmniIiIiIhIxKKe3lZUNL1NRCT2NL1NRETiUZGt3iYiIiIi\nIlIaKekREREREZG4pqRHRERERETimpIeERERERGJa0p6REREREQkrinpERERERGRuKakR0RERERE\n4pqSHhERERERiWtKekREREREJK4p6RERERERkbimpEdEREREROKakh4REREREYlrSnpERERERCSu\nKekREREREZG4pqRHRERERETimpIeERERERGJa0p6REREREQkrinpEZFC+9//YP/+WEchIiIikreo\nkh4zq29m/zGz/5rZN2Y2OJdyE8zsWzNbZmato+lTRGJv06ZNTJkyhT59PuaKK57hwIEDsQ5JRERE\nJFcJUdZPB/7o7l+bWTXgKzP7zN1XZRYws0uB0929sZm1A54H2kfZr4iUkIyMDFasWMG8efOYOXMm\nycnJpKamkp5+iMOHv+e3v/0NlSpVinWYIiIiIrmKKulx923AtuDrvWa2CqgLrMpSrAfwarDMQjOr\nYWa13X17NH2LSPHYu3cvCxcuZM6cOXz66acsW7aMChUqkJ6enm1Ep1Klthw+XJu5c8uxfz9UrhzD\noEVERETyEO1IT4iZnQq0BhbmeKsesCnL/g/AyYCSHpFSYPPmzSQnJzNr1iw+//xzvvvuOypXrkxq\naiqHDh0COGL6WuXKlenW7RU+/rgcFSrAp59Cr16xiF5EREQkf0WS9ASntr0L3O3ue8MVybHvRdGv\niBTeyy+/zP3338+ePXuoUKECKSkpoffS09NzrVelShXuuedeXnqpKenpkJ4OL7+spEdERERKr6iT\nHjOrALwHvO7u08IU2QzUz7J/cvDYEYYPHx56nZiYSGJiYrThiUguLrroIgYOHMiBAwciXoigcuXK\nXHLJJVx11TCeeOLX4zNnoiluZVRSUhJJSUmxDkNERKRYmXvhB13MzAjcr/OTu/8xlzKXAne5+6Vm\n1h4Y7+5HLGRgZh5NLCJScGPHjmXkyJHs27cv37IVKlTgrLPOYuHChTz88DE89VRglAegenX4xz80\n2hMPzAx3zzk6LyIiUqZFm/RcAMwBlvPrlLUHgFMA3H1isNwzQHdgH3Czuy8J05aSHpESdujQIc44\n4wzWr1+fb9natWuzfPlyTjyxFiefDFu2ZH+/Rw/44INiClRKjJIeERGJR1ElPUVJSY9IbHzxxRf8\n9re/ZX8eTxmtWrUqCxcupFmzZixfDh06QGpq9jKVKsGuXZriVtYp6RERkXgU1cNJRaRs2759Ow89\n9BD79++nfPnyYctUrlyZd955h2bNmgHwxhu/TmvLKnMVNxEREZHSRkmPyFEoIyODCRMmUKdOHWbN\nmkWfPn2oUqXKEeWqVKnCI488wiWXXAKAO7z+evikJyUlsIqbiIiISGmjpEfkKPPVV19RpUoV7r77\nbipVqsRXX33FW2+9xbhx46hatWqoXJUqVejduzdDhgwJHVuxAnbvzr3tzFXcREREREoTJT0iR4nd\nu3dz/fXXc95555GWlsYzzzzD3r17OeeccwC45ZZbaNSoEWZGxYoVadWqFRMnTiSwSGNAblPbMmmK\nm4iIiJRGWshAJM65O2+88QY33XQTAL/97W954403qFWr1hFlly9fznnnnUedOnVYvnw5NWrUyNIO\nYVdty0mruJVtWshARETikZIekTi2evVqfvvb37J5c+B5wLNnz6Zz58551vl//+//cd1113H66adn\nO75sGZx//pGrtuWkVdzKNiU9IiISjzS9TSQO7d+/n7vvvpumTZuyefNmhg8fzsGDB/NNeAAeeOCB\nIxIegDffzHtqWyZNcRMREZHSRiM9InHmo48+4vLLLwegZcuWTJ8+nQYNGkTVZqRT2zJpilvZpZEe\nERGJRwmxDkBEisamTZvo1asXS5YsAeCDDz6gR48eRdL21q3QsCGcdFL24199Ffjz3HOzHz90CA4f\nhnIaSxYREZFSQCM9ImVceno6jz/+OA8++CAAd955J2PGjMm2/HRxMYOmTWHlymLvSkqIRnpERCQe\naaRHpAybN28enTp1AqB27drMmjWLs846K8ZRiYiIiJQumnwiUgbt2LGDSy65JJTwvPLKK2zdulUJ\nj4iIiEgYSnpEypDDhw/zwgsvUKtWLT755BOuvfZadu3aRb9+/bI9RFREREREfqXpbSJlxLJly+jU\nqRMpKSmUK1eO+fPn07Zt21iHJSIiIlLqaaRHpJRLSUnh5ptvplWrVqSkpDBu3DjS0tKU8IiIiIhE\nSCM9IqWUu/POO+/Qu3dvADp16sTbb7/NSTnXjRYRERGRPCnpESmF1q5dy8UXX8z69esB+Pe//82F\nF14Y46hEREREyiZNbxMpRQ4cOMDQoUNp3Lgx69ev5y9/+QsHDhxQwiMiIiIShahHesxsMnAZ8KO7\ntwjzfiLwAbA+eOg9dx8dbb8i8eazzz6jW7duADRu3JhPPvmE0047LcZRiYiIiJR9RTHS8zLQPZ8y\ns929dXBTwiOSxZYtW+jQoUMo4Xn33XdZs2aNEh4RERGRIhJ10uPuc4Gf8ymmB4iI5HDo0CGeeOIJ\n6tWrx4IFC7jttttISUnh6quv1jN3RERERIpQSSxk4MD5ZrYM2Azc4+4rS6BfkVJrwYIFdOjQAYAa\nNWowZ84cWrQ4YnaoiIiIiBSBkljIYAlQ393PBp4GppVAnyKl0q5du7jyyitDCc9LL73ETz/9pIRH\nREREpBgV+0iPu6dkeT3DzJ4zs+PdfVfOssOHDw+9TkxMJDExsbjDEykR7s7LL7/MrbfeCsAVV1zB\nyy+/TM2aNWMcmRztkpKSSEpKinUYIiIixcrcPfpGzE4FPsxl9bbaBFZ2czNrC/zT3U8NU86LIhaR\n0ua///0vXbp04aeffgIgOTmZ888/P8ZRFQ0zaNoUVmrCatwwM9xdN5WJiEhciXp6m5m9BXwBnGFm\nm8zsFjMbYGYDgkWuAVaY2dfAeKBPtH2KlAX79u3jD3/4A82bN+enn37i0UcfJT09PW4SHhEREZGy\nokhGeoqCRnokXrg706ZN46qrrgKgTZs2TJ06lZNPPjnGkRU9jfTEH430iIhIPCqJ1dtEjhobNmzg\nsssuY9WqVQDMmDGD7t3ze4yViIiIiBSnkli9TSTupaWl8fDDD3PaaaexatUq/vznP5OamqqER0RE\nRKQU0EiPSJT+85//0LVrVwBOOeUU/v3vf9O4ceMYRyUiIiIimTTSI1JI27dv5ze/+U0o4XnrrbfY\nuHGjEh4RERGRUkZJj0gBZWRkMGHCBOrUqUNSUhI33ngjv/zyC3369MFM93+LiIiIlDaa3iZSAF9+\n+SUdO3YkLS2NypUrk5ycTOvWrWMdloiIiIjkQSM9IhHYvXs3ffr0oU2bNqSlpfHss8+yd+9eJTwi\nIiIiZYBGekTy4O68/vrr/O53vwPgoosu4vXXX6dWrVoxjkxE4pWZ3QIYcDkw3N2XxTikfJW1mMta\nvCISPSU9IrlYvXo1F154IVu2bAFg9uzZdO7cOcZRiUg8M7PuwGJ3X2FmO4F/AGfHOKw8lbWYy1q8\n0Yjn5C6ez02Kh6a35eLZZ59l4cKFJdLX4sWL2blzZ4n0VRqlpaXxr3/9K9ZhhKSmpjJ48GCaNm3K\nli1bGDlyJGlpaUp4RKQkNAEGBF+vBU6NXSgRK2sxl7V4CyVLcjcJeIVAchcX4vncpPgc9SM9GRkZ\nDBw4kLS0tGzHN2zYwKuvvsqiRYuKtf/FixezYsUK2rRpA8DatWuZNGkSVapUYffu3ZQrV45HHnmE\nihUrhq2fX/m0tDSeeuopMjIyMDN++uknhg8fTrVq1XKN6ccff2TChAlkZGTw9ddf0759ex588EES\nEnL/zyUtLY3777+fWrVqkZGRwc6dO3n88cdDdW644Qauu+46WrZsyQknnECFChUASEhIoGLFivz8\n88/885//5LrrrivU51hU/vWvf3HFFVcA0LJlS6ZPn06DBg1iGpOIlC5mdhJwo7s/HmH5O4Ev3T3s\nL2k52nsOyPwLuiMwowhCziu2iM4l5zmU9phLU7wx1ITAKMhdxF9yF8/nJsXF3UvFFgildNizZ4/3\n7dvX9+zZU6z97N+/36+++upsx1q3bu1vvfVWaP/cc8/1W2+9Ndc28iv/0EMP+datW0P7X375pffv\n3z/X9g4fPuy33367p6amhmI866yzfNCgQXmey9ChQ/2uu+4K7Q8ZMsTvvffe0H6LFi3czI7Y7rvv\nvlCZG264wb/77rs8+yku3333nbdu3doBB/yDDz6ISRxlDbg3bRrrKKQoBf8ujvm/CaV5AxoAw8Ic\nLw9MBF7OsSUBiwrSHlAD+DdwYkmeS6TnUFpiLgvxZunvdOBR4CHgSeBxoGI0dYCKwFDgAeAvwBNA\nteB7CUCN4OvfA1OK+fxqAaOD8c4AhgEJeZSvCIwD7gceBJ7KWh54A+gJNASqA5WCW0JJn5u2+Nhi\nHkAokFKU9JSUxx57zN95551sx1q3bu1//OMfQ/vXXXedn3nmmbm2kV/58847z/fu3ZutTufOnXNt\n73//+5+fddZZvnTp0tCxRx991CtWrOgHDx4MW+fAgQN+7LHH+ty5c0PHkpOTvWbNmqH9vn37+ty5\nc33+/Pm+cOFCT05O9muvvTZbm4sWLfLbbrst19iKQ1pamo8aNSqU7Nx1112+b9++Eo2hLFPSE3+U\n9ET071XYpCeXstWBN4HqkbYX/CL/BHBSaTiXcOdQXDEDxwBnRRNzScZbwHNbAvTJsv8l8FI0dYCR\nQJ0s++cCL+doI6rkLsJrYsCLQOXgfiXgv8CEPOo8BjydZf8pYGyW/eXA4TDbmKI6N21H16Z7emLo\njTfe4Morr8x2bMmSJYwbNy60v3bt2tDUt3DyK1+7dm26devGunXrAEhKSqJr1665tlexYkV+/PFH\nvv3229CxqlWrkp6ezp49e8LWWbZsGSkpKTRq1Ch0rEGDBuzatYulS5eyd+9ebr31Vi644ALat29P\n27ZtmTdvHmPGjMk2ba9NmzbMnTuX1NTUXOMrSnPnzqVixYo89NBD1KlTh5UrV/L0009TpUqVEulf\nROKfu6e4+/XunlKAan8AnnD3rWZ2Q3HFFqkIz6GoYq4DXBtF/ZKOt6DaZnm9jsD0umjqXAKEztPd\nvwJOy9w3s/LAX4Gb3H1HYQImsmtyejCuM4JxHABeAwaY2RHz883sGOAO4O0sh98Bbsmy/w3QGTgf\naA9cALxLYNSrqM5NjiJHbdIzadIknnjiCW6//fbQsZYtW/L++++XSP+rV6/m+OOPp3z58rmWmT9/\nPrt27eLJJ5+MqM1w5ceOHcu6deto2bIlgwcPZsGCBQwbNizXNho0aMCOHTu49tpf/35btGgRzZs3\n54QTTghbZ9OmTUAgOcpUvXp1ADZv3ky1atW48MILQ+8lJSVRt25dGjZseERbbdu2ZdasWRGdb2Ht\n2LGD7t27hxYmePXVV9myZQtNmzYt1n5FRPJjZtcS+AV8hZntAG6McUj5KmsxF1W8ZpYQwRb6R97d\nz3H3P2Vp4nRgcV59RFBnOzDTzBoFY0oEsv4jWlLJXRqB6W2NsxzbB1QAjg1T/mwCI3Lrshz7Djje\nzFqbWTVgkrvPc/cF7r6IQNIz1N0zb8IuVT8OSOl3VC5ksHz5cqpWrUrnzp1p3749f//73wHo3Lkz\nGzdujLidQ4cOMXDgQNLT0/Mt26dPHy6++OLQ/tKlS2nSpEnYshs2bODjjz9mypQpvPLKK5x44ol5\ntp1X+TPPPJM77riD2bNn88wzz3DOOefQt2/fiG/OX79+Pe+99x6fffZZrmX2798PQKVKlULHjjnm\nGABSUrL/0Jaens64ceOYPn162LYaNWrE119/zeWXXx5RfAVx+PBhXnzxRe644w4Arr32WiZOnMhx\nxx1X5H1J7p599lnOO+882rVrV6TtLl68mIYNG+aanJcmaWlpzJw5s1j+O5eSZ2a3AscBZ7j77cFj\nywlMpyrQL2nu/g6BX7xjxswqA4OAA0Ab4AUCv7R3AB5295VZy5eSmO8kMCKR4e7Dg8e6AH9x9+5Z\nyxZFvGGSi9wkAUdMrzCzDsDxwJ8L0Ge4OvcF41huZpOALe4+Ilg+M7kbbmYQmBr3RqT9FYS7fwfk\n/LLSFvjG3cMtT1s/+Oe+LMcyvzDUc/elwOeZbwQ/7y3uviG4X2LnJvHjqEx6UlJSuOaaaxg5cmS2\n6WVXXXVVRAlMpoSEBF588cVCxfDjjz/m+mW7YcOG3HnnnfTr148WLVpwww03MHr06Fzbyqt8//79\n6d+/P8OGDWPatGkMHDiQHj16sGxZ/svZp6WlcfPNN/P3v/+djh1zH4GvUaPGEcf27t0LZE+EAF5/\n/fU8R1Rq1qzJ6tWr842toL7++ms6derE3r17KV++PPPnz89z2qBEryRXRsy5CmJ+qwmGU9BVC+Nl\nxUKJjpmdTeCL2xxgAZA5fWAOJbiilJklEFiVrEIExae4+6d5vD+IwL0W+81sGoHlnW8BHiawaMDK\nPOpGywpcwewC4H/BbSwwPPhWT2B9kUWW3ZfAeRGUy/bLn5k1BC4F+gD9I5mWlU+d1cDzQBcCK5kt\nMbO33P27IkxGC3NNTgOuBi7KpUjl4J8Hshw7GPyzeo62KgB/cvcemcdKQ6ItZVCsbyrK3CjhhQwO\nHz7s9erV89mzZ4eOTZo0yffv3+/u7i+88EK294ramDFj/IEHHsi33N133+3lypWLeFWzzPIbN270\n5ORkv/HGG7O9/8MPP/gJJ5zgy5cvz7etAQMG+LRp0/Itt2TJEjcz//HHH0PHNmzY4GbmX375Zbay\nHTt29MmTJ+fa1sSJE/0Pf/hDvn1G6pdffvF+/fqFFip46qmn/NChQ0XW/tGuoAsZFMfKiOFWQcxv\nNcGcCrNqYVlfsTA3aCGDSP69yrp6WEcCPyCOJPsN5l2BiwvaXizPJbhvQMcs+6uAS4ojZgI3u/+d\n7KuuvQMs5cjV2LrkEXOn4J+vEnhIZebxr4Bri+sz5tdVxPLayudStxqwARhdgP6OqEPg+TRdg697\nAVuAZSV9TXK0URGYDdyQRz+XE1iUIOtKdDWDx67MUfZmsixeoE1bYbeoR3rMbDJwGfCju7fIpcwE\nAjfbpRL4lWJptP1Ga9WqVezcuZMLLrggdGz37t0APP3007z66qvZFggIJz09nTvvvLNQ09tq1arF\nhg0bspVJTk6mV69eTJ48OfSsmAoVKuDu/PLLL0e0mVf5PXv2sG3bNurUqZOtTr169ejcuXNo+llu\nxowZQ8+ePbnkkksAePvtt7nsssvCPt+nZcuW1KxZk/Xr14em1q1cuZLq1avTosWv/0mkpKSwcOFC\n/vKXv+Ta765du/KdzhcJd+ef//wnffr0AQJTF99+++0jPg8pWdWrV+fNN98s0jb/9re/ha4zwMGD\nB3n++ef1/K0cAAAgAElEQVT56KOPQseuvfZaevTowdixY8O2sXbtWpKTk1mzZg2tWrWiUqVK3HTT\nTQwbNownnnjiiOdkRdJH8+bNee6550hISKBcuXIcOnSI8ePHM2rUqFCdu+++m1GjRoWm2ErZ4+7J\nFphfcwtwfZa3TiWwgliZ4u4OJAOYWT2gEYFRq+Lo6wC/jowR7LMBge8JIwrQzlwzqw5cA7QItvN/\nQEsC08uKXLTT29x9r5l9APzFzF509+/zayhHnb8D9QBz91nB96eZ2WLgazNr4e4rIj+jUB9FcU0m\nAOPc/YM8ymwO/vl/QObIVeYIT87P4lZgUoR9i+SqKKa3vQw8TS5PwzWzS4HT3b2xmbUjMAzbvgj6\njcrhw4c59thjKVcusJbD1KlT6dmzJ5UqVWLQoEEsWbKEwN/9uatQoUKhp7c1bNjwiPtaqlWrRqVK\nlUJf+t2dWbNm0bFjx1DyMGPGDKZOncrEiROpWrVqnuXr16/PI488wvfff88pp5wCwNatW6lUqVLo\nfqLp06czffp0XnzxxdBn8corr7BlyxZatWrFJ598Euq3d+/eR8RQrlw5ypcvT58+fXjnnXdC92m8\n9dZbDBgwINuXxY0bN5KRkUHlypXJzfbt2zn77LML9ZlmWrt2Ld26dQsllZ9//nmeK9ZJ2fbGG2+w\ndOmvv6Pkt5pg69atj2gj66qFrVq1ArKvWpjzPqH8+mjcuHFoxcJMY8eODbti4U033URqaqpWDSzb\nmgInAPOyHKsR/AIJgJkNAFa5e7EkEMEpQM9SBNPbzKycux8GLgS+cvd9weMXuPu83OrFWEcC93xk\nTmfrBPzPi29VrwJNbzOzjsA04BZ3/zD4XjqBkbX/C1cxnzrHEriHaVvWOu6+2czm8OtUsRJlZkOB\nD9x9RnC/N/CRu+/NUXQ58BOBleYyr9FZBD6vULIWTGbbEXj2j0hUok56gr+wnJpHkR4Ehpxx94Vm\nVsPMarv79mj7jkbz5s3p168fDz30EHXr1qVp06bZvsAUtw4dOnDbbbdlO3b22WczadIkZs+ezccf\nf8yGDRs4//zzeeSRR0Jl1qxZw2effcbBgwdp1apVnuVr1KjBm2++yYgRIzjppJMwMzIyMnj++edD\n7a1atYqpU6dyzz33cOaZZ7J69WoGDBhAeno6Tz/9dKhcp06dwsaQmcA89thjDBkyhNGjR5ORkUGN\nGjWOuA+pSpUq1KlTh1NPPTXXz+WLL77g3nvvLfgHChw4cICHH36Yxx8PPKT7wQcf5KGHHsp3VEuK\nz7PPPsu2bdsoX748w4cPB2D27Nk8+uijoYQ6GuFWQcxvNcFwSU/mqoVZ5bVqYSR9FHTFQi1qUKaV\nA/YEEwXM7Crgg+DrSgR+Oe8H/CnXFqLk7ukEHtIYFTO7BniGwBfqXgTukyG4mtb5ZE/sSpPKQNbn\nKvQhspGYQgl+iV9SgCp7Cdy/sgMgODrYFUjOHJExs0uAq4ABwf+W8qxjZpuAB83slMyRIjM7CTjg\n7v8rgtMsEDPrD9QlMNKUuXjEJe7+dvD9rOeXYWZTCCyFvTBYti8w0X9dnQ0CI6blgf3FfwYS70pi\nIYN6wKYs+z8AJxNYZjGmMr8cx0LFihVp06YNy5Ytyzay0a1bN7p165ZrvSFDhjBkyJCIy59xxhlM\nmpT7qPDQoUPp1q1baIremWeeycGDef9AlDMGCHz5y2+KTqNGjdiyZUuu7+/cuZOEhATq1q2bZzvh\nzJw5MzR9sEmTJsyYMYPTTjstn1pSnObNm0eTJk1o0qQJ9913Xyjp+eCDD7Jdm6JeBbEgqwnmJr9V\nC8vKioVSMtz9GzN71cxGEbinYpW7rwu+dwB42szOoRA3hMfAD8AcM/sz8CQwyMzuAKoQmLZUWn0M\nXGdmg4P7PQkkmqWCuy8LrvLXJTgDpiHwBfBglmJnELjx/xhgf3513H23mV0PDDOzrQTuXS1P4Pk3\nJcrMziSwyEUFAgthZJqb5XW28wPuB8ab2V8JxL2bwHN3skolMJq1sVgCl6NKSa3elvMv+rznjZUS\nwWUQi82IESN47LHH8kxKSsKCBQsYMGBATGMAjrjfIRKbN2/mmmuuYcGCBQC89957XHnllcV+7SR/\n7s5FF11Ev3796NmzZ+j4nDlzGDp0aGi/qFdBLMhqguFEsmphWVixUEqWuxduiLqUcfcFQNYlBZNL\nOIQ0CverfmV37wtgZm0IjJB8XJSBRcvdZwIz83h/PDC+gHXWELjnpTjle03cfTWBZCavMtnOLzht\n8vbca0Dwx4OC/xIqEkZJJD2b+XU9dgiM8mwOVzDzl2CAxMREEhMTizOufOV3T0+0GjduTIMGDViw\nYAHt28fmNqdvvvmGhg0bhu7niZV169axY8eOiO+9OXToEE899RT33XcfALfffjvjxo0Lu9CCxEan\nTp1ISUnh3XffZcWKwBTtX375hWXLlhXZ/9sHDx48YknpevXqhfrKvN8tc/Ql8962vAwePJg//elP\n2RK1nArSx6RJk7j11ty/k1SsWJGMjIx84youSUlJJCUlxax/kazcfSuBZacjZmZnAN+YWQtgLYEv\n1r/Pek+VFF5hrolIaVQSSc90AmvHTzGz9sDu3O7nyZr0xNJzzz3HokWLcHcyMjKK9Sb4hx9+mBEj\nRnDaaadRq1atYusnN82bN6d58+Yl3m9WaWlpPP/88/ztb3+LqPz8+fM5//zzATjuuOOYM2dOzM9B\nwktOTqZu3bqh6Wxz586lSZMm2VboK+pVECNdTTCcSFctLI0rFhZWzh+YRoyIeNEsKbgyMcuhDNpC\n4AGqXQmMejzs7p/nXUVEjjZFsWT1WwQeinVC8Ka6YQRXj3H3ie7+sZldamZrCTzA7eZo+yxuAwcO\nZODAgSXW37Bhw0qsr9KoYsWKPPHEE/mW27VrF7fccgsffBBYBfOll17i5ptvjvkoleRu//79HHvs\nsaH9KVOmHPEjQlGvghjJaoLRrlpY2lYslBKXTuGmYOU277aw7RWFwvZdamJ29xSy30eSbx0ROfpY\ncU/hipSZeWmJRUoXd2fy5Mmh1e569OjB5MmTqVmzZowjEzNo2hRW5vJ89oMHD9K/f386dOgABFbU\ne/XVV7nqqquKpP+0tDSaNWvGt99+m+34vn37GDJkCA0aNCAjI4OdO3fy5JNPhhKSMWPGMGbMGL74\n4ovQqoVnn332EaNNnTp1Yvbs2UDgnrPx48ezatUqKleunG8fEJi22alTJ+bNm5frwhrt2rXj/fff\nL9QCHsXBzHB33RRXhMxsIHAnsBj4R+ZzVUREpOQo6ZFS7ZtvvqFLly7s2rULCCxpnfkFWmIvv6Rn\n9+7doZv+Fy9ezKWXXsqmTZsiWlAgUtdffz1Dhw4t8GjJ0qVLSUhIyHfKW3HauXMnPXv2JDm5pO8V\nz52SHhERiUeaFySl0t69exkwYAAtWrRg165dPPbYY6SnpyvhKUPWrFnDiSeeyOrVqzl06BBDhgzh\nxRdfLNKEBwL3oEyYUPCVdBcsWECzZs2KNJaCKsyKhSIiIlJwJbVktUhE3J3333+fq6++Ggg8uHHq\n1Kmh1bKk7Khbty5/+MMfmDVrFpMmTWLkyJHZHthZVAqzCmJpWLWwoCsWioiISOFpepuUGhs2bODS\nSy8NPbPkk08+ybZSl5Q++U1vK0kjRozgjjvuiMkqiAWVlpbGAw88wOjRo4t85Ctamt4mIiLxSEmP\nxNzBgwcZPXo0o0ePBuCee+5h1KhRpe7LoBypNCU9UjSU9IiISDzS9DaJqVmzZoWmPDVo0IDPPvuM\nxo0bxzgqEREREYknWshAYmLbtm0kJiaGEp4pU6awYcMGJTwiIiIiUuSU9EiJysjIYPz48Zx00knM\nnj2bm266iV9++YXevXtjphk1IiIiIlL0NL1NSsyXX35Jhw4dOHToEFWqVCE5OZlWrVrFOiwRERER\niXMa6ZFit3v3bnr37k2bNm04dOgQzz33HCkpKUp4RERERKREaKRHio2789prr9GvXz8ALr74Yl57\n7TVOPPHEGEcmIiIiIkcTJT1SLFavXk3Xrl3ZunUrAHPmzKFTp04xjkpEREREjkaa3iZFKjU1lcGD\nB9O0aVO2bt3KyJEjSUtLU8IjIiIiIjGjkR4pMh9++CE9evQA4Oyzz2b69OmccsopMY5KRERERI52\nSnokat9//z09evRg2bJlAEyfPp0rrrgixlGJiIiIiARoepsUWnp6OqNGjaJBgwYsW7aMQYMGsW/f\nPiU8IiIiIlKqaKRHCmXu3Ll07twZgDp16jBr1iyaNm0a46hERERERI6kkR4pkB07dtC9e/dQwvOP\nf/yDLVu2KOERERERkVJLSY9E5PDhwzz//PPUqlWLTz/9lOuuu46ff/6Zm266CTOLdXgiIiIiIrmK\nenqbmXUHxgPlgZfcfUyO9xOBD4D1wUPvufvoaPuVkrN06VI6derEvn37SEhIYP78+Zx33nmxDktE\nREREJCJRjfSYWXngGaA7cBbQ18zCzXOa7e6tg5sSnjJiz5499OvXj3POOYd9+/Yxfvx4Dhw4oIRH\nRERERMqUaEd62gJr3X0jgJlNAXoCq3KU0/ynMsTdefvtt+nbty8AXbp0YcqUKdSpUyfGkYmIiIiI\nFFy0SU89YFOW/R+AdjnKOHC+mS0DNgP3uPvKKPuVYvLtt9/SrVs3Nm7cCMDnn39O165dYxuUiIiI\niEgUol3IwCMoswSo7+5nA08D06LsU4rBgQMHuPfee2nSpAkbN27kwQcf5MCBA0p4RERERKTMi3ak\nZzNQP8t+fQKjPSHunpLl9Qwze87Mjnf3XTkbGz58eOh1YmIiiYmJUYYnkfj000/p3r07AGeccQYz\nZsygYcOGMY5KREpCUlISSUlJsQ5DRESkWJl7JIM1uVQ2SwDWABcCW4BFQF93X5WlTG3gR3d3M2sL\n/NPdTw3TlkcTixTc5s2bufrqq1m4cCEAU6dOpVevXlqCWiJmBk2bwkpNWI0bZoa76y8BERGJK1FN\nb3P3Q8BdwKfASuBtd19lZgPMbECw2DXACjP7msDS1n2i6VOid+jQIcaMGcPJJ5/MwoULuf3220lJ\nSeHKK69UwiMiIiIicSeqkZ6ipJGekjF//nzOP/98AI4//nhmz55N8+bNYxyVlFUa6Yk/GukREZF4\nFO1CBlJG/PTTT/Ts2TOU8EyaNImdO3cq4RERERGRuBftQgZSyh0+fJiXX36Z2267DYAePXowefJk\natasGePIRERERERKhpKeOLZixQq6dOnCzz//DASmtrVv3z7GUYmIiIiIlCxNb4tDe/fu5fe//z0t\nW7bk559/ZsyYMaSnpyvhEREREZGjkkZ64oi7M3XqVK655hoA2rdvz7vvvku9evViHJmIiIiISOwo\n6YkT69ev59JLL2XNmjVA4IGj3bp1i3FUIiIiIiKxp+ltZdzBgwd58MEHadSoEWvWrOGee+5h//79\nSnhERERERII00lOGzZo1iwsvvBCAU089lZkzZ9K4ceMYRyUiIiIiUrpopKcM2rZtG4mJiaGEZ8qU\nKaxfv14Jj4iIiIhIGEp6ypCMjAzGjx/PSSedxOzZs/nd737HL7/8Qu/evTHTA9RFRERERMLR9LYy\nYvHixXTo0IGMjAyqVq3KvHnzaNWqVazDEhEREREp9TTSU8rt3r2b6667jrZt25KRkcHzzz/Pnj17\nlPCIiIiIiERIIz2llLvz2muv0a9fPwAuvvhiXnvtNU488cQYRyYiIiIiUrYo6SmFVq1aRdeuXdm2\nbRsAc+bMoVOnTjGOSkRERESkbNL0tlIkNTWVQYMGcdZZZ7Ft2zZGjRpFWlqaEh4RERERkShopKeU\nmD59Oj179gSgdevWTJs2jVNOOSXGUYmIiIiIlH0a6Ymx77//nlatWoUSng8//JAlS5Yo4ZGSsXJl\nrCMQERERKXZKemIkPT2dkSNH0qBBA5YtW8bgwYPZt28fl19+eaxDk6NJixZQvz789a/w3//GOhoR\nERGRYmHuHl0DZt2B8UB54CV3HxOmzATgEiAV6O/uS8OU8WhjKSvmzJlDly5dAKhbty6ff/45Z555\nZoyjkqNSuXLgDhUqBLbjj4d+/aBvX2jWLN/qZtC0qQaM4omZ4e562rGIiMSVqEZ6zKw88AzQHTgL\n6GtmTXOUuRQ43d0bA78Hno+mz7Jsx44dXHzxxaGE5x//+Ac//PCDEh6JvfR0SE2FH36AsWOhbVuN\nAImIiEjciGqkx8w6AMPcvXtw/34Ad38sS5kXgP+4+9vB/dVAF3ffnqOtuBrpmfPRR8ycMIGEgwdJ\nP+YYDjVqxNjnA/le7969eeGFF6hRo0aMo5SjXuZIT27yGQHSSE/80UiPiIjEo2hXb6sHbMqy/wPQ\nLoIyJwPbiVNzPvqIT+++m0fWrQsd6w38X/nyfL5wIeeee27sghMpiPT0X0eBxo6Fp57KngCR/xQ4\nERERkViLNumJdGgm56+G8TOkE8bMCROyJTwAbwN/zcjg3PPOi01QItHKmQA9+iiQQYVd22BnApxw\nQqwjFBEREQkr2qRnM1A/y359AiM5eZU5OXjsCMOHDw+9TkxMJDExMcrwYiPh4MHwx48/Hi64oISj\nEcnD9OkFr1OxIpQvD/Xq8VGH1znuigvghFOLPDQpGUlJSSQlJcU6DBERkWIV7T09CcAa4EJgC7AI\n6Ovuq7KUuRS4y90vNbP2wHh3bx+mrbi5p+evF1/M6Jkzjzj+0MUXM+qTT2IQkUgu8runJ1NmolO7\nNvzud4GpbVqAIy7pnh4REYlHUY30uPshM7sL+JTAktWT3H2VmQ0Ivj/R3T82s0vNbC2wD7g56qhL\nuW6DB/PgunXZprg90KgR3QcNimFUIgWkREdERETiRNTP6Skq8TTSA4HFDD57+mnKHzhARqVKXDRo\nEJ0vuyzWYYlkl3OkR4nOUU8jPSIiEo+U9BTC4sWLadiwIScUw43baWlpzJw5k8svv7zI2y6tnn32\nWc477zzatcu58F90ivM6FYeYXPty5QJLUivRkSAlPSIiEo+U9BTQ4sWLWbFiBbfccgsQ+KJ6//33\nU6tWLTIyMti5cyePP/44CQm5zxzMr85rr73GMcccw3XXXVci51RSMjIyGDhwIGlpadmOb9iwgdTU\nVBYtWlRkfeW8TmvXrmXSpElUqVKF3bt3U65cOR555BEqVqyYaxv51UlLS+Opp54iIyMDM+Onn35i\n+PDhVKtWLdc2S921b9YMrrlGiY6EKOkREZG45O6lYguEUrrt37/fr7766mzHhg4d6nfddVdof8iQ\nIX7vvffm2U4kdW644Qb/7rvviiDq0m3Pnj3et29f37NnT5G1Ge46tW7d2t96663Q/rnnnuu33npr\nnu3kV+ehhx7yrVu3hva//PJL79+/f55t6tpLaRf8uzjm/yZo06ZNmzZtRbmVi3HOVab87W9/o0+f\nPqH9gwcP8vzzz9O7d+/QsWuvvZbJkyfn2kakde6++25GjRpVhNGXTtWrV+fNN9+kevXqRdZmzuuU\nKetIUqNGjUhOTs63rbzqzJgxI1vc5557LuvXr8+1LV17ERERkdhQ0lMAb7zxBldeeWVof9myZaSk\npNCoUaPQsQYNGrBr1y6WLl0ato1I67Rp04a5c+eSmppaDGcS33JeJ4AlS5Ywbty40P7atWtp06ZN\nnu3kV6d27dp069aNdcFV+pKSkujatWuu7enai4iIiMSGkp4IrV69muOPP57y5cuHjm3atAmAqlWr\nho5l/vK/eXPY568WqE7btm2ZNWtWEURfOuzfv5+xY8cyYcIEbrrpJpKTk3nyySe55pprWLlyZZH0\nEe465TR//nx27drFk08+GXG74eqMHTuWdevW0bJlSwYPHsyCBQsYNmxYrm0czddeREREJJaiek7P\n0WTp0qU0adIk27H9+/cDUKlSpdCxY445BoCUlJSw7RSkTqNGjfj666/jZiW3p59+mkGDBlG5cmV6\n9erFxIkTmTx5MiNHjmTAgAGcddZZABw6dIiBAweSnp6eb5t9+vTh4osvDu2Hu06ZNmzYwMcff8yU\nKVN45ZVXOPHEE/NtP686Z555JnfccQezZ8/mmWee4ZxzzqFv3740aNAgbFtH87UXERERiSUlPRH6\n8ccfOe6447Idq1GjxhHl9u7dC2T/YlvYOjVr1mT16tWFire0cXc6duxI5cqVAVizZg3jxo0jISGB\nX375JVvZhIQEXnzxxUL1E+46ZWrYsCF33nkn/fr1o0WLFtxwww2MHj06z/byqtO/f3/69+/PsGHD\nmDZtGgMHDqRHjx4sW7YsbFtH67UXERERiTVNb4vQwYMHj1iGul69egDZvrRn/mJ/yimnhG2nIHUq\nVqxIRkZGlJGXDmZGx44dgcBUrnXr1tG5c+ci7yfcdcqpWrVq9OzZk0cffZTvv/8+onaz1vnuu+/4\n4osvcPfQPTy9evVi8eLFbNmyhRUrVoRt42i99iIiIiKxppGeCNWqVYsNGzZkO9ayZUtq1qzJ+vXr\nQ9OeVq5cSfXq1WnRokXYdgpSZ9euXRFNwSorDh8+TLly5fj8888599xzQ/e2zJs3jwsuuCBULj09\nnTvvvLNQ09vCXafk5GR69erF5MmTueKKKwCoUKEC7n7EKFMkdfbs2cO2bduoU6dOtjr16tWjc+fO\noSlrOR3N115EREQklpT0RKhhw4ZMnz4927Hy5cvTp08f3nnnHdq1awfAW2+9xYABA0IPsJw+fTrT\np0/nxRdfpFy5chHVybR9+3bOPvvsEji74vfuu+9y1113sW3bNqZNmxa672bv3r188cUX2ZKeChUq\nFHp6W7jrVK1aNSpVqhRKItydWbNm0bFjx1CyMWPGDKZOncrEiRMpV65cvnXq16/PI488wvfffx8a\npdm6dSuVKlUKnZuuvYiIiEjpYO4e6xgAMDMvLbGEk5aWRrNmzfj222+zHd+3bx9DhgyhQYMGZGRk\nsHPnTp588snQl9gxY8YwZswYvvjiC84MPvE+vzqZ2rVrx/vvv0/dunVL5iSL0YIFCxg3bhzt2rWj\nffv2PP3003Tp0oXU1FTuvPPOXO+BKqjcrtPMmTNZunQp+/btY8OGDdSoUYNHHnmEY489FoDx48cz\nfvx4Vq1aFbrvKL86a9asYezYsZx00kmYGRkZGdx///2h93XtpSwyM9zdYh2HiIhIUVLSUwDXX389\nQ4cOLfAv8EuXLiUhISHXKW/h7Ny5k549e0b0AE3JrrDXqTjo2ktZo6RHRETikRYyKIARI0YwYcKE\nAtdbsGABzZo1K1Cd8ePHM2rUqAL3JYW/TsVB115EREQk9pT0FEDjxo1p0KABCxYsiLjON998Q8OG\nDSlXLvKPet26dezYsSO0MpgUTGGuU3HQtRcREREpHTS9rRBGjBjBHXfcQa1atYq87bS0NB544AFG\njx5dZPe5HK2K8zoVB117KQ00vU1EROKRkh4REQlR0iMiIvFI09tERERERCSuFfo5PWZ2PPA20ADY\nCFzn7rvDlNsI7AEygHR3b1vYPkVERERERAoqmpGe+4HP3L0J8HlwPxwHEt29tRIeEREREREpadEk\nPT2AV4OvXwV65VFW88NFRERERCQmokl6arv79uDr7UDtXMo58G8z+9LMbo+iPxERERERkQLL854e\nM/sMqBPmrQez7ri7m1luS691dPetZnYi8JmZrXb3uYULV0REREREpGDyTHrc/aLc3jOz7WZWx923\nmdlJwI+5tLE1+OcOM3sfaAuETXqGDx8eep2YmEhiYmJ+8YuISBSSkpJISkqKdRgiIiLFqtDP6TGz\nscBP7j7GzO4Harj7/TnKVAHKu3uKmVUFZgIj3H1mmPb0nB4RkRjTc3pERCQeRZP0HA/8EziFLEtW\nm1ld4O/ufpmZnQZMDVZJAN5w90dzaU9Jj4hIjCnpERGReFTopKeoKekREYk9JT0iIhKPolm9TURE\nREREpNRT0iMiIiIiInFNSY+IiIiIiMQ1JT0iIiIiIhLXlPSIiIiIiEhcU9IjIiIiIiJxTUmPiIiI\niIjENSU9IiIiIiIS15T0iIiIiIhIXFPSIyIiIiIicU1Jj4iIiIiIxDUlPSIiIiIiEteU9IiIiIiI\nSFxT0iMiIiIiInFNSY+IiIiIiMQ1JT0iIiIiIhLXlPSIiIiIiEhcU9IjIiIiIiJxTUmPiIiIiIjE\ntUInPWZ2rZn918wyzOycPMp1N7PVZvatmQ0tbH9lVVJSUqxDKDY6t7InXs8L4vfc4vW8RERESlI0\nIz0rgCuBObkVMLPywDNAd+AsoK+ZNY2izzInnr+w6NzKnng9L4jfc4vX8xIRESlJCYWt6O6rAcws\nr2JtgbXuvjFYdgrQE1hV2H5FREREREQKorjv6akHbMqy/0PwmIiIiIiISIkwd8/9TbPPgDph3nrA\n3T8MlvkP8Gd3XxKm/tVAd3e/Pbh/I9DO3QeFKZt7ICIiUmLcPc8hfBERkbImz+lt7n5RlO1vBupn\n2a9PYLQnXF/6R1ZERERERIpcUU1vyy1h+RJobGanmllFoDcwvYj6FBERERERyVc0S1ZfaWabgPbA\nR2Y2I3i8rpl9BODuh4C7gE+BlcDb7q5FDEREREREpMTkeU+PiIiIiIhIWVfcq7dFLNKHnZYl8fpg\nVjObbGbbzWxFrGMpSmZW38z+E/zv8BszGxzrmIqKmVUys4Vm9rWZrTSzR2MdU1Eys/JmttTMPox1\nLEXJzDaa2fLguS2KdTwiIiJlValJeojgYadlSZw/mPVlAucVb9KBP7p7MwLTNu+Ml2vm7geA37h7\nK6Al8BszuyDGYRWluwlMoY23oWsHEt29tbu3jXUwIiIiZVWpSXrcfbW7/y/WcRSh0INZ/397d++y\nYxiHcfx7EHlnIBHFYqZMLBYGyWyQMphNFv8EfwCLlIUoFu9PKaXUY5Ld4K1IyaL8DM+lLOTpuW7n\nfUpmteAAAAH8SURBVJ++n7q6z/G4uqej87zOX1V9A34OZp15VfUE+NQ6x9iq6m1VvRjWX1gYoru9\nbarxVNXXYbkSWA58bBhnNEl2AEeBS/z+UpVZ1uM7SZL0T01N6emQg1lnWJJdwD7gWdsk40myLMkL\n4B3wuKpets40kgvAOeB76yATUMCDJM+TnGkdRpKkWfXHOT1j+5thpx3p7ZjNfyPJOuA6cHbY8elC\nVX0H9ibZCNxNcqiq5hrHWpIkx4D3VTWf5FDrPBNwsKreJNkC3E/yathplSRJi/BPS88Iw05nyV8P\nZtX0SLICuAFcrapbrfNMQlV9Hq6V3w/MNY6zVAeA40mOAquADUmuVNWpxrlGUVVvht8PSW6ycGzW\n0iNJ0iJN6/G2Hs6wO5h1xiQJcBl4WVUXW+cZU5LNSTYN69XAYWC+baqlq6rzVbWzqnYDJ4BHvRSe\nJGuSrB/Wa4EjLFz4IkmSFmlqSs/vhp3Oqp4Hsya5BjwF9iR5neR060wjOQicZOFms/nh6eWWum3A\no+GbnmfA7ap62DjTJPR0rHQr8OSX/+xOVd1rnEmSpJnkcFJJkiRJXZuanR5JkiRJmgRLjyRJkqSu\nWXokSZIkdc3SI0mSJKlrlh5JkiRJXbP0SJIkSeqapUeSJElS1yw9kiRJkrpm6ZEkSZLUNUuPJEmS\npK79AAUCc0cKCtHcAAAAAElFTkSuQmCC\n",
       "text": [
        "<matplotlib.figure.Figure at 0x1061fc510>"
       ]
      }
     ],
     "prompt_number": 10
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "So, the sums of squares we are trying to explain can be expressed as the sum of the squared distance of each point from the origin, where the points (vectors) are the columns of $\\mathbf{X}$:"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "# Plot points and lines connecting points to origin\n",
      "plt.scatter(X[0], X[1])\n",
      "for point in X.T:  # iterate over columns\n",
      "    plt.plot(0, 0)\n",
      "    plt.plot([0, point[0]], [0, point[1]], 'r:')\n",
      "plt.axis('equal')"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 11,
       "text": [
        "(-5.0, 4.0, -3.0, 3.0)"
       ]
      },
      {
       "metadata": {},
       "output_type": "display_data",
       "png": "iVBORw0KGgoAAAANSUhEUgAAAXAAAAEACAYAAACqOy3+AAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3Xl4lNX1wPHvTSAkbAIqIoKgoqJYEURFrRDQEEVFEGkr\noihWflVrXSKiomURBcXgbgE3gqBWQRSrJKRKxJQKiqyyakVW2XcC2c7vj5vJO0Mmk0lmT87neeZp\n7sw773snljM35733HiMiKKWUij1xke6AUkqpqtEArpRSMUoDuFJKxSgN4EopFaM0gCulVIzSAK6U\nUjEqoABujEk0xiwwxiwxxqw0xowJVseUUkr5ZgKdB26MqSsih40xtYBc4GERyQ1K75RSSpUr4BSK\niBwu+TEBiAd2B3pOpZRSFQs4gBtj4owxS4BtwFwRWRl4t5RSSlUkGCPwYhG5AGgBdDHGJAfcK6WU\nUhWqFawTicg+Y8znQCcgx/W8MUY3W1FKqSoQEePr9UBnoZxgjGlU8nMSkAIs9tKJqHoMHz484n2I\nhT5Fa7+0T9qnmtAvfwQ6Aj8ZyDDGxGG/DN4VkS8DPKdSSik/BBTARWQ50DFIfVFKKVUJNXIlZnJy\ncqS7UEY09gmis1/aJ/9on/wXrf2qSMALeSq8gDES6msopVR1Y4xBQnkTUymlVORoAFdKqRilAVwp\npWKUBnCllIpRGsCVUipGaQBXSqkYpQFcKaVilAZwpZSKURrAlVIqRmkAV0qpGKUBXCmlYpQGcKWU\nilEawJVSKkZpAFdKqRilAVwppWKUBnCllIpRGsCVUipGaQBXSqkYpQFcKaVilAZwpZSKURrAlVIq\nRmkAV0qpGKUBXCmlYpQGcKWUilEawJVSKkZpAFdKqUrKysqiR4++9OjRl6ysrIj1w4hI1d9sTEtg\nCtAUEGCSiLx8zDESyDWUUiqaZGVl0afPQPLyngUgKWkoM2dmkJqaGtTrGGMQEePzmAADeDOgmYgs\nMcbUBxYBvUVkldsxGsCVUtVGjx59yc7uBQwseSaDlJRZzJkzI6jX8SeAB5RCEZHfRGRJyc8HgVVA\n80DOqZRSyj+1gnUiY0xroAOwIFjnVEqpqPLtt7xRt4hzkoaSl2efSkoaSlpaRkS6E1AKpfQkNn2S\nA4wWkU+OeU1TKEqp6uHgQfjlF7K2bCE9fRIAaWmDg57/hjDkwEsuUhv4FzBbRF708roMHz68tJ2c\nnExycnJA11RKqbAZMQJ694YLLgjpZXJycsjJySltjxw5MuQ3MQ2QAewSkQfLOUZH4Eqp2DVvHrRr\nB8cfH9bLhvwmJnA5MADoZoxZXPK4OsBzKqVU5GzfbkfdroFnly5hD97+CugmpojkoouBlFLVSaNG\n0KKFDeDG5wA44oJyE9PnBTSFopSKdl99BYmJcNllke5JKX9SKEGbRqiUUjErPx/i4yPdi0rTEbhS\nquYpLIQPPoD+/SEuOrPA4biJqZRSsScuDhYtgn37It2TgOgIXClVM+zaBevXw4UXRronftERuFJK\nuSxbBhHc+jUUdASulKq+Vq6EM8+E2rUj3ZNK0xG4UqpmGzMGfvwx0r0IGR2BK6WqDxHYtAlatox0\nTwKmI3ClVFQJeSmydevg9tudZfDVnI7AlVJhEbJSZEePQnExJCXZdnFx1M7trgwdgSulokZ6+qSS\n4D0QsIHctad2QB57DD76yGlXg+Dtr5rzSZVS1Ud+vvPzmDFw221BO3W0VJz3h+6FopQKvZ07eb1p\nEucnPkLeEftUlUuRHT4MnTrBd99BvXpQp07Qunlsmic3d2BIKs4Hi+bAlVKhd/QoZGSQ1bIl6S+8\nCVShFJn79q579kDjxkHvZo8effky+zqa0YMtnEKoKs77Q3cjVEpFTnExbNli99auUwcGDyYVSL3m\nmsqf64MP4Icf4LnnbDsEwdvlapZxPQu4mwkhu0awaA5cxYRYykuqErm58MgjwTnXNdfYm5WhUFgI\nkyZBcTFpaYOZm/ged9MZyChJ8wwOzXWDQFMoKuqFbPqZCr2qTukTgX79YOxYaNMm+P069lrDhtkv\nm0aNyMrKCnnFeX+EpSq9H53QAK4C0qNHX7Kze2Gnn0Ek85KqAtOnw44dcPfdgZ9r6VI477zQFFqY\nMMGmYf74x+CfO0g0B66UCq9OnaCgoGrvXbkS3nwTxo+37fbtg9cvgEOH7KwVgK5d4bjjgnv+CNAA\nrqJeWtpgcnMHkpdn21WefqZCY9UqaNbMjmhbt676eU47DXr1Clq3PKxZAwMHwrff2vY554TmOmGm\nNzFV1EtNTWXmTJs2SUmZxdor2pParFmkuxURUXkz97334Pvvq/bef/zDzucGuxQ+OTlo3WLuXDhS\nMun87LMhJyd4544SmgNXsWfpUjuCSkiIdE/CKqpu5rrPyQ7El1/C6afb0Xew3X8/3HOPDd4xSG9i\nKlWNRM3NXBG4/np4/nlo27Zy792/395AHDIkOF8A7r7+Gn79NajL6iNJN7NS1duiRZBR83LhJ/Fb\nZDtgDLzwQtVGtnXr2vcXFganL+6Dw5NOCs1IPoppAFexq3FjaNIk0r0Im7S0wZxbJ43pdAEmh3eR\nSWEhTJvmBMwzz/R/BL1kCfz3v/bnWrXs6DsYJc4OHIDf/c4u0wf718AVVwR+3hiiAVzFrtNPt3/K\n1xCpqamM/3Qao686j5SUz8Kb/y4qgoUL7VS8ytq2DbZuDU4/tmyBvXvtzw0aQHZ2UDezijWaA1ex\nr7gYZs2CG24Ifl61pjtyBBITK/ceEfj8c7v8PdiLcB54AK6+2j6qOc2Bq5qhoMAGjP37I92T0Fm5\nEh56KLzX/Pln6NatauXJPvvMjrwDtX69nabo8uKLNSJ4+yvgEbgx5m3gWmC7iPzOy+s6AlcqUAcO\n2OmTv/99eK+7dy80alTxcXl58NNPNicdTBs32i+De+4J7nljQLhG4O8A+pWoosOuXdVzJN6gQXiC\n94oVMGWK0/YneIO9UfnGG4FfXwSuu87JmbdsWSODt78CDuAi8g2wJwh9USpw48fbEVt1sWcPLF8e\n1FP6XM1Zr57/Oe9Nm5wZIJdeCi+/XLUOFRY6X7rGwFNPQdOmVTtXDROUm5jGmNbAZ5pCUREXrBWC\n0WLePMjMhGeeCcrpvK3m/PT9iaQkJ1d+c6eBA+HOO6FLl8A6NXasnVaYlhbYeaqZqNmNcMSIEaU/\nJycnkxzM/Q6UcucevPPzY3+5fZcugQdIN56V4W3qesPDw+Cvd9ml5xXZtQuOP97+PHmy31+W7nts\nP3LvbVwVF+dMAX34YTs/vIbLyckhp5L7tegIXFVPa9fCoEHwzTexOSIP0V8SPXr0pTC7Da1oy2Tu\nADJIvepTMufMqPh6mzfboPv995Uq0nDsqP+kxCF8160TLT//PDb/24SJTiNUNddZZ9m54bEaICZP\nhuHDg37atLTB7KzzFpfzDp0YTlLSUB58+P/K/z0VFzt57lNOgQULKl1hJz19Es/knU83WgAD2XZk\nHHcWJsXuf5soEnAAN8a8D8wHzjLGbDTG3BF4t5QKglheZt+/P/zlL8E734EDcOAAqampjPt0Gj92\nMpya/EPFqzmfeQZef91pV2YJvNt+J7Up4gx+rkLHlS+6ElNVf4MG2TzruedGuieRM2KEXQ4/alTF\nI1/3OpaHDtl9uitb1/Ktt+xe399/H13b4MYQTaEoBXYecagL4wbLf//rVI0JpmHD7HTEjRt9H1dQ\nYMui7dpl2/Xq+Re8Rfj2pZdKpyd+2bAh3GH/GD+2IIcG7+DREbhS0WT2bDtCLme5eKUqpo8fbyvc\ndOxYufdv2QLNm1eq23P+9S86Xt+L6xjGAtroKDsI/BmBIyIhfdhLKBUFvvtO5P33I92LKsvMzJSk\npJMEJgtMlqSkkyQzM7P8N3z8scill4rk5fl+f1aWyP33V75DU6eKzJwpmZmZ0qTJGXIK4wWKxE6h\nmSwpKTdW8ZMqEZGS2OkzvurkS1Vz1K/vVCWPNgcO2P75yE97m8Odnj7Jc5S7dastMGwM9Olj9+0u\nWVnpen8CfyKfOs77Z0yuWnGGH37g13Xr6PPvheTlnQY0Zj6X04eZBGEbK+UHzYGrmqNt2+jdP3zk\nyOBUF7r1VluyzOW888ockkA+r3IP7dhkn2jQAFq1qvjcCxd6ni89neG7jnJC3iPACOBR+tOXbWSG\nt9hEDaYBXNU8xcV2+9kA780EtUL8s8/CLbf4POSpa35P3cRHgAwgw3uQnDAB/ve/sm/+/HMev60P\nSUlDOcjHvMvxbE18ueIgu2WL83tq3x5SU+2NzhKX79jMLcwHUoEM1jODJk1Ga/47XCrKsQT6QHPg\nKtocPiwycKDIgQNVPkWl89GBKiwUuekmmTttmqSk3CgpKTfa6xUU2Pz19u2+3z96tMjtt0tmZqbn\n+ytyyikikyY57aVLRdLSSptzp02TqfGJ4fs91CD4kQPXWShKVUHQKsS/+KLdJrZTp6p35v777ajY\nfcGNiJ2SeNlltn30qF2ZetNNvueBL10Kv/wCvXvb9k8/2f91TcPcvx+uvRbefx9atAAqOTNG+S1q\nNrNSKmrt2mVv8lVwc/PYIBU07dqVBkKvPv0ULr4YTj7Z8/mtW53nxo2zpc/cHTgAf/ubnUrYpYut\nG9mvX8X9+eILmDPHCeCnnQbnn2+r4rRvDw0b2m1jTzqp9C2pqakatCNEc+CqZnvmGZsP98G1kjA7\nuxfZ2b3o02cgXbt2JClpKD7z0f5ISbGzRsrzyy92T3B3+/bZUfs339h2QoINrAUFsH27fa5hQ/vZ\nGjb0ff2dO+3oPz/fth99FAYPtnuegK1pOWgQrFvnvKdDh+BUlVeBqyjHEugDzYGraFZUVOEhKSk3\nluR4xWOOc6Xzye5+/NHmr6tq1iyRb7/1fO6NN0R69BApLvb93vx859pFRSIDBohs2eK8/tprIrfc\nUvW+iQT2u1Ei4l8OXEfgqmZzXybuNruiPIbi0p9TU1OZc/E5zBl6T+VTCGPGlF9p59NPy1YV2rTJ\n5rqLS65//fVwySXOyBlsgYVOneDwYd/XvuwyO9IG+/nvvBNuvtl5/dZb4cEHK/d53Hj7iyXgWTrK\nu4oifKAPdASuYsHy5SLdu3t9yTXj5C5ulzH09JxpsWqVyI4dwe3Ld9+JLFrk+VxenkinTiLz53s+\nf/nlIu++6/t8e/aIZGc77R9+EHnpJWekfviwSHKyyL59gfddnL9YnmSkNOU3XZVZRegIXCk/tWsH\nH3zg9SXXZkzbuu1i3pUJnnOc27aFE06o1KUqnD/eqZPdv0TEKe6bmAjz59vak+4eeqjiEfecOaUb\nSwFwwQXw4Ye2VBvY3QbnzoWGDQOb2/7BBx5/OXzOtRzBz/qaqmoqivCBPtARuFJWcbFsu+QSOafO\n8WXnTc+bJ/Lww57Hf/WVyBlniOze7Tz3v//Z/U3y88u/TlGRyHXXiezc6Tz30EMir77qtL/7TmTb\nNo+3VXpu+/z5ItOmOe1Fi0R+/DH8c+SrKXQErmqCoK6IFLGjVdf852MVFzszPcCOOl96yb9zG8MT\npj6rjj4P3AbYPbLT0yfZmR0DBnge360bPPec5xTHVq3gxBPh11+99x1sXlsEvv7aee2ss2D9eqfd\nqVOZyu+ee6249c3ll19spSCX446zfXHp2BHOPVe3jw2niiJ8oA90BK5CKCSjvfnzRY4e9f7a7Nki\nt93mtDdvFvnlF79PbfPD78i1fCYgEsfbnvnhr74SGTHC801Tp9qctS933CHSt6/T/te/RH73u4pn\npJTIzMyUHg1PkQf5U+lsmwb8Q55rd6Fz0MaNIhMn+nU+FTj8GIFrAFcxrbwpfiHjZ0As48MPRQ4f\n9vjCaUm6/DeutmR+8YVz3IIFIhde6DnF8J//FLn66rL9WLrUaX/zjb0R6doeoLhYZOZMuwS/PNu2\niUycWNqnlvxZ/k6SwDsCk6VxYlP5X9++Vf/MKiD+BHBNoahq40S28xVjMa5UQqAWLoRPPvF8riqF\neAsLIScH9u0jtXnz0vRC25T/sPDhB3lt3AR6d+9l0z8XX2yvO2YM7N1r3/+HP9gVku6WLYPOne2i\nHrALe44ehR9/dPrZu7ddiOPeD/fzJCbCokW8+tzr5OU9y0YmcQrJ1Od1mjR5ivc/mcJp06eX+5mD\nmrpSVVNRhA/0gY7AVQh5plDekQ51jndSKDt2BDY1bvFikc8+K/v8oUPOdL5ly0TuvLPcvnksZnHd\nqNyzRzIzM6VDh64SF3e8vEZ7yaWNZ/qnc2ebOnH3+ON22qJLcrLIkCFO29tIeeNG54an6+bmhg3O\n63fcIfdf1KXSf8XojcrQQ1MoqiYod9XfO++UzScH4Vr9uvSUuSe1sNfav9/OIfdyXFLSSZLAG54B\n7uhR2d+6tTRPPLE0+NXhROlMuqTykBM4vaU+unQR+fOfnfbSpWXnixcXe6ZfrrrKpldc7rtPZMoU\npz1+vCy//35JSGgk0FmgsyQkNPIdjIuLw5+6qoE0gCvlPip94w3PJeP+KioSycqq1KgzJeVGqccE\nWcOZcgZrBd4pDXA9u98gI7he/spLpcHvErrKDurLNVf2dk7y4osiKSlOOytLpG1b330dPNhz+9eR\nI0WefNJpf/SRyFNPebwlMzNTEhKcL5SEhBM9P9f8+c6WA4WFIs2aybXdb9AAHmL+BHDNgavqzT1/\nu2ePXbRSWYcPw+TJvPLcP3xPszvGIRK5ham8wWB6spS4ktx8QXw8rdnJaJ7g5JKqOAs4ytmJdXn4\nrj85J7j4YsjNhW0lBcp69ICVKz0vMnWq3bTKpUMHyM522qmpnpth3XQTPPGExynS0yeRnz+u9HM9\nmN+ViWNfdQ545hnYvdv+HB8Pq1Zx3yN3B2czLxWYiiJ8oA90BK4iqNz0yurVNh9cCe5pg9b8LPcw\nQC5sf4XIJZeUyT+7j9YNb8v78Yny3ahRIkVFpa8t5FSZxsUSF9dYOnS4XLJnzBCJj/dMebz1ll0K\n77JkiciYMU571iyRdu2c9saNIv36+f4g27bZ1E+J9XXry/X8rXQ0/Wdul75dr63w96EbVoUWmkJR\nNZnPlEdRkciaNc7BmzeLHDxY4fma1T5e6jBRvqGNvE4daVj7OJnvvq+IiD3PLbfIl1OnOgFu9myR\nxo1FOnYsPdcNydfJyN9dJPNfeMF572WXiQwf7rR37xaZMMFpr1gh0qSJkyM/fFgkNdX3zoavvCKS\nm+u0W7QQGTeutLlkyBA5o84JekMyymgAVzWa+4g5jkLfedrnn/dcal6OD05oJr05S6CPQKb3c+7b\nJ3LRRSIXX+y5yOe99+wIe+1a57n27e1yeZf8fLtYyPWFcOiQDfzr1zvH9O3rWUKtuNhz4dGtt4o8\n84zn8e5fAnl5ZT6XjqajjwZwVaO5B/BOLHSC7dat9nEs91H0iy967kFSIvWqPv7fvBs/XuTUU0W+\n/NJ5buJEu1DHZcUKkbFjRfLybBC9qo/sTEiURX//u3PMnXfa/cNdcnNF5sxx2l27ivTq5bQnTxb5\n97+99ykINNiHhwZwFbXCEQTKTaFMm+aZR163zjOgFxWJPP201+X07ufszhD5j6klW7p0sfPBi4vt\n/7qrX1+kUSOnvWyZHYW7j6BPP13W3Xxz6XnHkSqptd2m8r32mi1c7HLPPZ7L+XfsEDlypAq/ocrT\n+d/hE5YADlwNrAbWAUO9vB6WD6tix+jRoyUurnFYgoBfXxQvv2zTGy6LFnnu1Jeb6xEwMzMz5e5L\nusnaesfJrvPOE/nPf+wS9u+/t8H5rruc9/72m0hcnJ3O59Ktm+eI+cEH5ZPmrUtH9iN5QpZxijOy\nnz5dZNy48I18i4pE9u512ocPi3z9tYjYv2qO4zW5lQydPhhiIQ/gQDzwE9AaqA0sAc4RDeCqHJmZ\nmRIXd7xHGqIeEyTlqj6R7ppj1CjPmSBz5ngumFm/3gY1b2mYN9+0H6q323zuDh1Ezj3XaQ8aJNKq\nldO+7z7ZV6t26e/kZDbJ6TzrERgrHPm638Q8etRuF+ty4ICdA++ye7fIvfc67d9+szdPXTZtEjnn\nHKe9fbvIzTeLiA3gDXld7mKiBvAQC0cAvxTIdGs/CjwqGsBVOWxeurPbzcUCOUq8E8ALCkQaNnTy\n0QUFNpi42kVFIj17erbT0jzb7ntUFxfbqXfu7UOHKtfptDSRn35y2n36eK5mPFbduiX/tEr85S92\nZO4ya5ZnHrywUL6ZNKk0QNdhktxeq4EToPftk8+btyr9nR3HHvmSc5zAuX27yEknOefbvdtz86v9\n++0yfJe8PLugx6WgwPPz+aAplPAJRwC/CXjDrT0AeEU0gKty2ACeJmCDQBxvy4MmyQkC+fk24Lkc\nOeK5GjEvz3M14qFDdlqdy8GDIrVqOe0DB8q24+I820lJnudr3tzzeu7BNi9P5LTTnC+BQ4ek9E8J\nVx565EjPAL5qlWcefNs2kYQEp/3bbyJJSaUpkpu69JQi94C/a5fsqJNYGsDjKZD2jHQCeHFxWHcM\n1JuY4eFPAK8V6Dogfw4aMWJE6c/JyckkJycHeFkVq9LSBpObO5C8vAHABIhbx/Gjhjkb/teuDf/4\nh/OGOnVsSTCXxERYtcpp160Lu3Y57Xr1PIsT16vnWYChbl27k5/7+d5+22nHx8PttzttETjtNKdd\nWAjHH++s6MzIcF7buhVat4aHH7a7D7qcfrotHOzSpAnMmOG0TzwRli8n9Ywz7O/B9ZXgdvyiTz8h\nqc9A8vKgCFib9DrPppVcuyo7JAYgNTVVCzSEQE5ODjnu/7/xgxHxKwZ7f7MxnYERInJ1SfsxoFhE\nnnU7RgK5hqp+srKySpegp6UNjr1g0KgRJCc7W83efbetnvOHP4T0sjH/e1OVYoxBRHx+OwcawGsB\na4ArgS3AQuBmEVnldowGcBXbxo61e4y4AqarZFlenh3BKxUC/gTwgDazEpFC4K9AFrAS+Kd78FYq\nJv3nP56FD9asgZ49nfb27TbVcvbZnu8rLLT1NN1rTyoVQgHvRigis0XkbBFpIyJjgtEppcJq1y6Y\nP99px8XBunVOu39/W8z4gw9s+4QTbL7blf9evRoaN7YplEGDoHnz8PVd1Wi6nayqmdxvbG7eDJ99\n5rQ7dYIHHoArr7TtlBSYONE+l5trnzvxRJsHB7LXruXR087l+r1FZB0+DAkJ4fkMqsYLKAfu1wU0\nB66igYgzW2P3brj8cls/Mq5kDPPyy9CggU2BAJxzjn3PypXOMd26wZIlMHcuXHABYG8s9ukzsGSf\ncEhKGsrMmRmkNm5svySuuy6cn1JVIyHPgSsVE0TgootgyxbbbtLEBuajR51jXn/dpj9cVqyA88/3\nnNI4d64thuCW+05Pn1Ra5MFwK3XznrAzRdyLCSsVIhrAVfX09NPw/ff2Z2Pg00/h5JOd17/4Atq2\ntbltgHnz4MwzwVVdPT4exo3j4NNP82GrM53K62lpXqv6nMlafqQdY/jIPnHhhTr6ViGnAVxVD7m5\nnjcik5OhRQun3awZTJvmLJDp3h02bIA//tG2mza1QX7mzNISZFmrV3PvjoPU21CXG7N306fPQBvE\n3aSlDSYpaSj/I5fLuZf7k9Z4lhYrLvYscaZUEGkAV7Fpzx6b5nA5dMjWrnS5/HIbtF3i4mDkSCfH\nnZQEr75qA2xRkX3unHPg0kth1CgoKiI9fRJTCl9hPONZT6rXGpipqanMnJnB+R2mYJq8SNu2bTz7\neegQvPOOnTOuVJBpAFex48AB5+elS+H99512aipcdZXn8ePG2VE32DRKYiK8+67z+j33wH33wbff\nOs/17w8HD9qceYmvuJJnedRn11avXs3u3U/y8+L+PH7DLc5IvUEDeO+9qhVTVqoCOgtFxYZff7U5\n5eXLyz+muBg2boRWrWz7pZdgyBA7Mq9Vy960fOMNmy5xLXvftMkeM2mSDbYAO3fCjBmsz8qi/ez/\nsP/Ic4DbDJNjlrD36NGX7OxePMmvnMxW8viFzJR6zJkzw+M4du2ye7Po6k3lB52FomJXURF07gz7\n99t2q1aweLHv9yxYALfeCvn5tn3PPXZ15Ouv23adOjY33qmTs+FVixZ2qXy3bjbdAXahzh130Pqm\nm/hwZgYpKbNISZnlNXi7G8cQ7uU10rjZ+wEjR3qu8FQqUBVtVxjoA91OVvlr1CjPfalXr7b7ex/D\ntZ1pj6v6yLoBA5xtXIuLRerVsxVvXPLyRO67z7OQ74cf2rJk7rKzRRo0sNvZVpLfe2R7+SxKlYcw\nbCerVNXl5tqd/c47z7YvvNCmGFyO3WsEu3Dmxt63cbgkrZEb9xdEhDOnTrV57uRku0DHJTHRjuTd\n9eoFl10G06fDTTfZ5666yu6B8vHHcNZZdvMqP7luZLpucD426EW6edviNc7tD97CQpvWUSoAmkJR\n4bN/P/zyi9PeuNFzSXvPnp4zR7zYfs8D3HbkamAgMJDlxdfS6r33YO9ee8C//gUffuh5g7N/f5g6\n1blZWaeODfY//ACLFjnH/e53dhl8FXLUqampzJkzgzlzZtCtY0fPG6PHWrbMc3MspapIb2Kq0MrP\nd/YGmT7djo6HD6/wba69r5MKC3isV3c6P/AAALf+/moG/GcT17OIAuoAb7M94V5OXPgttG9v37xq\nFfz8s+dCmi+/tMUWTj/dWVK/Y4d9bs0azznjoSZir920afiuqWKOPzcxNQeuqqzC0lrLlnkWy63E\neV055bMYI2/EJ0nWrFmlr/1CnCynuWe++dVXPSupi9gq8e6lxpYuFbnpJs9j8vJsWbW33w5rWTIX\nLU+mykOoa2L689AAXj15u3E3Z9Yske7dnRuBRUWeNw/9UVwss5u3kka8Vlpq8i2ukKfbdSw9ZM2A\nATLpjHaeQe+VV2w1dbfzyI032gryLkVF9sboCy949mvzZpHRo4MbwDdsEJk40echmbNny5T4RDmD\nZ7VAsCpDA7gKGVuceLI4BRwn2yK7CxeKFBZW7mQFBXYUXOKxDpdJXSaWnjuVB6QYRHbtct7z228i\nH39c9lxEGv2TAAATKklEQVTHXjs/X2TlSs/XzzpLJCencn2srG3bRF5/3echKSk3SmeekFrke/4O\nlRL/ArjexFTBddFFld+Jb8QImDKltNl1zN85K+ExmjIeyGBe0vusveMOZ6EN2EU57kvpwRZZ6NzZ\nWRoPdu7488877fh4m/Pu2tXmxUv2PQm6pk1trcwKfEsbCqkdmj6o6q+iCB/oAx2BV0t+z3325rff\nRP75T6ftZe51QUKCrKvf0DNNMm+eyNatvs+9ZYv35++6q+yo+8knRZYv96/PIRDQ71BVe+gIXIWK\na+6zv6sUy1i92vm5dm27V/fIkaVP1Vq2jDZbNjFnzgznvAsWlK03uXSp3SzK5eST7aj72Wc9jzPG\nWWnpMmqUMwc9FDZsgEfL30Ml4N+hqvF0GqEKPRG45hpblqxVq9IpgmC3Y01NTYVXX0X+9jd6dutF\nUXy8ff7KK+3imq5dyz/3+vV2PveNNzrP7dhhK+ekpJQ9/vBhW8yhU6fgfkZvDh2yW8n27h36a6lq\nx59phBrAVWjs3g1HjjgFfletImv9eh4b9ixLl66guPh2svmCsbW3MuQzWyz41t63suPIOMBuHPWv\njFfoPn263c3PPa++ZYtdjHP88c5zIna3woYNnef697cj8vR057nvv7eLel58MVSfXKmg0ACuIic9\nHY47Dv78Z6Bs7UgYyk4O8yEXMjOlCQDZ2b1oytXUpoDNfElKyqyyO/qBvenZrh306+c8N3Om3Sjq\njTec52bPtqsqu3ULzWdUKoQ0gKvw2bTJ7rX92GNeX+7Roy+nZDfiWvbSjxlABvApcAMpKbMAG8AH\nYv+/koHxDODiVpTYGxE7M6V2bTsL5djK8D/+COee6/scobBpE/z1r/DJJ+G9rop5up2sCi3XlqwA\njRt7XxruqjkJLOVUFnGh24tbSUoaSlra4NLSZBkYMjClzwM2OKem2ul/7tautVMBwSnY8PDD8Pvf\nex4nYr9Yfv656p+1qk4+2RZCVioEdASuqu7SSyEjw+7e501Bgd2c6uuvydq82SOFEhf3IO3bn8uY\nMU+WzrzwenPTZf16uye4+wj6v/+1QXzgQOe5AwdsoL7gAs++VDSCVyrKaApFBdfatXZzKtfUu717\n7Xaw7goL7eyL446z7Y8+gmuvhbp1fQfoYx04YAsxPPJI2cB7bDDOz7e7HB67/eyRI/Y8J55YhQ8b\nZPoFoipJUygquBYv9ixpdmzwBls0+PrrnXa/flC3LuC55WqF850TE+1+2YWFns9nZMBTT3k+99FH\ncMUVHukaAD7/HJ5+uoIPFQY7dtiCyTqQUUGmI3BVvo0b4e9/91woc6ziYluYoUsX296504403af4\n+auw0F7ztNPKP2bvXpuaOXZUvWePzcN7619cZMcpWVlZvDHmFfYn1Kn4Lw+lSoR0BG6M6WeM+dEY\nU2SM6VjV86gos2OHM5Jt3tzWmPT1Bbxrl12k4yrMcMIJVQveYL8IRo8u+/y2bXZVI9hR/4kn2gU5\nb77pHOMevF3FHSAqgnefPgOZ8XU/srN70afPQKdivVIBqvII3BjTFigGJgJpIvJDOcfpCDyWdO8O\nL7/se4n5rFnQpIkz2+PIkcAqrbvnh73liidNsl8qf/mL89yaNbZ82urVTr4dbD68UyeYO7fqXyRB\n5KpYDwOpzwEO8nH589uVcuPPCLzKRflEZLXrIiqGLVhgbzp2727b//53xaPW99+3FWxcATyQ4P38\n83aXwf/7P9v29v+nwYPLPnf22XZF5rHHJyTAd9/ZlZpRpBYFFR+kVCVpVdWayH2Ue/SoHUG7eAve\n69bZjaamTrVt93qTgerfv3SbWPdZKs+3P53zO3eGvn2dYw8ehLvusiPyBg3KzkSJj7ePKAreaWmD\nyc0dSF4eHISS+e0Zke6WqiZ8BnBjTDbgrcrs4yLymb8XGTFiROnPycnJJCcn+/tWFWybN8Mf/mDz\nzcY4Nx99adwYfv3V3jysHYS9qydNstXgmzQp3Svl2KX2f573EOnnnssV7u9LSLD58G3bPPcGB3jl\nFXsTdOjQwPsXRMdWrE9L0x0HlXc5OTnk5ORU6j0Bz0IxxsxFc+DRbelSaNvWjkxF7KIYXzM9wK58\nvPVWGDAg+P157TXo1Qtatix9yj1XbGVULldcUGBH4fXqBb27SkVCOOeBayI8grKysujRoy89evT1\nPsPhpZfsIhywo+7ygrf7nOtrr4U2bYLXSdcsEoB77/UI3j4VFtpVlV9/7f1114yZ2rU1eKuap6KK\nD+U9gD7ARiAP+A2YXc5xgZWlUD55q+ry3TPPiEyZUrkTvfeeyBlnhKaT+/eLdOwocvhwuYf4rE4z\ndqzImjVl37Rnjz3vkSOh6bdSEYQfFXl0IU+M85Z6GHTpVN56YTRcconvN2/ebDdbiouz86o//9zm\npkMxs8iPBTXuNzGH/HUgKb16VXzezZvhlFOC0UOloooupa+hNtZvWCZ4l0mziEDHjvDhh/aAunXt\nsvdgBe/vv4dBg5y2Hwtq3JfapwwbBsOHez/QfUCgwVvVYDoCj3HHzt5IShpaprai65iGeY/Rgt2s\nTJpoj2nTxu7wVysEs0nz8+2ugOecU7X3L11qZ5qcfnrZ1554Ajp08JxiqFQ1o7sR1hAV7fLnSrM8\nymb68z7n83BoVgMuXGhHxxWlbsqzcaOdsli/vu/jNm2ypdPcy6cpVc2EdCWmih6pqane5xYfPQoT\nJmBKvkCf5VFe5EHgw9B0ZOfOwN7/wAOwf78tBOxLixaBXUepakJz4NVZQgLs28eQe28jKWkowrsc\n4UPPajeB2r7dmcrXs6d9+MHr1McPPrBl2bzZsMGu2tS/5pQqpSmU6ua99+wNyd69PZ6uVDGFyujT\nx65+7NzZ77e45+3bsoWWCc+RNusD330qKrL7tlx2WRA6rVT00xx4TbR4sQ3gx1anCZXCwkrfBHWf\n+ng3r3Efo7n/qkuZk/1xaPqoVAzSaYQ1wbZtdltVVxqjQ4fQBu+8PLjuOluqDAKewfIP7qU9o8uf\nvjhqFPzgdZcGpWo8vYkZi1zV4GvXhpNOggkTwle4ICnJTuOraKaID2lpgzkyrz/fHLXtWknDyt+h\n74orKt63RakaSlMosWjQILjhBvsIh927Yd68Mnn1KjtwgO1XXcWd9ZtxNL6WlhlTygvNgVcXxcW2\n6voZZ9j2oUPh3bhpwwZbvmzUqPBc75dfbKHiRx4Jz/WUikKaA68uVqyAIUOcdoiCt/vUvjlffGHn\nZAOcempwgvfkyfbLpyL16nlfgamU8qA58Gg1e7YttlCvHpx/PswIbQ3FY5fkn5fzJ07/U2/aTJkS\nnAuI2JG1P389NG1qN9VSSvmkI/BolZtrl5a7hLj2aHr6pJLgPRAYyCsFL3Dv1oPBu4Axtixb06bl\nHzNxolPdXilVIQ3g0WLFCrsIx+Xpp20VnQgppBYSjC+NsWNhyZKKjxOxo/NwzaZRqhrQFEq0qFMn\nNLsC+sm9+C4Esfhux47+7V1iDDz0UODXU6oG0VkokVJcbGtOvvKKLe4bBYK23N696n1FNmyANWsg\nJaVq11KqmtJphNHIPbjNnWv39qhTJ7J9Crbnn7dfSu4FHcqzcCEsWgR33x36fikVQzSAR5u337YV\n4cM1nzoMvI7at261X0pR8peFUrFIA3g02LsXGjWyP+/ebbd4DWAZejTxnHooNE4cyvufTPEv9ZKZ\nadMm8fEh76dSsUgX8kTaoUO2Oo3rzmCTJtUmeIPn1MN+1OWpI+eVjsZ9Kiy0e3/v2RPyPipVneks\nlGDbvNmOKps1swtWVqywm05Vc9O5iS/Yw2XMqfjgWrXsqkylVEB0BB5skyfbRTgu1Th4p6UNJilp\nKJCB8C7FScN9V/rZssVzcZJSKiCaAw/Uvn0wZw706xfpnkREpaYeTpsGO3bY2pdKKZ/0JmY47N1r\nZ5Wkp4d8ubtSqubQAB4qr74K3bvDuedGuiex4aefoE2bSPdCqZiis1BCpXVrW3dSVWz/frjtNjh8\nONI9Uara0RG4PxYvtjvlTZgQ6Z7EpsosrVdKASFOoRhjxgHXAfnAz8AdIrLPy3GxGcCLipxFJnl5\ndr+OCy6IbJ9iye7d0KBBtZ6Fo1QohTqFMgdoJyLtgbXAYwGcK/p07w4rV9qfk5I0eFdWejq8+26k\ne6FUtRaUFIoxpg/QV0QGeHktNkbgRUWwa5dTcGDnTjjhhMj2KZYVFdm0ie7vrVSVhPMm5iDgiyCd\nK+Tcaz9mZWXZJ6dPh2eecQ7S4F01Bw7Y/42P1+CtVIj5HIEbY7KBZl5eelxEPis5ZhjQUUT6lnOO\nqBqBOxswjaUrq1mY+A4zP5lCakqKHTHqzbaq27gReveG777T4K1UgPwZgfvcC0VEfO6yb4y5HegJ\nXOnruBEjRpT+nJycTHJysq/DQ8q1AVMcA7iXP7H8yN9JT59U9eIFytGyJXzzjQZvpaogJyeHnJyc\nSr0nkFkoVwPpQFcR2enjuKgagffo0Zfs7F7Y4r0AGaSkzGLOnNBWfa/WCgp0tolSQRbqHPgrQH0g\n2xiz2BjzegDnChv3DZggo6T2o48NmFTFBg0C170EpVTY1MiFPEGr/VhDlfn9XXSRLVqhqROlgkb3\nQlFBd2wVnqSkR5k5M0O/BJUKMt0LRQWd6ybwBZzPDGaRl/esf1V4lFJBpwFcVckSLuAhxke6G0rV\naFpSTVVKWtpgcnMHkpcHv0LJTeCMSHdLqRpJc+Cq0vQmsFKhpzcxlVIqRulNTKWUqsY0gCulVIzS\nAK6UUjFKA7hSSsUoDeBKKRWjNIArpVSM0gCulFIxSgO4UkrFKA3gSikVozSAK6VUjNIArpRSMUoD\nuFJKxSgN4EopFaM0gCulVIzSAK6UUjFKA7hSSsUoDeBKKRWjNIArpVSM0gCulFIxSgO4UkrFKA3g\nSikVozSAK6VUjKpyADfGPGWMWWqMWWKM+dIY0zKYHVNKKeVbICPw50SkvYhcAHwCDA9Sn0IuJycn\n0l0oIxr7BNHZL+2Tf7RP/ovWflWkygFcRA64NesDOwPvTnhE43+saOwTRGe/tE/+0T75L1r7VZFa\ngbzZGPM0cCtwGOgclB4ppZTyi88RuDEm2xiz3MvjegARGSYipwKTgRfC0F+llFIljIgEfhJjTgW+\nEJHzvLwW+AWUUqoGEhHj6/Uqp1CMMWeKyLqS5g3A4qp0QCmlVNVUeQRujJkOnA0UAT8Dd4vI9iD2\nTSmllA9BSaEopZQKv7CsxDTGjDDGbDLGLC55XB2O6/rDGJNmjCk2xjSJgr5E3eIoY8w4Y8yqkn59\nbIw5Lgr61M8Y86MxpsgY0zHCfbnaGLPaGLPOGDM0kn1xMca8bYzZZoxZHum+uBhjWhpj5pb8d1th\njPlbFPQp0RizoOTf20pjzJhI98nFGBNfEis/83VcuJbSCzBeRDqUPDLDdF2fSgJkCvBrpPtSIhoX\nR80B2olIe2At8FiE+wOwHOgDzItkJ4wx8cCrwNXAucDNxphzItmnEu9g+xRNCoAHRaQddsrxvZH+\nXYnIEaBbyb+384FuxpjfR7JPbu4HVmJjZ7nCuRdKNN7MHA88EulOuETj4igRyRaR4pLmAqBFJPsD\nICKrRWRtpPsBXAz8JCLrRaQA+AB7Qz+iROQbYE+k++FORH4TkSUlPx8EVgHNI9srEJHDJT8mAPHA\n7gh2BwBjTAugJ/AmFcTNcAbw+0r+DH/LGNMojNf1yhhzA7BJRJZFui/ujDFPG2M2AAOBsZHuzzEG\nAV9EuhNR5BRgo1t7U8lzygdjTGugA3ZAEFHGmDhjzBJgGzBXRFZGuk/YNTVDgOKKDgxoJaY7Y0w2\n0MzLS8OAfwCjStpPAenAncG6dhX79BjQw/3wUPengj49LiKficgwYJgx5lHsf8g7It2nkmOGAfki\n8l6o++Nvn6KAzgCoJGNMfWA6cH/JSDyiSv66vKDk3k6WMSZZRHIi1R9jzHXAdhFZbIxJruj4oAVw\nEUnx5zhjzJtAWP4BltcnY8x5wGnAUmMM2LTAImPMxaGeCunv7wl4jzCNdivqkzHmduyfdFeGoz9Q\nqd9TJG0G3G80t8SOwpUXxpjawAxgqoh8Eun+uBORfcaYz4FOQE4Eu3IZ0MsY0xNIBBoaY6aIyG3e\nDg7XLJST3Zp9sDehIkZEVojISSJymoichv1H1zHS89iNMWe6NctdHBVOJTOGhgA3lNz0iTaRvLfy\nPXCmMaa1MSYB+CMwK4L9iVrGjpTeAlaKyIuR7g+AMeYEVzrXGJOEndAQ0X9zIvK4iLQsiUt/Ar4q\nL3hD+HLgzxpjlhljlgJdgQfDdF1/RcufwmNK9ppZAiQDaRHuD8Ar2Buq2SXTml6PdIeMMX2MMRux\nsxk+N8bMjkQ/RKQQ+CuQhZ0x8E8RWRWJvrgzxrwPzAfOMsZsNMaEPA3nh8uBAdiZHtEynfhk4KuS\nf28LgM9E5MsI9+lYPmOTLuRRSqkYpSXVlFIqRmkAV0qpGKUBXCmlYpQGcKWUilEawJVSKkZpAFdK\nqRilAVwppWKUBnCllIpR/w8MpWoOCIdCTQAAAABJRU5ErkJggg==\n",
       "text": [
        "<matplotlib.figure.Figure at 0x106121610>"
       ]
      }
     ],
     "prompt_number": 11
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "Put another way, we are trying to explain the squares of the lengths of the dotted red lines on the plot.\n",
      "\n",
      "At the moment, we have not explained anything, so our current unexplained sum of squares is:"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "np.sum(X ** 2)"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 12,
       "text": [
        "155.66928985835315"
       ]
      }
     ],
     "prompt_number": 12
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "For the following you will need to know how to use vector dot products to project one vector on another.  There is good background in [this Khan academy video on projection](https://www.khanacademy.org/math/linear-algebra/matrix_transformations/lin_trans_examples/v/introduction-to-projections) if you need to revise that - and more background from the same series of videos if you need to freshen up on [vector length](https://www.khanacademy.org/math/linear-algebra/vectors_and_spaces/dot_cross_products/v/vector-dot-product-and-vector-length), [mathematical properties of dot products](https://www.khanacademy.org/math/linear-algebra/vectors_and_spaces/dot_cross_products/v/vector-dot-product-and-vector-length), [unit vectors](https://www.khanacademy.org/math/linear-algebra/matrix_transformations/lin_trans_examples/v/unit-vectors) and [angles between vectors](https://www.khanacademy.org/math/linear-algebra/vectors_and_spaces/dot_cross_products/v/defining-the-angle-between-vectors)."
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "Let us now say that we want to try and find a line that will explain the maximum sum of squares in the data.\n",
      "\n",
      "We define our line with a unit vector $\\hat{u}$.  All points on the line can be expressed with $c\\hat{u}$ where $c$ is a scalar.\n",
      "\n",
      "Our best fitting line $c\\hat{u}$ is the line that comes closest to the points, in the sense of minimizing the squared distance between the line and points.\n",
      "\n",
      "Put a little more formally, for each point $\\vec{v_j}$ we will find the distance $d_j$ between $\\vec{v_j}$ and the line.  We want the line with the smallest $\\sum_j{d_j^2}$."
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "What do we mean by the *distance* in this case?  The distance $d_i$ is the distance between the point $\\vec{v_i}$ and the projection of that point onto the line $c\\hat{u}$.  The projection of $\\vec{v_i}$ onto the line defined by $\\hat{u}$ is, as we remember, given by $c\\hat{u}$ where $c = \\vec{v_i}\\cdot\\hat{u}$."
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "Looking at the scatterplot, we might consider trying a unit vector at 45 degrees angle to the x axis:"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "u_guessed = np.array([np.cos(np.pi / 4), np.sin(np.pi / 4)])\n",
      "u_guessed"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 13,
       "text": [
        "array([ 0.70710678,  0.70710678])"
       ]
      }
     ],
     "prompt_number": 13
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "This is a unit vector:"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "(u_guessed ** 2).sum()"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 14,
       "text": [
        "1.0"
       ]
      }
     ],
     "prompt_number": 14
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "plt.scatter(X[0], X[1])\n",
      "plt.arrow(0, 0, u_guessed[0], u_guessed[1], width=0.01, color='r')\n",
      "plt.axis('equal')\n",
      "plt.title('Guessed unit vector')"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 15,
       "text": [
        "<matplotlib.text.Text at 0x106a48fd0>"
       ]
      },
      {
       "metadata": {},
       "output_type": "display_data",
       "png": "iVBORw0KGgoAAAANSUhEUgAAAXAAAAEKCAYAAAALoA6YAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAGhxJREFUeJzt3X+cXHV97/HXexMSVsBAGuQ3hcqvBu/lR5UicHWqj00o\nl4IBrT+KRvwR67W04gK5MY97CYgPqhiL2mt9pEYItehtiygI7iYVBwixaLlJQEhUFBFEUAQpmFAk\n+dw/ztnNZLOzMztnZs45s+/n4zEP5sfZcz4zYT/z3c/3c75HEYGZmZVPX94BmJlZa5zAzcxKygnc\nzKyknMDNzErKCdzMrKScwM3MSsoJ3HqKpIqkRzq071slvb0T+zZrhRO47ULSWyTdLek5SU9I+jdJ\n7887rrxFxBkR8Q8Akt4p6c5uxyCpKund3T6uFZMTuO1E0iBwNfAxYL+I2A/4c+BUSTNyDc4AWj7z\nTql2BmP5cgK3UZJmAZcB74+Ir0TEbwAiYkNEnBcRL6Tb7TQKHDsalXSMpDWSfiVps6Q31bx2hqT7\nJf2HpEfTLwwkzZH0dUlPpz93x0iykXSgpBsk/ULSjyVdULO/fknXSnpK0v3AqyZ4f4dJ2i6pr+a5\n0feSvo+1kq5K9/djSaeP3VbSMcDngFdLelbSU+Mc682SvjvmuQslfS29P1PSJyQ9LOlxSX8nafea\nbc+WtEHSM5IelDRf0keB/wb8bXrcT6fbniLpu5J+Lek7kl49JuYrJN0F/AY4vN7nY+XjBG61Xg3M\nBL7WYLugzkhQ0h7AGuCLwL7AW4DPpkkPYCWwKCJeChwL3JY+Pwg8AswBXgYsiYhIk+3NwHrgQOD1\nwAclzUt/7lKSpPR7wHxgYb3YmnwvJwGbgd8BPp7Gu9O2EbEZeB/w7YjYKyJmj7Pfm4CjJR1R89zb\ngH9M7/81cARwXPrfg4D/DSDpJGAVMBgRs4DXAD+JiKXAncAH0uP+paTZwC0kfzXNBj4J3CJpn5rj\nnge8B9gT+GnTn4wVnhO41ZoDPBkR20eekLQuHRVvkXRaE/s4E3goIlZFxPaI2AB8BfjT9PUXgGMl\nvTQinomI9TXPHwAcFhHbIuKu9PlXAXMi4oqIeDEiHgI+T/LFAPAm4KMR8euIeBT4FJClTPBwRKyM\nZJGg64ADJL1snO0mPEZEbCX5InwrgKQjgaOBm9K/LN4LfCiN+zngypr39G5gZUR8M93XYxHx/TrH\n/u/A9yPiH9PP+8skX0BnjYQCXBsRm9LXX2z2g7DicwK3Wr8C5tSWGCLilIjYJ32tmf9ffhf4wzTp\nPy3paZKR537p6+cCZwA/Sf+8Pzl9/irgQWC1pB9JWlyzvwPH7G8JySgdklF5bddJ1hHm4yN3ImJL\nenfPFvd1PWkCJ/kMboyI50n+MnkJcE/Ne/oGyRcowMHAjybYb+1fDAey63t+OH1+REe6cix/TuBW\n69vAfwJvaLDdb4A9ah7vX3P/p8DtEbFPzW2viPgAQET8e0S8gSSJfRX4p/T55yLiooh4Ocno8UOS\nXpfu76Ex+3tpRJyZHu/nwKE1x6+9P17ckCTP8WKfjGbKNP8K7CvpOJLR9fXp808CW4G5Ne9p77Ss\nBEnCPWLX3Y173J+RfMnV+t30+cnEaiXkBG6jIuLXJJOYn5V0rqS9JPVJOp6dE/YG4Jx0AvEIkj/5\nR9wCHCXpPEm7pbdXpRObu0n6M0mzImIb8CywDUDSmZKOSMsL/5E+vw34DvCspEvS402T9ApJr0yP\n90/AEkl7SzoYuIA6IuKXJInt7el+3gW8vMWP6wngYEm7TXC83wL/DHwC2IdkboC0RPX3wNWS9k3f\n/0E1df2VwPmSXpd+/gdJOrrmuLUx30ryeb9V0nRJbwaOAb5es407T3qUE7jtJCKuAj4EXEJSTnic\npOPiEpIROsDfkNSsnwCuIZmwjPTnnwXmkYw4f0YyQr4SGGlBPA94SNIzwCLgz9LnjyBJcM8C64D/\nExG3p8nuTOB44MfAL4EVwMho9TKSksFDwBBJ3XqiEed7gYtJRsFzgbtqXhtvcrbevr4J3A88LukX\nExzvepKJ13+unVsAFpOUjP4t/SzWAEcBRMR3gfNJPudfA1V2/GXxKeCNaZfM1RHxFMnnM5i+p4uA\nM9PnG70HKzlluaBD2vZ0O0nnwgzgaxGxpE2xmZnZBDIlcABJL4mILZKmA2uBiyJibVuiMzOzujKX\nUGpm6mcA04BdTmowM7P2y5zA00mWDST10G9FxAPZwzIzs0baMQLfHhHHk/SuvkZSJXNUZmbW0PR2\n7SginpF0C/BKkllzACR5BtzMrAURMWELaKYRuJIFiPZO7/cDAyRrVowNolC3Sy+9NPcYyhBTUeNy\nTI5pKsTVjKwj8AOAVemp133AP0S6foOZmXVWpgQeEfcBJ7YpFjMzm4QpeSZmpVLJO4RdFDEmKGZc\njqk5jql5RY2rkcwn8jQ8gBSdPoaZWa+RRHRyEtPMzPLjBG5mVlJO4GZmJeUEbmZWUk7gZmYl5QRu\nZlZSTuBmZiXlBG5mVlJO4GZmJeUEbmZWUk7gZmYl5QRuZlZSTuBmZiXlBG5mVlJO4GZmJeUEbmZW\nUk7gZmYl5QRuZlZSTuBmZiXlBG5mVlJO4GZmJeUEbmZWUk7gZmYl5QRuZlZSTuBmZiXlBG5mNknD\nw8PMm3cu8+ady/DwcG5xKCJa/2HpEOA64GVAACsi4tNjtoksxzAzK5Lh4WEWLFjI1q0fA6C/fzE3\n3riK+fPnt/U4kogITbhNxgS+P7B/RGyQtCdwD/CGiNhUs40TuJn1jHnzzmXNmrOAhekzqxgYuInV\nq29o63GaSeCZSigR8XhEbEjvPwdsAg7Msk8zM2vO9HbtSNJhwAnA3e3ap5lZ0QwOLmLt2oVs3Zo8\n7u9fzODgqlxiyVRCGd1JUj6pAldExFfHvOYSipn1lOHhYZYvXwEkCb3d9W/oQg08PchuwNeBb0TE\n1eO8Hpdeeuno40qlQqVSyXRMM7NeU61WqVaro48vu+yyjk9iClgF/CoiLqyzjUfgZmaT1I0ulNOA\nO4B7SdoIAZZExFDNNk7gZmaT1JUSShNBOIGbmU1Sx9sIzcwsP07gZmYl5QRuZlZSTuBmZiXlBG5m\nVlJO4GZmJeUEbmZWUk7gZmYl5QRuZlZSTuBm1jVFuRRZr/Cp9GbWFd26FFmv8FooZlYY3boUWa/w\nWihmZpNUpjJP2y6pZmY2kSJdiqyesWWetWsXFrrM4xKKmXVNNy5FlkWRyjzNlFA8Ajezrpk/f37h\nknaZuQZupVCmuqSV1+DgIvr7F5NcKXJVWuZZlHdYdbmEYoXn9jPrpqKUedxGaD2hSHVJs25xG6GZ\nWQ/zJKYVXhnaz8zy4BKKlUJR6pJ58+cwdbgGbtZDPJk7tTiBm/UQT+ZOLZ7ENDPrYZ7ENCsJT+ba\nWC6hmJWIJzGnDtfAzcxKyjVwM7MeljmBS/qCpCck3deOgMzMrDntGIFfA5zehv2YmdkkZE7gEXEn\n8HQbYjGzLvDSvL3DbYRmU0jZLhlmE+tKAl+2bNno/UqlQqVS6cZhzWyM5ctXpMk7OZtz69bkuU4n\ncLc/NlatVqlWq5P6ma4ncDPrsG3bYNq0vKMY5VF/c8YObi+77LKGP+M2QrNesW4dvO1tsGVL3U3y\nuGTYzqP+JJGPjMYtm3a0EX4JWAccJekRSednD8vMmrZuHZx6anKbOxf22qvupvPnz+fGG5NFsAYG\nbvJIuOR8JqZZWa1bBxdfDBs2JKPuPfaAn/98wgSeBy+D2xqfiWnWi0ZG3AMDyf0tW6C/HxYvzjV5\n12tP9Ki/czwCNyuLdet4+j3vof/7P2D37dt2fm3PPeGxxxom8E51g3iU3X7NjMCJiI7ekkOYWcvu\nuivilFPitzNnRsCut/7+iMsvb7iboaGh6O/fL+DagGujv3+/GBoayhze0NBQzJ798nS/I2FdGwMD\n52Te91SW5s4J86tP5DErsuefhwsugI0bmb5t2/jbTJsGH/xgw111ogd8x8j78Jb3Ya1zAjcrst13\nh9tug9NO44X7H2BGbN/p5ef7+tj9kktyq33v+FLYnx2XevPFJrrFk5hmLeramiKzZsHatWzfZ2/G\nziZNnzmzqdE3dLoHfH66388xe/ZHXP/ulkY1lqw3XAO3HtSpenJd11wzWvP+T/VFQLw4c2ZTte9a\nQ0NDMTBwTgwMnNO2+ndXP4cphCZq4O5CMWtBV68Qf+21cP75yUk6a9fCa18LmzfDzJlNdZ50mtc5\n6YxmulBcAzdrwtgk1TW1yfvee5MJyzvvhNe8Bt74xtyTNyR93k7a+XACN2tgvMWYli69gLVrF3f2\nCvHjJW9IauJ33AF9nsKa6pzAzRoYr/3u9tuTMwp3jMrbPGlXL3mPmDWrfcfqAJdVusMJ3KxFHSsd\nNEreBeflY7vHCdysgcHBRaxdu7Cz5ZIRJU/ekN9FI6YiF9HMGmj3Ykx1+8cLkLx9vcySadRnmPWG\n+8DNRtXtmx7p8547N+LFF4sVW077mepoog/cCdxKr90np3TSwMA5uyz69PG5f5B78q4XW6sLUpXp\n36SomkngroFbqZV9wuwd3MnFD9yTe817eHiYe+7ZCJzVlv25N7w7nMCt1Mo2YVY7IfoO7mQVK3nu\n0EPZM+fknXwJngdcNPq8F6QqPk9imnXRyITox+d+hlWs5Jf77ssbj/oD5v3xn+Y2abjjS/ATwBdp\ndkEqT3gWQKMaS9YbroFbBxV5wqxuHTidsHxw95fEdM0OGMw19lZq30X+3HsFnsS0qaCbE2bNHqtR\nt8kDmhZ9fCF9fb+AodyuYjM0NBQzZuwdcHLAyTFjxt4NP8d2Tnja+JpJ4K6BW+l1a8JsMhOm49Xm\n7/3QUuY/cA8P77EXr/jNp9jO+TU/sYJ2TSC2Zjfgz9P7F+cYh02GE7hZk7JMmNZ2m7zvwKPZ/q9j\np58ey23ScPnyFbzwwlWMvK8XXmj8vrp6dqrV5UlM62mdnmh78slfjft87dVv3sF7RrtNuPdeLrzo\nfTtdGaev70JOOGFaqdof2312qrWoUY0l6w3XwC0n7Z5oS2rF+47uD+ZMWC8eGhoaPUnn2UMP3ekk\nnSKd6OIJyWLCV+SxqawTV8058cTTWL9+G3AgsAh4vP4+C7C2SbO8/GvxNHNFHpdQzCZhzpz9SCb7\nbiC5kG8dLSTvPPuq58+fz+rVN7B69Q0Nk7f7vwuk0RA96w2XUGwc3SghdKI00NQ+W1iYqixljLLE\n2QvoRh84cDqwGfghsHic17vyZq08rrjiiujr26crSaATXxQT7rPFVQWb6asuQt3c/d/d0/EEDkwD\nHgQOI2kk3QD8fjiBWx1DQ0PR1/c7vZkEMiwJ2ygxFmXk6wTePc0k8Kx94CcBD0bETwAkfRk4G9iU\ncb/Wo5YvX8H27UfmHUb7ZZywbNRXXZRFu9z/XSxZE/hBwCM1jx8F/jDjPq3nnQosHn3U13chg4Nf\nyi+crK67LnO3yUhfdccuktwmZYlzqsiawJvqD1y2bNno/UqlQqVSyXhYK6sdI7jzgM/R1/dDLr98\nsNxJ4Ikn4HWvg9WrM7UKTrQkQJFGvl7ruzOq1SrVanVSP5OpD1zSycCyiDg9fbwE2B4RH6vZJrIc\nw3pPT/YcR4AmbNnNrCc/N6urmT7wrAl8OvB94PXAY8B3gLdGxKaabZzAzcwmqZkEnqmEEhEvSvoL\nYJikI2VlbfI2M7PO8an0ZmYF5FPpzbrAp5ZbXjwCN8tg7EUe+vsXe2lVa4uOT2I2GYQTuPWsTqx4\naAYuoZhl5vKIFZkvqWZWRzPXwCzSCTY29XgEblbHzuuPJIl85ESaESOnlp9wwjXMnv0RjjnmiFxi\ntanJCdysDTZv3sxTT/0v1q9/LwsWLHS5xbrCJRSzOpotjxRlpUCbepzAzerwyntWdG4jtNIp2qJO\n7gW3TnAfuPWcoibLon2pWPk5gVvP8YkzNlX4RB4rLZ9AY9aYE7i1rFNJdqRMsmbNWaxZc9ZObXmD\ng4vo718MrAJWpZ0hi9p27G7zF5Vl0uiqx1lv+Kr0PamTV0lv5grtAwPnxMDAOblcmb1dinKleSsm\nunBVepui8ux97pVrMrp/3LJyArfC8foiZs1xAreWdDLJTpUTaPxFZVm5jdBa5t7n7PwZWj3uA7dC\nqpe0nMzMdnACt0IZHh5myZIr2bjxe2zf/k7gv4yeSQkU8gxLs7w4gVthjD0FHkZ6uR9nYOAmAJ9h\naVajmQTuSUzrirEtc4kVwFk5RWRWfk7glqPHduq8cEeG2eQ4gVtXjG2Z6+u7kOOOm8uVV+6oc0+F\n1kGzdnIN3LrGXSZmzfMkpplZSXk5WbMC8IqD1iktJ3BJb5J0v6Rtkk5sZ1BmvWKipXHNssoyAr8P\nWADc0aZYzHrOzu2TSR/8yDyAWVYtd6FExGZI6jRmZtZ9roFbYfRirbjXriBkxTLhCFzSGmD/cV76\ncETc3OxBli1bNnq/UqlQqVSa/VGbIsaear927cKeWAtlqiyNa9lVq1Wq1eqkfiZzG6GkbwGDEfH/\n6rzuNkJryFebN9tZN9sIXQjPUS+WHsyssZYnMSUtAD4NzAFukbQ+Iv64bZFZU3ql9OCr05hNns/E\nLLleKj34VHuzHbycrI0qQ3LslavNm3WLE3jJNVN66JUyi5ntzCWUHtBodN1LZRazqcIllCnCpQez\nqclnYk4BRTwb0K2PZtm5hDJFFGkSc2xN3legN9uVL+hgheSavFljvqCDmVkP8ySmdZ3PujRrD5dQ\nLBdFqsmbFZFr4GZmJeUauE2KW/vMysUjcAPc2mdWNC6hWNPc2mdWLC6hmJn1MLcRGuDWPrMycgnF\nRrm1z6w4XAM3Mysp18Ct49x6aJYfj8CtZW49NOscl1Cso9x6aNY5LqGYmfUwtxFay9x6aJYvl1As\nE7cemnWGa+BmZiXlGriZWQ9zAjczKykncDOzkmo5gUu6StImSRslfUXSrHYGZmZmE8syAl8NHBsR\nxwE/AJa0JyQzM2tGywk8ItZExPb04d3Awe0JyczMmtGuGvi7gFvbtK+O8wJMZtYLJuwDl7QG2H+c\nlz4cETen2ywFToyIc+vso1B94F6AyczKoJk+8AlPpY+IgQYHeCdwBvD6ibZbtmzZ6P1KpUKlUplo\n845avnxFmryTBZi2bk2ecwI3szxVq1Wq1eqkfqblMzElnQ4sB14bEU9OsF2hRuBeQc/MyiDzCLyB\nzwAzgDWSAL4dEf8jw/66wgswmVmvmJJroXgBpmz8+Zl1nhezsrbzJLBZdziBW9t5DsGsO7waoZlZ\nD/MVeWxSPAlsVhwuodikeRLTrPNcAzczKynXwM3MepgTuJlZSTmBm5mVlBO4mVlJOYGbmZWUE7iZ\nWUk5gZuZlZQTuJlZSTmBm5mVlBO4mVlJOYGbmZWUE7iZWUk5gZuZlZQTuJlZSTmBm5mVlBO4mVlJ\nOYGbmZWUE7iZWUk5gZuZlZQTuJlZSTmBm5mVlBO4mVlJtZzAJX1E0kZJGyR9U9Ih7QzMzMwmlmUE\n/vGIOC4ijge+Clzappg6rlqt5h3CLooYExQzLsfUHMfUvKLG1UjLCTwinq15uCfwZPZwuqOI/1hF\njAmKGZdjao5jal5R42pkepYflvRR4O3AFuDktkRkZmZNmXAELmmNpPvGuf0JQEQsjYhDgWuBv+lC\nvGZmllJEZN+JdChwa0S8YpzXsh/AzGwKighN9HrLJRRJR0bED9OHZwPrWwnAzMxa0/IIXNK/AEcD\n24AfAe+PiF+0MTYzM5tAW0ooZmbWfV05E1PSMkmPSlqf3k7vxnGbIWlQ0nZJswsQS+FOjpJ0laRN\naVxfkTSrADG9SdL9krZJOjHnWE6XtFnSDyUtzjOWEZK+IOkJSfflHcsISYdI+lb67/Y9SX9ZgJh2\nl3R3+vv2gKQr845phKRpaa68eaLtunUqfQCfjIgT0ttQl447oTRBDgAP5x1LqognR60Gjo2I44Af\nAEtyjgfgPmABcEeeQUiaBvwtcDowF3irpN/PM6bUNSQxFclvgQsj4liSluMP5P1ZRcTzwB+lv2//\nFfgjSaflGVONvwIeIMmddXVzLZQiTmZ+Ergk7yBGFPHkqIhYExHb04d3AwfnGQ9ARGyOiB/kHQdw\nEvBgRPwkIn4LfJlkQj9XEXEn8HTecdSKiMcjYkN6/zlgE3BgvlFBRGxJ784ApgFP5RgOAJIOBs4A\nPk+DvNnNBH5B+mf4Skl7d/G445J0NvBoRNybdyy1JH1U0k+BhcBf5x3PGO8Cbs07iAI5CHik5vGj\n6XM2AUmHASeQDAhyJalP0gbgCeBbEfFA3jGRnFNzMbC90YaZzsSsJWkNsP84Ly0F/g64PH38EWA5\n8O52HbvFmJYA82o373Q8DWL6cETcHBFLgaWS/ifJP+T5eceUbrMUeCEiru90PM3GVADuAJgkSXsC\n/wL8VToSz1X61+Xx6dzOsKRKRFTzikfSmcAvImK9pEqj7duWwCNioJntJH0e6MovYL2YJL0COBzY\nKAmSssA9kk7qdCtks58TcD1dGu02iknSO0n+pHt9N+KBSX1OefoZUDvRfAjJKNzGIWk34AbgixHx\n1bzjqRURz0i6BXglUM0xlFOAsySdAewOvFTSdRHxjvE27lYXygE1DxeQTELlJiK+FxH7RcThEXE4\nyS/diXn3sUs6suZh3ZOjuintGLoYODud9CmaPOdW/h04UtJhkmYAbwZuyjGewlIyUloJPBARV+cd\nD4CkOSPlXEn9JA0Nuf7ORcSHI+KQNC+9BbitXvKG7tXAPybpXkkbgdcCF3bpuM0qyp/CV6ZrzWwA\nKsBgzvEAfIZkQnVN2tb02bwDkrRA0iMk3Qy3SPpGHnFExIvAXwDDJB0D/zciNuURSy1JXwLWAUdJ\nekRSx8twTTgVOI+k06Mo7cQHALelv293AzdHxDdzjmmsCXOTT+QxMyspX1LNzKyknMDNzErKCdzM\nrKScwM3MSsoJ3MyspJzAzcxKygnczKyknMDNzErq/wOHt7A24V7/HQAAAABJRU5ErkJggg==\n",
       "text": [
        "<matplotlib.figure.Figure at 0x1061217d0>"
       ]
      }
     ],
     "prompt_number": 15
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "Let's project all the points onto that line:"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "u_guessed_row = u_guessed.reshape(1, 2)  # A row vector\n",
      "c_values = u_guessed_row.dot(X)  # c values for scaling u\n",
      "projected = u_guessed_row.T.dot(c_values)  # scale u by values to get projection"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [],
     "prompt_number": 16
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "plt.scatter(X[0], X[1], label='actual')\n",
      "plt.scatter(projected[0], projected[1], color='r', label='projected')\n",
      "for i in range(X.shape[1]):\n",
      "    # Plot line between projected and actual point\n",
      "    proj_pt = projected[:, i]\n",
      "    actual_pt = X[:, i]\n",
      "    plt.plot([proj_pt[0], actual_pt[0]], [proj_pt[1], actual_pt[1]], 'k')\n",
      "plt.axis('equal')\n",
      "plt.legend(loc='upper left')\n",
      "plt.title(\"Actual and projected points for guessed $\\hat{u}$\")"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 17,
       "text": [
        "<matplotlib.text.Text at 0x1061e8bd0>"
       ]
      },
      {
       "metadata": {},
       "output_type": "display_data",
       "png": "iVBORw0KGgoAAAANSUhEUgAAAW4AAAEMCAYAAADknlzeAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3Xl8VNX5+PHPmYRIgCBBtrCETUUBZVH4AREIywRqEQla\nRdFirfDFaosSLEVQUIMbhGoVa7EqYLUiIogUsyCEilU2wQpqFSibKCChCiQQknl+f9xJmIRkMsnc\nycwkz/v1ui8yc++ce85MeHLmueeeY0QEpZRS4cMR7AoopZSqHA3cSikVZjRwK6VUmNHArZRSYUYD\nt1JKhRkN3EopFWY0cCulVJjRwK2UUmFGA3eIMsbsNcYMqYbzLDTGPBbo87jPtcMYM6A6zlVBPQLS\n5upqnzGmkzFmuzHmJ2PMvYE+Xyjz5bM0xkQbY+pWV52qgwbuchhjso0xOcaYKB+P32uMGWxjFcS9\nBVp1nQcR6Soi//S3HBve64C0uTLt87MNvwc+EJGGIvJ8FcuoKbx+lsaYCGAW8IgxpsbEuxrTEDsZ\nY9oBvYEjwEgfXyaACVCVAs3vehtjIu2oiI/seK+D/Vn504a2wBeVfVE1f0bVydv7mAikAX8EBlVL\nbaqBBu6y/RJYA7wGjPPcYYxpY4x5xxhzxBjzgzHmOWPMYiAeeM8Yc8IYM8V9rMsY08HjtSW+1hlj\n/mCM2eX+yrvTGDPKl8p5e527J5dijPnMGPM/Y8ybxpgLPPb3MMZ86n7tm4DXr5Du8v7gPk+OMeaV\novLc+35vjPk3cMIY4zDGXO7+tnLcnTq4rlRZQzwetzTGLHO/l3uMMb/18j7/yf38a6Xfa2/lVLbN\n3trr3l+Z9pX5WZTVBvfxU40xB931/KqsHrkxZi1WMHrefdzFPtSpxGdURpk9jTHb3OW9ZYxZUur3\ntNzfYx/e+zLb5OV52z5LABH5QESOiMj3IvKBt2PDiojoVmoDdgFjgUuAfKCZ+/kI4DOsv+DRwAVA\nP/e+/wKDS5XjAjp4PH4VeNTj8Y1AC/fPNwEngebllVeJ130CtABisXpm/+feFwXsAya523KDu32P\nenkv9gL/Blq5y9sAPOax71P3vguAOu737g9AJFYP5yfgktJtwuo0bAVmuI9tD+wGksp5nxM86uRZ\njimvnKq0uYL2+tw+Hz6L0sd2AvZ7fK7xnr87peq4DrjTxzqV+IzKKKvo/fmt+/1JBs5Q8ve0zN9j\nH977Mtvk5flyfyeq+vtbU7egVyDUNuAaIA+IcT/eDtzn/rkvVvrEUcbrfA3cj3k59zbguvLKq8Tr\nbvXY9xTwZ/fPA4BvS732I2+/+O7yJng8/hmwy2PfHR77+gPflXr9G8DM0m0C/h+wr9Sx04BXvL3P\nlSmnKm2uoL0+t8+Hz6L0sRcDh4EhQJ0KPm/PwO1Lne7wUtYA4GCp5z6k4sD9mA/vfZlt8vK8bZ8l\n0AYYDfzd/bgOsMaX/0/hsGmq5HzjgEwROeF+vJRz6ZI2WL9YLjtOZIz5pfsr6nFjzHGgK9DEhtd9\n7/FzHtDA/XNL4NtSxe2j4lzrAY+f97vLKWtfy1KPi8pvyfnaAi2L2uBuxzSgGdAa399nb+UU1amy\nbS6vvZVpX5HSn0X9sg4SkV3AfVgX0g4bY/5ujInzUm4RX+pUen/p15d+fw7gW/49Hi/vfXlt8tJW\nOz/Ly4BNWN80wP1HwYc2hQUN3B6MMdFYqYfBxpjvjDHfASlAN2PMlVi/0PHGulJdWllXtnOBeh6P\n44qOM8a0BRYA9wCNRSQW2EEF/2Gq+jq37zj3i1ykbTl19xRf6mfP/zyerz0EtDHGeNalLef/ZwPr\nvfyviMR6bA1FZARwkPLf59Ln3O+lHKham0u391AV2leR884vIn8Xkf4e9XvKh3J8qZO3tpb1/sSX\nek15v8fePkOvbSrneds+SxHJAu4A/uZ+agiQ5eV9CCsauEsaBRQAlwPd3NvlWF8dfwlsxPrledIY\nU88YU9cY08/92sNAx1LlbQfGGmMijDHDsb7qFamP9Qv3A+AwxvwKq+dckaq+DuBjoMAY8ztjTB1j\nzGigVwWvMcBvjDGtjDGNgenAknKO/QTrP/nv3eUnAiOAN8s4diPWxbLfG2ucbYQxpqsx5mq8v89Q\n8r3e7KWcqrS5rPYW1X9jJdrnrfzSbcAYc6kxZrCxLoSeAU4DhT6U42+d/gUUGmPuNcZEGmOu5/z3\np7zf4014ee/La5OXtnotj8p/ln2wrlEADAXWGWOG+fi+hDQN3CX9EiufdlCsK9FHROQw8Dxwq/uY\n67BydPuxehw3uZ9/Apjh/oo32f3cJPfxx92vX150IhH5Auvi28dYX6e7cu6XrFxVeJ24N0QkHyvv\ndwdwzF33ZRWdEitnmol1oegbILWcup3Fau/PgKNY79vtIvJ1Gce6sAJMd2CP+/gFQEP3vvLeZ/B4\nr7He4zLLqWKby22vuyyf2uel7KLeYenflwvczx3F+qPVBCtN4K0sv+vk/sxGA7/G+j0dC6zCuuhX\npMzfY2+foft15bWpzOcrKq8Kn+UKYKQx5hfu8q7FuvAc9ow7ce9fIdZX2i1YFzmuq+h4FT6MMf8F\nfi0ia20oax8wVkQq/AMVLHa2N1wZYzYCL4jIomDXRZXNrh73JKyhTv7/FVA1kjGmGdAUa3iaCiHG\nmAHGmBbuVMk4rG9x6cGulyqf34HbGNMa6yvIXwn+3WgqBBljegH/Af4kIgeDXR91nk5YeezjwP3A\nje4UoQpRfqdKjDFLgcex8lBTNFWilFKB5VeP2xgzAjgiItvQ3rZSSlULv3rcxpjHgduxhtDVxep1\nLxORX3oco3lvpZSqAhEps0PsV49bRB4UkTYi0h4YA6z1DNoex1V6mzlzZtBvKw3kpu0L760mt68m\nty2c2ueN3eO4tXetlFIBZtv8vCKyHlhvV3lKKaXKFrJ3TiYmJga7CgGl7QtvNbl9NbltUDPaZ8ud\nk15PYIwE+hxKKVXTGGOQci5OBm0po5KTmalg0j+sSoWXoK5BpwEj+PQPqFLhJ2Rz3EoppcqmgVsp\npcKMBm6llAozGrhDmMPhYM+ePcGuhlIqxGjgtlG7du1Yu7bWzr+vlKomQR1VUlV79uzh73+3ltS7\n9dZbaN++fZBrZHGPuwx2NZRSNVxI9rjXrl3LnDlzeOutt3C5XCX27dy5k27d+jBr1vfMmvU93br1\nYefOnbbX4cknn+Tiiy+mYcOGdOnShRUrVhTve+mll+jcuXPxvm3btnH77bezf/9+rrvuOmJiYpg7\ndy7Z2dm0adOmRLmevfJNmzbRt29fYmNjadmyJb/97W85e/as7W1RStUw1TDDlZSlvOdnz35a6tVr\nJ3Xq3C/16/eSkSPHiMvlKt4/atRYMWaugAiIGDNXkpNvK1HGxo0bpWPHbhId3Uj69BkqBw4cKPNc\n3ixdulS+++47ERFZsmSJ1K9fX7777jt56623pFWrVrJlyxYREdm1a5fs27dPRETatWsnH3zwQXEZ\n69atk9atW5co1/OYrVu3ysaNG6WwsFD27t0rl19+uTzzzDPFxxpjZPfu3ZWue2WU9zkopYLL/X+z\nzLgaUj3ukydP8sgjj5Cbu4GzZ+dx6tSHfPDBp3z88cfFx+Tk/IQ1i6xFpD3Hjv1Y/Pjw4cMMHXod\nu3c/SF7eN2zefA2DB193Xs+9IjfeeCMtWrQA4KabbuKSSy5h06ZNvPzyy0ydOpWrrroKgI4dOxIf\nH1+l9vbs2ZPevXvjcDho27YtEyZMYP16nadLKeVdSAXun376CYejHtDS/cwFRER04NixY8XHjBkz\ngnr1ZgE7gB3UqzeLMWNGFO/ftGkTxvQEbgKaUFj4MAcOfMvhw5VbQm/x4sX06NGD2NhYYmNj2bFj\nBz/88AMHDhygY8eO/jXU7euvv2bEiBHExcVx4YUXMn369BJtVUqpsoRU4G7RogUtW7bA4XgS+BFY\ngcu1lV69ehUfM3HieKZNG0uTJiNo0mQE06aNZeLE8cX7Y2Njcbn2AvnuZw5TWHiKmJgYn+uxb98+\nJkyYwPz588nJyeH48eN07doVEaFNmzbs2rWrzNeVvn28fv365ObmFj8uLCzk6NGjxY/vvvtuOnfu\nzK5du/jxxx+ZPXt2pb8ZKKVqn5AK3A6Hg7Vr36N793SioloRHz+djIwVxSkLsILjjBlTOXp0L0eP\n7mXGjKklAma/fv245prO1K8/CIfjD9Sv358HH3yQBg0a+FyPU6dOYYyhSZMmuFwuXn31VXbs2IEx\nhrvuuou5c+fy6aefIiLs2rWL/fv3A9C8eXN2795dXM6ll17K6dOnWb16NWfPniU1NZUzZ84U7z95\n8iQxMTHUq1ePr776ij//+c/+vH1KqdqivOS3XRuVvDhph4KCAnnttdckNTVV3n///SqVMX36dGnc\nuLE0adJEJk+eLImJifLyyy+LiMiLL74onTp1kgYNGsgVV1wh27dvFxGRd999V+Lj46VRo0aSlpYm\nIiILFy6UuLg4adasmcydO1fat29ffHHyn//8p1x22WXSoEED6d+/vzz88MPSv3//4jo4HA69OKlU\nLYWXi5NBm49bxzyHBv0clApN3ubjDqlUiVJKqYr5FbiNMXWNMRuNMduNMV8YY56wq2JKKaXK5tct\n7yJy2hgzSERyjTGRwAZjzDUissGm+imllCrF71SJiBSNd4sCIoAcf8tUSilVPr8DtzHGYYzZDhwG\n1onIF/5XSymlVHns6HG7RKQ70BoYYIxJ9LtWSimlymXbtK4i8qMx5h/A1UC2575Zs2YV/5yYmEhi\nYqJdp1VKqRohOzub7Oxsn471axy3MaYJUCAi/zPGRAMZwCMi8oHHMTqOO4Tp56BUaArkOO44YK07\nx70ReM8zaNd2H374IZdddlmwq8Edd9zBQw89FOxqKKVs4u9wwM+BnjbVpcbp378/X331ld/lOBwO\ndu3aRYcOHar0emPMeRNgKaXCV3jeOblnD8yebW3//W/QqlFQUFBt5/I3naHpEBUUGRmQlGRtGRnB\nrk2NEZqBe+1amDMH3noLSk9zunMndOsGs2ZZW7du1nM2ateuHU8++SRdunShcePG3HnnnZw5c4bs\n7Gxat27N008/TVxcHL/+9a/Jz8/nvvvuo1WrVrRq1Yr777+f/HxrStnSS5cdOnSIG264gWbNmtGh\nQweee+654n0ul4vHH3+8eLm0Xr16cfDgQQYMGABAt27diImJYenSpQCsWrWK7t27ExsbS0JCAp9/\n/nlxWdu2baNnz540bNiQMWPGcPr0aVvfH6V8kpEBycmQlWVtyckavO1S3uxTdm1UdnbA2bNF6tUT\nqVNHpH59kZEjRTyWLpNRo0SMEY+1y0SSk0uWsXGjSMeOItHRIn36iFRy6bK2bdvKFVdcIQcPHpSc\nnBxJSEiQGTNmSHZ2tkRGRsof/vAHyc/Pl7y8PHnooYekb9++cvToUTl69Kj069dPHnroIREpuXRZ\nYWGh9OzZUx577DE5e/as7NmzRzp06CAZGRkiIvL000/LFVdcIV9//bWIiHz22Wdy7NgxETl/CbNP\nP/1UmjVrJps2bRKXyyWLFi2Sdu3aSX5+vpw5c0bi4+PlmWeekYKCAnn77belTp06xXXy+XNQyl9O\npwjIP0FcRf9fnc5g1yps4GV2wNAK3CdOiERFnQvKYAXvjz46d8yAASX3g/Vcke+/F4mJObcvIkLk\nkktECgt9fsPatWsnf/nLX4ofr169Wjp27CjZ2dkSFRUlZ86cKd7XsWPHElPHZmRkSLt27USkZOD+\n5JNPJD4+vsR5Hn/8cfnVr34lIiKXXnqprFy5ssz6lA7cEydOPC8Qd+rUSdavXy/r16+Xli1bltjn\n+cekNA3cKmCcTskF6Qoyoyh4a+D2mbfAbds4blv89BM4SmVvIiLAczmvMWNgyxYoWlmmXj3ruSKb\nNoHnhbjCQjhwAA4fhrg4n6vimeKIj4/n0KFDADRt2pSoqKjifYcOHaJt27ZlHutp3759HDp0iNjY\nWI+qFRanQg4ePOjzkmj79u1j8eLFJVItZ8+e5bvvvkNEaNWqVYnj27ZtqzluVf1SUojesIG1eXkM\nBoiM5NHJk9HL5P4LrRx3ixbQsmXJ4O1ygcfSZUycCNOmQZMm1jZtmvVckdjY8/PihYVQiaXLgOJV\nbYp+btnSWgez9OiMli1bsnfv3jKP9dSmTRvat2/P8ePHi7effvqJVatWFe8vb0m00uLj45k+fXqJ\nsk6ePMnNN99MXFwc3377bYnj9+3bp6NKVPUbNgyWL6ep08nagQNZ0bo1D3/0kXYi7FBeV9yujcrm\nuPfuFenZ00qZxMeXTJP4orBQZPhwK8XicFj/zpxZqSLatm0rV155pRw8eFCOHTsmCQkJMn36dMnO\nzi5OfRSZMWOG9OvXrzjHnZCQUGaOu6CgQHr27ClPPfWU5ObmSkFBgXz++eeyefNmERGZM2eOXHnl\nlfLNN9+Iy+UqkeNu0aKFZGZmFp9zy5Yt0qZNG9m4caO4XC45efKkrFq1Sk6cOCH5+fkSHx8vzz77\nrOTn58uyZcs0x61CwpEjR6Rr164yY8YMcXlet1JlImxy3HYpKBB57TWR1FSRKixd1q5dO3nyySel\nc+fO0qhRI7njjjskLy9P1q1bJ23atClx7OnTp+V3v/udxMXFSVxcnEyaNKk4B+4ZuEVEDh06JLfc\ncou0aNFCYmNjpW/fvsXLmBUWFkpqaqq0b99eYmJipHfv3vLtt9+KiLVUWlxcnDRq1EiWLl0qIiLp\n6enSq1cvadSokcTFxclNN90kJ06cEBErsPfo0UNiYmLk5ptvljFjxmjgViFBg7fvvAVuXbqsDO3b\nt+fll19m8ODBfpWzdu1axo8fX2IB4VATyp+DqpmOHj3K4MGDGTVqFI8++qim8crh7Zb30Lo4WcPs\n2LGjync7KlVTNW3alLVr1xZ3jDR4V54G7gCZNGkSq1atYtGiRcGuilIhR4O3fzRVUsvp56CCSdMm\n5dNV3pVS9rB57pGinveKFSt4+OGHtRPhI+1x13L6OSifFc09kpeHACY6GpYvt8Zr+0l73ufTHrdS\nyn9paZCXx+vA/wGuvDzrORtoz7tygnpxUv+qKhV+RgIvAhOBF0Vs6/3pBUvfBS1w619UpcJMSgps\n2EBMXh6rgWsdDibWqcOLLheO0nMMVZEGb99oqkQp5Rv33CM4ncQ4nax++22+PHGCiRMn4io9P5Af\nNG1SsaBdnFRKhb8TJ05w7bXXcvnll/Piiy/a1vOGkhcsExISmDfvJQBSUiYwzIYLoqHO28VJvwO3\nMaYNsBhoBgiwQET+5LFfA7dSNVigg3fv3r05ePAoBQXPA4bo6KksX76oxgfvQI8qOQvcLyJdgD7A\nPcaYy20oVykVBmJiYli9ejVffvllQNImbdt2oaAgFtgN/JK8vKdIS1tg2znCkd+BW0S+F5Ht7p9P\nAl8C509IrZSqsQIZvKOiLgB+D6wA1tpWbjiz9eKkMaYd0APYaGe5SqnQF6jgnZIygejo2cBvgQNE\nR08lJWWCLWWHK9suThpjGgDZQKqIrPB4XmbOnFl8XGJiIomJibacUykVegKR887IyChOj9TUi5PZ\n2dlkZ2cXP37kkUcCd3ESwBhTB1gFvC8iz5TapxcnlaplAnnBsrYI9KgSAywCjonI/WXs18CtVC2k\nwds/gR5VkgDcBgwyxmxzb8NtKFcpFSg2z/JXlkBesKzt9AYcpWobj1n+ALBxlr+yaM+7anR2QKXU\nOe5Z/p4D3gYrgNs0y19ZtOdtPw3cStVS/YF7gWXVcC4N3vbSwK1UbZOSAtHRdAfSgXuAZf/v/wX8\ntBq87aOBW6naxmOWv+5OJ+nz53PPSy+xbFng+94avO2hFyeVUmzfvp3hw4czf/58GjRoEPCbXfSC\nZcUCOo7bh5Nr4FYqDGzfvp3Bgwdz6pSL/PxnAQI6E58Gb+90VIlSqkLdu3enU6ce5OcL0AAYF9CZ\n+DRtUnUauJVSxWJiGgEpwH3A99VwvnPBe8SIETido0lKuoGMAN0UVFNo4FaqJvHzjkhrJr4XgIeA\njGqZiS8mJobJkyeTkbGGNWtyyMoaQXLyOA3eXmiOW6mawuOOSAFMFe+IDMZMfElJN5CV5QReB7oB\nvXA6V5KZWR2jzEOTtxx30FZ5V8pftWGqz0px3xG5CPg3MDcvD5OWVunAPWzYsCC9l9HAauBTYG8Q\nzh8+NFWiwlJGRgbJyePIyhpJVtZI/WrtYSTWxPhTIGxWSLdSNFOBd4C9ulhCBTRwq7CUlraAvLyn\ngIFAp9q7DqFnTnvgQIiOJhZYA2Qbw5SGDcMieA8bNozlyxfhdK7E6VxZKxYD9oemSlSY2w3cAtTC\n3lnpWf42bIDp02H9eit4T5jA0CeeYMqUKcydOxdr6vzQFbwUTfjRHrcKS+e+Wh8Efgk8zs9+dk2Q\na1U1GRkZJCXdULlhcBkZcOutkJfHO0A+WAF8/XrIzITMTGJvvJE1a9aQnZ3NlClTwqLnrXwkIgHd\nrFMoZb/09HRxOkeL0zlaUlNTpWnTpvLRRx8Fu1qVkp6eLtHRzQVeFVgo0dHNJT093fuLUlNFHA4R\nkAKQ60FGgZwBEafzvMNzcnKkZ8+eMnnyZHG5XAFqibKbO3aWHVfL22HXpoFbVZf09PSwC95O52h3\n0B4qkCmwUJzO0eW/ID29OGi/CnI3SB7ISJBRDoecee+9Ml+mwTv8eAvcmipRNcawYcN47bXXGDVq\nFP/617+CXZ1KMMBMYCyww/uhaWngcpEBLAC2ApOBJY0a4erdm5tffpn8/PzzXhYbG6tpkxpEA7eq\nUcIteJ/L1e8GxgNpDBp09fkHFo0e2boVACfQFYjAHbyvuYYl69bhcrm4+eabNXjXdOV1xX3dgFeA\nw8Dn5eyvjm8VSpUQTmkTz1z93LlzpWnTppKZmel5gEh0tJXZ9NgKQcaD9G3TRnr37i1333235OXl\nyciRI2XUqFFy5syZMs+naZPwgJdUid+3vBtj+gMngcUickUZ+8XfcyhVFRkZGdx+++2sWLGCfv36\nBbs6PtuwYQOjR4/m9fvvx7lundXLzsnhFeC/wKOAadwYrroK1/33M3H5cnbs2EFhYSFXXXUV8+bN\n4+abb8bhcLBkyRKioqLOO8fx48cZOnQoiYmJYTFUsDYK6LSuIvIhcNzfcpSyW7ilTYpcc801vDN1\nKmMffJCsrCzIyQFgBLACKxu+RRwkEUOWw8GLL75I165diYiIYOvWrUyePJklS5Zo2qQGs2WSKWNM\nO+A97XGrUBSWPe+LL2bD7t2MxspFjnA/fQQYBJymD3v4P6Kj/+C+49DJxIkT/ep5O51O5s17CdC5\nX0JBwFfAqShwz5w5s/hxYmIiiYmJfp9TqcoIq+CdkQHDhwNwP/AXYA/QAtjauCkpOTeyng+BZKAD\nTud7ZGYuw+VyVTl49+7dm717D1FQMB8wAV35RpUtOzub7Ozs4sePPPJIuYHbrrHa7dCLkyrEhcUF\nS4+ba3JBuoAkgHQFOVK/vnvc90KBwwL3C7xcYtx3YWGhjB8/Xvr27VupC5aJiSME2gpMFnBVPJ5c\nBRw6jlupMMh5z54NM2aAy8V8YCmwDvgf0AgYXL8+v/71Te7hg+8D3YiOfrDELHqOKua869SJAn6P\nNa/g+wFvqvJTeRHd1w34O3AIOAMcAH4l2uNWISzket7p6SI9ehQP88sA+QKkJcgikCMREdKlWTO5\n5pprpGvXrvLmm28WDx8s7/b4yva8z916/xeBV3279V4FFHrLu1IlhUzwTk+XgsjI4qD9LkgDkHs8\ng/eUKXLkyBHp0qWLdOnSRerXbygDB/68wsDqLXj369dPhgwZVSL4e44n16AdfN4Cty5dpmqtULhg\n+ePFF3Ph7t0AbAF6Ys11+C4wDrjnd79j6Ntv88QTT3DBBRcwZsxYoAOQS926+axY8ZrXC4hlXbBs\n2rQpjz76BHAF8Buio6frhcgQFPBRJRWcXAO3ClnBDt4/1YmibsFZZgBvA/dira/+S+DdqCjGjR/P\nPffcw9ChQ4mNjWPnznFY40xigf8ycGB3srNXeT2HZ/A+duwYX3+9D3gOWAVcCVxc69d3DEUBvQFH\nqXBWmQuWVZo3uwKH69YjEmu1xSXAC8DTxrB49Wqu/8UvWLRoEfPnz2fNmjXs2rUT+BzrkmVHoDtb\nt37I0aNHvZ7D84Ll4cM/YK2f8qb7jL+3pR2qmpWXQ7FrQ3PcKgxUlPM+d/Fuoe/zZvtgc2qq5HnM\nP/I1SOvGjSUtLU0KCwtl7Nix0qBBA7nnnntkwYIFAg6B8QILpW7dZnLLLbdI165d5ciRIxWeq7Cw\nUFq1aicQJ1DXfSFyoTgcsZrTDkHoxUmlKlY6eHterOvRI8EdtL9xx1j7xjlvTk2VLY2bypbGTWVz\naqrs379fOnbsWG7wvuCCutKly1WSnp4uLpdLZsyY4XPwXr16tURERAu0EuglDsdFkpqaaks7lL00\ncCvlo6LgPW/evBI9bIcjVuA+gU4C8wJ+g4q34P3FF19Iy5YtZdGiRSIiVQrerVq1k0aNLpJ33nkn\nYG1Q/tHArVQlpKenS506FwhMF/ih+E5Ch+MigTSBplKnToOApxcCGbyLhgp26dJFBg26TocAhiAN\n3EpVkpUaiXEvKXavwKvSo8dAcTpHyzXXDJe4uDiZN29ewOsR6J63lTa5VODPetNNiNHArVQlpaen\nS1RUI4EGAi0lIiJa3n///eL9+/btkw4dOpQZvO2+kcWX4D1lyhRxOkfL0KHJPl+wtOY9ecV9sfN2\nnZ8kxGjgVqoK0tPTpUePBImMjJL4+Hi59957S6wYU1bwDtTok4ouWFZltMm5CasKBY5q4A4xGrhV\nrRGI27bT09Ploosuks6dO1cYvIuC4dPcIHuJkDM4JD0u3pZ6lBe8W7fuIDBboKXAIoGFMnRocoVp\nk0D9kVH20MCtaoWSgegVWwORr8Hb6Rwt6XSW50A6gRwCcYHIuHG21KOs4O1wRAgMFvhCYIDAS+J0\njvYp5622zH2gAAAZBUlEQVTzk4QuDdyqVjj31f8HgR4CT9v61d+X4P3wJZdYgRqkO0h7989nMbYF\nRs/gvXr1ajGmjgDu4P2KREU1LT6XZ/D2ZVZBFTo0cKta4VzgFoH5AhdJQsIwv8v17JWmpqaeF7yL\n9j8df4l0AJkH8izIOJBIkEkgZzC2fgMoCt6XXnqF+wLjIIE6Ai2ke/d+JY51uVxyyy23iDGRAn/S\ntEiY0MCtaoXSOds6dWKkWbNmsnv3btvKjI5uXiJ4jxw5UurWbSZDuU/O4JB9IBe5g3ZT9791QIbR\nwPaLf/v375fo6PoCAwWS3cG7ucTHX1zi24CIyNChyQLXCXQV+EkvRIYBb4FbJ5lSNcawYcPcC+eu\nxOlcyXvvLWXmzJkMHjyYPXv2VKnMtLQF5OU9BfwcGEde3lOsX/8pr7/+OocPH+ajjCxuPH2COjzD\nQlzEA48D/wCeBCKAB2iFNSXVclvaWaRNmzbcc8/dwD+BC7Amgs2lsDCPyZMnF3WcAGumORgNpAEN\nbK2HCoLyIrpdG9rjVkE2f/58adasmSQkDKt0frdHj4ECrwr0FXhKYKH06DFQRETSU1PlIpDOILeD\ntAb5mzunvQykOchMugk0EZgoxkTI2LFjbW2blR5KE+gosFRgoSQmjpCrr75a7rvvvuKet44gCT9o\nj1vVZh07duT48Tw++mgLWVl9SE4eV4lpWQuAB7B6q08Cv3E/B8PWr+d14DCwFbgJSHS/alREBDc0\nb83syC/o1KkVTucR3njjb2zbto1Zs2bZ1zgALsLqdf8csNaPzMzMZMOGDcU979LfRnThhDBXXkS3\na0N73CrIzl20HCPQrFKjTazXpggME4gSqCeJLduKOJ0ijRuLgKSD9AY5UDQ9q8NhrSMpIsuWLZPm\nzZvL888/L07naBkw4FqJj4+XmTNn2tI2bz3pnJwcufrqqyU5OVmGDk3W0SRhhkBenASGA18B3wBT\ny9hfTc1UNUXRHYuNG3eUHj0G+h1szgXuj923sMf4PNqkZGBMlSh3CuQpjzm0BaSg6GdjREpNkzpj\nxgz3nY2PuO9sbGJ78C5vmN/SpUvdo0mSRBcBDi8BC9xY1152Ae2AOsB24HLRwK2q6NwcIU2Ke5Ge\n45KrWua54DtDwMiFF17o82gTz8D4r9atpWnp4B0dbfW+e/Qo7ml7sv5w3CvQXOCgwEIZMOBa6dy5\ns4wdOzagY6utc88XuFrgQR1NEka8BW5/c9y9gV0isldEzmKth3S9n2WqWiwtbQH5+Zdhjc2IAz4h\nPz+VtLQFVS6zZH73C/74x3kUFhbSt29fn0abDBs2jMzMZWRmLqNvbi5rARfwFnAKIDoajh2DTz+F\ncvPGVwMfAC0BuOCCujz00EO88cabZGUVkpU1spK598qoD2QCNwagbBUU5UV0Xzas34SXPB7fBjwn\n2uNWVWT1EPsIPOrudQ8R6CyDB4+09Twff/yxNGjQwPdx3unpVl47JkYEZAfIwaIed48eFby07Dy0\n1dZnBToLfBCQ3rCOJglfeOlxR/ob9305yPMqemJiIomJiX6eVtVUKSkTWL9+DPn5u4D/A57FmDPk\n5ESRm5tLvXr1bDlPnz59yMrKwul00rdvXz7++GM6dOhQ9sGzZ8PDD4PLVfxUl6IfoqLgiSe8nquo\nx1/0rSElxRrRYT2+EPgEa2z1Yj9b5fu5VejJzs4mOzvbt4PLi+i+bEAfIN3j8TRKXaBEe9yqkjwv\nTl5ySXdp2LChDBkyRIYMGSKnTp2y9Vxee97p6VZv2t2zXg1SWNTLbtzY6oHblnvX3rAqCS89bmPt\nrxpjTCTwH2AIcAjYBNwiIl96HCP+nEOpjz76iFGjRtGtWzcAVq5caVvPG+CTTz7B6XRSr169cz3v\njAxIToa8PAD+CLyCdVHnJcDhdEJmpt/nzsjI8OgNT9DesCpmjEFETJn7/A2qxpifAc9gjTB5WUSe\nKLVfA7fyW7UG76eeokNKCuTkAPBv4AQwCmgB9AKade/HoCcf1kCrAiaggduHk2vgVraoluA9eDD1\n8vL4GOgA/A+4EngYuBwreBsacpRu1K37H1asWKzBWwWEt8Ctt7yrsJGQkMCKFSv47LPPABg5ciS5\nubm2ld+nTx+yGjcmF+gL7AEaYQ3iewTIviCa9ozhKFHAcU6f7sTcuX+x7fxK+UoDtworgQ7evb//\nniys1EjRqOeLgbUvv8xjwGaigRVYWe8jfPHFp7g8RpsoVR00cKuwE5DgnZEBSUm4Cgu5CugEdMS6\n0SY3IoJL7ryTF154HmMWAS8D32LMYaKiDOPHj9fgraqVBm4VlmwN3kUjSLKyiAQigQ+BBKwbFZa0\n6wTAnXfeyaRJvwMWAY8i8mu+//4Umzdv1uCtqpUGbhW2bAveaWmQl8dfgHmAAaJwcBv1eTSyPq3m\nzys+dOfO/Vi34+cDl3H69ByaNGnPrl27NHiraqOBW4W1hIQEpk2bxvr1H7Jt279JSEiocs/7WuAF\nrOD9eeOLuNU5jH6rlpUxaqQFsBZrHmyIjIzkH//4hwZvVX3KuzPHrg29c1IF0Lm7D6cLNBCHo450\n7969cndYpqdbM/yB7AfpaIykjR9fwfnOv9vxxIkTMmDAALnzzjulsLDQjuapWgxdAUfVFBkZGSQl\n3UBS0g3Fdx1aa0IOA57D5bqA/fu/rVzaZNgwWL4cnE7aOJ2sW7yYF9auZd68eWUcWv5KMg0aNNCe\nt6oWegOOqjb+3t6dkZFBcvI4d6CG6OipXHbZZWzb9iusSZruAa4jMnIxAwf2B6p+k86BAwcYNGgQ\nv/nNb5g8eXKlXnvy5El+/vOfU7duXVyu+hhj9HZ2VWnebsDRVImqFudSDK8KvFKlCZXOrWTzP4Ec\n98K9CR6pi3sEHJKSkiJNmjTxe2Kq/fv3S8eOHSUtLa3Sr12+fLk4HHUE+le5vap2Q1MlKtjOpTTy\ngbXk5T3hx+IIi4GhwCmaNGnukbr4junTp/G3v/2NOXPmlBxt8u67kJRkbT4uVtCmTRvWrVvHCy+8\nUGbaxJsXXngNl+s5rDEqG/xsr1Il+Tsft1KVdBvW2jEvIdKiUq9MSZnAhg3jyMt7EmiKMZOYMOHv\nDBs2rEQaonv37tx7773MmTOHBx54gG4NGzJy1ChWAvUANmywcto+pC6KgvegQYMAKpk2qQv8A3iU\nopXhlbJFeV1xuzY0VaKk9GiMv4jDESWDBw+WgoKCMo8tbx3Gon1DhybL6NGjpWfPnpKTk3NeGUuX\nLpXmzZvLq6NHSxOQIe7tVNF82k5npepf2bSJzrWt/EUgV3mvaNPArYp4BuQVK1bIkCFD5LbbbisR\nvFNTU8XhuMi9fFlKccArK5i7XC6ZPHly+cF7+nRpDvIqSBOQW0AOVTFwi1QteAdyIWBVs2ngViHp\n1KlTJYJ3enq6OByx7l7qHIFGAikeFyBfPa/36jV4O52y1L0i+2uea0Q6HFVeucafC5ZKVYa3wK3D\nAVVQ5ebmMnLkSOLi4vj++1OsWXO9e08BMBloQePGheTkTAUWAn8FNuF0riQzcxlgdT6mTJlCdnY2\na9asITY21ioiKQmyslgF9ABaFZ00NRWmT69ynf0ZKqiUr3Q+bhWy6tWrx8qVK/nuu+/YuXML1nx8\ncVjLl94AfEOjRnWBKGAi1miSQyXKMMYwd+5cEhMTGdq7N8cHDbKC9sCBEB3NCNxB2+HwO2iDf6NN\nlLJFeV1xuzY0VaJ8cOrUKenevbtERNQVeEVgioCRUaNGSaNGjSQq6kJ3CmW8gEMWLFhwXhmu99+X\nyZGR0hMkB6zb2FNTrXy2nwv7lkXTJiqQ0FSJCge5ubkkJCRw+PAxunS5mkGDruaZZ57h8ccf54EH\nHqBt28to1qwl3bp14I033mDNmjVcfvnl5wpISkKyspgCHASWANi0qG95PNMmXbp00YV/lW0Ccuck\n8AtgJ1AI9PRyXDX8bVI1RekLlhkZGdK0aVN56aWXpFmzZvLuu++KiMjixYulZcuW8sWCBed61D16\niIC4inrcVRw9Uln79++XuLg4iYxsoMP/lG0IxKgS4DLgUmCdBm5lJ5+D95Qp0hLki6IgHRVlbUWP\no6NtT4+Up3//4QLNBNLcp18oTufoajm3qpm8Be4qX5wUka9E5Ouqvl6p8nhesLzjjjsoKCigTZtL\n+c1v7uWWW25h/PjxrJw1i9tfeYUnsS5XfgmQnw9duljpEafT57sj7VC3bj1gKvA6cLhazqlqsfIi\nuq8b2uNWAVLeBcvH+vaVZiDvunvWi0EGulMk1ZEaKUvJSbQ0VaL8h5cet9e5SowxWVjLfZT2oIi8\n5+sfh1mzZhX/nJiYSGJioq8vVbVYvXr1aNw4nsJCa2IquJee/Jc/fbyMx4Hx7uNuB24CTHQ0pKQE\npa5F83Sfuzi5SC9OqkrJzs4mOzvbp2P9HlVijFkHpIjIp+XsF3/PoaqPv3Nm2y0p6QaysoYBbwBb\nmIGhPye5DWvlx4VANhDZuDG88Ua1pUaUCrTquAGn7CErKqwULVSQlTWSrKyRJCePI8PHKVADJSVl\nAtHRDwO30ooL+CMn+Qr4G/AqsB6IdDg0aKtapcqB2xiTbIw5APQB/mGMed++aqlgODdndj9gOXl5\nj5aYQ7r0smHV4dxSYRlsifiJbsCDwLfAP4EIgEcf1aCtapUqz8ctIsuB5TbWRYWMDkAs8EcKCy8F\nPJcNexIwbNgwrsR6i4FUPN/2RReRlZPDncBVuIN2TIzft7ArFW50rhJVzEpLTMVKRAwgImIvP/yw\nl1OnTnn0xtcBdcnLe6r6V3SZPJl6wJvAlUXPTZ1avXVQKgRo4FbFSq5gvoqVK5fRs2dPRowYQWFh\n0QouU4D7gI0BrcuW2bPZelEztl7UjC2zZ1tPTp9uTRLVuLG12TBhlFLhSOcqUV4VFhZy11138emn\nn/L1199z+vTTwAFgJtOmTeXxxx+3/ZxbZs+m84wZFK3Nngt8kZrK1RqkVS3ibVSJBm5VIc/g3aRJ\nOyIiIhk92skjjzzCM888w80332zr+bZe1Iyrco6yFrgEaANsbdyUq44dsfU8SoUyb4FbFwtWFYqI\niOCvf/0rd911F3v37mXlylXUr1+fhIQEkpKSAGwP3gD/BuYAOlxJqZI0x618UhS827Vrx4gRIzh1\n6hRXXHEFmZmZ3HfffSxZssS2c8nkSeRiZdLfxkqVyORJtpWvVLjTVImqlKK0yd69e1m1yup5f/75\n5wwcOJDWrS+hRYvWttxxuWX2bMy8ZwEraGt+W9U2muNWtiodvDds2MD119/KmTOFwK1ER79TbWO8\nlaqpNHAr23kGb4ejIWvXjgZOAo8Bo3E6Dxcv5quUqjxdLFjZzjPnvW3bv7Ay0e8AvYDX+f77g8Gt\noFI1mAZuVWVFwfvqq3vgcEwCRgG7cThOc+DA17ZesFRKnaOpEuW3wsJCfvazn7FlyzauvLI3P/54\niNatW7N582aeffbZgAwVVKqm01SJCqiIiAjef/99rr9+BMbkkpmZSW5uLr169WLSpEklet5l3squ\nlKoU7XEr23hesHzrrbcYM2YM9erVs3red97J8LfeosHu3YA1s5/eyq5U+XRUiao2ZQbvU6fYvHEj\nzwI3A9cBdwHXo7eyK1UeDdyqWp0XvDt0oN7Jk2wGngW6YM323QoN3EqVR3Pcqlp5DhW8aehQ3szN\nJRdroOB84FKsoK23sitVNRq4VUBERETw11/8gnY7d3KTy8WbQAusuUeigFMREZrfVqqKNFWiAicp\nicKsLO7CWk36laLno6Jg5UpdJ1IpLwI2rasxZg4wAsgHdgO/EpEf/SlT1SwRwF+B40VPNG6sK7Ir\n5Sd/UyWZQBcR6QZ8DUzzv0oqrGVkQFKStQ0cCNHRRABNAKKjNWgrZQPbUiXGmGTgBhG5rdTzmiop\nJSMjo3ihXTumQA0ZGRmQnAx5eQhgoqOtNSHXr7f2p6Ro0FbKR9W1As6dwN9tLK9GysjIIDl5nHvF\ndNiwYVzNmQI1LQ3y8vgb8C/g+bw8HOvXQ2ZmsGumVI1SYeA2xmRhDQgo7UERec99zHQgX0TeKKuM\nWbNmFf+cmJhIYmJiVepaI6SlLXAH7SuBVPLyHiMtbUHNCNxuI4EXgHuB50V06JJSPsjOziY7O9un\nY/1OlRhj7gDGA0NE5HQZ+zVV4iEp6QayskYCY4Fbgf8weHB7PvhgRZBrZgOPVMlPwHCHg+7XXsvz\n776Lw6HhW6nKCNgNOMaY4cADwPVlBW11vpSUCURHTwVeB4YTEfEfTpw4xOnTNeDtGzYMli8Hp5OG\nTifpS5ey/dgxrr/+epzO0SQl3UBGRkawa6lU2POrx22M+Qbrfooc91Mfi8hvSh2jPe5SPC9O3nff\nr1m4cCEnT57knXfeoW7dukGunb3eeecdfvGLMbhc/YHbiI6eVnNy+koFkM5VEuIKCgq49dZba2Tw\ntlJDTmAxcBVwNU7nSl3WTKkK6FwlIS4yMpI33niDBg0aMHr06JqRNikhGkgHrg12RZSqETRwh4ia\nGrzP5fSXA0eIjp5KSsqEYFdLqbCmqZIQUxPTJjX2hiOlAkhz3GGmJgZvpVTlaI47zNTUtIlSyh4a\nuEOUBm+lVHk0cIcwDd5KqbJo4A5xGryVUqVp4A4DGryVUp40cIcJDd5KqSIauMOIBm+lFGjgDjsa\nvJVSGrjDkAZvpWo3DdxhSoO3UrWXBu4wpsFbqdpJA3eY0+CtVO2jgbsG0OCtVO2igbuG0OCtVO1R\n5cBtjHnMGPOZMWa7MeYDY0wbOyumKk+Dt1K1Q5Xn4zbGxIjICffPvwW6ichdZRyn83FXM53PW6nw\nF5D5uIuCtlsD4IeqlqXsVaLnPWAAp4cMgaQkyMgIdtWUUjbwawUcY8xs4HYgF+gjIv8r4xjtcQdJ\nwerV3DpyJCcLC3kHqBsdDcuXgy4dplTIq/LSZcaYLKBFGbseFJH3PI77A9BJRH5VRhkauIMlKYmC\nrCxuBZoALwA4nZCZGdx6KaUq5C1wR3p7oYg4fTzHG8Dq8nbOmjWr+OfExEQSExN9LFb5KxLrw8kJ\ndkWUUl5lZ2eTnZ3t07H+XJy8RES+cf/8W6C3iNxexnHa4w6WjAxIToa8POuxpkqUChsBWeXdGPM2\n0AkoBHYDd4vIkTKO08AdTBkZkJZm/ZySokFbqTARkMBdiZNr4FZKqUoKyHBApZRSwaGBWymlwowG\nbqWUCjMauJVSKsxo4FZKqTCjgVsppcKMBm6llAozGriVUirMaOBWSqkwo4FbKaXCjAZupZQKMxq4\nlVIqzGjgVkqpMKOBWymlwowGbqWUCjMauJVSKsxo4FZKqTCjgVsppcKMBm6llAozfgduY0yKMcZl\njGlsR4WUUkp551fgNsa0AZzAPnuqc052drbdRYYUbV94q8ntq8ltg5rRPn973POA39tRkdJqwpvr\njbYvvNXk9tXktkHNaF+VA7cx5nrgoIj828b6KKWUqkCkt53GmCygRRm7pgPTgCTPw22sl1JKqXIY\nEan8i4zpCnwA5Lqfag18C/QWkSOljq38CZRSSiEiZXaIqxS4zyvEmP8CV4lIjt+FKaWU8squcdza\nq1ZKqWpiS49bKaVU9Qn5OyeNMb81xnxpjNlhjHkq2PWxW029gckYM8f9uX1mjHnHGHNhsOtkB2PM\ncGPMV8aYb4wxU4NdHzsZY9oYY9YZY3a6/7/9Lth1CgRjTIQxZpsx5r1g16WqQjpwG2MGASOBK0Wk\nKzA3yFWyVSBvYAoBmUAXEekGfI01CimsGWMigOeB4UBn4BZjzOXBrZWtzgL3i0gXoA9wTw1rX5FJ\nwBeEcYo3pAM3cDfwhIicBRCRo0Guj90CdgNTsIlIloi43A83Yo08Cne9gV0istf9O/kmcH2Q62Qb\nEfleRLa7fz4JfAm0DG6t7GWMaQ1cC/yVMB7CHOqB+xJggDHmE2NMtjHm6mBXyC617AamO4HVwa6E\nDVoBBzweH3Q/V+MYY9oBPbD+6NYkfwQeAFwVHRjKvN6AUx0quMknEogVkT7GmF7AW0CH6qyfP2r6\nDUxe2vegiLznPmY6kC8ib1Rr5QIjbL9aV4YxpgHwNjDJ3fOuEYwxI4AjIrLNGJMY7Pr4I+iBW0Sc\n5e0zxtwNvOM+brP7It5FInKs2iroh/La5r6BqT3wmTEGrDTCVmPMeTcwhTJvnx2AMeYOrK+lQ6ql\nQoH3LdDG43EbrF53jWGMqQMsA/4mIiuCXR+b9QNGGmOuBeoCDY0xi0Xkl0GuV6WF9HBAY8z/AS1F\nZKYx5lJgjYjEB7tedquJNzAZY4YDacBAEfkh2PWxgzEmEvgP1h+iQ8Am4BYR+TKoFbOJsXoRi4Bj\nInJ/sOsTSMaYgcAUEbku2HWpilDPcb8CdDDGfA78HQi7v4w+Ct2/nlX3HNAAyHIPvXoh2BXyl4gU\nAPcCGVijEpbUlKDtlgDcBgxyf2bb3H+Aa6qw/X8X0j1upZRS5wv1HrdSSqlSNHArpVSY0cCtlFJh\nRgO3UkqFGQ3cSikVZjRwK6VUmNHArZRSYUYDt1JKhZn/D5A2DotiBshvAAAAAElFTkSuQmCC\n",
       "text": [
        "<matplotlib.figure.Figure at 0x1061fcc50>"
       ]
      }
     ],
     "prompt_number": 17
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "The projected points (in red), are the positions of the points that can be explained by projection onto the guessed line defined by $\\hat{u}$.  The red projected points also have their own sum of squares:"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "(projected ** 2).sum()"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 18,
       "text": [
        "133.38132074316005"
       ]
      }
     ],
     "prompt_number": 18
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "As we will see later, this is the sum of squares from the original points that have been explained by projection onto $\\hat{u}$."
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "Once I have the projected points, I can calculate the remaining distance of the actual points from the projected points:"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "remaining = X - projected\n",
      "distances = np.sqrt((remaining ** 2).sum(axis=0))\n",
      "distances"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 19,
       "text": [
        "array([ 0.9264258 ,  0.71426675,  0.29312499,  0.41527826,  0.06212573,\n",
        "        0.79318814,  0.68455384,  1.68654891,  0.34062881,  0.00674608,\n",
        "        0.30113815,  0.40539657,  0.99582768,  0.17135551,  1.09474195,\n",
        "        0.78058277,  0.18356587,  0.9747337 ,  0.73200779,  0.49583326,\n",
        "        0.9632401 ,  1.36281666,  0.26286832,  0.09259664,  0.47780329,\n",
        "        0.04151855,  0.84132951,  0.33800965,  0.0198244 ,  0.85335614,\n",
        "        0.06981433,  0.24426315,  0.34796767,  0.47006229,  0.70514523,\n",
        "        1.17370933,  0.83870882,  1.00606869,  0.73159401,  0.74942988,\n",
        "        0.34328108,  0.55683988,  0.28791204,  0.47947506,  0.9777346 ,\n",
        "        0.06430814,  0.1273749 ,  0.15742452,  0.0101704 ,  0.51999721])"
       ]
      }
     ],
     "prompt_number": 19
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "I can also express the overall (squared) remaining distance as the sum of squares:"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "(remaining ** 2).sum()"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 20,
       "text": [
        "22.287969115193118"
       ]
      }
     ],
     "prompt_number": 20
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "I'm going to try a whole lot of different values for $\\hat{u}$, so I will make a function to calculate the result of projecting the data onto a line defined by a unit vector $\\hat{u}$:"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "def line_projection(u, X):\n",
      "    \"\"\" Return columns of X projected onto line defined by u\n",
      "    \"\"\"\n",
      "    u = u.reshape(1, 2)  # A row vector\n",
      "    c_values = u.dot(X)  # c values for scaling u\n",
      "    projected = u.T.dot(c_values)\n",
      "    return projected"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [],
     "prompt_number": 21
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "Next a small function to return the vectors remaining after removing the projections:"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "def line_remaining(u, X):\n",
      "    \"\"\" Return vectors remaining after removing cols of X projected onto u\n",
      "    \"\"\"\n",
      "    projected = line_projection(u, X)\n",
      "    remaining = X - projected\n",
      "    return remaining"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [],
     "prompt_number": 22
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "Using these little functions, I get the same answer as before:"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "(line_remaining(u_guessed, X) ** 2).sum()"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 23,
       "text": [
        "22.287969115193118"
       ]
      }
     ],
     "prompt_number": 23
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "Now I will make lots of $\\hat{u}$ vectors spanning half the circle:"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "angles = np.linspace(0, np.pi, 10000)\n",
      "x = np.cos(angles)\n",
      "y = np.sin(angles)\n",
      "u_vectors = np.vstack((x, y))\n",
      "u_vectors.shape"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 24,
       "text": [
        "(2, 10000)"
       ]
      }
     ],
     "prompt_number": 24
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "plt.plot(u_vectors[0], u_vectors[1], '+')\n",
      "plt.axis('equal')\n",
      "plt.tight_layout()"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "display_data",
       "png": "iVBORw0KGgoAAAANSUhEUgAAAaYAAAEZCAYAAADc7YGjAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAGItJREFUeJzt3W2MXOdd9/Hfz7tr1r29KLaCbEiCNqIGNdigIDABW+pE\ncKNtEI0XASE8F0QtFCPgDSGAlL2lm4e8SlS59K5KqfoCkSKojStCQ0AeIBYEIkpq07jEwiuctjEP\nSatNa8sP+79fzKy93ux5mJkz51wz8/1Iq+zsdfbMleOz5zfXw7mOI0IAAKRiS9MVAABgPYIJAJAU\nggkAkBSCCQCQFIIJAJCU6breyDbT/wBgQkWEy25ba4spIvT4448rIvhq4Itjz7GfxC+OffPHvld0\n5QEAkkIwAQCSUnswtVqtut8SXRz75nDsm8Oxb06/x9799P/19UZ21PVeAIB02FakOvkBAIAiBBMA\nICkEEwAgKQQTACApBBMAICkEEwAgKQQTACApBBMAICkEEwAgKQQTACApBBMAICkEEwAgKQQTACAp\nBBMAICkEEwAgKQQTACApBBMAICkEEwAgKQQTACApBBMAICkEEwAgKQQTACAphcFk+w9sX7R9Omeb\n99l+xfZLtu+ttooAgElSpsX0EUkLWYW2H5D09ojYI+m9kj5QUd2AJLXbkj38r337mv4/BZoxXbRB\nRPyd7fmcTd4t6aPdbV+wfZvtXRFxsZoqAvWYm5PefLPpWtx05kwnoIrY0urq8OsD1KWKMaY7JF1Y\n9/pVSXdWsF+gcnNz2S2UlEKpFxH5LS9g1BS2mEraePrHZhstLS3d+L7VaqnValX09sCtFhel48eb\nrkUaNgun7dullZX664LJ0G631W63+/59R2yaIbdu1OnK+0REvKXX2/b/k9SOiKe7r89KeufGrjzb\nUea9gF499ZT0K7/SdC1G3/y8dP5807XAOLKtiCjdfq+ixXRC0hFJT9u+T9IXGV/CMN19t7S83HQt\nxs/y8q2tq6kp6dq1xqqDCVYYTLb/SNI7Jd1u+4KkxyXNSFJEfDAinrH9gO1zkr4s6T3DrDAmz759\nnYkAqNf167cG1cyMdOVKc/XB5CjVlVfJG9GVhx6kPmhfx6m8bZt0+fLw36dfBw5Izz/fdC0wCnrt\nymPlByTh4MHmZ5LNz3cCp8xXHS5dKleXJ5+spz4bnTp1899r27Zm6oDxRIsJjdm9W7pY82jkpA3w\nt9vS/ffX+55btnS6AYE1vbaYCCbUqq6JCwzc5+PfAXWiKw/JWVy82eUzjIuh/dbuLS6G+c6ff+sx\n2769+vdZm0Bhd25uBsogmDA0axekqm903b791gsqy/FUY2Xl1uN64EC1+3/zzZvnxOJitfvGeKEr\nD5UaxrjR7GxnIgCaNYxp+4xHTQa68tCItU/CVYXSyZM3P7kTSmk4ffrWFtXMzOD7XF29ee4cPDj4\n/jAeaDGhb1V+gmaQfLRVOZmCG3nHDy0mDN22bZ1PuIOG0o4dTFYYF+snUxw6NNi+rl5lZfRJRzCh\ntLWLxSCrEayfuPD669XVDek4dqy6CRRr59yRI9XUDaOBrjwUGvSTKwPckKqZGMMySKOJrjxUYv3j\nw/v15JOdT82EEiTptddutqS29HnlWVsG6e67q60b0kKLCbcYdAkbHkCHXgz6QMdJW2JqVLEkEfo2\nSOvo5EmJBxJjENPT/beu9+7tTGdHmujKQ8/67bKbmbnZNUMoYVDXrnXOpb17e//dM2dYUWKcEEwT\nbGqqv0BaezwE95pgGNZu5D15svffPX68c06325VXCzWiK28Czc111i3r1SOPSEePVl8foMiWLf09\nB4tLThroykOmtYfx9RpKa8sDEUpoyupq5xycmurt92xp69bh1AnDQ4tpQvTTZcc/F1LVz2PnmSDR\nHGbl4Rb9zHTinwmjYuvWzhJGveD8rh9deZDUWWDV7i2U1mbYAaPiypXeu/jo3ksfwTSGel1gdW2F\nBmBUrU01L9tlvbZQLI/aSBNdeWOk19l2rDuGcdXrmCqXpuGiK28Cra1rVzaU1h43QShhXEV0bm8o\ni/X30kKLacT1MjuJVb4xiXpd1ZzLVPVoMU2QXp6NxCrfmFRrq5qXnSBhdyYPoTkE0wjaubN8H/re\nvXwCBKSbEyTKOHOm/0dzYHAc+hFjS2+8Ubzd2hIu3FAI3Cqis95jme1YGLYZBNOIWFws30o6eZJu\nOyDP+fPlp5cfP94Zy0V9CoPJ9oLts7Zfsf3oJuW32/6k7X+xfcb2zwylphNs69ZyD1Pbvp1HUAC9\nWF2VDh0q3u7y5cGeV4be5M7Ksz0l6bOSvlfS5yT9k6SHI+LlddssSfqqiHjM9u3d7XdFxLUN+2JW\nXh/K/jFwaIHBlF2+i/v/elf1rLz9ks5FxHJEXJX0tKQHN2zzBUlf3f3+qyX9z8ZQQu/WVgIvstZK\nAjCYa9fK3ft06hRde8NWFEx3SLqw7vWr3Z+t9yFJ32z785JekvRL1VVvMm3b1jn5i5w8Ka2sDL8+\nwKQ4erTc2BNde8M1XVBe5rP4r0v6l4ho2f4GSc/Z/taIeMslc2lp6cb3rVZLLQZD3qLMyT4zw9Nj\ngWFaXe30WhR9QLR5gOZm2u222gM8RrhojOk+SUsRsdB9/Zik1Yh4Yt02z0j6rYg41X3915IejYgX\nN+yLMaYCZUKJPwKgXmX+LufnOzP9sLlex5iKWkwvStpje17S5yU9JOnhDducVWdyxCnbuyR9k6R/\nL1sBdO4yL7MaOLkO1C+ieIHk5eXO5IlrjK5XIneMqTuJ4YikZyV9RtLHIuJl24dtH+5u9tuSvt32\nS5L+StKvRsTrw6z0OJmbKw6lXbsIJaBJKyudx8PkuX6dcaeqsIhrg6amOn3ZeU6e5L4kICVlwodL\n3a1YxHVE2MWhxM2yQHoiOrdp5LGlI0fqqc84IpgaUPSJi3uTgLStrBTf8/T+97NKeb8IphqtPdAv\nz6FD3JsEjIK1e57ynDnTeRoAesMYU00WF4vXu5vgwwOMtLXV/PPKJ3lh5aqni6MCZaaDE0rA6Fpd\n7bSMsh5Js7ra6S3h77wcuvKGbOfO/FCamuJkBcbB6693FnjNw3TycgimIZqezn+o365d3JAHjJPn\nn+/c4pGHcCrGGNOQlJnkcOxYPXUBUL+ia8AEXQ4ZY0pB0QnJTbPA+CtapZwxp2wEU8X4lARgDeHU\nH8aYKkQoAdio6O+eMae3IpgqQigByFKm5YSbCKYKEEoAiqyudh7ymYVwuolgGhChBKCsK1ek2dns\ncsKpg2AaAKEEoFeXLuWvTk44EUx9I5QA9GtlhXDKQzD1YWoqv5xQAlBkZYVuvSwEU4/m5vIf8Eco\nASjr0iXCaTMEUw/27ZPefDO7nFAC0KtLl/J7YSYxnAimkhYX81cJJ5QA9KtoMedJCyeCqYSnnsp/\nyB+hBGBQRdeRubl66pECVhcvIe/Tyoj+LwFIVN71Zn5eOn++tqpUptfVxWkxFcg7SYqeuwIAvcr7\nsLu83BlWGHe0mHLkhdKBA52HggHAMIxTT02vLSaCKcPUVPa08B07Oo9RBoBhGpdwoiuvAjt3ZofS\nli2EEoB6HDqUXTbOM/VoMW1w8KB06lR2+Qj8LwAYI7t3Sxcvbl62ZYt0/Xq99ekHLaYBtNuEEoC0\nvPZa9uMyVlfHcxo5wbTO/fdnlxFKAJpy5Up22Ztvjt9MvcJgsr1g+6ztV2w/mrFNy/anbJ+x3a68\nljUomoEHAE3K+3CctwDAKMoNJttTko5KWpB0j6SHbb9jwza3SXq/pB+IiL2SfmhIdR2abduyy2Zn\nmRYOIA1PPpldNk6TIYpaTPslnYuI5Yi4KulpSQ9u2ObHJP1pRLwqSRHx39VXc3gOHpQuX84uv3Sp\nvroAQJ5f/uXO6g9Zih7JMyqKgukOSRfWvX61+7P19kjaafuk7Rdt/2SVFRw2JjsAGCXnz3dm421m\ndVW6++566zMM0wXlZS7NM5K+TdL3SHqbpL+3/Q8R8crGDZeWlm5832q11Gq1Sld0GPKavnlNZgBo\n0vXr2dev5eVaq7Kpdrutdrvd9+/n3sdk+z5JSxGx0H39mKTViHhi3TaPStoWEUvd178v6ZMR8Scb\n9pXUfUxbt0pXr25exsoOAEbBqKwMUfV9TC9K2mN73vZWSQ9JOrFhmz+TdND2lO23SfpOSZ/ppdJ1\nW1zMDiWJUAIwGvLGm6aL+sMSlhtMEXFN0hFJz6oTNh+LiJdtH7Z9uLvNWUmflPRpSS9I+lBEJB1M\nPFsJwDg4fz671XT9euep26No4pYkymv6HjokHTtWX10AoAqpd+mxJFGO3buzy2ZmCCUAo2nc7m+a\nqBZT6p8qAKBfeRO6du3qrLnXFFpMGXgSLYBxlreeXtbq5KmaiBbT3FxnocPNbN8urazUWx8AGJYU\ne4ZoMW0iK5QkQgnAeNm+PbtsVB6RMfYtphQ/PQDAMKV23aPFtM7Ondlls7P11QMA6pQ3bj4Ks/TG\nusWU2qcGAKhLSrP0aDF1sUArgEk2yrP0xjKYDh7MLtuypfNMEwAYd488kl2WcpfeWHbl0YUHAB0p\nLMPWa1fe2AVTXr/q/Hxn0UMAmCRNf1if+GBq+h8AAFLT9CIDEx1MhBIAbK7J6+PEzsrLm/CQ8iAf\nANTh0KHsstSukWPTYqK1BAD5im6jGdaM5YnsysvrP92xg0elA8CaJj7ET2RXXt4irYQSANy0Jeeq\nnzckUqeRbzFNT3eebb+ZvXul06crf0sAGGl1t5omriuPsSUA6E3dwx8T1ZWXF0p5S3EAwCTLu2/p\njTfqq0eWkW4x0VoCgP7s3p29mOvsrHTpUnXvNTEtprxQynsWCQAg/7EXly/XV4/NjGyLidYSAAzm\n7rul5eXNy2Zm8h+d0YuJmPxAKAFANeq4nk5MVx4AYHB792aXbd1aXz3WG7kWE60lAKjWsK+rtJgA\nAD2Zn88ua6LVNFLBxEw8AKhe3gNUsx68OkyFwWR7wfZZ26/YfjRnu++wfc32D1ZbxXJarSbeFQDG\nw44d2WVzc/XVQyoIJttTko5KWpB0j6SHbb8jY7snJH1S0lCe7JG38CCrPADAYPKWIcpbKHsYilpM\n+yWdi4jliLgq6WlJD26y3S9K+hNJ/1Vx/W7IG4A7enRY7woAk2NqKruszpXHi4LpDkkX1r1+tfuz\nG2zfoU5YfaD7o8rnxm3bll22a1fV7wYAk+nateyyU6fqq8d0QXmZkHlK0q9FRNi2crrylpaWbnzf\narXUKjkwlLc8Rt6yGgCA6rTb5cbz2+222u123++Tex+T7fskLUXEQvf1Y5JWI+KJddv8u26G0e2S\nviLp5yPixIZ99XUf08GD2Uk9NZWf8ACA3lV9X1OlSxLZnpb0WUnfI+nzkv5R0sMR8XLG9h+R9ImI\n+PgmZX0FEzfUAkC9mg6m3DGmiLgm6YikZyV9RtLHIuJl24dtH+69egCA1OUtUzRdNABUgaSXJCq6\noZZ7lwBgOKpsNU3MkkSEEgAMT5NTx5NtMeU9XXH79vxHAwMABldVq2lsnsfEpAcAaFZTwTSyXXkA\ngOE6cCC7bJiTIJIMpryUfvLJ+uoBAJPs+eezy65fH977JtmVRzceAKShitnRI9+Vd+RIdlneAQIA\nVC+vMXD//cN5z+RaTLSWACAtg16XR77FBABIS933NCUVTPv2ZZfNztZXDwDATXU/DiOprjy68QAg\nTYNcn+nKAwBUbmYmuyyvt6sfyQQT3XgAkK4rV7LLzpyp9r2S6cqjGw8A0tbvdZquPADAUNQ1Oy+J\nYFpczC7LOxAAgPrUNTsvia48uvEAYDT0c72mKw8AMNIIJgBAaSdPZpdVtZ5p48G0JacGPOICANJS\nZjXxQTU+xsT4EgCMll6v24wxAQCGavv27LLduwfff6PBxDRxABg9KyvZZRcvDr7/Rrvy6MYDgNHU\ny/WbrjwAwEgjmAAAPdu7N7ts27bB9t1YMOWtq5S3vDoAoHmnT2eXXb482L4bG2NifAkARlvZ6zhj\nTACAkVYqmGwv2D5r+xXbj25S/uO2X7L9adunbH9L9VUFAKRkx47sskHuZyrsyrM9Jemzkr5X0uck\n/ZOkhyPi5XXbfJekz0TEl2wvSFqKiPs27IeuPAAYM2Wu5cPoytsv6VxELEfEVUlPS3rw1jePv4+I\nL3VfviDpzrwdEkoAgCxlgukOSRfWvX61+7MsPyfpmUEqBQCYXNMltindhrF9v6SflXRgs/KlpaV1\nr1rdLwDAeGlLauuWS34Pyowx3afOmNFC9/VjklYj4okN232LpI9LWoiIc5vs58YYE115ADAeFhel\n48c3L5ua6jyOfRhjTC9K2mN73vZWSQ9JOrF+A9tfr04o/cRmobReu122agCA1B07ll12/Xp/+yx1\ng63td0l6StKUpA9HxO/YPixJEfFB278vaVHSf3R/5WpE7N+wj4gIWksAMGaKruu9tphqX/mBYAKA\n8VJ1MLHyAwAgKQQTAGAgs7PZZfv29b4/uvIAAAPLu7ZLCXfl5SVnXuICACZHrS2mvHt1aS0BwOga\n2RYTAABFCCYAQFIIJgBAUggmAMDAHnmkun0x+QEAUInsCRAjOPlhZqbpGgAAUpFEMF250nQNAACp\nSCKYAABYQzABAJJCMAEAkkIwAQCSQjABAJJCMAEAKlHVTbZJ3GDLzbUAMB42v8l2BG+wBQBgDcEE\nAEhK48G0Y0fTNQAApKTxMSbGlwBgfDDGBAAYOwQTACApBBMAICkEEwAgKQQTACApBBMAICmFwWR7\nwfZZ26/YfjRjm/d1y1+yfW/11QQATIrcYLI9JemopAVJ90h62PY7NmzzgKS3R8QeSe+V9IEh1RUA\nMAGKWkz7JZ2LiOWIuCrpaUkPbtjm3ZI+KkkR8YKk22zvqrymAICJUBRMd0i6sO71q92fFW1z5+BV\nAwBMoumC8rILBm1caiLj95bWfd/S7Gyr5O4BAKNgdla6fLktqd33PoqC6XOS7lr3+i51WkR529zZ\n/dlbRCzJZn08ABhXly5JUktS68b13v4/Pe2jqCvvRUl7bM/b3irpIUknNmxzQtJPSZLt+yR9MSIu\n9lQLAAC6cltMEXHN9hFJz0qakvThiHjZ9uFu+Qcj4hnbD9g+J+nLkt6Tt8/Z2YpqDgBIWr/X+1of\ne1HXewEA0mHz2AsAwAgjmAAASSGYAABJIZgAAEkhmAAASSGYAABJIZgAAEkhmAAASSGYAABJIZgA\nAEkhmAAASSGYAABJIZgAAEkhmAAASSGYAABJIZgAAEkhmAAASSGYAABJIZgAAEkhmAAASSGYAABJ\nIZgAAEkhmAAASak9mNrtdt1viS6OfXM49s3h2Den32NPME0Qjn1zOPbN4dg3Z2SCCQCAPAQTACAp\njoh63siu540AAMmJCJfdtrZgAgCgDLryAABJIZgAAEkhmAAASRl6MNn+Ydv/avu67W/L2W7B9lnb\nr9h+dNj1mgS2d9p+zva/2f5L27dlbLds+9O2P2X7H+uu5zgpcx7bfl+3/CXb99Zdx3FVdOxtt2x/\nqXuef8r2bzZRz3Fj+w9sX7R9Omebns75OlpMpyUtSvrbrA1sT0k6KmlB0j2SHrb9jhrqNu5+TdJz\nEfGNkv66+3ozIakVEfdGxP7aajdmypzHth+Q9PaI2CPpvZI+UHtFx1AP15C/6Z7n90bE/621kuPr\nI+oc9031c84PPZgi4mxE/FvBZvslnYuI5Yi4KulpSQ8Ou24T4N2SPtr9/qOSDuVsW3oqJzKVOY9v\n/JtExAuSbrO9q95qjqWy1xDO84pFxN9JeiNnk57P+VTGmO6QdGHd61e7P8NgdkXExe73FyVlnQwh\n6a9sv2j75+up2lgqcx5vts2dQ67XJChz7EPSd3e7k56xfU9ttZtsPZ/z01W8q+3nJO3epOjXI+IT\nJXbBzVR9yjn2v7H+RUREzk3OByLiC7a/RtJzts92PwWhN2XP442f2jn/B1fmGP6zpLsi4iu23yXp\nuKRvHG610NXTOV9JMEXE/x5wF5+TdNe613epk6ookHfsuwOSuyPiNdtfK+k/M/bxhe5//8v2MXW6\nRQim3pU5jzduc2f3ZxhM4bGPiJV13/+F7d+zvTMiXq+pjpOq53O+7q68rP7dFyXtsT1ve6ukhySd\nqK9aY+uEpJ/ufv/T6nxCvIXtt9me637/vyR9nzoTVtC7MufxCUk/JUm275P0xXXdrehf4bG3vcu2\nu9/vV2flG0Jp+Ho+5ytpMeWxvSjpfZJul/Tntj8VEe+y/XWSPhQR3x8R12wfkfSspClJH46Il4dd\ntwnwu5L+2PbPSVqW9COStP7Yq9MN+PHu3+u0pD+MiL9sprqjLes8tn24W/7BiHjG9gO2z0n6sqT3\nNFjlsVHm2Ev6IUm/YPuapK9I+tHGKjxGbP+RpHdKut32BUmPS5qR+j/nWSsPAJCUVGblAQAgiWAC\nACSGYAIAJIVgAgAkhWACACSFYAIAJIVgAgAk5f8D14MPDd8Rp1oAAAAASUVORK5CYII=\n",
       "text": [
        "<matplotlib.figure.Figure at 0x106a482d0>"
       ]
      }
     ],
     "prompt_number": 25
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "I then get the remaining sum of squares after projecting onto each of these unit vectors:"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "remaining_ss = []\n",
      "for u in u_vectors.T: # iterate over columns\n",
      "    remaining = line_remaining(u, X)\n",
      "    remaining_ss.append((remaining ** 2).sum())"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [],
     "prompt_number": 26
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "plt.plot(angles, remaining_ss)\n",
      "plt.xlabel('Angle of unit vector')\n",
      "plt.ylabel('Remaining sum of squares')"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 27,
       "text": [
        "<matplotlib.text.Text at 0x106b2eb50>"
       ]
      },
      {
       "metadata": {},
       "output_type": "display_data",
       "png": "iVBORw0KGgoAAAANSUhEUgAAAYgAAAEPCAYAAABY9lNGAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3XecVPX1//HXkaISbKixotixIGJBDSprjRUwGKygYKxf\nRQ0/FdQIGjWKvREjsaCJWJAYsKGii6gBRFG6CEoCFowRFGy08/vjc1fXdZad2Zk7987s+/l47GNn\n7ty593CBOfMp93zM3REREalptaQDEBGRdFKCEBGRjJQgREQkIyUIERHJSAlCREQyUoIQEZGMYksQ\nZna/mS0wsyk1tp9vZjPMbKqZ3VBtez8ze9/MZprZYXHFJSIi2Wkc47EfAO4EHqraYGYHAp2AXd19\nmZltGG3fCTge2AnYDHjJzLZ395UxxiciIqsQWwvC3ccCC2tsPgf4k7svi/b5b7S9MzDU3Ze5+1xg\nNtA+rthERKRuxR6D2A44wMzGmVmlme0Zbd8UmF9tv/mEloSIiCQkzi6m2s63nrvvY2Z7AY8DW9ey\nr2qAiIgkqNgJYj4wHMDd3zSzlWa2AfAR0LLafptH237CzJQ0RETqwd0t1/cUu4vpKeAgADPbHmjq\n7p8DI4ATzKypmW1F6IqakOkA7l6yP/379088BsWffByKv/R+Sjl29/p/r46tBWFmQ4GOwPpmNg+4\nErgfuD+a+roU6AHg7tPN7HFgOrAcONfz+VOJiEjeYksQ7n5iLS91r2X/64Dr4opHRERyozupi6ii\noiLpEPKi+JOl+JNTyrHnw0qpJ8fM1PMkIpIjM8NLYJBaRERKhBKEiIhkpAQhIiIZKUGIiEhGShAi\nIpKREoSIiGSkBCEiIhkpQYiISEZKECIikpEShIiIZKQEISIiGSlBiIhIRkoQIiKSkRKEiIhkpAQh\nIiIZxbainIgU3sKF8N57MGcOfPQRfP01fPcdNGsGa68Nm20G228ffpo1SzpaKXVaMEgkxb7/HkaP\nhhEjYOxY+M9/YMcdYeutYfPNoXlzWGMN+OYb+OormDcPZs2CDz6ANm2gY0fo3Bn22QdWU39Bg1Xf\nBYNiSxBmdj9wFPCZu7ep8Vof4EZgA3f/ItrWD+gFrAB6u/sLGY6pBCENwuTJcNddMGwY7LwzHHts\n+LBv2xYaZ9Hu//ZbGD8eXn45HGPJEujeHc49N7QypGFJY4LYH1gCPFQ9QZhZS2AwsAOwh7t/YWY7\nAY8AewGbAS8B27v7yhrHVIKQsjZmDAwYAO+/D2efDT175v+B7g5Tp8K998Lf/w5HHAFXXgk77FCQ\nkKUEpG7JUXcfCyzM8NItwCU1tnUGhrr7MnefC8wG2scVm0jaTJoEv/51SAi9esGHH8IVVxTm275Z\n6G66885w3DZtYL/94IwzwjiGSG2K2itpZp2B+e4+ucZLmwLzqz2fT2hJiJS1JUvgoovg8MPDWMHM\nmaErqEmTeM63zjrQt28Yp2jRAnbbLXRlrVgRz/mktBUtQZhZM+AyoH/1zat4i/qSpKyNHg077RRm\nJk2dGsYHmjYtzrnXWw9uuCF0aT32GHToEJKGSHXFnOa6DdAKeNfMADYH3jKzvYGPgJbV9t082vYz\nAwYM+OFxRUUFFRUVsQQrEpdly8IYwEMPwQMPwGGHJRfLTjuFJPHnP4ckcdNN0KNH6JaS0lVZWUll\nZWXex4l1mquZtQJG1pzFFL32IT8fpG7Pj4PU29YckdYgtZS6jz6Crl1D986DD8Ivf5l0RD+aMgWO\nPx723BP+8hdYc82kI5JCSd0gtZkNBd4AtjezeWbWs8YuP3zSu/t04HFgOvAccK4ygZSbN9+EvfeG\nTp3g6afTlRwgDF5PnBhaOB07wscfJx2RJE03yokUwWOPwXnnweDB0KVL0tGsmjtcdx3ccw/885+w\n++5JRyT5St19EHFQgpBSdNddYUB45Mgwa6hUPPkknHMOPP44aKivtClBiKSMO1x7bRhrePFF2Gqr\npCPK3SuvhHGJv/41dI1JaapvglCxPpEYuMMll8CoUaGG0iabJB1R/Rx4IDzzDBxzTCgMeOKJSUck\nxaQEIRKDyy8P9zlUVoYZS6Vsr73gpZfgkEPCfRpduyYdkRSLEoRIgV17bai+Wg7Jocouu8Bzz4U7\nvldfHY4+OumIpBhUAFikgG65BYYMCWMOG2yQdDSF1a5dGGjv1Su0KKT8KUGIFMjf/w633x66lkp1\nzKEu7duH8uEnnQTvvpt0NBI3JQiRAqisDEX3nnkGWrasc/eSdsABYeru0UeHBYqkfGkMQiRPM2aE\nqaBDh4a++oagW7eQHI44Al57DdZdN+mIJA66D0IkDwsWhOU8BwyAU09NOpricocLLggJ8rnnslvp\nTpKhG+VEimzZMjj44NDlcs01SUeTjOXL4cgjQx2nm29OOhqpTeqK9YmUuz59YO214eqrk44kOY0b\nw6OPhppNf/tb0tFIoalRKFIPQ4bA88/DhAmwWgP/mtWiBTz1VLjrescdYY89ko5ICkVdTCI5euut\nMDhbWRkW3JFg+PAwk+vtt2H99ZOORqorSheTmbUws11zPYlIuVi0CI47LqzApuTwU7/5Dfz2t2Gw\nXt/jykOdLQgzGwMcQ+iOegv4L/C6u18Uf3g/i0UtCEmMe/gA3HRTuOOOpKNJp2XLYP/9w3Xq0yfp\naKRKnC2Iddz9K+A3wEPu3h44JNcTiZS6e++FOXNg4MCkI0mvJk3CoPXAgTBuXNLRSL6ySRCNzGwT\noBvwTLRNX+OlQZkyBa64Inz4rbFG0tGkW6tWYU3rE06AhQuTjkbykU2CuBoYBcxx9wlmtg3wfrxh\niaTH11+HO6Vvvhl22CHpaEpDly5hgaGzztJ4RCmLbRaTmd0PHAV85u5tom03AkcDS4E5QE93/zJ6\nrR/QC1gB9Hb3FzIcU2MQUnTnnguLF8PDDycdSWn59lvYc0+47DI4+eSko2nYYhuDMLMdzGy0mU2L\nnu9qZldkcewHgMNrbHsB2Nnd2wKzgH7RMXcCjgd2it4zyMwa+OxySYMXXoCnn4Y770w6ktKz5prh\n5rkLL1RRv1KVzYfwYOAywrd+gClAnQsPuvtYYGGNbS+6+8ro6Xhg8+hxZ2Couy9z97nAbKB9FrGJ\nxGbhQjj9dLj/fhWjq6927UKCOO00WLmyzt0lZbJJEM3cfXzVk6iPZ1kBzt0LeDZ6vCkwv9pr84HN\nCnAOkXrr3Rs6dw5LbUr9XXpp6G5SK6z0ZFNq479mtm3VEzM7Dvgkn5Oa2eXAUnd/ZBW7ZRxsGDBg\nwA+PKyoqqKioyCcUkYyGD4fx42HSpKQjKX2NG8NDD4Wqt4cdFspxSLwqKyuprKzM+zjZ3Ci3DXAv\nsC+wCPgQODnqCqrrva2AkVWD1NG204AzgIPd/btoW18Ad78+ev480L96yyXarkFqid1nn0HbtiFJ\n7Ltv0tGUj7vvhkcegbFjVb+q2GIZpDazRsA57n4w8Eugtbt3yCY51HK8w4GLgc5VySEyAjjBzJqa\n2VbAdsCE+pxDJF+9e0OPHkoOhXbOOWAGgwYlHYlkK5sWxDhg31y/upvZUKAjsAGwAOhPmLXUFPgi\n2u1f7n5utP9lhHGJ5cAF7j4qwzHVgpBYPf10GFSdMiXMwpHCeu896NAhFDzccsuko2k4YlswyMzu\nIQwiPwF8E212dx+ec5R5UoKQOC1eDDvvDA8+CAcdlHQ05etPf4IxY8IqdJbzR5bUR5wJ4sHo4U92\ndPeeuZ4sX0oQEqfevWHJkjCtVeKzbBm0bx9Kg/fokXQ0DYOWHBXJw7/+FcpVT5sWFsCReL39Nhx+\neOjK22ijpKMpf3G2INYETifc5bwmUUvC3XvVI868KEFIHJYuhd13hz/8IdRckuK45BL45BOVMCmG\nOMt9PwxsRCiBUQm0BJbkeiKRtBo4MFQg7dYt6UgaliuvDGMRBZiuLzHJpgXxjrvvZmaT3X1XM2sC\nvObuexcnxJ/EohaEFNSHH4aCcm+9FZKEFNc//gGXXw7vvANNmyYdTfmKswVRVYPpSzNrA6wLbJjr\niUTS6MIL4fe/V3JISpcusNVWcMstSUcimWTTgjgDeBJoAzwINAf+4O73xB7dz2NRC0IK5plnQoKY\nOhVWXz3paBquDz4Is5p0b0R8NItJJAfffRfuebj77jCbRpJ1zTUwcSI89VTSkZSnOGcx9a/29Ied\n3f3qXE+WLyUIKZSrr4Z334Unn0w6EgH4/nvYdVe46SY45pikoyk/9U0Q2VRz/ZofE8OahBXhpud6\nIpG0+OADuOOOMBdf0mH11eGuu+Dss+HQQ7Xud1rk3MVkZqsDL7h7x3hCWuW51YKQvHXqFArx9euX\ndCRSU5cuoSx4375JR1JeijYGYWYtgAnuvm2dOxeYEoTk65lnQomHKVM0MJ1Gs2eHBDF5Mmy6adLR\nlI84xyCmVHu6GqHs99XuXvT1oZQgJB/LlkGbNnDzzXDUUUlHI7Xp2xc+/TQUTZTCiDNBtKr2dDmw\nwN0LseRozpQgJB933BFaEM8/ryqiabZ4MbRuHW6ia6+V6QsizgSxytJl7v7Fql4vJCUIqa8vvggf\nOi+/DLvsknQ0UpchQ+DPf4Y33tDqc4UQZ4KYC2wBLIw2rQf8hzCzyd1961xPWl9KEFJfF1wQupi0\nmllpWLkyjEWcfz507550NKUvzgQxGPiHuz8bPT8CONbdz6xXpHlQgpD6mDEDDjgApk+HDVUkpmT8\n619w3HFhFbrmzZOOprTFmSCmuvsudW0rBiUIqY+jjgorxPXpk3Qkkqvu3UP5jWuuSTqS0hZngngB\neBX4G2DAScAB7v7r+gSaDyUIydWoUXDeeWEhIFULLT3z5sFuu4W73jffPOloSlec1VxPJExt/Qcw\nPHp8YhYB3W9mC6pPkzWzFmb2opnNMrMXzGzdaq/1M7P3zWymmR2W6x9EpKbly0Ol1ptuUnIoVS1b\nwplnhsWcpPhyulHOzBoBzd39yyz23Z+wsNBD7t4m2jYQ+NzdB5rZpcB67t7XzHYCHgH2AjYDXgK2\nd/eVNY6pFoRkbdAgGDYMRo/WtNZS9uWXsMMOoTXYtm3S0ZSm2FoQZjbUzNY2s18AU4DpZnZJXe9z\n97H8OPOpSidgSPR4CNAletwZGOruy9x9LjAb0AxoqbclS+CPfww3xSk5lLZ11oErroCLL046koYn\nmy6mndz9K8KH+XNAK6C+E882cvcF0eMFhKVMATYF5lfbbz6hJSFSL7ffDh07Qrt2SUcihXDWWTB3\nbmhFSPFkU821cbTMaBfgbndfZmZ59/O4u9dxnIyvDRgw4IfHFRUVVFRU5BuKlJkvvoBbbw3TJKU8\nNGkCN9wQWhGHHAKNGiUdUbpVVlZSWYDFvrOZxdQbuBSYDBxFuGnuYXffv86DhzIdI6uNQcwEKtz9\nUzPbBHjF3VubWV8Ad78+2u95oL+7j69xPI1BSJ0uvRQWLoR77006Eikk93A/S8+e0KtX0tGUlmJW\nczWgkbsvz2LfVvw0QQwE/ufuN0RJYd0ag9Tt+XGQetua2UAJQury8cehlMbkyZoWWY7Gj4euXcPN\nc7/4RdLRlI7ULTlqZkOBjsAGhPGGK4F/Ao8TWiFzgW7uvija/zKgF6Eg4AXu/rPeRiUIqcs554QP\njptuSjoSicsJJ4TlYjX1NXupSxBxUIKQVZkzB/beG2bOhA02SDoaicsHH8Bee4XSKRttVPf+EsM0\nVzP7bfS7aMX4RPJx5ZXQu7eSQ7nbems45RS47rqkIyl/tbYgzGySu7er+l3kuDJSC0JqM3kyHHYY\nvP8+rLVW0tFI3D77DHbcEd56C1q1Sjqa9Ct4F5OZvUSYaroXMLbGy+7unXKOMk9KEFKbY46Bgw+G\nCy9MOhIplgEDQnfTQw8lHUn6xZEgmgK7E4r0nU4o1FfF3X1MfQLNhxKEZPL663DSSWFmyxprJB2N\nFMtXX8F228FLL4WlZKV2cVZz3dDd/2tmzQHcfUk9Y8ybEoTU5A4VFXDqqZob3xDddltYJXDEiKQj\nSbc4q7lubGaTgOmEOkxvmZkWbZRUGDUq9Ef36JF0JJKEs88OpcBffz3pSMpTNgniXuD37r6Fu28B\n9Im2iSRq5Uq47LJQlK9xNkVjpOyssQZcdRX07Rtak1JY2SSIZu7+StUTd68EdA+jJG7YsLCgfdeu\nSUciSerePdTfeu65pCMpP9kkiA/N7A9m1srMtjKzK4AP4g5MZFWWLw930l53ncp5N3SNGsG110K/\nfqFVKYWTTYLoRVhFbjjwJLBhtE0kMQ8+CJtuCocemnQkkgadO0OzZvDoo0lHUl5UakNKznffhemN\nTzwB++yTdDSSFmPGhJlsM2Zoidma4pzFJJIqgwbB7rsrOchPdewI228PgwcnHUn5UAtCSkrVzVGj\nR4ey3iLVTZoERx4Js2erHHh1akFIg3DLLaHmkpKDZNKuXWhJ3H570pGUh2zupN4aOJ+wFnXVbHPV\nYpKi++9/oXVrePPNUNFTJJNZs+BXvwq/W7RIOpp0iLPUxmTgr8BUoGoSmWoxSdH9/vfw/fdw991J\nRyJpd8YZsP76cP31SUeSDnEmiAnu3r7ekRWQEkTDNW8e7LYbTJ0Km2ySdDSSdvPnQ9u2+vdSJc4E\n0R3YBhgFfF+13d3fzvVk+VKCaLh+9zvYcEP405+SjkRKRZ8+YUq0WpzxJojrge7AbH7sYsLdD8z1\nZPlSgmiY3nsP9tsv9Cmvt17S0Uip+PzzMGY1YYLGrOJMEHOAHd19aX2Dy3DMfsAphIQzBehJqO/0\nGLAlMBfo5u6LarxPCaIB6tYt3PfQt2/SkUipueqqMOX14YeTjiRZcSaIp4Cz3H1BfYOrcbxWwMuE\npPO9mT0GPAvsDHzu7gPN7FJgPXfvW+O9ShANzNtvw9FHh//kzZolHY2UmsWLw30zL77YsBcVivM+\niPWAmWb2gpmNjH7yWZ7jK2AZ0MzMGgPNgI+BTsCQaJ8hQJc8ziFl4rLL4IorlBykftZaCy69NPwb\nktxl04KoyLQ9Kvtdv5OanQncDHwLjHL37ma20N3Xi1434Iuq59XepxZEAzJmDPTsCTNnqraO1F9V\n7a7HH4d99006mmTUtwVR5zIr+SSCTMxsG+BCwo13XwJPmNkpNc7pZpYxEwwYMOCHxxUVFVRUVBQy\nPEkJ91C++eqrlRwkP2usAf37h9boyy83jPLwlZWVVFZW5n2cbFoQS4CqnZoCTYAl7r52vU5odjxw\nqLv/LnreHdgHOAg40N0/NbNNgFfcvXWN96oF0UCMHBn+Q7/zTqj3L5KP5cth553hrrsaZon42MYg\n3L25u6/l7msBawK/AQbVI8YqM4F9zGzNqCvpEMJ61yOBU6N9TgWeyuMcUsJWroTLLw+LwCg5SCE0\nbhyWpr3sMi1NmoucivW5+0p3fwo4vL4ndPd3gYeAicDkaPO9wPXAoWY2i9Ca0E3yDdTQodC8ORxz\nTNKRSDk57jhYsQKGD086ktKRTRdT9RV/VwP2ADq6e9GHe9TFVP6WLoUdd4T77gMNL0mhPf88XHQR\nTJkSWhUNRWyD1MAx/DgGsZxwE1vnXE8kko377oNtt1VykHj8+tfwy1+GG+d69kw6mvTTgkGSGt98\nE6YjjhgBe+yRdDRSrl5/HU46KZRuWX31pKMpjtgGqc1soJmtbWZNzGy0mX0ezTwSKag77wx1/JUc\nJE4dOoS7qu+5J+lI0i+bMYh33b2tmR0LHA38Hhjr7rsWI8AasagFUaYWLQqth7FjQ4E1kTi9+27o\nbpo9O0yIKHdxltqoGqc4Ghjm7l/y45iESEHceCN06qTkIMXRti0cdBDcdlvSkaRbtuW+uwDfAe2B\ndYGR7r53/OH9LBa1IMrQp5+Gm5gmTYIttkg6GmkoZs+GffYJ5eTXXz/paOIVWzXX6ODrA4vcfYWZ\n/QJYy90/rUeceVGCKE/nnx+mHN56a9KRSENz9tmw9towcGDSkcQr1gSRFkoQ5efDD2HPPUNBvg03\nTDoaaWg++igMWE+ZApttlnQ08VGCkJJ06qnQqlVY2EUkCRdfHNaNKOdZTUoQUnKmTQsDhe+/H5r5\nIkn43/9ghx1g3Lhwk2Y5inNFuT34+aylL4F/u/vyXE+YDyWI8nLssWGt6T59ko5EGro//hFmzIBH\nHkk6knjEmSDGEeovVRXWawNMA9YBznH3UbmetL6UIMrH+PGheNqsWbDmmklHIw1d1dKko0aFKbDl\nJs77ID4GdnP3Pdx9D2A34APgUKDMx/4lDu6h7PKVVyo5SDqstVZYoEpLk/5UNgliB3efVvXE3acD\nrd19DrphTurhpZdg/nwVS5N0OftsmDw51GqSIJsEMc3M/mxmHc2swswGAdPNbHVgWczxSZlZuTJ8\nU7vmmoZVblnSb/XVYcAALSpUXTYJ4jRgDmEd6QsI3UunEpLDQbFFJmVp2LDw+7jjko1DJJPu3eGz\nz8JYhGiaqxTRsmWhpMagQXDIIUlHI5LZsGFw3XUwcSKsltOam+kVZ7nv/czsRTN738w+jH4+qF+Y\n0pA98ECotaTkIGnWtWtIDFWt3YYsm2mu7xG6l94GVlRtd/fP4w0tYyxqQZSoqsWAnnoK9tor6WhE\nVu2FF0KNsGnTymOsLM5provc/Tl3X+Dun1f91CPGH5jZumY2zMxmmNl0M9vbzFpELZVZZvaCma2b\nzzkkXaoWA1JykFJw6KGw6aYwZEjSkSQr23LfjYDhwPdV29397Xqf1GwIMMbd7zezxsAvgMuBz919\noJldCqzn7n1rvE8tiBK0cCFsvz289looaSBSCsaNg27dws2ca6yRdDT5ifNO6koy3O/g7gfmerLo\neOsAk9x96xrbZwId3X2BmW0MVLp76xr7KEGUoL59Q72bwYOTjkQkN507Q0UFXHRR0pHkp2SK9ZnZ\nbsBfgOlAW+AtwhjHfHdfL9rHgC+qnld7rxJEifnoI9h117DE4+abJx2NSG6mTAmTKmbPDndbl6r6\nJohah1/MrLu7P2xmffhpC8IAd/db6hFn1Tl3B85z9zfN7DbgJ11J7u5mljETDBgw4IfHFRUVVFRU\n1DMMKYY//hFOP13JQUpTmzZhPOKWW6B//6SjyV5lZSWVlZV5H6fWFoSZneXufzGzAWROEPWq4B91\nH/3L3beKnu8H9AO2Bg5090/NbBPgFXUxlbZZs8LA9KxZ0KJF0tGI1M+cOdC+fViadIMNko6mfkqm\niwnAzF4Ffufus6IE1Cx66X/ufoOZ9QXW1SB1afvtb6Fdu1C6QKSUnXtuKCx5881JR1I/cQ5S/xI4\nA2jFj11S7u69cj1ZtWO2Bf4KNCWU8ehJmCn1OLAFMBfo5u6LarxPCaJEvPEGnHBCWEq0WbO69xdJ\ns08+CVUAJk8uze7SOBPEv4BXCYPJK6PN7u5P5hxlnpQgSoM7dOgAZ50VlhQVKQd9+8IXX8C99yYd\nSe7iTBDvuPtu9Y6sgJQgSsOwYXDttaGWTaNGSUcjUhhffBHu53njjfC7lMR5J/XTZnZUPWKSBmjp\n0vBN68YblRykvLRoEe6HuPLKpCMpnmxaEEsIg8hL+XH9B3f3oi8zrxZE+t1+eyiV/OyzSUciUnhL\nloSaYk8/DXvskXQ02SupWUz1pQSRbosWhab3yy/DLrskHY1IPAYPhocfhjFjwHL+yE1GwROEme3o\n7jPMbPdMr+dTi6m+lCDS7ZJLQt0lldSQcrZiBey+e+hq6to16WiyE0eCGOzuZxS6FlM+lCDSa+7c\n0OSeOhU22STpaETiNXo0nHkmTJ8elipNO3UxSaJOOil0L1WrhCJS1jp1gv33h4svTjqSusWaIMys\nDbAj8EPRW3d/KNeT5UsJIp3GjQtN7ffeg+bNk45GpDiqSslMnw6//GXS0axanPdBDAA6AjsDzwBH\nAK+5e9GXnVeCSJ+VK2GffeC886BHj6SjESmuCy+E776De+5JOpJVi/M+iOOAQ4BP3L0noUS3VnsT\nIMzmWG01OOWUpCMRKb4rr4Thw0NZ8HKUTYL41t1XAMujxX4+A1rGG5aUgsWLoV+/cO/Datn8SxIp\nMy1awBVXQJ8+ocRMucnmv/WbZrYeMBiYCEwC3og1KikJ11wDhx0Ge++ddCQiyTnnHPjPf8rz5tCc\nZjGZ2VbAWu4+Ob6QVnl+jUGkxPvvw777hqa1prVKQ/fss3DBBWGadxqnvcY9i6ktodx3I35cMGh4\nrifLlxJEenTqFCq2Xnpp0pGIpEOnTmHCRhrXP4lzFtMDQBtgGj+W+yYasC4qJYh0GDUK/u//YNq0\ndH5bEknChx/CXnvB22/DFlskHc1PxZkgpgM7p+GTWQkieUuXQtu2cMMN4RuTiPzoqqtCt+uwYUlH\n8lNxTnN9E9gp95CkHN1yC2yzDRxzTNKRiKTPJZfApEnw4otJR1IY2bQgKoARwKfA99Fmd/dd4w0t\nYyxqQSRo7lzYc0+YMAG23jrpaETSaeTIUH5j8mRo2jTpaII4u5jmABcBU/npGMTcXE+WLyWIZHXq\nFKa0Xn550pGIpNvRR4c6TWmZxBHrmtTuvm+9I6v9uI0I91XMd/djzKwF8BiwJTAX6Obui2q8Rwki\nISNGhObzu+9qYFqkLnPmhC9TkyZByxTcVhznGMQkM3vEzE40s67Rz2/qEWNNFwDT+bGUeF/gRXff\nHhgdPf+Zd94pwJklJ19/Db17w6BBSg4i2dhmm3BfxMsvJx1JfrJpQTwYPfzJjvlMczWzzYEHgWuB\n30ctiJlAR3dfYGYbA5Xu3rrG+3zPPZ1x47TecTH17Qvz5sHf/550JCJSH/VtQTSuawd3P61eEa3a\nrcDFQPV1rTdy9wXR4wXARpneuNZacMcdYfFwid+0aXDffeVbjExEaldngjCzHYBBwMbuvrOZ7Qp0\ncvdr6nNCMzsa+MzdJ0UzpH7G3d3MMjZtdt55AJdfDv/+N3TpUkFFRcZDSAGsWAGnnw5//CNsvHHS\n0YhItiorK6msrMz7ONl0Mb1K+LZ/j7u3MzMDprr7zvU6odl1QHdgOWEBorWB4cBeQIW7f2pmmwCv\nZOpicneCXC3CAAAPEElEQVT+9Cd49dVQ/6RUFg0vRbfeCv/8Z+hHVbVWkdIV5yB1M3cfX/Ukmka0\nLNcTVXv/Ze7e0t23Ak4AXnb37oR7LU6NdjsVeKq2Y/y//wcffwxDh9Y3CqnLnDlw7bXw178qOYg0\nVNn81/+vmW1b9cTMjgM+KWAMVU2Y64FDzWwWcFD0PKMmTWDw4FCD/X//K2AkAoS69mecEQant922\n7v1FpDxl08W0DXAvsC+wCPgQODkNN8r16QMffQSPPlrsSMrb4MHh5403oHGdo1QiknaxlvuOTtCc\nUOp7CeEmtsdyPVm+aiaIb7+F3XeHAQPg+OOLHU15mj8f2rUL4w5t2iQdjYgUQsHHIMysuZn1MbNB\nZnYu8A1hbeppwMn1D7Vw1lwThgwJN3F9UshOrwZq5Uro1QvOP1/JQURW0YIws+HAV8C/gMMI61B/\nB/R290TuZ66t1MYf/hDusB4xQrOa8nHnneFmuNdeU9eSSDkpeBeTmU2uqtga1U36BNjS3b/NK9I8\n1JYgli6F9u1DS6JXrwQCKwMzZsABB4Rxh+22SzoaESmkOKa5rqh64O4rgI+STA6r0rQpPPRQqJw4\nd27S0ZSepUvhlFPgmmuUHETkR6tqQawgjDtUWROoShDu7mv//F3xqqua6003wfDhMGZMmAor2fnD\nH8IyiU8/rS46kXIU+yymNKgrQaxcCUcdFWbhXHddEQMrYWPGwAknhLLEKqchUp6UICKffRYSxJAh\ncMghRQqsRH32WZgmfN998OtfJx2NiMRFCaKal16CU08N3SYbZawJKytXwhFHhCVEr7026WhEJE5x\n1mIqOYccAqedBj16hA9C+bnrrw83Gl51VdKRiEhalWULAmD5cjj4YOjYEa6+OubASsyrr0K3bjBx\nImy+edLRiEjc1IKooXFjePxxePBBeKrWurANz7x5YVB6yBAlBxFZtbJtQVSZMCHMbBo7Flq3rnv/\ncvbNN7D//iFBXHxx0tGISLFokHoV7rsPbrwxJIu1i373Rjq4h5vh3EM5Dd3vINJwKEHU4Zxzwl3W\nI0c2zDpDN90EjzwS6iw1a5Z0NCJSTBqDqMMdd4Rvz//3f+F3QzJsWFg+9KmnlBxEJHsNJkE0aQJP\nPAHjx8PAgUlHUzyvvRZaT08/DVtskXQ0IlJKGlRny1prwTPPwL77wpZbhsHacjZzJnTtGsYc2rVL\nOhoRKTUNKkEAbLZZ+DZ96KFhwPrII5OOKB7z5oU7pW+4AQ47LOloRKQUFb2LycxamtkrZjbNzKaa\nWe9oewsze9HMZpnZC2a2blwx7LprWFzotNNg9Oi4zpKcjz+Ggw4KK8OddlrS0YhIqSr6LCYz2xjY\n2N3fida5fgvoAvQEPnf3gWZ2KbCeu/et8d56z2LKZMwYOO64MHjboUPBDpuozz4Ld4/36AH9+iUd\njYikQcnMYnL3T6uWLHX3JcAMYDOgEzAk2m0IIWnEqmPH0D9/7LEhWZS6Tz8N5UWOP17JQUTyl+gs\nJjNrBbQDxgMbufuC6KUFQFHqsB52GDz6aGhJPPNMMc4Yjw8/hP32CzWW+vdPOhoRKQeJDVJH3UtP\nAhe4+2Krdmuvu7uZZexLGjBgwA+PKyoqqKioyDuWgw4KA9edOsFtt8GJJ+Z9yKKaOjUMSPfrB+ee\nm3Q0IpK0yspKKisr8z5OIndSm1kT4GngOXe/Ldo2E6hw90/NbBPgFXdvXeN9BR2DqGnKlFC36cwz\n4fLLS6McxYsvhhIat94KJ52UdDQikkYlMwZhoalwHzC9KjlERgCnRo9PBYpeg7VNm3Aj3YgRcPLJ\nYb2EtHIPd4f36BFuAFRyEJFCS2IW037Aq8BkoOrk/YAJwOPAFsBcoJu7L6rx3lhbEFW+/RZ69oTZ\ns8P4xLbbxn7KnHz9NZx3XljPYcQI2GqrpCMSkTRTsb4Cc4e77gqLDd16a+jGSYNJk8Id4PvuC3fe\nGe4OFxFZFSWImLz7bpg22rYt3H47bLxxUU//g6VLQw2p228PP+pSEpFslcwYRKlp2xbefhu23jrc\ngX3vvbBiRXFjGDsWdtstjI9MnKjkICLFoRZEDqZMCdNIFy6Ea66Bzp3jnek0ZUqYTfXOO3DLLaHw\nXinMrBKRdFEXU5G4hxvqLr88LDx04YXh5rTVVy/c8ceMCTOUXn8d+vYN5brXWKMwxxeRhkcJoshW\nroRnnw0f5JMnw29/G346dIBGjXI/3nvvhemqjz4aurDOPz9MYW3evPCxi0jDogSRoFmz4PHHwwf8\nvHkhSfzqV9C6dZgiu+GG8ItfhBbH4sWwaBF88AG8/35YJ3vsWPj++1Duo1u3UDJDXUkiUihKECnx\nySeha2jcuJAAZs+G//0PliyB5ctDi2CddcK9C9ttFxbyOeAA2GEHJQURiYcShIiIZKRpriIiUlBK\nECIikpEShIiIZKQEISIiGSlBiIhIRkoQIiKSkRKEiIhkpAQhIiIZKUGIiEhGShAiIpJRqhKEmR1u\nZjPN7H0zuzTpeEREGrLUJAgzawTcBRwO7AScaGY7JhtVYVVWViYdQl4Uf7IUf3JKOfZ8pCZBAO2B\n2e4+192XAY8CnROOqaBK/R+Z4k+W4k9OKceejzQliM2AedWez4+2iYhIAtKUIFTHW0QkRVKzHoSZ\n7QMMcPfDo+f9gJXufkO1fdIRrIhIiSnpBYPMrDHwHnAw8DEwATjR3WckGpiISAPVOOkAqrj7cjM7\nDxgFNALuU3IQEUlOaloQIiKSLmkapP5BNjfMmdkd0evvmlm7Yse4KnXFb2YVZvalmU2Kfq5IIs5M\nzOx+M1tgZlNWsU+ar/0q40/ztQcws5Zm9oqZTTOzqWbWu5b9Uvd3kE3sab7+ZraGmY03s3fMbLqZ\n/amW/VJ37SG7+HO+/u6eqh9C99JsoBXQBHgH2LHGPkcCz0aP9wbGJR13jvFXACOSjrWW+PcH2gFT\nank9tdc+y/hTe+2j+DYGdoseNyeMy5XEv/8sY0/79W8W/W4MjAP2K4Vrn0P8OV3/NLYgsrlhrhMw\nBMDdxwPrmtlGxQ2zVtne8JfzjIJicPexwMJV7JLma59N/JDSaw/g7p+6+zvR4yXADGDTGrul8u8g\ny9gh3df/m+hhU8KXvS9q7JLKa18li/ghh+ufxgSRzQ1zmfbZPOa4spVN/A78KmqiPmtmOxUtuvyl\n+dpno2SuvZm1IrSGxtd4KfV/B6uIPdXX38xWM7N3gAXAK+4+vcYuqb72WcSf0/VPzSymarIdNa+Z\nBdMy2p5NHG8DLd39GzM7AngK2D7esAoqrdc+GyVx7c2sOTAMuCD6Nv6zXWo8T83fQR2xp/r6u/tK\nYDczWwcYZWYV7l5ZY7fUXvss4s/p+qexBfER0LLa85aELL2qfTaPtqVBnfG7++KqpqC7Pwc0MbMW\nxQsxL2m+9nUqhWtvZk2AJ4G/uftTGXZJ7d9BXbGXwvUHcPcvgWeAPWu8lNprX11t8ed6/dOYICYC\n25lZKzNrChwPjKixzwigB/xwB/Yid19Q3DBrVWf8ZraRmVn0uD1hunGmvsI0SvO1r1Par30U233A\ndHe/rZbdUvl3kE3sab7+ZraBma0bPV4TOBSYVGO3VF57yC7+XK9/6rqYvJYb5szsrOj1v7j7s2Z2\npJnNBr4GeiYY8k9kEz9wHHCOmS0HvgFOSCzgGsxsKNAR2MDM5gH9CbOxUn/toe74SfG1j3QATgEm\nm1nVf+7LgC0g9X8HdcZOuq//JsAQM1uN8OX5YXcfXSqfPWQRPzlef90oJyIiGaWxi0lERFJACUJE\nRDJSghARkYyUIEREJCMlCBERyUgJQkREMlKCkFQxsy5mttLMdsjzOKeZ2Z0FimloVLvmggIca7CZ\ntY4eX5Z/dKs8V0cz2zfOc0h5U4KQtDkReDr6nY+C3OBjZhsDe7p7W3e/Pd/jufsZ7j4zetov3+PV\n4UDgV7m8wcwaxRSLlCAlCEmNqMjb3sB5hBIlVdsrzKzSzJ4wsxlm9rdqrx0ZbZtoYSGXkVUvVdtn\nQzMbZmYTop+ffWhaWGzlATObbGZvm1lF9NILwGYWFlfZr8Z7HjSzrtWeL8ki3koz28PMrgfWjI77\ncI3jnmVmA6s9/6E1ZGanWFgUZpKZ3RPdNVu1SNVbFhaLedHMtgTOAi6K9u0QlX95OWoNvWRmLav9\nOe4xs3HADXX+RUnDkfQCF/rRT9UPcDJwT/T4VWD36HEFsIiwtoABbxC+Ga8B/AfYMtrvEaLFUIDT\ngDurbe8QPd6CUCuo5rn7AH+NHu8A/JtQU39Lal986AGga7Xni1cVb/TaK9X+XItrOe4GwPvVnj8b\n/Xl3JNQCahRtHwR0BzascR3WjX73B35f7Tgjge7R457AP6LHD0bHtaT/DegnXT9qQUianAg8ET1+\ngp92M01w94/d3Qmr9G0FtAY+cPd/R/sMJfNiKIcAd0X1gf4JrGVmzWrs0wH4G4C7v0dIENvXcrxs\n1Iy3VbZvdPfPgQ/MbG8zWx9o7e5vAAcDewAToz/LgYTrsDfwatV1cPdF1Q5XPf59CMkSwp+1qkXk\nwBNRrCI/SF2xPmmYLJQcPhDYxcycUOjQgYujXb6vtvsKwr/dmh9otX2YG7C3uy+tK4ycgoblRN20\nUVdP02qvZYo3F48C3YCZwPBq24e4+08Gt83s6ByOW9uf8ZtatksDphaEpMVxwEPu3srdt3L3LYAP\nzWz/WvZ3wprHW0f97RDGLTJ9C34B6F31xMx2y7DPWEIXF2a2PaEr6r06Yp5L+EYPYSnKJnXsX9My\nM6stcfwD6EJoRT0abRsNHGdmG0ZxtjCzLQhrDx9gYRW3qmQLsBhYq9ox3+DH6p0nE7rxRGqlBCFp\ncQLhQ7G6JwkfkE6GD353/w44F3jezCYCXwFfVr1c7T29gT2jwdlpwJkZzj8IWM3MJhM+kE/1sKY4\n1D4jajDQ0cISj/sA1VdPy6a75l5CaeyHa74QdRNNB7Zw94nRthnAFcALZvYuIfFtHHVJnQkMj2IZ\nGh1mJHBs1SA1cD7QM3rvyUD1abvqXpKfUblvKWlm9gt3/zp6fDcwywswHVVE1IKQ0ndG9A15GrA2\n8JekAxIpF2pBiIhIRmpBiIhIRkoQIiKSkRKEiIhkpAQhIiIZKUGIiEhGShAiIpLR/wfX16/Qm8nS\n7QAAAABJRU5ErkJggg==\n",
       "text": [
        "<matplotlib.figure.Figure at 0x106b06310>"
       ]
      }
     ],
     "prompt_number": 27
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "It looks like the minimum value is for a unit vector at around angle 0.5 radians:"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "min_i = np.argmin(remaining_ss)\n",
      "angle_best = angles[min_i]\n",
      "angle_best"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 28,
       "text": [
        "0.49862061618631881"
       ]
      }
     ],
     "prompt_number": 28
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "u_best = u_vectors[:, min_i]\n",
      "u_best"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 29,
       "text": [
        "array([ 0.87824304,  0.47821456])"
       ]
      }
     ],
     "prompt_number": 29
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "plt.scatter(X[0], X[1])\n",
      "plt.arrow(0, 0, u_best[0], u_best[1], width=0.01, color='r')\n",
      "plt.axis('equal')\n",
      "plt.title('Best unit vector')"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 30,
       "text": [
        "<matplotlib.text.Text at 0x106e3cc50>"
       ]
      },
      {
       "metadata": {},
       "output_type": "display_data",
       "png": "iVBORw0KGgoAAAANSUhEUgAAAXAAAAEKCAYAAAALoA6YAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAGTxJREFUeJzt3XuUZXV14PHv7obGQuTRgjwEA+NjIjLyiHGYCWoZUzQ+\nBgR0GWISUEbWTCaYYKs9bU+mC9FBRttxxcnI6kCkkwwwMxGQh3ZRgiU0r4zYNBiahxoFw9BE5SW0\ndKD3/HFOdVWX9bh1n+fc+n7WuqvvvefUOftWr9r1q/3b53ciM5Ek1c+iXgcgSWqOCVySasoELkk1\nZQKXpJoygUtSTZnAJammTOBakCLi6Yg4tNdxSK0wgaslEfHDiHi2TIg/i4hrI+LgNh33N9sR43Qy\n8yWZ+cPyXJdExHmdOtdMImJ7RPyzbp9X/cMErlYl8K7MfAlwILAF+GKbjhttOE7VNfUZI2KXdgei\n+jGBq20y8zngK8Dh4+9FxG4R8bmI+FFEPBoRX4qIF5Xb9i1H7I9HxE8j4qYo/BXwCuCacmT/0ann\niogzIuLmKe/tGNGWo+o/K4//VETcPnm0W+77yog4C/gd4OPlub46zbm+FBGfnfLeVyPinPL5QRHx\nlYh4LCJ+EBFnT9pvUUR8IiK+V8bxfyPi4Ii4qdxlU3ne95b7fygiHiy/H1+NiAOnxPwHEfEgcH9j\n/yvqZyZwtUMARMTuwPuA2yZt+wzwKuDI8t+XA/+53LYceBjYF3gZsDILvwc8RDmyz8zPNRnX+4Bh\nYB/ge8Cnp2zPzFwL/E/ggvJcJ01znEvLY1F+zn2AIeCyiFgEXANsBA4C3gb8cUQcP+kz/jbw9szc\nEzgTeDYz31xuf3153v9Tloz+C/Beir9mfgRcPiWWk4BfZ9IvSS1cJnC1KoCrIuJx4AmKBPY5gIgI\n4EPARzLzicz8OXA+RUID2EaRqA7NzBcy85Y2xpXAFZn57cx8gSJJHzXH55jJBiAj4k3l6/cAt2bm\noxTJdN/M/FRmPp+Zfw9cxMRn/LfAqsx8ECAz787Mn81wnvcDF2fmXZm5DVgJ/KuIeMWkfc4vv5fP\nzRKvFggTuFqVwEmZuQ+wG3A28K2IeBmwH7A7cGdZJnkc+DrFiBvgsxQj4+sj4vsRsaLNsW2Z9Hwr\nsEczB8lixbfLgdPKt36H4hcCwK8AB41/vvIzrqT4iwLgYOD7DZ5qfNQ9ft5ngJ9S/NUy7uFmPoP6\nkwlcbVOWP64EXgCOA35CkTgPz8x9ysfeZSmBzPx5Zn40M18JnAh8JCLeOn64OU73DMUvBwAi4oBW\nQm9gn8uA90TErwBvpKj1Q1Hq+ftJn2+fzNwzM99Vbn+YonTUiEeAQ8dfRMSLgZcC/zDPWLVAmMDV\nDuM18IiIkyhqzpszczvw58AXImK/cp+Xj9eHI+KdEfGqstTyFEXi314ecwvwylnOuQl4XUQcWU6K\nDk8XU4O2ALO282XmXRS/kC4C1mfmU+WmvwWejoiPR8RARCyOiCMi4g3l9ouA88Y/Z0S8PiKWTjrv\n5M94GfCB8jPtRlEPvz0zH5rHZ9ECYgJXO1wTEU8DTwLnAb+fmZvLbSsoyiS3R8STwCjwmnLbq8vX\nTwO3An+Wmd8qt50P/KeyLPGRqSfMzAeATwLfoOjIuJmdR6fJL49Wp24fdzFweHmuK2b5nJcCv1n+\nOx7HduBdFPX1HwD/CKwF9ix3+Tzwv4HrKb4/fw68qNw2DKwrz/uezLwB+BOK0f0jwGFM1NKnxiwR\nrdzQoRz5fIui9rkE+GpmrmxTbJKkWbSUwKFoHcvMZ8sLCzYAH83MDW2JTpI0o5ZLKJn5bPl0CbAY\nmKlFSpLURi0n8PJKs7soJmS+mZn3th6WJGku7RiBb8/Moyj6Xd8cEYMtRyVJmlPbFsTJzCcj4jrg\nDcDY+PsR4cy5JDUhM2dth21pBF4uRrR3+XyAYn2IjdMEUanH6tWrex5DHWKqalzGZEwLIa5GtDoC\nP5Cij3URxS+Dv8qil1WS1GEtJfDMvAc4pk2xSJLmYUFeiTk4ONjrEH5JFWOCasZlTI0xpsZVNa65\ntHwhz5wniMhOn0OS+k1EkJ2cxJQk9Y4JXJJqygQuSTVlApekmjKBS1JNmcAlqaZM4JJUUyZwSaop\nE7gk1ZQJXJJqygQuSTVlApekmjKBS1JNmcAlqaZM4JJUUyZwSaopE7gk1ZQJXJJqygQuSTVlApek\nmjKBS1JNmcAlqaZM4JJUUyZwSaopE7gk1ZQJXJLmaWRkhOOPP5Xjjz+VkZGRnsURmdn8F0ccAvwl\n8DIggbWZ+adT9slWziFJVTIyMsLJJ5/O1q0XADAwsIIrr1zHsmXL2nqeiCAzY9Z9WkzgBwAHZOZd\nEbEHcCfw7szcPGkfE7ikvnH88acyOnoicHr5zjqGhq7m+uu/0tbzNJLAWyqhZOajmXlX+fznwGbg\noFaOKUlqzC7tOlBEHAocDdzRrmNKUtUsX34WGzacztatxeuBgRUsX76uJ7G0VELZcZCifDIGfCoz\nr5qyzRKKpL4yMjLCmjVrgSKht7v+DV2ogZcn2RW4Fvh6Zn5hmu25evXqHa8HBwcZHBxs6ZyS1G/G\nxsYYGxvb8frcc8/t+CRmAOuAn2bmOTPs4whckuapG10oxwE3AXdTtBECrMzM9ZP2MYFL0jx1pYTS\nQBAmcEmap463EUqSescELkk1ZQKXpJoygUtSTZnAJammTOCSVFMmcEmqKRO4JNWUCVySasoELqlr\nqnIrsn7hpfSSuqJbtyLrF66FIqkyunUrsn7hWiiSNE91KvO07ZZqkjSbKt2KbCZTyzwbNpxe6TKP\nJRRJXdONW5G1okplnkZKKI7AJXXNsmXLKpe068wauGqhTnVJ1dfy5WcxMLCC4k6R68oyz1m9DmtG\nllBUebafqZuqUuaxjVB9oUp1SalbbCOUpD7mJKYqrw7tZ1IvWEJRLVSlLtlrfh8WDmvgUh9xMndh\nMYFLfcTJ3IXFSUxJ6mNOYko14WSuprKEItWIk5gLhzVwSaopa+CS1MdaTuAR8RcRsSUi7mlHQJKk\nxrRjBP5l4IQ2HEeSNA8tJ/DMvBl4vA2xSOoCl+btH7YRSgtI3W4Zptl1JYEPDw/veD44OMjg4GA3\nTitpijVr1pbJu7iac+vW4r1OJ3DbH+c2NjbG2NjYvL6m6wlc0sLiqL8xUwe355577pxfYwlFWkB6\ncTVnr0b9C0E72ggvA24FXhMRD0fEB1oPS1InLFu2jCuvLBbBGhq62pFwzXklpqSOchnc5ngpvaSu\nmW2i0knM+TOBS32m1UTYqUTqKLv9GkngZGZHH8UpJLVq/fr1OTCwf8IlCZfkwMD+uX79+q59/WzH\nXbr0leVxs3xckkNDp7R87IWszJ2z5le7UKSaaLWboxPdIBMj78OaPoaaZwKX1LSJXwoHMHGrN282\n0S0mcKlJ3Z6Ya7WHu7M94MuAdcAwS5f+I5deav27K+aqsbT6wBq4+lCn6smNnHdo6JQcGjqlqfO1\n+vXTHa8X34eFgAZq4HahSE3wDvETbBHsjEa6UCyhSA2YmqRqZ8sWuPvu4nH44fD2t7ft0MuWLTNp\n94gJXJrDdIsxrVp1Nhs2rKjeHeK3boV774V77oFvfxvuuAPuvx+2bYPnnoPf+i348Id7HaXaxAQu\nzWG69rtvfatYR2RiVN6jSbvRUbj9drjttmJ0vWUL7L47vPACPPPMxH5LlsCv/RpcfTXsumvHw7Ks\n0h0mcKlJlSgdfOMb8PnPw/PPT7z31FM777PLLnDYYXDDDTAw0PGQXD62e5zElOZQ6cvEn3wSDjkE\nnn56+u2LF8OBB8J3vgP77deVkJzgbY9GJjHbcVNjqa+1ewnWtt6Tcq+9YHi4KJtMFQFLl8Kttzac\nvL1fZs3M1WfY6gP7wKUd2t43fdttOWkBkp0fe+2Vef/9XY/N3vD2oIE+cBO4aq/dF6d00tDQKe1Z\n9OmBBzJf+tKJZL18eeaLXzzxeo89Mr/znd7ElvX6P6mqRhK4k5iqtQU3YfbYY3DKKXDLLcXrtWvh\nzDOL59deW7QM7r47fP3rcPTRDR92ZGSEO+/cBJzYljArMcG7EMyV4Vt94AhcHdTOUWM3NF1eePrp\nzDPPnBhhf/zjmb/4xc773Hhj5uLFmdde22RMyxP2tfRRETQwAncSU+qiqROiq1adzZo1a2eeNHz+\nebjgAnjJS+Dii+Hd74bHHy/e2223nfd961uLfvB3vnNeMU30uX8O+GvgQpYuPW/Ov2Sc8KyAuTJ8\nqw8cgauDqjxhNlsdeP369Xn00W/JRYteWo58p8S+fXvm5ZdPjLhf+9rMhx7qSJzN/BVT5e97v8BJ\nTC0E3Zwwa/RcsyW4qdtg/4T1E4nzllsmEndE5qZNHf9MS5bsnXBswrG5ZMnec34f61a6qqNGEriT\nmKq9bk2YzWfCdLa730zdVljLa/h1rvjm1TB6RfHW6GixdklX7Ar8u/L5x7p0TrXKBC41qBO3JAPY\nnye4ihGO5Qp4nqLWfcYZsKg7U1Rr1qxl27bPMv65tm2b+3N19uYQapSTmOprnZ5o+8lPfjrt+8uX\nn8XAwAqKu9SsKxPcWTtt24MLuYTjeJQ/5lie4funnQa/+AV88INdS97NavfVqWrSXDWWVh9YA1eP\ntHuiragV7zepdr3vrPXiGevl27blA6efvqPO/f/e9KbMJ55oOq5WOSFZTXhHHi1knVhU6ZhjjmPj\nxheAg4CzgEcbP2YmXHYZvP/9xesjjiguuDn44KbjaReXf60e78gjtdm+++5PcbXixC+Fhtx8M7z5\nzcXzXXctVgc84oiddullEp3PRLDJvkLmGqK3+sASiqbRjda/TpQG5n3Me+/deY2SG2/sWqydUJc4\n+wHd6AMHTgDuAx4EVkyzvSsfVvXxqU99Khct2qcrSaATvygaOuYjj2Qec8xE4l63rrg4ZwaN9FVX\nYYEo+7+7p+MJHFgMfA84lKKR9C7gtVP26dLHVR2sX7++vPqwT5PAU09lvv/9E4l79erM556b88vm\nSoxVGfmawLunkQTeag38jcD3MvOHABFxOXASsLnF46pPrVmzlu3bX93rMNpv2zb4zGdg9eri9Wmn\nwYUXwp57NvTlc/VVd6oHfb7s/66WVhP4y4GHJ73+MfAvWzym+t5vACt2vFq06ByWL7+sd+G06oor\n4NRTi+dHHQXXXQcHHTSvQ4z3Vff8JslzqEucC0WrCbyh/sDh4eEdzwcHBxkcHGzxtKqriRHc7wIX\nsmjRg3zyk8vrnQR+8APYf3+48UY4/PCmDzNbJ0iVRr6u9d0ZY2NjjI2NzetrWuoDj4hjgeHMPKF8\nvRLYnpkXTNonWzmH+k/ftaFlFvef7LC++75pVo30gbeawHcB7gfeBjwC/C1wWmZunrSPCVyS5qnj\nF/Jk5vMR8YfACEVHysWTk7ckqXO8lF6SKqiREXi1lzyTasBbi6lXHIFLLZh6k4eBgRUuraq26Pgk\nZoNBmMDVtzqx4qEEllCkllkeUZW5nKw0g0bugVmlC2y08DgCl2aw8/ojRSIfv5Bm3Pil5Ucf/WWW\nLj2PX/3VV/UkVi1MJnCpDe677z5+9rM/YePGD3HyyadbblFXWEKRZtBoeaQqKwVq4TGBSzNw5T1V\nnW2Eqp2qLepkL7g6wT5w9Z2qJsuq/VJR/ZnA1Xe8cEYLhRfyqLa8gEaamwlcTetUkh0vk4yOnsjo\n6Ik7teUtX34WAwMrgHXAurIz5Ky2nbvb/EWllsx11+NWH3hX+r7UybukN3KH9qGhU3Jo6JSe3Jm9\nXapyp3lVE124K70WqF72PvfLPRntH1erTOCqHNcXkRpjAldTOplkF8oFNP6iUqtsI1TT7H1und9D\nzcQ+cFXSTEnLZCZNMIGrUkZGRli58nw2bfou27efAfyLHVdSApW8wlLqFRO4KmPqJfAw3sv9KEND\nVwN4haU0SSMJ3ElMdcXUlrnCWuDEHkUk1Z8JXD30yE6dF3ZkSPNjAldXTG2ZW7ToHI488nDOP3+i\nzr0QWgeldrIGrq6xy0RqnJOYklRTLicrVYArDqpTmk7gEfHeiPi7iHghIo5pZ1BSv5htaVypVa2M\nwO8BTgZualMsUt/ZuX2y6IMfnweQWtV0F0pm3gdFnUaS1H3WwFUZ/Vgr7rc7CKlaZh2BR8QocMA0\nmz6Rmdc0epLh4eEdzwcHBxkcHGz0S7VATL3UfsOG0/tiLZSFsjSuWjc2NsbY2Ni8vqblNsKI+Caw\nPDO/M8N22wg1J+82L+2sm22EFsJ7qB9LD5Lm1vQkZkScDPwpsC9wXURszMy3ty0yNaRfSg/enUaa\nP6/ErLl+Kj14qb00weVktUMdkmO/3G1e6hYTeM01UnrolzKLpJ1ZQukDc42u+6nMIi0UllAWCEsP\n0sLklZgLQBWvBrT1UWqdJZQFokqTmFNr8t6BXvpl3tBBlWRNXpqbN3SQpD7mJKa6zqsupfawhKKe\nqFJNXqoia+CSVFPWwDUvtvZJ9eIIXICtfVLVWEJRw2ztk6rFEook9THbCAXY2ifVkSUU7WBrn1Qd\n1sAlqaasgavjbD2UescRuJpm66HUOZZQ1FG2HkqdYwlFkvqYbYRqmq2HUm9ZQlFLbD2UOsMauCTV\nlDVwSepjJnBJqikTuCTVVNMJPCI+GxGbI2JTRFwREXu1MzBJ0uxaGYFfD7wuM48EHgBWtickSVIj\nmk7gmTmamdvLl3cAB7cnJElSI9pVA/8g8LU2HavjXIBJUj+YtQ88IkaBA6bZ9InMvKbcZxVwTGae\nOsMxKtUH7gJMkuqgkT7wWS+lz8yhOU5wBvAO4G2z7Tc8PLzj+eDgIIODg7Pt3lFr1qwtk3exANPW\nrcV7JnBJvTQ2NsbY2Ni8vqbpKzEj4gRgDfCWzPzJLPtVagTuCnqS6qDlEfgcvggsAUYjAuC2zPyD\nFo7XFS7AJKlfLMi1UFyAqTV+/6TOczErtZ2TwFJ3mMDVds4hSN3haoSS1Me8I4/mxUlgqTosoWje\nnMSUOs8auCTVlDVwSepjJnBJqikTuCTVlAlckmrKBC5JNWUCl6SaMoFLUk2ZwCWppkzgklRTJnBJ\nqikTuCTVlAlckmrKBC5JNWUCl6SaMoFLUk2ZwCWppkzgklRTJnBJqikTuCTVlAlckmrKBC5JNWUC\nl6SaajqBR8R5EbEpIu6KiBsi4pB2BiZJml0rI/D/mplHZuZRwFXA6jbF1HFjY2O9DuGXVDEmqGZc\nxtQYY2pcVeOaS9MJPDOfnvRyD+AnrYfTHVX8z6piTFDNuIypMcbUuKrGNZddWvniiPg08HvAs8Cx\nbYlIktSQWUfgETEaEfdM8/g3AJm5KjNfAVwC/LcuxCtJKkVmtn6QiFcAX8vMI6bZ1voJJGkBysyY\nbXvTJZSIeHVmPli+PAnY2EwAkqTmND0Cj4i/Af458ALwfeDfZ+ZjbYxNkjSLtpRQJEnd15UrMSNi\nOCJ+HBEby8cJ3ThvIyJieURsj4ilFYilchdHRcRnI2JzGdcVEbFXBWJ6b0T8XUS8EBHH9DiWEyLi\nvoh4MCJW9DKWcRHxFxGxJSLu6XUs4yLikIj4Zvn/9t2I+HAFYnpRRNxR/rzdGxHn9zqmcRGxuMyV\n18y2X7cupU/g85l5dPlY36XzzqpMkEPAj3odS6mKF0ddD7wuM48EHgBW9jgegHuAk4GbehlERCwG\n/jtwAnA4cFpEvLaXMZW+TBFTlfwTcE5mvo6i5fg/9Pp7lZm/AN5a/ry9HnhrRBzXy5gm+SPgXorc\nOaNuroVSxcnMzwMf73UQ46p4cVRmjmbm9vLlHcDBvYwHIDPvy8wHeh0H8Ebge5n5w8z8J+Byign9\nnsrMm4HHex3HZJn5aGbeVT7/ObAZOKi3UUFmPls+XQIsBn7Ww3AAiIiDgXcAFzFH3uxmAj+7/DP8\n4ojYu4vnnVZEnAT8ODPv7nUsk0XEpyPiIeB04DO9jmeKDwJf63UQFfJy4OFJr39cvqdZRMShwNEU\nA4KeiohFEXEXsAX4Zmbe2+uYKK6p+Riwfa4dW7oSc7KIGAUOmGbTKuBLwCfL1+cBa4Az23XuJmNa\nCRw/efdOxzNHTJ/IzGsycxWwKiL+I8V/5Ad6HVO5zypgW2Ze2ul4Go2pAuwAmKeI2AP4G+CPypF4\nT5V/XR5Vzu2MRMRgZo71Kp6IeBfwWGZujIjBufZvWwLPzKFG9ouIi4Cu/ADOFFNEHAEcBmyKCCjK\nAndGxBs73QrZ6PcJuJQujXbniikizqD4k+5t3YgH5vV96qV/ACZPNB9CMQrXNCJiV+ArwF9n5lW9\njmeyzHwyIq4D3gCM9TCUfw2cGBHvAF4E7BkRf5mZvz/dzt3qQjlw0suTKSaheiYzv5uZ+2fmYZl5\nGMUP3TG97mOPiFdPejnjxVHdVHYMfQw4qZz0qZpezq18G3h1RBwaEUuA9wFX9zCeyopipHQxcG9m\nfqHX8QBExL7j5dyIGKBoaOjpz1xmfiIzDynz0m8DN86UvKF7NfALIuLuiNgEvAU4p0vnbVRV/hQ+\nv1xr5i5gEFje43gAvkgxoTpatjX9j14HFBEnR8TDFN0M10XE13sRR2Y+D/whMELRMfC/MnNzL2KZ\nLCIuA24FXhMRD0dEx8twDfgN4HcpOj2q0k58IHBj+fN2B3BNZt7Q45immjU3eSGPJNWUt1STpJoy\ngUtSTZnAJammTOCSVFMmcEmqKRO4JNWUCVySasoELkk19f8BC5BqF7nCXjUAAAAASUVORK5CYII=\n",
       "text": [
        "<matplotlib.figure.Figure at 0x106b8c6d0>"
       ]
      }
     ],
     "prompt_number": 30
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "Do the projections for this best line look better than before?"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "projected = line_projection(u_best, X)\n",
      "plt.scatter(X[0], X[1], label='actual')\n",
      "plt.scatter(projected[0], projected[1], color='r', label='projected')\n",
      "for i in range(X.shape[1]):\n",
      "    # Plot line between projected and actual point\n",
      "    proj_pt = projected[:, i]\n",
      "    actual_pt = X[:, i]\n",
      "    plt.plot([proj_pt[0], actual_pt[0]], [proj_pt[1], actual_pt[1]], 'k')\n",
      "plt.axis('equal')\n",
      "plt.legend(loc='upper left')\n",
      "plt.title(\"Actual and projected points for $\\hat{u_{best}}$\")"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 31,
       "text": [
        "<matplotlib.text.Text at 0x1071e6d50>"
       ]
      },
      {
       "metadata": {},
       "output_type": "display_data",
       "png": "iVBORw0KGgoAAAANSUhEUgAAAXAAAAEMCAYAAADd+e2FAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3XlcVPX6wPHPFwFFBQVxwRVTyS3X1NS8YTnQYpba3i3L\nyurWvS2Uimjqz33BzOreMk1tN+vqNTMQF2y3zX0397QyNZdEEXh+f5xhlWWAWRh43q/XvGTmfM85\nz5mRhzPP+Z7v14gISimlvI+PpwNQSilVMprAlVLKS2kCV0opL6UJXCmlvJQmcKWU8lKawJVSyktp\nAldKKS+lCbwCMMbsN8Zc54b9zDfGjHP1fuz72mKM+Zs79lVEHC45ZncdnzHmcmPMBmPMaWPMk67e\nn32fAcaYKu7YV3mnCdwNjDHJxpgTxhh/B9vvN8Zc68QQxP5wNXftBxFpKyKfl3Y7TnivXXLMxTm+\nUh7DUGCViASJyCsl3IbDjDGVgDHAWGOM5p9S0jfQxYwx4UBX4Hegn4OrCWBcFJKrlTpuY4yvMwJx\nkDPea09/VqU5hibAtuKuVIrPKBKIB14EepdwG8pOE7jr3Q+sBN4GBuVcYIxpZIz5rzHmd2PMH8aY\nl40xbwGNgU+MMWeMMc/Z22YYYy7LsW6ur+7GmOHGmD32r8JbjTG3OhJcYevZz+xijDEbjTF/GmM+\nMMZUzrG8ozHmJ/u6HwCFfi22b2+4fT8njDFvZm7PvmyoMWYTcMYY42OMaWX/9nLSXlK4Oc+2rsvx\nvL4x5mP7e7nXGPPPQt7nWfbX3877Xhe2neIec2HHa19enOPL97PI7xjs7YcZYw7b49yR3xm6MWY1\nVkJ9xd6uuQMx5fqMCv608yciq0TkdxH5VURWFXd9lYeI6MOFD2APcC/QAkgF6thfrwRsxDobCQAq\nAz3sy/YB1+bZTgZwWY7n84D/y/H8NqCe/ec7gLNA3YK2V4z1vgXqAcFYZ2qP2pf5AweAp+zHMtB+\nfP9XyHuxH9gENLBv70tgXI5lP9mXVQb87O/dcMAX62ztNNAi7zFhnYj8CIy0t20K/AxEFfA+98wR\nU87tmIK2U5JjLuJ4HT4+Bz6LvG0vBw7m+Fwb5/y/kyfGNcBgB2PK9Rl5+ndLH6Jn4K5kjLka6z/7\nUhHZjfVLd499cVcgDHheRFJE5IKIfF3cXWT+ICIficiv9p8/BHbb91EoB9abJdbZ0kngE6CD/fWr\nAF8ReUlE0kXkY+D7onYHvCIiv9i3NwG4O8eyWfZlF+zbryYik0UkTUTWAMvIfv9y6gKEish4e9t9\nwBzgLvJ/n78qIL6uhWynJMdc2PEW5/gyFfRZ5JWO9YeqjTHGT0QOisjeQrabqaiY8n5Gudi/6Qww\nxrxvf+5njFnpwH5VCWkCd61BwAoROWN/vojsMkoj4ICIZDhjR8aY+40x6+1ffU8CbYFQJ6z3a46f\nU4Dq9p/rA7/k2dwBiq7FHsrx80H7dvJbVj/P88zt1+dSTYD6mcdgP45YoA7QEMff58K2kxlTcY+5\noOMtzvFlyvtZVMuvkYjsAZ7Gulj4mzHmfWNMWCHbzeRITHmX59QS+A7rpAWgm3195SKawF3EGBOA\nVZK41hhz1BhzFIgB2htj2mH9IjQ21lX5vPLr1XAOqJrjeVhmO2NME2A28AQQIiLBwBaKSKYlXc/u\nKNm/qJmaFBB7To3z/JwzIeZc9wjQyBiTM5YmXJpAwXov94lIcI5HkIj0BQ5T8Pucd58HC9kOlOyY\n8x7vkRIcX1Eu2b+IvC8ivXLEN8WB7TgSU4HHKiJJwAPAO/aXrgOSAIwxYcaYYtW8jTHPGGPqFWed\nikYTuOvcCqQBrYD29kcr4AusC5vrsBLCZGNMVWNMFWNMD/u6vwHN8mxvA3CvMaaSMeZ6IGcf4WpY\nv1h/AD7GmAexzqSLUtL1AL4B0owx/7J/VR6AVcoojAH+YYxpYIwJAeKAhQW0/Rbrj9ZQ+/Yjgb7A\nB/m0XYd1UW2osfoYVzLGtDXGXEnh7zPkfq+/L2Q7JTnm/I43M/51xTi+wraf9xgwxkQYY661XzC9\nAJzHKqsUtR1nxHQVVq0foA+wxhgTbY+xuL1dIjLLeyp/msBd537gTRE5LNZV999F5DfgFbJrijcD\nzbHO/A5hnbEDTAJG2r/GP2t/7Sl7+5P29Rdn7khEtmFdpPsG62t2W7J/iQpUgvWy+jyLSCowAOuM\n67g99o+L2iXwHrAC6+LgbmB8AbFdxDreG4BjWO/bfSKyK5+2GViJpgOw195+NhBkX1bQ+ww53mus\n9zjf7ZTwmAs8Xvu2HDq+QradeTac9/9LZftrx7D+eIVilYIK25YzYgJYAvQzxtyO9R7eiHUhtyNg\njDEPG2M6GMsgY8w19gSPMWagMeZGY8zV9utH4caYq4qx7wrHiOiMPMo9jDH7gIdEZLUTtnUAuFdE\nivxD5SnOPF5vZ6zumJ8AdYGeWN8KUrD+qO0UkWPGmA+BZSLylr28d5eIOFL6qbBKdQZu/zq6zli3\n4m4zxkxyVmBKFcQYUweojdWtTXmH+iKyH7gG61teK+CjzD/AxpjHsb4lXG6/RtQT+MoY07iA7Sms\nvp4lJiLnjTG9ReScse7M+tIYc3VZPitS3s0Y0wWrJDFLRA57Oh7lsG+MMVHAaRH5zhjjB/Q1xvwB\nbMfqy94C2IVVK78M60K9fsaFcFoJxRhTFVgLDLLXVpVSSrlQqS9iGuuW5w1YV5nXaPJWSin3KHUC\nF5EMEemAdcPE3+xdj5RSSrmY00Z9E5FTxphPgSuB5MzXjTHazUUppUpARAq9qa60vVBCjTE17T8H\nADZgfT5BlKnH6NGjPR6DN8RUVuPSmDSmihCXI0p7Bh4GLDDWsJI+wNuiQ0QqpZRblLYb4Wagk5Ni\nUUopVQwV8lb6yMhIT4dwibIYE5TNuDQmx2hMjiurcRXF5bfSG2PE1ftQSqnyxhiDFHER051zD+aS\ne8RK5Un6B1Yp7+SxBA6aOMoC/UOqlPeqkDVwpZQqDzSBK6WUl9IErpRSXkoTeBnl4+PD3r2OTCSu\nlKqoNIE7SXh4OKtXV/iJV5RSbuTRXigl9dVXX7FiRRIhIcEMHjyYwMBAT4eU2WfT02EopSqQMncG\nnp6ezgcffMC0adP4/PPPL1n+7rvvExV1O+PGpTF8+Fd06NCTs2fPOjWGyZMn07x5c4KCgmjTpg1L\nlizJWvbGG2/QunXrrGXr16/nvvvu4+DBg9x8880EBgYyffp0kpOTadSoUa7t5jxL/+677+jevTvB\nwcHUr1+ff/7zn1y8eNGpx6GUKufcMKKW5Ce/19PT0yUq6lapVq27+Pk9I1WrNpIZM2blahMa2kTg\nWwEREKla9VZ57bXXcrWZN2+B1K4dLtWrh8q99z4sKSkp+cZQkEWLFsnRo0dFRGThwoVSrVo1OXr0\nqHz44YfSoEED+eGHH0REZM+ePXLgwAEREQkPD5dVq1ZlbWPNmjXSsGHDXNvN2ebHH3+UdevWSXp6\nuuzfv19atWolM2fOzGprjJGff/65WHGXREGfj1LKs+y/m4Xm1zJ1Br569Wq+/vpn/vprLRcvzuDc\nuS8YNmxYrjPTc+dOA02znqemNuXUqVNZz9esWcMTT4zg2LEPOXt2Ix9//DtPPPFcseK47bbbqFev\nHgB33HEHLVq04LvvvmPu3LkMGzaMzp07A9CsWTMaNy7ZnKudOnWia9eu+Pj40KRJE4YMGcLatWtL\ntC2lVMVUphL4iRMnMKY54Gd/pTHG+OYqkdxwQ18qV/4XcBBYiZ/fO0RFRWUtX7YsgXPnHge6APU5\nf34qy5Z9Vqw43nrrLTp27EhwcDDBwcFs2bKFP/74g0OHDtGsWbNSHqVl165d9O3bl7CwMGrUqEFc\nXBzHjx93yraVUhVDmUrg3bt3JyPjS+BT4BSVKo2hWbPLqVmzZlabBQv+w623VqVGjato3PhpFi2a\nR4cOHbKWh4YG4++/O8dWd1OjRk0cdeDAAYYMGcKrr77KiRMnOHnyJG3btkVEaNSoEXv27Ml3vby3\npFerVo1z585lPU9PT+fYsWNZzx9//HFat27Nnj17OHXqFBMmTCAjI8PhOJVSqkwl8EaNGvHpp4to\n2DAGf/8GdO78JUlJS3Ilx2rVqvHBB2/y559HOHBgCzfddFOubTz22KPUqfMNVarcia9vDFWrDuaV\nVyY5HMNff/2FMYbQ0FAyMjKYN28eW7ZswRjDww8/zPTp0/npp58QEfbs2cPBgwcBqFu3Lj///HPW\ndiIiIjh//jzLly/n4sWLjB8/ngsXLmQtP3v2LIGBgVStWpUdO3bwn//8p6Rvm1KqgipTCRzgmmuu\n4dChHVy4cJZ161bRoEGDYq0fHBzM5s3rmDatF+PG1eabb1bmKrEUpXXr1sTExNC9e3fq1avHli1b\nuPrqqwGrNh4XF8c999xDUFAQAwYM4OTJkwDExsYyfvx4goODmTFjBjVq1ODf//43Dz/8MA0bNqR6\n9eq5eqVMnz6d9957j6CgIIYMGcJdd92V6w+VDjKllCqKx8YD137TZYN+DkqVTY6MB17mzsCVUko5\nRhO4Ukp5KU3gSinlpTSBK6WUl9IErpRSXkoTuFJKeSlN4Eop5aU0gSullJfSBO4iX3zxBS1btvR0\nGDzwwAOMGjXK02EopVxAE7iL9OrVix07dpR6O6WdG9MYo7flK+VkiYmJREUNJCpqIImJiR6Lo1RT\nqhljGgFvAXUAAWaLyCxnBFaor76CFSsgJAQGDwYPTKmWlpaGr697ZqQr7a3uequ8Us6TmJhI//6D\nSEmZAsCXXw5i8eIFREdHuz2W0p6BXwSeEZE2wFXAE8aYVqXaYno6fPABTJsG+UypxrvvQlQUjBsH\nw4dDhw7gxCnVwsPDmTx5Mm3atCEkJITBgwdz4cIFkpOTadiwIVOnTiUsLIyHHnqI1NRUnn76aRo0\naECDBg145plnSE1NBbhkSrUjR44wcOBA6tSpw2WXXcbLL7+ctSwjI4OJEydmTePWpUsXDh8+zN/+\n9jcA2rdvT2BgIIsWLQJg2bJldOjQgeDgYHr27MnmzZuztrV+/Xo6depEUFAQd911F+fPn3fae6OU\ngvj42fbk3Qa4n5SUKcTHz/ZMMEVN2VOcB7AEuC7Pa4VNF5RberpIVJRItWoifn4iVauKzJiRu01o\nqGTNp2bNqSaSZ0o1mTdPpHZtkerVRe69V6QYU6o1adJErrjiCjl8+LCcOHFCevbsKSNHjpTk5GTx\n9fWV4cOHS2pqqqSkpMioUaOke/fucuzYMTl27Jj06NFDRo0aJSK5p1RLT0+XTp06ybhx4+TixYuy\nd+9eueyyyyQxMVFERKZOnSpXXHGF7Nq1S0RENm7cKMePHxeRS6dW++mnn6ROnTry3XffSUZGhixY\nsEDCw8MlNTVVLly4II0bN5aZM2dKWlqafPTRR+Ln55cVk8Ofg1KqQDbbAIF5Ap0FFgrMF5ttgNP3\ngwNTqjkzeYcDB4DqeV4vLLjckpKspJszQfv5iaSmZrepWjX3cl9fkSlTspevXp27TZUqIoMHO/ym\nhYeHy+uvv571fPny5dKsWTNJTk4Wf39/uXDhQtayZs2ayWeffZb1PDExUcLDw0UkdwL/9ttvpXHj\nxrn2M3HiRHnwwQdFRCQiIkKWLl2abzx5E/hjjz12SUK+/PLLZe3atbJ27VqpX79+rmU5/6jkRxO4\nUsWTkJAgAQF1BYYK1JYqVepIQkKC0/fjSAJ3ykVMY0x14CPgKREpeT3jxAnIe8HNmNwlkhtugMqV\ns5/7+VkllUzLlkGOmXA4f956rRhylj4aN27MkSNHAKhduzb+/v5Zy44cOUKTJk3ybZvTgQMHOHLk\nSNYUbcHBwUyaNInff/8dgMOHDzs8VduBAweIj4/Pta3Dhw9z9OhRjhw5csn46U2aNNEauFJOFB0d\nzeLFC7DZ9lCrlg8PPjjQI/VvKOVFTABjjB/wMfCOiCzJr82YMWOyfo6MjCQyMjL/jXXvDjmnFatU\nCZo1gxxTqrFgATz0ECQkQI0a8O9/W3XwTKGh4O8P9lo0YLUrhsxZdjJ/rl+/PnDpJAv169dn//79\ntGrV6pK2OTVq1IimTZuya9eufPeXOVVb69ati4ytcePGxMXFMWLEiEuWrV27ll9++SXXawcOHKB5\n8+ZFblcp5bjo6Giio6PZtGkTNpuNyZMnExQUVKptJicnk5ycXLyVijpFL+wBGKxeKC8W0qawrweX\nSk4WadhQxN9fpGtXkcOHi/e948QJa/0qVazyStWqIvZasyOaNGki7dq1k8OHD8vx48elZ8+eEhcX\nJ8nJyVklkUwjR46UHj16ZNXAe/bsmW8NPC0tTTp16iRTpkyRc+fOSVpammzevFm+//57ERGZNm2a\ntGvXTnbv3i0ZGRm5auD16tWTFStWZO3zhx9+kEaNGsm6deskIyNDzp49K8uWLZMzZ85IamqqNG7c\nWF566SVJTU2Vjz/+WGvgSrnY/X36SFzTpiI2m4gTSym4ugYOXA1kABuA9fbH9VKaBO4MJ0+KvPyy\nyKRJIhs3FmvV8PBwmTx5srRu3Vpq1qwpDzzwgKSkpMiaNWukUaNGudqeP39e/vWvf0lYWJiEhYXJ\nU089lVUjz5nARUSOHDkid999t9SrV0+Cg4Ole/fusmrVKhGxLnKOHz9emjZtKoGBgdK1a1f55Zdf\nRETktddek7CwMKlZs6YsWrRIRKwaXJcuXaRmzZoSFhYmd9xxh5w5c0ZErATfsWNHCQwMlDvvvFPu\nuusuTeBKuUpCghyoXFlCQH4BkYAApyVxRxK4TqmWR9OmTZk7dy7XXnttqbazevVqHnnkkVwTHZdF\nZfVzUMorREVBUhJDgZPAGwA2m3WfSik5MqWae+5EqYC2bNnCZZdd5ukwlFJuEAts8cB+NYG7wFNP\nPcWyZctYsGCBp0NRSrlSTAx8+SXBKSn0AggIsF5zEy2hVHD6OShVSomJEB9v/RwTA07qUuhICUUT\neAWnn4NSZZMjCVxHI1RKKS+lCVwpVbElJlq9SaKirJ+9iEdLKKps0BKKqrASE6F/f0hJsZ4HBMDi\nxU6rY5dGma6BK6WUx9n7cQOcBaqD0/pxl5bWwJVSygFbgE5AalENyxhN4EoptykrU5FliYmBgADa\nApcBs/383NqPu7S0hKKUcou8U5EFBAzz2FRkeQKD+Hg2njlD9M6d7Nq/v9QjCzqD1sCVUmVGVNRA\nkpL6AfWAFOAUNttSVqz42MORZRs0aBCNGzdm3Lhxng5Fa+BKqbKoKvA0ZbHiPG7cOF566SX+9rcb\ny06ZpxCawJVSbhETM4SAgGHAXiAYX99niYkZ4umwctm+fTspKel88cU5kpL60b//oDKdxLWEopRy\nm8TEROLjZ3P27Gm2bfueffv2ERwc7OmwslhlnuuAMcBa4DuPlXm0hKKUKlOio6NZseJjvv46idtv\nv51JkyZ5OqR8VANmAueKauhxmsCVVyhz3c9UqY0dO5a5c+fmmoPW07LLPBeBLQQEDCtzZZ6ctISi\nyrwy2/1MldrIkSM5fPgw8+fP93QoWTLLPGAldE/9P9NuhKpcyO5+dg/wF/C/Mtf9TJXM6dOnadGi\nBStWrKB9+/aeDqdM0Rq4KmdmAs94OgjlREFBQYwcOZLhw4d7OhSvpAlclXnZdcnqwH+pXLnsdT9T\nJffoo4+ye/duVq1a5elQvI6WUJRXyKxLHjy4h6Cgynz33XeeDskjykp91tkWLlzI1KlT+f777/Hx\n0fNK0Bq4KodSU1Np1aoVb7zxBtdee62nw3Gr8nwxNyMjg6tateIZf3/uDgtz6tyS3koTuCqXFi5c\nyLRp0/juu+8q1Nla9sXcvwG/AD+Xn4u5iYkk9+vHg6mp7AAql6GJFTxFL2Kqcun222/HGMOHH37o\n6VA85ABwH1Zf5XIiPp7I1FTaAP8Ba4aczJneVYE0gSuv4+Pjw9SpU4mLi+PChQueDsdtsi/mHgBq\n4Of3TLm7mDsNaOvpILyIJnDllXr37k3Lli157bXXPB2K20RHR7N48QJstqV07x5K1aqV6Natm6fD\ncg77xAqtgD5gzU3pRRMreIrWwJXX2rx5M3369GHXrl3UqFHD0+G43eDBg6lbt24ZHU+kBOwTKwB6\nERO9iKkqgMGDB1OvXj0mTpzo6VDc7vDhw7Rv356NGzfSsGFDT4dzKU3IpeKWBG6MeRO4CfhdRK7I\nZ7kmcOUymUls06ZNNGjQwNPhuF1sbCy//fYbb775pqdDyS0xEfr3ty5GglUSqeC9SorLXQm8F3AW\neEsTuPKE2NhYjh07xpw5czwditudOnWKiIgIVq5cyRVXXPLr5zlRUZCUhAC/A3UBbDZYscKzcXkR\nt3QjFJEvgJOl3Y5SJTVs2DA++eQTtm7d6ulQ3K5GjRrExsYWaywRdw7NuwbromS6S/dScWkvFOX1\natasyfDhwyvsgEiPP/4427dvJzk5uci2mXdzJiX1c86UYYmJ1tl2VJT1cyZ7r5LeQE3gLT8/7VXi\nAk65iGmMCQc+KaiEMnr06KznkZGRREZGlnqfSuV04cIFWrZsyfz587nmmms8HY7bvf/++8yYMYN1\n69YVendq9t2cXYEjwOGS3c2ZmAixsbBxI2RkWK/lrXPbL2J+8+ef3LpzJ22vjKRSJd9yNYaLMyUn\nJ+f6Izx27NgiSyiISKkfQDiwuYBlopQ7vPvuu9K1a1fJyMjwdChul56eLp07d5YPPvig0HY22wCB\n+QJfCDQWmC0224Di7SwhQcTfXwREQE6BHLb/LDZbPs0TxMenssBtAvMlIKCuJCQkFG+fFZA9dxaa\ne7WEosqNu+66i7S0ND766CNPh+J2Oe9OTU1NLbBd9t2cPwO1SjYzfGwspKZyAdgIzAceL6R5fPxs\nMjLGYlXEbyIlZUrWiIqqdEqdwI0x7wNfAxHGmEPGmAdLH5ZSxZeZxGJjYwtNYuXVtddeS4sWLXj9\n9dcLbJPzbs4ePUKpVq0SXbp0Kd6ODhzgItAF6wLlncAm4At//0Lq3PWAV4HCKwKqePRGHlXuXH/9\n9fTt25cnn3zS06G43aZNm7DZbOzevZugoKAi2z/66KMEBQUxbdo0x3fSqROsX8/TQCIwAGhVpQqv\nhofz9bZtGJM7SZfnYXBdyZFuhE6pgRf2QGvgys02bNggdevWlVOnTnk6FI8YNGiQxMXFOdT2yJEj\nEhISIvv3789+MSHBqmXbbNbPedlr4MdAatofB996S5o1aybt2nUTm23AJTXuhIQEsdkG5LtM5Q8H\nauCawFW5dP/998vIkSM9HYbTOZIIDxw4ICEhIXL48GGH1h81apTcd999mQ1EAgKyLlBKQEDBSdxm\nk4nNm0tEgwYSHR0t/v41BeoKzNELlU6gCVxVWJlJ7JdffvF0KE6TkJAgAQF17b1ICu/NMXToUHn4\n4YcdWv/06dNSNzhY1l91lUhIiAjInyDHCulZkumvv/6SkJAQMcZHYIKATeBVgfnF792icnEkgWsv\nFFUuNW7cmIceeogxY8Z4OhSniY+fba8j+wA1C+3NMXz4cP73v/+xbdu2fNaPAPqQkjKF5NhxBF5z\nDaP+/JNh334LJ04AMAMY5kBMX3zxBWfOpCHiBywCpgCfluYwVTFoAlflVmxsLEuWLGH79u2eDsXJ\nwoDngLQCWwQHBxdyd+pS6jOYXQxl4vqvYP16hoiwF1hpb/EM8AmwpXLlQu+gjI+fzcWLM7FGO9mI\nlcRvJyBgWLmbbKJMKuoUvbQPtISiPGj69OnSr18/l2zb3RfmcpdA2oqfX2Ch+z1//rw0adJE1q5d\nm7X+zf41JZlGUgdkk71Eshnk3yArQb7NLJuEhMiMiAi5qWvXQmPKvjFomUBzgWoSHHyZ1r+dAK2B\nq4ouJSVFmjRpIl988YVTt1ucerSz92uzDZCrrrpWgoODi+xp8/bbb0u3bt0k47PPRDp2lAxjREBe\nBLkR5G6Qn0BCQA7kuXB5/vx5CQ8Pl+Tk5ELjsd6HeQJviDG+MmLECGcfdoWkCVwpEXnrrbfkqquu\ncuot9tlnnocEMjxy0e6+++6TUaNGFdomPT1dOoSFySJ74haQQ/Yz7a4gj4E8ChIHcj+IdOyYq9fJ\nu+++K126dCn0vcv5TWTixInSvHlzSU1NddpxVlSawJUSK4m1b99ePvrooxJvI2+5xErg8wTaCSz3\nSALfv3+/hISEyJEjRwpulJAgK4yR5iC/28+yl4C0BbkIchwkFOQHkLrVq8uGDRtyrZ6eni6dOnWS\nhQsXOhxXVFSUvPLKKyU9LGWnCVwpu8TERGnRokWJzgzzK5eMHz/e/tq/BBpKlSp1PFL3fe6552TI\nkCEFN7DZRECiQO4AuQEkA+RqkHkgEhgoU+vXl/49esisWbMkOjr6kk2sXLlSmjVrJhcuXHAopvXr\n11foG6mcRRO4UjnYbDZ59dVXS7Be5tn2pwJpWWfbCQkJ0qdPf6lZs5Y8++yzLoi4aMePH5fQ0FDZ\ntm1b/g3sCXwHyHqQZiCrQL6OiJCGoaFy7tw5SUlJkcaNG8uaNWukWbNmsnLlyks2Ex0dLbNmzXI4\nrmuvvVaaNm2pd16WgiZwpXL46aefpF69enL69OlirZedwHva/81dLvn666+lYcOGcu7cOWeH7JCp\nDz0kt9Sunf+t73nurPzAGOlcv76kp6fLgAEDZPLkySIismDBAunRo4d88MEH0qlTJ0lPT8+1meIM\nT5CQkCCVK4cKVBOYqXdllpAmcKXyuPfee2X06NHFWie7hDJSICTfcknOZOhWCQmSUqWKNAb5sqBb\n33OMbZK+fLl06dJF3n//fdm5c6eEhobKH3/8IWlpadKuXTv5+OOPpUuXLvLuu+9esitHhyfIvsAb\nI/C43pVZQprAlcpj3759EhISIkePHi3WepkXLuvUqS+DBw++ZPmOHTuykqGj23JKecFeIlkA0t1e\n3y7s1ncRkTVr1kjTpk3l/Pnz8vjjj8szzzwjIiKfffaZNGjQQDp16ilVqlSVpUuX5lrP0eEJshP4\nCYGDmsD4XMAWAAAgAElEQVRLSBO4Uvl49tln5bHHHivRuoUl6pzJsCBO7z9uT+BpIC+BXHAggYuI\n3HTTTTJz5kz59ddfpVatWrJ371757LPPxMfHX+B+gfbi51f9ktief/55eeSRR9x7jBWUJnBVIRT3\njPaPP/6Q0NBQ2bFjR4n2V1Ci/vXXXyUkJET27t1b4LrZ9fRv7GXpUp6dOjp6YB6bN2+WOnXqyJ9/\n/iljxoyRe+65xx7bGIEwgXUCgRIZ2TfXeidOnJDQ0FDZunVrEWHp8LGlpQlclXslPdubMmWK9O/f\nv0T7LCxRZybDglhJ8g2BJmLNS+mE8kJR43cX4MEHH5TY2Fg5c+aMhISESFBQQ/v7+KBAokAvCQ+P\nuGQ9Vw5PoLJpAlflXna99YDAGocT4rlz56RRo0by5Zdflmi/BSXqM2fOSFhYmPz444/5rpf9B+cR\ngWYe6z8uInLo0CEJCQmRt99+W/z8AsWa5LhWVk+bKlVCJTAwUA4ePJhrvczhCT7//HOPxF1RaAJX\n5V52Av9WoKHA6w6f0c6fP1969OhRolvsC0vU//nPf+S6664rcLuZ/cerV68hd99996WlhhKeUZfE\n8OHDpX79JgJzBCIExgtcJSEhzSQhIUFiY2PlwQcfvGS9t956S1q2bCl9+vTXMomLaAJX5V7uEkoX\n8fW99MJbQdLS0uSKK66QxYsXl2jfBSXq1NRUiYiIkPHjxxdYB05ISJBmza4Q8BF4Oqv88/348Vk1\n7Yxi1LRL6uTJk+LnV9meuP8rcEOubzF//vmn1KlTRzZt2pRrveXLl4sxvgJP6IVKF9EEriqEzAtm\nPXtGSVBQkBw7dszhdZcvXy6XX365XLx4sVj7stkGyLJlyyQiIiLfxDVy5Eh7gnvzkgSX849OCI3l\nXowsJkB+oInsql5DBCQJ5D4HZsRxhkcffdQeazeBruLvXzPXMc2cOVNuvPHGXOtY33yeE2ghkKpd\nBV3AkQSuEzoorxcdHc2KFR/z5ZeJ3HfffYwfP97hda+//noaNGjA3Llzi2ybObt6UlI/kpL6cfvt\nD3HHHXcwbNgw0tPTc7X99tutiIQDG4COuWbPyZwZZxDfEs5BliMMJYX2HKDZ2VMA9ABWA985fCQl\nd9NNNyGSAUQC/wD8ci1/7LHH2L59O2vWrMmzZlvgYeAvN0Sp8lVUhi/tAz0DV27022+/Sa1ateTn\nn392eJ0ffvhBwsLC5MyZM4W2y663H7V3s5svffr0l+7du8uCBQvyaRsnECxwXa4zVJttgEQRI6kg\n7UF6gYSDzM484/bxEQF5A+QaHx9rLG8XsmJ9VKCLFDQ07nvvvSdXXnll1i322tfb9dAzcFXR1KlT\nh6eeeoq4uDjAOmuOihpIVNRAEhMT812nc+fOREZGMmPGDAf3sg24G0jj+PETTJs2jVGjRnH+/Pms\nFjExQwgImAOEA+vw9386a4qxmJghDPV5GT9gKrAeqAK0y1y5fXuw2Xjguus41rAhn6YVPHWa83QD\nBFiX79I777wTEWHRokWA9a1n8eIF2GxLsdmWsnjxAqKjo90Qp8qlqAxf2gd6Bq7c7OzZs1K/fn2Z\nNWuWw2eJe/fulZCQEPn1118L3G5CQoL4+9e2b+8KgapZ9eJbb71Vpk6dekn7Hj1s4u9TSVr7+Ul6\ncHDWhAnHOnaUzJtvZoE0BkkBEWNyXbRcunSptGnTRtLS0pzz5hRwXNb7NLvQ92nVqlVy2WWXOTys\nrCod9CKmqqhef/11CQ6ube/TLAK/F3mh7amnnpJ//OMfhW63Y8eeAlcJNBOoLPBvsdkGyPbt2yU0\nNFSOHz+ee4WEBHnEx0fCQN7JLJH4+4uMH2/9a3+tH8h0sF7PISMjQ3r16iVz584t8XvhCEfvnLzh\nhhvkpZdecmksyqIJXFVYFy9elGrVAgWeFWtQpVCBFwtN4MeOHZNatWrJzp07C2yTXQc/IuAvcF3W\nNocMGSLPPfdc3hXkCEgQSEOQ8zl7liQkWGfkISGyrVUrqVm1qkRG9r0kiX777bfSoEED+euvv0r3\npjjBxo0bpWbNmhIZebP2/3YxTeCqzHLHWBmjR4/O0ZXvBqlUKaDIfU2cOFEGDhxY4PLcF+9sAsg7\n77wjIiJH3n1XQipVkv01amTPLWkfbGocyAMgpwroGpiQkCCVKgVk9cPOW8a47bbbZOLEiaV4N5zD\nirOKQF+9eOlibkngwPXADmA3MCyf5W45WOU9xo8fLz4+wS7vwZCRkSGtW7eWNm06S2RkX6lRo0aR\ngzD99ddf0rBhQ/nmm28KbJP5x6d375ulcuXKcuutt1rJ2t9fRuXsv51PqSTr9TzHa53ZzxQIEWtY\ngNzlnl27dklQUJBcc81NHj3zteKMt8d5WPt/u5DLEzhQCdiDdandD6vTayvRBK4KkJCQID4+tezJ\nWxyqTZfG119/LY0aNZJz587J9OnT5eabby5ynblz50qvXr0cusV+6kMPiZ8xsqVSJRH7GXZt+/Rl\n+ZVK8s76nim7NLNc4Pgl70n2GbrNo2e+2XEuENirCdyF3JHAuwMJOZ4PB4aLJnBVACsBXCXWiHef\nCtQWiHdpEhgwYIBMmTIlaxCmtWvXFto+LS1N2rRpI//73/8K33BCgqT6+UktkG72hL0XJBDks2Le\nRVlUv2rrfZsl1mBTP3sscWr/b/dxRwK/DXgjx/O/Ay+LJnBVACsRxQjUEGhqr6X6uzQJ5JyE4e23\n35Zu3boVeXa9bNkyadWq1aW32Oc8mw4MFAF5D8QfZI09aQ8BiSmgVFKYwq4LZJ/5ThZ4y6NnvjrW\nt3s4ksCN1a5kjDEDgetF5BH7878D3UTknznayOjRo7PWiYyMJDIyssT7VN4t83b0lJR7gdcBqF7d\n8OWXX9K+fXuX7fcf//gHAQEBTJs2jc6dOxMXF8dtt91WYHsRoXfv3tx777088sgjmcFDv36Qmpq7\nLTAWeBqoCRytVIm2wI9z5xI+aJBT4s9+36YAEBAwTG+eKWeSk5NJTk7Oej527FhExBS6UlEZvrAH\ncBW5Syix5LmQiZ6Bqzwyz+A6d+4lYWFh8uKLL8r111/v0n1mTsKwb98+WbFihTRv3lxSU1MLXWfd\nunVSv359ObtkiVUKCQkRATkL8q79TFvyPuxn3aNGjZK///3vTj0GPfOtWHBDCcUX+BnrIqY/ehFT\nFdMNN9wg8fHx0qxZM1m5cqVL9zV69Gi59957RUTEZrPJK6+8UuQ6d7RtK+ONyZWkF4O0BQkD+R6s\nUkqeC5SnT5+WunXryk8//eTSY1Lll8sTuLUPbgB2YvVGic1nuVsOVnmnTZs2SZ06dWT+/PnSqVOn\nrMGSXOH06dNSr149+fHHH+Wnn36SunXryunTpy9tmNl/u2NH2Q0SArI0c3xu+7/d7f26extT4GBT\nr7zyithcPBSsKr8cSeClHsxKRD4TkctFpLmITCrt9lTFcsUVV3DjjTeyc+dOfH19Wbhwocv2FRgY\nyAsvvMCwYcPo2LEjNpuNadOm5W6UmAj9+0NSEqxfT3OsYaseBFbYm5jq1ZkaGMgqX19+qV+fhAKu\nIw0ZMoR9+/aRlJTksmNSFVxRGb60D/QMXBXh4MGDEhISIosWLZLw8HA5f/68y/aVOVtOYmKi7N+/\nX0JCQuTIkSPZDex3Th4CmW8/4z4Lssg+9Gt6jsGmbrnlFhk0aJCEh4cXOLXYokWLpEOHDi79ZqHK\nJ/RWeuUthg0bJg899JD07dtXXnzxRZfu6+OPP5b27dtLenq6xAwcKEMaNMi+4caewH8DqQXyc56y\nyVs5brPftm2bBAUF2W/XfyjfftEZGRnSrVs3efvtt116TKr80QSuvMbJkyeldu3asmTJEqldu7ac\nPHnSZfvKyMiQ7q1ayVtNmshxYyQUZBtY80/mmJNyLMhdPj7WxUmbTb6YPl0aN24sKSkpWdtq0CBc\nrKnIGgmcy7d/9tq1a6VJkya51lOqKI4kcJ3QQZUJNWvWJDY2ljlz5nDzzTczZcoUl+3LrFjB1J9/\nZuSBA1QVYShW/1dSUmDtWli8GGw2nu3dm5XVqtHNryZRBPJX27Z07NiRV155JWtbzZq1ArYAQwvc\n39/+9jfatWvHq6++6rJjUhVUURm+tA/0DFw56Pz58xIeHi6LFi2SkJAQOXTokGt2ZC+T3AIyDWsi\nhR/zufU9ISFB/PwCBVoJzJOAgLoye/bsXON+JyQkiK9vNYEehd5avnXrVqldu7acOHHCNcekyh30\nDFx5k8qVKzN+/HimTZvGkCFDeOGFF1y6v6lAB6zpzDoBBARATEzW8vj42Vy8OBPIAFqQkjKFRYsS\nGDBgAJMmWR2uoqOj+fDDt/H1/Z6goJG0bNk83321bt2aW265hcmTJ7v0mFTFoglclSl33303Fy9e\npGXLlnz66ads2bLF+TuJiYGAACKAPgA+PtCxo1U6ueTW9ErA51jzxFvGjBnDm2++yYEDBwCoWrUq\nxlTh9OmarF//CP37D8p3/s2xY8cyZ84cDh486PxjUhVTUafopX2gJRRVTElJSdK8eXOZPn263HTT\nTa7ZSWaPk8zeJ/k2KXjkvVGjRsl9990nIpkDTc0RaC6wotCBpkaMGCGDBg1yySGp8gUHSii+Hv77\nodQl+vTpQ7NmzfDx8WHbtm0kJyc7fwC06Oh8zrbzNrFmXo+Pnw1ATEz24FHPP/88LVq0YOPGjfbW\nvsAUYCsQXOA2hw4dSkREBJs2baJdu3YFtlPKIUVl+NI+0DNwVQIbNmyQunXryty5c+XKK6/MdSNM\nWRnU6eWXX5bo6Ohij5H90ksvuXzwLuX90IuYylu1b9+eqKgo9u3bh4iwaNEiIHtY1aSkfiQl9Suw\n3uwOQ4YM4eeff6ZSpUosXrwAm20pNtvSIod5feyxx9i1axerVq1yY7SqPCrVeOAO7cAYcfU+VPl0\n4MABOnXqxGuvvUZsbCzbtm2jb9+7MUlNuYdvCaMq8bRDbPtYseJjj8S4aNEiJk2axA8//ICPj+Pn\nQwsXLmTq1Kl8//33xVpPVRzGGKSI8cD1f44qkxITE3nkkWcJDKzFvHnziIiI4PXXX6fzH7/xHi8x\njK8II4nFzKLzH795LM7bbrsNPz8/3n///WKtd/vtt+Pj4+PSwbtUBVBUjaW0D7QGXm65qhadu6b8\nqvhjZEHTplLH11f+qFJFBOQlkBvtN98c69jRafsuieTk5BINwrV69WqpV6+eXHfdLR6v56uyB62B\nK1dxZS06Pn42KSlTuJaarOV1xiH8d98+bkhL48Xz5wF4DKtndjoQGhrqlP2W1DXXXEPbtm3597//\nXaz1UlNT+f33E6xaVdXj9XzlnbQGrkokKmogSUn9gG7ABOA6bLZPnFKLjooaiElqSgtmUo10xgKX\nAzOAx7FmDgnKbBwQUMANOO61detWevfuza5du6hZs6ZD61jvYUfgZWAXsASbbanH6vmqbNEauHKD\nZsD3WAM6OUdMzBCG+rxMLOnMAY4B44HvsHpZZyXvkJAykbwB2rRpQ79+/Upwq3wjoB+w0gVRqXKv\nqBpLaR9oDbxcyl2n/qcY4yuffvppaTaY687IYx07ioCMAHkhx3RmWY+AgALvoPSUw4cPS0hIiBw8\neNCh9tnv4TyH+o+rigUHauBaQlEllpiYSHz8bESEI0d2MmzYMO6///6SbMiaxiwlxXoeEABxcTBh\nAhdSUvADq6td06YQFAShodZ4JmXgzDuvuLg4jhw5wrx58xxqn/kegvXNo7D+46picaSEoglcOcVX\nX33FPffcw86dO6lSpUqhbS9JWvHxkJTEX0A1e5s/OnYkdNIkiI/H3rBMJuy8Tp06RUREBCtXruSK\nK67wdDjKi2kCV27Vv39/evbsyXPPPZfv8sTERGJjJ3HD+u94BgEq8bIv/POKCKqtX08E8CXQBFjl\n40/a8qVeeUY6a9YsEhISWL58uadDUV5ME7hyqx07dtCrVy927txJSEhIrmWZ3Q5fTWnOA3zFfcAT\nwFXAirDGXPPbL0zMSGc/8BoB9OcfHr3DsjRSU1Np1aoVc+bMoXfv3p4OR3kp7YWi3Kply5YMHDgw\na7KDnOLjZ9Mr5e805CteAWoCmefp3Y8dZWz7q+hESz6jMjcyhxV4b/nB39+fCRMmMHToUDIyMjwd\njirHNIErpxo9enSuyQ5yiiGBZsAYYBPWjTgGqBJQhchJo7gn4CR/MIu1XCQgYBgxMUPcGrsz3XHH\nHQB8+OGHHo5ElWdaQlFO98K997J/zRreats26+JjYmIivjf247qMVDYAaVi9n3cCgePHQ1xcueuR\nsWbNGh5++GG2b9+Ov7+/p8NRXkZr4Mr9EhM5c/PNRFy8yGdAB39/WLoUoqP5YcIE2r4wmioZ6QDc\nA7Ts0IEX1q/3aMiudOONN3L99dfzr3/9y9OhKC+jCVy5X6dOsH49s4BPgUSw5pv86SdreWJiVtfA\nfffey5XPPsu2bduoW7euhwJ2rc2bN9OnTx927dpFjRo1PB2O8iKawJX71apF6okTtAGiscYv8Q8J\ngePH823+zDPPkJqayquvvurOKN0qKiqKPXsO0rx5m3JRGlLu4dJeKMaY240xW40x6caYTiXdjipn\nmjTBH7gLOAP4218ryMiRI/nwww/ZtWuXe+Jzs8TERD7/fD379h0mKamXjjionKo0vVA2A/2Bz50U\niyoPJk0Cf3+exyqfbPT1tV4rQK1atYiJiWHEiBFuC9Gd4uNnc+HCdOBJYAspKVOyLtQqVVolTuAi\nskNEyudpkyq56GhYupQgm42Rl1/OsPbti7wF/qmnnmLdunV8++23bgrSE4YDdQAtJyrn0X7gyvmi\no2HFCoZs2sSeP/8scvLegIAA/u///o+HH34Ym20AUVEDy02ZISZmCAEBw4D/AZcTEDDcq/u3q7LF\nt7CFxpgkoF4+i0aIyCeO7mTMmDFZP0dGRhIZGenoqsqL+fv7M3HiRIYOHVrk5L1169Zl27adbN3a\nB+jIl18OKnJ2d28QHR3N4sULcvRv9/5jUq6RnJxMcnJysdYpdS8UY8waIEZEfipgufZC8SY5uvk5\nYwRAEaFbt248/fTT3HPPPQW2s2anaQJ8hnV55V2dnUZVaO4cC6XQnSjXSkxMJCpqYOlLD5njcicl\nWY/+/a3XSsEYw7Rp04iLi+PChQtFtG4HPA1cLNU+laowiprxoaAHVg+UQ0AK8CvwWQHtSjMphSpC\n7plxSjmri80mYp/95njmzDc2m1Pi7Nu3r8yYMaPA5U49DqXKAXRGnvIve3JhG9aZa3LJSw9RUZCU\nxJfAo8BGwNdmgxUrSh2nI5P+lrexUJQqDR1OtkJZCBQ83kbOMssPEyZYyToqKneJJCYGAgLoCYQC\nC/z8rNecoE2bNtx8881MmTKlwDbR0dGsWPExK1Z8rMlbKUcUdYpe2gdaQnGp7NLDGwKh4u8ffEnp\nIWd5IooY+auwyYHtkwt/27WrNAgNlb/++stpsR46dKhYk/4qVZGhJZSKIbP0cPToQdLSzrJt2zaM\nyf7mFRU1EJLC8Wc+b3CKMNJJwT4WN0ABZZLbb7+dTp06ERsb67RYR4wYwa+//sqbb77ptG0qVR7p\nYFYVTEZGBldeeSUjRozgtttuy3o9ttPVjF6/jmdJowrWAFOPAM2BYVBgAt+9ezfdu3dnx44dhIaG\nOiVGnfRXKcdoAq+AkpKSeOKJJ9i6dSt+fn4A/NGpE6Hr1/Ml1qXO3cBfwNXAjipVqLVkSYH9vZ98\n8kl8fX2ZOXOm02J86aWXePfdd6lZsxGgFyyVyo9exKyAbDYbTQMDeaNdu6yLlJlnz+2BQOAYcDlw\nW+XKTLjhhkJv1nnhhRd4++232bt3r9NivOyyy/jhh59ISmpBUlI/HaFPqRLSBF7eJCYyZetWxu3Y\nwZnMm3GuuQb8/QkERmMNq4S/P6PnzWPB2rXs37+/wM3VqVOHp59+mri4OKeF+PLL8xF5BFgF3Kcj\n9ClVQprAy5v4eDpcuEAfrFo3KSmwdq01rVnHjgwJDmZf5cokjRlDvbvv5p///CcjR44sdJPPPvss\na9eu5YcffnBioF2Byli3zSulSqSobiqlfaDdCN3LfjflUZDfC7ibctGiRdKxY0dJT0+X06dPS716\n9eTHH38sdLOvv/669O7dWzIyMkodYna3xjf1rkulCoAD3Qj1DLy8sd+MUw+oDRAQcMnNOAMHDsTf\n35/333+fwMBAXnjhBYYOHZr5BzdfgwcP5ujRoyQkJJQ6xMwR+my2ZdhsS8vFqINKeYL2QimPHBhR\n8PPPP+f+++9nx44dVKpUibZt2zJr1qxCE+mSJUsYNWoUGzZsoFKlSq6KXimFdiNURejXrx+RkZE8\n++yz/Pe//+X555/nssvaY4zJt2ufiNCrVy8efvhhHnjgAc8ErVQFoQlcFWrbtm1ERkayc+dO1q1b\nx0039SMj4wGgJwEBw/ItbXz99dfcdddd7Ny5k4CAAI/ErVRFoAlcFemRRx4hJCSE9ev3kJTUEngH\n2AksLHBUwwEDBtCtWzeGDRvm7nCVqjD0Rh5VpLFjxzJnzhxSUs4BEcBdwJFC15k0aRLTp0/n+PHj\n7ghRKVUATeAVXP369Xn88cepUiXNPvlua+ALAgKGFTj57uWXX85tt93GhAkT3BqrUio3LaEoTp8+\nTYsWLRgzZgyLF68Eih6f5Ndff6VNmzb8+OOPhIeHuylSpSoOrYErh7388st89tlnLF++3OF1xowZ\nw549e3jnnXdcGJlSFZMmcOWw1NRUWrduzezZs7n22msdWufMmTM0adKEiIgOBAUF66iCSjmRXsRU\nDvP392fChAkMHTqUjIwMh9b5+uuvOXs2jXXrfiMp6WYdVVApN9MErrLcfvvt+Pj48OGHHzrUPj5+\nNhcvvgikAWE6qqBSbqYJXGXx8fFh6tSpjBgxggsXLji4li8wEZjvusCUUvnSBK5yiYyMpHXr1rz2\n2mtFto2JGWLvengGiCq066FSyvn0Iqa6xJYtW7juuuvYtWsXNWrUKLRt5oTKoFOjKeVM2gtFldhD\nDz1E3bp1mThxoqdDUapC0gSuSuzw4cO0b9+ejRs30rBhQ0+Ho1SFowlclUpsbCy///47c+fO9XQo\nSlU4msBVqZw6dYqIiAhWr15NmzZtPB2OUhWKS2/kMcZMM8ZsN8ZsNMb81xhT+NUu5XVq1KhBbGws\nw4cP93QoSql8lKYb4QqgjYi0B3YBsc4JSZUljz/+OFu3bmXt2rWeDkUplUeJE7iIJIlI5j3X6wC9\n0lUOVa5cmQkTJvD8888XOumxUsr9nHUjz2DA8WHsPCwxMZGoqIFERQ3UsTsccOedd5KRkcGiRYs8\nHYpSKodCL2IaY5KAevksGiEin9jbxAGdRGRgAdsoUxcxExMT6d9/ECkpk4EMAgJG5Dv3o8pt9erV\nDBkyhG3btuHv7+/pcJQq91zeC8UY8wDwCHCdiJwvoI2MHj0663lkZCSRkZEl3mdpRUUNJCmpH7AF\nqA+EFDj3o8rthhtu4KabbuLJJ5/0dChKlTvJyckkJydnPR87dqzrErgx5nogHrhGRP4opF2ZOgPP\nTuBX4kdPPiCMJiHH6fze26Bn4YXatGkTUVFR7Nq1i6CgIE+Ho1S55tIzcGPMbsAfOGF/6RsR+Uc+\n7cpUAs8sofRK+TthzKAewmSAgABYvFiTeBEeeOABGjVqxLhx4zwdilLlmt7IU4DExERC77mPeieO\n0Q7YADQCsNlgxQrPBlfGHTx4kLZt29Khw9VUqRKgA1gp5SI6I08BoqOj6dy5Aw2A4cAOTwfkRbZv\n305KSjpffHGOpKR+OguPUh5UIRM4ADExEBDA84ANrBJKTIyHgyr74uNnk5Y2DdgGdNVZeJTyoIqb\nwKOjrZq3zWY9tP5dDNWA6UCB166VUm5QIWvgquSy+9FPASAgYJj2o1fKBfQipnIJnYVHKdfTBK6U\nUl5Ke6EopVQ5pglcKaW8lCZwpZTyUprAlVLKS2kCV0opL6UJXCmlvJQmcKWU8lKawJVSyktpAldK\nKS+lCVwppbyUJnCllPJSmsCVUspLaQJXSikvpQlcKaW8lCZwpZTyUprAlVLKS2kCV0opL6UJXCml\nvJQmcKWU8lKawJVSyktpAldKKS+lCVwppbxUiRO4MWacMWajMWaDMWaVMaaRMwNTSilVuNKcgU8V\nkfYi0gFYAox2Ukwul5yc7OkQLlEWY4KyGZfG5BiNyXFlNa6ilDiBi8iZHE+rA3+UPhz3KIsfVlmM\nCcpmXBqTYzQmx5XVuIriW5qVjTETgPuAc8BVTolIKaWUQwo9AzfGJBljNufzuBlAROJEpDEwH3jR\nDfEqpZSyMyJS+o0Y0xhYLiJt81lW+h0opVQFJCKmsOUlLqEYY1qIyG7701uA9SUJQCmlVMmU+Azc\nGPMRcDmQDvwMPC4ivzsxNqWUUoVwSglFKaWU+7nlTkxjzBhjzGFjzHr743p37NcRxpgYY0yGMSak\nDMRS5m6OMsZMM8Zst8f1X2NMjTIQ0+3GmK3GmHRjTCcPx3K9MWaHMWa3MWaYJ2PJZIx50xjzmzFm\ns6djyWSMaWSMWWP/3LYYY/5VBmKqYoxZZ/9922aMmeTpmDIZYyrZc+UnhbVz1630AswQkY72R4Kb\n9lsoe4K0AQc8HYtdWbw5agXQRkTaA7uAWA/HA7AZ6A987skgjDGVgFeA64HWwN3GmFaejMluHlZM\nZclF4BkRaYPV5fgJT79XInIe6G3/fWsH9DbGXO3JmHJ4CtiGlTsL5M6xUMrixcwZwFBPB5GpLN4c\nJSJJIpJhf7oOaOjJeABEZIeI7PJ0HEBXYI+I7BeRi8AHWBf0PUpEvgBOejqOnETkVxHZYP/5LLAd\nqO/ZqEBEztl/9AcqASc8GA4AxpiGwI3AHIrIm+5M4P+0fw2fa4yp6cb95ssYcwtwWEQ2eTqWnIwx\nE9eCc9EAAAJXSURBVIwxB4FBwGRPx5PHYGC5p4MoQxoAh3I8P2x/TRXCGBMOdMQ6IfAoY4yPMWYD\n8BuwRkS2eTomrHtqngcyimpYqjsxczLGJAH18lkUB/wH+D/783FAPPCQs/ZdwphigaiczV0dTxEx\njRCRT0QkDogzxgzH+iAf9HRM9jZxQKqIvOfqeByNqQzQHgDFZIypDnwEPGU/E/co+7fLDvZrO4nG\nmEgRSfZUPMaYvsDvIrLeGBNZVHunJXARsTnSzhgzB3DLL2BBMRlj2gJNgY3GGLDKAj8aY7q6uiuk\no+8T8B5uOtstKiZjzANYX+muc0c8UKz3yZN+AXJeaG6EdRau8mGM8QM+Bt4RkSWejicnETlljPkU\nuBJI9mAoPYB+xpgbgSpAkDHmLRG5P7/G7uqFEpbjaX+si1AeIyJbRKSuiDQVkaZYv3SdPN2P3RjT\nIsfTAm+Ocid7j6HngVvsF33KGk9eW/kBaGGMCTfG+AN3Aks9GE+ZZawzpbnANhGZ6el4AIwxoZnl\nXGNMAFaHBo/+zonICBFpZM9LdwGrC0re4L4a+BRjzCZjzEbgGuAZN+3XUWXlq/Ak+1gzG4BIIMbD\n8QC8jHVBNcnerenfng7IGNPfGHMIqzfDp8aYzzwRh4ikAU8CiVg9BhaKyHZPxJKTMeZ94Gsgwhhz\nyBjj8jKcA3oCf8fq6VFWuhOHAavtv2/rgE9EZJWHY8qr0NykN/IopZSX0inVlFLKS2kCV0opL6UJ\nXCmlvJQmcKWU8lKawJVSyktpAldKKS+lCVwppbyUJnCllPJS/w9cpsDO848j7gAAAABJRU5ErkJg\ngg==\n",
       "text": [
        "<matplotlib.figure.Figure at 0x106e02b10>"
       ]
      }
     ],
     "prompt_number": 31
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "Now we have found a reasonable choice for our first best fitting line, we have a set of remaining vectors that we have not explained.  These are the vectors between the projected and actual points."
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "remaining = X - projected"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [],
     "prompt_number": 32
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "plt.scatter(remaining[0], remaining[1], label='remaining')\n",
      "plt.arrow(0, 0, u_best[0], u_best[1], width=0.01, color='r')\n",
      "plt.annotate('$\\hat{u_{best}}$', u_best, xytext=(20, 20), textcoords='offset points', fontsize=20)\n",
      "plt.legend(loc='upper left')\n",
      "plt.axis('equal')"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 33,
       "text": [
        "(-1.0, 0.60000000000000009, -1.5, 2.0)"
       ]
      },
      {
       "metadata": {},
       "output_type": "display_data",
       "png": "iVBORw0KGgoAAAANSUhEUgAAAXcAAAEACAYAAABI5zaHAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3Xt8VeWV//HPgnAJGgG5iNxEuYn1UhxL+dVWT60EZBws\nL2vRimRoK8XiD5RY8YKSjiCigoiC4owi3sbqINWKeIiOQaBKVRDxAgUKFlRUCHcit6z5Yx9CgCQk\nOck5Jzvf9+uVl2fv/Zy91za6eFj7eZ5t7o6IiIRLnWQHICIiVU/JXUQkhJTcRURCSMldRCSElNxF\nREJIyV1EJITiSu5m1s7M3jKzT8zsYzMbXkq7KWa2ysyWmVn3eK4pIiLHlhbn9/cBN7r7h2Z2PPCB\nmeW6+2cHG5hZX6CTu3c2sx8CjwA947yuiIiUIa6eu7tvdPcPY593Ap8BrY9o1g+YGWuzGGhiZifF\nc10RESlbldXczawD0B1YfMShNsD6YtsbgLZVdV0RETlalST3WEnmf4ARsR78UU2O2NaaByIi1Sje\nmjtmVg+YBTzj7n8uockXQLti221j+448jxK+iEgluPuRHei4R8sY8DjwqbtPLqXZK8CgWPuewFZ3\n/7qUAEPzM2bMmKTHoPvX/ev+w3/vpYm3534+MBD4yMyWxvbdBrSPJevp7v6amfU1s9XALmBwnNcU\nEZFjiCu5u/tCytH7d/fr47mOiIhUjGaoVpNIJJLsEJJK9x9JdghJVZvvP1Xu3cqq2SSSmXmqxCIi\nUlOYGV7CA9W4R8tUt+CZraQi/WEskrpSPrmDkkgq0h+6IqlNNXcRkRBSchcRCSEldxGREFJyr2HG\njx/PtddeW+VtRSRcUn4oZGyYTxIikrLo9yKSGmrsUMjKcndeeOEFPv74E7p1O50rr7ySOnWq5y8q\n+/fvJy0ttP8qRaQGqrFlmS1btvDoo48yefJkVq1addTxX/96GL/5zb2MHWsMGfIgV1/92yrtaXbo\n0IF7772Xs88+m4yMDBYtWsSPfvQjmjZtyve//33mz59f1DYSiXDHHXdw/vnnk5GRQb9+/di0aRNX\nX301jRs3pkePHnz++edF7UeMGEH79u1p3Lgx5513HgsXLiw6lpOTwzXXXAPAunXrqFOnDk899RSn\nnHIKLVq04O67765U24KCArKysjjxxBM544wzuPfee2nXrvhiniJSoyR7BbViK5t5SUra/8033/jJ\nJ3f09PRfeoMG1/lxxzX3d955p+j42rVrvWHDFg7bHdxhlzdq1NpXrFhR1Gbfvn0+bNhIz8ho6See\n2NYnTXqwxOuX5pRTTvHu3bv7hg0b/IsvvvBmzZr53Llz3d09NzfXmzVr5ps2bXJ39wsvvNA7d+7s\n//jHP3zbtm1+xhlneKdOnfzNN9/0/fv3+6BBg3zw4MFF537mmWc8Pz/fDxw44BMnTvRWrVr5nj17\n3N09JyfHBw4cWHSfZuZDhgzx7777zpctW+YNGjQous+KtB01apRHIhHfunWrb9iwwc866yxv165d\nqfdf2u9LRBIr9v/iUTm1RvbcJ058kG+/7UVBwZ/Ys2cau3Y9wPXX31Z0fPv27dSr1xzIiO1pRFra\nSWzbtq2ozZ13jmXGjA/YseM98vPnMnr0Q7z44v+UOwYzY/jw4bRp04ann36avn370qdPHwAuvvhi\nzjvvPObMmVPUdvDgwZx66qmccMIJXHLJJXTp0oWLLrqIunXrcsUVV7B06dKic1999dU0bdqUOnXq\nMHLkSPbs2cPKlSuBkid0jRkzhgYNGnD22WdzzjnnsGzZsgq3ffHFF7ntttto3Lgxbdq0YcSIEaqp\ni9RgNTK5f/NNPvv3dyu2pxubN28u2uratSsZGfuoU2ci8AVmD9Gw4Ra+973vFbV56aW57N49jmB1\n4jPZvfsmZs2aW6E4DpYtPv/8c1588UWaNm1a9LNo0SI2btxY1Pakkw69NrZhw4a0bNnysO2dOw+9\nwOr+++/njDPOoEmTJjRt2pRt27axadOmUuNo1apV0edGjRoddq7ytv3yyy8PK8O0bas3IYrUZDUy\nuffrl0mjRlOAT4FvSE+/g0svzSw63qBBAxYsiHLeea9xwgn/Qvfus1iwIMpxxx1X1KZZs6bAoVp9\nWtoqWrRoUqE4Dk7Bb9++Pddccw1btmwp+tmxYwc333xzmd8ryYIFC7jvvvt48cUX2bp1K1u2bKFx\n48bV3os++eSTWb/+0Ktui38WkZqnRg7x+PnPf864cevJyfkZe/cW8ItfDOD++8ce1ua0005j8eI3\nSz3HAw/8Bxdd9K/s2bOEunW3kZHxFrfc8k6l4hk4cCA/+MEPmDdvHj/72c/Yt28f7777Lp07d6ZN\nmzbA4SWSshL1jh07SEtLo3nz5uzdu5d77rmH7du3VyquivjlL3/J+PHj+cEPfsCuXbt4+OGHtX6M\nSA1WI3vuADfc8P/ZuvUrdu/eylNPTadBgwYV+n6PHj1YsmQR48a15Z57uvPJJ+8XJeKKatu2LS+/\n/DJ33303LVu2pH379kycOPGwJF48UZrZUYnz4HafPn3o06cPXbp0oUOHDqSnp9O+fftSv1tWAq5I\n2zvvvJO2bdty6qmnkpmZyRVXXEH9+vXLcfcikoo0iUlK9Mgjj/DCCy/w1ltvlXhcvxeR1FDaJKYa\n23OXqrVx40YWLVpEYWEhK1euZNKkSfTv3z/ZYYlIJdXImrtUvb179zJ06FDWrl1LkyZNuOqqq/j9\n73+f7LBEpJJUlpFK0e9FJDWoLCMiUosouYuIhFDcyd3MnjCzr81seSnHI2a2zcyWxn5Gx3tNEREp\nW1U8UJ0BPAQ8VUab+e7er7IX0GQaEZGKiTu5u/sCM+twjGaVzs56aCciUnGJqLk78CMzW2Zmr5nZ\nGQm4ptQQ0WiUzMzLycy8nGg0muxwREIjEePclwDt3H23mV0C/BnoUlLDnJycos+RSIRIJJKA8CRZ\notEo/ftnUVAwAYCFC7OYPXsmvXv3TnJkIqkrLy+PvLy8Y7arknHusbLMX9z9rHK0XQv8i7vnH7G/\nxHHuEl6ZmZeTm9sPyIrtmUmvXq8wb96sZIYlUqMkbZy7mZ1ksSeiZtaD4A+U/GN8TURE4hB3WcbM\n/hu4EGhuZuuBMUA9AHefDvwCuM7M9gO7gSvjvaaEQ3b2EBYuzKKgINhOTx9FdvbM5AYlEhIpv/yA\nhFs0GmXixMeAINmr3i5SMaWVZZTcRURqMK0tIyJSiyi5i4iEkJK7iEgIKbmLiISQkruISAgpuYuI\nhJCSu4hICCm5i4iEkJK7iEgIKbmLiISQkruISAgpuYuIhJCSu4hICCm5i4iEkJK7iEgIKbmLiISQ\nkruISAgpuYuIhJCSu4hICCm5i4iEkJK7iEgIKbmLiIRQ3MndzJ4ws6/NbHkZbaaY2SozW2Zm3eO9\npoiIlK0qeu4zgD6lHTSzvkAnd+8MDAEeqYJriohIGeJO7u6+ANhSRpN+wMxY28VAEzM7Kd7riogU\n9/DDDzNt2rRkh5EyElFzbwOsL7a9AWibgOuKSC0xbdo0li5dyvvvv8/06dOTHU5KSEvQdeyIbS+p\nUU5OTtHnSCRCJBKpvohEJBSeeeYZNm3axOOPPw7AmDFjeP7557nyyiuTHFn1yMvLIy8v75jtzL3E\nPFshZtYB+Iu7n1XCsUeBPHd/Pra9ArjQ3b8+op1XRSwiUrsUFBSQnp5+zH1hZWa4+5Ed6ISUZV4B\nBsWC6AlsPTKxi4hUVklJvLYk9rLEXZYxs/8GLgSam9l6YAxQD8Ddp7v7a2bW18xWA7uAwfFeU0RE\nylYlZZmqoLKMiKxZs4bJkyezbt06Bg4cyIABA4qOTZkyhblz5zJ37twkRph6klmWERE5psLCQu6/\n/34mT57MxRdfzN13333Y8ZkzZ9KyZcskRVfzKLmLSEp46623yMzMpG7durz++ut07dq16NiOHTv4\n6KOPuPDCC5MYYc2SqKGQIiJl6tatGy1atOCLL74gNzeXWbNmFR1btGgRBw4cUHKvACV3EUkJrVu3\nBuCFF14gIyODvn37Fh17++23ad26NR07dkxWeDWOyjIiklKi0Sg//elPqVevXtG+t99+mwsuuACA\ntWvXAjB16lROOOEEFi9eXCXXXbVqFX/961+r5FypQMldRFLK+vXrD6u3FxQU8MEHH/CTn/wEgIkT\nJwIwaNAg0tPT6dGjR5Vcd8KECXz22WdVcq5UoOQuIimlc+fObN68uWh77Nix7N+/nw4dOvDxxx/T\nrVs3IHgAe8EFF2B21CjASnnjjTfIzMysknOlAtXcRSSlPPDAAwwdOpQRI0bg7lx77bU0btyYhx56\niI4dOxb13OfNm4eZ8dxzz/H2228zbNgwzjrrLAoLC3nggQdIS0ujSZMm5Ofnc+ONNwIwZ84c1q5d\ny/HHH4+ZkZWVxauvvsrcuXM5cOAATz31FBdccEHR3xJqNHdPiZ8gFBGR8unatau/88477u7+6quv\ner9+/dzd/be//a1PmDDB3d137tzpt956q7u7b9++3c855xx3d//222/9kksuKTrXk08+6UOHDk1k\n+FUmljuPyqnquYtIjfPPf/6TwsJCevbsCcDGjRv59ttvWblyJc8++yzTp0/n2WefpaCggFtuuQWA\nevXqsW3bNr7//e/Tq1cvnnzyyaLz5eXlHTY6Jwy0/ICI1DgvvfQSc+bMKVrmd8CAAZx55pl07dqV\n++67j/fee++o77g7e/bsIRqNMmXKFH74wx8WzYLt1KkT7777Ls2aNSM/P59mzZol9H7ioeUHRCQ0\nGjduTJMmTYBgCOPy5cvJzs6mS5cuNGzY8LC2jz/+OGvXrqVFixbUrVuXyy67jOuuu462bYN3Bm3e\nvJl69erRvHlznn76aQoKChJ+P9VBPXcRqXHcnZtvvpkzzzyTv/3tb9xxxx20atUKgMmTJ2NmNG/e\nnF27dnHppZfSuHFjpk6dSps2bdi9eze7du3ihhtuAII1bQYOHEivXr1o27YtvXr1SuatVVhpPXcl\ndxGRGkxlGRGRWkTJXWqlcePG0axZJ5o168S4ceOSHY5IldNQSKl1xo0bx+jR9wJTABg9ejgAt99+\nexKjEqlaqrlLrdOsWSfy8+8AsmJ7ZnLiiXexefPqZIYlUimquYuUoaBgV7JDqF3274dVq0Adumqj\n5C61zsiRg4HhwMzYz00UFGxX7b26fPMNvPEGTJwIV1wBnTpBRga88w5U0aJfcjSVZaRWatSoJQUF\nHYHWwBBgo0oz8fruO/j0U1i+HN5/HxYvhhUrYO9eaNgQCgqCz3XqwH33wciRyY44FEory+iBqtRK\n6eknUFAwlOJ1dyknd/jnP+Gjj+DDD+Hdd2HZMvj6a2jUCAoLYefOw7+zZ0/wz0aN4IYblNgTQMld\naqWRIwcXjZIJDGfkyJuTFk/Ke+YZ+N//hffeg9Wrg953Whrs3h3Uzw/avr30czRqBNdcA2PHVn+8\nEn9yN7M+wGSgLvBf7j7hiOMR4GXgH7Fds9xdv11JqoPDHidNuguAkSNv1lDIsqxYAU8/fXgir4hG\njeDf/g2mTVOdPUHieqBqZnWBh4E+wBnAVWbWrYSm8929e+xHiV1Swu23387mzauL6uya1FSGUaPg\nuOMq9930dPjxj4Pefx2N4UiUeP9N9wBWu/s6d98HPA9cVkI7/VEtKevgpKb8/DvIz7+D0aPvVYI/\nUkYGjBtX8QTfoAGcfTa8/HJQxpGEiTe5twHWF9veENtXnAM/MrNlZvaamZ0R5zVFqtSkSTMIZqtm\nxX6mxPbJYYYMgdgyu+VSv34w7DE3NxgtIwkV7x+l5Rm7uARo5+67zewS4M9Al5Ia5uTkFH2ORCJE\nIpE4wxORKlOvHjz4IGRlwa5jTPqqWxdat4b584Nev1SZvLw88vLyjtku3uT+BdCu2HY7gt57EXff\nUezzXDObZmYnunv+kScrntxFEkUjZ8ppyxbIyzt2YjeDFi1g0SKoQW80qimO7Pj+8Y9/LLFdvMn9\nfaCzmXUAvgQGAFcVb2BmJwHfuLubWQ+CiVNHJXaRZNHImWP47rugxx57FylNmgSTkXbvLrl9kyaw\ncGHQc5ekiXuGaqzUcnAo5OPuPt7Mfgfg7tPNbBhwHbAf2A2MdPd3SziPZqiKpJLCQnj2WRg06NC+\nF16AX/wCMjODce+FhYd/JyMD/vpXOPPMxMZai+lNTCJSfrm5cPnlsCNWVX3wQbjuuqDuDsG49+7d\ng179QccdF6wh07Nn4uOtxbQqpEgFRKNRMjMvJzPzcqLRaLLDSZylS4Ned2ZmkNhvuSWYdTp8+KHE\nDnD66cEiYAf3pafD7NlK7ClEPXeRI0SjUfr3z6KgIJhsnZ4+itmzZ9K7d+8kR1aNPv8chg2DOXOC\n7V/9Cu6/H04+ufTvfPUVdOwYfH766aCnLwmnsoxIOWVmXk5ubj+OfJnHc89NDV+Cz8+H0aPhkUeC\n7fPPh//6r6BnXh633QannQa//W31xShlUllGJA75+S3o3z8rPCWaggK4++5gqOIjj0CbNsEIl4UL\ny5/YIVgETIk9JannLnKEI8syMIpgSeCN9Or1CvPmzUpidHE6cACeegp+/etD+2bNgv79taBXDaWy\njEgFRKNRfvWrYeTntwAuJZho/SXdu9dlyZKFSY6uEtwhGg3q4gfHp0+dCtdee/iDUqlxVJYRqYDe\nvXvz3HNTqV9/BcE0jn7AUD755O81rzTzwQfQrRtcckmQ2G+/PRgJ8/vfK7GHmHruImU499wIS5cO\npvjD1e7dZ7BkSV4SoyqntWuDsekH/zC65prg9XYnnZTcuKRKqecuUgnNmx+9NsqyZR+ndu990yb4\n3e+CUSzRKFxwAaxcGdTaldhrDSV3kTJkZw+hTp0bCR6ozgSyKSw0Lr10UOqt+b57N9x1V7Bo12OP\nQfv2wVIA8+dDlxIXYpUQU1lG5BjOPffHLF16APgO+DswLXZkOGPHpsAiYwcOwIwZwcNRCN52NGsW\nXHaZRsDUAhotI1JJh4ZG1gPGAq2Ax4AvOf74z9mx48vkBOYOr70WjIDZsyfY9+ij8Jvf6K1HtYhq\n7iKV1Lt3b2bPnkla2l5gOXAlwQrXsHPntuTU3997Lyi1XHppkNjvvBN27gxq7UrsgnruIuUWvGt1\nPJAO3B/bexMnn3wCX365JjFBrFkTJPA33wy2Bw+Ge+6Bli0Tc31JOSrLiFSBRo3aUFBwN8WHRsKN\njB2bXb2192+/DVZofOKJYPuii2D69OAdpVKrqSwjUgVOP71zCXvbVN8LtXftgpycoGf+xBNw6qnw\n7rtBz12JXcqg4pxIBYwffyt9+gwotucPwCD27Xuhai+0f3+QzH/3u2C7Xj146SX413/VCBgpF/Xc\nRSqgd+/eZGX9HLgBeBQYBMykoGB71TxYdYeXXw5efnEwsT/2WDCG/dJLldil3FRzF6mETp3OZs2a\n4wheHZwG7Il/UbHFi+HKK2HdumD7j3+Em26CRo3iD1hCSzV3kSp02mmdgfOB1cBgYChLl35cuVmr\nq1ZBJBK8om7dumAy0rffBsMbldilkpTcRSohWJbgSWACwciZVkA3Ro+eVP4E/8038O//HoxXnz8/\neG/pmjVBGaZ58+oKXWoJlWVEKunQipGtgAFAt9iRj3j99ZdKfyXfzp0wYULwFiOAzp3huefgvPOq\nP2gJHY1zF6li0WiUvn2vorDwBGAXQQ9+EfB3jj/e2LFj0+Ff2L8f/vM/g3XUARo2DEbA9OmjB6VS\nadVWczezPma2wsxWmdmoUtpMiR1fZmbd472mSCro3bs3//Ef2cB2gsT+eOxIF3buLDg0esYdZs+G\n+vUPJfbHHw968JdcosQu1SKunruZ1QVWAhcDXwDvAVe5+2fF2vQFrnf3vmb2Q+BBd+9ZwrnUc5ca\nyawpQWlmE8WXJejevStLHr4XBgyADRuC3XfdBdnZwVBHkSpQXT33HsBqd1/n7vuA54HLjmjTj2CO\nNu6+GGhiZnpjgITGxRefB3xNkNizgCy6kM205R/A+ecHiX3o0OAlGqNHK7FLQsSb3NsA64ttb4jt\nO1abtnFeVyRl5ObmkpFxKGFP5TpWcis9938XlF3WroVHHoFmR7/VSaS6xLv8QHnrKEf+laHE7+Xk\n5BR9jkQiRCKRSgUlkmgvvvgE/fpdw969sIPPWUsdvnroQX50/fXJDk1CJi8vj7y8vGO2i7fm3hPI\ncfc+se1bgUJ3n1CszaNAnrs/H9teAVzo7l8fcS7V3KVGi0ajTJz4GHUKC/lJ5F+Y//ZSIBgTX+qw\nSJE4VctQSDNLI3ig+jOCtxf8jbIfqPYEJuuBqoTZoTc3BX2c9PRRzJ49UwleqkVpyT2usoy77zez\n64EowSIbj7v7Z2b2u9jx6e7+mpn1NbPVBIOBB8dzTZFUN3HiY7HEHqz5XlAQ7FNyl0SKe8lfd58L\nzD1i3/QjtlV4FBFJIK0tI1LFsrOHkJ4+imAE8ExgBO+8807lFhUTqSQtPyBSDaLRKMOG3cKaNeuA\n3wBnAcPJyurPk08+mdTYJFy0toxIgjVr1on8/Ds4/H2rN/D668+r/i5VRuu5iyRVlODNTcbgwUOS\nHYzUAkruItVk5MjBwHDgJmAgMBR4gK++2qr6u1Q7lWVEqlHr1h356qu9wFiKl2cyMu5k+/bPkxiZ\nhIXKMiJJMGPGNILpHYcrKPgu8cFIraKeu0g1a936FL76aiswJbZnOB07tmX16k+SGZaEhHruIkky\nY8ZjpKUdAEYDo0lLO8DUqZOSHZaEnHruIglwcFExgAsvPJf585cAWlRM4qdx7iIp4NCiYj8GFgO7\nycr6N01skkpTWUYkBQSLiv0YeDW2pxEzZz6voZFS5ZTcRRIuF2hAMDxyLNCAsWNVg5eqpeQukkDZ\n2UMIFmOdwsH3rcIUvtPISKliSu4iCdS7d2/sqOoo1KlTwk6ROMS9nruIVMxpp7VizZrhxfYM59RT\n9c54qVrquYsk2NSpkw4b916nzh5OOKEFmZmXE41Gkx2ehISGQookwcFx75s2fc3y5R+zf383AOrX\nX8Err2hJYCk/jXMXSUHnnvtjli5dCdwf23MT3bt3ZcmShckMS2qQanlBtojE5/PPNxIk9qxi++5K\nWjwSHqq5iyTRKacc/SC1pH0iFaXkLpJE48ffSv36f+Dgy7TT0kYA6OGqxE01d5Ek08NViUeVP1A1\nsxOBPwGnAOuAX7r71hLarQO2AweAfe7eo5TzKblLraaHq1IZ1bFw2C1Arrt3Ad6MbZfEgYi7dy8t\nsYvIkQ9Xs4D7Y/tEKi6e5N6PoFBI7J8/L6Ot5laLHIMerkpVimco5Enu/nXs89fASaW0c+ANMzsA\nTHf3/4zjmiKhNX78rfTrdw179wbb9ev/gfHjn05uUFJjlZnczSwXaFXCoduLb7i7m1lpBfPz3f0r\nM2sB5JrZCndfUFLDnJycos+RSIRIJFJWeCKh0rt3b1555emiNzZlZz+th6lylLy8PPLy8o7ZLp4H\nqisIaukbzexk4C13P/0Y3xkD7HT3iSUc0wNVEZEKqo4Hqq9waFpdFvDnEi7ayMwyYp+PAzKB5XFc\nU0REyiGe5H4P0MvM/g5cFNvGzFqb2ZxYm1bAAjP7kOCFka+6+7x4AhapTaLRKJmZl2tSk1SYJjGJ\npKhDL9OeAEB6+ihmz56pOrwcRqtCitQwmZmXk5vbj0PVz5l07DiZ1auXJjMsSTHVUXMXkQRbs2Yd\n48aNS3YYUgOo5y6SoqLRKH36DAAejO0ZBQzkxBP/zObNq5MYmaQSlWVEaqCMjNbs3HkK0BoYAmzk\nxBPvUnKXIirLiNRAt9wyDPiUYLWPjcBwRo4cnNygpEZQz10kxY0bN45Jk2YAMHLkYG6//fZjfENq\nE5VlRERCSGUZEZFaRMldRCSElNxFREJIyV1EJISU3EVEQkjJXUQkhJTcRWo4LQssJdE4d5EaTMsC\niyYxiYRQScsC9+r1CvPmzUpmWJJAmsQkIlKLpCU7ABGpvOzsISxcmEVBQbCdnj6K7OyZyQ1KUoLK\nMiI1XDQaZeLEx4Ag2aveXruo5i5SCyjR1z5K7iIhp5EztZOSu0jIaeRM7aTRMiIitUilk7uZXWFm\nn5jZATM7t4x2fcxshZmtMrNRlb2eiJQtO3sI6emjgJnAzNjImSHJDkuSpNJlGTM7HSgEpgPZ7r6k\nhDZ1gZXAxcAXwHvAVe7+WQltVZYRiZMeqNY+1VZzN7O3KD25/z9gjLv3iW3fAuDu95TQVsldRKSC\nklVzbwOsL7a9IbZPRESqUZkzVM0sF2hVwqHb3P0v5Th/hbriOTk5RZ8jkQiRSKQiXxcRCb28vDzy\n8vKO2a66yzI9gZxiZZlbgUJ3n1BCW5VlREQqqLrLMkedOOZ9oLOZdTCz+sAA4JUquqaIiJQinqGQ\n/c1sPdATmGNmc2P7W5vZHAB33w9cD0SBT4E/lTRSRkREqpZmqIqI1GCaoSoiUosouYuIhJCSu4hI\nCCm5i4iEkJK7iEgIKbmLiISQkruISAgpuYuIhJCSu4hICCm5i4iEkJK7iEgIKbmLiISQkruISAgp\nuYuIhJCSu4hICCm5i4iEkJK7iEgIKbmLiISQkruISAgpuYuIhJCSu4hICCm5i4iEUKWTu5ldYWaf\nmNkBMzu3jHbrzOwjM1tqZn+r7PVERKT84um5Lwf6A28fo50DEXfv7u494rhejZKXl5fsEJJK95+X\n7BCSqjbff6rce6WTu7uvcPe/l7O5VfY6NVWq/IKTRfefl+wQkqo233+q3Hsiau4OvGFm75vZtQm4\nnohIrZdW1kEzywValXDoNnf/Szmvcb67f2VmLYBcM1vh7gsqGqiIiJSfuXt8JzB7C8h29yXlaDsG\n2OnuE0s4Fl8gIiK1lLsfVfous+deASXW1M2sEVDX3XeY2XFAJvDH8gYnIiKVE89QyP5mth7oCcwx\ns7mx/a3NbE6sWStggZl9CCwGXnX3efEGLSIiZYu7LCMiIqlHM1SrkZndZ2afmdkyM3vJzBonO6ZE\nKu9EtzASA8tYAAAByklEQVQxsz5mtsLMVpnZqGTHk0hm9oSZfW1my5MdSzKYWTszeyv23/zHZjY8\nmfEouVevecD33P0c4O/ArUmOJ9HKO9EtFMysLvAw0Ac4A7jKzLolN6qEmkFw77XVPuBGd/8eQbl6\nWDJ//0ru1cjdc929MLa5GGibzHgSrYIT3cKgB7Da3de5+z7geeCyJMeUMLEhzluSHUeyuPtGd/8w\n9nkn8BnQOlnxKLknzq+B15IdhFSrNsD6YtsbYvukljGzDkB3gk5dUlTVUMhaqzwTvczsdmCvuz+X\n0OASoIomuoWFRicIZnY88D/AiFgPPimU3OPk7r3KOm5m/w70BX6WkIAS7Fj3X8t8AbQrtt2OoPcu\ntYSZ1QNmAc+4+5+TGYvKMtXIzPoAfwAuc/fvkh1PktWGSWrvA53NrIOZ1QcGAK8kOSZJEDMz4HHg\nU3efnOx4lNyr10PA8QRr6iw1s2nJDiiRSpvoFlbuvh+4HogCnwJ/cvfPkhtV4pjZfwN/BbqY2Xoz\nG5zsmBLsfGAg8NPY/+9LYx28pNAkJhGREFLPXUQkhJTcRURCSMldRCSElNxFREJIyV1EJISU3EVE\nQkjJXUQkhJTcRURC6P8Afc7vrD1OVmEAAAAASUVORK5CYII=\n",
       "text": [
        "<matplotlib.figure.Figure at 0x106254210>"
       ]
      }
     ],
     "prompt_number": 33
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "Now it is obvious the next line we need to best explain the remaining sum of squares.  We want another unit vector orthogonal to the first.  This is because we have already explained everything that can be explained along the direction of $\\hat{u_{best}}$, and we only have two dimensions, so there is only one remaining direction along which the variation can occur."
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "u_best_orth = np.array([np.cos(angle_best + np.pi / 2), np.sin(angle_best + np.pi / 2)])\n",
      "plt.scatter(remaining[0], remaining[1], label='remaining')\n",
      "plt.arrow(0, 0, u_best[0], u_best[1], width=0.01, color='r')\n",
      "plt.arrow(0, 0, u_best_orth[0], u_best_orth[1], width=0.01, color='g')\n",
      "plt.annotate('$\\hat{u_{best}}$', u_best, xytext=(20, 20), textcoords='offset points', fontsize=20)\n",
      "plt.annotate('$\\hat{u_{orth}}$', u_best_orth, xytext=(20, 20), textcoords='offset points', fontsize=20)\n",
      "plt.axis('equal')"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 34,
       "text": [
        "(-1.0, 0.60000000000000009, -1.5, 2.0)"
       ]
      },
      {
       "metadata": {},
       "output_type": "display_data",
       "png": "iVBORw0KGgoAAAANSUhEUgAAAXcAAAEACAYAAABI5zaHAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3Xucj3X+//HHy7GZkPNhaCmH6IzWaitNYgbf0nnVdpC2\npMOqzG5qWembQyqRorJfZKlVm5QO+hitTw6VIiJh8aOQFCqH+Ygx798f16dxmhkz85n5XDPXPO+3\nm5vP9b7en+t6XdFr3l7X+3pf5pxDRESCpZzfAYiISNFTchcRCSAldxGRAFJyFxEJICV3EZEAUnIX\nEQmgmJK7mZ1sZnPNbKWZfWlmfXPpN8bM1prZF2bWOpZziojI8VWI8fsHgAecc8vMrAqwxMzSnXOr\nfu1gZt2AZs655mb2O+B5oH2M5xURkTzENHJ3zn3nnFsW/bwHWAUkHdWtOzA52mcRUN3M6sVyXhER\nyVuR1dzNrAnQGlh01K6GwKbDtjcDjYrqvCIicqwiSe7RkszrwH3REfwxXY7a1poHIiLFKNaaO2ZW\nEZgOTHXOvZlDly3AyYdtN4q2HX0cJXwRkUJwzh09gI55towBE4CvnHOjc+k2E7gl2r898JNzblsu\nAQbm1yOPPOJ7DLp+Xb+uP/jXnptYR+4XADcBy81sabTtb8Bvosn6Refce2bWzczWAXuBXjGeU0RE\njiOm5O6cW0A+Rv/OuXtjOY+IiBSMnlAtJsnJyX6H4Ctdf7LfIfiqLF9/Sbl2y6tmE09m5kpKLCIi\npYWZ4Yr6hqqIiJRMSu4iIgGk5C4iEkBK7iIiAaTkLiISQEruIiIBpOQuIhJASu4iIgGk5C4iEkBK\n7iIiAaTkLiISQEruIiIBpOQuIhJASu4iIgGk5C4iEkBK7iIiAaTkLiISQEruIiIBpOQuIhJASu4i\nIgGk5C4iEkAxJ3czm2hm28xsRS77k83sZzNbGv01MNZziohI3ioUwTEmAc8C/8yjz4fOue5FcC4R\nEcmHmEfuzrn5wI/H6WaxnkdERPIvHjV3B/zezL4ws/fM7PQ4nFNKiVAoRErKNaSkXEMoFPI7HJHA\nKIqyzPF8DpzsnMsws67Am0CLnDoOHjw4+3NycjLJyclxCE/8EgqFuOqqnkQiIwBYsKAnM2ZMJjU1\n1efIREqucDhMOBw+bj9zzsV8MjNrArztnDsrH303AG2dczuPandFEYuUHikp15Ce3h3oGW2ZTOfO\nM5k9e7qfYYmUKmaGc+6Y0nexl2XMrJ6ZWfRzO7wfKDuP8zUREYlBzGUZM/sXcDFQ28w2AY8AFQGc\ncy8C1wJ3mVkmkAFcH+s5JRjS0nqzYEFPIhFvOyGhP2lpk/0NSiQgiqQsUxRUlimbQqEQI0eOB7xk\nr3q7SMHkVpZRcheJs+eee45y5cpx9913+x2KBIBvNXcROWTcuHEsXbqUxYsX8+KLL/odjgSYkrtI\nnEydOpXt27czYcIEJk6cyLfffsu0adP8DksCSmUZkTiJRCIkJCQct02kIFRzFxEJINXcRUTKECV3\nkQJav349f/7zn7n88st59dVXj9g3ZswYunbt6lNkIocouYsUQFZWFk899RSjR4+mU6dODBs27Ij9\nkydPpm7duj5FJ3KIkrtIAcydO5eUlBTKly/P+++/z2mnnZa9b/fu3SxfvpyLL77YxwhFPPFYFVIk\nMFq1akWdOnXYsmUL6enpTJ9+aJGzhQsXcvDgQSV3KRGU3EUKICkpCYDXXnuNqlWr0q1bt+x98+bN\nIykpiaZNm/oVnkg2lWVECiEUCnHJJZdQsWLF7LZ58+bRoUMHADZs2BDT8deuXctHH32Uvb1t2zZu\nvPFG/eCQfFNyFymETZs2HVFvj0QiLFmyhIsuugiAkSNHxnT8ESNGsGrVquztevXq0bFjR7p06RLT\ncaXsUHIXKYTmzZuzY8eO7O0hQ4aQmZlJkyZN+PLLL2nVqlVMx58zZw4pKSlHtIVCIa2aKfmmJ1RF\nCmHDhg306dOHli1b4pzjjjvuYNasWYTDYZo2bcrIkSOpVKkSzjlGjRqFc44aNWqwatUqnnjiCbZs\n2cLrr7/O9OnTadCgAS1atGDs2LFMmTKFWbNmMXPmTPr06UOHDh246KKLyMrKokGDBjz++ONUrVqV\n9957j9tuu40LL7zQ7/8U4jMtPyDig7vuuotTTjmFBx98EIBbb72Vbt26Ua1aNVJTU2nbti39+/en\nR48efP311zRu3JjJkyfzySef8Pzzz2cf57PPPuPaa69l2bJl1KhRgxdeeIGVK1fy7LPP+nVpUkLk\nltw1W0ZKHecc0Tc3lmhLlixh+vTpbNmyJbtt586dbNq0ibS0NHbu3MmWLVvo0aMHAI0bNwa8FyAf\nPgsHYPbs2fTu3ZsaNWoA8Omnn3LeeefF6UqkNFLNXUqd1796nZXfr/Q7jOOaO3fuETNqMjIyWLBg\nAZdeeikA//nPf3KcEz9//nwuueQSnHPZdf05c+Zkfy8SiTBz5kyuvfZafvrppzhdjZQ2Su5S6tRK\nrMUFEy9g0eZFfoeSp6SkJBITE7O3n3zySXr16sW5554LeAm7Y8eOR3xnx44dVKxYkdq1azNlyhQi\nkQh79+5l5cqVtGvXDoBZs2bRpk0bqlSpwuuvvx6/C5JSRcldSp2z6p7Fz7/8TMd/dmTO+jl+h5Or\nG264gfr16zN+/HhGjRpF3bp1j5giuXbt2mOmNtaoUYPWrVszadIkkpKSaNSoEcuXL6dLly6UK+f9\n73rqqadSs2ZNJk+ezC233BLXa5LSQzdUpVSq/nh1fv7lZxIrJDL16qlc1eoqv0MS8YXWc5dAaVXb\nm0eekZnBjW/cyKSlk3yOSKRkUXKXUqldo3bZnyOZEe557x5GfhTbU6EiQRJzcjeziWa2zcxW5NFn\njJmtNbMvzKx1rOcUOa/BeVSpVCV7O5IZYVB4EH/74G+ovCdSNCP3SUCuC16YWTegmXOuOdAbeD63\nviL5dXa9szGOLDNmHMjgmUXP0OfdPmS5LJ8iEykZYk7uzrn5wI95dOkOTI72XQRUN7N6sZ5XyraW\ntVsSyYwc055xIIOpy6dy/evXk5mV6UNk4pfnnnuOcePG+R1GiRGPmntDYNNh25uBRnE4rwRY5QqV\naVClQY77Mg5k8O7ad+n2cjf2Ze6Lc2Tih3HjxrF06VIWL17Miy++6Hc4JUK8lh84eppOjkXRwYMH\nZ39OTk4mOTm5+CKSUu+c+uewademHPdlHMhg/jfzufili5lz8xyqVq4a5+gkXqZOncr27duZMGEC\nAI888gjTpk3j+uuv9zmy4hEOhwmHw8ftVyTz3M2sCfC2c+6sHPa9AISdc9Oi26uBi51z247qp3nu\nUiDD5w9nUHhQnuWXyuUr07RmU+bdOo9aibXiGJ3ESyQSISEh4bhtQeXnPPeZwC3RINoDPx2d2EUK\n45z655BYMTHPPr8c/IW1O9bSdnxbtuzakmdfKZ1ySuJlJbHnpSimQv4L+Ag4zcw2mdltZnanmd0J\n4Jx7D/h/ZrYOeBG4O9ZzioC3DMGBgweO2+9A1gE279pM2/FtWb9zfRwiE/Gflh+QUss5x4nDTsxx\n1kxODKP6CdUJ3xrm7HpnF3N0Uhjr169n9OjRbNy4kZtuuil7OWSAMWPGMGvWLGbNmuVjhCWPlh+Q\nwDEzWtRqke/+CRUTKGflWLtjbTFGJYWVlZXFU089xejRo+nUqRPDhg07Yv/kyZOpW7euT9GVPkru\nUqr9tuFvj9uncvnKnFjxRAZ2GMimBzZxzenXxCEyKai5c+eSkpJC+fLlef/99494Afnu3btZvnx5\njuvfS870JiYp1X6b9FteWfEKGQcyjtlXzspRuXxl/tT6Tzx6yaPUTKjpQ4SSX61ataJOnTps2bKF\n9PR0pk+fnr1v4cKFHDx4UMm9AJTcpVQ7u97ZVCh35F9jw6hcoTL7MvdxR9s7eKbLMz5FJwWRlJQE\nwGuvvUbVqlWPeNXgvHnzSEpKomnTpn6FV+qoLCOl2pl1zzxi1J5YMZHzTz6fhbctpEqlKoxZNIat\nu7f6GKEUVCgUOuL1hOAl9w4dOgCwYcMGAMaOHUu1atVYtKho3si1du1aPvrooyI5Vkmg5C6lWpVK\nVaiVUIuECgk0q9mMN3u8ycLbFtKmQRum/8H7Z/1f0//qc5RSEJs2bTqi3h6JRFiyZAkXXXQRQPbb\nrG655RYSEhKyXz8YqxEjRrBq1aoiOVZJoOQupd51p1/H+MvHs+beNXRu2jm7PaVpCqfXOZ2XV7zM\nmu1rfIxQCqJ58+bZLwYHGDJkCJmZmTRp0oQvv/ySVq28F7XMnTuXDh06YHbMLMBCmTNnDikpKUVy\nrJJA89wl0JZuXUqb8W244OQLWHDbAr/DkXzYsGEDffr0oWXLljjnuOOOO5g1axbhcJimTZsycuRI\nKlWqxL333sv333/PlVdeybx587jnnns466yzyMrKYtSoUVSoUIHq1auzc+dOHnjgAQDeffddNmzY\nQJUqVTAzevbsyTvvvMOsWbOYOXMmffr0oUOHDtn/SigNcpvnjnOuRPzyQhEpet2mdnMMxi34eoHf\noUgROu2009zHH3/snHPunXfecd27d3fOOXf77be7ESNGOOec27Nnj3v44Yedc87t2rXLnXPOOc45\n53744QfXtWvX7GO99NJLrk+fPvEMv8hEc+cxOVVlGQm8sf8zFoAer/fQW5oC4ptvviErK4v27dsD\n8N133/HDDz+wZs0aXn75ZRo0aMDLL7/Mv/71Lx566CEAKlasyM8//8y5557LiBEjeOmll7KPFw6H\n6dixox+XUmyU3CXwmlRvwl3n3cWW3VuYsXqG3+FIEVi8ePERpZPZs2fTtWtXvvjiC8444wxuvvlm\nbrzxRm6//XaqVasGQOXKlVm1ahWPPvoon3/+OaNHj87+/vz587nkkktwzh1R7y/NlNylTBjScQgA\n1752rd7QFAAnnXQS1atXB7wpjCtWrCAtLY0WLVpwwgknHNF3woQJbNiwgTp16lC+fHmuuOIK7rrr\nLho18t4ZtGPHDipWrEjt2rWZMmUKkUj+1ioq6XRDVcqMYfOHMeA/AxjXbRx3/fYuv8ORGDjnePDB\nBznzzDP59NNP+fvf/079+vUBGD16NGZG7dq12bt3L5dddhknnXQSY8eOpWHDhmRkZLB3717uv/9+\nwFvT5qabbqJz5840atSIzp0753XqEie3G6pK7lJmRA5ESBzmrf++++HdVKlUxeeIRGKnVSGlzEuo\nmMDEKyYC8PiCx32ORqR4aeQuZcrBrINUHV6VSGaE6v9oTLlIBfr168WAAQP8Dk2kUDRyFwHKlyvP\ndVnXAfDTbxuxc+ffGTjwCYYOHepzZCJFSyN3KXNq1mrKj3/MgNrfwbNrYMfH1Kz5GDt2rPM7NJEC\n08hdJMowmN7H27i8NwCRyF4fIyqDMjNh7VrQgK7YKLlLmdOvXy/Y+jSsOxOafAiN7iMS2aXSTHH5\n/nuYMwdGjoTrroNmzaBqVfj4YyiiRb/kWCrLSJmUmFiXSOWGcP8y+KkujB5BzZpDVJqJxb598NVX\nsGIFLF4MixbB6tWwfz+ccAJEIt7ncuXgySehXz+/Iw6E3MoyehOTlEkJCdWI7LwfliyEtv+A05bC\nD35HVUo4B998A8uXw7Jl8Mkn8MUXsG0bJCZCVhbs2XPkd375xfs9MRHuv1+JPQ6U3KVM6tevFwMH\n9oUPhkJboMcY7qv4qN9hlVxTp8J//gOffQbr1nmj7woVICPDq5//ateu3I+RmAg33wxDhhR/vBJ7\ncjezLsBooDzwf865EUftTwbeAv5ftGm6c05/uuKrX+e1P/30aDI+rsG+83+kXpd6PkdVgq1eDVOm\nHJnICyIxES6/HMaNU509TmKquZtZeWAN0AnYAnwG3OCcW3VYn2Sgn3Ou+3GOpZq7+CLjQAYnDjsR\ngBrjTiGt75/0UNPRdu+Gk0+Gn38u+HcTEuCii+Ddd73RvhSp4poK2Q5Y55zb6Jw7AEwDrsjp/DGe\nR6TYjHpiFLzlrST44+ln66GmnFStCkOHwoknFux7lSvD2WfDW28pscdZrMm9IbDpsO3N0bbDOeD3\nZvaFmb1nZqfHeE6RIvX005Ng2TjIrATJb0HiEK9NjtS7N0SX2c2XSpW8aY/p6d5sGYmrWH+U5qeO\n8jlwsnMuw8y6Am8CLXLqOHjw4OzPycnJJCcnxxieSD65cvDqG3DjZdDp36DXrR6rYkV45hno2RP2\nHuehr/LlISkJPvzQG/VLkQmHw4TD4eP2i7Xm3h4Y7JzrEt1+GMg6+qbqUd/ZALR1zu08ql01d/HF\n0KFDGTjwCeAZ6Nsfan7PA+Uf4OmBT/sdWsny448waBA891ze/cygXj1YssRL8FKsimue+2KguZk1\nAb4FegA3HHXiesD3zjlnZu3wfqDsPPpAIn45NHNmCJnpFdnVA5b/ZrnPUZUg+/Z5I/bou0ipXt17\nGCkjI+f+1avDggVK7D6LqebunMsE7gVCwFfAq865VWZ2p5ndGe12LbDCzJbhTZm8PpZzihSHAQMG\nsGPHOn7+ajMdT+nIBxs+4NMtn/odlr+ysrzpjwkJhxL7a6/Bzp3w+997c92PVrUqzJsHTZvGN1Y5\nhpYfEDnKup3raP5sc5rWaMraP6/FyuK87PR0uOYabwokeCP3u+7y6u7gzXtv3dob1f/qxBO9NWTa\nt49/vGWYVoUUyadmNZuRUieF9T+up831FxAKhfwOKX6WLoUzz4SUFC+xP/SQ99Rp376HEjtAy5be\nImC/tiUkwIwZSuwliJK7yFFCoRDz/r4UgGUtPuPKq28JfoL/+mu47DJo0wZWroQ//hG+/RaGD899\ntsuIEd7c9YQEr3xTyl4sHXRK7iJHGTlyPPt2PgnhQVAhk32n/Q9//OM9wUzwO3fC3XdDkybeE6QX\nXACrVsHLL0ODBnl/t0EDbxGwMWO8Eo6UKEruIrlZ+KD3+xWT2Lm7Jldd1TM4CT4SgWHDoFYteP55\naNjQm+GyYIFXcsmvIUPg9tuLL04pNCV3kaOkpfUmIaE/HHgd3u7pNZ7fkkhkBCNHjvc3uFgdPAiT\nJnkLef26fs706bBpkzdqL6icZsxIiaA/GZGjpKamMmPGZGrWfAyWroYsg45TIPFZtm/f5nd4heMc\nvP8+VKsGt93mtY0d681Xv/pqrdQYQEruIjlITU3llVfGUqnCGni1itd4aRVWrvxv6SvNLFkCrVpB\n167eg0cDBngzYe6++8gZMBIoSu4iuUhNTeWMM86BNWPgx8bQ9kP2n/hXHn54uN+h5c+GDdClC5x3\nHqxZ470o47vvvDp5lSp+RyfFTMldJA+1a9cCDKZP8xr+55988cWXJXv0vn073HknnHoqhELQoYOX\n3P/5T2/NFykTlNxF8pCW1pty5R6AzWtg42nQ7Euy6mdy2WW3lLw13zMy4LHHoE4dGD8efvMb+Ogj\nb2XGFjkuxCoBpuQukofU1FTOOed04AV4O3rT8eoTyMwcUXJe6nHwIPzf/3mP/w8a5M1gmTEDNm6E\n88/3OzrxidaWETmOUCjEVVf1JBKpCFc2hnMXwtTfw7osqlT5mt27v/UnMOfgvfe8B4h++cVre+EF\n+NOf9NajMkRry4gU0q9TIytU2A/pZ3mNf/gEzLFnz8/+1N8/+8wrtVx2mZfYBw2CPXu8WrsSu6Dk\nLpIvqampDB7cF/ZOgQ8ToFIWnHMmkEivXnfHL5D166FTJ2jXDtatg169YNs2ePTRgr/fVAJNyV0k\nnwYMGEBCwkmwcJTXcOUEqDCMrVt/LP7a+w8/eOWWZs3ggw+gY0dYuxYmToS6dYv33FIqKbmLFEDL\nls1h/wnwzjiv4fz3gYbF90LtvXth8GAvgU+cCKecAp984iX4Zs2K55wSCEruIgUwfPjDwH3weWWv\n4dI3IKEDBw7sL9oTZWZ60xmrVPFKLhUrwttve2WZ3/2uaM8lgaTkLlIAqamp9Ox5JWSlwbTmXuMl\nE4hEdhXNjVXn4K23vDXS74y+qXL8eG8O+2WXaQ0YyTdNhRQphGbNzmb9+kTotxSq7YfR59L6lBP5\n/PMFhT/ookVw/fXe/HTwRux/+Yu3gqNILjQVUqQInXpqc+BC+Hd0hkq3LJYu/bJwN1bXroXkZO8V\ndRs3wh13eDdQBw1SYpdCU3IXKQRvWYKXYNMo+OYCaLEc6v+GgQOfzn+C//57uPVWb776hx967y1d\nv94rw9SuXZzhSxmgsoxIIbVpk8zSpb2g1gH48x3wfQKMOwdYzvvvv0FqamrOX9yzx3v/6JAh3nbz\n5vDKK97qjSIFlFtZRsldpJBCoRDdut1AVlY1uHornL0fprSE9d9TpYqxe/f2I7+QmQn/+Ie3jjrA\nCSfAG294y/LqRqkUUrHV3M2si5mtNrO1ZtY/lz5jovu/MLPWsZ5TpCRITU3lf/83DdgFs6NvN/rD\nf8Gas2dP5NDsGee8hbwqVTqU2CdM8EbwXbsqsUuxiGnkbmblgTVAJ2AL8Blwg3Nu1WF9ugH3Oue6\nmdnvgGecc+1zOJZG7lIqmdUA6sOlX8NFEXjjDlg+g9atT+Pz556AHj1g82av82OPQVqaN9VRpAgU\n18i9HbDOObfROXcAmAZccVSf7sBkAOfcIqC6memNARIYnTqdB2yD+U97DVf/g2YV+jJuxRLvpdOb\nN0OfPt5LNAYOVGKXuIg1uTcENh22vTnadrw+jWI8r0iJkZ6eTtWqCbA/Ad4bA8DtvxtE+8x9Xtll\nwwZ4/nmoVcvnSKUsiXVt0PzWUY7+J0OO3xs8eHD25+TkZJKTkwsVlEi8/fvfE+ne/Wb2Lx7OyWef\nRMJPP/PRs8/y+3vv9Ts0CZhwOEw4HD5uv1hr7u2Bwc65LtHth4Es59yIw/q8AISdc9Oi26uBi51z\n2446lmruUqqFQiFGjhxPuawsLkpuy4fzlgLenPhcp0WKxKhYpkKaWQW8G6qXAt8Cn5L3DdX2wGjd\nUJUgO/TmJm+Mk5DQnxkzJivBS7HILbnHVJZxzmWa2b1ACCgPTHDOrTKzO6P7X3TOvWdm3cxsHbAX\n6BXLOUVKupEjx0cTe08AIhGvTcld4inm93E552YBs45qe/GobRUeRUTiSGvLiBSxtLTeJCT0x5sB\nPBm4j48//rj439YkchgtPyBSDEKhEPfc8xDr128E/gScBfSlZ8+reOmll3yNTYJFa8uIxFmtWs3Y\nufPv/Fp790bx9/P++9NUf5cio/XcRXwVAl4AjF69evsdjJQBSu4ixaRfv15AX+AvwE1AH2AUW7f+\npPq7FDuVZUSKUVJSU7Zu3Q8M4fDyTNWqg9i162sfI5OgUFlGxAeTJo3De7zjSJHIvvgHI2WKRu4i\nxSwpqTFbt/4EjIm29KVp00asW7fSz7AkIDRyF/HJpEnjqVDhIDAQGEiFCgcZO/Zpv8OSgNPIXSQO\nfl1UDODii9vw4YefA1pUTGKnee4iJcChRcUuBBYBGfTsebkebJJCU1lGpATwFhW7EHgn2pLI5MnT\nNDVSipySu0jcpQOV8aZHDgEqM2SIavBStJTcReIoLa033mKsY/DmvfcExrBPMyOliCm5i8RRamoq\ndkx1FMqVy6FRJAYxr+cuIgVz6qn1Wb++72EtfTnlFL0zXoqWRu4icTZ27NNHzHsvV+4XqlWrQ0rK\nNYRCIb/Dk4DQVEgRH/w673379m2sWPElmZmtAKhUaTUzZ2pJYMk/zXMXKYHatLmQpUvXAE9FW/5C\n69an8fnnC/wMS0qRYnlBtojE5uuvv8NL7D0Pa3vMt3gkOFRzF/FR48bH3kjNqU2koJTcRXw0fPjD\nVKr0V359mXaFCvcB6OaqxEw1dxGf6eaqxKLIb6iaWU3gVaAxsBH4g3Pupxz6bQR2AQeBA865drkc\nT8ldyjTdXJXCKI6Fwx4C0p1zLYAPots5cUCyc651boldRI6+udoTeCraJlJwsST37niFQqK/X5lH\nXz1bLXIcurkqRSmWqZD1nHPbop+3AfVy6eeAOWZ2EHjROfePGM4pEljDhz9M9+43s3+/t12p0l8Z\nPnyKv0FJqZVncjezdKB+DrsGHL7hnHNmllvB/ALn3FYzqwOkm9lq59z8nDoOHjw4+3NycjLJycl5\nhScSKKmpqcycOSX7jU1paVN0M1WOEQ6HCYfDx+0Xyw3V1Xi19O/MrAEw1znX8jjfeQTY45wbmcM+\n3VAVESmg4rihOpNDj9X1BN7M4aSJZlY1+vlEIAVYEcM5RUQkH2JJ7o8Dnc3sv0DH6DZmlmRm70b7\n1Afmm9kyvBdGvuOcmx1LwCJlSSgUIiXlGj3UJAWmh5hESqhDL9MeAUBCQn9mzJisOrwcQatCipQy\nKSnXkJ7enUPVz8k0bTqadeuW+hmWlDDFUXMXkThbv34jQ4cO9TsMKQU0chcpoUKhEF269ACeibb0\nB26iZs032bFjnY+RSUmisoxIKVS1ahJ79jQGkoDewHfUrPmYkrtkU1lGpBR66KF7gK/wVvv4DuhL\nv369/A1KSgWN3EVKuKFDh/L005MA6NevFwMGDDjON6QsUVlGRCSAVJYRESlDlNxFRAJIyV1EJICU\n3EVEAkjJXUQkgJTcRUQCSMldpJTTssCSE81zFynFtCyw6CEmkQDKaVngzp1nMnv2dD/DkjjSQ0wi\nImVIBb8DEJHCS0vrzYIFPYlEvO2EhP6kpU32NygpEVSWESnlQqEQI0eOB7xkr3p72aKau0gZoERf\n9ii5iwScZs6UTUruIgGnmTNlk2bLiIiUIYVO7mZ2nZmtNLODZtYmj35dzGy1ma01s/6FPZ+I5C0t\nrTcJCf2BycDk6MyZ3n6HJT4pdFnGzFoCWcCLQJpz7vMc+pQH1gCdgC3AZ8ANzrlVOfRVWUYkRrqh\nWvYUW83dzOaSe3I/H3jEOdcluv0QgHPu8Rz6KrmLiBSQXzX3hsCmw7Y3R9tERKQY5fmEqpmlA/Vz\n2PU359zb+Th+gYbigwcPzv6cnJxMcnJyQb4uIhJ44XCYcDh83H7FXZZpDww+rCzzMJDlnBuRQ1+V\nZURECqhIkcFIAAADuElEQVS4yzLHHDhqMdDczJqYWSWgBzCziM4pIiK5iGUq5FVmtgloD7xrZrOi\n7Ulm9i6Acy4TuBcIAV8Br+Y0U0ZERIqWnlAVESnF9ISqiEgZouQuIhJASu4iIgGk5C4iEkBK7iIi\nAaTkLiISQEruIiIBpOQuIhJASu4iIgGk5C4iEkBK7iIiAaTkLiISQEruIiIBpOQuIhJASu4iIgGk\n5C4iEkBK7iIiAaTkLiISQEruIiIBpOQuIhJASu4iIgGk5C4iEkCFTu5mdp2ZrTSzg2bWJo9+G81s\nuZktNbNPC3s+ERHJv1hG7iuAq4B5x+nngGTnXGvnXLsYzleqhMNhv0Pwla4/7HcIvirL119Srr3Q\nyd05t9o59998drfCnqe0Kil/wH7R9Yf9DsFXZfn6S8q1x6Pm7oA5ZrbYzO6Iw/lERMq8CnntNLN0\noH4Ou/7mnHs7n+e4wDm31czqAOlmtto5N7+ggYqISP6Zcy62A5jNBdKcc5/no+8jwB7n3Mgc9sUW\niIhIGeWcO6b0nefIvQByrKmbWSJQ3jm328xOBFKAR/MbnIiIFE4sUyGvMrNNQHvgXTObFW1PMrN3\no93qA/PNbBmwCHjHOTc71qBFRCRvMZdlRESk5NETqsXIzJ40s1Vm9oWZvWFmJ/kdUzzl90G3IDGz\nLma22szWmll/v+OJJzObaGbbzGyF37H4wcxONrO50b/zX5pZXz/jUXIvXrOBM5xz5wD/BR72OZ54\ny++DboFgZuWB54AuwOnADWbWyt+o4moS3rWXVQeAB5xzZ+CVq+/x889fyb0YOefSnXNZ0c1FQCM/\n44m3Aj7oFgTtgHXOuY3OuQPANOAKn2OKm+gU5x/9jsMvzrnvnHPLop/3AKuAJL/iUXKPn9uA9/wO\nQopVQ2DTYdubo21SxphZE6A13qDOF0U1FbLMys+DXmY2ANjvnHslrsHFQRE96BYUmp0gmFkV4HXg\nvugI3hdK7jFyznXOa7+Z3Qp0Ay6NS0BxdrzrL2O2ACcftn0y3uhdyggzqwhMB6Y65970MxaVZYqR\nmXUB/gpc4Zzb53c8PisLD6ktBpqbWRMzqwT0AGb6HJPEiZkZMAH4yjk32u94lNyL17NAFbw1dZaa\n2Ti/A4qn3B50CyrnXCZwLxACvgJedc6t8jeq+DGzfwEfAS3MbJOZ9fI7pji7ALgJuCT6//vS6ADP\nF3qISUQkgDRyFxEJICV3EZEAUnIXEQkgJXcRkQBSchcRCSAldxGRAFJyFxEJICV3EZEA+v+CzmbF\nkEHWzQAAAABJRU5ErkJggg==\n",
       "text": [
        "<matplotlib.figure.Figure at 0x1073d1650>"
       ]
      }
     ],
     "prompt_number": 34
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "Now the projections onto $\\hat{u_{orth}}$ are the same as the remaining points, because the remaining points already lie along the line defined by $\\hat{u_{orth}}$."
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "projected_onto_orth = line_projection(u_best_orth, remaining)\n",
      "np.allclose(projected_onto_orth, remaining)"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 35,
       "text": [
        "True"
       ]
      }
     ],
     "prompt_number": 35
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "If we have really found the line $\\hat{u_{best}}$ that removes the most sum of squares from the remaining points, then this is the *first principal component* of $\\mathbf{X}$. $\\hat{u_{orth}}$ will be the second principal component of $\\mathbf{X}$."
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "Now for a trick.  Remember that the two principal components are orthogonal to one another.  That means, that if I project the data onto the second principal component $\\hat{u_{orth}}$, I will (by the definition of orthogonal) pick up no component of the columns of $\\mathbf{X}$ that is colinear (predictable via projection) with $\\hat{u_{best}}$.\n",
      "\n",
      "This means that I can go straight to the projection onto the second component, from the original array $\\mathbf{X}$."
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "# project onto second component direct from data\n",
      "projected_onto_orth_again = line_projection(u_best_orth, X)\n",
      "# Gives same answer as projecting remainder from first component\n",
      "np.allclose(projected_onto_orth_again - projected_onto_orth, 0)"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 36,
       "text": [
        "True"
       ]
      }
     ],
     "prompt_number": 36
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "For the same reason, I can calculate the projection coefficients $c$ for both components at the same time, by doing matrix multiplication:"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "# Components as rows in a 2 by 2 array\n",
      "components = np.vstack((u_best, u_best_orth))\n",
      "components"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 37,
       "text": [
        "array([[ 0.87824304,  0.47821456],\n",
        "       [-0.47821456,  0.87824304]])"
       ]
      }
     ],
     "prompt_number": 37
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "# Calculating projection coefficients with array dot\n",
      "c_values = components.dot(X)"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [],
     "prompt_number": 38
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "# Result of projecting on first component, via array dot\n",
      "u = u_best.reshape(1, 2)  # first component as row vector\n",
      "c = c_values[0].reshape(1, 50)  # c for first component as row vector\n",
      "projected_1 = u.T.dot(c)\n",
      "# The same as doing the original calculation\n",
      "np.allclose(projected_1, line_projection(u_best, X))"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 39,
       "text": [
        "True"
       ]
      }
     ],
     "prompt_number": 39
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "# Result of projecting on second component, via array dot\n",
      "u = u_best_orth.reshape(1, 2)  # second component as row vector\n",
      "c = c_values[1].reshape(1, 50)  # c for second component as row vector\n",
      "projected_2 = u.T.dot(c)\n",
      "# The same as doing the original calculation\n",
      "np.allclose(projected_2, line_projection(u_best_orth, X))"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 40,
       "text": [
        "True"
       ]
      }
     ],
     "prompt_number": 40
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "## The principal component lines are new axes to express the data"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "My original points were expressed in the orthogonal, standard x and y axes.  My principal components give new orthogonal axes.  When I project, I have just re-expressed my original points on these new orthogonal axes.  Let's call the projections of $\\vec{v_1}$ onto the first and second components: $proj_1\\vec{v_1}$, $proj_2\\vec{v_1}$.\n",
      "\n",
      "For example, here is my original first point $\\vec{v_1}$ expressed using the projections onto the principal component axes:"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "# Show v1 as sum of projections onto components 1 and 2\n",
      "x, y = v1\n",
      "# Projections onto first and second component\n",
      "p1_x, p1_y = projected_1[:, 0]\n",
      "p2_x, p2_y = projected_2[:, 0]\n",
      "# Make subplots for vectors and text\n",
      "fig, (vec_ax, txt_ax) = plt.subplots(1, 2, figsize=(14, 4))\n",
      "# Show 0, 0\n",
      "vec_ax.plot(0, 0, 'ro')\n",
      "# Show vectors with arrows\n",
      "vec_ax.arrow(0, 0, p1_x, p1_y, color='r', length_includes_head=True, width=0.01)\n",
      "vec_ax.arrow(0, 0, x, y, color='k', length_includes_head=True, width=0.01)\n",
      "vec_ax.arrow(p1_x, p1_y, p2_x, p2_y, color='b', length_includes_head=True, width=0.01)\n",
      "# Label origin\n",
      "vec_ax.annotate('$(0, 0)$', (-0.5, -0.5), fontsize=16)\n",
      "# Label vectors\n",
      "vec_ax.annotate(r'$\\vec{{v_1}} = ({x:.2f}, {y:.2f})$'.format(x=x, y=y),\n",
      "                (x / 2 - 2.2, y / 2 + 0.4), fontsize=16)\n",
      "vec_ax.annotate(r'$proj_1\\vec{{v_1}} = ({x:.2f}, {y:.2f})$'.format(x=p1_x, y=p1_y),\n",
      "                (p1_x / 2 + 0.5, p1_y / 2), fontsize=16)\n",
      "vec_ax.annotate(r'$proj_2\\vec{{v_1}} = ({x:.2f}, {y:.2f})$'.format(x=p2_x, y=p2_y),\n",
      "                (x + 0.3, y - 0.1), fontsize=16)\n",
      "# Make sure axes are right lengths\n",
      "vec_ax.axis((-1, 7.5, -1, 3))\n",
      "vec_ax.set_aspect('equal', adjustable='box')\n",
      "vec_ax.set_title(r'first and and second principal components of $\\vec{v_1}$')\n",
      "# Text about length\n",
      "txt_ax.axis('off')\n",
      "txt_ax.annotate(\n",
      "    r'$\\|\\vec{v_1}\\|^2 = \\|proj_1\\vec{v_1}\\|^2 + \\|proj_2\\vec{v_1}\\|^2$ =' +\n",
      "    '\\n' + \n",
      "    '${p1_x:.2f}^2 + {p1_y:.2f}^2 + {p2_x:.2f}^2 + {p2_y:.2f}^2$'.format(\n",
      "    p1_x=p1_x, p1_y=p1_y, p2_x=p2_x, p2_y=p2_y),\n",
      "    (0, 0.5), fontsize=16)"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 41,
       "text": [
        "<matplotlib.text.Annotation at 0x1070ce550>"
       ]
      },
      {
       "metadata": {},
       "output_type": "display_data",
       "png": "iVBORw0KGgoAAAANSUhEUgAAAz0AAAEACAYAAACZExgvAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzs3Xd4VGX68PHvDaEX6VJEjIgCAgpIUTQirHUR0AVemlJX\npKn4E+w0BQFXRcBCFxFRUGRZgV0QCGJoUapIEQyiIELoPcnkfv84k3GSzCSTzKQQ7s91nYucc54+\nkzD3nPM8R1QVY4wxxhhjjMmr8uV0A4wxxhhjjDEmK1nQY4wxxhhjjMnTLOgxxhhjjDHG5GkW9Bhj\njDHGGGPyNAt6jDHGGGOMMXmaBT3GGGOMMcaYPM2CHmOMMcYYY0yeZkGPMcYYY4wxJk+zoMcYY4wx\nxhiTp1nQY4wxxhhjjMnTLOgxxhhjjDHG5GkW9BhjjDHGGGPyNAt6jDHGGGOMMXmaBT3GGGOMMcaY\nPM2CHmOMMcYYY0yeZkGPMcYYY4wxJk+zoMcYY4wxxhiTp1nQY4wxxhhjjMnTLOgxxhhjjDHG5GkW\n9BhjjDHGGGPyNAt6jDHGGGOMMXmaBT3GGGOMMcaYPM2CHmOMMcYYY0yeZkGPHyJyk4hsEZHTIjJQ\nRH4UkYicbld6ROQjEXktr9STFULd9lC8N0L5/hKR/SLSMhRlmfSl+FsxIAP5XhGRolnZNmOMMcY4\nwnK6AbnYEGCFqt7q3p+YmUJEZD/QU1VXhqph6VD3llfqyQohbbuq1skNZXgXx+X72mSZLPxdTPm3\nwl/9jwJVvQ7dCWwEloW4PcYYY4xJwYIe/6oBa9NLJCJhqpqQRhIFJGStCkx21Zfd/QqloNsewGtv\ncpes+l0M6G+Fqi5I+llECgM/qeryLGiPMcYYY1Kw29t8EJGVQHNgkvuWlRruW4ZauM/vF5EhIrIN\nOCMi+UXkeRH53Z1+l4i0EJHZwLXAf0TkjIg856e+F0RkrzvvDhFp63Vuv4j8n4hsFZGTIvKZiBTy\nOl9fRDa5834GFE6jX37rSa+ujNTjTp9qPNzHK4vIlyJyRER+EZGBXnmqisgC97lYEZnoda6WiESK\nyAn3rWAPZ9EY7XeP0w4ROS4iM1KU5eu1T/neSKstKfs4wStfy0Dakd7rmM7r4nOM0xpfr/Y8JyLb\n3O/l6SJytYgsFZFTIrJcREoFOIbBvJZpvX/Sev/6/F309z71MW4+2yyp/1bcEMjroKoXLeAxxhhj\nspGq2uZjA1bh3AqTtB8DtHD/vB/YBFQBCgE3AQeAiu7z1wLXp8yXRl3tvPJ2AM4CV3vlXw9UBEoD\nPwF93OcKAr8CTwP5gX8AccDIDNRTMUUfU9WViXp8jgdOkP0D8ArOVcZwYB9wn7vcrcBbQBH3uDZz\n5y8A7AVecOe7BzgN1PB6PUI1RvuBbe7XtjTwHfBaivOe197Pe8NfW3z18Q5f75O02hHg65jqPeev\n/jTG98YUZa4FygOVgT/d43CLu5wVwNAA2h7Ma+n3/ZNeXj9j7Pf3NsW4pdfmZH8rbLPNNttss822\n3LfZlZ60+bsVRoEJqnpQVS8BLpwPfjeLSAFVPaCqvwRaiap+oaqH3T/PA34GGnslmaCqh1X1BPAf\nIGnuQFMgTFXfVVWXqn4JRAdRj7+6MlQP/sejEVBOVV9X1QRVjQGmAR3d7agEDFbVC6p6SVWjvPpZ\nTFXHuPOtAr4GOid1LVRj5C5rkvu1PQGMAjqlOO/92vvK768tvvro77Yov+0I8HX0xV/9/sa3U4r8\nE1X1qKoeAtYA61R1q3scvgLqp9f2NOoK5LVM6/2TXl5fAv29Ta/N4OdvhYhcIyJtRWSee3+Q+Lni\na4wxxpisY0FP2tKaDP6bJ5HqXuAZYDjwp4jMFZFKgVYiIo+LyGb3rTMngDpAOa8kh71+vgAUd/9c\nGTiYorhf8f8BzFc9ZVMk867rvLuuShmpJ43xqAZUTqrf3YYXgQrANcCvqproo8jKeI23V/2V/bQ7\n02Pk5l3XgRT1pDzvi7+2VMV/HwNuR4Cvoy/+6vc3vlVSHPvT6+cLKfYv8lc//bY9jboCeS3Tev+k\nlzeVDPzeBtJmf38rbgQ241x9AvgMCPT1N8YYY0yIWNCTeck+5KjqXFW9C+eDmQJjfaVLSUSqAVOA\n/kAZVS0N/EhgE67/IPUH06T6c6SeJH7G4wAQo6qlvbaSqtoK+B24VkTy+yjuEFBVRLzbWo3UwUxI\n2o5zm5P3z4dSnM/symi/4b+PgbTjoIhcC0wlc6+jv/r9je/v6ZSXsk7vcfE3hsG8lr/h//3jS8rX\nKdXrlsbvrbdMt1mdleJ6AR+5D5UBvhGRq0TkURF5Mb0yjDHGGBM8C3rSFtBKTyJyozgLFxQCLuF8\n6+1yn/4TqJ5G9mI4H7ZigXwi0gPnm/tArAMSROQpESkgzpK4jUJYT1L/M1JPWuOxEWfy/xARKSLO\nIgB1ROQ2YANOgDJGRIqKSGERucNd5Aacq05D3PU3B1rhfGuengy13d3nfiJSRUTKAC8HWE8gNuK/\nj4G043Oc1zGRzL1f/I3xejI/vinb7K/tSWUF81qm9f5Jqz1Jkv0upvN76y2Q8Unrb0VDnH4D1FfV\nbap6Cmd+UsE08hljjDEmRCzoSVug3+gXAt4AjuJ8qCyHc9sN7uOvuG/HeTZVBao/4UwsX4dza04d\nnInfabVJ3XnjgEeB7sAxnEntX/rMlPF6PHWpanyg9bj5HA/3bVWtcOZZ/OI+PwUo6T73MHADzhWh\n39z1JPXzYeBBd55JwGOquietdnvlzUjbFfgU59kp+3Dmy7yeRvr0eLfFhZ8+BtoOVd1Jxl9H3PX7\nHGP365uR8fVuY6p++mu7uw3BvJYu/Lx/0svrlvJ3Ma3f278KCWx80vpb8RnQ1h1wp7u0tTHGGGNC\nT1TtGYbGJBGRGKCXZt/DZHN1OzLjcm57dnLfctpdVUfkdFuMMcaYvC6oKz3u22M2iMgWEflJRN7w\nk26CiPwszvMz6vtKY4wxV5jL+eG+xhhjzGUlqKBHVS8C96jqrUA94B4RudM7jYg8BNygqjWAJ4AP\ngqnTGGMudyJSHOeZUQ1FJNA5WcYYY4zJpJDd3iYiRYHVQDf3/JGk4x8Cq1T1c/f+LuBuVf3Td0nG\nGGOMMcYYEzpBL2QgIvlEZAvOykirvAMetyokf8bF7zjPZDHGGGOMMcaYLBd00KOqie7b264BItzL\nuaaU1vM8jDHGGGOMMSbLhIWqIFU9JSKLgduASK9TB3GeBJ/kGnw81E9ELBAyxphcQFVtkQVjjDF5\nSrCrt5UTkVLun4sA9wKbUyRbBDzuTtMUOOlvPo+q5upt2LBhOd4Ga6e183Ju5+XQxiu9ncYYY0xe\nFOyVnkrALBHJhxNAzVbVFSLSB0BVJ6vqEhF5SET2AueAHkHWaYwxxhhjjDEBCyroUdXtQAMfxyen\n2B8QTD3GGGOMMcYYk1lBL2RwJWnevHlONyEg1s7QsnaGzuXQRrB2GmOMMXlNyJ7TEywR0dzSFmOM\nuVKJCGoLGRhjjMlj7EqPMcYYY4wxJk+zoMcYY4wxxhiTp1nQY4wxxhhjjMnTLOgxxhhjjDHG5GkW\n9BhjjDHGGGPyNAt6jDHGGGOMMXmaBT3GGGOMMcaYPM2CHmOMMcYYY0yeZkGPMcYYY4wxJk+zoMcY\nY4wxxhiTpwUV9IhIVRFZJSI7RORHEXnKR5rmInJKRDa7t1eCqdMYY4wxxhhjMiIsyPzxwCBV3SIi\nxYEfRGS5qu5MkW61qrYOsi5jjDHGGGOMybCgrvSo6mFV3eL++SywE6jsI6kEU48xxhhjjDHGZFbI\n5vSIyHVAfWBDilMK3CEiW0VkiYjUDlWdxhhjjDHGGJOeYG9vA8B9a9sXwNPuKz7eNgFVVfW8iDwI\nLARuDEW9xhhjjDHGGJOeoIMeESkAfAl8oqoLU55X1TNePy8VkfdFpIyqHk+Zdvjw4Z6fmzdvTvPm\nzYNtnjHGmDRERkYSGRmZ080wxhhjspSoauYziwgwCzimqoP8pLkaOKKqKiKNgXmqep2PdBpMW4wx\nxgRPRFBVm4dpjDEmTwn2Sk8zoCuwTUQ2u4+9BFwLoKqTgXZAXxFJAM4DHYOs0xhjjDHGGGMCFtSV\nnlCyKz3GGJPz7EqPMcaYvChkq7cZY4wxxhhjTG5kQY8xxhhjjDEmT7OgxxhjjDHGGJOnWdBjjDHG\nGGOMydMs6DHGGGOMMcbkaRb0GGNC6r333mPDhg3ZUld0dDSxsbHZUldeEBcXx9dff53TzTDGGGOy\nnS1ZbcwV7ty5c9SpU4cZM2Zwzz33BJzP5XLRr18/4uLikh2PiYnh/PnzbNy4MdRNTSY6Oprt27fT\ns2fPTOWPi4vjhRdeoEKFCrhcLmJjY3nzzTcJC0v/8WVLlixh5syZzJ8/P+gyM9uOtNrwzjvv4HK5\nEBGOHTvG8OHDKV68OACzZ8+mUKFCdOjQwWe5tmS1McaYvMiCHmMM06ZNo0uXLhQpUiSocs6cOUOf\nPn2YPHkyJUqUCFHrUrt48SJdu3bliy++yHQZL7zwAufOnWPixIkADBo0iAIFCjBu3Di/ef7973/z\n7bffsn37dhISEli5cmXQZWY0T3ptGDp0KP369aNixYoA/PDDD0yaNImZM2d60nTt2pXRo0dz7bXX\npirfgh5jjDF5kQU9xpjLztixY6levTrt2rXLVP5Lly5RoUIFFi9ezJ133gnA2rVrad26dUC3y40Y\nMYLIyEhWrVoVVJnBtMNXGwAaNWpEZGQkxYoV8xy7++67Wb16tWc/OjqaKVOmMHXq1FTlWtBjjDEm\nL7I5PcaYy86cOXN45JFHMp1/69atnDlzhurVq3uOVatWjePHj7N58+Z08/v6giYzZQbTDn9fEl19\n9dXcd9997Nu3D4DIyEhatGiRLE2jRo1Ys2YN58+fT7MOY4wxJq9I/+Z1Y0yOGDlyJDt27OCJJ55g\n165dJCQkEBUVxbhx47juuusYPXo0O3bs4MUXXyQ6Oprjx49z6tQpRo4cCcC+ffuYMGEC4eHhHD9+\nnPLlyzNw4EBP+b///jvz5s1j9+7d1K5dm6effjqgdk2fPp0TJ06we/duz5WCevXqMWLEiKACkUDt\n2rWLMmXKkD9//kyX8dtvvwEkuxqSdDvewYMHqV+/fraUmRXtGDduHC1atKBevXr06tWLypUrM2zY\nsFTpGjduzMqVK2nVqlWG6zDGGGMuNxb0GJMLLV68mHbt2qGqPPfcc6xevZqSJUuSP39+hgwZQu/e\nvWndujWnTp1i4MCBLF68mLFjx7J06VJGjhzJ1q1bad++PStWrKBq1aoA1KlTh8aNG9OkSRNUlUmT\nJjFmzBgWLlzIyy+/HFDQs23bNooVK0ZERARNmzb1BD0RERHs378/4P4lJCTQr18/4uPj003bsWNH\n7r//fs/+5s2bufHGGwOuy5cLFy4AULhwYc+xQoUKAc68pOwqMyvaUbNmTfr27cvq1auZNGkSDRo0\noFOnTlSrVi1ZuurVq7NlyxYLeowxxlwRggp6RKQq8DFQAVBgiqpO8JFuAvAgcB7orqrp3z9izGXu\n3DlYv1656aaDXHPNNRnKe9VVV1G7dm3Wr19Pnz59KFmyJOBM4N+5cyfHjh3jvvvuIyoqiieeeIKi\nRYvSq1cvevbsiarSuXNnBgwY4Al4wPlgHRUVRZMmTTh8+DC9evUCYP369dxyyy0BtevMmTO0a9eO\nkSNHJruq8+ijjwYUwCQJCwtjypQpAaf3duTIEUqXLp3q+MyZM1m2bFmaeQsUKMC0adMoVapUqnNn\nz54FkgcgGZGZMrOiHd27d6d79+4MGzaMhQsX0q9fP1q3bs3WrVuTpStbtiy7du3KVB3GGGPM5SbY\nKz3xwCBV3SIixYEfRGS5qu5MSiAiDwE3qGoNEWkCfAA0DbJeY3KluLg4Nm3axHfffcfSpUuJjJzF\noEEx/OtfGQt67rzzTuLj44mKimL8+PGe4+vWraNGjRp06tSJc+fOER0dzWeffQbgWYlr7dq17Ny5\nk86dO3vyXbp0iT179nhW9KpUqZLn3DfffMOTTz7p2Z8xYwaqytdff83w4cOTBUTNmjVDVZkxYwaf\nfvqp5/j+/fs99U2ePJlatWoRERGRoT4H6tKlSz6Xc+7Rowc9evQIqIwqVaoAcOrUKcqXLw/8dWXF\n14pmWVVmqNuxdu1aVNUzh6dt27Y0atSIW2+9le3bt1O3bl1P2oIFC+JyuTJchzHGGHM5CiroUdXD\nwGH3z2dFZCdQGdjplaw1MMudZoOIlBKRq1X1z2DqNiY3OHHiBGvXriUyMpLly5ezc+dOChcuzMWL\nF4mPj6dSpUTefvsu6taFbt0yVnZ0dDQlSpTgpptuAuD8+fOsWLGC2bNnA7BmzRquvfbaVFeRDhw4\nQMWKFSlXrpzn2MKFCylSpAht2rRJ1f4tW7bQsmVLAJYuXUqjRo2oW7cu5cqV4/HHH091hWDnzp3E\nxsZ6VhsDOHnyJAATJ05k1qxZvP3222n2LT4+nv79+2fq9rYKFSoQExOTbr601KtXj7Jly/LLL794\ngo2ffvqJEiVKJAsM/BFJvbhZZsoMph2+2nD48GFPYJukSpUqREREeG6bS5I0z8sYY4y5EoRsTo+I\nXAfUB1I+ir0K8JvX/u/ANYAFPeayoqrExMQQFRXF8uXLiYyM5PDhwxQpUoSzZ8+SmJgIOFd7Chcu\nTOvWralWrSoTJkDfvnD8OAwaFHh9K1eupEyZMp790aNH89BDD/Hwww97zicFK94aNmxIfHw8CQkJ\nhIWFceTIEYYOHcrHH3+cbMI8wKpVq6hatSrh4eEA/PzzzyxevJhJkyZxww03+Jynk5iYSMmSJcmX\nz1n8ccGCBbRp04bChQszcOBANm3a5HdlsSQFChTI9O1t4eHhLFq0KFN5k+TPn5+OHTsyf/58mjRp\nAsDcuXPp06cPBQsWBJwAcMGCBUyePNnT1yS++hdImYsWLWLRokVMmTKFfPnyBdUOX21o2bIlo0aN\n4sCBA54rRX/88QeFCxdONQ/qzz//DPi2RmOMMeZyF5Kgx31r2xfA06p61leSFPs+PxENHz7c83Pz\n5s1p3rx5KJpnTNBUlbvvvpsNGzZQsGBBz7wLINXVirCwMG644Qbmzp3L/PlC8eJw9iy88gocOwav\nvQY+vqRPZeXKlbRo0YI33niDxMREwsLC+OSTTzzn9+zZQ79+/VLlq1GjBu+88w6DBw/mhhtuYM+e\nPcydO5cGDRqkSrtixYpkgVO/fv08fYuKiuLBBx9MladOnTp069aNV199lcqVK1OrVq1kSy5ntdtv\nv53evXsHXc6YMWN45plneP3113G5XJQqVYrXX3/dc3737t0sX76cS5cueR7aumzZMr744gsWL17M\niRMn6NKlC82aNfO8DumVuXPnThYsWMBzzz1HzZo1M9WOtNpw1VVX8emnnzJixAgqVaqEiOByufjg\ngw9S9X/t2rUMHjyYyMhIIiMjgx5PY4wxJjcL+uGkIlIA+BpYqqrjfZz/EIhU1c/c+7uAu1Pe3mYP\nJzW53YoVK2jdunW6zzYpX74827Zto2LFimzZAhERkLQQV9Gizm1ukyZBvjSeknXp0iVKly7N9u3b\nszSguPHGGxk5ciQdO3ZMdvzkyZO0a9eOuXPnZvgWqB49etC9e3fuvvvuUDY1mc6dO/P8889fllcq\nNm/eTFhYWEC30WWV2NhY2rRpQ1RUVKpz9nBSY4wxeVFQDycV56by6cBPvgIet0XA4+70TYGTNp/H\nXI5atmzJvffeS4ECBfymKVasGN98841nXkWtWuBelRiA8+dh1izo3BkSEvzXtW7dOsqUKRPygOfU\nqVM0bNiQ6Oholi1bhqrStm3bZGlcLhevv/46s2fPzvScD1/zTUJpxIgRTJiQaqHIy8L69eu5+eab\nc7QN48eP57XXXsvRNhj/RKSniPQSka9E5PKL7C8DNsbZw8bZ5CZBBT1AM6ArcI+IbHZvD4pIHxHp\nA6CqS4BfRGQvMBlIfT+OMZeJiRMn+p18X6RIET799FPq1avnOVaoEHgtlAY4gc9//gOtWsHFi6nL\n+d///sdzzz2Hy+Vi9OjRoWw+qkrBggVZsmQJ8+fPZ926damWRv7www957rnnqFSpEnPmzMl0PVmp\nRo0aVKtWjfXr12dpPaH2448/Eh4enmqOUHbat28fR48e9azwZnIXEXkAiFbV6cBHOI+FMCFkY5w9\nbJxNbhP07W2hYre3mdwuKirKs2JZgQIFkgU/RYsWZdiwYQwZMiRVvlYPuVi8ND8A+SSRRHU+8BYp\nArfcAsuXQ/Hi2dCBAMyfP5+ePXt6AqHbbruNpUuXBpz//fff57333qNRo0Y8/vjjWf7BesSIEfTt\n25cKFSpkaT15RVxcHC+99BKvv/663+cA2e1tOUtEngJuVNUBInIzsFZVr8rpduUlNsbZw8bZ5DYW\n9BiTjtjYWFq1asWGDc7ChN9++y29e/dmz549gBPwPProo3w8axZy8CBs2+Zsa9fC1q2M+v1xhutQ\nEiiYquxChaBGDVi9GrwWajMmx1jQkz4RqQR0VdU3A0zfH/heVVOubpqqPBEJA4qr6kkReQJooaod\nfeULhYz2JbeyMc4eaY1zyn5m9zgbk56QLVltTF6TmJjIpEmTePrppwF49dVXGTp0KGFhYcyeOZPm\nLVqQkJBAnWLFmL51K5J0uaZgQTh3zjNp51Y2U5QLnPYR9Fy6BDt3wgMPODGSj2duGmNyn4JA0ZQH\nRSQ/8L77vLdwoBvQOL3yVDUBOCkipYAOQKcQtdkfn33JDiJSDPgR6KmqqwLMY2OcAZkZY3e+zIxz\nsn7mwDgbkyb7iGWMD9u2bfOsDHbdddfx3XffUaVKFc/5xk2b8v8KF2b5qVMsPXqUgkeP/pXZe+UC\noK7sIF5TBzz580OBAs6iBqNHW8BjzOVOVV1AH+9jIlICZz5rH5+ZfHB/4HwFeExVj6aX/nKlqudE\nZBQQ8OS83DbGIlIIqK6qP2W2jKyUmTF258tV42xMKOTcbFpjcqEzZ87w6KOPegKeJUuWEBMTkyzg\nASBfPt6ePp2VRYqQ5l1pIlQtc865jy2FFi1g61aYPh2uvjp0fTDG5B6qekZVO6vqmQxkexL4l6r+\nISJdsqptuYGqTlPVC+mnTLOMnBzjikD7IPJnuVCMsbscey+by5oFPcbgrDY2d+5cSpYsyVdffUXv\n3r05f/68z4dzJin9j39w4y23pP2k0ZIlkajvqHGT86tWrBjUrw+rVsGyZXDjjaHuiTHmciYi7YEx\nwHYROYqzQqoJIRvj7BGKcRaRsAC2/CFvvMmT7IYac8Xbt28ft956K2fPnqVQoUJs376dGjVqBJa5\nc2fwt2xysWKwciXcdBONGsHJkzBxIjz8cNpxkjHm8iIivYDSwE2q+k/3sW3AMFX9KiNlqep8YH7o\nWxk4ERkK3AxMAWrifFZoBgxR1f0i8pL7/BtAI6AMcJWqDnXnrw48BcS4zx1V1Yle5V+DM8fjJpzn\n/L0bQJtsjC+jMYbgx1lEmgMrA0gaCdga/CZdFvSYK9alS5cYNGgQH3zwAQCzZ8+mS5cugT1Yc9s2\naNr0r/k7+fODy/XX+aJFYckSaNAAgGHD4MMPbd6OMXmN+4GL54BvceZN/NN96lvgumxsRxjOxHP/\nT0/+y2eq+j8/5fwd+AIQ4F/A3ap6WkRcwDgRmYbz0PGrgInA34HngQeBoe7xmA+0VNXf3GX+KCIb\nVXWD+6HmA1T1BRFpC4wC0vxALiL1yAVj7EeGv8KyMQ7Y98BtAaTLyO125gp2xX8Ee++997jtttto\n0qRJltcVHR1NeHg45cqVy/K6cqO4uDiWLVtGq1atcropLFu2jPvvvx+AVq1aMWfOHEqWLJl+xgMH\noE0b2LLF2Z8xA5o3h9q1/wp6ihSBefMgIsKTrWrVEHfAGJNbFMf5ADsU8P42fAGBBSAh4V4p64kQ\nFHVKVX8SkabAZFU97T5eGKgFlFXVZSLSDJiiqudFZDoww/1h+1NgUtKHcbeLOFcxNuDMgZnuPt4U\n2BpAm0qQC8ZYRArjBCHen52KAzeIyHUpkn+kqqv9FGVjHABVPeu+0pRu0ixvjMkTrpigx+Vy0a9f\nP+Li4pIdj4mJYdasWWzcuDFL64+Ojmb79u00atQIgL179zJ9+nSKFi3KyZMnyZcvH6NGjaJgwdSr\nfAWSPi4ujnfeeQeXy4WIcOzYMYYPH07xNJ56eeTIESZMmIDL5WLLli00bdqUl19+mbA0LkfExcXx\nwgsvUKFCBVwuF7Gxsbz55puePF26dKFDhw7Uq1ePcuXKUaCA87cyLCyMggULcuLECebNm0eHDh0y\nNY7BOnToEM2bN+fnn38GYNOmTdSvXz/9jCdOwJNPOsEMwKuvwssv/7VAQa9eMHWqc8Vn2jT4+9+z\nqAfGmNxEVaPcH0R7Ap29Tl2H8+H0sqKq34lIAZwP0M94nbod+FlV57qXQW4EdHTnOQAgInfgfGj3\n9Nu9utmNwGF32j+8yvwb8GEAbcoVY6yqF/nrCggAIlIN6K6qIzJQjo1xAOz2NhNyqporNqcp2ev0\n6dPaqVMnPX36dJbWc+HCBf3HP/6R7Fj9+vV17ty5nv2GDRtqr169/JaRXvpXX31V//jjD8/+999/\nr927d/dbXmJiov7zn//U8+fPe9pYu3ZtHThwYJp9ef7553XAgAGe/WeeeUYHDx7s2a9bt66KSKpt\nyJAhnjRdunTRX3/9Nc16Qi0+Pl5HjhypON8I6VtvvaUulyv9jBcuqL78sio4W8eOqidOpE4XG6ta\nvLjqpEmhb7wx2cj9tzjH/0/IzRtQDWeeQ9J+bZxv2vN5HXs2RZ4+QEQg5WWyTQVw5ofMDGC7P626\ngTuAg177RYGTwMPu/QdwPpynbENH4FCKY/8P+BMoluJ4aSABCA9wjHJ8jDNbbm4ZY5yAphfOlZxb\ncvsY41xFaxDAViPUr6tteXPL8QZ4GpIDQU92GTNmjM6fPz/Zsfr16+ugQYM8+x06dNCaNWv6LSO9\n9Lfddpu7E1GYAAAgAElEQVSePXs2WZ6IiAi/5e3Zs0dr166tmzdv9hx74403tGDBgnrp0iWfeS5e\nvKglS5bUNWvWeI5FRUVp2bJlPfudOnXSNWvW6Lp163TDhg0aFRWl7du3T1bmxo0btXfv3n7bFmrr\n16/3BDsNGjTQP//8M/1MLpfqtGnqCXYaNFA9cCDtPLt2habBxuQgC3oC+v8qZdBTBzjitf8ozrNb\nwLllaSDO/IRs/UCemb64j70CbPfafx341Gt/HPChj7JqAEeBMPd+BWA3XkFWijGKycAY5coxDiLo\nya4x/sX984NAXffPbYCtl8sY22ZbqLYr5va2nDRnzhw2b96c7NimTZuS7e/du9dz65sv6aW/+uqr\nue+++/j444+pXr06kZGRtGjh/2pvwYIFOXLkCD///DO33norAMWKFSM+Pp7Tp0/7nHe0detWzpw5\nQ/Xq1T3HqlWrxvHjx9m8eTM1atSgV69e3HnnnZ7z48aNY+zYsclu22vUqBGPPfYY58+fp2jRrHtI\n9YkTJ2jbti3ffvstAKtXrybCa56NX8uWgXu+D0WLwrp1UK9e+vluuimI1hpjLleq+qOIzBKR14BD\nwE5V3ec+dxGYKCINyMSk9xzSAlgpIi/iPNoigeTLDd+Is2hCMqr6s4gMAt4Ukb3udJ1UdVPKtEBL\nYIU7X7pjZGPsyMQYJ90eVgNnQYQBwF58LE6QB8fYmGSCDnpEZAbOL9IRVa3r43xz4N/AL+5DX6rq\n68HWG6jp06dz4sQJdu/ezdSpUwGoV68eI0aM4JFHHsny+nft2kWZMmXIn9//MvLr1q3j+PHjvPXW\nWwGV6Sv9uHHjaNGiBfXq1aNXr15UrlyZYcOG+S2jWrVqHD2a/OHIGzdupE6dOn4XWvjtN2fOZLFi\nxTzHSpQoAcDBgwepX78+LVu29JyLjIykcuXKhIeHpyqrcePGrFy5MksWNVBVJk+eTN++fQEYPHgw\no0aN8swv8mvzZmjcGBISnP1vvgGv/hhjjD+qOjin2xAK7vkhTYF/Jn3gTUlV2/rLr6qfAJ8EUNW9\nOJPmA5ZLxzgOyNCDP3NojN/HuV0MnLlES/2UnRvH2JiQCMWVnpk4q5l8nEaa1araOgR1Zci2bdso\nVqwYERERNG3a1BP0REREsH///oDLSUhIoF+/fsTHx6ebtmPHjp5VwQA2b97MjX6eQBkTE8OSJUv4\n7LPP+OijjyhfvnyaZaeVvmbNmvTt25fVq1czadIkGjRoQKdOnahWrVpAffzll1/48ssvWb58ud80\nF9zLMxcuXNhzrJB7Iv+ZM8lXjIyPj+ftt99m0aJFPsuqXr06W7ZsCXnQs2PHDurUqQNAxYoV2bBh\nA9dee23amX79FR56CH76ydmfNQu6doV89uxeY8wV53bguL8P45klIlfhXHV4EmeuiQALQ1lHTlBn\n0YBxGcyW7WOszup+J0WkFM7zezqFsm5jLgdBBz2qusbHUo0p5cil0DNnztCuXTtGjhyZ7KrOo48+\nGlAAkyQsLIwpU6Zkqg1HjhyhdOnSPs+Fh4fTv39/unXrRt26denSpQuvv+7/Ilha6bt370737t0Z\nNmwYCxcupF+/frRu3ZqtW9NfqTIuLo4ePXowdepUmjVr5jddqVKlUh07e/YskDwQAvjkk0+oVauW\n37LKli3Lrl270m1boM6dO8c///lP5s6dC8BXX31F27Z+vyhzHD8O//wnLFjg7I8YAc8//9eKbMYY\nE1q5emldEXkAZ25JfhF5SVVHh7J4nKsiDwHXALe7b5lKKdgxsjH2M8Yikh9nLtFjqnrUdxEBydVj\nbIw/2fFVtgJ3iMhWEVkiIrWzoU4AmjVrRv78+ZkxYwaPP/645/j+/fu5++67AZg8ebJnzkdWuHTp\nUppLQAMUL16cNm3a8MYbb3DgwIF0y/RO/+uvv7J27VpU1TOHp23btkRHR3Po0CG2b9+ebnlPPfUU\nzz77LF26dEkzXZUqVQA4deqU51jSFZ6UV1OmT59OzZo1/ZZVsGBBXN4P8wzCl19+SfHixZk7dy6P\nPfYY586dSzvguXjRCW7KlnUCnq5d4eRJGDrUAh5jTCDiyeAtTW7+vgDMbHmh4KlbVf+rqrepaqUQ\nfxhHVU+q6u2qOkJV/6mqsX6SBvslqY2x/zF+EviXqv4hImn/h5+23DjGxqQrOxYy2ARUVefhWg/i\nXGr1eb/X8OHDPT83b96c5s2bB135zp07iY2NTTa5/uTJkwBMnDiRWbNm8fbbb6dZRnx8PP3798/U\n7W0VKlQgJiYmWZqoqCjatm3LjBkzePjhhwEoUKAAqposoAgk/enTpzl8+DAVK1ZMlqdKlSpERER4\nbj/zZ+zYsbRp04YHH3wQgM8//5y///3vPp/vU69ePcqWLcsvv/ziubXup59+okSJEtSt+9d0rjNn\nzrBhwwZefPFFv/UeP3483dv50rN//34aNWpEbGyspy1pXV3C5XKeofPkk85+o0ZO0HPNNUG1w5jL\nWWRkJJGRkTndjMuKqh4iA7c0iUg/oLHzo+RX1WTPHsloeaGUk3V7S2+Mgs1/pY+xiLQHxgDDncfx\n8D0wJ4Nl5NoxNiYgoVgCDmcVkO0Bpo0Byvg4rllh+/btWr58ec/+l19+qXv37vXsd+/eXSMjI7Ok\nblXVyMhIfeSRR5Id27Jli15zzTW6bt06VXWemdOgQQO98847PWmWLFmivXv3VpfLpZs3b04z/YkT\nJ7RBgwbJnn1z6NAh7dy5s2f/3//+t/bq1SvZs2lmzpypTz31lP73v//VpUuX6tKlS7Vbt24+25Bk\nwIAB+n//93+e/a5duyZ7To+q6rZt21REdMWKFX7H5ZlnntGZM2f6PZ+WS5cu6aBBgzzLUE+bNk0T\nExP9Z0hMVF2yRD3LT191ler27Zmq25i8Dluy2jbbbLPNtjy4iWrwt2a65/T8R32v3nY1zspuKiKN\ngXmqep2PdBqKtvgyePBgChcuTOXKlalVq1ayK0g9evSge/funtvdQi0uLo6bb76Zn3/+OdnxZcuW\nsXnzZs6dO0dMTAylSpVi1KhRlCxZEoDx48czfvx4du7cSZEiRdJNv3v3bsaNG0elSpUQEVwuFy+8\n8ILn/NixYxk7dixr166lZs2a7Nq1i1tuuSXV1au77rqL1atX+2wDOHNnnnnmGapVq4bL5SI2Npa3\n3nor2ZLU+/bt46677uK7777j+uuv9zkuTZo04auvvqJy5coZGs9Vq1Z5buNr2bIlX3zxhc+5Rh4/\n/AC33eZdAITgCqIxeZWIoKq2JK0xxpg8JeigR0TmAncD5XCeCDwM58nQqOpkEekP9MVZg/48ztN9\n1/soJ8uCnrRkddAD0LlzZ55//nluueWWLKsjEJs3byYsLCzZrWg5ITY2ljZt2hAVFRVwnj///JN7\n773XM0dp48aNaT7XiJgYeOAB2LPH2Z8zBzp1ArHPcsakxYIeY4wxeVHQCxmoaidVrayqBVW1qqrO\nUNXJqjrZff49Va2jqreq6h2+Ap6cJln8QXjEiBFMmDAhS+sIxPr167n55ptzuhmMHz+e1157LaC0\nLpeLN998k4oVK7J9+3beeOMNEhIS/Ac8x45B69Zw/fVOwDNqFFy6BJ07W8BjjDHGGHOFyo6FDHK9\nrL7CVKNGDapVq8b69etp2rRpltblz48//kh4eDj5cvjZM/v27ePo0aOeW9TSsmnTJho2bAhA7dq1\nWbFiRaoFGzwuXIBXX4WkB7b26AHjx4P79j5jjDHGGHPluqKDnvfff5+NGzeiqrhcroA+iGfW0KFD\nGTFiBNdffz0VKlTIsnr8qVOnjuehnTklLi6ODz74gHfffTfNdKdOnaJjx47897//BeCbb76hZcuW\nvhO7XPDhhzBggLN/xx0wfz5kcK6QMcYYY4zJu0KykEEo5NScHpN7qCofffQRPXv2BJznB40bN873\nstuqsHgxuJfwplw5WL0aamfbY6CMyZNsTo8xxpi86Iq+0mNyj927d3PzzTfjcrkoVaoUmzZtIjw8\n3Hfi6Gho3Piv/W+/hbvuyp6GGmOMMcaYy07OTvAwV7wLFy7Qo0cPatasicvl4vPPP+f48eO+A559\n+6B69b8Cns8+g8REC3iMMXmaiPQUkV4i8pWI5OwyoFkgr/cvo/LyeOTlvpncz670mBzzn//8h9at\nWwPQvn17ZsyYQfHixVMnPHoUunWDpUud/bFjYdAgKFAgG1trjDHZT0QeAKJVdbuIxAIfA3nmw2Je\n719G5eXxyMt9M5cHu9Jjst1vv/1G1apVPQHP9u3bmTdvXuqA5/x5ePppqFDBCXieeAJOn4YhQy6r\ngOe9995jw4YN2VJXdHQ0sbGx2VJXbhQXF8fXX3+d080wJpRuBPq4f94LXJdzTckSeb1/GZWXxyMv\n981cBuxKj8k28fHxDB06lDFjxgBOMNC3b9/Uz0lKSID333cCHoCICPj8c/C3XHWInTt3jjp16jBj\nxgzuueeegPO5XC769etHXFxcsuMxMTHMmjWLjRs3hrqpyURHR7N9+3bPM4z27t3L9OnTKVq0KCdP\nniRfvnyMGjWKggULBlzmQw89xLRp06jsXg2vS5cudOjQgXr16lGuXDkKuIPPsLAwwsL8/zlZsmQJ\nM2fOZP78+enWGRcXxzvvvIPL5UJEOHbsGMOHD/cExZcuXeKVV16hVKlSJCQkADBs2DAAChYsyIkT\nJ5g3bx4dOnQIuJ/GZISI3AD0wnngdikgEXhZVePSzPhX/iVAb1U95N6fA8wDtgGxQLw7aQLwPpD0\njVAzYGmIupFeGx8Ceqhq+wDSFgQGAfkBBcoCw1X1rPt8IeB14CTuzx2qOsKdPUf6Fyh338YAR3D6\nVw4YrKoJAeT1OYbplJmt45HZ/qXTN3/vhVz9WpsrgKrmis1pismr1qxZozh/APWOO+7Q2NjY1IkS\nE1W/+krVWZtN9eqrVXftyv7GqurUqVP1/PnzQZdz+vRp7dSpk54+fToErfLvwoUL+o9//CPZsfr1\n6+vcuXM9+w0bNtRevXoFXObnn3+uIqK//vqr51jdunVVRFJtQ4YM8VnGwoUL9dlnn9V7771X77nn\nnoDqffXVV/WPP/7w7H///ffavXt3z37nzp116NChnv3HH39c33///WRldOnSJVm7TeDcf4tz/P+E\n3LwBm4COXvvfA9MCzNsBJ0i61uvYNvexlNtYrzSlgG+A8kG0uxBQO500bYC3gGXAygDLHQlU9Npv\nCMz02p8DjPDanwX0TVFGUP0LpG+ZLHcMMNFr/x1gXDBjGEiZ2TUeGe1fAH1L870Qir7ZZltmtxxv\ngKchFvTkSUePHtVGjRp5Ap61a9f6Trh+vXqCHVD97rvsbehlbsyYMTp//vxkx+rXr6+DBg3y7Hfo\n0EFr1qwZUHmnTp3SgQMHpgp6OnXqpGvWrNF169bphg0bNCoqStu3b6+XLl1Ks7zhw4dr8+bNA6r7\ntttu07NnzyY7FhERoaqqe/fuVRHR6Ohoz7mPPvpIGzZsmCz9xo0btXfv3gHVZ5KzoCeg/682AW97\n7X8O7AwgX0lggo+g51PgTqAp0Bi4A+fKT0H3+fzAv4BKQba7GjAswLTDgFUBpo0GiqU4ttr9b3V3\nf2/zOtcN+N5rP+j+ZaRvGSizEHAKuNPr2B1AbGbHMJAys2s8gumfv/dHWu+FUPXNNtsyu9mcHpMl\nEhMTeffddylfvjzR0dEMGzaM+Ph4br/99uQJf/4Zrr0WmjZ19ufPd1Zka9Ys+xt9GZszZw6PPPJI\nsmObNm3i7bff9uzv3bvXc+tbeiZPnkyfPn2SHTt79iy9evXizjvvpGnTpjRu3JjvvvuOsWPHpnvL\nnGrgz+C6+uqrue+++9i3bx8AkZGRngcHb9q0CYCyZct60leqVIktW7Zw+vRpz7FGjRqxZs0azp8/\nH3C9xgRKVRuo6rNeh27A+bCXnj7AZO8DIlIcmK6q36nqelXdiBMAPa9/3S73JPAvVf1DRLqEoAuB\nyMizmv4ElolIdQARaQ6sdJ9r4P73mFf6P4BbRaSkez8n+heIW4ASwD6vY78CZUSkfgD5fY1hIGVm\n13gE0z9/74+03guQe19rcwWwOT0m5LZu3cqtt94KwPXXX8+3335LlSpVkic6cgS6dIFvvnH233oL\nnnoK0pgXMnLkSHbs2METTzzBrl27SEhIICoqinHjxnHdddcxevRoduzYwYsvvkh0dDTHjx/n1KlT\njBw5EoB9+/YxYcIEwsPDOX78OOXLl2fgwIGe8n///XfmzZvH7t27qV27Nk8nzSlKx/Tp0zlx4gS7\nd+9m6tSpANSrV48RI0akCkSywq5duyhTpgz58+f3m2bdunUcP36ct956K93yvv/+e2rVqkWxYsWS\nHS9evDgtW7b07EdGRlK5cmX/z1PKpHHjxtGiRQvq1atHr169qFy5crI5OynFx8eTmJjI77//Tm2v\nh9M2btyYlStX0qpVq5C2zxhvInI7UAb4v3TS3QbsBM55H1dnrsMKr3TNgUOqGuPeb49zC9Jw9/zH\n73FuF8tNhuB8sN0mItNx2p80Z8fXPKcCOAspVRWR2uTe/lV1/+v9mp1x/1sF2BzqMt3BQnaNR1b0\nz+974TJ5L5s8LOigR0RmAH8HjqhqXT9pJgAP4kz67K6qmflFMrncmTNneOyxx/j3v/8NwNKlS3ng\ngQeSJzp3zll97f33nf1+/ZwlqH0tVe1l8eLFtGvXDlXlueeeY/Xq1ZQsWZL8+fMzZMgQevfuTevW\nrTl16hQDBw5k8eLFjB07lqVLlzJy5Ei2bt1K+/btWbFiBVWrOn/n69SpQ+PGjWnSpAmqyqRJkxgz\nZgwLFy7k5ZdfDijo2bZtG8WKFSMiIoKmTZt6gp6IiAj2798f8NglJCTQr18/4uPj003bsWNH7r//\nfs/+5s2bufHGG32mjYmJYcmSJXz22Wd89NFHlC9fPs2yXS4XX3/9NcOHD0+z/fHx8bz99tssWrQo\n3fZmVM2aNenbty+rV69m0qRJNGjQgE6dOlGtWjWaNWtGgQIFOHr0qCfY+umnnwCSXekBqF69Olu2\nbLGgx2QJEQkHHgI64vy/djSNtPmBVqo6XESuSyNdAeBZVW2ddExV5wPprwASYLNDVE5Ku4APgLuB\nAcAmEZmrqr8CUTgLM5QHYtzpk76dKBHC/mVF34q4/73odeyS+98SWVGmqs4Fvshk2d4CGY+s6J/f\n90KI38vGZFgorvTMBCbirLeeinuFjxtUtYaINMH5ZWgagnpNLqGqzJ07ly5dnCvVTzzxBO+++y6F\nCxf+K1FCAkyYAP/n/jL0b3+DTz6Bq68OqI6rrrqK2rVrs379evr06UPJks5dERcvXmTnzp0cO3aM\n++67j6ioKJ544gmKFi1Kr1696NmzJ6pK586dGTBggCfgAShcuDBRUVE0adKEw4cP06tXLwDWr1/P\nLbcE9uiAM2fO0K5dO0aOHJnsqs6jjz4aUACTJCwsjClTpgSc3tuRI0coXbq0z3Ph4eH079+fbt26\nUbduXbp06cLrr7/ut6yPPvqIHj16pFvnJ598Qq1atTLV3vR0796d7t27M2zYMBYuXEi/fv1o3bo1\nW7dupVy5crz66qvMmzePxo0bc/HiRdatWwdAuXLlkpVTtmxZdu3alSVtNMZ9JeY9EZkFbBeROar6\nip/k3XH+r0xPV5yrQUETkcI4/zd7/z9fHLjBR+D1kaquDqK6j9xljBCRtjirdC0CblHVWBF5DWcB\nh43udiXd55yp9fWD6ZuI9ADuS6eKeKA3zmpzKSV9Q3fRx7lAhLzMIMYjK/r3EX7eC5ksz5jQCcXE\nIJy11rf7Ofch8P+89ncBV/tIp+by8/PPP2vRokUV0MKFC+uePXuSJ0hMVJ0/Xz0LFFStqpoyTYDi\n4uK0RIkSustrRbdHH31UH3nkEVVVPXv2rBYsWFB/++23ZPmioqJURPTo0aOeYxcvXtQSJUronDlz\nUtXTsGFDnTp1arJjH374oa5evdpnuxITE7VKlSrJzk+fPl0vXLiQbt5QGDt2rL700kvppnv66ac1\nX758flc1O3jwoE6aNMmzHxMToyKi+/fvT5W2WbNmOmPGjIDbOGzYsIAWMoiKitKuXbsmO/b7779r\nuXLldNu2bZ5jc+fO1XfffVenTp2qCxYs0HLlyqUqa/Lkyfrkk08G3EbjwBYyyMz/geMBF16LE3id\nqwz099q/DmdifzUfab/DWQY4q9qZkYUMhhPAQgY4E99npzhWBTgK1PU61hF4CieYeAQ4mlN9y0CZ\n9d2vVXmvY0mvX8PMjGGwZYZyPIJpi5++BfResM22nNqyY05PFeA3r/3fgWtwJruZy9TFixd59tln\n+eCDDwCYPXs2Xbp0Sf7MnXXr4I47ku83zfxFvujoaEqUKMFNN90EwPnz51mxYgWzZ88GYM2aNVx7\n7bVcc801yfIdOHCAihUrJrsSsHDhQooUKUKbNm2SpT1x4gRbtmzxzF25ePEiU6dOZdasWckWBfC2\nc+dOYmNjufPOOz3HTp50vkCbOHFimnmTxMfH079//0zd3lahQgViYmKSpYmKiqJt27bMmDGDhx9+\nGIACBQqgqpw6dcpnuStXruSXX37hxRdfBJwrSODMsWnSpAmPP/444Fzd2rBhgyddIFI9i8mPw4cP\nUzHF85iqVKlCREQEhQoVAmD+/PncddddnnliL730kqdt3pLmbRkTSiLSDFgI9FTV/7gPx+PcTnSV\njywtgOtF5A33fgX3v0NEZIOqfuwutwTQBHjDRxk5IdDVRyoCh5NlVD0oIt/ivlXKPZdjjaoedO+P\nxs/dIbnMNpwFGK7H+eAOzq15Z4DtAeT3NYbBlhlKwbTFV9/SfS8Yk5OyayGDlJ94fP4xHT58uOfn\n5s2b07x586xrkcm0//3vf565Og8//DCffPKJ53YzAHbvhubN4bD7b9+CBdC2LQT4wdeflStXUqZM\nGc/+6NGjeeihhzwf6leuXJlson2Shg0bEh8fT0JCAmFhYRw5coShQ4fy8ccfp5qsv2rVKqpWreqZ\nL1K4cGEGDhzIpk2bUPX9GSAxMZGSJUuSL18+d3cX0KZNm4DyJilQoECmb28LDw9PNbemePHiFC5c\n2POhX1VZuXIlzZo1o25dZ+rd0qVLWbBgAZMnTyZfvnx07dqVrl27esqIjIxk5syZPP/881x77bWe\n4/v378flclGkSBFSWrRoEYsWLWLKlCme8Uiq35eUbWjZsiWjRo3iwIEDnjr/+OMPChcu7Jm3NGDA\nACZNmkT79u357bffWLFiBUuWLElV9p9//hnwbYpXssjISCIjI3O6GZeTszi3/hwFECeibwFEqep2\n97EHgUeBPqr6CfBJUmb3QgU9cJ7Bc8Cr3OtwlvO9kPVdCIjPP9gp+paIswjDyyJybVJ/RKQScFFV\n97izTcKZ3zFfRKoCLXHmQ+VqquoSkc+A9sAG9+FOwGR1r6znYzy8pRrDQMrMLkH2z9f7I5D3gjE5\nJjuCnoP8tUIIOFd5DvpK6B30mNzn0KFDREREeJYS3rx5s2eVNsAJcjp1gqQPUO++6yxUkMaKbBmx\ncuVKWrRowRtvvEFiYiJhYWF88onnswR79uyhX79+qfLVqFGDd955h8GDB3PDDTewZ88e5s6dS4MG\nDVKlXbFihc/AKS116tShW7duvPrqq1SuXJlatWpRvXr1jHcwk26//XZ69+6d7Ngtt9zC9OnTWb16\nNUuWLCEmJoY77riDUaNGedLs3r2b5cuXc+nSpVQBzJtvvsnixYsREfr3788jjzxCz549AShatCgV\nK1bkuuuuS9WWnTt3smDBAp577jlq1qzJsmXL+OKLL1i8eDEnTpygS5cuNGvWzPM6pWzDVVddxaef\nfsqIESOoVKkSIoLL5fJcUQTn6tm2bdvYtWsXhw4d4rPPPku2hHWStWvXMnjw4EyP65Ui5RdMI0aM\n8J/YoKpbRaQXcLd7zmo4sBZ42SvZTcC9OM9B8QQxIjIYZ+EfxZkP9JWqznCfPo/zLfn+LO9EGkTk\nPqAdTjtLi8gcnIDOvfpM8r6p6ikR6QwME5E/cPqWH+jrVexAoJ6I1MS53a+jqnovYZ2bvQCMF5FX\ncPp1EvCeu5XqtQ5gDNMrMztlqH9p9S3A94IxOScU98iR9pyeh4Al7p+bAuv9pFOTO8XHx+vw4cM9\nDxh955131OVy/ZXgzBnVJ55Qz7ydp55STfFwyWBdvHhRixQponv37g1puSnVqFFD586dm+p49+7d\nNTIyMlNlBpM3UJ06ddItW7ZkaR2B2rRpU7L5Nznh6NGjescdd+RoGy5X2JyePLsBlYAhOd0O65uN\nh2225cQWiiWr5+IsTVhORH7DeUpvAXcUM1lVl4jIQyKyF2ct+PSXhjK5xoYNG2jqnofTsGFDli5d\n+tc8ifh4GD/eWYIa4IEH4OOPIQvmUaxbt44yZcqE/ArKqVOnaNGiBR9++CEnTpxAVWnbtq3PtIHO\nSwl13kCMGDGCMWPGMH369CytJxBJK+zlpPHjx/Paa6/laBuMyW1U9Q9gXE63Iyvk5b5lho2HMakF\nHfSoaqcA0gwIth6TvY4fP07btm1Zs2YNAN9++y133XWXc1IV5s2Djh2d/fBwWLYMbrghS9ryv//9\nj5dffhmXy8Xo0aN56aWXQla2qlKwYEGWLFnC77//zrp165IvtZ0ibTD1ZKUaNWpQrVo11q9f7wlS\nc8KPP/5IeHh4svk82W3fvn0cPXqUFi1a5FgbjDHGGJO7ZNdCBuYyoapMnjyZvn2dW3Cff/55Xnvt\nNQoUKOAk+O47SAp+ADZsgMaNs7RN999/f7LVykKpVKlSnue8+PP++++zceNGVBWXy5WhD9PB5M2o\noUOHMmLECK6//noqVKiQfoYsUKdOHerUqZMjdQPExcXxwQcf8O677+ZYG4wxxhiT+0hWfwMdKBHR\n3LPY7+YAABO6SURBVNKWK9WOHTs8H1grVarEhg0b/nqY586dEBEBse5nyS1aBK1aBb0imzEmdxER\nVNV+sY0xxuQpOXcPisk1zp07R8eOHT0Bz8KFCzl06JAT8PzxBzRrBrVrOwHPpEmQkAAPP2wBjzHG\nGGOMuSxY0HMFU1W++OILihcvzueff063bt04d+6c88DOM2egZ0+oXBnWroVnn4Vz56B/f8ifP6eb\nbowxxhhjTMBsTs8Vav/+/TRs2JDjx48DzvNVatasCXFx8MYbkLRYQKtWMHMmlCuXg601xhhjjDEm\n8+xKzxUmLi6OQYMGER4ezvHjx5k2bRqJiYnUvOkm+PRTKFTICXhq1IB9++A//7GAxxhjjDHGXNbs\nSs8VZNWqVZ7Vw/72t78xf/58SpUqBatXg9cT2fn+e2jYMGcaaYwxxhhjTIhZ0HMF+PPPP/nb3/7G\njz/+CMD3339Pw4YNYccO5xk7J086CRcvhgcftAUKjDHGGGNMnmK3t+VhLpeLsWPHUrFiRX788UfG\njBlDQkICDStWhKZNoU4dJ+D54ANnRbaHHrKAxxhjjDHG5Dl2pSeP+uGHH7jtttsAuPnmm/nmm2+o\nWKQI9OgBs2c7iYYMgeHDoUiRnGuoMcYYY4wxWcyu9OQxJ0+e5L777vMEPCtWrODHTZuoOGUKlCrl\nBDyPPALHjsHYsRbwGGOMMcaYPM+CnjxCVZkxYwalS5dm+fLlPPPMM1y6cIEWBw86K7ING+Y8YDQm\nBhYsgDJlcrrJxhhjjDHGZIugb28TkQeA8UB+YJqqjk1xvjnwb+AX96EvVfX1YOs1f9m1axc333wz\niYmJlC5dmh9++IHw/fv/uooTFgYbN0L9+jnaTmOMMcYYY3JCUEGPiOQHJgF/Aw4C0SKySFV3pki6\nWlVbB1OXSe3ChQv07duXWbNmATB//nz+ceONSN26cO6ck+i//4X/3979B2tZ1nkcf38BDyhSKIkG\nIhFhbiVoLkirMidHjXQDnTS0YRybatAptdrdkSgbcFoTmybD0jGThlp/pOzaorgrlJ3E/BEkIAqs\nQDmhJsLiOshv4bt/nMcz/DjneDjPOdwP93m/Zs7w3Ndz3ff9fQ6jnM+5rvu6PvWpAquUJEmSilXt\n9LZRwOrMfCkzdwL3AeOb6eeSYB3soYce4ogjjmDWrFlMmDCBt1au5OLvfY8YMaIx8PzsZ7Brl4FH\nkiRJXV61oWcgsHaP45crbXtK4B8iYmlEPBIRH6nynl3a2rVrGThwIOPGNQ6cLfvDH7ivWzd6n3QS\nPPssTJkCW7fCF78I3XxkS5IkSar2mZ5sQ59ngUGZuSUiPg38GjixuY5Tp05tel1fX099fX2V5ZXH\nzp07uf7665k+vfGRqdtmzODK118nzjijscMll8Add8BRRxVYpaRDTUNDAw0NDUWXIUlSp4rMtuSW\nFk6OGA1MzcyxleNvArv3Xcxgn3P+ApyWmRv3ac9qaimzBQsWMGbMGADOOussfn3JJRx9zTWNb44Y\nAXPmwAknFFihpLKICDLTKcmSpFKpdv7TImBYRHwgIuqACcCcPTtExLEREZXXo2gMWhv3v5T2tX79\nekaOHNkUeJ760Y94fMGCxsBz+OGwZEnjl4FHkiRJalFVoScz3wa+CjwKLAd+lZkrImJSREyqdLsY\nWBYRS2hc2vrSau7ZFezevZtbbrmF/v37s2jRIqZeeSU7e/Zk9LXXNnaYPx+2bGkc5ZEkSZLUqqqm\nt3Ukp7c1Wrp0KaeccgoAQ084gcd792bAisoK4D//OVx+uQsUSOo0Tm+TJJWRPz3XiE2bNnHhhRc2\nBZ7/PvNMVv/1r42B5zvfgW3b4IorDDySJEnSAXKkp2CZyT333MPEiRMBuPLUU/nh4sX0Avj85+En\nP4G+fQutUVLX4UiPJKmMDD0FWr16NcOHD2fr1q0cXlfHczt28CGA006DBx+EQYOKLlFSF2PokSSV\nkXOlCrBt2zauuuoqhg0bxtatW7kb2LxjBx868kh47jlYtMjAI0mSJHWQajcn1QF69NFHGTt2LADj\ngV8CfQAeeww++ckCK5MkSZLKydBzkLz66quMGTOGNWvWALAEGAHwy182PrvjAgWSJElSp/An7U72\n9ttvM23aNAYOHMiaNWu4BdgFjLjhBti+HSZONPBIkiRJnciRnk709NNP84lPfAKAkcAjwPsuvxxm\nzID3vrfQ2iRJkqSuwtDTCTZu3MiF48ez4IknAFgAnDl6NMyeDQMHFlucJEmS1MU4r6oDZSa33XYb\n/fr1Y8ETTzAZ2NG3L2c+/zw89ZSBR5IkSSqAIz0d5IUXXuBjH/sYAAOBp4BBv/89jBlTaF2SJElS\nV+dIT5U2b97MhAsuaAo8/wm8fM89DNq928AjSZIk1YCqQ09EjI2IlRGxKiKua6HPjMr7SyPi1Grv\nWQsykwdmzuTII4/k/kce4Qpgyw03MG7HDrjsMgg3NJckSZJqQVXT2yKiO/Bj4BzgFWBhRMzJzBV7\n9Dkf+FBmDouI04HbgdHV3Pdge3zuXObNmEGP7dt5u2dPho8fz5Xf+AZvbN9ON2D5Zz/Lh2fOhPe8\np+hSJUmSJO2j2md6RgGrM/MlgIi4DxgPrNijzzhgFkBmPhMRfSPi2MxcV+W9D4rH587l0Wuv5V8r\nm4oCTJg3j53AzGHDuKKhgRgwoLgCJUmSJLWq2ultA4G1exy/XGl7tz7HV3nfg2bejBl7BR6AXwFf\nGT2aL7z4ooFHkiRJqnHVjvRkG/vt+4BLs+dNnTq16XV9fT319fXtKqoj9di+vdn2Xj17HuRKJKnj\nNTQ00NDQUHQZkiR1qmpDzyvAoD2OB9E4ktNan+MrbfvZM/TUirdbCDe7evU6yJVIUsfb9xdM06ZN\nK64YSZI6SbXT2xYBwyLiAxFRB0wA5uzTZw5wOUBEjAb+71B5ngfgvGuu4VtDh+7VNmXoUM69+uqC\nKpIkSZJ0IKoa6cnMtyPiq8CjQHfgrsxcERGTKu/fkZmPRMT5EbEa2Ax8oeqqD6IxF1wAwPW33kr3\nbdvY1asXY6++uqldkiRJUm2LzLY+ltO5IiJrpRZJ6qoigsx0ozFJUqlUvTmpJEmSJNUyQ48kSZKk\nUjP0SJIkSSo1Q48kSZKkUutyoWfhwoVs2LChU669Y8cOHn744U65tiRJkqT2qXZz0kPKwoULWbZs\nGSNHjgQaQ8rkyZPp378/u3btYsOGDXz/+9+nR4+Wvy2tnVNXV8cbb7zB/fffz+c+97mD9bEkSZIk\ntaLLLFm9bds2Jk6cyOzZs5vaJk+ezObNm7n11lsB+PrXv85hhx3GzTff3OJ12nLOxIkTufHGGznh\nhBM66dNIUudwyWpJUhl1mdAzffp0hg4dysUXXwzA9u3b6d+/P3PnzuXMM88E4Mknn2TcuHEtTn9r\n6zkLFy7kpz/9KXfeeWenfR5J6gyGHklSGXWZZ3ruvvtuLrrooqbjpUuXsmnTJoYOHdrUNnjwYDZu\n3MjixYubvUZbzxk5ciQLFixgy5YtnfBJJEmSJB2ILhF6Vq5cydFHH0337t2b2tauXQtA7969m9r6\n9OkDwCuvvNLsdQ7knFGjRvHYY491QPWSJEmSqtElQs/ixYs58cQT92rbunUrAL169Wpq69mzJwCb\nNm1q9joHcs7QoUNZsmRJlZVLkiRJqla7Q09EHB0R8yPixYiYFxF9W+j3UkQ8FxGLI+KP7S+1/V5/\n/XWOOuqovdr69t2/3LfeegvYO9S095x+/frx2muvtateSZIkSR2nmpGeycD8zDwR+G3luDkJ1Gfm\nqZk5qor7tdv27dv3W4Z64MCBALz55ptNbe+M1rS06tqBnFNXV8euXbuqrFySJElStaoJPeOAWZXX\ns4ALW+lb6EpA/fv3Z+PGjXu1DR8+nH79+vHnP/+5qW358uX06dOHk08+udnrHMg5Gzdu5JhjjunA\nTyFJkiSpPaoJPcdm5rrK63XAsS30S+A3EbEoIr5cxf3abciQIaxbt26vtu7du3PppZfywAMPNLXd\ne++9TJo0ibq6OgDmzJnDl770JXbv3t3mc96xbt06PvjBD3bWR5IkSZLURq3u0xMR84HjmnnrW8Cs\nzDxqj74bM/PoZq7x/sz8W0QcA8wHrs7MBc3067R9enbs2MFHP/pRVq1atVf75s2b+drXvsbgwYPZ\ntWsXGzZs4Ac/+EFTgJk+fTrTp0/nySef5KSTTmrTOe84/fTTefDBBxkwYECnfCZJ6gzu0yNJKqMe\nrb2Zmee29F5ErIuI4zLztYh4P/B6C9f4W+XP9RHxIDAK2C/0AEydOrXpdX19PfX19e9Wf5vU1dUx\ncuRIli5dyogRI5rae/fu3eoGotdddx3nnXceO3fubPM5ABs2bKBHjx4GHkk1r6GhgYaGhqLLkCSp\nU7U60tPqiRE3A/+bmdMjYjLQNzMn79PnCKB7Zm6KiN7APGBaZs5r5nqdNtIDsGrVKm666Sbuuuuu\nAzrv9ttvZ9KkSXTr1vaZgN/+9rc5++yzOfvssw+0TEkqlCM9kqQyquaZnpuAcyPiReDsyjERMSAi\n5lb6HAcsiIglwDPAw80FnoNh2LBhDB48mKeffrrN5zz//PMMGTLkgALPmjVrWL9+vYFHkiRJqhHt\nHunpaJ090vOOadOmcdVVV9G/f/8Ov/aOHTuYMmUK3/3ud1vc60eSapkjPZKkMupyoUeS1DJDjySp\njKqZ3iZJkiRJNc/QI0mSJKnUDD2SJEmSSs3QI0mSJKnUDD2SJEmSSs3QI0mSJKnUDD2SJEmSSs3Q\nI0mSJKnUDD2SJEmSSs3QI0mSJKnUDD2SJEmSSq3doSciLomIFyJiV0R8vJV+YyNiZUSsiojr2ns/\nSZIkSWqPakZ6lgEXAY+31CEiugM/BsYCHwEui4i/q+KehWpoaCi6hDaxzo5lnR3nUKgRrFOSpLJp\nd+jJzJWZ+eK7dBsFrM7MlzJzJ3AfML699yzaofIDhnV2LOvsOIdCjWCdkiSVTWc/0zMQWLvH8cuV\nNkmSJEk6KHq09mZEzAeOa+atKZn5UBuun+2qSpIkSZI6SGRWl0si4nfAP2Xms828NxqYmpljK8ff\nBHZn5vRm+hqQJKkGZGYUXYMkSR2p1ZGeA9DSP5CLgGER8QHgVWACcFlzHf1HVpIkSVJnqGbJ6osi\nYi0wGpgbEf9VaR8QEXMBMvNt4KvAo8By4FeZuaL6siVJkiSpbaqe3iZJkiRJtayzV29rs7ZudlqU\nQ2GT1YiYGRHrImJZ0bW0JiIGRcTvKn/fz0fENUXXtK+I6BURz0TEkohYHhHfK7qm1kRE94hYHBFt\nWWCkEBHxUkQ8V6nzj0XX05KI6BsRsyNiReXvfnTRNe0rIj5c+T6+8/VmLf53JElSraiZkZ6IOAnY\nDdxBCwsjFKWyyer/AOcArwALgctqbapeRJwFvAX8IjNPLrqelkTEccBxmbkkIo4E/gRcWIPfzyMy\nc0tE9ACeAP45M58ouq7mRMQ3gNOAPpk5ruh6mhMRfwFOy8yNRdfSmoiYBfw+M2dW/u57Z+abRdfV\nkojoRuP/l0Zl5tp36y9JUldUMyM9bdzstCiHxCarmbkAeKPoOt5NZr6WmUsqr98CVgADiq1qf5m5\npfKyDugO1OQP6xFxPHA+8DNaXlSkVtR0fRHxXuCszJwJjc8l1nLgqTgHWGPgkSSpZTUTemqcm6x2\nksrKfqcCzxRbyf4ioltELAHWAb/LzOVF19SCHwL/QuNIaS1L4DcRsSgivlx0MS0YAqyPiJ9HxLMR\ncWdEHFF0Ue/iUuCeoouQJKmWHdTQExHzI2JZM1+fOZh1tENtzAEsmcrUttnAtZURn5qSmbsz8xTg\neGBMRNQXXNJ+IuIfgdczczE1PooCnJGZpwKfBr5SmY5Za3oAHwduy8yPA5uBycWW1LKIqAM+AzxQ\ndC2SJNWyjtqnp00y89yDeb8O9AowaI/jQTSO9qidIuIw4N+Bf8vMXxddT2sy883KMux/DzQUXM6+\n/gEYFxHnA72A90TELzLz8oLr2k9m/q3y5/qIeJDGaaMLiq1qPy8DL2fmwsrxbGo49NAYIP+UmeuL\nLkSSpFpWq9Pbau031k2brFZ+szoBmFNwTYesiAjgLmB5Zt5SdD3NiYj3RUTfyuvDgXOBxcVWtb/M\nnJKZgzJzCI3TnB6rxcATEUdERJ/K697AeUDNrTKYma8BayPixErTOcALBZb0bi4D7i26CEmSal3N\nhJ6WNjutBYfKJqsRcS/wJHBiRKyNiC8UXVMLzgAmAp/cY8ndsUUXtY/3A49Vnul5BngoM39bcE1t\nUatTMY8FFuzx/Xw4M+cVXFNLrgbujoilwHDgxoLraVYlPJ4D/EfRtUiSVOtqZslqSZIkSeoMNTPS\nI0mSJEmdwdAjSZIkqdQMPZIkSZJKzdAjSZIkqdQMPZIkSZJKzdAjSZIkqdQMPZIkSZJKzdAjSZIk\nqdQMPZIkSZJKzdAjSZIkqdQMPZIkSZJKzdAjSZIkqdQMPZIkSZJKzdAjSZIkqdQMPZIkSZJKzdAj\nSZIkqdQMPZIkSZJKzdAjSZIkqdQMPZIkSZJKzdAjSZIkqdQMPZIkSZJKzdAjSZIkqdQMPZIkSZJK\nzdAjSZIkqdQMPZIkSZJKzdAjSZIkqdQMPZIkSZJKzdAjSZIkqdQMPZIkSZJKzdAjSZIkqdQMPZIk\nSZJKzdAjSZIkqdQMPZIkSZJKzdAjSZIkqdQMPZIkSZJKzdAjSZIkqdT+H7zGWMuBZCiNAAAAAElF\nTkSuQmCC\n",
       "text": [
        "<matplotlib.figure.Figure at 0x1074b22d0>"
       ]
      }
     ],
     "prompt_number": 41
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "We have re-expressed $\\vec{v_1}$ by two new orthogonal vectors $proj_1\\vec{v_1}$ plus $proj_2\\vec{v_1}$.  In symbols: $\\vec{v_1} = proj_1\\vec{v_1} + proj_2\\vec{v_1}$."
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "The sum of component 1 projections and the component 2 projections add up to the original vectors (points).\n",
      "\n",
      "Sure enough, if I sum up the data projected onto the first component and the data projected onto the second, I get back the original data:"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "np.allclose(projected_1 + projected_2, X)"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 42,
       "text": [
        "True"
       ]
      }
     ],
     "prompt_number": 42
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "Doing the sum above is the same operation as matrix multiplication of the transpose of the components with the projection coefficients (seeing that this is so involves writing out a few cells of the matrix multiplication in symbols and staring at it for a while):"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "data_again = components.T.dot(c_values)\n",
      "np.allclose(data_again, X)"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 43,
       "text": [
        "True"
       ]
      }
     ],
     "prompt_number": 43
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "## The components partition the sums of squares"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "Notice also that I have partititioned the sums of squares of the data into a part that can be explained by the first component, and a part that can be explained by the second:"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "# Total sum of squares\n",
      "np.sum(X ** 2)"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 44,
       "text": [
        "155.66928985835315"
       ]
      }
     ],
     "prompt_number": 44
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "# The data projected onto the first component\n",
      "proj_onto_first = line_projection(u_best, X)\n",
      "# The data projected onto the second component\n",
      "proj_onto_second = line_projection(u_best_orth, X)\n",
      "# Sum of squares in the projection onto the first\n",
      "ss_in_first = np.sum(proj_onto_first ** 2)\n",
      "# Sum of squares in the projection onto the second\n",
      "ss_in_second = np.sum(proj_onto_second ** 2)\n",
      "# They add up to the total sum of squares\n",
      "ss_in_first, ss_in_second, ss_in_first + ss_in_second"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 45,
       "text": [
        "(143.97317154347922, 11.696118314873956, 155.66928985835318)"
       ]
      }
     ],
     "prompt_number": 45
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "Why is this?"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "Consider the first vector in $\\mathbf{X}$ : $\\vec{v_1}$. We have re-expressed the length of $\\vec{v_1}$ with the squared length of $proj_1\\vec{v_1}$ plus the squared length of $proj_2\\vec{v_1}$.  The length of $\\vec{v_1}$ is unchanged, but we now have two new orthogonal vectors making up the sides of the right angled triangle of which $\\vec{v_1}$ is the hypotenuse.  The total sum of squares in the data is given by:\n",
      "\n",
      "$$\n",
      "\\sum_j x^2 + \\sum_j y^2 = \\\\\n",
      "\\sum_j \\left( x^2 + y^2 \\right) = \\\\\n",
      "\\sum_j \\|\\vec{v_1}\\|^2 = \\\\\n",
      "\\sum_j \\left( \\|proj_1\\vec{v_1}\\|^2 + \\|proj_2\\vec{v_1}\\|^2 \\right) = \\\\\n",
      "\\sum_j \\|proj_1\\vec{v_1}\\|^2 + \\sum_j \\|proj_2\\vec{v_1}\\|^2 \\\\\n",
      "$$\n",
      "\n",
      "where $j$ indexes samples - $j \\in 1..50$ in our case.\n",
      "\n",
      "The first line shows the partition of the sum of squares into standard x and y coordinates, and the last line shows the partition into the first and second principal components."
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "## Finding the principal components with SVD"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "You now know what a principal component analysis is.\n",
      "\n",
      "It turns out there is a much quicker way to find the components than the slow and dumb search that I did above.\n",
      "\n",
      "For reasons that we don't have space to go into, we can get the components using Singular Value Decomposition (SVD).\n",
      "\n",
      "In fact we need to do SVD on the variance / covariance matrix of the variables.\n",
      "\n",
      "See http://arxiv.org/abs/1404.1100 for a detailed explanation."
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "# Finding principal components using SVD\n",
      "unscaled_cov = X.dot(X.T)\n",
      "U, S, V = npl.svd(unscaled_cov)"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [],
     "prompt_number": 46
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "As a side note, we have done the SVD on the *unscaled* variance / covariance matrix.  *Unscaled* means that the values in the matrix have not been divided by $N$, or $N-1$, where $N$ is the number of samples.  This matters little in our case, but see below for more detail. "
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "The components are in the rows of the returned matrix $\\mathbf{V}$:"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "V"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 47,
       "text": [
        "array([[-0.87829753, -0.47811447],\n",
        "       [-0.47811447,  0.87829753]])"
       ]
      }
     ],
     "prompt_number": 47
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "Remember that a vector $\\vec{r}$ defines the same line as the vector $-\\vec{r}$, so we do not care about a flip in the sign of the principal components:"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "u_best"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 48,
       "text": [
        "array([ 0.87824304,  0.47821456])"
       ]
      }
     ],
     "prompt_number": 48
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "u_best_orth"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 49,
       "text": [
        "array([-0.47821456,  0.87824304])"
       ]
      }
     ],
     "prompt_number": 49
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "The returned vector $\\vec{S}$ contains the explained sum of squares for each component:"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "S"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 50,
       "text": [
        "array([ 143.97317326,   11.6961166 ])"
       ]
      }
     ],
     "prompt_number": 50
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "## Sums of squares and variance from PCA"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "As we said above, we have done our SVD on the unscaled variance covariance matrix.\n",
      "\n",
      "The standard *variance* of a vector $\\vec{x}$ with $N$ elements $x_1, x_2, ... x_N$ indexed by $i$ is given by $\\frac{1}{N-1} \\sum_i \\left( x_i - \\bar{x} \\right)^2$.  $\\bar{x}$ is the mean of $\\vec{x}$: $\\bar{x} = \\frac{1}{N} \\sum_i x_i$.  If $\\vec{q}$ already has zero mean, then the variance of $\\vec{q}$ is also given by $\\frac{1}{N-1} \\vec{q} \\cdot \\vec{q}$.\n",
      "\n",
      "The $N-1$ divisor for the variance comes from [Bessel's correction](http://en.wikipedia.org/wiki/Bessel%27s_correction) for bias.  \n",
      "\n",
      "The covariance between two vectors $\\vec{x}, \\vec{y}$ is $\\frac{1}{N-1} \\sum_i \\left( x_i - \\bar{x} \\right) \\left( y_i - \\bar{y} \\right)$.  If vectors $\\vec{q}, \\vec{p}$ already both have zero mean, then the covariance is given by $\\frac{1}{N-1} \\vec{q} \\cdot \\vec{p}$.\n",
      "\n",
      "Our unscaled variance covariance has removed the mean and done the dot products above, but it has not applied the $\\frac{1}{N-1}$ scaling, to get the true variance / covariance.\n",
      "\n",
      "For example, the standard numpy covariance function `np.cov` completes the calculation of true covariance by dividing by $N-1$."
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "# Calculate unscaled variance covariance again\n",
      "unscaled_cov = X.dot(X.T)\n",
      "# When divided by N-1, same as result of 'np.cov'\n",
      "N = X.shape[1]\n",
      "np.allclose(unscaled_cov / (N - 1), np.cov(X))"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 51,
       "text": [
        "True"
       ]
      }
     ],
     "prompt_number": 51
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "We could have run our SVD on the true variance covariance matrix.  The result would give us exactly the same components.  This might make sense from the fact that the lengths of the components are always scaled to 1 (unit vectors):"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "scaled_U, scaled_S, scaled_V = npl.svd(np.cov(X))\n",
      "np.allclose(scaled_V, V)"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 52,
       "text": [
        "True"
       ]
      }
     ],
     "prompt_number": 52
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "The difference is only in the *singular values* in the vector `S`:"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "S"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 53,
       "text": [
        "array([ 143.97317326,   11.6961166 ])"
       ]
      }
     ],
     "prompt_number": 53
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "scaled_S"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 54,
       "text": [
        "array([ 2.93822803,  0.23869626])"
       ]
      }
     ],
     "prompt_number": 54
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "As you remember, the singular values from the unscaled covariance matrix were the sum of squares explained by each component.  The singular values from the true covariance matrix are the *variances* explained by each component.  The variances are just the sum of squares divided by the correction in the denominator, in our case, $N-1$:"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "S / (N - 1)"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 55,
       "text": [
        "array([ 2.93822803,  0.23869626])"
       ]
      }
     ],
     "prompt_number": 55
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "So far we have described the PCA as breaking up the sum of squares into parts explained by the components.  If we do the SVD on the true covariance matrix, then we can describe the PCA as breaking up the *variance* of the data (across samples) into parts explained by the components.  The only difference between these two is the scaling of the `S` vector."
     ]
    }
   ],
   "metadata": {}
  }
 ]
}