{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "# Latent Semantic Indexing\n", "\n", "Here, we apply the technique *Latent Semantic Indexing* to capture the similarity of words. We are given a list of words and their frequencies in 9 documents, found on [GitHub](https://github.com/ppham27/MLaPP-solutions/blob/master/chap12/lsiDocuments.pdf)." ] }, { "cell_type": "code", "execution_count": 1, "metadata": { "collapsed": false }, "outputs": [ { "data": { "text/html": [ "
\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
012345678
000204000000
100002000000
1913000300000
1977000000200
2001000000020
4002000000
5002000000
500200000000
\"02713231948563522
(423740584
)423640584
*200000002
,343027832674394938
-6012220515167
.384139412536303138
:004030332
;020030330
?047003004
[000002000
]000002000
\n", "
" ], "text/plain": [ " 0 1 2 3 4 5 6 7 8\n", "000 2 0 4 0 0 0 0 0 0\n", "100 0 0 2 0 0 0 0 0 0\n", "1913 0 0 0 3 0 0 0 0 0\n", "1977 0 0 0 0 0 0 2 0 0\n", "2001 0 0 0 0 0 0 0 2 0\n", "4 0 0 2 0 0 0 0 0 0\n", "5 0 0 2 0 0 0 0 0 0\n", "500 2 0 0 0 0 0 0 0 0\n", "\" 0 27 13 23 19 48 56 35 22\n", "( 4 2 3 7 4 0 5 8 4\n", ") 4 2 3 6 4 0 5 8 4\n", "* 2 0 0 0 0 0 0 0 2\n", ", 34 30 27 83 26 74 39 49 38\n", "- 6 0 12 2 20 5 15 16 7\n", ". 38 41 39 41 25 36 30 31 38\n", ": 0 0 4 0 3 0 3 3 2\n", "; 0 2 0 0 3 0 3 3 0\n", "? 0 4 7 0 0 3 0 0 4\n", "[ 0 0 0 0 0 2 0 0 0\n", "] 0 0 0 0 0 2 0 0 0" ] }, "execution_count": 1, "metadata": {}, "output_type": "execute_result" } ], "source": [ "%matplotlib inline\n", "\n", "import numpy as np\n", "import pandas as pd\n", "import matplotlib.pyplot as plt\n", "from sklearn import preprocessing\n", "\n", "plt.rcParams['font.size'] = 16\n", "\n", "words_list = list()\n", "with open('lsiWords.txt') as f:\n", " for line in f:\n", " words_list.append(line.strip())\n", "words = pd.Series(words_list, name=\"words\")\n", "word_frequencies = pd.read_csv('lsiMatrix.txt', sep=' ', index_col=False,\n", " header=None, names=words)\n", "word_frequencies.T.head(20)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Now as per part (a), we compute the SVD and use the first two singular values. Recall the model is that\n", "\n", "\\begin{equation}\n", "\\mathbf{x} \\sim \\mathcal{N}\\left(W\\mathbf{z},\\Psi\\right),\n", "\\end{equation}\n", "\n", "where $\\Psi$ is diagonal. If the SVD is $X = UDV^\\intercal,$ $W$ will be the first two columns of $V$." ] }, { "cell_type": "code", "execution_count": 2, "metadata": { "collapsed": false }, "outputs": [], "source": [ "X = word_frequencies.as_matrix().astype(np.float64)\n", "U, D, V = np.linalg.svd(X.T) # in matlab the matrix is read in as its transpose" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "In this way, we let $Z = UD$, so $X = ZV^\\intercal$. Now, let $\\tilde{Z}$ be the approximation from using 2 singular values, so $\\tilde{X} = \\tilde{Z}W^\\intercal$, so $\\tilde{Z} = \\tilde{U}\\tilde{D}$. For some reason, the textbook chooses not to scale by $\\tilde{D}$, so we just have $\\tilde{U}$. Recall that all the variables are messed up because we used the tranpose." ] }, { "cell_type": "code", "execution_count": 3, "metadata": { "collapsed": false }, "outputs": [ { "data": { "text/plain": [ "array([[-0.29579069, -0.51925267],\n", " [-0.3133317 , -0.33578154],\n", " [-0.27364718, -0.45865789],\n", " [-0.34873574, 0.47093077],\n", " [-0.22977276, 0.04358919],\n", " [-0.44143778, 0.28243982],\n", " [-0.3498604 , 0.22347862],\n", " [-0.35250449, 0.17379354],\n", " [-0.35143587, -0.1538357 ]])" ] }, "execution_count": 3, "metadata": {}, "output_type": "execute_result" } ], "source": [ "Z = V.T[:,:2]\n", "Z" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Now, let's plot these results." ] }, { "cell_type": "code", "execution_count": 4, "metadata": { "collapsed": false }, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAiMAAAIHCAYAAABNF1yRAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3XuYHGWZ8P/vPZwZhYCirkIGlJNGJXJaUYEJoICsoK4I\nuqDzqi/qukFR5CfIOoyKoICg8QCrwCwgsCuoqC+Ccggqrq4IEQwqoGQ4i0JA6RAkzPP7o2pCpzM9\np0x1TXd9P9fVV9JPV1U/fXdN9d313PV0pJSQJEkqS1fZHZAkSdVmMiJJkkplMiJJkkplMiJJkkpl\nMiJJkkplMiJJkkplMqJKiIieiBiOiHMmuPye+fKfKLpv7SQi1o2Iz0TEHRHx94h4KiJeXna/xjLa\nez/Z/UFSsUxGKqruYFx/Wx4Rd0bE2RHxwibr7RoRg/mHUS2//S4izoqIfxznOb+VP8+txbyqcaX8\nNtKfliYcEfGsiLgiIu7On/cvEXFtRPxXK54/78Nr8vfhqohYHBEfj4i1J7GJY4CPAUuAk4EB4IEC\nujoha7hPrbI/lKnJ3+NjEXFPRPwwf59eUHY/O4VfNmaeyRyE1JluAy7M/78R0Av8H+CNEfGPKaU7\nACKiCzgdmA88AVwNfAsYBrYBDgXeExHvSCl9o/FJIuLZwAH58ttFxCtTSj8v8oU1uBd4MfBoC59z\nFSmlh4D9IuIY4CTgjSmln7bq+SNiDvAB4NCU0t8jYgfgx8CuwEET3Mx+wN+A16WUhovp6cSs4T5V\n+v7QRP3f4/rA84DdgE8Cx0fEsSmlM8rqnFQUkxHdllL6ZH1DRJwLvAP4OFliAtmH53zgl8BbUkp3\nN6zzTLJvzLOaPM87yPa304CjgXcBLUtGUkoryA709aJVz9/gNcByWvj6c58G/m9K6e8AKaVfR8RZ\nwEciYv+U0g8msI1/AB4qOxHJTXmfarI/zASr/T0CRMT+wLnAaRHxt5TS2a3vWkcp629fzaSUvFXw\nBvSQfaP87iiP7ZI/dkt+fxtgBfAnYNNxtrtOk/ZbyL6Frp///xFggwn2dRPgKeAbDe375P18HFi/\n4bE/A78a5fWek9/vz+8/lf87XHd/NrBnfv8TwE7Aj4C/5v3+FtCzBrH/M7CwhPf8b8Bv6t8j4A35\n6/zcOOs2i9c1dcusTTaMcwuwDHgY+AGwxyjbq4/vq4Gr8tg+NInXM6F9qvG9b9bWsM4/AwvzbS4D\nbiRL5MZ6HVPeT8b6e6xb5jX5Mg82vs7JxL7u9V2dL7cMuB04E9i8bpl35s/3jrFe9xjv6cJ8n7uP\nbEgv8uX+D3Bz/rx/AN41xmue9vdhjH35KWB23XIHk505fJDsGHMP8P+AfVr9t1uFmzUjGs3It4aR\n8fS+vO3MlNLDY62YUnpytY1F7ALMAS5JKS0HzgeeSfbHPq6U0lKyg+y8hod683/XBV5V93xzgGcB\n146x2YXAINnrWgicUHd7pG65XckOSMvJDta/BN4I/Cgi1p1I/+tFxIvzvl0/2XWnwRDZmY116ruU\n/7vBOOteSxabkQN8f35/sG6Zb5N96AB8EbiE7H25JiLe2mS7rwGuIRv6OzPfxrjWdJ8aZ9unAN8k\nS0ovzvu1DnBWRJzWZLVp3U9Gk7IhvR+T7T97Nzw84dhHxBfIXt9LgP8GzgBuIIvdjo1PO4WuvhL4\nIdmH+JlkCc9HgZMi4iPAKWTx+TrZe/a1iNizcSMFvg/XMs7ffkR8APgvsmGyi4HPkyXM2wCvm1w4\nNCFlZ0Peyrkx9pmRc/LHvp7fv4bsW0PvFJ/rzHz9efn9F5CdaVk4iW2ckW9ju7q2nwI/AWrAp+ra\nP5Av+4ZRXm/9t+PVvt2N8thTZMNS9Y/9Z97+1inE4j35uvuX8J53A89qaDs270/fBLdxJ/DHUdr7\n8nhdDnTVtb84f38eBp7RJL7/UuQ+1eS9H/XMCLBv3n4pq55BWovsA/8pYKfp3k/G+ntsWG4g3+YJ\nU4k9cGC+7P/Wvx/5Y+sBs+ruvzN/rsmeGXkK2K+ufUOysyM14C5WPfuy42ivu+j3YbT+N6zzK+Bu\nYL1RHtukyL/Tqt48M6JtI6I/v50WEb8kO7g9BHwmX+Z5+b/3TnbjEbE+cAhwb0rpWoCU0r1k30h2\nj4gXTXBTC8m+yczLt7sB2XDSlcAvWPWsyTyyA82PJ9vfUVyXUrqkoe2cvC+7TGF7u5N92/zZRFeI\niPMj4meTuH1htO2klGopK6Kt906y2onVio4n6Z1kr+tjqa6eJKX0W7J4bUz2DbXRr9IoBc9jmcZ9\najQfINt33p/qzvKllJ4Cjid73w8ZZb3p3k+auS/f5rPr2iYT+/fny34wpfRY/YZTSk+klOrPCk7V\n1SmlK+q2u4xseGN9srOr99Q9diPZUM3LGrYxE96HvzPKmaGUnanVNLOAVduQjbMCPEl2sPs6cGJK\naWgatv8WsoPhmQ3t5wN7kY0fHz+B7VxHdmCYl29rd7L991qyA83xEbFhfuDbHbg5pTQdV0rcOErb\nyMG0WbHuWF4F3DqZvqWUDp/C84wrIt5J9o18rzTK8NokvRz4W0rp5lEeW0j24bIDcEHDYzdM4bmm\na58aza5kdQ7vj1itxnHkNP/2o6w33ftJM6MVXk4m9jsDy1JK/zONfWo0Wj/uH+OxB8jiXq/s9+G/\nyIr2fxMRF5PF8X9SSo9PYhuaBJMR/b+U0oHjLPMAsB3ZqfDbJ7n9d5ElEY3ffr8FfBl4Z0T8e8rP\nfzaTUloaEbfwdJ1IL1lR2S/I5ssZAF4TEfcCm7H6h95U/XWUthX5v2tNZkMR8VzgRaz+ITry+IbA\ns1NKd02qh1MQEVuQ1Ri8dZo+mDYC/tjksQfqlmn0pyk817TsU01sSva+Npt/IpENOzSatv1kHP+Q\n//vnurbJxH5jsjliijRaLJ4a47EVrP5ZVOr7kFL6XEQ8BLyP7KrC44EnIuIS4CMppQcnui1NjMM0\nmoifkX0j22syK0VED9nYLMDN9RM6kV0FsSHwfLLx4YlYCDw7L1DtJfumsoLscs7H87ZesgPVwsn0\ntUVek//brHj13Yx+gJ1W+RDXf5FdlfC9adrsX4HnNnnsuXXLNJpUwlDAPtXor8CfUkprjXHbZ4rb\nng69ZDGrP6M0mdg/wtMJzXiGyf7uR/vSuvEEtzFVpb8PKaWzU0q7AM8hOxt3JfAvZH87mmaeGdFE\nDJLNIXJERJyRxriiJiLWTfk8FmQfriMV63eMsvgmZJfuvQu4YpTHGy0km+vkDWSnm0+A7AqeiPgf\nsmTpbiZeLzLybW06v7mO5dVkHySrTXQW2bno3pTSglEeuxDYehLP84uU0vwxHj8TOC2l9P265z40\npXTRJJ6j0SKgNyJePspwQS/Z6160BtsfMd37VKP/BfaNiNmtOEM1GRGxO9kQ5INkReUjJhL7m/L7\nvySbeO9VKaXx6pZGaiNGm/m18aqb6Vb0+zDhv/28zupbwLci4kZgj4h4ZkrpbwX0q7JMRjSulNId\nEXEq2TwGl0fEWxsPEBHxDLKE5X7gy/kH3Dt4+mqJ+xu3m8/qejfwhojYdKwkJ3dd/u9RZAeRhXWP\nLSS73HQbJl4vMvJ8W0xg2enwGuC+JrU47yf7oFhNSunt09WB/NLKhSmlS+uat+bpIuWpOo+snuek\niHjDSCFlfinzu8m+kV+2Jk9Q0D7VaAHZLLNnR8RbGvej/MwM01RPNWER8XqyLwUJOK6hdmEisf9u\nvuxXgf2BL0TE3imllWerImI9YMO6As1f5c93aER8LqX0RL7cNsCRFDuVftHvw5h/+xGxR0rpxw1t\nG5CdEVrB08mMpklbJSMRsTnZJZ77kH07ugr4UGqYDXSM9V9MVlswj+wyx7uAL4/2bVSrOY6sGn4+\ncFtEXAXcSnYW4kXAa8nmDBgptnwt2fwAPxjtQwMgpTQcEeeTJTmHkc2P0FRd3cjLySZA+t+6hxeS\nTZk9i4kP0fyOrGD30Ij4O1mhWxqvH1MRERsBc8m+YdW3r0d2YD+RrC6nMBHxOrJE7tqIqB9yexHw\nqTXc/Hlkp7JfDyyKiMvJxv0PISs47Gu8emMKpn2fGmX9H0TESWSJ9e0RcSXZfrEZWcHkbsDbyOZs\nKcK2EdGf/389siTxVcC2ZHNnHJVSavxxvwnHPqX0/YhYAPwb2d/xd8g+mHvIhrbeRZ64pJTuj4iL\n8tf7q4i4gmzI4k1kE6q9pYDXT/7cRb8PTf/28zMel0XEI2Q1aUNkx779gS2BBXmhvKbTZK4DLvNG\nNinT7WTV2G/IbzfnbePO5El2Wv9R4Dtk19rvSTbnw4fKfm0lxbOHLLu/bJLr7Up2qdztwGNkScHv\ngf9g1ev+L8y3f/A429suX+6mCT7/6fnyVza0r5P3ZwV184uM8nrPbmjfheyU9yP54/UzsD4F/PtE\ntzXG6/sR2fwcT5F9a7+c7GD+M7IPgqeAn7XgPX+o7jXW31YAW01wG3cCf2jy2Fpkk1vVzwJ6OfCa\nUZZtGt8xnntK+9Ro79d47yHZB/P3yIZElufv27XAh6ibhXga95ORZetvj+XPeyXZh/Lzx1h/wrHP\nl38rT89s+hjZ3/CXgRc0LLde/jd3X77dm8iSnNVe9zix6M8fG2023muBFa1+H2jyt58/9l6yz4o/\n5q/7T2RDv28v+u+0qreR6XlnvIj4IHAqsG1K6c68bUuyD8WPpjF+PCo/vXsL8LuUUmHZvCRJmrx2\nSkauIpsNb/eG9oVASik1ThVev8xeZN9Od0/jF21JkqQWaqdLe+eQ/chXo8Vkv7Ewllfn/24YEf8T\nEX+PiD9FxBfy2RwlSVJJ2ikZ2ZSnLzWr9zDZ5XxjeT5ZwevFZJf77QN8lqxmZE2nwZYkSWugra6m\nYfRLyUabHrlRV77u+SmlgbztxxGxNtnlcNunlH63ykYj2mP8SpKkaZJSmshn6rRrpzMjS8nOjjTa\nhNHPmNQb+XGwqxraf0iWzMwdbaWyq4s7/dbf3196H6pwM87GuBNuxrj4W5naKRlZTFY30uglZPNd\njLcurH5mZSQDHEYtt2TJkrK7UAnGuXjGuHjGuLO1UzLyXeCV+eW8wMpLe1/N+DM7/oDs56D3a2jf\nj9V/50GSJLVQOyUjXyP7tcnLIuLAiDiQbFKaIbIJtwCIiNkRsSIiVv6EeMqmhD4JeF9EnBgRe0fE\nx4B/BwZTSs1+8VIF6uvrK7sLlWCci2eMi2eMO1vbzDMCK6eDP51sWuiR6eCPSnW/k5L/ZsEfgRNS\nSp9qWP9DwL+SzbB5P9lvPXw6pbTa7wxERGqn2EiStCYigmQB6/hSSveklA5OKc1KKW2cUvrn1PCD\nbSmloZT9vPRqv7WRUjojpbRtSmn9lNJWKaWB0RIRtcbChQvL7kIlGOfiGePiGePO1lbJiCRJ6jxt\nNUzTSg7TSJKqxGEaSZJUWSYjKo1jwK1hnItnjItnjDubyYgkSSqVNSNNWDMiSaoSa0YkSVJlmYyo\nNI4Bt4ZxLp4xLp4x7mwmI5IkqVTWjDRhzYgkqUqsGZEkSZVlMqLSOAbcGsa5eMa4eMa4s5mMSJKk\nUlkz0oQ1I5KkKrFmRJIkVZbJiErjGHBrGOfiGePiGePOZjIiSZJKZc1IE9aMSJKqxJoRSZJUWSYj\nKo1jwK1hnItnjItnjDubyYgkSSqVNSNNWDMiSaoSa0YkSVJlmYyoNI4Bt4ZxLp4xLp4x7mwmI5Ik\nqVTWjDRhzYgkqUqsGZEkSZVlMqLSOAbcGsa5eMa4eMa4s5mMSJKkUlkz0oQ1I5KkKrFmRJIkVZbJ\niErjGHBrGOfiGePiGePOZjIiSZJKZc1IE9aMSJKqxJoRSZJUWSYjKo1jwK0x2Tj/8Ic/ZO+99+Yf\n/uEfWH/99dliiy045JBD+O1vf1tMBzuA+3LxjHFnW7vsDkiaWR5++GF23nlnPvCBD7DZZptx1113\ncdJJJ7Hbbrtxyy23sMUWW5TdRUkdxpqRJqwZkZ522223sf3223Paaadx1FFHld0dSQUos2bEMyNS\nhxsaGmJwcJDh4WG6urro6+ujp6dnUtvYdNNNAVhnnXWK6KKkirNmRKVxDLh4Q0NDHHPMMRx99NEM\nDAxw9NFHs2DBAoaGhsZdd3h4mCeffJLbb7+d9773vTz/+c/n0EMPbUGv24/7cvGMcWczGZE62ODg\nIH19fXR3dwPQ3d3NwMAAg4OD4677j//4j6y33npst912/OY3v+Hqq6/m2c9+dsE9llRFJiMqTW9v\nb9ld6HjDw8Psv//+q7R1d3czPDw87roXXHABv/jFL7jooovYaKON2GeffbjrrruK6mpbc18unjHu\nbCYjUgfr6uqiVqut0lar1ejqGv9Pf7vttmOXXXbhkEMO4aqrruKxxx7j5JNPLqqrkirMZESlcQy4\neH19fbzrXe9amZDUajX6+/vp6+ub1HY23nhjtt56a+64444Cetn+3JeLZ4w7m1fTSB2sp6eHN73p\nTZx66qkrr6aZP3/+pK+m+dOf/sTvfvc7Dj/88IJ6KqnKnGekCecZUVW9+c1vZscdd+TlL385G220\nEb///e8544wzePDBB/nFL37B1ltvXXYXJRWgzHlGTEaaMBlRVZ1yyin893//N3/4wx/4+9//zhZb\nbMG8efP42Mc+xuzZs8vunqSCmIzMQCYjxVu4cKEV8i1gnItnjItnjIvnr/ZKkqTK8sxIE54ZkSRV\niWdGJElSZZmMqDTOG9Aaxrl4xrh4xrizmYxIkqRSWTPShDUjkqQqsWZEkiRVlsmISuMYcGsY5+IZ\n4+IZ485mMiJJkkplzUgT1oxIkqrEmhFJklRZJiMqjWPArWGci2eMi2eMO5vJiCRJKlVb1YxExObA\nGcA+QABXAR9KKd09ye0cC5wI/DSltEeTZawZkSRVRpk1I22TjETEBsDNwOPAx/PmE4ENgJenlB6f\n4HZeCPwaeAy43WREkiQLWCfqCGBL4KCU0vdSSt8DDszb3juJ7XwFuAD43XR3UJPjGHBrGOfiGePi\nGePO1k7JyBuAn6eU7hxpSCktAa4HDprIBiLi7cArgGOL6KAkSZq8dhqmuR/4Tkrp/Q3tXwbeklJ6\n7jjrzyI7G3JMSum8iLgWWMthGkmSHKaZqE2BpaO0PwxsMoH1TwV+n1I6b1p7JUmS1kg7JSMAo52q\nGDeLi4jdgcOA9017jzRljgG3hnEunjEunjHubGuX3YFJWEp2dqTRJox+xqTemcDZwH0RsTFZArM2\n0JXffzyl9PfGlfr6+thyyy0BmDVrFnPnzqW3txd4+g/D+1O/v2jRohnVH+97f6r3Fy1aNKP604n3\nPV5M//2R/y9ZsoSytVPNyNXAOo01HnntBymleWOsO0x2VmW0sygJOCql9MWGdawZkSRVRpk1I+10\nZuS7wCkRsWV+FQ0RsSXwauCYcdbtHaXtC2TDVP8G/GG6OilJkiannWpGvgYsAS6LiAMj4kDgO8AQ\n8B8jC0XE7IhYERHHj7SllH7ceAMeAR5NKf0kpXRfa1+KwDHgVjHOxTPGxTPGna1tkpGU0jJgL+A2\n4DzgfLIzGnvnj42Iutu4m53ufkqSpMlpm5qRVrNmRJJUJc4zIkmSKstkRKVxDLg1jHPxjHHxjHFn\nMxmRJEmlsmakCWtGJElVYs2IJEmqLJMRlcYx4NYwzsUzxsUzxp3NZESSJJXKmpEmrBmRJFWJNSOS\nJKmyTEZUGseAW8M4F88YF88YdzaTEUmSVCprRpqwZkSSVCXWjEiSpMoyGVFpHANuDeNcPGNcPGPc\n2UxGJElSqawZacKaEUlSlVgzIkmSKstkRKVxDLg1jHPxjHHxjHFnMxmRJEmlsmakCWtGJElVYs2I\nJEmqLJMRlcYx4NYwzsUzxsUzxp3NZESSJJXKmpEmrBmRJFWJNSOSJKmyTEZUGseAW8M4F88YF88Y\ndzaTEUmSVCprRpqwZkSSVCXWjEiSpMoyGVFpHANuDeNcPGNcPGPc2UxGJElSqawZaaIVNSPXXXcd\n8+bNW6191qxZPPzww4U+tyRJ9cqsGVm7jCfV0yKCBQsWsPPOO69sW3tt3xZJUnU4TDMDbL/99uy6\n664rbzvuuGPZXWoJx4BbwzgXzxgXzxh3NpORkjlMJkmqOmtGmphKzcjQ0BCDg4MMDw/T1dVFX18f\nPT09TZcfqRl57nOfy5///GdmzZrFvvvuy8knn8wWW2yxpi9BkqQJK7NmxGSkickmI0NDQyxYsICB\ngQG6u7up1Wr09/czf/78pgnJokWLuPDCC9lzzz3ZaKONuOmmmzjxxBNZd911uemmm3j2s589XS9H\nkqQxOelZBxgcHFyZiAB0d3czMDDA4OBg03Xmzp3L5z73OQ444AB23313jjzySK644goeeOABvvjF\nL7ao5+VxDLg1jHPxjHHxjHFnMxmZJsPDwysTkRHd3d0MDw9PajuveMUr2HbbbfnlL385nd2TJGnG\nMhmZJl1dXdRqtVXaarUaXV2TD3FKiYhSzpS1VG9vb9ldqATjXDxjXDxj3NlMRqZJX18f/f39KxOS\nkZqRvr6+SW3nhhtu4LbbbuOVr3xlAb2UJGnmsYC1iVZcTXPYYYex1VZbseOOOzJr1ixuvPFGTj75\nZJ7xjGfwq1/9ik033XRNX8aMtnDhQr/ttIBxLp4xLp4xLp4zsHaInp4e+vv7J7z8S1/6Ui6++GK+\n9KUvsWzZMp73vOfxlre8hRNOOKHjExFJkkZ4ZqSJVvw2jSRJM4WX9kqSpMoyGVFpnDegNYxz8Yxx\n8YxxZzMZkSRJpbJmpAlrRiRJVWLNiCRJqiyTEZXGMeDWMM7FM8bFM8adzWREkiSVypqRJqwZkSRV\niTUjkiSpskxGVBrHgFtjsnG+/PLL2XPPPXnmM5/JxhtvzK677up7NQ7jUzxj3NlMRiStdNZZZ/HG\nN76RXXbZhe985ztccsklHHzwwSxbtqzsrknqYNaMNGHNiKpmaGiIF7/4xXz2s59l/vz5ZXdHUov5\nq72Spt3Q0BCDg4MMDw/T1dVFX18fPT09TZc/++yzWWuttXjve9/bwl5KksM0KpFjwMUZGhpiwYIF\nHH300cybN4+jjz6aBQsWMDQ01HSd66+/nu23356LLrqIrbfemnXWWYdtttmGr3zlKy3seXtyXy6e\nMe5sJiNSBxocHGRgYIDu7m4Auru7GRgYYHBwsOk69913H7fddhvHHHMMxx13HD/60Y943etex7/9\n27+xYMGCFvVcUhU5TKPS9Pb2lt2FjjU8PLwyERmJc3d3N8PDw2Ou89hjj3Heeedx0EEHrVz3zjvv\n5KSTTrKOZAzuy8Uzxp3NMyNSB+rq6qJWq63SVqvV6Opq/if/rGc9C4B99tlnlfbXve51/OlPf+KB\nBx6Y/o5KEm2WjETE5hFxSUQ8EhGPRsSlEbHFBNbbKSLOiojfRkQtIoYi4oKI2LL4XqsZx4CL09fX\nR39/P7VajYULF1Kr1ejv76evr6/pOnPmzBm1feSqsrESmapzXy6eMe5sbXN0iYgNgGuBbYHDgcOA\nbYBr8sfGcijwEuALwH7A/wfsCNwQES8orNNSSXp6epg/fz6nnnoq5557Lqeeeirz588f82qaN73p\nTQBceeWVq7RfccUVbL755jznOc8ptM+Sqqtt5hmJiA8CpwLbppTuzNu2BG4HPppSOmOMdZ+VUnqo\noW02cCfwqZTSCaOs4zwjqpy9996bm2++mU9/+tO88IUv5Jvf/CZnn302g4ODHH744WV3T1KBypxn\npJ2SkauA9VJKuze0LwRSSmneFLb5APC9lNL/HeUxkxFVzmOPPcaxxx7LJZdcwtKlS9l+++059thj\nOeSQQ8rumqSC+UN5EzMH+M0o7YvJhmAmJSJeDDwHuHUN+6Upcgy4NSYT52c84xksWLCA+++/n+XL\nl7No0SITkQlwXy6eMe5s7ZSMbAosHaX9YWCTyWwoItYCzgQeBM5Z865JkqSpaqdhmieAU1NKH29o\n/zRwTEpp3Uls60zg/wCvTyld3WQZh2kkSZXhb9NMzFKysyONNmH0MyajioiTgPcA72iWiIzo6+tj\nyy23BGDWrFnMnTt35cQ7I6cMve9973vf+95vx/sj/1+yZAlla6czI1cD66SU9mhovxZgIgWsEfFx\n4JPA/JTSmD+44ZmR4i1cuHDlH4eKY5yLZ4yLZ4yLZwHrxHwXeGX9RGX5/18NXDbeyhFxJPAp4Ljx\nEhFJktQ67XRmZENgEfA48O958yeBbmCHlNKyfLnZwB+BE1JKn87bDgW+AVyRr1Pvryml347yfJ4Z\nUceaN28e11133aiP7bffflx++eUt7pGkslkzMgEppWURsRdwOnAeEMBVwFEjiUgu6m4j9s3/3S+/\n1bsO2KuQTksz1Fe/+lX++te/rtL2s5/9jI985CMrfyRPklqlbc6MtJpnRornGHBrTDTO7373u7nw\nwgu5//77mTVrVvEd6yDuy8UzxsWzZkRSqZYvX84ll1zCgQceaCIiqeU8M9KEZ0bUToaGhhgcHGR4\neJiuri76+vrG/FG8Rt/4xjd4xzvewfe+9z1e//rXF9hTSTOVv00zA5mMqF0MDQ2xYMECBgYG6O7u\nplar0d/fP+6v9Nbbd999ufnmm7n33nvp6vKEqVRFDtOokuon3tHUDQ4OrkxEALq7uxkYGGBwcBAY\nP873338/V199NYcddpiJyBS5LxfPGHc2jzxSmxseHl6ZiIzo7u5meHh4Quuff/75pJR4xzveUUT3\nJGlcJiMqjZXx06Orq4tarbZKW61WW3mWY7w4n3/++eywww687GUvK6qLHc99uXjGuLOZjEhtrq+v\nj/7+/pUJyUjNSF9f37jr/upXv2Lx4sUTWlaSimIBaxMWsBbPeQOmz1hX04wV5yOPPJKzzjqLe+65\nh80226yFPe4s7svFM8bFcwZWSWukp6eH/v7+Sa2zYsUKLr74Yvbff38TEUml8sxIE54ZkSRViZf2\nSpKkyjIZUWmcN6A1jHPxjHHxjHFnMxmRJEmlsmakCWtGJElVYs2IJEmqLJMRlcYx4NYwzsUzxsUz\nxp3NZETvLWhIAAAgAElEQVSSJJXKmpEmrBmRJFWJNSOSJKmyTEZUGseAW8M4F88YF88YdzaTEUmS\nVCprRpqwZkSSVCXWjEiSpMoyGVFpHANuDeNcPGNcPGPc2UxGJElSqawZacKaEUlSlVgzIkmSKstk\nRKVxDLg1jHPxjHHxjHFnMxmRJEmlsmakCWtGJElVYs2IJEmqLJMRlcYx4NYwzsUzxsUzxp3NZESS\nJJXKmpEmrBmRJFWJNSOSJKmyTEZUGseAW8M4F88YF88YdzaTEUmSVCprRpqwZkSSVCXWjEiSpMoy\nGVFpHANuDeNcPGNcPGPc2UxGJElSqawZacKaEUlSlVgzIkmSKstkRKVxDLg1jHPxjHHxjHFnMxmR\nJEmlsmakCWtGJElVYs2IJEkVsd9++9HV1cUnPvGJsrsyY5iMqDSOAbeGcS6eMS5ep8T4oosu4uab\nbyailBMQM5bJiCRJLfDII4/w4Q9/mNNPPx3LAFZlzUgT1oxIkhoNDQ0xODjI8PAwXV1d9PX10dPT\nM6F1jzjiCJYsWcIPf/hDurq6OP744/nkJz9ZcI8nrsyakbXLeFJJktrN0NAQCxYsYGBggO7ubmq1\nGv39/cyfP3/chOSnP/0pF1xwATfffHOLetteHKZRaTplDHimM87FM8bFmwkxHhwcXJmIAHR3dzMw\nMMDg4OCY661YsYL3ve99fPSjH2XrrbduQU/bj8mIJEkTMDw8vDIRGdHd3c3w8PCY65188sksX76c\n4447rsjutTWHaVSa3t7esrtQCca5eMa4eDMhxl1dXdRqtVUSklqtRldX8+/1d999N5/5zGc4++yz\nWb58OcuXL19ZvPrEE0/w6KOP8sxnPnPMbVSBBaxNWMAqSao3lZqR6667jr322gtglSto8mJRIoKb\nbrqJl7/85S15DWMps4DVZKQJk5HiLVy4cEZ82+l0xrl4xrh4MyXGk72a5q9//SuLFi1arb23t5fD\nDz+c97znPey0005suOGGRXZ7QryaRpKkNtDT00N/f/+El99oo43YY489mm5r9913n66utbVqD1Kp\nVDPhW04VGOfiGePidVqMI8JZWOt4ZkSSpBZ76qmnyu7CjOKZEZVmJswbUAXGuXjGuHjGuLOZjEiS\npFJ5NU0TXk0jSaqSMq+maaszIxGxeURcEhGPRMSjEXFpRGwxwXXXi4hTIuK+iFgWET+LCMuYJUkq\nWdskIxGxAXAtsC1wOHAYsA1wTf7YeM4B3g0cDxwA3A9cGRHlzzRTUY4Bt4ZxLp4xLp4x7mztdDXN\nEcCWwLYppTsBIuIW4HbgvcAZzVaMiB2AtwF9KaXz8rYfA4uBTwJvLLTnkiSpqQnXjOQf6OcBPcAV\nwJEppQcj4u3A4Sml/YvrJkTEVcB6KaXdG9oXAimlNG+Mdf8d+DgwK6W0vK79BOD/AzZKKT3ZsI41\nI5KkymiXmpETgH7g1cDVwAUR8byU0oXATgX0rdEc4DejtC8GXjLOui8B7qxPROrWXRfwN50lSSrJ\nZJKR76eUvpNSWpxS+hrwFuBDEfHcgvrWaFNg6SjtDwObrMG6I4+rxRwDbg3jXDxjXDxj3Nkmk4yk\niHhpRCyIiI1TSn8FjgUOBCZSQDodRhs3mcgppViDdSVJUoEmVMAaEbullM6JiP3ICkYfg6xQA/ha\nRPy5wD6OWMroZzA2YfSzHvUeBka7BHiTusdX09fXx5ZbbgnArFmzmDt37srfRxjJ0r2/ZvdHzJT+\ndOL93t7eGdWfTrw/0jZT+tOp90fMlP60+/2R/y9ZsoSyTaiANSJ2AbZPKZ2f3/8nYGfg0ymlFcV2\ncWUfrgbWSSnt0dB+LYAFrJIkTV07FLAuBuZGxJ4AKaXvA3cCXy+qY6P4LvDKiNhypCH//6uByyaw\n7rrAwXXrrgW8FbiyMRFRazR+21ExjHPxjHHxjHFnm2gy8m1gIXB7RLwib7sBeHMRnWria8AS4LKI\nODAiDgS+AwwB/zGyUETMjogVEXH8SFtK6dfAfwFnRMS7I2Kv/P6WZFcISZKkkkx0mOYu4KCU0k0R\nsR3wFPD6fP0vFNzH+n5sDpwOvJas+PQq4KiU0l11y/QAfwROSCl9qq59PeBE4O3ALODXwDEppZ80\neS6HaSRJlVHmMM1Ek5FXkk2/fmRKaTgidgLellI6uugOlsVkRJJUJTO+ZiSl9POU0r+llIbz+78C\nFkZEK4dp1GEcA24N41w8Y1w8Y9zZpvzbNCml70fEVtPZGUmSVD0T/m2aqnGYRpJUJTN+mEaSJKko\nJiMqjWPArWGci2eMi2eMO5vJiCRJKpU1I01YMyJJqhJrRiRJUmWZjKg0jgG3hnEunjEunjHubCYj\nkiSpVNaMNGHNiCSpSqwZkSRJlWUyotI4Btwaxrl4xrh4xrizmYxIkqRSWTPShDUjkqQqsWZEkiRV\nlsmISuMYcGsY5+IZ4+IZ485mMiJJkkplzUgT1oxIkqrEmhFJklRZJiMqjWPArWGci2eMi2eMO5vJ\niCRJKpU1I01YMyJJqhJrRiRJUmWZjKg0jgG3hnEunjEunjHubCYjkiSpVNaMNGHNiCSpSqwZkSRJ\nlWUyotI4Btwaxrl4xrh4xrizmYxIkqRSWTPShDUjkqQqsWZEkiRVlsmISuMYcGsY5+IZ4+IZ485m\nMiJJkkplzUgT1oxIkqrEmhFJklRZJiMqjWPArWGci2eMi2eMO5vJiCRJKpU1I01YMyJJqhJrRiRJ\nUmWZjKg0jgG3hnEunjEunjHubCYjkiSpVNaMNGHNiCSpSqwZkSRJlWUyotI4Btwa48X5+uuvZ999\n9+W5z30uG2+8MTvttBPnnntuazrXIdyXi2eMO5vJiFRht9xyC6997WtZsWIFX//61/nWt77Frrvu\nyrvf/W7OOuussrsnqSKsGWnCmhFVwXHHHcfnP/95li5dygYbbLCyfbfddqOrq4vrr7++xN5JaiVr\nRiSV4sknn2SdddZh/fXXX6V91qxZDA8Pl9QrSVVjMqLSOAY8/YaGhhgYGKC/v5+BgQGGhobGjHNf\nXx8ARx55JPfffz+PPvooX/va17jmmmv48Ic/3JpOdwD35eIZ4862dtkdkDQ9hoaGWLBgAQMDA3R3\nd1Or1ejv72fnnXduus6cOXO49tpredOb3sSXv/xlANZdd13OPPNMDj744FZ1XVLFWTPShDUjajcD\nAwMcffTRdHd3r2yr1Wqceuqp9Pf3j7rOHXfcwd57782cOXOYP38+66+/Ppdddhlf+cpX+M///E/e\n9ra3tar7kkpWZs2IZ0akDjE8PLxKIgLQ3d09Zu3Hsccey7rrrsv3vvc91lprLQDmzZvHX/7yFz74\nwQ+ajEhqCWtGVBrHgKdXV1cXtVptlbZarcbQ0FDTdX7zm9+www47rExERuy666489NBDPPjgg4X0\ntdO4LxfPGHc2kxGpQ/T19dHf378yIRmpGdlvv/2arvO85z2PRYsWsWLFilXaf/7zn7P++uuz6aab\nFtpnSQJrRpqyZkTtaGhoiMHBQYaHh+nq6qKvr4+enp6my1966aW89a1v5bWvfS3/+q//ygYbbMBl\nl13GV7/6VT784Q9zyimntLD3kspUZs2IyUgTJiOqiiuvvJLPfvazLF68mOXLl/OiF72I9773vRxx\nxBFElHJcklQCk5EZyGSkeAsXLqS3t7fsbnQ841w8Y1w8Y1w8Z2CVJEmV5ZmRJjwzIkmqEs+MSJKk\nyjIZUWmcN6A1jHPxjHHxjHFna5tkJDLHRsSdEfF4RCyKiDdPYL1nRsQnIuL6iPhLRCzN/39QK/ot\nSZLG1jY1IxFxIvBh4DjgRuBQ4AjggJTSFWOsNwf4EXAO8BNgGHgb0Ad8IKX01SbrWTMiSaoML+0d\nR0RsBtwNfCal9Mm69quAZ6eU5o6x7gZASiktb2i/Ctg6pbRlk/VMRiRJlWEB6/j2A9YBvtHQfgHw\nsohoOsVkSunxxkQkdwPw/OnroibLMeDWMM7FM8bFM8adrV2SkZcAT6SU/tDQvhiI/PHJ2hP43Zp2\nTJIkrZl2GaY5C3hDSun5De0vAm4HDk8pNZ41GWt7RwBfBf4lpXRxk2UcppEkVUblhmkiYu+IGJ7A\n7ZqRVYDRMoNJBy0ieoEvAOc1S0QkSVLrrF3S814PbD+B5Zbl/z4MbDLK45vUPT6uiNgFuAy4CnjP\neMv39fWx5ZZbAjBr1izmzp278rcRRsYvvT/1+4sWLeJDH/rQjOlPp96vH2ufCf3pxPtnnHGGx4eC\n73u8mP77I/9fsmQJZWuXYZrDgUFgm5TSH+va+4CzgRemlIbG2cbLgIXAzcB+KaUnxlneYZqCLfSH\nr1rCOBfPGBfPGBfPS3vHUXdp74kppU/VtV8FbJZS2mGc9bcBfgzcBeydUnpsAs9pMiJJqowyk5Gy\nhmkmJaX054g4HTg2Ih7j6UnPeoED65eNiKuB2SmlbfL7m5FNerYOcAIwJ2KVWN+YUnqy6NcgSZJG\n1y6X9kI28+qngSOBK4DdgINTSpc3LNfFqq/rJcAWZPUl3wd+1nD7h2K7rWbqxy1VHONcPGNcPGPc\n2drizAhkU6gCn8lvYy03r+H+dcBaBXZNkiStgbaoGSmDNSOSpCqp3DwjkiRJI0xGVBrHgFvDOBfP\nGBfPGHc2kxFJklQqa0aasGZEklQl1oxIkqTKMhlRaRwDbg3jXDxjXDxj3NlMRiRJUqmsGWnCmhFJ\nUpVYMyJJkirLZESlcQy4NYxz8Yxx8YxxZzMZkSRJpbJmpAlrRiRJVWLNiCRJqiyTEZXGMeDWMM7F\nM8bFM8adzWREkiSVypqRJqwZkSRViTUjkiSpskxGVBrHgFvDOBfPGBfPGHc2kxFJklQqa0aasGZE\nklQl1oxIkqTKMhlRaRwDbg3jXDxjXDxj3NlMRiRJUqmsGWnCmhFJUpVYMyJJkirLZESlcQy4NYxz\n8Yxx8YxxZzMZkSRJpbJmpAlrRiRJVWLNiCRJqiyTEZXGMeDWMM7FM8bFM8adzWREkiSVypqRJqwZ\nkSRViTUjkiSpskxGVBrHgFvDOBfPGBfPGHc2kxFJklQqa0aasGZEklQl1oxIkqTKMhlRaRwDbg3j\nPDn33nsv8+fP51WvehXd3d10dXVx1113jbmOMS6eMe5sJiOSVOeOO+7gkksuYdNNN2WPPfYgopSz\n1lKlWDPShDUjks4++2yOOOII7rzzTmbPnl12d6RCWTMiSZIqy2REpXEMuDWqHOehoSEGBgbo7+9n\nYGCAoaGhQp6nyjFuFWPc2dYuuwOSVIShoSEWLFjAwMAA3d3d1Go1+vv7mT9/Pj09PWV3T1Idz4yo\nNL29vWV3oRKqGufBwcGViQhAd3c3AwMDDA4OTvtzVTXGrWSMO5vJiKSONDw8vDIRGdHd3c3w8HBJ\nPZLUjMmISuMYcGtUNc5dXV3UarVV2mq1Gl1d03/Yq2qMW8kYdzaTEUkdqa+vj/7+/pUJyUjNSF9f\nX7kdk7Qa5xlpwnlGpPY3NDTE4OAgw8PDdHV10dfXN6Hi1UsvvRSAq666irPOOouvfOUrbLbZZmy2\n2WbsscceRXdbKkWZ84yYjDRhMiJVV1dX16gzr+65555cc801JfRIKp6TnqmSHANuDeM8ecPDwzz1\n1FOr3ZolIsa4eMa4s5mMSJKkUjlM04TDNJKkKnGYRpIkVZbJiErjGHBrGOfiGePiGePOZjIiSZJK\nZc1IE9aMSJKqxJoRSZJUWSYjKo1jwK1hnItnjItnjDubyYgkSSqVNSNNWDMiSaoSa0YkSVJltU0y\nEpljI+LOiHg8IhZFxJunsJ2tImJZRAxHxAuL6KsmxjHg1jDOxTPGxTPGna1tkhHg08AngC8C+wH/\nA3wzIvab5Ha+CiwFHIORJGkGaIuakYjYDLgb+ExK6ZN17VcBz04pzZ3gdt4OnAacBJwObJNS+mOT\nZa0ZkSRVhjUj49sPWAf4RkP7BcDLIqJnvA1ExCyyROQjwKPT3kNJkjQl7ZKMvAR4IqX0h4b2xUDk\nj4/nFODWlNKF0905TY1jwK1hnItnjItnjDvb2mV3YII2BR4Zpf3husebiojXAIcBExrOkSRJrVNK\nMhIRewM/msCiC1NKe5Gd/RitgGPcsa2IWAc4E/h8Sun3k+lnX18fW265JQCzZs1i7ty59Pb2Zh3L\ns3Tvr9n9ETOlP514v7e3d0b1pxPvj7TNlP506v0RM6U/7X5/5P9LliyhbKUUsEbE+sDsCSy6LKV0\nT0ScDByZUtqwYTu7AL8ADkgp/aDJcx0DfBDYCXg8b/4XYEHedkdK6bFR1rOAVZJUGZUrYE0pLU8p\n3TaB2z35KouB9UaZF2QO2RmTW8d4uhcDzwPuI7ukdynwJbKzKjcCP57O16aJa/y2o2IY5+IZ4+IZ\n487WLjUjVwBPkp3R+FRd+2HAb1JKQ2OsexJwbkPb/sAx+fZum8Z+SpKkSWqLeUYAIuIksuGWj5Od\n0TgU+L/AgSmly+uWuxqYnVLaZoxtvRM4B+cZkSQJKHeYpl3OjAAcB/wNOJJs2OX3wMH1iUiui/a5\nZFmSpMprmw/tlPlMSmmrlNIGKaW5KaVvj7LcvJTSi8bZ1n+mlNZqdlZEreEYcGsY5+IZ4+IZ487W\nNsmIJEnqTG1TM9Jq1oxIkqqkcpf2SpIkjTAZUWkcA24N41w8Y1w8Y9zZTEYkSVKprBlpwpoRSVKV\nWDMiSZIqy2REpXEMuDWMc/GMcfGMcWczGZEkSaWyZqQJa0Yk6WmXXnopF110ETfccAMPPvggs2fP\n5s1vfjPHHXccz3jGM8runqZBmTUjJiNNmIxI0tN22203enp6OOigg9h888256aab6O/v58UvfjE/\n+9nPyu6epoHJyAxkMlK8hQsX0tvbW3Y3Op5xLl4VYvzQQw/xrGc9a5W2888/n76+Pq6++urCX38V\nYlw2f7VXktRSQ0NDDA4OMjw8TFdXF319ffT09DRdvjERAdhll11IKXHvvfcW2VVVgGdGmvDMiKRO\nNTQ0xIIFCxgYGKC7u5tarUZ/fz/z588fMyFpdOaZZ/KBD3yAX/7yl+y4444F9lit4DDNDGQyIqlT\nDQwMcPTRR9Pd3b2yrVarceqpp9Lf3z+hbdx7773suOOOvOIVr+CKK64oqqtqISc9UyU5b0BrGOfi\ntVuMh4eHV0lEALq7uxkeHp7Q+rVajYMOOoh1112Xc845p4gurqbdYqzJMRmRpIrp6uqiVqut0lar\n1ejqGv8j4YknnuANb3gDS5Ys4corr+T5z39+Ud1UhThM04TDNJI61VRrRlasWMFBBx3ET37yE66+\n+mp22WWXFvZaRbNmZAYyGZHUySZ7NU1KiUMOOYTvf//7XH755V5m24FMRmYgk5HiOW9Aaxjn4lUh\nxu9///s566yzOP744znggANWeWzzzTfnBS94QaHPX4UYl80CVknSjHbFFVcQEZx44om86lWvWuV2\n9tlnl909tTnPjDThmRFJUpV4ZkSSJFWWyYhK47wBrWGci2eMi2eMO5vJiCRJKpU1I01YMyJJqhJr\nRiRJUmWZjKg0jgG3hnEunjEunjHubCYjkiSpVNaMNGHNiCSpSqwZkSRJlWUyotI4Btwaxrl40xXj\ne+65h7e85S3MmjWLjTfemH/+53/m7rvvnpZttzv3485mMiJJM8Djjz/OvHnzuO222zj//PO54IIL\nuP3229lrr714/PHHy+6eVChrRpqwZkRSK33hC1/g6KOP5rbbbmOrrbYCYMmSJWyzzTaccsopfOhD\nHyq5h+p0ZdaMmIw0YTIiqZX22WcfnnjiCX7yk5+s0t7b20tEcO2115bUM1WFBayqJMeAW8M4F2+0\nGA8NDTEwMEB/fz8DAwMMDQ2NuY3Fixfz0pe+dLX2OXPmcOutt05XV9uW+3FnW7vsDkhSpxkaGmLB\nggUMDAzQ3d1NrVajv7+f+fPn09PTM+o6Dz/8MJtssslq7ZtuuilLly4tustSqTwzotL09vaW3YVK\nMM7Fa4zx4ODgykQEoLu7m4GBAQYHB8fcTsTqZ8gdLs64H3c2kxFJmmbDw8MrE5ER3d3dDA8PN11n\nk0024eGHH16tfenSpaOeMZE6icmISuMYcGsY5+I1xrirq4tarbZKW61Wo6ur+SF3zpw5LF68eLX2\nW2+9lZe85CXT0s925n7c2UxGJGma9fX10d/fvzIhGakZ6evra7rOgQceyM9//nOWLFmysm3JkiVc\nf/31HHTQQQX3WCqXl/Y24aW9ktbE0NAQg4ODDA8P09XVRV9fX9PiVYBly5Yxd+5cNthgAz71qU8B\n8IlPfIJarcavf/1rNtxww1Z1XRXlPCMzkMmIpFa75557OOqoo/jRj35ESol99tmH008/ndmzZ5fd\nNVWA84yokhwDbg3jXLzpivHmm2/ON7/5TR555BEeffRRLr30UhORnPtxZzMZkSRJpXKYpgmHaSRJ\nVeIwjSRJqiyTEZXGMeDWMM7FM8bFM8adzWREkiSVypqRJqwZkSRViTUjkiSpskxGVBrHgFvDOBfP\nGBfPGHc2kxFJklQqa0aasGZEklQl1oxIkqTKMhlRaRwDbg3jXDxjXDxj3NlMRiRJUqmsGWnCmhFJ\nUpVYMyJJkirLZESlcQy4NYxz8Yxx8YxxZzMZkSRJpWqbmpGICOBjwBHA84DfA59MKX1rguuvn6//\ndmA28Ajwv8CbU0orRlnemhFJUmWUWTOydhlPOkWfBj4MHAfcCBwKfDMiDkgpXTHWihGxNnAF0AN8\nBvgtsBnwWmAtYLVkRJIktUZbDNNExGbAR4CTUkqnp5SuSym9H7gWOHkCmzgamAu8OqX0tZTST1NK\n304p/WtK6YkCu64xOAbcGsa5eMa4eMa4s7VFMgLsB6wDfKOh/QLgZRHRM8767wf+O6V0XxGd09Qs\nWrSo7C5UgnEunjEunjHubO2SjLwEeCKl9IeG9sVA5I+PKiK2ALYA7oyI/4iIRyPi8Yi4KiJ2KK7L\nGs8jjzxSdhcqwTgXzxgXzxh3tnZJRjYlKzht9HDd4808P//3Y8BWwFvJ6k02A66NiM2nq5OSJGny\nSklGImLviBiewO2akVWA0S5tmUjV78hrrAH/lFK6MqV0GXAAsCHwgTV+QZqSJUuWlN2FSjDOxTPG\nxTPGna2US3vzy2xnT2DRZSmleyLiZODIlNKGDdvZBfgFcEBK6QdNnmtb4HfApSmlgxseWwQ8kFLa\nb5T1vK5XklQplbq0N6W0HLhtEqssBtaLiBemlP5Y1z6H7IzJrWOs+0fgcZqfWRlu0sdS3hBJkqqm\nXWpGrgCeBP6lof0w4DcppaFmK+YTmv0/YPeI2GCkPSJmA9uRTXwmSZJK0haTnqWU/hwRpwPHRsRj\nPD3pWS9wYP2yEXE1MDultE1dcz/ZcM7lEXEasAHwCbIC2C8X/wokSVIzbZGM5I4D/gYcydPTwR+c\nUrq8YbkuGs74pJR+GxF7AZ8FLiY7y3IN8NGU0p+L7rgkSWquXYZpSJnPpJS2SiltkFKam1L69ijL\nzUspvWiU9htSSnsDzwQ+B+wILI6IRRHx5sn2JyK2iohl+VU/L2x4rL/J1UET+h2dThCZYyPiznxe\nl2mPc/74ayLi+nyZ+yPitLxAuuOtSYwj4qSI+HVELI2IWkT8NiKOrx/KzJc7d5T9+KmI+Hwxr2pm\naUWM82XfGBE35s+xJCI+HhFtc3xeE1ONcUQ8MyI+kf/9/yWP8/URcdAoy3pMbkGc8+WndkxOKVXq\nBpxIVtB6FLAn8FXgKWC/SW7nCuDefN0XNjzWn7e/Eti17rZ12a+/w+L8cmAZcCkwD3gX2dDbRWW/\n/pkeY+BLwAfJZjeeB3w839a3G5Y7F3gA2KVhX96i7NffQTHel+z3sb6aP8eH8uVOKvv1z+QYk13A\ncB/Z75btS/ZbY+eQXZTw/oZlPSa3Js5TPiaXHqAWvxmbAcuBTzS0XwUsmsR23g7cTzZkNFYy0lX2\na+7wOH+bbLhurbq2w/Nl55Ydh3aIccO6n8ljt2ld27nAXWW/3g6P8Y3ANQ3L/Xv+3M8pOw4zNcZk\ntX/rj9J+FbCkoc1jcmviPOVjciVOA9ZZ09+4ISJmAaeR/XDfo+MtPpVOdoDC4xzZLzHvC/xXSump\nuof+m6wmaNRTiB1kjWM8ipEZjZ9ck451kMJjHNkM0HPzbdY7H1gX2H8Kz9FOphzjlNLjKZsmotEN\nPD3zdiOPyauatjiv6TG5asnIlH/jps4pwK0ppQsnsOzdEbEiHwM+uSq1DLQmzi8C1s+3uVLKfoX5\nDxN8jnY2HTEmItaKiO6I2Ifs9O3ZKaW/NSz2nIj4c0Q8GRG/j4hjKlLP0IoYj8yV1LgfLyE73e1+\nPHl7kk10ORqPyauazjiv0TG5na6mmQ5r8hs3RMRryOY2mTvO89xB9ls4N5EdaF5HdhB6BVnm2Ola\nEeeRbSxt8jxjPkcHWKMYA0TEHOCWuqb/BN7bsNhNZN+AFpMdaN4EnARsDRwxuS63nVbEeKz9eOlE\nnqPNrXGM60XEEWS1II1zUnlMLj7Oa3RMbutkJCL2Bn40gUUXppT2Yg1+4yYi1gHOBD6fUvr9WMum\nlBpPhV0dEfcCp0fEXimla0Zbb6aaoXEe2dZUf7NoRmlljOvcAewMdAOvIrt8fh2yRBCAlNIXG9a5\nIiJqwAcj4rOjfNOasWZojN2Pp+m1R0Qv8AXgvJTSxfWPeUxuSZzXaF9u62QEuB7YfgLLLcv/fRjY\nZJTHN6l7vJmj8uUWRMTGeVt3/u9GEfGMlNJjY6x/EXAG2VUJbbXjMzPjPFZGvwnZFTjtpJUxBlae\nPr0xv/uTiHgAOCcivphSGmtm4ovIrvjYmez0a7uYKTE+ty7GY+3HsybyHDNMy2MMK3+n7DKyosr3\nTGQdPCbD9MZ5jY7JbZ2MpNb+xs2LySZbu2+Ux24EFpHNXTKetvsBvhka5z8AT+TbXCki1gNeSFY0\n1czX1dYAAANTSURBVDZaHONmbiD7BrM1Y/9MwljfgGasGRRjeDrGI2P2c8hmiQYgLyjccIrPUZoy\nYhwRLyObAuBG4C0NxZMT0Vb7MczYOK/RMbkKRWj1pvwbN2Tj5PPIpqAfuX2W7I18O+Nn44fly/5i\nnOU6QeFxTik9mT/PWxuKKQ8muwrhu2v4Gma6NYlxM71kcR7vbMe/kM0x8MspPEc7KTzGKaW7gV+P\n8hyHA38HRv018g6yRjGOiG2AH5INh70hPxM1UR6TpzHOa3xMLvv651bfyD7slrHqxC8rgNc3LHc1\ncPs423ono89/cSPZaez9yS6p+jzZgeX7Zb/+DovzDkCNbIKdvYB3Aw8BF5f9+mdyjIGXAVeSJXZ7\n5fvpyfm2vle33GzgOuD9ZBMd/RPZZEcrgC+V/fo7Icb5svvn2zwzf46jyCanOrns1z/DY7wZsAT4\nSx7Df2y4rVO3rMfk1sR5ysfk0gNUwhsSZEVkd+Z/8IuAN42y3LXAH8bZVrMPyQuB24HH8uf4Tf6c\n60zX65jpt1bEOX/sNWTjp8vIJkg7jVEm6OnE21RjDDyHbH6BP+QHjj+TfTt8X8OBZRPgW/n2l+XL\n3kDDrIudfCs6xnXLv5HsSo/H8wP/x4Eo+/XP8BjvmR8Xmt1m1y3rMbkFcc6Xn9IxOfKVJUmSSlG1\nmhFJkjTDmIxIkqRSmYxIkqRSmYxIkqRSmYxIkqRSmYxIkqRSmYxIkqRSmYxIkqRSmYxIkqRSmYxI\nkqRSmYxIKlVEvDIi7oqIKyNiw4jYJSJ2K7tfklrH36aRVKqI+BJwNrA50AssSimdnz92BPDblNJP\nyuuhpKKZjEgqVUR0pZSGI+L5wDYppesiYn3gPWS/2PyRlNKPy+2lpCKtXXYHJFVbnohsB2yYUrou\nb1sOfCkidiq3d5JawZoRSaXKE46nUko35ff/qeQuSWoxz4xIKk2eePQCd0XE1cDOwFbA98vsl6TW\n8syIpFJExFbAuimlo4EA/geYB3y61I5JajkLWCXNWBFxLnCuBaxSZ3OYRtKMFBHvB3bJ/htrpZSu\nLbtPkorhmRFJklQqa0YkSVKpTEYkSVKpTEYkSVKpTEYkSVKpTEYkSVKpTEYkSVKpTEYkSVKpTEYk\nSVKp/n895a+YMIX9pwAAAABJRU5ErkJggg==\n", "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "plt.figure(figsize=(8,8))\n", "def plot_latent_variables(Z, ax=None):\n", " if ax == None:\n", " ax = plt.gca()\n", " ax.plot(Z[:,0], Z[:,1], 'o', markerfacecolor='none')\n", " for i in range(len(Z)):\n", " ax.text(Z[i,0] + 0.005, Z[i,1], i, \n", " verticalalignment='center')\n", " ax.set_xlabel('$z_1$')\n", " ax.set_ylabel('$z_2$')\n", " ax.set_title('PCA with $L = 2$ for Alien Documents')\n", " ax.grid(True)\n", "plot_latent_variables(Z)\n", "plt.show()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "I, respectfully, disagree with the book for this reason. The optimal latent representation $Z = XW$ (observations are rows here), should be chosen such that\n", "\n", "\\begin{equation}\n", "J(W,Z) = \\frac{1}{N}\\left\\lVert X - ZW^\\intercal\\right\\rVert^2\n", "\\end{equation}\n", "\n", "is minimized, where $W$ is orthonormal." ] }, { "cell_type": "code", "execution_count": 5, "metadata": { "collapsed": false }, "outputs": [], "source": [ "U, D, V = np.linalg.svd(X)\n", "V = V.T # python implementation of SVD factors X = UDV (note that V is not tranposed)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "By section 12.2.3 of the book, $W$ is the first $2$ columns of $V$. Thus, our actual plot should be below." ] }, { "cell_type": "code", "execution_count": 6, "metadata": { "collapsed": false }, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAd8AAAIHCAYAAADehVaHAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzs3XmYHGW5/vHvM+wZkCQQRJZMkDUEJbIpYZuwBUSWo0FQ\nQEZQImiQJXrYmwFljYclsuTIMhoF1ICCHghbMsgPBEEIqwgImQCGsGSBTAIx6ef3R9UMPT3dk57J\nVHfX2/fnuuaadHV1zXt3dffTVU9VxdwdERERKZ+6Sg9ARESk1qj4ioiIlJmKr4iISJmp+IqIiJSZ\niq+IiEiZqfiKiIiUmYqvJM7MGswsa2Y3lTj/XvH85yU9tjQxs9XN7CIze9XMlprZcjP7fKXH1ZNC\n6763rweREKn4VkDOh0/uz0dm9rqZ3Whmny3yuF3MrCX+8G2Pf14ys8lm9sUV/M074r/zYjKpVsjj\nn47xlLXAmtl6ZjbNzN6I/+57ZjbDzH5bjr8fj2H3eD08YGYvmNnZZrZqLxbxY+AMYBZwCdAMvJ3A\nUEuykq+pLq+HSiryflxkZm+a2X3xetq40uMMhb5cR3rzxpf+9zJwS/zvTwGNwLeBw8zsi+7+KoCZ\n1QFXAOOBj4EHgTuALLAlcCTwHTP7lrv/Jv+PmNn6wEHx/Fub2Zfc/bEkg+V5CxgOLCzj3+zC3d8H\nDjCzHwMXA4e5+/8r1983sxHA94Ej3X2pmW0P/AXYBTi0xMUcAHwI7O/u2WRGWpqVfE1V/PVQRO77\ncU1gQ2BX4ALgHDM7092vrNTgJCwqvpX1srtfkDvBzG4GvgWcTVSIISoW44EngLHu/kbeY9Yh2iIa\nWOTvfItoXf8MmAAcB5St+Lr7MqIPtlxWrr+fZ3fgI8qYP/YT4LvuvhTA3Z8xs8nA6WZ2oLvfU8Iy\nPgO8X+nCG+vza6rI66EadHs/ApjZgcDNwM/M7EN3v7H8QwtKpd771cXd9VPmH6CBaIvhrgL37Rzf\n91x8e0tgGTAXGLyC5a5WZPpzRFsZa8b/XgCsVeJYBwHLgd/kTd83HucSYM28+94F/l4g703x7Ux8\ne3n8O5tzeyiwV3z7PGBH4H7gg3jcdwANK/Hcvwu0VmCdfwg8n7uOgIPjnJet4LHFnq/pOfOsSrRb\n+jlgMTAPuAfYs8Dycp/f3YAH4uf2/V7kKek1lb/ui03Le8zXgNZ4mYuBp4i+uPSUo8+vk57ejznz\n7B7P805+zt489zn5HoznWwy8AlwPbJIzz7Hx3/tWT7l7WKet8Wvu30QtCovn+zbwbPx3/wUc10Pm\nfl8PPbyWlwNDc+Y7nGjP0DtEnzFvAv8H7Fvu925SP+r5Vp+Ob4Ud/bCmeNr17j6vpwe6+3+6Lcxs\nZ2AEMNXdPwKmAOsQvbhXyN3nE32ojM67qzH+vTowKufvjQDWA2b0sNhWoIUoVytwfs7Pgpz5diF6\nA35E9OH0BHAYcL+ZrV7K+HOZ2fB4bI/09rH9oI1oy3W13CHFv9dawWNnED03HR9omfh2S848fyD6\nkAW4GphKtF6mm9nXiyx3d2A6USvj+ngZK7Syr6kVLPty4PdEX8Jui8e1GjDZzH5W5GH9+jopxKMW\nxV+IXj/75N1d8nNvZlcR5dsW+B1wJfAk0XO3Q/6f7cNQvwTcR1S0ricq8D8CLjaz04HLiZ6fG4jW\n2S/MbK/8hSS4Hmawgve+mX0f+C3Rbv/bgP8h+oK4JbB/756OKlbp6l+LP/S85XtTfN8N8e3pRN8K\nG/v4t66PHz86vr0x0ZZ0ay+WcWW8jK1zpv0/4GGgHbgwZ/r343kPLpA3d+un27f3AvctJ9rNnnvf\nL+PpX+/Dc/Gd+LEHVmCd1wPr5U07Mx5PU4nLeB14rcD0pvj5uhuoy5k+PF4/84C1izy/RyX5miqy\n7gtu+QJj4um303UPwSpEBW45sGN/v056ej/mzdccL/P8vjz3wCHxvH/LXR/xfWsAA3NuHxv/rd5u\n+S4HDsiZPoBo67cdmE3XresdCuVOej0UGn/eY/4OvAGsUeC+QUm+T8v5oy3fytrKzDLxz8/M7Ami\nN/P7wEXxPBvGv9/q7cLNbE3gCOAtd58B4O5vEX3j3MPMNi9xUa1E31RHx8tdi2j3+L3A43TdKh5N\n9Mb6S2/HW8BD7j41b9pN8Vh27sPy9iDamni01AeY2RQze7QXP1cVWo67t3t00FeuY4l6n90Okuul\nY4lyneE5/WB3/wfR87Uu0RZIvr97gQP0etKPr6lCvk/02jnRc/biuPty4Byi9X5Egcf19+ukmH/H\ny1w/Z1pvnvsT43l/6O6Lchfs7h+7e+5en7560N2n5Sx3MdHu2jWJ9p69mXPfU0S7nj+Xt4xqWA9L\nKbDl79GeuCDogKvK2pKoTwLwH6I39w3AT929rR+WP5bozX993vQpwN5E/Z9zSljOQ0RvhNHxsvYg\neu3MIHpjnWNmA+I3+h7As+7eH0eyPlVgWseHR7GDy3oyCnixN2Nz92P68HdWyMyOJdri2tsLtAt6\n6fPAh+7+bIH7Wok+TLcHfp1335N9+Fv99ZoqZBeiPuWJZt2OyenYbblNgcf19+ukmEIHCvXmud8J\nWOzuf+3HMeUrNI45Pdz3NtHznqvS6+G3RAeZPm9mtxE9j3919yW9WEbVU/GtrP9z90NWMM/bwNZE\nu/Ze6eXyjyMqmvlbN3cA1wDHmtm5Hu/PKcbd55vZc3zS520kOgjicaJzxZuB3c3sLWAI3T/k++qD\nAtOWxb9X6c2CzOzTwOZ0Lxod9w8A1nf32b0aYR+Y2aZEPcKv99MH8aeA14rc93bOPPnm9uFv9ctr\nqojBROu12PmfTrQbNV+/vU5W4DPx73dzpvXmuV+X6BztJBV6Lpb3cN8yuteBiq4Hd7/MzN4Hvkd0\n1sc5wMdmNhU43d3fKXVZ1Uy7navfo0TfuPfuzYPMrIGotwLwbO4FBIiOUh0AbETU3ylFK7B+fEBV\nI9E30WVEp5csiac1Er0xW3sz1jLZPf5d7GCr4yn8gdKv4l32vyU6avRP/bTYD4BPF7nv0znz5OtV\ngUzgNZXvA2Cuu6/Sw8++fVx2f2gkes5y9xj05rlfwCcFfEWyRO/7QhtI65a4jL6q+Hpw9xvdfWdg\nA6K9LfcCRxG9d4KgLd/q10J0Du8JZnal93DEs5mt7vF5pETFpOOIwlcLzD6I6FSC44BpBe7P10p0\nrvHBRLvPzofoCGsz+yvRl4M3KL3f2/FtvD+3THqyG9EHZ7cLa1i0b63R3ScVuO8WYIte/J3H3X18\nD/dfD/zM3f+c87ePdPdbe/E38s0EGs3s8wV2fzYS5Z65Esvv0N+vqXx/A8aY2dBy7IHoDTPbg6il\n8g7RQZAdSnnun45vP0F0oZdR7r6i4w46epuFrqyVf1R0f0t6PZT83o+Pk7gDuMPMngL2NLN13P3D\nBMZVViq+Vc7dXzWziUTnEd5tZl/Pf0OY2dpEBXoOcE38gf4tPjmadU7+cuOrZr0BHGxmg3sq6rGH\n4t+nEr1pWnPuayU6/WVLSu/3dvy9TUuYtz/sDvy7SC/9RKIPxm7c/Zv9NYD4VI9Wd789Z/IWfHJQ\nXV/9iqgff7GZHdxx4E98atXxRFtcd67MH0joNZVvEtFVvG40s7H5r6N4y5t+Oh6iZGb2ZaIvwQ6c\nldd7LOW5vyue9zrgQOAqM9vH3Tv3RpjZGsCAnAOK/h7/vSPN7DJ3/zieb0vgZJK9NGfS66HH976Z\n7enuf8mbthbRFv8yPineqZba4mtm04jO+fqJu5+XM30gMJHokn1rAX8FTnX35ysy0P5xFtHRiuOB\nl83sAeBFoq3MzYH9iM7Z6zg4aD+i8/PuKfQhCeDuWTObQlTUjyY6P7GonL7v54lOuP9bzt2tRJfg\nG0jpu5xfIjrA7EgzW0p0YIavaBx9YWafAkYSfYPOnb4G0QfZT4n66okxs/2JvrjMMLPcFsLmwIUr\nufhfEe2a+zIw08zuJurbHUF0gExT/tG1fdDvr6kCj7/HzC4m+iL5ipndS/S6GEJ0gM+uwDeIzplO\nwlZmlon/vQbRl6JRwFZE566e6u75/xlEyc+9u//ZzCYBPyB6H/+RqBA1EO2qP464ULv7HDO7Nc77\n9/jzbgPgv4gu4DE2gfzEfzvp9VD0vR9v0d5pZguIjilpI/rsOxAYBkyKD+xMv96cl1QtP0Qr/t9E\n34AuyLvvYaLz2b5OVJxbiQ6Q2KjS484ZY0M89jt7+bhdiA7dfwVYRFQE/wn8L13Pu7slXv7hK1je\n1vF8T5f496+I5783b/pq8XiWkXN+b4G8N+ZN35loF96C+P7cK1wtB84tdVk95Luf6PzY5URbZXcT\nfXg9SvTBtxx4tAzr/P2cjLk/y4DNSlzG68C/ity3CtHFFHKvsnQ3sHuBeYs+vz387T69pgqtrxWt\nQ6JC9CeiXbwfxettBnAKOVd568fXSce8uT+L4r97L1ERKvr50ZvnPp7/63xy5ahFRO/ha4CN8+Zb\nI37P/Tte7tNERb1b7hU8F5n4vkJXO5sBLCv3eqDIez++bxzwR6ID2RYTHRj4F+CbSb9Py/nTccmx\n1Ii3bP9B9AK4lZwtXzM7lGjrZrTHuy3irZ7XgSnufkplRi0iIvKJNB7tfBnRdY8LHfV2MFFfr7Nf\n4FFf5U+U/j/HiIiIJCpVxdfMdifqJZ1UZJYRRBevz/cCMDQ+l1NERKSiUlN8LfpPx68HLvf4/7kt\nYDCfHKKfq+PoukFJjE1ERKQ30nS08xlER71d1MM8RuFD8Hv8/yPNLF2NbxERKQt3T+T/H07Flm98\nOb6zgHOBNc1s3fjAK4A14tt1RFu4gwssomOLt+hFuSt95FuSP5lMpuJjUD5lU77wfkLPl6RUFF/g\ns0SH3f+aqIDOJyq0TnSI/zxgO6Le7ogCj98WmO2hnB/WS7Nmzar0EBIVcr6Qs4HypV3o+ZKUlt3O\nT9P9P3OH6Fy5KUT/E9CrRCeoN5nZHu7+MHSeanQw/XexfxERkZWSiuLr0elC3a4XHP93V205hfYu\nogv9/9rMfkx0AveZ8eyXl2e01aepqanSQ0hUyPlCzgbKl3ah50tS6i6ykcvMlhNdZCOTM63j8pKH\nER2g9ShwmvdweUkz8zQ/DyIi0v/MDK/lA66K8ei/tsrkTVvg7t9x9/XdfW1337+nwlsLWltbKz2E\nRIWcL+RskK58b775JmPHjmXgwIGsu+66fO1rX+ONN97o8TFpytcXoedLUqqLr4hIOSxZsoTRo0fz\n8ssvM2XKFH7961/zyiuvsPfee7NkyZIVL0AkT6p3O/cX7XYWkZ5cddVVTJgwgZdffpnNNtsMiI70\n3XLLLbn88ss55RRdNj5ESe52VvFFxVdEerbvvvvy8ccf8/DDD3eZ3tjYiJkxY8aMCo1MkqSer6yU\n0PsyIecLORtUd762tjaam5vJZDI8/vjjNDQ0dJtnxIgRvPjii0WXUc35+kPo+ZKk4isikqetrY1J\nkyYxYcIEmpubWbp0Ka+88gptbV3///jBgwczf37RC+eJFKXdzmi3s4h01dzczIQJE6ivrwdgjTXW\n4OSTT2bttdcmk/nkBItzzjmHyy67jKVLl1ZqqJIg7XYWESmjbDbbWXgBBg0axKJFi8hms13mmz9/\nPoMG6T9Lk95T8a0BofdlQs4Xcjao3nx1dXW0t7d33h4xYgTPPvssdXVdPzJffPFFtt1226LLqdZ8\n/SX0fElS8RURydPU1EQmk+kswGPGjOGxxx5jn3326Zxn1qxZPPLIIxx66KGVGqakmHq+qOcrIt21\ntbXR0tJCNptl2bJl3HrrrayzzjpceOGFAJx33nm0t7fzzDPPMGDAgAqPVpKg83wTpuIrIivy5ptv\ncuqpp3L//ffj7uy7775cccUVDB06tNJDk4TogCtZKaH3ZULOF3I2SFe+TTbZhN///vcsWLCAhQsX\ncvvtt6+w8KYpX1+Eni9JKr4iIiJlpt3OaLeziIh0p93OIiIiAVHxrQGh92VCzhdyNlC+tAs9X5JU\nfEVERMpMPV/U8xURke7U8xUREQmIim8NCL0vE3K+kLOB8qVd6PmSpOIrIiJSZur5op6viIh0p56v\niIhIQFR8a0DofZmQ84WcDZQv7ULPlyQVXxGRlLj99tsZO3Ysw4YNY8CAAWyzzTacddZZLFq0qNJD\nk15Szxf1fEUkHXbddVcaGho49NBD2WSTTXj66afJZDIMHz6cRx99tNLDC47+P9+EqfiKSBq8//77\nrLfeel2mTZkyhaamJh588EEaGxsrM7BA6YArWSmh92VCzhdyNlC+UrS1tdHc3Ewmk+HnP/85bW1t\nXe7feeedcXfeeuutlf5bvRX6+kvSqpUegIiIFNbW1sakSZNobm6mvr6e9vZ2MpkM48ePp6GhAYgK\noJkxfPjwCo9WekO7ndFuZxGpTs3NzUyYMIH6+vrOae3t7UycOJFMJsNbb73FDjvswBe+8AWmTZtW\nwZGGSbudRURqUDab7VJ4Aerr68lms7S3t3PooYey+uqrc9NNN1VohNJXKr41IPS+TMj5Qs4Gyrci\ndXV1tLe3d5nW3t5ONpvl4IMPZtasWdx7771stNFGK/V3+ir09ZckFV8RkSrV1NREJpPpLMDt7e2c\ne+65PPLIIzz55JPcc889bLvtthUepfSFer6o5ysi1autrY2Wlhay2SxmxpNPPsn06dO5++67dWpR\nwnSeb8JUfEUkDU488UQmT57MOeecw0EHHdTlvk022YSNN964QiMLkw64kpUSel8m5HwhZwPl661p\n06ZhZvz0pz9l1KhRXX5uvPHGfv1bpQh9/SVJ5/mKiKTE66+/XukhSD/Rbme021lERLrTbmcREZGA\nqPjWgND7MiHnCzkbKF/ahZ4vSSq+IiIiZaaeL+r5iohId+r5ioiIBCQ1xdfM9jezB81sjpl9ZGZv\nmNlvzWx43nybmNlUM1tgZgvN7HYz27RS464GofdlQs4XcjZQvrQLPV+SUlN8gcHAk8D3gf2AM4AR\nwF87iquZrQXMALYCjgGOBrYEpsf3iYiIVFyqe75mthXwEnC6u19hZj8EJgJbufvr8TzDgFeAH7n7\nlUWWo56viIh0oZ5vcfPi30vj3wcDj3UUXgB3nwU8Ahxa3qGJiIgUlrria2Z1ZraamW0JTAb+DdwW\n3z0CeL7Aw14Aavb/3Qq9LxNyvpCzgfKlXej5kpTGazs/DuwY//sVYB93fz++PRiYX+Ax84BBZRib\niIjICqWx+B4NfAr4LDABeMDMdnP32fH9hZq3K9xn39TUxLBhwwAYOHAgI0eO7Py/Mju+3aX1dse0\nahmP8pV+u7GxsarGo3zKF3K+1tZWWlpaADrrQVLSfsDVusAs4FZ3P8nM3gb+4O4n5s13DTDW3T9d\nZDk64EpERLrQAVdFuPtC4FVgi3jSC0R933zbAi+Wa1zVpuObXahCzhdyNlC+tAs9X5JSXXzN7NPA\nNkQFGOAu4Evx6UUd8wwDdgPuLO/oRERECkvNbmczuwN4CngW+ADYGjgF2AD4oru/amYDgJnAEuDc\n+KEXAPXA9u6+uMiytdtZRES6SHK3c5qK74+ArwObA6sDbxBdzeqSnIOtMLNNgCuIroJlwAPAqbnz\nFFi2iq+IiHShni/g7pe7+87uPtjd13b34e5+Un5Rdfc33f1wdx/o7uu6+9d6Kry1IPS+TMj5Qs4G\nypd2oedLUmqKr4iISChSs9s5SdrtLCIi+bTbWUREJCAqvjUg9L5MyPlCzgbKl3ah50uSiq+IiEiZ\nqeeLer4iItKder4iIiIBUfGtAaH3ZULOF3I2UL60Cz1fklR8RUREykw9X9TzFRGR7tTzFRERCYiK\nbw0IvS8Tcr6Qs4HypV3o+ZKk4isiIlJm6vminq+IiHSnnq+IiEhAVHxrQOh9mZDzhZwNlC/tQs+X\nJBVfERGRMlPPF/V8RUSkO/V8RUTyvPXWW4wfP55Ro0ZRX19PXV0ds2fPrvSwREqi4lsDQu/LhJwv\n5GywcvleffVVpk6dyuDBg9lzzz0xS2QDZaVo/UkxKr4ikkp77bUXc+bM4c9//jNjx46t9HBEekU9\nX9TzFUmDtrY2WlpayGaz1NXV0dTURENDAwA33ngjJ5xwAq+//jpDhw6t8EglFEn2fFdNYqEiIv2p\nra2NSZMm0dzcTH19Pe3t7WQyGcaPH99ZgEXSRLuda0DofZmQ84WcDUrP19LS0ll4Aerr62lubqal\npSW5wfUDrT8pRsVXRKpeNpvtLLwd6uvryWazFRqRyMpR8a0BjY2NlR5CokLOF3I2KD1fXV0d7e3t\nXaa1t7dTV1fdH2Faf1JMdb9yRUSApqYmMplMZwHu6Pk2NTVVdmAifaTiWwNC78uEnC/kbFB6voaG\nBsaPH8/EiRPJZDJMnDgxFQdbaf1JMTraWURSoaGhgUwm02Xa7bffDsCTTz6Ju3P33XczZMgQhgwZ\nwp577lmJYYqUROf5ovN8RdKqrq6u4JWt9tprL6ZPn16BEUlIdJ6viEgBOtpZ0ko93xoQel8m5Hwh\nZwPlS7vQ8yVJxVdERKTM1PNFPV8REelO/5+viIhIQFR8a0DofZmQ84WcDZQv7ULPlyQVXxERkTJT\nzxf1fEVEpDv1fEVERAKi4lsDQu/LhJwv5GygfGkXer4kqfiKiIiUmXq+qOcrIiLdqecLmNnXzGyq\nmc0ys8Vm9pKZXWRma+fNN9DMbjCzd81skZndb2bbVWrcIiIi+VJTfIEJwDLgTGAMcC1wInBf3nx/\nAvYHvg98FVgNmGFmG5VvqNUl9L5MyPlCzgbKl3ah50tSmv5Xo6+4+/s5tx82s/lAi5k1unurmR0K\njAJGu/tfAMzsMeB14MfAKWUftYiISJ5U93zNbBvgReAYd/+Nmd0AjHH3TfPmawH2cvfNiixHPV8R\nEelCPd/iGgEnKsAAI4DnC8z3AjDUzAaUaVwiIiJFpbb4mtnGQDNwv7s/HU8eDMwvMPu8+Pegcoyt\n2oTelwk5X8jZQPnSLvR8SUpl8TWzeuBOYClwXO5dRFvC3R5SjnGJiIiUInU9XzNbA7gH+Dywp7u/\nmHPfY8B8dz8w7zE/Ai4B1nH3xQWW6cceeyzDhg0DYODAgYwcOZLGxkbgk293uq3buq3buh3u7dbW\nVlpaWgAYNmwYzc3NifV8U1V8zWxVoi3ePYB93P2JvPtvBPZz96F5028GGnXAlYiIlEoHXAFmZsAt\nwGjgkPzCG7sL2NjM9sh53KeAg4mKdk3q+GYXqpDzhZwNlC/tQs+XpDSd53stMBb4CbDEzL6Yc9+b\n7v4WUfF9DPi1mf0YWEB0UQ6Ay8s5WBERkWJSs9vZzF4Hhha5u9ndL4jnGwhMBA4D1gQeBU5z90Kn\nIHUsW7udRUSkiyR3O6em+CZJxVdERPKp5ysrJfS+TMj5Qs4Gypd2oedLkoqviIhImWm3M9rtLCIi\n3Wm3s4iISEBUfGtA6H2ZkPOFnA2UL+1Cz5ckFV8REZEyU88X9XxFRKQ79XxFREQCouJbA0Lvy4Sc\nL+RsoHxpF3q+JKn4ioiIlJl6vqjnKyIi3annKyIiEhAV3xoQel8m5HwhZwPlS7vQ8yVJxVdERKTM\n1PNFPV8REelOPV8REZGAqPjWgND7MiHnCzkbKF/ahZ4vSSq+IiIiZaaeL+r5iohId+r5ioiIBETF\ntwaE3pcJOV/I2UD50i70fElS8RURESkz9XxRz1dERLpTz1dERCQgKr41IPS+TMj5Qs4Gypd2oedL\nkoqviIhImanni3q+IiLSnXq+IiIiAVHxrQGh92VCzhdyNlC+tAs9X5JUfEVERMpMPV/U8xURke7U\n8xUREQmIim8NCL0vE3K+kLOB8qVd6PmSpOIrIiJSZur5op6viIh0p56viIhIQFR8a0DofZmQ8/U2\n2yOPPMKYMWP49Kc/zbrrrsuOO+7IzTffnMzg+kHI6w6UT4pT8RUJxHPPPcd+++3HsmXLuOGGG7jj\njjvYZZddOP7445k8eXKlhyciOdTzRT1fCcNZZ53F//zP/zB//nzWWmutzum77rordXV1PPLIIxUc\nnUj6qOcrIiv0n//8h9VWW40111yzy/SBAweSzWYrNCoRKUTFtwaE3pcJOd+KsrW1tdHc3Ewmk2Hx\n4sW4OyeffDJz5sxh4cKF/OIXv2D69Omcdtpp5RlwL4W87kD5pLhVKz0AEembtrY2Jk2aRHNzM/X1\n9bS3t7Nw4UJuv/12rrnmGgBWX311rr/+eg4//PAKj1ZEcqnni3q+kk7Nzc1MmDCB+vp6AF599VX2\n3ntv1lhjDa6++mrWXHNN7rzzTq699lp++ctf8o1vfKPCIxZJlyR7vqnZ8jWzjYEzgB2B7YG1gGHu\nPjtvvjWAnwBHAQOBmcB/u/vD5R2xSLKy2Wxn4QU488wzWWONNTjyyCM58MADARg9ejTvvfceP/zh\nD1V8RapImnq+WwBjgXnAX4Bim6o3AccD5wAHAXOAe83s8+UYZDUKvS8Tcr6estXV1dHe3t55+/nn\nn2fEiBGsumrX79S77LIL77//Pu+8805Sw+yzkNcdKJ8Ul5ri6+4Puftn3P0rwNRC85jZ9sA3gFPc\n/SZ3nwF8HZgNXFC+0Yokr6mpiUwm01mAhwwZwkMPPcTRRx/dZb7HHnuMNddck8GDB1dimCJSQCp7\nvmZ2PPC/wGa5u53N7FzgbGCgu3+UM/184L+BT7n7fwosTz1fSaW2tjZaWlrIZrO89NJLTJ06lf32\n24+TTjqJtdZaizvvvJPrrruO0047jcsvv7zSwxVJlSR7vqEV31uBke4+PG/+w4HbgO3c/R8Flqfi\nK0G49957ufTSS3nhhRf46KOP2HzzzRk3bhwnnHACZol8hogESxfZKN1gYH6B6fNy7q85ofdlQs7X\n22xjxoxh+vTpzJ07l4ULF/LUU08xbty4qi28Ia87UD4pLrTiaxQ+EKs6P3lERKQmpeZUoxLNAzYt\nMH1Qzv0FNTU1MWzYMCC6HN/IkSNpbGwEPvl2l9bbHdOqZTzKV/rtxsbGqhqP8ilfyPlaW1tpaWkB\n6KwHSQn6SxhwAAAgAElEQVSt56sDrkREpF+o51u6u4DVgc5r6ZnZKkSnG91bqPDWgo5vdqEKOV/I\n2UD50i70fElK1W5nM/ta/M+diPq4Xzazd4F33f0v7v6Mmf0WuNLMVgdeB04ChhGd/ysiIlJxqdrt\nbGZZCh9Q9ZC77x3PswbwU+CbRJeXfAb4cU+Xl9RuZxERyafzfBOm4isiIvnU85WVEnpfJuR8IWcD\n5Uu70PMlScVXRESkzLTbGe12FhGR7rTbWUREJCAqvjUg9L5MyPlCzgbKl3ah50uSiq+IiEiZqeeL\ner4iItKder4iIiIBUfGtAaH3ZULOF3I2UL60Cz1fklR8RUREykw9X9TzFRGR7tTzFRERCYiKbw0I\nvS8Tcr6Qs4HypV3o+ZKk4isiIlJm6vminq+IiHSnnq+IiEhAVHxrQOh9mZDzhZwNlC/tQs+XJBVf\nERGRMlPPF/V8RUSkO/V8RUREAqLiWwNC78uEnC/kbKB8aRd6viSp+IqIiJSZer6o5ysiIt2p5ysi\nIhIQFd8aEHpfJuR8IWcD5Uu70PMlScVXRESkzNTzRT1fERHpTj1fERGRgKj41oDQ+zIh5ws5Gyhf\n2oWeL0kqviIiImWmni/q+YqISHfq+YqIiARExbcGhN6XCTlfyNlA+dIu9HxJUvEVEREpM/V8Uc9X\nRES6U89XREQkICq+NSD0vkzI+ULOBsqXdqHnS5KKr4iISJmp54t6viIi0p16viIiIgFR8a0Bofdl\nQs4XcjZQvrQLPV+SVHxFRETKTD1f1PMVEZHu1PPtJTPbxMymmtkCM1toZreb2aaVHpeIiAgEWHzN\nbC1gBrAVcAxwNLAlMD2+r+aE3pcJOV/I2UD50i70fElatdIDSMAJwDBgK3d/HcDMngNeAcYBV1Zu\naCIiIr3o+ZrZ9sCvgAZgGnCyu79jZt8EjnH3A5MbZunM7AFgDXffI296K+DuPrrAY9TzFRGRLqql\n53s+kAF2Ax4Efm1mG7r7LcCOCYytr0YAzxeY/gKwbZnHIiIi0k1viu+f3f2P7v6Cu/8CGAucYmaf\nTmhsfTUYmF9g+jxgUJnHUhVC78uEnC/kbKB8aRd6viT1pvi6mW1nZpPMbF13/wA4EzgEqLYDmQrt\nQ05k14GIiEhvlXTAlZnt6u43mdkBRAcuLYKogQr8wszeTXCMvTWfaOs33yAKbxED0NTUxLBhwwAY\nOHAgI0eOpLGxEfjk211ab3dMq5bxKF/ptxsbG6tqPMqnfCHna21tpaWlBaCzHiSlpAOuzGxnYBt3\nnxLf/gqwE/ATd1+W6Ah7ycweBFZz9z3zps8A0AFXIiJSimo44OoFYKSZ7QXg7n8GXgduSGJQK+ku\n4EtmNqxjQvzv3YA7KzKiCuv4ZheqkPOFnA2UL+1Cz5ekUovvH4BW4BUz+0I87Ungq0kMaiX9ApgF\n3Glmh5jZIcAfgTbgfys5MBERESh9t/Ns4FB3f9rMtgaWA1+OH39VwmPsNTPbBLgC2I/oQKsHgFPd\nfXaR+bXbWUREukhyt3OpxfdLRJdpPNnds2a2I/ANd5+QxKDKTcVXRETyVbzn6+6PufsP3D0b3/47\n0Gpm1bjbWfKE3pcJOV/I2UD50i70fEnq87Wd3f3PZrZZfw5GRESkFuj/80W7nUVEpLuK73YWERGR\n/qPiWwNC78uEnC/kbKB8aRd6viSp+IqISDAOOOAA6urqOO+88yo9lB6p54t6viIiIbj11ls5/fTT\nmTt3LmeffTYXXHDBSi1PPV8REZEeLFiwgNNOO40rrriCNGxMqfjWgND7MiHnCzkbKF/aVSpfW1sb\nzc3NZDIZmpubaWtr48c//jGf+9znOOKIIyoypt7q83m+IiIi5dbW1sakSZNobm6mvr6e9vZ2jj/+\neO68806ee+65Sg+vZOr5op6viEhaNDc3M2HCBOrr6wFYtmwZn//859lggw06t8Tr6uo455xz1PMV\nERHpD9lstrPwAlxyySUsXbqU3XbbrYKj6j0V3xqgvlN6hZwNlC/tKpGvrq6O9vZ2AN544w0uuugi\nzj77bLLZLAsXLmTBggUAfPzxxyxcuJBsNlv2MZZCxVdERFKjqamJTCZDe3s7r732Gh9//DHHH388\nl156KYMGDWLw4MGYGZdffjmDBw/m+eefr/SQC1LPF/V8RUTSpK2tjZaWFpYsWcLcuXM58MAD2XDD\nDTvvb2xs5JhjjuE73/kOO+64IwMGDOjT30my56ujnUVEJFUaGhrIZDIrnGePPfYo04h6T7uda4D6\nTukVcjZQvrSr1nxmhlkiG6z9Rlu+IiISlOXLl1d6CCukni/q+YqISHc6z1dERCQgKr41oFr7Mv0l\n5HwhZwPlS7vQ8yVJxVdERKTM1PNFPV8REelOPV8REZGAqPjWgND7MiHnCzkbKF/ahZ4vSSq+IiIi\nZaaeL+r5iohId+r5ioiIBETFtwaE3pcJOV/I2UD50i70fElS8RURESkz9XxRz1dERLpTz1dERCQg\nKr41IPS+TMj5Qs4Gypd2oedLkoqviIhImanni3q+IiLSnXq+IiIiAVHxrQGh92VCzhdyNlC+tAs9\nX5JUfEVERMpMPV/U8xURke7U8xUREQmIim8NCL0vE3K+kLOB8qVd6PmSpOIrIiJSZur5op6viIh0\np54vYGanmdldZvZvM8ua2Xk9zHuYmT1lZkvMbJaZnW1mqckqIiJhS1NB+g4wBPgDUHQz1czGAFOB\nx4EDgCuBc4CflmGMVSn0vkzI+ULOBsqXdqHnS9KqlR5Aqdx9WwAzWwU4sYdZLwb+4u4d8zxkZusA\nZ5vZFe7+TsJDFRER6VHqer5x8f0PcL67X5B33ybAbOA77n5TzvRhwGvAt939lwWWqZ6viIh0oZ5v\n6UYQ7ZJ+IXeiu88CFgPbVmBMIiIiXYRWfAfHv+cXuG9+zv01JfS+TMj5Qs4Gypd2oedLUkV6vma2\nD3B/CbO2uvvevVl0/LvQPuQedx00NTUxbNgwAAYOHMjIkSNpbGyMBhG/wNJ6e+bMmVU1HuXTbd3W\n7Wq83draSktLC0BnPUhKRXq+ZrYmMLSEWRe7+5t5j+2p53sA8H/AKHd/PO++RcA17v7fBcajnq+I\niHSRZM+3Ilu+7v4R8HICi36BaAt3BNGpRgCYWQMwAHgxgb8pIiLSK0H1fN39DeAZ4Ki8u44BlgL3\nlH1QVaBjt0qoQs4XcjZQvrQLPV+SUnOer5ntCAwDVoknbWtmX4v//X/x1jTAWcCfzOx64FZgB+Bs\n4Eqd4ysiItUgNef5mtnNwLeK3L2Zu8/OmfcwIANsA8wFfgFcVKyxq56viIjkS7Lnm5rimyQVXxER\nyaeLbMhKCb0vE3K+kLOB8qVd6PmSpOIrknKjR4+mrq6u4M+Xv/zlSg9PRArQbme021nS7aWXXuKD\nDz7oMu3RRx/l9NNP59prr2XcuHEVGplIuqnnmzAVXwnN8ccfzy233MKcOXMYOHBgpYcjkkrq+cpK\nCb0vE3K+vmT76KOPmDp1KoccckjVF96Q1x0onxSXmvN8ReQTbW1ttLS0kM1mqauro6mpiYaGBgBu\nv/12Fi1axLHHHlvhUYpIMdrtjHY7S7q0tbUxadIkmpubqa+vp729nUwmw/jx42loaGDMmDE8++yz\nvPXWW9TVaeeWSF9pt7OIdGppaeksvAD19fU0NzfT0tLCnDlzePDBBzn66KNVeEWqmN6dNSD0vkzI\n+Qply2aznYW3Q319PdlslilTpuDufOtbxS4GV11CXnegfFKciq9IytTV1dHe3t5lWnt7O3V1dUyZ\nMoXtt9+ez33ucxUanYiUQj1f1POVdCnW821sbOSQQw7hyiuv5OSTT670MEVST+f5JkzFV9Km0NHO\nP/vZz5g8eTJvvvkmQ4YMqfQQRVJPxTdhoRff1tZWGhsbKz2MxIScr9Rsy5YtY6ONNmLUqFH88Y9/\nTH5g/STkdQfKl3ZJFl+d5ysSgFVXXZV33tF/Vy2SFtryJfwtXxER6T2d5ysiIhIQFd8aEPq5eCHn\nCzkbKF/ahZ4vSSq+IiIiZaaeL+r5iohId+r5ioiIBETFtwaE3pcJOV/I2UD50i70fElS8RURESkz\n9XxRz1fCdPfdd3PppZfy1FNPUVdXx9Zbb81ll10W9BWJRPqTer4i0iuTJ0/msMMOY+edd+aPf/wj\nU6dO5fDDD2fx4sWVHpqIoC1fIPwt39Cvvxpyvr5ka2trY/jw4Vx66aWMHz8+mYH1k5DXHShf2una\nziLSo9z/5ai1tZW6ujrGjRtX6WGJSBHa8iX8LV8JW/7/79vY2MhLL73Ej370I6677jra2toYNmwY\np556KieddFKlhyuSGtryFZGiWlpaOgsvwNy5c1myZAmZTIarr76az372s/z+97/nBz/4AcuXL6/6\nXdEitUAHXNWA0M/FCzlfKdmy2Wxn4e24vWjRIr7yla9w3HHH0djYyDXXXMMBBxzAxRdfnOBoey/k\ndQfKJ8Wp+IqkXF1dHe3t7Z2311tvPQA233zzLvPtv//+zJ07l7fffrus4xOR7lR8a0DIRyNC2PlK\nydbU1EQmk+kswFtvvTXuztFHH91lvo7jGurqqudtH/K6A+WT4tTzFUm5hoYGxo8fz8SJE8lmsyxd\nuhQz4x//+AfDhw/vnG/atGlssskmbLDBBhUcrYiAim9NCP1cvJDzlZqtoaGBTCbTefvtt99m3Lhx\nvPvuu50HXD3wwAO0tLQkN9g+CHndgfJJcSq+IgG68847OfPMMzn//POZP38+22yzDbfccgtHHHFE\npYcmIug8X0Dn+YqISHe6trOIiEhAVHxrQOjn4oWcL+RsoHxpF3q+JKn4ioiIlJl6vqjnKyIi3ann\nKyIiEhAV3xoQel8m5HwhZwPlS7vQ8yVJxVdERKTMUtHzNbMtgR8AjcBngQ+BJ4Bz3f3ZAvN/FzgN\n2AyYBVzh7pN7WL56viIi0oV6vrA/sBdwM/AV4ERgCPC4mX0hd8a48F4P/B4YA/wOuNbMxpV1xCIi\nIkWkpfje6u4j3f1Kd3/I3e8EDgCWAD/smMnMVgF+AvzS3c+L5z0PaAEujO+vOaH3ZULOF3I2UL60\nCz1fklJRfN19XoFpHwAvAxvnTN4VWB/4Td7sU4D1gN2TGqOIiEipUtHzLcTMBgFvADe6+w/jaeOA\na4GN3H1uzrxDgLnA9939ugLLqrqe70MPPcTo0aO7TR84cCDz5nX7LiIiIv0syZ5vmv9Xo5/Hv6/K\nmTY4/j0/b955efengpkxadIkdtppp85pq66a5lUmIiJQod3OZraPmWVL+Jle5PFnAkcSbcm+lntX\n/Lu6NmNXwjbbbMMuu+zS+bPDDjv0ehmh92VCzhdyNlC+tAs9X5IqtRn1CLBNCfMtzp9gZt8Dfgqc\n5e6/zLs7dwt3bs70wXn3d9PU1MSwYcOAaNfuyJEjO/+T6I4XWNK3N9tsM1paWnjttdeYO3cuHbvC\nV3b5M2fOLMv4K3U79Hy6rdu6XZ7bra2ttLS0AHTWg6SkqudrZscQHbk80d3/u8D9ewAPAfu6+/Sc\n6XsBM4DR7v5QgcdVvOfb1tbGpEmTaG5upr6+nmnTpnHggQcyZMgQ5s2bx8CBAxkzZgyXXHIJm266\naUXHKiJSC3SeL2Bm/wXcBPxvocIb+yvwHnBU3vRjgPeJtrirUktLS2fhBdhwww055ZRT2G+//Zgx\nYwbnnXceDzzwAKNGjeK9996r8GhFRGRlpKL4mtmewC3AM8CvzOyLOT8jO+Zz92XAucCxZnahme1l\nZhcATURXw1pWifGXIpvNdhZegJEjR3LFFVewxRZbsMcee3DyySczbdo03n77ba6++upeLbtjt0qo\nQs4XcjZQvrQLPV+S0nLo7GhgdeALwP/Lu6+N6JKTALj7ZDPLAqcDE4DZRAdmFb28ZDWoq6ujvb29\nSwFub2+nru6T70df+MIX2GqrrXjiiScqMUQREeknqer5JqUae77t7e1kMhnGjx9PQ0ND53zbbrst\nw4YN4+67767gaEVEwqfzfGtAQ0MD48ePZ+LEiWSzWerq6roV3ieffJKXX36ZI488soIjFRGRlaUt\nX6pjyzff0UcfzWabbcYOO+zAwIEDeeqpp7jkkktYe+21+fvf/87gwaVfL6S1tbXzsPoQhZwv5Gyg\nfGkXej5t+dag7bbbjttuu42f//znLF68mA033JCxY8dy/vnn96rwiohI9dGWL9W55SsiIpWl83xF\nREQCouJbA0I/Fy/kfCFnA+VLu9DzJUnFV0REpMzU80U9XxER6U49XxERkYCo+NaA0PsyIecLORso\nX9qFni9JKr4iIiJlpp4v6vmKiEh36vmKiIgERMW3BoTelwk5X8jZQPnSLvR8SVLxFRERKTP1fFHP\nV0REulPPV0REJCAqvjUg9L5MyPlCzgbKl3ah50uSiq+IiEiZqeeLer4iItKder4iIiIBUfGtAaH3\nZULOF3I2UL60Cz1fklR8RUREykw9X9TzFRGR7tTzFRERCYiKbw0IvS8Tcr6Qs4HypV3o+ZKk4isi\nIlJm6vminq+IiHSnnq+IiEhAVHxrQOh9mZDzhZwNlC/tQs+XJBVfERGRMlPPF/V8RUSkO/V8RURE\nAqLiWwNC78uEnC/kbKB8aRd6viSp+IqIiJSZer6o5ysiIt2p5ysiIhIQFd8aEHpfJuR8IWcD5Uu7\n0PMlScVXRESkzNTzRT1fERHpTj1fERGRgKj41oDQ+zIh5ws5Gyhf2oWeL0kqviIiImWWip6vma0N\n3AjsAHwG+A/wT2CSu/8mb14DzgBOADaM57vA3e/oYfnq+YqISBfq+cLqRAX3IuBg4BvAP4ApZvbD\nvHl/ApwHXA0cAPwV+L2ZHVC+4YqIiBSXiuLr7vPc/Wh3v9ndZ7j7NHf/NvAYcFzHfGY2BDgduNjd\nr3D3h9z9RGAGcEllRl95ofdlQs4XcjZQvrQLPV+SUlF8e/A+0RZxhwOA1YDf5M33a+BzZtZQroGJ\niIgUk4qeby4zWwVYFxgLTAKO6+j7mtnFwA/dfUDeY3YGHgcOcvd7CixTPV8REekiyZ7vqkksNClm\n9n2igguwlKjQ5m7lDgYWFHjovJz7RUREKqoiu53NbB8zy5bwMz3vobcBOxHtXr4B+LmZfTd30UCh\nTdhEvrmkReh9mZDzhZwNlC/tQs+XpEpt+T4CbFPCfItzb7j7+0R9XoD7zKwemGhmN7n7cqIt3EEF\nltMxbV6B+wBoampi2LBhAAwcOJCRI0fS2NgIfPICS+vtmTNnVtV4lE+3dVu3q/F2a2srLS0tAJ31\nICmp6/nmindDXw1s6u7/NrNjgBZgS3d/LWe+JqLzhD/r7m0FlqOer4iIdKHzfItrBBYB78S3pxEd\n/XxU3nxHA88XKrwiIiLlloria2YnmNlNZvZNM9vTzP7LzG4Dvgpc6O7LANz9XeAK4EwzO9XM9jKz\n64iK9JkVC1BhHbtVQhVyvpCzgfKlXej5kpSWo52fAw4BLic6Yvk9oitcHeTu0/LmPQv4EDiZTy4v\nebi7312+4YqIiBSX6p5vf1HPV0RE8qnnKyIiEhAV3xoQel8m5HwhZwPlS7vQ8yVJxVdERKTM1PNF\nPV8REelOPV8REZGAqPjWgND7MiHnCzkbKF/ahZ4vSSq+IiIiZaaeL+r5iohId+r5ioiIBETFtwaE\n3pcJOV/I2UD50i70fElS8RURESkz9XxRz1fCdN9993HppZfy4osvMn/+fIYMGcKoUaM4//zzGT58\neKWHJ1L1kuz5qvii4ithuu2223j66af54he/yJAhQ5g9ezYXX3wxb775Js899xybbrpppYcoUtV0\nwJWslND7MiHnW5lsRx55JJdeeilf/epX2WOPPTjqqKO44447+OCDD5g6dWr/DXIlhLzuQPmkOBVf\nkRoyePBgAFZbbbUKj0Sktmm3M9rtLOFoa2ujpaWFbDZLXV0dTU1NbLrppixfvpxZs2Zxxhln8Pjj\njzNz5kzWX3/9Sg9XpKoludt51SQWKiLl19bWxqRJk2hubqa+vp729nYymQz3338/zz33HABbbrkl\nDz74oAqvSIVpt3MNCL0vE3K+3mRraWnpLLwA9fX1NDc3s+eee/L4449z66238qlPfYp9992X2bNn\nJzTi3gl53YHySXEqviKByGaznYW3Q319PYMHD2bnnXfmiCOO4IEHHmDRokVccsklFRqliICKb01o\nbGys9BASFXK+3mSrq6ujvb29y7T29nbq6j55m6+77rpsscUWvPrqq/01xJUS8roD5ZPiVHxFAtHU\n1EQmk+kswB0936amps555s6dy0svvcQWW2xRoVGKCKj41oTQ+zIh5+tNtoaGBsaPH8/EiRPJZDLs\ntNNOADzzzDO0trYyefJkGhsbWX311TnttNMSGnHvhLzuQPmkOB3tLBKQhoYGMpkMAGuvvTa/+93v\nuOmmm1i6dCmbbropo0eP5owzzmDo0KEVHqlIbdN5vug8XxER6U6XlxQREQmIim8NCL0vE3K+kLOB\n8qVd6PmSpOIrIiJSZur5op6viIh0p56viIhIQFR8a0DofZmQ84WcDZQv7ULPlyQVXxERkTJTzxf1\nfEVEpDv1fEVERAKi4lsDQu/LhJwv5GygfGkXer4kqfiKiIiUmXq+qOcrIiLdqecrIiISEBXfGhB6\nXybkfCFnA+VLu9DzJUnFV0REpMzU80U9XxER6U49XxERkYCo+NaA0PsyIecLORsoX9qFni9JKr4i\nIiJllsqer5l9A/gN8Ka7Dy1w/3eB04DNgFnAFe4+uYflqecrIiJdqOebw8zWBf4HmFPk/u8C1wO/\nB8YAvwOuNbNxZRukiIhID1JXfIHLgZnAffl3mNkqwE+AX7r7ee7+kLufB7QAF8b315zQ+zIh5ws5\nGyhf2oWeL0mpKr5mthvwTeD7RWbZFVifaJd0rinAesDuyY2ues2cObPSQ0hUyPlCzgbKl3ah50tS\naoqvma0KTAYuc/fXisw2Iv79fN70FwADtk1oeFVtwYIFlR5CokLOF3I2UL60Cz1fklJTfIEzgNWB\nS3qYZ3D8e37e9Hl594uIiFRMRYqvme1jZtkSfqbH828BnAV8392X9rTo+LcOXc4xa9asSg8hUSHn\nCzkbKF/ahZ4vSRU51cjM1gS6nSJUwGJ3f9PM7gaWA0d3LAK4BtgT2A742N0/MrPvxdM3cve5OX9v\nCDCXqHhfV2A8KtYiItJNUqcarZrEQlfE3T8CXu7FQ4YTFev83ckQ7VK+iui83o7e7giiYtuho9f7\nYpHxJPLkioiIFFKR4tsHRwBr5k07E9gBGAu8FU/7K/AecBQwPWfeY4D3gUeSHaaIiMiKpaL4uvvf\n8qeZ2beJdjc/nDPfMjM7F7jGzP4NPADsAzQBP3D3ZWUasoiISFGpKL496NardffJZpYFTgcmALOJ\ner1FLy8pIiJSTmk61agLd/+2uzcUue8X7r6Nu6/l7lvHBfk0M7vLzP4dH0l9XqHHmtnFZvaMmc03\ns3Yz+4eZnWNmaxWY9zAze8rMlpjZLDM728wqdQT5CvOZ2Tpmdp6ZPWJm78UZHzGzQ4ssc/f4/sVm\nNsfMfhYfLFdWvVh3XzGz35jZP81secfR8kXm3dbM7jOzD+Pn4iYzG5RciuJKzRfPW9JrrlrWXTFm\ntl78nL8Tj/ExM9u/yLzfjd+HH5nZS2m4VKyZDTazq8zsX3G+18xskpmtX2DeqvkcKYWZHdvDGSrL\nzWyDvPnTuP42il+fc+Jxv2ZmPy0wX5+zVe0KTsB3gCHAH+j5VKR1gJuAbwBfAX4NnA3ckjuTmY0B\npgKPAwcAVwLnAN1WUJmUkm8o8D1gBlFf/OvAP4E/mNmJuTOa2eeJLuH5NnAQ0XPwbeDmJAa/AqWu\nu8OA7Yl6/28Um8nMPgO0Ep03/lXgJGBf4E/9M9xeKylfqa+5Klt33ZjZ6kSvwf2J9k79F9Eeqj+b\n2Z5586b1Wu1/Ao4ELiVaV5cRfabcmTtTFX6OlOLPwJfyfnYlOq7mb+7+TseMaVx/ZtYA/A3YAhgP\n7AdkgGV5861cNnevqR9gFSALnNeLx1xEdKrT4JxpTwHT8+Y7F/gI2KAa8wFrAWsWmP4AMCtv2h+I\nCvMqOdOOiZ+HkdWWrcC8D+evn5z7riA6Sn6dnGl7xMs+rBrXXW9ec9W47vLGfHQ8lj3ypj8DPJb3\nfMwFbsqb70bgndx81fQDbBmvx+Pzpo+Lc2/Z23Va7T8575/vBbD+pgGPAXU9zLPS2Wppy3dldFwh\n6z8AZrYJMJJoqzjXFKKtqQPLN7TSufsSj07zyvcksFHHDYsu5TkG+K27L8+Z73dEz0HB3dQpcjDw\nf+7+YccEjw7cm02VZiv1NZeSdfdFYInnHCwZuw/YOd4zAem9Vvvq8e8P86YvjH/XQXo/R4o4FvgY\n+G3OtNStPzP7LNEemavdPdvDrCudTcW3CDNbxczqzWxf4FTgxpwP6xFEuwdfyH2Mu88CFpO+a0jv\nBbyUc3tzolO78vN9DPyL9OXrFPc9N6P79b8hylut2Up9zaVh3S0n/iKb5+P493bx71Req93dXwAe\nAs41sx3jz5FdiLZo73b3f8azBvE5Er+nxgJ/cvfcazGkcf3tRrROPo6PCfnIzOaZ2S/NLPfyxCud\nTcW3ADMbQfTh8CHRt/H7iHYZdSh2DemOaam5hrSZnQDsQrRrvUNP+eaRonwFDCJ6c6QtW6mvuTSs\nu38CnzKzrfOmj4p/ryhLGq7VfhDRhYSeIPoceYzoy8/YnHlC+Rz5L6JjZX6ZNz2N628jos+HG4le\npwcAPyZan9Ny5lvpbKksvtbLa0P3wavATkRbhGcSHZQzJXcI8e9CB8es9NWyypCv4+80El0d7Ffu\nflvuXfHvfs9Xrmw9DSH+nbZ1V+q4E81XSB8y30J0MZxfmdl2Fh35fBZR3xCi3uGKspRNH9fpDUS7\n1/wlXD4AAAVRSURBVE8gugzuOGBn4PbcRce/y7auCumH1+yxwLvAPfmLjn9XbP31IVtHTZzh7uPd\nvdXdbyA6KHPH+AA56IdsaT3P9xFgmxLmW9yXhce76J6Kbz5sZm8DN5vZ1R5d8KOnbzcDc+7vq0Tz\nAZjZzkRHXj5AdLRtrp7yDeKTK4r1ReLZVmA+0RumWLZqXXelvuaSXHfF9Cqzuy80s68SbSk9Q/RB\n9irREaUXAnPi+XOz5F4udnDe/UnrVT4zO4joSOe93b01vu//mdnrwH1mdrC7/4nkP0dK1efXrJlt\nSHQho6sK9EirYf31Ntv78e8H8u6/j+h1+gXgXvohWyqLr/f+2tAr68n49xZEh6DnXkP68Y6Z4kPU\nB1DkGtKlSjqfmX2OaBfKU8DYvANzINo99jGf9DU6HrcG8Fmig3f6pALrLv/vLzGzWeRli21LdArS\nyiw/qXylvuYSW3fF9CWzuz8CbGFmmxMdGfqymf03sIRPvvj26Vrt/a0P+bYj+oL3ZN70jiv1DSc6\nFSnRz5FSreRr9hiircVfFbiv4uuvD9k6+u/FtmizOfOtVLZU7naugEailfEvAHd/g+gb+1F58x0D\nLKX77peqYWZbEn2LexU4ON7K78Ld/0NUnL9uXU/2P5zoKMy7yjHWBN0FHGRm63RMMLPdgQbyzsOs\nFqW+5tK27tz9X3HhXZtoD8yv3L1jKyT3Wu25qv1a7W/Hv3fJm/6l+PdbkO7PkRzHAM+6+7MF7kvj\n+nuMaP0dkDf9QKIa8ER8e+WzVfqcqjKeu7Uj8DWiC0tkgdvi218jPvcV+BzRLoXvAHvHT/glRLsk\n/pS3vAOJTrq+nqg3fCrRt/ZLqjjfEGBW/KI5kKgnlfuzWs7ytgfaiXpUewPHxy+q26oxWzzf0Hja\nWKJvns/lzDc0Z76NiM7FayU6LeeI+Hl5tFrXXW9ec9W07nrIfFGcb6/4/fYS0ZGjA/PmGxdnvjCe\n94L49vcqMe4Ss60DvBn/fI/oy/uJRLvTXwcG9HadVuMP0X9skwV+2MM8aVx/3yI6Iv86ogtsnES0\nG/mB/sxW8aBlfEJvjp/QQj9D43k2IDrn7l/xh9e7RLuDvkdOYcpZ5mHA0/GbZRbRlYSsivPt1cM8\nnfPlLHN3om9wi+MPjp9R4CId1ZAtnu/Y+MOg0HzfylvmCKIvWh8SFaYbgUHVuu56+5qrlnXXQ+Yb\nic6r/ij+fSV5hTdn3u8SFeclREegjqv0+EvItzHwi/izZHH8+3rgMwXmrZrPkV5mvDJef0NWMF8a\n199RwLPxmN+Ksw4oMF+fs1m8ABERESkT9XxFRETKTMVXRESkzFR8RUREykzFV0REpMxUfEVERMpM\nxVdERKTMVHxFRETKTMVXRESkzFR8RUREykzFV0REpMxUfEVqmJl9ycxmm9m9ZjbAzHY2s10rPS6R\n0OnaziI1zMx+TvSfHGxC9L/vzHT3KfF9JwD/cPeHKzdCkTCp+Ir8//buECeCIIgC6C8ghDPgVhAO\nAHjw3AGNhiNwBPxeAkvIYpDcgVNgoBA7DiTpSXbeU92jvvup6U56wapqr7u/q+o4yUl3b6rqKNtn\n/m6S3HX367wpYfcczB0AmM9UvKfZPpe2mb59JnmsqrN508HucuYLCzYV7Fd3v0/765kjwSKYfGGh\npqK9TPJRVc9JzpOskjzNmQuWwOQLC1RVqySH3X2fpJK8JblK8jBrMFgIF66AP1XVOsnahSv4f347\nA79U1W2Si+2y9rv7Ze5MsEtMvgAwmDNfABhM+QLAYMoXAAZTvgAwmPIFgMGULwAMpnwBYDDlCwCD\n/QCjyx4/bQ1XmQAAAABJRU5ErkJggg==\n", "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "W = V[:,:2]\n", "Z = np.dot(X, W)\n", "plt.figure(figsize=(8,8))\n", "ax = plt.gca();\n", "plot_latent_variables(Z, ax=ax)\n", "ax.set_aspect('equal')\n", "plt.show()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Note that this is very similar with the $y$-axis flipped. That part does not actually matter. What matters is the scaling by eigenvalues for computing. Before that scaling the proximity of points may not mean much if the eigenvalue is actually very large.\n", "\n", "Now, the second part asks us to see if we can properly identify documents related to abductions by using a document with the single word *abducted* as a probe." ] }, { "cell_type": "code", "execution_count": 7, "metadata": { "collapsed": false }, "outputs": [ { "data": { "text/plain": [ "array([[ 0.],\n", " [ 6.],\n", " [ 3.]])" ] }, "execution_count": 7, "metadata": {}, "output_type": "execute_result" } ], "source": [ "probe_document = np.zeros_like(words, dtype=np.float64)\n", "abducted_idx = (words=='abducted').as_matrix()\n", "probe_document[abducted_idx] = 1\n", "X[0:3,abducted_idx]" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Note that despite the first document being about abductions, it doesn't contain the word *abducted*.\n", "\n", "Let's look at the latent variable representation. We'll use cosine similarity to account for the difference in magnitude." ] }, { "cell_type": "code", "execution_count": 8, "metadata": { "collapsed": false }, "outputs": [ { "data": { "text/plain": [ "[(0, 0.59127529086736064),\n", " (2, 0.57730256865377172),\n", " (1, 0.45835648369851389),\n", " (8, 0.31235652085315346),\n", " (4, 0.15438034299529768),\n", " (7, 0.076686002588720159),\n", " (6, 0.039651047230294334),\n", " (5, 0.039384780001176511),\n", " (3, -0.13308361596893792)]" ] }, "execution_count": 8, "metadata": {}, "output_type": "execute_result" } ], "source": [ "from scipy.spatial import distance\n", "z = np.dot(probe_document, W)\n", "similarities = list(map(lambda i : (i, 1 - distance.cosine(z,Z[i,:])), range(len(Z))))\n", "similarities.sort(key=lambda similarity_tuple : -similarity_tuple[1])\n", "similarities" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Indeed, we find the three alien abduction documents, $0$, $2$, and $1$ are most similar to our probe." ] } ], "metadata": { "kernelspec": { "display_name": "Python 3", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.5.2" } }, "nbformat": 4, "nbformat_minor": 1 }