{ "cells": [ { "cell_type": "markdown", "metadata": { "collapsed": true }, "source": [ "### MidTerm Assignment: notebook 1: Revision" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "\n", "

Given date : March 30

\n", "\n", "Due date : April 17\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "#### Total : 10 pts " ] }, { "cell_type": "markdown", "metadata": { "collapsed": true }, "source": [ "#### Question 1.1. Statistical learning: Maximum likelihood (Total 5pts)\n", "\n", "This exercise contains a pen and paper part and a coding part. You should submit the pen and paper either in lateX or take a picture of your written solution and join it to the Assignment folder. \n", "\n", "We consider the dataset given below. This dataset was generated from a Gaussian distribution with a given mean $\\mathbf{\\mu} = (\\mu_1, \\mu_2)$ and covariance matrix $\\mathbf{\\Sigma} = \\left[\\begin{array}{cc}\n", "\\sigma_1^2 & 0 \\\\\n", "0 & \\sigma_2^2\n", "\\end{array}\\right]$. We would like to recover the mean and variance from the data. In order to do this, use the following steps:\n", "\n", "1. Write the general expression for the probability (multivariate (2D) Gaussian with diagonal covariance matrix) to observe a single sample \n", "2. We will assume that the samples are independent and identically distributed so that the probability of observing the whole dataset is the product of the probabilties of observing each one of the samples $\\left\\{\\mathbf{x}^{(i)} = (x_1^{(i)}, x_2^{(i)})\\right\\}_{i=1}^N$. Write down this probability\n", "3. Take the negative logarithm of this probability\n", "4. Once you have taken the logarithm, find the expression for $mu_1, \\mu_2$, $\\sigma_1$ and $\\sigma_2$ by maximizing the probability. " ] }, { "cell_type": "code", "execution_count": 66, "metadata": {}, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAX4AAAD8CAYAAABw1c+bAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi40LCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcv7US4rQAAH41JREFUeJzt3X+wXPV53/H3w+ViX7DHEtFNAheE\nREYRNiZB5g44licGt0aCAFJCZyzqpuCSUeKatnZbTUTdMUR2a001UyYpbrCGaGxmHEHNj1s54Mhq\n5QytqRyukGRZ2MKywEaLp5KRREx0iyXx9I89K87du2f37N3z+3xeM3fYPT92n3O0POec5/s932Pu\njoiI1MdZeQcgIiLZUuIXEakZJX4RkZpR4hcRqRklfhGRmlHiFxGpGSV+EZGaUeIXEakZJX4RkZo5\nO+8AOpk3b54vWLAg7zBEREpj586dP3P30TjLFjLxL1iwgMnJybzDEBEpDTP7cdxlVeoREakZJX4R\nkZpR4hcRqRklfhGRmlHiFxGpGSV+EZGaUeIXEamZQvbjFxGpi4ldDTZs3c8rx6e4cM4Ia5YtZuWS\nsVS/U4lfRCQnE7sa3P34XqZOngagcXyKux/fC5Bq8lepR0QkJxu27j+T9FumTp5mw9b9qX6vEr+I\nSE5eOT7V1/SkKPGLiOTkwjkjfU1PihK/iEhO1ixbzMjw0LRpI8NDrFm2ONXvVeOuiEhOWg246tUj\nIlIjK5eMpZ7o26nUIyJSM0r8IiI1o8QvIlIzPRO/mV1sZt8ys+fNbJ+Z/asOy5iZ/ZmZHTCz75rZ\n+0LzbjezHwZ/tye9ASIi0p84jbungH/j7s+Z2TuBnWa2zd2fDy1zA7Ao+LsG+HPgGjM7H7gHGAc8\nWHeLux9LdCtERCS2nmf87v5Td38ueP1z4PtAexP0CuAhb9oBzDGzC4BlwDZ3Pxok+23A8kS3QERE\n+tJXjd/MFgBLgO+0zRoDXg69PxRMi5ouIiI5iZ34zewdwGPAp9z975IOxMxWm9mkmU0eOXIk6Y8X\nEZFArMRvZsM0k/5X3f3xDos0gItD7y8KpkVNn8HdN7r7uLuPj46OxglLRERmIU6vHgP+Avi+u//n\niMW2AP806N3zfuA1d/8psBW43szmmtlc4PpgmoiI5CROr56lwO8De81sdzDt3wHzAdz9AeAp4Ebg\nAHAC+Hgw76iZfQ54NlhvnbsfTS58ERHpV8/E7+7/G7AeyzjwyYh5m4BNs4pOREQSpzt3RURqRolf\nRKRmlPhFRGpGiV9EpGaU+EVEakaJX0SkZpT4RURqRolfRKRmlPhFRGpGiV9EpGaU+EVEaibOIG0i\nIrmZ2NVgw9b9vHJ8igvnjLBm2WJWLtHznAahxC8ihTWxq8Hdj+9l6uRpABrHp7j78b0ASv4DUKlH\nRAprw9b9Z5J+y9TJ02zYuj+niKpBiV9ECuuV41N9TZd4lPhFpLAunDPS13SJR4lfRAprzbLFjAwP\nTZs2MjzEmmWLc4qoGtS4KyKF1WrAVa+eZPVM/Ga2CbgJOOzu7+0wfw3wsdDnvRsYDZ63+xLwc+A0\ncMrdx5MKXETqYeWSMSX6hMU54/8ycD/wUKeZ7r4B2ABgZjcDn257oPp17v6zAeMUqTT1VZcsxXnY\n+tNmtiDm590GbB4kIJG6UV91yVpijbtmdi6wHHgsNNmBb5rZTjNbndR3iVSJ+qpL1pJs3L0Z+HZb\nmeeD7t4ws18GtpnZD9z96U4rBweG1QDz589PMCyRYlNfdclakt05V9FW5nH3RvDfw8ATwNVRK7v7\nRncfd/fx0dHRBMMSKTb1VZesJZL4zexdwIeA/x6adp6ZvbP1Grge+F4S3ydSJeqrLlmL051zM3At\nMM/MDgH3AMMA7v5AsNjvAt90978PrforwBNm1vqev3T3v04udJFqUF91yZq5e94xzDA+Pu6Tk5N5\nhyEiUhpmtjPuvVIaskFEpGaU+EVEakaJX0SkZpT4RURqRqNzSmI03oxIOSjxSyI03oxIeajUI4nQ\neDMi5aHEL4nQeDMi5aFSjyTiwjkjNDok+X7Gm1EbgUg2dMYviRh0vJlWG0Hj+BTOW20EE7saKUQr\nUm9K/JKIlUvG+MLvXcHYnBEMGJszwhd+74rYZ+xqIxDJjko9kphBno2aZxuBSkxSN0r8koleyTWJ\nNoLZxqVuqFI3KvUUwMSuBkvXb2fh2idZun575eracer3eY1JrxKT1JESf87q0KgZJ7kO2kYwW+qG\nKnWkUk/OuiXFqpQa4ibXQdoIZiuvEpNInnTGn7M6nHEW+Zmyeuxhvqpe5iwqJf6cFTkpJqXIyTWv\nEpPUo8xZVHGeubsJuAk47O7v7TD/WpoPWX8xmPS4u68L5i0H/hQYAh509/UJxV0Za5YtntarBIqT\nFJNS9GfK5lFiknqUOYsqTo3/y8D9wENdlvlf7n5TeIKZDQFfBD4CHAKeNbMt7v78LGOtpKInxaQo\nuUq7OpQ5i6pn4nf3p81swSw++2rggLsfBDCzh4EVgBJ/GyVFqSM1rOcnqV49v2Vme4BXgH/r7vuA\nMeDl0DKHgGsS+j6RSEneiVv3u3rT3P46lDmLKonE/xxwibu/bmY3AhPAon4/xMxWA6sB5s+fn0BY\nUkdJ3olb97t6097+upQ5i8jcvfdCzVLPX3Vq3O2w7EvAOM3kf6+7Lwum3w3g7l/o9Rnj4+M+OTnZ\nMy6RdkvXb+9YPhibM8K31344t88qo7pvf9mY2U53H4+z7MDdOc3sV83MgtdXB5/5KvAssMjMFprZ\nOcAqYMug3yfSTZINhnVvfKz79ldZnO6cm4FrgXlmdgi4BxgGcPcHgH8EfMLMTgFTwCpvXkacMrO7\ngK00u3NuCmr/IqlJssGw7o2Pdd/+Kut5xu/ut7n7Be4+7O4XuftfuPsDQdLH3e9398vd/Tfd/f3u\n/kxo3afc/dfd/dfc/T+kuSEikOzNYkW+8SwLdd/+KtNYPVIpSTYY1r3xse7bX2WxGnezpsZdEZH+\n9NO4qzN+kYKp+70Dkj4l/opR0ii3ut87INnQ6JwVotEOy09PBJMs6Iy/QjTaYW9FvyJKsu980bdV\n8qPEXyG64aa7MpRRkuo7X8Rt1YGoOFTqqZA6PNRlEGUooyTVd75o26oyZLEo8VeIbrjprgxXREk9\nEaxo21q0A1HdqdRTIbrhpruyDEGQxPMZiratRTsQ1Z0Sf8XooS7R6jT+e9G2tWgHorpTqUdqo04P\nVi/atqoMWSwaskFEMqFePenSkA0iUjgqQxaHSj0iIjWjM34pLJUGRNKhxC+JSyJhF/HOU5GqUOOu\nJKo9YQMY4DR7lrQfBKIOEnrQt0h/Em3cNbNNwE3AYXd/b4f5HwP+mOb/3z8HPuHue4J5LwXTTgOn\n4gYls5d3eaTTHZqtU4v2s/ZuZ/W64UckPXFKPV8G7gceipj/IvAhdz9mZjcAG4FrQvOvc/efDRSl\nxFKE8kivxBweLbTbbfxzzh3m2ImTM9Yf9IafvA6MeR+QRcJ6Jn53f9rMFnSZ/0zo7Q7gosHDktnI\nc1jmVmKLUzhsHRyiDhKN41MMn2Uzpg8P2axv+JnY1eDeLfs4PvXWwSSrA2MRDsgiYUl357wT+Ebo\nvQPfNLOdZrY64e+SNnmVR8IjL8bROmuPOnsfMuPkmzMPIeedc/asEmUrvnDSb8lioDANUCZFk1ji\nN7PraCb+Pw5N/qC7vw+4Afikmf12l/VXm9mkmU0eOXIkqbBqJa9hmTsltpb28/bwbfpRt/Gfjuhw\n8FqHxD1ofJD+gVHtFVI0iSR+M/sN4EFghbu/2pru7o3gv4eBJ4Croz7D3Te6+7i7j4+OjiYRVu3k\nNR5KVAIz4L6PXhk5XkzUeDJjCR/AeiXYtA+Mek6CFM3A/fjNbD7wOPD77v5CaPp5wFnu/vPg9fXA\nukG/T6LlNSxzt5EXe92mHzU/yZElo+ILf26aja9FGylTJE53zs3AtcA8MzsE3AMMA7j7A8BngV8C\n/quZwVvdNn8FeCKYdjbwl+7+1ylsg4TkMR5KnMTWT2JN+gDWKT6AuecOc8/NlwOk2viq5yRI0egG\nLklEt8Te6aaukeGhTIcJ7hafbhaTKtDonNJVGmWNblcaaXUz7fcqImqeGl+lbpT4ayaPPuVpJNYk\ntyOqDWDOucMsXb89tfLMIAdg3RAmg9CwzDWTR5/yNHq1JLkd1102OqPb6fCQ8fr/O0Xj+BTOWweW\niV2NWcccFr73od/PH2RdEVDir508yhppdDNNajsmdjV4bGdj2h3HBgyfNfMmsiQPkIMcuHRDmAxK\npZ6ayeOh12n0aklqO6IGlTtx8s2Oy8e9O7mXQQ5c3Ya6SLM0JdVRqcSvumdvefUpT7qbaVLb0e8V\ngtH8nQ26LYMcuKLWNd46MGk8IOmmMqWeutU9J3Y1WLp+OwvXPsnS9dtjb2fU3bJlSw5JbUdUop0z\nMjyj7g/Nq4EkSiqDlL86rdt65kGYyj8SpTL9+OvUF7sI/eKrotu+/NQjuzuuY8CL638nke9OqldP\nVAkqqVil+GrZj79OfbHzHH45LXmV6bq1P2zYuj/V9pBByl/t60ad+Gg8IOmkMok/j0bLvGR9kEs7\nKec1Xn37dt330SunfV+ZxtgpU6ySv8rU+PMamTIPWY72mEXbSR7dE+NsV5naQ8oUq+SvMmf8dRoI\nK8uzuyzKSnmU6eJuVx6D3omkrTKJH+rzP2mWB7ksknIeZbqqtQklVS5Tl+h6qFTir5OsDnKDJuU4\niSSP+nQeB5s0k2oSV2Z6NnB9VKbGL+kYpO0kbvtAHvXpNNqEut1bkXZbSRJXMBoKoj50xi9dDVJW\n6ucsNOsy3Wy3K+qsvdfZclJn5Pdu2XfmofGtB8msXDKWyBVM1cpfEk2JX3qabVIueiLpd7u6Jfde\niX3QfTGxq8Gar+2ZNnDcsRMnWfPoHiCZclmdukTXnRK/pKbsiaT97P7v3zgVmdx7JfZB98WGrftn\njBYKcPK0s2Hr/jN3pw/ShqB7AeojVuI3s03ATcBhd39vh/kG/ClwI3ACuMPdnwvm3Q78+2DRz7v7\nV5IIXJKhh4x31unsPkq3YRNaiX3QfdHtyqA1b9ByWdLlLymuuGf8XwbuBx6KmH8DsCj4uwb4c+Aa\nMzuf5sPZx2mOIbXTzLa4+7FBgpZkpN2Lo8z3VnQq3URpbVe3xD7ovug2Hk+SV1BJlr/K8O9cV7ES\nv7s/bWYLuiyyAnjImyO+7TCzOWZ2AXAtsM3djwKY2TZgObB5kKAlGVncnFXWeyvi1t5byT1OYh9k\nX6xZtnhGjR+aTwrL8wqqiuNG1UFSNf4x4OXQ+0PBtKjpUgBFb3zNU9QZ9txzhzn3nLM7Jvc0D3Kt\nz43q1ZMX/YbKqTCNu2a2GlgNMH/+/JyjqYeyN76mKap0k2ei7XVgSbPWHvXZ+g2VU1I3cDWAi0Pv\nLwqmRU2fwd03uvu4u4+Pjo4mFJZ0U5WB7Wb7UJpuyjboWZo3iHX77Kr8huomqTP+LcBdZvYwzcbd\n19z9p2a2FfiPZjY3WO564O6EvlMGVObG15YkGxc7ndWW5SE+adbau312Et1IJXtxu3NuptlQO8/M\nDtHsqTMM4O4PAE/R7Mp5gGZ3zo8H846a2eeAZ4OPWtdq6JViKGvja8u9W/Z1HWYgbkIqe++UNGvt\nvT677L+hOorbq+e2HvMd+GTEvE3Apv5DE+luYlfjTENnu1bijpvIi9Q7ZTa1+jRr7arjV48GaZPY\n0qilD6Lb4GFDZn0NOBbnjDmL7Z9trT7NWrvq+NVTmF49UmxFLIV0K2Oc9pnDG3Rbp9dZbVbb32uE\nzKgrgTTba6rQFiTTmUf8D5Kn8fFxn5yczDsMCYl6mPfYnJHYDaBJdzeMiqnV176feNsTOzTPals9\neQbd/rjbvnDtk0T9HzkyPDQtPgM+9v75fH7lFT2/X6rPzHa6+3icZVXqkViSGF0y6e6GUSWIe26+\nvO/yRK/um4Nsfz/bHlU371S6cuCrO37S8XOKVpaTYlGpR2JJYnTJpBtP45Qg+rnC6NY7ZZDt72fb\no24cixo3yIPPD39OEctyUixK/BJLWqNLdpreT0moW7LuNq/fstMg29/PtkcdzDZs3R85SFv75xSp\nh5IUkxK/xJLW6JLtZ8xZnK12+o5PPbKbP/n6vsghGQbZ/n6vFqIOWJ9+ZHfH+n/752j8HOlFiV9i\nG3R0yThnzIOercY5k48acvnYiZNdDzLtyb/V06ZXXEk8l2DlkjEmf3yUr+74ybTk3+lz1O9eelHj\nrmQi7tg3WTSidvusbn39Z9tAndS4P59feQX3ffTKnp+jfvfSi874ZYa0RnmMc8WQRSNqt4eaQPTT\ntga5GklqWIM4n6N+99KLEr9Mk3ePkCwaUTt9R5jR3A9JXo1kTePnSDcq9cg0ve4cTdsgZZGoq4L2\n6a3vmDMy3HH5VhfJ2X6+SNHpjF+mKcJZ7WzPVvu5Wmh9x4K1T3b8rE7bW+aHx4uE6YxfpinzWe1s\nrhbG+tjesj2cRSSKzvhlmrKf1fZ7tdDv9ha5dp7moxerFJMo8UubPHuE5JEkZru9WcTaz3fk3Shf\nlpikSaNzSiH0Gh2zSLKItd/vSGL01KQVMaYq0+icUjp59ybqRxax9vsdaTfKz2a0zyJ0FJDOYiV+\nM1tuZvvN7ICZre0w/z4z2x38vWBmx0PzTofmbUkyeKmOMiWJqJgax6cSGwY57v5oJeSo6/YkGuVn\ne8dymTsKVF3PxG9mQ8AXgRuA9wC3mdl7wsu4+6fd/Up3vxL4L8DjodlTrXnufkuCsUuFlClJdIsp\nqWcNxNkf4YTcycjwENddNjrwuPyzvcLR0BHFFeeM/2rggLsfdPdfAA8DK7osfxuwOYngpD7KlCQ6\nxdpu0NJPnP0RNdgcNOvot141xmM7GwM//Ga2V2Pq/lpccXr1jAEvh94fAq7ptKCZXQIsBLaHJr/d\nzCaBU8B6d5+IWHc1sBpg/vz5McKSKkmjN1GaYw6FY40qswxSporaH9BsNO32vQZ8e+2HWbp+eyLj\n8g8yflKRu7/WWdLdOVcBj7p7+Nd2ibs3zOxSYLuZ7XX3H7Wv6O4bgY3Q7NWTcFxSAkkmibS7EoZj\njeq9MmiZqn1/dOrp00nre5NqNyn7vR0yU5xSTwO4OPT+omBaJ6toK/O4eyP470Hgb4AlfUcpfdHz\nVrPtJZRVmapbaafT9ybVbqKSTfXEOeN/FlhkZgtpJvxVwD9uX8jMLgPmAv8nNG0ucMLd3zCzecBS\n4D8lEbh0pptmmmZztjvb0lBWN711i91gxvcmeaaukk219Ez87n7KzO4CtgJDwCZ332dm64BJd291\n0VwFPOzT7wh7N/AlM3uT5tXFend/PtlNkDA9b7Wp37r0oAfMLBJj1DZF3RClcfklSqwav7s/BTzV\nNu2zbe/v7bDeM8AVA8QnfSpTf/g09Xu2W4YD5mzO4HWmLp1orJ6K0fNWm/o92y3DAbP13N3N33mZ\n0+4MmXHrVcVJ7GUYkK0MMWZBib+Euv141QPjLf2c7ZbhgDmxq8FjOxucDqqpp915bGeD8UvOzz15\nlaFtqQwxZkVj9ZRMr9vn1QNjdspwA1mRxzMqcmwtZYgxKzrjL5k4tWjVdftXhobQIpejihxbSxli\nzIoSf8noxxvfxK4Gf/L1fRw7cRKAOSPD3HvL5ZHJvOgHzNmUo7KqaZehVFaGGLOiUk/JlGkwszxN\n7Gqw5tE9Z5I+wPGpk6z52p7S3tDWbzlqtqNqZhFbHsoQY1aU+EtGP954Nmzdz8nTM0f+OPmml7am\n22/7TZY17TK0LZUhxqyo1FMyZahFF0G30lcaZbGsSir9lKOyLgsWvVQG5YgxC0r8JaQfb29R9dzW\nvCQVtZugatoSRaUeqaQ1yxYzPGQzpg+fZZkMnlaEboIqC0oUnfFLJbXOtPvp1TNbRe1ppbKgRFHi\nl8rKqiRW5JKKyoLSiUo9IgPqVFIx4LrLRvMJSKQHJX6RAa1cMsatV40RblFw4LGdjdLeMyDVpsQv\nkoBv/eDIjGfgFqGBV6QT1fhFElDUBl7QUMQyk874RRJQ1KE0shy2QcpDiV8kAUXtM1/UewwkX7ES\nv5ktN7P9ZnbAzNZ2mH+HmR0xs93B3x+E5t1uZj8M/m5PMniRokhqHJiJXQ2Wrt/OwrVPsnT99oHP\nzItcgpL89Kzxm9kQ8EXgI8Ah4Fkz29LhoemPuPtdbeueD9wDjNPs6LAzWPdYItGLFMigfebTGPqh\nyPcYSH7inPFfDRxw94Pu/gvgYWBFzM9fBmxz96NBst8GLJ9dqCLVlkZZpqglKMlXnMQ/Brwcen8o\nmNbuVjP7rpk9amYX97kuZrbazCbNbPLIkSMxwhKpljTKMhqKWDpJqjvn14HN7v6Gmf0h8BXgw/18\ngLtvBDYCjI+PzxxIXaTi0irLaNgGaRfnjL8BXBx6f1Ew7Qx3f9Xd3wjePghcFXddEWlSWUayEifx\nPwssMrOFZnYOsArYEl7AzC4Ivb0F+H7weitwvZnNNbO5wPXBNBFpo7KMZKVnqcfdT5nZXTQT9hCw\nyd33mdk6YNLdtwD/0sxuAU4BR4E7gnWPmtnnaB48ANa5+9EUtkOkEvIsy+gO3/ow9+KV08fHx31y\ncjLvMERqo70rKTTLTLriKA8z2+nu43GW1Z27IqI7fGtGiV9EdIdvzSjxi0hhB5mTdCjxi4i6ktaM\nxuMXET2YvWaU+EUE0B2+daJSj4hIzSjxi4jUjBK/iEjNKPGLiNSMEr+ISM0o8YuI1Iy6c0qlaIRJ\nkd6U+KUy0nhYuUgVqdQjlaERJkXiUeKXytAIkyLxKPFLZWiESZF4lPilMjTCpEg8sRK/mS03s/1m\ndsDM1naY/6/N7Hkz+66Z/U8zuyQ077SZ7Q7+trSvK5IUPaxcJJ6ez9w1syHgBeAjwCGaD06/zd2f\nDy1zHfAddz9hZp8ArnX3jwbzXnf3d/QTlJ65KyLSn6SfuXs1cMDdD7r7L4CHgRXhBdz9W+5+Ini7\nA7ion4BFRCQ7cRL/GPBy6P2hYFqUO4FvhN6/3cwmzWyHma2MWsnMVgfLTR45ciRGWCIiMhuJ3sBl\nZv8EGAc+FJp8ibs3zOxSYLuZ7XX3H7Wv6+4bgY3QLPUkGZeIiLwlzhl/A7g49P6iYNo0ZvYPgc8A\nt7j7G63p7t4I/nsQ+BtgyQDxiojIgOIk/meBRWa20MzOAVYB03rnmNkS4Es0k/7h0PS5Zva24PU8\nYCnwPCIikpuepR53P2VmdwFbgSFgk7vvM7N1wKS7bwE2AO8AvmZmAD9x91uAdwNfMrM3aR5k1od7\nA4mISPZ6dufMg7pzioj0p5/unIVM/GZ2BPhxH6vMA36WUjhJKkOcZYgRFGfSFGey8ojzEncfjbNg\nIRN/v8xsMu6RLk9liLMMMYLiTJriTFbR49RYPSIiNaPELyJSM1VJ/BvzDiCmMsRZhhhBcSZNcSar\n0HFWosYvIiLxVeWMX0REYip04o/xHIC3mdkjwfzvmNmC0Ly7g+n7zWxZznEW4nkFMeK8w8yOhOL5\ng9C8283sh8Hf7TnHeV8oxhfM7HhoXib708w2mdlhM/texHwzsz8LtuG7Zva+0Lws92WvOD8WxLfX\nzJ4xs98MzXspmL7bzFK9sSZGnNea2Wuhf9vPhuZ1/b1kHOeaUIzfC36P5wfzMtufPbl7If9o3iX8\nI+BS4BxgD/CetmX+OfBA8HoV8Ejw+j3B8m8DFgafM5RjnNcB5wavP9GKM3j/eoH25x3A/R3WPR84\nGPx3bvB6bl5xti3/L2jeTZ71/vxt4H3A9yLm30hzlFoD3k/zeRWZ7suYcX6g9f3ADa04g/cvAfMK\nsj+vBf5q0N9L2nG2LXszsD2P/dnrr8hn/D2fAxC8/0rw+lHgH1hzzIgVwMPu/oa7vwgcCD4vlzi9\nGM8riLM/oywDtrn7UXc/BmwDlhckztuAzSnFEsndnwaOdllkBfCQN+0A5pjZBWS7L3vG6e7PBHFA\njs/SiLE/owzyu+5bn3Hm8tuMo8iJP85zAM4s4+6ngNeAX4q5bpZxhs3qeQUJiBvnrcGl/6Nm1hqV\ntZD7MyiZLQS2hyZntT97idqOLPdlv9p/mw5808x2mtnqnGIK+y0z22Nm3zCzy4NphdyfZnYuzQP6\nY6HJhdmfiY7HL93ZAM8ryMjXgc3u/oaZ/SHNq6kP5xRLHKuAR939dGhakfZnaVjz8al3Ah8MTf5g\nsC9/GdhmZj8Iznjz8BzNf9vXzexGYAJYlFMscdwMfNvdw1cHhdmfRT7jj/McgDPLmNnZwLuAV2Ou\nm2WcRXheQc843f3VUGwPAlfFXTfLOENW0XYpneH+7CVqO7Lcl7GY2W/Q/Pde4e6vtqaH9uVh4AnS\nK5f25O5/5+6vB6+fAoatOdR74fZnoNtvM/f9mXsjQ9QfzauRgzQv5VuNNpe3LfNJpjfu/rfg9eVM\nb9w9SHqNu3HiXEKzAWpR2/S5wNuC1/OAH5JSw1TMOC8Ivf5dYEfw+nzgxSDeucHr8/OKM1juMpqN\nZZbH/gy+YwHRjZG/w/TG3b/Nel/GjHM+zTawD7RNPw94Z+j1M8DyHOP81da/Nc2E+ZNg38b6vWQV\nZzD/XTTbAc7Lc3923Ya8vjjmDr4ReCFImp8Jpq2jedYM8Hbga8EP92+BS0PrfiZYbz9wQ85x/g/g\n/wK7g78twfQPAHuDH+te4M6c4/wCsC+I51vAZaF1/1mwnw8AH88zzuD9vTSf7xBeL7P9SfNs7qfA\nSZp15TuBPwL+KJhvwBeDbdgLjOe0L3vF+SBwLPTbnAymXxrsxz3Bb+IzOcd5V+i3uYPQgarT7yWv\nOINl7qDZuSS8Xqb7s9ef7twVEamZItf4RUQkBUr8IiI1o8QvIlIzSvwiIjWjxC8iUjNK/CIiNaPE\nLyJSM0r8IiI18/8BRE2u+Cep26QAAAAASUVORK5CYII=\n", "text/plain": [ "
" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "import numpy as np\n", "\n", "\n", "from scipy.io import loadmat\n", "X = loadmat('dataNotebook1_Ex1.mat')['X']\n", "\n", "\n", "plt.scatter(X[:,0], X[:,1])\n", "plt.show()\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "5. Once you have you estimates for the parameters of the Gaussian distribution, plot the level lines of that distribution onb top of the points by using the lines below. " ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": true }, "outputs": [], "source": [ "import matplotlib.pyplot as plt\n", "from matplotlib import cm\n", "x1 = np.linspace(0, 1.85, 100)\n", "x2 = np.linspace(0.25, 2.5, 100)\n", "\n", "xx1, xx2 = np.meshgrid(x1, x2)\n", "\n", "from scipy.stats import multivariate_normal\n", "\n", "xmesh = np.vstack((xx1.flatten(), xx2.flatten())).T\n", "\n", "mu1 = # complete with your value\n", "mu2 = # complete with your value\n", "sigma1 = # complete with your value\n", "sigma2 = # complete with your value\n", "sigma = np.zeros((2,2))\n", "sigma[0,0] = sigma1**2\n", "sigma[1,1] = sigma2**2\n", "\n", "y = multivariate_normal.pdf(xmesh, mean=[mu1, mu2], cov=sigma)\n", "\n", "\n", "\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "#### What you should obtain:" ] }, { "cell_type": "code", "execution_count": 77, "metadata": {}, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAX4AAAD8CAYAAABw1c+bAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi40LCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcv7US4rQAAIABJREFUeJztvX9wXdd13/tZAC4BggBIkCAFmgJ/\niKICSrYl2nyiDGn8i7YsqbLVTDpTOalrx86wdeM8py/1vCZ5E3eU91pPM5MmGbtxNArH8Uwip/Wv\nsq1siaKVqhIsSpSo34QogL9AChAEECQAggBxgf3+OOdcnHvuOfeec+65v3DXZ4YjYJ9f+x5dfNfa\na6+9thhjUBRFUeqHhkp3QFEURSkvKvyKoih1hgq/oihKnaHCryiKUmeo8CuKotQZKvyKoih1RkHh\nF5EeEXlKRN4UkTdE5Os+53xcRC6LyMv2vz9yHbtHRN4SkUER+bdJfwBFURQlGk0hzkkDv2eMeUlE\n2oEXReSwMeZNz3n/2xhzv7tBRBqB7wCfBs4DL4jIIZ9rFUVRlDJR0OM3xowYY16yf54GTgBbQt7/\ndmDQGHPKGHMN+AHwQNzOKoqiKMUTxuPPICLbgT3AUZ/DHxGRV4B3gH9jjHkDy0AMu845D+wLuPcB\n4ABA86qWD79v0/VRuqYoilLXnD4/OG6M2Rjm3NDCLyJtwI+A3zXGTHkOvwRsM8bMiMh9wE+BXWHv\nDWCMeRh4GOCGnl3moa//WZTLFUVR6povfOP+s2HPDZXVIyIpLNH/W2PMj73HjTFTxpgZ++fHgJSI\ndAEXgB7XqdfbbYqiKEqFCJPVI8BfAyeMMX8acE63fR4icrt93wngBWCXiOwQkVXAg8ChpDqvKIqi\nRCdMqOdO4AvAayLyst32B8BWAGPMd4F/AnxVRNLAVeBBY5X9TIvI14DHgUbgoB37VxRFUSpEQeE3\nxjwDSIFzvg18O+DYY8BjsXqnKIqiJI6u3FUURakzVPgVRVHqDBV+RVGUOkOFX1EUpc5Q4VcURakz\nVPgVRVHqDBV+RVGUOkOFX1EUpc5Q4VcURakzVPgVRVHqDBV+RVGUOkOFX1EUpc5Q4VcURakzVPgV\nRVHqDBV+RVGUOiPMDlw9IvKUiLwpIm+IyNd9zvkNEXlVRF4TkX4RudV17Izd/rKIHEv6AyiKoijR\nCLMDVxr4PWPMSyLSDrwoIoeNMW+6zjkNfMwYMyki92Jtmr7PdfwTxpjx5LqtKIqixCXMDlwjwIj9\n87SInAC2AG+6zul3XfIc1qbqiqIoShUSKcYvItuBPcDRPKd9BfiZ63cDPCEiL4rIgagdVBRFUZIl\nTKgHABFpA34E/K4xZirgnE9gCf9drua7jDEXRGQTcFhEBowxT/tcewA4ALBh3cYIH0FRFEWJQiiP\nX0RSWKL/t8aYHwec80HgEeABY8yE026MuWD/dwz4CXC73/XGmIeNMXuNMXs72tZG+xSKoihKaMJk\n9Qjw18AJY8yfBpyzFfgx8AVjzElX+xp7QhgRWQPcDbyeRMcVRVGUeIQJ9dwJfAF4TURettv+ANgK\nYIz5LvBHwAbgP1t2grQxZi9wHfATu60J+DtjzM8T/QSKoihKJMJk9TwDSIFzfgv4LZ/2U8CtuVco\niqIolUJX7iqKotQZKvyKoih1hgq/oihKnaHCryiKUmeo8CuKotQZoVfuKoqiKJVnYGSK/qEJpucW\naG9J0bdzA72bOyLdQ4VfURSlRhgYmeLIiTEWlpYAmJpb4MiJscj30VCPoihKjdA/NJERfYeFpSX6\nhyYCrvBHhV9RFKVGmJ5biNQehAq/oihKjdDekorUHoQKv6IoSo3Qt3MDqYZs2U41NNC3c0Ok++jk\nrqIoSo3gZO9oVo+iKEod0bu5I7LQe9FQj6IoSp2hwq8oilJnqPAriqLUGWG2XuwRkadE5E0ReUNE\nvu5zjojIX4jIoIi8KiIfch37ooi8bf/7YtIfQFEURYlGmMndNPB7xpiX7P1zXxSRw8aYN13n3Avs\nsv/tA/4S2Cci64FvAnsBY197yBgzmeinUBRFUUJT0OM3xowYY16yf54GTgBbPKc9AHzfWDwHrBOR\nzcBngMPGmIu22B8G7kn0EyiKoiiRiBTjF5HtwB7gqOfQFmDY9ft5uy2o3e/eB0TkmIgcm5q5HKVb\niqIoSgRCC7+ItAE/An7XGDOVdEeMMQ8bY/YaY/Z2tK1N+vaKoiiKTSjhF5EUluj/rTHmxz6nXAB6\nXL9fb7cFtSuKoigVIkxWjwB/DZwwxvxpwGmHgH9uZ/fcAVw2xowAjwN3i0iniHQCd9ttiqIoSoUI\nk9VzJ/AF4DURedlu+wNgK4Ax5rvAY8B9wCAwC/ymfeyiiPwx8IJ93UPGmIvJdV9RFEWJSkHhN8Y8\nA0iBcwzw2wHHDgIHY/VOURRFSRxduasoilJnqPAriqLUGSr8iqIodYYKv6IoSp2hwq8oilJnqPAr\niqLUGSr8iqIodYbuuasoSk0zMDJV9Obj9YYKv6IoNcvAyBRHToyxsLQEwNTcAkdOjAGo+OdBQz2K\notQs/UMTGdF3WFhaon9ookI9qg1U+BVFqVmm5xYitSsWKvyKotQs7S2pSO2KhQq/oig1S9/ODaQa\nsmUs1dBA384NFepRbaCTu4qi1CzOBK5m9URDhV9RlJqmd3OHCn1ECgq/iBwE7gfGjDHv9zn+DeA3\nXPfbDWy0N2E5A0wDi0DaGLM3qY4rykpEc9KVchAmxv894J6gg8aYPzHG3GaMuQ34feB/eXbZ+oR9\nXEVfUfLg5KRPzS1gWM5JHxiZqnTXlBVGQeE3xjwNhN0u8fPAo0X1SFHqFM1JV8pFYlk9ItKKNTL4\nkavZAE+IyIsicqDA9QdE5JiIHJuauZxUtxSlZtCcdKVcJJnO+VngWU+Y5y5jzIeAe4HfFpGPBl1s\njHnYGLPXGLO3o21tgt1SlNpAc9KVcpGk8D+IJ8xjjLlg/3cM+Alwe4LPU5QVheakK+UiEeEXkbXA\nx4D/5mpbIyLtzs/A3cDrSTxPUVYivZs72L97Ex0tKQToaEmxf/cmzepREidMOuejwMeBLhE5D3wT\nSAEYY75rn/arwBPGmCuuS68DfiIiznP+zhjz8+S6rigrD81JV8pBQeE3xnw+xDnfw0r7dLedAm6N\n2zFFURSlNGitHkVRlDpDhV9RFKXO0Fo9Smy0vICi1CYq/EosdMs7RaldVPiVWOQrL1Drwn/85Gil\nu1CQPTd1V7oLSg2jwq/EIqnyAuUOF4UV9Q/23VSyPhTLq/0nQ30ONQ5KECr8SizaW1JM+Yh8lPIC\npQwX5RPGahb1MITpfz7joAZBUeFXYtG3c0OWaEP08gJJhItWssAXQ77Pfrz/pG+7GoT6QYVfiUUS\nW97FCRf5CX0xAl+PmUl+78tvhKCGYOWiwq/EJl95gTCCGiZc5BWjJL14zUxaxu+9+o0M1BisDFT4\ny0C9eZVhBdUvXNS6mKZndXNG8EsZrlnJmUlJ4H333lGBGoHaRYW/xNSjVxlWUHs3dzD8ziTDk1e5\nll5kVVMj+z55c9nei258Eg2vIfCOCNQQ1A4q/CWmHr3KQoLq9hq72pv55Gc+UJZ+eUkiM6mecRuC\nUowG6m2kXE5U+EtMPXqVfoLanF6guamxLCGcsCSRmaRYJG0E6nGkXE5U+EtMPXqVjqA2XJvPtDWK\nsO9jvVX1R5tEZpKSSxJGoB5HyuUkzEYsB4H7gTFjzPt9jn8ca+et03bTj40xD9nH7gH+HGgEHjHG\nfCuhftcM9ehVXp2e5aa1KYYn01zqWlfVghp145OjZy+UsDfZ7Nu2pWzPKhVxjUA9jpTLSRiP/3vA\nt4Hv5znnfxtj7nc3iEgj8B3g08B54AUROWSMeTNmX2uSevEq3X/Q1RDGiUoUQb/1tq0l7InFKy+f\nC92nWjEQQUbAzwDU40i5nITZgetpEdke4963A4P2TlyIyA+AB4C6En5Y2dvpVZvgF5oQzCemfoI+\nMDpN/+A403Np2lua6LuxqyT9DtMXP/IZiFIZhCQmXZ3vSpABqMeRcjlJKsb/ERF5BXgH+DfGmDeA\nLcCw65zzwL6gG4jIAeAAwIZ1GxPqllIKqk3sHbwTgtPpaQ6fnGHo4hq62poz54UV1YHRaY6ceJf0\norHuN5fmyIl3Aejtbk+49/HI91mOvnwup61YY5D0pKv7++NOD3WMwEofKVeKJIT/JWCbMWZGRO4D\nfgrsinoTY8zDwMMAN/TsMgn0S0mYahV8sDz548OTpBuWEHtfubn2NgDO0sT+GOGZ/sHxjOg7pBcN\n/YPjVSP8+fAzCsUag1JOuvqNAr58146i7qn4U7TwG2OmXD8/JiL/WUS6gAtAj+vU6+02pcaoRsH3\nC29cXt3qe+70XDrWM4Kui3u/asBrDPxCRfkMQTkmXQuFgZTiKVr4RaQbeNcYY0Tkdqx9fCeAS8Au\nEdmBJfgPAr9e7POU8lFtgu8VKK+IvfjMaV9Rbm+J9zVvb2lK9H7VSCFD4DUC5Zx0VQNQOsKkcz4K\nfBzoEpHzwDeBFIAx5rvAPwG+KiJp4CrwoDHGAGkR+RrwOFY650E79q9UOdUk+G4RKhSb77uxKysm\nD9DUKLEnZJO+Xy3gfcfe0FAlJl3VACSPWBpdXdzQs8s89PU/q3Q36pJqWFkbRey9+GXhFBOPz3e/\n/tHzse8bRF/39YnfMylesY3A+Mw8w5OzzM+tqsik66v2JLCKfzZf+Mb9Lxpj9oY5d+WMWZWiqLTg\nFyP2bnq724ueePUKeu+NKexBLhe5TP/oZQD29vZ4Ly2a/oHlRDhHYK+ll1jV1EBPZ2smO6kSBsL9\n/+UV10ig3Jk2znf0uBqA2Kjw1yhJFbCqZFinUMy+1OTz2Esh6mFwnjswMsVzI2OkpQVSsAAMTsP2\nLWuZuXw5sO/lMgju/1fucFA5F5N9sO8mDf/ERIW/Bkkql7pSXn5S3n0UgoSyUgJfiGeHJkgvZqdN\npheXeHZogq8EpDgeGxj2/ZylNgbO/0P3xHC5DIA3/q/iHw4V/hqk2FzqSnn55RR8PwFsW7uWZ4cm\nmJlboK0lxZ1VvAp0JiA9Mqgd/I2YYwzcYaO1ix1Fz334UQ0GQMM/4VDhr0GKyaWuhJfviEApxd5P\n6N1CODAyxZMDYxkvenpugScHqrfMb1tLyvf/Z1vEtMm9vT1ZYaMUs1xunOLxM9OcnLFWNCc9IvAL\nAxUyAEmFLt3hHxX/YFT4a5C4udTlFv1SC75X7POFbfKFTqpR+O/cuSHLUAE0NTbEGqW4P/tCanmR\n24W5Rrrast9jqYxAPgNQijIQGvvPjwp/DRI1l3olCX4UsXcTJ3SStx+TZ2JdF4a+zu0ZwfOGpuII\nYb7Pvrd3eb7AO0eQpBHIZwBKUQZCY//5UeGvQaKUei6n6Jcyhu8WpDgTslFDJ4WE/fbtpZsU7j9j\nP7sFdt8iwCoALnKR/smL9HVuj3S/sJ/d/V7dRqDUBqCUZSA09OOPCn+NEqbUc7lEv1SCX6zYu8kX\nOvET+VIKeyEKPTtjGDwEGYQ4YSP3+3avLUjKCNx629bMJHBzyzXm5lblnJNUGQgV/1xU+Fcox0+O\nltXLr1bBd3CM5JPvneJaepFVTY1s7VzNxZaLFRX5OPj19/kzwzkGzDEExYaNnP8HSY8CnO/M+Mw8\np8avsLiwXDo76TIQmvWTjZZsWGHUspdfCsGH3LBNrQl9HJ4/M5zTFjVEFMSxEowAjjzzdibdtK2p\nvaRlIFZqyQct2VCnlFv0q1Xw61HovXg/s3dEUIwRKMUIYP9d1hYeTimIUmZaaehHPf4VQyVFP25h\nND/BHxiZih2ScAtbpcR+YPZE6HN7W3eXsCfBuEcDSYwCkh4BvFKmEhArzfNXj7/OKIfoB3n5cbYn\nPDQ4lBnWt7SvzUwyxllkVS6xDyvod3SHHwU9NxrunkkbCPd7ck8UxzUCSY8AvCuASyX+9ez5q8df\nozgrHa/NzLKqqZF9H+st2fA4X2jnYMDmJ4Cv939ocIjT41eYb1qdaWtqbOBTvZt41k5Pzb1PKqc+\nTSkFP0jko4h6Ujw3mrtVIiRvDJIcBTgjgCS9f/X8C5Ooxy8iB4H7gTFjzPt9jv8G8H8DAkwDXzXG\nvGIfO2O3LQLpsJ1S8uOsdGy4Ng/AZNe6olY65qNQPD/fNoRu7/8iVinj4cnZLNGH5RW0YRZZFSP4\nfmUBWJu7hWOpBH5gZJr+IVdIbGcXvZvzh8SC+uI3WijGGPiNAoodAfQnYADcaZ/q+SdHmFDP94Bv\nA98POH4a+JgxZlJE7sXaMH2f6/gnjDHjRfVSycJZ6dgMzHZbYZKkNrx2k0/0nbh+IdKLhiPDZ9jT\n08ne3h6ePX/V97yZuQWaU43MLSzmHGtrSWUEvxixd1jXbRnJFybfYUfjGu6/cVeke8ZhYGSaIwOe\nkNiAHRIrIP5+eA3Cc6PnskYqSRiBJAyAE/6pFfGvFwoKvzHmaRHZnud4v+vX57A2VVdKyPTcAs3p\nhYzou9uTopDoe7ck9GNpzSwA19LLXmDQKtLmVCML6aWcdrNhho1da7h9+42RP8PAyBRH7DkDR+wz\nzLaxBIylmyD6rSPTPzSe877Si4b+ofFYwu/FbQiSMgJJGAC3+EN879+94reU4n+8/2RdeP0NCd/v\nK8DPXL8b4AkReVFEDuS7UEQOiMgxETk2NXM54W6tLNrJFUhIbqXj0bMXuPW2rYHhnf7BXBHz4oj+\nQqqVlva1mfY7d26gqTH7a+f8vuiab1paP83S+mkaRbjv/fGUuX9ograNo8uiP9u2/M8mX6gqSYKe\nU4rn39G9NfMPrDkL518cMgZg8kysGkV7e3uWwz9Fbld5621bczbwSRp32fKVSmJZPSLyCSzhv8vV\nfJcx5oKIbAIOi8iAMeZpv+uNMQ9jhYm4oWdX9c04VwnHT47S07maE6nVUIINrx3Rz0c+sWpqFK61\nXAEs0feWBghaRfr4G8t/bEvrpwEwi634RH4K4ghc47qJLJH3o70l/p/A1MLR0Ofu2THJvM+Iprmp\ngamFi5nfO1L7cs4phqCRQNRRgCP+znqAYr3/YkM/pfL86yXen4jwi8gHgUeAe40xE067MeaC/d8x\nEfkJcDvgK/xKeD75mQ/wvoTql7sJ60m1tzT5in97SxNd3UsMTzYwKy20B+Th+9UZenZogsutlgCa\nxVbXPcOPYtwe7R3dW3lzcJFp8hupvp1dBSdd8wn8ns6dofq2et4Kjy24RkqpRuEju6+jt9N61vHJ\nocBnJWEQHCOQhAGIG/6pJfFfyYRK57Rj/P8jIKtnK/AL4J+74/0isgZoMMZM2z8fBh4yxvy80PM0\nndOfUtbfibIa1y/G39Qo9PQIXW3Nmc0/oizEOjT8llWvJZ2d5rm/d1NBg+YV/Ey7Z0LVjSPwQM45\n7982yA1da9jQtlw7JqzA5+1nzIVuxyeHctqSMATuVNE48wBOCmgc7z+JlM9Sp3q+WmPx/ijpnAWF\nX0QeBT4OdAHvAt8EUgDGmO+KyCPArwFn7UvSxpi9InID8BO7rQn4O2PM/xemUyr8ucRZpBV2V6M4\nJRi8ItbVvZQl+n7VID8VIOJO3LijeW3kUYwj+kFpj4W8eWcdws09b2ddt+rqRr4csLdtpfEagmKN\nQDEGYCWLf63l9ycq/JVAhT+XqN6+d1cjsOYB9u/OFt8k6u44E3bOBN5fP3M61EKsuCmaAM+Mvczw\nxatcm1odOifey9TCUY6eykQmmZvszPwswNc/Vfo0T4g/EgDLCEzMzDN8cZb59BLD7+2O9S5g2QCU\n0/uvBfFficKfdFaPUgLiZBnk29XIS5KiD+F2uypW9E+PX+HalBUWcnLiB0amQ10/tXA0E0tfdXUj\nc5OdWaIPVihoYHSag8+c5s+ffJuDz5xmYDTc/aPghM2m5tIYYMpe9Bb2WavnN/HW4Couv2dlTvVs\nPMGFmWdCvws33iygKLgzf6KQRLZPKfdy/mDfTSsyy0eFv0aIGtsPs6tRsWlxfqIPwbtaOe2xF2PZ\ngjQ22sTSzJqsY05OfD7cgr+ncyd7OneyvWuN77nrWlNFCXJY+gfHsyZ8ARYWTajFcd7rHQO2tGQY\nm302UtaRgzcNNAqVFv9Sp3muJFT4q5y4E7pB2TBOe7EhniDRh+BcffduV3FEHyxhipoT7yf4DmfG\nr/hec37yqq8g/6+3xnzP97Im/UTgvzB9Dpvf73fe3GQnU++tZWJmnqeHfs7htx6zRiwRRgG1KP5Q\nvDPjx0r0+qu+OmfYCUolmzAbshc7RA6qnZ9vx6f+yeg7XnkncPOlk3pxC74fQQK7FDD1dXVhiYHR\naXq723NE3M22NR8IPHb2yvJ1996Qm99/5Nze0OsL2luamPL5DC2pBt4aXMXCYoqWzkl6Np7gyIDl\nQISN/9/RvTWT+hkl7n/79h7fjWAK4aR6xsUp7aAUpqqF3ztBOTW3ULJiZNXI4RfPMjx5ladmT0Y2\nevk2ZE8qxFPo+d6+9k+eKVr0Afp2duWkYDo5+Q6FBN8hSDgbJFf89289Zh2bbWBNujOvuOfDfd3V\nzuz8/s3XXeDu7S+yo2sNa9JWptGVprsD79V3Y5fv+gADWSEggJved5Kx2UF6uSd0X4sR//4z8RZ6\nFZvjX4qaPiutnENVC3++CcqVLvwDI1OcHr/C9HXrgXhGL9+G7KXy9oOIs9Q/KFXT8ViD0jTDij4E\nC+fuzR28ev5yRuwBRt61xESAf7w7mYwfJ3vHyeq5cnmbldVzndV+9sprmZGFnwHwXu9kBT3+em5o\nYm6yEzonmVo4GikFNK74A5FX+arXXx6qWvjDTFCuVPrfeCerdg0kY/SS8Pbjir7X288XxiuUn9+7\nud03ZBFW9N0plC2pBhobhPmFpYxwfrjrl2xOXSS9uCz4DsWUefCjt7s9MH3TPTpwh4jcRsDv+v7B\ncd+RzKqrG4Glsoi/E/KJI/79A8NV5/UDK6aUQ1VP7haaoFzJXEsv5lTfhGSMXj5vv1QpjH6if2Rg\nLPN5pucWODIwxsDIVEHRDyKK6Lszdq4uLLG4ZPjM+7v5nTve5sNdv2Tbmg9wy/oPMT6eLT6pRqHv\nxi7/G5eYbWs+kDEE+eYXwBrJpBolq83puzPBHTXrJ86EbzGb5MSd6C1VemeptzUtJ1Ut/H07N5Bq\nyO5iUsXIqp1VTY2+7aU0eo4gOhOe054UxmK8/Zz2oYmslb1gbcjirDMoleiDfwrlR7e8QMPsY1ni\n2tvdzv7d19HR0oQAHS1N7N99XejFVaXC6aNflpBDmL4XI/5RiZvlo5SGqhb+3s0d7N+9iY6WlP3l\nTeWsPF2p9HSuTtzoFQrz+JVbTkfIKQ/Cz+sLGrk0roseiooi+tazs0MgThz/7IXNOef2drfz5bt2\n8PVP7eLLd+2ouOi7Cev9FyJOvn+5vP64lDKvfyWkdlZ1jB/yT1CuVI6fHC1ZBc58w+B8OeVRht1O\ngbbLrRdZ1dRIR/PanH63B2zIsqqpMZJXGVX0rWcvZ/I4oj/y7hY6Eo7dB1FMiQYv29Z8IDMB7I77\nO6M3Z2TjLEADcrx+pyqoX8zfr9bRHZu3Bu4FnI845ZyLzfBJmpVSubOqPf56p3dzh+1t3mR5myU2\ngEGTlk57mOG3U6DNEfX5+eZM7N5Nn88ir/Xve4+e9dn78YYhauVMJ/69f+sxRt7dwsi7W8oWuy+2\nRIMf7tCPQ5QVwUHvz6lumhX6s0tj3NG9teRev4Z7SkfVe/xKMoQZ9jqpjTn58Td2ZTZLL8SzeWL3\nbsPlt85gR1e0/W/jhCjA8ni3tfwDwxcbEIjkdRfy1gsdzyfIYb1+5xlTc+nMeoOOlib6btzOmnbL\n84+zItjr9Zd6u8hyUMp9emsZFf46olC2Q1BOeG93O/2j4YTfKcTm7KLl4BfW8Ybx4mwNGKdO/pr0\nE6xpa+bD1+2FCGnphcInYcIrxZZo8D7DWWTmPOuOm+fpanuC9pZdvumcQaM6J+QTpk/u9qi5/eUM\n92hOfzAq/EoWvjnhEbJ53JupR9lJa2D2ROyMkTiEWXXr9d6vLS7l9dbDePNBK4XDrg3we4b7Wa+8\nvYH9e2YCF6YVCme5vf5CpTGc3P6wxCnlUOyCLsWfUDF+ETkoImMi8nrAcRGRvxCRQRF5VUQ+5Dr2\nRRF52/73xaQ6rkSnHGWGgwq0JZ2CO7VwNLa3Hwa/WPzcgv8m9+4YeL7jkD+/PgyFRgbO8Q93/TJy\nKqr3ffbt7KLJ01dvaYx8DIxM2d+3k3aRuKnCF9UItZ7ZE9bj/x7wbeD7AcfvBXbZ//YBfwnsE5H1\nWDt27QUM8KKIHDLGTBbTaSU64zPzDLji99MBWR7F4oRunnxvhvlFqrKwXhhvP59n7cXxgMN48/nC\naWGf5fcM93En0yffiuAwFCqNkQ9ngZ4z3+Ms0NuxEejMf221sxIye0IJvzHmaXvf3SAeAL5vrO28\nnhORdSKyGWvLxsPGmIsAInIYuAd4tJhOK9EZnpwlvbo1qy0dcVIxLL2bO7jYsq4i+dtJETbm7vbW\nw4ZXihFkv2fke1axBJXGKETQAr1zk/NQu1+LFUNSMf4tgDsQd95uC2rPQUQOAAcANqzbmFC3ao9S\nbah+LZ0/TKFkE+RZr041kGps8PXWi/Xmw+B+Rm5Wz/KzxmfmefT10yXrRyGCFuhdSy+WrQ9KMFUz\nuWuMeRh4GKw9dyvcnYqx56ZujvefTFz8VzU1cNWnPemCY+UkyYVQXoK894/9yqa8zyg2vBKGQs8Y\nGJ3m9MSVjOEKWrxVLAMj0xwfvsTj53PLhudboKdUnqQWcF0gewB3vd0W1K6UmZ7OVv+JugoVHCuW\niZn52Auhzl55reA51VqnJwz9g+MsLoVbvBUXZ3GX48G7i+yB/wK9psYGtnZGX6CnJE9S7t4h4Gsi\n8gOsyd3LxpgREXkc+Pci4kzn3A38fkLPVCLQ1dbM/u7OkoYhysmZiSs0dsyw4Nok3S1uQZ/zStPd\ngZk9fiOIL9+1o/QfJmHa1p7lyXN7c9oLhfW8efz58F/ctbxQL2gjoKn5cOtBqplXV8CGLKGEX0Qe\nxZqo7RKR81iZOikAY8x3gcc01jiIAAAbhklEQVSA+4BBYBb4TfvYRRH5Y+AF+1YPORO9SvkpJgxx\nbGC45Evonxs9FyqXf2BkmlfP3MjNPW/nHHM8/0I1anLuGbK2TTmJG8pa1eQ/kA8T1gtboz84dXU5\nvONXZ+v5M7Uv/CuBsFk9ny9w3AC/HXDsIHAweteUSuIVna5u/8nhIPo6t9N/JvxWi72tu0Ov3O0f\nCg5ZNAgFF1Fdabqbs1eeyErrDFtKoZTzCm7iGqKzV16jZ30rqbMSefFWFNpbmphuuMSl0U2e9uCF\nenH24T1W5IYsij9apK1O2LdtS+jl6351+U+PX6maBThub7OlM3tJSNAm6X4eqjvWH6o8QQkKrAVR\nqMia32I85/OsXvfZyPMTUcI8YC3uamjwzhkVXqgXZw/euGi5hmBqN6VDKRl+dfmXjOHZiNs+js/M\nc/CZ05HKSocJ9zilBN4c3sXNPW/T0jnJ3GRnJtUyTEkEJ9Z/9sprbFvzgVCLr4otsBZltJDPEPmN\nBl5970U2zbewqftXgdywnmMo8j07ylaMl2SSHV1rWLiYSrRseNJogTZ/1OOvUiq5MjBIdOamw8dn\nB0amODfYyMzCZfueC77lmd2ELfblLiXw5rBVzdNJtYxSEsGpX3/2ymuhriumwFrU0UK+EtleA7T5\nugsY4NHX3+97v0LPPj45FEn0He7adFvosuFxwzxKaVDhr0JKmTEQZvjrJzoNV1oDJw39yFee2Q+n\nrsvRUxP85fOvMjASHD7p3dzO/t7rMv1sbmrgV268lvFyo4Q5HPFf3X6m4HWF9ivIR5T6+JC/po/b\n0Gy+zsqOPnJuL8Z+TpRnRw3xALE2YYF4YZ648f1ShXlWQkYPaKinrti3bUtRdfl7OltDZ/c45Zkb\nLraztH46U6nTb1FPVl2X0U2s6x7j8NkhYGdguYDsUgK7mFo4yvHJIfZ07gy1wMkJuTSnGhB2Mrew\nxL03vMStu1qtcs153kucSdOoo4W8JbIHx1mz9ixgCX6h+wU949rq94ANsbz9KKWY43j7SaBhnmBU\n+JUc8tflD7cFY5tn5aY0zmIWW32zPrx1XS7Z4h9lw493xm9mbPZZjp56nlVXNwbGz73xcXfFzcdO\nfYi7l14EjtHV1pxTzK23u513Ll3l9QuXWTJWBtHuzR2h4vtxyjH7GbCzV17j1l3zDI3hm6vvdz+/\nZ7d0TtLc1BBZ9J8bPRdJ9B2ievuazVNaNNRTpey5qbskcf6w2T35NhkPE3t1l2duuGhdG5T14TcK\nuDS6iemGSwWfA8urSI+ftsoKX1v9XmD8vFDVzSfOfJhHX38/YImsO/NnYHSaEyNTmcyhJQMnRqZC\nZfUUW47Z3ZfV6z7L2NJ+xHNO0P28z27pnKShQdjUemeoZzvECfE8f2Y4lugXg4Z5CqMefx0TJye9\nr/v6UF6/M9H37NAEM3MLrGpqZGsnvhOA+eq6hMnyca8idTJ9Gjsu0j/YmPN5wkzETs+lszYuP3vF\nWuk7MDlJV9cSI+8uhxDCZvXEKeDmNjru/gB8sncT71u3OtT93M++tvo9mpsa2NR6Z6Sqm47olyvE\nU6y3r2Ge/KjwVzmvlqBg275tW/ifz7zNwILErs8fJtbvXbnZP3mG588M5yzq6tu5Iat2O1ijgz1r\nbwEuFBR/r5g7mT4397zN8cmlrA1GCtWzB2hJZQ+EHdH92am3+eTWY5kJVYfRd8OJTL65B7/6QV6x\nj3I/L1ebx9hzC3Sk7gt1vptiRH8lefsrCRX+KmbPTd0l2+lneHKWpvQS6fa2TFvY+vyO1x+1jENf\n53Zf8Q+q62K1dzAweyKv+AdtETj83m723TCWyVzZ07kzbz17h6Aj7S1NOZOp+7ceY9uWEc5emcvz\nycNRSOjj4nz+OJO4lRD9avX2V0qYB1T4a4JSeP3zc6ugKVeswtbnDxvyybkuj/gH5YE75RwcEfIa\ngL6dXRwZyM1C6tvZRUfKKrKWyfrptrx/p569H/MBWyz6GY2nL/wf7N99HVeaqq/YXTGCD/FE3yHu\nCt1iRP+Vl8+VRPRXmrcPOrlb9ZTKy3Cya1qmZzzt4X2Bvu7rYw3NHVGIEgPubd2dESDvJKM3r7+9\npYn9vddlxbA7UvvoSO3j+OSQHfZYoiNiXn6tlGo+PjnE8ckh3hm/mR8e3bRc1iHP2gg3z42ey2Tv\nRBX9OJO5UL0hHoeV5O2Devw1Q9Jef9/ODRw5sUi6YZaW6Rnm2tti1+ePU7kzyPMvhNv7d3v+YbcI\ndLzfqYWjbN12lVPjV5idWJc5XijTphwbrcTBvRCrI7Uvk+mUNYczYM/h5HlPcb38qOGdgZGpzMR/\nq5mjp7OVz924s/CFeVBvPzwq/DVAKWL97rj6THo6dqXJuPF+WBb/x14fZPRSY+iaL8uev1XNM0wp\nZy8dqX10dAI8zXDDZebTSzQ3NfCR634l9DsoZ6VOv+d4V926Qzr+9fJN4NoI9yiqHKL/pD2Zn1qY\n5RowPGwYaJuO9f5KFeJxWGnePqjw1wyl2JLRias7q3ljbwBehPivn1vP8+ODLBkDtGZq+jj9y9v/\nArH/MOzo/Cg77L1cphaOcpUxjk9az3dnA3kpV/1+73OurX6Pp0fGOb+whg1tzYHx+7ArhYsRfIg3\nkeuU80gtzAJWOZA04YvduSlliGelevsQfiOWe4A/BxqBR4wx3/Ic/0/AJ+xfW4FNxph19rFFwMlV\nO2eM+VwSHa9XSpXeefTsBV55+Ry33hZdPCG++D87NAFzbbB+Gmm0hCC92JrZyakQjlgVawAgdxL0\n+OTRnHMcYxBU/+aJN0Z5/PXRREYAxyeHePndSRo7lnDvVPv62Rs5+25Tzu5gAyPT9A9ZIwMRMD7p\nSc78RbGCD/DY64Ocm7zK4lgrJ1pOc2fI6pwzcwtZou8QNrHAwRH9UoZ4VqK3DyGEX0Qage8AnwbO\nAy+IyCFjzJvOOcaYf+06/3eAPa5bXDXG3JZcl+uXUqZ3Vkr83TV9AJZsAzA915rvshySNAAOXkPg\nZAYBXFs9QYtn+9i5yc7Mqt6wI4BCRdKc1chevCLpjen7iX5To7B63Rx/+fyrXEsvsnhpi7WSOtqr\n5vkzw4zPzHNusJG0qwbTkyFHaq1mjmtkiz5ESywopeg7rFTRh3Ae/+3AoDHmFIC9r+4DwJsB538e\na2tGpQSUIuTjUAnx99b0cYq6NTfPR574hdIYAAe3IRh+73SO+PptBfnLdy9xtbkzp93vnn60t+Q+\nx2rP/tP1i+kDGc9/VcdV1rWmGJ+Z5+I7G+2j4cNqDk5o572zq0gvZq+2Ti8uFdyz4djAMD2drQwP\nG9J40m9DJhaUWvRXcojHIYzwbwHcuVbnsTZUz0FEtgE7gF+4mltE5BiQBr5ljPlpwLUHgAMAG9Zt\n9DtFcVGKkA8kJ/4A/XaKXj4DcOfODZmJPodVl9fyqd5NXORiLPEH4PIW+ocmaFx3gePnLtGzfjVd\nbc2JGQG/tQPOimEvH93p3x73Oc4aBXdox5fWGQyw74YNQDP9L7XklMZwb5CeD28s/4U5f3Gc8Sm9\nAbmLswba4k2Ml0v0V7K3D8lP7j4I/NAYs+hq22aMuSAiNwC/EJHXjDE541tjzMPAwwA39OwKXlap\nZEI+1Sz+EM7799b0aWtJuWLFdubRmTMAoQ2At8wzwMWRBm69ZSYrtl2MEXAyY8LG1JN6TntLE307\nLc/YaxAAaM1el7F4aQu9rdZcwOMBYu1XJ8nBvdbCPYHrHam52734rciNkxarop8cYb6VFwD3X9z1\ndpsfD+LZdN0Yc8H+7ykR+Qes+H/03R+ULPbc1M3hF8/yNz94nktd6xLf+q4U4g/+3n++VbuQnfMP\nhQ2At8wzWJ7tyZNrucueEHWHghzWmc4cgc2X8+5eO+CNscOyZ14sfmsUDj5zevlZHrF3NkBvamxg\nf+9yNdSgYnh+pbKDBN/Bb6TW1NjAnZ7qq0mVYVDRT5Ywwv8CsEtEdmAJ/oPAr3tPEpFeoBP4paut\nE5g1xsyLSBdwJ/Afk+h4vTMwMsXJyws0pBdZPTrBVPcGjpyIFq8tRJLiD8RO+YRl8QljAII8WHe7\nN5PlmbGXeWF8iKUGA60wDRwZsAauYRaGBXnmUSpghsExVtMNlzKTso7Qu/FzBIKK4blLZRcSfIf8\nI7XkBB/KM5EL9SP6EEL4jTFpEfka8DhWOudBY8wbIvIQcMwYc8g+9UHgB8ZkDXh3A38lIktY5SG+\n5c4GUuLTPzTBwtISNKVoTi/QOjrBbPeG0GmQYUlK/CGc91/wHi4xckJAkG0Eoni2DidPrs3KJFrX\nPUa6eZqnhme5JOuyzg0KEYVdPRyGfLXve1t303/pdOBn9KZ5LvfPvxje1Pxlnj+zvJ9y2Jx8v5Ga\nu/RCrYj+SqqzH5ZQAUhjzGPAY562P/L8/u98rusHPuBtV4rH/Uc/7xL/6e7cjU6KxfmjO2r/ESbp\n/UM8AwDBo4Awnq0Xr4i6veje3ux5FGfFcCkplFvft3MDh0+8y9LSsp/V0CB5PyNki/XzZ4aZmrcE\nP25RNTdBXn6cFc7l8vLrUfRBV+5WLQMjUwFlii3aW1JM+Yj/uvFwu1bFIWnvH4oL/2Tu5TMK2LER\nRi+lQpeBiDJKiLvgKXG8k8khUiLChnKikC+s46w8jrLvQ6VFv9Df3kpAhb8KGRiZ4siJMSuUA0zN\nLeTE760ia8vnACytaqZnbapk2T6QLf5QnPcP0VI/Q93PEbNO6G87g/MVtzzbPBPIMUYJlaR/aMIu\nc7HMkjG+oT5vFdSkBR+Cwzr9gwE1g3zKM7jLL5Q6tAP+Mf0wf3srARX+KiQTv3exsJSdb51v85JS\npnqCK/STkPcPuQYAijMCAyNTnBgymYnHjduuZQmgd2I4/2Yw1Ue+CexSCb1DlInbsDWDyunlQ/BE\nbpi/vZWACn8VEiYrBYLTIEud5++QtPcPycwBuKs/gvXeZt4S2q5fzSd7r7Pu75oYdrh9e0/N/HE7\noSmnvpHDqqZG+jr9Q1HuUsjeLJxCxJ20DdodzVnfUC4vH8KlbIb926t1VPirEG/83t0elnKKPyTr\n/UNxBuBZnzx+A7x2/jLvW7ua3s0dOV6we4LYS6yVwwnj7Vv3ukVmx6+yZJbrHDU1NvCp3tzUTvA3\nhmFq6xSbpePsWpazvuHGrrJ5+RA+Tz+Jv71aQIW/CvGL36caosebyyX+kGzmjxu32IQNAwWVDTAQ\nWEsmKBySzyB46WheGzlUFGUXsqw+dsJAk+3BU9iD9zOGQbV1kkzJdOL47qyebaSZH50si+BDtMVZ\nSf3tVTsq/FVIkvFm58t+3P7yl8MAJB3+cfAbBUCuEQgqJwDBRiHwmRE2F/n5GWtfAWmEmQV48q0p\nTk2soautuej7+1FoxbOboM/ttHu3PkwiB9/BKc+wHNZprCov302tzfXERYW/SonyRx2Ginj/JTYA\n4G8E7ty5gcffGPXNbvSrJZMEzr4C3k2s37uS4nN3bS/JM6PgZwxTC7OsampIdJWtH+WM4zsUU4Ih\n6b+9akSFv45wiz+Ux/uH0hkAyBUrJxz0vpYlxqbnWEgtr8b1qyWTFIU86kpz584NPPXK6awU0FVz\na0q6WXwlBR/qqwRDVFT464xM6KdM3j9k/9EfdYlB0kYAlg1BXzccGhxieHKWa+klVjU10NPeWjJP\nLkq1ynLhDd/s6FrD+GjD8gra3aXZI7gSgg/1V2itGFT465Rye/8O5RgFOHzuxtzdq/oH/CdTi104\ndqddQmHRVUKhsUFKNsJw4xV4B9/QzY2l6UOlxB7Uy4+DCn8dUwnv38HPAEDpjICDnxh65wm8hDYK\nMUoohCVf/6B08flCVFLwQb38uKjwK2XP/HGTFQYqwyjAj3yiWcgoOBwfnqQhvZQ1ubuQas1Jlwxz\nrzj9LCeVFntQL79YVPiVDJUK/ziUcy4gLGHF9oXXr+Zk9KTWzDK3kHyp4mIpplomVE7sQQU/KVT4\nlSy84R8ovwGAYCMAlTUEQfiVJmi40moJaxWIvUPYapmveN55JcUeVPCTJpTwi8g9wJ9jbcTyiDHm\nW57jXwL+hOUtGb9tjHnEPvZF4P+x2/9fY8zfJNBvpcRUiwGA4HAQVI8RyFeaIA5xvPIw5KuWOT86\nmdVeabEHFfxSUVD4RaQR+A7waeA88IKIHPLZSevvjTFf81y7HvgmsBdrqutF+9pJlJqgmgwA5IpR\ntYwG/EoTxBXrODXsw+IelbRML+/Vm56Gfbd8sKh7J4kKfmkJ4/HfDgwaY04BiMgPgAeAMFsofgY4\nbIy5aF97GLgHeDRed5VKUW0GwKGQIYDyGQOnNEGxRKlhHxZnlLT26izzaasOjUm3ZI5XSxEyFfzy\nEEb4twDuVITzwD6f835NRD4KnAT+tTFmOODayo8fldhUqwFwyDEEntCQm2oJE3kJW8Pej6DPCta7\nWbuqvSqLkKngl5ekJnf/O/CoMWZeRP4F8DfAJ6PcQEQOAAcANqzbmFC3lFLh/uM87vqjrSYjAMFx\n6nwGASprFKLUsPcjX2y+moqQucUeVPDLSRjhvwC4V7Bcz/IkLgDGmAnXr48A/9F17cc91/6D30OM\nMQ8DDwPc0LMrwaUvSqmp9lGAH/nEsZBRCCKOsfB7zjbSnJq5klVXp0GEbS1rEqlhX+kiZOrdV54w\nwv8CsEtEdmAJ+YPAr7tPEJHNxpgR+9fPASfsnx8H/r2IdNq/3w38ftG9VqqSWhkFFCKuqPrNL8R9\nVrVs+J1UP9S7ry4KCr8xJi0iX8MS8UbgoDHmDRF5CDhmjDkE/J8i8jkgDVwEvmRfe1FE/hjLeAA8\n5Ez0Kisbv1EA1J4RiEKS6Y+V9sqh+I3HVeyrFzGm+qIqN/TsMg99/c8q3Q0lYY6fHM36fSUbgZXA\nwWdO+25D2NGS4st37fC9RsW+cnzhG/e/aIzZG+ZcXbmrlI2sUFAdjQRqlbAbj6vY1x4q/EpFyGcE\nQA1BNRC08fi68Usq9jWOCr9ScbyioYagOnA2Hk+9816mrVGEnq41KvQ1jgq/UnWoIags7ne9e2Ge\n4aZGpmlYsRuP1yMq/ErVE8YQgBqDOPi9R9DQzUpHhV+pOfxEKcgYgBoECBZ4UJGvR1T4lRVBkHjl\nMwiwsoxCvs8JKvDKMir8yoomn9gVMgpuKmkgwvYRVNyVcKjwK3VLFJE8HkF8k0bFXEkaFX5FCYGK\nr7KS8O4PrSiKoqxw1ONXVgTVUs1SUWoBFX6l5im2iqSi1Bsa6lFqnv6hiaytBAEWlpboH5oIuEJR\n6hsVfqXmCVtFUlEUCxV+peZpb0lFaleUeieU8IvIPSLylogMisi/9Tn+f4nImyLyqogcEZFtrmOL\nIvKy/e9Qkp1XFLCqSKYasr/KqYYG+nZuqFCPFKW6KTi5KyKNwHeATwPngRdE5JAx5k3XaceBvcaY\nWRH5KtZm6//UPnbVGHNbwv1WlAzOBK5m9ShKOMJk9dwODBpjTgGIyA+AB4CM8BtjnnKd/xzwz5Ls\npKIUohr2qFWUWiGM8G8Bhl2/nwf25Tn/K8DPXL+3iMgxrI3Yv2WM+anfRSJyADhg/zr/hW/c/3qI\nvlUTXcB4pTsREe1z+ajFfmufy0NSfd5W+BSLRPP4ReSfAXuBj7k7Y4y5ICI3AL8QkdeMMUPea40x\nDwMP2/c5FnbT4GpB+1wearHPUJv91j6Xh0r0Oczk7gWgx/X79XZbFiLyKeAPgc8ZY+addmPMBfu/\np4B/APYU0V9FURSlSMII/wvALhHZISKrgAeBrOwcEdkD/BWW6I+52jtFpNn+uQu4E9fcgKIoilJ+\nCoZ6jDFpEfka8DjQCBw0xrwhIg8Bx4wxh4A/AdqA/yoiAOeMMZ8DdgN/JSJLWEbmW55soCAejvdx\nKor2uTzUYp+hNvutfS4PZe+zGGPK/UxFURSlgujKXUVRlDpDhV9RFKXOKKvwhyj90Cwif28fPyoi\n213Hft9uf0tEPlNFfa7KchUh+v0lEXnP1b/fch37ooi8bf/7YhX1+T+5+ntSRC65jlXkXYvIQREZ\nExHfdSdi8Rf2Z3pVRD7kOlap91yoz79h9/U1EekXkVtdx87Y7S/b63Oqpc8fF5HLru/AH7mO5f1e\nVbDP33D193X7O7zePlba92yMKcs/rInhIeAGYBXwCnCz55x/BXzX/vlB4O/tn2+2z28Gdtj3aayS\nPn8CaLV//qrTZ/v3mXK93xj9/hLwbZ9r1wOn7P922j93VkOfPef/DlaiQaXf9UeBDwGvBxy/D2tB\nowB3AEcr+Z5D9rnP6Qtwr9Nn+/czQFcVvuePA/+j2O9VOfvsOfezwC/K9Z7L6fFnSj8YY64BTukH\nNw8Af2P//ENgv1hpQg8APzDGzBtjTgOD9v0q3mdjzFPGmFn71+ew1jlUmjDvOojPAIeNMReNMZPA\nYeCeEvXTTdQ+fx54tAz9yosx5mngYp5THgC+byyeA9aJyGYq954L9tkY02/3CarkOx3iPQdRzN9C\nUUTsc1m/z+UUfr/SD1uCzjHGpIHLwIaQ15aCqM/1LVchIs+JyD8uRQcDCNvvX7OH9D8UEWeRXtW/\nazuctgP4hau5Uu+6EEGfq1LvOSre77QBnhCRF8Uqs1JNfEREXhGRn4nILXZb1b9nEWnFMvo/cjWX\n9D3r1osJIUWUq6gQ/x141BgzLyL/Amuk9ckK9yksDwI/NMYsutqq+V3XJCLyCSzhv8vVfJf9njcB\nh0VkwPZsK81LWN+BGRG5D/gpsKvCfQrLZ4FnjTHu0UFJ33M5Pf4wpR8y54hIE7AWmAh5bSmo1XIV\nBfttjJlw9fUR4MNhry0RUZ77IJ5hcQXfdSGCPlel3nMoROSDWN+LB4wxmT0sXe95DPgJ5Qm5FsQY\nM2WMmbF/fgxIiVUtoKrfs02+73Np3nM5JjnsyYomrAmsHSxPstziOee3yZ7c/S/2z7eQPbl7ivJM\n7obp8x6syaNdnvZOoNn+uQt4m/JNKoXp92bXz78KPGf/vB44bfe/0/55fTX02T6vF2viS6rhXdvP\n3E7wpOM/Inty9/lKvueQfd6KNY/W52lfA7S7fu4H7qmSPnc73wkskTxnv/NQ36tK9Nk+vhZrHmBN\nOd9zWT686wPdB5y0hfIP7baHsDxlgBbgv9pfuueBG1zX/qF93VvAvVXU5yeBd4GX7X+H7PY+4DX7\ni/Ya8JUqe9f/AXjD7t9TQK/r2i/b/w8Ggd+slj7bv/87rNIf7usq9q6xPLURYAErfvwV4F8C/9I+\nLlgbGQ3ZfdtbBe+5UJ8fASZd3+ljdvsN9jt+xf7u/GEV9flrru/zc7iMlt/3qhr6bJ/zJazEFfd1\nJX/PWrJBURSlztCVu4qiKHWGCr+iKEqdocKvKIpSZ6jwK4qi1Bkq/IqiKHWGCr+iKEqdocKvKIpS\nZ/z/2ZYwklmtImkAAAAASUVORK5CYII=\n", "text/plain": [ "
" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "\n", "plt.scatter(X[:,0], X[:,1])\n", "plt.contourf(xx1, xx2, np.reshape(y, (100, 100)), zdir='z', offset=-0.15, cmap=cm.viridis, alpha=0.5)\n", "plt.show()" ] }, { "cell_type": "markdown", "metadata": { "collapsed": true }, "source": [ "#### 1.2. We consider the following linear regression problem. (Total 5pts)" ] }, { "cell_type": "code", "execution_count": 78, "metadata": {}, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAX4AAAD8CAYAAABw1c+bAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi40LCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcv7US4rQAAFYFJREFUeJzt3X+MZeV93/H3p4DVzZp2oWwIu0Cw\nKrQWNoW1R8QuroV/hF9FhqLIBaUuJEQbR3ZrVy4OOFJdOX9ARePWEpHJ1lCwQogje8GopsAKuyJW\n/Wt2FxswUFyCww5rdhyyBscr2Yu//WPOlPFwh70z5869c+95v6TRnPucc+55ro0+c/a53/M8qSok\nSd3x90bdAUnScBn8ktQxBr8kdYzBL0kdY/BLUscY/JLUMYcN/iQnJflKku8meTTJh5r2G5I8nuQ7\nSe5MsmGJ859O8nCSh5JMD/oDSJKWJ4er409yAnBCVe1OcjSwC7gEOBH4clUdSvKfAKrq93uc/zQw\nVVU/HHTnJUnLd9g7/qraV1W7m+0XgceAzVV1f1Udag77OnN/CCRJa9yRyzk4ySnAVuAbi3b9NvC5\nJU4r4P4kBfxJVW1f4r23AdsA1q9f/+bXv/71y+maJHXarl27flhVG/s5tu/gT/Ja4AvAh6vqhQXt\nfwAcAm5f4tS3VdVMkl8GdiZ5vKoeXHxQ8wdhO8DU1FRNT/t1gCT1K8n3+z22r6qeJEcxF/q3V9WO\nBe1XAhcBv1lLfFlQVTPN7/3AncBZ/XZOkjR4/VT1BLgZeKyqPrmg/Xzgo8B7quonS5y7vvlCmCTr\ngXOBRwbRcUnSyvRzx3828D7gnU1J5kNJLgRuBI5mbvjmoSQ3ASTZlOSe5tzjga8m+TbwTeBLVXXv\n4D+GJKlfhx3jr6qvAumx654ebVTVs8CFzfZTwBltOihJGiyf3JWkjllWOackafDu2jPDDfc9wbMH\nDrJpwzquPm8Ll2zdvGrXM/glaYTu2jPDtTse5uDPXgJg5sBBrt3xMMCqhb9DPZI0Qjfc98T/D/15\nB3/2Ejfc98SqXdPgl6QRevbAwWW1D4LBL0kjtGnDumW1D4LBL0kjdPV5W1h31BG/0LbuqCO4+rwt\nq3ZNv9yVpBGa/wLXqh5JGiNtyzEv2bp5VYN+MYNfkloYRTlmW47xS1ILoyjHbMvgl6QWRlGO2ZbB\nL0ktjKIcsy2DX5JaGEU5Zlt+uStJLYyiHLMtg1+SWhp2OWZbDvVIUsf0s+buSUm+kuS7SR5N8qGm\n/dgkO5M82fw+Zonzr2iOeTLJFYP+AJKk5elnqOcQ8JGq2t0snL4ryU7gSuCBqro+yTXANcDvLzwx\nybHAx4EpoJpz766qvx3kh5CkNoa9EMqoHfaOv6r2VdXuZvtF4DFgM3AxcFtz2G3AJT1OPw/YWVXP\nN2G/Ezh/EB2XpEGYf/J25sBBipefvL1rz8you7ZqljXGn+QUYCvwDeD4qtrX7PoBcHyPUzYDzyx4\nvbdp6/Xe25JMJ5menZ1dTrckacXG8cnbtvoO/iSvBb4AfLiqXli4r6qKuaGcFauq7VU1VVVTGzdu\nbPNWktS3cXzytq2+gj/JUcyF/u1VtaNpfi7JCc3+E4D9PU6dAU5a8PrEpk2S1oRxfPK2rX6qegLc\nDDxWVZ9csOtuYL5K5wrgiz1Ovw84N8kxTdXPuU2bJK0J4/jkbVv9VPWcDbwPeDjJQ03bx4Drgb9I\nchXwfeC9AEmmgPdX1e9U1fNJ/hD4VnPeJ6rq+YF+AklqYRyfvG0rc8Pza8vU1FRNT0+PuhuSNDaS\n7KqqqX6O9cldSeoYg1+SOsbgl6SOMfglqWMMfknqGINfkjrG4JekjjH4JaljDH5J6hiDX5I6xsXW\nJY29rq2g1ZbBL2msza+gNb+YyvwKWoDhvwSHeiSNtS6uoNWWwS9prHVxBa22DH5JY62LK2i1ZfBL\nGmtdXEGrrcN+uZvkFuAiYH9VvbFp+xww/7/qBuBAVZ3Z49yngReBl4BD/S4SIEn96uIKWm31U9Vz\nK3Aj8Nn5hqr6l/PbSf4I+NGrnP+OqvrhSjsoSYdzydbNBv0yHDb4q+rBJKf02tcsxP5e4J2D7ZYk\nabW0HeP/Z8BzVfXkEvsLuD/JriTbXu2NkmxLMp1kenZ2tmW3JElLaRv8lwN3vMr+t1XVm4ALgA8k\neftSB1bV9qqaqqqpjRs3tuyWJGkpK35yN8mRwKXAm5c6pqpmmt/7k9wJnAU8uNJrSppMTrkwXG3u\n+N8NPF5Ve3vtTLI+ydHz28C5wCMtridpAs1PuTBz4CDFy1Mu3LVnZtRdm1iHDf4kdwBfA7Yk2Zvk\nqmbXZSwa5kmyKck9zcvjga8m+TbwTeBLVXXv4LouaRI45cLw9VPVc/kS7Vf2aHsWuLDZfgo4o2X/\nJE04p1wYPp/clTRSTrkwfAa/pJFyyoXhcz5+SSPllAvDZ/BLGjmnXBguh3okqWMMfknqGINfkjrG\n4JekjjH4JaljDH5J6hiDX5I6xuCXpI4x+CWpYwx+SeoYg1+SOsbgl6SOMfglqWP6WXrxliT7kzyy\noO0/JplJ8lDzc+ES556f5Ikk30tyzSA7LmntuGvPDGdf/2Ved82XOPv6L7te7hrXzx3/rcD5Pdr/\nS1Wd2fzcs3hnkiOAPwYuAE4DLk9yWpvOSlp7XCx9/Bw2+KvqQeD5Fbz3WcD3quqpqvop8OfAxSt4\nH0mrrM0du4ulj582Y/wfTPKdZijomB77NwPPLHi9t2nrKcm2JNNJpmdnZ1t0S9JytL1jd7H08bPS\n4P808I+BM4F9wB+17UhVba+qqaqa2rhxY9u3k9SntnfsLpY+flYU/FX1XFW9VFU/B/4bc8M6i80A\nJy14fWLTJmkNaXvH7mLp42dFwZ/khAUv/wXwSI/DvgWcmuR1SV4DXAbcvZLrSVo9be/YL9m6mesu\nPZ3NG9YRYPOGdVx36emuobuGHXax9SR3AOcAxyXZC3wcOCfJmUABTwO/2xy7CfhMVV1YVYeSfBC4\nDzgCuKWqHl2VTyFpxa4+bwvX7nj4F4Z7lnvH7mLp4yVVNeo+vMLU1FRNT0+PuhtSZ9y1Z4Yb7nuC\nZw8cZNOGdVx93haDfMwk2VVVU/0ce9g7fkmTzzv2bnHKBknqGINfkjrG4JekjjH4JaljDH5J6hir\neqQ1wHJKDZPBL43Y/CRp8w9QzU+SBhj+WhUO9Ugj5rTGGjaDXxoxpzXWsBn80og5rbGGzeCXRsxp\njTVsfrkrjdj8F7htqnqsCtJyGPzSGtBmkjSrgrRcDvVIY86qIC2XwS+NOauCtFyHDf4ktyTZn+SR\nBW03JHk8yXeS3JlkwxLnPp3k4SQPJXFlFWkVWBWk5ernjv9W4PxFbTuBN1bVPwH+D3Dtq5z/jqo6\ns9+VYSQtj1VBWq7DBn9VPQg8v6jt/qo61Lz8OnDiKvRNUh9c7FzLNYiqnt8GPrfEvgLuT1LAn1TV\n9qXeJMk2YBvAySefPIBuSd3h0olajlbBn+QPgEPA7Usc8raqmknyy8DOJI83/4J4heaPwnaYW2y9\nTb+kYbOOXuNkxVU9Sa4ELgJ+s6p6BnVVzTS/9wN3Amet9HrSWjVfRz9z4CDFy3X0d+2ZGXXXpJ5W\nFPxJzgc+Crynqn6yxDHrkxw9vw2cCzzS61hpnFlHr3HTTznnHcDXgC1J9ia5CrgROJq54ZuHktzU\nHLspyT3NqccDX03ybeCbwJeq6t5V+RTSCFlHr3Fz2DH+qrq8R/PNSxz7LHBhs/0UcEar3kljYNOG\ndcz0CHnr6LVW+eSu1JJ19Bo3TtImtTSI2TWlYTL4pQGwjl7jxOCXsA5f3WLwq/Ocz15d45e76jzr\n8NU1Br86zzp8dY3Br85zPnt1jcGvzrMOX13jl7vqPOvw1TUGv4R1+OoWh3okqWMMfknqGINfkjrG\n4JekjjH4JaljDH5J6pi+gj/JLUn2J3lkQduxSXYmebL5fcwS517RHPNkkisG1XFJ0sr0e8d/K3D+\norZrgAeq6lTggeb1L0hyLPBx4NeAs4CPL/UHQpI0HH0Ff1U9CDy/qPli4LZm+zbgkh6nngfsrKrn\nq+pvgZ288g+IJGmI2ozxH19V+5rtHwDH9zhmM/DMgtd7m7ZXSLItyXSS6dnZ2RbdkiS9moF8uVtV\nBVTL99heVVNVNbVx48ZBdEuS1EObuXqeS3JCVe1LcgKwv8cxM8A5C16fCPyvFteUenLpRKl/be74\n7wbmq3SuAL7Y45j7gHOTHNN8qXtu0yYNzPzSiTMHDlK8vHTiXXtmRt01aU3qt5zzDuBrwJYke5Nc\nBVwP/HqSJ4F3N69JMpXkMwBV9Tzwh8C3mp9PNG3SwLh0orQ8fQ31VNXlS+x6V49jp4HfWfD6FuCW\nFfVO6oNLJ0rL45O7GnsunSgtj8GvsefSidLyuAKXxp5LJ0rLY/BrIrh0otQ/h3okqWMMfknqGINf\nkjrG4JekjjH4JaljDH5J6hiDX5I6xuCXpI7xAS6tCc6nLw2Pwa+Rm59Pf35q5fn59AHDX1oFDvVo\n5JxPXxoug18j53z60nCtOPiTbEny0IKfF5J8eNEx5yT50YJj/kP7LmvSOJ++NFwrDv6qeqKqzqyq\nM4E3Az8B7uxx6F/OH1dVn1jp9TS5nE9fGq5Bfbn7LuD/VtX3B/R+6hDn05eGa1DBfxlwxxL73prk\n28CzwL+vqkd7HZRkG7AN4OSTTx5QtzQsbcsxnU9fGp5UVbs3SF7DXKi/oaqeW7TvHwA/r6ofJ7kQ\n+FRVnXq495yamqrp6elW/dLwLC7HhLmhmusuPd0wl4Ykya6qmurn2EFU9VwA7F4c+gBV9UJV/bjZ\nvgc4KslxA7im1hDLMaXxMojgv5wlhnmS/EqSNNtnNdf7mwFcU2uI5ZjSeGk1xp9kPfDrwO8uaHs/\nQFXdBPwG8HtJDgEHgcuq7diS1pxNG9Yx0yPkLceU1qZWd/xV9XdV9Y+q6kcL2m5qQp+qurGq3lBV\nZ1TVW6rqf7ftsNYeyzGl8eJcPWrNckxpvBj8GgjLMaXx4Vw9ktQxBr8kdYzBL0kdY/BLUscY/JLU\nMQa/JHWMwS9JHWMdv4D20ypLGh8Gv14xrfLMgYNcu+NhAMNfmkAG/4Roc8f+atMqG/zS5DH4J0Db\nO3anVZa6xS93J0DbhVCWmj7ZaZWlyWTwT4C2d+xOqyx1i8E/AdresV+ydTPXXXo6mzesI8DmDetc\nL1eaYI7xT4Crz9vSc7Hz5dyxO62y1B2tgz/J08CLwEvAocWrvDdr7n4KuBD4CXBlVe1ue129zIVQ\nJC3HoO7431FVP1xi3wXAqc3PrwGfbn5rgLxjl9SvYYzxXwx8tuZ8HdiQ5IQhXFeS1MMggr+A+5Ps\nSrKtx/7NwDMLXu9t2n5Bkm1JppNMz87ODqBbkqReBhH8b6uqNzE3pPOBJG9fyZtU1faqmqqqqY0b\nNw6gW5KkXloHf1XNNL/3A3cCZy06ZAY4acHrE5s2SdIItAr+JOuTHD2/DZwLPLLosLuBf505bwF+\nVFX72lxXkrRybat6jgfunKvY5Ejgz6rq3iTvB6iqm4B7mCvl/B5z5Zy/1fKakqQWWgV/VT0FnNGj\n/aYF2wV8oM11JEmD45QNktQxBr8kdYzBL0kd4yRta4Rr3koaFoN/DXDNW0nD5FDPGtB2BS1JWg6D\nfw1wzVtJw2TwrwGueStpmAz+NcA1byUNk1/uDkibqhxX0JI0TAZ/o01wD6IqxxW0JA2LQz28HNwz\nBw5SvBzcd+3pb/Zoq3IkjRODn/bBbVWOpHFi8NM+uK3KkTRODH7aB7dVOZLGicFP++C+ZOtmrrv0\ndDZvWEeAzRvWcd2lp/tlraQ1acVVPUlOAj7L3CpcBWyvqk8tOuYc4IvAXzVNO6rqEyu95moZRDml\nVTmSxkWbcs5DwEeqanez7u6uJDur6ruLjvvLqrqoxXWGwuCW1BUrHuqpqn1VtbvZfhF4DDA5JWmN\nG8gYf5JTgK3AN3rsfmuSbyf5n0ne8CrvsS3JdJLp2dnZQXRLktRD6yd3k7wW+ALw4ap6YdHu3cCv\nVtWPk1wI3AWc2ut9qmo7sB1gamqqltsPFzKRpP60uuNPchRzoX97Ve1YvL+qXqiqHzfb9wBHJTmu\nzTV7afvkrSR1yYqDP0mAm4HHquqTSxzzK81xJDmrud7frPSaS3HKBEnqX5uhnrOB9wEPJ3moafsY\ncDJAVd0E/Abwe0kOAQeBy6pq2cM4h+OUCZLUvxUHf1V9FchhjrkRuHGl1+jXpg3rmOkR8k6ZIEmv\nNBFP7jplgiT1byLm43chE0nq30QEP/jkrST1ayKGeiRJ/TP4JaljDH5J6hiDX5I6xuCXpI7JKjxI\n21qSWeD7Kzz9OOCHA+zOOPAzT76ufV7wMy/Xr1bVxn4OXJPB30aS6aqaGnU/hsnPPPm69nnBz7ya\nHOqRpI4x+CWpYyYx+LePugMj4GeefF37vOBnXjUTN8YvSXp1k3jHL0l6FQa/JHXMRAV/kvOTPJHk\ne0muGXV/VlOSk5J8Jcl3kzya5EOj7tOwJDkiyZ4k/2PUfRmGJBuSfD7J40keS/LWUfdptSX5d81/\n148kuSPJ3x91nwYtyS1J9id5ZEHbsUl2Jnmy+X3Malx7YoI/yRHAHwMXAKcBlyc5bbS9WlWHgI9U\n1WnAW4APTPjnXehDwGOj7sQQfQq4t6peD5zBhH/2JJuBfwtMVdUbgSOAy0bbq1VxK3D+orZrgAeq\n6lTggeb1wE1M8ANnAd+rqqeq6qfAnwMXj7hPq6aq9lXV7mb7RebCYOIXJEhyIvDPgc+Mui/DkOQf\nAm8Hbgaoqp9W1YHR9moojgTWJTkS+CXg2RH3Z+Cq6kHg+UXNFwO3Ndu3AZesxrUnKfg3A88seL2X\nDgQhQJJTgK3AN0bbk6H4r8BHgZ+PuiND8jpgFvjvzfDWZ5KsH3WnVlNVzQD/GfhrYB/wo6q6f7S9\nGprjq2pfs/0D4PjVuMgkBX8nJXkt8AXgw1X1wqj7s5qSXATsr6pdo+7LEB0JvAn4dFVtBf6OVfrn\n/1rRjGtfzNwfvU3A+iT/arS9Gr6aq7VflXr7SQr+GeCkBa9PbNomVpKjmAv926tqx6j7MwRnA+9J\n8jRzQ3nvTPKno+3SqtsL7K2q+X/NfZ65PwST7N3AX1XVbFX9DNgB/NMR92lYnktyAkDze/9qXGSS\ngv9bwKlJXpfkNcx9GXT3iPu0apKEuXHfx6rqk6PuzzBU1bVVdWJVncLc/79frqqJvhOsqh8AzyTZ\n0jS9C/juCLs0DH8NvCXJLzX/nb+LCf9Ce4G7gSua7SuAL67GRSZmsfWqOpTkg8B9zFUB3FJVj464\nW6vpbOB9wMNJHmraPlZV94ywT1od/wa4vbmheQr4rRH3Z1VV1TeSfB7YzVz12h4mcPqGJHcA5wDH\nJdkLfBy4HviLJFcxNzX9e1fl2k7ZIEndMklDPZKkPhj8ktQxBr8kdYzBL0kdY/BLUscY/JLUMQa/\nJHXM/wNzW/nyZ61XoAAAAABJRU5ErkJggg==\n", "text/plain": [ "
" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "import numpy as np\n", "import matplotlib.pyplot as plt\n", "\n", "from scipy.io import loadmat\n", "X = loadmat('MidTermAssignment_dataEx2.mat')['MidTermAssignment_dataEx2']\n", "\n", "plt.scatter(X[:,0], X[:,1])\n", "plt.show()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Solve the $\\ell_2$ regularized linear regression problem __through the normal equations__ (be careful that you have to take the $\\ell_2$regularization into account). Then double-check your solution by comparing it with the regression function from scikit learn. Plot the result below." ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": true }, "outputs": [], "source": [ "\n", "\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "2.3. __Kernel Ridge regression__. Given the 'Normal Equations' solution to the regularized regression model, we now want to turn the regression model into a formulation over kernels. \n", "\n", "\n", "__2.3.1. Start by showing (one line) that this solution can read as__ \n", "\n", "$$\\mathbf{\\beta} = \\mathbf{X}^T\\left(\\mathbf{K} + \\lambda\\mathbf{I}_N\\right)^{-1}\\mathbf{t}$$\n", "\n", "where $\\mathbf{K}$ is the kernel matrix defined from the scalar product of the prototypes, i.e. $\\mathbf{K}_{i,j} = \\kappa(\\mathbf{x}^{(i)}, \\mathbf{x}^{(j)}) = (\\mathbf{x}^{(i)})^T(\\mathbf{x}^{(j)})$. \n", "\n", "__2.3.2.__ Given this, the classifier can read as $f(\\mathbf{x}) = \\mathbf{\\beta}^T\\mathbf{x} = \\sum_{i=1}^N \\alpha_i \\kappa(\\mathbf{x}, \\mathbf{x}_i)$. What are the $\\alpha$ in this case?\n", "\n", "__2.3.3.__ We will apply this idea to text data. Using kernels with text data is interesting because it is usually easier to compare documents than to find appropriate features to represent those documents. The file 'headlines_train.txt' contains a few headlines, some of them being about finance, others being about weather forecasting. Use the first group of lines below to load those lines and their associated targets (1/0). " ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": true }, "outputs": [], "source": [ "# Start by loading the file using the lines below \n", "import numpy as np\n", "\n", "f = open('headlines_train.txt', \"r\")\n", "lines = f.readlines()\n", "\n", "f.close()\n", "\n", "sentences = ['Start']\n", "target = [0]\n", "\n", "for l in np.arange(len(lines)-2):\n", " \n", " if l%2 == 0:\n", " \n", " \n", " lines_tmp = lines[l]\n", " lines_tmp = lines_tmp[:-1]\n", " sentences.append(lines_tmp)\n", " if lines_tmp[-1] == ' ':\n", " target.append(float(lines_tmp[-2]))\n", " else:\n", " target.append(float(lines_tmp[-1]))\n", " \n", "sentences = sentences[1:]\n", "target = target[1:]" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "__2.3.4.__ Now use the lines below to define the kernel. The kernel is basically built by generating a TF-IDF vector for each sentence and comparing those sentences through a cosine similarity measure. the variable 'kernel' the kernel matrix, i.e. $\\kappa(i,j) = \\frac{\\phi_i^T\\phi_j}{\\|\\phi_i\\|\\|\\phi_j\\|}$ where the $\\phi_i$ encodes the tf-idf vectors. Use the lines below to compute the kernel matrix. " ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": true }, "outputs": [], "source": [ "import numpy as np\n", "\n", "from sklearn.feature_extraction.text import TfidfVectorizer\n", "\n", "vect = TfidfVectorizer(max_features=100, stop_words='english',\n", " decode_error='ignore')\n", "\n", "tfidf = vect.fit_transform(sentences)\n", "\n", "\n", "\n", "from sklearn.metrics.pairwise import cosine_similarity\n", "kernel = cosine_similarity(tfidf)\n", "\n", "import matplotlib.pyplot as plt\n", "plt.imshow(kernel)\n", "plt.show()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "__2.3.4.__ Once you have the kernel matrix, compute the weights $\\alpha$ of the classifier $y(\\mathbf{x}) = \\sum_{i\\in \\mathcal{D}}\\alpha_i \\kappa(\\mathbf{x}, \\mathbf{x}_i)$. " ] }, { "cell_type": "code", "execution_count": 60, "metadata": { "collapsed": true }, "outputs": [], "source": [ "# compute the alpha weights\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "__2.3.5.__ Now that you have the weights, we want to apply the classifier to a few new headlines. Those headlines are stored in the file 'headlines_test.txt'. Use the lines below to load those sentences and compute their TF-IDF representation. the classifier $y(\\mathbf{x}) = \\sum_{i\\in \\mathcal{D}}\\alpha_i \\kappa(\\mathbf{x}, \\mathbf{x}_i)$ " ] }, { "cell_type": "code", "execution_count": 61, "metadata": { "collapsed": true }, "outputs": [], "source": [ "# Start by loading the file using the lines below \n", "import numpy as np\n", "\n", "f = open('headlines_test.txt', \"r\")\n", "lines = f.readlines()\n", "\n", "f.close()\n", "\n", "sentences_test = ['Start']\n", "\n", "for l in np.arange(len(lines)):\n", " \n", " if l%2 == 0:\n", " \n", " \n", " lines_tmp = lines[l]\n", " lines_tmp = lines_tmp[:-1]\n", " sentences_test.append(lines_tmp)\n", " \n", "sentences_test = sentences_test[1:]\n", "\n", "tfidf_test = vect.transform(sentences_test)\n", "\n", "test_F = np.hstack((tfidf_test.todense(), np.zeros((4, 100-np.shape(tfidf_test.todense())[1]))))" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "__2.3.6.__ Once you have the tf-idf representations stored in the matrix test_F (size 4 by 100 features) the value $\\kappa(\\mathbf{x}, \\mathbf{x}_i)$ that you need to get the final classifier $y(\\mathbf{x}) = \\sum_{i\\in \\mathcal{D}}\\alpha_i \\kappa(\\mathbf{x}, \\mathbf{x}_i)$ and hence the target of the new sentences, you need to compute the cosine similarity of the new \"test\" tf-idf vectors with the \"training\" tf-idf vectors which you computed earlier. each of those cosine similarities will give you an entry in $\\kappa(\\mathbf{x}, \\mathbf{x}_i)$ (here $\\mathbf{x}$ denotes any of the fixed test sentences). once you have those similarities, compute the target from your $\\alpha$ values as $t(\\mathbf{x}) = \\sum_{i\\in \\text{train}} \\alpha_i\\kappa(\\mathbf{x}, \\mathbf{x}_i)$. print those targets below. " ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": true }, "outputs": [], "source": [] } ], "metadata": { "kernelspec": { "display_name": "Python 2", "language": "python", "name": "python2" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 2 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython2", "version": "2.7.13" } }, "nbformat": 4, "nbformat_minor": 2 }