{ "metadata": { "name": "" }, "nbformat": 3, "nbformat_minor": 0, "worksheets": [ { "cells": [ { "cell_type": "heading", "level": 1, "metadata": {}, "source": [ "What is machine learning?" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "In this section we will begin to explore the basic principles of machine learning.\n", "Machine Learning is about building programs with **tunable parameters** (typically an\n", "array of floating point values) that are adjusted automatically so as to improve\n", "their behavior by **adapting to previously seen data.**\n", "\n", "Machine Learning can be considered a subfield of **Artificial Intelligence** since those\n", "algorithms can be seen as building blocks to make computers learn to behave more\n", "intelligently by somehow **generalizing** rather that just storing and retrieving data items\n", "like a database system would do.\n", "\n", "We'll take a look at two very simple machine learning tasks here.\n", "The first is a **classification** task: the figure shows a\n", "collection of two-dimensional data, colored according to two different class\n", "labels. A classification algorithm may be used to draw a dividing boundary\n", "between the two clusters of points:" ] }, { "cell_type": "code", "collapsed": false, "input": [ "# Start matplotlib inline mode, so figures will appear in the notebook\n", "%matplotlib inline" ], "language": "python", "metadata": {}, "outputs": [], "prompt_number": 0 }, { "cell_type": "code", "collapsed": false, "input": [ "# Import the example plot from the figures directory\n", "from figures import plot_sgd_separator\n", "plot_sgd_separator()" ], "language": "python", "metadata": {}, "outputs": [ { "output_type": "display_data", "png": "iVBORw0KGgoAAAANSUhEUgAAAW0AAAD7CAYAAAChScXIAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3XeYVNX5B/Dv997pswWWLr0qigiCva3BElFs0RixRk1M\nTKKJiZpilBgTuzEmQc1PxRITbNHYBcsKdkEQRaT3LsuyZfrc9/fHDLoOM8DuzOydu/t+nmcemZ0z\n976D8OXMOfeeQxGBUkopZzDsLkAppdTu09BWSikH0dBWSikH0dBWSikH0dBWSikH0dBWSikHcRX7\nBCT1mkKllGoFEWHmz4oe2rlOvDtIThKRSQUuxxb6WUpPe/kcgH6WUpTv58jV4dXhEaWUchANbaWU\ncpBSD+0auwsooBq7CyigGrsLKJAauwsooBq7CyigGrsLKJCaYhyUxV57hKS0dkxbKaU6qlzZWeo9\nbaWUUs1oaCullIO0ySV/HRHJPQEMBLBARFbaXY9Sqn3QnnYReP2BX/uDZXOG7Lv/VI/Pv8Dldk+0\nuyalVPugE5EFRnKoLxD85I5navydu/XE6iVf4HfnnhhJxGLdRaTB7vqUUs6gE5FtZ8AeA4fEOnfr\nCQDoO2QvBMoqEgB62VuWUqo90NAuvAVrli7yrFg4HwAw7723EG5qtACstrcspVR7oMMjRWC6XGea\npvmwL1BmRcKhZDwaOVlE3rK7LqWUc+TKTg3tIiEZANATwFoRidpdj1LKWTS0lVLKQXQiUiml2gG9\nuUa1GkkPgLgU++taHkj6ABwBgADeEZEmm0tSKi/a01YtRrJ7oLzifRpG2OV2h1wu98V215QNySpf\nIDivz+A9n+4/bO8nvf7A5yR72F2XUvnQ0FYtFigrf+LICWfu/8j7S42bpk7z+YLBu0keanddmbz+\nwI2HHH9y/1uemF5+09RXK751+jm9/MHyO+yuS6l8aGirFotGIgd/59JfuE2XC70HDsERJ53hBnC4\n3XVlcnu8e4485CgPmZrLGXnwEW7T5Rpmc1lK5UVDW7WY2+OpXb7gUwCAZVlYMu/jGIAN9la1o1gk\nPOO1Jx8JxaIRJOIxTH/ykXAiHnvb7rqUykfel/yRXAGgHkASqUmpAzNe10v+2hmSx3t9/v+OPvIY\nWbd8iWxet3peuKmxWkTibVxHdwCHIvXn7y0RSWa87vEHy55IxOMn0KCYpmtmuKnxZBEJt2WdSrVG\n0a7TJrkcwBgRqW3JiZWzkRyG1FUZtQCeF5FEG59/tMfrqxk8YhRqN20w6mu/nBtuahwnIrEsbbsi\ndfXIl6V8pYtSzRU7tMeKyJaWnFipfATLK+adf9UNI46ccAatZBI3XnpW6IuPP/iliNxrd21KFUIx\nb64RAK+RnEXyBwU4nlK7lEwm+wwfczABwDBNjDjo8IBhmgNsLkupoivEzTWHich6kt0ATCf5hYjM\nbN6A5KRmT2tEpKYA51UdmGm6Pnr53/cffe6V17kb6mpR8+zjTVYy+b7ddSnVWiSrAVTvsl0hh/hI\nXg+gUUTuaPYzHR5RBUeyuy9YNs1KJIYnkwmaLvft0XDot3bXpVSh5MrOvHra6ZXsTBFpIBkEcByA\nP+RzTKV2h4hsIjkaQBWAUCIe1ytCVIeQ7/BIDwDPpG9ecAF4TESm5V2VUrshfSVI1glwpdorXZq1\nhJB0AzgZqd7jDBFZaHNJSimbFGV4RBUOSY8vWDajR5/++/QeOMSYXTMNJE8XkVftrk0pVTo6XGiT\nNAB0BVCX7UYMG53dd/CeI65/8L9BwzDw2Qdv485fXjIFwB52F6aUKh0dau0Rknt5/YGVXn9gpely\n17s9novsrqmZHoNHjPIaRup/yYDhI5CIx6psrkkpVWI6VGj7AsGXJ17xu95T3lnou+WJ6V63x/s3\nkiPtrivt7ZnPPxVbu2wxEvE4npx8e9zj9b1rd1FKqdLSYSYiSQYM06x/9MPl5valOu+6+keNH772\n4s9E5CF7q0txudwX0eDfk8mk1xcIfhBubDhZRL60uy6lVNvTPSKBsGm6QovnfQwAiIRDWPrpHAJY\nZW9ZX0sk4g8m4vGgWJYv1FB/qAa2UipTh+lpAwDJk7w+/+NDR45JrF660IxFwk+Fmxq/79SV30hW\nAfAC2FDoz0Cyi2EYl5kud9d4LPqCiEwv5PGVUjtXtFX+Wntiu5AcAGB/AOsBvO/EwCZp+AJl/5dM\nxM41XW6L5PxwU+OxIrK1QMfv7PUHPj1w3PhuvQcN9bz4yH2hUEP95YlE/IHtbTxe31Ukr7XEcrtc\n7scioabLMtfTZmocag8AkVyrQCqlstPhkTQRWSEi/xWR95wY2ABA8qIeffqddc9rcz33z/jcd/Bx\nE/b1l5XfV8BTTNz34COrfnzDXzwnX3gZrvn7owGXx3Pz9hcNw/huRecuk/7075cr/vr8u/7+w/ae\n6AsEb8yoscofLJvtC5QtcXu86/zBsinpyy2VUnnQv0QO5A0ED6s+9exgoKwchmHg2O9e4CF4UAFP\nEezSo9dX1/B37tYdVjLp3/7cHyw/7dSLfxboPXAIOnfrgbOv+G3AdLlObX6AQFnFfYccf8o+98+Y\n77vntTmeHn36nwmyJHdtV8pJNLQdKBYOL5z3bk3EsiwAwKfvz7RALCvgKV6qeXZq/KM3XsaapQtx\n7/W/DLvc7qe3v5iIxzauWbboq51q1i1fIgA2ZxzjoGO/e77HMAwEyspRferZQV8gWHI7tivlNB1u\nTLs9IOn3BctmVHbuslewopO1dtmiaDQSPlRElrTgGG6vP/Bnl8t9OoCtTQ3bftF8HXSS3wqUV/xN\nLOkMyHPhpsafi0gk/VpPj8//yajDqsuD5Z2Md15+JhGLRo4Skdnb3x+sqHzjlO//9KgJF/7YsJJJ\n3P7zi8Lz3nvrD8lk4pYstZQD6A9gjYjU5fWbo1Q7oROR7QxJF4DDAPgBvCci21ryfn+wbHK/ocMv\nOO9X1wc2rl6Bf95wdSgWCR8gIp/v5vm7AvguAA9Se0QuzXh9sNfnf3ePgUN8jfXbjIa62vmR1Oa/\nkYx2J7i93ifLO1VZDVtrXclk4vvJROLxlnwWpdojDW31DV6/f9ttT71Z0W2PPgCAR267PvHKfx68\nTkRuKtQ5SFYAOBhAGKl/WBIZr5e7vd71v7t3anDYfmOxatECXHfhKaFYJDxERNYXqg6lnEivHlHf\nYNCINdV/PRJRv3VLEkAk9ztaTkTqRWSaiMzMsVt7//JOVdaw/cYCAPoNG44effvHAQwtZB1KtSca\n2h1UPB7//S0/PS/0yn8exJSbr03Mqpm2DcC/2riMNQ1ba12rFi8AAGxcvQKbVq/0AFjexnUo5Rg6\nPNKBkZzgCwRPS8TjmxPx2F12DEmYLtf3XG7PAz37DohvXL3CY4n1y1gkck+6Pg+AcgC1Tr2mXqnW\n0jFtlTeSewE4CsBWAM8Waj1yknsgNSSyXERWAYDb4/2xiHWXabpgmObqSKjpWBFpcQ+cZCWAaOYE\nqFKlTse0bULyZH+wbJ3H62sMlFc8l56ccxyS3/b6/bMPG3/anQOHj3zAHyx7h6S3EMcWkXUi8tb2\nwCZ5oNfnv/22p97wTHl3kefUSy4f6A+WPdfCeqsCZeXvu9yezYZpNvj8gZu5fXlHpRxMQ7uISI7y\nBYJTr/rrlF53v/R+cP8jxh0XKCv/j911AaneLcljSA7Znfa+QPCBK+98IPCTG+8O/PHR58v6DR0+\nHMDEIpV3wNhvfZs9+w0ESYw/5xIjEmrauyW3wQfKKu4/+LgJox96d5F78rTZrsqu3X8K4Mwi1atU\nm9HQLq5xR5x0hjl8zCGorOqKC675ozcaiYyzuyjDME7z+PyLB+098ilfIDjPFwj+dlfvScRjVQP2\nHLH9/Ri0z35eAD2KVOLqRXM/SsZjUQDAwrmz4Pb6akXE2t0DWJZ12InnXeoxTBMVnbtg3OnnBN0e\n72FFqlepNqOhXVxb1y5bHN8+b7Bh1XK43O5GOwsi6Tfd7seuf+DpwI3/erHyjmfe8huGeW16vDon\nj9f39hOTb40n4jGsWboQM55/MgZgRpHKfKHuy03Trzz1yMabLjun/rYrLgzFIuFzWnIAwzTWLJzz\noQCAZVmYP+vdcDwWXVGUapVqQwWZiCRpApiF1G3IEzJe67ATkST9vkBw1pB99x/Qb+henppnp8bC\nTY0XW5Y1tY3roGm6LvaXlV9kWcmEWNbYB2Yu+GoBqEnfP23bok9mTRSRl3ZyjC7+svJnIk1Nh7nc\nrrCVtH6WSMSnFLNmAEcC6A7gQxFZ2cL3j/T4fDOGjhzDus2bWLtp/aJwU+PhOiGpnKKoV4+QvBLA\nGADlInLy7py4oyAZAHAugC4A3hCRD9q6BrfHc0Wnrj3+dP5Vk4LbtnyJR+/4A875xbU49szzsWbZ\nIvz+vAnhaDi0z+5cnZH+B9pywiV4JHsAOBxAE4DXM9f7VqqUFS20SfYB8BCAPwG4UnvapSdYUbn8\n6rsfGTBsvzEAgCfvuV2ef/jeeLC8ItbUsM0llvXDRDz+qM1lKqWayZWdrmyNW+gvAK4C4MhL2ToI\nsazkV0+sZBIQa/K2LZvvBbBeROrtK00p1RJ5hTbJkwBsEpE5JKt30m5Ss6c1IlKTz3lVy0RDoT/f\nfc2P/zrx59cGtm3ZJC8/dn8oEY/fJyIL7a5NKZWSztDqXbbLZ3iE5J8BnAcgAcCHVG/7aRE5v1kb\nHR4pAYZhnBWsqLzQSiYbQo0NN4rIPLtryiY9AVkBoFFEkrtqr1R7VfTb2EkeBeBXOqatWovkIF8g\nOC0Rj/UTQVIs64fJZELH2lWH1Fa3sZf8FQWqdPkCZS+d/sOfD3z4/aXum6a+4vP6/feSHGl3XUqV\nkoKFdnrtiJN33VK1RyQ9/rLyR1xud9Tj9TV6vL5ft/D93mgkNHT8uT80SKLPoGEYdfi3BMABRSpZ\nKUfSOyJVQfgCwVsH773fGZOnfey55YnpwYqqLr83DOO7LThEzOVyNy2b/0nqSTSCpfM/EQBri1Kw\nUg6lS7Oqgiir6LTkmn88OnjIiNEAgOlPPoKpf7v5X6GG+vN29xgkT/b6/f/Z54DDkqsWL2BT/bZX\nw02NZ7bFjTwkh/iDZZNMl6truLHx8WQy8ZATbiBS7Vcxr9NWCgLZvGbJwq9Ce9WiBfFELLqhRccQ\neY7kqI9nvHYAgA0A3myjwO7j8fpmnXjepeU9+vY3npx8++FbN2/sAeDmYp9bqZbSnrYqCJJjPF5f\nzUHHnOhqrK+zPv/o3bpoJDxKRDbbXduukLzq6FO/d+MPrrvNAwBrli7E788/uTYSaupid22q49Ke\ntioqEZlNcuTMF58eDyAK4CkRqdvV+0qEYbrdX/3lMN0eQETne1RJ0tDuAEh6fIHgXQDOMAyjKdTY\ncKWIPFPo86QXnPpHoY/bBp6c8dyT1/YeMMTVve8A/uevf2oSYLLdRSmVjQ6PdAD+YNnkgXuPvPCS\n393s37JhHf7yqx+Ewk2Nx4jIe3bXVipIjgyWV95Mw+gSDYeeiMeif2nJpgtKFZpu7NuB+YNlm/70\n2EvdevUfBAB46t475NkH/n5LMhH/jc2lKaVy0I19OzAaRtOWDeu+er553Zq4lUxus7EkpVQraU/b\nAdKLKB0HYCiAeSLSom2+SJ7qD5Y9dsI5l/g3rV0d/+j1l2qjkfC+IvJlUQpWSuVNh0ccLFBWfm+g\nvPLcfQ8+wvh4xmsSDYdui4SaJrXkGCQPMUzXBCuZrAfkfg3s/JDsGSgr/wcNc2/LSs4KNzZcLiJb\n7a5LtR8a2g5FcnigvGL23S++7w+UlWPbls24/KRDovFotK8TroFuj0j6vP7AgmPOPK/32Orj3TX/\nezz6wfQXF0RCjWN08lIVil6n7Vzdu/bsHQuUlfsBoLJLNwQrOsXqNm/sCkBD2x5jOnfrUTXxit+5\nSWLoyDHe2TWvDgMwCMASu4tT7ZtORJa+TzesXs6P3ngZiXgcNc9OlXBDfQTAMrsL68DisWjEECvV\nqU7EY0gkEgaAmL1lqY5Ah0ccgORBXn/g6WgkvIfPH1wWCTWeIiLz7a6royJp+oNl7+59wKH7jq0+\n3j/j+SdDyxd89mYk1DhBF5lShaJj2u0ASUPHTEsDyYDb4/21x+cbEQ2HP0zEY3eISNzuupQ90ld4\n9QfgbfZYLyJr8jimhrZSqv0gOQzfDMntj1eyfeMh+Wuk9rLNbP/DbJ0hkh8B8Ge09QDomrl/aTq0\nlwGII7X2ThTAP0RkSh6fTycilVKtR3If7Bh6HgAv5Ai9awEEsGNIXiQiiSztZwMoy9K+s4hkmy94\nBqktDqMZj9eQCs9M5UhtQt4A4Mtm7XN1Kn+E1DxF5vF3+KzpfyQG5jhOQWlPWykbkTSQ+jufrWe4\nH3bs6XkBPJsj9Cbhm6HnSf/3QhGJZmk/F0BlluNXikgoS/vZSAVc8wCLATgjx/F/j1TAZYbeYznq\nH4Fv9lS3H7+pI84V6PCI6vDSX2HdAOI5QnIssvcMn8w2Xk3yjwAqsrQ/V0TCWdrPA9Alo60bqZCs\nz9L+XQAmdgy9XMe/Gtl7nk/lqH9P7BiS0Vy/P6ptaWi3MyTH+QLBR+PRaFeP3z873NjwHRFZt+t3\nto10QLqQCqasPSWShwAI4pu9Qi+AqTl6bjcB6IQdQ3KiiDRkaT8fQPdmx/cASALolm2tb5Kvp9tk\nhtglItKUpf3l+GbPc/tX6eeyfZ0nORBZepIakCobDe12hOQAj8//2ZV3/F9w6MgxeG7KPxKvTp3y\nRSTUdDBSAVWXY4zxCGQfM/xXjp7brdixZ+gFcFaO0FsAoFezdtu/GvfPdts8yReRGiPNDMmf5gjh\nHyF1b0Fm+1dyhGRvfB2SMQBRvfpGOUVRQpukD8Bb+Lon8z8R+U1Gm3YX2iRNfDPENmfOJqfbHY3s\nX58fytFzuwOpnqEno/0ZIlLbrN3Zo48Yd9+GVcvLt2xch3gshvSNHmGkAmpPEdmU5fhPIdWzzZxc\nuTLH1/PvI/vX8zdz9IS7o1lPMtvviVJq9xTl6hERiZA8WkRCJF0A3iZ5uIi8nc9xm0tP1GSG2Poc\nExnHIfvEyv05em53AeiZpf3pOUJvMVIzxJm9vVEA1mcp/wdI9WwzQ8+d4+PORur/SWaoZgb8lvUr\nlnHSQ8/C4/OhdtN6XHPGMbFkMlG+s6AUkTNyvZajfYsuV8r2e6aUKqy8L/lrNsvsQapXVruT5i1C\ncilS6zlkhthBANZmecv3kOrZZoZkrs/5Tvq1zPa51poela4lsTvjkCIycVdtMtr/ezebvratdvP7\nN1028eA9R431vPvKcwkaxlWS0J6tUu1d3mPa6Z7wxwAGA7hHRK7OeL3VwyMkPdCZ7KzSQzRnAugN\n4H0RecfmkpRSBVS0m2vSEzujSFYCeJVktYjUZJx8UrOnNZmv7+TYugBPDulhkKl216GUKgyS1QCq\nd9mukJ3Y9MX0YRG5vdnP2t1EpFJKFVtR9ogk2ZVkp/Sv/QCOBTAnn2MqpZTKLd/hkV4AHk6PaxsA\nHhWR1/MvSymlVDZ6c41SSpWgogyPKKWUalsa2kop5SC6nrZSSuUpy9IWXgD1IrK10OfS0FZKOUr6\nwodsO9a05JG5NEa+x8q2kNmtAP5e6M+voa2UyinH2j9tHYqZx8u29MTuPLLtQtOE1NIb+R5vt5a2\nKAQNbaVKRHoN8paEXTGDcfvDjexht7uh2PwRBlC3m8fa2fE69NIWGtqqQ8rYpKEtHzsLWg+y7yTT\n0nDc/qgvwLF0k4YSo6Gt2kR66d626jHubnhm27+wtY/Gnby22z1V3aRB7YqGdjuUYya70BMvLX0P\n0Powy3xs2812Oz2+btKgnEjviMxTxky23cG4/bF9Jru1oZhvjzFbQO6waYVSKreiLc3alloxUVPM\nYNz+yLbTTGsfTQC2FuA4bTaTrZRqWyUd2iQ/QGpzhe0B6cbuTdTsTm8xgt3/mr2zR4eeyVZKta2S\nHh4h2RNAEl8HrU7UKKU6hKLsxp7PiZVSSuWmq/wppVQ7oKGtlFIOoqGtlFIOoqGtlFIOoqGtlFIO\noqGtlFIOUtI31yillNM0W9rCEpFooY+voa2UcqwcmzSsyXaXMsmTAPiw41IUd2e7aY/kQ1naewAc\nldk+vcRGfbr99k0arkNq95qCyiu0SfYF8AiA7gAEwD9F5O5CFKaUyi4dEOMADAAwR0Rmt+F5MwPS\nA2B5jpA8DYAfO4bkHdlWWCT5CIBglvaHi0g8S/vGdPvMtX8GIrXcRaZLkLrDOnMpCgOpZXozvYHs\ny2bsQESE5B5og6Ut8rojMn2beU8RmUuyDMBsAKeKyIJmbfSOSKVagaQbWRYp85m8ptxrnrlXtwA+\nXtfIaFJ+FU9a95D8LoBAlvfckiP0/g2gPEv7Q0QkkqV9CDtu9RUDMDxH+6lIdeYyQ++3Oeo5C0Ai\ny/E/zNET9qIdb9LQJrexk3wWwN9E5PVdnVipUkHyAACDACwGsAI7htj8HD3DiQDK8M1epxfAn7ON\nZZJ8HECnLMc/UESasrSvQ2qRtOYhxkqv0eW+CUOCfreBDY0x/OTFZbGEhc4AJgMgdgzJG3LUcxqy\n9zw/yfF5DV37p+0UfWlWkgMAjAbwQaGOqdo3kp2QfYxxbrb1t0mej+w9wxtEJJyl/dMAqrDjMrxj\nRaQeAAJu87YKr3nZ0CpfYvb6pnKk9jFswDdD7ND0zzIdCcDEN3uFUaSCM5uHkX23nB16qQAgIp2y\nfKYJfSq8j/rdqQu/epZ54DGNZMKyqkTkwhznzUpEnmlhew3sElCQ0E4PjTwF4AoRaczy+qRmT2tE\npKYQ51UtQ7IbUmOMmeOSs3J8Xb0IQCV2DMnrcvQMnwHQLUv7/UWkNktJHyM1JpkZYuOQ2gA20xjs\n2PNsQO6QnIyvv543H/dsStc7osxjXDb5xEGBcq+JdQ0x/Oyl5WbCkiHZ/hxnEpEf7apNRvuXWtI+\nhzlLaiOuzzaFsHc3P6YtqRNLZCuA9QU4trIRyWoA1btsl+/wSHrc7QUAL4vIXVle77DDI+mJiWwT\nMe+JSCxL+x8C6IwdQ/Xa7T3DjPb/A9Ary/H3E5HNWdrPB1CBHSduxmcLVZK3pmvJDNV7coT2kcg+\nhrmqFLf2IvntPbv4pt563IDK7T879+nFTQ2x5L4istzO2naG5LEek4/Hk9LJ5zKWhxPWSc3nkVT7\nUJThkfRs8gMAPs8W2G2NZH9kn4iZmWOi5McAumRp/xtJ9V4y278AoE+W9iNFZF2Wkp5HKoQzQ+x0\nAF9mad8HqeGCzE0acgXeTcgektl6tRCRfXIcJysRubqF7We0pH0J+HRFXdS98Msw9uzqxzur6hFL\nWhEAa+0ubGdEZDqAKpKeUDy5wz/+qn3L9+qRwwHMADAPqfAAUoH3SrM2+WyC8BKA/tgxJEeIyOos\n7d9BKoQzQ2yiiGzK0v53SIV8Zvt/icgOY5gk90eOkCzFnqTaNZInug0+TsJNYFs0KSe01SV0Su2M\nIzdBIDki/cvMkKzXSRFVKOnd6zsj9Y+v/rlSJcGRoa2UUh2V7lyjlFLtgIa2Uko5iIa2Uko5iIa2\najdImj6X8ecyj7m8zGN+RvJ4u2tSqtB0IlK1G363cWufCu9PfjS2R6A2nMCd760LRxJylIh8ZHdt\nSrWUTkSqdo/geZcf1CswtIsfB/Upxyl7VflM4jt216VUIWloK9uQDPrdxp0VXtdbfrdxN8nyfI5n\nEOH66NfrTNVFkglLEMq7UKVKiA6PqDZB0ofUYlIbRCRO0vC7jHdG9wqOOmpAhe+dVQ3Rj9Y2Lggn\nrAOyrfC3OwzyrKDHePCMvbsEvgwlktOW1m2LJWXfHEsM7E7N/ZFaUH+hiOiCTKpN6c01yjYGearL\n4L89JiVuSSyWlBMB1FZ4zVkPnTokaBqEJYKL/7e0sTacOFJE5rT2XCTHeV08M2lJXcLC30VkTWuO\n43UZVxC4aY9yT2xdQ8wdt+TCpCVPtrYupVqq6OtpK5UNyd5ek4/9+Zj+/iFVPsxa1xi45e21NSZR\nm7TEv2pbFAM7+1JtC3C+9AYcr++y4c5rHuRz8aa/jR/k7x50+5dtjeDqaSsfJvlKtjVplGpLGtqq\n2PbuV+mND6lKBfPYPcoQdBvunxzQs8fmcAK/eW0VfjS2B+ZubIqG4tYyAJ/aWy4AYGDvcm+se9Dt\nB4BBnX0IeoxkXSS5B4CFNtemOjgNbVVsq9bUxzxbwwl09ruwelsU0aRgnx4BBNwm3lvdEJv80YYV\nJN6IJOTXrR3PLrBFa+qjnlXbouhX6cX8TSGE4hYA7LCypFJtTce0VdH53cZ1JnlN/07exLKtkYoL\nRnXD+KFVEBH8/JUVDSvqohNF5AW762zOZXCiQd5f4TUTDbEkY0k5HUANgL2RWt/8c10RUBWTTkQq\nW5HcB8Agt4GTuwU9Z48f2in42aZQ5JMNoSXhhDU228azdkvvYdkbwEoAHr/LmBlwG/0SljCWlLnh\nhHVstr0plSoEDW1VEtK7HU30uVgdS8pyS/DXbFuXlZqgx3zk8H7lZ112QE+PJcDNb6+NzN3QdGc0\nYf3O7tpU+6RXj6iSIKlewmPph2OYxMgj+lV4SMIkcHi/ct/nm0Oj7a5LdTx6R6RSuyFhYe6MVfUx\nEUHSEsxYWR+OJXVbMtX2dHhEtQrJgMfktV6To6NJ+TiWlBudOL6bHq5xi8hON8gl2dnvMmb4XMaA\npAjjSZkTTljHOfEzK2fQMW1VMCRNv8t4e2TPwKjD+1b4ZqysD3+2KTQrnLCqnXRFhcvgxST+nrTg\n8buNWaG4NQFAF4M4WwRJAR4VkWXb25N0ARiO1NUjXzjpsyrn0dBWBUNyZCef+e6Dp6RuQU9aggue\nWRJqiCXHiMgXdte3O0geUuYxXrvl2P6BXmUePDR3U/zVJXWfCrDniUM7+xOWyKtL60KxpBzolM+k\n2hediFSFZJqkGOk/TiRgGhA468/T4dUDKt19KrwAgO+N6OqevrRu/wtGdccJQzsDADr5XMGnF2y5\nDsBEG+tyPW2nAAAN0klEQVRU6hvynogk+SDJjSRL4fZj1TY+C8WtVfd8tDH2yYYm/OPDDdFoQpYC\nWGB3YS2wYdGWcMxKf9NcUhuByyC6BdxfNegadBkm0cmuApXKJu/hEZJHAGgE8IiI7JvldR0eaYdI\nVvldxp0uA/slLMwJJ6wrRaTO7rp2F0m332XUdA+6R/au8HjnbmhyVw+owILNYVxxcC8kLMHNb68N\n1YYTP0xa4qjLE1X7UNQxbZIDADyvoa2cJD2xeJJB/GT80M7jLtm/O/+7oBYvL96KpriVjCXlynjS\nuruVx3YDOBNAdwAzRfTyQNUyOqatHIWkKSLJPI9BAF0B1IlIPPP19OJUz5JcOX1p3dsDOnkDfSs9\nINkUS8pv8wlsv8t4s3eFZ7/BnX3umavqLdPgD7THrgqhTXraAP7Q7Ec1IlKT90lVu0RyrM/FZ6IJ\n6e11cU0kIae0ZlMEkvt6Tb4KoCopEEvkgqQlT+yk/aFlHuMGgmXhRPKBhIX7pZV/OUieMbCTd8od\nxw8oMw0ivR53QyxpVbTmeKpjIFkNoLrZj663ractIpPa4jzK2UiWe0xOv/ygXp0O7lOOd1Y19P3b\nh+tfJ9m3JeuTkDS8Jl+9dGzPnuMGVXL51gh+/drKKSQ/FpEl2d4jIu8COKZAH6Vr/05ew0xfXtO3\nwouEJcFCfHtQ7Ve6M1uz/TnJ67O109vYVSkZ3jXgMg7rVwHTII4cUIHOPpcJYM8WHqcrgKpxgyoJ\nAAM7+7BX10ACwKhCF5zDzPfXNODzzSFEEhYenrsp7ncbH2pgq0LIu6dN8j8AjgLQheRqANeJyJS8\nK1Md0ebacMLTEE2i3GuiPppAXSThAbC5hcfZmhTIiq0RDOjsQyiexPKtERNttImBiMwnefakN1c/\nGLekk89lfBCKW99pi3Or9k/viFQlxe827wi4jUtH9wwaH69vtMIJ62/huPWblh7HNPg9j8kH9uoa\nSKyoi5jRhDwaiid/XIyaC4nkuDKPcQUAaYxZd4jIDLtrUvbQ29iVY5A8Gqk1PuaLyFt5HGcIUkMi\nq0Xkg0LVVywkjwu4jWe+P7p7wBLBlDmbwpGEnJDP74FyLg1tpUpchdd87eL9e4w7emAlAGDakjo8\n/MmmFxqiyQk2l6ZsoNdpq3aB5GAAhwEoB/ABgNmtvTSv9NDcfsUJABipywRMu6pRpUlDWzmGQX7H\na/Jfe3b1e1dvizKelHhS5FWSpxbyygySBoAL3AZHxi35DMBDbXHlR0Msedd9szYcaBABywIe+HhT\nOBRv3Q0+qv3S4RHlCCRNl8H6W4/tHxhclboi5BevrIABhNc1xn8sIg8X6Dz0u4ypvcrdJx7WryL4\n3uqGprX1sVfCCevMtujRkzyl3GP8CoDVELNuEZGXin1OVZp0eEQ5XSUJ1+AqHwAg4DYxqLMPlohv\nfWN8cAHPM9ggJtx8TH+/12VgwrDOwYv+t+QEAMMALCzgebISkf8B+F+xz6OcS2+uUSWJZDeSR6aX\nSACArSaxedrS1EKCK+uimL8phCW1kbAAcwt46rKA20h4Xam/Gl6XgaDbTAAoK+A5lGo1HR5RJYfk\neI/JJ3qVeRIbm2IeEdwQSVg3k9zHa/J1AN2TAopIwm0akyMJ6+eFGrog6fW5uPg7w7vscUT/CvPt\nVQ3Jpz7/ckMkIUN1P0jVlvSSP+UIJD1ug7V//Fbf4PBuAWwJxfHTl5aHQnHrwPSdhkRqudNKALUi\n8mURaugfdBuPJkWGm+QXTXHrPBFZ0ez14QB6A/hURDYW+vxKATqmrZyjm9ukMbxbAADQJeDGoM6+\nxGebQkORutlGAGxMP4pCRFYCODLbawG3eVvAbfykV5kntro+6iJ5mohML1YtSmXS0FalZlPCktic\n9U3+0b2CWN8Qw9LaiAvA53YXRvLgSq952T9OHOQv95r++ZtCuP7N1U+R7NR+rhVXpU5DW5UUEYmT\nPPnPM9c8X+41WR9JugH8UkQW2V0bgMF7dvUny72p+1326R6AAH4AFQC22VqZ6jA0tFXJEZEZJHtv\nCSUGAFgvIlvsrints882hcyNjTH0KPPgnVX1MInaBFBvd2Gq49CJSKVawGMaPwVwe9BjxMNxKxZN\nynG6/6MqBr16RKkCIdkJqStYVopI1O56VPukoa2UUg6SKzv1jkillHIQDW2llHIQDW2lSghJM33X\np1JZaWgrVQJIdg56zBoCMZfBJrfJS+2uSZUmnYhUqgQEPeZLh/UtH3fp2J6ejU0x/Hr6qlBDLDle\n94fsuHQiUqkSlrDkiHNGdvO4TaJPhRfHDa70Icf6Jy1Fct+gx/zE5zLqgh7zLZK9C3FcZY+8Q5vk\nt0l+QXIxyWsKUZRSHY1Jblm2NQIAEBEs2hKJoACLYpGs8piccdHo7vtOPmlQ5YnDOh/qdxlvprdU\nUw6U1/AISROp3TyOAbAWwEcAzhaRBc3a6PCIUrtAcpzX5HMH9SnHuoaYtbY+tiScsA4RkUiexz1u\naJXviduPH1AJpP5BOOfpxaGmuDVcRFYVpnpVDMVamvVAAEu2rzVMciqAUwAs2NmblHK6dIdlPIBu\nAN4Rkby2IhOR10mOnrGyvhpAHYBnRSSWf6Worw0nzHhS4DaJhpiFaFJcABoKcGxlg3xDuzeA1c2e\nrwFwUJ7HVKqkkTT9LmNal4DrwP6VXs5a10iSZ4nIC/kcN72SYaFXM/wwFLfe+c1rKw8fs0cwWLOi\nvsll4J/xpGwt8HlUG8k3tHUNYdURnd6jzH3gnccPKDMNYsHmEK5/c/XDALrYXVgmEbFInrS4NnL+\nktrIYAFmA3jG7rpU6+Ub2msB9G32vC9Sve1vIDmp2dMaEanJ87xK2ann0C4+l2mkhhuHVPkQS0on\npgchba5tByKSAPCg3XWonSNZDaB6l+3ynIh0ITUROQ7AOgAfQiciVYGQ3DfoNv5PgD4AZobi1o9E\nxPbNBkiOCbiNGbcc2z/Qp8KDx+Z9mXhp8dY5TbHkgXbXptqPoq3yR/IEAHcBMAE8ICI37c6JldoZ\nkj08JhdePLp7xfBuAf53wZboh2sbP2yKJQty7XK+TIPnG8R9lsDjdRnzwnHrRBFZZ3ddqv3QpVmV\no5D87n49A/ffcHS/cgBIWoIzn1iYSAqqRKQkrnxIrxHi0TW1VTHoHZHKaULbIkls71Q0xpLbZ71L\nJiAlpWTqUR2D9rRVSSLp9buMD/ftERg2onvA9/LirU214cTkSMK62u7alGoLOjyiHIdkkMDPPC4O\niCbkLQBTS/HqDKWKQUNbKaUcRMe0lVKqHdDQVkopB9HQVkopB9HQVkopB9HQVkopB9HQVkopB9HQ\nVkopB9HQVkopB9HQVkopB9HQVkopB9HQVkopB9HQVkopB9HQVkopB9HQVkopB9HQVkopB9HQVkop\nB9HQVkopB9HQVkopB2l1aJM8k+R8kkmS+xeyKKWUUtnl09P+FMBpAGYUqBallFK74GrtG0XkCwAg\ndc9epZRqKzqmrZRSDrLTnjbJ6QB6ZnnptyLyfHFKUkoplctOQ1tEji3ESUhOava0RkRqCnFcpZRq\nL0hWA6jeZTsRyfdEbwL4lYjMzvG6iIgOfCulVAvkys58Lvk7jeRqAAcDeJHky/kUqJRSatfy7mnv\n8gTa01ZKqRYreE9bKaVU29PQVkopBynp0E7PprYL+llKT3v5HIB+llJUrM9R0qGN3bj8xUGq7S6g\ngKrtLqBAqu0uoICq7S6ggKrtLqBAqotx0FIPbaWUUs1oaCullIO0ySV/RT2BUkq1U9ku+St6aCul\nlCocHR5RSikH0dBWSikHKfnQdvq2ZiS/TfILkotJXmN3Pa1F8kGSG0l+anct+SLZl+Sb6T9Xn5G8\n3O6aWoukj+QHJOeS/JzkTXbXlA+SJsk5JB299DPJFSTnpT/Lh4U8dsmHNhy8rRlJE8DfAXwbwN4A\nziY53N6qWm0KUp+jPYgD+IWI7IPUgmc/cer/FxGJADhaREYBGAngaJKH21xWPq4A8DkAp0+2CYBq\nERktIgcW8sAlH9oi8oWILLK7jlY6EMASEVkhInEAUwGcYnNNrSIiMwFstbuOQhCRDSIyN/3rRgAL\nAOxhb1WtJyKh9C89AEwAtTaW02ok+wAYD+B+AO1hkbmifIaSD22H6w1gdbPna9I/UyWC5AAAowF8\nYG8lrUfSIDkXwEYAb4rI53bX1Ep/AXAVAMvuQgpAALxGchbJHxTywK3e2LeQ2vG2Zk7/iteukSwD\n8BSAK9I9bkcSEQvAKJKVAF4lWe203aFIngRgk4jMaSdrjxwmIutJdgMwneQX6W+reSuJ0C7UtmYl\naC2Avs2e90Wqt61sRtIN4GkA/xKRZ+2upxBEZBvJFwGMBVBjczktdSiAk0mOB+ADUEHyERE53+a6\nWkVE1qf/u5nkM0gNlRYktJ02POK0ca5ZAIaSHEDSA+AsAM/ZXFOHR5IAHgDwuYjcZXc9+SDZlWSn\n9K/9AI4FMMfeqlpORH4rIn1FZCCA7wF4w6mBTTJAsjz96yCA45C6oKIgSj60nbytmYgkAPwUwKtI\nzYg/LiIL7K2qdUj+B8C7AIaRXE3y+3bXlIfDAJyL1JUWc9IPp14Z0wvAG+kx7Q8APC8ir9tcUyE4\neWixB4CZzf6fvCAi0wp1cL2NXSmlHKTke9pKKaW+pqGtlFIOoqGtlFIOoqGtlFIOoqGtlFIOoqGt\nlFIOoqGtlFIOoqGtlFIO8v9dSx/Qx+MLEQAAAABJRU5ErkJggg==\n", "text": [ "" ] } ], "prompt_number": 1 }, { "cell_type": "markdown", "metadata": {}, "source": [ "This may seem like a trivial task, but it is a simple version of a very important concept.\n", "By drawing this separating line, we have learned a model which can **generalize** to new\n", "data: if you were to drop another point onto the plane which is unlabeled, this algorithm\n", "could now **predict** whether it's a blue or a red point.\n", "\n", "If you'd like to see the source code used to generate this, you can either open the\n", "code in the `figures` directory, or you can load the code using the `%load` magic command:" ] }, { "cell_type": "code", "collapsed": false, "input": [ "%load figures/sgd_separator.py" ], "language": "python", "metadata": {}, "outputs": [], "prompt_number": 2 }, { "cell_type": "code", "collapsed": false, "input": [], "language": "python", "metadata": {}, "outputs": [], "prompt_number": 3 }, { "cell_type": "markdown", "metadata": {}, "source": [ "The next simple task we'll look at is a **regression** task: a simple best-fit line\n", "to a set of data:" ] }, { "cell_type": "code", "collapsed": false, "input": [ "from figures import plot_linear_regression\n", "plot_linear_regression()" ], "language": "python", "metadata": {}, "outputs": [ { "output_type": "display_data", "png": "iVBORw0KGgoAAAANSUhEUgAAAXsAAAEKCAYAAADzQPVvAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAGMBJREFUeJzt3X+0XWV95/H3J0TKJaloAo0ItDCrpaUSQGFQKSWxIwRm\nVjVxtU5humRGBrEDAuWqQ0dWiT+p0htpQGhAsGhbKzorFKe1JFZvoYtVfkwJv4I/OouIDhAdgw1U\nxgHynT+efbknN/fenHPv3ufZPz6vtc7KuecezvluNnx5ePazn48iAjMza7cFuQswM7PqudmbmXWA\nm72ZWQe42ZuZdYCbvZlZB7jZm5l1wMLcBUxHkteDmpnNQURoutdr2exh5oL3RtLaiFhbcjlZ+Fjq\npy3HAT6WuprPscw2UPY0jplZB7jZm5l1QBub/XjuAko0nruAEo3nLqAk47kLKNF47gJKNJ67gBKN\nV/GhquPeOJJirnP2ZmZdNVvvbOPI3szMpnCzNzOrgKRV0tJN6aFV2evxNI6ZWblSc3/5Rlg/kl65\n8DnYuSYibq/4e2fsnbVdZ29m1lxLRmHdCJw98cIIXDIKVNrsZ+NpHDOzDvDI3sysdDvG4MKTgd5p\nnLGcFXnO3sysAmnefslo+mnHWNXz9cV3ztg73ezNzFrC6+zNzAY07KWTEkslrpH4mSo+383ezGyK\nyaWT605Nj5dvrKrhS0jiLOBhYBfwXBXf4wu0ZmZ7GM7SSYkjgOuAg4HVEdxd5uf38sjezGzIJBZK\njAL3Al8HTqiy0YNH9mZm06hu6aTE8cANwA7gDRH8Uxmfu9fv9WocM7M9lb10UmIx8EHgt4H3AZ+L\noNQG7KWXZmYZSZxBmpu/E7gkgh9U8z3eG8fMbOgklgFXAScC50awOVctvkBrZlayYjnlOcBDwOPA\n8pyNHjyyNzMrlcSRwPXA/sBpEWzJXBLgkb2ZWSkk9pW4DLgL2Ai8sS6NHjyyNzObN4mTSMspHwNe\nF8HjmUvag5u9mdkcSRwAXAGsBi4CvlT2csqyeBrHzGwOJN4GPEIaNL8mgi/WtdFDhc1e0k2Stkt6\nqOe1tZK+J+n+4nF6Vd9vZlYFiUMlNgIfA86K4F0RPJ27rr2pcmT/GWBqMw9gXUS8tnj8TYXfb2ZW\nGol9JM4H7ge2AMdGcEf63XC3Q56LyubsI+JOSYdP8yvfGWtmjSKxnLSc8gXglAgenfzdS9shT+yj\nc7KkNcNIphpEjjn790h6QNKNkl6R4fvNzPoiMSLxUeBrpNmKFb2NPlkyCuuL7ZDPJj2f2FOnPoa9\nGuc64EPF8w8DY8A5071R0tqeH8cjYrzSyszMeki8iTSan5iyeSJzSXuQtBJY2dd7q9wIrZjG+XJE\nLB/wd94IzcyykFgKXAm8Gbgggttmf//ENM763u2Qs0zj1GYjNEkHR8STxY9rSPtGmJllJyHgTNKM\nwy2k5ZTP7O2vi4jbJa0pkqyAnfPeDrkKlY3sJX0eWAEcCGwHLif978ZxpFU5jwHnRcT2af5aj+zN\nbGiKeMBrgUNIu1NWmhpVFe9nb2Y2DYmFwMXApaSpm3URPJ+3qrmrzTSOmVld5IoHzMXN3sw6YTJm\n8IB9YOOP4NiTqSgesI68N46Ztcp0d7NOrpj5wqnw4K/BN94KK94TwWe70OjBI3sza5GZ7maFY34P\nbh2BI4p3/t994OH/TFp10wlu9mbWIktGU6M/e+KFEdi2Di76efhuzsKyc7M3sxZbBew8FD5yMdww\nNuXGp7GspQ2Zl16aWWukaZylG2HzCBwFfPB5WP/WiH/5yuQFWoAdtbzxab689NLMOiKegZ3fh62L\nYMWjcM9HJ5p68WfrGny/PLI3s8ZrUjxglWbrnV56aWaNJrGGFA+4Dw2IB8zF0zhm1kgShwJXkybn\nz5pIjbLpeWRvZo0yJR7wAXriAW1mHtmbWWNMiQdcEcHWzCU1hkf2ZlZ7RTzgR9g9HtCNfgAe2ZtZ\nrTUhHrAJ3OzNrJYklgB/SJ/xgDY7T+OYWa1ISOIs0nLKZ0jLKd3o58kjezOrjSIe8Drg1cDqpsYD\n1pFH9maWncRCiVHgXmAcON6Nvlwe2ZtZVl2LB8zFI3szy0JiscQY8NfAVcCpbvTVcbM3s6GTOAN4\nGDgIOLpL8YC5eBrHzIZGYhlpFH8icG4EmzOX1Bke2ZtZ5YrllO8EHiLlAy53ox8uj+zNrFISRwIb\ngMXAaRFsyVxSJ3lkb2aVkNhX4jLgLuBW0kobN/pMPLI3s9JJnERaTvkY8LoIHs9cUue52ZtZaRwP\nWF+exjGzUjgesN48sjezeXE8YDN4ZG9mc+J4wGbxyN7MBtYTD/gicEoEj2YuyfbCI3sz61sRD/hR\nJuMB3egbwiN7M+tLEQ+4AXgQOCaCJzOXZANwszezWUksBa7E8YCN5mkcM5tWTzzgw8CzOB6w0Tyy\nN7M9FPGA15LiAd8awT2ZS7J58sjezF5SxAO+lxQP+HfACW707eCRvZkBIHECaTml4wFbyCN7s47r\niQf8KxwP2Fpu9mYd5njA7nCzN+sgiWUSnweuIcUDviOCH8z987RKWropPbSqvEqtLJ6zN+sQCQHv\nJG1D/BngnAh+PNhnaBUsGU0/7RhLf758I6wbSc8vPFnSmoi4vay6bf7c7M06oogHvB5YxBzjAVOj\n372xw4uPwvoROHvibSNwySjgZl8jnsYxa7ly4wGXjE429rNJz3/q50or1irjkb1ZixXxgNcD26gs\nHnDXd+DC/YGJ0f5zsHOs/O+x+VBE/S68S4qIUO46zJpqSjzgxVBOatTkNM763sa+Jj2fnMf3fH0e\ns/VON3uzliniAa8mrZu/NIKny/383S/QurHXh5u9WQf0xAP+EnCeU6O6Z7be6Qu0Zg03TTzgcW70\nNpUv0Jo1WE884AvAigi2Zi7Jasoje7MGkthP4iPA35JujnKjt1lV1uwl3SRpu6SHel5bImmzpG9J\n2iTpFVV9v1lbFfGADwJHAsdGcH0EuzKXZTVX5cj+M8DpU167FNgcEUeSRiSXVvj9Zq0isUTiJuBm\nYDSCtzsH1vpVWbOPiDthjyVfbyH9g0rx5+qqvt+sLXriAR8BniHFA345c1nWMMO+QLssIrYXz7cD\ny4b8/WaN4nhAK0u2C7SRFvjXb5G/WQ0U8YCjOB7QSjLskf12Sa+KiKckHQx8f6Y3Slrb8+N4RIxX\nXZxZHUgcT1pO+TSOB7RZSFoJrOzrvVXeQSvpcODLEbG8+PkTwA8j4uOSLgVeERF7XKT1HbRWZ1Vt\nFyCxGPgg8NvA+8GpUTaYLNslSPo8sAI4kDQ///vAXwK3AD9L2oXv7RHxo0EKNstppo3A5tvwi3jA\na4E7SStt5pwaZd3lvXHMSiIt3QTrTp0M6rgZuGRzxA9Pm9vnsYwU8n0i8O4INpdUqnWQ98Yxq5li\nOeU5wEPA48ByN3qrkvfGMRvIjrEUxTf3oI4y4gHNBuWRvdkA0tz8zjVwyeb06H++Xjro30l//G3Y\n+TB89VHmFQ9oNhjP2ZsNgfThi+E318Eiwd3AOaVc2DXr5Tl7s0wkDpC4Fs7/A3hKcBjwG6TVPBPL\nN82q52ZvVpEiHvARYCGcdBd8J3dJ1mG+QGtWsp54wKOAsyK4Q/rmKrjwDczjwq7ZfHjO3qwkEguA\n3wHWAp8CrojgJ5O/d1C3Vcs3VZlVrCce8EXgXU6Nshx8gdasIj3xgF8D/gQ4xY3e6shz9mZzVMQD\nbiBFBB4bwROZSzKb0V5H9pIulPTKYRRj1gQSS3viAd8bwW+40Vvd9TONswy4V9Itkk6X5Ll066Qp\n8YD/AhwdwW3pd1olLd2UHlqVtVCzafR1gVbSAuA04D8CJ5C2Kb4xIv5XJUX5Aq3VjMThwHXAIcC5\nEdw9+btqtj02G9S8L9BGxC7gKdK+9C8CrwS+JOnK0qo0q6GeeMD7gDuA43sbfbJkNDX6s0kP3x1r\n9bPXC7SSLgLeAfwQ+DTw3oh4vhjtfxt4X7UlmuVRxAPeAOzA8YDWcP2sxlkCvC0idrvXOyJ2Sfr1\nasoyy0diEfAh+o4HnP+2x2ZV801VZj0kTifNzQ8UD+i7Y60OfAet2V4U8YCfBF4P/E4EmzKXZDYw\n30FrNoMZ4gHd6K11fAetddaUeMBTI3ggc0lmlfHI3jpHYl+Jy4C7gI2klTZu9NZqHtlbp0icRFpO\n+Rjwuggez1yS2VC42VsnSBwAXAGsBi4CvjT7ckqzdvE0jrWexNt4KR6Q10TwRTd66xqP7K21injA\na4BfoogHzFySWTYe2VvrSOwjcQGwBXiAtNe8G711mkf21ioSR5MuwL4A/GoEj2YuyawWPLK3VpAY\nkfgo8HVSPOAKN3qzSW721nhFPOADwJHAMRFsiGDXYJ/h8BFrN++NY40lsRS4EngzcMFEatTgn+Pw\nEWsH741jrdITD/gw8AxpOeWcGn3i8BFrP1+gtUaROIK0BfHBwOo9U6PMbDpu9tYIEguBi4FLgT8E\nxiJ4vpxPd/iItZ/n7K32psQDvruKeECHj1gbOLzEGkliMfBBUjzg+4DPeZsDs5n5Aq01jsQZpAuw\nBwFHR+wtB9bMZuM5e6uVIh7wKlI84LucGmVWDo/sbU7KvgmpWE75TibjAY92ozcrj0f2NrDJm5DW\nTaxeOVnSnG9CKuIBNwCLgdMi2FJasWYGeGRvc1LOTUhT4gFvJcUDutGbVcAje8uiiAe8HtiG4wHN\nKudmb3Mw95uQHA9olofX2duczOUmJIk1wNXAV4D3R/B0pUWadYxvqrKsinjAq4GjSMspnRplVgHf\nVGVZSCyQOB+4H3gQxwOaZeM5e6tEEQ94PfAicIpTo8zy8sjeSiWxn8RHSPGAN+N4QLNa8MjeSlPE\nA24gRQQeG8ETmUsys4Kbvc1bWfGAZlYdT+PYnJUfD2hmVfHI3uakJx7w1Tge0Kz2PLLvuEF3r5RY\nKDEK3AuMA8e70ZvVX5aRvaRtwE7SsrznI+LEHHV03aC7V06JB3xDFfGAZlaNXNM4AayMiB2Zvt+A\ntN3BumL3SgBG4JJRYLdm73hAs+bLOY3j7RAaoCcecBmOBzRrrJwj+69KehHYEBE3ZKqj42bevdLx\ngGbtkqvZ/0pEPCnpIGCzpG9ExJ29b5C0tufH8YgYH2aBXRARt0taU0zdkBp9bCriAf8A+BPgnAh+\nnK9KM5uJpJXAyr7em3vXS0mXA89GxFjPa971MoMp8YDnOjXKrFlqteulpP0l/XTxfBFwGilk2jLZ\nPR7wb78JP/M0LP1EGUHiZlYPOaZxlgEbJU18/59FhOeDM5F4I2k55Tb4zd+FTRtSpizMN0jczOpj\n6M0+Ih4Djhv299ruinjAjwFrgN8FboGv3T4ZJA7MsBTTzJrHd9B2UBEP+AjwMtJ+Nl/wckqzdvPe\nOB0icQhwDSke8D9E8He7v2PuQeJmVm/ZV+NMx6txyiWxAHg36S7YTwFXRPCT6d87eJC4mdWDA8db\nqp/GPCUe8LwItg61SDMbGjf7FprcxGx975TLSytnJEaAy4B3AR8APh3Brlz1mln1ZuudnrNvrJk3\nMeuJB9wCHBPBk7mqNLN6cLNvlWUvk7iJFA94fgRfzl2RmdWDm31jTV058/f/D/7xGOBB0nLKZzIW\nZ2Y14zn7Bkvz9iddBlcdBUfuhAN+K4J7ctdlZnl4zr6FJBZCHE1aM38lsC6C5zOXZWY15WbfQI4H\nNLNBebuEAQ0a0F3ud7NYYgz4a+CPgFPd6M2sHx7ZD2DQgO5yv5t/C1wL3EGKB/xB1d9pZu3hZj+Q\n/gK6y9QTD3giKVBkc1XfZWbt5WmcmpKQxDmkYJfHgeVu9GY2Vx7ZD2Q4u0IW8YDXA4uA0xwPaGbz\n5XX2A6pyV0iJfYH3AxcDHwauieDFsj7fzNrNG6HVxGz/oZA4ibSc8jHgv0TweJYizayx3OxrYKZd\nKiH+AbgCWE0a0X/RqVFmNhe+g7YWplvJ848fBw4krZt/TQRPZyvPzFrNzT6bNwG/9gvAGRHcMfW3\nTowyszJ5GmdIUvM+YCP8zQgcB/zRC/DJNRFP/Y/p3ztzMImZ2XQ8jVML8QT8cBs8eTD8+lb46odm\nbt7Dv3nLzNrNzb5iEvuR4gHPg6WXwdIbIjY7HtDMhsrNvkI98YAPMlA84HBu3jKz7vCcfQUklpL2\nmH8zcEEEtw3+Gb5Aa2aD8Tr7IZEQcCYwBnwR+IDjAc1sWHyBdggkDgeuAw4BVkdwd96KzMwmedfL\neZJYKDEK3Efaa/54N3ozqxuP7OfB8YBm1hQe2c+BxCLHA5pZk7jZD0jidOBh4CBSPODN3rjMzOrO\n0zh9KuIBPwm8Hjgvgk2ZSzIz65tH9ntRxAO+k93jAd3ozaxRPLKfRREPuAFYjOMBzazBPLKfhsS+\nEh8A7gJuJa20caM3s8byyH6KIh7wemAbac38d/JWZGY2f272BYkDgI8Ba3A8oJm1jKdxAIk1wCPA\nvqR4wFvc6M2sTVo/sp9t90iJQ4GrgaOAs6aLBzQza4NWj+wn4/3WnZoeL98oaZXEPhLnA/eT9po/\n1o3ezNqs5SP76eL9/vxy4HJgF7Aigq3ZyjMzG5JWj+z39FrglhOAzwKnuNGbWVe0vNnvGEuRfrcD\n/wxsexHOPDuCP47AObBm1hktn8aJ++Bbd8IrT4EPfAM+danj/cysi1oZS1jEA/4WsA64BbjM8YBm\n1nadiiWUOAK4Fng1jgc0MwNaNmcvcTRwLzAOnOBGb2aWtGoaR2IBcJj3szGzLpqtd7aq2ZuZddls\nvbNV0zhzle6qXbopPbQqdz1mZmXr/Mh+ckuF9SPplQufg51rvETTzJqmdiN7SadL+oakb0v6rzlq\nmLRkNDX6s0mP9SOTG6eZmbXD0Ju9pH2Aa4DTgV8GzpR01LDrMDPrkhzr7E8E/ikitgFI+gvgrcCj\nGWqh2FLhZKB3GmcsTy1mZtXI0ewPAb7b8/P3gNdnqAOAiLhd0hq4pJi62Tnm+Xoza5sczb52V4SL\n5u4Gb2atlaPZ/2/gsJ6fDyON7ncjaW3Pj+MRMV5tWWZmzSJpJbCyr/cOe+mlpIXAN4F/AzwB3AOc\nGRGP9rzHN1WZmQ2oVksvI+IF4ALStMlW4Au9jX6+iv/StYKPpX7achzgY6mrqo4lyzr7iPhKRPxi\nRPx8RFxR8sevLPnzclqZu4ASrcxdQElW5i6gRCtzF1CilbkLKNHKKj7U2yWYmXWAm72ZWQfUdm+c\n3DWYmTVRo7Y4NjOzcnkax8ysA9zszcw6oFXNvl5bJ8+PpG2SHpR0v6R7ctfTL0k3Sdou6aGe15ZI\n2izpW5I2SXpFzhr7NcOxrJX0veK83C/p9Jw19kvSYZK+LukRSQ9LurB4vVHnZpbjaNx5kbSfpLsl\nbZG0VdIVxeuVnJPWzNkXWyd/E3gzaUuGe5lyZ26TSHoMOD4iduSuZRCSfhV4FvhsRCwvXvsE8H8i\n4hPFf4RfGRGX5qyzHzMcy+XAMxGxLmtxA5L0KuBVEbFF0mLgfwKrgf9Eg87NLMfxdpp5XvaPiB8X\nOwv8PfBe4C1UcE7aNLJ/aevkiHgemNg6uckat2VERNwJPD3l5bcANxfPbyb9y1l7MxwLNPO8PBUR\nW4rnz5K2FD+Ehp2bWY4Dmnleflw83RfYh/TPWyXnpE3Nfrqtkw+Z4b1NEMBXJd0n6dzcxczTsojY\nXjzfDizLWUwJ3iPpAUk31n3aYzqSDgdeC9xNg89Nz3H8Q/FS486LpAWStpD+3n89Ih6honPSpmbf\njvmoSb8SEa8FzgDOL6YUGi/SvGGTz9V1wBHAccCTQKOCboqpj/8OXBQRz/T+rknnpjiOL5GO41ka\nel4iYldEHAccCpwi6U1Tfl/aOWlTs+9r6+SmiIgniz9/AGwkTVM11fZirhVJBwPfz1zPnEXE96MA\nfJoGnRdJLyM1+s9FxK3Fy407Nz3H8acTx9Hk8wIQEf8M/BVwPBWdkzY1+/uAX5B0uKR9gX8P3Ja5\npjmRtL+kny6eLwJOAx6a/a+qtdtIae4Uf946y3trrfiXb8IaGnJeJAm4EdgaEVf1/KpR52am42ji\neZF04MR0k6QR4FTgfio6J61ZjQMg6QzgKtKFjhsr2FFzKCQdQRrNQwqY+bOmHIukzwMrgANJ842/\nD/wlcAvws8A24O0R8aNcNfZrmmO5nLQj4XGk/7V+DDivZ361tiSdDNwBPMjktMDvkfIkGnNuZjiO\n/wacScPOi6TlpAuwC4rH5yLiSklLqOCctKrZm5nZ9No0jWNmZjNwszcz6wA3ezOzDnCzNzPrADd7\nM7MOcLM3M+sAN3szsw5wszcz6wA3e7M+SPrXxY6KPyVpURGc8cu56zLrl++gNeuTpA8D+wEjwHcj\n4uOZSzLrm5u9WZ+K3RbvA54D3hj+l8caxNM4Zv07EFgELCaN7s0awyN7sz5Jug34c+BfAQdHxHsy\nl2TWt4W5CzBrAknvAH4SEX8haQFwl6SVETGeuTSzvnhkb2bWAZ6zNzPrADd7M7MOcLM3M+sAN3sz\nsw5wszcz6wA3ezOzDnCzNzPrADd7M7MO+P+LXE3wcUXQQAAAAABJRU5ErkJggg==\n", "text": [ "" ] } ], "prompt_number": 4 }, { "cell_type": "markdown", "metadata": {}, "source": [ "Again, this is an example of fitting a model to data, such that the model can make\n", "generalizations about new data. The model has been **learned** from the training\n", "data, and can be used to predict the result of test data:\n", "here, we might be given an x-value, and the model would\n", "allow us to predict the y value. Again, this might seem like a trivial problem,\n", "but it is a basic example of a type of operation that is fundamental to\n", "machine learning tasks." ] }, { "cell_type": "heading", "level": 1, "metadata": {}, "source": [ "An Overview of Scikit-learn" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "*Adapted from* [*http://scikit-learn.org/stable/tutorial/basic/tutorial.html*](http://scikit-learn.org/stable/tutorial/basic/tutorial.html)" ] }, { "cell_type": "code", "collapsed": false, "input": [ "%matplotlib inline\n", "import numpy as np\n", "from matplotlib import pyplot as plt" ], "language": "python", "metadata": {}, "outputs": [], "prompt_number": 5 }, { "cell_type": "heading", "level": 2, "metadata": {}, "source": [ "Loading an Example Dataset" ] }, { "cell_type": "code", "collapsed": false, "input": [ "from sklearn import datasets\n", "digits = datasets.load_digits()" ], "language": "python", "metadata": {}, "outputs": [], "prompt_number": 6 }, { "cell_type": "code", "collapsed": false, "input": [ "digits.data" ], "language": "python", "metadata": {}, "outputs": [ { "output_type": "pyout", "prompt_number": 8, "text": [ "array([[ 0., 0., 5., ..., 0., 0., 0.],\n", " [ 0., 0., 0., ..., 10., 0., 0.],\n", " [ 0., 0., 0., ..., 16., 9., 0.],\n", " ..., \n", " [ 0., 0., 1., ..., 6., 0., 0.],\n", " [ 0., 0., 2., ..., 12., 0., 0.],\n", " [ 0., 0., 10., ..., 12., 1., 0.]])" ] } ], "prompt_number": 7 }, { "cell_type": "code", "collapsed": false, "input": [ "digits.target" ], "language": "python", "metadata": {}, "outputs": [ { "output_type": "pyout", "prompt_number": 9, "text": [ "array([0, 1, 2, ..., 8, 9, 8])" ] } ], "prompt_number": 8 }, { "cell_type": "code", "collapsed": false, "input": [ "digits.images[0]" ], "language": "python", "metadata": {}, "outputs": [ { "output_type": "pyout", "prompt_number": 10, "text": [ "array([[ 0., 0., 5., 13., 9., 1., 0., 0.],\n", " [ 0., 0., 13., 15., 10., 15., 5., 0.],\n", " [ 0., 3., 15., 2., 0., 11., 8., 0.],\n", " [ 0., 4., 12., 0., 0., 8., 8., 0.],\n", " [ 0., 5., 8., 0., 0., 9., 8., 0.],\n", " [ 0., 4., 11., 0., 1., 12., 7., 0.],\n", " [ 0., 2., 14., 5., 10., 12., 0., 0.],\n", " [ 0., 0., 6., 13., 10., 0., 0., 0.]])" ] } ], "prompt_number": 9 }, { "cell_type": "heading", "level": 2, "metadata": {}, "source": [ "Learning and Predicting" ] }, { "cell_type": "code", "collapsed": false, "input": [ "from sklearn import svm\n", "clf = svm.SVC(gamma=0.001, C=100.)" ], "language": "python", "metadata": {}, "outputs": [], "prompt_number": 10 }, { "cell_type": "code", "collapsed": false, "input": [ "clf.fit(digits.data[:-1], digits.target[:-1])" ], "language": "python", "metadata": {}, "outputs": [ { "output_type": "pyout", "prompt_number": 12, "text": [ "SVC(C=100.0, cache_size=200, class_weight=None, coef0=0.0, degree=3,\n", " gamma=0.001, kernel='rbf', max_iter=-1, probability=False,\n", " random_state=None, shrinking=True, tol=0.001, verbose=False)" ] } ], "prompt_number": 11 }, { "cell_type": "code", "collapsed": false, "input": [ "clf.predict(digits.data[-1])" ], "language": "python", "metadata": {}, "outputs": [ { "output_type": "pyout", "prompt_number": 13, "text": [ "array([8])" ] } ], "prompt_number": 12 }, { "cell_type": "code", "collapsed": false, "input": [ "plt.figure(figsize=(2, 2))\n", "plt.imshow(digits.images[-1], interpolation='nearest', cmap=plt.cm.binary)" ], "language": "python", "metadata": {}, "outputs": [ { "output_type": "pyout", "prompt_number": 14, "text": [ "" ] }, { "output_type": "display_data", "png": "iVBORw0KGgoAAAANSUhEUgAAAIcAAACMCAYAAABWILrJAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAACZRJREFUeJztnV+MVFcdxz9f/jQUqGwMRmzZdHkopiaNC6nECLSLqYY2\n1kJiUkmMgIlPaqHGpugT+1TexMT4oi2lFttEKk2b2mitLJEYsbRMy99aGjYBCojaxbYEA/Lz4V6a\ndTtn59wzc2fu7P4+yWTv3Ht/9/xm5rvnnnN+5/6OzAzHqceUTjvgVBcXhxPExeEEcXE4QVwcThAX\nhxNkWrMXkOR94S7HzFRvf9PiyC9ed//mzZvZvHlz4euNZ7du3bqgXa1Wo7+/v+6xoaGhoN3IyAg9\nPT2FyxsaGmJgYCB4fOPGjXX3b9myhU2bNgXtQr6U8X1KdXUBRNxWJK2UdEzSW5IeLuyZ07WMKw5J\nU4GfAiuBzwBrJN3aDsecztOo5lgCHDezYTO7DDwN3Bd78fGq3DLs5s2bl2Q3Y8aMJLu+vr4ku2XL\nliXZtfv7bCSOm4CTo96fyvdF4eKoz0QRh/dEJjGNeiungd5R73vJao//Y3RLeGBgIFmpTvkMDQ2N\n23MbTSNx7AdukdQHvAPcD6wZe1JK98rpDGP/eQcHB4PnjisOM7si6bvA74CpwKNmdrQ1bjpVp+Eg\nmJm9CLzYBl+ciuGxFSeIi8MJ0pLYSgrDw8NJdtu3b0+yu/nmm5PsUscyJgJeczhBXBxOkJio7GOS\nzkk62A6HnOoQU3NsI4vKOpOMhuIwsz8B77bBF6dieJvDCeLicIK0ZJzDo7LdQ5GorGIepM6jss+b\n2W11jlnKw9ipg2ALFixIsksdBGsm4rxq1aoku9AE4zKQFJx9HtOVfQr4M7BQ0klJ61vtoFNNYqKy\nH5m/4UwOvEHqBHFxOEE6FpVNjXbOmTMnyW5kZCTJLrXhDOmfMdXXVuM1hxPExeEEienK9kraLemw\npEOSHmiHY07niWlzXAYeNLOapNnAq5Je8lnoE5+YqOxZM6vl2+8DR4Eby3bM6TyF2hz5MPoiYF8Z\nzjjVIloc+S1lJ7Ahr0GcCU7UOIek6cAzwJNm9uzY4x6V7R5aGpVVlhdoO/BPM3uwzvGkqGwq7YxY\nQjh1Uwxbt25NsmvnIFhTUVlgKfANYIWkA/nL55ROAmKisnvxwbJJif/oThAXhxMkaprguBdoc4P0\n2Wc/0lmKYvXq1S32pDFr165Nsnv88cdb68g4NNsgdSYpLg4nSExUdoakfZJqko5IeqQdjjmdJ6Yr\ne0nSCjO7KGkasFfSsryL60xgom4rZnYx37yOLHHcv0rzyKkMUeKQNEVSDTgH7DazI+W65VSB2Jrj\nqpn1A/OBOyQNlOqVUwkKzT43swuSXgBuB4au7feobPfQ6qjsXOCKmY1Iup4sYe2gmb2cH/dBsADd\nPggWU3N8CtguaQrZbeiX14ThTGxiurIHgcVt8MWpGD5C6gRxcThBXBxOkI49SJ1K6rzM1Aewm6GZ\nh7CrgNccTpDY4fOp+cTi58t2yKkOsTXHBuAIviDgpCJmPsd84B7gF0B4bWtnwhFTc/wYeAi4WrIv\nTsVotFz5V4C/m9kBvNaYdDTqyn4B+Kqke4AZwMckPWFm3xx9kkdlu4eWZzAGkHQn8AMzu3fM/rZG\nZVOFV6vVWutIBP39/Ul2sT9eK2jlowneW5lERI+QmtkeYE+JvjgVw0dInSAuDieIi8MJ0rGobGqL\nfM+etGbPtm3bkuyaWXR4xYoVSXapc0jXrVuXZBfCaw4nSGzCuGHg38B/gctmtqRMp5xqEHtbMWDA\nzPwxyElEkduKx1YmGbHiMOAPkvZL+naZDjnVIfa2stTMzkj6BPCSpGP5StXOBCZKHGZ2Jv97XtIu\nYAnwoTg8Kts9FInKNhSHpJnAVDN7T9Is4MvA4Ohzmll71WkvY/95BwcHg+fG1ByfBHZlWa6ZBuww\ns98356LTDcQ8K3sCSJuY4HQ1PkLqBHFxOEFcHE6QrovKtru8ZqKyqVTlGVuvOZwgMU+89UjaKelo\nnsH48+1wzOk8MbeVnwC/NbOv5RmMZ5Xsk1MRxhWHpDnAcjNbC2BmV4AL7XDM6TyNbisLgPOStkl6\nTdLP8+F0ZxLQSBzTyDIJ/szMFgMfAJtK98qpBI3aHKeAU2b2Sv5+J3XE4VHZ7qFlUVkzOyvppKSF\nZvY34C7g8NjzPCrbPbQ6Kvs9YIek64C3gfVN+ud0CTFR2deBz7XBF6di+AipE8TF4QRxcThBOrbo\n8MjISFJ5qRmMU6OyzURIUyO6qWvK9PT0FLbxRYedJGKisp/Osxdfe12Q9EA7nHM6S0xX9k1gEWSr\nRAKngV0l++VUgKK3lbuAt83sZBnOONWiqDi+DvyqDEec6hEtjnz4/F7g1+W541SJIhOM7wZeNbPz\nYw94VLZ7aOmzsqNYAzxV74BHZbuHIlHZ2MV4ZpE1Rn/TpG9OFxG7lv0HZjbXzN4rcvHUUcm9e/cm\n2aWOZqaO1l66dKmt5aV+L6m/Q6kjpC6O1pY3ocThdDcuDidIS6KyLfLF6RChqGzT4nAmLn5bcYK4\nOJwgpYlD0kpJxyS9JenhSJvHJJ2TdLBgWb2Sdks6LOlQ7HwTSTMk7ZNUyzMIPFKw3MIrdUsalvRG\nbvfXAnaFsx00PRfHzFr+AqYCx4E+YDpQA26NsFtONnfkYMHy5gH9+fZs4M2Y8vLzZ+Z/pwF/AZYV\nKPf7wA7guQI2J4CPJ3yn24FvjfJ1TkH7KcAZoDfWpqyaYwlw3MyGzewy8DRwXyMjy7Iiv1u0MDM7\na2a1fPt94ChwY6TtxXzzOjJRRyX/b3Kl7kLnj8p28Bhk2Q7MrGi2g8JzccoSx03AaCdO5ftKR1If\nWe2zL/L8KZJqwDlgt5kdiSwqdaXulDzyrch2UHguTlni6Ej/WNJssoe9N+Q1SEPM7KqZ9QPzgTsk\nDUSU08xK3UvNbBHZFIjvSFoeYdNUtoPUuThlieM00DvqfS9Z7VEakqYDzwBPmlnhuf15Nf0CcHvE\n6ddW6j5BNo3hi5KeiCznwzzyZHNxYxY2qpftYHFMeTnBuTjjUZY49gO3SOrLVXs/8FxJZaEs9/aj\nwBEzi36wRdJcST359vXAl4ADjezM7Edm1mtmC8iq6z/amCXcA+XNlHRDvn0tj3zDnpmZnQVOSlqY\n76qb7WAcgnNxGhVcVo/lbrJew3Hgh5E2TwHvAP8ha7Osj7RbRnbvr5H9uAeAlRF2twGv5XZvAA8l\nfM47ieytkLUdavnrUOz3ktt+FngFeJ1sXk1Ub4Ush9s/gBuKfjYfPneC+AipE8TF4QRxcThBXBxO\nEBeHE8TF4QRxcThBXBxOkP8BGQ6+Sh33LfcAAAAASUVORK5CYII=\n", "text": [ "" ] } ], "prompt_number": 13 }, { "cell_type": "code", "collapsed": false, "input": [ "print(digits.target[-1])" ], "language": "python", "metadata": {}, "outputs": [ { "output_type": "stream", "stream": "stdout", "text": [ "8\n" ] } ], "prompt_number": 14 }, { "cell_type": "code", "collapsed": false, "input": [], "language": "python", "metadata": {}, "outputs": [], "prompt_number": 15 } ], "metadata": {} } ] }