{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# An Introduction to Image Classification with scikit-learn\n",
    "\n",
    "By [Ellie Birbeck](https://elliebirbeck.com) for [Hyperion Dev](https://blog.hyperiondev.com/)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Introduction\n",
    "\n",
    "This tutorial is an introduction to machine learning with [scikit-learn](http://scikit-learn.org/), a popular and well-documented Python framework. We’ll be using [Python 3](https://www.python.org/downloads/) to build an image recognition classifier which accurately determines the house number displayed in images from Google Street View. You’ll need some programming skills to follow along, but we’ll be starting from the basics in terms of machine learning - no previous experience necessary.\n",
    "\n",
    "So what is machine learning? It’s an area of artificial intelligence where algorithms are used to learn from data and improve their performance at given tasks. For example, if we previously had wanted to build a program which could distinguish between an image of the number 1 and an image of the number 2, we might have set up lots and lots of rules looking for straight lines vs curly lines, or a horizontal base vs a diagonal tip etc. What machine learning allows us to do instead, is feed an algorithm with many examples of images which have been labelled with the correct number. The algorithm then learns for itself which features of the image are distinguishing, and can make a prediction when faced with a new image it hasn’t seen before. \n",
    "\n",
    "Typically for a machine learning algorithm to perform well, we need lots of examples in our dataset, and the task needs to be one which is solvable through finding predictive patterns. There are different types of tasks categorised in machine learning, one of which is a classification task. This simply means that we are aiming to predict one of several discrete classes (labels). An example of this could be predicting either yes or no, or predicting either red, green, or yellow. This is in contrast to regression, a different type of task which makes predictions on a continuous numerical scale - for example predicting the number of fraudulent credit card transactions. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Dataset\n",
    "\n",
    "For this tutorial we’ll be using a dataset from [Stanford University](http://ufldl.stanford.edu/housenumbers). It contains images of house numbers taken from Google Street View. Each one has been cropped to 32x32 pixels in size, focussing on just the number. Some examples are shown below. \n",
    "\n",
    "![house-numbers](http://ufldl.stanford.edu/housenumbers/32x32eg.png)\n",
    "\n",
    "We need to download the dataset and save it in the same directory as this notebook. There are three different datasets available of different sizes. In this tutorial we will be using the largest at 1.3GB in size, available to download [here](http://ufldl.stanford.edu/housenumbers/extra_32x32.mat). If you want to save on space or time, download [this one](http://ufldl.stanford.edu/housenumbers/train_32x32.mat) at only 182MB, but expect worse results due to the reduced amount of data.\n",
    "\n",
    "There are a total of 531131 images in our dataset, and we will load them in as one 4D-matrix of shape `32 x 32 x 3 x 531131`. This represents each 32x32 image in RGB format (so the 3 red, green, blue colour channels) for each of our 531131 images. We’ll be predicting the number shown in the image, from one of ten classes (0-9). Note that in this dataset the number 0 is represented by the label 10. The labels are stored in a 1D-matrix of shape `531131 x 1`. You can check the dimensions of a matrix `X` at any time in your program using `X.shape`.  \n",
    "\n",
    "Although this tutorial focuses on just house numbers, the process we will be using can be applied to any kind of classification problem. Autonomous vehicles are a huge area of application for research in computer vision at the moment, and the self-driving cars being built will need to be able to interpret their camera feeds to determine traffic light colours, road signs, lane markings, and much more. With this in mind, at the end of the tutorial you can think about how to expand upon what you’ve developed here.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Feature Processing\n",
    "\n",
    "Now let’s begin! To understand the data we’re using, we can start by loading and viewing the image files. First we need to import three libraries."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "import scipy.io\n",
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "%matplotlib inline"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "Then we can load the training dataset into a temporary variable `train_data`, which is a dictionary object. The dictionary contains two variables `X` and `y`. `X` is our 4D-matrix of images, and `y` a 1D-matrix of the corresponding labels. So to access the i-th image in our dataset we would be looking for `X[:,:,:,i]`, and its label would be `y[i]`. Let’s do this for image 10."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# load our dataset\n",
    "train_data = scipy.io.loadmat('extra_32x32.mat')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# extract the images and labels from the dictionary object\n",
    "X = train_data['X']\n",
    "y = train_data['y']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[2]\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAP8AAAD8CAYAAAC4nHJkAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAGjVJREFUeJztnW2opVd1x//rvNy3c9/mLZNpDI2BfAlSR7kEQRFbUVIR\nol+CUiQfgiOtlQr2Q0ihpt9sqYqUIh2b4FisGqpiKKElBiEIJXW0MRmNrS9ETJjMxJnMzL3nnvez\n+uE8ae/c7PW/555773NnZv9/MMy5e5/97HX2edZ5ztn/Z61l7g4hRH5U9tsAIcT+IOcXIlPk/EJk\nipxfiEyR8wuRKXJ+ITJFzi9Epsj5hcgUOb8QmVLbyWAzuxvAFwBUAfyju3+GPb/RmPXl5aXgYHSm\n7Ru3FzcuRndDWmzfBJZvDTmoBSayOznpUrE7QCd53ROv1aQreR3fwUrWKno/X710Gc3m+liLNbHz\nm1kVwN8DeA+AFwH8wMwec/efRmOWl5fwx3/yR8m+SiU2xSf5gjKMu2yCRQWAQb+fbK9Vq+GYaiXu\no45FqFTj9bBh+pj9Xtp2AOgHrwvg61GpxHZUgzWp1OK1r5D3uQqyjuRUj94zZjv7wPCJP4O2P9DI\n+9zr9ZLtf/cPp8Y+/k6+9t8F4Bfu/it37wL4OoB7dnA8IUSJ7MT5bwHwmw1/v1i0CSGuA/Z8w8/M\nTpjZaTM73Wyu7/V0Qogx2YnzvwTg1g1/v6Fouwp3P+nuK+6+0mjM7WA6IcRushPn/wGAO8zsjWY2\nBeBDAB7bHbOEEHvNxLv97t43sz8F8O8YSX2PuPtP2BgzQ7VaD/sihuHOPdk5Jjulg8Eg7HOyK2vB\njvOwH+8O16fjJa5Uya4y2WVna9UbpneBO/10O8DXY0BUgiEZV6+lX7eR3f5acG4AwMz0DBkXKwFW\nTc/nZEefKQFsz56/Z9u/zg7iEx+tTjvZPiRjNrMjnd/dHwfw+E6OIYTYH3SHnxCZIucXIlPk/EJk\nipxfiEyR8wuRKTva7d8uDqdySEQok5BDMfmKSWVMzIlUQGMBLrW4r0KCfqKAFADo97thXyeQgNjd\nlevrcV+7nT4eAAy6sXxYC6S+aj1ej3otlvoajUbYt7QURIoCaMymbywbDpncG3axQDtUKpOdVxZc\ng1vr8dqvrq0l27cj9enKL0SmyPmFyBQ5vxCZIucXIlPk/EJkSqm7/QYLd9pZlqMoLx2NsiA7r053\nXtm4NEOWFowcj+3M9shuf7sd77KvraZ37ldXV8Mxly5dDPuazWbY123Fu9GVQKGp1mOFY6oe7/Yv\nLsyHfU6ClmaP3pxsZymy2LlTmTAFHFMConFRqi4AaAfqjXb7hRBbIucXIlPk/EJkipxfiEyR8wuR\nKXJ+ITKlVKkPQCij7HapJi7JsHHb/zxkuexYNZweCYxh0txaENQBAOvN9LgrV+LjdbuxZMekvhYJ\nFvJIciJvZa0Wy4BN+ppjG3vd9PofuelIOKY+NRX31WOXYcFktVp8TI+qLJFzZy04P9i5uBld+YXI\nFDm/EJki5xciU+T8QmSKnF+ITJHzC5EpO5L6zOwFAKsABgD67r6yG0ZtJiqtxCLw9oJqVBaKmhF3\n9vuxLNMlMiDvG1/qGccOFiU2JEkUo74+iVTrdOPjMVmU5XI8dLCVnqsd50GsV4ksV5msjBqTlweD\n9Bp3251wTLeTtn8YyIYpdkPn/313/+0uHEcIUSL62i9EpuzU+R3Ad83sh2Z2YjcMEkKUw06/9r/D\n3V8ys5sAPGFmP3P3pzY+ofhQOAEAy8uLO5xOCLFb7OjK7+4vFf+fB/BtAHclnnPS3VfcfaXRSBdQ\nEEKUz8TOb2YNM1t47TGA9wI4s1uGCSH2lp187T8K4NuFvFED8M/u/m+7YtVmoqSfbMj2q4Jtecyo\nvJaR5IyDXiyVdQK5BgDW19MSFQC023HfIChDFcqUABYX43JXc3Pxt7VuN5aiorJha804Oq9FyoZ1\nOvFcq1euhH0XLlxIti8sLIRjZmZmwj6rkuSeZI2jyD0gjt4bTCSzliD1ufuvALx50vFCiP1FUp8Q\nmSLnFyJT5PxCZIqcX4hMkfMLkSmlJ/AMa/WRsKdJZDsSYEUjrFhfNZL6yGS9AUng2Yv7WKRdvx9L\nQJEcOT09HY5ZXorvvKxU4usDS/y5tpaW33574ZVwjAcyJQCsdWOJcK0VJ/B89dV0HcJDhw6HY5aW\nYumzSpKMVkhfFJkKxJGTbK7wnGMn/iZ05RciU+T8QmSKnF+ITJHzC5Epcn4hMqX8cl0lwXbtOWS3\ndAKlgpVw4n2xEsDy6tXr9WT73FwjHHPgwIH4eKR01XAQ59W7eDFtR7tNSoOtxYE9rUoczMQCpJrN\n9DgWHMWUlirrq6VfMwDQKnDRKUcVq+Ac2MaJryu/EJki5xciU+T8QmSKnF+ITJHzC5Epcn4hMqV8\nqW8CDW6SYCDWt0V9rZjgmGwuWu6K9DGpr9eLpa1KkE+Q2VitxacBCwiCx+NmZmaT7bPTcU7AuZm4\nrzMTv+ZhvFToddOdrKwZK3nV78XjpmNVFBYEXAEAguCpej1e32otPYYFmb1u2rGfKYS4oZDzC5Ep\ncn4hMkXOL0SmyPmFyBQ5vxCZsqXUZ2aPAHg/gPPu/qai7SCAbwC4DcALAO5191e3OpYDGEYyG1Hm\ntiNf/D+xjAaPj8dmsig0i0X1MdmIRPX1ghJOAC9dFR6vF0fgsUi7WjW+PlTIag0iSYy8LTWLZcVa\nNS6hVbXty4AVJ/nxSB/Ln0gDQklJt0pwfk8HcikA1Ovptdptqe/LAO7e1PYAgCfd/Q4ATxZ/CyGu\nI7Z0fnd/CsDmFKj3ADhVPD4F4AO7bJcQYo+Z9Df/UXc/Wzx+GaOKvUKI64gdb/j56L7R8IetmZ0w\ns9NmdrrZjDO1CCHKZVLnP2dmxwCg+P989ER3P+nuK+6+0mjE924LIcplUud/DMB9xeP7AHxnd8wR\nQpTFOFLf1wC8C8BhM3sRwKcBfAbAo2Z2P4BfA7h3/CmDzxtSk4tH6AWHYxkTSZ8xGTCQUQYkOo+V\nu5qdi78JzazF0tbF3oWwb309/dOq34+lPqsQ+2+Kt3MW5hbCvulaOsStYrGMFsqDAHqtWN7sB5F7\nAFAPkmo2V+OfoMNBfL4dOBgnO52aIpF7rBxdcF5NkYjKozffnGyvBQlck8/d6gnu/uGg691jzyKE\nuObQHX5CZIqcX4hMkfMLkSlyfiEyRc4vRKaUnMDTAN+9zxuiDm5hBZHzWHRhEKFXJZFUtWos/9RJ\nX5X0scitfj8d4dZqNcMxvd5i2NclyUJZFFu1kj61jJ1yRGbFkMmzpC+SdUmUHXtd9Rp5X9gxCZGU\nHUXuAcD8Qvo9Y+fNZnTlFyJT5PxCZIqcX4hMkfMLkSlyfiEyRc4vRKaUX6svgCbOnFDSKwsjkXs1\nUqOtHshhADA1FUdnsWivSOqrBdFtoz5yGpAISJbMsjdMR+g5SWjKgje5/XGRvPpUuq/RiCMSqyQy\nrk9qKFbZWUzOkahmYyeI0ASAVrO1rWMlTRr7mUKIGwo5vxCZIucXIlPk/EJkipxfiEwpdbffEJcm\n4uO2/xlVcbbryYI6JrEvpl6Pl9iH8S713Gyc3295eSnsi/LIVYnqMDc7H/axcf1+nHOv20nnDOyT\nMmRsJ31I6rlVyBpPz6VLXjWW4mCmWqAQjOwgQUTExgEplxblV7x88VI4Zm0tHag1IO/JZnTlFyJT\n5PxCZIqcX4hMkfMLkSlyfiEyRc4vRKaMU67rEQDvB3De3d9UtD0E4KMAXime9qC7P75XRl4rRLnW\nWE69WpUs8XQ8bn4hlt8GiOWchU4j2V4hATrz8/FclWoc5NLqxPn9oorMq+txLsFWOy7JRSpo0UCc\n+UDSayzGgT0VElTF8vQ5CarpduO1Wl9PB+l0u7E8GB1vSCXuqxnnyv9lAHcn2j/v7seLfze84wtx\no7Gl87v7UwAulmCLEKJEdvKb/xNm9qyZPWJmcelSIcQ1yaTO/0UAtwM4DuAsgM9GTzSzE2Z22sxO\nR78DhRDlM5Hzu/s5dx+4+xDAlwDcRZ570t1X3H2l0YjvVxdClMtEzm9mxzb8+UEAZ3bHHCFEWYwj\n9X0NwLsAHDazFwF8GsC7zOw4AAfwAoCPjTshk8XCMTRuLo3TeUi5LiKJhTnmWI5BmtYt7pyZnokH\nVlk+uHRkHFv3Cskl2CPyW5PkmLu8tpZsX22m2wGgG0S3AQCIZDo9F6/V4oHlZPss+RZaJ9JhheXi\nI3pkJ4hyBID19XayvUvW3geBpLeNfJdbOr+7fzjR/PD4UwghrkV0h58QmSLnFyJT5PxCZIqcX4hM\nkfMLkSnXTLkuOPkcYn0RE9b4iiL3ACAKmHKi5xE1j8oyFSJtzdbi9ahU0wk3jaxHJDUBQJtEo62u\nxrLd5StXku1MHuySBJ6zs3GJssZiHJUYJeqcmomPx+S8Wi1OaNodxHJepxPLdp1g/VdXV8MxEez8\n3Yyu/EJkipxfiEyR8wuRKXJ+ITJFzi9Epsj5hciUkqU+m0y2C2BHGtJIQNLnE2pz0QgyhMqKpI/V\nz4ukPsZgECcEbbXTySUBnoxzvZnuY7X6mMQ2RaS+2aAeHwDMBHX3bEjWl5wfVcQRf8NhLJn2OvHr\n7rTTcmq3FcuD03PxeoyLrvxCZIqcX4hMkfMLkSlyfiEyRc4vRKZcM4E928/Sx/HhpEdkO/DbP9qQ\nHY8FBJFd+yrp6/fSO/ftbrxr3wrKRQHAqxcuhX0XL1yIj9lO73yzJVxYTAfhAMCRw4fDvsXldJ4+\nZsfZc+fCMfXq74R9LJ9ki+zOX7kcB0Fdunw52e6IS2/NBzkIqyS/42Z05RciU+T8QmSKnF+ITJHz\nC5Epcn4hMkXOL0SmjFOu61YAXwFwFCOl5qS7f8HMDgL4BoDbMCrZda+7v7rV8SoTBPZE8lAshABU\nPGT5/UhZKx+mZ3QSJMLVQRJcYvFbw0qRDYLAmXYzDjpZvRTnimsGZbcAoE0kwigwqdFohGOOHDkS\n9h296WjYxy5h5155Odk+VYsDY1abS/FUJPio24nzHXa7cX6/bi89rl4nZeWqOxfHx/HEPoBPufud\nAN4G4ONmdieABwA86e53AHiy+FsIcZ2wpfO7+1l3/1HxeBXA8wBuAXAPgFPF004B+MBeGSmE2H22\n9R3czG4D8BYATwM46u5ni66XMfpZIIS4Thjb+c1sHsA3AXzS3a9Kyu6jH3jJH3lmdsLMTpvZ6WaQ\n4EEIUT5jOb+Z1TFy/K+6+7eK5nNmdqzoPwbgfGqsu5909xV3X2GbPUKIctnS+c3MADwM4Hl3/9yG\nrscA3Fc8vg/Ad3bfPCHEXjFOVN/bAXwEwHNm9kzR9iCAzwB41MzuB/BrAPdudSADYJFMxfLZjWHk\n6ycjkh07Is25F83FzGClvEgfkZSGRFrsBOW11ohkd2U1XVoLADrtOFKNMTOTzqu3fCCOwDtyJI7c\nO3jwQNi31oylynYQaTeoxkJxpxPLcpgnpbyqcX4/FolZqaTdsNuL5dnJitFdzZbO7+7fR3x6v3sX\nbBBC7AO6w0+ITJHzC5Epcn4hMkXOL0SmyPmFyJTyE3gG8haVLibRNUjknk34mVeppG1ncl6VSHYV\nI30seiyQ8wCgEySsbJLSWt2gXBQAuvZzM/FNW9ENXUuLsdQ3O5tOSgkAw0Esv7Wa62FfJ+gb1tJl\nvACgS+TNwSBekIqRpKv1WAasVNNuOER8vMiK7biKrvxCZIqcX4hMkfMLkSlyfiEyRc4vRKbI+YXI\nlFKlPkec2JHBpLR4rsninviotB0VUh+tViNLTCYb9NM19wCg04mlqPUgqWaH1JFrNUkiTiJtzczM\nhH2zs+movql6LLENeunkowBwaT2W8y5duBj29QLZrjYby2gYkNSw5PytTsXv9cxsLIvOzKYjLlsk\nurAXJATdjn/pyi9Epsj5hcgUOb8QmSLnFyJT5PxCZEr5gT0s4d1uzkKCZiZRHABgGBUII5vDdZZL\nkNjR7sT521gJrfW19K54uxUfr7UW7/ZXWO65aVI2LNgxb67FAUadVmxHi+Tpu3jhQtgXrdVULQ60\n6QXlswCgRd6X6ZlYyajU4vPRgjVuE1Xn0uXLyfbBIFaJXmfT2M8UQtxQyPmFyBQ5vxCZIucXIlPk\n/EJkipxfiEzZUuozs1sBfAWjEtwO4KS7f8HMHgLwUQCvFE990N0f3+JoiKQ+Js1FETCsbBXL4cdg\nJbT6vbR8NbRY62O54vokFx/LS9dcjWWvVis9js01IPnxKiT+xYjG2e2kZbv1VixTDolMtXopLW0B\nwCopNxblXWTnh5PX1R/E8tuUkXJdNZLfL5AB2fl9OTgHtiP1jaPz9wF8yt1/ZGYLAH5oZk8UfZ93\n978dezYhxDXDOLX6zgI4WzxeNbPnAdyy14YJIfaWbf3mN7PbALwFwNNF0yfM7Fkze8TM4jKqQohr\njrGd38zmAXwTwCfd/QqALwK4HcBxjL4ZfDYYd8LMTpvZ6WYzvrVTCFEuYzm/mdUxcvyvuvu3AMDd\nz7n7wN2HAL4E4K7UWHc/6e4r7r4SFXIQQpTPls5voxxaDwN43t0/t6H92IanfRDAmd03TwixV4yz\n2/92AB8B8JyZPVO0PQjgw2Z2HCMd7gUAHxtnwiiQjUfapfvomAkj9xhRPj6WL5DFMNZJfr/Z6Tg/\n3iqRRZuraSktivYDgNGXtzRMfmuuxZJjp5+WD/vDeC6Wq7HbiyW22nQspy4szCfbZ+fT7QBgJAJv\nSNaKRQPOzEyHfUeP3pxs7w/inIZra2npk0Vhbmac3f7vI30Ob6HpCyGuZXSHnxCZIucXIlPk/EJk\nipxfiEyR8wuRKSUn8HQMMX7U0f8xDMpkVViSzvhwTFJyEknlHo2LxwyZtEX62AsYssSf7aBcVzeW\nylg1NNpH1j+KpqsQ8bNSJdInkUXr9bhvPpD65onUVyfSYY2U5Iqi8wCgQqL6pufS8y23l8MxU1Pp\nCMIaWcPX2TT2M4UQNxRyfiEyRc4vRKbI+YXIFDm/EJki5xciU/ahVt/2YdIcGRX28IC/eJwFSR+Z\n9OZBzToAGPRj2bNLIsR63TjhZqQ6sjWsEqlsjuRgYPkZesP0a6MJJqvkWkQiGev1OHFmY2423U5s\nX1hcCPvYetSIHSwpaJRMdHF5KRwyPZOO+qwSSXQzuvILkSlyfiEyRc4vRKbI+YXIFDm/EJki5xci\nU25gqW/356pYOjKLRapVWbE7IgOygD9GlGR0bm4uHFMlEtuBQwfDvsWlWIqKloTJm0zqYxGErA7e\n1FQ6cSZLkNpoxGs1NR0n4qwwqZLJwUHf/NxiOGZ2Or2OtW0k8NSVX4hMkfMLkSlyfiEyRc4vRKbI\n+YXIlC13+81sBsBTAKaL5/+Lu3/azA4C+AaA2zAq13Wvu786qSF0Rz/YKGUBNSy/36RYlJcuUAEA\nwJzkGSSfvbQUGQvSmUrng6uR5V2Yj4NVDh46FPYdOBBXZY/Wf0DKXbHXxcpQ0cCq4OSZqsd5+uqk\nj+7oT3AOM6rkNUeqDlNFNjPOMzsA/sDd34xROe67zextAB4A8KS73wHgyeJvIcR1wpbO7yNeq/5Y\nL/45gHsAnCraTwH4wJ5YKITYE8b6jmBm1aJC73kAT7j70wCOuvvZ4ikvAzi6RzYKIfaAsZzf3Qfu\nfhzAGwDcZWZv2tTvCH7VmNkJMzttZqebzeaODRZC7A7b2hVz90sAvgfgbgDnzOwYABT/nw/GnHT3\nFXdfYdlThBDlsqXzm9kRM1suHs8CeA+AnwF4DMB9xdPuA/CdvTJSCLH7jBPYcwzAKTOrYvRh8ai7\n/6uZ/QeAR83sfgC/BnDvOBPGagiRaywtD9F4n2AMn4ll8AMilWpISnyBSC+9bjvs6/f7YR8rAVYL\n8vHVSW63ZSLZLS3HJaMWluLAk0jqY7bT1a/GfX2SFzBaRxpwRU4sbj45h2lpNnLMgEgG3E4I3JbO\n7+7PAnhLov0CgHdvYy4hxDWE7vATIlPk/EJkipxfiEyR8wuRKXJ+ITLFaPTYbk9m9gpGsiAAHAbw\n29Imj5EdVyM7ruZ6s+N33f3IOAcs1fmvmtjstLuv7MvkskN2yA597RciV+T8QmTKfjr/yX2ceyOy\n42pkx9XcsHbs229+IcT+oq/9QmTKvji/md1tZv9tZr8ws33L/WdmL5jZc2b2jJmdLnHeR8zsvJmd\n2dB20MyeMLOfF//HoXZ7a8dDZvZSsSbPmNn7SrDjVjP7npn91Mx+YmZ/VrSXuibEjlLXxMxmzOw/\nzezHhR1/VbTv7nq4e6n/AFQB/BLA7QCmAPwYwJ1l21HY8gKAw/sw7zsBvBXAmQ1tfwPggeLxAwD+\nep/seAjAn5e8HscAvLV4vADgfwDcWfaaEDtKXROMInPni8d1AE8DeNtur8d+XPnvAvALd/+Vu3cB\nfB2jZKDZ4O5PAbi4qbn0hKiBHaXj7mfd/UfF41UAzwO4BSWvCbGjVHzEnifN3Q/nvwXAbzb8/SL2\nYYELHMB3zeyHZnZin2x4jWspIeonzOzZ4mfBnv/82IiZ3YZR/oh9TRK7yQ6g5DUpI2lu7ht+7/BR\nYtI/BPBxM3vnfhsE8ISoJfBFjH6SHQdwFsBny5rYzOYBfBPAJ939ysa+MtckYUfpa+I7SJo7Lvvh\n/C8BuHXD328o2krH3V8q/j8P4NsY/STZL8ZKiLrXuPu54sQbAvgSSloTM6tj5HBfdfdvFc2lr0nK\njv1ak2LubSfNHZf9cP4fALjDzN5oZlMAPoRRMtBSMbOGmS289hjAewGc4aP2lGsiIeprJ1fBB1HC\nmtioVtvDAJ53989t6Cp1TSI7yl6T0pLmlrWDuWk3830Y7aT+EsBf7JMNt2OkNPwYwE/KtAPA1zD6\n+tjDaM/jfgCHMCp79nMA3wVwcJ/s+CcAzwF4tjjZjpVgxzsw+gr7LIBnin/vK3tNiB2lrgmA3wPw\nX8V8ZwD8ZdG+q+uhO/yEyJTcN/yEyBY5vxCZIucXIlPk/EJkipxfiEyR8wuRKXJ+ITJFzi9Epvwv\nPjBWMQY1ufIAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x10daba518>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# view an image (e.g. 10) and print its corresponding label\n",
    "img_index = 10\n",
    "plt.imshow(X[:,:,:,img_index])\n",
    "print(y[img_index])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "> The first code block will take a short while to load if you are using the largest dataset. If you are getting errors here, check two things. Firstly, ensure that your downloaded dataset is saved in the same directory as this notebook. And secondly, ensure the filename matches the code (if you downloaded the smaller dataset, you will have a different filename of `'train_32x32.mat'`)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "As you can see, we load up an image showing house number 2, and the console output from our printed label is also 2. You can change the index of the image (to any number between 0 and 531130) and check out different images and their labels if you like. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now, to use these images with a machine learning algorithm, we first need to vectorise them. This essentially involves stacking up the 3 dimensions of each image (the width x height x colour channels) to transform it into a 1D-matrix. This gives us our feature vector, although it’s worth noting that this is not really a feature vector in the usual sense. Features usually refer to some kind of quantification of a specific trait of the image, not just the raw pixels. Raw pixels can be used successfully in machine learning algorithms, but this is typically with more complex models such as convolutional neural networks, which can learn specific features themselves within their network of layers. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "from sklearn.utils import shuffle\n",
    "\n",
    "X = X.reshape(X.shape[0]*X.shape[1]*X.shape[2],X.shape[3]).T\n",
    "y = y.reshape(y.shape[0],)\n",
    "X, y = shuffle(X, y, random_state=42)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We’re also shuffling our data just to be sure there are no underlying distributions. The library we’ve used for this ensures that the index pairings between our images in `X` and their labels in `y` are maintained through the shuffling process. You can use the parameter `random_state=42` if you want to replicate the results of this tutorial exactly."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Machine Learning Algorithms\n",
    "\n",
    "Now that we have our feature vector `X` ready to go, we need to decide which machine learning algorithm to use. We don’t need to explicitly program an algorithm ourselves - luckily frameworks like scikit-learn do this for us. Scikit-learn offers a range of algorithms, with each one having different advantages and disadvantages. We won’t be going into the details of each, but it’s useful to think about the distinguishing elements of our image recognition task and how they relate to the choice of algorithm. This could include the amount of data we have, the type of problem we’re solving, the format of our output label etc.\n",
    "\n",
    "If you don’t have any prior experience in machine learning, you can use this helpful [cheat sheet](http://scikit-learn.org/stable/tutorial/machine_learning_map/) to guide you in which algorithms to try out depending on your data. But I would really recommend reading up and understanding how the algorithms work for yourself, if you plan to delve deeper into machine learning. This will be especially useful for tuning hyperparameters. Hyperparameters are input values for the algorithm which can tune its performance, for example adjusting the maximum depth of a decision tree. \n",
    "\n",
    "For now we will be using a Random Forest approach with default hyperparameters. You can learn more about Random Forests [here](http://scikit-learn.org/stable/modules/generated/sklearn.ensemble.RandomForestClassifier.html ), but in brief they are a construction of multiple decision trees with an output that averages the results of individual trees to prevent fitting too closely to any one tree. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "First we import the necessary library and then define our classifier."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "RandomForestClassifier(bootstrap=True, class_weight=None, criterion='gini',\n",
      "            max_depth=None, max_features='auto', max_leaf_nodes=None,\n",
      "            min_impurity_split=1e-07, min_samples_leaf=1,\n",
      "            min_samples_split=2, min_weight_fraction_leaf=0.0,\n",
      "            n_estimators=10, n_jobs=1, oob_score=False, random_state=None,\n",
      "            verbose=0, warm_start=False)\n"
     ]
    }
   ],
   "source": [
    "from sklearn.ensemble import RandomForestClassifier\n",
    "\n",
    "clf = RandomForestClassifier()\n",
    "\n",
    "print(clf)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can also print the classifier to the console to see the parameter settings used. Although we haven’t changed any from their default settings, it’s interesting to take a look at the options and you can experiment with tuning them at the end of the tutorial. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Training the model\n",
    "\n",
    "We’re now ready to train and test our data. But before we do that, we need to split our total collection of images into two sets - one for training and one for testing. You can also add a third set for development/validation, which you can read more about [here](https://en.wikipedia.org/wiki/Training,_test,_and_validation_sets).\n",
    "\n",
    "Keeping the testing set completely separate from the training set is important, because we need to be sure that the model will perform well in the real world. Once trained, it will have seen many example images of house numbers. We want to be sure that when presented with new images of numbers it hasn’t seen before, that it has actually learnt something from the training and can generalise that knowledge - not just remember the exact images it has already seen.\n",
    "\n",
    "Usually we use between 70-90% of the data for training, though this varies depending on the amount of data collected, and the type of model trained. For example, neural networks are often used with extremely large amounts of data and may sample 99% of the data for training. In this tutorial we’ll go with 80%.  \n",
    "\n",
    "Again, using the largest dataset means that fitting the model may take a little while."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "RandomForestClassifier(bootstrap=True, class_weight=None, criterion='gini',\n",
       "            max_depth=None, max_features='auto', max_leaf_nodes=None,\n",
       "            min_impurity_split=1e-07, min_samples_leaf=1,\n",
       "            min_samples_split=2, min_weight_fraction_leaf=0.0,\n",
       "            n_estimators=10, n_jobs=1, oob_score=False, random_state=None,\n",
       "            verbose=0, warm_start=False)"
      ]
     },
     "execution_count": 29,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from sklearn.model_selection import train_test_split\n",
    "\n",
    "X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)\n",
    "\n",
    "clf.fit(X_train, y_train)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Test Results\n",
    "\n",
    "Now we’re ready to use our trained model to make predictions on new data."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Accuracy: 0.76013631186\n"
     ]
    }
   ],
   "source": [
    "from sklearn.metrics import accuracy_score\n",
    "\n",
    "preds = clf.predict(X_test)\n",
    "\n",
    "print(\"Accuracy:\", accuracy_score(y_test,preds))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Your accuracy result should be ~76% if you opted to use the original large dataset, and will be less if you used a smaller one."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "So our model has learnt how to classify house numbers from Google Street View with a relatively high accuracy simply by showing it a few hundred thousand examples. Given a baseline measure of 10% accuracy for random guessing, we’ve made significant progress. There’s still a *lot* of room for improvement here, but it’s a good result from a simple untuned learning algorithm on a real-world problem. You can even try going outside and creating a 32x32 image of your own house number to test on. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Extensions\n",
    "\n",
    "What can you do next? How can you expand upon this tutorial?\n",
    "\n",
    "If you’re interested in experimenting further within the scope of this tutorial, try training the model only on images of house numbers 0-8. Then test it on images of number 9. Do you think we can transfer the knowledge learnt to a new number? Why or why not? \n",
    "You could also perform some error analysis on the classifier and find out which images it’s getting wrong. Try to spot patterns in the errors, figure out why it’s making mistakes, and think about what you can do to mitigate this. \n",
    "Take a look into the distribution of different digits in the dataset, and you’ll realise it’s not even. Is this having an effect on our results? \n",
    "\n",
    "If you want to go further into the realms of image recognition, you could start by creating a classifier for more complex images of house numbers. This [dataset](http://ufldl.stanford.edu/housenumbers/extra.tar.gz) contains uncropped images, which show the house number from afar, often with multiple digits. Next you could try to find more varied data sets to work with - perhaps identify traffic lights and determine their colour, or recognise different street signs. Finding or creating labelled datasets is the tricky part, but we’re not limited to just Street View images! There are a ton of resources available online so go ahead and see what you can build next.\n",
    "\n",
    "A Github repo with the complete source code file for this project is available [here](https://github.com/elliebirbeck/sklearn-tutorial).\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "anaconda-cloud": {},
  "kernelspec": {
   "display_name": "Python [conda root]",
   "language": "python",
   "name": "conda-root-py"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.5.2"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}