{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Let's learn to train a machine learning algorithm and test it\n",
    "\n",
    "This notebook will teach you to use one of the popular machine learning package called  <a href=\"http://scikit-learn.org/stable/\"> Scikit Learn</a> to train a simple machine learning algorith to do a classification task.\n",
    "\n",
    "Take a look at this <a href=\"http://scikit-learn.org/stable/\"> website</a> and different <a href=\"http://scikit-learn.org/stable/auto_examples/index.html\"> examples </a> to explore further.\n",
    "\n",
    "I encourage you to play around with the code and see what happens !\n",
    "\n",
    "\n",
    "We will start by loading the necessary libraries to the workspace. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "# You don't want to change anything here now\n",
    "\n",
    "import numpy as np   # For some numerical stuff\n",
    "import matplotlib.pyplot as plt # For making beautiful plots\n",
    "from sklearn.datasets import load_iris\n",
    "from sklearn.neighbors import KNeighborsClassifier  # A simple machine learning model known as KNN\n",
    "from sklearn.cross_validation import train_test_split # A utility to split data\n",
    "from sklearn.metrics import precision_score\n",
    "%pylab inline\n",
    "\n",
    "# You may see some messages in the next line, don't worry about them"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Now lets load the data to our workspace "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "dataset = load_iris() # Load the complete iris data structure to this variable\n",
    "\n",
    "# Now lets get the features\n",
    "features = dataset['data']\n",
    "\n",
    "# Lets also get the name of the features\n",
    "feature_names = dataset['feature_names']\n",
    "\n",
    "# The class labels\n",
    "labels = dataset['target']\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "# Lets have a look at the names of the features and dimensions (shape) of the feature array and also see how many classes are present.\n",
    "# Verify if the number of feature names are equal to the number of columns\n",
    "\n",
    "print 'Feature names are :', feature_names\n",
    "\n",
    "print '\\nThe feature array has %d rows and %d columns'%(features.shape[0],features.shape[1])\n",
    "\n",
    "print '\\nThere are %d classes of objects in the dataset'%(len(np.unique(labels)))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Lets plot the data in a two dimensional space with the first feature on the x-axis and second on the y-axis "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "index_1 = 0 # Modify this to change the x-axis . Now it will take the first column. [In python index 1 starts at '0']\n",
    "index_2 = 1 # Modify this to change the y-axis\n",
    "\n",
    "plt.scatter(features[:,index_1],features[:,index_2],c=labels) # Make the scatter plot\n",
    "plt.xlabel(feature_names[index_1])\n",
    "plt.ylabel(feature_names[index_2])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Split the data into train and test sample\n",
    "\n",
    "Generally when training a machine learning algorithm, we have to validate its learning accuracy againts a set of test data whose labels are known. Performing this test will help us evalute how good the algorithm has learned. As a general practise we split our data into training and test samples. Usually 70% of the total data is used for training and the rest 30% for validation.\n",
    "\n",
    "The following peice of code splits the data into training and test sets."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# train_data --> feature samples for training\n",
    "# test_data  --> feature samples to evaluate / test\n",
    "# train_labels --> class labels for the training data\n",
    "# test_labels --> class labels for the test data\n",
    "\n",
    "train_data,test_data,train_labels,test_labels = train_test_split(features,labels,test_size=0.3,random_state=0)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "# Lets have a look at the size of the train and test data\n",
    "\n",
    "print 'Train data has %d samples'%(train_data.shape[0])\n",
    "print 'Test data has %d samples'%(test_data.shape[0])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Training the machine \n",
    "\n",
    "In this example we will train a simple machine learning algorithm called K-nearest neighbors to classify the 3 different classes in the data we have loaded."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "mymodel = KNeighborsClassifier(n_neighbors=5,)  # Create the classifier object to a variable 'mymodel'\n",
    "\n",
    "mymodel = mymodel.fit(train_data,train_labels) # Train the algorithm and save the model mymodel "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    " That's it ! We have trained our first machine learning algorithm. Now lets test it.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Testing the algorithm \n",
    "\n",
    "Testing the algorithm is simple as training it. To evaluate the performance we will use an evaluation metric \n",
    "called 'Percision Score'. The precision score is defined as\n",
    "\n",
    "$\\mathrm{precision = \\frac{Number \\ of \\ correctly \\ classified \\ samples}{Number \\ of \\ correctly \\ classified \\ samples \\ + \\ Number \\ of \\ incorrectly \\ classified \\ samples}}$\n",
    "\n",
    "The higher this number better the performance of the machine learning algorithm. This simply means the machine learning algorithm has learnt the pattern well."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "# Test the performance of the algorithm on the test data which was generated through the splitting before.\n",
    "\n",
    "predictions = mymodel.predict(test_data)\n",
    "\n",
    "# Now we have the class labels predicted by the algorithm for each test samples in the variable 'predictions'\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "# Time to check the precision score\n",
    "\n",
    "score = precision_score(predictions,test_labels,average='micro')\n",
    "\n",
    "print 'The precision score is %f'%(score*100)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As an excercise change the values of the following parameters in the above code and check how it affects \n",
    "the precision score.\n",
    "\n",
    "* test_size=0.2 in test_train_split [ Change it values like 0.5, 0.2 etc] \n",
    "\n",
    "* \"n_neighbours=5\" in  clf = KNeighborsClassifier(n_neighbors=5) [Change the value between 1 and 25]  \n"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 2",
   "language": "python",
   "name": "python2"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 2
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython2",
   "version": "2.7.11+"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 0
}