{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Introduction to keras\n",
    "\n",
    "This is a short introductive tutorial for [keras](http://keras.io). Keras is a high-level interface for machine learning with **neural networks**.\n",
    "\n",
    "In order to introduce and illustrate the principle of neural networks, we will consider the well-known classification problem of **iris species**."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "## Classification of iris species\n",
    "\n",
    "*Iris setosa*, *Iris versicolor* and *Iris virginica* are closely-related species. They differ by the **size** of their petal and sepal.\n",
    "\n",
    "![flowers](./images/flowers.png)\n",
    "\n",
    "For instance the dataset below list the **height** and **width** of the petals of different specimens, and **whether** these specimens belong to the species *setosa*. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>petal length (cm)</th>\n",
       "      <th>petal width (cm)</th>\n",
       "      <th>setosa</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1.3</td>\n",
       "      <td>0.2</td>\n",
       "      <td>1.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1.6</td>\n",
       "      <td>0.4</td>\n",
       "      <td>1.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>4.7</td>\n",
       "      <td>1.2</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>5.5</td>\n",
       "      <td>2.1</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>1.3</td>\n",
       "      <td>0.3</td>\n",
       "      <td>1.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>3.7</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   petal length (cm)  petal width (cm)  setosa\n",
       "0                1.3               0.2     1.0\n",
       "1                1.6               0.4     1.0\n",
       "2                4.7               1.2     0.0\n",
       "3                5.5               2.1     0.0\n",
       "4                1.3               0.3     1.0\n",
       "5                3.7               1.0     0.0"
      ]
     },
     "execution_count": 1,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import pandas as pd\n",
    "df = pd.read_csv('./data/setosa/train.csv')\n",
    "df.head(6)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Problem statement\n",
    "\n",
    "Based on this dataset, and given the petal height and width of a **new specimen**, we would like to predict the **probability** that this specimen belongs to the species setosa.\n",
    "![Model](./images/Model_Schematic.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Using neural networks for this problem\n",
    "\n",
    "### General structure of fully-connected neural networks\n",
    "\n",
    "Neural networks are an extremeley versatile machine learning technique. \n",
    "\n",
    "They consist in stacking individual units (a.k.a. *artificial neurons*) that :\n",
    "- perform a **weighted sum** of their inputs ($\\,S = \\sum_i w_i x_i + w_0\\,$)\n",
    "- apply a **non-linear function** to this sum ($\\,y = f(S)\\,$) and return it as output\n",
    "\n",
    "![neural_nets](./images/Schematic_neural_net.png)\n",
    "\n",
    "- The number of units and layers are ** *arbitrarily* chosen** by the user.\n",
    "- The non-linear function $f$ are ** *arbitrarily* chosen** by the user (typical example include the sigmoid function or the `tanh` function).\n",
    "- The values of the weights are **determined by an algorithm**, so as to produce the right output on a known dataset.\n",
    "\n",
    "**Training the network consists in finding the right values for the weights $w_i$.**"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Using a neural network for the classification of Iris setosa\n",
    "\n",
    "In order to solve the problem of classification for *Iris setosa*, we will start with the simplest kind of neural network: a **single layer** network. \n",
    "\n",
    "For the non-linear function, we will choose the **sigmoid** function, since its output is between 0 and 1 and can thus easily be interpreted as a probability.\n",
    "\n",
    "![single_layer](./images/single_layer.png)\n",
    "\n",
    "With this model, we have: $ p_{setosa} = f( w_0 + w_1\\times height + w_2 \\times width )$, and training the model consists in finding $w_1$ and $w_2$."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Next steps in this tutorial\n",
    "\n",
    "We will first build a neural network by hand [here](./Single_layer_by_hand.ipynb), before using `keras` to automate the process."
   ]
  }
 ],
 "metadata": {
  "anaconda-cloud": {},
  "kernelspec": {
   "display_name": "Python [conda env:thw]",
   "language": "python",
   "name": "conda-env-thw-py"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.5.2"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}