{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Combining Everything Together\n",
    "\n",
    "This file will perform binary classification on the iris dataset. We will only predict if a flower is I.setosa or not. \n",
    "\n",
    "We will create a simple binary classifier by creating a line and running everything through a sigmoid to get a binary predictor. The two features we will use are pedal length and pedal width.  We use these two features because we know that Iris setosa is separable by these two features.  We aim to find the line that separates it out.\n",
    "\n",
    "We will use batch training, but this can be easily adapted to stochastic training (i.e. set batch size equal to 1).\n",
    "\n",
    "We start by loading the necessary libraries and resetting the computational graph."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "import matplotlib.pyplot as plt\n",
    "import numpy as np\n",
    "from sklearn import datasets\n",
    "import tensorflow as tf\n",
    "from tensorflow.python.framework import ops\n",
    "ops.reset_default_graph()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Load Iris Data\n",
    "\n",
    "We load the data and mark the target if it is I. setosa or not.  The input data will be the 3rd and 4th features of the data set. (Petal Length and Petal Width)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# Load the iris data\n",
    "# iris.target = {0, 1, 2}, where '0' is setosa\n",
    "# iris.data ~ [sepal.width, sepal.length, pedal.width, pedal.length]\n",
    "iris = datasets.load_iris()\n",
    "binary_target = np.array([1. if x==0 else 0. for x in iris.target])\n",
    "iris_2d = np.array([[x[2], x[3]] for x in iris.data])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We set the batch size to 20."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "batch_size = 20"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Then we start a computational graph session."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# Create graph\n",
    "sess = tf.Session()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Placeholders\n",
    "\n",
    "We declare the placeholders for the model.  Just to illustrate that we can feed in multiple x-features separately, we create two separate placeholders for the two Iris features."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# Declare placeholders\n",
    "x1_data = tf.placeholder(shape=[None, 1], dtype=tf.float32)\n",
    "x2_data = tf.placeholder(shape=[None, 1], dtype=tf.float32)\n",
    "y_target = tf.placeholder(shape=[None, 1], dtype=tf.float32)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Model Variables\n",
    "\n",
    "We are going to be doing a linear model, so we will need to create two variables, `A` (slope) and `b` (intercept)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# Create variables A and b\n",
    "A = tf.Variable(tf.random_normal(shape=[1, 1]))\n",
    "b = tf.Variable(tf.random_normal(shape=[1, 1]))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Model Operations\n",
    "\n",
    "A line can be defined as $x_{1}=A\\cdot x_{2} + b$. To create a linear separator, we would like to see which side of the line the data points fall. There are three cases:\n",
    "\n",
    "- A point exactly on the line will satisfy: $0 = x_{1} - (A\\cdot x_{2} + b)$\n",
    "- A point above the line satisfies: $0 > x_{1} - (A\\cdot x_{2} + b)$\n",
    "- A point below the line satisfies: $0 < x_{1} - (A\\cdot x_{2} + b)$\n",
    "\n",
    "We will make the output of this model:\n",
    "\n",
    "$$x_{1} - (A \\cdot x_{2} + b)$$\n",
    "\n",
    "Then the predictions will be the sign of that output:\n",
    "\n",
    "$$Prediction(x_{1},x_{2}) = sign(x_{1} - (A \\cdot x_{2} + b))$$\n",
    "\n",
    "So we add the corresponding operations to the computational graph."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# Add model to graph:\n",
    "# x1 - A*x2 + b\n",
    "my_mult = tf.matmul(x2_data, A)\n",
    "my_add = tf.add(my_mult, b)\n",
    "my_output = tf.subtract(x1_data, my_add)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Loss Function\n",
    "\n",
    "Since we are doing a categorical prediction (I.setosa or not), we will use the sigmoid cross entropy loss.  This is a function provided to us by TensorFlow."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# Add classification loss (cross entropy)\n",
    "xentropy = tf.nn.sigmoid_cross_entropy_with_logits(my_output, y_target)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Optimizing Function and Variable Initialization\n",
    "\n",
    "We use the standard Gradient Descent Optimization function with a learning rate of `0.05`.  We then add and run a variable initialization operation."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# Create Optimizer\n",
    "my_opt = tf.train.GradientDescentOptimizer(0.05)\n",
    "train_step = my_opt.minimize(xentropy)\n",
    "\n",
    "# Initialize variables\n",
    "init = tf.global_variables_initializer()\n",
    "sess.run(init)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Run Classification\n",
    "\n",
    "We run the classification for 1000 iterations and output the values of `A`, `b`, and loss every 200 iterations."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Step #200 A = [[ 8.48966789]], b = [[-3.37898731]]\n",
      "Step #400 A = [[ 10.06670666]], b = [[-4.63564396]]\n",
      "Step #600 A = [[ 11.00876522]], b = [[-5.43504715]]\n",
      "Step #800 A = [[ 11.8202858]], b = [[-5.87730837]]\n",
      "Step #1000 A = [[ 12.35187435]], b = [[-6.31382132]]\n"
     ]
    }
   ],
   "source": [
    "# Run Loop\n",
    "for i in range(1000):\n",
    "    rand_index = np.random.choice(len(iris_2d), size=batch_size)\n",
    "    #rand_x = np.transpose([iris_2d[rand_index]])\n",
    "    rand_x = iris_2d[rand_index]\n",
    "    rand_x1 = np.array([[x[0]] for x in rand_x])\n",
    "    rand_x2 = np.array([[x[1]] for x in rand_x])\n",
    "    #rand_y = np.transpose([binary_target[rand_index]])\n",
    "    rand_y = np.array([[y] for y in binary_target[rand_index]])\n",
    "    sess.run(train_step, feed_dict={x1_data: rand_x1, x2_data: rand_x2, y_target: rand_y})\n",
    "    if (i+1)%200==0:\n",
    "        print('Step #' + str(i+1) + ' A = ' + str(sess.run(A)) + ', b = ' + str(sess.run(b)))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Visualize Results\n",
    "\n",
    "We pull out the slope and intercept and plot the predictions"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXYAAAEhCAYAAABr1YsqAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJztnXmcFNW1+L9nGJB1YFh0BGRxVEBNjEvcUBxQE8wTQX2u\nIG4x0YdGY/I+cUPgodG8+ItLMD5UcItb1BkiLlEDDCQo4gIqghubOMgiDAMMss2c3x9VPdPT093T\n3VNd1d1zvp9Pfbr71q17z63l9K1zzz1XVBXDMAwjd8gLWgDDMAzDW0yxG4Zh5Bim2A3DMHIMU+yG\nYRg5hil2wzCMHMMUu2EYRo7RYhW7iNSKyOyg5TAMw/CanFLsrrKuSTC7ultOIyL7iMhvRWSBiGwR\nkV0islZE3heRP4vIkKBlDAIROcW9X27PIFlibTUi0icg2cpdGQK/T0RklYisCFqObCA/aAECZBCw\nI2gh0omIdADmAUcC3wIvAuuAjsARwFVAZzePETyrgMejpCuwxVdJGtadKR2gTJEj42mxil1Vvwha\nBh/4NY5S/wdwlqruDd8pIp1x/uBaIhK0AFFYpar/E7QQRvaTU6aYZIhmYxeRiaHXThH5TxF5V0Sq\nRWSTiDwrIj1jlFUoIneJyFIR2eGaPP4pIqdHyVsgIv8tIrNEZI1rGtkgIn8XkePiySoi+4nIoyLy\njYjsFZGxTTTzBJxezv9FKnUAVa1S1QVR6mslIv8lIu+ISJV7Dj4UkXEiIhF5+7ryTReRASIywz1f\n20XkX36dAxE5WETuFpH33LJ2uq/uU0WkV0RZjwGz3XMTuuYhk8eQsHxtROQmEfnIPQdVIjJPRM6L\nIl/4eThYRJ4XkfWRZXqFiBwlIi+5dYTa+qCIFEXJ+7grWz8Ruc5tz47I+98jufYVkXtE5DP3Hqh0\nvz8mIv2i5P+piLwmIhvddnwlIv/rdjpCeU4RkVqgD9BPGpqppkeUd6qI/MO9B78Xkc/dZ7MgSt39\nReRhEfnSPR+bRORjEXlIRArD8iV9vwZNi+2xxyD02jkOGAG8DJQDxwEXAD8UkR+p6p7QAeLYPufi\n3HT/Al4HOgBnAv8QkV+o6rSwOgYBd7jHvAJUuseeBZwhImeq6ptRZOsKLAC2AS8BtcD6Jtqzyf08\nJJHGu+3Jd+X6CfAZ8DSwExgK/Bk4Frg0yqEHAu8AnwD/B+yPc85eF5GLVPWFsLzpOAfnAL8A5gDz\ngd3AYcDPgTNF5BhV/dbNW4ZznS/Dub7lYXWscs9Da+BNYAiwDJgCtAf+E3heRI5Q1duiyHgQ8C7w\nOfBXoB2wNUq+lBGRM3HMarifq4GjgWuAs0RksKp+HXZI6L5+ADgJeNXdEh2PSlSudsDbQH/gLZzn\nR4C+ONf2Bdzz6+a/HZiIc5++AmwAfgj8Fuc+OEFVt7vHTMR5A1XgXurfuBaHlfdL4C/AdreuDUAJ\n8Duce2Cwqm518xYB7+OYJV/DOY9tXdnH4NzrlW7Rqd6vwaGqObPhPOg1SeSdHZE2wU3fAhwase9p\nnAfhPyPSy4G9wHkR6QXAIqAa6BGW3gnoGkWenkAF8GmsdgGPAXlJnI//cI/dCTwI/AwoauKYie4x\n9wESli7Ao64cI8LS+4bJd3dEWUfhKNhNQMd0ngOcP5LWUdJPc6/PgxHpp7hl3h7jPNzs7p8ZXh/Q\nHVjpynJ8jPMwOcn7NiTLCvcejNxOCcvbAfgO2AOcGFHOf7vl/CMi/TE3fQ3QJ0nZ5rhtGpJA3jPd\neu6Jsi8f6BD2e6ib919Ap4i8Y919/y8ifSWwIkbdfdz7fAtwcMS+B93y/i8s7Vq3XddGKasdsE9z\n7tegt8AF8LQx3in2SVHyl7j7/jcs7Ydu2vMx6jjLvXmuTlCm+938vaPI+j3QPYVzci2w2S231t3W\n4vQmT47IK8BG92aNpjw7u+U8F5YWUmibwx/csP2PucdcEuA5+Aj4KiKtKcX+Jc4fwsFR9l3hHvto\nlPOwlih/ME3IF5KlJsZ2e1jei928T0UppxXOn0OD8xd2DRopsQRkS0Wx35FA3jK33EEx9n8IrItI\ni6fYbyXGnyrQBajC6WS1DnsuaoGfJ3tOErlfg97MFNMYBT6Ikr7G/SwMSzvB/ewsIhOiHLMvjrJs\nMEApIoOB64Hj3TxtIurvBXwTUdYqVf0ukQaEo6pTRORR4HTgRJzB1BOBi4CLRWSyqoZkPwToBnwB\njBdpNL4oOMo12oDrh6paHSW9HMd0cyTwVF1BaTgHIjLGresInOvUKmz3rljHRSmnI1AMfKOqX0bJ\nErJNHxll30caZqpLkrmqOqyJPEfhnJ85kTtUtUZE5gGXuLJFnr/3UpQrUebidApuEpGjcUwc84HF\nqlobkfd4nLeO86PcZ+DcDz1EpFBVK6NliCB0LaKdly0isgg4GRiIYy58Gfg98BcRGQ68AcxX1aXR\nCk/xfg0MU+zRieZaFhp8DFcW3dzP090tGorz+gyAiJyNY//7HscOuRynJ1GL83o6BNgnSjnrEpS9\nsQCqO3FMCjNdGfJxXB0fAG4TkVJV/SisPQcD8fy7O0RJi2XvD8kdPhjm+TkQkXtxHry1OF5AFW75\nAJfjvKonSkjWb2PsD6V3SUZGj8hY2VR1mzuYOAnnbfUnOJ2B70TkLzg9+dBz1A3nWYp3nymODTwR\nxZ7UeVHVr0Xkxzimx+HA2YCIyBocU9KfQwc2434NDFPszaPK/bxeVackeMxknN7j0RrhcimO100s\nDwrPfHjdh+shETkBGI1zc35EfXvKVPU/kyx2vxjpIS+NqrA0T8+BiPQArgM+xrE774jYf3F80RsR\nkrWRh4nL/hH5mpTRQzJZNlR1LU6n4SoRGQQMw3FGuB1HyYfeDqtwxnC6e1R1+HlZFmV/o/Oiqp8D\nF4lIHs5b3mk499F9IrJdVR9zs6Z6vwZGi3V39IiQq+DJSRxTDCyNcoNIkuV4wbZQ9e7nZzhvK8eL\nSKvoh8TkKHEmREUyFEehLApL8/ocHIhzL78VRan3dvdHEvIIadROdTwxlgO9RKQ4yrEhc0k0k126\nWYRzvUoid7jX7CT354c+yhQVVV2mqg/i9NwBRoXtXgAUuso/UWqIcr1c4p2XzsCPcAZXGyl9Va1V\n1UWq+kecMQyJkDWTntmEMMXeDFT1A5xR/XNE5PJoeUTkcLdHGWIVcHAUf+OJeDxZSER+GcvPVkQG\nAiF/7H+BY6PFcfPqCfxZRNpGOa4oxsPYmfreWCjvMTgPyhacwbIQq/D2HKxyP09ye1+h+jsCjxD9\nzTTkChrLRDMd5/n4Y0SZ3YHxOH9Wj8U4Np3MwBmovijKtf01zp/YW6qaVnuvex8MCPcPF5FDRWTf\nKNlD1zl8DCbksviIiOwfeYCItI/Svk04dvdoZo+/4tjsr4vyZ3wHjpfaU6HxD3HmATTybQ+TNbyD\nsAqfnlmvyElTjDgTUGJxjWtz9oqLgVnAoyLyKxwf5i1AbxyvmcNwBlk3uvnvBR4CFovISzg342Cc\nG+RlHP95rxiOY3JZhTOItQbHFngw8FOc63+/qr4fdsxkV+5fAiPEmcRSgTNgdLAr6y007vnMA650\nH8b5OH8O5+M8vL90e8EhPD0HqrpeRJ7D8ZtfLCJv4vzRnI5jF12M86odzuduuy4UkT3A1zjK+klV\nXQPcA5wBjAQ+EpHXcPzYzwN6AH9Q1beTkdMLVLVaRK4A/gbMFZEXXNmPxukZrwWu9kGUu3HcEi8D\nnnTTTsf5I3wHZwB+A85zMBKnt/3HsHbMFpHfAXcBX7rndyWOTb0vjqfQv3BcdEPMAo4B3nAHiXfh\nDFa/oqqrReQGnPkGH4rI33CeuVNwnr+lwE1hZV0C/FJE/o3zdlaJ0zMfgdOzvy8sr5/PrDcE7Zbj\n5UZsd7HQthcoCMs7K+L4CcRw7cK52WqAaVH2dcC5ad7DmYxSjXOzzASuBNpF5B+L86q8DefmfxHn\nDyBq/dFkTfB8HITTi3sV50HbhqPoVuEMBv0szrGjcQaKvsO50dfgKO/fAb0izkstTg93AE7PfBPO\nJJF5wGkxyvf0HOBMLpnstnMHzqSdB3C8Y+YAe6Mcc7Tbxkr33mhQL47nw004tvtqHPvsXOD8ZO6P\nBK7TKcleY1f2l3AGrXe613QKUeYp4LxZ7CVJH3b32DnusZHXI+RCOTYsbSDOH+JCV67vcdwvnyfM\n5z+inBOB53A8Sna6x32I8ydwVETe9jg+6V/jzI+oAaZH5DkNZ/B8k1v/Fzh/HgUR+X7slrXIvcer\n3byPEjGHJZX7NehNXKENIyVEpC9OT+txVb0iaHkMwzAbu2EYRs5hit0wDCPHMMVueIFisbINI2Mw\nG7thGEaOYT12wzCMHCOtil1EDhGRReIs0rBInIUKfpXOOg3DMFo6vpli3Nl73wDHqTMBxDAMw0gD\nfs48PQ1YHk2pi4gZ+g3DMJJEVaPGPPbTxn4B8GysnUHP1ApimzBhQuAyZGK7a2qUnj2Vzz4LXla7\n3tbuTG13PHxR7O76kaE1Dw0jLgsXQpcuMGBA0JIYRnbilynmDOADVd0YK8PEiRPrvpeUlFBSUpJ+\nqYyMpLQUzj47aCmMXGT1ypU8Pn48tRUV5PXqxWWTJ9O3f/+gxUqI8vJyysvLE8vsxysJjgnm0jj7\ntSUyZ86coEUIhHjtrq1VPegg1Q8+8E8ev7DrHSyrVqzQ3xQX63ZQBd0O+pviYl21YkVa6kt3u129\nGVWnpt0rRkTa4URjO1BVt8XIo+mWw8gOPvkERoyAlSsh+lKYhpEak8aM4bdPP91gXcdq4J7Ro5nw\n178GJVbKiAgaY/A07aYYVf0eJ361YTRJaSmcc44pdcN7aisqGi3W2wGoXbs2CHHSis08NTIKs68b\n6SKvV68GSziB02PP69kzCHHSiil2I2NYvhzWrYMTTwxaEiMXuWzyZCYUF9cp92pgQnExl02eHKRY\naSEjgoCZjd0AuOce+PJLmDo1aEkML0i3B0oq5dcds3YteT17ZpVXTCTxbOym2I2M4cQTYcIE+OlP\ng5bEaC6rV67kz6efzqTly+lAfe/4urfe8kSRprv8bCCeYjdTjJERfPstfPYZDB0atCSGFzw+fnyd\n0gVnkHLS8uU8Pn58VpSf7fgZK8YwYjJjBvzsZ9CmTdCSGF6QigdKMqaVluThkgqm2I2MoLQUrrkm\naCkMrwh5oET6jMfyQIlqWlmwIKZpJdnyWxyxZi75udFCZ54aDps2qRYUqG7fHrQkhlckO8tz4ujR\ndXk17JiJo0d7Un4uQpyZp9ZjNwLnlVdg2DDoEPlubWQtffv357q33uKeMA+U6zw0rSRbfkvDFLsR\nOKWlcO65QUtheE3f/v0TnqqfimklmfJbGubuaATK9u3QsyesXg2FhUFLYwSFuS8mj/mxGxnLiy/C\nI4/AG28ELYkRNLk0ecgPTLEbGcvo0XDyyXD11UFLYnhNNsc+zwZMsRsZya5dUFQES5fC/vsHLY3h\nJWZaST8289TISGbPhsMOM6Wei9jM0GAxrxgjMEKx1w3/SbeZxGaGBospdiMQamrg5ZfhlluClqTl\nkewsz1SwmaHBYqYYIxDmz3fcHM3c6j9+mElaUuzzTMR67EYgmBkmOPwwk9jM0GAxxW74jiqUlcFr\nrwUtScvELzOJzQwNDjPFGL7z4YfQti0cemjQkrRMzEyS+5gfu+E7t97qDJ7efXfQkrRckp3lmSuT\njfxoh1/nKp4fe+Ahe9XC9rY4Bg5UfffdoKUwEiVXQuT60Q4/zxVxwvb6obQ7Ay8Ay4BPgeOi5PG8\n0UZmsnSpau/eqjU1QUtiJEqysdIzFT/a4ee5iqfY/Rg8vR94TVXPE5F8oL0PdRoZSmkpnH025Nno\nTqC0xGXo/GhHppyrtCp2EekEnKyqlwGo6l5gazrrNDKbsjL44x+DlqJl01KXofOjHTsKCqLWsaNT\nJ8/qSIR095sOBL4TkcdE5EMReVhE2qW5TiNDWb3a2U4+OWhJWjbJTlDKFS8aP9qxV4TxbtmhOsa7\n6X6SblNMPnAUME5V3xeR+4CbgAmRGSdOnFj3vaSkhJKSkjSLZvhNWRmcdRbk2+yJQMnUZejS7U3S\nt39/zp4+nbGXXkqHLVuo7tKFG6dP97SOgqoqrgDuAWpxes7XA9O3Nt9QUV5eTnl5eWKZYxnfvdiA\n/YAVYb9PAmZGyef5wIKReZx8suorrwQthZGJg6G54rGSKYOnfnjFzAUOcb9PAP4QJY/njTYyi3Xr\nVDt3Vv3++6AlMTLRfTFXPFYyxd3Rj5fiXwFPi0hrYAVwuQ91GhnGyy/D8OHOjFMjPrlgkkiWVLxJ\nkj1PLSlGTtoVu6p+BPw43fUYmU1pKVxuf+lN4kdI3dUrV1J2xRU8uWqVU8eWLUy44gp6B7i6UbIe\nK6mcpxYVIydWV97PDTPF5DSVlaqdOqlu3Rq0JJlPrpgkkiVZE0YqbchEE1RzIGBTjNHCefVVOOUU\n8NmVNytpSZNowknWhJFKGzLFTOIHptiNtFNW5sw2NZrGD3NBxk840qYDAja7DQnUkSoZETAtVlfe\nzw0zxeQs1dWqBQWqGzcGLUl2kCtuf+mWKZU25Nq5JUh3x0Q2U+y5S1mZ6rBhQUuRXaxasUInjh6t\ntw8dqhNHj06LUvCjjmRI1WaeTBtybfwinmI3U4yRVnLNDOPHa7avXhVpNEkkQ21FBd/RcMbmZSRo\n90+wDanY5efPm8efLr2UDpWVVBcWcuMTTzB4yBBP60gHptiNtLFnD7zyCtx5Z9CSeIMfroh+kInt\n2Nq5M/cDk6FOpvGAFhREze+Hu+P8efN45NRTeXLvXqeOqirGnXoqzJoVU7lnzPhFrK68nxtmislJ\n3nxT9bjjgpbCOzLRTTAVMrEdvz3rrKgy/fass6Lm98Pd8Zx+/aLWcU6/fp7V0RwwU4wRBKWlcM45\nQUvhHZnymt1caisqWAz8ifre8Y0E2472W7dGPbftt22Lmt8Pd8cOlZVR6+iwZYtndaQLU+xGWqit\nhRkzYN68oCXxjox5zW4mFfn5PAI8Sb1iHwfkt2oVmEzJxjFP9VokM35RXVhIdVVVozqqu3TxrI50\nYevYGGlhwQLo3h0OPjhoSbwjV+KSb1y2jAepV4odgAfd9KBINo65H9fixieeYFx+foM6xuXnc+MT\nT3hWR7qwHruRFnLNDAOZ85rdXDpv3x7VxNC5ujpadl8oqKriJ8BYGpqH3owRxzzVa5GMV9PgIUNg\n1qyGwdKa8IrJFEyxG56j6ij20tKgJfGeTHjNbi6pmhjSydbOnXmJhuaheF4xkPy1SMWTZvCQIQxe\nuTLxhmQIZooxPOfjj53PI44IVg4jOploYshXrXN1xP2c7KZ7RbJLAmYz1mM3PCdkhvF5mUcjQVIx\nMaR7YlayXjGpkCteTYlgit3wnNJSmDo1aCmMeCRjYvBjQpMfHkfJet5kM2aKMTzlyy/hu+/g+OOD\nlsTwCj9MGH54uSTreZPNWI/d8JSyMhg1CvKsy5AzpGrCSMZ844fHUbKeN9mMKXbDU0pLIcvcuo0m\nSMVMkor5Jt0eR6l43mQr1q8yPKOiwjHFlJQELYnhJamYSTLRA8UPz5tMwXrshmfMmAH/8R/QunXQ\nkmQWyXqU+BEaOFkzyQl33MFZP/85XXfuZHPbtvzXHXfElSkTPVBS8bzJiNWQUiFWdDA/Nyy6Y04w\nbJizsIZRjx8rA6Vbpn/PnauX5uc3yH9pfr7+e+7cmHUkG63RD5KVKRNXmgqHIFdQAlYBHwGLgIUx\n8qT3DBhpZ+NGZwm86uqgJckskg0vm4mr/KQSvvaGkSP1126+UP5fg94wcqRn7UiWZGXKxPDG4cRT\n7H6YYmqBElWt9KEuIyBmzoTTToP27YOWJLNI1iThhwkj2TpSCV9bUFXFFTRcEel6YHqAHijJypSJ\n5qRE8UOxCzZIm/OUlcEFFwQtReaRrEdJqh4oydiBk62jurCQZVVV/I16hXg+8WPL5PXqxY6ItB1N\ntCPd5PXqRXdgQlia19ciY4jVlfdqA1YA7wPvAVfFyJPWVxYjvWzdqtqpk2plZdCSZB7J2qf9sMkn\ne8yLzz6rYyJMGGNAX3z2Wc/a7QeZON7RHAjYxl7kfvYAFgMnRcmT3jNgpJXnn1cdPjxoKTKTiaNH\n61LQiaC3u59LE1jCbeLo0Xr70KE6cfTouIokVTtwuuvIVPt0Mu1OJb+fxFPsaTfFqOo693OjiJQB\nxwL/jsw3ceLEuu8lJSWUmDN01lBaCmefHbQUmUltRQWDaPj6D00v4ZboRJ1U7cDfrFnDx/Pn06Gy\nkuqVK/lmzZqY5ptU6qitqOA7GtqzL0tArnST7CSoTArTXF5eTnl5eUJ506rYRaQ9kKeq20WkA/AT\nYFK0vOGK3cgedu6Ef/wD7r8/aEkyk01ueNxIO+0mj5ahS8UOPH/ePB459VSe3LvXmYFZVcW4U0+F\nWbOiRnhMpY6tnTtzP9RNCMrlWZ5+EdnhnTQpqip1iNWV92ID+uOYXxYBnwA3xciXzjcWI4288orq\nyScHLUXmMrxXr6gudsN79fKk/FTswMm6L6ZSRyb6secaBGWKUdWVwI/SWYcRLLm4BF48kvVA6bF9\nO9fT2MVuvEfL0KUSPKtDZWVUM0ks98VU6vAjvroRGwspYKTM3r3w8stw++1BS+IPqQS2qi4spHtV\nVSMXOy+XoUvWDvxNfn5UM8k3ccxDydaR1a6CuUCsrryfG2aKyUrmzFE96qigpfCPVDw9MtHt76S2\nbaO246S2bT2rI9NdBXMBAp55auQomWaGSXfAplS8QwYPGcK6p55qGEDr0UcDXYauaM+eqO0o2rPH\nszr8iK9uxCGWxvdzw3rsWUdNjWqvXqpLlwYtiYMfPcRxp50Wtac77rTTPJPLj3YM69AhajuGdejg\nWR1G+iHICUqJbKbYs49331UdODBoKerxY0JMKh4umRgELJWZpEbmEU+xmynGSIlMm5Tkx/JtPbZv\n51waL602NY6HSyYGATv3wgsBGpmHQulG9mOK3UgaVUexP/NM0JLU48fybRs7duSlqqpGS6tt7BCp\niuvZUVAQVa4dnTp51o5UOPfCC02R5zAWddFImqVLnRmnRx8dtCT1+LF8W/GgQVGXViseNChmHXtF\nGO/KE5JrvJvuVTsMIxLrsRtJE/KGiaGbAiEVL4xkzR7d3Cn4kfm71dTErKOgqoo2OLE09gPWAz8D\ndseIAd63f3/Onj6dsZdeSoctW6ju0oUbp08PfCk9I7swxW4kTWkp3Hdf0FI0JtlJNH6YSRZs2MC+\nwJvUm2+uBjasXx81/+qVKym74gqeXLXKyb9lCxOuuILeMcxDqUyaMnIfM8UYSbFyJVRUwEknBS1J\n8/HDTLJt2TL+j4bmm/9z06ORrHko2fxGy8B67EZSlJXByJHgUXBCT0nWJFFQVcVPaOzl8mYcM8kJ\nd9zR0Jvkjjvi1lGkGn0ykOPm24hM9KIxsg9T7EZSlJbCrbcGLUVjUjFJbO3cmZegkZdLrNCy8+fN\nY+Yll/ByKNxtdTXjLrmEop49Y84kXS9CdYRyr3bTo+HHUnpGCyCWg7ufGzZBKSv49lvVLl1Ud+4M\nWpLGpDKxJ9nQssmGu1VVveO226JOBrrjttui5s/EmapGZoJNUDK84O9/hzPOgH32CVqSxqRikkg2\ntGyHysqo+WOFuwW4dfJk7gR+cued7KfKehF+duut3BrDLp+sd4/FZDGi0aRiF5EewFVAv/D8qnpF\n+sQyMpHSUrjqqqCliE4qJom8Xr1YBvyN+rjk58c5prqwkOqqqkZ1NBWC99bJk2Mq8mhk8/JtRmaQ\niFfM34HOwD+BV8M2owVRWQnvvAPDhwctSXRS8Vg5/Mwz+T3wW5z1Gn8L/N5Nj8aNTzzBOHepu1Ad\n4/LzufGJJzxqhWF4g2iM0fm6DCKLVTWtqyCJiDYlhxEsTz0FL70EM2YELUls6rxiXJNEU14x5/bv\nX+cvHqIaGNuvHy+tXBn1mPnz5vGn8MlDTzwRNwSvYaQLEUFVo47CJ2Jjf0VEfqaqr3ksl5FFZFrQ\nr7gk2ElIxWY+eMgQBsdQ+rGwmaGG38RU7CKyDVBAgFtEZBewx/2tqmrLjbcQqqth9myYNi1oSWKT\nirvjxnbtotrMN7ZtG6hchtFcYtrYVbWTqha4n3mq2i7styn1FsQbb8Cxx0LXrkFLEptUZmDW1NZG\nnXlaU1sbqFyG0VwS8YqZpaqnNpVm5C5eLYGXTpNEKu6ORbt2RY2v/t3u3Z7IlKpchtFc4pli2uLc\ng91FpBDHBANQANi0thbC7t3w6qvwv//bvHLSbZJIxd1x3T77RJ15uq5Nm2bL0xy5DKPZxJq5BFwP\nrAR2uZ+h7SPg2ljHxSgrD/gQeDnGfu+nZRme8I9/qJ5wQvPLSfeSb6nMwBxeVBRVpuFFRZ7IlKpc\nhpEIpDLzVFXvB+4XketU9c/N/P+4HliK09s3sgivzDDpNkmkEse8x/ffR5Wpx86dnsgUkstmhhp+\nE88UE3qcK8K+16GqpYlUICK9cdYWuBPHhGlkCTU1jt/62283v6xkY58nS7JxzAHWt24dVab1+d5G\n2rCZoYbfxJt5OsLdrgCmAaPd7VE3LVHuBf4bx3XSyCLeeQeKiqC4uPllJRv7PFlS8T6p2rYtqkxV\nMWLFGEa2EM8UczmAiLwFHKqq37q/9wceT6RwEfkPYL2qLhaREuoHYBsxceLEuu8lJSWUlJQkUoWR\nRrwyw4AT+/wK4B7q47JcD0yPEfsckvOiScXUc8DevVwfRaY1e/d6IpNheEl5eTnl5eWJZY5lfNf6\ngc1l2nggdFlTx7l5fw98DawAvgW2A09GyZfWQQYjeWprVfv2Vf34Y2/KS3bwNNlBx2RD8KqqDuvQ\nIeoxwzp08EQmw0gnxBk8TSQI2CwReUNELhORS3ECgP0zwT+NW1S1j6oeCFwIzFbVsYn95RhBsmgR\n5OfD4Yd7U16yQbqSNa2kYur5r0cf5eqIY652072QyTCCoslRIlW91h08PdlNelhVy9IrlhE0ZWWO\nGcYjE3jhaxq1AAAgAElEQVTS3iG1FRV8R0MzyWXENq2kYuo598ILARoudffoo3Xp0WSyyUZGNpDQ\n8L86HjAJecHEKWMuMLc5ZRj+UVrqfWyYZLxDtnbuzP3AZBJbti6vVy+6AxPC0hKZCHTuhRfGVOTR\n6rDJRkY2ENMUIyL/dj+3icjWsG2biMTuBhlZz2efwZYtTnyYoMhXrVPquJ+T3fRopBKPPVn8qMMw\nvCCeV8xJ7qc3jsZG1lBWBqNGQV4iIzBJkIxHSbLL1vkxEcgmGxnZQrwJSvcB84H5qmpGxBZEWRnc\ndZe3ZSYbKyYVs4cfE4FsspGRDcTrk30FnA28LSKrROQZERknIkeKiMd9OSNT+PprWLECvF4UKFmP\nEjN7GEbqxDPFTAGmQN2kpMHAicCvgX2xuC85yYwZMGIEtG7tbbnJepSY2cMwUieuV4yICPADHIU+\nGDgUpyf/VPpFM4KgrAx+/Wvvy81U04rNJDVykZiLWbuhBAqAxcACYIGqLkuLELaYdUawcSMcdBCs\nWwft2nlbdlQbe3FxoEvEZaJMhpEo8RazjqfYpwJHADtwFPs7wDuq+l0aBDTFngFMm+Ysg/e3v6Wn\n/LresWtaCbp3PGnMGH779NON3iLuGT3aBkiNjCeeYo9nY/+le3ABcDyOOWaciPQAlqjqpekQ1giO\n0lIYMyZ95X+zZg0fz59Ph8pKqleu5Js1awJV7DaT1MhVEpl5ugun1/69+7034N3aYUZGsHUr/Otf\n8Oyz6Sl//rx5PHLqqTy5d69j9qiqYtypp8KsWQz22gUnQWwmqZGrxDPF3IvTSz8Yx87+dmhT1S2e\nCmGmmMB57jl48kl47bX0lH9u//51i2CEqAbG9uvHSytXpqfSJjAbu5HNpGSKwVnf9GlgkarWpEUy\nI2MoLYVzz01f+R0qK6OaPTps8bSPkBTmUmnkKjF77L4KYT32QPn+e2elpK++gh490lNHJvbYDSOb\niddjtxmkBv/8Jxx5ZPqUOsCNTzzBuPz8BjNJx+Xnc+MTT6SvUsNooZhiNzxdAi8Wg4cM4fh77mF4\nq1acCwxv1Yrj77knsIFTw8hl4g2edo13oKpu9kwIM8UExp49sP/+zopJBxyQvnpCXjEPhrxicHrs\nVwXoFWMY2UyqE5RWAkr0BajVXe7OKwFNsQfErFlw882wcGF66zEbu2F4S6oTlMw1oAVQWgpnn53+\nejLRK8YwcpWElsYTkUIcf/a2oTRVnZcuoQx/qK11ojnOnp3+uqoLC6muqmrUY6/u0iX9lRtGC6PJ\nwVMR+TkwD3gDmOR+TkyvWIYfLFwIXbrAgAHpr8u8YgzDPxLxirke+DGwWlWHAkcC9v6cA/jhDRNi\n8JAhXDVrFmP79WNsly6M7dfPBk4NI000OUFJRN5T1R+LyGLgOFXdJSKfquphnglhg6e+owoHH+xE\ncjzqqKClMQwjWVINKRDiGxHpAswA3hKRSmB1ghXvg2PGaePW9aKqTkpMbCOdLFkCe/c6E5MMw8gt\nkgopICKnAJ2B11V1T4LHtFfVHSLSCmdx7F+p6sKIPNZj95lJk6CqCv70p6AliY2tbmQYsWlWj11E\nnlLVSwBUdW4oDbgkkcpVdYf7dR+3PtPgGUBpKTz4YNBSxCZq5MUFCyzyomEkQCKDpw1s6W7P++hE\nKxCRPBFZBKwD3lLV95IT0fCa5cth/Xo44YSgJYnN4+PH1yl1cHzeJy1fzuPjxwcplmFkBTF77CJy\nM3AL0E5EtlI/A3U38HCiFahqLXCkuxLTDBE5VFWXRuabOHFi3feSkhJKSkoSrcJIkrIyGDkSWrUK\nWpLY2OpGhtGQ8vJyysvLE8qbiFfMXap6swdyISK3A9tV9U8R6WZj95ETT4QJE+CnPw1aktjYeqSG\nEZ/mhu29VUTGiMh4t7ADROTYBCvuLiKd3e/tgNOAzxKU20gDa9fCZ5/B0KFBSxKfyyZPZkJxcYMJ\nTROKi7ls8uQgxTKMrCCRHvtDQC0wTFUHueEF3lTVHzdZuMgPgCdw/kDygOdV9c4o+azH7hN/+Qu8\n8w489VTQkjRNnVeMu7qRecUYRj0pRXcMO/hDVT1KRBap6pFu2keqeoSHAppi94nTToP/+i//Zpwa\nhpEemmuK2eN6wqhbWA+cHryRZWzeDO+9l9m2dcMwmk8iiv0BoAzYV0TuBP4N/D6tUhlpYeZMOPVU\n6BDpbmIYRk7R5AQlVX1aRD4ATsVxeRylqsvSLpnhOaWlcN55QUthGEa6ibeCUlvgauAg4BNgmqru\nTYsQZmNPO9u3Q8+esHo1FBYGLY1hGM0lVRv7E8AxOEr9DOCeNMhm+MQ//uHMNDWlbhi5TzxTzKGq\n+gMAEZkGpHlVTCOd+Bl73TCMYInXY6+L3pguE4zhD7t2weuvO2EEDMPIfeL12I9wY8SAM2gaHjNG\nVbUg7dIZnjBrFhx2GBQVBS2JYRh+EFOxq2oGh4gykqGszMwwhtGSSGqhjbQJYV4xaaOmBvbfH959\nF2w2vmHkDs2deWpkMf/+N/TubUrdMFoSpthznLIyOPvsoKUwDMNPElnM2shSVB03x9dfD1oSwzD8\nxHrsOcwHH0C7dnDooUFLYhiGn5hiz2FKSx0zjEQdXjEMI1cxxZ7DmJujYbRMTLHnKMuWOYG/jjkm\naEkMw/AbU+w5SsgMk2dX2DBaHPbY5yghxW4YRsvDZp7mIKtXOyaYb7+FfHNoNYycxGaetjDKyuCs\ns0ypG0ZLxRR7DmKx1w2jZWOmmBxj/XoYMADWrYO2bYOWxjCMdBGYKUZEeovIbBFZKiKfiMiv0lmf\nAS+/DMOHm1I3jJZMuk0xe4EbVfVQ4ARgnIgMTGuNDz4IGzbE3r9hg5Mn0+tIETPDGIaRVsWuqutU\ndbH7fTuwDOiVtgoffBCuvRaGDo2ueDdscPZde23qitePOlKkqgrmz4czzvC1WsMwMgzfBk9FpB/w\nI+DdtFVy3nlOxKulSxsr3pDCXbrUyXPeeZlbR4q8+iqccgp06uRrtYZhZBi+OMSJSEfgReB6t+fe\niIkTJ9Z9LykpoaSkJPmK9t0X5sypV65Dhzq/oaHCnTPHyZsKftSRImaGMYzcpby8nPLy8oTypt0r\nRkTygVeA11X1/hh5vPWKCe859+jhpG3c6K3C9aOOJNixw1kCb/ly6N7d16oNwwiAeF4xfij2J4Hv\nVPXGOHm8d3fcsAEOP9xRtuAo3yVLvFW4ftSRIDNmwJ//DLNm+V61YRgBEKS742BgNDBMRBaJyIci\nMjyddabksZLsMX7UkSS2BJ5hGHWoauCbI4YHTJmiCqqHHKI6YIDzvUcPZwMn7ZBDnO9TpjQ85tBD\nVdevb1zm+vXOvtAxftSRJLt3q3btqrpmTdKHGoaRpbh6M7pOjbXDz80zxb5+fb1SDSnZ9eudLaSE\nQ0o5pGDDlWqk4o22z486kuTNN1WPO64Z580wjKwjnmLPvVgx4evAhdvtw7+H5wl5uUS6MEa6LoYP\niPpRRxKYN4xhGA2IpfH93PDaFLPvvvW96nAzySGHOPuimTzCe87hx0T2okN1FBfHrqO4uHl1hOqJ\n13tfv151yhStqVEtKlL94ovmnTrDMLILWowppkuXelPIAw/UK86QIn3ggfrfXbo0Pn79+sbHRFO4\nIQW+ZEnj/EuWNLaxp1pHAjb5+Te+qIcfnvopMwwjO4mn2HPLFHPfffXff/Urx7k7xI4dTlq0vMkQ\nmnn6xRdw7rlQW1u/r7bWSfviC99mt5Z+f4aZYQzDaEgsje/nRrwee+/eqjNmxN4/Y4aTJ0Tr1vW9\nYVDt0MHZwtNat25YxpQpTk87lplkyZKGve/DD1c94ID68rp1c7bQ7wMO0Kjd6GRMMbEGbsPSatet\n1/79VRcvjn16DMPITchaU0zv3vXKMppynzGjfn/v3qrt2jVU4PG2du2cMkJmj332ia1EQ/umTFE9\n8sjGZUUq9tB25JH1siagqOMq9yh/BIsWqR54oGptbfwbwDCM3CN7FXu44o5U7tH23Xln4or9zjud\ncpYsqVfc++zj/A4RbV9kvQUF0b+Hy9scd8c4Nvnbb1f9zW9iXHXDMHKaeIo9s23sI0fCL35R/3vU\nKPj7351t1Kj69F/8wsnbuXPiZYfylpfDrl3OAqG7dsH559e7Ip5/fsN95eXwzTcNy9m6Nfp3qM/7\nwguxXRojXSFfeCHhJpibo2EYUYml8f3ciNVjD5lJwk0ykVto35Qpqnl5iffY8/KcOsJNK3371veK\nQ73kUFq4aWXKFNXHH49d9uOPN/aISdB9sVFaDFPM529/p0VFqjU1sYs0DCN3IWtNMeGKLZpyD6WF\nTBg335y4Yr/5ZqeOuXNVRZw0kYYuk126NNw3d25juaJtKc4gjdn2KDb5u/f9f3rNZdXNq8NoMfTt\n21cB27Jw69u3b9Rrmr2KXTU5JRqvFx25Pf54fR0lJU3nLympl6eoyEkL782HtlBaUVHqyj0Bm/yx\nLNA3+1zZ/D8Qo0UQ9xkzMppY1y67Fbtq02aPEKHedSKbEypY9bzznN/5+bHzhvadd17y+VOhiQlK\naxZt1K6tKnU3+SkFDTNaHqbYs5dUFHva47EnQtx47JEDpdGYMcMZPE0kb+Qxn34KRx0Fu3fHz9+m\nDXz4oRNzfdAg2Lw5JLwzqApQUuKodoCuXWHZstRjsz/4oDNRKcrxU6bAe//6nieGTIdx41Ir32hR\nuLG7gxbDSIFY1y6weOzNJlJRhy/mGf495C3z9deJlx3K26MH9O7ddP7evZ28n30GlZX16X36wMCB\nztanT316ZaWTN1XGjYv5p1BaCudc3M6UumEY0YnVlfdzI9ZrYviAaeh7uHdI5P5UvGJCZo/994+d\nN7QvfIKSSGwvmpBJKHyCkkds3Oi4y+/Y4XnRRg7T6BlLxUsrEi/KMJokln4kq23sPXs29n6J9Jbp\n2dPJu2RJ4oo9fCLSDTfEt8+LOHlCHHmk4yETa/LQ3LlpUeqqqtOnq557blqKNnKYBs+YFwu/pHnx\nGKOe3FPs4R4o8WZshjxQmuvu2NRga8jdMVyGpiI1esyZZ6o+/XRaqzBykAbPmBcLv6R58ZhEWLVq\nlYqI1uT4ZI7cU+whD5TIqf4hwqf8n3de8ycohbxZROqVfbj3S6zYL00F9PKIrVtVO3VS3bIlLcUb\nOUyjZyyV2EWReFFGM1i5cqXm5eXp3r1701ZHJpB7ij3ZHrtq4r3vcAYOrFfgoT+R8D+N/HwnT7S6\nfbyhn3tOdfhwz4s1WgBRnzEvOicednDuvvtu7dWrl3bq1EkHDhyos2fP1traWr3rrru0uLhYu3fv\nrhdccIFWVlaqqmqfPn00Ly9PO3bsqJ06ddIFCxZobW2tTp48Wfv27av77befXnrppVpVVaWqqjt3\n7tQxY8Zot27dtEuXLnrsscfqhg0bVFX1scce00GDBmmnTp20uLhYp06dmpTs6ST3FLtqcko0MkBX\nvC0UoCvZIGABvoJecIHqww97WqTRQojbeWquOdGDMj7//HM94IADdN26daqqunr1al2xYoXee++9\nesIJJ+jatWt19+7devXVV+tFF12kqo4pJi8vT2vDwptOmzZNDz74YF21apVWV1frOeeco2PHjlVV\n1alTp+pZZ52lO3fu1NraWv3www9127Ztqqr62muv6cqVK1VVdd68edq+fXtdtGhRcuchTeSmYldN\nvFcQHg4gZJYJN6uEm2pCKygla+4JaNDo++9VO3e2iaZGamS6Yv/qq690v/3203/+85+6Z8+euvRB\ngwbp7Nmz636vXbtWW7durTU1NXWmmHAb+6mnnqoPPfRQ3e/PP/9c27RpozU1NTp9+nQdPHiwfvzx\nx03KM2rUKH3ggQeSakO6yDjFDkwD1gMfN5Gv6dYlcvOsX6/aqlV9njZt6s0qbdrUp7dq1fDY886L\nrtRDLFnScBZpAG5eM2eqDhniaZFGCyIbTDHPPvusnnTSSVpYWKgXXXSRrl27Vtu3b6+dO3fWwsJC\nLSws1C5dumj79u117dq1dT32cMU+aNAgfe211+p+79y5U0VE165dq3v27NH/+Z//0UMPPVR79eql\nv/vd7+rs86+99poef/zx2rVrV+3SpYvus88+evvttyclf7rIRMV+EvAj3xR7pAIvLq433YQWmA5X\n+FnE5Zer3ntv0FIY2Uo2DZ5u27ZNL7roIr3kkkt04MCB+vbbb0fNt3r16qR67JHHHnrooTp9+nTd\ntWuXtm/fXktLS+vyjRo1SsePH5+07OkgFcWe1pmnqvpvoLLJjE0RWudz40Zn9mePHs73yPVAJ01y\nQgO0aQPFxbB8ORx+uLMtX+6ktWnj5Jk0qdli+cXevTBzJpx9dtCSGDlBxLq5dWsERK4NEPl8eV1G\nGF988QVz5sxh9+7dtGnThnbt2pGfn8/VV1/NLbfcwtfuTPGNGzfy8ssvA9CjRw/y8vJYvnx5XTkX\nXXQR9957L6tWrWL79u3ceuutXHjhheTl5VFeXs6SJUuora2lY8eOtG7dmvz8fHbv3s3u3bvp3r07\neXl5vP7667z55pvNP89BEkvje7UBfWlOjz3ZXkHIrBKrhx9pVskCZs9WPfrooKUwspkGz1gGTlD6\n+OOP9dhjj9WCggLt1q2bjhgxQr/99lutra3Ve++9VwcMGKAFBQV60EEH6a233lp33IQJE7RHjx5a\nWFio7777bp1XzAEHHKD77ruvjh07Vre4/sHPPvusDhgwQDt27KhFRUV6ww031PXQ//KXv+h+++2n\nhYWFOnbsWL3ooouyusee9iBgItIXmKmqP4yTRydMmFD3u6SkhJKSkti9Aoi/L7T/8MOdnj04vfwl\nS1IPyhUg110H++8Pt9wStCRGttIokFScIHOA8/y88EL8eERelGE0SejalZeXUx4KOAhMmjQJjREE\nLGMUe1Q5HnwQrr02uuKGhsp9ypT6Gyg8vUcPJ23jxtjlZDC1tU5ssbfecoJKGkYqWHTH7CWV6I75\naZcKxN2SJ6SoY/UKQva88F5BtJ481KcNHZpVyv39951AlqbUDcNIlLT22EXkGaAE6Ibj9jhBVR+L\nki96jz1ZmmO6yVBuuglatYI77wxaEiObsR579pJx8dhV9WJV7amq+6hqn2hK3VNeeCG24o4crX/h\nhbSK4gWqbuz1c4KWxDCMbMIPU4x/pGK6yWCWLoVdu5wFngzDMBIl85fGa8FMngybNsF99wUtiZHt\nmCkme8k4U4zRPMwMYxhGKphiz1BWroSKChg8OGhJDMPINkyxZyhlZTBypOMRYxiGkQym2DMUM8MY\nLYl+/fpRVFTE999/X5c2bdo0hg4dGqBUDcnLy2PFihVBi5EQptgzkG+/hU8/hWHDgpbEaAmsXrmS\nSWPGMGHoUCaNGcPqlSt9L0NEqKmp4b4ITwGR1OY2poNMkqVJYgWR8XMjkbC9LYiHHlJ1F4kxDE+I\n9YytWrFCf1NcrNvdYHnbQX9TXKyrVqxIuGwvyujXr5/+4Q9/0G7dutUtZffoo4/q0KFDVVV1/vz5\n+uMf/7huSbvwUL4lJSU6fvx4HTx4sHbq1El/+tOf6qZNm2LW9dhjj+mBBx6onTp10gMPPFCfeeaZ\nun3Tpk3TQYMGadeuXXX48OH69ddfq6rqkCFDVES0Q4cO2qlTJ/3b3/6mqqoPP/ywHnTQQdqtWzcd\nOXKkrl27tq6sG264Qffdd1/t3LmzHnHEEfrpp5+qquqrr76qRx55pBYUFGifPn104sSJcc9NrGtH\n1q+g1ML4yU9UX3ghaCmMXCLWMzZx9Og6haxhinni6NEJl+1FGf369dNZs2bpueeeq7fddpuq1iv2\nzZs3a2FhoT799NNaU1Ojzz77rBYWFurmzZtV1VHsBx10kH711Ve6c+dOLSkp0ZtvvjlqPdXV1VpQ\nUKBffvmlqqquW7dOly5dqqqqZWVlevDBB+vnn3+uNTU1euedd+qJJ55Yd6yI6IqwP6tZs2Zp9+7d\ndfHixbp792697rrrdIi7Gs4bb7yhxxxzjG7dulVVVT/77LO6Zf/mzp2rS9z1ID755BMtKirSv//9\n7zHPTSqK3UwxGUZlJbzzDgwfHrQkRkugtqKCDhFpHYDatWt9LSPEpEmTmDJlCps2bapLe/XVVznk\nkEO4+OKLycvL48ILL2TgwIHMnDmzLs/ll19OcXEx++yzD+effz6LFy+OWUerVq345JNP2LlzJ/vt\ntx+D3EBMDz/8MDfffDOHHHIIeXl53HTTTSxevJg1a9bUHath/uTPPPMMV155JUcccQStW7fmrrvu\nYsGCBXz99de0bt2abdu2sXTpUlSVAQMGsN9++wEwZMgQDjvsMAAOP/xwLrzwQubOnZv0uYqHKfYM\n45VXHNt6x45BS2K0BPJ69aI6Iq0ayOvZ09cyQhx22GGceeaZ3HXXXYCjSNeuXUvfvn0b5Ovbty8V\nFRV1v4uKiuq+t2/fnu3btwNwzTXX0KlTJwoKCrj77rtp3749zz//PA899BD7778/I0aM4IsvvgBg\n9erVXH/99XTt2pWuXbvSrVs3RKRBPeFEytWhQwe6du1KRUUFQ4cO5dprr2XcuHEUFRVx9dVX18m0\ncOFChg0bxr777kuXLl2YOnUq3333XdLnKh6m2DOMsjJbKcnwj8smT2ZCcXGdYq4GJhQXc9nkyb6W\nEc7EiRN55JFHqKioQETo1asXq1atapDn66+/plevXk2W9dBDD7Ft2za2bt3KTTfdBMDpp5/Om2++\nybp16xgwYABXXXUVAAcccABTp05l8+bNbN68mcrKSrZv387xxx8fteyePXuyevXqut/V1dVs2rSp\nTq5rr72W999/n08//ZTPP/+cP/7xjwBcfPHFjBo1ioqKCrZs2cIvf/lLz2cFm2LPIKqrYdYsGDEi\naEmMlkLf/v257q23uGf0aCYMHco9o0dz3Vtv0bd/f1/LCKe4uJgLLriABx54AIAzzjiDL7/8kuee\ne46amhqef/55li1bxogUHpQNGzYwc+ZMduzYQevWrenYsSOt3MkiV199Nb///e9ZunQpAFVVVbz4\n4ot1xxYVFTVwd7z44ot57LHH+Pjjj9m1axe33HILJ5xwAn369OH9999n4cKF7N27l3bt2tG2bVvy\n853QXNu3b6ewsJDWrVuzcOFCnnnmmZTOU1xiGd/93LDBU1VVfekl1dNOC1oKIxfJ9Gesf//+OmvW\nrLrfa9as0Xbt2umwYcNU1fGKOfroo7VLly56zDHHNPCKGTp0qE6bNq3u9+OPP64nn3xy1Hq+/fZb\nPeWUU7RLly5aWFioQ4cO1WXLltXt/+tf/6o/+MEPtHPnztqnTx+98sor6/ZNnTpV999/fy0sLNQX\nXO+GqVOnanFxcd1yfhUVFarqDKz+8Ic/1E6dOmmPHj10zJgxWl1draqqL730kvbt21cLCgp0xIgR\net111+kll1wS89zEunYEuTReIlgQMIcxY5wQAtdcE7QkRq5hQcCyl1SCgJlizxB274aiImdZ1hTG\nnAwjLqbYsxeL7pjFzJkDAweaUjcMo/mYYs8QLDaMYRheYaaYDKCmxumpv/02FBcHLY2Ri5gpJnsx\nU0yW8s47jn3dlLphGF5gij0DMDOMYRhekluLWWchqo5iDwt7YRie07dv3+wKO2vUERlOIRHSrthF\nZDhwH87bwTRV/UO668wmFi2C1q3h8MODlsTIZSKn5Bu5TVpNMSKSB0wBfgocBlwkIgPTWWc2UV5e\nXhcbpiV1psrLy4MWIRCs3S2LINudbhv7scCXqrpaVfcAzwEj01xn1lBeXt4i7ev2oLcsrN3+k27F\n3gtYE/b7GzfNAL77DrZsgWOPDVoSwzByiXQr9mgGBnOmdVm2DEaNgjzzTTIMw0PSOkFJRI4HJqrq\ncPf3TTgRyf4Qkc+UvWEYRpIEEgRMRFoBnwOnAt8CC4GLVHVZ2io1DMNo4aTV3VFVa0TkWuBN6t0d\nTakbhmGkkYyIFWMYhmF4h2/DdiIyXEQ+E5EvROR3Ufa3EZHnRORLEXlHRPr4JVs6SaDdl4rIBhH5\n0N2uCEJOrxGRaSKyXkQ+jpPnAfd6LxaRH/kpX7poqt0icoqIbAm73rf5LaPXiEhvEZktIktF5BMR\n+VWMfDl1vRNpd2DXO9bSSl5uOH8gXwF9gdbAYmBgRJ5rgL+43y8AnvNDtgxo96XAA0HLmoa2nwT8\nCPg4xv4zgFfd78cBC4KW2ad2nwK8HLScHre5CPiR+70jzrha5H2ec9c7wXYHcr396rEnMlFpJPCE\n+/1FnAHXbCfRCVo5N+9UVf8NVMbJMhJ40s37LtBZRPbzQ7Z0kkC7Iceut6quU9XF7vftwDIaz1fJ\nueudYLshgOvtl2JPZKJSXR5VrQG2iEhXf8RLG4lO0DrHfT39m4j09ke0wIk8NxW0nMlrx4vIIhF5\nVUQODVoYLxGRfjhvLO9G7Mrp6x2n3RDA9fZLsScyUSkyj0TJk20k0u6XgX6q+iNgFvVvLblOS528\n9gHQV1WPxImjNCNgeTxDRDrivG1f7/ZgG+yOckhOXO8m2h3I9fZLsX8DhA+G9gbWRuRZAxwAdf7v\nBara1CttptNku1W10jXTADwCHO2TbEHzDe71dol2T+QcqrpdVXe4318HWufAmykiko+j3J5S1b9H\nyZKT17updgd1vf1S7O8BB4lIXxFpA1yI01MNZybOQCLAecBsn2RLJ022W0SKwn6OBJb6KF+6EWLb\nF18GxkLdDOUtqrreL8HSTMx2h9uVReRYHJfjzX4JlkamA0tV9f4Y+3P1esdtd1DX25eFNjTGRCUR\nmQS8p6qvANOAp0TkS2ATjhLMahJs969E5CxgD7AZuCwwgT1ERJ4BSoBuIvI1MAFogxNS4mFVfU1E\nfiYiXwHVwOXBSesdTbUb+E8RuQbnen+P4wGW1YjIYGA08ImILMIxsdyC4w2Ws9c7kXYT0PW2CUqG\nYTGaLKEAAALvSURBVBg5hsUVNAzDyDFMsRuGYeQYptgNwzByDFPshmEYOYYpdsMwjBzDFLthGEaO\nYYrdyApEpMYNe/qJiDwvIm2byH9zguWujDYTMFa6V4jISBEZGPZ7jogcla76jJaFKXYjW6hW1aNU\n9Qc4kz2ubiL/LQmWG2siR7oneIwCDktzHUYLxRS7kY38CzgIQERGi8i7bm/+IRHJE5G7gHZu2lNu\nvjIRec/t8f88rKxYIQ8apYtIe3chjXdF5AMRGeGmXyoiL4nI6yLyuYj8IeyYK920BSLysIj8WURO\nAM4C/teV8UA3+/lu2Z+5sxoNIyV8CSlgGB4gUBd06QzgddeUcQFwohu+4UHgYlW9WUTGqWq4aeNy\nVd3imnDeE5GXUggydyswS1WvFJHOwEIR+ae77wicsK17gM9F5AGgFrjNTd8OzAEWq+o7IvIyMFNV\nS912AbRS1eNE5AxgInB6kvIZBmCK3cge2onIh+73eTixhX4JHIWjqAVoC6xz80T2uG8QkVHu997A\nwcDCJGX4CTBCRP7b/d2G+uids0IhW0XkU5x4IT2AclWtctNfcOuNRan7+YF7vGGkhCl2I1vYEdED\nx1XmT6jqrfEOFJFTgGHAcaq6S0Tm4PwJpMK5qvplRPnHA7vCkmpxnq140S2jESqjBns2jWZgNnYj\nW4imIGfhRM/rASAihSISivm9243rD9AZqHSV+kDg+BTrfAOoW7BYml6QeSEwREQ6uyakc8P2bQMK\nkqjbMBLGFLuRLTTyUlHVZTg27DdF5COc8Mj7u7sfxgmn+hQQWuDgU+D3wDvxyg1L/0hE1ojI1yJy\nDzDZLedjEfkE+J94sqrqWre+hTgDviuBKjfPc8B/u4OwB0aRw8KuGiljYXsNI42ISAdVrXbfHspw\nYvJHW2HIMDzDeuyGkV4muoswfAKsMKVu+IH12A3DMHIM67EbhmHkGKbYDcMwcgxT7IZhGDmGKXbD\nMIwcwxS7YRhGjmGK3TAMI8f4/2jTa4gSsC/2AAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7fa9c4752358>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Pull out slope/intercept\n",
    "[[slope]] = sess.run(A)\n",
    "[[intercept]] = sess.run(b)\n",
    "\n",
    "# Create fitted line\n",
    "x = np.linspace(0, 3, num=50)\n",
    "ablineValues = []\n",
    "for i in x:\n",
    "  ablineValues.append(slope*i+intercept)\n",
    "\n",
    "# Plot the fitted line over the data\n",
    "setosa_x = [a[1] for i,a in enumerate(iris_2d) if binary_target[i]==1]\n",
    "setosa_y = [a[0] for i,a in enumerate(iris_2d) if binary_target[i]==1]\n",
    "non_setosa_x = [a[1] for i,a in enumerate(iris_2d) if binary_target[i]==0]\n",
    "non_setosa_y = [a[0] for i,a in enumerate(iris_2d) if binary_target[i]==0]\n",
    "plt.plot(setosa_x, setosa_y, 'rx', ms=10, mew=2, label='setosa')\n",
    "plt.plot(non_setosa_x, non_setosa_y, 'ro', label='Non-setosa')\n",
    "plt.plot(x, ablineValues, 'b-')\n",
    "plt.xlim([0.0, 2.7])\n",
    "plt.ylim([0.0, 7.1])\n",
    "plt.suptitle('Linear Separator For I.setosa', fontsize=20)\n",
    "plt.xlabel('Petal Length')\n",
    "plt.ylabel('Petal Width')\n",
    "plt.legend(loc='lower right')\n",
    "plt.show()"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.5.2"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}