{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Logistic regression with an evolutionary algorithm"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In this notebook, we will show how to use Auxein and a simple evolutionary algorithm to perfom a [logistic regression](https://en.wikipedia.org/wiki/Logistic_regression)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "%matplotlib inline  \n",
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "import os\n",
    "import logging\n",
    "\n",
    "logging.getLogger().setLevel(logging.CRITICAL)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As a first step, we will generate a sigmoid function and we sample (and round to the closest integer) $100$ data points from it (in the form of $(x_{i},y_{i})$) which will represent our observations of a binary classification $y = f(x)$ where $y_{i} \\in \\{0, 1\\}$"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "x = np.sort(np.random.choice(np.arange(-50, 50, 0.01), 100))\n",
    "\n",
    "alpha = -1.5\n",
    "b0 = 0.125 \n",
    "y = 1 / (1 + np.exp(-(alpha + b0*x)))\n",
    "y_rounded = np.array(list(map(lambda yi: round(yi), y)))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "And then we visualise our observations $(x_{i},y_{i})$:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXQAAAD4CAYAAAD8Zh1EAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjMsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+AADFEAAATzUlEQVR4nO3dfbBcd13H8ff3bjZwA9hbyEXoTUI6TiwWSgneaet0RkHBhIppwFYSrKIyZBipDyNGW2CAqTBFM4I41ocOMooitQiGDBMnItRxhiG1N8TSSUswFiE3KTQ8BB8aaB6+/rG7YbM5u3tu7t7e5tf3a6bTe875nnO+v7N7Pzk552w2MhNJ0vlvbLEbkCSNhoEuSYUw0CWpEAa6JBXCQJekQixZrB0vX748V69evVi7l6Tz0p49e76emZNVyxYt0FevXs3MzMxi7V6SzksR8eV+y7zkIkmFMNAlqRAGuiQVwkCXpEIY6JJUCANdkgphoEtSIQx0SSrE0A8WRcQHgFcAD2fm8yuWB/A+4BrgEeAXM/Nzo25UUsv2vYfYtms/h48e46KJcbauu4SNa6d46/b7+JvdXzmrfqpdA/COHfs4euz4wO2PBZzK76038+Vv8uG7D3Ky67sTure5bdd+Dh09RgCdiguXNXn7Tz+PjWunzui7e/8XLmvyUy94Nnd94QiHjh6jEcHJzNPb7qzbPd4LxptEwNFHjp8x9jrHrHfsVT32W7fqePcu7z0Gy5pjPKnZ4FuPHD89tm5PWdrgXa+8rFb/dcWwL7iIiB8F/hf4YJ9Avwb4VVqBfiXwvsy8ctiOp6en00+KSnOzfe8hbv7YfRw7fvL0vPFmgxetuoDP/Oc3+67XHGsFyqk5fp/NGHBqwDYJOH6yeqPNRrDtusvZuHaK7XsPsfUj93K8ZgPjzQa3vuoygLPGW1U3KBQH7bu7x37rVh3vzj6rls9FYyz4g+v7779KROzJzOmqZUMvuWTmvwL93ylwLa2wz8zcDUxExLNrdyeptm279p8VHseOnxwY5gDHT809zKF/mHe22S/MoRX023btB1p91w1zaI1p2679leOtqhtk0L67e+y3btXx7h7XuYY5wMlTg/c/V6P4t1ymgINd07PteQ/1FkbEFmALwKpVq0awa+mJ5fDRY4vdwpx0+j2XvuuuM6xuPsv7LZvPuOay/7kaxU3RqJhX+cdhZt6emdOZOT05WfmPhUka4KKJ8cVuYU46/Z5L3xdNjNdab1jNfJb3Wzafcc1l/3M1ikCfBVZ2Ta8ADo9gu5J6bF13CePNxhnzxpsNrv6Bpw9crzkWjFWdeg0xKCCaY0Gz0X+jzUacvnG6dd0lrWvuNY03G2xdd0nleKvqBhm07+4e+61bdby7xzWov2EaY4P3P1ejCPQdwC9Ey1XAtzPzrMstkuZv49opbn3VZUxNjBO0nja59VWX8aHX/wg3XFV9GXNqYpxt11/Oe372hUyMN4fuo5N9UxPjvOfVL+SGq1bRiDMDsbPNbdddzlT7DLO74sJlzTNuNm5cO8W26y8/Y/8XLmtyw1WrTq/f2UdnTBvXTp013onxJhcua54x9mE3FKv2XdVjv3Wrjnf3uDrLe4/BsuYYFy5rnjG2bk9Z2pjzDdFh6jzl8mHgxcBy4GvA24EmQGb+WfuxxT8G1tN6bPGXMnPo4ys+5SJJczfoKZehN0Uzc/OQ5Qm88Rx7kySNiJ8UlaRCGOiSVAgDXZIKYaBLUiEMdEkqhIEuSYUw0CWpEAa6JBXCQJekQhjoklQIA12SCmGgS1IhDHRJKoSBLkmFMNAlqRAGuiQVwkCXpEIY6JJUCANdkgphoEtSIQx0SSqEgS5JhTDQJakQBrokFcJAl6RCGOiSVAgDXZIKYaBLUiEMdEkqRK1Aj4j1EbE/Ig5ExE0Vy1dFxF0RsTciPh8R14y+VUnSIEMDPSIawG3Ay4FLgc0RcWlP2VuBOzNzLbAJ+JNRNypJGqzOGfoVwIHMfDAzHwXuAK7tqUng+9o/XwAcHl2LkqQ66gT6FHCwa3q2Pa/bO4AbImIW2An8atWGImJLRMxExMyRI0fOoV1JUj91Aj0q5mXP9GbgLzNzBXAN8NcRcda2M/P2zJzOzOnJycm5dytJ6qtOoM8CK7umV3D2JZXXAXcCZOZngScDy0fRoCSpnjqBfg+wJiIujoiltG567uip+QrwEwAR8UO0At1rKpL0GBoa6Jl5ArgR2AU8QOtpln0RcUtEbGiXvQl4fUTcC3wY+MXM7L0sI0laQEvqFGXmTlo3O7vnva3r5/uBq0fbmiRpLvykqCQVwkCXpEIY6JJUCANdkgphoEtSIQx0SSqEgS5JhTDQJakQBrokFcJAl6RCGOiSVAgDXZIKYaBLUiEMdEkqhIEuSYUw0CWpEAa6JBXCQJekQhjoklQIA12SCmGgS1IhDHRJKoSBLkmFMNAlqRAGuiQVwkCXpEIY6JJUCANdkgphoEtSIQx0SSpErUCPiPURsT8iDkTETX1qfjYi7o+IfRHxt6NtU5I0zJJhBRHRAG4DXgbMAvdExI7MvL+rZg1wM3B1Zn4rIp65UA1LkqrVOUO/AjiQmQ9m5qPAHcC1PTWvB27LzG8BZObDo21TkjRMnUCfAg52Tc+253X7QeAHI+IzEbE7ItZXbSgitkTETETMHDly5Nw6liRVqhPoUTEve6aXAGuAFwObgfdHxMRZK2XenpnTmTk9OTk5114lSQPUCfRZYGXX9ArgcEXNxzPzeGZ+CdhPK+AlSY+ROoF+D7AmIi6OiKXAJmBHT8124CUAEbGc1iWYB0fZqCRpsKGBnpkngBuBXcADwJ2ZuS8ibomIDe2yXcA3IuJ+4C5ga2Z+Y6GaliSdLTJ7L4c/Nqanp3NmZmZR9i1J56uI2JOZ01XL/KSoJBXCQJekQhjoklQIA12SCmGgS1IhDHRJKoSBLkmFMNAlqRAGuiQVwkCXpEIY6JJUCANdkgphoEtSIQx0SSqEgS5JhTDQJakQBrokFcJAl6RCGOiSVAgDXZIKYaBLUiEMdEkqhIEuSYUw0CWpEAa6JBXCQJekQhjoklQIA12SCmGgS1IhagV6RKyPiP0RcSAibhpQd11EZERMj65FSVIdQwM9IhrAbcDLgUuBzRFxaUXd04BfA+4edZOSpOHqnKFfARzIzAcz81HgDuDairrfBX4f+M4I+5Mk1VQn0KeAg13Ts+15p0XEWmBlZn5i0IYiYktEzETEzJEjR+bcrCSpvzqBHhXz8vTCiDHgvcCbhm0oM2/PzOnMnJ6cnKzfpSRpqDqBPgus7JpeARzumn4a8HzgXyLiv4CrgB3eGJWkx1adQL8HWBMRF0fEUmATsKOzMDO/nZnLM3N1Zq4GdgMbMnNmQTqWJFUaGuiZeQK4EdgFPADcmZn7IuKWiNiw0A1KkupZUqcoM3cCO3vmva1P7Yvn35Ykaa78pKgkFcJAl6RCGOiSVAgDXZIKYaBLUiEMdEkqhIEuSYUw0CWpEAa6JBXCQJekQhjoklQIA12SCmGgS1IhDHRJKoSBLkmFMNAlqRAGuiQVwkCXpEIY6JJUCANdkgphoEtSIQx0SSqEgS5JhTDQJakQBrokFcJAl6RCGOiSVAgDXZIKYaBLUiEMdEkqRK1Aj4j1EbE/Ig5ExE0Vy38zIu6PiM9HxKci4jmjb1WSNMjQQI+IBnAb8HLgUmBzRFzaU7YXmM7MFwB/D/z+qBuVJA1W5wz9CuBAZj6YmY8CdwDXdhdk5l2Z+Uh7cjewYrRtSpKGqRPoU8DBrunZ9rx+Xgf8Y9WCiNgSETMRMXPkyJH6XUqShqoT6FExLysLI24ApoFtVcsz8/bMnM7M6cnJyfpdSpKGWlKjZhZY2TW9AjjcWxQRLwXeAvxYZn53NO1Jkuqqc4Z+D7AmIi6OiKXAJmBHd0FErAX+HNiQmQ+Pvk1J0jBDAz0zTwA3AruAB4A7M3NfRNwSERvaZduApwIfiYh/j4gdfTYnSVogdS65kJk7gZ09897W9fNLR9yXJGmO/KSoJBXCQJekQhjoklQIA12SCmGgS1IhDHRJKoSBLkmFMNAlqRAGuiQVwkCXpEIY6JJUCANdkgphoEtSIQx0SSqEgS5JhTDQJakQBrokFcJAl6RCGOiSVAgDXZIKYaBLUiEMdEkqhIEuSYUw0CWpEAa6JBXCQJekQhjoklQIA12SCmGgS1IhltQpioj1wPuABvD+zHx3z/InAR8Efhj4BvDqzPyv0bYK2/ceYtuu/Rw6euyM+WMBr7lyFe/ceNkZdYePHuOiiXFe8txJ7vrCkdPTW9ddwsa1U2dtsxHByUwmxptEwNFHjp9RX1U71Wd5ANnT/4XLmrz9p5931r6r+hp2DA4fPcYFXX1e0NPzoDH3euv2+/jw3Qc5mb0dw1OWNnjXK1vHtWrsL3nuJB/dM8ux46fOWOeVL5o6a/+dbXR6/7/vHqdrtdPHrPf4947lJc+d5BP3PsTRY8crj2u/4/aOHfvOWqe3p0dPnOSRrqb6vbe6j8N4c4zvnjjFqYRGBJuvXMn0c55e+V7tbPNUnj3OOq+/NEhkxS/xGQURDeCLwMuAWeAeYHNm3t9V8yvACzLzDRGxCXhlZr560Hanp6dzZmamdqPb9x7i5o/dx7HjJ/vW3HDVKqaf8/ShdePNBre+qvULOqy2U/8zPzzFR/ccqqwdtrxbsxFsu+7yyn13+ur3S13nGAwaQ9W237r9Pv5m91cGrhvAkkZw/OTg98ogzbGAYF7bGLj99nGtOnbb9x5i60fu5fipM/c9FtAYqzeuuu+tjsZYcPLU3MY67PWXACJiT2ZOVy6rEeg/ArwjM9e1p28GyMxbu2p2tWs+GxFLgK8Ckzlg43MN9Kvf/enKs51ujQiedcGTh9YBTE2MA9Sq7Wy76gy27vK6+56aGOczN/145Xp1jsGw/fZu+wdu3lm778e7fsduvscN5vbemo9Br78EgwO9ziWXKeBg1/QscGW/msw8ERHfBp4BfL2nkS3AFoBVq1bVar7jcI1fpJOZterqbq932/NZXnff57rsXPdbSphD/+Mz3+MGc3tvzcdjsQ+Vq85N0aiY15sCdWrIzNszczozpycnJ+v0d9pF7bPaQRoRteo626tb29n2fJbX3fegnubSb93159L34925HNO65vLemo/HYh8qV51AnwVWdk2vAA73q2lfcrkA+OYoGuzYuu4SxpuNgTWbr1xZq2682WDruktq1XbqN1+5sm/tsOXdmo3ou+9OX/3U7bdfj1Xb3nzlyorqMwWtvuejORbz3sbA7bePa5Wt6y5pXcPvMRb1x1X3vdXRqNjfMMNef2mYOpdc7gHWRMTFwCFgE/CanpodwGuBzwLXAZ8edP38XHRuFNV5yqVTV+cpl+5tDnvKpfvJhaqnXLqX13nKpbfPYU85dB+DUT3l0jlmJT/l0pk/iqdcqo6DT7no8WLoTVGAiLgG+ENajy1+IDPfFRG3ADOZuSMingz8NbCW1pn5psx8cNA253pTVJI0/5uiZOZOYGfPvLd1/fwd4Pr5NClJmh8/KSpJhTDQJakQBrokFcJAl6RCGOiSVAgDXZIKYaBLUiFqfbBoQXYccQT48qLsfH6W0/OPjj1BOO4nFsf9+PWczKz8x7AWLdDPVxEx0+9TWiVz3E8sjvv85CUXSSqEgS5JhTDQ5+72xW5gkTjuJxbHfR7yGrokFcIzdEkqhIEuSYUw0OcgIn4rIjIilrenIyL+KCIORMTnI+JFi93jKEXEtoj4Qnts/xARE13Lbm6Pe39ErFvMPhdCRKxvj+1ARNy02P0spIhYGRF3RcQDEbEvIn69Pf/pEfHJiPiP9v8vXOxeF0JENCJib0R8oj19cUTc3R7330XE0sXusS4DvaaIWAm8DPhK1+yXA2va/20B/nQRWltInwSen5kvAL4I3AwQEZfS+irC5wHrgT+JiHP7stPHofZYbqP1+l4KbG6PuVQngDdl5g8BVwFvbI/3JuBTmbkG+FR7ukS/DjzQNf17wHvb4/4W8LpF6eocGOj1vRf4bc78qtBrgQ9my25gIiKevSjdLYDM/KfMPNGe3E3rC8KhNe47MvO7mfkl4ABwxWL0uECuAA5k5oOZ+ShwB60xFykzH8rMz7V//h9a4TZFa8x/1S77K2Dj4nS4cCJiBfBTwPvb0wH8OPD37ZLzatwGeg0RsQE4lJn39iyaAg52Tc+255Xol4F/bP9c+rhLH19fEbGa1ncD3w18f2Y+BK3QB565eJ0tmD+kdaLW+WbwZwBHu05kzqvXvtZ3ij4RRMQ/A8+qWPQW4M3AT1atVjHvvHoOdNC4M/Pj7Zq30Ppr+Yc6q1XUn1fjHqL08VWKiKcCHwV+IzP/u3WyWq6IeAXwcGbuiYgXd2ZXlJ43r72B3paZL62aHxGXARcD97bf4CuAz0XEFbT+9F7ZVb4COLzArY5Uv3F3RMRrgVcAP5Hf+9DCeT/uIUof31kiokkrzD+UmR9rz/5aRDw7Mx9qX0p8ePE6XBBXAxsi4hrgycD30Tpjn4iIJe2z9PPqtfeSyxCZeV9mPjMzV2fmalq/7C/KzK8CO4BfaD/tchXw7c5fUUsQEeuB3wE2ZOYjXYt2AJsi4kkRcTGtm8L/thg9LpB7gDXtpx2W0roBvGORe1ow7evGfwE8kJnv6Vq0A3ht++fXAh9/rHtbSJl5c2auaP9ebwI+nZk/B9wFXNcuO6/G7Rn6/OwErqF1U/AR4JcWt52R+2PgScAn23872Z2Zb8jMfRFxJ3A/rUsxb8zMk4vY50hl5omIuBHYBTSAD2TmvkVuayFdDfw8cF9E/Ht73puBdwN3RsTraD3ddf0i9fdY+x3gjoh4J7CX1h925wU/+i9JhfCSiyQVwkCXpEIY6JJUCANdkgphoEtSIQx0SSqEgS5Jhfh/Kvo+GLSmxRcAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.scatter(x, y_rounded);"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "From now on, we will only assume that we have our observations $(x_{i},y_{i})$ and we will pretend that we do not know the function $y = f(x)$ that generated them.\n",
    "\n",
    "Our goal is to find a function $\\hat{f}$ such as $\\hat{f} \\sim f$, which means finding a function that can approximate the underlying (unknown) function that generates our observations."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The first thing to do is to use the $(x,y)$ observations and wrap them with a [Fitness function](https://github.com/auxein/auxein/blob/master/auxein/fitness/core.py#L15) $\\phi$ that Auxein can explore.\n",
    "\n",
    "Auxein comes with some pre-defined fitness functions. In this case, given that our problem can be modeled as a [logistic regression](https://en.wikipedia.org/wiki/Logistic_regression), we will use the [MaximumLikelihood](https://github.com/auxein/auxein/blob/master/auxein/fitness/observation_based.py#L73)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "from auxein.fitness.observation_based import MaximumLikelihood\n",
    "fitness_function = MaximumLikelihood(x.reshape(100, 1), y_rounded)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Then, the second step is to create an initial `population` of individuals. Each `individual` maps to candidate solution, which in this case would be a vector $(\\alpha, \\beta_{0})$ that fully specify a logistic regression model.\n",
    "\n",
    "Auxein provides some utility functions to create initial populations, like the `build_fixed_dimension_population` used below."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "from auxein.population.dna_builders import UniformRandomDnaBuilder\n",
    "from auxein.population import build_fixed_dimension_population\n",
    "population = build_fixed_dimension_population(2, 100, fitness_function, UniformRandomDnaBuilder((-0.01, 0.01)))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Once we have a `fitness_function` and an initial `population`, we need to set up a [Playground](https://github.com/auxein/auxein/blob/master/auxein/playgrounds/static.py#L27).\n",
    "A playground is basically the object that represents our experiment."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [],
   "source": [
    "from auxein.playgrounds import Static\n",
    "from auxein.mutations import SelfAdaptiveSingleStep\n",
    "from auxein.recombinations import SimpleArithmetic\n",
    "from auxein.parents.distributions import SigmaScaling\n",
    "from auxein.parents.selections import StochasticUniversalSampling\n",
    "from auxein.replacements import ReplaceWorst"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In order to instantiate a `playground` the following must be specified:\n",
    "* `mutation` strategy, which describes how `individual` dna will mutate. In this case we will use the [SelfAdaptiveSingleStep](https://github.com/auxein/auxein/blob/master/auxein/mutations/core.py#L62).\n",
    "* parents `distribution`, which gives a probability distribution for parents `selection`. We here use [SigmaScaling](https://github.com/auxein/auxein/blob/master/auxein/parents/distributions/core.py#L43) for distribution and [StochasticUniversalSampling](https://github.com/auxein/auxein/blob/master/auxein/parents/selections/core.py#L27) for selection.\n",
    "* `recombination` defines how fresh dna are created when `individual`s breed. Here we use the basic [SimpleArithmetic](https://github.com/auxein/auxein/blob/master/auxein/recombinations/core.py#L23).\n",
    "* for `replacement` we will use the basic [ReplaceWorst](https://github.com/auxein/auxein/blob/master/auxein/replacements/core.py#L34) which basically only replaces the 2-worst performing individuals."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [],
   "source": [
    "offspring_size = 4\n",
    "playground = Static(\n",
    "    population = population,\n",
    "    fitness = fitness_function,\n",
    "    mutation = SelfAdaptiveSingleStep(0.05),\n",
    "    distribution = SigmaScaling(),\n",
    "    selection = StochasticUniversalSampling(offspring_size = offspring_size),\n",
    "    recombination = SimpleArithmetic(alpha = 0.5),\n",
    "    replacement = ReplaceWorst(offspring_size = offspring_size)\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Invoking `playground.train(max_generations=200)` will trigger the evolution process up to a maximum of $2o0$ generations."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "stats = playground.train(200)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Once the training phase has ended, the `playground` returns a dictionary with some basic statistics on the population."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'generation_count': 200,\n",
       " 'size': 100,\n",
       " 'mean_age': 11.382943091392518,\n",
       " 'std_age': 2.7466001141174465,\n",
       " 'max_age': 19.073707103729248,\n",
       " 'min_age': 7.5454676151275635,\n",
       " 'mean_fitness': 95.94936639758845,\n",
       " 'min_fitness': 89.88305025536772,\n",
       " 'max_fitness': 98.5427431177904,\n",
       " 'std_fitness': 1.5003278526771748}"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "population.get_stats()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To get the most performant `individual` we can invoke `playground.get_most_performant()` and grab the dna of the individual."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [],
   "source": [
    "[alpha_star, *coeff] = playground.get_most_performant().genotype.dna"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[-6.100514545336547, 0.689517381351941]"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "[alpha_star, *coeff]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Once we have $\\alpha$ and $\\beta_{0}$, it might be useful to plot $\\hat{f(x)} = \\frac{1}{1 - e^-{(\\alpha + \\beta_{0}x)}}$ against our observations $(x_{i},y_{i})$, to visually inspect the quality of our regression:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXQAAAD4CAYAAAD8Zh1EAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjMsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+AADFEAAAbf0lEQVR4nO3dfXQc9X3v8fd3VytZxmBhLB4sS5bhGINDCAZdHuJzeQhJMIQC6SUX3JuUUG7cnJaQpo4bQ3JISpvwFG7CPaVNnTQnSZtCSSGOLzXxJYGcEG5MsbEJx4CNszKWZGMM2DzEsq2H7/1jdp31eqQdSbOa3dXndY7PamZ+M/P9raSPf5r9za65OyIiUv1SSRcgIiLxUKCLiNQIBbqISI1QoIuI1AgFuohIjahL6sTTp0/39vb2pE4vIlKV1q1b97q7N4dtSyzQ29vbWbt2bVKnFxGpSmb2ylDbdMlFRKRGKNBFRGqEAl1EpEYo0EVEaoQCXUSkRijQRURqhAJdRKRGKNBFRGpEyRuLzOy7wOXAa+5+Wsh2A+4FLgP2Ap9092fjLlREAivW93D36k1s39PLjKZGll4yl6vmt/ClFc/zL2u2Hda+JdcG4CsrN7Knt2/Y46cMBv33+6195U3uf7qLgYLPTig85t2rN9GzpxcD8i2Onpzhy3/wHq6a33JI3YXnP3pyho+cfgJPvLSLnj29pM0YcD947Py+hf2d2pjBDPbs7Tuk71Ges+K+h9U41L5hz3fx9uLnYHImRUMmze69fQf7VuiI+jRf/eh7I9UflZX6gAszOx94F/jBEIF+GfAZgkA/B7jX3c8pdeKOjg7XnaIiI7NifQ83P/w8vX0DB9c1ZtKc2TaVp3775pD7ZVJBoAyO8PNsUsDgMMfEoG8g/KCZtHH31e/jqvktrFjfw9IfPUdfxAIaM2lu/8P3AhzW37B2w4XicOcurHGofcOe7/w5w7aPRDpl3POxoc8fxszWuXtH6LYon1hkZu3AI0ME+j8Cv3D3+3PLm4AL3X3HcMdUoIuM3II7HqdnT++o9z9y/+847dXfcsI7uzj23d0c3fs2R+17lykHepnUv5+G/j7qB/qoGxygbnCAlA+SHhwk5fl/juUyw4Dfj0cDVpQndSmjbdpktr25l/4R/m9SlwrOUGq//DmGUurcw+0/1L5j6RfAXRdcx0/nLgCCv3aeWvaByPsOF+hxvJdLC9BVsNydW3dYoJvZYmAxQFtbWwynFplYto8wzM0Hec/OLBdk13F+57Oc1fMidf77Mfe+unrebjiCd+sb6c1M4kA6w4G6DPvq6hlIpRmwFAOpNINmDFgKN8MxsN/Huee+znMOXW47YwbrN2wfXYcjajtjxpDbopx7qP2H23cs/drdeNTBr0f6PR1OHIFuIetC/8ty9+XAcghG6DGcW2RCmdHUGGmE3rZ7Bzf9vwe4MLuW6XvfAuD5407iW+dezdOtp9E19Th2TjmG3vpJZa23pamRK5d9gLtG8ZdFS1MjQMn98ucYSqlzD7f/UPuOpV/FZuT6GYc4Zrl0A60FyzOB8v53LDJBLb1kLo2Z9CHrGjNpFpw0DYD04ACf/dW/8tg//RmXbnqKX84+k7+4fAnn3PRDrrz+Xr5+/h/z5Owz2TqtJVKYDxcQmZSRSYeN53Lb03bwhdOll8wNrrlH1JhJs/SSuaH9DWs3nOHOXVjjUPuGPd+F/RquvlLSqeHPP1JxjNBXAjea2QMEL4q+Ver6uYiMTv7Fs7BZF1/+0bNc/Ff/k/O3rucnp17AVy/6E1478hhamhq5OeFZLvnH0cxyKezvaGa5hJ07rMaRPt/F26tllsv9wIXAdGAn8GUgA+Du38pNW/w7YCHBtMXr3b3kq516UVQkZnfeCcuWwbe+BX/6p0lXI2Uy5lku5aBAF4nRtm1w6qnw4Q/Dj3+cdDVSRsMFuu4UFakFn/scuMM3v5l0JZKgxD6CTkRi8tOfwsMPw9e+BrNmJV2NJEgjdJFqtm8f3HgjzJ0LS5YkXY0kTCN0kWp2113w29/Cz34G9fVJVyMJ0whdpFpls3D77XDNNXDxxUlXIxVAgS5Sjdzhppugrg7uuSfpaqRC6JKLSDX65S/hP/4jCPOW+G5MkeqmEbpINVq3Lnj85CcTLUMqiwJdpBpls9DUBNOmJV2JVBAFukg16uyE2bOTrkIqjAJdpBpls3DiiUlXIRVGgS5SbQYHgxG6Al2KKNBFqs2OHbB/vwJdDqNAF6k2nZ3Bo66hSxEFuki1yWaDR43QpYgCXaTaZLPBhzTrnRWliAJdpNpks9DaqjfjksMo0EWqTTar6+cSSoEuUm00ZVGGoEAXqSa9vbB9uwJdQinQRarJ1q3BowJdQijQRapJfsqirqFLCAW6SDXJ31SkEbqEUKCLVJNsFiZPhmOPTboSqUAKdJFqkn+XRbOkK5EKpEAXqSZ621wZhgJdpFq464MtZFgKdJFq8frr8O67GqHLkBToItVC77IoJUQKdDNbaGabzGyLmS0L2d5mZk+Y2Xoz+42ZXRZ/qSITnAJdSigZ6GaWBu4DLgXmAYvMbF5Rsy8BD7r7fOBa4O/jLlRkwsvPQW9vT7QMqVxRRuhnA1vcPevuB4AHgCuL2jhwVO7rqcD2+EoUESAYoR9/fDAPXSRElEBvAboKlrtz6wp9Bfi4mXUDq4DPhB3IzBab2VozW7tr165RlCsygWnKopQQJdDD7mDwouVFwPfcfSZwGfDPZnbYsd19ubt3uHtHc3PzyKsVmcgU6FJClEDvBloLlmdy+CWVG4AHAdz918AkYHocBYoI0NcHXV2agy7DihLozwBzzGy2mdUTvOi5sqjNNuBiADM7lSDQdU1FJC7btsHgoEboMqySge7u/cCNwGrgRYLZLBvN7DYzuyLXbAnwKTN7Drgf+KS7F1+WEZHR0pRFiaAuSiN3X0XwYmfhulsLvn4BWBBvaSJykAJdItCdoiLVIJuF+nqYMSPpSqSCKdBFqkFnZ3BDUUq/sjI0/XSIVANNWZQIFOgi1UCBLhEo0EUq3Z49sHu3Al1KUqCLVLr8m3LppiIpQYEuUuk0ZVEiUqCLVLp8oGuELiUo0EUqXTYLxxwDU6cmXYlUOAW6SKXLZjU6l0gU6CKVrrNT188lEgW6SCUbGICtWxXoEokCXaSS9fQE74WuQJcIFOgilUwzXGQEFOgilSx/U5FG6BKBAl2kkmWzkE5Da2vptjLhKdBFKlk2C21tkMkkXYlUAQW6SCXTHHQZAQW6SCXTHHQZAQW6SKX63e9g504FukSmQBepVJrhIiOkQBepVHrbXBkhBbpIpdJNRTJCCnSRStXZCUceGbx1rkgECnSRSpX/YGizpCuRKqFAF6lU+UAXiUiBLlKJ3INLLrp+LiOgQBepRDt3Qm+vRugyIgp0kUqkKYsyCgp0kUqkQJdRiBToZrbQzDaZ2RYzWzZEm/9uZi+Y2UYz+9d4yxSZYPKBPmtWsnVIVakr1cDM0sB9wIeAbuAZM1vp7i8UtJkD3AwscPfdZnZsuQoWmRA6O6GlBSZNSroSqSJRRuhnA1vcPevuB4AHgCuL2nwKuM/ddwO4+2vxlikywWjKooxClEBvAboKlrtz6wqdDJxsZk+Z2RozWxh2IDNbbGZrzWztrl27RlexyESgQJdRiBLoYbepedFyHTAHuBBYBHzHzJoO28l9ubt3uHtHc3PzSGsVmRj274eeHs1BlxGLEujdQOEHGs4Etoe0+Ym797l7J7CJIOBFZKReeSW4sUgjdBmhKIH+DDDHzGabWT1wLbCyqM0K4CIAM5tOcAkmG2ehIhOGpizKKJUMdHfvB24EVgMvAg+6+0Yzu83Mrsg1Ww28YWYvAE8AS939jXIVLVLTFOgySiWnLQK4+ypgVdG6Wwu+duAvc/9EZCyy2WC64vHHJ12JVBndKSpSabLZ4AVRvW2ujJACXaTSdHbqcouMigJdpJK4aw66jJoCXaSSvPkmvP22Al1GRYEuUkn0wdAyBgp0kUrS2Rk8aoQuo6BAF6kkGqHLGCjQRSpJNgvHHgtTpiRdiVQhBbpIJcnPQRcZBQW6SCXRHHQZAwW6SKXo7w/eaVGBLqOkQBepFF1dMDCgQJdRU6CLVArNcJExUqCLVAq9ba6MkQJdpFJ0dkJdHcycmXQlUqUU6CKVIpuF9nZIp5OuRKqUAl2kUuhdFmWMFOgilUI3FckYKdBFKsHbb8Mbb2iELmOiQBepBHqXRYmBAl2kEmjKosRAgS5SCXRTkcRAgS5SCTo7oakJjj466UqkiinQRSqBpixKDBToIpVAgS4xUKCLJG1wMLjkouvnMkYKdJGk7dgBBw5ohC5jpkAXSZqmLEpMFOgiSVOgS0wiBbqZLTSzTWa2xcyWDdPuajNzM+uIr0SRGpfNQioFbW1JVyJVrmSgm1kauA+4FJgHLDKzeSHtjgRuAp6Ou0iRmpbNBu+BXl+fdCVS5aKM0M8Gtrh71t0PAA8AV4a0+xvgLmBfjPWJ1L7OTl1ukVhECfQWoKtguTu37iAzmw+0uvsjwx3IzBab2VozW7tr164RFytSkzQHXWISJdAtZJ0f3GiWAr4BLCl1IHdf7u4d7t7R3NwcvUqRWrV3bzBtUYEuMYgS6N1Aa8HyTGB7wfKRwGnAL8xsK3AusFIvjIpEsHVr8KibiiQGUQL9GWCOmc02s3rgWmBlfqO7v+Xu09293d3bgTXAFe6+tiwVi9QSvQ+6xKhkoLt7P3AjsBp4EXjQ3Tea2W1mdkW5CxSpaZqDLjGqi9LI3VcBq4rW3TpE2wvHXpbIBJHNwhFHgF5TkhjoTlGRJG3dCu3tYGFzD0RGRoEukqSuLmhtLd1OJAIFukiSursV6BIbBbpIUvbvh507g9v+RWKgQBdJSk9P8KgRusREgS6SlO7u4FEjdImJAl0kKV25t0jSCF1iokAXSYpG6BIzBbpIUrq6oKkJpkxJuhKpEQp0kaRoyqLETIEukpSuLl1ukVgp0EWSortEJWYKdJEk7NsHu3ZphC6xUqCLJEE3FUkZKNBFkpCfsqhAlxgp0EWSkL+pSJdcJEYKdJEkKNClDBToIkno7oajjw4+rUgkJgp0kSRoyqKUgQJdJAnd3brcIrFToIskQSN0KQMFush46+2F119XoEvsFOgi4y1/U5EuuUjMFOgi400fbCFlokAXGW/btgWPGqFLzBToIuNtyxZIp6G9PelKpMYo0EXG2+bNQZjX1yddidQYBbrIeHv5ZZgzJ+kqpAYp0EXGk7sCXcpGgS4ynl59Fd59F04+OelKpAZFCnQzW2hmm8xsi5ktC9n+l2b2gpn9xsx+bmaz4i9VpAa8/HLwqBG6lEHJQDezNHAfcCkwD1hkZvOKmq0HOtz9dODfgbviLlSkJijQpYyijNDPBra4e9bdDwAPAFcWNnD3J9x9b25xDaAJtiJhXn4ZMhmYpT9iJX5RAr0F6CpY7s6tG8oNwKNhG8xssZmtNbO1u3btil6lSK3YvBlOOimYhy4SsyiBbiHrPLSh2ceBDuDusO3uvtzdO9y9o7m5OXqVIrVCM1ykjKIEejdQ+KYTM4HtxY3M7IPAF4Er3H1/POWJ1JDBweAuUQW6lEmUQH8GmGNms82sHrgWWFnYwMzmA/9IEOavxV+mSA3o7oZ9+zRlUcqmZKC7ez9wI7AaeBF40N03mtltZnZFrtndwBTgR2a2wcxWDnE4kYlLM1ykzOqiNHL3VcCqonW3Fnz9wZjrEqk9CnQpM90pKjJeXnoJGhuhZbhJYiKjp0AXGS8bNsDpp0NKv3ZSHvrJEhkP7kGgz5+fdCVSwxToIuPhlVfgrbfgjDOSrkRqmAJdZDysXx88aoQuZaRAFxkPGzYE185POy3pSqSGKdBFxsOGDTB3LkyenHQlUsMU6CLjYf16XW6RslOgi5Tbjh3Q1QVnnZV0JVLjFOgi5bZmTfB43nnJ1iE1T4EuUm5PPgkNDbrkImWnQBcpt0cfhQsugEmTkq5EapwCXaSctm4N3sPl0kuTrkQmAAW6SDk9mvs0xssuS7YOmRAU6CLltGoVnHii3jJXxoUCXaRc9u2Dxx8PLrdY2EfzisRLgS5SLk8+CXv36vq5jBsFuki5rFoVTFe86KKkK5EJQoEuUg59fbBiRTBdUe/fIuNEgS5SDt/+djBl8TOfSboSmUAU6CJxe+cd+Ou/hvPPh498JOlqZAKpS7oAkZpzzz3w2muwcqVmt8i40ghdJE6vvgpf/zpcfTWcc07S1cgEo0AXidNtt8H+/fC1ryVdiUxACnSRuGzeDMuXw+LFujNUEqFAF4nLLbdAYyPcemvSlcgEpUAXicMvfgEPPQSf/zwcd1zS1cgEpUAXGavvfS+4vf/EE2HJkqSrkQlMgS4yWvv3w6c/DddfD+9/f/BRc1OmJF2VTGCR5qGb2ULgXiANfMfd7yja3gD8ADgLeAO4xt23xlsqrFjfw92rN9Gzp/eQ9SmDPzqnjb+96r2HtNu+p5cZTY1cdEozT7y06+Dy0kvmctX8lsOOmTZjwJ2mxgxmsGdv3yHtw9q2DLHdAC+q/+jJGb78B+857NxhdZV6Drbv6WVqQZ1Ti2oers/FvrTiee5/uosBL64YjqhP89WPBs9rWN8vOqWZh9Z109s3eMg+Hz2z5bDz54+Rr/13+/so2O3gc1b8/Bf35aJTmnnkuR3s6e0LfV6Het6+snLjYfsU13Sgf4C9BUWF/Wzdt3I95/3qEa5f93+YvXs73znvau7s+AR99/wnaTMWndNKx6xpoT+r+WMO+uH9jPL9FxmOecgv8SENzNLAZuBDQDfwDLDI3V8oaPNnwOnu/mkzuxb4qLtfM9xxOzo6fO3atZELXbG+h5sffp7evoEh23z83DY6Zk0r2a4xk+b2Pwx+QUu1zbf/b2e18NC6ntC2pbYXyqSNu69+X+i583UN9Usd5TkYrg9hx/7Siuf5lzXbht3XgLq00Tcw/M/KcDIpA2NMxxj2+LnnNey5W7G+h6U/eo6+wUPPnTJIp4buV2pwgLY9r3Ly69v4WONbnLK7m3ef/Q0nvt5Fw0AfG044mXsXLOKJk/7LYfumU8bA4Mj6Wur7LwJgZuvcvSN0W4RAPw/4irtfklu+GcDdby9oszrX5tdmVge8CjT7MAcfaaAvuONxevb08r7tmzi/89nQNikzpjTU8fa+vpLHO2pSBiBS2/yxB4d5rkptj3ruoyZl+NT5J4bu9+1fZiPXO9R5i4/9jcc2R6670uTvwbRc/UdNquOGBe3Bynyf3PneU1t5Z18fhmPupNxJDw5QNzhA2gfIDAxQP9DHpP4DNPTvZ3LfPqbtfZvZu7czqf/AwfO92nQsL05rZfP0Wfz05PezvuWU2PvU0tTIU8s+EPtxpXYMF+hRLrm0AF0Fy91A8S1wB9u4e7+ZvQUcA7xeVMhiYDFAW1tbpOLztuf+dJ2/fRNLfvXDEe1bdX4WvvpTZTj25+I4ZiX5ecHXudvuP5GLfgcGLYUb9KfqGEil6U+lGUilOJDKsC/TQG+mgd66BnqOauZX7WeweXobL0+fxcvHtLK3YfJhl9Hitj3kEo1IVFECPezNKIp/rqO0wd2XA8shGKFHOPdBM5oa6dnTy/fPupwfnBn+hkdpM46fOin0umWxlqZGgEht88cOu8YcdXvUc7c0NfLkF8JHaP/1zscj1zvUeYuPPeeWVZHrrkRu+bC2YHR788WHtTn/jrE9bxB8f2dE/Nkaixm5nw2R0Ygyy6UbaC1YnglsH6pN7pLLVODNOArMW3rJXBozadxSDKbSof+uOa+dJZfOo6Ghfsg2g6k0DQ31LLl0XqS2+fbXnNc+ZNtS2wv/pTN1Q547XxfpdOi/qPUO1+fiY15zXnvJfT1X92jOe7DfdXVjPkZobZbCLUWmLsXSheGXQJZeMje4hl8kZcG19ygWndN68GcwinTI+UppzKQPvngsMhpRRujPAHPMbDbQA1wL/FFRm5XAdcCvgauBx4e7fj4a+ReKosxyybeLMsul8JilZrkUzlwIm+VSuD3KLJfiOkvNcih8DuKa5ZJ/zmp5lkt+fRyzXMKeh8ZMiv39gww6muUiiSr5oiiAmV0GfJNg2uJ33f2rZnYbsNbdV5rZJOCfgfkEI/Nr3T073DFH+qKoiIiM/UVR3H0VsKpo3a0FX+8DPjaWIkVEZGx0p6iISI1QoIuI1AgFuohIjVCgi4jUCAW6iEiNUKCLiNQIBbqISI2IdGNRWU5stgt4JZGTj810it50bIJQvycW9btyzXL35rANiQV6tTKztUPdpVXL1O+JRf2uTrrkIiJSIxToIiI1QoE+csuTLiAh6vfEon5XIV1DFxGpERqhi4jUCAW6iEiNUKCPgJl93szczKbnls3M/reZbTGz35jZmUnXGCczu9vMXsr17cdm1lSw7eZcvzeZ2SVJ1lkOZrYw17ctZrYs6XrKycxazewJM3vRzDaa2Wdz66eZ2WNm9nLu8eikay0HM0ub2XozeyS3PNvMns71+9/MrD7pGqNSoEdkZq3Ah4BtBasvBebk/i0G/iGB0srpMeA0dz8d2AzcDGBm8wg+ivA9wELg780s2odtVoFcX+4j+P7OAxbl+lyr+oEl7n4qcC7w57n+LgN+7u5zgJ/nlmvRZ4EXC5bvBL6R6/du4IZEqhoFBXp03wD+ikM/KvRK4AceWAM0mdkJiVRXBu7+f929P7e4huADwiHo9wPuvt/dO4EtwNlJ1FgmZwNb3D3r7geABwj6XJPcfYe7P5v7+h2CcGsh6PP3c82+D1yVTIXlY2YzgY8A38ktG/AB4N9zTaqq3wr0CMzsCqDH3Z8r2tQCdBUsd+fW1aI/AR7NfV3r/a71/g3JzNoJPhv4aeA4d98BQegDxyZXWdl8k2Cglv9k8GOAPQUDmar63kf6TNGJwMx+BhwfsumLwC3Ah8N2C1lXVfNAh+u3u/8k1+aLBH+W/zC/W0j7qup3CbXev1BmNgV4CPgLd387GKzWLjO7HHjN3deZ2YX51SFNq+Z7r0DPcfcPhq03s/cCs4Hncj/gM4Fnzexsgv+9WwuazwS2l7nUWA3V7zwzuw64HLjYf3/TQtX3u4Ra799hzCxDEOY/dPeHc6t3mtkJ7r4jdynxteQqLIsFwBVmdhkwCTiKYMTeZGZ1uVF6VX3vdcmlBHd/3t2Pdfd2d28n+GU/091fBVYCf5yb7XIu8Fb+T9RaYGYLgS8AV7j73oJNK4FrzazBzGYTvCj8n0nUWCbPAHNysx3qCV4AXplwTWWTu278T8CL7v6/CjatBK7LfX0d8JPxrq2c3P1md5+Z+72+Fnjc3f8H8ARwda5ZVfVbI/SxWQVcRvCi4F7g+mTLid3fAQ3AY7m/Tta4+6fdfaOZPQi8QHAp5s/dfSDBOmPl7v1mdiOwGkgD33X3jQmXVU4LgE8Az5vZhty6W4A7gAfN7AaC2V0fS6i+8fYF4AEz+1tgPcF/dlVBt/6LiNQIXXIREakRCnQRkRqhQBcRqREKdBGRGqFAFxGpEQp0EZEaoUAXEakR/x/1dlykQFUaagAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "y_pred = 1 / (1 + np.exp(-(alpha_star + coeff*x)))\n",
    "plt.scatter(x, y_rounded);\n",
    "plt.plot(x, y_pred, color='red');"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python [conda env:Python3]",
   "language": "python",
   "name": "conda-env-Python3-py"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.10"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}