{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "## Programming Exercise 2: Logistic Regression\n", "#### Author - Rishabh Jain" ] }, { "cell_type": "code", "execution_count": 1, "metadata": {}, "outputs": [], "source": [ "import matplotlib.pyplot as plt\n", "import seaborn as sns\n", "import pandas as pd\n", "import numpy as np\n", "%matplotlib inline" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### 1    Logistic Regression\n", "\n", "##### Problem Statement\n", "In this part of the exercise, we will build a logistic regression model to predict whether the student gets admitted into a university. \n", "\n", "Suppose that we are the adminstrator of a department and want to determine each applicant's chance of admission based on their results in two exams. We have hisotrical data from previous applicants that we can use as a training set for logistic regression. For each applicant, we have their scores in two exams and their admission decision.\n", "\n", "Our task is to build a classification model that estimates an applicant's probability of admission based on the score from those two exams." ] }, { "cell_type": "code", "execution_count": 2, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "TRAINING DATASET SHAPE : 100 X 3\n" ] }, { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
exam1exam2decision
360.18259986.3085521
3764.17698980.9080611
7982.22666242.7198790
8568.46852285.5943071
6444.66826266.4500860
\n", "
" ], "text/plain": [ " exam1 exam2 decision\n", "3 60.182599 86.308552 1\n", "37 64.176989 80.908061 1\n", "79 82.226662 42.719879 0\n", "85 68.468522 85.594307 1\n", "64 44.668262 66.450086 0" ] }, "execution_count": 2, "metadata": {}, "output_type": "execute_result" } ], "source": [ "data=pd.read_csv('./ex2data1.csv')\n", "print(f'TRAINING DATASET SHAPE : {data.shape[0]} X {data.shape[1]}')\n", "data.sample(5)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "#### 1.1    Visualizing the data\n", "Before starting to implement any learning algorithms, it is always good to visualize the data if possible." ] }, { "cell_type": "code", "execution_count": 3, "metadata": { "scrolled": true }, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYgAAAEKCAYAAAAIO8L1AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvOIA7rQAAIABJREFUeJzt3XuUFPW16PHvHmB4OcrD0SDjCPGixhhEMj4QrxFR1GhUjk/iER9JMEe9ijErMcm5aIxGPZqrsnKNIXrwcXBEUZBrTI5KICZziMmgqAiyjB6ECQQmIygvHaD3/aOqh2bsnunp6ar6VdX+rDWrp2uqu39d3VO7fq/9E1XFGGOMaa8i6gIYY4xxkwUIY4wxeVmAMMYYk5cFCGOMMXlZgDDGGJOXBQhjjDF5WYAwxhiTlwUIY4wxeVmAMMYYk1fPqAvQHfvuu68OGzYs6mIYY0ysLFmy5B+qWt3ZfrEOEMOGDaOxsTHqYhhjTKyIyAfF7GdNTMYYY/IKLECIyL+LyAYRWZazbZCIvCQi7/q3A/3tIiLTReSvIvKmiIwOqlzGGGOKE2QN4hHg9HbbbgIWqOoIYIF/H+AMYIT/MwX4RYDlMsYYU4TAAoSqvgJ82G7zOcCj/u+PAufmbH9MPX8CBojIkKDKZowxpnNh90Hsr6rrAPzb/fztQ4E1Ofs1+duMMcZExJVRTJJnW96VjERkCl4zFLW1tUGWqaBMRmnZ2krrzl1U9uzB4P6VVFTkewvGGBNfYQeI9SIyRFXX+U1IG/ztTcCBOfvVAGvzPYGqzgBmANTV1YW+HF4mo6xcv5lvPdZI08bt1Azsy68m13Ho/lUWJIwxiRJ2E9N84DL/98uA53K2T/ZHMx0HfJRtinJNy9bWtuAA0LRxO996rJGWra0Rl8wYY8orsBqEiNQDJwH7ikgTcDNwJ/CUiHwDWA1c4O/+AvBV4K/ANuCKoMrVXa07d7UFh6ymjdtp3bkrohIZY0wwAgsQqjqpwJ/G59lXgWuCKks5VfbsQc3AvnsEiZqBfans2SO4F81kYFsz7GyFnpXQrxoqbI5jotlnbhxg37guGty/kl9NrqNmYF+Atj6Iwf0rg3nBTAY2LIeHToH7jvBuNyz3tptkss/cOEK8i/d4qqur0yhyMYU6imnLeu8EsWn17m0DauGbL8Ne+wfzmsYT1VW8feYmYCKyRFXrOtvPlWGusVJRIVRX9Q7nxXa27nmiAO/+TusUL6v2waDvYGh+B56c5B3vAbVwcT3sd3jwQSIOn7k1gaWCfaKu61npnZxyDaj1tpvyyNek89Ga3cEBvNsnJ3knxaCV+plnMl7tY9Ma7zaoJilrAksNCxCu61ftXblmTxjZK9l+naZyN8Xa1vzZYLBlfXRX8aV85mGetPMdr7CCpwmVNTF1wIkZ0xUVXrPGN1+26nxQ8jXpbG32Tszt+wHCqLmV8pkXOmkH0W8RhyYwUxYWIApwasZ0RYV1TgYp26STe9JbWg8XzYLZl+zZBxFWza2rn3mYJ+18x8uaPRPJLkMLsBnTKZKvSWfcD3dfxU9d5t2G0UFdqjD7qqzZMzWsBlGAzZhOkY6adOJSc8uetNuPugripG3NnqlhAaKASGZMm+jEKRjkE/ZJO+7HyxTFQn4Boc+YNqa7siftAQd6t3ZFb7rJahAFVFQIh+5fxdyrx9q6D8aYVLIA0YFQZ0wbUwybwWxCZAHCmLjIToaLIv2HSSX7VhkTFzaD2YTMahDGxEVcZjBbM1hi2KdmTFzEIXGjJfJLFAsQxsRFHGYwWzNYolgTkzFxEYcZzNlmsJo6GDsV+g6E7RutBhFTFiCMiRPXZzD3rIRDz4Rjr4L51+4ebXXRLJu8F0ORfFoicr2ILBORt0Vkqr9tkIi8JCLv+rcDoyibMalTzoWG+lXDabfvDg7g3c6+xJqZYij0ACEiRwDfAo4BjgTOEpERwE3AAlUdASzw7xtjglTuTuWKCqjoGY/RVqZTUdQgvgD8SVW3qepO4PfAROAc4FF/n0eBcyMomzHpkK01fLQaNq+FvfbztpejUzkOo61MUaIIEMuAE0VksIj0A74KHAjsr6rrAPzb/SIom4laWOsqp1lureH+I+HXN8LJ07yOZej+1X4cRluZooTeSa2qK0TkLuAlYAvwBrCz2MeLyBRgCkBtbW0ne5tYsVQS4cg3FHX+tXDaT2H2P3f/aj8Oo61MUSL5xFT1YVUdraonAh8C7wLrRWQIgH+7ocBjZ6hqnarWVVfbFUmgwr6atzH04Sg0I7vvwPJd7cc99bjVZIGIhrmKyH6qukFEaoF/AsYAw4HLgDv92+eiKJvxRXE1H5dUEnFXaE3pAQd5V/1pv9q3mmybqN7tMyKyHPh/wDWquhEvMJwqIu8Cp/r3TVSiuJq3zs1wFOoj2HtoPK/2y81qsm0iqUGo6v/Ms60FGB9BcUw+UVzNh7mucpoV6iMArzklbf0G7ZMLZjJWk/XZTOokCCJ7ZqFmiCCv5q1zMzztZ2SntVkl3/u+aJY3G3zlr3fvl9KabII/+ZQIKntmVEMV4965GVdpbVbJ975nX+LNBg/ru+9wh7jVIOKu0D/2N1/uXs4eu5pPl7QOECj0vit6hvPdd7zmFn0JTPcE+Y+d5Kt5h6/aIpHWAQIdve8wvvuO19wS9B+fUmn9x+4OW9Rmt2yg3NkKk+d7be+QngECUc/6drzmZk1McWcjf7ouqGa57ohimc5CHbRn/sx77TQ0KUbdlBrFYJAusAARd1F/wePItau2qNqhC3XQRhkooxDlGhuOX+BZgEgC1xeRcY1rV21h1mhyayrgZXHNPQ4ONW+kguMXeG6UwpgwRd3u3F5YNZr2fS+PfBXG37I7iys41byRGg4PBrEahEkf167awqrR5KupPHe11+cw64LoA6VxjgUIk04uNcuF1Q5dqKay7yEwdVn0gdI4xwKEMVELq0ZTqKbSq687wdI4xS4VjHFBGO3QrvW9GOdZDcKYtHCt78U4zwKEMWniUt+LcZ5dOhhjjMnLAoQxxpi8LEAYY4zJy/ogjOmOKJLsGRMSCxDGlMrxxV6M6a5IvsUicoOIvC0iy0SkXkT6iMhwEXlVRN4VkdkiYglhjNscX+zFmO4KPUCIyFDgOqBOVY8AegAXA3cB96rqCGAj8I2wy2ZMl7iWNtyYMouqHtwT6CsiPYF+wDrgZGCO//dHgXMjKpsxxbHV/EzChR4gVPVvwD3AarzA8BGwBNikqjv93ZqAoWGXzZgusdQVJuFC76QWkYHAOcBwYBPwNHBGnl21wOOnAFMAamtr8+2SKpmM0rK1ldadu6js2YPB/SupqJCoi5UOlrrCJFwUo5hOAf5bVZsBRORZ4HhggIj09GsRNcDafA9W1RnADIC6urq8QSQtMhll5frNfOuxRpo2bqdmYF9+NbmOQ/evsiARFhdSV9hQWxOQKL5Fq4HjRKSfiAgwHlgOLATO9/e5DHgugrLFSsvW1rbgANC0cTvfeqyRlq3WSZoa7VeJe+gU734mE3XJTAJE0QfxKl5n9GvAW34ZZgDfB74jIn8FBgMPh122UmQySvPmT/nbxm00b/6UTCa8Sk3rzl1twSGraeN2WnfuCq0MqZbJwJb1sGmNdxvFSdmG2poARTJRTlVvBm5ut/l94JgIilOyqJt4Knv2oGZg3z2CRM3AvlT27BH4a6eeK5PkbKitCZA1VHZD1E08g/tX8qvJddQM7AvQFqAG97dhloFz5crdhtpGz4WaZEAs1UY3RN3EU1EhHLp/FXOvHmujmMLmypV7WOtZm/xcqUkGxAJEN7jQxFNRIVRX9Q7t9Yyv0PrOYV+5x3WobVJGXhWqSX7z5ehHt5VBDD8RdwTVxBNlx7cpkkuT5MJYz7qckjTyypWaZECsBtENQTTxRN3x3W0uXhkGUaa4Xrm7IElX3a7UJANi3+ZuyjbxDB3Yj+qq3t0+iUfd8d0tLl4ZBlmmuF25B6GUDtokXXW7VJMMgNUgShRUiouoO767xcUrQxfLlBSldtAm6ao74TXJZLyLkGWbgSY+0MDYuxYy8YEGVq7fXJa+gmzHd67YzG1w8crQxTIlRalDfZN21Z3gmmRy3kmIgmwGivXcBhfH5LtYpqQoNfjmXnVPXebdJmRYaNJYE1MJgmwGivXcBhfH5LtYpqTI11R06Jkg4vVJdNTc4kKSQ9MpCxAlCHr+Q2znNrjYHutimZKiffA99Ez4yvdg5hmJnDSWRqIa3zH2dXV12tjYGPrrxn4oqjHlkjuEWGR3cMgaUGsDAhwkIktUta6z/awGUYLOmoFsEZ8YcXHeRpzkNhVtWmMDAhLGAkSJCjUDWe0iRgoN06z6HOzYbgGjq5I0fNUANoqp7GI90S1tCg3TXPuaOxP94iRpw1ddElHGWKtBlFmsJ7qlTaFhmr367f7dJtUVzwYEBCPCjLH2yZVZrCe6pU2hORLbN+6+b23oXZPgSWORiXDtEfv0yqzcE90ss2uA8jWJnPMANNy3ex9rQzdRizAbgDUxlVk5J7pZh3fA2jeJ9KiETzfDlg3e360N3bggws5/mwfhsObNnzLxgYbPTMibe/XYeE6kiwMb9mpcE0AfhLPzIETkUGB2zqbPA9OAx/ztw4BVwIWqurH949PEOrwjYCkg9mQBM3oRdv6H/kmr6kpVHaWqo4AvA9uAucBNwAJVHQEs8O+nmnV4OyBJC9J39b24uL5Hd8T5s4yo8z/qS4HxwHuq+gFwDvCov/1R4NzISuWIWGd2zSdu/6BJOkGW8l4iHD1Tdkn6LEMUaR+EiPw78Jqq/lxENqnqgJy/bVTVgR09Pul9EJCgtB0RjuUu2Zb13okkCbmFSnkvm9Z4J9P2pi7zrmTjJEmfZRkU2wdR1H+miPTKs23fUgqW8/hK4Gzg6S4+boqINIpIY3NzDK9kuqjcS5pGJo5Xo0labKiU91JongjEowaYy9XPsrNadcS17g4DhIiME5EmYK2IvCgiw3L+/GI3X/sMvNrDev/+ehEZ4r/uEGBDvgep6gxVrVPVuupqG34YG67+g3YkSYsNlfJeCs0TmXN5/JpoXPwsO2v2cqBZrLMaxL8Bp6lqNTADeElEjvP/1t1L2UlAfc79+cBl/u+XAc918/mNS1z8B+1MknILlfJeckfPXP8GnPkzWHALNDXGowaYy8XPsrNatQO17s6GuVaq6tsAqjpHRFYAz4rITUDJnRci0g84FbgqZ/OdwFMi8g1gNXBBqc9vHBTHld2SlFuo1PeSHT2zaQ3Mavcv6XoNMJeLn2VntWoHat2dBYgdIvI5Vf07gKq+LSLjgeeBg0t9UVXdBgxut60Fb1STSSIX/0GLkaR5Ed15L0lI5e3aZ9nZMXXgmHf233kTsMcRVdUm4CS8K35jimeJ3OLLxSaauOvsmDpwzC3VhjGmODaruvw6O6YBHfOyptoQkbOAnwAH+Y8RQFV1726V0oQqMXMq4i6uJ1rXmmiSoLNjGvExLzYX033APwFvaZyrHClmmWEdEccJgya1iv1GrgGWWXCIL1sK1REODF00RD4BLS6KrUF8D3hBRH4PfJrdqKr/J5BSmbKzzLCOcGDoYuoFVYuLa9NhB4ot/e14WVf7AFU5P6ZMgl45zjLDOiKOEwaTJohanAOznoNQbA1ikKpOCLQkKRZG/0A2M2z714hNZtikXJ3FccJg0gRRiysUdGKeDLDYAPGyiExQ1e7mXzJ5FOofKOfKceVcCjV0SerYjeuEwSQJYgJaQpsOi/1WXgP8VkS2i8jHIrJZRD4OsmBpElb/QGwzwyatY9cmDEYriAloCW06LKoGoarW3xCgbP9A+7WnrX/Al9CrMxORIGpxCW06LHpNahEZCIzA66gGQFVfCaJQaRP7/oGgOZCTxiRMuSegJbTpsNiZ1N8ErgdqgKXAccBi4OTgipYese4f6KKSZnMn9OrMJEwCZ5oXW4O4Hjga+JOqjhORw4AfB1es9Mn2DyRZyaO1Enp1Zozriv0P+0RVPwEQkd6q+g5waHDFMknUrdnc1rFrTOiKrUE0icgAYB7eqnIbgbXBFcskkc3mNiZeih3FNNH/9RYRWQjsA/w2sFKZRLLRWsbES1H1dBE5Jfu7qv5eVefjrSltTNGyo7WyKT9stJYxbiu2iWmaiJwHfBfYC3gIL2nfo0EVzCRPmkZr5ZWUdCEmHA58X4p9ta8A7+ENcf0j8ISqnh9YqUxiVVQIg/tXUtmzB607d9GytbXsiQmdlNBkbiYgjnxfig0QA4Fj8YLEp8BBIlLyZZ+IDBCROSLyjoisEJExIjJIRF4SkXf924GlPn+Ygs7CmjTZoa4TH2hg7F0LmfhAAyvXb07+cUtauhATLEe+L8UGiD8Bv1HV0/HmQxwANHTjde8HfquqhwFHAiuAm4AFqjoCWODfd1pqT3btdCVIpnbhIksXYrrCke9LsQHiFGCHiExT1e3APZR4AheRvYETgYcBVLVVVTcB57C7T+NR4NxSnj9MqT3Z5ehqkAxsqKvrK4QlNJmbCYgj35diA8QP8NJrZEcubQZ+VuJrfh5oBmaKyOsi8pCI9Af2V9V1AP7tfiU+f2hsXH/Xg2QgCxc50l7boSAyiJrkcuT7UuwopmNVdbSIvA6gqhtFpNRQ1hMYDfwvVX1VRO6nC7UREZkCTAGora3tZO9g2bj+rgfJQBITxmGxFksXYrrCke9LsQFih4j0ABRARKqBUi/PmoAmVX3Vvz8HL0CsF5EhqrpORIYAG/I9WFVnADMA6urqIm3styysXQ+SgQx1daS9tlMJTOZmAuTA96XYADEdmAvsJyK3A+cD/1rKC6rq30VkjYgcqqorgfHAcv/nMuBO//a5Up4/TKkf109pQbLsiQktHbgxgRDV4i7C/Qyu4wHBG220ouQXFRmFN9muEngfuAKvP+QpoBZYDVygqh929Dx1dXXa2NhYajFMmZSUwru8BUjOkqTGhEBElqhqXaf7FRsgXGQBwrRxYNapMXFRbIAoekU5Y5zmQHutCYFdCITKAoQxJh6sKTF0dlRTyNKDGKcVmvToSPqJNLEaRMqUvOynMWHoqJYQl+HMCWI1iJSx9CDGaR3VEhxJP5EmFiBSxtKDGKd1VEtwJP1EmlgTU8pYehDjtI4mPTqSfiJN7MimjC37aZzWWS0hO5x5wIHerQWHQNlEuRSKfOazMR2xuQ6Bs4lypqCy50Iyppxs0qMzLCwbY4zJywKEMcaYvCxAGGOMycv6IIxxjXXSGkdYgDDOSuVoK0tIZxxiAcKUJOiTtws5oyIJUHFYX9ukhgUI02VhnLwL5Yyae/XYUIboRhagLCGdcYjVWU2XhZHwL+qcUZElNbSEdMYhFiBMl4Vx8s7mjMoVZs6oyAKUJaQzDrEAYbosjJN31DmjIgtQuQnppi7zbq2D2kQkklxMIrIK2AzsAnaqap2IDAJmA8OAVcCFqrqxo+exXEzRCKt9PspRTDt3Zli5YTNXPb7EFlYyiVNsLqYoA0Sdqv4jZ9u/AR+q6p0ichMwUFW/39HzWICITqGTdxKGpmYD4L0vreS8Lx/I4P6V7FfVmwP26UvPnnYlb+Ivjsn6zgFO8n9/FFgEdBggSpGEE5gL8iX8c2FoajnkdlC/uHwD4DUvhTWCyhhXRHU5pMCLIrJERKb42/ZX1XUA/u1+5X7R7Als4gMNjL1rIRMfaGDl+s1kMvFNee6SpCxnGvUIKmNcEVWAGKuqo4EzgGtE5MRiHygiU0SkUUQam5ubu/SiSTmBuSopJ9aoR1AZ44pIAoSqrvVvNwBzgWOA9SIyBMC/3VDgsTNUtU5V66qruzb0LyknMFcVOrHuymisamlRj6BKrUwGtqyHTWu820wm6hKlXugBQkT6i0hV9ndgArAMmA9c5u92GfBcuV/brgyDNbh/Jb+89Mt7nFjvOm8kt/16eaxqaRUVwqH7VzH36rE0fH8cc68eG7t+lNjJ5qB66BS47wjvdsNyCxIRC30Uk4h8Hq/WAF4n+ROqeruIDAaeAmqB1cAFqvphR8/V1VFMSelEddn6j7aztOkjBvTtxabtO3hw0Xu8vmYTDd8fx9CB/aIunnHVlvVeUMhNMzKg1nJQBcTZUUyq+j5wZJ7tLcD4IF8798rQRjEFo6Kigp88v3yPpjyrpcVUmGnHLQeVk1I3qDs7PHPowH5UV/Uu+8Su5s2f8reN22je/Gms2t3LxdrvEyLsJh/LQeWkSCbKlYtLE+Ws+Wq39nNNBvbtxcbtO6zWloez83LCbvKxdTBC5WwTU1JFnZ7aJbmT6CxwFhb5semoCSnsJp/cHFS2kp4z7OiXiQ2hzc/mnhQW6bHprAkpiiafigqvdjLgQO/WgkPk7BMoExtC+1mZjNK6cxc/u+BIfnnplznqwAGABc6sSC8qCq1ct82ffGppx0uTsLkc1sRUJtnO2fbNBWntnM3XfHLXeSO55z9X0rzl01QHzqzsRUUkI746a0JKQ5NPuUdpJbAfxTqpy8jZDscING/+lIkPNHzm5PeTc47gc/v0sT4IIu6DSPu8gyBO5jE6ptZJHYF8GU7TqlDzycH77UXNgL6pDw4Q8bycbBNS+xNkOZqQwpw/UapCTWzdOZkncC6HBQgTiELNJ3179bDgkCOyi4qgmpDi0swSxMk827HfvgYR47kcDn1i4SnHhDabFNcxmzAXA0GMGuqs89sVQYzSSmDHfupqEOVo9418/HoMWFqTlCrnlXmQTVVBNLElsGM/dZ3UhTpPC01oy9fx3LK1tUvPYUxqlKujNoymqtwA1KMSKnrAju2JOLF3pthO6uQegQK6Mva88Ap0GZsUZ7olsU2U5WpmCaOpKtvEtvdQ2NoMvxpnqcbbSV0TU1fGnhea6frUVWOiG79uyiqKocmJbqIsspml0+Me5oigIEY0JUTqahBd6TwtVNtQVeuATYCo1ihPfPqRTjq/izruYab6SODw1HJJXQ2iK52nHdU2Dt2/r3XAxlxUCRbTnrerqOMe5DyN9hI4PLVcUhcgoPix5x2lz7BJcfEX1Yk60hQbvihn/Rd13MMcERRmMIqZVAaIYtlQzWSL6kQddd6uqPtAij7u2aaqoCVweGq5pG6YqzFZYZ4oXVpEqatDvcst6gBlLBeTMZ0Kq4bo2gkx6j6QigphRPVePHXVGHbuytCzRwX77VXe5X9NeURWhxKRHiLyuog8798fLiKvisi7IjJbRKyHyAQuyDXKs1wbtRT12iWZjPJu8xYu/OViTrx7ERf+cjHvNm9JzlyQBImyke16YEXO/buAe1V1BLAR+EYkpTKmzKK+Ym8v6jxZrgVMU1gkTUwiUgOcCdwOfEdEBDgZ+Lq/y6PALcAvoiifMeXkwqilXFEPvnAtYJrCouqDuA/4HlDl3x8MbFLVnf79JmBoKU+8Y8cOmpqa+OSTT7pfSlM2ffr0oaamhl69epXl+eK0OFPUo5byiXKYtmsB0xQWeoAQkbOADaq6REROym7Os2veBkkRmQJMAaitrf3M35uamqiqqmLYsGF4FRMTNVWlpaWFpqYmhg8f3u3nc63TtzNRX7G7xsWA2RVxujjprtCHuYrIHcClwE6gD7A3MBc4Dficqu4UkTHALap6WkfPlW+Y64oVKzjssMMsODhGVXnnnXf4whe+0O3ninqYpum+uJ5k43ZxUoiz2VxV9QeqWqOqw4CLgd+p6iXAQuB8f7fLgOdKfQ0LDu4p52dibdjxF8bosSCU0sEe58y9Lk0V/D5eh/Vf8fokHo64PCUTEW688ca2+/fccw+33HJLh4+ZN28ey5cv73CfI488kkmTJhX8+6pVqzjiiCO6VNZp06bx8ssvA3Dfffexbdu2tr/99Kc/7dJzATzyyCNce+21XX5cV0Q9TNOkV1cvTqJKCFkukQYIVV2kqmf5v7+vqseo6v9Q1QtU9dMoy9YdvXv35tlnn+Uf//hH0Y/pLECsWLGCTCbDK6+8wtatW8tRTABuvfVWTjnlFKA8ASIMUQ/TNOnV1YuTuA/pTf1M6iDaQnv27MmUKVO49957uf322/f42wcffMCVV15Jc3Mz1dXVzJw5k6amJubPn8/vf/97brvtNp555hkOPvjgPR73xBNPcOmll7JixQrmz5/fVpNYsmQJV155Jf369eOEE05o2/+RRx5h3rx57Nq1i2XLlnHjjTfS2trK448/Tu/evXnhhRcYNGgQl19+OWeddRZr165l7dq1jBs3jn333Zdjjz2W7du3M2rUKL74xS8ya9Ys/uM//oPp06fT2trKscceywMPPECPHj2YOXMmd9xxB0OGDOGQQw6hd+9g+wGs09dEpasd7HFvDnWpiSl0QVb/rrnmGmbNmsVHH320x/Zrr72WyZMn8+abb3LJJZdw3XXXcfzxx3P22Wdz9913s3Tp0s8EB4DZs2dz0UUXMWnSJOrr69u2X3HFFUyfPp3Fixd/5jHLli3jiSee4M9//jM/+tGP6NevH6+//jpjxozhscce22Pf6667jgMOOICFCxeycOFC7rzzTvr27cvSpUuZNWsWK1asYPbs2TQ0NLB06VJ69OjBrFmzWLduHTfffDMNDQ289NJLnTaTlUtc27BNvOVenDR8fxxzrx7bYQd13JtDUx0ggqz+7b333kyePJnp06fvsX3x4sV8/evefMBLL72UP/7xj50+11/+8heqq6s56KCDGD9+PK+99hobN27ko48+YtOmTXzlK19pe75c48aNo6qqiurqavbZZx++9rWvAfClL32JVatWden9LFiwgCVLlnD00UczatQoFixYwPvvv8+rr77KSSedRHV1NZWVlVx00UVdel5j4qYrFydxbw5NdRNT0NW/qVOnMnr0aK644oqC+xQzuqe+vp533nmHYcOGAfDxxx/zzDPPcP7553f4+NymnoqKirb7FRUV7Ny5s9DD8lJVLrvsMu644449ts+bN89GjRlTQNybQ1Ndgwi6+jdo0CAuvPBCHn5494Cs448/nieffBKAWbNmtfUbVFVVsXnz5s88RyaT4emnn+bNN99k1apVrFq1iueee476+noGDBjAPvvs01YLmTVrVrfK274MvXr1YseOHQCMHz+eOXPmsGFMutGfAAAMPElEQVTDBgA+/PBDPvjgA4499lgWLVpES0sLO3bs4Omnn+5WGYxJmjg3h6Y6QIRR/bvxxhv3GM00ffp0Zs6cyciRI3n88ce5//77Abj44ou5++67Oeqoo3jvvffa9n/llVcYOnQoQ4fuzjxy4oknsnz5ctatW8fMmTO55pprGDNmDH377hnsumrKlCmcccYZjBs3ru3+yJEjueSSSzj88MO57bbbmDBhAiNHjuTUU09l3bp1DBkyhFtuuYUxY8ZwyimnMHr06G6VoZA4jyU3Jq4St2DQihUrujRbN64zOuOoq59NVlJmrxrjCmdnUrsmztW/tIj7WHJj4ir1AcK4L+5jyY2JKwsQxnlxH0tuTFxZgDDOi/tYcmPiKtXzIEw8xH0suTFxZQHCxEKUK6AZk1bWxBSQuXPnIiK88847ef9++eWXM2fOnKKfb+3atZx/vrdcxtKlS3nhhRfa/rZo0SL+67/+q8tlHDZsWJcyzhpj0sUCREDq6+s54YQT2mZNd9cBBxzQFlDKFSCMMaYjFiAyGdiyHjat8W4zmW4/5ZYtW2hoaODhhx9uCxCqyrXXXsvhhx/OmWee2ZayArwr+R/+8IeMGTOGuro6XnvtNU477TQOPvhgHnzwQWD3YkCtra1MmzaN2bNnM2rUKO666y4efPBB7r33XkaNGsUf/vAHmpubOe+88zj66KM5+uijaWhoAKClpYUJEyZw1FFHcdVVVxHnSZLGmOCluw8ik4ENy+HJSbBpNQyohYvrYb/DoaL02Dlv3jxOP/10DjnkEAYNGsRrr73GqlWrWLlyJW+99Rbr16/n8MMP58orr2x7zIEHHsjixYu54YYbuPzyy2loaOCTTz7hi1/8It/+9rfb9qusrOTWW2+lsbGRn//85wBs376dvfbai+9+97sAfP3rX+eGG27ghBNOYPXq1Zx22mmsWLGCH//4x5xwwglMmzaNX//618yYMaPk92iMSb50B4htzbuDA3i3T06Cb74Me+1f8tPW19czdepUwMuxVF9fz44dO5g0aRI9evTggAMO4OSTT97jMWeffTbgpeLesmULVVVVVFVV0adPHzZt2tSl13/55Zf3WJfh448/ZvPmzbzyyis8++yzAJx55pkMHDiw5PdojEm+dAeIna27g0PWptXe9hK1tLTwu9/9jmXLliEi7Nq1CxFh4sSJRaXmzk3Lnb3f1dTcmUyGxYsX503eZ6m5jTHFSncfRM9Kr1kp14Bab3uJ5syZw+TJk/nggw9YtWoVa9asYfjw4QwaNIgnn3ySXbt2sW7dOhYuXFjya7RPy93+/oQJE9qan8Dr1AYvC2w2JfhvfvMbNm7cWHIZTPxZhlzTmdADhIj0EZE/i8gbIvK2iPzY3z5cRF4VkXdFZLaIBD9Ntl+11+eQDRLZPoh+1SU/ZX19PRMnTtxj23nnncff//53RowYwZe+9CX+5V/+pW0VuFKMGzeO5cuXM2rUKGbPns3XvvY15s6d29ZJPX36dBobGxk5ciSHH354W0f3zTffzCuvvMLo0aN58cUXqa2t7eSVTFIFudyuSY7Q032L18bRX1W3iEgv4I/A9cB3gGdV9UkReRB4Q1V/0dFzlSPdN5mM1xexs9WrOfSr7lYHtSms1HTfpvyaN3/KxAca9kiCWDOwL3OvHmsTElPA2XTf6tni3+3l/yhwMpCdOfYocG4oBaqo8DqkBxzo3VpwMClgGXJNMSI5G4pIDxFZCmwAXgLeAzaparY3tgkYWujxxpjusQy5phiRBAhV3aWqo4Aa4BggX7tD3rYvEZkiIo0i0tjc3BxkMY1JLMuQa4oR6TBXVd0kIouA44ABItLTr0XUAGsLPGYGMAO8PogC+9hwTsfYrG23WIZcU4woRjFVi8gA//e+wCnACmAhcL6/22XAc6U8f58+fWhpabETkkNUlZaWFvr06RN1UUwOW27XdCaKGsQQ4FER6YEXoJ5S1edFZDnwpIjcBrwOPFzKk9fU1NDU1IQ1P7mlT58+1NTURF0MY0wXhB4gVPVN4Kg829/H64/oll69ejF8+PDuPo0xxqSejek0xhiTlwUIY4wxeVmAMMYYk1foqTbKSUSagQ9KfPi+QFzW27SyBidO5bWyBiNOZYXylPcgVe006VysA0R3iEhjMblIXGBlDU6cymtlDUacygrhlteamIwxxuRlAcIYY0xeaQ4QcVqQ2coanDiV18oajDiVFUIsb2r7IIwxxnQszTUIY4wxHUhFgHBqmdMi+OtlvC4iz/v3nSwngIisEpG3RGSpiDT62waJyEt+eV8SkYFRlxNARAaIyBwReUdEVojIGBfLKiKH+scz+/OxiEx1saxZInKD/7+1TETq/f85J7+3InK9X863RWSqv82JYysi/y4iG0RkWc62vGUTz3QR+auIvCkio8tdnlQECOBT4GRVPRIYBZwuIscBdwH3quoIYCPwjQjLmOt6vAy3Wa6WM2ucqo7KGXp3E7DAL+8C/74L7gd+q6qHAUfiHWPnyqqqK/3jOQr4MrANmIuDZQUQkaHAdUCdqh4B9AAuxsHvrYgcAXwLL+/bkcBZIjICd47tI8Dp7bYVKtsZwAj/ZwrQ4RLNJVHVVP0A/YDXgGPxJpv09LePAf7TgfLV+F+Ck4HnAXGxnDnlXQXs227bSmCI//sQYKUD5dwb+G/8fjeXy9qufBOABpfLirf64xpgEF4C0OeB01z83gIXAA/l3P/fwPdcOrbAMGBZzv28ZQN+CUzKt1+5ftJSg4jTMqf34X1hM/79wbhZziwFXhSRJSIyxd+2v6quA/Bv94usdLt9HmgGZvrNdw+JSH/cLGuui4F6/3cny6qqfwPuAVYD64CPgCW4+b1dBpwoIoNFpB/wVeBAHD22vkJlywbmrLIf49QECO3GMqdhEZGzgA2quiR3c55dXRp6NlZVR+NVd68RkROjLlABPYHRwC9U9ShgK4400RTit9mfDTwddVk64reJnwMMBw4A+uN9H9qL/Hurqivwmr5eAn4LvAHs7PBB7gr83JCaAJGlqpuAReQsc+r/qeAypyEaC5wtIquAJ/Game7DvXK2UdW1/u0GvHbyY4D1IjIEwL/dEF0J2zQBTar6qn9/Dl7AcLGsWWcAr6nqev++q2U9BfhvVW1W1R3As8DxOPq9VdWHVXW0qp4IfAi8i7vHFgqXrQmv9pNV9mOcigAhAS9zWi6q+gNVrVHVYXhNC79T1UtwrJxZItJfRKqyv+O1ly8D5uOVExwpr6r+HVgjIof6m8YDy3GwrDkmsbt5Cdwt62rgOBHpJyLC7mPr6vd2P/+2FvgnvGPs6rGFwmWbD0z2RzMdB3yUbYoqm6g7jULq9BmJt4zpm3gnsGn+9s8Dfwb+ileN7x11WXPKfBLwvMvl9Mv1hv/zNvAjf/tgvI72d/3bQVGX1S/XKKDR/x7MAwY6XNZ+QAuwT842J8vql+3HwDv+/9fjQG+Hv7d/wAtgbwDjXTq2eMFqHbADr4bwjUJlw2ti+r94/alv4Y0iK2t5bCa1McaYvFLRxGSMMabrLEAYY4zJywKEMcaYvCxAGGOMycsChDHGmLwsQBgTMT/tw0IR2SIiP4+6PMZk9ex8F2NMwD7BSxp3hP9jjBOsBmFMASLyz+KtI7JURH4pIgf5Ofn3FZEKEfmDiEzw953nJyx8OydpIX6t4C7/by+LyDEiskhE3heRswFUdauq/hEvUBjjDAsQxuQhIl8ALsJLRjgK2AV8BS/R24PAjcByVX3Rf8iVqvploA64TkQG+9v7A4v8v20GbgNOBSYCt4b1fowphTUxGZPfeLzFev7ipReiL16m3VtE5ALg23ipO7KuE5GJ/u8H4i3i0gK04mUNBS8dwqequkNE3sLL+2+MsyxAGJOfAI+q6g/22OitIVDj390L2CwiJ+ElgByjqttEZBHQx99nh+7OZ5PBW90QVc3kZDo1xknWxGRMfguA83Myfw4SkYPwmphmAdOAX/n77gNs9IPDYXip5I2JPbuCMSYPVV0uIv+Kt1peBV52ze8AR+P1S+wSkfNE5ArgCeDbIvIm3rKPf+rq6/lrgOwNVIrIucAEVV1eprdjTEksm6sxxpi8rInJGGNMXhYgjDHG5GUBwhhjTF4WIIwxxuRlAcIYY0xeFiCMMcbkZQHCGGNMXhYgjDHG5PX/AZzbS+tHPFUrAAAAAElFTkSuQmCC\n", "text/plain": [ "
" ] }, "metadata": { "needs_background": "light" }, "output_type": "display_data" } ], "source": [ "sns.scatterplot(x='exam1',y='exam2',data=data[data['decision']==0],label='Not Admitted');\n", "sns.scatterplot(x='exam1',y='exam2',data=data[data['decision']==1],label='Admitted');" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "#### 1.2    Implementation\n", "We will add another dimension to the Design Matrix to accomodate the $\\theta_0$ intercept term. We will also initialize the fitting parameters to 0." ] }, { "cell_type": "code", "execution_count": 4, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "X : (100, 3)\n", "y : (100, 1)\n", "theta : (1, 3)\n" ] } ], "source": [ "m=data.shape[0]\n", "X=np.ones(shape=(m,3))\n", "X[:,1:]=data.values[:,:2]\n", "print(f'X : {X.shape}')\n", "\n", "y=data.values[:,2]\n", "y=y[:,np.newaxis]\n", "print(f'y : {y.shape}')\n", "\n", "theta=np.zeros(shape=(1,X.shape[1]))\n", "print(f'theta : {theta.shape}')" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "##### 1.2.1    Sigmoid/Logistic Function\n", "\n", "Before starting with the implementation, let's try to understand the sigmoid function. \n", "For large positive values of $x$, sigmoid function is close to 1, while for large negative values, sigmoid function is close to 0. And sigmoid of 0 is exactly 0.5 .\n", "\n", "$$ \\sigma(x)=\\frac{1}{1+e^{-x}} $$" ] }, { "cell_type": "code", "execution_count": 5, "metadata": {}, "outputs": [], "source": [ "def sigmoid(z):\n", " sigma=1/(1+np.exp(-z))\n", " return sigma" ] }, { "cell_type": "code", "execution_count": 6, "metadata": {}, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYUAAAEKCAYAAAD9xUlFAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvOIA7rQAAIABJREFUeJzt3Xl8VPW9//HXZyYrJGFJwh4FBBRQQYiC2goqIlIvorUKt70V24re1l791WuLtdr+tP4eWK2/3v5qq7i3Lri0thQpghVbrQs7QkAkIktIIGEnhKzz/f0xwziECQSSk5NJ3s/HI4+c5Tszn5w5mfec79nMOYeIiAhAwO8CRESk9VAoiIhIlEJBRESiFAoiIhKlUBARkSiFgoiIRCkUREQkSqEgIiJRCgUREYlK8ruAE5WTk+P69u3rdxkiIgll2bJlO51zucdrl3Ch0LdvX5YuXep3GSIiCcXMNjemnbqPREQkSqEgIiJRCgUREYlKuH0K8dTU1FBUVERlZaXfpUgzSEtLo0+fPiQnJ/tdiki70yZCoaioiMzMTPr27YuZ+V2ONIFzjl27dlFUVES/fv38Lkek3fGs+8jMnjazUjNb08B8M7Nfm1mhmX1sZiNO9rUqKyvJzs5WILQBZkZ2dra2+kR84uU+hWeBCceYfwUwMPIzHfhdU15MgdB26L0U8Y9n3UfOuX+aWd9jNLkK+L0L3w/0QzPrbGY9nXMlXtUkIonPOUdtyFFVG6K6NkRVbR01tY7qujqqax21oRA1dY7auhB1IUdNyFEXClEX4ovfzhEKOULOURdyOAch5whFfrsjhsO/w68dmRYZBgiPfTF+uMYv5h/dtn77I/6+I//YI+ZdOrg7w/I6n9yCayQ/9yn0BrbGjBdFph0VCmY2nfDWBKecckqLFHeiHnjgAV588UWCwSCBQIDHH3+cUaNGMXbsWB5++GHy8/MpLy/nzjvvZMGCBWRlZREIBLjlllu46aab2LRpE/369eMnP/kJ999/PwA7d+6kZ8+e3HzzzfzmN78BYNasWTzyyCMAZGVl8cgjj/ClL30J4IjX6tu3L5mZmQDU1dVxzTXXcM8995CamnpU7du3b+f2229nyZIlpKam0rdvX371q19RXFzMww8/zNy5c6Ntp02bxpVXXsm1117L2LFjKSkpIS0tjZSUFJ544gmGDx/OtGnTOP/887n55pujj/vzn//MrFmzmDdvHsFgkLPOOis6b8qUKcyYMaOZ3xFprWrrQuw6WM3O8ip2H6xm98Fq9hysZt+hWvYdquFAZQ3lVbUcqKylvKqWQ9V1VNSEfx+qrqOyNvxh317Ebjh3y0pr06EQr48g7jvtnJsFzALIz89vdWvDBx98wNy5c1m+fDmpqans3LmT6urqo9p95zvfoX///mzYsIFAIEBZWRlPP/10dH7//v2ZO3duNBReffVVhg4dGp0/d+5cHn/8cd577z1ycnJYvnw5kydPZvHixfTo0eOo11u0aBE5OTmUl5czffp0pk+fznPPPXdEG+ccV199NTfccAOzZ88GYOXKlezYsaNRf/sLL7xAfn4+zzzzDHfeeScLFy5k6tSpzJw584hQmD17NlOnTgUgPT2dlStXNur5JfHUhRzFew+xcedBPi8rZ+ueQ2zbc4htew+xfX8lu8qraOgzvWNKkE7pyWSkJZGRmkRmWhLds1LpmJJEWkqQ9OTwT2pSgNTkAKlJQZKDAVKSwj/JASM5GCApGPkdMJKCRjAQIGhGMHD4BwJmBCLTAgHDgGDAMAMjPN0IfyibHZ4eftzhNvV7Og9P/2L48HSLGY5t3/q6Sv0MhSIgL2a8D1DsUy1NUlJSQk5OTvRbeE5OzlFtPvvsMxYvXsyLL75IIBDelZObm8uPfvSjaJv09HQGDx7M0qVLyc/P5+WXX+a6666juDi8WB588EEeeuih6POPGDGCG264gUcffTQaJPFkZGTw2GOPkZeXx+7du+natWt03qJFi0hOTuaWW26JThs+fDgA77zzTqOXwfnnn89DDz0EwLhx45g2bRolJSX07NmTiooK3nrrLZ544olGP58khqraOtZs28+qrXtZV7KfT7Yf4NMdB6iqDUXbpCUH6N05nd5dOjCkZxbds1LJzUojNyOFrh1T6doxhS4dkslKTyY5qFOn/OZnKMwBbjWz2cAoYF9z7E/4338tYG3x/iYXF2tIryx++m9DG5w/fvx47rvvPgYNGsS4ceO4/vrrGTNmzBFtCgoKGDZsWDQQGjJlyhRmz55Njx49CAaD9OrVKxoKBQUFjBw58oj2+fn5R337jycrK4t+/fqxYcMGRo0aFZ2+Zs2ao57zZMyfP5/JkycDEAwGueaaa3jllVe47bbbmDNnDhdffHG0O+vQoUPR4AG46667uP7665tcg3ivqraOZZv38N6GnXywcRcF2/ZTXRcOgJyMVAb3zOQ/Rp/KgG4Z9MvpSL/cjuRmpLbKb8QSn2ehYGYvAWOBHDMrAn4KJAM45x4D5gETgUKgArjRq1q8lpGRwbJly3j33XdZtGgR119/PTNnzmTatGkNPuaBBx7g1VdfpbS0NPqhDzBhwgTuueceunfv3qgPSudco//hXEN7thrQ0PPGTv/617/OwYMHqaurY/ny5dHpU6dO5c477+S2225j9uzZfPOb34zOU/dRYtlfWcPb60qZt7qEf24oo7ImRDBgDM/rzI0X9uWcU7ow4pTOdMtK87tUaQZeHn009TjzHfC95n7dY32j91IwGGTs2LGMHTuWs846i+eee+6IUBgyZAirVq0iFAoRCAS4++67ufvuu8nIyDjieVJSUhg5ciS//OUvKSgo4K9//esRz7Fs2TIuueSS6LTly5czZMiQ49Z34MABNm3axKBBg46YPnToUF577bW4j8nOzmbPnj1HTNu9e/cR3WMvvPACw4YNY8aMGXzve9/jT3/6EwAXXnghJSUlrFq1ivfffz+6v0ISQyjkeLdwJy8v2cJba0uprgvRIyuN6/LzuGhgLqP6dyUzTWect0Vt4oxmv61fv55AIMDAgQOB8I7aU0899Yg2AwYMID8/P3p0UTAYpLKyMu639zvuuIMxY8aQnZ19xPQf/vCH/OhHP2L+/PlkZ2ezcuVKnn32WT766KNj1ldeXs53v/tdJk+eTJcuXY6Yd8kll/DjH/+YJ554gptuugmAJUuWUFFRwejRoykuLmbdunUMHjyYzZs3s2rVqiO6fgCSk5P5+c9/zmmnnRZta2Zcd9113HDDDUycOJG0NH2LTATlVbW88OFmfv/BZrbtPUSXDsl8Y/SpfOXsnpyT15lAQN1AbZ1CoRmUl5fz/e9/n71795KUlMSAAQOYNWvWUe2efPJJ7rzzTgYMGEDXrl1JT0/nwQcfPKrd0KFDjzjq6LBJkyaxbds2LrjgAsyMzMxMnn/+eXr27Bm3rosvvjh8vHUoxNVXX80999xzVBsz4/XXX+f2229n5syZpKWlRQ9JTU1N5fnnn+fGG2+ksrKS5ORknnzySTp16nTU86Snp3PHHXfw8MMP89RTTwHhLqSHHnqImTNnHtG2/j6FCRMmHNVGWtbeimqe/tcmnnt/E/sO1TC6f1dmXHEG44d2JzUp6Hd50oLsRPuZ/Zafn+/q32Tn8LdTaTv0nraMmroQz3+4mV+9tYF9h2oYP6Q73714AMM9PhZeWp6ZLXPO5R+vnbYURNqpdzeU8dM5BWwsO8iXBuRw91cGM7hnlt9lic8UCiLtTEV1Lf9n3jqe/3AL/XI68tQN+VxyRjcdNipAGwqFEzk0U1q3ROvSTCTLNu/hB6+sZMvuCr7zpX789+Wnk5asfQbyhTYRCmlpaezatUuXz24DDt9PQUcrNb+XFm/h3r+soXtWGi/dNJrR/bOP/yBpd9pEKPTp04eioiLKysr8LkWaweE7r0nzqKkL8fO5a3nug81cNCiX/zf1HDql6xwDia9NhEJycrLu0iUSR0V1LTf/YRnvbtjJTV/ux4wrBhPUuQZyDG0iFETkaOVVtXzrmSUs3bybX3z1bK47N+/4D5J2T6Eg0gbtO1TDtGcW83HRPn499RyuPLuX3yVJglAoiLQxB6tq+eZTH7G2ZD+//foILh969L02RBqiUBBpQ2rrQnz/pRWs3raPx/8jn8uGdPe7JEkwCgWRNsI5x0/nFPD2J6U8cPWZCgQ5KbrNkUgb8fg/N/LCR1u4ZcxpfH3Uqcd/gEgcCgWRNuDdDWU8OP8Trjy7Jz+8/HS/y5EEplAQSXCl+yv5Xy+vZEBuBg9dO0z3PJAm0T4FkQRWF3LcNnsl5VW1vHjTaNJTdB0jaRqFgkgC+83bhXywcRe/+OrZDOqe6Xc50gao+0gkQa3cupf/+funTB7ei6/l61pR0jwUCiIJqKYuxIw/fkxuZir3TT5TVweWZqPuI5EENOufG/lk+wFm/cdIstJ0xVNpPtpSEEkwG8vK+Z+/b2DiWT0Yr0tYSDNTKIgkEOccd/1pNWlJAX42aajf5UgbpFAQSSBzVhXz0ee7+fHEwXTL1N3ppPkpFEQSRGVNHb+Yv54hPbO4Ll/3RhBvKBREEsSz729i295D/OQrg3XWsnhGoSCSAHaVV/Ho24VcekY3LhiQ43c50oYpFEQSwK//voGKmjrumniG36VIG6dQEGnlPt95kBc+2sKUc/MY0E2XshBveRoKZjbBzNabWaGZzYgz/xQzW2RmK8zsYzOb6GU9Iono0UWFBAPGbeMG+l2KtAOehYKZBYFHgSuAIcBUMxtSr9lPgFecc+cAU4DfelWPSCLauruC11ds499HnaJDUKVFeLmlcB5Q6Jzb6JyrBmYDV9Vr44CsyHAnoNjDekQSzm/fKSRoxs0XneZ3KdJOeHnto97A1pjxImBUvTY/AxaY2feBjsA4D+sRSSjb9h7itWVFXH9uHj06aStBWoaXWwrxDqR29canAs865/oAE4E/mNlRNZnZdDNbamZLy8rKPChVpPV5/B+f4RzcMkZbCdJyvAyFIiD2tMs+HN099G3gFQDn3AdAGnDUQdjOuVnOuXznXH5ubq5H5Yq0HqX7K5m9ZCvXjuxDny4d/C5H2hEvQ2EJMNDM+plZCuEdyXPqtdkCXApgZoMJh4I2BaTde+6DTdTUhfjPsdpKkJblWSg452qBW4E3gXWEjzIqMLP7zGxSpNkdwE1mtgp4CZjmnKvfxSTSrlTW1PHiR1u4bHB3Ts3u6Hc50s54epMd59w8YF69affGDK8FLvSyBpFE8+cV29hTUcONF/bzuxRph3RGs0gr4pzj6X99zuCeWYzu39XvcqQdUiiItCLvf7aLT3eUc+OFfXXfZfGFQkGkFXn6vc/J7pjCpGG9/C5F2imFgkgrsWnnQd5eX8rXR51CWnLQ73KknVIoiLQSLy7eQtCMb4w+1e9SpB1TKIi0AtW1If64rIhLB3ejW5YuaSH+USiItAIL1+5g18Fqppx3it+lSDunUBBpBWYv2ULvzulcNFCXcRF/KRREfLZ1dwXvbtjJ1/L7EAzoMFTxl0JBxGevLN2KGVyXn3f8xiIeUyiI+Ki2LsQrS7cyZlAuvTqn+12OiEJBxE//+LSMHfurmHKudjBL66BQEPHRH5cXkd0xhUsHd/O7FBFAoSDim32HanhrXSn/NqwXyUH9K0rroDVRxCd/W11CdW2Iq8/p7XcpIlEKBRGfvL5iG/1zOnJ2n05+lyISpVAQ8UHRngo++nw3V5/TW5fIllZFoSDig7+sLAZgsrqOpJVRKIi0MOccr6/Yxrl9u5DXtYPf5YgcQaEg0sIKivdTWFqurQRplRQKIi3szyu2kRw0vnJWT79LETmKQkGkBYVCjjdWl3DRwFw6d0jxuxyRoygURFrQiq17KNlXyZXDtJUgrZNCQaQFzf24hJSkAOMGd/e7FJG4FAoiLSQUcsxbXcKYQblkpiX7XY5IXAoFkRaydPMeduyv4sqz1XUkrZdCQaSFvPFxMalJAS5V15G0YgoFkRZQF3LMW7OdS87oRkZqkt/liDRIoSDSAhZ/vpuyA1V8RV1H0sopFERawBuri0lLDnDJGbqZjrRuCgURj4VCjjcLdnDx6d3okKKuI2ndPA0FM5tgZuvNrNDMZjTQ5jozW2tmBWb2opf1iPhhxdY9lB2oYsKZPfwuReS4PPvaYmZB4FHgMqAIWGJmc5xza2PaDATuAi50zu0xM21bS5szf812UoLqOpLE4OWWwnlAoXNuo3OuGpgNXFWvzU3Ao865PQDOuVIP6xFpcc455hds58IB2TphTRKCl6HQG9gaM14UmRZrEDDIzP5lZh+a2YR4T2Rm081sqZktLSsr86hckea3tmQ/W3cfUteRJAwvQyHePQZdvfEkYCAwFpgKPGlmnY96kHOznHP5zrn83NzcZi9UxCvz12wnYOhaR5IwvAyFIiAvZrwPUBynzV+cczXOuc+B9YRDQqRNmL9mO6P6ZZOdkep3KSKN4mUoLAEGmlk/M0sBpgBz6rX5M3AxgJnlEO5O2uhhTSItprC0nA2l5eo6koTiWSg452qBW4E3gXXAK865AjO7z8wmRZq9Cewys7XAIuBO59wur2oSaUlvFmwHYPxQdR1J4vD0TBrn3DxgXr1p98YMO+AHkR+RNmVBwXaG5XWmZ6d0v0sRaTSd0Szige37KllVtI/LtZUgCUahIOKBhet2ADB+iEJBEotCQcQDCwq20z+nI6flZvhdisgJUSiINLP9lTV8uHEXlw3pjlm803VEWi+Fgkgze2d9GTV1TkcdSUJSKIg0swUF28nJSGV4Xhe/SxE5YQoFkWZUVVvHO+vLGDe4G8GAuo4k8SgURJrRhxt3U15Vq64jSVgKBZFmtKBgOx1SglxwWo7fpYicFIWCSDMJhRwL1+5gzKBc0pKDfpcjclIUCiLNZPW2fZQeqOIynbAmCazBUDCzeWbWt+VKEUlsC9fuIBgw3XZTEtqxthSeBRaY2d1mpvsIihzHwrU7OLdvFzp3SPG7FJGT1uBVUp1zr5jZG8C9wFIz+wMQipn/SAvUJ5IQtuyqYP2OA9xz5RC/SxFpkuNdOrsGOAikApnEhIKIfGHB2si9E7Q/QRJcg6FgZhOARwjfLW2Ec66ixaoSSTAL1+7gjB6Z5HXt4HcpIk1yrC2Fu4GvOecKWqoYkUS052A1Szbt5nsXD/C7FJEmO9Y+hS+3ZCEiiertT0oJOXQoqrQJOk9BpIkWrt1Bj6w0zurdye9SRJpMoSDSBJU1dfxzQxnjhnTTvROkTVAoiDTB+5/tpKK6jvFDevhdikizUCiINMGCgh1kpiYxun+236WINAuFgshJqgs53lq3g7FndCMlSf9K0jZoTRY5SSu27GFnebVOWJM2RaEgcpIWrt1BctAYe3qu36WINBuFgshJcM7xZsF2zj8th8w0XS9S2g6FgshJKCwtZ9OuCnUdSZujUBA5CQvW7gB0FrO0PQoFkZOwYO0OhuV1pntWmt+liDQrhYLICSree4hVW/eq60jaJE9DwcwmmNl6Mys0sxnHaHetmTkzy/eyHpHmsKAgfO+EK87UWczS9ngWCmYWBB4FrgCGAFPN7KjbUplZJvBfwEde1SLSnOYXbGdQ9wz652b4XYpIs/NyS+E8oNA5t9E5Vw3MBq6K0+5+4BdApYe1iDSLXeVVLP58NxOGaitB2iYvQ6E3sDVmvCgyLcrMzgHynHNzPaxDpNm8tW4HIQeXq+tI2igvQyHedYRddKZZAPi/wB3HfSKz6Wa21MyWlpWVNWOJIidm/prtnNK1A0N6ZvldiognvAyFIiAvZrwPUBwzngmcCbxjZpuA0cCceDubnXOznHP5zrn83FxdUkD8sb+yhvcKdzLhzB66d4K0WV6GwhJgoJn1M7MUYAow5/BM59w+51yOc66vc64v8CEwyTm31MOaRE7aok9KqalzXK79CdKGeRYKzrla4FbgTWAd8IpzrsDM7jOzSV69rohX5q/ZTrfMVM7J6+x3KSKeSfLyyZ1z84B59abd20DbsV7WItIUFdW1vLO+jGtH9iEQUNeRtF06o1mkERZ9UsahmjomntXT71JEPKVQEGmEuR8Xk5uZynn9uvpdioinFAoix3Gwqpa3Pyll4pk9CKrrSNo4hYLIcfz9k1KqakN85exefpci4jmFgshxzF1VTPesVPJP7eJ3KSKeUyiIHMOByhre+bSMiWf11FFH0i4oFESO4e/rSqmuDXHl2TrqSNoHhYLIMcz9uJhendI4J09dR9I+KBREGrCvooZ/frqTK9R1JO2IQkGkAfPWlFBdF2Ly8N7HbyzSRigURBrw+vJtnJbbkTN76zLZ0n4oFETi2Lq7gsWbdnPNiD66TLa0KwoFkTj+snIbAJOG6YQ1aV8UCiL1OOd4fcU2zuvblbyuHfwuR6RFKRRE6lm9bR+flR3k6hHawSztj0JBpJ7XV2wjJRhg4pk6YU3aH4WCSIzauhB/XVXMpYO70alDst/liLQ4hYJIjLc/KWVneTVXn6OuI2mfFAoiMV5espXczFQuPqOb36WI+EKhIBJRsu8Qi9aX8rWRfUgO6l9D2iet+SIRry4tIuTg+nPz/C5FxDcKBREgFHK8vGQrFw7I5tTsjn6XI+IbhYII8G7hTrbtPcSUc0/xuxQRXykURICXl2yhS4dkxg/t7ncpIr5SKEi7V3agioVrd3DNiD6kJgX9LkfEVwoFafde/GgLNXWOfx+lriMRhYK0a9W1IZ7/aDNjT8/ltNwMv8sR8Z1CQdq1N1YXU3agihsv7Od3KSKtgkJB2i3nHE+/t4kB3TK4aGCO3+WItAoKBWm3lm3ew+pt+5h2QV/dXU0kQqEg7dYz/9pEp/RkrtF9E0SiPA0FM5tgZuvNrNDMZsSZ/wMzW2tmH5vZ383sVC/rETls295DzC/YzpTz8uiQkuR3OSKthmehYGZB4FHgCmAIMNXMhtRrtgLId86dDbwG/MKrekRiPf6PzwgY3HB+X79LEWlVvNxSOA8odM5tdM5VA7OBq2IbOOcWOecqIqMfAn08rEcEgB37K5m9ZCvXjuxDr87pfpcj0qp4GQq9ga0x40WRaQ35NvC3eDPMbLqZLTWzpWVlZc1YorRHj/9jI3Uhx3+OGeB3KSKtjpehEO9wDhe3odk3gHzgoXjznXOznHP5zrn83NzcZixR2pud5VW8uHgzk4f35pTsDn6XI9LqeLmHrQiIvTB9H6C4fiMzGwfcDYxxzlV5WI8IT7y7keraEN+7+DS/SxFplbzcUlgCDDSzfmaWAkwB5sQ2MLNzgMeBSc65Ug9rEWHPwWr+8MFm/m1YL/rrkhYicXkWCs65WuBW4E1gHfCKc67AzO4zs0mRZg8BGcCrZrbSzOY08HQiTfbookIO1dRx68XalyDSEE8P0HbOzQPm1Zt2b8zwOC9fX+SwzbsO8twHm7huZB4Du2f6XY5Iq6UzmqVd+MX89SQFAvxg/CC/SxFp1RQK0uYt27ybN1aXcPOY/nTPSvO7HJFWTaEgbZpzjp+/sY5umalMv6i/3+WItHoKBWnT5qwqZsWWvfz3+NN1jSORRlAoSJu1t6Ka++eu5ew+nfjqSF1BRaQx9NVJ2qwH3ljHnooafv+tUQQDul+CSGNoS0HapPc27OTVZUXcfFF/hvTK8rsckYShUJA251B1HT9+fTX9cjryX5cO9LsckYSi7iNpc2b+bR1bdlcwe/po0pKDfpcjklC0pSBtyvw1JTz3wWa+dWE/RvfP9rsckYSjUJA2Y+vuCu587WOG9enEjCvO8LsckYSkUJA2obo2xK0vrQDgN/8+gpQkrdoiJ0P7FCThOee4f+5aVm3dy+++PoK8rrp5jsjJ0tcpSXhPvfc5f/hwM9Mv6s8VZ/X0uxyRhKZQkIQ2b3UJP39jHRPP6sGMCdqPINJUCgVJWEs37eb2l1cy8tQuPHLdcAI6a1mkyRQKkpCWbNrNtGeW0LtzOk98M1/nI4g0E4WCJJz3P9vJN59aTLesVF66aTRdO6b4XZJIm6FQkITyzvpSbnxmCX26pDN7+mh6dNJNc0Sakw5JlYTgnOOZf23i52+s5fQeWTz/7fPIzkj1uyyRNkehIK1eVW0dP3l9Da8uK2L8kO48cv1wMlK16op4Qf9Z0qp9VlbOD15eyaqiffzXJQO4fdwgHWUk4iGFgrRKoZDjuQ82MfNvn5CeEuSxb4xgwpk6MU3EawoFaXXWFu/nZ38tYPHnu7n49Fwe/OrZdMvSDmWRlqBQkFaj7EAVjyxcz+wlW+mUnszMa87i+nPzMFN3kUhLUSiI77bvq+TJdzfy4uItVNeGuPGCftx26UA6dUj2uzSRdkehIL5wzrF62z5e+HALr6/YRp1zTBrWi1svGcBpuRl+lyfSbikUpEWVHqjkb6u38/KSrawt2U9acoCv5ffhljGn6ZLXIq2AQkE85Zzjs7Jy/vHpTuavKWHp5j04B0N7ZXH/5DOZNKwXndLVTSTSWigUpFmFQo4NpeUs37KHpZv28K/CnWzfXwnAGT0yue3SgVxxZk9O75Hpc6UiEo+noWBmE4D/AYLAk865mfXmpwK/B0YCu4DrnXObvKxJmodzjrLyKj4vO8hnZQf5ZPt+1pXsZ13JAcqragHo0iGZC07L4cIBOXx5YI66h0QSgGehYGZB4FHgMqAIWGJmc5xza2OafRvY45wbYGZTgAeB672qSRqnLuTYU1HN7oPV7CyvonR/FTv2V1Kyr5Jtew9RtOcQRbsrOBD58AfISE3ijB6ZXH1Ob4bndWbEqV3om91Bh5OKJBgvtxTOAwqdcxsBzGw2cBUQGwpXAT+LDL8G/MbMzDnnPKwroTjnqAs56g7/jvzUhhy1dY6aulBkOERVbYiauhDVtSGqI7+rakNU1tRRWRPiUE0dh6prqaiuo6K6jvKqWsoraymvqmV/ZQ17K2rYd6iG/ZU1xHsHOqYE6dOlA727pHNu3y70y+lI/9wM+ud0pE+XdAWASBvgZSj0BrbGjBcBoxpq45yrNbN9QDaws7mLeWXJVma9uzE63lDuuAZGDg8652KG4fCYcxzxQRqvXSjaJjwccg5X73fIOUKh8HBdZHpzSwqXWtYHAAAITklEQVQY6SlBMlOTyEhLIiM1ia4dU+iX05FO6cl07pBCdscUunZMITsjhe5ZaXTPStNF6ETaAS//y+N9baz/EdeYNpjZdGA6wCmnnHJSxXTpmMLp3evt3Gzgi23s5NhvvxadFjtsX7Q3ODx2uM3hhxtGIBAZMgiaRdsEAkYg8jzBgGFmBCw8HDAjGIj5MSMpaCQFjGAgQFLQSA4aSYEAKUkBUoIBkoMBUpMDpCaFp6UnB0lLDpKWFCQ9JUhKkm6jISLxeRkKRUBezHgfoLiBNkVmlgR0AnbXfyLn3CxgFkB+fv5JfXe+bEh3LhvS/WQeKiLSbnj5lXEJMNDM+plZCjAFmFOvzRzghsjwtcDb2p8gIuIfz7YUIvsIbgXeJHxI6tPOuQIzuw9Y6pybAzwF/MHMCglvIUzxqh4RETk+T/ccOufmAfPqTbs3ZrgS+JqXNYiISONpj6OIiEQpFEREJEqhICIiUQoFERGJUiiIiEiUJdppAWZWBmw+yYfn4MElNJqB6joxquvEtdbaVNeJaUpdpzrnco/XKOFCoSnMbKlzLt/vOupTXSdGdZ241lqb6joxLVGXuo9ERCRKoSAiIlHtLRRm+V1AA1TXiVFdJ6611qa6TozndbWrfQoiInJs7W1LQUREjqHNhYKZfc3MCswsZGb59ebdZWaFZrbezC5v4PH9zOwjM9tgZi9HLvvd3DW+bGYrIz+bzGxlA+02mdnqSLulzV1HnNf7mZlti6ltYgPtJkSWYaGZzWiBuh4ys0/M7GMze93MOjfQrkWW1/H+fjNLjbzHhZF1qa9XtcS8Zp6ZLTKzdZH1/7Y4bcaa2b6Y9/feeM/lQW3HfF8s7NeR5fWxmY1ogZpOj1kOK81sv5ndXq9Niy0vM3vazErNbE3MtK5mtjDyWbTQzLo08NgbIm02mNkN8dqcEOdcm/oBBgOnA+8A+THThwCrgFSgH/AZEIzz+FeAKZHhx4D/9LjeXwL3NjBvE5DTgsvuZ8B/H6dNMLLs+gMpkWU6xOO6xgNJkeEHgQf9Wl6N+fuB7wKPRYanAC+3wHvXExgRGc4EPo1T11hgbkutT419X4CJwN8I34hwNPBRC9cXBLYTPo7fl+UFXASMANbETPsFMCMyPCPeeg90BTZGfneJDHdpSi1tbkvBObfOObc+zqyrgNnOuSrn3OdAIXBebAML33vzEuC1yKTngMle1Rp5veuAl7x6DQ+cBxQ65zY656qB2YSXrWeccwucc7WR0Q8J38XPL435+68ivO5AeF261GLv6+oB51yJc255ZPgAsI7wPdATwVXA713Yh0BnM+vZgq9/KfCZc+5kT4ptMufcPzn6rpOx61FDn0WXAwudc7udc3uAhcCEptTS5kLhGHoDW2PGizj6nyYb2BvzARSvTXP6MrDDObehgfkOWGBmyyL3qW4Jt0Y24Z9uYHO1McvRS98i/K0ynpZYXo35+6NtIuvSPsLrVouIdFedA3wUZ/b5ZrbKzP5mZkNbqKTjvS9+r1NTaPiLmR/L67DuzrkSCIc+0C1Om2Zfdp7eZMcrZvYW0CPOrLudc39p6GFxptU/9KoxbRqlkTVO5dhbCRc654rNrBuw0Mw+iXyjOGnHqgv4HXA/4b/5fsJdW9+q/xRxHtvkQ9gas7zM7G6gFnihgadp9uUVr9Q40zxbj06UmWUAfwRud87trzd7OeEukvLI/qI/AwNboKzjvS9+Lq8UYBJwV5zZfi2vE9Hsyy4hQ8E5N+4kHlYE5MWM9wGK67XZSXjTNSnyDS9em2ap0cySgGuAkcd4juLI71Ize51w10WTPuQau+zM7AlgbpxZjVmOzV5XZAfalcClLtKZGuc5mn15xdGYv/9wm6LI+9yJo7sGmp2ZJRMOhBecc3+qPz82JJxz88zst2aW45zz9Bo/jXhfPFmnGukKYLlzbkf9GX4trxg7zKync64k0p1WGqdNEeF9H4f1Ibw/9aS1p+6jOcCUyJEh/Qgn/uLYBpEPm0XAtZFJNwANbXk01TjgE+dcUbyZZtbRzDIPDxPe2bomXtvmUq8f9+oGXm8JMNDCR2mlEN70nuNxXROAHwGTnHMVDbRpqeXVmL9/DuF1B8Lr0tsNBVlzieyzeApY55x7pIE2PQ7v2zCz8wj//+/yuK7GvC9zgG9GjkIaDew73G3SAhrcWvdjedUTux419Fn0JjDezLpEunvHR6advJbYs96SP4Q/zIqAKmAH8GbMvLsJHzmyHrgiZvo8oFdkuD/hsCgEXgVSParzWeCWetN6AfNi6lgV+Skg3I3i9bL7A7Aa+DiyQvasX1dkfCLho1s+a6G6Cgn3m66M/DxWv66WXF7x/n7gPsKhBZAWWXcKI+tS/xZYRl8i3G3wccxymgjccng9A26NLJtVhHfYX9ACdcV9X+rVZcCjkeW5mpijBj2urQPhD/lOMdN8WV6Eg6kEqIl8fn2b8H6ovwMbIr+7RtrmA0/GPPZbkXWtELixqbXojGYREYlqT91HIiJyHAoFERGJUiiIiEiUQkFERKIUCiIiEqVQEGkCC1+d9HMz6xoZ7xIZP9Xv2kROhkJBpAmcc1sJXx5kZmTSTGCW8/HiaiJNofMURJoocnmJZcDTwE3AOS58BVWRhJOQ1z4SaU2cczVmdicwHxivQJBEpu4jkeZxBeHLFJzpdyEiTaFQEGkiMxsOXEb4rmH/q4VvECPSrBQKIk0QuYrm7wjfv2AL8BDwsL9ViZw8hYJI09wEbHHOLYyM/xY4w8zG+FiTyEnT0UciIhKlLQUREYlSKIiISJRCQUREohQKIiISpVAQEZEohYKIiEQpFEREJEqhICIiUf8fnS4lzmbbqYcAAAAASUVORK5CYII=\n", "text/plain": [ "
" ] }, "metadata": { "needs_background": "light" }, "output_type": "display_data" } ], "source": [ "df=pd.DataFrame({'X':np.linspace(-10,10,100)})\n", "df['Y']=sigmoid(df['X'])\n", "sns.lineplot(x='X',y='Y',data=df,label='SIGMOID CURVE');" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "###### Model Representation\n", "\n", "The hypothesis in Linear Regression (i.e. $h_\\theta(x)=\\theta^Tx$) cannot be used for the classification problems because the linear hypothesis output ranges from $-\\infty$ to $\\infty$, whereas $y$ (i.e. target variable) in classification problems are either 0 or 1. Hence the range of hypothesis output in Logistic Regression should be:\n", "\n", "$$ 0\\le h_\\theta(x) \\le 1 $$\n", "\n", "In order to keep $h_\\theta(x)$ in the range mentioned above, Sigmoid Function comes to rescue:\n", "\n", "$$ h_\\theta(x)=g(\\theta^Tx) $$\n", "$$ g(z)=\\frac{1}{1+e^{-z}} $$\n", "\n" ] }, { "cell_type": "code", "execution_count": 7, "metadata": {}, "outputs": [], "source": [ "def predict(X,theta):\n", " '''Predicts by applying logistic function on linear model'''\n", " z=np.dot(X,theta.T)\n", " h=sigmoid(z)\n", " return h" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "##### 1.2.2    Cost function\n", "We learnt about the cost function $J(\\theta)$ in **Linear Regression**, the cost function represents the optimization objective i.e. we create a cost function and minimize it to develope an accurate model with minimum error.\n", "$$ J(\\theta)=\\frac{1}{2m}\\sum_{i=0}^m(h_\\theta(x^{(i)})-y^{(i)})^2 $$ \n", "On replacing $h_\\theta(x)$ with $\\frac{1}{1+e^{(-\\theta^Tx)}}$ in the above equation,it becomes '**Non Convex**' which does not guarrante to converge and find the global minimum. So we need a new cost function for Logistic Regression. \n", "\n", "\n", "\n", "The figure in left is a '**Convex**' function and has only one minimum called global minimum and will always converage whereas the figure in right is a '**Non Convex**' function and has mutiple minimums also called local minimums. And there is no guarrante that the non convex function will always converge.\n", "\n", "For **Logistic Regression** cost function is defined as: \n", "\n", "$$\\begin{equation}\n", " Cost(h_\\theta(x),y)=\\begin{cases}\n", " -log(h_\\theta(x)) & \\text{if }y=1 \\\\\n", " -log(1-h_\\theta(x)) & \\text{if }y=0\n", " \\end{cases}\n", "\\end{equation}$$\n", "\n", "Let's try to understand the cost function. \n", "\n", "For $y=1$\n", "- $J(\\theta)\\approx\\infty$ if $h_\\theta(x)\\approx0$ (Penalty for wrong prediction)\n", "- $J(\\theta)\\approx0$ if $h_\\theta(x)\\approx1$ \n", "\n", "For $y=0$\n", "- $J(\\theta)\\approx\\infty$ if $h_\\theta(x)\\approx1$ (Penalty for wrong prediction)\n", "- $J(\\theta)\\approx0$ if $h_\\theta(x)\\approx0$ " ] }, { "cell_type": "code", "execution_count": 8, "metadata": {}, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXwAAAEKCAYAAAARnO4WAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvOIA7rQAAIABJREFUeJzt3Xd4VNed//H3Ue+9o46EQPSOwdjGuPdu4hI7seNssm7xJvHPm03fbOIUb+xN4rjFnRiXuOFu44LBFNFBiKYCEqijjtrM+f1xRiDbgEagmTvl+3qeeUbSjLjfC/bnnjn3FKW1RgghhO8LsLoAIYQQ7iGBL4QQfkICXwgh/IQEvhBC+AkJfCGE8BMS+EII4Sck8IUQwk9I4AshhJ+QwBdCCD8RZHUBgyUlJenc3FyryxBCCK+ybt26Rq118lDv86jAz83NpaSkxOoyhBDCqyilqpx5n3TpCCGEn5DAF0IIPyGBL4QQfsKj+vCFGI6+vj6qq6vp7u62upQRFxYWRmZmJsHBwVaXInyIBL7wWtXV1URHR5Obm4tSyupyRozWmqamJqqrq8nLy7O6HOFDpEtHeK3u7m4SExN9KuwBlFIkJib65CcXYS0JfOHVfC3sB/jqeQlr+Ubg93TAJ/fD3tVWVyKEEMP2+sYanl3l1FD6k+IbgR8QCJ/8D1Qut7oSIYZUUVHB7NmzKSws5Nprr6W3t9fqkoTFXllfw8sl+1x+HN8I/OBwCIuD9gNWVyLEkO69915+8IMfsGvXLuLj43niiSesLklYrK61m5SYMJcfxzcCHyAmA9ok8IX7/PSnP+XBBx88/P1PfvITHnrooeP+jtaaZcuWcdVVVwFw00038dprr7m0TuH5atu6SXND4PvOsMzodGnh+7FfvrmN0v1tI/pnFmfE8POLxx/z9VtuuYUrrriCu+66C7vdzgsvvMCyZcuYMmXKUd+/ePFiUlJSiIuLIyjI/K+XmZlJTU3NiNYtvEt3n43WQ32kxUrgOy86HepLra5C+JHc3FwSExPZsGEDdXV1TJ06lZycHDZu3HjM32loaPjaz2REjn+razPDb1OiQ11+LN8J/Jh06KgDu83cxBV+5XgtcVe69dZbeeqpp6itreXb3/427e3tzJ8//6jvXbx4MePGjaOlpYX+/n6CgoKorq4mIyPDzVULT1LX1gMgLfxhiU4DbYeOehP+QrjB5Zdfzs9+9jP6+vpYvHgxgYGBx23hAyxYsICXX36ZRYsW8fTTT3PppZe6qVrhiWodLfxUuWk7DNGOVlL7fmvrEH4lJCSEBQsWcM011xAY6Nwny/vvv58HHniAgoICmpqauOWWW1xcpfBkda3uC3zfaeEPtOrba62tQ/gVu93OqlWreOmll5z+nfz8fNasWePCqoQ3qWvrJjw4kJgw18exD7XwHYHfJi184R6lpaUUFBSwcOFCCgsLrS5HeKnatm5SY0LdcvPed1r4kcmgAqWFL9ymuLiY8vJyq8sQXq6+rcct3TngSy38gECISpWx+EIIr2Ja+BL4wxcjk6+EEN5Da01dW7dbhmSCrwV+dLosryCE8Bqth/ro6bdLC/+ERKfLsEwhhNc4Mgbf9bNswdcCPyYduluht8vqSoQ4pr/85S8UFBSglKKxsdHqcoSFDs+ylRb+CRgYmin9+MKDzZs3jw8//JCcnByrSxEWc+ekK/DZwJehmcL1TmR5ZICpU6eSm5vrwsqEtzi8cJqbunR8Zxw+mDXxQVr4/uid/we1W0b2z0ybCOf/7pgvn8jyyMXFxSNbo/BqtW3dJESGEBrkngUffSvwo9PMs8y2FW5wIssjCzFYXVu3W5ZFHuBbgR8aA8GR0qXjj47TEnel4S6PLC18MVhdW4/bxuCDrwW+UqaVL0MzhZucyPLIQgyobeumOD3GbcfzrZu2YPrxpYUv3ORElkd+6KGHyMzMpLq6mkmTJnHrrbe6uErhifptdho7ekj1tRa+UioQKAFqtNYXufRg0emwb7VLDyHEgBNZHvnOO+/kzjvvdGFVwhs0dPSgtfvG4IP7Wvh3AdvdcqToNNPC19othxP+S5ZHFiejttW9s2zBDS18pVQmcCHwG+AeVx+PmAyw9cChgxCR4PLDCf8lyyOLkzEwy9Zdk67APS38PwM/BuxHe1EpdZtSqkQpVdLQ0HDyR5OhmX5F++gnOV89L3FEnRv3sh3g0sBXSl0E1Gut1x3rPVrrR7XWM7TWM5KTk0/+oIf3tpUbt74uLCyMpqYmnwtHrTVNTU2EhbkvCIT71bV1ExyoSIwMcdsxXd2lMw+4RCl1ARAGxCilntNa3+CyIx7e21Za+L5uYKTLiHwy9DBhYWFkZmZaXYZwodq2blKiwwgIcP3WhgNcGvha6/uA+wCUUmcAP3Rp2ANESZeOvwgODiYvL8/qMoQ4IXVt3W5bQ2eA743DDwqB2Gxo3GV1JUIIcUx76jvJS4x06zHdNtNWa/0J8IlbDpZaDHXb3HIoIYQYrpauXmrbuilKi3brcX2vhQ+QOh6adkF/r9WVCCHE15TVtgMw1o3LKoCvBn5KMdj7oXGn1ZUIIcTX7BgIfGnhj4DU8eZZunWEEB6orLaNuIhgty6NDL4a+IkFEBgC9RL4QgjPU1bbzti0aJRy35BM8NXADwyGpCKoK7W6EiGE+BK7XbOztp2xae7tvwdfDXyQkTpCCI9UffAQnb02t4/QAV8O/JRiM9v20EGrKxFCiMPKatsAJPBHVOoE8yzdOkIIDzIwQqcoVQJ/5KQ69g6Vbh0hhAcpq20nOyGCyFD37zDru4EfnQ5hcTJSRwjhUcpq2yzpzgFfDnylTLeOdOkIITxEd5+NisZOxkngu0BqMdSXgv2oe68IIYRb7a7vwK6hyIIhmeDrgZ9SDL0d0LrX6kqEEOLwGjrSpeMKh5dYkG4dIYT1dtS2ERIUQG5ihCXH9+3ATxlnnmWkjhDCA5TVtjMmNYqgQGui17cDPzQaEkZDzTG31BVCCLew2zVbalopdvOSyIP5duAD5MyFvSvBbrO6EiGEHyurbaelq4/ZeYmW1eD7gZ87H7pbpVtHCGGp1RVNAMzOT7CsBj8I/HnmufJza+sQQvi1VeVNZCWEkxlvzQ1b8IfAj82E+DwJfCGEZex2zeqKZuZY2J0D/hD4ALmnQtUKmYAlhLDEjjrTfz8nXwLf9XJPhe4WqNtqdSVCCD+0qtz6/nvwl8DPcfTjV62wtg4hhF9aXd5MZry1/ffgL4EflwXxudKPL4RwO9N/32R5dw74S+CD6dap/Fz68YUQbrWzvp2DHtB/D/4U+DmOfnxZH18I4Uar9jj67/Os7b8Hfwp8GY8vhLDAqvJmRsWFk5Vgbf89+FPgx2VDXA6Uf2J1JUIIP9Fvs7PKQ/rvwZ8CH6DofNjzMfR0WF2JEMIPrKlspqWrj7PGpVhdCuBvgT/uErD1wO4PrK5ECOEH3ttaS1hwAKcXJVtdCuBvgZ89ByKSoPQNqysRQvg4u13z7rZaTh+TTERIkNXlAP4W+AGBMPZC2PU+9HVbXY0QwodtrG6hrq2H8yakWV3KYf4V+GC6dXo75OatEMKl3ttaS1CA4syxqVaXcpj/BX7eaRAaA9vftLoSIYSP0tp058wtSCI2PNjqcg5zaeArpcKUUmuUUpuUUtuUUr905fGcEhQCY86DHW+Brd/qaoQQPqistp2qpi7OG+853Tng+hZ+D3Cm1noyMAU4Tyk1x8XHHFrxJXDooCymJoRwiXe31qIUnDPec7pzwMWBr42BQe/Bjod25TGdMnohBIXDdhmtI4QYee9urWVmbgJJUaFWl/IlLu/DV0oFKqU2AvXAB1rr1V95/TalVIlSqqShocHV5RghEVB0Hmz9F/T3uOeYQgi/sLWmlR117Vw4Md3qUr7G5YGvtbZpracAmcAspdSEr7z+qNZ6htZ6RnKyGycnTL0RDjVD2VL3HVMI4fOWrN1HSFAAl00ZZXUpX+O2UTpa6xbgE+A8dx3zuPIXQGw2rH/G6kqEED7iUK+N1zbWcMGENGIjPGd0zgBXj9JJVkrFOb4OB84Cylx5TKcFBMC0G814/OYKq6sRQviAt7ccoL27n0Wzsq0u5ahc3cJPBz5WSm0G1mL68D2nD2XK9aACYMOzVlcihPABL6zdS15SpEesfX80rh6ls1lrPVVrPUlrPUFr/StXHm/YYkdBwdmw4XkZky+EOCm76ztYW3mQa2dmoZSyupyj8r+Ztl81/SboqDXr6wghxAlasnYvQQGKK6Z53s3aARL4hedAVCqse8rqSoQQXqqn38Yr62tYOC6FlOgwq8s5Jgn8wGCYfrNp4TfstLoaIYQXenV9Dc2dvdw4J9fqUo5LAh9g1m0QFAYrH7S6EiGEl7HZNY9+Vs74jBjmFXjGVobHIoEPEJkEU2+ATUugbb/V1QghvMgHpbWUN3byb6eP9tibtQMk8AfMvR20DVb9zepKhBBeQmvNw5+Wk50QwfketNHJsUjgD4jPhfGXQ8lTcKjF6mqEEF5gVXkzm/a18J3T8gkK9Pw49fwK3WneXdDbDiVPWF2JEMIL/P3TPSRFhXD19EyrS3GKBP5g6ZNh9Jnwxd+gp2Po9wsh/NaW6lY+3dnAzXNzCQsOtLocp0jgf9UZ/wldjdKXL4Q4rvvfLSM+Ipib5uZaXYrTTirwlVLfGqlCPEbWTBh7Eax4EDobra5GCOGBlu9q4PPdjdx+ZiHRYZ63KuaxnGwL3/o9al1h4c+grws++6PVlQghPIzdrrn/3TJGxYVzwxzPXBXzWIKGeoNjpcujvgR41oaNIyW5yIzLL3kC5nwP4nOsrkgI4SHe2nKArTVt/OnqyYQGeUff/YAhAx8T6ucCB7/ycwWsHPGKPMUZ98HmF+Hj38AVj1pdjRDCA/TZ7Pzp/R0UpUZz2VTPXSTtWJzp0lkKRGmtq77yqMTsYOWbYjJg9r/B5iWwb63V1QghPMCTKyqobOri3vOLCAzw7Fm1RzNk4Gutb9Faf36M164b+FopFT+ShXmE034I0Rnw1j2yXr4Qfu5A6yH+/OEuFo5NYUFRitXlnJCRHJb50Qj+WZ4hNBrO+x+o3SyTsYTwc79eWorNrvnFJeM9fs2cYxnJwPfOv4GhFF9mJmMt+29or7O6GiGEBT7d2cDbW2q5fUEBWQkRVpdzwkYy8PUI/lmeQym44I/Q3w3v/5fV1Qgh3Ky7z8bPX99KXlIkt52eb3U5J0Vm2jojcTTMuxu2vAg7ZStEIfzJgx/torKpi19dOt7rhmF+lXTpOOu0H0JKMbxxB3Q1W12NEMIN1u89yCOf7uHaGVnML0y2upyT5nTgK6WeHeJnC0ekIk8VFAqXPWzW2XnnXqurEUK42KFeGz98cRPpseH810XjrC5nRAynhT9+8DdKqUBg+sD3Wmvfb/ZmTIHTfmy6dkpft7oaIYQL/f69MsobO/nDVZO8ar2c4xky8JVS9yml2oFJSqk2x6MdqAf8L/Xm3wPpU2DpD6C91upqhBAusGJ3I0+uqOSmU3KYW5BkdTkjxpmJV7/VWkcDf9Baxzge0VrrRK31fW6o0bMEBsPlj0BvF7xyK9htVlckhBhB9e3d3PXCRkYnR3Lv+WOtLmdEDadLZ6lSKhJAKXWDUuoBpZR/riqWMhYu/BNULodP77e6GiHECLHZNXe/sJGOnj7+dv10IkKcWW7Mewwn8B8GupRSk4EfA1XAMy6pyhtMvR6mXA+f/h72fGx1NUKIEfB/y3axck8Tv7p0AkVp0VaXM+KGE/j9WmsNXAo8qLV+EPC9v5HhuOAPZinlf30H2vZbXY0Q4iR8vquRBz/axRXTRnnNHrXDNZzAb1dK3QfcCLzlGKXjG7euT1RIJFz9NPQdgheuM89CCK9T2djJvy9eT2FKFL++dILXrpUzlOEE/rVAD/BtrXUtMAr4g0uq8iYpY+GKx2D/RjMpS/vmChNC+Kr27j5ufaaEAAWPf3MmkaG+1W8/mNOB7wj554FYpdRFQLfW2n/78AcbewEs/ClseQk+/1+rqxFCOMlm19z1wkYqGzv52/XTyU703oXRnDGcmbbXAGuAq4FrgNVKqatcVZjXOfUemHAVfPQrKH3D6mqEEE747dvbWVZWz88vGc8poxOtLsflhvPZ5SfATK11PYBSKhn4EHjZFYV5HaXg0r9Ay14zPj/yNciZa3VVQohjeHx5OY9/XsHNc3O5cY5/jDAfTh9+wEDYOzQN8/d9X3A4XLcE4rLhn4ugfrvVFQkhjuL1jTX891vbuWBiGj+9qNjqctxmOIH9rlLqPaXUzUqpm4G3gLeP9wtKqSyl1MdKqe1KqW1KqbtOplivEJEAN7wCQWHw3JXQWm11RUKIQVbsbuSHL21iVl4CD1wzxSv3pj1RzqylU6CUmqe1/hHwCDAJmAx8ATw6xK/3A/+htR4HzAH+XSnl+5fT+By4/mXoaYdnLpWdsoTwECWVzdz6dAn5SVE8duMMwoK9e3374XKmhf9noB1Aa/0vrfU9WusfYFr3fz7eL2qtD2it1zu+bge2Y4Zz+r70SSb02w6Y0O9ssroiIfzapn0t3PzkWtJjw3ju1tnERvjfNCJnAj9Xa735qz/UWpcAuc4eSCmVC0wFVn/l57cppUqUUiUNDQ3O/nHeIXu26dM/WAHPXgqHDlpdkRB+qXR/G9/8xxriI4N5/juzSY4OtbokSzgT+GHHeS3cmYMopaKAV4C7tdZtg1/TWj+qtZ6htZ6RnOz9O8p8Td58WPQ8NOyApy+GzkarKxLCr2yubuG6x1cRGRLI4lvnkB7rVGz5JGcCf61S6jtf/aFS6hZg3VC/rJQKxoT981rrfw2/RB9QcBZ845/QuAueulDW0RfCTdZVNXP9Y6uJCg1iyXdPISvBtydWDUXpIZYCUEqlAq8CvRwJ+BlACHC5YwbusX5XAU8DzVrru4cqZsaMGbqkpMTJ0r1QxXJYfC1Ep8FNb0Csby7QJIQn+GJPE7c8vZbUmDAWf2e2T7fslVLrtNYzhnqfMxug1Gmt5wK/BCodj19qrU85Xtg7zMMstnamUmqj43HBkNX7qrz5cOOr0NkAT5wj4/SFcJF3thzgpifXMCounCW3+Xc3zmBDtvDdyedb+ANqt5gx+v095qZu9hyrKxLCZzy7qoqfvb6VKVlx/OOmmcRHhlhdksuNWAtfuEDaRLjlfYhINEM2ty+1uiIhvJ7Wmj+9v4OfvraVM4tSWHzrHL8I++GQwLdKfK4J/dTxsOQGWPGQLK0sxAnq7rNxxz838H/LdnPNjEweuXE64SH+NanKGRL4VopMgpuWQvEl8MFP4c07wdZndVVCeJWG9h6+8dgqlm4+wI/PK+L+KycRFCjRdjS+u9K/twiJgKuego9/A8v/CM0VcPVT5mIghDiurTWtfPfZdTR19vDw9dM4f2K61SV5NLkMeoKAALOByuWPQvVaeOR02L/B6qqE8GivbajhyodXYteaF797ioS9EyTwPcnka+Hb75m19f9xHmxcbHVFQnicPpudXy8t5e4lG5mSFcebd5zKpMw4q8vyChL4niZjCtz2CWTOhNe+B6/fLpujC+Gwv+UQ1z7yBU84Ni557tbZJEX557o4J0L68D1RZBLc+Bp88lvTr1+zHq55GpIKra5MCMt8vKOee5ZspM+m+ct1U7loUobVJXkdaeF7qsAg069//SvQfgAeOQ3WPytDN4Xf6em38eulpXzrybWkxYbz5h2nStifIAl8T1d4FnxvBYyaDm/cDi/dLMssC7+xu76dy/66kic+r+CmU3J49ftzyUuKtLosryVdOt4gJgO++TqsfAiW/bcZyXPpX2D0mVZXJoRL2O2aZ76o5HfvlhEREsQTN81g4bhUq8vyetLC9xYBgXDqD+CWDyAkEp69HN76D+jttLoyIUZU9cEurn98Nb94s5RT8hN59675EvYjRFr43mbUNPjuZ/DRr2HV32D3R3DJ/5mVOIXwYna75p9r9/Lbt8vQWvO7KyZy7cwszCrrYiRIC98bBYfDef8DNzsWXXv6InjzbuhutbYuIU5QZWMn1z2+ip+8upVJmbG8e/dpLJqVLWE/wqSF781yT4XvrTTLMqz6G+x8D86/H8ZdbCZvCeHh+mx2HltezkMf7SI4IEBa9S4mge/tQiLg3N/A+CvM4msv3ghjzoML/gBx2VZXJ8QxlVQ285+vbmFnXQfnjk/ll5dMIC32eFtoi5Mlge8rMqebGbqrHjYTtv46G+b/B8y9A4JkJqLwHI0dPdz/ThkvratmVFw4j39zBmcVy01Zd5Adr3xRy1549z4oWwoJ+XD+76HwbKurEn6u32bnuVVV/OmDnXT32fj2qXnceWYhkaHS7jxZzu54JX/TviguGxY9D7s/hHfuheevgsJz4JzfQPIYq6sTfmj5rgZ+vbSUnXUdzC9M4ucXj6cgJcrqsvyOBL4vKzjL3NRd/Xf47I/w8Ckw8ztw+o8hIsHq6oQf2F3fwW/f3s5HZfVkJ0Tw9xumce74NLkpaxHp0vEXHQ1mNM/6pyE02vTvz/ouBMtNMjHy6tu7efDDXbywdh/hwYHccWYBN8/LJTRIth10BWe7dCTw/U3dNvjwF7DrfYjNggX/CZOuNTN5hThJ7d19PL68gseWl9Pbb+f62dncsbBQljB2MQl8cXzln8IHP4MDGyF5LJz5XzD2Ihm/L05Id5+N51ZV8dePd3Owq4/zJ6Txo3OLyE+Wfnp3kJu24vjyTzfDOLe/YRZkW3IDpE+BBT8xI3ok+IUTevptvLh2H3/9eA+1bd3ML0ziR+cWyQ5UHkoC358pBcWXQtGFsPkF+PT3sPhqGDUDzrgPChZK8Iuj6um38cq6Gv768W5qWg4xIyeeB66dzNzRSVaXJo5DunTEEbY+2Pg8fPoHaKuGjKlw+r1m5q4Ev8B03bxYso+HP9nDgdZupmTFcc/ZY5hfmCQjbywkffjixPX3wqbFsPwBaKmC1Ilw6t0w/nK5ueunOnr6eX5VFY9/XkFDew8zcuK5c2GhBL2HkMAXJ8/WB1tegs//Fxp3mlm7c++Eyd+Q4Zx+orGjh2dWVvLUykrauvs5tSCJ7y8YzSn5iRL0HkQCX4wcu90s0/D5A7B/A0Qmw+zvwoxbZAKXj6po7OSx5eW8sq6aXpudc4pT+f4ZBUzOkpuxnkgCX4w8raFyOax4CHZ/AMERMOU6mPN9SBxtdXXiJGmtWVPRzOOfV/Dh9jqCAwO4ctoobp2fz2gZXunRZFimGHlKQd5p5lG3Db74G6x/BtY+YW7szv4u5J8hN3i9TE+/jaWbDvDkygq21rQRHxHMHQsKuOGUHFKipevOl0gLX5yc9jpY+ziU/AO6Gs0krlm3mdm7odIq9GS1rd0sXl3F4jV7aezopSAlim/Ny+XKaZmEBcvNeW8iXTrCvfq6YdursPphOLAJQmPMzd2Zt8oKnR5Ea80X5U08+0UV75fWYdeahWNTuHluHvMK5East5LAF9bQGqrXwprHoPQ1sPVC7nyYfrPZelE2Y7FES1cvL6+rZvGavZQ3dBIXEcy1M7K4fnYO2YkRVpcnTpJHBL5S6h/ARUC91nrCUO+XwPcxHQ2w8TkoedKM549INK3+6TdDUqHV1fk8rTWrK5p5Yc1e3t5aS2+/nWnZcVw3O4eLJqVLt40P8ZTAPw3oAJ6RwPdjdjvsWQbrnoSd74K9H3LmwdQbzNIOIZFWV+hT6tu6+deGGl5cu4/yxk6iQ4O4dGoG183KoTgjxuryhAt4ROA7CskFlkrgC8Dc5N202IzuaS6HkGiYcDlMuR6yZssInxPU229nWVkdL6+r5uMdDdjsmhk58Syalc2FE9MJD5HWvC/zmsBXSt0G3AaQnZ09vaqqyqX1CA+hNez9AjY8B9teg75OSBhtunwmXQPxOVZX6PG01myubuVf66t5Y9N+Dnb1kRIdypXTM7l6eqYsTexHvCbwB5MWvp/q6TDLNG9cbCZ2genymXSN6fIJj7e2Pg+zr7mL1zfW8K8NNZQ3dBISFMA5xalcNT2TUwuSCAoMsLpE4WYS+MI7HayCLS/CpiXQtAsCQ8wG7BOvhjHnQnC41RVaoqmjh7e3HOC1jftZV3UQgFm5CVw+bRQXTEwnNjzY4gqFlWSmrfBO8Tlw2o9g/g9h/3rY8jJsfcWs5RMSDWMvhAlXwugFEOjbIdfW3cf72+p4Y9N+VuxuxGbXjEmN4kfnFnHJ5AyyEmQ4pRgeV4/S+SdwBpAE1AE/11o/caz3SwtfHJXdBhWfmeDf/gZ0t5punrEXwYQrIPc0CPSNtktHTz8fba/jzU0H+GxnA702O5nx4Vw8OYOLJ2UwLj1aJkeJr/GYLp3hkMAXQ+rvhT0fmVm9ZW9DbzuEJ5iWf/FlZp2foBCrqxyWtu4+Ptpex9tbavl0ZwO9/XbSYsK4cFI6F05KZ2pWnIS8OC7p0hG+KSgEis43j75u2P0hlL5uRvpseBbCYmHM+WZW7+gzIcQzuz2aOnr4oLSOd7fVsmJ3I302TXpsGNfPzuaCielMz44nIEBCXowsCXzhvYLDYNxF5tHfYyZ3bX8Tyt4ye/QGR5jQH3uRueFr8dr9+5q7eG9bLe9vq6Okqhm7hqyEcL41L49zx6cxNStOQl64lAS+8A1BoUda/rY+qPzc3Ogte8s8q0DImQtFF5j3JOS5vCS7XbN1fysflNbxQWkdZbXtAIxNi+b2Mws5pziV8Rkx0l0j3Eb68IVvs9vNLl073jJ9/g3bzc+Tx5o1/IvOh8yZI7ZXb1dvPyt2N/HR9jqWldVT395DgIIZuQmcU5zK2cWp5CTKUhJiZMlNWyGOprkcdrwLO9+BqpVmXZ/weCg423T7jD5z2F0/+5q7+HhHPcvK6lm5p4nefjtRoUGcPiaZheNSWFCUQnykd91IFt5FAl+IoRxqMf3+u943j64mUAEwaoaZ7FWwENKnQMCXZ65299lYW9nMJzsa+GRHPXsaOgHIS4pkQVEKZ45NYVZeAiFBMuNVuIcEvhDDYbeZrp+B8N+/wfw8IgldcCYNKaeyrG8C71fZ+WJPE4f6bIQEBjA7P4EFRSmcUZQsa9cIy0jgC3HH+MW1AAAPqElEQVQS2hr2U7HmTey7PiS3ZRXxtAGwMyCfuqS5RI0/m6KZZxMRIf3xwnoS+EIMQ0+/jfVVLazY3cjnuxvZXN2CXUNUaBCn5MVzWVoDc9lE/P7lUL3G9P0HhZuRP/lnmEfqhK91/wjhDjLxSojjsNk12/a3smJ3Eyv3NLK2spnuPjuBAYopWXHcvqCA+WOSmZIVR/Dh1ScvNE897WbY556PofwT+OCn5ucRiWY7x/zTIe90SMiX9f2FR5HAF37BbteU1bazqryJlXuaWF3RRHt3PwBjUqNYNDObeQVJzM5PICZsiEXZQqOPjPkHaNsP5Z9CxafmufQ18/OYUWaph9z5kHuqrPEvLCddOsInDQ741RVNrK5opqWrD4CcxAjmjk5kTn4ip4xOJCU6bOQOrDU07TaLvVV8Ztb372oyr8Vlm/DPmScXADGipA9f+JU+m51t+9tYU9HEmopm1lYepPWQCfjshAhm5yVwiiPkM+LcuKa+3Q4NZaYLqPIzqFwBh5rNa7FZ5h5AzlxzEUgskC4gcUIk8IVP6+rtZ+PeFtZWHmRNZRMb9rbQ1WsDID8pkll5CczKS2B2fiKj3BnwQxl8AahaYSZ/ddab1yKTIXsOZM81z2mTfGbZZ+FactNW+JT69m7WVR6kpOogJZXNbNvfRr9doxSMTYvh6umZzMxLYFZuAikxI9hFM9ICAiC12Dxm33akC6hqpXnsXWkWgAMIjoTMGZB9CmTPNktAhEZbW7/watLCFx6n32ZnR1076/e2sL7qIOuqDrK3uQuA0KAAJmfFMTM3nhm5CUzLjve97f1aa2DfKqj6wjzXbQNtN7OAU8dD1mzzyJwJ8bnSDSSkS0d4j6aOHjbua2HD3hbW7z3Ipn0tdDq6Z5KjQ5mRE8+07Him58YzISPW/5Ys6G6D6rWwbw3sWw3VJWbjF4DIFMiaZT4JZM6EjKkQIpPB/I106QiP1NNvo3R/Gxv3tRx+VDWZ1ntggKI4PYarpmcyzRHymfHhsnxwWIxZ16dgofneboP6UnMBGLgQlC01r6lA0100ynEBGDUdksbIhDABSAtfuJDdrqls6mRTdQub9rWyYV8L2/e30WuzA5AaE8rUrHimZMcxLTueiaNiCQ8ZmWWK/U5nE9SUmPCvKYGa9dBjloMgNMa0/EdNP/KISbe2XjGipEtHuJXWmv2t3WypbmFTdSubq1vYXN16eHJTREggE0bFMjUrjilZcUzOinPv8Eh/Y7dD406oWWcuANUl5lOB3fx7EJ0OGdNg1FTznDHV8h3BxImTLh3hMlpr6tp62FzdwtaaVrY4Ho0dvQAEBSjGpkdz8eQMJmfGMiUrnoKUKAJl+z73CQiAlLHmMfV687O+Q1C7xXERWA/715uNYQbE55rgT58CGVMgfbLZK0D4DAl8cVxaa2paDrG1po1t+02wbx0U7gEKxqRGc0ZRCpMzY5mYGcfYtGjCgqVrxuMEh5sbvFmzjvzsUAsc2GTCv2a9uRhse/XI6/G5X74ApE+RTwJeTAJfHGazayoaO9m2v5XS/W1s29/G1v2th5ckCAxQFKZEcUZRChNHxTJhVCzF6THS7+7NwuPMYm/5px/5WVez2Q/gwEbYv9FcDAbWBwIzQzhtkuMCMMl8HZMhw0O9gAS+nzrUa2NHXfvhcC890EbZgXYO9ZnhkCGBARSlRXPe+DTGj4plQkYM49JjpOXuDyISvjwqCMxFoHazuQDUboYDm2HH24DjHmB4AqRNPHIBSJsIiYUyU9jDyL+Gjxvob99+wIT6wHNlYyd2x/+r0aFBjMuIYdGsLIrTYxifEUthatSgZYGF34tIOLLu/4CeDjMprHaz6Raq3QKrHwGb6e4jMBRSxkHaBEid6HgeL/cFLCSB70MO9drYWddOWW0bZbXtbD9gnge6ZABGxYVTnBHDRZMyKE6PZnxGrIx1FycmNMos+ZA9+8jPbH3QuMtcBGq3QN1W2PEObHjuyHtiMk3wp02AlGKzcUxigXwacAP5G/ZC/TY7Vc1d7Khtp6y2nR21beyobaequYuBUbbhwYEUpUVz/oQ0xqaZ7pix6dFDr/UuxMkIDD6yVtDkReZnWkN7rfk0ULfF8bwN9nx0ZJhoYAgkFZnfSyk2F4SUcWZPAWmMjBgJfA82MEJmZ107O+s62Fnbzo66dnbVd9DbbyYvBSjITYxkXHoMl00dxdi0GMamRZOdEEGADIMUnkApM9ErJh0Kzzry8/4eM1egrtR8EqgvhYrlsHnJkfeExprgTxlnLgQDX0cmuf88fIBMvPIAA5OWdtW1s6uug52OUN9d30FHT//h96XHhjEmNZoxqVEUpcVQlBpNQUqUjJIRvqWr2SwhXV9qLgb126F+G3S3HnlPZDIkjzWPlLGQPM58HZloXd0WkolXHshu11QfPMSu+nZ213ewy/HYXdd+eLEwgKSoUMakRnHV9EwKU6NMyKdEExsh3THCD0QkHNkYZoDW0H7AhH9D2ZHnTS8cWUgOBl0Iisxz0hjzHJUiXUNI4LtET7+NysYudjta6XsazHN5YwfdffbD70uJDqUgJYqrZ2RRkGKCvSAlioTIEAurF8IDKWXG+sdkfHm4qNbQVuO4CJRBw3Zo2AmbX4KeQZ8IwuLMRSCp0NwrGPg6LgcC/OcTsgT+SWju7KW8wQR6eUPn4WDf29x1eMgjmJExBSlRzB2dSGFqFAUpURQkS4tdiJOmFMRmmkfBoPsDA58IGnaY+wQNZeZCsPO9L48YCgozI4SSCs2ngaQx5vvEAjMKycdI4A+ht9/O3uZOyhs6KW/sZE99B+WNnZQ3dHBw0HDHkKAA8pMiKc6I4ZLJGYxOiWJ0chT5yZFEhMhfsxBuNfgTwegFX36tq9lcBAYeDTvNPILS181GMwNiRjkuBmPMBWHgwhCT6bXLTUsSYfrWa9u6qWg0oV7R0ElFYwcVjZ3sO3gI26DmenJ0KPlJkZw3IZ3RyZGHQz0zPkIWBxPCG0QkOPYOnvPln/f3QNMeaNrluBjsMo/NS44sNQ3mU0HCaEgcfeTTwMAjIsGj7xW4PPCVUucBDwKBwONa69+5+phHo7WmoaOHysYuKhs7qWgywV7ZZB6D+9bDgwPJS4pk/KhYLp6cQX5yJPlJUeQlR8o4diF8VVDokTkEg2kNHfVm7+Emx0WgaY+5cbzj7SNzCQDCYk3wD1wQDl8YRpvXLObSwFdKBQJ/Bc4GqoG1Sqk3tNalrjie1pqG9h4qm7qobOqkqqmTysYuKhrN14NHwgQHKrISIshLjOTUgiRykyIPB3tqTKjMPBVCGEpBdKp55M778mu2fmipOvLJoGmPuTDs/QK2vMThtYYAIhIHXQjyv/wIj3PLqbi6hT8L2K21LgdQSr0AXAqMaOAf7OzlG4+tYm9zF12DQj0oQJEZH05uUiSz8hLIS4okJzGC/KQoMuLCCJK1YoQQJyMw6EgLnnO+/FrfIThYaS4CzXscz+VQ8Rls+ueX3xvuGIq66HmXluvqwB8F7Bv0fTUwe/AblFK3AbcBZGdnn9BBYsODyUqIYO7oJHKTIshOiCAvKZJRceES6kIIawSHH5kZ/FV9h6C5wlwAmsvhYIVbNp93deAfrV/kS1N7tdaPAo+CmWl7IgcJCFA89s0hJ5kJIYRnCA4/+v0CF3N187cayBr0fSaw38XHFEIIcRSuDvy1QKFSKk8pFQIsAt5w8TGFEEIchUu7dLTW/Uqp24H3MMMy/6G13ubKYwohhDg6l4/D11q/Dbzt6uMIIYQ4PhnCIoQQfkICXwgh/IQEvhBC+AkJfCGE8BMetcWhUqoBqBrmryUBjS4ox9PJefsXOW//MtzzztFaJw/1Jo8K/BOhlCpxZi9HXyPn7V/kvP2Lq85bunSEEMJPSOALIYSf8IXAf9TqAiwi5+1f5Lz9i0vO2+v78IUQQjjHF1r4QgghnOAVga+UOk8ptUMptVsp9f+O8nqoUmqJ4/XVSqlc91c58pw473uUUqVKqc1KqY+UUjlW1OkKQ537oPddpZTSSimfGMnhzHkrpa5x/LtvU0otdneNruDEf+vZSqmPlVIbHP+9X2BFnSNJKfUPpVS9UmrrMV5XSqmHHH8nm5VS0076oFprj35gVtncA+QDIcAmoPgr7/k+8HfH14uAJVbX7abzXgBEOL7+ni+ct7Pn7nhfNPAZsAqYYXXdbvo3LwQ2APGO71OsrttN5/0o8D3H18VApdV1j8B5nwZMA7Ye4/ULgHcwG0nNAVaf7DG9oYV/eF9crXUvMLAv7mCXAk87vn4ZWKi8fxfyIc9ba/2x1rrL8e0qzAYzvsCZf3OAXwO/B7rdWZwLOXPe3wH+qrU+CKC1rndzja7gzHlrIMbxdSw+sJGS1vozoPk4b7kUeEYbq4A4pVT6yRzTGwL/aPvijjrWe7TW/UArkOiW6lzHmfMe7BZMa8AXDHnuSqmpQJbWeqk7C3MxZ/7NxwBjlFIrlFKrlFLnua0613HmvH8B3KCUqsYst36He0qz1HAzYEguXw9/BAy5L66T7/E2Tp+TUuoGYAZwuksrcp/jnrtSKgD4X+BmdxXkJs78mwdhunXOwHyiW66UmqC1bnFxba7kzHl/A3hKa/0npdQpwLOO87a7vjzLjHiueUML35l9cQ+/RykVhPnId7yPSt7Aqf2AlVJnAT8BLtFa97ipNlcb6tyjgQnAJ0qpSkz/5hs+cOPW2f/WX9da92mtK4AdmAuAN3PmvG8BXgTQWn8BhGHWm/FlI74nuDcEvjP74r4B3OT4+ipgmXbc9fBiQ563o1vjEUzY+0Jf7oDjnrvWulVrnaS1ztVa52LuX1yitS6xptwR48x/669hbtajlErCdPGUu7XKkefMee8FFgIopcZhAr/BrVW63xvANx2jdeYArVrrAyfzB3p8l44+xr64SqlfASVa6zeAJzAf8XZjWvaLrKt4ZDh53n8AooCXHPeo92qtL7Gs6BHi5Ln7HCfP+z3gHKVUKWADfqS1brKu6pPn5Hn/B/CYUuoHmG6Nm729UaeU+iemay7JcW/i50AwgNb675h7FRcAu4Eu4FsnfUwv/zsTQgjhJG/o0hFCCDECJPCFEMJPSOALIYSfkMAXQgg/IYEvhBB+QgJfiCEopXKPtaKhEN5EAl8IIfyEBL4QzglUSj3mWIP+faVUuNUFCTFcEvhCOKcQsyzxeKAFuNLieoQYNgl8IZxTobXe6Ph6HZBrYS1CnBAJfCGcM3glUhtesA6VEF8lgS+EEH5CAl8IIfyErJYphBB+Qlr4QgjhJyTwhRDCT0jgCyGEn5DAF0IIPyGBL4QQfkICXwgh/IQEvhBC+AkJfCGE8BP/H/DudeeNp6VjAAAAAElFTkSuQmCC\n", "text/plain": [ "
" ] }, "metadata": { "needs_background": "light" }, "output_type": "display_data" } ], "source": [ "# In Logistic Regression, 0" ] }, "metadata": { "needs_background": "light" }, "output_type": "display_data" } ], "source": [ "alpha=0.3\n", "iterations=1500\n", "\n", "theta,jHistory=gradientDescent(X,y,theta,alpha,iterations)\n", "df=pd.DataFrame({'Iterations':range(iterations),'Cost Function':jHistory})\n", "print('FINAL COST : %.4f'%jHistory[iterations-1])\n", "sns.lineplot(data=df,x='Iterations',y='Cost Function');" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "**Plotting Decision Boundary** \n", "A decision boundary is hypersurface that partitions the underlying vector into two sets or classes.\n", "\n", "Genral Equation of Line : \n", "$$ ax+by=c $$\n", "$$ \\theta_1x_1+\\theta_2x_2=-\\theta_0 $$ \n", "$$ x_2=(\\frac{-1}{\\theta_2})(\\theta_0+\\theta_1x_1)$$" ] }, { "cell_type": "code", "execution_count": 15, "metadata": {}, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYgAAAEKCAYAAAAIO8L1AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvOIA7rQAAIABJREFUeJzsnXd4VNXWh9+VBgmEHkBAilwMBFKA0L0UQXoRlQ8RCWBBIBFQr2KlqNcrilKkBBAREQEBKSoiHQQRBIyUJIh0BEmkd0iyvz9mEkJIz8ycMzP7fZ48Z87JKb9z5sxeu6y1tiil0Gg0Go0mIx5GC9BoNBqNOdEGQqPRaDSZog2ERqPRaDJFGwiNRqPRZIo2EBqNRqPJFG0gNBqNRpMp2kBoNBqNJlO0gdBoNBpNpmgDodFoNJpM8TJaQEEoU6aMqlq1qtEyNBqNxqnYuXPnP0qpgJz2c2oDUbVqVXbs2GG0DI1Go3EqRORobvbTXUwajUajyRRtIDQajUaTKXYzECLymYgkiMjedNtKichqETlgXZa0bhcRmSgif4rIbhGpZy9dGo1Go8kd9mxBfA60z7DtVWCtUqoGsNa6DtABqGH9GwBMtaMujUaj0eQCuxkIpdQm4GyGzd2A2dbPs4GH023/Qln4BSghIvfYS5tGo9FocsbRYxDllFKnAKzLstbtFYHj6fY7Yd12FyIyQER2iMiOxMREu4rNjpQUReKlG/x17iqJl26QkqInXtJoNK6FWdxcJZNtmZa4SqnpwHSA8PBwQ0rllBTF/tOXePaLHZw4d41KJX2ZERFOYDl/PDwyuxWNRqNxPhzdgjid2nVkXSZYt58A7k23XyXgpIO15ZozV26mGQeAE+eu8ewXOzhz5abByjQajcZ2ONpALAf6Wj/3BZal2x5h9WZqDFxI7YoyIzeTktOMQyonzl3jZlKyQYo0LklKClw+DeePW5YpKUYr0rgZdutiEpF5QEugjIicAEYC7wNfi8jTwDGgh3X3FUBH4E/gKtDfXrpsgY+XJ5VK+t5hJCqV9MXHy9N+F01JgauJkHQTvHzALwA8dBiLy5KSAgmxML8XnD8GJSrD4/OgbJD+3jUOQ5Ry3sHV8PBwZUSqDYePQejCwv24fBo+bWP5vlMpURmeWQNFyxmnS+MSiMhOpVR4TvuZZZDaqfDwEALL+bNkcDNuJiXj4+VJ6SI+9hugvpp42ziAZTm/ly4sbE3GVppvabh2xphWW9LNO40DWNaTTDLOpVu0boE2EPnEw0MI8C/kmIuZvbBwBTK20gI7QYtX4Os+xrTavHws18zYgvDyyf4eHFFo6xat26C/TWcgtbBIT06FhSZvZGylhfW6bRzgdqvtqoNib/wCLIVu6veeWgj7ZZGhObXQ/rQNjK9jWSbE2mdgO6sWraOejcZh6BZEDqSkKM5cuemYrqSsSC0sMtbYsiosNHknYyvNt6SxrTYPD0uN/Jk1uWsROLIbUrdo3QZtILLBNAFxeS0sNHknY5fOtXN57+KxNR4euS/cHVlo56f7S+OU6BImG0wVEJdaWJS417LUxsG2ZOzSiZkH/zcn9108RuPIbsi8dn9pnBbdgsgGHRDnRmTWSvMt7TytNkd2Q+oWrdugDUQ2GBIQpzGOzLp0nMWN2NGFdl66vzROi1ua/A1HNtB1XleOnD+S7X6li/gwIyKcSiV9AdLGIEoX0X2tGhOiuyE1NsYtWxDHLhxj7eG1BE0OYmSLkbzQ5AV8PO8u9B0eEKfRaDQmwi2rGBGhEcRFxtHuX+14de2r1J1Wl01HN2W6b2pAXMWSfgT4F9LGQWMsOoGfxoG4pYEAqFy8Mkt6LmH548u5cvMKLT5vQf9l/Um8ooN9NCbFkcFwGg1ubCBS6RLYhX2D9zG82XC+3P0lgZMC+XTXp6Qo/aPTmAxnimDWLR2XwO0NBEARnyK83+Z9Yp6LoU7ZOjz77bM88NkD7D6922hpGs1tnCWCWbd0XAZtINJRu2xtNvbbyKxuszhw9gD1ptXjP6v+w+Wbl42WptE4T04uZ2rpaLJFG4gMiAj9wvoRHxlP/7D+fLT1I2pNrsWSuCU489wZGhfAWSKYk25C0bLQ80vo971lWbSs+Vo6mhzREwblwM/Hf2bgdwPZk7CHTjU68UmHT6hWsppdr6nRZIk9Unrb+pxX/oGEOFg2+HZUd7cpULYWFClTMK0am5DbCYMMaUGIyFAR2Ssi+0RkmHVbKRFZLSIHrMuSRmjLSNN7m7JzwE7GPjSWDUc2UHtKbf730/+4maxrQxoDsHUwnD3GC1KSbxsHsCyXDbZs1zgVDjcQIlIHeBZoCIQCnUWkBvAqsFYpVQNYa103Bd6e3rzU9CXiIuPoUKMDr697nbDoMDYe2Wi0NI0mf6R6GV04BpdOWrqAwDbjBclZDKbrSpXTYUQLohbwi1LqqlIqCdgIdAe6AbOt+8wGHjZAW7bcW/xeFv/fYr7r9R3Xkq7RcnZL+i7tS8KVBKOluQ7aPdL+pG81TAiF71+CB0dAJWuPQ0E9o5xlMF2TI0YYiL1AcxEpLSJ+QEfgXqCcUuoUgHVZNrODRWSAiOwQkR2JicZ4RXS6vxP7Bu/jtQdeY96eedScVJPpO6fr2ImCot0jHUNmXkbLo6DZMMt6QQtzZxlM1+SIIYPUIvI0EAlcBmKBa0B/pVSJdPucU0plOw7hiEHqnIhLjGPQ94PYeHQjjSs1ZmqnqYSVDzNUk81w9MT0l09bjELGiWjsMSuaO3P+uMUAZ6Tf97B0kG3ml3b0u2NrnF1/Dph6kFopNVMpVU8p1Rw4CxwATovIPQDWpVP029QKqMX6vuuZ/fBsDp49SP3p9Xnxxxe5dOOS0dIKhhG1eWcJBHN2suoCKlHFYowLahzAuTPL6pZsGkZ5MZW1LisDjwDzgOVAX+sufYFlRmjLDyJCRGgE8VHxPFvvWcb9Mo5ak2uxOHax88ZOGBHspPuuHUNWXUDFKjpfYW4PdKBfGka9CYtFJBb4FohUSp0D3gceEpEDwEPWdaeilG8pojtHs/XprZTxK8NjCx+j01edOHTukNHS8o4RtXndd+0Y0k8uNGzvna0Gd3QSyOyedUsWMGg+CKXUvzPZdgZobYAcm9O4UmN2DNjBJ9s+YcSGEdSeUps3//0m/2n6Hwp5FbLPRW3dZ2rExPR6KkvHkdmMcKldKxmnLbVFl5NZyeyee86FwE6w//vb+7lpS9ZFv3Xj8fLw4oUmLxAXGUfn+zvz5vo3CY0OZf3h9ba/mD36TI2qzTtz37Wz445dK5nd84Le0O6/jnv3Tdxq06k2HMQPB34gckUkh88f5smQJxn70FjK2cozx17ePy7uyeHy95dXsvJuGrbXYrBdkezu2cvH/u+GQa02U3sxuSMdanRg3+B9vPnvN1mwdwE1J9ckeke0bWIn7DVe4Mq1ee2pcpvUGqxKht4LbwfMget3rWTnGOGId9/krTYX+sWbH19vX9558B12D9pNWPkwBn0/iCYzm/Dbqd8KdmLt/ZN3zPjDNKKrIbOo6tajLEbCHZwEjHaMMLlrtzYQBlCzTE3WRaxjTvc5HDl/hPAZ4QxbOYyLNy7m74RGv+TOiNl+mEa1aDIzlMsGw2Of2y4mwsxk59HlCExeuXPhb97ciAhPhjxJfGQ8A+oNYOK2idSaXIuv932d99gJo19yZ8RsP0xHtmjSt1RS525IT6oGV+tWzAoju1JNXrlzg2/f3JT0LcnUzlPZ+vRWyhUpR89FPekwtwMHzx7M24lcebzAHpjth+moFk3GlsrnHW93KaViohqsy2Pyyp05VGhoVKkR25/dzvh24/n5+M/UnlKbdza+w42kG0ZLc03M9sN0VIsmqy6lFsNvX9NENVi3wMSVO/Mo0eDl4cXQxkOJi4yjW81ujNgwgpDoENYeWmu0NNfETD9MR7VosmqplLnfHIZSYyr0W2BCKharyILHFrCy90qSU5JpM6cNvb/pzd+X/zZamsZeOKpFk1VLxdvXHIZSYyr0m2Bi2v2rHXsG7eGt5m+xKHYRNSfVZPL2ySTrqRtdE0e0aMw29qIxNTqS2knY/89+IldEsvbwWsIrhBPdKZr6FeobLUvjjOgIcrdHR1K7GIFlAlndZzVzH5nL8QvHafhpQ4b8MIQL1y8YLU3jbJhp7EVjavSb4USICE8EP0F8VDyDwgcxafskak2uxYK9C5x33gmNRmNatIFwQkoULsGkjpPY9sw27vG/h8cXP067L9tx4MwBo6VpNBoXQhsIJ6ZBxQZsf2Y7E9tP5JcTvxA8NZjRG0ZzPem60dLcBxOnatZoCopRU46+ICL7RGSviMwTkcIiUk1EtonIARFZICI6lDMXeHp48nyj54mPiufhmg8zauMogqcGs/rgaqOluT46I6zGxXG4gRCRisAQIFwpVQfwBB4HxgDjlFI1gHPA047W5sxU8K/A/Mfms+rJVQC0/bItvRb34tSlUwYrc2HMmBFWo7EhRnUxeQG+IuIF+AGngAeBRdb/zwYeNkibU/NQ9YfYM2gPI1uM5Ju4b6g5uSaTtk/SsRP2wGwZYTUaG+NwA6GU+gsYCxzDYhguADuB80qpJOtuJ4CKjtbmKhT2KsyolqPYO2gvjSo24vkfnqfRp43YcdI9YkYchtkywmo0NsaILqaSQDegGlABKAJ0yGTXTP02RWSAiOwQkR2JibopD5CSoki8dIO/zl0l8dINUlIsj65G6Rr8+OSPzHt0Hn9d+ouGMxoStSKK89fPG6zYRdBRyRoXx+GR1CLSA2ivlHrauh4BNAF6AOWVUkki0gQYpZRql9253CmSOitSUhT7T1/i2S92cOLcNSqV9GVGRDiB5fzx8JC0/S5cv8Cb695kyo4pBPgFMK7dOB6v8zgiks3ZNTlihqhkM2jQOBVmjqQ+BjQWET+xlE6tgVhgPfCYdZ++wDIDtOWLrGrwjuDMlZtpxgHgxLlrPPvFDs5cubMfvHjh4nzS8RO2P7Ode4vfyxPfPMFDcx7ijzN/OEyrxg5oTyqNHTFiDGIblsHoXcAeq4bpwHDgRRH5EygNzHS0tvyQWoPvPmULzcasp/uULew/fclhRuJmUnKacUjlxLlr3EzKfFC6foX6/PL0L0zqMIlfT/5K8NRgRq4fqWMn8oMZCmftSaWxI4a0Q5VSI5VSNZVSdZRSfZRSN5RSh5RSDZVS/1JK9VBKOcVMObmtwdsLHy9PKpX0vWNbpZK++Hh5ZnmMp4cnkQ0j2R+1n8eCHuPtTW9TZ0odfvzzR3vLdS3MUDhrTyrjceFgSd1RWUDyWoO3NaWL+DAjIjzNSKSOQZQukrMnTfmi5Zn7yFxW91mNh3jQfm57ei7qyclLJ+0t2zUwQ+GsPamMxQytSDuiDUQByU8N3pZ4eAiB5fxZMrgZW4a3YsngZncNUOdEm/vasHvQbka3HM2y+GXUnFSTidsmkpSSlPPB7owZCmdn9aRylVq3GVqRdkTPB1FAcutFlJ/znrlyk5tJyfh4eVK6iE+Bzpdb/jz7J5ErIll1cBV1y9clunM0DSs2tPt1nZLU2mNqAZFaODt6yk5n82Iyy3OzBeePW1oOGRm215JO3aTk1otJGwgbYOvC3F5GJ7copVgYu5BhK4fx9+W/GRg+kPdav0eJwiVydwKzFVj21GO2e3UGLp+2dMWk754rUdkyzWrRcsbpyg9Oei9mdnN1GVLdW09dsIxB3FPclwD/QgUuxI0e+BYR/q/2/xEfFc+QRkOYtnMagZMCmbt7bs7zTpitT9beetx98p38dBWZYezGVjhrF18ucbO32XbY073V6IHvVIoVKsb49uP59dlfqVK8Ck8ueZI2c9oQ/0981geZrU/WbHpcifwaXzOM3dgKDw9L19gzayzdSs+scc6usixwjbswAHvW8o0e+M5IvXvqsfXprUzpOIWdJ3cSMjWEt9a9xbVb1+7e2Wy1Q7PpcSXya3xdrdbtwq1I17kTB2PPWn5BXFfthaeHJ4MaDGJ/1H561unJuz+9S52pdVj558o7dzRb7dBselyJzIxv0bLW7dl0Obl4rduV0N9IPrFnLd8Wrqv2olzRcszpPoe1EWvx9vCmw9wO9FjYg78u/mXZwWy1Q7PpcSUyGt9K4dB6FHzeMecuJxeudbsS2ospnxjtaWQGbiTdYOzPY3n3p3fx8vDinVbvENUwCi88zOXZoz2N7ENGd9XeC+H7l5zOo8cd0W6uDiA791aj4hiM4ODZg0T9EMXKP1cSVj6M6E7RNKrUyGhZuUMbj4KR/vmpZJgQevc+Jo8JcEe0m6sD8PAQAvwLUbGk3x3urUYn8HM01UtVZ8UTK1jYYyEJVxJoMrMJA78byLlr54yWlj1ZeeFc+cf5I3wdRfquIm9fPd7jYmgDYQeMjmMwAhHhsaDHiI+MZ2ijoczYNYPASYHM+X1OzrETRpGVF87JXeaI4XA29HiP/TAoNYk2EHbALHEMRuBfyJ9x7cexc8BO7it5HxFLI3jwiweJS4wzWtrdZOUC6+13+7OOmcg92jvJPhgYfKq/OTtgtjgGIwgrH8bPT/9MdKdoYv6OITQ6lDfWvsHVW1eNlnabrFxg03eN6ZiJvKG9k2yPgcGe+tuzA7aOYzByxrqC4CEePBf+HPuj9vN4ncd5b/N71JlShxUHVhgtzUJmXSLdpsCW8bf30X3oGqMxMNhTezHZCVt5MbmSO+2GIxsY9P0g4v+J55FajzCh/QQqFatkrKj0XjiePnDjEnzZ3fmzjGpcBzskBDStm6uIBAIL0m26DxgBfGHdXhU4AvyfUipbNxgzGwhbkXjpBt2nbLljTKNSSV+WDG5GgH8hA5Xlj5vJNxn781je2fQOXh5ejG45miGNhuDl4WW0NAva7fVO9PMwHjukRzetgbjj4iKewF9AIyASOKuUel9EXgVKKqWGZ3e8OxiIv85dpdmY9Xdt3zK8FRVL+hmgyDYcPneYqB+iWHFgBSHlQojuFE2Te5sYLetuXK2AzMv9uNK8DeDc36WNtTtLHERr4KBS6ijQDZht3T4beNgwVSbC5Qa8re561cSL77rMZPFjCzl77SxNP2vKgG8HcPbaWaMV3sZsqcsLSl7vx5Uy4Tr7d2nQ4L/RBuJxYJ71czml1CkA67KsYapMhBkT9+WbDD9SmfkQj5SuSezAvbzY+EU+++0zAicFMjtmtjliJ1ypgIS8348rZcJ1te/SQRhmIETEB+gKLMzjcQNEZIeI7EhMdP0v18yJ+/JMFj9S/+TrfNTuI3YO2EmNUjXot6wfLWe3JDYx1li9rlRAQt7vJys3YHC+KHOzfpc5BcAZPHe3kS2IDsAupdRp6/ppEbkHwLpMyOwgpdR0pVS4Uio8IMA9IjSzSunhdOTwIw0tH8rmpzYzvfN09pzeQ2h0KK+tec242AlXSxWe1/vJyg14UT/n66Ix43eZU7eXCbrFjDQQvbjdvQSwHOhr/dwXWOZwRRr7kosfqYd48Gz9Z9kftZ/ewb15f8v7BE0O4rs/vnOwWFwvdURe7yd9ZPTQ36HTR7B2FJzY4XxdNGb8LnPq9jJBt5ghXkwi4gccB+5TSl2wbisNfA1UBo4BPZRS2Y5YuoMXk0uRD6+YTUc3Mej7QcQmxvJwzYeZ0H4ClYtXznRfu2l2Vs+XzMjv/Zw/bqnFZsSZMrWa7bvM6Zna8Znn1ovJEOdzpdRVoHSGbWeweDVpXJX0NdJc/kibV2nOb8/9xsdbP+btjW8TNDmIUS1HMbTRULw9vR2j2ZXmMsjv/aS2/jIGazlTd5vZvsucnqkJnrkTV4U0Tkk+3PV8PH149YFXiY2MpVW1Vry8+mXqT6/PlmNbHCBYA5izi8bZyemZmuCZ61QbboazT2SklGLZ/mUM+WEIxy8e5+m6TzOmzRhK+5XO+WAzYbbujtzgjJrNTk7P1E7P3CkiqQuKNhB5w5XyOl2+eZnRG0Yz7pdxlChcgg8f+pC+YX3xECcosFwtQlnjdDhLJLXGgbjSREZFfYryYdsP+e2536hZpiZPLX+KFp+3YG/CXqOl5YwJvFPcHnvEFxgcs2APcmUgROSu0UARKWN7ORp74ooTGQWXC2ZT/0182uVTYhNjqTutLsNXD+fKzStGS8saswZtuQv2iC8wQcyCPcjWQIhIKxE5AZwUkVUiUjXdv1fZU5g7Yu95H5w6r1M2tTMP8eDpek+zP2o/ESERfPDzBwRNCWL5/uUGCs4GMwZtuRP2aMG5aKswpxbEB0A7pVQAMB1YLSKNrf9zrk5rk5M6PtB9yhaajVlP9ylb2H/6kk2NhNPmdcpl7ayMXxlmdpvJT/1/wt/Hn27zu9FtfjeOnj9qkPAsMIF3iltjjxaci7YKsx2kFpHflVKh6dZrA98ArwJvKaXq2V9i1rjSILWj5n1wSi+mfEyYciv5FuN+GcfojaMBGNliJC80fsExsRO5QXsEGYcdJuCxyzntiK0GqW+JSPnUFaXUPizBbCOBGgWTqEmPo8YHnDKvUz5qZ96e3rzS7BViB8fy0H0PMXzNcOpOq8tPR3+ys9hcouduNg57tOBctFWYUyT1q0A54O/UDUqpEyLSEssEPxobkTo+kLEF4RTjA/amABGlVUpUYenjS1m+fznP//A8zT9vTv+w/nzw0AeU8dN+Fm5JPiL6DTmnCdBxECbBlWIUsiNfXVw2ihu4cvMK72x6h4+2fkSxQsX4oM0H9K/b3zliJzQaG2LTQDkR6Qy8A1TB0uoQQCmlihVUaEFwJQMBTjo+kAcKZARt2Ge/N2Evg74fxOZjm2l6b1OiO0UTXC44X+fSaJwRWwfKjceSgru0UqqYUsrfaOPgijjl+EAeKFCgng377OuUrcPGfhv5rOtn7P9nP3Wn1eXlVS9z+eblfJ9To3FFcvsrOw7sVc7cH6UxHDMF6nmIB/3r9md/1H76h/Vn7NaxBE0OYmn8UnNMd6rRmIDcGohXgBUi8pqIvJj6Z09hGtfDjIF6pf1KM6PrDDb330yJwiXovqA7Xed35cj5I4Zp0mjMQm4NxH+Bq0BhwD/dn0aTa8wcqNescjN2DtjJ2IfGsv7weoImB/H+5ve5mWzDQCcXzNWjsSMmeF9yO0i9IzcDGo7G1Qap3YGUFMX5aze5djOZZKUo7O1JmSLmGm85duEYQ1cOZWn8UoICgpjaaSrNqzQv2El1BldNXrDz+2LrQeo1ItK2gJrSEJESIrJIROJFJE5EmohIKRFZLSIHrMuStrqePbF3/iRX5PTFG/Sc/gvNP9jAI1N+tnlKkYJSuXhllvRcwre9vuXKzSu0+LwF/Zb2I/GKztWjcRAmeV9yayAigZUick1ELorIJRG5WIDrTgBWKqVqAqFAHJagvLVKqRrAWuu6qXFE/iRnIC9G0plSjne+vzOxkbG82uxV5u6ZS+CkQGbsnEGKykdT30Vz9WjshEnel1wZCKtbq4dSyregbq4iUgxoDsy0nvumUuo80A2Ybd1tNvBwfs7vSJypsLMXeTWSdvNkslN/rZ+3H/9r8z9+H/g7weWCGfDdAB747AF2n96dtxPpDK6avGCS9yXXnVkiUlJEGopI89S/fF7zPiARmCUiv4nIpyJSBCinlDoFYF2Wzef5HYaZ3DaNIq9G0i6eTA7IxR8UEMSGvhv4vNvnHDh7gHrT6vHSjy9x6cal3J3ARXP1aOyESd6X3E4Y9AywCfgRGG1djsrnNb2AesBUpVRd4Ap56E4SkQEiskNEdiQmGtt/a0a3TUeTVyNpF08mB/XXigh9w/qyP2o/T9V9io9/+ZigKUF8E/dNzrET6XP1DNtrWeoBak1WmOR9ye3VhgINgKNKqVZAXSytgPxwAjihlNpmXV+ExWCcFpF7AKzLhMwOVkpNV0qFK6XCAwKMrX2Z2W3TUeTVSHp4CIHl/FkyuBlbhrdiyeBmBc835eD+2lK+pZjeZTo/P/UzpXxL8ejXj9J5XmcOnzuc/YE6g6smL5jgfcntFa8rpa4DiEghpVQ8EJifCyql/gaOi0jq8a2BWGA5lnQeWJfL8nN+R2KXws7JyI+RtHlKEYP6a5vc24SdA3byUduP2HhkI0FTgnjvp/dsGzuh0RhIbuMglgD9gWHAg8A5wFsp1TFfFxUJAz4FfIBD1nN7AF8DlYFjQA+l1NnszqPjIMyB4UkGTRBjcOLiCYatHMbiuMXUKlOLKZ2m0LJqS4dcW6PJKzbN5prhxC2A4ljcVA2tKmkDoUnDJDO0rTiwgqgVURw+f5g+IX0Y23YsZYuY3t/CeTDJ9+zs2DRQTkTapH5WSm1USi0HehVAn0ZjW0zQXwvQsUZH9g7ey+sPvM78vfMJnBTItB3T8hc7obkTB3irae4kt7+iESIyVUSKiEg5EfkW6GJPYRr7oiPA7Yeftx//bf1ffh/4O2Hlwxj4/UCazmxKzN8xRktzDrKKaTFJdLE7kVsD0QI4CMQAm4GvlFKP2U2Vxq7oCHDHUCugFusi1vHFw19w6Nwh6k+vzwsrX8h97IQ7kl0rwSTRxe5Ebg1ESaARFiNxA6giIu7jquNi6AhwxyEi9Antw/6o/Txb71kmbJtArcm1WBS7SM87kRnZtRJMEl3sTuTWQPwC/KCUao8lHqICsMVuqjR2RUeAO56SviWJ7hzN1qe3ElAkgB4Le9Dxq44cPHvQaGnmIrtWgkmii92J3BqINsAtERmhlLoGjMUJkulpMkdHgBtHo0qN+PXZXxnXbhybj22mztQ6vLvpXW4k3TBamjnIrpVgkuhidyK3T/Y1oDG3PZcuAR/ZRZHG7ugIcGPx8vBiWONhxEfG0/n+zry1/i1Co0NZf3i90dKMJ6dWgkm81dyF3AbK7VJK1ROR36z5kxCR35VSoXZXmA06DiL/GB7cpklj5Z8riVwRyaFzh+gd3JuP2n5EuaLljJZlHDrWwe7YesKgWyLiCSjryQMA7XzsxNg83YUm37T/V3v2DtrLm/9+k6/3fU3gpECm/jqV5BQ3HRPSrQTTkNsnPxFYApQVkf9icXV9z26qNBo3w9fbl3cefIc9g/ZQv0J9Bq8YTNPPmrLr1C6jpWncmNxOGDQXeAX4H3AKeFgptdCewjQadySwTCBr+qydrvlNAAAgAElEQVThy+5fcuT8ERrMaMDQH4Zy8UZBJnDUaPJHnnMxmQk9BqFxSax98OcuJ/DGLx8R/fsXlC9anvHtx9MjqAc6BElTUGw9BqHROBwzpANxuIZ0kcQlox9gypGd/NJ9LuWLlqfnop50mNuBP8/+aV8NGo0VbSA0+caehacZ0oEYoiGTSOKG695je69vmdB+Aj8f/5k6U+rw9sa3deyExu5oA6HJF/YuPM2QDsQQDVlEEnulpDCk0RDio+J5uObDjNwwkpDoENYeWms/LRq3RxsITb6wd+FphnQghmjIId9QBf8KzH9sPj8++SPJKcm0mdOGJxY/wd+X/7afJo3bog2EJl/Yu/A0QzoQQzTkMt9Q2+pt2TNoDyOaj2Bx3GJqTqrJ5O2T3Td2QmMXDDEQInJERPaISIyI7LBuKyUiq0XkgHVZ0ghtmtxh78LT6HQgKSkKTw+Y9mR9x2rIQ74hX29fRrcazZ5Be2hQsQFRP0TReGZjdp7caT99GrfCEDdXETkChCul/km37QPgrFLqfRF5FSiplBqe3Xm0m6txpI5BpHYzpRaeNQKKcu7aLZuk8DAqHUj6ewsoWoghrWtQrUwR/Ap5UqaIOaPOlVLM3zufF1e9SMKVBAaHD+bdB9+leOHiRkvTmBC7zUltC7IwEPuBlkqpUyJyD7BBKRWY3XnyayB0HiLbkPE5lvT15kDi5buMRmA5f6d6vomXbtB9ypY7utAqlfRlyeBmBPgXMlBZzly4foE3173J5F8nU65oOca1G0fP2j117ITmDsweB6GAVSKyU0QGWLeVU0qdArAu7TLTuxncJ12FjPmczl27ZbjnkS0wwwB5fileuDifdPyE7c9up6J/RXot7kXbL9ty4MwBo6VpnBCjDEQzpVQ9oAMQKSLNc3ugiAwQkR0isiMxMe9z0ZrBfdJVceaCNT1mGCAvKOEVwtn2zDYmdZjE9r+2Ezw1mNEbRnM96brR0jROhCEGQil10rpMwJIEsCFw2tq1hHWZkMWx05VS4Uqp8ICAvM8k5SqFmBnJqmBNTlFO1UIzeoDcVnh6eBLZMJL4yHi61+rOqI2jCJ4azOqDq42WljkpKXD5NJw/blmm6ITRRuNwAyEiRUTEP/Uz0BbYCywH+lp36wsss8f1XaF2aFZKF/FhWp87vX7GPBrCu9/HOlULzcNDCCznz5LBzdgyvBVLBjdzunGU9Nzjfw/zHp3HqidXIQhtv2xLr8W9OHXplNHSbpMuxQjj61iWCbHaSBiMwwepReQ+LK0GAC/gK6XUf0WkNPA1UBk4BvRQSp3N7lz5GaTOyvvGmQsAM3H6wjViTlyghK8356/dInrDQX47fp4tw1tRsaSf0fLcnutJ1xmzeQz/2/w/CnkV4t1W7zK4wWA8PTKpIDly4p7Lpy1GIX0UeYnKFjdfd548yU6Y2ovJVpjRi8ndPaSc2QPInThw5gBRP0Sx6uAq6t9Tn+jO0YRXSFdepNboU/NCpQbs2WsO6PPHLS2HjAzba5k4SGNTzO7FZCj2mk1Ne0hl3X9f0tfb8MysZsSojLU1StdgZe+VzH90PicvnaThjIZErYji/PXzlh0ySRrI/F6W7fYghxQjGmNwyxaEvdC1ZwuuGh9hawzv7rR2IV248g9vbRvH5JhZBPgF8HG7j+lV6QFkQvDdx9irRu/oFoubo1sQBqA9pCykb6GVLuJDwuUbXLmRxFudg6h7bwntWmzFUJfrdIPCxac2ZeKhbWzv/hWVi1em9ze9eWjpk/zhn6Hv3541+jykGNE4Di+jBbgSqR5SGVsQ7uohlVkNecyjIYz9cT+/HT/vdoYzI4ZWKDLpQqq/9l22PvUj0/Yv5fW1rxN86yrDC5fgtevX8S1RJdOkgTbFw8P5B6QdObDvAJxXuQlxFf95W5FZDXn44t0MbFndrQ1nKoa6XGcx74RnSjKDGwwmPiqeHkE9eOdGIsElSvFjmzdcr0Zv67gLF3TVdaFv23hczX++oGRVQ041pO5qOFMxtEKRw6Bw+aLl+fLRuazpswZPr0K0X9yLnt/04uSlkzmf2xkC3uxRmDt6YN8BaANhY+zlIeWMZFVDrlDC160NZyqGVihyOe9E6/tas3vgbt5u+TbL4pdRc1JNJvwygaSUpMzP6yy1aHsU5lm0ykhy3rE2tzUQtnAvNMpF0VnIqoZcvlhhtzcOqRhWocjDoHAhr0K81eIt9g3eR7PKzRj24zAazmjI9r+2331eZ6lF26Mwd0FXXbccpLaFe6HhLopOQPoasrsGDpqaPA4KVy9VnRVPrGBR7CKG/TiMxp825rn6z/Fe6/co6Wud38uWBa89B3xTC/OMkdsFKcxTW2UZXXXtObBvZ9wyDiKv8QqZRUefuXJTxzxo3JaLNy4yYv0IPtn+CWX8yvBR24/oHdwbuZJgm5QZ9o6LyOz8PedCEasRyq8xchIvJh0HkQ15cS/MOjo6Rcc8aPKNs3dPFitUjPHtx7Pj2R1ULVGVPkv60PqL1sRfPZOrsY0csXdXVcYutojlsOF9+LhmwcZNUltlJe61LE1oHPKCc6vPJ3lxL8wqmClZobPCugBGFNSulJKl7j11+fmpn5naaSq//f0bIdPCeHPfPK71/TbbsY0cn7sjBnxTC3MvH/iiK+z//vZ1zDhuYgBuaSDy4l6YVWtDKaVjHpwcowpqV5u0ytPDk4HhA4mPjKdnnZ78d/N71JnTmh8S92Zai87Vc3fkgK8Leh/ZCrc0EHlxL8yutaFjHpwbowpqM6RksUfLqVzRcszpPod1Eevw9vCm41cdeezrxzhx8cQd++XquefSDdcmuKD3ka1wSwMBuXcvzK61oWMenBujCmqjJ62yd8upVbVW/D7wd95t9S7fH/ieWpNrMW7ruLTYiVw9d0fmZnKkMXIy3NKLKa+4+xwProojs++mf4e8vTy4fD2JiM+2G+Ii7cj7PnTuEFErovjhzx8ILRdKdOdoqhevaz4PQCfxPrIVesIgjSYHHBXLktV1yhUrxLWbjq90/HXuKs3GrL9ru71m/VNK8U3cNwxdOZSTl07yTL1n6F/ndV7++lDa85jWpz6BZf3x8nLdQtlMmN5AiIgnsAP4SynVWUSqAfOBUsAuoI9SKtvOYG0gNAXFEa1Ds80TYpSeSzcuMXLDSCZum0gp31K81vS/PFDhEc5evcXincd54aFAPY7nIJwhDmIoEJdufQwwTilVAzgHPG2IKo1b4YhxJDMMSqfHqCSB/oX8+bjdx+wcsJPKxe/jxTUD+PesVjz15VJWxSY4tSeXq2KIgRCRSkAn4FPrugAPAousu8wGHjZCm8Y5cKZAM6MHpTNidNbh0PKhLHlsDaVuRnHL4zAnCz3POa/ZHDt3TgeamgyjWhDjgVeA1FDF0sB5pVRqisgTQEUjhGnMj7MFmplxnhCjPfAKe3tTq1h3KlyPpkhyCy56LyTBL4qNx1Y5VIcmexxuIESkM5CglNqZfnMmu2b6axeRASKyQ0R2JCbqSEd3xNkCzYyusZuRVKNZpeQ9lLn1AnV8PqZi8eL0XtqdR79+lOMXjhstMUucqfVaUIzI5toM6CoiHYHCQDEsLYoSIuJlbUVUAjKdmUQpNR2YDpZBasdI1pgJs/Xp54bUGrvGwt2ZftvgX3gw4375mHc2vcOPf/7I6JajGdJoCN6e3kbLTSM/nm/O7CZvqJuriLQE/mP1YloILFZKzReRaGC3UmpKdsdn5sV069YtTpw4wfXr1+2mW2MsydYaXFK6mpuXtQD29BAKFy5MpUqV8PY2T8GiyT2Hzx3m+R+e5/sD3xNSLoSpnabS9N6mRssC8pcJ2ozTAuTWi8lM80EMB+aLyLvAb8DM/JzkxIkT+Pv7U7VqVSxj3xpXQynF9VspHD1zhZvJKfh4elCldBEKe1t6TM+cOcOJEyeoVq2awUo1+aFayWp82+tblsYvZcjKITT7rBnP1H2G99u8T2m/0oZqy2vrNavuUGeZFsDQqBSl1AalVGfr50NKqYZKqX8ppXoopW7k55zXr1+ndOnS2ji4MCJCYW8PqpctSs3y/lQvW5TC3h6ICCJC6dKldQvSyRERutfqTlxkHP9p8h9mxcyi5uSazI6ZjZG9Hnn1SHPG7tD0uGTYojYOro+I4O3pgY+XJ96eHnd85/r7dx2K+hTlw7Yfsuu5XdQoVYN+y/rRcnZL9iXsM0RPXj3SzObinFdc0kAYjaenJ2FhYdSuXZvQ0FA+/vhjUvI5afuIESNYs2ZNlv+Pjo7miy++yK/UNI4cOYKvry9hYWGEhobStGlT9u/fX+Dz5kTRokXtfg2N8xNSLoTNT21mRpcZ7E3YS9i0MF5b8xpXb111qI68eqSZ0cU5TyilnPavfv36KiOxsbF3bXM0RYoUSft8+vRp1bp1azVixAgDFeXM4cOHVe3atdPWo6OjVUREhN2vm/5Z5URKSopKTk7O1b5meA809iHhcoLqt7SfYhSqyrgq6tv93xotKVuSk1NUwsXr6sTZKyrh4nWVnJxitCQF7FC5KGN1C8LOlC1blunTpzNp0iSUUiQnJ/Pyyy/ToEEDQkJCmDZtWtq+H3zwAcHBwYSGhvLqq68C0K9fPxYtsgSYv/rqqwQFBRESEsJ//vMfAEaNGsXYsWMBiImJoXHjxoSEhNC9e3fOnTsHQMuWLRk+fDgNGzbk/vvv56effspR98WLFylZ0jIR/fXr1+nfvz/BwcHUrVuX9estid4+//xzoqKi0o7p3LkzGzZsACwtgzfeeIPQ0FAaN27M6dOnATh8+DBNmjShQYMGvPXWW2nHXr58mdatW1OvXj2Cg4NZtmwZYGnZ1KpVi8GDB1OvXj1GjX6bIUOHcis5xTJp04wZvPjii3n8VjTOTECRAGZ1m8Wmfpso6lOULvO60H1Bd45dOJbzwQZgdFBiQTCTF5PNGbZyGDF/x9j0nGHlwxjffnyejrnvvvtISUkhISGBZcuWUbx4cX799Vdu3LhBs2bNaNu2LfHx8SxdupRt27bh5+fH2bNn7zjH2bNnWbJkCfHx8YgI58+fv+s6ERERfPLJJ7Ro0YIRI0YwevRoxo+3aE1KSmL79u2sWLGC0aNHZ9ptdfDgQcLCwrh06RJXr15l27ZtAEyePBmAPXv2EB8fT9u2bfnjjz+yvecrV67QuHFj/vvf//LKK68wY8YM3nzzTYYOHcqgQYOIiIhIOy9A4cKFWbJkCcWKFeOff/6hcePGdO3aFYD9+/fz2Wef8dH4T4g7nkC3B5vy7Esj+Ff5EsyaNesOI6txH/5d5d/sem4X47aOY/TG0dSaXIvRLUcztNFQU8VOODO6BeEglNXzYtWqVXzxxReEhYXRqFEjzpw5w4EDB1izZg39+/fHz8+SbrlUqVJ3HF+sWDEKFy7MM888wzfffJO2XyoXLlzg/PnztGjRAoC+ffuyadOmtP8/8sgjANSvX58jR45kqrF69erExMRw8OBBxo8fz4ABAwDYvHkzffr0AaBmzZpUqVIlRwPh4+ND586d77rmli1b6NWrF0DaOVOfz+uvv05ISAht2rThr7/+Smt1VKlShfCGjTh65gpehXxp2Kw5a378gfW//MbNW7cIDg7OVovGdfHx9GH4A8OJjYyldbXWvLz6ZepNr8eWY1uMluYSuHQLIq81fXtx6NAhPD09KVu2LEopPvnkE9q1a3fHPitXrszW+8bLy4vt27ezdu1a5s+fz6RJk1i3bl2uNRQqZPG59vT0JCkpKYe9oWvXrvTv3x+4bdwy05R+8D29a6m3t3fa/WS8Zmb3OXfuXBITE9m5cyfe3t5UrVo17XxFihRBKcXNZMu1uvfqw8xPPqbqv2oQEdE3x3vRuD5VS1Rlea/lLItfxvM/PM8Dsx7g6bpPM6bNGMNjJ5wZ3YKwM4mJiQwcOJCoqChEhHbt2jF16lRu3boFwB9//MGVK1do27Ytn332GVevWrwyMnYxXb58mQsXLtCxY0fGjx9PTMydXWfFixenZMmSaeMLc+bMSWtN5IfNmzdTvXp1AJo3b87cuXPT9B47dozAwECqVq1KTEwMKSkpHD9+nO3bt+d43mbNmjF//nyAtHOCpQVUtmxZvL29Wb9+PUePHr3jOBHBx9PyuobUDefvU3+xculiHre2RjQagG41uxEbGcvLTV9m9u+zCZwUyKzfZpGi8udF6O64dAvCKK5du0ZYWBi3bt3Cy8uLPn36pA2kPvPMMxw5coR69eqhlCIgIIClS5fSvn17YmJiCA8Px8fHh44dO/Lee++lnfPSpUt069aN69evo5Ri3Lhxd1139uzZDBw4kKtXr3Lfffcxa9asPOlOHYNQSuHj48Onn34KwODBgxk4cCDBwcF4eXnx+eefU6hQIZo1a0a1atUIDg6mTp061KtXL8drTJgwgSeeeIIJEybw6KOPpm3v3bs3Xbp0ITw8nLCwMGrWrHnHcV4eQpXSRdKipzt26c6xP+MIKF0q4yU0bk5Rn6J88NAHRIRGMOj7QTy1/Ck+i/mMqZ2mUqdsHaPlORUuN+VoXFwctWrVMkiRxp4opUhKsbjfde/WlRdeeIE2bdpkuq9+DzQAKSqFz2M+55XVr3DhxgVebPwiI1qMoIhPEaOlGYozzCin0eQJEeHKpYvUCaqFn59flsZBo0nFQzx4qu5TxEfF0ze0Lx/8/AFBU4JYFr/MaGlOgTYQGqeiRIkS/PHHHyxcuNBoKRonooxfGT7t+imb+2+mWKFiPLzgYbrN78bR80dzPtiN0QZCo9G4Dc0qN2PXgF180OYD1hxaQ9CUID7Y8gG3km8ZLc2UaAOh0WjcCm9Pb15u9jJxkXG0rd6W4WuGU3daXX46mnOGAXdDGwiNRuOWVC5emSU9l7D88eVcvnmZ5p8356llT/HP1X+MlmYatIHQaDRuTZfALuwbvI/hzYYzZ/ccAicF8umuT3XsBNpA2AUR4aWXXkpbHzt2LKNGjcr2mKVLlxIbG5vtPqGhoWlpKjLjyJEj1KmTNz/v9OnEx48fnxaoB9wRh5FbMibw05iXFOvUrX+du0ripRukpDivy3tBKeJThPfbvE/MczHUDqjNs98+y79n/Zs9p/cYLc1QHG4gRKSwiGwXkd9FZJ+IjLZuryYi20TkgIgsEBEnSZh+N4UKFeKbb77hn39y31TNyUDExcWRkpLCpk2buHLlii1kAvD222+nuYvawkBonIPUuZK7T9lCszHr6T5lC/tPX3JrIwFQu2xtNvbbyKxus/jjzB/UnVaXl1e9zOWbl42WZghGtCBuAA8qpUKBMKC9iDQGxgDjlFI1gHPA044QY49alJeXFwMGDMg02vno0aO0bt2akJAQWrduzbFjx/j5559Zvnw5L7/8MmFhYRw8ePCu47766iv69OlD27ZtWb58edr2nTt3EhoaSpMmTe7Ijvr555/z8MMP06VLF6pVq8akSZP4+OOPqVu3Lo0bN05L5ZGaTnzixImcPHmSVq1a0apVK1599dW0iPDevXsD8OWXX9KwYUPCwsJ47rnnSE62TJs4a9Ys7r//flq0aMGWLTpJmjOQ1VzJZ67cNFiZ8YgI/cL6ER8ZT/+w/ozdOpagyUEsiVti6HSnRuBwA2GdryLVHHtb/xTwILDIun028LC9tdizFhUZGcncuXO5cOHCHdujoqKIiIhg9+7d9O7dmyFDhtC0aVO6du3Khx9+SExMTFoOpPQsWLCAnj170qtXL+bNm5e2vX///kycOJGtW7fedczevXv56quv2L59O2+88QZ+fn789ttvNGnS5K5Z6IYMGUKFChVYv34969ev5/3338fX15eYmBjmzp1LXFwcCxYsYMuWLcTExODp6cncuXM5deoUI0eOZMuWLaxevTrHbjKNOXD2uZIdQWm/0szoOoMtT22hROESPPL1I3Sd35Uj548YLc1hGDIGISKeIhIDJACrgYPAeaVUasrPE0DFLI4dICI7RGRHYmJigXTYsxZVrFgxIiIimDhx4h3bt27dyhNPPAFY0l1v3rw5x3P9+uuvBAQEUKVKFVq3bs2uXbs4d+7cXSm+06fPBmjVqhX+/v4EBARQvHhxunTpAkBwcHCWKb+zYu3atezcuZMGDRoQFhbG2rVrOXToENu2baNly5YEBATg4+NDz54983RejTE4+1zJjqTpvU3ZOWAnYx8ay/rD6wmaHMT/fvofN5Ndv7VliIFQSiUrpcKASkBDILOkOZlW45VS05VS4Uqp8ICAgALpsHctatiwYcycOTPbMYPsUnynMm/ePOLj46latSrVq1fn4sWLLF68GKVUtsenpvgG8PDwSFv38PDIVcrv9Cil6Nu3LzExMcTExLB///60gffc3IPGXDj9XMkOxtvTm5eavkRcZBwdanTg9XWvExYdxsYjG42WZlcM9WJSSp0HNgCNgRIikppdthJw0t7Xt3ctqlSpUvzf//0fM2fOTNvWtGnTO9JdP/DAAwD4+/tz6dKlu86RkpLCwoUL2b17N0eOHOHIkSMsW7aMefPmUaJECYoXL57WCkmfPjs/ZNTg7e2dlpa8devWLFq0iISEBMCSjvzo0aM0atSIDRs2cObMGW7duqVTYDgJHh5CYDl/lgxuxpbhrVgyuBmB5fydajpMI7i3+L0s/r/FfNfrO64lXaPl7Jb0W9qPxCsF680wK0Z4MQWISAnrZ1+gDRAHrAces+7WF7B7Ni1H1KJeeumlO7yZJk6cyKxZswgJCWHOnDlMmDABgMcff5wPP/yQunXr3jFIvWnTJipWrEjFird73Jo3b05sbCynTp1i1qxZREZG0qRJE3x97zR2eWXAgAF06NCBVq1apa2HhITQu3dvgoKCePfdd2nbti0hISE89NBDnDp1invuuYdRo0bRpEkT2rRpk6uU3xpz4MxzJRtNp/s7sW/wPl574DW+2vMVgZMCmb5zusvFTjg83beIhGAZhPbEYqC+Vkq9LSL3AfOBUsBvwJNKqRvZncsW6b5TUhRnrtzkZlIyPl6elC7io38oLoBO961xFHGJcQz6fhAbj26kSaUmTO00ldDyoUbLyhbTpvtWSu1WStVVSoUopeoopd62bj+klGqolPqXUqpHTsbBVuhalEajKQi1Amqxvu96Zj88mz/P/kn96fV58ccXuXTj7i5jZ0NHUms0Gk0BEREiQiOIj4rnmXrPMO6XcdSaXIvFsYudOnZCGwiNRqOxEaV8SxHdOZqtT2+ljF8ZHlv4GJ3ndebwucNGS8sX2kBoNBqNjWlcqTE7Buzg47Yfs+noJoKmBPHeT+85XeyENhAajUZjB7w8vHihyQvERcbR+f7OvLHuDUKjQ9lwZIPR0nKNNhAajUZjRyoVq8TCHgtZ8cQKbiTdoNXsVkQsiSDhSoLR0nJEGwg7sWTJEkSE+Pj4TP+fmiQvt5w8eZLHHrOEicTExLBixYq0/23YsIGff/45zxqrVq2ap4yzGo0m/3So0YF9g/fx5r/fZP7e+QROCmTajmmmjp3QBsJOzJs3jwceeCAtarqgVKhQIc2g2MpAaDQax+Lr7cs7D77D7kG7CSsfxsDvB9J0ZlNi/o4xWlqmaAORkgKXT8P545ZlSsGt+eXLl9myZQszZ85MMxBKKaKioggKCqJTp05pKSvAUpN//fXXadKkCeHh4ezatYt27dpRvXp1oqOjgduTAd28eZMRI0awYMECwsLCGDNmDNHR0YwbN46wsDB++uknEhMTefTRR2nQoAENGjRIS8F95swZ2rZtS926dXnuueec2v1Oo3FmapapybqIdczpPofD5w9Tf3p9Xlj5gvliJ5RSTvtXv359lZHY2Ni7tmVJcrJSp/YoNa6OUiOLWZan9li2F4A5c+aop556SimlVJMmTdTOnTvV4sWLVZs2bVRSUpL666+/VPHixdXChQuVUkpVqVJFTZkyRSml1LBhw1RwcLC6ePGiSkhIUAEBAUoppQ4fPqxq166tlFJq1qxZKjIyMu16I0eOVB9++GHaeq9evdRPP/2klFLq6NGjqmbNmkoppZ5//nk1evRopZRS3333nQJUYmJige7VrOTpPdBoDOTs1bNq4LcDlYwSVeGjCmrhvoUqJSXFrtcEdqhclLFeORkQl+ZqIszvBeePWdbPH7OsP7MGipbL92nnzZvHsGHDAEuOpXnz5nHr1i169eqFp6cnFSpU4MEHH7zjmK5duwKWVNyXL1/G398ff39/ChcuzPnz5/N0/TVr1twxL8PFixe5dOkSmzZt4ptvvgGgU6dOlCxZMt/3qNFobENJ35JM7TyVfmH9GPT9IHos7EH7f7VnUodJVC9199wwjsS9DUTSzdvGIZXzxyzb88mZM2dYt24de/fuRURITk5GROjevXuuUnOnT8udup7X1NwpKSls3bo10+R9OjW3RmNOGlVqxPZntzN5+2TeWv8WdabW4Y1/v8HLTV+mkFehnE9gB9x7DMLLB0pUvnNbicqW7flk0aJFREREcPToUY4cOcLx48epVq0apUqVYv78+SQnJ3Pq1CnWr1+f72tkTMudcb1t27ZMmjQpbT0mxjIA1rx587SU4D/88APnzp3LtwaNRmN7vDy8GNp4KHGRcXS5vwtvrX+L0OhQ1h1eZ4ge9zYQfgHw+LzbRqJEZcu6X/4nIpo3bx7du3e/Y9ujjz7K33//TY0aNQgODmbQoEFps8Dlh1atWhEbG0tYWBgLFiygS5cuLFmyJG2QeuLEiezYsYOQkBCCgoLSBrpHjhzJpk2bqFevHqtWraJy5co5XEmj0RhBxWIV+brH1/zQ+weSUpJo/UVrnvzmSU5fPu1QHQ5P921LbJHum5QUy1hE0k1Ly8EvADzc2266Ajrdt8ZVuHbrGv/b/D/GbBmDr5cvc7rPoUtglwKd07Tpvk2Hh4dlQLrEvZalNg4ajcZE+Hr78nart9k9cDdN723K/aXvdyT79bsAAAfESURBVNi13XuQWqPRaJyEwDKBrOi9IucdbYiuLms0Go0mU4yYk/peEVkvInEisk9Ehlq3lxKR1SJywLrMt5O+M4+raAqO/v41GttgRAsiCXhJKVULaAxEikgQ8CqwVilVA1hrXc8zhQsX5syZM7qQcFOUUpw5c4bChQsbLUWjcXocPgahlDoFnLJ+viQicUBFoBvQ0rrbbGADMDyv569UqRInTpwgMTHRJno1zkfhwoWpVKmS0TI0GqfH0EFqEakK1AW2AeWsxgOl1CkRKZvFMQOAAUCmfvze3t5Uq1bNToo1Go3GfTBskFpEigKLgWFKqYu5PU4pNV0pFa6UCg8IyH9Am0aj0WiyxxADISLeWIzDXKXUN9bNp0XkHuv/7wHMP92SRqPRuDBGeDEJMBOIU0p9nO5fy4G+1s99gWWO1qbRaDSa2zg81YaIPAD8BOwBUmfneR3LOMTXQGXgGNBDKXU2h3MlAkcLIKcM4Cxzbmqt9kFrtQ9aq/2whd4qSqkc++idOhdTQRGRHbnJR2IGtFb7oLXaB63VfjhSr46k1mg0Gk2maAOh0Wg0mkxxdwMx3WgBeUBrtQ9aq33QWu2Hw/S69RiERqPRaLLG3VsQGo1Go8kCtzAQIlJYRLaLyO/WDLKjrduricg2awbZBSKS/8mobYyIeIrIbyLynXXdzFqPiMgeEYkRkR3WbTbLzmtLRKSEiCwSkXhrRuEmZtQqIoHW55n6d1FEhplRK4CIvGD9be0VkXnW35wp31kRGWrVuU9Ehlm3meK5ishnIpIgInvTbctUm1iYKCJ/ishuEalnaz1uYSCAG8CDSqlQIAxoLyKNgTHAOGsG2XPA0wZqzMhQIC7dupm1ArRSSoWlc7+zSXZeOzABWKmUqgmEYnnGptOqlNpvfZ5hQH3gKrAEE2oVkYrAECBcKVUH8AQex4TvrIjUAZ4FGmL5/juLSA3M81w/B9pn2JaVtg5ADevfAGCqzdUopdzqD/ADdgGNsASbeFm3NwF+NFqfVUsl64vwIPAdIGbVatVzBCiTYdt+4B7r53uA/SbQWQw4jHXszcxaM+hrC2wxq1Ys2ZiPA6WwJAD9DmhnxncW6AF8mm79LeAVMz1XoCqwN916ptqAaUCvzPaz1Z+7tCBSu2xisOR4Wg0cBM4rpZKsu5zA8qKbgfFYXtrUSPPSmFcrgAJWichOa7ZdyJCdF8g0O6+DuQ9IBGZZu+8+FZEimFNreh4H5lk/m06rUuovYCyWDAingAvATsz5zu4FmotIaRHxAzoC92LC55qOrLSlGuZUbP6M3cZAKKWSlaW5XglL87JWZrs5VtXdiEhnIEEptTP95kx2NVxrOpoppephafJGikhzowVlgRdQD5iqlKoLXMEEXTTZYe237wosNFpLVlj7xLsB1YAKQBEs70JGDH9nlVJxWLq+VgMrgd+xTGLmjNi9XHAbA5GKUuo8lsmIGgMlRCR1ToxKwEmjdKWjGdBVRI4A87F0M43HnFoBUEqdtC4TsPSTN8Sc2XlPACeUUtus64uwGAwzak2lA7BLKXXaum5GrW2Aw0qpRKXULeAboCkmfWeVUjOVUvWUUs2Bs8ABzPlcU8lK2wksrZ9UbP6M3cJAiEiAiJSwfvbF8kLHAeuBx6y7mSKDrFLqNaVUJaVUVSxdC+uUUr0xoVYAESkiIv6pn7H0l+/FhNl5lVJ/A8dFJNC6qTUQiwm1pqMXt7uXwJxajwGNRcRPRITbz9Ws72xZ67Iy8AiW52vG55pKVtqWAxFWb6bGwIXUriibYfSgkYMGfUKA34DdWAqvEdbt9wHbgT+xNOELGa01g+6WwHdm1mrV9bv1bx/whnV7aSwD7Qesy1JGa7XqCgN2WN+FpUBJE2v1A84AxdNtM6vW0UC89fc1Byhk4nf2JywG7HegtZmeKxZjdQq4haWF8HRW2rB0MU3GMp66B4sXmU316EhqjUaj0WSKW3QxaTQajSbvaAOh0Wg0mkzRBkKj0Wg0maINhEaj0WgyRRsIjUaj0WSKNhAajcFY0z6sF5HLIjLJaD0aTSpeOe+i0WjszHUsSePqWP80GlOgWxAaTRaIyJNimUckRkSmiUgVa07+MiLiISI/iUhb675LrckK96VLWIi1VTDG+r81ItJQRDaIyCER6QqglLqilNqMxVBoNKZBGwiNJhNEpBbQE0siwjAgGWiBJdFbNPASEKuUWmU95CmlVH0gHBgiIqWt24sAG6z/uwS8CzwEdAfedtT9aDT5QXcxaTSZ0xrLRD2/WtIL4Ysly+4oEekBDMSStiOVISLS3fr5XiyTuJwBbmLJGgqWdAg3lFK3RGQPlrz/Go1p0QZCo8kcAWYrpV67Y6NlDoFK1tWiwCURaYklAWQTpdRVEdkAFLbuc0vdzmeTgmV2Q5RSKekynWo0pkR3MWk0mbMWeCxd5s9SIlIFSxfTXGAEMMO6b3HgnNU41MSSSl6jcXp0DUajyQSlVKyIvIllpjwPLNk1XwQaYBmXSBaRR0X+v707NEIgCIIo2mOQhHWaFHBERixoxKWziEO2gTr5XgJbo37Vipm5J3kmeczMO8fZx9ev733vf1yTXGbmlmRba+0njQN/sc0VgMoXEwCVQABQCQQAlUAAUAkEAJVAAFAJBACVQABQfQB72SCKvyolcAAAAABJRU5ErkJggg==\n", "text/plain": [ "
" ] }, "metadata": { "needs_background": "light" }, "output_type": "display_data" } ], "source": [ "sns.scatterplot(x='exam1',y='exam2',data=data[data['decision']==0],label='Not Admitted');\n", "sns.scatterplot(x='exam1',y='exam2',data=data[data['decision']==1],label='Admitted');\n", "# Only need two points to define a line, so choose two endpoints\n", "x1=np.array([min(X[:,1]),max(X[:,1])])\n", "x2=np.multiply((-1/theta[0][2]),(theta[0][0]+np.multiply(theta[0][1],x1)))\n", "# De-normalizing the data\n", "x1=x1*data['exam1'].std()+data['exam1'].mean()\n", "x2=x2*data['exam2'].std()+data['exam2'].mean()\n", "sns.lineplot(x=x1,y=x2,label='Decision Boundary',color='green');" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "##### 1.2.4    Evaluating Logistic Regression" ] }, { "cell_type": "code", "execution_count": 16, "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
actualprediction
6400.075530
7310.878757
6300.000157
1000.905205
4910.999988
2110.998391
4010.967775
3800.209379
9910.999695
610.998787
\n", "
" ], "text/plain": [ " actual prediction\n", "64 0 0.075530\n", "73 1 0.878757\n", "63 0 0.000157\n", "10 0 0.905205\n", "49 1 0.999988\n", "21 1 0.998391\n", "40 1 0.967775\n", "38 0 0.209379\n", "99 1 0.999695\n", "6 1 0.998787" ] }, "execution_count": 16, "metadata": {}, "output_type": "execute_result" } ], "source": [ "h=predict(X,theta).reshape(m)\n", "result=pd.DataFrame({'actual':data['decision'],'prediction':h})\n", "result.sample(10)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Calculating the admission probability of student with Exam 1 score of 45 and Exam 2 score of 85." ] }, { "cell_type": "code", "execution_count": 17, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "ADMISSION PROBABILITY : 0.770\n" ] } ], "source": [ "x=[1,45,85]\n", "# Normalizing the data\n", "x[1]=(x[1]-data['exam1'].mean())/data['exam1'].std()\n", "x[2]=(x[2]-data['exam2'].mean())/data['exam2'].std()\n", "print('ADMISSION PROBABILITY : %.3f'%predict(x,theta)[0])" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Calculating the training accuracy." ] }, { "cell_type": "code", "execution_count": 18, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "TRAINING ACCURACY : 89.0%\n" ] } ], "source": [ "p=np.round(h).reshape(m,1)\n", "accuracy=np.mean(y==p)*100\n", "print(f'TRAINING ACCURACY : {accuracy}%')" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### 2    Regularized Logistic Regression\n", "\n", "##### Problem Statement\n", "In this part of the exercise, we will implement regularized logistic regression to predict whether microchips from a fabrication plant passes quality assurance (QA). During QA,e ach microchip goes through various tests to ensure it is functioning correctly.\n", "\n", "Suppose we are the product manager of the factory and we have the rest results for some microchips on two different tests. From these two tests, we would like to determine whether the microchips should be accepted or rejected. To help us make a decision, we have a dataset of test results on past microchips, from which we can build a logistic regression model." ] }, { "cell_type": "code", "execution_count": 19, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "TRAINING DATASET SHAPE : 118 X 3\n" ] }, { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
test1test2result
114-0.5938900.494880
160.1664700.538741
920.1031100.779970
380.062788-0.163011
790.085829-0.755120
\n", "
" ], "text/plain": [ " test1 test2 result\n", "114 -0.593890 0.49488 0\n", "16 0.166470 0.53874 1\n", "92 0.103110 0.77997 0\n", "38 0.062788 -0.16301 1\n", "79 0.085829 -0.75512 0" ] }, "execution_count": 19, "metadata": {}, "output_type": "execute_result" } ], "source": [ "data=pd.read_csv('./ex2data2.csv')\n", "print(f'TRAINING DATASET SHAPE : {data.shape[0]} X {data.shape[1]}')\n", "data.sample(5)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "#### 2.1    Visualizing the data" ] }, { "cell_type": "code", "execution_count": 20, "metadata": {}, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAZQAAAEKCAYAAAA1qaOTAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvOIA7rQAAIABJREFUeJztnXm0FPWV+D8XZBEFWTXoA0HHER1FwBdHxCQSwWVUkBEd9GckbiSYzcnJHPGX/HTiaILCOVEykgSXiJqASlzIGTNuoDl5auJjBhQJm4ryhMDLQ8xDkOW9+/ujqrHo13vX2nU/5/Tp7qpvdd2u7qpbd/neK6qKYRiGYVRLp6gFMAzDMGoDUyiGYRiGL5hCMQzDMHzBFIphGIbhC6ZQDMMwDF8whWIYhmH4gikUwzAMwxdMoRiGYRi+YArFMAzD8IWDohYgTPr3769DhgyJWgzDMIxEsWzZsr+q6oBi41KlUIYMGUJjY2PUYhiGYSQKEXm/lHHm8jIMwzB8wRSKYRiG4QumUAzDMAxfMIViGIZh+IIpFMMwDMMXUpXlZRjZtLcrLZ/sYc++Nroe1Jl+h3SlUyeJWizDSCSmUIzU0t6urNnSyvUPN9L00S7q+hzMfVfVc/wRPU2pGEYFmMvLSC0tn+zZr0wAmj7axfUPN9LyyZ6IJTOMZGIWipFa9uxr269MMjR9tIs9+9oikihYzL1nBI0pFCO1dD2oM3V9Dj5AqdT1OZiuB3WOUKpgMPeeEQbm8jJSS79DunLfVfXU9TkYYP9Ftt8hXSOWzH/MvWeEgVkoRmrp1Ek4/oiePHXDmJp3A6XNvWdEgykUI9V06iQM6NktajECJ03uPSM6zOVlGCkgTe49IzrMQjGMFJAm954RHaZQDCMlpMW9Z0RHpC4vEXlQRLaKyMo860VE5ojIehF5U0RGedZNFZF17mNqeFIbhmEYuYg6hvIQcF6B9ecDx7mPacDPAESkL3Ar8I/AacCtItInUElrkPZ2pbl1Nx9+tJPm1t20t2vUIsUeO2aGkZ9IXV6q+nsRGVJgyETgYVVV4HUR6S0iA4GzgBdUdRuAiLyAo5gWBCtx7ZDWiW7VzBZP6zEzjFKJ2kIpxlHARs/7JndZvuVGiaRxoltGIUya28CYO5cyaW4Da7a0lmxlpPGYGUY5xF2h5Lrt0wLLO36AyDQRaRSRxubmZl+FSzJpnOhWrUJI4zEzjHKIu0JpAgZ53tcBmwos74CqzlPVelWtHzBgQGCCJo3MRDcvkU50a2+HHVtg+0bnub3d911UqxBid8wMI2bEXaEsBq5ys71OBz5W1c3Ac8A5ItLHDcaf4y4zSiRWE93a22HrKrh/HNx9kvO8dZXvSqVahRCrY2YYMUSceHdEOxdZgBNg7w9swcnc6gKgqj8XEQH+EyfgvhO4WlUb3W2vAf6v+1F3qOovi+2vvr5eGxsb/f4aiSU25cx3bHGUyPYPPlvWezBc9yIceoRvu/EjqB6bY2YYISIiy1S1vui4KBVK2JhCiSnbNzqWSTY3roTegzour4KkKoSkym3UBqUqFJspb1SFLxe6g7o6Fkm2hXKQ/66ksGaL+6kALF3ZSApxj6EYMabaNNz99BgAUxY4SgSc5ykLnOUJxLfj4mLpykZSMIWSMvyc6e3bha5TJzj8RCdmcuNK5/nwE53lCcRvBWDpykZSMJdXivDbdeLrha5TJ18D8FHitwKwXialYXGm6EnmLaBREX7fOdu8jNz4fVwsXbk4frsZjcowhZIi/L5ztgtdbvw+Lt5eJg03jeWpG8ZYQD4LizPFA3N5pQi/XSfWtCk3QRwX62VSGIszxQOzUFJEEBZF5kJ3VJ8eDOjZLfXKJIMdl3Ax92s8sImNKcMCl0YtYnN1gsUmNho5MdeJUYuY+zUemEIxDKNk4mzh2s1S9JhCMYxctLfDzmbYt8cpAdNjQGInWvqFuZWMYqT7DDGMXIRUTj9pWGquUQxTKEbt4FeTrp3NsPDyz4pVbv/Aeb8z3R0/LTXXKIYpFKM28NOq2LfnwMrH4Lzfl+47cUvNNYphCsWIBr9b/vppVWTK6XvxsZy+nwU6g/i8fFhlBKMYFpQ3widjTWQUQKZcfTUVhv20KjLl9LPl86Gcvt+B7TAD5ZaaaxQjUgtFRM4TkTUisl5EZuRY/xMRWe4+1orIds+6Ns+6xeFKblRFEDEKP62KAMvp+x3YDjtQ7lcFgLCsKiNcIrNQRKQzcC8wHmgC3hCRxaq6KjNGVf/VM/5bwEjPR+xS1RFhyWv4SBAxCr+tioDK6fsd2E5ioNzSj2uXKC2U04D1qvququ4BFgITC4y/HFgQimRGsAQRo0hIky6/A9tJDJRb+nHtEuXZdhSw0fO+yV3WARE5GhgKLPEs7i4ijSLyuohcHJyYhu8E1fI3Y1X0HuQ8x0yZgP+B7SQGyv2yqsxtFj+iDMrnsm3z/SOmAItU1fuPG6yqm0TkGGCJiLylqu902InINGAawODBg7NXG1HgtSZSNhPd78B2EgPlfrRRMLdZPInyDG4CBnne1wGb8oydQpa7S1U3uc/vAi9zYHzFO26eqtarav2AAdVn6Rg+kQBrIij8Lm2ftFL5flhV5jaLJ1FaKG8Ax4nIUOBDHKVxRfYgETke6AO85lnWB9ipqrtFpD8wBrgrFKkTQJAF/GJZHDBudbfiJk/M8MOqSmIyQhqITKGo6j4R+SbwHNAZeFBV3xaR24BGVc2kAl8OLNQDG7ecAPxCRNpxrKyZ3uywNBOkKyByN0OuCzX4P6elWhnjJE8xIlJ+1VYG9rv7qOEP1mArAoK8y29u3c2kuQ0dTrSnbhhTdWnvID+7KPku1IcMgAfGHZiG3HuwE58plPYb1IV0xxan7Eu58kRB0pSfh8hvblKGNdiKKUGfCEG6AiJ1M+SbDDn1t+XPaQnyQpqkOmD5jmk1yi8kiyeJyQhpIN63ITVI0MHEIOclRDrnId+FWjqXP6clyGrCAdcB8xW/lV/IZf/jkoxg6cufYQolZIK+yw9yXkKkcx7yXai7HFz+nJYgrYig5tgEgd/KL4Vl/zMeh0lzGxhz51ImzW1gzZbW1CoVc3mFTNDBxCBdAZG6GfKVVjm4L3TvDVf/Dtr2QucucOjnCrtZMhfS7DiHH1ZEkubY+F2uJknuPp/I53EIJa4YQ0yhhEzmLj87huLnXX5ZGTRl+rwj69ud70IN0Ly6vHhIgNWE98satwB8LvxWfkEq6ioIMgnG0pcPxBRKyMQqmJi0LJ9cF+odW8oPLCfJiggaP5Vf0Iq6AoJOgrH05QOxtOE0k6QU13xs3+gEgLO5caUzC9/oQKCTU2M2qTPoVPe0pC9b2rBRnFrwecfUzRJXAr8AxszdF7RLKlYehxiQQhs/4fjZOjdJKa75SFJWVQyolRpYpabqhpHqHpf05ThgFkqS8DvmEUOfd9lYPKQsaiGIXI6VFUYSjPEZFkNJEkHEPGLm8041IfwW2TGFkYN6cdu4z3Hi4d3o3KVbIn7/cuMisSxomjAshlKLBBHziJnPO7WElHHnvWMfcGgXfvlPh9D7mX9ORpafS7lWVmSp7ikkvv8aoyO1EPMwchPSLHNvEHnh//k7ej8zNXEz25PY9jgtmEJJEhaA9g8/kxv8IMSMu8wdezf2JTLLL4ltj9OCubxCwhc/rgWgyyNfTCKOEzrzpT937uoovCB+74SmXFuqbnyxK1EI+FpALsWtc8uiUOXbOBYxzGd97m4Nrnpvgi1eS9WNJ5blFQKRNqZKK4Uy4vbtiefs+myLSjrDfWOLZ/VVkx1mWX5GCViWV4yohdz/xFEoJhFXV092xt32jcVjHNW67yzLz/CRSG9FROQ8EVkjIutFZEaO9V8VkWYRWe4+rvOsmyoi69zH1HAlL4+az0qJW4AbCmfEJcXVU0pWXxzddzWCNc4qn8gsFBHpDNwLjAeagDdEZLGqrsoa+piqfjNr277ArUA9oMAyd9uPQhC9bIKYrRubyVpxDHBD4SoASUluKKWSQS3UY4shaSn66DdRurxOA9ar6rsAIrIQmAhkK5RcnAu8oKrb3G1fAM4DFgQka1X4nZUSqz97EH3J/aCY0kiCq6cUxRdX912lxCSmY42zKiPKW7KjgI2e903usmwuEZE3RWSRiGQipqVui4hME5FGEWlsbo7ODeBnVkqsCvxVe4ccpLusFjLiin2HpLjvSiHknvSFsLhnZUR5huW6omY7KX8LDFHV4cCLwPwytnUWqs5T1XpVrR8wIIEnWQ4q/rMHcfGuZvZ+jC4gicVrxdy40nmO2t1YKTGKB9V83DMgovzXNQHeHM06YJN3gKq2qOpu9+19wKmlblvLVPRnD+riXc0dcowuIImmFiwxiFU8yGbjV0aUMZQ3gONEZCjwITAFuMI7QEQGqupm9+0E4M/u6+eAH4lIH/f9OcDNwYscDyoK8gcV66gmwB2jC4gRA6KKB+WI23Tq1Mlm41dAZApFVfeJyDdxlENn4EFVfVtEbgMaVXUx8G0RmQDsA7YBX3W33SYi/4GjlABuywTo00BFQf4gL96VBrhrLaBsVEcU/XkKZCl26tTJAvBlYjPl00Ic+8fHNeXYiI6ws7zieF7EEJspbxxIHLszJmU+iBEeYadzm9vVV0yhpIW4XryTMB/EqF3M7eordiuYJmolGyhK4lhmxqicWprHEwPMQjGMUrGYT+0RV8s9odhRM4xSsXkztYlZ7r5hR84wSsUCuIZREHN5GckiyuKBFsA1AiA2lcN9wBSKkRyijmHEMfXaSDSxqhzuAzax0UgOcZiEFpPy6kZtkJT24DaxsVawC9hnxCGGYfNmDB+ptTL5Kb0yJQQr734g1ZTKN4wYUmtl8k2hFCHSvtJJS1MNetKfTUIzaoxaK5NvLq8CRB4wi4OLp1TCCJjbJDSjxvC7PXjU2JlYgEpa7fpq0STJxROWNWWT0Iwaw8/24FFjZ2MByg2YZSyaSXMbGHPnUibNbWDNltbKlUqSXDxJsqYMwwgEc3kVIBMwy07pyxcwy2fRVJwCmCQXj036M1JKLU1MrJYYXpniQ7kBs0BSAJPi4kmSNWUkixhXePbdK5FwIrVQROQ84B6cFsD3q+rMrPXfBa7DaQHcDFyjqu+769qAt9yhH6jqBL/lKzdgVq5FU1MkyZoykkPU1RGK4LtXIuFE9ouISGfgXuB84ETgchE5MWvY/wL1qjocWATc5Vm3S1VHuA/flUmGcgJmtZYCWDZJsaaM5BDz1Plam5hYLQUtFBE5F6gDXlLVDZ7l16jqg1Xu+zRgvaq+637mQmAisCozQFWXesa/DlxZ5T4DpdZSAA0jcmKe7JFqr0QO8t5CisiPgO8DJwMvici3PKu/6cO+jwI2et43ucvycS3wO8/77iLSKCKvi8jFPsjjC7WUAmgYkRPz1PnUeyWyKGShXASMVNV9IvLvwK9F5BhV/VfAj6tkrs/IGckSkSuBeuBLnsWDVXWTiBwDLBGRt1T1nRzbTgOmAQwePDh7tVEIqyNmRE3MKzybV+JACimUg1R1H4CqbheRi4B5IvIE4If6bQIGed7XAZuyB4nIOBxL6UuqujuzXFU3uc/visjLwEigg0JR1XnAPHCqDfsgd/KoRDHEPBhqpIQEJHtkvBJG4aD8OyKy3yJQ1TZVvRZYA5zgw77fAI4TkaEi0hWYAiz2DhCRkcAvgAmqutWzvI+IdHNf9wfG4Im9GB4qLTAZ82CokSIs2SMxFPplLgX+lL1QVX/AgZZFRbjWzzeB54A/A4+r6tsicpuIZLK2ZgGHAk+IyHIRySicE4BGEVkBLAVmqqoplFxUqhhiHgw1DCN+5HV5qeouABF5SVXPzlr9MJC9rGxU9Vng2axlt3hej8uz3as4yQJGMSpVDDbz3TCMMimU5dVdRPoC/V0XU1/3MQQ4MiwBjSqpNEsmrjPfYzxr2jDSTqGg/NeAG3GUxzI+y8r6G86ERCMJVJolE8dgqCUKGLmwbMTYULSnvIh8S1V/GpI8gZLanvK1csLFoae8ES/sJiMUSu0pX8oR/4uI9HQ/9Aci8qSIjKpaQiM8aiVLxhIFjGwsGzFWlHJl+X+q2ioiZwLnAvOBnwUrlmHkIOazpo0IsJuMWFGKQslUObsA+JmqPoM/ExsNozzimihgRIfdZMSKUsrXfygivwDGAXe6EwoT6jMxEk0cEwWMaIl5aZa0UYpCuQw4D5jtlmAZCPxbsGIZRh4y8SDDALvJKELY3SSLKhRV3SkiW4EzgXU4za7WBSaRYRhGOdhNRk4y3SQzDcAylZCPP6JnYEqlqBoXkVuBm4Cb3UVdgEcDkcYwDMMAHIXQ3LqbDz/aSXPr7rLbCufrJtnySXAJC6W4vCbhVPL9H3Cq/GbSiA3DMAz/8cO6iKKbZCmOxj3qzH5UABE5JDBpDMMwDF+si0w3SS9Bd5MsRaE87mZ59RaR64EXgfsCk8gwDCPl+GFdRNFNshSX1wBgEU4Nr+OBW3BSiA3DMIwA8KNXfRTdJEup5fU/qjoqa9mbqjo8MKkCIi61vMJO5fOdWqkNZhgxJYoMrUKUWssrr4UiItOBG4BjRORNz6qeQEP1IqaTuP1RysaPYnymkAyjIEntVZ/XQhGRw4A+wI+BGZ5Vraq6LQTZfCcOFkpz624mzW3oYMo+dcOYZPSlrrbir1WHNYzEUXW1YVX9WFU3qOrlqvq+5+GbMhGR80RkjYisF5EZOdZ3E5HH3PV/dJt7Zdbd7C5fIyLn+iVT0PiWyhdVo6lqi/FZddj0YM3QyqLaeSdxoJSgfCCISGecRl3jgSbgDRFZnNUb/lrgI1X9OxGZAtwJ/IuInAhMAf4BpwHYiyLy96oaXIK1T/gRbIv0Lr/a1sBWHTYdmCVaFol3hbtE+cueBqxX1XdVdQ+wEJiYNWYiTrl8cDLNzhYRcZcvVNXdqvoesN79vNjjSypflHf51Vb8teqw6cAs0bKIYlZ7EERmoQBHARs975uAf8w3RlX3icjHQD93+etZ2x4VnKj+4UuwLcq7/GqL8VVaHdYC+cnCLNGyiGJWexBEqVByXUGznYb5xpSyrfMBItOAaQCDBw/ONSR0OnWS6gLw1bqdqqWaYnyVKCRznySPkP+jSU/F98UVHgOiPBubgEGe93XApnxjROQg4DBgW4nbAqCq81S1XlXrBwyokR4JSW80VW5LYnOfJI8Q/6OZ+MOkuQ2MuXMpk+Y2sGZLa6KC2lHMag+CohMbA9uxoyDWAmcDHwJvAFeo6tueMd8ATlbVr7tB+X9W1ctE5B+AX+PETY4EXgKOKxaUj0PasG+kyQW0fSPcfVLH5TeudJSSEU9C+o8mPhXfJc5WVtUTG4PGjYl8E3gO6Aw8qKpvi8htQKOqLgYeAB4RkfU4lskUd9u3ReRxYBVOf5ZvJCHDy1dqoAdEySdQ1C4+ozJC+o/WSvyhald4DIgyhoKqPgs8m7XsFs/rT4FL82x7B3BHoALGjDjfwZRLWWmS1ubVKECtxB9qgchcXlGQZJdXreSpZyjbTZEmF59RFrV2bsSR2Lu80oQflkW+PPWk+YkzlO2mqAEXnxEMca17VUsehVIxhRIwft091YqfOIO5KQw/iVv8Ia1Wk/kMAsavGbBRdF8LklpJkzSMXNTKzPdyMQslYPyyLDIX4Ow7nqRegOPqpjAMP6g1j0KpmEIJGL9cO7V4AY6bm8Iw/CKtLl1zeQWMn66dzAX4qD49GNCzm6/KpBZKZxtGXEirS9fShkMg7tkeaQ0gGjVIjNLL437el0PVDbYM/wjSsvCDtAYQjRojU0T0/nFOqZ77xznvI2rsFffzPghMoRipDSAGjnUsDBcrIho5plCMmktJjgUxu1tOBdaDJXJMoRipDSB2wE+Lwu6WDyQMa63abqBmUVaNpQ0b4aYkxyho2kEuP5t42d3yZ4TVIK2aIqLWxM0X7EgZQEgBxDi7gfy2KKq9W64lwrLWvN1Ab1zpPJeqEMyi9AVTKDGi5ueClHvShumC8NuiSHpXTT8J01ortxtoBDLW8nluLq+YkIq5IOWctGG7IPxu4uW9W46bey9sktAgLSQZa/08T+G/O56kYi5IOW6gsF0QQVgUld4tB0kUgeckWGshyVjr53kkFoqI9AUeA4YAG4DLVPWjrDEjgJ8BvYA24A5Vfcxd9xDwJeBjd/hXVXV5GLIHRSrmgpQTNA07qJ0GiyKqwHNYx7aahI+QZKz18zwql9cM4CVVnSkiM9z3N2WN2QlcparrRORIYJmIPKeq2931/6aqi0KUOVBSUUyunJM2CjdJrTfxymf1Xfdi8N876GPrh7IM4fev9fM8qtuvicB89/V84OLsAaq6VlXXua83AVuBGNnIhSk38Ja0uSAVBxZLdQMlwU2SNGo5lTkhWVpJO8/LJSoL5QhV3QygqptF5PBCg0XkNKAr8I5n8R0icgvwEjBDVXcHJm2ZVBJ4S1J5+lACi2lwQYVNEoLjlZIQZZmk87wSAjs7ReRFEVmZ4zGxzM8ZCDwCXK2qmQjizcAw4PNAXzq6y7zbTxORRhFpbG4O526l0sBbrIvJeYK5ba1buPuF1cEHFqsNatvM5wOpZasvQfN+Yn2eV0lgFoqqjsu3TkS2iMhA1zoZiOPOyjWuF/BfwA9U9XXPZ292X+4WkV8C3ysgxzxgHjjl68v/JuVTc4G3LP90l96D+dFF89na2ov/3fg3IIbfz2Y+d6SWrb5qZskbvhHVP2kxMNV9PRV4JnuAiHQFngIeVtUnstYNdJ8FJ/6yMlBpy6Tmii3m8E/3++1UfnDWZydr7L5fQnzqoRPHVGY/qGaWvOEbUR3tmcB4EVkHjHffIyL1InK/O+Yy4IvAV0VkufsY4a77lYi8BbwF9AduD1f8wiQl8FZyYD2Pf3rgoc7fJ5bfLyE+dcNHalVZJohIgvKq2gKcnWN5I3Cd+/pR4NE82385UAGrJAmBt7IC63mCuQN696LhprGx/H41HYA2jJhiKjwg4h54KytxIE8wt0vPw2P7/Wo6AG0YMcVqeaWUshIHkhjMTaLMhpFwTKGklLJn7CZxFnkSZTaMBGO3ayklKYkDhmEkB7NQUkoSEgcMw0gWplBSTCZxoNZob1daPtljitKIhDT//0yhGDVFrTcwMuJN2v9/FkMxaopab2BkxJu0///MQjFqinzp0Nre5hSI9DuFuJqmTkbNUXN1/MrEFIpRMknwDedKhz73xAH03/kOPHaFv4Ui41SA0hRbLKj1BlrFsH9cAqm4uVWV+1yzpZVJcxsYc+dSJs1tYM2W1lD2XQ650qF/cmEdnTLKBPwrFBmXApQZxXb/OLj7JOd566pYlOuP4r8aJWlPxxfV2v6BvdTX12tjY2PUYlRFVEG/5tbdTJrb0OHO66kbxsQuUyzbkurfthW5+6SOA29c6RQSrJTtG50LuN+fWy47tjhKJLtuWRitfQuQ1gB1Eiz5chGRZapaX2ycWSgJI6qgX5J8w9l11CSo5ktxaeoU08rKaQ1Qx72OX5CYQkkYUV3Yk9rjpb1d+UgOo+1ffu1/oci4FKCMi2LLIkk3IYY/WFA+YUQV9Mv4hrPdF3H2DXtdLgMO7cJt5z/J8f270qVrd+QQH4LWcSlAGdNuhWkPUKcRi6EkjCj90knzDScp7lM1MczySmsMpRYpNYZiFkrCiLIGV9JKtaTK5RLDyspWLy59mEJJIEm7sEeFuVyix/6r6SISm1hE+orICyKyzn3uk2dcm6ef/GLP8qEi8kd3+8dEJL6OfCMy0j4nwDDCJpIYiojcBWxT1ZkiMgPoo6o35Ri3Q1UPzbH8ceBJVV0oIj8HVqjqz4rttxZiKEZ5JC3uYxhxJO7zUCYC893X84GLS91QRAT4MrCoku2NdJGYOQHt7c4Exe0bnecYzHI3jHKJKoZyhKpuBlDVzSJyeJ5x3UWkEdgHzFTVp4F+wHZV3eeOaQKOyrcjEZkGTAMYPHhwvmGGER1xqgkWFTHMUisHs4QdAlMoIvIi8Lkcq75fxscMVtVNInIMsERE3gL+lmNcXr+dqs4D5oHj8spev3fvXpqamvj000/LEMvw0r17d+rq6ujSpUvUoiSTfDXBIi6d4gulKIqgFWrAysrSoz8jMIWiquPyrRORLSIy0LVOBgJb83zGJvf5XRF5GRgJ/AboLSIHuVZKHbCpUjmbmpro2bMnQ4YMwfGmGeWgqrS0tNDU1MTQoUOjFieZxLR0StWUqiiCVKghWH/5SszU5HynIkRlUy4GprqvpwLPZA8QkT4i0s193R8YA6xSJ4tgKTC50Pal8umnn9KvXz9TJhUiIvTr188svGqIaemUqim1GnOQCtXnitC5qienar5TEaJSKDOB8SKyDhjvvkdE6kXkfnfMCUCjiKzAUSAzVXWVu+4m4Lsish4npvJANcKYMqkOO35VEpeaYMUoN3GgVEURpEL1UVnla+FwcNdk1rkLgkgUiqq2qOrZqnqc+7zNXd6oqte5r19V1ZNV9RT3+QHP9u+q6mmq+neqeqmq7o7ie/hF586dGTFiBCeddBIXXXQR27dvL7rNGWecUdG+nn76aVatWlV8YBaHHtohe9vwC29NsBtXOs9xC8hX0nOlVEURpEL1UVnlc23ta1eb7+QSo39sejn44INZvnw5K1eupG/fvtx7771Ft3n11Vcr2lelCiVthN4YKlM6pfcg2nscTvMneyNrSpXzu1fiOipVUQSpUH1UVvlcW3v3te8vMdNw01ieumFMKgPyYKVXyibo9MDRo0fz5ptv7n8/a9YsHn/8cXbv3s2kSZP44Q9/CDgWw44dOwqOefjhh5k9ezYiwvDhw5k+fTqLFy/mlVde4fbbb+c3v/kNAN/4xjdobm6mR48e3HfffQwbNoz33nuPK664gn379nHeeef59v2SQNQFOKPMGMq3/2EH70HKdR2VUI35wPOpN/16+Zxu62NF6EKlfKzEjINZKGUQdBvctrY2XnrpJSZMmADA888/z7p16/jTn/7E8uXLWbZsGb///e8P2CbfmLfffps77riDJUuWsGLFCu655x7OOOMMJkyYwKxZs1i+fDnHHnss06ZN46c//SnLli1j9uz+B1vKAAAPzklEQVTZ3HDDDQB85zvfYfr06bzxxht87nO5sr9rlygbQ5W07wAnQeZ160iXylxHHsuLQ4/ooExCaStdQIZysFI+xTELpQyCSg/ctWsXI0aMYMOGDZx66qmMHz8ecJTF888/z8iRIwHYsWMH69at44tf/OL+bfONWbFiBZMnT6Z///4A9O3bt8N+d+zYwauvvsqll166f9nu3U44qqGhYb8F85WvfIWbbupQGadmiTJrp+i+A06Dzbf/j+jF4T73XElauq1VTy6OKZQyCOpCk4mhfPzxx1x44YXce++9fPvb30ZVufnmm/na176Wd9t8Y+bMmVM0+6q9vZ3evXuzfPnynOvTmr0VZZXiovsOeBJkvv1Lp86+NxNLYrqtubYKYy6vMgi6De5hhx3GnDlzmD17Nnv37uXcc8/lwQcf3B8r+fDDD9m69cA5oPnGnH322Tz++OO0tLQAsG3bNgB69uxJa2srAL169WLo0KE88cQTgKOcVqxYAcCYMWNYuHAhAL/61a98+X5JIUrXRtF9BzwJsuD+fXIdZUhqW2kjP6nv2PjnP/+ZE044oaTtgwqYegPsABdddBGXXXYZX/nKV7jnnnu4//7794979NFHOfbYYw9QDPnGzJ8/n1mzZtG5c2dGjhzJQw89RENDA9dffz3dunVj0aJFdOrUienTp7N582b27t3LlClTuOWWWw4Iyl9yySXcfvvtB8iYTTnHsVSirI8U233v2OKk7HqVSu/BvpZpCeu7R52AYJROqdWGTaGUeSGMQxG4lpYWRo0axfvvvx/qfgvht0Kxi00eaqyQZBzOJ6M41gI4IKL2oW7atImzzjqL733ve5HJEAZJC9iGho9psHEg6vPJ8BdTKAnjyCOPZO3atVGLEThJDNiGRgz7xxsGWFDeiCkWsDWM5GEKxYglNonMMJKHubyMWGKTyAwjeZhCMWKLBWwNI1mYyysmPPXUU4gIq1evDmV/d999Nzt37ixrm5dffpkLL7wwIIkMw0g6plBiwoIFCzjzzDP3z04PmkoUimEYRiFMoZRLAJVed+zYQUNDAw888MABCuWuu+7i5JNP5pRTTmHGjBkArF+/nnHjxnHKKacwatQo3nnnHcApYf/5z3+e4cOHc+uttwKwYcMGhg0bxtSpUxk+fDiTJ09m586dzJkzh02bNjF27FjGjh0LOEUmR48ezahRo7j00kv3z4r/7//+b4YNG8aZZ57Jk08+WfV3NQyjhlHV0B9AX+AFYJ373CfHmLHAcs/jU+Bid91DwHuedSNK2e+pp56q2axatarDsry0talufkv1Jyep3trLed78lrO8Ch555BG95pprVFV19OjRumzZMn322Wd19OjR+sknn6iqaktLi6qqnnbaafrkk0+qququXbv0k08+0eeee06vv/56bW9v17a2Nr3gggv0lVde0ffee08B/cMf/qCqqldffbXOmjVLVVWPPvpobW5uVlXV5uZm/cIXvqA7duxQVdWZM2fqD3/4Q921a5fW1dXp2rVrtb29XS+99FK94IILcn6Hso6jYVRIW1u7bv3bp9q07RPd+rdPta2tPWqRUgHQqCVcY6OyUGYAL6nqccBL7vsDUNWlqjpCVUcAXwZ2As97hvxbZr2q5i6X6zeVdK0rgQULFjBlyhQApkyZwoIFC3jxxRe5+uqr6dGjB+CUn29tbeXDDz9k0qRJAHTv3p0ePXocUMJ+1KhRrF69mnXr1gEwaNAgxowZA8CVV17JH/7whw77f/3111m1ahVjxoxhxIgRzJ8/n/fff5/Vq1czdOhQjjvuOESEK6+8sqrvmXZC7wJZY4TWP8WomKiyvCYCZ7mv5wMvA4UabkwGfqeq0Tr9A6j02tLSwpIlS1i5ciUiQltbGyLCJZdc0qF8vOapu6Z5Sthv2LChw2fkKkmvqowfP54FCxYcsHz58uWpLWHvN1abrHqsHE/8icpCOUJVNwO4z4cXGT8FWJC17A4ReVNEfiIi4fybDupaWde6AixatIirrrqK999/nw0bNrBx40aGDh1K3759efDBB/cHzrdt20avXr2oq6vj6aefBpxmWDt37ixY5v6DDz7gtddeAz4L/MOBZexPP/10GhoaWL9+PQA7d+5k7dq1+1sBZ+I02QrHKJ0ou0BmSLqFZOV44k9gCkVEXhSRlTkeE8v8nIHAycBznsU3A8OAz+PEY/JaNyIyTUQaRaSxubk61xQ9BjiVXTNKxYeudQsWLNjvwspwySWXsGnTJiZMmEB9fT0jRoxg9uzZADzyyCPMmTOH4cOHc8YZZ/CXv/yFc845hyuuuILRo0dz8sknM3ny5P3K4oQTTmD+/PkMHz6cbdu2MX36dACmTZvG+eefz9ixYxkwYAAPPfQQl19+OcOHD+f0009n9erVdO/enXnz5nHBBRdw5plncvTRR1f8PdNO1BfDWnAXWTme+BNJ+XoRWQOcpaqbXYXxsqoen2fsd4B/UNVpedafBXxPVYtOkPCjfD3t7U7MJAGVXjds2MCFF17IypUrA99XEP1Qaonm1t1MmtvQoRNiWO6aqPfvB+Y2jI64l69fDEwFZrrPzxQYezmORbIfERnoKiMBLgaCv2JmsEqvRgVkapNlXwzDqk0WtYXkB1aOJ/5EpVBmAo+LyLXAB8ClACJSD3xdVa9z3w8BBgGvZG3/KxEZAAhO2vDXwxE7WQwZMiQU68QoTtQXw6K96hOCleOJN5EoFFVtAc7OsbwRuM7zfgNwVI5xXw5SPsMIgigvhlFbSEY6sOKQOGmzlh5bOVHE4YzyiNpCMtJB6hVK9+7daWlpoV+/fqZUKkBVaWlpoXv37lGLYhTB3EVG0KReodTV1dHU1ETVKcUppnv37tTV1UUthmEYEZN6hdKlSxeGDh0atRiGYRiJJ54TKAzDMIzEYQrFMAzD8AVTKIZhGIYvRFJ6JSpEpBl4P8Rd9gf+GuL+ysXkqw6TrzpMvuoIU76jVbVo0cJUKZSwEZHGUurfRIXJVx0mX3WYfNURR/nM5WUYhmH4gikUwzAMwxdMoQTLvKgFKILJVx0mX3WYfNURO/kshmIYhmH4glkohmEYhi+YQqkSEekrIi+IyDr3uU+OMWNFZLnn8amIXOyue0hE3vOsGxG2fO64No8Miz3Lh4rIH93tHxMRX+udl3j8RojIayLytoi8KSL/4lkXyPETkfNEZI2IrBeRGTnWd3OPx3r3+AzxrLvZXb5GRM71Q54yZfuuiKxyj9VLInK0Z13O3zlk+b4qIs0eOa7zrJvq/hfWicjUiOT7iUe2tSKy3bMujOP3oIhsFZGczYzEYY4r/5siMsqzLvDjVxBVtUcVD+AuYIb7egZwZ5HxfYFtQA/3/UPA5KjlA3bkWf44MMV9/XNgetjyAX8PHOe+PhLYDPQO6vgBnYF3gGOArsAK4MSsMTcAP3dfTwEec1+f6I7vBgx1P6dzyLKN9fy/pmdkK/Q7hyzfV4H/zLFtX+Bd97mP+7pP2PJljf8W8GBYx8/dxxeBUcDKPOv/CfgdToPB04E/hnX8ij3MQqmeicB89/V8nJbEhZgM/E5VdwYq1WeUK99+xKnn/2VgUSXbl0hR+VR1raquc19vArYCRSdZVcFpwHpVfVdV9wALXTm9eOVeBJztHq+JwEJV3a2q7wHr3c8LTTZVXer5f70OhFkKupRjl49zgRdUdZuqfgS8AJwXsXyXAwt8lqEgqvp7nJvOfEwEHlaH14HeIjKQcI5fQUyhVM8RqroZwH0+vMj4KXT8g97hmq4/ERG/G1aUKl93EWkUkdcz7jigH7BdVfe575vI0UEzJPkAEJHTcO4s3/Es9vv4HQVs9LzP9b33j3GPz8c4x6uUbYOWzcu1OHezGXL9zn5SqnyXuL/ZIhEZVOa2YciH6yocCizxLA76+JVCvu8QxvErSOrL15eCiLwIfC7Hqu+X+TkDgZOB5zyLbwb+gnORnAfcBNwWgXyDVXWTiBwDLBGRt4C/5RhXdlqgz8fvEWCqqra7i6s+frl2lWNZ9vfON6aUbauh5M8XkSuBeuBLnsUdfmdVfSfX9gHK91tggaruFpGv41h6Xy5x2zDkyzAFWKSqbZ5lQR+/Uojqv1cUUygloKrj8q0TkS0iMlBVN7sXvK0FPuoy4ClV3ev57M3uy90i8kvge1HI57qSUNV3ReRlYCTwGxxz+iD3LrwO2BSFfCLSC/gv4AeumZ/57KqPXw6agEGe97m+d2ZMk4gcBByG46YoZdugZUNExuEo7C+p6u7M8jy/s58XxKLyqWqL5+19wJ2ebc/K2vZlH2UrST4PU4BveBeEcPxKId93COP4FcRcXtWzGMhkU0wFnikwtoM/1r2IZuIVFwM5MzuClE9E+mRcRSLSHxgDrFIn0rcUJ+6Td/sQ5OsKPIXjN34ia10Qx+8N4DhxMty64lxYsjN6vHJPBpa4x2sxMEWcLLChwHHAn3yQqWTZRGQk8Atggqpu9SzP+Tv7KFup8g30vJ0A/Nl9/RxwjitnH+AcDrTmQ5HPlfF4nMD2a55lYRy/UlgMXOVme50OfOzeWIVx/AoTZgZALT5w/OYvAevc577u8nrgfs+4IcCHQKes7ZcAb+FcCB8FDg1bPuAMV4YV7vO1nu2PwbkgrgeeALpFIN+VwF5guecxIsjjh5NJsxbn7vP77rLbcC7SAN3d47HePT7HeLb9vrvdGuD8AP5zxWR7EdjiOVaLi/3OIcv3Y+BtV46lwDDPtte4x3Q9cHUU8rnv/x2YmbVdWMdvAU4m414cq+Na4OvA1931Atzryv8WUB/m8Sv0sJnyhmEYhi+Yy8swDMPwBVMohmEYhi+YQjEMwzB8wRSKYRiG4QumUAzDMAxfMIViGAEgIr1F5IYKt71RRHp43t8hIhtFZId/EhqG/5hCMYxg6I1TkbgSbgR6eN7/Fn8LTBpGIFjpFcMIhpnAsSKyHKfq61ac0jvdcMrv3Coih+C0B6jDKav+H8AROCX6l4rIX1V1rLqlZpxiAIYRX0yhGEYwzABOUtURInIOTnmW03BmOS8WkS/ilODfpKoXAIjIYar6sYh8Fxirqn+NSnjDqARzeRlG8JzjPv4X+B9gGE6Nr7eAcSJyp4h8QVU/jlBGw6gas1AMI3gE+LGq/qLDCpFTcWpL/VhEnlfVakvvG0ZkmIViGMHQCvR0Xz8HXCMihwKIyFEicriIHAnsVNVHgdk4bV+ztzWMxGAKxTACQJ2eHw0ishIYD/waeM1tXLYIR2GcDPzJDdx/H7jd3Xwe8DsRWQogIneJSBPQQ0SaROTfw/02hlEaVm3YMAzD8AWzUAzDMAxfMIViGIZh+IIpFMMwDMMXTKEYhmEYvmAKxTAMw/AFUyiGYRiGL5hCMQzDMHzBFIphGIbhC/8fmeL0ykfWicEAAAAASUVORK5CYII=\n", "text/plain": [ "
" ] }, "metadata": { "needs_background": "light" }, "output_type": "display_data" } ], "source": [ "sns.scatterplot(x='test1',y='test2',data=data[data['result']==0],label='Rejected');\n", "sns.scatterplot(x='test1',y='test2',data=data[data['result']==1],label='Accepted');" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Figure above shows that our dataset cannot be separated into two different classes by a straight line. Therefore, a straight forward application of logistic regression will not perform well on this dataset since logistic regression will only be able to find a linear decision boundary." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "#### 2.2    Feature Mapping\n", "\n", "One way to fit the data better is to create more features from each data point. We will map the features into all polynomial terms of $x_1$ and $x_2$ up to $n^{th}$ power. As a result of this mapping, our vector of two features will be transformed into higher-dimensional feature vector. A logistic regression classifier trained on higher-dimension feature vector will have a more complex decision boundary and will appear non-linear. \n", "\n", "While the feature mapping allows us to build a more expressive classifier and helps us solve the **Underfitting** problem, it is also more susceptible to **Overfitting** problem.\n", "\n", "From $[x_1,x_2]$ features to $[1,x_1,x_2,x_1^2,x_1x_2,x_2^2,x_1^3...x_2^n]$ features." ] }, { "cell_type": "code", "execution_count": 21, "metadata": {}, "outputs": [], "source": [ "def mapFeature(x1,x2,degree):\n", " result=np.ones(shape=(x1.shape[0],1))\n", " for i in range(1,degree+1):\n", " for j in range(0,i+1):\n", " column=np.multiply(np.power(x1,i-j),np.power(x2,j))\n", " result=np.column_stack((result,column))\n", " return result" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "#### 2.3    Cost Function and Gradient\n", "\n", "In this part of the exercise, we will implement the **regularized** logistic regression to fit the data and also see for ourselves how parameters regularization can help combat the overfitting problem.\n", "\n", " \n", "If we have too many features, the learned hypothesis will have large parameters and fit the training set very well such that $J(\\theta)\\approx0$, but fail to generalize new examples. What are our options available?\n", "1. Reduce the number of features. (we will be throwing away the important information here.)\n", "2. **Regularization** - this process keeps all the features and adds a penalty to the **RSS (Sum of squared residuals)** to reduce the freedom of the model. Hence, the model will less likely to fit the noise of the training data and will improve the generalization abilities of the model. In layman terms, regularization tries to prevent the parameters from growing larger.\n", "\n", "##### Types of Regularization\n", "1. **L1 Regularization (used in Lasso Regression)** \n", "LASSO stands for Least Absolute Shrinkage and Selection Operator. \n", "This regularization will shrink some parameters to **Zero**. Hence, this type of regularization can also help in **Feature Selection** in the model. As $\\lambda$ grows bigger, more shrinkage in parameters will occur.\n", "$$ J(\\theta)=RSS+\\lambda \\sum_{j=1}^n|\\theta_j| $$ \n", "2. **L2 Regularization (used in Ridge Regression)** \n", "Parameters decrease progressively as $\\lambda$ grows larger, **but are never cut to zero.**\n", "$$ J(\\theta)=RSS+\\lambda \\sum_{j=1}^n(\\theta_j^2) $$ \n", "3. **L1/L2 Regularization (used in Elastic Net Regression)** \n", "Elastic-net is a mix of both L1 and L2 regularization. Lambda is a shared penalization parameter while alpha sets the ratio between L1 and L2 regularization in the Elastic Net Regularization.\n", "$$ J(\\theta)=RSS+\\lambda[(1-\\alpha)\\sum_{j=1}^n(\\theta_j^2)+\\alpha\\sum_{j=1}^n|\\theta_j|]$$ \n", "\n", "##### Geometric Perspective on Regularization\n", "Lasso, Ridge and Elastic-Net regularization can also be viewed as a **constraint** added to **optimization** process. Lasso error minimization can be written as :\n", "$$ minimize(RSS)\\;\\;\\;\\;subject\\;to\\; \\sum_{j=1}^n|\\theta_j|\n", "\n", "For the L1 regularization, the constraint region ($|\\theta_1|+|\\theta_2|" ] }, "metadata": { "needs_background": "light" }, "output_type": "display_data" } ], "source": [ "degree=6\n", "lmbdas=[0,1,100]\n", "alphas=[10,0.03,0.03]\n", "iterations=[40000,10000,10000]\n", "\n", "labels=['No Regularization (Overfitting)','Regularization (Just Right)','Too much Regularization (Underfitting)']\n", "\n", "nrows,ncols=2,3\n", "fig,ax=plt.subplots(nrows,ncols,figsize=(18,8))\n", "fig.suptitle('Models with different lambda values',size=16)\n", "\n", "for i in range(ncols):\n", " X=mapFeature(data['test1'],data['test2'],degree)\n", " y=data.values[:,2]\n", " y=y[:,np.newaxis]\n", " theta=np.zeros(shape=(1,X.shape[1]))\n", " theta,jHistory=regularizedGradientDescent(X,y,theta,alphas[i],lmbdas[i],iterations[i])\n", " axis1=ax[0][i]\n", " axis1.set_title(f'{labels[i]}\\nlambda = {lmbdas[i]} | Cost = {jHistory[iterations[i]-1]:.3f}')\n", " plotDecisionBoundary(theta,data,y,axis1)\n", " axis2=ax[1][i]\n", " c=sns.distplot(theta[0],ax=axis2,kde=False)\n", " c.set(xlabel=f'Theta{i+1} Values',ylabel='Frequency')" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "" ] } ], "metadata": { "kernelspec": { "display_name": "Python 3", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.7.1" } }, "nbformat": 4, "nbformat_minor": 2 }