{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "94bcb454-032d-49a4-8b9f-3f0964ec614a",
   "metadata": {
    "jp-MarkdownHeadingCollapsed": true,
    "tags": []
   },
   "source": [
    "# Perceptron"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "6e5e8870-aafd-43f7-9c54-204f874f35dd",
   "metadata": {},
   "outputs": [],
   "source": [
    "import matplotlib.pyplot as plt\n",
    "import random\n",
    "import torch\n",
    "import torch.nn as nn\n",
    "import numpy as np\n",
    "import math"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0529a320-c3fb-4869-a6ab-609415a74264",
   "metadata": {},
   "source": [
    "The **perceptron** is the building block of artificial neural networks. The concept of a perceptron is based off of the neuron. A perceptron analogous to a neuron as an artificial neural network is analogous to a biological neural network of a living organism."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d6b8685f-ccaa-462d-a67a-6c07f0537491",
   "metadata": {},
   "source": [
    "The architecture of a perceptron consists of three main components: a set of **weights**, a **bias**, and an **activation function**. Input data is fed into the perceptron, the **weighted sum** is calculated using the input data and the weights and bias values, then, that sum is fed into the activation function which transforms the sum, and then that value is output as the final result."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0379131a-05b8-4bb7-b59c-85c4bd5742d7",
   "metadata": {},
   "source": [
    "The weights of a perceptron are usually represented with a vector:"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "661ea5aa-a5e4-4e10-ae3b-3f2de05b84a2",
   "metadata": {},
   "source": [
    "$$w = \\begin{bmatrix} w_1 & w_2 & w_3 & \\ldots & w_d \\end{bmatrix}$$"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b060dec5-4344-4839-86c3-79d4cc211aa1",
   "metadata": {},
   "source": [
    "where $w_1, \\ldots, w_d$ are numeric weight values. $d$ is the number of dimensions of an input value (the number of features of the input values). Bias is a scalar value represented with the letter $b$ or sometimes $w_0$. If I wanted to, I could include the bias $w_0$ as its own \"weight\" in the weight vector instead of denoting it as a separate variable $b$."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "bbf36f17-9d35-4b87-88b4-a67a08b588ea",
   "metadata": {},
   "source": [
    "Each input data point $x_i$ with $d$ number of features can also be represented as a vector:"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ef56dc02-5510-4e82-92fc-16bace952a3c",
   "metadata": {},
   "source": [
    "$$x_i = \\begin{bmatrix} x_{i,1} & x_{i,2} & x_{i,3} & \\ldots & x_{i,d} \\end{bmatrix}$$"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7e2a27dd-3b6c-4b9f-a09f-e9802be8f9ea",
   "metadata": {},
   "source": [
    "I can also include $1$ as $x_{i,0}$ if the bias value is included in the weight vector. The weighted sum is calculated by taking the dot product of the weight and input vectors."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "799e4cd7-7d5a-4264-a783-f3657407e9df",
   "metadata": {},
   "source": [
    "The activation function of a perceptron is any one-variable function that transforms the output of the weighted sum into an new output."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "aed2f600-9a6b-4d39-8050-30cd0c794fec",
   "metadata": {},
   "source": [
    "When tasked with creating a good model for a particular regression/classification problem, a collection of interconnected perceptrons (a neural network) is often used. A perceptron/neural network \"learns\" to model a dataset by finding the weight and bias values that best fit the dataset. The ideal weight and bias values can be found by processing the same data points in the dataset over and over again while trying different combinations of weight and bias values. The performance of a model is usually measured with a **loss function**. A loss function calculates the difference between the current output of a model and the expected output. If the learning/training steps are implemented correctly, I would expect the loss to decrease with over several epochs until it converges onto a value. A loss function also helps me adjust the weight and bias values appropriately so I do not have to guess and try random weight and bias combinations and just hope I stumble across the best values to model the data."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "12f2a312-765b-4136-be0b-3f3f152fa904",
   "metadata": {
    "tags": []
   },
   "source": [
    "## Linear Regression (Least Squares)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1ed234fc-05e2-442d-90d4-ebb1a3b3bb3e",
   "metadata": {},
   "source": [
    "Say I have a collection of $(x,y)$ data points and I want to find the line that best fits the data. For a more concrete example, let me use the same dataset used in the linear regression IPython notebook:"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7c3dee82-f99a-4e4c-b5db-d1e7e6be892f",
   "metadata": {},
   "source": [
    "$$D = [(-1.5, 4),(-0.1, 4),(0.6, 3.1),(2.3, 0.3),(-1.2, 2.4),(1.6, 0.4),(1.7, 1.2),(-0.2, 2.4),(0.7, 2.3),(0.5, 1.75)]$$"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "95fbd130-ede1-4a77-940a-f7ac56f6abb0",
   "metadata": {},
   "source": [
    "Scatter plot of the dataset $D$:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "df1b66f3-0f24-4c98-b5af-b4bc90b32dc1",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYIAAAEWCAYAAABrDZDcAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/YYfK9AAAACXBIWXMAAAsTAAALEwEAmpwYAAApYklEQVR4nO3deZxcVZn/8c+XEKBla4GIpAmEzbiAEsiwCDODKEYRQkQUFJQwKjIjAzoShyjDIC4omeHngoiMKCAYEQghIE6IAirI1iGBACFD2IQOSgATEmkhhOf3xzkNlU519e3lVi/1fb9e9cqtc7enbnXuU/eec89RRGBmZo1rvYEOwMzMBpYTgZlZg3MiMDNrcE4EZmYNzonAzKzBORGYmTU4JwKzKiSdIenSOu1rP0kPSVolaXI99mlWyYnA+kTS/pL+IGmFpOck3Srp7/q4zSmSbulUdpGkr/Ut2nX2c5Gkl/IJ+DlJcyW9uRfbeUzSe/oQypnAuRGxSUTM6mL77ZJWSlqej/cJkgr9/5U0VlJIWr8PMQ6a/Vj/cyKwXpO0GXAd8D1gC6AF+Arw4kDGVU2Nk9PZEbEJsC3wNHBR3YJ6zfbA/d0sc2hEbJqX/Sbw78CFZQdmjcGJwPriTQARMSMi1kREe0TcEBH3diwg6dOSFuVfsw9I2iOXnyrp4YryD+bytwDnA/vmX+rLJR0PHA18MZddm5cdLekqScskPSrppIr9niHpSkmXSnoemFLrg0TEC8DPgF2rzZc0SdL9OZ6bc5xI+imwHXBtju2LXaz/aUlL8pXHbEmjc/nDwI4V62/YTZwrImI2cCRwrKRd83Y+IGm+pOclPSHpjIrVfpf/XZ73sa+knSTdKOlZSc9IukxSc0W8/y6pLX8/iyW9O5evV/HdPSvpF5K26Go/tT6LDSIR4ZdfvXoBmwHPAhcD7wde32n+h4E24O8AATsD21fMG036MXIk8FdgmzxvCnBLp21dBHyt4v16wDzgdGAD0sn0EWBinn8GsBqYnJdtqhL/q9sENiElgt9XrH9pnn5Tju8gYCTwRWAJsEGe/xjwnhrH6UDgGWAPYEPSFdTvKuZ3t37V+cAfgX/O0wcAu+XP+nbgz8DkPG8sEMD6FevunD/PhsAo0kn823neOOAJYHTF+jvl6ZOB20lXUBsCPwRmdLUfv4bGy1cE1msR8TywP+k///8Ay/Kv3a3zIp8i3Xq5K5IlEfF4XveKiFgaEa9ExOXAQ8BePdj93wGjIuLMiHgpIh7JMRxVscxtETEr76O9i+2cImk56cS+CdWvHI4EfhkRcyNiNfBfQBPwzoKxHg38OCLujogXgWmkK56xBdfvylLSLTki4uaIWJg/673ADOAfu1oxfxdzI+LFiFgGnFOx/BrSSf6tkkZGxGMR8XCedwLw5Yh4Mn+WM4AjXC8wtDkRWJ9ExKKImBIR25Juq4wGvp1njwEerraepE9IWpBvtSzP627Vg11vD4zuWD9v40vA1hXLPFFgO/8VEc0R8caImFRxwqs0Gni8401EvJK33VIw1s7rryJdSRVdvystwHMAkvaWdFO+TbaCdMLu8nhK2lrSz/Ptn+eBSzuWj4glwOdIJ/mn83Kj86rbA1dXHPNFpMSxNTZkORFYv4mIB0m3Wzrusz8B7NR5OUnbk369nwhsGRHNwH2k20eQrjDW2Xyn908Aj+aTeMdr04g4uMY6vbWUdALsiF+kJNdWcD+d198Y2LJi/R7LLbNagI7WVT8DZgNjImJzUj1LreP5jVy+W0RsBhxTsTwR8bOI2D/HHcC38qwngPd3Ou4bRURbF/uxIcCJwHpN0pslfUHStvn9GOCjpHvIAD8i3XrZU8nOOQlsTDppLMvrHcfalbR/BraVtEGnsh0r3t8JrMyVmk2SRkjaVX1sutqFXwAfkPRuSSOBL5BaRv2hi9g6mwEcJ2n3XBn8DeCOiHisp4FI2kzSIcDPSXUYC/OsTYHnIuJvkvYCPlax2jLglU4xbgqsAlZIagGmVuxjnKQDc6x/A9rz+pASzNfz94ikUZIOq7EfGwKcCKwvVgJ7A3dI+ispAdxHOlESEVcAXyf9Wl0JzAK2iIgHgP8GbiOdRHcDbq3Y7o2k5pR/kvRMLruQdM96uaRZEbEGOATYHXiUVBn7I2Dz/v6QEbGY9Iv5e3k/h5Kac76UFzkLOC3HdkqV9X8N/AdwFfAU6SrpqM7LdeNaSStJv8i/TLqnf1zF/H8BzszLnE5KXh37f4H0PdyaY9yH1Mx3D2AF8EtgZsW2NiQ1UX0G+BPwBlK9BsB3SFceN+R93U76G+hqPzYEKMJXc2ZmjcxXBGZmDc6JwMyswTkRmJk1OCcCM7MGN+SeBtxqq61i7NixAx2GmdmQMm/evGciYlS1eUMuEYwdO5bW1taBDsPMbEiR9HhX83xryMyswTkRmJk1OCcCM7MG50RgZtbgnAjMzBpc6a2GJI0AWoG2iDik07wNgUuAPUn9sx/Zmx4ZuzNrfhvT5yxm6fJ2Rjc3MXXiOCaP72tX8FZP/g7NylOPK4KTSYNXVPNJ4C8RsTPw/3itz/N+M2t+G9NmLqRteTsBtC1vZ9rMhcya3+uu4K3O/B2alavURJD7qf8AqXvgag4jjXcLcCXw7jzoR7+ZPmcx7avXrFXWvnoN0+cs7s/dWIn8HZqVq+wrgm+TBvp+pYv5LeThBCPiZVLf6Ft2XkjS8ZJaJbUuW7asRwEsXV59qNquym3w8XdoVq7SEkEeRenpiJjX121FxAURMSEiJowaVfUJ6S6Nbm7qUbkNPv4OzcpV5hXBfsAkSY+RhtU7UNKlnZZpI439iqT1SaNLPdufQUydOI6mkSPWKmsaOYKpE8f1526sRP4OzcpVWiKIiGkRsW1EjCUNy3djRBzTabHZwLF5+oi8TL8OmTZ5fAtnHb4bLc1NCGhpbuKsw3dzi5MhxN+hWbnq3umcpDOB1oiYTRqH9qeSlgDP0fNxXAuZPL7FJ40hzt+hWXnqkggi4mbg5jx9ekX534AP1yMGMzOrzk8Wm5k1OCcCM7MG50RgZtbgnAjMzBqcE4GZWYNzIjAza3BOBGZmDc6JwMyswTkRmJk1OCcCM7MG50RgZtbguk0EkvaTtHGePkbSOZK2Lz80MzOrhyJXBD8AXpD0DuALwMOkAefNzGwYKJIIXs5jBBwGnBsR3wc2LTcsMzOrlyLdUK+UNA04BvgHSesBI8sNy8zM6qXIFcGRwIvAJyPiT8C2wPRSozIzs7rp9oogn/zPqXj/R1xHYGY2bBRpNXS4pIckrZD0vKSVkp6vR3BmZla+IreGzgYmRcTmEbFZRGwaEZt1t5KkjSTdKekeSfdL+kqVZaZIWiZpQX59qjcfwszMeq9IZfGfI2JRL7b9InBgRKySNBK4RdKvIuL2TstdHhEn9mL7ZmbWD4okglZJlwOzSCd3ACJiZq2VcpPTVfntyPyK3oVpZmZlKXJraDPgBeC9wKH5dUiRjUsaIWkB8DQwNyLuqLLYhyTdK+lKSWO62M7xkloltS5btqzIrs3MrCClH+4l70RqBq4G/jUi7qso3xJYFREvSvoMcGREHFhrWxMmTIjW1tZS4zUzG24kzYuICdXmFWk1tK2kqyU9nV9XSdq2JwFExHLgJuB9ncqfjYiO200/AvbsyXbNzKzvitwa+gkwGxidX9fmspokjcpXAkhqAg4CHuy0zDYVbycBvamUNjOzPihSWTwqIipP/BdJ+lyB9bYBLpY0gpRwfhER10k6E2iNiNnASZImAS8DzwFTehS9mZn1WZFE8KykY4AZ+f1HgWe7Wyki7gXGVyk/vWJ6GjCtWKhmZlaGIreG/gn4CPAn4CngCOC4MoMyM7P6KdLX0OOk+/dmZjYMdZkIJH0xIs6W9D2qPAgWESeVGpmZmdVFrSuCjhY8brRvZjaMdZkIIuLaPPlCRFxROU/Sh0uNyszM6qZIZXG1Vj1u6WNmNkzUqiN4P3Aw0CLpuxWzNiO1+zczs2GgVh3BUlL9wCRgXkX5SuDzZQZlZmb1U6uO4B7gHkk/i4jVdYzJzMzqqMiTxWMlnQW8FdioozAidiwtKjMzq5uinc79gFQv8C7SwPWXlhmUmZnVT5FE0BQRvyGNXfB4RJwBfKDcsMzMrF6K3Bp6UdJ6wEOSTgTagE3KDcusscya38b0OYtZuryd0c1NTJ04jsnjWwY6LGsQRa4ITgZeB5xEGjjmGODYMoMyaySz5rcxbeZC2pa3E0Db8namzVzIrPltAx2aNYgiVwRrImIVaSB69zpq1s+mz1lM++o1a5W1r17D9DmLfVVgdVHkiuC/JS2S9FVJu5YekVmDWbq8vUflZv2t20QQEe8itRZaBvxQ0kJJp5UemVmDGN3c1KNys/5W5IqAiPhTRHwXOAFYAJxeew0zK2rqxHE0jRyxVlnTyBFMnThugCKyRtNtIpD0FklnSFoIfA/4A7BtgfU2knSnpHsk3S/pK1WW2VDS5ZKWSLpD0tjefAizoWzy+BbOOnw3WpqbENDS3MRZh+/m+gGrmyKVxT8Gfg5MjIilPdj2i8CBEbFK0kjgFkm/iojbK5b5JPCXiNhZ0lHAt4Aje7APs2Fh8vgWn/htwNS8IpA0Ang0Ir7TwyRAJKvy25H51Xmks8OAi/P0lcC7Jakn+zEzs76pmQgiYg0wRtIGvdm4pBGSFgBPA3Mj4o5Oi7QAT+R9vQysALassp3jJbVKal22bFlvQjEzsy4UuTX0KHCrpNnAXzsKI+Kc7lbMiWR3Sc3A1ZJ2jYj7ehpkRFwAXAAwYcKEdcZPNjOz3ivSauhh4Lq87KYVr8IiYjlwE/C+TrPagDEAktYHNgee7cm2zcysb7q9IoiIrwBIel1EvFB0w5JGAasjYrmkJuAgUmVwpdmk7ipuA44AbowI/+I3M6ujIs1H95X0APBgfv8OSecV2PY2wE2S7gXuItURXCfpTEmT8jIXAltKWgL8G3Bqrz6FmZn1WpE6gm8DE0m/3omIeyT9Q3crRcS9wPgq5adXTP8N+HDRYM3MrP8VfbL4iU5Fa6ouaGZmQ06RK4InJL0TiPxg2MnAonLDMjOzeilyRXAC8FlSm/82YPf83szMhoEirYaeAY6uQyxmZjYAirQaOlvSZpJGSvqNpGWSjqlHcGZmVr4it4beGxHPA4cAjwE7A1PLDMrMzOqnSCLouH30AeCKiFhRYjxmZlZnRVoNXSfpQaAd+Of8xPDfyg3LzMzqpchQlacC7wQmRMRqUsdzh5UdmJmZ1Ue3VwSSNgKmAPtLCuAW4Aclx2VmZnVS5NbQJcBK0jCVAB8Dfoq7hjAzGxaKJIJdI+KtFe9vyp3QmZnZMFCk1dDdkvbpeCNpb6C1vJDMzKyeurwikLSQNMbwSOAPkv6YZ21H7pLazMyGvlq3hg6pWxRmZjZgukwEEfE4gKR3AW/LxfdHxE31CMzMzOqj1q2hFmAm6eGxebn4w5K+BXwwItrqEJ+ZmZWs1q2hc4EfRMRFlYWSPgGcRzcPlUkaQ2p6ujWpruGCiPhOp2UOAK4BHs1FMyPizOLhm5lZX9VKBG+NiA92LoyISyR9ucC2Xwa+EBF3S9oUmCdpbkR0bnr6+4hwfYSZ2QCp1Xy06jxJ6wEjuttwRDwVEXfn6ZWkUc1aehOkmZmVp1YiuE7S/0jauKMgT58PXN+TnUgaSxrI/o4qs/eVdI+kX0l6W5X5SDpeUquk1mXLlvVk12Zm1o1aieCLwArgcUnzJM0jjUfwPHBK0R1I2gS4CvhcHteg0t3A9hHxDlIXFrOqbSMiLoiICRExYdSoUUV3bWZmBdRqProaOEXSf5AGowF4OCJeKLrxPNj9VcBlETGzyj6er5i+XtJ5krbKw2OamVkdFBmzuB1Y2NMNSxJwIbAoIs7pYpk3An+OiJC0F+kK5dme7mugzZrfxvQ5i1m6vJ3RzU1MnTiOyeNdHdITPoZmA6dIp3O9tR/wcWChpAW57EukLiqIiPOBI0iD3bxMGvjmqIiIEmPqd7PmtzFt5kLaV68BoG15O9NmprzpE1kxPoZmA6u0RBARtwDqZplzSc8rDFnT5yx+9QTWoX31GqbPWeyTWEE+hmYDq9veR5UcI+n0/H67fBvHgKXL23tUbuvyMTQbWEW6oT4P2Bf4aH6/Evh+aRENMaObm3pUbuvyMTQbWEUSwd4R8VnygPUR8Rdgg1KjGkKmThxH08i1n69rGjmCqRPHDVBEQ4+PYfdmzW9jv2/eyA6n/pL9vnkjs+a7qy/rP0XqCFZLGkHqLwhJo4BXSo1qCOm4h+0WL73nY1ibK9OtbOqukY6ko4EjgT2Ai0ktfU6LiCvKD29dEyZMiNZWD5BmjWO/b95IW5X6kpbmJm499cABiMiGIknzImJCtXlFniO4LD9V/G5SK6DJEbGon2M0sy64Mt3KVms8gi0q3j4NzKicFxHPlRmYmSWjm5uqXhG4Mt36S60rgnmkeoFqzwIEsGMpEZnZWqZOHLdWHQG4Mt36V62+hnaoZyBmVp0r061shZ4slnQ4sD/pSuD3ETGrzKDMbG2Tx7f4xG+lKfJk8XnACaSO5+4DTpDkB8rMzIaJIlcEBwJv6egMTtLFwP2lRmVmZnVT5MniJeQeQ7MxuczMzIaBWs1HryXVCWwKLJJ0Z36/N3BnfcIzM7Oy1bo19F91i8LMzAZMreajv61nIGZmNjCKtBraR9JdklZJeknSGkmdB6E3M7Mhqkhl8bmksQgeApqAT+HxCMzMho0iiYCIWAKMiIg1EfET4H3drSNpjKSbJD0g6X5JJ1dZRpK+K2mJpHsl7dHzj2BmZn1R5DmCFyRtACyQdDbwFMUSyMvAFyLibkmbAvMkzY2IByqWeT+wS37tDfwg/2tmZnVS5IT+cWAEcCLwV9JzBB/qbqWIeCoi7s7TK4FFQOdn5A8DLonkdqBZ0jY9iN/MzPqoyHgEj+fJduArvdmJpLHAeOCOTrNagCcq3j+Zy57qtP7xwPEA221X+WybmZn1Va0Hyn4RER+RtJA8TGWliHh7kR1I2gS4CvhcRPSqtVFEXABcAGmEst5sw8zMqqt1RdBRuXtIbzcuaSQpCVwWETOrLNJGutXUYdtcZmZmdVLrgbKn8qD1F0XEu3q6YUkCLgQWRcQ5XSw2GzhR0s9JlcQrIuKpLpY1M7MS1KwjiIg1kl6RtHlErOjhtvcjVTQvlLQgl32J3IFdRJwPXA8cTOrE7gXguB7uw8zM+qhI89FVpJP5XFKrIQAi4qRaK0XELVQf5rJymQA+WyAGMzMrSZFEMDO/zMxsGCrSfPTiegRiZmYDo9tEIGkX4CzgrcBGHeURsWOJcZmZWZ0UebL4J6SuH14G3gVcAlxaZlBmZlY/ReoImiLiN5KUnzI+Q9I84PSSYzMbNmbNb2P6nMUsXd7O6OYmpk4cx+TxnXtcMRsYRRLBi5LWAx6SdCLpga9Nyg3LbPiYNb+NaTMX0r56DQBty9uZNnMhgJOBDQpd3hqS9MY8eTLwOuAkYE/gGODY8kMzGx6mz1n8ahLo0L56DdPnLB6giMzWVuuKYIGk+4AZwEMR8SR+4Musx5Yub+9RuVm91aosbgGmA/sDiyVdI+koSU31Cc1seBjdXP2/TFflZvXWZSLIo5HNiYjjSB3D/Zg0fsCjki6rV4BmQ93UieNoGjlirbKmkSOYOnHcAEVktrYilcVExEuSHiANLrMn8JZSozIbRjoqhN1qyAarmolA0hjgKNLg9RuT6gsmRcSDdYjNbNiYPL7FJ34btGoNTPMHUj3BL4BPR8S8ukVlZmZ1U+uK4FTg97mHUDMzG6ZqDUzzu3oGYmZmA6NIX0NmZjaMdZsIJO1QpMzMzIamIlcEV1Upu7K/AzEzs4FRq9XQm4G3AZtLOrxi1mZUjEtQY/0fA4cAT0fErlXmHwBcAzyai2ZGxJmFIzczs35Rq9XQONKJvBk4tKJ8JfDpAtu+CDiXNH5BV34fEYcU2JaZmZWkVquha4BrJO0bEbf1dMMR8TtJY/sSnJmZla9IFxNLJH0JGFu5fET8Uz/sf19J9wBLgVMi4v5qC0k6HjgeYLvttuuH3ZqZWYciieAa4PfAr4E13SzbE3cD20fEKkkHA7OAXaotGBEXABcATJgwwQ+4mZn1oyKJ4HUR8e/9veOIeL5i+npJ50naKiKe6e99mZlZ14o0H70u/2LvV5LeKEl5eq8cy7P9vR8zM6utyBXBycCXJL0EvAQIiIjYrNZKkmYABwBbSXoS+E9gJGnl84EjgH+W9DLQDhzlfo3MzOqv20QQEZv2ZsMR8dFu5p9Lal5qZmYDqEgXE5J0jKT/yO/H5Fs5ZmY2DBSpIzgP2Bf4WH6/Cvh+aRGZmVldFakj2Dsi9pA0HyAi/iJpg5LjMjOzOilyRbBa0gggACSNAl4pNSozM6ubIongu8DVwBskfR24BfhGqVGZmVndFGk1dJmkecC7SU1HJ0fEotIjMzOzuqjVDfUWFW+fBmZUzouI58oMzMzM6qPWFcE8Ur2AgO2Av+TpZuCPgEcpM7MhYdb8NqbPWczS5e2Mbm5i6sRxTB7fMtBhDRpd1hFExA4RsSOps7lDI2KriNiSNEbBDfUK0MysL2bNb2PazIW0LW8ngLbl7UybuZBZ89sGOrRBo0hl8T4RcX3Hm4j4FfDO8kIyM+s/0+cspn312h0nt69ew/Q5iwcoosGnyHMESyWdBlya3x9NGj/AzGzQW7q8vUfljajIFcFHgVGkJqRXA2/IZWZmg97o5qYelTeibhNBRDwXESdHxPj8OtkthsxsqJg6cRxNI0esVdY0cgRTJ44boIgGn25vDUl6E3AK6w5VeWB5YZmZ9Y+O1kFuNdS1InUEVwDnAz+if4eqNDOri8njW3zir6FIIng5In5QeiRmZjYgilQWXyvpXyRtI2mLjlfpkZmZWV0UuSI4Nv87taIsgB1rrSTpx6SHz56OiF2rzBfwHeBg4AVgSkTcXSRoMzPrP0U6nettVxIXkYaivKSL+e8HdsmvvYEf5H/NzKyOigxV+TpJp0m6IL/fRdIh3a0XEb8DajUzPQy4JJLbgWZJ2xQN3MzM+keROoKfAC/xWrcSbcDX+mHfLcATFe+fzGVmZlZHRRLBThFxNrAaICJeIPVCWjeSjpfUKql12bJl9dy1mdmwVyQRvCSpideGqtwJeLEf9t0GjKl4v20uW0dEXBAREyJiwqhRo/ph12Zm1qFIIvhP4H+BMZIuA34DfLEf9j0b+ISSfYAVEfFUP2zXzMx6oEirobmS7gb2Id0SOjkinuluPUkzgAOArSQ9SUooI/M2zweuJzUdXUJqPnpcLz+DmZn1QZHnCAD+EdifdHtoJKkX0poiomYPpRERwGcL7t/MzEpSpPnoecAJwELgPuAzkr5fdmBmZlYfRa4IDgTekn/BI+li4P5SozIzs7opUlm8hDR4fYcxuczMzIaBIlcEmwKLJN1JqiPYC2iVNBsgIiaVGJ+ZmZWsSCI4vfQozMxswBRpPvpbSdsDu0TEr/PDZetHxMrywzMzs7IVaTX0aeBK4Ie5aFtgVokxmZlZHRWpLP4ssB/wPEBEPAS8ocygzMysfookghcj4qWON5LWJ/c7ZGZmQ1+RRPBbSV8CmiQdRBrM/tpywzIzs3opkghOBZaRniz+DKmPoNPKDMrMzOqnSKuhVyTNAmZFhAcDMDMbZrq8IsjdQ58h6RlgMbBY0jJJfq7AzGwYqXVr6POk1kJ/FxFbRMQWpMHl95P0+bpEZ2ZmpauVCD4OfDQiHu0oiIhHgGOAT5QdmJmZ1UetRDCy2gA0uZ5gZHkhmZlZPdVKBC/1cp6ZmQ0htVoNvUPS81XKBWxUUjxmZlZnXV4RRMSIiNisymvTiCh0a0jS+yQtlrRE0qlV5k/JLZEW5Nen+vJhzMys54qOWdxjkkYA3wcOAp4E7pI0OyIe6LTo5RFxYllxmJlZbUWeLO6tvYAlEfFI7qvo58BhJe7PzMx6ocxE0AI8UfH+yVzW2Yck3SvpSkljqm1I0vGSWiW1Llvmh5vNzPpTmYmgiGuBsRHxdmAucHG1hSLigoiYEBETRo0aVdcAzcyGuzITQRtpoPsO2+ayV0XEsxHxYn77I2DPEuMxM7MqSqssBu4CdpG0AykBHAV8rHIBSdtExFP57SRgUYnxmNkwNWt+G9PnLGbp8nZGNzcxdeI4Jo+vdifaqiktEUTEy5JOBOYAI4AfR8T9ks4EWiNiNnCSpEnAy8BzwJSy4jGz4WnW/DamzVxI++o1ALQtb2fazIUATgYFKWJoDTY2YcKEaG1tHegwzGyQ2O+bN9K2vH2d8pbmJm499cABiGhwkjQvIiZUmzfQlcVmZn2ytEoSqFVu63IiMLMhbXRzU4/KbV1OBGY2pE2dOI6mkSPWKmsaOYKpE8cNUET9b9b8Nvb75o3scOov2e+bNzJrflv3K/VAma2GzMxK11EhPFxbDdWjMtyJwMyGvMnjW4bNib+z6XMWv5oEOrSvXsP0OYv77TP71pCZ2SBWj8pwJwIzs0GsHpXhTgRmZoNYPSrDXUdgZjaI1aMy3InAzGyQK7sy3LeGzMwanBOBmVmDcyIwM2twTgRmZg3OicDMrMENufEIJC0DHu/l6lsBz/RjOP1tsMcHgz9Gx9c3jq9vBnN820dE1UHfh1wi6AtJrV0NzDAYDPb4YPDH6Pj6xvH1zWCPryu+NWRm1uCcCMzMGlyjJYILBjqAbgz2+GDwx+j4+sbx9c1gj6+qhqojMDOzdTXaFYGZmXXiRGBm1uCGdSKQ9GFJ90t6RVKXTbokPSZpoaQFkloHYXzvk7RY0hJJp9Yxvi0kzZX0UP739V0styYfuwWSZtchrprHQ9KGki7P8++QNLbsmHoY3xRJyyqO2afqHN+PJT0t6b4u5kvSd3P890raY5DFd4CkFRXH7/Q6xzdG0k2SHsj/f0+ussyAHsMei4hh+wLeAowDbgYm1FjuMWCrwRgfMAJ4GNgR2AC4B3hrneI7Gzg1T58KfKuL5VbV8Zh1ezyAfwHOz9NHAZcPsvimAOfW+++tYv//AOwB3NfF/IOBXwEC9gHuGGTxHQBcN4DHbxtgjzy9KfB/Vb7jAT2GPX0N6yuCiFgUEYsHOo6uFIxvL2BJRDwSES8BPwcOKz86yPu5OE9fDEyu035rKXI8KuO+Eni3JA2i+AZURPwOeK7GIocBl0RyO9AsaZv6RFcovgEVEU9FxN15eiWwCOg8WMCAHsOeGtaJoAcCuEHSPEnHD3QwnbQAT1S8f5J1/+jKsnVEPJWn/wRs3cVyG0lqlXS7pMklx1TkeLy6TES8DKwAtiw5rnX2nXX1fX0o3zK4UtKY+oRW2ED+zRW1r6R7JP1K0tsGKoh823E8cEenWUPhGL5qyI9QJunXwBurzPpyRFxTcDP7R0SbpDcAcyU9mH+VDJb4SlMrvso3ERGSumprvH0+fjsCN0paGBEP93esw8i1wIyIeFHSZ0hXLwcOcExDyd2kv7lVkg4GZgG71DsISZsAVwGfi4jn673//jTkE0FEvKcfttGW/31a0tWky/t+SQT9EF8bUPmLcdtc1i9qxSfpz5K2iYin8mXt011so+P4PSLpZtIvpLISQZHj0bHMk5LWBzYHni0pns66jS8iKmP5EakuZjAp9W+urypPuhFxvaTzJG0VEXXr7E3SSFISuCwiZlZZZFAfw84a/taQpI0lbdoxDbwXqNpaYYDcBewiaQdJG5AqP0tvmZPNBo7N08cC61zBSHq9pA3z9FbAfsADJcZU5HhUxn0EcGPkGrw66Da+TveKJ5HuMQ8ms4FP5JYv+wArKm4RDjhJb+yo85G0F+k8Vq9ET973hcCiiDini8UG9TFcx0DXVpf5Aj5Iujf3IvBnYE4uHw1cn6d3JLXsuAe4n3TLZtDEF6+1QPg/0q/sesa3JfAb4CHg18AWuXwC8KM8/U5gYT5+C4FP1iGudY4HcCYwKU9vBFwBLAHuBHas899dd/Gdlf/W7gFuAt5c5/hmAE8Bq/Pf3yeBE4AT8nwB38/xL6RGi7sBiu/EiuN3O/DOOse3P6le8V5gQX4dPJiOYU9f7mLCzKzBNfytITOzRudEYGbW4JwIzMwanBOBmVmDcyIwM2twTgQNSNKqHi5/gKTryoqnwP57FG+ndadIGl2l/FhJMzqVbZV7Bd2w4LYnSPpugf2f28W8nn4Pm0u6JPdo+XCe3jzPG9DvqCLGAyS9s+L9CZI+MZAxWfecCGy4m0J6LqOzq4GDJL2uouwI4NqIeLG7jUpaPyJaI+Kk/gmzkAuBRyJi54jYCXiU9GRyKfJT2T11AOnZEgAi4vyIuKTfgrJSOBE0sPzr7ebc8dmDki6reGLzfbnsbuDwinU2Vuov/k5J8yUdlsunSLomb+8hSf9Zsc4xefkFkn4oaUQuXyXp67nzsNslbZ3Ld5B0m9IYEV/rFPNUSXcpddj2lVw2VtIiSf+j1D/8DZKaJB1Bevjtsrzvpo7tROqm4LfAoRWbPwqYIelQpXEM5kv6dUVcZ0j6qaRbgZ9W/gqXtFeOeb6kP0gaV7HdMdWOS3efq9P8nYE9ga9WFJ8JTJC0U36/maRfKo2FcL6k9SSNkHSRpPvy8fx83t5Okv5XqaPF30t6cy6/KK97B3C20lgdzRVxPCRp62rHSKkDthOAz+fj/ff5mJ2S1909f8/3SrpaeXyLfGy+lf9G/k/S3+fyt1X83dwrqe79CTWMgX6iza/6v8jjB5B+va0g9YOyHnAb6anJjUg9J+5CekLyF+T+34FvAMfk6WbSE7Qbk355P0V6GrmJ1E3HBNKYC9cCI/M65wGfyNMBHJqnzwZOy9OzK5b5bEW87yUNDq4c73WkvuvHAi8Du+flflER4810PdbDEcDVeXo0sJQ0nsDreW08708B/52nzwDmAU0Vx6/juGwGrJ+n3wNclaerHpdO30PVz9Up1kkdsXYqvzrPOwD4G+lJ+RHA3Pz59gTmVizfnP/9DbBLnt6b1A0HwEV5/yPy++8Ax1Us9+s8XesYnVKxv1ffk57E/cc8fSbw7YrvqGP9gyv28T3g6Dy9Qcdx96v/X0O+0znrszsj4kkASQtIJ9VVwKMR8VAuvxTo6J77vcCkjl95pKSxXZ6eG7lDNUkzSUnlZdLJ6K58sdHEa53XvUQ66UA6wR6Up/cDPpSnfwp8q2Lf7wXm5/ebkJLVH3O8Cyq2NbbAZ/8lcJ6kzYCPkE7eayRtC1yu1CfQBqRbMB1mR0R7lW1tDlycf7UGMLJiXrXjUjkSXlefq6cdH94ZEY/k/czI+/kNsKOk7+XPe4NSr5nvBK7Qa8M0VNaLXBERa/L05cDpwE/Ig/zk8lrHaB1KdRnNEfHbXHQxqRuQDh0dt1V+d7cBX87fx8yOv0frf741ZJX3w9fQfY+0Aj4UEbvn13YR0dFpWuf+SiIvf3HF8uMi4ow8f3Xkn3tV9l2t7xMBZ1Vsa+eIuLCXn4N8Qv9fUp9PR5H6uIH0S/TciNgN+Awp2XX4axeb+ypwU0TsSrrdVLlOteNS9HN1eADYXdKr/2fz9O681snfOvuJiL8A7yD96j6BVKewHrC8Yn+7R8RbuviMtwE7SxpFGpio44Rd6xj1Rsf39+p3FxE/I13ttAPXS3JX3SVxIrBqHgTGVtx7/mjFvDnAv0qv1iWMr5h3kNI4x02kk8atpF+kRyiN9dAxDvL23ez/VtKJGeDoTvv+p/yLFkktHdutYSVpOMGuzAD+jTTozm25bHNe6zL42GorVVG5zpRO86odl0rdfq6IWEK6Yjitovg04O48D2CvXL+yHnAkcItSj7DrRcRVefk9ItWPPCrpw3l/kvSOah8qJ+qrgXNIvW129PLZ1TGqerwjYgXwl477/8DHSXU0XVIa3+KRiPguqefbt9da3nrPicDWERF/I90K+qVSZXHlOARfJd32uFfS/axdeXknqY/2e0m3WVoj4gHSCegGSfeS7l13N2TfycBnJS2kYlSniLgB+BlwW553JbVP8pDueZ+vTpXFFeaS6gcur7g6OYN022QeULSP+7OBsyTNZ92rkXWOS+XMHnyuTwJvUmo6+jDwplzW4S7gXFK31o+STuAtwM35tt+lwLS87NHAJyV19LpbazjNy4FjeO22EHR9jK4FPthRWdxpO8cC0/Pfwe6keoJaPgLcl2PfFXDro5K491HrF5KmkCpBTxzoWMysZ3xFYGbW4HxFYGbW4HxFYGbW4JwIzMwanBOBmVmDcyIwM2twTgRmZg3u/wNqTgb7v3tm2AAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "dataset = [(-1.5, 4),(-0.1, 4),(0.6, 3.1),(2.3, 0.3),(-1.2, 2.4),(1.6, 0.4),(1.7, 1.2),(-0.2, 2.4),(0.7, 2.3),(0.5, 1.75)]\n",
    "\n",
    "plt.title(\"Scatter Plot of Dataset\")\n",
    "plt.xlabel(\"Independent Variable Observations\")\n",
    "plt.ylabel(\"Dependent Variable Observations\")\n",
    "plt.scatter(x=[p[0] for p in dataset], y=[p[1] for p in dataset])\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "985d2fd2-a4d0-4184-b3e4-c274f958caf1",
   "metadata": {},
   "source": [
    "I can use a perceptron to solve the least squares linear regression problem. In fact, the way the architecture of the perceptron is definied makes it really easy to use the stochastic gradient descent method to solve the least squares problem."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d6ef0232-9a94-4156-a116-d7ae573aa88b",
   "metadata": {},
   "source": [
    "If I define the activiation function as the identity function $f(x)=x$, the loss function as the square of the difference between the true value $y_i$ and the predicted value $\\hat y_i$, what I have is basically everything needed to perform least squares stochastic gradient descent, just in perceptron form."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c5e650b5-69f9-4696-9590-5fbdc8c27f50",
   "metadata": {},
   "source": [
    "Recalling the formulas from my section on linear least squares stochastic gradient descent:"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8964becc-e91a-4ef6-9156-62ecd3cd2c8b",
   "metadata": {},
   "source": [
    "$$\n",
    "MSE = \\frac{1}{n} \\sum\\limits_{i = 1}^{n} (y_i - \\hat{\\beta}_0 - \\hat{\\beta}_1 x_i)^2\n",
    "$$\n",
    "$$\n",
    "\\nabla MSE =\n",
    "\\begin{bmatrix}\n",
    "    MSE_{\\hat{\\beta}_0} \\\\\n",
    "    MSE_{\\hat{\\beta}_1}\n",
    "\\end{bmatrix}\n",
    "=\n",
    "\\begin{bmatrix}\n",
    "    \\frac{1}{n} \\sum\\limits_{i = 1}^{n} -2(y_i - \\hat{\\beta}_0 - \\hat{\\beta}_1 x_i) \\\\\n",
    "    \\frac{1}{n} \\sum\\limits_{i = 1}^{n} -2x_i(y_i - \\hat{\\beta}_0 - \\hat{\\beta}_1 x_i)\n",
    "\\end{bmatrix}\n",
    "$$\n",
    "\n",
    "$$\n",
    "\\begin{bmatrix}\n",
    "    {\\hat{\\beta}_0}_{n+1} \\\\\n",
    "    {\\hat{\\beta}_1}_{n+1}\n",
    "\\end{bmatrix}\n",
    "=\n",
    "\\begin{bmatrix}\n",
    "    {\\hat{\\beta}_0}_n \\\\\n",
    "    {\\hat{\\beta}_1}_n\n",
    "\\end{bmatrix}\n",
    "- \\eta\n",
    "\\nabla MSE ({\\hat{\\beta}_0}_n, {\\hat{\\beta}_1}_n)\n",
    "$$"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "446f58f5-b1c1-4509-a6d6-9b1ccde3e611",
   "metadata": {},
   "source": [
    "I want to rewrite and rearrange these formulas a bit so that it is a little bit more clear how these formulas fit into the perceptron model."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6b17ca31-86ec-498a-9802-372514105305",
   "metadata": {},
   "source": [
    "The perceptron typically separates the bias component and the weight components. Above, I have the bias and weight(s) in one gradient vector, so let me rewrite them separately:"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d20cdede-c709-4353-a6ed-ed3e92f1e17d",
   "metadata": {},
   "source": [
    "$$\n",
    "b_{n+1} = b_n - \\eta MSE_{b}(b_n, w_{1\\, n}) = b_n - \\eta\\left(\\frac{1}{m} \\sum\\limits_{i = 1}^{m} -2(y_i - b_n - w_{1\\, n} x_{i,1})\\right) \\\\\n",
    "w_{1\\, n+1} = w_{1\\, n} - \\eta MSE_{w_1}(b_n, w_{1\\, n}) = w_{1\\, n} - \\eta\\left(\\frac{1}{m} \\sum\\limits_{i = 1}^{m} -2x_{i,1}(y_i - b_n - w_{1\\, n} x_{i,1})\\right)\n",
    "$$"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d661add9-dcbd-4a3a-854d-854419ff768b",
   "metadata": {},
   "source": [
    "Note that I have replaced $\\hat{\\beta}_0$ with $b$ and $\\hat{\\beta}_1$ with $w_1$ so that the variables fit the context of the perceptron. Also a small replacement of $n$ with $m$ is used two avoid confusion with what would otherwise be two different values represented by the same variable $n$."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b1c28229-4720-4bd1-b935-40eba9eb47b6",
   "metadata": {},
   "source": [
    "I can still make some more simplifications. Perceptrons generally process one data point at a time, so I can set $m=1$. Also, $b_n + w_{1\\, n} x_{i,1}$ is the same as prediction value $\\hat y_i$. So rewriting and rearranging, I have:"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "bee7cb1d-264c-4e38-b8ff-51757514fe8f",
   "metadata": {},
   "source": [
    "$$\n",
    "b_{n+1} = b_n + 2\\eta (y_i - \\hat y_i) \\\\\n",
    "w_{1\\, n+1} = w_{1\\, n} + 2\\eta (y_i - \\hat y_i)x_{i,1}\n",
    "$$"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b732d0de-7240-447f-b975-c8a50cd30fd6",
   "metadata": {},
   "source": [
    "Since $\\eta$ is some small constant value, I can pretend that the $2$ is \"absorbed\" by $\\eta$ since I can set $\\eta$ to whatever value I want anyways. Now I have:"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6338b171-ad79-4d2d-a96c-1c4b602a260f",
   "metadata": {},
   "source": [
    "$$\n",
    "b_{n+1} = b_n + \\eta (y_i - \\hat y_i) \\\\\n",
    "w_{1\\, n+1} = w_{1\\, n} + \\eta (y_i - \\hat y_i)x_{i,1}\n",
    "$$"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "71bbb58a-3f8d-44e5-b314-e7cfbfdac004",
   "metadata": {},
   "source": [
    "as formulas for updating the weights (weight in this case) and bias."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "132f8bb4-6210-41ae-860f-5d79850c5d75",
   "metadata": {},
   "source": [
    "Now I have everything needed to code out a perceptron. The perceptron algorithm in words is as follows:\n",
    "1. Initialize the weights and bias to zero (or with random values, it doesn't really matter)\n",
    "2. Set the number of epochs\n",
    "3. Iterate through each epoch\n",
    "    1. For each epoch, randomly shuffle the order of data points in the dataset\n",
    "    2. Iterate through each point in the dataset\n",
    "        1. For each point, feed the $x_i$ value into the perceptron\n",
    "        2. Update the weights and the bias based on the values calculated from the gradient of the loss function"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "53c10948-f0c8-4e9c-b1e7-fd3b8e2f4cc3",
   "metadata": {},
   "source": [
    "After enough epochs, the weights and bias values should converge onto some stable values."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "94e539fe-146d-4d42-8380-d42fdb56cdfa",
   "metadata": {},
   "outputs": [],
   "source": [
    "lr = 0.001\n",
    "\n",
    "class Perceptron():\n",
    "    def __init__(self, input_dataset):\n",
    "        self.dataset = input_dataset.copy()\n",
    "        self.num_features = len(input_dataset[0]) - 1\n",
    "        self.weights = [0] * self.num_features\n",
    "        self.bias = 0\n",
    "        self.activation_func = lambda x: x\n",
    "    \n",
    "    def shuffle_dataset(self):\n",
    "        random.shuffle(self.dataset)\n",
    "    \n",
    "    def process_one_point(self, data_point):\n",
    "        x = data_point[0:self.num_features]\n",
    "        y = data_point[len(data_point) - 1]\n",
    "        weighted_sum = sum([w_d * x_d for w_d, x_d in zip(self.weights, x)]) + self.bias\n",
    "        return x, y, self.activation_func(weighted_sum)\n",
    "    \n",
    "    def update_weights_bias(self, lr, x, y, y_hat):\n",
    "        self.weights = [w_d + lr * (y - y_hat) * x_d for w_d, x_d in zip(self.weights, x)]\n",
    "        self.bias += lr * (y - y_hat)\n",
    "    \n",
    "    def process_dataset(self, lr):\n",
    "        self.shuffle_dataset()\n",
    "        for data_point in self.dataset:\n",
    "            x, y, y_hat = self.process_one_point(data_point)\n",
    "            self.update_weights_bias(lr, x, y, y_hat)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4a4069b5-fc68-4119-a5ac-97bafd379b71",
   "metadata": {},
   "source": [
    "Now that the structure of the perceptron is coded, I can train the perceptron to find a model for the dataset and then graph the result."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "3e887d48-5170-4677-a287-0e67cd4ac539",
   "metadata": {},
   "outputs": [],
   "source": [
    "p = Perceptron(dataset)\n",
    "for epoch in range(0, 100000):\n",
    "    p.process_dataset(lr)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "2131f6e6-9732-483a-b0f1-dee70e316f93",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYIAAAEWCAYAAABrDZDcAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/YYfK9AAAACXBIWXMAAAsTAAALEwEAmpwYAAA6rElEQVR4nO3dd3gU5fbA8e8hCRBqkE4okWIAQUCKdAIWioiIBfTasCA2bKDiT71e9drAjg0RsWJBBKSIKAlVSmjSOwihg5RAgJTz+2Mm3hhSNmWzm+z5PM8+zk49O8E9O3PeeV9RVYwxxgSuYr4OwBhjjG9ZIjDGmABnicAYYwKcJQJjjAlwlgiMMSbAWSIwxpgAZ4nA+IyI1BaReBEJ8sK+nxORL3Ox3Yci8kx+x+Pu+0UROSQi+7yxf2NyyxKB8ZiI3C4iq0XklIjsE5EPRCQsB9vvEJHLUt+r6p+qWkZVk70ScOZxRInI7oyWqepgVX3BC8esDTwGNFbVavm0TxWRk24yPSwiv4lI/xxsn+l5yE8FdRyTe5YIjEdE5DHgVWAYUB5oC9QBZolIcV/GVkjUBg6r6oGcbigiwVksbqaqZYBIYBwwSkT+nbsQTcBSVXvZK8sXUA6IB25IN78McBC4w33/HDAB+BY4ASzH+aIC+AJIARLcfT0ORAAKBLvrxAAvAgvddX4CKgJfAceBpUBEmuO/Dexyly0DOqVZ9hzwZSafJwrYncmyccCLadfD+SV/ANgLDEyzbglgJPAnsB/4EAjNYJ+XuZ87xf1c49z5fYC1wFH3szdKs80O4AngD+BM6jlKt18F6qebdx1wGqjovh8IrHf/HtuAe9z5pdPFFA/UANoAv7sx7QVGAcXdbQR40z0Xx4HVQJOszkVmx/H1v2l7/fNlVwTGE+2BksDEtDNVNR6YDlyeZvbVwPfAecDXwCQRCVHVW3C+JK5S53bQa5kcawBwCxAO1MP5UvrU3d96IO2v3aVA8zTH+l5ESub+Y2aoGs4VUDhwJ/CeiFRwl70CXODGUN9d59n0O1DVX4GewB73s98uIhcA44GHgco45/GndFdXNwJXAmGqmuRhvJOBYJwvdHC+tHvjJPOBwJsicrGqnkwXUxlV3QMkA48AlYB2wKXAfe6+rgA6u5+5PHADcDirc5HFcYwfsURgPFEJOJTJl9Fed3mqZao6QVUTgTdwEkjbHBzrU1XdqqrHgBnAVlX91T3290CL1BVV9UtVPayqSar6Os6v0sicfbRsJQLPq2qiqk7H+UUbKSICDAIeUdUjqnoCeAknkXmiPzBNVWe552okzi/o9mnWeUdVd6lqgqfBuvs6hJMcUdVp7vlUVZ0D/AJ0ymL7Zaq6yD2nO4CPgC7u4kSgLNAQEFVdr6p78+FcGB/L6t6jMakOAZVEJDiDZFDdXZ5qV+qEqqa4RcIaOTjW/jTTCRm8L5P6RkSG4vxKr4Fzm6Qc/0xK+eFwus98yo2hMlAKWOZ8DzohAZ62gKoB7Ex9456rXTi/pFPtOmerbIhIiBvbEfd9T5yrqAtwfviVwrmlk9n2F+Ak8FbuusE4t91Q1dkiMgp4D6gjIhOBoTjJPi/nwviYXREYT/yOc5+6X9qZIlIG57L/tzSza6VZXgyoCaTeCsi3rm5FpBNOneEGoIKqhgHHcL6ACsIhnMR0oaqGua/y6hRuPbEHp9gOgPuruhYQl2ad3Jyvq4EkYImIlAB+wLnaqOqeo+n87xxltP8PgA1AA1UtBzyVZn1U9R1VbQk0xkkuw8j+XFgXx37OEoHJlnub5j/AuyLSQ0RCRCQC+A6nmPpFmtVbikg/t6XLwzgJZJG7bD9QN5/CKovzhXcQCBaRZ3GuCDwmIiXTvTxOIqqaAnyMc8+9iru/cBHp7uEuvgOuFJFL3V/xj+Gcq4U5+QypROQ8EfkXzq/1V1X1MFAc53bZQSDJvTq4Is1m+4GKIlI+zbyyOIXgeBFpCNyb5hitReQSN96TOEXpFA/ORUbHMX7EEoHxiFvcfQrn1+VxYDHOrYtLVfVMmlUn49z//gun6NvPvW8N8DLwtIgcdW/r5MVM4GdgE84tltPk7FZKOM6v2LSvejmM4QlgC7BIRI4Dv+JhjUJVNwI3A+/i/KK+CqeQfjaHMawSkXg3jrtw7tM/6x7jBDAEJ+n8BdwETEkTwwacgvU2929SA+dWz004rYw+xmkBlqqcO+8vnHN+GBjhLsv0XGRyHONHRNWu2kz+EJHncJoz3uzrWIwxnrMrAmOMCXCWCIwxJsDZrSFjjAlwdkVgjDEBrtA9UFapUiWNiIjwdRjGGFOoLFu27JCqVs5oWaFLBBEREcTGxvo6DGOMKVREZGdmy+zWkDHGBDhLBMYYE+AsERhjTICzRGCMMQHOEoExxgQ4r7caEpEgIBaIU9Xe6ZaVAD4HWuJ0YNXfHQwjX01aEceImRvZczSBGmGhDOseSd8W4dlvaPyG/Q2N8Z6CuCJ4CGeIwYzcCfylqvVxxkJ9Nb8PPmlFHMMnribuaAIKxB1NYPjE1UxaEZfttsY/2N/QGO/yaiIQkZo4Y66OyWSVq4HP3OkJwKXZ9Qm//dBJNuw77nEMI2ZuJCEx+R/zEhKTGTFzo8f7ML5lf0NjvMvbVwRv4YwilZLJ8nDcPuTd4QCPARXTryQig0QkVkRiT55Jotfb83jqx9Ucij+TftVz7Dma8XCvmc03/sf+hsZ4l9cSgYj0Bg6o6rK87ktVR6tqK1Vt1ah6OW5tF8G3S3fRdUQMo+du5UxScqbb1ggLzdF843/sb2iMd3nziqAD0EdEdgDfAN1E5Mt068ThjnHrDm1YHqdonKmgYsJzfS5k5sOdaBVRgZemb+CKN+cyc+0+MupJdVj3SEJD/jmGdmhIEMO6ezSQlPED9jc0xru8lghUdbiq1lTVCGAAMDuDkaumALe509e563jUL3b9KmX5dGAbxg1sTUhQMe75Yhk3fbyYdXv+WT/o2yKcl/s1JTwsFAHCw0J5uV9Ta3FSiNjf0BjvKpDxCEQkChiqqr1F5HkgVlWniEhJnIHPWwBHgAGqui2rfbVq1UrTdzqXlJzC10v+5M1ZmziakMiA1rV49PJIKpct4ZXPY4wxhY2ILFPVVhkuK2wD02SUCFIdO5XIO7M389nCHZQMCeKBbvUZ2CGCEsFBGa5vjDGBIqtEUKSeLC5fKoRnejfml0c607buebwyYwOXvzGXn9fszbB+YIwxpoglglR1K5dhzG2t+eLONoSGBDH4y+UMGL2INXHHfB2aMcb4nSKZCFJ1alCZaUM68mLfJmw+EM9Vo+bzxIQ/OHDitK9DM8YYv1GkEwFAcFAxbm5bh+ihUdzV8XwmrthN1xExvB+zhdOJmT9/YIwxgaLIJ4JU5UND+L8rG/PLI11oX78Sr/28kcvemMP01VY/MMYEtoBJBKnOr1Saj29txVd3XUKZEsHc99Vy+n+0iNW7rX5gjAlMAZcIUnWoX4lpQzrx0jVN2Xownj7vzWfY96s4cNzqB8aYwBKwiQCc7ipuuqQ20cOiGNS5LpNX7iFqZAyjZm+2+oExJmAEdCJIVa5kCMN7NmLWo53p3KAyI3/ZxKWvz+GnVXusfmCMKfIsEaRRp2JpPrylJePvbku50BAeHL+C6z/8nVW7jvo6NGOM8RpLBBloV68iUx/syKvXNmXH4ZNc/d4CHv1uJfuOWf3AGFP0WCLIRFAxoX/r2kQPjeLeqHpMXbWXriNjeOe3zSSctfqBMabosESQjbIlQ3iiR0N+fbQLXRtW5o1Zm7j09Rgmr4yz+oExpkiwROCh2hVL8f6/WvLtoLZUKF2ch75ZSb8PFrLiz798HZoxxuSJJYIcuqRuRaY80JHXrruI3X8lcM37C3n4mxU2fq4xptCyRJALQcWEG1rVInpoFA90rc/0Nfvo9noMb87axKmzSb4OzxhjcsQSQR6UKRHM0O6R/PZoFy5rVJW3f9tMt5Fz+HHFblJSrH5gjCkcLBHkg1rnlWLUTRfz/eB2VClXgke+XcU1Hyxk2U6rHxhj/J/XEoGIlBSRJSKySkTWish/MljndhE5KCIr3ddd3oqnILSOOI9J93Xg9eubsfdoAtd+sJAh41cQZ/UDY4wfC/bivs8A3VQ1XkRCgPkiMkNVF6Vb71tVfcCLcRSoYsWEa1vWpEeTanw0Zysfzd3GzLX7uKdzXe7pUo/SJbx5yo0xJue8dkWgjnj3bYj7Cpgb56VLBPPoFZHMHhpF9wur8c7sLXR7PYYflln9wBjjX7xaIxCRIBFZCRwAZqnq4gxWu1ZE/hCRCSJSK5P9DBKRWBGJPXjwoDdDznfhYaG8c2MLfri3HdXKh/LY96vo+/4CYncc8XVoxhgDgBTE07EiEgb8CDyoqmvSzK8IxKvqGRG5B+ivqt2y2lerVq00NjbWq/F6S0qKMnlVHK/O2Mi+46fpfVF1nuzZkJoVSvk6NGNMESciy1S1VYbLCqqbBBF5FjilqiMzWR4EHFHV8lntpzAnglSnzibx0ZxtfDR3KykKgzrV5d4oqx8YY7wnq0TgzVZDld0rAUQkFLgc2JBunepp3vYB1nsrHn9Sqngwj1x+AbMfi6JXk2qMit5C1MgYvo/dZfUDY0yB82aNoDoQLSJ/AEtxagRTReR5EenjrjPEbVq6ChgC3O7FePxOjbBQ3hrQgon3tSc8LJRhE/6gz3vzWbztsK9DM8YEkAK7NZRfisKtoYyoKlNW7eGVGRvYe+w0vZpWY3jPRtQ6z+oHxpi888mtIZMzIsLVzcOZ/VgUj1x2AdEbDnLp63N49ecNnDid6OvwjDFFmCUCPxNaPIiHLmtA9NAoejerzgcxW+k6cg7fLv2TZKsfGGO8wBKBn6pWviRv3NCcSfd3oE7FUjzxw2quenc+v2+1+oExJn9ZIvBzzWuFMWFwO969sQXHEhK58eNFDP5iGTsPn/R1aMaYIsISQSEgIlzVrAa/PdaFoVdcwNzNB7n8jbm8PGO91Q+MMXlmiaAQKRkSxAPdnPpBn+Y1+GjONrqOjGH8EqsfGGNyzxJBIVS1XElGXt+Mnx7oyPmVSjN84mqufGceC7cc8nVoxphCyBJBIda0Znm+u6cd7//rYuLPJHHTmMXc/Xks2w9Z/cAY47lsE4GIdBCR0u70zSLyhojU8X5oxhMiQq+m1fn10S483iOShVsOccWbc/jvtHUcS7D6gTEme55cEXwAnBKRZsBjwFbgc69GZXKsZEgQ90XVJ3pYFP1a1GTM/O10HRnDl4t2kpSc4uvwjDF+zJNEkKROPxRXA6NU9T2grHfDMrlVpWxJXr3uIn56oCP1q5Th6UlruPKd+czfbPUDY0zGPEkEJ0RkOHAzME1EiuGMNmb8WJPw8nw7qC0f3nwxpxKTuPmTxdz12VK2HYzPfmNjTEDJttM5EakG3AQsVdV5IlIbiFJVn9weKqqdznnT6cRkxi3cwajZWzidmMxt7SMY0q0B5UtZPvcXk1bEMWLmRvYcTaBGWCjDukfSt0W4r8MyRYhfDEyTXywR5N7BE2d4Y9ZGvlm6i7DQEB69/AJubFOb4CBrPOZLk1bEMXziahISk/+eFxoSxMv9mloyMPkmT72Pikg/EdksIsdE5LiInBCR4/kfpvG2ymVL8HK/i5j6YEciq5Xlmclr6fn2POZuKlzjQBc1I2Zu/EcSAEhITGbEzI0+isgEGk9+Cr4G9FHV8qpaTlXLqmo5bwdmvOfCGuUZf3dbPrqlJWeTU7h17BLuGLeULQesfuALe44m5Gi+MfnNk0SwX1UDYgjJQCIidL+wGr880pmnejVk6fYj9HhrLs9NWcvRU2d9HV5AqREWmqP5xuQ3TxJBrIh8KyI3ureJ+olIP69HZgpEieAgBnWuR/SwKG5oXYvPf99BlxExjFuwnUR7/qBADOseSWhI0D/mhYYEMax7pI8iMoHGk1ZDn2YwW1X1jmy2KwnMBUoAwcAEVf13unVK4Dyc1hI4DPRX1R1Z7deKxd61fu9xXpy2jgVbDlOvcmme7t2YrpFVfB1WkWethoy3+aTVkIgIUFpV40UkBJgPPKSqi9Kscx9wkaoOFpEBwDWq2j+r/Voi8D5V5df1B/jvtHXsOHyKLhdU5ukrG9Ggqj1HaExhlddWQzVF5EcROeC+fhCRmtltp47U6mOI+0qfda4GPnOnJwCXugnE+JCIcHnjqvzySBeevrIRy//8ix5vz+Pfk9fw10mrHxhT1HhSI/gUmALUcF8/ufOyJSJBIrISOADMUtXF6VYJB3YBqGoScAyomMF+BolIrIjEHjxoTR0LSvHgYtzVqS5zhnXlpja1+WLRTrqMiGbsfKsfGFOUeJIIKqvqp6qa5L7GAZU92bmqJqtqc6Am0EZEmuQmSFUdraqtVLVV5coeHdrko/NKF+eFvk2Y8VBnmtUK4/mp6+j+1lxmb9hPYXsg0RhzLk8SwWG3++kg93UzTmHXY6p6FIgGeqRbFAfUAhCRYKB8TvdtCk5ktbJ8fkcbxt7eChTuGBfLrWOXsGn/CV+HZozJA08SwR3ADcA+YC9wHTAwu41EpLKIhLnTocDlwIZ0q00BbnOnrwNmq/3E9GsiQreGVZn5SGee7d2YVbuO0uOtuTwzaQ1HrH5gTKHkzVZDF+EUgoNwEs53qvq8iDwPxKrqFLeJ6RdAC+AIMEBVt2W1X2s15F/+OnmWt37dxJeL/6RU8SAeurQBt7aLoHiw9V9kjD/JVfNREXlcVV8TkXc5t7UPqjokf8P0jCUC/7R5/wlenLaeOZsOcn6l0vxfr0Zc2qgK1gjMGP+QVSIIzmK71G4l7FvXZKtB1bJ8dkcbojce4MWp67jr81g61K/IM70b07CadU1ljD/LNBGo6k/u5ClV/T7tMhG53qtRmUKra2QVOtavxNeL/+TNXzfR6+15DGhTm0cvv4BKZUr4OjxjTAY8uZE73MN5xgAQElSM29pHEDM0itvaR/Dd0l10HRHD6LlbOZOUnP0OjDEFKqsaQU+gF06LoW/TLCoHNFbVNt4P71xWIyh8thyI56Xp65m94QB1KpbiqV6NuKJxVasfGFOActvFxB6c+sBpYFma1xSge34HaYqu+lXKMPb21nx2RxuKBxXjni+WcdPHi1m3x8Y3MsYfeNL7aIiqJhZQPNmyK4LCLSk5hfFL/uSNWZs4mpDIgNa1ePTySCqXtfqBMd6Up95HRaQB8DLQGCiZOl9V6+ZnkJ6yRFA0HDuVyDuzN/PZwh2UDAni/q71GdghgpLp+uU3xuSPPPU+itPB3AdAEtAVZ/yAL/MvPBOIypcK4Znejfnlkc60rXser/68gcvfnMOM1Xut/yJjCpgniSBUVX/DuXrYqarPAVd6NywTKOpWLsOY21rzxZ1tKBUSzL1fLaf/6EWsiTvm69CMCRieJIIzIlIM2CwiD4jINUAZL8dlAkynBpWZNqQjL/ZtwpYD8Vw1aj6PT1jFgROnfR2aMUWeJzWC1jhPGYcBL+A0Hx2RdqSxgmQ1gqLvWEIio2ZvZtzCHRQPKsZ9XetzZ8fzrX5gTB7ktVh8saou90pkuWCJIHDsOHSSl6av55d1+6lZIZThPRvRq2k1e/7AmFzIa7H4dRFZLyIv5HZgGWNyI6JSaUbf2oqv77qEMiWCuf/r5dzw0e+s3m31A2PyU7aJQFW74rQWOgh8JCKrReRpr0dmjKt9/UpMG9KJl/s1ZdvBk/R5bz5Dv1/F/uNWPzAmP+RoPAIRaQo8DvRX1eJeiyoLdmsosB0/nch70Vv4dP4OgoOE+6LqcVenulY/MCYbebo1JCKNROQ5EVkNvAssxBmD2JgCV65kCMN7NmLWo53p3KAyI3/ZxKWvz+GnVXvs+QNjcsmTGsFY4C+gu6pGqeoHqnrAy3EZk6U6FUvz4S0tGX93W8qHhvDg+BVc9+HvrNp11NehGVPoZJkIRCQI2K6qb6vqnpzsWERqiUi0iKwTkbUi8lAG60SJyDERWem+ns1h/CbAtatXkZ8e7Mir1zZl5+FTXP3eAh79biX7jln9wBhPZTVCGaqa7H6hF1fVnI5MngQ8pqrLRaQssExEZqnqunTrzVPV3jnctzF/Cyom9G9dm15Nq/N+zFY+mbedGav3cW9UPe7uVJfQ4lY/MCYrWSYC13ZggYhMAU6mzlTVN7LaSFX3Anvd6RMish4IB9InAmPyRdmSITzRoyE3tanNyzPW88asTXyz5E+e6NmQPs1q2PMHxmTCkxrBVmCqu27ZNC+PiUgE0AJYnMHidiKySkRmiMiFmWw/SERiRST24MGDOTm0CUC1zivF+/9qybeD2nJemeI89M1K+n2wkBV//uXr0IzxSx43HxWRUqp6KscHECkDzAH+q6oT0y0rB6SoaryI9ALeVtUGWe3Pmo+anEhJUX5YvpvXZm7k4Ikz9G1eg8d7NKRGWKivQzOmQOW1+Wg7EVkHbHDfNxOR9z08cAjwA/BV+iQAoKrHVTXenZ4OhIhIJU/2bYwnihUTrm9Vi+ihUTzQtT7T1+yj2+sxvDlrE6fOJvk6PGP8gid9DS0GrgOmqGoLd94aVc2yuwlxbsh+BhxR1YczWacasF9VVUTaABOAOppFUP54RTBpRRwjZm5kz9EEaoSFMqx7JH1bhPs6rEKloM7hriOnePXnDUz9Yy/VypXkiZ6RXN0snGLFrH5gira89jWEqu5KNyvZg806ALcA3dI0D+0lIoNFZLC7znXAGhFZBbwDDMgqCfijSSviGD5xNXFHE1Ag7mgCwyeuZtKKOF+HVmgU5DmsdV4pRt10Md8PbkeVciV45NtVXPPBQpbttPqBCVyeXBFMAN4ARgGXAA8BrVR1gPfDO5e/XRF0eGU2cUcTzpkfHhbKgie7+SCiwsdX5zAlRflxRRyv/ryBAyfO0KdZDZ7o2ZBwqx+YIiivVwSDgftxmn7GAc3d9wbYk8EXWFbzzbl8dQ6LFROubVmT6KFRDOlWn5lr99FtZAyv/7KRk2esfmAChye9jx5S1X+palVVraKqN6vq4YIIrjDIrPWJtUrxnK/PYekSwTx6RSSzh0bR/cJqvDt7C11HxjBh2W5SUgrVnUpjcsWTVkOviUg5EQkRkd9E5KCI3FwQwRUGw7pHEpqu58vQkCCGdY/0UUSFj7+cw/CwUN65sQU/3Nue6mGhDP1+FX3fX8DSHUcKNI6MTFoRR4dXZnP+k9Po8Mpsq0GZfOXJraErVPU40BvYAdQHhnkzqMKkb4twXu7XlPCwUATny+Tlfk2t1VAO+Ns5bFmnAj/e2563+jfnwPEzXP/h79z/9XJ2HcnxYzT5whokGG/zpFi8RlWbiMgYYIKq/iwiq1S1WcGE+E/+Viw2Rdups0mMnruND+dsJUXh7k7nc29UfcqU8KR3lvxhDRJMfshrsXiqiGwAWgK/iUhlwLp2NAGhVPFgHr7sAqKHRnFl0+q8F72VriNj+C52V4HVD6xBgvE2T4rFTwLtcZqMJuJ0PHe1twMzxp9ULx/Km/2b8+N97alZIZTHJ/xBn/fms3ib99tN+LqYboo+T4rFJYHbge9F5AfgHuCod8Myxj+1qF2Bife25+0BzTkSf5b+oxdx31fLvFo/8Jdiuim6PKkRfAecAL50Z90EhKnq9V6OLUNWIzD+IuFsMh/P28YHMVtJTlHu7HQ+90XVo2zJkHw/lnVjYvIqqxqBJ4lgnao2zm5eQbFEYPzNvmOneW3mBiYuj6NSmRIM634B17WsRZD1X2T8SF6LxctFpG2anV0C2DexMa5q5Uvyxg3NmXx/B+pULMUTP6zmqnfn8/tWe+7SFA6ZXhGIyGpAgRAgEvjTXVQb2GBXBMacS1WZ+sdeXpmxgbijCfS4sBrDezWkTsXSvg7NBLhc3RoSkTpZ7VRVd+ZDbDlmicAUBqcTkxkzbxvvx2wlKVkZ2CGC+7vVp5wX6gfGeCKvNYKuQOoQkmtVNTqf48sRSwSmMNl//DQjZm5kwrLdVCxdnMeuiKR/a6sfmIKX2yuCcGAizsNjy9zZLYFQ4BpV9cnz7ZYITGG0evcxnp+6lqU7/qJhtbI827sx7evbYHym4OQ2EfwITFbVcenm3wpcq6o+eajMEoEprFSVGWv28dL09ez+K4HLG1flqV6NOL+S1Q+M9+U2EWxU1QyfWMlqmbdZIjCF3enEZMYu2M57s7dwNjmF29tH8EC3BpQPtfqB8Z7cNh/NcJmIFAOCMlpmjMleyZAg7ouqT/SwKPq1qMmY+dvpOjKGLxftJCk5xdfhmQCUVSKYKiIfi8jf163u9IfA9Ox2LCK1RCRaRNaJyFoReSiDdURE3hGRLSLyh4hcnKtPYUwhVKVsSV697iJ+eqAjDaqU4elJa7jynfnM23zQ16GZAJNVIngcOAbsFJFlIrIMZzyC48BQD/adBDzmPm/QFrhfRNI/e9ATaOC+BgEf5Cx8Ywq/JuHl+WZQWz68+WJOJSZxyydLuOuzpWw7GO/r0EyA8KT5aCjOYDQAW1U1V71richkYJSqzkoz7yMgRlXHu+83AlGqujez/ViNwBRlZ5KS+XTBDkbN3sLpxGRuax/BkG4NKF/K6gcmb/LUxYSqJqjqaveV2yQQAbQAFqdbFA7sSvN+tzsv/faDRCRWRGIPHrTLZlN0lQgOYnCXekQPjeL6VrUYu2A7USOj+fz3HVY/MF7jSV9DeSIiZYAfgIfdIS9zTFVHq2orVW1VuXLl/A3QGD9UuWwJXu7XlGkPdqJhtXI8O3ktPd+ex5xN9kPI5D+vJgIRCcFJAl+p6sQMVokDaqV5X9OdZ4wBGtcox9d3X8LoW1pyNjmF28YuYeCnS9hywOoHJv94MjDNOTcnRSTbRyJFRIBPgPWq+kYmq00BbnVbD7UFjmVVHzAmEIkIV1xYjV8e6cz/9WpE7I6/6PHWXJ6bspajp876OjxTBGSaCESkq4jsBvaKyC/uff5Uv3iw7w7ALUA3EVnpvnqJyGARGeyuMx3YBmwBPgbuy9WnMCYAlAgO4u7OdYkeFkX/1rX4/PcddBkRw7gF20m0+oHJg6yeLF4K3K6qa0XkOuBl4BZVXSQiK1S1RUEGmspaDRnj2LDvOC9MXceCLYepV7k0T/duTNfIKr4Oy/ip3LYaKq6qawFUdQLQF/hMRPrijFNgjPGhhtXK8eWdlzDm1lakKAz8dCm3jV3C5v0nfB2aKWSySgSJIlIt9Y2bFC4FnsN5AMwY42MiwmWNqzLz4c48fWUjlv/5Fz3ense/J6/hr5NWPzCeySoRPAlUTTtDVXcDXYBXvBmUMSZnigcX465OdZkzrCs3tanNF4t20mVENJ/M387ZJKsfmKxl+2Sxv7EagTHZ27jvBC9OW8e8zYeoW6k0/3dlI7o1rILTmM8EojyNUOZvLBGYwmjSijhGzNzInqMJ1AgLZVj3SPq2OOch+nylqkRvPMCLU9ez7dBJOjWoxNNXNiayWlmvHtf4J0sExvjQpBVxDJ+4moTE5L/nhYYE8XK/pl5PBgCJySl88ftO3vp1E/Fnkrjpkto8ctkFVCxTwuvHNv4jT30NpdlJqfwLyZjAMWLmxn8kAYCExGRGzNxYIMcPCSrGHR3PZ86wrtzStg7jl+wiamQMY+Zts/qBATx7sri9iKwDNrjvm4nI+16PzJgiYs/RhBzN95YKpYvzn6ub8PNDnWhZpwIvTlvPFW/OYda6/RS2OwMmf3lyRfAm0B04DKCqq4DO3gzKmKKkRlhojuZ7W4OqZRk3sA2fDmxNcFAx7v48lps/Wcz6vbnqE9IUAR7dGlLVXelmJWe4ojHmHMO6RxIa8s/RXUNDghjW3SfDfv+ta2QVZjzUif/0uZC1e45z5TvzeOrH1RyKP+PTuEzB8yQR7BKR9oCKSIiIDAXWezkuY4qMvi3CeblfU8LDQhEgPCy0wArF2QkJKsZt7SOIGRrFbe0j+G7pLrqOiGH03K2cSbLfe4HCkxHKKgFvA5cBgtPh3EOqetj74Z3LWg0Z4z1bDsTz0vT1zN5wgDoVS/FUr0Zc0biqPX9QBFjzUWNMjszZdJAXp65j84F42tWtyDO9G9O4Rjlfh2XyIFeJQETeJYvO5VR1SP6ElzOWCIwpGEnJKYxf8idvzNrE0YREBrSuxaOXR1K5rD1/UBhllQiCs9jOvm2NCWDBQcW4pV0EfZqF8+7szYxbuIOfVu3l/q71GdghgpLpCuCm8PL41pCIlANUVX3ax61dERjjG9sOxvPS9A38un4/tc4L5amejejRpJrVDwqJPD1ZLCKtRGQ18AewRkRWiUjL/A7SGOPf6lYuw5jbWvHlnZdQKiSYe79aTv/Ri1gTd8zXoZk88qT56FjgPlWNUNU6wP3Ap94Nyxjjrzo2qMS0IR357zVN2HIgnqtGzefxCas4cOK0r0MzueRJIkhW1Xmpb1R1PpCU3UYiMlZEDojImkyWR4nIsTTjGT/redjGGF8KDirGvy6pQ/TQKO7uVJcfV8TRdUQM70Vv4XSiPX9Q2GTVauhid/JWIBQYj9OKqD9wWlUfzXLHIp2BeOBzVW2SwfIoYKiq9s5JwFYjMMb/7Dh0kpemr+eXdfsJDwvlqV6N6NXU6gf+JLethl5P9/7faaazrTCr6lwRicg+PGNMYRdRqTSjb23Fwi2HeH7qOu7/ejmtIyrwTO/GXFQzzNfhmWx49YEyNxFMzeKK4AdgN7AH5+pgbSb7GQQMAqhdu3bLnTt3eiliY0xeJaco38XuYuTMjRw+eZZrL67J4z0iqVqupK9DC2h5frJYRK4ELgT+/kuq6vMebBdB5omgHJCiqvEi0gt4W1UbZLdPuzVkTOFw/HQi70Vv4dP5OwgOEu7tUo+7O9e15w98JK/NRz/EqQs8iNPX0PVAnbwGparHVTXenZ4OhLj9GhljioByJUMY3rMRsx7tTOcGlXl91iYufX0OU1btsfEP/IwnrYbaq+qtwF+q+h+gHXBBXg8sItXErSSJSBs3Fp90ZGeM8Z46FUvz4S0t+WZQW8qHhjBk/Aqu+/B3Vu466uvQjMuTRJA6jNIpEakBJALVs9tIRMYDvwORIrJbRO4UkcEiMthd5TrcB9SAd4ABaj8TjCmy2tatyE8PduS1ay9i5+FT9H1vAY9+u5J9x+z5A1/zpBvqZ4B3gUuB93BaDI1R1We8H965rEZgTOEXfyaJ96O3MGb+doJEGNylHoM61yW0uNUPvCXfuqEWkRJASVX12TPllgiMKTp2HTnFKzM2MG31XqqXL8mTPRvSp1kNe/7AC3LbDXU3VZ0tIv0yWq6qE/MxRo9ZIjCm6Fmy/QjPT13LmrjjtKgdxjO9G3Nx7Qq+DqtIye0DZV2A2cBVGSxTwCeJwBhT9LQ5/zym3N+RH5bv5rWZG+n3/kL6Nq/B4z0aUiMs1NfhFXlZ3hoSkWLAdar6XcGFlDW7IjCmaDt5JokPYrYyet42ignc07ke93SpS6niWf1uNdnJ9XMEqpoCPO6VqIwxJgOlSwQztHsksx/rwmWNqvL2b5vpNnIOP67YTUqKNSz0Bk+aj/4qIkNFpJaInJf68npkxpiAVrNCKUbddDETBrejSrkSPPLtKq75YCHLdv7l69CKHE+aj27PYLaqal3vhJQ1uzVkTOBJSVF+XBHHazM3sP/4Gfo0q8ETPRsSbvUDj+Vb81F/YInAmMB18kwSH83ZykdztwEwqHNdBnepR+kSWdcPJq2IY8TMjew5mkCNsFCGdY+kb4vwggjZb+RHp3NNgMb8s9O5z/MtwhywRGCMiTuawGs/b2Dyyj1UKVuCx3s0pF+LcIoVO/f5g0kr4hg+cTUJaQbMCQ0J4uV+TQMqGeS107l/4zxZ/C7QFXgN6JOvERpjTA6Eh4Xy9oAW/HBve6qHhTL0+1Vc/d4Clu44cs66I2Zu/EcSAEhITGbEzI0FFa7f86RYfB1O9xL7VHUg0Awo79WojDHGAy3rVODHe9vzVv/mHDxxhus//J37v17OriOn/l5nz9GEDLfNbH4g8qjTObcZaZI7hsABoJZ3wzLGGM8UKyb0bRHO7KFdePiyBvy2fj+XvjGHETM3EH8mKdMH0uxBtf/xJBHEikgY8DGwDFiO06uoMcb4jVLFg3n4sguIHhrFlU2r8170VrqOjKFzg0qUDP7nV11oSBDDukf6KFL/k1VfQ+8BX6vqgjTzIoByqvpHwYR3LisWG2M8seLPv3h+6jpW/HmUmmGhnE5K5nD8WWs1lIGs2lxtAkaKSHXgO2C8qq7wRoDGGJPfWtSuwMR72/PTH3t5Zfp6DsWfpWeTagzv2YjaFUv5Ojy/4skDZXWAAe4rFBiPkxQ2eT+8c9kVgTEmpxLOJjNm3jbej9lKcopyR8fzub9rPcqWDPF1aAUmP8cjaAGMBS5SVZ+MIGGJwBiTW/uOnWbEzI38sHw3lcoUZ+gVkVzfqhZBGTx/UNTk9TmCYBG5SkS+AmYAG4EMxyhIt91YETkgImsyWS4i8o6IbBGRP0Tk4uz2aYwxeVGtfElev6EZk+/vQETF0jw5cTW9353P71sDe7j0TBOBiFwuImOB3cDdwDSgnqoOUNXJHux7HNAji+U9gQbuaxDwgadBG2NMXjSrFcb3g9sx6qYWHE9I5MaPF3HPF7HsPHzS16H5RFZXBMOBhUAjVe2jql+rqsdnSVXnAuc+5vc/VwOfq2MREOYWpo0xxutEhN4X1eC3x7owrHsk8zYf4vI35vLy9PUcP53o6/AKVKaJQFW7qeoYVfVWn6/hwK4073e784wxpsCUDAni/q71iRkaxdXNazB63ja6jojh68V/khwg4x948kCZz4nIIBGJFZHYgwcP+jocY0wRVKVcSUZc34wp93ekXuUyPPXjaq58Zx4LtxzydWhe58tEEMc/u6qo6c47h6qOVtVWqtqqcuXKBRKcMSYwNa1Znm/vacv7/7qY+DNJ3DRmMXd/Hsv2Q0W3fuDLRDAFuNVtPdQWOKaqe30YjzHGAE79oFfT6vz6aBce7xHJwi2HuOLNOfx32jqOJRS9+oHXBqYRkfFAFFAJ2A/8GwgBUNUPRUSAUTgti04BA1U12wcE7DkCY0xBO3DiNK/P3MR3y3ZRoVRxHrn8Am5sXYvgoEJxdx2wEcqMMSZfrIk7xgtT17F4+xEuqFqGZ3o3plODwnG7Ok8PlBljjHE0CS/PN4Pa8uHNF5OQmMwtnyzhznFL2Xow3teh5YklAmOMyQERoUcTp37wZM+GLN5+hO5vzuX5n9Zx7FThrB9YIjDGmFwoERzE4C71iB4axfWtavHpwu10GRnN57/vICk5xdfh5YglAmOMyYPKZUvwcr+mTHuwE42qlePZyWvp+fY85mwqPM88WSIwxph80LhGOb6++xJG39KSs8kp3DZ2CQM/XcKWA/5fP7BEYIwx+UREuOLCavzySGf+r1cjYnf8Rfe35vLclLUcPXXW1+FlyhKBMcbksxLBQdzduS4xw6IY0LoWn/++gy4jYhi3YDuJflg/sERgjDFeUrFMCf57TVOmP9SJpuHlee6ndfR4ay7RGw/4OrR/sERgjDFe1rBaOb64sw2f3NaKFIWBny7l1rFL2Lz/hK9DAywRGGNMgRARLm1UlZkPd+aZ3o1Z+edf9Hh7Hs9OXsORk76tH1giMMaYAlQ8uBh3djyfmGFd+dcltflq8Z9EjYjmk/nbOZvkm/qBJQJjjPGB80oX5/mrmzDjoU40qxXGC1Od+sFv6/dT0H3AWSIwxhgfuqBqWT6/ow2f3t4aBO78LJZbxy5h476Cqx9YIjDGGB8TEbo2rMLMhzvz76sa88fuY/R8ey5PT1rN4fgzXj++JQJjjPETIUHFGNjhfGKGRnFruwjGL9lF1MgYxszb5tX6gSUCY4zxMxVKF+e5Phcy8+FOtKxTgRenreeKN+fwy9p9XqkfWCIwxhg/Vb9KWcYNbMOnA1sTHFSMQV8s419jFrN+7/F8PY4lAmOM8XNdI6sw46FO/KfPhazbe5wr35nH8ImrOZRP9QOvJgIR6SEiG0Vki4g8mcHy20XkoIisdF93eTMeY4wprEKCinFb+whihkZxW/sIvo/dRdcRMXw0ZytnkpLztG9vDl4fBGwCLgd2A0uBG1V1XZp1bgdaqeoDnu7Xxiw2xhjYciCel6avZ/aGA9Q+rxRP9WpE9wurIiIZru+rMYvbAFtUdZuqngW+Aa724vGMMSZg1K9ShrG3t+bzO9pQMqQYg79cxo0fL2LtnmM53pc3E0E4sCvN+93uvPSuFZE/RGSCiNTKaEciMkhEYkUk9uDBwjPqjzHGeFvnCyozfUgnXujbhI37TtD73fk8+cMfHDzhef3A18Xin4AIVb0ImAV8ltFKqjpaVVupaqvKlSsXaIDGGOPvgoOKcUvbOsQM68qdHc5nwrLddB0ZwwcxWzmdmH39wJuJIA5I+wu/pjvvb6p6WFVT09YYoKUX4zHGmCKtfGgIT/duzC+PdKZt3Yq8+vMGLn9zDjNW781yu2AvxrQUaCAi5+MkgAHATWlXEJHqqpoaYR9gvRfjMcYUUZNWxDFi5kb2HE2gRlgow7pH0rdFRneiA0PdymUYc1sr5m8+xAtT13HvV8uzXN9rVwSqmgQ8AMzE+YL/TlXXisjzItLHXW2IiKwVkVXAEOB2b8VjjCmaJq2IY/jE1cQdTUCBuKMJDJ+4mkkr4rLdtqjr2KAS04Z05L/XNMlyPa81H/UWaz5qjEmrwyuziTuacM788LBQFjzZzQcR+SdfNR81xhiv25NBEshqvjmXJQJjTKFWIyw0R/PNuSwRGGMKtWHdIwkNCfrHvNCQIIZ1j/RRRPlv0oo4Orwym/OfnEaHV2bne/3Dm62GjDHG61JbBxXVVkOpxfAE93mA1GI4kG+f0RKBMabQ69sivMh88ac3YubGv5NAqoTEZEbM3Jhvn9luDRljjB8riGK4JQJjjPFjBVEMt0RgjDF+rCCK4VYjMMYYP1YQxXBLBMYY4+e8XQy3W0PGGBPgLBEYY0yAs0RgjDEBzhKBMcYEOEsExhgT4ArdeAQichDYmcvNKwGH8jGc/Obv8YH/x2jx5Y3Flzf+HF8dVc1w0PdClwjyQkRiMxuYwR/4e3zg/zFafHlj8eWNv8eXGbs1ZIwxAc4SgTHGBLhASwSjfR1ANvw9PvD/GC2+vLH48sbf48tQQNUIjDHGnCvQrgiMMcakY4nAGGMCXJFOBCJyvYisFZEUEcm0SZeI7BCR1SKyUkRi/TC+HiKyUUS2iMiTBRjfeSIyS0Q2u/+tkMl6ye65WykiUwogrizPh4iUEJFv3eWLRSTC2zHlML7bReRgmnN2VwHHN1ZEDojImkyWi4i848b/h4hc7GfxRYnIsTTn79kCjq+WiESLyDr3/9+HMljHp+cwx1S1yL6ARkAkEAO0ymK9HUAlf4wPCAK2AnWB4sAqoHEBxfca8KQ7/STwaibrxRfgOcv2fAD3AR+60wOAb/0svtuBUQX97y3N8TsDFwNrMlneC5gBCNAWWOxn8UUBU314/qoDF7vTZYFNGfyNfXoOc/oq0lcEqrpeVTf6Oo7MeBhfG2CLqm5T1bPAN8DV3o8O3ON85k5/BvQtoONmxZPzkTbuCcClIiJ+FJ9Pqepc4EgWq1wNfK6ORUCYiFQvmOg8is+nVHWvqi53p08A64H0gwX49BzmVJFOBDmgwC8iskxEBvk6mHTCgV1p3u/m3H903lJVVfe60/uAqpmsV1JEYkVkkYj09XJMnpyPv9dR1STgGFDRy3Gdc2xXZn+va91bBhNEpFbBhOYxX/6b81Q7EVklIjNE5EJfBeHedmwBLE63qDCcw78V+hHKRORXoFoGi/5PVSd7uJuOqhonIlWAWSKywf1V4i/xeU1W8aV9o6oqIpm1Na7jnr+6wGwRWa2qW/M71iLkJ2C8qp4RkXtwrl66+TimwmQ5zr+5eBHpBUwCGhR0ECJSBvgBeFhVjxf08fNToU8EqnpZPuwjzv3vARH5EefyPl8SQT7EFwek/cVY052XL7KKT0T2i0h1Vd3rXtYeyGQfqedvm4jE4PxC8lYi8OR8pK6zW0SCgfLAYS/Fk1628alq2ljG4NRi/IlX/83lVdovXVWdLiLvi0glVS2wzt5EJAQnCXylqhMzWMWvz2F6AX9rSERKi0jZ1GngCiDD1go+shRoICLni0hxnOKn11vmuKYAt7nTtwHnXMGISAURKeFOVwI6AOu8GJMn5yNt3NcBs9Wt4BWAbONLd6+4D849Zn8yBbjVbfnSFjiW5hahz4lItdSaj4i0wfkeK6hEj3vsT4D1qvpGJqv59Tk8h6+r1d58Adfg3Js7A+wHZrrzawDT3em6OC07VgFrcW7Z+E18+r8WCJtwfmUXZHwVgd+AzcCvwHnu/FbAGHe6PbDaPX+rgTsLIK5zzgfwPNDHnS4JfA9sAZYAdQv431128b3s/ltbBUQDDQs4vvHAXiDR/fd3JzAYGOwuF+A9N/7VZNHizkfxPZDm/C0C2hdwfB1x6op/ACvdVy9/Ooc5fVkXE8YYE+AC/taQMcYEOksExhgT4CwRGGNMgLNEYIwxAc4SgTHGBDhLBMar3F4au6eb97CIfJCDfTwvIlk+mCciMZJBD65uT5+jcnCsim7M8VltJyLFReQtt3fJzSIyWURqussiMus5syCJSHP3ydvU932kAHuvNYWHJQLjbeNxHqpKa4A7P1siEqSqz6rqr/keWcZOA88AQ7NZ7yWcnicjVbUBTjcHE73VuZ37hHRONcdp3w6Aqk5R1VfyLShTZFgiMN42AbjSfco2tZOuGsA8EfnA7axurYj8J3UDccaHeFVElgPXi8g4EbnOXfasiCwVkTUiMjrdF+8t4vRPv8Z94vQfRKSyiPzgbr9URDqkX0dVT6rqfJyEkCERKQUMBB5R1WR3u09xHgxM7TMoWES+EpH1bsdypdxtXxGnH/s/RGRkVnGJyHMi8oWILAC+EKdTvwvTxBEjIq1EpI2I/C4iK0RkoYhEuuf7eaC/e076p706cq9aZrtx/CYitd3548TpR3+hiGxLc96ri8jcNOe3U2bnxxQ+lgiMV6nqEZyne3u6swYA36nzJOP/qWor4CKgi4hclGbTw6p6sap+k26Xo1S1tao2AUKB3mmWlVLV5jjjEYzNIJy3gTdVtTVwLU4/P7lRH/hTz+1oLBZI/aKOBN5X1UbAceA+EamI8zT5hap6EfCiB3E1Bi5T1RuBb4Eb4O9uKqqraiywAeikqi2AZ4GX1OkC+1mcsRiaq+q36WJ9F/jMjeMr4J00y6rjPD3bG0i9grgJ58n35kAznKdpTRFR6DudM4VC6u2hye5/73Tn3yBOt9/BOF8+jXEe2wfnSy8jXUXkcaAUcB5OVwM/pTkOqjpXRMqJSFi6bS8DGqe5iCgnImVUNT4Pny0zu1R1gTv9JTAEeAvnSuMTEZkKTM0qLnd6iqomuNPfAb8A/8ZJCBPc+eWBz0SkAU7XByEexNcO6OdOf8E/O76bpKopwDoRSe16fCkwVpzO1iap6koPjmEKCbsiMAVhMs7gMBfj/GpfJiLn49yHv9T9VToNp4+gVCfT70RESgLvA9epalPg43TbpO8vJf37YkBb9xdyc1UNz2US2ArUFrezwjRa4iSmDGNRZ2yENjhf4L2Bnz2I62SaHcQBh90rp/78L1m+AES7V0lX8c9zkhtn0kyLe+y5OCOHxQHjROTWPB7D+BFLBMbr3C+1aJzbNalF4nI4X3LH3F+dPTPZPK3UL7hD7i/m69It7w8gIh1xens8lm75L8CDqW9EpHkOPsbfVPUkzhgCb4hIkLuvW3GuUma7q9UWkXbu9E3AfDfm8qo6HXgE5xZLTuP6Fnjc3U/q1VN5/tfF8e1p1j2BU9DOyEL+V8T/FzAvi2MiInWA/ar6Mc6tK/8eg9fkiCUCU1DG43zxpd6+WQWswLm//TWwIPNNHap6FOcqYA0wE+d2RVqnRWQF8CH/u/2U1hCglVsgXYfTW+Q5RGQH8AZwu4jsFpHGGaw2HOc2zyYR2QxcD1yj/+vFcSNwv4isByoAH+B8KU8VkT+A+cCjOYnLNQG3zpJm3mvAy+5nT3u7NxrnltNKEemfbj8PAgPdWG4BzhmAPZ0oYJV7jP44dQ1TRFjvo8YYE+DsisAYYwKcJQJjjAlwlgiMMSbAWSIwxpgAZ4nAGGMCnCUCY4wJcJYIjDEmwP0/3vRV9o+oHVYAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Weight Vector: [-0.8546742291537157]\n",
      "Bias: 2.560876320399005\n",
      "Residual Sum of Squares: 5.467462560012633\n"
     ]
    }
   ],
   "source": [
    "plt.title(\"Optimal Line for Dataset\")\n",
    "plt.xlabel(\"Variable 1 Observations\")\n",
    "plt.ylabel(\"Variable 2 Observations\")\n",
    "plt.scatter(x=[p[0] for p in dataset], y=[p[1] for p in dataset])\n",
    "plt.axline(\n",
    "    (dataset[0][0], p.weights[0] * dataset[0][0] + p.bias),\n",
    "    (dataset[1][0], p.weights[0] * dataset[1][0] + p.bias)\n",
    ")\n",
    "plt.show()\n",
    "\n",
    "print(\"Weight Vector:\", p.weights)\n",
    "print(\"Bias:\", p.bias)\n",
    "print(\"Residual Sum of Squares:\", sum([(y_i - p.bias - p.weights[0] * x_i)**2 for x_i, y_i in dataset]))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7add9a5b-63be-4a9f-a8c2-1146c3078100",
   "metadata": {
    "tags": []
   },
   "source": [
    "### PyTorch Implementation"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b16ae1b4-3067-4ede-a532-23499e4266e6",
   "metadata": {},
   "source": [
    "I can implement a perceptron using PyTorch. I try to structure the PyTorch model such that it is similar to the way I coded the perceptron from scratch above. This is just so I can attempt to understand a bit about what PyTorch is doing."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "1a0104cd-5da6-4871-8c2b-b1121357d5ee",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Weight: -0.8551109433174133\n",
      "Bias: 2.559089422225952\n"
     ]
    }
   ],
   "source": [
    "class Perceptron(nn.Module):\n",
    "    def __init__(self):\n",
    "        super(Perceptron, self).__init__()\n",
    "        #define a linear layer with # of features as input (one in this case) and one output (and with a bias value)\n",
    "        self.ws = nn.Linear(1, 1, bias=True)\n",
    "        #define a linear layer for the activation function\n",
    "        self.af = nn.Identity()\n",
    "    \n",
    "    def forward(self, x):\n",
    "        ws_output = self.ws(x) #feed input data into the linear layer and store the weighted sum as a variable\n",
    "        af_out = self.af(ws_output) #send the weighted sum through an activation function\n",
    "        return af_out #return final output\n",
    "\n",
    "p = Perceptron()\n",
    "\n",
    "#prepare dataset\n",
    "xs, ys = torch.split(torch.tensor(dataset), 1, dim=1)\n",
    "\n",
    "#define measure of loss and how weights should be updated\n",
    "criterion = nn.MSELoss()\n",
    "optimizer = torch.optim.SGD(p.parameters(), lr=0.001)\n",
    "\n",
    "#train dataset\n",
    "for epoch in range(0, 10000):\n",
    "    for i in range(0, len(dataset)):\n",
    "        #plug in input and get predicted output\n",
    "        y_hat = p(xs[i])\n",
    "        \n",
    "        #calculate the loss (the measure between expected output and predicted output)\n",
    "        loss = criterion(ys[i], y_hat)\n",
    "\n",
    "        #backpropagation step\n",
    "        optimizer.zero_grad() #sets the gradients of all optimized torch.Tensors to zero\n",
    "        loss.backward() #calculate the gradient of the loss function\n",
    "        optimizer.step() #update the weights and bias\n",
    "\n",
    "print(\"Weight:\", p.ws.weight.item())\n",
    "print(\"Bias:\", p.ws.bias.item())"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0b015621-4a85-4dbe-a424-7a4003c492cc",
   "metadata": {},
   "source": [
    "Though the code above helps me relate elements to the perceptron I coded from scratch, there are a couple of simplifications I should make to the PyTorch code. I can eliminate the identity linear layer as it doesn't transform my weighted sum at all. The PyTorch is also smart enough to process the whole input dataset at once - I don't need an inner for loop to feed the model every individual point."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "42344ed9-f352-4d87-9713-960b0dc52940",
   "metadata": {},
   "outputs": [],
   "source": [
    "class Perceptron(nn.Module):\n",
    "    def __init__(self):\n",
    "        super(Perceptron, self).__init__()\n",
    "        self.ws = nn.Linear(1, 1, bias=True)\n",
    "    \n",
    "    def forward(self, x):\n",
    "        return self.ws(x)\n",
    "\n",
    "p = Perceptron()\n",
    "\n",
    "xs, ys = torch.split(torch.tensor(dataset), 1, dim=1)\n",
    "\n",
    "criterion = nn.MSELoss()\n",
    "optimizer = torch.optim.SGD(p.parameters(), lr=0.001)\n",
    "\n",
    "for epoch in range(0, 100000):\n",
    "    y_hat = p(xs)\n",
    "\n",
    "    loss = criterion(ys, y_hat)\n",
    "\n",
    "    optimizer.zero_grad()\n",
    "    loss.backward()\n",
    "    optimizer.step()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8823616b-9074-4964-b878-b5d1185088e4",
   "metadata": {},
   "source": [
    "Now I can graph the line that the PyTorch perceptron came up with:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "ffc3c43e-6d34-4339-b905-9052ac577e25",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYIAAAEWCAYAAABrDZDcAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/YYfK9AAAACXBIWXMAAAsTAAALEwEAmpwYAAA6vElEQVR4nO3dd3gU5fbA8e9JAUINvUNoBlAEJCCdICJVRC4qdiwXsIEFVO69lqv+RAVRVOy9o4iIFFGBUAUJvYaOEDpINUBIzu+PmXhDSNmUzW6y5/M8+zg79ewE9+zMeed9RVUxxhgTuIJ8HYAxxhjfskRgjDEBzhKBMcYEOEsExhgT4CwRGGNMgLNEYIwxAc4SgfEZEaklIidFJNgL+35aRD7PwXZvi8gTeR2Pu+/nROSQiOzzxv6NySlLBMZjIjJQRNaIyF8isk9E3hKR8Gxsv0NErkx5r6p/qGpJVU3ySsAZxxEtIrvTW6aqQ1T1WS8csxbwCNBYVavk0T5VRE65yfSwiMwSkRuysX2G5yEv5ddxTM5ZIjAeEZFHgBeBEUAZoDVQG/hFRIr4MrYCohZwWFUPZHdDEQnJZHFTVS0JRAIfA2+IyFM5C9EELFW1l70yfQGlgZPA9WnmlwQOAne6758GJgITgBPAcpwvKoDPgGQgwd3Xo0AEoECIu04M8BywyF3nR6A88AVwHFgKRKQ6/jhgl7tsGdAh1bKngc8z+DzRwO4Mln0MPJd6PZxf8geAvcAdqdYtCowB/gD2A28DYens80r3cye7n+tjd34fYB1w1P3sjVJtswN4DFgNnEk5R2n2q0D9NPP6A6eB8u77O4AN7t9jGzDYnV8iTUwngWpAK+A3N6a9wBtAEXcbAV5xz8VxYA1wSWbnIqPj+PrftL3Of9kVgfFEW6AYMCn1TFU9CUwHuqaafQ3wLVAO+BKYLCKhqnorzpfE1ercDnopg2MNAG4FqgP1cL6UPnL3twFI/Wt3KdAs1bG+FZFiOf+Y6aqCcwVUHbgLGC8iZd1lLwAXuTHUd9d5Mu0OVPVXoAewx/3sA0XkIuAr4EGgIs55/DHN1dWNQC8gXFXPeRjvD0AIzhc6OF/avXGS+R3AKyJymaqeShNTSVXdAyQBDwEVgDZAF+Bed19XAR3dz1wGuB44nNm5yOQ4xo9YIjCeqAAcyuDLaK+7PMUyVZ2oqonAWJwE0jobx/pIVbeq6jFgBrBVVX91j/0t0DxlRVX9XFUPq+o5VX0Z51dpZPY+WpYSgWdUNVFVp+P8oo0UEQEGAQ+p6hFVPQE8j5PIPHEDME1Vf3HP1RicX9BtU63zmqruUtUET4N193UIJzmiqtPc86mqOhf4GeiQyfbLVHWxe053AO8AndzFiUApoCEgqrpBVffmwbkwPpbZvUdjUhwCKohISDrJoKq7PMWulAlVTXaLhNWycaz9qaYT0nlfMuWNiAzH+ZVeDec2SWnOT0p54XCaz/yXG0NFoDiwzPkedEICPG0BVQ3YmfLGPVe7cH5Jp9h1wVZZEJFQN7Yj7vseOFdRF+H88CuOc0sno+0vwkngUe66ITi33VDV2SLyBjAeqC0ik4DhOMk+N+fC+JhdERhP/IZzn7pf6pkiUhLnsn9Wqtk1Uy0PAmoAKbcC8qyrWxHpgFNnuB4oq6rhwDGcL6D8cAgnMV2squHuq4w6hVtP7MEptgPg/qquCcSnWicn5+sa4Bzwu4gUBb7Dudqo7J6j6fzvHKW3/7eAjUADVS0N/CvV+qjqa6raAmiMk1xGkPW5sC6O/ZwlApMl9zbNf4HXRaS7iISKSATwDU4x9bNUq7cQkX5uS5cHcRLIYnfZfqBuHoVVCucL7yAQIiJP4lwReExEiqV5eZxEVDUZeA/nnnsld3/VRaSbh7v4BuglIl3cX/GP4JyrRdn5DClEpJyI3Izza/1FVT0MFMG5XXYQOOdeHVyVarP9QHkRKZNqXimcQvBJEWkI3JPqGC1F5HI33lM4RelkD85FescxfsQSgfGIW9z9F86vy+PAEpxbF11U9UyqVX/Auf/9J07Rt5973xpgFPAfETnq3tbJjZnAT8AmnFssp8nerZTqOL9iU7/qZTOGx4AtwGIROQ78ioc1ClWNA24BXsf5RX01TiH9bDZjWCUiJ9047sa5T/+ke4wTwFCcpPMncBMwJVUMG3EK1tvcv0k1nFs9N+G0MnoPpwVYitLuvD9xzvlhYLS7LMNzkcFxjB8RVbtqM3lDRJ7Gac54i69jMcZ4zq4IjDEmwFkiMMaYAGe3howxJsDZFYExxgS4AvdAWYUKFTQiIsLXYRhjTIGybNmyQ6paMb1lBS4RREREEBsb6+swjDGmQBGRnRkts1tDxhgT4CwRGGNMgLNEYIwxAc4SgTHGBDhLBMYYE+C83mpIRIKBWCBeVXunWVYU+BRogdOB1Q3uYBh5avKKeEbPjGPP0QSqhYcxolskfZtXz3pD4zfsb2iM9+THFcEwnCEG03MX8Keq1scZC/XFvD745BXxjJy0hvijCSgQfzSBkZPWMHlFfJbbGv9gf0NjvMuriUBEauCMufp+BqtcA3ziTk8EumTVJ/z2Q6fYvP+ExzGMnhlHQmLSefMSEpMYPTPO430Y37K/oTHe5e0rgldxRpFKzmB5ddw+5N3hAI8B5dOuJCKDRCRWRGJPnTlH93HzefKHtRw5lXXX7XuOpj/ca0bzjf+xv6Ex3uW1RCAivYEDqrost/tS1XdVNUpVoxpWLc1NrWrx+eKdRI+ewwcLtnP2XEZ5BqqFh2VrvvE/9jc0xru8eUXQDugjIjuAr4ErROTzNOvE445x6w5tWAanaJyhkCDh2b6XMGNYR5rWDOfZqevp/uo8Zm3YT3o9qY7oFklY6PljaIeFBjOim0cDSRk/YH9DY7zLa4lAVUeqag1VjQAGALPTGblqCnC7O93fXcejfrEjq5Ti0ztb8dHAliBw1yex3Pbh78TtO79+0Ld5dUb1a0L18DAEqB4exqh+TazFSQFif0NjvCtfxiMQkWhguKr2FpFngFhVnSIixXAGPm8OHAEGqOq2zPYVFRWlaTudS0xK5vPFO3n1182cOJ3ITZfX4qErL6J8yaJe+TzGGFPQiMgyVY1Kd1lBG5gmvUSQ4s9TZxk3azOfLd5J8SLBDOvSgNvaRFAkxJ6bM8YEtswSQaH6hixboghP97mYmQ92oEXtsjw3bQNXvTKXn9ftS7d+YIwxppAlghT1K5Xi4zta8dEdLQkJDmLQZ8u4+f0lbNh73NehGWOM3ymUiSBF58hKzBjWgf/2uZj1e4/T67X5jJy0hkMnz/g6NGOM8RuFOhEAhAYHcXvbCOYO78zAtnX4NnYXnUfH8M7crZw5l5T1DowxppAr9IkgRZnioTx5dWNmPtSRVnXKMWrGRrqOncdPa61+YIwJbAGTCFLUq1iSDwa25NM7W1EsNIghny/jxvcWs27PMV+HZowxPhFwiSBFx4sqMn1oB57tewlx+07Q+/UFPDZxNQdOnPZ1aMYYk68CNhEAhAQHcWvr2sSM6Mxd7erw3fLddB4dw5sxWzidaPUDY0xgCOhEkKJMWCj/6d2Ynx/qSJt6FXjppziuHDuX6Wv2Wv3AGFPoWSJIpW7Fkrx/exRf3H05JYuGcO8Xy7nhncWsjbf6gTGm8LJEkI529SswbWgHnr+2CVsPnuTqNxYw4ttVHDhu9QNjTOFjiSADwUHCTZfXYs6IaAZ1qMvklfFEj4lh/ByrHxhjChdLBFkoXSyUkT0b8ctDnejQoAKjZ8bR5eW5/Lhqj9UPjDGFgiUCD0VUKME7t0bx5T8vp3RYKA98tYLr3v6NVbuO+jo0Y4zJFUsE2dS2XgWmPtCeF/o1YcfhU1wzfiEPf7OSfcesfmCMKZgsEeRAcJAwoFUt5gyP5p7oekxdtZfOY2J4bdZmEs5a/cAYU7BYIsiFUsVCeax7Q359uBOdG1Zk7C+b6PJyDD+sjLf6gTGmwLBEkAdqlS/Omze3YMKg1pQtUYRhX6/kH28tYsUff/o6NGOMyZLXEoGIFBOR30VklYisE5H/prPOQBE5KCIr3dfd3oonP1xetzxT7m/PS/0vZdefCVz75iIe/HoFe44m+Do0Y4zJUIgX930GuEJVT4pIKLBARGao6uI0601Q1fu9GEe+Cg4Sro+qSc8mVXkrZgvvzd/OT+v2MbhjPQZ3qkvxIt485cYYk31euyJQx0n3baj7Cpgb5yWLhjCiW0NmPdyJLo0qM27WZq4YM5fvV+wmOTlgToMxpgDwao1ARIJFZCVwAPhFVZeks9o/RGS1iEwUkZoZ7GeQiMSKSOzBgwe9GXKeq1muOONvuoxvh7ShUumiPDRhFde+tYhlO61+YIzxD5IfrVtEJBz4HnhAVdemml8eOKmqZ0RkMHCDql6R2b6ioqI0NjbWq/F6S3Ky8v2KeF78aSMHTpyhT9NqPNajIdXDw3wdmjGmkBORZaoale6y/GrmKCJPAn+p6pgMlgcDR1S1TGb7KciJIMWpM+d4Z+5W3pm3DYDBHesyuFM9ShS1+oExxjsySwTebDVU0b0SQETCgK7AxjTrVE31tg+wwVvx+JMSRUN4+KpIZg+PptvFVXht9hY6j4lh4jKrHxhj8p83awRVgTkishpYilMjmCoiz4hIH3edoW7T0lXAUGCgF+PxO9XDw3jtxuZ8d08bqoaHMfzbVfR9cyFLdxzxdWjGmACSb7eG8kphuDWUnuRk5YdV8bw4I459x0/T69KqPN69ITXLFfd1aMaYQsAnt4ZM9gQFCdc2r8Hs4Z0Y1qUBszbsp8vYuYyeuZGTZ875OjxjTCFmicDPFC8SwkNdL2LO8Gh6NanK+Dlb6Twmhm9id1n9wBjjFZYI/FTVMmG8ckMzvr+3LTXKhvHoxNX0Gb+AJdsO+zo0Y0whY4nAzzWvVZZJ97Rl3IBmHDl5lhveXcy9Xyxj15G/fB2aMaaQsIbrBYCIcE2z6lzVuArvzd/GWzFb+XX9Ae5sX4f7OtejVLFQX4dojCnA7IqgAAkrEszQLg2YMzya3k2r8vZcp37w9e9/kGT1A2NMDlkiKICqlCnG2Oub8cN97ahdvgSPT1pD79cX8NtWqx8YY7LPEkEB1rRmOBOHtOGNm5pzPCGRG99bzODPYtl5+JSvQzPGFCBZJgIRaSciJdzpW0RkrIjU9n5oxhMiQu9LqzHrkU6M6BbJ/M2H6Dp2HqOmb+D46URfh2eMKQA8uSJ4C/hLRJoCjwBbgU+9GpXJtmKhwdzXuT4xw6O5plk13p2/jc6jY/hiyU6rHxhjMuVJIjinTj8U1wBvqOp4oJR3wzI5Val0MUZf15Qp97WnXsWS/Pv7tfR6bT4LtxzydWjGGD/lSSI4ISIjgVuAaSIShDPamPFjTWqUYcLg1rx582WcPHOOm99fwt2fxLL9kNUPjDHny7LTORGpAtwELFXV+SJSC4hWVZ/cHiqsnc550+nEJD5cuJ3xs7dwNimZ29tE8ECXBpQJs3zuLyaviGf0zDj2HE2gWngYI7pF0rd5dV+HZQoRvxiYJq9YIsi5AydOM/bnTUyI3UXZ4kV4qOtF3NiyJiHB1njMlyaviGfkpDUkJCb9PS8sNJhR/ZpYMjB5Jle9j4pIPxHZLCLHROS4iJwQkeN5H6bxtkqlivHCPy5l6gPtaVCpJE9MXkvP1+Yzf3PBGge6sBk9M+68JACQkJjE6JlxPorIBBpPfgq+BPRR1TKqWlpVS6lqaW8HZrzn4mpl+HpQa96+pQWnE5O59YPfuevjpWw9eNLXoQWkPUcTsjXfmLzmSSLYr6oBMYRkIBERul9ShV8e7sjIHg1Zsv0I3V6Zx39/XMfRv876OryAUi08LFvzjclrniSCWBGZICI3ureJ+olIP69HZvJF0ZBgBneqx5zh0VwXVZNPFu0gekwMnyzaQWJSsq/DCwgjukUSFhp83ryw0GBGdIv0UUQm0HjSauijdGarqt6ZxXbFgHlAUZxeTieq6lNp1imK83BaC+AwcIOq7shsv1Ys9q71e47z3LT1LNp6mPqVSvKfXo2Ijqzk67AKPWs1ZLzNJ62GRESAEqp6UkRCgQXAMFVdnGqde4FLVXWIiAwArlXVGzLbryUC71NVft1wgP+btp4dh/8iOrIi/+nViPqV7DlCYwqq3LYaqiEi34vIAff1nYjUyGo7daRUH0PdV9qscw3wiTs9EejiJhDjQyJC18aV+fmhTvynVyOW7fyTbq/O5+kp6/jzlNUPjClsPKkRfARMAaq5rx/deVkSkWARWQkcAH5R1SVpVqkO7AJQ1XPAMaB8OvsZJCKxIhJ78KA1dcwvRUKCuLtDXWKGR3Njq5p8+ptTP/ho4XarHxhTiHiSCCqq6keqes59fQxU9GTnqpqkqs2AGkArEbkkJ0Gq6ruqGqWqURUrenRok4fKlyzKc32bMGNYR5pUL8N/f1xPt1fnMXvjfgraA4nGmAt5kggOu91PB7uvW3AKux5T1aPAHKB7mkXxQE0AEQkBymR33yb/RFYpxWd3teKD26NA4c6PY7ntw9/ZtP+Er0MzxuSCJ4ngTuB6YB+wF+gP3JHVRiJSUUTC3ekwoCuwMc1qU4Db3en+wGy1n5h+TUTo0qgyPz3YkSd6N2bVrqP0GDefJyav5YjVD4wpkLzZauhSnEJwME7C+UZVnxGRZ4BYVZ3iNjH9DGgOHAEGqOq2zPZrrYb8y5+nzvLqr5v4fMkfFC8SzLAuDbitTQRFQqz/ImP8SY6aj4rIo6r6koi8zoWtfVDVoXkbpmcsEfinzftP8Oy0DczbdJA6FUrw756N6NKoEtYIzBj/kFkiCMlku5RuJexb12SpQeVSfHpnK+bEHeC5qeu5+9NY2tevwH96N6JhFeuayhh/lmEiUNUf3cm/VPXb1MtE5DqvRmUKrM6RlWhfvwJfLN7JK79upue4+QxoVYuHu15EhZJFfR2eMSYdntzIHenhPGMACA0OYmC7OswdEc1tbSKYsHQXnUfH8O68rZw5l5T1Dowx+SqzGkEPoCdOi6EJqRaVBhqraivvh3chqxEUPFsOnOD/pm1gTtxBapcvzr96NuKqxpWtfmBMPsppFxN7cOoDp4FlqV5TgG55HaQpvOpXKsVHd7TikztbUSQ4iMGfLeOm95awfo+Nb2SMP/Ck99FQVU3Mp3iyZFcEBdu5pGS++v0Pxv6yiaMJiQxoWZOHu0ZSsZTVD4zxplx1OgdEiMhEEVkvIttSXnkcowkQIcFB3NomgpjhnbmzXR2+jd1N5zExvD3X6gfG+Iqnnc69BZwDOuOMH/C5N4MyhV+Z4qE80bsxPz/UkdZ1y/HCjI10HTuPGWv2Wv9FxuQzTxJBmKrOwrmNtFNVnwZ6eTcsEyjqVizJ+7e35LO7WhEWGsw9XyznhncXszb+mK9DMyZgeJIIzohIELBZRO4XkWuBkl6OywSYDg0qMm1oe57rewlbDpzk6jcW8OjEVRw4cdrXoRlT6HlSLG6J85RxOPAsTvPR0alHGstPViwu/I4lJPLG7M18vGgHRYKDuLdzfe5qX4diacb1NcZ4LldDVYrIZaq63CuR5YAlgsCx49Apnp++gZ/X76dG2TBG9mhEzyZV7PkDY3Igt62GXhaRDSLybE4HljEmJyIqlODd26L48u7LKVk0hPu+XM717/zGmt1WPzAmL2WZCFS1M05roYPAOyKyRkT+4/XIjHG1rV+BaUM7MKpfE7YdPEWf8QsY/u0q9h+3+oExeSFb4xGISBPgUeAGVS3itagyYbeGAtvx04mMn7OFjxbsICRYuKdTPf7Zsa7VD4zJQq5uDYlIIxF5WkTWAK8Di3DGIDYm35UuFsrIHo345eGOdGxQkZd/2USXl+cyZdUee/7AmBzypEbwIfAn0E1Vo1X1LVU94OW4jMlU7fIlePvWFnz1z9aUCQtl6Fcr6P/2b6zcddTXoRlT4GSaCEQkGNiuquNUdU92diwiNUVkjts1xToRGZbOOtEickxEVrqvJ7MZvwlwbeqV58cH2vPSPy5l5+G/6Dt+IQ9PWMm+Y1Y/MMZTmY1QhqomuV/oRVQ1uyOTnwMeUdXlIlIKWCYiv6jq+jTrzVfV3tnctzF/Cw4Srm9Zk56XVuXNOVt4f8F2Zqzdx5BO9RjUsS5hRax+YExmMk0Eru3AQhGZApxKmamqYzPbSFX3Anvd6RMisgGoDqRNBMbkiZJFQ3i0e0NubFWLF2Zs5JVfN/H10j94rHtD+jStRlCQPX9gTHo8qRFsBaa665ZK9fKYiEQAzYEl6SxuIyKrRGSGiFycwfaDRCRWRGIPHjyYnUObAFSzXHHG33wZ3wxuQ/mSRXhwwkr6vbWI5X/86evQjPFLHjcfFZHiqvpXtg8gUhKYC/yfqk5Ks6w0kKyqJ0WkJzBOVRtktj9rPmqyIzlZ+W75bl6aGcfBE2e4plk1HuvekGrhYb4OzZh8ldvmo21EZD2w0X3fVETe9PDAocB3wBdpkwCAqh5X1ZPu9HQgVEQqeLJvYzwRFCRcF1WTmOHR3N+5Pj+t3ccVL8cw9pdN/HX2nK/DM8YveNLX0BKgPzBFVZu789aqaqbdTYjTIcwnwBFVfTCDdaoA+1VVRaQVMBGorZkE5Y9XBJNXxDN6Zhx7jiZQLTyMEd0i6du8uq/DKlDy6xzu/vMvXvwpjh9X7aFy6aI81r0hfZtVt/qBKfRy29cQqrorzSxPhpJqB9wKXJGqeWhPERkiIkPcdfoDa0VkFfAaMCCzJOCPJq+IZ+SkNcQfTUCB+KMJjJy0hskr4n0dWoGRn+ewRtnivH5jcyYOaUOV0sV4+JtVXPvmQpbtPJLnxzKmoPDkimAiMBZ4A7gcGAZEqeoA74d3IX+7Imj3wmzijyZcML96eBgLH7/CBxEVPL46h8nJyuSV8bz400b2Hz/D1U2r8Vj3SGqULe61YxrjK7m9IhgC3IfT9DMeaOa+N8CedL7AMptvLuSrcxgUJPS7rAZzhkcztEsDfl63jy4vz2XMzDhOnbH6gQkcnvQ+ekhVb1bVyqpaSVVvUdXD+RFcQZBR6xNrleI5X5/D4kVCeLjrRcweHk33S6rwxpwtdB4Tw7exu0hOLlB3Ko3JEU9aDb0kIqVFJFREZonIQRG5JT+CKwhGdIskLE3Pl2GhwYzoFumjiAoefzmH1cPDGDegOd/d09YpWE9czTXjF/L7dt/XDyaviKfdC7Op8/g02r0w22pQJk95cmvoKlU9DvQGdgD1gRHeDKog6du8OqP6NaF6eBiC82Uyql8TazWUDf52DlvULsuke9oybkAzDp08w/Xv/MZ9Xyxn15FsP0aTJ6xBgvE2T4rFa1X1EhF5H5ioqj+JyCpVbZo/IZ7P34rFpnBLOJvEu/O28fbcrSSpcnf7OtzbuT4li3rSO0vesAYJJi/ktlg8VUQ2Ai2AWSJSEbCuHU1ACCsSzLArGzB7eCd6N6nKmzFbiR4dw4Slf5CUT/UDa5BgvM2TYvHjQFucJqOJOB3PXePtwIzxJ1XLhDH2hmZMvq8dtcqF8dh3a7j69QUs3ub9dhO+Lqabws+TYnExYCDwrYh8BwwGjno3LGP8U7Oa4Xx3T1teu7E5R/86y4B3FzPks2X8cdh79QN/KaabwsuTGsE3wAngc3fWTUC4ql7n5djSZTUC4y9OJybx3rxtvBmzlaRk5Y72EdzfuT6lioXm+bGsGxOTW5nVCDxJBOtVtXFW8/KLJQLjb/YfP81LP8Xx3fLdVChZhEeuiuT6qJoEW/9Fxo/ktli8XERap9rZ5YB9Exvjqly6GC9f35Qp97ejToUSjJy0hl6vzWfR1kO+Ds0Yj2R4RSAiawAFQoFI4A93US1go10RGHMhVWX6mn08P30D8UcTuKpxZf7VsxERFUr4OjQT4HJ0a0hEame2U1XdmQexZZslAlMQnE5M4oMF23lzzhbOJiVzR7s63H9FfUp7oX5gjCdyWyPoDKQMIblOVefkcXzZYonAFCQHjp9mzM9xfLtsN2WLF+HhrhcxoGVNQoI96gHemDyT0yuC6sAknIfHlrmzWwBhwLWq6pPn2y0RmIJobfwxnpm6nt+3HyGycime6N2Y9g1sMD6Tf3KaCL4HflDVj9PMvw34h6r65KEySwSmoFJVflq7j+dnbGDXkQSubFSJf/VsRN2KJX0dmgkAOU0Ecaqa7hMrmS3zNksEpqA7nZjEx4t28MbsLZxOTOL2thEMvaIBZYpb/cB4T06bj6a7TESCgOD0lhljslYsNJghneoxZ3g010XV4MOF24keM4fPftvBuaRkX4dnAlBmiWCqiLwnIn+3e3On3wamZ7VjEakpInNEZL2IrBORYemsIyLymohsEZHVInJZjj6FMQVQxVJFGdXvUqY+0J7IKqV44od19Bg3n7mbDvo6NBNgMksEjwLHgJ0iskxEluGMR3AcGO7Bvs8Bj7jPG7QG7hORtM8e9AAauK9BwFvZC9+Ygu/iamX46p+teefWFpxNSub2D3/njo9+Z8uBk74OzQQIT5qPhuEMRgOwVVVz1LuWiPwAvKGqv6Sa9w4Qo6pfue/jgGhV3ZvRfqxGYAqzM+eS+GTRDl6ftYWExCRuaV2bB69sQHjxIr4OzRRwuepiQlUTVHWN+8ppEogAmgNL0iyqDuxK9X63Oy/t9oNEJFZEYg8etMtmU3gVDQlmUMd6zBkRzfUta/LpbzvoNDqGjxduJ9HqB8ZLvP5Ui4iUBL4DHnSHvMw2VX1XVaNUNapixYp5G6AxfqhCyaI8f20Tpg/rwCXVS/P0j+vp/uo85sQd8HVophDyaiIQkVCcJPCFqk5KZ5V4oGaq9zXcecYYoGGV0nx+1+W8f1sUyQp3fLSU2z/8nc37T/g6NFOIeDIwzQWNm0Uky0ciRUSAD4ANqjo2g9WmALe5rYdaA8cyqw8YE4hEhCsbV2bmgx35T69GLP/jT7qPm89TP6zlz1NnfR2eKQQyTAQi0llEdgN7ReRn9z5/ip892Hc74FbgChFZ6b56isgQERnirjMd2AZsAd4D7s3RpzAmABQJCeLuDnWZO6IzN7WqxWeLd9Jp9Bw+WLCds+esfmByLrMni5cCA1V1nYj0B0YBt6rqYhFZoarN8zPQFNZqyBhH3L4TPDdtPfM3H6JuhRL8u1cjrmhYCedi3Jjz5bTVUBFVXQegqhOBvsAnItIXZ5wCY4wPRVYpxad3tuLDgVEgcNcnsdz24e/E7bP6gcmezBJBoohUSXnjJoUuwNM4D4AZY3xMRLiioVM/eOrqxqzefYwe4+bxn8lrOHzyjK/DMwVEZongcaBy6hmquhvoBLzgzaCMMdkTGhzEHe3qEDM8mtvaRPDV77uIHhPD+/O3Wf3AZCnLJ4v9jdUIjMnalgMneG7aBmLiDhJRvjj/6tmIro0rW/0ggOVqhDJ/Y4nAFESTV8QzemYce44mUC08jBHdIunb/IKH6PPcnLgD/N+0DWw5cJK29crzRO/GNKpa2uvHNf7HEoExPjR5RTwjJ60hITHp73lhocGM6tckX5JBYlIyXy75g1d+3cTxhERuaFmLR666iAoli3r92MZ/5KqvoVQ7KZ53IRkTOEbPjDsvCQAkJCYxemZcvhw/NDiI29tGEDM8mtvbRvBt7C46j47hnblbOXMuKesdmELPkyeL24rIemCj+76piLzp9ciMKST2HE3I1nxvCS9ehKeuvpiZD3WkVZ1yjJqxka5j5/HT2n0UtDsDJm95ckXwCtANOAygqquAjt4MypjCpFp4WLbme1u9iiX5YGBLPr2zFcVCgxjy+TJufG8x6/Yc80k8xvc8ujWkqrvSzLLrSWM8NKJbJGGh54/uGhYazIhuPhn2+28dL6rI9KEdeLbvJcTtO0Hv1xfw+HerOXDitE/jMvkvxIN1dolIW0DdDuiGARu8G5YxhUdKQdgXrYayEhIcxK2ta9OnaTVen7WZjxft4MdVe7jvivrc2a4OxUJtePJA4MkIZRWAccCVgOB0ODdMVQ97P7wLWashY7xn28GTPD99I79u2E+NsmH8q2cjelxSxZ4/KASs+agxJlsWbD7Es1PXE7f/BK0iyvHk1Y25pHoZX4dlciFHiUBEXieTzuVUdWjehJc9lgiMyR/nkpKZELuLsT9v4shfZ+l/WQ1GdIukUulivg7N5EBmiSCzGoF92xoTwEKCg7j58tpc3bQa42dv4cOF25m2Zi/3da7PXe2tflCYeHxrSERKA6qqPu3j1q4IjPGNHYdOMWrGBmau20/18DBG9mxIryZVrX5QQOTqyWIRiRKRNcBqYK2IrBKRFnkdpDHGv0VUKME7t0bx5T8vp3RYKPd/uYLr3v6N1buP+jo0k0uePEfwIXCvqkaoam3gPuAj74ZljPFXbetVYOoD7XmhXxN2HD5FnzcW8vA3K9l3zJ4/KKg8SQRJqjo/5Y2qLgDOZbWRiHwoIgdEZG0Gy6NF5Fiq8Yyf9DxsY4wvBQcJA1rVYs7waIZ0qsfUVXvpPCaG12ZtJuGsPW9a0GTWaugyd/I2IAz4CqcV0Q3AaVV9ONMdi3QETgKfquol6SyPBoarau/sBGw1AmP8zx+H/+KFnzYwfc0+qpUpxmM9GtKnaTWrH/iRnLYaejnN+6dSTWdZYVbVeSISkXV4xpiCrlb54rx5cwuWbDvMM1PXM+zrlXyyaAdP9G5M81plfR2eyYJXHyhzE8HUTK4IvgN2A3twrg7WZbCfQcAggFq1arXYuXOnlyI2xuRWUrLy3fLdjJ4Zx8ETZ7i2eXUe7R5J1TK+6WTPOHL9ZLGI9AIuBv5+kkRVn/FguwgyTgSlgWRVPSkiPYFxqtogq33arSFjCoaTZ87xVswW3pu/nSCBwR3rMbhTXYoX8aSLM5PXctt89G2cusADOH0NXQfUzm1QqnpcVU+609OBULdfI2NMIVCyaAgjujVk1sOd6NKoMuNmbeaKMXP5fsVukpMLVtc2hZ0nrYbaquptwJ+q+l+gDXBRbg8sIlXErSSJSCs3Fp90ZGeM8Z6a5Yoz/qbL+HZIGyqWKspDE1Zx7VuLWLbzT1+HZlyeJIKUYZT+EpFqQCJQNauNROQr4DcgUkR2i8hdIjJERIa4q/THfUANeA0YoAWtBzxjjMdaRpTjh/vaMea6puw9msA/3lrE0K9WEJ/PI7WZC3nSDfUTwOtAF2A8Touh91X1Ce+HdyGrERhT8J06c4535m7lnXnbABjcsS6DO9WjRFGrH3hLnnVDLSJFgWKq6rMx7SwRGFN4xB9N4MUZG5myag+VSxfl0W4NubZ5dYKC7PmDvJbTbqivUNXZItIvveWqOikPY/SYJQJjCp9lO4/wzNQNrNp1lEtrlOHJ3o2Jiijn67AKlZw+UNYJmA1cnc4yBXySCIwxhU+L2uX4/p62/LAqnhdnxNH/7d/odWlVHu/ekJrlivs6vEIv01tDIhIE9FfVb/IvpMzZFYExhdtfZ8/xztxtvDNvK8kK/+xQh3ui61PS6ge5kuPnCFQ1GXjUK1EZY0w6ihcJ4aGuFzH7kWh6XlKF8XO20nlMDN/E7rLnD7zEk+ajv4rIcBGpKSLlUl5ej8wYE9CqhYfx6oDmfH9vW2qUDePRiavpM34BS7bZ40Z5zZPmo9vTma2qWtc7IWXObg0ZE3hUlSmr9vDijI3sOXaank2qMLJHI6sfZEOeNR/1B5YIjAlcCWeTeG/+Nt6K2UpSsnJn+zrc17kepYqFZrrd5BXxjJ4Zx56jCVQLD2NEt0j6Nq+eT1H7h7zodO4SoDHndzr3aZ5FmA2WCIwx+46d5qWZG5m0PJ4KJYsw/KpIrouqSXA6zx9MXhHPyElrSEj834A5YaHBjOrXJKCSQW47nXsK58ni14HOwEtAnzyN0BhjsqFKmWKMvb4ZP9zXjtrlS/D4pDX0fn0Bv229sH4wembceUkAICExidEz4/IrXL/nSbG4P073EvtU9Q6gKVDGq1EZY4wHmtYMZ+KQNrx+Y3OOJyRy43uLGfxZLDsPn/p7nT0Z9GWU0fxA5FGnc24z0nPuGAIHgJreDcsYYzwjIlzdtBqzHunEiG6RzN98iK5j5zFq+gaOn06kWnj6A+JkND8QeZIIYkUkHHgPWAYsx+lV1Bhj/Eax0GDu61yfmOHRXNOsGu/O30bn0TG0q1+eYiHnf9WFhQYzolukjyL1P5n1NTQe+FJVF6aaFwGUVtXV+RPehaxYbIzxxJrdx3h26np+33GEamWKceZcMkdOnbVWQ+nI7JntTcAYEakKfAN8paorvBGgMcbktSY1yjBhcGtmrN3H89M3cPjUWa5sVJl/92pEnQolfB2eX/HkgbLawAD3FQZ8hZMUNnk/vAvZFYExJrtOJybx4cLtjJ+9hbNJydzeJoIHujSgTFjmzx8UJnk5HkFz4EPgUlUNzqP4ssUSgTEmpw6cOM3LMzfxzbJdlC1ehIe6XsSNLWsSEuxJubRgy+1zBCEicrWIfAHMAOKAdMcoSLPdhyJyQETWZrBcROQ1EdkiIqtF5LKs9mmMMblRqVQxXux/KT/e354GlUryxOS19HxtPvM3H/R1aD6VYSIQka4i8iGwG/gnMA2op6oDVPUHD/b9MdA9k+U9gAbuaxDwlqdBG2NMblxSvQxfD2rN27e04HRiMrd+8Dt3fbyUrQdP+jo0n8jsimAksAhopKp9VPVLVT2VyfrnUdV5wJFMVrkG+FQdi4FwtzBtjDFeJyJ0v6QKvzzckZE9GrJk+xG6vTKPZ35cz7G/En0dXr7KsNWQql7h5WNXB3aler/bnbfXy8c1xpi/FQ0JZnCnevS7rAZjf9nEx4u2M2nFbh668iJuurwWoQFQPygQn1BEBolIrIjEHjwY2PfyjDHeUbFUUUb1a8LUBzrQqEppnpqyjh7j5hMTd8DXoXmdLxNBPOd3VVHDnXcBVX1XVaNUNapixYr5EpwxJjA1rlaaL/95Oe/e2oLEpGQGfrSUgR/9zpYDJ3wdmtf4MhFMAW5zWw+1Bo6pqt0WMsb4nIhw1cVV+Pmhjvy7ZyOW7fyTbq/O5+kp6/jz1Flfh5fnvDYatIh8BUQDFURkN/AUEAqgqm8D04GewBbgL+AOb8VijDE5UTQkmH92rEu/y6rzyq+b+PS3HXy/Ip4Hr2zALa1rF5r6gY1QZowxHorbd4Jnp65nwZZD1K1Ygid6NSY6siIiFw6I429y9UCZMcYYR2SVUnx2Vys+uD0KFO74eCm3f7SUTfsLdv3AEoExxmSDiNClUWV+erAjT/RuzMo//qTHuPk8MXktRwpo/cASgTHG5ECRkCDual+HmBGdufnyWnz5+x90Gj2H9+dv4+y5ZF+Hly2WCIwxJhfKlSjCM9dcwk/DOtC8Vlmem7aBbq/O49f1+ykoNVhLBMYYkwcaVC7Fp3e24qM7WhIkcPensdz6we9s3Hfc16FlyRKBMcbkoc6RlfjpwY48fXVj1sQfo+e4+fz7+zUcPnnG16FlyBKBMcbksdDgIAa2q8PcEdHc1iaCr5fuInp0DO/O28qZc0m+Du8ClgiMMcZLwosX4ek+FzPzwQ5ERZTl+ekbueqVecxct8+v6geWCIwxxsvqVyrFR3e04uM7WhIaHMTgz5Zx03tLWL/HP+oHlgiMMSafREdW4qdhHXjmmovZuO84vV6fz8hJqzl4wrf1A0sExhiTj0KCg7itTQQxwztzZ7s6fBu7m85jYnh7ru/qB5YIjDHGB8oUD+WJ3o35+aGOtK5bjhdmbKTr2Hn8tHZvvtcPLBEYY4wP1a1Ykvdvb8lnd7UiLDSYIZ8vZ8C7i1kbfyzfYrBEYIwxfqBDg4pMG9qe5/pewuYDJ7n6jQU8OnEVB06c9vqxLREYY4yfCAkO4pbWtZkzPJq729fh+xXxdB4dw/g5Wzid6L36gSUCY4zxM2XCQvl3r8b8/FAn2tWvwOiZcVw5di7TVnunfmCJwBhj/FSdCiV497Yovrz7ckoWDeG+L5dz/Tu/sWZ33tYPLBEYY4yfa1u/AtOGdmBUvyZsO3iKPuMXMPzbVew/njf1A68mAhHpLiJxIrJFRB5PZ/lAETkoIivd193ejMcYYwqq4CDhxla1mDMimkEd6zJl5R46j4nh9Vmbc10/8NqYxSISDGwCugK7gaXAjaq6PtU6A4EoVb3f0/3amMXGGAM7D59i1PSN/LRuH9XDw3isR0OuvrRqhuMn+2rM4lbAFlXdpqpnga+Ba7x4PGOMCRi1y5fg7Vtb8NU/W1MmLJShX62g/9u/sXLX0Wzvy5uJoDqwK9X73e68tP4hIqtFZKKI1ExvRyIySERiRST24MGD3ojVGGMKpDb1yvPjA+158R9N2Hn4L/qOX8jDE1ay75jn9QNfF4t/BCJU9VLgF+CT9FZS1XdVNUpVoypWrJivARpjjL8LDhJuaFmLOcM7cU90Paau2UvnMTGM+3UzCWezrh94MxHEA6l/4ddw5/1NVQ+rakq3e+8DLbwYjzHGFGqlioXyWPeGzHq4E1c0rMQrv27iipdj+GFlfKbbeTMRLAUaiEgdESkCDACmpF5BRKqmetsH2ODFeIwxhdTkFfG0e2E2dR6fRrsXZjN5ReZffIVdzXLFGX/zZXwzuA3lSxZh2NcrM13fa4lAVc8B9wMzcb7gv1HVdSLyjIj0cVcbKiLrRGQVMBQY6K14jDGF0+QV8YyctIb4owkoEH80gZGT1gR8MgBoVaccU+5rz+j+l2a6nteaj3qLNR81xqTW7oXZxB9NuGB+9fAwFj5+hQ8i8k++aj5qjDFetyedJJDZfHMhSwTGmAKtWnhYtuabC1kiMMYUaCO6RRIWGnzevLDQYEZ0i/RRRHnP28XwkDzdmzHG5LO+zZ3nVEfPjGPP0QSqhYcxolvk3/MLupRieILbn1BKMRzIs89oicAYU+D1bV690HzxpzV6ZtzfSSBFQmISo2fG5dlntltDxhjjx/KjGG6JwBhj/Fh+FMMtERhjjB/Lj2K41QiMMcaP5Ucx3BKBMcb4OW8Xw+3WkDHGBDhLBMYYE+AsERhjTICzRGCMMQHOEoExxgS4AjcegYgcBHbmcPMKwKE8DCev+Xt84P8xWny5Y/Hljj/HV1tV0x30vcAlgtwQkdiMBmbwB/4eH/h/jBZf7lh8uePv8WXEbg0ZY0yAs0RgjDEBLtASwbu+DiAL/h4f+H+MFl/uWHy54+/xpSugagTGGGMuFGhXBMYYY9KwRGCMMQGuUCcCEblORNaJSLKIZNikS0R2iMgaEVkpIrF+GF93EYkTkS0i8ng+xldORH4Rkc3uf8tmsF6Se+5WisiUfIgr0/MhIkVFZIK7fImIRHg7pmzGN1BEDqY6Z3fnc3wfisgBEVmbwXIRkdfc+FeLyGV+Fl+0iBxLdf6ezOf4aorIHBFZ7/7/OyyddXx6DrNNVQvtC2gERAIxQFQm6+0AKvhjfEAwsBWoCxQBVgGN8ym+l4DH3enHgRczWO9kPp6zLM8HcC/wtjs9AJjgZ/ENBN7I739vqY7fEbgMWJvB8p7ADECA1sASP4svGpjqw/NXFbjMnS4FbErnb+zTc5jdV6G+IlDVDaoa5+s4MuJhfK2ALaq6TVXPAl8D13g/OnCP84k7/QnQN5+OmxlPzkfquCcCXURE/Cg+n1LVecCRTFa5BvhUHYuBcBGpmj/ReRSfT6nqXlVd7k6fADYAaQcL8Ok5zK5CnQiyQYGfRWSZiAzydTBpVAd2pXq/mwv/0XlLZVXd607vAypnsF4xEYkVkcUi0tfLMXlyPv5eR1XPAceA8l6O64JjuzL6e/3DvWUwUURq5k9oHvPlvzlPtRGRVSIyQ0Qu9lUQ7m3H5sCSNIsKwjn8W4EfoUxEfgWqpLPo36r6g4e7aa+q8SJSCfhFRDa6v0r8JT6vySy+1G9UVUUko7bGtd3zVxeYLSJrVHVrXsdaiPwIfKWqZ0RkMM7VyxU+jqkgWY7zb+6kiPQEJgMN8jsIESkJfAc8qKrH8/v4eanAJwJVvTIP9hHv/veAiHyPc3mfJ4kgD+KLB1L/YqzhzssTmcUnIvtFpKqq7nUvaw9ksI+U87dNRGJwfiF5KxF4cj5S1tktIiFAGeCwl+JJK8v4VDV1LO/j1GL8iVf/zeVW6i9dVZ0uIm+KSAVVzbfO3kQkFCcJfKGqk9JZxa/PYVoBf2tIREqISKmUaeAqIN3WCj6yFGggInVEpAhO8dPrLXNcU4Db3enbgQuuYESkrIgUdacrAO2A9V6MyZPzkTru/sBsdSt4+SDL+NLcK+6Dc4/Zn0wBbnNbvrQGjqW6RehzIlIlpeYjIq1wvsfyK9HjHvsDYIOqjs1gNb8+hxfwdbXamy/gWpx7c2eA/cBMd341YLo7XRenZccqYB3OLRu/iU//1wJhE86v7PyMrzwwC9gM/AqUc+dHAe+7022BNe75WwPclQ9xXXA+gGeAPu50MeBbYAvwO1A3n//dZRXfKPff2ipgDtAwn+P7CtgLJLr//u4ChgBD3OUCjHfjX0MmLe58FN/9qc7fYqBtPsfXHqeuuBpY6b56+tM5zO7LupgwxpgAF/C3howxJtBZIjDGmABnicAYYwKcJQJjjAlwlgiMMSbAWSIwXuX20tgtzbwHReStbOzjGRHJ9ME8EYmRdHpwdXv6fCMbxyrvxnwys+1EpIiIvOr2LrlZRH4QkRrusoiMes7MTyLSzH3yNuV9H8nH3mtNwWGJwHjbVzgPVaU2wJ2fJREJVtUnVfXXPI8sfaeBJ4DhWaz3PE7Pk5Gq2gCnm4NJ3urczn1COrua4bRvB0BVp6jqC3kWlCk0LBEYb5sI9HKfsk3ppKsaMF9E3nI7q1snIv9N2UCc8SFeFJHlwHUi8rGI9HeXPSkiS0VkrYi8m+aL91Zx+qdf6z5xeh4RqSgi37nbLxWRdmnXUdVTqroAJyGkS0SKA3cAD6lqkrvdRzgPBqb0GRQiIl+IyAa3Y7ni7rYviNOP/WoRGZNZXCLytIh8JiILgc/E6dTv4lRxxIhIlIi0EpHfRGSFiCwSkUj3fD8D3OCekxtSXx25Vy2z3ThmiUgtd/7H4vSjv0hEtqU671VFZF6q89sho/NjCh5LBMarVPUIztO9PdxZA4Bv1HmS8d+qGgVcCnQSkUtTbXpYVS9T1a/T7PINVW2pqpcAYUDvVMuKq2oznPEIPkwnnHHAK6raEvgHTj8/OVEf+EMv7GgsFkj5oo4E3lTVRsBx4F4RKY/zNPnFqnop8JwHcTUGrlTVG4EJwPXwdzcVVVU1FtgIdFDV5sCTwPPqdIH9JM5YDM1UdUKaWF8HPnHj+AJ4LdWyqjhPz/YGUq4gbsJ58r0Z0BTnaVpTSBT4TudMgZBye+gH9793ufOvF6fb7xCcL5/GOI/tg/Oll57OIvIoUBwoh9PVwI+pjoOqzhOR0iISnmbbK4HGqS4iSotISVU9mYvPlpFdqrrQnf4cGAq8inOl8YGITAWmZhaXOz1FVRPc6W+An4GncBLCRHd+GeATEWmA0/VBqAfxtQH6udOfcX7Hd5NVNRlYLyIpXY8vBT4Up7O1yaq60oNjmALCrghMfvgBZ3CYy3B+tS8TkTo49+G7uL9Kp+H0EZTiVNqdiEgx4E2gv6o2Ad5Ls03a/lLSvg8CWru/kJupavUcJoGtQC1xOytMpQVOYko3FnXGRmiF8wXeG/jJg7hOpdpBPHDYvXK6gf8ly2eBOe5V0tWcf05y4kyqaXGPPQ9n5LB44GMRuS2XxzB+xBKB8Tr3S20Ozu2alCJxaZwvuWPur84eGWyeWsoX3CH3F3P/NMtvABCR9ji9PR5Ls/xn4IGUNyLSLBsf42+qegpnDIGxIhLs7us2nKuU2e5qtUSkjTt9E7DAjbmMqk4HHsK5xZLduCYAj7r7Sbl6KsP/ujgemGrdEzgF7fQs4n9F/JuB+ZkcExGpDexX1fdwbl359xi8JlssEZj88hXOF1/K7ZtVwAqc+9tfAgsz3tShqkdxrgLWAjNxblekdlpEVgBv87/bT6kNBaLcAul6nN4iLyAiO4CxwEAR2S0ijdNZbSTObZ5NIrIZuA64Vv/Xi2MccJ+IbADKAm/hfClPFZHVwALg4ezE5ZqIW2dJNe8lYJT72VPf7p2Dc8tppYjckGY/DwB3uLHcClwwAHsa0cAq9xg34NQ1TCFhvY8aY0yAsysCY4wJcJYIjDEmwFkiMMaYAGeJwBhjApwlAmOMCXCWCIwxJsBZIjDGmAD3/6S2LxB+IoKDAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Weight: -0.8544847369194031\n",
      "Bias: 2.5609138011932373\n",
      "Residual Sum of Squares: 5.467461935727148\n"
     ]
    }
   ],
   "source": [
    "plt.title(\"Optimal Line for Dataset\")\n",
    "plt.xlabel(\"Variable 1 Observations\")\n",
    "plt.ylabel(\"Variable 2 Observations\")\n",
    "plt.scatter(x=[p[0] for p in dataset], y=[p[1] for p in dataset])\n",
    "plt.axline(\n",
    "    (dataset[0][0], p.ws.weight.item() * dataset[0][0] + p.ws.bias.item()),\n",
    "    (dataset[1][0], p.ws.weight.item() * dataset[1][0] + p.ws.bias.item())\n",
    ")\n",
    "plt.show()\n",
    "\n",
    "print(\"Weight:\", p.ws.weight.item())\n",
    "print(\"Bias:\", p.ws.bias.item())\n",
    "print(\"Residual Sum of Squares:\", sum([(y_i - p.ws.bias.item() - p.ws.weight.item() * x_i)**2 for x_i, y_i in dataset]))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6a60ec6c-f159-4272-bbff-c8d768ca5aa7",
   "metadata": {},
   "source": [
    "## Binary Classification"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "79ccb04e-75fa-4ea2-bc3c-c1be8413396b",
   "metadata": {},
   "source": [
    "Consider this problem: I have a collection of data points that can be graphed on a two-dimensional Cartesian coordinate plane. Each point has two feature values: an $x_1$ value and an $x_2$ value, and a classfication label. There are only two labels, meaning points fall into one class or the other. This type of problem I'm presenting is known as a **binary classification** problem. I also assume here that the data here is **lineary seperable**, meaning the two classfications of observed and unobserved data points can be divided cleanly and completely with a straight line. The goal is to find the equation for a line that does just this. A perceptron can be used to solve this problem."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e3e97366-3c80-4cf0-a64a-a94526adc9c5",
   "metadata": {},
   "source": [
    "Assume I have already collected a small sample of data points. The data is graphed below:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "59abd20c-a12c-4c52-bf74-303c268bd68f",
   "metadata": {},
   "outputs": [],
   "source": [
    "X1_LOWER_BOUND = -10\n",
    "X1_UPPER_BOUND = 10\n",
    "NUM_SAMPLES = 10\n",
    "TRUE_BOUNDARY_FUNCTION = lambda x1: 1.5 * x1 + 0.5 #same as 3x1 - 2x2 + 1 = 0\n",
    "X2_LOWER_BOUND = min(TRUE_BOUNDARY_FUNCTION(X1_LOWER_BOUND), TRUE_BOUNDARY_FUNCTION(X1_UPPER_BOUND))\n",
    "X2_UPPER_BOUND = max(TRUE_BOUNDARY_FUNCTION(X1_LOWER_BOUND), TRUE_BOUNDARY_FUNCTION(X1_UPPER_BOUND))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "id": "30911cab-41a1-41e9-801e-ef73543b0d85",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYQAAAD5CAYAAAAndkJ4AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/YYfK9AAAACXBIWXMAAAsTAAALEwEAmpwYAAAuw0lEQVR4nO3deVxUZfvH8c8tLpWampHtafVoDiioqA9lCmnaYos95ZOZSw5LKJm2aGaWSyuuhYp75pq45r7hgAu4ixtGmtvjWu4SKTLcvz8Y+aGBoMzMmRmu9+vFy1kO53y55zgX17mHc5TWGiGEEKKE0QGEEEK4BikIQgghACkIQgghbKQgCCGEAKQgCCGEsJGCIIQQAoCS9liJUmoC0BL4Q2vta3usLxAK/Glb7FOt9eIbradixYr68ccft0ckh/rrr78oW7as0TEKJDntJzU1FavVislkMjrKDbnDWILktIesrCyOHj3KH3/8AXBKa+1d5JVqrYv8BTQG6gK7cj3WF/joZtZTvXp17Q4sFovREQpFctpPkyZNtJ+fn9ExCuQOY6m15Cyq5cuX66pVq2pAd+nSRQObtR3ey+1yyEhrvRo4Y491Cc8TFRWFxWK55jGLxUJUVJRBiYRwT2fPnqVTp040b96cMmXKsGbNGoYPH2639Tt6DiFSKbVDKTVBKVXJwdsSLqp+/fq0bt06pyhYLBZat25N/fr1DU4mhPuYO3cuJpOJSZMm0atXL5KTk2nUqJFdt6G0nU5doZSqCizU/z+HUAU4BWhgAHCf1rpTHt8XBoQBeHt714uNjbVLHkdKS0ujXLlyRscokCvl3LZtG/369ePll19m/vz5fPHFF9SpUwdwrZz56datG1arlejoaKOj3JA7jCVIzptx5swZfvjhBxISEnj88cf5+OOPqV69+jXLBAcHb9FaBxR1W3aZVM6L1vrk1dtKqbHAwnyWGwOMAahRo4YOCgq65vkrV65w5MgRLl265KioN61ChQrcdtttRscokBE5b7vtNh588EFKlSp1zeNBQUGcPXuWAQMG0KdPH7p3757zXHx8PNe/7q6mYsWKnDt3zuVzusNYguQsDK01kyZNonv37qSnp/P111/z0Ucf/eP/lj05rCAope7TWh+33W0F7LqV9Rw5coTy5ctTtWpVlFL2C1gEFy9epHz58kbHKJCzc2qtOX36NEeOHKFatWrXPGexWIiJiaFPnz7ExMQQHBxMcHCw07IJ4U4OHTpEeHg4y5Yt48knn2T8+PE88cQTDt+uXeYQlFLTgSSghlLqiFLKDEQppXYqpXYAwUD3G64kH5cuXaJy5couUwxE/pRSVK5c+R/d3NU5g9jYWPr3709sbOw1cwpCiGxZWVkMHz4cHx8f1q5dS3R0NGvWrHFKMQA7dQha6zZ5PDzeHusGpBi4kbxeq02bNhEbG5vTEQQHBxMbG8umTZukSxDCJjU1FbPZzLp162jRogWjR4/mkUcecWoGhx0yEuKqHj16/OMxOWQkRLYrV64waNAg+vXrxx133MHEiRNp3769Ib8IS0EowOnTp2natCkAJ06cwMvLi8qVK1OiRAk2btxI6dKli7yNoKAgjh8/zu23387ly5fp3r07YWFhRV7vjXTs2JGWLVvy+uuvO3Q7Qoj8bdu2DbPZzLZt23j99deJjo7m3nvvNSyPFIQCVK5cmeTkZAD69u1LuXLlCA8Pz5mszczMpGTJog/j1KlTCQgI4MyZMzz22GN07NjRLsXGXuz1cwohsudG+/fvT1RUFHfffTezZ8/mtddeMzqWnNzuVrz77ru8++67NGzYkB49etC3b18GDRqU87yvry8HDx4EYMqUKTRo0AB/f3/Cw8OxWq03XHdaWhply5bFy8sLgOnTp1OrVi18fX3p2bNnznK5Pxs9a9YsOnbsCGT/5t+1a1eefPJJateuzaxZs4DsTwBFRkZSo0YNmjVrdvX8JwD079+f+vXr4+vrS1hY2NVTjxAUFES3bt0ICAjgq6++olq1aly5cgWACxcuXHNfCFE4a9euxc/Pj2+++Yb27duzZ88elygG4GYdQrdu3XJ+W7cXf39/hg0bdtPfd+TIERITE/Hy8qJv3755LrNnzx5mzJjBunXrKFWqFJ07d2bq1Km0b9/+H8u2bduWMmXKsHfvXoYNG4aXlxfHjh2jZ8+ebNmyhUqVKtG8eXPmzZvHq6++esNsx48fZ+3atWzZsoU2bdrw+uuvM3fuXFJTU0lJSeHkyZOYTCY6dcr+O8HIyEg+//xzANq1a8fChQt56aWXAMjIyGDz5s0AHDx4kEWLFvHqq6/y888/89prrzn0M9FCQPapT+rXr3/NnJPFYmHTpk15zk+5qosXL9KrVy9GjBhB1apVWbZsGc2bNzc61jWkQ7hFb7zxRs5v8fmJi4tjy5Yt1K9fH39/f+Li4ti/f3+ey06dOpUdO3Zw+PBhBg0axKFDh9i0aRNBQUF4e3tTsmRJ2rZty+rVqwvM9uqrr1KiRAmeeOIJTp7M/vvA1atX06ZNG7y8vLj//vt55plncpa3WCw0bNiQWrVqsWrVKnbv3p3z3H//+9+c2yEhIfz4448A/Pjjj7zzzjsFZhGiqDzh1CfLli3D19eXkSNH0rVrV3bu3OlyxQDcrEO4ld/kHSX3KXFLlixJVlZWzv2rn8PXWtOhQwe++eabQq/X29ubunXrsmHDBsqUKZPvcrk/gXD95/5zf19Bpya5dOkSnTt3ZvPmzTz00EP07dv3mvXl/jmfeuopDh48SHx8PFarFV9f30L/XELcqqsfU27dujURERHExMRc8zFmV3bmzBm6d+/OpEmTeOKJJ1i7di1PPvmk0bHyJR2CHVStWpWtW7cCsHXrVg4cOABA06ZNmTVrVs7x+jNnznDo0KEbris9PZ1t27bx2GOP0aBBAxISEjh16hRWq5Xp06fTpEkTAKpUqcKePXvIyspi7ty5BWZs3LgxM2bMwGq1cvz48Zzftq6++d99992kpaXlzDnkp3379rz11lvSHQinCg4OJiIiggEDBhAREeEWxWDWrFnUrFmTadOm0bt3b7Zt2+bSxQDcrENwVf/5z3+YNGkSPj4+NGzYMOfEUyaTiS+//JLmzZuTlZVFqVKlGDFiRJ5/bNK2bducj5127NiRevXqAfDtt98SHByM1poXX3yRV155Jefxli1b4u3tTUBAAGlpaTfM2KpVK1atWoXJZOLhhx8mMDAQyD5HT2hoKL6+vtx7770FtuFt27bls88+o02bvP4WUQjHcKdTnxw/fpwuXbowd+5c6taty7Jly/D39zc6VuHY46IK9vrK6wI5KSkpN3npCMe7cOGC0REKxRE5Z86cqd9+++0bLnOzr5mrXoQkN7lAjn3dTM5Vq1bpu+++W69atSrP+450MzmzsrL0hAkTdMWKFXWZMmX0t99+q69cueK4cLlgpwvkSIcgCu29995jyZIlLF58wyuhCmFX7nDqkwMHDhAWFsbKlSt5+umnGTdu3D9OUe0OpCCIQnP1awEIz+TKpz6xWq2MGDGCXr16UaJECUaOHEl4eDglSrjn9KxbFASttZzgzk1oO11wSQhXt2fPHsxmM0lJSTz//POMGjWKhx9+2OhYReLyZey2227j9OnT8kbjBrTtegjucPEgIW7VlStX+Oqrr/D39yc1NZXJkyezaNEity8G4AYdwoMPPsiRI0f4888/jY6S49KlS27xpmdEzqtXTBPCE23ZsoVOnTqxY8cOWrduTXR0NPfcc4/RsezG5QtCqVKl/nH1LaPFx8fnXA/YlblLTiFc3d9//51zzrIqVaowd+7cAk8h445cviAIIYSRVq9eTUhICHv37iUkJISBAwdSsWJFo2M5hBQEIYTIw4ULFxg6dCjz58+nWrVqrFy5MufaKJ7K5SeVhRDC2RYvXoyvry8LFiyge/fu7Ny50+OLAUhBEEKIHKdOnaJdu3a8+OKLlC9fnuHDhzNkyJBrTvLoyaQgCCGKPa01M2bMwGQy8fPPP/P555+zdetWTCaT0dGcSuYQhBDF2rFjx4iIiGD+/PkEBASwcuVKateubXQsQ0iHIIQolrTWjBs3DpPJxPLlyxk0aBBJSUnFthiAdAhCiGJo//79hIaGsmrVKpo0acK4ceN4/PHHjY5lOOkQhBDFhtVqZejQofj6+rJp0yZGjx7NqlWrpBjYSIcghCgWdu/ejdlsZsOGDbz44ouMGjVKTrNyHekQhBAeLSMjg/79+1OnTh1+//13pk2bxoIFC6QY5MEuBUEpNUEp9YdSaleux+5SSq1QSu21/VvJHtsSQojC2rRpE/Xq1eOLL77gjTfeICUlhTZt2sjp9PNhrw5hIvDcdY99AsRprf8FxNnuCyGEw6Wnp/PRRx/x73//m7NnzzJ//nymTp2Kt7e30dFcml0KgtZ6NXDmuodfAX6y3f4JeNUe2xJCiBuJj4+ndu3aDB48mNDQUHbv3s1LL71kdCy34Mg5hCpa6+O22yeAKg7clhCimDt//jzh4eE5l9ZctWoVo0aNokKFCgYncx/KXlciU0pVBRZqrX1t989prSvmev6s1vof8whKqTAgDMDb27tebGysXfI4UlpaGuXKlTM6RoEkp/1069YNq9Xq8teVdoexBPvnTExMZOjQoZw5c4Y33niDjh072uXiUO4ynsHBwVu01gFFXpHW2i5fQFVgV677qcB9ttv3AakFraN69eraHVgsFqMjFIrktJ8mTZpoPz8/o2MUyB3GUmv75fzjjz90mzZtNKB9fX31hg0b7LLeq9xlPIHN2g7v4448ZDQf6GC73QH4xYHbEkIUI1prpk2bRs2aNZk1axb9+vVjy5YtNGjQwOhobs0uf5imlJoOBAF3K6WOAF8A3wKxSikzcAhobY9tCSGKtyNHjhAREcHChQtp2LAh48ePx8fHx+hYHsEuBUFr3Safpzz/ihJCCKfIyspi7NixfPzxx2RmZjJkyBC6du2Kl5eX0dE8hpy6Qgjh8vbt20doaCjx8fE888wzjB07lkcffdToWB5HTl0hhHBZmZmZDBo0iFq1arF161bGjh3LypUrpRg4iHQIQgiXtGPHDsxmM5s3b+bll19m5MiRPPDAA0bH8mjSIQghXMrly5f54osvqFevHocOHWLGjBnMmzdPioETSIcghHAZ69evx2w2k5KSwttvv82wYcOoXLmy0bGKDekQhBCG++uvv/jggw948sknuXDhAosWLWLy5MlSDJxMOgQhhKHi4uIIDQ3lwIEDRERE8O2333LnnXcaHatYkg5BCGGIc+fOERoaSrNmzShZsiQJCQmMHDlSioGBpCAIIZxu7dq1mEwmJkyYQI8ePdi+fTuNGzc2OlaxJ4eMhBBOc/LkSbp27UpsbCy1a9dm/vz5BAQU/SSdwj6kQxBCOJzWmilTpmAymZg3b17O3xdIMXAtUhCEEA51+PBhXnzxRdq1a0eNGjVITk7m7bffplSpUkZHE9eRgiCEcIisrCxiYmLw8fEhISGB77//njVr1lCzZk2jo4l8yByCEMLufvvtN0JCQlizZg3NmjVjzJgxVKtWzehYogDSIQgh7CYzM5PvvvuO2rVrs3PnTiZMmMDy5culGLgJ6RCEEHaxfft2OnXqxNatW2nVqhUjRozgvvvuMzqWuAnSIQghiuTSpUt89tlnBAQEcPToUWbNmsWcOXOkGLgh6RCEELcsMTERs9nMr7/+SocOHRgyZAh33XWX0bHELZIOQQhx09LS0nj//fdp1KgR6enpLF26lIkTJ0oxcHNSEIQQN2X58uX4+vryww8/0KVLF3bt2kWLFi2MjiXsQAqCEKJQzp49yzvvvEOLFi247bbbWLNmDdHR0ZQvX97oaMJOpCAIIQo0Z84cTCYTkydPplevXiQnJ9OoUSOjYwk7k0llIUS+Tpw4QWRkJLNnz8bf35/FixdTp04do2MJB5EOQQjxD1prfvrpJ0wmEwsXLuTrr79m48aNUgw8nHQIQohrHDp0iPDwcJYtW8ZTTz3FuHHjeOKJJ4yOJZxAOgQhBJB9Mrrhw4fj4+PD2rVriY6OZvXq1VIMihHpEIQQ/Prrr4SEhLBu3TpatGjB6NGjeeSRR4yOJZxMOgQhirErV67w9ddf4+fnR0pKCj/99BNLliyRYlBMObxDUEodBC4CViBTay2XSBLCBWzbto1OnTqRnJzM66+/zvDhw6lSpYrRsYSBnNUhBGut/aUYCGG8S5cu0atXL+rXr8+JEyeYPXs2M2fOlGIgZA5BiOJk7dq1mM1mfvvtN9555x0GDx5MpUqVjI4lXITSWjt2A0odAM4CGhittR5z3fNhQBiAt7d3vdjYWIfmsYe0tDTKlStndIwCSU776datG1arlejoaKOj3FB+Y5mens7YsWOZN28e9957Lx9++KGhF7h3h9cc3CdncHDwFrscgdFaO/QLeMD27z3AdqBxfstWr15duwOLxWJ0hEKRnPbTpEkT7efnZ3SMAuU1lkuWLNEPP/ywVkrp999/X1+8eNH5wa7jDq+51u6TE9is7fB+7fA5BK31Udu/fwBzgQaO3qYQAk6fPk2HDh14/vnnKVu2LOvWrWPYsGFu8RuvMIZDC4JSqqxSqvzV20BzYJcjtylEcae1ZtasWZhMJqZNm8Znn33Gtm3bCAwMNDqacHGOnlSuAsxVSl3d1jSt9VIHb1OIYuv06dP85z//Ye7cudSrV4/ly5fj5+dndCzhJhxaELTW+wHZG4VwMK01P/74I++//z6ZmZl89913fPDBB5QsKR8kFIUne4sQbu7AgQOEhYWxcuVKateuzcyZM6levbrRsYQbkoIghJuyWq0MHz6cTz/9FC8vL0aOHEmNGjWkGIhbJucyEsINpaSk8PTTT9OtWzeaNGnC7t27iYiIoEQJ+S8tbp3sPUK4kStXrvDll19Sp04dfvvtN6ZMmcKiRYt46KGHjI4mPIAUBCEMFBUVhcViueYxi8VCVFTUP5bdsmULAQEB9OnTh1atWpGSkkLbtm2xfYpPiCKTgiCEgerXr0/r1q1zioLFYqF169bUr18/Z5m///6bnj170qBBA/7880/mzZvHzz//zD333GNUbOGhZFJZCAMFBwcTGxtL69atiYiIICYmhtjYWIKDgwFISEggJCSEffv2ERISwsCBA6lYsaKxoYXHkg5BCIMFBwcTERHBgAEDiIiIIDg4mAsXLhAREUFQUBBZWVmsXLmSsWPHSjEQDiUdghAGs1gsxMTE0KdPH2JiYrj99tsZOXIkx44d44MPPqB///6ULVvW6JiiGJCCIISBrs4ZxMbGUqtWLRITE/n000955JFHSExMpGHDhkZHFMWIFAQhDLRp0yZmzJjBH3/8gclk4uzZs7Rv354aNWpIMRBOJwVBCAO1bduWzp07M3/+fAICAoiLi6NWrVpGxxLFlEwqC2EArTVjx47FZDKxfPlyBg0aRFJSkhQDYSjpEIRwst9//53Q0FAsFgtBQUGMHTuWxx9/3OhYQkiHIISzWK1WhgwZQq1atdiyZQujR48mLi5OioFwGdIhCOEEu3btwmw2s3HjRlq2bElMTAwPPvig0bGEuIZ0CEI4UEZGBv369aNu3brs37+fadOmMX/+fCkGwiVJhyCEg2zcuBGz2cyuXbt46623GDZsGN7e3kbHEiJf0iEIYWfp6el89NFHBAYGcvbsWRYsWMDUqVOlGAiXJx2CEHZksVgICQlh//79hIeH891331GhQgWjY91QUlIS8fHxBAUFERgYaHQcYSApCELYwfnz5+nRowdjxozhsccey/lIqatLSkqiadOmZGRkULp0aeLi4qQoFGNyyEiIIlqwYAEmk4lx48bx0UcfsWPHDrcoBgDx8fFkZGRgtVrJyMggPj7e6EjCQFIQhLhFf/75J2+99RYvv/wylStXZv369QwcOJA77rjD6GiFFhQUROnSpfHy8qJ06dJuU8iEY8ghIyFuktaa6dOn07VrVy5cuEC/fv345JNPKF26tNHRblpgYCBxcXEyhyAAKQhC3JT//e9/REREsGjRIho2bMj48ePx8fExOlaRBAYGSiEQgBQEIQpFa83p06fx8fHBarUydOhQ3nvvPby8vIyOJoTdSEEQogB79+5l+/btnD9/nqZNmzJmzBgeffRRo2MJYXcOn1RWSj2nlEpVSu1TSn3i6O0JYS+ZmZkMGjSI2rVrk5aWxoMPPsiKFSukGAiP5dAOQSnlBYwAngWOAJuUUvO11imO3K4QRbVjxw7MZjObN2/mlVde4eTJk/z9998opYyOJoTDOLpDaADs01rv11pnAD8Drzh4m0LcssuXL/P5559Tr149Dh06xIwZM5g7dy5lypQxOpoQDqe01o5buVKvA89prUNs99sBDbXWkXktf8cdd+gGDRo4LI+9nDt3jooVKxodo0CS8+ZcuHCB1NRU0tPTqVKlCo899hilSpUCIDk5mczMTAICAgxOeWOuMpYFkZz2lZCQsEVrXeSd0/BJZaVUGBAGUKpUKc6dO2dsoEKwWq2S046Mzmm1Wjlx4gSnTp2iVKlSVKtWjTvvvJO//vorZ5nMzEy01i4/nkaPZWFJTheltXbYFxAILMt1vxfQK7/lq1evrt2BxWIxOkKhSM6CrVy5UlerVk0DunPnzvr8+fN5LtekSRPt5+fn3HC3QF5z+3KXnMBmbYf3bEfPIWwC/qWUqqaUKg28Ccx38DaFKNC5c+cICQmhWbNmlCxZkoSEBEaMGMGdd95pdDQhDOPQgqC1zgQigWXAHiBWa73bkdsUoiDz5s3DZDIxceJEevbsyfbt22ncuLHRsYQwnMPnELTWi4HFjt6OEAU5efIk7733HjNnzsTPz48FCxZQr149o2MJN1Ecrhth+KSyEI6mtWbKlCl069aNtLQ0vvzyS3r06JHzCSIhClJcrhshp78WHu3w4cO8+OKLtG/fnho1apCcnEzv3r2lGIibUlyuGyEFQXikrKwsRo4ciY+PD6tXr+aHH35gzZo11KxZ0+howg0Vl+tGyCEj4XF+++03QkJCWLNmDc8++yxjxoyhatWqRscSbqy4XDdCCoLwGJmZmQwePJgvvviC22+/nR9//JEOHTrI+YeEXRSH60ZIQRAeITk5GbPZzNatW2nVqhUjRozgvvvuMzqWEG5F5hCEW7t06RK9e/cmICCAo0ePMmvWLObMmSPFQIhbIB2CcFuJiYmYzWZ+/fVXOnTowJAhQ7jrrruMjiWE25IOQbidtLQ0unbtSqNGjUhPT2fp0qVMnDhRioEQRSQFQbiV5cuX4+vry/Dhw+nSpQu7du2iRYsWRscSwiNIQRBu4cyZM7zzzju0aNGC2267jdWrVxMdHU358uWNjiaEx5CCIFze7NmzMZlMTJ48mV69epGcnEyjRo2MjiWEx5FJZeGyTpw4QWRkJLNnz6ZOnTosWbKEOnXqGB1LCI8lHYJwOVprJk6ciMlkYuHChXzzzTds2LBBioEQDiYdgnApBw8eJDw8nOXLl9OoUSPGjRtHjRo1jI4lRLEgHYJwCVlZWURHR+Pr60tiYiLDhw8nISFBioEQTiQdgjDc4cOHady4MevWraNFixaMHj2aRx55xOhYQhQ7UhCEYa5cucLAgQPp27cv5cqV46effqJdu3ZyMjohDCIFQRhi69atmM1mkpOTadKkCTNmzKBKlSpGxxKiWJOCIJzq77//pn///gwcOBBvb2/mzJlDpUqVpBgI4QJkUlk4zdq1a/H39+fbb7+lQ4cOpKSk0KpVK6NjCSFspCAIh7t48SKRkZE8/fTTZGRksGLFCsaPH0+lSpWMjiaEyEUKgnCoJUuW4OPjw8iRI3n//ffZuXMnzZo1MzqWECIPUhCEQ5w+fZr27dvzwgsvUK5cOdatW8ewYcMoV66c0dGEEPmQgiDsSmvNzJkzMZlMTJ8+nT59+rBt2zaPvxatEJ5APmUk7Ob48eN07tyZefPmUa9ePZYvX46fn5/RsYQQhSQdgigyrTUTJkygZs2aLF26lKioKNavXy/FQAg347CCoJTqq5Q6qpRKtn294KhtCeMcOHCA5s2bYzab8fPzY/v27Xz88ceULCnNpxDuxtEdwlCttb/ta7GDtyWcyGq18v333+Pr68uGDRuIiYnBYrFQvXp1o6MJIW6R/BonblpKSgpms5n169fzwgsvMGrUKB566CGjY9lFUlIS8fHxBAUFedxEuCf/bMI+HF0QIpVS7YHNwIda67MO3p5woIyMDL777ju+/PJLypcvz5QpU3jrrbc85mR0SUlJNG3alIyMDEqXLk1cXJzHvHF68s8m7KdIBUEptRK4N4+negMxwABA2/4dDHTKYx1hQBiAt7c38fHxRYnkFGlpacUuZ2pqKlFRUezfv59nnnmGyMhIKlWqREJCQpHX7SrjOXXqVC5fvkxWVhaXL19mwoQJXL58GYBz585htVpdIueN5DeWN/rZjOAqr3lB3CWn3WitHf4FVAV2FbRc9erVtTuwWCxGRygUe+RMT0/XH3/8sS5RooS+//779S+//FL0YNdxlfFMTEzUt99+u/by8tK33367TkxMzHmuSZMm2s/Pz7hwhZTfWN7oZzOCq7zmBXGXnMBmbYf3aocdMlJK3ae1Pm672wrY5ahtCcdISEggJCSEffv2ERoaSlRUFBUrVjQ6lsMEBgYSFxfnkcfZPflnE/bjyDmEKKWUP9mHjA4C4Q7clrCjCxcu0LNnT0aNGsWjjz5KXFwczzzzjNGxnCIwMNBj3yw9+WcT9uGwgqC1bueodQvHWbRoEe+++y7Hjh3jgw8+oH///pQtW9boWEIIJ5C/VBYAnDp1irfffpuWLVtSoUIFEhMTGTx4sBQDIYoRKQjFnNaan3/+mZo1axIbG8sXX3zB1q1badiwodHRhBBOJn+YVowdPXqUzp07M3/+fOrXr8/48eOpVauW0bGEEAaRDqEY0lozduxYTCYTK1asYNCgQSQlJUkxEKKYkw6hmPn9998JDQ3FYrEQFBTE2LFjefzxx42OJYRwAdIhFBNWq5UhQ4ZQq1YttmzZwpgxY4iLi5NiIITIIR1CMbBr1y7MZjMbN27kpZdeIiYmhgceeMDoWEIIFyMdgge7cuUK/fr1o27duuzfv5/p06fzyy+/SDEQQuRJOgQPtXHjRsLDwzlw4ABvvfUW33//PXfffbfRsYQQLkw6BA+Tnp7Ohx9+SGBgIBcvXmTBggVMnTpVioEQokDSIXgQi8VCSEgI+/fvJzw8nJYtW9KyZUujYwkh3IR0CB7g/PnzhIWF8cwzz1CiRAksFgujRo2iXLlyRkcTQrgRKQhubsGCBZhMJsaPH8/HH3/M9u3bCQoKMjqWEMINSUFwU3/++Sdt2rTh5ZdfpnLlymzYsIGoqCjuuOMOo6MJIdyUFAQ3o7Vm2rRp1KxZk9mzZ9O/f382b95MQECA0dGEEG5OJpXdyP/+9z8iIiJYtGgRDRs2ZPz48fj4+BgdSwjhIaRDcANZWVmMGjUKHx8fLBYLQ4cOZd26dVIMhBB2JR2Ci9u7dy+hoaEkJCTQtGlTxowZw6OPPmp0LCGEB5IOwUVlZmYycOBAateuTXJyMuPHj2fFihVSDIQQDiMdggvasWMHZrOZzZs388orrzBy5Ejuv/9+o2MJITycdAgu5PLly3z++efUq1ePw4cPExsby9y5c6UYCCGcQjoEF5GUlITZbGbPnj20a9eOoUOHUrlyZaNjCSGKEekQDPbXX3/RrVs3nnrqKdLS0li8eDGTJk2SYiCEcDrpEAy0cuVKQkNDOXjwIJ07d+abb77hzjvvNDqWEKKYkg7BAOfOncNsNvPss89SqlQpVq9ezYgRI6QYCCEMJQXByebNm4fJZOKnn37ik08+Yfv27Tz99NNGxxJCCDlk5CwnT57kvffeY+bMmfj5+bFgwQLq1atndCwhhMhRpA5BKfWGUmq3UipLKRVw3XO9lFL7lFKpSqkWRYvpvrTWTJ48GZPJxC+//MJXX33Fpk2bpBgIIVxOUTuEXcBrwOjcDyqlTMCbgA9wP7BSKVVda20t4vbcyuHDhwkPD2fp0qU8+eSTjBs3jpo1axodSwgh8lSkDkFrvUdrnZrHU68AP2utL2utDwD7gAZF2ZY7ycrKYsSIEfj4+LBmzRp++OEH1qxZI8VACOHSHDWH8ACwPtf9I7bHPF5qaiohISGsXbuWZ599ljFjxlC1alWjYwkhRIEKLAhKqZXAvXk81Vtr/UtRAyilwoAwAG9vb+Lj44u6SodLS0v7R06r1cqMGTOYOHEiZcqUoWfPnrRo0YKDBw9y8OBBl8npitwh57lz57BarS6f0x3GEiSny9JaF/kLiAcCct3vBfTKdX8ZEFjQeqpXr67dgcViueb+tm3bdN26dTWgX3vtNX38+HFjgl3n+pyuyh1yNmnSRPv5+Rkdo0DuMJZaS057AzZrO7yXO+rvEOYDbyqlyiilqgH/AjY6aFuGuXTpEr179yYgIICjR48ya9YsZs+ezb335tVQCeH+kpKS+Oabb0hKSjI6inCAIs0hKKVaAdGAN7BIKZWstW6htd6tlIoFUoBMoIv2sE8YrVu3DrPZTGpqKh07dmTw4MHcddddRscSwmGSkpJo2rQpGRkZlC5dmri4OAIDA42OJeyoqJ8ymqu1flBrXUZrXUVr3SLXc19prR/TWtfQWi8pelTXkJaWxg8//MDTTz/NpUuXWLZsGT/++KMUA+Hx4uPjycjIwGq1kpGRUbyOrRcTcuqKm7B8+XJ8fX2ZN28ekZGR7Nq1i+bNmxsdSwinCAoKonTp0nh5eVG6dGmCgoKMjiTsTE5dUQhnzpzhww8/ZOLEidSoUYPvv/+e9957z+hYQjhVYGAgcXFxxMfHExQUJIeLPJAUhALMnj2bLl26cOrUKT799FP69OnD+vXrC/5GITxQYGCgFAIPJgUhH8ePHycyMpI5c+ZQp04dli5dir+/v9GxhBDCYWQO4TpaayZOnIjJZGLRokV8++23bNiwQYqBEMLjSYeQy8GDBwkLC2PFihU0atSIcePGUaNGDaNjCSGEU0iHQPbJ6KKjo/H19SUpKYkRI0aQkJAgxUAIUawU+w5hz549hISEkJiYyHPPPceoUaN45JFHjI4lhBBOV2w7hCtXrvD111/j7+/Pr7/+yqRJk1i8eLEUAyFEsVUsO4StW7fSqVMntm/fzhtvvEF0dDRVqlQxOpYQQhiqWHUIf//9N5988gkNGjTg5MmTzJkzh9jYWCkGQgi7cPeT/xWbDmHNmjWEhITw22+/YTabGThwIJUqVTI6lhDCQ3jCyf88vkO4ePEiXbp0oXHjxmRkZLBixQrGjRsnxUAIYVeecPI/jy4IS5YswcfHh5iYGLp168auXbto1qyZ0bGEEB7IE07+55GHjE6fPk337t2ZPHkyNWvWZN26dW7XuokbS0pKkpOsCZfiCSf/86iCoLVm5syZREZGcvbsWfr06UPv3r0pU6aM0dGEHXnCsVrhmdz95H8ec8jo2LFjvPbaa/z3v//l4YcfZsuWLfTv31+KgQfyhGO1Qrgity8IWmvGjx+PyWRi6dKlREVFsX79emrXrm10NOEgnnCsVghX5NaHjPbv309YWBhxcXE0btyYcePG8a9//cvoWMLBPOFYrRCuyC0LgtVqJTo6mt69e+Pl5UVMTAxhYWGUKOH2DY8oJHc/ViuEK3K7gpCSkoLZbGb9+vW88MILjBo1ioceesjoWEII4fbc5lfqjIwMBgwYgL+/P3v37mXKlCksXLhQioEQQtiJW3QImzZtwmw2s3PnTt58802+//577rnnHqNjCSGER3HpDiE9PZ0ePXrw73//m9OnT/PLL78wffp0KQZCCOEALtshJCQkEBISwr59+wgNDWXgwIFUqFDB6FhCCOGxXK5DuHDhAhEREQQFBZGVlUVcXBxjxoyRYiCEEA7mUh3CX3/9hY+PD8eOHeODDz5gwIAB3HHHHUbHEkKIYsGlCsLRo0fx8fFh1qxZNGzY0Og4QghRrCittdEZciilLgKpRucohLuBU0aHKATJaV/ukNMdMoLktLcaWuvyRV2JS3UIQKrWOsDoEAVRSm2WnPYjOe3HHTKC5LQ3pdRme6zH5SaVhRBCGEMKghBCCMD1CsIYowMUkuS0L8lpP+6QESSnvdklp0tNKgshhDCOq3UIQgghDOL0gqCUekMptVsplaWUCrjuuV5KqX1KqVSlVIt8vr+aUmqDbbkZSqnSTsg8QymVbPs6qJRKzme5g0qpnbbl7DLrfzOUUn2VUkdzZX0hn+Wes43xPqXUJwbkHKiU+lUptUMpNVcpVTGf5Zw+ngWNjVKqjG1/2GfbD6s6I9d1GR5SSlmUUim2/0vv57FMkFLqfK594XNn57TluOFrqLL9YBvPHUqpugZkrJFrnJKVUheUUt2uW8aQ8VRKTVBK/aGU2pXrsbuUUiuUUntt/1bK53s72JbZq5TqUKgNaq2d+gXUBGoA8UBArsdNwHagDFAN+B3wyuP7Y4E3bbdHARFOzj8Y+Dyf5w4Cdzt7THNtvy/wUQHLeNnG9lGgtG3MTU7O2Rwoabv9HfCdK4xnYcYG6AyMst1+E5hhwOt8H1DXdrs88FseOYOAhc7OdrOvIfACsARQwL+BDQbn9QJOAI+4wngCjYG6wK5cj0UBn9huf5LX/x/gLmC/7d9KttuVCtqe0zsErfUerXVef3z2CvCz1vqy1voAsA9okHsBpZQCngFm2R76CXjVgXGvYdt+a2C6s7bpAA2AfVrr/VrrDOBnssfeabTWy7XWmba764EHnbn9GyjM2LxC9n4H2fthU9t+4TRa6+Na66222xeBPcADzsxgR68Ak3S29UBFpdR9BuZpCvyutT5kYIYcWuvVwJnrHs69D+b3HtgCWKG1PqO1PgusAJ4raHuuNIfwAPC/XPeP8M+dvDJwLtebSV7LONLTwEmt9d58ntfAcqXUFqVUmBNz5RZpa70n5NNKFmacnakT2b8h5sXZ41mYsclZxrYfnid7vzSE7ZBVHWBDHk8HKqW2K6WWKKV8nJssR0Gvoavtj2+S/y98rjCeAFW01sdtt08AVfJY5pbG1SF/qayUWgncm8dTvbXWvzhim0VVyMxtuHF30EhrfVQpdQ+wQin1q63COyUnEAMMIPs/4QCyD291suf2C6sw46mU6g1kAlPzWY3Dx9OdKaXKAbOBblrrC9c9vZXswx5ptrmkecC/nBwR3Og1tM1Hvgz0yuNpVxnPa2ittVLKbh8VdUhB0Fo3u4VvOwrkvh7mg7bHcjtNdktZ0vbbWV7L3JKCMiulSgKvAfVusI6jtn//UErNJfsQhF13/sKOrVJqLLAwj6cKM85FVojx7Ai0BJpq20HPPNbh8PG8TmHG5uoyR2z7RAWy90unUkqVIrsYTNVaz7n++dwFQmu9WCk1Uil1t9baqeflKcRr6JT9sZCeB7ZqrU9e/4SrjKfNSaXUfVrr47bDa3/kscxRsuc9rnqQ7HnbG3KlQ0bzgTdtn+KoRnb13Zh7AdsbhwV43fZQB8BZHUcz4Fet9ZG8nlRKlVVKlb96m+yJ0115Leso1x17bZXP9jcB/1LZn9YqTXaLPN8Z+a5SSj0H9ABe1lqn57OMEeNZmLGZT/Z+B9n74ar8Cpqj2OYsxgN7tNZD8lnm3qtzG0qpBmT/X3dq4SrkazgfaG/7tNG/gfO5Doc4W75HAFxhPHPJvQ/m9x64DGiulKpkO3Tc3PbYjRkwa96K7ONZl4GTwLJcz/Um+1MeqcDzuR5fDNxvu/0o2YViHzATKOOk3BOBd6977H5gca5c221fu8k+NOLssZ0M7AR22Haa+67Pabv/AtmfTPndoJz7yD6+mWz7GnV9TqPGM6+xAfqTXbwAbrPtd/ts++GjBoxfI7IPC+7INYYvAO9e3UeBSNu4bSd74v5JA3Lm+Rpel1MBI2zjvZNcnzx0ctayZL/BV8j1mOHjSXaBOg5csb1vmsmes4oD9gIrgbtsywYA43J9byfbfroPeKcw25O/VBZCCAG41iEjIYQQBpKCIIQQApCCIIQQwkYKghBCCEAKghBCCBspCEIIIQApCEIIIWykIAghhADg/wBEXFEPwFKR8gAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "#generate samples\n",
    "samples = [\n",
    "    (random.uniform(X1_LOWER_BOUND, X1_UPPER_BOUND), random.uniform(X2_LOWER_BOUND, X2_UPPER_BOUND))\n",
    "    for i in range(0, NUM_SAMPLES)\n",
    "]\n",
    "true_classifications = [1 if samples[i][1] > TRUE_BOUNDARY_FUNCTION(samples[i][0]) else 0 for i in range(0, NUM_SAMPLES)]\n",
    "\n",
    "#graph samples\n",
    "plt.grid()\n",
    "plt.axhline(0, color=\"black\")\n",
    "plt.axvline(0, color=\"black\")\n",
    "plt.xlim([X1_LOWER_BOUND, X1_UPPER_BOUND])\n",
    "plt.ylim([X2_LOWER_BOUND, X2_UPPER_BOUND])\n",
    "plt.axline(\n",
    "    (X1_LOWER_BOUND, TRUE_BOUNDARY_FUNCTION(X1_LOWER_BOUND)),\n",
    "    (X1_UPPER_BOUND, TRUE_BOUNDARY_FUNCTION(X1_UPPER_BOUND)),\n",
    "    color='black',\n",
    "    label='True Boundary'\n",
    ")\n",
    "for i in range(0, NUM_SAMPLES):\n",
    "    plt.plot(samples[i][0], samples[i][1], color=\"black\", marker=\".\" if true_classifications[i] == 0 else \"x\")\n",
    "plt.legend()\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1a68975a-1a86-4504-aaf3-c5bf0dff7d33",
   "metadata": {},
   "source": [
    "The black line represents the \"true\" boundary - the (typically unknown) line/hyperplane that divides the space into two distinct regions/classes. As I can see from the graph above, points on one side of the line are marked with x's and points on the other side of the line are marked with dots. This boundary is typically written in the form of $\\beta_0 + \\beta_1x_1 + \\beta_2x_2 = 0$ where $\\beta_i$ values are the true values of the boundary line and $x_i$ values are the inputs for the feature values. In the code above, I rearranged the equation to express the value of $x_2$ in terms of $x_1$. I also technically defined the boundary such that points that satisfy $\\beta_0 + \\beta_1x_1 + \\beta_2x_2 > 0$ are x's and $\\beta_0 + \\beta_1x_1 + \\beta_2x_2 \\leq 0$ are dots in the code above."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "cea37166-e687-4f02-b157-f2f75c943bc5",
   "metadata": {},
   "source": [
    "Since I don't actually know what the equation for the true boundary line is, I need to train a perceptron to figure that out for me. In linear least squares, I needed to find the perceptron's weights and the bias values that minimize the squared distances between my input data points and my regression line. This would give me the best fit line. Again in this problem, I need to get the equation of a line/hyperplane, but unlike linear regression, it's not about finding the best fit line that best fits the data. It's about finding a line that cuts the space into two sections."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e27c7e57-69a4-4d96-8a84-cee65a989b57",
   "metadata": {},
   "source": [
    "As before, the main things needed to consider when training a perceptron are its weight values, bias value, activation function, and a loss function to measure the perceptron's performance. The activation function and loss function ultimately determine what weight and bias values I end up with, so let me formulate what those should look like."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "95ef6971-45d3-4c26-a086-ebffa463e0a1",
   "metadata": {},
   "source": [
    "The loss function for linear regression was pretty straightforward. By attempting to minimize the mean squared distances between my points and the line, I could train the perceptron to find the parameters for the line that produces the smallest loss value. For a binary classification problem, my loss function will look a bit different."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3e99260a-d3a6-4b45-8a96-789b71baeaef",
   "metadata": {},
   "source": [
    "In binary classification, I only have two output $y$ values (unlike $y$ values in regression, which can take on any number in $\\mathbb R$). A point marked with an x is represented with a value of $y=1$ and point marked with a dot is represented with a value of $y=0$. The perceptron must also only have two outputs, either $1$ or $0$."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "496003a8-45a7-4e76-b1cc-0ef7614f17f1",
   "metadata": {},
   "source": [
    "When calculating the difference between a true and a predicted classification, there are only four possible difference combinations. They are:"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "75f4d599-7add-4763-8083-e6a5981e643d",
   "metadata": {},
   "source": [
    "$$\n",
    "0 - 0 = 0 \\\\\n",
    "1 - 1 = 0 \\\\\n",
    "0 - 1 = -1 \\\\\n",
    "1 - 0 = 1\n",
    "$$"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a06bf6e8-ca1b-4e28-9020-c94ab65fcc52",
   "metadata": {},
   "source": [
    "Just as in linear least squares, the loss function to I need to minimize is the sum of squared differences between the true and the predicted value for each point:"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2b9e60d9-a616-489f-86bf-3dd4ea6b5205",
   "metadata": {},
   "source": [
    "$$\n",
    "MSE = \\frac{1}{n} \\sum\\limits_{i = 1}^{n} (y_i - \\hat y_i)^2\n",
    "$$"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a329b0ed-365e-451f-917f-a899dfdf43d4",
   "metadata": {},
   "source": [
    "I can't just set the loss as something like $y - \\hat y$ because the sum of misclassfied values $0-1=-1$ and $1-0=1$ would just cancel each other out. I could take the absolute value of each difference but the absolute value function does not have a nice derivative. Squaring each term in this case actually produces the exact same loss if I was using absolute value except squares have nicer derivatives. Taking the mean of the sum of squared differences is not really necessary, but it allows for more generalization since it scales the sum of squares by the number of data points. Also, it only scales the total loss by some constant value and does not have any effect on the values needed to minimize the loss function."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7b23c73f-8266-4963-a03e-0bfc0a159521",
   "metadata": {},
   "source": [
    "Reminder that inside the perceptron, I have weights and a bias that represent a boundary line/hyperplane:"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "59edf394-f02e-4188-897a-a707b23d7f0a",
   "metadata": {},
   "source": [
    "$$\n",
    "b + w_1 x_1 + w_2 x_2 + \\ldots + w_d x_d = 0\n",
    "$$"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1676ddf1-ab5f-429a-804d-0bf002d3a669",
   "metadata": {},
   "source": [
    "When I plug in some input $x$ from my dataset and compute the weighted sum, I will get an output value that is either on one side of the line (positive), the other side of the line (negative), or on the line (0). Earlier, I stated that I defined the true boundary line such that points that satisfy $\\beta_0 + \\beta_1x_1 + \\beta_2x_2 > 0$ are x's and $\\beta_0 + \\beta_1x_1 + \\beta_2x_2 \\leq 0$ are dots in the code above and so similarly, points that satisfy $b + w_1 x_1 + w_2 x_2 > 0$ are x's (class $1$) and $b + w_1 x_1 + w_2 x_2 \\leq 0$ are dots (class $0$)."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5333b9a0-b480-42bb-a804-e605e85b640a",
   "metadata": {},
   "source": [
    "So I need to transform the output of the weighted sum so that it always maps to either $0$ or $1$. I can set my activation function as one variant of the **Heaviside step function**:"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ec39c95b-9cf9-44b9-b862-80d75d23878a",
   "metadata": {},
   "source": [
    "$$\n",
    "H(x) =\n",
    "\\begin{cases}\n",
    "1 & \\text{if } x > 0 \\\\\n",
    "0 & \\text{if } x \\leq 0\n",
    "\\end{cases}\n",
    "$$"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9c5026cc-7a09-4e59-89e8-aef1ca9efa4e",
   "metadata": {},
   "source": [
    "So now I can write down my loss function that I need to minimize as:"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0f324b0a-cbd9-4074-8f3b-5e1438493514",
   "metadata": {},
   "source": [
    "$$\n",
    "MSE = \\frac{1}{n} \\sum\\limits_{i = 1}^{n} (y_i - H(b + w_1 x_{i,1} + w_2 x_{i,2}))^2\n",
    "$$"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "dbb69348-bafd-464d-be9d-4aa5c9e93190",
   "metadata": {},
   "source": [
    "The problem here though is that the Heaviside step function is a piecewise function that is not continous - it is not differentiable at $x=0$. Everything I've done up to this point requires taking first-order derivatives in order to find to find the values that minimize the loss function. So unfortunately, it would be too difficult for me to optimize this function that contains the Heaviside step function. I need a differentiable function that is similar to the step function."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ad535d15-4541-4132-9de9-69393c0b40e9",
   "metadata": {},
   "source": [
    "This differentiable function I need must satisfy some conditions:\n",
    "1. The $y$ domain must be defined everywhere for all values in the $x$ domain\n",
    "2. There must be no points of discontinuity and no vertical asymptotes ($x$ equals some constant)\n",
    "3. There must be two horizontal asymptotes: one at $y=0$ when $x\\to-\\infty$ and one at $y=1$ when $x\\to\\infty$\n",
    "4. The function must \"switch\" from \"off\" ($0$) to \"on\" ($1$) at around $x=0$"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "55a8a4af-a400-4fc6-bba0-a74c1ef95cf6",
   "metadata": {},
   "source": [
    "Let me graph some basic functions and see how I can maniuplate them to fit my needs:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "id": "58121090-e1cb-468d-be5c-28ce94b8e363",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAX4AAAD8CAYAAABw1c+bAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/YYfK9AAAACXBIWXMAAAsTAAALEwEAmpwYAABRs0lEQVR4nO3dd3hUVfrA8e9J772ThAQCoUOAUJUi9o6uiGsFERXd1d11betPd3WL6+rquuq6rAI2RFFZG+oKGEV6D5BQQkvvmfSZTDm/P2YSAyRkkkxNzud58kw7c+47dybv3Dn33PcKKSWKoihK/+Hh7AAURVEUx1KJX1EUpZ9RiV9RFKWfUYlfURSln1GJX1EUpZ9RiV9RFKWfsTrxCyGWCSHKhRAH2t0XIYT4Vghx1HIZbp8wFUVRFFvpzhb/CuDSM+57FFgvpRwCrLfcVhRFUVyY6M4BXEKIFOALKeUoy+3DwCwpZYkQIh7IklKm2yVSRVEUxSa8evn8WCllieV6KRDbWUMhxGJgMYCfn9+E5OTkXi7a/kwmEx4err8bxBlxehka8W8uoSkgEaOnn1XPcfX1WVBQgJQSV/xsajXg6Q3egebb1q5LX10VPi0a6oMHAcKuMXbE1d/zVq4eZ7VWUtciaSnNq5RSRve6Qyml1X9ACnCg3W3NGY/XWNPP0KFDpTv47rvvnB2CVZwSZ02+lE+FSLn9P1Y/xdXX58yZM+XYsWOdHcZZDC1G+crd6+X2L4633Wf1unznOilfm2afwKzg6u95K1eNs0lnkIvf3iEHPvKFfOrTAxLYKbuRszv76+1XXJlliAfLZXkv+1PcRWgi+IVC6YGu2yq9Ul+jBSA4wrpfVqcpPQCxo2wckeIIFfU65v9nK//LKePJK0fw+6tH2qzv3ib+z4DbLddvBz7tZX+KuxDCnFDKDjo7kj6vodqc+IO6m/gbK6GhFOJU4nc3eeX1zH1tE4dL63j9lgksPC/Vpv13Zzrn+8AWIF0IUSiEuBN4FrhICHEUuNByW+kvYkeaE7/J5OxI+rT66h5u8ZfuN1+qLX63suVYFde9thmt3sgHi6dyycg4my/D6p27UsqbOnloji0C0ev1FBYWotVqbdGdTYSGhpKbm+vsMLrUmzj9/PxITEzE29u7+0+OHQX6RtCchIhBPVq+0rX6ah0ICAr37d4TW3+NxY22fVCKXXyyu5BHPs5mYGQgy+/IJCkiwC7L6e2sHpspLCwkODiYlJQUhHD87IOO1NfXExwc7OwwutTTOKWUVFVVUVhYSGpqD35Ktm5Jlh5Qid+O6qu1BIb44OnVzZHZsgMQFAeBUfYJTLEZKSUvr8/jxXVHmDooktdvmUBoQA82xqzkMvOXtFotkZGRLpP0+wMhBJGRkT3/lRUzHBBqnN/OGqq13R/fB/MXshrfd3ktBhMPrc7mxXVHuG78AN5aOMmuSR9caIsfUEnfCXq1zn0CIDLtp7FkxS7qq7RED+zmLzpDC1QcgjSbjMQqdlLbrOeed3ax5XgVD144hAfmDHFIHnSpxK+4ofgxULDd2VH0WdIkaajRMWhcN4/ZqTwCJr0a33dhBdVNLFyxg5NVjbxww1iun5DosGW7zFCP4qbix0JtATRVOzuSPqmpvgWjwURwZDeHesosx1eoGT0uKbtQw9zXNlNap+WthZMcmvRBJX6lt+LHmi9L9jk3jj6qoVoH9GAOf/Fe8A6AqCG2D0rplf8dLOXGf2/Fz9uDNUumMW2w43e+q8Rv8eSTT/LSSy+13f7d737Ha6+91qs+r7nmGt5++20A/v3vf3PzzTf3qj+XFDfGfKkSv138NIe/m1M5S/aZh3k8PO0QldJTy348wd3v7mJobBBrlkwnLcY5swZdcoz/D58fJKe4zqZ9jkgI4amrOj/keeHChVx33XU8+OCDmEwmVq1axfr1689qd/7551NfX3/W/c8//zwXXnjhafctXbqU6dOnk5qaygsvvMDWrVt7/0JcTUAEhCarxG8ndZXNAIRE+lv/JJMJSrNh3M/tFJXSXUaT5Jkvclix+SQXj4jlH/Mz8Pdx3peySyZ+Z0hJSSEyMpI9e/ZQVlZGRkYGkZGRZ7XbuHGj1X3Gxsby9NNPM3v2bNasWUNERIQtQ3Yd8WPMiUaxubrKZvyCvPHx78a/avVxaGn4aRhOcaqmFgMPrNrLtzllLJyeyu+uGI6nh3NnMLpk4j/Xlrk9LVq0iBUrVlBaWsrChQs7bNOdLX6A/fv3ExkZSXFxsc3jdRnx4+DQF6CtA78QZ0fTp9RWNBMS1Y2tfYCSveZLlfidrrxey6K3dnKgqJY/XD2S26elODskwEUTv7PMnTuXJ598Er1ez8qVK2lqajqrTXe2+Ldv385XX33Fnj17mDlzJhdffHHPjpB1dfGWcf6yAzBwmnNj6WPqKpuJTenml2nJPvD0gehh9glKscqRsnoWLN9BdWMLS2+dyIUjOj1dicOpnbvt+Pj4MHv2bObNm4enZ+/G33Q6HXfddRfLli0jISGBF154gYULF7aet6BvUTN77MJkNFFfrSMkurtb/PvMBfQ87Xv0p9K5zXmVXP+vzbQYTXx491SXSvqgtvhPYzKZ2Lp1K6tXr+51X76+vuzb91MivPrqq7n66qt73a9LCo6DoFgoUeP8tlRfrUOaZPeGeqQ0J/6R19otLuXcPtpVyKMfZzMoOpBld2SSGG6fQmu9obb4LXJyckhLS2POnDkMGaLmPndb/Fi1xW9jrTN6QruT+DX55vM0qvF9h5NS8vdvj/DQ6n1MHhTB6numuWTSB7XF32bEiBEcP37c2WG4r7gxkLce9M3g3c2hCaVDbVM5uzPU0/rlqxK/Q+kMRh77eD+f7CnihgmJ/GnuaHy6W03VgVTiV2wjfixII5TnwIAJzo6mT6irbMbDSxAY1o2Dt0r2gfCEGOfMjOuPapv03P3uTrYer+Y3Fw3l/gvSXL7gpEr8im2038GrEr9N1FY0ExLpj0d35nyX7DWXy/buQRlnpdsKqpu4Y/l2CqqbeenGcVybMcDZIVlFJX7FNsKSwS9MjfPbUF2ltvs7dov3wtBL7BaT8pM9+TXc9fZO9EbJ23dOYsqgsw/4dFWuOwiluBchzPP51cwem6mrbCY0qhtb7vUl0FSpxvcd4OsDpcxfuhV/H08+WTLNrZI+qMTvVCtXrsTHx4dnnnnmtPu//fZbJkyYwOjRo5kwYQIbNmxwUoTdFD/WfDYuo97Zkbg9baMeXZNB7dh1MVJK3th4nHvf28Xw+BDWLJnO4OggZ4fVbTZJ/EKIXwkhDgohDggh3hdCqAHGLmzYsIHnnnuOnJwc1q1bx1tvvdX2WFRUFJ9//jn79+/nrbfe4tZbb3VipN2QkAFGnToVow20zejpzlBPyT5AqBr8dmI0SX7/2UH++GUul4yIY9XiKUQFdbNqqovodeIXQgwAfglMlFKOAjyB+b3t19EcWZZ5//79PPHEE3zzzTekpaWxdu1aVq5cyTfffANARkYGCQkJAIwcOZLm5mZ0Ol2vYnGIhPHmy+Ldzo2jD6it6EHiL95rrr/v635boK6uUWdg8ds7eWvLKe46P5XXbh6Pn7f7lry21c5dL8BfCKEHAoDeVST76lHbn8c1bjRc9mynDzuyLPPo0aPZvHlzW7vAwMC2pH+mjz/+mPHjx+Pr6wZbFuEp4B8BRbthYsdF7hTr1Jab60SFWjvUIyUU7VLn2LWD8jotC9/aQU5xHc9cM5Jbp6Y4O6Re63Xil1IWCSGeB/KBZuB/Usr/ndlOCLEYWAwQHR1NVlbWaY+Hhoa2JVRffQseRkNvQzuNSd+CroOE3SoyMpKwsDB+/PFHysvLGT16NGFhYWcl+bVr13bax5ltAwICeOyxx5g9ezYrV67E29u7wy+NzuTm5vLb3/6W//73v+d8ntFo7Fa/Z9JqtWe9Hz012j8F3yM/sLOD/hoaGmy2HHvQaDQYjUaXiLFwnwnvANi0peOigGeuS19tBVMbyznaGEyRC8TfytXf81adxVlYb+Lvu7Q06iW/zPAlSXeSrKyTDo/P1nqd+IUQ4cA1QCqgAVYLIW6RUr7bvp2UcimwFCA9PV3OmjXrtH5yc3MJDracjebqv/c2rA75dPH43XffzerVqyktLWXx4sV4enr+FJNFd8sy5+XlERkZSU1NzVl9nUthYSG33HIL7777LmPHnntnXX19fbf6PpOfnx8ZGRk9fv5pTJtg4/PMmpYJPoGnPZSVlcWZ77srCQsLQ6PRuESMq7fsIDjZi1mzOn5fzlqXOZ/CVhgyaz5DEl3nOApXf89bdRTnj0cr+eu7u/D38eHjuzIZNSDUOcHZgS2Gei4ETkgpKwCEEJ8A04B3z/ksF+QqZZk1Gg1XXHEFzz77LNOnT+/Wa3C6AeNBmszTOgdOdXY0bklKiaa8maGTulHRsWgXeHhDnNqxawsf7ijg8TX7SYsJYtkdmSSE9a0yJLaY1ZMPTBFCBAjzccpzgFwb9OtwrlKW+ZVXXiEvL4+nn36acePGMW7cOMrLy3sVj8O07uAt2uXcONxYc72elmYDYbHdKPBVtNu8H8vLDfYFuTApJc9/c5iHP85m6uBIVt8ztc8lfbDNGP82IcRHwG7AAOzBMqTjblylLPMTTzzBE0880esYnCI4FkIS1cyeXtCUNQIQbm3iNxnNM3rG3mi/oPoBncHIwx9l8+neYuZnJvHMtaPw9uybhzrZZFaPlPIp4Clb9OUsOTk5XHnllcydO1eVZe6tARnmLVClRzRl5qmcVm/xVx6FlnpVI6kXGlokt76xne0nq/ntJeksmTXY5Qut9Yaq1WOhyjLb0IAJkPs5NFVDQB89wbwd1ZQ14enlQVCElcdBtg6rqcTfI6eqGvnjtmaqtVr+MX8c14xzj0JrvdE3f8cozqUO5OoVTVkToTHdqMpZvBt8giFS/VLtrl2napj72mbqWyTvLprcL5I+qMSv2EPCOPNl0R6nhuGuNGVN1o/vg3mLP2EceKh/5+74an8JP//PVoL9vPi/Kf5MSu0/v07VJ0WxPb9Q89anmtnTbUajibqKZkKtTfwGHZQeUMM83SClZOkPx1iycjcjE0L45N5pxAX2r1SoxvgV+xgwAY5/Zy4l0Id3ktlafaUWk0lav8VfegBMepX4rWQwmvj95wd5d2s+V4yO54V5Y9265k5P9a+vORfTWVlmgD179nDnnXcCcOjQIaZOnYqvry/PP//8WW0ffPBBNm3axG9/+1uGDRvGmDFjmDt3LhqNBjAXhbvjjjvs+VLONmA8NJRBXe/KNvU3NWXmgwatntHTtmN3vJ0i6jsadQbuensn727N5+6Zg/jnTRn9MumDSvxOc66yzAB//vOf+eUvfwlAREQEL7/8Mg899FCHfe3YsYMpU6Zw0UUXceDAAbKzsxk6dCh/+ctfAHNRuMLCQvLz8+37otprO5Brp+OW2QfUlJrn8Hcr8QfFQkj/2CnZU6W1Wm54fQs/HK3kT3NH8dhlw7t3Sss+RiV+C1cqy1xfX092dnZbjZ6YmBgyMzPx9vY+axm5ubmkpaXh6enJxRdfjJeXefRuypQpFBYWtrW76qqrWLVqVa9eT7fEjwFPHyjc4bhl9gHVxY0EhvrgF3j2e92h4t3mYR41nNap3JI65r62iVNVjbxx+0RunjzQ2SE5nUuO8f91+185VH3Ipn0OixjGI5Me6fRxVyrLvHPnTkaNsq7myldffdVhcbhly5Zx440/Hck5ceJEnn32WR5++GGr+u01L1/zmaAKVOLvjuriRiIGWFlPv1kDlUdgzDy7xuTOvj9SwX3v7SbQ15MP75nKyIS+U2itN1wy8TtDSkoKkZGR7Nmzh7KyMjIyMoiMPPs8mt0p0hYbG8vTTz/N7NmzWbNmDRER1k0XKykpITo62qq233zzDf/85z9Pu+9Pf/oTXl5ebb8wwPyrobjYwePtSZNh+3/A0AJeXdVGVUwmSU1JIyNnWDlsU2gZRkucZL+g3Nj72/N54r8HGBITxPIFmcSH9r2aOz3lkon/XFvm9rRo0SJWrFhBaWkpCxd2fCKR7pZl3r9/P5GRkd1Kuv7+/mi12i7bNTU1odFoiI+Pb7tvxYoVfPHFF6xfv/60Q861Wi3+/g7+4CdmwpZXoDQbEic6dtluqK6yGYPeRERCYNeNAQq2gfBQM3rOYDJJnv/fYV7LOsaModG8+vMMgv2sHDrrJ1wy8TuLq5RlHj58OC+88EKX7b777jtmz57ddvvrr7/mueee4/vvvycg4PSdg0eOHLF6+MhmkiabLwu2q8Rvhepi847dyAQrh3oKt5vPr6tOtdhGqzfy0Op9fJFdwk2Tknn6mpF9ttBab6jE305rWeawsDCblWVevnz5aWWZN2zY0GXxp2HDhlFbW9t2gpXS0lImTpxIXV0dHh4evPTSS+Tk5PDVV1/xs5/9rO15999/Pzqdjosuuggw7+B9/fXXAfOXxBVXXNGr19RtIfEQmmTeMp26xLHLdkOtiT883ooZPdJoHuoZ63ant7ab6sYWFr+9k52nanj0smHcPWNQny601hsq8bfjKmWZwbyz+YMPPmDRokXExcWdNkOn1ebNm3nxxRfbhoXy8vI67Eun07Fz587TZi05TNIkyN/q+OW6oeriBoIj/fDx6/rfMrAxH1oafvpV1c+drGxkwYodFGmaeeXnGVw5JsHZIbk09RvIIicnh7S0NObMmeMSZZnvvffeLk+wvnv37g6neJ4pPz+fZ599tm2qp0MlToK6Iqg9+4tLOV1VcSORVo7vh9ZaZr0lqR27u05VM/e1TWiaWli5aLJK+lZQW/wWrlaW2c/Pj1tvvdUmfQ0ZMsR5X2atialgO9B/imB1l9FoQlPWRMros2eSdSSk7pD5wK2w/j0n/YvsYn794T4GhPmz/I5MUqKs3DHez6ktfsW+4kaDl78l8SudqS1rxmSURFi5Yze09pB51lQ/HcOWUvKvrGPcv3IPYwaE8sm901TS7waV+BX78vQ215EpVIn/XKqKGwCsm8rZUIG/trTfju8bjCYeX3OAv359iKvGJvDuosmEB6rjRLpDJX7F/hIzoSQbD6PO2ZG4rKrCBoSHIDzOihk9rV+i/TDxN+gM3PnWTt7fns+SWYP5x43j+m2htd5QiV+xv6TJYNITXH/M2ZG4rIqCBiLiA/GyJokVbMMkvMwlMfqRktpmbnh9Cz/mVfKX60bz8KXD+nWhtd6wSeIXQoQJIT4SQhwSQuQKIabaol939/vf/77DMspnal+CuTOvvPIKy5Yts1VojpWYCVh2SCodqiioJzrJygOxCrZTHzwYvK08J28fcLC4lmtf3URBdRPL7sjkpknJzg7Jrdlqi/8fwNdSymHAWCDXRv32C+1LMHdm4cKFZ9XkcRtB0RAx6KcpiMppGmt1NNe1EJUU3HVjQwsU76EuJN3+gbmI7w6XM+/1LXgIwep7pjJzqHV1rJTO9TrxCyFCgRnAmwBSyhYppaa3/TqaPcoytzdr1iweeeQRJk2axNChQ9tKP5xZgvmBBx7g6aefBswF2GbMmIHJZCIgIICUlBS2b3fTnaTJUwmtzTGfkUs5TUW+ufZTdLIVW/zFe8CgpTZ0hJ2jcg3vbTvFord2MjAykDVLpjM8PsTZIfUJtpjHnwpUAMuFEGOBXcADUsrG9o2EEIuBxQDR0dFkZWWd1kloaGhb8TPNC3+n5cgRG4T2E5+hQwn7za87fXzevHnccsst3HnnnZhMJlauXMn69evPKsh2ySWX0NDQcNbz//jHP55WNwfMR8x6e3tTX1+P0WikqamJ9evX88033/Dkk0/y2Wef8cMPPzBs2LC25Tz++OPMmjWLCRMm8OCDD/LRRx/R2GhelaNHj2bdunUMHz78tOUYjcYOC8dZS6vVnvV+2FpccwTDDPVsX/sOTYGu+TNdo9FgNBrtvi7OVHHQ/GWYe3IvR4rOPWadfOojBgHFXgOpdHCcPdHQ0NCj9WmSko+O6Fl7Qs+YaE/uHWng0J6t2Os3Y0/jdFe2SPxewHjgF1LKbUKIfwCPAv/XvpGUcimwFCA9PV3OmjXrtE5yc3MJDjb/1G308cbUy1o5Z/L28W7rvyOjRo0iOjqavLw8ysrKmDBhAtHR0Wc9p30d/a74+vri6+tLcHAwnp6ezJ8/n+DgYM4//3weffRRgoODqa2tJT4+vm05wcHBvPnmm8yYMYMXX3yx7ZcAQFJSEocOHTorptaaPj3l5+dHRkZGj59vlepkOPxPJsXoIXOWfZfVQ2FhYWg0Gs78bNrbV4f3o4tuYM5FVuwae/cViB6GT3iCw+PsiaysrG7HqdUb+c2H+1h7ooSbJyfzh6tH4mXnQms9idOd2SLxFwKFUsptltsfYU78PRb3+OO9Dqon7FGWub3WEgyenp4YDAag4xLMnZVydkppZVsJT0XnE4Hvqc2Qee4d2f1NZUE90clWDGGYjOa6R2NusH9QTlLVoOOut3eyO1/D45cP467zVaE1e+h14pdSlgohCoQQ6VLKw8AcIKf3oTmercsyW+PMEsynTp3ihRdeYM+ePVx++eVce+21TJ5snq995MgRpk+fbtPlO4wQaMJGEntqs3mcX/0zA6Br0lNXqWXEeVbUlyndDy31MHA6VNk/Nkc7XtHAghU7KK3V8trN47l8dHzXT1J6xFa/n34BvCeEyAbGAX+2Ub8O1VqWed68eb0uy2yt9iWYpZTceeedPP/88yQkJPDmm2+yaNGitl8EmzZtaiu57I5qQ0dCfTHUnHB2KC6jssC8vyjamhk9pzaZLwdOs2NEzrHjZDXX/Wsz9VoDK++aopK+ndmkSJuUci/g9mfasGVZZjDP42/VfsdRVFQUJ0+ebLvdvgTzunXr2u6fMGEC+/fvB8xz/UeOHNnh6SDdhSZspPnKqc0QMci5wbiIcsuMHqumcp7aDOGpEJIA2HbygzN9ureI367OJjHcn+ULMhkYqWru2Js6ctfCmWWZrSnBXFlZyTPPPOOgiOyjKSAJAiLNCUwBoOxELcGRfgSEdFFrxmQyr7eBbjrU1wEpJa9+l8cDq/YyLimMT5ZMU0nfQVRZZgtnlmW2pgSzOw/xtBECkqf+NGShUHaijvjBoV03rDwMzdV9ZphHbzTxxJoDfLCzgGvGJfDcz8bg66Vq7jiK2uJXHGvgdKg5CbVFzo7E6Ro1OhpqdMSmWpH4+9D4fp1Wz8IVO/hgZwG/uCCNl24cp5K+g6nErzhWa+LK3+LcOFxA2Yk6AGJTrZjKeWozBCdAeIp9g7KzYk0z817fwpZjVTx3/Rh+c3G6mq7pBCrxK44VNxp8guHkj86OxOlKT9Ti4SW6ntEjpWV8f5pbT4M9UGQutFZU08yKBZOYl5nk7JD6LTXGrziWhycMnKoSP+Yt/qjEYDy9u9j+qjoG9SVuPcyz4VAZ96/cQ5i/Nx/dO430uJ4faa70ntrid6KcnBxiYmK49NJL247k7RdSZ0DVUagr7rptH2Uymig/VUecNcM8J7LMl4Nm2TMku3lny0kWvbWTQdGBrLlvukr6LkAlficpLi5m3rx5rFmzhpEjR7J48WJnh+Q4rQns+PdODcOZqksaMbSYrBvfP/49hCa53bEPJpNk1SEd//fpQWanx/DB4qnEhvSfcwi4MpX4LexRlrmiooLrr7+ezMxMMjMz2bTJPDOjrq6OG2+8kaVLlzJ9+nReeOEFoqOjefLJJwHYsWMHY8aMQavV0tjYyMiRIzlw4ECvYnEpMSPN8/lP9N/EX3rcyh27JhOc3AipM91qfF+rN3Lfyt18fdLAbVMHsvS2iQT6qpFlV+GS78TGD4+0HcpuK1FJQZw/b2injy9cuJDrrruOBx98EJPJxKpVq1i/fv1Z7bpTpO2BBx7gV7/6Feeddx75+flccskl5ObmEhISclbNn7/+9a9t1zMzM7n66qt54oknaG5u5pZbbmHUqFHdfcmuy8PDPNxz/Pt+W7en+KiGgFAfQqK6KLpXmg3NNTBopmMCs4FKS6G1vQUabhrmwx+uHqlm7rgYl0z8zpCSkkJkZCR79uyhrKyMjIyMDssjdKdI27p168jJ+aleXV1dHQ0NDQQFdX3CjSeffJLMzEz8/Px4+eWXrV6m20idCQfXQFUeRDn2SGlnk1JSfFRDwpCwrhNi66+i1Bn2D8wG8sobWLBiOxX1Ov518wT8Kg+ppO+CXDLxn2vL3J5sXZa5tfaPn1/3xzWrqqpoaGhAr9ej1WoJDOxjh7K3bsEez+p3ib+usplGjY4BQ8K6bnz8e4geBsFxdo+rt7Ydr2LxO7vw8hC8f9cUMpLDycpSp9t0RWqMv525c+fy9ddfs2PHDi655JIO22zcuJG9e/ee9ddRLf6LL774tPPk7t271+pY7r77bp555hluvvlmHnnkkW6/FpcXngqhyebE388UHdEAkDAk/NwNDTrzgW6prj/M8989Rdz65naignxYs2Q6GcldvDbFqVxyi99ZWssyh4WF2aQs88svv8x9993HmDFjMBgMzJgxg9dff73L57399tt4e3vz85//HKPRyLRp09iwYQMXXHBBr2NyGULAoBmQ+7n5BCMe/eeQ/ZKjGvyCvAmPDzh3w8IdoG9y6fF9KSWvbMjjhW+PMDk1gqW3TiQ0wNvZYSldUIm/HVuXZY6KiuKDDz7o9vNuu+02brvtNsB8tq5t27Z18Qw3lToL9rwLJftgwHhnR+MwRdaO7x//HoSHy1bk1BtNPP7JflbvKmRuxgCevX60qrnjJtRQj4UzyzL3W607LPvRtM76ai31VVoS0sK6bnzie0jIAH8r2jpYbbOeO5ZvZ/WuQn45Zwh/nzdWJX03orb4LZxZlrnfCo6FmBHmcf7zfuXsaByi+EgNAAlDw87dUFsHRbtg2i/tH1Q3FdY0sXDFDo5XNPL8DWP52YREZ4ekdJNLbfFLKZ0dQr/j9HU++AJzAbKWRufG4SAFh2rwDfQickAXU3pPfA8mA6TNcUxgVtpfWMvc1zZTUqvlrYWTVNJ3Uy6T+P38/KiqqnJ+IupHpJRUVVX1aLqpzaRdCMYWOPGD82JwECklBTnVJA2PwMOji/H9o9+CbwgkTXZMcFZYl1PGvH9vwcfTg0/uncb0tChnh6T0kMsM9SQmJlJYWEhFRYWzQ2mj1WqdmxSt1Js4/fz8SEx04lbbwGngHWhOdOmXOS8OB6gqaqSproXkERHnbigl5K0zz+bxdI0ZMm9tPskfPj/IqAGhvHH7RGKCXf//QumczRK/EMIT2AkUSSmv7O7zvb29SU1NtVU4NpGVlUVGRoazw+iSu8TZIS9fc4LL+7bPl2/Iz6kCIGn42UeEn6biENQVwcyHHRDVuRlNkj99mcuyTSe4cHgsL980jgAfl9leVHrIlkM9DwC5NuxP6S/SLgRNPlQedXYkdlWQU01EQiBB4b7nbnj0W/NlmnPPs9zcYuTed3exbNMJFkxP4d+3TlBJv4+wSeIXQiQCVwBv2KI/pZ8ZYklwed86Nw470rcYKcmrJamrYR4wr4eYERA6wP6BdaKiXsf8pVv4NreMJ68cwVNXjcSzq/0Situw1df3S8DDQKdnWBBCLAYWA0RHR5OVlWWjRdtPQ0ODitOGzhVnZkAiuu0fkq0b6dig2tFoNBiNRrusy/piidEg0egLycrq/ETznoYmpp/cTGHiVRw/Rxz2fM+LG0z8fZeWOp3kF+N8GWQ4RVbWqR711Rc+m32SlLJXf8CVwGuW67OAL7p6ztChQ6U7+O6775wdglX6RJxfPy7l01FS6hocFs+ZZs6cKceOHWuXvr9feUi+fv93Uq8znLth7hdSPhUi5bGsczaz13u+Ka9Cjn7qaznhmW/l3vyaXvfXJz6bLgTYKXuZs6WUNhnqmQ5cLYQ4CawCLhBCvGuDfpX+pG1ap/Vlr92FlJIT2ZUkjYjAy6eLo1uPfgs+QZA81THBtfPJ7kJuX7admBA/1iyZxtikMIfHoDhGrxO/lPIxKWWilDIFmA9skFLe0uvIlP6ldVpnHxznr8ivp6FGR+rY6HM3lBKO/s98akovH4fEZl6s5KV1R/j1h/vITIng43unkRTRRQE5xa25zAFcSj/XOq3z8NfmBNiHnNhXiRCQMqaLaZwle83TONMvd0hcAC0GE79ZvY+X1h3l+vGJrFgwiVB/1zh2QLEfmyZ+KWWW7MEcfkUBYNgVUFdoPt1gH3JiXwXxaWH4B3WxFX9orbka59BLHRJXbbOe25dt55PdRfzqwqE8f8MYfLzUtmB/oN5lxXUMvdSc+A596exIbKa2oomqokZSx1pR3uDwWkiaAoFd/DKwgYLqJq7/12Z2nqrm7/PG8sCFQ9QpEvsRlfgV1xEYZd6p2YcS//G9lQBdj+/XnISyAzDM/sM8+wo0zH1tE+V1Wt5eOJnrxqtCa/2NSvyKa0m/3JwAq084OxKbOLqjjJiBwYRG+5+74eGvzJd2Ht//38FSbly6BT9vTz5ZMo2pg+3/60JxPSrxK66ldYv38FrnxmEDmrImKvLrGZIZ23XjQ1+aT6oeOdhu8Sz78QR3v7uL9LgQ1iyZTlpMp8dbKn2cSvyKa4kYBDEjzTs63dyRHWUgIG1CF4m/qdp8TgI7be0bTZLff3aQp7/I4eIRsay6awrRwV3UC1L6NFVxSXE9w66Ajc9DY5VDdnTag5SSozvKGDA0zLqibNJoft021tRi4Jfv72Vdbhl3npfK45cPVzV33IxJmqhoqiC/Pt9mfarEr7ieYVfAD8/Bka8gwz2PBazIr0dT1kTGRcldNz64BoITIMG2J5wvr9dy54qdHCyu5Q9Xj+T2aSk27V+xHYPJQGljKfn1+RTUFVBQX2C+Xl9AYX0hWqPWpstTiV9xPfFjIWygOSG6aeI/vLUUDy/BoIwuZvM0a+DYesi8CzxsN/J6pKyeBct3UN3YwtJbJ3LhCCv2Myh2ZTQZKW0q5VTtKU7Vn+JUnfmvoL6AooYiDCZDW1tfT1+SgpNICk5iWsI0koOTSQpOYjrTbRKLSvyK6xECRs6FLa+45XCPocXI4W2lDB4XjV9gF0fBHv7KXKNo5FybLX9TXiX3vLsLP29PPrx7KqMTQ23Wt3JuUkoqmys5WXeS/Lr8tuR+qu4U+fX56E36trYBXgEkhySTHp7ORQMvakv0ycHJRAdE4yHstwtWJX7FNY26Hja9BLmfwcQFzo6mW47tqUDXZGDEeQldNz64BkKTIHGiTZa9emcBj32yn0HRgSxfMIkBYV1MI1V6pFZX25bcz0zyTYamtnbeHt4kByczMGQgMxJnMDBkIMkhyaSEpBDlH+W0g+ZU4ldcU9xoiEyDg5+4XeLP+bGYkGh/BgwNP3fD5ho4tgGm3NPrU05KKXnx2yO8vCGP89KieO2W8YT4qZo7vaE36SmoL+CE5gQn6k5wovZEW5LX6DRt7TyEBwOCBpAcksz42PEMDBnIwOCBDAwdSFxAHJ4eXVRkdQKV+BXXJASMvM48u6e+DILdY4y6prSR4qMaps4djOhq9syhL8Gk7/Uwj85g5NGP97NmTxE3TEjkz9eNxttTzdS2VkNLAyd1J/ns2Gcc1xznRK050RfUFWCQP427x/jHkBKawkUDLzInd8vWe1JQEt6e7vUlqxK/4rpGXW+e3ZPzKUxe7OxorHLghyI8PATDpsZ33fjgGvNO7F7M5qlt0rP4nZ1sO1HNQxcP5b7ZaarmTgeklJQ3lXOi7sRPyd3yV95cbm5UCl7Ci+SQZAaFDmJO8hwGhQ4iNTSV1NBUAr0DnfsibEglfsV1xQwzn3v2wMdukfh1zQZyN5WQlhlDQEgXlTgbq+B4Fky9v8fDPPlVTdyxYjuF1c38Y/44rhnnvHP0ugopJWVNZRytOUqeJo88TZ450dedoFHf2NYuyDuI1NBUpiRMITU0lcaCRq6efjWJwYl4e7jX1ntPqMSvuLaR18F3fwRNPoRZMSfeiXJ+LEavMzJujhVxHvgITAYYM69HyzqmMfKb1zZhMEneuXMSkwe518yn3pJSUqWtMif3mry2JH9Mc4wGfUNbu2j/aAaFDeLqwVeftvUe7R992i+jrKosUkNTnfFSnEIlfsW1jbnBnPj3fQAzf+vsaDplMprI3lDAgKFhRCdbUQNn70rzDuzY7p9c/usDJTy7XUt8WADLF2QyODqoBxG7j1pdLUdrjnJMc4yjGvNlnibvtB2sYb5hpIWlceWgK0kLSyMtPI20sDRCfdVU1o6oxK+4tvAUSDkf9r4HMx7q9ewXezm2u4KGGh0zbkrvunF5rvlsW5f8pVvLkFLy5o8n+NPaXAaFePDhkmlEBvWdmjsGk4GTtSc5XHOYwzWHOVJ9hCM1R6hormhrE+QdRFpYGnOS5zAkfAiDwwaTFpZGpF+k2rfRDSrxK65v3M3w33sgf4v53LwuRpokO786SXhcACmjrBhy2fc+eHjB6BusXobBaOLpL3J4e8spLhsVx9z4OrdO+nUtdRyuPsyRmiMcrj7MoepDHNMco8XUApjnvw8OG8zUhKkMCRvStgUfGxCrErwNqMSvuL4RV8Pah2DPey6Z+PN2l1Nd3MjFi0Z2PYXTZITsDyHtIgjqopyDRaPOwC/f38P6Q+UsnjGIRy8dxg8/fG+DyO1PSklBXQGHag5xuNq8JX+4+jAljSVtbSL8IhgaPpSfD/85Q8OHkh6RTmpoar/YydpKSolRo8FYVYWhsgpDVeXp1yurMFRX22x5KvErrs8nEEZeCwfWwGV/BV/XGdM2mSQ7vjhBeHwgg8fHdP2E499BfQlc+qxV/ZfVaVm4Yge5JXU8c81Ibp2a0ruA7UhKSUF9ATlVORysOkhOVQ77y/bTnN8MmA90SglJYVz0OG5Mv5H0iHTSw9OdegSrvcmWFgyVlRgqKtCXl2OoqDD/tV6vbJfUDYazO/DywisiAs+oSLwirTh9p5V6nfiFEEnA20AsIIGlUsp/9LZfRTnNuFtgz7vmEg7jfu7saNrk7SyjprSJixeNxMOacse73wb/CEi/rMumh0rrWLh8B5pmPW/cPpELhrnOQWytSb41wedU5ZBblUu9vh4wD9Wkh6czIXACF46+kGERwxgcNhg/Lz8nR24bJp3OksDbJfGKM65XVGCsqTn7yR4eeEVG4hUTg3d0DH7Dh+MVGYVXVCSekZGnXfcMDUW0L973xn9sEr8ttvgNwG+klLuFEMHALiHEt1LKHBv0rShmyVPMJ2nZ867LJH6D3sjW/x4nMjGINGu29utLzUfrTrkXvM49Pr/xaAVL3t2Nv4+50NqoAc6dnVLaWEp2RTYHKg9wsOpgh0n+stTLGBE5ghGRI0gLS8Pb05usrCxmDZ3l1Ni7y6TVYigrQ19SiqGsFH1JKfrSEgylZehLSzGUlmLUaM5+opcXXlFReEVH452UhP/4DLyio/GKiTFfRkfjHRODZ0QEwtO5ZRx6nfillCVAieV6vRAiFxgAqMSv2I4QMP42WPd7KMuB2BHOjoh96wuor9Zyze0ZXY/tA+x5xzx3f8K5aw99uKOAx9fsJy0miGV3ZJLg4EJrTfom8zBN5X6yK7LJrshuO7r1XEneHUiDAX1pGfriIgylpW3JPezAAY6//DKGktIOt9I9w8LwiovDOy4O/3Fj8Y6NNSf0dkndMzz89K1zFyaklLbrTIgU4AdglJSy7ozHFgOLAaKjoyd8+OGHNluuvTQ0NBAU5DrjyZ3pL3F6t9QxdctCSuIv5OjQe2wYmdmDDz6I0Wjkn//8Z5dt9c2SvC8lgbGQfL4V/+zSyJStd9MUkED22Kc7biIlnxzV8/lxPaMiPbkvwxd/r46/UGz1nkspKTeUc1J30vzXcpLilmJMmACI8ooixSeFFF/z3wCfAXgJ67cXHf7ZNJnw0GjwrKo6/a/SfOlRU4MwmU5/SmAg+pAQiIrEGBaOKTwcY3gYxvDW6+Hg08WR2A4ye/bsXVLKXpdytdnOXSFEEPAx8OCZSR9ASrkUWAqQnp4uZ82aZatF201WVhYqTtuxSZyNP2NA7ucMuG0p+IXYJK5WYWFhaDQaq2JctzwHZBlXL55MWGxA150f+QZ0Ffhd8wKzRpzdv85g5Lers/n8eDHzM5N45tpR5yy01tN1qTfpya3KZXfZbnaV72Jv+d62A6GCvIMYFTWKy6MvZ0zUGEZHjybCL6Lby7BFnOdirK+n5eQpWk6dQl+QT0tREfrCIvRFRehLSs7aSeoVE4P3gAF4p6fjPSAB7wED8BkwAK+4eLzjYvEICHCb/yFbsUniF0J4Y07670kpP7FFn4rSocy7zPPg961yWv2e/JwqDm8rZeLlKdYlfYAdb0JQbIcnVK9pbOHud3ax/WQ1D1+azr0zB9tslkuTvol9FfvYXb6bPWV7yK7MptlgnmWTHJzMrKRZZMRkMDZ6LKmhqXY9+Ud3GOvqaDmVT8upU7Tkn0J/6lTb7TOHYjyjo/BJGID/6NGEXHaZOckPGGBO8gkJePi67/EO9mKLWT0CeBPIlVL+vfchKco5JE6AhAzY8QZMusvhR/LqdUay3jtMWGwAEy4baN2TKo7A0W9g5qNwxlj4qapGFizfQWFNMy/flMHVY604ecs5NOmb2FW2i+2l29lZupPc6lyM0oiH8CA9PJ3rhlzH+JjxjI8dT5S/7aYH9oQ0GtEXFKA7fhzdsWO05B2j5eRJWvLzz0ruXvHx+CQnE3zRRfgMTMY7ORmfgQPxSUrCw1+dbKa7bLHFPx24FdgvhNhrue9xKeVaG/StKGebtBj+e6/5XLVpFzp00Vv+e4z6Ki1zf5OBl7eVMzO2vgqevpC56LS7d52q4a63d2KSkvfumkxmSveHVfRGPfsq9rG9dDvbSraRXZGNQRrw9vBmdNRoFo5ayPjY8YyNHkuwjxU1hOzApNOZE/qxY+iOHUd3/KckL/U/nYrQKzYWn5QUS3IfiM9Ac3L3TkrCw69vTAN1FbaY1fMj0DePvlBc06jrYf3T8ONLDk38J/dXsv+7QsbMTiRhSBdn12rVUAF734dxN512pO7a/SU8+MFe4kP9WLFgEqlR1tV6l1JypOYI62rXserbVewu302zoRmBYETkCG4beRuT4yeTEZOBv5djt4SllBiKi9EePozu8GG0hw4TuXcPhysqoXWHqhB4JyXhO2gQgTPOx3dwGr6DB+EzaBCewc75YuqP1JG7ivvx8oUpS+Db/4OiXTBggt0X2VirY/1buUQmBjH1usHWP3HHG2DUwZT7AHNyXPrDcf7y1SEmDAxn6a0Tuqy509DSwNaSrfxY9CMbizZS3mSeWjlIDuLatGuZHD+ZibETHVqJ0qTVojt6FO2hQ+gOHUZ7+BC6w0cw1de3tfFOTsYQn0Dc9dfjM3gwvoMH45OSorbeXYBK/Ip7mnAH/PC8eav/xnfsuiijwcT/3jiIQWfk4oUjrR/i0TfDjv/A0EsheigGo4mnPjvIe9vyuWJ0PC/MG4tfB31JKTlee5yNhRvZWLSR3WW7MUgDQd5BTE2YyvkDzsfjlAfXXHiNjV9px6Rej+7oUZoPHEC7/wDNBw6gO3q0bfaMR0AAvkOHEnLF5fgNG4bv0HR8hw7FMyiQrKwsMvrRbBl3oRK/4p78QiDzTvjxRajMg6g0uy3qxw+PUnxUw0ULRxCR0I3T7+1cDk1VMP0BGnQGfrFyN98druCemYN5+JL000o8mKSJA5UHWJe/jg35GzhVdwqAoeFDuW3kbZw/4HzGxoxtK1yWVZRly5fYRkqJvqCA5j17aN5/AO3+/WgPHULqdAB4hIbiP3IkQXfeid/IEfgNG4Z3YqLbHLikmKnEr7ivKffClldh4wsw9192WcT+rEIO/FBExsXJDJ0UZ/0TW5rMX0qpMygNG8/C17dwuKyeP88dzc8nm8/QZTAZ2FW2i3Wn1rGhYAPlTeV4CS8mxU/ithG3MSNxBnGB3VhmD0i9Hu2hQzTt2kXz7j007d6NsbISABEQgP+IEYTfdBN+o0fhP3o03klJfbagWn+iEr/ivoJizFM6t74G0x8wn6PXho7uKOOHD46QMiaKKdd2Y1wfYNdyaCzn5OjXmP/qJuq1et68fSIzh0azr2IfXxz7gm9OfkONrgY/Tz+mD5jOnOQ5zEicYdexetnSQvO+fTRu2WpO9tnZyGbzvH7vxEQCp00lYPwE/DMy8E0b7PSaMop9qMSvuLfzfg27VsB3f7LpWP+pg1WsW55DQloYl1hbebNVSyP8+BI1sdO48lMTQb4e/OPWRA42rua5NV9SUF+Ar6cvs5JmcUnKJUxPmE6At5UHgnWTNJnQHT5M4+YtNG7dStPOneZE7+GB37BhhP3sZwSMz8B//Hi8Y12n+qdiXyrxK+4tMBKm3g/fPwtFu2HA+F53eTK7kq+XHiBiQCCXLxmDl083t3o3vwKN5SyqvZuIhGzik/bwq837EQgmxU9i8ZjFXJh8IUE+9qlhY9RoaNj4Iw1ZWTRu3tx2MJTPoEGEXXedeas+MxPPENuWvFDch0r8ivubeh9sXwrr/wC3/rdXR/MOjszgq9f3E5UUxFW/GIevf/f+RUy1xRzZ8g9eDB/GsdDVGEUToYYUfjPhN1yWehmxgbbfqpZS0nLiBA3ffUfDd1k07dkDRiOeEREEzTifgKlTCZw6VW3RK21U4lfcn18IzHwEvn7EXO9++JXd7kJKSXrENEZHX0Dc4FCuWDIGn24kfSklPxRs4tVvHiI3IRIPdFw08CJuHDaPibETbb9DVEq0R45Q99VX1H/1NS0nTwLgO2wYkXctInj2bPxGj1azbZQOqcSv9A2Zi2D3W/D1Y5A2B7ytP2rV0GJkwzuHGBNzIccq93DPLx60enjHYDLwzclveGP/cvI0h4nyMDLPOJglN60g0t+KE693k+74cerWfkXkRx9xorQUPDwImDyJ8NtuJXjWLLwTelfrR+kfVOJX+gZPL7j8b7DiCvM0ytmPW/W0ysIGvl12kOqSRvaXr2fr8S/x8vlNl88zSRPrTq3jlb2vcKL2BJ6GGO6uEdxmaCHkgXfAP6yXL+gnxvp66tZ+heaTj9HuywYhMKWlEXf3YoIvugivKOcWW1Pcj0r8St+Rch6MvgE2/h1GXAOxIzttajJJ9q0vYOunx/AL8ObK+8ey+r6OT5Bypm0l23hh5wvkVueSEJCCR/nt3NZ0nPvlTphnm6QvpaR55040H31E3Tf/Q2q1+A5JI+aRRwi54nI25eQwVh0Rq/SQSvxK33Lps3A8C9bcA4vWg9fZZ04qPV7LD6uOUJFfT+rYKGbfOgz/oK7PsFTZXMnfdvyNtSfWkhCYwM+SH+K99VFMDtHwoOffYPAVMPyqXoVv0umo+3It1W+/je7QITyCggi99hrCrr8ev1GjftpXkKPObKr0nEr8St8SGAVXvgQf3Aw//A0u+F3bQ3WVzez48gSHtpQSGOrDxXeOJG1iTJc7XqWUfH78c57d9ixao5a7x9yNrLmAF745wdSBgSyXf8Gj3h+ueL7HM4oMNTXUvPMuNatWYayuxndIGnFP/4HQq65S9eYVm1OJX+l7hl8JY38OG5+H5CnUhU9j19enOLSpBDwg46JkJl6Rgo9f1x//Rn0jz2x9hi+Pf8n4mPH835SnePO7Bt7ffoKrxibwYsj7eO3YD/Pfh5Du71g1VFdTvWwZ1SvfRzY3EzRzJhG33UrA1KmqNIJiNyrxK32SvPxvFB+tZf9rOzmuFQgPwYjzE5hw6UCCwq0rC1zWWMb9G+7nSM0R7h93PzcOuZ1frsrmhyMV3Dd7ML9JOIjHx/+GSXfDsLNPqXguxro6qpYupfq9lUitlpDLLyfq3nvwTbNfsTlFaaUSv9Kn1FY0c3RnGUe2l1FTsghfjwbGRW5k9JIlBMdHd92BRXFDMbd/fTt1ujpenfMqg4MmMH/pdo6WN/DsdaOZH18GK+6F5Glw8TNW9yv1emo+/JDKf76CsbaWkCuvNCf8QYN68nIVpUdU4lfcmjRJKgsbOHWwipPZlZSdqAMgblAos28dxpDIQ3h/+E/4agvc8gn4dF0Tp1pbzeJvF9Oob2TFpSsw6RK49tVNNOqMLL8jkxkhZfDWPPPQzo3vmk8MY4XGbdsp/cMfaDl+nIDJk4l95GH8Rozo1etXlJ5QiV9xKyajiariRsqO11JyvJaC3Bqa61oAiE4OZurcwaRNjCEksnWHaAJc/x9YvQDeuwFuWgl+nVe/lEie2vwUJQ0lvHHJG5RWRnL/e1sI8fdm9T1TGc5JePta8PKDWz8x1wrqgrG2lvLnn0ez+iO8k5JIfO1VgmbPVmP4itOoxK+4rBatgeqSRqqLG6kuaqSyqJ6yk/UYdEYA/IO9SUwPJ3lUJEnDIwgM7WTLe+RcMBnNUzyXXQbz34OI1I7bDoGsgiwemvgQB49H8NRnOxkWF8yyOzKJLV4PH99lnqd/++cQ0fXwTMPGHyl+/DGMVdVE3LmQ6PvvV7N0FKezSeIXQlwK/APwBN6QUj5ri36VvktKSYvWSHNdCw0aHfVVzdRVaam3/NVVNtNQo2tr7+XtQURCIMOnxBE7KJS4QaGERPlZv9U8+mcQEAEf3gH/ngGXPw9j5p02/VIiMWWaSAlJofDkRP6z8QCz06N55WdDCdz4pLkQXEIGzF8JIfHnfn0tLZT/4x9Uv7kM3yFpJL3+Ov4jOz+gTFEcqdeJXwjhCbwKXAQUAjuEEJ9JKdURJn2UySQxGkwY9eY/g96EocVIS7MBXbMBvdZIi9ZAS7PlUmtA12Sg6JSJDzfvoLm+hab6FkwGeXrHAoLCfAmO9CNhaBjhcYFExAcSOSCQ4Ej/7tXE78jgC+CejfDxIliz2Hw+3Bm/hcFzwNMLU6AJYsGjYSr/OXiShZlR/C5hN55Lb4f6Epi0GC76Q5d1gPTl5RT94pc079tH2PwbiX30UXWCccWlCCll163O1YEQU4HfSykvsdx+DEBK+ZfOnhMZGiPnXnAHgs7+kX+6//QNuo7bW9XP6UFb1Y9Wq8XPz6/X/XQZT/tbp121rh+tToefry8IgQceCCEQp116IBBnXP70uEcnj3t6eOEpvPAQXngKT/Olhxcewvr69FJK9CYdepOWJl0dBnRoDY3ojI3tLhto1NfQpK9DYrK6757yQHJJXDV3phYT5WugUufFHk0wn+kMZM0M5+qCNC7THCXTvxBfT8leTRBvHI/nQF3X9fMT9XrurdEQKE28HRrKbjslfI1GQ1hYmF36tiUVp219//33u6SUE3vbjy2GegYABe1uFwKTz2wkhFgMLAZIihpKRuylNli065LypwR2+ler7ORqx1/And1Puy/s9m1M0oSUJiQmpJSYpBGJNN8nTebHLbdNre1MJkxSYkJ/WjuDSY9JGjCaDBhMeowmA0ZpuTQZMEoDRsv9elMLeqOWFqMWvUFLi1FHi7EZg0nf9kKNRiOeLnIqv/c1nqw+nMiMuCYuTWxkbGQD5TG+ZAEPGr9HY/Tgk1PBfF0YyL5qP8AAaM7Z52ijiV/r9TQCT/l4c0qrBa3WLvEbjUY0mnPH4wpUnK7JYTt3pZRLgaUAQ4emyztfOB84Y6NZdLZl3XGfp43vdny18y1o0Vmbn3z//ffMtBTC6rRPF5iZkZWVxSw3KNjlqnFuP1HNJe/spKmogtD1bxH9+T6iPTwYAtxkZR/1WVkU/eKX+Awbxqil/+bTmBh7huyy6/JMKk7bslW+sUXiLwKS2t1OtNzXKSHAL9DbBou2L+Ehej+urLi0T/cW8dvV2SSG+xO4aTWNZeXQzZOX1G/YQOEDD+KXnk7yG//B0w2GDJT+zRan59kBDBFCpAohfID5wGc26FdR7EZKyavf5fHAqr2MSwrjkyXT8NZput1P0+49FP3q1/gNH07ysjdV0lfcQq+3+KWUBiHE/cA3mKdzLpNSHux1ZIpiJ3qjiSfWHOCDnQVcMy6B5342Bl+v7u97aDl5ksIlS/CKiyXp36+rk5crbsMmY/xSyrXAWlv0pSj2VKfVc997u9l4tJJfXJDGry8a2qNxU1NjIwVL7gMhSF66FK/wcDtEqyj2oY7cVfqNIk0zC5fv4FhFA89dP4Z5mUldP6kDUkpKn36alhMnSF72Jj4DB9o4UkWxL5X4lX7hQFEtC1fsoLnFyIoFkzhvSM/PU1v35VpqP/2MqPvvJ3DqVBtGqSiOoRK/0uetzy3jF+/vITzAh3funUx6XHCP+zLU1FD25z/jN3YMUffeY8MoFcVxVOJX+rR3tpzkqc8OMiIhhGW3ZxIT0rsjacuf+xvGujqSn34a4SIHoylKd6nEr/RJJpPkz2tzeePHE1w4PIZ/zM8g0Ld3H/fm/fupXbOGyLvuwi893UaRKorjqcSv9DnNLUZ+9cFevj5Yyu1TB/LkVSPxtMGBeBUvvohneDiRdy+2QZSK4jwq8St9SmWDjkVv7WRfoYb/u3IEC6en2OQw98at22jcvIXYxx7FM6jrYm2K4spU4lf6jLzyBhas2E5FvY5/3TyBS0fF2azv6uXL8YyMJGz+fJv1qSjOohK/0idsPV7F3e/swttTsGrxVMYlhdmsb92JEzR8/z1R99+Ph69159dVFFemEr/i9tbsKeThj7JJjghgxYJJJEV0fUL17qh59z2Etzfh82+0ab+K4iwq8StuS0rJPzfk8fdvjzBlUAT/vmUioQG2rfoqW1qo/eILgi++GK+onh/0pSiuRCV+xS21GEw8vmY/H+0q5LqMATx7/Rh8vGxRbPZ0DRs3YqqtJfSaq23et6I4i0r8itupbdZz77u72HysigfmDOHBC4fY7YQ4tV98gWdEhCrNoPQpKvErbqWwpokFy3dworKR528Yy88mJNptWVKvp/HHTQRfcjHC2/VPHKQo1lKJX3Eb2YUa7nxrJ1q9kbcXTmJamn3H3Jv37sVUX0/QjBl2XY6iOJpK/IpbWJdjLrQWEejDykWTGRLb80Jr1mr44Qfw8iJw2jS7L0tRHEklfsXlrdh0gqe/yGHUgFDeuH0iMcG9K7RmrcZt2/EfN1Ydqav0ObafBqEoNmI0SZ7+PIfff57DnOGxrFo8xWFJ36TToc3NJSAjwyHLUxRHUlv8iktqbjHywKo9/C+njAXTU3jiihE2KbRmLW1ODuj1+I8d67BlKoqjqMSvuJyKeh2L3tpBdlEtT101ggXTUx0eQ/O+fQD4jRnj8GUrir31KvELIf4GXAW0AMeABVJKjQ3iUvqpo2X1LFixg6qGFpbeOpGLRsQ6JQ5tdjZeCfF4x8Q4ZfmKYk+9HeP/FhglpRwDHAEe631ISn+1+Vgl1/1rM1q9iQ/unuK0pA+gPXIEv+EjnLZ8RbGnXiV+KeX/pJQGy82tgP2OplH6tI93FXL7su3EhfixZsk0xiSGOS0WaTDQciof30GDnBaDotiTLcf4FwIfdPagEGIxsBggOjqarKwsGy7aPhoaGlScNtRRnFJKPj2m5795eoZHeHD/aBPHsrdzzAnxaTQajEYjP370EVF6Pcf1LeS46Hp15/fcFblLnLbSZeIXQqwDOjqjxe+klJ9a2vwOMADvddaPlHIpsBQgPT1dzpo1qyfxOlRWVhYqTts5M84Wg4lHP8nmv3lFXD8+kb9cN9ouhdasFRYWhkajYWxkJEXA2CuuwH/0aKfFcy7u+p67KneJ01a6TPxSygvP9bgQ4g7gSmCOlFLaKC6lj6tt0nPPu7vYcryKX180lF9ckGa3QmvdpS8oBMBn4EAnR6Io9tHbWT2XAg8DM6WUTbYJSenrCqqbWLBiB6eqGnnxxrHMzXCtXUP60hI8AgLwCLZ/WQhFcYbejvG/AvgC31q21rZKKe/pdVRKn7W3QMOit3bQYjDx9sLJTB0c6eyQzmIoKcUrId5lfoEoiq31KvFLKdNsFYjS9+0qM/Cf9VuIDvZl1eIppMW45ha1vqQE77h4Z4ehKHajavUoDvHmjyd4ZY+O9LgQ1iyZ7rJJH0BfWop3fEfzGRSlb1AlGxS7Mpokz3yRw4rNJ5kQ68m7d03B38fT2WF1ylNKjJWVeMWpxK/0XSrxK3bT1GLgl+/vYV1uOYvOS2VaYJlLJ32AEMulV6Q6sbrSd6mhHsUuyuu03PjvrWw4VM7T14zkiStH4OEGO0sDLTOSPUNDumipKO5LbfErNnekrJ4Fy3dQ3djCf26byJzhzqu5012BlkuPEJX4lb5LJX7FpjblVXLPO7vw8/Hkw7unMjox1NkhdUtQ6xZ/iHvFrSjdoRK/YjOrdxbw2Cf7GRQdyPIFkxgQ5u/skLqtdYtfDfUofZlK/EqvSSn5+7dH+OeGPM5Li+K1W8YT4uft7LB6pG2MXw31KH2YSvxKr+gMRh75KJv/7i1m3sRE/jR3NN6e7jtnIFACQqhyDUqfphK/0mOaphYWv7OL7Seqeejiodw323UKrfVUIOARHIzwdO1pp4rSGyrxKz2SX9XEHSu2U1jdzD/mj+OacQOcHZJNBEqphnmUPk8lfqXbdufXcNdbOzGYJO8umsyk1Ahnh2QzgajxfaXvU4lf6Zav9pfw4Ad7iQ3xY/mCTAZHBzk7JJvykxKPwMCuGyqKG1OJX7GKlJI3fzzBn9bmMi4pjDdum0hkkK+zw7I5b0D49r3XpSjtqcSvdMlgNPGHz3N4Z+spLhsVx4s3jsPPu2/u/PQBhJ9K/ErfphK/ck6NOgO/eH8PGw6Vc/eMQTxy6TA8PNx75s65eAMePirxK32bSvxKp8rqtCxcsYPckjqeuXYUt07p++eg9ZES4efn7DAUxa5U4lc6dKi0joXLd6Bp1vPm7ZnMHhbj7JAcwgcQvj7ODkNR7EolfuUsPxypYMl7uwn0NRdaGzWg/xQs8wY8fNUWv9K3qcSvnOaDHfk8vuYAQ2KCWL4gk/hQ9yu01hvmLX41xq/0bTYpqiKE+I0QQgoh1GmL3JTJJPnbN4d45OP9TE+LYvU9U/td0veQEk/UUI/S9/V6i18IkQRcDOT3PhzFGXQGI79dnc1n+4qZn5nEM9eOcutCaz3lbanMqYZ6lL7OFkM9LwIPA5/aoC/FwWoaW7j7nV1sP1nNw5emc+/MwW5faK2nWo9MEF598xgFRWnVq8QvhLgGKJJS7usqWQghFgOLAaKjo8nKyurNoh2ioaGhT8dZ1mji77u0VGkl9471ZQSFfP99oe0DtHD19VlbWwvA0bxjNLtwnOD667KVitM1dZn4hRDrgLgOHvod8DjmYZ4uSSmXAksB0tPT5axZs6yP0kmysrLoq3HuOlXDr9/eiQkv3l88kcwU+xdac/X1GRYaCuUVDBkyhAgXjhNcf122UnG6pi4Tv5Tywo7uF0KMBlKB1q39RGC3EGKSlLLUplEqNvVldgm/+nAvCaF+LF8widQoVZTsNP10qEvpP3o81COl3A+0HdUjhDgJTJRSVtogLsUOpJT8+4fjPPvVISYMDOc/t00kIlDNYGnVlu5V4lf6ODWPv58wGE089dlB3tuWzxVj4nnhhrF9ttBar6m8r/RxNkv8UsoUW/Wl2FaDzsD9K3eTdbiCe2YO5uFL0vt0obWeEtLZESiKY6gt/j6utFbLghU7OFJWz5/njubnk5OdHZILs2R+NdSj9HEq8fdhOcV1LFyxg3qtnjdvn8is9P5RaK2nWtN9fz2OQek/VOLvo7IOl3Pfe7sJ9vNm9T3TGJGgziNrNZX4lT5OJf4+aOW2fP7v0wMMjQ1m+R2ZxIWqEgTdoxK/0repxN+HmEyS5745zOvfH2Pm0GhevXk8Qb7qLbaWSvdKf6GyQh+h1Rv5zep9fJldws2Tk/nD1SPx6oeF1mxCDfUofZxK/H1AfYvk5je2setUDY9dNozFMwapHZQ9IM66oih9k0r8bu5EZSN/3NpMTYuWV38+nivGxDs7JLeljtxV+guV+N3YzpPV3PX2Tgx6yft3TWHCQPsXWusP1K8lpa8TUjr+cEUhRD1w2OEL7r4owB1qD6k4bccdYgQVp625S5zpUsrg3nbirC3+w1LKiU5attWEEDtVnLbjDnG6Q4yg4rQ1d4rTFv2oaR+Koij9jEr8iqIo/YyzEv9SJy23u1SctuUOcbpDjKDitLV+FadTdu4qiqIozqOGehRFUfoZlfgVRVH6GbslfiHEDUKIg0IIkxBi4hmPPSaEyBNCHBZCXNLJ81OFENss7T4QQtj95LCW5ey1/J0UQuztpN1JIcR+SzubTK/qDiHE74UQRe1ivbyTdpda1nGeEOJRB8f4NyHEISFEthBijRAirJN2TlmXXa0bIYSv5fOQZ/kcpjgqtnYxJAkhvhNC5Fj+lx7ooM0sIURtu8/Ck46O0xLHOd9HYfayZX1mCyHGOyHG9Hbraa8Qok4I8eAZbZyyPoUQy4QQ5UKIA+3uixBCfCuEOGq5DO/kubdb2hwVQtxu1QKllHb5A4YD6UAW5pOwt94/AtgH+AKpwDHAs4PnfwjMt1x/HbjXXrF2Ev8LwJOdPHYSiHJkPGcs//fAQ1208bSs20GAj2Wdj3BgjBcDXpbrfwX+6irr0pp1AywBXrdcnw984IT3OR4Yb7keDBzpIM5ZwBeOjq277yNwOfAV5soYU4BtTo7XEygFBrrC+gRmAOOBA+3uew541HL90Y7+h4AI4LjlMtxyPbyr5dlti19KmSul7Ojo3GuAVVJKnZTyBJAHTGrfQJiPmb8A+Mhy11vAtfaK9UyW5c8D3nfUMu1gEpAnpTwupWwBVmFe9w4hpfyflNJgubkVSHTUsq1gzbq5BvPnDsyfwznCwbUcpJQlUsrdluv1QC4wwJEx2NA1wNvSbCsQJoRwZmGpOcAxKeUpJ8bQRkr5A1B9xt3tP4Od5cBLgG+llNVSyhrgW+DSrpbnjDH+AUBBu9uFnP1hjgQ07RJHR23s6XygTEp5tJPHJfA/IcQuIcRiB8bV3v2Wn8zLOvkJaM16dpSFmLf2OuKMdWnNumlrY/kc1mL+XDqFZagpA9jWwcNThRD7hBBfCSFGOjayNl29j670eQTzr7jONuxcYX0CxEopSyzXS4HYDtr0aL32qmSDEGIdENfBQ7+TUn7am77txcqYb+LcW/vnSSmLhBAxwLdCiEOWb2yHxAn8C3gG8z/bM5iHpRbacvnWsGZdCiF+BxiA9zrpxu7r0t0JIYKAj4EHpZR1Zzy8G/NwRYNlX89/gSEODhHc6H207C+8Gnisg4ddZX2eRkophRA2m3vfq8QvpbywB08rApLa3U603NdeFeafgl6Wra2O2vRIVzELIbyA64AJ5+ijyHJZLoRYg3nowKYfcmvXrRDiP8AXHTxkzXruFSvW5R3AlcAcaRmQ7KAPu6/LDlizblrbFFo+E6GYP5cOJYTwxpz035NSfnLm4+2/CKSUa4UQrwkhoqSUDi04ZsX7aPfPYzdcBuyWUpad+YCrrE+LMiFEvJSyxDIsVt5BmyLM+yVaJWLer3pOzhjq+QyYb5k1kYr523R7+waWJPEd8DPLXbcDjvoFcSFwSEpZ2NGDQohAIURw63XMOzEPdNTWXs4YG53byfJ3AEOEeXaUD+aftp85Ij4wz5oBHgaullI2ddLGWevSmnXzGebPHZg/hxs6+/KyF8s+hTeBXCnl3ztpE9e670EIMQnz/7RDv6CsfB8/A26zzO6ZAtS2G8ZwtE5/0bvC+myn/Wewsxz4DXCxECLcMuR7seW+c7PjXuq5mMebdEAZ8E27x36HeVbFYeCydvevBRIs1wdh/kLIA1YDvvaK9Yy4VwD3nHFfArC2XVz7LH8HMQ9rOHoGwDvAfiDb8uGIPzNOy+3LMc8EOeboOC3vWwGw1/L3+pkxOnNddrRugKcxf1EB+Fk+d3mWz+EgJ7zP52Eezstutx4vB+5p/YwC91vW3T7MO9GnOSHODt/HM+IUwKuW9b2fdjP9HBxrIOZEHtruPqevT8xfRCWA3pI378S8T2k9cBRYB0RY2k4E3mj33IWWz2kesMCa5amSDYqiKP2MOnJXURSln1GJX1EUpZ9RiV9RFKWfUYlfURSln1GJX1EUpZ9RiV9RFKWfUYlfURSln/l/Jw3j5MvXC68AAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.grid()\n",
    "plt.axhline(0, color=\"black\")\n",
    "plt.axvline(0, color=\"black\")\n",
    "plt.xlim([-10, 10])\n",
    "plt.ylim([-5, 10])\n",
    "plt.plot(np.linspace(-10, 10, 1000), np.linspace(-10, 10, 1000), label=\"y = x\")\n",
    "plt.plot(np.linspace(-10, 10, 1000), np.linspace(-10, 10, 1000)**2, label=\"y = x^2\")\n",
    "plt.plot(np.linspace(0, 10, 500), np.linspace(0, 10, 500)**(1/2), label=\"y = x^(1/2)\")\n",
    "plt.plot(np.linspace(0.00001, 10, 500), np.log(np.linspace(0.00001, 10, 500)), label=\"y = ln(x)\")\n",
    "plt.plot(np.linspace(-10, 10, 1000), np.exp(np.linspace(-10, 10, 1000)), label=\"y = e^x\")\n",
    "plt.legend()\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0329a32c-1ec7-4795-9ed0-aa002897c607",
   "metadata": {},
   "source": [
    "Of these basic functions I graphed, $\\ln(x)$ and its inverse $e^x$ seem to have the most potential as a basis for creating an approximated Heavisde step function."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "284ebf9c-96c7-465b-98fe-7a060b5a366d",
   "metadata": {},
   "source": [
    "$y=e^x$ is already bounded by $y=0$. I can rewrite $y=e^x$ as $y=\\frac{1}{e^{-x}}$ Looking at the behavior of this function, as the $x$ goes to infinity, the term $e^{-x}$ goes to $0$ and so the fraction $\\frac{1}{e^{-x}}$ approaches infinity. However, If I add a $+1$ to the denominator of the fraction, then, the function will approach $\\frac{1}{1}$ or just $1$ when $x\\to\\infty$."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "id": "aa888b4a-d165-4bda-ad8b-1cc130b9a524",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYcAAAD8CAYAAACcjGjIAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/YYfK9AAAACXBIWXMAAAsTAAALEwEAmpwYAAAh90lEQVR4nO3deZhU9Z3v8fe3N5q9W2jZN5UmLEJYVMwEbaOJSLhuo4YYlYgGE8ebOLn3zqPDjDHJzBOzjDG7Q5CgRtGYICEJhmikxVGRRVpWgUZRGhqQhgYa6KWqfvePOt0WfXop6FObfF7PU0+d5Vfn9+3Tp+vTZ6lT5pxDREQkVlaqCxARkfSjcBARER+Fg4iI+CgcRETER+EgIiI+CgcREfHpcDiY2SAzW25mm81sk5l9o4U2ZmY/NbNyM1tvZhM62q+IiCROTgDLCAH/xzn3lpl1B9aa2YvOuc0xba4ChnuPi4Bfec8iIpKGOrzn4JyrdM695Q0fBbYAA5o1uwZ4wkWtBArMrF9H+xYRkcQIYs+hiZkNBcYDbzabNQDYFTNe4U2rbPb62cBsgPz8/ImDBw8OsryEiEQiZGWl/6kb1RmcXbt24ZxD22dwMqHOTKgRYNu2bQecc0UdXU5g4WBm3YA/APc6546czjKcc3OBuQAjRoxwW7duDaq8hCktLaWkpCTVZbRLdQanpKSE6upqysrKUl1KuzJhfUJm1JkJNQKY2ftBLCeQGDSzXKLB8JRzblELTXYDg2LGB3rTREQkDQVxtZIBjwFbnHMPt9JsCXCbd9XSZOCwc66ylbYiIpJiQRxW+gfgVmCDmZV50/4VGAzgnHsUWApMA8qB48DtAfQrIiIJ0uFwcM79D2DttHHAP3W0L5EzWUNDAxUVFdTW1sb9mp49e7Jly5YEVhWMTKgz3WrMz89n4MCB5ObmJmT5gV6tJCKJU1FRQffu3Rk6dCjRo7ntO3r0KN27d09wZR2XCXWmU43OOaqqqqioqGDYsGEJ6SP9r8sSEQBqa2vp1atX3MEgH19mRq9evU5pL/JUKRxEMoiCQRoleltQOIiIiI/CQURSZsWKFUyYMIHCwkJ+//vft9n2q1/9Kq+99ppv+iOPPMITTzwBwHPPPcfo0aPJyspizZo1Cam5udLSUrp168add97pm1dZWcn06dNPaXkzZsxg+/btTeNXXHEFhw4d6nCdp0rhICIpM3jwYBYsWMCNN97YbtuVK1cyefLkk6aFQiHmz5/PzTffDMCYMWNYtGgRl1xySVz9L1iwgAcffPCU6260ceNG7r77blauXMnRo0f59re/fdL8hx9+mK985SuntMyvfe1r/OAHP2gav/XWW/nlL3952jWeLoWDiMTlgQce4JFHHmkanzNnDj/5yU86tMyhQ4cyduzYdu9ZtGXLFoqLi8nOzj5p+ssvv8yECRPIyYleeDly5EhGjBjRoZqa27FjB1OnTuWSSy5hypQpvPPOOwDs3r2bO+64g8WLFzNmzBiefvpptm7dyvz585te+4c//IGpU6f6lhkKhbjgggsoLS0F4P7772fOnDkATJkyhZdeeolQKATA1VdfzcKFCwP9meKhS1lFMtC3/7SJzXvav4VZOBz2vaG2ZlT/Hnzrf41udf6sWbO4/vrruffee4lEIjzzzDOsWrXK127KlCkcPXrUN/1HP/oRV1xxRVy1NPfCCy+0+Cb72muvMXHixNNaZrxmz57No48+St++fdm8eTN33303L7/8MgMGDODNNz+6x2h2djZPP/100/h7771HYWEhnTp18i0zJyeHBQsWcMMNN/Czn/2Mv/71r03LysrK4rzzzuPtt99m4sSJFBYWUldXR1VVFb169Uroz3pSjUnrSUQy2tChQ+nVqxfr1q1j3759jB8/vsU3q1dffTXwvpctW8ZvfvMb3/TKykpGjhx5Ssuqqqri8ssvB+DgwYPU19ezePFiAJ588knOP//8prY1NTW8/vrr3HjjjU13Za2rq4urn8rKSoqKWr856ujRo7n11luZPn06b7zxBnl5eU3zzj77bPbs2dMUfI3jCgcRaVNb/+HHCvqDW3feeScLFixg7969zJo1q8U2Qe85HD9+nOrqavr37++b17lz51O+1r9Xr15Nd9RdsGABO3fubPW8QyQSoaCggLKyslNel81ru/3221m3bh39+/dn6dKlAGzYsIGCggL2799/0mtra2vp3Llzq+PJoHAQkbhdd911PPDAAzQ0NJx0CCVW0HsOy5cv57LLLmtx3siRIykvLw+0v1g9evRg2LBhPPfcc0ydOhXnHOvXr2fcuHHtvra4uJidO3c2jTff81m0aBEHDx5kxYoVTJ8+nVWrVlFQUADAtm3bGDNmDBD9NPTevXsZOnRoUD9WXHRCWkTilpeXx2WXXcZNN90U97mMtqxevZqBAweyePFi7rrrLkaP9u8RtXa+AeCqq65ixYoVTePPP/88AwcO5I033uDzn/88V155ZYdrfOqpp3jsscf41Kc+xejRo/njH/8Y1+u6du3Kueee22J4HThwgPvuu4958+ZRXFzMPffcwze+8Q0A9u3bR+fOnenbty8Aa9euZfLkyU0n3ZPGOZeWj+LiYpcJli9fnuoS4qI6g3PppZe6cePGJb3fzZs3n/Jrjhw5EmgN4XDYjRs3zm3bti3Q5bZV5/jx4119fX2r86+99trA62nJ6azLRYsWuTlz5pzSax5++GE3b968pvGvf/3r7qWXXmqxbUvbBLDGBfAerD0HEYnL5s2bOe+887j88ssZPnx40vp966232rzz6EMPPURlZXp+Pcx11113yoeDCgoKmDlzZtP4mDFjmk6gJ5POOYhIXEaNGsW7776b6jJ8RowYEfhnG4LU0ien23L77Sd/3c2pfoguKNpzEMkg0aMGIonfFhQOIhkiPz+fqqoqBYQ0fZ9Dfn5+wvrQYSWRDDFw4EAqKir48MMP435NbW1tQt9AgpIJdaZbjY3fBJcoCgeRDJGbm3vK3/pVWlrK+PHjE1RRcDKhzkyoMUg6rCQiIj4KBxER8VE4iIiIj8JBRER8FA4iIuKjcBARER+Fg4iI+CgcRETEJ5BwMLP5ZrbfzDa2Mr/EzA6bWZn3eCCIfkVEJDGC+oT0AuDnwBNttHnVOTc9oP5ERCSBAtlzcM6tAA4GsSwREUm9ZJ5zuNjM3jazF8wsvm9HFxGRlEjWjffeAoY452rMbBqwGPB9lZSZzQZmAxQVFVFaWpqk8k5fTU2N6gxQJtRZXV1NOBxO+zohM9YnZEadmVBjoIL4rlHv/vJDgY1xtt0J9G6rjb5DOliqMzip+g7p05EJ69O5zKgzE2p0LsO+Q9rM+pqZecMXEj2cVZWMvkVE5NQFcljJzBYCJUBvM6sAvgXkAjjnHgVuAL5mZiHgBDDDSzgREUlDgYSDc+6L7cz/OdFLXUVEJAPoE9IiIuKjcBARER+Fg4iI+CgcRETER+EgIiI+CgcREfFROIiIiI/CQUREfBQOIiLio3AQEREfhYOIiPgoHERExEfhICIiPgoHERHxUTiIiIiPwkFERHwUDiIi4qNwEBERH4WDiIj4KBxERMRH4SAiIj4KBxER8VE4iIiIj8JBRER8FA4iIuKjcBARER+Fg4iI+CgcRETEJ5BwMLP5ZrbfzDa2Mt/M7KdmVm5m681sQhD9iohIYgS157AAmNrG/KuA4d5jNvCrgPoVEZEEyAliIc65FWY2tI0m1wBPOOccsNLMCsysn3OuMoj+RST9hMIRjtWHOVEf5lh9iON13nN9iNqGCPWhCPXh6POWnQ1sfWUHDeHG6Y76UKRpPBRxRJwj7D03DocjnDS9aX4Ews2mOwcOiL4NRUWnuZjhxuneVG+CA44fP07+6uUtv+6jRX702ph2sfNbE0eTuJYTlEDCIQ4DgF0x4xXetJPCwcxmE92zoKioiNLS0iSVd/pqampUZ4Ayoc7q6mrC4XDa1wnBrc9QxHG4znGoznGoNjp8rMFR4z2O1RN99h61YQhFTrGTd95pGswxyMlqfBjZBlnewyx6yCM6bE3TP5pGzLTo/BxvvFHjoFnMuLUwLeY51DVCbk5d0zgG5o21uLw2prUmnjbtNVobzzLikKxwiItzbi4wF2DEiBGupKQktQXFobS0FNUZnEyos6CggOrq6rSvE+Jfn845qo7Vs/PAMd47cIydVcfYeeA4Hxw8zt4jtRyoqWvxv9bu+TkUdMmjsEseAwpzKeySR8/OuXTtlEOXvGy65GXHDOfQNS+bLp1yyM/NIi87i7yc6POqlW9w2aVTyMvJIifLMIvrbTKpMmHbBHj8nmCWk6xw2A0Mihkf6E0TkSSLRBw7Pqxh054jbK48wqY9h9m05wjVxxua2mRnGYMKOzO4V1dG9etBn5759O2RT9+enejTI5+zu+dT0CWX3OxgTlt2yzO6dkqr/1XPeMn6bSwB7jGzZ4CLgMM63yCSHA3hCBt3H2bVewdZvfMgq947yJHaEAB5OVl8om93po7uS3Gf7gzr3ZWhvbsysLBzYG/8kpkCCQczWwiUAL3NrAL4FpAL4Jx7FFgKTAPKgePA7UH0KyItqz5ez/Kt+1lYVsv/Xv4iR+uiYXBO7658fmw/Jg45izEDenBuUTeFgLQoqKuVvtjOfAf8UxB9iUjLjtY28MKGvTy/bjerdh4kHHH0yDOmjR3IlOLeXDjsLM7unp/qMiVD6CCfSAZzzvH6jioWrvqAFzfvoy4UYVjvrtxdci6Xj+zDwfJ1fOaysakuUzKQwkEkA52oD/P8ut0seP09tu2roaBLLjdNGsT1EwbwyUEFTVf7lO5Iv6t+JDMoHEQyyLG6EE+88T5zV+zg0PEGRvfvwY9uHMf0sf3Iz81OdXnyMaJwEMkAJ+rDLHh9Z1MolIwo4u6S87hgaGFafiZAMp/CQSSNOef48/pKvrd0C3sO11IyoohvXD6c8YMLU12afMwpHETS1JbKI3zrj5tYtfMgI/v14OEvfJLJ5/RKdVlyhlA4iKSZ+lCEny8v55fLy+nROZfvXX8+N00aRHaWDh9J8igcRNLI+opq/t9z69m67yjXjx/Av08fRWHXvFSXJWcghYNIGohEHHNffZcfLdtK726dmP/lSXzmE31SXZacwRQOIilWVVPHN3/3Nq9s+5Bp5/fle9ePpWfn3FSXJWc4hYNICq19/xB3P7WWQ8cb+I9rx/Cliwbr0lRJCwoHkRT5/doK/nXRBvoV5LP47gsZ1b9HqksSaaJwEEmycMTx/b++w9wV7/Kpc3vxyy9NoKCLTjpLelE4iCTR8foQ9zy9jpff2c9tFw/h36eP0i2zJS0pHESS5NCxem5fsJr1FdV895rR3Hrx0FSXJNIqhYNIEuypPsFt81fxwcHj/OqWiVw5um+qSxJpk8JBJMHK9x/l1sdWUVMX4slZF3KRboEhGUDhIJJA7+w9ws2/fpPsLON3d13MyH66Ikkyg8JBJEE27znCl+atpFNONgtnT2ZY766pLkkkbrpMQiQBNu4+zM3zVpKfm80zCgbJQNpzEAnYhorD3PLYm3TrlMPCr0xmcK8uqS5J5JRpz0EkQI2Hkrp1yuGZ2QoGyVwKB5GAvHfgGLfNf5OuXjAMOkvBIJlL4SASgN3VJ7hl3ps4B0/ecZGCQTKewkGkgw7U1HHrvDc5cqKBx2ddyHlnd0t1SSIdphPSIh1w+EQDtz22ij2HT/DkHRcxZkDPVJckEgjtOYicpuP1IWYtWM32/Uf571snccHQs1JdkkhgAgkHM5tqZlvNrNzM7mth/pfN7EMzK/MedwbRr0iqOMvmrifXsu6DQ/xkxnguLS5KdUkigerwYSUzywZ+AXwWqABWm9kS59zmZk2fdc7d09H+RFLNYRwfdxOvbj/AD24Yy7Tz+6W6JJHABbHncCFQ7px71zlXDzwDXBPAckXSTiTiqDrnShr6juGB6aO4adKgVJckkhBBnJAeAOyKGa8ALmqh3T+a2SXANuCfnXO7mjcws9nAbICioiJKS0sDKC+xampqVGeA0rlO5xxPv1NPzdnnk7ftRc4JdaW09P1Ul9WmdF6fsTKhzkyoMUjJulrpT8BC51ydmd0FPA58pnkj59xcYC7AiBEjXElJSZLKO32lpaWozuCkc50/fnEbL76/nR6Va7B3SykpeSTVJbUrnddnrEyoMxNqDFIQh5V2A7H71gO9aU2cc1XOuTpvdB4wMYB+RZJm3qvv8pO/b+emSQMpfH85luqCRBIsiHBYDQw3s2FmlgfMAJbENjCz2DN2VwNbAuhXJCl+t3oX//GXLUw7vy/fu36sgkHOCB0+rOScC5nZPcAyIBuY75zbZGbfAdY455YAXzezq4EQcBD4ckf7FUmGv6yv5L5F67mkuIgff+GTZGcpGuTMEMg5B+fcUmBps2kPxAzfD9wfRF8iyVK6dT/3PruOiUMK+e9bJtIpJzvVJYkkjT4hLdKC1TsP8tXfrqW4T3ce+/IFdM5TMMiZReEg0sza9w9x+29W07+gM4/PupAe+bmpLkkk6RQOIjHe+uAQM+evoqh7J56+czK9u3VKdUkiKaFwEPGs++AQMx9bRe9ueSz8ymT69sxPdUkiKaNwEAHKdlVz22OrOKtbHgtnKxhE9H0OcsZb+/5Bvvyb1RR2je4x9OvZOdUliaSc9hzkjPbKtg/50rw36d2tEwtnT6Z/gYJBBLTnIGewv6yv5N5n1zH87O48cceFOvksEkPhIGekhas+YM7zG5g4pJB5My+gZ2ddrioSS+EgZ5RIxPHDv23lV6U7KBlRxK++NFEfcBNpgcJBzhi1DWG++bsylm7Yy80XDeY7V48mJ1un3URaonCQM8Lew7Xc9du1rK+o5t8+P5I7Pj0MM91ET6Q1Cgf52Hu9/ABff2Ydx+vDPHrLRK4c3TfVJYmkPYWDfGxFIo5HV+zgR8u2Mqx3V56ZPZnzzu6e6rJEMoLCQT6W9h2p5V9+v55Xtn3I9LH9eOgfx9KtkzZ3kXjpr0U+dv709h7+bfFG6kJhvnvNaG6ZPETnF0ROkcJBPjb2H6nlO3/ezJ/XVzJuUAE/vmkc5xR1S3VZIhlJ4SAZLxSO8NuV7/Nff9tGXSjCNz9bzN0l5+oyVZEOUDhIRnu9/AD/uXQLm/YcYcrw3nznmjEM69011WWJZDyFg2Skt3dV88NlW/mf8gP075nPL26ewLTz++rcgkhAFA6SMZxzrN55iLkrdvDSlv2c1TWPf/v8SG6ZPIT8XN0CQyRICgdJe/WhCC9u3sevX32Xsl3VFHbJ5Z+vKGbWp4fSXd/vLJIQCgdJW+X7j/Ls6l0sems3VcfqGXxWF757zWhumDhIN8sTSTCFg6SVvcci/GJ5OX/duJcNuw+Tk2VcMbIPX7hgEJcUF5GdpXMKIsmgcJCUqm0Is+q9g7xWfoDSrR+ydd8JYCvjBhUwZ9pIrh0/gKLu+hIekWRTOEhSHaipo+yDat6uqGbNzkOs/eAQ9aEIudnGxCGF3PyJPO659tP6uk6RFFM4SELUNoR578Axtu+vYfu+o2zfV8PGPYepOHQCgOws4xN9uzPz4iF8engRFwwtpEteDqWlpQoGkTQQSDiY2VTgJ0A2MM8591Cz+Z2AJ4CJQBXwBefcziD6luRzznHkRIgDx+rYf6SO3dUnqDh0nIpDJ9h96AQV1cfZfegEERdtn51lDOnVhbEDe3LbxUP45KBCzh/QUyeVRdJYh8PBzLKBXwCfBSqA1Wa2xDm3OabZHcAh59x5ZjYD+D7whY72LafGOUco4qgLRahrCFMXilDbEOZEQ5ia2hA1dSGO1oY4WhfiaG0DNbXeeG0DVcfqqaqpp+pYHVU19YQa3/k9ZnB2904MLOzChMGFXPvJAQzv053iPt0Y1rsrnXIUBCKZxJxz7bdqawFmFwMPOueu9MbvB3DOfS+mzTKvzRtmlgPsBYpcG53nF5ztRk69tXEJJz3FDLQ43zWf3kq71ua7Vuf7l1NbW0d+fv5Jc1289TaNfjTuMLAsnGWBWfSZxmFv3kltsoCYeWZAFi4rG5eVG322HCKWDdk5Xvs4RcJkhevICteTFTpOdsNHj6ym4WPk1B0hp/4o5sLxL7sV1dXVFBQUdHg5iVRWVkYoFGLSpEmpLqVdmbA+ITPqzIQaAV555ZW1zrkOb5xBHFYaAOyKGa8ALmqtjXMuZGaHgV7AgdhGZjYbmA2Q1/c8Dg39TADlBcRFmo1/FEF10Qmtzid2vms23my+RSLR4UgEiICLYC76jHPNxiPRds2nOYdFGrBwiKxICCIhXKieLBfGvHELe8+RBgiHsFAtFqrznqPDREKtxqMDQt6jrp1VdyrC4TDV1dUBLjF4oVAI51za1wmZsT4hM+rMhBqDlFYnpJ1zc4G5AMOLR7g13/pc0z/VjW9SjffO+Wi8cb6dNE4789tdXpz36CktLaWkpCSutqmkOoNTUlJCdXU1ZWVlqS6lXZmwPiEz6syEGiH+9672BBEOu4FBMeMDvWkttanwDiv1JHpiulVZBj0769YIIiKpEMQN71cDw81smJnlATOAJc3aLAFmesM3AC+3db5BRERSq8N7Dt45hHuAZUQvZZ3vnNtkZt8B1jjnlgCPAU+aWTlwkGiAiIhImgrknINzbimwtNm0B2KGa4Ebg+hLREQST9+jKCIiPgoHERHxUTiIiIiPwkFERHwUDiIi4qNwEBERH4WDiIj4KBxERMRH4SAiIj4KBxER8VE4iIiIj8JBRER8FA4iIuKjcBARER+Fg4iI+CgcRETER+EgIiI+CgcREfFROIiIiI/CQUREfBQOIiLio3AQEREfhYOIiPgoHERExEfhICIiPgoHERHxUTiIiIhPh8LBzM4ysxfNbLv3XNhKu7CZlXmPJR3pU0REEq+jew73AX93zg0H/u6Nt+SEc+6T3uPqDvYpIiIJ1tFwuAZ43Bt+HLi2g8sTEZE00NFw6OOcq/SG9wJ9WmmXb2ZrzGylmV3bwT5FRCTBctprYGYvAX1bmDUndsQ558zMtbKYIc653WZ2DvCymW1wzu1ooa/ZwGyAoqIiSktL2ysv5WpqalRngDKhzurqasLhcNrXCZmxPiEz6syEGgPlnDvtB7AV6OcN9wO2xvGaBcAN7bUrLi52mWD58uWpLiEuqjM4l156qRs3blyqy4hLJqxP5zKjzkyo0TnngDWuA+/rjY+OHlZaAsz0hmcCf2zewMwKzayTN9wb+Adgcwf7FRGRBOpoODwEfNbMtgNXeOOY2SQzm+e1GQmsMbO3geXAQ845hYOISBpr95xDW5xzVcDlLUxfA9zpDb8OnN+RfkREJLn0CWkREfFROIiIiI/CQUREfBQOIiLio3AQEREfhYOIiPgoHERExEfhICIiPgoHERHxUTiIiIiPwkFERHwUDiIi4qNwEBERH4WDiIj4KBxERMRH4SAiIj4KBxER8VE4iIiIj8JBRER8FA4iIuKjcBARER+Fg4iI+CgcRETER+EgIiI+CgcREfFROIiIiI/CQUREfBQOIiLi06FwMLMbzWyTmUXMbFIb7aaa2VYzKzez+zrSp4iIJF5H9xw2AtcDK1prYGbZwC+Aq4BRwBfNbFQH+xURkQTK6ciLnXNbAMysrWYXAuXOuXe9ts8A1wCbO9K3iIgkTofCIU4DgF0x4xXARS01NLPZwGxvtM7MNia4tiD0Bg6kuog4qM5g9TazjKiTDFmfpH+dmVAjwIggFtJuOJjZS0DfFmbNcc79MYgiGjnn5gJzvX7XOOdaPY+RLlRnsFRnsFRncDKhRojWGcRy2g0H59wVHexjNzAoZnygN01ERNJUMi5lXQ0MN7NhZpYHzACWJKFfERE5TR29lPU6M6sALgb+YmbLvOn9zWwpgHMuBNwDLAO2AL9zzm2KY/FzO1JbEqnOYKnOYKnO4GRCjRBQneacC2I5IiLyMaJPSIuIiI/CQUREfFIaDm3dfsPM7vdut7HVzK5s5fXDzOxNr92z3gnvRNf8rJmVeY+dZlbWSrudZrbBaxfIpWWnwsweNLPdMbVOa6VdSm9tYmY/NLN3zGy9mT1vZgWttEvJ+mxv/ZhZJ2+bKPe2xaHJqs3rf5CZLTezzd7f0jdaaFNiZodjtoUHklljTB1t/g4t6qfeulxvZhNSUOOImPVUZmZHzOzeZm1Ssj7NbL6Z7Y/9/JeZnWVmL5rZdu+5sJXXzvTabDezmXF16JxL2QMYSfQDG6XApJjpo4C3gU7AMGAHkN3C638HzPCGHwW+luT6/wt4oJV5O4HeKVy3DwL/t5022d66PQfI89b5qCTX+Tkgxxv+PvD9dFmf8awf4G7gUW94BvBskmvsB0zwhrsD21qosQT4czLrOp3fITANeAEwYDLwZorrzQb2AkPSYX0ClwATgI0x034A3OcN39fS3w9wFvCu91zoDRe2119K9xycc1ucc1tbmHUN8Ixzrs459x5QTvQ2HE0ses+OzwC/9yY9DlybwHJP4vV/E7AwWX0mQNOtTZxz9UDjrU2Sxjn3Nxe9og1gJdHPwaSLeNbPNUS3PYhui5dbO/eTCZJzrtI595Y3fJToFYEDktV/wK4BnnBRK4ECM+uXwnouB3Y4595PYQ1NnHMrgIPNJsduf629B14JvOicO+icOwS8CExtr790PefQ0i03mm/wvYDqmDeWltok0hRgn3NueyvzHfA3M1vr3RYkFe7xds/nt7K7Gc96TqZZRP9zbEkq1mc866epjbctHia6bSadd0hrPPBmC7MvNrO3zewFMxud3MqatPc7TLftcQat//OXDusToI9zrtIb3gv0aaHNaa3XhN9byZJ4+42gxFnzF2l7r+HTzrndZnY28KKZveMlf1LqBH4FfJfoH+R3iR4CmxVk//GKZ32a2RwgBDzVymISvj4zmZl1A/4A3OucO9Js9ltED43UeOeeFgPDk1wiZNDv0Dt/eTVwfwuz02V9nsQ558wssM8mJDwc3OndfiOeW25UEd3tzPH+Ywvsthzt1WxmOURvVT6xjWXs9p73m9nzRA9RBPqHEO+6NbNfA39uYVZSbm0Sx/r8MjAduNx5B0lbWEbC12cL4lk/jW0qvO2iJ9FtM2nMLJdoMDzlnFvUfH5sWDjnlprZL82st3MuqTeRi+N3mE632rkKeMs5t6/5jHRZn559ZtbPOVfpHYLb30Kb3UTPkzQaSPQ8b5vS9bDSEmCGdyXIMKKpvCq2gfcmshy4wZs0E0jWnsgVwDvOuYqWZppZVzPr3jhM9KRrUu8w2+xY7XWt9J/yW5uY2VTgX4CrnXPHW2mTqvUZz/pZQnTbg+i2+HJrAZcI3vmNx4AtzrmHW2nTt/E8iJldSPTvPtkBFs/vcAlwm3fV0mTgcMwhk2Rr9chAOqzPGLHbX2vvgcuAz5lZoXd4+XPetLYl+4x7s7Po1xE9/lUH7AOWxcybQ/RKka3AVTHTlwL9veFziIZGOfAc0ClJdS8AvtpsWn9gaUxdb3uPTUQPnyR73T4JbADWextQv+Z1euPTiF7hsiNFdZYTPR5a5j0ebV5nKtdnS+sH+A7RMAPI97a9cm9bPCfJ6+/TRA8dro9Zh9OArzZuo0RvX7PJW38rgU+l4Pfc4u+wWZ1G9IvBdnjb7qRk1+nV0ZXom33PmGkpX59Ew6oSaPDeN+8gen7r78B24CXgLK/tJGBezGtnedtoOXB7PP3p9hkiIuKTroeVREQkhRQOIiLio3AQEREfhYOIiPgoHERExEfhICIiPgoHERHx+f/B1CYDKxThJgAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.grid()\n",
    "plt.axhline(0, color=\"black\")\n",
    "plt.axvline(0, color=\"black\")\n",
    "plt.xlim([-10, 10])\n",
    "plt.ylim([-1, 2])\n",
    "x = np.linspace(-10, 10, 1000)\n",
    "y = 1 / (1 + math.e**(-1 * x))\n",
    "plt.plot(np.linspace(-10, 10, 1000), y, label=\"y = 1 / (1 + e^(-x))\")\n",
    "plt.legend()\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f403b524-049d-4a71-be4d-1d1166955b78",
   "metadata": {},
   "source": [
    "This is a type of **sigmoid function** known as a **logistic function**. If I instead use a really big number for the base of the exponent rather than $e$, I can get a pretty good approximation for the Heaviside step function:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "id": "e82e1303-e8b3-4b8d-a3f9-8467a558061a",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYcAAAD8CAYAAACcjGjIAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/YYfK9AAAACXBIWXMAAAsTAAALEwEAmpwYAAAdxElEQVR4nO3de5CU9b3n8fd3blyih4ERo1wM4MAseGFRUbM56Cgeb+UB9RAlVatEQzBZTY5bm9roUqVujlUxuzlZTY6JxRpFUxFyVA6SBCVemGDcKBAdREBuwgkzGZQMDpfAzNAz3/2jHybDPN0z3dNP34bPq6qLfp7n18/vy6+fmc88l37a3B0REZHuSvJdgIiIFB6Fg4iIhCgcREQkROEgIiIhCgcREQlROIiISEjG4WBmY81stZltNrNNZvaPCdqYmf3QzHaY2ftmdkGm/YqISPaURbCOGPDf3P1dMzsV+IOZverum7u1uQ6YGDwuAX4S/CsiIgUo4z0Hd29y93eD54eALcDoHs1mA8963NtApZmdmWnfIiKSHVHsOXQxs3HANOCdHotGA3u6TTcE85p6vH4BsABg8ODBF5511llRlpcVnZ2dlJQU/qkb1RmdPXv24O5o+4xOMdRZDDUCbNu27c/uPjLT9UQWDmZ2CvAicK+7H+zPOtx9EbAIoKamxrdu3RpVeVlTV1dHbW1tvsvok+qMTm1tLS0tLdTX1+e7lD4Vw3hCcdRZDDUCmNm/R7GeSGLQzMqJB8PP3X1ZgiaNwNhu02OCeSIiUoCiuFrJgJ8CW9z9B0marQBuD65auhQ44O5NSdqKiEieRXFY6QvAbcBGM6sP5v0P4CwAd38CWAlcD+wAjgB3RNCviIhkScbh4O6/A6yPNg7cnWlfcvI4duwYDQ0NtLa25ruUkAcffBB3Z8uWLfkupU/Dhg1TnREptBoHDx7MmDFjKC8vz8r6I71aSSQqDQ0NnHrqqYwbN474kcvCUVJSQiwWY/LkyfkupU+HDh3i1FNPzXcZfSqGOgupRnenubmZhoYGxo8fn5U+Cv+6LDkptba2UlVVVXDBIFIIzIyqqqqs7lkrHKRgKRhEksv2z4fCQUREQhQOIjmyZs0aLrjgAsrKynjhhRd6bfu1r32Nt956KzT/0Ucf5dlnnwXg+eef55xzzqGkpIT169dHUmNv6/zud79LdXU1NTU1rFq1qmv+K6+8Qk1NDdXV1TzyyCNd83ft2sUll1xCdXU1t956K+3t7QC0tbVx6623Ul1dzSWXXMLu3buLoo+pU6ee0MdxdXV1nHLKKcyfPz80nk1NTdxwww29jnlPc+fOZfv27V3TV111FZ9++mla64iEuxfkY9KkSV4MVq9ene8SUlJsdW7evDm/hfTiww8/9A8++CDt1+3atcs3bNjgt912mz///PO9tp06darHYrET5h07dszPO+88P3bsmLvHx+jDDz/0yy+/3NetW5dwPQcPHux6/vTTT/uDDz7Ya7/J1rlp0yY///zzvbW11T/66COfMGGCx2Ixj8ViPmHCBN+5c6e3tbX5+eef75s2bXJ39y9+8Yu+ZMkSd3e/6667/Mc//rG7uz/++ON+1113ubv7kiVL/JZbbvGDBw9mvY9M/x8HDx48oQ93940bN/rkyZN948aNfsstt/hDDz10wnh+61vf8uXLl/c65j3V1dX5/Pnzu6YXL17sDz/8cNL3qydgvUfwO1h7DiIJPPDAAzz66KNd0wsXLuSxxx7LaJ3jxo3j/PPP7/P+PFu2bGHSpEmUlpaeMP+NN97o2vMAmDx5MjU1NRnV1FOydb700kvMnTuXQYMGMX78eKqrq1m7di1r166lurqaCRMmUFFRwdy5c3nppZdwd9544w3mzJkDwLx581i+fHnXuubNmwfAnDlzeP3113H3ouujsbGRr3zlKyxfvpxzzz2X5557jq1bt/LUU091jduLL77ItddeGxrPWCzG9OnTqaurA+D+++9n4cKFAMyYMYPXXnuNWCwGwKxZs1iyZEl/39J+06WsUvD+5y83sflP/bpdV1JTRv0ND/79OUmX33nnndx8883ce++9dHZ2snTpUtauXRtqN2PGDA4dOhSa//3vf5+rrrqqX7W9/PLLCX+hvPXWW1x44YX9WmemGhsbufTSS7umx4wZQ2Nj/A44Y8eOPWH+O++8Q3NzM5WVlV1B1r19Y2Nj12vKysoYNmwY+/fvz3ofzc3NkfYxevRo3nnnr/cYLS0t5bnnnuua3rVrF8OHD2fQoEGh8SwrK2Px4sXMmTOHH/3oR7zyyitd6yopKaG6upoNGzZw4YUXMnz4cNra2mhubqaqqirJOxQ9hYNIAuPGjaOqqor33nuPjz/+mGnTpiX8wXzzzTcj73vVqlU8/fTToflNTU1pf7aiubmZGTNmALB//37a29u7/vL92c9+xnnnnZdxvZJYU1MTI0cmvznqOeecw2233cYNN9zA73//eyoqKrqWnX766fzpT3/q+mPg+LTCQaSb3v7Cz6b58+ezePFi9u7dy5133pmwTdR7DkeOHKGlpYVRo0aFlg0ZMiTt69qrqqq67h67ePFidu/ezUMPPZR2XaNHj2bPnr/edb+hoYHRo+Nf25JoflVVFS0tLcRiMcrKyk5of3xdY8aMIRaLceDAAUaMGJH1PqqqqjLqo2f7vvR8v+644w7ee+89Ro0axcqVKwHYuHEjlZWVfPLJJye8trW1lSFDhiSdzgWdcxBJ4qabbuKVV15h3bp1XHPNNQnbvPnmm9TX14ce/T2ktHr1aq644oqEyyZPnsyOHTv6td5MzZo1i6VLl9LW1sauXbvYvn07F198MdOnT2f79u3s2rWL9vZ2li5dyqxZszAzrrjiiq6rsp555hlmz57dta5nnnkGgBdeeIErr7wSMyu6PvoyadKkE66Sevrpp6mvr+8KhmXLlrF//37WrFnDN77xDVpaWrrabtu2jXPPPReIXzS0d+9exo0b1493LgNRnNXOxkNXK0Wr2OoslKuV7rrrLv/2t799wrz+Xq20du1aHz16tA8dOtRHjBjhU6ZMCbW5++67k75Xu3fv9hkzZnRNL1u2zEePHu0VFRV++umn+9VXXx16TbpXK/W2zocfftgnTJjgkyZN8pUrV3bN//Wvf+0TJ070CRMmnHBVzc6dO3369Ol+9tln+5w5c7y1tdXd3Y8ePepz5szxs88+26dPn+47d+7sqjObfWT6/xg/fvwJfaTiyiuv9O3bt4fm79u3zydOnOh//OMf3d39scce89tvv93d3ffu3evTp0/vartu3Tq/+eabE64/m1cr5T0Ekj0UDtEqtjoLIRw6Ojp86tSpvm3bthPm9zccUjFt2jRvb29PuvzGG28M1dOb7uFQyIqhzv7UuGzZMl+4cGFar/nBD37gTz75ZNf0N7/5TX/ttdcSttWlrCI5tnnzZqqrq5k5cyYTJ07MWb/vvvtur3fZfOSRR2hq0lehFIubbrop7cNBlZWVXZfhApx77rnMnDkz4sr6phPSIglMmTKFjz76KN9lhNTU1ET+2QbJrkSfnO7NHXec+HU3X/3qV6MsJ2Xac5CCFd9DFpFEsv3zoXCQgjR48GCam5sVECIJuMe/z2Hw4MFZ60OHlaQgjRkzhoaGBvbt25fvUkL27t0bP2HXx20wCkFra2tWf4FEpRjqLLQaj38TXLYoHKQglZeXZ+0brjL19a9/nZaWlq4PlxWyuro6pk2blu8y+lQMdRZDjVEq/D99REQk5xQOIiISonAQEZEQhYOIiIQoHEREJEThICIiIQoHEREJUTiIiEhIJOFgZk+Z2Sdm9kGS5bVmdsDM6oPHA1H0KyIi2RHVJ6QXA/8CPNtLmzfd/YaI+hMRkSyKZM/B3dcA+6NYl4iI5F8uzzl83sw2mNnLZpafb4wXEZGU5OrGe+8Cn3P3w2Z2PbAcCH29lpktABYAjBw5krq6uhyV13+HDx9WnREqhjpbWlro6Ogo+DqhOMYTiqPOYqgxUlF812hwz/1xwAcptt0NnNZbG32HdLRUZ3Quv/xynzp1ar7LSEkxjKd7cdRZDDW6F9l3SJvZGWZmwfOLiR/Oas5F3yIikr5IDiuZ2RKgFjjNzBqAB4FyAHd/ApgDfN3MYsBRYG6QcCIiUoAiCQd3/1Ify/+F+KWuIiJSBPQJaRERCVE4iIhIiMJBRERCFA4iIhKicBARkRCFg4iIhCgcREQkROEgIiIhCgcREQlROIiISIjCQUREQhQOIiISonAQEZEQhYOIiIQoHEREJEThICIiIQoHEREJUTiIiEiIwkFEREIUDiIiEqJwEBGREIWDiIiEKBxERCRE4SAiIiEKBxERCVE4iIhIiMJBRERCFA4iIhISSTiY2VNm9omZfZBkuZnZD81sh5m9b2YXRNGviIhkR1R7DouBa3tZfh0wMXgsAH4SUb8iIpIFZVGsxN3XmNm4XprMBp51dwfeNrNKMzvT3Zui6F8kX/bsP8I//2Yrm5sOcuDoMTo9Pt/9eAtP9tKcaG9vp+J3r+a1hlQUQ53FUGOUIgmHFIwG9nSbbgjmnRAOZraA+J4FI0eOpK6uLkfl9d/hw4dVZ4SKoc6WlhY6Ojr49aurWfi7o7TGnMlVpXz2VKPEwI43tBP+yYtjx5zy8s48VpCaYqizGGoE+ENE68lVOKTE3RcBiwBqamq8trY2vwWloK6uDtUZnWKos7KykpaWFhoHnUVL24e8dPcXmDq2Mt9lJVQM4wnFUWcx1Aiw+J5o1pOrq5UagbHdpscE80SK1mtbPuGcUX9TsMEgkolchcMK4PbgqqVLgQM63yDFzDE+aDzA9HEj8l2KSFZEcljJzJYAtcBpZtYAPAiUA7j7E8BK4HpgB3AEuCOKfkXypXPIMI60d/Afzjg136WIZEVUVyt9qY/lDtwdRV8ihaBzSHyP4ayqoXmuRCQ79AlpkX7oHDIcgLHDFQ4yMCkcRPrBK+KhUHVKRZ4rEckOhYNIP3j5UCrKShhSXprvUkSyQuEg0g9ePpThQ8sxy+dH3ESyR+Eg0g9eMZTKITqkJAOXwkGkHzrLh1A5tDzfZYhkjcJBpB/ih5W05yADl8JBpB+8fKj2HGRAUziIpMkBrxhCpfYcZABTOIikyUvKoaRMew4yoCkcRNLUWRrfYzhlUEHd8V4kUgoHkTR5SXyPQR+Ak4FM4SCSJi+J7zEMqVA4yMClcBBJk5dqz0EGPoWDSJo6jx9W0p6DDGAKB5E0dR1W0p6DDGAKB5E0ufYc5CSgcBBJU2ep9hxk4FM4iKTJS+Kfc9CegwxkCgeRNOmcg5wMFA4iaTp+zmGwwkEGMIWDSJo6S8ug4xilJfoWOBm4FA4iafKScqzjWL7LEMkqhYNImrykHDra812GSFYpHETS1FlShnVqz0EGNoWDSJp0WElOBgoHkTR5qQ4rycAXSTiY2bVmttXMdpjZfQmWf9nM9plZffCYH0W/IvmgPQc5GWT8VVZmVgo8Dvwd0ACsM7MV7r65R9NfuPs9mfYnkm+dJfFLWUUGsij2HC4Gdrj7R+7eDiwFZkewXpGCpD0HORlE8SW4o4E93aYbgEsStPsHM7sM2Ab8V3ff07OBmS0AFgCMHDmSurq6CMrLrsOHD6vOCBVDnR1WSlmsreDrhOIYTyiOOouhxijl6hvSfwkscfc2M7sLeAa4smcjd18ELAKoqanx2traHJXXf3V1dajO6BRFnb96EeuMFX6dFMl4Uhx1FkONUYrisFIjMLbb9JhgXhd3b3b3tmDySeDCCPoVyQsvKdfnHGTAiyIc1gETzWy8mVUAc4EV3RuY2ZndJmcBWyLoVyTnjnV0QkmpzjnIgJfxYSV3j5nZPcAqoBR4yt03mdl3gPXuvgL4ppnNAmLAfuDLmfYrkg9Hj3XEn+hzDjLARXLOwd1XAit7zHug2/P7gfuj6Eskn1rb4+GgPQcZ6PQJaZE0HN9zMO05yACncBBJw18PK2nPQQY2hYNIGo7qsJKcJBQOImn4azjosJIMbAoHkTTosJKcLBQOImnoOiGtD8HJAKdwEEnD8cNK2nOQgU7hIJKGIzrnICcJhYNIGg63xQCwWFsfLUWKm8JBJA2H22LQ2QGdsXyXIpJVCgeRNBxujVHS0Y7luxCRLFM4iKThL20xSjp0SEkGPoWDSBoOtcV0MlpOCgoHkTTE9xwUDjLwKRxE0nBY4SAnCYWDSBoOt8YwnXOQk4DCQSQN2nOQk4XCQSQNCgc5WSgcRFLU0ekcae9QOMhJQeEgkqK/tAe3zlA4yElA4SCSok//Eg+F0tjRPFcikn0KB5EU/flw/CqlkmN/yXMlItmncBBJ0b5DwZ7DsSN5rkQk+xQOIinaF+w5lGrPQU4CCgeRFDV8eoSK0hKFg5wUFA4iKfpj8xHGjBii23XLSUHhIJKirR8fYsJpp+S7DJGciCQczOxaM9tqZjvM7L4EyweZ2S+C5e+Y2bgo+hXJlZYj7Xy07y9MHTMs36WI5ETG4WBmpcDjwHXAFOBLZjalR7OvAJ+6ezXwf4DvZdqvSC798v0mAC6bNDLPlYjkhrl7Zisw+zzwkLtfE0zfD+Du3+3WZlXQ5vdmVgbsBUZ6L50PqTzdJ1/7n5P26xkf+e3j9X2uPt7g6NGjDBkyJC819P3O/XUFra1HGTw43TpTGOMUxylVra2tDB48OOXXe8T9J9JZNoTDI8+h4sg+ztj0HBvq64nFYlx00UUZrzvbWlpaqKyszHcZfSqGOouhRoDf/va3f3D3jDfOsghqGQ3s6TbdAFySrI27x8zsAFAF/Ll7IzNbACwAqDijmv3jZkZQXva15ruAFKnOXnhn8mUdMcr3bWPQll9xoO0QsVgMd6elpSVn5fVXR0eH6oxIMdQYpSjCITLuvghYBDBxUo2vf+Dq3l/Q1x/eGe4cWB8rMODNN99kxowZ/eo/vo4++sjwD9/jr1+zZg2XXXZZVvqPYhyPq/ttHbWX16ZVQ1/rj863AaitraWlpYX6+voc9dt/dXV11NbW5ruMPhVDncVQI0T38xBFODQCY7tNjwnmJWrTEBxWGgY097bSEoNhQ8sjKC+7BpcZnxlUUBmbUHmJMaisNN9l9KnEjJISXSwqkm9RXK20DphoZuPNrAKYC6zo0WYFMC94Pgd4o7fzDSIikl8Z/8kbnEO4B1gFlAJPufsmM/sOsN7dVwA/BX5mZjuA/cQDREREClQkx0PcfSWwsse8B7o9bwW+GEVfIiKSffqEtIiIhCgcREQkROEgIiIhCgcREQlROIiISIjCQUREQhQOIiISonAQEZEQhYOIiIQoHEREJEThICIiIQoHEREJUTiIiEiIwkFEREIUDiIiEqJwEBGREIWDiIiEKBxERCRE4SAiIiEKBxERCVE4iIhIiMJBRERCFA4iIhKicBARkRCFg4iIhCgcREQkROEgIiIhGYWDmY0ws1fNbHvw7/Ak7TrMrD54rMikTxERyb5M9xzuA15394nA68F0Ikfd/T8Gj1kZ9ikiIlmWaTjMBp4Jnj8D3Jjh+kREpABkGg6fdfem4Ple4LNJ2g02s/Vm9raZ3ZhhnyIikmVlfTUws9eAMxIsWth9wt3dzDzJaj7n7o1mNgF4w8w2uvvOBH0tABYAjBw5krq6ur7Ky7vDhw+rzggVQ50tLS10dHQUfJ1QHOMJxVFnMdQYKXfv9wPYCpwZPD8T2JrCaxYDc/pqN2nSJC8Gq1evzncJKVGd0bn88st96tSp+S4jJcUwnu7FUWcx1OjuDqz3DH6vH39kelhpBTAveD4PeKlnAzMbbmaDguenAV8ANmfYr4iIZFGm4fAI8Hdmth24KpjGzC4ysyeDNpOB9Wa2AVgNPOLuCgcRkQLW5zmH3rh7MzAzwfz1wPzg+f8DzsukHxERyS19QlpEREIUDiIiEqJwEBGREIWDiIiEKBxERCRE4SAiIiEKBxERCVE4iIhIiMJBRERCFA4iIhKicBARkRCFg4iIhCgcREQkROEgIiIhCgcREQlROIiISIjCQUREQhQOIiISonAQEZEQhYOIiIQoHEREJEThICIiIQoHEREJUTiIiEiIwkFEREIUDiIiEqJwEBGREIWDiIiEZBQOZvZFM9tkZp1mdlEv7a41s61mtsPM7sukTxERyb5M9xw+AG4G1iRrYGalwOPAdcAU4EtmNiXDfkVEJIvKMnmxu28BMLPeml0M7HD3j4K2S4HZwOZM+hYRkezJKBxSNBrY0226AbgkUUMzWwAsCCbbzOyDLNcWhdOAP+e7iBSozmidZmZFUSdFMp4Ufp3FUCNATRQr6TMczOw14IwEixa6+0tRFHGcuy8CFgX9rnf3pOcxCoXqjJbqjJbqjE4x1AjxOqNYT5/h4O5XZdhHIzC22/SYYJ6IiBSoXFzKug6YaGbjzawCmAusyEG/IiLST5leynqTmTUAnwd+bWargvmjzGwlgLvHgHuAVcAW4F/dfVMKq1+USW05pDqjpTqjpTqjUww1QkR1mrtHsR4RERlA9AlpEREJUTiIiEhIXsOht9tvmNn9we02tprZNUleP97M3gna/SI44Z3tmn9hZvXBY7eZ1Sdpt9vMNgbtIrm0LB1m9pCZNXar9fok7fJ6axMz+99m9qGZvW9m/2ZmlUna5WU8+xofMxsUbBM7gm1xXK5qC/ofa2arzWxz8LP0jwna1JrZgW7bwgO5rLFbHb2+hxb3w2As3zezC/JQY023cao3s4Nmdm+PNnkZTzN7ysw+6f75LzMbYWavmtn24N/hSV47L2iz3czmpdShu+ftAUwm/oGNOuCibvOnABuAQcB4YCdQmuD1/wrMDZ4/AXw9x/X/M/BAkmW7gdPyOLYPAd/qo01pMLYTgIpgzKfkuM6rgbLg+feA7xXKeKYyPsB/AZ4Ins8FfpHjGs8ELgienwpsS1BjLfCrXNbVn/cQuB54GTDgUuCdPNdbCuwFPlcI4wlcBlwAfNBt3v8C7gue35fo5wcYAXwU/Ds8eD68r/7yuufg7lvcfWuCRbOBpe7e5u67gB3Eb8PRxeL37LgSeCGY9QxwYxbLPUHQ/y3Aklz1mQVdtzZx93bg+K1Ncsbdf+PxK9oA3ib+OZhCkcr4zCa+7UF8W5xpfdxPJkru3uTu7wbPDxG/InB0rvqP2GzgWY97G6g0szPzWM9MYKe7/3sea+ji7muA/T1md9/+kv0OvAZ41d33u/unwKvAtX31V6jnHBLdcqPnBl8FtHT7xZKoTTbNAD529+1JljvwGzP7Q3BbkHy4J9g9fyrJ7mYq45xLdxL/yzGRfIxnKuPT1SbYFg8Q3zZzLjikNQ14J8Hiz5vZBjN72czOyW1lXfp6Dwtte5xL8j/+CmE8AT7r7k3B873AZxO06de4Zv3eSpbD229EJcWav0Tvew1/6+6NZnY68KqZfRgkf07qBH4C/BPxH8h/In4I7M4o+09VKuNpZguBGPDzJKvJ+ngWMzM7BXgRuNfdD/ZY/C7xQyOHg3NPy4GJOS4Riug9DM5fzgLuT7C4UMbzBO7uZhbZZxOyHg7ev9tvpHLLjWbiu51lwV9skd2Wo6+azayM+K3KL+xlHY3Bv5+Y2b8RP0QR6Q9CqmNrZv8X+FWCRTm5tUkK4/ll4AZgpgcHSROsI+vjmUAq43O8TUOwXQwjvm3mjJmVEw+Gn7v7sp7Lu4eFu680sx+b2WnuntObyKXwHhbSrXauA9519497LiiU8Qx8bGZnuntTcAjukwRtGomfJzluDPHzvL0q1MNKK4C5wZUg44mn8truDYJfIquBOcGseUCu9kSuAj5094ZEC83sM2Z26vHnxE+65vQOsz2O1d6UpP+839rEzK4F/jswy92PJGmTr/FMZXxWEN/2IL4tvpEs4LIhOL/xU2CLu/8gSZszjp8HMbOLif/c5zrAUnkPVwC3B1ctXQoc6HbIJNeSHhkohPHspvv2l+x34CrgajMbHhxevjqY17tcn3HvcRb9JuLHv9qAj4FV3ZYtJH6lyFbgum7zVwKjgucTiIfGDuB5YFCO6l4MfK3HvFHAym51bQgem4gfPsn12P4M2Ai8H2xAZ/asM5i+nvgVLjvzVOcO4sdD64PHEz3rzOd4Jhof4DvEwwxgcLDt7Qi2xQk5Hr+/JX7o8P1uY3g98LXj2yjx29dsCsbvbeA/5eF9Tvge9qjTiH8x2M5g270o13UGdXyG+C/7Yd3m5X08iYdVE3As+L35FeLnt14HtgOvASOCthcBT3Z77Z3BNroDuCOV/nT7DBERCSnUw0oiIpJHCgcREQlROIiISIjCQUREQhQOIiISonAQEZEQhYOIiIT8f+nk7CRPcWYwAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.grid()\n",
    "plt.axhline(0, color=\"black\")\n",
    "plt.axvline(0, color=\"black\")\n",
    "plt.xlim([-10, 10])\n",
    "plt.ylim([-1, 2])\n",
    "x = np.linspace(-10, 10, 1000)\n",
    "y = 1 / (1 + 100000000000**(-1 * x))\n",
    "plt.plot(np.linspace(-10, 10, 1000), y, label=\"y = 1 / (1 + 100000000000^(-x))\")\n",
    "plt.legend()\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8decc4ab-d743-4621-8dc2-a929db22dd56",
   "metadata": {},
   "source": [
    "Even though visually it looks like having a large base value would make for a better approximation, it turns out that it doesn't actually really matter how steep the function is - the math works out (for the most part) regardless of steepness."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a5d93a24-c327-46c2-8cec-e4005025721e",
   "metadata": {},
   "source": [
    "Remember that when I defined the problem, I stated that the data points must be linearly separable. If even one point is misclassified, I can expect the error between the true and the predicted to have a value that is at least $>0.5$. If that point ends up being classified correctly, the error between the true and the predicted should have a value $<0.5$. So total loss, which is the sum of all these smaller losses, should always be smaller when the points are classified correctly. Since I defined the problem to have linear separability, there must exist a solution that linearly separates the space and classifies all the points correctly. If a point manages to land right in the middle of the sigmoid at a value of $0.5$, it technically means that the computer doesn't know which class the point falls into. Of course, this scenerio can happen when you have an infinite number of input data points to cover the whole Cartesian space, which is unrealistic and woud never happen in a real-world scenario, or if the computer runs out of significant digits needed to classify the point correctly, which is an extreme that I don't really care about right now."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3aaf18b8-bd0f-407c-9641-1c42de4398bb",
   "metadata": {},
   "source": [
    "So given that it doesn't matter for the most part which variation of steepness I can use as my activation function, I'll choose to use:"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "81a43957-a0db-4c6c-94e7-1ace0dd0746a",
   "metadata": {},
   "source": [
    "$$\\frac{1}{1+e^{-x}}$$"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7c7b96c7-4d0b-4888-ab8b-995903e0f117",
   "metadata": {},
   "source": [
    "as my activation function and the one I use to calculate loss because taking the first-order derivative of this function is a lot nicer than if the base in this function was not $e$."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "50477dc1-8c27-4c12-a35d-a622dafd39c0",
   "metadata": {},
   "source": [
    "$$\n",
    "\\frac{d}{dx} \\frac{1}{1+e^{-x}} = \\frac{d}{dx} \\frac{e^x}{1+e^x} = \\frac{e^{-x}}{(1+e^{-x})^2} = \\frac{e^x}{(1+e^x)^2}\n",
    "$$"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "da226ce2-362f-4eb5-9e8f-9c13b3386fb6",
   "metadata": {},
   "source": [
    "Great! So now I can use the logistic function as my activation function that transforms the output of my weighted sum. I can rewrite my $MSE$ loss as:"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4b3b7bbc-cec9-42a6-b2f9-9531661a491b",
   "metadata": {},
   "source": [
    "$$\n",
    "MSE = \\frac{1}{n} \\sum\\limits_{i = 1}^{n} \\left(y_i - \\frac{1}{1 + e^{-(b + w_1 x_{i,1} + w_2 x_{i,2})}}\\right)^2\n",
    "$$"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "98783638-b5c3-4716-9fd3-8a1e535ea98c",
   "metadata": {},
   "source": [
    "and I can actually find the first-order partial derivatives of this function."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3849ba9a-99a9-42a6-adda-64f182314b71",
   "metadata": {},
   "source": [
    "There is one more thing important thing to be aware of. My original approach was to use gradient descent to find the point of smallest loss, but minimizing a function using gradient descent only really works for convex functions. In least squares linear regression, my $MSE$ function took that of a parabolic shape and so there was always a guaranteed minimum point to work towards. This $MSE$ function has a sigmoidal \"S\" shape, and doesn't have a true minimum point. A gradient descent algorithm will keep going in the direction of $0$ for every iteration, but it will never converge onto a constant set of weights $w$ and bias $b$."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d4df9715-8051-4bb5-b86a-9fd906927198",
   "metadata": {},
   "source": [
    "The problem I presented above is an important component of understanding why the **vanishing gradient** problem occurs and it is a problem that must be addressed when constructing deep neural networks (which I will get into a bit right now, but not too much). Luckily for me, my neural network only consists of one node so this problem is unlikely <span style=\"color:red\">(or impossible? I'm not really sure as of now)</span> to happen."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f8f81021-c443-41dd-948d-0bf96a21781c",
   "metadata": {},
   "source": [
    "I can either run the algorithm a set number of epochs as usual, or I can keep it running until the weights+bias values do not update anymore. If I keep iterating until the weights and bias no longer change with each iteration, then the only thing left to do is just test the performance of the perceptron. At this point, if the model classifies all the points in my dataset correctly, then everything is fine, I have a working model. If not, then I have experienced the vanishing gradient problem. This is because the magnitude of the gradient gets so small it has no effect on changing the weights+bias. This due to the nature of the sigmoid function and/or the fact computers that can't hold numbers that small in memory."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8263185b-a483-4313-9ad3-51ce939953f6",
   "metadata": {},
   "source": [
    "So let me go ahead and write out the gradient for by sigmoid based $MSE$ loss function and derive the weight+bias update formulas from the gradient:"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d6b3d249-fc7b-41e0-9e2b-7b057a15fefd",
   "metadata": {},
   "source": [
    "$$\n",
    "\\nabla MSE =\n",
    "\\begin{bmatrix}\n",
    "    MSE_{b} \\\\\n",
    "    MSE_{w_1} \\\\\n",
    "    MSE_{w_2}\n",
    "\\end{bmatrix}\n",
    "=\n",
    "\\begin{bmatrix}\n",
    "    \\frac{1}{n} \\sum\\limits_{i = 1}^{n} -2\\left(y_i - \\frac{1}{1+e^{-(b + w_1 x_{i,1} + w_2 x_{i,2})}}\\right)\\left(\\frac{e^{-(b + w_1 x_{i,1} + w_2 x_{i,2})}}{(1+e^{-(b + w_1 x_{i,1} + w_2 x_{i,2})})^2}\\right) \\\\\n",
    "    \\frac{1}{n} \\sum\\limits_{i = 1}^{n} -2 x_{i,1} \\left(y_i - \\frac{1}{1+e^{-(b + w_1 x_{i,1} + w_2 x_{i,2})}}\\right)\\left(\\frac{e^{-(b + w_1 x_{i,1} + w_2 x_{i,2})}}{(1+e^{-(b + w_1 x_{i,1} + w_2 x_{i,2})})^2}\\right) \\\\\n",
    "    \\frac{1}{n} \\sum\\limits_{i = 1}^{n} -2 x_{i,2} \\left(y_i - \\frac{1}{1+e^{-(b + w_1 x_{i,1} + w_2 x_{i,2})}}\\right)\\left(\\frac{e^{-(b + w_1 x_{i,1} + w_2 x_{i,2})}}{(1+e^{-(b + w_1 x_{i,1} + w_2 x_{i,2})})^2}\\right)\n",
    "\\end{bmatrix}\n",
    "$$\n",
    "$$\n",
    "b_{n+1} = b_n - \\eta MSE_{b}(b_n, w_{1\\, n}, w_{2\\, n}) = b_n - \\eta\\left(\\frac{1}{m} \\sum\\limits_{i = 1}^{m} -2\\left(y_i - \\frac{1}{1+e^{-(b_n + w_{1\\, n} x_{i,1} + w_{2\\, n} x_{i,2})}}\\right)\\left(\\frac{e^{-(b_n + w_{1\\, n} x_{i,1} + w_{2\\, n} x_{i,2})}}{(1+e^{-(b_n + w_{1\\, n} x_{i,1} + w_{2\\, n} x_{i,2})})^2}\\right)\\right) \\\\\n",
    "w_{1\\, n+1} = w_{1\\, n} - \\eta MSE_{w_1}(b_n, w_{1\\, n}, w_{2\\, n}) = w_{1\\, n} - \\eta\\left(\\frac{1}{m} \\sum\\limits_{i = 1}^{m} -2 x_{i,1} \\left(y_i - \\frac{1}{1+e^{-(b_n + w_{1\\, n} x_{i,1} + w_{2\\, n} x_{i,2})}}\\right)\\left(\\frac{e^{-(b_n + w_{1\\, n} x_{i,1} + w_{2\\, n} x_{i,2})}}{(1+e^{-(b_n + w_{1\\, n} x_{i,1} + w_{2\\, n} x_{i,2})})^2}\\right)\\right) \\\\\n",
    "w_{2\\, n+1} = w_{2\\, n} - \\eta MSE_{w_2}(b_n, w_{1\\, n}, w_{2\\, n}) = w_{2\\, n} - \\eta\\left(\\frac{1}{m} \\sum\\limits_{i = 1}^{m} -2 x_{i,2} \\left(y_i - \\frac{1}{1+e^{-(b_n + w_{1\\, n} x_{i,1} + w_{2\\, n} x_{i,2})}}\\right)\\left(\\frac{e^{-(b_n + w_{1\\, n} x_{i,1} + w_{2\\, n} x_{i,2})}}{(1+e^{-(b_n + w_{1\\, n} x_{i,1} + w_{2\\, n} x_{i,2})})^2}\\right)\\right)\n",
    "$$"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "771f0e19-c542-44f7-94da-a623fc2b84d9",
   "metadata": {},
   "source": [
    "Like before, set $m=1$, use predicted value $\\hat y_i$ in the formulas, and let $\\eta$ \"absorb\" the factor of $2$:"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d65b1ba6-a227-4284-bd4d-48c81dc228ea",
   "metadata": {},
   "source": [
    "$$\n",
    "b_{n+1} = b_n + \\eta (y_i - \\hat y_i)\\left(\\frac{{\\hat y_i}^2}{e^{b_n + w_{1\\, n} x_{i,1} + w_{2\\, n} x_{i,2}}}\\right) \\\\\n",
    "w_{1\\, n+1} = w_{1\\, n} + \\eta (y_i - \\hat y_i)\\left(\\frac{{\\hat y_i}^2}{e^{b_n + w_{1\\, n} x_{i,1} + w_{2\\, n} x_{i,2}}}\\right)x_{i,1} \\\\\n",
    "w_{2\\, n+1} = w_{2\\, n} + \\eta (y_i - \\hat y_i)\\left(\\frac{{\\hat y_i}^2}{e^{b_n + w_{1\\, n} x_{i,1} + w_{2\\, n} x_{i,2}}}\\right)x_{i,2}\n",
    "$$"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ea0f8e8d-45ea-4534-b665-2042ca6864a8",
   "metadata": {},
   "source": [
    "Like before, now I have everything needed to code out the perceptron. Once again, the perceptron algorithm in words is:\n",
    "1. Initialize the weights and bias to zero (or with random values, it doesn't really matter)\n",
    "2. Set the number of epochs\n",
    "3. Iterate through each epoch\n",
    "    1. For each epoch, randomly shuffle the order of data points in the dataset\n",
    "    2. Iterate through each point in the dataset\n",
    "        1. For each point, feed the $x_i$ value into the perceptron\n",
    "        3. Update the weights and the bias based on the values calculated from the gradient of the loss function"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "id": "df902d0c-8df9-4f8c-ba24-9422f1fbdfbe",
   "metadata": {},
   "outputs": [],
   "source": [
    "lr = 0.001\n",
    "\n",
    "class Perceptron():\n",
    "    def __init__(self, input_dataset, classifications):\n",
    "        self.dataset = input_dataset.copy()\n",
    "        self.true_labels = classifications.copy()\n",
    "        self.num_features = len(input_dataset[0])\n",
    "        self.weights = [0] * self.num_features\n",
    "        self.bias = 0\n",
    "        self.activation_func = lambda x: 1 / (1 + math.e**(-1 * x))\n",
    "    \n",
    "    def shuffle_dataset(self):\n",
    "        zipped_dataset = list(zip(self.dataset, self.true_labels))\n",
    "        random.shuffle(zipped_dataset)\n",
    "        self.dataset, self.true_labels = zip(*zipped_dataset)\n",
    "    \n",
    "    def process_one_point(self, data_point, label):\n",
    "        x = data_point\n",
    "        y = label\n",
    "        weighted_sum = sum([w_d * x_d for w_d, x_d in zip(self.weights, x)]) + self.bias\n",
    "        return x, y, self.activation_func(weighted_sum)\n",
    "    \n",
    "    def update_weights_bias(self, lr, x, y, y_hat):\n",
    "        weighted_sum = sum([w_d * x_d for w_d, x_d in zip(self.weights, x)]) + self.bias\n",
    "        self.weights = [w_d + lr * (y - y_hat) * (y_hat**2 / math.e**(weighted_sum)) * x_d for w_d, x_d in zip(self.weights, x)]\n",
    "        self.bias += lr * (y - y_hat) * (y_hat**2 / math.e**(weighted_sum))\n",
    "    \n",
    "    def process_dataset(self, lr):\n",
    "        self.shuffle_dataset()\n",
    "        for i in range(0, len(self.dataset)):\n",
    "            x, y, y_hat = self.process_one_point(self.dataset[i], self.true_labels[i])\n",
    "            self.update_weights_bias(lr, x, y, y_hat)\n",
    "    \n",
    "    def get_current_true_loss(self):\n",
    "        weighted_sums = [sum([w_d * x_d for w_d, x_d in zip(p.weights, s)]) + self.bias for s in self.dataset]\n",
    "        pred_labels = [1 if ws > 0 else 0 for ws in weighted_sums]\n",
    "        loss = sum([(t - p)**2 for t, p in zip(self.true_labels, pred_labels)])\n",
    "        return loss\n",
    "    \n",
    "    def get_current_approximated_loss(self):\n",
    "        weighted_sums = [sum([w_d * x_d for w_d, x_d in zip(p.weights, s)]) + self.bias for s in self.dataset]\n",
    "        pred_labels = [1 / (1 + math.e**(-1 * ws)) for ws in weighted_sums]\n",
    "        loss = sum([(t - p)**2 for t, p in zip(self.true_labels, pred_labels)])\n",
    "        return loss"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8ab4b3b8-3519-4655-9610-0387c1e68035",
   "metadata": {},
   "source": [
    "Train the perceptron classifier and graph the result:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "id": "f350be36-4430-4914-9d2e-172235217ab7",
   "metadata": {},
   "outputs": [],
   "source": [
    "p = Perceptron(samples, true_classifications)\n",
    "loss_vals_perceptron_from_scratch = []\n",
    "for epoch in range(0, 100000):\n",
    "    p.process_dataset(lr)\n",
    "    loss_vals_perceptron_from_scratch.append(p.get_current_true_loss())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "id": "ca920083-5984-48ef-b791-b6664ef2c126",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYQAAAD5CAYAAAAndkJ4AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/YYfK9AAAACXBIWXMAAAsTAAALEwEAmpwYAABAMklEQVR4nO3dd1yV5f/H8dclztRckeYoR2niIkVNc4CW1rfpLLXMBMGVK3NkmlqO1DRzgjjSHJB7pqHgxHKmppGLzK04EHHA4fr9cSM/NBCUc859DnyejwcPOefc577f3NyeD9d9X/d1Ka01QgghRDazAwghhHAMUhCEEEIAUhCEEEIkkoIghBACkIIghBAikRQEIYQQAGS3xkqUUrOAt4CLWuvKic8NBToBlxIX+0JrvfZh6ylYsKB+/vnnrRHJpm7evEnevHnNjpEmyWk9ERERWCwW3NzczI7yUM6wL0FyWkNCQgJnzpzh4sWLAJe11q4ZXqnWOsNfQAOgOnAo2XNDgb6Psp7y5ctrZxAaGmp2hHSRnNbTsGFDXa1aNbNjpMkZ9qXWkjOjNmzYoEuXLq0B3a1bNw3s1lb4LLfKKSOt9RbgijXWJTKfMWPGEBoaet9zoaGhjBkzxqREQjinq1ev0rFjR5o0aUKuXLnYunUrkydPttr6bX0NobtS6oBSapZSqpCNtyUcVM2aNWndunVSUQgNDaV169bUrFnT5GRCOI9ly5bh5ubG3LlzGThwIPv376devXpW3YbSVhq6QilVGlit//8aQlHgMqCBr4FntNYdU3ifL+AL4OrqWiM4ONgqeWwpJiaGfPnymR0jTY6Uc9++fQwbNox33nmHlStX8tVXX/HSSy8BjpUzNb169cJisTBp0iSzozyUM+xLkJyP4sqVK/zwww9s3ryZ559/ns8//5zy5cvft4yXl9cerbVHRrdls4KQ3teSq1Chgo6IiLjvubi4OE6fPs3t27etktMabt++Te7cuc2OkSZHy3nt2jWuX79OgQIFKFiwYNLzj5ozd+7clCxZkhw5ctggZco8PT25du0a+/fvt9s2H0dYWBienp5mx0iT5Eyb1pq5c+fSu3dvYmNj+eqrr+jbt2+Kx71SyioFwSq9jFKilHpGa30u8WEz4NDjrOf06dPkz5+f0qVLo5SyXsAMuHHjBvnz5zc7RpocKWd0dDR37tzh2Wef5dKlS5QoUYInn3wSeLScWmuioqI4ffo0ZcqUsWVkIUzzzz//4Ofnx/r166lbty4zZ87kxRdftPl2rXINQSm1EAgHKiilTiulvIExSqmDSqkDgBfQ+3HWffv2bYoUKeIwxUA8uujoaE6cOEHZsmUpUaIEZcuW5cSJE0RHRz/yupRSFClSxKFajEJYS0JCApMnT6ZSpUps27aNSZMmsXXrVrsUA7BSC0Fr3SaFp2daY92AFAMnFxsbS9myZZNaBE8++SRly5YlNjY26blHIceDyIwiIiLw9vZm+/btNG3aFH9/f5577jm7ZpA7lYXNFStW7D8f/E8++STFihUzKZEQjiMuLo5Ro0ZRrVo1Dh8+zJw5c1i3bp3diwHY8BpCZhEVFUXjxo0BOH/+PC4uLhQpUoRs2bLx+++/kzNnzgxvw9PTk3PnzpEnTx7u3LlD79698fX1zfB6H6ZDhw689dZbtGzZ0qbbEUKkbt++fXh7e7Nv3z5atmzJpEmTTP1DSQpCGooUKZLUs2To0KHky5cPPz+/pIug8fHxZM+e8d04f/58PDw8uHLlCuXKlaNDhw5WKTbWYq2fUwhhXBsdPnw4Y8aM4amnnmLJkiU0b97c7FhyyuhxdO7cmc6dO1O7dm369evH0KFDGTduXNLrlStXJjIyEoCffvqJWrVq4e7ujp+fHxaL5aHrjomJIW/evLi4uACwcOFCqlSpQuXKlenfv3/Scsn7Ri9evJgOHToAxl/+PXr0oG7dulStWpXFixcDRu+c7t27U6FCBV599dV7458AMHz4cGrWrEnlypXx9fW9N/QInp6e9OrVCw8PD0aMGEGZMmWIi4sDjAvFyR8LIdJn27ZtVKtWjVGjRtG+fXuOHDniEMUAnKyF0KtXL6v3A3d3d+f7779/5PedPn2aHTt24OLiwtChQ1Nc5siRIwQFBbF9+3Zy5MhB165dmT9/Pu3bt//Psu3atSNXrlwcPXqU77//HhcXF86ePUv//v3Zs2cPhQoVokmTJixfvpz33nvvodnOnTvHtm3b2LNnD23atKFly5YsW7aMiIgIDh8+zIULF3Bzc6NjR+M+we7duzNkyBAAPvroI1avXs3bb78NwN27d9m9ezcAkZGRrFmzhvfee49FixbRvHlzu94LILKmMWPGULNmTby8vJKeCw0NZdeuXfTr18/EZI/mxo0bDBw4kClTplC6dGnWr19PkyZNzI51H2khPKZWrVol/RWfmo0bN7Jnzx5q1qyJu7s7Gzdu5MSJEykuO3/+fA4cOMCpU6cYN24c//zzD7t27cLT0xNXV1eyZ89Ou3bt2LJlS5rZ3nvvPbJly8aLL77IhQsXANiyZQtt2rTBxcWF4sWL06hRo6TlQ0NDqV27NlWqVGHTpk38+eefSa+9//77Sd/7+Pgwe/ZsAGbPns0nn3ySZhYhMiozDH2yfv16KleuzNSpU+nRowcHDx50uGIATtZCeJy/5G0l+ZC42bNnJyEhIenxvT7yWms+/vhjRo0ale71urq6Ur16dX777Tdy5cqV6nLJu14+2Cc/+fvSuhP99u3bdO3ald27d1OqVCmGDh163/qS/5yvvPIKkZGRhIWFYbFYqFz5oTeeC2EVXl5eBAcH07p1a7p06cK0adMIDg6+r8XgqK5cuULv3r2ZO3cuL774Itu2baNu3bpmx0qVtBCsoHTp0uzduxeAvXv3cvLkSQAaN27M4sWLk87XX7lyhX/++eeh64qNjWXfvn2UK1eOWrVqsXnzZi5fvozFYmHhwoU0bNgQgKJFi3LkyBESEhJYtmxZmhkbNGhAUFAQFouFc+fOJf21de/D/6mnniImJibpmkNq2rdvT9u2baV1IOzKy8uLLl268PXXX9OlSxenKAaLFy+mYsWKLFiwgEGDBrFv3z6HLgbgZC0ER9WiRQvmzp1LpUqVqF27dtLAU25ubnzzzTc0adKEhIQEcuTIwZQpU1LsX9yuXbukbqcdOnSgRo0aAIwePRovLy+01rz55pu8++67Sc+/9dZbuLq64uHhQUxMzEMzNmvWjE2bNuHm5sazzz5LnTp1AChYsCCdOnWicuXKFCtWLM1meLt27fjyyy9p0yalexGFsI3Q0FCmTZvG4MGDmTZtGl5eXg5bFM6dO0e3bt1YtmwZ1atXZ/369bi7u5sdK32sMamCtb5SmiDn8OHDjzRxhD1ER0ebHSFdbJHz559/1h9++KFV1/k4Oe19XMgEOdb1KDk3bdqkn3rqKb1p06YUH9vSo+RMSEjQs2bN0gULFtS5cuXSo0eP1nFxcbYLlwxWmiBHWggi3T799FPWrVvH2rUPnQlVCKvatWvXfdcM7l1T2LVrl8O0Ek6ePImvry8hISHUr1+fwMDA/wxR7QykIIh0c/S5AETmlFLXUkc5ZWSxWJgyZQoDBw4kW7ZsTJ06FT8/P7Jlc87Ls1IQhBDiMRw5cgRvb2/Cw8N54403mD59Os8++6zZsTLEOcuYEEKYJC4ujhEjRuDu7k5ERATz5s1jzZo1Tl8MQFoIQgiRbnv27KFjx44cOHCA1q1bM2nSJJ5++mmzY1mNtBCEECINt27don///tSqVYtLly6xbNkygoKCMlUxACkI6eLi4oK7uzuVK1emVatWxMbGPva6OnTokHTzl4+PD4cPH0512bCwMHbs2PHI2yhdujSXL19O8fkqVarg7u5OlSpVWLFixSOv+1F5enomjYUkhDPasmUL1apVY8yYMXTs2JHDhw+nOZ6Ys5KCkA558uRh//79HDp0iJw5czJr1qz7Xo+Pj3+s9QYGBuLm5pbq649bEB4mNDSU/fv3s3jxYnr06GHVdVtDWqPBCmEv0dHRTJgwgYYNGxIfH09ISAgzZsygYMGCZkezGSkIj6h+/focP36csLAw6tevzzvvvIObmxsWi4XPP/+cmjVrUrVqVfz9/YGHDzud/K/nX375herVq1OtWjUaN25MZGQk06dPZ8KECbi7u7N161YuXbpEixYtqFmzJjVr1mT79u2AMYlPkyZNqFSpEj4+PmmOXwTGwV6oUKGkx+PHj6dy5cpUrlw5acyoyMjI+8YrGjduXNLIrp6enklN6PLly7N161bAaFp/8MEHVKxYkWbNmnHr1q2k93fp0gUPDw8qVarEV199lfT8vaG9q1evzujRo6levXrSa0ePHr3vsRD2sHbtWipXrsyqVavo3bs3Bw8eTJooKzNzrovKvXqBlYe/xt0d0jloXnx8POvWrcPT0xMwxi06dOgQZcqUISAggAIFCrBr1y7u3LnDK6+8QpMmTdi3b1+qw07fc+nSJTp16sSWLVsoU6YMV65coXDhwnTu3Jl8+fLRt29fANq2bUvv3r2pV68ep06domnTphw5coRhw4ZRr149hgwZwpo1a5g5M/XprO8Ng3HixAmCg4MB40LZ7Nmz+e2339BaU7t2bRo2bHhfwUhtf/z++++sXbuWYcOGERISwrRp03jiiSc4cuQIBw4cuO/DfMSIERQuXBiLxULjxo05cOAAVatWBYyJiO6NBxUSEsL+/ftxd3eXUVWFXV2+fJnevXvz008/4ebmxuTJk+natavZsexGWgjpcOvWLdzd3fHw8ODZZ59Nms+gVq1alClTBoANGzYwd+5c3N3dqV27NlFRURw9evShw07fs3PnTho0aJC0rsKFC6eYIyQkhO7du+Pu7s4777xDdHQ0MTExbNmyhQ8//BCAN99886Ef5KGhoRw6dIiDBw/SvXt3YmJi2LZtG82aNSNv3rzky5eP5s2bJ/3F/zD3JvWoUaNG0oRAybNUrVo16QMfIDg4mOrVq/PSSy/x559/3nf9JKVhti0WC0FBQbRt2zbNLEJkhNaaoKAg3NzcWLRoEUOGDGHv3r0PPaWbGTlXC8Gk4a/vXUO458aNG8D9Q0NrrZk0aRJNmza9773WHOYhISGBnTt3kjt37gyvq1y5chQtWvShF7VTG9b7nnvDbLu4uKR5HeXkyZOMGzeOXbt2UahQITp06JDqMNstWrRg2LBhNGrUiBo1alCkSJFH+tmEeBRnz56lS5curFy5Eg8PD0JCQu77QyYrkRaClTRt2pRp06YlTSn5999/c/PmzVSHnU7u5ZdfZsuWLUnDZl+5cgWA/PnzJxUfgCZNmtw3fMS9ItWgQQMWLFgAwLp167h69WqaeS9evMjJkyd57rnnqF+/PsuXLyc2NpabN2+ybNky6tevT9GiRbl48SJRUVHcuXOH1atXp7ne5FkOHTrEgQMHAOOaRd68eSlQoAAXLlxg3bp1qa4jd+7cNG3alC5dusjpImEzWuukjh0bNmxg3LhxhIeHZ9liAM7WQnBgPj4+REZGUr16dbTWuLq6snz58lSHnU7O1dWVgIAAmjdvTkJCAk8//TS//vorb7/9Ni1btmTFihVMmjSJH374gW7dulG1alXi4+Np0KAB06dP56uvvqJNmzZUqlSJunXrPvSOSS8vL1xcXIiLi2P06NEULVqUokWL0qFDB2rVqpX0s7z00ksADBkyhFq1alGiRAlefPHFNPfDvQ/xihUrUrFixaRhvKtVq8ZLL73Eiy++SKlSpXjllVceup527dqxbNkyh5xVSji/EydO0KlTJzZt2kTDhg0JDAzk+eefNzuW+awxZKq1vmT4a+ty5pxjx47VX375ZarvkeGvU5YZh7+2pvj4eD1+/HidJ08enT9/fu3v768tFkuqyzvL/kSGvxaZVbNmzTh+/DibNm0yO4rIRP7880+8vb357bffePPNN5k+fTolS5Y0O5ZDkYIgHE56pgQVIr3u3r3L6NGj+eabbyhQoAALFizggw8+uG9ecmGwykVlpdQspdRFpdShZM8VVkr9qpQ6mvjvwzu1P4ROx41WIuuQ40Gk165du6hRowZfffUVrVq14vDhw7Rp00aKQSqs1ctoDvD6A88NADZqrV8ANiY+fmS5c+cmKipKPgQEYBSDqKgoq3S9FZlXbGwsffv25eWXX+bq1ausXLmS+fPn4+rqanY0h2aVU0Za6y1KqdIPPP0u4Jn4/Y9AGND/UdddsmRJTp8+zaVLlzIS0apu377tFB9ImTVn7ty55dyvSFVYWBg+Pj4cP34cPz8/vv32WwoUKGB2LKdgy2sIRbXW5xK/Pw8UfZyV5MiRI+kOXkcRFhaW1C3TkUlOkZVcv36dfv36ERAQQLly5di0aZNDTLPpTJS1TsUkthBWa60rJz6+prUumOz1q1rr/1xHUEr5Ar4Arq6uNe6Nr+PIYmJiyJcvn9kx0iQ5radXr15YLBaHn1faGfYlWD/njh07mDBhAleuXKFVq1Z06NDBKq1jZ9mfXl5ee7TWHhlekTX6riYWldLAoWSPI4BnEr9/BohIax0p3YfgiJylb7LktB65D8G6rJXz4sWLuk2bNhrQlStX1r/99ptV1nuPs+xPrHQfgi2HrlgJfJz4/ceA7WdjEUJkCVprFixYQMWKFVm8eDHDhg1jz549SXfbi8djlWsISqmFGBeQn1JKnQa+AkYDwUopb+AfoLU1tiWEyNpOnz5Nly5dWL16NbVr12bmzJlUqlTJ7FjmsWIPTGv1MmqTykuZf0YJIYRdJCQkMGPGDD7//HPi4+MZP348PXr0wMXFxexo5rh6FebNg8TJuKxB7lQWQji8Y8eO0alTJ8LCwmjUqBEzZsygbNmyZseyP61h506jCAQFwe3bYMXTZDL8tRDCYcXHxzNu3DiqVKnC3r17mTFjBiEhIVmvGFy/DlOmQLVqULcuLFkCHTrAvn3w229W24y0EIQQDunAgQN4e3uze/du3nnnHaZOnUqJEiXMjmU/WsPu3UZrYOFCiI2F6tWNx23aQP78Vt+kFAQhhEO5c+cOI0eOZOTIkRQqVIigoCBatWqVdcYfunEDFiwwPvj37YMnnjAKQOfO4JHxWw0eRgqCEMJh7Ny5E29vbw4fPsyHH37I999/n3WmUN23zygC8+dDTAxUrWqcJmrXDuw09IYUBCGE6W7evMngwYP5/vvvKVGiBGvWrOF///uf2bFs7+ZNWLTIKAS7dkGePPD+++DnB7Vrg51bRVIQhBCm2rhxI506deLkyZN06dKF0aNH8+STT5ody7YOHjSKwLx5EB0Nbm4wcSJ89BEUeuyZAjJMCoIQwhTXrl3j888/JzAwkBdeeIHNmzfToEEDs2PZzq1bEBxsFILwcMiVC1q1MloDr7xi99ZASqQgCCHsbtu2bbRt25YLFy7Qr18/hg4dSp48ecyOZRuHDxtFYO5cuHYNKlSA8eOhfXtwsOsjUhCEEHZz4cIFevToQXBwMFWrVmXlypV42LjnjClu3zbuFfD3h61bIUcOaNHCaA00bOgQrYGUSEEQQtic1pr58+fTs2dPYmJi8Pb2Ztq0aeTIkcPsaNYVEQEBAfDjjxAVBc8/D2PGGDeROcFsbVIQhBA2derUKTp37sy6deuoU6cOM2fO5MKFC5mnGNy5A8uWGa2BsDDInh3ee89oDTRqBNmcZ0AIKQhCCJtISEjA39+ffv36kZCQwMSJE+nWrRsuLi5cuHDB7HgZd/y40RqYPRsuXYLSpWHkSPjkEyhWzOx0j0UKghDC6v7++298fHzYunUrr776KgEBAQ43Fe5jiYuDFSuM1kBICLi4wDvvGK2B115zqtZASqQgCCGsJj4+nu+++46vvvqKPHnyMGvWLDp06OD8w05ERsKMGTBrFpw/D6VKwfDh4O0NxYubnc5qpCAIIazijz/+oGPHjuzdu5dmzZoxZcoUnnnmGbNjPb74eJ7atg2+/RbWrzd6Br35ptEaeP11o3WQyUhBEEJkyO3bt/nmm2/49ttvKVKkCIsXL6ZFixZmx3p8p05BYCDMnEnls2eNFsDgweDjY7QMMjEpCEKIx7Zjxw68vb3566+/+Pjjjxk/fjyFCxc2O9ajs1hg3Trj2sDatcbQ06+/zsEuXagyYIDRcygLcO4rIEIIU8TExNCzZ0/q1atHbGwsv/zyC3PmzHG+YnDmjHEtoEwZePttY/6BAQPgxAlYu5aoevWyTDEAaSEIIR7Rhg0b8PX15Z9//qF79+6MHDmS/DaYrMVmEhJgwwajNbBqldE6eO01mDDB6DGUWe6PeAxSEIQQ6XL16lX69OnDnDlzqFChAlu3bqVevXpmx0q/8+eNXkIzZhi9hlxdoW9f6NQJypUzO51DkIIghEjT0qVL6datG5cuXWLgwIEMGTKE3Llzmx0rbQkJsGmT0RpYvhzi48HLC0aPhmbNIGdOsxM6FCkIQohUnT9/nu7du7NkyRLc3d1Zu3YtL730ktmx0nbxIsyZY9xJfPy4Mapoz57g6wvly5udzmFJQRBC/IfWmrlz59K7d29iY2MZOXIkffv2dezxh7Q2xhLy94elS427iuvXh2HDjJFGnaFFYzIpCEKI+/zzzz/4+fmxfv16XnnlFQIDA3nxxRfNjpW6qChjdNGAAGO00YIFoWtXozXg5mZ2OqciBUEIARiD0U2dOpUBAwYAMGnSJLp27Uo2RxyfR2vYts1oDSxebIw4WreuURhatTLmJhaPTAqCEIK//voLHx8ftm/fTtOmTfH39+e5554zO9Z/Xb1qzEPs72/MRPbkk8YdxH5+UKWK2emcnhQEIbKwuLg4xo4dy7Bhw8ibNy8//vgjH330kWMNRqc17NxpFIGgIGM2slq1YOZMeP99yJvX7ISZhs0LglIqErgBWIB4rXUmnC9PCOezb98+OnbsyP79+2nZsiWTJ0+maNGiZsf6f9evw08/GYXg4EHIl8+YeczPD9zdzU6XKdmrheCltb5sp20JIR7i9u3bDBs2jLFjx+Lq6sqSJUto3ry52bEMWhvDR/j7w8KFEBsL1asbj9u0AWe6I9oJySkjIbKQbdu24e3tzd9//80nn3zCd999R6FChcyOBTduwIIFxgf/vn3wxBNGAejcGTzkpIK9KK21bTeg1EngKqABf611wAOv+wK+AK6urjWCg4NtmscaYmJiyJcvn9kx0iQ5radXr15YLBYmTZpkdpSHSm1fxsbGMmPGDJYvX06xYsX47LPP8DDxg/ZeznxHj1J81SqeDgkh+61bxJQty9m33+bCq69icYBjwhmOTQAvL689Vjkdr7W26RdQIvHfp4E/gAapLVu+fHntDEJDQ82OkC6S03oaNmyoq1WrZnaMNKW0L9etW6efffZZrZTSPXv21Ddu3LB/sORiYvSRvn21rllTa9A6Tx6tO3TQOjxc64QEc7M9wBmOTa21BnZrK3xe2/yUkdb6TOK/F5VSy4BawBZbb1eIrC4qKoo+ffowd+5cKlasyPbt26lTp455gQ4eNE4JzZvHi9HRxk1jEyfCRx+BI5y2ErYtCEqpvEA2rfWNxO+bAMNtuU0hsjqtNUuWLKFbt25cuXKFL7/8ki+//JJcuXLZP8ytWxAcbBSC8HDIlQtatWJfrVq81L27MS2lcBi2biEUBZYl9mnODizQWv9i420KkWVFRUXRokULli1bRo0aNdiwYQPVqlWzf5DDh40iMHcuXLsGFSrA+PHQvj0UKcL1sDApBg7IpgVBa30CMOFoFCJr0Voze/ZsevbsSXx8PN9++y19+vQhuz1n+7p9G5YsMQrB1q3GRDMtWhj3DTRsKAXACUi3UyGc3MmTJ/H19SUkJISqVavy888/U96eQzxHRBgDy/34ozHQ3PPPw5gxxk1krq72yyEyTAqCEE7KYrEwefJkvvjiC1xcXJg6dSoVKlSwTzG4cweWLTNaA2FhxrzD771ntAYaNQJHHBBPpEkKghBO6PDhw/j4+BAeHs4bb7yBv78/pUqVIiwszLYbPn7caA3Mng2XLkHp0jByJHzyCRQrZtttC5uTgiCEE4mLi+Pbb7/l66+/Jn/+/Pz000+0bdvWtoPRxcXBihVGayAkBFxcjMno/fyMyemlNZBpyG9SCBONGTOG0NDQ+54LDQ1lzJgx/1l2z549eHh4MHjwYJo1a8bhw4dp166d7YpBZCQMGgTPPmvMMRARAcOHw6lTxoxkTZtKMchk5LcphIlq1qxJ69atk4pCaGgorVu3pmbNmknL3Lp1i/79+1OrVi0uXbrE8uXLWbRoEU8//bT1A8XHG5PRv/EGlC1rTEZfsyasXg0nT8LgwVC8uPW3KxyCnDISwkReXl4EBwfTunVrunTpwrRp0wgODsbLywuAzZs34+Pjw7Fjx/Dx8WHs2LEULFjQ+kFOnYLAQGOOgbNnjQ/9wYONyWdKlbL+9oRDkoIghMm8vLzo0qULX3/9NYMHD8bLy4vo6Gj69+/P9OnTKVu2LCEhITRu3Ni6G7ZYYN0649rA2rXG0NOvvw5Tp8Kbbxo9h0SWIr9xIUwWGhrKtGnTGDx4MNOmTSNPnjxMnTqVs2fP0qdPH4YPH05ea84KduaM0RIIDIR//zV6Bw0YAJ06Gb2GRJYlBUEIE927ZhAcHEyVKlXYsWMHX3zxBc899xw7duygdu3a1tlQQgJs2GC0BlatMloHr70GEyYYPYZy5LDOdoRTk4IghIl27dpFUFAQFy9exM3NjatXr9K+fXsqVKhgnWJw/jzMmgUzZhi9hlxdoW9fozVQrlzG1y8yFSkIQpioXbt2dO3alZUrV+Lh4cHGjRupUqVKxlaakACbNhmtgeXLjZ5DXl5Gj6FmzSBnTqtkF5mPFAQhTKC1JjAwkL59+3L37l3GjRtHz549MzYY3cWLlFq40OgZdPw4FCkCPXuCry/Yc2wj4bSkIAhhZ8ePH6dTp06Ehobi6enJjBkzeP755x9vZVobYwn5+8PSpZSLi4P69WHYMGOk0dy5rZpdZG5SEISwE4vFwsSJE/nyyy/JkSMH/v7++Pj4kO1x7vaNijJGFw0IMO4gLlgQunbld3d3anXoYO3oIouQgiCEHRw6dAhvb29+//133nrrLaZNm0bJkiUfbSVaw7ZtRmtg8WJjxNG6dY3C0KoV5MlDrK0HtxOZmhQEIWzo7t27jBo1ihEjRlCgQAEWLFjABx988GjjD129CvPmGYXg8GF48knjOoGfH2T0ArQQyUhBEMJGfv/9d7y9vTl06BBt27bl+++/xzW9E8ZoDTt3GkUgKMiYjaxWLeOGsvffB2veqCZEIikIQlhZbGwsQ4YMYcKECTzzzDOsWrWKt956K31vvn4dfvrJKAQHD0K+fMbMY35+4O5uy9hCSEEQwppCQ0Px8fHhxIkT+Pn58e2331KgQIGHv0lr2L3bKAILF0JsLFSvbjxu0wby57dp5vDwcMLCwvD09KROnTo23ZZwbFIQhLCC69ev069fPwICAihXrlxSl9KHunEDFiwwPvj37YMnnjAKQOfO4OFhl9zh4eE0btyYu3fvkjNnTjZu3ChFIQuT+RCEyKBVq1bh5uaWdKPZgQMHHl4M9u0zPvSLFzf+tVhgyhRj2OnAQLsVA4CwsDDu3r2LxWLh7t27tp+CUzg0aSEI8ZguXbpEz549WbhwIVWqVGH58uX3TWxzn5s3YdEiozWwaxfkyWNcHPbzg9q1wZZTYD6Ep6cnOXPmTGohpNmqEZmaFAQhHpHWmoULF9KjRw+io6MZNmwYAwYMIGdKYwQdPGgUgXnzIDoa3Nxg4kT46CMoVMj+4R9Qp04dNm7cKNcQBCAFQYhH8u+//9KlSxfWrFlD7dq1mTlzJpUqVbp/oVu3IDjYKATh4ZArl3HjmJ8fvPKKaa2B1NSpU0cKgQCkIAiRLlproqKiqFSpEhaLhQkTJvDpp5/i4uLy/wsdPmwUgblz4do1qFABxo+H9u2NgeaEcHBSEIRIw9GjR/njjz+4fv06jRs3JiAggLJlyxov3r4NS5YYhWDrVmOimRYtjNZAw4YO1xoQ4mFs3stIKfW6UipCKXVMKTXA1tsTwlri4+MZN24cVatWJSYmhpIlS/Lrr78axSAiAj77DEqWhA8/hHPnYMwYY3rKhQvB01OKgXA6Nm0hKKVcgCnAa8BpYJdSaqXW+rAttytERh04cABvb292797Nu+++y4ULF4i/eRMVFGS0BsLCjEno33vPaA00agSPM2qpEA7E1qeMagHHtNYnAJRSi4B3ASkIwiHduXOHESNGMGrUKAoVKkRQUBCtqldnYaNGvH7unHHjWOnSMHIkfPKJMUG9EJmErQtCCeDfZI9PA6lOFPvvv/86RT/oa9euUbBgQbNjpElyPpro6GgiIiKIjY2lxNNP83HBgpT180Ndu0ZrYK2LC1urVGF3oULo9eth/XqzI/+Ho+zLtEhOx2T6RWWllC/gC5AjRw6uXbtmbqB0sFgsktOKzM5psVg4f/48ly9f5vns2elVoACtr1zB9eJFzuXIwZRixZhw/TrnlOKFbNmMAegclNn7Mr0kp4PSWtvsC6gDrE/2eCAwMLXly5cvr51BaGio2RHSRXKmLSQkRD9furR+F/ShZ5/VCUppnS2b1m+/rfXq1VrHx2uttW7YsKGuVq2aaTnTS37n1uUsOYHd2gqf2bZuIewCXlBKlQHOAB8AbW28TSHSdO3aNUZ27ky+oCC2ubhQFCA+HgYPNiafKVXK7IhC2J1NC4LWOl4p1R1YD7gAs7TWf9pym0I8lMVC+JAhxHz3HaPu3EEBunFj6NoV3nzT6DkkRBZl86Nfa70WWGvr7QjxUGfOEDNxIrcnT6bOrVtcyp6di598wjNDhhi9hoRIQ1aYN0L+HBKZV0ICbNiA9vdHr1xJvoQEdirFtTZteDcwkBxPPGF2QuEkssq8EXInjch8zp837hMoVw7eeIPra9cyJiGBVi+9RIk//6TlggVSDMQjySrzRkhBEJlDQgKEhBijipYqBYMGcTpnTj7OlYsy2bOT94cfWLRrFxUrVjQ7qXBC9+aNcHFxydTzRsgpI+HcLl6EOXMgIACOH4ciRbjSvj3d//iDhXv28Nprr7EvIIDScp1AZEBWmTdCCoJwPlobYwn5+8PSpRAXB/XrYxkyhAmnTvHlN9+QJ08eZs+ezccff4ySQeaEFWSFeSOkIAjnERUFP/5otAYiIqBgQaO7qK8v++/exdvbm71799KsWTOmTJnCM888Y3ZiIZyKXEMQjk1rY56BDz+EEiWMIaeLFDEKw9mz3B49mkHz5+Ph4cGZM2dYvHgxS5culWIgxGOQFoJwTFevGvMQ+/sbM5E9+aRxB7GfH1SpAsCOHTvw9vbmr7/+4uOPP2b8+PEULlzY5OBCOC8pCMJxaA07dxpFICjImI2sVi2YORPefx/y5gUgJiaGL774gsmTJ1OqVCl++eUXmjZtanJ4IZyfFARhOpeYGJgyxSgEBw9CvnzQoYPRGnB3v2/ZDRs24Ovry6lTp+jWrRsjR44kf/78puQWIrORgiDMoTXs3g3+/tSdP99oDVSvbhSFNm3ggQ/5K1eu8NlnnzFnzhwqVKjAli1bqFevnknhhcicpCAI+7pxAxYsMD749+2DJ57gYqNGPDNsGHh4pPiWJUuW0K1bNy5fvszAgQMZMmQIuXPntnNwITI/6WUk7GPfPujcGYoXN/61WIzTRGfPEvH55ykWg/Pnz9OyZUtatmxJ8eLF2bVrFyNHjpRiIISNSAtB2M7Nm7BokdEa2LUL8uQxLg77+UHt2pDKDWNaa3788Uf69OlDbGwso0aN4rPPPiNHjhx2/gGEyFqkIAjrO3jQKALz5kF0NLi5wcSJ8NFHUKjQQ98aGRmJn58fGzZsoF69egQGBlKhQgU7BRcia5OCIKzj1i0IDjYKQXg45MplDDTn5wevvJJqa+CehIQEpkyZwsCBA1FKMXnyZLp06UK2bHJWUwh7kYIgMubwYaMIzJ0L165BhQowfjy0b2/cUZwOp06dokGDBmzfvp2mTZvi7+/Pc889Z9vcQoj/kIIgHt3t27BkiVEItm6FHDmgRQujNdCwYZqtgXvi4uIYO3YsQ4cOJV++fPz444989NFHMhidECaRgiDSLyLCGFjuxx+Ngeaefx7GjDFuInN1faRV7d27F29vb/bv30/Dhg0JCgqiaNGitskthEgXKQji4e7cgWXLjNZAWJgxCf177xmtgUaN4BHP8d+6dYvhw4czduxYXF1dWbp0KYUKFZJiIIQDkIIgUnb8uNEamD0bLl0yJqIfORI++QSKFXusVW7btg1vb2/+/vtvOnbsyLhx4yhUqFCmnY5QCGcjBUH8v7g4WLHCaA2EhICLC7zzjtEaeO21R24N3HPjxg0GDhzIlClTKF26NL/++iuvvvqqlcMLITJKCoKAyEiYMQNmzTImqC9VCoYPB29v487iDFi3bh1+fn6cPn2anj178s0335AvXz7r5BZCWJUUhKwqPh5WrzZaA+vXGz2D3nzTaA28/rrROsiAqKgoevfuzbx586hYsSLbt2/P9NMPCuHspCBkNadOQWCgMcfA2bNGC2DwYGPymVKlMrx6rTWLFy+me/fuXLlyhcGDBzNo0CBy5cplhfBCCFuSgpAVWCywbp3RGli71hh6+vXXYepUo1WQ3TqHwblz5+jatSvLly+nRo0abNiwgWrVqlll3UII25OCkInlvHTJuBYQGAj//mv0DhowADp1MnoNWYnWmtmzZ9OnTx/u3LnDmDFj6N27N9mtVGiEEPZhs/+xSqmhQCfgUuJTX2it19pqeyJRQgJs2AD+/tRZudJ4/NprMGGC0WPIyiOGnjx5El9fX0JCQmjQoAEzZsygfPnyVt2GEMI+bP0n3ASt9Tgbb0OA0Tto1iyjt1BkJLi68u/77/Ps119DuXJW35zFYmHy5Ml88cUXuLi4MG3aNHx9fWUwOiGcmPzvdWYJCcb9Aq1aGReEBw2CMmWMOQhOn+aEr69NisHhw4epV68evXr1wtPTkz///JPOnTtnimIQHh7OqFGjCA8PNzuK1WXmn01Yh61bCN2VUu2B3cBnWuurNt5e1nDxIsyZY9xJfPy4Mapoz57g6ws2PF1z9+5dvv32W7755hvy58/PTz/9RNu2bTPNYHTh4eE0btyYu3fvkjNnTjZu3Jhpuspm5p9NWE+GCoJSKgRIaRyDQcA04GtAJ/77HdAxhXX4Ar4Arq6uTjGMQUxMjP1zak3B/fspvmoVT23dSrb4eK5VrcrZL77gcsOGJOTMaXQjPXvWJjkjIiIYM2YMJ06coFGjRnTv3p1ChQqxefPmDK/blP2Zgvnz53Pnzh0SEhK4c+cOs2bN4s6dOwBcu3YNi8XiEDkfJrV9+bCfzQyO8jtPi7PktBqttc2/gNLAobSWK1++vHYGoaGh9tvY5ctaf/ed1hUqaA1aFyyodc+eWv/5Z5pvtUbO2NhY/fnnn+ts2bLp4sWL6xUrVmR4nQ+y6/58iB07dug8efJoFxcXnSdPHr1jx46k1xo2bKirVatmXrh0Sm1fPuxnM4Oj/M7T4iw5gd3aCp/Vtuxl9IzW+lziw2bAIVttK9PRGrZtM+4bWLzYGHG0bl1j2OlWrYy5ie1g8+bN+Pj4cOzYMTp16sSYMWMoWLCgXbZthjp16rBx40bCwsLw9PTMVKdUMvPPJqzHltcQxiil3DFOGUUCfjbcVuZw9aoxD7G/vzET2ZNPGncQ+/lBlSp2ixEdHU3//v2ZPn06ZcuWZePGjTRq1Mhu2zdTnTp1Mu2HZWb+2YR12KwgaK0/stW6MxWtYedOowgEBRmzkdWqZQwt8f77kDevXeOsWbOGzp07c/bsWfr06cPw4cPJa+cMQghzyK2kZrl+HX76ySgEBw9CvnzGzGN+fuDubvc4ly9fplevXsyfP59KlSqxePFiateubfccQgjzSEGwJ61h926jCCxcCLGxUL268bhNG8if34RImqCgID799FOuX7/OV199xRdffEHOnDntnkUIYS4pCPZw4wYsWGB88O/bB088YRSAzp3Bw8O0WGfOnKFr166sXLmSmjVrMnPmTKrY8VqFEMKxSEGwpX37jCIwfz7ExEDVqjBlCrRrBwUKmBZLa01gYCB9+/YlLi6OcePG0atXL1wyOAeCEMK5SUGwtps3jaEj/P1h1y6ji+j77xvXBmrXNiaiMdHx48fp1KkToaGheHp6MmPGDJ5//nlTMwkhHIMUBGs5eNAoAvPmQXQ0uLnBxInw0UdQqJDZ6bBYLEycOJEvv/ySHDlyEBAQgLe3d6YYf0gIYR1SEDLi1i0IDjYKQXg45Mpl3Djm5wevvGJ6a+CeQ4cO4e3tze+//87bb7/NtGnTKFGihNmxhBAORgrCY3giMtIYTG7uXLh2DSpUgPHjoX17Y6A5BxEXF8ewYcMYMWIEBQoUYOHChbz//vuZZjA6IYR1SUFIr9u3YckS8Pen1tatxkQzLVoYrYGGDR2mNXDP77//jp+fHydPnqRt27ZMnDiRp556yuxYQggHJieQ0xIRAZ99BiVLwocfwrlzHPfzgzNnjHsJPD0dqhjExsby2WefUadOHW7cuMGqVauYP3++FAMhRJqkhZCSO3dg2TLj2kBYmDEJ/XvvGa2BRo34d8sWyrm6mp3yP0JDQ/Hx8eHEiRP4+fnx1ltv8dZbb5kdSwjhJKSFkNzx49C/vzH7WJs2xlSUI0caE9T//DO8+io4YK+c69ev4+vrS6NGjciWLRuhoaFMnz6dfPnymR1NCOFEpIUQFwcrVhitgZAQcHExJqP38zMmp3fAApDcqlWr6Ny5M+fPn+fzzz9n6NChPPHEE2bHEkI4oaxbECIjjQnpZ80yJqgvVQqGDwdvbyhe3Ox0abp06RI9evRg0aJFVKlShRUrVuBh4jAYQgjnl7UKQnw8rF5ttAbWrzcuBr/5ptEaeP11o3Xg4LTWLFy4kB49ehAdHc3w4cPp37+/DEYnhMiwrFEQTp2CwEBjjoGzZ40WwODBxuQzpUqZnS7d/v33X7p06cKaNWuoXbs2M2fOpFKlSmbHEkJkEpm3IFgssG6d0RpYu9YYevr112HqVKNVkN15fvSEhAQCAgLo168fFouFCRMm8Omnn8pgdEIIq3KeT8X0OnPGaAkEBhq9g4oVgwEDoFMnKF3a7HSP7OjRo3Tq1InNmzfTuHFjAgICKFu2rNmxhBCZUOYoCAkJsGGD0RpYtcpoHbz2GkyYYPQYypHD7ISPLD4+ngkTJjBkyBBy5crFzJkz+eSTT2TYCSGEzTh3QTh/3uglNGOG0WvI1RX69jVaA+XKmZ3usR04cABvb292797Nu+++y9SpUynuBD2fhBDOzfkKQkICbNpktAaWLzd6Dnl5wejR0KwZOHFvmzt37jBixAhGjRpF4cKFCQ4OpmXLltIqEELYhfMUhIsXYc4cCAgw7iguUsQYcdTXF8qXNztdhoWHh+Pt7c2RI0f46KOPmDBhAkUcaORUIUTm59gFQWtjLCF/f1i61LiruH59GDbMGGk0d26zE2bYzZs3GTRoED/88AMlS5Zk7dq1vPHGG2bHEkJkQY5ZEKKi4McfjdZARAQULAhduxqtATc3s9NZTUhICJ06dSIyMpKuXbsyatQonnzySbNjCSGyKIcqCC63bhlDTC9ebIw4WreuURhatTLmJs4krl27xmeffcasWbN44YUX2LJlC/Xr1zc7lhAii3OogvDEv/8a3UZ9fIzhJKpUMTuS1S1fvpyuXbty8eJFBgwYwJAhQ8iTiYqdEMJ5OVRBuF20qHHBOG9es6NY3YULF/j000/5+eefqVatGqtWraJGjRpmxxJCiCQZGttZKdVKKfWnUipBKeXxwGsDlVLHlFIRSqmm6VlfXIECma4YaK2ZN28ebm5urFixghEjRrBr1y4pBkIIh5PRFsIhoDngn/xJpZQb8AFQCSgOhCilymutLRncnlM5deoUfn5+/PLLL9StW5fAwEAqVqxodiwhhEhRhloIWusjWuuIFF56F1iktb6jtT4JHANqZWRbziQhIYEpU6ZQqVIltm7dyg8//MDWrVulGAghHJqtriGUAHYme3w68blMLyIiAh8fH7Zt28Zrr71GQEAApZ1wUD0hRNaTZkFQSoUAxVJ4aZDWekVGAyilfAFfAFdXV8LCwjK6SpuLiYn5T06LxUJQUBBz5swhV65c9O/fn6ZNmxIZGUlkZKTD5HREzpDz2rVrWCwWh8/pDPsSJKfD0lpn+AsIAzySPR4IDEz2eD1QJ631lC9fXjuD0NDQ+x7v27dPV69eXQO6efPm+ty5c+YEe8CDOR2VM+Rs2LChrlatmtkx0uQM+1JryWltwG5thc9yW80gvxL4QCmVSylVBngB+N1G2zLN7du3GTRoEB4eHpw5c4bFixezZMkSihVLqUElhPMLDw9n1KhRhIeHmx1F2ECGriEopZoBkwBXYI1Sar/WuqnW+k+lVDBwGIgHuulM1sNo+/bteHt7ExERQYcOHfjuu+8oXLiw2bGEsJnw8HAaN27M3bt3yZkzJxs3bqROnTpmxxJWlNFeRsu01iW11rm01kW11k2TvTZCa11Oa11Ba70u41EdQ0xMDD/88AP169fn9u3brF+/ntmzZ0sxEJleWFgYd+/exWKxcPfu3ax1bj2LsNUpo0xpw4YNVK5cmeXLl9O9e3cOHTpEkyZNzI4lhF14enqSM2dOXFxcyJkzJ56enmZHElbmUENXOKorV67w2WefMWfOHCpUqMDEiRP59NNPzY4lhF3VqVOHjRs3EhYWhqenp5wuyoSkIKRhyZIldOvWjcuXL/PFF18wePBgdu7cmfYbhciE6tSpI4UgE5OCkIpz587RvXt3li5dyksvvcQvv/yCu7u72bGEEMJm5BrCA7TWzJkzBzc3N9asWcPo0aP57bffpBgIITI9aSEkExkZia+vL7/++iv16tUjMDCQChUqmB1LCCHsQloIGIPRTZo0icqVKxMeHs6UKVPYvHmzFAMhRJaS5VsIR44cwcfHhx07dvD6668zffp0nnvuObNjCSGE3WXZFkJcXBwjR47E3d2dv/76i7lz57J27VopBkKILCtLthD27t1Lx44d+eOPP2jVqhWTJk2iaNGiZscSQghTZakWwq1btxgwYAC1atXiwoULLF26lODgYCkGQgircPbB/7JMC2Hr1q34+Pjw999/4+3tzdixYylUqJDZsYQQmURmGPwv07cQbty4Qbdu3WjQoAF3797l119/JTAwUIqBEMKqMsPgf5m6IKxbt45KlSoxbdo0evXqxaFDh3j11VfNjiWEyIQyw+B/mfKUUVRUFL1792bevHlUrFiR7du3O13TTTxceHi4DLImHEpmGPwvUxUErTU///wz3bt35+rVqwwePJhBgwaRK1cus6MJK8oM52pF5uTsg/9lmlNGZ8+epXnz5rz//vs8++yz7Nmzh+HDh0sxyIQyw7laIRyR0xcErTUzZ87Ezc2NX375hTFjxrBz506qVq1qdjRhI5nhXK0QjsipTxmdOHECX19fNm7cSIMGDQgMDOSFF14wO5awscxwrlYIR+SUBcFisTBp0iQGDRqEi4sL06ZNw9fXl2zZnL7BI9LJ2c/VCuGInK4gHD58GG9vb3bu3Mn//vc/pk+fTqlSpcyOJYQQTs9p/qS+e/cuX3/9Ne7u7hw9epSffvqJ1atXSzEQQggrcYoWwq5du/D29ubgwYN88MEHTJw4kaefftrsWEIIkak4dAshNjaWfv368fLLLxMVFcWKFStYuHChFAMhhLABh20hbN68GR8fH44dO0anTp0YO3YsBQoUMDuWEEJkWg7XQoiOjqZLly54enqSkJDAxo0bCQgIkGIghBA25lAthJs3b1KpUiXOnj1Lnz59+Prrr3niiSfMjiWEEFmCQxWEM2fOUKlSJRYvXkzt2rXNjiOEEFmK0lqbnSGJUuoGEGF2jnR4Crhsdoh0kJzW5Qw5nSEjSE5rq6C1zp/RlThUCwGI0Fp7mB0iLUqp3ZLTeiSn9ThDRpCc1qaU2m2N9TjcRWUhhBDmkIIghBACcLyCEGB2gHSSnNYlOa3HGTKC5LQ2q+R0qIvKQgghzONoLQQhhBAmsXtBUEq1Ukr9qZRKUEp5PPDaQKXUMaVUhFKqaSrvL6OU+i1xuSClVE47ZA5SSu1P/IpUSu1PZblIpdTBxOWsctX/USilhiqlziTL+r9Ulns9cR8fU0oNMCHnWKXUX0qpA0qpZUqpgqksZ/f9mda+UUrlSjwejiUeh6XtkeuBDKWUUqFKqcOJ/5d6prCMp1LqerJjYYi9cybmeOjvUBl+SNyfB5RS1U3IWCHZftqvlIpWSvV6YBlT9qdSapZS6qJS6lCy5worpX5VSh1N/LdQKu/9OHGZo0qpj9O1Qa21Xb+AikAFIAzwSPa8G/AHkAsoAxwHXFJ4fzDwQeL304Euds7/HTAkldcigafsvU+TbX8o0DeNZVwS921ZIGfiPnezc84mQPbE778FvnWE/ZmefQN0BaYnfv8BEGTC7/kZoHri9/mBv1PI6Qmstne2R/0dAv8D1gEKeBn4zeS8LsB54DlH2J9AA6A6cCjZc2OAAYnfD0jp/w9QGDiR+G+hxO8LpbU9u7cQtNZHtNYp3Xz2LrBIa31Ha30SOAbUSr6AUkoBjYDFiU/9CLxnw7j3Sdx+a2ChvbZpA7WAY1rrE1rru8AijH1vN1rrDVrr+MSHO4GS9tz+Q6Rn37yLcdyBcRw2Tjwu7EZrfU5rvTfx+xvAEaCEPTNY0bvAXG3YCRRUSj1jYp7GwHGt9T8mZkiitd4CXHng6eTHYGqfgU2BX7XWV7TWV4FfgdfT2p4jXUMoAfyb7PFp/nuQFwGuJfswSWkZW6oPXNBaH03ldQ1sUErtUUr52jFXct0Tm96zUmlKpmc/21NHjL8QU2Lv/ZmefZO0TOJxeB3juDRF4imrl4DfUni5jlLqD6XUOqVUJfsmS5LW79DRjscPSP0PPkfYnwBFtdbnEr8/DxRNYZnH2q82uVNZKRUCFEvhpUFa6xW22GZGpTNzGx7eOqintT6jlHoa+FUp9VdihbdLTmAa8DXGf8KvMU5vdbTm9tMrPftTKTUIiAfmp7Iam+9PZ6aUygcsAXppraMfeHkvxmmPmMRrScuBF+wcEZzod5h4PfIdYGAKLzvK/ryP1lorpazWVdQmBUFr/epjvO0MkHw+zJKJzyUXhdGkzJ7411lKyzyWtDIrpbIDzYEaD1nHmcR/LyqllmGcgrDqwZ/efauUmgGsTuGl9OznDEvH/uwAvAU01oknPVNYh8335wPSs2/uLXM68ZgogHFc2pVSKgdGMZivtV764OvJC4TWeq1SaqpS6imttV3H5UnH79Aux2M6vQHs1VpfePAFR9mfiS4opZ7RWp9LPL12MYVlzmBc97inJMZ124dypFNGK4EPEntxlMGovr8nXyDxgyMUaJn41MeAvVocrwJ/aa1Pp/SiUiqvUir/ve8xLpweSmlZW3ng3GuzVLa/C3hBGb21cmI0kVfaI989SqnXgX7AO1rr2FSWMWN/pmffrMQ47sA4DjelVtBsJfGaxUzgiNZ6fCrLFLt3bUMpVQvj/7pdC1c6f4crgfaJvY1eBq4nOx1ib6meAXCE/ZlM8mMwtc/A9UATpVShxFPHTRKfezgTrpo3wzifdQe4AKxP9togjF4eEcAbyZ5fCxRP/L4sRqE4BvwM5LJT7jlA5weeKw6sTZbrj8SvPzFOjdh7384DDgIHEg+aZx7Mmfj4fxg9U46blPMYxvnN/Ylf0x/Madb+TGnfAMMxihdA7sTj7ljicVjWhP1XD+O04IFk+/B/QOd7xyjQPXG//YFx4b6uCTlT/B0+kFMBUxL390GS9Ty0c9a8GB/wBZI9Z/r+xChQ54C4xM9Nb4xrVhuBo0AIUDhxWQ8gMNl7OyYep8eAT9KzPblTWQghBOBYp4yEEEKYSAqCEEIIQAqCEEKIRFIQhBBCAFIQhBBCJJKCIIQQApCCIIQQIpEUBCGEEAD8HwqlZ+gLK7A/AAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "predicted_boundary_function = lambda x1: (-1 * p.weights[0] / p.weights[1]) * x1 - p.bias / p.weights[1]\n",
    "\n",
    "plt.grid()\n",
    "plt.axhline(0, color=\"black\")\n",
    "plt.axvline(0, color=\"black\")\n",
    "plt.xlim([X1_LOWER_BOUND, X1_UPPER_BOUND])\n",
    "plt.ylim([X2_LOWER_BOUND, X2_UPPER_BOUND])\n",
    "plt.axline(\n",
    "    (X1_LOWER_BOUND, TRUE_BOUNDARY_FUNCTION(X1_LOWER_BOUND)),\n",
    "    (X1_UPPER_BOUND, TRUE_BOUNDARY_FUNCTION(X1_UPPER_BOUND)),\n",
    "    color='black',\n",
    "    label='True Boundary'\n",
    ")\n",
    "plt.axline(\n",
    "    (X1_LOWER_BOUND, predicted_boundary_function(X1_LOWER_BOUND)),\n",
    "    (X1_UPPER_BOUND, predicted_boundary_function(X1_UPPER_BOUND)),\n",
    "    color='red',\n",
    "    label='Predicted Boundary'\n",
    ")\n",
    "for i in range(0, NUM_SAMPLES):\n",
    "    plt.plot(samples[i][0], samples[i][1], color=\"black\", marker=\".\" if true_classifications[i] == 0 else \"x\")\n",
    "plt.legend()\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "108e7c31-b116-4cab-91a5-f2575b719002",
   "metadata": {},
   "source": [
    "Let me look at the graph of the total loss per epoch as well:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "id": "3e637b75-6646-47b8-8d6e-ca7768420038",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYUAAAEWCAYAAACJ0YulAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/YYfK9AAAACXBIWXMAAAsTAAALEwEAmpwYAAAYc0lEQVR4nO3de7QlZX3m8e9jNxfljt3DAA00xNaVJgmCHQbUlTDiBRiFtSYaIRIvIWHGDCqDOjbRIYRx1vIyMUpEBUeHiAriJaajHdFwMcYL0qigNHZsEaURQiOCokZp+M0fVafYHHqfvfuyz+4++/tZa69T9da7q946dc55TlW9+61UFZIkATxm3A2QJG07DAVJUsdQkCR1DAVJUsdQkCR1DAVJUsdQkLYjSW5Kcsy426G5y1DQNinJrUmeOaZtPzXJVUl+muS+JH+fZOksbfv+ntdDSX7RM/+iqjq0qq6ZjbZoMhkKUo8kRwOfBf4O2A84GLgB+GKSQ7bytpLkEb+DVbXr1Av4AfC8nrIPbc3tSxtjKGi7kmSnJG9P8sP29fYkO7XLFiT5VJJ7k9yT5AtTf3STvC7J7e1//2uSHNtnE28BPlBV76iqn1bVPVX1BuArwLntum5O8tyeNs1Psj7JEe38UUm+1Lbjht7LPUmuSfK/k3wR+DmwSUHTewaV5NwkH03ywXa/vpnkiUnOTnJXktuSPLvnvXskeV+SO9rvxRuTzNuU7WvuMxS0vXk9cBTwZOAw4EjgDe2yVwPrgIXAPsCfAZXkScAZwG9X1W7Ac4Bbp684yeOApwIf3ch2Lwee1U5fCpzSs+w5wN1V9bUk+wOfBt4I7A28Bvh4koU99f8QOB3YDfj+8Lu+Uc8DLgH2Ar4OXEHze70/cB5wYU/di4ENwBOAw4FnA3+8hdvXHGMoaHvzIuC8qrqrqtYDf0HzRxbgAWBf4KCqeqCqvlDN4F4PAjsBS5PsUFW3VtV3N7LuvWl+J+7YyLI7gAXt9IeBE9sQAfgDmqAAOBVYWVUrq+qhqvocsAo4oWddF1fVTVW1oaoe2IzvQa8vVNUVVbWBJswWAm9q13sZsDjJnkn2adtwZlX9rKruAv4KOHkLt685xlDQ9mY/Hvnf9ffbMoC3AmuBzya5JclygKpaC5xJc/nnriSXJdmPR/sx8BBNsEy3L3B3z/puBp7XBsOJNEEBcBDwgvbS0b1J7gWePm2dt23KDg/wrz3Tv6A5Y3mwZx5g17ZdOwB39LTrQuDfbcW2aA4wFLS9+SHNH7gpB7ZltPcAXl1Vh9D8oT5r6t5BVX24qp7evreAN09fcVX9DPgy8IKNbPf3gSt75qcuIZ0ErG6DApo/+JdU1Z49r12q6k29m9rkvd5ytwG/BBb0tGv3qjp0DG3RNsxQ0LZshyQ797zm0/wxfkOShUkWAOcAHwRI8twkT0gS4D6ay0YPJXlSkme0N6T/jeY/6If6bHM58JIkr0yyW5K9krwROJrmUtWUy2iuyb+ch88SaNvyvCTPSTKvbfcxSRZtrW/K5qiqO2h6Vf1lkt2TPCbJryX53XG2S9seQ0HbspU0f8CnXufS3MBdBdwIfBP4WlsGsAT4R+B+mv/431VVV9PcT3gTzeWfO2kumZy9sQ1W1T/T3Dj+zzT3Eb5Pc1P26VX1nZ56d7TbeCrwkZ7y22jOHv4MWE/zH/pr2TZ+114M7AisprlU9jE2fqlMEyw+ZEeSNGVb+O9FkrSNMBQkSR1DQZLUMRQkSZ35427AplqwYEEtXrx43M2QpO3K9ddff3dVLRxUb7sLhcWLF7Nq1apxN0OStitJhhpny8tHkqSOoSBJ6hgKkqSOoSBJ6hgKkqTOyEIhyfvbRwJ+q8/yJDk/ydokN049ylCSND6jPFO4GDhuhuXH04xquYTm0YTvHmFbJElDGFkoVNU/AffMUOUkmgekV1V9BdgzyciG8b3u1nu4+Ivfw1FhJam/cd5T2J9HPpZwXVv2KElOT7Iqyar169dv1sYu/tKtnPv3q7ntnl8MrixJE2q7uNFcVRdV1bKqWrZw4cBPaW/Us359HwAe9ExBkvoaZyjcDhzQM7+oLZMkjck4Q2EF8OK2F9JRwH3tIw4lSWMysgHxklwKHAMsSLIO+HNgB4Cqeg/N83dPANYCPwdeNqq2SJKGM7JQqKpTBiwv4L+NavuSpE23XdxoliTNjokLBT+nIEn9TUwoJONugSRt+yYmFCRJgxkKkqSOoSBJ6hgKkqSOoSBJ6kxcKNghVZL6m7hQkCT1ZyhIkjqGgiSpYyhIkjqGgiSpYyhIkjoTEwppR8RzkFRJ6m9iQkGSNJihIEnqGAqSpI6hIEnqGAqSpI6hIEnqTEwoPPyIZvukSlI/ExMKkqTBDAVJUsdQkCR1DAVJUsdQkCR1DAVJUmdiQqEdJNVRUiVpBhMTCpKkwUYaCkmOS7Imydokyzey/MAkVyf5epIbk5wwyvZIkmY2slBIMg+4ADgeWAqckmTptGpvAC6vqsOBk4F3jao9kqTBRnmmcCSwtqpuqapfAZcBJ02rU8Du7fQewA9H2B5J0gCjDIX9gdt65te1Zb3OBU5Nsg5YCbxiYytKcnqSVUlWrV+/fhRtlSQx/hvNpwAXV9Ui4ATgkiSPalNVXVRVy6pq2cKFC2e9kZI0KUYZCrcDB/TML2rLep0GXA5QVV8GdgYWjKIxacdJtUeqJPU3ylC4DliS5OAkO9LcSF4xrc4PgGMBkvw6TSh4fUiSxmRkoVBVG4AzgCuAm2l6Gd2U5LwkJ7bVXg38SZIbgEuBl1b58TJJGpf5o1x5Va2kuYHcW3ZOz/Rq4GmjbIMkaXjjvtEsSdqGGAqSpM7EhEIyuI4kTbqJCYUp3saWpP4mLhQkSf0ZCpKkjqEgSeoYCpKkjqEgSepMTCjYI1WSBpuYUJhSjpMqSX1NXChIkvozFCRJHUNBktQxFCRJHUNBktSZmFBwlFRJGmxiQmGKo6RKUn8TFwqSpP4MBUlSx1CQJHUMBUlSx1CQJHUmKBTskypJg0xQKDTskipJ/U1cKEiS+jMUJEkdQ0GS1DEUJEkdQ0GS1BlpKCQ5LsmaJGuTLO9T5/eTrE5yU5IPj64to1qzJM0dA0MhydOS7NJOn5rkbUkOGuJ984ALgOOBpcApSZZOq7MEOBt4WlUdCpy56buwaQr7pEpSP8OcKbwb+HmSw4BXA98FPjDE+44E1lbVLVX1K+Ay4KRpdf4EuKCqfgxQVXcN3XJJ0lY3TChsqKqi+YP+zqq6ANhtiPftD9zWM7+uLev1ROCJSb6Y5CtJjtvYipKcnmRVklXr168fYtOSpM0xTCj8NMnZwKnAp5M8BthhK21/PrAEOAY4BXhvkj2nV6qqi6pqWVUtW7hw4VbatCRpumFC4YXAL4HTqupOYBHw1iHedztwQM/8oras1zpgRVU9UFXfA/6FJiQkSWMw1JkC8I6q+kKSJwJPBi4d4n3XAUuSHJxkR+BkYMW0Op+kOUsgyQKay0m3DNVySdJWN0wo/BOwU5L9gc8CfwhcPOhNVbUBOAO4ArgZuLyqbkpyXpIT22pXAD9Kshq4GnhtVf1o03djMHukStJg84eok6r6eZLTgHdV1VuS3DDMyqtqJbByWtk5PdMFnNW+ZoWjpEpSf8OcKSTJ0cCLgE9vwvskSduZYf64n0nzAbO/bS//HEJzqUeSNMcMvHxUVZ8HPp9k1yS7VtUtwCtH3zRJ0mwbZpiL30zydeAmYHWS65McOvqmSZJm2zCXjy4Ezqqqg6rqQJqhLt472mZtfXFEPEkaaJhQ2KWqunsIVXUNsMvIWiRJGpthuqTekuR/Ape086fiB8wkaU4a5kzhj4CFwCeAjwMLgJeNslGSpPEYpvfRj5nW2yjJR2jGRJIkzSGb+yG0o7dqKyRJ2wQ/mSxJ6vS9fJTkiH6L2HrPU5g1dkiVpMFmuqfwlzMs+/bWbogkafz6hkJV/cfZbMhscZRUSerPewqSpI6hIEnqGAqSpM7m9D4CoKq+tvWbI0kap83tfVTAM7ZyW0bKQVIlabCJ630kSepvmFFSSfIbwFJg56myqvrAqBo1SoV9UiWpn4GhkOTPgWNoQmElcDzwz8B2GQqSpP6G6X30fOBY4M6qehlwGLDHSFslSRqLYULhF1X1ELAhye7AXcABo22WJGkchrmnsCrJnjTPZb4euB/48igbJUkaj2EesvOn7eR7knwG2L2qbhxts7Y+u6RK0mADLx8luXJquqpuraobe8skSXPHTJ9o3hl4HLAgyV48/EiC3YH9Z6FtI+EoqZLU30yXj/4LcCawH9A7pMVPgHeOsE2SpDGZ6RPN7wDekeQVVfXXs9gmSdKYDNP76MIkrwR+p52/Briwqh4YWaskSWMxzOcU3gU8pf06Nf3uYVae5Lgka5KsTbJ8hnq/l6SSLBtmvZKk0ZjpRvP8qtoA/HZVHdaz6KokNwxacZJ5wAXAs4B1wHVJVlTV6mn1dgNeBVy7OTswrGCfVEkaZKYzha+2Xx9M8mtThUkOAR4cYt1HAmur6paq+hVwGXDSRur9L+DNwL8N12RJ0qjMFApT/1q/Brg6yTVJrgGuAl49xLr3B27rmV/HtK6s7YN8DqiqT8+0oiSnJ1mVZNX69euH2HR/9kiVpP5mutG8MMlZ7fSFwLx2+kHgcODqLdlwkscAbwNeOqhuVV0EXASwbNky/65L0ojMFArzgF3hURfj5wO7DbHu23nkwHmL2rIpuwG/AVyTZgyKfw+sSHJiVa0aYv2SpK1splC4o6rO24J1XwcsSXIwTRicDPzB1MKqug9YMDXfXpp6jYEgSeMzzD2FzdL2XDoDuAK4Gbi8qm5Kcl6SE7dk3ZvFzkeSNNBMZwrHbunKq2olzdPaesvO6VP3mC3dniRpy/Q9U6iqe2azIZKk8RvmE81zSjlMqiT1NXGhIEnqz1CQJHUMBUlSZ2JCwR6pkjTYxISCJGkwQ0GS1Jm4ULBDqiT1N3GhIEnqz1CQJHUMBUlSZ2JCoX1mgyRpBhMTCpKkwQwFSVJn4kLBQVIlqb+JCwVJUn+GgiSpYyhIkjoTEwp2SJWkwSYmFCRJgxkKkqTOBIaCfVIlqZ8JDAVJUj+GgiSpYyhIkjoTEwoOkipJg01MKEiSBjMUJEmdiQsFR0mVpP5GGgpJjkuyJsnaJMs3svysJKuT3JjkyiQHjbI9kqSZjSwUkswDLgCOB5YCpyRZOq3a14FlVfVbwMeAt4yqPZKkwUZ5pnAksLaqbqmqXwGXASf1Vqiqq6vq5+3sV4BFI2yPJGmAUYbC/sBtPfPr2rJ+TgP+YWMLkpyeZFWSVevXr9+sxsRxUiVpoG3iRnOSU4FlwFs3tryqLqqqZVW1bOHChbPbOEmaIPNHuO7bgQN65he1ZY+Q5JnA64HfrapfjrA9kqQBRnmmcB2wJMnBSXYETgZW9FZIcjhwIXBiVd01wrZIkoYwslCoqg3AGcAVwM3A5VV1U5LzkpzYVnsrsCvw0STfSLKiz+q2XrtGvQFJ2o6N8vIRVbUSWDmt7Jye6WeOcvuSpE2zTdxong0OiCdJg01MKEiSBjMUJEkdQ0GS1DEUJEmdiQsFh86WpP4mLhQkSf1NTCjYI1WSBpuYUJAkDWYoSJI6hoIkqWMoSJI6ExcKZZ9USepr4kJBktTf5ISCfVIlaaDJCQVJ0kCGgiSpYyhIkjqGgiSpM3GhYIdUSepv4kJBktTfxIRC7JMqSQNNTChIkgYzFCRJHUNBktQxFCRJnYkLBQdJlaT+Ji4UJEn9TUwoxB6pkjTQxISCJGkwQ0GS1BlpKCQ5LsmaJGuTLN/I8p2SfKRdfm2SxaNsjyRpZiMLhSTzgAuA44GlwClJlk6rdhrw46p6AvBXwJtH1R5J0mDzR7juI4G1VXULQJLLgJOA1T11TgLObac/BrwzSapG13H0NR+9gcftOG9Uq5ekkXnlsUt43mH7jXQbowyF/YHbeubXAf+hX52q2pDkPuDxwN29lZKcDpwOcOCBB25WYw7db3de8JRF/OxXGzbr/ZI0bns8doeRb2OUobDVVNVFwEUAy5Yt26yziN123oG3vuCwrdouSZprRnmj+XbggJ75RW3ZRuskmQ/sAfxohG2SJM1glKFwHbAkycFJdgROBlZMq7MCeEk7/XzgqlHeT5AkzWxkl4/aewRnAFcA84D3V9VNSc4DVlXVCuB9wCVJ1gL30ASHJGlMRnpPoapWAiunlZ3TM/1vwAtG2QZJ0vD8RLMkqWMoSJI6hoIkqWMoSJI62d56gCZZD3x/M9++gGmflp4A7vNkcJ8nw5bs80FVtXBQpe0uFLZEklVVtWzc7ZhN7vNkcJ8nw2zss5ePJEkdQ0GS1Jm0ULho3A0YA/d5MrjPk2Hk+zxR9xQkSTObtDMFSdIMDAVJUmdiQiHJcUnWJFmbZPm427MpkhyQ5Ookq5PclORVbfneST6X5Dvt173a8iQ5v93XG5Mc0bOul7T1v5PkJT3lT0nyzfY95yfJ7O/poyWZl+TrST7Vzh+c5Nq2nR9ph2UnyU7t/Np2+eKedZzdlq9J8pye8m3uZyLJnkk+luTbSW5OcvRcP85J/nv7c/2tJJcm2XmuHeck709yV5Jv9ZSN/Lj228aMqmrOv2iG7v4ucAiwI3ADsHTc7dqE9u8LHNFO7wb8C7AUeAuwvC1fDry5nT4B+AcgwFHAtW353sAt7de92um92mVfbeumfe/x497vtl1nAR8GPtXOXw6c3E6/B3h5O/2nwHva6ZOBj7TTS9vjvRNwcPtzMG9b/ZkA/gb443Z6R2DPuXycaR7J+z3gsT3H96Vz7TgDvwMcAXyrp2zkx7XfNmZs67h/CWbpgBwNXNEzfzZw9rjbtQX783fAs4A1wL5t2b7Amnb6QuCUnvpr2uWnABf2lF/Ylu0LfLun/BH1xrifi4ArgWcAn2p/4O8G5k8/rjTP7Ti6nZ7f1sv0Yz1Vb1v8maB58uD3aDuATD9+c/E48/Bz2vduj9ungOfMxeMMLOaRoTDy49pvGzO9JuXy0dQP3pR1bdl2pz1dPhy4Ftinqu5oF90J7NNO99vfmcrXbaR83N4O/A/goXb+8cC9VbWhne9tZ7dv7fL72vqb+r0Yp4OB9cD/ay+Z/d8kuzCHj3NV3Q78H+AHwB00x+165vZxnjIbx7XfNvqalFCYE5LsCnwcOLOqftK7rJp/BeZM/+IkzwXuqqrrx92WWTSf5hLDu6vqcOBnNKf8nTl4nPcCTqIJxP2AXYDjxtqoMZiN4zrsNiYlFG4HDuiZX9SWbTeS7EATCB+qqk+0xf+aZN92+b7AXW15v/2dqXzRRsrH6WnAiUluBS6juYT0DmDPJFNPDOxtZ7dv7fI9gB+x6d+LcVoHrKuqa9v5j9GExFw+zs8EvldV66vqAeATNMd+Lh/nKbNxXPtto69JCYXrgCVtj4YdaW5QrRhzm4bW9iR4H3BzVb2tZ9EKYKoHwkto7jVMlb+47cVwFHBfewp5BfDsJHu1/6E9m+Z66x3AT5Ic1W7rxT3rGouqOruqFlXVYprjdVVVvQi4Gnh+W236Pk99L57f1q+2/OS218rBwBKam3Lb3M9EVd0J3JbkSW3RscBq5vBxprlsdFSSx7VtmtrnOXuce8zGce23jf7GeZNplm/ynEDTa+e7wOvH3Z5NbPvTaU77bgS+0b5OoLmWeiXwHeAfgb3b+gEuaPf1m8CynnX9EbC2fb2sp3wZ8K32Pe9k2s3OMe//MTzc++gQml/2tcBHgZ3a8p3b+bXt8kN63v/6dr/W0NPbZlv8mQCeDKxqj/UnaXqZzOnjDPwF8O22XZfQ9CCaU8cZuJTmnskDNGeEp83Gce23jZleDnMhSepMyuUjSdIQDAVJUsdQkCR1DAVJUsdQkCR1DAWpleTBJN/oeW21ETWTLO4dIVPaVs0fXEWaGL+oqiePuxHSOHmmIA2Q5NYkb2nHq/9qkie05YuTXNWOeX9lkgPb8n2S/G2SG9rXU9tVzUvy3jTPDvhskse29V+Z5lkZNya5bEy7KQGGgtTrsdMuH72wZ9l9VfWbNJ8WfXtb9tfA31TVbwEfAs5vy88HPl9Vh9GMXXRTW74EuKCqDgXuBX6vLV8OHN6u57+OZtek4fiJZqmV5P6q2nUj5bcCz6iqW9qBCe+sqscnuZtmrPoH2vI7qmpBkvXAoqr6Zc86FgOfq6ol7fzrgB2q6o1JPgPcTzOsxSer6v4R76rUl2cK0nCqz/Sm+GXP9IM8fE/vP9GMdXMEcF3P6KDSrDMUpOG8sOfrl9vpL9GMugnwIuAL7fSVwMuhe8b0Hv1WmuQxwAFVdTXwOpqhoB91tiLNFv8jkR722CTf6Jn/TFVNdUvdK8mNNP/tn9KWvYLmKWmvpXli2sva8lcBFyU5jeaM4OU0I2RuzDzgg21wBDi/qu7dSvsjbTLvKUgDtPcUllXV3eNuizRqXj6SJHU8U5AkdTxTkCR1DAVJUsdQkCR1DAVJUsdQkCR1/j+xEsfa3B0IrQAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.title(\"Loss Over Time\")\n",
    "plt.xlabel(\"Epochs\")\n",
    "plt.ylabel(\"Total Loss\")\n",
    "plt.plot(loss_vals_perceptron_from_scratch)\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "77453452-e2e1-4eb9-8e56-235fc073df31",
   "metadata": {},
   "source": [
    "I can see that the predicted boundary cleanly separates the data into two classes. However, it is unlikely that the equation for the predicted boundary will be equivalent to that of the true boundary. One way I can try to get a better estimate of the true boundary is to collect more data points. While that might help a little, there are still an infinite amount of lines that can be drawn to linearly separate the data. The perceptron algorithm will not always give the same boundary line if the input data is processed in a different order. While the perceptron is good at finding a line that divides linearly seperable data, other techniques might be useful, like **support vector machines**, in order to find boundaries that might be better generalizations for unobserved data."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2945b5ad-f8a3-40ba-8c3d-447d31afea28",
   "metadata": {},
   "source": [
    "### PyTorch Implementation"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9242cb72-11d6-4d69-8dff-ff5407e78b69",
   "metadata": {},
   "source": [
    "Here is my PyTorch implementation of the binary classifier perceptron."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "id": "a0fe9d69-0104-4048-abc7-faca69ef9aee",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "class Perceptron(nn.Module):\n",
    "    def __init__(self):\n",
    "        super(Perceptron, self).__init__()\n",
    "        self.ws = nn.Linear(2, 1, bias=True)\n",
    "        self.af = nn.Sigmoid()\n",
    "    \n",
    "    def forward(self, x):\n",
    "        weighted_sum = self.ws(x)\n",
    "        return self.af(weighted_sum)\n",
    "\n",
    "p = Perceptron()\n",
    "\n",
    "samples_tensor = torch.tensor(samples)\n",
    "true_classifications_tensor = torch.FloatTensor(true_classifications).unsqueeze(dim=1)\n",
    "\n",
    "criterion = nn.MSELoss()\n",
    "optimizer = torch.optim.SGD(p.parameters(), lr=0.001)\n",
    "\n",
    "loss_pytorch_perceptron = []\n",
    "\n",
    "for epoch in range(0, 100000):\n",
    "    y_hat = p(samples_tensor)\n",
    "\n",
    "    loss = criterion(true_classifications_tensor, y_hat)\n",
    "    loss_pytorch_perceptron.append(loss.detach())\n",
    "\n",
    "    optimizer.zero_grad()\n",
    "    loss.backward()\n",
    "    optimizer.step()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f2b20677-e994-4dc4-85a3-029cc0eed3d1",
   "metadata": {},
   "source": [
    "Graph the result of the trained perceptron binary classifier:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "id": "c0b9b028-a229-4342-8bda-e0d3938f043a",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYQAAAD5CAYAAAAndkJ4AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/YYfK9AAAACXBIWXMAAAsTAAALEwEAmpwYAABACklEQVR4nO3dd1xV9RvA8c9XnLk1UnNraYIDFTXLAVlaZpqmZlppgiBKjjLNzFnmLC0HiDPTHLnS1ByIe6+MNEeu3Hsgyrh8f38cJPQHgnLvPffC8369eHnHuec8HK73ud9xnq/SWiOEEEJkMjsAIYQQjkESghBCCEASghBCiHiSEIQQQgCSEIQQQsSThCCEEAKAzNbYiVJqGtAEuKS1rhj/2CCgE3A5frMvtNYrHrWffPny6eeee84aIdnUnTt3yJkzp9lhpEjitJ7Dhw9jsVhwc3MzO5RHcoZzCRKnNcTFxXH27FkuXboEcEVr7ZrmnWqt0/wD1AOqAeGJHhsE9Hqc/ZQrV047g7CwMLNDSBWJ03rq16+vq1SpYnYYKXKGc6m1xJlWq1ev1qVKldKA7tq1qwZ2ayt8lluly0hrvRG4Zo19ifRn5MiRhIWFPfBYWFgYI0eONCkiIZzT9evX6dixIw0bNiRbtmxs2rSJ8ePHW23/th5DCFRKHVBKTVNK5bfxsYSDqlGjBq1bt05ICmFhYbRu3ZoaNWqYHJkQzmPx4sW4ubkxc+ZM+vbty/79+6lTp45Vj6G0lUpXKKVKAb/p/8YQCgFXAA18BRTRWndM4nV+gB+Aq6tr9fnz51slHluKiIggV65cZoeRIkeKc9++fQwePJimTZuydOlSBg4cSNWqVQHHijM5PXr0wGKxMG7cOLNDeSRnOJcgcT6Oa9eu8cMPP7Bhwwaee+45PvvsM8qVK/fANt7e3nu01p5pPZbNEkJqn0usfPny+vDhww88FhMTw5kzZ7h3755V4rSGe/fukT17drPDSJGjxXnjxg1u3rxJ3rx5yZcvX8Ljjxtn9uzZKVasGFmyZLFBlEnz8vLixo0b7N+/327HfBLr16/Hy8vL7DBSJHGmTGvNzJkz6dmzJ5GRkQwcOJBevXol+b5XSlklIVhlllFSlFJFtNbn4+82B8KfZD9nzpwhd+7clCpVCqWU9QJMg9u3b5M7d26zw0iRI8V569YtoqKiKFGiBJcvX6Zo0aLkyZMHeLw4tdZcvXqVM2fOULp0aVuGLIRpTp06hb+/P6tWreKll15i6tSpvPDCCzY/rlXGEJRSc4BtQHml1BmllA8wUin1p1LqAOAN9HySfd+7d4+CBQs6TDIQj+/WrVscP36cMmXKULRoUcqUKcPx48e5devWY+9LKUXBggUdqsUohLXExcUxfvx43N3d2bx5M+PGjWPTpk12SQZgpRaC1vq9JB6eao19A5IMnFxkZCRlypRJaBHkyZOHMmXKEBkZmfDY45D3g0iPDh8+jI+PD1u2bKFRo0ZMmjSJkiVL2jUGuVJZ2FzhwoX/74M/T548FC5c2KSIhHAcMTExDBs2jCpVqnDw4EFmzJjBypUr7Z4MwIZjCOnF1atXadCgAQAXLlzAxcWFggULkilTJnbu3EnWrFnTfAwvLy/Onz9Pjhw5iIqKomfPnvj5+aV5v4/SoUMHmjRpQsuWLW16HCFE8vbt24ePjw/79u2jZcuWjBs3ztQvSpIQUlCwYMGEmSWDBg0iV65c+Pv7JwyCxsbGkjlz2k/j7Nmz8fT05Nq1a5QtW5YOHTpYJdlYi7V+TyGEMTY6ZMgQRo4cydNPP83ChQtp0aKF2WFJl9GT6Ny5M507d6ZWrVr07t2bQYMGMXr06ITnK1asyMmTJwGYNWsWNWvWxMPDA39/fywWyyP3HRERQc6cOXFxcQFgzpw5VKpUiYoVK9KnT5+E7RLPjV6wYAEdOnQAjG/+3bp146WXXqJy5cosWLAAMGbnBAYGUr58eV599dX79U8AGDJkCDVq1KBixYr4+fndLz2Cl5cXPXr0wNPTk6FDh1K6dGliYmIAY6A48X0hROps3ryZKlWqMGzYMD788EMOHTrkEMkAnKyF0KNHD6vPA/fw8GDs2LGP/bozZ86wdetWXFxcGDRoUJLbHDp0iHnz5rFlyxayZMlCly5dmD17Nh9++OH/bduuXTuyZcvG0aNHGTt2LC4uLpw7d44+ffqwZ88e8ufPT8OGDVmyZAlvv/32I2M7f/48mzdvZs+ePbz33nu0bNmSxYsXc/jwYQ4ePMjFixdxc3OjY0fjOsHAwEAGDBgAwAcffMBvv/3GW2+9BUB0dDS7d+8G4OTJkyxfvpy3336buXPn0qJFC7teCyAyppEjR1KjRg28vb0THgsLC2PXrl307t3bxMgez+3bt+nbty8TJkygVKlSrFq1ioYNG5od1gOkhfCEWrVqlfAtPjmhoaHs2bOHGjVq4OHhQWhoKMePH09y29mzZ3PgwAFOnz7N6NGjOXXqFLt27cLLywtXV1cyZ85Mu3bt2LhxY4qxvf3222TKlIkXXniBixcvArBx40bee+89XFxcePbZZ3nllVcStg8LC6NWrVpUqlSJdevW8ddffyU89+677ybc9vX1Zfr06QBMnz6djz76KMVYhEir9FD6ZNWqVVSsWJGJEyfSrVs3/vzzT4dLBuBkLYQn+SZvK4lL4mbOnJm4uLiE+/fnyGutad++PcOGDUv1fl1dXalWrRo7duwgW7ZsyW6XeOrlw3PyE78upSvR7927R5cuXdi9ezfFixdn0KBBD+wv8e/58ssvc/LkSdavX4/FYqFixUdeeC6EVXh7ezN//nxat25NQEAAQUFBzJ8//4EWg6O6du0aPXv2ZObMmbzwwgts3ryZl156yeywkiUtBCsoVaoUe/fuBWDv3r2cOHECgAYNGrBgwYKE/vpr165x6tSpR+4rMjKSffv2UbZsWWrWrMmGDRu4cuUKFouFOXPmUL9+fQAKFSrEoUOHiIuLY/HixSnGWK9ePebNm4fFYuH8+fMJ37buf/g//fTTREREJIw5JOfDDz+kbdu20joQduXt7U1AQABfffUVAQEBTpEMFixYQIUKFfj555/p168f+/btc+hkAE7WQnBU77zzDjNnzsTd3Z1atWolFJ5yc3Pj66+/pmHDhsTFxZElSxYmTJiQ5Pzidu3aJUw77dChA9WrVwdg+PDheHt7o7XmzTffpFmzZgmPN2nSBFdXVzw9PYmIiHhkjM2bN2fdunW4ublRokQJateuDUC+fPno1KkTFStWpHDhwik2w9u1a8eXX37Je+8ldS2iELYRFhZGUFAQ/fv3JygoCG9vb4dNCufPn6dr164sXryYatWqsWrVKjw8PMwOK3WssaiCtX6SWiDn4MGDj7VwhD3cunXL7BBSxRZx/vLLL/r999+36j6fJE57vy9kgRzrepw4161bp59++mm9bt26JO/b0uPEGRcXp6dNm6bz5cuns2XLpocPH65jYmJsF1wiWGmBHGkhiFT7+OOPWblyJStWPHIlVCGsateuXQ+MGdwfU9i1a5fDtBJOnDiBn58fa9eupW7dukyZMuX/SlQ7A0kIItUcfS0AkT4lNbXUUbqMLBYLEyZMoG/fvmTKlImJEyfi7+9PpkzOOTwrCUEIIZ7AoUOH8PHxYdu2bbzxxhsEBwdTokQJs8NKE+dMY0IIYZKYmBiGDh2Kh4cHhw8f5qeffmL58uVOnwxAWghCCJFqe/bsoWPHjhw4cIDWrVszbtw4nnnmGbPDshppIQghRAru3r1Lnz59qFmzJpcvX2bx4sXMmzcvXSUDkISQKi4uLnh4eFCxYkVatWpFZGTkE++rQ4cOCRd/+fr6cvDgwWS3Xb9+PVu3bn3sY5QqVYorV64k+XilSpXw8PCgUqVK/Prrr4+978fl5eWVUAtJCGe0ceNGqlSpwsiRI+nYsSMHDx5MsZ6Ys5KEkAo5cuRg//79hIeHkzVrVqZNm/bA87GxsU+03ylTpuDm5pbs80+aEB4lLCyM/fv3s2DBArp162bVfVtDStVghbCXW7duMWbMGOrXr09sbCxr165l8uTJ5MuXz+zQbEYSwmOqW7cu//zzD+vXr6du3bo0bdoUNzc3LBYLn332GTVq1KBy5cpMmjQJeHTZ6cTfnn///XeqVatGlSpVaNCgASdPniQ4OJgxY8bg4eHBpk2buHz5Mu+88w41atSgRo0abNmyBTAW8WnYsCHu7u74+vqmWL8IjDd7/vz5E+5/9913VKxYkYoVKybUjDp58uQD9YpGjx6dUNnVy8sroQldrlw5Nm3aBBhN6zZt2lChQgWaN2/O3bt3E14fEBCAp6cn7u7uDBw4MOHx+6W9q1WrxvDhw6lWrVrCc0ePHn3gvhD2sGLFCipWrMiyZcvo2bMnf/75Z8JCWemZcw0q9+gBVi5/jYcHpLJoXmxsLCtXrsTLywsw6haFh4dTunRpQkJCyJs3L7t27SIqKoqXX36Zhg0bsm/fvmTLTt93+fJlOnXqxMaNGyldujTXrl2jQIECdO7cmVy5ctGrVy8A2rZtS8+ePalTpw6nT5+mUaNGHDp0iMGDB1OnTh0GDBjA8uXLmTo1+eWs75fBOH78OPPnzweMgbLp06ezY8cOtNbUqlWL+vXrP5AwkjsfO3fuZMWKFQwePJi1a9cSFBTEU089xaFDhzhw4MADH+ZDhw6lQIECWCwWGjRowIEDB6hcuTJgLER0vx7U2rVr2b9/Px4eHlJVVdjVlStX6NmzJ7NmzcLNzY3x48fTpUsXs8OyG2khpMLdu3fx8PDA09OTEiVKJKxnULNmTUqXLg3A6tWrmTlzJh4eHtSqVYurV69y9OjRR5advm/79u3Uq1cvYV8FChRIMo61a9cSGBiIh4cHTZs25datW0RERLBx40bef/99AN58881HfpCHhYURHh7On3/+SWBgIBEREWzevJnmzZuTM2dOcuXKRYsWLRK+8T/K/UU9qlevnrAgUOJYKleunPCBDzB//nyqVatG1apV+euvvx4YP0mqzLbFYmHevHm0bds2xViESAutNfPmzcPNzY25c+cyYMAA9u7d+8gu3fTIuVoIJpW/vj+GcN/t27eBB0tDa60ZN24cjRo1euC11izzEBcXx/bt28mePXua91W2bFkKFSr0yEHt5Mp633e/zLaLi0uK4ygnTpxg9OjR7Nq1i/z589OhQ4dky2y/8847DB48mFdeeYXq1atTsGDBx/rdhHgc586dIyAggKVLl+Lp6cnatWsf+CKTkUgLwUoaNWpEUFBQwpKSR44c4c6dO8mWnU7sxRdfZOPGjQlls69duwZA7ty5E5IPQMOGDR8oH3E/SdWrV4+ff/4ZgJUrV3L9+vUU47106RInTpygZMmS1K1blyVLlhAZGcmdO3dYvHgxdevWpVChQly6dImrV68SFRXFb7/9luJ+E8cSHh7OgQMHAGPMImfOnOTNm5eLFy+ycuXKZPeRPXt2GjVqREBAgHQXCZvRWidM7Fi9ejWjR49m27ZtGTYZgLO1EByYr68vJ0+epFq1amitcXV1ZcmSJcmWnU7M1dWVkJAQWrRoQVxcHM888wxr1qzhrbfeomXLlvz666+MGzeOH374ga5du1K5cmViY2OpV68ewcHBDBw4kPfeew93d3deeumlR14x6e3tjYuLCzExMQwfPpxChQpRqFAhOnToQM2aNRN+l6pVqwIwYMAAatasSdGiRXnhhRdSPA/3P8QrVKhAhQoVEsp4V6lShapVq/LCCy9QvHhxXn755Ufup127dixevNghV5USzu/48eN06tSJdevWUb9+faZMmcJzzz1ndljms0bJVGv9SPlr63LmOEeNGqW//PLLZF8j5a+Tlh7LX1tTbGys/u6773SOHDl07ty59aRJk7TFYkl2e2c5n0j5a5FeNW/enH/++Yd169aZHYpIR/766y98fHzYsWMHb775JsHBwRQrVszssByKJAThcFKzJKgQqRUdHc3w4cP5+uuvyZs3Lz///DNt2rR5YF1yYbDKoLJSappS6pJSKjzRYwWUUmuUUkfj/330pPZH0Km40EpkHPJ+EKm1a9cuqlevzsCBA2nVqhUHDx7kvffek2SQDGvNMpoBvP7QY58DoVrr54HQ+PuPLXv27Fy9elU+BARgJIOrV69aZeqtSL8iIyPp1asXL774ItevX2fp0qXMnj0bV1dXs0NzaFbpMtJab1RKlXro4WaAV/ztH4H1QJ/H3XexYsU4c+YMly9fTkuIVnXv3j2n+EBKr3Fmz55d+n5FstavX4+vry///PMP/v7+jBgxgrx585odllOw5RhCIa31+fjbF4BCT7KTLFmyJFzB6yjWr1+fMC3TkUmcIiO5efMmvXv3JiQkhLJly7Ju3TqHWGbTmShrdcXEtxB+01pXjL9/Q2udL9Hz17XW/zeOoJTyA/wAXF1dq9+vr+PIIiIiyJUrl9lhpEjitJ4ePXpgsVgcfl1pZziXYP04t27dypgxY7h27RqtWrWiQ4cOVmkdO8v59Pb23qO19kzzjqwxdzU+qZQCwhPdPwwUib9dBDic0j6Sug7BETnL3GSJ03rkOgTrslacly5d0u+9954GdMWKFfWOHTusst/7nOV8YqXrEGxZumIp0D7+dnvA9quxCCEyBK01P//8MxUqVGDBggUMHjyYPXv2JFxtL56MVcYQlFJzMAaQn1ZKnQEGAsOB+UopH+AU0NoaxxJCZGxnzpwhICCA3377jVq1ajF16lTc3d3NDssct27B7NlW2521Zhm9l8xT6X9FCSGEXcTFxTF58mQ+++wzYmNj+e677+jWrRsuLi5mh2Z/+/ZBcLCRDO7csdpupdqpEMLhHTt2jAYNGtC5c2dq1KhBeHg4PXv2zFjJIDISZsyAF1+EatXgp5/g3Xdh506rHUJKVwghHFZsbCxjx46lf//+ZM2alcmTJ+Pj45OxrjT++2+YNMlIBjduQIUK8P338OGHYOX1nSUhCCEc0oEDB/Dx8WH37t00bdqUiRMnUrRoUbPDso/oaFiyBIKCYP16yJIF3nkHAgKgbl2wUUKUhCCEcChRUVF88803fPPNN+TPn5958+bRqlWrjNEqOHkSQkJg6lS4dAlKlYJhw6BjR3jmGZsfXhKCEMJhbN++HR8fHw4ePMj777/P2LFj0/8SqhYLrFxptAZWrjS+/TdpYrQGGjaETPYb6pWEIIQw3Z07d+jfvz9jx46laNGiLF++nMaNG5sdlm2dP2+0BEJC4N9/oUgR6N8ffH2heHFTQpKEIIQwVWhoKJ06deLEiRMEBAQwfPhw8uTJY3ZYtqE1rFtnTBldsgRiY+G112DsWHjrLWOswESSEIQQprhx4wafffYZU6ZM4fnnn2fDhg3Uq1fP7LBs4+pV+PFHY7bQkSNQsCD06AF+fvD882ZHl0ASghDC7jZv3kzbtm25ePEivXv3ZtCgQeTIkcPssKxLa9i+3WgNzJsHUVHw8stGt1DLluCApeklIQgh7ObixYt069aN+fPnU7lyZZYuXYqnZ9qLdDqU27eNK4iDg+GPPyB3bvDxAX9/qFzZ7OgeSRKCEMLmtNbMnj2b7t27ExERgY+PD0FBQWQxuc/cqv74w5gpNHs2RESAh4fRRdS2LThBCW2QhCCEsLHTp0/TuXNnVq5cSe3atZk6dSoXL15MH8ng7l345RcjEWzfbnQDtWljTBmtUcNmF5DZitQyEkLYRFxcHEFBQbi7u7Nhwwa+//57Nm3aRIUKFcwOLe2OHIFPPoGiRaF9e6OkxNixcO4cTJ8ONWs6XTIAaSEIIWzgyJEj+Pr6smnTJl599VVCQkIcbincxxYTY0wVDQ42po5myQItWkDnzlC/vlMmgIdJQhBCWE1sbCzffvstAwcOJEeOHEybNo0OHTo4d9mJU6dg8mTjIrILF6BkSfjmG6OcRKEnWireYUlCEEJYxR9//EHHjh3Zu3cvzZs3Z8KECRQpUsTssJ6MxQK//07FoUNhxw7jsTffNFoDjRpBOi27LQlBCJEm9+7d4+uvv2bEiBEULFiQBQsW8M4775gd1pO5cAGmTTPKSZw6Re4CBeCLL6BTJyhRwuzobE4SghDiiW3duhUfHx/+/vtv2rdvz3fffUeBAgXMDuvxaG2UmA4OhkWLjHISDRrAt9+yPW9e6r/6qtkR2o3MMhJCPLaIiAi6d+9OnTp1iIyM5Pfff2fGjBnOlQyuXYMxY+CFF+CVV2DtWujeHQ4fNm6/8w46c8b6zpyxflshRJqtXr0aPz8/Tp06RWBgIN988w25c+c2O6zU0doYE7hfTuLePXjpJfjyS6OcRHorn/GYJCEIIVLl+vXrfPLJJ8yYMYPy5cuzadMm6tSpY3ZYqXP7Nvz8s5EI9u83rhz+6COjnESVKmZH5zAkIQghUrRo0SK6du3K5cuX6du3LwMGDCC7AxZn+z8HDhhJYNYsIylUqWLcb9vWqDEkHiAJQQiRrAsXLhAYGMjChQvx8PBgxYoVVK1a1eywHu3evf/KSWzbZpSTePddY8porVrp4gIyW5GEIIT4P1prZs6cSc+ePYmMjOSbb76hV69ejl1/6OhRo5jc9OnGgHG5cvDdd0ZpCWca7DaRJAQhxANOnTqFv78/q1at4uWXX2bKlCm88MILZoeVtJgYWLrUaA2EhkLmzNC8uVFczstLWgOPSRKCEAIwitFNnDiRzz//HIBx48bRpUsXMtlxkfdUO33aKCcxZYpxMVmJEjB0qFFOonBhs6NzWpIQhBD8/fff+Pr6smXLFho1asSkSZMoWbKk2WE9yGKBVauMQeHly40ppPfLSbz+erotJ2FPkhCEyMBiYmIYNWoUgwcPJmfOnPz444988MEHjlWM7uLF/8pJnDxpFJTr29coJ+FoScvJ2TwhKKVOArcBCxCrtU5n6+UJ4Zz27dtHx44d2b9/Py1btmT8+PEUcpTqnVrDhg3/lZOIiTGuJh45Epo1g6xZzY4wXbJXC8Fba33FTscSQjzCvXv3GDx4MKNGjcLV1ZWFCxfSokULs8MyXL8OM2caieDvvyF/fggMNC4gK1/e7OjSPekyEiID2bx5Mz4+Phw5coSPPvqIb7/9lvz585sblNawa5eRBObONZalfPFFmDEDWrfO8OUk7ElprW17AKVOANcBDUzSWoc89Lwf4Afg6upaff78+TaNxxoiIiLI5QSLZkuc1tOjRw8sFgvjxo0zO5RHSu5cRkZGMnnyZJYsWULhwoX59NNP8fQ0r/c2IiKCvC4uPBMayrNLl5L76FEs2bNz8bXXONe0KRHPPWdabIk5w3sTwNvbe49VuuO11jb9AYrG//sM8AdQL7lty5Urp51BWFiY2SGkisRpPfXr19dVqlQxO4wUJXUuV65cqUuUKKGVUrp79+769u3b9g8ssQMH9JlmzbTOnVtr0LpyZa2DgrS+edPcuJLgDO9NrbUGdmsrfF7bvMtIa302/t9LSqnFQE1go62PK0RGd/XqVT755BNmzpxJhQoV2LJlC7Vr1zYnmHv3YMECo1toyxaKZMkCbdoYF5C9+KJcQOYgbJoQlFI5gUxa69vxtxsCQ2x5TCEyOq01CxcupGvXrly7do0vv/ySL7/8kmzZstk/mGPH/isncfUqPP88fPstW8uWpU6zZvaPRzySrVsIhYDF8XOaMwM/a61/t/Exhciwrl69yjvvvMPixYupXr06q1evpoq9yzvHxMCyZUZrYM0ao5zE228bF5B5e0OmTMSuX2/fmESq2DQhaK2PA1JsXAgb01ozffp0unfvTmxsLCNGjOCTTz4hsz1X/Pr3X6OUxJQpcO4cFC8OX30FPj5QpIj94hBPTKadCuHkTpw4gZ+fH2vXrqVy5cr88ssvlCtXzj4Hj4uD1auN1sCyZcYU0jfeMO43bizlJJyMJAQhnJTFYmH8+PF88cUXuLi4MHHiRMqXL2+fZHDpkjEuMGkSnDgBzzwDffqAnx+UKmX74wubkIQghBM6ePAgvr6+bNu2jTfeeINJkyZRvHhx1tuyb15r2LTJKDW9cKExVuDlBcOGGSWnpZyE05OEIIQTiYmJYcSIEXz11Vfkzp2bWbNm0bZtW9sWo7tx479yEocOQb580KWLMUjsqOskiCfigIXOhcg4Ro4cSVhY2AOPhYWFMXLkyP/bds+ePXh6etK/f3+aN2/OwYMHadeune2Swa5dxoDws89C9+7GGsTTp8PZszB2rCSDdEgSghAmqlGjBq1bt05ICmFhYbRu3ZoaNWokbHP37l369OlDzZo1uXz5MkuWLGHu3Lk888wz1g/ozh1jlpCnJ9SsCfPmwQcfwJ49sGMHdOgATz1l/eMKhyBdRkKYyNvbm/nz59O6dWsCAgIICgpi/vz5eHt7A7BhwwZ8fX05duwYvr6+jBo1inz58lk/kPBwY4B45ky4dQsqVoQJE6BdO8ib1/rHEw5JEoIQJvP29iYgIICvvvqK/v374+3tza1bt+jTpw/BwcGUKVOGtWvX0qBBA+seOCrKGBwOCoLNm41B4datjbGBl16SchIZkCQEIUwWFhZGUFAQ/fv3JygoiBw5cjBx4kTOnTvHJ598wpAhQ8iZM6f1DvjPP/+Vk7hyBcqWhVGjjO6gp5+23nGE05GEIISJ7o8ZzJ8/n0qVKrF161a++OILSpYsydatW6lVq5Z1DhQb+185idWrjQvGmjUzWgMNGkAmGU4UkhCEMNWuXbuYN28ely5dws3NjevXr/Phhx9Svnx56ySDM2eMQeLJk41yEsWKweDB4OtrzB4SIhFJCEKYqF27dnTp0oWlS5fi6elJaGgolSpVSttO4+KMonL3y0nExUGjRsZYQePGRrE5IZIg7wwhTKC1ZsqUKfTq1Yvo6GhGjx5N9+7d01aM7vJlis+ZY1w7cPw4uLrCZ59Bp05Qpoz1ghfpliQEIezsn3/+oVOnToSFheHl5cXkyZN57kmXjNTamCEUHAwLFlA2Ohrq14ehQ41yEmasgSCcliQEIezEYrHw/fff8+WXX5IlSxYmTZqEr68vmZ5kQPfmTfjpJyMR/PWXca1A587srFqVmh06WD12kTFIQhDCDsLDw/Hx8WHnzp00adKEoKAgihUr9vg72rPHGAuYMwciI6FGDZg6Fd59F3LmJFIWnhFpIAlBCBuKjo5m2LBhDB06lLx58/Lzzz/Tpk2bx6s/dOcOzJ1rtAZ27zZKR7Rta0wZrV7ddsGLDEcSghA2snPnTnx8fAgPD6dt27aMHTsWV1fX1O/g4EEjCcycaXQRubvD+PHw/vtSTkLYhCQEIawsMjKSAQMGMGbMGIoUKcKyZcto0qRJ6l4cFQWLFhmJYONGo5xEq1ZGa+Dll6WchLApSQhCWFFYWBi+vr4cP34cf39/RowYQd7UfJs/fhxCQmDaNLh82SgnMXKkUU7icVoVT2Dbtm2sX78eLy8vateubdNjCccmCUEIK7h58ya9e/cmJCSEsmXLJkwpfaTYWFi+3GgNrFpllI9o2tRoDbz6ql3KSWzbto0GDRoQHR1N1qxZCQ0NlaSQgUkBEyHSaNmyZbi5uSVcaHbgwIFHJ4OzZ2HIEChdGt5+G/78EwYOhFOnjO6ihg3tVlto/fr1REdHY7FYiI6Otu0SnMLhSQtBiCd0+fJlunfvzpw5c6hUqRJLlix5YGGbB8TFQWioMWV06VKwWIxyEuPGQZMmppWT8PLyImvWrAkthBRbNSJdk4QgxGPSWjNnzhy6devGrVu3GDx4MJ9//jlZk1pk/soVmDHDKDd97JhRXvrTT8HPzxgnMFnt2rUJDQ2VMQQBSEIQ4rH8+++/BAQEsHz5cmrVqsXUqVNxd3d/cCOtYetWozXwyy8QHQ116xrdRC1aOFw5idq1a0siEIAkBCFSRWvN1atXcXd3x2KxMGbMGD7++GNcXFz+2+jmTZg1yxgkDg+HPHnA39/4eThpCOGAJCEIkYKjR4/yxx9/cPPmTRo0aEBISAhlElcP3bvXSAI//2xcVVy9urEGQZs2YM2VzoSwMZtPZVBKva6UOqyUOqaU+tzWxxPCWmJjYxk9ejSVK1cmIiKCYsWKsWbNGiMZREYaS1DWrGkkgFmzjASwa5dRXsLHR5KBcDo2bSEopVyACcBrwBlgl1Jqqdb6oC2PK0RaHThwAB8fH3bv3k2zZs24ePEid+/eRf39t9Ea+PFHo4vIzQ1++AE++ADy5TM7bCHSxNZdRjWBY1rr4wBKqblAM0ASgnBIUVFRDB06lGHDhpE/f37mzZtHq2bN+KpqVRqfPm0kgCxZoGVL4wKyunWlnIRIN5TW2nY7V6ol8LrW2jf+/gdALa11YFLbP/XUU7pmzZo2i8dabty4QT4n+DYocT6eW7ducfjwYSIjIylUqBB1ihal+eXLvHHhAgViYjihFKtLlWJl4cLcSGqKqQNwlHOZEonTujZs2LBHa+2Z1v2YPqislPID/ACyZMnCjRs3zA0oFSwWi8RpRWbHabFYuHDhAleuXCFb5sz4PvMM7SMjeWnvXjSwMU8exmbKRGimTDyXJ48xfhAZaVq8j2L2uUwtidNBaa1t9gPUBlYlut8X6Jvc9uXKldPOICwszOwQUkXiTNnatWt16dKldRHQS2vW1JZixbQGrZ99VuuBA7X+91+ttdb169fXVapUMS3O1JK/uXU5S5zAbm2Fz2xbtxB2Ac8rpUoDZ4E2QFsbH1OIFN24cYPPPv2UE9OmMTFXLhq5uKB27oTXXjMGiZs0McYKhMhAbJoQtNaxSqlAYBXgAkzTWv9ly2MKkZLlM2eyOzCQz27fphygs2VDBQQY5SSedLF7IdIBm48haK1XACtsfRwhHklrri1fTnjXrjQ4fZo3gdseHtCrF+qddyB7drMjFA4uI6wbYfqgshA2desWetYsbgwfToF//8UDCK9ViyoTJ5K7WjWzoxNOIqOsGyHrIYj0af9+8PcnrkgRVNeunPj3X4aXKcO5Xbvw3L6dLJIMxGPIKOtGSAtBpB9378K8ecaVxDt2EJMlC3O1ZnL27LQaMYLPunZ9sBidEKmUUdaNkIQgnN/ffxvrDcyYATduEFWmDJPKlGHg8ePUeO01ZoaEUKpUKbOjFE4so6wbIQlBOKfoaFiyxGgNhIVBlizENW/O3Hz56DhjBjmeeoox06fTvn17lJSWEFaQEdaNkIQgnMvJkzB5MkydChcvQqlSMGwY4TVq0L53b/bu3Uvz5s2ZMGECRYoUMTtaIZyKDCoLx2exwG+/GReLlSkDw4dDrVqwYgX3wsPpd/s2Ho0acfbsWRYsWMCiRYskGQjxBKSFIBzXhQtGSyAkBE6fhiJF4MsvwdcXSpRg69at+Hh68vfff9O+fXu+++47ChQoYHbUQjgtSQjCsWhtjAkEBRljBLGx8Oqr8N130LQpZMlCREQEX3Trxvjx4ylevDi///47jRo1MjtyIZyeJAThGK5dM2YJTZoER45AgQLQvbtRTqJcuYTNVq9ejZ+fH6dPn6Zr165888035M6d27y4hUhHJCEI82gN27fzwrBhsGEDREXBSy8Z3UKtWj1QTuLatWt8+umnzJgxg/Lly7Nx40bq1KljYvBCpD+SEIT93b4Ns2cbU0b/+IOnc+SAjh2NFcgqV/6/zRcuXEjXrl25cuUKffv2ZcCAAWSX2kNCWJ0kBGE/f/xhjA3Mng0REeDhAZMmsa1YMeo2bvx/m1+4cIHAwEAWLlxI1apVWblyJVWrVrV/3EJkEDLtVNjW3bswcybUrm0kgB9/NNYj3r4d9u4FPz8sTz31wEu01syYMQM3Nzd+++03hg0bxo4dOyQZCGFj0kIQtnHkiNElNGMGXL8O5cvDmDHw4YfGgHEyTp48ib+/P6tXr6ZOnTpMmTKF8uXL2y9uITIwSQjCemJi/isnsW4dZM4MLVoYYwNeXvCIEhJxcXFMmDCBvn37opRi/PjxBAQEkCmTNGKFsBdJCCLtTp36r5zEhQtQsiQMHWoMFBcunOLLT58+Tb169diyZQuNGjVi0qRJlCxZ0g6BCyESk4QgnozFAr//brQGVqwwppC++abRGnj9dUhFmemYmBhGjRrFoEGDyJUrFz/++CMffPCBFKMTwiSSEMTjuXABpk0zykmcOmW0AL74wign8Rjf6vfu3YuPjw/79++nfv36zJs3j0KFCtkwcCFESiQhiJRpDevXG62BRYuMchINGsDo0dCsGWTJkupd3b17lyFDhjBq1ChcXV1ZtGgR+fPnl2QghAOQhCCSd+2aMWU0OBgOH4b8+aFbN6OcxBPM/Nm8eTM+Pj4cOXKEjh07Mnr0aPLnz59ulyMUwtlIQhAP0hp27jQuIJs3D+7dM64h+PFHo5xEjhyPvcvbt2/Tt29fJkyYQKlSpVizZg2vvvqqDYIXQqSFJARhiIj4r5zE/v2QKxd06GAMElep8sS7XblyJf7+/pw5c4bu3bvz9ddfkytXLquFLYSwHkkIGd2BA0YSmDXLqDFUubLROmjXDtJQRfTq1av07NmTn376iQoVKrBly5Z0v/ygEM5OEkJGdO8e/PKLkQi2boVs2eDddyEgwFiJLA3TPrXWLFiwgMDAQK5du0b//v3p168f2bJls+IvIISwBUkIGcnRo8Z6A9OnGwPG5coZC8+0b//IchKpdf78ebp06cKSJUuoXr06q1evpkoaupuEEPYlCSG9i4mBpUuN1sDatUY5iebNjbEBb+80tQbu01ozffp0PvnkE6Kiohg5ciQ9e/Ykc2Z5ewnhTGz2P1YpNQjoBFyOf+gLrfUKWx1PPOTffyk1bRq0bQvnz0OJEvD110Y5CSsuQH/ixAn8/PxYu3Yt9erVY/LkyZRLtMKZEMJ52Por3Bit9WgbH0PcZ7HA6tXGoPDy5ZTUGho3NloDb7yRqnISqT+UhfHjx/PFF1/g4uJCUFAQfn5+UoxOCCcmbfr04OLF/8pJnDwJzzwDn3/OjkqVeLFNG6sf7uDBg/j4+LB9+3YaN25McHAwxYsXt/pxzLBt2zbWr1+Pl5dXupsVlZ5/N2Edtk4IgUqpD4HdwKda6+s2Pl7GoTVs3Gi0BhYtMsYKvL1hxAh4+23ImpV7Vr4CODo6mhEjRvD111+TO3duZs2aRdu2bdNNMbpt27bRoEEDoqOjyZo1K6GhoenmgzM9/27CetKUEJRSa4Gk6hv3A4KArwAd/++3QMck9uEH+AG4uro6RRmDiIgI0+LMfPs2hVat4tlly8h5+jQxuXJxoVkzzr/1FpElShgbbd1q9TgPHz7MyJEjOX78OK+88gqBgYHkz5+fDRs2pHnfZp7PxGbPnk1UVBRxcXFERUUxbdo0oqKiALhx4wYWi8Uh4nyU5M7lo343MzjK3zwlzhKn1Witbf4DlALCU9quXLly2hmEhYXZ94BxcVrv2KH1Rx9pnSOH1qB1rVpaz5ihdWRksi+zRpyRkZH6s88+05kyZdLPPvus/vXXX9O8z4fZ/XwmY+vWrTpHjhzaxcVF58iRQ2/dujXhufr16+sqVaqYF1wqJXcuH/W7mcFR/uYpcZY4gd3aCp/VtpxlVERrfT7+bnMg3FbHSrciImDOHGPK6N69kDMnfPCBMUhsh/WFN2zYgK+vL8eOHaNTp06MHDmSfPny2fy4ZqlduzahoaHpsp89Pf9uwnpsOYYwUinlgdFldBLwt+Gx0pc//zSSwE8/GeUkKlWCiRONchJ58tj88Ldu3aJPnz4EBwdTpkwZQkNDeeWVV2x+XEdQu3btdPthmZ5/N2EdNksIWusPbLXvdOnePViwwEgEW7YY5SRatzZaA7VrW+UCstRYvnw5nTt35ty5c3zyyScMGTKEnDlz2uXYQghzybRTsx079l85iatX4bnnjIVn2reHp5+2WxhXrlyhR48ezJ49G3d3dxYsWECtWrXsdnwhhPkkIZghJgaWLTNaA2vWGBeMvf220Rp45RWw48VdWmvmzZvHxx9/zM2bNxk4cCBffPEFWbNmtVsMQgjHIAnBnv79F6ZMMX7OnYNixWDIEPDxgWeftXs4Z8+epUuXLixdupQaNWowdepUKlWqZPc4hBCOQRKCrcXFGeUkgoONVoHW8PrrxgVljRsbxebsTGvNlClT6NWrFzExMYwePZoePXrgYsXSFkII5yMJwVYuXTLGBSZNghMnwNUVevc21iMuXdq0sP755x86depEWFgYXl5eTJ48meeee860eIQQjkMSgjVpDZs2Gd/+Fy40xgrq14dhw4yS0yb2y1ssFr7//nu+/PJLsmTJQkhICD4+PlKMTgiRQBKCNdy4ATNnGt1Chw5B3rzQpQv4+0OFCmZHR3h4OD4+PuzcuZO33nqLoKAgihYtanZYQggHIwkhLXbvNloDc+bA3btQs6ZRdfTdd+Gpp8yOjpiYGAYPHszQoUPJmzcvc+bM4d133003xeiEENYlCeFx3blDkeXLoVcv2LPH+OB//32jNVC9utnRJdi5cyf+/v6cOHGCtm3b8v333/O0Ha9rEEI4H0kIqfXXX0aX0MyZlL91C9zdYfx4IxnkzWt2dAkiIyPp378/Y8eOpUCBAixbtowmTZqYHZYQwglIQniUqChjcDg42BgszpoVWrViX61aVA0MtFs5idQKCwvD19eX48eP4+/vT5MmTSQZCCFSTaaYJOWff6BPH+PCsXbtjIvIRo6Es2dh1ixuVqrkUMng5s2b+Pn58corr5ApUybCwsIIDg4mV65cZocmhHAi0kK4LzYWfvvNGCRevdooJ9G0KQQEQIMGdi0n8TiWLVtG586duXDhAp999hmDBg3iKQcY0BZCOB9JCGfPGqUkJk82bhctCoMHG+UkHHhq5uXLl+nWrRtz586lUqVK/Prrr3h6epodlhDCiWXMhBAXB2vXGq2BZcvAYoFGjWDCBHjzTVPKSaSW1po5c+bQrVs3bt26xZAhQ+jTp48UoxNCpJnjfvLZwuXL/5WTOH7cKC/dq5dRTqJMGbOjS9G///5LQEAAy5cvp1atWkydOhV3d3ezwxJCpBPpPyFoDZs3GzOFFiyA6GioVw++/hpatDAWonFwcXFxhISE0Lt3bywWC2PGjOHjjz+WYnRCCKtKvwnh5k1jCcrgYOMagjx5jIvH/P2NawicxNGjR+nUqRMbNmygQYMGhISEUMYJWjNCCOeT/hLCnj1GEvj5Z4iMBE9PY9C4TRtjkXonERsby5gxYxgwYADZsmVj6tSpfPTRR1J2QghhM+kjIURGwty5RiLYtcsoJ9G2rdEacMKZNwcOHMDHx4fdu3fTrFkzJk6cyLMmLKAjhMhYnDshHDxoDBD/+KPRReTmBuPGGeUk8uUzO7rHFhUVxdChQxk2bBgFChRg/vz5tGzZUloFQgi7cL6EEBUFixYZrYGNG41yEi1bGusR16njUFcQP45t27bh4+PDoUOH+OCDDxgzZgwFCxY0OywhRAbiPAnh+HEICTHKS1++bEwTHTECPvrIWI3MSd25c4d+/frxww8/UKxYMVasWMEbb7xhdlhCiAzIsRNCbCwsX260BlatMr79N21qtAZee81hy0mk1tq1a+nUqRMnT56kS5cuDBs2jDx58pgdlhAig3LMhHD2LEydapSTOHMGnn0WBgwAX1+j4JyTu3HjBp9++inTpk3j+eefZ+PGjdStW9fssIQQGZxDJYTMkZHwzjvw669GOYmGDY1B4iZNHLqcxONYsmQJXbp04dKlS3z++ecMGDCAHDlymB2WEEI4VkLIceYM3LsHn35qlJMoW9bskKzm4sWLfPzxx/zyyy9UqVKFZcuWUd2BVlgTQog0dcIrpVoppf5SSsUppTwfeq6vUuqYUuqwUqpRavZ3r3Bho4toxIh0kwy01vz000+4ubnx66+/MnToUHbt2iXJQAjhcNLaQggHWgCTEj+olHID2gDuwLPAWqVUOa215VE7i8mTxylqC6XW6dOn8ff35/fff+ell15iypQpVKhQweywhBAiSWlqIWitD2mtDyfxVDNgrtY6Smt9AjgG1EzLsZxJXFwcEyZMwN3dnU2bNvHDDz+wadMmSQZCCIdmqzGEosD2RPfPxD+W7h0+fBhfX182b97Ma6+9RkhICKVKlTI7LCGESFGKCUEptRYonMRT/bTWv6Y1AKWUH+AH4Orqyvr169O6S5uLiIj4vzgtFgvz5s1jxowZZMuWjT59+tCoUSNOnjzJyZMnHSZOR+QMcd64cQOLxeLwcTrDuQSJ02FprdP8A6wHPBPd7wv0TXR/FVA7pf2UK1dOO4OwsLAH7u/bt09Xq1ZNA7pFixb6/Pnz5gT2kIfjdFTOEGf9+vV1lSpVzA4jRc5wLrWWOK0N2K2t8Fluq0t9lwJtlFLZlFKlgeeBnTY6lmnu3btHv3798PT05OzZsyxYsICFCxdSuHBSDSohnN+2bdsYNmwY27ZtMzsUYQNpGkNQSjUHxgGuwHKl1H6tdSOt9V9KqfnAQSAW6KpTmGHkbLZs2YKPjw+HDx+mQ4cOfPvttxQoUMDssISwmW3bttGgQQOio6PJmjUroaGh1K5d2+ywhBWldZbRYq11Ma11Nq11Ia11o0TPDdVal9Val9dar0x7qI4hIiKCH374gbp163Lv3j1WrVrF9OnTJRmIdG/9+vVER0djsViIjo7OWH3rGYRzV4ezs9WrV1OxYkWWLFlCYGAg4eHhNGzY0OywhLALLy8vsmbNiouLC1mzZsXLy8vskISVOVTpCkd17do1Pv30U2bMmEH58uX5/vvv+fjjj80OSwi7ql27NqGhoaxfvx4vLy/pLkqHJCGkYOHChXTt2pUrV67wxRdf0L9/f7Zv357yC4VIh2rXri2JIB2ThJCM8+fPExgYyKJFi6hatSq///47Hh4eZoclhBA2I2MID9FaM2PGDNzc3Fi+fDnDhw9nx44dkgyEEOmetBASOXnyJH5+fqxZs4Y6deowZcoUypcvb3ZYQghhF9JCwChGN27cOCpWrMi2bduYMGECGzZskGQghMhQMnwL4dChQ/j6+rJ161Zef/11goODKVmypNlhCSGE3WXYFkJMTAzffPMNHh4e/P3338ycOZMVK1ZIMhBCZFgZsoWwd+9eOnbsyB9//EGrVq0YN24chQoVMjssIYQwVYZqIdy9e5fPP/+cmjVrcvHiRRYtWsT8+fMlGQghrMLZi/9lmBbCpk2b8PX15ciRI/j4+DBq1Cjy589vdlhCiHQiPRT/S/cthNu3b9O1a1fq1atHdHQ0a9asYcqUKZIMhBBWlR6K/6XrhLBy5Urc3d0JCgqiR48ehIeH8+qrr5odlhAiHUoPxf/SZZfR1atX6dmzJz/99BMVKlRgy5YtTtd0E4+2bds2KbImHEp6KP6XrhKC1ppffvmFwMBArl+/Tv/+/enXrx/ZsmUzOzRhRemhr1akT85e/C/ddBmdO3eOFi1a8O6771KiRAn27NnDkCFDJBmkQ+mhr1YIR+T0CUFrzdSpU3Fzc+P3339n5MiRbN++ncqVK5sdmrCR9NBXK4Qjcuouo+PHj+Pn50doaCj16tVjypQpPP/882aHJWwsPfTVCuGInDIhWCwWxo0bR79+/XBxcSEoKAg/Pz8yZXL6Bo9IJWfvqxXCETldQjh48CA+Pj5s376dxo0bExwcTPHixc0OSwghnJ7TfKWOjo7mq6++wsPDg6NHjzJr1ix+++03SQZCCGElTtFC2LVrFz4+Pvz555+0adOG77//nmeeecbssIQQIl1x6BZCZGQkvXv35sUXX+Tq1av8+uuvzJkzR5KBEELYgMO2EDZs2ICvry/Hjh2jU6dOjBo1irx585odlhBCpFsO10K4desWAQEBeHl5ERcXR2hoKCEhIZIMhBDCxhyqhXDnzh3c3d05d+4cn3zyCV999RVPPfWU2WEJIUSG4FAJ4ezZs7i7u7NgwQJq1apldjhCCJGhKK212TEkUErdBg6bHUcqPA1cMTuIVJA4rcsZ4nSGGEHitLbyWuvcad2JQ7UQgMNaa0+zg0iJUmq3xGk9Eqf1OEOMIHFam1JqtzX243CDykIIIcwhCUEIIQTgeAkhxOwAUknitC6J03qcIUaQOK3NKnE61KCyEEII8zhaC0EIIYRJ7J4QlFKtlFJ/KaXilFKeDz3XVyl1TCl1WCnVKJnXl1ZK7Yjfbp5SKqsdYp6nlNof/3NSKbU/me1OKqX+jN/OKqP+j0MpNUgpdTZRrI2T2e71+HN8TCn1uQlxjlJK/a2UOqCUWqyUypfMdnY/nymdG6VUtvj3w7H492Epe8T1UAzFlVJhSqmD8f+XuiexjZdS6mai98IAe8cZH8cj/4bK8EP8+TyglKpmQozlE52n/UqpW0qpHg9tY8r5VEpNU0pdUkqFJ3qsgFJqjVLqaPy/+ZN5bfv4bY4qpdqn6oBaa7v+ABWA8sB6wDPR427AH0A2oDTwD+CSxOvnA23ibwcDAXaO/1tgQDLPnQSetvc5TXT8QUCvFLZxiT+3ZYCs8efczc5xNgQyx98eAYxwhPOZmnMDdAGC42+3AeaZ8HcuAlSLv50bOJJEnF7Ab/aO7XH/hkBjYCWggBeBHSbH6wJcAEo6wvkE6gHVgPBEj40EPo+//XlS/3+AAsDx+H/zx9/On9Lx7N5C0Fof0londfFZM2Cu1jpKa30COAbUTLyBUkoBrwAL4h/6EXjbhuE+IP74rYE59jqmDdQEjmmtj2uto4G5GOfebrTWq7XWsfF3twPF7Hn8R0jNuWmG8b4D433YIP59YTda6/Na673xt28Dh4Ci9ozBipoBM7VhO5BPKVXExHgaAP9orU+ZGEMCrfVG4NpDDyd+Dyb3GdgIWKO1vqa1vg6sAV5P6XiONIZQFPg30f0z/P+bvCBwI9GHSVLb2FJd4KLW+mgyz2tgtVJqj1LKz45xJRYY3/SelkxTMjXn2Z46YnxDTIq9z2dqzk3CNvHvw5sY70tTxHdZVQV2JPF0baXUH0qplUopd/tGliClv6GjvR/bkPwXPkc4nwCFtNbn429fAAolsc0TnVebXKmslFoLFE7iqX5a619tccy0SmXM7/Ho1kEdrfVZpdQzwBql1N/xGd4ucQJBwFcY/wm/wuje6mjN46dWas6nUqofEAvMTmY3Nj+fzkwplQtYCPTQWt966Om9GN0eEfFjSUuA5+0cIjjR3zB+PLIp0DeJpx3lfD5Aa62VUlabKmqThKC1fvUJXnYWSLweZrH4xxK7itGkzBz/7SypbZ5ISjErpTIDLYDqj9jH2fh/LymlFmN0QVj1zZ/ac6uUmgz8lsRTqTnPaZaK89kBaAI00PGdnknsw+bn8yGpOTf3tzkT/57Ii/G+tCulVBaMZDBba73o4ecTJwit9Qql1ESl1NNaa7vW5UnF39Au78dUegPYq7W++PATjnI+411UShXRWp+P7167lMQ2ZzHGPe4rhjFu+0iO1GW0FGgTP4ujNEb23Zl4g/gPjjCgZfxD7QF7tTheBf7WWp9J6kmlVE6lVO77tzEGTsOT2tZWHup7bZ7M8XcBzytjtlZWjCbyUnvEd59S6nWgN9BUax2ZzDZmnM/UnJulGO87MN6H65JLaLYSP2YxFTiktf4umW0K3x/bUErVxPi/btfElcq/4VLgw/jZRi8CNxN1h9hbsj0AjnA+E0n8HkzuM3AV0FAplT++67hh/GOPZsKoeXOM/qwo4CKwKtFz/TBmeRwG3kj0+Arg2fjbZTASxTHgFyCbneKeAXR+6LFngRWJ4voj/ucvjK4Re5/bn4A/gQPxb5oiD8cZf78xxsyUf0yK8xhG/+b++J/gh+M063wmdW6AIRjJCyB7/PvuWPz7sIwJ568ORrfggUTnsDHQ+f57FAiMP29/YAzcv2RCnEn+DR+KUwET4s/3nySaeWjnWHNifMDnTfSY6ecTI0GdB2LiPzd9MMasQoGjwFqgQPy2nsCURK/tGP8+PQZ8lJrjyZXKQgghAMfqMhJCCGEiSQhCCCEASQhCCCHiSUIQQggBSEIQQggRTxKCEEIIQBKCEEKIeJIQhBBCAPA/kwZiVLRXuGAAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "predicted_boundary_function = lambda x1: (-1 * p.ws.weight[0][0].item() / p.ws.weight[0][1].item()) * x1 - p.ws.bias.item() / p.ws.weight[0][1].item()\n",
    "\n",
    "plt.grid()\n",
    "plt.axhline(0, color=\"black\")\n",
    "plt.axvline(0, color=\"black\")\n",
    "plt.xlim([X1_LOWER_BOUND, X1_UPPER_BOUND])\n",
    "plt.ylim([X2_LOWER_BOUND, X2_UPPER_BOUND])\n",
    "plt.axline(\n",
    "    (X1_LOWER_BOUND, TRUE_BOUNDARY_FUNCTION(X1_LOWER_BOUND)),\n",
    "    (X1_UPPER_BOUND, TRUE_BOUNDARY_FUNCTION(X1_UPPER_BOUND)),\n",
    "    color='black',\n",
    "    label='True Boundary'\n",
    ")\n",
    "plt.axline(\n",
    "    (X1_LOWER_BOUND, predicted_boundary_function(X1_LOWER_BOUND)),\n",
    "    (X1_UPPER_BOUND, predicted_boundary_function(X1_UPPER_BOUND)),\n",
    "    color='red',\n",
    "    label='Predicted Boundary'\n",
    ")\n",
    "for i in range(0, NUM_SAMPLES):\n",
    "    plt.plot(samples[i][0], samples[i][1], color=\"black\", marker=\".\" if true_classifications[i] == 0 else \"x\")\n",
    "plt.legend()\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4f7153bc-ca3c-48a4-b773-60b14ece609d",
   "metadata": {},
   "source": [
    "Let me look at the graph of the total loss per epoch as well:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "id": "e239ab4f-791c-4dd2-b92e-7430d19e2a14",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYsAAAEWCAYAAACXGLsWAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/YYfK9AAAACXBIWXMAAAsTAAALEwEAmpwYAAAoNElEQVR4nO3de5xcdX3/8dd7Zva+yW6y2YTcyIWES7jrFgGtReMFrUprQaFe+Fn8YW2tteqvQm39WWp/FVvvokIrLeIFKKCiotDKpdZiYCPXAIEQArlnc092s/fP749zkkyW2UsuszPZfT8fj3nMme85M+czmc2+93u+Z75HEYGZmdlQMqUuwMzMyp/DwszMhuWwMDOzYTkszMxsWA4LMzMblsPCzMyG5bAwGwMkLZN0XqnrsLHLYWFHFUmrJL2uRPs+V9I9knZJ2iHpx5IWjdK+d+fd+iXtyXv8rog4OSLuG41abHxyWJiNgKRzgLuBHwEzgHnAo8CvJM0/wvuSpAP+b0ZE/d4b8CLw1ry27x7J/ZsV4rCwMUFSlaQvSVqX3r4kqSpdN0XSTyRtl7RV0i/3/jKW9AlJa9PewnJJiwfZxeeAb0fElyNiV0RsjYi/Bn4NfDp9rackvSWvppykNkkvSx+fLel/0joezT9sJOk+SX8v6VdAB3BQAZTf45L0aUn/Luk76ft6XNLxkq6UtEnSaklvyHtug6RvSVqf/lt8RlL2YPZvY5/DwsaKTwJnA2cApwNnAX+drvsYsAZoBqYBfwWEpBOADwG/FRETgDcCqwa+sKRa4Fzg3wvs9xbg9eny94FL8ta9EdgcEb+RNBP4KfAZYDLwceA2Sc15278HuByYALww8rde0FuBG4FJwMPAXST/32cCVwHX5m37b0AvsAA4E3gD8P7D3L+NMQ4LGyveBVwVEZsiog34W5JfvgA9wHRgTkT0RMQvI5kUrQ+oAhZJqoiIVRHxXIHXnkzyf2V9gXXrgSnp8veAt6XhAvCHJAEC8G7gzoi4MyL6I+I/gFbgzXmv9W8RsSwieiOi5xD+DfL9MiLuiohekpBrBj6bvu5NwFxJjZKmpTV8JCLaI2IT8EXg4sPcv40xDgsbK2Zw4F/jL6RtAP8IrADulrRS0hUAEbEC+AjJYaRNkm6SNIOX2gb0kwTOQNOBzXmv9xTw1jQw3kYSIABzgIvSQ1DbJW0HXjXgNVcfzBsexsa85T0kPZy+vMcA9WldFcD6vLquBaYewVpsDHBY2FixjuQX317Hpm2kYwwfi4j5JL/AP7p3bCIivhcRr0qfG8DVA184ItqBB4CLCuz3HcAv8h7vPRR1AfBkGiCQBMGNEdGYd6uLiM/m7+qg3/XhWw10AVPy6poYESeXoBYrYw4LOxpVSKrOu+VIfkn/taRmSVOATwHfAZD0FkkLJAnYQXL4qV/SCZJemw6Ed5L8xd0/yD6vAC6V9GFJEyRNkvQZ4BySQ1573URyzP+D7O9VkNbyVklvlJRN6z5P0qwj9Y9yKCJiPclZXp+XNFFSRtJxkn6nlHVZ+XFY2NHoTpJf7HtvnyYZOG4FHgMeB36TtgEsBP4T2E3SQ/h6RNxLMl7xWZLDSBtIDr1cWWiHEfHfJAPWbycZp3iBZDD4VRHxbN5269N9nAvcnNe+mqS38VdAG8lf9P+H8vg/+F6gEniS5JDbrRQ+5GbjmHzxIzMzG045/FVjZmZlzmFhZmbDcliYmdmwHBZmZjasXKkLOFKmTJkSc+fOLXUZZmZHlaVLl26OiObhthszYTF37lxaW1tLXYaZ2VFF0ojmIfNhKDMzG5bDwszMhuWwMDOzYTkszMxsWA4LMzMblsPCzMyG5bAwM7NhjfuwWL9jD1+4ezkr23aXuhQzs7I17sOibVcXX7lnBSvb2ktdiplZ2Rr3YVGRTf4JevsHu0CamZk5LNKw6O7zRaDMzAYz7sOiMg2Lnl73LMzMBjPuw6IiJwB6+hwWZmaDcVjs7Vk4LMzMBuWwyHjMwsxsOA4LH4YyMxuWw8ID3GZmwxr3YZHLpD2Lfh+GMjMbzLgPC0lUZjM+DGVmNoRxHxYAFVn5MJSZ2RAcFkBFzj0LM7OhOCyAXCbjU2fNzIbgsAAqs3LPwsxsCA4LfBjKzGw4DguS71r0+jCUmdmgHBYkYdHtnoWZ2aAcFnjMwsxsOEUNC0nnS1ouaYWkKwqsr5J0c7p+iaS5aftcSXskPZLevlnMOiv8pTwzsyHlivXCkrLANcDrgTXAQ5LuiIgn8za7DNgWEQskXQxcDbwzXfdcRJxRrPryVWQz9PR6zMLMbDDF7FmcBayIiJUR0Q3cBFwwYJsLgBvS5VuBxZJUxJoKymXlMQszsyEUMyxmAqvzHq9J2wpuExG9wA6gKV03T9LDku6X9NtFrNNzQ5mZDaNoh6EO03rg2IjYIunlwA8lnRwRO/M3knQ5cDnAsccee8g785iFmdnQitmzWAvMzns8K20ruI2kHNAAbImIrojYAhARS4HngOMH7iAirouIlohoaW5uPuRCK3L+noWZ2VCKGRYPAQslzZNUCVwM3DFgmzuAS9PlC4F7IiIkNacD5EiaDywEVhar0Iqs6PKss2ZmgyraYaiI6JX0IeAuIAtcHxHLJF0FtEbEHcC3gBslrQC2kgQKwKuBqyT1AP3AH0fE1mLVWpXLOizMzIZQ1DGLiLgTuHNA26fyljuBiwo87zbgtmLWlq8ql6Grt2+0dmdmdtTxN7iB6oosXT3uWZiZDcZhAVRXJHND9fk63GZmBTksSHoWAN0etzAzK8hhQTJmAdDZ43ELM7NCHBbs71l0epDbzKwghwXJmAXgQW4zs0E4LEi+ZwHuWZiZDcZhwf6eRad7FmZmBTksgOq9PQsPcJuZFeSwAKrSAW5P+WFmVpjDAp86a2Y2HIcFeafOOizMzApyWJB36qwPQ5mZFeSwYH/Poss9CzOzghwW5I9ZuGdhZlaIw4K8noW/lGdmVpDDAqjIZshm5J6FmdkgHBap2oosHd3uWZiZFeKwSNVWZWnv6i11GWZmZclhkaqrytHe7bAwMyvEYZGqq8z5MJSZ2SAcFqnayiy7fRjKzKwgh0WqvipHhw9DmZkV5LBI1VblaO/yYSgzs0IcFqm6Sp8NZWY2GIdFqq4q57AwMxuEwyJVV5mlo6eP/v4odSlmZmXHYZGqrcoRAZ2eH8rM7CWKGhaSzpe0XNIKSVcUWF8l6eZ0/RJJcwesP1bSbkkfL2adkByGAnz6rJlZAUULC0lZ4BrgTcAi4BJJiwZsdhmwLSIWAF8Erh6w/gvAz4pVY766ymTm2Q6fEWVm9hLF7FmcBayIiJUR0Q3cBFwwYJsLgBvS5VuBxZIEIOn3gOeBZUWscZ/aSvcszMwGU8ywmAmsznu8Jm0ruE1E9AI7gCZJ9cAngL8dageSLpfUKqm1ra3tsIqtTw9D+YwoM7OXKtcB7k8DX4yI3UNtFBHXRURLRLQ0Nzcf1g4nVCdhsavTYWFmNlCuiK+9Fpid93hW2lZomzWSckADsAV4BXChpM8BjUC/pM6I+Fqxim2oqQBgx56eYu3CzOyoVcyweAhYKGkeSShcDPzhgG3uAC4FHgAuBO6JiAB+e+8Gkj4N7C5mUIDDwsxsKEULi4jolfQh4C4gC1wfEcskXQW0RsQdwLeAGyWtALaSBEpJ7D0M5bAwM3upYvYsiIg7gTsHtH0qb7kTuGiY1/h0UYobIJfNUF+Vc1iYmRVQrgPcJdFQU8FOh4WZ2Us4LPJMrKlgZ6fDwsxsIIdFnoYaH4YyMyvEYZGnoabCYWFmVoDDIo/DwsysMIdFHoeFmVlhDos8DTUVdPb00+VrWpiZHcBhkWdSXSUA29rduzAzy+ewyNNUVwXA5t1dJa7EzKy8OCzyNE9IehZb2rtLXImZWXlxWOTZ17PY5Z6FmVk+h0Wepvq9PQuHhZlZvmHDQtIrJdWly++W9AVJc4pf2uirr8pRlcuwebcPQ5mZ5RtJz+IbQIek04GPAc8B3y5qVSUiiSn1VR7gNjMbYCRh0ZtekOgC4GsRcQ0wobhllc6U+kq2uGdhZnaAkVzPYpekK4F3A6+WlAEqiltW6TTVV7FxZ2epyzAzKysj6Vm8E+gCLouIDSTX0v7HolZVQk117lmYmQ00op4F8OWI6JN0PHAi8P3illU6UydW0ba7i77+IJtRqcsxMysLI+lZ/BdQJWkmcDfwHuDfillUKR3TUENff3iQ28wsz0jCQhHRAbwd+HpEXAScUtyySmdGQzUA67bvKXElZmblY0RhIekc4F3ATw/ieUel6Q01AKzf4UFuM7O9RvJL/yPAlcAPImKZpPnAvUWtqoRmNLpnYWY20LAD3BFxP3C/pHpJ9RGxEvhw8UsrjYaaCqorMu5ZmJnlGcl0H6dKehhYBjwpaamkk4tfWmlIYkZDDRscFmZm+4zkMNS1wEcjYk5EHEsy5cc/F7es0preWM26HT4MZWa210jCoi4i9o1RRMR9QF3RKioD0xtqPGZhZpZnJGGxUtLfSJqb3v4aWFnswkpp9qRaNu7sorPH1+I2M4ORhcUfAc3A7cBtwBTgfSN5cUnnS1ouaYWkKwqsr5J0c7p+iaS5aftZkh5Jb49K+v0Rv6MjYO6UWgBWb+0Yzd2amZWtkZwNtY0BZz9JuplkzqhBScoC1wCvB9YAD0m6IyKezNvsMmBbRCyQdDFwdfq6TwAtEdEraTrwqKQfR0TvQby3QzanKTnKtmpLBwunjdkJds3MRuxQv1x3zgi2OQtYERErI6IbuIlkmvN8FwA3pMu3AoslKSI68oKhGohDrPOQzG1KehYvbGkfzd2amZWtYn4TeyawOu/xmrSt4DZpOOwAmgAkvULSMuBx4I8L9SokXS6pVVJrW1vbESu8sbaSxtoKVjkszMyAIQ5DSXrZYKsYhetZRMQS4GRJJwE3SPpZRHQO2OY64DqAlpaWI9r7mNNUx6rNHrMwM4Ohxyw+P8S6p0fw2muB2XmPZ6VthbZZIykHNABb8jeIiKck7SaZvLB1BPs9IuY21bL0hW2jtTszs7I2aFhExGsO87UfAhZKmkcSChcDfzhgmzuAS4EHgAuBeyIi0uesTge455BcQ2PVYdZzUOY21fHjR9fR2dNHdUV2NHdtZlZ2ijZmkY4xfAi4C3gKuCWdiPAqSW9LN/sW0CRpBfBRYO/pta8iOQPqEeAHwJ9ExOZi1VrI8dMm0B+wYtPu0dytmVlZGsmV8g5ZRNwJ3Dmg7VN5y53ARQWedyNwYzFrG84Jx9QD8MzGXZwys6GUpZiZldyYvS7F4ZrbVEdlNsPyDbtKXYqZWckdytlQAETEb458OeUjl81w3NR6lm90WJiZHerZUAG89gjXUnZOPGYCv165ZfgNzczGuGKeDXXUO37aBH7w8Fp27OmhoaboXy0xMytbIxrglnQKsIhk6g0AIuLbxSqqXJw4PZkX6sl1OznnuKYSV2NmVjojuVLe/wW+mt5eA3wOeNuQTxojTkvPgnpszfbSFmJmVmIjORvqQmAxsCEi3gecTvJN6zGvqb6K2ZNreNRhYWbj3EjCYk9E9AO9kiYCmzhwGo8x7fRZjTy6ekepyzAzK6mRhEWrpEaS624vBX5DMj3HuHDG7EbWbt9D266uUpdiZlYyI7n40Z+ki9+U9HNgYkQ8VtyyysfpsxuBZNxi8UnTSluMmVmJjGSA+xd7lyNiVUQ8lt821p0yo4FcRp6B1szGtaG+wV0N1AJTJE0iuY4FwEReehGjMaumMsupsxpY8vzWUpdiZlYyQ/UsPkAyRnEiyTjF0vT2I+BrxS+tfJw9v4lHV2+no3tULgFuZlZ2Bg2LiPhyRMwDPh4R8/Jup0fEuAuL3v7woSgzG7dGcjbUtZI+LOnW9PYhSeNq7ouWOZPIZuR5osxs3BrJdB9fJ7nm9tfTx+8BvgG8v1hFlZu6qhynzWrggeccFmY2Pg01wJ1Lr3b3WxFxet6qeyQ9WvzSyssrj5vCN+5/ju0d3TTWVpa6HDOzUTXUYagH0/s+ScftbZQ0H+gralVl6DUnTqWvP7j/mbZSl2JmNuqGCou9p8p+HLhX0n2S7gPuAT5W7MLKzRmzG5lcV8m9T28qdSlmZqNuqDGLZkkfTZevBbLpch9wJnBvMQsrN9mMOO/4Zu5Zvom+/iCb0fBPMjMbI4bqWWSBemACSagoveXStnHntSdNZXtHDw+/6FNozWx8GapnsT4irhq1So4Crz6+mcpshp89sYGWuZNLXY6Z2agZyZiFpSZWV/Dq45v56WPr6e+PUpdjZjZqhgqLxaNWxVHkradPZ8POTlr9bW4zG0eGmu7DM+cV8LqTplFdkeHHj64rdSlmZqNmJNN9WJ66qhyLT5zGnY+vp6evv9TlmJmNCofFIfj9M2eypb2bXzzl71yY2fhQ1LCQdL6k5ZJWSLqiwPoqSTen65dImpu2v17SUkmPp/evLWadB+u8E5qZNrGKmx56sdSlmJmNiqKFhaQscA3wJmARcImkRQM2uwzYFhELgC8CV6ftm4G3RsSpwKXAjcWq81Dkshne0TKb+59pY822jlKXY2ZWdMXsWZwFrIiIlRHRDdwEXDBgmwuAG9LlW4HFkhQRD0fE3hHkZUCNpKoi1nrQ3tEyG4BbWteUuBIzs+IrZljMBFbnPV7DSy/Hum+bdIbbHUDTgG3+APhNRHQVqc5DMntyLb9zfDPfW/IiXb3jbl5FMxtnynqAW9LJJIemPjDI+ssltUpqbWsb/dlg3/+q+Wze3cWPHvZptGY2thUzLNYCs/Mez0rbCm4jKQc0AFvSx7OAHwDvjYjnCu0gIq6LiJaIaGlubj7C5Q/vlQuaOGn6RP75lyuJ8De6zWzsKmZYPAQslDRPUiVwMXDHgG3uIBnABrgQuCciQlIj8FPgioj4VRFrPCySuPzV83h2027u83UuzGwMK1pYpGMQHwLuAp4CbomIZZKukvS2dLNvAU2SVgAfBfaeXvshYAHwKUmPpLepxar1cLzltBnMaKjma/escO/CzMYsjZVfcC0tLdHa2lqSfX93yQt88gdP8K/v+y1ec0JZZpqZWUGSlkZEy3DblfUA99HiopfPZvbkGj5/93L3LsxsTHJYHAGVuQx/vvh4nli7k7uWbSh1OWZmR5zD4gj5vTNmsGBqPf/ws6f9vQszG3McFkdILpvhU29ZxAtbOrj+v1eVuhwzsyPKYXEEvfr4Zl530jS+es+zbNzZWepyzMyOGIfFEfY3bzmJ3r7gMz99qtSlmJkdMQ6LI2xOUx1/+poF/PjRddztwW4zGyMcFkXwwfOO48RjJvDJHz7B9o7uUpdjZnbYHBZFUJnL8E8Xnc7W9m6u+vGTpS7HzOywOSyK5JSZDfzpecdx+8Nr+dEjA+dPNDM7ujgsiujDixfSMmcSf3X746xs213qcszMDpnDoohy2QxfueRMKnIZ/vR7D9PZ4y/rmdnRyWFRZDMaa/j8Rafz1Pqd/NXtj3vuKDM7KjksRsHik6bx0dcfz+0Pr+Ub9xe8jpOZWVnLlbqA8eLPXruAZzft5nM/X878KfWcf8oxpS7JzGzE3LMYJZL4xwtP4/TZjfz5TQ/z4PNbS12SmdmIOSxGUXVFlusvbWHWpBou+7eHeGLtjlKXZGY2Ig6LUdZUX8WNl72CiTUVvPf6B1mxaVepSzIzG5bDogRmNNbwnfe/gozEO6/9NU+t31nqkszMhuSwKJF5U+q45QNnU5nLcPF1v+aR1dtLXZKZ2aAcFiU0v7meWz5wDg01Fbz7X5bwqxWbS12SmVlBDosSmz25lls+cA4zG2u49PoHufmhF0tdkpnZSzgsysAxDdX8+wfP4dwFU/jEbY/z2Z89TX+/v+ltZuXDYVEmJlZXcP2lLbz77GP55v3P8Uc3PMS2dl8Lw8zKg8OijOSyGf7uglP4u987hf9ZsYW3fPW/efjFbaUuy8zMYVFuJPGes+dw6wfPQYJ3XPsA//LLlT4sZWYl5bAoU6fNauSnf/bbnHfCVD7z06e45J9/zeqtHaUuy8zGKYdFGWuoreC697ycz114GsvW7eT8L/0X33/wRU9zbmajrqhhIel8ScslrZB0RYH1VZJuTtcvkTQ3bW+SdK+k3ZK+Vsway50k3tEym59/5Lc5fXYjV97+OO+89tcs3+BpQsxs9BQtLCRlgWuANwGLgEskLRqw2WXAtohYAHwRuDpt7wT+Bvh4seo72syaVMt3LnsFn337qTyzaRdv/sov+cxPnmR3V2+pSzOzcaCYPYuzgBURsTIiuoGbgAsGbHMBcEO6fCuwWJIioj0i/pskNCyVyYiLzzqWez92Hu9omcW3fvU8iz9/H7c8tJrevv5Sl2dmY1gxw2ImsDrv8Zq0reA2EdEL7ACaRroDSZdLapXU2tbWdpjlHj0m1VXyD28/jds/eC7HNNTwl7c9xpu+/Ev+48mNHs8ws6I4qge4I+K6iGiJiJbm5uZSlzPqzjx2Ej/8k3P55rtfRl9/8L+/3cqF33yA/3qmzaFhZkdUMcNiLTA77/GstK3gNpJyQAOwpYg1jTmSOP+U6dz9F6/m//3+qazdtof3Xv8gF1zzK37+xAZ/P8PMjohihsVDwEJJ8yRVAhcDdwzY5g7g0nT5QuCe8J/EhySXzfCHrziW+//yPD779lPZsaeHP/7OUt74pf/iltbVdPb0lbpEMzuKqZi/myW9GfgSkAWuj4i/l3QV0BoRd0iqBm4EzgS2AhdHxMr0uauAiUAlsB14Q0Q8Odi+WlpaorW1tWjv5WjT29fPTx9fzzfue46nN+xiUm0F7/ytY3n32ccya1JtqcszszIhaWlEtAy73Vj5Q95hUVhE8MDKLXz7f17g7ic3APC6k6bx7rPn8MoFU8hmVOIKzayURhoWudEoxkpHEuceN4Vzj5vC2u17+O6vX+Cmh1Zz95Mbmd5Qze+fOZM/ePksjmuuL3WpZlbG3LMYhzp7+vjFU5u47TdruP+ZNvr6gzNmN/IHL5vJ+adMp3lCValLNLNR4sNQNiKbdnXyo4fXcdtv1vD0hl1IcNbcybz51Omcf8oxTJtYXeoSzayIHBZ2UCKC5Rt38bPHN3Dn4+t5dtNuJGiZM4nXL5rGa06YyoKp9Uge4zAbSxwWdlie3biLnz2RBMfT6aSFMxtreM2JzbzmhKmcc1wTtZUe8jI72jks7IhZt30P9y1v497lm/jVis10dPdRmctw1tzJnHNcE2fPb+K0WQ1UZI/qCQHMxiWHhRVFV28fDz2/jXueToJj+cak11FbmaVl7mTOmd/E2fMnc+rMBnIOD7Oy51NnrSiqclletXAKr1o4BYAtu7tY8vxWfr1yCw88t4Wrf/40kITHabMaOPPYSZw5u5Ezjm1k6gQPlpsdrdyzsCOqbVcXS57fQuuqbTz84jaWrdtJbzo/1axJNfvC4/TZDZx4zETqqvz3ilkp+TCUlYXOnj6WrdvBwy9uT2/bWLcjuUyJBPOm1HHyjAZOnjExvTUwua6yxFWbjR8+DGVloboiy8vnTOblcybva9uwo5Mn1u5g2bqdPLFuB795YRs/fnTdvvVTJ1Rx/LQJLJxWz/HTJnD8tHoWTpvAxOqKUrwFM8NhYSVwTEM1xzRU87pF0/a1bWvv5sn1O1m2bgdPb9jFsxt3c9ODq9mTN1vuMROr9wXIvCl1zG2qY05TLTMaazzHlVmROSysLEyqq+SVC6bwygVT9rX19wdrt+/hmY27eGbjbp7duItnNu3iu0teoLNn/2VkK7MZZk+uYd6UOuY01TG3qZa5aZg4SMyODIeFla1MRsyeXMvsybUsPml/L6S/P9i4q5NVmzt4YUs7z29p54XNHaza0s6vVmw5oDdSkRWzJyWvMaOxhpmN1cycVMOMhhpmNNZwTEO1vx9iNgIOCzvqZDJiekMN0xtqOOe4Ay/ZHhFs2tXF85vbeWFLO6u2dLBqcztrtu3hibU72NLefeBrCaZNrGZGY00aJkmgzGisYdrEaqZOrKKprsq9Exv3HBY2pkhi2sRqpk2s5uz5TS9Zv6e7j/U79rB2+x7Wbd/D2u2drEuXH1uznbue2EB3X/8Bz8lmxJT6SqZOqGbqhCqmTtx7X8XUCdVMS++n1Ff6i4g2ZjksbFypqcwyv7me+YNcv6O/P9jc3sXabXvYuLOLtl2dbNzZxaZdnWza1cW6HZ08umY7W9q7GXjWuQSTayuZXJfcmur3LlcxubaCyfVVNO1dV1fJpLpKHwKzo4bDwixPJqO0BzH0t817+vrZvLuLTTu72LgzCZJNOztp293NtvZutrZ3s3zDLra2d7N9T89LgmWvidU5muqr9gdMGiINNRU01FTQmN5PTO8baiuYUJXz7L826hwWZoegIpvZN24ynN6+frbv6WFrezdbdidBsrW9iy1pqGxpTwJm9dYOHlm9nW3t3fu+9V5IRuwLk/wgaaytOLC9uoL66hz1VTkmVOeor0oe11ZkyXgMxg6Sw8KsyHLZDFPqq5hSXwXTht8+Iujo7mPHnh527Olhe0dyvzN9vGNPD9v3dLNjT+++x6u3diTbdPbSN0TQQHK4rL4yDZA0TOqrkx5Lsrw/YPJDpq4yS01lltrKHLWV2fSW8+D/OOGwMCszkqirylFXlWNG4/A9l3wRwe6uXrZ39LC7qze5dfayK73f3dXDrs5ednXuX7e7Kwmdtds69rW1d/cNv7NUZS6TBEdFEiZ1VTlqKvaHSU0aLDWVWerSoNnXVpE8rqvav1ybF0oOovLhsDAbQyQxobqCCYc5NUpffxwYNp09tHf3sae7l47uPjq6+9iT3nf09LKnu4/2rj729Oxfv3l3Nx3dHcl2PX10dPW95Eyz4VTmMtRUZKnKZaiuyFJdkaEqN+A+b/2+7XJZqioyVOeS9dUVmby2bOHnpK9ZkZXHhApwWJjZS2Qz2jf2cST19vXT0ZMXNN29ecvJ44FB1NXTT2dPH129B953dPeyrSNZ7uzpp6u3n66ePjp7++jpO/QJUjNiXxBV5jJUZJP7ygH3+9r3thVs1771FQVeY+C6qgHPr0jbKrOZko8zOSzMbNTkshkmZjNFnxSyrz/o6u1LgqZ3b5gk9/mBs3e564C2/dt29/bT09dPV9/+5e7e5Nbe1UtXbz/dfQe29/RFsnyQvajhZDOiMpshl03uK7IZKnKiIpth8YlT+eTvLjqi+xvIYWFmY042o3QgvnQ1REQSHAOCpis/dPr66ektHEY9ff37w6g36O7r2xdEPX399PZFsn1fP8eM4Ky8w+WwMDMrAklU5pLDUFSVuprD56+PmpnZsIoaFpLOl7Rc0gpJVxRYXyXp5nT9Eklz89ZdmbYvl/TGYtZpZmZDK1pYSMoC1wBvAhYBl0gaOAJzGbAtIhYAXwSuTp+7CLgYOBk4H/h6+npmZlYCxexZnAWsiIiVEdEN3ARcMGCbC4Ab0uVbgcVKTnC+ALgpIroi4nlgRfp6ZmZWAsUMi5nA6rzHa9K2gttERC+wA2ga4XORdLmkVkmtbW1tR7B0MzPLd1QPcEfEdRHREhEtzc3NpS7HzGzMKmZYrAVm5z2elbYV3EZSDmgAtozwuWZmNkqKGRYPAQslzZNUSTJgfceAbe4ALk2XLwTuiYhI2y9Oz5aaBywEHixirWZmNoSifSkvInolfQi4C8gC10fEMklXAa0RcQfwLeBGSSuArSSBQrrdLcCTQC/wpxEx5DSYS5cu3SzphcMoeQqw+TCef7QZb+8X/J7HC7/ngzNnJBspBruE1zgjqTUiWkpdx2gZb+8X/J7HC7/n4jiqB7jNzGx0OCzMzGxYDov9rit1AaNsvL1f8HseL/yei8BjFmZmNiz3LMzMbFgOCzMzG9a4D4vhplEvZ5JmS7pX0pOSlkn687R9sqT/kPRsej8pbZekr6Tv9TFJL8t7rUvT7Z+VdGle+8slPZ4+5ysqkyvZS8pKeljST9LH89Jp7lek095Xpu0HPQ1+Of5MSGqUdKukpyU9Jemcsf45S/qL9Of6CUnfl1Q91j5nSddL2iTpiby2on+ug+1jSBExbm8kXxZ8DpgPVAKPAotKXddB1D8deFm6PAF4hmQ6+M8BV6TtVwBXp8tvBn4GCDgbWJK2TwZWpveT0uVJ6boH022VPvdNpX7faV0fBb4H/CR9fAtwcbr8TeCD6fKfAN9Mly8Gbk6XF6WfdxUwL/05yJbrzwTJ7MzvT5crgcax/DmTTBz6PFCT9/n+r7H2OQOvBl4GPJHXVvTPdbB9DFlrqf8TlPgH8hzgrrzHVwJXlrquw3g/PwJeDywHpqdt04Hl6fK1wCV52y9P118CXJvXfm3aNh14Oq/9gO1K+D5nAb8AXgv8JP2PsBnIDfxcSWYQOCddzqXbaeBnvXe7cvyZIJkz7XnSE1IGfn5j8XNm/8zTk9PP7SfAG8fi5wzM5cCwKPrnOtg+hrqN98NQI5oK/WiQdrvPBJYA0yJifbpqAzAtXR7s/Q7VvqZAe6l9CfhLoD993ARsj2SaeziwzoOdBr8cfybmAW3Av6aH3v5FUh1j+HOOiLXAPwEvAutJPreljO3Pea/R+FwH28egxntYjAmS6oHbgI9ExM78dZH86TBmzo+W9BZgU0QsLXUtoyhHcqjiGxFxJtBOcuhgnzH4OU8iuQjaPGAGUEdy1cxxZTQ+15HuY7yHxVE/FbqkCpKg+G5E3J42b5Q0PV0/HdiUtg/2fodqn1WgvZReCbxN0iqSqy++Fvgy0Khkmns4sM6DnQa/HH8m1gBrImJJ+vhWkvAYy5/z64DnI6ItInqA20k++7H8Oe81Gp/rYPsY1HgPi5FMo1620jMbvgU8FRFfyFuVP/X7pSRjGXvb35ueVXE2sCPtit4FvEHSpPQvujeQHM9dD+yUdHa6r/fmvVZJRMSVETErIuaSfF73RMS7gHtJprmHl77ng5kGv+x+JiJiA7Ba0glp02KSGZnH7OdMcvjpbEm1aU173/OY/ZzzjMbnOtg+BlfKQaxyuJGcYfAMyZkRnyx1PQdZ+6tIuo+PAY+ktzeTHKv9BfAs8J/A5HR7Adek7/VxoCXvtf6I5FrnK4D35bW3AE+kz/kaAwZZS/z+z2P/2VDzSX4JrAD+HahK26vTxyvS9fPznv/J9H0tJ+/sn3L8mQDOAFrTz/qHJGe9jOnPGfhb4Om0rhtJzmgaU58z8H2SMZkekh7kZaPxuQ62j6Funu7DzMyGNd4PQ5mZ2Qg4LMzMbFgOCzMzG5bDwszMhuWwMDOzYTkszIYhqU/SI3m3IzZDqaS5+TOOmpWr3PCbmI17eyLijFIXYVZK7lmYHSJJqyR9Lr1ewIOSFqTtcyXdk15z4BeSjk3bp0n6gaRH09u56UtlJf2zkms33C2pJt3+w0quVfKYpJtK9DbNAIeF2UjUDDgM9c68dTsi4lSSb8d+KW37KnBDRJwGfBf4Str+FeD+iDidZG6nZWn7QuCaiDgZ2A78Qdp+BXBm+jp/XJy3ZjYy/ga32TAk7Y6I+gLtq4DXRsTKdELHDRHRJGkzybUCetL29RExRVIbMCsiuvJeYy7wHxGxMH38CaAiIj4j6efAbpLpPX4YEbuL/FbNBuWehdnhiUGWD0ZX3nIf+8cSf5dkLqCXAQ/lzbZqNuocFmaH55159w+ky/9DMospwLuAX6bLvwA+CPuuId4w2ItKygCzI+Je4BMkU26/pHdjNlr8l4rZ8GokPZL3+OcRsff02UmSHiPpHVyStv0ZyVXt/g/JFe7el7b/OXCdpMtIehAfJJlxtJAs8J00UAR8JSK2H6H3Y3bQPGZhdojSMYuWiNhc6lrMis2HoczMbFjuWZiZ2bDcszAzs2E5LMzMbFgOCzMzG5bDwszMhuWwMDOzYf1/f6aCtlZEMtYAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.title(\"Loss Over Time\")\n",
    "plt.xlabel(\"Epochs\")\n",
    "plt.ylabel(\"Total Loss\")\n",
    "plt.plot(loss_pytorch_perceptron)\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "92a2681a-d9af-4941-98d6-caccedfa86b1",
   "metadata": {},
   "source": [
    "# Appendix"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "adddecb1-6cde-4785-847e-02ecdb5938a1",
   "metadata": {
    "tags": []
   },
   "source": [
    "#### Links"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d45149e6-a05e-4bd8-9604-f3737b8eaab4",
   "metadata": {},
   "source": [
    "Some possibly helpful links for understanding the perceptron. I looked at all of them but some were more helpful than others."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "bfa194a6-05be-4902-984a-4431ec1be5b7",
   "metadata": {},
   "source": [
    "https://natureofcode.com/book/chapter-10-neural-networks/  \n",
    "https://en.wikipedia.org/wiki/Perceptron  \n",
    "https://www.youtube.com/watch?v=4Gac5I64LM4&ab_channel=ritvikmath  \n",
    "http://web.mit.edu/6.S097/www/resources/L01.pdf  \n",
    "https://stats.stackexchange.com/questions/200445/machine-learning-intuition-behind-perceptron-learning-algorithm  \n",
    "https://stats.stackexchange.com/questions/387695/derivation-of-perceptron-weight-update-formula  \n",
    "https://stats.stackexchange.com/questions/411212/how-are-the-weights-updated-in-the-perceptron-learning-rule  \n",
    "https://datascience.stackexchange.com/questions/36450/what-is-the-difference-between-gradient-descent-and-stochastic-gradient-descent  \n",
    "https://towardsdatascience.com/the-hitchhikers-guide-to-optimization-in-machine-learning-edcf5a104210  \n",
    "https://www.reddit.com/r/pytorch/comments/mvpj9k/how_to_change_pytorch_sigmoid_function_to_be_more/  \n",
    "https://stackoverflow.com/questions/67203664/how-to-change-pytorch-sigmoid-function-to-be-steeper"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b2e0acf4-6328-48bd-9bf1-a9b86b8e7f79",
   "metadata": {},
   "source": [
    "#### Versions Used"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7688c6fd-cc5b-4f4b-8fc0-efbc9404ba60",
   "metadata": {},
   "source": [
    "python 3.8.9  \n",
    "jupyterlab 3.3.4  \n",
    "matplotlib 3.5.1  \n",
    "numpy 1.21.5  \n",
    "torch 1.11.0"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.9"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}