{
"cells": [
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "slide"
}
},
"source": [
"Sveučilište u Zagrebu
\n",
"Fakultet elektrotehnike i računarstva\n",
"\n",
"# Strojno učenje\n",
"\n",
"http://www.fer.unizg.hr/predmet/su\n",
"\n",
"Ak. god. 2015./2016.\n",
"\n",
"# Bilježnica 2: Osnovni koncepti strojnog učenja\n",
"\n",
"(c) 2015 Jan Šnajder\n",
"\n",
"Verzija: 0.7 (2015-10-21)"
]
},
{
"cell_type": "code",
"execution_count": 1,
"metadata": {
"collapsed": false
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Populating the interactive namespace from numpy and matplotlib\n"
]
}
],
"source": [
"import scipy as sp\n",
"import scipy.stats as stats\n",
"import matplotlib.pyplot as plt\n",
"from numpy.random import normal\n",
"%pylab inline"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Sadržaj:\n",
"\n",
"* Tipični koraci primjene algoritma SU\n",
"\n",
"* Prostor primjera\n",
"\n",
"* Hipoteza i model\n",
"\n",
"* Empirijska pogreška\n",
"\n",
"* Prostor inačica\n",
"\n",
"* Složenost modela\n",
"\n",
"* Induktivna pristranost\n",
"\n",
"* Tri komponente svakog algoritma SU\n",
"\n",
"* Primjer: regresija\n",
"\n",
"* Problem šuma\n",
"\n",
"* Odabir modela"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Tipični koraci primjene algoritma SU\n",
"\n",
"\n",
"\n",
"1. Priprema podataka\n",
"\n",
"2. (Označavanje podataka za učenje i ispitivanje)\n",
"\n",
"3. (Redukcija dimenzionalnosti)\n",
"\n",
"4. **Odabir modela**\n",
"\n",
"5. **Učenje modela**\n",
"\n",
"6. **Vrednovanje modela**\n",
"\n",
"7. **Dijagnostika i ispravljanje (debugging)**\n",
"\n",
"8. Instalacija (deployment)\n",
"\n",
"\n",
"* Naš fokus su koraci 4-7"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Prostor primjera\n",
"\n",
"\n",
"* Prostor primjera (ulazni prostor): $\\mathcal{X}$\n",
"\n",
"\n",
"* Dimenzija ulaznog prostora: $n$\n",
"\n",
"\n",
"* Primjer je vektor u ulaznom prostoru: $\\mathbf{x} = (x_1, x_2, \\dots, x_n)^T \\in \\mathcal{X}$\n",
"\n",
"\n",
"* Oznaka (engl. *label*) klase (za klasifikaciju) ili ciljna vrijednost (za regresiju): $y$\n",
"\n",
"\n",
"* Skup oznaka klase: $\\mathcal{Y} = \\{0, \\dots, K\\}$\n",
" * Broj klasa: $K$\n",
" * Binarna klasifikacija: $K=2$, $\\mathcal{Y} = \\{0,1\\}$\n",
"\n",
"\n",
"* Broj primjera: $N$\n",
"\n",
"\n",
"* Skup označenih primjera za učenje: $\\mathcal{D} = \\big\\{(x^{(i)}, y^{(i)})\\big\\}_{i=1}^N \\subseteq \\mathcal{X}\\times\\mathcal{Y}$\n",
"\n",
"\n",
"* Matrično:\n",
"\\begin{array}{lllll|l}\n",
"&x_1 & x_2 & \\cdots & x_n & \\mathbf{y}\\\\\n",
"\\hline\n",
"\\mathbf{x}^{(1)} = & x_1^{(1)} & x_2^{(1)} & \\cdots & x_n^{(1)} & y^{(1)}\\\\\n",
"\\mathbf{x}^{(2)} = & x_1^{(2)} & x_2^{(2)} & \\cdots & x_n^{(2)} & y^{(2)}\\\\\n",
"& \\vdots\\\\\n",
"\\mathbf{x}^{(N)} = & x_1^{(N)} & x_2^{(N)} & \\cdots & x_n^{(N)} & y^{(N)}\\\\\n",
"\\end{array}\n",
"Matrica $\\mathcal{D}$ sastavljena je od matrice $\\mathbf{X}_{N\\times n}$ i vektora $\\mathbf{y}_{N\\times 1}$\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Hipoteza i model\n",
"\n",
"* Hipoteza: $h : \\mathcal{X} \\to \\mathcal{Y}$\n",
" * Funkcija koja svakom primjeru (iz prostora primjera) dodjeljuje oznaku klase (iz skupa oznaka klase)\n",
"\n",
"\n",
"* Binarna klasifikacija: $h : \\mathcal{Y} \\to \\{0, 1\\}$\n",
" * Definicija: Primjer $\\mathbf{x}\\in\\mathcal{X}$ **zadovoljava** hipotezu $h$ akko $h(\\mathbf{x})=1$\n",
" * Definicija: Hipoteza $h$ je **konzistentna** s primjerom $(\\mathbf{x}, y)$ akko $h(\\mathbf{x})=y$\n",
"\n",
"\n",
"* Općenitije: $h(\\mathbf{x} | \\theta)$\n",
" * Funkcija parametrizirana parametrima $\\theta$ (vektor parametara)\n",
" * Npr.:\n",
" * Linearna regresija: $h(x) = \\theta_1 x + \\theta_0$\n",
" * Linearan klasifikacijski model: $h(x_1,x_2|\\theta_0,\\theta_1,\\theta_2) = \\mathbf{1}\\{\\theta_1 x_1 + \\theta_2 x_2 + \\theta_0 \\geq 0\\}$\n",
"\n",
"\n",
"\n",
"* Model $\\mathcal{H}$: skup hipoteza $h$\n",
"\n",
"\n",
"* Formalno: $\\mathcal{H} = \\big\\{ h(\\mathbf{x} | \\theta)\\big\\}_{\\theta}$\n",
" * Familija funkcija parametriziranih s $\\theta$\n",
" \n",
" \n",
"* Učenje (treniranje modela) svodi se na **pretraživanje** prostora hipoteza $\\mathcal{H}$ i nalaženje **najbolje** hipoteze $h\\in \\mathcal{H}$\n",
" * Najbolja hipoteza: ona koja najtočnije klasificira primjere (klasifikacija) odnosno daje vrijednosti najbliže ciljnim vrijednostima (regresija)\n",
" * Optimizacijski problem!\n",
" \n",
" \n",
"* [Primjer: Ulazni prostor + prostor parametara]\n",
"\n",
"\n",
"* $\\mathcal{H}$ je vrlo velik, pa nam često treba heuristička optimizacija"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Empirijska pogreška\n",
"\n",
"* Iskazuje koliko točno hipoteza klasificira primjere (klasifikacija) ili koliko su vrijednosti blizu ciljnih vrijednosti (regresija)\n",
"\n",
"\n",
"* Pogreška klasifikacija (engl. *misclassification error*):\n",
"\n",
"$$\n",
"E(h|\\mathcal{D}) \n",
"= \\frac{1}{N} \\sum_{i=1}^N \\mathbf{1}\\{h(\\mathbf{x})^{(i)} \\neq y^{(i)}\\}\n",
"$$\n",
"\n",
"* Specifično, za binarnu klasifikaciju s $\\mathcal{Y}=\\{0,1\\}$:\n",
"\n",
"$$\n",
"E(h|\\mathcal{D}) = \\frac{1}{N} \\sum_{i=1}^N |h(\\mathbf{x})^{(i)} - y^{(i)}|\n",
"$$\n",
"\n",
"* [Primjer]\n",
"\n",
"\n",
"* Vrijednost pogreške načinjene na pojedinačnom primjeru (funkcija unutar sume) zove se **funkcija gubitka** (engl. *loss function*)\n",
" * Gubitak $\\mathbf{1}\\{h(\\mathbf{x})^{(i)} \\neq y^{(i)}\\}$ zove se **gubitak nula-jedan** (engl. *zero-one loss*)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Prostor inačica (engl. *version space*)\n",
"\n",
"\n",
"* $\\mathit{VS}_{\\mathcal{H},\\mathcal{D}} \\subseteq \\mathcal{H}$\n",
"\n",
"\n",
"* Skup hipoteza iz $\\mathcal{H}$ koje su konzistentne s primjerima za učenje $\\mathcal{D}$\n",
"\n",
"$$\n",
"\\mathit{VS}_{\\mathcal{H},\\mathcal{D}} =\n",
"\\Big\\{h\\in\\mathcal{H} \\mid \\forall(\\mathbf{x},y)\\in\\mathcal{D}.\\ \\big(h(\\mathbf{x})=y\\big)\\Big\\}\n",
"$$\n",
"\n",
"* [Primjer]\n",
"\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Složenost modela\n",
"\n",
"\n",
"* Idealno, u modelu $\\mathcal{H}$ postoji hipoteza $h$ koja je konzistentna s $\\mathcal{D}$, tj. hipoteza za koju vrijedi $E(h|\\mathcal{D}) = 0$\n",
"\n",
"\n",
"* No, moguće je da takva $h$ ne postoji, tj. $\\forall h\\in\\mathcal{H}. E(h|\\mathcal{D}) > 0$\n",
"\n",
"\n",
"* Tada kažemo da model $\\mathcal{H}$ nije dovoljne **složenosti** (ili kapaciteta)\n",
"\n",
"\n",
"* [Primjer]\n",
"\n",
"\n",
"* [Zadatak: 6 primjera]"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"#Induktivna pristranost (engl. *inductive bias*)\n",
"\n",
"* Učenje hipoteze je **loše definiran problem**: $h$ ne slijedi deduktivno iz $\\mathcal{D}$\n",
"\n",
"\n",
"* Primjer 1: Učenje Booleove funkcije\n",
"\n",
"\\begin{array}{ccc|c}\n",
"x_1 & x_2 & x_3 & y\\\\\n",
"\\hline\n",
"0&0&0&\\color{red}{\\textbf{?}}\\\\\n",
"0&0&1&\\color{red}{\\textbf{?}}\\\\\n",
"0&1&0&1\\\\\n",
"0&1&1&0\\\\\n",
"1&0&0&1\\\\\n",
"1&0&1&0\\\\\n",
"1&1&0&\\color{red}{\\textbf{?}}\\\\\n",
"1&1&1&1\\\\\n",
"\\end{array}\n",
"\n",
"* $N = |\\mathcal{D}|=5$, $n=3$, $\\mathcal{X} = \\{0,1\\}^3$, $|\\mathit{VS}| = 2^{2^n - N} = 8$\n",
"\n",
"\n",
"* **Generalizacija** - sposobnost klasifikacije još neviđenih primjera\n",
"\n",
"\n",
"* Učenje i generalizacija nisu mogući bez **dodatnih pretpostavki**\n",
" * *Futility of bias-free learning*\n",
"\n",
"\n",
"* **Induktivna pristranost** (engl. inductive bias)\n",
" * $\\mathcal{L}$ - algoritam učenja\n",
" * $h_\\mathcal{L}$ - hipoteza inducirana pomoću $\\mathcal{L}$ na $\\mathcal{D}$\n",
" * $h_\\mathcal{L}(\\mathbf{x})$ - klasifikacija primjera $\\mathbf{x}\\in\\mathcal{X}$\n",
" * Induktivna pristranost od $\\mathcal{L}$ je bilo koji skup minimalnih pretpostavki $\\mathcal{B}$ takvih da \n",
" \n",
"$$\n",
" \\forall \\mathcal{D}.\\,\\forall\\mathbf{x}\\in \\mathcal{X}.\\,\\big((\\mathcal{B}\\land\\mathcal{D}\\land\\mathbf{x})\\ \\vdash\\ h_\\mathcal{L}(\\mathbf{x})\\big)\n",
"$$\n",
"\n",
"\n",
"* Skup pretpostavki koje *od indukcije čine dedukciju*\n",
"\n",
"\n",
"* Dvije vrste induktivne pristranosti:\n",
"\n",
" * **Pristranost jezika** (pristranost ograničenjem): odabiremo model $\\mathcal{H}$ koji ograničava skup prikazivih hipoteza\n",
" \n",
" * **Pristranost preferencijom** (pristranost pretraživanja): definiramo način pretraživanja unutar $\\mathcal{H}$\n",
" \n",
" \n",
"* Većina aloritama SU kombinira obje vrste pristranosti\n",
"\n",
"\n",
"* [Primjer 2: Ulazni prostor + prostor parametara]\n",
"\n",
"\n",
"* Zadatak 3: \n",
" * Učenje Booleove funkcije u $\\mathcal{X}=\\{0,1\\}$, $\\mathcal{H}$ je skup pravaca\n",
" * Q: Koja je ovo vrsta pristranosti?\n",
" * Q: Koliko različitih hipoteza postoji?\n",
" * Q: Postoji li za svako označavanje konzistentna hipoteza u $\\mathcal{H}$?\n",
" \n",
" \n",
"* Razmotrimo opet Primjer 1, uz $\\mathcal{H} = \\text{skup ravnina u $\\mathbb{R}^3$}$"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"#Tri komponente svakog algoritma SU\n",
"\n",
"\n",
"* **(1) Model** $\\mathcal{H}$\n",
" * $\\mathcal{H} = \\big\\{ h(\\mathbf{x} | \\theta)\\big\\}_{\\theta}$\n",
"\n",
"\n",
"* **(2) Funkcija gubitka** $L(y, h(\\mathbf{x}))$\n",
"\n",
" * Izračunava kolika je pogreška hipoteze (naučenog modela) na primjeru $\\mathbf{x}^{(i)}$\n",
"\n",
" * Uobičajene funkcije gubitka:\n",
" * Kvadratno odstupanje (regresija): $L\\big(y,h(\\mathbf{x}^{(i)}|\\theta)\\big)=(h(\\mathbf{x}^{(i)}|\\theta) - y^{(i)})^2$\n",
" * Gubitak 0-1 (klasifikacija): $L\\big(y,h(\\mathbf{x}^{(i)}|\\theta)\\big) = \\mathbf{1}\\{h(\\mathbf{x})^{(i)} \\neq y^{(i)}\\}$\n",
" \n",
" \n",
"* **Funkcija pogreške** definirana je kao očekivana vrijednost funkcije gubitka na primjerima iz $\\mathcal{X}\\times\\mathcal{Y}$\n",
"$$\n",
"E(h) = \\mathbb{E}_{\\mathbf{x},y}[L]\n",
"$$\n",
" * Međutim, prava distribucija primjera i oznaka, $P(\\mathbf{x}, y)$ je nepoznata, pa umjesto toga računamo *empirijsku pogrešku* (pogrešku na skupu označenih primjera $\\mathcal{D}$)\n",
"$$E(h|\\mathcal{D}) = \\mathbb{E}_{D}[L] = \\frac{1}{N} \\sum_{i=1}^N L\\big(y^{(i)}, h(\\mathbf{x}^{(i)})\\big)$$\n",
" * Budući da su hipoteze indeksirane preko parametara $\\theta$, možemo pisati\n",
"$$E(\\color{red}{\\theta}|\\mathcal{D}) = \\mathbb{E}_{D}[L] = \\frac{1}{N} \\sum_{i=1}^N L\\big(y^{(i)}, h(\\mathbf{x}^{(i)}|\\color{red}{\\theta})\\big)$$\n",
"\n",
"* **(3) Optimizacijski postupak**\n",
"\n",
" * Postupak kojim nalazimo hipotezu $h^*$ koja minimizira empirijsku pogrešku\n",
"$$\n",
"h^* = \\mathrm{argmin}_{h\\in\\mathcal{H}} E(h|\\mathcal{D})\n",
"$$\n",
"tj. \n",
"$$\n",
"\\theta^* = \\mathrm{argmin}_{\\theta} E(\\theta|\\mathcal{D})\n",
"$$\n",
"\n",
"\n",
"* Optimizacija može biti analitička ili heuristička\n",
" * Analitičke postupke koristimo kada postoji rješenje u **zatvorenoj formi**\n",
"\n",
"\n",
"* Gornje tri komponente definiraju i induktivnu pristranost svakog algoritma\n",
" * Q: Koja vrsta induktivne pristranosti je vezana uz koje komponente?"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Primjer: regresija\n",
"\n",
"* $y \\in \\mathbb{R}$\n",
"\n",
"\n",
"* Na temelju $\\mathcal{D}=\\{(\\mathbf{x}^{(i)},y^{(i)})\\}$ učimo funkciju $h$ koja aproksimira nepoznatu funkciju $f:\\mathcal{X}\\to\\mathbb{R}$\n",
"\n",
"\n",
"* Idealno, $y^{(i)}=f(\\mathbf{x}^{(i)})$, ali zbog šuma $y=f(\\mathcal{x}^{(i)})+\\varepsilon$ \n",
"\n",
"\n",
"* [Primjeri: Box Office Revenue, prosjek ocjena, cijena automobila]\n",
"\n",
"\n",
"* Funkcija gubitka je kvadratna:\n",
"$$\n",
"L(y, h(\\mathbf{x})) = (y - h(\\mathbf{x}))^2\n",
"$$\n",
"pa je empirijska pogreška hipoteze\n",
"$$\n",
"E(h|\\mathcal{D})=\\color{red}{\\frac{1}{2}}\\sum_{i=1}^N\\big(y^{(i)}-h(\\mathbf{x}^{(i)})\\big)^2\n",
"$$\n",
"NB: Umjesto $1/N$, kod pogreške regresije koristimo $1/2$ zbog kasnije matematičke jednostavnosti. To međutim nema utjecaja na optimizaciju (radi se o konstanti)\n",
"\n",
"* **Linearan model**: hiperravnina u $\\mathbb{R}^n$\n",
"\n",
"$$\n",
"h(\\mathbf{x}|\\mathbf{w}) = w_1 x_1 + w_2 x_2 + \\dots + w_n x_n + w_0 = \n",
"\\sum_{i=1}^n w_i x_i + w_0 = \\mathbf{w}^T\\mathbf{x} + w_0\n",
"$$\n",
"\n",
"* Za $n=2$ imamo $\\mathcal{X}=\\mathbb{R}$. Model je\n",
"$$\n",
"h(x|\\mathbf{w}) = w_1 x + w_0\n",
"$$\n",
"funkcija gubitka je\n",
"$$\n",
"L(y^{(i)}, h(x^{(i)})) = \\big(y^{(i)}-(w_1 x^{(i)} + w_0)\\big)^2\n",
"$$\n",
"a pogreška je\n",
"\n",
"$$\n",
"E(h|\\mathcal{D})=\\frac{1}{2}\n",
"\\sum_{i=1}^N\\big(y^{(i)}-(w_1 x^{(i)} + w_0)\\big)^2\n",
"$$"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"**(1) Model**:"
]
},
{
"cell_type": "code",
"execution_count": 2,
"metadata": {
"collapsed": false
},
"outputs": [],
"source": [
"def h(x, w): return w[1] * x + w[0]"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"**(2) Funkcija gubitka** (i njoj odgovarajuća funkcija pogreške):"
]
},
{
"cell_type": "code",
"execution_count": 3,
"metadata": {
"collapsed": false
},
"outputs": [],
"source": [
"def quadratic_loss(y, hx):\n",
" return (y - hx)**2\n",
" \n",
"def error(h, X, y):\n",
" err = 0\n",
" for xi, yi in zip(X, y):\n",
" err += quadratic_loss(yi, h(xi))\n",
" return 0.5 * err"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Funkcija koja generira podatke (i koju zapravo želimo naučiti):"
]
},
{
"cell_type": "code",
"execution_count": 4,
"metadata": {
"collapsed": false
},
"outputs": [
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAXEAAAEACAYAAABF+UbAAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAEz1JREFUeJzt3X+s3XV9x/HnGxCVsYikeOkcscsyrVuYlRgjVOrxB0uz\nLdD+w2ZmbAwzmmzIsFXBZONG/pjTIt2yQOSXKW5jIi4M3FxpkbNgzNjUFuu9xbIFMnBr1dlqC/vD\nO9/743xve7jcnl/3nPM93/N9PpKbfM/3fM/3vnPavvPt67zP5xuZiSSpmk4ruwBJ0uBs4pJUYTZx\nSaowm7gkVZhNXJIqzCYuSRXWsYlHxMsi4rGI2BcR8xHxp8X+2Yh4NiL2Fj8bx1OuJKlddJsTj4iz\nMvP5iDgD+BqwDXgncCwzPzOGGiVJp9A1TsnM54vNM4HTgSPF4xhVUZKk3nRt4hFxWkTsAw4Dj2Tm\nXPHU1RHxeETcGRHnjLRKSdKyusYpJw6MeAWwC7gOmAd+UDx1I7A6M68aSYWSpFM6o9cDM/PHEfEP\nwJsys7m4PyLuAB5cenxEuCiLJA0gM3uOq7tNp6xajEoi4uXAZcDeiDi/7bDNwP5TFOJPJjfccEPp\nNUzKj++F74XvReeffnW7El8N7IyI02g1/M9n5sMRcXdErAMSeAr4QN+/WZK0Yh2beGbuBy5aZv97\nR1aRJKlnfmNzDBqNRtklTAzfi5N8L07yvRhcz9MpfZ84Ikd1bkmaVhFBDuuDTUnSZLOJS1KF2cQl\nqcJs4pJUYTZxSaowm7gkVZhNXJIqzCYuSRVmE5ekCrOJS1KF2cQlqcJs4pJUYT3f2UeSNDoLC3DH\nHf2/ziYuSSXbsweuvRZWrer/tTZxSSrJk0/C1q0wNwfbt8OmTXBanyG3mbgkjdnRo63mffHFcOml\nMD8PmzdD9LyK+Ek2cUkak4UFuPVWWLsWjh9vXYF/5CPw0pcOfk7jFEkag927W7n3eefBrl3whjcM\n57w2cUkaoYMHYdu2VmSyfTtcccVgscmpGKdI0ggcPQof/jCsXw8bNrSik02bhtvAwSYuSUO1mHu/\n7nXw3HOt5r1t28py7046xikR8TLgn4GXAmcCf5+Z10fEucAXgNcATwNXZubR0ZQoSdWwmHu/6lXw\n0EPDy707iczsfEDEWZn5fEScAXwN2AZcDvwwMz8VER8DXpmZ1y15XXY7tyRNg2Hm3hFBZvb86q5x\nSmY+X2yeCZwOHKHVxHcW+3cCm/qsU5IqbzH3vuSS1rz3qHLvTro28Yg4LSL2AYeBRzJzDpjJzMPF\nIYeBmRHWKEkTpT33Hta896C6jhhm5s+AdRHxCmBXRLx9yfMZEcvmJrOzsye2G40GjUZjRcVKUtl2\n725dfa9aNZzcu9ls0mw2B35910z8BQdH/DHwv8DvA43MPBQRq2ldoa9dcqyZuKSpMep570VDzcQj\nYlVEnFNsvxy4DNgLPABsKQ7bAtw/WLmSNNkmIffupFucshrYGRGn0Wr4n8/MhyNiL3BvRFxFMWI4\n2jIlabwWFuD222F2tnXVPTcHMxP46V9fcUpfJzZOkVRR7et779gxnnnvRf3GKa6dIkmFxfW9R517\nD5Nfu5dUe0vX95603LsTm7ik2mqf9z52rNx570EZp0iqpfbce1zrnIyCTVxSrYxr3ntcjFMk1cKk\nz3sPyiYuaapN0jono2CcImlqtd/Xssq5dyc2cUlTZ9py706MUyRNjWnNvTuxiUuqvGnPvTsxTpFU\naXXIvTuxiUuqpDrl3p0Yp0iqlDrm3p3YxCVVwjSsczIKximSJl577r1rF6xbV3ZFk8MmLmlimXt3\nZ5wiaeKYe/fOJi5pYrTn3s89Z+7dC+MUSROh7vPeg7KJSyrVYu49N9fKvY1N+mOcIqkUS3Pv+XnY\nvNkG3i+buKSxqvM6J6PQsYlHxAUR8UhEzEXEdyLiQ8X+2Yh4NiL2Fj8bx1OupCrbvbs1433vva15\n79tug5mZsquqtsjMUz8ZcT5wfmbui4izgW8Cm4ArgWOZ+ZkOr81O55ZUH8579y4iyMye352OV+KZ\neSgz9xXbx4EDwKsXf9fAVUqqhaNHYetW571HqedMPCLWAG8E/qXYdXVEPB4Rd0bEOSOoTVJFLebe\na9eae49aTyOGRZRyH3BNZh6PiFuBTxRP3wjcBFy19HWzs7MnthuNBo1GY4XlSpp0S9c5cd67s2az\nSbPZHPj1HTNxgIh4CfBl4CuZuWOZ59cAD2bmhUv2m4lLNeK893AMNROPiADuBObbG3hErG47bDOw\nv99CJU0H573L1S1OWQ+8B/h2ROwt9n0ceHdErAMSeAr4wOhKlDSJFhbg9tthdrY1bTI357hgGbrG\nKQOf2DhFmlrtuffNN7u+9zD1G6e4doqknjnvPXn82r2krlzfe3LZxCWdkut7Tz7jFEnLcn3varCJ\nS3qBgwdbX5U/cMDcuwqMUyQBcOTIydx7wwZz76qwiUs1177Oibl39RinSDVm7l19NnGphpz3nh7G\nKVKNtM97v/Wt5t7TwCYu1cBy894f/ai59zQwTpGmnLn3dLOJS1PK9b3rwThFmjJLc2/X955uNnFp\nSph715NxijQFzL3ryyYuVZjz3jJOkSrIeW8tsolLFWLuraWMU6SKMPfWcmzi0oRz3ludGKdIE8p5\nb/XCJi5NGHNv9aNjnBIRFwB3A68CErgtM/8iIs4FvgC8BngauDIzj464VmnqmXurX5GZp34y4nzg\n/MzcFxFnA98ENgHvA36YmZ+KiI8Br8zM65a8NjudW9JJzntrUUSQmT3/6XeMUzLzUGbuK7aPAweA\nVwOXAzuLw3bSauyS+rSYe69fD5de6ry3+tdzJh4Ra4A3Ao8BM5l5uHjqMDAz9MqkKbZc7u19LTWI\nnkYMiyjlS8A1mXks2i4TMjMjYtncZHZ29sR2o9Gg0WispFZpKuzZ08q9V60y9xY0m02azebAr++Y\niQNExEuALwNfycwdxb4ngEZmHoqI1cAjmbl2yevMxKU2Tz4JW7eae6uzoWbi0brkvhOYX2zghQeA\nLcX2FuD+fguV6uLo0Vbzvvhi1znR8HXLxNcD7wHeHhF7i5+NwCeByyLiIPCO4rGkNu259/Hjzntr\nNLrGKQOf2DhFNdY+771jh7m3etdvnOLaKdIQOe+tcfNr99IQtK9z4ry3xskmLq2A894qm3GKNCDn\nvTUJbOJSn5z31iQxTpF61D7vbe6tSWETl7pYzL3Xrj05723urUlhnCJ10D7vvWuXubcmj01cWobz\n3qoK4xSpjfPeqhqbuITz3qou4xTV3uK8t/e1VBXZxFVbzntrGhinqHYW570vuQQ2bDD3VrXZxFUb\ny817b9tm7q1qM05RLbTn3s57a5rYxDXVzL017YxTNJVc50R1YRPXVHGdE9WNcYqmRvv63ubeqgub\nuCpvcZ2TuTm46SZzb9WLcYoqa+k6J/Pz5t6qH5u4Ksd1TqSTujbxiLgrIg5HxP62fbMR8WxE7C1+\nNo62TKll925Ytw6++MXWOief/SzMzJRdlVSeyMzOB0RcChwH7s7MC4t9NwDHMvMzHV6X3c4t9cr1\nvVUXEUFm9vy3u+uVeGY+ChxZ7nf1U5g0CNf3ljpbSSZ+dUQ8HhF3RsQ5Q6tIwtxb6tWgI4a3Ap8o\ntm8EbgKuWnrQ7Ozsie1Go0Gj0Rjw16lO2ue9Xd9b067ZbNJsNgd+fddMHCAi1gAPLmbivTxnJq5+\ntc97b99ubKJ6Gnomfopfsrrt4WZg/6mOlbpZbt5782YbuNSLrnFKRNwDvA1YFRHPADcAjYhYByTw\nFPCBkVapqbSwALffDrOzravuuTnHBaV+9RSnDHRi4xR10J5779hh7i0t6jdOce0UjVX7vPenP23u\nLa2UX7vXWCw3723uLa2cTVwj1b6+t/Pe0vAZp2hkXN9bGj2buIZu8b6WzntLo2ecoqFZel9L572l\n0bOJa8W8r6VUHuMUrcju3a2pE3NvqRw2cQ3EeW9pMhinqC/Oe0uTxSaunri+tzSZjFPUVXvu7fre\n0mSxieuUzL2lyWecohcx95aqwyauE8y9peoxThHQyr2vvRbOO8/cW6oSm3jNeV9LqdqMU2rK+1pK\n08EmXjPm3tJ0MU6pEee9peljE6+B9nnv7dvhiiuMTaRpYZwyxZab9/aDS2m62MSnUHvu7fre0nTr\nGqdExF3AbwHfz8wLi33nAl8AXgM8DVyZmUdHWKd61D7vvWsXrFtXdkWSRqmXK/HPARuX7LsO2J2Z\nrwUeLh6rRAcPwuWXwwc/CDfeCF/9qg1cqoOuTTwzHwWOLNl9ObCz2N4JbBpyXeqR895SvQ2aic9k\n5uFi+zAwM6R61KP2+1o67y3V14pHDDMzIyKXe252dvbEdqPRoNForPTXCdizp5V7e19LqfqazSbN\nZnPg10fmsv33hQdFrAEebPtg8wmgkZmHImI18Ehmrl3ymuzl3Oqd65xI0y8iyMye/2UPGqc8AGwp\ntrcA9w94HvXA3FvSqXRt4hFxD/B14HUR8UxEvA/4JHBZRBwE3lE81pA57y2pm57ilIFObJyyIu3z\n3jff7LigVBf9ximunTJhzL0l9cOv3U8Ic29Jg7CJl8z1vSWthHFKibyvpaSVsomXwPW9JQ2LccoY\nHTni+t6ShssmPgYLC3DLLa5zImn4jFNGzPW9JY2STXxEzL0ljYNxypB5X0tJ42QTHxLnvSWVwThl\nCJz3llQWm/gKmHtLKptxygDMvSVNCpt4H1zfW9KkMU7pkfPekiaRTbwLc29Jk8w45RTMvSVVgU18\nicXc23VOJFWBcUqbpbm3896SJp1NHHNvSdVV6zjF3FtS1dWyiTvvLWlarChOiYingZ8A/wf8NDPf\nPIyiRsl5b0nTZKWZeAKNzPzRMIoZpcXce26ulXsbm0iaBsOIUya6FS7NvefnYfNmG7ik6bDSJp7A\nnoj4RkS8fxgFDYu5t6Q6WGmcsj4z/zsizgN2R8QTmfno4pOzs7MnDmw0GjQajRX+ut64vrekqmg2\nmzSbzYFfH5k5lEIi4gbgeGbeVDzOYZ27V9/9biv3PnDAeW9J1RQRZGbPnWvgOCUizoqIny+2fw74\nDWD/oOdbiSNHWrn3+vWwYYPz3pLqYyWZ+AzwaETsAx4DvpyZDw2nrN4sLMAtt7TWOTH3llRHA2fi\nmfkUUNqUtbm3JFVw7RTXOZGkkyrztXvXOZGkF5v4Ju68tySd2kTHKXv2tHLvVavMvSVpORPZxJ98\nErZuNfeWpG4mKk45erTVvC++2NxbknoxEU28Pfc+dszcW5J6VXqcYu4tSYMrrYk77y1JKzf2OMV5\nb0kanrE1cee9JWn4xhKnuM6JJI3GSJu4ubckjdZI4xRzb0karaHd2edFJ47IQ4eSmZmRnF6SplK/\nd/YZaRMf9+3ZJKnqxnZ7NklS+WziklRhNnFJqjCbuCRVmE1ckirMJi5JFTZwE4+IjRHxREQ8GREf\nG2ZRkqTeDNTEI+J04C+BjcCvAu+OiNcPs7Bp0mw2yy5hYvhenOR7cZLvxeAGvRJ/M/Dvmfl0Zv4U\n+FvgiuGVNV38C3qS78VJvhcn+V4MbtAm/mrgmbbHzxb7JEljNGgT9/v0kjQBBlo7JSLeAsxm5sbi\n8fXAzzLzz9qOsdFL0gBGvgBWRJwBfBd4J/BfwL8C787MA32fTJI0sIFuCpGZCxHxh8Au4HTgThu4\nJI3fyJailSSN3ki+sekXgVoi4oKIeCQi5iLiOxHxobJrKltEnB4ReyPiwbJrKVNEnBMR90XEgYiY\nLz5nqqWIuL74N7I/Iv4mImpz+/SIuCsiDkfE/rZ950bE7og4GBEPRcQ5nc4x9CbuF4Fe4KfAtZn5\na8BbgD+o8Xux6BpgHiec/hz4x8x8PfDrQC3jyIhYA7wfuCgzL6QVz/5umTWN2edo9cp21wG7M/O1\nwMPF41MaxZW4XwQqZOahzNxXbB+n9Q/1F8qtqjwR8YvAbwJ3ALW942pEvAK4NDPvgtZnTJn545LL\nKstPaF3snFUMTJwFfK/cksYnMx8FjizZfTmws9jeCWzqdI5RNHG/CLSM4orjjcBj5VZSqpuBjwA/\nK7uQkv0S8IOI+FxEfCsibo+Is8ouqgyZ+SPgJuA/aU26Hc3MPeVWVbqZzDxcbB8GOt6peBRNvO7/\nTX6RiDgbuA+4prgir52I+G3g+5m5lxpfhRfOAC4CbsnMi4Dn6PJf5mkVEb8M/BGwhtb/Us+OiN8r\ntagJUtyouGNPHUUT/x5wQdvjC2hdjddSRLwE+BLwV5l5f9n1lOgS4PKIeAq4B3hHRNxdck1leRZ4\nNjP/rXh8H62mXkdvAr6emf+TmQvA39H6u1JnhyPifICIWA18v9PBo2ji3wB+JSLWRMSZwO8AD4zg\n90y8iAjgTmA+M3eUXU+ZMvPjmXlBZv4SrQ+uvpqZ7y27rjJk5iHgmYh4bbHrXcBciSWV6QngLRHx\n8uLfy7toffBdZw8AW4rtLUDHi7+BvuzTiV8EeoH1wHuAb0fE3mLf9Zn5TyXWNCnqHrtdDfx1caHz\nH8D7Sq6nFJn5ePE/sm/Q+qzkW8Bt5VY1PhFxD/A2YFVEPAP8CfBJ4N6IuAp4Griy4zn8so8kVZe3\nZ5OkCrOJS1KF2cQlqcJs4pJUYTZxSaowm7gkVZhNXJIqzCYuSRX2/yhJKYnZf+vwAAAAAElFTkSu\nQmCC\n",
"text/plain": [
""
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"def f(x): return 3 * x + 2\n",
"xs = sp.linspace(0, 10)\n",
"plt.plot(xs, f(xs));"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Skup primjera za učenje $\\mathcal{D}=(\\mathbf{X},\\mathbf{y})$ dobiven je iz $f(x)$, uz dodatan šum:"
]
},
{
"cell_type": "code",
"execution_count": 5,
"metadata": {
"collapsed": false
},
"outputs": [],
"source": [
"X = linspace(0, 10)\n",
"y = f(X) + 2 * stats.norm.rvs(scale=3, size=50)"
]
},
{
"cell_type": "code",
"execution_count": 6,
"metadata": {
"collapsed": false
},
"outputs": [
{
"data": {
"text/plain": [
"array([ 0. , 0.20408163, 0.40816327, 0.6122449 ,\n",
" 0.81632653, 1.02040816, 1.2244898 , 1.42857143,\n",
" 1.63265306, 1.83673469, 2.04081633, 2.24489796,\n",
" 2.44897959, 2.65306122, 2.85714286, 3.06122449,\n",
" 3.26530612, 3.46938776, 3.67346939, 3.87755102,\n",
" 4.08163265, 4.28571429, 4.48979592, 4.69387755,\n",
" 4.89795918, 5.10204082, 5.30612245, 5.51020408,\n",
" 5.71428571, 5.91836735, 6.12244898, 6.32653061,\n",
" 6.53061224, 6.73469388, 6.93877551, 7.14285714,\n",
" 7.34693878, 7.55102041, 7.75510204, 7.95918367,\n",
" 8.16326531, 8.36734694, 8.57142857, 8.7755102 ,\n",
" 8.97959184, 9.18367347, 9.3877551 , 9.59183673,\n",
" 9.79591837, 10. ])"
]
},
"execution_count": 6,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"X"
]
},
{
"cell_type": "code",
"execution_count": 7,
"metadata": {
"collapsed": false
},
"outputs": [
{
"data": {
"text/plain": [
"50"
]
},
"execution_count": 7,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"len(_)"
]
},
{
"cell_type": "code",
"execution_count": 8,
"metadata": {
"collapsed": false
},
"outputs": [
{
"data": {
"text/plain": [
"array([ 0.67064434, 7.31239169, 9.56400499, 0.2890283 ,\n",
" 6.96746129, 14.86178311, 12.60975181, -0.28934441,\n",
" 6.44890713, 7.28992995, 13.72389263, 19.61341887,\n",
" 15.62669111, 14.43066191, 8.76710654, 19.48489724,\n",
" -1.27170224, 10.81539578, 21.36130674, 14.3632114 ,\n",
" 14.6825962 , 12.37886072, 5.04860612, 27.01349903,\n",
" 14.15906195, 8.62295154, 16.36435167, 15.92878647,\n",
" 12.01783068, 25.11064324, 24.71867488, 25.64192171,\n",
" 21.53657671, 26.70048261, 30.36258764, 22.37268596,\n",
" 27.29906374, 32.14498482, 12.14117217, 16.54494565,\n",
" 32.29949169, 13.98599392, 26.27138558, 28.5969641 ,\n",
" 29.10679964, 26.91455084, 32.36537598, 31.40052367,\n",
" 29.39580935, 23.59296505])"
]
},
"execution_count": 8,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"y"
]
},
{
"cell_type": "code",
"execution_count": 6,
"metadata": {
"collapsed": false
},
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
"/usr/local/lib/python2.7/dist-packages/matplotlib/collections.py:590: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison\n",
" if self._edgecolors == str('face'):\n"
]
},
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAXkAAAEACAYAAABWLgY0AAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAGrBJREFUeJzt3X+UXHV5x/H3kwRhAypBS4iQElQo2OMPUAMiMYNldlF0\nIYmkop6uAU3PoeVH3NgkxJb1qIcQGxUEe0QJLCKYHGNCRNjN1jKxaSgSgRJIKGhZ2WATKkiRJsrC\nPv3jziabYXYzd+bO3B/zeZ0zJzOzc/c+u0me+c73+3yfa+6OiIhk07i4AxARkfpRkhcRyTAleRGR\nDFOSFxHJMCV5EZEMU5IXEcmwCbV+AzPrB14AXgEG3X26mR0BrAKOBfqBue7+fK3nEhGRcKIYyTuQ\nc/eT3X168bnFQJ+7nwD8tPhYREQaLKrpGit53A50F+93A+dFdB4REQkhqpH8P5vZFjP7bPG5ye6+\nq3h/FzA5gvOIiEhINc/JA+939/82sz8B+szssZFfdHc3M/VOEBGJQc1J3t3/u/jn/5jZWmA6sMvM\njnL3nWY2BXim9DglfhGR6rh76RT5qGqarjGziWb22uL9Q4FWYCuwHugovqwDWDdKoKm9XXnllbHH\noPjjj0Pxp++W5tjdw4+Nax3JTwbWmtnw9/q+u28wsy3AajO7iGIJZY3nERGRKtSU5N39SeBdZZ5/\nDjirlu8tIiK1047XKuVyubhDqInij5fij0+aY6+GVTPHE8mJzTyuc4uIpJWZ4Y1aeBURkWRTkhcR\nyTAleRGRDFOSFxHJMCV5EZEMU5IXEckwJXkRkQxTkhcRyTAleRGRDFOSFxHJMCV5EZEMU5IXEckw\nJXkRkQxTkhcRyTAleRGRDFOSFxHJMCV5EZEMU5IXEckwJXkRkQxTkhcRyTAleRGRDFOSFxHJMCV5\nEZEMU5IXEckwJXkRkQxTkhcRybBIkryZjTezB83sx8XHR5hZn5k9bmYbzOzwKM4jIiLhRDWSvwzY\nBnjx8WKgz91PAH5afCwiIg1Wc5I3s2OADwPfBaz4dDvQXbzfDZxX63lERCS8KEbyXwc+DwyNeG6y\nu+8q3t8FTI7gPCIiElJNSd7MPgI84+4Psm8Uvx93d/ZN44iISANNqPH404F2M/swcAjwOjP7HrDL\nzI5y951mNgV4ptzBXV1de+/ncjlyuVyN4YiIZEuhUKBQKFR9vAUD7dqZ2Uxgobt/1MyWA8+6+9Vm\nthg43N0Xl7zeozq3iGRTb28vK1bcAEBn53za2tpijih+Zoa7l505KafWkXyp4ay9DFhtZhcB/cDc\niM8jIhnX29vLrFkd7NlzNQCbNnWwdm23En1IkY3kQ59YI3kRGUNr6xz6+tqBjuIz3eTz69mwYU2c\nYcUu7EheO15FRDIs6ukaEZFIdHbOZ9OmDvbsCR63tCyis7N77IPkVTRdIyKJpYXXVws7XaMkLyKS\nIpqTFxGRvZTkRUQyTEleRCTDlORFRDJMSV5EJMOU5EVEMkxJXkQkw5TkRST1ent7aW2dQ2vrHHp7\ne+MOJ1G0GUpEUq20W2VLy6JMd6vUjlcRaSrN1q1SO15FRGQvdaEUkVRTt8qxabpGRFKvmbpVak5e\nRCTDNCcvIiJ7KcmLiGSYkryISIYpyYuIZJiSvIhIhinJi4hkmJK8iERGjcKSR3XyIhKJZmsUFhfV\nyYsI0PhR9YoVNxQTfAcQJPvhXagSHyV5kQwaHlX39bXT19fOrFkdTTd9smULDA3FHUX8lORFMiiO\nUXVn53xaWhYB3UB3sVHY/Lqes5Q7/OQn8IEPwNy58PTTDT19ItXUhdLMDgE2AgcDrwHucPclZnYE\nsAo4FugH5rr78zXGKiIJ1tbWxtq13SMahTVuPn5wEFatguXLYdw4WLwYPvYxmKA+u7UvvJrZRHff\nbWYTgE3AQqAd+K27LzezRcAkd19ccpwWXkXqpNkWQW+5BVauhCVLoLUVrOJlyfSJrQulmU0kGNV/\nGlgDzHT3XWZ2FFBw9xNLXq8kL1JHzdR+1z3biX2khid5MxsHPAC8Bfgnd/87M/udu08qft2A54Yf\njzhOSV4k5Rr9RjIwAEceCQcfXNfTJFrYJF/zjJW7DwHvMrPXA71mdmbJ193Mymbzrq6uvfdzuRy5\nXK7WcESkQUqnhDZt6qjblNC2bcF8+/r10NMD06dHforEKhQKFAqFqo+PdDOUmf09sAf4DJBz951m\nNgW4R9M1ItnSiAtob94MV18N//7vcOmlcPHFMGnSgY/LsoZuhjKzN5rZ4cX7LUAeeBBYz76/+Q5g\nXS3nEZHms3EjfPKT0NYGTz4JS5cqwVej1umaKUB3cV5+HPA9d/+pmT0IrDaziyiWUNZ4HpG6aaYF\nyijV+wLaM2bAE0+oDLJW6l0jTa3ZSg2jFsUb5O7dQXXMoYc27pxpFna6BneP5RacWiRe+fxsh5s9\nSDPucLPn87PjDqspPPus+xe/6H7kke6rV1d2TE9Pj7e0TC7+nd3sLS2TvaenZ9TX5vOzPZ+fPepr\n0qiYOyvOtWprICINNTAACxbAW98K/f1QKMD551d2bKXtGtS7Zx/NdklTq/e8suyvvx9OOQXmzYOH\nH4ZjjqnPefZ/M4A9e4Lnmm1qB5TkpcnF2W+lGU2bBr/+Nbz2tdUdrzfl8LTwKiKRcw9GzxMnRv+9\nK1l4LbegvnTpJWzc+MCYx6VBbL1rwlKSF8mekd0gP/5xuOKK+GIZ+WYwc+YpfOUr38xEFZWSvEgT\niruscPduuPFGWLEimJJZvDjYxJSUpmGN2J3bKA3vXSMi8WpkD5lyXnwRTjgBTjstGMWfempDTisV\nUpIXSbm4K0kOOwx+8QuYMqXyYxr9yaOZF2yV5EWkYi+/XL7NQNgE3+hPHs1cRaU5eZGYRDWabURr\nhnvvhWXLgl7u3/lObd8rS/PjcWhoF0oRqU6UOzKHR6n5/Hry+fVjJvje3l5aW+fQ2jrngOcbeVHs\nT3wiWEi99tqqQpQ4hemBEOUN9a6RGqS9L0kcPXPC9H155RX3973P/R3vcL/tNvfBwXjikFcjZO8a\nzclL6sRdTZJWoy3QDn8N9k0bjRsHN90UVM1EXQbZzPPjsQjzjhDlDY3kpUpZ6BxZy2i22k8x5X5v\nJ5/8fj/kEI2q0wSN5EWSr9rRbC2fYkrLCA8++Bp27FjCH/5wKvCnQHM38soqJXlJnazUPLe1tYVO\nprXUxA+/sXzxi3fR338ev//9fUyceBdwH8NJXrJH1TWSOmGqSdImTPVLNdasaeO//usaLr74TJ56\n6iC+/e1DaGm5BOgGuotvmPMjP6/ER3XyIglRSb17rTXxDzwAJ50ELS37n7eZL6eXNmpQJpJSlW4S\nUlJubmpQJpJxY83lD3eDvOuuYCPTuAgnZPXmkk5K8iIJUcuC8rPPwvXXw3XXwRlnQFdX9AleexPS\nSQuvIglRbkEZOOBC7IoVcPzxwWX1fvYz+NGPom/3W+kFtCV5NJIXSZCRUzGVjp5nzICtW+Hoo6OL\no3RqRtJLSV4koSqtiZ8+PdrzlntzWbr0EjZtWpT6vQnNSEleJDVyvPLK3XU/S7k3l40b16vfTEop\nyYtELKoqlMsu+2sKhTsYHPwoAAcddBWdnRdGFmdY1ezQlfipTl4kQlFdwGPdOrj8cnj9659j/Pjr\necMbHmLhwsaULTbiIiRSvYZuhjKzqcAtwJGAAze4+7VmdgSwCjgW6AfmuvvzJccqyUvmRHXVo/vv\nh6Gh+C6KrZr45Gr0ZqhBYIG7P2RmhwG/MLM+YB7Q5+7LzWwRsLh4E5EKvPe98Z5fUzPZUVOdvLvv\ndPeHivdfBLYDRwPtBB2PKP55Xi3nEUmLzs75tLQsopKGX9u2wWc/G2xkEqmXyDZDmdk04GSCvqWT\n3X1X8Uu7gMlRnUckySrpkLl5M7S3wwc/CNOmwUEHxROrNIdIqmuKUzVrgMvc/fc24nph7u5mVnby\nvaura+/9XC5HLpeLIhyRWI021bFlC3zuc7BjByxcCKtW7d8NUqScQqFAoVCo+viaq2vM7CDgTuBu\nd/9G8bnHgJy77zSzKcA97n5iyXFaeJWmsnUrPPIInH8+TFDxslQp7MJrTdM1FgzZbwS2DSf4ovXs\nKy/oANbVch5Jtnpf6CIO9fiZ3v52uOACJXhprFpLKM8AfgY8TFBCCbAE+DmwmuCaYv2ohDKzslhT\nXcvPNNwN8lOfgje/ud6RSjNq6Eje3Te5+zh3f5e7n1y89bj7c+5+lruf4O6tpQlesiPq7oTlRtCN\n/qRQzc80MAALFuzrBqnRenhZ/ESYBPqnKIkxWmOsr3zlm4ntY/7UU3DllXDHHXDhhfDww3DMMXFH\nlT7qV19H7h7LLTi1pF1PT4+3tEx2uNnhZm9pmew9PT1Vfa98fnbx+3jxdrMfccRbXvVcPj874p9i\nf2F+piefdP/Sl9yffbauIWVeub/7ev89p1Uxd1acazWSl5oM14VnqTthmJ9p2jT4whcaGJxIWGHe\nEaK8oZG8lCg3gv7yl78c2SeFar30kvv3vuf+0EMNPW1TifITYdYRciSvLpSSKOUaY8XVLGv3bli5\nEv7xH+G444I/3/3uhpy6KakpWmUa2oWyFkryklQvvADXXBNcFPv002HRIjjttLijEgk0ugulSOa8\n8krQeqBQgJNOijsakdpoJC8ikiIN3QwlkiRhN9Ns3gz33deAwERipCQvmTC8maavr52+vnZmzeoo\nm+jd4a674AMfCFoP7NwZQ7AiDaTpGsmEA1127+WX4Qc/gOXLwSxYTJ07V+0HJH208CoywnBZ3tDQ\nBP74x2tYvvwo2tqCRC/SDJTkJRM6O+ezaVMHe/YEj1taFjFz5iUl3STfhVk3Zqq/luahOXnJhOFW\nBDNm/Bvvfe9jrF3bzcaND0TaIVMkjZTkJRO2bYPbb2/jkUdu4IILrkrcbkm10ZW4KMnLqNKQmDZv\nhnPPhTPPDHq5/+pXQV93CKZwWloWAd1ANy0ti+jsnB/p+Sv5HVVa+SNSF2Ea3UR5Qw3KEi0NDaOG\nhtzPOcf9+uvdd+8u/5qenh7P52d7Pj878vgr/R2pja5ECbUalijsf3Uk2LMneC5J0yBmcOedY7+m\nra2tbjGn4XckoiQvibd7dzDn/p73xB1JdcpV/nR2dscblDQNJXkpKwmJafii2NdfH8y7Jy3JV/o7\nyuKFVSQ9tONVRhVXf++BAfja16C7G2bNgs9/Hk48sSGnDk090KXR1E9eEqGW5DdnTnBZvQULdFFs\nkVJK8hK74ZLBfTtNF7F2beVTFO5qOyAyGrUalv3EUeu+f9VJ+Z2m7rB9e/njleBFoqMkn2FJ3IQz\nOAi33grvfCdcdBEMDTU+hjRs8hKJTJii+ihvaDNU3cW1CafcJqF16zb4tde6H3usey7nfvfdwWam\nRkvDJi+RsaDNUBK3ciWDa9bk+e1vYdUqOPXU+GLTBiZpNkryGRZnrXvpTtOzzoLx4xtyahEZoebq\nGjNbCZwDPOPuby8+dwSwCjgW6AfmuvvzJcd5reeWA2t0HfdvfgNvelNdT1GTWit/ROLW8BJKM5sB\nvAjcMiLJLwd+6+7LzWwRMMndF5ccpySfIffeC8uWwf33w+OPw2GHxR3R6LSBSdIsljp5M5sG/HhE\nkn8MmOnuu8zsKKDg7ieWHKMkn3LDF8W++mrYsQMWLoR586ClJe7IRLIrKdd4nezuu4r3dwGT63Qe\nidFVV8Hq1bB4MXzsY7ootkgS1f2/pbu7mZUdsnd1de29n8vlyOVy9Q5HIrRgASxZos1LIvVUKBQo\nFApVH1/P6Zqcu+80synAPZquSa8XXoDXvS7uKEQEktPWYD3DhcjBn+vqdJ5EyOoOyoEB+NzngmZh\nTz4ZdzQiUo2ak7yZ3Q5sBv7MzAbMbB6wDMib2ePAB4uPMymJrQNqtW0bfPrTQeuBcePg4YfhuOPi\njkrSKquDoLRQF8oatbbOoa+vnX0fXLrJ59ezYcOaOMOq2urVcOmlcMklcPHFMGlS3BFJmmlfQvSS\nMl0jKXXOOcHUzNKl5RO8RmUSRiUdSaW+VPRWoyRcJq8ag4NBm4FxJW/zhx46+jGlo7JNmzo0KhNJ\nujDdzKK8kaEulD09PZ7Pz/Z8fnbiOxq++KLv7QZZKOx7vpKfIa6ulpJe6voZPdSFsvFKm3El0fBF\nsa+7Ds44Y/9ukBqhS73oIubx08JrE7jvPvjQh0a/KHali8daRBOJX1LaGjS9JDXBOvnkoAyy1oti\na1Qmkj4ayYdUSfKOc8RbzUWwNUIXSY+wI3ktvIZQ6SJSoxcoh4bc77zTfcYM95tuqu57pGnxWKSZ\nEXLhVXXyISSt5vfll+H73w92pl5xBZx++n9w663nV1XD3tbWxoYNa9iwYY1G8KPQHgFJI83J10Ej\naucHBmDGjKCvzPLlAL3Mnq0KmXpRBZKkVphhf5Q3MjxdM/zaek5/DA25b9my73HSa9jTPh2U9N+v\nNA9UJ18/YapL6l07bwbvfnfdvn2kNAoWiVGYd4Qob6RwJN9ojz7q3tHhvmzZgV+b5J2FWRgFJ/n3\nK80FLbym3733wrnnwplnwvHHw/z5Bz5m+FNGPr+efH69RsoR0+9X0kp18gmyezecfXawqLpwIVx4\nYTYuiq06fJHohK2TV5JPmL6+YASftYtiJ2kHsEiaKcmnTLXJT0lTpDkpySfcc88FnSBfeglmzKhu\nGiMr0x96oxIJT20NEuqpp9wvv9x90iT3efPct2+vvupE1SoizQtV1ySLO3zmM0HrgfHjg26QK1e+\nut1vs0laiwiRrMrY8l7ymMFHPgJf/eqrr5labfuDtF5yUEQaT3PyMWvWhdesrCuINJoWXmMwOBhc\nTu+RR2DZsrijSY+0v1GJxEFJvoF274Ybb4QVK4JukEuWgPKUiNSTLv/XICtWBC1+3//+/S+KLSKS\nJEryVZo6FTZuVJWMiCSbpmtERFIk7HSN6uRTQpeeE5Fq1G0kb2ZnA98AxgPfdferS76ukfwoSqtO\nAJUbigiQkOoaMxsP/CdwFvA0cD9wgbtvH/EaJfkyytWPn3jiiTz44DyC3aEAQV/zDRvWxBaniMQj\nKdU104Ffunt/MagfAOcC28c6SEq3+8OePfDrX38p3qBEJLXqNSd/NDAw4vGO4nNNrdp59WOPPYqW\nlkVAN9BdbGNQweWiRKTp1Wskr3mYEpVezLpcX5qrrgr60lRyAXERkZHqleSfBqaOeDyVYDS/n66u\nrr33c7kcuVyuTuHEr9w0zIoVN7wqWQ9fS7RcQldiF2k+hUKBQqFQ9fH1WnidQLDw+hfAb4Cf0+QL\nr62tc+jra0eLpyJSi0QsvLr7y2b2t0AvQQnljSMTfDNSe2ARiYN2vDaQui6KSK0SUSdf0YmbMMmL\niNRKbQ1ERGQvJXkRkQxTkm9CanYm0jw0J99kdG1VkXTTwquMSfX6IummhVcREdlLl/9rMtqUJdJc\nNF3ThLQpSyS9NCcvIpJhmpMXEZG9lORFRDJMSV5EJMOU5EVEMkxJXkQkw5TkRUQyTEleRCTDlORF\nRDJMSV5EJMOU5EVEMkxJXkQkw5TkRUQyTEleRCTDlORFRDJMSV5EJMOU5EVEMkxJXkQkw5TkRUQy\nrOokb2bnm9mjZvaKmZ1S8rUlZvaEmT1mZq21hykiItWoZSS/FZgF/Gzkk2b2NuAvgbcBZwPfMrPM\nfWIoFApxh1ATxR8vxR+fNMdejaqTr7s/5u6Pl/nSucDt7j7o7v3AL4Hp1Z4nqdL+D0Xxx0vxxyfN\nsVejHiPsNwE7RjzeARxdh/OIiMgBTBjri2bWBxxV5ktXuPuPQ5zHQ0UlIiKRMPfa8q+Z3QN0uvsD\nxceLAdx9WfFxD3Clu99XcpwSv4hIFdzdKn3tmCP5EEaecD1wm5l9jWCa5njg56UHhAlSRESqU0sJ\n5SwzGwBOA35iZncDuPs2YDWwDbgbuNhr/bggIiJVqXm6RkREkivW+nUz+6qZbTez/zCzH5nZ6+OM\np1JmdnZxo9cTZrYo7njCMLOpZnZPcSPbI2Z2adwxhWVm483sQTMLs/ifCGZ2uJn9sPjvfpuZnRZ3\nTGEUNzo+amZbzew2Mzs47pjGYmYrzWyXmW0d8dwRZtZnZo+b2QYzOzzOGMcySvyh8mbcm5Q2AH/u\n7u8EHgeWxBzPAZnZeOA6go1ebwMuMLOT4o0qlEFggbv/OcFU29+kLH6AywimA9P4MfQa4C53Pwl4\nB7A95ngqZmbTgM8Cp7j724HxwMfjjKkCNxH8Xx1pMdDn7icAPy0+Tqpy8YfKm7EmeXfvc/eh4sP7\ngGPijKdC04Ffunu/uw8CPyDYAJYK7r7T3R8q3n+RIMm8Kd6oKmdmxwAfBr7L/gv+iVcccc1w95UA\n7v6yu/9vzGGF8QLBIGGimU0AJgJPxxvS2Nz9X4HflTzdDnQX73cD5zU0qBDKxR82b8Y9kh/pQuCu\nuIOowNHAwIjHqd3sVRyZnUzwDyUtvg58Hhg60AsT6Djgf8zsJjN7wMy+Y2YT4w6qUu7+HLACeAr4\nDfC8u/9zvFFVZbK77yre3wVMjjOYGh0wb9Y9yRfnvraWuX10xGuWAi+5+231jicCaZwieBUzOwz4\nIXBZcUSfeGb2EeAZd3+QlI3iiyYApwDfcvdTgP8j2VMF+zGztwCXA9MIPv0dZmafjDWoGhUr/1L5\nf7rSvBlVnfyo3D0/1tfN7NMEH7//ot6xRORpYOqIx1PZv41D4pnZQcAa4FZ3Xxd3PCGcDrSb2YeB\nQ4DXmdkt7v5XMcdVqR3ADne/v/j4h6QoyQPvATa7+7MAZvYjgr+T78caVXi7zOwod99pZlOAZ+IO\nKKwweTPu6pqzCT56n+vuf4gzlhC2AMeb2TQzew1Bx831McdUMTMz4EZgm7t/I+54wnD3K9x9qrsf\nR7Dg9y8pSvC4+05gwMxOKD51FvBojCGF9Rhwmpm1FP8dnUWwAJ4264GO4v0OIE0DndB5M9Y6eTN7\nAngN8FzxqXvd/eLYAqqQmX0I+AZBdcGN7n5VzCFVzMzOIGgP/TD7PqYucfee+KIKz8xmErTTaI87\nljDM7J0Ei8avAX4FzEvT4quZ/R1BYhwCHgA+UyxASCQzux2YCbyRYP79H4A7CDZs/inQD8x19+fj\ninEsZeK/kqCapuK8qc1QIiIZlqTqGhERiZiSvIhIhinJi4hkmJK8iEiGKcmLiGSYkryISIYpyYuI\nZJiSvIhIhv0/tPhebNOUa0UAAAAASUVORK5CYII=\n",
"text/plain": [
""
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"plt.plot(xs, f(xs), '--')\n",
"plt.scatter(X, y)\n",
"plt.show()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Dvije hipoteze iz našeg modela:"
]
},
{
"cell_type": "code",
"execution_count": 7,
"metadata": {
"collapsed": false
},
"outputs": [],
"source": [
"def h1(x): return h(x, [0,1])\n",
"def h2(x): return h(x, [0,2])"
]
},
{
"cell_type": "code",
"execution_count": 8,
"metadata": {
"collapsed": false
},
"outputs": [
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAXkAAAEACAYAAABWLgY0AAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3X94VOWZ//H3DaIEQcFfgIKCVhJ1WyutVqtAVEIUl2RQ\ny7p1v0t1V7uXbWnd2IK1rXjZrsp3qdXV3e/aaqXr2opVJ1CRiC7RUrFV0UXFBFR+ioAF0SoBArm/\nf5yZOMQAM5kfZ+bk87quuZiZzMl5EuCeZ+7nPvdj7o6IiERTj7AHICIi+aMgLyISYQryIiIRpiAv\nIhJhCvIiIhGmIC8iEmEHZPsNzGwV8CGwG2h19zPM7DDgIeA4YBUwyd23ZnsuERHJTC5m8g5Uuvtp\n7n5G4rlpwAJ3HwE8nXgsIiIFlqt0jXV4XAPMStyfBcRydB4REclArmbyT5nZi2Z2VeK5ge6+MXF/\nIzAwB+cREZEMZZ2TB85293fN7EhggZk1pX7R3d3M1DtBRCQEWQd5d3838ed7ZvYYcAaw0cwGufsG\nMxsMbOp4nAK/iEjXuHvHFPleZZWuMbM+ZtYvcf9gYBzwKjAHmJx42WQgvpeBluztxhtvDH0MGn/4\n49D4S+9WymN3z3xunO1MfiDwmJklv9d/u/uTZvYiMNvM/oFECWWW5xERkS7IKsi7+0rg8508vwUY\nm833FhGR7OmK1y6qrKwMewhZ0fjDpfGHp5TH3hXWlRxPTk5s5mGdW0SkVJkZnsHCay5KKEVECi6x\nFhhpuZgIK8iLSMmKcjYgV29iysmLiESYgryISIQpyIuIRJiCvIhIDg0bNoynn3467GG0U5AXEckh\nM+t00bS1tZVLL72U4cOH06NHD5555pmCjEdBXkSkQEaPHs0DDzzAoEGDClYCqiAvIpJjL7/8Mqee\neir9+/fnsssuY8eOHfTq1YspU6Zw9tln07Nnz4KNRUFeRCSH3J2HH36YhoYGVq5cydKlS7n//vtD\nG48uhhKRaMpVOiTDC67MjClTpjBo0CAAJkyYwCuvvJKbsXSBZvIiEk3uubl1QTLAA5SVlfHRRx/l\n6qfKmIK8iEgehd1jR0FeRCSPUvvr7Nixg+3bt3/qfj4pyIuI5FFq3Xx5eTl9+vRh/fr1VFdXc/DB\nB7NmzZr8nl/95EWkFCX6qoc9jLzZ28+XaT95zeRFRCJMQV5EJMIU5EVEIkxBXkQkwhTkRUQiTEFe\nRCTCFORFRCJMQV5EJMIU5EVEckjb/4mIRNjetv97/vnnqaqq4vDDD+eoo45i0qRJbNiwIe/jyUmQ\nN7OeZvaymc1NPD7MzBaY2XIze9LM+ufiPCIipWrr1q380z/9E6tXr2b16tX069ePK664Iu/nzdVM\n/tvAMiDZaGEasMDdRwBPJx6LiHQLnW3/d8EFF3DJJZfQt29fysrK+MY3vsEf/vCHvI8l6yBvZkOA\n8cAvgORnlBpgVuL+LCCW7XlEREpButv/Pfvss/zVX/1V3seTi+3/bge+CxyS8txAd9+YuL8RGJiD\n84iIpM1uys1mHX5j7rf/W7p0KTfffDNz5szJyRj3Jasgb2Z/DWxy95fNrLKz17i7m1l0+4GKSFHK\nNDjnUsft/9avX9/++M0332T8+PHceeednH322XkfS7Yz+S8DNWY2HugNHGJm/wVsNLNB7r7BzAYD\nmzo7ePr06e33KysrqayszHI4IiLFJbXSZvXq1VRVVfGjH/2Iyy+/PK3jGxsbaWxs7PL5swry7v59\n4PsAZjYGuM7d/4+ZzQAmA7cl/ox3dnxqkBcR6aihoYGZM+8BoK7uaqqrq0MeUeaSG3+88847nHfe\neXzzm9/k6quvTvv4jhPgm266KaPz57pOPvn56FagysyWA+clHouIpK2hoYGJEyezYEENCxbUMHHi\nZBoaGsIeVsaSdfP33nsvK1euZPr06fTr149+/fpxyCGH7P8bZHt+bf8nIsVo3LhLWLCghiAZADCL\nqqo5PPnkI4C2/0v3++iKVxGRCMtFCaWISM7V1V3NokWTaWkJHpeVTaWubta+D5JPUbpGRIrWvhZe\nla5J8/soyItIKVKQT49y8iIiEaYgLyISYVp4FZGS1VnfdtmTgryIlKQo5+NzSekaEZEIU5AXEYkw\nBXkRkQhTkBeRktfQ0MC4cZcwbtwlJdnELJ90MZSIlLRkt8qWltuAoP3BY4/NKsm2xOnQFa8i0q3s\nr1tl1OiKVxERaac6eREpaepWuW9K14hIyYvCNoHpUk5eRCTClJMXEZF2CvIiIhGmIC8iEmEK8iIi\nEaYgLyISYQryIiIRpiAvIjmjRmHFR3XyIpIT3a1RWFhUJy8iQOFn1TNn3pMI8JOBINgnr0KV8CjI\ni0RQcla9YEENCxbUMHHi5G6XPnnxRWhrC3sU4VOQF4mgMGbVdXVXU1Y2FZgFzEo0Crs6r+fsyB0e\nfxxGj4ZJk+Cddwp6+qKUVRdKM+sNPAMcBBwI1Lv79WZ2GPAQcBywCpjk7luzHKuIFLHq6moee2xW\nSqOwwuXjW1vhoYdgxgzo0QOmTYNLL4UD1Gc3+4VXM+vj7tvM7ABgEXAdUAP82d1nmNlUYIC7T+tw\nnBZeRfKkuy2C/upXcN99cP31MG4cWNrLkqUntC6UZtaHYFb/NeARYIy7bzSzQUCju1d0eL2CvEge\ndaf2u+7RDuypCh7kzawHsAQ4AfgPd/+emb3v7gMSXzdgS/JxynEK8iIlrtBvJGvXwlFHwUEH5fU0\nRS3TIJ91xsrd24DPm9mhQIOZndvh625mnUbz6dOnt9+vrKyksrIy2+GISIF0TAktWjQ5bymhZcuC\nfPucOTB/PpxxRs5PUbQaGxtpbGzs8vE5vRjKzH4ItAD/CFS6+wYzGwwsVLpGJFoKsYH2c8/BbbfB\n88/DlClwzTUwYMD+j4uygl4MZWZHmFn/xP0yoAp4GZjDJ3/zk4F4NucRke7nmWfg8suhuhpWroQb\nblCA74ps0zWDgVmJvHwP4L/c/WkzexmYbWb/QKKEMsvziORNd1qgzKV8b6A9ahSsWKEyyGypd410\na92t1DDXcvEGuW1bUB1z8MGFO2cpyzRdg7uHcgtOLRKuqqqLHe73IMy4w/1eVXVx2MPqFjZvdr/p\nJvejjnKfPTu9Y+bPn+9lZQMTf2f3e1nZQJ8/f/5eX1tVdbFXVV2819eUokTsTDvWqq2BiBTU2rVw\n7bXwmc/AqlXQ2Ahf+Up6x6bbrkG9ez6hbJd0a/nOK8ueVq2CkSPhiitg6VIYMiQ/59nzzQBaWoLn\nultqBxTkpZsLs99KdzRsGKxeDf36de14vSlnTguvIpJz7sHsuU+f3H/vdBZeO1tQv+GGb/HMM0v2\neVwpCK13TaYU5EWiJ7Ub5GWXwfe/H95YUt8MxowZyU9+8m+RqKJSkBfphsIuK9y2De69F2bODFIy\n06YFFzEVS9OwQlydWygF710jIuEqZA+Zznz0EYwYAWeeGcziv/SlgpxW0qQgL1Liwq4k6dsXXnoJ\nBg9O/5hCf/Lozgu2CvIikrZduzpvM5BpgC/0J4/uXEWlnLxISHI1my1Ea4bFi+HWW4Ne7j//eXbf\nK0r58TAUtAuliHRNLq/ITM5Sq6rmUFU1Z58BvqGhgXHjLmHcuEv2e77UTbG/+tVgIfXOO7s0RAlT\nJj0QcnlDvWskC6XelySMnjmZ9H3Zvdv9rLPcP/c59wcfdG9tDWcc8mlk2LtGOXkpOWFXk5SqvS3Q\nJr8Gn6SNevSAX/4yqJrJdRlkd86PhyKTd4Rc3tBMXrooCp0js5nNdvVTTGe/t9NOO9t799asupSg\nmbxI8evqbDabTzEdywgPOugO1q27nu3bvwQcC3TvRl5RpSAvJScqNc/V1dUZB9NsauKTbyw33TSP\nVati/OUvf6RPn3nAH0kGeYkeVddIycmkmqTUZFL90hWPPFLN22/fwTXXnMuaNb34z//sTVnZt4BZ\nwKzEG+bVOT+vhEd18iJFIp1692xr4pcsgZNOgrKyPc/bnbfTKzVqUCZSotK9SEhBuXtTgzKRiNtX\nLj/ZDXLevOBCph45TMjqzaU0KciLFIlsFpQ3b4a774a77oJzzoHp03Mf4HVtQmnSwqtIkehsQRnY\n70LszJlw4onBtnrPPguPPpr7dr/pbqAtxUczeZEikpqKSXf2PGoUvPoqHHNM7sbRMTUjpUtBXqRI\npVsTf8YZuT1vZ28uN9zwLRYtmlry1yZ0RwryIiWjkt27n8j7WTp7c3nmmTnqN1OiFORFcixXVSjf\n/vbXaWysp7V1AgC9et1CXd2VORtnprpyha6ET3XyIjmUqw084nH4znfg0EO30LPn3Rx++Ctcd11h\nyhYLsQmJdF1BL4Yys6HAr4CjAAfucfc7zeww4CHgOGAVMMndt3Y4VkFeIidXux698AK0tYW3KbZq\n4otXoS+GagWudfdXzKwv8JKZLQCuABa4+wwzmwpMS9xEJA2nnx7u+ZWaiY6s6uTdfYO7v5K4/xHw\nBnAMUEPQ8YjEn7FsziNSKurqrqasbCrpNPxatgyuuiq4kEkkX3J2MZSZDQNOI+hbOtDdNya+tBEY\nmKvziBSzdDpkPvcc1NTAeefBsGHQq1c4Y+2OPtj+AU+9/VTYwyionFTXJFI1jwDfdve/WMp+Ye7u\nZtZp8n369Ont9ysrK6msrMzFcERCtbdUx4svwj//M6xbB9ddBw89tGc3SMmPdz58h/rmeuqb61m8\ndjHnH38+5w0/jx5WGhf8NzY20tjY2OXjs66uMbNewO+AJ9z9Z4nnmoBKd99gZoOBhe5e0eE4LbxK\nt/Lqq/Daa/CVr8ABKl7OG3dn2XvLiDfFqW+u563332L8ieOJlceo/kw1fQ/sG/YQs1Lo6hojSD5u\ndvdrU56fkXjuNjObBvR392kdjlWQj4goVmJE8WeKst1tu3lu7XPUN9cTb4rT2tZKbXktteW1jD5u\nNL16RicnVuggfw7wLLCUoIQS4HrgT8Bsgj3FVqESysiKYk11Nj9Tshvk3/0dHH98vkfavbW0trDg\n7QXUN9Uzd/lcBvcbTKw8RqwixucHfZ7UtHGUZBrk097xO9e34NRS6qqqLna438ETt/u9quriLn+/\n+fPne1XVxV5VdbHPnz9/r8/lU1d+pjVr3L/zHfcBA9yvvNJ99eq8DzNy0vl7/vPHf/ZZr8zyib+Z\n6IfccohX3l/pty++3d/e8naBRxueROxMO9YqMyhFY2+NsX7yk38r2j7ma9bAjTdCfT1ceSUsXQpD\nhoQ9qtKzr46bq7auor6pnnhznCXvLuG84edRW17LPRPu4Yg+R4Q88hKQyTtCLm9oJh8J8+fP97Ky\ngYmZ7/1eVjawy7PtzmbQhx12Qk4/KaQjk59p5Ur3m29237w5r0OKvD3/7tucQTf58VdW+Kn/caof\nOeNIvzJ+pc9pmuPbdm4Le6ihQzN5KaRkXXiUuhNm8jMNGwY/+EEBBxdRbdYGw5dBxRQor4e27exu\nOZS7xt/FWUPOomePnmEPsWSpQZkUjc4WPDuma8JY2G1tDWraP/tZOPXUgp028j7a+RENbzYQb45T\nv6yej9a24MtqoHkkvf9yJ/HHflXyE4Z8KGh1TTYU5KUznZUuhlXOuG0b3Hcf/Ou/wvDhwZ9f+EJB\nTh1ZGz/ayNzlc4k3xXlm9TOcOeRMJlZMpKa8htcXv66y1TQoyItk6cMP4Y47gk2xv/xlmDoVzjwz\n7FGVrhWbVxBvihNvjvP6ptep/kw1sfIYF554If179w97eCWn0F0oRSJn9+6g9UBjI5x0UtijKT1t\n3saL619sv+L0/Zb3qS2v5Yejf8i5w87loAMOCnuI3Ypm8iKStR27drBw1ULqm4IeMYf2PpSJFROJ\nVcT44tFfLJk+MaVA6RrptjLN3T/3HPTsGd7GHKXug+0fMG/FPOLNcRrebOCUo04hVh6jtqKWEYeP\nCHt4kaUgL91Suq0I3OGJJ+DWW4OUzO23Q21tGCMuTR07Oo4+bjS15bXUlNcwsK86iheCgrx0S/vb\ndm/XLvjNb2DGDDALFlMnTVI3yP3xiHd0LEVaeBVJkUzhtLUdwI4ddzBjxiCqq4NAL53b3babxesW\ntwf2nbt3EiuPcevYWxl17KhIdXTsDhTkJRLq6q5m0aLJtLQEj8vKpjJmzLc6pHA+j9kszFR/3VHH\njo5H9zua2vJaZl86O9IdHbsDpWskMhoaGvjJTx5h+/bDufnmSmbOvGefKZzubvO2zfxu+e+ob67n\n6ZVPc9qg04hVxKgtr2X4gOFhD0/2Quka6ZaWLYNf/7qa116r5oc/hOpq2ittikGxbEKS2tHxpfUv\nMfb4seroGHWZdDPL5Q11oSx6he7j3hV/+IN7TY37UUe5//jH7lu2fPK1XHbI3Jt0fkeFGMfetLW1\n+ZL1S/zGhTe2d3S8In6Fx9+I+8c7Py7IGCS3yLALpYK8dCrMwJSutjb3iy5yv/tu92176UCbzzeq\ndH9Hud5YZX927trpT7/9tE+ZN8WPvf1YP+GOE7yuoc5/v/r3vmv3rrydVwoj0yCvdI10aubMexIL\nlkE+u6UleK6YmkaZwe9+t+/XVFdX523MxfQ7SnZ0rG+u5/EVj3PCgBOoLa9l3lfncfKRJ2vhtBtT\nkJeit21bkHP/4hfDHknXdFb5U1c3K+vvm9rR8dnVz3LmkDOpLa/lX87/F4Ycou2pJKAgL53KV2DK\nRHJT7LvvDq5KLbYgn+7vKJcbqyzfvLx94TTZ0fHyz17OAxc/oI6O0imVUMpehVURsnYt/PSnMGsW\nTJwI3/0uVFQU5NQZy/fvKLWjY7wpztbtW6kpryFWEVNHx25KbQ2kKGQT/C65JNhW79pru+em2Dt3\n72ThyoXtV5z2792f2vJaJp40UR0dRUFewpdus7C9ce9+bQeSHR3rm+tpeKuBk488WR0dpVMK8rKH\nMFIu+2sWBkEgb2rq3ptyrPtwHXOa5xBvivP8uucZfdxoYhUxJoyYoI6Osle64lXadZxRL1o0ueCb\nYHeU3BR7xgzo2xcWLYIeBc4+hLXW4CkdHePNcd5+/23Gnzier3/h6zz6N4+qo6PkhWbyEZbOjDof\nOkvX/PrX/8WaNVXMnBlsij11KqF0g8w2lZSpvXV0rK2oVUdH6RLN5CV0nZUMPvJIFX/+czCLD3Mn\npkJcwNTS2sJTbz9FvCnO71b8jsF9BxOriKmjo4RCQT7Cwqx173il6dixwVZ7UdWxo+PIwSOpLa/l\nB6N/oI6OEqqs0zVmdh9wEbDJ3T+beO4w4CHgOGAVMMndt3Y4TumaAih0/nn9ejj66LyeIiu5TNes\nfH9l+1Z4qR0dLxpxkTo6St4UvLrGzEYBHwG/SgnyM4A/u/sMM5sKDHD3aR2OU5CPkMWLg31TX3gB\nli8PFlWLVVff+NydVza8Qn1zPfGmOOv/sp4JIyZQW1HL2OPH0qdXn3wOWwQIqYTSzIYBc1OCfBMw\nxt03mtkgoNHdKzocoyBf4txh3jy47bZgU+zrroMrroCysrBHljutu1v5/Zrfty+c9urRi9ryWmIV\nMb489Mv07BHhHJQUpWJZeB3o7hsT9zcCKvqNoFtugdmzYdo0uPTS6GyKvbeOjo9/9XFOOfIULZyW\nsrY22LQJBg0KeyQFk/f/lu7uZtbplH369Ont9ysrK6msrMz3cCSHrr0Wrr8+Glenbvp4E3Ob5xJv\njvPMqmc4a+hZ6uhYyrZtgxUrgivuUm/Ll8Mxx0Bzc8n8w21sbKSxsbHLx+czXVPp7hvMbDCwUOma\n0vXhh3DIIWGPIvc66+gYK49x4YkXqqNjKXCHDRuCgN0xmG/cCCecAOXlwWXVFRXBbcSIkv/HXCzp\nmjkEhcjJguR4ns5TFIpl/85cW7sWbr8d7r8fXnopuIiplLV5Gy+880L7wmmyo+MPR/9QHR2L2c6d\n8OabQfDuGNAPPPCTAF5eHtTqVlQEHe6ikj/MUi6qa34NjAGOIMi//wioB2YDxxLxEspCX0FZCMuW\nBW0H5syBK6+E73yndLtB7ti1g4WrFlLfVL9HR8failrOOOYMdXQsgLQnQZs3dz4rX7MGjjsuCOKp\nAb2iAg4/vIA/SXFQg7ICC6t1QL7Mng1TpsC3vgXXXAMDBoQ9osx9sP0DnnjzCeJNcXV0DFnHSVDf\n3t9j3r/fyqgjj/z0zHznzk8CeGqK5YQTghm7AMWTrpESddFFMGHC3ssgizU1lezoWN9cz+K1i9s7\nOt5xwR3q6BiGDz+E5mZeqfsBN7ScTgVzqKCJE7ZvYes134TK0UEwHzkSLr88COYDB5bMYmgpUZDP\nUjFsk9cVra1Bm4GOHSAPPnjvxxRTV0t35/X3Xm9fOE12dLxq5FU8MukRdXQshLa24AKJ1NRKcma+\ndSuUl3P65vdoZCizmUQz5SznT5wzqoEnnyjNT7qlSOmaHCjW2W1nPv4Y7rsPZs4MttcbMyZ4Pp2f\nIezUVGpHx3hTnJ27d7bvmKSOjnm0r3LE/v0/Sa+k5syHDIEePSK5ZhU2pWtC0LEZVzFKbop9111w\nzjl7doMsphl6R6kdHecun8vR/Y4mVhHj4a88rI6OuZQsR+ysgiVZjpjMl190EdTVpVWOmMtNzKVr\nNJPvBv74R7jwwr1vip3uDL1Qs7LN2zbz+IrHiTfF2zs6xspj1JTXqKNjtlLLETsG9NRyxNQKFpUj\nFhXN5ItEMaVwTjsNli7Nvgwyn7Oy1I6OS95dwnnDz2NixUR+PuHnHN6n+5XJZW3z5s5n5R3LEceM\nga9/PXjcDcsRuwPN5DOUTvAOMw/ZlU2wwxhvsqNjsvFXsqNjrCLG2OPHUtYrQl3O8mXXLli1qvNZ\nebIcsWNJosoRS57q5PMo3WBY6AXK1G6QV14JX/ta5t+jEJ88OuvoGKuIUVteq46O+5IoR/xUIH/r\nraDRVmowTwZ0lSNGltI1eVSIreMysWtXsIB6223B/+cLL/xfHnjgxzz4YFvGgTpfi8fJjo7x5jjz\nVsxjeP/hTKyYWJIdHfP6RthZOWLy9sEHwSJncjY+aVIQzE88Efqoh73sm4J8HhSidn7tWhg1KlgT\nmzEDoIGLLy6OCpmNH21k7vK5xJviPLv6Wc4cciaxihi3nH9LyXZ0zFkFUsdyxNQZev/+e6ZXYrE9\nyhFFukLpmgxkkrvOd/rDHZYsgS98IXgcdg37is0r2tMwr216jQs+cwG15bXtHR2LaSG6KzL6/WZS\njphaY17i3RGlMJSuyaNMqkvyXTtv9kmAD0NnHR2TG1d37OhYzHX42TigrQ3eeKPzFMtBB+15cdDY\nscHj4cNVjigFpZl8EUt2gzzpJJg6dd+vLUSFTLKjY7wpzpzmOfTv3b994fT0Y07fa0fHsD9lZG3z\nZv44axb3T7uJ4a1nU8G7nGSvcvwBPeg5bNinK1hUjih5pJl8BCQ3xX7++aAj5NVX7/+YfNWwf7D9\nA+atmEd9cz3z35zPKUedQm15LQsnL6T8iPKsv3/RSC1H7JhiaW3lSxUVDBlzBg2r1vByn74c9s3/\n4MTJk1WOKEVPM/kism0bXHBBsKh63XVBOWQYm2K/8+E77RcmpXZ0nDBiQpc6OhZV/5LUcsTUgP7W\nW0HZYeoVn8lZ+aBBKkeUoqE6+RK3YAGce25h07buzrL3lrXn1996/y3GnzieWHmM6s9U56SjY0EX\nXvdXjphc8EwN5ipHlBKhIF9iuhr8sg2au9t289za59oDe2tba7BjUnkto48bXRodHTvrjtjcvGc5\nYmq+vLxc5YhS8hTki9yWLUEnyJ07YdSorqUxupr+aGltYcHbC6hvqmfu8rkM7jeYWHmMWEUslI6O\nab1RpZYjdsyXp5YjppYkVlSoHFEiS0G+SK1dCz/9adDDPRaD730PpkzpWtVJJtUqnXV0TM7Yw+zo\n2PGN6pDe3+PxO27mnOS2cJ2VI3bcFm7YsGDnE5FuRNU1RcYdrroKHn00WEjNRTfI/Vm1dVX7jkkv\nrX+J848/n9ryWu6ZcA9H9Dkivyffny1boKmJ1/75+0xvGUkFj1BBE8du38Kfr62D888NAvjo0eqO\nKJIDmskXQDwedHTtuCl2V9MuHY/rXfY9/vWBH7FpwKY9OjrWVtQy9vix9OlV4AXF/ZQjUlHBk2ve\nZeG7Z9DEV2mmnDdZTGXVvNKpnRcJidI1JaarC6iPP/E4N953K5sOe5cdw/5Cv4P7EasI8utnDTmr\nMB0dO5YjJoN6sjtixwqWlM2ai6qsUqSEKMiHoLU16Ab52mvBRUz50rGj4/EDjg/2OK2YyMlHnpyf\nhdOO5YipQT2xWfMeuwhlUI5Y6v1sRMKgIF9A27bBvfcGm2IPGwbXXw+5jlN76+hYU16T246O+9us\nOTWQJxdAVY4oUnAK8gUyc2bQV+bss4O+MslNsXMh2dEx3hzn9U2vU/2ZamLlsfaOjl3mHpQddnaR\nULIcsWMPFnVHFCkqCvIFMns2fO5zn94UuyvavI0X178YBPZER8ea8hpiFbFPdXRMS+pmzR1z5qmb\nNaemWLRZs0hJUJAvEcmOjvVN9cxZPodDDzq0feH0i0d/ca8dHfewefOng3hnmzWnzsqPCLmEUkSy\noiBfxJIdHePNcRrebOCUo04hVh6jtqKWEYeP6PygRDnikgcf5MUHHmbIto84q//BDNi4MZixd7xA\nSJs1i0Ra0QR5M7sA+BnQE/iFu9/W4evdIsiv+3Adc5rnEG+K8/y65xl93Ghqy2uZUD6BQX0HffLC\nlHLEt+bNY/3/PMvQj//C0NYd7BgwgEXvvc+ytjE0MZiVB85l2v13c+5ll6k7okg3UxRB3sx6As3A\nWOAd4AXgb939jZTXRDLIJzs6JrfC26Oj4/FV9N209dN15SnliO8eeii/WPQCr+36e5oYzLred3Hc\nSSfx8stXULKbbohIzhRLW4MzgDfdfVViUL8BaoE39nVQqdrdtpvF6xa3txLYuWsHsSPO4dae1Yxq\nMXrNXgFNP4blfx+UI6bmymOx4PHQodCjB5PHXcKCXXfTHtC3Hwurbw715xOR0pWvIH8MsDbl8Tog\nh0WG4WudctQ0AAAGq0lEQVTZuY2nlvyW+P/+hrkbf8/Rrb2pffdQZi/Zxuff2IKdsBQqEjnziy6C\nujqeWrOGGf/vvwGoq61N++Kf444bREvLVFpagsdlZVOpq5uVrx9NRCIkX0E+enmYFEuffZhR8ycx\n8r1exD4ewg8OuYjhJ54OXyqHaZ2XIzY0NDDx767Z72bWdXVXs2jR5D0C+i23BAE911v7iUj05SvI\nvwMMTXk8lGA2v4fp06e336+srKSysjJPw8mtk04fz1vDmzhiaPp7nM6ceU8iwAdpmJaW4LmOwXpf\ne7UqsIt0P42NjTQ2Nnb5+HwtvB5AsPB6PrAe+BPdZOF1bzLpAS8isjdFsfDq7rvM7JtAA0EJ5b2p\nAb476iwNo7y6iOSbLoYqIHVdFJFsFUWdfFon7oZBXkQkW5kGefWJFRGJMAV5EZEIU5DvhhoaGhg3\n7hLGjbuEhoaGsIcjInmknHw3o71VRUqbFl5ln1SvL1LatPAqIiLttN9bN6OLskS6F6VruiFdlCVS\nupSTFxGJMOXkRUSknYK8iEiEKciLiESYgryISIQpyIuIRJiCvIhIhCnIi4hEmIK8iEiEKciLiESY\ngryISIQpyIuIRJiCvIhIhCnIi4hEmIK8iEiEKciLiESYgryISIQpyIuIRJiCvIhIhHU5yJvZV8zs\ndTPbbWYjO3ztejNbYWZNZjYu+2GKiEhXZDOTfxWYCDyb+qSZnQz8DXAycAHw72YWuU8MjY2NYQ8h\nKxp/uDT+8JTy2Luiy8HX3ZvcfXknX6oFfu3ure6+CngTOKOr5ylWpf4PReMPl8YfnlIee1fkY4Z9\nNLAu5fE64Jg8nEdERPbjgH190cwWAIM6+dL33X1uBufxjEYlIiI5Ye7ZxV8zWwjUufuSxONpAO5+\na+LxfOBGd/9jh+MU+EVEusDdLd3X7nMmn4HUE84BHjSznxKkaU4E/tTxgEwGKSIiXZNNCeVEM1sL\nnAk8bmZPALj7MmA2sAx4ArjGs/24ICIiXZJ1ukZERIpXqPXrZvZ/zewNM/tfM3vUzA4NczzpMrML\nEhd6rTCzqWGPJxNmNtTMFiYuZHvNzKaEPaZMmVlPM3vZzDJZ/C8KZtbfzH6b+He/zMzODHtMmUhc\n6Pi6mb1qZg+a2UFhj2lfzOw+M9toZq+mPHeYmS0ws+Vm9qSZ9Q9zjPuyl/FnFDfDvkjpSeAUdz8V\nWA5cH/J49svMegJ3EVzodTLwt2Z2UrijykgrcK27n0KQavtGiY0f4NsE6cBS/Bh6BzDP3U8CPge8\nEfJ40mZmw4CrgJHu/lmgJ3BZmGNKwy8J/q+mmgYscPcRwNOJx8Wqs/FnFDdDDfLuvsDd2xIP/wgM\nCXM8aToDeNPdV7l7K/AbggvASoK7b3D3VxL3PyIIMkeHO6r0mdkQYDzwC/Zc8C96iRnXKHe/D8Dd\nd7n7ByEPKxMfEkwS+pjZAUAf4J1wh7Rv7v574P0OT9cAsxL3ZwGxgg4qA52NP9O4GfZMPtWVwLyw\nB5GGY4C1KY9L9mKvxMzsNIJ/KKXiduC7QNv+XliEhgPvmdkvzWyJmf3czPqEPah0ufsWYCawBlgP\nbHX3p8IdVZcMdPeNifsbgYFhDiZL+42beQ/yidzXq53cJqS85gZgp7s/mO/x5EAppgg+xcz6Ar8F\nvp2Y0Rc9M/trYJO7v0yJzeITDgBGAv/u7iOBjynuVMEezOwE4DvAMIJPf33N7PJQB5WlROVfSf6f\nTjdu5qpOfq/cvWpfXzezrxF8/D4/32PJkXeAoSmPh7JnG4eiZ2a9gEeAB9w9HvZ4MvBloMbMxgO9\ngUPM7Ffu/vchjytd64B17v5C4vFvKaEgD3wReM7dNwOY2aMEfyf/HeqoMrfRzAa5+wYzGwxsCntA\nmcokboZdXXMBwUfvWnffHuZYMvAicKKZDTOzAwk6bs4JeUxpMzMD7gWWufvPwh5PJtz9++4+1N2H\nEyz4/U8JBXjcfQOw1sxGJJ4aC7we4pAy1QScaWZliX9HYwkWwEvNHGBy4v5koJQmOhnHzVDr5M1s\nBXAgsCXx1GJ3vya0AaXJzC4EfkZQXXCvu98S8pDSZmbnELSHXsonH1Ovd/f54Y0qc2Y2hqCdRk3Y\nY8mEmZ1KsGh8IPAWcEUpLb6a2fcIAmMbsAT4x0QBQlEys18DY4AjCPLvPwLqCS7YPBZYBUxy961h\njXFfOhn/jQTVNGnHTV0MJSISYcVUXSMiIjmmIC8iEmEK8iIiEaYgLyISYQryIiIRpiAvIhJhCvIi\nIhGmIC8iEmH/H1TiKAOCPR4oAAAAAElFTkSuQmCC\n",
"text/plain": [
""
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"weights = [[0,1], [0,2], [1,2]]\n",
"plt.plot(xs, f(xs), '--')\n",
"plt.scatter(X, y)\n",
"plt.plot(xs, h1(xs), 'r', label='h1')\n",
"plt.plot(xs, h2(xs), 'g', label='h2')\n",
"plt.legend();"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Empirijske pogreške hipoteza na skupu $\\mathcal{D}$:"
]
},
{
"cell_type": "code",
"execution_count": 9,
"metadata": {
"collapsed": false
},
"outputs": [
{
"data": {
"text/plain": [
"5025.9784792784731"
]
},
"execution_count": 9,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"error(h1, X, y)"
]
},
{
"cell_type": "code",
"execution_count": 10,
"metadata": {
"collapsed": false
},
"outputs": [
{
"data": {
"text/plain": [
"2156.5878715137965"
]
},
"execution_count": 10,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"error(h2, X, y)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"**(3) Optimizacijski postupak**\n",
"\n",
"* Tražimo $h\\in\\mathcal{H}$ koja minimizira empirijsku pogrešku\n",
"\n",
"$$\n",
"h^* = \n",
"\\mathrm{argmin}_{h\\in\\mathcal{H}} E(h|\\mathcal{D}) = \n",
"\\mathrm{argmin}_{h\\in\\mathcal{H}} \\frac{1}{2}\n",
"\\sum_{i=1}^N\\big(y^{(i)}-h(x^{(i)})\\big)^2\n",
"$$\n",
"\n",
"* Hipoteza $h$ je indeksirana parametrima $(w_0, w_1)$, dakle zapravo tražimo\n",
"\n",
"$$\n",
"(w_0,w_1)^* =\n",
"\\mathrm{argmin}_{w_0,w_1} \\frac{1}{2} \n",
"\\sum_{i=1}^N\\big(y^{(i)}-(w_1 x^{(i)} + w_0)\\big)^2\n",
"$$\n",
"\n",
"* U ovom slučaju postoji **analitičko rješenje** (rješenje u zatvorenoj formi)\n",
"\n",
"\\begin{eqnarray*}\n",
" && \\nabla_{w_0,w_1} E(h|\\mathcal{D})=0\\\\\n",
" &&\\frac{\\partial}{\\partial w_0}\\Big[\n",
"\\frac{1}{2}\\sum_i^N\\big(y^{(i)}-(w_1 x^{(i)}+ w_0)\\big)^2\\Big] = 0 \\\\\n",
"&&\\frac{\\partial}{\\partial w_1}\\Big[\\frac{1}{2}\\sum_i^N\\big(y^{(i)}-(w_1 x^{(i)}+\n",
"w_0)\\big)^2\\Big] = 0\\\\\n",
"&&\\vdots\\\\\n",
"&& w_0= \\bar{y} - w_1\\bar{x}\\\\\n",
"&& w_1 = \\frac{\\sum_i^N x^{(i)}y^{(i)} - N\\bar{x}\\bar{y} }\n",
"{\\sum_i^N(x^{(i)})^2 - N\\bar{x}^2}\n",
"\\end{eqnarray*}\n"
]
},
{
"cell_type": "code",
"execution_count": 11,
"metadata": {
"collapsed": false
},
"outputs": [],
"source": [
"N = len(X)\n",
"x_mean = sp.mean(X)\n",
"y_mean = sp.mean(y)\n",
"w1 = (np.dot(X, y) - N * x_mean * y_mean) / (sum(X**2) - N * (x_mean**2))\n",
"w0 = sp.mean(y) - w1 * sp.mean(X)"
]
},
{
"cell_type": "code",
"execution_count": 12,
"metadata": {
"collapsed": false
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"2.82127363032 2.57918900236\n"
]
}
],
"source": [
"print w1, w0"
]
},
{
"cell_type": "code",
"execution_count": 13,
"metadata": {
"collapsed": false
},
"outputs": [],
"source": [
"def h_best(x): return h(x, [w0,w1])"
]
},
{
"cell_type": "code",
"execution_count": 14,
"metadata": {
"collapsed": false
},
"outputs": [
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAXkAAAEACAYAAABWLgY0AAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3Xl8VeW1//HPEhQDqIADoFJQq5fqxTriiMRqiFYbBW4p\nrVUUFSuWqdSC2nvFtv5UWiSIoqWiBmcUQbSaEJVoKU6o1AEVJypQCQqiIkGGrN8f+wRDDJAz7nP2\n+b5fr/3iTDt7nQDrPOfZ61nb3B0REYmmHcIOQERE0kdJXkQkwpTkRUQiTEleRCTClORFRCJMSV5E\nJMKaJ/sDzGwx8CWwCdjg7t3NrB3wINAZWAz0c/fVyR5LRETik4qRvAOF7n64u3ePPTYaqHT3g4Cn\nY/dFRCTDUjVdYw3ulwBlsdtlwNkpOo6IiMQhVSP5p8xsvpldHHusvbtXx25XA+1TcBwREYlT0nPy\nwAnu/omZ7QlUmtk79Z90dzcz9U4QEQlB0kne3T+J/fmpmc0AugPVZtbB3ZebWUdgRcP9lPhFRBLj\n7g2nyLcqqekaM2tpZrvEbrcCegFvALOAAbGXDQBmbiXQnN2uvvrq0GNQ/OHHofhzb8vl2N3jHxsn\nO5JvD8wws7qfda+7zzaz+cA0M7uQWAllkscREZEEJJXk3f0j4LBGHl8FnJrMzxYRkeRpxWuCCgsL\nww4hKYo/XIo/PLkceyIskTmelBzYzMM6tohIrjIzPFMnXkVEJLspyYuIRJiSvIhIhCnJi4hEmJK8\niEiEKcmLiESYkryISIQpyYuIRJiSvIhIhCnJi4hEmJK8iEiEKcmLiESYkryISIQpyYuIRJiSvIhI\nhCnJi4hEmJK8iEiEKcmLiESYkryISIQpyYuIRJiSvIhIhCnJi4hEmJK8iEiEKcmLiESYkryISIQp\nyYuIRFhKkryZNTOz18zssdj9dmZWaWaLzGy2mbVJxXFERCQ+qRrJDwMWAh67PxqodPeDgKdj90VE\nJMOSTvJmti/wY+B2wGIPlwBlsdtlwNnJHkdEROKXipH8eOByoLbeY+3dvTp2uxpon4LjiIhInJJK\n8mZ2JrDC3V/j21H8Ftzd+XYaR0REMqh5kvsfD5SY2Y+BnYFdzexuoNrMOrj7cjPrCKxobOcxY8Zs\nvl1YWEhhYWGS4YiIREtVVRVVVVUJ72/BQDt5ZtYT+K27/8TMxgIr3f0GMxsNtHH30Q1e76k6tohE\nU0VFBePGTQZg5MhBFBcXhxxR+MwMd2905qQxyY7kG6rL2tcD08zsQmAx0C/FxxGRiKuoqKB37wHU\n1NwAwNy5A5gxo0yJPk4pG8nHfWCN5EVkG3r16ktlZQkwIPZIGUVFs5g9e3qYYYUu3pG8VryKiERY\nqqdrRERSYuTIQcydO4CamuB+QcEoRo4s2/ZO8h2arhGRrKUTr98V73SNkryISA7RnLyIiGymJC8i\nEmFK8iIiEaYkLyISYUryIiIRpiQvIhJhSvIiIhGmJC8iOa+iooJevfrSq1dfKioqwg4nq2gxlIjk\ntIbdKgsKRkW6W6VWvIpIXsm3bpVa8SoiIpupC6WI5DR1q9w2TdeISM7Lp26VmpMXEYkwzcmLiMhm\nSvIiIhGmJC8iEmFK8iIiEaYkLyISYUryIiIRpiQvIimjRmHZR3XyIpIS+dYoLCyqkxcRIPOj6nHj\nJscS/AAgSPZ1q1AlPEryIhFUN6qurCyhsrKE3r0H5N30yfz5UFsbdhThU5IXiaAwRtUjRw6ioGAU\nUAaUxRqFDUrrMRtyh7//HU46Cfr1g2XLMnr4rJRUkjeznc3sRTNbYGYLzey62OPtzKzSzBaZ2Wwz\na5OacEUkWxUXFzNjRtDLvahoVkbn4zdsgHvugR/+EK66CgYPhkWLoFOnBi9ctgwm59cUUtInXs2s\npbuvNbPmwFzgt0AJ8Jm7jzWzUUBbdx/dYD+deBVJk3w7CTp1KtxxB1xxBfTqBdbwtOTLL0NpKTzx\nBJx7Ltx4IzTPzU7roXWhNLOWwLPA+cB0oKe7V5tZB6DK3bs2eL2SvEga5VP7XfdGEvvGjTBzJowf\nH4zghwyBCy+ENrk9sZDxJG9mOwCvAgcAt7r778zsc3dvG3vegFV19+vtpyQvkuMy/UGyZAnstRe0\naLGNF61eDVOmwMSJsO++MGIEnHVWzo7cG4o3ySf9rt29FjjMzHYDKszs5AbPu5k1ms3HjBmz+XZh\nYSGFhYXJhiMiGdJwSmju3AFpmxJauBDGjoVZs6C8HLp3b+RF778PEybAvffC6afDQw/B0UenPJZM\nq6qqoqqqKuH9U7oYysz+F6gBLgIK3X25mXUE5mi6RiRaMnEB7Xnz4IYb4IUXYOjQ4IRq2/pzAu4w\nZ04w3/7883DxxXDZZbDPPimLIdtkdCRvZnsAG919tZkVAEXANcAsgr/5uhqumckcR0Tyz7PPwvnn\nw+WXw/33Q8uW9Z5cty54sLQ0KK0ZPhweeKDBiwSSHMmbWTeCotgdYtvd7v5nM2sHTAO+BywG+rn7\n6gb7aiQvWSGfTlCmUroreGprg22LqfTqarj1VrjtNjj88CC5N1pOE126xqtIHPKt1DDVUvEBuXZt\nMOvSqtU2XrRgQTDfPnMmS44/njGfr2dJ613z8kM53iSPu4eyBYcWCVdRUR+HuzxIM+5wlxcV9Qk7\nrLywcqX7Nde477WX+7Rpjbxg40b3Rx91Lyx032cf9+uu86enTfOCgvaxv7O7vKCgvZeXlzf688vL\ny72oqI8XFfXZ6mtyUSx3NjnXRqOmSERyxpIlwVqksjI4+2yoqoIf/KDeC776Cu68E266Cdq1C0og\n/+d/YMcdub5X33rtGqCmJmjh0HA0n8nKn2ynJC95beTIQcydO4CamuB+0G+lLNygImzxYjjiCLjg\nAnj99aCMfYsnJ06Eu+6CU04JlrEed1xC8+1b9u7Z+odBPlCSl7xW12/l23nl/BztZUqXLvDvf8Mu\nu8QecId//jOokpkzBwYOhFdfhc6dG91fH8rx04lXEUk592D0vNWKxvXrg8VKpaXwxRcwbBgMGACt\nW2/3ZzflZG9jJ9SvumoIzz776jb3ywWqrhGR0GzYAA8+GKxO7d8frryywQs++yzoAnnLLdC1a1AC\necYZsEPqu57X/zDo2fMIrr12YiSqqJTkRfJQ2LX+a9cG7WLGjQumZEaPhuLietPpCxcGo/aHHoI+\nfYKR+6GHZiy+TKzOzZSM964RkXCFXUmyZg0cdBAce2wwij/mmNgTtbVQXhEk99dfh1/9Ct59N+gw\nJhmjJC+S48KuJGndGl55BTp2jD3w9ddw993B4qWddw6mZGbN2qJ1ZKa/eeTzCVsleRFpso0bG+/Y\n27EjQQH8LbfA7bfDCScE7Qd69vxOCWQY3zzyuooqnpVTqdzQilfJc6lakVleXt7kVaCJmjfPvaTE\n/aKLGnnyhRfc+/d3b9vWfehQ9/ff3+bP0irj5BDnilddyFskBHWj2crKEiorS+jdewAVFRUJ/ax4\nrq1aUVFBr1596dWr73aPV/+i2L/4RXAi9aabYk9u3AjTpgWLlfr3Dxq8f/RRMEVzwAEJvQ9Jk3g+\nEVK5oZG8JCHX+5KEMZqNZ8S/aZP7cce5H3qo+333uW/YEHti1Sr3sWPdO3Vy79HDffr0oMdMmuKQ\n70K9ayTqwq4myVVbO0Fb9xx8exJ0hx2C9jEHHRSbUl+0KBjG33dfUNc+YwYceWRCceT1/HgY4vlE\nSOWGRvKSoCjM6SYzmk30W0xjv7fDDz/Bd955K3HU1ro/9ZT7GWe477mn++9/775sWSJvV1IIjeRF\nsl+io9lkvsU0LCNs0WICS5dewbp1xxBc3ycY3U8ceyvFS5cG9e21tUEJ5EMPQUFBYm9WwhXPJ0Iq\nNzSSlwTl85xust9iysvL/bjjhnrHjs9469brvXPnmQ7THNw78B+/hhJftVML99NPd6+oCEbzklVQ\ndY1EXTzVJLkmnuqXREyfXsyHH05g8OCT+fjjHfnrX3fm2Ba/4i5OZCHfp0OzSt665WZ44om8u6xe\nZMXziZDKDY3kRbbQlG8oyX6LeeUV97VrPaiImTHD/aSTfO0ee/jfvn+I9yk8M2++EeUy4hzJq0GZ\nSJZoahOtpFoCfPnlt1dd2nPP4KpLffrAjjum5k1I2qlBmUjEFRcXbzWx13WDfOKJYCHT5g6+H30U\nJPapU6GoCO65J1jIFIewO11KYpTkRbJEMk20Vq4M2sbcfDOceCKMGQM7mMNz/wiqZJ57Di68EBYs\ngE6d4o5NaxNyl068imSJxk4oA9s9ETtuHBx4YHBZveeeg0fu/4Zj3p0aLFa6+GI49dTgyRtuSCjB\nB8eov5AqSPZ1o3rJbhrJi2SR+lMxTR099+gBb7wB++z0Kdx2G0yaBP/93/CnP8FppyV01aWGUzOS\nu5TkRbJUU/vEd2/5JlxdCtOnQ9++UFkZJPkENfbhctVVQ5g7d1Re9mPPdUryIjmjkE2bngxu1tZC\neTmMHw9vvQWDBwf9ZfbcM+mjNPbh8uyzs9RvJkcpyYukWKqqUIYNu4SqqkfZsOEnAOy443X87rJf\nBNMxEyZAq1ZBCeTPfgY77ZSy+LdmW1U9kr1UJy+SQg2nOgoKRiVUhTJzZtAyZrfdVtGs2S10bTWP\nazvuwn7PPBM0eB8xIiijScOK1FS9B0mPeOvkk0ryZtYJmArsBTgw2d1vMrN2wINAZ2Ax0M/dVzfY\nV0leIqepC5q25+WXgxmZY/yFoASyshLOOw+GDIH990953A2pJj57ZXox1AZghLsvMLPWwCtmVglc\nAFS6+1gzGwWMjm0isj0bNnD0B9OD5L5iBQwbBpMnw667ZiwETc1ER1J18u6+3N0XxG6vAd4G9gFK\ngLpT72XA2ckcRyRXjBw5iIKCUQT/7MtiVSiNlyAuXBiUsa9cGXtg1aqgln3//YNSyNGj4b33giSf\nwQQv0ZKyxVBm1gU4HHgRaO/u1bGnqoH2qTqOSDZrSofMefOgpAR+9CPo0gVaLH43qI454IAg8z/6\nKFRVwdlnQ7NmYbwNiZCUVNfEpmqmA8Pc/SurdzLI3d3MGp18HzNmzObbhYWFFBYWpiIckVBtbapj\n/nz4zW9g6VL47UjnoUFP0eLWUrhpPlxySZDgO3YMIWLJZlVVVVRVVSW8f9LVNWa2I/A48KS7l8Ye\newcodPflZtYRmOPuXRvspxOvklfeeAPefrWGvuvupdnE0mAl6rBhcM45sPPOYYcnOSLeE69JTddY\nMGSfAiysS/Axs/i2vGAAMDOZ40h2S/eFLsKQ8vf0n//Q7cHf0+/yzjR7/NGgzv1f/wqahinBSzrF\n03y+4QacCNQCC4DXYttpQDvgKWARMBto08i+KWifL2GL4qX4knlPn33mfs017h98EHtg/nz3X/7S\nvW1b98suc3/33fQFLnmBOC8aoitDSVKSveZoQ+Xl5V5U1MeLivpsTqyNPZZOibynjz92Hz48yOUX\nnr/RV9w23b1HD/dOndzHjnVftSrtcee6TP8956p4k7zaGkjW2FpjrGuvnZi1fcw//hiuvjooiBl8\nzhd8OOQO2ky9Cd7tGCxZ7dMHmuu/2faoX30axfOJkMoNjeQjIZXTNY2NoNu1OyCl3xSaIp739NFH\n7hNHfOA1lwwLhvH9+7u/8EJa44uiVH8jjDI0kpdMqqsLj1J3wia9J3d47jm6jB/Pr+fOhYsuCk6k\nJnhRDpG0iecTIZUbGslLA42NoP/0pz+FfmJ3/Xr3u+92X7DA3detc7/rLvfDDnP/r/9yv/VW9zVr\nMhpPFEXxBH66EOdIXl0oJas01hgrrGZZa9fCHXfAX/4Ch++zglu63cbej94K3boFXSCLixO66pI0\nTk3RmiajXSiToSQv2erLL4My9ptvhp8f8jpXtprAXnMfgZ/+NFi8dMghYYcoeSzTXShFImfThlra\n/vMJPtxvPK3eeRsuuwzufA/22CPs0ETippG8SJ01a6CsLBjG77JLMCXTr19Grrok0lQZbWsgkk3i\nbUUwbx68+CJBsfvllwctIefMCSbi58+HX/5SCV5ynqZrJBKaupjGHZ58Eq6/ztnrg+eZsF8pvPM0\nnH9+cDmm/fYLIXqR9FGSl0gYN25yLMEHffFqaoLH6pL8xo3wwANw4w0bKPriYR7ZqZTdW67E+g+D\n86cE0zMiEaQkL5FWV5bX+ptNFP/7YOatmUqLbgdiw6+EM8/URTkk8pTkJRJGjhzE3LkDqKkJ7hcU\njKJnzyGMPuscLvmmGz/jJR5vVsGrN43j+MGDww1WJINUXSORUVFRwbXXTmddTTsmnb0LPn4i31tZ\nw18ZxiQGU00FRUWzmD17etihiiRM1TWSlxYuhIfvPokjXzmKJ5c8xlHTpvFs+33ozF+4mj9QTYdQ\n44vihVUkR8TTAyGVG+pdk/Vyob/3P//pfn7RUh/f8kpf02pP/6b4TPenn3avrc1IP5Sm/I7Ul0VS\nCV00RFIhFxJT7Usv+zN7n+M1Ldv6hkuHuC9a9J3XpPODqqm/I7XRlVSKN8nrxKs0ansliaHZuDG4\nQsf48diSJZw8YghcdDO0adPoy4uLi9MWc9b+jkTqUZKXrLd2Lbz74moOf3UKTJwI++4bXHXp7LNz\n4qpLjVX+jBxZFm5Qkjey/3+IhCIbEtPKlXD/H96j4G830b/2XuhzGkybBt27ZzSOrWnq7yiKF1aR\n3KESStmqsPp7L/nYmfWbOew/q5QTmz3PN+ddzB7/OzgYwWcZ9UCXTFM/eckKCSW/devg/vtZPKKU\ngmbr2eny4bQdei60bJnmaEVyh5K8hK5hs7CCglGNNgvbrLoabr0VbrsNDjsMHz4C61Wkqy6JNEKL\noWQLYSzC2bLqJEj2daP6Ou7w4Yx/wcCB0LUrLF8OzzwD5eXYabqsnkiq6H9ShNWNqCsrS6isLKF3\n7wGhr7bcsG4TVb+Zxcu7/IjW/c+g9vsHwvvvB6P4gw/OSAxafSp5JZ6i+lRuaDFU2oW1CKexRUKP\n3z/Tq/re5B81P8Df3uUoX/C7e732m/Vpj6UpsWXbIi+RbUGLoSRs9UsG29d8zR/3OoE9zh/I67v/\niK9vKeOQi48Ha/KUYkppAZPkGyX5CAut1t2d4latKN7V4JWX4ZgL2LTwRo7fv3P6jy0iW0i6usbM\n7gDOAFa4e7fYY+2AB4HOwGKgn7uvbrCfJ3ts2b6M1nGvX8/nkx+ibVkprF4NQ4cGl9XLoqsuxV35\nI5JlMl5CaWY9gDXA1HpJfizwmbuPNbNRQFt3H91gPyX5qPjsMz7+/WRald3CwtquHHX3cAr+54ys\nrZDRAibJZaHUyZtZF+Cxekn+HaCnu1ebWQegyt27NthHST7H+VsLWTKylLZPP0T5zr2pHTqckt8f\nSkFB2JGJRFe8ST5dc/Lt3b06drsaaJ+m40im1dbC7Nkwfjxfz3udx1tdSseb36X3hXvlQq8wkbyT\n9v+W7u5m1uiQfcyYMZtvFxYWUlhYmO5wJFFr18LUqTBhArRoASNG0OzBWVy6W4uwCmVE8kJVVRVV\nVVUJ75/O6ZpCd19uZh2BOZquyVFLl/LNjbfQ4u7b4YQTgha/PXuGVgIpku+ypa3BLOoKkYM/Z6bp\nOFkhkisoX3qJr8/6OWu/fyh3TfqaJdOeh5kzobBQCV4kh6SiuuZ+oCewB8H8+/8BjwLTgO8R8RLK\nSJXkbdwIM2aw9v+Vsva9ZZTWDoULL+RXo3bLxi6/kiNUzZRa6kKZYb169aWysoRvv7iUUVQ0i9mz\np4cZVnxWr4bbb4eJE/m05fcY9ckIvv+bEi4d0py2bcMOTnJZpAZBWSJbqmskF7z3XnAi9b774Iwz\nYPp0Wv7gKG7Zga2WQWpUJvFQG4nwKcknKRsukxcXd3jmGWpvLMVefhEbNAjefBP23huAVtvYteGo\nbO7cARqViWS7eLqZpXIjQl0oy8vLvaiojxcV9cnejoY1Ne5TpvimQ7r5Zx0O9lHtJvtzFWs3P92U\n9xBWV0vJXer6mXqoC2XmFRcXZ+9odvlymDSJ2tv+ygdtjuSqFX9h48lFjBptHHNM8BKN0CVddBHz\nLBDPJ0IqNyI0ks9Kr73mft557m3a+PI+l3r3Xd/2gQPd3377uy9t6ghdozKR8BHnSD47O0hFQCi1\n85s2waOPBrXsP/lJcKWlDz6g7f2TmP5WV6ZMCa60l6i6UVlR0SyKimZptC+SC+L5REjlRo6O5Jsy\nd53xEe+XX7qXlrrvv7/Xdu/uft997uubftUljdBFcgdxjuSV5OPQ1GSYsROUH37oPmKE17Zr58t6\n9PNLfvi833lnYj8qJ04ei4ima9Jpy5rf4ERl3QmljHGHf/wD+vTBjz6ahYuaU7zna5z+xYO0Oa2A\ne+75aUJTRMXFxcyePZ3Zs6drCmYrItm+QiJP1TVpkJba+fXr4cEHobQUvvqKzwcM54T5U9lrTWtG\nlwJU0KePKmTSRRVIkrPiGfanciPC0zV1r03J9Menn7r/8Y/uHTu6n3KK++OPu2/a5LW17vPnf/uy\nbK9hz/XpoGz//Ur+QHXy6RNPzW/StfNvvRWM2h9+GPr2hYoK6NZt89MGHHlk4j8+kzQKFglRPJ8I\nqdzIwZF82m3a5P73v7sXFbl36ODVv/6D/7pftV9//fZ3zeYKmSiMgrP59yv5BY3kc9DXX3971aWW\nLXnvzBGM3qkfc6e1YOhQGDRo+z9CKwvTS79fyVVqNRymJUvg5pthyhTo0YN1l46g1x97sGSp8dvf\nwsCBW+8GmUvUblYkddRPPhe88EIw3z57NgwYAEOGwP77A1BZCSefTOQuiq0WxSKpoSSfrTZuhOnT\nYfx4WLEChg6FgQOpeP75hJKfkqZIflKSzzaffw5/+1swLbPffnx14XBKPyzhm43N6NEjsWmMqEx/\n6INKJH7xJnlV16TLO++4Dx7s3rat+7nn+iePz/fhw4O7F1wQdINMtOpE1Soi+Qu1NQiROzz1FJx5\nJpx0Euy+O/7mW1y001QOPvdImjWD11+HO+5IrhtkFGRFiwiRPBCx03shqamBe+8NTqaawfDh8NBD\nUFCAEeT8P/+Z71wUO9H2Bzl3yUERCY3m5JPxyScwaRJMngxHHx0k91NOCRJ9EyU6L53r89lROa8g\nkmk68ZoJS5fClVfCY4/BL37BhsHDePC1g3jzTbj++rCDyx25/kElEgYl+UxYuRKmTGHtORcz5ZG2\njBsHXbrAFVeA8pSIpFO8SV5z8onYfXfGNfsdY4+AE04IOgDXXRRbRCSbKMknqFMnePZZVcmISHbT\ndI2ISA6Jd7pGdfI5QpeeE5FEpG0kb2anAaVAM+B2d7+hwfMayW9Fw6oTQOWGIgJkSXWNmTUD3gVO\nBZYBLwM/d/e3671GSb4RjdWPd+3alddeu4BgdShAGUVFs5g9e3pocYpIOLKluqY78L67L44F9QBw\nFvD2tnaShsv9g8W0//73H8MNSkRyVrrm5PcBltS7vzT2WF5LdF69c+cOFBSMAsqAslgbgyZcLkpE\n8l66RvKah2mgqRezbqwvzXXXBX1pdOk5EYlXupL8MqBTvfudCEbzWxgzZszm24WFhRQWFqYpnPA1\nNg0zbtzk7yTrbV1LVIldJP9UVVVRVVWV8P7pOvHanODE6ynAf4CXyPMTr7169aWysgSdPBWRZGTF\niVd332hmvwYqCEoop9RP8PlI7YFFJAxa8ZpB6rooIsnKijr5Jh04D5O8iEiy1NZAREQ2U5IXEYkw\nJfk8pGZnIvlDc/J5RtdWFcltOvEq26R6fZHcphOvIiKymS7/l2e0KEskv2i6Jg9pUZZI7tKcvIhI\nhGlOXkRENlOSFxGJMCV5EZEIU5IXEYkwJXkRkQhTkhcRiTAleRGRCFOSFxGJMCV5EZEIU5IXEYkw\nJXkRkQhTkhcRiTAleRGRCFOSFxGJMCV5EZEIU5IXEYkwJXkRkQhTkhcRibCEk7yZ/dTM3jKzTWZ2\nRIPnrjCz98zsHTPrlXyYIiKSiGRG8m8AvYHn6j9oZgcDPwMOBk4DJplZ5L4xVFVVhR1CUhR/uBR/\neHI59kQknHzd/R13X9TIU2cB97v7BndfDLwPdE/0ONkq1/+hKP5wKf7w5HLsiUjHCHtvYGm9+0uB\nfdJwHBER2Y7m23rSzCqBDo08daW7PxbHcTyuqEREJCXMPbn8a2ZzgJHu/mrs/mgAd78+dr8cuNrd\nX2ywnxK/iEgC3N2a+tptjuTjUP+As4D7zOxGgmmaA4GXGu4QT5AiIpKYZEooe5vZEuBY4O9m9iSA\nuy8EpgELgSeBwZ7s1wUREUlI0tM1IiKSvUKtXzezP5vZ22b2LzN7xMx2CzOepjKz02ILvd4zs1Fh\nxxMPM+tkZnNiC9neNLOhYccULzNrZmavmVk8J/+zgpm1MbOHY//uF5rZsWHHFI/YQse3zOwNM7vP\nzFqEHdO2mNkdZlZtZm/Ue6ydmVWa2SIzm21mbcKMcVu2En9ceTPsRUqzgUPc/YfAIuCKkOPZLjNr\nBtxMsNDrYODnZvaDcKOKywZghLsfQjDVdlmOxQ8wjGA6MBe/hk4AnnD3HwCHAm+HHE+TmVkX4GLg\nCHfvBjQD+ocZUxPcSfB/tb7RQKW7HwQ8HbufrRqLP668GWqSd/dKd6+N3X0R2DfMeJqoO/C+uy92\n9w3AAwQLwHKCuy939wWx22sIksze4UbVdGa2L/Bj4Ha2POGf9WIjrh7ufgeAu2909y9CDiseXxIM\nElqaWXOgJbAs3JC2zd3/AXze4OESoCx2uww4O6NBxaGx+OPNm2GP5OsbCDwRdhBNsA+wpN79nF3s\nFRuZHU7wDyVXjAcuB2q398IstB/wqZndaWavmtnfzKxl2EE1lbuvAsYBHwP/AVa7+1PhRpWQ9u5e\nHbtdDbQPM5gkbTdvpj3Jx+a+3mhk+0m911wFrHf3+9IdTwrk4hTBd5hZa+BhYFhsRJ/1zOxMYIW7\nv0aOjeJjmgNHAJPc/Qjga7J7qmALZnYAMBzoQvDtr7WZnRNqUEmKVf7l5P/ppubNVNXJb5W7F23r\neTM7n+CQ/YF6AAABiklEQVTr9ynpjiVFlgGd6t3vxJZtHLKeme0ITAfucfeZYccTh+OBEjP7MbAz\nsKuZTXX380KOq6mWAkvd/eXY/YfJoSQPHAXMc/eVAGb2CMHfyb2hRhW/ajPr4O7LzawjsCLsgOIV\nT94Mu7rmNIKv3me5+7owY4nDfOBAM+tiZjsRdNycFXJMTWZmBkwBFrp7adjxxMPdr3T3Tu6+H8EJ\nv2dyKMHj7suBJWZ2UOyhU4G3QgwpXu8Ax5pZQezf0akEJ8BzzSxgQOz2ACCXBjpx581Q6+TN7D1g\nJ2BV7KHn3X1waAE1kZmdDpQSVBdMcffrQg6pyczsRIL20K/z7dfUK9y9PLyo4mdmPQnaaZSEHUs8\nzOyHBCeNdwI+AC7IpZOvZvY7gsRYC7wKXBQrQMhKZnY/0BPYg2D+/f+ARwkWbH4PWAz0c/fVYcW4\nLY3EfzVBNU2T86YWQ4mIRFg2VdeIiEiKKcmLiESYkryISIQpyYuIRJiSvIhIhCnJi4hEmJK8iEiE\nKcmLiETY/wc/Nyw93H9n4wAAAABJRU5ErkJggg==\n",
"text/plain": [
""
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"plt.plot(xs, f(xs), '--')\n",
"plt.scatter(X, y)\n",
"plt.plot(xs, h_best(xs), 'r');"
]
},
{
"cell_type": "code",
"execution_count": 15,
"metadata": {
"collapsed": false
},
"outputs": [
{
"data": {
"text/plain": [
"892.91671935457725"
]
},
"execution_count": 15,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"error(h_best, X, y)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"* U gornjem primjeru radili smo s modelom prvog stupnja\n",
"\n",
"$$\n",
"h_1(x) = w_1 x + w_0\n",
"$$\n",
"\n",
"* Međutim, mogli smo odabrati i složeniji model, npr. polinom drugog stupnja:\n",
"$$\n",
"h_2(x) = w_2 x^2 + w_1 x + w_0\n",
"$$\n",
"ili četvrtog stupnja:\n",
"$$\n",
"h_4(x) = w_4 x^4 + w_3 x^3 + w_2 x^2 + w_1 x + w_0\n",
"$$\n",
"\n",
"* Ovo je i dalje linearna regresija, i dalje ima analitičko rješenje"
]
},
{
"cell_type": "code",
"execution_count": 16,
"metadata": {
"collapsed": false
},
"outputs": [],
"source": [
"from SU import PolyRegression"
]
},
{
"cell_type": "code",
"execution_count": 18,
"metadata": {
"collapsed": false
},
"outputs": [],
"source": [
"X1 = X.reshape((50,1))\n",
"h2 = PolyRegression(2).fit(X1, y)\n",
"h4 = PolyRegression(4).fit(X1, y)"
]
},
{
"cell_type": "code",
"execution_count": 19,
"metadata": {
"collapsed": false
},
"outputs": [
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAXkAAAEACAYAAABWLgY0AAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3XmcjXX/x/HX19gGSWTJUqRbJJXqhxKmW2PcLcrSdrdM\nSiWtkkj3fafFLYr2uuMOQyWVtSxjwih7ShKjsW9liBthZMZ8fn+cQ4zBnJkzc8655v18PM7DWa5r\nrs8ZfM73fK/P93M5M0NERLypWKgDEBGRgqMkLyLiYUryIiIepiQvIuJhSvIiIh6mJC8i4mHF8/sD\nnHPrgT3AISDDzJo45yoCY4BzgPXALWa2K7/HEhGRwARjJG9AjJk1NrMm/ud6A0lmVg+Y4X8sIiKF\nLFjTNS7b43ZAgv9+AnBTkI4jIiIBCNZI/ivn3GLn3P3+56qaWZr/fhpQNQjHERGRAOV7Th5obma/\nOucqA0nOuZVHv2hm5pxT7wQRkRDId5I3s1/9f253zo0HmgBpzrlqZrbVOXcWsC37fkr8IiJ5Y2bZ\np8hPKF/TNc65Ms650/z3ywJtgGXAJCDev1k8MOEEgUbs7bnnngt5DIo/9HEo/si7RXLsZoGPjfM7\nkq8KjHfOHf5ZH5nZdOfcYuBT59x9+Eso83kcERHJg3wleTNbB1ySw/M7gWvy87NFRCT/tOI1j2Ji\nYkIdQr4o/tBS/KETybHnhcvLHE9QDuycherYIiKRyjmHFdaJVxERCW9K8iIiHqYkLyLiYUryIiIe\npiQvIuJhSvIiIh6mJC8i4mFK8iIiHqYkLyLiYUryIiIepiQvIuJhSvIiIh6mJC8i4mFK8iIiHqYk\nLyLiYUryIiIepiQvIuJhSvIiIh6mJC8i4mFK8iIiHqYkLyLiYUryIiIepiQvIuJhSvIiIh6mJC8i\n4mFK8iIiHhaUJO+ci3LOLXHOfeF/XNE5l+ScS3XOTXfOVQjGcUREJDDBGsk/DqwAzP+4N5BkZvWA\nGf7HIiJSyPKd5J1zNYFrgf8Czv90OyDBfz8BuCm/xxERkcAFYyT/GtATyDrquapmlua/nwZUDcJx\nREQkQPlK8s6564FtZraEP0fxxzAz489pHBERKUTF87n/lUA759y1QGmgvHNuFJDmnKtmZludc2cB\n23LauW/fvkfux8TEEBMTk89wRES8JTk5meTk5Dzv73wD7fxzzrUCnjKzG5xzA4EdZjbAOdcbqGBm\nvbNtb8E6toh4U2JiIoMGDQGgR48HiIuLC3FEoeecw8xynDnJSX5H8tkdztovA5865+4D1gO3BPk4\nIuJxiYmJtG8fT3r6AADmzIln/PgEJfoABW0kH/CBNZIXkZNo06YjSUntgHj/MwnExk5i+vSxoQwr\n5AIdyWvFq4iIhwV7ukZEJCh69HiAOXPiSU/3PY6O7kWPHgkn30mOo+kaEQlbOvF6vECna5TkRUQi\niObkRUTkCCV5EREPU5IXEfEwJXkREQ9TkhcR8TAleRERD1OSFxHxMCV5EYl4iYmJtGnTkTZtOpKY\nmBjqcMKKFkOJSETL3q0yOrqXp7tVasWriBQpRa1bpVa8iojIEepCKSIRTd0qT07TNSIS8YpSt0rN\nyYuIeJjm5EVE5AgleRERD1OSFxHxMCV5EREPU5IXEfEwJXkREQ9TkheRoFGjsPCjOnkRCYqi1igs\nVFQnLyJA4Y+qBw0a4k/w8YAv2R9ehSqhoyQv4kGHR9VJSe1ISmpH+/bxRW76ZPFiyMoKdRShpyQv\n4kGhGFX36PEA0dG9gAQgwd8o7IECPWZ2ZjB5MrRsCbfcAlu2FOrhw1K+ulA650oDs4FSQElgopk9\n45yrCIwBzgHWA7eY2a58xioiYSwuLo7x4xOOahRWePPxGRkwZgwMHAjFikHv3tCpExRXn938n3h1\nzpUxs/3OueLAHOApoB3wm5kNdM71As4ws97Z9tOJV5ECUtROgo4cCcOGwTPPQJs24E5wWvK7X76j\n94zeTP77ZEpGlSzcIIMkZF0onXNl8I3q7wHGAq3MLM05Vw1INrP62bZXkhcpQEWp/a7ZiRP7YZ8t\n/4xuU7ox5PohtG/QvnACKwCFnuSdc8WA74G6wHtm9rRz7n9mdob/dQfsPPz4qP2U5EUiXGF/kGza\nBFWqQKlSud/HzHhh9gsM+2EYE26dQOOzGhdcgIUg0CSf7xkrM8sCLnHOnQ4kOueuzva6OedyzOZ9\n+/Y9cj8mJoaYmJj8hiMihST7lNCcOfEFNiW0YoVvvn3SJJg2DZo0yd1++zP203liZzbs2sDCLgup\nRjmYNQuuvvrUO4eJ5ORkkpOT87x/UBdDOef+CaQDXYAYM9vqnDsLmKXpGhFvKYwLaM+bBwMGwIIF\n8Nhj0K0bnHHGqfcDWLNzDbd+fiv1K5zHf7Oup/TnEyApCVq0gIkTISoqaHEWpkJdDOWcO9M5V8F/\nPxqIBZYAk/jzbz4emJCf44hI0TN7NtxxB8TFwbp18OyzuUvwZsbIRUNp9m5j7vw+k1H3T6H0iA/h\n2mt9P+jLLyM2wedFvkbyzrlG+Ipii/lvo8zsFX8J5afA2ZyghFIjeQkXRekEZTAVdAVPVpbvlusy\nyAMH2D15LA/N7cMPmZsZvbEJF19/H7RvD5UqBSWmcKBrvIoEoKiVGgZbMD4g9+/3VceULRv4MZ96\n/F7aOAdjxjD323Hc2S6Tv1Vswqt3jqRM9XMCjiUSBJrkMbOQ3HyHFgmt2NgOBiPMl2bMYITFxnYI\ndVhFwo4dZs8/b1alitmnn+Zun2nTplnZ0lXsap6292ll23G27uLz7aH+za3qgMo2IWXCMdvGxnaw\n2NgONm3atAJ6F4XPnztznWu1HkxECtWmTTB4MCQkwE03QXIyNGhwip3MYNEi9j3wMKkHDrKVJD7m\nNuo3qsHv142j84UxpLT+gjOifZP2hVn5E+6U5KVI69HjAebMiSc93ffY128lIbRBedj69XDppdC5\nM/z4I9SseYodVqyAjz+G0aOheHH2FS/B1fQitVJ7uO5hKJPK/6U04z/9/nPMbsf27oH0dN9zSvIi\nRUwo+60URbVrw4YNcNppJ9lo40b45BNfcv/tN7j9dvjsM2jcmKgvR7Pmgy5wfj/45npKL13Gi+N6\nn+SHiU68ikjQmflGz2XK5HKHnTvh88/ho49g+XLo2BH+/ndfTXuxYuxM38mAOQMY+v1QWp/Rmu0T\n0imZWeqEJ3tzOqH+7LOPMnv290BkV1GpukZEQubobpC33QZ9+pxk4/R0X836Rx/5VqHGxfkK49u2\nPdK34Lf9v/HOond4a9FbdGzQkX+1+hc1ytfIVSxHV+G0anUp/fq95YkqKlXXiBRBoa4k2bfP7M03\nzc45x6xVK7OpU82ysnLY8NAhs1mzzO691+yMM8xatzYbNsxs165jNkv9LdUe+vIhq/ByBesysYv9\n/NvP+YrPS1VUqLpGpGgJdSXJ3r1Qrx40a+YbxTdtmsNGy5fDqFG+UXulSnDnnfDCC1Djz1G5mTF7\nw2zeXPgm32z8hq6XdSXl4RSqlatWKO/Dq5TkRSJcqCtJypWD776Ds87K9kJamq8qZtQo3/077oAp\nU6BRI99USufHALjn0Y5sqLCBD5Z8QHSJaLpe1pVR7UdRtmQuV0flQlGuolKSF5Fcy8zMuc3AkQR/\n4ICvVWRCgq+7WLt2vgn6mJgj/WISExO56Za7OVCzE1w8j6R5d/K3s9vyUYePaFKjCe5UjeHzoChX\nUenEq0iIBKtnTmG0Zpg/H15+2dfLfejQbC+a+RL6yJG+CplLL4W774YOHY7pVbA/Yz9TV03l4fce\nJ63cTtjcAn66DVYcJLbV9KB2r/SyQu8nLyKBC+Y8eiCj1EA+WMx8sysDBvhWqfbs6VvEdMSGDb7E\nPnKkb3gfHw9Llx6zwunX339lyqopTF41mZnrZnJ59cupuKsKacN6w/5H/FsVjWmTkAnkLG0wb6i6\nRvIh1NUk+RWKao9p06ZZdHRV/3FHWHR01RP+7g4dMrviCrOLLjL7+GOzjAz/C3v3miUkmP31r2aV\nKpl162a2cOGRUpo/Mv+wr9d/bf+c+U+79P1LrcLLFeyWz26xhB8SbNvebQHHIcdD1TXidaGuJolU\nJzpBe/g1+HN0X6wYDB/uq5pxGMyZAyNGwLhx0Lw5PPQQ3HADh0oUZ2naUmbMe5UZ62Ywb9M86lWq\nR+s6rRncZjBX1rqSElEljomjKM+Ph0QgnwjBvKGRvOSRF2qe8zOazeu3mJx+b40bN7fSpU8Qx8aN\nZi+9ZHbeeWYXXGD2yiu2b+Mam7Vulr04+0VrM6qNle9f3hq83cAenvywjVsxznbs35GXX4cEAI3k\nRcJfXkez+fkWk72MsFSpN9i8+RkOHGiK7/o+kJV+kMU9/0Xc4MHY4m9Zf/vfmP/K3cwvmcaCLWNY\nMeo5GlVpRIuzW9Dt8m581OEjzixzZt5+CVIoVF0jEacoX+gjv9dVTUxM5Pnnp7B+/U38/vtVVKo0\nhQ0bDnIJ53Fb6feoW30UM/8SxeZrzmehbSaqWBRX1LqCK2peQbOazbjsrMuILhFdYO9PTk3VNeJ5\nXp7TLehLEY4dG8fatXF0ffggrdp+w8wxg/lhzTek1CjGv8pDVJrj2qbXcfdVt/F2jSbUKl+rQOrW\npfBoJC8SJnLzDSUv32LMjE17NrFg8wImfTefNdun8+Oen6m7w2hS4hyqVrqEWYt3U2Z/eXr26OqZ\nD0yvUhdKkQiV26mYU432Dx46yJJflzB301zmbprL/E3zyTqUSbMDZ9LsuzSa/X46l7V7kNPu6uKp\nC1wXFZquEfG4uLi4YxL7voP7mL95PrPXzyZ53dcs2vwdpfbV5Y6rrqBjeh1eXbCb2l8txt18Ffzj\nfrj8csjDFExBTyVJwVCSFwkTuW2idSDzAHM3zmXGuhnM3jCbpVuX0rDSxZTaGsOySX34e9WzeKna\nZ9ToNhxq1YIuXWD4BF8nsTzS2oTIpekakTCSfbQM8Oqg99lbZjcX3nguG0tsZN6meVxY5UKuqXMN\nMbVjWDTuCl7rV4LnLv2C+INDKJey2NfxsUsXaNQoKHHlt6pHgkfTNSIR7PBUzN6De3l17Ku89OlA\nDjUqBZkl+XbCfJ69rSdjuo+hQukKvh3WraPB8n48XWo4URnnwQP3Q6fxEJ2/MsecPmwkMinJi4SJ\nbfu2MWHlBMaljGPuprmU3BbNoa03wTd9YUc9DpHA/PRJPN+pLIwfD++/D4sXU/2uu2DWDGjQIChx\n5DQ18+yzjzJnTq8i2Y890inJi4TQ5j2bGZcyjrEpY1m6dSltz2vLvY3v5dObP6XTDZ1JWhAH1AOg\nFn/hrlUroXZtOPdcePBBmDABSpcOakw59biZPXuSZ9cmeJ2SvEiQnaoKZWf6Tj5f8TkfL/uYZduW\ncUO9G3jqiqeIrRtL6eJ/JuzHH3+Qr2eNp3VmKboygitJ5vdGbaF/P2jYsFDfExxf1SORQUleJIhO\nVIUS0zqGL1K/4MMfP2TW+lnE1Y2je7PutD2vLaWKlzru50wdkcby7otZ4yazs8wXJJ5TldL9xxB7\n440F/h6K8qXyvChf1TXOuVrASKAKYMAQM3vTOVcRGAOcA6wHbjGzXdn2VXWNeM5xVShV+nH2TSNI\nP283F1a5kLsvvpsODTpQvlT543c2g6+/hvfeI3NKIjuv7kSVf3WFyy4r1PcAqokPZ4W64tU5Vw2o\nZmY/OOfKAd8BNwGdgd/MbKBzrhdwhpn1zravkrx4Tps2HUlKjoMLo+CyIVB+FXV2VyFp4GTqVqyb\n8067d/sudv3ee75E/9BDvsvnnX564QYvEaFQSyjNbCuw1X9/r3MuBagBtANa+TdLAJKB3jn9DBGv\nWLNzDeU7OWjcFTZdDLOvpvSW9bw37o0cE/ya8T+yuc87tPj1U4rFtYF334WWLfO0GlXkRIoF6wc5\n52oDjYGFQFUzS/O/lAZUDdZxRMKJmTF9zXSu+/g6mn3QjLp16jLiyuHE/nYusbU3MGHcyGOnOg4e\nJPX50ayoeBXRna6F6jXYt2gFjBkDrVopwUvQBeXEq3+qZizwuJn9fnRrUjMz51yO8zJ9+/Y9cj8m\nJoaYmJhghCNS4DIOZfDJT5/w6vxXybIsujfrzuc3f36k13r8jfHH7rBlC7889z6lRg7lt6gG7Lmz\nO3UGtaNV+RI5/HSRPyUnJ5OcnJzn/fPd1sA5VwL4EphqZq/7n1sJxJjZVufcWcAsM6ufbT/NyUvE\n+f2P3xny3RBeX/g69SrVo+eVPYmrG5dzz/XDJ1LfeQe++oodcX9nwaXdiOt+AcVV1yZ5FOicfL6m\na5zvX/YHwIrDCd5vEn82uYgHJuTnOBLeEhMTadOmI23adCQxMTHU4QRF9ve068AuXpz9Iue+eS6L\nf13MxNsmMuPuGbQ9r+3xCX7fPhgyBC6+GLp29U3DrF9PpdFvc11PJXgpZIFcEDb7DbgKyAJ+AJb4\nb22BisBXQCowHaiQw755uoithJf8XJA6XB3znqLftuJtylr5l8pb/Ph4+/m3n0+845o1tv+hJ21f\nmUq295obzZKSzLKyCi9wKRII8ELe+Ury+bkpyXtDbGwHf4I3/22ExcZ2yPPPmzZtmsXGdrDY2A5H\nPixyeq4gxcZ2MEq9a1z9T+PpisYNLa359XE5b5yVZTZ9uu1vfb39Hn2mvVGqp/W8eZ1t2FDgYXpO\nYf89R6pAk7y+OErYOFFjrH793iq0Pub7M/azvkYqPJYEqR1hyHewazZlYicdu+HevTBqFBmD3+LX\nHSUY9MejlLl/DA/3LEPNmgUSmqepX30BCuQTIZg3NJL3hGBO1+T0raBixbpB/aZwIgczD9o7i96x\n6oOq21VvXmWlalbK+T2tXWv25JNmFSuadehgv4xOthdfyLIdO4IeUpES7G+EXoZG8lKY4uLiIro7\noZkx8eeJPJ30NLUr1GbSbZO4rPplJNY7aln/kyOIK1UK2reHb76Bzp3hu++gdm3OAv4R2rcgcnKB\nfCIE84ZG8pJNTt8KXnrppQI7sfvtlm+t5fCW1vCdhjZ11dTjN0hPNxs2zLIuuth2Va9vm559z2zv\n3qAcW47lxRP4BYUAR/K6/J+ElZwaYwW7WdaWPVvoPaM3M9bO4PmY5+ncuDPFix31pXbrVnjvPew/\n77OxUmP+ueMJNjeI5ZVBxULRK6zIUFO03CnUBmX5oSQvhe2PzD94bcFrvDrvVR687EF6X9Wb00qd\n9ucGP/wAr7+OTZjId/Vu44k1j1G5ZQN69YJmzUIXt8jRdI1XkRx8mfolT0x7goZVGrKwy8I/G4Zl\nZcHkyTB4MKxaBY88wu7vBzN0QEWGJgTtinoiIaORvHja2v+t5bGpj7Fq5yreaPsGbc9r63th3z4Y\nMQLeeAPKl4cnn4Sbb4YS6iUj4U0jeSmyjp7TfbT7PfxY9kdeW/AaT135FONuHUfJqJLwyy/w1lsw\ndCg7GrYgrccHXPDAVer+KJ6lJC+ecMximtopfDW1PU3qXs7iBxZTu0JtWLoUBg/GvviCDc3voHed\nhSzaVJfXqsEFyu/iYUry4gmDBg0h3f0DbpoJdWZhU7txWsomatdLgUFdsJQUfrjqUR4563X2bjyD\nXr3gw1tQszDxPM3JS8QzMy66oyk/1fgZlt1LiVn/4I6DvXiu7GdUqlKJ98pUZGaV2uzLeJtnn61G\nXJxmZyRyaU5eipQNuzbQdXJX9jbeTqX3inH/us08ynksL7afeTf/nS6fTCH9wHOwHKKjL8G5BJxT\n/bUUHUG7/J9IYcqyLN5c+CaXDbmMFqddSOrm6/h1eyatqizlqYYdyJoyiRFb9pB+YCC+Sxr45usP\nn5gVKSqU5CXipO5IpeXwlny2aDhzlzelT+dh7P69DD3bruTvGan8330fhN1qSS9eWEUig5K8nFC4\nJaZDWYd4de4rXPmf/+OW5O3MHrCNUqddzW1N19Fw8kAqX1KDNWuge3ff9j16PEB0dC8gAUggOroX\nPXo8ENSYcvM7Olz5k5TUjqSkdrRvHx8Wv08pIgJpdBPMG2pQFtbCrWHUil9/tKYv17WYh8va6iZ/\nMUtIsKwDf9h115m9847Z/v0571eQF6LI7e9IbXQlmFCrYQmGQYOG+C/g4LtUb3q677nCngY5tPd3\nXn/3bvr/bxIvbDiXrnd8QrG3rgPncMCXX558/7i4uAKLOVx+RyInoyQv4WnHDla//Tz3/PIexU47\nnY/P/4w2/TuEOqo86dHjAebMiSc93ffYN22UENqgpMjQnLzkqDDms3O0cSNZTzzO2x1r0ezQEKqd\n3pOVo7bx+bfhl+Bz+zs6fGGV2NhJxMZO0mXtpFBpMZScUKH2905JgYED2TRzPHfdcTqrqcTvn4ym\n09Xn07Mn1K9fcIfOD/VAl8KmfvISFnKd/BYtgv79sXlzGfVIS3qUTKbKuidpU/ZpenQvrotii2Sj\nJC8hd0yzMHxz0MdMUZjBjBnQvz+sXs32Hl158MwFrNq9lpE3jeSSao3VdkDkBAJN8pqT97hQ1Lof\nW3Vy1ErTrCwYPx6aNsUee4xfrrmbSVNe46KMNzmv8vl8e/+3ND5LCV4kmFRd42HZR9Rz5sSH5KRf\ncTJp/ctGuPBCssqU5Zur+vDw7L+ybe2TlPtqFp92+pQW57QotHg0jy5FSiBF9cG8ocVQBS5Ui3AO\nLxIqxRB7kLttrYuybRdebOO7Tbdzzs6yS278xqr1r2P3TrzP9hzYU+Dx5BRbuCzyEgkUWgwloRbX\nvDnf39mOygmPsap8Bbb1eYX+Kd1J2/oHLV98hqTtCbx//fu0O79docemBUxS1CjJe1ihL8LZtQve\nfhvefJP6MTGwcD6VLrkEgJK//Mg9k+6iTsk6LO26lCplqxRcHCLyp0CG/TndgGFAGrDsqOcqAklA\nKjAdqJDDfgX9rUasYHu3HLF9u1mfPmYVK9q+TnebpaQceSnzUKYNmDPAzhx4pg1fMtyysrIKJoZc\n0nSNRDoCnK7Jdwmlc64FsBcYaWaN/M8NBH4zs4HOuV7AGWbWO9t+lt9jS4j9+isMGgTDhpHW8mb+\nsbc3k1fUITUVypWDtf9bS/yEeKJcFCNuGuG71moY0IlXiWQhqZN3ztUGvjgqya8EWplZmnOuGpBs\nZvWz7aMkH6k2bYKBA7GPPmL9VXfxVFpPlmyvyVNPQefOULq0MfT7oTw781l6N+9N9yu6U8ypWlck\nGMLl8n9VzSzNfz8NqFpAx5HCtH69bwHTZ5/Bfffx5oMrGD61Gr17w5hOvotib9mzhS7jurBt3zaS\n45NpWKVhqKMWKdIK/MSrmZlzLsche9++fY/cj4mJISYmpqDDkbxYvdqX3CdMgK5dITUVzjyTB9Lh\nsX/7LoptZnz048d0T+zOw//3MH1a9KFEVIlQRy4S8ZKTk0lOTs7z/gU5XRNjZludc2cBszRdE4FS\nU6FfP7K+nEyxRx+Bxx6DihWP2yxtbxrdpnRj5W8rGXnTSC6rflkIghUpGsKlrcEkDhci+/6cUEDH\nCQvhdpm8fFu5Eu68k0NXNGfqqvOok7WGdfF9j0vwZsYnP33CRf+5iPPOOI/vHvhOCV4k3ARSipPT\nDRgN/AIcBDYBnfGVUH5FESih9FRJ3ooVZrffbhkVK9tnjfvZ2RV2W48eZps2Hb/p1t+3WocxHazB\n2w1s4eaFhR+rRIxCKeMtQgiwhFJtDfLJE9fvXL7c7LbbzKpUsaW397dzK++xl14y27nz+E2zsrLs\nox8/siqvVLFnvnrG0jPSCz9eiRieGgSFiUCTvFa8FmUrVsCLL8LMmfDkkzB0KHVdOX4qBtHRx2++\nYdcGbk64mZQtP3PB6ktp1agVpYuXLvy4JWKojUToqXg5n0J2mbz8SEkh69bbsauvhksugTVroFcv\nKFeOsmWPT/CHsg7xxoI3uOidi1gyMYW9gwazaPw9tG8f741zECJeFsiwP5g3PDJdYxZBc44pKZZx\n8+22r1xle7lCf/t6yu9HXjrRe1i6dak1GdrEWg5vaVe2i438qSkpVJquCT40Jy/HSU21A53usL1l\nzrSXyvSz26/fYwsW/PlyTv8Rx3451p6Y+oRVHljZ3l/8vh3KOuSN8w9S6CJmEBQhlOTlT2vWmN1z\njx08vZL1K/2Cdbtz99G9w444NnlnGRd2tVLPlLZ7J9xr2/ZuO7KdRmUioRdokteJ1wIS0iZYGzbA\nSy/BuHHwyCNY6mruPljh1BfFrrwcrn0Uoldz0c9N+eDfHxzzclxcHOPHJxz1vgr/KlMiEqBAPhGC\neSNCR/K5+eoZshHv5s2W1fUhs4oVfa1/d+zI1W6ffPGJRd0YbfQ8zWhyh5UuU0UjdJEwRWG3Gs6r\nSGxrkP2aqdHRvXK8ZmqbNh1JSmrHn4t+E4iNncT06WMLJC77dSvru75MpSkj2XjNfVw48mmoXPmU\n+6VnpPP6gtcZNH8QLU9vyf8mZlAis6Ta74qEsXBpa+BJx9b8+pL94amLUMhM28HyG3qxp9YFzJtr\nvPngZzyRsZ42d3Q9aWljZlYmCT8k0OCdBiz+dTELuixg3IPjmDXlC6ZPH6sEfwKea18hRYLm5AtA\ngV92b/dudj83mKy33mZN1ZvZNvxHKlVezv0d7jjyLWPOnPjjvmVkZmUyetloXvz6RaqfVp1R7UfR\n4pwWwYvLw7J/i8vp9ysSlgKZ2wnmjQickz8y1+6GGeUHn3SuvUDKxvbuNevf36xyZcuKj7cfJ6w5\n8tLJyhszD2Xah0s/tHpv1bMWw1rYzLUzgxNPACK9jE7loxIuUHVNwTlcXfL8e6+w+MJvuLF2B5pf\n3fyE2wZtlPfHH/D++76e7i1bwtdf4+rXp9EpdsuIymDw/MG8tegtapavyTvXvkPrOq1xLtfTeUGh\nUbBICAXyiRDMGxE4kj/apt2b7M5xd1r1QdVtxJIRdijrUNCPsXxphn1w5X9t9+m1zK6/3mzJkhNu\ne0xFT8UBFnV9tJV7sZzd/vntIe8S6YVRsNYISLhAI/nCUbN8TUa1H8WCzQt4YtoTvP3t27zc+mX+\nWuev+R6qFLoqAAAJ6ElEQVQpz5+bxdzHP6X9D//ir2fXgDFjIO6Kk+7TrFUzur7/d0Z835O9ZXfT\n4Zz2vHrrq9Qsf6rieMkNrRGQSKUSyiDIsixGLxtN/zn9OXjoIPc1vo97LrmHquUCu7Tt/n1G36ZT\nuCf1WSpVL0mFd/5NqWtb+66vl4N9B/cxY90MPvzxQxLXJNK6TmvuvOhOrv3LtWHVHTK3pacicmqB\nllAqyQeRmbFg8wKGfj+UcSnjaH1ua25teCvNazWnRvkaJ9959mzo04e9W3YRPagfUR1uPC65H8o6\nxPe/fk/S2iSS1iax+JfFXF79cm6/8HY6XdCJitHHX5ovXIR0BbCIhyjJh4k9f+xh9LLRTF41mXmb\n5lGuZDman92c5rWa0+DMBlQuW5nKZSrz8/BPiH7+39Tcv5ftD3ejfr8X2Hsond0HdrN652qWb1/O\niu0rWLF9BT9t+4ma5WsSe24sFXdVZNaIxRTPKq6kKVKEKMmHITPj5x0/M3fjXGaunsu8lLWkZ24h\nM2sDu6MyKP5HKQ5QHErtI6p4FOVLlee0UqdR94y6XFD5AhpWbuj7s0pDzixzpmemPzS6FwlcoEle\n1TWFZONGsyeeMGtw+habef6DllGhkg2re4GVce8aZbYZ0b8ZUUNzVXWiahWRoosAq2vU1qCAmUGX\nLtCy0f+4YW5vlrlGXH3DaRRf/TOjz63PfisD+ytDeiU4VCLU4RaacGsRIeJVKqEsYO5AOk9kvMWQ\n4q9Q7OKbYNxSDvf8zWv7gwJvmyAinqE5+YKSmQkjRsDzz0OTJtCvH9Svf9xmeZ2XjvT5bK+cVxAp\nbDrxGgIZGb71Sj/9BC/3N5g4EZ55BqpWhQEDoGnTUIcYliL9g0okFJTkC9H+/fDBBzBoENSuDQNu\n+IamY3vBvn3w8svQtu0JFzKJiOSF+skXkkGDoE4dmDULJr28guTy7Wj61l3w0EOwZAn87W9K8CIS\nckryeVSrFsz9dAvjKnbhosdiICYGVq6Eu+6CYvq1ikh4UHVNXuzezS0/DPC1/73/fkhNhQoVQh2V\niMhxNOTMi127YNs2+OEH39x7ISR4XXpORPKiwE68OufaAq8DUcB/zWxAttcj/sRrQcledQKo3FBE\ngDCprnHORQE/A9cAW4BvgdvNLOWobZTkc5BT/Xj9+vVZsqQzvtWhAAnExk5i+vSxIYtTREIj0CRf\nUHPyTYDVZrbeH9QnwI1Aysl2kuzL/SE9HTZseDG0QYlIxCqoOfkawKajHm/2P1ek5XVe/ZxzqhEd\n3QtIABL8bQweKLA4RcQ7Cmokr3mYbHJ7Meuc+tL07+/rS6NLz4lIoAoqyW8Bah31uBa+0fwx+vbt\ne+R+TEwMMTExBRRO6OU0DTNo0JDjkvXJriWqxC5S9CQnJ5OcnJzn/QvqxGtxfCdeWwO/AIso4ide\n27TpSFJSO3TyVETyIyxOvJpZpnPuESARXwnlB0cn+KJI7YFFJBTUoKwQqeuiiORXWNTJ5+rARTDJ\ni4jkl7pQiojIEUryIiIepiRfBKnZmUjRoTn5IkbXVhWJbDrxKielen2RyKYTryIicoSuDFXEaFGW\nSNGi6ZoiSIuyRCKX5uRFRDxMc/IiInKEkryIiIcpyYuIeJiSvIiIhynJi4h4mJK8iIiHKcmLiHiY\nkryIiIcpyYuIeJiSvIiIhynJi4h4mJK8iIiHKcmLiHiYkryIiIcpyYuIeJiSvIiIhynJi4h4mJK8\niIiH5TnJO+duds4td84dcs5dmu21Z5xzq5xzK51zbfIfpoiI5EV+RvLLgPbA10c/6Zy7ALgVuABo\nC7zrnPPcN4bk5ORQh5Avij+0FH/oRHLseZHn5GtmK80sNYeXbgRGm1mGma0HVgNN8nqccBXp/1AU\nf2gp/tCJ5NjzoiBG2NWBzUc93gzUKIDjiIjIKRQ/2YvOuSSgWg4v9TGzLwI4jgUUlYiIBIUzy1/+\ndc7NAnqY2ff+x70BzOxl/+NpwHNmtjDbfkr8IiJ5YGYut9uedCQfgKMPOAn42Dk3GN80zV+ARdl3\nCCRIERHJm/yUULZ3zm0CmgGTnXNTAcxsBfApsAKYCnSz/H5dEBGRPMn3dI2IiISvkNavO+decc6l\nOOeWOufGOedOD2U8ueWca+tf6LXKOdcr1PEEwjlXyzk3y7+Q7Sfn3GOhjilQzrko59wS51wgJ//D\ngnOugnPuc/+/+xXOuWahjikQ/oWOy51zy5xzHzvnSoU6ppNxzg1zzqU555Yd9VxF51yScy7VOTfd\nOVchlDGezAniDyhvhnqR0nSgoZldDKQCz4Q4nlNyzkUBb+Nb6HUBcLtzrkFoowpIBtDdzBrim2p7\nOMLiB3gc33RgJH4NfQOYYmYNgIuAlBDHk2vOudrA/cClZtYIiAJuC2VMuTAc3//Vo/UGksysHjDD\n/zhc5RR/QHkzpEnezJLMLMv/cCFQM5Tx5FITYLWZrTezDOATfAvAIoKZbTWzH/z39+JLMtVDG1Xu\nOedqAtcC/+XYE/5hzz/iamFmwwDMLNPMdoc4rEDswTdIKOOcKw6UAbaENqSTM7NvgP9le7odkOC/\nnwDcVKhBBSCn+APNm6EeyR/tXmBKqIPIhRrApqMeR+xiL//IrDG+fyiR4jWgJ5B1qg3DUB1gu3Nu\nuHPue+fcUOdcmVAHlVtmthMYBGwEfgF2mdlXoY0qT6qaWZr/fhpQNZTB5NMp82aBJ3n/3NeyHG43\nHLXNs8BBM/u4oOMJgkicIjiOc64c8DnwuH9EH/acc9cD28xsCRE2ivcrDlwKvGtmlwL7CO+pgmM4\n5+oCTwC18X37K+ecuyOkQeWTv/IvIv9P5zZvBqtO/oTMLPZkrzvn7sH39bt1QccSJFuAWkc9rsWx\nbRzCnnOuBDAW+NDMJoQ6ngBcCbRzzl0LlAbKO+dGmtndIY4rtzYDm83sW//jz4mgJA9cDswzsx0A\nzrlx+P5OPgppVIFLc85VM7OtzrmzgG2hDihQgeTNUFfXtMX31ftGMzsQylgCsBj4i3OutnOuJL6O\nm5NCHFOuOecc8AGwwsxeD3U8gTCzPmZWy8zq4DvhNzOCEjxmthXY5Jyr53/qGmB5CEMK1EqgmXMu\n2v/v6Bp8J8AjzSQg3n8/HoikgU7AeTOkdfLOuVVASWCn/6n5ZtYtZAHlknPub8Dr+KoLPjCz/iEO\nKdecc1fhaw/9I39+TX3GzKaFLqrAOeda4Wun0S7UsQTCOXcxvpPGJYE1QOdIOvnqnHsaX2LMAr4H\nuvgLEMKSc2400Ao4E9/8+7+AifgWbJ4NrAduMbNdoYrxZHKI/zl81TS5zptaDCUi4mHhVF0jIiJB\npiQvIuJhSvIiIh6mJC8i4mFK8iIiHqYkLyLiYUryIiIepiQvIuJh/w+AEzNip/PbsQAAAABJRU5E\nrkJggg==\n",
"text/plain": [
""
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"plt.plot(xs, f(xs), '--')\n",
"plt.scatter(X, y)\n",
"plt.plot(X1, h2.predict(X1), 'r');\n",
"plt.plot(X1, h4.predict(X1), 'g');"
]
},
{
"cell_type": "code",
"execution_count": 20,
"metadata": {
"collapsed": false
},
"outputs": [
{
"data": {
"text/plain": [
"885.87751643487434"
]
},
"execution_count": 20,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"error(h2, X, y)"
]
},
{
"cell_type": "code",
"execution_count": 21,
"metadata": {
"collapsed": false
},
"outputs": [
{
"data": {
"text/plain": [
"845.62372708741623"
]
},
"execution_count": 21,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"error(h4, X, y)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"\n",
"* Možemo očekivati da vrijedi:\n",
"$$\n",
"E(h_4|\\mathcal{D}) \\leq E(h_2|\\mathcal{D}) \\leq E(h_1|\\mathcal{D})\n",
"$$\n",
"Q: Zašto?\n",
"\n",
"\n",
"* Q: Koji model odabrati u ovom slučaju?\n",
"\n",
"\n",
"* Q: Koji model općenito odabrati za neke podatke $\\mathcal{D}$?"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Problem šuma\n",
"\n",
"* **Šum** je neželjena anomalija u podacima\n",
"\n",
"\n",
"* Mogući uzroci:\n",
" * Nepreciznost pri mjerenju značajki\n",
" * Pogreške u označavanju (engl. *teacher noise*)\n",
" * Postojanje skrivenih značajki (latentnih varijabli)\n",
" * Nejasne granice klasa (subjektivnost)\n",
"\n",
"\n",
"* Zbog šuma je granica između pozitivnih i negativnih primjera složenija nego što bi idealno bila!\n",
"\n",
"\n",
"* [Primjer 1: binarna klasifikacija po značajkama dobi i prihoda]\n",
"\n",
"\n",
"* Jednostavni modeli ne mogu doseći $E(h|\\mathcal{D})=0$\n",
"\n",
"\n",
"* S druge strane, složeni modeli uče šum, a ne pravu klasifikaciju!\n",
"\n",
"\n",
"* [Primjer 2]\n",
"\n",
"\n",
"* Šum u načelu nije moguće odvojiti od pravih podataka\n",
" * Moguće je samo za stršeće vrijednosti (engl. *outliers*)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Odabir modela\n",
"\n",
"* Moramo odabrati model $\\mathcal{H}$ (učenje bez pristranosti je uzaludno)!\n",
"\n",
"\n",
"* Često radimo odabir modela unutar neke familije modela (npr. kod regresije: odabir stupnja polinoma)\n",
"\n",
"\n",
"* Stupanj polinoma je **hiperparametar** modela ($w_i$ su parametri)\n",
"\n",
"\n",
"* ** Odabir modela = optimizacija modela, optimizacija hiperparametara **\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Primjer: regresija"
]
},
{
"cell_type": "code",
"execution_count": 22,
"metadata": {
"collapsed": false
},
"outputs": [
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAYAAAAEACAYAAAC6d6FnAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3XmUVNW1x/HvZhSc0IiCgAYTjOI8MAgKjYpBo6IRAaPG\nAecBNRqDQ0JHfZGYKHEIifOAwgsiIkQEASnUmICzAiKgkCeoCJqAMRgber8/TjW0nUbo6qo6t6p+\nn7V6WXWrb9W2tO++Z9rH3B0RESk9DWIHICIicSgBiIiUKCUAEZESpQQgIlKilABEREqUEoCISInK\nWQIws3ZmNsPM5prZHDMbnD6+vZlNNbMFZvasmbXIVQwiIrJxlqt1AGbWCmjl7m+Y2VbAq8AJwFnA\nSne/xcx+Bmzn7kNyEoSIiGxUzloA7v6xu7+Rfvwv4B2gDXA88HD61x4mJAUREcmzvIwBmNm3gQOA\nWcBO7r48/dJyYKd8xCAiIl+X8wSQ7v55ArjM3T+v/pqH/ifVohARiaBRLt/czBoTLv4j3X18+vBy\nM2vl7h+bWWvgk1rOU1IQEcmAu9vm/m4uZwEZcD8wz91/V+2lCcAZ6cdnAONrngvg7vpxZ+jQodFj\nSMqPvgt9F/ouNv7z4Yd1v2/OZQugO3Aa8JaZvZ4+dg0wDBhjZoOAJUD/HMYgIlL03OGAA+p+Xs4S\ngLu/yMZbGEfm6nNFRErNwoXQpEndz9NK4IQrKyuLHUJi6LvYQN/FBvouYOZM6Nmz7uflbCFYfZiZ\nJzEuEZEkOu20kADOO8/wJAwCi4hI7rln3gJQAhARKWBLlsDatdChQ93PVQIQESlgVXf/ttkdPxso\nAYiIFLCZM6FHj8zOVQIQESlgzz+fWf8/KAGIiBSspUth9Wro2DGz85UAREQK1MyZcNhhmfX/gxKA\niEjBqk/3DygBiIgUrEzn/1dRAhARKUAffwzLl8M++2T+HkoAIiIF6Pnn4dBDoWHDzN9DCUBEpADV\nt/8flABERApSffv/QdVARUQKzsqVsNtu8Nln0Kjari5mqgYqIlLUXnwRunX7+sU/E0oAIiIFJhvd\nP6AEICJScAo6AZhZHzObb2YLzexnMWIQESlEq1aFPYAPPrj+75X3BGBmDYG7gD5AR+AUM9sz33GI\niBSiF1+Ezp0z2wS+phgtgM7AIndf4u4VwP8CfSPEISJScLLV/QNxEkAb4INqz5emj4mIyCbUZwOY\nmmIkAE3wFxHJwL/+BXPnQpcu2Xm/es4izcgyoF215+0IrYCvKS8vX/+4rKyMsrKyXMclIpJoL70E\nBx4IzZqF56lUilQqlfH75X0lsJk1At4FjgA+BGYDp7j7O9V+x2fN+u+4zGDXXaFly8w3QBARKVTX\nXQcNGsCNN9b+el1XAue9BeDua83sEmAK0BC4v/rFv8oll/z3uevWweLF4eK/xx6w554bfvbeG3bZ\nJefhi4hEk0rBL3+ZvfcruFpA7rBiBbzzztd/3nwTtt8e+vYNP126hEwpIlIMVq+GNm3gk082dAHV\nlPgWQH2ZwY47hp/qU6EqK+GVV+Cpp+Dcc0OxpOOOC8ngiCM2/oWJiBSCVAq6ds3utazgWgCba9Gi\nkAyeeiqMmg8aBIMHQ9u2WQpSRCSPBg8OLYCffUPtBFUDTfvud+HKK8OmCa+8AhUVsO++cOqp8Npr\nsaMTEambadPgyCOz+55FmwCqa98ehg+H99+H/fcP3UK9esHEiaHrSEQkyZYtC/v/7r9/dt+3JBJA\nlRYt4Kc/DYngvPPCaPp++4XMKiKSVNOmweGH12//39qUVAKo0rgxnHIKvPwy3HBDSAYnnBDGDURE\nkmbaNOjdO/vvW5IJoIoZnHgizJsXRte7dAkDLKtXx45MRCRwz03/P5R4AqiyxRYwZAjMmRPm2H7v\ne3D//WHhmYhITPPmhamfu+2W/fdWAqimdWt48EGYMCEkgB494L33YkclIqVs6tTc3P2DEkCtOnUK\nmy706xe6hu69NzTDRETyLVf9/1DEC8GyZe5cOO20sIDsvvtgp51iRyQipaKiAnbYIfRE7LDDpn9f\nC8GybK+9YNYs2GefMAd3/PjYEYlIqZg1Kyxq3ZyLfyaUADZDkybwq1/B2LFhdfHZZ8Pnn8eOSkSK\nXS77/0EJoE66d4c33giPu3aFhQvjxiMixS2X/f+gBFBnW28NDzwQCjN17w6TJsWOSESK0erV8NZb\n4TqTK0oAGTr/fHjyyVB6+le/0iwhEcmumTPD4tRclrJXAqiH7t1h9uywbuDkk8OGzSIi2ZDr/n9Q\nAqi3Nm1Cpt522zAuoHpCIpINue7/ByWArGjaNKwRuOii0CpIpWJHJCKFLFfln2vKSQIws9+Y2Ttm\n9qaZjTOzbau9do2ZLTSz+WZ2VC4+PwazkABGj4b+/eHxx2NHJCKFavr03JR/rilXLYBngb3cfT9g\nAXANgJl1BAYAHYE+wAgzK6pWyOGHw7PPwuWXw113xY5GRApRPvr/IUcJwN2nunvVXluzgKqdePsC\no929wt2XAIuAzrmIIab99w+1hO68E667TjOERGTzVZV/znX/P+RnDOBsoGq2/M7A0mqvLQXa5CGG\nvGvfPiSBadPCyuGKitgRiUghyGX555oaZXqimU0FWtXy0rXuPjH9O9cBX7n7qG94q1rvj8vLy9c/\nLisro6ysLNNQo2nZEp57LowJ9O0bxgW23DJ2VCKSZHXp/kmlUqTqMeskZ9VAzexM4FzgCHf/Mn1s\nCIC7D0s/nwwMdfdZNc5NTDXQbKioCNtOzpsHkyfDdtvFjkhEkqpPn7DA9KST6n5uXauB5iQBmFkf\n4Fagp7uvrHa8IzCK0O/fBpgGfLfm1b7YEgCEfr2rroIZM0KG/9a3YkckIknzxRfQqlWYBrrNNnU/\nPynloO8EtgKmmtnrZjYCwN3nAWOAecAzwEVFd6XfCDP47W/DwM4RR8DKlZs+R0RKy/TpYUOqTC7+\nmdCGMHnmDtdfDxMnhgHiHXeMHZGIJMUFF0CHDqHsfCbq2gLIeBBYMmMGN90UFnj06hUGibXLmIi4\nw9NPh3VE+aIEEIEZ3HADNGoEZWUhCbRuHTsqEYlpzhxo3Bj22CN/n6kEENEvfhFaAlVJoE1RrogQ\nkc3x9NNwzDHhBjFfiqoMQyG67jo466zQHbR8eexoRCSWSZPgBz/I72dqEDghbrgBnngiVBLVOgGR\n0vKPf8Cuu4abwPpsAJOUaaBSRz//eVj9d8wx2lhGpNRMmQI9euR296/aKAEkRNU6gb33DmUjvvwy\ndkQiki8xun9AXUCJs24dnHoqrFkDY8eGWQEiUrzWrQurf195JXQD1Ye6gApcw4bwyCOwdm0YHK6s\n3PQ5IlK4XnklrAWq78U/E0oACdSkSbj7X7oULr5Y+wmIFLOq6Z8xKAEkVLNmMGECvPoqXHtt7GhE\nJFdi9f+DEkCibbMNPPMMPPkk/P73saMRkWz76CN47z3o1i3O52slcMJ961shCRx6KOy8M5x4YuyI\nRCRbJk8OFYJjTfZQAigA7duH7qCjjw6DRbHuFkQku55+Go47Lt7naxpoAZk8Gc48M6wWzmfBKBHJ\nvq++CuXg3303exWBNQ20iPXpA8OGhZbARx/FjkZE6uMvf4Hdd49bDl5dQAXmzDPhgw/CrIGZM2Hr\nrWNHJCKZiDn9s4paAAXo+uvh4IOhX7+w4byIFJ6Y0z+rKAEUIDMYMSIsGDv/fC0UEyk0ixfDp5/C\nQQfFjSOnCcDMrjSzSjPbvtqxa8xsoZnNN7Ojcvn5xaxRIxg9Gl5/HX7zm9jRiEhdTJwYxvIaRL4F\nz9kYgJm1A3oDf692rCMwAOgItAGmmdnu7q6KNxnYaqvwP1LXrmEjaa0RECkMTz4JV1wRO4rctgBu\nA66ucawvMNrdK9x9CbAI6JzDGIpe27Ywfjycdx689lrsaERkU1asCH+rvXvHjiRHCcDM+gJL3f2t\nGi/tDCyt9nwpoSUg9XDwwfDHP4Z9BJYtix2NiHyTCRPg+9/P/+Yvtcm4C8jMpgKtannpOuAaoHr/\n/jctTKh1CLO8vHz947KyMsrKyuocYyk56SRYsACOPx6efx623DJ2RCJSm3Hj4PTTs/NeqVSKVCqV\n8flZXwlsZnsD04F/pw+1BZYBXYCzANx9WPp3JwND3X1WjffQSuAMuIc9BFatCvsLxx5gEpGvW7UK\n2rULpd632Sb77x99JbC7z3H3ndy9vbu3J3TzHOjuy4EJwEAza2Jm7YEOwOxsx1CqzODuu8P0MpWQ\nFkmeSZPC3r+5uPhnIh8rgdffyrv7PDMbA8wD1gIX6VY/u5o2DU3Mrl1hzz3hjDNiRyQiVcaNgx/+\nMHYUG6gYXJGaNw/KysI00S5dYkcjImvWhL1/33sPdtghN58RvQtIkqFjR7j//jA4/OGHsaMRkWef\nDSt/c3Xxz4QSQBE77ji48MKwQOzLL2NHI1Laktb9A+oCKnruMGAANG8ODz4YBopFJL8qKkL3z5tv\nhsWbuaIuIPkas3Dhf+MNuP322NGIlKZUKpRryeXFPxPaD6AEbLllKBdxyCGw995w5JGxIxIpLePG\nJbNWl7qASsjMmdC/f9iJ6LvfjR2NSGlYtw7atAkr9HffPbefpS4g2aiePWHo0FAz6PPPY0cjUhr+\n9jdo2TL3F/9MKAGUmAsvhG7dQskINbJEci+Js3+qKAGUGDO4666wr/Cvfx07GpHi5p7sBKBB4BLU\ntGkoFte5Mxx4IBylfdlEcuLNN0NRxn33jR1J7dQCKFFt24YtJX/847A/qYhkX9Xdf1LX3ygBlLCe\nPWHIkPA/6L//venfF5G6SXL3D2gaaMlzh9NOg4YN4eGHk3unIlJo3n0XevUKtf/ztTeHpoFKnZjB\nvffCW2+FwWERyY5Ro8K6myRvzKQWgADw/vthpfDYsXDYYbGjESls7mHe/6hR0KlT/j5XLQDJyG67\nhS6ggQNVPlqkvl5+Ofzz4IPjxrEpSgCyXp8+YaFY//6heqGIZOaxx+DUU5M/pqYuIPmayko4/vhQ\nuXD48NjRiBSetWvDNOt81P6pSV1AUi8NGsDIkfDUUzBmTOxoRArPc89Bu3bJrP1TU84SgJldambv\nmNkcM/t1tePXmNlCM5tvZlqDmkDbbRdWCl98cdhbWEQ2X1X3TyHISReQmfUCrgWOcfcKM2vp7ivM\nrCMwCugEtAGmAbu7e2WN89UFlAAPPgi33AKzZ8PWW8eORiT51qyBnXcON06tW+f/85PSBXQhcLO7\nVwC4+4r08b7AaHevcPclwCKgc45ikHo666wwJfTss1U5VGRzTJwYZv7EuPhnIlcJoAPQw8z+ZmYp\nM6uaDLUzsLTa7y0ltAQkoe64I9QK+t3vYkciknyjRhVO9w/UoxqomU0FWtXy0nXp993O3buaWSdg\nDLDbRt6q1nvL8vLy9Y/LysooKyvLNFSphy22CIvDunQJdzZaJCZSu88+gxkzwnqafEmlUqRSqYzP\nz9UYwDPAMHefmX6+COgKnAPg7sPSxycDQ919Vo3zNQaQMJMnw6BB8Oqr0Kq2tC9S4u65B6ZOhccf\njxdDUsYAxgOHpwPaHWji7iuBCcBAM2tiZu0JXUWzcxSDZFGfPnDOOWGl8Nq1saMRSZ5C6/6B3LUA\nGgMPAPsDXwFXunsq/dq1wNnAWuAyd59Sy/lqASTQunVwzDFhE5mbb44djUhyfPAB7L9/KKPStGm8\nOOraAtBKYKmTFSvgoINC5dDjj48djUgy3HILLFwYKuvGlJQuIClSLVvCn/4UuoPefz92NCLJUIjd\nP6AEIBk45BC4/nro1w++/DJ2NCJxzZ0LK1dCjx6xI6k7JQDJyKWXhoJxgwfHjkQkrlGj4JRTkr3x\ny8YUYMiSBGZw332h4mE+5z2LJEllZeF2/0A9FoKJbL11KBpXVgYHHAD77hs7IpH8SqVgq61gv/1i\nR5IZtQCkXvbaK+wb0K8frFoVOxqR/LrvPjj33ORv/LIxmgYqWXHhhWGK6OOPF+4fg0hdfPopfOc7\nYTbc9tvHjibQNFCJYvhwWLIEbr89diQi+TFyJBx7bHIu/plQC0CyZvFi6NoVnnwSunWLHY1I7rjD\n3nvDiBHQs2fsaDZQC0Ciad8e7r8/1AtasWLTvy9SqP7611ATqxDn/lenBCBZdeyxcNpp8KMfhdpB\nIsXo3nvDavhCH+9SF5Bk3dq10Lt3uDv65S9jRyOSXatWwa67woIFsOOOsaP5OnUBSXSNGsHo0WGK\n3OTJsaMRya7Ro+HII5N38c+EEoDkRKtW4Q/lzDPh//4vdjQi2XPvvWHufzFQApCc6dEDrrwS+veH\nr76KHY1I/b32Wpj/37t37EiyQwlAcuqqq6B165AIRArdvfeGrVELsfBbbTQILDn3z3+GDeVvuCHM\nDhIpRF98Ae3awVtvQdu2saOpnQaBJXFatAhF4y67DObNix2NSGbGjIHu3ZN78c+EEoDkxX77hW3z\nTjoJPv88djQidVdMg79VcpIAzKyzmc02s9fN7GUz61TttWvMbKGZzTezo3Lx+ZJMZ50Fhx4a/ojU\nwyeFZO5c+Pvf4ZhjYkeSXblqAdwC/NzdDwB+kX6OmXUEBgAdgT7ACDNTK6SE3HlnWEBz552xIxHZ\nfPfdF25gGhXZDiq5+tf5CNg2/bgFsCz9uC8w2t0rgCVmtgjoDPwtR3FIwmyxBYwdG4rGdeoU9hcW\nSbIvvoBHH4XZs2NHkn25SgBDgBfN7LeEVkbVn/nOfP1ivxRok6MYJKF22y0UjRswAF59FVq2jB2R\nyMY9/DAcdlgodlhsMk4AZjYVaFXLS9cBg4HB7v6kmZ0MPABsbOlErb3B5eXl6x+XlZVRVlaWaaiS\nQMcdFyoqDhwIU6YUX9NaikNlJfzud+GGJYlSqRSpVCrj83OyDsDMVrv7NunHBvzT3bc1syEA7j4s\n/dpkYKi7z6pxvtYBlIB166BPn7BG4OabY0cj8t/+/GcoL4eXXy6Myp9JWQewyMyqtkk4HFiQfjwB\nGGhmTcysPdABKMKeNdkcDRvCqFHh58knY0cj8t9uuw2uuKIwLv6ZyFXD+zzg92bWFFiTfo67zzOz\nMcA8YC1wkW71S1vLlmEf4WOPDRvM77577IhEgjfeCDPWTj45diS5o1IQkgj33AN33AGzZsGWW8aO\nRiRUst1jDxgyJHYkm6+uXUBKAJII7qHI1pdfwmOPFW+TWwrDRx9Bx47w3nuFtel7UsYAROrEDH7/\ne5g/X4vEJL4RI+CUUwrr4p8JtQAkURYvDovEnngilI0Qybc1a8KWjy++WHhjUmoBSEFr3x4eeiis\nD/joo9jRSCl69FHo0qXwLv6ZUAKQxDn6aDj/fOjXTzuJSX65w/DhYepnKVACkES67rowRfTyy2NH\nIqVkyhRo3Bh69YodSX4oAUgiNWgAjzwCzz2X3GX4UnyGD4ef/KR0ZqFpEFgSbf78sLn8xImhX1Yk\nV+bMCZu9L1kCTZvGjiYzGgSWorLHHqEWe79+8PHHsaORYjZ8OFx0UeFe/DOhFoAUhPJymD49/DRp\nEjsaKTaLF4eihAsWwLe+FTuazGklsBSlyko44QTYZRe4667Y0UixOeccaN0abrwxdiT1owQgRWvV\nKujcOdRmOeus2NFIsXj//fD/1YIFhb/yVwlAito770DPnjBhQlgxLFJfgwZBmzZwww2xI6k/JQAp\nehMnwgUXhMqhbdvGjkYKWdXd/8KFsN12saOpv7omAG3EJwXnuONg7twwJvD889C8eeyIpFDddBNc\nfHFxXPwzoRaAFCR3OP30sK3kqFGls3BHsue998LakmK5+wetA5ASYQb33hv+iLWfsGTippvgkkuK\n5+KfCXUBScFq1gzGjw99uHvtBX37xo5ICsWiRWEsadGi2JHEpRaAFLSdd4Zx48I87rffjh2NFIqb\nboJLL4UWLWJHElfGCcDMTjazuWa2zswOrPHaNWa20Mzmm9lR1Y4fZGZvp1+7vT6Bi1Tp3Bluvz20\nAFasiB2NJN2iRfDnP8Nll8WOJL76tADeBk4Enq9+0Mw6AgOAjkAfYITZ+iG6PwCD3L0D0MHM+tTj\n80XW+9GPYMCAUDPoP/+JHY0k2Y03wuDBuvuHeiQAd5/v7gtqeakvMNrdK9x9CbAI6GJmrYGt3X12\n+vceAU7I9PNFavqf/wl1XM49N8wSEqlp4UKYNEl3/1VyMQawM7C02vOlQJtaji9LHxfJigYNwnZ+\n77wT+nhFarr++nDx33bb2JEkwzfOAjKzqUCrWl661t0n5iakoLy8fP3jsrIyysrKcvlxUiSaNw+z\nO7p2he98J3QNiQC88AL89a/w4IOxI8meVCpFKpXK+Px6LwQzsxnAle7+Wvr5EAB3H5Z+PhkYCvwd\nmOHue6aPnwL0dPcLanlPLQSTepkzBw4/PMwQOvTQ2NFIbOvWQadOcPXVMHBg7GhyJ9ZCsOofOAEY\naGZNzKw90AGY7e4fA6vNrEt6UPh0YHyWPl/ka/beG0aODIPCpT7XW+Chh0LrcMCA2JEkS8YtADM7\nEbgD2AFYBbzu7kenX7sWOBtYC1zm7lPSxw8CHgKaAZPcffBG3lstAMmKu++G226Dl14q7I0+JHOr\nV8P3vhemfh50UOxockvVQEVq+OlPYfZsePbZ0truT4Krr4aVK+GBB2JHkntKACI1VFaGrqDmzeGR\nR8JsISkNCxfCIYeEMaFWtU1nKTIqBidSQ9X00Pffh5/9LHY0kk9XXRVagKVw8c+EisFJSWjePPQB\nH3po2Pv1Jz+JHZHk2tSpYd+IMWNiR5JcSgBSMrbfHqZMge7dYaed4NRTY0ckubJ2LVxxBfz2txr3\n+SZKAFJS2rWDZ54JawRatoSjjtr0OVJ47r47dPuoRPg30yCwlKQXX4QTTwx1YTp1ih2NZNNnn8Ee\ne8D06bDPPrGjyS/NAhLZTBMmwPnnh32FO3SIHY1ky9lnw5Zbwp13xo4k/7QpvMhmOv54+OQT+P73\n4S9/CYPDUtiefhpSKXjrrdiRFAYlAClp55wDH38cxgJmzIAddogdkWTqH/8ILbpHH4WttoodTWFQ\nF5CUPHe45powbXD6dG0UUqh+/OPw3+6OO2JHEo+6gETqyAxuvhnWrIGjjw4lI7beOnZUUhdPPRXq\nPb35ZuxICotaACJp7nDBBTB/fpgq2rx57Ihkc3z6aZjt86c/wWGHxY4mLs0CEqmHyko488wwLjBh\nAmyxReyIZFN+9KOwsG/48NiRxKcEIFJPa9eGi8qXX8LYsdCkSeyIZGOeeCKM37zxhlpsoAQgkhUV\nFaGCaJMmMHo0NNJoWeKsWAH77huSQLdusaNJBiUAkSz5z3/ghBPCgPCjj6olkDT9+8Ouu8JvfhM7\nkuRQOWiRLGnaFMaPD4ngpJNCl5AkwwMPhBr/N9wQO5LCphaAyCZUVMDpp4fZJuPHhzIDEs/s2fCD\nH8ALL4SaP7JB3loAZnaymc01s3XpvX6rjvc2s1fM7K30P3tVe+0gM3vbzBaa2e2ZfrZIPjVuDI89\nBm3bQp8+YY9ZiWP58jA2c999uvhnQ326gN4GTgSeB6rfrq8AjnX3fYEzgJHVXvsDMMjdOwAdzKxP\nPT5fJG8aNoT77w+DjkceGSpOSn599RWcfDKcdZbKPGdLxgnA3ee7+4Jajr/h7h+nn84DmplZYzNr\nDWzt7rPTrz0CnJDp54vkW4MGcNdd0LMn9OoVCslJ/lx5JWy7LQwdGjuS4pHryW0nAa+6e4WZtQGW\nVnttGdAmx58vklVmcMstYRygZ8+ww9guu8SOqvg99FD4rmfPDolYsuMbE4CZTQVq2075WnefuIlz\n9wKGAb0zD08kecygvDwUHuvWLawYPvDA2FEVr5dfDhu7z5ypQn3Z9o0JwN0zunibWVtgHHC6uy9O\nH14GtK32a23Tx2pVXl6+/nFZWRllZWWZhCKSM5dfHu7++/SBBx8MM1Mkuz75JEzBvece6NgxdjTJ\nk0qlSKVSGZ9f72mgZjYDuMrdX00/bwHMBIa6+/gavzsLGAzMBp4G7nD3ybW8p6aBSsH429/C9pK/\n+AVceGHsaIrHf/4TNus57DC48cbY0RSGvK0ENrMTgTuAHYBVwOvufrSZXQ8MARZW+/Xe7r4yPV30\nIaAZMMndB2/kvZUApKC89x4cc0zYZezXv1Y/dX1Vzfhp3DhU+WzYMHZEhUGlIEQi+eyzUDpixx1h\n5Eho1ix2RIVp7VoYODAkARXjqxuVghCJZPvtw65iTZvC4YfDhx/GjqjwrFsXdvb64gt4/HFd/HNN\nCUAki5o2DYXjfvADOPhgeO652BEVjspKGDQoDPyOGxe+S8ktdQGJ5Mi0aaGG0KWXwpAhGhf4JpWV\nYTe2d9+FSZNUbylTGgMQSZBly2DAgLCCdeTI0E0kX+cekuRrr4XFXtqPOXMaAxBJkDZtYMaMULjs\nwAPDoibZoLISfvKTsML3mWd08c83tQBE8mTcODj//FDL5uKLw4riUrZ6NZx2Gvzzn6HMtlpH9acW\ngEhC/fCH8NJL8MgjoaLo++/HjiiehQuha9fQQpo2TRf/WJQARPKoQ4eQBI4+Gjp3hjvuCN0gpWTK\nFDj0UBg8GP7wB031jEldQCKRLFgQpj26h70Gvve92BHlljvcemv4GTMmlHiQ7FIXkEiB2H33UOFy\n4EDo3j2UmV67NnZUubFmTZgSO2oUzJqli39SKAGIRNSgAVxySZgd9OyzcNBBYTZMMTWAX3wxdHet\nWxcea/+E5FAXkEhCuMOTT8K110KrVjBsWBgoLVQrVsDVV4fyGLfeCv37a+ZTrqkLSKRAmYWZQnPm\nhO6Sk08Oz995J3ZkdbNuHdx9N+y1F2y3HcybFxbD6eKfPEoAIgnTqFEYHF6wAA45BHr0gHPOgcWL\nN31ubK++GmIeOTJM77ztNthmm9hRycYoAYgkVLNmYSvEBQtCielOncL00fHjkzVY7B42xTn99FAE\n76KL4PnnYd99Y0cmm6IxAJECsWZNqI//xz/CkiWhlXDOOfEGVb/4AkaPhhEjYNWqsBvaoEGh20fi\nUDE4kRIh4Kp1AAAFGklEQVQwZ07oZx81KnS59OsHRxwB7drl/rPnzw9JaOTIsKDroougd29VO00C\nJQCREvLFF6FVMGlS2HugRYuQCI44Anr1gh12qN/7u8OiRWH18l/+Ev65cmW40z/vPNh11+z8e0h2\nKAGIlKjKSnj77ZAIpk+HF16Ab387LDhr2zbU3an+zx13DN1Kq1eHn1WrNjxetgz++tdwwW/SJCxU\n694dunWD/fYLA9WSPPncFP5koBzYA+jk7q/VeH0XYB4w1N1vTR+r2hR+C8Km8Jdt5L2VAETqqaIC\nXn89zB5atgyWLg0/VY8/+QSaNw97FWyzzdd/dtwxrEHo3j0/3UqSHflMAHsAlcDdwJW1JICxwDpg\ndrUEMBu4xN1nm9kk4A53n1zLeysBpKVSKcrKymKHkQj6LjbQd7GBvosN8rYQzN3nu/uCjQRxAvA+\noQVQdaw1sLW7z04fegQ4IdPPLxWpVCp2CImh72IDfRcb6LvIXNbH7c1sK+BqQvdQdW2ApdWeL0sf\nExGRCL5xKMfMpgKtannpWnefuJHTyoHh7v5vMy3+FhFJqnrPAjKzGVQbAzCz54GqYaMWhHGCnwPj\ngBnuvmf6904Berr7BbW8pwYAREQyUJcxgGxN5lr/ge7eY/1Bs6HA5+4+Iv18tZl1AWYDpwN31PZm\ndfkXEBGRzGQ8BmBmJ5rZB0BX4Gkze2YzTrsIuA9YCCyqbQaQiIjkRyIXgomISO4lqnqHmfUxs/lm\nttDMfhY7nljMrJ2ZzTCzuWY2x8wGx44pNjNraGavm9nGJh+UBDNrYWZjzewdM5tnZgW8ZUz9mNk1\n6b+Rt81slJk1jR1TvpjZA2a23MzernZsezObamYLzOxZM2uxqfdJTAIws4bAXUAfoCNwipntGTeq\naCqAK9x9L0IX28Ul/F1UuYywrqTUm6y3E1bR7wnsCxTYdjHZYWbfBs4FDnT3fYCGwMCYMeXZg4Rr\nZXVDgKnuvjswPf38GyUmAQCdCeMCS9y9AvhfoG/kmKJw94/d/Y30438R/sh3jhtVPGbWFjiGMH5U\nshMEzGxb4DB3fwDA3de6+6rIYcWymnCj1NzMGgHNCWuLSoK7vwD8o8bh44GH048fZjMW2iYpAbQB\nPqj2fClaKFZ1p3MAMCtuJFENB35KmFJcytoDK8zsQTN7zczuNbPmsYOKwd0/A24F/g/4EPinu0+L\nG1V0O7n78vTj5cBOmzohSQmg1Jv2/yW9qnoscFm6JVByzOxY4BN3f50SvvtPawQcCIxw9wOBL9iM\nZn4xMrPvAJcD3ya0jrcys1OjBpUg6WJqm7ymJikBLGPDAjLSj5du5HeLnpk1Bp4AHnX38bHjiagb\ncLyZLQZGA4eb2SORY4plKbDU3V9OPx9LSAil6GDgJXf/1N3XEhaadoscU2zLzawVrK+99smmTkhS\nAngF6GBm3zazJsAAYELkmKJIl9C4H5jn7r+LHU9M7n6tu7dz9/aEQb7n3P3HseOKwd0/Bj4ws93T\nh44E5kYMKab5QFcza5b+ezmSasUnS9QE4Iz04zOATd44JmZbB3dfa2aXAFMII/r3u3tJznAAugOn\nAW+Z2evpY9do4RygrsJLgcfSN0nvAWdFjicKd38z3RJ8hTA29BpwT9yo8sfMRgM9gR3SC3J/AQwD\nxpjZIGAJ0H+T76OFYCIipSlJXUAiIpJHSgAiIiVKCUBEpEQpAYiIlCglABGREqUEICJSopQARERK\nlBKAiEiJ+n+UjKRoMB6ItgAAAABJRU5ErkJggg==\n",
"text/plain": [
""
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"def g(x): return x**3 - 10 * x**2 + 2 * x - 2\n",
"xs = sp.linspace(0, 10)\n",
"plt.plot(xs, g(xs));"
]
},
{
"cell_type": "code",
"execution_count": 23,
"metadata": {
"collapsed": false
},
"outputs": [
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAYAAAAEACAYAAAC6d6FnAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAHadJREFUeJzt3XmcVPWZ7/HPwyatEpchAUQUN4KomVEWtyhtsGmVSEDH\nOHpHuQ4JbjFEehICaCBRXhPnpo3Gm+hFTWyT4LhgI0btplFbRaPoiIoLolEUiKADQVAhsjz3j3Na\nupvqpqq6qs6pOt/369Uvq845VfXYtr+nzu/5LebuiIhI8nSKOgAREYmGEoCISEIpAYiIJJQSgIhI\nQikBiIgklBKAiEhCdcnnm5vZcmADsA3Y4u7DzGxf4G7gQGA58G13X5/POEREZGf5vgNwoNzdj3b3\nYeGxHwMN7j4AeDR8LiIiBVaILiBr9Xw0UBM+rgHGFCAGERFppRB3AAvM7AUz+254rJe7rwkfrwF6\n5TkGERFJIa81AOBEd//AzL4MNJjZ0uYn3d3NTGtRiIhEIK8JwN0/CP/5kZnVAsOANWbW291Xm1kf\n4MPWr1NSEBHJjru37nZvU966gMxsdzPrET7eAxgJLAHmAePCy8YBc1O93t2L9mf69OmRx5DE2BV/\n9D+KP9qfTOXzDqAXUGtmTZ/zR3efb2YvAPeY2XjCYaB5jEFERNqQtwTg7u8C/5Ti+Drg1Hx9roiI\npEczgfOgvLw86hCyVsyxg+KPmuIvLpZNv1G+mZnHMS4RkTgzMzwORWAREYk3JQARkYRSAhARSSgl\nABGRhFICEBFJKCUAEZGEUgIQEUkoJQARkYRSAhARSSglABGRhFICEBFJKCUAEZGEUgIQEUkoJQAR\nkYRSAhARSSglABGRhFICEBFJKCUAEZGEUgIQEUkoJQARkYRSAhCRxKuvr2fkyLMZOfJs6uvrow6n\nYMzdo45hJ2bmcYxLREpPfX09Y8eOY9Om6wAoK5tMbW0NlZWVEUeWOTPD3S3t6+PY0CoBiEihjBx5\nNg0No4Fx4ZEaKirmMX/+nCjDykqmCUBdQCIiCdUl6gBERKJUVTWBhQvHsWlT8LysbDJVVTXRBlUg\n6gISkcSrr6+nunoWECSEYuz/B9UAREQSSzUAERFJixKAiCTae+/BZ59FHUU0lABEJNEuvRQeeSTq\nKKKhGoCIJNbf/w5f/nJwF7DPPlFH03GqAYiIpOnpp+GII0qj8c+GEoCIJNb8+TByZNRRREcJIIdS\nLSiV1EWmRIqBEkAEzOw0M1tqZm+Z2eQoYsi1pgWlGhpG09AwmrFjxzFz5sydjqVKAkoSIoW3dSsc\neSQMGxZ1JNEpeBHYzDoDbwKnAquA54Hz3P2NZtcUXRE41YJS++57DevWXU17i0yV0kqEIhKtYigC\nDwPedvfl7r4F+C/gWxHEEQvV1bPCxn8cECSCpinpIiL5FMVicH2BFc2erwSOjSCOnEq1oNSkSVcw\nc+bkRC4yJSLxF0UCSKtvZ8aMGV88Li8vp7y8PE/h5EZlZSW1tTXNFpQKunGGDBmy07HmkrwSoYh0\nTGNjI42NjVm/PooawHHADHc/LXw+Bdju7tc1u6boagBN3ngD5s5teaxTJxgyBEaMSP2aUlmJUESi\nFfvVQM2sC0EReATwV2ARJVAEbrJkCcye3fLYli1wyCHBlPPWNm6EsjLoop0ZRAqivr6eyy77G337\nPsa0aWeX1Beu2CcAADM7HbgB6Azc7u7/0ep80SaATN14I/z0p/CNb8CoUXDOObDnnlFHJVKa6uvr\nGTPmcjZvfh24l7KyqpIadVcUCWBXkpQAAD74ABoaYM4ceOopOPdcmDoV+vWLOjKR0hIM174CKA+P\nFO/+v6kUwzBQaaVPH7jwQnjggaALqU8fSFD+Eymw/aMOIDZ0ByAiifHww/WMGjUEeBz4tOQmXuoO\nIOY6suzDiy8GdwnKjSLZ2WefSvr370ZFxV1UVMwrqcY/G7oDKKCOLvvw9NNw+eVBkfiWW4J1TDL5\nbA01laTbtAlWrIABA6KOJD9UBI6xVOsFZVqA2r4dZs2Cn/wExo+Hq6+G3Xdv/zVab0gkGdQFVOI6\ndYJLLoFXXgl2MTr55F13CbW13pBWIRVJNk0/KqBcLvvQu3cw4Wz1arC08/0O//M/a1rcFSxcOE53\nBSIJozuAAmpaL6iiYl7OClAvv7zrb/FVVRMoK5sM1AA14eMuWoVUJOF0B1BglZWVOfuW3bpvv+lb\n/PbtcP31t2G2/YuCb+uF6tTYS5L8/e/w2WfJ3fu3LUoARaT1SJ6WffvBCIcpU67h1VePZMuWm4Cn\nWnTttE48WoVUkqK+Hm66KZhxLzsoARSJVN/2Bw4cuNN17723mi1bJgC9gXPYtGkPqqtn7dT4t7V8\ntUgp+tOfgrW2pCUlgCKR6ts+3EpZWcsNZw48cCDr1jUfFnQKK1e+n/I9c9kdJRJX7kEC+NGPoo4k\nfpQAiljPnr2orb26xbd4ILxTCK7Zbbdb+PTTBn7xC/j3f48qUpHoLF4MX/oSHHpo1JHEjxJAkWhr\nCGmqb/Etu3ZmcMIJe/LJJ4WOWCQeHngAvvnNqKOIJ80ELiJazkEkcz/7GZx+OgwdGnUk+aelIERE\nEkpLQUhatm+HZ5+NOgoRiZISQEKtWAFnnw2//nXUkYhIVFQETqgDDwy2nzztNFi7NlhVNJs1hUSk\neOkOIKYKsVLnwQfDk0/CffcFY6RVdhFJFiWAGGqa9dvQMJqGhtGMHTsub0mgd29obIQnnoDrr8/L\nR4hE4gc/gFWroo4i3jQKKIZysXFMpjZsgG3btFiWlIaVK+FrXwuWS+/WLepoCifTUUCqAQgQzJQU\nKRX33w9nnpmsxj8bSgAxlMuNY0SSaM4cLX2SDnUBxVQcZv1u3RoUhrt2LfhHi2RtzRr46leD7p/u\n3aOOprA0E1hyproaFi2CP/4RumRxrxiHJCbJc+ut8NhjcNddUUdSeEoAkjObN8OYMUFh+Pe/zywJ\ntN6/oKxssvYcloLYuhXWr4eePaOOpPCUACSnNm+G0aPhK1+Bmhro3Dm910Uxkkkk6bQWkORU9+7B\ncrqrVsFll2mymEgpUQKQXSorg3nzgmJw08ikXamqmkBZ2WSgBqgJRzJNyGeYkiDpzpQvxIz6YqYu\nIMkbFYElH9KtLyWxDqUagMSakoJ0VFv1pZ//fA4HHgj/8A/tX1fKdSjVACS2CrnGkSTPRRfBkiVR\nR1FclAAka59/Ds8/n/711dWzwtvxcUBwa950NyCSrlT1pbPOmsS6dXDyye1fpzpUS0oAkrW33oJR\no2DhwqgjkSSprKyktjbozqmomEdtbQ3Ll5/I+edDp07tX6cux5byUgMwsxnAd4CPwkNT3f2R8NwU\n4N+AbcD33X1+iterBhCBdPvnm19XXj6FG28cwoIFcNRRu37/pBXlJP+2b4f+/eGhh3b9N1jqMq0B\n4O45/wGmA5NSHB8EvAR0BfoDbwOdUlznUlh1dXVeVtbL4Q6HO7ysrJfX1dWldd3kyS/7/vu7L1+e\n3udUVJzlFRVnpXx/kUw1NrofdVTUUcRD2Ham3Vbn6w5gOvCJu1e3Oj4F2O7u14XP64AZ7v5sq+s8\nH3FJ29IdMdHWdaNGzeHmm4PuoEyn4GtkkHTE0qXw/vswcmTUkUQvTvsBXGFmFwIvAFXuvh7YD2je\n2K8E+uYxBimQiRNh770zX32xdbfQwoXj1C0kGRk4MPiRzGWdAMysAeid4tQ04GbgZ+Hza4BqYHwb\nb6Wv+jGQ7h4E7V03btxOl+9Sy5FBwUzj6upZSgAiBZB1AnD3inSuM7PbgAfDp6uAfs1O7x8e28mM\nGTO+eFxeXk55eXk2YUqamkZM7OiKSf0tPN3rRCT/GhsbaWxszPr1+aoB9HH3D8LHVwJD3f18MxsE\nzAaGEXT9LAAObd3hrxpAcmhkkEjuxGIpCDO7E/gngu6dd4GL3X1NeG4qwTDQrcBEd99pKqgSQGnY\ntAmefRZOOaX961QElmxs3py8Hb92JRYJoKOUAOKjI43zsmVw0kkwezaMGJGvCCWphgyBm2+GoUOj\njiQ+tBaQ5ExH1+4ZMADuuQfOOw9efTWPgUpJS7Wk83//N6xdC4MHRxxckdMdgLQpk9UU27tT+MMf\n4Kqr4M9/hj59ChC4lIy2akT331/JAQfAtGkRBxgzcZoHIAmxq7H8//qv8O67cOaZ0NgIe+4ZYbBS\nVFINE77uujt56aVK3VXmgBKAtCnduQHpjOW/6iro1SuzjeVFUlmz5kROOgn226/lcQ0myJz+d5Q2\n5XLMvxlM0Eq8kqFUX0JGjHiU0aNbXqcZ5dlRDUA6TGP5JZ/S+WafxN2/UlENQApOs4MlnyorK/X3\nlCe6A5DIbNgAK1fCoEFRRyJxkk1fvu5CA5oIJkWjvh7Gjw+Gh/brt+vrpfR1pCFXEVgJQIrML34B\nv/99sI9Ajx5RRyNRU19+x2gmsBSVqio4/ng491zYujXqaCSuXn892H9acksJQCJlBjfdBNu2BZvK\n6MYv2aqqJlBWNhmoAWrCuScTuOEGOPbYqKMrPUoAErmuXYM1g77ylWCDb0i9/ouUvqYRZRUV86io\nmEdtbQ2DB1dy771wySVRR1d6VAOQ2NGIDmlu5kx45x24/faoI4k/FYGl6KkQKE0+/xz69w9GjB11\nVNTRxJ+KwCJSMt59F04/XY1/vugOQGJnRxfQDcA2ysqqqK0NFqFL+jhvkfaoC0hKQn19PZdeuoGN\nGw+mpmYtnTu76gIlSJO3cksJQErGli3B2O8BA+DNN89mwQLVBUqJiv25p8XgpGR07Qr33gsnnghb\nt2oWUKlJZx8JyS8lAIm1vfaChx6CwYPPp1u3i/n88+B4W5vTSGlwDyYJSn6pC0iKwvPPw403vs2H\nH04G1F9cCtrqAho6tJIRI4JFArt3jzjIIqMagCSGCojFL9V/w+nTYcUK+O1vIw6uCCkBSCKogFia\n1q+HQw+F556DQw6JOprioyKwJIIKiKXpppuCkV9q/AtDCUCKWLeoA5Ac2rABfvUreOaZqCNJDi0F\nIUWpqmoCnTr1AF6i+bLBUrw++AC+/3047LCoI0kO1QCkaN199+NcdNHhHHTQfVx//WHq/pHE02Jw\nkhjnnnsKixf3Zu3a7+Guxr8QtE9DadEdgBS9p5+GMWOgrg4GD446mtKlkVfxp2Ggkkhz5wabhkya\nFHUkpUv7NMSfhoFKIo0ZE3UEko2NG4P9oPfeO+pIkkk1ACl56rfOjbY2bO+ImTNh2rSchCdZUBeQ\nlDT1W+dWLpffWL48qNm88gr07ZujABNONQCR0OrVcMEF/8yCBWeifuv4Oe88+OpXYcaMqCMpHRoG\nKhK6/HJYtuzCqMNIpF11uz33HDz5JPzwhxEEJ1/IOgGY2Tlm9pqZbTOzY1qdm2Jmb5nZUjMb2ez4\nYDNbEp67sSOBi+zKbbdBp06n0KXLCtLpt07VaKl+kLmmbreGhtE0NIxm7NhxLX537lBVBddeC3vs\nEWGgAu6e1Q8wEBgAPA4c0+z4IIL5+V2B/sDb7OhqWgQMCx8/DJzWxnu7SDbq6uq8ouIsr6g4y+vq\n6nzVKvfevT/1ww//zRfH2npdWVkvhzsc7vCysl5+7bXX7nSsrdfLDhUVZ4W/Mw9/7vCKirNaXPPC\nC+5bt0YUYAkL28602/Gsh4G6+1II+pxa+RZwl7tvAZab2dvAsWb2HtDD3ReF190JjAHqso1BpLnW\nBd+FC8dRW1vDk09WcvLJl3LNNZfSVLNsXcxMtbro9ddfoxVH09D6d5kOTdiLh3zMA9gPeLbZ85VA\nX2BL+LjJqvC4SE60tUT0/PmVPPQQPBv+VaZKFAMHDowo6uKW6nc5bdoVLFw4mU2bgmu0fWd8tZsA\nzKwB6J3i1FR3fzA/IYnk3jHHBD+QOlHArZSVtWy0Jk26gpkz1ZC1J9Xv8okn5lFbW9PsrkDDbuOq\n3QTg7hVZvOcqoF+z5/sTfPNfFT5ufnxVW28yo9nYsPLycsrLy7MIRZKkqmoCCxeOy6rB7tmzF7W1\nV+/UaA0ZMkQNWRYqKyv1uyqAxsZGGhsbs3+DTAoGqX4IisCDmz1vKgJ3Aw4C/sKOIvBzwLGAoSKw\n5EHrInBb1yS5uJvO7yiT90rndzl7tvt//meHPkrSQIZF4I40/GOBFcAmYDXwSLNzUwlG/ywFKpsd\nHwwsCc/9qp33zu9vSRKvrq7Ojz9+oh9//MTENf65Tn67SigffeTeq5f7c8916GMkDZkmAM0ElsS6\n+2648kpYsAAGDYo6msKIYkXPCy6Anj3hl7/M20dISKuBiqTp3HODlShPPRXmz4cjj4w6otJTVwcL\nF8Krr0YdiaSiBCCJdv750KkTVFQESeCoo6KOKL86UijP1CefwCWXwKxZmvEbV+oCEmFHd9Brr8E+\n+0QdTX7lckXP9qxfD3PmwPjxeXl7SUGrgYpk6Z134OCDo45CJHtaDVQkS0lu/LXoXTLpDkAk4bRp\nTulQF5BIDn38Mey1V9RR5Fcuh4Zu3QpdNLQkMuoCEsmR7dth+HC4+eaoIykO998P3/521FFIJpSr\nRdrQqVPQqFVUBNtLzpgBO69+XvxyMTT0nXeCIZ9/+lMeApS8UReQyC6sWQOjR8Nhh8Htt8Nuu0Ud\nUe51ZGjounVw0klw6aXwve/l5zMkPaoBiOTBZ5/BhRfC2rXw6KPB3UFzSW3cNm8O7pCGDYPq6rav\nU6G5MJQARPJk+/ZgU5kTTmh5PFXjNm3aFTzxxItA8SaEdJLaLbdAYyPMnr1zUmwuijWIkkhrAYnk\nSadOOzf+kGpTlCX85CfVbN8erH7WtDVlMSWBtrbXbP3vcPHFwUzf9hp/iS8lAJGcezps/It3L+G2\nttds/e9gBl277vr9CrkGkaRPeVukg84444d07z4TqAFq6NTprahDaiEOs3wrKyuprQ26fSoq5hXd\nHVGp0h2ASAd99NFx7LvvYg444Gp69HiP4cOvjM1ewul25bSW6hv7pEk1bNwIPXpkF4u2iYwfFYFF\ncmD2bJg4EaZPh8sug4aG9EYF5Xv0UEeKr81ju/LKCTz6aCXvvw/33JPTECWHVAQWicD558OQIcFQ\n0blz4Xe/q2T+/PYb82y/nRdK0zf2rVvhO9+BZcs00avUqAYgkiMDBgS7X33jG7Bq1a6vb1loDRJB\n0zfuXKmqmkBZ2WSa6hNBd9SEtF//2Wdw9tnBZLiGBth335yGJxHTHYBIDnXpAlOnRh3FDk3F1x3d\nTOnfYaxfDyeeCEOHBrt6deuWz0glCqoBiEQk7rNj3eGpp+Dkk6OORNKlmcAiMfXTn8LRR8OZZ+5Y\nVC6pS0hIfigBiMTU/PnBSKF+/YLRQieeGHVEUmq0H4BITI0cCS+/HBRVL7gg2GtgwYKoo4LHHw+W\nuPjLX6KORApNCUCkgLp1C9bPWbYMvvtdWLRo52sKNXN3yRIYNSoY4jlxIhx0UN4+SmJKXUAiMVKI\nwvDixUGD/+abMG1asJGLRviUBtUARIrYzjN3H+fkk//AE0/cnrPPWLECXngh+Pavhr+0aCawSEk5\nlGeeuYFDD4Wvfx1+/WvYY4+dr3r/fdi4Mfj5+GN47bXg57bbdt7Gsl+/4EdECUAkRlItwjZnTg0H\nHFDJwoVtf2MfMSJYlrlHj+Dn8MOD8fvbtgWT00RSUReQSMxoboBkSzUAkTyLooFWUpB0KAGI5FEU\nyzfEfckIiQ8lAJE8imJzc22oLunSTGCRhIjDVo9S3DQ+QCQDUWxunuozhw+/ItabyUhxUBeQSIbi\nUASurp6lbiHZScFqAGZ2DjADGAgMdfcXw+P9gTeApeGlf3b3y8Jzg4E7gO7Aw+4+sY33VgIQaYfq\nApJKIWcCLwHGAv8vxbm33f3oFMdvBsa7+yIze9jMTnP3ug7EIJJIUXRFSenJugjs7kvdfVm615tZ\nH6CHuzetf3gnMCbbzxdJsqatHisq5lFRMU/9/5KVfBWBDzKzxcDHwFXuvhDoC6xsds2q8JiIZKGy\nslKNvnRIuwnAzBqA3ilOTXX3B9t42V+Bfu7+NzM7BphrZkdkGtiMGTO+eFxeXk55eXmmbyGSOJox\nnCyNjY00NjZm/foOjwIys8eBqqYicFvngQ+Ax9z98PD4ecBwd78kxWtUBBbJkGYMS1QTwb74QDPr\naWadw8cHA4cB77j7B8AGMzvWzAy4AJibo88XSbzq6llh4z8OCBJB092ASCpZJwAzG2tmK4DjgIfM\n7JHw1HDg5bAGcC9wsbuvD89dBtwGvEUwUkgjgEREIqKJYCIlQl1AosXgRBJMReBkUwIQiSk1zpJv\nSgAiMaTuGSkEJQCRGNLaPVII2g9ARETSov0ARApAi7dJHKkLSKRAVASWfFMNQEQkoVQDEBGRtCgB\niIgklBKAiEhCKQGIiCSUEoCISEIpAYiIJJQSgIhIQikBiIgklBKAiEhCKQGIiCSUEoCISEIpAYiI\nJJQSgIhIQikBiIgklBKAiEhCKQGIiCSUEoCISEIpAYiIJJQSgIhIQikBiIgklBKAiEhCKQGIiCSU\nEoCISEIpAYiIJJQSgIhIQikBiIgklBKAiEhCZZ0AzOz/mNkbZvaymd1vZns1OzfFzN4ys6VmNrLZ\n8cFmtiQ8d2NHgxcRkex15A5gPnCEu/8jsAyYAmBmg4BzgUHAacBvzMzC19wMjHf3w4DDzOy0Dnx+\nbDU2NkYdQtaKOXZQ/FFT/MUl6wTg7g3uvj18+hywf/j4W8Bd7r7F3ZcDbwPHmlkfoIe7LwqvuxMY\nk+3nx1kx/xEVc+yg+KOm+ItLrmoA/wY8HD7eD1jZ7NxKoG+K46vC4yIiEoEu7Z00swagd4pTU939\nwfCaacDn7j47D/GJiEiemLtn/2Kz/w18Fxjh7pvDYz8GcPefh8/rgOnAe8Dj7n54ePw8YLi7X5Li\nfbMPSkQkwdzddn1VoN07gPaEBdwfEjTim5udmgfMNrPrCbp4DgMWubub2QYzOxZYBFwA/Kqj/wIi\nIpKdrO8AzOwtoBuwLjz0Z3e/LDw3laAusBWY6O714fHBwB1AGfCwu3+/Q9GLiEjWOtQFJCIixSu2\nM4Hbm2gWV2Z2Wjj57S0zmxx1PJkws35m9riZvWZmr5pZUd6dmVlnM1tsZg9GHUumzGxvM7sv/Lt/\n3cyOizqmdIWTP18LJ3rONrPdoo6pPWb2WzNbY2ZLmh3b18wazGyZmc03s72jjLE9bcSfcZsZ2wRA\nGxPN4srMOgP/l2Dy2yDgPDM7PNqoMrIFuNLdjwCOAy4vsvibTAReB4rx1vZGgq7Rw4GvAW9EHE9a\nzKw/wWCQY9z9KKAz8C9RxpSG3xH8v9rcj4EGdx8APBo+j6tU8WfcZsY2AbQz0SyuhgFvu/tyd98C\n/BfBpLii4O6r3f2l8PEnBI3PftFGlRkz2x84A7gNKKqBBOG3tZPc/bcA7r7V3T+OOKx0bSD4ArG7\nmXUBdieY5xNb7v4U8LdWh0cDNeHjGmI8UTVV/Nm0mbFNAK00n2gWV32BFc2eN02AKzrhN7qjCf6I\niskvCUambd/VhTF0EPCRmf3OzF40s1vNbPeog0qHu68DqoH3gb8C6919QbRRZaWXu68JH68BekUZ\nTAel1WZGmgDC/rYlKX7ObHZNsUw0K8Yuh52Y2Z7AfQSjtz6JOp50mdk3gQ/dfTFF9u0/1AU4BviN\nux8DfEq8uyC+YGaHAD8A+hPcNe5pZv8r0qA6yIPRMUX5/3QmbWbW8wBywd0r2jsfTjQ7AxhRkIA6\nZhXQr9nzfrRc+iL2zKwrMAf4g7vPjTqeDJ0AjDazM4DuwJfM7E53vzDiuNK1Eljp7s+Hz++jSBIA\nMAR4xt3XApjZ/QT/Pf4YaVSZW2Nmvd19dbh22YdRB5SpTNvM2HYBNZto9q1WE83i6gWCFU77m1k3\nghVR50UcU9rCFVtvB1539xuijidT7j7V3fu5+0EEBcjHiqjxx91XAyvMbEB46FTgtQhDysRS4Dgz\nKwv/jk4lKMQXm3nAuPDxOKCovgRl02bGdh5AexPN4srMTgduIBgFcbu7/0fEIaXNzL4OPAm8wo5b\n3ynuXhddVNkxs+FAlbuPjjqWTJjZPxIUsLsBfwEuKpZCsJn9iKDR3A68CHwnHAwRS2Z2FzAc6EnQ\n3/8T4AHgHuAAYDnwbXdfH1WM7UkR/3SCUT8ZtZmxTQAiIpJfse0CEhGR/FICEBFJKCUAEZGEUgIQ\nEUkoJQARkYRSAhARSSglABGRhFICEBFJqP8Pk2b1V1XPB7gAAAAASUVORK5CYII=\n",
"text/plain": [
""
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"X = sp.linspace(0,10)\n",
"y = g(X) + 5 * stats.norm.rvs(scale=3, size=50)\n",
"plt.plot(xs, g(xs), '--')\n",
"plt.scatter(X, y)\n",
"plt.show()"
]
},
{
"cell_type": "code",
"execution_count": 24,
"metadata": {
"collapsed": false
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"error(h1) = 45943.12\n",
"error(h2) = 12409.02\n",
"error(h3) = 4554.71\n",
"error(h4) = 4480.06\n",
"error(h5) = 4262.50\n",
"error(h6) = 3971.45\n",
"error(h7) = 3663.57\n"
]
},
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAYAAAAEACAYAAAC6d6FnAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzs3XdUFGcXwOHf0hQBRbCAAoLd2AsaxQIqWJIYe4uKvccY\nNZZYYk3UxNjyxV5j7L1jQWxRsYBib4C9N1A69/tjEVFBWQUW9H3OmRN2dmb2nfVk7r71akQERVEU\n5fNjoO8CKIqiKPqhAoCiKMpnSgUARVGUz5QKAIqiKJ8pFQAURVE+UyoAKIqifKaMUvPiGo0mCHgG\nxABRIlJRo9FYASuAfEAQ0FxEnqRmORRFUZS3pXYNQABXESkrIhXj9g0GdopIYWB33GtFURQljaVF\nE5DmjdcNgEVxfy8CGqZBGRRFUZQ3pEUNYJdGozmm0Wi6xO3LLSJ34/6+C+RO5TIoiqIoiUjVPgDA\nRURuazSanMBOjUZzPuGbIiIajUatRaEoiqIHqRoAROR23H/vazSadUBF4K5Go7ERkTsajcYWuPfm\neSooKIqifBgRebPZPUmp1gSk0WiyaDQai7i/zQAPIADYCHjGHeYJrE/sfBHJsNsvv/yi9zJ8jmVX\n5df/psqv301XqVkDyA2s02g0Lz/nXxHZodFojgErNRpNJ+KGgaZiGRRFUZQkpFoAEJFAoEwi+x8B\ntVPrcxVFUZTkUTOBU4Grq6u+i/DBMnLZQZVf31T5MxbNh7QbpTaNRiPpsVyKoijpmUajQXToBE7t\nYaCKoigpIq4/UYmTEj+SVQBQFCXDUC0DWikVDFUfQEo7eRJiY/VdCkVRlPdSASAliUD37rBqlb5L\noiiK8l6qEzilbd8O/fpBQAAYGuq7NIryyYjr4NR3MdKFpL4LXTuBVQ0gpdWpA9mywcqV+i6Joih6\n0r59e4YPH67vYryXCgApTaOBUaNg9GiIidF3aRRF0QONRqNTR21UVBRNmzbFyckJAwMD9u7dm4ql\ne0UFgNTg7g5WVrB8ub5LoiiKnujaXFW9enWWLFmCjY1Nmg15VQEgNSSsBURH67s0iqKkMj8/P8qV\nK0fWrFlp2bIl4eHhOp1vbGxMnz59cHFxwTAN+w5VAEgttWpBrlywbJm+S6IoSiqKjIykYcOGeHp6\n8vjxY5o1a8aaNWvQaDRcv34dS0tLsmfPnui2XM+tBGoiWGp5WQvo1g1atQIj9VUrSmrSjEqZZhP5\nRbemm8OHDxMdHc0PP/wAQJMmTXB2dgbA3t6eJ0+epEi5UoN6KqUmNzfIkwf+/Rc8Pd9/vKIoH0zX\nB3dKuXXrFnnz5n1tX758+TLEkFXVBJSaXtYCxoyBqCh9l0ZRlFRga2vLzZs3X9sXHBwc3wRkbm6O\nhYVFotsyPTcRqwCQ2lxdIV8+WLhQ3yVRFCUVVKlSBSMjI6ZNm0ZUVBRr167l6NGjgLYJKDQ0lJCQ\nkES3Vq1axV8nIiIivvM44d+pSc0ETguHD0Pz5nDxImTOrO/SKEqGlJ5nAh8/fpwuXbpw+fJl6tev\nj0ajoVChQowePTrZ13B0dOTatWvx96nRaAgMDMTBweGtY1NqJrAKAGnl22+1fQJ9++q7JIqSIaXn\nAJDWVADIaE6dAg8PuHwZzM31XRpFyXBUAHhFrQWU0ZQqBTVrwtSp+i6JoigKoGoAaevSJahcWdsX\nYGWl79IoSoaiagCvqBpARlSoEDRuDL//ru+SKIqiqBpAmrt+HcqUgTNnwMZG36VRlAxD1QBeUZ3A\nGdmPP2qXip42Td8lUZQMQwWAV1QAyMju3YMvvgBfX8ifX9+lUZQMQQWAV1QfQEaWKxf06QPDhum7\nJIqifMZUANCXfv3AxweOH9d3SRRFSWEqJaTybubmMGIEDBoEqlqrKJ8UXVNCHj58GHd3d6ytrcmV\nKxfNmzfnzp07qVhCLRUA9KlTJ+2ooB079F0SRVFSmC79FU+ePKF79+4EBwcTHByMhYUFHTp0SMXS\naal8APpkbAy//aatBbi7g4GKx4qSEfn5+dGpU6fXFoPTRd26dV973atXL1xdXVOwhIlTTxx9a9QI\nsmTRJo1RFCXDSY2UkPv27aNEiRKpXnZVA0hh4eHhZNZlyWeNBiZOhDZtoFkztVy0onwoHX91J0nH\nPrmUTgl56tQpxowZw8aNG3U670OoGkAKEhGqV6/OoUOHdDuxalXt7OD//S91CqYonwORlNl0lJIp\nIV82IU2bNg0XFxedz9eVCgApSKPR0KNHDwYPHqz7P/5vv8GECfDwYeoUTlGUVJFSKSGDg4Nxd3dn\nxIgRfPfdd2lSdhUAUljbtm25f/8+Xl5eup1YrJi2CUiHDEKKouhfSqSEvHnzJjVr1qR379507do1\nzcquAkAK8vLyon79FpiYZKVXr17Exsbi5eWFh0cTPDyavD8ojBwJS5fChQtpUl5FUT6esbExa9eu\nZeHChVhbW7Ny5UqaNGmi0zXmzp1LYGAgI0eOjK8dZM2aNZVK/Ipe1gLSaDR1gSmAITBXRCa88X6G\nWwvIy8uLRo08CQubAAgaTReaNm3E5s374vaBqekg1q1bRJ06dd46d9Kk2QBMz2dFkTt3YNOmtL4F\nRUnX1FpAr2TYxeA0Go0hcAGoDdwEjgKtRORcgmMyXADw8GjCzp0NAM+4PYMxMJhEbOxMoFPcvkW4\nu29kx4418ee9HjggW+aB3MpuSJbFi6F27bS8BUVJ11QAeCUjLwZXEbgsIkEiEgUsB77VQzlSWTEM\nDIyBfe88atKk2XEPf0/Ak6fhE/k9pwP0769dMlpRFCWV6CMA5AWuJ3h9I25fhta/f1dMTQeSN28A\nmTPfx9BQg5vbNGAbMBtYhKnpIPr3f38Hz8FcecDSEubPT+1iK4ryGdPHRLBk1eFGjhwZ/7erq2ua\nTIv+GHXq1GHdusUsW/Y9xYtHEhTUmMDA7hgatsTIKJhixTby229vt//379+VAwc8CQvTvjY1HUT/\nAYsgRw74+mto0QLSoDNIUZSMx8fHBx8fnw8+Xx99AF8CI0WkbtzrIUBswo7gjNgH8NLBM5Hs2Qh2\ntn+RN++vHDv2HaNG3cPaejGlSxszdao2NXBCCTuB+/fv+ipIeHpCnjzaOQKK8plTfQCvZOROYCO0\nncC1gFuAL59AJ/BLFQ744R/5DLMIE5wintA+058cWXGMg4fy0LGdH39NNqFLFxg6VLsidEgImJqC\nUWJ1sVu3oFQpOHwYChZM83tRlPREBYBXMmwAANBoNPV4NQx0noj89sb7GTYAAETGxnI9IoLg8HCC\nw8N5+GALy7r1wSKvOZnaz+T6bHcenMjCiqUaTp0wYNQoqFkTvvpKOxfM3DzBxSZMgAMH1LBQ5bOn\nAsArGToAvE9GDwAJRT2J4sWZFzy4exfXXpXp1CWKMuVrMW/7ELYvLYNz5yf81MOEp/9lY90aDfv3\na5v9f/4Z7O2BiAgoWRKmTIH69fV9O4qiNyoAvKICQDr1aNcjQnxDCDkRQuiJUKLuR5GleBYMzQ05\n9/wcPU/0ZGafWtjU3MWd9R0ZNW84WYxjaFTyInmdM1OilgNbT1nTvqMGB4e4i27bps0hfPo0ZMqk\n1/tTFH3JSAGgffv22NvbM2bMmFS5fkaeB/BJe7DuAdGPo8nZJCeltpWi6pOqlD9cnjK7ytDqUCsm\nTJ/AmG3nKVHxMKV/fMrfO0tQstEV/hfkzLnbppz+6TQV/9rPnRFneLDxATFhMVCvnnatoMmT9X17\niqIkg64pIc+ePUuFChWwsrLC0tISFxcXDhw4kIol1FL5AFJY4f8Vfuf7Dg4OPHoUQokSHsyfP4vq\n1QeQI+dQCpe0ZtKkaQz/LT83Cwdzce0d3Mc9wb5DLHZd8mA36HcMv3Jhd662eHTIm2JLnyuKkjp0\nqa3kzZuXVatW4ejoCMBff/1F06ZNUz0vsKoBpCEvLy8aN27P7dtDuXMnC19/3Qhv78uUKLGWXr06\nM3t2DyaNuUXwvzBl/Jc8WO3ADzMN2H7zAUe+uYe/9XRif5hAtWra1iBdPzvZi9IpiqITPz8/ypUr\nR9asWWnZsiXh4eE6nZ8tWzacnJzQaDTExMRgYGCAra1tKpU2ARFJd5u2WJ8ed/fGAgvjsk6EC1QU\nS0trefDggYiIxMbGypUr3lK58n/i7HxYLl7cKU8jI2XY1atit3WfzB5wTA4YrZUdjnukTPZQGTxY\n5Pnz93/u9u3bxdQ0d9xnLxRT09yyffv2VL5bRUlZ6fW5EBERIQ4ODjJlyhSJjo6W1atXi7GxsQwf\nPlyuXbsm2bJlE0tLy0S3ZcuWvXatbNmyiZGRkTg4OMjly5eT/Mykvou4/cl+1qomIL3JBHTH0nI8\nLi4ubN26lfz585M/vxv79gl9+17Czc2WyZM7M8CjO13dnBnqGMjoWlmZOvIY054ZcXJtbupsc2Sf\nn/E7m4ReX28IwsKIn3iW6AQ0RcmANB8xIzYh0XHVgZRMCfnkyRNevHjBqFGjaNasGcePH9c5wbxO\ndIkWabWRTiP9x0rql/hff/0ltra2cuTIkdeO/+efGLGyCpOxY7tJQEBDCQ09K75PnkjZZcukweIN\ncqzLGdlnfUCuT78uMZExSX7u6zUPEVgoZcu6qFqBkqGk1+fCsmXLxNnZ+bV9rVq1kmHDhn3wNWNj\nY8XMzExOnjyZ6PtJfRfoWAPQ+8M+0UKl03/olLB9+3Zxd28s7u6NX3vgrl+/XqytrWXAgAHy7Nmz\n+P3Hj4vkyxcjvXodkr17c8m5c50k5Nw++a1TJ7Heu1cGz9oh860Wy1KL5eI1yyvJz3zzYV+2bI23\ngoK7e+NUv39F+VDp9bng4+MjefLkeW1flSpV4puAzMzMxNzcPNFt6dKliV4zKipKTE1N5dKlS4m+\nn1IBQHUCp7E6deqwY8caduxY81qTy7fffsvp06e5f/8+RYsW5d9//0VEKFcOjh414OzZL/n11xu8\neGGP/4OGtKx6hfVTJzAx8godR+ZkqaEJYd3C2dVtF9u2eOHu3iy+w1e7UJ02F4G7+0bWrVtEjhzW\nevwWFOXTkRIpIXft2oW/vz8xMTE8e/aMfv36UaRIEQqm9hIwukSLtNpIp5E+rfz3339Srlw5qVq1\nqhw9elRERKKiRBo1ChRT01vi7t5Ndu2oJvtXauQnDw/RNAoW1h2QPNXWyDyzhTJZs1xyEyywMsmm\nHdUxrGQ06fm5cOzYMSlbtqxYWFhIixYtpGXLljJ8+PBkn79q1SopWrSomJubi42NjbRs2VKuXbuW\n5PFJfRfoWANQM4HTqZiYGObNm8fYsWOxs7OjWrVqTJu2kPDwVUBFjI2HMMB1E183Cua0phTT1kzg\nzHexZA46QcNfi9IiKifTKMgejuPuPu+1LGQvJbkKqaKkQxlpJnBqU0tBfCaio6PZvHkznTp149Gj\ncKAX0AOwIHv2w0x+9h2Rbg5Ydg3ixMkoVmQbQGAeFwoMLcmo4CD+w5K9RX04fa67nu9EUT6OCgCv\nqADwmdHmHHYG7gBLgRxYWIzDwrQGqx7V4p+qljQZmh1DQ2+mbc/Hhi//wHx+KMO33MPcpDwRg0rR\nd5SJnu9CUT6cCgCvqLWAPjPalJNTgLLAeExMblOt2lxCwkZQI3YXt09Wxi7POJyd9/F9nWgWR7bG\nscNldm4vSPU+2akw5zhPDz/V920oipKOqBpABpJYm310dDRLFvsysJsZz7iHg9M4mjevSvXqRphm\n+R9HTJqwxsCThcGOPOhxlQJ/FMCmnY1+b0RRPoCqAbyimoCU10QFnGdsxQ1Mz/QDVauv4cyZERga\ngoeHCeVqhLPA9mfGGX+DScsgbDrakG9YvtSdYagoKUwFgFdUAFDeNnYsR7fep+2jKZQtCx06nGDD\nhvmsWPEPefKGk9u9NE0bzaZ8nxjMy5jzuENhKldVrYBKxqACwCuqD0B528CBOIfuwW/wSmxsNHh6\nlsfF5X/cvPmAX0bMI9r/FgPqV6B/8V+5cD4Y39oBzPgjWt+lVhRFT1QN4FPj6wsNGsDp0/hezUGX\nLmBjAzNmgJOTsOfwRKZN/oUdXhqqWtbE5VZzTPq2ZPDETCrHgJKuqRrAK6oG8In74PX7K1aE1q2h\nb18qVoRjx6BWLe3uP/7QUK3CIBbMP8VfKwtg2eAMM7IMYt2kOvRqeQr1/5aipIz27dszfPhwfRfj\nvVQASIe8vLxo1MiTnTsbsHNnAxo18tQtCIwZA//9B1u2YGwMAwfCkSOwaxeULQsnjxWmXe0T1G7/\nNdNXReH8jQ3/rqxBjQrduX//furdmKJ8JnRNCZnQ6NGjMTAwwNvbO4VL9TYVANKh19fv9yQsbEL8\n8M9kMTODefOgWzd4/BiAAgVg+3YYPRo8PaFtGxPq5/6Lp44zqf3jbpbMbEWuCw8pWrgo48ePJyoq\nKlXuTVE+Fx/SXHXlyhVWr15Nnjx5UqFEb1MB4FPl5gaNGkGfPvG7NBpo3BjOnYOCBaFMGXi4rhnP\n7HZxs+hBvl/5kL8sJrB7424qVapEQECAHm9AUTKOj00J+VLv3r2ZMGECxsbGKVzCxKkAkA5pZ/0O\nAhYBizA1HUT//l11v9D48XD4MKxf/9ruLFm0rUSHD8P+/TCufmnu3NrDRvNc2MwfzYjoFnSo24Ga\nNWsyYcIEYmJiUuS+FOVTFBkZScOGDfH09OTx48c0a9aMNWvWoNFouH79OpaWlmTPnj3Rbfny5fHX\nWbVqFZkzZ6ZevXppVnY1CiidSrGVOg8cgGbNICAAcuRI9JCtW6FvX7B2jMK0/RyG5R6O8f96k7Vt\nU/pO+4Hw8DAWLVpE4cKFP/R2FOWjvW8UkI/GJ0U+x1VcdTp+3759tGrVips3b8bvc3FxoVatWowe\nPTpZ1wgJCaF8+fLs2rULBwcHnJycmDdvHjVr1kz0+JQaBaRyAqdTderUSZnlmatW1Y4K6tULVqxI\n9JD69aF2bZg2zZhx33fnuzo1mN2+CZFbgylmtIbCLZbg4uLCzJkzadKkSbI/Wi03raQlXR/cKeXW\nrVvkzZv3tX358uXTqQ9g5MiRtG3bFgcHh/h9afEjWDUBfQ7GjoVTp2DlyiQPMTGBAQPg3BkDKmkK\n07izH7stbWhUrTGBu5uxefN2+vXrx7Bhw4iNjX3vR370SCZFySBsbW1f+/UPEBwcHN8EZG5ujoWF\nRaLbsmXLAPD29mbatGnY2tpia2vL9evXad68Ob///nvqFl6X7DFptZGOM/9kWIcPi+TKJXL7drIO\nX7YzTEwKhkipkidl3qi6MrD5ebl1665Uq1ZNvv76a3ny5Mk7z08sEb3KOax8jPT6XIiMjBQHBweZ\nOnWqREZGypo1a8TY2FinjGAPHz6Uu3fvyt27d+XOnTtib28vq1evltDQ0ESPT+q7QOUE1q/NFzdz\n5t6Z9DdjsVIl6NoVOnQgOTO+WtbOzJkThtyoZUHfP1YRHL6Dv0YEsXOnto2yUqVKXLhwIQ0Krijp\nm7GxMWvXrmXhwoVYW1uzcuVKnZpKAaysrMiVKxe5cuUid+7cGBoakj17dszMzFKp1FqqEziF9ffq\nz/oL63ke+Rw3JzdqOtakplNN8mfPr//VN6OitH0CbdtC797JOuV6eDg1vM9iM9KU4OuZmDT6IS27\nODNnzhyGDx/Ohg0bqFSp0lvnvWwC0s5nAFPTQaxbt0j1AygfTC0F8YpaDTSdC3oSxJ7APXgHeeMd\n6I2RgRE1nWri5uhGTaea2GW100/BLl2CKlXAxweKF0/WKbcjInDz9+ebOU9YvKE4tVzD+XtOQQ4c\n2EyHDh1Yvnw5tWrVeus81QmspCQVAF5RASADEREuPbqEd6A2GOwJ2kP2zNmp6aStHbg6upLLLFfa\nFWjePJg2TbtwXKZMyTrlVkQErv7+DFl/iw3+Tzh8ohmzZptjabmXZs2aMXv2bBo2bPje66igoHwo\nFQBeUQEgA4uVWALuBrAnaA/egd7sC96HQzaH+IBQPV91LDNbpl4BRKBpU3B0hEmTkn3azYgIXP38\n+HX5Q+5kX8jv82bi4WGFp6cfzZrVZ+LEibRr1y7J81WzkPIxVAB4RQWAT0h0bDQnbp9g99Xd7Ana\nw6EbhyiWo1h8QHCxd8HMJIU7gx4+hNKlYcECcHdP9mk3wsNx9fNjwqwXmFUdwpRV0wkMqsjYsYH0\n61eDIUOG0LNnz0TP1Sa2b4B2jSOARbi7b2THjjUffz/KJ08FgFfURLBPiJGBERXzVqRi3ooMqTaE\niOgIjtw8gnegN2P2jcHvth/lbMvFB4RKeSuRySh5TTdJsraGhQuhfXvw84OcOZN1ml3mzHiXLUvN\nLn78OmYCPb7pyfG7XejVqxsdO57kt9/KkjlzZjp27Phx5VMUJdWlSg1Ao9GMBDoDL9cW/llEtsW9\nNwToCMQAfURkRyLnf1Y1gPd5Hvmcg9cP4h3oze7A3Zx/cJ7KdpXjA0I523IYGXxgLB80SDtJbMsW\nvHbuTFb7vJeXF2PnLONk88bMGGNCpo5jMMhXhsmTpyMSxuXLzkyd+gstWrR46zzVBKR8KFUDeCVd\nNwFpNJpfgBAR+fON/V8ASwFnIC+wCygsIrFvHKcCwDs8CX/CvuB98Z3K155eo1q+avFDTkvmLomB\nJplTPKKiwNWVi0WLUmbZlvc+nF97iOcxwuxXUxaMMCey1+84VTVj9erVrFoVw4sXdVi4sD/ffPPN\nW+erTmDlQ6gA8EpGCAChIjLpjf1DgFgRmRD3ejswUkQOv3GcCgA6uPf8Hj5BPvEB4XH4Y1wdXeMD\nQmHrwu+eg3D9Oo8LFuKbyP4cZFzczsTb599qx3f4F5tRlswebM6DPn9S0iOcc+c20aePhtjYbqxd\n+12iQ0QTUkFBSQ4VAF7JCCkhv9doNCc1Gs08jUbzckhLHuBGgmNuoK0JKB8hl1kumhdvzsyvZ3Lx\n+4v4dfPjm8Lf4HvLF/d/3LGbbEfbdW2Z7zefoCdBb1/A3p4/vyjHUmZizQPdPvxaNI67NjJ8siG2\nf/wID8wpVaoBO3bEYmr6N998E8CBA4eSPF2tGaR8ij75lJAajWanRqMJSGRrAMwAnIAywG3gXWMN\nVUhPYXZZ7WhXuh0Lvl1AcN9g9rXfR3WH6uy4soNKcyuRf2p+Om/szNKApdwOuQ1A1fG/sMYonEXU\nQsOCJHMQJJarYGTbxsz+qhSjxxsS0q43PDMjU6aG+PtDwYJtqV37CadPX0m0rB+d/UxR0iFdU0IG\nBQVhYGDw2kJx48aNe/+JH+mDRwGJSLLGDmo0mrnApriXNwH7BG/bxe17y8iRI+P/dnV1xdXV9UOK\n+dnTaDQUsCpAAasCdCnfBRHh7P2zeAd6s+rsKnpv7Y2NuQ01nWpiPf8HHPvMYFbOP3GYnnjnbJ06\ndVi3blGCJptXx01oXpJfHwcwqMX3WK2bzr17TfD1XUuNGgVxdn5MQMBDCha0TtP7VxR9+ZDmqmfP\nnukUOHx8fPDx8dH5c15KrT4AWxG5Hff3j4CziLRO0AlckVedwAXfbPBXfQBpJyY2Bv87/tr+gyBv\ngk/tZ/+MCFaMaIzjt55Uc6iGRSaLZF9v+8OH/DP+DF03asi+bDIYRfLFF2uoXNmXs2fzcvJkHgoW\nzBx/vBoZpCRXeu4D8PPzo1OnTly+fJn69euj0WgoWLAgY8aMSdb5QUFB5M+fn6ioKAwNDd97fEr1\nAaTWcs6LgVPASWA9kDvBez8Dl4HzQJ0kzk9spVMlDURER0jAP5PkmbWFNP3zSzEbZyaV51aWobuH\nyu6ru+VF5Iv3XmPl3bvSt+1e2VvxP/He1VACAhpKZGS4lCr1j2TJck9On4557fjt27eLu3tjcXdv\nLNu3b0+tW1MyuPT6XIiIiBAHBweZMmWKREdHy+rVq+OXg7527Zpky5ZNLC0tE92WLVsmIiKBgYGi\n0Wgkb968YmdnJx06dJAHDx4k+ZlJfRfouBy0mgmsJO6332DDBjb//BMj/5nCo2z3yVzMgGsR16iY\nt2L8HATnPM4YG76dwHrerVvc7nAZE98sOK8ZQW4bKxwdZ1G69J/cvt0Xb28LKlbUw30pGdZ7U0L6\npMxqu66uuj17UiIl5PPnz7lw4QJlypThwYMH9OrVi5CQELZv357o8el6GOjHUgEgHRDhbtWqbDri\nR5eYGYC2eWbJqr/JXDhz/KS0K4+u4OLgEj/ktIxNGQwNtFXYyVeuYdI8iOvXs/L12r7kylWM7NlH\nU7r0UJ4/n8ru3aZUqKDPm1QygpfDhHfuXJsum4CWL1/On3/+ia+vb/y+1q1bU6BAgWQ3Ab3p7t27\n2NraEhISkmhOgIwwDFTJyDQaumWyxiXGko5E83KEzsyp/1K/UH3+8PgDv25+BP4QSJdyXdgfsJ8a\nU93IPMIUl2kuTD8yHQ+LEO7Pz0OJLM/4t81fPHp0nGfPfmf37h+Brnh4RHL8uL5vVEnPEg4TTq9S\nIiVkUpKTfvVjqACgJOmFkTGN+J7xDMYZ30SPsc5ijdk1M3b18+X5xOlET/2do4tOs/XEVhosb8Df\nO134a9Q5aofeZ9YPf3D//jZMTdfw778t0Gh6ULduDH5+aXxjSobx+jDh9KlKlSoYGRkxbdo0oqKi\nWLt2LUePHgXA3t6e0NBQQkJCEt1atWoFgK+vLxcuXCA2NpaHDx/Sp08f3NzcsLBI/gCMD6ECgJKk\n/v27cs10Kl1pxWrq4ZR5QKJzA177nzT0B6JOTCNmXRau9LmCb+cjdClpwvSRN2hwPoKx6yw5eeV3\nInOuoFNPa7JnH0a9esLJk2l/f4qSElIiJeTVq1epV68eWbNmpWTJkpiamr63dpASVB+A8k4v21+/\nu3KOb41jsfTzA1PT145JzjLPUbGx9Pv7BHVGhXJ37iFymY9hzlUD9vytwSK6Iy9uj2fTtjCqOVul\n3c0p6d7LJiBr6x+5cWNwuuwD0AfVCaykLRFo3Vr799KlkGCySnLH8r+IiWHY8KO4LY+m6n4zzgbV\nJcZ8MC2b/I7Gpi23T/em8E8dqVu+eNokxlEyBC8vLy5e7EqfPtdUAIijAoCS9sLCwM0N6tWDX355\n7a3kLuiPN7e5AAAgAElEQVT2JCqKSR19qXDBgOpbojl7sTEWFrPx8OhBg2/3s9snL99NnsWhh1s5\nfONw6ifGUdK9Fy8u4edXhapVH6gAEEcFAEU/7tyBSpVg4kR4Y73/5LodFs6/X/niZGGG0y+3CAvr\nwsOHY+nWbQTffHOR8+fN2bEDNEYRHL5xOD6P8onbJ1I+MY6S7rz5Y8LBYRmZMzuRP/9IFQDiqACg\n6I+/vzaN5JYtfOhsrsuPnuNd9Rj+Yk3vNSd4+nQ427Y1w9vblzx5fIiJMWDlSkg4Kz5hYhzvQG/O\nPTjHl3Zfxs9BKJ+n/IcnxlHShTebE52c+jNvXjSagKW4/fCVCgBxVABQ9GvDBujRAw4cgPz5P+gS\n/oGPOV/lJFvt7Rj+7zoePVrEyJG2FClSjlOnJvDFFzB9+mvdDa9J0cQ4Srrw5oCCgQOrY252n82/\nW+IbelgFgDhqIpiiX99+Cz//DHXrwgMdcwjEKeOUnbzri9Ho0g1G9WmJpWUtBg58wfr1y+nQYSP7\n9sHUqUmfb5nZkgZFGjCl7hRO9TjFxe8v0rZUWy48vECzVc3I9Xsumq1qxoyjM7jw4IJ6eGQwtrZX\nqVLFj+PLCnI1zFzfxfkkqRqA8nGGDIE9e8DbG7Jk+aBLbFkbTFTHQHY2K0aPH3px+vRTvv/en5Ur\nj9C6dX4WLoQPWRz0+tPr7Anaw56gPey+upsYidH2H8TVEPJZ5vug8iqpJ2ET0IAB83n8+Bg7F9hx\nz2w/4SG5VRCPo5qAlPRBBDw94fFjWLcOjD6sDX7Fnxdhwi3K/FeMF8+asXBhCPPnX6Vw4R5cuTKK\nI0cyUaTIxxRTuPL4CnsC9+AdpG0yMjcxjw8Gbk5u2JjbfPgHKCnGy8uLuXMn06HDHvaNrsds3za8\nKORKxMWcKgDEUQFAST+iouDrryFfPpg1K+lG+/dYMeAUYVsfk2X8fcIet2DECHuuXXPA2Lg8uXKN\nJSDAjOzZU6bIIsKZ+2fiA8LeoL3YWtjGB4QajjWwMlWT0vTlwoWuhF9+StWvsxCVdQrfNDdl9dxM\nGSYAtG/fHnt7+w9eDO59VB+Akn4YG8Pq1XDsGCTI5Kar5r+XJPMXWQgcbcHgX/5gzJgoTE2vEBX1\nHENDb5o3h+jolCmyRqOhRK4SfF/pe9a1WMf9n+6zuOFi7LPZM/vEbBynOFJ+dnl+2vETWy9tJSQi\nJGU+WHmvsLBA7t9ZxZ8tbhKhmYym4RNG9DHRd7F0omtKSIAXL17Qs2dPcubMiaWlJTVq1Eil0r2i\nagBKyrl7F6pXh65doX//D7pETEQMM0ts5XweU7bdy07vbu4MGBCBs3NVsmXzokiRd3cMp5SomCiO\n3joav+z1sVvHKJmrZPwchMp2lTE1Nn3/hRSdnfNtxN3pPtResgNHVxPKlyjCiumZ03VGsDd16NAB\nOzs7nWoAbdq0ITY2lunTp2NlZYW/vz9ly5ZN9FjVBKSkT9eva4PAoEHQvfsHXWLbGi/u9ohhs7Mx\n1x7sokL539m61ZZx42bTtWsZnJzWMGlSkTRNGxkWFcahG4fih5yeunsK57zO8U1GFfNWTDQxjqKb\n0Mu7OHmmDh06zOd5aBnq+pkwr3gx4NNOCXn+/HkqVarEzZs3MTd//4indJ0S8mM30mnqNyWZLl8W\nyZtXZPHiD77E5kXbZVU2b3FtsVe2busrtWubiqlpZoH1AmGSKZOrXtNHPgt/JlsubpH+Xv2l7Myy\nYvGrhdRdUlcmHpgox24ek+iYaL2VLSNJmA7U+99/5dSfZjKgySAx4IbU/HG3XHr+PP7Y9PpcSImU\nkIsWLZKSJUvKjz/+KDly5JCSJUvKmjVrkvzMpL4LdEwJqfeHfaKFSqf/0IoOzpwRsbERWb36gy9x\n6+BjWW/hI98MPi1DhxaW3LmNxchotmiHHj0VV9c2KVjgj/Pg+QNZfWa19NrSS4r9VUyyj88uDZc3\nlGmHp8npu6clNjZW30VMd7Zv3y6mprkFFkoOpon/FwYyb2o5MTR4IO1zzZdWAadfO/59zwUgRTZd\n7d27V/LkyfPavipVqsjw4cOTfY1x48aJRqORUaNGSVRUlOzdu1fMzc3l3LlzSd7rO/Yn+1mr5s0r\nqeOLL2DrVu1EscyZ4auvdL6EbRVLCs8uSvvu5xnReDxffz2KZct6ExNTGRFDzpz5nthYMEgHQxms\ns1jT5IsmNPlCuw787ZDb8fMPJh+ezIuoF7g5ucU3GeXPnl/nTsJPzcs8EpY0wAs3TrfOR69Bq3Ay\nmsm1wdWY4qjbPA3RU/PQrVu3yJs372v78uXLp1N5TE1NMTY2ZtiwYRgYGFC9enXc3NzYsWMHRYsW\nTekix0sH/+son6yyZWHjRujYUfvfD1CsZW5sf3Ji8FYrNj1oyVdfWZE9e2UyZXLH2rowEyemcJlT\niK2FLa1Ltmbet/O4+sNVDnU6hHt+d/Zd20e1BdVwnOpIhw0d+OfkP9x4dkPfxdUbS57jRR2OlivE\nmH9mExWxjf5li5PTw5qSyWgLTw9SIiVkqVKlgLeDWKr/SNClupBWG6oJ6NNy9KhIrlwf1Ry0zfOC\n/F1kj1T6sZ+4uBhJ/frF5Pp1bSvTrl0pWNY0EBsbK+fvn5e/ff+WpiubivUEayk8vbB039RdVpxe\nIXdD7+q7iGli94oV4q8xkj9xlzp1pouxkY90ztRdyizYJ8VqP5ewsNePT6/PhcjISHFwcJCpU6dK\nZGSkrFmzJr4PILmioqKkYMGCMmbMGImKipIDBw6IhYWFXLhwIdHjk/ou0LEJSI0CUtKGn582j8DU\nqR+0jLSIcLhVAPtvP8ZhpoaBdWrSpk0DvLxKcfr0DyxYcJLWrV1TvtxpIFZiOXX3VPyktH3B+8iX\nLV/8kNNPMjHO3btQqxZXS5Sg1xMnDv7Xnew0YYT7NGY2ykFJ78LMn//6Kel5FNDx48fp0qXLa6OA\nChUqxOjRo5N9jbNnz9K5c2dOnTqFo6Mj48aN49tvv030WDUMVMl4AgK0i/pMnAht2uh8emxkLAc9\n/Nhi/ZzYWluZNWQCOXL05OrVLhgYPGfTplDq10+7oaGpJTo2muO3jrMnaA/egd4cunHo00qMc/Mm\n1KoF333HifqDqVX7GQY0YUxMS/5cWJxHAyty1CsTBQq8flp6DgBpTQUAJWM6exY8PLQZxbp00fn0\n6KfRHKh6jPnlQth/eiI88iEs7BR371qSP/8+rlz5sCQ16VlEdARHbh5h99XdGT8xTnAw1KwJ3btz\np+1PVKjwDHv7gTg8fEzlWgNYXzkH+XY7sWjR26eqAPCKCgBKxnX5srYm0L49DBum89pBEbcj2Fxk\nF3PaZiZgWz+K5b/G4cN/EhX1Ldu2WVGzZuoUO71ImBhnd+Buzj84T2W7yvEBoZxtufSZGOfMGahf\nHwYMIKzz97i6RpMjxwT8jk1lduwiei2xILTLlxzebUShQm+frgLAKyoAKBnbnTvaPgEXF22/QMLU\nX8mwY+4OQnsa81vv51xc0IImjSIoVGg/f/1VmRMnIHfuVCp3OpRYYpzq+arHB4QSuUroPzHOvn3Q\nrBlMnoy0as1338GjR0c5daomP+f6jdtfVYQmeci8xY7hwxO/hAoAr6gAoGR8z55Bw4ZgbQ1LlkAm\n3Zox1o7dg8EYQ37qGsC9f/qybKkHBw9u5uhRDdu3p4/5Afpw7/k9fIJ84gPC4/DHuDq6xs9BKGxd\nOG3nIKxeDT17wrJlUKsWY8fChg1h5MxpS/YId7rc6U/nWcKZKpXI9I5/NBUAXlFLQSifhvBwkaZN\nRdzcRJ480fl0/xn3ZW3mfeLQ5nspVDiLnDz5k1SrJjJ2bCqUNYO69uSaLPRbKJ7rPMX+T3vJMymP\ntFnbRuafmC9Bj4N0ulbCpRuStRTHtGkiefKI+PmJiMiqVSL29iLjxtUTezsr2WGzQ5rNPSxL7tx5\n76XUc+GVpL4L1FIQSoYTHS3y/fcixYpp1xHS0b6ht2V5ln1i9WU1+bZhVvH1XSy5c4scOJAKZc3g\nYmNj5fLDyzL72GxpsaqF5Po9l+Sfml86begk/576V249u5XkuQmXboCFYmqaO+kgEB0tMmCASOHC\nIlevioh2OkiOHCLr1++SHDkM5Z8mC2R9q+NS6dgxiUnGUhnqufBKSgUA1QSkpB8zZsCoUbB8Obi6\n6nTqtg7XebjtHF0NPfmpSxiFC+3g558r4u9PiiWR+RSJCGfvn9U2F8UlxrExt4nvP3B1dI1PjPNm\nwnZYhLv7RnbsWPP6RZ89g9atITRU2/yTIwc3bsCXX8K0aTHMnGmNg3VlPL2H0X6BsLpaacpaWLy3\nrKoJ6BWVEEb59PTooe0LaNEC5szR6dR6C+yp0qkg47P8xvgpMaCpS/36d+ncWbt0nJI4jUZD8VzF\nX0uM80+jf8iXLR9zTszBcYoj5WaVY8COAdzPfgdMwt59wcuXtU96e3vYuRNy5OD5c2jQAPr0gbt3\nu3PpUhTtLw7lwA9Z8HDMjVPM+x/+SupQNQAl/bl4Eb75RjtK6I8/kp1nWES42OMik3fOZwX/sGS6\nIT8PuUyNGpc5f34EAP37d03TPAIZXWRMJL43fdkTuIfVJ1Zz6kEA3C0AgcUwuXmAVZMX0qBeA+3B\nu3drf/mPHKkN5kBsLDRpoq2FDRp0gi+/rMD83lOw2l2Z7yZG0mx7JSIfG/G//72/LBmpBpBRUkLq\nvb0/sQ3V1qc8eiTi4SFSo4bIraTbpd8UGx0rAc0DpJptXSnkUVzmLagqGs2DuDwC72m3Vt5rw9YN\nUr5pVXHqVESK/VFMzMaZidsCV/HqXU8iclpJ1M4drx0/aJBI9eoiISERUqpUDhnUv6rsz3FA6iw7\nLFMC7oiVlciNG8n77Iz0XGjfvr1OawEtWbJEzM3N47csWbKIRqOREydOJHp8Ut8Fqg9A+WTExMC4\ncTBzprZpKJkzvGIjYzn67VGaHWiDY1tzHMPcWbJoPCIGJNlurXyQkDvXeN62JZHBV+npac0+g+tU\ndahKTaeahPo2Zclf+ThyRMPw4c0ICNjM3057OWdmwsxukGtCaYoW0SQ7jXRGqgF8SErIhBYtWsTY\nsWO5dOlSou+rPgDl02doCCNGwKJF8N13MHastk3hPQxMDKiwrgK9jf/g+OwgpNheSpXZC7z/XEUH\nvr5YVK6BzRfOOAQEs3noGQJ/CKRzuc6sW/OQUUPNCKpdiuqDi7Bm7VqGt/uZZ4di6dcklK5hhdi/\nT8NPP+n7JlKGn58f5cqVI2vWrLRs2ZLw8PCPut7ChQtp165dCpXuHXSpLiTcgGbAGSAGKPfGe0OA\nS8B5wCPB/vJAQNx7U99x7WRXnZTPxI0bIi4uInXqiNy+naxT7l+Plu5Gf4iFcQ4ZOLeyWGa/IcbG\nXZJsAkpsjLvO494/B7GxIpMni+TMKZJI2sL58/cK3BXwErKMkMxZkCHjrGW9+Xop9ds0KbblTynQ\nfLaMn3VZp0xp6fW5kBIpIRMKCgoSQ0NDCQoKSvIzk/ouSKt5AEBRoDCwJ2EAAL4A/AFjwBG4zKvO\nZl+gYtzfW4G6SVw7ed+88nmJjBT5+WeR3LkTffCIvP3AvnY+SuoaeEq+bKVk8NRvJWu2p3LlSuLn\nvTnGfezYsckf9/65uHlTpH59kQoVJLEv8skTETOzawKHBCLExKS0tG9vIgsKT5KN/QIk/3/7Zcbx\nBVJvThux/cNWHCY7iOc6T1nkv0iuP73+zo9+33NBO97r4zddpURKyIRGjx4tbm5u7zxG7wEg/gJv\nB4AhwKAEr7cDXwK2wLkE+1sCM5O4pk5fmPKZOXhQpEABkbZtRR4/jt+d1ESl04deSFFNBXEv3lra\n9hopJUrflE2bvF4LFO7ujePOe/kgWChWVgXe2ufu3liPN65HsbEi//6rTezzyy/aYBzn5fdXq1ZT\nKVfuvtjbb4n73vpKsWLWMqJJO1liuVJsfPbL0adPE1xSmxjnf77/SzQxzr3Qe68VIb0+F5YtWybO\nzs6v7WvVqpUMGzbsg65XsGBBWbhw4TuPSakAkBpLBuYBDid4fQPIC0TF/f3Szbj9iqKbKlXA3x8G\nDoTSpWH+fKhVKz7H7MuJSmFh2ryzO3bUYerKdbRtWZESjvnIZHqaPoOuEnhWO3zxwAHPVM27muHd\nv68d1nn+vDbPc/ny8W95eXnRqJFn3PdeEQODU/zyiz9jx07AyAh6ds2Kff9G/D0vK73y2VEha9b4\nczUaDUVyFKFIjiL0dO5JrMQScDcA70BvFp9cTJdNXV5LjJNeJZUSsmDBgly/fp1ixYolufbS7Nmz\nadWqVfzrgwcPcvv2bZo2bZqqZX7pnQFAo9HsBGwSeetnEdmUOkVSlGQwN4e//4bt27XLSteqRbbI\niCQP92hqx+ZdG/Bw96BpuWjW3f+BIpWvcuFQNcLCAOZgajoo7m8wNR1Ev37fM27c6/v6909kofpP\nlYh2Abf+/aFdO+1IrMyZXzvkzaAbG+vH5s07yZJF+PXXcHLMG8iVHhY8K2XJYAeHd36cgcaA0jal\nKW1Tmh8r/xifGMc70JvpvtNT6y4/WpUqVTAyMmLatGn06NGDTZs2cfToUWrVqoW9vT2hoaHJvtai\nRYto2rQpZmZpk/DnnQFARNw/4Jo3AfsEr+3Q/vK/Gfd3wv2vh80ERiYYG+bq6oqrjksDKJ+JunW1\nSWZ++YUlGzaQw3g/s6JiEQzeemA7uzrz9+y/+b5TH2oW+4JdF+pSpNYeLuyGHDlys27dcCZNmg1A\n//6LqFOnDhUqVHhr32fh7Fno3RuePIH166FSpXccnCfB3/c4deowEycWpYqpG88t69Gr+XMOFS2K\nkY7LsxoZGFHJrhKV7CoxpNoQNO3ScAVTHRgbG7N27Vq6dOnCsGHDqF+/Pk2aNNH5OuHh4axatYq1\na9cm+xwfHx98fHx0/qx4urQXJbah7QMon+D1y05gE8AJuMKrTuAjQCVAg+oEVlLaiRPyuEgRCbC0\nli5f1kqyw7ZVk1ZSgIJSw8lbSpTeJ8WauXxWnbvvHNkUEiIycKB21bbp07WLur3DrFn7Be4LbBWY\nLRqNkXToUEkO7y4vB+z2So2tR2TWzZuydKnIxIkfV271XHglqe+CNBwF1Ai4DoQBd4BtCd77Ge3o\nn/NAnQT7Xw4DvQxMe8e1U/CrUj4r0dEiM2ZoH2CdO2tHrbwhJiZGKjtXluqGHlIk53lp2uZ3+Sdw\nT9qXVQ+SXNEzOlpk0SLtWs1t2iRrqO29eyL584v0739KatduJDY29uLq6iz7fKxkf+ml8tvyM/L1\nqVNy716s5M4tcuTIx5VdPRde0XsASM1N/UMrH+3RI5GffhKxshIZOlQkwegTEZFnz56JXd7i8rXB\naMlh/lSGTWguf17ao9O49Izo7dFOC2RYmcoiJUqIVKkisn9/sq4THi5StarIkCHa1+PGjZPy5cvJ\n/r1l5WD3QbLx13OS77//5F5EhLRpI9K378eXXT0XXkmpAKBmAiufpuzZYeJE8POD69ehcGGYMgWe\nPwfAwsKCffs3cSDrDMqFefG/UXMxP9Gfwef3EiMZY7mBj1WFg+zjNzpdOqNdcuPAAaha9b3niUDX\nrpArl3Zy9sKFC5kzZw5//VUdgyBz5GlrOlZ7wLoSJTjubcKBA9rjlHRIl2iRVhsq0ispzd9fpFEj\n7Tj2MWO0NQQROXr0qFhY5JByhv5S3OaqrNzhKG38d0voe9q+M6rt27ZJQxNL2UMRCcRaOhtnle1b\ntuh0jbFjRcqXFwkNFdm8ebPkzp1bjh5dLvt25JD/qm+VQnv+k2V37khIiEi+fCJeXilTdvVceCWp\n7wLVBKQo73D2rIinp7ZpaOBAkZs3ZdOmTZItm63YGdyXluV2yAZvR3E9skVuhIfru7QpJzJSZPFi\nkRIl5JmTk/xawlnq1mqoc+f30qUiDg7aBVoPHTokOXPmlP37t8kBbzvZ99Wv0nDLMRkUl9Xt8WOR\nuXNT7hbUc+GVlAoAajVQ5fMUHKzNNbBkCbi5McvOjrHrDhEdcpDvKy2g/KDf+CnTZOaXrPXa5KUM\nJzhYO1Fu/nwoVAgGDQIPD/iApPD792vX9t+9G4yNz+Pq6srcuXOwzTaJ56sd2FthIIdKx7KpZEkM\nUyHpfEZaDTS1qdVAFeVj5MsH06fDtWtQty7dDh6kXcglrDM15s//OnJ3ememhP1A21M7WX3vnr5L\nq5uICFi1CurU0c7affwYNm8Gb2/tvkQezl5eXnh4NMHDowleXl5vvX/xIjRrpo2XVlY3qVu3LuPH\nj6egzW5C/cK4WWIw/xR+ztJixVLl4f+SRqNRW0p+v7pUF9JqQ1X1FD2I9fWVtgULSknDjpLf5KZs\nafqj+GzJI2UPrJLhV69KdHoeIfTihcj69SLt2mmbt9zctGv3vHjx3lPfl+z9/n2RggVF5swRuXHj\nhhQqVEgmTpwoN84vEZ+VeWTD5P8kz8GDcuH589S8QyUZUE1AivLhoqKiaNm8Of6HW5KfEoz4cgKx\n323iL4OxPLUqzlJnZ3Kk0TT99woKgr17YdMmbf7dChWgUSNo2BDs7N57+kvvSvYeHg61akH16tCr\n1w3c3Nzo0qULXdu443+8JuFn59K+qg27SpemhLk50dHJzuCppAJdm4DUP5WiJGBsbMyyFSto1Kg5\nR4+OpeX+OSzO+jO9Gw7H16cF5e/dY+W2bVQqUgScnbWL0eXIkfoFi4rStsMcPqx96O/dq23qqVED\nvv4aZs0Ca+sU/ciYGG2KX3t76NFD+/Dv2rUrfbp25MjOssRcGUg7l9xsK1WKEubmrF2rbSLSYSUD\nRc9UDUBREhEeHk69em3x9f0f5mbWTGs8j9z1BvEiZhLtLR355dQpeq5Zg+bUKciaVRsISpUCJyft\nE/PlpksHckQE3L4Nt25pt8uX4fRpCAjQPvzt7bW/8l1dtQ/+woU/qDP3Ta+v6Kld9G7t2kWsXVuH\nwECYMeM6deu60aNHD3p36IrvBjdinheleenurCtVEpds2bh6Fb78UtvVULHiRxdJ+UC61gBUAFCU\nJLx48YIaNX4gIOBPChQwo63zVr5s2A7TOyPpVu5LnLJkYXbBguS8dQtOntQ+qIODtRPPrl3T/tfA\nACwsIEsWMDPTbiYm2od9RASEh2u3Z88gJARsbCBPHu3m6AglS2q3YsW010glXl5eCRa968qBA3XY\nvh0WLAimYcNa9OzZk54tu3F0TX1isprRwmkwS0uWpFb27Dx6BNWqaVeM7t07+Z/x2Sysl4ZUAFCU\nFBQaGkrFiuMJCupHtWoW2GXxp137r8l8ohMbv+3AgvD7zClSBKNjx95+uIloH+zPn7/aXrzQPvgz\nZdIurfxyMzfXNiXpuGJmapg+XbvNmHGKdu3qMWTIEDp4tOf4yhZEFn1G29yjWVy8NO5WVoSHg7u7\n9lf/pElJXzOxWsa6dZ/R6qppRAUARUlhL168wNl5McHBX7F4sRV1a93lxL7ayMGKxDj/SgurYB5u\n3UXMX0UgXDA1HcTQod+zd+8JIGP92l2xQrv8/9ixPvTt2xBHx6KUzFyajlUeE+52kT7Wf7K2VEVK\nmpsDMHMm+PjA0qXvjl3v6mhWUo6aB6AoKSxLliz4+3fCySkAT88rPArJRqW6xzGtcw/N7ZYM6n0U\n45jKMKcIlG1AWFgbRoyYxM6dDdi5swGNGnkmOrY+vdmyBfr0gU6d1tCvX2PCwgwwPdmPVvljuFfl\nAIPNRrK3vEv8wx+gWzf45590UXFRPoCqAShKMsXECKVK+XPt2kNOnSpMvnx5uHzuJwID/uXJ4F8Z\nY+HBmaE3IeAwzMgBj9vGnZn+f+1u3Qrt2wtffz2Lffv+wDZXQYof6s5XLXcS0WY93c+Op7TPZnZv\nWfFB11dNQGlD1QAUJZUYGmrw8yuLrW1hSpXyxs/vFIWLT+ZORDfMp/TmV4tF9GvzgKw3bsH8PNDw\nJhjo/4fM+2b5bt0Knp6xFC48gPPnF+O9wptu59pS/cfFPOywnR47V/BwcCyGUdEfXIY6deqwbp02\nELq7b1QP/3RC1QAURUdPn0Lp0k+4d28xU6aYEhTUmYMHfej/YwOe/+eM9ez+zMnkxdpB9Yk112Cy\n6A82jBhAXT088N73y3vbNvjuuygyZWpO69b56e/SnyvfX+DWr2O4a3OboSP7E3rYOH5oqItLHSws\n0vw2lGRSncCKkgYePIDq1cO5d285X33lg6vr34wZ84IpUzqRO3swz0f15f4RM5bXvcrRHs4UzJ6d\ncfnz45It22vXSe2hke/qfN22TWjePAwjoybM/qMjxfcU557vDQKn/IyhlRUWT79nxqSFAPz4Y1d2\n767DtWuwcmWKFlFJQWomsKKkgRw54MCBzNSu3ZYjR/Lg51eZmTNXM2LEeqpXn83XI36izJNB2Pep\nRpbHWTjby4LW4WcpbmbGGCcnyltYvPXr/MABzzRrGpk79ym9ekF+px+Z02AosYPAu91FLGYNJK91\nVdxLzkGjMaRRnW+IjobOnbVz0TZvTvWiKWlI1QAU5SM8eQJ16womJmc5d64mI0b8wtOn3ahZ8xIm\nJq3JZGxHNr8R3BoTjXFeE051N2eE4wPyZc7M08UrOf2/YhCdekMj3wwymTINxNl5Nv/9V4Hmbv/Q\nJbAKNx2N2d1nG99l/xunfIMp4NA/fsXJFy+gVSuIjITVq7Xz2JT0SzUBKUoae/YM6teH3Lkfc/du\nI6Kiwpg1axalSn1BcPCv3Lz5F3Z5f8TkYBtujLuLgbkBd1tY0D3LUW7ktoEtBWCLLdxbkSqjhV42\nM4WEhHLhQnvCQiswzjaAAplyMavHM9q5zCKfXKDEF8uxsCgTf96TJ+Diol3yaPZs7QRmJX1TAUBR\n9CA0FBo3hthY4Ztv/uXXX/vTpk0bRo0ahaHhPS5f7seLF2cokH8qmqOVuLPwDve23mOvgT9baxbm\neHGeCN8AAAxfSURBVLvMyMMg2jrlY8CXX1LCzCzF1n1/9OgRowb+wd8LqmOLEz0r3mNvKwNq1r5B\nlaeDyGldnwIF/sDQ8PWlJkS0SWCqV0+RYihpQAUARdGT6GgYMkS7Gua8eY9YsOBHdu/ezYABA+jc\nuTMLFuzniy9+IGfOwuTLN5QsMRXwGe7D/SWPyBZmQXjlTFyvbc3yoi+4Z6fBw8qKchYWlDU3p6SZ\nGZkNDZNdlpiIGHw3nGPyhMls9T+BQewqbO2MqTsrlMblI8nx8A+ePvGmcOFZ5MjRIBW/FSUtqQCg\nKHq2dCn88AP8/Tc4OR1jwoQJ+Pj44OHRnRMnutGw4Qbq1PmDbNnscXAYhJVVfSKuR/DY+zFPvJ/w\nePdjogzgcSljrtvCuZxR+OWMIlP+zOTKmRlLQyOyGxuTzdgQSwMjuB9N1NVwJDACo6Aowi7c5KD/\nUnbE7MDW9jfuPOtC28GRTOj7mLs3f+XevVXY2fXBzu5HjIzUmM5PiQoAipIO+Plpm4SaN4fRo+Ha\ntUtMmjSJlStXUrx4U65caUiDBg9o0eJPzM1jyJv3B3LmbISxsTUiQtjFMEL8QggPDCfsShgvroYR\ncjmMmGfRaNM+vczmB5HWBty3DeWQHODQ3V0E3TvPt4278/jRSK5fN2Xu3EtYWc3izp352Nh0xMFh\nMCYmr3IY7NkDQ4dql3QoUEB/35ny8VQAUJR04sED6NIFzpyBadOgbl24c+cOixYtYuPGTfj7n8bJ\nqRY9ejhRpMhZTEwOkDWrM/fvF2XGjEBCQ02TnBvw5MkTTp06hb+/P5s3b8bX15d69erRokULjI3r\n062bhrp1j+Pp2RMjowf8v727D9KqLOM4/v3tAgGKLEkoIgkmTosCI1JSQ4IpQpogf5Q6jlFqxuAU\nBZMCNco/WjMMKU3qZLyoY9oUoqFtJKVrMWlkohLvlC/gCyIqCoH5cvXHOeiGy/I8z7J7zsP5fWZ2\n5pz7nGfPxfDsfZ1zv5y7V68L6NNnCp07f7hS2KpVMH06rFsH11+frPnrd/pUNycAs5xpaEhesjZ4\nMNxwQ7IePcC2bdtoaGhgyZIlLF++nJ07d9Kr12F06rSd+voaamu7sn37Lnr3rqdHj+OB7mzZ8ipP\nP72aV1/dzsknD2TIkEGcfvpgRozoxcqVu5k9exjPPVfH1KlXcd55n6BXrws54ojPIX1Ys69cmTRR\nrV+f3PlPmuQRPocKJwCzHNqzB2bPhrlzk0VTrrgiWfOlqbfeeouzzz6fxx6rp6bmGLp330CPHpvo\n338Tgwf3IOIVevT4DyecUEPv3pD0CQfPPz+K+fOvYe3aeqZOXcvll3ekrm4INTXNz/PcvBkefxzO\nPdcV/6HGCcAsx555JmluWbQoGWN/6aXJkr57K+KPvrphMx061HHccd0YMQJuuimZjLVtW/L65sWL\nYcWKJKGMH58sHbxjR9LstHo1zJt3UFaNtCrhBGBWBXbtgnvugQULYM2apPIeOBB27vwH1103mbff\nvhIIOneewcKFd3LYYV/kkUeSV1A88EDSfj96NIwbBxMmwNCh0LFjsvpkt27JCpLDhsHFF0MHv/Cl\nMJwAzKrMpk2wdGnSJr9+PTz55G62b6+htvZt3nvvcOrqaujZM6n8Bw1KksUZZyQrSZo15QRg1sba\nY3HzXbuS2cVHHpncwXtBdSuFE4BZG8piZSuvpmWlcgIwa0NZLG7uBdWtVF4S0qwgDrTUo9mBeHyA\nWRmmTbuC5csnsnt3st+ly9VMm3Z7u19z5MhvZ7aYjB063ARkVqYsOmT3veacObe6Wcg+ot36ACR9\nBZgFfBr4TEQ8kZb3A9YC69JTH42IyemxU4HbgM5AQ0RM2c/vdgIwa4H7Baw57bkm8CpgAvDzZo5t\niohTmim/BbgsIlZIapA0NiKWtiIGs0LKoinKDj0VdwJHxLqI2FDq+ZJ6A90iYkVadAdwfqXXNyuy\nMWPGcO+9yV3/6NFL3P5vFWmrTuD+klYCO4AfRsRyoA+wpck5L6RlZlaBMWPGuNK3VmkxAUhaBhzd\nzKGZEXH/fj72ItA3Il6XNBS4T9JJ5QY2a9asD7ZHjRrFqFGjyv0VZoXjGcPF0tjYSGNjY8Wfb/Uo\nIEkPA9P2dgLv7zjwEvBQRNSn5RcBIyNiUjOfcSewWZk8Y9iymgj2wQUl9ZRUm24fDwwA/h0RLwFv\nSjpNkoBLgPsO0vXNCm/OnFvTyn8ikCSCvU8DZs2pOAFImiBpMzAc+J2k36eHRgJPpX0AvwG+FRFv\npMcmA/OAjSQjhTwCyMwsI54IZnaIcBOQ+WVwZgXmTuBicwIwyylXztbWnADMcsjNM9YenADMcsjv\n7rH24PUAzMysJF4PwKwd+OVtlkduAjJrJ+4EtrbmPgAzs4JyH4CZmZXECcDMrKCcAMzMCsoJwMys\noJwAzMwKygnAzKygnADMzArKCcDMrKCcAMzMCsoJwMysoJwAzMwKygnAzKygnADMzArKCcDMrKCc\nAMzMCsoJwMysoJwAzMwKygnAzKygnADMzArKCcDMrKCcAMzMCsoJwMysoJwAzMwKygnAzKygnADM\nzArKCcDMrKCcAMzMCqriBCBptqS1kp6StFhS9ybHZkjaKGmdpLOblJ8qaVV6bG5rgzczs8q15gng\nQeCkiBgCbABmAEgaCFwADATGAjdLUvqZW4DLImIAMEDS2FZcP7caGxuzDqFi1Rw7OP6sOf7qUnEC\niIhlEfF+uvs34Nh0ezxwd0S8ExHPApuA0yT1BrpFxIr0vDuA8yu9fp5V85eommMHx581x19dDlYf\nwKVAQ7p9DLClybEtQJ9myl9Iy83MLAMdWjooaRlwdDOHZkbE/ek5PwD+GxF3tUF8ZmbWRhQRlX9Y\n+jrwTeDMiNiTlk0HiIgfp/tLgWuB54CHI6I+Lb8IGBkRk5r5vZUHZWZWYBGhA5+VaPEJoCVpB+73\nSSrxPU0OLQHukvQTkiaeAcCKiAhJb0o6DVgBXAL8tLX/ADMzq0zFTwCSNgKdgNfSokcjYnJ6bCZJ\nv8C7wJSI+ENafipwG9AFaIiI77QqejMzq1irmoDMzKx65XYmcEsTzfJK0th08ttGSVdnHU85JPWV\n9LCk1ZL+Kakqn84k1UpaKen+rGMpl6Q6SYvS7/0aScOzjqlU6eTP1elEz7skfSzrmFoiaYGkrZJW\nNSn7uKRlkjZIelBSXZYxtmQ/8ZddZ+Y2AbCfiWZ5JakW+BnJ5LeBwEWS6rONqizvAN+LiJOA4cCV\nVRb/XlOANUA1PtrOJWkarQcGA2szjqckkvqRDAYZGhGDgFrgwixjKsFCkr/VpqYDyyLiROBP6X5e\nNRd/2XVmbhNACxPN8uqzwKaIeDYi3gF+RTIpripExMsR8WS6vZOk8jkm26jKI+lY4BxgHlBVAwnS\nu7UvRMQCgIh4NyJ2ZBxWqd4kuYHoKqkD0JVknk9uRcRfgNf3KR4H3J5u306OJ6o2F38ldWZuE8A+\nmk40y6s+wOYm+3snwFWd9I7uFJIvUTW5gWRk2vsHOjGH+gPbJC2U9ISkX0jqmnVQpYiI14A5wPPA\ni8AbEfHHbKOqyFERsTXd3goclWUwrVRSnZlpAkjb21Y183Nek3OqZaJZNTY5fISkw4FFJKO3dmYd\nT6kkfRl4JSJWUmV3/6kOwFDg5ogYCuwi300QH5D0KeC7QD+Sp8bDJV2caVCtFMnomKr8my6nzqx4\nHsDBEBGjWzqeTjQ7BzizXQJqnReAvk32+/L/r77IPUkdgXuAOyPivqzjKdPngXGSzgE6A0dIuiMi\nvpZxXKXaAmyJiL+n+4uokgQADAP+GhHbASQtJvn/+GWmUZVvq6SjI+Ll9N1lr2QdULnKrTNz2wTU\nZKLZ+H0mmuXV4yRvOO0nqRPJG1GXZBxTydI3ts4H1kTEjVnHU66ImBkRfSOiP0kH5ENVVPkTES8D\nmyWdmBadBazOMKRyrAOGS+qSfo/OIumIrzZLgInp9kSgqm6CKqkzczsPoKWJZnkl6UvAjSSjIOZH\nxI8yDqlkkkYAfwae5sNH3xkRsTS7qCojaSQwLSLGZR1LOSQNIenA7gT8C/hGtXQES7qKpNJ8H3gC\nuDwdDJFLku4GRgI9Sdr7rwF+C/wa+CTwLPDViHgjqxhb0kz815KM+imrzsxtAjAzs7aV2yYgMzNr\nW04AZmYF5QRgZlZQTgBmZgXlBGBmVlBOAGZmBeUEYGZWUE4AZmYF9T8vP7LPYZ1wVAAAAABJRU5E\nrkJggg==\n",
"text/plain": [
""
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"plt.plot(xs, g(xs), '--')\n",
"plt.scatter(X, y)\n",
"\n",
"X1 = X.reshape((50,1))\n",
"for degree in range(1, 8):\n",
" h = PolyRegression(degree).fit(X1, y)\n",
" plt.plot(X1, h.predict(X1), label=\"d=%d\" % degree);\n",
" print \"error(h%d) = %.2f\" % (degree, error(h, X, y))\n",
"plt.legend()\n",
"plt.show()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"\n",
"* Model koji odgovara pravoj funkciji koja je generirala podatke je $h_3$, tj. **optimalan hiperparametar je $d=3$**\n",
"\n",
"\n",
"* Modeli $h_1$ i $h_2$ imaju veću pogrešku od $h_3$ i njih sigurno ne bismo uzeli\n",
"\n",
"\n",
"* Međutim, modeli $h_4$ i $h_5$ imaju manju pogrešku od $h_3$\n",
"\n",
"* Očito, što je veći kapacitet modela $\\mathcal{H}$, to je manja pogreška $E(h|\\mathcal{D})$, $h\\in\\mathcal{H}$\n",
"\n",
"\n",
"* Ali model mora moći **generalizirati!**\n",
"\n",
"\n",
"* Preferiramo jednostavne modele\n",
" * bolja generalizacija\n",
" * lakše učenje/uporaba\n",
" * lakše tumačenje\n",
"\n",
"\n",
"* **Occamova britva**\n",
"\n",
"![](\"http://muslimsi.com/wp-content/uploads/2014/12/quote-occam-s-razor-no-more-things-should-be-presumed-to-exist-than-are-absolutely-necessary-i-e-the-william-of-occam-372636-846x398.jpg\")
\n",
"\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"\n",
"* Trebamo odabrati model koji točno odgovara *pravoj složenosti* funkcije koju nastojimo naučiti\n",
"\n",
"\n",
"* Dvije krajnosti:\n",
"\n",
" * **Podnaučenost (engl. underfitting)** - $\\mathcal{H}$ je prejednostavan u odnosu na stvarnu funkciju $\\Rightarrow$ loša klasifikacija na viđenim i neviđenim primjerima\n",
"\n",
" * **Prenaučenost (engl. overfitting)** - $\\mathcal{H}$ je previše složen u odnosu na stvarnu funkciju $\\Rightarrow$ loša klasifikacija na neviđenim primjerima (loša generalizacija)\n",
" \n",
" \n",
"* [Primjer: Podnaučenost/prenaučenost kod klasifikacije]\n",
"\n",
"\n",
"* Drugi pogled: \n",
" * Jednostavan model ima **visoku pristranost** (engl. high bias)\n",
" * Složen model ima **visoku varijancu** (engl. high variance)\n",
" * Odabir modela $\\Rightarrow$ kompromis između pristranosti i varijance (engl. bias-variance tradeoff)\n",
" * Optimalan model minimizira zajednički pristranost i varijancu\n",
"\n",
"\n",
"* **Pretpostavka induktivnog učenja**\n",
" * Ako je **(1)** pogreška hipoteze na dovoljno velikom skupu primjera za učenje mala i **(2)** ako model nije suviše složen, hipoteza će dobro klasificirati i nove, **(3)** slične primjere"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Unakrsna provjera (engl. *cross-validation*)\n",
"\n",
"* Metoda za procjenu sposobnosti generalizacije modela\n",
" * Skup primjera dijelimo na **skup za učenje** i **skup za ispitivanje**\n",
" $$\n",
" \\mathcal{D} = \\mathcal{D}_{\\mathrm{train}} \\cup \\mathcal{D}_{\\mathrm{test}}\n",
" $$\n",
" * Model učimo na skupu za učenje, a ispitujemo na skupu za ispitivanje\n",
" * Primjeri iz skupa za ispitivanje model dosad nije vidio, pa na tom skupu dobivamo dobru (pravednu) procjenu pogreške generalizacije\n",
"\n",
"\n",
"* Računamo dvije pogreške za $h\\in\\mathcal{H}$:\n",
" * **Pogreška učenja** (engl. *train error*): empirijska pogreška hipoteze na skupu za učenje, $P(h|\\mathcal{D}_{\\mathrm{train}})$\n",
" * **Ispitna pogreška** (engl. *test error*): empirijska pogreška hipoteze na skupu za ispitivanje, $P(h|\\mathcal{D}_{\\mathrm{test}})$ \n",
" \n",
"\n",
"* $P(h|\\mathcal{D}_{\\mathrm{train}})$ pada sa složenošću modela, dok $P(h|\\mathcal{D}_{\\mathrm{test}})$ tipično prvo opada a zatim raste\n",
"\n",
"\n",
"* Optimalan model je onaj koji minimizira $P(h|\\mathcal{D}_{\\mathrm{test}})$\n",
"\n",
"\n",
"* [Graf: Pogreške s obzirom na složenost modela]\n",
"\n",
"\n",
"* Što ako želimo optimirati parametre modela?\n",
" \n",
" * Ne možemo to raditi na skupu za provjeru!\n",
" \n",
" * Trebamo još jedan skup: **skup za provjeru** (engl. *validation set*)\n",
" \n",
" * Tročlana particija skupa primjera:\n",
" $$\n",
" \\mathcal{D} = \\mathcal{D}_{\\mathrm{train}} \\cup \\mathcal{D}_{\\mathrm{val}} \\cup \\mathcal{D}_{\\mathrm{test}}\n",
" $$\n",
" $$\n",
" \\mathcal{D}_{\\mathrm{train}} \\cap \\mathcal{D}_{\\mathrm{val}} = \n",
" \\mathcal{D}_{\\mathrm{train}} \\cap \\mathcal{D}_{\\mathrm{test}} =\n",
" \\mathcal{D}_{\\mathrm{val}} \\cap \\mathcal{D}_{\\mathrm{test}} = \\emptyset\n",
" $$\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Primjer: Regresija"
]
},
{
"cell_type": "code",
"execution_count": 25,
"metadata": {
"collapsed": false
},
"outputs": [],
"source": [
"XY = np.column_stack((X1, y))\n",
"np.random.shuffle(XY)"
]
},
{
"cell_type": "code",
"execution_count": 28,
"metadata": {
"collapsed": false
},
"outputs": [],
"source": [
"X_train, y_train = XY[:30,0:1], XY[:30,1]\n",
"X_test, y_test = XY[30:,0:1], XY[30:,1]"
]
},
{
"cell_type": "code",
"execution_count": 29,
"metadata": {
"collapsed": false
},
"outputs": [
{
"data": {
"text/plain": [
"(30, 20)"
]
},
"execution_count": 29,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"len(X_train), len(X_test)"
]
},
{
"cell_type": "code",
"execution_count": 30,
"metadata": {
"collapsed": false
},
"outputs": [
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAYAAAAEACAYAAAC6d6FnAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAH5lJREFUeJzt3Xt8XHW57/HP05sM0M1lo22BclGogOLe0FJABKJlJiAQ\nCXkpB86GHEQjolhg1EqLtrrt0W4Nihw3ngpKUKsgJVAFM0mRAEG5bRDLTUAotBUqGywXCdLSZ/+x\nVso0naQzk5msNbO+79crL2fWrJl5iOnvWev3/C7m7oiISPKMiToAERGJhhKAiEhCKQGIiCSUEoCI\nSEIpAYiIJJQSgIhIQo2r5oeb2UrgZeBNYL27zzSznYGrgT2BlcDH3H1dNeMQEZEtVfsOwIEGdz/I\n3WeGx74E9Lj7NODm8LmIiIyy0egCskHPm4CO8HEHcNIoxCAiIoOMxh3AcjO718w+GR6b5O5rw8dr\ngUlVjkFERAqoag0AOMLdnzWztwM9ZvZo/ovu7mamtShERCJQ1QTg7s+G//u8mXUCM4G1ZjbZ3Z8z\nsynAXwe/T0lBRKQ87j64231IVesCMrNtzWxi+Hg7IAOsAJYBreFprcD1hd7v7jX7M3/+/MhjSGLs\nij/6H8Uf7U+pqnkHMAnoNLOB7/mZu3eb2b3ANWZ2FuEw0CrGICIiQ6haAnD3p4B/LXD8ReCYan2v\niIgURzOBq6ChoSHqEMpWy7GD4o+a4q8tVk6/UbWZmccxLhGRODMzPA5FYBERiTclABGRhFICEBFJ\nKCUAEZGEUgIQEUkoJQARkYRSAhARSSglABGRhFICEBFJKCUAEZGEUgIQEUkoJQARkYRSAhARSSgl\nABGRhFICEBFJKCUAEZGEUgIQEUkoJQARkYRSAhARSSglABGRhFICEJHEy+VyZDItZDIt5HK5qMMZ\nNebuUcewBTPzOMYlIvUnl8vR3NxKf/8iAFKpOXR2dtDY2BhxZKUzM9zdij4/jg2tEoCIjJZMpoWe\nniagNTzSQTq9jO7upVGGVZZSE4C6gEREEmpc1AGIiEQpm22jr6+V/v7geSo1h2y2I9qgRom6gEQk\n8XK5HO3ti4EgIdRi/z+oBiAikliqAYiISFGUAEQk0Z5+Gl57LeoooqEEICKJ9ulPw29+E3UU0VAN\nQEQS6x//gLe/PbgL2GmnqKMZOdUARESKdMcd8J731EfjXw4lABFJrO5uyGSijiI6SgAVlMvlaMlk\naMlkNi0oldRFpkRqgRJABMzsWDN71MweN7M5UcRQablcjtbmZpp6emjq6aG1uZmFCxfS3NxKT08T\nPT1NNDe3FkwChRKHiFTXhg3w3vfCzJlRRxKdUS8Cm9lY4E/AMcAa4B7gVHd/JO+cmisCt2QyQcMf\nPu8A5u/8dp5+8VsMt8jUQOJYFM5Dn5NK0dHZWbMzEUUkOrVQBJ4JPOHuK919PfAL4CMRxBELi9vb\nWdTfTytBmljU38/i9vaowxKRBIhiMbjdgFV5z1cDh0YQR0W1ZbO09vVB3pX8uRfMZuHCOYlcZEpE\n4i+KBFBU386CBQs2PW5oaKChoaFK4VRGY2MjHZ2dm67eO7JZGhsbmTFjRt4iU1tuMlEocXRks6Mb\nvIjUpN7eXnp7e8t+fxQ1gMOABe5+bPj8QmCjuy/KO6fmagADHnkErr9+82NjxsCMGTBrVuH35HK5\nTYmjLUwcIiKliv1qoGY2jqAIPAv4C3A3dVAEHrBiBSxZsvmx9evhXe8KppwP9sorkErBOO3MIDIq\ncrkc55/zHHvu1s15886oqwuu2CcAADM7DvguMBa4wt2/Mej1mk0ApbrkEvjqV+FDH4Ljj4ePfhS2\n3z7qqETqUy6X4/STzuCV1x/nEqbwlZTX1ai7mkgAW5OkBADw7LPQ0wNLl8Ltt8Mpp8DcuTB1atSR\nidSXlkyGyT078BRnchPH0wEsS6dZ2t0ddWgVUQvDQGWQKVPgjDPghhuCLqQpUyBB+U9kVD3ACZzA\nr6MOIxZ0ByAiiXHTTTlOOP5gvsUMduGZupt4qTuAmBvJsg/33RfcJSg3ipRnp50a2XOvbfhd+t0s\nS6frqvEvh+4ARtFIl3244w74zGeCIvEPfhCsY1LKd2uoqSRdfz+sWgXTpkUdSXWoCBxjhdYLKrUA\ntXEjLF4MX/kKnHUWfPnLsO22w79H6w2JJIO6gOrcmDFw9tnwxz8GuxgdddTWu4SGWm9Iq5CKJJum\nH42iSi77MHlyMOHsuefAis73b3nhv/97s7uC1r4+3RWIJIzuAEbRwHpBy9LpihWgHnhg6xvOtGWz\nQbIh6Haak0qxAbQKqUjC6Q5glDU2NlbsKjuXy9Hc3Ep/f7CMUl9fK52dHWzcCBdffDlmG8lm2wou\nVKfGXpLkH/+A115L7t6/Q1ECqCG5XC5vZdE22tsXh41/UFbu74cLL/wGDz64H+vXXwrcvikpFEo8\nWoVUkiKXg0svDWbcy1uUAGpEoav9/fbbb4vznn56NevXfxyYDHyU/v7taG9fvEXjP9Ty1SL16Ne/\nDtbaks0pAdSIQlf78ENSqc03nNlzz3148cX8YUEfZPXqZwp+ZiW7o0Tiyj1IAF/8YtSRxI8SQA3b\nZZdJdHZ+ebMNZ4DwTiE4521v+wF//3sP3/42fP7zUUUqEp3774d/+ifYZ5+oI4kfJYAakc220dfX\nusX2koWu4js7O/KSwgLe//7tefXV0Y5YJB5uuAFOOCHqKOJJM4FryOAisLpvRLbua1+D446DQw6J\nOpLq01IQIiIJpaUgpCgbN8Kdd0YdhYhESQkgoVatgpYW+P73o45ERKKiInBC7blnsP3kscfCCy8E\nq4qWs6aQiNQu3QHEVC639TV+Ruqd74TbboNrrw3GSKvsIpIsSgAxNDDrt6eniZ6eJpqbW6uWBCZP\nht5euPVWuPjiqnyFSCTOOw/WrIk6injTKKAYymRa6OlpgrytY9LpZXR3L63ad778Mrz5phbLkvqw\nejW8733BcukTJkQdzegpdRSQagACBDMlRerFddfBiScmq/EvhxJADA0161dEirN0qZY+KYa6gGIq\nDrN+N2wICsPjx4/6V4uUbe1aePe7g+6fbbaJOprRpZnAUjHt7XD33fCzn8G4Mu4V45DEJHl++EP4\n7W/h5z+POpLRpwQgFfP663DSSUFh+Cc/KS0JDN6/IJWas2ljGpFq2rAB1q2DXXaJOpLRpwQgFfX6\n69DUBO94B3R0wNixxb0vipFMIkmntYCkorbZJlhOd80aOOccTRYTqSdKALJVqRQsWxYUgwdGJm1N\nNttGKjUH6AA6wpFMbdUMUxKk2JnyozGjvpapC0iqRkVgqYZi60tJrEOpBiCxpqQgIzVUfemb31zK\nnnvCP//z8OfVcx1KNQCJrdFc40iS58wzYcWKqKOoLUoAUrY33oB77in+/Pb2xeHteCsQ3JoP3A2I\nFKtQfenkky/gxRfhqKOGP091qM0pAUjZHn8cjj8e+vqijkSSpLGxkc7OoDsnnV5GZ2cHK1cewWmn\nwZgxw5+nLsfNVaUGYGYLgE8Az4eH5rr7b8LXLgQ+DrwJfM7duwu8XzWACORyORa3twPQls0O+Y8l\n/7zpDf+XSy6ZwfLlcOCBW//8pBXlpPo2boS99oIbb9z632C9K7UGgLtX/AeYD1xQ4PgBwB+A8cBe\nwBPAmALnuYyurq4un5RK+ZXgV4JPSqW8q6urqPPmzHnAd9/dfeXK4r4nnT7Z0+mTC36+SKl6e90P\nPDDqKOIhbDuLbqurdQcwH3jV3dsHHb8Q2Ojui8LnXcACd79z0HlejbhkaC2ZDE09PXnjJWBZOs3S\n7u6izjvq+G4uuyzoDip1Cr5GBslIPPooPPMMZDJRRxK9OO0HcK6ZnQHcC2TdfR2wK5Df2K8Gdqti\nDDJKZs+GHXcsffXFwd1CfX2t6haSkuy3X/AjpSs7AZhZDzC5wEvzgMuAr4XP/x1oB84a4qN0qR8D\nbdksrX19m6b6zkml6MhmSzqvtXWL07dq85FBwce2ty9WAhAZBWUnAHdPF3OemV0O/Cp8ugaYmvfy\n7uGxLSxYsGDT44aGBhoaGsoJU4rU2NhIR2fnpuJuxxBF4GLPE5Hq6+3tpbe3t+z3V6sGMMXdnw0f\nnw8c4u6nmdkBwBJgJkHXz3Jgn8Ed/qoBJIdGBolUTiyWgjCzq4B/JejeeQr4lLuvDV+bSzAMdAMw\n2923mAqqBFAf+vvhzjvhgx8c/jwVgaUcr7+evB2/tiYWCWCklADiYySN82OPwZFHwpIlMGtWtSKU\npJoxAy67DA45JOpI4kNrAUnFjHTtnmnT4Jpr4NRT4cEHqxio1LVCSzr/13/BCy/A9OkRB1fjdAcg\nQyplNcXhZhH/9Kdw0UXw+9/DlCmjELjUjaFqRNdd18gee8C8eREHGDNxmgcgCZHL5WhtbmZRODS0\nta+Pjs7OTUng3/4NnnoKTjwRenth++0jDFZqSqFhwosWXcUf/tCou8oKUAKQIWWzbfT1tW7aBSxY\nTbFji/MWt7ezqL9/030C/f0sbm/f7C7gootg0qTSNpYXKWTt2iM48kjYddfNj2swQen0z1GGNLCa\n4lv/qMofnmkGbVqJV0pU6CJk1qybaWra/DzNKC+PagAyYoO7gOakUpt1AYmMRDFX9knc/asQ1QBk\n1Gl2sFRTY2Oj/p6qRHcAEpmXX4bVq+GAA6KOROKknL58zSgPaCKY1IxcDs46KxgeOnXq1s+X+jeS\nhlxFYCUAqTHf/jb85CfBPgITJ0YdjURNffkjo5nAUlOyWTj8cDjlFNiwIepoJK4efjjYf1oqSwlA\nImUGl14Kb74ZbCqjG79ky2bbSKXmEOw11xHOPWnju9+FQw+NOrr6owQgkRs/Plgz6B3vCDb4hqA/\ntyWToSWTKWn9IaltA3NP0ullpNPL6OzsYPr0Rn75Szj77Kijqz+qAUjsaF6B5Fu4EJ58Eq64IupI\n4k9FYKl5xW5QL/XvjTdgr72CEWMHHhh1NPGnIrCI1I2nnoLjjlPjXy26A5DYGegC+nJ/irG8wYLU\nm3R0dgIkfpy3yHDUBSR1IZfLcd6nn+dvr+zLjzpeYuxY10zPOjTcPhJSOiUAqRvr1wdjv6dNgz/9\nqYXlyzVBqJ6o2F95WgxO6sb48fDLX8IRR8CGDZoFVG+K2UdCqksJQGJthx3gxhth+vTTmDDhU7zx\nRnB8qM1ppD64B5MEpbrUBSQ14Z574JJLnuCvf50DqAhcD4bqAjrkkEZmzQoWCdxmm4iDrDGqAUhi\naPXH2leoCDx/PqxaBT/6UcTB1SAlAEkErf9en9atg332gbvugne9K+poao+KwJII7e2Lw8Y/KCH2\n9wfHlABq26WXBiO/1PiPDiUAqWETog5AKujll+F734Pf/S7qSJJDS0FITcpm2xgzZiLwB/KXDZba\n9eyz8LnPwb77Rh1JcqgGIDXr6qtv4cwz92fvva/l4ov3VfePJJ4Wg5PEOOWUD3L//ZN54YXP4q7G\nfzRon4b6ojsAqXl33AEnnQRdXTB9etTR1C8t3RB/GgYqiXT99cGmIRdcEHUk9Uv7NMSfhoFKIp10\nUtQRSDleeSXYD3rHHaOOJJlUA5C6p37rymjLZoNuH4Kr/zmpFG3Z7Ig+c+FCmDevIuFJGdQFJHVN\n/daVVcn1+1euDGo2f/wj7LZbhQJMONUARELPPQfnnN7IR5Z3q986hk49Fd79bliwIOpI6oeGgYqE\nPvMZWPHYJ9GlxOjL5XJkMi1kMi0Fu93uugtuuw2+8IUIgpNNyk4AZvZRM3vIzN40s4MHvXahmT1u\nZo+aWSbv+HQzWxG+dslIAhfZmssvhw1jGjl33EVF9VsXqhVsrSGTLQ0s1NfT00RPTxPNza2b/e7c\nIZuFr38dttsuwkAF3L2sH2A/YBpwC3Bw3vEDCObnjwf2Ap7gra6mu4GZ4eObgGOH+GwXKUdXV5en\n0yd7On2yd3V1+Zo17pMn/90P2v9iPzmd9q6uriHfNymV8ivBrwSflEr517/+dU+lJjlc6XClp1KT\nhny/vCWdPjn8nXn4c6Wn0ydvds6997pv2BBRgHUsbDuLbsfLHgbq7o9C0Oc0yEeAn7v7emClmT0B\nHGpmTwMT3f3u8LyrgJOArnJjEMk3eInovr5WOjs7uO22Ro466nzm/fv5DNQsB+8lUGh7wvkXX0J/\n/7fQiqPDG/y7LIYm7MVDNeYB7Arcmfd8NbAbsD58PGBNeFykIoZaIrq7u5Ebb4Q7w7/KQoli5n67\nRhR1bSv0u5w371z6+uYQDrzS9p0xNmwCMLMeYHKBl+a6+6+qE5JI5R18cPADhRPFy/yQOakU5A0X\nPfeC2SxcqIZsOIV+l7feuozOzo68uwJt1BNXwyYAd0+X8ZlrgKl5z3cnuPJfEz7OP75mqA9ZkDc2\nrKGhgYaGhjJCkSTJZtvo62stq8HeZZdJfKOzc9MY945wjPuMGTPUkJWhsbFRv6tR0NvbS29vb/kf\nUErBoNAPQRF4et7zgSLwBGBv4M+8VQS+CzgUMFQElioYXAQe6pwkF3eL+R2V8lnF/C6XLHH/j/8Y\n0VdJESixCDyShr8ZWAX0A88Bv8l7bS7B6J9Hgca849OBFeFr3xvms6v7W5LE6+rq8sMPn+2HHz47\ncY1/pZPf1hLK88+7T5rkftddI/oaKUKpCUAzgSWxrr4azj8fli+HAw6IOprRkcm00NPTBHlzo9Pp\nZXR3L63ad55+OuyyC3znO1X7CglpNVCRIp1ySrAS5THHQHc3vPe9UUdUf7q6oK8PHnww6kikECUA\nSbTTToMxYyCdDpLAgQdGHVF1jaRQXqpXX4Wzz4bFizXjN67UBSTCW91BDz0EO+0UdTTVNXjiVrVG\n66xbB0uXwllnVeXjpQCtBipSpiefhHe+M+ooRMqn1UBFypTkxl+L3iWT7gBEEm7wcg6p1Bw6OzXp\nrRapC0ikgl56CXbYIeooqquSQ0M3bIBxGloSGXUBiVTIxo1w9NFw2WVRR1IbrrsOPvaxqKOQUihX\niwxhzJigUUung+0lFyyALVc/r32VGBr65JPBkM9f/7oKAUrVqAtIZCvWroWmJth3X7jiCnjb26KO\nqPJGMjT0xRfhyCPh05+Gz352+O+o1IbyUphqACJV8NprcMYZ8MILcPPNwd1BvqQ2bq+/HtwhzZwJ\n4X9+QblcjtbmZhblLbfd0dmZmN/TaFECEKmSjRuDTWXe//7Njxdq3M6dN49bb70PqO5kq2oqJqn9\n4AfQ2wtLlmyZFPO1ZDI09fTklZlhWTrN0u7uisedZFoLSKRKxozZsvEHCm4nOfsrX+OljUGXysDW\nlLWUBAYntda+voJX7J/6VDDTd7jGX+JLCUCkCt7cuC+1vJdwoaS2uL19i/8GMxg/fuuf15bN0trX\nt9mOax3ZbGWDlpIpb4uM0JEf/hpf2GYyHQRdG+eNGcurHBt1WJvkcjlaMhlaMpnIZvk2NjbS0dnJ\nsnSaZem0+v9jQncAIiP0/POHMX7nJ/jJHheww8Sn+PzRR7Nw4aX09wdLi0a5l3CxXTmDFbpiv/KC\nLK+8AhMnlheLtomMHxWBRSpgyRKYPRvmz4dzzoGenuKGVVZ7Zc6RFF/zi8CfOD/LzTc38swzcM01\nFQ1RKkhFYJEInHYazJgRDBW9/nr48Y8b6e4evjEfvAZP3IrFA1fsGzbAJz4Bjz2miV71RjUAkQqZ\nNi3Y/epDH4I1a7Z+fnv74rDxbwWCRDBwN1ApbdlsUHAluPqfk0rRVkLx9bXXoKUlmAzX0wM771zR\n8CRiugMQqaBx42Du3KijeMtA8XWgK6ejhElq69bBEUfAIYcEu3pNmFDNSCUKqgGIRCTuyzC7w+23\nw1FHRR2JFEszgUVi6qtfhYMOghNPfGtRudHanlGSQQlAJKa6u4ORQlOnBqOFjjgi6oik3mg/AJGY\nymTggQeCourppwd7DSxfHnVUcMstwRIXf/5z1JHIaFMCEBlFEyYE6+c89hh88pNw991bnjNaM3dX\nrIDjjw+GeM6eDXvvXbWvkphSF5BIjIzGssn33x80+H/6E8ybF2zkohE+9UE1AJEaNjBz97dcyU78\njQ3czlNHjePGW6+u2HesWgX33htc/avhry+aCSxSB7K0cxMf5hecycO/O4p99oEPfAC+/33Ybrst\nz3/mGXjlleDnpZfgoYeCn8sv33Iby6lTgx8RJQCRGBlYhG1R/wqmsILnUt/jhqWd7LFHI319Q1+x\nz5oVLMs8cWLws//+wfj9N98MJqeJFKIuIJGYSer2kjJyqgGIVFkUDbQmjEkxlABEqiiKzc3jvmSE\nxIcSgEgVRbG5eSbTQk9PE+R9azq9jO7upVX7TqlNmgkskhC5XI5MpoVMpiWyrR6ltml8gEgJotjc\nPJtto6+vdeArSaXmcPTR58Z6MxmpDeoCEilRHIrA7e2L1S0kWxi1GoCZfRRYAOwHHOLu94XH9wIe\nAR4NT/29u58TvjYduBLYBrjJ3WcP8dlKACLDUF1AChnNmcArgGbg/xd47Ql3P6jA8cuAs9z9bjO7\nycyOdfeuEcQgkkiFuoWy2Y5og5KaU3YR2N0fdffHij3fzKYAE919YP3Dq4CTyv1+kSRrbGykszO4\n6k+nl6n/X8pSrSLw3mZ2P/AScJG79wG7AavzzlkTHhORMjQ2NqrRlxEZNgGYWQ8wucBLc939V0O8\n7S/AVHf/m5kdDFxvZu8pNbAFCxZsetzQ0EBDQ0OpHyGSOJoxnCy9vb309vaW/f4RjwIys1uA7EAR\neKjXgWeB37r7/uHxU4Gj3f3sAu9REVikRJoxLFFNBNv0hWa2i5mNDR+/E9gXeNLdnwVeNrNDzcyA\n04HrK/T9IonX3r44bPxbgSARDNwNiBRSdgIws2YzWwUcBtxoZr8JXzoaeCCsAfwS+JS7rwtfOwe4\nHHicYKSQRgCJiEREE8FE6oS6gESLwYkkmIrAyaYEIBJTapyl2pQARGJI3TMyGpQARGJIa/fIaNB+\nACIiUhTtByAyCrR4m8SRuoBERomKwFJtqgGIiCSUagAiIlIUJQARkYRSAhARSSglABGRhFICEBFJ\nKCUAEZGEUgIQEUkoJQARkYRSAhARSSglABGRhFICEBFJKCUAEZGEUgIQEUkoJQARkYRSAhARSSgl\nABGRhFICEBFJKCUAEZGEUgIQEUkoJQARkYRSAhARSSglABGRhFICEBFJKCUAEZGEUgIQEUkoJQAR\nkYRSAhARSaiyE4CZfcvMHjGzB8zsOjPbIe+1C83scTN71Mwyecenm9mK8LVLRhq8iIiUbyR3AN3A\ne9z9X4DHgAsBzOwA4BTgAOBY4D/NzML3XAac5e77Avua2bEj+P7Y6u3tjTqEstVy7KD4o6b4a0vZ\nCcDde9x9Y/j0LmD38PFHgJ+7+3p3Xwk8ARxqZlOAie5+d3jeVcBJ5X5/nNXyH1Etxw6KP2qKv7ZU\nqgbwceCm8PGuwOq811YDuxU4viY8LiIiERg33Itm1gNMLvDSXHf/VXjOPOANd19ShfhERKRKzN3L\nf7PZ/wE+Ccxy99fDY18CcPdvhs+7gPnA08At7r5/ePxU4Gh3P7vA55YflIhIgrm7bf2swLB3AMMJ\nC7hfIGjEX897aRmwxMwuJuji2Re4293dzF42s0OBu4HTge+N9D9ARETKU/YdgJk9DkwAXgwP/d7d\nzwlfm0tQF9gAzHb3XHh8OnAlkAJucvfPjSh6EREp24i6gEREpHbFdibwcBPN4srMjg0nvz1uZnOi\njqcUZjbVzG4xs4fM7EEzq8m7MzMba2b3m9mvoo6lVGa2o5ldG/7dP2xmh0UdU7HCyZ8PhRM9l5jZ\n26KOaThm9iMzW2tmK/KO7WxmPWb2mJl1m9mOUcY4nCHiL7nNjG0CYIiJZnFlZmOB/0cw+e0A4FQz\n2z/aqEqyHjjf3d8DHAZ8psbiHzAbeBioxVvbSwi6RvcH3gc8EnE8RTGzvQgGgxzs7gcCY4H/FWVM\nRfgxwb/VfF8Cetx9GnBz+DyuCsVfcpsZ2wQwzESzuJoJPOHuK919PfALgklxNcHdn3P3P4SPXyVo\nfHaNNqrSmNnuwIeBy4GaGkgQXq0d6e4/AnD3De7+UsRhFetlgguIbc1sHLAtwTyf2HL324G/DTrc\nBHSEjzuI8UTVQvGX02bGNgEMkj/RLK52A1blPR+YAFdzwiu6gwj+iGrJdwhGpm3c2okxtDfwvJn9\n2MzuM7Mfmtm2UQdVDHd/EWgHngH+Aqxz9+XRRlWWSe6+Nny8FpgUZTAjVFSbGWkCCPvbVhT4OTHv\nnFqZaFaLXQ5bMLPtgWsJRm+9GnU8xTKzE4C/uvv91NjVf2gccDDwn+5+MPB34t0FsYmZvQs4D9iL\n4K5xezP735EGNUIejI6pyX/TpbSZZc8DqAR3Tw/3ejjR7MPArFEJaGTWAFPznk9l86UvYs/MxgNL\ngZ+6+/VRx1Oi9wNNZvZhYBvgn8zsKnc/I+K4irUaWO3u94TPr6VGEgAwA/idu78AYGbXEfz/8bNI\noyrdWjOb7O7PhWuX/TXqgEpVapsZ2y6gvIlmHxk00Syu7iVY4XQvM5tAsCLqsohjKlq4YusVwMPu\n/t2o4ymVu89196nuvjdBAfK3NdT44+7PAavMbFp46BjgoQhDKsWjwGFmlgr/jo4hKMTXmmVAa/i4\nFaipi6By2szYzgMYbqJZXJnZccB3CUZBXOHu34g4pKKZ2QeA24A/8tat74Xu3hVdVOUxs6OBrLs3\nRR1LKczsXwgK2BOAPwNn1koh2My+SNBobgTuAz4RDoaIJTP7OXA0sAtBf/9XgBuAa4A9gJXAx9x9\nXVQxDqdA/PMJRv2U1GbGNgGIiEh1xbYLSEREqksJQEQkoZQAREQSSglARCShlABERBJKCUBEJKGU\nAEREEkoJQEQkof4HkpkE0ir06BsAAAAASUVORK5CYII=\n",
"text/plain": [
""
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"plt.plot(xs, g(xs), '--')\n",
"plt.scatter(X_train, y_train, c='b')\n",
"plt.scatter(X_test, y_test, c='r');"
]
},
{
"cell_type": "code",
"execution_count": 31,
"metadata": {
"collapsed": false
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"train_error(h1) = 29464.92; test_error(h1) = 17336.67\n",
"train_error(h2) = 7791.35; test_error(h2) = 4718.77\n",
"train_error(h3) = 3002.65; test_error(h3) = 1657.94\n",
"train_error(h4) = 2912.17; test_error(h4) = 1787.16\n",
"train_error(h5) = 2612.13; test_error(h5) = 1754.30\n",
"train_error(h6) = 2158.16; test_error(h6) = 2150.77\n",
"train_error(h7) = 1937.27; test_error(h7) = 1916.90\n"
]
},
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAYAAAAEACAYAAAC6d6FnAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzs3XdcVfUbwPHPYSgXBJEhOHAiOHLmthQHODIHmIHmyrTU\n1BJLzcyZiaPclZVK5czRz5HgChVX7i0pLkRBxQmy7/P746KiYoJyAfX7fr3Oq8u555z7Pbc6z/3O\nRxMRFEVRlFePSW4XQFEURckdKgAoiqK8olQAUBRFeUWpAKAoivKKUgFAURTlFaUCgKIoyivquQOA\npmlzNU2L1jTtSLp9dpqmbdA07V9N09Zrmmab7r1hmqad0jTtpKZpXs/7+YqiKMqzyY4awDygxSP7\nhgIbRMQN2JT2N5qmVQTeBSqmnTNb0zRVC1EURckFz/3wFZFtwI1HdrcBAtNeBwLt0l63BRaJSLKI\nnANOA7WftwyKoihK1hnr17eTiESnvY4GnNJeFwUupjvuIlDMSGVQFEVR/oPRm1/EsNbEf603odai\nUBRFyQVmRrputKZpziISpWlaEeBK2v5IwCXdccXT9j1E0zQVFBRFUZ6BiGiZPdZYNYBVQLe0192A\nP9Pt99U0LZ+maaWBcsA/GV1ARF7YbeTIkblehlex7Kr8ub+p8ufullXPXQPQNG0R0Ahw0DQtAvgK\nmAAs1TStJ3AO6Jj2UD+uadpS4DiQAvSVZym1oiiK8tyeOwCIiN8T3mr2hOPHA+Of93MVRVGU56PG\n4BuBh4dHbhfhmb3IZQdV/tymyv9i0fJiC4ymaaplSFEUJYs0TUOy0AlsrFFAiqIo2UrTMv1ceyVk\nx49kFQAURXlhqJYBg+wKhqoPQFEU5RWlAoCiKMorSgUARVGUV5QKAIqiKNmse/fujBgxIreL8VQq\nACiKomQzTdOy1FGbnJxMhw4dKF26NCYmJmzZssWIpXtABQBFURQjyOqIpYYNG/L777/j7OycY0Ne\n1TBQRVGU53TgwAF69uzJ6dOnadWqVZYf4Obm5gwYMAAAU1NTYxQxQ6oGoCiK8hySkpJo164d3bp1\n48aNG7zzzjssX74cTdOIiIjA1taWQoUKZbgtXrw4V8uuagCKorwUtNHZ02wiI7PWdLNr1y5SUlIY\nOHAgAD4+PtSqVQsAFxcXbt68mS3lMgYVABRFeSlk9cGdXS5dukSxYg9nti1ZsuQLMWtZNQEpiqI8\nhyJFihAZ+XBiw/Pnz99vAipQoADW1tYZbosWLcqlUhuoGoCiKMpzqF+/PmZmZkyfPp0+ffqwevVq\n9uzZQ9OmTXFxcSE2NjZT10lMTLxfa0hMTCQhIQELCwtjFl3VABRFUZ6Hubk5K1asYP78+djb27N0\n6VJ8fHyyfB13d3csLS25dOkSzZs3x8rKigsXLhihxA+ofACKorwQ0ta6z+1i5AlP+i6ymg9A1QAU\nRVFeUSoAKIqivKJUAFAURXlFqQCgKIryilIBQFEU5RWlAoCiKMorSgUARVGUV5QKAIqiKK8oFQAU\nRVGymUoJqSiK8orKakrIXbt24enpib29PYULF6Zjx45ERUUZsYQGRg0Amqad0zTtsKZpBzRN+ydt\nn52maRs0TftX07T1mqbZGrMMiqIouSEry1bcvHmTjz76iPPnz3P+/Hmsra3p0aOHEUtnYOwagAAe\nIlJdRGqn7RsKbBARN2BT2t+KoigvrAMHDlCjRg1sbGzw9fUlISEhS+e3aNECHx8fChQogE6no1+/\nfmzfvt1IpX0gJ5qAHq0HtQEC014HAu1yoAyKoihGYYyUkFu3buW1114zetmNnQ9AgI2apqUCP4rI\nT4CTiESnvR8NOBm5DIqivAqymIj9ibK44mh2p4Q8fPgwY8eOZdWqVVk671kYOwA0EJHLmqY5Ahs0\nTTuZ/k0REU3T1PquiqI8v1xaKjo7U0KePn2aVq1aMX36dBo0aJBdRXwiowYAEbmc9s+rmqatBGoD\n0ZqmOYtIlKZpRYArGZ07atSo+689PDzw8PAwZlEVRVGeyZNSQrq6uhIREUGFChWeOCJozpw5+Pn5\n3T/H09OTr776is6dO2fqs0NCQggJCXnmshstIYymaZaAqYjc0TTNClgPjAaaATEiEqBp2lDAVkSG\nPnKuSgijKMpD8mpCmOTkZFxdXfH397+fEtLX15ehQ4cyZsyYTF0jMjKShg0b0rdvX/z9/Z96fHYl\nhDFmACgNrEz70wxYICLfaJpmBywFSgDngI4icvORc1UAUBTlIXk1AADs27ePXr163W/C0TSNcuXK\nZToAjB49mtGjR2NlZXV/n6Zp3L59O8Pj83wAeB4vbAAQgZ9/hs6dwdIyt0ujKC+VvBwAcppKCZkX\naRps3AiTJuV2SRRFUZ5K1QCy2/nzUKMGHDgAJUrkdmkU5aWhagAPqBpAXlWyJHz8MQwZktslURRF\n+U+qBmAMcXFQvjwsXAhvvpnbpVGUl4KqATygagB5mZUVTJwIAwdCampul0ZRFCVDKgAYi6+vYSTQ\n/Pm5XRJFUZQMqSYgY9q3D1q3hpMnoWDB3C6NorzQVBPQA2oewIuiZ0+ws1NDQxXlOakA8IAKAC+K\nqCh47TUIDTV0DCuK8kxepADQvXt3XFxcGDt2rFGurzqBXxTOzjB8OPTvn2urFSqKkrOymhLy+PHj\n1KxZEzs7O2xtbWnQoAGhoaFGLKGBCgA54eOPDTWB5ctzuySKouSQrNRWihUrxh9//EFMTAw3btzA\n19eXDh06GLF0BioA5ARzc5g1Cz79FGJjc7s0iqJks+dNCVmwYEFKly6NpmmkpqZiYmJCkSJFjFTa\nB1QAyCkNG0KjRjBuXG6XRFGUbJSdKSFtbW3R6XRMnDiRZcuWGb3sqhM4J12+DJUrqw5hRXkGT+sE\n1p4jMUp6ksXkU1u3bsXPz++hpDANGjSgadOmmV4OOr27d+8yevRoNmzYwL59+zLsS8iuTmBjp4RU\n0itSBL780tAhvH599uUwVRQlyw/u7JKdKSEBLC0tmTBhArNmzeLIkSNUqVIlO4qZIdUElM2e+i9d\ndQgrykvlSSkh7zUBFShQAGtr6wy3RYsWZXjN1NRU9Ho9lkbOK6ICQDbz8PDg+PHjTz7AzOxBh/Cd\nOzlXMEVRjKJ+/fqYmZkxffp0kpOTWbFiBXv27AHAxcWF2NhY7ty5k+F2Lx/wxo0bOXjwIKmpqdy+\nfZtBgwbh7u6Oq6urUcuuAkA2q1mz5mMdO49p2BCaNoWRI3OmUIqiGI25uTkrVqxg/vz52Nvbs3Tp\nUnx8fLJ0jZs3b+Ln54etrS3u7u5cvXqVVatWGanED6hO4Gy2Z88eOnfuTFhY2H9PBLl2DSpVgnXr\nDAlkFEX5Ty/STGBjUzOB86iaNWui1+vZv3//fx/o4AABAdC7t1oyWlGUXKECQDbTNA1fX9+nNwMB\ndOsGBQoY+gQURVFymGoCMoKjR4/SqlUrzp07h4nJU2LsyZPwxhtw8CAUL54zBVSUF5BqAnpANQHl\nYa+99ho2Njbs2LHj6QeXLw/9+hmyhymKouQgFQCMxM/PL3PNQADDhsGRI5ADvf6Koij3qCYgIzl9\n+jQNGjQgMjISM7NMTLjevBm6d4ejR8HGxujlU5QXjWoCekA1AeVxrq6ulCxZkr///jtzJzRpAp6e\nhtqAoihKDlABwIgyPRronsmT4c8/Yds24xVKURQljWoCMqKLFy9StWpVLl26RP78+TN30p9/wuef\nw6FDoNMZt4CK8gJ5kZqAVErIV1BwcDA+Xl74eHkRHBxM8eLFKVasGHXqeODl5UNwcPDTL9KuHVSr\nBs+wjKyiKHlDVlNCpjdmzBhMTEzYvHlzNpfqcbmyHLSmaS2AqYAp8LOIBORGObJTcHAw3dq3JyA+\nHoBuoaH0Hz6ckyfPkZxcEWhDaGg3Vq4MpHnz5o+dO2fKFAB6+/vTfMYMqFIF3nlHLROhKC+oZ6mt\nhIeHs2zZMooWLWqEEj0ux2sAmqaZAjOBFkBFwE/TtAo5XY7sNmfKFALi46lNed7DhID4eH76dhrJ\nyV8DJ4G2xMcHMGXKnIfOuxc42mzYQJsNG+jWvj3BBw/CpEnw/vuQnJwr96MoSuY9b0rIez7++GMC\nAgIwNzfP5hJmLDeagGoDp0XknIgkA4uBtrlQDqMYzGTc+Jf1DEAvVoAN0A74LsPj7wWObkA3ICA+\n3lAb6NIFnJ1h4sScK7yiKFmWXSkh//jjDywsLGjZsmWOlT03moCKARHp/r4I1MmFcmSr3v7+dAsN\nZVrl1hzbX5eJMhiThG8wMwskJaU+MAgLC0v8/X/L3AU1DebMgddfhzZtDKkkFUV5ohAtJFuu4yEe\nWTp+165dpKSkMDBtNr+Pjw+1atUCDPkAbt68+dRr3Llzh+HDh7Nx48Ysl/d55EYAyFTD2KhRo+6/\n9vDwwCOX0r1lVvPmzQlcuYIzYe9TZdBRVtrcwN3dksGDW7BmjQM63W08Pfc81v5/L3CQ1ncwRKcj\n0N/f8GaJEjBhgmHRuN27IYeqhYryIsrqgzu7ZEdKyFGjRtGlSxdKlChxf19mzg8JCSHkeXIhi0iO\nbkBdICjd38OAIY8cIy+qpUv10rr1HFm50kG8vH4VExMRTRNxdEwQM7Mdsnt39GPnBAUFibenp3h7\nekpQUNDDb+r1Iq1aiYwcmTM3oCh5VF59LoSEhEjRokUf2le/fn0ZMWKEXLhwQaysrKRAgQIZbgsX\nLhQRkWrVqomDg4M4OzuLs7OzmJqaip2dnUycODHDz3zSd5G2P/PP46wcnB0bhlpHOFAKyAccBCo8\nckwmvva87c6dQ7Jrl7ucONFDEhPj5OZNkTffXCUWFrflt98Mz3URkXnzRJo0ERkxQmTjRpHk5Awu\nFhkp4ugosndvTt6CouQpefW5kJSUJCVKlJBp06ZJUlKSLF++XMzNzWXEiBGZvkZMTIxER0dLdHS0\nREVFiYuLiyxbtkxiY2MzPD67AkCOdwKLSArwMRAMHAeWiMiJnC6HsRUoUIXXX9+LSDIHD9YCdrJs\nWR3y5Xub0aOT6NwZYmMNw/79/Q05Yb74AooWNeSIeSitcNGi8N13hqagxMTcuiVFUTKQHSkh7ezs\nKFy4MIULF8bJyQlTU1MKFSqElZWVkUptoGYCG5mIcOXKYsLD/bG3f4t586yJiorHxOR79u41TPwt\nU+bB8efOwbJl0Lixof833YXAxwfc3Az9AoryinmRZgIbW3bNBFYBIIekpNzi7NkRnDgRiK9vHG9U\nq0O56j+xbFlFFiyAZs0ycZErV5CqVdFWrIB69YxeZkXJS1QAeEAFgDwq9OZNLiYmEpOSwrXkZK4l\nJ3MzJYV8msa1yEjObv6R8uFriYxIQStXgXqv/cTvU2ry2Wcan35qGP35JBEREFBnBeNShqA7cYD8\n9gUyXa7HZhs/MhpJUfI6FQAeUAEgj+odFsatlBTszc1xSNtszcxIFmHGjBm4nT1LaTNTfv5rBe/1\nNKGMZxnmxAzm9EgfCrsl0e/bODydC1KtQAFMH4kGIrBzJ8R2fJ/rt0y4O+1nunaFp6UbeHSZiiE6\nHYErVwKooKC8MFQAeEAFgBeQj5eXYbkH4AjQAHivQ0U6fXib+MvufDNmDmevFKZrxeNYmt3F3tQc\nB+v8lKlUkMIVrdGV02HpZol5vgTiK1Tna9tJLE1qz++/Q+3amftcgEBgXvXqnDx58rGgoIKAklep\nAPBAdgWAXFkM7lXV298f/63/YpP4BpY0ZBCXmLViFu3jf8Op7T5Gz27IH8s+5/s/PuDbAXHYlLnN\nwahbBIVdpuLiK5S7ZILV2RTMbc2wrz2PgZsn4Dm1FsWKZT2Z/MXz5+8vQQFA2hIUKgAoyqtDBYAc\nkHIrhSt/XKHwb4WZkX8eRxz3c8p5N62/7IjupI5hf3zJ1s5bsbAYgnvVFZSvOoSBI75iwIBIvvii\nBqkmOoJv3ODXK1f469o12t+woMthS+4e/ACt51HW227muNMpmn3zBs3bNic4OPj+onP+/r0znG3s\nWrIkXL+em1+Loii5TDUBGVHq3VQipkRw8buL2HrY4tTVCftW9pjkezD9QkTo2rUrSUlJLF68+P4a\n4ocO7efddwtRosReAgJ24+bWg9DQi0ycPpfLldy528IDAd6b/CNWG92Q5HpUJpXY+ncYuW8SpxOH\nAonodJ+zcmUg8HB7P5Bhv4CqASh5lWoCekD1AeRhoheiF0Zz9ouz2NSzocyEMuhKPzm7V0JCAh4e\nHjRt2pRx48bdDwJ370K/frGEhCQwdGhHkpNDWbmyJ9u2vY6p2ZcUb92SS1XaYFHZFNP/OWGyJJx3\n9AdonNCCMziwCBf2swVPz99Zv375Y5+rRgYpLxIVAB5QASCPuhl6k/BB4aCB63euFKxfMFPnRUVF\n0aZNG+zs7Jg7d+5DCSGWLIGuXW9RvPhG+vT5nrJlDxMc/Dqhocc4dmws7YqaY9LpLCveqItu3WpS\nfqtB47tedOE8VzAn2OUE6851xkTlf1NeYC9SAFApIV9BIsKlWZcoPrA4NXbWyPTDH8DZ2Znt27dT\nr149qlevzpIlS+6/9+67UKvWMM6cachnn22kT59d6PUmjB8fxdSpvbhTYSttp60hoH8v3JvUJ3lB\nQdZ32kn3/GFsM5vPR5dL8UPho5xYF2eM21YU5RFZTQl57tw5TExMsLa2vr99/fXXRiyhgaoB5EF7\n9uyhS5cu1KhRg1mzZlGoUCGCg4Np164nCQnrAFfMzT9n1armmJruIiLiJ5ydblImJB8lXh/FQrvK\nfHn6HDeLF+ETnQWj6zZmWadLOG2JoGT3wpT+ujRm1qr/X3mxvEg1gB49elC8ePFM1wDOnTtHmTJl\nSE1NzVTgUDWAl1itWrXYv38/Dg4OlC5dmvfee4/k5GT++OMHPD1HUbv2SEqU+JrJk1tTqtQ43n8/\nGo/GxzD39uGA7jNqWY/myDsObGpYj42FCtH01GEqLCxEo7O1SY1NZU/FPVz982pu36aivDSyKyWk\nXq/P5pL9NxUA8ihLS0umT59OWFgY9evXZ+LEiXTr1g0XF1sGDXqdBQtO0bhxPPXqwddfg5mZK2Xr\nzafu5ak4/xzJxQtTMDlRnWWOG+hZOD9vHTlC32tncPihDOV/K8+ZIWc42v4oCRef7T9URVEMsisl\nJBgSybi4uPD+++8TExNj9LKrJqAXSEREBEuXLmX79u2Eh4cTHh6OubkrMIOkJFeqVFlOxQqHsN4W\nglXBgpi9VZekpH9ISTlKIYfa7M/fgL/zFWNEjRr0LFeJa9OvcWHGJdy+LYtzN6cstVkqSk57WhNQ\nSEj2/Pfr4ZG1Z8/WrVvx8/MjMjLy/r4GDRrQtGlTxowZk6lrxMXFERYWRrVq1bh27Rr9+vXjzp07\nBAUFZXi8GgWkpC01fYXw8HA2b9bz66/uxMWZ0qLJZkqt7kVC0ybEubhw504MMTGHuHHjX67f0nHm\nqhkJ125TQKdDSypC5eQS1K1YndYBLanVsBaWlpa5fWuK8pi82gewePFivv32W/7555/7+zp16kTZ\nsmWfeRRQdHQ0RYoU4c6dOxnmBFBLQbzkHp3Nm9EYfU3TcHJywsnJifr1YfhwWLsWhg/vwNHiLRi+\nsQ9v7eyDaUV3AFJTE4iKms+FiIlE6Usx7UYb6mp12P1NBCF7NrGmzUecNzlPhUoV8Pb2xs/PjzLp\nkxUoivKYIkWKPPTrH+D8+fO4uroSERFBhQoVnli7njNnDn5+fk+8ttH7BLKSPiynNvJo6recEhQU\nJDqdk8B8gfmi0zk9niv4P6SmiixeLFKrVLSUMb8g3wUkyM2b6d9PlqioBbJ1V0X5fYu79Ptnmkz8\nJV5a2FyVdQU3yeJui6VPnz7i6OgoderUkalTp8rly5eNcKeKknl59bmQHSkhd+/eLSdPnpTU1FS5\ndu2adOzYUZo0afLE45/0XZDXU0IqTzdlyhzi4wOAbkA34uMD7tcGMsPExDB3YHe4I797BrJr1n5K\nlxb694fDh8HExAwnp068UfsITcuPp0XSLGzK1qb1mu18WagBpU+78+HpDzl3+ByjRo1i//79VKxY\nkb59+3Lx4kWj3beivIiyIyXkmTNnaNmyJTY2NlSuXBmdTseiRYuMVOIHVB9AHuTl5cOGDW0g3QLO\nnp6rMlzO4ani46F+fS6278+Pye8TGAj29tC1K3TqBE5OIJLK9nO/cPnCaJLMS9PUbQpxEx24+sdV\nKi2vhHUNa65evcrkyZP5+eef8fPzY9iwYRQrViw7b1tR/lNe7QPIDWoewEvM3783Ot0QDCv3B6LT\nDcHfv/ezXUyng+XLKT5zKGNb7eTcOZgyBQ4dgvLloXVrWLDAlCr2vfGqd4qLFk3ZdbQ1l98bgctk\nKw43P0zUr1E4OjoSEBDAiRMnsLCwoHLlygwePJi4ODW7WFFeVKoGkEdlphM4S9asgT59YO9ew89+\nIC4OVq40rDW0dSs0aQIdO0JyzbPsvzyC1vxFSYtPiOneGLsmTrhOdcXEzPCbISoqis8++4wdO3Yw\nd+5cGjVqZPx7UF5pqgbwgBoGqmTdyJGwaRNs3gz58j301s2b8L//GYLB9u1Qr0kKl2rvo2+dkZRN\nPYUWOBDHq42otLQSZgUfDB5btWoVffr0wdvbm2+++YYCBQx5ioODg2nfvltaXwbodENYuTJQBQHl\nmakA8IAKAErW6fXg4wOOjvDjj0/MQB8TY6gZLFoshP6jx6H2KfxqfkvjIpE4LP6YKos80JV6sLz1\n9evX+fTTT9m2bRuBgYG8+eab2duPoSioAJCe6gNQss7EBH79FXbsgNmzn3iYvT188AFs2qhx4ZQp\nzZsVZfqaSXT8aiHfFj3Nsr4/cGvXrfvH29nZERgYyLRp0+jQoQOzZ89W/6MqygtA1QBeRWfOQP36\nsGgRNG6cqVOOxsbScnU4+l8sid9XkKK2l+nRypp+U8pgYfHguPDwcNq0aUPJkiX5++99JCRMBB40\nAQGqX0B5JqoG8IBqAlKez+bNhnGgO3dC6dKZOuVmcjJ+x05w8HgiA8JO8+caR86E12XgYCv69NGw\ntzccd/v2bbp06cKZM2ewty9BvnwW90cxqX4B5VmpAPCACgDK85sxA376ydDra22dqVP0Iow+d465\nUVEs1qXw77oJrFjfn+2HvOjUyQR/f0M80ev1jBw5kt9++41Vq1ZRpUoV1S+gPBcVAB5QfQDK8/v4\nY6hXD3x9ISUlU6eYaBqjS5dmZrlytI8zIX+bGQzzWMQv06piZnqamjXhk0/gxg0Txo4dy4QJE/D0\n9GT37t1GvhlFyTu6d+/OiBEjcrsYT2WUAKBp2ihN0y5qmnYgbWuZ7r1hmqad0jTtpKZpXsb4fCWT\nNA1mzjQ8/AcOJDgoCB8vL3y8vAgODn7iacHBwfzaqRNVZs/G/8p5/vIZQYkdA/Fu+iZr/jecpKRU\nypeHgABo29aXefPm8fbbb+PpWTf7JrgpSh6W1ZSQAHfv3qVv3744Ojpia2ub4dyabJeVhYMyuwEj\ngUEZ7K8IHATMgVLAacAkg+OeuAiSYgQ3b8rtUqXkS3NzmQ8yH8RJp8twAbqgoCBx0unuH+fo5CSV\nN22SGv87JOvb7pCts96UvbvryKFDl8TbW8TFRWTJEpENGzaKg4ODjBs3Tjw9vcXT0/v+9YOCgh7b\npyiPepGeC927d5cvv/wyS+d07txZ/Pz85Nq1a6LX62X//v1PPPZJ3wVZXAzOmAHAP4P9w4Ah6f4O\nAupmcFyWvri8ZNquafLDnh8k5GyIXL5zWfR6fW4XKVN6v/GGxIHhP4m0h7u3p+djx3l7esr8R45r\n17y5tPr7hJjO2y3L2hyT0E8+ltCtThITEyTbtolUrCji7S2yatVucXR0lJUrV96/3vOufKq8OvLy\nc2H//v1SvXp1sba2lnfffVd8fX2zFABOnDghNjY2cufOnUwdn10BwJh9AP01TTukadovmqbZpu0r\nCqRfTvIi8FKtKGZmYsbuyN18sfkLXpv9GrYBttT+qTZdVnZh3NZxLDu+jCPRR0hIyVupGK/pdGx+\nxnNN9HrWNHLnk/JF6NjtBicvfowWMJITx3pQrNgo9u5Nxd0dPvigNgMG7ObDDz9i2bJlwPOvfKoo\nuS07UkL+888/lCxZkq+++gpHR0eqVKnCihUrjF72Z04Io2naBsA5g7eGA98D93KhjQWmAD2fcKkM\nu/VHjRp1/7WHhwceHh7PWNKc1bdW34f+jrkbQ1hMGGHXwgiLCWPBkQWEXQvjzI0zFLEugru9u2Fz\ncKe8Q3nc7d0pal00x9Mz9vb3p1toKAvj46kLzMqfn7H+/k88jvh4AIbodAT6+6NpGpPrulD2oI7e\nXcL4Zmlj3ujnwvUfJnLnzm7GjFmMt3dBevQoTfnyYXz0UQNsbGxy9B6Vl1t2/T8jWRxptGvXLlJS\nUhg4cCAAPj4+1KpVCwAXFxdu3rz51GtcvHiRo0eP0qFDBy5fvsyOHTt46623qFixIuXLl3/ieSEh\nIYSEhGSpvOkZfRiopmmlgNUiUlnTtKEAIjIh7b0gYKSI7H7kHDF2uXJbij6FszfOPhQc7r2OS47D\nzd7tseBQzq4cVvkeTw+XXYKDg5kzZQrNIyLofPs2VgcOQOHCTzwODAHh0XH8a8Lu4H3yCNOOF6TK\nrBhsly4k1jSEypVXYWrqxpgx8NNPiSQltWfkSE+GDw9QcwOUp8qrw0CzIyXkd999x9ChQ4mPj8fE\nxNAw06ZNG5o1a8aAAQMeOz5PzwPQNK2IiFxOe/0pUEtEOmmaVhFYCNTG0PSzEXB99Gn/KgSA/3Iz\n4eaDoJAuOJy+fhpHS0fcHdwfCg7u9u64FHTBRMvGFr2RI2HNGjYOH87EHxYAWZu5G5GQwNtHjuC9\ny5ym4+IosmIfl/maChV+w87Oi7VroXPnRFJTRzFxYklWrtyQ5c9QXi15NQBs2bKFTp06ZZgUvlev\nXplKCblp0yZatWrF3bt3MTU1BQwBwNPTk/79+z92XnYFAGN1Av8KHAYOAX8CTune+wLD6J+TQPMn\nnJ+pjpDJDm1IAAAgAElEQVRXTUpqioRfD5e//v1Lvtv5nXy0+iPxmO8hRSYXEcuvLaXq91Xl3T/e\nla82fyULDi+QvZF75U5i5jqVHqPXy4WWLWWzST7Jx0/P1EF7OzlZWh06JH2m75FtjqESsXWtbN/u\nLBERU0Wv18vp0yIlSlwXnW6ZHD165tnKqbz07o0Sy6vPhexICZmcnCyurq4yduxYSU5OltDQULG2\ntpawsLAMj3/Sd0EWO4HVTOCXxO3E2xnWGk7FnMJOZ5dhraFEwRKYmpg+8ZotPL3ptfECqZTBj0Xo\n+f2JM3ef1CyUotfT79Qpbmy8Qf/RKbgtsuaCTVdsbOpSrtwsEhLMaNw4jIMH9YSG2lOr1uNNTsqr\n6+FlxbvnyRoAwL59++jVqxenT5+mVatWaJpGuXLlGDNmzNNPTnP8+HE++OADDh8+TKlSpfj6669p\n27Zthsfm6Sag56UCQPbRi56IWxGcvHbyseAQczeGsnZl73c+pw8OBS0K4uXlw9YNLVjHQo5TkY+p\nhafn6scCQHBwMN3atycgfcfwypX3g4CIMOHCBYJWXmTM10KFX4sTVawPoFGx4hJMTa1p2TKYzZur\nsW2bNXXqWOb016TkUQ8vH5I3m4BygwoAynOLTYrlVMyp+4HhZMxJwq6F8W/MvxTIV4DCJoU5vu00\nVpEt2XQ4lBCz21T8YxmtWr710HV8vLxos2FDuhV+YJWnJ8vXr79/jF4Prw+JRl/kX6Z+o1HppzLc\nqDCSO3f2UbnyGvLlK0LDhrPZs6czwcE2NGqkVilRHgSAwoUbc+VKSRUA0qgAoBiNiBB5J5Kwa2Gs\n3LaSVTuCyM8t/rfqBisqwEIfN8PIJPvyuDu4M3/sbN5Zt4d+hgpAhgEAICkJmn12g5gyx5j2tVB5\nthsJtX7h0qUfqVLlL8zMXKlVaxjh4WNYsKAA7drl/L0recu9JqDPPy/B6NF7VABIowKAkvOio9E3\nakR0h+aE+r1xvylpd/huTl8/RYEUcI6BiJumdG7dnTb12+Bu706ZQmUwNzUHDMsOtfs0ljNlDjF1\nvJ6qM9yg8UZOn/6USpX+IDX1NapX78WtW78zebIlvXrl8j0ruW79+gncvTuW9u3vqgCQRgUAJXdE\nRkKjRoaVRD/55P7uoKAgps/4hlhdHNXa1MfMyYywmDBOXjtJ5O1IStqWvN/PUM7OncWL3LlazISJ\nAalUm+KGxVvHOH7cDze3Ody8WYl69bpiZraZL77QkcEwaOUlFhwcnC5p0PsUKvQ5pUqNxcnJRwWA\nNCoAKLnn/HlDEBg6FD766KmHJ6Ykcvr66fsB4V6fw/HrEThajmLiyFIcfe8f7DpEUUl+x9bZn5uX\n6vOOz2dYWOzkyy/z0adPDtyXkuseHvUDfn4f88EHbsz8YhArd7+nAkAaFQCU3BUeDk2awOefQ79+\nz3QJEeHCnSj8g8Po1E/PmS7HOVxnOW1sQ/lfpLB+kx13ggtjylbaf3SEnj3B3cEdJyunHF8qQ8kZ\n6Uf9FCoUxdy5bkz89HN2n+uCnlIqAKTJrgDwzGsBKa+4smUhJASaNjX07n76aZYvoWkaJW2KsLSD\nMyNsT/J6F1Nalnkb17ZmuBzyon/N2ow1ucrRw74s/34ROy99z3V3b1L1qRnOayhnXw4LM4unf7Dy\nQujdeygzx41hz7m3GWj7J989fUkdJYtUDUB5PhcuGIJAz56GJqFnJCLM3HaGIh0jKDW0JDEV9nH9\n6gdcjbZk3u+FqVmzC3/95c+kSeDV7lqGk97O3jhLUeuiGQaH3FhgT8m6e01ApUr1otpr+Vj1xzu8\nmT8A+0kfsWBAXVUDSKOagJS849IlQ3NQp04wYoQh09gz+Ocf+GD8BYbvOsPqfCvxujodx68hNMqE\nub85MmjQL3z77Vv89BO0bv34+ekX2Dt57eRDweFu8l3c7N0em/TmZu+GpbmaeJaXBAX9xS+zjrF1\nQ0uszHswqc5UAgIs2FOz5gsTALp3746Li0umF4PLKtUEpOQdRYvCli2GmkB8PIwf/0xBoHZt6PpG\nCT6PiCPgYltsiuTHa+gkZJSefL0SmTixO1OmbOP998uzahXUrfvw+WYmZpSzL0c5+3K0dns4QtyI\nv8G/Mf/eDw5Ljy8l7FoY4TfC7y+wd29ew73gUNymePYusKc8lV4PC6fmZ++hpqSYt2Wc2Tg+eUdj\n+F1X9uR24bIgqykhFyxYwEfpBlTo9Xri4+PZt28f1atXN0YRAVUDULLTtWvQqhVUqQI//ABmz/b7\nomr52dxNbMS4+Ciw305C2EgSpztx6YqOgIBoXF0/JzJyGLt25cfN7fmKnKpP5fyt8/drC+lHKd1K\nvEU5u3IZBocC+Qo83wcrj0lKgvfqHeFkfAxxdz7kQ+eepLp5MqV2fq4OqIiJyYuzFESPHj0oXrz4\nM9cAAgMDGTduHKdOncrwfdUEpORNsbHg4wM6HSxaZPhnFgUHB/PO22txLPQx47QLrC95jLJtbhMX\nO579+3Vs3OiIiUlLHBzGc+CAJc4ZpSXKBrcTbxtqDY8Eh4wW2CvvYAgQJQqWULWGLLi3iGBKSj5u\nnRyLVugSKTaf4u7YkO5He9Jscgrz7Wrh62GRZ5eDBjhw4AA9e/Z8aDE4V1fXZw4AjRs3pkmTJowY\nMSLD91UAUPKupCTo3h0uXoRVq8DW9qmnPCo4OJgvBx/AuVAUnc+1JbTSVWZvjKPvh3vZvPl3Tp+u\nRYkSPbGx8SMkBHIyuZhe9Fy4deGxjuiT105yPf46rnauGQYHm/wqA1p69xYRHBmfj59Zg7N9OAWa\n9Cc6qhpfX/6GOb5WbLOw4czwMkDezQeQlJREuXLlGDRoEB9//DF//vknfn5+DB06lF69elG5cuUn\nNgd9//33+Pr6PrTv/PnzlC1blvDwcEqWLJnheSoAKHmbXm8YGvr33xAUZOgneEY3I+NZXWsbf1fQ\nmLe9ET06fcLq1XMoUaIKNWvu5+xZWLsWzM2zsfzPKDYp9qFaw70A8W/Mv1jnt35sdFJ5h/KUsi31\nn8tyv6x8vLxosuEQPxFMI5O/sfzQn9m/m7Oxeyhx58xp2v0uISVq82ZNQ1Pi0wJAdg3yyuqjZ+vW\nrfj5+WWYECYry0HfM3bsWP7++282b35ylm7VCazkbSYmMHUqTJgA9eoZagJVqz7TpWyL6bD7Tnij\nfxKmHlv5eUVdOnjOY8OGQ/zwwxZGjmzEgAEwe3b2PQSeVYF8BahRpAY1itR4aL9e9ETejnyoxhAc\nHkzYtTCi46IpU6hMhsGhkK5QLt2J8ZnftmEqW+jMYqq9NZpugeBVxoukpUmM/VXjo4TS9x/+mZFb\nvxkvXbpEsWLFHtpXsuSzr1z666+/8uWXX2ZH0Z5K1QAU41uyxLB20Lx5GY/fzKSgpcFED0xhf01T\nqn6scfPYj0yduobNm4/Stq0bffoYPuZFE58cf3+E0qM1BwsziwznNaRfYO9FFP5XGM3etqASM2lV\nZDKjkyBfSn4Cbf4i+vOiTKsTx44aNTBJF9HzahNQdqSEvGf79u00b96c6OhorKyenP9bNQEpL5Zd\nu8DbGz77zLCI3DP+VJ82LpkSC/axt0ISjr0TWPiVLyK36dNnKcOHexMYCF5e2Vz2XCIiRMVGZZjM\nJ/J2JKVsS2UYHBwsHfL0pLfjP27Dq29ZRnS/SNF2lxns74NzYVtGlFxMwduOtBpyh7+rVeO1Ag+P\ntMqrASA5ORlXV1f8/f3p06cPq1evxtfXl6FDh2a5Cah3794kJSUxf/78/zxOBQDlxXPuHLz9NjRo\nADNmPFOjfUgIdOuQQoDtDg6USmBS7DXswwfi4RFD3brrCAhozpYtUKFCtpc+T0lISSD8eviD4JAu\nQJhoJo81Jbnbu1PWriz5TPPlXqFF2D90KW9N9mDS8Ft0GuWKt3dtYmPPsizgFMdanuD7JdaUKWXN\nmNKlHzs9rwYAyJ6UkAkJCRQpUoQVK1bQuHHj/zxWBQDlxXT7Nvj6wt27hqYhJ6csX2LnTmjeKIZR\n5oe5XsOU8RZW5A99k8GD9bi4XGTiRAd27wZ7eyOUP48TEa7evWrI8JYuOJy8dpKIWxG4FHR5KCjc\nCxKFrQobt9YQH8++jgG0WtefH2Yk076PM4MH9yE4+Gf+3ribcy30RL1fkMG1r3OwZk0sTB/vFM/L\nASCnqQCgvLhSU2HUKJg/H/744/EpvZlQt+5gDuwexej8p0iqfIcxztvJv/krAgMLs3PnMfbvt2X9\n+rwxMiivSEpNIvx6+GMpQMNiwh5aYC99cHC1c33+BfbOn2dfi+G0OjuTOfPz09ZXx/fff8+ECYP5\n889BFFrbk5jtt/AaHMso00r098h42LAKAA+oAKC8+Fatgg8+gNGjDXkFsvALNDg4mLZtv0afuIQv\nzQ5iUS6VXxoGY7ptJdOnmTB58gnc3KyYPt2I5X+JXLv7+AJ7J6+d5NzNcw8tsJcancqhNTspEG/J\ngI+/oEWLFv994U2b2N9xAi2T/mTOb5a0baexdu1a3n+/Mz/+6ELjols42voECxcUYl6IGVvauVGr\nVsaXUgHgARUAlJfDqVOGzuHXXzeM47TM/MJswcHBTJ48BxPRGKz14WpMfgY5jKdOgRj6fniH/v0P\nMmxYPnr0MGL5X3LJqcmcvXmWsGthrNqxigWr5+Fim8plB4g1A1e7ctQqU+uhSW/l7MthaWoBkyax\nf9ImWqau4cd5+WjXztBW3qKFF+PGpeLb7m9O1E8kYaQzbztewnt1bQJnP3nYpwoAD6gAoLw84uLg\nww/hwAHD8hFVqmT5EqIXTg04xaHNEXRN7MOArtWpWi6KAQM2sXatGXXqGKHcrxgfLy/abNhAt7S/\nZ1vAHy1r0f2rfg91RMddOE3gSo3o+Fr0uLUav+Hb6ehjTv7b+fFr7cfgwc60a9eG+C/eQWxMeKvz\nLW4ElOH0fEccHZ/8+SoAPJBdAUAtWqLkPisr+O03GDLEsKLojBlZntWjmWiUm1GO171L80Pq10yZ\nupEr2ut89tkH+Pgkc/mykcr+CrNKALtYW7pV68b4puNZ3nE5R0tP4kxgIcwafMmH8evpMGQrphVX\nMXL1SJo0a4JD02gKVzhO4ORkzuw+wwDfM8QeTmZkS91/PvwV41A1ACVvOX0a/PwMo4PmzSOrTwUR\nWNPlAifWr2B48limrfiEvb+Zsnb129SrOpQ+nw2gefPmRir8y+3e2j0B8fEADNHpCFy50vB9JibC\nsGGwbBlnAv6g0ed1+OYbeO89iImJoWHDhvj5vY2Hxy+YpkzhrncxFk4+wxIXB0y3jCdZd+yxBfbc\nHQxNSvcW2FM1gAdUE5Dy8kpKMiSW+f13+PHHLM0eTk6Gtm2h6qXL3Ar/ml8LrafO+yVJDOqD+YkY\nTiQPfPDQUrLs3uqdAL39/Q3f4/790KMHlClD5Ni5NGxbCH9/6NsXYmNjadasGW+++QadOu2mUMG3\nuNq+Kfl7F6ZZtYsEV6lCdWvrxxbYSz/57d4Ce0f6Hsnlu89bVABQXm4hIYZUk/XrG9YVyuTA/qQk\nw/NIt/8q4WEdCXdNoO2H7qyZ9hlNk6Zzo8pZlq9fb9yyvwoSE2HcOEOQ/vZbrjXvTMNGGl27GrKD\nJiYm0rp1a0qWLMnw4SW4dWsLujmzSLqaQq8hCfg6OfGJi8tTP+beAnuPZnn7N+ZfbPLbZDiv4VVd\nYC/HAoCmae8Ao4DyQC0R2Z/uvWHA+0AqMEBE1qftfx2YD1gAf4nIwCdcWwUAxSAuDr780jBpbOZM\nw4ihTNDrwd8ftv2wn1sJ71C2QmXMP6jBtjEf0bTBMJav/eWxczL6dRscHMyUKXMA8PfvrWoO9+zd\ne/9XPz/8wC3LIjRpAs2bGxLCGYZ69gZg/vxB2NhMonjYWqImprB2iT37tLus+Y9lkjPj0QX20k98\nuxJ35f4Ce48Gh5d5gb2sBgBE5Jk2DA9+N+BvoEa6/RWBg4A5UAo4zYNA8w9QO+31X0CLJ1xbFOUh\noaEibm4i77wjEhn5xMOCgoLE09NbPD29Zd26IHn//TApxe9ih628X7q7lBvxm9jbR8qxY9seO89J\np5P5IPNBnHQ6GTdunOh0TgLzBeaLTuckQUFBxr7TvO32bZHBg0UKFxZZsEBEr5f4eJGGDUX69hXR\n60XWrl0rpqb5BaqITjddFi40lfVzRkpo4VDZuOeyFN2+XaITE41azLikODl4+aAsObpExoSMkc7L\nO8vrP74uBcYXEMeJjvLG3Dek5/96ysTQifK/k/+Tk1dPSlJKklHLlBPSnp2Zf45n5eAML/B4ABgG\nDEn3dxBQFygCnEi33xf44QnXNOJXpLyw7t4VGT5cxN5eZNIkkUceIkFBQRk+sP/8c714VW4oBSko\n3zaeL6V77JQKFXdJzw8aiaentwQFBYm3p6fMN/Qhi6QFgZJ2jmnXurd7vnh6eufSzecyvV5k8WKR\n4sVFunUTiYoSEZHkZJH69aPEyWmbNGvmI2vXrhVnZxeB1wTiZdiw92T4oMayOt8aObHmshTZvl36\nzY159F9dDt6GXiJvR8rmM5vl+z3fyyfrPpEWv7eQ0lNLS/6x+eWvf//KnYJlk6wGAGPkAygK7Er3\n90WgGJCc9vqeyLT9ipI5Op2hzblbNxg4EH75xTBktFkzAKZMmUN8fACkjVSPjzfsW79+OW3bejJn\nxhzGDRnBhBtT6WtRgYhLXbmbeIj27btRu/yzJ6x56Z04YVhn+9o1wzyNN94ADCHx7bcvsmvXafT6\nCKKj3+Lvv33Q6fIDI2jWbDkVyu8h5cOJhLoe4IvChal10oXD8+ww6/bfH2ksmqZR1LooRa2L0rj0\nwwuuJaQk5E6hctF/BgBN0zYAGWVc/UJEVhunSAajRo26/9rDwwMPDw9jfpzyIilXzpACbPVq6N0b\nqleHb7556mm9+/fm1LlTfDVzKF8lzWC4gw/dOh9F59CemLNHGKLTGaIGhiGO/QcN5Ouvh9zbhU43\nBH//QGPeWd5y9aoh4C5caBiV1bcvmD14ZIwYATt2JKDXRwKdgQ9JTS1JmTKOJCUNo1+/JM4O/pqo\nlL1smdaIMgk2bPcvzu5dhnxBec1zr3mUC0JCQggJCXn2C2SlupDRxuNNQEOBoen+DgLqYAgk6ZuA\n/FBNQMrzuntXZPx4EQcHudCqlZSxcPjPNvuUlBRxdCgiZWgp37JPzPMnyOjvWkv/SR73m4K8PT3v\nn5e+T+Flaf9/6j3duSMyZoyhqa1/f5Ho6McOmTbN0CXTqFF3gbkCHwrUF/heWrduLZs2FZU5fn4S\naPu7dPxrgzTYfUCKl0qVP//MgRt8hZFLfQCvp/v7XidwPqA0EM6DTuDdacFAQ3UCK9np2jWRzz6T\nRGtrWVTKTdp7tH7iA3vRov8JlJMyDJIvOSD5zW/I9EU1Zf7BwZKSmprDBc9ZT+onERFDn8rs2SJF\nioj4+YmcPp3hNRYtMnQFnDt3r8PXQsBN4HuxtCwsGzfWkX2LusvuCrtl/pELUnbnTmnUJkk+/zwH\nb/QVlWMBAGgPRADxQBSwLt17X2AY/XMSaJ5u/+vAkbT3pv/HtY37LSkvr4gIkQ8+ELGzExk69H5n\n5aPmzv1dTEwcpFT+qdLL8rSUd74jgcGVZOLOrhKTWz2UOcDT0/uxju23m7QR+e47w1O9RQuRffue\neP769YYBQEeOiCQlJYmvr69Ur15dmjRpI56e3rJhQ2fZtba27HDdJiHHosUxNFT2XomVESMMHcaK\nceV4DcAYmwoAynM7c8YwLrFQIZE+fUTCwx87ZM+evWJl5SglCm0Sz0Ix4uVyRZZvqSpjtrSSrdev\n5EKhjS99ALDlunxJe7lhnl+kQweRvXv/89w9e0QcHUW2bhVJSEiQ9u3bS6tWrSQ+Pl5ERK5eXS3b\nNhaV0IqrZPeBK+IYGirrY2Jy4raUNFkNAHmwK0ZRskHp0jBrlmEES6FCULs2vPuuYXax4UcGNWu+\nzp9/LuSW3o9bZS9yKcGaTe+uoD4J7DvchrFnjpGi1+fufWQzf//e1Mo/iNk0IRwX3EzXcXT2TENi\nntdff+J5p04Zsnn+9BPUqpWAt7c3IsKKFSuwsLDg7t1TnDjcAxn1Ffnn1uHtuH/52d0dTzu7HLw7\nJcuyEi1yakPVAJTsduuWyIwZIhUripQvb2jySPt1unjxYilevLjs2HFBnG1TZELBI7Jr9buyYGtF\nabF3o1xI+4X7QktIMEzcatBA4u3tJbBMBfF9s2WmOrYvXRIpXVrkp59Ebty4IY0aNRJfX19JSjJM\nnEpMjJbtm8vIVt/PZX9IlDiFhsqyKy9nDSqvQzUBKcp/0OsNbRidOokULGj45+rVMv3bb8XNzU3W\nrbsu9rapMq/4Idkzo6+s21pSKm9bLIujo0Wv1+d26bMmNdVwr336GNpumjYVWb5cJOnxGa9PGhl0\n86ZI1aoi48aJREZGSpUqVaR///6SmtZZnpx8R3ZuqCZb+nWXA0GGiV6/XoySu3dz7C6VdLIaANRi\ncMqr6+pVWLrUsM7QsWN86eJCcHw8A7/Yx9AvrJhf/V+sSgSS+O4vTMw3HiubuswuVw7HfPlyu+RP\nJmJYnXPJEli8GAoWNCyv7etrWLcnA8HBwbRv3y1tEp1hvsPKlYG8+WZzmjc3TLPo1+9fWrRoTu/e\nvRk6dCiapqHXJ7N3nRcJ/xSgkOcvtDA5zfhSZVj7iTOVKsHIkTl54wqo1UAV5dlERCBLl/JhQABh\n16/j6f4zc86/zaQm0RRNXIt8HsB+x2EExNZjZrly+OSl7CU3b8KGDfDXXxAUBNbW0KGD4cFfufJT\nT/fy8mHDhjZwP9dXIE2b/kW+fEtwcIC+ff+hffu2jBs3jp49ewKg1+vZv/wd7kZGI28s4d3Ec3xT\npgz7xxfh6FFYtw4sXrx5VS88FQAU5Tno9Xp6d+lC2J69lIyfSfRFmGX1LdddW5A04huwf4veFh9Q\n3aYg01xdKZwbtYHISNi507Dt2AHHjsGbb0LLloatbNksXe7xAPArTk5lqFPnDXr0WEuvXt355Zdf\naNOmDWBIv7n/l4+IM9vBpeor6B93hd8rVGD/j3YsXgxbtxoqHkrOUwFAUZ6TXq/nww8/5PjxU5ib\nbeB2+HV8UxbTyuk2MT0XY3k3hmWmfZlXpQ5jb9+ml7MzJhUqgIND9hYkNhZOnjSMZLq37dsHd+9C\nvXoPtrp1DeskPaNHm4BMTTVee+0t3n33V2bOnMzy5cupW7cuACm3Utg/azAJ5f5kt+syZiUks7py\nZXYuLMA338D27VD0CcsqZZhMRslWKgAoSjbQ6/X06dOHgwfPEhu7jgoVTNmyBf4YfRXTa++juR1D\nF9mVAcUrkJKYyIwpU6hw/DjXLSwo4OqKfcWKhrSWjo5gaQn58xvaRPLnN6ynk5BgWHfo3nb7Nly+\n/PAWG2tY96hChQdbtWqGfc+xjn5G7uU9OHWqM+bmjalTZzhHjuxg1apVlChRAoA7h+9w6NdPkIYh\nLHb5mX/0Nqx67TWc8+Vn4EDo399QtCdd/4npJJVsowKAomQTvV5Pv3792LMnkpiYlXTvbsqQIWCW\nksKRX0Zzq9RMitydwo92Lky7G0uzv/+mS2AgqxMS6PT220QfDsM2KZE6VStQ0tnZkEErIQFSUgzB\nQKd7sFlbQ5EiD2+OjmCac1mtRo2CxYtTsLVtR9Gi+ejatSu/zZ4NwEfVBpM/dS54HWZEoanYWxbl\ntwoVsMpk+Xy8vGizYUO6XgZY5empMrNls6wGAGMsB60oLwUTExNmzZrFJ598QnDwW/z44xrKljXj\nvffMqD5wLFF7GhN28T2aBFWj0txI9r3fhE8DA2m8ZAld/1zD7YSZAOiuGEbV5OVfu6NGwW+/xZOS\n0ogOHbxo0KABPXx8CIhPwU7rS9JrAcR5RvKJ1XcMLVaJD4sWfa5sXkoekZUxozm1oeYBKHmIXq+X\nKVOmSOHCHmJvnyQrVz54LzHhmiydXlY2z3ST44W7yfGiFeT10aNFW7JCaHlJMNHneiKZjFY5vUev\nFxkxQi/OzjFiZ1delixZIiIi3p6esoiqstNkrmz7opr8srK0FFn4k5yMi3vmMjyace1lWV01L0FN\nBFMU41i+fLnY2jaRggUTZP16w76UFJEKFWKkd6d6smm5pWxq7C+9ta6icx8vTNsvzNstNPtDmjbv\nkCtl/q8Hr14v8tln8WJjc14qVWosp06dEhGRpJgkmVn0O9lQ8CcJmlNcvg2uLt4fdJF2zZvL5csi\nH30kzzTR678CkZI9VABQFCPavXu32Nu3FSuru7J5s2Hfvn0iZcveknp15svyRYVk3dTastJ2nnRg\nuZi/vlhMps6Uwps3y/SICIlNSXnoesbON5BRqktvT0/R60V69rwk5uYnpXv3zyU+Pl70er1ELYqS\n7c7bZekHs2VxsL18vqSZfFXeTZx0Opk8eZcUKyby1VeGwKfkPVkNAGoxOEXJgtq1a7Nnz3c4OPSh\nZcvbLFx4lxo14MQJG9r7dKPfp5eIsGyMzdJBtG+znmVHrQg+6M7/Slfl75s3Kb1rFyPPnuV8QsL9\n4ZcbNrRhw4Y2tG/fjeDgYKPfQ0oK1K69l/nzY5g16yRz504gdn0s+2rtI2z8OTbNXI9F5y+4atKX\nUz9rHClekrc7H2DSpDr8/DOMHp2jfdOKEalRQIryDOLi4ujSZSqrVvVi2LA7jB1rmHwVHW1Ilejt\nvRs7u+7kS3bHfNEAbizIh2MHR5J72fOD7XUWXbkC/57h+u86CPWGZFMgEE/PVaxfvzzbyvno8MtP\n8xUiwXQpVlZ2hIQ4UviUBefGnOd2UgqreyZjVeNbGpodpm6V/1HQ+jWSk6FjR7h0ybBgaNqIUCWP\nUsNAFSUHzZixnk8+KU/z5v+yenVjTNP9NE5NTeDChQlERs7AybYXpqveI2rmbXRuOuzedcT34I/s\ncuP8mskAAA2YSURBVK0PbtawzRF2HqKJ7XI2rV2SrWUMDg5m9sSJHPn3Jhcuz6FerYIEdsxP1LzL\nxOhT+OU9PTYeh+mSOhnngrWoUH4OZmYPpvLOnQudOxumMCh5mwoAipLD/vnnMo0bJ2Fvv42//qrB\na69VfOj9hISLnD07nOvXN1Ci+Cjy72vD1aUxXFl9hQMJR9hkVZzQxhqxTSLJX6USDe3seMvenqaF\nCuFuaYnpcwy31Ov1LFmyhM8/+4kbMQto5ZTABzEXOPqmKaua6qnawpR3U2egxe3CzW0W9vZvPe/X\noeQiFQAUJRdER6dSt240kZFH6dJlIxMmfIbjIwvGBQbuJSlpECVLXqd8+c8oYvcOId9s5/TP5ygT\nUxLzYmY4ehXlXDVTNrgnsq5ALJeTkqhcoAA1ChSgeoEClLO0pLC5OU758mFrZoZJBsFBRLhwNY7F\n01fww5zxJNxuRmxiAG+VvIhNj+sU9nakaTEbysavIOLcVzg7d6dUqZEkJFhhZZVT35hiDCoAKEou\nSU6GTz6J57ff4v/f3t0HV1XfeRx/fyGgl4aHKNSkCU/dEgXpusF1dTBuGDDGhVaFVVfdFspaU3wY\nus6tS5XOwu5MuzpbqVpHp661G8ciI7TUuIpId0nlwQYVVlGUAqtCVBKoIRRKIA/f/eMcMdKQ5t6Q\n3HN7Pq+ZO3Pu75yb8w3c/L7n/B7OD7PrWbhwGvPnz+f08LGY7lBT4yxf/gLFxfdy9tlvcOaZt1FS\nMo/+NozDrx+maX0T21ds50jt78lpzyFn9GnkjM+jIR/+b0Q7dYNaaWpvpdHbOEI7uQNyGHwIhta3\nM6zBGVbfzoHdb/Bf+x6jPucwZ3z2SRqav0jywd/x9ZmDyet3lA8/fJS6uvtIJD7PF75wH3v2nMeS\nJVBdDbt2QW5uhv8hJW1KACIZ9vTTcOONrRQUVNHU9K/Mm/cN5s6dS0FBwfFj3nsPli17naNHl1BW\n9jT5+dcwYsTVvPxyC3NmXsM9R47Qj6E8fNpo/uVb3+fsYWfT/E4zrU2t0Abe5rS1OUdb2rCh/WkZ\n0cILH6ziqU1L+X3LYb429x5WrLyG8eONRx6BROJ96uoe4MMPHyUvr5yioiRbtlzAkiVQWwu33go3\n3xw8fUKylxKASAS8806wBHEi0Uhh4T2sWvUjysrKqKyspKKi4nhncVsbtLZ+QH39E+zfv5KG+ldo\nXdfKhS9C3qvw0+aTPzOnpaWFTZs2UVVVxfLly5k6dSo33TSPXbumsXixsWjRe8yY8TiNjas4fPgt\n8vNnU1R0O4nEWBYsCJYPuO02mD27Rw8TlQhRAhCJiKNH4XvfgwcfhLlzjzJq1JM88cRD7N69m9LS\nUi6++GImT55MSUkJA8N1BWbP+muu9HXkXzKIQxOcpn1nsGdvIedMmEFR0UQOHjzChg2vUVPzMi+9\n9Cpjxozmqqsu5dprL+SVV/L47nf/giFDPuL2229i1KjfkpMznXPOmc7QoaX06/fJMJ5jxyDKC5tJ\nepQARCKmrg6+8x1YvTp46NrUqe9QW7uBjRs3snHjRnbu3ElxcTF5eXk0NzezubaWS9raaLZBvDYo\nl8HDTufQoQMcOvQ7RowYyOTJg5g0aQAlJcaQIcfYvn0KP/zhIvbuHU5p6ToSiRbWrStlwICx3HBD\nMHFL4kEJQCSiNm+GZDJY0OsrXwmW6S0uhoMHD7J9+3aamppoampiw4YN/PLZZzHgS1dfzfTp0yks\nLKSgoICBA0/DLFg+oLoali6FrVvhjjtg4kQYMgQ+8xkYPTpYhkDiRQlApJf1ZGUrd/j1r4P12p96\nKlg967rrYMYMGDPm5JX2c8+t5u67l9PYOJHc3BvYtu2zlJUFs3RnzVJlLwElAJFedCpXtmprg1/9\nKkgGa9cGTUW5ucHV+6hRMGBAULZz5xEaGvoDbcB+Bgy4h2XLZjJr1rRT+8tJ1lMCEOlFvbmyVXs7\n7NsHu3cHw0SPHoWRI2HhwptZv34y8NXjZy0vryaZrOTeex8BIJmsjPSCM9I3tCKYSJbq1y9YRvis\ns+CCCz4pTyQagPZPHbt/f/2nFnJfv35O5Fcdk+hJ+3HQZnaNmb1pZm1mNqlD+RgzO2JmW8LXQx32\nnW9mW81sh5nd39PgRfpaZTIZNPsQXP0vSCSoTCZ79ZzJZCWJxILwjFXhdk5Y+c8BgkTw8d2ASHf1\nZD2ArcBM4MVO9u1095LwdUuH8oeBG919HDDOzC7vwflF+lxFRQVVK1dSXV5OdXl52u3/qZ5z5cqg\n2ae8vJqVK6sYPvzMXj2nxEPaTUDu/jbQ7YWhzawAGOzum8Kix4GrgOfTjUEkEyoqKvq8qaWzc65f\nP4ewL5pEYgHJZFWfxiTZr7dWBBsbNv/UmFlpWFYI1HU45v2wTERS1Nldgdr/JVVd3gGY2Rogv5Nd\nd7n7Myf52AfASHdvDPsGfmFm5/YwThE5QWd3BatXr9bIIOm2LhOAu5en+gPd/RhwLNzebGa7gHEE\nV/xFHQ4tCss6tXjx4uPbU6ZMYcqUKamGIhIrH68xrJFB8VFTU0NNTU3an+/xPAAzWwt8y91fDd8P\nBxrdvc3MPk/QSTzR3Q+YWS0wH9gEPAs84O5/0AegeQAiqbvssr9lzZoroMMshVO9xrBEW6rzAHoy\nDHSmme0BLgKeNbNV4a4y4DUz2wIsB77h7gfCfbcAjwI7CEYKqQNYRCRDNBNY5E/EiU1AicQCNQHF\njB4FIRJRfdFBq07geFMCEIkgXZ1LX1ACEIkgddBKX+izTmAREcluehqoSB9IJiv16AaJHDUBifQR\nddBKb1MfgIhITKkPQEREukUJQEQkppQARERiSglARCSmlABERGJKCUBEJKaUAEREYkoJQEQkppQA\nRERiSglARCSmlABERGJKCUBEJKaUAEREYkoJQEQkppQARERiSglARCSmlABERGJKCUBEJKaUAERE\nYkoJQEQkppQARERiKu0EYGb/bmZvmdlrZvZzMxvaYd+dZrbDzN42s8s6lJ9vZlvDfff3NHgREUlf\nT+4AXgDOdffzgN8AdwKY2QTg74AJwOXAQ2Zm4WceBm5093HAODO7vAfnj6yamppMh5C2bI4dFH+m\nKf7sknYCcPc17t4evq0FisLtK4En3b3F3d8FdgIXmlkBMNjdN4XHPQ5cle75oyybv0TZHDso/kxT\n/NnlVPUB/APwXLj9OaCuw746oLCT8vfDchERyYCcrnaa2Rogv5Ndd7n7M+ExC4Fj7r60F+ITEZFe\nYu6e/ofNvgbcBExz9+aw7NsA7n53+P55YBHwHrDW3ceH5dcDZe4+r5Ofm35QIiIx5u72x48KdHkH\n0JWwA/cOgkq8ucOuamCpmS0haOIZB2xydzezg2Z2IbAJ+CrwQE9/ARERSU/adwBmtgMYCHwUFr3k\n7reE++4i6BdoBb7p7qvD8vOB/wQSwHPuPr9H0YuISNp61AQkIiLZK7IzgbuaaBZVZnZ5OPlth5kt\nyHQ8qTCzkWa21szeNLM3zCwr787MrL+ZbTGzZzIdS6rMbJiZrQi/99vM7KJMx9Rd4eTPN8OJnkvN\n7LRMx9QVM3vMzOrNbGuHsjPMbI2Z/cbMXjCzYZmMsSsniT/lOjOyCYCTTDSLKjPrDzxIMPltAnC9\nmY3PbFQpaQFud/dzgYuAW7Ms/o99E9gGZOOt7f0ETaPjgT8H3spwPN1iZmMIBoNMcvcvAv2B6zIZ\nUzf8hOBvtaNvA2vcvRj47/B9VHUWf8p1ZmQTQBcTzaLqr4Cd7v6uu7cAywgmxWUFd9/r7v8bbh8i\nqHw+l9moUmNmRcB04FEgqwYShFdrl7j7YwDu3uruTRkOq7sOElxADDKzHGAQwTyfyHL3dUDjCcVX\nAFXhdhURnqjaWfzp1JmRTQAn6DjRLKoKgT0d3n88AS7rhFd0JQRfomzyA4KRae1/7MAIGgvsM7Of\nmNlmM/sPMxuU6aC6w90/Au4FdgMfAAfc/ZeZjSotZ7l7fbhdD5yVyWB6qFt1ZkYTQNjetrWT15c7\nHJMtE82yscnhD5hZLrCCYPTWoUzH011m9iWgwd23kGVX/6EcYBLwkLtPAg4T7SaI48zsz4B/BMYQ\n3DXmmtnfZzSoHvJgdExW/k2nUmemPQ/gVHD38q72hxPNpgPT+iSgnnkfGNnh/Ug+/eiLyDOzAcDP\ngCfc/ReZjidFk4ErzGw6cDowxMwed/fZGY6ru+qAOnd/OXy/gixJAMBfAhvd/bcAZvZzgv+Pn2Y0\nqtTVm1m+u+8Nn13WkOmAUpVqnRnZJqAOE82uPGGiWVS9QvCE0zFmNpDgiajVGY6p28Intv4Y2Obu\n92U6nlS5+13uPtLdxxJ0QP5PFlX+uPteYI+ZFYdFlwJvZjCkVLwNXGRmifB7dClBR3y2qQbmhNtz\ngKy6CEqnzozsPICuJppFlZn9DXAfwSiIH7v7v2U4pG4zs1LgReB1Prn1vdPdn89cVOkxszIg6e5X\nZDqWVJjZeQQd2AOBXcDcbOkINrN/Iqg024HNwNfDwRCRZGZPAmXAcIL2/n8GngaeAkYB7wLXuvuB\nTMXYlU7iX0Qw6ielOjOyCUBERHpXZJuARESkdykBiIjElBKAiEhMKQGIiMSUEoCISEwpAYiIxJQS\ngIhITCkBiIjE1P8DZREfrD4TZBIAAAAASUVORK5CYII=\n",
"text/plain": [
""
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"plt.plot(xs, g(xs), '--')\n",
"plt.scatter(X_train, y_train, c='b')\n",
"plt.scatter(X_test, y_test, c='r');\n",
"\n",
"for degree in range(1, 8):\n",
" h = PolyRegression(degree).fit(X_train, y_train)\n",
" plt.plot(X1, h.predict(X1), label=\"d=%d\" % degree);\n",
" print \"train_error(h%d) = %.2f; test_error(h%d) = %.2f\" % (degree, error(h, X_train, y_train), degree, error(h, X_test, y_test))\n",
"plt.legend()\n",
"plt.show()"
]
},
{
"cell_type": "code",
"execution_count": 32,
"metadata": {
"collapsed": false
},
"outputs": [],
"source": [
"train_errors = []\n",
"test_errors = []\n",
"degrees = range(1,8)\n",
"for degree in degrees:\n",
" h = PolyRegression(degree).fit(X_train, y_train)\n",
" train_error = error(h, X_train, y_train)\n",
" test_error = error(h, X_test, y_test)\n",
" train_errors.append(train_error)\n",
" test_errors.append(test_error)"
]
},
{
"cell_type": "code",
"execution_count": 33,
"metadata": {
"collapsed": false
},
"outputs": [
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAYEAAAEACAYAAABVtcpZAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3Xl4FFX28PHvyc6SgAEnEHYRFQRRYAR0kKjAgCiiooCK\nG+qAK87iC+qMgXH8wbgvI4jAgKiI4gKoIIjEHRmQ1YCAgkJYZA9bhyzn/aMq0MSEbL2ku8/nefrp\n6ttV954anD6pe6vuFVXFGGNMZIoKdgDGGGOCx5KAMcZEMEsCxhgTwSwJGGNMBLMkYIwxEcySgDHG\nRLCTJgERSRCRb0VkuYhkisj/ueXJIjJfRNaJyDwRqe11zAgRWS8ia0Wkh1d5exFZ5X73nFd5vIhM\nd8sXiUgTf5yoMcaY3zppElBVD3Cxqp4LnANcLCJ/AIYD81X1DGCB+xkRaQX0B1oBPYGXRETc6sYC\ng1W1BdBCRHq65YOB3W75M8AYX56gMcaYkpXaHaSqh93NOCAa2Av0Aaa45VOAvu72lcA0Vc1V1U3A\nBqCjiNQHElV1sbvfq17HeNf1DnBphc/GGGNMuZSaBEQkSkSWAzuAhar6PZCiqjvcXXYAKe52KrDF\n6/AtQINiyrPcctz3zQCqmgfsF5Hkip2OMcaY8ogpbQdVLQDOFZFawMcicnGR71VEbO4JY4wJQaUm\ngUKqul9EPgTaAztEpJ6qbne7en51d8sCGnkd1hDnCiDL3S5aXnhMY2CriMQAtVR1T9H2LdEYY0z5\nqaqc7PvS7g6qW3jnj4hUA7oDy4BZwM3ubjcD77vbs4ABIhInIs2AFsBiVd0OZItIR3egeBAw0+uY\nwrr64Qw0l3QyYfl69NFHgx6DnZ+dn51f+L3KorQrgfrAFBGJwkkYU1V1gYgsA94SkcHAJuA690c6\nU0TeAjKBPOAuPR7JXcBkoBrwkarOdcsnAlNFZD2wGxhQpsiNMcZU2kmTgKquAtoVU74H6FbCMY8D\njxdTvhRoU0x5Dm4SMcYYE1j2xHAVkJaWFuwQ/MrOL7TZ+YU3KWu/UbCJiIZKrMYYUxWICFrKwHCZ\n7w6qCn76CU47LdhRGBM5jj/wb6q6iv6RHFJJ4M034aGHgh2FMZHFrsCrvsok65AaE3jzzWBHYIwx\n4SWkksDevfD998GOwhhjwkdIJYH+/e1qwBhjfCmkksCAATBtGlgXpTHG+EZIJYH27UEEliwJdiTG\nmHAwdOhQHnvssWCHEVQh95zAP/4Bhw7BU08FOyJjwp97n3mwwyhR06ZNmTRpEpdcckmwQwmqkv6d\nyvKcQEhdCYDTJTR9OhQUBDsSY0ywnSxJ5eXlBTia0hUXU35+frnqKO/+pQm5JNCqFdSpA19+GexI\njDHBNGjQIH755ReuuOIKEhMTeeKJJ4iKimLSpEk0adKEbt2c6c2uvfZa6tevT+3atenatSuZmZnH\n6rjlllv4+9//DkBGRgYNGzbk6aefJiUlhdTUVCZPnlxqHDk5Ofz1r3+lSZMm1KtXj6FDh+LxeE6o\n89///jf169fntttuY+TIkfTr149BgwZRq1YtpkyZwtatW+nTpw916tShRYsWTJgw4Vj96enpv9nf\nl0IuCcDxAWJjTPCJVP5VEVOnTqVx48Z88MEHHDhwgOuuc+ah/Pzzz1m7di0ff/wxAL1792bDhg3s\n3LmTdu3accMNN3jFLic8aLVjxw6ys7PZunUrEydO5O6772b//v0njWP48OFs2LCBFStWsGHDBrKy\nshg1atQJde7du5dffvmF8ePHo6rMmjWLa6+9lv3793P99dczYMAAGjduzLZt25gxYwYPPfQQCxcu\nPFZH0f19KtjzXZdjXmwt9NNPqnXrqh49qsYYP/L+/11V1LRpU12wYIGqqm7cuFFFRDdu3Fji/nv3\n7lUR0ezsbFVVveWWW/SRRx5RVdWFCxdqtWrVND8//9j+v/vd7/Tbb78tsb6CggKtUaOG/vjjj8fK\nvv76a23WrNmxOuPi4jQnJ+fY948++qh27dr12OdffvlFo6Oj9eDBg8fKRowYobfcckux+xenpH8n\nt/ykv60hNW1EoWbN4PTTYcEC6Nkz2NEYY6qSRo2OL25YUFDAQw89xIwZM9i5cydRUU7nx65du0hM\nTPzNsXXq1Dm2D0D16tU5ePBgiW3t3LmTw4cP0759+2NlqkqB16DlqaeeSlxc3AnHNWx4fKHFrVu3\nkpycTI0aNY6VNW7cmCVet0F67+9rIdkdBNYlZIwpfs4c77LXX3+dWbNmsWDBAvbv38/GjRuBE+dD\nqsy8O3Xr1qVatWpkZmayd+9e9u7dy759+8jOzi6x/qJdUKmpqezZs+eEZPPLL7+c8MPvz4n8QjYJ\nXHcdzJoF7viLMSYCpaSk8OOPP5b4/cGDB4mPjyc5OZlDhw7xUJEZKPV4d3OFREVFcccddzBs2DB2\n7twJQFZWFvPmzSvxmKLtNWrUiAsuuIARI0aQk5PDypUrmTRpEjfeeGOF4yqPkE0C9evDeefBRx8F\nOxJjTLCMGDGCxx57jOTkZN55553f/MV800030aRJExo0aEDr1q3p3LnzCfsU/au8In9xjxkzhtNP\nP51OnTpRq1Ytunfvzrp160qss2ibANOmTWPTpk2kpqZy9dVXM2rUqGPPPhS3vy+F3MNi3l55BebP\nh7feClJQxoS5qv6wmHFU5mGxkE4Ce/Y4g8RbtkAxYzzGmEqyJBAaIuqJYW/JydClC8ycGexIjDHh\n7OyzzyYxMfE3r2lhcHdKSF8JALz+unOX0AcfBCEoY8KcXQmEhojtDgI4cAAaNnTWH65TJwiBGRPG\nLAmEhojtDgJnLOCPf4R33w12JMYYE3pCPgmAPThmjDEVFfLdQeA8MFa/vrP+cGpqgAMzJoxZd1Bo\niOjuIICEBLjySnj77WBHYowxoSUskgBYl5AxxlTESZOAiDQSkYUi8r2IrBaR+9zydBHZIiLL3Fcv\nr2NGiMh6EVkrIj28ytuLyCr3u+e8yuNFZLpbvkhEmlTkRC691LlDyJ0fyhgTAZo2bcqnn35aqTom\nT55Mly5dfBRR6CntSiAXeEBVzwY6AXeLSEtAgadV9Tz3NQdARFoB/YFWQE/gJTk+6cVYYLCqtgBa\niEjhJNCDgd1u+TPAmIqcSGwsXHMNvPlmRY42xoSiqjxmURWXkizOSZOAqm5X1eXu9kFgDdDA/bq4\nwYYrgWmqmquqm4ANQEcRqQ8kqupid79Xgb7udh+gcL20d4BLK3guDBxoScCYSFF0ecknn3ySRYsW\nccEFF3DKKadw7rnn8tlnnx3bf/LkyTRv3pykpCROO+003njjDdauXcuQIUP45ptvSExMJDk5+aRt\nhvpSksUp86IyItIUOA9YBFwI3CsiNwFLgL+o6j4g1f2+0BacpJHrbhfK4ngyaQBsBlDVPBHZLyLJ\nqrqnvCfzhz/A7t2QmemsRWyM8T8ZWfkZLvXR8v81P3XqVL788ksmTpzIJZdcQlZWFm3btuW1116j\nZ8+efPLJJ1xzzTX88MMPJCQkcP/997NkyRJatGjBjh072L17N2eddRYvv/wyEyZM4Isvvii1zeHD\nh7Nx40ZWrFhBTEwM119/PaNGjeLxxx8HTlxKMj8/n9GjRzNr1ixmzJjB1KlT8Xg89OjRg3POOYcZ\nM2awZs0aunfvTvPmzbn44osBfrO/v5UpCYhITWAGcL+qHhSRsUDhIpr/BJ7C6dYJqqgo6N/fGSD+\n5z+DHY0xkaEiP+D+8Nprr3HZZZfR011usFu3bnTo0IEPP/yQfv36ERUVxapVq2jYsCEpKSmkpKQA\nv53fvySqyiuvvMLKlSupXbs24ExlfcMNNxxLAlFRUYwcOZLY2FhiY2MBuOCCC+jTpw/grET29ddf\nM2fOHOLi4mjbti233347r7766rEk4L1/QkKCj/7XKVmpSUBEYnG6aV5T1fcBVPVXr+8nALPdj1lA\nI6/DG+JcAWS520XLC49pDGwVkRigVklXAenp6ce209LSSEtL+80+Awc6r1GjKr6AtTEm9Pz888+8\n/fbbzJ49+1hZXl4el1xyCdWrV2f69Ok8+eSTDB48mAsvvJCnnnqKM888s8z1h8JSkhkZGWRkZJTr\nmJMmAXdQdyKQqarPepXXV9Vt7sergFXu9izgDRF5GqebpwWwWFVVRLJFpCOwGBgEPO91zM043Uj9\ngAUlxeOdBErSvj2owtKl0KFDqbsbY0KY92IrjRs3ZtCgQYwfP77YfXv06EGPHj3Iycnh4Ycf5o47\n7uDzzz8v84It3ktJ1q9fv9R4Cj+XtJRkzZo1Ad8uJVn0j+ORI0eWekxpdwddCNwIXFzkdtAxIrJS\nRFYAXYEHAFQ1E3gLyATmAHd5PeZ7FzABWA9sUNW5bvlEoI6IrAeGAcPLcK4lErEBYmMihffykjfc\ncAOzZ89m3rx55Ofn4/F4yMjIICsri19//ZWZM2dy6NAhYmNjqVGjBtHR0cfq2LJlC7m5uSdtKxyW\nkixW4RqbVf3lhFo2q1erNmyomp9f5kOMMcUoz//vgmHmzJnauHFjrV27tj711FP67bffateuXTU5\nOVlPPfVUvfzyy3Xz5s26bds27dq1q9aqVUtr166tF198sa5Zs0ZVVY8ePaq9e/c+dszJeDwefeih\nh/S0007TpKQkbdmypb7wwguqqrpw4UJt1KjRCfunp6froEGDTijbsmWLXn755ZqcnKzNmzfXl19+\n+aT7l0VJ/05u+Ul/W8Ni7qDinHMOvPgiXHSRH4MyJsxV5fvwzXERP3dQcaxLyBhjShe2SaB/f5gx\nA0rp5jPGmBOE81KSxQnb7iCATp1g5Ehn0RljTPlZd1BosO6gEtjMosYYc3JhfSWwdSu0bu28B+DB\nO2PCjl0JhIbKXAmUee6gUJSaCueeC3PmwFVXBTsaY0JTZR5eMlVfWCcBON4lZEnAmPKzq4DwF9bd\nQeDMKnraabBlCyQm+iEwY4ypoiJ+YBigTh1niulZs4IdiTHGVD1hnwTAHhwzxpiShH13EMCBA9Cw\nobP+cCkLBxljTNiw7iBXYiL06AHvvBPsSIwxpmqJiCQA1iVkjDHFiYjuIIAjR5znBjIzoYT1IIwx\nJqxYd5CXatWgTx94++1gR2KMMVVHxCQBsLmEjDGmqIjpDgJnWunUVFi8GJo181FgxhhTRVl3UBGx\nsdCvH0yfHuxIjDGmaoioJADWJWSMMd4iLgl06QK7djl3CRljTKSLuCQQFeUsPWnPDBhjTAQmATj+\n4FiIjIkbY4zfRGQS6NABCgrgu++CHYkxxgRXRCYBERsgNsYYiLDnBLx9/z307Ak//+yMExhjTLix\n5wRO4uyzoXZt+OqrYEdijDHBE7FJAGxmUWOMidjuIICffoJOnWDrVoiJ8WnVxhgTdJXuDhKRRiKy\nUES+F5HVInKfW54sIvNFZJ2IzBOR2l7HjBCR9SKyVkR6eJW3F5FV7nfPeZXHi8h0t3yRiDSp+CmX\nz2mnOXMIffppoFo0xpiqpbTuoFzgAVU9G+gE3C0iLYHhwHxVPQNY4H5GRFoB/YFWQE/gJREpzEJj\ngcGq2gJoISI93fLBwG63/BlgjM/OrgwGDrS7hIwxkeukSUBVt6vqcnf7ILAGaAD0Aaa4u00B+rrb\nVwLTVDVXVTcBG4COIlIfSFTVxe5+r3od413XO8CllT2p8rjuOpg5EzyeQLZqjDFVQ5kHhkWkKXAe\n8C2Qoqo73K92ACnudiqwxeuwLThJo2h5lluO+74ZQFXzgP0iErDl4FNToW1bmDs3UC0aY0zVUabh\nUBGpifNX+v2qeuB4Dw+oqopIQEaX09PTj22npaWRlpbmk3oLHxzr27f0fY0xpqrKyMggIyOjXMeU\neneQiMQCHwBzVPVZt2wtkKaq292unoWqepaIDAdQ1dHufnOBR4Gf3X1auuUDgYtUdai7T7qqLhKR\nGGCbqp5aTBw+vzuo0O7dziBxVhbUrOmXJowxJuB8cXeQABOBzMIE4JoF3Oxu3wy871U+QETiRKQZ\n0AJYrKrbgWwR6ejWOQiYWUxd/XAGmgOqTh34wx9g1qxAt2yMMcF10isBEfkD8DmwEijccQSwGHgL\naAxsAq5T1X3uMQ8BtwF5ON1HH7vl7YHJQDXgI1UtvN00HpiKM96wGxjgDioXjcVvVwIAU6fCW2/B\n7Nl+a8IYYwKqLFcCEf2wmLfsbGjUCDZuhOSADUsbY4z/2NxB5ZCUBN27w7vvBjsSY4wJHEsCXmwu\nIWNMpLHuIC9HjjjPDaxZA/Xq+bUpY4zxO+sOKqdq1eCKK5wBYmOMiQSWBIqwLiFjTCSx7qAicnOd\nLqH//Q+aNvV7c8YY4zfWHVQBsbFwzTUwfXqwIzHGGP+zJFAMW4TeGBMpLAkUo0sX2LnTuUvIGGPC\nmSWBYkRHQ//+NkBsjAl/lgRKUNglFCLj5sYYUyGWBErw+99Dfj4sWxbsSIwxxn8sCZRAxAaIjTHh\nz54TOInVq+Gyy2DTJoiydGmMCTH2nEAltW4NtWrB118HOxJjjPEPSwKlsC4hY0w4C6kksHz78oC3\nOWAAzJgBeXkBb9oYY/wupJLAPR/dQ6DHBZo3hyZN4NNPA9qsMcYEREglAU+eh9dWvhbwdm1mUWNM\nuAqpu4O+2fwNV0+/mrX3rCUpPilgbWdlQZs2sG0bxMcHrFljjKmUsLs7qFPDTvQ8vScjM0YGtN0G\nDeCcc2Du3IA2a4wxfhdSSQBgdLfRvLryVb7/9fuAtjtwoN0lZIwJPyHVHVQY6wvfvsD7P7zPJ4M+\nQeSkVzo+s2uXM0iclQU1awakSWOMqZSw6w4qNPT3Q9l5aCczMmcErM26deHCC2H27IA1aYwxfheS\nSSAmKoYXL3uRv8z7CwePHgxYu/bgmDEm3IRkd1ChG969gSa1mvD4pY8HJIbsbGjUyJlL6JRTAtKk\nMcZUWNh2BxV6ovsTjF86nnW71wWkvaQk6N4d3n03IM0ZY4zfhXQSSE1MZfgfhnP/3PsD9iSxdQkZ\nY8JJqUlARCaJyA4RWeVVli4iW0Rkmfvq5fXdCBFZLyJrRaSHV3l7EVnlfvecV3m8iEx3yxeJSJPy\nnMB9He9j075NzF4XmBHb3r1hyRLYvj0gzRljjF+V5Urgv0DPImUKPK2q57mvOQAi0groD7Ryj3lJ\njt/DORYYrKotgBYiUljnYGC3W/4MMKY8JxAXHcfzPZ9n2NxhHMk9Up5DK6RaNbjiCnj7bb83ZYwx\nfldqElDVL4C9xXxV3GDDlcA0Vc1V1U3ABqCjiNQHElV1sbvfq0Bfd7sPMMXdfge4tOzhO7o37067\n+u3491f/Lu+hFWJzCRljwkVlxgTuFZEVIjJRRGq7ZanAFq99tgANiinPcstx3zcDqGoesF9Ekssb\nzFM9nuL5xc+zce/G8h5abt27w7p18PPPfm/KGGP8KqaCx40FRrnb/wSewunW8av09PRj22lpaaSl\npR373KR2Ex7o9AB/nvdn3uv/nl/jiI2Fq6+G6dPhwQf92pQxxpRZRkYGGRkZ5TqmTM8JiEhTYLaq\ntjnZdyIyHEBVR7vfzQUeBX4GFqpqS7d8IHCRqg5190lX1UUiEgNsU9VTi2mn1DWGPXkeWr/Umhcv\ne5GepxcdxvCtjAx44AFYtsyvzRhjTIX57TkBt4+/0FVA4Z1Ds4ABIhInIs2AFsBiVd0OZItIR3eg\neBAw0+uYm93tfsCCisQEkBCTwHM9n+O+OfeRk5dT0WrKpEsX2LED1q71azPGGONXZblFdBrwNXCm\niGwWkduAMSKyUkRWAF2BBwBUNRN4C8gE5gB3ef35fhcwAVgPbFDVwomZJwJ1RGQ9MAwYXpkT6n1G\nb86seybPLnq2MtWUKjoa+ve3AWJjTGgL6WkjSvLjnh/pOKEjy4csp2FSQ7/FtHgxDBrkXA0EaDJT\nY4wps7CfNqIkzZObM7TDUP46769+bef3v3cWoF++3K/NGGOM34RlEgAY0WUE32z5hoUbF/qtDRGb\nRsIYE9rCNglUj63OM398hnvn3Etufq7f2hkwwBkXKCjwWxPGGOM3YZsEAK466ypSE1P5z//+47c2\n2rRxZhf95hu/NWGMMX4T1klARHi+1/M89vljbD/ovxnfrEvIGBOqwvLuoKIenP8gOw7tYErfKaXv\nXAEbNjhLT2ZlQUxFn8E2xhgfi9i7g4r6+0V/55OfPuGrX77yS/2nnw5NmsBC/41BG2OMX0REEkiM\nT+SJ7k9wz5x7yC/I90sbhQPExhgTSiIiCQAMbD2QpPgkxi8d75f6+/eH99+HHP/OVmGMMT4VMUlA\nRHih1ws8mvEouw7v8nn9DRpA69Ywd27p+xpjTFURMUkA4JyUcxjYeiAPL3jYL/XbYjPGmFATEXcH\nedvn2UfL/7Rk9sDZdEjt4IPIjtu1yxkkzsqCGjV8WrUxxpSb3R1UjNoJtXn8kse556N7KFDfPuZb\nty507gyzA7PmvTHGVFrEJQGAm891li+YvHyyz+seONAeHDPGhI6I6w4qtHTrUnq/0Zs1d6/hlGqn\n+Kze7Gxo1Ag2bYJTfFetMcaUm3UHnUT71Pb0Pasvj2Y86tN6k5KgWzd4z7/LHBtjjE9EbBIA+Ncl\n/+LN1W+ycsdKn9ZrcwkZY0JFxHYHFRq3ZBxvrHqDz275DPHR8mBHjkBqqrPiWEqKT6o0xphys+6g\nMrij3R0cPHqQN1a94bM6q1WDyy+Ht9/2WZXGGOMXEZ8EoqOi+c9l/+HBTx4kOyfbZ/XaXELGmFAQ\n8d1BhW6deSt1q9XliR5P+KS+o0edLqHvvoPGjX1SpTHGlIt1B5XD6EtHM3nFZNbsXOOT+uLi4Oqr\n7WrAGFO1WRJwpdRM4ZEuj3Df3Pvw1RWHzSVkjKnqLAl4ufv8u9l+cDvvrnnXJ/VddBFs3w4//OCT\n6owxxucsCXiJiYrhhV4v8Od5f+bQ0UOVri86Gq67zq4GjDFVlyWBItKapnFBowv4vy//zyf1Fc4l\nFCLj78aYCGNJoBhPdn+ScUvGsWHPhkrXdf75zp1Cy5f7IDBjjPExSwLFaJDUgAcvfJBhc4dVui4R\ne2bAGFN1lZoERGSSiOwQkVVeZckiMl9E1onIPBGp7fXdCBFZLyJrRaSHV3l7EVnlfvecV3m8iEx3\nyxeJSBNfnmBFDes0jPV71vPBug8qXVfhXUIFvl2+wBhjKq0sVwL/BXoWKRsOzFfVM4AF7mdEpBXQ\nH2jlHvOSHJ+QZywwWFVbAC1EpLDOwcBut/wZYEwlzsdn4qLjeKHXC9w/9348eZ5K1dW6NdSsCYsW\n+Sg4Y4zxkVKTgKp+AewtUtwHmOJuTwH6uttXAtNUNVdVNwEbgI4iUh9IVNXF7n6veh3jXdc7wKUV\nOA+/6NG8B21T2vLEV5V7iljEFpsxxlRNFR0TSFHVHe72DqBwrsxUYIvXfluABsWUZ7nluO+bAVQ1\nD9gvIskVjMvnnv7j0zz77bNs2repUvUMGOBMKJeX55u4jDHGF2IqW4GqqogE5AbI9PT0Y9tpaWmk\npaX5vc2mtZsyrOMw/jLvL7xz3TsVruf0050VxzIynEVnjDHG1zIyMsjIyCjXMWWaQE5EmgKzVbWN\n+3ktkKaq292unoWqepaIDAdQ1dHufnOBR4Gf3X1auuUDgYtUdai7T7qqLhKRGGCbqp5aTAx+nUDu\nZDx5Hs5+6WzG9h5Lj+Y9Sj+gBE89BZmZMHGiD4MzxpgS+HMCuVnAze72zcD7XuUDRCRORJoBLYDF\nqrodyBaRju5A8SBgZjF19cMZaK5SEmISePaPz3LfnPs4mn+0wvX07w/vvw85OT4MzhhjKqEst4hO\nA74GzhSRzSJyKzAa6C4i64BL3M+oaibwFpAJzAHu8vrz/S5gArAe2KCqc93yiUAdEVkPDMO906iq\nufyMy2me3JznFj1X+s4laNjQuVPo4499GJgxxlSCrSdQDut3r6fzxM6sGLKCBkkNSj+gGGPHwhdf\nwBu+W8jMGGOKVZbuIEsC5fTwgofZuG8jb1xTsV/xnTuhRQvIyoIaNXwcnDHGeLFFZfzgoS4P8dXm\nr/hs02cVOv7UU6FTJ5g928eBGWNMBVgSKKcacTV4qsdT3DPnHvIKKnbTvy02Y4ypKiwJVMA1La8h\npUYKL/3vpQod37cvLFwI+/b5ODBjjCknSwIVICI83+t5/vn5P9lxcEfpBxRRqxZceim8954fgjPG\nmHKwJFBBrU5txc1tb2bEghEVOt7mEjLGVAV2d1AlZOdk0/I/LZlx7Qw6N+pcrmMPH4bUVGf94ZSU\n0vc3xpjysruD/CwpPol/d/s398y5h/yC/HIdW706XH45zJjhp+CMMaYMLAlU0vVtrqd6bHUmfDeh\n3McOGGBdQsaY4LLuIB9YsX0FPV7rQeZdmdSpXqfMxx096nQJffcdNG7sxwCNMRHJuoMCpG29tlzX\n6joe+fSRch0XFwdXXw3Tp/spMGOMKYUlAR8ZdfEo3lv7Ht9t+65cxw0YABMmwJYtpe9rjDG+ZknA\nR06pdgr/uuRf3P3R3RRo2VeUT0uDG26A886Dl16yxeiNMYFlScCHbj3vVgq0gFdXvFrmY6Ki4B//\ngM8+g9dfhy5dnIVnjDEmECwJ+FCURPFirxcZsWAE+zzlmxOiVStniukbb4SuXSE93RafMcb4nyUB\nH/t9g99zxRlXkJ6RXu5jo6Jg6FBYtsx5nXcefPWV72M0xphCdouoH+w6vItW/2nFgpsW0CalTYXq\nUIV33oH774crr4TRoyEpyceBGmPCmt0iGiR1q9clPS2de+fcS0UTlwj06werV0NuLpx9NsycWfpx\nxhhTHpYE/ORP7f/E/pz9vLm6cgsHnHIKvPIKTJ0Kf/sbXHstbNvmoyCNMRHPkoCfREdF82KvF/nb\n/L9xIOdApetLS4MVK+CMM6BtW+fZghDpHTPGVGE2JuBnN79/M/Vq1GNM9zE+q3PlSrj9dmcSuvHj\nncRgjDFF2ZhAFTCm2xgmLpvI2l1rfVbnOefAN984K5RdcAE8/rgzbmCMMeVlScDP6tWsx8NdHua+\nOfdVeJDb0PvWAAAN8ElEQVS4ONHRMGwYLF0KX34J7dvD4sU+q94YEyEsCQTAPeffQ9aBLN5f+77P\n627SBD78EIYPhz59nMRw8KDPmzHGhClLAgEQGx3Li71e5IGPH+Bw7mGf1y8C11/v3E66dy+0bg1z\n5vi8GWNMGLKB4QAaMGMAZ9Q5g1EXj/JrO/Pnw5/+BJ07wzPPwO9+59fmjDFVlA0MVzFP9niSl/73\nEj/u+dGv7XTvDqtWOQvWtGkDU6bY7aTGmOLZlUCAjf5yNF9v/ppZA2cFpL2lS+GOO6BOHXj5ZTjt\ntIA0a4ypAvx+JSAim0RkpYgsE5HFblmyiMwXkXUiMk9EanvtP0JE1ovIWhHp4VXeXkRWud89V5mY\nqroHOj3A2l1r+XDdhwFpr/CuoR494Pzz4cknIS8vIE0bY0JAZbuDFEhT1fNU9Xy3bDgwX1XPABa4\nnxGRVkB/oBXQE3hJRAoz1FhgsKq2AFqISM9KxlVlxcfE83yv57l/7v148jwBaTMmxply4ttvYe5c\n6NjRWdfYGGN8MSZQ9FKjDzDF3Z4C9HW3rwSmqWquqm4CNgAdRaQ+kKiqhXe5v+p1TFjqeXpPWv+u\nNU9/83RA223e3Bk0vvde6NULHnwQDvv+ZiVjTAjxxZXAJyKyRETucMtSVHWHu70DSHG3UwHvlXS3\nAA2KKc9yy8PaM398hqe/eZpf9v8S0HZF4JZbnKknNm92Bo4/+SSgIRhjqpDKJoELVfU8oBdwt4h0\n8f7SHckN/dFcP2h2SjPuPf9e/jLvL0FpPyUFpk2D55+HwYOdxLB7d1BCMcYEUUxlDlbVbe77ThF5\nDzgf2CEi9VR1u9vV86u7exbQyOvwhjhXAFnutnd5VnHtpaenH9tOS0sjLS2tMuEH3YMXPkirl1rx\nyU+f0O20bkGJoXdv5yGzRx5xHjJ7+mkYMMC5YjDGhJaMjAwyMjLKdUyFbxEVkepAtKoeEJEawDxg\nJNAN2K2qY0RkOFBbVYe7A8Nv4CSKBsAnwOmqqiLyLXAfsBj4EHheVecWaS8sbhEtaubamdw7514e\nuegRrm9zPTXjagYtlkWLnNtJGzeGsWOdd2NM6PL3LaIpwBcishz4FvhAVecBo4HuIrIOuMT9jKpm\nAm8BmcAc4C6vX/W7gAnAemBD0QQQzq4860om9pnInA1zaPxMY+7+8G5W/7o6KLF06uQ8V9C5M7Rr\nB889B/n5QQnFGBMg9rBYFbIlewsTvpvAK9+9QrPazRjaYSj9WvUjPiY+4LH88APceSd4PM4CNm0q\ntlSyMSaIynIlYEmgCsrNz+WDdR8wdslYlm9fzi3n3sKd7e/k9OTTAxpHQYGTAB5+2JmL6JFHICEh\noCEYYyrBkkAY2LBnAy8veZnJKybTrn47hrQfwhVnXkFMVKXG9Mtl61bn2YLVq531ji+6KGBNG2Mq\nwZJAGPHkeZiROYNxS8axad8mbm93O3e0u4MGSYF7pOK995xk0Ls3jBkDtWuXfowxJnhsFtEwkhCT\nwI3n3MiXt33JRzd8xM5DO2kztg1XTb+KeT/Oo0AL/B7DVVfB999DVBScfTa8+67fmzTG+JldCYSw\nAzkHmLZ6GmOXjCU7J5s/tf8Tt557K6fWONXvbX/xhTNwfNZZ8OKL0CDsn/E2JvTYlUCYS4xP5M72\nd/Ldnd/xxtVvkLkzkxYvtODGd2/ky1++9OmaxkV16QLLlzuL3p97Lowb5wwkG2NCi10JhJk9R/Yw\nZfkUxi0dR1x0HEPaD2FQ20EkxSf5rc3vv4fbb3dmKx0/Hlq29FtTxphysIHhCKaqLNy0kHFLxjH/\np/lc2+pahnQYQrv67fzSXn6+85Rxejrcd5+z8H1cnF+aMsaUkSUBA8C2A9uYtGwS478bT72a9RjS\nfgj9W/enemx1n7e1eTPcdRds3OjcTtq5s8+bMMaUkSUBc4L8gnzmbJjD2CVjWbRlEYPOGcSQDkM4\nq+5ZPm1HFd5+G4YNg6uvhpEjoWZNiI52XjY5nTGBYUnAlGjTvk2MXzqeScsm0fLUlgztMJS+Z/Ul\nLtp3fTh79jgrmk2b5ixpmZ/vDB6LHE8IRV8xMSV/F6h9y7t/fDzUqAHVqx9/Ff0cH2/JzwSeJQFT\nqqP5R3lvzXuMWzqOtbvWctu5t3FH+ztoWrupX9pTdRJBfr7zKkwOZX35e//yHpOXB0ePOiu0HT4M\nhw4d3/b+nJd38iThi89xcZZozIksCZhyWbtrLeOWjGPqyql0btiZIR2G0Ov0XkRHRQc7tJCXlwdH\njpScJE5WVtbP3onGl0kmKQlq1XLmjbIkE1osCZgKOZx7mOmrpzNu6Ti2H9zOne3uZHC7wdSrWS/Y\noZmTKEw0FUkkJe1z6BAcOAD79jlXcbVqOdOFVOQ9KcnpPjOBY0nAVNrSrUsZt2QcM9bMoPtp3Rna\nYShpTdMQ+5Mw4ng8sH+/89q3r/zvBw86VxZFk0N5Ekm1anY1Uh6WBIzP7PfsZ+rKqYxbMo68gjyG\ndBjCTW1vIrlacrBDMyGioMC5qqhoEtm/3xmHqezVSEzgJuANOksCxudUla82f8XYJWP5cN2H9D2r\nL0M7DOX8Bufb1YHxu5yckpPEnn15/Lr/ALsOZLP74AH2Hc5m35EDHMjJ5mDuAQ7lZ5NTcICYmtnE\n1cwmpsYBohIOEhMVS6wkECsJxEU5r/ioBOKjE0iISSA+JoFqsQlUi0mgetzxV42EBGrEJ1AzwXkl\nVnNeSdWdV81qMVSrJsTHB2/Q3pKA8audh3by3+X/5eWlL1MrvhZDOgwJ+jrJJrTkF+Rz4OgBsnOy\nOZDjvpfw+WTfHcg5QE5+DolxiSTFJ5EY77wnxScdL4tLJDEuiTgSiclPIio3kYKcmniO5nE4x8Oh\nox6OHPVwONeDJ9fDkTwPnjwPOfnO62i+h6MFHo6qh1z1kKce8vCQJx7y8ZAf5aHAfWm0ByiA/ATI\nTYC8BMhPIKoggaj8BKI0gWhNIEYTiCaBWBJ+m4iiE4hzE1GCVyKqFpdAjTIkoprVYjjjjChLAsb/\nCrSA+T/OZ9zScXy26TMGth7IkA5DaJNia1KGo/yCfA4ePVjqj3JZfsA9eR5qxtU89kPt/QPu/eNd\n4g+7177VY6tXqavRvII8cvJy8OR5OHzUQ/ZhD9lHnPeDRzwcOOLhgMfDoRwPhzxOEjpcUiLK85BT\n4CSinAInCZWWiKAARhVYEjCBVXSd5CEdhnDFGVcAzv8pcgtynff83BM+l7Ws8HNZy06oq7z7lzEe\nRREEESn2PUqiSvwuFPY5nHv4hB/ww7mHqRFX46Q/yN7vxX7nfq4eW50oscmM/SGvII/Y6FhLAiY4\nvNdJ/mrzV8RExRATFUNsVKzzHh1bYlnh5xLLylBHheqtYJnb74qixb4XaEGJ3ynu9xU83p91Fx5f\nPbb6CT/eNeNq2g93iLAxAWOMiWC2qIwxxpiTsiRgjDERzJKAMcZEMEsCxhgTwSwJGGNMBLMkYIwx\nEazKJAER6Skia0VkvYj8v2DHY4wxkaBKJAERiQZeBHoCrYCBItIyuFEFTkZGRrBD8Cs7v9Bm5xfe\nqkQSAM4HNqjqJlXNBd4ErgxyTAET7v8R2vmFNju/8FZVkkADYLPX5y1umTHGGD+qKknA5oMwxpgg\nqBJzB4lIJyBdVXu6n0cABao6xmuf4AdqjDEhJiQmkBORGOAH4FJgK7AYGKiqa4IamDHGhLkqsdqm\nquaJyD3Ax0A0MNESgDHG+F+VuBIwxhgTHFVlYLhYIjJJRHaIyKpgx+IPItJIRBaKyPcislpE7gt2\nTL4kIgki8q2ILBeRTBH5v2DH5GsiEi0iy0RkdrBj8TUR2SQiK93zWxzseHxNRGqLyAwRWeP+99kp\n2DH5ioic6f67Fb72l/T7UqWvBESkC3AQeFVVw27BWhGpB9RT1eUiUhNYCvQNp64wEamuqofdcZ8v\ngb+q6pfBjstXROTPQHsgUVX7BDseXxKRjUB7Vd0T7Fj8QUSmAJ+p6iT3v88aqro/2HH5mohEAVnA\n+aq6uej3VfpKQFW/APYGOw5/UdXtqrrc3T4IrAFSgxuVb6nqYXczDme8J2x+UESkIXAZMAGoOiuc\n+1ZYnpeI1AK6qOokcMYlwzEBuLoBPxaXAKCKJ4FIIiJNgfOAb4MbiW+JSJSILAd2AAtVNTPYMfnQ\nM8DfgIJgB+InCnwiIktE5I5gB+NjzYCdIvJfEflORF4RkerBDspPBgBvlPSlJYEqwO0KmgHc714R\nhA1VLVDVc4GGwEUikhbkkHxCRC4HflXVZYTpX8vAhap6HtALuNvtng0XMUA74CVVbQccAoYHNyTf\nE5E44Arg7ZL2sSQQZCISC7wDvKaq7wc7Hn9xL7U/BDoEOxYfuQDo4/abTwMuEZFXgxyTT6nqNvd9\nJ/Aezhxf4WILsEVV/+d+noGTFMJNL2Cp+29YLEsCQSQiAkwEMlX12WDH42siUldEarvb1YDuwLLg\nRuUbqvqQqjZS1WY4l9ufqupNwY7LV0Skuogkuts1gB5A2Nylp6rbgc0icoZb1A34Pogh+ctAnD9S\nSlQlHhYriYhMA7oCdURkM/APVf1vkMPypQuBG4GVIlL44zhCVecGMSZfqg9Mce9OiAKmquqCIMfk\nL1X3NruKSQHec/5OIQZ4XVXnBTckn7sXeN3tMvkRuDXI8fiUm7y7AScdz6nSt4gaY4zxL+sOMsaY\nCGZJwBhjIpglAWOMiWCWBIwxJoJZEjDGmAhmScAYYyKYJQFjjIlglgSMMSaC/X/R71j/JYNnBAAA\nAABJRU5ErkJggg==\n",
"text/plain": [
""
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"plt.plot(list(degrees), train_errors, label=\"train_error\")\n",
"plt.plot(list(degrees), test_errors, label=\"test_error\")\n",
"plt.legend()\n",
"plt.show()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Odabir modela kao minimizacija rizika*\n",
"\n",
"* Pogled na problem optimizacije modela iz **statističke teorije učenja**\n",
"\n",
"\n",
"* Rizik = očekivanje gubitka = pogreška hipoteze\n",
"\n",
"\n",
"* **Empirijski rizik $R_{\\mathrm{emp}}$** = procjena pogreške na skupu primjera, $E(h|\\mathcal{D})$\n",
"\n",
"\n",
"* **Strukturni rizik $R_{\\mathrm{struct}}$** = kvantifikacija složenosti modela\n",
" * Npr.: broj parametara, veličina zapisa modela i sl.\n",
" * Što je model složeniji, to je veći strukturni rizik\n",
" \n",
"\n",
"* Želimo modele $\\mathcal{H}$ koji (za naučenu $h\\in\\mathcal{H}$) minimiziraju i empirijski i strukturni rizik\n",
"$$\n",
"R_{\\mathrm{emp}}(h) + \\lambda R_{\\mathrm{struct}}(h)\n",
"$$\n",
"gdje $\\lambda$ definira važnost empirijskog rizika u odnosu na strukturni\n",
"\n",
"\n",
"* [Grafikon: empirijska + strukturna pogreška]\n",
"\n",
"\n",
"* Dakle, umjesto minimizacije\n",
"$$\n",
"h^* = \\mathrm{argmin}_{h\\in\\mathcal{H}} E(h|\\mathcal{D})\n",
"$$\n",
"imamo\n",
"$$\n",
"h^* = \\mathrm{argmin}_{h\\in\\mathcal{H}} E(h|\\mathcal{D}) + \\lambda R_{\\mathrm{struct}}(h)\n",
"$$\n",
"\n",
"\n",
"* Minimizacija strukturnog rizika je alternativa unakrsnoj provjeri\n",
"\n",
"\n",
"* U praksi ipak koristimo unakrsnu provjeru (jer je pouzdanija)\n",
"\n",
"\n",
"* Često kombiniramo unakrsnu provjeru s minimizacjom strukturnog rizika (tzv. regularizacija)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Sažetak\n",
"\n",
"* **Hipoteza** je funkcija koja klasificira primjere (kod klasifikcije) ili daje brojčanu vrijednost (kod regresije), a **model** je skup hipoteza\n",
"\n",
"\n",
"* Različiti modeli imaju različite **složenosti** (kapacitete)\n",
"\n",
"\n",
"* Učenje nije moguće bez **induktivne pristranosti**, koja može **pristranost jezika** ili **pristranost preferencijom**\n",
"\n",
"\n",
"* Svaki algoritam SU ima tri komponente: **model**, **funkciju gubitka** i **optimizacijski postupak**\n",
"\n",
"\n",
"* Empirijsku pogrešku hipoteze izračunavamo kao očekivanje funkcije gubitka na skupu primjera\n",
"\n",
"\n",
"* Učenje modela svodi se na **optimizaciju parametara** modela s empirijskom pogreškom kao kriterijem\n",
" * Konkretno, kod regresije postoji **analitičko rješenje** za taj problem\n",
"\n",
"\n",
"* Model koji je **podnaučen** ili **prenaučen** loše generalizira\n",
"\n",
"\n",
"* Odabir modela svodi se na **optimiranje hiperparametara** modela\n",
"\n",
"\n",
"* **Unakrsnom provjerom** možemo procijeniti **pogreška generalizacije** i odabrati optimalan model\n"
]
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 2",
"language": "python",
"name": "python2"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 2
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython2",
"version": "2.7.6"
}
},
"nbformat": 4,
"nbformat_minor": 0
}