{
 "cells": [
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "collapsed": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The watermark extension is already loaded. To reload it, use:\n",
      "  %reload_ext watermark\n",
      "CPython 3.5.4\n",
      "IPython 6.2.1\n",
      "\n",
      "numpy 1.14.0\n",
      "scipy 1.0.0\n",
      "sklearn 0.20.0\n",
      "pandas 0.22.0\n",
      "matplotlib 2.1.2\n"
     ]
    }
   ],
   "source": [
    "%load_ext watermark\n",
    "%watermark -v -p numpy,scipy,sklearn,pandas,matplotlib\n",
    "\n",
    "# 파이썬 2와 파이썬 3 지원\n",
    "from __future__ import division, print_function, unicode_literals\n",
    "\n",
    "# 공통\n",
    "import numpy as np\n",
    "import os\n",
    "\n",
    "# 일관된 출력을 위해 유사난수 초기화\n",
    "np.random.seed(42)\n",
    "\n",
    "# 맷플롯립 설정\n",
    "%matplotlib inline\n",
    "import matplotlib\n",
    "import matplotlib.pyplot as plt\n",
    "plt.rcParams['axes.labelsize'] = 14\n",
    "plt.rcParams['xtick.labelsize'] = 12\n",
    "plt.rcParams['ytick.labelsize'] = 12\n",
    "\n",
    "# 한글출력\n",
    "matplotlib.rc('font', family='AppleGothic')\n",
    "plt.rcParams['axes.unicode_minus'] = False"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Chapter 4. 모델 훈련"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- 이번 장에서 배울 내용\n",
    " - 가장 간단한 모델 중 하나인 선형 회귀\n",
    " - 비선형 데이터셋에 훈련시킬 수 있는 조금 더 복잡한 모델인 다항 회귀\n",
    " - 학습 곡선<sup>learning curve</sup>을 사용해 모델이 과대적합되는지 감지하는 방법\n",
    " - 훈련 세트의 과대적합을 감소시킬 수 있는 규제 기법"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---\n",
    "## 4.1 선형 회귀"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "다음은 1장에서 본 삶의 만족도에 대한 간단한 선형 회귀 모델"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "&nbsp;&nbsp;&nbsp;&nbsp;삶의 만족도 = $\\theta$<sub>0</sub> + $\\theta$<sub>1</sub> * 1인당_GDP"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- 이 모델은 입력 특성인 1인당_GDP에 대한 선형 함수<br>\n",
    "- &theta;<sub>0</sub>과 &theta;<sub>1</sub>이 모델 파라미터"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "일반적으로 선형 모델은 [식 4-1]에서처럼 입력 특성의 가중치 합과 **편향**<sup>bias</sup>(또는 **절편**<sup>intercept</sup>)이라는 상수를 더해 예측을 만듦"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "![equation4-1](./images/equation4-1.png)\n",
    "<br>\n",
    "<center>**식 4-1 선형 회귀 모델의 예측**</center>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- $\\hat{y}$은 예측값\n",
    "- n은 특성의 수\n",
    "- x<sub>i</sub>는 i번째 특성값\n",
    "- $\\theta$<sub>j</sub>는 j번째 모델 파라미터(편향 $\\theta$<sub>0</sub>과 특성의 가중치 $\\theta$<sub>1</sub>, $\\theta$<sub>2</sub>, ..., $\\theta$<sub>n</sub>을 포함)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "이 식은 [식 4-2]처럼 벡터 형태로 더 간단하게 쓸 수 있음"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<img src=\"./images/equation4-2.png\" width=\"40%\">\n",
    "<center>**식 4-2 선형 회귀 모델의 예측(벡터 형태)**</center>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- $\\theta$는 모델의 파라미터 벡터\n",
    "- $\\theta$<sup>T</sup>는 $\\theta$의 전치(열 벡터가 아니고 행 벡터)\n",
    "- x는 x<sub>0</sub>에서 x<sub>n</sub>까지 담고 있는 샘플의 **특성 벡터**. x<sub>0</sub>는 항상 1\n",
    "- h<sub>$\\theta$</sub>는 모델 파라미터 $\\theta$를 사용한 가설<sup>hypothesis</sup> 함수"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- 모델 훈련 : 모델이 훈련 세트에 가장 잘 맞도록 모델 파라미터를 설정하는 것<br>\n",
    "이를 위해 먼저 모델이 훈련 데이터에 잘 들어맞는지 측정해야 함.<br>\n",
    "성능 측정 지표로는 평균 제곱 오차<sup>Mean Square Error</sup>(MSE)를 최소화"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "훈련 세트 X에 대한 선형 회귀 가설 h<sub>$\\theta$</sub>의 MSE는 [식 4-3]처럼 계산함"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<img src=\"./images/equation4-3.png\" width=\"50%\">\n",
    "<center>**식 4-3 선형 회귀 모델의 MSE 비용 함수**</center>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 4.1.1 정규방정식"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**정규방정식**<sup>Normal Equation</sup> : 비용 함수를 최소화하는 $\\theta$ 값을 얻을 수 있는 수학 공식(식 4-4)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<img src=\"./images/equation4-4.png\" width=\"30%\">\n",
    "<center>**식 4-4 정규방정식**</center>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- $\\hat{\\theta}$은 비용 함수를 최소화하는 $\\theta$ 값<br>\n",
    "- y는 y<sup>(1)</sup>부터 y<sup>(m)</sup>까지 포함하는 타깃 벡터"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "이 공식을 테스트하기 위해 선형처럼 보이는 데이터를 생성"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "\n",
    "X = 2 * np.random.rand(100, 1) #균일 분포[0,1) , 100행 1열\n",
    "y = 4 + 3 * X + np.random.randn(100, 1) #정규분포"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAY0AAAETCAYAAADKy1riAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvNQv5yAAAG8dJREFUeJzt3X+wJWV95/H3ZwYFZFeTDCggTkE2gVDxNze4s7J4V0wMwVKoLLuWpkKMYXQlKmWyKTCwIU4JLpSK2azGK5odTEzciL+S0RVFhxVmUO8Y4hD5YSVRXANxHFBcRZThu3/0udnD4Zx7nr6n+3m6z/28qqjxdPc9/T1tn/708zzdfRQRmJmZpdhQugAzM+sPh4aZmSVzaJiZWTKHhpmZJXNomJlZMoeGmZklc2iYmVkyh4aZmSVzaJiZWbKDSheQ4vDDD49jjz22dBlmZr2yZ8+eb0XEEU2+Zy9C49hjj2V5ebl0GWZmvSLpa02/p7unzMwsmUPDzMySOTTMzCyZQ8PMzJI5NMzMLJlDw8zMkjk0zMwsmUPDzMyStRYaks6WdOSEeYdJuljSIW2t38zMmtdKaEg6BrgCGBsawNuB1wMODTOzHmn8MSKSdgDPBDZNmH8e8GXgn5pet5mZtavxlkZEnBERRwG7RudJOhlYBC5ver1mZta+bA8slLSJKizOioiQlGvVZmbWkCxXT0naALwH+K2IuDfxb7ZKWpa0vG/fvnYLNDOzJLkuuf0Z4BnAuyXdLOlm4Gjgs5LeOu4PImIpIhYiYuGIIxp9HLyZma1Rlu6piPgysHl4mqSvAv82Ir6dowYzM5udb+4zM7NkDg0zM0vWWvdURCxOmX9sW+s2M7N2uKVhZmbJHBpmZpbMoWFmZskcGmZmlsyhYWZmyRwaZmaWzKFhZmbJHBpmZpbMoWFmZskcGmZmlsyhYWZmyRwaZmaWzKFhZmbJHBpmZpbMoWFmZskcGmZmlsyhYWZmyRwaZmaWrLXQkHS2pCOHXj9G0oWSPi9pr6SbJC22tX4zM2teK6Eh6RjgCuDIocmbgQeAUyPiKcA5wB9LOqKNGszMrHkHNf2GknYAzwQ2DU+PiNuA24Ze3y5pD/B04JNN12FmZs1rvKUREWdExFHArtWWk3QYsADc0XQNZmbWjiID4ZIeB3wEeG9EfK1EDWZmVl/20JB0AvBZ4GMRcfEqy22VtCxped++ffkKNDOzibKGhqRfBD4MvDIi3rLashGxFBELEbFwxBEeKzcz64LGB8InkfQE4L8Bz3OXlJlZP+VsafwqcKUDw8ysv7K1NIDjgdMk/aeR6W+PiLdnrMPMzNaotdCIiMWR1+e2tS4zM8vDz54yM7NkDg0zM0vm0DAzs2QODTMzS+bQMDOzZA4NMzNL5tAwM1vF7t1w2WXVv5b35j4zs17ZvRtOOw1++EN49KPhuutgy5bSVZXlloaZ2QQ7d1aBceBA9e/OnaUrKs+hYWY2weJi1cLYuLH6d3GxdEXluXvKzGyCLVuqLqmdO6vAWO9dU+DQMDNb1ZYt/QiL3bvzhJtDw8ys53IO2HtMw8ys53IO2Ds0zMx6LueAvbunzMx6LueAvUPDzGwO5Bqwd/eUmdkM1ttjRlpraUg6G/hsRNw9NO2XgDcCBwN3AC+PiP1t1WBm5eS6BLSklKuW5m07tBIako4BrgDOBO4eTPtJ4A3Az0fEtyS9GLgKOKuNGsysnPXyzKZxVy0Nf842tkPpEGq8e0rSDuALwNEjs14HXB4R3wKIiD8HHi/pp5uuwczKWi/PbJp21VLT22ElhC6+uPq3RJdY46EREWdExFHArpFZi8CnRqZdN5huZnNkvTyzaeWqpW3bxrcimt4OXQjjnFdPPSEi7hmZdjfwpIw1mFkG6+mZTatdtdT0dlgJoZXurhJhnDM0DpkwPcZNlLQV2AqwefPmtmoys5b05ZlNbWtyO3QhjHOGxn5JPx4R9w5NOxK4c9zCEbEELAEsLCyMDRYzs7UoPZg8i9JhnDM0bgCeC1wzNG0ReHnGGsxsnevqlV19CbKcofEW4J2SPh0R90p6CfDNiPhKxhrMbJ2bdplsCaNBduWVsH9/NwMkW2hExBclXQp8WtKhwC3AubnWb2YG5QeTx7UohoPsgQfgvPMgolstoRWthUZELI6Z9iHgQ22t02y960MXR+kaSw4mT+oaGw6yDRuq8Hjooe60hIb5gYVmc6KrffXDulJjqcHkSV1jw0G2aROcf37V4tiwoXrdJQ4NsznRxb76UX2osU2rdY2NBtl551Xb6fzz4SlP6c52cmiYzYnSffUp+lBjm1K7xvbvr8Y0uthF5dAwmxNduPFrmj7U2LaUrrEuh6siun/f3MLCQiwvL5cuw8x6qPTA+1o1UbekPRGx0GRdbmmY2dzqysD7WpS+83sS/3Kfmc2V4V/S68JTYeeNWxpmNjfG3Vnd1bGBvnJomNncGG1Z7N+fd+C9r+MndTg0zGxuDF91dNBBcOfgGdoXXtj+unONn5QOJo9pmFk2w+MNbVi5pPfcc6v7HN71rnw/i5pj/GQuf+7VzGycXAe8LVtg8+bq4N3kb3NPC7scP3HbhYF9d0+ZWRY5HyHS5M1xqd1OOW5c7MJNfw4NswJK90uXkPOA1+QBvE7YtX1vRRfuqE8KDUl/BLwCeGJE/OPIvBOAvcA7IuK1zZdoNl/6fMNZikmBuHLAu/rqPHVMO4CnBncXzu6Hlb7pL7WlsZsqNE4GPjwy763AfcAlzZVlNr/m+UmvKYG4fXs1f/v2coFZJ7i7cHbfJakD4TcN/j15eKKkM4DTgf8SEfc2WZjZvMoxYFrKtIHaOgO5bV5pVXdAecuW6rLd9R4YkNjSiIjbJd3DUGhIehTV737fAryznfLM5s88n7lO68pJ7epJaQnMMi7UtS6nPqkzEH4T8GxJiurRuK8FjgeeFxEHUt9E0ouA3wUeAxwA3hQRf1ajDrPey9kvnXPQfVogpgbmtC68WceF5jm421Y3NH4JOGHQ6rgY+HBEXJf6BpJ+Fvh94Bci4puSngTslLQ3Im6pU7iZTVdi0H1aIDbxexJNjAuVHlDuqzqhsdKzeDJwKnAw8Fs11/evgesi4psAEfF1STcAJ1B1c5lZg/o66D6pJbDSatq0yd1LpdQJjc8BDwEvB04BroiIv6+5vj3AJZL+LCKWJb0AeBbwmprvY2YTDHdH9bnvfrQlMO4Jtvv3u3spt+TQiIjvSvoyVSvjbuCNdVcWETdLugC4TtL9wKOA50fEd+q+l5k90rjuqKb77kvcmLh7N1xyCTzwwP//3ez9+/M8iNAeru4d4Z8HngxcGBHfrbsyST8FvAr4OPAh4DTgWklnRcT1I8tuBbYCbN68ue6qzDqvjYPvuO6oJi8VLTFGsrLOlcDYsKF/raamdOFJAsmhMbjEdhFYBravcX1/AvxBRLxv8Pr9kj4FvAF4zvCCEbEELEH1G+FrXJ9ZJ7V18G27O6rEGMnKOlcC43nPq1od661LqitPEqjzlNvfBo4DXj245HYtjgf+58i0vwYeu8b3M+ultp5WujKAvG1bOweVlBsTm74pb3GxWp9U/UbGegwM6MYTbmFKS0PSTwDPB54K/GfgLRFx02p/M8UngIslbYuIByUdDrwN+KMZ3tOsd9psEbR5Kem0+xvaOhuWHv7vetSVixqmdU89H3gf8E2qZ0xdMOP6tgLbgJslPQT8X6og+sCM72vWK32+uWy1UGqj+2rnTnjwwepHlR58sD+XDTetK/vMqqExuFO7sbu1B4Pn5zf1ftZNXRis64N5vLmsjbPhrpxhd0EX9hn/noY1qiuDdVZGG2fDXTnDtopDwxrV1zuQ+6irLbo2zoZX3m9l8LdLn3e9cWh0VFcPCOPMyx3IfbLeWnTr7fN2mUOjg/r0BclxB3IJXQ/t9daiW2+ft8scGh3Upy9I23cgl9CH0F5vLbr19nm7zKHRQX36gvSp1lRNhXabrZW2B4e71tLyYHh3aO03d+ezsLAQy8vLxdZf6gFtffmC9KnWFE20NLreWlnt/7Ou127pJO2JiIUm39MtjSlKfYG6cD12qj7VmhJwTZzVdrmLcdo+3eXarTyHxhT+AvXXaEDUOQGYNQi73G03bZ/ucu1WnkNjCn+B+mlcQOQ8AVhprVx9dTvvP6xu9+C0fdrjB7Yah8YU/gLV14UxjnEBUeIEYPv2an3bt7fTtbmW7tOUfbpPXY6Wl0Mjgb9A6boyiDouIHKfAORo2ax1Hd6nba0cGtaorowBTQqInAfLtbRsmu5qMmuaQ8Ma1aWDWOmz6botm7a6msya5NCwRq3lQDnPB7w6weWuJusDh4Y1LvUg1pXxj67oUivNbJI6vxFuc6bp33Kua9yZdVdqK6Ht3/c2a4JbGutUF87yJ51Zd6G2tZq1u81dTdZ12UND0uOpfm/8mcABYE9EnJO7jvWuC1c5TRr/aLq2XOMmfQ47s1RZQ0PSIcAngT+IiJdK2gj8as4arNKV/vNxZ9ZN1tbWgXxcEHUhiM3alrul8Wrgxoh4N0BEHAD+OHMNvdD22XGXL9VssrY2DuSTgqhEEM/71WfWPblD42zgvMzr7J1c3Rxd6z8fPQA2UVsbB/JJQZQ7iN0dZiXkDo0TgXskLQEnAz8A3hwRf5G5jk7rQjfHtDPYps9w2zoAtnEgXy2IcgZxF/YTW39yh8ZG4F3A70fEVknHAv9L0l0RccPwgpK2AlsBNm/e3FgBfWjOlx5vmHYAb+MA3+YBsOkDeVe69krvJ7Y+5Q6N/cD5EfElgIj4qqS3AGcCDwuNiFgClqD65b4mVt6X5nzqQamtAJx0/8TKuto4wPsAWF9XwsvWl9yhcR3VpbZfGpq2Ebg/x8r71JyfdnbcZgCOHsA3bXr4uq68svkDfJ8OgCnbPleLtmvjUjb/cofGpcDHJN0cETdLOoZqYPw/5lh5n89mRw9CbXfnDB/AR9e1f387B/i+HACnbfu+tGjN1iJraETEHZJ+BViS9DjgHuC1EfG3OdbfxtlsjjPKcQehtgNw9AA+7rcpch4IuzQWNW3b527Rdmnb2PzLfkd4RNxEdeVUEbMe7Ia/oJDnjHLcQejCC/N155TuOuramfu07ZGzRdu1bWPzz8+eqmH0C3rOOXnOKCcdhHKe7ZfsOuriWNRq2yNnyHZx29h8m7vQaLOpPvoFhTxnlKXP9Evr41hUrpDt47axflNEI1eztmphYSGWl5enLtd2U33c+0O9g3mX+5/Xa21d/twp+l6/tUfSnohYaPI956ql0XZTfdIZf+o6JoVaF770Xe8bb+vMveufO0Vfrjqz+TBXoZGjqT7LF3TSTXOztl6asF77xtfr5zZbq7kKja71/Y+2IMaF2uhB6+qrYfv2/Ge+dQO3C62jJnhMwKyeXofGuANXakug7YPepG6PcaE2fNCCMme+dQJ3Hrp0VnTtRMOs63obGrMcuHIc9CZ1e4yG2uhBCx7e0sh55psauPPWpeMxAbN0vQ2NWQ5cOQ56dbo9Rg9ak858u9IlNPzZNm6EO++saltLTV35TGaWKCI6/99JJ50Uo3btijj00IiNG6t/d+16xCITzfK3dezaFXHppc28f66a69TzyldGHHzw5Jqmff6ufSazeQMsR8PH4962NGbpi87Vjz2u22OtZ9Zd6xJaeWjigw+OrymlC7Brn8nMputtaEBaX3TKQXppCa65Bn75l2Hr1vp1pAbBLGMpXbzKZ7WaUgKhi5/JzFbX69CYZrWb6VamS9XZMsC111b/1gmOOkEwy5l1F6/yWa2mlEDo4mcys9X1JjTW0q0z6SA9PH3UNdfUC406QTDrmXUXr/KZVFNqIHTxM5nZZL0Ije99b23dOpMO0sPTh1saUHVR1VH3Kqn1dGbtQDCbP714YOExxyzE3Xcvc+BAdYnntm3V70msWK0VMmne8PS9e/OMaZiZ5dTGAwt7ERonnrgQX/va8tiWxjzdnWxm1qQ2QmNDk2/WlsMOq8Jg27ZHhsKkhwBas3bvhssuq/41s/Wr6JiGpCOBX4uIN01bdlL/uC/bbJ9bc2a2olhLQ9JBwFXABbO8z8rg8rhWiDXDrTkzW1GypXEZ8F7glFnfyFfptMutOTNbUSQ0JJ0JHBwR75f0zhI1NGG9XDW13i4VNrPJsoeGpJ8CXgW8IPe6m7Te+vndmjMzyDymIelQYAk4NyJ+OGXZrZKWJS3v27cvT4E1lOjn9xVMZlZa7pbGqcBxwEckrUz7l5JuBv40Iq5YmRgRS1QBw8LCQuduJsndz7/eWjZm1k1ZQyMiPkEVGv9M0rcj4uk562hC7n5+P0bczLqgF8+e6qqc/fy+gsnMusCh0RO+gsnMuqB4aETEj5WuoS98BZOZldaLZ0+V4CuVzMweqXhLo01rvfnOVyqZmY03t6Exy4HfVyqZmY03t91Ts9x8t3Kl0saNs12p5C4uM5s3c9vSmOUS1SauVHIXl5nNo7kNjVkP/LNeqeQuLjObR3MbGlD2ElXfjGdm82iuQ6Mk34xnZvPIodEi34xnZvNmbq+eMjOz5jk0zMwsmUPDzMySOTTMzCyZQ8PMzJI5NMzMLJlDw8zMkjk0zMwsmUPDzMySZQ0NSYdLulTSFyXtlfQZSU/LWYOZma1d7pbGzwK3Az8XEU8BLgI+KMmPMzEz64GsB+uIuB64fuj1jZLuBY4DvpKzFjMzq6/omIako4AnAt8oWYeZmaUpFhqSjgY+DlwcEd8vVYeZmaUrEhqStgA7gcsj4qoJy2yVtCxped++fVnrMzOz8bKHhqSXAe8AXhgR75u0XEQsRcRCRCwcccQR+Qo0M7OJsg6ES3oq8Drg1Ii4N+e6zcxsdrlbGq8ALnBgmJn1U+7QOB54s6RbRv57UeY6zMxsDXLfp/HzOddnZmbN8rOnzMwsmUPDzMySOTTMzCyZQ8PMzJI5NMzMLJlDw8zMkjk0zMwsmUPDzMySOTTMzCyZQ8PMzJI5NMzMLJlDw8zMkjk0zMwsmUPDzMySOTTMzCyZQ8PMzJI5NMzMLJlDw8zMkhUJDUm/JmmvpNskvUfSoSXqMDOzerKHhqRTgF8BnhURPwPcBlyWuw4zM6uvREvjAuD1EfH9wesrgNMlPa5ALWZmVkOJ0HgG8IWVFxERwOeAkwvUYmZmNWQNDUk/DnxvEBTD7gaelLMWMzOr76DM6zsUGA2MFQ+bLmkrsHXw8gFJt7RZWEMOB75VuogErrNZfaizDzWC62zaCU2/Ye7Q2Af8xJjpRwLXDk+IiCVgCUDSckQstF/ebFxns1xnc/pQI7jOpklabvo9s3ZPRcSPgNslnbQyTZKABYbGOczMrJtKDIRfDlwq6ZDB6wuAj0fEdwrUYmZmNeTuniIiPirpaOALkg4GPgO8ZsqfLbVfWSNcZ7NcZ3P6UCO4zqY1XqceeSGTmZnZeH72lJmZJSsWGnWePyXpZEk3SbpF0mckHTcy/6cH02+R9DlJz8hdp6SNkn5T0o2SviTpryX9+6H5vy3pDkk3D/33sQJ1vk3SrSN1XDWyTCvbs0aNbxyp72ZJ/7CyPdvelkN1nC3pyCnLFNs3U2osvV/WqLPYflmzzmL7pqTDJV0q6YuD79FnJD1tleXb2TcjIvt/wCnAp4DHDF7/DnDlhGUfC+wFjhu8PhX43ND8jcAe4KTB6xOBW4BDC9R5KfDYwesjgVuBJw9eXwKc34Ht+T+AM1d5r1a2Z50ax/ytgL8Fjmp7Ww6t8xjgq8DTV1mm2L5Zs8Yi+2XNOovsl3XrLLlvAs8BzgE2Dl4/G/g74KCc+2aplkad509tBd4bEf8AEBH/G/h7SacN5p9FtTH2DObfCvwl8JKcdUbEfRHx+oi4b/D6bmAHsKWBOhqrM0Fb23OWGp8P7I2Iu2asIYmkHVSXgB89ZdFi+2ZqjYX3yzrbcpo2v+ez1Jlt34yI6yNie0QcGLy+EbgXOG7M4q3tm6VCo87zpxapzlCHfRJ4buL8XHU+jKSNVGcGtzZQxzRNPs9rkXa25yw1/ibwhzOuP1lEnBERRwG7piy6SKF9s0aND5N5v1xznWMs0t73fJY6s+6bwyQdBTwR+MaY2Yu0tG+WeDR63edPPWkwb9Ky0+bnqnP4bx8NbAduj4gbhmadKmnHoF/505KeM0uNM9R5lqRrJf3NoJ6nDs1rfHvOuC2PA44e2Y7QwrZcgyL75lrl3C/XKOt+OauS+6aq2xY+Dlw81Hof1tq+mf0+DWo8f2rK8pE4f63q1gn8c/p/ALgReP3QrBuA7wLviYh7JS0AH5R0WkR8JWOdO6iem7M9Ir4v6ReAHZKeFhH3rPJ+s2zPNW3LgVcBbx+Z1ta2rKvUvllbgf2yrhL75ayK7JuStlCF/yUR8b4Ji7W2b5YIjeTnTw3842DeXSPLfn1k/uh7fZ3Z1K0TVY9H+VPgdyLio8PzIuJTDDUHI2JZ0tXA6cAsO1OtOiPiL0ZeXyvpeqqB6o/SzvasvS0BVF1ddRYwfMbZ5rasq9S+WUuh/bKWQvvlmpXaNyW9DHgt8MKIuG2VRVvbN7N3T0X950/dwCP72Z4LfDpxfpY6VT0W5U+Al4x+MVfxWODbOetMqKPx7TlDjS8BPjKh+T1q5m25BkX2zTpK7ZcNaXW/nFH2fXPQXfc64N9NCQxoc9+sc8lXU/8BLwQ+ARwyeH0h8ObB/34rcOrQsk8EvgxsHrxeBHYNzT+M6pK3pw1eP5nqUrNDMtd5NnDFKu91JnDK0OvTB5/rxzLXeQ7w1KHXLwNuAh7V5vasU+PQ3ywD/yrnthxZz06GLr/s0r5Zo8Zi+2XNOovsl3XrLLlvAv8dOGPCvGz7ZonuKWL1508dBzxhaNlvSHoF8AFJ/4KqWfXiofnfk3Q28A5JhwP3AS+NiB/krBM4HnixpNNH3ubDEXER1fXfF0l6G/Ao4A7gFyNi5jOQmnXeCbxJ0hMGdXyBakf80eC9WtmeNWtE0r8B/iki/m7M232VlrblFJ3ZN1NrpOB+WbPOIvvlGuosuW8eD5wm6b+OTP/d0Trb3Df97CkzM0vmZ0+ZmVkyh4aZmSVzaJiZWTKHhpmZJXNomJlZMoeGmZklc2iYmVkyh4bZFJIOlfR/JN05uDFxeN5Vkg5IevGkvzebJw4Nsyki4n7g96geG/2qlemSLgNeDrw6Iv68UHlmWfmOcLMEgx8v+hvg8cBPAr9B9byf34uIN5SszSwnh4ZZIkkvoPpJzOuongj6hxHxmtX/ymy+uHvKLFFE/BXwReA04P1Uv2vwMJLOk/R5ST+QtDNziWatK/KUW7M+kvQfgKcPXn43xjfT7wLeBPwcsCVXbWa5ODTMEgx+fvS9wIeAHwG/LumtEXHr8HIR8cHB8pvzV2nWPndPmU0h6VnAB6l+X/ulwEXAQ8BlJesyK8GhYbYKSScCO6h+TOfMiHhg8OM77wZeJOnZRQs0y8yhYTbBoIvpWuA7wOkRcd/Q7DcA9wOXl6jNrBSPaZhNEBF3Ut3QN27eXcBj8lZkVp5Dw6xBkg6i+l4dBGyQdAjwUET8sGxlZs1waJg16yKqR46suB+4HlgsUo1Zw3xHuJmZJfNAuJmZJXNomJlZMoeGmZklc2iYmVkyh4aZmSVzaJiZWTKHhpmZJXNomJlZsv8H37EWTcKceKUAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x116c56860>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.plot(X, y, \"b.\")\n",
    "plt.xlabel(\"$x_1$\", fontsize=18)\n",
    "plt.ylabel(\"$y$\", rotation=0, fontsize=18)\n",
    "plt.axis([0, 2, 0, 15])\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "정규방정식을 사용해 $\\hat{\\theta}$을 계산"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {},
   "outputs": [],
   "source": [
    "X_b = np.c_[np.ones((100, 1)), X]  # 모든 샘플에 x0 = 1을 추가합니다. 두 개의 1차원 배열을 칼럼으로 세로로 붙여서 2차원 배열 만듦\n",
    "theta_best = np.linalg.inv(X_b.T.dot(X_b)).dot(X_b.T).dot(y) #inv():역행렬 계산, dot():행렬 곱셈 "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[4.21509616],\n",
       "       [2.77011339]])"
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "theta_best"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "$\\hat{\\theta}$을 사용한 예측"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[4.21509616],\n",
       "       [9.75532293]])"
      ]
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "X_new = np.array([[0], [2]])\n",
    "X_new_b = np.c_[np.ones((2, 1)), X_new]  # 모든 샘플에 x0 = 1을 추가합니다.\n",
    "y_predict = X_new_b.dot(theta_best)\n",
    "y_predict"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "아래 그래프는 모델의 예측을 나타냄"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 71,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAY0AAAETCAYAAADKy1riAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvNQv5yAAAIABJREFUeJzt3XucXHV9//HXZ2+5QQBzISEhJFwSuQbDEjJZf2RqsIoghYflVwQLpUrqT6qitf2ByqMUBFR+CLX+tA1WTaK0VBHqryhNWFiqyQbYhEC4BKgIkZALhECAXPb2+f1xZndnJrs7Z3Znzjkz+34+HnksM3Nm5jOHM+d9vt/vOd8xd0dERCSMmrgLEBGRyqHQEBGR0BQaIiISmkJDRERCU2iIiEhoCg0REQlNoSEiIqEpNEREJDSFhoiIhFYXdwFhTJw40WfOnBl3GSIiFWXdunWvu/ukUr5mRYTGzJkzaWtri7sMEZGKYmYvl/o11T0lIiKhKTRERCQ0hYaIiISm0BARkdAUGiIiElpFnD1VyO7du9mxYwcdHR1xl1IR6uvrmTx5MuPHj4+7FBGpMBUfGrt372b79u1MmzaNMWPGYGZxl5Ro7s7evXvZsmULgIJDRIpS8d1TO3bsYNq0aYwdO1aBEYKZMXbsWKZNm8aOHTviLkdEKkzZQsPMLjSzKQM8Ns7MrjWz0cN9n46ODsaMGTPclxlxxowZo+48ESlaWULDzKYDtwD9hgbwXeDLwLBDI/N+pXiZEUXrTESGouShYWb3AY8BRwzw+JXAM8D2Ur/3SNPV1dX73+4eYyUiMlKUPDTc/Rx3nwqsyX/MzOYDaeCbpX7farJlyxamT58OQFNTEw888EC/y33ta1/jlFNOYf78+Vx33XURVigiI1VkA+FmNoEgLJa4DosBuP/++5kzZw4zZ87kiiuuoLOzEwi6jurr6wEYPXo0dXV9J7l94QtfYO7cuSxYsIBVq1YxduxYampqePjhh1m0aBHz5s3jK1/5SiyfR0SqXySn3JpZDfAD4K/cfVfI5ywBlgDMmDGjjNXFY9euXXz605+mubmZmTNncv755/OjH/2IT33qU7g7tbW1QHBNRXbG3nrrrdTUBFn/zjvvsGDBAsyMlpYWRo8uyRCRiMiAomppvBd4H/DPZrbBzDYQjHn82sxu6+8J7r7U3RvdvXHSpJJOB58Id955JxdddBHHHHMMtbW1XHPNNdx111184hOf4Kyzzupdric8etTU1NDV1cXKlStZtGgRn/3sZ7niiitoamqiubk56o8hIiNMJC0Nd38GyGkumNlLwP9w9zejqCFpXnjhBVKpVO/tk046ic2bN7Nq1SpeeeUV0uk0EHRV9bQ0Xn75ZS699FKef/55Fi1axIoVKzjhhBMASKVS3HjjjVx++eWceeaZrFixQmdIiUjJVfwV4f2Ka2dZxFBNV1dXbzcTBC2I7u7uAV42eN2jjjqK5cuXM3ny5AOuTTn99NO599576ejo4K233lJgiEhZVGdoVICjjjqKTZs29d5+8cUXmTVrFrfddhubNm3qDYqe02qfffZZzjvvPMyMmpoa6urqaGho6B0wb29vZ9++fezdu7f378aNG6tyPEhE4lO20HD3dIHHZ5brvYs54o/LJZdcQlNTE5dddhnTpk3jG9/4BpdeeiknnXQSM2fOZNWqVQB0d3fj7syePZv169fT0NBAQ0PDoC2Jjo4Ourq6GDVqVFQfR0RGCLU0YjJ16lS+/e1v89GPfpR9+/Zx4YUXcskll2BmTJw4sberqic0amtrOfjgg1mxYgXXX389o0aNoq6uDnenvb2dzs5OOjs7aW9v54UXXmDs2LExf0IRqUYKjRide+65nHvuuf0+lt891ePjH/84F1988QFnVQHs27ePMWPGDDg2IiIyXAqNBOrs7OwNjZ6WRo/sC/3yqTtKRMqt4qdGr0aTJ0/mrrvuAoKWRtgL6HuuKBcRKRe1NBJo7Nixvddw/OIXv+DQQw8N9bz6+nq2bt3KuHHjylmeiIxgCo2EO+yww4pafsqUgWajFxEZPnVPiYhIaAoNEREJTaEhIiKhVUVo6Oc5iqd1JiJDUfGhUV9fz969e+Muo+Ls3bu3d94qEZGwKj40Jk+ezJYtW9izZ4+OnkNwd/bs2cOWLVuYPHly3OWISIWp+FNux48fD8Crr75KR0dHzNVUhvr6eg4//PDedSciElbFhwYEwaEdoIhI+VV895SIiERHoSEiIqEpNEREJDSFhoiIhFa20DCzC81sStbtsWZ2jZk9amYbzWytmaXL9f4iIlJ6ZQkNM5sO3AJkT7k6A9gPnOnuJwOXAT80s0nlqEFEREqv5Kfcmtl9wDxgQvb97r4J2JR1+zkzWwecCqwqdR0iIlJ6JW9puPs57j4VWDPYcmY2DmgEni91DSIiUh6xDISb2SHAvwMr3P3lOGoQEZHiRR4aZjYH+DXwS3e/dpDllphZm5m1vfbaa9EVKCIiA4o0NMzsw8C9wKfd/VuDLevuS9290d0bJ03SWLmISBJENveUmR0O/ANwlrqkREQqU5QtjUuB2xUYIiKVK8pZbmcDi83sf+Xd/113/26EdYiIyBCVLTTcPZ13+4pyvZeIiERDc0+JiEhoCg0REQlNoSEiIqEpNEREJDSFhoiIhKbQEBGR0BQaIiKDaG2Fm28O/kq0F/eJiFSU1lZYvBja26GhAZqbIZWKu6p4qaUhIjKAlpYgMLq6gr8tLXFXFD+FhojIANLpoIVRWxv8Tafjrih+6p4SERlAKhV0SbW0BIEx0rumQKEhIjKoVKoywqK1NZpwU2iIiFS4KAfsNaYhIlLhohywV2iIiFS4KAfs1T0lIlLhohywV2iIiFSBqAbs1T0lIjIMI22akbK1NMzsQuDX7r4t676PADcCo4DngU+6+85y1SAi8YnqFNA4hTlrqdrWQ1lCw8ymA7cA5wPbMvcdDVwPfNDdXzezi4DvAxeUowYRic9ImbOpv7OWsj9nOdZD3CFU8u4pM7sPeAw4Iu+hLwLfdPfXAdz9X4HJZnZcqWsQkXiNlDmbCp21VOr10BNC114b/I2jS6zkoeHu57j7VGBN3kNp4IG8+5oz94tIFRkpczb1nLV0ww39tyJKvR6SEMZRnj11uLu/kXffNuDICGsQkQiMpDmbBjtrqdTroSeEerq74gjjKENj9AD3e393mtkSYAnAjBkzylWTiJRJpczZVG6lXA9JCOMoQ2OnmR3m7ruy7psCbO5vYXdfCiwFaGxs7DdYRESGIu7B5OGIO4yjDI3fAB8A7s66Lw18MsIaRGSES+qZXZUSZFGGxreAfzKzB919l5ldDOxw9xcirEFERrhCp8nGIT/Ibr8ddu5MZoBEFhruvt7MbgIeNLMxwFPAFVG9v4gIxD+Y3F+LIjvI9u+HK68E92S1hHqULTTcPd3PffcA95TrPUVGukro4oi7xjgHkwfqGssOspqaIDy6u5PTEsqmCQtFqkRS++qzJaXGuAaTB+oayw6yCRPgqquCFkdNTXA7SRQaIlUiiX31+SqhxnIarGssP8iuvDJYT1ddBSefnJz1pNAQqRJx99WHUQk1llPYrrGdO4MxjSF1Ue3dC21tsCZ/Uo7SUGiIVIkkXPhVSCXUWG5husaKCtft22H16r5/69dDR0cJK85l7sm/bq6xsdHb2triLkNEKlDcA+9D1W/d3d3wzDO5IfHii7lPNAv6s5qasO99b527N5ayLoWGiFStpAy8D9m778IjjwRdTatXBx/orbdylxk3DhYsgKam4N8ZZ8AhhwBgZiUPDXVPiUhVyT5Cr7iB91deCcKhJyQ2bAiKzzZjBixc2BcSJ58MddHtyhUaIlI1+ruyOrED752dsHFjbkhszpuKr7YWTjstNySmT4+n3gyFhohUjfyWxc6d0Q68Dzp+sns3rF3bNxbxyCPwzju5yxxySPDEpqYgKObPh4MOKm/RRVJoiEjVyD7rqK6u78D9mmvK/965rRynecVWUvse6guJjRuD82izHX10Xwti4UI48cTgir4C7zNSpkYXkRGu3Du8nlN6ly+HH/wA7rgDli2LYAC8o4OWFdto3zeNLq+hfW8nLX/8D6T4et8y9fUwb15uSEyZUtTbJGFgX6EhIpGIaoeXSgXB1NVVugHwA8LujTeCO3vGIx59lPTeuTTQTDv1NNBB+uD1sOjcvpBobIQxY4b12ZIwsK/QEJFIRLnDK+WV561rnMWLnfb90FDTSfP0y0i9/K8HLJeas4vm426npeGDpP94EqmL7g+umSihJFxRr9AQiUHc/dJxiHKHN6wrz/fvh3XresciWh5opH3f1XRRR3uX0fLyTFKjRsHpp/d1My1cCBMnkgLK+b8zCVfUhwoNM/tH4C+Aae7+at5jc4CNwPfc/fOlL1GkuiShX7qcBgrE7PGGKBSarqO3zrm7SLU/3Hfaa1tb8D8nI812Gvgr2oGGekh/52K47DoYNarcH6FflfJzr60EoTEfuDfvsduA3cB1pStLpHoloV+6XMIE4rJlweORDFDn6+6GTZtoXf4Ci//Ph2nvqqWBUTTzDVKs7VvuxBN7xyJSTU00bx9Ny8OWCcKTIyw4ecKGRs/azAkNMzsHOBu40t13lbg2kaqUhH7pcikUiMUEZkm68Pbsgcce6xuwXrMGdu2ihatp55ygywmn5eg/J/XxxUFQLFgAhx2W8zKpYyC1cIg1VJlQoeHuz5nZGwShAYCZ1RP87vdTwD+VpzyR6pOEfulyKRSIYQMzTIul31DZujV3Mr/HHw+uvM42bRrpObU0/Npp73YaGupI//iK8g5GVJFiBsLXAk1mZh7Mcvh5YDZwlrt3Df7UPmb2R8BXgLFAF/B1d/+XIuoQqXhR9ktHOeheKBDDBmahFkkQKpkzmmo7af6DG0k9vwxeein3hWpq4NRT+wasm5pgxgxSZjSPwJMRSqHY0PgIMCfT6rgWuNfdm8O+gJmdCPwd8IfuvsPMjgRazGyjuz9VTOEiUlgcg+6FAnHIvyfx9tu9M762rJhO+95Lg+6lbmhZuZ8UL8HBB+fO+Dp/PowfP+Q65EDFhEZr5u984ExgFPBXRb7fAqDZ3XcAuPvvzew3wByCbi4RKaFKHXRPpaD5J9touWs76c4HSF35E3jiCVq759NCmgm00sBFwRlNtd2kr2qEP90AJ50UTPInZVNMaDwCdAOfBN4P3OLuLw7+lAOsA64zs39x9zYzOxc4A/hcka8jIgPI7o6qmEH3zk544om+sYg1a0i98krOMENr7ftZbKuCK67rndtveJudXWNJp+tIpT4WW+kjTejQcPe3zewZglbGNuDGYt/M3TeY2dVAs5ntBeqBD7n7WwWeKiIh9NcdVepB95KMkbz55oEzvu7Zk7vMoYf2jkO0HvJhrrvnFPY/VBf8bnYX7Ow6LJKJCCVXsVeEPwqcBFzj7m8X+2ZmdizwGeBXwD3AYmClmV3g7g/nLbsEWAIwY8aMYt9KJPHKMUDdX3fUNdeU7vWHNEbiHvwkac/Fc6tXw9NPHzjj67HH5k7md/zxUFPT+5779weXWdTUJLzVVEZJmEkgdGhkTrFNA23AsiG+34+Bb7v7nZnbd5nZA8D1wKLsBd19KbAUgp97HeL7iSRSuQaoy90dFWqMpL0d1q/P/XGh7dtzl2loCH5cKDskJk8e9D17AuOss+C66ypjbKaUkjKTQDEtjS8Bs4BLfOg/LD4b+Le8+x4H+j+9QaRKlWuAutzXgPQbSjt39oXDmjW0PlJDS3uKNC19V1lPnJh72utpp8Ho0aHfs7Y2CI26upEZGJCckxoGDQ0zew/wIeAU4K+Bb7n72sGeU8B/Atea2Q3u3mlmE4G/B/5xGK8pUnHK2SIo56mkqQVO8w820/LT10i3ryR1+XJ47rnex1tZwGKaaaeBhtpumv/3SlKXzYbjjhvWjK89Ty3xpLEVJSknNRRqaXwIuBPYQTDH1NXDfL8lwA3ABjPrBt4hCKKfDfN1RSpKxVwVvm9fMA1HVksitXNn7sXTo0cH10M0NdGy5U9p/8kYurqMdqDloHNJzR5eCS0twclV7sHfSjltuNSSss0MGhqZK7VLdrV2ZvD8qlK9niRTEgbrKkEiLy7bvj13LGLdOujoyF1mypS+sYimpuCK64YGANKt0PDT0h4NJ+UIOwmSsM3o9zSkpJIyWCchdHfDM8/khsRvf5u7jBmcfHJuSMycOWA/UTmOhpNyhC0BhYaUVFIG60aColt0774Ljz7ad9prayu8lXeJ1LhxwTQcPQPWCxbAIYcUVVc5joZ7Xq+lJfe2RE+hkVCV1MVTkVcgV7hQLbotW3JnfN2wIUjzbEcemXtW0ymnBKcoJYxasMmRvK1DKuoLEsUVyHFIemgf0KJ7sJvU6CdyL6DbvDn3SbW1MG9e7rURRx4ZS/3FUgs2ORQaCVRJX5ByX4Ech0oI7fTp79JQN5r2bmjwDtI3ngNffTB3ofHjg8KzZ3w96KB4Ch4mtWCTQ6GRQJX0BamkWsMqVWiXrLXiDi+/nDuZ38aNNGdmfE17C6m9a+Hoo/u6mZqa4IQThjzja9JaWhoMTw6FRghRf4Eq6QtSSbWGVYogHFZrpaMjGH/ICglefTV3mfp6Uqc7qYX7oelLQVhMnVpUfQP9P0tqSysJp5uKQqOguL5AlfQFqaRawxwAlCIIi2qt7NoVFNYTEo8+Cnv35i7znvcEwdDTkjj9dBgzpvjCKLxNV1L3qERPoVGAvkCVKz8gijkAGG4QDthacYf//u/cayOeeebAF5g9O3fAes6cYLa+Eii0TVdjl6OUjkKjAH2BKlN/ARHlAUBPa2X5Dzthx2vwkwfhmz8LgmLHjtyFR42Cxsa+kEilYNKk0O9VbPdpoW26GrscpXQUGgXoC1S8JAyi9hcQkRwAvPZaXwvi/jdZtvF22pnEMi6gme+QYkcQCNlXWM+bFwTHEAyl+zTMNl1JXY4SLYVGCPoChZeUQdT+AqLkBwDd3cEMr9kX0L3wQu/DLVxNOw10UUe7GS0f+w6pmw+BY44p2XStQ209aZuWoVJoSEklZQxooIAY1s5yz57cGV9bW+GNN3KXGTMGzjgDmppIH/YRGq6tzQRXLekvngbHDv4Wpe5qEik1hYaUVJJ2YsM+mt66NXfAev36YG7ubEcckTtgfeqpUF8fvD/QvDB8CJSrq0mklBQaUlLF7sSSMP4BBE2jp5/ODYnf/S53mZoamDs3NySOOmrQrqZigktdTVIJFBpScmF3YrGOf7zzDjzySN9YxNq1sHt37jIHHRTM8toTEmecEUzNUSZJaqWJDEShMYLFfZQ/2JF1yWvbvDl3Mr8nnggGsrMddVTujK8nnzzkaTiGQl1NUgkUGiNUEs5yGujIeti1dXYGoZDd1fTKK7nL1Nb2XRvRExLTpg37Mw037NTVJEkXeWiY2WSC3xufB3QB69z9sqjrGOmScJbTQEfWRdf21lt903CsWRN0O737bu/DrSygZfQVpE99k9S5E4KQmD8/+MGhEkpCEIuUW6ShYWajgVXAt939EjOrBS6NsgYJJKX/vL8j60Frcw8GqLOvjXj66eD+bMccA01NtE65gMXfPo/2DqPhCaP5W6XZkffXokhCEIuUW9Qtjc8Cq939nwHcvQv4YcQ1VIRyjzckuf88p7amDlI16+FbWTO+btuW+4T6ejjttNyzmg4/HICWm6G9o7Q78oFaFHEEcdzjUjLyRB0aFwJXRvyeFSeqbo6k9Z+3tkLLL98lPf5xUm/cR2r1arj+Mdi3L3fBCRNyxyIaG2H06H5fsxw78oFaFFEHsbrDJA5Rh8bxwBtmthSYD+wDbnX3n0ZcR6IloZuj0BFsSY5w3eH552H1alrv2cbi+75Au4+igXk089ekWBss99735obE7Nmhp+Eox458sCCKMoiTsJ3IyBN1aNQCdwB/5+5LzGwmcL+ZbXX332QvaGZLgCUAM2bMKFkBldCcj3u8odAR7JCPcPftg7a2vm6mNWvg9deBnnma6oN5moCWRX9L6kudwQtPmDCsz1PqHXlSuvbi3k5kZIo6NHYCV7n7kwDu/pKZfQs4H8gJDXdfCiwFaGxs9PwXGopKac6H3SmVKwD7O4LtuT+dLuIId/v23Gsj1q0LfpUu2+GHB/M0TTuFhqVGe6fT0FBH+uYPB/NwyICSEl4yskQdGs0Ep9o+mXVfLbC3/8VLq5Ka84WOjssZgPlHsBMm5L7X7bf3c4Tb3R38mFB2SPz2t7kvbBZcMJfd1TRrFpgF8zR9vDJ2gGHWfVQt2qSNS0n1izo0bgJ+aWYb3H2DmU0nGBj/kyjevJKb8/k7oXIGYP4RbP577dwJzf+xl5Z/eZV0za9J3fBvQYFvvpn7QuPG9c74ysKFwZQchx466PtWwg6w0LqvlBatyFBEGhru/ryZfQJYamaHAG8An3f3p6N4/3I056M4ouxvJ1TuAMzfgTfUd9Pu0GCdpJd/mtS1y0l1deU+afr03NNe586FutJsYkkaiyq07qNu0SZp3Uj1i/yKcHdfS3DmVCyGezSb/QWFaI4o+9sJXXNNGfuzu7pg48bebqbUmjU075tKC2nS3S2kNq0NZnx93/tyf4HuyCNLWESfpB25Fzr4iLJFm7R1I9VPc08VIf8Letll0RxRDrQTKll3zu7dB874+s47OYukxu8itWA8NJ0NTV8Lup0OOqgEb15YEseiBlv3UQ5QJ3HdSHWrutAoZ1M9/wsK0RxRlnQn5A4vv5w7md/GjQfO+DprVu6A9YknRjrja7ZKHIuKanymEteNVDbz/Dl7EqixsdHb2toKLlfupnp/rw/F7cwj73/u6IANG3JD4tVXc5epq4N582iddTEttR8gfdFUUh+dGEFx4ZVzvVX6mECl1y/lY2br3L2xlK9ZVS2NcjfVBzriD/seA4VaSb/0u3blzvj66KPBb1tnO+ywvhZEZhqO1ifG9tV2T/L6xst15F4NYwKVctaZVIeqCo0omurD+YIOdNHckFsv7sG1ENkzvj7zzIHLHXdc7oD1nDnBQHaB2kbCjmikfm6Roaqq0EjaFbL5LYj+Qi1/p7V8OSxbNsCR7/79wVXVPd1Ma9bAjh25b9rQAKef3teSWLgQJk0qWGuxgVstXSIaExApTkWHRn87rrAtgXLv9Abq9ugv1LJ3WpAdIk7Ld58lde+yICTa2oLgyDZpUu6A9WmnwahRRddbTOBWQ5dOj6QdaIgkXcWGxnB2XFHs9Abq9sgPtd6d1kPdpGdthiefZBkfop1aGrraSf/4k9Az2yvACSfkhsSxx4ae8bWQsIFbbV06GhMQCa9iQ2M4O64odnoFuz327oXHHgsunlu9mlRrK7zxBgDNLAgupGtoJZUaBU1fDn6Frvb9tKwfH/sRcfZnq62FzZuDIB5KTdXSzSUyUlRsaAynLzqqAfOcbo9Z2+DurAHr9euhszP3SVOnQlMTqcw/Tr0++FU6ktUl1PPZli+HH/4Q7rgjGIfpbwr1wQIhSZ9JRMKp2NAYTl902fuxu7rg6adJPbGG1DOr4Y7Vwe9aA60soIUPkKaW1Nw9uae+HnXUgF1NSesS6pk0sbOz/5rCBELSPpOIFFaxoQHh+qLDdH8sXQp33w0f+xgsWVJ8Ha3Ne2i581XSNf9F6veZGV93785d6KCDaH3v5SzecCvt3bU0jDKav2ehd5JJPMtnsJrCBEISP5OIDK6iQ6OQwS6m67nfrK+XaOXK4G/B4Pj973u7mVpXvs3i579LOzNp4AiauYMUu2HGjNwZX08+mZZb6mh/HLq6iz+yTuJZPoPVFCYQkviZRGRwFRMaQxkwHehoN/v+fHffnRcanZ3w5JO5F9C98krfe3A17TQEP1NqRsuf/BOpW94TTBOeZ7hH1kk8y2egmsIGQhI/k4gMrCJC4913hzZgOtBOOvv+7JYGwMc+sgfu/6++C+geeSQoINshhwQFNDWRPvRsGv6mNvMetaQ/dwocmBfAyDuyViCIVJ+KmLBw+vRG37atja6u4BTPG24Ifk+ix2CtkIEea22Floec9OxX2fjLzdy9ajwf6/4pS7ZeH0zPke2YY3IHrE84IWcaDp02KiJJVI4JCysiNI4/vtFffrmt35ZGUadttrfD44/nzvi6bVvuMvX1wVXVPWMRCxfClCll/XwiIuUwYme5HTdu4G6dQc/SeeONvnBYvTq4mG7fvtwXnzAhd56mxkYYMyaSz1VJ1JoSEYg5NMxsCvBn7v71QssO1D/eNz7hNNQ76Xd+CZ+6NwiJTZsOfMKcOblnNc2ZU7JpOKqVLsITkR6xhYaZ1QHfB94PFAyNA+zbB21twe9Xn/Z6ML3GnvtI3ZQ1T9OoUcGMrz0hkUrBxGT9uFAl0EV4ItIjzpbGzcAKgtAobMeO3LGIdet6f3M1lfnH5MnQdEFfSMyb1zd1rAyZLsITkR6xDISb2fnAB9z9c2b2prsfOtjyjaNHe1v+lOBmwe9WZ8/4evTRkXY1jaR+/pH0WUWqRVUMhJvZscBngHNDP2n/fhg7Fs44oy8kUik4dNCsKauR1s+vay5EBCIODTMbAywFLnf39gLLLgGWABwzZUow/3ZmxtckiKOfX0f7IhK3qFsaZwKzgH+3vm6kg81sA/ATd7+l5053X0oQMDQ2NnqSAgOi7+cfaS0bEUmmSEPD3f+TIDR6ZcY0To2yjlKIekoQncEkIklQERf3JVWU/fw6g0lEkkChUSFG2mSHIpJMsYdGodNtpY/OYBKRuNUUXmRkam2Fm28O/oqISCD2lkY5DfUUVZ2pJCLSv6oNjeHs+HWmkohI/6q2e6q/HX9YPWcq1dYO70wldXGJSLWp2pbGcE5RLcWZSuriEpFqVLWhMdwd/3DPVFIXl4hUo6oNDYj3FFVdjCci1aiqQyNOuhhPRKqRQqOMdDGeiFSbqj17SkRESk+hISIioSk0REQkNIWGiIiEptAQEZHQFBoiIhKaQkNEREJTaIiISGgKDRERCS3S0DCziWZ2k5mtN7ONZvaQmc2NsgYRERm6qFsaJwLPAae7+8nAV4Gfm5mmMxERqQCR7qzd/WHg4azbq81sFzD2e8bHAAAIz0lEQVQLeCHKWkREpHixjmmY2VRgGrAlzjpERCSc2ELDzI4AfgVc6+574qpDRETCiyU0zCwFtADfdPfvD7DMEjNrM7O21157LdL6RESkf5GHhpldDnwPOM/d7xxoOXdf6u6N7t44adKk6AoUEZEBRToQbmanAF8EznT3XVG+t4iIDF/ULY2/AK5WYIiIVKaoQ2M2cKuZPZX3748irkNERIYg6us0Phjl+4mISGlp7ikREQlNoSEiIqEpNEREJDSFhoiIhKbQEBGR0BQaIiISmkJDRERCU2iIiEhoCg0REQlNoSEiIqEpNEREJDSFhoiIhKbQEBGR0BQaIiISmkJDRERCU2iIiEhoCg0REQlNoSEiIqHFEhpm9mdmttHMNpnZD8xsTBx1iIhIcSIPDTN7P/AJ4Ax3fy+wCbg56jpERKR4cbQ0rga+7O57MrdvAc42s0NiqEVERIoQR2i8D3is54a7O/AIMD+GWkREpAiRhoaZHQa8mwmKbNuAI6OsRUREilcX8fuNAfIDo0fO/Wa2BFiSubnfzJ4qZ2ElMhF4Pe4iQlCdpVUJdVZCjaA6S21OqV8w6tB4DXhPP/dPAVZm3+HuS4GlAGbW5u6N5S9veFRnaanO0qmEGkF1lpqZtZX6NSPtnnL3DuA5Mzut5z4zM6CRrHEOERFJpjgGwr8J3GRmozO3rwZ+5e5vxVCLiIgUIeruKdz9F2Z2BPCYmY0CHgI+V+BpS8tfWUmoztJSnaVTCTWC6iy1ktdpB57IJCIi0j/NPSUiIqHFFhrFzD9lZvPNbK2ZPWVmD5nZrLzHj8vc/5SZPWJm74u6TjOrNbO/NLPVZvakmT1uZn+c9fiXzOx5M9uQ9e+XMdT592b2bF4d389bpizrs4gab8yrb4OZ/a5nfZZ7XWbVcaGZTSmwTGzbZpga494ui6gztu2yyDpj2zbNbKKZ3WRm6zPfo4fMbO4gy5dn23T3yP8B7wceAMZmbv8NcPsAy44HNgKzMrfPBB7JerwWWAeclrl9PPAUMCaGOm8CxmduTwGeBU7K3L4OuCoB6/NHwPmDvFZZ1mcxNfbzXAOeBqaWe11mved04CXg1EGWiW3bLLLGWLbLIuuMZbssts44t01gEXAZUJu53QT8FqiLctuMq6VRzPxTS4AV7v47AHf/L+BFM1ucefwCgpWxLvP4s8D/Ay6Osk533+3uX3b33Znb24D7gFQJ6ihZnSGUa30Op8YPARvdfeswawjFzO4jOAX8iAKLxrZthq0x5u2ymHVZSDm/58OpM7Jt090fdvdl7t6Vub0a2AXM6mfxsm2bcYVGMfNPpQmOULOtAj4Q8vGo6sxhZrUERwbPlqCOQko5n1ea8qzP4dT4l8B3hvn+obn7Oe4+FVhTYNE0MW2bRdSYI+Ltcsh19iNN+b7nw6kz0m0zm5lNBaYBW/p5OE2Zts04pkYvdv6pIzOPDbRsocejqjP7uQ3AMuA5d/9N1kNnmtl9mX7lB81s0XBqHEadF5jZSjN7IlPPKVmPlXx9DnNdzgKOyFuPUIZ1OQSxbJtDFeV2OUSRbpfDFee2acFlC78Crs1qvWcr27YZ+XUaFDH/VIHlPeTjQ1VsnUBv+v8MWA18Oeuh3wBvAz9w911m1gj83MwWu/sLEdZ5H8G8OcvcfY+Z/SFwn5nNdfc3Bnm94azPIa3LjM8A3827r1zrslhxbZtFi2G7LFYc2+VwxbJtmlmKIPyvc/c7B1isbNtmHKERev6pjFczj23NW/b3eY/nv9bvGZ5i68SC6VF+AvyNu/8i+zF3f4Cs5qC7t5nZcuBsYDgbU1F1uvtP826vNLOHCQaqf0F51mfR6xLAgrOrLgCyjzjLuS6LFde2WZSYtsuixLRdDllc26aZXQ58HjjP3TcNsmjZts3Iu6e8+PmnfsOB/WwfAB4M+XgkdVowLcqPgYvzv5iDGA+8GWWdIeoo+focRo0XA/8+QPM737DX5RDEsm0WI67tskTKul0OU+TbZqa77ovAHxQIDCjntlnMKV+l+gecB/wnMDpz+xrg1sx/3wacmbXsNOAZYEbmdhpYk/X4OIJT3uZmbp9EcKrZ6IjrvBC4ZZDXOh94f9btszOf69CI67wMOCXr9uXAWqC+nOuzmBqzntMGHBPlusx7nxayTr9M0rZZRI2xbZdF1hnLdllsnXFum8D/Bc4Z4LHIts04uqfwweefmgUcnrXsFjP7C+BnZnYQQbPqoqzH3zWzC4HvmdlEYDdwibvvi7JOYDZwkZmdnfcy97r7VwnO//6qmf09UA88D3zY3Yd9BFJknZuBr5vZ4Zk6HiPYEDsyr1WW9VlkjZjZQmC7u/+2n5d7iTKtywISs22GrZEYt8si64xluxxCnXFum7OBxWb2jbz7v5JfZzm3Tc09JSIioWnuKRERCU2hISIioSk0REQkNIWGiIiEptAQEZHQFBoiIhKaQkNEREJTaIgUYGZjzOwVM9ucuTAx+7Hvm1mXmV000PNFqolCQ6QAd98L/C3BtNGf6bnfzG4GPgl81t3/NabyRCKlK8JFQsj8eNETwGTgaOBTBPP9/K27Xx9nbSJRUmiIhGRm5xL8JGYzwYyg33H3zw3+LJHqou4pkZDc/T+A9cBi4C6C3zXIYWZXmtmjZrbPzFoiLlGk7GKZ5VakEpnZ/wROzdx82/tvpm8Fvg6cDqSiqk0kKgoNkRAyPz+6ArgH6AD+3Mxuc/dns5dz959nlp8RfZUi5afuKZECzOwM4OcEv699CfBVoBu4Oc66ROKg0BAZhJkdD9xH8GM657v7/syP7/wz8Edm1hRrgSIRU2iIDCDTxbQSeAs42913Zz18PbAX+GYctYnERWMaIgNw980EF/T199hWYGy0FYnET6EhUkJmVkfwvaoDasxsNNDt7u3xViZSGgoNkdL6KsGUIz32Ag8D6ViqESkxXREuIiKhaSBcRERCU2iIiEhoCg0REQlNoSEiIqEpNEREJDSFhoiIhKbQEBGR0BQaIiIS2v8HUTsYOn47lO8AAAAASUVORK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x116344240>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.plot(X_new, y_predict, \"r-\", linewidth=2, label=\"예측\") # x범위, y범위, 스타일=빨간색 선\n",
    "plt.plot(X, y, \"b.\")\n",
    "plt.xlabel(\"$x_1$\", fontsize=18)\n",
    "plt.ylabel(\"$y$\", rotation=0, fontsize=18)\n",
    "plt.legend(loc=\"upper left\", fontsize=14)\n",
    "plt.axis([0, 2, 0, 15])\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "아래는 같은 작업을 하는 사이킷런 코드"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(array([4.21509616]), array([[2.77011339]]))"
      ]
     },
     "execution_count": 20,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from sklearn.linear_model import LinearRegression\n",
    "lin_reg = LinearRegression()\n",
    "lin_reg.fit(X, y)\n",
    "lin_reg.intercept_, lin_reg.coef_"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[4.21509616],\n",
       "       [9.75532293]])"
      ]
     },
     "execution_count": 21,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "lin_reg.predict(X_new)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 4.1.2 계산 복잡도"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- 정규방정식은 (n+1) x (n+1) 크기가 되는 X<sup>T</sup>$\\cdot$X의 역행렬을 계산(n은 특성수)<br>\n",
    "- 역행렬의 **계산 복잡도**<sup>computational complexity</sup> : O(n<sup>2.4</sup>) ~ O(n<sup>3</sup>)&nbsp;&nbsp;&nbsp;&nbsp; (구현 방법에 따라 차이가 있음)<br>\n",
    "특성 수가 두 배로 늘어나면 계산 시간이 대략 5.3 ~ 8배 증가<br>\n",
    "하지만 이 공식의 복잡도가 훈련 세트의 샘플 수에는 선형적으로 증가(즉, O(m))<br>\n",
    "- 예측 계산 복잡도는 샘플 수 와 특성 수에 선형적. 학습된 선형 회귀 모델은 예측이 매우 빠름"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---\n",
    "## 4.2 경사 하강법"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- **경사 하강법**<sup>Gradient Descent</sup>(GD) : 비용 함수를 최소화하기 위해 반복해서 파라미터를 조정해가는 최적화 알고리즘<br>\n",
    "$\\theta$를 임의의 값으로 시작해서 한번에 조금씩 비용 함수가 감소되는 방향으로 진행하여 알고리즘이 최솟값에 수렴할 때까지 점진적으로 향상 (그림 4-3)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<img src=\"./images/figure4-3.png\" width=\"50%\">\n",
    "<center>**그림 4-3 경사 하강법**</center>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- 경사 하강법에서 중요한 파라미터는 스텝의 크기로, **학습률**<sup>learning rate</sup> 하이퍼파라미터로 결정<br>\n",
    "- 학습률이 너무 작으면 알고리즘이 수렴하기 위해 반복을 많이 진행해야 하므로 시간이 오래 걸림 (그림 4-4)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<img src=\"./images/figure4-4.png\" width=\"50%\">\n",
    "<center>**그림 4-4 학습률이 너무 작을 때**</center>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- 학습률이 너무 크면 골짜기를 가로질러 반대편으로 건너뛰어 이전보다 더 높은 곳으로 올라가게 되어 알고리즘을 더 큰 값으로 발산하게 만들어 적절한 해법을 찾지 못함 (그림 4-5)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<img src=\"./images/figure4-5.png\" width=\"50%\">\n",
    "<center>**그림 4-5 학습률이 너무 클 때**</center>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "[그림 4-6]은 경사 하강법의 두 가지 문제점을 보여줌\n",
    "- 무작위 초기화 때문에 알고리즘이 왼쪽에서 시작하면 **전역 최솟값**<sup>global minimum</sup>보다 덜 좋은 **지역 최솟값**<sup>local minimum</sup>에 수렴<br>\n",
    "- 알고리즘이 오른쪽에서 시작하면 평탄한 지역을 지나기 위해 시간이 오래 걸리고 일찍 멈추게 되어 전역 최솟값에 도달하지 못함"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<img src=\"./images/figure4-6.png\" width=\"50%\">\n",
    "<br>\n",
    "<center>**그림 4-6 경사 하강법의 문제점**</center>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- 선형 회귀를 위한 MSE 비용 함수는 곡선에서 어떤 두 점을 선택해 선을 그어도 곡선을 가로지르지 않는 **볼록 함수**<sup>convex function</sup><br>이는  지역 최솟값이 없고 하나의 전역 최솟값만 있다는 뜻<br>\n",
    "- 또한 연속된 함수이고 기울기가 갑자기 변하지 않음<br>\n",
    "- 이 두 사실로부터 경사 하강법이 전역 최솟값에 가깝게 접근할 수 있다는 것을 보장"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- 비용 함수는 그릇 모양을 하고 있지만 특성들의 스케일이 매우 다르면 길쭉한 모양일 수 있음<br>\n",
    "[그림 4-7]에서 왼쪽의 경사 하강법 알고리즘이 최솟값으로 곧장 진행하고 있어 빠르게 도달하지만 오른쪽 그래프는 돌아서 나가기 때문에 시간이 오래 걸림<br>\n",
    "- 경사 하강법을 사용할 때는 반드시 모든 특성이 같은 스케일을 갖도록 만들어야 함(예를 들면 사이킷런의 StandardScaler를 사용)<br>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<img src=\"./images/figure4-7.png\" width=\"60%\">\n",
    "<center>**그림 4-7 특성 스케일에 따른 경사 하강법**</center>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- 앞의 그림은 모델 훈련이 (훈련 세트에서) 비용 함수를 최소화하는 모델 파라미터의 조합을 찾는 일을 설명함. 이를 모델의 **파라미터 공간**<sup>parameter space</sup>에서 찾는다고 말함<br>\n",
    "- 모델이 가진 파라미터가 많을수록 이 공간의 차원은 커지고 검색이 더 어려워짐"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 4.2.1 배치 경사 하강법"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- 비용 함수의 **편도함수**<sup>partial derivative</sup> : $\\theta$<sub>j</sub>가 조금 변경될 때 비용 함수가 얼마나 바뀌는지 계산"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<img src=\"./images/equation4-5.png\" width=\"50%\">\n",
    "<center>**식 4-5 비용 함수의 편도함수**</center>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "편도함수를 각각 계산하는 대신 [식 4-6]을 사용하여 한꺼번에 계산<br>\n",
    "그래디언트 벡터 $\\nabla$<sub>$\\theta$</sub>MSE($\\theta$)는 비용 함수의 (모델 파라미터마다 한 개씩인) 편도함수를 모두 담고 있음"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<img src=\"./images/equation4-6.png\" width=\"60%\">\n",
    "<center>**식 4-6 비용 함수의 그래디언트 벡터**</center>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- 이 공식은 매 경사 하강법 스텝에서 전체 훈련 세트 X에 대해 계산하기 때문에 이 알고리즘을 **배치 경사 하강법**<sup>Batch Gradient Descent</sup>이라고 함<br>\n",
    "- 이런 이유로 매우 큰 훈련 세트에서는 아주 느림. 하지만 특성 수에 민감하지는 않음"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- 내려가는 스텝의 크기를 결정하기 위해 그래디언트 벡터에 학습률$\\eta$를 곱함"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<img src=\"./images/equation4-7.png\" width=\"40%\">\n",
    "<center>**식 4-7 경사 하강법의 스텝**</center>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "아래는 알고리즘을 구현한 코드"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 94,
   "metadata": {},
   "outputs": [],
   "source": [
    "theta_path_bgd = [] # 알고리즘이 훈련 과정 동안 파라미터 공간에서 움직인 경로\n",
    "np.random.seed(42)  # 초기 난수 생성값 지정\n",
    "\n",
    "eta = 0.1 #학습률\n",
    "n_iterations = 1000\n",
    "m = 100\n",
    "theta = np.random.randn(2,1)\n",
    "\n",
    "for iteration in range(n_iterations):\n",
    "    gradients = 2/m * X_b.T.dot(X_b.dot(theta) - y)\n",
    "    theta = theta - eta * gradients\n",
    "    theta_path_bgd.append(theta)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 95,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[4.21509616],\n",
       "       [2.77011339]])"
      ]
     },
     "execution_count": 95,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "theta"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "[그림 4-8]은 세 가지 다른 학습률을 사용하여 진행한 경사 하강법의 스텝 처음 10개를 보여줌(점선은 시작점)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 96,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAmgAAAEjCAYAAACGmP17AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvNQv5yAAAIABJREFUeJzsnXmcTfUbxz9nGGPPloiERBvZQxQqKkVFSSmton6JyE4RslXSSosQKZStRSGVJUXZSpKyJMpuGMbM3Of3x8fpLnPvnbuce++59z7v1+u8Zuae7/me7zlznvt9zvN9FkNEoCiKoiiKotiHlFgPQFEURVEURXFHFTRFURRFURSboQqaoiiKoiiKzVAFTVEURVEUxWaogqYoiqIoimIzVEFTFEVRFEWxGaqgKYqiKIqi2AxV0BRFURRFUWyGKmhJgmEY5xmGMccwjKOGYRwzDOMjwzAqWXm8YRgdDMOYaxjGTsMwThqGsdUwjOcMwyhm/RUpivWEIyeGYVQ0DONlwzBWG4aRYRiGGIZRObIjVpTIEqZMND8jB57bkUiPOxEwtJJA4mMYRmEAGwBkAhgMQACMAFAYQC0ROWHF8YZhfAdgF4D5AP4CUAfAMwB+BdBERBxWX5uiWIUFctIcwAcA1gHIB6AVgCoisiNyo1aUyGGRTHwFoAeAH1x2ZYvI2kiMOZHIH+sBKFHhYQBVAdQQkd8BwDCMjQC2AXgEwAsWHX+ziOx3Oe5rwzAOAZgKoDmAZZZcjaJEhnDl5BsROefMcQ+BCpqixDPhyoTJFhH5LjJDTFx0iTMGGIYx5MwS4GWGYSwwDCPdMIy/DcMYZRiGEYFTtgXwnSlgACAifwJYCaCdVcd7KGcm5ltThRDGrSQx8SYnaiFWIk28yYQSHqqgxYbaAE4BeAfAPAC3gg/8AAANPRsbJH8AWz4f57sUwGYvn/8M4JIAxhvO8Vef+bklgPMoiivxJieKEmniVSZmGIaRYxjGQcMwZgbj/5zM6BJnbKgN4ASAG0TkIAAYhrELQAcAlQGs8Wh/NbiOnxdfg0uJnpQCcNjL54cAlAyg35CONwyjAoDhAJaov4ESAvEmJ4oSaeJNJo4CeP5M/8dAv+SBAFYbhlFHRP4NoI+kRRW0KGMYRnEAVQAMMQXsDAXP/PS2TLgOQIMAuk/3s89bNEgwJvGgjjcMoygYLJAN4P4gzqMo8SwnihIR4lEmROQnAD+5fPS1YRjfAPgeDBwYHEg/yYoqaNGnNvhwL/H4vNaZn5u8HHMcwPoA+vYVknsYfBPypCS8vx2FdbxhGAUBLACdS68Wkb8COIeiuBKPcqIokSQhZEJEfjQM4zcEpjgmNeqDFn3qAMgBsNHj87oAdvtwtL8aQFYA21If5/wZ9CXw5BIAvwQw5oCPNwwjFcBc0B/iRhHx9qWhKHkRj3KiKJEkkWTCgG+lUDmDWtCiT20AW0XkpMfn9UBztDfCNVMvADDeMIyqIvIHAJxJoHklgP4B9BvQ8YZhpACYAeAaAG00rFoJg3iUE0WJJAkhE4Zh1AdQHcCHoRyfTGii2ihjGMZ6AJtFpLPLZwboTDlWREZE4JxFwGSDJ+FMNvgsgGJgssHjZ9pdDb5JPSAi00I4/nUA3QCMBLDIYxh/6VKnEijxKCdn9nU48+s1oCw8CvoG7ReRr60es5I8xKNMGIYxA8CfAH4EcAS0Ag4AkAGgrogcsHrMiYQucUaRM8t/FyO3T0B18IH39RYUFmeyPbcE8BuA6aCV608ALU0BM4cIZkBPCfH4G878HARgtcf2kLVXpSQq8SonZ5h9Zut25u/Xzvw9LBJjVpKDOJaJzWAutSkAFgPoCeAjAFeocpY3akGzAYZhdAIwE0A5Efkn1uNRFDuicqIo7qhMJDZqQbMH9cAlQBUwRfGNyomiuKMykcCogmYP6oFr9Iqi+EblRFHcUZlIYHSJU1EURVEUxWaoBU1RFEVRFMVmxEUetDJlykjlypVjPQxFAQCsW7fugIicHcsxqEwodkJlInxEgF9+ARwO4NJLgRQ/5pPsbODnn4HUVODiiwEjj8JL27YB6elsW6iQ/7bHjrF9qVJAlSr+2+bkAFu28OfFFwMFCvhvn0xYIRNxoaBVrlwZa9dqrW3FHhiGsTPWY1CZUOyEykT4DB8O/Pgj8OmnwA03+G97111U0FatAmrX9t/2hReA3r2BV14BHnvMf9sdO4B69YBatYDVq4HChX23zc4G2rThOJYvB6680n/fyYYVMhEXCpqiKIqiJCpbtwIjRwIdO+atnM2eDbz/PvDss3krZ+vWAf37A7fcAjz6qP+2p04BHTpQ8Zo7179yBrDfL74AJk9W5SxSqIKmKIqiKDFCBOjWjQrRhAn+2/7zD9C9O9CgARUkf6SnA3feCZxzDvD223kvgz7xBBW6efOAatX8t502DXj+eVrkHn7Yf1sldFRBUxRFUZQY8e67XCKcNAkoV853OxGga1fg+HFg6lQgfx6z92OPAX/8AXz1Ff3J8hrD5MlU+tq189/2++85jubNgRdf9N9WCQ9V0BRFURQlBuzfD/TpwyXCh/Iohjd9OrBgAS1XF1+cd9vp04Gnnwauusp/2/XraZVr0YLLpv7Yuxe49VagfHkutaam+m+vhIcqaIqiKIoSA3r35lLkpEn+ozZ37wZ69ACaNeNSpD+2baO/WbNmwODB/tseOQK0b08L26xZ/q1yp05ROTtyhAEEZcr471sJH1XQFEVRFCXKLFlCK9egQUyr4QsR4MEH6bw/ZQqQL5/vtqdPA5060bI1Y4Z/hcvhAO69F9i1C/j6a6BsWf9j6N4dWLMGmDOHUZ5K5FEFTVEURVGiyMmTDAyoVo0Kmj8mTQK+/BJ4/XXgggv8tx0wgI7+H38MnHee/7ZjxgALFwITJwJNmvhvO3Ei/dSGDqXFTYkOqqApiqIoShQZMQLYvp1WNH+JY7dvp4/addcBjzziv8/PPmPOs0cfZVoNfyxdyuXPTp2A//3Pf9slS7gUe8st9GlTooeWelIURVGUKLF5MzB2LJcXr7nGd7ucHOD++7mkmVeajL17gS5dgJo1gfHj/Z//r7+YfuOiixi56a/f7duBO+5g22nT/PvJKdYTsdttGMbthmF4DRo2DKOIYRhDDMMoGKnzK4rdUJlQFHeSTSYcDlrCzjqL0Zj+eOkl4Ntvubzob7nS9CU7fpyO/v4scqdPA7ffTof/uXOBokV9t01PZ8oNwwDmzweKFfM/XsV6IqKgGYZREcA4AL6yurwGYCCAhBE8RfGHyoSiuJOMMvHmmyzPNH68/yjILVuAgQOBtm2pfPlj3DguQ770EnDJJf7b9u4NfPcdgw0uush3O4cDuOce4NdfgQ8/zNv3TYkMlvugGYbxCYC6AEr72P8YgF8A/GP1uRXFjqhMKIo7ySgTe/cC/fox31iXLr7bZWdzf9GiDBDwtwS5Zg19yW6/Pe88ajNmsB7nk0+ypJM/nnmGVrMJE/wvwyqRxXILmoi0EZHyAFZ57jMMoyGA5gDGWn1eRbErKhOK4k4yykTPnlxafOMN/0rXmDHADz8watNfZYGjR+lLVqFC3r5kmzcz+3+zZsDo0f7HOWcOE9befz9zrymxI2pRnIZhlAYF7lYRESOPwmCGYXQF0BUAKlWqFPkBKkqUUZlQFHcSVSY+/ZRLhcOHA9Wr+263fj0wbBgVr9tv991OhL5su3cD33wDlCjhu+2xY0yNUbw48MEH/rP/b9hA612jRlQQ86rfqUSWqMRkGIaRAuAdAL1F5HAgx4jIZBGpLyL1zz777MgOUFGijMqEoriTqDJx4gRTX1x8MdC3r+92mZlUjkqX5lKkP6ZMobI1bJj/HGYitIRt38725cv7bnvgAIMCSpQAPvoISEvzPwYl8kTLgnYRgDoA3nZ5IzoXwLeGYSwRkV5RGoei2AWVCUVxJyFl4plngJ07aenyp/QMHw5s3MjksaW9euaRLVuAxx+nL1v//v7P/fzzVLbGj/dfkzMrixa7ffsYOepPkVOiR1QUNBH5BYCb/dkwjB0AmonIkWiMQVHshMqEoriTiDKxfj3w4ot04G/WzHe7NWvoG/bAA8BNN/lud+oUk8sWLgy8957/sk9ff00Frn17Bgb4o2dPYPlylp5q0MB/WyV6aCUBRVEURbGYnBzg4YdpDRvrJ9whI4OpNCpWpDLnj7596Se2aBFw7rm+2/39N9CxI0tJvfOOf1+yyZOB115jxYLOnf2fX4kuqqApiqIoisW8+iqwdi0wcyZQsqTvdoMGAb/9xlxmxYv7brdgAfDyy8ATTwBt2vhul5VF5Sw9nSWd/PW5YgVLPbVunXd0pxJ9IqagiUjzPPZXjtS5FcWOqEwoijuJKhO7d1Pxat2aEZm+WL6cucYee8x/vrE9e+jsX6cO03D4o18/Kl4zZwKXXuq73a5dwG23AZUrA++/73+5VIkNWllLURRFUSykRw8ucfpLVZGeTqXrggv8K105OcDddzPKc9Ys/4EGs2dzmfTxx+mr5ouMDBY/P3WKCWn9WfiU2KFLnIqiKIpiEfPmcRs9GqhSxXe7p55idOe33wJFivhuN2oUHf6nTPGfQ23LFgYZNG7sv2C6CPDggwxgWLCA6T8Ue6IWNEVRFEWxgGPH6NNVq5b/yMnFi1nGqU8f4MorfbdbuZJpOu66y395qOPHGa1ZqBAT4hYo4LvtmDG0xI0c6T9iVIk9akFTFEVRFAsYPJgRlHPn+s7Yf/gwLViXXMLcZ744fJiKWeXK/pdKRZjGY+tW4MsvGQ3qi08+YRH2jh3zzqGmxB5V0BRFURQlTL7/nhUAHn0UuOIK3+2eeIIJYefPBwoW9N7GVLr+/ptWNH+RmC+/zCoBzz0HtGzpu92WLVT4atfOO/WGYg9UQVMURVGUMMjKYjHy8uW5dOiLjz9mMtinnwbq1fPdbvJkVgAYOxZo2NB3u5Urgd69gbZt/ZeROnKEZZzS0ugfV7hw3tekxB5V0BRFURQlDF56iQlk58wBzjrLe5v9+1ngvE4dpuDwxebNzOzfqhWVL1/88w9wxx3A+ecDU6cCKT48ynNyGNG5Ywfzotm4przigSpoiqIoihIiO3bQInbzzcwr5g0RoFs34OhRYNky3/5pGRnMm1a8uH+lKzub7Q4fBj77jAXOfTFgAPD55wxK8FduSrEfqqApiqIoSgiI0OfMMOh/5suv6/33uWQ5Zgxw2WW+++vdG/j5Z0Z5livnu93gwUxyO3UqI0Z98d57wLhxQPfuXIJV4gtV0BRFURQlBD78kBasF1/0vXT499+sFNC4sf8ly7lzgTfeYH60Vq18t5s3j4pet26s4emLtWsZaHDVVVyCVeIPzYOmKIqiKEFy+DAjMuvVY+Z+b5jRmJmZwLvv+i6ntHMn2zVoAIwY4fuc27YxH1qDBiwR5Yu9e1kpoFw5+sX5WlJV7I1a0BRFURQlSPr3p+P/p5/6VrzefpsWtokTfVcByM5mKaecHC6F+koye+IEk9GmplLp8lXyKTOT7Q4fZpTn2WcHf22KPVAFTVEURVGCYOVKpsLo1QuoW9d7mx07uL9FCy5x+mL4cPY3YwbrcnrDDDLYvJkO/76WU02fuNWrufxau3ZQl6XYDFXQFEVRFCVATp+mw32lSr4rATgcrItpGEwK6ysac/lyLml26cIksr544w06/A8f7t8/7ZVXeL7Bg4Hbbw/4khSbogqaoiiKogTIuHHAL78ACxcCRYt6b/Pqq8BXXwFvvcVSTd44cIBLmxdeSMXKF2vW0Nftxhv9509bupQWu7ZtgWHDAr4cxcaogqYoiqIoAbBtG/Dss0CHDr4Ljf/2G9CvHxWqBx7w3kaE+w4cABYt8q3o7d/Pc1WowAoEvixxf/zBpLU1avhvp8QXqqApiqIoSh6YfmBpab7TVuTkcLmyYEHgzTd950V75RVa4CZMYGUBX33ddReVtFWrgFKlvLdLT6fVTARYsMB/3U4lvlAFTVEURVHyYPp0VgF47TXg3HO9txk/HvjuO2DmTN9tNmwA+vQB2rQBevTwfb5nngGWLOEyqa9ABIeDudC2bGHwgK8gAyU+iZgh1DCM2w3DKOfyd2HDMAYYhvG9YRibDMP4zjCM5pE6v6LYDZUJRXEnXmTiwAHgySeBRo1YT9MbmzYBQ4dySfLOO723OXEC6NgRKF0amDLFt4Vt0SIGDzzwAPDgg77HNXw4E9c+/zxw3XXBXZNifyKioBmGURHAOACuxSoqAcgEcJWI1ATQBcAUwzA0S4uS8KhMKIo78SQTTz3FOpqTJ3v37zp9mkubJUrQwuZL8erRgz5q773nOz/ZH38A99zDpU9/wQNz5zIYoEsXBhEoiYflS5yGYXwCoC6A0q6fi8ivAH51+XurYRjrANQG8KXV41AUu6AyoSjuxJNMfPUVqwD07w/UrOm9zciRwE8/AR9/7FvxmjWLKTAGDgRatvTe5uRJJpkFmIy2UCHv7TZupGJ2xRVMweFLIVTiG8staCLSRkTKA1jlr51hGEUA1Afwm9VjUBQ7oTKhKO7Ei0ycOsUlzapVuXzpjbVrqaDdcw/LK3njjz/YT+PG9C3zxf/+B6xfTwtb1are2xw4ALRrx2CAjz5iQIKSmMQkSMAwjLMAzAUwXUR2+mjTFUBXAKjkK22yoiQIKhOK4o4dZGLUKKbWWLzYuzXr1Ck66Zcr5zuyMyuL0ZiGweABX3Ux33rLmWS2TRvffd1xB2ttfvON70AEJTGIerYUwzBqAPgWwKciMsRXOxGZLCL1RaT+2VpMTElgVCYUxR07yMSWLcDo0VSufGXvHzKE7d5+GyhZ0nebNWuYdsNX0tp162g9u+46/xa2J5/kkuvkyUDDhsFcjRKPRNWCZhjG9QBeBPCgiPg1bStKMqAyoSju2EEmHA4uSRYtCrz4ovc2K1cyevKRR4DWrb23+fJLYMwY4OGHfZdeOnSIkZ9ly9LC5qvw+ltvMWjgySdptVMSn6gpaIZhnAPgZQDX+jJXK0oyoTKhKO7YRSbeeQf49lsqRWXL5t5/4gSd9CtXZuknb/z7LxWpiy9mQlpvOBxA587Anj3AihVAmTLe261cySLo111HhU9JDqJpQbsXwASdiBTlP1QmFMWdmMvEP/8wrcZVV/ku1dSvHx3/v/oKKFYs936Hgwrc4cPAF18AhQt772fECOCzz5iaw9eS5e7djOw8/3zggw+A/JpePmmI5r+6OoBrDMPo7vH5ayLyWhTHoSh2QWVCUdyJuUz06gVkZACTJnlPX7FkCYuh9+oFXH219z4mTGBm/1df9Z2aY/Fi+pvdcw9LSHnj5Eng1ls5nmXLfPu5KYlJxBQ0EWnu8ffDkTqXosQDKhOK4o7dZOLzz4H33weefhq46KLc+48epVWtRg2m1vDGunXMmXbLLUB3TzXzDDt3Mvjgsst85zETAR56CPjxR2D+fOCSS0K/LiU+UWOpoiiKkvRkZNDPq3p1Klje6NWL/mKrV3tPu5GezjJP55zDyE5vildmJoMCsrNZDcDX8ue4cQwaGDECuPnm0K9LiV9UQVMURVGSnuHDgT//pF+Zt+SvCxeyfubAgb79xR57jL5py5cDpUp5b/PEE0xu+/HHwIUXem/z6adUEm+/nedTkpOo50FTFEVRFDuxcSMwfjxw//1A8+a59x88yFQZtWr5rigwfTq3oUOBZs28t5k6lb5t/fr5rjqwdSuXPy+/3H9BdSXxUQuaoiiKkrTk5ABdu9IB31fKjMceY76yxYuBtLTc+7dto79Zs2bAoEHe+9iwgcEALVpw2dIbR44Abduy2sC8eUCRIqFdk5IYqIKmKIqiJC1vvMFM/9OnA6VL597/wQfcRoygVcuT06fpd1agADBjhvc0GEeOMFVGqVIMQvDWJieHlrM//gCWLmVaDSW5UQVNSUhWr6YfSPPmLFCsKMmOykRu9uwBBgwArrkGuPvu3Pv37WPgQIMGXJb0xoABjLScNw8477zc+82caDt3Al9/zQACbwwcyJxor7/OHGxK5LG7TKiCpiQcq1fzC/f0ab7VLl1qT+FTlGihMuGdJ55gAXJvqS5EWMYpIwOYNs271euzz4AXXuASaLt23s8xdiywYAGLqTdp4r3NzJls162b75xoirXEg0xokICScCxfTqHLyeHP5ctjPSJFiS0qE7lZuJBpLoYMAapVy71/2jQqVqNGec+JtncvLWM1a/r2XVu6lD5pd94JPP649zZr1wIPPkj/tZdeCv16lOCIB5lQBU1JOJo35xtRvnz86S0qS1GSCZUJd44fp9Xr0kuBPn1y79+9G+jRg0uNTzyRe7/DwTqbx4/TP81bTrS//gI6daJy9+ab3qMx9+1jpYCyZYE5c/i/UaJDPMiELnEqCUfjxnxztbNvgaJEE5UJd4YMoRK2cmVupUiE1QJycpjmIsWLGWPsWJZ8evNNFkP35PRp5jA7eZJWuqJFc7fJzGTgwMGDwKpV3ouyK5EjHmRCFTQlIWnc2FqBc3UmVZR4RGWCrFsHTJxIXy9vPmFvvEHl6403gKpVc+//7jtg8GAqYA8+6P0cffqw3ezZ3pdHRWjBW7WKFrjatcO7JiU07C4TqqApSh54OpMCxTQ7kZLUxKtMZGcz4WzZssBzz+Xev307latWrZgbzZOjR7lsWbEiMHmy92XL998HXn4ZePJJlnTyxquvshTUwIHAHXeEd02KPYiETKiCpih54OlMChQvFuMhKUpMiVeZePll4KefaLUqUcJ9X04OcN99TBLrrY6mGdW5ezfw7be5jweAn39mgfOmTYHRo72P4auvgJ49WV/z2WctuSzFBkRCJjRIQFHywNOZFDiWHuMhKUpMiUeZ2LWLvmc33sjlSU8mTABWrODyZ8WKufdPmULFbvhw78tix44Bt90GFCsGfPghFT1P/vyT565eHXjvPe/+bUp8EgmZUAuaouSBpzNpkybpJ2I9JkWJJfEmE6bPlwiXFz2tY7/8wnQY7doB99yT+/gtW5gmo2VL7wlrzcCC7duBZcuA8uVztzl+nP3n5ADz5wPFi1tzbYo9iIRMqIKmKAFgtTOposQ78SQTc+cCixaxIHrlyu77srKYz6xoURYy91TeTp1iHrPChVkOKl++3P2/8ALPMX689yoAZjWBn39mctsLL7Ts0hQbYbVMqIKmJAV2L+mhKNEmWWTi6FHmNKtd23tOs9GjmSx29mzvZZj69gU2bqSCd+65ufd/8w2tau3bMzDAGyNGAB99BDz/PAMQFHtiN5lQBU2JOtEWAs/omgkTmHvILkKoKCoTkWPgQOCff7is6Fmuaf16+pR16uQ94nLBAgYW9OwJtGmTe//evUDHjsAFFwDvvOM9qvPjj4Gnn+bSaa9e1lxTMqAyAUBEIrIBuB1AOY/PbgTwE4BfAMwDUDqQvurVqydKYrBqlUihQiL58vHnqlWRP+eoUTwfIJKSIpI/f2DnX7WKx3q2AbBWVCYUi1CZiJxMrFolYhgiPXrkvpZTp0Rq1hQpV07k4MHc+3fvFilVSqROHbb15PRpkaZNRQoXFtm82fv92rhRpEgRkQYNRE6e9N5GyU0yy4TrFpEYEsMwKgIYB6Ccy2dVAQwHcJ2IXAJgFoC3InF+xb7Eov6Za3RNvnz0B8nr/Obb1JAh/Ll6dXhjUJlQfKEyERmZyMpiLrMKFbjE6MmwYcCmTcBbbwGlSrnvy8kBOndmtv9Zs4C0tNzH9+/PqM8332TJKE8OHmRQQPHitKIVLBjKVSQnySoTnliuoBmG8QmAHwB4rtY/CWCsiBwAABGZBaCsYRjqLplExKL+mRld8+yzwCuv8Ms2r/Nb+QWhMqH4I95lYseO4M8fDZl4/nlg82ZeXzGPjFTffQeMGcNKAN6WLkeNAr7+mhGf1avn3j97NgMD/vc/4K67cu/PzmYC2j176HtWoUKwo09u4l0mLCNcE5yvDcByALVd/t4MoJRHm+EAHs6rL13OSSx8mYTtdH5/JnaEvpyjMqF4JR5l4osvRHr35lKQ3WTi999FChYUufXW3Ndx4oRI9eoilSqJHD2ae/+333KJ6+67RRyO3Pu3bBEpWlSkUSORzEzv96pHDy6VTZni+34q/olHmbBinnDdohkkcI6IHPL4bB+A87w1NgyjK4CuAFCpUqUID02JJrEOzw/k/FEqpKsyoQCIL5lYuhQ4fJhO9ceOWT6UsGVCBOjenYliJ07MfczAgcBvv/E6PHORHT5Mi1iVKsBrr+V2+j9+nMloCxWiFc2z0DrAYIGJExlYcN99AVyx4pV4kolIzRPRVNB8rcCL1w9FJgOYDAD169f32kZRIhnpE4UvCJUJxXIiJRMiwIED9NnauZOfFS1KpcVCwpaJ998HvvzSe0WAr74CXnqJS5MtW3r2xTJNe/eyiLmn8ibCOp5btwJffOG92sCqVSzCfu21wLhxeV+sEh3idZ6IpoJ20DCMkiJy2OWzcgB2RXEMSgLhGRa9dGnoghKj/DcqE4qlREomChZkJn7TCbpAAaffTa9ewIsvWnUF4cnEoUO0XDVsCDz6qPu+9HTg/vuBatW818mcNIn+YuPGAQ0a5N7/8ssMGBg1ivfYk7/+onWtUiWWhPJM6aHEhnieJ6L5CK0A0BLAXJfPmgN4MIpjUBIIbw6aoQiNqwDnz88v8XvvjYoAqkwolmK1TGRm8m+Hgz9TUug8ffIkc4fVqUOLlIWEJRN9+1JJ+/LL3Bn/e/d2FjovUsR93+bNVDRbt/aebHbVKh7ftq33Uk8nTwK33gqcOEEFwDMqVIkd8TxPRLNU6wsA+huGURIADMO4C8C/IrItimNQEggrIn1WrwaeeYYTUU4Of06aFJmQaS+oTCiWEq5MrF4NPPcclzFPnqRiZipnAH9v0IBJPDdtokJkcYRiyDJx/Djw9ttUsC6/3H3fZ58xHUafPkCTJu77MjJYyumss4CpU3MXMP/nHxY4P/987/vNpc+1a1kA3VvKDSV2WCUT06Y5Fb1ozRNRs6CJyI+GYYwCsMwwjEJgtM7D0Tq/Yg1Wm3jD6S9cB01XK4HrJCQS3ptWoKhMJAaJIhOrV9MvKzOTMuCNG26g4taLC4npAAAgAElEQVSzJ5cKZ89miSNPpSVUwpGJnTtZZ/Ppp90/P3yYvmWXXsrcZ548+SRrZC5enLvUU3Y2LYWHDjE1R4kSuY9//nlgxgymZ2jXLpCRJj6JJBOuVjMzPxpjdKMwT4QbBhqNTVMK2AOrszvHIlu0K56Zoxs2FElNZebxtDTf44EF4dPhbioT9iCRZOK++8xph5spG+ZmGPx59tkiL7/sTDGRnm4PmQDqyaef5r6uzp2ZCmTt2tz75szhNfXt6/2e9OvH/e++633/Z5/xu6NDB+8pOZKRRJIJ1zkiXz6Rbt24RWueiOYSpxLnWJ2ULxbZol1xNX2npTFppRlWLxojqQRAIsjEpk2UhXffdf88JYV1Jl1l4r77gN9/ZxQkwISeF1wQ+TEGQunStPC58tFHXHYcPBioV899386dtKw1aEDrlyfz5jGZ7SOPAF265N7/229cGr3sMt47b3U4k5FEkAkTz+XRe+/lFq15QhU0JWCszu4ci2zRrrhmjl66lKVZcnIodDk50VcYlfgjnmVi3z4qXJdfzqz5KSnOvF6tWtEpfv585ySUksKs+kWKUOm56CLg8ceBiy+O3BiDoXJl97///ZcpL+rWZe4zV7KzgbvvppzPmpU7n9m2bVTK6tenv50nR48yYCA1lffIM+ggmYlnmfDEc45o3JjzQrTmCQ0EVgLG6qR8UUoGm+cYXM9boIAzHDvaCqMSf8SjTGRkMJXEc885ozSLFGEE4pVX8rxvvslcX1deSef37GwqI6mpjNzctIk/P/+cypxVPmhWIWeS1R49ytxnqanu+4cNA1auBGbOBKpWdd+XkUG/uvz5gTlzctfQzMmhcrd9O7BkSW7FMNmJR5nI6/yu5zQVxqjME+GukUZjU38bxZNIlQEJpF/YwN9GZULxJK9nNyeHpYdKl5b//MqKFuXPSy4RGTZMpGZN/t2ggcjy5c5+u3YVufxy7qtWTWTWLPZnYjeZeO89jnXs2Nz34auv6D9033259zkcIvfcw/2ff+79Pvbvz75ffdX7fsU+xPs8EXPlK5BNJyPFlVgHF9htMlKUvGRi2TKRiy6S/xSzQoX4s2xZkUGDRFq04N9Vq4p88IHT4X3DBpE2bbivXDmR118XOX069/ntJBN//SVSooRIkyYi2dnu49y/X+Tcc0Vq1GBwgyevv85rHTbM+32eOZP7u3bVoAC7kwjzhM0M04riH8+8ZbEILlAUO+FPJn79lUuQLVvy99RUp4Nzjx5cnhk5Eti4kQlnt2wB7rgD+PNPoHNnoHZtLgU+9xyX9Lp1y71caCfkTLmmzEw67rsmqxUBHniA5aref59lqlz5/nvgiScYaDB4cO6+f/yRgURNm7KqgAYF2JdEmSfUB02JGzzzlplOzeorpiQrvmTi8suBrl2ZcFaEykRqKieqzp3pc/bGG2w/cCATzp51FpOyPvssMHkyfbD69eO+kiV9j+Gff6J3vXnx1lv0i3v5ZeDCC933vfIKsHAhnf7r1HHfd+AAi7+fey4DILwlq23XDihThn5p3oqkK/YgkeYJVdCUuMEMtzaF7tpr+ZYUqNNojOptKkrE8JSJFi2omHTowISyAFC4MB3fr76aKSGmTKHz/P33MxLxl19oPfrmG9bUPHWKmfGHDKHC4ot//wXGjgVeey0ql5onmZlMOtuyZe46nOvXs4rATTfRcuhKTg5w1128npUrc5dpOn2aQQMHDwIrVuROZqvYi0SaJ1RBU+IGM3omM5NLF+3bByd0wRbMtZOgKoo3TJk4dYpWsu+/57MNOBWzatVYY/KDD1ij8sYbWSz8+PHclQM6dqQFzdP6ZLJ6NbBoEXOIffwxz3v33cD06VG5XL/s2MF78M477hawEyeYr6x0aSqnnkuTw4bxvrz5Zu5caSLM+bZyJZdF69aN+GUoYZJQ80S4TmzR2NQhOnEJNspm0iRmBU9JCc7x0zMj9KhReY/Ll4MpbOQQrSQewcrEU0+JW8b/1FT+LF9epHdvZ/RlvXoMFhARycoSue029+Mee8z/eT77jLJntm/VSuTXX7nPDjIB1JO338497gceYFSmee2uLFrEa3ngAe/X/Oqr3N+/v/97o0SWZJ0n1IKmxIxQ3lYOHuT04HD4r4Pm+VYTbO4ab9mr1YqmRJpgZOL33+nU/umnufd17w5s3co6kVWq0Ppz3nlMSLt6NQs/b91Ka5JhsJLG3Xd7P8+hQ+xn/HjmQwNooWreHKhRw5LLtoSzzuKyrSuzZtGiNmgQl39dMQMh6tShf5ony5fz/rZpA4wYEbFhK3mQzPOEKmhKzAjl4Q5EgHwJdDDJDqOajFBRzhCITBw6BAwdSif/nBwqWOKS7b9pU+4rWZI+Zd27MwKxeXP2CTC56scfA2XLUmnzJhOHDwMvvMDoTnM5dMUKKml2lIlq1dyXL//4g4ESTZrQB8mVkye59AXQ6b9QIff9O3YAt9/OPmfMcI8GVaJLUs8T4ZrgorHpck5kiVQyv0DOG0qemrzGG6yZOtjzwAbLOSoTkcWOMpGZKfL88yJFikiufGYNG4rUrStSoIBIwYIs8n34MI/74Qcml3Utej5ihO8xHD4sMnSoSPHibN+hg8imTc7xxYNMnD7Ne3LWWSI7duS+xgcf5LUtXJh7X3q6SK1aPHbrVt/3Kdmwo0zkdVy8zxMxV74C2XQyihzRTubn+TBPmkRflkmTrOsz0tdkt8lIsRa7yYTDITJnDhOsmkpWwYL8Wbu2yCOPiJQs6cyOv2sX++rVy5mAtkQJ+qb588k5ckTkmWeomAD0UduwIbBrsJtM9OvHa5g9O/dY33qL+wYPzr3P4RBp3573yVclgWTEbjJhRZ/xME8E1gh4A4AAONfLvhoATgN4KdzB+Np0MoocVr1FBIKnQEyaFL6A+BKySL7t2W0yUqzFTjLx1lu0jJmKWYEC/Fmhgki3biKVKvHv6693KlPz5jnHbzrAHz3qWyaOHhV59lkqcYDILbeI/PRTcNdhJ5n44gv5L9u/J+vWiaSliVx3Xe4qAyIiw4fz2HHjgrv+RMdOMpFM80SgClqXMwraLV72fQrgAICS4Q7G16aTUeSI5puRp5C3auVd6IMRmmh+cZjYaTJSrMcuMuG6paTIf0uaDzzgjMysU0fkyy/Z14EDIn36uEdapqT4loljx0RGjhQpVYpt27YV+fHH0K7DLjLxzz8i55zD2qInTriP8eBBkcqVRc47jyWfPJk3j/ehc2ct4+SJXWQi2eaJQIMEvjvzsyGAeeaHhmG0AXADgMdE5HAILnBKjAnWKTIcPB0q27cHvv3W3cHSm+Mm4Ht86syvWE0sZeLGG53PvEm+fHSQBph37J13gHLlWJKpRw+WYxo5kklj09NZqmjpUt/O/MePM2px/HhGu910E53oPXOAxSNdujAJ75dfMg+cicMB3HMPsGcPv3PKlHE/bvNmRnTWr88qClrGyR2dJ2JEoJocgIMAlrj8nQpgK4BNAPKFqyn629RaYE9CMQ978wNw/dvzTadbt7zf3KLtvAqbWAsUexHqc7hqFZcYe/USKVbMaf0y85m5OvgDIo0b0wctJYVtTAtYu3b+nfmPHxcZM0akTBm2v+EGkTVrrLl2O8hExYr1BBB57bXc4zOXLr3tO3iQReLLlWOhdcU6wpEJnSeCCBIA8AmAIwCMM3/3AZc9rwnqhEA7AN8D2AxgA4BOeR2jk5H9iJTJ27Pfbt2ib5rOC6snI5WJ+CdUeXA4RBYs4LKbqYClpfFn3boinTo5/c4Mg4rZI4/wd7N95cr+z3fiBH2qzj6b7Vu3Flm92prrFmGyWjvIhGHUk1tvzb08uXgx75e3pcusLJFrr+U9jnZ0YqITyWXRZJkngsmD9h2AGwHUMAzjEIAhAOaJyFL/hzkxDONSAMMAtBKRfw3DOA/AcsMwNonI5iDGosSYSCXoczWlly4N/PSTMwdRXJmmA0RlIjEIRR5++ok5ytas4d+pqUBWFnOTXX018MknbHPPPcAttwBbtgB79zLprJzJe1agAPN0eTtXRgbzoY0ZwzqT113HskZWLU+tXAmMGwcsWGBNfyahykT+/CyW7ro8uXMn62xedhkwaVLupcunngKWLOGysSaitpZIJnFNlnkiGAVt9ZmfDQFcBSANQO8gz9cIwFIR+RcARGS3YRgrwEhQnYzihNWrgV27+IUIWC8QphCbPgb587N48733JuSXqMpEAlC6NJPEiuQtD3/9xcLdH3zAv1NS6CNVqBCTo65YAbz3HtCqFZWr2rWpDA0bBmzYwGPy5wcefJA+V54ycfIk/ahGjwb27aMcDRsGXHll+NeZkwPMn0//tdWrWVh88GDW77SQkGSienX3QueZmbyfWVnA3LnuPmkA8O67wIQJ9OPzrECghE8wMhEKSTFPBGpqA1AMQA6Ar8/8HB2suQ5AbQC7AdQ/8/dNAH4FcJa/43Q5xz64mpYLFKBpOS/TdShr/6NGOSPX/EWixQJYuJyjMhH/mDKRksIISl+5mtLTWdPRNcoyXz5ut93mjMysXZupIkRENm4Uuflmfl6smHNp05tMnDwpMnEia3ACzIH29dfWXGNGhsjrrzv94apWFXnlFfq1idhTJrp141g//jj39axeze+vli25zKlYS6Ay4do+FP+wRJ8nAragiUi6YRi/gNazfQBGhqAMrjcMoz+ApYZhnAQDDVqLyFHPtoZhdAXQFQAqVaoU7KmUIPGsSeYLV7M1AFSq5L99KHXUAL59ORz83eHg39Em0HsSDioT9iVYmXA4uIR28KD7/pwc4O23gb59GWHoSt26tJx99BFlafp0Lsnt3MmlzRkzgOLFgVGjgKJFae0B3GUiM5NLe889xyjFq64CZs60xmJx4ADw6quM+jxwALj4YqBTJ+DRR1lSKhJYIRPTpnF5t29fLg+7smcPcOutQMWKwIcfOlcClLyxSiY8+wxljgCSYJ4IRpsD8DYYGHBfKNoggGoAVgKYBaAjgMkADgG42t9xai2ILME4cwbr+Blq/plYvxn5u05Yay1QmbAhVsnE55/T2uRqMXONyDQMJogdN44WsH37RP73P0ZnFiwo0rcvowxFcsvE8OGMSqxYkZ81bSqydKk1Oby2bRPp3t1ZSuqmm0RefZVjsrtMbNjA8TVvnts6dvKkSIMGLJdlRrwqgRGpeSKcHGWJPk+kBKrIGYaRCqA5gLUApoaoD74H4FURuVNEPhCRrgC6ARgeYn+KBXhz5vSF6Zz57LOBvemY+Wfy5ePP0qX5pr96dd7H5c/Pt6/8+aPv9BnMPQkTlQkbEq5MbN5MS9b117Not2mlOfts+uWY3HknMGsW85c9+ihwwQXA668DDzwA/P47fdBMvypTJgDKxauv8phKlZj365tvWNA8nBxea9YAHTrQn+vtt2nN+/lnYOFCWv+ysuwtE0eOALfdxkLxs2a5W8dEWDz9hx9oqbzssoiNPyGJ1DzhOUeYec50ngguSKAPgCoA7j6jHYZCdQAfenz2E4DiIfanWECwSfwaNw7clOsZbdOzZ+CmbHOiiUXSyCgmNlSZsCGhysS+fXTcnz6dCoEZAFC0KPv45hv+Xbs2MGQIla/rrmNSWYBLPa+9RgXJk+xs53JOTg6TrU6bxuPDkRGHA1i0iBGZK1YAJUoA/fsDjz8OlC8f+j0Jg5Bl4r77uDy8fDlwzjnu+158kf+XYcO4xKkER6TmCc8kuEBwS54JPU/4M68BKAWgE4DnAGQDGB+OuQ7A+2D4dP4zf5cBS0U94u84Xc6JPNFI4heMKTsWpTk88XVPYO1yjsqETQlGJk6cEBk61JmzzFxySU0VadNG5Pzz+dk117AeZFaWyNtvOwuT+1uiOX2aNTlLlnRvO3JkeNd38qTI5MkiNWqwz/PPF5kwgSWgfGFnmahQgYlqJ0zIPe7PP+c9a99eJCcnhJuliIjOE96IpEzkJSidQJ+zfwCMQ5gVA8BI0AlgqPRGAKsAdMjruEScjKKd1dgORNLXLZpYPBmpTJwhHmUiJ0dkyhRnNn/XCgAtW4rUqsXfa9WikpCTIzJ3rshFF/HzSy6hUuftOc/KYt+mD9tFF7FtSkp4MnHwoMiIEaxZadbznDkzvGhGO8gEUE86dsztg/fbb/Tzq1mTkbTxRDzKRLjoPBGggmaXLdEmIzs/VJEmmC8cu345WTkZhbqpTMSeZctEqlfPrZjVrSvSpAl/P+88kalTRbKz6cDfoAE/v/hikY8+ojLh+ZxnZfEYM6VF3boiCxd6bxsMf/wh8vjjIoULs9/rr7cuqMAOMlGwYL1cCtjRo7zXpUvz+uOJeJQJq9B5gpsGGMeASGZYjgRWhhEH47+mJA/xJBO//soUGGvX8m+zmHmFCnTyX7aMqTHGjKEf1y+/sID5l18C553HrPX33ON0YDdlIieHCWqHDwe2baOf2rx5QNu24fnXrFtH/7LZs+kTd9ddTJRbs2b498JOVK9OXz+TnBzg7ruB337jva9SJXZjC4V4kglA54lIoAqaH6zOb2L2V7p01JxtwyacHDXxeF7FP8ksE/v3AwMHMrpRXMKkChfmvfjmG+DvvxkIM2gQ84Z16ULFqHRp4IUXWNqpYEH3fnNyWFVg+HBg61agVi3mRbvlFnfFLBiZEAE++4yK2fLlVBh792YetYoVLb81tiA11f3voUMZ/PDyy0CLFpE5ZyRyYMWTTJjoPBEZVEHzgdX/eM/+Jkxg8r5IJUG16osjVm9x8fb2mAwkq0ycOkXl6tln+bsn2dnAF1/QMjViBBWFAQNoKStYkIpC795UklxxOJgodfhw1tm87DJgzhxGGLqm4jAJRCYyM1mrc/x4pseoWJG/P/xw7vMnMh98wOS+Dz0EPPZYZM4RCeUgXmTCE50nIoMqaD6w+h/v2d/Bg/wSjwRWfnFEMbTeFudVfJNsMiFCZadnT1rPAC5LZmc702cAzK4/eTKX0EaPpsUmJ4eKwaBBLH7uisNBRWzYMC5/XnopFbX27b0rZib+ZOLIERYDf+klFlSvVYspODp2ZNtk4qefWFvzyiuZKy5S6RcioRzYXSZ8ofNEZFAFzQdW/+Oj+SAF8sUR6JuTZ46aQATYireyUM6rRJZkkgmHA3jkEVqhAKdiduml/Pnzz8yz1bcvk59OnAiMHQscO0b/smHDgMqV3ftetoy/z5rFRLYXX8zfb7/dv2Jm4k0mdu2ileXNN4Hjx4Frr2UR8HBzo8Ur//4LtGvH5cG5cyOrnEbi+bWzTOg8EQPCjTKIxhZOxFo4ER5WR4dEK9okr+gfq6ODXK8rGSKPYIOINZWJ4M8TiEykpDhLx7iWZqpSxRmBWaEC019kZLD8kZmuom1bZ/kg1+taudI9P1qlSkxrkZ0d+vX89JPI3Xc7i63ffbfIjz+G3l+42EEm6tatJ82asRzV2rXBjT/U5zASz6/dZELnidCwQiZirnwFsoU6GSXDQ+BLmP0JuZXJ/TzvcbdusU8cGGnsMBmpTPgmFJkYPNipQLlupUqJXHkla2YWLy7y3HPMpTVjhjM/WbNmVMJcz2MqewUKMMWDa+3NESNCuy6HQ2TxYpFrr2VfRYuK9OolsnNnaP1ZiR1kokwZJqqdOTO4satM6DwRCayQiYRe4kxkB8LVq+lj8s47vD5PHwJ/YcpWmtE97zGQ2D4B8Y7KhPsxp0/TZ2z8ePfPCxQAGjZkiorvv2f046BBrOPYtCmwYQNw+eXAp5+y3qbrcuJXX9FZ3+Fg/1lZDBzIyQHS0lgvMxiysrgUOn48sHEjyy+NHs0l2BIlgr9PicqBA0C/fkCnTsEdpzLh/VidJ2xAuBpeNDa1FrhjXpdhiNtSTDBvIZMmibRqxZ/BnNfzbcvbPbZr4kCrgA2sBSoT7gQrEw6HyOzZIuXKiZt1y1zONCsD3HmnyPbttJA1a8bPqlallSYnx/1ZdzhEFi1yVgkwk9d+801oMnH0qMj48SIVK8p/VQfeeUfk1Clr7pmV2EEmihevF9KyscqEb3SeCB0rZCLmylcgW6z8bSJJOONyNT2bE0swXyyhfCH5O8au9zhS2GEyUplwJxiZWLNGpHZt90nLc2mzTh2R77+nT1nbtvysXDmR114Tycx0jtd1KfOSS9iucmWRgQNFnn02tGv56y+Rp57ikiog0ry5yCef2LuGpB1kom7dxJKJcMek80RsUQUtTgn3jc31+LQ0rucH00covgV2KEprF+wwGalM+D7el0zs2CHSrp1zwjKDASpXFilTxv3zPn1E7r3X6Xs2cqTI8ePu/Y0c6R5QUKKEyJtvsrh5KGzcKNKlC61uKSkiHTuK/PBDaH1FgxMn6IvXqpXKhNVYYdXTeSK2WCETCe2DZlfC9XkIN7TYn2+Br9DnRM83o8SWSMrE0aNMezFxIvs3KVECqFQJWL8eKFOGfmLZ2fQnmzCBqTX69KHP2E8/0f+rcWOqY0uW0C/MzIWWmgrMnw9cdVVw1y1Cn7Vx44DPP2dVgm7dgF697FmaSARYsQKYOpW529LTeQ8Va7HCL07niQQgXA0vGlsivRmJ2MPnIVA/gbyOSUag1gLLiYRMnD4t8tJLjHZ0Xb5MSxOpV49WqmLFaAnbu1fk/vu5VJkvn8jDD4vs3u0+roIFRSZOFGnalP1UrMilyOHDgx9vVhb92OrWZV9lyzK68+DB8K87Evzxh8gzzzgjV4sUobVv2TIuvapMWIsd5ghzHDpPhIYVMhFz5SuQLZEEz8TXgx/LB1vN04Ghk1FksEomHA6R+fNFzjtP3JYtDYO+ZYUKieTPL/L44yK7dom88IIzFcYdd4hs3ersy9OPBxA591zmPwvFWT89XWTCBJHzz2dfNWpwWfTkyeD7ijTHjjEo4eqr5T8fppYtRaZO5XW4ojJhPaGkxogGOk8EhipoCYQd3pjsMIZ4QCej6BDK8/jjj86Esq5+Zpdc4ozMvOMOkS1bqHyYStx113n393r1VXc/s169QlOm/v5bZMAA+qkBtMLNn28/x//sbJEvvxTp3FmkcGGO9cILad3zl29NZSI62OE72g5jiAeskAn1QbOQcEpXePocTJsW/fIVCV82Q4k60ZKJPXvotzV7Nv82DKpUFSvy+F9+oX/YmDGsVXnbbSxQ3qABMGUKaxK6smIF8PTTLM9UujTQqBH90YL1qdmyhfnL3nuP+cxuu439NGoUXD+R5rff6Fc2bRrw11/AWWcBnTsDXbrwXidj2ahIofOEEjDhanjR2OLhzcjqKDTTFyYSbyixNpHHO1BrQUBEQybS00X69eOSpesyZOnSItWq8feLLxZZsEBk6VKRhg352UUXicydy+VQ81yjRjHfk5mpv2xZLn9mZAQ3bodD5OuvRW66if0ULCjSvbvItm3B9RNpDh0Sef11kUaN5D9r4w03iMyaFfw1q0wEhs4TyYMVMqEWNIuwMgpt1y4WP86rr1DexFavprXAjLJxzSqtKFYSSZlYtgzYtAl46ikWKDcpUIDWn/37+fvkyUCtWsDQocAXX9Ci9vbbwL33MkoToEy0aMHs/wCjO8ePB7p3Z1RloOTkAB99xIjMH35gZOgzzwCPPgqcfXbg/USS7Gxg8WJayxYs4DVfeikLvd99N3DuubEeYWKj84QSFOFqeMFuAMoCmAFgC4DNAKbmdUwyvBkF21cw53N9E1IHz/CBxdYClYng+ipQQKR8eXHzM0tJYQUA87P8+UXefZf+ZqZF7fnnnf5jpiy89ZZI9erO4wyD0YrBcPy4yCuvOCMcq1WjZerEidCv12o2bhTp3dtZOaF0aQZJrF3rtCKGg8pEYOg8kTxYIRPRVs4KAtgA4MEzf+cDcH9ex8VC8EIx71ppEs6rr0AFyFNAJ00K7QtCzd1OrJyMVCYCZ/p0kUqV3BUz04ndzOhvlrUxDP5dpIjIkCEiR464jyktzdlPkSJU6FJSgpOJRYtEWrRwZvxv1IjLpqGUG4oE//7LiNE6dZxKa7t2Ih9/7KyGYBUqE5E9JtS+dJ6IHfGooD0F4LVgj4u24MUySiXQBzzQMXoT0GCFSKN23LF4MlKZyIOFC91LM5lK2HnniZQsyd/bt6c1zNUXrUMHkX373Ptat869VqZhiDz9dHAysXWryC23iJui+MYbEbn0oMnMFPnoIypi5r2oW5f54P79N3LnVZnQeULnCXeskIlo+6DdDuCxKJ8zaKzI4uy57h+IH0Aw6/6BRtJ4y+zcuHFw12PF/VB8ojLR3HtfGRnA//7HKEtXSpYE0tKA3buBpk1ZJWDNGqB3b46vTh1g0CCgfXvnMevX0x9s/nygWDH6nzkc7Kd168BkYtUq+pfNnw/ky+eMFDUM4NCh4O6FlYgA69bRr+z994GDB4Fy5YCePRmFedllsRtbiKhMNNd5QkHULWjpAC4AMBnAegDfAbjdR9uuANYCWFupUqU8tVWrzcZWRdoEYy6O1Lp/uPdG34zcgbXWApUJj/5ycpijzLSOuW5mHrEaNUTmzGGeMtOv6uab6WvlyoYNIrfeyv1nnSUybBiXOwO9N9nZtEg1bsw+SpUSGTyYy5uxlok9e0TGjHEWaU9Lo8/dJ5+wUkE0UZnQeULnCXeskIloK2gZAJYBuPrM35UB/Aqgqb/j8jJdR+LB8PawBvoAuwpQSgqdhk1/mWD8AKx6wK34UlLfAicWT0YqEy4ysWyZMz2G63KmuZUqRQf86dNFLrhA/kv6umKF+/k2beKyJ0AfsaefFjl8OPBxZ2TwPBdeyD6qVBF5+WX3gumxkImMDJH33xe5/nrn/WvcmEushw5FbxyeqEz4/8wbOk8kNvGooO0GUMvjs64Axvs7Li/Bi0bESbDRMKbTsuskE4gTstUPuK9xqyCFjsWTkcpEIdalbNEit2JWpQrbFCgg8uCDdMK//HLuq1WL1iLXKMSff6YVyTBYZ3Pw4NyKiz+ZGGllyM0AACAASURBVDRI5KGHRM4+m+eoX1/kgw+ib5FyxeGgAvrww7QCmv53AweK/Ppr7MblisoESaR5QueI8LBCJqLtg7YUQF0AG10+ywfgZDidels/twrTJ2DXrsDX1811/2eeAZYsoa9LSgpw7bX8zN+6fLDr/nnhzS8A0Bw3NiKpZeKqq4AiRZiDS8Tp11W+PHD4MM/xyCPAjTfS/6t9e6BqVWDGDODOO9kHwIz9w4cDH3zA/gYMoE9aqVK5x+JNJv76C+jUiZ8BQJMmrEpw1VWxy6K/cycwfTqzxW/bxpxs7dvTr6xFC+e1JyBJLRN2mCemTaNPo84RMSZcDS+YDUB1AL8DqH3m74pgjptL/R0XSHROMFEtgb4VuL5VhJK12QpTdCT8AkJ5k9S3KZKZabm1ICllIiWFUYYFCjitZgD9zkw/s9tuYwWAtm359znn0Ofs66+d4/31V5G77qL1oUgRkf79RfbvD+4aWrRwX0pNSYmdTKSnM3+bqzWxeXORKVNYvNyuqEwk1jzRrVto1kadJ5xYIRNRVdA4ZjQC8D2ArQBWA7gmr2OsCJ9etYoPXTDC46nIdOsW3Zw3VvkaeI4h2H6T3flz/36RqVOZtqFYMWsnI0kymXA4mAjWLMRtboUKsbQSINKkicjs2SJdulBxKl5cZORI+n+5Knj58vFn4cIiffsGnkYiJ0dk7FiRypXlv+CBe+5hSaZYyERODn3vunShkgkw6e2wYSJ//hl6v9FEZSKx5olQ+kz2ecKTuFTQQtnCFTzzwXF9Sw7krSDWD1wkfSaC+UJItqzSDgd9mUaPFrnySqePSPny9AOyejIKZYtHmVixgnUxXRWzlBSRChX4e/XqtBT16MEJMi1NpE8fkQMHnH307u0+5mbNRP75J7DznzzJXGlmHrRKlURefNFpmYq2TGzbRh+588+X/4IZHnpI5NtvrcnuH01UJhJvnghWaUy2eSIvklpBC/XL1HTGDOatIFYm21gLvt3GEUlOn2Yx7Z49nSV7AGZhHzpU5IcfaOkQse9kZFeZ+P13kdat3c8FOBWzsmVZhmnQIJGiRam0PfSQyO7dzj62bxe57z7nmA2DFq9AxnDoEC1w55zDY2vXFpkxg//zUAlVJg4fZjqFJk3kPwW1VSsGSdipNFSwqEzoPGGXcdiFpFXQwlmiS0ujCdrzGCsFLFZ9RfJLIhF9Cw4eFHnvPZGOHZ3RcWlpIjfeyBQLrgqCK3acjOwoEwcPsl/TAmlu55zD8xYuTKXsuedEypThvg4dRLZscfbx55+M3syXjwpZz56sLBDIuP78U+T2251+bq1biyxZYp11KtD7k50t8tlnInfeyWsAaEkcPVrkr7+sGUusUZkIHp0nEpukVdCsdnJ3FcwCBbwLZqDE6i1C314C49dfRcaNE7nqKuczVLasyAMPsEaha54rX9hxMrKTTGRm0serUCFxU8xclzW7duXyollb87rraKU02bGDy8n583Oy7NFD5O+/Azv/unVUhlytbWlp0ZeJzZtFnnrKWdi9ZEmRxx4T+f77+FvCzAuVieDQeSLxSVoFzeqHzJtp29cbVDB9RXMdXtf/vZOVJbJ8OX2Xqld3/o9r1aL15rvvnEuXgXDihD0nIzvIhMPB7P5mZn9zc62PCTDjv5n9vn59WrVMdu4UeeQRkdRUToKPPRaYlcnhEPn0U2f0Y7Fi9E8LJPGnlRw4wIS29es7z3vzzbwvp05F/vyxQmUi9L50nkhMklZBE7HGlGr2YZbY8MxaHowPgmufsUgMq29GTg4fZrb1u+5ylgsqUIC+Pq+8QutMoDgcLB80bpzItdfyC9mOk5FIbGVizRoqvZ5tS5d2/m5uAEs1jRzJbdUqLid3707FLDWVv+/alfd4MzMZWHDZZfKfX9u4cc5yTtGQidOnRebNY0mp1FT5z8/txRcDD2CId1QmdJ5Q3ElKBS0vE3QouWsKFaLwdevGCTjYKJ68xhEtoUjm9f/ff+eE2LKl02JTpgxTF8yZE1wOqf376bTdpYtzeQoQufRSkSeftN9kFEuZ2LGDFiLPAAAzE3+1alzubNhQ/ltOfustkW++cU+XYSpmjzxCK1peHDnCOpTnnst+a9ZkKpTMzNCvPxgcDi6l9ujh9J8rW1akVy+R9eutPVc8YCeZyOt/rvNE8s4T0STpFDR/D3CwD7cvU++qVcHnwckLNStbT3Y20xH07eueuuGSS5isdOVKtgmE06eZ/HTgQJF69ZxfvKVKMYDgnXfcAwbsNhnFQia++ILKieuSj6kUmwra0KF00jfv5fjxrCUpwv+R6wTXoEFgls1du6gkMx+dyDXXiHz+efR8uvbu5XXUrCn/WWY7dGDgQjhRofGOXWQir2de5wklWlghE9Eu9RQW3kq0mOUn/O3zhq+yH2YJjXvvZR/m5889x99DKXdhniszk2VjSpf23s4sFxLqeRKdY8eAL74AFi4EPvkEOHgQyJ8fuPpqlgO6+WaWAQqE7duBxYvZ37JlQHo6kC8f7/uwYUDr1kC9evzMzkRbJpYu5X1v2xY4dcp5bPHiwIkT3Hr2BI4eBUaOBNLSgMGDgT59gLPOAvbtYxmm11+nagbwHj/0EHD++bnHZMpEhQrAl18Cs2bxuI4dWcqpbt0wb2AAnDoFLFjA0jeLF/N+NmwIvPYax+GtnJQSG/J65nWeUOKKcDW8aGyRsBaYx+Rl6rXK7DxpEpdwXAvhup5ffQO88+efIhMnMsrP9O8pWVKkc2cWsT5yJLB+jh2jn9Cjj4pccIHTclO5MpfVPvoo8L5gE2uBSPRkwuFg2SUzd5nrVqAArWH33sv7W6gQ/1f/+x8tTiIi+/bR8mVm67//fpERI9xlYtIk93OvXOleCqpQIabZCMaHMFQcDpHVq2klMUtPVahAy59rGhCF2EUmrLagmcfEap7wlAmdJ+IHK2Qi5spXIFsk/G0CPdYqs7O3ciCughZq7bNEIzub/4MBA5yO36ZTeZ8+XIrMysq7n5wckbVr6YR+1VVOv7QiRURuuomRdr/9FtrSmF0mI5NIy8SUKc6oRG9b9epUnEqUoKLWuTMTy4rQSb5PH6e/WZcuzKAv4i4TZm1OM9/Z0KHu/n+GITJkSPDXECy7dvGZMSN+CxUSuftuLukGumSejNhJJqz0QQv02EjME64yYSpjugwaPySlguaJpxXqllvokDxpUt7tXT+z0grh67z+FDJPhS2Z3ozS02nBuv9+Zz3GfPlYJPr556lIBcLff7PQdKdOTl8ogNUA+vcX+eqr3E7kgZKRwXqJQ4faazLyhlUyYSZV9dzMepHm/8m81zfdJLJhA4/fv5/+gYULc6Lp3Flk69bc5zWf+dTU3NFxlSvntjpHguPHRaZPpz+bOYZmzUTeflvk6NHInDPRSBaZiOY8YT77rsqYWtDih6RX0FwfVtNZ0/UL3lP4fD3ceb2VWBX1kteSZjJF1+zaJfLqqyLXX2+mrmA2/zvvZBmeQ4fy7uPkSZEvv6SFxnTaBpip/p57OOnu2xfa+DIyWPpp6FBa4MylNn5h2ncyskIm0tOpoHgqZma1hQsuELnjDjr+A6xX+u237O/AASrDRYpQ2bnrLiYH9sWqVWzfqpXzPCkpTJWRkxM5mcjJYX68++9neSlApEoVkaefdlr/lMBJdJkQif48Yab1iHYqDsUakkZBu+iienmalT3fvgF+6ftq7xmRE6uszskiaDk5zKA+ZIjI5Zc7/0fVqjE1wbJleUfBORwiv/zCdBrXX+/MVJ+aygSlo0eL/PRTcIlnTUyFbMgQKieuCln9+lQCFy2in5odJqNIyERKiki7diza7c1yVro0Lb1mvrOaNXlPHA6WdRo4kMqOYVDR/uUX//d80ybW1zQtBddcQx+2SMrD779T6a5cmddQtCirSHz9dWjPjUISVSZiPU8k0xyRaCSNgmYY9fI0K4fzZmTuU0GwlhMnRObPZ+FrM7t8SopI06bMYbVlS94+YIcOiXz4Ifs47zzn/7Z6dZHHH6eCEEh5Jm9jW7JEZPBgjsdVIWvQgCV6TIXMEztMRlbLhGsSWXMzlx0LFKDv2JVXyn+Wpvfeo0/WoUO8h2baizvuYIkjXzgcVMZvuIHtCxfm/zGSVqujR5l7rWlT5yR97bW0sIby7Ci5SUSZ0HlCCQcrZMJgP/bGMOoLsBb58gEPPwxUquQMMXYNOQaAsWOBv/8GHnwQ6No1d18aohxZ/v4bWLSIaQmWLmWKgmLFgOuvZxqMG24AypTxfXx2NvD990x/sXgxf3c4mKLhmmuY/qJVK6By5eDGlZEBrFoFfP01//9r1gBZWUBKCtNpNG/OrWlTpozwh2EY60SkfnAjsBarZGLzZqBTJ/40SUlh+pKsLODWW4Hjx/n/KFsWGDqU58vIACZMAF58kelPOnQAnn4auOwy7+PNzgbmzAHGjwfWrWNfjz8OdO/uO51AOOTk8PmbOhX4+GPg5EmgRg2gSxegc2fgvPOsP2cyk0gyAeg8oYSPJTIRroYXjc18MypQgP5K6iBpH8yM6s88wySvrg7ePXrQRywvx/ydO0UmTxZp396Z0iAlReSKK7gctXJlYJGbrhw/znMPGkTLj5miI18+Ogf37cvajaE4gcNG1oJQZWLvXgZTeFrNzGXjli1F2rbl/uLFmRIjPZ0WxWHDnP5ot93mDAzwRnq6yEsvOZcUq1fn//rkycDGGSxbttCnzUwHUqIEl2VXr068AuV2IhFkQlGsxAqZiItEtTVqMCHgrl3Am28GnmTQDiTim9ipU0zuumABrWV79jCxYqNGwKhRtJRdeik/88aJE7RkLV7MbetWfl6xItC+PS1k114bXALQEydoIVu+nNsPP9AClC8fUL8+8OSTwNVXA1dembeFLB4IVSYyMphMc+xYtjcpUoT3sEYN4IILmAzYMHjfBgwAUlNpMXvhBeDwYaBdO+CZZ4Datb2fZ98+4OWXmZD28GHe9wkT+GysWUPLm1UycegQE9hOnUqLa758tNi++CLPV7Bg+OdQ3Nm9m/JmbnZA5wkl4QhXw4vGFkhSTjsSb+P1x9699ONp145+Q2a6hdtuY54sf0WhHQ7WJxwzhpYZ09+rYEE6+7/wgsjPPwdn4UhPF1m8mI7pTZo485zly0fLW79+Ip99FlwNTn9kZTEA4fXX7WEtCFYmcnJYssq0UHpazCpXZsmiIkVovXzwQUbaHjvGvGBmxObNN9Ni6ostW3ismbz2ttsik47g9Gkmzm3f3vk81azJtCxmclzFGjIzRdasYXDOHXeIVKzo/vxcfXV8yoRdiLfxKoFhhUzE1IJmGEY5APeJyOhA2jduTL+SeHnTCLasiJ0QATZupCVl4UJaJgD67tx3Hy0TzZv7tk78+y9L83zxBbd9+/j5ZZfR96h1a6BZs8CtG8eP57aQZWfTWtKgAUsJNW8ONGlCn7dw+ecf4Lvv+Gb73Xc8X0ZG+P3mRSRk4quv6JOzfbvzs7Q0lpQpVIj/i2+/pY9Y+/bAiBG0Zr7yCn3GDh4E2rShxay+F48KEWDFCmDcOD4rBQvSt6dXL+DCC93bhisTGzbQUjZjBp+xMmXox9alC615vqy2SuDs38/n3rSO/fCDs6xXpUr002zShFutWrSuRvK+6zyhJC3hanihbgDyA1gE4Ehebf0lILQz8fZmdOoUC08/+qhIpUrOt+SGDUWefZZWMF9WrsxM5pUaMECkbl3nsaVLM+XClCkie/YEPpb0dI6lf3+RRo2cFrL8+UUaN+Z5Pv+c7cLFtBC89BLHavpLmZGMDRvSn27mTJE//oictcBqmfj1V+Zwc7WYpaRwS0tjYlnTGnLNNUyDcvw4LZ1m8tkbbuC98UZ2tsjs2bw/5v/66adF/v3X95hCkYl//qGV1UzPkppKy9z8+cldoNwKsrNFNm4UeeMNluqqVs392b/iCqbBmT1b5K+/fPcTLzJhR+JtnlACwwqZiKUF7TkA0wE0jdYJo73OHw9vcvv3s/D4woW0dB0/TqvKddcBQ4bQclK+vPdjf//d6Uf21Vc8Nn9+XueIEbTM1KkTWMHx9HRg5UpnlOUPP/CNMn9+WsieesppIStaNLxr3r2bVjHTQvbjj7QmAbQcNW5MK1+jRhx/oULhnS8ILJGJ/ft5v6ZN41RrkppKv7xrrqGfzqJFwEUXAQ88wMjGb74BbrqJlqnWrWkxa9Qod/8ZGcC779Ifbft2+qy99hqtWIUL+x9boDKRmcnxvfsu8NlnfBbq16dfW6dOkYn8TAaOHqUP4KpVTuvwsWPcd/bZ9BV8+GHKWb16UX32faHzhJK8hKvhhbIBuAXAxDO/R+XNSN9SiMPBPFWjRtESZUbxnXsui4YvWsSkrd44elTk448ZFVe1qvNNu2pVke7duS/QqMhjx+gj1q8f39LN3ET589OnbOBA1j8MN09VRgaz3I8bR38l12LfBQsywrNPH5E5c0R27w6sT0TAWmCFTJw8KTJ8uLMyg7mZf19xhTPJbPXqjMxMS3OP5LzuOkbNeuPffxlVW7q0s785c6yrUelw0Fr36KMiJUs6n8u+femjqASHw8EyaVOnUrZr1nT+rw2Dz0K3biLTpjGBbzhRrnaViWDReUKxCitkIuoWNMMwqgF4FMBNebTrCqArAFSqVCns84azzh/vETanT9M6YvqT/fknP69bl3mtbr6Zv3v6kTgczFm1eDGta6tX0++raFGgRQtG+LVuDVSrlvcY0tPpp2T6kK1bx/9FairQsCHQvz+jLJs0YURhKIjw2kzLwHffAevXc8wAULUqz9G4MS1DtWoBBQqEdi4rCVcmRICZM4EnnqC/mInpZ1alCv3y1qwBKlRghFvHjrREmZZDgPmgJk3Kfd5t24Dnn6fv16lTQNu2zGd36BBw7rmBWUj9sWcPMH06+//1V/qw3XILfR2vvTb8/pOFkyeBtWvdoysPHOC+4sX53HfoQBlr2NDe0cw6TygKomtBA1AIwDIA57t8Zus3o3h9ozpwgG/Gt9/uLN1TsKBImzb0N/HlT7JnD6P97rzTaSkB6Fc2YAD9zAIpOH70KPOM9e1LHyXTQpaaSqvVoEHMUxaOhezYMWalHzmS0YVnn+0cb5EiLP80YACj/fxFmQYLLLQWhCsTK1aI1KghbhYzM+dbhQq0kgK0SI0bx8z/EyeKlC8vbj5p3p7tVatEbr2V1pa0NJGHH2aUphUyceIEa662auUsCH3llcyR5q16g5Kb3btFPvhApGdPypjpp2laSLt0YZb8TZsiX8bKTjIRDsk2TyiRwwqZiLYF7SoAVQDMN5zmmmKGYawHMENExkXqxKGu88dLhI0I84mZVrKVK2kBO+cc4I47aCW75prc1qlTpxjBZ/qSmRnly5Wj/1nr1rRilC3r//zHjuW2kDkctJBdcQVzaZn3PS8/JW84HMBvv7lHVm7ezM8B+lK1aUPLWOPGzMMWJ5aXkGQiM5N+gkuWOD8zDD4HhQsDl1zCyNvDh4FBg+hTN2cOULMmLVZXXUWrW1qau0w4HMxvN24cLTAlSwIDB/L4c87heZ57LjSZkDPRnlOnAh9+SKvq+edzfPfeG5glNlnJymIEq6t1bPdu7itYkBaxPn1oHWvUiP5kcYzOE4oCRFdBE5HFoOD9h2EYR0TER7pLa2ncOHihad6cy2CnT/OnWSrEDmRlccIzlbLff+fnl1/OSfXmm+lYnZLiPEYE2PL/9s49PKrq3P/fFUIgQEIIIAQIECAEQQNIuCSoUFDQUtCWiigoRS168IaXY6Voqe3xWnus9veznnpr9Dw9VnqorbdK5QhiBRS5g1wE5H5PgAAhIZl1/vhmnb333DKTzGXP5P08zzwzmbVnz8qe/e713e/7rnd9bQmypUsp0tLSWPbi5ptZKLawMPjU+ZMnnYJs9WpLkI0YwUHXhBMbIsjKyykwjCBbuRI4cYJtbdvyO77/fT4PGxZeUdtw0BrYt4+hUvOI7P4bZhMbNzqXZ1KKx37wYA7kX34J3HEHJwt88AHPg337WCLhjTcYoja/b3Exw2P/8R8MZW7fzqW0XngBmDnTd1JGuDaxaxe/8403gJ07eZPwwx9yUsGoUc7zUyDHjvmWuqisZFturlXmoqSE9t68eXz7G0lknBAEkhArCcQTt82wKS8H/v53CrIPP6RoSUsDxoxh3anvfY+1iuyUldHTYnLJ9u3j+/36AbffTkE2alTw3K9AgiwtjR6yefN4fEaMCF+Q1dYCmzY5Z1Zu2cI2pVg7bcoU7nvECFYMj8agfv48v9cuxtau5fEzuM3Lk5rKHLvBgynQv/gCuPFG/h7LllF0793Lgfz11+lFtQvv48c5A/O3v+XszyFDWJV/8mTu2x+h2ERFBT12paW8CQB4js6fD/zgB42fiZtMeDzA5s3WzMrPP6e3GOBvMHgw8wNLSnisZR1R9+G2cUJIDkSghUBD7qgiyfbtlpds2TIKmo4dmUg9cSJDXfbirDU19DgZQfbll9aC41dcwYkB48YxvBSIEyecgmzNGkuQjRgBPPKIJcjCnYp/9KglxlasoKg4fZptHTpwnzfdxOehQyNTeNabkydZiNcuxDZutJY/atmS4cDJk1kAddAg/p2R4Y5iqCac2bcvy2KsXm0Vk127lkn8u3fzGL7yCs8Re7937mSZjNdeo2fmu9+lt23UqND+P3824fGw3Mof/gAsXMhyHH36AL/8JX/PYOdbU+LUKZ7zxju2YgXPR4Dnf0kJS5+YUhcN8UALsSfe44SQfMRdoGmts6K170SdVVNTw77/7W8UZWatyosuAh56iKJs2DBnjtW331qCbPFiXvBTUrjdo48yl2zo0MBekRMnKP6WLKHHwy7Iiou5j1GjwhdkJnfGLshMRfvUVIZnZsywZlb26hVZAeQvRLl2LQWKoUMHeinuvdcSY337Bj5W0SYUm2jenP3evJm1q95+m6HEKVP4PHQo18G86irn8Xz1Va6JuXkzz5/p04EHHmDOXkPZto2esjffpLeubVvu1/yubhC08UJrnmtGjC1fDmzYQNsy3uGpUy3vWJ8+Tft4BULGCaEpEneB1lgCGdfy5QznmJyAxYvdbXwnT1Jgvfsuc4bKyjgIjxoFzJ5NUZZny8o4fZr/96JF/JwJieTmAtddR0E2diyTvP1hF2TGQ6Y1k8ZHjKAgGz2a4ctwBNmBA84yF6tWWcvE5OTwN7j9dj5fcklkvQOhhCjz8+mVuPVWS4zl5CTeoFhdzby7F1/k/3fbbRQCRUU8X6qrgaws/l8eD8+pRx+1cuhSUxmCnDSpYd9fXg786U8UZitW8GZg3DhOLpg0yRUFTuNCZSUnyNjzx44cYVtGhmVbJSW0rbZt49vfpkKyjBNC0yKhBVow40qEWTU7d1qhy6VL6Tlr356hqokTKbJMrSKPh4OrSe7/7DMKkvR0XnRmz+YA2a+ff7FRXu4UZGvXWoKsuJi5QUaQhbo+5rlzFHZ2QWZmlqWlUQj9y79YuWO5uZETQo0JUSYDPXvS8/Xgg8w9GzyYHtf27RnGrq6mwL/vPuCddzgxpG1bKzSqNfP+whFoNTW8ISgtBf76V84kHTAAeOYZYNo01kRrahw44JxZuXo17RKgN+yqq6xk/v79E2ZmcVKR6OOE0HRJaIEWzLhCmVUTa9d2bS1zw4wo27SJ7/frx4F00iT2w1zEDx/moPvRR1x4/PBhvl9YCMyZQ0F26aX+BVVZmTNkaRdkJSXMVRo1KnRBpjVzmuxlLtassQajHj0YajNibNAgfldjScQQZSw4eJAzLAcNogCbNIni68knKZw8Hp5vTz7JMPJ//icnj4wfH75NbNhgLVB+6BBF4KxZDGH6K3CcrJw/z5sCe7hy9262tWzJsPJ991nhyvpK0wixIdHGCUEwJPQQFsy46ptVEyvXdkUFvQ7vvss1L48dowC7/HKG2SZOtGYGmor/xktmwlEdOjDJe/x4ijJ/a2PaBdmSJcz70poDR3ExBdno0cxJC0WQnTnD8KRdkBmBmJ7Owej++ynGhg8PvF5nODSlEGVjSUlhIv4111gzWnfv5vEyteFSUoDnnmMNM3N8QrWJ5s0Ziv70Uwrx1FR6dmfM4LMbVmCINseP87w3guyLLzjxAeCKDCUlvFEqKeF52BSOSSKSCOOEIPgjoQVafcYVbFZNNF3be/ZQkP3tb9b3ZGVxZt2kSQx7ZGVRQG3fzhIHixZxBtyZMxwMS0qAxx+3Fhz3LitRVsbB0wiy9estQVZSAjz2mCXI6vNkmX7Yy1xs2MBjA1AUjRtnJfJffHHjvVOJFqLUmrlEJtcv3vTvzzpwAMNqzz7LiQJK8fzKz+dSTuHYxMcfO71vzz9PD9nzz3NfCV78NCgeD28O7OFKMzmnWTPa4G23WeFKKXWROLh1nBCE+khogQY0fGpzJAsLejwsZWFCl+vX8/38fOCuu+glGzmSXomTJ3mxMMn9337LbXv3pndi/Hj2xXudvOPHLUG2dKlTkI0cGZ4gO3nSKgJrHsZLlZFhVf4vLub+OnRo+LHRmnlpRoStW+cbouzYkQLMDSHKc+coVrdudT62bbMK5bqFv/+dSfn/8z/83ebM4TEMRzxoTYH3hz9wFqbxvqWmsm7a9OlR6XrcqaiwSl0sX86H+X3bt6cImzGDz0VFDV8fVnAHbhgnBCFcFJeMcjdFRUV61apVEd9vY3ILzpxhXpgJXR4+TC/XpZdSkE2cyIKqtbUMFRpBtmIF38vIYOFOE7bs3du5f7sgMx4ygOHFkhL2efRohhqDCTKPhwni9kT+zZs5MCtFT4zJGysuZj5cQxOZz5/nd3mHKMvL2a4UReugQcyLileIUmsud+QtwrZuZZjQbhLduvF3tD+uukp9pbUuil2PfWnVqkhXVq5Cly4UZrNmhTcj8OBB5qWVljIXskULhkuHDeNM4cEFHwAAIABJREFUxLFjk8dToDVLj9hnVq5fb4nRAQOclfnz85teyLyxKBV/m3DjOCE0XSJhE01aoIXLvn3Ae+9RlC1ezHBQZiZDShMnMoTZvj23M4Ls44/pnVKKuVPjxlGUFRc7l2c5dswpyDZs4Pvp6fSQ2QVZsFyX48c5EcGEKr/4goUxAZZlMGLMLJHU0Gn+J09a3jDz2LTJGaIsLLREmAlRxrKCfEUFPV/enrBt2yiwDW3a0GPnLcTy8/331w2DUXp6kX7ppVW44YbQc58qKzn7srSU56fHw/NwxgzWTwtUkiXROHeOXkF7uNLkT7Zpw3PfiLHhw5luIDQON9iEW8YJQQAiYxMJH+KMJh4PL/QmdLlmDd/v1YvrHE6cyKV0amsprh5/nKJs82Zul5Njlcu44gpnDs/Ro86QpRFkrVpRkF1/ff2CrKaGn7Pnjm3fzrZmzSiQpk2zBFlDPAPeIUrz2LXL2qZjR+bozJljibH8/NiEKGtr6fXy5w07cMDaLiWFpSn69uUEDbsQ69Il8TwmAwZQWNWH1hQoZoHykycZAn34Ya67WlAQ/b5Gm4MHncskffWVdaPQu7eVP1lSwsKwUupCEIREQASaF5WV9I69+y69ZQcOcPAuLmbJgkmTGAbcvJli7OmnKbSqqhgmuuwylj8YP56DgRn4jx4F/vu/LQ+ZWejaCLKpUynIiooCC7JDh5x5Y19+ac0q69SJfbz1VoqxhuTNhBqiHDoU+PGPLTHWuXP0BU55uX8R9s03PPaGdu0oOq680inCevcOvb5bMrB7N3PK3niDor1VK064mDGDC6Un6gLl5qbE7h0zeZwtWvC8v/deq9RFp05x7a4gCEKDEYEG3oGb0OXHH1OktWlDkTVxItcpTElhztmzzzI8tH8/P3vhhSzGOn48PTOmMv6RI05BZmqetWrFPLUbb6QgGzLEvyCrrqbHzi7IzEDUvDk9Vj/+seUd69EjPJEUaojyuutiF6I8f57LQPlL0D961NouNZWCq6CAv41diHXokHjesEhx+jTPudJSzggGWOtu7lzghz9MzCK9ZWW+pS5MeDonhzc399xjlbqIRO09QRAEN9AkBZrWFCemFIZJW+jenYsUT5rEC/6aNRRjEyZwG63pobniCiu538yYO3KEkwVMyNIIstatKcimTbM8ZPbcM9OfffucifyrV1ueodxcirB77uHz4MGhe4PcFqI05Sr8ecN27rRKewD0fhQUcFF4e45YXp7vMWyqeDw830pLuXTTmTMMwT/2GBcoty8P5nY8Hp4H9nDl11+zrVkzTiyZOdPKH+vevemKcUEQkp8mI9DOnaNXwYQu9+7lxX3YMODf/o2estat6SX73e/oOTp1ip6zESO4FJJZcLxZM4qMpUuBp56iKDN5Z0aQTZ9ueci8xYR9vT4jyEy+VMuWFHF3380QzfDhLIoZCm4KUVZWMrTmnaS/dSu9d4aWLdmngQOZqG5EWN++krwdjKoqrun45psMZ2ZksFbZjBn0KiWCcDl9mmF6e2V+c65mZ/P8nz6dYmzoUCl1IQhC0yKpBdrhw/RqvfsuhdeZMwwxXnmlVVl/0ybmkk2ezHwmgHfm119vLTielcV9LV3KnJ4lS6w7+zZtKMhuvpn7u+QSpyDTmmE7e0X+deuYSwMwVPed71hlLgoLQ/MOuSFEaTx//rxhe/b4L1cxbZozJNm9e2LlQ9lnf8aTjRv5e195JfDEE/QyRnLh+Uhjlgqz546tX295TPv3pw2a3LG+fRPrvBAEQYg0SSXQtObAZWZdrlzJ97p2ZbhnwgSWwViyhELrjjvodWrViuLq7rspyvr2tQTZ3Ll8tguyyy6jp8KfIKuo4JJLdkF27Jj12WHDgIcespZIqm+9PjeEKP2VqzC5YWaSgvn/CgrowbnlFme5ikTwflRVUXDu3Rv4YTw88aZrV57foXpXY01VFcP09tpjBw+yrXVrnv9z51KQjRiRPCU+BEEQIkXCC7SqKgooE7o0ifRDhtBLVlzM8OGiRRQNJtl84EAubGwWHC8v535+8xsKuC1buJ0RZD/6kSXIjPAxOTP2MhebNlkFMPv1Y+jUJPIPGBB8in91tf+1KGMRoqyt5bEzwqu+chUFBTwedm+Ym9fErKmhQPAWXHv2WK+PHPH9XPv2zAHs0YPnSW4uhUW86dzZXeLs0CGnGFu1yvLm5uWxKLPJHbvoouReyF4QBCESJORl8tgx4IMPKMo++ogenpYtmbz/4IOcybd6NReTnj+fn+nYkWLMPLSmIFuwALjzTmvdvYwMCrJbbqEAGTzYGkzKyznL0wiylSut5WGysugRmzzZKgIbzCtw4oQVojTPsQhRlpUFLldhvhsIXK6iTx/3zZTzeCi87WLL+3HggCWcDRkZDLHm5lJ45+Y6H926+Q8bukGgxZOaGnqq7eFK49FNS2MO5T33WMvr5OTEt7+CIAiJSMIItM2brdDl8uUcbHNymCt2ySXMDfrkE4YPz55l2HHkSObnjB/PUOKyZfSOPfGEryC79VanIKut5SD06quWIDNetZQUegGmTLEWEA+UM6M1hYO3V8x4+gD2bfBgiqFIhCirqzkj0p8QM+FWgMfIlKv43vecCfpuKVehNYVxsLDjvn1OcQlQ4BqhNXasr/jKzW34KgpNjRMnnKUuVq5kgj9AT15JCW9ySkpoi24T8IIgCIlITJd6Ukp1AHA/gKsANAdwDMAcrfW6YJ9r2bJIV1WxFsagQRQy7dvT67NoEQUQQO/O+PF8FBRwpqSpQ7ZtG7fJzKQgM0snDRpEIXTkiLPmmL3eUseOzvUqi4r815SqrvY/i9J42ZSi+LEvf2RClOGiNfPk7Plg9ZWr8H7k5cU/1HT6dGDhtWcPxZd3Yn5qKsN7/kSX8Yi1bx89gRnJZW0aahPRWtZGa55Ldu+YmaGcksLUAPu6leHW3xOSk2S2CUFoCIm41NMAAFsBPKq1rlVKjQSwUClVoLWuCfShFi0YMqmpoXj69a/pQcvIoHdk7lyGA7/9lmLsgQesJY8yM1lAdtYsS5DV1nIG2fLlwHPPcZ87d3L71FRuM3OmJcjy8nwHIXuI0j6L8vx5tqens0/XX+8MUYabLG/KVfhL0PcuV2HEn71cRUFB/DxF9qT7QOFHI14NSlGw5ubSS3n11ZboMo9OnZJquZ4G2USkOHPGt9RFWRnbsrIowm68kXYwbFhs11IVmixxtQlBcAtxXyxdKbUKwA1a6+2BtklNLdK1taugFL1X48czlHLqFPDZZ8wlM4KsbVtfD5lZIsnMqvzqK9ZFA7gOowlTjhjB/drzjsIJUdq9Yvn5oYsIj4crE4RSriI31783LDc3tmUJamqY1xUs9Bgo6d5bcNkfXbqEvvh3vIj2wtCh2ERDvAXmXLYn869da3lb+/VzescKCqTUhRAaiWoTghAtEtGD5kAplQOgK4D9wbbLymJl9GbNKK7eeovFZQEKsssvZ8kMM6tw3ToKsaef5mC0bx+3TUvj7M7Zsy1B1q2b5R0LNUQ5fDhw++3hhygrKvyLsO3b3VWuwuOhuAoWejx40DfpPjPTElrhJN0LFqHaRCiY5cLs4UozI7dVK57HDz9slbrIzm7sNwpC5ImkTQhCIhE3gaaU6gLgA9CNfdZP+ywAswAgNXUQ7rqL72dlUZDNns11BjMzGaJZsYJrYq5ZY4UZe/ZkaQTjIRs40EpgNiHKhQsjG6K0l6vwfpg6UAA9E3l5FF5jxjiXMopmuQp/Sffe4cf9+4Mn3V9xhf+8r8zM6PS5qRCOTXTv3t3n84cPW96x5ctpF2a5sJ49eQNTXExBVlgY//xDQaiPxtqEICQycQlxKqWKAZQC+LnW+o/1bZ+VVaTnz1+F4cOZk2UE2YoVHJQAegSGDrXyxoYPp2crWiHKUMtVZGf7D0n27h2d2W4VFcHDjnv3Or11gP+ke+8wZDST7hOJ8+eBtLTIh3PCtYmioiL92murHN6xHTvYlpZGD6YJVRYXM3QsCNEiGiHOhtiEhDgFtxAJm4i5QFNKzQRwL4CpWustoXzmgguKdNeuq7B+vRVW69vXObPyoovYFslZlNXVHPT8CbHjx63t7OUqvB8dOjT8WHlz7lz9le79Jd3n5ATO+UrCpPuQOH+ensTjxym2y8pCe11RAQCRHYwaYhPNmhVpj4eDUadOztyxSy6hx1MQYkWkBVpDbEIEmuAmEi4HTSlVCE6fvlxrHfKiOcePU0jNm2ctkdSsmTWL8vnnGx6i1JqTCPyJsF27nOUqOnem6Jo82RmSjES5ikBJ9/bwo1kFwU6HDhRZeXkM/SZi0n1jMEIrVIFl/qbQ8k9KCj2f7dvzuUsXnjvZ2Xz87GeR639DbaJ9e+Df/52CzN8sY0FIVBpqE4KQbMQ6C+V2AA+Ha3T9+7MQ5tq1wMsv87U9RNmpEwXY+PGBQ5RnzzL86E+InTplbWfKVQweDEyd6ize2tByFSbpPlil+/qS7ocM8Q1BdutGIZoM1NQ4RVSogsv+23ljhJYRWzk5XG7LLr78vc7ICD57MZICDQ20ie7dgenTI9oPQXALDbIJQUg2Yi3Q+gIYq5R62uv9eVrrvwb60MaNwLXX0ktQUEAv2h13UIgNHGiFKD0ehgC3buWSTN7lKuyYchU33dS4chVaUyjUV+neePYM/pLuvfO+EjHpvqYmPI+WeV2f0GrXzhJSnTtbQiuY2MrMTIgyEQ2yCUFIYsQmBAExFmha6ysb8rnu3YG332aeWevWHMxN5fwXX3QWb62stD6XkUHRddllThGWnx96uYdgSfem0r2/pPtu3Siyiov95325PeneLrTCCR/ai+d6Y4SWEVKdOgEXXhhYYJm/E0RoNYiG2oQgJCtiE4JAEmKivccDvP56aOUq7EKsc+fgIsiedB8o/OgtOOxJ94WFwIQJ7k66r6nhxAF/eVjBXocqtLKzLaFVn0erbdvkFVqCIAiCEEkSQqDt2wcsWEDRZdbZrK9cxfnzwXO+6ku679WLdda88766dOGszVhTWxu+R+v48eBCSymnR6tjRx7TYPlZIrQEQRAEIfokhEAbOJATBAweD+uf7d0LvPee/9DjoUP1J91753zFIum+ttbXo1XfjMOyMt/yGXbsQis72xJawbxZ2dks+itCSxAEQRDcR0IItMOHgWnTnJXuAyXdd+8OjBvnP+8rkkn3RmgFE1X+XtcntLKyLCHVvj3z5ULxaLklpCoIgiAIQuNJCIF26BArpefmsu5TJJPua2sZBgy3YOmJE85FzO0YoWUXUvn59Zd3EKElCIIgCAKQIAJtyBCgvgLRRmiFW94hmNACnB6t7GxLaAUTW1lZIrSSCY+Hq0pUVfmuUSoIgiAI0SAhBFpZGfDb3wYXXOXl9Qstu5Dq3dt/SQf7axFascNbBFVVOV+H09bYz3u/V1MT76MjCIIgNDUSQqDt2gXccw9f20OHRmjV59Fq106EFuBfBLlFDEVaBDVrxtm9LVpwqSt/zy1asFZeoLZAn7vrrsj2VRAEQRC8SQiBNmAAsGQJxVlj17yMNsFEUKQ9O+FuEy0RVJ+oCSSCQhFDDWlLS4uuIBeBJgiCIEQbl8sd0rIl65MZjAiKpuBJNBHUpg29hdEQPP7aoi2CBEEQBKEpkxACbd06io9YiaBAosQugqLl/bG3iQgSBEEQhKZJQgi0rCxg6tToiCERQYIgCIIguI2EEGg9enAWpyAIgiAIQlMgIQQaqquB3butuJ/d/SUITRFjE/Z4eIsW8VkoVhDcQHU11/nzHifEJoQEJTEE2s6dQM+ezveGDwdWrODr4mJg2zanYV5+OfDqq2y/4Qbg2DGnsBsxApgzh+2PPkrjtn9+4EDgqqvY/sc/Mg5qb+/Zk1VrtQY2b/YVjq1b+1/FXRAiwfbtvjYxbhzw0Ud8PWAAl+Cwn5Pf/S7wm9+wfdIkoLLSKfDGjAFmzWL7v/4rF2q1txcVcZvaWtqE9znfqxcfNTXA1q2+s0rEJoRosmULwy12rrsOePttvs7NBc6edZ6T118P/PKXbB871nmdb9ECmDABmD6dawv+5CfOMSAtDRg5ko/KSuDPf/Y95/v25fqDVVXAN9/43lC1bi2OBiEgiSHQunQB5s51Tt284AKr/dprgX37nFMqe/Wy2mtraZjl5dY+7NNCS0uBo0f5WVPtduZMS6DdfDP3Yefuu4EXXuBnLrrIt88//Snw+OMUhl27+hr23LnAHXcABw4AP/iB03DT0oDbb+f3798PPPGE713hNdcAF1/MQfj9930vDEOGAJ06cXmFnTt9B9PsbD6b/7ch62QJ8aNrV2DePOc537271T5lCs8901Zd7Ry8amuBM2csm6iqAvLy2KY18PvfW1OWDXPmUKCdO0eb8OaRRzjYHT/u3yaefhp46CFgxw6gf3/fc/bxx7no7rZtwIwZvufsnDm88frmG+C553w/P2UKB8Q9e4B//MPXpoYPZ4HEsjJfT0taGtuaN+c0caXEJhKN3FzLJswjP99qnzYNOH3aOY5068Y2rWkTlZVOm7r4YrafOwe88orVZpg/nwKtvNy/TfzqV8CDD9Lb7c8mXnqJ1/rVq7mOofc5+fzzvNavXg3ceafvOf/Tn/LGacMG7stbAE6fTrv/5hvg0099baqkhGsMHj3Kscb7+zt0YG0rsYm4kBgCLTOTgikQP/lJ8M+bO6hA7NnDZ2Ok3uv5bN3qNOrqaqBzZ7alpnL/9rbqahoNQEO4/37f9txc6zvbtuV7dhF58iTbjx937t9cHPLyePHYuhW47Tbf/+mdd2jYy5YBEyf6ti9ezMH2T3+ih9FbAH7wATBoELBwIQdO7wvDSy/x4rZoEbBgga/hP/AA/68vvgC++sp3/xMmcDDctQs4csT387m5vBhUV/M5NVUuDnaysoLbxPz5wT///vuB25Syzj+7TZjjn57OC773Od21K9szM3nO2tuqqjiQmfb77nMOpNXVvBEz35+ZyffsIrKigu2HDzv3b2xi0CAKtLVr/dvEJ58Ao0cDH37IgcubVat4Y/Pyy7x58j7nly1jZew33uDA6T0Ylpbyxuedd3h8vQXivHl8b9kyYONG3/1fcw3/92++od17799cM6qq6N0Um3DgadsOuOWWwBs89VTgNqVYbDMQGRnAqVN8bWyiqsqaYXbBBfRq28/nqirLy925M6+13uNISQnbO3bkDYi3TXTqxPaUFPahutoSmVVVHDMAOii89w/whqZHD2D5cuDWW33/rzVraDcLFlAAerN9O9CnD/Dss8DDD/tex9es4f/+4ovAa6/5XscXLGCdrP/6L9qf9+cfe4zfs3gxb8y8Pe4TJrB961Zek+yfTU+3rhnmt2jWLKlsIjEEWqwwQsC7Gm7v3oE/k5pKN3ogMjKAJ58M3N61qxWW8kdhIe9uDObiYE7CESMoML0N39w5FhVxwPBuLyhg+4ABwM9+5jvYtmvH9lataATmfXNx8HjYvmsXxZx3YbrZsynQ3nvPCiHYqaigQHvhBSvsZsfs/847eecKOD0d337L9x54gCLRbvhduvCCAPAO1gyGxrC7dOFdLQC89RYHfPvnO3ViuBCgwDx3znnhaEr4s4mUlOA2kZ4e3CY6dgw+WObnB7eJkSN9baKmhv0CgCuvpMfCe7Dr14/to0ZZNuHPw3jJJbQJb4GZkcH2Nm2AnBynTZSVWf3Zvt2yCbuInDuX7X/+M897Oykplpf+iSeA1193trdty4WDAeCmmzjwKWWdkz17AuvXs33WLA7I9oGyTx/Ljh57jF51+zndq5dVgbm0lN9lb+/WjeIWAD7/nH21798FrFnDn6VPH56e5tm8zs6O0Bf5s4nUVH5JIDIz6eENRG5ucJsYNIjXuUBcfTU95gZvm5g8mee9t02YceLqq4G//MVXYJpIVUkJPeTe40irVtb/17mz1V5RwWcjYL/+mmOBff+AJdDefJPnnZ3sbN6oAPQULlzobO/enXYOMGVj0SKnTRQWAp99xvYbbuA4YD+nBw7kjRZA8WlSQsx53b+/JWpffpneVfs4kZdn3XR++im/O8LjhNLBFrCMEkqpHwF4AEBzAJ8DuFNrXRlo+6KiIr2qvtXSBfdgD5uePWsZq33AKyzkxWPLFoo8e9v585Z36IMPeOW1t6emAs88w/bnnqMR2vffrh0vNgDvqD/5xPn5Pn0YMgBoYJ9/7uy/Pb+xsJDhAxsK+EprXRTJQyY2keTYbeLMGYo6+zlZXc1BGOBA4n3TpZTl9fvLX4BNm5yfzciwBrvHH+f57e2xf/NNtk+ZwhsP+/6HDLE8SBddxP3bsec39uhhRR3qcINNdO1apMePX4UdO+iEPHDA2Z6V5SvazHNOTlI5XhIDra2DXlFBu7Cfkx6PFWJeu5Y/qP0636KFJXrfesvy6tvFpbkpmjePItHefuGFjAQBTCfassW5/zFjgL/+le25ufRS2rHnN2ZlWVGHOiJhEzEXaEqpSwH8HMAkrfVZpdRDALporecE+owMRkLUOHvWuTyEEYAmNLFqFQ3P5glRU6dGdDASmxBcxZkzvuIxLc3KcVy50ikwXWoTZ8/SUbhjB/5PtJnn3budacXp6XQi+vO+9ejh/iUGhShjz100Ii493cph/Oc/rfzFCNpEPATaewB+obX+ou5vBWALgGFa65P+PiODkeAmlFKRHozEJoSEJtFs4vx5OgGNaLMLuB07mNVgSE2lSPPnfevVi+O0IHgTCZuIx33BYABfmj+01loptRLAMAD/iEN/BCHeiE0IgpOwbcI4OUJJ/2ne3BJb3ng8wMGDvl63HTuY+eAVyULXrv7FW+/ejHwJQkOJqUBTSrUDcEb7uu0OAcj12nYWgFl1f1YppTbGoIuNoQOAY/VuFV+kj5GhIFI7EpuIO9LHyOAKm2jRIvY2sX8/H0uXhrR5IvyW0sfI0GibiLUHLR1AoJiq432t9e8B/B4AlFKrIp2AGmmkj5EhUfoYwd2JTcQR6WNkEJsIDeljZEiUPjZ2HymR6EgYHAXgb7JzZwB7Y9wXQXADYhOC4ERsQhAQY4GmtT4PYKtSaoh5ry75swi2fANBaCqITQiCE7EJQSCx9qABwDMAnlBKtaz7+2EAHwaamVPH76PfrUYjfYwMTbGPYhPxQ/oYGcQmQkP6GBmaRB/jVaj2DgB3AmgB4BMA92itq4J/ShCSF7EJQXAiNiE0deIi0ARBEARBEITAxCPEKQiCIAiCIATBNQJNKfUjpdQGpdQWpdRrSqmA9ZmVUsOUUiuUUhuVUp8opfLc1Eel1PNKqa+VUmttj1di0ce6779OKdW5nm3icgxt3x+0j/E6hkqpDkqpJ5RSq+t+60+UUgODbB+14yg2EdF+ik00vF9iE1Hoo9hE/TR5m9Bax/0B4FIAHwNoVff3QwB+E2DbTAAbAOTV/X05gJUu6+MfAFwbp2PZDcC3AAYF2SYuxzDMPsblGAIYBWAGgGZ1f48EsANAaiyPo9hEzM83sYnA3ys2EZ0+ik00vo9JbRNu8aA9DOCnWuuzdX//CsDVSqm2fradBeBNrfUuANBafwpgp1JqrIv6GBeUUu+D09C71LNpvI5hOH2MC1rrpVrrUq11bd3f/wRQDsDfHU80j6PYRAQQm2g8YhNR62NcEJtoPLGyCbcINJ911wCYdde8GQ3eodj5B4Ax0epcHeH0MS5orSdorXMAfF7PpqMRn2MYTh9dgVIqB0BXAPv9NI9G9I6j2EQEEJuIPGITQRGbiABiEyTuAk2Fse5aHbl1baFsGxEa0EcA+L5SapFSap1S6n2lVGG0+tcAYn4MG0hcj6FSqguADwE8arsjthOV4yg2ERfEJkJAbCIwYhNxI2ltItZrcfoj5HXX6tk+mvVCwu3j++BirqVa67NKqXEA3ldKDdRal0Wrk2EQj2MYLnE9hkqpYgClAH6utf5jgM2idRzFJmKP2EQ9iE3Ui9hE7Elqm4i7Bw3hr7t2oK4tlG0jRVh91Fov0Fr/zqhprfUiAEvBBFI3EI9jGBbxPIZKqZkAfgdgUhCjA6J3HMUmYo/YRBDEJkJCbCLGJLtNxF2g6fDXXfsMvrHbMQD+x0V99EcmgBNR6F5DiPkxjBBRP4Z17vH7AXxHa72lns2jchzFJuKC2EQAxCai1kd/iE00nuSxCR3CVM9oPwBMAvARgJZ1f88F8Ou6188BuNy2bVcAmwF0r/t7NIDPXdbHGQAKbX/PBLACQPMYHc8lsE1NdssxDLOPcTmGAP4/gAkB2mJ2HMUmxCbccgzFJsQmYnkMw+xjUtuEG3LQoLX+W12i3ZdKqf9bd62uOQ9AJ9u2+5VStwP4s1KqDeg+nOqmPgLYA+AppVQnAM3Bu6cJmndY8cAVx7Ae3HIM+wIYq5R62uv9ed59jOZxFJuIOq44hvXglmMoNhGFPkJsoiG45RjGxCZkLU5BEARBEASXEfccNEEQBEEQBMGJCDRBEARBEASXIQJNEARBEATBZYhAEwRBEARBcBki0ARBEARBEFyGCDRBEARBEASXIQJNEARBEATBZYhASyKUUulKqX1KqT11RRLtba8opWqVUvEuNCgIMUNsQhCciE0kDiLQkgitdSWA+QByAcw27yulngRwK4C7tdZvxal7ghBzxCYEwYnYROIgKwkkGUqpZgDWAbgAQC8At4Frg83XWv8inn0ThHggNiEITsQmEgMRaEmIUup7AN4FsBjAGAD/T2t9T/BPCULyIjYhCE7EJtyPhDiTEK31ewBWAxgL4E8A7vXeRil1p1LqC6XUOaXUkhh3URBiitiEIDgRm3A/qfHugBB5lFJTAAyq+7NC+3eTHgTwFIChAIpj1TdBiAdiE4LgRGzC/YhASzKUUuMAvAngLwDOA7hFKfWc1vpr+3Za64V123ePfS8FIXaITQiCE7GJxEBCnEmEUmo4gIVqJ4PMAAABG0lEQVQA/glgGoBHAHgAPBnPfglCvBCbEAQnYhOJgwi0JEEpdSGA9wFsA3Ct1rpKa70DwKsArlFKjYxrBwUhxohNCIITsYnEQgRaElDnfl4E4CSAq7XWp2zNvwBQCeCZePRNEOKB2IQgOBGbSDwkBy0J0FrvAYsO+ms7CKBVbHskCPFFbEIQnIhNJB4i0JooSqlU8PdPBZCilGoJwKO1ro5vzwQhPohNCIITsYn4IgKt6fIIuNyHoRLAUgCj49IbQYg/YhOC4ERsIo7ISgKCIAiCIAguQyYJCIIgCIIguAwRaIIgCIIgCC5DBJogCIIgCILLEIEmCIIgCILgMkSgCYIgCIIguAwRaIIgCIIgCC5DBJogCIIgCILL+F8+MMVbeE/kSAAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x1a23823748>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "def plot_gradient_descent(theta, eta, theta_path=None):\n",
    "    plt.plot(X, y, \"b.\")\n",
    "    for iteration in range(n_iterations):\n",
    "        if iteration < 10:\n",
    "            y_predict = X_new_b.dot(theta)\n",
    "            style = \"b-\" if iteration > 0 else \"r--\"\n",
    "            plt.plot(X_new, y_predict, style)\n",
    "        gradients = 2/m * X_b.T.dot(X_b.dot(theta) - y)\n",
    "        theta = theta - eta * gradients\n",
    "    plt.xlabel(\"$x_1$\", fontsize=18)\n",
    "    plt.axis([0, 2, 0, 15])\n",
    "    plt.title(r\"$\\eta = {}$\".format(eta), fontsize=16)\n",
    "    \n",
    "np.random.seed(42)            # 초기 난수 생성값 지정\n",
    "theta = np.random.randn(2,1)  # random initialization\n",
    "\n",
    "plt.figure(figsize=(10,4))\n",
    "plt.subplot(131); plot_gradient_descent(theta, eta=0.02)\n",
    "plt.ylabel(\"$y$\", rotation=0, fontsize=18)\n",
    "plt.subplot(132); plot_gradient_descent(theta, eta=0.1, theta_path=theta_path_bgd)\n",
    "plt.subplot(133); plot_gradient_descent(theta, eta=0.5)\n",
    "\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- 왼쪽의 알고리즘은 최적점에 도달하겠지만 학습률이 너무 낮아 시간이 오래 걸릴 것<br>\n",
    "- 가운데는 학습률이 아주 적당하여 반복 몇 번 만에 최적점에 수렴<br>\n",
    "- 오른쪽은 학습률이 너무 높아 알고리즘이 널뛰면서 스텝마다 최적점에서 점점 더 멀어져 발산"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- 적절한 학습률을 찾으려면 그리드 탐색을 사용<br>\n",
    "- 그리드 탐색에서 수렴하는 데 너무 오래 걸리는 모델을 막기 위해 반복 횟수를 제한<br>\n",
    "반복 횟수를 아주 크게 지정하고 그래디언트 벡터가 아주 작아지면, 즉 벡터의 노름이 어떤 값 $\\varepsilon$(**허용오차**<sup>tolerance</sup>)보다 작아지면 경사 하강법이 (거의) 최솟값에 도달한 것이므로 알고리즘을 중지함<br>\n",
    "- $\\varepsilon$ 범위 안에서 최적의 솔루션에 도달하기 위해서는 O(1/$\\varepsilon$)의 **반복**이 걸릴 수 있음. 다시 말해 허용오차 $\\varepsilon$을 1/10로 줄이면 알고리즘의 반복은 10배 늘어남"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 4.2.2 확률적 경사 하강법"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- 배치 경사 하강법의 가장 큰 문제는 매 스텝에서 전체 훈련 세트를 사용해 그래디언트를 계산하여 훈련 세트가 커지면 매우 느려진다는 것<br>\n",
    "- **확률적 경사 하강법**은 매 스텝에서 딱 한 개의 샘플을 무작위로 선택하고 그 하나의 샘플에 대한 그래디언트를 계산. 매 반복에서 매우 적은 데이터만 처리하기 때문에 알고리즘이 훨씬 빠르고 매우 큰 훈련 세트도 훈련시킬 수 있음<br>\n",
    "- 반면, 확률적이기 때문에 이 알고리즘은 배치 경사 하강법보다 훨씬 불안정함<br>\n",
    "비용 함수가 최솟값에 다다를 때까지 부드럽게 감소하지 않고 위아래로 요동치면서 평균적으로 감소함. 시간이 지나면 최솟값에 매우 근접하겠지만 최적치는 아님 (그림 4-9 참조)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<img src=\"./images/figure4-9.png\" width=\"40%\">\n",
    "<center>**그림 4-9 확률적 경사 하강법**</center>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- [그림 4-6]처럼 비용 함수가 매우 불규칙할 경우 지역 최솟값을 건너뛰어 전역 최솟값을 찾을 가능성이 높음<br>\n",
    "- 지역 최솟값에서 탈출하지만 전역 최솟값에 다다르지 못하는 딜레마를 해결하는 한 가지 방법은 학습률을 점진적으로 감소시키는 것<br>\n",
    "시작할 때는 학습률을 크게 하고(수렴 빠름, 지역 최솟값에 빠지지 않음) 점차 작게 줄여서 알고리즘이 전역 최솟값에 도달하게 함<br>\n",
    "- 매 반복에서 학습률을 결정하는 함수를 **학습 스케쥴**<sup>learning schedule</sup>이라고 부름"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "아래은 확률적 경사 하강법의 구현 코드"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 79,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAY0AAAETCAYAAADKy1riAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvNQv5yAAAIABJREFUeJztnXmYVNWZ/79vd/UKdDd0s0PT7LIJaisiyC6g4BoXRuMWE2LMTJLJ5JcxJmYcMzGT+IyOMXEicYlLnCQmxg1RQW0VBKVRlGYVVEBla2j2bqC7z++Pb9+5t24tfau66lZV9/t5nnqaqnOXU5db53vf5bxHjDFQFEVRFC9kpboDiqIoSuagoqEoiqJ4RkVDURRF8YyKhqIoiuIZFQ1FURTFMyoaiqIoimdUNBRFURTPqGgoiqIonlHRUBRFUTwTSHUHvFBWVmYqKipS3Q1FUZSMYvXq1bXGmO6JPGZGiEZFRQWqq6tT3Q1FUZSMQkS2JfqY6p5SFEVRPKOioSiKonhGRUNRFEXxjIqGoiiK4hkVDUVRFMUzKhqKoiiKZ1Q0FEVRFM+oaCiKoiieSZpoiMgVItIrQlsnEbldRPKTdX5FURQl8SRFNESkH4C7AYQVDQAPALgNgIqGoihKBpHwMiIisgjA6QBKI7R/G8B6ALsTfW5FURQluSTc0jDGzDXG9AbwjrtNRM4CMBXArxJ9XkVRFCX5+FawUERKQbG41BhjRMSvUyuKoigJwpfsKRHJAvAIgH8xxtR53GeBiFSLSPXevXuT20FFURTFE36l3J4C4DQAD4vIGhFZA6APgLdF5N5wOxhjFhpjKo0xld27J7QcvKIoihInvrinjDHrAZQ7PxORzwCca4w54EcfFEVRlLajk/sURVEUz6hoKIqiKJ5JmnvKGDO1lfaKZJ1bURRFSQ5qaSiKoiieUdFQFEVRPKOioSiKonhGRUNRFEXxjIqGoiiK4hkVDUVRFMUzKhqKoiiKZ1Q0FEVRFM+oaCiKoiieUdFQFEVRPKOioSiKonhGRUNRFEXxjIqGoiiK4hkVDUVRFMUzKhqKoiiKZ1Q0FEVRFM+oaCiKoiieUdFQFEVRPJM00RCRK0Skl+N9oYj8SETeE5G1IrJSRKYm6/yKoihK4kmKaIhIPwB3A+jl+LgcwHEAk40xYwBcD+BREemejD4oiqIoiSeQ6AOKyCIApwModX5ujNkIYKPj/SYRWQ1gHIAlie6HoiiKkngSbmkYY+YaY3oDeCfadiLSCUAlgM2J7oOiKIqSHFISCBeRYgDPAXjCGLMtFX1QFEVRYsd30RCR4QDeBvCSMeb2KNstEJFqEaneu3evfx1UFEVRIuKraIjIHADPArjZGHNPtG2NMQuNMZXGmMru3TVWriiKkg4kPBAeCRHpCeB+ADPVJaUoipKZ+GlpXAfgv1UwFEVRMhffLA0AwwDMEJFvuT5/wBjzgI/9UBRFUeIkaaJhjJnqev+NZJ1LURRF8QetPaUoiqJ4RkVDURRF8YyKhqIoiuIZFQ1FURTFMyoaiqIoimdUNBRFURTPqGgoiqJEYcUK4Be/4F/F38l9iqIoGcWKFcCMGcCJE0BuLvDaa8CECanuVWpRS0NRFCUCVVUUjKYm/q2qSnWPUo+KhqIoSgSmTqWFkZ3Nv1OnprpHqUfdU4qiKBGYMIEuqaoqCkZHd00BKhqKoihRmTAhM8RixQp/xE1FQ1EUJcPxM2CvMQ1FUZQMx8+AvYqGoihKhuNnwF7dU4qiKBmOnwF7FQ1FUZR2gF8Be3VPKYqitIGOVmYkaZaGiFwB4G1jzC7HZxcA+DmAPACbAdxkjNmXrD4oipI6/EoBTSVespZScR1OngTefTc5x06KaIhIPwB3A7gEwK6WzwYBuBPAecaYWhGZD+AhAJcmow+KoqSOjlKzKVzWkvN7JuM6RBKhrVuBV18FXnkFeP114PDhtp0nEgkXDRFZBOB0AKWupu8D+JUxphYAjDF/EpF/EpGhxpiPE90PRVFSR2uDaXvBylqyRMGdtZTo6+AWoX/7N2DbNorF1q3cZsAA4MILgb17gSVL4j9XJBIuGsaYuQAgIlWupqkAfur67LWWz1U0FKUd0dpg2l5oLWspkdehqQl48kmgoQEwBqivB269FejUCZg0CRg8GNi4Efj8c+Cpp+I/T2v4mT3V0xiz3/XZLgD9feyDoig+0JFqNkXLWmrrdfjiC9vltHQpsM8RAc7KAnr1Ampr2e4XfopGfoTPTbgPRWQBgAUAUF5enqw+KYqSJDKlZlOyieU61NcDb71FEXj1VWDdOn7eowcwYgSwaxewfTstl+Zm4Msvwx+npARobASOHEnMd3Dip2jsE5Guxpg6x2e9AGwPt7ExZiGAhQBQWVkZVlgURVHiIV0yu4wBampskXjrLeD4cbqyRo0Chg4Fdu4E9uzhKxKdO3M2+MGDfH/0KN1ZycBP0VgGYDqAvzk+mwrgJh/7oChKByfVmV1WgPrVV/nauZOf9+rF+ER2NnDsGPDBB5GPkZcHFBYCdS2P4PX1we3NzXwlAz9F4x4AD4rI68aYOhG5GsAezZxSFMVP/M7sOnGCQmVZE++/Twujc2ege3eguJgWwq5dkY8RCHD7Q4coBsePB1sSzc08pkV2NjBsGLBhQ+K/j2+iYYx5X0TuAvC6iBQAqAHwDb/OryiKAiQ/s8sYYMsWWyTeeIOxBRGgd29aE0eP8jOvMYfGRloflvUgws8scnKAIUPozqqvp8tr48bEfi8LMSb9wwWVlZWmuro61d1QlLQnXXz10UiHPia6DwcP0s1lZTp99hk/LynhX8tCSBQ5ObQkRo6k1bFmDVNtrXNYmVVffimrjTGViTuzFixUlHZDqn31XkiXPrY1s6upCVi1yo5LrFjBATs3F8jPp3uoqQk4cCD6cXJyWPIDsPeJtN3w4cC4cZynsWYNLQkruwqgq6tnT/578+bImVVtRUVDUdoJmTALOxP6GIkdO2hFWHMmLEHo1InuIoDf6cSJ8PsHAnw1NNifWYIBBAtGVpZtNWRnA337Aps20e1k0bUrhcIY4JNPGGDfvx8YPRqYPp3Hfu21tn/vkO+R+EMqipIKMmEWdib00eLoUeDNN2lJvPwyB22A/XYO8EePht8/KwsoKLDbGxsjWxLZ2cAppwATJ3K7xYvtrKqmJrq7unSheBjDuRp1dTz26NHAeedRjDZtAj78kK/i4oRchhA0pqEo7Yh0iBe0Rrr20RgOtlZc4u23+bSelUVLwsu8h4ICxhicAetwQ2xWFkViyhS2v/8+XU2RBMh5/DFjgNJSbrt+PWeEA4yf9O5N0dmxw7JoEh/TUNFQFKVdE02kdu/mnIlXXqE1YQ3ATvdQNPLyKCZWJlM0kRg+nPGcrCzOwfjwQwbInccqKeHxrH7k5jLg3b8/g+3r1tkT+EpL6aI6dsyOX2RnA6edRotl4kTgyitVNBRFUTzjDrwvXsxB+ZVXgBdf5JM6EHmwdxMIcFsrFhFpPxEO9uedRzFYs4ZCsd9RfS8nBygqCg6Yl5RwJniXLnQ/rV1LUQCAsjLGT/bvt8uel5QA55zD18SJwJlnchu7H5o9pSiKEhWnZfHGG7a7qL6eAhLOzRRJMLKy+PRuiYRzboR7v6FDgdmzOQlv7Vqguhr4zW/s9uxsxhmamjg/4+RJisuYMZzdvW8f91u+nNuXlVFUTpzgeWtrKRKXXUaBOOcc1qPK8nn9VRUNRVHaDZZlcfw4B+S8vGA3U2txCRFaE5ZIRCvHMWgQMGcO3UTr1nGlvN/+1haSrCxaDI2NFKymJsYkhg9nWu6uXcyGevttnrdrVzsmAtB1dcYZwFe/SpGYMMFOqU0lKhqKomQ0jY3Ae+8BTz8NPP54cB0my7UTjUDAtiCMCU6DdTJgAC2Jvn05R2LlSuB3vwsOeufm2llSzc20DIYM4Tm+/JLismsXBaWoiC6qpiaeNyuL62JY8YgzzqC4pBsqGoqiZByffQb8+c/A3/7GeEGkgT4c7kl0bpeTRd++wKxZXNxoyxbgnXeAhx8O3tcZCDeGriRLAPr1Y2rsjh08Z2GhHWBvbmamkxWLmDiR7i1rvkc0Up19pqKhKIpvxDvgHTlCgXjqKT7hO7OOYiWSi6pnT1oSp5xCUVqxAvjjH4Mn6+XmBh+jvJwvYxjotqrOute6yMlhVpMlEhMm0K0VK+kwo15FQ1EUX4hlwGtu5qzrRx/lGhOxlsTwkg1VWsrspnHjuELe8uXAX/4SPGPbypayjtW7N91NjY20Pj7+mK9AIPR8kyYBF11EoTjtNFtw2kI6zKhX0VAUxRdaG/DWrmWMYMkSlsWIZREht0iEE4ziYmDmTKCykplKy5cz7fZPf7K3yc62j2XN0s7P5+S75mZg2za+nC4uEWYxTZzIFfaOHQMuvZRikWjSYUa9ioaipIBU+6VTgXvAGzECuP12DtwbNthZQ0Drvn0vIpGfD1RU8LylpYxJLF1KN5eFM101J4dzJLp352zrmprgWk8WgQAwebIdizj77OSV7HCTDmuve5rcJyK/A/BNAH2NMV+62oYDWAvgf4wx301GJ3Vyn9KeSAe/dDKJJIh79wI//zkD2LW1wQHo1txJXtxN+fksy3HuuTz2u+8yq2rfvvDHycmhQPTpw882bLCzrZzblZZy8p0x3GfJEp4jE0jl5L4VoGicBeBZV9u9AA4BuCNx3VKU9ks6+KWThVMQc3KAH/yAk9zefdcOEoejNUEI156Twyf9adMovu+9Z6+QF46CAhb3KyoCPv2ULrAvvwyOl4jQ2pg2zQ5a9+/fMS3DSHgVjZUtf4NEQ0TmAjgfwLeNMVFuCUVRLNLBL50M9u8H7r/fnifR1AT8x38k7vjZ2cBZZzEuUVxMMVq2jIN5ODp3pgAUFjLQvXUr18AIx8yZwK23AuPHcz83bV1/oz3hufaUiOwD8IExZmbL+xxwydYTAMYZY2IIW8WGuqeU9kZ7eHKtq2Nm07PPMlbw+efJOU8gAPzjP7JQX1UV02HDDVtFRRSJnBwGq7dvD92uZ0/GIyoqKHAnT7ZPF6FFMtxTsYjGIgATAXQ1xhgR+QGAuwHMNMZ4XupDRC4G8GMAhQCaAPynMeZ/o+2joqEo8ZMogaqrY8mLV18FXnqJLh4nXov+RUKE7qNZs5jG+sIL3o6XlcUSHM7YhfX50KG0Iix3U+/ednt7EO7WSHXBwpUALgAwXET2A7gdwLMxCsYoAP8OYJYxZo+I9AdQJSJrjTFh8hQURWkLbQm6HzhAkXj9dXsRomiDeDyCMXy4PaFu82b27777ws/S7tqV29XXc1sraN3cTMEoKADGjmU9qGnTWPG1oCDyudXlFB+xiMaKlr9nAZgMIA/Av8R4vrMBvGaM2QMAxpgdIrIMwHDQ1aUoSgKJJeh+8CBFoqqK1kRNTdssh3BUVADnn8/B/bPPmIn04IPB6bYWRUUs+JeTw3hJXR1F0E1ODvDYY8BVV/lf8bUjEotovAugGcBNACYBuNsY80mM51sN4A4R+V9jTLWIzAMwHsB3YjyOoigRcLpdogXdDx1iIPmNN2hNrFnDp/a2upmc9O1LS2L8eC5fumQJ8Ic/BBcVtCgtZb2mI0eY0XTokF0uJCsL6NWLabvG8Lvcdx8tjPbsXkpHYlqESUTWAhgNYBeAYcaYwzGfUOQaAA8AqAeQA2C2MSZqwEJjGorijXDuKIAicuaZDPxWVVEoVq9OvEh0786YxKRJdG+9+ipTYcMtY1paysD04cNcQc9Z4ykvj4sYzZzJ9SOamoC77mLAvbmZmVQ/+xnwox8lpt/tlVTHNADgPVA0fhSnYAwBcAuAxQD+DmAGgFdF5FJjzJuubRcAWAAA5eXlsZ5KUdKeZARi3e6ohQs5MFdVcfZ1U5O95rVV0rstgtGlCy2J6dPpYlq8mAHsP/4xdNuuXSkUR47QYti3zw5eFxcznfaCC4DLL2d9J2tW+IoVFCJrMaWsrPaVqhwLaRG8N8Z4eoFWwVYAq9BiocT6AoPpV7s+uxLAm9H2O+OMM4yitCfeeceYggJjsrP595132n7Mw4eNufdeYwIBYygFfGVl8RzOz+J95eYGH3fECGO6dg2/bXGxMeXlxnTrZoyI/bmIMb16GTNvnjEPPWTMvn3Rv9ddd/E6WeecNSsx1yvTiOeeAVBt4hiro71isTR+AGAggGtaOhMPwwD8xfXZBwCK4jyeomQkiZgVfvQo6ym98Qb3X7WKWUfWinVWcNla6jQeCgroajr/fFoVTz3F81nH3bDB3rZLF65PfegQM5sOHuQrEAAGDmTK61VX0TLJyfHeh6lT6Y5qbuax7rijY8Yw0qWSQFTREJFuAGYDOBXA/wNwjzFmZbR9WuEVALeLyM+MMY0iUgbgPgC/a8MxFSXjiGdW+LFjFAkrJvHeexSJ7GygWzd7BTpjwmcjeSEvj26iCy9kxdYlSxhHWLo01I2Vn0+ROHqU5cQPH+bLIhBg1dqvfc3b4kLRsPZv63EymXSpJNCapTEbwFMA9oA1pm5t4/kWAPgZgDUi0gzgCChEf23jcRUlo/BSrbS+3haJqirWbzp5kiLRqxef7Ovq+OS5d298/bAWB7rwQqbDLl1KofjXfw0ViYICbn/sGMWpoYGvLl14jDlzGK+45x57CdM9e9o+0FdV2WLY2Ni+anXFQjpUuAVizJ5KFZo9lVmkRbAuA6mv56p0lrvp3Xf5VJmdzaJ5TU1MRY1lnQk32dmcdX3hhcDIkTzP4sWszWQFxi3y8hh0bmgIFpCyMuD007nA0NVXM8BtkYwKvu29KnAySWkZkVSiopE56A/cOw0NwSKxciWvW1YWs4esGkpHjsR/jqwszqKeO5cD/bJlwKJFXLfaLT45ObQKnKmvIpxrcfbZwBVXAPPmsQBgNJLx0KAPIvGRDim3ihKVdAnWpSMNDbQerJjEypWMPVgD+xlnsJ7Trl0skxEPIsCgQUxdnTiR53jhBeDee0NLcwQCtC4sC8NyfQ0ZwsH5qqu4bkReXmx9SEZ5Dut4VkVbvadSh4pGmpJJT1ZeZyB3NI4ft0WiqorXqaGBA/uYMbw227dzren16+M/jwgtgPnzuSzp888zAH3//cHbZWUFu6AaG/l/dMopnAdx+eUUrkAajgpqwaYPaXh7KJn0AwnX13QI1rWVeET7+HGmvVrupnfesUVi7Fi6iHbvBj76yH7FQ+/enLD30UcUAWPocnrhheDt3DO9m5sZzB43jn258ELGNzKhXpNasOmDikYakkk/kHB9/dGP0re/XvAq2idOhIpEfb0tEldeyXjEe++xrtOaNfH1p6yM4jV7NhcSeuEFFhN0Wg3uIDZAwSgqYgrtJZfwOw0fnplpq2rBpg8qGmlIJv1AMqmvXokk2idOcLU4KyaxfLk9aW7sWOC66xgTWLYMWLcufpEoKWE84vzzWeTvhRf4+quHxPTu3RmHmDeP62UPHBifSKSbezRd0k0zBWOStyiWZk95IBU/oHT70UYjk/rqBaelEQgAN97I9aSXLbPXcBgzhoNy9+78/J13whfl80LnzsxOmjOHJcAXLeJMa2cWUyT69WNfZ83iinT9+nn/jpH+zzLJPapwrk5NDbB2rf2qqeFsfEBTbn1Hf0Dti2iD5cmTDCS/8QaXMH3/fX4G0Pc/dSrnNqxaBbz4YvwT6goKGHCeNYuWyiuv0DLxMot7yBDuN306LYoePWI/f2v39C9+YRc31Gqy6cPx43yYcIrD2rWcY2NRXMwHGut1yy2acus7mRRfUIJxC4R7sHzlFaaTWu6mZcvsORGjRgELFtAC2LIFePxx4IEHwscOvDBiBCfDiXDG9apVPF80nMufTpnCGlDOiXTx0to93R5djplEczNTr93i8PHH9tya3FzeU9OmURxGj+bffv2C3ZG33JL4/qlotIL+gDKTcE/Tr79ul9eur+daDZYLaORIxiQmT6Zw3H8/y4r/9rexnzsnh8cbMoSC88UX/MH/8pfR9wsEgMpKWhFTpnAg79LF23eNxT3Y2j2t8QP/2LMnVBzWrbPdoADn3YwZw5RoSxyGDg0u+mgJzQsvcP+aGv5NBuqe8kB789knm3S4Xk4XiwgX9Nmxw/4xijCjaP58mvS/+AWznOKpBpudzR/xOefwWFVV/NFarq1I5OXx+kybRrEaPz76mtbhiNd9mg7/Rx2Jo0c5iDtjDmvXUjQsune33UqWOIwaxZiXhTG8jy1RsARiw4Zgoenfn/u+/LK6p1KCLkDvnVTGgJqagA8/pKtp0SLblDeGYnDttSyJ8cUXwMaN7Nvf/x77ebKyWNzvzDOZDvvWW/zRbtwYfb+cHF6bKVMoEpWVvEZtIV73qd7TyaGxkValUxjWrmUihfV8XljIAX3u3OD4Q8+e9nGMYebcihXBlsO6dcGVhHv3tl2po0bxNXIkH16A5KRXq2goCcXPGFBTEye4WTGJt96yMkZoWVx8MZ/mp00DPvgA+NvfuFJcrMa1CH3F48bxh718Oct8fPJJ+G2t43fuzKc/a03rJUsYvI5Gol1NSnIwhg8fbnHYsMFOaMjK4n142ml0fVriMHAgrVOLvXt5jKefDhaIujp7m7IyWh/XXUdhGD2af7t18/d7AyoaSoJJ5iDW3BwqEgcOsG3oUE6mmzyZZcOXLgWeeIJPa39xL/vlgV69+MPs1o1ZVJ98QrdANHr0oBVhWRKjRtllRLyIQDxWmsYfks/Bg+FTWp2Dep8+FISZM21xGDGCa45Y1NVRDJYuDXYtObPwSkp431x5pW05jB4dX5ZcstCYhpJwYnlajrZtczN/oFbtpjfftH+ogwfTgpg0yR7Y//AHVoWN55YuLeWPvEsX1oHavr314/TrFywSw4a1zR2gqa6p5cQJuhjdgWnnw0JRkR1vcMYfnE/8hw7xHnK7lb780t6mc+dgUbD+3adPYl1KWuVWyQi8+svdT9ZLlvBHaZXlePNNTnYDmEFy6aWcKW0N7E89BTz8cHwiUVLCYxYU0IrYubP1FNhBgyhukydTKAYMSOwPXF1N/tDczIcLtzhs3mxXAs7JYSHHc88NFofycvv//NgxuqOsjCVLJLZvt89VUMCHkZkzgwXCeZxMQy2NDkyqM2juuotP1tbch8JCOwOkd28+dc2YwR/Zhg2s3rphQ3wi0bkzM0pyc+mLrq1tfZ8RI2xL4txzGURPNqn+P2lv1NaGT2l1rlFSURGatTRsmJ2kcPw4LRC35eAMblvVgt2Wgzt+4Te6CJOSMFKR5WQMLQTL3bRkiR24FmGtpTlzWDPnnntC13+Ihbw8Ck9ODtMarfNEwppIN22aLRLdu8d+Xh30U8OxY7y33Cmtu3bZ25SWhk9pLSpi+8mTzHyyhMH6u2WLnYmXnU1BsYTB+jtkSHqWlG8X7ikR6QGuN346gCYAq40x1/vdj46OH1lOxtAysESiqsoO+pWXc55Ely6clBQIcNvvfCe+cwUCHOSzsxn3OHoU+OyzyNtnZTEbasYMupsmTaLLqi1oyZnk09TEQdwtDlu22E/9+fkcyGfPDo499OrFh4OmJlYLdgala2ronrLm1mRlMW42ahRXLLQEwmmBdFR8FQ0RyQewBMCvjTHXiEg2gOv87INCkuE/N4ZmvFMkrMlL/frxRzxqFN9bYrJtW3znyspiSQ0R5q0fP864RGv7XHstcM01HMydk6ZiJZxFoSVnEoc1T8EtDuvXc40SwF4Wd8wYrlVuicPgwXx4sGIXNTXAY4/ZloMzLRage2r0aFYGtiyHU06JfaJlR8FvS+OfACw3xjwMAMaYJgCP+tyHjCDZbo5EpGoaw6czK3BdVcVFhgD6/887jz9qgD/2JUuAJ5/ke/cCQV7o0oX7HT3KgXnfvsjbBgKcfFdYyP41N3Pf4cPZr7YQyaJIRSC7PbjDDh3igO5Oa7WSIABaCSzAZ4vDyJEc2K0y4OvWMe7lnCXtrDzcrx8FwYqTjR7NuFVbHh46In6LxhUAvu3zOTMOv9wcsc4KNoY+X2ueRFWV7TPu04c1kyoqOKCvX89Z2dY8CvdSo14EIz+fA31DA7d3zoR1k5vLxYbmzmVM4owz+Jn7WiZiII9kUfg9ZyLT3GEnTvAhwx2YdlqbnTtzMP/KV4LjD2VlvAd27aIoLFvGJW2toPShQ/YxevWiKNx0k205jBzZdvejQvwWjREA9ovIQgBnAWgA8F/GmKd97kdakw5ujhUrKAxDhjCIbFkSVq55aSmFYto0xhLWrQOee87OfsrJCQ5kx1Md1nJDhCMvj7WaLr6YA/TYseGzVJIxkEezKPwsz5EO90k4jGHaqVscNm2yYwaBAK2+CRNYAsMSiPJyPmDU1trupL/8xf630/ooLaUofPWrwUHp0tLUfO+Ogt+ikQ3g9wD+3RizQEQqALwsIjuNMUFZ8iKyAMACACgvL09YBzLBnE9Vvr4xDBA+/DBw9912xgjAp7dzzqHbaft25qbv28fBAKAbyOknbq1YH8BB3hhvgpKfT5H4ylcoVCNHel/bOtEDebrMwk6HeR3794eKQ01NsFVYXk5BmDfPFofhw9nnAwdsa2HJEtu15CzkV1xMMbj88tBZ0pk61yGT8TXlVkR2AJhrjPnI8dkCAMOMMT+ItF+iUm4zyZz3Im5tFUBjmLnkjEm4l4gUYe2cI0fomjImvniEdSz3fpGOlZvL73TVVfw/Gzo0vQaIdHn48Ksf9fWhCwDV1ATPcu7aNXSm9OjRHPQPH7ZnSTvTWZ0LCHXqZIuC03Lo2ze9/u8zifaQcvsamGr7keOzbABxFKSOnXQ158PR2tNxPAJoDNNQnTEJq0RCjx6MCcycSUvijTe4vTGsHOu0OuJ9zjAmcmyjsJDpjAMH0t1w2WXxncMPvFx7vwbzRFtRTU2ctObOWvr4Y/v/LS+Plt7MmcElNfr0oUtxwwbu9/zzLI1SUxMct8jPZwB6+vRggbBcU0p647do3AXgJRFZY4xZIyL9wMD4VX6cPB3M+XhxD0JeBdASCUsorBIHZWUszX3uubQiPvqIS5gCzFLq04eug/r6YMGIlezs4P2tgadTJwarr7uOk/r69In/HH7T2rXPBIus/0ZxAAAgAElEQVTWGGa6ucVh3Tp7TRERpq+OGcMCepY4DBnC775pU2hQeutW+0HAKsUxYQLwjW/YAjFoUGpnSSttw1fRMMZsFpGvAlgoIsUA9gP4rjEmSWtMBZMMX7QfT5ThBqFIArhtW/A8CWuCW7dudDONH89MkzVrgJdfZltZGdMRBw6ku+Hw4eiZStEIBDigWAOHJRiWSNx4I33bZWWxX4N0cAcBrT98+G3RtnZtjhwJn9LqLKXSsyef9r/5zeCU1txcTpxzB6WdS49ai1CNG8c5MM5Z0s7V5ZT2gZYRiRHnDxTw54kyUvXTFSuAZ5+lu+Dzz9mvTz/lPt26AaeeSn/ygQN0MVnpr/37UyAOHuT2znTFWMnNZZaUO5jdqRNF6mtfY6HBtqQ7puOTe7SB2s/+us/1+99TuJ3iYN0TgB03cMYexozh/fLJJ6H1lTZutJMaLMvDXV9p+HDeg0r60R5iGhmN+wd6/fX+PFE6n2xzcug3vukmns9aCKhrV3smq5XRUlXFtuHDOXdh/34+IX7+eetrQ4RDhP5oa51twF5ju1MnitTXv86yC17WtvZKOsaiosUSkp1dZU1mW7sW+PWvbXdSfT3jQYBdI+nMMynczpTWzz+3heHRR+1Z0s4U5wEDKAznnx88S7qwMLHfRck82p1oJNON4R68gOTHSL74gsIwcyZXjNu/H7jzTloQI0bQYqitZWbKsmUc2MeOpQ+6ro6Dwdat9D/HSnY2RaKhwXY5WQNUYSFF4qab6JJIZsmFTIxFJSpAXVcXfgEgZwFGKwMtEABuu41JBMOH876wLIfnnmNV4XXrgmdJ9+1LQZg2LXiWdCJFX2lftCv3VLLdAuGOD8QmUq2J2pdfBsckPv6YnxcV8cmxsJA57Js382nfKpdhxSqsQaU+jny07Gwer7ExNPhdUMAB5aabaGE5VyTzg2Q+DKRDvOT48fAprc4U6OLi0JTWUaPY/2ee4T1y9KjtWnIKS48eoZVZR43SWdLtHS2N3gp+rHzWlgEmnOhUVAALF7Lkxq5dttuoSxcGEvPy+LkV0C4o4HknTuT3XL0aeO+94KUnvZKXRxE6etS2nCxyczm43HgjXU5+i4Rf+B0vaW5mjMEtDps320Kdm8unfWcJ7zFj+H+/fn1o6W5nDa5u3YKFwfoba+KB0j7QmEYr+OHGaIvbweneamgALrooOIPFKqhXW8uB4IMP+HQ5aRKzWoqK6IJ6+217HkUsFBbyZVWFtV5AcGpsdjZwxx0dY6nRZMZL9uwJFQe3e2jQIArCZZfZ4tCjB0XEEoYlS/jXKgYJ8F4YNYpJBk6B6NlTJ8IpyaVdiUa6lHewWLGCcx8KCuh2eukle2A2hsXZAgG76J8xTJk9eZI//JwcYNYsDjgvvxz7fImiIp774EGK1LFjdm2o3Fy6u669Frj5Zg5QsRT2SweXTiJIxIOG5RJyzndYuza4FEZZGQXhpptscRgwgJalOyjtdEkVFjL19fzzgy2Hfv1UHJTUkNHuqba6ipIx6O3Zw7Wt//Qn4O9/D14OcuBADt47d9ruoF69uDCRMRSJsWPpboqHkhKKhDUpz0lODtMlr76a5aXDFXXzek3SMQW2LXj93o2N9spuTgvCuexnQUFoSuuQIXQfOl1KNTXBi0Tl5dEl5XYtDRigs6SV+NGYhoO2DFyJHPT27qVIWIHrdS3TFK2AshtrsfrJk1m2Y/ly4H/+h4NPtKqu4ejWjU+i+/fbFoRFIECRuuoqikTv3vF8u/D4ETtKJcYwa80tDs7Fe7KyOKHNKQ7Dh/Oecq8nvXWrnaKck8Pt3PWVrIWDFCWRaEzDQVt80W3Zt7YWeOstu3ZTTQ0/z8tjKYzycrqinIIRCAD//u8MKn/0Ed0QP/xh6yvNuenShYNVQwMHr/377VLR2dl8Kr3sMrqbBg+O7dix4HTpZGezNMmKFfEJb6rdXAcPhk9pdSYW9OlDUZg5k39HjOD3//hjWxz+9Ce+t/7fs7NpYZx6KvAP/2ALxNChOktayWzU0mhl3337KBJW7SarFHhuLoOOjY2MSVjupTPPtC2JQIBZUdZELOcTpxe6dmVcYu/eUEtChDO7L76YQfKRI/31ca9YATz+OAWwsTH8dWxNEPx0c1kWgLuMt3OSY1FRcLbSyJEU6i++CLYcNm603YsiDGaHmyXdXjPOlMxBLQ0HbQl6R9t3/35bJKqqaBlYgtCjBwOatbUcNGprue83v0mRGD+eT64PPgj8+MccYLysK2FRUsJsKUsk6urCp9JmZ3OC3223eT92orGKJlpzOtwWmxdBSEbmkrUutNty2LTJtgKsQnrnnktxGDWKrr66Ort89yOP0B3ljA2Vl1MYrLXOrYlwOkta6UhkrGgA3tJfW3vaPXSIPv9FizjgWJkrgQAFoqiIQnDyJAu/TZpEgZg8GTj9dA5Ed9zBUg07dwYvRNQahYUcrKyYxIEDdn0ogJbMrFl0awUCHKysQXjaNO/nSRbRMo+8CEJbM5dqa0PFoaaG/08WFRUUhosv5kDfsyfde1aF1mefBX7+8+B9+vThtjffbFsOI0fyXlCUjk7Guqe8EO5pd8QIFnW77bbwgWprEAM4wFiupsmT+WQpwlXrFi5kENs567Y1cnKiWx5lZRw4r78eOO+80CJwqfb/hyNSn7y6nrx8p2PHaAG4U1qtVGWA2WDOyXB9+/Ih4LPPggvwOUW5e/fws6S7dm3zZVGUtKBDZ0/df391zAOmM8tHhBlEzpXGwtG7N10/kyfbq8W9+y6XP62qCp592xoFBbalEi4zqksXfp9rrmG58E6dvB87E4hV5JqaWIbbLQ5bttgprfn5fOq34g4VFUwO2LnTdi3V1ARPmrSKOboFonv3JHxpRUkjOqxojBhRabZtq/YUMD10iDOmq6o4sW7jRrutc2cO3k4Lw73c6IMPsiLsnXfyPHv2xDbzuqgoeKa1u234cAawb76Z1kRHxBgO8u6spfXrbXEVYfaRJQ6DB/P//uDB4GVDnRloXbqEprKOGsUHAZ0Ip3REOmwg/PDhyP7xw4fpKnruOT5dbtxoLyvaubMtCiK0HCxXU14eg9xTp7JMw4MPcmbvt77lPcMpEOA5Dh+2Z2s716bo1Ilxj8svB+bPZyC9o3H4cKg4rF1rpwoDnOA4ZgxjS1ZRxmPH7MV/HnkkOMupoIDWxqxZwQLRv7+Kg6Ikm4y0NH7+c/qzq6qAVavCWwJW+qslEuecw8ykpiauWvfAAyzNsWuXd5HIyuKAFmkJ1Lw8rl520UVc16C8vE1fO6M4cYL1ktzi4FwbunNnO+Zwyim0vBobuY1lOXz6afAs+nCzpC2XlKIo0emw7qlhwyrN2WdX4403mDNvWQ75+aHlMmbMAH7yE862torzrVzJyVeLF9tzKrwQbtEhJzk5HMjmzmXwesiQ9vuka8UnpkxhkNktDps22UH+QIBuOGuuQ7duvH579tjisGWLfU0DAVoY7pjD4MFsUxQlPtqdaIhILwA3GGP+M/p2lUakGjk5dmZTSQnTXwcMAB56yJ5g9uST9Iu/9BLw6quc8xALzuwpN9ZqaLNnAzfcwNm+7VUkAHsFwOefB+67L7x1VV5uz3Xo3p0WwIEDFBGr5LcVQ8rKorC6LYehQ3ndFUVJLO1KNEQkAOBZAJOMMVGXghGpND16VGPKlOD016Ymrn395JMUiG3bQmdOt4azJHjoeTnbd8YMWhLjx7fP+kANDXZKqzNzKVymmQivx8yZdtzBmiXtDP5bs6SdAnHKKTpLWlH8pL2Jxt0AqgE82JpojB5dadaurcaBA3STvPMO15VYsSKyVRD5vLZ7yp05BfDJecoU4Lrr+Lc91QlqamJFVndK68cf266ivDy6lEaPZvntggKWP3nyyfDi2r9/cKaSNUu6vaUPK0om0m6yp0TkEgB5xpg/i8iDrW2/ezddIFYF2fjPGywSxjBzZ9IkzpWYMye2J+F0nGwH8Hvt3h0qDuvW2TEgEcYMRo/mWg2dO9ONtHs3y2c8+yzjQRalpXQJnnEGU4WtWdLFxan5joqipAbfLQ0RGQLgAQDzjDEnROSAF/cUjZLYyMriAOr8iqWlwNlns/LoRRcxtz8e0mVNiSNH7AWAnC/n5LYePex1HYqKeD3q6uxSGs76VmVloZaDVZtJUZTMIuMtDREpALAQwI3GmKiOJRFZAGAB353h6fjZ2RwQLVdLczOfhCsrgSuu4CtRg18ylwkNx8mTwNNPs4RJXh4H+rVrmaJqUVjIQX7OHH5PEVoLn37KOSmvvWZvW1JCMbjyymCB6IhzSRRF8Y6vloaIzAbwOwDOik1jAKwF8EdjzN3h9wtvaQQCFAmnr71TJ+C007iuxPz5iV18yEmyLA1j7FLq7tnSzpnsAwbQVVRWxrjLsWOcALd+fXAAu3Pn8LOk+/Rp35lfiqK0s0D4/3UgBveUlbPvHDzz8zkYWhPqBg5MZm+DaWtM48CBUHGoqQkugtivH2MHe/dyUqL131VcHLxdQQED0G7XUv/+OhFOUToqHV40AHsthAsuYIbTiBHp/8R8/DiDy+6UVqsMO0ARGDWKE+c6daIw1tYys8m5BjXA7ztzJsXKEoiBA9tnOrCiKPHTYUUjP7/SfOMb1bjhBrqe0vXJubmZ8QO3OGzebLvQcnMpegMGMAhvBaU/+YSprdZ21kRC5yS4Eye43fTp6ZWtpShKetIuRcML8a6nkUz27Amf0nr0qL3NoEGsk+QMSu/YQRGxSm5Y1Vzds6SHDdNZ0oqikOPH7ZU86+pYrcH973Cf7d6d4dlTmYQVrxg/nhaBO/awZ4+9bVkZS2FMn0732dGjDEZbriWLigoKwrx5wbOkCwr8/naKovhNUxPjmLEM+ta/W6t0UVzMdWO6duVD6ujR/Pfvf5/479GuLY1YAtWNjfYCQIsXA489FlqksKCA4tCzp13IcPduioPzP7Vfv1DLYcQIZjIpipK5GEOPQayDfl1d66t8Fhbag75TAFr7rLg4cmHPjJ+n4SeRUmKNoRXgthw2bLBrJ7lnjg8eTGtj61bOd7Do2ZOCMGWKLRAjR3IOhKIo6YkxrIwQq6vHekWqVQfQ0+Ac4Hv3tis9uwd997/dyzunK+1WNJyT744fB/75nykeNTXBM6B79WJa6qRJdlB6y5bgEhp79zKr6dprgy2I0tLofUjXMiOK0h44eTI+V09dXfiVNS2ysvjg5xzUBw6M/tRv/buwMP2zOdtKuxCNEydYZdVpOaxebT8RNDfzs8GDWU4DsIPSu3bxBdhpr/PnM+310CFOErzggthvhHQpM6Io6UxTE902sT71798fnHQSji5dggf1ESNaH/S7dmWpnXTN0EwHMko0mptZ/tw9GW7TJnvCX04OK9VWVNB9tHcvS39bZTcACsKoUcCFFwZbDn37Ju4pwe8yI4qSKoxhDbR4nvoPHoy+KFpBQfCgXlHBtPvWnvpLStpXhep0IiNEY9s2Drg1Nbw5Lfr0oTBUVlIYdu8Gdu5k7GHrVgarw82SLi9P/pPE1Kn2gk65uXyvKOlMQ0Prbp1I7c4qDW4CgeBBvUcPZg16eerX9VfSj4zIngoEKs2wYdXo1Il+zNpaLvtqYc0Sd9dYGjQotbOkNaah+E1jY3yD/v79FI1IiNh+/lgyfLp2ZdZge/fzpysddnKfVUYkO5spr27LYcgQNUWV9kNzM+Np8Tz1OxM4wtG5c+yDfrdujPepnz/z6LAptwMHclGg4cMzJy1N6dgYw7k7rQVzw4nCgQOhc4Sc5OUFD+r9+3O9+tYEoKREqwwobScjRKNbN/4oFMVvrPIN8Tz1W6ViwpGdHTyoW1UFvDz1awUBJZVkhGgoSluwyjfE89TvpXyDc1Dv18+b26dLF/XzK5mJioaSERhDP388T/2HDkU/dqdOwYP6kCHefP0lJVqOXul4qGgovuEs3xDrU/+BA9HLN+TmBg/qffrYRdtaEwD18yuKd1Q0lJg5cSL6k340ATgRZWX4rCx7QLcG9UGDvD31d4TyDYqSDqhodFCs8g3xBHhbK99QVBQ8qDsLtkUL8HbpommdipLuqGhkMFb5Bq9F2pyfeSnf4BzUBw4ETj/dW1pnpDLNiqJkPvrzTgPcfn6vT/0HDrRevsE5wPfqxbIqrT31a/kGRVEi4atoiEgZgO8DmAMgB0AtgO8ZYz70sx/J4OTJ1tM6IwmAl/INzkG9osJbgLdTJ/XzK4qSWPy2NEYB2ATgdmNMk4hMBPCMiAw3xkR5ZvaH5ma7THOsT/3OQorh6Nw5eIB3F2yL9NSv5RsURUknfBUNY8ybAN50vF8uInUABgL4ODHnYKDWa5E2d1pnND+/Vb7BGtTLy4Fx41p/6tcyzYqitBdSGtMQkd4A+gL4Itp2R44AL7zg/ak/mp8/Ozt4UO/eHRg2zNtTv5ZvUBSlo5My0RCRPgBeAl1VUYs1bNoEXHRR8Gfu5Rj79/dWuVPLNCuKosRPSkRDRCYAeAzAHcaYpyJsswDAAgDo0WMIXnzRFoDiYi3foCiKkgp8X09DRG4E8F0A840xG73sU1lZaaqrq5PbMUVRlHZGxq+nISKngim3k40xdX6eW1EURWk7fidzfhPArSoYiqIomYnfojEMwH+JSI3rdbHP/VAURVHiwO95Guf5eT5FURQlsehcY0VRFMUzWrBQUZSUYwzL+DQ1pdcrHfsUS9+TgYqGosSA34NbJg9asfQ/WQNcshDhXLFEv3JzE3u8//iPxH93FY12SqIGt44yaMVyvEwiHQa3rKzk9MGvV7j+Z0pViQ4rGidPAjt2pH4ASodBSwe32Ae39jhoeX1lyuCmZA4ZIRoffcSKsulAvINbaz/8vLzUD07JGrh0cFOU9kNGiEZ5OXD77akfOHVwUxSlo5MRotG9O/D1r6e6F4qiKIrO01AURVE8o6KhKIqieEZFQ1EURfGMioaiKIriGRUNRVEUxTMqGoqiKIpnMiLlFidOAFu3Bk+YyM/nouEAcPQo/7pnoymKoigJJTNE45NPgCFDgj8bPx5YuZL/njABWLs2uH3mTGDJEv571Chg+/ZgUZk3D3jkEbafdRZw4IDdFggAF10E3Hkn26dPZ20O5/4XXQTccgvQ2AjMnx86C3DePODyy4Fjx4B//dfQ9tmzgalTgYMHgd/8JrR96lRg7Figrg545pnQ9rPOAgYOZPuKFaHtI0cCZWXAoUPAli3297La+/YFCguB+nr2wb1/QQH/WmvI66xGRVGQKaLRpw+nhDc1cZBuauKMP4vvfx/Ysye4+NLAgXb7NdcAtbXB7WPH2u2nn86B02prbLStGADIyWExp5MngYYGbmNZN83NwIYNocWfRoxge0MD8Mc/hrYXF1MY9u8HfvKT0O/861+zjzt2hJ/Z+Oij/I4bNgBz54a2//WvwFe+QkGZMye0/eWXKVyLFgFXXBHavnw5cM45wGOPATfeGFpH5N13KcYPPwz8+MeholNVBfTrx/Zwovjii7wGjz4K/OUvoe1PPcXr/sQTwOuvB7fl5QH33st+Pv008MEHwYLYuTPwve+x/cUXgU8/Dd6/uNj+zm++yXvH2V5SAkyaxPY1a4AjR4Lbi4qAoUPZ/tlnvF/cgmvdP0eOhNaeUStYyWAyQzSKioBrr43cfsMN0fe/7bbo7b/7XfT2V16J3JabC6xbF7m9WzcKQyQqKoDjx0NFpaCA7aecAmzbFtreuzfbx4yhxeVuHzOG7aedBjz7bLAgutsfeCB0/4oKto8dC/z0p6HtpaVsHzQIuPji0Pb8fLYXFQH9+4e2WwNnfT2wb19ou2XZbN4MvPZacFsgYIvG4sUUlsZG+5qWldmi8fDD/P7ua26Jxs9+xuM7GTOGBc8A4OabKZBOzjmHogpQsNevD26fPZuiDNDi27EjuP3yyyl2AIXVsvQs4bv6avv7DR9Oa88pOtdcA/zwh3TbTp8eKrj/8A/A9dfTyvza10Lbr7wSuOACPkjdcUdo+8UX03rfvRt46KHQ9lmz+FC0axcfOpxtgQAwcSIt2T17gOrq0PYxYyjMdXUUXffx+/fn/XP0KL+Du71TJ94/agWnBDHWhffzpCI3APgXADkA3gHwbWNMfaTtKysrTXV1tU+9UzIWq6RwczOtEYBP+pZ16BSkfv3Yvm0bcPhwcHteHnDqqWxftYquS2d7SQkweTLbX3yRg5+zvW9f2/p78MFgK9ayQq+8ku23387B0SnoEyZw0AeA666jhevcf948Wp8NDTyPW3BvvJFit28f++luv/VW4FvfogVWWRnaft993P+DD2iFu3n8cT7EvfUWMGVKaPvf/w5ccgnw0kvhreClS4EZM4A//5muXTcrV9L9/PDD4a3sdesoxv/938A//3OoFbxxI/9/77kH+NWvQkVn9Wpam/feS0va3f7mm/x7//38Dk7BKygAnnyS/Vi4kJa808otLgbuuovtTzxBT4Dz2GVlvPYA3c5ffBHc3r07RRuge93pNrf2P/tstldX8x5wtnftygc5APjkE8jgwauNMZWhFzF+fBcNEZkE4A4AFxljjonIDwH0McZ8L9I+KhqKkgKMCRWspibGwvLzaSHv3h3c1thIS6GoiAPe5s3BgtjUROu2WzcOmKtWhR5/zhwOjhs30s0ZThRLSzlgv/xyaPtPf8rBe9Ei4PnnQ9t//3t+hz/8gQLnbl+6lA8Wv/wl8Le/BX+33Fzg/fd5fb77Xe7v/G6lpcCmTWy//HLguef4uTXODhkCfPwx/z1tGr+fk3HjKNYABX316uD2c8+lWAP0QljnsrjgAn5vAOjbF/Lll+1CNF4EcKcx5r2W9wJgI4CzjDEHw+2joqEoSkZjjG0F5+bys8OHQ13T2dmM4QK0BC0r1Hp16kQrC6BFduRIcHv37rYl8swzkK98JeGikYqYxmkAVllvjDFGRN4FcBaAJSnoj6IoSnIRoRvLSZcufEXCmcwTDkscInHZZd76FiO+pnGISFcAR02oebMLQH8/+6IoiqLEjt+WRgGASP6woM9FZAGABS1vj4tITTI7liDKANSmuhMe0H4mlkzoZyb0EdB+JprhiT6g36KxF0C3MJ/3AvCq8wNjzEIACwFARKoT7ZdLBtrPxKL9TByZ0EdA+5loRCThwWBf3VPGmJMANonIGdZnLYHwSjjiHIqiKEp6koqpqb8CcJeItMz+wq0AFkfKnFIURVHSB9+zp4wxz4tIHwCrRCQPwBsAvtPKbguT37OEoP1MLNrPxJEJfQS0n4km4f1MyYxwRVEUJTPRymmKoiiKZ1ImGiJyg4isFZGNIvKIiBRE2fYsEVkpIjUi8oaIDHS1D235vEZE3hWR0/zup4hki8g/ishyEflIRD4Qkcsd7T8Qkc0issbxeikF/bxPRDa4+vGQa5ukXM8Y+vhzV//WiMin1vVM9rV09OMKEenVyjYpuze99DHV92UM/UzZfRljP1N2b4pImYjcJSLvt/yO3hCRsVG2T869aYzx/QVgEoClAApb3v8QwH9H2LYIwFoAA1veTwbwrqM9G8BqAGe0vB8BoAZAQQr6eReAopb3vQBsADC65f0dAL6XBtfzDwAuiXKspFzPWPoYZl8BsA5A72RfS8c5+wH4DMC4KNuk7N6MsY8puS9j7GdK7stY+5nKexPAFADXA8hueT8RwFYAAT/vzVRZGrcCuM0Yc6zl/d0AzheR4jDbLgDwhDHmUwAwxrwF4BMRmdHSfil4MVa3tG8A8AKAq/3spzHmkDHmNmPMoZb3uwAsAjAhAf1IWD89kKzr2ZY+zgaw1hizs4198ISILAJTwPu0smnK7k2vfUzxfRnLtWyNZP7O29JP3+5NY8ybxpjHjDFNLe+XA6gDEK7eSNLuzVSJRkj9KQBW/Sk3U8EnVCdLAEz32O5XP4MQkWzwyWBDAvrRGnH3MwxTkZzr2ZY+/iOA37Tx/J4xxsw1xvQGy/ZHYypSdG/G0McgfL4v4+5nGKYieb/ztvTT13vTiYj0BtAXwBdhmqciSfem76Ihsdef6t/SFmnb1tr96qdz31wAjwHYZIxZ5miaLCKLWvzKr4tImMUIfOnnpSLyqoh82NKfUx1tCb+ebbyWA8HS+ctcTQm/lnGQknszXvy8L+PE1/uyraTy3hROW1gM4HaH9e4kafdmKqrceq4/1cr2xmN7vMTaTwD/p/5/BbAcgHPJwGUADgN4xBhTJyKVAJ4RkRnGmI997OcisG7OY4brmcwCsEhExhpj9kc5XluuZ1zXsoVbADzg+ixZ1zJWUnVvxkwK7stYScV92VZScm+KyARQ/O8wxjwVYbOk3ZupEA3P9ada+LKlbadr2x2udvexXGtsxkys/YSwPMofAfzQGPO8s80YsxQOc9AYUy0ijwM4H0BbbqaY+mmMedr1/lUReRMMVD+P5FzPmK8lAAizqy4F4HziTOa1jJVU3ZsxkaL7MiZSdF/GTaruTRG5EcB3wUXsNkbZNGn3pu/uKRN7/allCPWzTQfwusd2X/opLIvyJICr3T/MKBQBOOBnPz30I+HXsw19vBrAcxHMbzdtvpZxkJJ7MxZSdV8miKTel23E93uzxV33fQDTWhEMIJn3ZiwpX4l6AbgIwCsA8lve/wjAf7X8+14Akx3b9gWwHkB5y/upAN5xtHcCU97GtrwfDaaa5fvczysA3B3lWJcAmOR4f37L9yrxuZ/XAzjV8f5GACsB5CTzesbSR8c+1QAG+3ktXeepgiP9Mp3uzRj6mLL7MsZ+puS+jLWfqbw3AfwWwNwIbb7dm6lwT8FErz81EEBPx7ZfiMg3AfxVRDqDZtV8R/tREbkCwP+ISBmAQwCuMcY0+NlPAMMAzBeR812HedYY8xMw//snInIfgBwAmwHMMca0+Qkkxn5uB/CfItKzpR+rwBvxZMuxknI9Y+wjROQcALuNMVvDHO4zJJINNgEAAAIqSURBVOlatkLa3Jte+4gU3pcx9jMl92Uc/UzlvTkMwAwR+aXr8x+7+5nMe1NrTymKoiie0dpTiqIoimdUNBRFURTPqGgoiqIonlHRUBRFUTyjoqEoiqJ4RkVDURRF8YyKhqIoiuIZFQ1FaQURKRCRz0Vke8vERGfbQyLSJCLzI+2vKO0JFQ1FaQVjTD2AfwPLRt9ifS4ivwBwE4B/Msb8KUXdUxRf0RnhiuKBlsWLPgTQA8AgAF8H6/38mzHmzlT2TVH8REVDUTwiIvPAJTFfAyuC/sYY853oeylK+0LdU4riEWPMiwDeBzADwJ/BdQ2CEJFvi8h7ItIgIlU+d1FRkk5KqtwqSiYiIlcCGNfy9rAJb6bvBPCfAM4EMMGvvimKX6hoKIoHWpYffQLA3wGcBPA1EbnXGLPBuZ0x5pmW7cv976WiJB91TylKK4jIeADPgOtrXwPgJwCaAfwilf1SlFSgoqEoURCREQAWgYvpXGKMOd6y+M7DAC4WkYkp7aCi+IyKhqJEoMXF9CqAgwDON8YccjTfCaAewK9S0TdFSRUa01CUCBhjtoMT+sK17QRQ6G+PFCX1qGgoSgIRkQD4uwoAyBKRfADNxpgTqe2ZoiQGFQ1FSSw/AUuOWNQDeBPA1JT0RlESjM4IVxRFUTyjgXBFURTFMyoaiqIoimdUNBRFURTPqGgoiqIonlHRUBRFUTyjoqEoiqJ4RkVDURRF8YyKhqIoiuKZ/w/1jgrMjJEI7wAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x1a23804550>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "theta_path_sgd = [] # 알고리즘이 훈련 과정 동안 파라미터 공간에서 움직인 경로\n",
    "m = len(X_b)        # 훈련 세트에 있는 샘플 수\n",
    "np.random.seed(42)  # 초기 난수 생성값 지정\n",
    "\n",
    "n_epochs = 50\n",
    "t0, t1 = 5, 50  # 학습 스케줄 하이퍼파라미터 learning schedule hyperparameters\n",
    "\n",
    "def learning_schedule(t):\n",
    "    return t0 / (t + t1)\n",
    "\n",
    "theta = np.random.randn(2,1)  # 무작위 초기화\n",
    "\n",
    "for epoch in range(n_epochs):\n",
    "    for i in range(m):\n",
    "        if epoch == 0 and i < 20:                    # 훈련 스텝의 첫 20개를 그림\n",
    "            y_predict = X_new_b.dot(theta)\n",
    "            style = \"b-\" if i > 0 else \"r--\"\n",
    "            plt.plot(X_new, y_predict, style)\n",
    "\n",
    "        random_index = np.random.randint(m)\n",
    "        xi = X_b[random_index:random_index+1]\n",
    "        yi = y[random_index:random_index+1]\n",
    "        gradients = 2 * xi.T.dot(xi.dot(theta) - yi) # 하나의 샘플에 대한 그래디언트를 계산하므로 식 4-6에서 /m 이 없음\n",
    "        eta = learning_schedule(epoch * m + i)\n",
    "        theta = theta - eta * gradients # 식 4-7\n",
    "        theta_path_sgd.append(theta)\n",
    "\n",
    "plt.plot(X, y, \"b.\")\n",
    "plt.xlabel(\"$x_1$\", fontsize=18)\n",
    "plt.ylabel(\"$y$\", rotation=0, fontsize=18)\n",
    "plt.axis([0, 2, 0, 15])\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 73,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[4.21076011],\n",
       "       [2.74856079]])"
      ]
     },
     "execution_count": 73,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "theta"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- 샘플을 무작위로 선택하기 때문에 어떤 샘플은 한 에포크에서 여러 번 선택될 수 있고 어떤 샘플은 전혀 선택되지 못할 수도 있음<br>\n",
    "- 에포크마다 모든 샘플을 사용하게 하려면 훈련 세트를 섞은 후 차례대로 하나씩 선택하고 다음 에포크에서 다시 썩는 식의 방법을 사용<br>\n",
    "- 그러나 이렇게 하면 보통 더 늦게 수렴됨"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "사이킷런에서 SGD 방식으로 선형 회귀를 사용하려면 SGDRegressor 클래스를 사용<br>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 74,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "SGDRegressor(alpha=0.0001, average=False, early_stopping=False, epsilon=0.1,\n",
       "       eta0=0.1, fit_intercept=True, l1_ratio=0.15,\n",
       "       learning_rate='invscaling', loss='squared_loss', max_iter=50,\n",
       "       n_iter=None, n_iter_no_change=5, penalty=None, power_t=0.25,\n",
       "       random_state=42, shuffle=True, tol=None, validation_fraction=0.1,\n",
       "       verbose=0, warm_start=False)"
      ]
     },
     "execution_count": 74,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from sklearn.linear_model import SGDRegressor\n",
    "sgd_reg = SGDRegressor(max_iter=50, penalty=None, eta0=0.1, random_state=42) # 학습률 0.1, 에포크 50번\n",
    "sgd_reg.fit(X, y.ravel()) # 다차원 배열(array)을 1차원 배열로 평평하게 펴줌, NumPy의 reshape()함수와 반대 기능"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 75,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(array([4.16782089]), array([2.72603052]))"
      ]
     },
     "execution_count": 75,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "sgd_reg.intercept_, sgd_reg.coef_"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 4.2.3 미니배치 경사 하강법"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- **미니배치 경사 하강법**<sup>Mini-batch Gradient Descent</sup> : 각 스텝에서 전체 훈련 세트나 하나의 샘플을 기반으로 그래디언트를 계산하는 것이 아니라 **미니배치**라 부르는 임의의 작은 샘플 세트에 대해 그래디언트를 계산<br>\n",
    "- 장점 : 행렬 연산에 최적화된 하드웨어, 특히 GPU를 사용해서 얻는 성능 향상"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- 미니배치를 어느 정도 크게 하면 이 알고리즘은 파라미터 공간에서 SGD보다 덜 불규칙하게 움직여 최솟값에 더 가까이 도달<br>\n",
    "- 하지만 지역 최솟값에서 빠져나오기는 더 힘들지도 모름"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 80,
   "metadata": {},
   "outputs": [],
   "source": [
    "theta_path_mgd = [] # 알고리즘이 훈련 과정 동안 파라미터 공간에서 움직인 경로\n",
    "\n",
    "n_iterations = 50\n",
    "minibatch_size = 20\n",
    "\n",
    "np.random.seed(42)            # 초기 난수 생성값 지정\n",
    "theta = np.random.randn(2,1)  # 무작위 초기화\n",
    "\n",
    "t0, t1 = 200, 1000\n",
    "def learning_schedule(t):\n",
    "    return t0 / (t + t1)\n",
    "\n",
    "t = 0\n",
    "for epoch in range(n_iterations):\n",
    "    shuffled_indices = np.random.permutation(m)\n",
    "    X_b_shuffled = X_b[shuffled_indices]\n",
    "    y_shuffled = y[shuffled_indices]\n",
    "    for i in range(0, m, minibatch_size):\n",
    "        t += 1\n",
    "        xi = X_b_shuffled[i:i+minibatch_size]\n",
    "        yi = y_shuffled[i:i+minibatch_size]\n",
    "        gradients = 2/minibatch_size * xi.T.dot(xi.dot(theta) - yi) # 식 4-6\n",
    "        eta = learning_schedule(t)\n",
    "        theta = theta - eta * gradients # 식 4-7\n",
    "        theta_path_mgd.append(theta)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 81,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[4.25214635],\n",
       "       [2.7896408 ]])"
      ]
     },
     "execution_count": 81,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "theta"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "아래 그림은 세 가지 경사 하강법 알고리즘이 훈련 과정 동안 파라미터 공간에서 움직인 경로를 나타냄"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 89,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAdUAAAEYCAYAAAAKz2ydAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvNQv5yAAAIABJREFUeJzsnXd4VNXWh9+dSSdAKKEFAihKUZAuTUAvCqh8CIKCcFW4CIjoFcF79aok2AtFxIJI1URQilRBVJqFjohCAEHpHYRUUmbW98fOhEkyk0ySSSaE/T7PeYZzzi5rMsP8ztp77bWViGAwGAwGg6Hw+HjbAIPBYDAYSgtGVA0Gg8Fg8BBGVA0Gg8Fg8BBGVA0Gg8Fg8BBGVA0Gg8Fg8BBGVA0Gg8Fg8BBGVA0Gg8Fg8BBGVA0Gg8Fg8BBGVA0Gg8Fg8BC+3jagOKhcubLUqVPH22YYDAaDoQSxffv2cyIS5sk2rwlRrVOnDtu2bfO2GQaDwWAoQSilDnu6TTP8azAYDAaDhzCiajAYDAaDhzCiajAYDAaDhzCiajAYDAaDhzCiajAYDAaDhzCiajAYDAaDh7gmltS4Q1xcHGfOnCEtLc3bphg8iK+vL4GBgYSFhREYGOhtcwwGQynHq6KqlFLAE0BfICTjiBGRl3OpcxMwCagGBAALgJdExFZQO+Li4jh9+jTh4eEEBQWhzTJc7YgI6enpJCQkcOTIEapWrUr58uW9bZbBYCjFeHv4NwioC/QUkRZAK+B2pdRDzgorpSzAQuAFEWkCNAMaA8MKY8SZM2cIDw8nODjYCGopQimFn58fFSpUoGbNmpw/f97bJhkMhlKOVz1VEUkCRjucxymlFgP1XVSpAySKyFZ7faXUV0DzwtiRlpZGUFBQYZowlHCCgoJISUnxthkGg6GU421PNQtKqQbAAPSQrjOOAlWUUgOVUj4Z5UcBMR7ou7BNGEow5vM1GAzFgddFVSlVXym1Uyl1APgd+AHY66ysiKQC9wFjgdPALmCyiGwqLnsNBoPBYHCF10VVRPaJSFMRqQeEAmXRgUg5UEoFoYeLT2e8vgm8rpR61knZoUqpbUqpbWfPni26N2AwGAwGQwZeF1VHRCQBeB7o4aJIJBAvIreJyKciMhYd3PSSUiogW1vTRKSliLQMC/Pozj5XBVarlZkzZ9K6dWvKly9P+fLladeuHRMmTODvv//OUvbw4cMMHz6cOnXqEBAQQNWqVenVqxdLly5FRACYPXs2SimUUvj4+FChQgVatmzJyy+/TFxcnDfeosFgMJQ4SpSoZlAJOOPiXkNgruMFETkCXAKCi9iuq4qHHnqIF154gSeffJLffvuNHTt2MHDgQN5//30mT56cWW716tXcdNNNnD59ms8++4wjR46wevVqbr75ZgYOHMjPP/+cWbZWrVr8/fffnD59ms2bN/PUU08xd+5cWrZsybFjx7zxNg0Gg6FkISJeOwALMBuonHFeHvgOGJhxPgno6FB+GLAcqJBx7ge8CnyeWz8tWrSQ3NizZ0+u9682tmzZIoBs2LAhx73ExETZtGmTiIjs3btXypQpI6NHj3bazp9//il//fWXiIjMmjVLateunaNMQkKCtGrVStq1a+cx+4uK0vY5GwyGwgFsEw/rmreX1FiVUrHANxnDt6nAeyISnVGkLlDVofzHSik/YG3GmtU0dKTwo8Vrecnm4MGDADRu3DjHveDgYG699VYAxowZQ4MGDXjrrbectlO3bt08+ypTpgwxMTE0aNCAlStX0r1790JYbjAYDFc3Xh/+FZG3RKSFiNwsIs1FZLbDvftEZH628u+LDmxqnFH+ddFRwSWDatVAqZxHtWrFZkLTpk1RSvHkk0+SmJjotMy5c+dYsWIFY8aMwWKxFKq/G264gc6dO7Nw4cJCtWMwGAxXO14X1VLH6dP5u14ENGjQgHfeeYd58+ZRs2ZNhg0bxtq1a7HZrmRy3Lp1KyJCp06dPNJn48aN2blzp0faMhgMhqsVI6qllNGjR7N3715GjRrFpk2buOOOO6hevTrPPfccSUlJXLhwAQDHyOg+ffoQGhqa5RgxYoRb/ZUpU8ZEARsMhmseI6q54WwYN6+jONvLg+uvv56xY8fy66+/cvDgQR5++GHGjx9Pnz59KFeuHAAXL17MLP/BBx+wc+fOzKNLly4kJSW51deFCxcIDQ0tlL0Gg8FwtWNENTdE8n8UZ3v54LrrruOdd97h3XffZeXKlZmi+tNPP2WWqVq1KnXq1Mk88sOWLVto0qSJx+w1GAyGqxEjqtcYHTt2BHQUcPv27Xnvvfdclo2NjXWrzW3btrFjxw769OnjERsNBoPhasWIqqepWjV/14uAZcuWsWrVKqf3vv76a8qXL0/jxo155513WL9+PRMmTHBabs+ePXn2deHCBR599FH+8Y9/0K1bt0LbbjAYDFczXl2nWio5dcrbFmCz2bjnnnvo1asXI0eOpFGjRpw7d4558+bx5ptvMmvWLAIDA2nbti3Tpk3jscceY9++fYwcOZLQ0FBWr17N2LFjadmyZY527XOw8fHxrFu3jqioKKpXr868efO88VYNBoOhRGFEtRTSs2dPNm/ezEcffcSgQYM4ceJE5nDvmjVr6NChQ2bZwYMH06BBA1555RU6dOiAxWKhU6dOrFq1ioULF3L48OHMskePHqVChQpYLBYqVqxI06ZNGTt2LAMGDMDX13yVDAaDQYkHg2FKKi1btpRt27a5vB8bG0vDhg2L0SKDNzCfs8FgcEQptV1EWuZd0n3MnKrBYDAYDB7CiKrBYDAYDB7CiKrBYDAYDB7CiKrBYDAYDB7CiKrBYDAYDB7CiKrBYDAYDB7CiKrBYDAYDB7Ca6KqNCOVUuuVUtuVUvuUUmPzqOOrlHpFKbVHKfWbUupbpVTN4rLZYDAYDIbc8GYanCCgLtBTRC4qpcoBS5RSB0Tkcxd1pgJWoLmIXFZKdQSCi8leg8FgMBhyxWuiKiJJwGiH8zil1GKgvrPySqlWwK3ALSJiy6izoThsLe3s27ePkJAQwsPD811XRFCF3PfVYDAYSgslZk5VKdUAGAAscFGkL/CpXVANnuOll15i/PjxBar75JNPMnTo0HzVmTlzJu3btwcgKSmJZs2acfz4cadl80qj2bt3b1555ZV89W8wGAxFhVdFVSlVXym1Uyl1APgd+AHY66J4Q+APpdSLSqltSqlflFKRSilLsRlcSsguVP7+/pQpU6ZAbQUGBmKzZX3OOXXqFHPmzCE9PT3z2pIlS9iyZQsAAQEBJCYmAmCxWNi5cycWi/OPcdy4cQQEBFCuXDmnnrTFYjGessFgKDF4VVRFZJ+INBWRekAoUBaY5KJ4IPAycBJoA3QAbgH+66ywUmpohvhuO3v2rOeNL+EkJyfj6+tLUFAQZcuWpWzZsgQEBODj40ObNm2ylA0ICMhRt23btuzfvz/PfgICAnKI9MmTJxk8eDA+Ple+XitWrGD+/PkAWUTQlZjaiYyMJCUlhbi4OKferL+/f542GgwGQ3FRYoZ/RSQBeB7o4aLIKWC6iMwQkXQRSQSeA3q7aG+aiLQUkZZhYWFFY3QJxs/PD6vVyvbt24mPjyc+Pp6UlBRmzJiRQ8iyC1NgYCCbNm1yqx9nW77ZvUdHUfXx8ckUb8freeEowCJCamoqSUlJmd6xr69vnkPEBoPBUFyUGFHNoBJwxsW974Hm2a5ZgOQitagQRK2L8lrfdrHLLmBKqRyimv3cLlJ5eZH2MtlFzd6n43Wr1Zp53dEmZ0O3x48fJzAwkCpVqlCjRg1CQ0Px8/MjJCSEsLAwwsLCMkXfWf8Gg8HgLby5TtWilJqtlKqccV4e+BCYnHE+KWPJjJ25QFOl1ICM+0HA68Cc4rXcfcatH+dtE5ySXSyzC5tdpNzxKJVSOUTNYrFgtVrx8/OjTJky+Pn5MX36dKei6qyP8PBwkpOTOXPmDCdOnGDRokUEBQWRmJjIpUuXSExMpF27di77NxgMBm/hzSU1VqVULPCNUioASAXeE5HojCJ1gaoO5VOUUt2Bj5RSrwBxwEwRmV4U9qlxngl+KWw7ElkwwcgePJSlTReeZfb77gYAZW/PXs8xUOmRRx5xKtZKKaf9OF7bsWMH8fHxnDp1imrVquXZv8FgMHgLbyZ/QETeAt5yce8+J9dOAjmuG3LiKGjZsVqtWc5dearZy2VvIyEhwamn6EzQAwICMq/7+Piwa9cuAgMDs/TnjLi4OKZMmUK9evWYOnUqUVFROWw3omowGEoKXhXVkkxBPURH1DjlkXYKgl1UHQVHRLBarbmKpWOdlJQUzp07x7Fjxzh69CjHjh1jz549bN++nT179jB27FisVmsOUbP3bbVasVgs2Gw2UlNTM6/bbDZuvvlmdu3aBbieu01OTqZv3760adOGqKgoWrduTdu2benatWtmGZvNZkTVYDCUGIyollLsXmGjRo0yh1jt11q1apWlbHZRsp83a9aM0NBQwsLCiIiIoEGDBjRv3pzBgwdz00034e/vT1RUVI76dtH29fXFx8cns9/Ro0dntu/oSTsTxdjYWAYOHEhoaCizZs0iODiYGTNm0Lt3byIjIxk9enSmYBtRNRgMJQUjqkVIZKdIr/UdEhLCpUuX8PX1xdfXFz8/P2w2G+np6S5F1E5AQACpqan4+fnl2Y8zUWvSpAkXL17MXFZjF3X70h1nouroPX/77bf07NmTJ554gtdffz3TjgceeICwsDCGDRtGhw4daNeunRFVg8FQojCiWoREdY7yav/lypXLcm6xWJwOtdrF1hF3BNVe11n0b/ny5V3WcRRVe13H/u+8804OHTpElSpVctS9/fbbsySlMIJqMBhKEiVtnarBC6Snp+ca2JQbBfEUHUXcVVCUM0H1VP8Gg8FQVBhRNZCenk5aWlqB6hZE0MLDw+nSpUuW+qmpqQXq34iqwWAoSahr4QepZcuWsm3bNpf3Y2NjadiwYTFaVHrw9tZva9eupWzZsrRs2TLPsuZzNhgMjiiltotI3j8e+cDMqRoKhbd3iLn99tu92r/BYDA4YoZ/DQaDwWDwEEZUDQaDwWDwEEZUDQaDwWDwEEZUDQaDwWDwEEZUDQaDwWDwEEZUDQaDwWDwEEZUDQaDwWDwEEZUDQaDwWDwEEZUr1FmzpxJ+/btAUhKSqJZs2YcP348R7m8Mm717t2bV155pUhsNBgMhqsNr4mq0oxUSq1XSm1XSu1TSo11s65FKfWsUqpaUdt5tXLq1CnmzJmTJVH+kiVL2LJlC6C3d0tMTAT0rjI7d+50uoPNuHHjCAgIoFy5coSHh+e4b7FYvJ5VyWAwGEoK3vRUg4C6QE8RaQG0Am5XSj3kRt2XgbGAEVUXnDx5ksGDB+Pjc+UjXrFiBfPnzweyphd0JqZ2IiMjSUlJIS4uzqkna98j1WAwGAxezP0rIknAaIfzOKXUYqB+bvWUUvcCocD2orXw6sbuQTqKqo+PDwEBAZn/dgdH8RUR0tLSSE9PJzAwEB8fH3x9fc0uMQaDwZBBiZlTVUo1AAYAC3IpUxf4N/BMcdlVEGJioE4d8PHRrzExxW+DXTQdBc9qtWZedxTV7MO3x48fJzAwkCpVqlCjRg1CQ0Px8/MjJCSEsLAwwsLC2LRpE6DF24iqwWAwaLy6S41Sqj7wBRAC1AEmA3tdlA0EPgEeE5GUvObxlFJDgaEAERERnjM6D2JiYOhQSErS54cP63OAAQOKzQwsFgtWqxU/Pz8CAgJITU0lPT2dyMhIgBwerCPh4eEkJydniu2aNWu47777iIuLy9GPUsqIqsFgMGTgVU9VRPaJSFMRqYce0i0LTHJR/D1gsoj85Wbb00SkpYi0DAsLy7dtShXsGDjwiqDaSUrS1wvaZkGwC2J6ejqJiYmkpaXx8MMPZwpgdk81+0OK4/mOHTuIj4/n1KlTTvsyomowGAyaEjP8KyIJwPNAj+z3lFJlgXuAV5RSO5VSO4GWwBdKqS+K19KrA5vNluNaQEBA5nUfHx927dpFYGAggYGBLoUxLi6OKVOmUK9ePaZOnZrjvvFUDQaD4QolRlQzqAScyX5RROJFJDzDq20qIk2BbcCDIvJgsVt5FWBfSmO1WgEtsvYhYPv5zTffzOXLl7l8+bLTwKXk5GT69u1LmzZtWLp0KRMmTOCbb77JUsZmsxlRNRgMhgy8uU7VopSarZSqnHFeHvgQPa+KUmqSUqqjt+wTKdgRHQ3BwVnbCg7W1wvaZkGwi6mvry8WiwWLxcKcOXNIS0vLeH+SZQ1rdmGMjY2lQ4cOpKenM2vWLBo2bMiMGTPo3bs3b7/9dhaxNqJqMBgMGm8uqbEqpWKBb5RSAUAq8J6IRGcUqQtU9ZZ9BcUejPTCC3DkCEREwGuvFW+QEkCTJk24ePFi5rIa+7ypfV2pM1G1C+W3335Lz549eeKJJ3j99dfx8/MD4IEHHiAsLIxhw4bRoUMH2rVrZ0TVYDAYHPBq9K+IvAW85eLefXnU7VwUNnmCAQOKX0SzY7FYKF++vMv7jqJqF0X7+Z133smhQ4eoUqVKjnq33347+/fvz9KOwWAwGDQlbU7VUEzYbLYcomr3VAGnguqqHSOspZ+odVHeNsFguCowonqNEh4eTpcuXYAropqamprvdoyoXhuMWz/O2yYYDFcFXh3+NXiPDh060KFDB6BwWZGGDh1K2bJlPWmaoQQhIqz5aw0AyWnJBPkFedkig6EQVKsGp09nnraAFp7uwoiqoVDcfvvt3jbBUASkWdN4cMGDfLX3q8xrwa/rsPbITpFEdY7ykmUGQyFwENSiwoiqwWDI5NLlS0zbPo33trzHsbhjWe4lv5BMoG+glywzGAqACJw8CbGx+igGjKgaDAYOXzzM5M2Tmb5jOvGp8Vnu3X3D3Xz9x9dGUA0lF5sNDh3SwrlnzxURjY0FPz8ICoKjR4vFFCOqBsM1zLYT25iwcQLzd8/HKjr6u2JQRS4kXwCgx409mN93Pm/8+IY3zTQYNKmpcOBATuHctw8qV4aGDfXRpg0MGgTx8TB7NqxZA889B2++WeQmGlHNQERyJJU3lB5MhPIVbGJjxf4VTNg4gfWH1wNgURYGNB5A+1rtee775wDo1aAX8/rMw9/ib+ZQDZ4jW7BQJlWrgn3TjsRE2Ls3q3Du2aO90YgILZyNGkG3bjBqFDRoAPaASZsNli+/koFn1CiYMUPfN6JaPPj5+ZGcnExw9vyChlJDcnJy5gbt1yqX0y/z2a+fMWHjBPad3wdAuYByDG0+lKdufYojl47QPaY78anx9G3Ul5jeMfhZ/LxstaHU4SpY6PRp6N5dC+jp03DDDVo4GzaEfv306403gqv/x8nJ8NlnMGGCFtBnn4X77wdfB5mrWrXIg5WMqKITHRw/fpzw8HCCgoKMx1pKsGeNio+P59y5c1StetVlvfQIZxPP8uHWD/lg6wecTToLQK1ytXi6zdMMaT6EcgHl2HB4A3fH3E1iWiL9bu7HZ70+w9fH/DwYcsGVx+njo73F7FStCtu25d7miBFaPOvWBYvFPTvOnoUPP9RH69YwbRp07Oh838xs21duV2q7e524j/lfA5QrVw6AEydOZCacN5QOfH19CQwMJCIigsDAayvQZv/5/UzcOJE5v87hcvplAJpXb86YtmPo06hPphe69q+13Dv3XpLSkhjYZCCzes66JgQ1al2UGdYuDK48PmeCai/ftGnubfbIsfOna/bvh0mTYN486NsX1q3TguxlSv//HDcpV65cprgaDFcrIsKPR35k/MbxLNu3DEHPJd97472MbjuaTrU7ZRmJ+e7P7/i/uf9HcnoyjzZ9lOk9pmPxcdNDuEqx2qzsPLWTcevHGVEtbs6ccd8DdYYI/PwzjB8PP/0Ew4frudcSNAplRNVgKAWk29JZFLuI8T+PZ+uJrQAEWAJ4+JaHGdVmFA3Dcj7Brzqwivvm3UeKNYUhzYbwcY+P8VGlM3PpqYRTfHPgG1YdXMXqg6szo5sNBcQhT3i+cLJvs9v9LV6sxfTsWXjmGb2fZpkyBWuvCMm3qCqlgoBngYeAOsBZ4DMgUkTM2KnBUIzEp8Qz85eZvLv5XQ5dPARApaBKPNHqCUa0GkHVEOdP8Cv2r6D3l71JtaYyvMVwPrjng1IlqGnWNDYe28iqA6tYdWAVv5z6xWk5NU577SZLlJv88Yfegmvr1oK34SpYyMfH+Txo2bIQFgZVqujgo549C+ftFjH5ElWlVHXgW+BG4CtgCXAv8DxQCRjmaQMNBkNOjscdZ8qWKUzdNpVLKZcAuKHiDTzT9hkevuVhgv1cR7Iv3beUPl/2Ic2WxshWI3mv+3ulIjjv6KWjWkQPruK7P78jLiXOabkASwBT753KoCWDkEiz1CpP4uNhzhx48skr1wID4fLlgrWXLVgoE1ffwfh4WLUK2rUrWH/FjNuiqpTyB5ahvdPbReSnjOuvALuBIUqpSBFx8RczGAyFZdfpXUzYOIG5v80lzaYHhjpEdGBM2zH0qN8jT29zUewiHlzwIOm2dJ6+9Wkmdp141QpqSnoKPxz5IdMb3X12d5b7DSs3pFu9blQKqsSkTZM4n3ye8LLhLHpwEa3DWzNoySAvWV5CcRXN60jlypCWBnfdBd9+Cxcv5iyTW/RvQblKBBXy56mOQWf0H2kXVAARSVBKfQX8G7gNmO9ug0r/b34C6AuEZBwxIvKyi/IW4HGgP1AWsAKviciCfLwPg+GqQkRYfXA1EzZO4Ns/vwXAR/nQt1FfRrcdza01b3Wrnfm759N/YX+sYuXZds/yVpe3rjpBPXjhYKY3uuavNSSlJWXeC/EPoct1Xeh2fTe61utKndA6fPrrpwxdNpQUawrtarVj4QMLqRZSDdBDvnauyUhgd0TUkRYtdJaifv2gUqWis+sqxy1RdZhHPQlMc1LkfMZrtXz2HwTUBXqKyEWlVDlgiVLqgIh87qR8GaAG0F1E4pRS1YC1Sqm9IvJ7Pvs2GEo0KekpzP19LhM3TuS3M78BUMavDP9q9i+ebvM0dSvUdbutub/N5Z9f/ROrWHm+w/O8dsdrxSaoBREse52ktCTWHVrHqgOrWHlgJQcuHMhS7paqt9CtXje61etGu1rt8Lf4Azpw6+lVTzN582QAhrUYxnvd38u8D2TadC7p3LUZCZwfQd21Cxo3Ljpb0tOLru1ixl1PtRcQCsxwEYxkXwCYr12uRSQJGO1wHqeUWgzUd1E+Dvifw/kppdQKoC1gRNVQKvg7+W+mbpvKlC1TOJlwEoDqIdV56tanGNZiGBWCKuSrvc9+/YxHlzyKTWyM7TiWqM5RxSaoF5IvMG79OLpc1wWb6A3tBcnyahNbjmvj1o9j47GNrD+0nhRrSmZ7oYGh3HX9XZneaI2yNXL0eS7pHA8ueJA1f63Bz8ePKd2nMKzllXCPVGsqPxz+gZUHVrLywEr2nN1TLH+Lq5qiFNS0NB38VEpwV1TvyXgNV0pFObnfJeO1UNsAKKUaAAOAf7lZ3gJ0AhYXpl+DoSTw599/8u6md5n5y0wS0xIBaFylMaPbjqZ/4/5ZvCx3mfXLLP619F8IwrjO4xjbaaynzc7CmcQzbDi8gfWH1rPhyAZ+O6097Ntm3ZbvtlYfXI1C0apGq0xvtHV461wTU/x66lfu++I+Dl08RNUyVVnwwAI6RHTgyKUjrPxDi+j3f31PQmpCjromEtgLpKXBQw9BUpLrqOAStAbVHdwV1Q4Zr/3yKJfvRz6lVH3gC/R8ah1gMrDXjXr+wExgn4j86OT+UGAoQERERH7NMhiKjc3HNjN+43gWxS7CJjrA467r72J029Hced2dBfYqP9n+CUOXDwXg9Tte5/nbnveYzXZOxJ9g/aH1rD+8ng2HNxB7Lvc9KyPKRVCnQh0UCqUUCoWP8kEpxZ8X/uTPi39mKS8Id99wd54iF7UuioaVGzJoySCS05O5peotjG47miV7lzB8+fAcQUyNqzTmUsoljlw6Qp3QOhy6eAjJ7GJcxkHWJO+lgbQ0+OorePddb1uibenXD1JSYNEi1zl9rzJUXrt3KKXKAAnA7yKSYwxAKRUCXABOi0itjGsduRLYVAMYJCKz8zRGtzURSBWRkbmUqw4sAH4C/iciuQ7It2zZUrbllXPSYChGrDYry/YvY/zP4/npqI778/Px46HGD/FM22doUrVJodr/aOtHjPh6BABvd3mbZ9s/W2ibQe+7uv7w+kwhPfj3wSz3g3yDaFerHR1rd6RT7U7cWvNWgl4LyvfSFTVOuV3HarPi+0pW/8DPxy8zOhqgrH9ZulzXhbtvuJtu9bpx8fJFmn/cnDRbGt/98zu6fNbFQVSzcTXtcOQq+CgsTCdM+OADOHHCdSpBZxTFg0VqqhbU9HSYP794BdXhb9QS2Cbi0bkQdzzV8IzXEy7udwX8gBUO10LQc5yfZhxukRFJ/DywA3AqqkqpFkAM8B8RWepu2wZDSSApLYk5O+cwcdPEzKCb0MBQhrcYzpO3Pul0jjC/TNk8hadWPQXAxLsmMqrtqAK1IyIcuHBAD+ce1iJ65NKRLGVC/EPoENGBTrU70bF2R1rWaFmgYeqCkpKeQq8veuW4nmZL4+YqN9O9Xne61+tO+4j2WYKYen3RizRbGkObD+Uf1/2DyHXFZnLR4ir46OxZva/oyJHQuTP83/9pQbvgJLNUUXvnqanwwAP6YaW4BRVKxC419v8hKS7u2xd7zbRfEJGvga8BlFKz82lTJeCMsxtKqUAgGnhIRHbks12DwWucTjjNB1s/4MOtH3I+WQfL1w2ty9NtnmZws8GE+Id4pJ9JGyfxzOpnAJjSfQojW7sc8MmBiBB7LvaKiB5anxkoZSc0MJTbIm6jU+1OdKrTiabVmuaZfN9x6YqdmJgr211GRMBrr+lYFft1jtioMyvndcfyf4RHMW79uBxt33vDvXx4z4fUKl/LqT0Tfp7AthPbqFWuFu/c9Q4AUevc+xtd1cTG6nR/nTvrPUafe674bUhJ0cnvfXy0oPoX3wNYceHO8G9ldCrCrSLSOtu9NsDPwCoRudtF/QT02tbZTu5ZgBnAGBE5p5QqDywEZotuGhVgAAAgAElEQVRItFJqEvCViGzIKN8XaC0i+RrLMsO/Bm8RezaWiRsn8tmuzzKjWFuHt2ZM2zH0atjLo7vBvP3T2/z3u/8C8NE9HzG85fBcy9vExm+nf8ucD91weEPm1nB2woLD6Fi7Y+ZwbuOqjQudzjAmBoYO1bEpdvz9dYDpzp1Z08r6+MB11+m9qR1XXfj5QblycOGCULFaIufbDiXly9l5esl7z+2l6dSmpFhTWDVgFV3rddU3cpu3vpqGf/N6Hy++qFMMrlxZ8Dy8BSUlBfr00R/evHneE1SHv5FXhn8zxC4WaKGUaiIiu7RdqjYwD7gEjChI5yJizWj7G6VUAHpJznsiEp1RpC7gGPp1I9BPKdU9W1OLReTFgthgMHgaEWHdoXVM2DiBFX/oWRGF4r4G9zG67Wja12rv8SUtr//wOi+seQGFYlqPaQxpPiRHmXRbOjtP7cyMzP3h8A/8ffnvLGWqh1SnU51OmcO5DSs39JitaWmwcaPeMtNRUEGPCG53srOlzQYHDuS8npYG588DKM6fDIFF0QT4+lC79hXvNjtWm5VBSwaRYk1hUNNBWlBFYGzRRkSXGL79FmbNgh07vCOo99+v0xvOnauFtbhJSICZM/MuV0jcfUx+FT2P+b1SKhqdhOEBQIB7RORQQQ0QkbeAt1zcuy/b+WvAawXty2AoStKsaczfM58JGyew46SenQj0DWRQ00GMajOKGyrdUCT9vrz+ZSLXRaJQzOw5k0ebPpppz7YT2zLnQ3868hPxqfFZ6kaUj9BDuRkiWq9ivUKJaPZh2mee0b+jq1bB999DnPN0vB5Ai8Thw9oLhpzCOnnzZDYd20SNsjWY2HWiVubHHtNeU4UK8Pff5OAqW86RK488ond2Ke73dPmyFtTgYPj88+IX1JMnYcoUvXl5585F3l2ew7+ZBZV6BPgPcD1wDj1nOk5EjudRz+Xwb3Fhhn8NRUlcShyfbP+EyZsnczROL9WuUqYKI1uN5PFWj1M5uHKR9Bu5NhJBeGXDK/goHz6+92NurHRjZmTuxmMbs6TxA6hXsR4dIzpmeqO1Q2t7zB5nw7r5QSnnI62uUsnmho+Pbss+/9q62x80mdqEy+mXWdZ/GfdW76Tn9tavh6VL4c47C2Z0ScNV9K+/v55DHZdzDrpIuXwZevXSY/XR0cUrqLt3w4QJesu4hx7S88inT0OHDplftKIY/nVbVAvcgRFVQynl6KWjTN48mU92fJK5I0qDyg0Y3XY0A5sMJNA3MI8WCo6I4PPylSG8IN8gbGLLkn0IdFJ5uxfasXZHwsuFZ2/KI6SkQHi4fUjWNRERUK+e3l86xcHU4GDtSM2Zk1WUXV3PD0qJ/g0tf5j2g77mx//2hnvu0ZtbL1sGd9xRsIavFl57DVav1kMFvsW4hXZyMtx3H1SsCJ99Vjx9i8DatXrf1V9+0dHOw4frXMW//qo3Avj0U+iq59KVUttFpKWHbRCPH+glNU0zjiRgbMa/I4qiv7yOFi1aiMHgCSLXRsr2E9vloYUPie/LvkIUQhRy++zbZfm+5WK1WYvFjo+2fpTZt/1QUUqafNREnvz6SVmwe4GcTjhd5HacOCESGSlStaqI/kVzfkycKLJnj4jNputFR4vUri2ilH6Njnb/eqVKIv7+uffn6ggiQaLpr08qVCjyv4/X2bBBfzjHjhVvv0lJInfeKdK/v0haWtH3l5qqvyTNmok0aCAyfbpIcvKV+3/8IVKjhsgXX2SpBmwTT+ufpxvUdtIZPd+a/ZhdFP3ldRhRNRQWq80qK/avyCJilnEWeWjhQ7L9xPZisyNybWQOMSUK6Te/n5xPOl9sdmzaJDJggIifX95CVru25/u3iyxooc2PsFpIuyKspZmzZ0Vq1hRZsaJ4+01MFOnSRX9BPCWorp7aqlQRGT9epFYtkc6dRZYvF7Fme7A9dkykbl2RadNytNlCfwdKvqiWtMOIqqGw/HD4h0wBK/t6WRn9zWg5fPGwV20iqnhFISVFi1nr1s5/30JDc4pscPAVb7OocPRiLRb3hFVhFbBm8YZLFTabyD33iIwZU7z9JiaK3HGHyD//KZKe7rl2c/sw+/UT2brVeb1z50QaNRJ56y2nbRaFqBZzXLXBcPURtS4qS0L4+NR4JmycwMxfij4835vExECdOjroJzRUZ7obOBC2bMlarn17+OILOHNGr9ioXVsHHdWurQMui3oDkgED9DpWm03PvQYH511H//T5ZEYLx8QUrY3FzqRJcO6cnk8tLhIT4d57oWZN/UWwWIqn37lzoWW2aVGLRX8JK1eGPXvgv//V58Vhk6dVuiQexlM1eIri9g5zI3JtZJG1HR0tEhjo2jnw9xd55BGRbduKzIQCEx0tEl4rTcjwRt0dFi6pXmtun7PTe5s3i4SFifz1V/46cjXEWrVq3nUTEkQ6dRJ59FHXHmph2s/tg0tOFjl5UiQ2VmTjRpGVK937wIvIUy3y6N+SgIn+NXiK/CR6v1o5dQpuuEGvlc+Oj49elTF0KFSpUvy2uYOI0O2zu1j913e0XNmfXzZ/itXtJfk6UHTyZO9v8Xk+6TzL9i9j0JJBLr9zOb6PFy9C8+Y6+rV37/x1WNCsUgkJOpr6+uvhk09ce4O5tX/qlLbd1fHmm67r+vvrdcahofoICdGRzq6oWxfatYOYGK8l1DcYDBk4y2NbWoiP18v6xo/XI3nOENGZ7koyszZ+yOq/vqNiEiw/8Q3flRnB0MRJJFHGrfrnz+th7n//u4Di6mqtaD4S1S/fv5whS4dwOtF58vdTCacYukxnuTh88bBebyyik1ncfXf+BbWgxMfr/urX12P9Bc3U1LjxFVF0FMjQUJ3RIzcc12bZbPCvPLbj/jNje8EiGvM3nqrBcI2TlgbTp0NUlJ4XzY3atfX8ZUnl+B/buWl2ay7524j+thwDpv4EN9+cmenp8GHXSSacERyc+7xw1LqonHu9FiKPcFxKHM988wwzfpnh9H5kp0j2n9/P3N/n5rwXfDdRn5/QuSADC7BGOr92x8dD9+7QsCF8/HHugmqz5T6f6ervsnMndOumk0hcupTzvuODiohO4bVlC/z8c959VatGy9OnPe6pen2+szgOM6dqMOTEZhNZtEjkxhtzTjfVrCkSEJD1WnFE8hYG2+7dcs+/goQopMegALH9/rvTco7LcQo73+p0jj23hnJh7V9rpfak2plR5qFvhspvp3/L7CMxNVEeX/54juVUIiKyc6dI5coi+/bl4y/mQGxs/uy+dEmkXTuRoUNzLmHJztGjes1qfv8u27fr+dYFC9x7Dy+/LNKkicipU273xdWyTrWkHUZUDYas/Pij/k10JqazZ+tYE1eJGEokP/wgn3Uop8XoeSXHf1mfZ5XoaP2g4L642qQSZ+TtkP6ysCHy6m1a1Gw2m35COXZMZNmyfItHUmqSjFo1KotQBrwSIOsP6fdAFLLjxA5p8H6DLGUeW/qYFtX4eP1k5M4HlFeWjtzszm+gkc2mbQoLE3n11fz9XbZt02tQFy7M+z2JiEyZInL99Tpg6V//ct2Pj0+WakZUC3gYUTUYNHv3ivTqlfO3pnx5kTff1IlwrjoWLJCTtStJhRf9hChk1qo33a4aHa0zNOVPY2xC0Bmhd/8sIhd5d5DIXXflSzy2Ht8qDd9vmCMz1oLd2juz2qxy56d3it/LflnKNJvaTJLTkiVyzViRgQNFBg927w3nZlteopkfUTx3TqRPH71GdMcOfc1dUd66VQvqV1+5/yHWrCny558i778vcvPN+kHDrT+HEVUjqgZDATh5UmT48JzJEfz8REaN0r+BVyWTJ4stvIb0+m9tIQrpPq2T9hyz4WpZysXki7Lx6EZ57NW1Elg2Xgumu+LqlyD07i+yZIke4rT364b4pKanSuTaSLGMs+QYzp2yeYqIiBy7dEz+MecfmdeDXwvOHBY+eOGgbmjWLC1cCQnu/b3yI4wFrbt8uU4J+MwzWVMFusOWLVpQFy92r/zSpVqUf/9dZO1a/e+DB93urihE1QQqGQylmIQEHdH7zjs5I3r799e5AerW9Y5thcJm07uuLFvGFz3q0q/MSsr6hbD7iT3UKl8rR3E1TrHukXXEnotlz9k9ma8n4k9kLbirP3z/OlyqDbgRv6LSif7M90ogU3Jy7tknRNhzdg8Pf/Uw209uR6FoH9Gen4/+jE1sPNf+Od7o8gYL9yzksWWP8fflvwkLDuORWx5h4qaJ2MTG0n5L6VG/B8TGQseOsG4d3HST6z5dRSNnp2pV5+UqVYK2bWH58rzbsFOhAly44F7ZgkZLr1+vdxpavlyXbdNGJ+7v0sVtM4siob5ZUmMwlELS0mDGDB3Rm/336o474O23oUULr5hWeFJSYNAgOHKEs727MtI2BYAJXSdmEdQLyReI3hWdGUnbeU7nHE0F+gZSv1J9GoU1omHlhjTs25BGU5PYusrKiOG+ee+MI75Zl99seFpH3l6+nKOorWoV3t04kf99/z9SrCnULl+bFzu+yPPfP49NbDx8y8P877b/MWTpkEybu9frzuv/eJ1u0d0yRbdH/R5avB94AN54wzOCCq7LnT+fP0EF53vT5rff3Ozevl0L6rx5+v136KCzJuVDUIsMT7u++TnQj4IjgfXAdmAfMDaPOq2BTcDvwFqgbl79mOFfQ2nHMagoLEykWrWcI3SNG+tkM05GR68eLl4Uuf12kd69RaKi5IFHQ4QopMunXcRms4nVZpVvD34r/Rb0czq0ShTSZU4XWbZvmRw4f0DSra7z00ZHi1TgTD6GhG1Syee8RH+SmKOtPy/8KR1ndcy0YfDiwbL/3H6p+25dIQrp+llX+enIT1LvvXqZgUpTNk+R1PRUiZgYkbkTUpo1I0H90KF6B5i8Psz8TRZ75Ijs7OZwsjs2OiM2Vn/Bv/pKv/8HH9TpvQrwxaa0zakCwcAEIDTjvFyGUD7konw54De7kAIdgc159WNE1VCaySuKtWZNPfXmyfzmXuHoUf1k8OSTIi+/LA8M0oJa5rUy8sPhH+TldS9LnXfrZAn46fpZV/ny9y8LlF5yx4kdct1T6ICkQPfFVSmRxx/XbdhsNpm+fbqEvK5trfpOVVm6d6nEXY6T5h83F6KQplObynPfPpf5ENDkoyby+2m9HOjZ1c8KUUj18dXlVPwp3ei8eSL16ullLdkpaHSvBw+icrmfPSipYsX8ierhwyIREfoLLSLyxhsirVrlf+42g6IQ1RI3p6qU+jdQUUQindwbA/iIyNsO1+YC00XEZV4qM6dqKM3UqgXHjuW8rpQeHXzqKQgKKn67PMru3TpzzxNPQFIS5xd/TuVefwBQ1r8siWmJ2MQGQET5CAY3HcyjTR/VmYZwI72kzaaHSs+eRYCZzeCJeyDFF5qfgPnzYePf/XkE91MeVqhopXa/ieys8h8A+jTqw0f3fES5gHL0mNuD1QdX4+fjx3UVrmPf+X0AjGozitf/8TqBvoEs3ruYXl/0AmDDoxu4rfZtcPCgziHp7Hfb1ZxoMXAyBBY3gBU36kOicilst/2nn/SwbW44vs8zZ+C22+Dxx+Hpp2HFCp0vc8sWCA8vkN2lfk5VKdUAGAC4yjPVGb3huSPfAncAuSR7NBhKHyIQHe1cUO3897/FZ0+RsX69nj+cNEkH53z1Fc+80BT2a1GNT43Hz8eP+xvez5DmQ/hH3X9g8cmavSczveTFi7B/vz727bvy+scfkJREkh88cTfMbqaLD90Gk1dBYDpch85iNJRP3Ep5+PcFC39/OAYVPIjHX9rH+33aAfDI4kdYfXA1AGm2NPad30f1kOrMuW8Od15/JwBPrXyKKVumZLbVcXZH/T72VSfKlSNUzIJ6KBS+agALG8FPtcgS16Wi9GvkOoha56TyDz/A/ffn3YljlidfX/jf/7Sg7tun59UXLy6woBYVXvdUlVL1gS+AEKAOMBl4TkTSnJT9FeguIiccrt0N9BORh7OVHQoMBYiIiGhxOK/8kQbDVcSBAzB8eO55w0t6SkG3+PJLGDlSb+/13XdE/TmTcY1y5lIc03YM79z1jj5JTdUeXXbh3L8fkpLgxhv1Ub/+ldcbbmD/deXp8wD8VhWC0mDqcnj415wmxdCffzOZ84Tl661UqgRtH5vH8sD+Wa73rN+T6f83ncrBlQFITkum7Yy2/Hr6V3o16MVXe7/SXvbTT+s8i4sX56tfT7KvkhbRRQ1he42c9zv/Bevq5uGprl+vBfXzz+Guu/JngM0GcXFw660wZgwMGZK/+tkoCk/V66LqiFIqBJgIpIrISCf39wOdROSkw7W7gQdF5BFX7ZrhX0NpITVVJ7x/5ZWsAabZ89nmlbO2RJJXpKqvL5w8CZUr89eFP7luyvXYOnyHsnuedvE8dgwiIrIKp/3f1as7zXG7YM8CBkf3JT4AbjwHC76ExrnlQa5alZhRW/n3c8GcpyJuLb8BwAYtP4R7nyTYL5h3u77LkOZDUA42DV4ymFk7Z1GvYj22PbaN0LdCkWZL9Dj+L79AxYpu9lV4BNhV9YqQ7naxM5GPDcauhxc3gG9kHqJaubKO2q1VS38m+SE9HXr21Bv9vv9+/uo6oShE1auBSs4OoBJw2MW9dUCzbNcGA6/m1qYJVDKUBn7+WSeLcYzj8PHRa+xnzLiKUgq6wp1AmAcfFGnaVKRMGR0Q07GjyJAhIu+8o5Mw7N0rkpLidpep6alZUgT27YtcCnAjYCY+XqR+fZE5c2TI0BTJz76tYBNLmQvyzkfHczQ7fft0IQoJfDVQdp7cKSIikYuf1gkRfv7Z/b9TboE/uQQzRXZGbCCbwpFn70Sufypr5HT555B/9kKG9ECC/6evVR+NrK2TtY1cbfj+e5EjRwpm///+p/dtTU3N77fLKZS26F+nBsGNwFYX914FRme7Fg3ckVubRlQNVzMXL+poUqWy/r40b14yNwkvMO78qEZH6zR2Fy8WepP2o5eOSrsZ7YQoxPdlX5l8qxaUPKNQbTaRhx6SyFHN5IfDP8h1k6/TEcJB+Vl+kzVKWERHGwe8EqBTLf4yS19MTdVJmt96S5/v2pV7o7lF/9ojb52UsYFsiNAiWXNUViENexYZei+y6nokwQ8Z1fXKve4DkDPB+RDFChVEzpxxv3z2o3ZtXd9DlDpRBSzAbKByxnl54DtgYMb5JKCjQ/lwYA8QkXHeGfg5r36MqBquRmw2vUFH9epZf1fKlBGZOFEkLc3bFnoYdz0tD7D6wGqp/HZlLSITa8rPR352OzetbepU+fPW+jnWv/566tcC5RKuVElk2qwELc5RyJAlQ6509vzzIt266Z1gzp8Xue46najZVWM+Pq7fQ7b3F9kZSfJFpjVHbhqR9b3UHIU81Q1ZVxtJV7r8gQpIy8f0fd+XkLfbIVZVQHEs6GHPI+whikJUvT6nqpT6L/AAEACkAu+JyOyMe4uBGBGZ71D+NvTa1hDgBDBYRI7k1oeZUzVcbRw9qlePLFuW9fo998AHH+ggpFJHbvt52ink75VNbLy64VWi1kUhCHdedycxvWMIK5Mz6MgmNo5cOsKeu5oR63uRPWGwJwxiw+BSLtuVRnaK5IbjUTw2IpHkuGDcm28V8I+n9oA32TttLIG+gXru1FlmIg8snTlYAer9GyolwXkXWRUdI3e/vAke6wFxgVD7IsxbAG1yiTovEsqX19HbHqTUByoVFUZUDVcLVquOv3jhhay5eqtVg/fegz593NOeq5IiFNWodVGMbD2SgYsG8s3Bb1AoxnYay0sdX0IQDl44mCMv8N5ze0lKyytPoYNpkTltGzECpk7Nj9nC448rPnzpJNRwEl5bCA5WgPk3wfxGsMOh6ZbHYdQm6LMHAl4iS5BRsi+M6gYfZ8hO7z0wfSlUyJmF0XPUrq2X3NSqpfNtdu2qo33feMPjXZX6daoGw7XML7/otezZn/+GDYM334TQUO/YVWzk5YFVrVrgpsetH8fMX2ZyNO4oAG1qtmHP2T00/bgp+8/vJ9Wa6rRetZBqNNp1iobnoNFZfTQ8C1USwSfqipCqcc4fCD78ENq3h4GPnYVkd5bgKD76SPjykyAm058BGWtjC4orIXVkWzjsrwT+1qzXYyvDg331EiP/dJj0DTy+1f045wLzzTdaUAFGj9a5lF99tah79RhGVA0GL5OYqBPfT5qkPVU7jRrBxx/nnXSmVCAC//qXTty+Zo1e1OkhVh1YBZApqAAbj23MUiaifASNwhrRqHIjGoY1zEywXyGoglsedGZyCScMGAB/hH/AmS+j3PRaFefTQxlIDMOZylSG5ymuUZ2vDNX+WUGL6Jc3ZRXSkBSoGQd7wyD5VQh6kRxLXyLXgQBzmuokGEn+eonRFwugaS4bxngU+zKbWbO0wG7erBM8FGQnGy9gRNVg8CIrV+ohQsckDQEB8OKL8J//gL+/10wrXsaNg6VLPSqoUeuiGLd+XI7rN1a6kd4NemeKZ4PKDQjxD3HeyN69LtuPXAdkaGlU56jcbekcBZ211/rvf8P580LePp8igXIMJIafaMeHPOmy5LjOOmHF/JuyJmUISYH/2wcP7IaDFWFMRq6FMy4SQo3+GR7uBdG36POBv2oh9qSgOj4A5MA+GrFpk04HtmGDHqLJbSeb7A89XhZaM6dqMHiB06d1gpx587Jev/12PQd3443escsrvPKK/kOsXQtVXGQXKAQigs/LPiS/kKwDgNzlp5905p/chqQL+Pt5S48f2bW8HeDjZg0hhPgcXqtVQf8+Wkzt2IW07x7oekCnWLz9UVhfJ2erjsFIO6vBg31gf2UIToUPvoZHdmYMc0fl/z26Qtnbq1jxyp6roaFuBSHlKsiOuPm5FMWcqrufqMFg8AA2G0yfDg0aZBXUihX1aNf3319jgvrGGxATo994EQgqkJmtKF+C+tVX0KsXfPqp67ncAs7xzt89n10tb0Pd/wiBQenoAde8uOK1jkDnBI5aG4lvZFZBBXhqM8Qsgvv2QoAVnrxbC6qPDV7YoMv4WeHIRC1QAnzQCtoM0YJ682nYNg36/Q4v3aHLn81l33VXRHXO+prmA8/e6XC/icMm5m5G9Y7rnH87ihsjqgZDMREbC507w2OPZf0N+ec/9Sjjo4+W4sheZ7zzjn6SWLNGhzcXIbnNeebg/fd1vuFVq3Ru2lOnnK+aLMAQY+zZWAZ//iAA7x6LJjnZj94NB1CJs7grrh/xBGW5RHqf/fjYQGVUkyh9vLZGn6da4KH74YPWEJAOC7+EVzPupVngjdvgYiD0eQBGZuzIM2wbbPkExrfTc66v6Tz+VPmP9jDtAukOdgEc11nPz/qPhfHtM95FlL7ubnupFvg0Y0jaI2OrIrkO7RcGM/xrMBQxly9rh+yNN/QKATvXXaeHeu+803XdUsukSXrB7fr1JWeXEZsNnn8elizRglqnjkebT0hNoPUnrYk9F8uDv8PcBVlnVUcwhY8YQX6GhPGPp2e94Sx5YG6WIdoEf7j/AVhdD8qmwNK50PmQvvfE3fBRKxAFFhtYfaDcZfhkGfTYp73TiW31fTu2KPeifk+UhZc7wdc3wNHyeZc//Q582Mr1kO5lX+j9AKx0MnrjcgcccD78e+iQfoCzH35+qCNHzDrVgmBE1eAt1q/XS2L27btyzddXb7Dx0ks68f01x3vvwbvv6j+OfemEt0lN1VuJHTqkA6Y8GH0Mel63/8L+fLH7Cxqe1d5giJNVPDGfWhn2uA+Jie4EMmW2jr8lnplWPd96PgjuGQCba0KVBFgVDc0cnGqbAouD497yuI7uPVEWBveEPypp79dRVHMVMODvQHirA7zdDqQA45/Z20/wh49baI/5VFl9LTANLvu5Ob8rAidO6Hl6u4gmJ8Mdd1w56tZF+fiYOVWD4WrgwgW9QqRz56yC2qYN7NihvdZrUlA/+EB7qWvXlhxBvXQJunfX28J9951nBbVaNVCK99r68MXuLwhJgUVfOBdUgAH/9CEhAR4PnAXY3OxEkWrV860P+07htsFaUOv8DT/OzCqoZ4Ph3oey1p65BCbfCh0HaUG96UxGZDNQ6xL858ecc5n2YdskP3izA9R5WotqQQQV4L8/6teLgfDabbq9MV2vCGqDjAcRtwgKgoYNoXFjWLQImjXTG5qfPKm3mxsyRA8TFdFci1lSYzB4EBG99eeoUXDGYeuwsmV1Aodhw8BicV2/VPPxx/D227BuXcnJs3j8ONx9N9x2G0ye7PkP5/Rpfqp1ZSnLrCXQ4Fwu5TN+6D8E2vMdw/iYREJwz2tVfJb+BHzyMDVvG85PW+ZSI/7K3XV1YEBvOFEOKibpOdREf2gyQt+32OD5DXr7trsztgxsfwSmtcjaS1RnLbLVEnTgUUKAG6blQfCL+tU/HVKzqdKgX2DK11Am7YrYZ2HUKO2J/vWXXtRt90RvuQV8it9vNMO/BkMhiYnRaQWPHNFrTC9nS+HWu7ce8SwpU4deYcYMvRZ17Vq4/npvW6PZvVsL6ogRelFwEXgup0MUzYbDybLwzM8wYXX+2xjB+3zE4+RvYPHKEpx+ai6vdtRznTYfuO0wfLwMHr/3yjKboDT4aYb2aie01V5icRKcqhNNOFLGN5ipX/swcMp6aNHCeUWA117TItqiBfj55atfk/u3gBhRNRQVMTE6tWCSkxSx4eF6tLNnz+K3q0Qxe7bOZrF2Ldxwg7et0WzYAH37woQJMHBgkXSRbkvnzsF+rKsLHQ7Dmjng5+6IbjZi6M/DPlOx2cqSv0SBQmi11Vwc3g0leklN+6Mw/F44nC3tpURB50dgfV3nLTU8qzcTKA6aVrmFL6J+58Yz1rwLF0LDzDpVg6GE8Z//OBfUsmX1EpprXlCjo7Ub//33JUdQ58/XOxPExBSZoAK88P0LrKsL1eLhy/kFF1SAy83mYhtbHv//G0AQcbi/sOf9GswAACAASURBVERx8dRdEGXlrsnvcyoEug/Ugtr8hJ6rtFPzGdeCCkUrqDXL1cz898hWI9m4p617ggp6hKEgw/azZ+e/jhsYUTUYCoDNpqfgTpxwfj8hQQvrNc3cufqp49tvr+Rz9Tbvvqvn4Favhi5diqybr2K/4u2f38Zi05G11RMK3tbuMJ3AAWDGobkkUZ7H+YD8CCv48M3FEUxfkYJlZ38i1+mlNHsdhPJ4uYLbWFBGbNGvx+KOEZoMC7+AKfe8T+D7U/PXkC0fTyxWq/5eFlGSfiOqBkM+OXJEry19+mnXZSIiis+eEsmXX8Izz2jxatTI29boH93Ro2HaNJ1+sGnTvOtkRO7mOPJIVLH//H4eib4fgLe+hY6HC272C3fonWKS/XTKwIG7tJTW6Pgk9B4AfvnzWrH5Y10cw4T9q/i1aPNt5MnjW69sKdfmKPzyMfSOLUSD7iQQiY+H++6DrVt1ov4iwIiqweAmIjBnjo7UX7PmyvXs8S3BwTp24ppl4UJ46im9w8jNNxd//87E0GKBKVPgxx/djzzOLYk76C/EyZNapMuXB6VI9Ffc/1J94v2F+/fAMxudN+Eur3eE3VWgfqX6vD9kETYfxdPddIIGn5vncm+f/HqtAIqEE3fBuDTY1b9wBhaCj1ppbxlgwyyoU9j9x/PauP3QIWjXTu9Tu3q1x9ci2/GqqCqlKiulXldK7VBK/aaUWquUuiWX8jcppVYrpXYppfYppV5TSpkHA0ORc+aMjuJ99FGIi9PXfHx0Ap5Zs/TvtFL6ddo0vd3XNcnixTqaduVKaNLEOza4+nFNS9NJlt0hr+HExo0hJES/x9GjIS4OAYb1gN+rQv1zev1nYeKJ5zs4+Pcu28eq//Sm9r+F99roa9OXwvL6MNH3SbrcUgCvVXxhUQy8v7MQVnqGwsw3u8WPP0LbtjpH6NSp+Y4Szg9ejf5VSnUC6gDRImJVSrUHPgXqi0h6trIWYDfwTxHZqpQKBuYBK0Xko9z6MdG/hsKwZImO8HVcd3r99TrXert23rOrxLFsmV5Y//XXuS+BKGpyWxoTHa0TL//9tz7s/87+Gh+fu7D+8otOIHDjjZki/mEreOIevTxkyydw01nX1XPDvg7UXW45Bb9W02s8m3/Tn03bp4MtiPxkZAKg7mp4pFv+jPUweWVuyhNnejZ7tp5D/fRT6Jb1/V0TS2qUUtuA/iLyR7br1wNfikgLh2uDgOYi4nqjQYyoGgrGpUt63jR7kODjj+tc8GVc7El5TfL119qNX7ECWrXyri25iWq/flChgj5CQ12/li+v80m6wv67mdHXppo6I1GaBT5fAP1/L/zb+LUqNH0c+v8Gcxvns/Ku/rBoFuBPvsW15Qdwb64/qR6h5XGYuhxaDvPg1nKOema1wnPP6R2Hli3TWZayURSiWqIyKimlqgPhwHEnt48CVZRSA4HPgRuBUcDQ4rPQcK2wdq3WiCNHrlyrUQNmzoSuxbwwvsTzzTf6j7V0qfcFNS/mzs27TD45G6yXvKZZ4MnNbghqdkfGYnHqFd+SMYq9K48d5vr9Bh8vh3IpOs1fheegXvhcDkTN1UO755rgbkYmALY9Adseg6h8bJWXD8pf1pmals4FS1H5dHFxeg4mIUEHJBXR/KkzSsx8pFKqBrASeElEcqz8E5FU4D5gLHAa2AVMFpFNxWqooVSTnKyDVu+4I6ug9u8Pv/1mBDUH332n965bvFgnNi5NuLGP6tjOepPwY+Wh7VEYX4CMSa6GmfdX0gK028U2s0EZOx59vlALKkBoRjavA3YNGdlUe57YyN8SHH+IssG4RDfruEf/32Dv+7Di8yuC6jT1YEGwfy6HDkH79kUekOSKEjH8q5RqC8wBokTkcxdlgoAZQC3gE6Ae8P/tnXu8zGUawL+P40gqK5fclUpKoovsJomuii7alm5baYtqdb9f5Ohiu2ypJe2qlJBqE0qJJYSKCJtLaCklhEiFOs5594/nd5w5Y+5nZn4zc57v5zOfOfP+3t+c5/2985tn3ud9Lr2BvzvnngjRvxfeKrZJkybHff11OfzajQrB/PmqH5YFuPXXrAlDhkCPHv7JlbFMm6YXZswYzZ2bKdSrF9pZqW7dhGqgRkL66+quzi+w4J/Q8KcoJ4SSIYS5en59TdKwMcwWQ4vvNf71zRZl9yB/3Aua3QAb9w1x0t+/gp+bEG9Gpt3PBeXLizxkAlyXyC5cpUrh97eDr+esWWo2uOceuOGGqKknc3JP1dsXvQm4yDkXtmqsiDwK7O+c6x3Q1gRYDNRxzv0a7lzbUzUiUVioVWMeegh2BbjHnXUWvPCC/uA1gvjwQ81K9MYbWoqnAjL9q+l0Gt4JgI6rofsSaL5ZPX8b/BSkukq+Z8Mp/ACmHaSl23aEcVD9y2fwj4lQrbBs+/Vnw3Ntowg9YRDMux6VLl7lmrhiffAD6PthAidWqaIrzbw8/cw1jZDy6aWX4K67YMSImE1KObenKiKtgFuBDs65LVG6HwEMDGxwzq0RkR+BakBYpWoY4fjiC7j8co0FL2GffeCpp9T7PkXVobKb2bNVoY4eXSEVasH0AvrP6F+mbXpTfZSw769w2GZPye7ch+aLX6N5reY027KBUIvIEnpcCG+ECe3d91cY+k7oPdt/t4BXYshnQdcb9BH3qjXxG6HNWrhnVhwnlPwAufde/bX7229aezecQi1xSBo3ThXv4YcnLGsy8Duk5lngPefcuyGODQTGOuc+9F73Bs5BQ2q2iEg+0A842Dl3SfD5gdhK1QimuBgGD9YftoFVZU48URM8ZEohlYzj4481ofHIkXDGGX5LkzjlMRE7B489BoMHI9es5a3ub7Fi8wqWb16uj03L2bxjc9jTG27T1WzJqrbk+T+HaJxrNALDTh7oBDsrwxMnRj9vD/57Mbw1kthXrQ4KYnPDuWEODPo95BfBZ/+ClgHhaDzwADz4YIR/47QO6h81KxVz5kDbMEvwQIekN9+Me/8051aqqAfvqSLyWFD7fUBTYLdHgHPuX54inebFrBYCbwJXpklWI0dYswZ69iybFalKFTX/3nZbBa53Go25c1WhDh+e3QoVomdLCsfOnRq0vGQJfPIJvNiYbkd026Pb5u2bWX5CM5ZX2sKKWrC8NiyvBV/W1By7a6vDBwdHFzOvGPrOgIJOe4adbKoGD50c/T3C0mq0PnZ7CEP50lXAbR/Bk+1grLdY7DsjSKHuvz88H6Xa+LJlpQp16tTwCnX1ajj3XP0lPGhQShM6xIPve6rpwFaqBugP4BEj1H+hJCsSaFKcESP8SwCUFcybp7VHhw2Drl39lqb8RLLrh/tO3LABunWDRo00eLlaNQqmF1DQsSDm/1Ek8FUNVbIrasEXtUvz3wZTaztMHAnHfwdSUFapzq+v8Z1JpaCIyKvW8CvVdmvgsSlwy5kwL0Td4N2r6wYN9EfZ6adHl2fcuPBlnmbOhO7d1UTcp0/C+zRW+s0wEmTjRv3xe8UVe6YZnDvXFGpEPvsMunTRFUYuKNREWLhQV0xnnAGvvaYJniG8Qg1DnoNDtsDZK+Gvc2FHGFthnzmwZqAqVCgbdnL+RSlQqKBOSLKD3U5JZQjVprwwHu74CC66UBVq5SJdoYL+EHAFnkKtUgWuukodGaIxfHh4hfrSS3ozv/xyTB6+6cZv869hpJy331ano+A0g8OHq+XIiMCiRbpC/ec/c6c4bLzWuXHj9AM0eHDSYqu256tT0oQQFfEmvwKnryrbVjAdfsuDmzvD+FT64fTzYngKgmuZ7un523U5/G0q9D8Z3jxS237/LTz/Nhz1fYBp2jmtMDFpksao9e+vijDcPDz+uHoPBlNUpE4Q48dnhENSOEypGjnLtm1aOnPYsLLt116raQb3jeSGaWi2i86ddb+q2577hlmJc2oyjLXvo49qoPLEidAmOVbCrVXhnIthVlCxnIs/h8Y/7qlQAb7bDy7sDh833vNYze3wQ7WkiFZKlNCZSSPgm+pwUk/Yujfs8xsMmKqr7z2SOsycqZ+hyy/XIsSNGmn5taFDS1eZr7+uKSSvuQbuuGPPf7htG1xyCWzfrnvZaU7oEA+2p2rkJNOna+a8wJwf9eurgu3sb87w7GDpUi3i/dRT+mWXKzzwgHqk5edrgHIwJd6/O3dqcYDly3Wl2jDERmE0Qpgl1+0LZ/4ZPg9I1rT/DnhuAvRYEvptZjXRfAbrA4re7/erZnBatX9A9qQ0ctAW+Gp//fuslSr/gT+G6dyokebd/d//NOvWqyHz+yihvK9Xr4ZzzoH27ZPukGR7qoYRhZ07Nc1gp05lFepFF8HixaZQY+KLL9SR5IknckuhPvSQPmrVglWrdCUa/Fi/Xh8dO6rSnTEjMYUKe6Q5/LImnPiXsgr1jC/h8yGhFaoDBreFTleUVagXLIXFQ2Dyof4oVChVqBcshXdHRVCooDVnV63S/dRPomSVDfa+njlTS0H17g3PPZcxHr6RMPOvkTPMn68WpqVLS9v231+td7mkG1LKihW6Qh0wILeKwv7tb7rKqVJFN9MbNQrdb8EC3Tu++mro27d8TjDr1+8+f2E96HwZbPC2HPYu1DzB130a2td2ez5c2xVGBFSXbrIVBr8H56yAU65IXKxkcPlCeHIy1N4jS3sIGjXSz1OfPpraMtbC9cOGaVKHLIuJNqVqZD27dul35oMPlk0z2LkzvPiipRmMmS+/hFNP1Qt5hc/f2snk73/XL+jGjXVcXbqE7vfWW7oiGjJE7a1Joud58NYRsM0r+tL2W3hlrCZ9CKagI1y5ELr1gIX1tS2vGG7+RJ2V/t4Ozo2Y6ia1lJh9h4+L0KluXS231q6d/oh59VXNwPXyy3DkkdH/SZY4JIXDlKqR1SxfrqvTuXNL26pV063AXr0yzts+c1m1ShVq375qpst2wmVMWrtWg5JDHatUST9I0QqsFxbC5s0ap7VxI2zaFPbvtw8XXj5G/VbyiuGBGXDvTKgcJj98/44wqG2p49Hxa+Ff78Axri78up7OjYWHOkBxujfuHNz2MfSfFiZ7U82aek0Afvih1JGoevVSF/twP2aCqVxZzbzr1+v7ZhmmVI2spLgYnn0W7ryzbJrBdu3Uunfoof7JlnV89ZXWurv7bv0lkgskkjGpuFjjrl55JbKy/Okn/bKvU6f0Ubu2PjdvrkqkTh0W5m/mghmXgivi8I0wYiy0+S78vx/oVc77oZo6Ig2YqubhvAPUeadgegH9r078kiRK6/Xwwtulsu+uilOpknpEX3SRutmD7kuXKNRvvlHrQCIUFkKLFkmvKpQOTKkaWcc332iawalTS9vy89UH5fbbLc1gXKxZowr19tvhuuv8lsZ/nnmmVEHWqaNJ3IMV5/77q0KJQMH0AvpPL026/0UdOL5X2by9u/t21BVqID/tBZse60deQHKJgo4FVL+3P7elqabvXrtU1ts+gvzglbVzmtNz5Uq4+ebS9gO84q9ffaVm9PIQLWVkhmIhNUbW4Jz6LNxwA/wY4G3YqpUuLlq3Dn+uEYJvv1Uv1z59yn4xZjsvvqiORomQrO/DAPNzcIrBcIw7HLpdBK5fCBmKivj1tVFU/TJ9e9195sCgiUGNJdfnnXf0c7NgQamJ9txztX32bE0Y8mMkl+AYSbF+spAao8KycaP6Olx+eem9WqmSWiznzjWFGjfffadxR9ddl1sKdfhw6NfPbykSWmWdX5K9T6Tso0YNFrY/NK0KFaDWjqCGEgW3Zo3+aBk9ulSh3nKLKtTBg/XGTIZCzVLM/GtkPKHSDB58sK5OLc1gAqxbpybfq69WE16uMGqUZkv64IOM8hgNzNsbb18HXHHKj4xonXoldeESTTcYcVVdWAgXX6zB4O3aqaLt1w+eflqP9+mTcjkzHTP/GhlLuDSDvXtrlISlGUyADRt0hXrppXDffX5Lkzxef10/LFOmqINLpHqpkHgt1VgI979LOPNMDRt54w3dny0hyFV9dQ04OMiIcGGLC3lz6ZvllzGABtvg2fd0pRzRVO2c3nS//JLU/x8RM/8aRnKYMUP3SgMVav368N57mtvdFGoCbNyoYTM9euSWQh0zRk3YkyaVeoxeeqkqrL/+VT3bgjMmhcumlAyimX6bNdNcwoEKNYh2fymrUJvWaAqQdIV67aew9NlS03PEVfX776dXoQZlpMoWTKkaGcXOnWqRDE4z2KOHphk86yz/ZMtqNm1Shdqtm+a/zRXGj1fFOXGieufecosq1sJCLQgweHD47El+MWiQxmKGoEh0tRiYOH/sa3DyQWGqkTs48vvQh047+DReOu+liKLUPfJ4frdlu5p/2NMzuQw9e0Z8r3KTqh85acbXPVURqQ3cCnQG8oFNwM3OuUVh+lcG+gF/BIqA9UBP59y36ZHYSCWffQZ//rOlGUw6P/yguXy7dNFsSdmaESOcWbVGDa2xOWKEerItXuxfGq3ZsyMfj7L6mn5Q6d/Vd8INc9UjmIUvhz5BYOkBQnCt05k9Z9K+SXvaD2sPQL196zFpQUuOqH4wVeoOZQf3U/U/01TBt22r1ywaWark0o3fjkpHAsuBvs65IhE5EXhLRJo753aF6P9PVJke65zbKSIdgGQXPTLSzK5dWmGrf/+yaQbPPFOjIxLNZ24AW7aoQi3J55utChXCm1W3btXg5CVLdI/AL7Zs0UoqkYigmAq67EP/40vNq9uqwiMd4C/LqvJ668r8/NvPu4+9dN5LTFgxgTHLxuA8hXpq01Ppd3I/pq6eypRVUzjppZNK/+3P62ndbD39TrwPZkPVgof1wLHHlgqQn6/evFkaH5opZJyjkojMAy52zq0Maj8eGAa0ds6FSfIVGnNUylzCpRl88km1SGWzDvCdrVtVoZ50kl7QbL+YkeT363ssmlNSMDHKKf2Fb2/5lrun3s3I/44sc+zyVpfzyn9f2eOcfif3oyAgWQRAsSsm78E83ODauoquVImCa5pFNvP6hQ9zmApHJb9XqmUQkfpAQ2BtiMN/Al6JV6EamUlJmsG77oIdAfFwJ5ygoTKWZrCcbNumFQVOOCE3FGqmEa8yhZgdb3YU6g1x2ODD2F5YtgzMS+e9xJVHX8nwbsPZUbiDagOqhU4W4clYacMGKED31Js3B/SlL9SurXvdoWJYs9QpKRQZ46gkIg2AiagpOFRBoSOAlSJyv4jME5EFItJPRCwpXZYwahQcdJDGhu+zD9x4Y6lCzc/XSjMzZ5pCLTc//aQeXcceq2n3TKEmn1gUagKONzsKd9BsUDOAPRTqfSfdx5VHX7n79d75e4d+k3r1dM49GeOJk00p48er9SSVntcZQEYoVRE5AZgOPO6ceyFMt6rAg8A64A9Ae6A1cFeY9+zlKd95GzduTL7QRlyMGqW52r/+Wu+hwCT4Rx0Fn36q2ZEsb285+flnTRHXsqV6vppCzRoKphdQbUA11v4UylAHj8x8BOkvFEwv2N3W7+QQ2aOCFH5GmHpFNFlEBcD3PVUR6QncBFzknPsiQr8RwKfOuX8EtB0GvBrNJm57qv7TuLGmmg2menXNlLTXXumXKef45Rf18D3kEHj++ahJ37OOSAkd0rHSSdH+aTC7ineR/1A+O+7bQX6lfPIq5SH9JbyZNxi/f0jVravhTH/8ozo+jRyZsYHlOZf8QURaoSE1nSIpVI+pwLFBbXlAcIZKI8P46KPQChXUUmkKNQls364JzQ86KDcVKqQ+aUM00uQVW7mSurpUrVyVvEoxmm5KTL5+KtSS+ZgyBX7/e80h+tZbGatQU4Xfd15v4G7n3JbgAyIy0AuZKWE0cLSIXOod3xsYAAxPi6RG3BQVwcMPQ4cO4fs0aZI+eXKWnTvh/PM1nOTFF3NToWYb5XS8CTbrhjTzwh77pzHTpYuaiaIVZI+Xd9/VvNL9+6uTRAX8LPpq/hWR/wCNgeCY1PuAnsAo59y/A/rXB54DWgHbgGGB5uBwmPk3/Xz7LVx2maYbDEe1ajB0qGaUMxJk507NklSjhiY/CJOpx0gC8awC0/W9mujKdNQoDbeqUyc5q9u6deGOO9TTfMwY9TrPAnIupMY5d3qEw+ND9F8HnJ86iYxkMH48XHWVJvIpoX17+NOf4KmntHJUkybwyCOmUMvFr79qPbz99jOFmkri3UvNhvCQSy7R512hcuwkwNlnayzcxx/DgQcm5z2zFLsLjaSxYwfcfrumFSyhUiXo2xfuv1+/82+80T/5corffoPu3XVDetQoU6ipJNbwmWwkWYUVtmzR5BIVbP80FHYnGklhyRLNzxuYQrRxY/2+P+mk8OcZCVBYqBdbRAtF5+f7LVHFJhtWpsEk26FpzJgKuX8aCrsKRrlwTkuxtWlTVqFecAEsXGgKNens2qWmu8JCrcdZpYrfElVscixxQULUrWsKNQBbqRoJ88MPcPXVMHZsaVvVqvD005rowe9wuZxj1y4t4/PLL3rRTaFWbOrVi69/3brJDwvKVrN3CrGfF0ZCfPghtG5dVqG2bAnz5lki/JRQVARXXgmbN2vsnwX3VkwC41EjKcjiYujaVYsTl8SPGmnBlKoRF7t2Qb9+WkQ8MKHDX/+qlWaOPNI/2XKWoiJ1p163Tl2rq1b1W6KKRbg9Uz/2UmNdaQ4cCBs3aqxovOfGSjbuJacBM/8aMfP11xoCE1iHuWZNGDYMzjvPP7lymuJitaWvWaOB9XuHSaJupI5s3DN97DH97KRqi6BSpey8LmnAVqpGTPz732ruDVSoHTvCokWmUFNGcTFcey2sXAnvvKPZMoyKRyIpCP/1Ly33lmzq1lVTclFR8t87RzClakTkl190odS9e2kZxLw8TT84ZQo0auSvfDmLc2pTX7pUV6gW/1dxScRsu3lzcmXIwRJtqcLMv0ZYFi3ScMgvAkodHHighkZmSRay7MQ5zZKxcCFMmqQZkwwjHq6+2m8JKiy2UjX2wDkYNAjati2rUHv00O95U6gpxDm45RaYMwfef1+TnhuGkTXYStUow6ZN0LMnTJhQ2latmirZnj0tVCalOKdJyWfNUtv6737nt0SGYV6+cWJK1djNBx9oZZl160rbjj4aXnsNmjf3T64KgXNwzz0wdao+atTwWyLDsPjWBDDzr0FhIdx7L5x2WlmFevPN8MknplBTjnNadWDiRF2h1qzpt0RGJuHXStFSDyaErVQrOKtWaSrZOXNK2+rUgZdf1mpORhp48EEYNw6mTYNatfyWxsg0wnncJroXEy1doa1Oy4Up1QrM6NGaUvCnn0rbTjtNyyLWr++fXBWKhx+G119XhVqnjt/SGLlMoLI054iUYev7CsjPP6vT0SWXlCrUypXh8cc1gsMUapp49FEYOVI3s80ZxIiXeD4z9vlKG74qVRGpLSIDROQzEflcRKaJSOsYzssTkTtEJM4yDcb8+XDssWreLeGQQ+Cjj9Tx1LZR0sQTT2h+xw8+iL/aiGGAmoWdi5ybOFzShkzKZ5xj+P0VeiSwHDjeOXcUcD/wlohEM0s/CDwA2LdRjBQXw5NPaozpypWl7X/+MyxYAMcf759sFY6BAzWN3LRp0KCB39IY2U6Jcg1+RMp+lMg5Rkz4uqfqnJsBzAh4PVtEtgBNgZWhzhGRrkANYH5ahMwBNmyAK65Q024J++4LQ4aoUjXSyKBB+pgxAxo29FsawzCSTEY5KolIfaAhsDbM8abATUBXYFKoPkZZJk2Cyy+H778vbWvTRp2UDj3UP7kqJEOGwFNPwfTp0Lix39IYhpEC/Db/7kZEGgATgb7Oue0hjlcFngd6Oed+jeH9eonIPBGZt3HjxuQLnOH89hvcfjt07lxWod55p1aaMYWaZoYO1XJcU6dqAmXDMHKSjFipisgJwHCgwDn3aphu/wCecc6tjuU9nXNDgaEAbdq0qVCBVytXwsUXq1NSCXXraqjMGWf4J1eFZdgweOgh3UM9+GC/pTEMI4X4rlRFpCdq0j3XOfdFmD77AV2AtiLykNd8KPC6iCx0zvVIj7SZjXMwYgRcf72WbCvhrLPU2/eAA3wTreIyfDg88IB6+Zp5wDByHl+Vqoi0Am4FOjjntoTr55z7Cd1rDTx3OnCzc25hSoXMErZtg+uug1cD1vn5+Rp7euONFirjCyNHav7HqVPhsMP8lsYwjDTg90q1N3B3KIUqIgOBsc65D9MvVnYxZ44mcli1qrTtsMM0Ef4xx/gnV4Vm9GjdwJ4yBQ4/3G9pDMNIE36vXw4DnhSRxUGP89CwGotEjkBxsSblad++rEK96irdTzWF6hNvvAG33gqTJ0OLFn5LYxhGGvE7TvX0CIfHRzm3Y3KlyS6++05DZaZOLW2rXl2dTHvYDrN/jBmj9vbJk6FlS7+lMQwjzfht/jUSYMIEzd27aVNp2x/+oPupTZv6J1eFZ/x49RJ7/31o1cpvaQzD8AG/zb9GHOzcCTfdBOecU6pQReC+++DDD02h+sqECdCrF7z3ntndDaMCYyvVLGHZMo09XbSotK1BA3Uw7dTJP7kMtLj4VVepYj3uOL+lMQzDR2ylmuE4By+8oN/VgQr13HP1tSlUn5k8WRMrv/02tG3rtzSGYfiMKdUMZutWdTq65hrYsUPb9toLnn0Wxo2D2rX9la/CM3UqXHYZjB2rm9qGYVR4zPybocyerbGna9aUtrVoobGnRx3ln1yGx/Tpao9/80048US/pTEMI0OoECvV2lm0pCsq0jSxHTqUVai9e8Onn5pCzQhmzoTu3TUetUMHv6UxDCNxNkXvEh/iXO7nmheRxcBOv+UoJ7VJwQcgzWT7GLJdfrAxZAo2hsygqnMuqQHlFcX8u9M518ZvIcqDiMyzMfhLtssPNoZMwcaQGYjIvGS/Z4Uw/xqGYRhGOjClahiGYRhJoqIo1aF+C5AEbAz+k+3yg40hU7AxZAZJH0OFcFQyDMMwjHRQUVaqhmEYhpFy9blxLQAACVBJREFUsk6pikhtERkgIp+JyOciMk1EWofp+4yILBORhQGPF4L6NPPeY7GIzBGRlGdDj3MMjwTJv1BEVovIhd7x20VkRdDx91Isv4hIHxGZISLzRWS5iDwQ5Zy2IvKJd52niUjToONpnYd4xyAieV7/2SLyXxFZUDIH3vGMn4dMvB8SGEPG3Q9B8tUTkbuj9Il4jUXkABF5W0SWeJ+zSCUyk06MY7hURD7w7oXPReT6gGM9ROR/wfOUesnLyBdxDLF8ThKeB+dcVj2Ak4ErgDzv9YnA/4DKIfq+DJwf4b3ygPnAcd7rI4DFwN6ZMoYQ5wqwBKjvvS4Abk7zHFQDngRqeK+rA9OAS8L0rw58DjT1XncA5vg5DwmOYQBQ3XtdD1gGtMyieci4+yHeMYQ43/f7IUCWysAEYGt5rjEwCTjb+7sBsAhomClj8Po9ARwQMGezgbO811cCT/sxB3HMQ9TPSaLzkHUrVefcDOfccOdckfd6NrAFSKTwWTf0y32+917LgHeAS5IlbyjKOYYzgc+dc+tSKWMknHPbnXO3Oee2eq+3AeOA5mFO6QWMcM6t9vp/CKwSkVO942mfh3jH4Jzb5py71+uHc2498C5wQqpkjEYC8xCNjJ+HEPh+PwTwN2BElD4Rr7GIHAcUOefe845/BwwBrk2V0EHEMgacc3c45773/t4GjAbap1i2WIlpDJEozzxknVINRkTqAw2BtQmc3hGYEtT2H+CUcooVF3GOoQ8wOLUSxYeIHA5cCrwZpktHIl/naMdTTgxjCO6fh1oclqVSrniIdwwh6EiWzQMZcj+IyPnAXs6516N07Ujka9wpyvGUEccYQnEKGXAvlHMMgSQ8D1mtVEWkATAR6Ouc2x6mWzcRmSwii0TkXRFpFXCsMbA+qP96rz0txDiGkr5NgQbOuVlBhzp4Y1vg7XOcnCp5A2Rp7u1DfImar2YCX4TpHu06+zIPcY4h8LwqwHBgedBcZPo8QAbeD+WYh4y4H0TkUOB64PYYumfqvRDPGALPExF5DKiJrlZLaCkiY7x98tki8sckihtOlnjHEOlzkvA8ZK1SFZETgOnA4865F8J0exf4BN1Hag08A7wrIjW943sDoWKK0hJnFOMYArkeNUEEMgvdz7jMOXcMcCcwQkSaJVPWYJxzy51zRzvnDgVqAPsBA8N0j3adfZmHOMcA7LYqTAO+Q/eOSsiGecjI+yGRefDw/X4Qkb3RWMdrnHO/xXBKxt0LCYyh5LzqqKm+Abr3WOQd+hy9R25wzh2H3iePiEjKqk8kMIZon5PE5yFdm8fJfAA9gYXA4QmcOxI41/v7ZaBb0PFTgJGZNgZvkr8EqsXQ92HgxjTPSS3g6zDHpgPHBLVdBTzs9zzEOgbv+HHoCurcGN8vo+YhTP+MuB/iHUOm3A/onu5q714ueRR5z3eE6B/xGqMONLcEHT8YmJXC6x3XGLxzmqLKs1eM/+Nq4KlMGkOkz0l55iHrEup75qpbgQ7OuS0JvEV1YKv39yz0Az024PgpwAflEjIKCY7hEmC8i2Ii9qgOrEhUvgSpBXwf5ljJdV4Q0HYKMCzoeFrnIQRhxyAiVVEFdIlz7rMY3y/T5iEUvt8PIYhlDBlxPzjnJhHkYCgiW51zR4c5Jdo1ngXcRNmVekrnIN4xiIgAo4C7nXPvxvhvAj9nSSeBeQhF4Ock8XlI1S+HFP4ieRboEubYQFRRlby+AmgV8Lonav7K917vg7rjt/Zet0R/fVXNlDEEtM8DDgnRfj7QPuD1WcBSvBCFFMmfh/7iru29/h26qX9ZmHlo6MnUxHvdEfgo4Hja5yGBMfwJeCLC+2XDPGTc/RDvGDLxfgghw9aAv+8EegS8jniN0S25T4DTvdeN0FCO+umSP4YxHA/8O8K5JwNdgUoB/VcAB2XQGCJ+TsozD1m3UgUOA071NscDuQ/9pVI3oG0N8KiI1AXygU9RZVYI4Jz7RUT+BDwnIrWBbcClzrlU116NZwyISDtgg3PufyHe6yvgfhF5Bh3jCqCz80IUUoFzrkhElgGTRGQv4DfgH865kV6XMmNwzq0Vkd7AmyKyL7ofeVHA8bTPQ7xjQOfsIhE5K+itxjnn7icL5oEMvB8SGEPG3Q9RaAgUl7yIdo2dc8WeU8/zIvI0ej1ucv6GDJUZA3ovdBCtUx3Ip865nujnrD/QX0Ty0fv9AufcV+kQNgzBY/iKCJ+T8syD5f41DMMwjCSRtd6/hmEYhpFpmFI1DMMwjCRhStUwDMMwkoQpVcMwDMNIEqZUDcMwDCNJmFI1DMMwjCRhStUwchAR2VtEHhCRL0Rkp4h8IyIDvLhBwzBShMWpGkaO4SX9/w8apD8WDXTvCrQAhjrnevsnnWHkNqZUDSOH8MrSfQQcDpzpnJvtte+LpsdrBDR0WmTdMIwkY+Zfw8gtbker6dxVolABnHM/o6vWSsBJPslmGDmPKVXDyBG8mpJ3AOvQ2pLBbPae66VNKMOoYJhSNYzcoRta5PvVkiT5QVT1nmMuRG0YRnxkY5UawzBC08V7bigiBSGOn+Y9f5MecQyj4mGOSoaRI4jI10CTGLo29bkMl2HkLGb+NYwcQET2QRXqYuecBD+A/YBC4NtAhSoi14vIai+Wdb6ImBOTYZQDU6qGkRs09J6/C3P8TLQY87slDSLSA3gGGAAcg4biTBSRWFa7hmGEwJSqYeQGVbznX8Mc7+k9DwtouxV42Tn3vHNumXPuBtRz+LoUyWgYOY8pVcPIDUqSOewRLiMifwDOBiY65+Z6bVXQeNbJQd0nA+1SKKdh5DSmVA0jB3DObQKWAceJSKuSdhE5EHgN+BG4PuCU2kAesCHorTZgcayGkTAWUmMYucPDwChgqoiMBPYBugMO6BLG4zfY/V9CtBmGESO2UjWMHME59ypwJfA9ui96NvAG0NI591FQ901AEXuuSg9gz9WrYRgxYnGqhlFBEZE5wCLnXK+AthXAGOfcPf5JZhjZi5l/DaPi8hQwQkTmArOBa4EGwD99lcowshhTqoZRQXHOvS4itYD7gfrAYuBs59zX/kpmGNmLmX8NwzAMI0mYo5JhGIZhJAlTqoZhGIaRJEypGoZhGEaSMKVqGIZhGEnClKphGIZhJAlTqoZhGIaRJEypGoZhGEaSMKVqGIZhGEnClKphGIZhJIn/A0uAODaSARC9AAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x1a233c0b38>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "theta_path_bgd = np.array(theta_path_bgd)\n",
    "theta_path_sgd = np.array(theta_path_sgd)\n",
    "theta_path_mgd = np.array(theta_path_mgd)\n",
    "\n",
    "plt.figure(figsize=(7,4))\n",
    "plt.plot(theta_path_sgd[:, 0], theta_path_sgd[:, 1], \"r-s\", linewidth=1, label=\"SGD\")\n",
    "plt.plot(theta_path_mgd[:, 0], theta_path_mgd[:, 1], \"g-+\", linewidth=2, label=\"미니배치\")\n",
    "plt.plot(theta_path_bgd[:, 0], theta_path_bgd[:, 1], \"b-o\", linewidth=3, label=\"배치\")\n",
    "plt.legend(loc=\"upper left\", fontsize=16)\n",
    "plt.xlabel(r\"$\\theta_0$\", fontsize=20)\n",
    "plt.ylabel(r\"$\\theta_1$   \", fontsize=20, rotation=0)\n",
    "plt.axis([2.5, 4.5, 2.3, 3.9])\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "지금까지 논의한 알고리즘을 선형 회귀를 사용해 비교함(m은 훈련 샘플 수, n은 특성 수). [표 4-1] 참조"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<img src=\"./images/table4-1.png\" width=\"80%\">\n",
    "<center>**표 4-1 선형 회귀를 사용한 알고리즘 비교**</center>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---\n",
    "## 4.3 다항 회귀"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- **다항 회귀**<sup>Polynomial Regression</sup> : 비선형 데이터를 학습하기 위해 각 특성의 거듭제곱을 새로운 특성으로 추가하고, 이 확장된 특성을 포함한 데이터셋에 선형 모델을 훈련시키는 기법"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "먼저 간단한 **2차방정식**<sup>quadratic equation</sup>으로 비선형 데이터를 생성(약간의 노이즈 포함)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 99,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYMAAAEUCAYAAADJB1rpAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvNQv5yAAAF7hJREFUeJzt3XuwrXVdx/H3lw0BMUgJKBoyWQZYYZZbbatjJzHNNHGG0S46jGYdUzIYuwyUR80z4ykZrzWVJ+xieatQu5hZnjgO2ibcx1BOKVlNaY4MCqSkCHj49sfaWzaLtfZ+1trP83su6/2aObNnr7P2Wr91eX6f3+35PZGZSJIW21FtF0CS1D7DQJJkGEiSDANJEoaBJAnDQJJEg2EQEc+MiNPGbvuRiPjniPjXiHhPRJzc1PNLkqprJAwi4nTgMuC0Tbd9G/BK4Icy8zuBdwCXN/H8kqTZ1B4GEfFe4CPAA8f+6yXAqzPzCwCZ+Q7gfhHxHXWXQZI0m9rDIDOfmpkPAP5x7L92AR8Yu+3A+u2SpBYdXfC57p+ZN4/ddgPwoEl3jojdwG6AE0444RFnn312w8WTpH768pfh+ushEyLgrLPghBPg0KFDX8jMU6s8RskwOG7K7RM3R8rM/cB+gOXl5VxbW2uqXJLUa/v2wZ49cOQIHHUUXHABXHopRMR/V32MkktLb4qIbx677TTgMwXLIEmDs2sXfMM3wNLS6OeuXbM/RsmewYeAJwBXbLptF/D8gmWQpMFZWYEDB+DgwVEQrKzM/hglw+C1wJsi4h8y85aI+Engxsz8VMEySNIgrazMFwIbioVBZn40Il4F/ENEHA8cBn6m1PNLkqZrLAwyc9eE294NvLup55Qkzce9iSRJhoEkyTCQJGEYSJIwDCRJGAaSJAwDSRKGgSQJw0CShGEgScIwkKReWV0dXb9gdbXexy25a6kkaQdWV+Hcc+GOO0bXLThwYGc7lW5mz0CSeuLgwVEQHDky+nnwYH2PbRhIUk/UcUWzaRwmkqSeqOOKZtMYBpLUIzu9otk0DhNJkgwDSZJhIEnCMJAkYRhIkjAMJEkYBpIkDANJEoaBJAnDQJKEYSBJW2rq+gFd495EkjRFk9cP6Bp7BpI0RZPXD+gaw0CSpmjy+gFd4zCRJE1Rx/UDVler/f3m+0Ez1yzYimEgSVvYyfUDqs45bL7f0UdD5mhoquQ8hcNEktSQqnMO4/e7887y8xTFwyAizouIayLicER8LCJ+onQZJKmEqnMO4/c75pjy8xRFh4ki4ruAXwOelJk3RsSDgIMRcV1mHi5ZFklqWtU5h/H7XXcdXHEFnH/+cOcMvh84kJk3AmTmZyLiQ8BZgGEgqXe2myCuOuewcb/VVbj44tEQ0VVXwTnnDHPO4BDwrIhYBoiIpwGPBj5QuByStGMbE7979ox+1nGWclvnNhQNg8y8FrgEOBARNwB/BDwnM784ft+I2B0RaxGx9vnPf75kMSWpkiYq7rbObSgaBhHxEOBFwPuAi4ArgL+LiB8Yv29m7s/M5cxcPvXUU0sWU5IqaaLi3pg/2Lu37PYXkZllngmIiKuBN2bm2zbd9izgwsy8VyBsWF5ezrW1tRJFlKSZVD2pbN777+TvI+JQZi5XedzSE8hnAn86dts/A/cpXA5JqsUsJ6XtdOO7JjfOKz2B/H5gT0QcDRARpwBvAH63cDkkqbidzjE0OblcOgx2AycB10bEx4G/BH4/M99UuBySVNxO5xianFwuOkyUmbcCF5d8Tknqip1ufFfHxnnTFJ1AnpcTyJI0u1kmkN2oTpJkGEiSDANJEoaBJAnDQJKEYSBJwjCQJGEYSJIwDCRJGAaSJAwDSRKGgSQJw0CShGEgScIwkCRhGEiSMAwkqRdWV2HfvtHPJhS97KUkdcHqajOXjmzK6iqcey7cccfo2scHDtRfbsNA0kIpUbHW7eDBUXmPHBn9PHiw/jI7TCRpoUyqWOvS1FDOrl2j4FpaGv3ctavexwd7BpIWzEbFutEzqKtibbLHsbIyerwmh7YMA0kLpamKtemhnJWVZoezDANJC6eJirWpHkcphoEk1aDEUE6TDANJqknTQzlNcjWRJMkwkCQZBpIkDANJEoaBJAnDQJJEC2EQEfeLiLdGxCci4nBE/FHpMkjSLJreProLip5nEBHHAX8PvDEznx0RS8AFJcsgSbPo4y6n8yjdM3gx8OHMfDNAZh7JzD8oXAZJBQylNd3kLqddUvoM5GcCFxZ+TkmFdak1vdML2fR9z6GqSofBQ4GbI2I/8Cjgq8BrMvPPxu8YEbuB3QBnnHFG0UJK2pkSF2Opoo5Q6vueQ1WVDoMl4PeAX8vM3RHxrcDfRsTnMvNDm++YmfuB/QDLy8tZuJySdqArrem6QqnPew5VVToMbgIuzsyPA2Tmf0XEa4FnAB/a8i8l9UZXWtNdCaU+KB0GB4DvAz6+6bYl4LbC5ZDUsC60prsSSn1QOgxeBfxNRFybmddGxOmMJpR/rHA5JC2ILoRSHxQNg8z8t4h4DrA/Ik4CbgYuysx/KVkOSdI9Fb+4TWZezWglkSTVbqdLSRdVpZPOIuJ3IyIj4oET/u+siLgjIt5Qf/EkqbqNpaR79ox+9v2Et5KqnoG88ZZOatG/DvgS8Io6CiRJ89q8lPT22+EVrzAQqqoaBlev/7xHGETEU4GnAC/LzFvqLJgkzWpjKelRR8Fdd8EHPmAPoapKYZCZ1zOa7P16GETEMcBrgcPAmxopnaRBaXq/oo2lpE984t2BMG0/oaHsnVSXWSaQrwYeGxGRmQlcBJwJPDEzjzRSOkmDUWq/opWV0fDQVVdNP9msS3sndcUsu5ZeDZwEnBUR9wP2AO/JzAONlEzSoJTc/XOjh7B37+SKflF2Ip3FLD2DzZPIjweOBX6h9hJJGqTSW0NsdbKZ21Tc2yxh8E/AXcDzgccBl2XmfzZSKkmD06WtIbpUlq6I0fB/xTtHXAd8N3ADcGZm3tpUwTZbXl7OtbW1Ek8lSYMREYcyc7nKfWe90tk16z8vLRUEkqTmVQ6D9aWku4A1wIvYS9KAzDJn8IvAg4Fn5yxjS5KkztsyDCLivsCTgYcBvwS8dn2jOUnSgGzXM3gy8DbgRkZ7EF3SeIkkScVtGQaZ+Xbg7YXK0lluiStp6Ipfz6BvPG1d0iKYdWnpwvG0dUmLwDDYxsZp60tLnrYuabgcJtqGp61L3eRcXr0Mgwq22vBK6pJFqSCdy6ufYSANRB8qyLrCatJcXtdea98YBtJAdL2CrDOs3IK6foaBNBBdryDrDCvn8upnGEgD0fUKctaw2m5Iybm8ehkG0oB0uYKcJaz6MP8xNJ5nIKmYlZVREBw8OKrwp5l2sufqKuzbt/Xfaj72DCQVU7XFP2lIyd5CsxauZ2DLQmpP1e1dNoaU9u69u9J3a5hmLVTPwJaF1K5ZJpHH5z+6vlqq7wYXBlutQOj6OmxpCLY6Budd8bTxmK9/Pdx0UzdXS/XdoMJgu5a/LQupWVV637OueLJHX0av5wzGx/+3G1OcNA4pqT5NjOs7V1BGb3sGk1oLVVr+XV6HLfVdE71ve/RltBoGEXEa8NzM/PVZ/3ZSa+HSS7t9BuYsFmX3SQ1LE2dBd/3M6qFoLQwi4mjgcuBxwMxhMK21MISWv2Ok6rMhHIOLqM2ewT7gjxmFwcyG3Fpw1ZN0NxtHZbQSBhHxDODYzHxnRLypyt9MGjYZagvEMVLpbjaOyigeBhHxEOBFwNO2ud9uYDfA/e//7QvVMhhyr0ealY2jMiIzyz1ZxPHAe4HnZeZ/r9/2v5n5TVv93emnL+cNN6xx5MjowvR7944miyUtBhdUzCciDmXmcpX7lu4ZPB54MPAXEbFx24kRcS3w1sy8bNIfnXgi3HyzLQNpUQ11SLhLioZBZr6fURh83XrP4OFb/d0JJzhsIklN6s1JZ7YMpMXmUFGzehMGkhaXy0ub1/reRNtNHkuS+xM1r/UwkBaRF1mazcby0qUlF5E0xWEiqbC+D3lsN3bfxNi+5940bxBhMOuXz4kotan0GbV1ft+3C7Img85FJM3qfRjM+uXre6tM/VfyjNq6v+/bBZlbR/RX7+cMtptYmvUCOFLTSl5kqe7v+3Zj947t91fvewZbtbLmvQCO1LRSQx51f9+3G7t3bL+/eh8GW335hn4BHGk7TV1sZqvHcWy/n4puVDev5eXlXFtbm/nvmpgfcPJZUl90eaO6oupuFTn5rD6wwaJ5DDoMoN4u67TJOA88dYUNFs1r8GFQp/HJuJNP9sBTt7i0U/Pq/dLSec2zHcD4ksCbbnKZ6iRutdAel3ZqXgvZM9hJV3p82MllqvfkMEW7XNqpeS1kGIx3pd/ylvkOHg+8e3OYon0u7dQ8FjIMNo/9Ly3Bm98MX/saHHPM7JXXoh944ytXPKlP6qeFDIPNLfprroH3vGd0+0YvoQuVex+WB04bErK3VE0fPmMtjoUMA7i7Rf/CF7Zdknvry7j7tCGhRe8tVdGXz1iLY2FXE2244AI49liIGP284IK2S9SfzfRcuTK/vnzGWhwL2zPYsLICV17Zre56X8bdHRKaX18+Yy2OQe9N1GeOJw+fn7GaNsveRIaBJA3ULGGw8HMGkiTDQOolt/xQ3RZ+AlnqmxLLUp3PWDyGgdQzTW/54TkQi8lhogZN6srbvddONX1+h+dALCZ7Bg2Z1LqCbrW4HArop6bP7/AciMVkGDRkWuuqKzt6OhTQbzvd8mOrhoAnEy4mw6AGkw6saa2reVtck55jJy37Jsed7XF0W5WGgPtLLR7DYIemHVjTWlfztLiaGHJqaihgKD2OIQea15zQJIbBDm11YE1qXc3T4mpiyKmpoYAhVDRDCbRpnBPQJIbBDtVxYG3XCq17yGlDE0MBQ6hohhBoW3FOQJMUDYOIOAV4CfDDwDHAF4CLM/NjJctRp50eWFXHb+sacmraECqaIQTadpwT0LiiG9VFxA8A3wr8SWYeiYjHAm8BzsrMr037u3k3quvDuO++fbBnz6gVurQEe/fCpZe2XSrV8d3pw/dPwzbLRnVFewaZ+UHgg5t+/3BE3AI8GPhUnc/Vl3HfSa1QK5F6jL+Ps7yvdSzd7MP3T9rQ6pxBRDwA+Bbgs3U/dl/GfceHVcBKpA7jlfHrXw8XX1zufS35/bPxoDq0FgYR8UDgb4A9mfmVCf+/G9gNcMYZZ8z8+H0a993cCt23rx8h1nXjlfEVV5R9X0t9/+yBqC6t7E0UESvAQeDVmXn5pPtk5v7MXM7M5VNPPXXm59hoce/d268DxOsK12P8fTz//LLva6nvn/sIqS7FewYR8TzgIuDpmfnJJp+riysmtuvSD2E1ThdMeh/POafs+7rxHBsVdBPP2acesLqt9GqihwFvBR6fmbdU/buhXPayT136UuPQQx7vLvV5D/k91M50djUR8ALgklmCYEj6MqldshLrSzjOo9Tn3cUesPqn9JzBmcBrIuLw2L/zCpejFX2ZDyg1Dj308e55Pm+vd6G2lD7P4IdKPl/TZu2e92U+oNQ49NDHu2f9vIfeU1K3uTfRnOY9cPvQpS8VWls9z1DGwWf5vPsyjKhhMgzmNPQDd7wS28nZvLM8z8ZzLWILeeg9JXWbYTDFvDuJDlHps3mHHrTT9GUYUcNkGEywk51Emy5XGxVF6bN5Fylox/VhGFHDZBhMULVlWvLAbXPoZLxyPv98uOqq5iprW8hSeYbBBF1smbY5dNLG2by2kKWyDIMJutgybTugxitnK2tpWAyDKbpW2XUxoOrW1JxIybmWoSyJ1eIxDHqk6YBqsyJrak6k5FzLoi6J1TC0soW1umejItuzZ/Sz9HYITW1NUXLLiza213D7CtXFnoGA9tf2NzUn0sTjTutBlZ7XsSeiOhkGAroxQd3EnEjdj7tVBVx6XqftANewGAYCujFB3dScSJ2Pu10FXHLhQdsBrmExDPR1XVtB1UVdqoC7EOAaDsNAmkHXKmADXHUxDKQZWQFriFxaKkkyDDSfRV/fvuivX8PjMJFm1sb69i5t89C19f1dem/UX4aBZlZ6fXvXKt8qr79UBd2190b95TCRZraxvHJpqczyyja2edjKdq+/5NYeXXtv1F/2DDSz0ssru7S2H7Z//SV7Tl17b9RfkZltl2Fby8vLuba21nYx1KI+jYuXHrrp03ujsiLiUGYuV7qvYdCcRThIF+E1zsP3RV0wSxg4TNSQJlqHXatgnLyczhPT1DdOIDek7om9tq83MImTl9JwGAYNqXvFTRcr3tKriiQ1x2GihtS94qaLq0a6tmnbvLo2/Ca1wQnkHulbpdWH8jrvoSFzAnmg+jQpWbWSbTswvFqYNGIYqBFVt2xou1XexeE3qQ2GgRpRpZLtQqt8KPMe0k4ZBmpElUq2K63yPg2/SU1pJQwi4rnALwDHAP8IXJiZt7VRFlUzz9j+dpWsrXKpO4qHQUQ8DngO8OjM/EpE/DKwD7i4dFlUTZNj+7bKpW5o46SzS4BfycyvrP9+GfCUiDiphbKogi6e8CapXm2EwfcCH9n4JUcnOvwT8KgWyqIKPNNYGr6iw0QR8c3Al/PeZ7rdADxo7L67gd3rv94eEYcLFLEtpwBfaLsQWzvxBLjPibfd9qVbH/OYW7884x/34PXtyJBf35BfGwz/9Z1V9Y6l5wyOB6ad8nyP2zNzP7AfICLWqp5F10e+vn4b8usb8muDxXh9Ve9bepjo88B9J9x+GvCZwmWRJK0rGgaZeSdwfUQ8YuO2iAhgmU3zCJKkstqYQH418KqIOG7990uA92XmF7f4m/3NF6tVvr5+G/LrG/JrA1/f17Wya2lE/CxwIXAscCXw85l5e/GCSJKAnmxhLUlqllc6kyR1Pwxi5Oci4oMRcSgiro+Il7VdrrpExCkR8aqI+GhEXBcRV0bE97RdriZExDMj4rS2y1GXiHju+mf2yYj4/Yg4vu0y1W1onxkM/5ibt87sfBgwOjfhwcB5mfkI4JHAD0bET7ZbrNp8F3A98MjMPAd4KfCuiBjUjrIRcTqjrUcGUbGM7bF1NvBJRntsDcbQPrNNhn7MzVVn9nLOICIuAu6bmS9vuyxNWD9R5Ccy81Ntl6UOEfFe4PuAk4FHZea1LRdpxyLir4FXZuY1678Ho0B41DYr43phiJ/ZVoZ2zI2rUmf2oWdwDxFxNvBs4M/bLksTIuIBwLcAn227LHXJzKdm5gMYbVc+FIPeY2ugn9lEQzzmNqtaZ/YiDCLirIi4NiL+HTgMXMWoFTYoEfFA4H3Ank27uqpjZtljS9021GNunjqzF2GQmddn5sMz8yHANwEnAq9ruVi1iogV4CDw6sy8vOXiaGuV99hSdw35mJunzuxUGETEiyPi8KZ/zxq/T2b+H3Ap8KPlSzi/rV5bRDwP+B3g6Zn5tvZKOb8qn92AuMdWzw3hmKuqap3ZqdnzzPxN4Dcr3PVk4MaGi1Oraa8tIh4GvAR4fGbeUrxgNZnhs+u9zLxzfbneIzLzENxjj60Xt1s6bWcox9yMtq0zO9UzmCQiliLiDyPilPXfTwJ+G3hDuyWrzQuASxboSzkU8+yxpW4Y9DE3b53ZqZ7BJJl5JCI+Abw/Io4F7gDemJl/0nLR6nImcG5E/MbY7b+amX/RRoG0vcz8y/XJx4+sfy+vBH6+5WKpmkEfc/PWmb08z0CSVK/ODxNJkppnGEiSDANJkmEgScIwkCRhGEiSMAwkSRgG0rYi4viI+J+I+PT6STyb/+/yiDgSET/eVvmkOhgG0jYy8zbg5Yy2p37Rxu0RsQ94PvDizHxHS8WTauEZyFIFEbEEfAy4H/BtwE8z2hL45Zn5yjbLJtXBMJAqioinAX8FHACeAPxWZrofkQbBYSKposz8a+CjwLnAO4GLxu8TERdGxDUR8dWIOFi4iNLcOr9rqdQV6xfsefj6r7dOuOwlwOeAXwceCayUKpu0U4aBVEFEPAn4Y+DdwJ3AT0XE6zLzE5vvl5nvWr//GeVLKc3PYSJpGxHxaOBdwIeBZwMvBe4C9rVZLqlOhoG0hYh4KPBe4N+AZ2Tm7Zn5H8CbgfMi4rGtFlCqiWEgTbE+1PN3wBeBp2Tmlzb99yuB2xhd/lLqPecMpCky89OMTjSb9H+fA76xbImk5hgGUo0i4mhGx9XRwFERcRxwV2be0W7JpK0ZBlK9Xspo64oNtwEfBHa1UhqpIs9AliQ5gSxJMgwkSRgGkiQMA0kShoEkCcNAkoRhIEkC/h93CCL9YP+GIwAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x116c28208>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "np.random.seed(42)\n",
    "\n",
    "m = 100\n",
    "X = 6 * np.random.rand(m, 1) - 3\n",
    "y = 0.5 * X**2 + X + 2 + np.random.randn(m, 1)\n",
    "\n",
    "plt.plot(X, y, \"b.\")\n",
    "plt.xlabel(\"$x_1$\", fontsize=18)\n",
    "plt.ylabel(\"$y$\", rotation=0, fontsize=18)\n",
    "plt.axis([-3, 3, 0, 10])\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "사이킷런의 PolynomialFeatures를 사용해 훈련 데이터를 변환"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 106,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([-0.75275929])"
      ]
     },
     "execution_count": 106,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from sklearn.preprocessing import PolynomialFeatures\n",
    "poly_features = PolynomialFeatures(degree=2, include_bias=False)\n",
    "X_poly = poly_features.fit_transform(X)\n",
    "X[0]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 107,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([-0.75275929,  0.56664654])"
      ]
     },
     "execution_count": 107,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "X_poly[0]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "이 확장된 훈련 데이터에 Linear Regression을 적용"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 108,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(array([1.78134581]), array([[0.93366893, 0.56456263]]))"
      ]
     },
     "execution_count": 108,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "lin_reg = LinearRegression()\n",
    "lin_reg.fit(X_poly, y)\n",
    "lin_reg.intercept_, lin_reg.coef_"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 109,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYMAAAEUCAYAAADJB1rpAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvNQv5yAAAIABJREFUeJzt3Xd4VGXax/HvkxCqICJGBKTpig1EjWgABV27WFZFERXbgr2u64pldS2g8u4qiA11dcWyuK6iro2ioEJQQRFQEcVFVhSx0kPa/f7xJCGElJlk5pw5k9/nuuaaZOo97dznPOV+nJkhIiINW0bYAYiISPiUDERERMlARESUDEREBCUDERFByUBEREhiMnDODXLOtat02dHOuY+cc5865yY557ZN1vOLiEjskpIMnHMdgdFAuwqXdQNuAQ4zs92BfwKPJOP5RUQkPglPBs65V4APgPaVrroKuMvMfgQws38C2c653yQ6BhERiU/Ck4GZHWNmOwCzKl01AJha6bJppZeLiEiIGgX4XNub2c+VLlsB7FjVjZ1zw4HhAC1atNh31113TXJ4IiLRtG4dfP45mIFz0L07tGgBc+fO/dHMtovlMYJMBk2rubzK4khmNh4YD5CTk2Nz5sxJVlwiIpE2ahTceCMUF0NGBgwdCiNGgHPu61gfI8ihpT8557apdFk74H8BxiAiknYGDIDGjSEz058PGBD/YwR5ZPAucAjw7wqXDQDOCzAGEZG0k5sL06bB9Ok+EeTmxv8YQSaDvwEPOefeNLNfnHNDgJVm9kWAMYiIpKXc3LolgTKBJQMz+9A5NxJ40znXDFgIDAvq+UVEpHpJSwZmNqCKy14AXkjWc4qISKlH4pvTG2QzUdKsXr2alStXUlhYGHYokZCVlUV2djatWrUKOxQRSYaFC+GCC+K6S+STwerVq/n+++/p0KEDzZo1wzkXdkgpzczYsGEDy5cvB1BCEEk3ZnDxxX6caRwiX7V05cqVdOjQgebNmysRxMA5R/PmzenQoQMrV64MOxwRSbRnnoG334a2beO6W+STQWFhIc2aNQs7jMhp1qyZmtVE0s3q1XD11f7vO++M666RTwaAjgjqQO+ZSBr6y1/gu+/ggAPg7LPjumtaJAMRkQbvk09gzBhfnOi++3xdijgoGURYcYUOIrMqSzyJSENgBhdd5DuNL7gA9tkn7odQMkhBy5cvp2PHjgD07duXqVMrV/72brvtNnr27Env3r25+eabA4xQRFLKk0/6TuPttoPbb6/TQygZhOj111+ne/fudOnShWHDhlFUVAT49vysrCwAmjZtSqNGm0YAX3nlley1114ccMABTJkyhebNm5ORkcGMGTPo378/++yzD9dff30or0dEQvDrr5s6jUePhm0q1wONTeTnGUTVL7/8wgUXXMC0adPo0qULJ5xwAo8//ji///3vMTMyMzMBP0GsYhPQX//6VzJK2wLXrl3LAQccgHOO6dOn07RpdVXCRSRt3XADrFwJBx7oa1fXkY4MQvL0008zePBgdtppJzIzMxkxYgQTJ07kjDPO4NBDDy2/XVlSKJORkUFxcTGTJ0+mf//+XHrppQwbNoy+ffsybdq0oF+GiAQsL8+vX5CXB8ydC/ff72tX33ef7zyuo/Q8Mghr2GQcnbhffPEFuRVKDO65554sW7aMKVOm8M033zCgtCC5c678yODrr79m6NChLF68mP79+zNhwgR23313AHJzc7n99ts555xzOOigg5gwYYKGj4qkmbw8+O1voaAAmmYVs6LrhWxlBldcAT161Oux0zMZREBxcXF5cw/4Pf6SkpIqb1uWDDp37swTTzxBdnb2FhPt9ttvPyZNmkRhYSGrVq1SIhBJQ9On+0RQXAxnlzzEVp99AB06wE031fux0zMZRGCYZefOnVm0aFH5/1999RVdu3bl7rvvZtGiReUJoGz46GeffcZxxx2Hc46MjAwaNWpE48aNyzuaCwoKyM/PZ8OGDeXnCxYsoFOnTsG/OBFJirIVzbbZuILbS0b4C8eOhZYt6/3Y6ZkMIuD000+nb9++nHXWWXTo0IE777yToUOHsueee9KlSxemTJkCQElJCWbGLrvswocffkjjxo1p3LhxjXv+hYWFFBcX06RJk6BejogEoGxFs1YXXMXW81fDMcfA736XkMdWMgjJDjvswNixYzn22GPJz89n0KBBnH766TjnaNu2bXmTUVkyyMzMpGXLlkyYMIFbbrmFJk2a0KhRI8yMgoICioqKKCoqoqCggC+++ILmzZuH/ApFJBly10yG+c9As2YwblzC+kiVDEI0cOBABg4cWOV1lZuJypx22mkMGTJki1FGAPn5+TRr1qzavgcRibj8fF+eGnw/QZcuCXtoJYMUVFRUVJ4Myo4MylScgFaZmoVE0tztt8OXX8Iee8BVVyX0oTXPIAVlZ2czceJEwB8ZxFp3qGwGs4ikoU8/3VSW+qGHoHTwSKLoyCAFNW/evHwOwksvvUTr1q1jul9WVhbfffcdLVq0SGZ4IhK0khIYPhwKC+H886Fv34Q/hZJBitsmzjoj7dq1S1IkIhKaRx6BmTOhXTu4446kPIWaiUREUtmKFXDNNf7vMWMgxpaCeCkZiIiksssvh1Wr4OijYdCgpD1NWiQDLewSP71nIhHw8svw7LPQooUvSJfEMjORTwZZWVls2LAh7DAiZ8OGDeWlLEQkBa1e7VcvAz+ktHPnpD5d5JNBdnY2y5cvZ/369drbjYGZsX79epYvX052dnbY4YhIda67Dr75Bnr3hksuSfrTRX40UatWrQD49ttvKSwsDDmaaMjKymL77bcvf+9EpHp5eb5a6IABvjZQIGbO9M1CjRrBww/79QqSLPLJAHxC0IZNRBKt4voBjRv7InFJTwgbN8KwYb768p/+BD17JvkJvcg3E4mIJEvF9QMKCvz/SXf77fDZZ7DLLn5Jy4AoGYiIVKNs/YDMTH9eugBh8nz8sV/TEuDRRyHAdc3ToplIRCQZytYPqE+fQax9DnnvFNHltPPYoaiIFSddzGPv9GNAZnD9FEoGIiI1yM2t+wY51j6HvDz4zyF/4/aiuSyjE3u/NIpVkwLsp0DNRCIiSRNrn8P85xZzQ5Ffx/h8HuKXopbB9lMQQjJwzh3vnHvfObfQOfexc+60oGMQEQlCTH0OxcWcNvVcmpHPE24obzU5kqysAPspSgXaTOSc2wP4C3C4ma10zu0ITHfOLTCzhUHGIiKSbDH1OYwbR6v5MynYth0/XXg3bx0NCxbAv/8NJ52Uvn0GBwDTzGwlgJn9zzn3LtAdUDIQkciprYO4xj6HL7+EESMAaPzog1x5fBvy8uCKK3wT0TvvQI8e6dlnMBc4xTmXA+CcGwjsD0wNOA4RkXor6yC+8UZ/npcXx51LSuC882DDBhgyBI4/HghpbgMBJwMzmwdcC0xzzq0A/gGcYWarKt/WOTfcOTfHOTfnhx9+CDJMEZGY1GvD/cAD8PbbkJ0NY8eWXxz43IZSgSYD59zOwEXAa8DlwL+Byc65/pVva2bjzSzHzHK22267IMMUEYlJnTfcS5ZsWrDm/vth223LryrrZ7j11uCGlQK4ICt9OudmA2PN7OkKl50CXGxmWySEMjk5OTZnzpwgQhQRiUu8hezyZpbQ4YwBdFr6DgweDM88k7Tnc87NNbOcWB436A7kXYBnK132EaAqcyISSfFMSsvLg+cPHsvownf4nu1ZdtY49ovjuZJZOC/oDuQ3gBudc40AnHNtgTHAgwHHISISuPn/+pxbCv3oofMzxjP1o21rucfmktm5HHQyGA5sDcxzzs0HXgL+bmYPBRyHiEiwiosZMvns8sllk5scF3fncDI7lwNtJjKzNcAVQT6niEhKGD2alp/MpqBte3668B6mHRV/E08iCudVJ9AO5LpSB7KIRNrHH8N++0FhIbz+OhxxRCBPG08HsgrViYgk08aNMHSoTwQXXhhYIoiXkoGISDLdfDPMnw877QSjR4cdTbWUDEREkmXWLLjrLsjIgH/8A1q0CDuiaikZiIgkw5o1cOaZvgbRH/8IffuGHVGNlAxERJLhyivhq69gr73gL38JO5paKRmIiCTapEl+QfsmTeCpp/x5ilMyEBFJpBUrYNgw//cdd8Aee4QbT4yUDEREEsXMr1Hw44++iNBll4UdUcyUDEREEuX+++HVV6F1a3j8cT+KKCKiE6mISCr75BO4+mr/9/jx0LFjuPHESclARKS+8vP90pX5+XDOOTBoUNgRxU3JQESkvq67zs8y3nnnzZawjBIlAxGR+njjDbj7bl9X+qmnYKutwo6oTpQMRETq6vvvfRE68BPLevcON556UDIQEamLkhI46yxYudIvLnDttWFHVC9KBiIidXH33b6JqE0bePJJ30wUYUoGIiLxmjsXRvi1jHnsMejQIelPmZcHo0b582QIdNlLEZFUkJdXj6UjV6+GU0/1i9Vccgkcd1wSItxcXp6f0FxQ4Nc+njYtsUtegpKBiDQw9dqwmsH558OSJb4aaUCL1Uyf7uMtLvbn06cnPhmomUhEGpSqNqwxe/RR+Oc//SI1zz4LTZtudnWymnIGDPCJKzPTnw8YkNjHBx0ZiEgDU7ZhLTsyiHnDunAhXHqp//vBB2GXXTa7OplNObm5/vHq3LQVAyUDEWlQ6rRhXbcOTjllU7mJM87Y4ibJbsrJzU1OEiijZCAiDU5cG1YzuOgi+Owz2G03uPfeKm9W5yOOFKFkICJSk8cegyeegObN4V//qnZR+yCacpJJyUBEpDrz58PFF/u/77+/1lXLkt2Uk0waTSQiUpU1a3wp6vx8OPdcX3oijSkZiIhUZubXMV68GPbcs9p+gnSiZCAiUtm4cTBxoi9H/dxzvr8gzSkZiIhUlJcHV13l/37sMejePdx4AqJkICJS5ocf/HyCoiK44go4+eSwIwqMkoGICPjZYkOGwDffQN++cNddYUcUqMCTgXMu2zn3lHPuM+fcQufcP4KOQURkCzfcAFOnQna27y/Iyiq/Ktnlo1NBoPMMnHNNgSnAWDM73TmXCQwNMgYRkS38+99wxx2+Etyzz262PkEQ5aNTQdBHBpcCM83sUQAzKzazxwKOQUQCEJm96c8+g7PP9n+PHg39+292db2qnEZI0DOQBwEXB/ycIhKwVNqbrnEhm9Wr4cQTYe1aGDzYdxpXEvWaQ7EKOhnsBvzsnBsP9Abygb+a2b8q39A5NxwYDtCpU6dAgxSR+gliMZZY1JiUSkpg6FBYtMhPLHvkEXBui8eIes2hWAWdDDKBh4G/mNlw51wX4HXn3Hdm9m7FG5rZeGA8QE5OjgUcp4jUQ6rsTdeYlG69FV58EVq3hkmTqi1AB9GuORSroJPBT8AVZjYfwMyWOuf+BpwAvFvjPUUkMlJlb7rapPTii3DzzZCR4Vcu22mncAJMIUEng2nAPsD8CpdlAhsCjkNEkiwV9qarTEqffQZnnulvMGoUHHFEiBGmjqCTwUjgVefcPDOb55zriO9QPjXgOESkgdgsKf38Mxx3nK9Ieuqp8Mc/hhpbKgk0GZjZYufcGcB459zWwM/A5Wb2SZBxiEgDVFTkE8CXX0KvXn5x+yo6jBuqwBe3MbPZ+JFEIiIJV+1Q0quv3jTD+MUXa+wwbohimnTmnHvQOWfOufZVXNfdOVfgnBuT+PBERGJXNpT0xhv9efmEt0cfhTFjfImJ558HDVffQqwzkMve0qr26O8GVgM3JyIgEZG6qjiUdONGP2Dok/tnwIUX+hs8+KAvQidbiDUZzC493ywZOOeOAY4C/mxmvyQyMBGReJUNJc3I8HPKvpqyhHYXnwiFhXDllX75SqlSTMnAzD7Hd/aWJwPnXBbwN2Ah8FBSohORtJLsekVlQ0kPPRS2cb/ykg1kW37my+7H+LpDAcYSNfF0IM8G+jrnnJkZcDmwC3ComRUnJToRSRtB1SvKzYWbbyhi7dRT2c0WsdDtyfpxT/uKpAHHEiXxVC2dDWwNdHfOZQM3ApPMbFpSIhORtBJY9U8zcp++lMNKJrOuxXYUPPcyvQ9tFU4sERLPkUHFTuSDgCbAHxIekYikpcDqFd19t+8obtKEFpMnsU+fLuHFEiHxJIP3gBLgPKAfMNrMvkpKVCKSdgKpV/TCC34+AcA//gF9+oQXS8Q43/wf442dWwDsCawAdjGzNckKrKKcnBybM2dOEE8lIlH1wQd+YZoNG2DkSBgxIuyIQuecm2tmObHcNt6Vzt4vPR8RVCIQEanVV1/BwIE+EZx7Llx7bdgRRU7MyaB0KOkAYA6gRexFJDX89BMcdRSsXOnHlD7wgGoO1UE8fQZXA12B0y2etiURkWTZsAGOPx4WL4aePf3C9o0bhx1VJNWYDJxzbYAjgJ7AH4G/lRaaExEJV3GxX7Zy5kzo2BFefRVatar9flKl2o4MjgCeBlbiaxCpIU5EwmfmF69/7jnYemt47TXo0CHsqCKtxmRgZs8AzwQUS8qqtiSuiITjjjtg3DjfJDRpkl/QXuol8PUMokbT1kVSzGOPwXXX+U7ip57SjLEEiXdoaYOjaesiKeQ//4Fhw/zfY8fCySeHG08aUTKoRdm09cxMTVsXCdU778CgQX7P7Lrr4JJLwo4oraiZqBaati6SAubN85PK8vP9kcFtt6kvL8GUDGKQm6svm0RDWm4gv/wSjjwSVq+Gk06CBx4gb7ZTX16CKRmIpIkoDHaIO1ktXw6HHQbff+9f3FNPQWZmlX15qfZao0bJQCRNpPoGMu5kVVZeYulS2G8/X5G0SRNAJaiTQclAJE2k+gYyrmT1669wxBGwaBH06AGvvw4tW5Zfrb68xFMyEEkTqb6BjDlZrV0LxxwD8+bx87a/4auRk8lp02aLm6kvL7HiWs8gLFrPQCQ91NpnsH69TwTTp/M/tyMHuXf5vkmnlOz/iIJkrmcgIlJnubk+EUyf7hPDZvLz4Xe/g+nTWdNyBw5z01ha0mmzyZ55eTBqVBX3lXpTM5GIBKbaTuSCAj+hbPJkyM5myT3TWHbeb8is0KQUhdFSUdbgjgy0ZyESnirLuxQWwuDBvtREmzYwdSq9TtuNadPg1ls3bfRVGia5GtSRgfYsRMJVuRP54H6FcOqpftho69YwZYofPcSWHcSpPloq6tIuGdTUQZXq47BF0kFNv8GKI54O7lfIAfcM3jwR7LNPjY95zz1+lctUHC0VdWmVDGrb89eehUhyxXL0nZsLuTmlTUPPP+8Xp5kyBXKqHvSiI/pgRLrPoHL7f21timV7JRXbIUUkcWJq19+40XcWx5AIYn5MqbfIHhlUtbcQy56/JqqIJE+tv8H8fF9s7tVXYZtt/OihGhJBTI8pCRFqMnDOtQPONrM74r1vVXsLI0ak9gzMeKRl9UlJezXOgl6/3s8jmDwZtt0Wpk6FXr3q95iSMKElA+dcI+ARoB9QczJYvtwvgO1c+UXV7S2kw56/2kglyqr8Da5ZA8ceCzNmQHa2/1Jr3eKUEmafwShgQky3XLECLrsMSkrKL0rn9n+1kUpa+flnX310xgxo395/oeNIBGU7Rzfe6M81Ryg5QkkGzrkTgCZmNjHGO8C4cczf71zy3ikqvzg31zcNpVMiAC21KWnk++/9F/j996FLF7905W67xfUQ2jkKRuDJwDm3M3ARcHUttxvunJvjnJvzY+t2rKM5PT/8BysGDGb2jI3BBBuSdD7qkQbk66/hwANhwQLo3t0ngm7d4n4Y7RwFI9Cqpc65ZsArwDlm9nXpZb+aWeua7texY451/W4sL5ccTWtW8d+dD6XrRy/AVlsFEbaIxOvTT+Hww31/X69e8MYbvq+gjjSgom7iqVoadDI4AngQWFXh4h7AAuApMxtd1f122y3Hvv56DrttnMerJUewPSv9ykevvgpt2wYQuYjE7P334aijfF/BgQfCyy/7+QQSuHiSQaCjiczsDaBrxctKjwxqHF/WokXZ0LJefLvzTLb/0+HwwQfQr58fptapU1LjFpEYvfGGn0ewbh0MHAjPPgvNmoUdlcQgMjOQyzqL9x60M8yc6YtZff459Onj2yRFJFwTJvgEsG4dnHmmn2GcwESgisPJFZlksJkddoC33/aHoMuX+3MNMRAJhxncdRcMHQpFRXDNNfD445CVlbCn0PDS5As9GdTWeVyt1q19E9HJJ8OqVX7x7ImxjVQVkQQpLvZzgP70Jz8EfMwYuPNOyEjspkXDS5Mv9GRQL02bwj//6b+MBQW+CuJdd/k9FZEUlhZNHuvWwYknwrhxfsxn2W8xCTS8NPkiW6iuXGamL3K+447wxz/6PZQlS/wXNIGHqSKJEvVyI3l58MHLKzj3hWPZatEcX3DuxRd9cy3JGQaq+kTJF/1kAOTNdkwvvJoTb+tC99vOhPHjYelS+Ne/oFWrLW+vMcsSoqAXWUrk9z0vDy49eCHPbRzIVnxNfvuuNH3zNT+pjOQmunSoO5bKIp8MKn75bm18Mu+N7UiP64/z/Ql9+vgxzl27Vnn7KO6VSfQFWZI50d/3ZQ+9xpsbT6UVa3iP/Zl99ktc3n3TZDKtJhhd0e4zYMsv339+PABmz4Zdd4VPPqFwn95MOP/dmBfAEUm2IMuNJOz7bgb33sspEwbSijVMdKdyVNO36D1w81nFatuPrsgfGVS5l9WtG+Tl8euRg2n93hucOv4QLnlsPMw4WwtlSEoIqskjId/3ggK49FIYPx4H/O/cP/NVt5t45ZCMGtc4VjNstARajqKucnJybM6cOdVeX12b6B23FdH8z3/gMhsLwPt9r6D39NHkfdBIX1ZpMOrVZ7BypR++/c470KQJPPoonH56EqKUZEjZ2kR1VVsyqE5Ze+nQ/PHcaxeTRZG/YOJEv9JSHajzWRqMjz6CE06AZcv8OgSTJvmaYBIZ8SSDyPcZ1KTskLXz7cP5/IG3Nq2wtN9+MH9+3I+nWZASBQmZw/Dkk34AxrJlcMABMGeOEkGai3yfQW02tc32g2Pm+D2dDz/0X/Dx4+GMM2J+rOo643SkIKmi3qOHCgvhD3+Ae+/1/597Ltx3n5/gKWktrY8MtrDjjvDuu3D22bBhgy+mdckl/pcTg8ojJbbdVkcKklrqNXro22/hkEN8IsjKgocegkceUSJoIBpWMgBfRfHvf2fJNQ9SnJnl93r69/eHw7WoPCTwp580TLUqaVFqIaLqPLRz2jTYe2+/s9S+vS8EOXy4rzckDYOZpfxp3333tUSaNcusWTOz3IzZ9j/X0QzM2rQxe+WVOj1OZqY/nzUroWFGkt6T8M2aZTZyZIzvfXGx2a23mjnnfweHHGK2YkXSY5RgAHMsxu1swzsyYNOhdF7J/uzrPmLhjqWrMh1zDFx7rW83jYHWKt6SJvWFr2ztj1q/j99/D0ce6ds5zfz55Mmw/faBxCmppUEmg4qH0r82asu+3/2HEYyiiExffvegg3xtoxjE/MNLU5WbhDQDNSKmToW99oIpU/zSsa++Crfc4j84aZDSep5BTcrmC7z/vh8+DdCPd3ixxRDarPvGr9n68MMwaFBCnzfe+FJ5lFJ1I1eiEHsqCOV9KiyEm26CO+7wRwMDBvhhpB06BBSABCmeeQah9wfEckp0n0FFF1zgm0rLTn84+0ez44/fdMG555qtXp20569KVNrdR470MYI/Hzky7IiiI5TP+PPPzXJy/AeWkWF2yy1mRUUBPLGEBfUZxG7oUD/L3jl/ftLwbeGFF/x6CE2bwt//Dr16BTo0Jirt7moSqrtAP2MzP0R077395LHOnWHGDN9HoGYhKdXgk0FuLrz1Ftx+uz/PzcVnhosv9j+cXr3gq6+gXz/4859jnpNQH1HZyKoDve4C+4xXrIDjjoNhw2D9el9X6OOP/fdZpIIG22cQs40bfRIYPdrvYfXqBU88AT16JPVp1e6e/pL+GT/7LFx4oR8pt/XWfk6Nisw1KCpUlwxvv+1nLv/3v3525s03wzXXQKO0r+ghUfPDD35m/bPP+v8PP9xXG+3YMdy4JHAqVJcMBx3ki9tdcIEfkXH99bD//jBvXtiRiXhmflH63Xf3iaB5c7j/fnj9dSUCqZWSQTy22goeeMBPzOnc2Re8y8nxiSE/P+zopAHZouTHN9/A734Hp50GP/7oawwtXOibiVRSQmKgZFAXhx3mf2iXXQYlJTByJPTsCW++GXZk0gBULKV+2CHF/Peqe/3RwIsvQsuWvhrv1Kmbrf1dl+dQfamGRcmgrrbaCsaM8YW9dtsNvvjC/0LPOsu32YokSdmw1B7FH/Fmfh+63n0ZrFnjy7N/+qkfOVSPowGt29EwKRnUV58+vt/gttv8RIUnnoDu3eGBB8h7t3iLvSvtcUl9/XbfX7mXS5lDDr15n43bdfBzY154ISF9A1GZ5yIJFuvstDBPyZyBnFBffGF22GHls5c/dPtY34xZ5TNMU21mcVzVLSV8xcVmjz1mlp1tBlackWnLT73SbNWqhD5Nqn1Ppe6IYwayxkUm0s47wxtvwPPPs+q8K9l71Ye8a32YkH8mc18cyZqtO26xxxXWHIJ6r4glwZo1Cy6/3E+EBOjXj4z77qN9z551eria5jiUTSbUPJcGJtasEeYp1Y8MqtrDnj1trd3Z6DrLp7EZWFGTZrbsvJutTdN1ddrjquo56rNnn8y6QjriSKClS82GDNlUK6t9e7MnnjArKanzQ2rPv+EgjiOD0Df0sZxSORnU9MOaNcvs/quX2I8Hn1z+Y97Ytr298ruHbdbbhfV6jvr+oJO1QUiXDU3oCe3nn82uvtqssd+ZsCZNzG64wWzNmno/tAoMNhzxJAM1E9VTVZ1tZYfVubmQm9sN+JefwXzVVTSeO5ejXxgGn9/te5KPPbbWkR/VdejVp8kpWU0BNb0fURFqE9qGDb5sxKhRvowEwJAhvnhWly4JeYqyukhlry9Va19JsJQM6inmH9ZBB/nFEyZO9JPUPv0Ujj8e9t+fT4fcxotrf8uAg12VG53qnqO+P2ifrOK/X03SYUMTSkIrLPQVcm+5xS9MD/7NGz3aT2xMIPUJSJViPYRIxAloC4wEPgQWAG8Be9V2v1RuJjKrQ5Pb6fcgAAAOHUlEQVRCfr7ZPfeYbbddefPRdPrbYY2nV/sYie4zSKZUjStWgTZ1FRSYPfKIWdeum/oF9t7b7NVX69UvIGIWXzNRoIXqnHP9gS7Ak2ZW7JzrCzwBdDezouruV9dCdSlf+XPtWt46eRx7vXEXbfgFgGVdDqTTw3/27RQqIxCaRHx3anyMjRv9nJSRIzctsdq9u68HftJJkKEpQFJ/kVrpDJgD/Kam29TlyCAqHZmzZplt3/RXu8n9xX5im/K9w2923N8WjXrejy2XOql8hBLkEUu137/Vq83+7//8qKCyI4FddzV76imtOiYJR1RGEwE7AN8BzWu6XV2SQZRGTJRtpN6bssqWXjDSVtK2fEOxvtMuZuPHm61fH3aYkVJ5Y/zQQ8HuHFT+/o259luz668322ZTwrc99zR7+ul6J4GoN8tJ8kQiGQDtgXnA76u5fnjpUcOcTp06xf0mROXIoLKRI81aZqy1Sxhr/6Xzpg1H27Z+aOG334YdYiRU3hgffniwOwdl3799Mj6yCZlDrbhR1qbPsk8fs5dfTkifQFS/5xKMlE8GQC6wGBgSy+3r2oEcxT2mij/ulk0LbPFNT/oOxbINSVaW2Wmnmc2YoQ7GGoR6ZJCfb/bkk7aqR59Nn1tGhtmJJ5q9+25CnypKR8ASvHiSQeArnTnnzgEuBwab2aJY7pMSK50lSCwdk1vcxsxXRx0zxhcjKynxN9xjDzj/fL+UYZs2gcQfJZXfx6QPKFi82K8o9vjjsHIlAEUtWvHRXueQeeVl7HNyt4Q/pcqKSE1StgMZ6IkfUrpNPPdL9aGlsUrIIf3XX/vmonbtNu11Nm5sduqpZm+8kbBOyKCOqqJ49LaZ1at98biDDtr0eYBZjx725TUPWtuma5J+NBL591CShlRtJgLuA46J937pkgwSekhfUGD27LO+Mdy5TRuhHXYwu+oqs7lz69yMFFQ7dGTbuwsK/DyA007zgZe99y1amJ17rtnMmWYlJWrCkdDFkwyCHsy8C/BX59zCSqfjA44jFGWzczMzEzA7NysLBg3yVVKXLvUzV7t1g+++g7/9Dfbd1y+6c8MNfr2FOJoDg6pnH6m6+YWFMGWKXzimXTs4+mh45hlfPuLAA+Hhh/17/+ijfo0L5+r0eWu9CwlL4H0GdZGqfQZ1aYNOaru1Gbz3Hjz1lC97UXHFtZ12guOO86e+fX0yqSHGINqhU769e80an2wnTYJXXoFff9103R57wODBcMYZNdYMiufzTvn3QyInnj4DJYM6SvkfblERzJgBzz0Hzz9f3qEJQOvWfh3nI47wpypWxwpq9nZ1zxPK7HEzv7b166/Da6/5TvvCwk3X77abPxo75RSfDBJs1Ci/1GRxsT+auPVWGDEi4U8jDYiSQQAi9cMtLvZb15degpdfhkWVBnHtuiscfDD07+9P7dpt8RBBjswJLNGa+ffinXfgzTfhrbc2T5oZGXDAAX5t4eOPh112SUIQm6T8DoZETjzJQFVLq1Hbxi5S1TkzM6FfP3+66y748kvf/DF5st8ILlrkTw884G/frdumkqa9ezN7XQ9+e3TT8td6zz1wxRXJ22glrWroL7/A3LnwwQf+A545c1OZ6DLt28Phh8ORR/qjpwCH7KqaqIRJRwZViHUPLeimjKQ8X0GBX0px+nR/mjkT1q/f7CbFGY1YULIH8+jFp25PsnrtwdMf78HXJR3JyMxI+FFRvfeQCwpgyRJfJnz+fFiwAD7+GL76asvbtm/v+1AOOcSffvMbFQiUtKFmonpKxSagwJoQior8xjMvD2bPhrlzsUWLcGUT3SpYTzOWuJ1pf9DObJvTDXbcETp18n0Q7dpBdjY0aVKnMGpMfOvW+ZE7333na/8vXQpff+3PFy+G//5308S8ipo2hb339usD7L+/TwKdO2vjL2lLzUT1lIpNQIEtuNKokd9g7r03XHQRAG7dOhY89THLX/mYXlmf0O6nTyic9wnNf/2BHrYAZiyAGdU8XuvWvqmldWt/atkSmjeHZs38KTPTnzIy/Aa8qAgKC8ktKCB37VqYuQ7WrvUjeX7+2Z/Wrav5NTgHXbv6vpCePaFHD3/afXf/+kRkC/plVCEV225DTVAtWtBjeB96DO9TflEW+A30kiXwxRd+z3zZMn/65hv4/nvfGfvrr5sPyUyEJk38kccOO/hmns6d/fDOzp1h5539MNqmTRP7nCJpTs1EEZLyi/VUVlLi9+R/+WVTUlizxk/UKjsVF/vbFRezdFkGXyzNYqfujei2a2No0QK22sqfb7ONP8Jo08ZfFkfTTpDvW+Q+I0lraiZKU8lYs7iihG/IMjKgbVt/iuG5y/tEpiWuTyTI4ZoaGipRprX1BNi0IbvxRn8edDmEZJWmCLLkRRjlNVS+QhJFRwYCBNhBXY1k9Ykk43GrO4IKul9HRyKSSEoGAoQ/gipZnfaJftyaNsBBDzwIO4FLelEyECA1RlAlq08kkY9b2wY42f06FYWdwCW9KBlIuSA3ZFGVShvgVEjgkj6UDETikGobYCVwSRQlA5E4aQMs6UhDS0VERMlA6qahj29v6K9f0o+aiSRuYYxvT6UyD6k2vj+V3huJLiUDiVvQ49tTbeMby+sPctnQVHpvJLrUTCRxKxtemZkZzPDKMMo81KS21x9kaY9Ue28kunRkIHELenhlKo3th9pff5BHTqn23kh0qYS1REKU2sWDbrqJ0nsjwdKylymiIfxIG8JrrAu9L5IKtJ5BCkjG3mGqbWDUeVk9TUyTqFEHcpIkumMv7PUGqqLOS5H0oWSQJIkecZOKG96gRxWJSPKomShJEj3iJhVHjaRa0ba6SrXmN5EwqAM5QqK20YpCvOr3kHSmDuQ0FaVOyVg3smEnDK0WJuIpGUhSxFqyIey98lRsfhMJg5KBJEUsG9lU2CtPl34PkfpSMpCkiGUjmyp75VFqfhNJllCSgXPubOAPQBYwC7jYzDaEEYvEpi5t+7VtZLVXLpI6Ak8Gzrl+wBnA/ma23jl3DTAKuCLoWCQ2yWzb1165SGoIY9LZtcB1Zra+9P/RwFHOua1DiEVikIoT3kQkscJIBnsDH5T9Y36iw3tA7xBikRhoprFI+gu0mcg5tw2wzrac6bYC2LHSbYcDw0v/3eicWxhAiGFpC/wYdhA1a9kCWrXcsGH1mj591qyL884ReH31ks6vL51fG6T/6+se6w2D7jNoBlQ35Xmzy81sPDAewDk3J9ZZdFGk1xdt6fz60vm1QcN4fbHeNuhmoh+ANlVc3g74X8CxiIhIqUCTgZkVAp875/Ytu8w554AcKvQjiIhIsMLoQL4LGOmca1r6/7XAa2a2qob7jE9+WKHS64u2dH596fzaQK+vXChVS51zFwAXA02At4DLzGxj4IGIiAgQkRLWIiKSXFrpTEREUj8ZOO8S59wM59xc59znzrk/hx1Xojjn2jrnRjrnPnTOLXDOveWc2yvsuJLBOTfIOdcu7DgSxTl3dulntsg593fnXLOwY0q0dPvMIP1/c3XdZqZ8MsDPTegKHG9m+wL7AQc754aEG1bC7AF8DuxnZj2AG4DnnXNpVVHWOdcRX3okLTYslWps7QoswtfYShvp9plVkO6/uTptMyPZZ+CcuxxoY2Y3hR1LMpROFDnNzL4IO5ZEcM69AuwDbAv0NrN5IYdUb865/wC3mNn7pf87fELoXcvIuEhIx8+sJun2m6sslm1mFI4MNuOc2xU4HXgu7FiSwTm3A9ABWB52LIliZseY2Q74cuXpIq1rbKXpZ1aldPzNVRTrNjMSycA51905N8859yWwEHgHvxeWVpxz7YHXgBsrVHWVFBNPjS1Jben6m6vLNjMSycDMPjezXma2M9AaaAncHXJYCeWcywWmA3eZ2SMhhyM1i7nGlqSudP7N1WWbmVLJwDl3qXNuYYXTKZVvY2ZrgRHAscFHWHc1vTbn3DnAA8BxZvZ0eFHWXSyfXRpRja2IS4ffXKxi3WamVO+5md0L3BvDTbcFViY5nISq7rU553oCVwEHmdkvgQeWIHF8dpFnZoWlw/X2NbO5sFmNrUvDjU5qky6/uTjVus1MqSODqjjnMp1zjzvn2pb+vzVwPzAm3MgS5nzg2gb0pUwXdamxJakhrX9zdd1mptSRQVXMrNg59xnwhnOuCVAAjDWzJ0MOLVF2AX7rnLuz0uXXm9mLYQQktTOzl0o7Hz8o/V6+BVwWclgSm7T+zdV1mxnJeQYiIpJYKd9MJCIiyadkICIiSgYiIqJkICIiKBmIiAhKBiIigpKBiIigZCBSK+dcM+fcN865ZaWTeCpe94hzrtg5Nzis+EQSQclApBZmtgG4CV+e+qKyy51zo4DzgEvN7J8hhSeSEJqBLBID51wm8DGQDXQDfo8vCXyTmd0SZmwiiaBkIBIj59xA4GVgGnAIMM7MVI9I0oKaiURiZGb/AT4EfgtMBC6vfBvn3MXOufedc/nOuekBhyhSZylftVQkVZQu2NOr9N81VSx7CfAdcAewH5AbVGwi9aVkIBID59zhwATgBaAQONc5d7eZfVbxdmb2fOntOwUfpUjdqZlIpBbOuf2B54GZwOnADUAJMCrMuEQSSclApAbOud2AV4DFwAlmttHMlgCPAsc75/qGGqBIgigZiFSjtKlnMrAKOMrMVle4+hZgA375S5HIU5+BSDXMbBl+ollV130HNA82IpHkUTIQSSDnXCP876oRkOGcawqUmFlBuJGJ1EzJQCSxbsCXriizAZgBDAglGpEYaQayiIioA1lERJQMREQEJQMREUHJQEREUDIQERGUDEREBCUDEREB/h8f2VDpoZwfjgAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x1a23632438>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "X_new=np.linspace(-3, 3, 100).reshape(100, 1) # 시작, 끝점을 균일간격으로 100개 생성\n",
    "X_new_poly = poly_features.transform(X_new)\n",
    "y_new = lin_reg.predict(X_new_poly)\n",
    "plt.plot(X, y, \"b.\")\n",
    "plt.plot(X_new, y_new, \"r-\", linewidth=2, label=\"예측\")\n",
    "plt.xlabel(\"$x_1$\", fontsize=18)\n",
    "plt.ylabel(\"$y$\", rotation=0, fontsize=18)\n",
    "plt.legend(loc=\"upper left\", fontsize=14)\n",
    "plt.axis([-3, 3, 0, 10])\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- 특성이 여러 개일 때 다항 회귀는 이 특성 사이의 관계를 찾을 수 있음(일반적인 선형 회귀 모델에서는 하지 못함)<br>\n",
    "- PolynomialFeatures가 주어진 차수까지 특성 간의 모든 교차항을 추가하기 때문"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.5.5"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}