{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 밑바닥부터 시작하는 딥러닝\n",
    "\n",
    "# Deep Learning from Scratch\n",
    "\n",
    "## Github \n",
    "\n",
    "https://github.com/WegraLee/deep-learning-from-scratch\n",
    "\n",
    "## 목차\n",
    "\n",
    "http://nbviewer.jupyter.org/github/SDRLurker/deep-learning/blob/master/%EB%AA%A9%EC%B0%A8.ipynb"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 3 신경망\n",
    "퍼셉트론 좋은 소식: 복잡한 함수도 표현가능\n",
    "\n",
    "퍼셉트론 나쁜 소식: 가중치를 설정하는 작업을 사람이 수동으로 함\n",
    "\n",
    "신경망은 이 나쁜 소식을 해결함."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 3.1 퍼셉트론에서 신경망으로\n",
    "\n",
    "### 3.1.1 신경망의 예\n",
    "\n",
    "입력층, 은닉층, 출력층\n",
    "\n",
    "신경망의 예 그림"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAABFcAAAMZCAYAAADFq9tvAAAAAXNSR0ICQMB9xQAAAAlwSFlzAAAX\nEgAAFxIBZ5/SUgAAABl0RVh0U29mdHdhcmUATWljcm9zb2Z0IE9mZmljZX/tNXEAAJ8cSURBVHja\n7f1vaGVnvi/4FZxzutR9Lrbe2BEYE9HQiQx+scEkFji0RdNMq9MhrfabVpMXpXclLtgWOB4XxjHl\n+GIZPFjEE6rSwVjj8aVlTGLFgVi+gZTmclHEcFyjydimyDG+lTseR3fujadO5sRozjn3nB098trV\nj1atvbW39lprrz8fwwfbVfqz97PWevbzfNezfs+FV1555QIAAAAA56MRAAAAAMagEQAAAADGoBEA\nAAAAxqARAAAAAMagEQAAAADGoBEAAAAAxqARAAAAAMagEQAAAADGoBEAAAAAxqARAAAAAMagEQAA\nAADGoBEAAAAAxqARAAAAAMagEQAAAADGoBEAAAAAxqARAAAAAMagEQAAAADGoBEAAAAAxqARAAAA\nAMagEQAAAADGoBEAAAAAxtDYN3bxBz/oXrhw4cSf//BHXa/rjBMheU3h9bkwAAAAYIQ5dZVezI+m\nfng8ub/Y/bM//dO7AURvwv/Di1Pdf/SjPx964t+mcCWPYKSscOWHU1N333/4bxchAAAAdVeJFxFC\nhnSg0s/UxYtDTcjPG2KErx3mdWQZJpgQrghXAAAAaJaJv4CwWmXUEOMHf/ZnhYUYwhXhCgAAAIw0\np57kL08HGWH1Sghb/vxHp0OH8HVhxcooAYtwpfyfMQzhCgAAAE0z0V8ePwo0zOM+IXT5kz/5k6Em\n52XVXInDmDaGK/H7KZpiuwAAAFTRxH5xHEoM85hP1veFcGaYSX/Z4cqwq1+EK8IVAAAA6m9ivzjs\n/nPekOFUcPKj7O9tS7iS/l3nPhGEKwAAAHC+OfWkfnE8KR/1e+O6Hf0CiraEK+mCwOf9mQIMAAAA\nOJ+J/eKqhit1K2gbHqnKY3VHUeFKCH9CPZ3wc+N6OT3hz8Lfha8JX+uiBAAAoG4m9ouFK+OHK/1e\n63l+bp7hyj/60Z/fs7vTKML3hp/hAgUAAKAOJvaL45oro65YiFdr5F1zpU7hSvyz4v8eVOi374mQ\nU7gSjmXWCpVRhZ9hJQsAAAB1MLFfXNXdgkbdWvmsn1FUzZW41kqv/eLQ6eIPLo70s/MIV9L1X3qv\nI/z5oPca/i58Tfja9PcLWAAAAKi6if7yEI6MEgaESXi8KmLQxLvJ4UpYrRO3Q+/nhD9PP14z9ImQ\nQ7gSv6ZwPM/zaE/4nvjYhZ/pQgUAAKDKJvrL02FACFtOVjmkHvUJ4UG6hsdZq12aHK7EIUZ4vCr+\nu/TqkWEDlnHfc/x786jbEh8/q1cAAACosom/gKxHSc4yzGNEkwxXinhdQVjVET/6068d4no2w64K\nGvc9x+FX3ltMhwLGLlYAAACqqhIvIqxUiR8RGiS9UqOfMsOV8D1BCAHC9xT1uFK8YuWsx2XSK33O\nKhArXAEAAIDzqdSL6RU1TQctYcIfQpVRanhMYregYcKA87yueOvpXlDSb5ekU9+XWsEy6BGbqj0W\nFK/Q8VgQAAAAVaYRUvIIVwY9hjNquJIOmoYNVnrirZEHPU6Vf0HbHyhoCwAAQCtohJRRwpUQVoQg\noLeyJqwwOSswGTVcSW9ZPUqwcvdnHH/PWd9rK2YAAAA4n0q9mDh4GPdnxY/S5FEDpIj3OMpjQcPW\nmjn3iZDTIz3nKVCc5awaMQAAAFAVlXoxbQhXKnsi5FgvJTzaky6oO4rwved5pAgAAAAmMqeu0osR\nrkzwRChg++kQkITVJyEsiY/tvY8O/eDka6xUAQAAoJZz6iq9mKqFK3kUtx1knNAnfm15bFVcRLhy\n1nGxxTIAAABNUKkXI1w532sTrgAAAMDkVOrFDHp0ZBIhRpvClbIIVwAAAGiaSr2YqoUrRb9H4Ypw\nBQAAgPqr1IsRrgxvkuFKHJBMimAGAACAqqjUi2nDbkHCFeFKaRf3hQvzx5aOXU1sH9vtYzv6uuVj\nC8emtCMAAMAQ868qvRjhyvCEK8KVUxfyhQtzx1aPbR27nWNbHx3bObZ2rKOtAQAAMuZkVXoxwpXh\n1bXmCjlevBcuLCZhymGJwdZh8juXHQMAAIBkflalFyNcGZ5wpaUX7PcrVDZKDlT6uXNsMzxC5NgA\nAACtnqtV6cUoaDu8IreJDq/RxVG5UGUhqY0y9HF88onHu7/+5c+7Lz//dPedt17v3th+b6AP3712\n8rWXlp86+d6pixdHOW/Co0grjhUAACBcEa4IV4QrtQxVQhDyi5/9tPvmqy92b974qPv3/+Yvc/HV\nzd2TYOa3v/lVd+bBB4QsAAAAwhXhinClNqFKePxnf9Bxmr7/vpNVJmHFSV5hylm+2Puk+8Izl4cJ\nWm55XAgAABCuUIi8whUaekFeuDB1bD3ZpSczuAiP+vzh9xvd777+rLRQJcvH7799sqLljJAlFL+d\ncWwBAIBGz+U0AlTkYvz+EaBbg0KVPB/5yfPRocuXfjeoRksofLvmGAMAAI2dz2kEqMCFeOHCWr9Q\nJdRSqWKokvbN53vdZy+vDApZwiqWKccbAABo3JxOI8AEL8DvHwPaygojQl2TMuup5FmXZf6xTr+A\nZd9jQgAAQOPmdhoBJnTxXbgwc+wgK4QIRWMnXVNlXGGXoVB0N+P9HR6bdw4AAACNmd9pBJjAhXfh\nwnwSMtyzA9CN7fdqHaqk67F0Hn0kK2AJBXuXnQsAAEAj5ngaAUq+6C5cWM3aDWjuJz8+CSOaEqz0\nhBU4oRhvn8eENpwTAABA7ed5GgFKvOAuXNjotxPQt19+2rhgJfby80/3C1h2FLoFAABqPdfTCFDS\nxfb9ipV7woUQOjQ5VIn94fcb/XYTuu4cAQAAajvf0whQwoX2fY2VU48ChZChjrsBjStsKz378ENZ\nAcuqcwUAAKjlnE8jQMEX2YULs+nitSFYCSFD24KVnm8+38sKWEL4tOCcAQAAajfv0whQ4AV24cLU\nsf30Ko02rlhJ+2Lvk6ytmu+EMMq5AwAA1GrupxGgwAvswoWtdLDy2kvPtT5Y6fn4/bezHg+6pcAt\nAABQq7mfRoCCLq4LF9aydgUSqpwWwqaMgGXbOQQAANRm/qcRoIAL68KFxXRg0Hn0ke53X38mUMnw\n29/8KitguepcAgAAajEH1AiQ80X1fZ2VUwVsQ22Rr27uClL6CKHT/GOdrICl45wCAAAqPw/UCJDz\nRXXhwkZ6Z6Ab2+8JUYbYQWjmwQfS4cq+cwoAAKj8PFAjQK7BSifZUvhuQPDCM5eFJ0MKuyhlrF5Z\ncW4BAACVngtqBMg1XDm17fLsww+pszKiUPQ3Fa4c2j0IAAAQrkA7gpWl9KqLsBJDYDKaUJsmPEqV\nass15xgAACBcgeaHKwe2Xc7Hy88/bfUKAAAgXIGWBSv3rFq5eeMjQckYuwdlFLddc64BAADCFWhu\nuGLVSs7efPVFq1cAAADhCrQkWFmwaqW01SurzjkAAEC4As0LV65btVKM1156Lh2u7DrnAAAA4Qo0\nK1iZOnbHDkHF7RyUXhV0bNa5BwAACFegOeHKcjzxD4+xhMdZBCP5efKJx9PhylXnHgAAIFyB5oQr\n2/HE/9nLKwKRnF1745V0uHLbuQcAAAhXoDnhyqlHgvZ2PhCI5OzbLz/tTl286NEgAABAuAINDFY6\n8YR/+v77hCHlPRq04hwEAACEK1D/cGUtnvBfWn5KEFKQl59/Oh2ubDoHAQAA4QrUP1w5VW/lnbde\nF4QU5Mb2e+quAAAAwhVoYLhyqt5K2DZYEFKMsANTRt2VGechAAAgXIH6Bisz6q2Ua/6xTjpcWXQu\nAgAAwhWYfEgynTzesznKSojjf+bjiX6Y+AtAinX50u/S4cqqcxgAABCuQDXClcNksh4e87lybGqI\n71uJJ/ph4i8AKdZrLz2XDlc2nMMAAIBwBaoRsIRHfLbiYqlnPXJy/M96PNEPE38BSLE+fPdaOlzZ\ncf4CAADCFahWyBIe9dmPJ+/H5vp87WY80Q8TfwFIsb7Y+8SOQQAAgHAFahKyrEaPCnWTVSrTqa85\ntQ1z2CpYAFKsbz7fE64AAADCFahRwDKdevQnhC0r0d/vxhP9vZ0PBCAlSIUrd5yrAACAcAWqH7LM\nJY8H9Sb0+8njQ6fCla9u7go/yg9Xus5RAABAuNK+ifpWMimnHBvHrubkIDWx/2vhSvlmH35IuAIA\nAAhXWh6u3E7feacZhCuTCVfStXAAAACEK80PVxYo1VpOq1ZC7ZU70YT+KFXwVrhSkun770uHK/+P\nY7P6FwAAQLgC1Q3EllNBSni0ayZdc+XmjY+EHxOoudLbNUjAAgAACFegeqHKfFK8tjeBDzVX5qO/\n37UVc7m++/qzrGDl76OApePcBQAAhCsw+VAlrEq5ntqCeTXj6zbjSf7H778tAClYePQqFax8FwUr\n3eTRLQELAAAgXIEJhSpTx66kaqus9yuYmgpgum+++qIApGB7Ox+kw5W/SOrfBB9GAcuCcxoAoD5+\nODV1d4wX/ruu7+PiD35w9338+Q9/VJn30XtN4fU534QrUGRns3jsVjRp3zk2d8b3rMUT/WcvrwhA\nCvbOW6+nw5XryXbb4b+3o9VEIWxZdG4DAAhX6h6u5BGMlBWuNOU4ClfgfB3NTDRZvzXspDwJZO52\nHr/42U8FIAV74ZnL6XDlSnL8jpL/7yS7O/UClmXnOACAcKUnBB59Nkg40zDBhHBFuAJtDlemklUo\nq+G/R/i+ubiznfvJjwUgBfv1L3+e/pBbSo7FlV7R4eT/r0Zfs+I8BwAoXxw0FGXUsEC4IlwRrkA1\ng5neionu1MWL3W+//FQIUqDZhx9Kf8h1ooDsMBW4rERft+Z8BQAQrghXhCvCFahmuHIQd7gfvntN\nCFLeTkFH8UqjqAbOQe/PUwHLVecsAEC7w5VRxWFMG8OVMo5hk4rtuvDh/J3UhqK2Eytmuz0g7FqL\n/mwxWmG04bwFAKiWf/SjP+/+yZ/8yd1xXvjv8GdVeG1Z4cqwq1+EK8IVYPhO6lRR2/DYiiCkGJeW\nn0p3wGsZx2Mp+bvD1KqWOGDZdO4CAFQnWPnBn/3ZPRPt8Gd//qPJb2U86XAl/buEK8IVaGq4MhXX\nXQnC4yvCkPzNPPhAZr2VjGOym/z9eurPw05Cd6Jtm6ecwwAAk/OjqR+eWrESApVQxzBewTLpuh2T\nDldCG+XxM5sUYAhXoLkBy27c4b320nPCkJzd2H4v/WF1OOB4dKKaLDMDApYdAQsAQLnCSpUQGPzZ\nn/5patXCxbuT/rgo6t2Q5eJULitZ6lbQNr2q57zhSFHhSjiWIRALPzcOyuJjF/4ufE34WuEKMKij\nWrMlc+mPBF0/45hs9/u6439mj91O/n7/2LTzGACgGCEQCSFDCEeyHv8JIUtWCBH+LOuRlPD1vYl6\n+JpRa7PUKVzp91rP83PzDFdCm8crjEYVvrcqNXWEK1CtcGUm/WjQF3ufCEVy8t3Xn2V13gsjHJPO\nGQHLgYAFACA/6ZUnWUJIMsxKhvA1WaFMWghvmhauxD8r/u/QdpMKV9KPcp1X+BlNXMmiA4DxA5bt\nuLN4+fmnBSPF7RJ0e8hjstFvV6Hk76ej3YVuhcDFuQwAkI9+E/Dw+M95QofwPeGGW7+fO+xKiFG3\nVh4noBknXIlrrYRwKfxZHDLFj1GVFa6k67/0XkdvFdGg9gpfE742/f1NC1hc/DB+uLIUdxKh+GpY\ncSEcGV/n0UfSnfD6kMdkJqqvsjBEwHJbwAIAkI+weqVXb+OkXsoP89v55+Qxo+OfH3527zGh8wQj\nVQ1XwqNUcYjU+znhz9OP15QZrsSvKQQl53m0J3xPvAon/EzhChB3VlPJ9r93O4o3X31RODKmD9+9\nlvVBNTfCcenVwzk449jtRFs4d5zTAADNVIdwJQ4x0o87pVePDBuwjPue49+bR92WOGBp0uoVFxnk\nE7CsWb1S+KqV7TFCr6UhA5Y7AhYAAOHKuKHBqOFKWNURP/rTexwoLQQu/XZaKipciWsg5r3F9KS3\n2xauQPXClXtWr9iWOfdVK51zHJflaFXK1BlfuxkFLIvOawCA4Q1TyLZoZ03UzxOuhO+JH0UatNLi\nvOFK+Np4xcpZj8ukN3w4q0CscEW4AnULWNbSy/S++XxPWDKBVSup49Krq7I2xNf2ApYjAQsAQHPD\nlSJ+x3nClXS7haAk1Fc58/suTg1dILZqjwXFK3Q8FgRkdVpTURHVE7/+5c+FJSMK9WpSHxRH4zyq\nE0KSaPXKzBBffzX6vSvObQAA4cowj+GMGq6EgrznCVbi0KO34qXfY0R5hCvB6YK2P1DQVrgChQcs\nq+kO+OP33xaaDCms9EkvcwzbKudwXLZH+VlRwNIVsAAAlBfMFPmYyCjhSggrQhDQ2/EovK6zApNR\nw5X49YTfN0qwcvdnHH/PWd9rK2bhCtQ1YNmPO43Zhx/qfvvlp8KTIYSVPqlON6w2mc7hmHSi1Sgz\nQ37PSvQ6rji3AQDqHa4U7byPBaV3BSpgfpLLIz1ZAct5nFUjRrgCxBP5o7gDefKJx+0edIYXnrmc\n1fmu5nhcNket35IKWK46vwEAhCs1nJ/kVi8lPNqTsdJ8aOF7z/NIkXAF2tuBrac7kmcvrwhR+vjD\n7zeyOt/9nI/JTBR6dUb4vsXo+647vwEAhpNHjZN+8ti1RrgyXsgSVp+EsCResXPvo0M/OPmaJq5U\nEa5AOR1YKG67m+5crr3xijAlZW/ng6z0OzwONFvAcdlIfv7OiN8XByybznEAgHqHK0W+tnFfX/za\n8ljJU0S4kqXtK5Bc9FBcwDJ97HZ6GdyN7feEKomvbu52Zx58IGt3oPkCQ6/D5Pcsjfi9nWg3qFAg\nd8p5DgAgXBGuCFeEK1B8wDKXrr8yff99J6FC24OVUIOm8+gjWR9GywUfk7Xk9xyc43vjgGVHwAIA\nUJ548i5cqfbxEa4ARUzml9KdbQgV2l7gNmNnoGC9hOMRVq/cOm/B3PC4UrQiaT+P3YwAACg3XCnC\neXYLEq4IV4DRJuRX0kHC/GOd7jef77Vyxcpvf/OrrGBlu8TjsRTVdpk6x/fHAcuBgAUAYHBI0IaC\ntk0IV+KAZFLqGsy46KG8Cf1WuuMI9UZu3vioNcFKCJP6PAp0UPYjNsnvDL/7yjm/fzr6GbeKKMAL\nAFRiDBdWvS6ER5ePXU1sJ5sXZNmOvm45+d5WPkosXBGuCFeAoj6YD7L2eg9bETc9WAkhUkbx2m6y\nAmR2AsdjPiqgO5NDwHJbwAIAjRizdZIabTvp2nljOkp+ZvjZHeGKcEW4IlwBzv9hPZ3czbinE3n2\n8kpjg5UQHmVst9xbsTI7wePROxYbY4ZmO9FjRq0YLAFAw8Zoi8kq48MSJ5GHye9cbmq71rV2SJvD\nFYQrULcP8I2sD9lQ5PXbLz9tVLDywjOX+w0otiddqyS5M3WUmMspYLkjYAGAWozH5pIx2eGk79Qn\n44fN8AiRcEW4IlwRrgCjfaAvZy01nfvJjxtRhyXUV/nFz37abwCxXqHjsJFXQd1kUNQbIC06zwGg\nkmOwhaQ2ytDhx5NPPH5yE+zl55/uvvPW690b2+8N9OG7106+9tLyUyff22cFbz/hUeOVpoUrRWhD\nuJK38Br1A8IVaOKHe6ff3ZLwmFBdV7G89tJz3en77+v3rHGllr6GeitRyNXJMWA5ErAAQP1ClRCE\nhBtEb776Yq43vL66uXsSzIRdE/vUoWtcyCJcEa4IV4CyJ/f7WZ1f+OC99sYrtQlVwp2asPJmwHPF\n8xU9Br2tsg9y+nlXo4Cl1oMiAGjAWGuu31irJ9wUCqtMwoqTssZNX+x9cvL49BBBy626Pi4kXBGu\nCFeAsj/0p7K2au6ZffihkzsdVQ5VwpLXAR35fpV30knav7eCaCnngKUrYAGAiX2+rw/a8Sc86hMK\n73/39WcTHUt9/P7bJytazpgYb513h0PqFa4gXAHGHwQsJncn+oYsYSVLVR4XCnd3zghVQu2RtZq0\n/Wp0d2gqp5+5ErXFFec4AJT2ub4waEwVQpUq1rgLjw5dvvS7QTVaajO2AuEKMOnBwFTymMrRoLsX\nk7rTElaphA/9PjVVYpt1u7uSbA3dzXPQkgpYrjrHAaDwz/O1fuOTUEulDhsHhI0BQv29ASHLVl43\ngwDhCjR9YDCT7GRzdFbRtRC0hBUt4dndIj7cQ4gTnkMesvjadl23Ig6PBEX1YaZy/LmL0XG87vwG\ngEI+x/s+Zh3GMGXWU8mzLsv8Y51Bj117TAiEK0CeIUv86FBvq8AQioSVJsN+gO/tfHDyzG/43rA6\npfPoI6MUyNqsa6iSau/dIraLTgUsm85tAMh9vHSQNUYJRWMnXVNlXKH2Xp9Vw5XdMACEK0CVBw1X\nk235zl0hPKx0CXVSggFLTYd1mBSKm21QO3einX5mCvjZd6IVPpbzAsD4n6/zUWH6UzsAjXKTqepC\nPZY+N77CmGXZuQDCFeB8k/TrWQOJEhwlS24XG9y+20U9wpMKWHYELAAw1ufqatbq3rmf/PgkjGhK\nsNITVuCEFcp9xmgbzgkQrgDjTdbXkkDgqKBAZT9ZobLQkjadidqyU8DPn41WIIW2nXYuA8DIn6cb\n/Yr+V2VnxaKER7j7jNncuAHhCpDTQCMsjV1OHiHaSmqIHI0Qouwk37uSbGPYyg/oaMC2XdDPjwOW\nAwELAIz0ObqaNZYJoUOTQ5VYqK3X5xFvxfNBuAJQmUHbTPTY1WJBv2M6Kr53q0m1awCgwM/o+fSN\noxAy1HE3oHGFbaXDZgYZAcuqcwWEKwBVGbyt9VaWFPg74oDltoAFAAZ+bs6ma86FYCWEDG0LVnq+\n+XwvK2A5asvj3FCZ/kkjAPQdwE1FA7ilgn/PTrQDU0f7A0Dm5+V+epVGG1espH2x90nWVs133LQB\n4QpAVQZyy1HoMVXg74kDljsCFgC457NyKx2svPbSc60PVno+fv/trMeDbilwC8IVgKoM5nqP7ayV\n8Ls2o4ClsdtdA8CIn49rWbsCCVVOC2FTRsCy7RwC4QpAFQZ0i9HqlZkSA5YjAQsAPofvfg7f1Xn0\nke53X38mUMnw29/8KitguepcAuEKQBUGdtvJ4GSjpN93NQpYVhwDAFr6+TuVLmAbaot8dXNXkNJH\nCJ3mH+tkBSwd5xQIVwAmPbjrRGHHTEm/82o0IBKwANDGz9+N9M5AN7bfE6IMsYPQzIMPpMOVfecU\nCFcAqjDAu172s8shVIkGRVccBwBa9LnbSW5q3A0IXnjmsvBkSGEXpYzVKyvOLRCuAEx6kDcTDfI6\nEwpYrjoWALTkc/fUtsuzDz+kzsqIQtHfVLhyaPcgEK4AVGGg11uevFPy712Mgp3rjgUADf+8XUqv\nuggrMQQmowm1acKjVKm2XHOOgXAFYNKDvbiw3tIEA5ZNxwOABn/eHth2OR8vP/+01SsgXAGo5IBv\nLRmcHEzgd4fnz+/0ar8YHAHQwM/Ze1at3LzxkaBkjN2DMorbrjnXQLgCMOlBX1i9cisZnKxOOGDZ\nEbAA0LDPWatWcvbmqy9avQLCFYBKDvyWJjk4Of5n9tjt3taKx6YdFwAa8Pm6YNVKaatXVp1zIFwB\nqMIA8GCSWySnApYDAQsADfhsvW7VSjFee+m5dLiy65wD4QpAFQaAnWRwEorMzkzoNUxHIU94VGnW\nsQGgpp+rU9Fjr3YIKmDnoPSqIOMGEK4AVGUguJ0MTjYm+BrigOW2gRIANf1MXY4n/uExlvA4i2Ak\nP08+8Xg6XLnq3APhCkAVBoKdZOVK0Jng65hKitt2k7t+HccHgJp9pm7HE/9nL68IRHJ27Y1X0uHK\nbeceCFcAqjIY3OhtjTzh1yFgAaDOn6enHgna2/lAIJKzb7/8tDt18aJHg0C4AlDJweBMsnKlW4VA\n4/ifzagWzKJjBEANPks78YR/+v77hCHlPRq04hwE4QpAVQaFV3q79lTk9QhYAKjT5+haPOG/tPyU\nIKQgLz//dDpc2XQOgnAFoCqDwvBIzmEySFmqyGu66q4UADX5HD1Vb+Wdt14XhBTkxvZ76q6AcAWg\n0gPD1WhL5CkBCwAM/Xl1qt5K2DZYEFKMsANTRt2VGechCFcAqjQ47G2JvFah17Riy0UAKvzZOaPe\nSrnmH+ukwxWPEINwBaBSA8SlZJByWJXVKwIWACr+2TkfT/TDxF8AUqzLl36XDldWnYuVuiamk0fl\nNq0qEq4AtPkDcTcZqGxU7HUtRrsaKV4HQFU+n+IbACcTfwFIsV576bl0uFKpMYtr4iRc6dXyu5Ns\nnFCZm3YIVwDK+kDsRDv1VOpug4AFgAp+bq7HE/0w8ReAFOvDd6+lw5Ud52LlrovwuNxWXHjY41vC\nFYA2fiD2dj24XsHX1okKB+64EwLAhD+XNuOJfpj4C0CK9cXeJ3YMqs/1ER6b24+DsGNz2ka4AtCW\nD8KZaIVIR8ACAH0/k05twxy2ChaAFOubz/eEK/W7TlajR4W6yYqvaW0jXAFow4fgRvLht13R1zeb\nLDHtJrsc+YAGYBKfR7vxRH9v5wMBSAlS4cod52ItrpXp1GN0IWxZ0TbCFYCmfwDORHcYKvmMrIAF\ngKqFK1/d3BV+lB+udJ2Ltbpm5pKVx73jFx4bmtc2whWAJn/4rfWCiwq/xukkWOkVS5t17AAY4XNk\nKpno7UbC/19NWYhF339buFK+2YcfSgcsu5RiI+PaOK+D1DG0dbNwBaDRA87e6pVlAQsADfysm47q\neI1NuDKxcIVmsBJZuALQ2EHnUvRcbGULx0Z3HrvJILnj+AEwQsASr0xZTt1h34zu3B8KVyZv+v77\n0m2/QCnWclq1sp4KNY+SP7dJgXAFoNGDzt6qkLWKv04BCwBFfcaEner+2Vl33m/e+Ej4oaAtg6+l\n5VRIueVxIOEKQFs+BBej1SuV//BL7jD27oJUshgvALUKVf7FsI812Iq5eN99/ZmtmOt5Lc0nxWvj\nR4AUshWuALTuA3E7+SDcqMnrFbAAMM7nyP/42H+eEaD83479fb9w5eP33xaAFCw8eiVcqdW1FHag\nvJ7agnlV2whXANr6wdiJwopaLN1Mnt3tfZCvOI4ADPHZ8T899i9Tk/e/TnZJ+R8e+8voz2+lw5U3\nX31RAFKwvZ0P7tkpyLlbyWspPK59JVVbZV3BWuEKgE73j3cdtmv0mgUsAAwbqvzr1KT962P/uFdk\n8/if16O/+yfRqs67nr28IgAp2DtvvZ4OV647hyt3PS2mwsdQE29O2whXAHjl7rLOo+RDslOj170S\nfbhfdSwBiD4jOqmQ5B+SWhA/z/i6f9erFZH82UE6XPnFz34qACnYC89cTocrV5zLlRsvxqu7PJ4t\nXAEg4wNzvY5LcAUsAJwRqvzVsf/o2P+gz9f/v5Kv+673eGzqcYcTcz/5sQCkYL/+5c/T4cqSc7pS\n19ZUsmXzqq2VhSsADP7APKzjYCZZotpbebPpeAIIVZKA5OqgOhDH/1yLvv7fS/5sOqpF1vts6U5d\nvNj99stPhSAFmn34oXS40nFug3AFoI4D07V4WbSABYAmhirJ9/2PkkeFwvf80+jP56LHHk49HvTh\nu9eEIOXtFHRkdQQIVwDqOkCdioqU1W4rvWSAfScqsGZQBiBU6ff9/7a3fWzqzxd7j8kmOwkpajuZ\nYra1KbIPwhUAsgabS73BZh3DCQELgFBliJ/xh+R7/z5diyWpJ3GyCjIKWk6Ex1YEIcW4tPxUOlxZ\nc76DcAWg7gPXgzpX6T/+Z/bY7d4jTqMMuAFobqiS/Jz5Y3+X/Ix/P+PvewXeryYrOo/iSX94fEUY\nkr+ZBx9QbwWEKwCNHMD2nneeEbAAUPdQJflZ09Fnw9U+X7OZ/P1K8v+78aT/tZeeE4bk7Mb2e+lg\n5dC5D8IVgKYMZnsD2Y0av4fpaBVOGEzPOrYA7QtVMj7bdgd8TS9MWUz+f82WzKU/EnTdNQDCFYCm\nDGrnom0oOwIWAOoaqqRCksNBqzKTlSt3el8T/p1+NOiLvU+EIjn57uvPTra5ToUrC64FEK4ANGmA\nu9GEiv3JM/M70SC94/gCtCNUSX72fBSQLJzj++PX1X35+acFI8XtEnTb9QDCFYCmDXTju3XzAhYA\n6hSqJD//zDorQ/yMpTgACMVXw4oL4cj4Oo8+kg5X1l0XIFwBaOKg90qvKGxD3s9mVKx30TEGaGao\nEv2eM+usDPEzppLHie6GAG+++qJwZEwfvnstHawEc64PEK4ANHHwGw8olwQsANQhVEl+11B1Vkb8\nWVavFLdqpdaPIYNwBYCzBpSryaDnVghbGvKerkaDuRXHGaA5oUry+8aqs5Lx8+5ZvWJb5txXrXRc\nKyBcAWj6oLi3485ag96TgAWgYaFK8jvHrrPS5+eeWr0Sdrn55vM9YYlVKyBcAWDoAeVStLS6EatX\nkve1Eg3srjrWAPUNVaLfPXadlT4/dyp5H3dDgV//8ufCkhGFejWpYOXIqhUQrgC0aaDc221no2Hv\nS8AC0IBQJfn9udVZ6fPzV9OPs3z8/ttCkyGFlT5hxU+qDRs1rgDhCgDDDJh7d5hmGvbeFqNn8zcd\nb4B6hSrJa8i1zsqA37MfhwOzDz/U/fbLT4UnQwgrfVLBymGZ5wgIVwCoyuB5u6kBhIAFoJ6hSvI6\nCqmzMuA9H8UhwZNPPG73oDO88MzlrCK2q64jEK4AtHEQPRMNKDsNnST0nqffaVJ9GYAmhirR6ymk\nzsqA37eeDgqevbwiROnjD7/fyApW9l1LIFwBaPOAeqPJlf0FLAD1CVWS11RonZU+vzMUt91NBwbX\n3nhFmJKyt/NBVp2VcKxmXVMgXAFo88B6JhkUhcHRYkPf42y0vPzA8+CAvr96oUryukqps9Lnd8eP\nIt3dnvnG9ntClcRXN3e7Mw8+kLU70LzrCoQrADrqP94lPGjwexSwAPr7ioYqGeHG1Qm9hrl0/ZXp\n++87CRXaHqyEGjSdRx/Jehxo2bUFwhUAXrm7HPqw6YOkZOB+kLzP25YwA0KV6gTNZddZGfA6ltIB\nQggV2l7gNmNnoGDd9QXCFQCyB5OHTa5LImABhCrVClWS11l6nZUzXs+VdJAw/1in+83ne61csfLb\n3/wqK1hpZK02EK4AkMdgshc6rDX8fU4lxW17E42O4w8IVSb2WidWZ+WM17WVDhRCvZGbNz5qTbAS\nwqQ+jwIdKBAPwhUA+g8kF6NB+MTvHApYAJobqiSvd+J1Vs74nDhIBwuhyG3YirjpwUoIkTKK11r1\nCcIVAIYcTPYG5Rsteb+b0W4HjdwtCRCq1OAzZ7eir2861a53PXt5pbHBSgiPMrZb7q1YEayAcAWA\nIQfovbCh0atXBCyAUGWir71SdVbOeK0bWQFLKPL67ZefNipYeeGZy1mhSjc5zyp/XkGj+3yNAFC7\nwfr1thWrSyYjvQHkivMAEKoU+vorWWfljNe8nN6mOZj7yY8bUYcl1Ff5xc9+2i9YsSsQVKEf0ggA\ntRu0z0QDyE6L3nccsKw6F4AK9lO1DlWS91DZOitDtv9hv8eE6rqK5bWXnutO339fVqgSxgLLrj0Q\nrgBw/gHkepWfgy/wfa9Eg8paDfoBoUpN3kul66wM8frDDYj9rIAlFIC99sYrtQlVbmy/d7Lyps9q\nlRAizbv+oEL9j0YAqOXgcSq6O7fUsve+Yik0UJH+qDGhSvJ+alNnZYjPyK0+oUR39uGHuu+89Xql\nQ5Unn3i8X6jSTcKjWdcgVKzv0QgAtR089gbBBy1874vRo1Gbzgeg5D6oUaFK8p5qV2dlyM+KW4NC\nlrCSpSqPC3347rWzQpVwnq25BqGifY5GAKjtoHEq2Xax28bBVjpgCe3hvAAK7ncaF6ok76u2dVaG\n/Ky8klXsNr2zUNji+LuvPyt9lcrlS7/rV1MlttmWXQKhtv2NRgCo9aBxKVrC3bpwIZno3EnaYEfA\nAhTY1zQuVIneX63rrAz5HmeSLZsHhixTFy+eBC1hRcsXe58UsutPCHEuLT91UgPmjEClt8Vyx3UI\nNehnNAJA7QeMvdUrV1o86YkDlkZMdoDK9C+NDVWS99iIOisjvN+hQpb40aEQtrz8/NMnoUhYaTJs\nkLK380H34/ffPvnesDql8+gjw4Qp8UqVjusQatS/aASARgz+e1sytnLJcCjsFy1pPxCwADn0q40O\nVZL32bg6KyO895nkmN4eIfDIXOkS6qQE4b/H+VlJwLWuWC3UtF/RCACNGCT2JgEbLW6DdMDi2XRg\n1H6kFaFK8l4bW2flnMf9erQLX5mOkp2NFl2DUPO+RCMANGJgOJcM0I7avIw4mSz0HpO67e4fMMLk\nuhWhSvSeG19nZYxzYS1pn6OCApX9ZIXKgjaHBvUfGgGgMQPCjV7xu5a3QzpgmXN+AAMm0q0KVZL3\n3ao6K2O2VXh0ajk5L8IKk90RQpf9pBZY+N6VEKYovA4N7i80AkBjBoAz0YBvvuVtMZUMaHuTpY5z\nBIj6iFaGKlFY0Mo6KwCF9q8aAaBRg+be3cgDbXFPwNLqwAlod6iSvH91VgCK6mM1AkDjAoVeQb4l\nbXLSJptR0UAFA6Gd/UCrQ5WoHdRZASiqj9UIAI0bPK8mg+dbnu2+2yZxwLKkTaA1175Q5Y9toc4K\nQJH9rEYAaOQgulfQdU173G2Tq9EEa0WbQKOvd6HK6fZQZwWg6L5WIwA0ciC9FN2htHrlj+0SByyr\n2gQad40LVe5tE3VWAMrobzUCQGMH1L1irhva41S7rEQTLxMNaMZ1LVTp3zbqrACU0d9qBIBGTzZ6\ndUY8X3+6beKAZV2bQK37OaFK//ZRZwWgrD5XI0AjB1Nhx5iFY8vJIPNqMvjc7WM7+rrl5Hs9StKM\nc2ErGVhvao972mYxqkGgffRJ1OucEqqc3UbqrACU2e9qBGjMIHMteQzkKBpsjuso+ZnhZ3e0dS3P\njZnonHAM722fUwGLCbw+iVqcW0KVs9tJnRWAsvtejQC1nhRuJUt9uyU5TH7nsmNQq3NlIzl+29qj\n72TtTtJGOwIWfRJClQa0lzorAGX3vRoBajVYmgv1IUqevPRzJ7nTv+DYVP68mYnOmSVtMlTAYsKm\nT0KoUtc2U2cFQLgC9BkoLSR1CIaeaDz5xOPdX//y592Xn3+6+85br3dvbL830IfvXjv52kvLT518\n79TFi6NMasLS4xXHqhaD7QPt0beNZqNl9Acmb/okhCo1bDd1VgCEK0DGIGkpmeQNnEiESccvfvbT\n7puvvti9eeOj7t//m7/MxVc3d08mQb/9za+6Mw8+MOyEZs1jFZU8l6ai1QUeoRg+YHHXV5+EUKUu\nbafOCoBwBUgNkObPmsBM33/fyR3dcHc3r4nLWb7Y+6T7wjOXh5nUHJrAV3Zi3Ds+JpuDJygH0eR8\nVpvokxCq1KAN1VkBEK4AycBoJqkZ0HeSEJbV/+H3G6VNXvr5+P23T+4enzGh2bWjR+XOsd4EeU17\njBSwzOmT9EkIVSrcjuqsAAhXgGRgtBoV1LxHuCOc5/L6vHzz+V732csrJ3etB0xo1q2UqMx5thBN\nYAzAB7fVVFLcttderZqU65MQqtSmLdVZARCuAMnA6Hq/SUC4ExvqDFRtApP27ZefnhSfHDCh2TVo\nrsz51pvQbGiPkQOWeX2SPgmhSoXaU50VAOEKkAyKMnfcmPvJj092zKj6BCbrrnF4TGBAccmOY1+J\nyU03udM5q02GarPNqM0W9Un6JIQqFWlXdVYAhCtgoBndbTq1y8ZrLz1XuwlMVv2D2YcfyprMhMnp\nknNg4udfb2XCtvY4V8DSuHNYn+QcF6rUrm3VWQEQrkDrB0TL0fPRd4WBfxVrGIyzLP/JJx7vd8f4\nqnNhoufgTHQOdrTJ0O12NTqHV/RJ+iSEKhNqX3VWAIQrYHKWNbAPA/4w8G/KJCYWikv2mcxsKSo5\n0XNx3XLysa/hVX2SPgmhSsltrM4KgHAFWj8g2soa0IeBfhMnMLFrb7xy8nhBxvs/MOie2Pk4lSwn\n73osYuS2W2nCigd9kj5JqFLLtlZnBUC4Aq0eDK1n1TIIA/ymT2J69nY+6Ldzx45zZGLnZe+Z/QPt\nMVbAsq5P0icJVYQqJfbZ6qwACFeglYOh5fTgfebBB04G9m2ZxPSELVw7jz6SNZmxLfBkzs2p5E59\nOAZr2mTk9luM6h5s6pP0SUIVoUqBba7OCoBwBVo/AD1KT2LCgL5tk5ie777+rDv/WCdrMlP7+hU1\nPUeXojuh6k2MGbBUvQ31SfokoUot212dFQDhCrR6MDST3to0LLtv493htG8+38vaFvXI3biJnasH\nBu1jTzjv9B4pqWrAok/SJwlVatv+6qwACFegtQOh8LjFfvpO6B9+v9H6SUxP2OI1o6BkGLDPOocm\nMnHqTSY9x59PwFKpSac+SZ8kVKntMVBnBUC4Aq0eDN2zC8cLz1w2gUn58N1rWUvxbxm4T+Sc3VZr\nYuw2nI1WhlRq1xl9kj5JqFLL46DOCoBwBVo9GLqSHpz/+pc/N3Hp47WXnsuazGw7l0o/b+eSQXzQ\n0Sa5BSwTv9OsT9InCVVqeSzUWQEQrkDrJ1anikWGnShCwUSTlv5++5tfZU1mlpxTpZ+/G8Kt3CZF\nvTo2tyf5WIk+SZ8kVKntMVFnBUC4AgZDduHIZbcOu9eUf/7ORBPxeW2Sa8Ayp0/SJwlVGPK4qLMC\nIFyBVg+GltJ3Ot9563UTlfGKSa47tyY2qD/QHmO35VRS3LY3ae3ok/RJQhXOODbqrAAIV6D1k6jD\neBD+5BOPm6CM6NnLK1lbobprN7lzeUmb5B6wzOuT9ElCFfocH3VWAIQr0PoB0Vo8AA93O8NdT5OT\n0Zfih8cWUpMZu9eUfz6vRI+zeDQrnzbdjCbni/okfZJQhYzjpM4KgHAFWj0YuucOcbjbaWJyPn/4\n/YbVK9U4r3v1Qta0RyEBy5I+SZ8kVCE6VuqsAAhXwIAofYf4m8/3TErGEHYzsXpl4uf1ksLChbTr\n1ei8XtEn6ZOEKqizAiBcAReRO8SF+PDda1avVOP83hFuFR6wrOqT9ElClVb3B+qsAAhXgON/lt0h\nLu1OsUHnZCZqwq1i2naliHNbn9TOPkmoUuu+QJ0VAOEKEN3ZP3Fp+SmTkJyELWNTE5nbzrmJnONb\nSftvao9CA5Z1fZI+SajSuj5AnRUA4QqQLOU9igfbezsfmITk5NsvPz25656azCw490o/z2ei87yj\nTXJv38WofTeH/J61rGOhT2pPnyRUacS1r84KgHAFSAZGq/Ege+4nPzYBydlvf/Or9ETG6onJnOsb\nSftva4/CA5atQQWEj/+Z7bdqQp/U/D5JqNKYa16dFQDhChANjnbjQfZrLz1n8lFOEUk710xmItAr\nkrqkTQqbNN9J2njnjICl93Vz+qR29ElClcZd7+qsAAhXgGRgNJVefq9oZP6++/qz7vT993k0qBrn\nfK82wIH2KCVg2e03cQ6rJZKvWdMnNbtPEqo0ui9VZwVAuAKEwXQ8uA67SJh4FOPXv/y5XYOqEyj2\nVq8sa5PC2nk2elzgIGsSHRXC3dUnNbNPEqo09vpWZwVAuAKkBkgb8eD62csrJh3l7dBhGfXkzvul\n6I6rx7PKC1hmU38/HT2SMq1Pak6fJFRp9HWtzgqAcAXIGCQdxIPrj99/26SjIF/d3FV3pZrn/pr2\nKHwi1mvr2xkBy25cA0efVO8+SajSimtanRUA4QqQGiBNpQbWJ8/hm3QUJ+x6kmrzjnNxYuf/QjT5\nUy+g3IClE/3dld5uNfqk+vZJQpXWXMvqrAAIV6DxE5ftZHIyM8L3zdnutFwZ258uOYcneu30JoMb\n2qOQtt3qrVRJgpOdaOLdSfVDh/qk+vVJQpVWXdPqrAAIV6AV4cphNLC9MszS7qjuxIlQ3NBko1gv\nPHM5PZG54hye6LXTiR6HmNUmhfRJR8lkeyojYFlIvr5Xv+Hf1yfVo08SqrTymlZnBUC4Aq0Y+Mwk\nd4m70dL7xTO+Z03hyIkXkNxy/k782ukVUN3WHoX2SYdRXZXNKHhZjI7BJ/qkavdJQpXWXs/qrAAI\nV6B1A6CwbHc/GviGu8Rzfb72ejyovvbGKyYbBdvb+SA9kdl33lYiBDhSAye39pxK6qscJH3Mq8f+\ns3hHmuTxnzhg6fVF/1qfVM0+SajS6mtanRUA4Qq0ejC0Gi3LD9bTg+BocnPixvZ7Jhvl785x2/la\nietlXdiVW1tOp/qenv9vFGJ1k9Uqr2d8nT6pQn2SUKX117M6KwDCFSCZ5KynluWvRH+/ayJTrrDz\niXClktfKVBQILGmT3CZlV5KJ+Z0+AcrfHPv/CFeq1ycJVVBnBUC4Atw7QJqLikh2k8eG5tPhyhd7\nn5hslCA9gXSOVuY66S19P9AehfVDq8mKua8GrVjRJ022TxKqkFyz6qwACFco+MN2K5mUU46NZGCb\nh4PUAPrb+P/D8nATjeJN339feiLjPK+Gf37sr5Nj8mVD3+NOjv3JuP6DY/9yULiiT5pYnyRUMdZT\nZwVAuEIJH7i3h7njSP2YyJRj9uGHnG+gT6p6n/RfJBPsWWOf1o3z1FkBEK5Q0ofuAqVay+ku8fqF\n03UPjlL/byIzuYlMWC3xj53rlfFFclz+owa+t8WKrFj5p8f+xbG/Fa7UIvA9ELS0Zow3fUGdFQDh\nCtB3sLR84fTuHeHRrrD9rJor1ahv0Au7lpyvlbheOtExsRx+vLaciQKdnQvZuwipuVLNPukPF07X\nXhG0tOOaVWcFQLiiESBjkBSW9u6nBsXz0d/bLWjyE5lwh/B69P9rzt1KTTA2tMe5ApWtAUHK3104\nvT2zrZir1Sf92+i/l5KVDCuCllZcu2sX1FkB8HmgEeCeyU08YQ8DpdUBE0gTmZJ88/lev21Pr0R/\nZkI/+WtoLqo50NEmI7XdwoXTRVF3k0eBPon+/L859lfR//+DcKU6fVLyGGlv9VYnOraCluZet+qs\nACBcgWhwNJVM0uNaKmGQPN3n6zfjwfEffr9hslGw8JhDekISHY/laHAb7vxPOa8nej1tJMdiW3uc\nK2CZ7dMnpbdh/osLf9yl6R/0SdXok5I+qBfOz2YcY0FLc65XdVYAEK5ANDgKdQ1uRYPbUN9g7ozv\nuRoPiF9+/mmTjYJ9+O619CRkK2NS2puI7l6wDeokr6mZKOya1yZj90lfZUzCN6NHEYK/1CdVo09K\ngrH9KCyZPmNyLmip77WqzgoAwhWIJoG9gWyYzCwO+X3L8SD4t7/5lclGwV576bn0xGM947h0oruI\nt0xKJnptraVXGDFyMHWYPAaUPvevJF/3N8n//9fH/hN9UnX6pNSKhp0hj72gpZ59nDorAAhXILnD\nGAZIq6M8ShLtinJi/rGOyUbBLl/6XXqysdLn2MwmE5HeoLfjXJ/YtdUrzLqkTUZqt//w2P8zOtf/\nq6iOx0rydRupRxg/0idVq09K6g/1VtNdH/E8ELRU+zpVZwUA4QrkOAG6O9idvv8+k42ChcliaoIx\nf8bEZCcqDLrovJ3IdbISFfpUB2f4dptJztt/deyz9Hkchbu9Oithovcf65Oq1ycljysejbOjmaCl\nctenOisACFcg5wHW7XiQu7fzgQlHQb77+rPu1MWL6UnF9BAB2Gb6jj+lXycHtso+V7v1XYEVBYfx\nrlmb6Ym3PqkafVIUMo69ikvQUolrU50VAIQrkPMAa1MByXJ8/P7bfXcKGuI4rUffd9W5W/p1shQF\nBNPaZKg2i2sHHcST5ag9/7v4cZPUY0L6pIr1Sf22aBa01O7aVGcFAOEKFDDIiu9Gdp984nGTjoI8\ne3klPXHYGPFYrca7rDh/S79Wds5z3FraVn13vUpWY/VWs/y3cb2H9A5m+qTq9UlnbdEsaKn8tanO\nCgDCFShooDUbD2DDEvFvv/zUxKMAnUcfSU8WFs5xvBajgfGOGiClXiud6K69u73922klOke30udo\ndNf8X/Um6dHf3ROu6JOq1SeNskWzoKVy16Y6KwAIV6DgAdepuivvvPW6iUfOvtj7JD05ODpvMJJM\n8u9EkwwT/fKulS0rhwa2z5VBqyCSIre93Zc+Se9AE4Urd/RJ1e2TzrNFs6ClEtenOisACFeg4AHX\nqToHv/jZT00+cvbCM5fTE4KdMY/ZbDS5Cf+ecy6Xcq3MRKsyOtrkVNtcj87v1TP6mt3ovzsZ4cq+\nPqnafdI4WzQLWiZyfaqzAoBwBUoYdHXSg9RvPt8zAcnRzIMPpCcByzlNMvajO/3zzudSw8ht7XH3\nMZHtaPXD0oDJ+MBgKqor9E/0SdXvk/LYolnQUko7qbMCgHAFShx8HcQD02tvvGICUtyOHOd+JOi8\nE1tyn9Ad5rElbUPa4mCYgC86T6+fNWHXJ9WnT8pzi2ZBS2Htos4KAMIVKHEAthYPRkOhQ5OQfPz6\nlz9PD/Q3Czh+8SMZa87p0q6Xgxa3wdCPpiWFmEcqBqxPqk+fVMQWzYKW3NpCnRUAhCtQ8gAsriVx\n4sN3r5mIjOnmjY+6GYP7xYKO4ZXzbvPMyG09FQULKy18/+miytNnfH1vFcoVfVIz+6Qit2gWtJz7\nvauzAoBwBSY0ENtwp7jwO8QHBR/D5UHb4JJrWy9FE5fWtHPyvofeDjyqozJyO+mT6tMnlbVFs6Bl\n6PeqzgoAwhWY4GDMneLi7xAvlXAcF6JVBbtVnOQ06Jo5aNOjWFFQMtSjJMmE+9z1afRJ9eqTyt6i\nWdAy1HG4qq8GQLgCkxmUnbpTHHaU+O7rz0xMziFsH1vmqpXUcexEg+tbdswoNMjqFXNt9LL7qK7G\n0I/3RP3JgT6pHX3SpLZoFrScej/qrNDv3JhKPrfCKterie3kRkyW7ejrlpPvtSIWhCvAkB+8M+nB\n5QvPXDYxGVG4uz6JVSupYzkbraw4rEKhyYZeM9tNrnOTDMY3o4KlKyP0JQO3XtYnNbNPmvQWzW0O\nWtRZIXU+dJJzYie9CnBMR8nPXDO2AOEKMPjDOL5D3Z26ePFkObkJynDCXfVwdz01ENmZ0LGcTgZA\nvdUVi87xQgavvcFmZSddY5w/u+c5f6LJ6ZY+qX19UlW2aG5T0KLOCsl5sJjUXDvMMUw5y2HyO5cd\nAxCuAKc/mOOdUE48+cTjluIP6fKl32Xd4Zmb8PEceeUBI7Vx79GV7Qa9p3OvfEomeSNtvaxPal6f\nVLUtmpsctKiz0vrPoLnkc6jMQKWfO8mYQ8AHwhUg+aBeTH9ghgG6icpg1954JWugsV6RYxrf/Tf4\nzrdtc3kEpkLvJ67ZczDqpDEKZdb1Se3uk6q6RXPTghZ1Vlr72bMQrS4cSgimw65hLz//dPedt17v\n3th+b6DwSGH42kvLT518b1g5OMLvu+2GDghXgNOD4rvefPVFE5Y+wiAkY9Bxu0rF30bd7YWR2vZq\n0q77DRisn3u3qaToYSFbVOuT6tcn1WGL5roHLeqsCFX6Cdd/KGQd+sk8H6X86ubuSTDz29/8KuuR\nQyELCFeAPoPGW+kPyY/ff9vEJWOgMX3/fVlL7zsVPK6L0SqLHZX/c51Ennvb4Yq8h5Xo3Nga9dxI\ntcGKPkmfFB23WmzRXLegRZ2V1n3OzEVhZaZw3YdVJmVuW//F3icnhcaHCFpuOU9BuAJt/iCfje5i\n3/3gDh+kJjDf+/bLT7tzP/nxxHcHGvG4dqLjeuBuZ27tulb2tts5vvYr0bm7cc6fcbXo969Pqmef\nVMctmqsetKiz0qrPlqnk0d6+O/6ER33+8PuNideiCmF3WNFyRsiyZdwBwhVo64f6PbUOZh9+6GQA\nbyLzlydLbjMGDldqEpzdjpbszjnfcxkAH9RtC9ow2Y3O3dVz/oyZsu6g65Pq2SfVeYvmKgYt6qy0\n5vxZyFqxF4cqVdw9LayeC3WxBtRoudOmfgCEK0D84b6WVRSt7bt1PHt5JfOOTM0G/fvRQGfe+T52\nmy4VVXOkoDBoO3pkZGmMn7VZ5o5J+qR69klN2KK5CkGLOivtHXv1hBC1DlvSf/P53km/NCBk2ar6\nZyUIV4AiPuSvZ90xaevd4j6TmP26DRLynGBzt033q75UP5nkHeQRrCWPmfXOn1l9kj7pjOPWmC2a\nJxG0qLPSinNjKquAdxDqmpRZTyXPuizzj3X6BSz7QkIQrkAbP+zvqU4fnu0Pyz/bVM8gTOD6VMOv\n7eAgNVFdc86P1ZZx2FC5cyLvR8KiMGlDn6RPGvLYNW6L5jKCFnVWWnEuzETB9ymhaGzdV+eFXYYy\nim33+gKrZ0G4Aq0bAN7Oqk4ftv1s+iQm3HnpUyjyThPuwOZR1JS7bbldxXbMKGY8PebPix+DmtYn\n6ZNGCMYau0VzgUHLXzVhy3f6Hvv5aMe1xvZnIfzuPPpIVr8VbkgsOxdAuAJt+vDve1flzVdfbOwk\nJgxsBtxtqX2wEh3f5XG24+VuO85G7ViJ8yMJQnLbhjuZIN+e9GonfVI9+6S2bNFcYNBSyPbOTOxY\nr2btBtTUlXhhBU6fFXdu7oBwBVo3COj7PHDYgq9pRSXDBK1NzwknuxP0Vjfsuqt87nbcKLPI6xAD\n9955u5nTz6zM1tP6pHr2SW3bonmMdgorGv67pJ3+haClsZ8Vrash9fLzT/fry3bc3AHhCrRtQJBZ\nyT4ULQvL1ZtQyyBMzPp88G82+YM/eXykd1f5lkH7udqwtO2Jz3gd63lvyZt6b0v6JH3SGMeslVs0\nj9A+99RZsaKlUcd3Net6DqFDW+pG/eH3G/12ExK4gnAFWjcwiFc53BU+KOtcfC0UXQtV+ftMYlox\nAUgebTlo4uNPE5jsH0zgd09FWySHyetKjj+7Mqty9En175PavkXzGW3TC1B2B4QvgpZ6Htv59KNA\noZ+q425A4wrbSs8+/FBW37bqXAHhCrRtgDCXrG6o/baB4QN+wHaBd9q29WUycN+J3v+ic37kgOOw\n7Eljctx2izhuqd2QKhm46ZNqea3YovneNlmLwu2ZIa97QUs9ju1sunhtCFbC9d62YKXnm8/3sgKW\nI1uOg3AF2jhQmO4zoDsRKsNXeUITBjQDiqv1Ho2Za+mxLWwFREvabyXaGneqhN9X6Iqjqu6EpE9q\nxLVii+Y/tsX8OI8VCloq/5m6nz4ubVyxkrUDWkah7jvOVRCuQFsHDctZWwnGE5rwfG1Vlubv7Xxw\n1gQmDG6vKqx2T+2Oq873kdruoIzHN1K1cg7yHpBGWy/fqUvhVH1SbSedB23td7PqrOTw8wQt1Tm+\n9xTffu2l51ofrPR8/P7b/cJkBW5BuAKtHRiuZ20r2BPuTFxafupkO9FJ3Bl59vJKv+d709XqDThP\nH9vcd51pSbstRXfkpwv6HYXu8pRMfEsJifRJ7e2TbNF8dp0VQUutj+1a1q5AQpXTQtiUcX5WrsYX\nCFeAMgcRs9HjJH2FCUWY1ISCjV/d3C3kOd5wZ/rypd91537y47MmL72J6YJj2Pe4LkaTVNsljj5h\n2ijgZ69Ex2SriGMSTQpqewdRn1Sb4xRv0Vzpx88KvM4Oi14dJmiZyGfnPSvnmrZVfF767Ix21bkE\nwhUQsgwxoYknNuFOTtiKMExARrmTHL42fE/43vAzhpy43L0rIlQZ+ph2osnPQVGrMRrYZr3HOmZy\n/LlXonN4o6DXPj2Jwrz6pPb2SW3conncOiuClkof26n044lhtVwR4W1ThNCpTzHvjnMKhCvgYv1+\nQnN9UP2DYYSK+k8+8fiJ8N/j/KwkILjuw/rcx/N2VKy1lQV/R2yz3rP2Wzn9vOtlbFkZbb28q0/S\nJ5V4fOItmhu9U1nedVYELZU7vhvpPmMSjyDWcQehjC3o951TIFwBTg80lpKJ5tGYE5Hz2kkKXXqk\nZfxB+H40KZzXLgPbayY65ztj/JypaOJzVORqkuQRjaOm3zHUJ1X2uLRii+Yi66wIWibehp10v/LC\nM5eFJ0MKuyhlnHsrzi0QrgDZk8SFZPeL3QInLgfJnaMlj7AUcgxLmeg3pL02xinOV3agFR3b6/ok\nfdKEjslWtEKucRP5MuusCFom0m776ccM1VkZTcYOaoeCaBCuAMMNROaTO7hXk0H17gh3k/eTr7+a\nDPQWTFxKO26lPKLSgHY6d/2Ssh/FSq6f3OvE6JP0SecIvBq5RfMk66wIWkppp6V0+4SVGAKT0YTa\nNBmPXK7pH0G4AtDkgWThxVUb0k69O9UHI3xP6UWEo62XrzhuVGAy36gtmqtUZ0XQUngfatvlMYWi\n4FavgHAFoG2DyeWitwVuQBtNRZOqlSG+vvTtr8PqIwNYKnbdNGqL5qrWWRG05NYW96xauXnjI0HJ\nGLsHZRS3XdM3gnAFoOmDyoVoErTrMYiBA++B4UUUcgSbJYY/jdl6mcb1LbXforkudVYELWO9f6tW\ncvbmqy9avQLCFYBWToI60eqMA8/fDxx8r/X5+/VoEHmlxNe1PupjS1Di+VnrLZrrWmdF0DLS+12w\naqW01StqvIFwBaAVA+rZKEA4bPJWqmMOwE8VjE1WjmxGf7dS4mvKZbtoKPg8reUWzU2psyJoOfM9\nXrdqpRivvfRc+pxpzGN1IFwBYJiB9E60dXDt7jQX3D7bcQ2JSbdX9Hq2HB8qfu3UbovmJtZZEbTc\n856mosdi7RBUwM5BGedKrc4REK4AMO5gcyIrMWrQNp2oXf4nk1zpk3otrd16mVr1K7XZorkNdVYE\nLXeLut997eExlvA4i2AkP08+8Xj6/LjqGgLhCkDbBs8TqSFSg3bZSNrku0nWqImCnXXHhRpNyCu/\nRXMb66y0NWhJv+ZnL68IRHJ27Y1X0ufEbdcNCFcA2jhoLn33mxq0yVPH/iFpk7+YxO5K0d1Wuy9Q\nt+un0ls0q7PSrqAl/UjQ3s4HApGcffvlp92pixc9GgTCFQBCHZHoLu5OmyfzSahxFA0Q/9MJvIZ4\n6+UV5yg1vI4qu0WzOiv1DVqSRyXvDLvSMnq08sT0/fcJQ8p7NMhnFwhXAFo7SO5Ed/gOJrFaowJt\ncCUaGP6HUcCxNKHXYetl6nw9VW6LZnVW6h20JKuijobd8jc63icuLT8lCCnIy88/nT7mVsKCcAWg\n1YPj2Wi5fPj3XIvee7xV52pqYF5acc7U1ssLzktqfl1VZotmdVaaEbSkQrulM7721O9/563XBSEF\nubH9nrorIFwBIGNQvB9tPTzf8Pc7FQ3Aj9KD9aio7FpJr6e3i9O285GGXGMT36JZnZVmBS3hGEZ9\n9vyArztVbyVsGywIKUbYgSmj7orVYSBcAWj9YHhg4NCwQf/AICm897IKy6a2Xp51LtKg/mSiWzSr\ns9K8oCUKou9kfU+yClC9lRLNP9ZJH9NKPA4IwhUAqjAQvudRmQa9t6EfgYomhlcLfk27Vd1hBXKY\nVE9ki2Z1VpoZtCSh3U7Uh0+n/n4+/plh4i8AKdblS79LH8dGjRsacs1tJ8GkvlC4AsAEPozjIq+N\nmPSPWrw3taJkpqDXFK+QmXbu0cC+pPQtmtVZaXbQknzfQdaqqFRtlpOJvwCkWK+99Fz62LlRUL3r\n7DBa8XWlzbtDClcAmNQHcrw98VadP4zPu+10NOjfKOA1TUV39decczS4Lylti2Z1VtoRtKRWIe5E\nf74ef2+Y+AtAivXhu9fSx2vHOV2562smqoPVW/Xl8S3hCgATmBT17jrv1nF1RViifN5tIpMBfG9S\n2Mn5da3ZepkW9SWlbNGszkp7gpbUasTryfduxl8fJv4CkGJ9sfeJHYPqc23NR4889242tWaHSOEK\nAFX4MO5EdwgP6lR0NXUX88o5f8ZG3jv5JHeR7gyzrSg0qC8pdItmdVZaGbTsxH18+uvCVsECkGJ9\n8/mecKV+19Vq9KhQN+mbp7WNcAWAcj6IZ6Nn3A+LmBjl/HqnojuYYSK3klMQspDT68s9sIGa9CWF\nbNGszkrrg5aeL+L/39v5QABSgtQxuON8rc31FN+AOhxnrIRwBYDRP4h3oqJolXxet4jXmecjPKlC\nuR3nFi3rR3LfolmdldZ+Hl2Pgu+ev4///6ubu8KP8sOVrnO0VtfSXGoFWOif57WNcAWAciZGuawI\nKej1FbLCJnnfh3k8xlNkkVyo0cQ4ty2a1Vlp3GfMTlLjqyf8///22P8pmfh9PWDlypFwpXyzDz+U\nPg67lCKsgr2ak4PUMbR1s3AFgJIGwGPXMingNRVaGyYqyHn7vHfbo62X7xi00PI+ZDaPLZrVWWlk\n8PZXA8KTkQhXJhau0AwHarEIVwAoZxB87l14CngtpexqFN3ZWTvH906N8/3QwD5krC2a1Vlp7Hnx\nrzImeX997Nax/yRZyfLPk77+0GNBkzd9/33p47VAKdZyWrWynnrE7ij58yl9knAFgPIGwYvR5GZn\nEh/Ex/8sR69hq8jXEK08ORw1wInusN8yYIFT1+/IWzSrs9Loc2IjCU7Wkz53JuNrOn0K296O///m\njY+EHwracnYfHIeUW1YBClcAmNwHcye641HqEtJk681umTVMzlMzJZkI5lKzBRrYh1wdtcizOiut\n/ryJQ5U7Ud96kKxosRVzib77+jNbMdfzWpqPiov3rh+FbIUrAFTgQ3o2umMY/j1Xwu+8Hg0KVkse\n3PcmgjNDfs+GiSAMvEaG3qJZnRWhShKqXE09WjYfFVw/8fH7bwtAChYevRKu1OpamkmNnw7LHEMh\nXAFguA/s6eguyJ2i7oAktUu2o4BjaQLvtTeA3xria+eiwX/HuQJ9r+vds7ZoVmdFqJKEKtOpOlbr\nydfHk8bum6++KAAp2N7OB/fsFOTcrWwfeyVVW2VdwVrhCgDV/vAuLPgoK8AZ4nXMDBuYRO1x3TkC\nZ17ffbdoVmdFqBJPBKPHye7WsYpWNZ149vKKAKRg77z1ejpc8VlXvetpMblOulGNvDltI1wBoB4f\n5Lk/sjOJR4/OeD0b/SaB0dcsjPoIEbS87+i7RbM6K0KV1Nfds4IpmUTenej/4mc/FYAU7IVnLqfD\nlSvO5UpdUzPRsbk1SuFwhCsAVOcDPbdis5MsmjvgNZ1ZpDZasm6wCcNfW/ds0azOilAl9fX7Wask\nkscw7070537yYwFIwX79y5+nw5Ul53Slrq2ppP9ctVOhcAWAen+oj71NchW2ex7w2noTvoOMv1uJ\nJoMGNDB639GbrK2psyJUib7nSrSCcSrj73vnSnfq4sXut19+KgQp0OzDD6XDlY5zG4QrABQzeF6I\nVp3sjrLqJLnT0huwbVbwvU1FjyqtpP7c1ssw3vXVq6nxD+qsCFWS74sLhC/2+ZqDeLL/4bvXhCDl\n7RR05GYCCFcAKH4gfTt6rGd2iO9Zr8Mz3CE8Sa9QiV77geMPY11fX0eTtllt0s5QJfr+3bPC9qge\nlqK25Rez3Xaeg3AFgOIH1bPRHcXDfkuHk1Ufm9GEaqUG7+0genxh6J2EgIHXVe+xu785a4tmmh2q\nJD8jftRyesDXnSpqGx5bEYQU49LyU+lwZc35DsIVAMoZYE8ndVN6g+vFUf6+wu9rPgqDehOILccc\nxrqmeiHl/2zQFs00O1SJftbV5JxYPOPrpuK6K0F4fEUYkr+ZBx9QbwWEKwBMcLCduTJl2JUtFX5f\n26nnzu1oAue7lqajMOVq1D/cyWP3MeoVqpzz9+/Gk/7XXnpOGJKzG9vvpYOVQ+c+CFcAmMzgO66p\n8r8ZtSZLRScTvffzv3eM4dzXUm9Svpv683u2aEao0ud1rNmSufRHgq67BkC4AsDkBuKrqcHZbtmD\n8Bzfy1L0Pv7Pji+MNSk+zFr9ldqiuRaPDQpVJvJ6ZtKPBn2x94lQJCffff3ZyTbXqc/vBdcCCFcA\nmNyAPEyU/jYanP2zOhasTG29/LeePYdzXUdxnZWFAV93NXr8znUmVOn32uLX1X35+acFI8XtEnTb\n9QDCFQAmN/C9Eg3M/mlUT+GgbqtXovdyEE389h1nGPoauqfOyhlfv9Wb1NmiWajS5zXGqwlPiq+G\nFRfCkfF1Hn0kHa6suy5AuALAZAa919NbNyYFK29HE6a5mryXePn5QmoVy5LjDUNdR5l1VgZ8/VRU\ntNQWzUKVfufIYRwCvPnqi8KRMX347rV0sNKty+c1CFcAaNLAfCoamB+lw4fk7vVBNGifr8F76gVF\n29GfrZn0wdDX0MA6KwO+b9oWzUKVIc8tq1eKW7Wy7RoB4QoA5Q5yw0Ro/6zg5KwApoITjt7rnE39\n3YFdTeDMa2ioOisDvt8WzUKVQa//ntUrtmXOfdVKx7UCwhUAyhvgjvzIT+rRodWKvq/dfpO66Hn/\nQ6tXIPP6GanOyoCfY4tmocqg93Jq9UrY5eabz/eEJVatgHAFgFoO0uNitaMs+4+L3lbqrnQqPJnu\n8zX7iv1B32topDorZ/wsWzQLVfq9p6noM+jEr3/5c2HJiEK9mlSwYrcuEK4AUOKgdjG6o7xznhUc\nx/+sRD9jqwqrQJLB+q2z7pSnHhuacU7A3WvjXHVWzviZtmgWqvR7f6vpx1k+fv9tocmQwkqfsOIn\n1YYewwPhCgATGMxujvmzFqI7j7uTHvDHBWuH+NptA1E4dU2MVWfljJ9ti2ahSr/3uh+HA7MPP9T9\n9stPhSdDCCt9UsHKYRPPERCuAFDFQex6NAi7muMk4Hb0eNHshN7bTBT0LA3x9bPRRLLj/KDlfUMu\ndVYG/HxbNAtVBr3nozgkePKJx+0edIYXnrmcVcS2knXQQLgCQJMGr2FisxktzV/J+efPRrvwHE4i\nrAgrUEYt5Hee74GG9hG51VkZMsCxRXPLQ5XU+19PBwXPXl4RovTxh99vZAUr+64lEK4AUOygdTqp\nq9IbsC8W+Ht2i/49AyYmR6PWdEitdllwvtDSPiL3OisDfpctmoUqWe0Qr2y669obrwhTUvZ2Psiq\ns3LocTsQrgBQ/ESmtBUlRa+QGfB7z10/ZZQ6LdDAPqKwOisDfmfrt2gWqmS2Sbyy6e72zDe23xOq\nJL66ududefCBrN2B5vVnIFwBoNjBe2+geqvMu1qpJd5XCv5dS9HkZOYc3z+VBE9D1WqBhk5mr5b8\nu1u5RbNQ5cz2mUvXX5m+/76TUKHtwUqoQdN59JGsx4GWnTsgXAGguAHqxHfxyXNXojN+z8G4d8Cj\nid6hQpu0qJ8ovM7KGb+/NVs0C1VGaquldIAQQoW2F7jN2BkoWHfOgHAFgOIGpsvRnb+tSYYFySC5\n91p28n4t0SM9t8b92XmENFCjfqK0OitnvI5Gb9EsVDl3u11JBwnzj3W633y+18oVK7/9za+yghWF\n2EG4AkBJA9JKFItMJhd3oi1Yp3P6udN5Ps4T3S09NPGh4f1E6XVWBryWRm7RLFTJpQ230oFCqDdy\n88ZHrQlWQpjU51Eg25mDcAWAAgei16OB11rFXttsVNsh/Hsuh5+5kfcjDeMUxoWa9BMTq7My5Guq\n9RbNQpVc23IqWlF4qsht2Iq46cFKCJEyitc2dpUXCFcAqMoAdDuqXbBU4UndQTThmB/jZ81Gd947\nOU+Meu044/yigf3FROusnHFN13aLZqFKoZ8b2xkBQ/fZyyuNDVZCeJSx3XJvxYpgBYQrABQ08NzP\nI7Ao6fXmEgRFP+N6Aa+xt5X0lnOMhvUXlaizMuD11W6LZqFKae28kRWwhCKv3375aaOClReeuZwV\nqnST88x5BcIVAAoYbOb+qE2Jrz1+hGn1HBOwwlaXhJ9ZxKoYmPA1V5k6K2e8zlps0SxUmdi5cZQO\nHeZ+8uNG1GEJ9VV+8bOf9gtW7AoEwhUAChzYx0Via/cIy3mL70aPFl0p8LVtNKH+AyTnc+XqrJzx\neiu7RbNQpRLtf9jvMaG6rmJ57aXnutP335cVqoRrYNmxB+EKAMUMLheL3N645PeyMsq20cnX9x5r\nmCrwdU3luRMRTPg6q2SdlTNec6W2aBaqVOrcmIkeh71nN6Frb7xSm1DlxvZ7Jytv+qxWOaz6o74g\nXAGgzoPK1WjgtdmQ97QQrcLZ7TdZKTvwiOpTHDj3qPH1Vek6KwNedyW2aBaqVPr82OoTSnRnH36o\n+85br1c6VHnyicf7hSrdJDyadaxBuAJAMYPJ9WjgdbVh760TPbaQuRtC9P4PSnpNU9FrWnEOUsPr\nqhZ1Vga8/olt0SxUqc05ElZy3hoUsoSVLFV5XOjDd6+dFarcqUsxZ2hln6MRAGo/eJyKdrA5aupE\nPynQexDdZe9EfzeRIrNhhUwZjyFBwcHE1Zr3C6Vt0SxUqe1n5JWsYrfpnYXCFsffff1Z6atULl/6\nXb+aKrHNOtZPg1b1NxoBoPYTpJ1okF/Z3TNyfL+76fcbLf/emsBrOqjT1rCQnLe1q7My4L0UvkWz\nUKUR5/xMUox8YMgydfHiSdASVrR8sfdJIbv+hBDn0vJTJzVgzghUelssdxxDEK4AUNxAse9Kjoa/\n7/RKnZeL3Hp5iNczP8nfD+c4Z2tZZ+WM91TIFs1ClUae/0OFLPGjQyFsefn5p09CkbDSZNggZW/n\ng+7H77998r1hdUrn0UeGCVPilSqt+FyHxvQvGgGgloPDuAbJrTYWtkvVmAnWJ/hatst6LAHGPFdr\nXWfljPeW2xbNQpVWXAszyTG9PULgkbnSJdRJCcJ/j/OzksBzXbFaqGm/ohEAajcgHGr3nJa0xfVo\nUPofT/B1dJIJXTDnPKWi10sj6qyc8R7H2qJZqNLaa6OTfJ4cjhmOnMdRct42+rFeaEVfohEAajUA\nXI7uOm+1uYhqauvlv+3tGDLBLVk3es/HO1ep6DXTmDorZ/QLI2/RLFQhdS6sJefDUUGByn6yQmVB\nm0OD+g+NAFCbAd+VaGDW+sdPoroRB8lg+E70/9MTeD0T2bEIRrxeGlNnZcB7HXqLZqEKQ5xP88mN\njavJTY3dEUKX/ST0D9+7kqw8tbMcNLW/0AgAtRjcxY+/rGmPU0HGQvJns9GE6vYkHs+Jaj4cOG+p\n2OSwkXVWBrzngVs0C1UAyP2zRyMAVHqCMBVNAMLkaEm7nAqbtlN/Ph3toBQmS/MTOF69R5UcK6pw\nrTS+zsqA937PFs1CFQAK+9zRCACVnhTtTyooqHC7dKKwaTbj7ycaSKUeV7L8m0lfL42vs3LG+4+3\naP6/C1UAKOwzRyMAVHJCMPFHXCrcNjvD1J1JPUq1WvJrPPAIFxW4VlpTZ2VAG3SS7ep7fcF/K1QB\noJDPHI0AUMnJQFyctZWToj5tsxRNFqeH+PqJFAFOvU6rV5jEtdK6OisZ/Wj8+M/fJP/+V/pUAAr5\n7NEIAJWaECxGE6IdE/NTbTMV3YFeG+H7ViaxfXW0Hey640fJ10qb66z0q6ny3zvPFs0AMPRnkEYA\nqMykYDWaEGxqk3vaZ+28O/EkhS17q4F2y3gkIFUbxp1yyrxWWldnZZhCtanQadu5AkCun0UaARo5\nyJxKJpPLyeDyajLo3O1jO/q65eR73dUr95itR5OCq9rknvaZGXcXnmTydTu6cz1b4iT3uj5Jn1RS\nW7eqzsqou/+ctUUzAJz7M0kjQGMGl2vJYyRH0SBzXEfJzww/u6OtC5t0bkbtvaJdMttpI4+7zcnE\n6iCafHYKft2z0TXZmmtInzSxdm9NnZVxtlRObdG86twBIJfPJo0AtR1YLib1Iw5znLic5TD5ncuO\nQS7HcDra+SZMDBa1S99J1FGik1O775bV7nkFQ/okfdIQ53Xj66yME6qkfk68RbO+FwDhCrRs8DyX\nTNTKnLz0cydZcbHg2JzrWJa6gqLmbbWd9xL+MlcMJY803WniagJ9UiWvk0bWWckrVEn9zKvRz9IH\nAyBcgRYMmheiO+1DefKJx7u//uXPuy8//3T3nbde797Yfm+gD9+9dvK1l5afOvneqYsXR5nU3PY4\ny8iThN4d5ltl1P6ocVstFlk/IlXr5kqB7+PcxXj1SfqkEc6vxtVZKSJUSf38reicUXgaAOEKtDlU\nCZOOX/zsp903X32xe/PGR92//zd/mYuvbu6eTIJ++5tfdWcefMCEJr9jWuquNTVvr4NRt14+x++I\nd2m6XtDvmBq3IK8+SZ/U55g0ss5K0aFK6tq0RTMAwhVo6GA5LLXfHzRpmL7/vpM7uuHubl4Tl7N8\nsfdJ94VnLg8zqbnlcaHM47ocTYK2DOLPbK+16HyaKvh3LUXHZqeI3xfVeDis27HXJ1X2uDSuzkpZ\nocqAdrRFMwDCFWjAoHIqeUyh7+4aYVn9H36/0f3u689Km8Bk+fj9t0/uHp8xodmyzPqeoMD2n8Nf\nC6Wu9EgmdXeiO9jTBfyOwlfi6JPa0yc1qc7KJEKV1O+3RTMAwhVoyCB5Ibm72ncCk+fy+jyX6V++\n9LtB9RDu1GUiWeCxvR61R6vbYoQ26+2ws1Py752N7mDfzrseTtE1ZPRJ7emTmlJnZdKhSsY5b4tm\nAIQr0IBB8j1C3YIqTmDSvvl8r/vs5ZVBE5rWPQaT3PXfjnakWXK+Dx1w9CY4nQn8/ulohUmY6M3n\n/PO3q353XJ9U+Wuk9nVWqhSqpF6XLZoBEK5ATSffW1kD/1BDoMzaBXnWQJh/rNNvMrPflseEkgn6\nflET9Ia33XaRxWVHuDYLCcaSSWXv51bqetAn1aZvqW2dlaqGKqnXaItmAIQrUKMB5kx0d/yUUKBx\n0vULxhV29AgFLjPe32HTg4Zk5cWt6NGSOef80G03X6XgIfVI12qOP3ezt3pCn6RPGvE41bLOSh1C\nldTrtUUzAMIVqMkE8jBrt40b2+/VegKTrn3QefSRrMlMmDgvN/TYpouiGpSP1n6VK/h6/M+V6Nxd\nzzHImNijT/qkevZJdayzUrdQJXrdtmgGQLgCFR+wrWbtvDH3kx+fDPybMonpCXe7Q+HLPkvyG7Uj\nQ1KstNDtfBt+baxUdavi5LXluo32pIr26pPq2SfVrc5KXUOV1HuYjlYh2qIZAOEKVGigttFv141v\nv/y0cZOY2MvPP91vMtOIECIKBoJN5/vI7Vf61svneI0L0aqk3XEniVV4z/qkevRJdaqz0oRQJfV+\nbNEMgHAFKjZAW80ayIcBfpMnMLE//H6j384d12t+bNej91LpiU+F27BXQHK/BhPH29GjArNj/rze\nYx4H+iR90oBjVfk6K00LVVLvbd4WzQAIV6B6A7MTYUBfx503xhW2cJ19+KGsyUztBqzJyoPNqGbD\nivP9XO1YqfojQ7ze2ag2zO1xXnNyDt0q+xrQJ9WnT6p6nZUmhyqp92mLZgCEK1CBidhhehITBvRt\nm8T0fPP5XtZk5qgOdQSi4zqdPD7Qm0wYbJ+/LbeqtnPOkMd/N4/jHx4JKrPWjD6pPn1SleustCVU\nSb3nK7ZoBkC4ApMZiIW70vvpO6JtvDuc9sXeJ1nbot4Z9zGLko5rvGXtoUH22BO0ymy9fI7rO5eV\nS2XtkqRPqk+fVNU6K20MVVLvf9MWzQAIV6D8QdhWehLz2kvPtX4S0/Px+29nLcW/VeUCt6maG7fq\nEAZV/BrZr3utmlTNnSvn/BnzZYRM+qT69ElVq7PS9lAlagdbNAMgXIGSB2BrWTtwmMCcFiZ2GZOZ\nSm55mfduMa6Rch+HKfi9rI5bDDWauG7ok9rdJ1WpzopQJbNNbNEMgHAFShp4LaYH551HH+l+9/Vn\nJi8ZfvubX2VNZq5W7JguR7UPttytHLs9422IVxoUFh2ddzvfZBJ7lJjTJ7WzT6pKnRWhypntY4tm\nAIQrUOKk8UR4jv+rm7smLX2ECd78Y52syUynIsc0vuNvEJ1vmx407H11ognXwagT0XB+5X03XJ9U\nnz6pCnVWhCrnDsJs0QwgXNEIkPNgayO9C8eN7fdMWIbYrWPmwQfSE5n9ChzP69HrWXOO59Km8dbL\n8w18f7PRBPn2KHV5itiWWp9Unz5pknVWhCrnbjdbNAMgXIGCBqdH8WD8hWcum6gMKexYknGneGVC\nx3IqmmiEY7rkHM89sGpsrYJkBcJBNEmdH+F7r+S1qkefVJ8+aVJ1VoQqubShLZoBEK5AzgOsU1uc\nzj78kJoGIwoFNlMTmdKLnSYT4/3zTIwZaiLXC6xmG/5ezxXQpR7jWdInNb9PmkSdFaFK7u1pi2YA\n4YpGgJwGVkvpO5zhrqfJyWhCHYjw2EKqLddKPI6z0S4Qt/MuLOo6OSn02qraNalHy1aH/J61cbd6\n1SfVo08qu86KUKWwdrVFM4BwRSNATgOrA1uc5uPl55+eyOqVjGKk7j4WE0DWfuvlc7z3K9H5vD5i\nn7KmT2pun1RWnRWhSinXuS2aAYQrQE6Txrtu3vjIpGSMnToyCkmuFXwMF8fZRpcz23cqmnSstbQN\nVkbZznucMEqfVI8+qYw6K0KV0q9zWzQDCFeAMQZT7hDn7M1XXyxt9Uoy6e39nk3ndCFt3Mitl8/R\nDgvRxGv3rAlu9JjBuj6pWX1S0XVWhCoTvc5t0QwgXAHOOVlyh7icO8WrBRy/9ejnX3VOF3KNzEQF\nWlu/VWky6b0dPX42e8bX9grizuiTmtEnFVlnRahSmevcFs0AwhVgxAHUdXeIi/HaS8+lJzK7OR63\nqWh3h6NJbfnckmtkQw2Ce9pkNlpdcnvQ9q3RRPm6PqkZfVIRdVaEKpW8zm3RDCBcAUaYoN+xG0dx\nu3Sk78DnsX1vctd4Jxr0uqtY3DUyl4RXRyYXmefh7lnnYbLyp/eIQWdAXxTCwv+5PqnafVLedVaE\nKpW/zm3RDCBcAYYYNMXLfk+WjIel4yYh+XnyicfTE5mrYx6zmWjFwKEJf+HXyLbCjgPbZ6gVVPHq\nn2TifCqsih4F+kKfVN0+Kc86K0KVWl3jtmgGEK4AQ04cTzx7ecXkI2fX3nglPZG5PeZkpFfn4FYe\nq2AY2N6LRe+E0qC2imv/XMn4+5loRcp2ushtFK78W31SNfukvOqsCFVqeX3bohlAuAKcMWA6tfx+\nb+cDk4+cffvlp92pixfHfjRo1F1ayOX6OGjz1svnaK/V6By/nvH3vcdJvkxP6qOtl/9Wn1TNPmnc\nOitCldpf37ZoBhCuAAMGuncH19P332fiUd4y/JURj9VytBR/y7LsUoOCW9p7pHZbis7VnbjtkscL\nDqOJdfj3fPJ3K+laIPqk6vRJ49RZEao06vq2RTOAcAUYMFg+cWn5KZOOgrz8/NPpiczmOY+Tu4Xl\nXBtxCLCkTUZuv04UnhzEk+ioztN38aNBWeGKPqkafdJ566wIVRp7fduiGUC4AqQGSKfqrbzz1usm\nHQW5sf3eeWscxFvSrjlvS7s2esVXd7THudtwNqrPcTt+7CR63OrutZAVruiTJt8nnafOilClFde3\nLZoBhCtANDg6VW8lbNFp0lGMsNtJRo2DmQHHZiqanBxZPVHqdXHmtsEM1Y5hMv3Osc+jSVjvEaBe\noeC/7z0alBWu6JMm3yeNUmdFqNK6a9wWzQDCFSCZQKptUKL5xzrpicxin2MT7hTvpyeklHZtbPcr\nyMpI7dh7rOqvjv3n6aAwNQkPOw39r/RJ1eqThq2zIlRp7TVui2YA4Qo0anAznQxqN0e5c5TcKb47\nqA6DbJONYl2+9Lv0RGY147jMRttdhruBc87zUq+n+SgEcCd2vIDqkygkDP51fO6nCmr/l8fe1idV\np08aps6KUAVbNAMIV6BpA5t4940rw9w9Si/BD4Nsk41ivfbSc+mJzEbGRCUuAmpyX/71ZOvlfPuk\nMDn/P6SClXi1ymb0//9cn1SNPumsOitCFVLngy2aAYQr0JiBzUyyPW83WvGweMb3rMeD6jDINtko\n1ofvXktPZHai47HYb/taSruOlqNHILT/eBOt/36qT/qvj/0fo///h96d7mP/Lvnvb/VJ1eiT+tVZ\nEaow4Lq3RTOAcAUaN7iJl+Hv9HusJHXH+GSQbbJRrC/2PsncnSO1imjTuTyRa8fWy/m040J0LodH\nBf6vx/5V9Gf/aSK+Dv4mY1WLPmlCfVJWnRWhCkNe/7ZoBhCuQOMGOKvRRLG3/H469TWntmEO23Ka\nbBTrm8/3siYy8Qqiq87fiV0zV5NjsK89xmrH2VTA28+Xx/560NfokybSJ/2/4zorQhXO0QfYohlA\nuAKNG+BMpybuIWxZif5+Nx5U7+18YLJRgj537I/iY0Pp14qtl4tp14VkFcT1IQOXrj5p4n1Sb2vs\n/51QhTGufVs0AwhXoJGDnLnk8aDeIHk/eXzoVLjy1c1dE43yJzK9SYvl09WYCGxpj8Lbej6ZpP+X\nZ4Ur+qSJ9Un/VqjCmNe5LZoBhCsM8YG5lXxgUo6NZGCbh4PUAPqvTWTKN/vwQ+mJzF84zyfqL6I7\n9nsNf687OfYnRfRJ/06fVIk+SahCHuNFWzQDCFc448Py9qhLu6kHE5mJTmQAfVKV+qT/Inmka9bY\nhzHGjLMXbNEMIFyh7wflAqVay+kO8Xo0wOnV+Dg0kSnf9P33pScxzvPJeenCH7cA/vda8H4XK7Ba\n5T849k+P/YtjfytcqWSflHYgaGGMcaMtmgGEK9CYgc1yKkgJj3aFAp6naq7cvPGRiUb59Q3uOEcn\ndl1MRSvxVrRJIW08EwU6O+lA9yz6pMn0ScnjHCsXUjvKCVoYcxxii2YA4QrUdjAT7hbtpwbF89Hf\nnwpXbHtavO++/uyerZidqxO7PtZ614X2yD1Q2RoQpPxd9N//1YU/7phlK+YK9kmCFnLsG2zRDCBc\ngVpObq5fOL0F82rG123GA+WP33/bZKNg4TEH4UplrpHeMvV5bZJr2y6kVkHsJo8CfZLqk/5J9Khi\n71j8tT6pun2SoIUc+gdbNAMIV6AWg5ap5M5QXFsl1FqZ7vP1cQDTffPVF002Cra380F6QrLr3J3I\ntbJxwQ4WRQcsswP6pP9FFKjsRBOu/0yfVI8+SdDCGOMUWzQDCFeg0gOWUNfgVjS4DROWuTO+Zy0e\nED97ecVko2DvvPV6ehJy3flb+rXSiVZLmPyV3CeFVXTRn20mX9t7hGhDn1S/PknQwjnOF1s0AwhX\noJIDlZloIHtr2GJxyeTn7iD4Fz/7qclGwV545nJ64nHFOVz69bJjW9BS+qSjdJ+UrFo5de5HjxHd\n1ifVv08StDDkeWKLZgDhClRykDKVDFhXR1lim9xFvjvwnfvJj002CvbrX/48PdlYcg6Xeq0sRTU/\nLEcvtk/a6PVJic1oxdBK9LXXo8eF9EkN6pMELZxxftiiGUC4Ao0a3PQGNt2pixe73375qQlHgWYf\nfig9weg4D0ud8B8k7b6mTUpr9+moxsKd9Mq66JGgeX1Sc/skQQt9zgtbNAMIV6AxA5uDeJD74bvX\nTDjK25XjyOqJUs91Wy+X3+azUR9zmJ64x48E6ZPa0ycJWkidD7ZoBhCuQCMGNQpITq5wpEJ+5Z3n\nM9EKCXdHy2nzTlJHpTdhns34muvpIqr6pHb1SYIWkvPAFs0AwhWo/YDmVAHJsETcpKMYl5afSk8c\n1pyDpYeIAq1y2nshKla5O2A7+F7gtaBP0icJWlrdZ9iiGUC4Ao0Y0BzFg9iwVNzEI38zDz6g3spk\nzvG55Bw/0ualtPdK1Kds9ZskRY8EHeqT9EmCFmzRDCBcgSYMaHbjgetrLz1n4pGzG9vvpScHh869\n0s7vbdt9ltbWV6JzfOOMr73nkSB9kj5J0NL6PsQWzQDCFaj1YGbN9qelL7+/7twr5dxejIqpeo6/\n2La+Hp3fq0N8/Ua/1RL6JH2SoKXVfYktmgGEK1DbgcxMehn+F3ufmIDk5LuvPzvZUjY1CVhw7pVy\nbtt6ufg2noomuqEfWdIn6ZMELYx5bG3RDCBcgdoOZE4NTl9+/mmTkOJ25LjtnCvlnF5N2vuW4oiF\nTm4Pom1U5/VJ+iRBCzkdU1s0AwhXoJaDmKV4MBoKHYa7myYi4+s8+kh6oL/unCv8fJ6KdqJZ0iaF\ntPFstNVy+PecPkmfJGgh52Npi2YA4QrUejJ64s1XXzQRGdOH717rZgzu55xzhZ/PvXoeO9qjkPbt\nREUnD/pttaxP0icJWsihH7BFM4BwBWo3iFlzp7jwO8S2lyz+PI7rdXS0Se7tuxS1706Rkx19kj5J\n0IItmgGEK1DHAcw9d4ptgZr7HWKT/eLP4207MhXWtqvRubypT9InCVoo6XjZohlAuAK1G8CculMc\ndpT45vM9ExN3iOty/naiXWs8n59v265H5/IVfZI+SdBCycfJFs3jt2EIrBeS3ZiuJraTR6+ybEdf\nt5x8r0ezQLgCjPDBeyceXP76lz83MRlRqA2RGqAfWbVSyvlr6+Vi+oTN6Dxe0SfpkwQt2mpCx8cW\nzaO1Vyc5Z3fS29uP6Sj5mWv6ERCuAIM/jFfTH6Qfv/+2CcqQwl31cHc91YaWMZc36D50Zy3XieZu\ntB3qoj5JnyRoEbRM+LjYonlw+ywe20o/Ulmww+R3LjsGIFwB7v1w3o8/OGcffqj77ZefmqgMIdxV\nzxh0TDuvCj1fbb2cf5vORiuBDic9idEn6ZMELUTHwxbNp9tjLnl0s8xApZ87yfFZcK6CcAV45e5S\n0lNLSJ984nE7dZzhhWcuZw00PBte/Pnau5O5rz1yu/5vRxPHiU8a9Un6JEEL0TGwRfP37bAQtcNQ\nQr8ZAteXn3+6+85br3dvbL83UCiEHb720vJTJ9+bsQpukNtlP0oKCFegqh/a6+kPymcvr5iw9PGH\n329kDSxM9os/T229nP9gvVfjZLdKKxz0SfokQQuptm/lFs1hhWa0srCvEIT84mc/Pam5dPPGR7n1\nLV/d3D0JZn77m191Zx58YNiQZc0juyBcgTYPXKay7ohce+MVE5eUvZ0Psu7mHBpUl3Ke9paHb2mP\nsdtyJQqqtqo2ENYn6ZMELaTaPN6ieb0F73f+rFBl+v77TlaZhBUnZfU3X+x9crJKboig5VBdFhCu\nQNsHi7fTd0LCUlETmD/ewckYUIQJ6rxzqPDz09bL+bXllToUO9Un6ZMELWQEDo3eojlZobk5KLgI\nj/qE1WqT7n9CsfGwouWMkGXXSlMQrkBbBy5z6VoH4c5IGMC3fRIT6j10Hn0ka+Dgzkw552ZvFcNV\n7TFWO16vUz0OfZI+SdBCqo2XoxCxUVs0J7ul3ekXVIRVKnk+8pPnLmXhsc3QNw8IWdY9KgTCFWjj\nwGUp/aEYBvBtLyaZsQtHK5YmV+yctPXy+dtwKprwHdVppyV9kj5J0EKqbeMtmuca8p6u9wsmwuqQ\nOgTKYVe3UBB3QMhSqdpegHAFyh643DX/WOfk7kQb7w73WfbaqqJ6Ew4Feo+GrGiTc0/yDqLJSO0e\nGdEn6ZMELaTatBFbNCfnRuYuQHM/+XEtH4MM/XKf8Ld3vDrOYRCuQNsGLlvpD8XwbH8Vl6QWOUDo\ns+y+tdtBTuA8XOu1ufY4V/vNRuHU7Trf5dUn6ZMELUTtGBe93q/j+Z/UErudtfPPay89V/v+KtRk\nmX34oaz+qlarJ0G4AuQ1cDnI+tCvQiG1ooUJW59q+LcNfEs7B2ei588V6DzfwP1ONGmb1ifpkxC0\nNKz9arlFc1I75ih93EMY0aTAODwq9OQTj/dbxXLVeQzCFWjbwCVr0HdSvKypk5gwUcvY2rQ32DXQ\nLe/82/C4w7nbbikauO80ZVWDPkmfJGjRVql2m03qcdWm5lAIFbL6sBBChDCiiX1Y6J/7BCxbVt2B\ncAXaOsm9ZzvApg0EXnjmcr8BwLZCbKWec/HWyyYUo7XdanTebuqT9EkIWhreXrXZojnr8camh8M9\n1954ZVBIrC8D4Qq0avCSuYQ1FFxrwhLWUMvgFz/7ad8tBJ0DpZ9vvUnFhvYYqd3Wo/P2ij5Jn4Sg\npSXttFT1LZpT/fPdxxpD6NCWulF7Ox/0201ox/UOwhVo2+ClEy2/veeuS13vGIfCcX0+7MMgbdmx\nn9gg2dbLw7fZVLR7xlFbdlbSJyFoEbRE7VPZLZqTMPiegtwhbGjbjmdhW+k+hbndTAHhCrRu8DKT\nVObvZg0U6nQHJmxxGO5y97kzfKiI6sRCgl7R0jVtMvSEazeaVFTyrq0+SZ+EoKWEdrletS2akxD4\nKN03hZChbcFKT9hSfv6xTlY/V+nHukC4AhQ1Ad7qMwE4qXb/zluvV3oCM6B6fW9bx9YNSitybtl6\nebT2mo3CqDD57uiT9EkIWtoatCR9wU5VtmhOwt/b6UeB2rhiJevRx4ytmkMIteC6BuEKtHFAtxht\ng5g5oQl3jauyNP/Dd6+dNYG5Y7XExCcIvUc8WrX64pzt1YkG7XaN0SeBoOWV6mzRnAQ996yqa8PW\n8cMK9bEyitze8XkGwhVo60BuKnnO+WjABOFkF48woAhLQcu+I3z50u/61S+IbVZlCXGLzyVbLw/f\nVgvJALSbPBI0rV30SSBoufveJ75Fc9ZqurD7l1Dl3pA54/y85XMNhCvQ5kHcTDI5HjihCXcowqQm\n3D3+Yu+TQpaZhgnTpeWnTp5pPmPy0tvOtOMYTvz8mUvOnSPH48y2Womusy1Ff/VJIGjJfM8T26I5\nKq57KtAVpvQv4p3VF7p2QbgCJjRDTGjiZfphwPHy80+fTEDCXd1RtvT7+P23T7433AnuU31+0F1h\nE5jqnDe2Xh59wK6t9EkgaBn8XkvfojlZNXOqvwl9Qdmr5Ormt7/5VVa/uOSaBeEKuFi/n9BcTRdy\nG1W4qxxqEgQZz+WOKiwRXvcsb+XOlYXo+HgMon87Xbejgj4JBC0jv8dSt2hOt2XbdwYacwehQ6sz\nQbgCnB5odJKJ4eGYE5HzOEoenVAgtbrnh62XB7fPVDRYP3InT58EgpaR31spWzRHK2XuqvJOZTUp\ncLvu+gThCtB/UrOWDN6OCpq87Cd3gxe0eeXPh9WoeJ27U9mTnf3oruu8dtEngaBl5PdT+BbNye84\nFdiGFW5Ck9E8e3klK5C2qhWEK8AQg5FQcG45Wa6/lex8cjTChGUn+d6V5PESE/R6DXYPPVfdt31m\no0dYbpexnB19EjQ1aCl6i+akHU49QhhWYghMRn88KKP4txpjIFwBYMBAdD0ZNO1oj3vaphNttXxg\nS0pA0JLL6y9ki+asVSthBYaw5HxCMXGrV0C4AsBwA9GZaDVAR5ucapvFqG12rHwABC25vu7ct2jO\nWrUStmEXlJxfxm5rVq+AcAWAjIFobzB+XXucapfVeGtebQIIWgp5vblt0WzVSjE+fPea1SsgXAHg\njIFox0Aps13Wo0HkFW0CCFoKfZ25bNGc1GiyaqWc1StXXWMgXAHgjwNRWy+fbo9w13MzCpxWtAsg\naCnl9Q29RXPyONFSxp/vxO/r0vJTgpGchG2sU+fMbdcVCFcAeOXUHb5DtUTuTjx2oruni84TQNBS\n2msaeovmZOewbvwak/d0ajexvZ0PBCM5+fbLT09WAqXOlQXXEwhXANo+sLb18un2mI1W8Rwq7AsI\nWsoPWobdojl6rcvRn8V1srpzP/mxUCRnv/3Nr9Lnh3pkIFwBaP1guvd8+762OKk7czuaSMw6RwBB\ny2SClmG2aI4+w9ajP9uNX+9rLz0nECmnsK1d9EC4AtDawbOtl//YFgvJI0DdZGA+7RwBmGzQctYW\nzdEOQzvJ/0+lHwlSyDZ/3339WXf6/vs8GgTCFQCSQWivYOtWy9thORqMb7n7BlCNoCVZUfj/67dF\nc7K65eQxzuT/F+LXFHa2EYYU49e//Lldg0C4AoCtl++2w5VoYLjh3ACoTtAStmNOrUS5Z4vmaNVh\neD0b8et49vKKIKS8XYN2XRcgXAFo46B4t+13mqLtPjOXmwMw+aAl+bmntv6NbwokOwqFP1+MCpKf\n+Pj9twUhBfnq5q66KyBcAWj9IHipzVsvJ8/kb0eDwSXnBUB1g5ZwIyD5/r9Pb9EcBeX/y/TvC7VB\nBCHFCTsxpdq84xoA4QpAm4KF3vaWKy0d9O9Hy8vnnRcA1Q9aojph/y7eojnaevn/YgvmiW/JvOS8\nB+EKQFsGumu9QW0L3/tstNXy7fRz+wBMPmhJ/v52Uj+lE31/uDmwkwpY1pMdhcJ//8v4Z4aCqwKQ\nYr3wzOX0cbziXK/cNbedBJOtra8nXAGgiA/ZmajwX6tWbCQFfO9Eg/hp5wRAJYOWv0r9/63ksaC5\n5PsOUo8I9QqT/51ithMvatvq3Qcrep0dRqt1r6iLI1wBIJ8P2Y14KXWL3vditNvEjoEFQG2Clr/J\nCFr+18e+ThVSPUp/77U3XhGAFGxv54P08dp3Tlfu+go31rZSBaEXtY1wBYDzf7h2ogHobIve92o0\noNh0LgDUNmj529T//13G19x1Y/s9AUj5Owbddh5X9tqaj2rO9W42eTxauALAOT5UewPVjRa953XP\ngQM0Mmj5d4OCFeFKOcJuTMKV2l1Xq9GjQr26RdPaRrgCwHAfpK3aejkpergZLRdfcR4ANDZoyfTF\n3icCkBKk2935WpvrKb4BdWisJFwBYLgP0V7xv7WWDBh2ouJtnisGaF4/fz0qUp4pPLIi/Cje9P33\nCVfqey3NRWOmbvLYUKs2PBCuADDKB2drtl5OtvA8iO7CdJwDAI2YAK4kRdn3BwQqR8KV8s0+/FD6\nOOxSio1kF608HKSOoa2bhSsApAak8RZ8jV7BkRTsvR1ttTzrHACo5efWUvLIwm7WLkBRP7+Z1I+Y\nT773tnClEuEKzXCgFotwBYA/DlJbsfXy8T8L0fLwXYMBgNr257czJnmHSa2Vq0l/P9Xne3fVXJl8\nzZXkGFG8tZxWraynHrE7Sv688TX6hCsADDtAnY22Xu40+H0uR3c2twwGAGrdp28kIcl6soJlZoTv\n3bVb0MTDFbsF1W8MFe8ctOVxIOEKAPd+YDZ+6+WwvXI0IGjNFtMADPzcE66U5JvP94Qr9bxW5lP1\niw4UshWuAJD9obkQLaVu5B2IZKeI3qBg1XEHaP1n32Y80f/D7zcEIAULj16l63Q4Fyt9jcykxk+H\nxlDCFQAGf3g2duvl8NhPdHcyPA605JgDkNSJuDvRf/n5pwUgBfvw3WvpcGXLuVjZsdOVVG2VdTXq\nhCsADP4AXUk+NG81rf5IsotEbxnrHUtYAYg+I5bjif5vf/MrAUjBXnvpuXS4su5crNx1sZiMCXvH\naCdsb65thCsADP4AnYoKky017L3NRrtI3DYwACD1OdGJJ/rzj3UEIAW7fOl36XBlxblYqWtiJjo2\nIWBZ1C7CFQCG+xBd792VaOCA+U5UdG3a8QYg9VkxFU/0p++/TwBSsBBgpcIVK0qrd02ELZtX7aYo\nXAFg+A/QmWhL4k6D3tdi9L52DA4AGPCZcTue7O/tfCAEKch3X3/Wnbp4MR2uTDsPQbgCUPcB5VbT\ntiRO7rT0BmybjjMAZ3xubCpqW46P33/bTkEgXAFo3GCyE+2e04itl6NHnIIrjjMAQ3x2rMQT/ief\neFwQUpBnL6+kw5XG3NwB4QpAeweTjdl6OXk+eDMKi1YcYwCG/AyZjSf84bGVb7/8VBhSgM6jj6TD\nlQXnIAhXAOo8kFyKdtCpdT2SZKvlnWirZVXtARj1s+RU3ZV33npdGJKzL/Y+SQcrR2qigXAFoM4D\nyMZsvZzcbeytwDlsUlFeAEr9PNmIJ/6/+NlPBSI5e+GZy+lwpVG7FIJwBaB9A8gryaBmv+bvoxPd\naQwBy6zjC8AYnymnJv/ffL4nFMnRzIMPpMOVZeceCFcA6jp4bMTWy+EZ7eQRoPA+dm3jCEAOny0H\n8eT/2huvCEWK2yXII0EgXAGo9cDxet23KA53uqKAaMvgDICcPl/W4gAgFF8VjOTj17/8eTpcqe04\nBIQrAAaNtd96OXqkyfaNAOT9GROv7jzx4bvXhCNjunnjo3Sw0lV8HoQrAHUeNO4mA5qrNX3916NB\n2ZpjCkABnzUbVq8UvmrlwLkGwhWAug4Wl6IddWr1GE2yu9F2tOpmyTEFoKDPHKtXil+14nMchCsA\ntRwohnDiVh0r84dCtWFXo+S1hwK2844pAAV/9pxavRJ2ufnu68+EJecQtrS2agWEKwBNGSSu1XFA\nE7ZWjrZaDv+eczwBKOHzZya92uKFZy4LS0YUVvxYtQLCFYAmDRB7WxbXZtVHUny397oP6lqAF4Da\nfn6ux6HA1MWLJ4+4CE2GE1b6hBU/qWBlx7kFwhWAug4Oe0ubt2v0mhej5913bLUMwAQ+i6ai1ZMn\nnnzicY8HDenypd+lg5UjK1BBuAJQ14FhJxnM1Gbr5eN/VqOB2KbjCMAEP5MW04+1hNBAeDLYtTde\nyXocaN05BcIVgLoOCns77GzU5PXGS7CvOoYAVOCzaSsdFLz56otClD5ubL938ghVqs1uW4UKwhWA\nug4GF+uy9XKy9HozWja84hgCUJHPqOlox727Pn7/bWFKylc3d7vT99+X9ThQx7kEwhWAug4GD5JB\nzWoNBq070VbLi44fABX7rJqNiqyfCCHCF3ufCFUS3375aXfuJz+2OxAIVwAaNQisxdbLyWD1IFph\n03H8AKjoZ9Y99VdmH37oJFQQrvxl9xc/+2lWsHLFuQPCFYC6Dv6mkqCiW+VVIEmx3d4uDGG59azj\nB0DFP2PX0gGCHYT+svvs5ZWsYGXLOQPCFYA6D/wqv/Xy8T8L0fLq3fBokGMHQE0+Z6+ng4Rf//Ln\nrV3B0idY2VfAFoQrAHUe8M1GWy/PVfQ1Lievr5vswGDwBUCdPmunkhsDpwKFUG8kFHRtU42VECpl\nBCthVeqMcwWEKwB1HvBVeuvl8Ox1NPiqxfbQAJDxeTYdPdp6qsht2Iq46cFKKOTbp3jtHfXTYEL9\nkkYAyG2gNx8Vhq3cHaPUMuo1xwyAmn/uzkRF2U9589UXGxushPAoY7tlhelh0n2SRgDIbZB3UMXg\nIlk+3VtRc2RLRgAa9Nk7lTziek/Y8Nvf/KpxhW5DaJT1XpMaKx4Fgkn2RxoBIJfB3Upv6+Uq1TBJ\nlk3vR0uF5x0vABr4ObyWFTrMP9Y5eYSmCfVVQljUJ1jZVD8NKtAPaQSAsQd08dbLSxV6XbPR8+i3\nq1pgFwBy+tyLd8K7a+rixe4Lz1yu7SqWd956vTvz4AP9gpU1xx4q0gdpBICxB3NXkwHOToVeUyca\nYB5YKgxASz6T547dygoiQkDx4bvXahOq3Lzx0cnKmz6hSviMX3DMoUL9j0YAGGsQNxNta9ypyGta\njF7TjqXCALTss3k6qjV2j86jj1Q6ZAmhSp8tlntuWY0KFex7NALAWAO4rSpta3z8z2r8DLZjBECL\nP6OXo8d2M0OWP/x+ozKPC+3tfHBWqHKUrJZ10wSq2OdoBIBzD9o60WBn4o/dHP+zHg3ArjpGAPis\nPlnFsh6t6LxH2Nb40vJTJ1sclx2ohGK7z15e6c4+/NCgUKW3EnXWMYUK9zcaAeDcA7b9KhSTSwrq\nbkZBz4rjAwCnPitno8/KvkLIEYKWUET2q5u7uYcp33y+d7Ja5vKl33XnfvLjswKVYFdtFahJP6MR\nAM41SFuKduGZmuDrmE7uZvWK2y06PgDQ93NzqJAlDlvCozovP//0SSgyyuqW8LXhe8L3hp8xZJjS\nsy1UgZr1LxoBYOSBWSW2Xk4GiAfJ6zisSkFdAKjBZ3n4DL0+qCbLMMI2z08+8fiJ8N/j/KzkJsl1\nn+dQ035FIwCMPCBbSwZB+xN8DZ1k1Uxv14BZxwYAzvWZupQUqD8aMxw5r52k+K5CtVDnvkQjAIw0\nAJv41sthmXByd6v3LPa0YwMAY3++TiWfsVeTz9eiwpSw6nQjCXV8hkNT+hCNADDSwOv6JLc5Tu5s\n9cKdLXe5AKDQz9355LP3avK5uzvCCpf95OuvJqteF4Qp0OD+QiMADD3AmujWy8f/XIkGbBuOCQAA\nVGSuoBEAhuww/7grz9UJ/O7rUbCy5ngAAECF5goaAWCIzvKPWy8flvkoTvL893a0YmbJ8QAAgIrN\nFzQCwBkd5fcBx60k4Fgu8fdOJ89r97ZnnHc8AACggnMGjQBwRkf5x62XD0r8nbPRVsvh33OOBQAA\nVHTOoBEABnSS32+9fJiEHPMl/c5OtNXywSSK5wIAACOM4TUCwIBO8sKFjSTk2C7p9y1GWzzu2GoZ\nAABqMG/QCAB9OsjvV5AclbX18vE/q9GOQJuOAQAA1GTuoBGgkaFAKMC6EIqvhm2DE2HHmd0+tqOv\nW06+t/UrJqJdejZK+F3rUbBy1XkMAAA1mjtoBGhECNBJiq7uRI+U5OEo+ZnhZ3da1qaLZWy9nARh\nm1F7rzinAQCgZvMHjQC1nvxvRcVWy3CY/M7lFrTvQfKeVwv8HdNJeNXbannRuQ0AADWcP2gEqNWE\nfy4psFpmoNLPnWTFxUID27nwrZeTrZYPotCq4xwHAICaziE0AtRisr+Q1EYZOvx48onHu7/+5c+7\nLz//dPedt17v3th+b6AP37128rWXlp86+d6pixdHCVpuN+VxluQxnV54tVjQ7+gkbRZ+x60QtDjP\nAQCgxvMIjQCVnugPFaqEIOQXP/tp981XX+zevPFR9+//zV/m4qubuyfBzG9/86vuzIMPtCJkKXrr\n5eSY3kl+Rzi20851AACo+dxNI0AlJ/jh8Z/9QUHG9P33nawyCStO8gpTzvLF3ifdF565PEzQcquO\njwslj+r0tl6eK+DnL0cFh7fsyAQAAA2Zw2kEqNTkfirZkrfvjj/hUZ8//H6j+93Xn5UWqmT5+P23\nT1a0nBGyhABhpkbtX9jWy1Edl1K2dgYAAEqcS2gEqMzEfiFZ8dE3VMnzkZ88Hx26fOl3g2q0hEdg\n1mrQ/vNRcdmZnH/29ag9Kt8WAADAiGN+jQCVmNiv9QtVQi2VKoYqad98vtd99vLKoJCl0o/BRDv3\nrOX4M6ei1TBhNdKS8x0AABo4p9MIMNEJ/VQSOtwTRoS6JmXWU8mzLsv8Y51+Act+FR8TSmqhdJOA\nZSqnnzkd1c0Jq3fmnfMAANDQuZ1GgIlN6Gei1RKnhKKxk66pMq6wy1Aoupvx/g6rFDSktl5eyuln\nzkaPeN0uojguAABQofmdRoCJTOjnown9qR2Abmy/V+tQJV2PpfPoI1kBS3hEZrkix+Jq8pp2cvp5\nnWir5YM6FfQFAADOOQ/QCFD6ZH41azeguZ/8+CSMaEqw0hNW4IRivH0eE9qY8LGYiY5FJ4eftxj9\nvB1bLQMAQEvmeRoBSp3Mb/TbCejbLz9tXLASe/n5p/sFLBMLIY7/2cwr5Dn+ZyV6T5vOdwAAaNFc\nTyNAaRP51axwIYQOTQ5VYn/4/Ua/3YSuT+B4dKJHlGbG/Fnr0Xu56nwHAICWzfc0ApQykZ9PPwoU\nQoY67gY0rrCt9OzDD2UFLKslH5P9cbdeTorhbkYhzYrzHQAAWjjn0whQ+CR+Nl28NgQrIWRoW7DS\n883ne1kBSwgnFko6JkvRTj5T5/wZ08kjTb2tlhed7wAAIFwB8p/ET0UrJO5q44qVtC/2PsnaqjmE\nFLMlHJPb42y9nNpG+zCPYrgAAIBwBciehG+lg5XXXnqu9cFKz8fvv531eNCtIgvchseAkt+zf87v\n70ThzK2iwyAAAEC4Am0OVtaydgUSqpwWwqaMgGW7oGMy1tbL4bGlZHVN+P7d8GiQcx0AANAIUMwk\nfjEdGHQefaT73defCVQy/PY3v8oKWK4WcFw2zrtV8vE/y1EwszWp7aMBAADhCrQhWJlKF7ANtUW+\nurkrSOkjhE7zj3WyApZOjsfl3Fsvp1YhbTjPAQAA4QoUG65spHcGurH9nhBliB2EZh58IB2u7Od4\nXHbOsyLm+J/r0etZc44DAADCFSg2WOlEj46ceOGZy8KTIYVdlDJWr6zkcFyWop19pob8nrACaTta\n7bLkHAcAAIQrUHy4cmrb5dmHH1JnZUSh6G8qXDkcp75JEpL0tk1eHvJ7pqNjGQrYzju/AQAA4QoU\nH6wspVddhJUYApPRhNo04VGqVFuujXFcevVSDob8+tlki+VusuXynPMbAAAQrkA54cqBbZfz8fLz\nT+eyeiXZerlXXHh+iK/vRFstH4xa+BYAABCuAOcPVu5ZtXLzxkeCkjF2D8oobrt2juPSKy68PcTX\nLkb1cnZstQwAAAhXoNxwxaqVnL356otjrV4Jj/MkYcmZWy+HornR79l0TgMAAMIVKDdYWbBqpbTV\nK6sjHJfeTj8bZ3zdevTzrzqnAQAA4QqUH65ct2qlGK+99Fw6XNkd8pgsnrXaJdlFaDPaannF+QwA\nAAhXoPxgZSoqgGqHoAJ2DkqvCgq7+QxxXA4GrXRJtlreibZaXnQ+AwAAwhWYTLiyHE/8w2Ms4XEW\nwUh+nnzi8XS4cvWMY7I6aOvlZAehg2hlS8e5DAAACFdgcuHKdjzxf/byikAkZ9feeCUdrtwecDym\noq2XFzP+Pmy1fDv5+1vDrIIBAAAQrkCx4cqpR4L2dj4QiOTs2y8/7U5dvDjUo0GDtl5OCg/3jtdu\neDTIOQwAAAhXYLLBSiee8E/ff58wpLxHg1YyjsdMtPXyXOrvlpM/D9+7NcqWzgAAAMIVKC5cWYsn\n/JeWnxKEFOTl559OhyubGccjc+vl1HHacO4CAADCFahOuHKq3so7b70uCCnIje33BtZdOf5nPipQ\nOxP9ebxN9przFgAAEK5AtcKVU/VWwrbBgpBihB2YMuquxCHKQRygJIVte+FXeBxoyTkLAAAIV6Ba\nwcqMeivlmn+skw5XFpNj0dsO+yAJVaaP7Sd/FgKweecsAAAgXIHiQpJOsnPM1VGKnEaPoZwIE38B\nSLEuX/pdOlxZTW29vBR2EUq2WO4mWy7POc8BAADhChQbrizGdTyGfXwk7FYTT/TDxF8AUqzXXnou\nHa5sJKFY+O+dJCi7E61imXGOAwAAwhUoJ2CZj2p29Cbqc2d8z3o80Q8TfwFIsT5891o6XLkRba/8\nj6P/3rHVMgAAIFyByYQsq6kitSFAme7ztZvxRD9M/AUgxfpi75N0uPLXyb//2aAtmgEAAIQrUG7A\nMp08btKNtvZdyfi6U9swh62CBSDF+ubzvXS4Evxd9N9XncMAAIBwBaoTsswlhW57E/f9eNeZ1N91\n93Y+EICUICNc6W21vOK8BQAAhCtQzZBlKSl0e/exk2Qb5lPhylc3d4UfkwlX/qq3JTMAAIBwpZ4T\n761kkk11bCU7yORpPVWLJfz3fyNcKd/sww+lw5WOvggAABCu1Dtcud3nMQVaRrgymXBFPwQAAAhX\n6h+uLFA5ywWsXNlMhSkhVPuXwpXyTd9/n3AFAAAQrkCNwrPp5JGgo+hxoCvHptI1V27e+Ej4UX7N\nlTvOUwAAQLgC1Q1WVpJtmHsT+VDLZSb6+11bMZfru68/S4crt52rAACAcAWqF6rMJ9suZ27BHH3d\nqUeFPn7/bQFIwcKjV8IVAABAuALVDVVmktUpvYl7WLWyPODrr8cT/TdffVEAUrC9nQ/S4cqucxcA\nABCuQDWClStRXZWjpM7K9BnfsxZP9J+9vCIAKdg7b72eDleuO38BAADhCkw+WFmOJuvbx2aH/L7F\neKL/i5/9VABSsBeeuZwOV644hwEAAOEKTD5cmU3qpyyO+H1z8UR/7ic/FoAU7Ne//Hk6XFlyDgMA\nAMIVqPPF88dHibpTFy92v/3yUyFIgWYffigdrnSchwAAgHAF6h2uHMST/Q/fvSYEKW+noBBsTTkP\nAQAA4QrUO1zZUNR2YsVst52DAACAcAXqH66cKmobHlsRhBTj0vJT6XBlzTkIAAAIV6D+4cpUXHcl\nCI+vCEPyN/PgA+qtAAAAwhVoaMCyG0/6X3vpOWFIzm5sv5cOVg6dewAAgHAFmhOurNmSufRHgq47\n9wAAAOEKNCdcmUk/GvTF3idCkZx89/VnJ9tcp8KVBeceAAAgXIFmBSzb8eT/5eefFowUt0vQbecc\nAAAgXIHmhStLcQAQiq+GFRfCkfF1Hn0kHa6sO+cAAADhCjQvXAm7Bh3GIcCbr74oHBnTh+9eSwcr\nwZxzDgAAEK5AMwOWNatXCl+1su1cAwAAhCvQ3HDlntUrtmXOfdVKx7kGAAAIV6DZAcup1Sthl5tv\nPt8Tlli1AgAACFeAIcOVsHrlThwK/PqXPxeWjCjUq0kFK0dWrQAAAMIVaE/Aspp+nOXj998Wmgwp\nrPQJK35Sbbjh3AIAACo9F9QIkHvAsh+HA7MPP9T99stPhSdDCCt9UsFKqGMz7bwCAAAqPQ/UCJB7\nuNJJHmW5GxI8+cTjdg86wwvPXM4qYrvqnAIAACo/D9QIUMCFdeHCejooePbyihCljz/8fiMrWNl3\nLgEAALWYA2oEKODC+r647W46MLj2xivClJS9nQ+y6qyEx4FmnUsAAEAt5oAaAQq6uC5cmD52O709\n843t94Qqia9u7nZnHnwga3egeecQAABQm/mfRoACL7ALF+bS9Vem77/vJFRoe7ASatB0Hn0k63Gg\nZecOAABQq7mfRoCCL7ILF5bSAUIIFdpe4DZjZ6Bg3TkDAADUbt6nEaCEC+3ChSvpIGH+sU73m8/3\nWrli5be/+VVWsLLtXAEAAGo559MIUNLFduHCVjpQCPVGbt74qDXBSgiT+jwKdBCKADtPAACAWs73\nNAKUdLF9v4PQQTpYCEVuw1bETQ9WQoiUUby2mxT9nXWOAAAAtZ3vaQQo8YL7fgeh7YyAofvs5ZXG\nBishPMrYbrm3YmXWuQEAANR6rqcRYAIX3oULG1kBSyjy+u2XnzYqWHnhmctZoUo3CZmmnQ8AAEDt\n53gaASZ08V24sJzepjmY+8mPG1GHJdRX+cXPftovWLErEAAA0Jz5nUaACV6AFy50jh32e0yorqtY\nXnvpue70/fdlhSohTFp27AEAgEbN7TQCTPgivHBh5th+VsASCsBee+OV2oQqN7bfO1l502e1SgiR\n5h1zAACgcfM6jQAVuBC/30loq08o0Z19+KHuO2+9XulQ5cknHu8XqnST8GjWsQYAABo5p9MIUKEL\n8sKFxWO3BoUsYSVLVR4X+vDda2eFKneOrTm2AABAo+dyGgEqdlF+v4rlSlax2/TOQmGL4+++/qz0\nVSqXL/2uX02V2GZ45MkxBQAAGj+P0whQ0Yvz+1osG2eFLFMXL54ELWFFyxd7nxSy608IcS4tP3VS\nA+aMQKW3xXLHMQQAAFozf9MIUPGLdMiQJX50KIQtLz//9EkoElaaDBuk7O180P34/bdPvjesTuk8\n+sgwYUq8UqXjmAEAAK2bt2kEqMnF+n3IcvXY7RECj8yVLqFOShD+e5yflewAtK5YLQAA0Or5mkaA\nGl64Fy50jl1Pwo1uyY6SnY0WHQsAAADhCtT/Iv4+aFlLap0cFRSo7CcrVBa0OQAAQGpephGgYRf1\nhQvzx5aTR4jCCpPdEUKXEKLsJN+7EsKUsHuRdgUAABgwD9MIAAAAAOenEQAAAADGoBEAAAAAxqAR\nAAAAAMagEQAAAADGoBEAAAAAxqARAAAAAMagEQAAAADGoBEAAAAAxqARAAAAAMagEQAAAADGoBEA\nAAAAxqARAAAAAMagEQAAAADGoBEAAAAAxqARAAAAAMagEQAAAADGoBEAAAAAxqARAAAAAMagEQAA\nAADGoBEAAAAAxqARAAAAAMagEQAAAADGoBEAAAAAxqARAAAAAMagEQAAAADGoBEAAAAAxqARAAAA\nAMagEQAAAADGoBEAAAAAxqARAAAAAMagEQAAAADGoBEAAAAAxqARAAAAAMagEQAAAADGoBEAAAAA\nxqARAAAAAMagEQAAAADGoBEAAAAAxqARAAAAAMagEQAAAADGoBEAAAAAxqARAAAAAMagEQAAAADG\noBEAAAAAxqARAAAAAMagEQAAAADGoBEAAAAAxqARAAAAAMagEQAAAADGoBEAAAAAxqARAAAAAMag\nEQAAAADGoBEAAAAAxqARAAAAAMagEQAAAADGoBEAAAAAxqARAAAAAMagEQAAAADGoBEAAAAAxqAR\nAAAAAMagEaAEF3/wg+6FCxdO/PkPf9Rt2+v64dTU3d8T/ts5AcRC/1NmH6FPgvpqyvVrDAbNoxGg\njx9N/bB78QcXu3/2p39690MpCB+GP7w41f1HP/rzoT+ghCs+2GHkD+io3ylSFd6rcAVo2/VrDAYN\nHLtpBLh3kJ8OVPqZunhxqA+p836AxhOOUYXf6YMdhCt5hivj/J5BfZJwBaja9WsMBghXYAxhtcqo\nH6A/+LM/K+wDtMof7PH3FmWY9wDClbqEKxcLDVf0SVBfVbx+jcH0dyBcgXNKf4iG1SshbPnzH53+\nwAtfF1asjBKwCFd8sENZE5Mir5txrudBoYlwBfRhwhX9HQhXoCHiR4GGedwnhC5/8id/MtSEoKyl\nn/FAwAc7tEM67B3lkcWixcvRB/UxwhUQrtT9+jUGAzQCpD4Qh3nMJ+v7QjhTxXBl2DsvRb6uOIQK\n/+2cg7wmJBdPXVtxSBz+bpTC20UI/ekwjx+VXXNFnwT1FPq09PU76X7OGAzo0QhwLOz+k8fdg/Qj\nRJlf06JwJQx44slVHGBVZTAEteyzpqbumWCE/ie9oi7896QKGIZrfNiVNGWFK/okqHew0u/67Tf+\nanO4or8D4QpMRBx+nGeSc9aHYxvDlfTkL0yu4scXJjnpgzoKk4dwDcXXVdZAOWtAHb4nfG+ZW8GP\nElqXEa7ok6C+Qg28+PoNfVzVrl9jMEAjQIXDlbptA9hv8hfftY7b6+4H/PEkrAp3naAqQkASrskw\noci6pnrXTvj7YQfX8feFR4bC34ffUcQdzHjVyqBHJosOV/RJUO9+MPRx8eOO6Z3Hirx+jcEA4QoI\nV0r5YI8nf2GgkzWJCwOirJ8bPsSzirGFrw+DgJNdmgqa9EHVZRWovWcwPGQAEa6hs35eEQVwR+1b\n8ghX9ElQbyePNh5fZ2Gyn/U4S9/r94fFXL/GYPo7EK7AOcTL1wfdCc4SDwDyrrlSxQ/2YSZqvW2s\nh3l/w1S8r8rOJ1CWfgPlUfunWNYd4J48B9Fx4BzfYS4qXNEnQY3HX6mVFONcv+FrskKZe7aFvzg1\ncr9kDOZcBeEKnCPEqNJuQaNu6zfO4GDY15Wu1J9eqnuepa39lrIWMfGDukw4Qp8SrqmTu4g5Ltnu\nLbUPP7t3pzKvnx1+7nl2pTirnxrU/+mToN5yv35/mM/1awymvwPhCpxTegvTYT4w4w+iQXcJmhSu\n9CZ+39dt+MH3z+rmWIjtZEnw8c8PP7s3uXR+QvXFwcqglXx5hyv6JKi3Mq/fUcJkYzD9HQhX4Lwf\nKD/6UeayyvQEoXdHJL1bx6Cf3bRwBWBQsDLq40vjhisAuY8LjcGAEWkEGDBBGMYwjxFNMlwp4nUB\n+Rum9kDRzlNMNv26z1MXpoytmAHO2y8ZgwHD0AiQ/jD90Y/6Fn08b1G0MsOV8D3xss4iHlcChCvp\nPuSsraGH7euEK6C/q0J/ZwwGCFcgJ3HBx/Ty9BCqjFLgaxK7BQ0zeBjlddX1rjqYbOR/jYX+Ma45\nFf57nKK75wlX9EmgvysrXDEG09+BcAVqLo8P9kHFyHywQ/MmKkVeJ1nbOeexXF64AsKVJoYrxmAg\nXAFqGK6E2i/hg7q3siZ8CJ71Ye2DHYQrw/78rMcl8/pdwhWgKv2dMZj+DoQrkLP4Qy/PgUCVnqv1\nvC+YbAwjvUNa6DtGeTSyiHAF0N/Vub8wBoPm0QgwxIdengMBH6BAHScbYXl7CFmK6MOEK0CV+juA\n89AI0HciIVw5j3A3O9RlCJOw0IZx0cu4+GX4u/A14WvzvAMOTRcKx/YKbsf91L3P+v/g5GvC145T\nbLaU91RguKJPgpr0bTnUOOmnLTe29HcgXAHhyoQHHXkMPML3D5ronV307QdW9cAZ11h4rv+811j4\n3qpeY0WEK/okEK7kOcYxBgOEKyBcKfyDPRRvy61g2kVLfCEtXfNkHOFnVS0QyTtc0SeBcKUt4Yr+\nDoQrUKtwpQofoFX9YE9/qIflpr2lpoN+Zvi73tLV9LJVH+5Q/WusquGKPgk46QtyfCTbGAwQrkCD\nwpWi3+N5Xld4TjevbfrS2wx6BhiqfY1VMVzRJwFFhCvGYMAwNAIM8aEnXMkW7nrk+ahB/OhD+NnO\nQ9quytdYFXf40SdBfbWtoK0xGAhXoJXhyrg/q6p3T8b9YM97O0TbK0J9rrEqhiv6JBCutCVc0d+B\ncAWEKw1duRK2fB3/9bhrAnW5xqq+ckWfBMKVtqxc0d+BcAWEKzX/YL/ned8xiqCld0PxvC9U+xqr\nYriiT4JmhCttWDlhDAbNoxFgiA+9cX9WU8OV9HvLrVK95ahQ+WusqhMhfRIIV+o2zjQGA+EKtOZD\nT0HbMz7cU1sBjsMWgFCPa6zIJfx5b8esT4J6hStFaGK4or8D4QoIVxr2wd4bFI3TZuF7q/hMNFRp\n4lGla6zK4Yo+CYQrbQlX9HcgXAHhSsM+2HvCc7q9pabh56eXm/aWrYa/6y1d9Wwv1O8aq3q4ok8C\n4UpbwhX9HQhXgAZ+sAMAYAwGbaMRAAAAAMagEQAAAADGoBEAAAAAxqARAAAAAMagEQAAAADGoBEA\nAAAAxqARAAAAAMagEQAAAADGoBEAAAAAxvD/Bw8JFhMv71cyAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<IPython.core.display.Image object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "from IPython.display import Image, display\n",
    "display(Image(filename='neurons.png', embed=True))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "위의 그림은 3층으로 구성되지만 가중치를 갖는 층은 2개뿐이기 때문에 '2층 신경망'임.\n",
    "\n",
    "출처: http://happycontrol.tistory.com/entry/인공신경망1"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 3.1.2 퍼셉트론 복습\n",
    "x1, x2 두 신호를 입력받아 y를 출력하는 퍼셉트론\n",
    "\\begin{equation*}\n",
    "y = 0 (b + w_1 x_1 + w_2 x_2 <= 0)\n",
    "\\end{equation*}\n",
    "\\begin{equation*}\n",
    "y = 1 (b + w_1 x_1 + w_2 x_2 > 0)\n",
    "\\end{equation*}\n",
    "\n",
    "편향: 뉴런이 얼마나 쉽게 활성화되느냐를 제어\n",
    "\n",
    "가중치: 신호의 영향력을 제어\n",
    "\n",
    "편향을 명시한 퍼셉트론(b = -1.5, w1 = 1, w2 = 1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAbIAAADJCAIAAADJgyD9AAAAGXRFWHRTb2Z0d2FyZQBBZG9iZSBJ\nbWFnZVJlYWR5ccllPAAAFepJREFUeNrsnUFMGukbh/WfbsLe3JN6KmQP1aSJ9KScdsgeCulhbbJJ\n9YRNTdTsoTXRqCf1pEYT18MGTTTqSUw2kR4aMNld2MtiT7hNE/GwgZ6kp3qr2Wzq/4XPThEQEQcY\n4HkOzYggUwYefu/3ffNO8/n5eRMAAHzmf7wEAABoEQAALQIAoEUAALQIAIAWAQDQIgAAWgQAQIsA\nAGgRAAAtAgCgRQAAtAgAgBYBABpSi4kNd3PzeJgXEgAaXItpG6awDQZ5EQGgwbUYHr+w4VjoPDTG\nSwgA9cWdkh4lQlzU0op8Vf5dTCaTwWDw+Pj44OBAfuzo6Ojq6tI0TTY4fgBgBi1qi+daxfZvfn5+\ndnb27OzsS1gNh9XG5OTk9PS0xWLhKAJAdYvoCpFIJBwOx9TUVKYTs4z54MGDw8NDjiIANIQW+/v7\nVdVcgFgs5na7T09POZAAUOdalCR4rRMVyWRyZGSEAwkA9axFKZ9nZ2eLv7/P5/P7/Y1wtIpfJaqv\noMrGvZHgXQ9Qc1oUx101nngV+/v79W5DVokCNLAWj4+Pb/qQIivuGuRWq0TlMdkEnll51wPUnBZL\ncNzbt2+l9K7TY6TkphaKAkBDavGmFbTw33//ORwOt9vt8/nq6wBpiwgRAC2WcPqKPOTk5MTj8Wxv\nb7e3t09NTdVveASAxtNid3d3aSbt6+sLBAKRSES2JTw6nc6tra0SsmfdsORkChqgElrMXPvhXMr6\n9Bnx4bPb7Td9yMOHD/Vtq9U6Nzcn4XFoaGh3d1fC48jISCwWa/AjHRy04UaAWk2LLpdLct+NouLA\nwEDu7So8RqPR1tZWt9st+bFxwqP1WeDSBHR83aXcOIIYAa7j3JRI1mtraytm/y0Wi4ivmL8pihRR\ntrS0DA8PHx0dmeW/mm/VTZ6FNV/uesUvr+dCjSU/HqBBMOnJf+LEzc1NUdi19/R6vUUW3RJCd3Z2\nRIh3795V4XF1dbWRzqe2fv9jyotv/yEvAtRaEa1bTBTW09NToHaORCJ5y+fCwp2cnIzH49PT03/+\n+Wd7e/vTp0+ruRpcW8z9sirTkhzrt/d5xwPUsBaVwkR8Eht7e3utVqt+o6Zpy8vLUjsXkGaR4VH8\neO/evf7+/s7OzroPj4l/3vKOB7iWZskmvApN6VNr1tbW/H6/KHhoaOg2wi0b4fHU1L/eGr20h7vW\n45wACFCzabGSiAcllkp47O7ulrJahcdkMlkbe586czqzt478fLnRTmpVVXo11dgETgS4Bmad8qJG\nLS0Wi1rlU70duZg+vmIgYD1+aTZbn2S+oqnE5/sDQO3NRJskPJ6cnHz33Xejo6M2m21+ft604VF7\npJbuPPpcXGuLOWJMKZPqGYCxRaM4PDxcW1vz+Xwul8vj8ci/vCYAaBFSrX3EjOJHiY1DQ0NSZRe5\n5hwAagiK6BtgsVhEhZFIJBAIvHv3rrOzsx5bmQGQFkmLtw6PiURCdCn5UV9cCQCkxYYOj6FQ6PT0\ntE774AKQFuEWiBO3t7cPDw8JjwCkRUhBH1wA0iJcHx4PDg5El8+fPy/hYgwAQFqsw/AYjUZbWlok\nOTZUH1wA0iJcTzAYlPAo/xIeAdAifCGZTEpmXFtba2tr83g8qmE4LwsAWoSL8Oj3+8WMZm1lBoAW\noXrh0WKxSGVNeARAi/AlPO7u7pq7Dy4AWoSKc3p66vP5VlZWZFvCoyiSbhQAaBFSqIsoiCLFjLQy\nA0CLkB0ez87OaGUGUBlYzm1qWlpahoeHj46OdnZ2jo+PbTZbf39/MBjklQEgLUIK+uACkBbhEvTB\nBSAtQlHhkT64AKRFuBQeVR/cBw8eEB4BSItwKTz6/X764AKQFuFLeKQPLgBpEQpBH1wA0iJcgj64\nAKRFuCY87u7uhsNhwiMAWoQv0AcXAC1CfuiDC4AWoVB4pA8uAFoEwiMAWoTroA8uAFqE/NAHF9Ai\nQKHwSB9cQIsAVQuPiUQiFovJM8r2u3fv5Ed1e0tLS1dXl2x0pLHb7RwXQIv1T2LDbRu8Hzpf1Ewc\nHre3t4vpg+t2u+U+otEi/7ioUP74y5cvDw8Pi7m/1WqVP/7kyRNmh8BgzsEExNczw9dYyPQ7fHR0\nNDw8LPFNYuPOzk7e+0QiEZFmPB4v/Kc+fvw4PT19m1Nu5FlevHhxcnLCGwkMgbRYdcLjzc6lCxs+\nepXaHjNtWszi2j64W1tbCwsL0WjUYrHk/Qurq6uzs7OSPTNvlDtLDLx3755sSxLMfGw4HJZ/379/\n7/f7cx81MTEhfmTdJZAWa53Q2Jd4KNu1kRaLD4+iS3Fc7kMCgUBWQpSHy5339vaKfeFCIZFglojl\njywvL/OugtuAFs2myJrUol4Ob25uSr6TqnZyclKVz3Kj3JKlKtFZrsvknqU9r5g0S46apn348IH3\nE6BFtGiaodJ4XLQochQ9iSglS4r4JNzJr8RWmbPYUvlOT08bojCv15s5/yNRVJ6XtxSgRbRoLqSg\nFgmKE0VS4qzffvsts3CW4trYeRLJm6LjzBCqXAyAFtGiiYhEIlIgq2mTO3fu6M6SOrp8LtZnaWQj\nEAjwxoIbcYdJJzB2bvrnn39+//794eFhIo3dbpfU9vz5c7XaUalKKuu+vr4y7YP8ZavV+vjxY3k6\n2Z/+/n5RMz13oXjQIhi83ltM1NraOj09LVWzLiO32607UeKbppV3AVJPT4+o0Ol0ipdll+TZ1ZUb\nOECAFqHSiApnZmaybhwdHQ0Gg2pbcmK5naiQwLi3t+dwOETTIkcJj6FQiAMExcAlrqC8+P1+KavV\n9uTkZPlq51ykfhcLq+1wODw1NcXhALQIVUaSmkRFtd3b2zs3N1fhHRAL63PTYuesE2MA0CJUmtXV\nVdX/xmKxeL3equyDuFj12hFHz87OclAALUI1o+LCwoLaVjMw1doTeXZd00W25wG0WF4SG+7m5vEw\nL3aBl+cC1TNiyanf4N5I1Ox/bGpqShWtqsNNFfdE6ne9+RiBEaqpRf3jbhsM8kI3IH6/X21MTExc\n1UGnYuglfG7rHYDKaDE8fmHDsdDFmRtwFdZnBc7CCDyz1ub/6uDgQI0qSlQcHh6u+v7Y7Xb9XGzd\n1wB5Kd+6Rb1pYPjV7f6QfLcHg8Hj42PVxb6jo6Orq0vTNM5bMDMvX77UC9iqR0XFkydP1PJJ2Tcz\nmBrMi8nP852bm7vqQzU5OVlyKyooN/qXlnmaNZycnKhdkncUbcegADcvolP1cXPuFMrFSKJxMytS\ngjkcjqmpqbOzs7x3mJ+ff/DgAROLJiSWpindw6Yy57QUg2p01pSeIqeOBkPHFrXF9IVHlhYy50gT\nGyODwSbXety4Zvv9/f2qai788XO73aenpxzIag0g5n3x9QNX/PWtKsMPP/ygNl6/fs3hKx/JZLKm\n80opUy7WZ14RY3BwRBdj+Je0FL2GzQ9IErzWifoBGBkZ4Y1YFXZ3dzs7O30+X27SVxvqeizmQS/t\nmYwuK6o9hySbGn2dS5uJ/izGX8KqrHYuGSlF+VDdaHGZfCypiarC8vLy3t7eysqKw+HITAfv37/X\n61ZT7bC+P1QY5f76icfj8qUo35oSca4aB6szLX4W45JzPGy0FNPrJ276Ou7v7/NerAqqhZfH45F0\nILFdpQM9I5hWi3qehTJhsVhmZmai0ejr169FjkYFFzW1kf8sh/TvjDkB4hbTNfrFjV3rcSNnoktY\nPGG325k+qy4fPnxQFyOVCKmfUiKfCtP1o//8oeWQVQx1lUeXy2XA1XWUdfIo58pfVLg7t/Xb+01N\nqXVg97+1Gvk9U+SoYiZv3rxxOp18RVcd0eLo6OhXX32l/2i2PZTAqLp284ap8MseTDM5OXmrRkrW\n7390NQWDv/6eeHapQE38/mvq9JEJQ6rW0rWY2HCnrvS+vv52cNA5/sjAC76XMBLx6dMnvR0AVIvT\n09OFhQX59+uvv1aLBE04qKSPKvKGqSThcFjijpQRT548ud1fsj6bGBsMLmV58cKKjwyy0K2ibLoy\nztg0poguYVWHRHRKleqyubkpiaCvr0+EqJ9mZ7YL70mlf/HZslo5ZBVbRS+faHlv7OzsGHmGyKVy\nOfeWCi/nVkkxtUxxLJROiCl7N6VnXwz6Yunu7i5Bi3whVwtJAQ6HY21tbW9vT976bWnUr8y2PkPf\nH67rUgGkVpiZmens7Ozq6orH48Y1ZtceiXE+L4RJZ9FXS2LFH783bDDPmLP51E1fbK3PxuTjOqkH\nAoGb/i+8Xi9fy1UJAgMDAyJBiYqZt+v16fLysql2WL+ci+RZDl9Zke9IeWMYfinw3GrV+KxYQlpU\nQ4qyDz9lVvHaYijtb5sRkVHesjf6YpGoKB9Ovpwrz+joqLz1j46Osl5/PYvpCxhNgjolscl8K4fq\nr4BYWFiQ0kHJ0fgnSE+8NC29CutZ0aDJltuNLZY/hhT5alosFhOuAmFBhjnHfPWvW7PFWCitaJW8\neN3cRkmruMz80SpmACirfAMz8PHjR/3YGbBUrQx7FY/HOUx14cX1lBWNLKBLn3KpBGrxp74wOG/t\nHIlEKJ9NiER4fTlB7hnTVVwjolbnyDvHarVymGoc7afUiXaDqXYMBk62pDH1Ja6kjhbxSR6Uz5j+\nPlbtoaQIktq5gDShuui9asxzXubu7q7auPXSOTAFaoCxyXgrNjWffz4dCsBAzs7OvvnmG7WcW77A\n1CVJq0gymbTZbGp/pAphRVc9kNhw2waD+nUAGiQtQn3U0U+fPq36/iwsLCgniqBxYn2Q7mdo3Jkt\naBEqwMTEhNo4PDysbue3RCKxurqqtjnnr26i4kLuSkG0CCZHcpneDKm6V2ceHR3Vo6LZGoZDiVIc\nGQwavVpRh7FFKCOZI3per7cq19sLh8N6s5y9vT20WOOV83hz6nySNGUYVSQtQtnJvEi0RDYxVOXL\n58ePH6vt3jQclDqhbE4kLULZkaioX9KgpaUlGo1WbM3g6empPLV+DcJKPjXUNKRFKC8Wi0U/MVZd\n+ahi11Hp7+9XTlT7gBOBtAhVQIrWp0+fioa6u7tFhR0dHXa7XZLawcGB0+nU5z3KLSkxr+yGPv1d\nrWFNQItghFZSC1Tvl3HUpPyIAdVlgt+9eyeWlB/Fhpqm/fvvv3/99Ze6j4hSzKguZl8ONT9+/Fi/\nEuGLFy+Wl5d5a8EN4IxzM3C5P6Wx3UCq36BBrCT5Uf5jDodDbajCthxdMiORSGaHEXEi7y6on1YR\nDUPqMo629Bqs0EU73zohFouNjIy0t7fv7+9L4dzT0/PHH3+EQiGlLYmQ8luprEu4olleJKJK4Szm\nVWOXSrvkRCgBtGgGVDys4cI5a1xva2tL9OR2u1tbW6PR6MTEhChyZ2dHVCVyzDxFOhwOyz2l5r3N\nhZvlGUdHR202mzyvXqQHAgHGE4Eiul5ayNVsEa36vIn7+vr6xEoX4wPxeFtbW9a1rqSyznWWPHZv\nb09+dYPXKxSSv5PVl1PTNNopQn22oUWLtcKHDx+kXFVNDOfm5jKv3SGOk3goN+Z9oMRG/RqBOsqq\nEi1Febl2k+eS28We6hoyWY9Vc9y8jQAtokVzxcNMXrx40dvbe23iu7btmLhVn6vJixiZPu1gFHcY\nRoASxvJ8Pt/KyopsezweVSbn3m1raysYDEokLPzXpOaV+8RiMb/fv7u7qy+syVr0c5UNRbuyD1Xv\n5wj1BFqEGyCa297eln9FRpLOCndH//vvv6WkLZzydKQGn0yTSCTEj/v7+2dnZ+LfLEvK3ZSCHz58\nKPtA50QoByznNhWqPciY2ZZzJ5NJiX5ra2viuOfPn0vJzOXngbQIDR0PJb6paRAungNoERqUrHjo\n9XqJh4AWoUFR8x4SEtXkMoN3gBahQUkkEpINJSFardahoaHNzc0ip0oA0CLUGz6fb3t7++DgQOJh\nKBQiHgIwE139oJa+2O0Vv3StxwNluYhPVjwUJxIPAUiLDR0PDw8PBwYGiIcApMXGJRaLraysiBN7\neno8Ho/EQ14TANJiI3J2diYqlHpZqmaJh1zmCYC0SDwkHgKQFomH6XiYTCbFhsRDANJi43JwcLC9\nvS1OdLlcIsTcVoYAQFpsCPQ2X5ITh4aGjo6O8rb5AgDSYkPEQymWxYmq5yDxEIC02OjxsKlgF1gA\nQIv1TzgcVm2+iukCCwAU0XULXWABSItwAV1gAUiLQDwEIC1CTjykCywAabHRUW2+fD5fW1sbbb4A\nSIsNTWYXWOIhAFps9Hiod4Et/nrKAIAW6zMe0gUWAC02OrFYTGwo8dBut3s8HqmXeU0A0GIjktUF\nNhKJ0OYLwOQwE13GeEgXWADSItAFFoC0CDnxkC6wAKTFhoYusACkRbiALrAApEW4FA+b6AILgBaJ\nhxIP6QILQBHd6CSTSTW5LNu0+QIgLTY0mV1giYcApMWGjod0gQUgLcKXeEgXWADSYqOTSCTU6CFd\nYAGgBtKilLQS346Pjw8ODuRHSXBdXV2aphkS5egCCwA1lhbn5+dnZ2fPzs5yfzU5OTk9PV1arMvq\nAks8BIAa0KKYq7+/XyXEq5Bwt7OzY7fbbxoPVRdYj8dDPASAmtGiw+Eo7ERFW1vb0dHRtTPFejxU\nXWBp8wUAV/E/09bOxTixKT3yODIyctVvpfoWFTqdTpGs/BiJRAKBAE4EgEKcl4v4+uXOCWOhoh8Z\nj990pG9vby/rj0iEHB4elhTpcrmk0D4HACiO8qTF8Hhzs20weOm2JWdz83i4mEf7/f68cywF2N/f\nz4yHkg3dbndra2s0GiUeAkDVi+jwuHOpqcm1Hs/Qb2hMqbEYMR4fH9/0KaXijsViUk23t7eLIqen\npyVyzszM0BkbAMwxtpgqmAPPMo2kLSozLr263otFjipm8ubNG4mHd+/eldpZSmZaHwJAyZRjObe2\nuJjvZmuHuCpYxONvWkELnz59knjI4QSAiqfF1KBhc+4YYWLDne/mEilhLSHLDwGgSlrUFtMTzEsL\nG4lMKY4MBlNjiYtagYcmfv9VouLYI+3aJ+nu7kaLAFAjWpRS+JlXxBgcHNHFGP4lLUXvpbHE3Jyp\n7vXT9VZsutFZK4qHDx9yLAHAGEpfkqgWIqYnUi7POhd+QFHcaEmNRMWPHz+y2AoADKHE5dyfPVeM\nFNV9rzPnZU5OToq8bpTFYolGoxxIAKiyFjNOYinsu1KcqAgEAsX0xN7c3OQoAoAZznKxfntfbdz/\n1lpg3to2GBQnBgqPO+bF5XIdHR0VuHyK1M6RSGRgYICREACo9tiiXkSvFxgzVOe1lJITs5A82Nvb\nq5+vIsW1pmnLy8uMJwJAOSipsVhiwy0pUGy4qGVs5tyhKftmAIA6XKDzeZniWChtPOuzibGsc51x\nIgDUMjdOi+HxZufSZeepmy6GEC+keCXYEgDqKS2K81RznEursrXF1DhicNBm0Ml/AAC1lBYBAEiL\nAABoEQAALQIAAFoEAECLAABoEQAALQIAoEUAALQIAIAWAQDKy/8FGACZWuJiYje0ugAAAABJRU5E\nrkJggg==\n",
      "text/plain": [
       "<IPython.core.display.Image object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "display(Image(filename='perceptron.png', embed=True))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\\begin{equation*}\n",
    "y = h(b + w_1 x_1 + w_2 x_2)\n",
    "\\end{equation*}\n",
    "입력신호의 총합이 h(x)를 거쳐 그 변환된 값이 y의 출력이 됨\n",
    "\n",
    "\\begin{equation*}\n",
    "h(x) = 0 (x <=0)\n",
    "\\end{equation*}\n",
    "\\begin{equation*}\n",
    "h(x) = 1 (x > 0)\n",
    "\\end{equation*}\n",
    "\n",
    "h(x) 함수는 입력이 0을 넘으면 1을 돌려주고 그렇지 않으면 0을 돌려줌"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "### 3.1.3 활성화 함수의 등장\n",
    "활성화 함수 h(x): 입력 신호의 총합을 출력 신호로 변환하는 함수. 활성화를 일으키는 지 정하는 역할.\n",
    "\n",
    "\\begin{equation*}\n",
    "a = b + w_1 x_1 + w_2 x_2\n",
    "\\end{equation*}\n",
    "가중치가 달린 입력 신호와 편향의 총합을 계산\n",
    "\n",
    "\\begin{equation*}\n",
    "y = h(a)\n",
    "\\end{equation*}\n",
    "a를 h(a) 함수에 넣어 y를 출력하는 흐름"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# 그림 3-5!"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "단순 퍼셉트론: 단층 네트워크에서 계단함수를 활성화 함수로 사용한 모델\n",
    "다층 퍼셉트론: 신경망(여러 층으로 구성되고 시그모이드 함수 등의 활성화 함수를 사용하는 네트워크)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 3.2 활성화 함수\n",
    "계단 함수(step function): 임계값을 경계로 출력이 바뀜\n",
    "\n",
    "\"퍼셉트론에서는 활성화 함수로 계단 함수를 이용한다\" 라고 할 수 있음"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 3.2.1 시그모이드 함수\n",
    "시그모이드 식\n",
    "\\begin{equation*}\n",
    "h(x) = \\frac{1}{1 + exp(-x)} \n",
    "\\end{equation*}\n",
    "\n",
    "exp(-x)는 e<sup>-x</sup>, e는 자연상수로 2.7182...\n",
    "\n",
    "신경망에서는 활성화 함수로 시그모이드 함수를 이용하여 신호를 변환.\n",
    "\n",
    "변환된 신호를 다음 뉴런에 전달."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 3.2.2 계단 함수 구현하기\n",
    "계단 함수: 입력이 0을 넘으면 1을 출력, 그 외에는 0을 출력"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "def step_function(x):\n",
    "    if x > 0:\n",
    "        return 1\n",
    "    else:\n",
    "        return 0"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "위 구현에서 인수 x는 실수(부동소수점)만 받아들임.\n",
    "\n",
    "넘파이 배열을 인수로 넣을 수 없음."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "def step_function(x):\n",
    "    y = x > 0\n",
    "    return y.astype(np.int)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "위 구현은 넘파이의 트릭을 사용하여 구현. 넘파이 배열도 인수로 넣을 수 있음."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([-1.,  1.,  2.])"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import numpy as np\n",
    "x = np.array([-1.0, 1.0, 2.0])\n",
    "x"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([False,  True,  True], dtype=bool)"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "y = x > 0\n",
    "y"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "넘파이 배열에서 부등호 연산을 수행하면 원소 각각에 부등호 연산을 수행한 bool배열이 생성\n",
    "\n",
    "배열 x의 원소 각각이 0보다 크면 True, 0 이하면 False로 변환한 새로운 배열 y가 생성"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([0, 1, 1])"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "y = y.astype(np.int)\n",
    "y"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "astype() 메서드: 넘파이 배열의 자료형을 변환\n",
    "\n",
    "파이썬에서는 bool을 int로 변환하면 True는 1로, False는 0으로 변환"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 3.2.3 계단 함수의 그래프\n",
    "계단 함수: 입력이 0을 넘으면 1을 출력, 그 외에는 0을 출력"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXQAAAD7CAYAAAB68m/qAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAEK5JREFUeJzt3X+M5HV9x/Hn60AaWn/bVJJDsBYUi21PrYgh1UnR3GmR\n+6dtuNaI1pj+EDVVW6iasIR/xDZRW9oYUzzrr5xFG8UGLVqYWlORU6TF807Oml7v1NIAGtMmFW7v\n3T/myzGseztzMLOzny/PR3LZ+c5+ZuYz3NyT7372s7OpKiRJ7du06AlIkmbDoEtSTxh0SeoJgy5J\nPWHQJaknDLok9cSJ6/lgSdwjKUkPQVVl0ph1P0Ovqt7+ufzyyxc+B5+fz83n178/03LJRZJ6wqBL\nUk8Y9BkaDAaLnsJc9fn59fm5gc/vkSLHsz7zsB8sqfV8PEnqgyTURvymqCRpPgy6JPWEQZeknjDo\nktQTBl2SesKgS1JPGHRJ6gmDLkk9MTHoSa5JcmeSf1tjzJ8n2Z/ktiRbZjtFSdI0pjlD3wlsPdYn\nk7wU+LmqOhP4XeC9M5qbJOk4TAx6VX0R+P4aQ7YDH+zGfhl4XJInz2Z6kqRpzeIXXGwGDo4df6e7\n7s4Z3Le0ru69Fz7/eVheXvRMtNGccgo873mLnsXaZhH01d4w5pjvwLW0tHT08mAw8F3StKHccgu8\n4hVw3nmLnok2mnPPXb+gD4dDhsPhcd9uqndbTHI68Omq+sVVPvde4Kaq+lh3vA94UVX92Bm677ao\nje7GG+HKK+GmmxY9E+kBs363xbD6mTjAdcAruwc9F/jBajGXWrC8DCeu62/alWZn4ks3yUeBAfCk\nJP8JXA6cBFRVva+qrk/ysiTfAv4XePU8JyzN0+HDBl3tmvjSrarfmmLMJbOZjrRYBl0t8ydFpTEG\nXS0z6NIYg66WGXRpjEFXywy6NGZ5GU44YdGzkB4agy6N8QxdLTPo0hiDrpYZdGmMQVfLDLo0xqCr\nZQZdGmPQ1TKDLo1xl4taZtClMZ6hq2UGXRpj0NUygy6NMehqmUGXxhh0tcygS2MMulpm0KUxhw+7\ny0XtMujSGH8FnVpm0KUxLrmoZQZdGmPQ1TKDLo0x6GqZQZfGGHS1zKBLY9zlopYZdGmMu1zUMoMu\njXHJRS0z6NIYg66WGXRpjEFXywy6NMagq2UGXRrjLhe1bKqgJ9mWZF+SO5Jcusrnn5LkxiS3Jrkt\nyUtnP1Vp/tzlopZNDHqSTcDVwFbgbGBHkrNWDHs78LGqeg6wA/irWU9UWg8uuahl05yhnwPsr6oD\nVXUfsAvYvmLMEeCx3eXHA9+Z3RSl9WPQ1bJpXrqbgYNjx4cYRX7cFcANSd4A/CTw4tlMT1pfBl0t\nm+alm1WuqxXHO4CdVfWuJOcCH2a0PPNjlpaWjl4eDAYMBoOpJiqtB4OujWA4HDIcDo/7dqla2eYV\nA0aBXqqqbd3xZUBV1VVjY74ObK2q73TH/w48v6ruWnFfNenxpEXasgU+8IHRR2mjSEJVrXZy/SDT\nrKHvBs5IcnqSk4CLgOtWjDlAt8yS5JnAT6yMudSC5WW3LapdE4NeVcvAJcANwB5gV1XtTXJFkgu6\nYW8BXpvkNuAjwMXzmrA0Ty65qGUTl1xm+mAuuWiDO/NMuP760Udpo5jlkov0iOEZulpm0KUxBl0t\nM+jSGIOulhl0aYy7XNQygy6N8QxdLTPo0hiDrpYZdGmMQVfLDLo0xqCrZQZdGmPQ1TKDLnWOHIEq\n2OS/CjXKl67U8dfPqXUGXeq43KLWGXSpY9DVOoMudQy6WmfQpY5BV+sMutQ5fNj3cVHbDLrUcZeL\nWmfQpY5LLmqdQZc6Bl2tM+hSx6CrdQZd6hh0tc6gSx13uah1Bl3quMtFrTPoUsclF7XOoEsdg67W\nGXSpY9DVOoMudQy6WmfQpY67XNS6qYKeZFuSfUnuSHLpMcb8ZpI9SW5P8uHZTlOaP3e5qHUTX75J\nNgFXA+cD3wV2J/lUVe0bG3MGcCnwgqr6YZKfnteEpXlxyUWtm+YM/Rxgf1UdqKr7gF3A9hVjXgv8\nZVX9EKCq7prtNKX5M+hq3TRB3wwcHDs+1F037unAM5J8Mcm/JNk6qwlK68Wgq3XTvHyzynW1yv2c\nAbwQOA345yRn33/GLrXAoKt107x8DzGK9P1OZbSWvnLMl6rqCPAfSb4JnAl8deWdLS0tHb08GAwY\nDAbHN2NpTgy6NorhcMhwODzu26Vq5cn2igHJCcA3GX1T9HvALcCOqto7NmZrd92rum+IfhXYUlXf\nX3FfNenxpEXZuRO+8IXRR2kjSUJVrbZa8iAT19Crahm4BLgB2APsqqq9Sa5IckE35h+Au5PsAf4R\neMvKmEsbnWfoat1UL9+q+izwjBXXXb7i+M3Am2c3NWl9GXS1zp8UlToGXa0z6FLHoKt1Bl3qGHS1\nzqBLneVl35xLbTPoUsczdLXOoEsdg67WGXSpY9DVOoMudQy6WmfQpY5BV+sMutTxV9CpdQZd6vgr\n6NQ6gy51XHJR6wy61DHoap1BlzoGXa0z6FLHoKt1Bl3quMtFrTPoUsddLmqdQZc6LrmodQZd6hh0\ntc6gSx2DrtYZdKlj0NU6gy513OWi1hl0qeMuF7XOoEsdl1zUOoMudQy6WmfQpY5BV+sMutQx6Gqd\nQZc67nJR66YKepJtSfYluSPJpWuM+/UkR5I8Z3ZTlNaHu1zUuolBT7IJuBrYCpwN7Ehy1irjHg28\nHrh51pOU1oNLLmrdNGfo5wD7q+pAVd0H7AK2rzLuSuAq4EcznJ+0bgy6WjdN0DcDB8eOD3XXHZVk\nC3BqVV0/w7lJ68qgq3XTvHyzynV19JNJgHcBF0+4jbShGXS1bpqX7yHgtLHjU4Hvjh0/htHa+rCL\n+ynAp5JcWFW3rryzpaWlo5cHgwGDweD4Zy3NgUHXRjEcDhkOh8d9u1TV2gOSE4BvAucD3wNuAXZU\n1d5jjL8JeFNVfW2Vz9Wkx5MW5QlPgG9/e/RR2kiSUFUTVz4mrqFX1TJwCXADsAfYVVV7k1yR5ILV\nboJLLmqQZ+hq3cQz9Jk+mGfo2sBOPhnuuWf0UdpIZnaGLj1SeIau1hl0CajyR//VPoMuAUeOwKZN\noz9Sq3z5Snh2rn4w6BK+MZf6waBL+A1R9YNBlzDo6geDLmHQ1Q8GXcKgqx8MuoS7XNQPBl3CXS7q\nB4Mu4ZKL+sGgSxh09YNBlzDo6geDLmHQ1Q8GXcJdLuoHgy7hLhf1g0GXcMlF/WDQJQy6+sGgSxh0\n9YNBlzDo6geDLuEuF/WDQZdwl4v6waBLuOSifjDoEgZd/WDQJQy6+sGgSxh09YNBlzDo6geDLjHa\n5eK2RbVuqqAn2ZZkX5I7kly6yuf/MMmeJLcl+VySp8x+qtL8eIauPpgY9CSbgKuBrcDZwI4kZ60Y\ndivw3KraAnwC+NNZT1SaJ4OuPpjmDP0cYH9VHaiq+4BdwPbxAVX1T1X1f93hzcDm2U5Tmi+Drj6Y\nJuibgYNjx4dYO9ivAT7zcCYlrTeDrj6Y5iWcVa6rVQcmrwCeC7zoWHe2tLR09PJgMGAwGEwxBWm+\nDLo2kuFwyHA4PO7bpWrVNj8wIDkXWKqqbd3xZUBV1VUrxr0YeA/wwqq6+xj3VZMeT1qEK6+Ee+8d\nfZQ2miRU1Won1w8yzZLLbuCMJKcnOQm4CLhuxYM9G3gvcOGxYi5tZJ6hqw8mBr2qloFLgBuAPcCu\nqtqb5IokF3TD3gn8FHBtkq8l+eTcZizNgUFXH0z1Eq6qzwLPWHHd5WOXXzLjeUnryqCrD/xJUQmD\nrn4w6BIGXf1g0CX8FXTqB4Mu4a+gUz8YdAmXXNQPBl3CoKsfDLqEQVc/GHQJg65+MOgS7nJRPxh0\nCXe5qB8MuoRLLuoHgy5h0NUPBl3CoKsfDLqEQVc/GHQJd7moHwy6hLtc1A8GXcIlF/WDQZcw6OoH\ngy5h0NUPBl3CoKsfDLqEu1zUDwZdwl0u6geDLuGSi/rBoEsYdPWDQZcw6OoHgy5h0NUPBl3CoKsf\nDLrEaJeL2xbVuqmCnmRbkn1J7khy6SqfPynJriT7k3wpyWmzn6o0P56hqw8mBj3JJuBqYCtwNrAj\nyVkrhr0GuKeqzgTeDbxz1hOV5smgqw+mOUM/B9hfVQeq6j5gF7B9xZjtwN90lz8OnD+7KUrzVeWS\ni/phmnOSzcDBseNDjCK/6piqWk7ygyRPrKp7ZjPN47d3L3zrW4t6dLXk/pgni56J9PBME/TVXuY1\nYUxWGQPA0tLS0cuDwYDBYDDFFI7f7t1w7bVzuWv10KtetegZSA8YDocMh8Pjvl2qVu3uAwOSc4Gl\nqtrWHV8GVFVdNTbmM92YLyc5AfheVf3MKvdVkx5PkvRgSaiqiV9DTrOGvhs4I8npSU4CLgKuWzHm\n08DF3eXfAG48nslKkh6+iUsu3Zr4JcANjP4HcE1V7U1yBbC7qv4euAb4UJL9wN2Moi9JWkcTl1xm\n+mAuuUjScZvlkoskqQEGXZJ6wqBLUk8YdEnqCYMuST1h0CWpJwy6JPWEQZeknjDoktQTBl2SesKg\nz9BDebvLlvT5+fX5uYHP75HCoM9Q319UfX5+fX5u4PN7pDDoktQTBl2SemLd3z533R5MknpkmrfP\nXdegS5LmxyUXSeoJgy5JPbGQoCd5fZJ9SW5P8o5FzGHekrwlyZEkT1z0XGYlyTuT7E1yW5JPJHns\nouc0C0m2da/HO5Jcuuj5zFKSU5PcmOQb3b+3Nyx6TrOWZFOSW5Os/OX1zUvyuCTXdv/u9iR5/lrj\n1z3oSQbAy4FnVdUvAH+23nOYtySnAi8GDix6LjN2A3B2VW0B9gN/suD5PGxJNgFXA1uBs4EdSc5a\n7Kxm6jDwpqr6eeAFwOt69vwA3gh8Y9GTmJP3ANdX1TOBXwL2rjV4EWfovw+8o6oOA1TVXQuYw7y9\nC/ijRU9i1qrq81V1pDu8GTh1kfOZkXOA/VV1oKruA3YB2xc8p5mpqv+qqtu6y//DKAibFzur2elO\nnl4G/PWi5zJrSR4D/EpV7QSoqsNV9cO1brOIoD8deGGSm5PclOSXFzCHuUnycuBgVd2+6LnM2e8A\nn1n0JGZgM3Bw7PgQPQreuCRPBbYAX17sTGbq/pOnPm7XexpwV5Kd3ZLS+5KcvNYNTpzHLJJ8Dnjy\n+FWM/oO/vXvMx1fVuUmeB/xtN/FmTHh+bwVesuJzzVjjub2tqj7djXkbcF9VfXQBU5y11f5+eheH\nJI8GPg68sTtTb16SXwPurKrbuqXcpv6tTeFE4DnA66rqK0neDVwGXL7WDWauql5yrM8l+T3g77px\nu7tvHD6pqu6ex1zm4VjPL8mzgKcC/5okjJYkvprknKr673Wc4kO21t8dQJKLGX2J+6vrM6O5OwSc\nNnZ8KvDdBc1lLpKcyCjmH6qqTy16PjN0HnBhkpcBJwOPSfLBqnrlguc1K4cYfbX/le7448Ca37Rf\nxJLLJ4HzAZI8HXhUSzFfS1V9vapOqaqnVdXPMvoLeXYrMZ8kyTbgj4ELq+pHi57PjOwGzkhyepKT\ngIuAvu2WeD/wjap6z6InMktV9daqOq2qnsbo7+3GHsWcqroTONh1EkbdXPObv3M5Q59gJ/D+JLcD\nPwJ68xewiqJfXwb+BXAS8LnRFyDcXFV/sNgpPTxVtZzkEkY7eDYB11TVmjsJWpLkPOC3gduTfI3R\na/KtVfXZxc5MU3oD8JEkjwK+Dbx6rcH+6L8k9YQ/KSpJPWHQJaknDLok9YRBl6SeMOiS1BMGXZJ6\nwqBLUk8YdEnqif8HX6cWY0SWHiEAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x105b28208>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "%matplotlib inline \n",
    "\n",
    "import numpy as np\n",
    "import matplotlib.pylab as plt\n",
    "\n",
    "def step_function(x):\n",
    "    return np.array(x > 0, dtype=np.int)\n",
    "\n",
    "x = np.arange(-5.0, 5.0, 0.1)\n",
    "y = step_function(x)\n",
    "plt.plot(x, y)\n",
    "plt.ylim(-0.1, 1.1) # y축의 범위 지정\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 3.2.4 시그모이드 함수 구현하기"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "def sigmoid(x):\n",
    "    return 1 / (1 + np.exp(-x))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "np.exp(-x)는 exp(-x) 수식에 해당. 인수 x가 넘파이 배열이어도 올바른 결과가 나옴"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([ 0.26894142,  0.73105858,  0.88079708])"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "x = np.array([-1.0, 1.0, 2.0])\n",
    "sigmoid(x)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "브로드캐스트: 넘파이 배열과 스칼라 값의 연산을 넘파이 배열의 원소 각각과 스칼라 값의 연산으로 바꿔 수행"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([ 2.,  3.,  4.])"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "t = np.array([1.0, 2.0, 3.0])\n",
    "1.0 + t"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([ 1.        ,  0.5       ,  0.33333333])"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "1.0 / t"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "np.exp(-x)가 넘파이 배열을 반환하기 때문에 1 / 1 + np.exp(-x))도 넘파이 배열의 각 원소에 연산을 수행한 결과를 냄"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "시그모이드 함수 그래프"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXQAAAD7CAYAAAB68m/qAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAGklJREFUeJzt3XmYVOWZ9/HvjQgqIC4EiSAI4hJQQEEwcTBl1JEYhdHR\nCPpO1JhEXGIwYyRoRtsrmUTcjTqRRNS4hajggmLEFyhGwws2IILsQ5RVWQV0Ilv3/f7xlNppm+5q\nuqqfqlO/z3Wdq6u6TlXdZTc/n77Pc55j7o6IiBS/JrELEBGR3FCgi4gkhAJdRCQhFOgiIgmhQBcR\nSQgFuohIQjRtzDczM82RFBHZA+5ude3T6CN0d0/sdsstt0SvQZ9Pn02fL3lbttRyERFJCAW6iEhC\nKNBzKJVKxS4hr5L8+ZL82UCfr1RYffozDX4zM2/M9xMRSQIzwwvxoKiIiOSHAl1EJCEU6CIiCaFA\nFxFJCAW6iEhCKNBFRBJCgS4ikhAKdBGRhKgz0M1stJmtNbO5tezzWzNbamZzzKxXbksUEZFsZDNC\nfxQ4c3cPmtm3gSPc/UjgCuChHNUmIiL1UGegu/ubwEe17DIIeDyz7wygtZkdkpvyREQkW7noobcH\nVla5vzrzPRERaUS5uGJRTQvG7HYFrrKyss9vp1IprZImIlJNOp0mnU7X+3lZrbZoZp2A8e7eo4bH\nHgKmuPufM/cXAd9097U17KvVFkUkmspK2LwZNm78Ytu0KWybN4dty5Yvtq1bw/bxx3DqqfDkk3Hq\nzna1xWxH6EbNI3GAl4CrgT+b2UnA5prCXEQkX3buhDVrYOVKWL063F6zBj78ENauDV/Xr4cNG6BF\nC2jTBg4+GA46KHw98MCwHXYYHHssHHAA7L//F1urVuF7ha7OQDezp4EUcLCZrQBuAZoB7u6/d/cJ\nZnaWmf0P8L/AZfksWERKjzusWwdLlsCyZWF77z14//2wrVsHbduGQO7QAQ49FL761RDO7drBIYeE\nx9u0gWbNYn+a/NEFLkSkYLiHkfXcufDuu7BgQdgWL4amTeHII6FrV+jSJWydO0OnTtC+fXg8qbJt\nuSjQRSQKd1i+HN56C2bOhFmzYM4c2Gsv6NkzjK67d4du3eDoo0NrpFQp0EWkoOzcCW+/DW+8Ebbp\n08EM+vWDPn2gd284/vjQIpF/pEAXkajcYf58mDgRJk2CN98M7ZH+/cP2jW+EnrfVGVOiQBeRRrdt\nWwjv8eNhwoTQPjnzTDj99DDtr5TbJg2R62mLIiI1+vRTeOUVeOaZMBrv0QMGDQq3jz5aI/DGpBG6\niNRbZSVMmQJ//GMYjffpA9/9bgjytm1jV5c8armISM6tWgUPPwyPPhpOyrn0UrjwQh3IzDe1XEQk\nJ9zDaPz++2HqVLjoInjxReilKx8UHAW6iNRoxw4YMwbuvjvc/slP4IknoGXL2JXJ7ijQReQfbNsW\nWiq33RbOyvzNb8JMlSa6YGXBU6CLCBBO/Bk9Gn71q3Cm5pgx8PWvx65K6kOBLlLi3MOUw5tuCmuj\nPP88nHhi7KpkTyjQRUpYeTkMGxbaLKNGwWmnxa5IGkJdMZEStGEDXH55mDd++eUh2BXmxU+BLlJC\n3MPJQMceGy7asGgRfP/7OuCZFGq5iJSIFSvgBz8Il1t75ZWwuqEki/6/LJJw7mH2Su/eYYGs6dMV\n5kmlEbpIgm3cGHrky5fD5Mlw3HGxK5J80ghdJKGmTg0XjDjiiDAqV5gnn0boIgnjDiNHwn33wSOP\nwLe/HbsiaSwKdJEE2bIlrID4wQdhKmKHDrErksaklotIQixdGq7P2a5daLcozEuPAl0kAdJp+Kd/\nguuug9/9Dpo3j12RxKCWi0iRe+wxGD4cnn5aZ3uWOgW6SJFyh1//OlxBaOpUOOaY2BVJbAp0kSJU\nUREuOPHmmzBtGnz1q7ErkkKgQBcpMjt3wve+Bx9+GEbmrVvHrkgKhQJdpIhs3w5DhoSvEybAvvvG\nrkgKSVazXMxsgJktMrMlZja8hscPM7PJZjbbzOaYmU5lEMmxbdvg3HPDyojPP68wly8zd699B7Mm\nwBLgNGANUA4MdvdFVfYZBcx291Fm9jVggrt3ruG1vK73E5Ev2749hHnr1uFCzU31t3VJMTPc3era\nL5sRel9gqbsvd/edwBhgULV9KoH9M7cPAFbXp1gR2b0dO+CCC2C//RTmUrtsfjXaAyur3F9FCPmq\nbgUmmtm1wH7A6bkpT6S07doFF10EZmGeucJcapPNr0dNw/zqfZMhwKPufo+ZnQQ8CXSv6cXKyso+\nv51KpUilUlkVKlJq3GHoUNi6FcaPh2bNYlckjSWdTpNOp+v9vGx66CcBZe4+IHP/54C7+8gq+7wL\nnOnuqzP3lwH93H1DtddSD10kSyNGhDXMJ02Cli1jVyMx5bKHXg50NbNOZtYMGAy8VG2f5WTaLJmD\nos2rh7mIZO+ee+CFF8Kl4hTmkq06Wy7uXmFm1wATCf8DGO3uC83sVqDc3V8Grgf+YGbXEQ6QXpLP\nokWS7Nln4e67wxmgbdrErkaKSZ0tl5y+mVouIrWaNg3+5V9g4kTo1St2NVIoctlyEZFGsGwZ/Ou/\nwh//qDCXPaNAFykAW7bA2WfDzTfrknGy59RyEYmsogIGDoTOneGBB2JXI4VILReRIjFiBHz6aZjZ\nItIQOu9MJKKnnoKxY+Gtt2DvvWNXI8VOLReRSObODZeMmzQJevSIXY0UMrVcRArY5s1w3nmhzaIw\nl1zRCF2kkVVWhrnmnTrB/ffHrkaKQbYjdPXQRRrZnXfC+vXw3HOxK5Gk0QhdpBFNmxYuVFFeDh07\nxq5GioV66CIFZuPGcD3QP/xBYS75oRG6SCNwh0GDoGvXsPCWSH2ohy5SQP7rv2DNGvXNJb80QhfJ\ns/nzIZWCv/4VjjoqdjVSjNRDFykA27aFvvnIkQpzyT+N0EXy6LrrYNUqeOaZcKFnkT2hHrpIZJMn\nh6sPzZ2rMJfGoZaLSB5s3gyXXQajR8NBB8WuRkqFWi4ieXDJJdCiRZjdItJQarmIRPL88+GM0Dlz\nYlcipUYjdJEc2rABjjsuzDc/+eTY1UhSZDtCV6CL5NCQIXDooXDXXbErkSRRy0WkkY0bB7NmhQOh\nIjFohC6SA2q1SD6p5SLSiP7t36BNG13oWfJDLReRRvLqq2GdlnnzYlcipU6BLtIAH38MQ4eGvnmL\nFrGrkVKnlotIA1xzDfz97/DII7ErkSTL6WqLZjbAzBaZ2RIzG76bfb5rZvPNbJ6ZPVnfgkWKzbRp\nYWaLpihKoaiz5WJmTYAHgNOANUC5mb3o7ouq7NMVGA583d23mlmbfBUsUgh27IAf/QjuvRcOPDB2\nNSJBNiP0vsBSd1/u7juBMcCgavv8EHjQ3bcCuPuG3JYpUljuuAMOPxwuuCB2JSJfyOagaHtgZZX7\nqwghX9VRAGb2JuF/Ere6+2s5qVCkwCxZEqYnzpqlZXGlsGQT6DX9ylY/stkU6AqcAnQE3jCz7p+N\n2EWSwh2uvBJuugk6dYpdjcg/yibQVxFC+jMdCL306vv8P3evBN43s8XAkcCs6i9WVlb2+e1UKkUq\nlapfxSIRPfkkfPQR/PjHsSuRJEun06TT6Xo/r85pi2a2F7CYcFD0A+AtYIi7L6yyz5mZ712aOSA6\nC+jl7h9Vey1NW5SitWkTdOsG48fDiSfGrkZKSc6mLbp7BXANMBGYD4xx94VmdquZnZ3Z5zVgo5nN\nByYB11cPc5FiN2IEnH++wlwKl04sEsnCtGlhRsuCBdC6dexqpNTk9MQikVK2a1c4EHrXXQpzKWwK\ndJE63H8/tG0LF14YuxKR2qnlIlKL1auhZ8/QcjnqqNjVSKlSy0UkB37609BuUZhLMdDyuSK78frr\nUF4Ojz0WuxKR7GiELlKD7dvh6qvhvvtg331jVyOSHQW6SA3uuguOOQbOOSd2JSLZ00FRkWrefx96\n94aZM6Fz59jViOigqMgeGzYsbApzKTY6KCpSxYQJMH8+jBkTuxKR+lOgi2Rs2xZWUXzwQdhnn9jV\niNSfWi4iGbffHk4iGjAgdiUie0YHRUWA996DPn1g9mxduEIKjw6KitTDsGHw7/+uMJfiph66lLxX\nXoGFC+GZZ2JXItIwCnQpadu2wbXXhgOhzZvHrkakYdRykZJ2++3Qq5cOhEoy6KColKz33guXk5s9\nGzp2rHt/kVh0UFSkDsOGheVxFeaSFOqhS0l6+WVYtEgHQiVZFOhScj79NBwIfeghHQiVZFHLRUrO\nyJFhNcV//ufYlYjklg6KSklZtgz69YM5c6BDh9jViGRHB0VFqnEPi2/dcIPCXJJJgS4l4/nnYfny\nMLtFJInUcpGS8Mkn0K0bPPEEfPObsasRqZ9sWy4KdCkJN9wAH34Ijz8euxKR+lOgi2S8+y6cemr4\nesghsasRqb+cHhQ1swFmtsjMlpjZ8Fr2O9/MKs3shPoUK5IvlZVw5ZVw660Kc0m+OgPdzJoADwBn\nAt2BIWZ2TA37tQR+DEzPdZEie+rRR2HHDrjiitiViORfNiP0vsBSd1/u7juBMcCgGvb7JTAS2J7D\n+kT22Pr1cOONMGoU7LVX7GpE8i+bQG8PrKxyf1Xme58zs15AB3efkMPaRBrkZz+Diy8Oy+OKlIJs\n1nKpqRH/+ZFNMzPgHuCSOp4j0mjSaZg0CRYsiF2JSOPJJtBXAVUXGO0ArKlyvxWht57OhHs74EUz\nG+jus6u/WFlZ2ee3U6kUqVSq/lWL1GLbttAzf+ABaNUqdjUi9ZdOp0mn0/V+Xp3TFs1sL2AxcBrw\nAfAWMMTdF+5m/ynAT9397Roe07RFybubb4b582Hs2NiViORGttMW6xyhu3uFmV0DTCT03Ee7+0Iz\nuxUod/eXqz8FtVwkkgUL4He/C4tviZQanVgkiVFZGU7rHzIErroqdjUiuaPVFqXkjBoFFRUwdGjs\nSkTi0AhdEmHlSjjhBJg6NSzCJZIkGqFLyXAPp/dfe63CXEqbrikqRe9Pf4IVK2DcuNiViMSllosU\ntbVroWdPGD8eTjwxdjUi+aHlcyXx3OH88+HII+G222JXI5I/OZuHLlKonn0WFi6Ep56KXYlIYdAI\nXYrSunXQowe89BL07Ru7GpH8UstFEuuzVkvXrjByZOxqRPJPLRdJrCefhCVL1GoRqU4jdCkqK1ZA\nnz4wcaLWOZfSoROLJHEqK+Gyy+C66xTmIjVRoEvRuO8++PTTcCUiEfkytVykKMyZA2ecATNmQJcu\nsasRaVxquUhi/P3vcNFFcPfdCnOR2miELgXvqqtg8+Ywq8V06RQpQZq2KInwwgvw6qvw9tsKc5G6\naIQuBev998NZoOPHQ79+sasRiUc9dClqO3bAhRfC8OEKc5FsaYQuBen662Hx4rBWi1otUurUQ5ei\nNW5cWElx9myFuUh9aIQuBWXxYujfH155RResEPmMeuhSdD75BM47D/7zPxXmIntCI3QpCO4weDC0\nbAkPP6xWi0hV6qFLUfn1r8M0xalTFeYie0qBLtG9+CI89FBYp2WffWJXI1K8FOgS1bx58IMfwIQJ\ncOihsasRKW46KCrRrFkDZ58dlsXVQVCRhlOgSxSffALnnAM//GFYSVFEGi6rQDezAWa2yMyWmNnw\nGh6/zszmm9kcM3vdzA7LfamSFLt2hRktPXvCTTfFrkYkOeoMdDNrAjwAnAl0B4aY2THVdpsN9Hb3\nXsBY4I5cFyrJ4B6Ww92+HUaN0owWkVzKZoTeF1jq7svdfScwBhhUdQd3n+ru2zJ3pwPtc1umJMUv\nfhGWwh03DvbeO3Y1IsmSTaC3B1ZWub+K2gP7cuDVhhQlyXTvvTB2bJjR0qpV7GpEkiebaYs1/VFc\n4+meZvZ/gN7AN3f3YmVlZZ/fTqVSpFKpLEqQYvfII+EScm++CV/5SuxqRApbOp0mnU7X+3l1nvpv\nZicBZe4+IHP/54C7+8hq+50O3Aec4u4bd/NaOvW/BD3xBIwYAVOmwJFHxq5GpPjkcnGucqCrmXUy\ns2bAYOClam92PPAQMHB3YS6l6c9/DhepeP11hblIvtUZ6O5eAVwDTATmA2PcfaGZ3WpmZ2d2ux1o\nATxrZm+b2Qt5q1iKxlNPwbBh8Npr8LWvxa5GJPm02qLkxejRcPPNMHEidO8euxqR4qbVFiWa+++H\nO++EdFptFpHGpECXnHGH//gPeOaZsAzu4YfHrkiktCjQJSd27YIrrgirJ/71r5qaKBKDAl0abMuW\nsDYLwOTJ4apDItL4tNqiNMjf/gbf+AZ06QLjxyvMRWJSoMsemzoVTj45LLb14IPQVH/viUSlf4JS\nb+7hNP477ghngZ5xRuyKRAQU6FJPW7eGS8a99164BminTrErEpHPqOUiWZsxA44/Hg4+GN54Q2Eu\nUmg0Qpc6VVTA7bfDPffAQw/BeefFrkhEaqJAl1otXgyXXgr77AMzZ0LHjrErEpHdUctFarRrF9x1\nV5jFcvHFMGmSwlyk0GmELl9SXg4/+lHolU+fDl27xq5IRLKhEbp8bv16uPJKGDgQrr8+rGGuMBcp\nHgp0YceOcMCzWzdo3hwWLAhtFqtzsU4RKSRquZSwigr405/CuuXHHAP//d+6EIVIMVOgl6CKChg7\nFn75S9h/f3jsMTjllNhViUhDKdBLyI4dMGYM/OY30Lo13HYbnHWWWisiSaFALwEffQS//324ktDR\nR8Nvfwunn64gF0kaHRRNsJkz4fLLoXNnePfdsLztpElhMS2FuUjyaISeMBs2wNNPh774pk0wdCgs\nWQJt28auTETyzdy98d7MzBvz/UrFxx/DSy+F/vgbb8DZZ8Nll8Gpp0IT/Q0mUvTMDHev8+9qBXqR\n2rAhtFBeeAHSaejfP1wGbuDAMHNFRJJDgZ4wFRUwaxa89hpMmADz54de+Lnnwne+AwceGLtCEckX\nBXqRq6iAd94JLZQpU8Ll3jp0CCF+1llhRN68eewqRaQxKNCLzAcfhFkpM2aE7a23QoD37x9O+vnW\nt6Bdu9hVikgMCvQCtWMHLF0aphG+8w7MnQuzZ8P27dC7N/TrByedFL62aRO7WhEpBAr0iCorYc0a\nWLYsbIsXh6mDixaFa3F26gTdu0PPntCjB5xwQlhrXHPDRaQmOQ10MxsA3Es4EWm0u4+s9ngz4HGg\nN7ABuNDdV9TwOkUf6O5hfveaNWFbvRpWrgzbihXw/vvh9oEHwhFHQJcu4ezMo44K29FHq/ctIvWT\ns0A3sybAEuA0YA1QDgx290VV9rkSOM7drzKzC4Fz3X1wDa9VUIHuDp98Aps3h9PjN20K28aNX2zr\n18O6dWFbuzZsLVpA+/Zw6KFhO+yw0O/u1AkOPzyMtvfbL/anE5GkyDbQszlTtC+w1N2XZ154DDAI\nWFRln0HALZnbzwEP1K/c3JswAf7ylxDYH3/8j9vWrbBlS7i9775wwAFhO+igMLI++ODQv/7KV8Ia\n4W3bhtvt2oXb++wT+9OJiHxZNoHeHlhZ5f4qQsjXuI+7V5jZZjM7yN035abM+mvVKlxtp2XLMKLe\nf//wvVatwu3WrcPtvfeOVaGISG5lE+g1DfOr902q72M17ANAWVnZ57dTqRSpVCqLEuqvf/+wiYgU\nm3Q6TTqdrvfzsumhnwSUufuAzP2fA171wKiZvZrZZ4aZ7QV84O5fWg6q0HroIiLFINseejZLN5UD\nXc2sU2Y2y2DgpWr7jAcuydy+AJhcn2JFRKTh6my5ZHri1wAT+WLa4kIzuxUod/eXgdHAE2a2FNhI\nCH0REWlEOrFIRKTA5bLlIiIiRUCBLiKSEAp0EZGEUKCLiCSEAl1EJCEU6CIiCaFAFxFJCAW6iEhC\nKNBFRBJCgS4ikhAK9Bzak+Uui0mSP1+SPxvo85UKBXoOJf2XKsmfL8mfDfT5SoUCXUQkIRToIiIJ\n0ejL5zbam4mIJEg2y+c2aqCLiEj+qOUiIpIQCnQRkYSIEuhm9mMzW2Rm88zsthg15JuZXW9mlWZ2\nUOxacsXMbjezhWY2x8zGmtn+sWvKBTMbkPl9XGJmw2PXk0tm1sHMJpvZgsy/t2tj15RrZtbEzGab\nWfWL1xc9M2ttZs9m/t3NN7N+te3f6IFuZingHOBYdz8OuLOxa8g3M+sAnA4sj11Ljk0Eurt7L2Ap\nMCJyPQ1mZk2AB4Azge7AEDM7Jm5VObUL+Km7dwO+DlydsM8H8BNgQewi8uQ+YIK7fw3oCSysbecY\nI/QrgdvcfReAu2+IUEO+3QP8LHYRuebu/9fdKzN3pwMdYtaTI32Bpe6+3N13AmOAQZFryhl3/9Dd\n52Ruf0IIhPZxq8qdzODpLODh2LXkmpm1Avq7+6MA7r7L3bfW9pwYgX4UcIqZTTezKWbWJ0INeWNm\n5wAr3X1e7Fry7PvAq7GLyIH2wMoq91eRoMCryswOB3oBM+JWklOfDZ6SOF2vC7DBzB7NtJR+b2b7\n1vaEpvmowsxeBw6p+i3Cf/BfZN7zAHc/ycxOBJ7JFF406vh8NwJnVHusaNTy2W5y9/GZfW4Cdrr7\n0xFKzLWafj6JCwczawk8B/wkM1Ivemb2HWCtu8/JtHKL6t9aFpoCJwBXu/tMM7sX+DlwS21PyDl3\nP2N3j5nZUGBcZr/yzIHDg919Yz5qyYfdfT4zOxY4HHjHzIzQkphlZn3dfV0jlrjHavvZAZjZJYQ/\ncb/VOBXl3SqgY5X7HYA1kWrJCzNrSgjzJ9z9xdj15NDJwEAzOwvYF2hlZo+7+/ci15Urqwh/7c/M\n3H8OqPWgfYyWywvAaQBmdhSwdzGFeW3c/V13b+fuXdy9M+EHcnyxhHldzGwAcAMw0N23x64nR8qB\nrmbWycyaAYOBpM2WeARY4O73xS4kl9z9Rnfv6O5dCD+3yQkKc9x9LbAyk5MQcrPWg795GaHX4VHg\nETObB2wHEvMDqIGTrD8D7weaAa+HP0CY7u5XxS2pYdy9wsyuIczgaQKMdvdaZxIUEzM7GbgYmGdm\nbxN+J29097/ErUyydC3wlJntDfwNuKy2nXXqv4hIQuhMURGRhFCgi4gkhAJdRCQhFOgiIgmhQBcR\nSQgFuohIQijQRUQSQoEuIpIQ/x/gQXz/zfULcAAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x105c25860>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "x = np.arange(-5.0, 5.0, 0.1)\n",
    "y = sigmoid(x)\n",
    "plt.plot(x, y)\n",
    "plt.ylim(-0.1, 1.1) # y축의 범위 지정\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "시그모이드(sigmoid): 'S자 모양'이라는 뜻"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 3.2.5 시그모이드 함수와 계단 함수 비교\n",
    "계단 함수(점선)와 시그모이드 함수(실선)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYMAAAEKCAYAAADw2zkCAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAHzxJREFUeJzt3Xl8VPW5x/HPA4iAICKoKJsKgoKKFreq6PSChVqUW29V\nUG+rtbViraK1WmqvxvbWK1i3ulMRW5dSF0QpqKg4VGuVACIIshSRLS4sAtIaluS5f/xGmoYQJsk5\nOTOT7/v1Oi9mJr+ZeebFyTz5bc8xd0dERBq2RkkHICIiyVMyEBERJQMREVEyEBERlAxERAQlAxER\nAZokHUC2zExrYEVEasHdbVdt8qpn4O4Fe9x4442Jx6DPp8+mz1d4R7byKhmIiEg8lAxERETJIFek\nUqmkQ4hVIX++Qv5soM/XUFhNxpSSZGaeL7GKiOQKM8MLbQJZRETioWQgIiJKBiIiomQgIiIoGYiI\nCEoGIiKCkoGIiKBkICIixJwMzGyMmX1iZnOqafNbM1tsZrPN7Kg44xERkarF3TMYCwzY2Q/N7BtA\nV3c/BPgh8EDM8YiISBViTQbu/gbwWTVNBgN/yLR9G2htZvvFGZOIiOwo6YvbdABWVLi/KvPYJ8mE\nI1J7W8q28MoHr1BWXrb9sbYt2nJipxN3aLvmn2v424q/7fC42hdm+1O6nELrZq13aJdLkk4GVRVP\n2mk1uqKiou23U6mUqg1KTpm+ajoXjL+AkzqftP2xI/c9ssovi1UbVzF61ugdHlf7wmx/+L6H11sy\nSKfTpNPpGj8v9qqlZtYFmOjuR1bxsweA19z9T5n7C4BT3X2HnoGqlkque/fjd3l87uOMOm1U0qFI\nDMrLYf16WLv2X8e6deFYvz4cGzb869i4MRyffw5f+xo89lgycWdbtbQ+ksGBhGRwRBU/Ox34kbt/\n08xOAO509xN28jpKBiISua1boaQEVqyAVavC7ZIS+Phj+OST8O/q1bBmDeyxB7RrB23bwt57h3/b\ntAnHXntB69bh3z33/NfRqlV4rGXLZD5ftskg1mEiM3sCSAFtzWw5cCPQFHB3H+3uk83sdDP7O/AP\n4KI44xGRhscdPv0UFi2CJUvCsXQpfPhhOD79FPbdFzp1go4d4YADYP/94fDDoX172G+/8PN27aBp\n06Q/TXx0cRsRKQju4S/6OXPgvfdg/vxwLFwITZrAIYdAt25w8MHhOOgg6NIFOnQIPy9UOTNMFBUl\nAxH5kjssWwbTp8OMGTBzJsyeDY0bQ+/e4a/6Xr2gZ0/o0SMM5zRUSgYiUjC2boV33oHXXw/HW2+B\nGRx/PBxzDPTpA0cfHYZ15N8pGYjUs5LPS1j9j9X0bt876VDynjvMmwdTpsCrr8Ibb4Qhnb59w3Hi\niWGM33b5FSdKBiL17NF3H+WlJS/x2FkJrSHMc6Wl4Yt/4kSYPDkM+QwYAP37h6WZDXmopy5yYjWR\nSENS5mU0aaRfqZr44guYNAmefDL0Ao48EgYPDrd79NBf/vVJZ65IRLaVb1MyyEJ5Obz2Gvz+96EX\ncMwxcM45cM89YQmnJENnrkhElAyqt3IlPPQQjB0bNmxdeCGMGqVJ31yhM1ckIkoGO3IPvYC774Zp\n0+C88+C55+AoXbkk5+jMFYlIxz070tgaJx1GTtiyBcaNg9tvD7evvBIefTS5kgyya1pNJCKRKS0N\nw0C33BJ2+15zTVgR1EgX2E2MVhOJSL3ZuhXGjIH//d+wA3jcOPjqV5OOSmpCyUBEas09LAu9/vpQ\n6+fZZ+HYY5OOSmpDyUBEaqW4GIYPD0NDDz4I/folHZHUhUbyRKRG1qyBiy8Om8MuvjgkBSWC/Kdk\nIBKRhWsW8sFnHyQdRmzcw0axww8PF2xZsAC+9z1NDhcKDROJROShWQ+xzx77cO1J1yYdSuSWL4fv\nfz9c4nHSpFAlVAqLcrpIRAqxNpF7WCXUp08oFvfWW0oEhaqwzlyRBBXaDuS1a8OcwLJlMHUqHLHD\nVcylkKhnIBKRQkoG06aFi8V07Rp6A0oEha8wzlyRHFAIycAdRo6Eu+6Chx+Gb3wj6YikvuT3mSuS\nQw5rdxidW3dOOoxa27AhVBL96KOwXLRjx6Qjkvqk2kQiwuLFcMYZYZL4zjth992Tjkiikm1tIs0Z\niDRw6TScfDJcdRXcf78SQUOlYSKRBuyRR+C66+CJJ7SLuKFTMhBpgNzh5pvDlcemTYNDD006Ikma\nkoFIA1NWFi4288Yb8OabsP/+SUckuUBzBiIRmVEyg9X/WJ10GNXauhUuuADmzQs9AiUC+ZKSgUhE\nrp96PTM/mpl0GDu1eTOcey5s3AiTJ0Pr1klHJLkk9mRgZgPNbIGZLTKz66r4eSczm2pms8xstplp\nm4vkpbLy3K1NVFoK3/pWqDD67LPQvHnSEUmuiTUZmFkj4B5gANALGGpmlaeqfgH8yd2/AgwF7osz\nJpG45OoO5M2b4ayzQk9g3Dho2jTpiCQXxd0zOA5Y7O7L3H0rMA4YXKlNObBn5vZewKqYYxKJRS4m\ngy1b4OyzoUULePRRaJJb4UkOifvU6ACsqHB/JSFBVHQTMMXMrgBaAP1jjkkkFrmWDLZtg/POA7Ow\nj0CJQKoT9+lR1RboyjUlhgJj3f0OMzsBeIwwpLSDoqKi7bdTqRSpVCqaKEUicFyH42jTrE3SYQBh\nH8Gll4bJ4okTNTTUkKTTadLpdI2fF2ttosyXe5G7D8zc/xng7j6yQpv3gAHuvipzfwlwvLuvqfRa\nqk0kkqURI8I1CF59FVq2TDoaSVKu1CYqBrqZWRczawoMAZ6v1GYZmaEhMzsM2L1yIhCR7N1xB0yY\nEC5PqUQg2Yp1mMjdy8zscmAKIfGMcff3zewmoNjd/wxcA/zOzK4iTCZ/N86YRArZU0/B7beHncXt\n2iUdjeQTlbAWKRBvvgn/+Z8wZQocdVTS0UiuyJVhIhGpB0uWwH/9F/z+90oEUjtKBiIRmbp0KqXb\nSuv9fTdsgEGD4IYbdJlKqT0lA5GInD/+fNZ9sa5e37OsLOwl6NcPhg2r17eWAqNkIBKRJGoTjRgB\nX3wRVhCJ1IX2JIpEpL53ID/+ODzzDEyfDrvtVm9vKwVKyUAkIvWZDObMgeHDw6aytm3r5S2lwGmY\nSCQi9ZUM1q8PVUjvuAOOPDL2t5MGQslAJCL9D+4fezIoL4fvfCesGrrggljfShoYbToTySOjRoWL\n00ybpuJzkp1sN50pGYjkiTffDFcrKy6Gzp2TjkbyhXYgixSQtWth6FD43e+UCCQe6hmI5Dh3GDwY\nunULRehEaiLbnoGWlorkuPvug5ISePrppCORQqZhIpEIbCnbwisfvBL5686bB0VF4bKVmjCWOCkZ\niERgQ+kGhjw9JNLXLC0N8wQjR0L37pG+tMgOlAxEIlDm0dclGjECevSAiy6K9GVFqqQ5A5EIRL37\neOrUcNWyOXPAdjn1J1J36hmIRCDKZLB+fegNjBkDe+8dyUuK7JKSgUgEokwGV14J3/wmDBgQycuJ\nZEXDRCIRaNakGakDU3V+nWefDTuNZ8+ue0wiNaFNZyI5Ys0aOOKIsJ/gpJOSjkYKhWoTieSZoUPh\ngAPgttuSjkQKiXYgi+SR8eNh5swwaSySBPUMRBKm4SGJk4aJRPLEf/83tGuni9pLPDRMJFKP1v5z\nLX9f93eO73h8jZ73wgvw17/C3LkxBSaSJe0zEInA7I9nM+LVETV6zuefw6WXwujRsMceMQUmkiUl\nA5EI1KY20YgR0K8f9O8fU1AiNRB7MjCzgWa2wMwWmdl1O2lzjpnNM7O5ZvZY3DGJRK2mO5DffDOs\nINIyUskVsc4ZmFkj4B6gH1ACFJvZc+6+oEKbbsB1wFfdfaOZtYszJpE41CQZbNkCl1wCd94JbdrE\nHJhIluLuGRwHLHb3Ze6+FRgHDK7U5gfAve6+EcDd18Qck0jkapIMbr0VDjwQzj473phEaiLu1UQd\ngBUV7q8kJIiKugOY2RuE5HSTu78Uc1wikdqnxT702b/PLtstWhSWkM6cqdLUklviTgZVne6VNws0\nAboBpwCdgdfNrNeXPQWRfNC3S1/6dulbbRt3GDYMrr8eunSpp8BEshR3MlhJ+IL/UkfC3EHlNn9z\n93LgQzNbCBwCzKz8YkVFRdtvp1IpUqlUxOGKxOexx+Czz+DHP046Eilk6XSadDpd4+fFugPZzBoD\nCwkTyB8B04Gh7v5+hTYDMo9dmJk8ngkc5e6fVXot7UCWvLVuHfTsCRMnwrHHJh2NNCTZ7kCOdQLZ\n3cuAy4EpwDxgnLu/b2Y3mdmgTJuXgLVmNg94FbimciIQyXcjRsC3v61EILlLtYlEYvbmm2Hl0Pz5\n0Lp10tFIQ5MTPQORhmLpZ0uZv3r+Do9v2xYmjW+7TYlAcpuSgUgEJiyYwO9m/m6Hx+++G/bdF849\nN4GgRGpAVUtFIlDVprNVq+DXvw7DRNpTILlOPQORCFSVDK6+OgwRde+eUFAiNaCegUgEKieDl1+G\n4mJ45JHkYhKpCfUMRCJQMRls3gw/+hHcdRc0b55wYCJZUs9AJAJd9+7Kbo12A8LKoUMPhTPOSDgo\nkRrQPgORCH34IfTpAzNmwEEHJR2NiPYZiCRi+PBwKBFIvtEwkUhEJk+GefNg3LikIxGpOSUDkQiU\nloZqpPfeC82aJR2NSM1pmEgkAqNGQe/eMHBg0pGI1I56BiJ1tHQp3PbYHF6c0AbolHQ4IrWyy56B\nmV1uZrpst8hODB8OXS+4nQWbX0k6FJFay2aYqD1QbGZPmtlAM1VZEfnSpEnw/vtwaM+yHcpRiOST\nXSYDd/8F4TKUY4ALgcVmdrOZdY05NpGcVloKV1wBv/0tuO1Ym0gkn2Q1gZzZ7fVx5tgGtAGeNrNR\nMcYmktNGjYKjjgqTxlUVqhPJJ7s8e83sCuC7wBrgIeCn7r7VzBoBi4Fr4w1RJPcsXRp6BLNmhftK\nBpLvsjl72wFnufuyig+6e/mX1zEWaWiGDw8lqjt3DvePbn80+7faP9mgROpAtYlEaujPf4af/ATm\nzIHdd086GpHqZVubSP1akRr44oswafzAA0oEUli0A1mkBkaODFVJv/71pCMRiZaGiUSytGQJHH88\nzJ4NHTsmHY1IdlTCWiRC7qEQ3bXXKhFIYVIyEMnCs8/CsmVhFVFV3lj+ButL19dvUCIRUjIQ2YVN\nm0ISuO8+aNq06jZXvHAFS9Ytqd/ARCKkZCCyC7/8JaRScOqpO29T5qpNJPlNZ69INd57D8aODf9W\nRzuQJd/F3jPIVDpdYGaLzOy6atp928zKzewrccckko3ychg2DG66Cfbbr/q2SgaS72JNBpn6RfcA\nA4BewFAzO7SKdi2BHwNvxRmPSE2MHQtbtsAPf7jrtkoGku/i7hkcByx292XuvhUYBwyuot2vgJHA\n5pjjEcnK6tXw85/Dgw9C48a7bn9ql1Np2bRl/IGJxCTuZNABWFHh/srMY9uZ2VFAR3efHHMsIln7\n6U/h/PNDiepsPDz4YfZruYuxJJEcFne/tqpdb9u3EWeumnYHoUR2dc8RqTfpNLz6Ksyfn3QkIvUn\n7mSwEuhc4X5HoKTC/VaEuYR0JjG0B54zszPdfVblFysqKtp+O5VKkUqlYghZGrLS0jBHcM890KpV\n0tGI1Fw6nSadTtf4ebHWJjKzxsBCoB/wETAdGOru7++k/WvA1e7+ThU/U20iid0NN8C8efDMM0lH\nIhKNnChh7e5lZnY5MIUwPzHG3d83s5uAYnf/c+WnoGEiScj8+XD//aEQnUhDo6qlIoQ9BaeeCkOH\nwmWX1fz5kxdPZkDXATRulMXSI5F6pKqlIjXw4INQVgaXXlq755/5xzMp9/JogxKpR9olIw3eihVh\nrmDaNGhUiz+P3J0yL1OvQPKaegbSoLmHkhNXXAE9e9buNcq9nEbWiEamXyfJX+oZSIP2xz/C8uUw\nfnztX0OlKKQQ6AyWBuuTT+Dqq2HixJ1fpyAbSgZSCNSvlQbJPawauvBCOPbYur2WmfGNbt+IJC6R\npGhpqTRITz4JRUUwaxY0a5Z0NCLxyXZpqZKBNDiffgpHHgnPPw/HHZd0NCLxUjIQqYI7fPvb0K0b\njByZdDQi8cuJchQiueaxx2DRInj88aQjEckt6hlIg7F8ORxzDEyZkv11CkTyncpRiFRQXg4XXQRX\nXRV9Iti0ZRPpD9PRvqhIPVMykAbhrrvgiy/CFcyitnzDcoZNGhb9C4vUI80ZSMGbPRtuvhnefhua\nxHDGl5WXadOZ5D31DKSg/fOfcN55cPvtcPDB8byHdiBLIVAykIJ2zTVhjuCCC+J7DyUDKQQ6g6Vg\nTZgAL7wA77wDFuP185QMpBDoDJaC9OGHcMkloQjdXnvF+16tm7Xm5E4nx/smIjHTPgMpOFu2QN++\ncM458JOfJB2NSLJUjkIarGuugYULQ+2hOIeHRPKBylFIgzR+PDz1VKhGqkQgkj31DKRgLFwYhocm\nTar7NQpECoXKUUiDsmkTnHUW/PrXSgQitaGegeQ9dxgyBFq2hIceqv/hoZLPS/jo84/oc0Cf+n1j\nkSyoZyANxs03h6Wk996bzDzBX5b9hVFvjqr/NxaJkCaQJa899xw88ECoO5TU5StVm0gKgc5gyVtz\n58L3vw+TJ8MBByQXh3YgSyHQMJHkpZISGDQolKZOesJ4W/k2mpiSgeQ3JQPJO5s2wRlnwA9+ECqS\nJk09AykEsScDMxtoZgvMbJGZXVfFz68ys3lmNtvMXjazTnHHJPlr27awcqh3b7j++qSjCTq17kTv\n9r2TDkOkTmJdWmpmjYBFQD+gBCgGhrj7ggptTgXedvdSM7sUSLn7kCpeS0tLGzh3+OEPYenSME+w\n225JRySS+3JlaelxwGJ3X+buW4FxwOCKDdx9mruXZu6+BXSIOSbJU7/4RShHPX68EoFI1OJOBh2A\nFRXur6T6L/uLgRdijUjy0p13wjPPhB5Bq1ZJRyNSeOKe9aqqa1LlWI+ZXQD0AU7d2YsVFRVtv51K\npUilUnWLTvLCww+Hy1a+8Qbss0/S0YjktnQ6TTqdrvHz4p4zOAEocveBmfs/A9zdR1Zq1x+4CzjF\n3dfu5LU0Z9AAPfoojBgBr70GhxySdDQi+SdX5gyKgW5m1sXMmgJDgOcrNjCzo4EHgDN3lgikYfrT\nn+C66+Dll3M7ESxYs4DFaxcnHYZIncSaDNy9DLgcmALMA8a5+/tmdpOZDco0GwXsATxlZu+Y2YQ4\nY5L88PjjMHw4vPQSHHZY0tFU7w/v/oEn5z2ZdBgidRL7Thl3fxHoUemxGyvcPi3uGCS/jBkDN9wA\nr7wCvXolHc2uadOZFAKdwZJT7r4bfvMbSKdze2ioIhWqk0KgM1hygjv8z//Ak0/CtGlw4IFJR5Q9\n9QykEOgMlsRt2xZ2Fs+dC3/9a/4tH1UykEKgM1gStWFDqDUEMHVquFpZvum5T08OanNQ0mGI1Iku\neymJ+eCDUH00lQqlqJvoTxORyOXKPgORKk2bBiedBJddFi5XqUQgkiz9Ckq9cg+lJW69NewuPk0L\ni0VygpKB1JuNG8NlKpcuDdcs7tIl6YhE5EsaJpJ68fbbcPTR0LYtvP66EoFIrlEykFiVlcH//V+Y\nKL71Vrj/fmjWLOmoojV91XQ+3vRx0mGI1ImSgcRm4UI4+WSYMgVmzICzzko6onj8ctovKV5VnHQY\nInWiZCCR27YNbrstrBY6/3x49VXo3DnpqOKjTWdSCHQGS6SKi+GSS8LcwFtvQbduSUcUvzJXbSLJ\nf+oZSCRWr4Zhw+DMM+Gaa8I1CBpCIgD1DKQwKBlInWzZAnfcAT17wu67w/z5YWjIdrnfsXAoGUgh\n0BkstVJWBn/8Y7juwKGHwl/+kvsXoYnLCR1OoG2LtkmHIVInqk0kNVJWBs88A7/6Fey5Z1g2esop\nSUclIjuTbW0i9QwkK1u2wLhx4cu/dWu45RY4/fSGNRwkUsiUDKRan30Go0eHK5D16AG//S30768k\nIFJoNIEsVZoxAy6+GA46CN57DyZODPsFTjtNiUCkEKlnINutWQNPPAGPPALr1sGll8KiRbDvvklH\nJiJx0wRyA/f55/D882E+4PXXYdAguOgi+NrXoJH6jVl55YNXOLHTibTYrUXSoYjsINsJZCWDBmjN\nmjDsM2ECpNPQt2+49OSZZ4YVQlIzne/ozOsXvU6XvVSKVXKPVhPJdmVlMHMmvPQSTJ4M8+aFsf+z\nzw5DQm3aJB1hftOmMykEOoMLUFkZvPtuGPZ57bVwicmOHUMC+NWvQk9g992TjrJwqDaRFAKdwQXg\no4/C6p+33w7H9Onhy79vXzjnHHjgAWjfPukoC5d6BlIIdAbnkS1bYPHisNTz3XdhzhyYNQs2b4Y+\nfeD442H48PBvu3ZJR9twKBlIIdAZnGPKy6GkBJYsCcfChWF554IF4drBXbpAr17Qu3fYB3DvveFa\nAVr7n5yvd/06TRs3TToMkTqJfTWRmQ0E7iRscBvj7iMr/bwp8AegD7AGONfdl1fxOnm/msg9rN8v\nKQnHqlWwYkU4li+HDz8Mt9u0ga5d4eCDw67f7t3D0aOHxvpFpGZyYmmpmTUCFgH9gBKgGBji7gsq\ntBkGHOHul5nZucC33H1IFa+VU8nAHTZtgvXrQ8mGdevCsXbtv47Vq+HTT8PxySfh2GMP6NABDjgg\nHJ06hfH9Ll3gwAPDX/kttFxdRCKSK0tLjwMWu/uyTFDjgMHAggptBgM3Zm4/DdwTc0y7NHkyvPhi\n+LL//PN/PzZuhA0bwu3mzWGvvcKx997hL/q2bcN4/T77hBr/++4bbrdvH24X2sXgRaQwxJ0MOgAr\nKtxfSUgQVbZx9zIzW29me7v7uphj26k9WpZT2nkSLZtDu93DX+rNm0OLFsag7oNo3RpatYLddgvt\ny72cSYsm7fA6ZsY3uw/a4fHq2g9S+7xvL5KP4k4GVXVNKo/1VG5jVbQBoKioaPvtVCpFKpWqQ2g7\nd+JJZfxm1eh/f7AUmmxpwo8P2vGXv6y8jNGzRu/weJNGTar8slD7wm4vkqR0Ok06na7x8+KeMzgB\nKHL3gZn7PwO84iSymb2QafO2mTUGPnL3HUqj5dqcgYhIPsh2ziDuUmTFQDcz65JZNTQEeL5Sm4nA\ndzO3zwamxhyTiIhUEuswUWYO4HJgCv9aWvq+md0EFLv7n4ExwKNmthhYS0gYIiJSj1S1VESkgOXK\nMJGIiOQBJQMREVEyEBERJQMREUHJQEREUDIQERGUDEREBCUDERFByUBERFAyEBERlAxyRm1KzuaT\nQv58hfzZQJ+voVAyyBGFfkIW8ucr5M8G+nwNhZKBiIgoGYiISJ6VsE46BhGRfJRNCeu8SQYiIhIf\nDROJiIiSgYiI5GEyMLMfm9kCM5trZrckHU8czOwaMys3s72TjiUqZjbKzN43s9lm9oyZ7Zl0TFEw\ns4GZ83GRmV2XdDxRMrOOZjbVzOZnft+uSDqmqJlZIzObZWbPJx1L1MystZk9lfm9m2dmx1fXPq+S\ngZmlgDOAw939COA3yUYUPTPrCPQHliUdS8SmAL3c/ShgMTAi4XjqzMwaAfcAA4BewFAzOzTZqCK1\nDbja3XsCXwV+VGCfD+BKYH7SQcTkLmCyux8G9Aber65xXiUDYBhwi7tvA3D3NQnHE4c7gJ8mHUTU\n3P0Vdy/P3H0L6JhkPBE5Dljs7svcfSswDhiccEyRcfeP3X125vYmwpdJh2Sjik7mD6/TgYeSjiVq\nZtYK6OvuYwHcfZu7b6zuOfmWDLoDp5jZW2b2mpkdk3RAUTKzM4AV7j436Vhi9j3ghaSDiEAHYEWF\n+yspoC/LiszsQOAo4O1kI4nUl394FeKSyoOBNWY2NjMMNtrMmlf3hCb1FFjWzOxlYL+KDxH+s35B\niHcvdz/BzI4FniR86Lyxi8/3c+C0Sj/LG9V8tuvdfWKmzfXAVnd/IoEQo1bV/0/BfbGYWUvgaeDK\nTA8h75nZN4FP3H12Zvg5r37XstAE+ArwI3efYWZ3Aj8DbqzuCTnF3U/b2c/M7FJgfKZdcWaSta27\nr623AOtoZ5/PzA4HDgTeNTMjDKPMNLPj3P3Tegyx1qr7vwMws+8SuuX/UT8RxW4l0LnC/Y5ASUKx\nxMLMmhASwaPu/lzS8UToJOBMMzsdaA60MrM/uPt3Eo4rKisJowwzMvefBqpd4JBvw0QTgH4AZtYd\n2C2fEkF13P09d2/v7ge7+0GE/8yj8yUR7IqZDQSuBc50981JxxORYqCbmXUxs6bAEKDQVqU8DMx3\n97uSDiRK7v5zd+/s7gcT/t+mFlAiwN0/AVZkvichfG9WO1Gecz2DXRgLPGxmc4HNQMH851XBKayu\n691AU+Dl0PHhLXe/LNmQ6sbdy8zscsJKqUbAGHevdsVGPjGzk4Dzgblm9g7hnPy5u7+YbGSSpSuA\nx81sN+AD4KLqGqschYiI5N0wkYiIxEDJQERElAxERETJQEREUDIQERGUDEREBCUDkVrJlHf+wMz2\nytxvk7nfKenYRGpDyUCkFtx9JXAfMDLz0C3AA+6+YufPEsld2nQmUkuZuj0zCDvjv08oH7It2ahE\naiffylGI5Ax332Zm1wIvAv2VCCSfaZhIpG5OJ1QqPSLpQETqQslApJbM7ChCNcgTgKvNbL9dPEUk\nZykZiNTefYQLvqwERgG3JRyPSK0pGYjUgpn9AFjm7lMzD90P9DCzvgmGJVJrWk0kIiLqGYiIiJKB\niIigZCAiIigZiIgISgYiIoKSgYiIoGQgIiIoGYiICPD/UYWPOl73bPAAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x104195a20>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "x = np.arange(-5.0, 5.0, 0.1)\n",
    "y1 = sigmoid(x)\n",
    "y2 = step_function(x)\n",
    "\n",
    "plt.plot(x, y1, label=\"sigmoid\")\n",
    "plt.plot(x, y2, linestyle=\"--\", label=\"step_function\")\n",
    "plt.xlabel(\"X\") # x축 이름\n",
    "plt.ylabel(\"y\") # y축 이름\n",
    "plt.ylim(-0.1, 1.1)\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "계단함수: 0과 1 하나만 돌려줌.\n",
    "\n",
    "시그모이드 함수: 연속적인 실수가 흐름.\n",
    "\n",
    "공통점: 입력이 작을 때는 출력은 0에 가깝고 커지면 출력이 1에 가까워짐. 출력은 0에서 1사이."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 3.2.6 비선형 함수\n",
    "계단 함수와 시그모이드 함수 모두 비선형 함수\n",
    "\n",
    "선형함수: 출력이 입력의 상수배만큼 변하는 함수. f(x) = ax + b\n",
    "\n",
    "비선형함수: 선형이 아닌 함수.\n",
    "\n",
    "선형함수의 문제는 '은닉층이 없는 네트워크'로 똑같은 기능을 할 수 있음.\n",
    "\n",
    "층을 쌓는 혜택을 얻고 싶다면 활성화 함수로 반드시 비선형함수를 사용해야 함."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 3.2.7 ReLU\n",
    "ReLU(Rectified Linear Unit): 입력이 0을 넘으면 그 입력 그대로 출력. 0 이하면 0을 출력하는 함수."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "def relu(x):\n",
    "    return np.maximum(0, x)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "넘파이 maximum 함수: 두 입력 중 큰 값을 선택해 반환하는 함수\n",
    "\n",
    "ReLU 함수의 그래프"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXMAAAD9CAYAAABOd5eOAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAFXBJREFUeJzt3X2wVfV97/H3F4FOcpM0pr3eMKGGJkwSo51LGzEl8WFP\nMSOmFosR1IFp0hg7NmlgJvW0vfaqx8CkjMQRaktGgkVBi4mEQInUhFvcN+TiQSBXBXzKFCWSFoVy\nrUoNCud3/1gnlpzAeeCss397r/1+zZxxH9l7re8S+Pg9v4e1IqWEJKm1jchdgCRp6AxzSaoAw1yS\nKsAwl6QKMMwlqQIMc0mqgJFlHCQingP+HegG3kgpnVPGcSVJA1NKmFOEeC2l9P9KOp4kaRDKCvOg\nnyGbiHB3kiSdhJRS9PeessbME/DdiNgaEdf0UVBlv2666absNXh9XpvXV72vgSqrM/9YSmlfRPxX\nYENEPJlS+kFJx5Yk9aOUME8p7ev55/6I+DZwDvALYd7Z2fnm61qtRq1WK+P0klQZ9Xqder0+6M/F\nYNr44x4g4q3AiJTSqxHxX4DvATenlL7X631pqOdqZvV6vdL/c6ry9VX52sDra3URQRrAmHkZYf7r\nwLcpxs1HAvemlOYf532VDnNJGg4NC/OBMswltaL77oODB+Hzn89z/oGGuTtAJekEdu2CL34RPvax\n3JX0zzCXpON45RX41KdgwQKYMCF3Nf1zmEWSekkJZsyAd70L7rgjby0DHWYpa525JFXGwoXw7LOw\nYkXuSgbOzlySjrFpE0yfDl1dMG5c7mqcAJWkQdu3D666Cu66qzmCfDAMc0kCjhyBK6+Ea66BKVNy\nVzN4DrNIEtDRATt2wPr1MKKJ2lwnQCVpgFavhlWrYNu25grywbAzl9TWnn4azjuv6MjPPjt3Nb/I\nCVBJ6sehQ8XGoHnzmjPIB8POXFJbSglmzYKRI4vVK9Fv75uHY+aS1IfFi2HnTnj44eYN8sGwM5fU\ndrq6YOpU2LwZxo/PXU3fHDOXpOPYv7+478rXv978QT4YduaS2sbRo3DRRTBxIvzVX+WuZmDszCWp\nlxtvLCY+587NXUn5nACV1BbWrYPly2H79mIFS9VU8JIk6eft3g1XXw1r18Jpp+WuZng4zCKp0l57\nrdgYdMMNMGlS7mqGjxOgkiorpaIj/+lP4d57W3M9uZuGJLW9O++ELVuKr1YM8sGwM5dUSdu3w8UX\nF08O+uAHc1dz8lyaKKltHTwIl19ebNlv5SAfDDtzSZXS3Q2XXAJnnAG33pq7mqGzM5fUlubNg1de\ngfnzc1fSWE6ASqqMBx+EO+4onhg0alTuahqrtM48IkZExA8j4h/KOqYkDdSePfCZz8DKlTBmTO5q\nGq/MYZY5wBMlHk+SBuTw4WLCs6MDzj8/dzV5lBLmETEW+CSwtIzjSdJgzJkDp58OX/pS7kryKWvM\n/DagA/jlko4nSQNy993w0EOwdWv1Nwb1ZcideUT8LvBCSulRIHq+JGnYPf44XHcdrF4N73hH7mry\nKqMz/zgwNSI+CbwFeHtELE8p/UHvN3Z2dr75ularUavVSji9pHb00ktw2WWwaBGceWbuaspTr9ep\n1+uD/lypm4Yi4gLgT1NKU4/za24aklSK7m6YNq0YJ7/99tzVDC9vtCWpsm65BV58Ee6/P3clzcPt\n/JJaysaNMHNmMeE5dmzuaoaf2/klVc7evUWQ33NPewT5YBjmklrC66/DjBkwezZMnpy7mubjMIuk\nljBnDjz7LKxZAyPaqA11AlRSZaxcCd/5TnEDrXYK8sGwM5fU1HbtgloNNmyACRNyV9N4ToBKankv\nvwyf+hQsWNCeQT4YduaSmlJKxYTnqafCkiW5q8nHMXNJLW3hwmLCc8WK3JW0BjtzSU1n0yaYPh26\numDcuNzV5OWYuaSWtG8fXHkl3HWXQT4YhrmkpvHGG3DFFXDNNTBlSu5qWovDLJKaRkcH7NwJDzzg\nevKfcQJUUktZvRpWrXJj0MmyM5eU3TPPwLnnwvr1cPbZuatpLk6ASmoJhw4VG4PmzjXIh8LOXFI2\nKcGsWTBqFCxb1t4PZD4Rx8wlNb3Fi4t7r2zebJAPlZ25pCy6umDqVHj4YXj/+3NX07wcM5fUtPbv\nL+67snSpQV4WO3NJDXX0KFx0EZxzDnzlK7mraX525pKa0o03FhOfX/5y7kqqxQlQSQ2zbh0sXw7b\nt8NI06dU/ueU1BC7d8PVV8PatXDaabmrqR6HWSQNu9deKzYG3XADTJqUu5pqcgJU0rBKqejIf/pT\nuPde15MPlpuGJDWFO++ERx4p1pUb5MPHzlzSsNm+HS6+uHhy0Ac/mLua1tSwzjwifgn4PjC653ir\nUko3D/W4klrbwYNw+eXwta8Z5I1QSmceEW9NKf1HRJwC/B9gdkrpkV7vsTOX2kR3N1xyCZxxBtx6\na+5qWltDNw2llP6j5+UvUXTnprbUxubNg1dfhfnzc1fSPkqZAI2IEcB24P3A36aUtpZxXEmt57vf\nhTvuKJ4YNGpU7mraR1mdeXdK6TeBscBHI+LDZRxXUmv58Y/h05+GlSthzJjc1bSXUpcmppRejog6\nMAV4ovevd3Z2vvm6VqtRq9XKPL2kjA4fLiY8r7sOzj8/dzWtq16vU6/XB/25IU+ARsSvAm+klP49\nIt4CfBeYn1Ja3+t9ToBKFXbttXDgANx/v+vJy9TITUNjgLt7xs1HAN/oHeSSqu3uu+Ghh2DrVoM8\nFzcNSRqSxx6DCy+Eeh3OPDN3NdXj/cwlDbuXXipuoLVokUGem525pJPS3Q3TpsHpp8Ptt+euprq8\n0ZakYXXLLfDii8WEp/IzzCUN2saNxdDK1q0wenTuagSOmUsapJ/8BGbOhHvugbFjc1ejnzHMJQ3Y\n66/D9OkwezZMnpy7Gh3LCVBJAzZnDjz3HHz72zDCVrAhnACVVKr77oMHHihuoGWQNx87c0n9euIJ\nuOAC2LABJkzIXU17cdOQpFK88gpcdhl89asGeTOzM5d0QinBFVfAO98JS5bkrqY9OWYuacgWLoTd\nu+EHP8hdifpjZy7puDZtKu5PvmULjBuXu5r25Zi5pJO2bx9ceWVxa1uDvDUY5pJ+zhtvFOPkn/sc\nTJmSuxoNlMMskn5ORwfs2FGsKT/llNzVyAlQSYP2rW8Vd0Hcvt0gbzV25pIAePppOPdcWL8eJk7M\nXY1+xglQSQN26FDxxKB58wzyVmVnLrW5lGDWLBg1CpYt84HMzcYxc0kDsngx7NoFmzcb5K3Mzlxq\nY11dMHVqEeTjx+euRsfjmLmkPu3fDzNmwNKlBnkV2JlLbejo0WJD0MSJ8JWv5K5GfbEzl3RCN91U\nTHzOnZu7EpXFCVCpzaxbV9xzxY1B1WKYS23kn/8Zrr4a1q6F007LXY3KNORhlogYGxEbI+KJiNgR\nEbPLKExSuV57rdgYdOONMGlS7mpUtiFPgEbEu4F3p5QejYi3AduBS1NKT/V6nxOgUiYpwWc/C4cP\nw733up68lTRs01BKaR+wr+f1qxHxJPAe4Kk+PyipYZYuhUceKR40YZBXU6lLEyNiHFAHzkopvdrr\n1+zMpQy2bYOLLy6eHPShD+WuRoPV8KWJPUMsq4A5vYNcUh4HD8L06fC1rxnkVVfKapaIGEkR5CtS\nSmtP9L7Ozs43X9dqNWq1Whmnl3Qc3d3FDbSmTSue5anWUK/Xqdfrg/5cKcMsEbEcOJBS+lIf73GY\nRWqgL38ZNmyAjRuLOyKqNQ10mKWM1SwfB74P7ABSz9f1KaUHe73PMJca5MEHi/Xk27bBmDG5q9FQ\nNCzMB8owlxpjzx746EeLx7+dd17uajRU3ptFakOHDxfj4x0dBnm7sTOXKuTaa+HAgaIrdz15Nfik\nIanN3H03PPQQbN1qkLcjO3OpAh5/HCZPLsL8rLNyV6MyOWYutYmXXipuoLVokUHezuzMpRbW3Q2X\nXQa/9mtw++25q9FwcMxcagMLFsALL8A3v5m7EuVmmEstauNGWLiwmPAcPTp3NcrNMXOpBe3dCzNn\nwj33wNixuatRMzDMpRbz+uswYwbMnl2sYJHACVCp5cyeDc89B2vWwAjbscpzAlSqoJUrYf364gZa\nBrmOZWcutYhdu6BWK25rO2FC7mrUKG4akirklVeKjUELFhjkOj47c6nJpVRMeJ56KixZkrsaNZpj\n5lJFLFwIzz4LK1bkrkTNzM5camKbNhUPZO7qgnHjclejHBwzl1rcvn1w1VVw110GufpnmEtN6MgR\nuOIK+NznYMqU3NWoFTjMIjWhjg7YuRMeeMD15O3OCVCpRa1eXTz2bft2g1wDZ2cuNZFnnoFzzy06\n8okTc1ejZuAEqNRiDh0qNgbNm2eQa/DszKUmkBLMmgWjRsGyZT6QWf/JMXOphSxeXNx7ZfNmg1wn\nx85cyqyrC6ZOLYJ8/Pjc1ajZOGYutYD9+4v7rixdapBraEoJ84i4MyJeiIjHyzie1A6OHi12eM6a\nVXTm0lCU1ZkvAy4q6VhSW7jxxmLic+7c3JWoCkqZAE0p/SAi3lvGsaR2sG5dcRfEbdvglFNyV6Mq\ncDWL1GC7dxf3XFmzBk47LXc1qgonQKUGeu21YmPQDTfApEm5q1GVNLQz7+zsfPN1rVajVqs18vRS\nVinBF74AZ5xR/FM6nnq9Tr1eH/TnSltnHhHjgHUppd84wa+7zlxtbelSuO022LIF3va23NWoVTR0\nnXlE/D2wGfhARPw4Iv6wjONKVbF9O1x/fXFHRINcw8EdoNIwO3gQPvIRWLAALr88dzVqNQPtzA1z\naRh1d8MllxTj5LfemrsatSK380tNYN48ePVVmD8/dyWqOteZS8PkwQfhjjuKjUGjRuWuRlVnmEvD\nYM8e+Mxn4BvfgDFjclejduAwi1Syw4eLic6ODrjggtzVqF04ASqV7Npr4cCB4qHMPmhCQ+WThqQM\nli+Heh0eecQgV2PZmUsleewxuPDCIszPPDN3NaoKlyZKDfTSS8UNtP76rw1y5WFnLg1RdzdMmwbv\nfW8R5lKZHDOXGuSWW+DFF4sJTykXw1wago0bYdEi2LoVRo/OXY3amWPm0kn6yU9g5ky45x4YOzZ3\nNWp3hrl0El5/HaZPh9mzYfLk3NVIToBKJ2XOHHj22eI5niNsiTSMnACVhsnKlfDAA8UNtAxyNQs7\nc2kQdu2CWg02bIAJE3JXo3bgpiGpZC+/XGwMWrDAIFfzsTOXBiAlmDEDTj0VlizJXY3aiWPmUolu\nu62Y8FyxIncl0vHZmUv92LSpuD/5li0wblzuatRuHDOXSrBvH1x1Fdx1l0Gu5maYSydw5AhccQVc\ncw1cfHHuaqS+OcwinUBHB+zcWawpdz25cnECVBqC1ath1So3Bql12JlLvTzzDJx7LqxfD2efnbsa\ntTsnQKWTcOhQsTFo3jyDXK2llDCPiCkR8VREPBMRf17GMaVGSwn+6I/gIx8pJj2lVjLkMfOIGAH8\nDTAZ+Bdga0SsTSk9NdRjS420eHEx4fnwwxD9/lArNZcyJkDPAX6UUtoDEBH3AZcChrlaRlcX3Hwz\nbN4Mb31r7mqkwStjmOU9wPPHfL+3599JLWH//uK+K1//Oowfn7sa6eSU0Zkf7wfSrMtWfvjD4pFe\n0kAsXFg8/u3SS3NXIp28MsJ8L3D6Md+PpRg7/wWdnZ1vvq7VatRqtRJO/4v+6Z/g+98flkOrgj78\nYZg7N3cVUqFer1Ov1wf9uSGvM4+IU4CnKSZA/xV4BLgqpfRkr/e5zlySBqlhO0BTSkcj4k+A71GM\nwd/ZO8glScPLHaCS1MTcASpJbcQwl6QKMMwlqQIMc0mqAMNckirAMJekCjDMJakCDHNJqgDDXJIq\nwDCXpAowzCWpAgxzSaoAw1ySKsAwl6QKMMwlqQIMc0mqAMNckirAMJekCjDMJakCDHNJqgDDXJIq\nwDCXpAowzCWpAgxzSaoAw1ySKsAwl6QKMMwlqQKGFOYRcXlE7IyIoxHxW2UVJUkanKF25juAacD/\nLqEWSdJJGjmUD6eUngaIiCinHEnSyXDMXJIqoN/OPCI2AP/t2H8FJOAvU0rrhqswSdLA9RvmKaVP\nlHWyzs7ON1/XajVqtVpZh86uXq9X6np6q/L1VfnawOtrNfV6nXq9PujPDWnMvJd+x82PDfOqqdof\nqN6qfH1Vvjbw+lpN70b35ptvHtDnhro08fcj4nngt4HvRMQ/DuV4kqSTM9TVLGuANSXVIkk6SZFS\nasyJIhpzIkmqmJRSv8PYDQtzSdLwcZ25JFWAYS5JFdDwMI+IL0bEUxGxIyLmN/r8jRAR10VEd0S8\nK3ctZYmIWyLiyYh4NCK+FRHvyF1TGSJiSs+fx2ci4s9z11OmiBgbERsj4omev2+zc9dUtogYERE/\njIh/yF1L2SLilyPi/p6/d7si4qN9vb+hYR4RNeD3gLNSSr8BfLWR52+EiBgLXAjsyV1Lyb4HnJlS\nmgD8CPgfmesZsogYAfwNcBFwJnBVRHwob1WlOgJ8KaX0YWAS8IWKXR/AHOCJ3EUMk0XA+pTSGcB/\nB57s682N7sz/GJifUjoCkFI60ODzN8JtQEfuIsqWUvpfKaXunm+7gLE56ynJOcCPUkp7UkpvAPcB\nl2auqTQppX0ppUd7Xr9KEQbvyVtVeXoap08CS3PXUraIeDtwXkppGUBK6UhK6eW+PtPoMP8AcH5E\ndEXEQxFxdoPPP6wi4veA51NKO3LXMsw+C1Rhg9h7gOeP+X4vFQq7Y0XEOGACsCVvJaX6WeNUxSV5\n7wMORMSynmGkJRHxlr4+UOZ2fqDPG3P9z57zvTOl9NsRMRH4Zk/RLaOf67se+ESvX2sZA7mpWkT8\nJfBGSunvM5RYtuP9/lQuGCLibcAqYE5Ph97yIuJ3gRdSSo/2DN+21N+1ARgJ/BbwhZTStohYCPwF\ncFNfHyhVXzfmiohrgdU979vaM0n4Kymlfyu7juFyouuLiLOAccBjPfd3Hwtsj4hzUkovNrDEk9bf\nTdUi4tMUP9b+TmMqGnZ7gdOP+X4s8C+ZahkWETGSIshXpJTW5q6nRB8HpkbEJ4G3AG+PiOUppT/I\nXFdZ9lL8lL+t5/tVQJ8T9I0eZlkDTAaIiA8Ao1opyPuSUtqZUnp3Sul9KaVfp/jN+M1WCfL+RMQU\n4M+AqSmlw7nrKclWYHxEvDciRgNXAlVbFfF3wBMppUW5CylTSun6lNLpKaX3Ufy+baxQkJNSegF4\nvicnocjNPid6S+/M+7EM+LuI2AEcBirzH/84EtX60e92YDSwoefBUl0ppc/nLWloUkpHI+JPKFbq\njADuTCn1uWKglUTEx4GZwI6I+L8UfyavTyk9mLcyDdBs4N6IGAXsBv6wrze7nV+SKsAdoJJUAYa5\nJFWAYS5JFWCYS1IFGOaSVAGGuSRVgGEuSRVgmEtSBfx/3PD2oB5IChoAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x105c87438>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "x = np.arange(-5.0, 5.0, 0.1)\n",
    "y = relu(x)\n",
    "plt.plot(x, y)\n",
    "plt.ylim(-1.1, 5.1) # y축의 범위 지정\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "ReLU 함수 수식\n",
    "\\begin{equation*}\n",
    "h(x) = x ( x > 0) \n",
    "\\end{equation*}\n",
    "\\begin{equation*}\n",
    "h(x) = 0 ( x <= 0)\n",
    "\\end{equation*}"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "## 3.3 다차원 배열의 계산\n",
    "넘파이의 다차원 배열 계산 설명. 신경망 구현."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 3.3.1 다차원 배열\n",
    "N차원으로 나열하는 것. \n",
    "\n",
    "1차원의 예시"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[1 2 3 4]\n",
      "1\n",
      "(4,)\n",
      "4\n"
     ]
    }
   ],
   "source": [
    "import numpy as np\n",
    "A = np.array([1, 2, 3, 4])\n",
    "print(A)\n",
    "print(np.ndim(A))\n",
    "print(A.shape)\n",
    "print(A.shape[0])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "np.ndim(): 배열의 차원 수\n",
    "\n",
    "shape: 배열의 형상\n",
    "\n",
    "2차원 배열 예시"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[[1 2]\n",
      " [3 4]\n",
      " [5 6]]\n",
      "2\n",
      "(3, 2)\n"
     ]
    }
   ],
   "source": [
    "B = np.array([[1, 2], [3, 4], [5, 6]])\n",
    "print(B)\n",
    "print(np.ndim(B))\n",
    "print(B.shape)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "처음 차원은 0번째 차원, 다음 차원은 1번째 차원에 대응\n",
    "\n",
    "행렬: 2차원 배열"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 3.3.2 행렬의 내적(행렬 곱)\n",
    "행렬의 내적 계산방법"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# 그림 3-11!"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "파이썬 구현 코드"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(2, 2)\n",
      "(2, 2)\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "array([[19, 22],\n",
       "       [43, 50]])"
      ]
     },
     "execution_count": 21,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "A = np.array([[1, 2], [3, 4]])\n",
    "print(A.shape)\n",
    "B = np.array([[5, 6], [7, 8]])\n",
    "print(B.shape)\n",
    "np.dot(A, B)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "A와 B는 2 X 2 행렬. \n",
    "\n",
    "np.dot: 배열 2개를 인수로 받아 두 행렬의 내적을 반환.\n",
    "\n",
    "np.dot(A, B)와 np.dot(B, A)는 다른 값이 될 수 있음.\n",
    "\n",
    "2 X 3 행렬과 3 X 2 행렬의 곱 파이썬 구현 코드"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(2, 3)\n",
      "(3, 2)\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "array([[22, 28],\n",
       "       [49, 64]])"
      ]
     },
     "execution_count": 22,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "A = np.array([[1, 2, 3], [4, 5, 6]])\n",
    "print(A.shape)\n",
    "B = np.array([[1, 2], [3, 4], [5, 6]])\n",
    "print(B.shape)\n",
    "np.dot(A, B)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "행렬의 내적을 구할 때\n",
    "\n",
    "행렬 A의 1번째 차원의 원소 수(열 수)와 행렬 B의 0번째 차원의 원소 수(행 수)가 같아야 함.\n",
    "\n",
    "위의 예시는 둘 모두 원소가 3개라 같음.\n",
    "\n",
    "2 X 3 행렬 A와 2 X 2 행렬 C를 곱하면 다음처럼 오류 발생."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(2, 2)\n",
      "(2, 3)\n"
     ]
    },
    {
     "ename": "ValueError",
     "evalue": "shapes (2,3) and (2,2) not aligned: 3 (dim 1) != 2 (dim 0)",
     "output_type": "error",
     "traceback": [
      "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
      "\u001b[0;31mValueError\u001b[0m                                Traceback (most recent call last)",
      "\u001b[0;32m<ipython-input-23-be6ab816c1cc>\u001b[0m in \u001b[0;36m<module>\u001b[0;34m()\u001b[0m\n\u001b[1;32m      2\u001b[0m \u001b[0mprint\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mC\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mshape\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m      3\u001b[0m \u001b[0mprint\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mA\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mshape\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m----> 4\u001b[0;31m \u001b[0mnp\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mdot\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mA\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mC\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m",
      "\u001b[0;31mValueError\u001b[0m: shapes (2,3) and (2,2) not aligned: 3 (dim 1) != 2 (dim 0)"
     ]
    }
   ],
   "source": [
    "C = np.array([[1, 2], [3, 4]])\n",
    "print(C.shape)\n",
    "print(A.shape)\n",
    "np.dot(A, C)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "두 행렬의 대응하는 차원의 원소 수를 일치시켜야 함"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# 그림 3-12!"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "A와 B의 대응하는 차원의 원소 수가 같아야 함.\n",
    "\n",
    "계산 결과인 행렬 C의 형상은 행렬 A의 행수와 행렬 B의 열수가 됨."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "A가 2차원 행렬이고 B가 1차원 배열일 때도 '대응하는 차원의 원소 수를 일치시켜라'는 원칙이 똑같이 적용됨."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# 그림 3-13!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(3, 2)\n",
      "(2,)\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "array([23, 53, 83])"
      ]
     },
     "execution_count": 26,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "A = np.array([[1, 2], [3, 4], [5, 6]])\n",
    "print(A.shape)\n",
    "B = np.array([7, 8])\n",
    "print(B.shape)\n",
    "np.dot(A, B)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 3.3.3 신경망의 내적\n",
    "행렬의 곲으로 신경망의 계산을 수행한다."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# 그림 3-14!"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "X, W, Y의 형상을 주의깊게 확인\n",
    "\n",
    "X와 W의 대응하는 차원의 원소 수가 같아야 함"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(2,)\n",
      "[[1 3 5]\n",
      " [2 4 6]]\n",
      "(2, 3)\n",
      "[ 5 11 17]\n"
     ]
    }
   ],
   "source": [
    "X = np.array([1, 2])\n",
    "print(X.shape)\n",
    "W = np.array([[1, 3, 5], [2, 4, 6]])\n",
    "print(W)\n",
    "print(W.shape)\n",
    "Y = np.dot(X, W)\n",
    "print(Y)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "행렬의 내적으로 한꺼번에 계산해주는 기능은 신경망을 구현할 때 매우 중요"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 3.4 3층 신경망 구현하기\n",
    "넘파이의 다차원 배열을 사용하여 신경망의 순방향 처리"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# 그림 3-15!"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 3.4.1 표기법 설명\n",
    "넘파이의 다차원 배열을 사용하여 신경망의 순방향 처리"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# 그림 3-16!"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<sup>(1)</sup> : 1층의 가중치\n",
    "\n",
    "<sub>12</sub> : 다음 층 번호(1), 앞 층 번호(2)\n",
    "\n",
    "다른 자료에서는 이 순서를 반대로 표기하는 경우도 있음."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 3.4.2 표기법 설명"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# 그림 3-17!"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "a<sub>1</sub><sup>(1)</sup> 의 수식\n",
    "\n",
    "\\begin{equation*}\n",
    "a^{(1)}_{1} = w^{(1)}_{11} + w^{(1)}_{12} + b^{(1)}_{1}\n",
    "\\end{equation*}\n",
    "\n",
    "행렬의 내적을 이용 1층의 '가중치 부분'을 다음 식처럼 간소화\n",
    "\n",
    "\\begin{equation*}\n",
    "A^{(1)} = XW^{(1)} + B^{(1)}\n",
    "\\end{equation*}\n",
    "\n",
    "행렬의 값\n",
    "\n",
    "\\begin{equation*}\n",
    "A^{(1)} = (a^{(1)}_{1} a^{(1)}_{2} a^{(1)}_{3}), X = (x_{1} x_{2}), B^{(1)} = (b^{(1)}_{1} b^{(1)}_{2} b^{(1)}_{3})\n",
    "\\end{equation*}\n",
    "\n",
    "\\begin{equation*}\n",
    "W^{(1)} = \\begin{vmatrix}\n",
    "\\mathbf{w^{(1)}_{11}} & \\mathbf{w^{(1)}_{21}} & \\mathbf{w^{(1)}_{31}} \\\\\n",
    "\\mathbf{w^{(1)}_{12}} & \\mathbf{w^{(1)}_{22}} & \\mathbf{w^{(1)}_{32}} \\\\\n",
    "\\end{vmatrix}\n",
    "\\end{equation*}\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(2, 3)\n",
      "(2,)\n",
      "(2,)\n"
     ]
    }
   ],
   "source": [
    "X = np.array([1.0, 0.5])\n",
    "W1 = np.array([[0.1, 0.3, 0.5], [0.2, 0.4, 0.6]])\n",
    "B1 = np.array([0.1, 0.2, 0.3])\n",
    "\n",
    "print(W1.shape) # (2, 3)\n",
    "print(B.shape)  # (2, 3)\n",
    "print(X.shape)  # (2,)\n",
    "\n",
    "A1 = np.dot(X, W1) + B1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# 그림 3-18!"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "활성화 함수를 시그모이드를 사용.\n",
    "\n",
    "파이썬 구현 코드"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[ 0.3  0.7  1.1]\n",
      "[ 0.57444252  0.66818777  0.75026011]\n"
     ]
    }
   ],
   "source": [
    "Z1 = sigmoid(A1)\n",
    "\n",
    "print(A1) # [0.3, 0.7, 1.1]\n",
    "print(Z1) # [0.57444252, 0.66818777, 0.75026011]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "1층에서 2층으로의 신호 전달"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# 그림 3-19!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(3,)\n",
      "(3, 2)\n",
      "(2,)\n"
     ]
    }
   ],
   "source": [
    "W2 = np.array([[0.1, 0.4], [0.2, 0.5], [0.3, 0.6]])\n",
    "B2 = np.array([0.1, 0.2])\n",
    "\n",
    "print(Z1.shape) # (3,)\n",
    "print(W2.shape) # (3, 2)\n",
    "print(B2.shape) # (2,)\n",
    "\n",
    "A2 = np.dot(Z1, W2) + B2\n",
    "Z2 = sigmoid(A2)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "2층에서 출력층으로 신호 전달. 활성화 함수만 지금까지 은닉층과 다름. (항등 함수)\n",
    "\n",
    "2층에서 출력층으로의 신호 전달"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# 그림 3-20!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "def identity_function(x):\n",
    "    return x\n",
    "\n",
    "W3 = np.array([[0.1, 0.3], [0.2, 0.4]])\n",
    "B3 = np.array([0.1, 0.2])\n",
    "\n",
    "A3 = np.dot(Z2, W3) + B3\n",
    "Y = identity_function(A3) # 혹은 Y = A3"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "항등 함수 identity_function 정의. 입력을 그대로 출력하는 함수.\n",
    "\n",
    "출력층의 활성화 함수를 다음처럼 표시하여 은닉층의 h()와는 다르게 표시.\n",
    "\n",
    "\\begin{equation*}\\sigma()\\end{equation*}\n",
    "\n",
    "출력층의 활성화 함수는 풀고자 하는 문제의 성질에 맞게 정함.\n",
    "\n",
    "회귀 => 항등함수\n",
    "\n",
    "2클래스 분류 => 시그모이드 함수\n",
    "\n",
    "다중 클래스 분류 => 소프트맥스 함수"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 3.4.3 구현 정리\n",
    "가중치는 대문자, 편향, 중간결과는 소문자로 표기함."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[ 0.31682708  0.69627909]\n"
     ]
    }
   ],
   "source": [
    "def init_network():\n",
    "    network = {}\n",
    "    network['W1'] = np.array([[0.1, 0.3, 0.5], [0.2, 0.4, 0.6]])\n",
    "    network['b1'] = np.array([0.1, 0.2, 0.3])\n",
    "    network['W2'] = np.array([[0.1, 0.4], [0.2, 0.5], [0.3, 0.6]])\n",
    "    network['b2'] = np.array([0.1, 0.2])\n",
    "    network['W3'] = np.array([[0.1, 0.3], [0.2, 0.4]])\n",
    "    network['b3'] = np.array([0.1, 0.2])\n",
    "    \n",
    "    return network\n",
    "\n",
    "def forward(network, x):\n",
    "    W1, W2, W3 = network['W1'], network['W2'], network['W3']\n",
    "    b1, b2, b3 = network['b1'], network['b2'], network['b3']\n",
    "    \n",
    "    a1 = np.dot(x, W1) + b1\n",
    "    z1 = sigmoid(a1)\n",
    "    a2 = np.dot(z1, W2) + b2\n",
    "    z2 = sigmoid(a2)\n",
    "    a3 = np.dot(z2, W3) + b3\n",
    "    y = identity_function(a3)\n",
    "    \n",
    "    return y\n",
    "\n",
    "network = init_network()\n",
    "x = np.array([1.0, 0.5])\n",
    "y = forward(network, x)\n",
    "print(y) # [0.31682708 0.69627909]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "init_network() : 가중치와 편향을 초기화하고 딕셔너리 변수인 network에 저장\n",
    "\n",
    "forward() : 입력 신호를 출력으로 변환하는 처리과정 모두 구현\n",
    "\n",
    "forward 의미는 신호가 순방향(입력에서 출력방향)으로 전달됨을 알림\n",
    "\n",
    "역방향(backward, 출력에서 입력 방향) 처리에 대해서도 살펴볼 계획"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 3.5 출력층 설계하기\n",
    "신경망은 분류, 회귀 모두 이용가능\n",
    "\n",
    "일반적으로 회귀에는 항등 함수, 분류에는 소프트맥스 함수를 사용.\n",
    "\n",
    "분류: 데이터가 어느 클래스(class)에 속하느냐는 문제\n",
    "\n",
    "회귀: 입력 데이터에서 (연속적인) 수치를 예측하는 문제\n",
    "\n",
    "<pre>\n",
    "19세기 후반 영국의 우생학자 골턴 경은 사람과 완두콩 등을 대상으로 그 키를 측정. 관찰 결과 키가 큰 부모의 자식은 부모보다 작고 작은 부모의 자식은 부모보다 큰, 즉 평균으로 회귀(regression)하는 경향을 발견. 그 사이에는 선형 관계가 있어 부모의 키로부터 자식의 키를 예측할 수 있고 그 예측 결과값이 연속적인 수치임.\n",
    "</pre>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 3.5.1 항등 함수와 소프트맥스 함수 구현하기\n",
    "항등함수: 입력 그대로 출력"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# 그림 3-21"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "소프트맥스 함수\n",
    "\n",
    "\\begin{equation*}\n",
    "y_{k} = \\frac{exp(a_{k})}{\\sum_{i=1}^{n} exp(a_{i})}\n",
    "\\end{equation*}\n",
    "\n",
    "exp(x)는 지수함수(exponential function)\n",
    "\n",
    "n은 출력층의 뉴런수. yk는 그중 k번째 출력임을 뜻함.\n",
    "\n",
    "소프트맥스의 분자는 입력 신호 ak의 지수 함수.\n",
    "\n",
    "분모는 모든 입력신호의 지수 함수의 합으로 구성.\n",
    "\n",
    "소프트 맥스의 출력은 모든 입력 신호로부터 화살표를 받음."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# 그림 3-22"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "소프트맥스 함수 구현"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[  1.34985881  18.17414537  54.59815003]\n",
      "74.1221542102\n",
      "[ 0.01821127  0.24519181  0.73659691]\n"
     ]
    }
   ],
   "source": [
    "a = np.array([0.3, 2.9, 4.0])\n",
    "\n",
    "exp_a = np.exp(a) # 지수 함수\n",
    "print(exp_a)\n",
    "\n",
    "sum_exp_a = np.sum(exp_a) # 지수 함수의 합\n",
    "print(sum_exp_a)\n",
    "\n",
    "y = exp_a / sum_exp_a\n",
    "print(y)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "소프트맥스 함수 정의"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 43,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "def softmax(a):\n",
    "    exp_a = np.exp(a)\n",
    "    sum_exp_a = np.sum(exp_a)\n",
    "    y = exp_a / sum_exp_a\n",
    "    \n",
    "    return y"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 3.5.2 소프트맥스 함수 구현 시 주의점\n",
    "컴퓨터로 소프트맥스를 계산할 때 오버플로 문제 발생가능\n",
    "\n",
    "오버플로(Overflow) : 표현할 수 있는 수의 범위가 한정되어 너무 큰 값은 표현할 수 없음\n",
    "\n",
    "소프트맥스 함수 개선한 수식\n",
    "\n",
    "\\begin{equation*}\n",
    "y_{k} = \\frac{exp(a_{k})}{\\sum_{i=1}^{n} exp(a_{i})} = \\frac{C exp(a_{k})}{C \\sum_{i=1}^{n} exp(a_{i})}\n",
    "\\end{equation*}\n",
    "\n",
    "\\begin{equation*}\n",
    "= \\frac{exp(a_{k}+logC)}{\\sum_{i=1}^{n} exp(a_{i}+logC)}\n",
    "\\end{equation*}\n",
    "\n",
    "\\begin{equation*}\n",
    "= \\frac{exp(a_{k}+C')}{\\sum_{i=1}^{n} exp(a_{i}+C')}\n",
    "\\end{equation*}\n",
    "\n",
    "첫 번째 번형에서 C라는 임의의 정수를 분자와 분보 양쪽에 곱함\n",
    "\n",
    "C를 지수 함수 exp() 안으로 옮겨 logC로 만듬\n",
    "\n",
    "logC를 C'라는 새로운 기호로 바꿈\n",
    "\n",
    "소프트맥스의 지수 함수를 계산할 어떤 정수를 더해도 (혹은 빼도) 결과는 바뀌지 않음\n",
    "\n",
    "오버플로를 막을 목적으로는 입력 신호 중 최대값을 이용하는 것이 일반적\n",
    "\n",
    "오버플로 막는 예시"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 44,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/Users/donglyeolsin/anaconda/lib/python3.5/site-packages/ipykernel/__main__.py:2: RuntimeWarning: overflow encountered in exp\n",
      "  from ipykernel import kernelapp as app\n",
      "/Users/donglyeolsin/anaconda/lib/python3.5/site-packages/ipykernel/__main__.py:2: RuntimeWarning: invalid value encountered in true_divide\n",
      "  from ipykernel import kernelapp as app\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "array([ nan,  nan,  nan])"
      ]
     },
     "execution_count": 44,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "a = np.array([1010, 1000, 990])\n",
    "np.exp(a) / np.sum(np.exp(a)) # 소프트맥스 함수의 계산\n",
    "# array([ nan,  nan,  nan])   # 제대로 계산되지 않는다."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 45,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([  0, -10, -20])"
      ]
     },
     "execution_count": 45,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "c = np.max(a)\n",
    "a - c"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 46,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([  9.99954600e-01,   4.53978686e-05,   2.06106005e-09])"
      ]
     },
     "execution_count": 46,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "np.exp(a - c) / np.sum(np.exp(a - c))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "소프트맥스 함수 다시 구현"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 47,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "def softmax(a):\n",
    "    c = np.max(a)\n",
    "    exp_a = np.exp(a-c) # 오버플로 대책\n",
    "    sum_exp_a = np.sum(exp_a)\n",
    "    y = exp_a / sum_exp_a\n",
    "    \n",
    "    return y"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 3.5.3 소프트맥스 함수의 특징\n",
    "softmax() 함수를 사용하면 신경망의 출력은 다음과 같이 계산 가능"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 48,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[ 0.01821127  0.24519181  0.73659691]\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "1.0"
      ]
     },
     "execution_count": 48,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "a = np.array([0.3, 2.9, 4.0])\n",
    "y = softmax(a)\n",
    "print(y)\n",
    "np.sum(y)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "소프트맥스 함수 출력의 총합은 1. 출력을 '확률'로 해석할 수 있음.\n",
    "\n",
    "y[0]의 확률 0.018(1.8%), y[1]의 확률 0.245(24.5%), y[2]의 확률 0.737(73.7%)로 해석가능\n",
    "\n",
    "\"2번째 원소의 확률이 가장 높으니, 답은 2번 클래스다\"\n",
    "\n",
    "소프트맥스 함수를 이용함으로써 문제를 확률적(통계적)으로 대응할 수 있게 됨\n",
    "\n",
    "소프트맥스를 적용해도 각 원소의 대소 관계는 변하지 않음\n",
    "\n",
    "이유: 단조 증가함수(a<=b일 때 f(a)<=f(b)가 성립하는 함수)이기 때문\n",
    "\n",
    "추론 단계에서는 출력층의 소프트맥스 함수를 생략하는 것이 일반적\n",
    "\n",
    "학습 시킬 때에는 출력층에서 소프트맥스 함수를 사용"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 3.5.4 출력층의 뉴런 수 정하기\n",
    "분류에서는 분류하고 싶은 클래스 수로 설정하는 것이 일반적\n",
    "\n",
    "이미지를 숫자 0부터 9 중 하나로 분류하는 문제라면 출력층의 뉴런을 10개로 설정"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 49,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# 그림 3-23!"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "출력층 뉴런은 위에서부터 차례로 0, 1, ..., 9에 대응.\n",
    "\n",
    "이 신경망이 선택한 클래스는 y2, 입력 이미지를 숫자 '2'로 판단했음을 의미"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 3.6 손글씨 숫자 인식\n",
    "추론 과정: 신경망의 순전파(forward propagation)\n",
    "\n",
    "신경망 문제 해결 2단계\n",
    "\n",
    "학습단계: 훈련 데이터(학습 데이터)를 사용해 가중치 매개변수를 학습\n",
    "\n",
    "추론단계: 앞서 학습한 매개변수를 사용하여 입력 데이터를 분류"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 3.6.1 MNIST 데이터셋\n",
    "MNIST: 손글씨 숫자 이미지 집합\n",
    "\n",
    "0부터 9까지의 숫자 이미지로 구성\n",
    "\n",
    "훈련 이미지가 60000장, 시험 이미지가 10000장 준비됨\n",
    "\n",
    "훈련 이미지로 모델을 학습. 시험 이미지들을 얼마나 정확하게 분류하는지를 평가함\n",
    "\n",
    "MNIST의 이미지 데이터는 28 X 28 크기의 회색조 이미지, 각 픽셀 값은 0~255. 각 이미지에는 실제 의미하는 숫자가 레이블로 붙어 있음."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 50,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(60000, 784)\n",
      "(60000,)\n",
      "(10000, 784)\n",
      "(10000,)\n"
     ]
    }
   ],
   "source": [
    "import sys, os\n",
    "sys.path.append(os.pardir) # 부모 디렉터리의 파일을 가져올 수 있도록 설정\n",
    "from dataset.mnist import load_mnist\n",
    "\n",
    "# 처음 한 번은 몇 분 정도 걸립니다.\n",
    "(x_train, t_train), (x_test, t_test) = load_mnist(flatten=True, normalize=False)\n",
    "\n",
    "# 각 데이터의 형상 출력\n",
    "print(x_train.shape) # (60000, 784)\n",
    "print(t_train.shape) # (60000,)\n",
    "print(x_test.shape)  # (10000, 784)\n",
    "print(t_test.shape)  # (10000,)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "부모 디렉터리의 파일을 가져오고 dataset/mnist.py의 lost_mnist함수를 import함\n",
    "\n",
    "load_mnist 함수로 데이터셋을 읽음\n",
    "\n",
    "최초 실행시 인터넷에 연결된 상태여야 함\n",
    "\n",
    "두 번째부터는 로컬에 저장된 pickle 파일을 읽기 때문에 순식간에 끝남.\n",
    "\n",
    "load_mnist 함수는 읽은 MNIST 데이터를 \"(훈련 이미지, 훈련 레이블), (시험 이미지, 시험 레이블)\" 형식으로 반환\n",
    "\n",
    "load_mnist 인수\n",
    "\n",
    "normalize 입력 이미지의 픽셀 값을 0.0~1.0 사이의 값으로 정규화할 지 결정\n",
    "\n",
    "flatten 1차원 배열로 할지 여부. False면 1X28X28의 3차원 배열, True면 784개의 원소로 이뤄진 1차원 배열로 저장\n",
    "\n",
    "one_hot_lable 원-핫 인코딩(one-hot encoding). 정답을 뜻하는 원소만 1이고 나머지는 모두 0인 배열.\n",
    "\n",
    "파이썬 pickle: 프로그램 실행 중에 특정 객체를 파일로 저장하는 기능. pickle 파일을 로드하면 실행 당시의 객체를 즉시 복원가능\n",
    "\n",
    "MNIST 이미지를 화면에 불러오기\n",
    "\n",
    "참고주소: http://qiita.com/Tatejimaru137/items/44646c9bb3799768fa81"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 51,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "5\n",
      "(784,)\n",
      "(28, 28)\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAP4AAAD8CAYAAABXXhlaAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzsvbuPZduW5vUb87Ee+xWZeU6ee+6tR18PD2GVgUNJIISB\n1BYIcEDY+A1WqzzAxQO1EEggEEYLLGgwbkttwR/QRWNQdLWq655bNx879l6P+RoYc+2IHXHyPDPP\nIzPXJw2NOVfsiFh7R3xrzDnmeIiqsmLFio8L5qe+gRUrVvz4WIm/YsVHiJX4K1Z8hFiJv2LFR4iV\n+CtWfIRYib9ixUeItyK+iPwbIvLnIvJPROTvvKubWrFixQ8L+b7n+CJigH8C/KvAXwH/F/DvqOqf\nP3rdGiiwYsVPBFWVN113b/Ez/wT4f1T1/wMQkf8B+NvAn3/5pX/3avwb4E/f4tf+0PgN6/29DX7D\nz/f+fsPP997g3d/fn33lV95mqf8HwF9ezf/Zcm3FihU/c7wN8d+0hFiX9StWvAd4m6X+PwP++Gr+\nh9S9/hvwm6tx9xa/8sfAr3/qG/gG/PqnvoFvwK9/6hv4Gvz6p76Bb8Cv3/L7/2KRb8bbOPcs8H9T\nnXv/HPg/gX9XVf/xo9fpwz3+ihUrfhz82bt37qlqFpH/CPgH1C3D33tM+hUrVvw88TZLfVT1fwX+\nhXd0LytWrPiRsEburVjxEWIl/ooVHyFW4q9Y8RFiJf6KFR8hVuKvWPERYiX+ihUfIVbir1jxEWIl\n/ooVHyFW4q9Y8RFiJf6KFR8hVuKvWPERYiX+ihUfIVbir1jxEWIl/ooVHyFW4q9Y8RFiJf6KFR8h\nVuKvWPERYiX+ihUfIVbir1jxEWIl/ooVHyFW4q9Y8RFiJf6KFR8hVuKvWPERYiX+ihUfIVbir1jx\nEWIl/ooVHyFW4q9Y8RHirXrnrXgfId+g38FvMIqYqjGX+XKNx92Z9cFvvnxdUES1aoBlrCpoMRQ1\nlCKLNncaFGMUY8q9SEGWa4KihXtRHswff0pfngsYUCOwyIOxPvyZlMc//919zm+DlfgfNOQNY1nE\nLCJv0G/xG61ifcE0BbvI/biSst6FLiS/J3fVBasFo/e6jjNGCyUbQmwI0RNjQ4iOED0hNYTYICi+\niTRNoPEB3xQaH2iagG8iopkSIQfIiy5XYy33n9BFrj81rEG9RRuDNvZLUlS+9DPv5uHhw+WnxEr8\nDxZvsugXsVSS2zeM35L4pmDajO8zfpNxm4tO+E3GunvSCwVzNRYUqwWnCVcSruSH45LI0TJMG4ap\nZ5gc4yQMUwNjT0o9YqBpBja90m8Smz7TbwKbfqDvRwyRNEAcq9yNB0gJSvnyY/DBI9FYSuvR3lM2\njrLx6MZTNqAbQ1KpP3ORtPzsCJS0En/FD4o3kf0yvxDcLfJ4/HZuHzEF20T8JtEcEs0h0h6gORTa\nQ8E1lfhmIb2hLA+AKk4zPkeaEmlKuB/niC+RNDuOJzieLcdzy9EawJNSzxT2C/Er6Q/7if0+czjM\n7PcD+/0tVmfCLcy3VG0hAHOCMEPhy2uha8E4StOSNy3l0FAOhXKAfDCUgyWqIRxhPnKnoZI+T2/1\n0b5TvBXxReQvgNfUzyuq6p+8i5ta8S7weMF6bbsuRPdXcpm/LfEztnG4baA9CN0z6J4p3bNM/0zw\n3YVElfSWgiHfEd+XSJdn2hxo80y36Mt4HhtevnZ0r1ucLYCQsmeae4zsEaM0TWSzmdgfhGdPC0+f\nBp4+HXj69IjTiekljB1MDiZgzDDNMMo98e1XaIwjNz15m8iHTH4G+ZkhP3PkZ0osyvhScB2YhV0l\nQZre+qN9p3hbi1+AP1XVl+/iZla8S3wV6S//xh5orvRF7Nv9VpOwzYzfCM2hkn7zWWbzmWHzGbSb\nSnqLYinYhfSWjCXTlEifAn2a6PNU9UXyyHTq6JoWZ7dAIWXDNHtOQ48xC/H9xKY/cdgLT59mnj+f\nef78zPPnr/FlZGjh7GAAhgTDBOcTDAYy92uiNwnGk5pE3hTSQcnPDOkzR/rMkz9T5gK2U4yrq6wL\n6eMJ5AMi/uW/acXPCo+deo8deJflvQfaR/K2xI/YRvAbaG+U/llh81lm96vI7ldCt2MhfRV3RXpL\nos2JbQps48QmjWzTwCZWvY0D421YSD+TcmaahdPQ0PjNlcU/sdl4DovF/+x54Je/HPjl50e8njk5\nOAG3CU4TnE7QNdAaSLx5A3QZq2lJTSFtIN4I6ZkjfdaQfpWJv1KmcrH0SklyR3rTfFjEV+B/ExEF\n/ktV/a/ewT2teCd4bOnfRPqGe8J3i7zdv8SF+G6jNIdC9yyx/cyy+5Xh8MdCf9DlDirpL8R3JByJ\nLkd2MbCPM7swsItndvHMftHDqwjckHJgCoXTILy69TS+R8weYwpN09NvPPu94emzzPPngV9+PvBH\nf3SkKbccgWOC4wzHE/SvofXgTXXCXTY+13K5pqYjNkrcGOLBEp81pM864q8y8Y+VYXHe1T29Ek6C\newW2Afl5nOQBb0/8f1lV/1pEngP/u4j8Y1X9R19+2W+uxr9e5EPGV+2vv8Vf/sFJmz70LAmIXBxi\neucku9ay/BC58kmLXhFf85fOr1HQYqqQ3+qdd0Q6ZvpFOqnSy0wvE70J2IXkF7kmfq8TW1NlZycO\nZWJ/Ja0z3PqJ22bitp247WeOm5njdua4C+AKh+3MoZ/ZdxO7ZmbrJ7ZuYmMmGiaSQDKQBLLU5f1F\nJ8DLvTh5OC8dzF1P6BJzp8y9MPeG0nty3yClRTqFVqEBcYpYvf97/qD4i0W+GW9FfFX960X/TkT+\nPvAnwBuI/6dv82veM1x7zt8k8vClPLyEBRoFD+J12Ybr3diaC0kiXhKOjF9I44n1AaBSKa+CQRDl\n7ppmRw4tObaUWPX1vJS3W+q3JbELI7thYnOc6F9MtN2IdxOWCbMPGDJCYqEckFAyhUwpM5oGNE1o\nCmiKaEqQCiTFnAvNMbCZRw56yzP/kmnXkaIDAbWFXzz95zztf8fOvKSJR+Q0EH4fOFOYEwx/DfPv\nIL2EcgsyggvQloXoFhoL3tXx3dxCvhHYWnLbgG1J2jOHLcOwZzg+4Zxbbm8zwzkzjYV5zsSYKbmg\nmuFLAUzvEr/moVH9h1/5yu9NfBHZAEZVTyKyBf514M++78/7cHAhvv8KMfcv+1J0CPUv0irSV+GR\nti7RMtGK0FLogFYSHTMtE56MqGKUR6KIggZLnBrS2BAXSaMn0pByQ35L4jclsQkzm2Fmc5zpu5nW\nzXhmbJox27ic2WdkIb7eSaGUQCkTmkc0z2iOkDNaCmTFzIXmHNjMAweOTE1L2jkw4LpEMcrT3Rc8\n3fyOrXlJE2/hdiQSOU0FF2H8XSV+vCK+DdAUUIHGgffQXKS5n6eDkLaWufWoacmlZ447hmHP8XjD\nObWcbyPDOTGOkTBHYozkHNHL8upngLex+L8A/v6yv3fAf6eq/+Dd3Nb7jGviP3acLc6zrwoLE8Ap\n0imyVdgXZKfIfpFdwTWBRqCnsCWwEWVLYiMzWwYajVhVTNGqVbHlXpfZMJ884dYznxzBeWY8IXvm\n2ZPf0lfrS6YLgX4IdMdI5wItAZ8CdoqYPi0Wf4lnJQOlkp6MakLLjOqMloCWiJYEmqGAyYUmBTZx\n4KC3pKZaetclusNEFmXrX7L1L+6JfxoIc+D8WjEB4gsILxeLfwQGcLEutBBo7UL4Dtq26qatEvaG\neWsZGg+2I5XF4o8Hbo833MaW6XZmPAemcWaeDSlCzqW+h58JvjfxVfX/Bf6ld3gvHwgeE79/JI8+\n8uvwsIX4tAW2itwU5ElBnuiiC7abaKWwIbITywFlL5kDM3sZ6XTClYItSxRcKdg7UcoojK8sU+uY\nnGPEMSXLNDtG40hvSXxXMk1ItEOkcYmWRJMifkq4U0TavFj8soTsVtLrFfmLBgoR1QgagXQXWG+k\n0JjAxowkcwQPrk10dmZnTiRVfLnFl+rI8+EI00gsgXMpyAT5FvIR0u2XLb4xlfhtU4nebqDtF72B\naSsMW4ttPZiWrD0hbDkPe472CbehIdxOzOeReTSEGWIs5JxQ/fl499bIvXeONxF/eyX+G2JrFLqC\n7ApyKMizgvm0IIu43tFIZCMTeww3ojwl8URmnsrARkdczkuIa7nXy7V0FobWMDjLgGVIhmG2nM+W\nwRjiWxLfloIPGTdU34NPGTdl/CljX2WMz3dhutXi13j9SvqqVRJKAql7f5W6DUAU4wtNF9h0A/Tg\nmkTXT+y6M0+6V0QFHQcYB3QYYa7jMETCWNARGEAHYKxaFotvC7gL8T103UL4LXQ7aHdgNkLTL8S3\nLalsmGPd49/qDcfQEm/PpLOpocBzIcZEyWEl/oeNryL+fpHm/mVviq1xirQZtgW5ycgnBfmsIJ9n\nzC8Kdiu0MtHLmb0YngCfSOITCXzKwFbP+JLxOeFyxpeEX7TLmXRUTtZwwnBKwmk2nAah94aTGMJb\nxuqbUrChBueYVLBTwZ7uE3bEXmLz75N06q63josUiimoFNQUVDKYUkXA9IXmJtwv75uJ3e5MODSE\nm4a5KOFVIBAJUyDEQLgNxFeB8LJQzmDCIrFa+rt5qQ68ztVz/a6Frl9If4DuANILrXdY71FbLf4c\ndgy65xhvOM4t+WTIA+SpUEIip0DOdiX+h42vI/7Ncu3qpY8Dwp1Cl5FtrsR/lpHPCuaXGfMHGbuH\nRs5sxLMTyxNRnkniM2Y+k4GDnvA50eS06PhgHl8VjgjHDLezcDzD8bXQNUJjYH7bd19AgiIJZNIH\nKbpitB5JAg+dXLrQHopR1IIaRe0iRpdDEcXsCo1EXJvpdKZ4Q9kZyieG8twwZThROE+F0+vCKRbC\nqRD+JnP660I+gS/3IqVaepvrvBXoLPSLxe820O8q6bsnoJ3QiMWKB2nv9/jxwK3ccJxa9Bb0XNAx\noXNA44QWi/48/HrASvzvCVmiMd6gaTF4RC0GsxzFF0QThnAXxXH5H3isucSuS6555JIx5l56c302\nPt2fkTOxkYleJxpNeI2LTjQaF52IptydYSeBeBGqvJ1Pf3kjl1O67/PtBrI3RG+YvWVygvcGi0GM\n0Igsz1W5jztanq2yrw8c3Qips4TGMlkYBE5ag3ZyWk5Ll5PSRqAxVRdlOUatconekUv4noVJDky6\nYypbJu2ZSsuknkktkxrmUeBW4Cw1+H9ePuD8LeM4fiSsxP/OEDAWxFZ9PRaLweO1xRXBl4DXM64k\nvA748gpRhy7JIGUp2lCWYJoCkApmLsgpY14XpC+YJiNSMCWz3Z7Zyt/Qyd/geIGRV6iciDIyEXCa\nSMuyPuSMLwWfFZ8VV5R0hNNvYfgbmF5BONW01Bx/upTRazqoMUTfMHYeek/qGqbOc+4aXvced7Do\nJ8AzgQOwFbQFrIDCrHCycGrg1MNpD6cncDvDKXNv8XN15vl8tQLI9eiubaF11fq3GdoA7VivDX7P\nX5Vf8IU+5UXZciyeQZVQZore1syf21s4n2EcYZ4h1iPJn5PJX4n/XSEL8Y0H6+/1MjZYfBbaInQ5\n0pVMlwe6LHQiGK2n10W51wWyqbngmhSZCuZckNcZ8UsFmVKQWNj0Axt5RSsv8bxE5DWFW5KMTBIw\nGvEPnHsFlwuuKK5AOsHw+yrjq5qemkYo8cf/v3ycUQAX4nvoe9KuZ972nHc9ftfjtz3m4OFG0BuB\ng6AboFmq4KgQqMk2QwPnDQx7GAIMuRrhsgOXqvj05bG3y9GdXVYDBZoAzVjno+35ojzli/Lky8Qv\ntzCbSvqV+B8apFp468G1YNuql7EALi0x5ymyTZEtia1GtiViKaSF9Em4H5c616TIXJCzIk1BTEGK\nIrEgQ6HrJrZypOMWJ0cMx2rxGZklIJoIC+ltycuRXj3DtwXyCNPLau3vLP7041eHeRy0+ID4TUPq\nN8y7HXKzX2SHudnDvkV3gm4FdoJuBFpBraBqiAqThaU2B9N+SbsFRg/lvHjwQ43Wux7buMTlN0vU\nHsuKINyPJ9PwomwX2XDMjqFcET9QCX+RlfgfCC4W39bjHHx/L66vOeVxoI+JrUQOnDnowI2cOcgZ\np/FuP5102Vtr3QYmgZK0OsXOChfSB0XGgtwqTRPYyJmWM54BkTOFM4lq8VVTLVVV7stXPQzgqVZ+\nPnFXkCItS31+5KX+mwIXVQzJN5SuJ+/25Jun5E+eUJ49IX/ylLLr0E6gM2gnaGvQRlBrUBWyCrOB\nuYGwgbkshTYchA7KAHauYmaw08O5K2ANOFvDd11ZHg65PhwilmPxvM6eY6ky5EIoEyUXiKWS/VpW\n4n8AkMXim8Xi+x6aLfgtNFsMCW8SnQxsiRz0zLPykqf5Fc94iWciaP1nDFTSB1nmWrO6mBdveFEI\nigwKt4psFOcSHROdTDgmDDNFJiITM4FMxCwRe7VenWJKqU5GrfEwd2Wnxvtx/hGX+o+X+NfEL8tS\nP/Q9827P/OQJ4ZNPmZ9/SvjsU9J+gzrzSC7EN2QVooXYQOoXx6WD2EHago5gJjBjFRnvx2YEm5ai\nG5eDhAwm3x0qkFQZMgxZGbJyLlWHPFPyBClXoj+WlfjvO64s/jXxmz20e4SAl4EOYauRm3LmmXnJ\np+YLnstvaTgzU51QM/cSlnnOwKSwHIsxUpN0lmQda/NdQo6XiJBQInUdUTPd7qrTLvH55q5iLWi6\nKgIZq5Sw7PF/5KX+myKX1VSLP/Y9592e4eYp52efMvzic86//Jyw26FL/R6VSx2fpbSHmlrs0kJu\nlow7D6WFvF0ebhPIGcxQtVx0U/PoZa5ElwQmLeEDqV4zCXJOhDwvEu7HaabkeflQ85tlJf57DHlE\nfNdXa9/uobvBMOF5RaewK5FDPvM0veS5/JbP+UtajkzUPeekj7TU4yYKdQlwd75/n557Scs1V8Uq\nC4W0RL5xFxizEErhQQnrR+WkH8tPhcsD4OLcG7sNp92e10+ecPzkOa8/+5zXv/oD5u2Bki2aDJot\n5YGuVl8NaAPqQLtH728GTiC31KPA9v7oTpaqZLIEM8hyLClxuTaDxpmSbynptuqcKWmi5HqdPHNf\nY/sN8jPBSvzvjOuom+UoT9zd4a9oworBi9KayMbM7MzA3t5yY1/R6ZGRu8zbWuFF75eX6RLJ+h1x\nSXl5J+9Q7oWrscBDE83V+OraXY6/Xuf71/njYMXH87hUF8hYAo4Jz5mGW1pe0zPqBs2mEj1aSqxa\ng6FEC+VrzsrfFCn5pqXH5W9wSdAP1AfGRN2X5Rmyq0n9mWrNU1xI/7YhUD8OVuJ/V1wO3nNZ9nPL\nGlCqy06ISIiYlLC5lpRyRmms1owvszj1dHEkLXtvuYSu/cQQAbuEJdiLmPu5PCbMo7GW6qcoqW4f\n7sbLFgO9/5Y3VfZPRXGxYMeIPQbMixlpR3BnlFu0l4X0Bk0GooVlTjJfT3yoJB6A8yKPxyMP92AX\nWXKFKulvIZ+hjFDmJZHoh861f7dYif99UBSy1jO4y4aQSyZZwKRKfFMyTgtOCv6K+LHUyLGwhIua\nJXT05/B/cyG+8zUH3fl6tHWZG8fDEGP7cFxy5UaaaznpNC9z7pfOX1e+OhddknwS5jYg3QRuAAY0\nnSgdyzK/Ev8itZzOtyB+pJL7q2Tm3vP6WBKLc+RczwXLWC18iUvK7c/gD/gtsRL/u0JZrH6pS7y4\nrAc1QQmIRKQkTE7YkmuFHFPwTmlEaSyEvBwT5RpwdqnKJMJP/r9zcWF4v6SlXtJTl3x063lYUMg9\nnOd030wieki2cs1kkFBX0NfPiseSiuJCxg4JexswbkYY0XRG51u0UTRLJf6i70ifaxDP1yJxv2y/\n1ndL+eU18Q2SqEv6Ml7JYvH15+W8+yasxP+uuF7qS+Ge9IuLXOpy35CwmrFcWXy7RIIlmPMVAXT5\nUT8DXCy+95XsXVcz1Pq+atfw5iqUi+S4NKto7ptVSAZCjcP/prr1qSg+FNyYMMeAYULSCNOZcj5R\nfA1z1CJoqTHwWgTNUq39NxF/uZd6lspDi/6A4Nzv8a/nOS1kn+/1utT/SKAL8cn1SZ+XDWyq+Z0i\nEWMSVhJOqsVvRGsySLkq5Mi9U0/MQpCfGNdL/aathN9s78W3fLmaWHM/TqFGzU12eW+ZSqap8vJx\n3XrzaB7vLH7EEpE0I9MI5zP6+hZ1GYosTsOF/OWK9N/EvcJDcr9JLo69/IZxyYuFj8sSP95b/JX4\nHzCUJY1rOR8q14e+ETERsQnjEtZmrC04UbwrNFbxek96p8seX6rwM1nq31n8plr5zRZ2e9jtoNnw\n5j4ci8S5FrO4kF5CPTsvrq7Iha8vRdoUxYWCI2FSwEwTnAdozqhvUbtUslHqA0C5J/y3yXd/WPHr\n4fj6mj7SdzVDlhJaF+F6vBL/A8ZVOh25mumLc09q4Lfxy1JfMs5mnLly7i3HeF6XY7yLf/BnkrH5\npaX+FfEPh1qN5q584HVZ/kXCtJC+LKQfqy8sOYhLcZ/rJhWPXQWxKD5mbE7YOSBmQswI5oyahiLp\nDbnM38LSX3B9eqJvkDe95oF+9A2P5+8JVuJ/L5R74utlYxgAi2og21SPf41hdg1T2zG0W87tgYxh\nioU5FlJUcqjx3RLrMVbJXFXNlyVK7b6fLAqiBcoS0luWeP7L+Fvc/eVM/sE5/TJvGnBeMM6Aq8kv\nxRiSCMFcXqxXLnm98k7WY+7ZCLMIk5E6NsJs67xYKFLwZulZv/SuFylYU3BqsCo1v6AUTM5IvJwN\nzvCt3uGKb8JK/O+M6+iOS+mK+3r5RQLBFYbGcux6Xmz2dH3E9cCmoTdn5hyYUmTOkTnV0ss2Rdoc\nMUWp1fJr24mytJyo1xyaFRMSJkQkprtx1Uuf56+ByHIub+7P6K/nvjE0rQPniDjO2RFnx2lwtDhs\nkLpseaMU4iwMLyzj0TKeLUOwjLnW9xudhVZpfaRbpPURXMT4iPiIUYcEWyUaJJirZAZ+9ESiDxUr\n8b8zLsS/eIIid7XyUYpJBHshfke3OeC2AjtP2m3ZuIFSRoqOlDLVcRkxOtEVcKoEGmYaMi2FhkRD\nWESjYocZO0xVj3UMICl/4+nAhfjOLefzj7TxgrQObEugJaQGmes6XlILk6lVK5qr+lV+CUzwhRSF\n6eirDJ5p9kzJM+GZnMdIYdOPbLuJ3I3QTdhuwncj0glGHWZwmMEig0VGgwwG4ZK++EP9XT8urMT/\nXri2+PLgWhGtFt9X4rsNsPekw5Zp/5RtO2C5xXLCcsJxwuKwCJ6MowAdmZ5Ehy460DPSobPijsMi\nZ/yxFsuSVDBT+MY7F5ZzerdUm1kaRbRN1XhDtI7gWgI9MW8I84aYe8K8oThbAxDcUrrmMl4CE1I2\nhHPLfG6rnltCbplpCa7F+UzYnMi7E2xP2O0ZvztRtoLsCqYYzNEhR4ccLXK8Iv3PqL/8+46V+N8Z\nj5f6l2vVw1uEO4vvuh42DWm3ZToUTk8yu36kM6/o5BW96eiMozfgJdOZmSJKoSOyQdiibMlsCWyZ\n2JKHQvPyFt81qKsrDUm5kt58c2nsi8X3bikh3VbpF12scMYRaQlsOKc957TjPO84sScat+Su5lqy\nxuYH85wtce4fSu6J0hNtT+MTefsa9q+wNx3+xtPdCOVQ4CbULLiXDtNZxC1hPcnUDKafUbfZ9x0r\n8b8zrol/mWfqRxkpYgnOMjQWuoa0sUx7w+nG8uqpZbcdOdgtB9eytw4cNDZjXKC1A2oKkZaJDcKO\nwoHEnpkDI3vyqVC6BnX3lt5OAXcaa/mpb8DdOf1i8fsWNj1suqqTMcTkOeeWmDac8o6X6YaX+YaX\n6YYJvxxdplqy5m5cjzOLOnLeklKVuzFbstvStRE2v8ceOpqnnv6ZkJ4VyrOAPBsxuSCdRZzFYJFk\nkEmQ0+XMc8W7wEr87wXlYaRHzSsDQxFPsB00ntR1TNuO066juelonnbs9xOf+JboLXihaTJbP2P9\nQOc92MJMh2ODsEe5IfGEwBNGnpCP+Yr0+Y70pfHfihhvsvibDnZb2G0giHCeHMwtIVWL/3K+4bfz\nM347P+Ocm1qQXuK9vhqrOAr7O8lX4+L2bNqA3XY0B0/3VNg+L8TnkfJ8RD471RwH55A70hvktJTB\nXS3+O8NK/O+M67zZwsO8VKGYQrANyVumrsds9pj9HnPYY57uOdzMxNZBSyV9O5PbAdPe0rYN4jLN\nl4j/lMAnjDwjv6orjcvy3p1G8qsTpXGofMel/tI0YtNX0h+2MCI0OMgtYe455R0v5if89vyMvxye\nc4xtPaAnPNKxauNRewPuBrU3qFtkubbrZpqNpzsI22eF6dNA/HykfH6L/LKpCwgcJllkspX0rwQa\n+dnEOnwIWIn/vfClCJKrr9il4TNLRJmF4kFb0J5SDL32bHXDji072XKWLaPZMpktxhSmZT8/sbmT\nsVbOJ9uEdR2+acldQ+k9bB1mb3E3tf48qjXMoIAWvQ87KPWWijVkb0iNIfeGtBHi1hAOhlm3jOwZ\n0o5z2HGSLbe64XXa8GruOYaGr467M2CapSrRUodQl56BUscGw1n6Rbp7baoUI4ymYTINwTiiOLIY\nigi6Mv+dYSX+u0ZZ8vSnCMMMx7H2XbbVGpchEpuRsU2cGuV142jbHtfsoX2G2MIrDrxiyys6XuG5\nxTBQmAjoOdH+PlLOudbgd+B2QvOppcPhbxwlKBqUEnUZQwl1XMQQbMPgq0s/9Q3j1nN7aOhvGgbd\n8Fc84Yt8w4t0w3HeMEyeYKVGzSE8zGa5xLleHoJLSGtZClOYcSlUsrz/ORDPZ6bjzKlNHJ3SGost\nLcQeTY7f/VXPyy9aXr9oOB890+BIwXxTiMKK74CV+O8aRSEmmCOc5wekJxfKKRGbkclHTh7axmF9\nD82B5DPGFo5suWXHkY4jjluEM4WRiJkD6RQppwS5YLxW4mPoOoc/O/KglKGQByWPVQNo0up8tA3q\ne1K3Yex7/G6D3/f4Jz2jbvmibPgibXkRthynDcPgCZaF+MpD0j8mvt5nK5alUs1lC6KFMkficGY6\nTpxd4rX0g9yiAAAgAElEQVQBWyyEhjxu0Bx58UXHiy86ji/ahfiWGGpG3op3g5X47xqqNUd/CjA8\nJD0hoX0mupHRJ04OrPPgO7LbM3tBTOFMdyWe00L8iYArgRQjOV1Z/L3QdJbuiaMZHelYSEchHQty\nBCho0iVDzhBcQ/QbpnaPbPbI9oAc9siTPWPZ8CI1vAgNL8eG26GuDu4t/iVX9aIfZLAs9bbyUsHT\n1Tx5WJJbEmVK1eK7mbNkbFEIljw2zKcNmhPHFw2vX1wsvmMaHDEYykr8d4ZvJL6I/D3g3wR+q6r/\n4nLtKfA/An8L+Avg31bV1z/gfb4/KHq/1LdLxMlCesZAaZXoJiYbsQ6wjmx7ZicMtkFMYcQz4plw\ny1gYKUxEGhNILlFcAlswTnGd0FhD5xzN5IkvM6YriKuOQE1QJkWMkMWQbUP2G0p7IPdPybun5P1T\n8s1TJu15HYTjaDgOwvFkGL15ZPHflK96bfGXHIa8vP+7elwBNZk4DEwyY0uCCHk0zKeG4XWPlsz5\n6Dm99lU/sPg/3p/xQ8e3sfj/NfBfAP/t1bX/GPg/VPU/F5G/A/wny7UVF4s/L8E9V6SnmygeosmM\nNoGBbD3BVNLfmlQTXRACwrzogCFQmAnQzKRdpOwy7AqmW/b4O0u3dTTBYTq5Ir1SJiWf6nFY3eN7\n5qYntHvm/inz9jnz4TnhyXPG0jGMmWHInE+Zoc0MPhNtvrL45Svk2uIv71/LctYfoEwUMlEmxjJB\nTORRmU+W4XXLbVe9kNNgGYdq6aezvdrjrxb/XeEbia+q/0hE/tajy38b+FeW8X8D/IaV+BV3Fp+H\npPcOnKVYc5fllo0wi2MwHi+CN1ILTlJIZBKFvJTOvlyz20D8JNZzg06rxd8JzaeG7hNHmxzGVYJU\n0hvS6dKOiyvn3oahO3DePOW8fc6w/yXnm8+ZtGMeJsJpJvQTczsR/Ey4I/5SMfOBvMHiFyrppVr6\n6uCzlFKIpTaZyGNi9srgLd43eC+ghRiqhX8sq8V/d/i+e/zPVPW3AKr61yLy/B3e0/uNslj8C+kv\nqaxGwBiKWCINWTwzDUYcQoORBoMHQAkUIkpAF13IKBF/E0hESpfgSakNfS5e/T90tLn+jHJn6Qv2\nVcE09T6uiX9s97zun/F695zj4XNeP/kjptJSbk/k7S2lP1FaQ/aFYmf0AfG50vDlpf5C+gctxA0l\nKTEUsikEo/WjMRZjBGPcUpJbKAVKEcpSYusyX/Fu8CM5935zNf71Ih8wylKs441wy8LYcB8AtHRy\noFlec7Gi5tE8kUiUp6Ue0xXqA6W1yNZibjymZOQVmB3IRpG2IL6AlXv+LaKmdplVKxRrKK4up4td\nmlBaUHOp/13Qb934/uLou5pefzz5cZLdpdTmirfDXyzyzfi+xP+tiPxCVX8rIp8DX3z9y//0e/6a\nDxGX5fGFRJe03mtr9rjy40MHWv0Jhlzr/JBwRDwBj5CXhGFdAonyXd8dBYwWmhLZ5BFNR2x8QTu3\nbCfHzQhTaQnzWCVMhDQy55FQIqrKw1If70/FmY8Dv+ahUf2HX/nKb0v86z4jAP8L8B8A/xnw7wP/\n83e4u48c10XcEg9JfyHSm0hfiV+/WxYx1E56F+I3GMrynWXxEeRlm1D/hJX4Ac0DNt7Sho7t7Gpj\n1zFV594UGUJkiJEhRWyuBSXT8rvv7/Xxfa94X/BtjvP+e6rJ/kRE/inwd4H/FPifROQ/BP4p8G/9\nkDf54eFxcg88zAF4XAb2scW/kN7eVeqJeOId8ZVIXkhv70p5ARgKjQZsHmjTLTm62iNiSuRhZtSW\n46QcZ+UYFJeArKSiTHpN9gsu11byv0/4Nl79f+8rvvSvveN7+UhwvdS/tpjXqb6XbcCbouPurf6X\nl/p5IX49Acik5crDpb4tAckjko5LQ8gE04SMZ0bteDFZ2mBx0UK0pGyZSk2T5UtL/ZX07yPWyL2f\nBBen3iUg5kJ6u3z9TbWf70l/sfgXq5+WxtmV+JlIXtyAtXbfpWDnZanvS8DnAZ/Ah4QLE34648fX\njNrRzS1ubiG2xNQy5pamtBi9OOHetMxfHwDvE1bi/+i4tvjXpH+8139c3P2+fPOl4m4lvSXiiIul\nN5QHpK9L/aWfPGCpe/w+Qx8TXZzo5zP91NCPnkF73LSFsCWFLVPacsqKV4vR9tE9ro6+9xUr8X8S\nXCz+l/P5v/JsfNH3ITMXi+9IC+kDuhA/Lnbf3S31L065e69+Zp9mdsGwnw37ybAfDYP2MN0Q5wNT\nDJxS4VU2NKVZNgsr2T8ErMT/yfDVOf3f9G05G1I0hNkxj8p0hvOt0L22ZAU9J3RKaIqoOsRYnBds\nJzRJ6XymM5lOoU/Qz9AP0N+CaqAbLO3c4NMGp7mW3/YWaZulrsCli1C5H2vhfWoa+bFjJf57hlKE\nHAxhcIxH4fTC4DuHcR7IbMRgvyi417X/nMseZy12Y3BPhCZBv6s98Iyp3A0zjKf6888qnE6WYXZM\npSGYnthuybsdqgeYW0ipdgpOqfYNvNO6kv89wUr89wxahBQMYYDxaPCdxbq6+C+pMBlD9yrTvQp0\n44zNHucc3cbQPa2depuuNsSUpZ99nGCgdoA+IZwmyzB7xtwy247Ubii6R91C/BAgxqpDWBpeaH0Y\nrMR/L7AS/z2DFkjBMA8Gf6Sm9qK1L/0Es7PshoiOtdlGmz3OWbqNZYfQ5lpX3yzdc0quFj8lmAe4\nRThlw5D9lcXfkN0O3RwgtDBNMM0wmqXApy5PkDWW/n3BSvz3DKUIKQhhMIyuOgRzEuJkmE9C8IaS\nZ2yeaPMI2eOso98Y9m0lfrlUx1q26Sndz28RTsZyNp7JtMymI7kN2ezAHCC1cPbg7SPSx59P588V\n34iV+O8Z6lLfEobaWT4nu5DeMrwyxNZg7URnR3a2AbdY/Nawt9Aue/o41ZbWKVYd5qqPCKfWMrSO\nqW0Ivic2W3K7Q9sD5LaS3sId6VOE+fIgWPE+YCX+e4bLHh8cOTni5JhPDtc4bOPInaHbjOw2A7Fv\n0Y3HN4vF74W2wHhe9vTpoXNvPC3E31kGPKOrFj92G8p2h+4W4pvlKDIvln7ytUvHavHfG6zEf89Q\nChAMJVni5DHGI6ZBjMeYhrIxbJ8MPHnSkqiVdOse37B/UokPi5Eelj3+BMMt3L6CowgnLIPzTH1L\nMNW5l3c79OkBSsv98j7BHKCZwK3Ef5+wEv99gwolC+T74hZVam17HxyD9Qy+ZWg7xk3PmHZMZWKS\nGcQyo4RSCKkQYiGGQpwKaSgkUbQrSMi4mGhyoi+RrQSSmWkM4GdoZuhm2MzovOwT0ow2YckLkLvQ\n4uu5qqB6Xev/Mta1E+6PiJX47x2u03ov4b6XevdQNBGSMISG47TlxTnRNYJzHkxPrwPTMTCdAtMQ\nmadADAFywGmgp5BzQOOAnY/48QX90LFrHDcWgmnQcAt6Qv0tbG9ROaHtLexvyTGTsHfpwo+lZKGE\nggZF7+r+13kJX1fAZMW7xEr89xLXMf4PQ2iLZkIWhthwnDZ0A5X00pP0QK8D+TiQbwfKMJDnkRwH\nyOA10VPQHDBxoJmPdFPH7uwYLYySiM6jaUAZUH+mbAe0PaP7AY0DqRQCLTMNAZgxS8FQy0xDioY8\nFMqQKUOhjIU8CIWCpPKort76EPihsBL/vcN1gcvHxC8U1XuLP4MbPJgNSQ9MObLRAXM+Yk5HzHDE\nTEdMAJMTTqca2Z8DTRzo5yPb0REsBEkEnUitR2WimBFtJko7oaZKkYkowkBmRBkxV+XBHYaGMFvy\nMZOOmbwIZDQB0+Mw5jXj74fCSvz3Eo/z9+8z/kqRavGDx00eZENSZcrKKSpbBprhJe3Y0Q6OdoI2\nJpo00aiphTpyIMeBHBxphCyJrBM5nyi9o3SB0gaKj5QuoG24uzY7xwk4YTjhOVPwyJLL3yCDJb3M\nSJeQu2rAYCYlm0sm4jXhV/L/EFiJ/17iq1N7i1pCcgzRw+RI6piy4xQdrybHloHd3LGbHdsJdnNC\nwkSTT3g1eCLkAHGAGUQS6AT5BOkVmi1ZMqVNFJ/J20TZZ8ohkfeZqW14jeGIo6PDL2VFwZFp0JPD\n3JFe7pt9nAxiLpWCVvL/0FiJ/97hukSXXM2X8tUKIXsIDUl7ptxzih3N3NP4np0OPI2OJxFSTJg0\n0cQT5Aanhp6CzQEbwZq6/Lf5hI0NNjRQLKUpZFVyo+StUp4V8idK/lQZ+o4Njo4WT8Qs95axBBrK\n0SOuPgo0gU6FfCpIU64CgN5E/hXvEivx30tckx+uc/qL1qV+0oYpbzDxgDF7jD1g7J49I1OGlBOm\nTLT5xDa/gtzgVegotDnQxETDRJsNTTQ0wdBOBsGQ90JWIXkhb4X8VMi/EPKvhNMu0dLh2WCWCkMZ\nIeCYaEivGq4t/X3d/4Tc1f/WN+gV7xIr8d9bvKlgB6gWcoGsAsUu3WobMG317IvQsKFlS8+O3uzY\nmj2DPzBxQytgRfGmYEzBaabNkV4LfayNOdJkyKHWBMjZkIohYWofe2PZMDEw0d/JTM9Mx0y0IC5C\nE6HL0Bd0q+ge9EZImKWpBndn/dfj+/f7sCrRw/GKb8JK/A8OSwaORGAGHQEHxYABlUC0gdnB2Xpu\n7ZbGPsW6ABYmuWFbwiKRUAKpBEoJoAEbC2lS0klJr5TUKclpLeedYdgmJmYiE4Uz0GCxNEBHJp9b\n7O8L9lxwqWBdwe4K7tOCQ0g3lhJqzYEShRLMInWs5eLbKI/04+adK74OK/E/OCiQQBfi47jrUlMK\nxSSSm5kaZWgbjs0O2wZoILcNE7cc4sAcB2IYyHFE44AEsDHhYiGOEE9KapXotFb1zYYYCkOfmQlE\nRgoesFjAU+iJlLnDnqRKFqwX/E7wCL4zxLMhDZY8WNJgSeMyxlKSXQJ8LmXH45W+vPeV+N8GK/E/\nOFwsYgAc6KU1VW3goRSinZlb5bzx2M0WesibhrDZMXFiHo+E8Ugaj5TpCIBJCVcmfIQ4KvFWCFaJ\nUojZEOdCPAtjl5gIJEYKFkGwZBoiHRNaOlx02ORxyeGcw+89rnP4J4YwWuLREY+eeHTI0QOekhwy\n+aXq4Ly8P7voy/tOP+Ln/H5jJf4Hh4X4GoFp8Y0tXXs0UASSi0ytYjce9lvyviHst4z7yKQnwukl\n6dRRnEMAmxJ+nmjVkCOEEaJTgtT+oGEuhLMQj8LUpDuLX6MMMrVN6EzHgJge51qs63C2xbkO3yne\nGmYn+Mkyv/TYrkFcAzRoashTg5hmeX++vjcePtS+3OxjxVdhJf4HB73f4yvckyIAE4ohWmFuBDa+\nkv6JMD4VTk9g1DOp7dAr0rsw0ZgTXTG1CfAEQZSwVO8JgxKOwtwLwWVmZiLUMFwilhnPQE+DaXrs\nbovbbfC7bSX9TvA7j98Kc7DYzmFc9QpoaslTRzy1YLrl/VxtX+72+JGV+N8eK/E/OFwsPiClHpYT\nQJf+9Hiia6Bt6/J+3+KftrhPWvynDaMOd6Q3KeHmiXY40ZuGbTGUAAFlTkKYlfmszI0QfNXRpGUH\nXpYufg6Lo8FhcNhtj/vkQCTiOsU7g995mk8L4RPBJbsUDm0oqSVPPfHUY5seMf3y/h5b+jc1Hl3x\ndViJ/8Hhyuutl+XvXV9sCi3J7iltQ9h4zH6LPNljPt0jv9gzluHLpPev2JiGvQoaYc4wzcpsWOQy\nloXwBSWiCxktgsHgEPzNBkcidop/Yoje43cd8dNM84dg8+IKTA156kinHvdqi2k2iGy5j1+4Jv0M\nX2rvteLrsBL/g8TXe7cLHeVCIDEgy1m/dIiBjWzYmC0bs6OXHZ3Z08mBTm7ohNqUUwuxVElSKFJA\natsOo/muzL5ZKm7fzwscGuRJg4wtZm4xqcOWHssEtqU4hzYO7Tzae3QbYZ/gJhFQVMtS0l8oxaDF\nocVTSvOG9/+mM/8VK/E/NpRSPXJThPMMxxEaB7YunwuB+CIwvYbz2XM7b2nTU6wE8LBpb1AbwAaw\nEUzA2iqtCSiFvJTZL4u+HlMUCQkzBOxxQl8MlM6jlzBekyi/H5HzgE0D3p3pdlu2n27YsSXcCDHM\npDATYyAFJQVLDB0pCKVcNxq9losvYCU/rMT/+FC0lsy6EP+K9ORSG3C9npleK+dzQzPvsLkemWXf\nsOUW7wecG/B+wLtxmYP3tYhfDJBmiGEpzBPqgrwUkFLuiG+OI6ZzuIX0kgq4GU4b7GmDzwOt3xB2\nJwIbQvf/t/d2IZZt233fb8w511wfe9eu7jrn9rkc3VzdkDwbQT4eIkFkEoIJAQWDEyNjbMcIP4jE\nYD/Y0YvA+CH2g0Ax6CGKbCQTkziGRPKL7YQgBRscy46UyEiOA+FEUW7u7Xtvddf+WB/zMw9z7apd\n1dXndB+d093qWn8YzLlW7ao996b+a4w5/3OO0eEOmrFPTH0q7ZAY+7LYF4MtKcHw3Nb4j7Zs8Dli\nIf5DQ84QYsmV198mPS6QJREOE+MhczhU6GkFEaJYJrNmrfZ09ZbWbunqLdpuURaaOtDZEUkwDUXy\nm4aSux8KH6Nn9vgR6R1qO6JNqRAsIaFGj9Qj2vdUoaMJLcG0xLOW0LTERy3jYOm3msNW028Vh21J\nORaDRUbNzR6Gaba7q/8L4BWILyI/B/x7wLdzzr9vvveTwI8BT+eX/UTO+e98aaNc8MUh5ZtQX4/l\n3kx6hqLz+8kzTZnDVMG0IkaLkxVD5ellz3nzjNg06NbQNKCbQNOMrFuFjiVb71iVwh1QPH08qm0n\noT7XpI+kMSD7Cd0OZNOSzYGkW7JpyE1L1g3ZtIxjw+5Zg23KXgBoiEEzjQ1KNRSPPlAW+xad/2V4\nFY//14C/AvzCnfs/lXP+qS9+SAu+VOQT4kMh/VRIz34kK4UPwhgFYlVIH4VBhH0Fgz6Q2ga1MjQd\nxC6gu5G623PWFeLrU9LHcrzfmZKEV649vi+0DBE1etJ+Qj23sKqRdYNaN8i6QZpjv0ZWDaNbYZs1\n2pxRiolY3GgY9g2izuYPeR/p9Zv7jn8P4DOJn3P++yLy/ff8aHl8/l5EmkP9kRtPbxxUBipNUhVB\nLCM1USxOagZqKqmpjGWse1RnqFdwtg6k9Yhe72lWlvVaUcXbpA++bPjRpggI+HmOD+QQkTGQ9xPK\nlpV8tanQH9QYGkxTo02NWdeYD2v0Bw1jWKNN8d6F9IlhrzG2QanyMHgxvD9u713+ZY/43czxf1xE\n/ijwj4E/m3O++oLGtODLxDHUP5JeSXHFSkApkq7x5oxYWZypUNUKMWeo6gxlzpjqnrqFs1VgOhtJ\nZ3vU5jnNxnK2KdV44WZO70eYDjPxT0J9CYk8Ht9XyKrU4dOPDJYa21jsoxpbWep1jf2wxn7N0sdz\nbki/ot9nmueayjaInHHj2U9JX7EQ/zY+L/F/BvgLOecsIn8R+CngT7785b980v/GbAveGtKnpLHW\nkGxDsgks8yEfA8qCadCS2UvHXjoOrOilY5AVg6wZZU1WickIUwVjDVML00qYJpgmcCaXcD+lYjGV\nKCAFJCU0GrMJVI8cdnDUk6UOE3Wy1FhQ0OmWVdWxtit6e2CoB4Z2ZOwmdDDk5Ek5zBZJKZFyegCZ\nuz+Z7bPxuYifc/7OyeXPAn/703/jhz/P2yx4G8jHipoe4gRxgHCM0yGpCWdGehXZorikoUlrTHgM\nbsLmNYc97IOwV8K+EfYbYa9h3wphSGjv0M7ftCd9SSXPfuoTcZsIl/EkRx8EGeE7B6rnW7rBcp40\nqQK9itSPHYfK4MKACz0+lraYx4X3nfzf4LZT/ZWXvvJViX/c91kuRL6ac/7WfPkHgX/6WuNb8A4j\nl902R+KHkwMxOZGUx+mRXiLbrGhSjfFnME2EIWHVSO+F3it6EfpG6LUwdIr+XEhToOoHqmG2uS89\n6DCX7HV5Jn4kNILM/6U5ZIIekecHqitLN2pSAmUi9cqxvug51BW98/TO0U+OwTn6yQOOEN934r86\nXkXO+xsUl/2BiPw28JPA7xeRH6BMpD4B/tSXOMYFbxJ5Pt2XHCQD4WShLAUSEScDfY5so8KEGqY1\nYSwbaSrjGJViFMWoFFOjGFu5vsfkqHd76t2Oercj6TInP54NIOVSZWcm/mkK7jQmktEwHKgGTTeA\nTpG6cpyteia149BatkOaLbMdSt6ukBKjT0uZrhmvsqr/o/fc/mtfwlgWvBPIJ6H+rPPnQnq0I+WE\nyyN9jBivwNWEYc3Ya/a2wTQRV2t8o3CNxtUK1yj83Fd+on32jNjUN0d/Y6SaxjKdSJRyWn1CTCzp\nOkOeE3NGsAJRU6Xye3VyrM1AWu2JzRWHzvJsr2gqhVGlrmCIitErlCwLfEcsO/cW3MEJ8aGQXoUS\nAcSxVNeNgT5EmBRhbBgrw960PK8CuoO40QStCa0iNpqw0YTz0lZ+INb1rfP+1TiR9oeyzS8ciV9O\nGRbSJ+I+op4rVF0UgqqKaDOhTI+udijToE3DwdU0lcXoksQjJMvoLfvRokRY9PyChfgLbiPPJbDh\nxtOLAzWf51cKFwScEJRm1Ia9brBasFqh1oqsNak1ZKVJjSGda/JXDOlDjY2HEq4zh/fjhN0fiLYq\nkt5xcQ/IQeZiGwmsIFZRtQm9ClSdo1kN1KuKxlTUK0O9quhDg9Ed0BJix+g79mOHNaDEsBC/YCH+\ngjs4evyTba7HfPeiSGic1GWTjxiU2Nnq0roKWgPnBhEDjUE2Br5ikO8zNHE3e/pINU3U+wPhqiVV\ntmj6c6ifQ7rJrqWkZAhWAiuhfixUjxWdUqxbYV0pzlaK9WOhTw2wIcQNo9+wnyLPD2CNQVTzFr/X\ndwsL8Rfcg/kQ/cnlTdcQUUQ0NwUvDMwJtKksajDIZBBvUNEgyaByeRBkFWikwSmLl4oohoQmiyAI\nMj93cszXb5256UcPWc+lAhqoJqgDNAk6AdREq1SJAkxTpgQWlNVIbSHV8wLmS+yBYCH+gtfEaa2+\n47HXk0WzFMleI4OBrSZfGqg1yRgUmpz25G8d4DsDPBth55AhIC6VjT3zu9ytn3OdVmOefYQJ/ADj\nHqoKTCkbUCTEnWYYKqZQ42gJ1ZrUnZHDOVT1fGoo3m8PhPwL8Rd8DpwS/zTX3bwHwClyr2GnodHz\nKbySFz/nPfk7B/J3Bng2wc7DEBAXUakk67qbP+cUOZeEHsHNR39t+fNqHlYvisOgGfuKydd4WqJZ\nkdozyOdga/AeQijt0aA8EBbiL1hwH44e/+5R1znXX/LgNPQKdops5pNyQcOkyPlAvjzAs+LxZeuQ\n3iM+ITnfEtvynb8O89rjicefZtLLnFF80ELvNYOvGK89/orIGdmcg6u52T/sThIGpPIweCBYiL/g\nc+CY0POUlnMUkHTx+IPAthTZzEHBqMgHRcoH8u5A3g6wm65DfeUikgqJT2sBH//6deg/R+lHj38k\nPUVtZKiEPmsGKiaKxw/VimQ25PYcfA3DAMa8SHp5OBr/QvwFr4nTOf7x+hj2a0iK7BTSz7Xug4JR\n4CBwpcj05L6HvodhhN5BHxCfUPMc/0j+++b7tzw+N54+jhB66CuhrwyjqZhMjataglmRqtnjh/p+\n0ju3EH/BgpfjtET3kfTqxlLR+DOCBMrR24OQrSCVFOK7A9kNJSGfc+WY7ry4d0r6+wpkXxOfuWzA\n7OmDAW+gr4W+1QxtxdTW+GoO9dszaM8h1veTXuuF+AsWfDqOSSvvBuVFh8dLqekxca3BFx0eEiM5\nHchpgDQiyZcjuTFee/xPs5zmbL1xJr0CI+UttYKhEQ5nc6hvalw7h/rtGXnzCI4puE9JP47lYbAQ\nf8HDxsmmnVsbeAQkI0dTxzbdXDMv0OW5n2dLGYmZLo+s854u99RpxOYJkz0qf3YizOvFvsytiX8G\nssymmYvpSok+jnb6WV5mDwgL8RfcgYDokj9L9At90RldRbSJcxtKe7wnCZ0jOiV0iqiTvs6RJo6c\n+R1n4cBZGFiHiSZ4TEjgc8kRwotlMI4hv5REQRgF1T2trjO9jTTaU+eJKoyYqUcd9gjbsoFnt4fD\nvMYwuSLnPSANHxbiL7gLkUJ0Vd02XVqpMrrxVLWjanyx2mGbTNUEjIpUMczmqWLARE+VynXtR9qp\np5l62mmknRzNFDBEiJkUbwuGd+vfCCWkN7qkCbQGrIZ6bpXNdDbS6kCdHTYMmPGAYg9hJv7hSPyh\nyHo+LMRf8NBx9PgV6PoFkyaju5FqNdGsRuqVUHeJehWoV1DrRB0CNriSMiu4uV+uq2mi6gdMX5Jw\nGD1R4TExlgQc3BD/vo081x7fgK2gqaCe26YCVWV2JtJqT83s8enRcY9M27K4N8ykv+XxH87mHViI\nv+Au5A7xTXvLpMmYdYXdaOoNtJtMuwm0G0V7lmlNovWe1k80fqT1I60br/tmmIp+bydEz0UvogcX\nyZLvDfFv0XFexCuVe6CuobXFuhqUzqwk0Yi/9vg69CjZlaKboYZpnDfxzO0S6i9YMIf6+oT41era\npM3otaY6F+rHmfZxZPXYsX6sWD2GlYl0LrByjm6aWLmBlevpXM9q6tGHiWA9XnsCnhA9wXvCEAkq\n3+vpb4X68/CMLh6/ttA2sGqga0CpTBcjbfTUafb4sUfFPRK3hfjezebndgn1Fzx03OfxqxVUZ2DP\nUE1Cr6E6T9QXgfZDx/pDw9mHirMPM2c2cTYG1pPjbBxZTz1n44H1tOds3CO7iVFHRiJDjIwuMo6R\nwUSC3BAfbhP+OtTnRY/fNNC1sO5AS6abIq0L1HGe408HtGvBNWXn3rGS5y1biL/gQeNkcc+cEN+e\nQX2OtAm9ylTnkfrC0X1lZPWR4ewj4dFXYWMj50NgMzrOh5HN0HM+7tkMO87HLTQTOzL7mNm5zG7M\ncMj4KsMd4t87ujtz/KPH77pCfEWmI9LEO6v6/R4ZbNmrn9NLbCH+gt/TuLPt5ZYmn1EqI5IQlVGz\nBjYejEcAACAASURBVK/UrMdrDVVAKg/GQ+WRyoFxiHI0kjhTE2cysZaJlZpYqZFOTbT3tiOdjHRq\noJORrBy+bO6jkvIPeDzf9yoV7I/E1xq0Lc+mqgHbQr2CmDM2JSofMTqg8ajkED+VjTr+ZasID4f0\nsBD/PcScV+6owd/pK50x1mMqT1UFTOUxVaCypdUqIihEEoqAMCH0KHZIeI6dEqvDltV2y8pesdJb\nurzFhivUtCXbHWnc46ceN46Mo8NMHjUmZMrkLewvob+CcVcO2vipRNt3Ha7c01clERBiQKqSkEMa\noANW3EgCgZMiOrmYPEyS34eF+O8d5jm6VLdNlVaqRNWO1M1E3Y40baRuM3UTqNuRSimUT6jgUX5C\n+x4VdijfokJLNUXqw57a7qn1npo9ddhTT3t0vwdzILqe4HrcNDI6h5rKefvsMnlfSN8/h2EHU18O\n3CR/m/hyTyvcLO5dE78GGqDlhvhH0k8ZxjyHFafEh4fs7WEh/nuIE+KrujDjpFVVxDSG+kxYrSPd\n2tGdZbp1YLWesJLQg0cPE7rv0YMtFi06WExO5b7uiz4eevTUY/oetevJeiD5keBHnBtR86p59pHo\nMnkonn7cF3P9ix7/BbKfXCtmj69ve3w59fhH0g9ANXt7dZfkdwn/sMi/EP99wzGF9NEdqvaWKRup\nWqFZR7pHjrNHirNHmc2jwNmjiQaP2U6YrS6mNCZqzGgwUSMhwmEk5wnCCNNI7kfYTuRnI+iJFBwh\nTEzBQXBk74kh4UOGqXj5qS+kP3r8GLhV7OJlh3Tm2p7XHp/7PP5ESdRZ5xLqm/wSj3/aPiwsxH/v\ncCLHqZn4egVqBXqFVAHTRuozx+rRwOZDxaMPEo8+8Dz6cKLLI9X3oLLz4lsQqrH0qwD4SMieEBx+\n8oTeEWpPqB2hdqA8MQZ89BADOR6vI1MEfPHwfn5uHPt35/j3kp5ZbVRztu/52fbCHH/I5WFgKR7f\n5PmXX5bU6+GRfyH+e4fTOf7R469An4E+Q1WeqnU064HuUcXZB4pHH2U++CjwwUcj63TA2oRVkSpE\n7Jiw+0glERsTaYpMMTBOkdEERhOLLm8CyUQSkZQjISVyisQU8dcHdTLEWTb3NxJ68uWM/d1Q/9i/\nJj3z4p7mVqhPA3Lq8XvgkGeP/7JQHx4i4Y9YiP/e4c4c/+jx9Rnoc8R6TDtSnx0K8T9UPP4o8+H3\nBZ58PHGWemrlsdFRj45677HWXd+LU2Q/ZQ4k9pI5kFGSSWQ8pU05k8nEu0dz5zO0R8n8vvbkU7xI\n+qOdruofPf6R+GnuX3t8XrK497CxEP+dxN0lrTvLXCqXBS5VQtjTVkQhSko4fK3XB5TyiDjWyrHR\nExs1caYm1qpo8Z0ai+6eR6xM1EzUecImR50mbCwHboKPhAwhg0tlDU1nUGnmFZ+9Tn59BH7+eKcS\nXbmpyg0USRTp5Dp3YJqMthllyv4DyOScySHTp5p9sPSxYkwGlzQhKVK+L5/Pw8VC/HcOR9+m72+1\noKqEsgll83UrttzTojEpYdKESfuT/gGTnrPC8dh/l0fDdznffYfu2SVVdQVyIISRKU7kbznSU0+8\nDIRtxB8Sdsq4mIllox19Lutnbn4IfNaOu+tPJ2XzzfUmnLlVcz8rTVAVXlmCWMLcelX6phGGTeTQ\nRPY6souRboisriIrIkO0fPPyjKfPV1zuWrZ9TT9VuKBI6WEl2/g0LMR/5zCvymNmq076BlGCqiO6\njeguobs4W+lXItQuUbuptN5h3Z7aVdSuos0Ta/+M9XDJeveMrnpGxRUS9oRxYEoT6Tue8NQTvhfx\nV5HqkHBTpipH5ulzWT+bclHO5kxYr058BaYq5+mv23nvfdKKQddk3RJUR9Ado+4YVMegO8Qq2taz\nbz2t9rTR0w6eFk/nPGMwPH3e8fR5x+WuYdtb+snggp69/gJYiP8O4pT49gUTJSgb0F2g2gTMJmI2\nAbMp17Uk2j7RDY62d3Q9dAO0Al2EJo/Ufks9bKl3W6xsqeIWpj1+PyDJES8D5lkgPAu4bcAcImZK\nmJhJc0m7496Yz+Px1TGJRl322lt704ZKkY0lmI7RbAh6w2A27GZLWtHoiVo7GjVRh4lmnKj9RH2Y\ncE643LXXduPxF+Kf4jOJLyJfA34B+Crlwf6zOef/XEQeA/8N8P3AJ8B/kHO++hLH+kBwDPUrruvR\n0XAtWCtBWY9eeczGU1147IXHXiiqC0UrnvXWc7YLrLeedRU4E886BtbOU6cR5Q+o4YCSQzmrPh6Q\n/YHQjuQ0EbYRvZttG9F9Qk8ZHSHNZHfctJ4SCbxOqF/NB2yaplg7t94qQmUZqxVUG0J1wWgu2FcX\nPK8u8KKxcaBKAzbO5sdyLw4El9n2NVeHmm1f3xDf6yXUP8GrePwA/Jmc86+LyBr4JyLy94A/AfyP\nOee/LCJ/DvhPgT//JY71AeHU4x+XrFugmz2+w3QOs3HYC0f9RFM/Udgnwkoi55eJ82cT59XAuRo4\njwPnbuC876nSSPIjcRiJYSKNI3E/Eu1IsCM+enSfUENC9QnVR/SQUFNGzSdXfZ7zWeZC+jt7bz4V\nR+IbUzz88WTd0VytGKzF2A7shmAvGOwTdvYJz+wTxmQwwwEzHqjGA8aV/vFeHBP9VBUbKw5zf/H4\nt/GZxM85fwv41tzfi8hvAV8DfgT4N+eX/TzwyyzE/wJw9PhH4tfcaFUrRCmUndDdRHVusBeF9M3H\n0HycWePZNIlH1cSFHLiIWy6mLRf9lsd6R+UGRu+ZomccA5N4RuVJ4vEqEGNEuYT4jLhc+m7uh1zS\nW3NTUmOW5l97jn/rLH0H6xWs1zC2il1tMXUH9Tm+vmCsn7CvP+ZZ/TF9qFDbHZodyu3QcYcaduht\ng9pa0hBwXuOCfqFdiH+D15rji8g3gB8A/iHwUc7521AeDiLylS98dA8Sxzn+aajfAetiSqFshe4M\nZqOoLgT7pJC+/XpmxcimSjxWjg/Tng/dc77SX/Lh9nt8RV+i88A+JHYpsY+ZfUzEmJhSJsSEixlJ\nQLq/zfn+He+vqpBfh/rmJtTvukL6zQaqTvGssVRNB82G0FwwNk/YNR/zrP06u8kiXCHuCjlcQeyQ\noUGuLPJdQ+4dKQkpy73tgoJXJv4c5v8t4E/Pnv81RNFfPul/Y7b3GZ+iwSP37Ec9uYdFUaGyRqEo\nEn1C5YDCUxlozESrJxrt5naiUROtmmgZadVAKwdaDnTsaNnS5Su6/ByVBnyAyYMJoDyIp5ShCiXn\n5Gd+utNU9Ecdfv4IJb990d7TPW22YLuMbTO2zlibmQxMOjNJZpKGSRrGo6mW4WjS0mPnShpz+izn\nYHSlFNfBlWKdDxafzPbZeCXii4ihkP6v55x/cb79bRH5KOf8bRH5KvD05X/hh19pMO8H7urwd7R4\nEdAyT+PnVt+0WjRVqrBZsClgc49NHpsP2GSpbKKSCRsmqmGi2k7Y705U1UTFiGWPevoMnm6J393j\nng2M24lDH6inhPKwD9BHGGPZhOPzvDj3Co/yax1ezzr8rL8fr5PSuFmHd2KJyuLmvlMWYwTXRMY6\n0teJg4rsUuRqiqxJjH7NN8cVT+uay1qzrRN97XB1T6q34Cr43g6eH2A3QD+Vp1iYVx4fNL7Bbaf6\nKy995at6/L8K/GbO+adP7v0S8MeBvwT8MeAX7/m9B4i7Ovwdk7nmU3Vi9qavBOqUaSN0MdAlTxeh\nS5kuQmUDWhwqOFTvUFuHrhwKh/KOKvfo710hl1vS9w745wPjznHoI5XLiC+k7wMMEaYIIb3+BhxT\nlXn6LT2+gmgUva7JqsXpjqg7JtXR62KiFKP2DCpwUJ6d9nQxsJo8nQ9MY8tTu+KptVxaxdZmeutw\ntifZLThTSP/8APs7xH9AqbN+t3gVOe8HgT8C/IaI/Brl/+MnKIT/myLyHwG/DfyhL3Ogv3dwd3Hu\nOFef+2KKh68EGoFaQX3sC0olbHB0wbGJE5vg2ER33drKk8WTgycPnnzlyy5576H32Dygrw7wfE+8\n2uOvCvGrIaCmjITi6cdTj59eT45Tsxx3rcPXZaHO1jc6vDcdMuvwk9lwMBu2ZkNC06eJQ3Q0aaI9\ntr60Tmouq47LquayUmyrRF95XNWTzBaCLp7+aAvxPxdeZVX/H1Bc2H34t7/Y4bwvOHr8ihsN/qjD\nmxIb25n0nUCroFPQCkp5rN+zCp6ND1yEgQu/Lxb2WDURJBBCIAyBQCD4QOgD4Sqi04TeD7AfSPse\ntx8Y9xOqDySXUR6mVMzF0oY51H8V3NLh51X5poW2La23Cl9ZhmqFVBtidcFYXXCoLriqLvDJUE8D\ndhyppxE7DdR+xE4j9TQQkmFrVlwZy9ZotibRa4czPcmYUna7n27b5MEvof7rYNm594XjZXLcbFLN\nepaCRhXSrxWsSqvUiPWezvWF+K7niXvOE/+MJ+4Sy8AkkSlEpiEy+cTUR6aryGQipIAeJhgn4uDw\n48Q4OPIYCFMJ9X0uXj7M7bXHf405vpk9fttCt7qxqVH01lLZDrEbor1gsk842Cdc2SeMwVDte8y+\np6Kn8j1V6jHTgWrfEz3ztKCm14qDyoX4uicpSgHMycPoS3vsLx7/tbAQ/wvH3S23p2dGu3KO9Fjh\nsdbF068VnGnYKJQ+YKeezgnnU+Bi6nnirvh4+g4f629j454+J/qQ6H0u/Zw45IzkRIoR7QL4QPQB\n7wPZl6hg8gmZ99ufWprtc+28m4m/Ppt1+O5Gh5f6nFhfMDVFh7+qP+bgKnS1R7NH+z1q2KPTHj21\n6P2eNEacquYFQY1TCaccTjJJhTmRZizm77QL8V8ZC/G/cNzdcnu6AWddiK81WAWNhlYXb7/R8Eij\ntMJOz+lGxWYMXNieJ+NzPtbf4evq/6V2W7Yus/OwmzI7B2beYJMduJBROUHKpJRxKRHmMtVqZnfm\nth5/V5v/1E93N9Q/If5mA9VK0bZFh5dmQ2wK8Q/Nxzxvv85utAhbxG+RfoeoLRK3yNggO1t0eJFi\nCEkSCUcST2Iog0hzeHJfu+CVsBD/pbh7Jv7YljPgSiWUzK3KKEnXZ8PJFaSqtEdLtrS6HD0tpsiq\naGNZabIoGploZKSRgUZ6Wg60sqeVLa1cUectPpbqT24u/2ZHqEYwA6STMvNHMt+V5l+qw8usw59q\n73d1+JqiwTfMOjxMJjOpzKRgku7aRmkZpWNUHaNqGaVlkBoIkENJuxP9/GGOobucjPz0EyxJNL5I\nLMR/Aad5X140rRJV5bHWY6swmy9mPQoDwZWEcmEAf4CwKyWaw4psLFFpYlYkr4iTJvaKqBUJzVrt\nWE3fpJ6eot0ledrip57BOXZTYppgP5X9KqMHF0qkG9NrnofXoM2NFv+CDi+zDi8nOrxYTCW4NjE2\nqejwktiFxNWYig7vzvjmsOFp3XLZVGxr6GuPq4eiw08WvruDZ3vY9nCYK9aGMKfgWQj+JrAQ/14c\nF+f0C6ZUKmWb2kjXJLrW07UjXTPStSNGZK7C2sN0gKmFqSuta0mqwosiJIUPCj8pglZ4FCEqWnVg\n5Z5Su5n4fot3M/F9opo3qfUOhgBTmPeuvGIFKJFC+Fs6vL2jw6tZh1cdUXVMuqNXxUQrRl1q3R10\nYKciXYyshkDn46zDb3hqWy5txdZCbz3ODrMOX8GzAzzrYTsU4o9z4cp0t3Le62wGXvA6WIj/Ak53\n3p0mwiitUgFbRbrWsVlnNuvAZj2yOTuwWR+wpFJ7va+hb2CYW11y28dcMSnBZcUUFG4SJhQuKiYn\n1DKw8pfU4RLtZ+KHnsEX4ptQPP3RXPxd6PDNrL83d3R4bfG6Q/Ssw+sNB71hqzckMfTJc8ieJnva\nFGiip/WeJgecWC6rDZdVy2Vl2FbQVwFXDaRqB14Xwm/HEx3+rseHJTHml4uF+C/glPgVtzfg2KKz\nW8eqUWzWmYtHnotHExePDlycX9FIhH0FOwt7W1pti4yXLCEaBiWMWRiDMKAYozA6YRgFg2MVttTh\nCh235DATPzp2oRB/CmXHnTvx+PEVt97JnOqqsvPiXDfr8F3pe6vwxjKYFWI2RF3Owx/MBVfmAp8q\nauewk6N2E9Y56sldtyFrtmbNlenYmoqtgV57nBlIRhUd/jDr7wdX+qMvHv8F4p+2C75ILMS/F6ce\n/7gyX0wpja16ulaxOUtcPAo8+WDkyYcHnnywpRMHVwYaA9aUuFoqSAacwTlFL0KfhT4IfRR6JxyU\n0Isg2dOlnjr26NiT0wEfC/ElJfSsXvl0o2T5+PqJMMzs8dtu1uDXxaZG0RtLVXVlA465YKqecKie\ncFU9YQyWaj9iDhPVfqLyI1WcMMNIdZiIAXpt6VVNrysOGnodcKon6QjxRIcfPYzziaFwX6jPPf0F\nXwQW4r+Aux7/VJJrZuJXdK3ifPb4Tz4c+fijAx9/tGUtQ5HorAKji4tNGryCUTMlYZ+FfeKmTUI7\n91OO2OSw2aGTIyeHz44hOWJOqFS8e8xzm2Yd/jXn+Ncevy2EX2+Kja1iZy2m6hB7TqzKBpy9/Zir\n6mMOrkZXA5oB7UdUP6DDgB4H9HYguYgThVOzCTjlcSqSZJozeaSbJ9bp0yvfDVsWwn9ZWIj/Ak53\n3p16/HIuXinBWjt7/Hzt8T/+6MDXP75iI4eyHVfP6kAU8AKjwAFGL2w9bDPswtwP0HiwAXzM8zHc\nIqblnPAkYk5MORVB8a4Wn2/6n/npTnX4poT3R+JvHs06vLVU9cnOu/oJB/sxz+uvsxsahB7xPdL3\niOqReECGHtn1sw4fSaS5jSQJ1304Pql4UYNPy0Lem8L7SfyjQH0qVl/3Z8392m5fQ4SkIKu51Set\n5lwmNjLnpJeRtSq2kpta8Pcew6f0FWWrbIgluvW+HCs/tvqe8/CJ+1Nb3U0rcTyJntX8RqqI9Hlu\nUUAj5DWklRBbIdYQKsEbmJQwyepEh581eDnV4VtgJmnMNwn2XS7ZN0dFScjludHfw2zHewveNt4/\n4h+XrV9iWicq7bC3zF/3VVbgRwgN+B58M/dbCA1nMvFRfMrj6ZL1YYu92pPrkcl49mSyAJcndgXs\nKWWdZsl678ux2DHMq/L51efocE8ejxNDC7lSZKvJVs1Wph7ZalKtcI3QN4rcKLwRhqTYDYoWoR83\nfNOe89R2XFYVWytlVf4ox40TfHeA5/Oq/GEo8qWfSoKM6yx8x6Tbp9r84s3fFbx/xGcmvp5FalPd\n9HWFqiJ11dNW0FWBrkp0laOrerqqx6QMY11smtuxue63TFzESx5Pl6wOV1RXBzAjDs8+JIIAzymE\nP7Y7CvGnkjCmDyfn4dN8Hv4VD8nMn/BWWanTFiXkWpNbQ+oMubtp6Qy50rh5p6BXmkEpqqSpRk3l\nNL1Z87Q656mZiV9RjsWaWY6bxkL6ZxPsxrI6P45zkftT4h8z8R3jlYX07xLeP+IfPb6poKpfMGUD\ntoauDmzqkU2d2NSOTd2zqbfYFOFQwcEW62c5jgqCxYpnHa9YTVesDlusOZAZccGXnXVQiL6f22N/\nJr6fF7Kvz8MfdfhX3Xl3Ynfz/CgAJSSrSJ1BNpa0qWBjyXM/mQofDCEYhmAQb5BgEGeQoOnpuDTn\nXJqOS1OxNUJvQiG+3oI3sHWwm0rbuxLGeFfyd5Vk29wm/uLt3zW8/8SvW7Cz1S2q8dg2sGpHNq3i\nok1ctI6Ltuei3dJEB1sD2wpsVZbAcwXBwFShCdi4x44HKr2n4gBhxI2evM8lcUE/23DSn0P96Obz\n8PHkPHx69fPwcH+RraOhIFlNXBnSpiJeNHBRky5quGjIyuL7ijhb6Cuiq4hjuR5iw1avuNIdW23Z\nHuU4PZD0UYcPcCiJPzj4EznuNNn20U7Jv+BdwftJfD2H+lVdCN+srk21E3Y10q33bFaKi1XiyWri\nyarnyWpLFwZoDdh58zoaogGnYdCzBDWWkJeBHEfyNDIdAlM9u+3p5ZbCzRn4kF7/PDy86PHNSYsS\nolVIZ4gbCxc1PGnJT1rkSUuSBv/cMj6zTFgmZxmTZRot07ZmmCy9quiVLTq8EnoVcGogqaMOH2/C\nlmluQywHb26R/u4Bm4X87wreP+Jzj8dvVtCeQXeGWo3YswPdmeX8THFxlniycXx81vPx2Za1PxQN\nXs/Bc1AwqZK91ShiSLjocZO/afXRSlHJ67Wt40L2yXW+ex4+8Xrn4bl9hOjo6Y8bi1GCWA2dgXNL\nvmjIT1rUxyvSxx2JFmcbBmr2ruawr9mnhsNYc7iqGXqDU4KT2RQ48TgJJDWVgd59aoV5vpLvknzZ\nb/+u4v0j/t1Q/+jxuzPozlFriz1/Tndu2ZwrLs4TT84dH5/3fP3Rlo3b3mjwgXJMdJAyVzfCJJlD\nzOxDYj8lcs5MOeFI7HPCnTq5l7QvnIN/jfPw8CLxT8troqQ8uLp5bn9RF2//8Qr5+pqcOhwNvWvZ\n7Rqem5ar1PB8bLjatgw7RSIU7f2owZ9cXxezv9bfud1fdt79nsB7SHwQXXbJSg3SglqBrDNylmnP\nMu1Zol0F2jbQNZ7OOrqq1Ijv1Et0+LlVuUT7JoGa16/SvBFtmuVs4SbP/HUa/eM2AgVZpMh+pwfh\nBUTJHYrcVeqZJXQhJkESSBJIUircJAEsgRWejkBHkGJeGoI0DKphoOWQG/apZRcarnzL1dTwbGyZ\nBrjZEXB8YkVK6OJYdPj3A+8d8UVljAlo6zDtgF5rzEbQ5wl9Hni0Glg3z2mqHYYDeRrwOAYf2O0z\neQK+A3yXIsdtuV6YYz4cs0/Qp5tqsT7flJAS5jzzak6Xr25fo4VkFLFSJKOJRpVrU66Tuk32fIf8\nKQrZaaJTOK8Qd2PKKVKyBNcS+5awbQmXLbFpCaYmYBiyYvtNYf8U+svEuE34PhBdICc3f4Lj6vxd\nLX7B+4L3jvhKMtpEbD1hW41dCXaTqR4H7GPHpus5k+c0ssXkPUwjwTuGQ2AnmTgCzyibb55RQvwD\nZXEulE1q/WxDKuWiw0nke13/XUOl5/R6+uaaSgi1ItSGUBv83EqtCXXJwHub7KWf5372ithrUm/I\nvSYNhtRrEoYcDDFVRGeJvSVuLbGpicYSsYRgmLJm9/RI/My0jfg+Epwnp2OIc58WvxD/fcJ7R3yR\nhDEBax1NKzTrTLMJtI8mmg8HzuqBtX9O47eYcAA34INj8JGdT/iB4uWv5vbo8Wfi+zjvTE3FXvD4\nUrx7pcvhvNrcbqnBdRrXGlxXoTuL6yroLKmtyNUxk7nMEcTtNk+asK3w2+qmpcKHijBWhGRIzpSH\nw1YTjSGhicGQRs2UFf2l0F8W4o/bhOsD0ZXpQiH+kfSnq/TLAt37hPeP+HOob63QtJnVKtBtJlaP\nK7oPKtbVwPrwjKbfYfwBpoHQO8ZDYHfITD3Fwx9tP7cj4Msq/DR7epdvPP5Rh5+jeSpdyN5Uxdq5\npRPGtWI6M+i1RZ3V5LOatK4JZzXJ6muSF/LLnOmveOPUa8KzGtdYRlMXSS5YxrFmUnYmPqReSEYK\nbYOQRiHtwaMYt8J4BeM2z6F+PCE+3K6Hu3j89xHvH/Elo3XA1om2DazWivVGcfZYc/aBYqUHWp7T\n+i06H8jTiN9NDM8CPM+YPYXkd232+GneW+/zTX344161TFmvO4b61hTCdxa6urR0QrVR6HODPKrg\nvCY9aojnLepRg6qra9If7XgtCHlfEZqayTQMNBxCQz829PuGXjV4r0gukftEIpFDIo+JtE/k56ko\njL3genB9xh8irofoMjkdmZ9eYgv53xe8d8RXkjAmYU9yvp9t4NGjzPkH0MpI5Z9THbaYXEL9sHMM\n34uEpxm15UZ/P13jmtscZz+Yb9prRYubxb1K3Xj8roZ1DesGWAn6TCOPDVxY0kVN+KDFXXToi47Y\nVrzk+A0ZIW0twbRMtPShYz+27PYtO9uxUy0uKbKby2oFTx497D3ZerINJBLBCdFBdIkwF54NLpOv\nU/Te1d8X0r9veO+IL+q4uJdo2shqHTnbJM4fRx5/EGnzCIcdUu0gl1Df7xzhe4Hx/8tlbn/8X0/3\n9O/R3k/tuLh3Herb4unXDWxaYCXIRsEjQ/qgInylxn+lYXrSob6yRrqKmyM3ch3iX18/t3hWTGFF\nP3bs9yuunq94bjuuZMWYBNxEDmM5PKMmUCNZjUVKJJBn+S+n4uVTSvO946LiosW/73jviE8GyRlS\nRFKAGJAYkRCQECCPJOfIUyCNidQL+aBJe0PaWtjdo52/3tuX7Hxyk6lvkptdu2Dn+u91qQWvGkY1\n14RXDZOy5JnwR0sn/VFZRjXr8arlIC0HaThIy15apixzKHK6V/64qff6KM+dEZ+2Cx4C3jvi5wTB\nCa4Xxq2iv9RUTUaZslpuc0X+Zk16GsmXibTN5F7ITpGSocT1nx+WIvmNcynqg4edgiuBNUBVcdh2\nHKqOg+o40HIIDQdnOQwG1+oXiH9q41ax/Sbsn2b6y8i0DfjeE91ETkdFYKJstlm0+AX3470jfkql\nnoXrFcM2Y05In4Jgcyqkf5rIl5m8laKHO0NOlt8t8atZ6htiOcS2c6Ug7groMqANQ9XSq4Yhtwyx\noZ9qhsEy7Ay+MfO04ZTwN6v6016xfyrsn6aZ+B7flw09Nzr8fcRf5LgFN/hM4ovI14BfAL5K+Q/6\nL3LOf0VEfhL4MeDp/NKfyDn/nS9tpK+InCCeeHxtyj97CoIfI1WuyJcz6S+FvNXk3pBdRU415SN+\nfpgMfYZDhCZAK3PZzAxNAtCMqmbKNWOsGaeaqa8Z9xXjlSFYDXfIfvMgKCvy/SUMl5nhMjFuA673\nd+S4uyuTR4+/YEHBq3j8APyZnPOvi8ga+Cci8j/MP/upnPNPfXnDe33kJNehvjYCaGIQwihMe0Ej\nsM3kq5n0W0PuK3A1OR2Fuc8PDdQJbIR6nufXGWyCOgBJ47LFhQo3WVxf4fYWd1XhuopYlYW971sd\nXQAABa9JREFUG+LfbsOkGLcwXWXG7THUlxM57rgB59SWUH/BbXwm8XPO3wK+Nff3IvJbwPfNP/7d\nrYR9CTh6fN+XEzExZPxMevtcoRFyL9DPnv5QQV+T3VzE8XfpGVWGKoGJczmO0+sAREWIBu8MYTCE\nvSE0N1t3k1YnOv7JVt3570cv+KMG30fcoVwHl07kuLsZcBbSL7iN15rji8g3gB8A/hfgh4AfF5E/\nCvxj4M/mnK++6AG+Lo5z/CPpw5iZ9hljFcZmBAVOk50pKXBcIrtUVuTScS78+SG5ZMrVgM6gEuhw\nc1gHJ8RJkypFNPr6wM6xn5XcIj5w6zpFmXX3RHRCdIHo0jzH9/Pr7kuGsZB/wQ1emfhzmP+3gD89\ne/6fAf5CzjmLyF8Efgr4k1/SOF8Zx1X9FEBGQVRG5tNxouA6NfQxt3vKt+0LgKTyAJB0+0huOaYr\n5OPRXCXXLcJM+k8PonI+1eDjrMHzGTr8srC34DZeifgiYiik/+s5518EyDl/5+QlPwv87Zf/hV8+\n6X9jti8JWcruuuup+luajbwRri2EXnCKT2b7bLyqx/+rwG/mnH/6eENEvjrP/wH+IPBPX/7rP/yK\nb7NgwYLPj29w26n+yktf+Spy3g8CfwT4DRH5NYqL+QngR0XkBygB8yfAn/q8w12wYMGbxaus6v8D\n5szNd/DWNfsFCxZ8PtzduL1gwYIHgIX4CxY8QCzEX7DgAWIh/oIFDxAL8RcseIBYiL9gwQPEQvwF\nCx4gFuIvWPAAsRB/wYIHiIX4CxY8QCzEX7DgAWIh/oIFDxBvgfifvPm3fC188rYH8Bn45G0P4DPw\nydsewKfgk7c9gM/AJ2/snRbiv4BP3vYAPgOfvO0BfAY+edsD+BR88rYH8Bn45I290xLqL1jwALEQ\nf8GCBwjJ+cvN2SYiS1K4BQveEnLO9yad/NKJv2DBgncPS6i/YMEDxEL8BQseIN4Y8UXkD4jIPxOR\nfy4if+5Nve+rQkQ+EZH/TUR+TUT+0Tswnp8TkW+LyP9+cu+xiPw9Efk/ROTvisj5Oza+nxSR3xGR\n/3W2P/AWx/c1EfmfROQ3ReQ3ROQ/me+/E9/hPeP7j+f7b+Q7fCNzfBFRwD8H/i3gm8CvAn845/zP\nvvQ3f0WIyP8F/Cs552dveywAIvJDwB74hZzz75vv/SXgeznnvzw/PB/nnP/8OzS+nwR270IhVRH5\nKvDV02KvwI8Af4J34Dv8lPH9h7yB7/BNefx/Hfg/c87/d87ZA/815UO+SyhVNt8R5Jz/PnD3IfQj\nwM/P/Z8H/v03OqgTvGR88I4UUs05fyvn/Otzfw/8FvA13pHv8CXje2PFaN/UP/r3Af/PyfXvcPMh\n3xVk4O+KyK+KyI+97cG8BE9yzt+G6yrGX3nL47kPPy4ivy4i/+XbnIqc4qTY6z8EPnrXvsM7xWjh\nDXyHb4r49z3B3jUd8d/IOf+rwL9L+eJ/6G0P6Pcgfgb4l3LOP0Aprf4uhPy3ir3yjv3f3TO+N/Id\nvini/w7w9ZPrr1Hm+u8MjnUA52Kg/x1levKu4dsi8hFczxGfvuXx3ELO+Tv5ZtHoZ4F/7W2O575i\nr7xD3+HLitG+ie/wTRH/V4F/WUS+X0Qs8IeBX3pD7/2ZEJFufvIiIivg3+FTi4C+MQi3o6VfAv74\n3P9jwC/e/YU3jFvjm4l0xGcUUn0jeKHYK+/Wd3hvMdqTn39p3+Eb27k3yxI/TXnY/FzO+T97I2/8\nChCRf5Hi5TOlnuB/9bbHJyJ/g1Jm+APg28BPAv898N8C/wLw28Afyjk/f4fG9/spc9XrQqrH+fRb\nGN8PAv8z8Bvc1BP/CeAfAX+Tt/wdfsr4fpQ38B0uW3YXLHiAeGfkqwULFrw5LMRfsOABYiH+ggUP\nEAvxFyx4gFiIv2DBA8RC/AULHiAW4i9Y8ACxEH/BggeI/x/VjAudSrz/AgAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x105c8bf28>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "%matplotlib inline\n",
    "import sys, os\n",
    "sys.path.append(os.pardir)\n",
    "import numpy as np\n",
    "from dataset.mnist import load_mnist\n",
    "from PIL import Image\n",
    "\n",
    "def img_show(img):\n",
    "    pil_img = Image.fromarray(np.uint8(img))\n",
    "    #pil_img.show()\n",
    "    plt.imshow(np.array(pil_img))\n",
    "    \n",
    "(x_train, t_train), (x_test, t_test) = load_mnist(flatten=True, normalize=False)\n",
    "\n",
    "img = x_train[0]\n",
    "label = t_train[0]\n",
    "\n",
    "print(label) # 5\n",
    "\n",
    "print(img.shape)          # (784,)\n",
    "img = img.reshape(28, 28) # 원래 이미지의 모양으로 변형\n",
    "print(img.shape)          # (28, 28)\n",
    "\n",
    "img_show(img)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "reshape() 메서드: 원하는 형상을 인수로 지정하면 넘파이 배열의 형상을 변환가능"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 3.6.2 신경망의 추론 처리\n",
    "입력층 뉴런을 784개, 출력층 뉴런을 10개로 구성\n",
    "\n",
    "첫 번째 은닉층은 50개의 뉴런, 두 번째 은닉층은 100개의 뉴런을 배치"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 52,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "import pickle\n",
    "\n",
    "def get_data():\n",
    "    (x_train, t_train), (x_test, t_test) = load_mnist(normalize=True, flatten=True, one_hot_label = False)\n",
    "    return x_test, t_test\n",
    "\n",
    "def init_network():\n",
    "    with open(\"sample_weight.pkl\", 'rb') as f:\n",
    "        network = pickle.load(f)\n",
    "    \n",
    "    return network\n",
    "\n",
    "def predict(network, x):\n",
    "    W1, W2, W3 = network['W1'], network['W2'], network['W3']\n",
    "    b1, b2, b3 = network['b1'], network['b2'], network['b3']\n",
    "    \n",
    "    a1 = np.dot(x, W1) + b1\n",
    "    z1 = sigmoid(a1)\n",
    "    a2 = np.dot(z1, W2) + b2\n",
    "    z2 = sigmoid(a2)\n",
    "    a3 = np.dot(z2, W3) + b3\n",
    "    y = softmax(a3)\n",
    "    \n",
    "    return y"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "init_network(): pickle 파일인 sample_weight.pkl에 저장된 '학습된 가중치 매개변수'를 읽음\n",
    "\n",
    "정확도(accuracy, 분류가 얼마나 올바른가) 평가"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 53,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Accuracy:0.9352\n"
     ]
    }
   ],
   "source": [
    "x, t = get_data()\n",
    "network = init_network()\n",
    "\n",
    "accuracy_cnt = 0\n",
    "for i in range(len(x)):\n",
    "    y = predict(network, x[i])\n",
    "    p = np.argmax(y) # 확률이 가장 높은 원소의 인덱스를 얻음\n",
    "    if p == t[i]:\n",
    "        accuracy_cnt += 1\n",
    "\n",
    "print(\"Accuracy:\" + str(float(accuracy_cnt) / len(x)))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "predict() : 각 레이블의 확률을 넘파이 배열로 반환\n",
    "    \n",
    "np.argmax() : 가장 큰(확률이 가장 높은) 원소의 인덱스를 구함 => 예측 결과\n",
    "\n",
    "신경망이 예측한 답변과 정답 레이블을 비교하여 맞힌 숫자(accuracy_cnt)를 세고, 전체 이미지 숫자로 나눠 정확도 계산\n",
    "\n",
    "normalize를 True로 설정. 각 픽셀의 값을 0.0~1.0 범위로 변환.\n",
    "\n",
    "정규화(normalize) : 데이터를 특정 범위로 변환하는 처리\n",
    "\n",
    "전처리(pre-processing) : 신경망의 입력 데이터에 특정 변환을 가하는 것\n",
    "\n",
    "전처리, 정규화 예시\n",
    "\n",
    "데이터 전체 평균과 표준편차를 이용하여 데이터들이 0을 중심으로 분포하도록 이동시킴\n",
    "\n",
    "데이터의 확산 범위를 제한하는 정규화를 수행\n",
    "\n",
    "전체 데이터를 균일하게 분포시키는 데이터 백색화(whitening)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 3.6.3 배치 처리\n",
    "각 층의 가충치 형상을 출력"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 54,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(10000, 784)"
      ]
     },
     "execution_count": 54,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "x, _ = get_data()\n",
    "network = init_network()\n",
    "W1, W2, W3 = network['W1'], network['W2'], network['W3']\n",
    "\n",
    "x.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 55,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(784,)"
      ]
     },
     "execution_count": 55,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "x[0].shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 56,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(784, 50)"
      ]
     },
     "execution_count": 56,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "W1.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 57,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(50, 100)"
      ]
     },
     "execution_count": 57,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "W2.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 58,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(100, 10)"
      ]
     },
     "execution_count": 58,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "W3.shape"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "최종 결과는 원소가 10개인 1차원 배열 y가 출력됨"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 59,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# 그림 3-26!"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "위는 데이터를 1장만 입력했을 때의 처리 흐름"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 60,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# 그림 3-27!"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "입력 데이터 형상은 100 X 784, 출력 데이터 형상은 100 X 10\n",
    "\n",
    "100장 분량 입력 데이터의 결과가 한 번에 출력됨\n",
    "\n",
    "배치: 하나로 묶은 데이터\n",
    "\n",
    "배치처리 이점 : 이미지 1장당 처리 시간을 대폭 줄임\n",
    "\n",
    "* 수치 계산 라이브러리 대부분이 큰 배열을 효율적으로 계싼\n",
    "\n",
    "* 버스에 주는 부하를 줄임. (CPU, GPU로 순수 계산을 수행하는 비율이 높아짐)\n",
    "\n",
    "배치 처리 구현"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 61,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Accuracy:0.9352\n"
     ]
    }
   ],
   "source": [
    "x, t = get_data()\n",
    "network = init_network()\n",
    "\n",
    "batch_size = 100 # 배치 크기\n",
    "accuracy_cnt = 0\n",
    "for i in range(0, len(x), batch_size):\n",
    "    x_batch = x[i:i+batch_size]\n",
    "    y_batch = predict(network, x_batch)\n",
    "    p = np.argmax(y_batch, axis=1)\n",
    "    accuracy_cnt += np.sum(p == t[i:i+batch_size])\n",
    "\n",
    "print(\"Accuracy:\" + str(float(accuracy_cnt) / len(x)))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "range(start, end, step) 처럼 인수를 3개 지정하면 start에서 end-1까지 step 간격으로 증가하는 리스트를 반환"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 62,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[0, 1, 2, 3, 4, 5, 6, 7, 8, 9]"
      ]
     },
     "execution_count": 62,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "list( range(0, 10) )"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 63,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[0, 3, 6, 9]"
      ]
     },
     "execution_count": 63,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "list( range(0, 10, 3) )"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "x[i:i+batch_n]은 입력 데이터 i번째부터 i+batch_n번째까지 데이터를 묶는다는 의미\n",
    "\n",
    "batch_size가 100이므로 x[0:100], x[100:200]와 같이 100장씩 묶어 꺼내게 됨\n",
    "\n",
    "argmax(): 최대값의 인덱스를 가져옴\n",
    "\n",
    "axis=1 인수를 추가. 100 X 10의 배열 중 1번째 차원을 구성하는 각 원소에서(1번째 차원을 축으로) 최대값의 인덱스를 검색\n",
    "\n",
    "argmax axis 예시"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 64,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([1, 2, 1, 0])"
      ]
     },
     "execution_count": 64,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "x = np.array([[0.1, 0.8, 0.1], [0.3, 0.1, 0.6], [0.2, 0.5, 0.3], [0.8, 0.1, 0.1]])\n",
    "y = np.argmax(x, axis=1)\n",
    "y"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "배치 단위로 분류된 결과를 실제 답과 비교. == 연산자 사용\n",
    "\n",
    "넘파이 배열끼리 비교하여 True/False로 구성된 bool 배열을 만들고 True가 몇 개인지 셈.\n",
    "\n",
    "True 개수 세는 예제"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 65,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[ True  True False  True]\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "3"
      ]
     },
     "execution_count": 65,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "y = np.array([1, 2, 1, 0])\n",
    "t = np.array([1, 2, 0, 0])\n",
    "print(y==t)\n",
    "np.sum(y==t)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 3.7 정리\n",
    "신경망의 순전파를 살펴봄\n",
    "\n",
    "신경망에서는 매끄럽게 변화하는 시그모이드 함수를, 퍼셉트론에서는 갑자기 변화하는 계단함수를 활성화 함수로 사용\n",
    "\n",
    "이 차이가 중요함. 다음 장에서 설명"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "이번 장에서 배운 내용\n",
    "\n",
    "신경망에서는 활성화 함수로 시그모이드, ReLU 함수 같은 매끄럽게 변화하는 함수를 이용\n",
    "\n",
    "넘파이의 다차원 배열을 잘 사용하면 신경망을 효율적으로 구현 가능\n",
    "\n",
    "기계학습 문제는 크게 회귀, 분류로 나눌 수 있음\n",
    "\n",
    "출력층의 활성화 함수로 회귀에서는 주로 항등 함수, 분류에서는 주로 소프트맥스 함수를 이용\n",
    "\n",
    "분류에서는 출력층의 뉴런 수로 분류하는 클래스 수와 같게 설정\n",
    "\n",
    "입력 데이터를 묶은 것을 배치. 추론 처리를 배치 단위로 진행하면 결과를 훨씬 빠르게 얻을 수 있음."
   ]
  }
 ],
 "metadata": {
  "anaconda-cloud": {},
  "kernelspec": {
   "display_name": "Python [conda root]",
   "language": "python",
   "name": "conda-root-py"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.5.2"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 0
}