{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Deep Learning with Python 2章勉強ノート\n",
    "\n",
    "ManningのMEAP（出版前の本のできた部分から読めるサービス）でTensorFlowとTheanoの両方に対応したラッパーツールkerasの作者Francois Chollet氏の「Deep Learning with Python」が発売されました。\n",
    "\n",
    "1章は、Francois Chollet氏のDeep Learningへの思いの丈が詰まっており、2章からDeep Learningの紹介が始まります。\n",
    "\n",
    "ここでは、2章の例題をSageのjupyterノートブックで試しながら、Deep Learningへの理解を深めたいと思います。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### kerasのデータセット\n",
    "kerasにも例題を試すためにデータセットが容易されています。\n",
    "\n",
    "MNISTは手書き数字の画像データセットで、70000の手書き数字データが収録されおり、ここでは、60000のトレーニング用データ（train_images, train_labels）と10000の検証用データ（test_images, test_labels）に分割しています。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Using TensorFlow backend.\n"
     ]
    }
   ],
   "source": [
    "from keras.datasets import mnist"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "(train_images, train_labels), (test_images, test_labels) = mnist.load_data()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "train_imagesのshapeをみると、60000のデータは28x28の画像情報からなり、グレースケールを0〜255で表現しています。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(60000, 28, 28)\n",
      "[  0   0   0   0   0   0   0   0   0   0   0   0   3  18  18  18 126 136\n",
      " 175  26 166 255 247 127   0   0   0   0]\n"
     ]
    }
   ],
   "source": [
    "print train_images.shape\n",
    "# 画像データの例として５行目の値を表示\n",
    "print train_images[0][5]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "ラベルには、０〜９の値がセットされています。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "60000"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "len(train_labels)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([5, 0, 4, ..., 5, 6, 8], dtype=uint8)"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "train_labels"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "画像の例として5番目の数字を表示します。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAP4AAAD8CAYAAABXXhlaAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJztfV2IZNt13reqq6vrp3/n3qt70b2xHBxIyIMQJBbEEqSN\njRHBIGNQosgYKTbCD0psIj9I1svMNX6w/HBBCPQQRxaSsXBsgyP5xZKNaIQMtuREciT0ZwiyLJw7\nsu7Mnenuquqqrt55mF5nvrNq7VPVf9U9ddYHm7Orprpqn5r69vpfW1JKCAQC9ULjuhcQCAQWjyB+\nIFBDBPEDgRoiiB8I1BBB/ECghgjiBwI1xIWILyJvEZFvisi3ReR9l7WoQCBwtZDzxvFFpAHg2wB+\nAsA/APgSgLenlL5pXheJAoHANSGlJN7zF5H4bwTwtymlv0spjQH8PoC3Zj68GLdv3y49vmkj1re8\n67vJa7uK9VXhIsR/HsDf0+PvnT4XCARuOC5CfE+FCLU+EHgC0LzA334PwA/R4xfwyNafwp07d4r5\n9vb2BT7y6rG7u3vdS6hErO/8uMlrAy6+vr29Pezt7c312os491YAfAuPnHv/D8AXAfzHlNI3zOvS\neT8jEAicHyKClHHunVvip5QmIvKfAXwWj0yGj1rSBwKBm4lzS/y5PyAkfiBwLaiS+JG5FwjUEEH8\nQKCGCOIHAjVEED8QqCGC+IFADRHEDwRqiCB+IFBDBPEDgRoiiB8I1BBB/ECghgjiBwI1RBA/EKgh\ngviBQA0RxA8EaoggfiBQQwTxA4EaIogfCNQQQfxAoIYI4gcCNUQQPxCoIYL4gUANEcQPBGqIIH4g\nUEME8QOBGiKIHwjUEEH8QKCGCOIHAjVEED8QqCGC+IFADRHEDwRqiCB+IFBDNC/yxyLyHQAPAJwA\nGKeU3ngZiwoEAleLCxEfjwi/m1K6fxmLCQQCi8FFVX25hPcIBAILxkVJmwB8RkS+JCLvvowFBQKB\nq8dFVf0fSym9LCLPAPgzEflGSukL9kV37twp5ru7u9jd3b3gxwYCAYu9vT3s7e3N9VpJKV3Kh4rI\nbQD7KaWXzPPpsj4jEAjMDxFBSkm8fzu3qi8iXRFZP533APwUgK+d9/0CgcDicBFV/1kAfywi6fR9\nfi+l9NnLWVYgELhKXJqqn/2AUPUDgWvBlaj6gUDgyUUQPxCoIYL4gUANcdE4fuCGQf0p7Fexz/E1\nN6+6Vs3ngYhUDn7dvPfJa+D3sfOq66zPXCYE8ZcQKSWcnJwUZObHJycn7phMJqXXeFfvfe2YhUaj\ngZWVlWI0m83S40aj4RKTCemtR68AivfJDREprjr4c+uAIP6SoYrcSvDj42McHx8Xc77q4M3AznXw\nZ1ny5bCysoLV1VW0Wi33urKyUqkN8Gd66xOR0obSbDZL89ymoGurC4L4SwglBxNZx/HxMcbjcXZ4\nm4Kd50g3mUxmEr/ZbKLdbqPdbmNtbW1q3mw2S9LYSuiUUuXaRASrq6vZYTUMJbu+d12kfhB/ycAq\nMEt3JcdoNMJoNMLR0ZF7tRuAnXvaAY9ZxF9dXUW320W320Wn0ynmOlZXV7OqeaPRQEqptCa7RhHB\n2toaWq3W1FU3qmazWdJO9P3rhCD+EsJT65XQo9EIw+EwO3Rj4Nfz3G4k9vHJyUnl2lqtFtbX10tj\nY2MDR0dHGI/HaLVaU+o4Pz45OSlpKLy+8XiMRqNR0iTa7XaxNjYTVldXATwm/bw+imVBEH8Joeow\nq/ZKkOFwiMFggH6/X7rqnMnPmgAPTxPQ6yzit9ttbG5uFmNra6v4jOPjY3Q6nRLZ2em3srKCk5OT\nyrU1Go1Ck9B1qXS3arySXt+3TgjiLxk8VV+l4dHRUYn4h4eHU2MwGODo6Kg0hsNhMdcNxEpaHZPJ\npHJ9nU4HOzs72N7eRr/fL95TJfJ4PJ6ywXlMJpOp9fFoNBro9XpTpAcw5blnLSIkfuBGY1YMngnu\nXXOE16Fk9Eiv2gBvJtZZ6ElOJpSq42y3s9d+NBpVEv/k5MRdGxPfOiJ5TRx9YInfbNaLCvW62yXA\nrHDdeDwuVPfc6Pf72aHktk4/VvNVOitxms1mEUZTkuUSbPS1Kt2HwyH6/X4RxhsMBlPqPav8Svyc\nObKysjL1HXHUQbUgXSer+iHxAzca9kfMc5bqSmaes02vaj8PdbJ5Dj5Wya36rBuATeaxCT5KfN2k\njo6OcHh4CACYTCYXcu6NRqPCY18V61fSiwiazSZWV1fnikgsE4L4TxjYfrfxd5WgBwcHxTg8PCw9\nVsLrUJVZ50ok67RTRx6rzZxRx6pyVZYfE//4+BjD4RAACqedhvPYFOChf5cLOTLxbd4BZyeqhqLJ\nQ0H8wI2GEoq99Tz6/T729/exv7+Phw8fFnMd/X6/UI+9WL6N1dtrSsmNr/NzVam+/JrxeAygTHpV\n+W3iTi6Bx16bzeaUfc/E1zU0m80ixm83tDogiP8EgiW+xuWVyAcHB9jf38eDBw/w8OFDPHjwoDQf\nDAZTnnm+Kglshh7H6FVqs4qvAyj7Iay9rX+nxNekIs/R5+XSK/Fza1xdXXUJr0PXv7q6WsT7eVOr\nC4L4Txg8iX90dFTY6IeHh4W0f/XVV6dGv9+vVJVtsY6V3kpATXVV4qvKrKTODVb7j4+Ps0U+ueq6\nXHGOXpX4ucxCTelttVrodDrFphfOvcCNho3R29g8S/xXX30V9+/fx71794rrYDCYSrm14S/9HC9c\nqI42WxCjarP+W86pZtXzqsy/WWW53vrUUWc/U4dm9nU6nVJ4MiR+4Eox74/Lq3dXKelJeXXksWqv\ng+39wWBQKZEVtnRVn1tdXS2lw9phie+RnsNvAArS6QZg79m75qDpuHZ4RUis3dSJ9EAQ/8aB1V7P\nK67SXclunXhqyz98+BAHBwdFbF7taQ5lsdqu788hNBtPVy/4PMT3pL068TiqwFEG1UY8E8NuTDmw\nT0C1EVXtdWiVHpfp1qUqTxHEvyFgiVYlkTXhhW15K+V1E2Dis+PO63jDhLFlrPy41WqViG4LYjQZ\nJjc0z0DvYW1trZTAo7kCNgyn3808kpmJz+TXKj0mP+cI1In8QfwbAKvWe9lmOqzEV1tex/7+/lQK\nriV+VZy82WxibW2tVNLKjz2y83OW+HYj48iDvqdGCQCUUm61zNZ+R1XgDYwdjyz16056IIh/7bCk\nZzXfK39l4rPEV+ed2vGclWdVfSa+bX+l3u5Op4N2uz01r5L2a2trpfRXLzowHA7x4MGDKdLr60Sk\nsMeZ9Ppv84DNFVXpLfEt+YP4gYUh58BjSW9DbqzqHxwcFGE7Jj4n5LAjjR1nNgavRGi32+j1euj1\neuh2u6Vrr9dDp9NxO+cw8XPJOykl9Pv9LOk1vMdE1Lj9vMS0Nj5n582y8etE/iD+NSFHepaQTHpb\nT2/j9ffv38crr7yCg4ODqao57p5jy1ItMTqdDnq9XtEgY2Njo5ivr6+j2+1OkZ7Jz00tPEdlv9/P\nkv7o6Ki0RiX9WaWy1WY8Bx9rOSHxAwuHF66yEp9Tc3POPSZ+Lo6uJGKJz8TX+Hav18PGxga2traK\nZhk67/V6rsRnG98jvo7Dw8PCkWdJr159/h508ztLayzPqz9L1a8b+YP414CqcJ2tp7c19Tnvvcbx\n+/1+6TNYs2CiK1GtLb++vl4iOw8mfm6wje9l1k0mk0LV9+zsWSr3LHJaT76Sne/Vfn7dSA8E8RcO\nK9HtfDQaVdbSP3jwoMjEe/jwYdE1R8NgCpWQtkd9q9UqGluq/c5zVvFZ1dfmmO12u9I7rvfo3d9k\nMimVBXNvAHZEahRCzROvl5+9L90wNItQNZh2u126R71PvY+6kj+Iv2BwdZk3WJXnbjms3mvoTomv\nXnvOp/dCdtqBVm343NB/V6LoXB173Aff84xbNZ2HbfzhbW7c8JPbZ6n2kivk4fp61UBU0vM9s+Tn\ne6gTgvjXAJWCXqOLwWBQqO6sxvNjq+azxGdyeK2rVJ3f2NgoGl7q3Ep32wJbiW8jAUx8W0tg728W\n8XUTY8ekl8Pvlewy8TnRSO+BIxNM/Do6+GYSX0Q+CuCnAdxNKb3+9LkdAP8DwOsAfAfAv08pPbjC\ndS4NbHWd7RnX7/dLefY2555tedsuS1V967hjoioB1HnHY3t7G71erzJWrx1y7BFYTHzWarg9Fmft\nsapv1fxc8Y46JfUevSQkJr46K62qH3b+fBL/YwA+DOAT9Nz7Afx5Sum3ROR9AH7t9LnADFji29x1\nDtFpVh5fVbW3jj/b4ZZj9HxMFXvtNzc3sb29jZ2dnWL0er1S2IvTXDkM5rXFUuJYic9dfuZR9b2M\nRVuyy6TnNVjiW1Vf7y+IPwMppS+IyOvM028F8G9P5x8HsIcg/lxgx5fX8lpVec7Gu3//fjH6/b7b\nEktt/FyRitq87MBTKX/r1i089dRTuHXrFnq93pSWYNX6nJrt2fh8j0z2Kqlf1Q/AU/O9ZB1r47PE\nt5V7unHUCee18V+TUroLACmll0XkmUtc09LDk/heNt69e/dw7949vPLKK8VQiZhLidXEGC9Grx5u\nlvg7OzsF8Z9++mn0ej23Kk/ntrDHXj1V325uTH4r9YfDoZvbYHP1PdLbMJ5Her0/7yDNkPiXjDt3\n7hTz3d1d7O7uLuJjrwVVySsppSJRhYnOTjxV6b2a+v39fQyHwyk7lzviWBWXnXSdTqd0io069Hh0\nu90paWrns5CT9jbfQIlum3xWwfNf8JUdkba2gNOKPTPhScfe3h729vbmeu15iX9XRJ5NKd0VkecA\nfL/qxUz8ZQer8t5QO17Jzle277XKTj32msfOP3zvquq8HRq/Vim/tbVVEL3dbk95t+dNqMndP5Oe\nswx1E1NfBcfr5y255Y4/torQJht1u91S2M5rDroskt4K1RdffDH72nmJL6dD8WkA7wLwQQDvBPCp\nM65xacE/fO+oqcFgMNX51g4lSI74nIPOQ215jsHbwVl5GrpT8lj73WoWZ71/1m60HdiDBw+KPgHc\nx/8sxNcqQtZqdGxvbxebGsfsORpR1dizLpgnnPdJALsAnhKR7wK4DeA3AfyhiPwCgO8CeNtVLvJJ\ngv7wOYTFg6WfbYOtXXO4np5Ddbaqzlacqdeei2q8k2k5Oy8n8c9LBrbvlfjsu+B7VDX/LD3v2HOv\nPgve6Jj4VRLf2+DqhHm8+u/I/NNPXvJalgIppZJ9az3X3C7L64unUp5r6lniA5hqcMnx9m63W7LZ\nrR3PaatqAijxrR1/UVVfE5J0s7MtwbjV93klvobp+PTd7e3tQtX3JH4u669OiMy9S4ZVde0hlUwA\nHvqcHnjhHU2tTTTYW2/VXSaBN2yCjg4+zMJT8c+r6rNjj3v7W1V/3vbWbOPzRqeEVzOGVX0r8etM\neEUQ/5LB6bhcO89SPXfYxcOHD4vSVHtCTE7V5wSVXEYej263WypNZTOB49lMiIs49zyJb0/wvYjE\nV1V/a2ur5LRkVd9KfL6ns25sy4Ig/iXD++Gres+hutzQOLbXxMI691TVt7F5lX7e6HQ6lcdQXzSs\nlSM+2/isyXA+/lkkvtr4vNnt7OwUacf6nXgSX9+nzgjinxFVMXoAJSmvks52w1XJ59XSHx0duSq2\nqqhWxWdJr6T34vTq3NMuObk4/bz3r3N75aYhnFbMCTv2QM5cEY5Vx0WklJSj5Oaio42NjVLuArf5\nWpZ4/WUgiH9GeHF6zi3v9/tT+fU8t/3u1autP/xcOa3O2aGlpLbNMjY3N0tVdqrOnzdOb9uE5dJp\nT05OivviegIr2b3DNqxGYzMHdbAPgzc0lfLszLMpuXWX8owg/hlh4/S2v52eZuORnh14Xttr+8P3\nfvyelGPHlpKe++OpqpsLY+XIbwnP959rJMIRCSY91xXYijtba29bZvFQ4rMmw1EK9WGwpFenZeAx\ngvhnhLVhbbye0261YQZvAAcHB6WTY5j4AEpE95J0csRXG1dj8zpU+uVaXOUIkSM9V955DUUs8T3y\n27PtbOVdo9EoFRZxdh6bM5b8atNzy6265uLPQhD/jPDCVVxkosS3hOeyWraBLfHZa+9JvCqJv7Oz\nU9jxNj/dU3nnUfWtTc8NPFnr0WFVfc+JZ4nvqfq2L6BXa5BT9Tl/n4kfeIwg/hmR81rbOD2fbsNO\nPe+YaiUE8Fjiey2htV8et5FSEqjXfn19fepv2Lk1T5zeq4pjldzr/qsj1zuP79ceq2Wr75j47MDs\ndrslX4aV9kp82yQkJP40gvhnRFVmGh9pZcN0ugkMh8MpaccHQuZKam09fU7i23p6HmeJ03sOvVxb\nLTZ1mPQ5555Hek/ie33+mfSW/Ep8W4hTx/bZsxDEPyOqElS0ss4bHKfPvS9Qdm7ZtFyuLc/Z+L1e\nL/vDP6uTKxe2tP0EbIcdzcjzVH1NPfZIb++fW2fZe/VUfc1K1Pc5T5FRXRDEN5iVRGJrzW3nHNsb\nz6s9Z5Vb50pK7gPvNbvkkF3Oxs3los+S7Hp/drCU1gpDriVgPwffP0cuuGOuQteUUio0Ea9XHkt7\nTsdlB6b6MVZXVy/hV7D8COJXwCaoAHA7y3DrayU8d4y14ToluldPz2fXeW2uvXp6G66bJy7v2e7W\nfveO4VLzJne+/f7+Pu7fv18qv+V8fAClDY9Ho9GYaoXN3YC9TsARsjsfgvgZeOQAUCJGrpsOq7vq\ntbe59jZkp3NOQ80NVnVtWa3nsbeE8NKB9arltPY0Hx4e6XVoHoNKfpX6NnLhbXyap2DLiG1WIuco\ncHJSEH9+BPEdWLvTs2+rJL5HfFZxrefeto6yjiy+2jp7T+ID03atzr3QHA/2zNvGmByqYwceP1az\nRzdCT+Lz/VvnI2s7THq+f87Q0z7/4bw7G4L4M2CdUF7baC5E0QMulAieqp8L2dm+96rWc5GN5qJz\nPz2up5+n6szG4/mqzT/5fniwr8K7cl6+PSTDfgd8/7oBslljJb5qOdzrPyT++RDEd1DlzbY2Pjv3\ntCDHnk2fU/VtD3hbacftr2/dulUk6NhsNu/HXxWus/n2nFjDxPfKiFWCWzPAhu/s3Nr4tsJQ78OG\nK63E9/r+B/HPjiB+Bh7pcwdFWFXf9tuz9q0n8Thkl+t7r+2vvRi9p+5WpePy/fCR3NzuW0uJua//\n4eGha/frsIlJPLSRiNfzXyX4LImv929zFULVPxuC+BWwpOeQlmfjqzpsm2iwqg/42Xk2Tm/73j/9\n9NN45pln0Ov1pkKBXuHNrHtiDYZz7jVcxwd0am//H/zgBzg4OJiS5izhNU7vOQ41bOdJfDVdqkiv\njURsW+xIyT07gvgGOTXfk/y29lxtXI578xx4rOYy4bm23B5Pzc4udWp5jkc1JXLptnr1wnV89XoH\ncDaimjJ26AZgj7O2yGUm2tRcHrZHftXGF5gPQfwzIBcG80iey0Nn4nPrKCW7bRSpXmtNdLG57rn0\nX7sGr7jGi9MfHh6Wju5SsttkHJuCOyvxie+fzRx7vh1vcPYYa6+HQGTlnQ9B/BnwQnu5I6xyzSV4\n2H55tk+eEl8bRXK/e/Yx2L58Xo8+b3hkt9V1VuLnsvDOS3x7zp01czhU5zURmbeBSCCPIH4FcoUq\nHum9GvMqie91ib1161apJzxnprHEzx3WoREEe5gmbxSW6PY57hHIV9s4xG4w9j5zYInvnenHDTK5\nmYZtnRUbwMUQxM/Akl6vnr1vJX7OPwD47aFtl1i2bVnVV/tZIwpeIxDdDLyr3Si8wWfY26GfYaMB\n80p8L6JhVX226askvr4fv3dgfgTxZ8BzkOXI77WTsoRgG5/Pqlfib25ulhpo5CS+zZizXW9mbQxV\nxOf3tnONx+d8CTkwWb2IxiyJP4v4gbMhiF+BHNk9ac8ZcPZv+TpL4m9ubrrnt6vE53Bi7sx5m1bL\nIyf5PbPBe2w1Gs+csbCS2Up8DuVZ4tuGmTZsF+Q/H4L4Djw1n+dVdr6q4zkiaCjLs/GV+F7VXk7i\n2w5A3pnz/FilNhM71xPPG3Yjq7pXYJqYXgzfSnzr3IsknctHEH/BsNEB6xw8Pj6GiBRXO0ajUanW\nnwtizkL8nJMv56CcR7Izcva3p+oz+b0W2V7bsMDFEMR3UPWj1SsnjXgZZDlVP6VUVMBpq661tbXi\naCfNzOPP4aEhN294DTI8VZ/DfrYxBnvNT05OCk1Dr1X3lfueeG5DeV5DTVbzo0vu1SCIX4EqAnpt\nrXTkHHsqOTVstr+/XyL98fExut3u1GfyY21hbZ16OeeeVyxjK/LY/8AbF5O90WhMSX0vx4G/O29w\nOW6O/LZJaKTkXj6C+DPglblWbQCNRsN1BCqY+AcHB2g2H/0XaAOMXM84vU4mk7nI7YXxVNpb34QS\nVsltN68c+fm+cqS3abU5NZ+Jb0t17Sm+gYtjJvFF5KMAfhrA3ZTS60+fuw3g3QC+f/qyD6SU/vTK\nVnkNqFLxZ0l8Jhb/vT6nqj6TXj30rVZr6jP5qn/vOeZsVp03Z6LzVd+fJb4n3ZX87MRkk4C/M88k\n8voQsGdf++JzBV6o+pePeST+xwB8GMAnzPMvpZReuvwl3RxY8s+r8nOyjSWFSnzttquSXlNluVmk\nF69Wz74dNnXXS9XlngB8T/b+9HP0aue8Hr2/nIZivyfbeUhr8bm56FWc4hsoYybxU0pfEJHXOf9U\ni+3X+1HnnHr2B6ok4aHEBx6T/vDwsJShZj+f57n8gaoiIX4MTB/MyeRkeLa7V33nefqrtCJuHa5h\nTVb17fqiL/7l4yI2/ntE5OcB/DWAX00pPbikNd1YzKvqK3ISUvPd9Uhsfh/9HO+z9T35vWc52zxV\nndcLoLh6drSXeejdn/273HeVk/is6tu1hH1/+Tgv8T8C4NdTSklEfgPASwB+MffiO3fuFPPd3V3s\n7u6e82OvHjmV3nNQeQko3kGSx8fHAKYbe+hzVWvhq30up1rr3CtosZKXNyyuCfAy83jd9n54jUx0\n7iC8srJSJOXYwZmKgfNhb28Pe3t7c71W5knIOFX1/0Sde/P+2+m/p3mTPm4CrLfbzvf393H37t1i\nvPzyy6XHDx48mGpOwXOvYQZfgfk2H1Z/vciCNUM8zcQjf0opu3lVDXUeishU52B+vLOzg+effx7P\nP/88Xvva1xZzHRsbG9fy/76MOPUtuarSvBJfQDa9iDyXUnr59OHPAvjaxZZ482Al58nJiZtuu7m5\nWSpeabVaU4k0GuLzKvcYHE+3vgTrU6giMEtZb1411BTJtdbSdbKkz2lF1nOvzTS18o6TdMKGXyzm\nCed9EsAugKdE5LsAbgP4cRF5A4ATAN8B8EtXuMaFw1Otlby2iYbtINtqtUoptUx6bjHN4T5P2ldJ\ncG40aZtOqnS16rM+pyTLEZ/PuOeUX41M2PyEyWQyZYPbIhw1h7SLsE3JDa/94jGPV/8dztMfu4K1\n3EjYhBxbUmsPiuCWz5b0w+Gw0B709V4SjBcCU3JYYntX236bHzPRbLhsZWUF4/G4lP/PJcG22chk\nMpnyuHsSXz32ullWddcJLAaRuecg9wMUKXfP6fV6JRIzAT3S21x+dopZtd9T670+/B7BuTElh8r4\nqC27oeh1NBoVrbcs6TVBSIlvw4H6HVni20MwWeJz5V1gcQjiZ+CFpwCUVH0r6fXH7JGe1WygnPhS\nlfFmY9+2D71HcEsy7lhriW+vw+GwdCwVk34wGBQOSt40bFSBVX3bU49t/JD414cgfgW8cJpKfCvp\n1dnXarWmSN/v911b1kvw0c+zqr5X2MKxbyZW7rTdXq+HtbU11+Ovc00l1g1JD9kYDAaFP0FV/lxy\njQ152i66VWW3gcUgiG8w68fH0kyJyz/ylFIpjGfDec1mM5tpp2TyHHc6Z7WZCW+72OSO2VaNxGbu\nscrOZoElpVe8YzUWL1HH+h5mddYJXC2C+GeEl8jDhS9KPj1VRlV6Je1gMMj2rOPX2jAcp7nm1HxP\npfZsatYsdN3aMowr+njT8s7H45p+W97rOSftybiRint9COKfA5b4/IM/Pj7G+vp6cZQUk77T6WA4\nHFYmCakGkYvRc4jMc+xZ555tY8Ult0D5nICUUqnSz5KeTwC2vfW91N4q0tvEokjLXSyC+OeA/qCZ\nQJzo40l6baqpx0zlUmKV+DmvuxfOs/OqcJ6u0avL1wIiK/Vt3T9n6uUkPpsQs8gfpF88gvhnhFX1\n9Tl+bEmvJ+BubW1hPB5n7WQljpWGuSIXa//rlW1om8gjIm4nYHXY2SYe3lHYXPrLEt86Kz3npCfx\ng/iLRxD/HPAKYPQHro9Z0jNxbNouUHaYVaXrcqgsl3lX5R+wlYNq46sGYE/p4cGqfs456X1HrLHM\nIn9gcQjinxEeMTm7T3/Ya2tr7hFWub77PD9LkY5dhw3T2ces3ivZqojvtfjKmSpVzr151Pwg/+IQ\nxD8HlOgAijRenaeUsLa2NrM99ZxVkdm5jfnz3Husz6mUtiE6PkW3yquvnYPsPXghvSqvfjj2rhdB\n/DPiSZdO6bQZB5Pe5hKw/e61+MrByzrMxfG9o7ECi0NkTdQQOadiVWLOLNjMvaqUXQ0vRlnu9SGI\nHwDgt/Li56tgzRDOObApu1VluUH8xSGIX2N49fVnlfheMZOV+F51HqfuRnXe4hE2fk3hSXXP+Vjl\njPQktLXv+ZQcLSDS51jiayg0sBgE8WuIKpt+XomfIylLfK86TysYw8F3vQji1xy5TYD/3cvDz8Gz\n8a2qb5txhqq/eATxa4pZ3vycxJ9HKrONb1X9brc71XI7JP7iEcSvIbwMQI+I50mwsXF82zJMKwRt\nRmGQfrEI4tcMSkwmJJfV2tbX7HgLci4Pgvg1RK6JiIhMdcAN59tyIohfM1Q1EeFmmzbGHs635UIQ\nv4bgRiJAeTPQzj1W4oeqv1wI4tcM7HzT/nusAXhZdXHSzfIhiF8zcBMR9uirre+p+mHjLx+C+DWE\nklh7CXA833PuBemXD6G/1Ri2t4Bt2mFfF1gehMSvIXKtvU9OTorjvbXhZq59duDJRhC/ZmCyc7cd\nfayddC0sqPvwAAAN4UlEQVTxOd4fePIRxK8ZNAffa601i/iB5cFMG19EXhCRz4nI10XkqyLyy6fP\n74jIZ0XkWyLyGRHZuvrlBi4DtquuNtRUNd8S3zYKDTz5mMe5dwzgvSmlfwng3wB4j4j8CwDvB/Dn\nKaV/DuBzAH7t6pYZuCx4ffSV+MPhEIPBoER8NgUCy4OZxE8pvZxS+srp/ADANwC8AOCtAD5++rKP\nA/iZq1pk4HLBqr4n8XPOvZD4y4MzhfNE5IcBvAHAXwJ4NqV0F3i0OQB45rIXF7h8sHOPic8n5YSN\nv/yY27knIusA/gjAr6SUDkRk7u3/zp07xXx3dxe7u7tnWGLgssFn5dnTc46OjorHrOaHxL/52Nvb\nw97e3lyvnYv4ItLEI9L/bkrpU6dP3xWRZ1NKd0XkOQDfz/09Ez9wvbBdddnetx7+IP2TBStUX3zx\nxexr51X1fwfA11NKH6LnPg3gXafzdwL4lP2jwM0EE59j+aza2+Ovg/jLhZkSX0TeBODnAHxVRL4M\nIAH4AIAPAvgDEfkFAN8F8LarXGjg8uAdm1Ul8YP8y4eZxE8p/QWAlcw//+TlLidw1bDNNL2z8jzS\nB5YLUaRTQ5xV4oe0Xz4E8WsIz8avIn1I/eVDEL+m8Ip1Zkn8IP/yIIgfKNXh2xFYTgTxawjbV5+P\nuvLabnHrrZD6y4Egfs3AJ+jYU24s8XOtt4L8Tz6iHr+GsJ119Yy7yWRSEH/WSbYppTAFnmAE8WsG\nj/StVqtw5vHxWZb485A/nIBPBoL4NYRHfvXge4dpVPXVz0n+IP/NRhC/ZsidZKthO+vcm6e9dpD/\nyUMQv4bgk3RsZp517lWp+oyw+Z8sBPFrBnt+PZNfRArHnjr3dPBZ9vNKcpsarEPXwAd5hHawWATx\nawgvpKdgae8NPmHXZvTxldN9bWYgk9xuAoHFIIhfQzDprbRVLz9f1Q+g0p9z/W21HzC7FgB4TPiT\nk5M4kPMaEMSvGTgdV8mvzzcajZkSXx2BqrLrXN/DSn2v+o/X0Wg0ogLwGhDEryH4xFx9rM+ppM9t\nAOoTmEwmJWceN+P0yn65ww+bGkH660EQv2ZgkiuYhFWE1zGZTIq/VdKy13+Wqm99DEH8xSOIX0Mo\n8fTK9vo8qn6O5AqvmSd39G00GphMJlhZWYkuP9eEIH4N4YXT1NZnVd/bBDgCwAT3UndzEl+Thyzp\ng/yLQxC/ZmCCegk3qs4r0dfW1tBut9Fut9HpdIqDNkajUUnqK/l1riQfj8el47lardaUhOdMQi9R\naNaaA2dHED9QgtbqK+k7nQ663S42NjawubmJlBKOjo6KoWRXqZ5SKhG+3+/j8PAQDx8+RLvdRkoJ\nnU5n6nguAIXZEY1Brh5B/EAJ3KRDpX2v18P6+jq2traQUsJgMCikM0t43gTG43Eh5Q8ODorin5QS\nRqORS3rNEVDHHzsBdW2By0EQP1BClcQfDodIKRWFOwAK0qvqr8TX47j6/X5BevXg89HbXDewuroK\nAKX0YAVrA4GLI4gfKIGJ32q10Ol00Ov1isM0lXws2Y+OjkqtuTyJzxoCS3rWMFqtVkFwTi7S1wUu\nD0H8QAlW4rfbbXS73UI954M4VNIPh8OCpOrJZxuf6/lt3J83mdFoVHj7NXrAIcfA5SGIHyjBU/XZ\nJucjtkejEQaDQVG+qym87Nxjs4CP27akb7fbGI1GpSIgTgmOfP7LRRA/UIJHfLXJARQ2uobo+v1+\nUbZrbXz73Hg8LkhsST8cDjEajQo730syCql/eQjiB6bAzra1tTWMx+PCLj8+Pi5s98FggHa7XWre\n4fkA2BEoIqWW3tzdt91uFz0BrMd/3tTe8AXMhyB+oASu2mOJrGr6aDRCt9tFt9tFv99Hp9NBp9Mp\nEny00MeT9sCjBKHBYIB+v1/aMNQPcHx8XCQMsWmgm5FV+YPo50MQP1BCrvW2EnA8HhfE18HkB1Bq\n0KnEBx5J/JWVFQwGA/fgDs3hVw3Dkp7LgXWt7A8IzI+ZxBeRFwB8AsBzACYA/ltK6cMichvAuwF8\n//SlH0gp/emVrTSwMLDEV5VbVXgmvhKe5wBKxAQeO/U0yccjPL+WSc+hPi3ssQjSnx3zSPxjAO9N\nKX1FRNYB/C8R+bPTf3sppfTS1S0vsGhwQs1kMkGr1So9r8S3hNfB9fXeHIBLfABTr2NfAyf92PXq\n38YGMD9mEj+l9DKAl0/nByLyDQDPn/5zfNNLBmvj2+cmkwn6/b5LfrXLNfQHoJhzONCT9LZ9l5Je\nnX9KfG7WadcdmB9nsvFF5IcBvAHAXwF4M4D3iMjPA/hrAL+aUnpw2QsMLBZMck6pZdXfs+91HB8f\no9FoYDQaFeo6J/tMJhOX+Jr4o1l77F/gkKJt1Bk4H+Ym/qma/0cAfuVU8n8EwK+nlJKI/AaAlwD8\n4hWtM7BAKNE90jPxLem16g54nMNvU3s1vGdLevW9ARSSXrMGx+NxKaSoiE3g/JiL+CLSxCPS/25K\n6VMAkFL6R3rJbwP4k9zf37lzp5jv7u5id3f3HEsNLALcmos74WqzDq7P73a76PV6GAwGRS4/gFI4\nD3gszdUx5+X56+sajUYRGux2u0ViD5sK3KSTawTqTv69vT3s7e3N9VqZJxtKRD4B4AcppffSc8+d\n2v8Qkf8K4EdTSu9w/jZFxtWTA3awec65fr+Pe/fu4d69e7h///7U/OHDh+j3+9mhff1yY3NzE695\nzWvwzDPPuKPX6xWVe3bwZhMowp3uFzJPOO9NAH4OwFdF5MsAEoAPAHiHiLwBwAmA7wD4pUtbceBa\n4UlQlbCq9rMazvH2tbU1HB4e4vDwcCqrT52DGudXDUBTfCeTCVZXV9Hv90tahHb94dZdHL/n1l1B\n/Pkwj1f/LwBMB0+BiNkvMTwCqVRV+9uSXjcE9fB7x22rD4Bh7XsmPRNf7Xxeo+0fGJgPkbkXcOER\nnwt42u32FOnVA++duKvEHw6Hha2uhOehmX3D4bCw763E5/VE2e75EMQPTMFT9fU5lfg50nc6nVKH\nXhu6azabBZmBx9Jen2s0GkUBkJX4lvhnPcQz8BhB/IALJj/PObGHSa/2frfbnSI9h+6U2AAKtV2J\nPxgMICIliW+Jryf4aF5/nMZzPgTxA1nk2lpr/zyO7XOMn6vt+O+YmEp2nuuG4Dn2mPzchDNIfz4E\n8QMlzPKKc8dbeyiGxtVttx5Ox9W/4zwBDhnaSIA1FaLN9uUgiB84F1j952SaXPceVfO1gacm/xwe\nHmJra6sIAbZarSKO//TTT2NnZwcbGxvo9XpYW1srneHH5b+xGZwNQfzAmTFPnF+Jz6TXXv3dbhfr\n6+tFQw515mn/vlu3buGpp57CrVu3sL29jY2NDXS73VJ4kDUB1gYC8yGIHzgXquL8Snx13qk3X6W9\n9uvzrs1mE1tbW6WxublZIj4nAAXpz4cgfuBcyMX51dNvJb0m/fR6vSJ0Z8d4PMbKygrW19eL03t0\nbolv03WD+GdDED9wZsyK83MdP5Nei220/TafoKvPrayslLL/eK4OP3u8Vjj9zo4gfuBcyMX5FSrp\n1cNvh2bu2atqDLnBJ+raEZgfQfzAueHF+bl1l9eCi2Pu9jneSKwab1X6XI5BYD4s/HiSeeuFrwux\nvmrkpK2Oz3/+86UOvarmcwhPbXc9entrawvb29vY2dnB9vY2tra2ihAeq/k2hHdWVf+6v7tZWOT6\ngvgGsb6L4Sav7yavDVhy4gcCgetHED8QqCHmar11oQ8QieqJQOCakGu9deXEDwQCNw+h6gcCNUQQ\nPxCoIRZGfBF5i4h8U0S+LSLvW9TnzgsR+Y6I/I2IfFlEvngD1vNREbkrIv+HntsRkc+KyLdE5DMi\nsnXD1ndbRL4nIv/7dLzlGtf3goh8TkS+LiJfFZFfPn3+RnyHzvr+y+nzC/kOF2Lji0gDwLcB/ASA\nfwDwJQBvTyl988o/fE6IyP8F8K9SSvevey0AICJvBnAA4BMppdefPvdBAK+klH7rdPPcSSm9/wat\n7zaA/ZtwkKqIPAfgOT7sFcBbAfwn3IDvsGJ9/wEL+A4XJfHfCOBvU0p/l1IaA/h9PLrJmwTBDTJ9\nUkpfAGA3obcC+Pjp/OMAfmahiyJk1gfckINUU0ovp5S+cjo/APANAC/ghnyHmfUt7DDaRf3Qnwfw\n9/T4e3h8kzcFCcBnRORLIvLu615MBq9JKd0FilOMn7nm9Xh4j4h8RUT++3WaIgx5fNjrXwJ49qZ9\nh7S+vzp96sq/w0UR39vBbloc8cdSSv8awL/Doy/+zde9oCcQHwHwIymlN+DR0eo3QeUvHfaKG/a7\nc9a3kO9wUcT/HoAfoscv4JGtf2Og5wCeHgb6x3hkntw03BWRZ4HCRvz+Na+nhJTSP9JBib8N4Eev\ncz3iHPaKG/Qdeutb1He4KOJ/CcA/E5HXiUgLwNsBfHpBnz0TItI93XkhIj0APwXga9e7KgCPNCXW\nlj4N4F2n83cC+JT9gwWjtL5TIil+Ftf/Hf4OgK+nlD5Ez92k73BqfYv6DheWuXcalvgQHm02H00p\n/eZCPngOiMg/xSMpn/CoR8HvXff6ROSTAHYBPAXgLoDbAP4ngD8E8E8AfBfA21JKr96g9f04Htmq\nxUGqak9fw/reBODzAL6KR/+vetjrFwH8Aa75O6xY3zuwgO8wUnYDgRrixoSvAoHA4hDEDwRqiCB+\nIFBDBPEDgRoiiB8I1BBB/ECghgjiBwI1RBA/EKgh/j8nIe1uID7bPwAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7f39a4b298d0>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "import numpy as np\n",
    "import matplotlib.pyplot as plt \n",
    "\n",
    "%matplotlib inline\n",
    "\n",
    "digit = train_images[4]\n",
    "plt.imshow(digit.reshape(28,28), cmap=plt.cm.binary)\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## kerasを使って数字を認識\n",
    "いきなり、kerasを使った例題がでてきます。\n",
    "\n",
    "ここでは、処理の詳細を理解する必要はなく、kerasでニューラルネットがまるでレゴのようにニューラルネットを積み重ねるだけで、ネットワークができることを示しています。\n",
    "\n",
    "以下の例では、以下の様なnetworkを生成しています。\n",
    "- Sequentialのインスタンスを作成し、networkにセット\n",
    "- ニューラルネットの重みの数を512個、活性化関数にReLu、入力が28*28の密なネットワーク（Dense）をnetworkに追加\n",
    "- 出力が10個の活性化関数がSoftmaxとするDenseをnetworkに追加"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "from keras.models import Sequential\n",
    "from keras.layers import Dense\n",
    "\n",
    "network = Sequential()\n",
    "network.add(Dense(512, activation='relu', input_shape=(28 * 28,)))\n",
    "network.add(Dense(10, activation='softmax'))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "次にセットするのは、どのようにニューラルネットワークを学習させるかをcompileメソッドで指定します。\n",
    "- optimizer: 学習方法として、rmsprop\n",
    "- loss: 損失関数として、categorical_crossentropy（複数のカテゴリから１つを選択する時に使用）\n",
    "- metrics: 尺度として、accuracy"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "network.compile(optimizer='rmsprop',\n",
    "                loss='categorical_crossentropy',\n",
    "                metrics=['accuracy'])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### データの加工\n",
    "イメージデータは、reshapeメソッドを使って(６００００， 28, 28)から(60000, 784)の2次元データに変換し、値も整数から0〜1の実数（float32）に変換します。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "train_images = train_images.reshape((60000, 28 * 28))\n",
    "train_images = train_images.astype('float32') / 255\n",
    "test_images = test_images.reshape((10000, 28 * 28))\n",
    "test_images = test_images.astype('float32') / 255"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "ラベルも0〜9の数字から数字をインデックスとして、１とし、他は0となるone-hot形式に変換します。これには、kerasのユーティリティto_categoricalを使用します。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "5\n",
      "[ 0.  0.  0.  0.  0.  1.  0.  0.  0.  0.]\n"
     ]
    }
   ],
   "source": [
    "from keras.utils.np_utils import to_categorical\n",
    "#  変換前のラベル\n",
    "print train_labels[0]\n",
    "train_labels = to_categorical(train_labels)\n",
    "test_labels = to_categorical(test_labels)\n",
    "# 変換後のラベル（one-hot形式）\n",
    "print train_labels[0]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "後は、fitメソッドでnetworkを学習させるだけです。全データに対して5回繰り返し（nb_epoch）、ミニバッチのサイズ（batch_size）を128としています。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 1/5\n",
      "60000/60000 [==============================] - 9s - loss: 0.2559 - acc: 0.9253     \n",
      "Epoch 2/5\n",
      "60000/60000 [==============================] - 9s - loss: 0.1044 - acc: 0.9693     \n",
      "Epoch 3/5\n",
      "60000/60000 [==============================] - 9s - loss: 0.0694 - acc: 0.9792     \n",
      "Epoch 4/5\n",
      "60000/60000 [==============================] - 9s - loss: 0.0513 - acc: 0.9846     \n",
      "Epoch 5/5\n",
      "60000/60000 [==============================] - 9s - loss: 0.0382 - acc: 0.9886     \n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "<keras.callbacks.History at 0x7f39a4766610>"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "network.fit(train_images, train_labels, nb_epoch=5, batch_size=128)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 検証用データで確認\n",
    "検証用データで学習したモデル（network）を評価してみましょう。\n",
    "evaluateメソッドで評価することができます。\n",
    "\n",
    "検証結果の精度はなんと９８％（0.98）と精度が高い結果となっているのですが、本文はこれをOverfitting（過学習）だと説明しています。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      " 9696/10000 [============================>.] - ETA: 0s"
     ]
    }
   ],
   "source": [
    "test_loss, test_acc = network.evaluate(test_images, test_labels)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "('test_acc:', 0.97889999999999999)\n"
     ]
    }
   ],
   "source": [
    "print('test_acc:', test_acc)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 2",
   "language": "python",
   "name": "python2"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 2
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython2",
   "version": "2.7.13"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 0
}