{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 程序说明\n",
    "时间：2016年11月16日\n",
    "\n",
    "说明：该程序是一个包含两个卷积层、一个池化层和一个全连接层组成的神经网络。\n",
    "\n",
    "数据集：MNIST"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 1.加载keras模块"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "from __future__ import print_function\n",
    "import numpy as np\n",
    "np.random.seed(1337)  # for reproducibility"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Using TensorFlow backend.\n"
     ]
    }
   ],
   "source": [
    "from keras.datasets import mnist\n",
    "from keras.models import Sequential\n",
    "from keras.layers import Dense, Dropout, Activation, Flatten\n",
    "from keras.layers import Convolution2D, MaxPooling2D\n",
    "from keras.utils import np_utils\n",
    "from keras import backend as K"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 如需绘制模型请加载plot"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "from keras.utils.visualize_util import plot"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 2.变量初始化"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "batch_size = 128 \n",
    "nb_classes = 10\n",
    "nb_epoch = 12\n",
    "\n",
    "# 输入数据的维度\n",
    "img_rows, img_cols = 28, 28\n",
    "# 使用的卷积滤波器的数量\n",
    "nb_filters = 32\n",
    "# 用于 max pooling 的池化面积\n",
    "pool_size = (2, 2)\n",
    "# 卷积核的尺寸\n",
    "kernel_size = (3, 3)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 3.准备数据\n",
    "需要将数据还原成28*28大小的数组\n",
    "\n",
    "注意：Theano和TensorFlow的维度表示含义是不一样的。\n",
    "\n",
    "'th'模式，也即Theano模式会把100张RGB三通道的16×32（高为16宽为32）彩色图表示为下面这种形式（100,3,16,32），Caffe采取的也是这种方式。第0个维度是样本维，代表样本的数目，第1个维度是通道维，代表颜色通道数。后面两个就是高和宽了。而TensorFlow，即'tf'模式的表达形式是（100,16,32,3），即把通道维放在了最后。这两个表达方法本质上没有什么区别。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "X_train shape: (60000, 28, 28, 1)\n",
      "60000 train samples\n",
      "10000 test samples\n"
     ]
    }
   ],
   "source": [
    "# the data, shuffled and split between train and test sets\n",
    "(X_train, y_train), (X_test, y_test) = mnist.load_data()\n",
    "\n",
    "if K.image_dim_ordering() == 'th':\n",
    "    X_train = X_train.reshape(X_train.shape[0], 1, img_rows, img_cols)\n",
    "    X_test = X_test.reshape(X_test.shape[0], 1, img_rows, img_cols)\n",
    "    input_shape = (1, img_rows, img_cols)\n",
    "else:\n",
    "    X_train = X_train.reshape(X_train.shape[0], img_rows, img_cols, 1)\n",
    "    X_test = X_test.reshape(X_test.shape[0], img_rows, img_cols, 1)\n",
    "    input_shape = (img_rows, img_cols, 1)\n",
    "\n",
    "X_train = X_train.astype('float32')\n",
    "X_test = X_test.astype('float32')\n",
    "X_train /= 255\n",
    "X_test /= 255\n",
    "print('X_train shape:', X_train.shape)\n",
    "print(X_train.shape[0], 'train samples')\n",
    "print(X_test.shape[0], 'test samples')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 转换类标号"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# convert class vectors to binary class matrices\n",
    "Y_train = np_utils.to_categorical(y_train, nb_classes)\n",
    "Y_test = np_utils.to_categorical(y_test, nb_classes)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 4.建立模型\n",
    "### 使用Sequential（）"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "model = Sequential()\n",
    "\n",
    "model.add(Convolution2D(nb_filters, kernel_size[0], kernel_size[1],\n",
    "                        border_mode='valid',\n",
    "                        input_shape=input_shape))\n",
    "model.add(Activation('relu'))\n",
    "model.add(Convolution2D(nb_filters, kernel_size[0], kernel_size[1]))\n",
    "model.add(Activation('relu'))\n",
    "model.add(MaxPooling2D(pool_size=pool_size))\n",
    "model.add(Dropout(0.25))\n",
    "\n",
    "model.add(Flatten())\n",
    "model.add(Dense(128))\n",
    "model.add(Activation('relu'))\n",
    "model.add(Dropout(0.5))\n",
    "model.add(Dense(nb_classes))\n",
    "model.add(Activation('softmax'))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 打印模型"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "____________________________________________________________________________________________________\n",
      "Layer (type)                     Output Shape          Param #     Connected to                     \n",
      "====================================================================================================\n",
      "convolution2d_1 (Convolution2D)  (None, 26, 26, 32)    320         convolution2d_input_1[0][0]      \n",
      "____________________________________________________________________________________________________\n",
      "activation_1 (Activation)        (None, 26, 26, 32)    0           convolution2d_1[0][0]            \n",
      "____________________________________________________________________________________________________\n",
      "convolution2d_2 (Convolution2D)  (None, 24, 24, 32)    9248        activation_1[0][0]               \n",
      "____________________________________________________________________________________________________\n",
      "activation_2 (Activation)        (None, 24, 24, 32)    0           convolution2d_2[0][0]            \n",
      "____________________________________________________________________________________________________\n",
      "maxpooling2d_1 (MaxPooling2D)    (None, 12, 12, 32)    0           activation_2[0][0]               \n",
      "____________________________________________________________________________________________________\n",
      "dropout_1 (Dropout)              (None, 12, 12, 32)    0           maxpooling2d_1[0][0]             \n",
      "____________________________________________________________________________________________________\n",
      "flatten_1 (Flatten)              (None, 4608)          0           dropout_1[0][0]                  \n",
      "____________________________________________________________________________________________________\n",
      "dense_1 (Dense)                  (None, 128)           589952      flatten_1[0][0]                  \n",
      "____________________________________________________________________________________________________\n",
      "activation_3 (Activation)        (None, 128)           0           dense_1[0][0]                    \n",
      "____________________________________________________________________________________________________\n",
      "dropout_2 (Dropout)              (None, 128)           0           activation_3[0][0]               \n",
      "____________________________________________________________________________________________________\n",
      "dense_2 (Dense)                  (None, 10)            1290        dropout_2[0][0]                  \n",
      "____________________________________________________________________________________________________\n",
      "activation_4 (Activation)        (None, 10)            0           dense_2[0][0]                    \n",
      "====================================================================================================\n",
      "Total params: 600810\n",
      "____________________________________________________________________________________________________\n"
     ]
    }
   ],
   "source": [
    "model.summary()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 绘制模型结构图，并保存成图片"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "plot(model, to_file='model-cnn.png')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 显示绘制的图片\n",
    "![image](http://p1.bpimg.com/4851/4c386bd0784d9b4d.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 5.训练与评估\n",
    "### 编译模型"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "model.compile(loss='categorical_crossentropy',\n",
    "              optimizer='adadelta',\n",
    "              metrics=['accuracy'])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 迭代训练"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Train on 60000 samples, validate on 10000 samples\n",
      "Epoch 1/12\n",
      "60000/60000 [==============================] - 13s - loss: 0.3825 - acc: 0.8821 - val_loss: 0.0861 - val_acc: 0.9734\n",
      "Epoch 2/12\n",
      "60000/60000 [==============================] - 12s - loss: 0.1333 - acc: 0.9609 - val_loss: 0.0611 - val_acc: 0.9801\n",
      "Epoch 3/12\n",
      "60000/60000 [==============================] - 12s - loss: 0.1041 - acc: 0.9694 - val_loss: 0.0545 - val_acc: 0.9824\n",
      "Epoch 4/12\n",
      "60000/60000 [==============================] - 12s - loss: 0.0861 - acc: 0.9736 - val_loss: 0.0441 - val_acc: 0.9852\n",
      "Epoch 5/12\n",
      "60000/60000 [==============================] - 12s - loss: 0.0780 - acc: 0.9767 - val_loss: 0.0410 - val_acc: 0.9862\n",
      "Epoch 6/12\n",
      "60000/60000 [==============================] - 12s - loss: 0.0701 - acc: 0.9795 - val_loss: 0.0384 - val_acc: 0.9869\n",
      "Epoch 7/12\n",
      "60000/60000 [==============================] - 12s - loss: 0.0624 - acc: 0.9815 - val_loss: 0.0378 - val_acc: 0.9873\n",
      "Epoch 8/12\n",
      "60000/60000 [==============================] - 10s - loss: 0.0608 - acc: 0.9821 - val_loss: 0.0348 - val_acc: 0.9893\n",
      "Epoch 9/12\n",
      "60000/60000 [==============================] - 11s - loss: 0.0556 - acc: 0.9838 - val_loss: 0.0337 - val_acc: 0.9885\n",
      "Epoch 10/12\n",
      "60000/60000 [==============================] - 12s - loss: 0.0535 - acc: 0.9841 - val_loss: 0.0316 - val_acc: 0.9899\n",
      "Epoch 11/12\n",
      "60000/60000 [==============================] - 12s - loss: 0.0512 - acc: 0.9849 - val_loss: 0.0306 - val_acc: 0.9896\n",
      "Epoch 12/12\n",
      "60000/60000 [==============================] - 12s - loss: 0.0458 - acc: 0.9867 - val_loss: 0.0321 - val_acc: 0.9896\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "<keras.callbacks.History at 0x7f261c07e8d0>"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "model.fit(X_train, Y_train, batch_size=batch_size, nb_epoch=nb_epoch,\n",
    "          verbose=1, validation_data=(X_test, Y_test))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 模型评估"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Test score: 0.0321088282784\n",
      "Test accuracy: 0.9896\n"
     ]
    }
   ],
   "source": [
    "score = model.evaluate(X_test, Y_test, verbose=0)\n",
    "print('Test score:', score[0])\n",
    "print('Test accuracy:', score[1])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 2",
   "language": "python",
   "name": "python2"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 2
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython2",
   "version": "2.7.12"
  },
  "ssap_exp_config": {
   "error_alert": "Error Occurs!",
   "initial": [],
   "max_iteration": 1000,
   "recv_id": "",
   "running": [],
   "summary": [],
   "version": "1.1.1"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 0
}