{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "# 卷积神经网络基础" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## 1. 计算机视觉\n", "\n", "随着深度学习的兴起，近些年来计算机视觉领域的发展十分迅猛。一方面，深度学习在计算机视觉的应用，使得很多几年前还无法解决的领域问题得以解决，比如人脸识别、自动驾驶；另一方面，在实践中计算机视觉社区对深度学习技巧的发现，也常常可以应用到其它领域里。\n", "\n", "计算机视觉领域的典型问题，包括图片分类、物体检测、神经风格迁移等。\n", "\n", "![Computer Vision Problems.png](img/Computer Vision Problems.png)\n", "\n", "由于图像数据的特点（高分辨率，特征多），非常容易多拟合，同时针对图像数据训练模型也需要极大的计算资源。而要使用高分辨率的图片训练模型，最好要实现卷积，卷积是卷积神经网络的基本组成单元。\n", "\n", "![Deep Learning on large images.png](img/Deep Learning on large images.png)" ] }, { "cell_type": "markdown", "metadata": { "collapsed": true }, "source": [ "## 2. 边缘检测实例\n", "\n", "在之前介绍的人类识别神经网络中，我们发现最终训练完成的神经网络会首先形成边缘检测的底层特征，在此基础上继续构建五官检测，进而再到人脸检测。可以说边缘检测是非常基础性的步骤。\n", "\n", "![Computer Vision Problem.png](img/Computer Vision Problem.png)\n", "\n", "卷积操作可以有效地进行边缘检测。卷积计算用到的这个3×3的矩阵，称为**过滤器 filter**，有些文献中也称为**核 kernel**。\n", "\n", "![Vertical edge detection.png](img/Vertical edge detection.png)\n", "\n", "卷积操作检测出来的边缘看起来比较粗，当输入图片的分辨率非常高时，这点损耗可以忽略不计。\n", "\n", "![Vertical edge detection 2.png](img/Vertical edge detection 2.png)" ] }, { "cell_type": "markdown", "metadata": { "collapsed": true }, "source": [ "## 3. 更多边缘检测的内容\n", "\n", "上面提到的过滤器，还可以区分由浅到深的边缘和由深到浅的边缘。\n", "\n", "![Vertical edge detection examples.png](img/Vertical edge detection examples.png)\n", "\n", "而将这个过滤器的矩阵转置，就是水平边缘检测的过滤器。\n", "\n", "![Vertical and Horizontal Edge Detection.png](img/Vertical and Horizontal Edge Detection.png)\n", "\n", "除去水平边缘和垂直边缘过滤器外，计算机视觉社区还发明过Sober过滤器和Scharr过滤器，它们分别有一些自己的特性。而在深度学习时代，一个非常重要的理念是，过滤器本身可以作为神经网络的参数，在反向传播的过程中进行学习。这样最终学得的边缘检测过滤器，可能不限于水平或垂直边缘，而是可以检测任意倾斜的边缘。\n", "\n", "![Learning to detect edges.png](img/Learning to detect edges.png)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## 4. 补全\n", "\n", "经过卷积操作，图片的分辨率会降低。如果原图是n×n的矩阵，而过滤器是f×f的矩阵，卷积之后的矩阵就变为了(n-f+1)×(n-f+1)维。这样有两个坏处：1）随着每一层神经网络的卷积计算，图片的大小都在不断缩小，限制了训练过大的神经网络；2）角和边上的像素点，参与卷积计算的次数会更少，从而造成边角的信息利用率低。所以实际使用中，卷积通常伴随着补全。\n", "\n", "![Padding.png](img/Padding.png)\n", "\n", "根据使用补全的策略，区分**正确卷积 Valid convolution**和**同一卷积 Same convolution**。所谓正确卷积，就是不包含补全，任由图片大小缩减；而同一卷积，是先进行补全，使得最终输出的图片大小和输入一致。注意要同一卷积的要求，使得 $p=\\frac{f-1}{2}$。这就要求过滤器是一个奇数维的矩阵，否则补全就需要是非对称的。过滤器是奇数维矩阵的另一个好处，是过滤器存在一个中心像素，方便定位位置。\n", "\n", "![Valid and Same convolutions.png](img/Valid and Same convolutions.png)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## 5. 步长\n", "\n", "前面看到的卷积操作，过滤器每次都只移动一格。而引入步长的概念之后，过滤器每次可以移动不只一格。\n", "\n", "![Strided convolution.png](img/Strided convolution.png)\n", "\n", "在有补全和步长的情况下，输出的数据量大小也会有所变化。\n", "\n", "![Summary of convolutions.png](img/Summary of convolutions.png)\n", "\n", "从严格数学的定义来说，实际上我们上面用到的应该称为**交叉相关性 cross-correlation**，而真正的卷积，在交叉相关性之前，还需要先进行垂直和水平的翻转，这样可以使得卷积服从结合律。不过这个特性对于神经网络意义不大（对于信号处理中使用卷积比较有用），所以在深度学习社区，实际上使用卷积时，并不会进行翻转，但是从命名习惯上，依然将其称之为卷积。\n", "\n", "![Technical note on cross-correlation vs convolution.png](img/Technical note on cross-correlation vs convolution.png)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## 6. 高维空间的卷积\n", "\n", "对于图片，如果要处理RGB值，就会有三个n×n的矩阵，形成一个n×n×3的立方体，这时相应的，过滤器也变成了一个f×f×3的立方体，最终输出仍然是一个矩阵。\n", "\n", "![Convolutions on RGB image.png](img/Convolutions on RGB image.png)\n", "\n", "在需要的情况下，也可以同时使用多个过滤器。\n", "\n", "![Multiple filters.png](img/Multiple filters.png)" ] }, { "cell_type": "markdown", "metadata": { "collapsed": true }, "source": [ "## 7. 一层卷积神经网络\n", "\n", "![Example of a layer.png](img/Example of a layer.png)\n", "\n", "![Summary of notation.png](img/Summary of notation.png)" ] }, { "cell_type": "markdown", "metadata": { "collapsed": true }, "source": [ "## 8. 简单的卷积神经网络示例\n", "\n", "一个39×39像素，RGB三通道表示的图片，经过三个卷积层，最后叠加一层logistic或softmax的卷机神经网络分类模型。注意到随着层数的增加，图片的像素在下降，而通道数在上升，这也是超参选择（每一层的过滤器大小f，过滤器数量$n_c$，步长s，补全p）的一个普遍趋势。\n", "\n", "![Example ConvNet.png](img/Example ConvNet.png)\n", "\n", "实际上，一个典型的卷积神经网络，除去卷积层之外，还会包含池化层(Pooling)，完全连接层(Fully connected)。下面会详细介绍。" ] }, { "cell_type": "markdown", "metadata": { "collapsed": true }, "source": [ "## 9. 池化层\n", "\n", "在过滤器区间内，取最大值，称为最大池化层。\n", "\n", "![Pooling layer Max pooling.png](img/Pooling layer Max pooling.png)\n", "\n", "池化层和卷积层的一个显著不同，是池化层的过滤器是针对通道独立的，不会跨通道取最大值。过滤器也包含两个超参，分别是f过滤器大小和步长s。注意和卷积层的另一个不同，是池化层的过滤器，只有超参，没有参数。所以在反向传播的过程中，这一层是没有更新的。\n", "\n", "![Pooling layer Max pooling 2.png](img/Pooling layer Max pooling 2.png)\n", "\n", "同时也有平均池化层，但平均池化层不常用。\n", "\n", "![Pooling layer Average pooling.png](img/Pooling layer Average pooling.png)" ] }, { "cell_type": "markdown", "metadata": { "collapsed": true }, "source": [ "## 10. 卷积神经网络实例\n", "\n", "上面说到，池化层是没有需要学习的参数的，而且池化层通常会跟在一层或多层卷积层之后，所以习惯上会将卷积层和池化层一起称为神经网络的一层。而所谓的完全连接层FC，就是普通的神经网络的一层，有权重和截距作为参数。\n", "\n", "下图是一个卷积-池化-卷积-池化-全连接-全连接-全连接-Softmax的卷积神经网络的例子。设计良好的卷积神经网络架构，每一层激活值的数量在前向传播的过程中，通常是逐渐递减的。\n", "\n", "![Convolution Neural Network Example.png](img/Convolution Neural Network Example.png)" ] }, { "cell_type": "markdown", "metadata": { "collapsed": true }, "source": [ "## 11. 为何卷积有效\n", "\n", "卷积层相比全连接层，需要学习的参数大幅减少，有以下两个原因：\n", " - 参数共享：一个特征检测器（比如垂直边缘检测器），可能不只对图像的单一区域有效。过滤器平移的过程，就是参数共享的过程。\n", " - 连接的稀疏性：每一层的输出值都只依赖与几个输入值。（与全连接层相比，所有输入和所有输出都是连通的）\n", " \n", "![Why convolutions.png](img/Why convolutions.png)" ] }, { "cell_type": "markdown", "metadata": { "collapsed": true }, "source": [ "## 12. 卷积神经网络：一步一步地搭建卷积模型\n", "\n", "这里我们将使用numpy来实现卷积神经网络的卷积层（CONV）和池化层（POOL），前向传播与反向传播。\n", "\n", "**标记**:\n", "- 上标 $[l]$ 表示第 $l^{th}$ 层的对应变量。 \n", " - 举例: $a^{[4]}$ 是第 $4^{th}$ 层激活层。 $W^{[5]}$ 和 $b^{[5]}$ 是第 $5^{th}$ 层的参数。\n", "\n", "\n", "- 上标 $(i)$ 表示第 $i^{th}$ 个样本的对应变量。 \n", " - 举例: $x^{(i)}$ 是第 $i^{th}$ 个训练样本的输入。\n", " \n", " \n", "- 下标 $i$ 表示向量的第 $i^{th}$ 个元素。\n", " - 举例: $a^{[l]}_i$ 表示第 $l$ 层的激活值的第 $i^{th}$ 个元素，假定这一层是全连接层（FC）。\n", " \n", " \n", "- $n_H$, $n_W$ 和 $n_C$ 分别表示给定层的高度、宽度和通道数。当需要确定地表示是第 $l$ 层时，可以写作\n", "$n_H^{[l]}$, $n_W^{[l]}$, $n_C^{[l]}$。\n", "- $n_{H_{prev}}$, $n_{W_{prev}}$ 和 $n_{C_{prev}}$ 分别表示上一层的高度、宽度和通道数。当需要确定地表示是第 $l$ 层对应的前一层相应参数时，可以写作 $n_H^{[l-1]}$, $n_W^{[l-1]}$, $n_C^{[l-1]}$。" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### 12.1 三方包\n", "\n", "首先引入在这个编程练习中所需要的包。\n", "- [numpy](www.numpy.org) 是Python生态圈中进行科学计算的基础包。\n", "- [matplotlib](http://matplotlib.org) 是Python生态圈中著名的绘图包。\n", "- np.random.seed(1) 用来保证所有函数调用中随机部分的一致性。" ] }, { "cell_type": "code", "execution_count": 1, "metadata": { "collapsed": true }, "outputs": [], "source": [ "import numpy as np\n", "import h5py\n", "import matplotlib.pyplot as plt\n", "\n", "%matplotlib inline\n", "plt.rcParams['figure.figsize'] = (5.0, 4.0) # set default size of plots\n", "plt.rcParams['image.interpolation'] = 'nearest'\n", "plt.rcParams['image.cmap'] = 'gray'\n", "\n", "%load_ext autoreload\n", "%autoreload 2\n", "\n", "np.random.seed(1)" ] }, { "cell_type": "markdown", "metadata": { "collapsed": true }, "source": [ "### 12.2 总览\n", "\n", "我们将会实现卷积神经网络的各个组件：\n", "\n", "- 卷积函数，包括：\n", " - 零补全 Zero Padding\n", " - 卷积窗口 Convolve window \n", " - 前向卷积 Convolution forward\n", " - 反向卷积 Convolution backward (optional)\n", "- 池化函数，包括:\n", " - 前向池化 Pooling forward\n", " - 创建掩码 Create mask \n", " - 分布值 Distribute value\n", " - 反向池化 Pooling backward (optional)\n", "\n", "这一节的内容都基于 `numpy` 从头开始实现；而在下一节中，我们会用Tensorflow来实现同样的模型。\n", "\n", "\n", "\n", "**注意** 每一个前向传播的函数，都有对应的反向传播的函数。因此，前向传播模块中的每一步，都需要将相应的参数保存到缓存中。这些参数将在反向传播过程中用于计算梯度。" ] }, { "cell_type": "markdown", "metadata": { "collapsed": true }, "source": [ "### 12.3 卷积神经网络\n", "\n", "编程框架使得卷积非常容易使用，但卷积的概念依然是深度学习中最难理解的概念之一。卷积层将输入立方体转换为另一个尺寸的输出立方体，如下图所示：\n", "\n", "\n", "\n", "在这一节，我们会逐步构建起一个完整的卷积层。首先，我们需要实现两个辅助函数：零补全和卷积计算。" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "#### 12.3.1 零补全\n", "\n", "零补全围绕着图像的边框补零值：\n", "\n", "\n", "
**Figure 1** : **Zero-Padding**
Image (3 channels, RGB) with a padding of 2.
\n", "\n", "补全的主要好处包括：\n", "\n", "- 使用卷积层时，高度和宽度不会缩减，这点对于构建深度网络来说十分重要。而一个重要的应用实例就是同一补全的卷积层，卷积计算前后，高度和宽度都不变。\n", "\n", "- 使得图片边框的信息得到充分利用。没有补全时，下一层只有很少的数值会收到当前图片的边角像素的影响。\n", "\n", "**练习**: 实现下面的函数，批量将样本 X 进行零补全。[使用 np.pad](https://docs.scipy.org/doc/numpy/reference/generated/numpy.pad.html)。注意到如果想要给形状为 $(5,5,5,5,5)$ 的数组 \"a\" 在第二维度补 1，在第四维度补 3，其它维度补 0，可以这样写：\n", "```python\n", "a = np.pad(a, ((0,0), (1,1), (0,0), (3,3), (0,0)), 'constant', constant_values = (..,..))\n", "```" ] }, { "cell_type": "code", "execution_count": 2, "metadata": { "collapsed": true }, "outputs": [], "source": [ "# GRADED FUNCTION: zero_pad\n", "\n", "def zero_pad(X, pad):\n", " \"\"\"\n", " Pad with zeros all images of the dataset X. The padding is applied to the height and width of an image, \n", " as illustrated in Figure 1.\n", " \n", " Argument:\n", " X -- python numpy array of shape (m, n_H, n_W, n_C) representing a batch of m images\n", " pad -- integer, amount of padding around each image on vertical and horizontal dimensions\n", " \n", " Returns:\n", " X_pad -- padded image of shape (m, n_H + 2*pad, n_W + 2*pad, n_C)\n", " \"\"\"\n", " \n", " ### START CODE HERE ### (≈ 1 line)\n", " X_pad = np.pad(X, ((0, 0), (pad, pad), (pad, pad), (0, 0)), 'constant')\n", " ### END CODE HERE ###\n", " \n", " return X_pad" ] }, { "cell_type": "code", "execution_count": 3, "metadata": { "collapsed": false }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "x.shape = (4, 3, 3, 2)\n", "x_pad.shape = (4, 7, 7, 2)\n", "x[1,1] = [[ 0.90085595 -0.68372786]\n", " [-0.12289023 -0.93576943]\n", " [-0.26788808 0.53035547]]\n", "x_pad[1,1] = [[ 0. 0.]\n", " [ 0. 0.]\n", " [ 0. 0.]\n", " [ 0. 0.]\n", " [ 0. 0.]\n", " [ 0. 0.]\n", " [ 0. 0.]]\n" ] }, { "data": { "text/plain": [ "" ] }, "execution_count": 3, "metadata": {}, "output_type": "execute_result" }, { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAUEAAACqCAYAAAAz+v3EAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAEtRJREFUeJzt3X2wXHV9x/H3J2AYSEiUBxNLCAoIKY42pGMkk6ZEBQzY\nEv+w9anloVUZhcrUjg9NmTHO2FT/cBQURzEYedBKZYQEJW3iQKVBEiNJIEqCYQATYhIRSGgSK4F8\n+8c5N7Pc7MPZ3d/ds3fP5zWzc8/u/u73fO/u2e89Z8/v/H6KCMzMqmpM2QmYmZXJRdDMKs1F0Mwq\nzUXQzCrNRdDMKs1F0MwqzUXQzFqSdKmk/yk7j5HgImhmRQ1kp2IXQTOrNBfBkkk6VdIzkqbn9/9I\n0m8l/XnZuVn/6GQ7kXSvpEWS1kjaI+kOSa+sef4/JO2Q9Jyk/5Z0Vs1zx0lalv/eauC0Ef0DS+Qi\nWLKIeBz4JHCrpKOBJcCSiLiv3Mysn3SxnfwtcBkwGXgJ+ErNc3eTFbdXA+uA79Q89zVgPzAJ+Hvg\n77r/K/qTfO1wf5B0J3AqcBB4c0QcKDkl60PtbCeS7gUeiIgF+f0/BtYDR8ewD36+h/gsMBHYB/wf\n8IaI2JI//6/AnIgYuCMU7wn2j8XAG4CvuABaE+1uJ9tqln8NjAVOkDRG0uclPSZpN/AE2YmPE4AT\ngSOAp4b97kByEewDksYBXwZuBBbWfm9jNqTD7eTkmuVTgBeA3wEfAP4SeFtEvBJ4LaD89jTw4rDf\nndpt/v3KRbA/XAf8LCI+TPY9zTdKzsf6Uyfbyd9ImibpGOCzwPfzQ+HxwB+A5/Li+m/kXWAi4iDw\nA7JCe3R+wuTS9H9Of3ARLJmki4ELgI/mD30cOFvS+8rLyvpNF9vJLcBNwG/IDoWvzh+/GdgKbAd+\nAfx02O/9A3AssAP4Vn4bSF2dGJH0KuA2st3sJ4G/jog9ddo9Cewh+zL3QETM7HilZlZIfmLklogY\n2AKWQrd7gp8GfhwRZwL3AP/coN1BYG5EnO0CaGb9pNsiOJ9sV5v857satFOCdZnZMJL+V9LzNbeh\n+7MZ0MvcUuv2cPjZiDiu0f2axx8HdpN11rwhIr7Z8UrNzBI6slUDSSvJeo0feojsP8w1dZo3qqiz\nI2KHpBOBlZI2RcSqtrM1M0usZRGMiPMbPSdpl6RJEbFL0mTgtw1i7Mh/Pi3pDmAmULcISvIufEVF\nhEZ6Hd6+qq3eNtayCLawjOy6xC+Q9SNaOrxB3j9pTETszfsjXUDWX6mh559/vsu0Xm7RokUsWLAg\nacwJEyYkjTeSFi9enDTe0qVLmT9/ftKYH/zgB5PGa2b27NlNn9+6dStTp3bfN9hxehOnaKz777+/\n7uPdnqz4AnC+pEeBtwOfB5D0Gkk/zNtMAlZJWg+sBu6KiBVdrtfMLImu9gQj4lngvDqP7wD+Il9+\nApjezXrMzEZKJbqtzJkzp+wUBsqZZ55Zdgp1SZonabOkX0n6VKdxJk6cmCQfx+lNnG5juQha26ZN\nm1Z2CoeRNAb4KvAOslFW3iepo0T77UPuOCMbqxJF0CphJrAlIn6dDzH1PbLO/GZNuQjaoDiJl4+d\n91T+mFlTLoJmVmnd9hM06xfbefnAn1Pyxw6zdevWQ8sTJ05M+t2U9Y89e/awZ89hg1odxkXQBsVa\n4HRJp5CNgfdeoO5Ye6k66Fp/G/4Pbtu2bXXbuQjaQIiIlyRdBawg+5rnxojYVHJaNgq4CNrAiIj/\nBPqzE6P1LZ8YMbNKcxE0s0pzETSzSktSBItcsynpOklbJG2Q5AEVzKwvdF0Ei1yzKelC4LSIeD1w\nBfD1btdrZpZCij3BItdszieb55SIWANMlDQJM7OSpSiCRa7ZHN5me502ZmY95xMjZlZpKTpLF7lm\ncztwcos2hyxatOjQ8pw5czwe4ADavHkzjz76aNlpmCUpgkWu2VwGXAncJukcYHdE7GoUMPWkSNZ/\npk2b9rLBWe+6666uY0q6kWxah10R8aauA1oldH04HBEvAUPXbP4S+F5EbJJ0haQP523uBp6Q9Bjw\nDeCj3a7XrI4lZL0UzApLcu1wvWs2I+Ibw+5flWJdZo1ExKr8iMSsMJ8YMbNKcxE0s0rzUFpWOR5Z\nuho8srRVlfJbQx5ZuhqKjiztw2EbGJK+C/wUOEPSVkmXl52T9T/vCdrAiIj3l52DjT7eEzSzSnMR\nNLNKcxE0s0pzETSzSnMRNLNK89lhs5ItX748SZwJEyYkibN48eIkcZYsWZIkzkjryURLks6VtFvS\nuvx2TYr1mpl1q+s9wZqJlt4O/AZYK2lpRGwe1vS+iLi42/WZmaXUq4mWoMWlTGZmZejVREsAs/I5\nh38k6awE6zU7RNIUSfdI+qWkjZI+VnZONjr06sTIg8DUiNifz0F8J3BGj9Zt1fAi8PGI2CBpPPCg\npBV1vpYxe5meTLQUEXtrlpdL+pqk4yLi2XoBv/jFLx5anjt3LnPnzk2QZlqXXnpp2SkUdt5555Wd\nwmEeeOABVq9enSxeROwEdubLeyVtIjsicRG0pnoy0ZKkSUMTK0maCahRAQRYuHBhgrSsn82aNYtZ\ns2Ydun/ttdcmiy3ptcB0YE2yoDawui6CEfGSpKGJlsYANw5NtJQ9HTcA75b0EeAA8HvgPd2u16ye\n/FD4duDq2iMQs0Z6MtFSRFwPXJ9iXWaNSDqSrADeEhFLG7XzyNLV4JGlrYq+BTwSEU2PrT2ydDV4\nZGmrFEmzgQ8Ab5O0Pr8yaV7ZeVn/856gDYSIuB84ouw8bPTxnqCZVZqLoJlVmougmVWai6CZVZqL\noJlVms8Om5Xs2GOPTRIn1fXsqa41r9TI0mZmo5WLoJlVmougmVWai6CZVVqq2eZulLRL0sNN2lwn\naUs+xP70FOs1GyLpKElr8uuGN0r6TNk52eiQak9wCfCORk/mQ+qfFhGvB64Avp5ovWYARMQfgLdG\nxNlkA6pemA/ga9ZUkiIYEauA55o0mQ/cnLddA0yUNCnFus2GRMT+fPEosu5fUWI6Nkr06jvB4TPS\nbaf+jHRmHZM0RtJ6srlGVkbE2rJzsv7Xl52la+cY6deJlqw7qSdaAoiIg8DZkiYAd0o6KyIeGd7O\nI0tXQ7+NLL0dOLnm/mEz0tXyREuDbyQnWoqI5yXdC8wDDiuCHlm6GsoYWVr5rZ5lwCUAks4Bdg/N\nPmeWgqQTJE3Ml48GzsfTbVoBSfYEJX0XmAscL2kr8BlgLPlscxFxt6SLJD0G7AMuT7FesxqvAW6S\nNIbsn/ttEXF3yTnZKJBqtrn3F2hzVYp1mdUTERuBGWXnYaOPrxgxs0pzETSzSnMRNLNKcxE0s0rr\ny87SZlUyefLkJHFuvfXWJHHmzUszZ/3xxx+fJM5I856gmVWai6CZVZqLoJlVmougmVWai6ANlHw4\nrXWSlpWdi40OLoI2aK6mzsgxZo24CNrAkDQFuAhYXHYuNnr0ZKIlSedK2p0fpqyTdE2K9ZoN8yXg\nE3hYfWtDTyZayt0XETPy2+cSrdcMAEnvBHZFxAaaj21p9jKphtJaJemUFs28UdpImg1cLOki4Gjg\nWEk3R8Qlwxt6eP1q6Lfh9QFmSdpANqz+J+rN/WDWqYhYACyA7OsX4J/qFUDw8PpVUXR4/V4VwQeB\nqRGxP5+D+E7gjEaNU11LOZJSXafZC6muBTUbRD0pghGxt2Z5uaSvSTouIp6t137v3kPNGTt2LGPH\nju1BltZL+/btY//+/a0bdiAifgL8ZESC28BJWQQbfhktadLQxEqSZgJqVAABxo8fnzAt60fjxo1j\n3Lhxh+4/88wzJWZjVdaTiZaAd0v6CHAA+D3wnhTrNTPrVk8mWoqI64HrU6zLzCwlXzFiZpXmkaXN\nSnb66acnibNw4cIkcUbLiNCpeE/QzCrNRdDMKs1F0MwqzUXQzCrNJ0ZsYEh6EtgDHAQORMTMcjOy\n0cBF0AbJQWBuRDxXdiI2evhw2AaJ8DZtbfIGY4MkgJWS1kr6UNnJ2Ojgw2EbJLMjYoekE8mK4aaI\nWFV2UtbfXARtYETEjvzn05LuAGYChxVBjyxdDT0bWTqf4etmYBLZF9PfjIjr6rS7DrgQ2Adcls8F\nYZaEpGOAMRGxV9I44ALgs/XaemTpaujlyNIvAh+PiA2SxgMPSloREZuHGuSjSZ8WEa+X9Bbg68A5\nCdZtNmQScIekINuuvxMRK0rOyUaBrotgROwEdubLeyVtAk4CNtc0m0+2t0hErJE0sXagVbNuRcQT\nwPSy87DRJ+nZYUmvJdsQ1wx76iSgdl90e/6YmVmpkhXB/FD4duDq2jlFzMz6Warh9Y8kK4C3RMTS\nOk22AyfX3J+SP1aXJ1oafCM50ZJZO1J1kfkW8EhEXNvg+WXAlcBtks4Bdjf7PtATLQ0+T7Rk/SJF\nF5nZwAeAjZLWk/XaXwCcQj7RUkTcLekiSY+RdZG5vNv1mpmlkOLs8P3AEQXaXdXtuszMUvO1w2ZW\naS6CZlZpLoJmVmkugjYw8iuRvi9pk6Rf5pdomjXlUWRskFwL3B0Rf5X3XT2m7ISs/7kI2kCQNAGY\nExGXAUTEi8DzpSZlo4IPh21QvA74naQlktZJukHS0WUnZf3PRdAGxZHADOD6iJgB7Ac+XW5KNhr4\ncNgGxVPAtoj4eX7/duBT9Rp6ZOlq6NnI0mb9ICJ2Sdom6YyI+BXwduCRem09snQ19HJkabN+8THg\nO5JeATyOr1G3AlwEbWBExEPAm8vOw0aXrk+MSJoi6Z68c+pGSR+r0+ZcSbvzs3brJF3T7XrNzFJI\ncXZ4aKKlNwCzgCslTavT7r6ImJHfPpdgvYW98MILyWM+9NBDlY25b9++5DH7SZEv0x2nf+J0G6vr\nIhgRO4emz8yH1R+aaGk4dbuuTrkIpjXoI0L324fccUY2Vq8mWgKYJWmDpB9JOivles3MOpXsxEiL\niZYeBKZGxP58DuI7gTNSrdvMrFOKiO6DZBer/xBY3mSekdr2TwB/GhHP1nmu+4RsVIqIEf/KxNtX\ntdXbxnoy0VLtROuSZpIV38MKYKMkzVLx9mXD9WSiJeDdkj4CHAB+D7yn2/WamaWQ5HDYzGy0KnUU\nGUmvkrRC0qOS/ktS3SvZJT0p6SFJ6yX9rEGbeZI2S/qVpLoXzku6TtKW/Cz19AL5NY3ZSSdwSTdK\n2iXp4SZt2s2zacwO82zZCb7dXPu9Y32RbahgnJbvccE4hd6DAnGOkrQm//xslPSZLvMak783y7qI\n0fIzXTBO96OJR0RpN+ALwCfz5U8Bn2/Q7nHgVU3ijAEeIzsEfwWwAZg2rM2FwI/y5bcAq1vkViTm\nucCyNv/mPyPrRvRwg+fbyrNgzE7ynAxMz5fHA48meE2LxGw710TbYsv3O9V7nPI9aCPWMfnPI4DV\nwMwu8vpH4NZu3qdWn+k24nwbuDxfPhKY0G6MsscTnA/clC/fBLyrQTvRfK91JrAlIn4dEQeA7+Wx\nh6/rZoCIWANMlDSpy5hDuRUWEauA55o0aTfPIjE7ybNIJ/i2ci0Ys+1cEyn6frdU8P0oEqfo61Uk\n1lAP96PIikVH34NJmgJcBCzu5PdrQ9HlkWjNaOJLIBtNPCLaHk287CL46sjPGkfETuDVDdoFsFLS\nWkkfqvP8SUDtODlPcfjGMrzN9jpt2o0J6TuBt5tnUR3n2aQTfMe59mHH+qLvdylavF5Ffn9MfuJy\nJ7AyItZ2mMqXgE/QYRGt0eozXUSS0cRHfBQZSSuB2r0Dkb0A9b7rafTCzo6IHZJOJHvhNuX/bcs2\nWjqBd5ynmneC70iLmKPlNe2ZFO9BRBwEzs73nu6UdFZE1B1vsUke7wR2RcQGSXPpbo89xWd6aDTx\nKyPi55K+TDaaeFvfeY74nmBEnB8Rb6q5vTH/uQzYNXT4JGky8NsGMXbkP58G7iA7dKm1HagdKXNK\n/tjwNie3aNNWzIjYO3SYERHLgVdIOq5JzCLazbOlTvNU1gn+duCWiFiaItdWMUfoNS2iyDbUcwXe\ng7bkh4v3AvM6+PXZwMWSHgf+HXirpJs7zKPVZ7qIeqOJz2g3SNmHw8uAy/LlS4HD3mRJx+T/CZE0\nDrgA+MWwZmuB0yWdImks8N489vB1XZLHOQfYPXQo3kDLmLXff6lFJ/DhfxaN/4u2m2fLmF3k2bQT\nfIe5tuxY32Gu3SqyDbWj2XvcjlbvQetEpBOU97zIDxfPBza3GyciFkTE1Ig4lez1uSciLukgnyKf\n6SL57AK2SRo6Umg4mnirQKXdgOOAH5Od9VoBvDJ//DXAD/Pl15GdqVsPbAQ+3SDWvDzOlqE2wBXA\nh2vafJXsDOBDwIwC+TWNCVxJ9uatB34KvKVAzO8CvwH+AGwlG/242zybxuwwz9nASzWv/br89eg4\n1yIxO8k14fZ42PvdYZzD3o8O49R9vTqI88b8dzcADwP/kuC16vgsftHPdMFYf0L2D2wD8ANgYrsx\n3FnazCqt7MNhM7NSuQiaWaW5CJpZpbkImlmluQiaWaW5CJpZpbkImlmluQiaWaX9P0BfssqyjIfe\nAAAAAElFTkSuQmCC\n", "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "np.random.seed(1)\n", "x = np.random.randn(4, 3, 3, 2)\n", "x_pad = zero_pad(x, 2)\n", "print (\"x.shape =\", x.shape)\n", "print (\"x_pad.shape =\", x_pad.shape)\n", "print (\"x[1,1] =\", x[1,1])\n", "print (\"x_pad[1,1] =\", x_pad[1,1])\n", "\n", "fig, axarr = plt.subplots(1, 2)\n", "axarr[0].set_title('x')\n", "axarr[0].imshow(x[0,:,:,0])\n", "axarr[1].set_title('x_pad')\n", "axarr[1].imshow(x_pad[0,:,:,0])" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "**预期输出**:\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "\n", "
\n", " **x.shape**:\n", " \n", " (4, 3, 3, 2)\n", "
\n", " **x_pad.shape**:\n", " \n", " (4, 7, 7, 2)\n", "
\n", " **x[1,1]**:\n", " \n", " [[ 0.90085595 -0.68372786]\n", " [-0.12289023 -0.93576943]\n", " [-0.26788808 0.53035547]]\n", "
\n", " **x_pad[1,1]**:\n", " \n", " [[ 0. 0.]\n", " [ 0. 0.]\n", " [ 0. 0.]\n", " [ 0. 0.]\n", " [ 0. 0.]\n", " [ 0. 0.]\n", " [ 0. 0.]]\n", "
" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "#### 12.3.2 单步卷积\n", "\n", "这节我们需要实现单步卷积，即将过滤器应用在输入的一个给定位置。单步卷积会用来构建一个卷积单元，卷积单元的作用是：\n", "\n", "- 取一个输入立方体\n", "- 将过滤器应用到输入的每个给定位置\n", "- 输出另一个立方体（通常会发生尺寸改变）\n", "\n", "\n", "
**Figure 2** : **Convolution operation**
with a filter of 2x2 and a stride of 1 (stride = amount you move the window each time you slide)
\n", "\n", "在计算机视觉应用中，左边矩阵中的每个值都对应着一个像素值。卷积的过程中，这个 3x3 的过滤器依次和原矩阵中对应位置做元素相乘，再求和，然后加上截距项。首先，我们要实现单步卷积，对应着将过滤器和矩阵中一个位置进行计算，并获得唯一实数值的过程。\n", "\n", "之后，我们会使用这个函数，对输入的不同位置进行计算，从而实现完整的卷积过程。\n", "\n", "**练习**: 实现 conv_single_step(). [提示](https://docs.scipy.org/doc/numpy-1.13.0/reference/generated/numpy.sum.html).\n" ] }, { "cell_type": "code", "execution_count": 4, "metadata": { "collapsed": true }, "outputs": [], "source": [ "# GRADED FUNCTION: conv_single_step\n", "\n", "def conv_single_step(a_slice_prev, W, b):\n", " \"\"\"\n", " Apply one filter defined by parameters W on a single slice (a_slice_prev) of the output activation \n", " of the previous layer.\n", " \n", " Arguments:\n", " a_slice_prev -- slice of input data of shape (f, f, n_C_prev)\n", " W -- Weight parameters contained in a window - matrix of shape (f, f, n_C_prev)\n", " b -- Bias parameters contained in a window - matrix of shape (1, 1, 1)\n", " \n", " Returns:\n", " Z -- a scalar value, result of convolving the sliding window (W, b) on a slice x of the input data\n", " \"\"\"\n", "\n", " ### START CODE HERE ### (≈ 2 lines of code)\n", " # Element-wise product between a_slice and W. Do not add the bias yet.\n", " s = a_slice_prev * W\n", " # Sum over all entries of the volume s.\n", " Z = np.sum(s)\n", " # Add bias b to Z. Cast b to a float() so that Z results in a scalar value.\n", " Z = Z + float(b)\n", " ### END CODE HERE ###\n", "\n", " return Z" ] }, { "cell_type": "code", "execution_count": 5, "metadata": { "collapsed": false }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Z = -6.99908945068\n" ] } ], "source": [ "np.random.seed(1)\n", "a_slice_prev = np.random.randn(4, 4, 3)\n", "W = np.random.randn(4, 4, 3)\n", "b = np.random.randn(1, 1, 1)\n", "\n", "Z = conv_single_step(a_slice_prev, W, b)\n", "print(\"Z =\", Z)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "**预期输出**:\n", "\n", " \n", " \n", " \n", " \n", "\n", "
\n", " **Z**\n", " \n", " -6.99908945068\n", "
" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "#### 12.3.3 卷积神经网络：前向传播\n", "\n", "在前向传播的过程中，我们需要接收多个过滤器，并使用它们对输入进行卷积过程的计算。每个过滤器产生的卷积都会输出一个2D矩阵。将这些矩阵叠加起来，就得到了一个3D的立方体：\n", "\n", "
\n", "\n", "
\n", "\n", "**练习**: 实现下面的函数，将一组过滤器 W 应用到输入 A_prev。A_prev 是上一层的激活输出值（包括共 m 个样本），W 表示过滤器/权重，b 表示截距向量，每个过滤器都有自己单独的解决。我们也可以访问超参字典，获取补全量和步长。\n", "\n", "**提示**: \n", "1. 要在矩阵 \"a_prev\" (shape (5,5,3)) 中选取左上角 2×2 的子矩阵，可以使用：\n", "```python\n", "a_slice_prev = a_prev[0:2,0:2,:]\n", "```\n", "在下面定义 `a_slice_prev` 时，定义 `start/end` 索引，在使用上面的方法即可。\n", "2. 要定义 a_slice，首先需要定义其四个角的坐标 `vert_start`, `vert_end`, `horiz_start` 和 `horiz_end`。下面的图展示了如果使用 h, w, f 和 s 来计算这四个坐标。\n", "\n", "\n", "
**Figure 3** : **Definition of a slice using vertical and horizontal start/end (with a 2x2 filter)**
This figure shows only a single channel.
\n", "\n", "\n", "**提醒**:\n", "输出尺寸和输入尺寸的关系如下：\n", "$$ n_H = \\lfloor \\frac{n_{H_{prev}} - f + 2 \\times pad}{stride} \\rfloor +1 $$\n", "$$ n_W = \\lfloor \\frac{n_{W_{prev}} - f + 2 \\times pad}{stride} \\rfloor +1 $$\n", "$$ n_C = \\text{卷积中使用的过滤器数量}$$\n", "\n", "在这个练习中，我们不考虑向量化，仅简单使用for循环来实现。" ] }, { "cell_type": "code", "execution_count": 6, "metadata": { "collapsed": true }, "outputs": [], "source": [ "# GRADED FUNCTION: conv_forward\n", "\n", "def conv_forward(A_prev, W, b, hparameters):\n", " \"\"\"\n", " Implements the forward propagation for a convolution function\n", " \n", " Arguments:\n", " A_prev -- output activations of the previous layer, numpy array of shape (m, n_H_prev, n_W_prev, n_C_prev)\n", " W -- Weights, numpy array of shape (f, f, n_C_prev, n_C)\n", " b -- Biases, numpy array of shape (1, 1, 1, n_C)\n", " hparameters -- python dictionary containing \"stride\" and \"pad\"\n", " \n", " Returns:\n", " Z -- conv output, numpy array of shape (m, n_H, n_W, n_C)\n", " cache -- cache of values needed for the conv_backward() function\n", " \"\"\"\n", " \n", " ### START CODE HERE ###\n", " # Retrieve dimensions from A_prev's shape (≈1 line) \n", " (m, n_H_prev, n_W_prev, n_C_prev) = A_prev.shape\n", " \n", " # Retrieve dimensions from W's shape (≈1 line)\n", " (f, f, n_C_prev, n_C) = W.shape\n", " \n", " # Retrieve information from \"hparameters\" (≈2 lines)\n", " stride = hparameters[\"stride\"]\n", " pad = hparameters[\"pad\"]\n", " \n", " # Compute the dimensions of the CONV output volume using the formula given above. Hint: use int() to floor. (≈2 lines)\n", " n_H = int((n_H_prev - f + 2 * pad) / stride) + 1\n", " n_W = int((n_W_prev - f + 2 * pad) / stride) + 1\n", " \n", " # Initialize the output volume Z with zeros. (≈1 line)\n", " Z = np.zeros((m, n_H, n_W, n_C))\n", " \n", " # Create A_prev_pad by padding A_prev\n", " A_prev_pad = zero_pad(A_prev, pad)\n", " \n", " for i in range(m): # loop over the batch of training examples\n", " a_prev_pad = A_prev_pad[i] # Select ith training example's padded activation\n", " for h in range(n_H): # loop over vertical axis of the output volume\n", " for w in range(n_W): # loop over horizontal axis of the output volume\n", " for c in range(n_C): # loop over channels (= #filters) of the output volume\n", " \n", " # Find the corners of the current \"slice\" (≈4 lines)\n", " vert_start = h * stride\n", " vert_end = vert_start + f\n", " horiz_start = w * stride\n", " horiz_end = horiz_start + f\n", " \n", " # Use the corners to define the (3D) slice of a_prev_pad (See Hint above the cell). (≈1 line)\n", " a_slice_prev = a_prev_pad[vert_start:vert_end, horiz_start:horiz_end, :]\n", " \n", " # Convolve the (3D) slice with the correct filter W and bias b, to get back one output neuron. (≈1 line)\n", " Z[i, h, w, c] = conv_single_step(a_slice_prev, W[:,:,:,c], b[:,:,:,c])\n", " \n", " ### END CODE HERE ###\n", " \n", " # Making sure your output shape is correct\n", " assert(Z.shape == (m, n_H, n_W, n_C))\n", " \n", " # Save information in \"cache\" for the backprop\n", " cache = (A_prev, W, b, hparameters)\n", " \n", " return Z, cache" ] }, { "cell_type": "code", "execution_count": 7, "metadata": { "collapsed": false }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Z's mean = 0.0489952035289\n", "Z[3,2,1] = [-0.61490741 -6.7439236 -2.55153897 1.75698377 3.56208902 0.53036437\n", " 5.18531798 8.75898442]\n", "cache_conv[0][1][2][3] = [-0.20075807 0.18656139 0.41005165]\n" ] } ], "source": [ "np.random.seed(1)\n", "A_prev = np.random.randn(10,4,4,3)\n", "W = np.random.randn(2,2,3,8)\n", "b = np.random.randn(1,1,1,8)\n", "hparameters = {\"pad\" : 2,\n", " \"stride\": 2}\n", "\n", "Z, cache_conv = conv_forward(A_prev, W, b, hparameters)\n", "print(\"Z's mean =\", np.mean(Z))\n", "print(\"Z[3,2,1] =\", Z[3,2,1])\n", "print(\"cache_conv[0][1][2][3] =\", cache_conv[0][1][2][3])" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "**预期输出**:\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "\n", "
\n", " **Z's mean**\n", " \n", " 0.0489952035289\n", "
\n", " **Z[3,2,1]**\n", " \n", " [-0.61490741 -6.7439236 -2.55153897 1.75698377 3.56208902 0.53036437\n", " 5.18531798 8.75898442]\n", "
\n", " **cache_conv[0][1][2][3]**\n", " \n", " [-0.20075807 0.18656139 0.41005165]\n", "
\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "最后，卷积层还包括激活函数，如果我们要添加激活逻辑的话，可以使用下面的代码：\n", "```python\n", "# Convolve the window to get back one output neuron\n", "Z[i, h, w, c] = ...\n", "# Apply activation\n", "A[i, h, w, c] = activation(Z[i, h, w, c])\n", "```\n", "\n", "这里，我们先不需要进行激活操作。\n" ] }, { "cell_type": "markdown", "metadata": { "collapsed": true }, "source": [ "### 12.4 池化层\n", "\n", "池化层会缩减输入的高度和宽度，在减少计算量的同时，也使得特征检测对于特征在输入图像中的位置不那么敏感。池化层包括两大类型：\n", "\n", "- 最大池化层：($f, f$) 的滑动窗口，针对输入值进行滑动，将窗口中的最大值作为对应的输出。\n", "\n", "- 平均池化层：($f, f$) 的滑动窗口，针对输入值进行滑动，将窗口中的平均值作为对应的输出。\n", "\n", "\n", "
\n", "\n", "\n", "\n", "\n", "\n", "\n", "
\n", "\n", "池化层对于反向传播来说，没有需要训练的参数。但池化层具有超参，即滑动窗口的大小 $f$ 和步长 $s$。\n", "\n", "#### 12.4.1 前向池化\n", "下面我们会在同一个函数中实现最大池化和平均池化。\n", "\n", "**练习**: 实现池化层的前向传播\n", "\n", "**提示**:\n", "由于没有补全，下面的公式给出了通过输入尺寸来计算输出尺寸的方法：\n", "$$ n_H = \\lfloor \\frac{n_{H_{prev}} - f}{stride} \\rfloor +1 $$\n", "$$ n_W = \\lfloor \\frac{n_{W_{prev}} - f}{stride} \\rfloor +1 $$\n", "$$ n_C = n_{C_{prev}}$$" ] }, { "cell_type": "code", "execution_count": 8, "metadata": { "collapsed": true }, "outputs": [], "source": [ "# GRADED FUNCTION: pool_forward\n", "\n", "def pool_forward(A_prev, hparameters, mode = \"max\"):\n", " \"\"\"\n", " Implements the forward pass of the pooling layer\n", " \n", " Arguments:\n", " A_prev -- Input data, numpy array of shape (m, n_H_prev, n_W_prev, n_C_prev)\n", " hparameters -- python dictionary containing \"f\" and \"stride\"\n", " mode -- the pooling mode you would like to use, defined as a string (\"max\" or \"average\")\n", " \n", " Returns:\n", " A -- output of the pool layer, a numpy array of shape (m, n_H, n_W, n_C)\n", " cache -- cache used in the backward pass of the pooling layer, contains the input and hparameters \n", " \"\"\"\n", " \n", " # Retrieve dimensions from the input shape\n", " (m, n_H_prev, n_W_prev, n_C_prev) = A_prev.shape\n", " \n", " # Retrieve hyperparameters from \"hparameters\"\n", " f = hparameters[\"f\"]\n", " stride = hparameters[\"stride\"]\n", " \n", " # Define the dimensions of the output\n", " n_H = int(1 + (n_H_prev - f) / stride)\n", " n_W = int(1 + (n_W_prev - f) / stride)\n", " n_C = n_C_prev\n", " \n", " # Initialize output matrix A\n", " A = np.zeros((m, n_H, n_W, n_C)) \n", " \n", " ### START CODE HERE ###\n", " for i in range(m): # loop over the training examples\n", " for h in range(n_H): # loop on the vertical axis of the output volume\n", " for w in range(n_W): # loop on the horizontal axis of the output volume\n", " for c in range (n_C): # loop over the channels of the output volume\n", " \n", " # Find the corners of the current \"slice\" (≈4 lines)\n", " vert_start = h * stride\n", " vert_end = vert_start + f\n", " horiz_start = w * stride\n", " horiz_end = horiz_start + f\n", " \n", " # Use the corners to define the current slice on the ith training example of A_prev, channel c. (≈1 line)\n", " a_prev_slice = A_prev[i, vert_start:vert_end, horiz_start:horiz_end, c]\n", " \n", " # Compute the pooling operation on the slice. Use an if statment to differentiate the modes. Use np.max/np.mean.\n", " if mode == \"max\":\n", " A[i, h, w, c] = np.max(a_prev_slice)\n", " elif mode == \"average\":\n", " A[i, h, w, c] = np.mean(a_prev_slice)\n", " \n", " ### END CODE HERE ###\n", " \n", " # Store the input and hparameters in \"cache\" for pool_backward()\n", " cache = (A_prev, hparameters)\n", " \n", " # Making sure your output shape is correct\n", " assert(A.shape == (m, n_H, n_W, n_C))\n", " \n", " return A, cache" ] }, { "cell_type": "code", "execution_count": 9, "metadata": { "collapsed": false }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "mode = max\n", "A = [[[[ 1.74481176 0.86540763 1.13376944]]]\n", "\n", "\n", " [[[ 1.13162939 1.51981682 2.18557541]]]]\n", "\n", "mode = average\n", "A = [[[[ 0.02105773 -0.20328806 -0.40389855]]]\n", "\n", "\n", " [[[-0.22154621 0.51716526 0.48155844]]]]\n" ] } ], "source": [ "np.random.seed(1)\n", "A_prev = np.random.randn(2, 4, 4, 3)\n", "hparameters = {\"stride\" : 2, \"f\": 3}\n", "\n", "A, cache = pool_forward(A_prev, hparameters)\n", "print(\"mode = max\")\n", "print(\"A =\", A)\n", "print()\n", "A, cache = pool_forward(A_prev, hparameters, mode = \"average\")\n", "print(\"mode = average\")\n", "print(\"A =\", A)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "**预期输出:**\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "\n", "
\n", " A =\n", " \n", " [[[[ 1.74481176 0.86540763 1.13376944]]]\n", "\n", "\n", " [[[ 1.13162939 1.51981682 2.18557541]]]]\n", "\n", "
\n", " A =\n", " \n", " [[[[ 0.02105773 -0.20328806 -0.40389855]]]\n", "\n", "\n", " [[[-0.22154621 0.51716526 0.48155844]]]]\n", "\n", "
\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "到这里，我们就实现了卷积神经网络前向传播过程的所有层。接下来，我们会处理反向传播。" ] }, { "cell_type": "markdown", "metadata": { "collapsed": true }, "source": [ "### 12.5 卷积神经网络中的反向传播\n", "\n", "在现代深度学习框架中，我们通常只需要实现前向传播，框架会自行处理反向传播的过程，所以大部分深度学习工程师都不需要了解反向传播的详细机制。卷积神经网络的反向传播过程尤其复杂。这一节会简要介绍卷积神经网络的反向传播过程。\n", "\n", "之前在实现简单（全连接）的神经网络时，反向传播是根据成本函数来计算偏导，从而更新参数。卷积神经网络也是类似的，只不过卷积神经网络的反向传播公式更难推导，这里我们会简要地展示在下面。\n", "\n", "#### 12.5.1 卷积层的反向传播\n", "\n", "首先实现卷积层的反向传播\n", "\n", "##### 12.5.1.1 计算 dA:\n", "下面是给定一个训练样本，对于某个特定的过滤器 $W_c$，计算 $dA$ 的公式：\n", "\n", "$$ dA += \\sum _{h=0} ^{n_H} \\sum_{w=0} ^{n_W} W_c \\times dZ_{hw} \\tag{1}$$\n", "\n", "其中$W_c$ 是过滤器， $dZ_{hw}$ 是一个标量，表示成本函数针对卷积层输入 Z 在第 h 行，w 列的梯度。每次，我们都用同样的过滤器 $W_c$ 乘以不同的 dZ，来更新dA。这主要是因为，在前向传播过程中，每次过滤器都是点乘一个区间再求和。而计算 dA 的过程中，我们将这个区间内的梯度相加。\n", "\n", "上面的公式，转换为相应for循环内的代码：\n", "```python\n", "da_prev_pad[vert_start:vert_end, horiz_start:horiz_end, :] += W[:,:,:,c] * dZ[i, h, w, c]\n", "```\n", "\n", "##### 12.5.1.2 计算 dW:\n", "下面是计算 $dW_c$ 的公式（$dW_c$ 是过滤器的梯度）：\n", "\n", "$$ dW_c += \\sum _{h=0} ^{n_H} \\sum_{w=0} ^ {n_W} a_{slice} \\times dZ_{hw} \\tag{2}$$\n", "\n", "其中 $a_{slice}$ 表示用来生成激活值 $Z_{ij}$ 的区间。因此，这里最终给了我们针对这个区间计算 $W$ 的梯度的公式。由于是同一个 $W$，全部相加就得到了 $dW$。\n", "\n", "Where $a_{slice}$ corresponds to the slice which was used to generate the acitivation $Z_{ij}$. Hence, this ends up giving us the gradient for $W$ with respect to that slice. Since it is the same $W$, we will just add up all such gradients to get $dW$. \n", "\n", "上面的公式，转换为相应for循环内的代码：\n", "```python\n", "dW[:,:,:,c] += a_slice * dZ[i, h, w, c]\n", "```\n", "\n", "##### 12.5.1.3 计算 db:\n", "下面是针对过滤器 $W_c$ 对成本函数计算 $db$ 的公式：\n", "\n", "$$ db = \\sum_h \\sum_w dZ_{hw} \\tag{3}$$\n", "\n", "和普通神经网络一样，db 是通过对 $dZ$ 求和来计算的。在卷积的情况下，我们对计算输入 Z 的所有卷积求和。\n", "\n", "上面的公式，转换为相应for循环内的代码：\n", "```python\n", "db[:,:,:,c] += dZ[i, h, w, c]\n", "```\n", "\n", "**练习**: 实现下面的 `conv_backward` 函数。这里需要对所有训练样本、过滤器、高度和宽度求和。请使用上面的公式1,2,3。" ] }, { "cell_type": "code", "execution_count": 10, "metadata": { "collapsed": true }, "outputs": [], "source": [ "def conv_backward(dZ, cache):\n", " \"\"\"\n", " Implement the backward propagation for a convolution function\n", " \n", " Arguments:\n", " dZ -- gradient of the cost with respect to the output of the conv layer (Z), numpy array of shape (m, n_H, n_W, n_C)\n", " cache -- cache of values needed for the conv_backward(), output of conv_forward()\n", " \n", " Returns:\n", " dA_prev -- gradient of the cost with respect to the input of the conv layer (A_prev),\n", " numpy array of shape (m, n_H_prev, n_W_prev, n_C_prev)\n", " dW -- gradient of the cost with respect to the weights of the conv layer (W)\n", " numpy array of shape (f, f, n_C_prev, n_C)\n", " db -- gradient of the cost with respect to the biases of the conv layer (b)\n", " numpy array of shape (1, 1, 1, n_C)\n", " \"\"\"\n", " \n", " ### START CODE HERE ###\n", " # Retrieve information from \"cache\"\n", " (A_prev, W, b, hparameters) = cache\n", " \n", " # Retrieve dimensions from A_prev's shape\n", " (m, n_H_prev, n_W_prev, n_C_prev) = A_prev.shape\n", " \n", " # Retrieve dimensions from W's shape\n", " (f, f, n_C_prev, n_C) = W.shape\n", " \n", " # Retrieve information from \"hparameters\"\n", " stride = hparameters[\"stride\"]\n", " pad = hparameters[\"pad\"]\n", " \n", " # Retrieve dimensions from dZ's shape\n", " (m, n_H, n_W, n_C) = dZ.shape\n", " \n", " # Initialize dA_prev, dW, db with the correct shapes\n", " dA_prev = np.zeros((m, n_H_prev, n_W_prev, n_C_prev)) \n", " dW = np.zeros((f, f, n_C_prev, n_C))\n", " db = np.zeros((1, 1, 1, n_C))\n", "\n", " # Pad A_prev and dA_prev\n", " A_prev_pad = zero_pad(A_prev, pad)\n", " dA_prev_pad = zero_pad(dA_prev, pad)\n", " \n", " for i in range(m): # loop over the training examples\n", " \n", " # select ith training example from A_prev_pad and dA_prev_pad\n", " a_prev_pad = A_prev_pad[i]\n", " da_prev_pad = dA_prev_pad[i]\n", " \n", " for h in range(n_H): # loop over vertical axis of the output volume\n", " for w in range(n_W): # loop over horizontal axis of the output volume\n", " for c in range(n_C): # loop over the channels of the output volume\n", " \n", " # Find the corners of the current \"slice\"\n", " vert_start = h * stride\n", " vert_end = vert_start + f\n", " horiz_start = w * stride\n", " horiz_end = horiz_start + f\n", " \n", " # Use the corners to define the slice from a_prev_pad\n", " a_slice = a_prev_pad[vert_start:vert_end, horiz_start:horiz_end, :]\n", "\n", " # Update gradients for the window and the filter's parameters using the code formulas given above\n", " da_prev_pad[vert_start:vert_end, horiz_start:horiz_end, :] += W[:,:,:,c] * dZ[i, h, w, c]\n", " dW[:,:,:,c] += a_slice * dZ[i, h, w, c]\n", " db[:,:,:,c] += dZ[i, h, w, c]\n", " \n", " # Set the ith training example's dA_prev to the unpaded da_prev_pad (Hint: use X[pad:-pad, pad:-pad, :])\n", " dA_prev[i, :, :, :] = da_prev_pad[pad:-pad, pad:-pad, :]\n", " ### END CODE HERE ###\n", " \n", " # Making sure your output shape is correct\n", " assert(dA_prev.shape == (m, n_H_prev, n_W_prev, n_C_prev))\n", " \n", " return dA_prev, dW, db" ] }, { "cell_type": "code", "execution_count": 11, "metadata": { "collapsed": false }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "dA_mean = 1.45243777754\n", "dW_mean = 1.72699145831\n", "db_mean = 7.83923256462\n" ] } ], "source": [ "np.random.seed(1)\n", "dA, dW, db = conv_backward(Z, cache_conv)\n", "print(\"dA_mean =\", np.mean(dA))\n", "print(\"dW_mean =\", np.mean(dW))\n", "print(\"db_mean =\", np.mean(db))" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "** 预期输出: **\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "\n", "
\n", " **dA_mean**\n", " \n", " 1.45243777754\n", "
\n", " **dW_mean**\n", " \n", " 1.72699145831\n", "
\n", " **db_mean**\n", " \n", " 7.83923256462\n", "
\n" ] }, { "cell_type": "markdown", "metadata": { "collapsed": true }, "source": [ "#### 12.5.2 池化层的反向传播\n", "\n", "接下来，我们需要实现池化层的反向传播。首先我们处理最大池化层。尽管对于反向传播来说，池化层没有需要更新的参数，我们计算反向传播时，依然需要计算这一层的梯度，这样才能继续计算上一层的梯度。\n", "\n", "##### 12.5.2.1 最大池化层的反向传播\n", "\n", "在开始池化层的反向传播之前，我们需要构建一个辅助函数 `create_mask_from_window()`，进行如下转换：\n", "\n", "$$ X = \\begin{bmatrix}\n", "1 && 3 \\\\\n", "4 && 2\n", "\\end{bmatrix} \\quad \\rightarrow \\quad M =\\begin{bmatrix}\n", "0 && 0 \\\\\n", "1 && 0\n", "\\end{bmatrix}\\tag{4}$$\n", "\n", "这个函数创建了一个“掩码”矩阵，记录矩阵中最大值的位置。真值（1）表示最大值在矩阵 X 中所在的位置，其它的值都应该为假（0）。平均池化层后面也会用一个类似但不同的掩码矩阵。\n", "\n", "**练习**: 实现 `create_mask_from_window()`。\n", "提示:\n", "- [np.max()]() 可以计算数组中的最大值\n", "- 给定矩阵 X 和标量 x: `A = (X == x)` 会返回和 X 同样大小的矩阵 A，使:\n", "```\n", "A[i,j] = True if X[i,j] = x\n", "A[i,j] = False if X[i,j] != x\n", "```\n", "- 这里我们不必考虑矩阵中存在多个取最大值的元素的情况。" ] }, { "cell_type": "code", "execution_count": 12, "metadata": { "collapsed": true }, "outputs": [], "source": [ "def create_mask_from_window(x):\n", " \"\"\"\n", " Creates a mask from an input matrix x, to identify the max entry of x.\n", " \n", " Arguments:\n", " x -- Array of shape (f, f)\n", " \n", " Returns:\n", " mask -- Array of the same shape as window, contains a True at the position corresponding to the max entry of x.\n", " \"\"\"\n", " \n", " ### START CODE HERE ### (≈1 line)\n", " mask = x == np.max(x)\n", " ### END CODE HERE ###\n", " \n", " return mask" ] }, { "cell_type": "code", "execution_count": 13, "metadata": { "collapsed": false }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "x = [[ 1.62434536 -0.61175641 -0.52817175]\n", " [-1.07296862 0.86540763 -2.3015387 ]]\n", "mask = [[ True False False]\n", " [False False False]]\n" ] } ], "source": [ "np.random.seed(1)\n", "x = np.random.randn(2,3)\n", "mask = create_mask_from_window(x)\n", "print('x = ', x)\n", "print(\"mask = \", mask)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "**预期输出:** \n", "\n", " \n", " \n", "\n", "\n", "\n", "\n", "\n", " \n", "\n", "\n", "\n", "\n", "\n", "
\n", "\n", "**x =**\n", "\n", "\n", "[[ 1.62434536 -0.61175641 -0.52817175]
\n", " [-1.07296862 0.86540763 -2.3015387 ]]\n", "\n", "
\n", "**mask =**\n", "\n", "[[ True False False]
\n", " [False False False]]\n", "
" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "##### 12.5.2.2 平均池化层的反向传播\n", "\n", "在最大池化中，对每个输入窗口，输出仅仅来自输入窗口中最大的那个值的影响。而在平均池化中，输入窗口中的每个元素对输出都有同等的影响。\n", "\n", "比如我们用 2x2 的过滤器来计算平均池化的前向传播，则方向传播过程中会用到的掩码为：\n", "$$ dZ = 1 \\quad \\rightarrow \\quad dZ =\\begin{bmatrix}\n", "1/4 && 1/4 \\\\\n", "1/4 && 1/4\n", "\\end{bmatrix}\\tag{5}$$\n", "\n", "这表明矩阵 $dZ$ 中的每个位置对输出都有同等影响。\n", "\n", "**练习**: 实现下面的函数，来平均分布 dz 到矩阵的各个维度中. [提示](https://docs.scipy.org/doc/numpy-1.13.0/reference/generated/numpy.ones.html)" ] }, { "cell_type": "code", "execution_count": 14, "metadata": { "collapsed": true }, "outputs": [], "source": [ "def distribute_value(dz, shape):\n", " \"\"\"\n", " Distributes the input value in the matrix of dimension shape\n", " \n", " Arguments:\n", " dz -- input scalar\n", " shape -- the shape (n_H, n_W) of the output matrix for which we want to distribute the value of dz\n", " \n", " Returns:\n", " a -- Array of size (n_H, n_W) for which we distributed the value of dz\n", " \"\"\"\n", " \n", " ### START CODE HERE ###\n", " # Retrieve dimensions from shape (≈1 line)\n", " (n_H, n_W) = shape\n", " \n", " # Compute the value to distribute on the matrix (≈1 line)\n", " average = np.ones((n_H, n_W))\n", " \n", " # Create a matrix where every entry is the \"average\" value (≈1 line)\n", " a = average * dz / (n_H * n_W)\n", " ### END CODE HERE ###\n", " \n", " return a" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "**预期输出**: \n", "\n", " \n", " \n", "\n", "\n", "\n", "
\n", "distributed_value =\n", "\n", "[[ 0.5 0.5]\n", " \n", "[ 0.5 0.5]]\n", "
" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "##### 12.5.2.3 完整的池化层反向传播\n", "\n", "现在我们已经有了计算池化层反向传播的所有组件。\n", "\n", "**练习**: 实现不同模式下（`\"max\"` 和 `\"average\"`）的函数 `pool_backward`。这里会用到四层循环（对训练样本、高度、宽度、通道）。使用 `if/elif` 语句来判断模式。如果是 'average'，则使用 `distribute_value()`，得到和 `a_slice` 同样大小的矩阵。 否则使用 `create_mask_from_window()`，并和对应的 dZ相乘。" ] }, { "cell_type": "code", "execution_count": 15, "metadata": { "collapsed": true }, "outputs": [], "source": [ "def pool_backward(dA, cache, mode = \"max\"):\n", " \"\"\"\n", " Implements the backward pass of the pooling layer\n", " \n", " Arguments:\n", " dA -- gradient of cost with respect to the output of the pooling layer, same shape as A\n", " cache -- cache output from the forward pass of the pooling layer, contains the layer's input and hparameters \n", " mode -- the pooling mode you would like to use, defined as a string (\"max\" or \"average\")\n", " \n", " Returns:\n", " dA_prev -- gradient of cost with respect to the input of the pooling layer, same shape as A_prev\n", " \"\"\"\n", " \n", " ### START CODE HERE ###\n", " \n", " # Retrieve information from cache (≈1 line)\n", " (A_prev, hparameters) = cache\n", " \n", " # Retrieve hyperparameters from \"hparameters\" (≈2 lines)\n", " stride = hparameters[\"stride\"]\n", " f = hparameters[\"f\"]\n", " \n", " # Retrieve dimensions from A_prev's shape and dA's shape (≈2 lines)\n", " m, n_H_prev, n_W_prev, n_C_prev = A_prev.shape\n", " m, n_H, n_W, n_C = dA.shape\n", " \n", " # Initialize dA_prev with zeros (≈1 line)\n", " dA_prev = np.zeros(A_prev.shape)\n", " \n", " for i in range(m): # loop over the training examples\n", " \n", " # select training example from A_prev (≈1 line)\n", " a_prev = A_prev[i]\n", " \n", " for h in range(n_H): # loop on the vertical axis\n", " for w in range(n_W): # loop on the horizontal axis\n", " for c in range(n_C): # loop over the channels (depth)\n", " \n", " # Find the corners of the current \"slice\" (≈4 lines)\n", " vert_start = h * stride\n", " vert_end = vert_start + f\n", " horiz_start = w * stride\n", " horiz_end = horiz_start + f\n", " \n", " # Compute the backward propagation in both modes.\n", " if mode == \"max\":\n", " \n", " # Use the corners and \"c\" to define the current slice from a_prev (≈1 line)\n", " a_prev_slice = a_prev[vert_start:vert_end, horiz_start:horiz_end, c]\n", " # Create the mask from a_prev_slice (≈1 line)\n", " mask = create_mask_from_window(a_prev_slice)\n", " # Set dA_prev to be dA_prev + (the mask multiplied by the correct entry of dA) (≈1 line)\n", " dA_prev[i, vert_start: vert_end, horiz_start: horiz_end, c] += dA[i, h, w, c] * mask\n", " \n", " elif mode == \"average\":\n", " \n", " # Get the value a from dA (≈1 line)\n", " da = dA[i, h, w, c]\n", " # Define the shape of the filter as fxf (≈1 line)\n", " shape = (f, f)\n", " # Distribute it to get the correct slice of dA_prev. i.e. Add the distributed value of da. (≈1 line)\n", " dA_prev[i, vert_start: vert_end, horiz_start: horiz_end, c] += distribute_value(da, shape)\n", " \n", " ### END CODE ###\n", " \n", " # Making sure your output shape is correct\n", " assert(dA_prev.shape == A_prev.shape)\n", " \n", " return dA_prev" ] }, { "cell_type": "code", "execution_count": 16, "metadata": { "collapsed": false }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "mode = max\n", "mean of dA = 0.145713902729\n", "dA_prev[1,1] = [[ 0. 0. ]\n", " [ 5.05844394 -1.68282702]\n", " [ 0. 0. ]]\n", "\n", "mode = average\n", "mean of dA = 0.145713902729\n", "dA_prev[1,1] = [[ 0.08485462 0.2787552 ]\n", " [ 1.26461098 -0.25749373]\n", " [ 1.17975636 -0.53624893]]\n" ] } ], "source": [ "np.random.seed(1)\n", "A_prev = np.random.randn(5, 5, 3, 2)\n", "hparameters = {\"stride\" : 1, \"f\": 2}\n", "A, cache = pool_forward(A_prev, hparameters)\n", "dA = np.random.randn(5, 4, 2, 2)\n", "\n", "dA_prev = pool_backward(dA, cache, mode = \"max\")\n", "print(\"mode = max\")\n", "print('mean of dA = ', np.mean(dA))\n", "print('dA_prev[1,1] = ', dA_prev[1,1])\n", "print()\n", "dA_prev = pool_backward(dA, cache, mode = \"average\")\n", "print(\"mode = average\")\n", "print('mean of dA = ', np.mean(dA))\n", "print('dA_prev[1,1] = ', dA_prev[1,1])" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "**预期输出**: \n", "\n", "mode = max:\n", " \n", " \n", "\n", "\n", "\n", "\n", "\n", " \n", "\n", "\n", "\n", "
\n", "\n", "**mean of dA =**\n", "\n", "\n", "0.145713902729\n", "\n", "
\n", "**dA_prev[1,1] =** \n", "\n", "[[ 0. 0. ]
\n", " [ 5.05844394 -1.68282702]
\n", " [ 0. 0. ]]\n", "
\n", "\n", "mode = average\n", " \n", " \n", "\n", "\n", "\n", "\n", "\n", " \n", "\n", "\n", "\n", "
\n", "\n", "**mean of dA =**\n", "\n", "\n", "0.145713902729\n", "\n", "
\n", "**dA_prev[1,1] =** \n", "\n", "[[ 0.08485462 0.2787552 ]
\n", " [ 1.26461098 -0.25749373]
\n", " [ 1.17975636 -0.53624893]]\n", "
" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "#### 恭喜 !\n", "\n", "这样，实现完所有卷积神经网络的组件，我们就更加了解了卷积神经网络的机制。在下个练习中，我们会使用Tensorflow来实现卷积神经网络。" ] }, { "cell_type": "markdown", "metadata": { "collapsed": true }, "source": [ "## 13. 卷积神经网络：应用\n", "\n", "在这部分，我们将会:\n", "\n", "- 实现一些辅助函数，帮助我们更好地实现Tensorflow模型\n", "- 使用Tensorflow实现一个功能完备的卷积神经网络\n", "\n", "**这部分的内容将教会我们**\n", "\n", "- 使用Tensorflow构建并训练卷积神经网络，来解决分类问题" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### 13.0 Tensorflow 模型\n", "\n", "上一节中，我们使用numpy来实现了一些卷积神经网络的辅助函数，从而更好第了解卷积神经网络背后的运行机制。而如今，绝大多数深度学习的实际应用都是基于编程框架来构建的，这些编程框架提供了很多自带的函数，我们只需要调用就可以了。\n", "\n", "首先，我们需要引入相关包" ] }, { "cell_type": "code", "execution_count": 17, "metadata": { "collapsed": false }, "outputs": [], "source": [ "import math\n", "import numpy as np\n", "import h5py\n", "import matplotlib.pyplot as plt\n", "import scipy\n", "from PIL import Image\n", "from scipy import ndimage\n", "import tensorflow as tf\n", "from tensorflow.python.framework import ops\n", "from cnn_utils import *\n", "\n", "%matplotlib inline\n", "np.random.seed(1)" ] }, { "cell_type": "code", "execution_count": 18, "metadata": { "collapsed": false }, "outputs": [], "source": [ "# Loading the data (signs)\n", "X_train_orig, Y_train_orig, X_test_orig, Y_test_orig, classes = load_dataset()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "提示一下，SIGNS数据集是一组手势的图片集，表示从0到5这6个数字。\n", "\n", "\n", "\n", "下面我们会给定一个打标的训练集的展示，可以修改 `index` 的值来重跑看看不同的结果。" ] }, { "cell_type": "code", "execution_count": 19, "metadata": { "collapsed": false }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "y = 2\n" ] }, { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAP4AAAD+CAYAAAAalrhRAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJztnWmQXMdx5//Z3TM9F2YwuEGAhHiCFEGCF25JpCRKpiSH\n5Y2NVch2eClrHbvhlYNS6FhT2g+O/bARlle2Dlt22LGSllZIax2WRWpXXh4Bk6JIAuABkhBJACRB\nXAQwuDGY++jaD914lZlvquZ1Y6YH9stfBAL1Jt+rqndUv8qXWZnknINhGPmiMNcdMAyj+djAN4wc\nYgPfMHKIDXzDyCE28A0jh9jAN4wcclEDn4juIaLdRLSXiP5opjplGMbsQo3a8YmoAGAvgPcDOALg\nWQAfd87tnrnuGYYxG1zMG389gNedcwecc+MA/h7AR2emW4ZhzCalizh2BYBDbPswqj8GAiIy10DD\nmCOcczTV3y9m4Gfms/f9Zzy9bQc2b1qPzRvXYfPG2u8DZZ1wUHAzdVYU3Ej4yp//Bb7wufsyth3v\nC0Xa4794U2lUX/nzr+Pzn/202lPVQ/ocXFAmfmLDl2xK4f/4ytfwhc9/BhS4ZlMdFupWfbjU5p/+\n2dfwXz73GaQeWceL6eOSoqtEWtPHkZBqvvLn38DnP3vfFKLICbP204eFj5tO7f6zr30Tn/vMp5C+\nEdXjnt62A89s25H89avf+OtgXRcz8N8GcAXbXln7W4rPf+ZT+MrXqv8bhjE7bN64Hps2rEu2Z2vg\nPwvgGiJaBeAogI8D+K34IeFXRuSdF5WmJJEfTf6C1D+uJOqMVRJ7Y0REqY6R/y9SR7qj/CTiu07V\nnCrKetxUwin2q5s67h+xXVLn11gHXHDj0oLYvU2fK6n/Q5Vka6vhge+cmySiPwTwCKofCb/lnHst\ntP/mjetCoqazeVPqU8ScsHnThrnuQsLmTRvnugsJWzZfOn25lO7Rpo0z99w2bM7L3ACRO7LvlQsb\nWpgU42/8VKVhUWxLvCz1nlnf+HqTgkL5Io+cYT23IHLuop7wpY7q8RSrv2Fib3wXkKRnMFmfVa3j\nyzd+7EmL6e3T/oGJGtPx5W71jAjHSr58+VVr5vbjXojYjcw6FuIPqlNbMVXDZdov2rHIx6jUh7jI\nw+9iIzjWfGw6H+iW3jf6o1dHX+IXKuN9j56Evmb8OP3Rk91b/bxEG3ShHeV1SqmNYeLfErNeM90C\nBSUhzGXXMHKIDXzDyCE28A0jhzRFxw/rjeGPe1PvNVXlSveqQ+fP3kpmz5jMTbuI3ZEyXpd0E+F+\nxsyj0W8akePiTkFh22nmc0rtyK5LSh0P6+PxSiNKd0ytdoXAjvEtcrFvO0xXj35m0nWG2wthb3zD\nyCE28A0jhzRnql+bhmn7cVZ7efapaHXvKStJbc0MfIZJsambPq7h9kSDgoZVBFZn7Bpp06m0hKlz\nD3cz3pfIlqw/Mi2PzuYbewrSluesqoU6aia8R6fxa82CvfENI4fYwDeMHGID3zBySFN0/AsaSVpv\nibnQRvzJubdkSnkOm0SE23JaGGwv6hIZdbl3QVl2fbzBlXt1QBkX8kfvXj1uq2KjDrft2D1q+ANO\n2PVWVh/7PhWRubA+Hg21kHonh5+lRrR8e+MbRg6xgW8YOaRJ5rzA32PT5NiqqegS04xTaNWp+Ewx\nMmUX6kO0kkCNU8myVhRRV2KajJIVIlE6omY5Cm6oOmJ6QHi+m4p5Er23MU/IcHOx6xKL4JFZPYt4\nGMZX6IbDh2n1QY4B89wzDCOADXzDyCE28A0jhzRpdV4GQaPmmDoWW0WraUxVj0Zi4faauMvuzJjs\nYou04ua1mEmt0dVrkfai94XfiEh7dZDVfBinHvNvtlrS33Ia7VuohTD2xjeMHGID3zByyBwH4mD7\nRNKmpOMj1jE1ZsSy3vC5vu5K405hzHQTC/IYXxIXqjJKPc1l7Uq6aabKxNLe1FlrFlFU7YjsWk+V\n4hmsI9CIrCK7GVCqJI2uLLWpvmEYAWzgG0YOsYFvGDmkOTp+paa9pDLpRMxIkRxxOtKNFGY0d+nD\nIopgPIBnzH7IzyHidhwx2UV1QnX9Yiv+RC/rMoFmC6WTDhwZazAr4Q8VsWy5s0E97WXO+DNDnZ6V\nCDxE9C0i6iOil9nfeonoESLaQ0QPE1FPA20bhjFHZJnqfwfAr6m/3Q/gMefcagBbAXxxpjtmGMbs\nMe1U3zn3y1oqbM5HAdxZKz8A4HFUfwwCTG9iiM0w0/ERKVCeptXYaqtoIPOMgQ/rcvnjK/5ippvY\nFFpVKUygkQAQqb4Eq5zuyCAzMonN6uF3ES266PK82LN18WeYDu4R3oof17xAHEucc30A4Jw7BmBJ\ng/UYhjEHzNTHvehPzle/8VfVAhE2bliHTRvWzVCzhmFc4JltO/DMth2Z9qUsXyBrU/2fOedurm2/\nBuAu51wfES0D8M/OuRsCx7oDe1++sKGEvpj2xgvHxyOx+CUWpKMOGs4LzyZNKStFTCUJTzHlNL2e\nABChOsK9mh7+VT/7VHj2p/paz3FhWeS4+FQ/TMoSE6wzOzFLiHySsk31L7/qBri0SyyA7G98Um0/\nBOATAL4M4F4AD8YPv9CxmE4z9RHpo7QwbGbRenv0trpsD3h6aMdW5zX6+PvjKtEAk+Hz0w8fZTQR\npn9Hs2mD8TOtw35Yl14fqjK7DtxwEpLIIsKGadAM2MiC1CzmvO8DeBrAdUR0kIh+D8CfAPgAEe0B\n8P7atmEY/0LI8lX/twOiu2e4L4ZhNImmeO55Ui5xvhydlkf0qdRf+IqxOkx2kVql2qF6NjmRlCuj\nw0JWKLf540qtdbTHziGWry5ai5Lx/Hip6Xw2XT2r/2K6L40dGTUtxlZhRrb1UZln6SltJaPHZuzZ\nretbQLZnN6uZ0Xz1DSOH2MA3jBxiA98wckhTdPyKqyYHSJmGRBATlUCA/L4pi13MlBJe1Kf0wogp\nTB/HqIyNiO1Tzz+RlCf7DglZcf7ipNyzdpOQlRcsDXUl6pYbFzE9PuLfkDJ3ZV2dF6uzHqKm04sn\nbv6PfY2IPEyRw+JuzjNFRrfxjNgb3zByiA18w8ghTY6rH56zp+JUuLBZR85a9Tw5bLaK56SLLAdk\nm6f3vyFEh7b9Mil3tZaFrHTiTFIePHtOyJa//zeScmv3/GBX4rO47B6GWb0dYypCymNsam/QFPEg\nFvW412Zqro4dpwujwp7BGfPOyypsbCVkVi9Ce+MbRg6xgW8YOcQGvmHkkOa47F5QkFLup7GVbcEN\nGWkm4kqZ1sv4dwPtzstbU6Y+VtH5c2eF7MQZr7uPtXcKWVeHL5f6jgnZke3/nJRXvudDQlbkrr4x\nPbquIDQRfZViZquwiVAH+wz1JXprU+p/ZkVebjWq//NzqGNVX6wvjTsJN7bMmSLXOoS98Q0jh9jA\nN4wc0lxzXtb5CvRqMv37FAuMkaEfUx3JTYuRjvasfIfYHmll8/lhuToPVEyKXeoUJt7cnZSPz18g\nZMtufxfrVlHIZDz+YDcRuxKx6Dx1BXlo0MQlPQWzV5J9NVv4qUide+bm64jOIzxSY/XMjI0wqvoG\nsDe+YeQQG/iGkUNs4BtGDmmSOa/2X0bzz/RCrrNF3EGjumzYFpaW+L90L1kqZMtvWZ+U39z6/2Qv\nK77OUlHW2sm+DZzeJUMity3ybfReeX243/HooWo74s6b9btBPYF0Yn2JPgaN6r0xc2XMtli/fpxu\nJHs0IFHFRewbOjLr9bM3vmHkEBv4hpFDmhtss565DT9MB+mYGE+KlYkxWWWLXyFXbJGr5eJBH7KZ\nCLVp8bpN707Kb+/5lZCdP3rYH6d+YivMSlccl7fh4FOPJuW2BTI7WXvPQt3xhFhuiKwz8ZifWTqQ\nIw9sqo/LOG9udNlbVHuow1QbiUISC0IqkftVhCS7517jxr2wR2oIe+MbRg6xgW8YOcQGvmHkkKa6\n7KZz4GXTd8bOnRKy088/5ffrl6vlSl0+mk3ntTcKWeeqa5MytbQE2451U5skO3t7k/LNH/x1IXvm\n+99OyqPn+4Xs/IgP2tm7qFfIelr9OR157nEhW/WuDyflYrk90mm1yc1W4aPi1TTsoqs7kzHSTF36\nf+RZiibfEOGHVI3hjx8U2u8iyOzMG7ucM+WyS0QriWgrEb1CRLuI6L7a33uJ6BEi2kNEDxNRT7Ym\nDcOYa7JM9ScAfNY5dyOATQA+RUTXA7gfwGPOudUAtgL44ux10zCMmSRL0sxjAI7VygNE9BqAlQA+\nCuDO2m4PAHgc1R+DdB3hyX643cpkUj787DNCdual55NyuShz0hVKfUn52JtvClnvzbcm5cvWvVvI\nSm1+2pyeGVaYLDyXWrH6BrF93fvuScrPPfQjIesmb5LsYvn3AMBVfHsDb70qZG+3d/n21t0lZCnz\npah0ymJ9pHW1bHXGpqZ19CaWBjxONpXSVcK5FrSHaDxwa2Nkv4ZhW2Y0rTqjro97RPQOALcA2AZg\nqXOuD0h+HJaEjzQM41Ii88c9IuoC8GMAn669+TN7JnzjL//mQi3YsP4ObNxwR/09NQwjyjPbd+CZ\nbTum3xEZBz4RlVAd9N91zj1Y+3MfES11zvUR0TIAx0PH3/eH/6lWj1kPDWO22LRhPTasX5dsf/0v\n/yq4b9Y3/rcBvOqc+zr720MAPgHgywDuBfDgFMcB4MEqtU0krCdVmJ7bd6xPyM6dG0rKnWWpH1Nh\nNCkXW6X+P/bic0l5RJkBl9+xOSl3LAprLSndUiiCMlrOjVvuTMpnjh8VsqM7t/m+jEq346ERfw7t\nkOdw6uVtCLFi3XuTclGZK6UBrTH9tPGkEo3p4/W1F/7eIOamqUhP/lqkX0vZ+hLzBNfPS9YrHzUR\nxtyVM160aQc+EW0B8DsAdhHRzlo7X0J1wP+QiD4J4ACAj2Vq0TCMOSfLV/2nABQD4rtntjuGYTSD\nJnnuVSc4qUxvwiNOygpF37X5V8tgFG/u8qvgJtTUpr3sTVqt41INKI357fNv7hGywRN+Jd1lG+4S\nsgW8/UiwRh3IscSm27ff/REh23rkYFIeOCVVmcJ5P+msVOT5dbT463L8+V8KGZV9XP+Vt22GFIZX\nqFEskEODU9ysxFSnWNqAWOr0+FQ4bLKrL08B2y2i/aX2VUpX1uMyt5CxEvvaZhg5xAa+YeQQG/iG\nkUOamzsvlQNv6rLe9+rb1gnJ+X6fr+7lR2WAy3mj3hV2fmeHkFHB/84VSZrJKuf86rkDT/6TkA0P\neNnSG24VsmIrM5vpSEGMrl4ZOWfN+3y+vKd/+L+EzA35xByFgrxFJWYyLJJs7+C2rUm5ffFyIVt4\n+VXBvrlIMEqZ2CR8/zSZc2akvplEosnE3I5nIJBP2vTGVzTWE2k085UJHzVzftVTYm98w8ghNvAN\nI4c0Zap/YcUQQU+F/e9ObDpYLEkvtFvu8u4DlaI8hW0P/jQpD4xI77zeTu8h190pg1h0dvnU1K4y\nJGT7fvFwUj7z9n4hu/x2v8qvc8FiIaNCkZXlua+8zgcJWbFGrl3Y/9zT/ji9YmzS19PRLlfjjY36\n8937y4eF7PZ/c29Sbm2TKpCcXqv2Ggz2LtW4sPda6r5njNFRXx6GMLEpfLyWsAqkW2iM2IrCiw/8\nYW98w8ghNvANI4fYwDeMHNIcl93aSjuX+p1hkW3iKREkTKe6efO7hIjrqNt/9pCQDZ48k5TnDQ4I\nWfeI1/lbSnJpQnuXl50/8LqQ7TlxJCkvuVHq6ouuWePr7OgSsmLBf7dY8+73C9nB17xL8pnTJ4Ss\nVPBXptQqb197h9fdzx/eJ2SHf+WjFl15+xZIIv6uDaqosUCVme90g6vz6vA6Fv2Mn2os2mZk1Wkd\n55BOvhFuItKzTNgb3zByiA18w8ghzTHnXTBBhZ200lN9YWKqKFnYQ27Nxg1JuaunW8ie+j9+6n/4\n4AEhK/WfT8rzlKlvOTPLlUrKk27Em/6OPffPQnZ6/+6kvOSmjUK24PJrknL3QunVd8Om9yTl5x76\ngZANjTGTpJsUMn4JO5S6cmi79+qbv/xyIeu9bBVCNBoYMxrEInMA+XBn6rHmUcb2UiIKS0nk4wvX\nFDt1PbWPBtSIqg98IM1CsE3DMP51YAPfMHKIDXzDyCFUX2KCBhogcr96oZbrLhVl1+stFaW386QS\naf2f67ZhXUi7Y/af9S6tzz/1lJDteMznpC9NyuCXi3q8mWxRd6eQ9c7zsrayysfHVs+Nqm72XnNT\nUr5y3Z1CVg1qXOXx7/2tkA0e9ZF75nVI19suthqRRyICAOf8qsW2lXKl3s2//jtJudwxDzNBNF8d\nV1cb1PFToshhsUQcWQOPRp1yKb6duc4GxyIfOnwcXb16DZzTsa2q2BvfMHKIDXzDyCFNmerveq4W\nFFJP9SlsEuEmO22+4zH3kVrxx+tP9cUfNSlbfP2VXUn5if+rPP5O+GCYPe0ygMd8Zvqbx1b4AUBn\nG9+Wprch5jnYpcxpq9/tg3SMT8jz2/bjB5JyYXxUyMosj0Bbm5zql7l5T0WqXHGHVzWue8+HhKwo\nzJfR8JdSMgvPlQgKEulJo1Pm2LQ/NZ2P1tOYkDK650U9EdnYuMqm+oZhcGzgG0YOsYFvGDmkKTr+\ny889WduSvzOOpAOjhLk9qqQSTqzqUzq+C5sBIXTEsNJ24qjMc/fEz3+WlA/ufkXIuCbd0SrNed3M\nvNbRJr8NdLYys2NB6v/jRb/vZWtlYozRYa/Xv/HME0JWKvrr26py57UyXb3cIt15S2Xf3jV33iNk\n77jNRxgqtshzEM9Ow9Fy1K4NP48ZleDIYfXkqajne0DWOhs9B+HezsoXZc4jojIRbSeinUS0i4j+\nuPb3XiJ6hIj2ENHDRNSTvauGYcwl0w5859wogPc6524FcAuADxHRegD3A3jMObcawFYAX5zVnhqG\nMWPUNdUnog4AvwDwBwC+C+BO51wfES0D8Lhz7vopjnEvPXthqq/jskdWOAnTn5aGTX3CDFjJvqqP\nT8908MShwcGk/Owv5Aq8F574RVIeO39eyDrZlHpJr/SIW7FsUVJuL6v22Iq/4Ql57uPtC5Jy/6lT\nQlYaH/FlFY+fCr6NznYVaLTDT+Fb2+Rxqzb4ICFXrn+fkPH8ho3nnYuHyZBEYtJHqpGL7CJ11hF0\nhE/1U2pAw1N9JksFE4msdhRqMfPcu/6mizPnEVGhliL7GIBHnXPPAljqnOurdsQdAxBOKm8YxiVF\npvX4rvqqvJWIugH8IxHdiCl8bkLH//Xffjsp33H7rVh3+20NdNUwjBjbtu/Atm07Mu1bVyAO51w/\nET0O4B4AfUS0lE31j4eO+4P/+MlaaQbyKhuGMSUbN6zHhnU+7uM3vvnXwX2nHfhEtAjAuHPuHBG1\nA/gAgD8B8BCATwD4MoB7ATwYqiPJ8R4Z9y6SQy2VsEDo/2lDCztQVsllqdWAvD0pa2M68bvu/qCQ\nLVuxMilv/clPhGzklP8t1GbH0VGvjxcL0kzWUuTb0tQ3ctbXeebEaSG7coXXthYvkFF9JiYmkrL+\n0iFMf5PjQrafRe5p710qZMtX35yU44kp6oh+mVGXjaIOjAYRYsq01quz2ijTvYqY+vhx+rGOWfMi\npmgXOy5Aljf+cgAPEFEB1W8CP3DO/ZyItgH4IRF9EsABAB/L2KZhGHPMtAPfObcLQEopd86dBnB3\n+gjDMC51mhNXPzj/iEyXiE/5pPFBTnV0rreweSaeutn/QQcF4ZNjnmobAK55p8+B19YmV+dt/7lf\n5Td6+piQnen3qbcHhqSXXYGvWlR94c1fsXS+kJVZzP1lC2TAkCIz753ulzkFhsd9Gy0tclXfyJA3\nUe75hUxJ3r3ksqTc2bsIjRAz50VzxNUTrz4SpJM/S7orXD1Le3qGzXlxb0B2XNSSWYe6EmsugPnq\nG0YOsYFvGDnEBr5h5JCmrM57/pnHL2wImYu6S4bNclyP16Y+EvpxJElHxJ3XqUQVYlvr/6IN2d75\nM96l9pWn5Uq6t158ISmPDQ0L2eQka0OtTFy10pvprrh8mZAdO3YyKS9ZIPX/9lb//aGgkoL0nTrn\nZUXl6svOd0Jds8vX+8g919/1EXkc+xgRSzjRaH68VL6JQPXVzWyNZA28qdtLmZtD/VJ/SUf14dcl\nslo15rLLV+ddbxF4DMNg2MA3jBzSJHPeZO1/HWyT7xQLbKBUBL5d0L9dfjulBnDTTUHVydqv6L5U\n+LZUA6TpT8q65vuVdHd88NeFbNk7rk7Kzz/8T0J2/rhPjb2wV6bXLrC+nDpxRsi4hnD2/JCQDZe8\nR15ZBdTgQTrOnpemvq4u335LUV6X43tfTsor1sgU4d1Llifl9KS1seR50bgfURMXXwWq4+pno1Fn\n89j6wvqOZHWkLmj9vbM3vmHkEBv4hpFDbOAbRg5pio5fmbyg+6p87tz0FjOJqEQc3FRE2ruWu/qm\nTH1c/1eHFVgASlVnRZj65IGVSb/qbfisjIgzNuBz9U2MSpMdRr0uffmVK4WovKw7Kfd2Sx3/+FG/\nOu/I8ZNCVmIuw0PDMtlGa8nr9VSR92HxEr+qT1/Oc0zn753fLWSTw1721gsyF+GaD/xmUi4UZHBP\nTtScHDNbhY+6CCKrCCPPZzofX+y4mMluhiKWZsDe+IaRQ2zgG0YOadJUvzodTgfUiAXb5J574Shf\nFSUTaoCeOolVWuE6U159FT+dHxuRZrJ9271H3sjRfUJWqvh02xPjMsAFsdVyvSrd9QRbZTevW8nG\nepPyoaNStTh93KsWFTX9nNflV+tNTsqpPrX6FXljTHUBgK4OH4SEtFcfU92O7nlZyC5fuyEp9y6T\nqkwU4dQXy4ugiEW4nIlZctTDVZubmUR7j2aM7Zm9NcRXCgawN75h5BAb+IaRQ2zgG0YOaY6OX9OR\ndfQaMBfe9GIknltO6fFMBU957Aq3zqISZcy5p3R8Yqvzzva9LWRHXnsxKS/ulhF45nV43fnc2REh\nGxn2unR3t0y2ceakT+DRfrZfyFpZfr7OTpkY40S/P25AmfPGJ/2FaS1Ll92TrI2ubllnz3xvThwf\nlXXyqzQ6Kl19396zKynPZ5F6qoRdaCWx6Dyqxox6fFrnzqYfx3XuiGNuY6Ip2si2Z2ylIMfe+IaR\nQ2zgG0YOadLqvNqkJpK7Lh4ZU+8aDn4h2o0l+4kE6Uj5r7Hp08BZuSKuwkx9PE01ALSwePXjKgfe\nGAuwMTomTX0DA97Lb15XWEUYm5Cmt0WLvKlv6IjMbzIw4qfpHWpl4ki/b7+7R071RaBK5fFXZOdX\nUHUefcOnE7+WBewAgNYyMxEiTDqGRFa1IFJrJJZ97BGMEZ1ep+qM5R/gdcaqjEz1MwbWsTe+YeQQ\nG/iGkUNs4BtGDmmOjp8q1IjmLWPH61V2YVVd6TjhbwqpVWEuUimvUbnethS9ybCkglgWmYsrqRVq\nQyx33lsH+4RsbMib5YYHpcvuyTPeZXh4VPZlXo83C3b3yIQaI2zf8UmVpINdp4LSq8eGvNsxiuFV\nkvx7BgAMnPbndP7UCSFbdNkVSdlF3KpjgSqzBtDU6KMK2RX5QE8uogNhD/YpEniEq6ToB7GpyfzG\nJ6ICEb1ARA/VtnuJ6BEi2kNEDxNRT9a6DMOYW+qZ6n8awKts+34AjznnVgPYCuCLM9kxwzBmj0xT\nfSJaCeDDAP47gM/W/vxRABfsNA8AeBzVH4M0NTOei/zOpFIGC6HeNzwt56vSCqkpe9gMKHLUpUx9\nXjY5oQJctPhL2NKiLiebu7WU5FR/YNDXs/+wnAqvWOi95YaGpcluYNirCHq6y1NhX7ZUpsk+wWLn\n9w/KFYZtRT9N72mX3oclpgYUS1LGFQ0qyvOrjHh15fSRA0K2kE3104SnrS4SbVPeMuXpmXHhXnT6\n3njUzCDRKuowLcYDf0xN1jf+VwF8AfL0lzrn+gDAOXcMwJKpDjQM49Jj2jc+EX0EQJ9z7kUiuiuy\na/Crwrce+PsLleHWtWtw2y031dlNwzCm45lt2/HM9u2Z9s0y1d8C4DeI6MMA2gHMI6LvAjhGREud\nc31EtAzA8VAF/+Hej1cLZNZDw5gtNm3cgE0b1yfbX/uLbwb3nXbgO+e+BOBLAEBEdwL4nHPud4no\nTwF8AsCXAdwL4MFIHagen1LWw+2yMkHnuZt6PwCgCl+BF3b5TK3SEgE1ZXsV5qo6PiKDZhaLPICn\n/GHj1ZRU9JqJCS8cGpGusC1lb8Ibr8g6K+ycKi4cKaitVZrzWlu8Dq7vw7xOr7sv7pUrBcWuKhHH\nILsuY2PyWwTv24m33xKya27fkpQL+ppFXbUjMr6b1nNjbqwN6uqNB/Xh5sp6Gs/oFpyxYxfzCv4T\nAB8goj0A3l/bNgzjXwB1OfA4554A8EStfBrA3bPRKcMwZpc59dzjU87YtDxlzuOpo3VgfRebSnFZ\nZKqvVqHx4JQTOhgF84KbHFd5A4p89ZqcXPHVbF2dcgpdLvupd/+QXJ03zFfyqVx2bSxo58iwPI5b\nE7W32uKeHrafFE6y6zkJOZ3np6RTi3Md4dSRw0I0NuzNieUOmTcgRiwHnos8L3WZ6bhIeNJln9xH\nV89Fj8tYf9TUZ4E4DMMIYAPfMHKIDXzDyCFN0fFDekfMzTKrrqJNby5i61NZ74L1VGI6vlrZNspW\nvY2MjgkZzzuvo+ycPe9dWsfVORw97d1rR0d0gEvflxWXSWdJHrFmYkL2pcR07u426Xq7mCXtGFbH\njUwwk92E0uPZx4JJlYiD2DXrPyVXH57uO5KUl7/jWlknz28oJdEVeVyvrstCFwl+KdtWh4W9h0PV\nT/EH/Z0ifA5Sr49o9RlP3t74hpFDbOAbRg5p0lR/aoR5JpWhONvESzv/Se0h5p0XNgdVKlJWYd6A\nFdXg4BALYqkCVbQybz1tImxn3nJvn5fT+f4TPqCnU3nuFrIAG6WSbK+/36sIba1SNslUlOWL5gtZ\noeTPaUy1x6efBWVHmmSeghMTUpXh6cNHlWlx15OPJeXu3sVC1tnt+xZPnV5PFPqYzY7P2bMrCdKz\nNNLeLAQddsyGAAAUI0lEQVTwaDQQJ8fe+IaRQ2zgG0YOsYFvGDmkKTp+KJCmi2Uz4Fa5iJ4UTVCQ\nqpK7fOqdeZ3q95DYyrZWGfzyzDnvfkoqECdfAVhqkXWuYLr6iQFpQjt5xueyKyoX2s527wZ87vRp\nIRtnprhyWYZA5CbJlqI8+0nn9fHOskyowa/hyJg0Ow4Os231bYC7BRfUA3D0DZ9Xb9fTS4Xslvf8\nWlLmLsgAIPPAy+vCv9Fk1unrIKrHzxDCJDkLEX849sY3jBxiA98wckiTVueF5vqsHDHLRfOBpcyA\nfEPPj2K/c9xEqKf6frtnuQwUOVzxbQyOyCn7Iubp1tFaFrK2Hr+9pU3Kxie9iUtfuyHmyTeozGTF\nFl9Pb7cKxMECeHZ3yOl8WyuL/6/nmGx67ZTaMc62RyNeki1qZeIkU0n27XxKyMrdC5Ly9bfcIWTt\nYuqvYvwHNySxCbp+zrLHq9emYV6HksVi50dMi9mNl7H8lB574xtGDrGBbxg5xAa+YeSQ5rjsZrB8\npCOqZNyKxVVM6f8xF+GIqY/p+AuWSR1/1TWrk/LlGBSy+e1e5x4dkuauIWb6ayGZjKKlzHLStcpb\nNL/L1zk2Kr8NlNi+ZXVcgbkTT6hVdqOjfrtYDH/fcBV90Vh0ngmpW05Muql2AyCjFo30nxWyHY/8\nNCmfPX1KyG7ddGdS7p6v3I5ZbsJYzr24HTdmIgweNUVrjX1kcCKXZPa+iJyCyjU8hL3xDSOH2MA3\njBzSXHNeHQ5VYsauAmpyz7rUtCecVk/Wmcq5R2EZW50HNZW6YfmipFzuV+auip9+jrfIOk8P+mns\n+IjKZdfBbovKuefYqjedJhtse0it3CM2hdfBS/hKt5LKgcen0BV1A0dGfV+GRmUgjhEWeHRyUqtq\nYTWgv/9YUt659WdCNnDSy669ZYOQrbzqmqTc2aVyAzBzol7Vl33KHrMb6wPDQWRFjRmn/dO1x3MY\njA/1h3dk2BvfMHKIDXzDyCE28A0jhzRHx7+gI2v9KpIDT+g/sYiFqXAkYbMcV221ZYpv6wg8PHrN\n6MmjQjZ2wm8PKzNZhXyUnYFxFaSTfBKNcwMnhazT+dsyUtFBQXkeP6njj014PVub5fh2Qbneymuo\nctnx66Ku2RgzSZ4ZlO7D3LpXUBk8eCSfSbWqr8IOJCfNowdeejopH37jNSG74p23JOXr1q4XshWr\nrkzK7e1yxR8VYs9S5CGMBM3M7DOs9HgeLFV/Tykg/A1qctRfp8FzMrBpiEwDn4j2AziHqiPwuHNu\nPRH1AvgBgFUA9gP4mHPuXLASwzAuGbJO9SsA7nLO3eqcu/Bzej+Ax5xzqwFsBfDF2eigYRgzT9ap\nPiH9I/FRABdcqR4A8DiqPwYpLkznUoEx2HQwbV7jJpGwaSOdcy881edTeD1tnWR/0LHzuafb4BkZ\n/OI88y6rjKtzKPjp/HlleRsc96vshgbklHYMXkVoGZdmMt63MRWrn0+9W0ry1ra2erOc08nz2I2Z\nVBdmfIJ5jKnjJtg0fUCtTOSz5tYW5UXI6tHnIPdTjxwzZY6dPS5Eb+54PCkf3vuKkF127Y1JefVa\nueJv5RVMDeiUKxp5Cu86YntKL7vo6lFdDTdz6hyGLOCrymEweNY/g6dPSLUxRNY3vgPwKBE9S0S/\nX/vbUudcX7WT7hiAJcGjDcO4pMj6xt/inDtKRIsBPEJEezDFt7PQwQ9870fVHQhYe9M7ccvNN4Z2\nNQyjQZ7d+RKe/OUvM+2baeA7547W/j9BRD8FsB5AHxEtdc71EdEyAMdDx9/7O/+uWs8Mxw0zDMOz\n7ta1uHrFwmT77370s+C+0w58IuoAUHDODRBRJ4APAvhvAB4C8AkAXwZwL4AHQ3Vc0AW1i+Ikc3/V\nK7+4Ca2iXExlggtZJ99Mmey4jq/bY9uTWsdn28PUImTHx73uPHhSriabqPjL2z8m9bLhUe+mW6wo\nnY2dX7ksXWh5v0dVcE/uNtvSIvtZZC68k/qacZOdEopgn0rnHmPfH/S3gQ72TaGoTJIjY/6+j6pv\nGCWm/7eVWoWMd7RVmSQnJ/03hpGTbwvZvrMnkvKxN3cL2dKrb0jK1yv9f9WVVyflclnmG5QW5ewf\nAGKrTqMWQsb4mPyecn7Au+m+sW9/pC+eLG/8pQD+kaoxmUoAvuece4SIngPwQyL6JIADAD6WqUXD\nMOacaQe+c+4tALdM8ffTAO6ejU4ZhjG7NMlzrzpvmVRTPj7V19Nr7tGV8u4SU31tBuT7hafzqitK\npuPOs+0WOeXrWbPJ7/f2QSHrP8tWSimTHYb89siY9HobYXHuaUhOhfkUsFKUXmiFed4cNT4+LGTj\nzLtLzyK5w6G+R2KFmlolyRcAdimVpMCE4+p6jrDpfSo0JA/uoY6bYNspHzvHPf6UkJlOR08eFqJD\n57wacPLAXil7561Jee36LULWu8CvykyZHSPx/2WfU3/JtK9WYQfYM/L6vgPBOjjmq28YOcQGvmHk\nEBv4hpFDmhNss6a7xKLlpHPZh6O0CFkkAGT0uGjQRSnhOqNeadbJdL323oVCdhlPLad053Gm504o\nk5YwcyrzGo8m097RJWRltvJsqF+aFvfufDIpn+6Teq6I5FNREX+Y80VBma2KBebmrC7aMHP11S6m\n/JvCuPp+UxL6sTRXdjLX31aVi7DEblJR6dztLPBoi4poNFnxbYyfOCRkrz7pXVOO7H9DyNas25yU\nV7HoPwDQ2dmdlNP6P9/Orv/zZ3dcmXGHR/2z5UiZQAPYG98wcogNfMPIIc1Jk12b0aQCMjgWAFKZ\nMoqROOKTPP64st1w856yPonUzZPqJ08GBQkHDNHmQxmrPxzgIhVbngcFqeg6eR3KTFb0t6xYCAfG\n7OiQK83mvfc3k/KBPS8K2cHXfVCL4SEZ+JNfQm1WHR72+w6Pq3jucumlEnlZe4s8h3kd3uOwXQUM\nbWWBQHX4/zY2hVdVCvVBL0wUIff1AzPhTaJn9r0qRE8d9wFY9rJAHwBwzWrvDbh0+WWyn0wdKxXV\ntFw8L/qa+e2BQWkaPn3ySFKenFDm3wD2xjeMHGID3zByiA18w8ghTdHxL+ie2j2zxBQsvcCJb1ZU\nvvoi+zaQtuaFo5hw/Tx9HNOvdFBQro/Lw+Rxuk5WTT0rBStCPdbZz/32RKpSb+bRrqLFlvakvHrt\nZiG7do0PTlmZkCu/+BmPjkjd8tWXn0/Kb+yWOvAI0//V4koU2fumXJZ6bk+nd4nuapMrDLkFr5Qy\nubopy4DU61NBfdiniXQCFt/xVqU7jzNz6aFfnRGyg6/7FYDz5i8QsvkLvclXJ/5oafW5ECsqcQtf\nNdnfL5NmHDnidfxTJ6QZN4S98Q0jh9jAN4wc0qSp/tS/LyLEuJq68UCHOtebiDmemgozUSp2frap\nfjrmfkR94AE8dGpjUZGOrMj6kl6jFuwL9+rTQUG5uU2rD3zqr2Pu86Ad5dZ2JfOPSHtnj5Ctf1dv\nUl6ybIWQ7dzhQ0CdUemu+UrMM4NSteAejYu65ErIRSxFeLlDqgjcgqc9DMVToTwo+aZKGyjue7Gg\n77u/1mU1LR8b9FHmzw3KafmZI34FZ0W9d4nC72F+zSaU2sFNeIVJS5NtGEYAG/iGkUNs4BtGDmnS\n6rwq+lfGcT1e+1Iy90mnjiyIjwPyMK7apgJ4CmHYhJby6oy67PIqtftwOCgoX62n+xlz55W6ntTn\n+Co/rf/zqEXa1Me/wZTU6jWu/7cqX1geiHPJiquE7I4tXh9/Zed2Ies74lfBVVQ/B1lSkvF+GZlo\ngK0i7B0uC1lPu9f5O8vysW4pcJOrOnem2OtvUUUmG1crKKng+61dmUsiR6O6t2DfYdS3q1guyQp7\nCMegTH2FyMMbwN74hpFDbOAbRg5pTrDNUCCOCLF9o4k5xGw+vHIv7S0XNvWJmPtRLzt5nGxPydgf\nYnWm+8mnmNqcF55+VkSgyphHY1i1GBsLp97W7nmltvlJ+eobZbx6Knr14cTb+4XMMdOYThXN1YDK\nkDQDjrDVgd3K429emZkrtbrC1ABSq/P4dWppVXkKRKDYcL4BrVrEQvBXos+uP7CsVmW6CvdkjQTk\nZ9gb3zByiA18w8ghNvANI4dk0vGJqAfA/wSwBlX/x08C2AvgBwBWAdgP4GPOuXNT11DVO7TuzJV1\nHYFHOrvqQJy8HEuaEdHjIzKdW47L9EqzWKIDF7HPxM4h9i1C5BTUK7giJkKxalHnbKuEdXVk7Iv2\nNnUsx2C5o1fIVl59U1IuKD/Zs8d8QghSOQW5xVc7OY+ycx9Mmd5853R8mjKL8qM8dsHT8+kAntq9\nl9NW8KZFfW+zksoXya51QedT5N92KjPrsvt1AD93zt0AYC2A3QDuB/CYc241gK0AvpixLsMw5php\nBz4RdQN4t3PuOwDgnJuovdk/CuCB2m4PAPjNQBWGYVxiZJnqXwngJBF9B9W3/XMAPgNgqXOuDwCc\nc8eIaEmoAj1FTBAz4XBAxpSZDJEpe8yEJlbZRabCdUzLpUweFz2/WF+EShLJG6jNedzEpE5ezBxJ\nm5h4XHYdaJSbu5SKwKa/Be0NyN4pJZVvcN78xUl55dVrhYzve/bom0IGlgpbxeEUpzSh7sMonyar\nlYmVcS/TdZbZvqlngt2jlqIaRjwoiFarmL6SumbCezXs1afh6q1OOx4iy1S/BOA2AN90zt0GYBDV\naX7kMTcM41Imyxv/MIBDzrnnatv/gOrA7yOipc65PiJaBuB4qIK/+/4/JOW1N92AtTe98yK6bBjG\nVLx14jzeOBb4vq6YduDXBvYhIrrOObcXwPsBvFL79wkAXwZwL4AHQ3X8+9/+t5k6YxhG41y5eB4u\nm+8DqTy5ty+4b1aX3fsAfI+IWgDsA/B7qFo/fkhEnwRwAMDHgkcnSoDWH8PagYu4kcbca+VxYVk6\nsKJHu7QKPU3rx6ys9SahVqv2uD5XqWidO+xeG443hOjyLh7FKLVCLRAhqdqCr6igz5DXWVG6Mw9w\nqa5ZqeTNXV3dMhjlyqu8qa+tXeYGPH7QB7EcGzkv6+TNpx4r9u1D3fh2FmFIR8AZF9dJn0MxJBJm\nwIK68fy+62eCXyZSxsWYq+8YW6XZFtuRkWngO+deArBuCtHdmVoxDOOSwjz3DCOHNGl1XrqU3kfL\neFKz7KawWHxLUbtO+SzmWfL3kDdR0LNrkTtPyriKolfgEfd6U6vCCizSCJH2xOLejprwSrPJSR5M\nVPUlUNakVDMeoFT1JuZlxy91sSQfwY4un2J62arrhayNyY7s+5WQDZ89kZQnVItieq/VjqKXFXX8\nfzbb1rkdqBie6vPrWyqq5yyyopGrY/r55KpFKpho0QcoqUyEA7dy7I1vGDnEBr5h5JCmDfyXdr06\n/U5N4pVXd0+/UxPYvfvS6AcAHNj/5vQ7NYm39r0x111IOHSyf/qdmsQbR8/OWF3NCbbpHF7e9SrW\nrrlBCbgenzooUE5VrbalYW6q1l55dTfW3PhOJfO/gaTMWzwoaDHVXrZEHAXlXlugCt54fS9uWrMG\nBa3jc3Oe1i3ZdioJCTvfgvo2ME7elTO9ShI4dPAtXHX1teKbRSL0LSoR+9aiXX3DR6m/pN2A9+9/\nE1ddcx3KZZncY+HSK5JyuU2a+g7s3ZmU+48fFjKxalFdawd+XdLfi/b1ncXC7k60pcyx/j60qkCx\nJRYhp6D8gKVbrrq3XKIuWkupiH0n+nHDFYtVHSpHI82cy65hGP/KsIFvGDmEGg0UkLkBvaTLMIym\n4bSbZo1ZH/iGYVx62FTfMHKIDXzDyCE28A0jh8z6wCeie4hoNxHtJaI/mu32VNvfIqI+InqZ/a2X\niB4hoj1E9HAtgnAz+rKSiLYS0StEtIuI7pur/hBRmYi2E9HOWl/+eK76Umu3QEQvENFDc9mPWtv7\nieil2rXZMVf9IaIeIvoREb1We2Y2zGQ/ZnXgU9Xb5C8B/BqAGwH8FhFdHz9qRvlOrW3OXEUHngDw\nWefcjQA2AfhU7Vo0vT/OuVEA73XO3QrgFgAfIqL1c9GXGp8GwF075zKCcwXAXc65W51z6+ewP7Mb\n2do5N2v/AGwE8E9s+34AfzSbbU7Rh1UAXmbbu1ENFAoAywDsbmZ/WD9+imo8gzntD4AOVAOorpuL\nvgBYCeBRAHcBeGiu7xGAtwAsVH9ran8AdAN4c4q/z1g/ZnuqvwLAIbZ9uPa3uWSJY9GBAQSjA88W\nRPQOVN+026CiFTerP7Xp9U4AxwA86px7do768lUAX4D03Z2Ta1LDAXiUiJ4lot+fo/4kka1rKtDf\nElHHTPbDPu41OTowEXUB+DGATzvnBqZovyn9cc5VXHWqvxLAeiK6sdl9IaKPAOhzzr2IaUIBzGY/\nFFtcNZr0h1FVx949Rfuz3Z9Zj2w92wP/bQBXsO2Vtb/NJX1EtBQAposOPNMQUQnVQf9d59yF4KRz\n1h8AcM71A3gcwD1z0JctAH6DiPYB+N8A3kdE3wVwbK6uiXPuaO3/E6iqY+vR/OsyVWTr22ayH7M9\n8J8FcA0RrSKiVgAfB/DQLLepIci3yUOoRgcGpokOPAt8G8Crzrmvz2V/iGjRhS/CRNQO4AMAXmt2\nX5xzX3LOXeGcuwrVZ2Orc+53Afysmf24ABF11GZkIKJOAB8EsAvNvy59AA4R0XW1P12IbD1z/WjC\nx5J7AOwB8DqA+5v1kabW9vcBHAEwCuAgqtGBewE8VuvTIwDmN6kvW1AN9/oigJ0AXqhdmwXN7g+A\nm2rtvwjgZQD/tfb3pveF9elO+I97c9IPVHXrC/dn14XndY7u0VpUX5wvAvgJgJ6Z7If56htGDrGP\ne4aRQ2zgG0YOsYFvGDnEBr5h5BAb+IaRQ2zgG0YOsYFvGDnk/wNKPet8hTX3lQAAAABJRU5ErkJg\ngg==\n", "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "# Example of a picture\n", "index = 6\n", "plt.imshow(X_train_orig[index])\n", "print (\"y = \" + str(np.squeeze(Y_train_orig[:, index])))" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "在课程2中，我们曾经对这组数据构建了一个全连接层的神经网络。但考虑到这是一个图像数据集，用卷积神经网络是更为自然的选择。\n", "\n", "开始之前，我们先确定数据的形状。" ] }, { "cell_type": "code", "execution_count": 20, "metadata": { "collapsed": false }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "number of training examples = 1080\n", "number of test examples = 120\n", "X_train shape: (1080, 64, 64, 3)\n", "Y_train shape: (1080, 6)\n", "X_test shape: (120, 64, 64, 3)\n", "Y_test shape: (120, 6)\n" ] } ], "source": [ "X_train = X_train_orig/255.\n", "X_test = X_test_orig/255.\n", "Y_train = convert_to_one_hot(Y_train_orig, 6).T\n", "Y_test = convert_to_one_hot(Y_test_orig, 6).T\n", "print (\"number of training examples = \" + str(X_train.shape[0]))\n", "print (\"number of test examples = \" + str(X_test.shape[0]))\n", "print (\"X_train shape: \" + str(X_train.shape))\n", "print (\"Y_train shape: \" + str(Y_train.shape))\n", "print (\"X_test shape: \" + str(X_test.shape))\n", "print (\"Y_test shape: \" + str(Y_test.shape))\n", "conv_layers = {}" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### 13.1 创建占位符\n", "\n", "Tensorflow要求我们为输入数据创建占位符（placeholders），之后在运行session时，再填充数据给模型。\n", "\n", "**练习**: 实现下面的函数，来为输入图像 $X$ 和 输出 $Y$ 创建占位符。在这个阶段，我们无法需要定义训练样本的数量。我们可以将批次大小设置为None，到后面再进行设置。因此，这时 X 的维度为 **[None, n_H0, n_W0, n_C0]** 而Y的维度为 **[None, n_y]**. [提示](https://www.tensorflow.org/api_docs/python/tf/placeholder)." ] }, { "cell_type": "code", "execution_count": 21, "metadata": { "collapsed": true }, "outputs": [], "source": [ "# GRADED FUNCTION: create_placeholders\n", "\n", "def create_placeholders(n_H0, n_W0, n_C0, n_y):\n", " \"\"\"\n", " Creates the placeholders for the tensorflow session.\n", " \n", " Arguments:\n", " n_H0 -- scalar, height of an input image\n", " n_W0 -- scalar, width of an input image\n", " n_C0 -- scalar, number of channels of the input\n", " n_y -- scalar, number of classes\n", " \n", " Returns:\n", " X -- placeholder for the data input, of shape [None, n_H0, n_W0, n_C0] and dtype \"float\"\n", " Y -- placeholder for the input labels, of shape [None, n_y] and dtype \"float\"\n", " \"\"\"\n", "\n", " ### START CODE HERE ### (≈2 lines)\n", " X = tf.placeholder(tf.float32, shape=[None, n_H0, n_W0, n_C0])\n", " Y = tf.placeholder(tf.float32, shape=[None, n_y])\n", " ### END CODE HERE ###\n", " \n", " return X, Y" ] }, { "cell_type": "code", "execution_count": 22, "metadata": { "collapsed": false }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "X = Tensor(\"Placeholder:0\", shape=(?, 64, 64, 3), dtype=float32)\n", "Y = Tensor(\"Placeholder_1:0\", shape=(?, 6), dtype=float32)\n" ] } ], "source": [ "X, Y = create_placeholders(64, 64, 3, 6)\n", "print (\"X = \" + str(X))\n", "print (\"Y = \" + str(Y))" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "**预期输出**\n", "\n", " \n", "\n", "\n", "\n", "\n", "\n", "\n", "
\n", " X = Tensor(\"Placeholder:0\", shape=(?, 64, 64, 3), dtype=float32)\n", "\n", "
\n", " Y = Tensor(\"Placeholder_1:0\", shape=(?, 6), dtype=float32)\n", "\n", "
" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### 13.2 参数初始化\n", "\n", "我们接下来会使用 `tf.contrib.layers.xavier_initializer(seed = 0)` 来初始化权重/过滤器 $W1$ 和 $W2$。这里我们不需要关心截距变量，很快我们会看到，Tensorflow自动处理了截距项。另外，我们也只需要初始化 conv2d 函数的权重/过滤器，Tensorflow会自动为全连接层初始化参数。\n", "\n", "**练习:** 实现 initialize_parameters(). 每组过滤器的维度下面已经提供了。提醒，在Tensorflow中，要初始化一个维度为 [1,2,3,4] 的参数 $W$，使用：\n", "```python\n", "W = tf.get_variable(\"W\", [1,2,3,4], initializer = ...)\n", "```\n", "[更多信息](https://www.tensorflow.org/api_docs/python/tf/get_variable)." ] }, { "cell_type": "code", "execution_count": 23, "metadata": { "collapsed": true }, "outputs": [], "source": [ "# GRADED FUNCTION: initialize_parameters\n", "\n", "def initialize_parameters():\n", " \"\"\"\n", " Initializes weight parameters to build a neural network with tensorflow. The shapes are:\n", " W1 : [4, 4, 3, 8]\n", " W2 : [2, 2, 8, 16]\n", " Returns:\n", " parameters -- a dictionary of tensors containing W1, W2\n", " \"\"\"\n", " \n", " tf.set_random_seed(1) # so that your \"random\" numbers match ours\n", " \n", " ### START CODE HERE ### (approx. 2 lines of code)\n", " W1 = tf.get_variable('W1', [4, 4, 3, 8], initializer=tf.contrib.layers.xavier_initializer(seed = 0))\n", " W2 = tf.get_variable('W2', [2, 2, 8, 16], initializer=tf.contrib.layers.xavier_initializer(seed = 0))\n", " ### END CODE HERE ###\n", "\n", " parameters = {\"W1\": W1,\n", " \"W2\": W2}\n", " \n", " return parameters" ] }, { "cell_type": "code", "execution_count": 24, "metadata": { "collapsed": false }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "W1 = [ 0.00131723 0.14176141 -0.04434952 0.09197326 0.14984085 -0.03514394\n", " -0.06847463 0.05245192]\n", "W2 = [-0.08566415 0.17750949 0.11974221 0.16773748 -0.0830943 -0.08058\n", " -0.00577033 -0.14643836 0.24162132 -0.05857408 -0.19055021 0.1345228\n", " -0.22779644 -0.1601823 -0.16117483 -0.10286498]\n" ] } ], "source": [ "tf.reset_default_graph()\n", "with tf.Session() as sess_test:\n", " parameters = initialize_parameters()\n", " init = tf.global_variables_initializer()\n", " sess_test.run(init)\n", " print(\"W1 = \" + str(parameters[\"W1\"].eval()[1,1,1]))\n", " print(\"W2 = \" + str(parameters[\"W2\"].eval()[1,1,1]))" ] }, { "cell_type": "markdown", "metadata": { "collapsed": true }, "source": [ "** 预期输出:**\n", "\n", " \n", "\n", " \n", " \n", " \n", " \n", "\n", " \n", " \n", " \n", " \n", "\n", "
\n", " W1 = \n", " \n", "[ 0.00131723 0.14176141 -0.04434952 0.09197326 0.14984085 -0.03514394
\n", " -0.06847463 0.05245192]\n", "
\n", " W2 = \n", " \n", "[-0.08566415 0.17750949 0.11974221 0.16773748 -0.0830943 -0.08058
\n", " -0.00577033 -0.14643836 0.24162132 -0.05857408 -0.19055021 0.1345228
\n", " -0.22779644 -0.1601823 -0.16117483 -0.10286498]\n", "
" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### 13.3 前向传播\n", "\n", "Tensorflow自带一些函数，可以用来实现卷积的过程。\n", "\n", "- **tf.nn.conv2d(X,W1, strides = [1,s,s,1], padding = 'SAME'):** 给定输入 $X$，一组过滤器 $W1$, 这个函数会用 $W1$ 中所有的过滤器，以卷积过程应用到 X 上。第三个输入 ([1,f,f,1]) 代表过滤器针对输入的所有维度 (m, n_H_prev, n_W_prev, n_C_prev) 分别对应的步长。函数的完整文档可以参考[这里](https://www.tensorflow.org/api_docs/python/tf/nn/conv2d)\n", "\n", "- **tf.nn.max_pool(A, ksize = [1,f,f,1], strides = [1,s,s,1], padding = 'SAME'):** 给定输入 A，这个函数会将大小为 (f, f) 的滑动窗口以步长 (s, s) 对输入实现最大池化。函数的完整文档可以参考[这里(https://www.tensorflow.org/api_docs/python/tf/nn/max_pool)\n", "\n", "- **tf.nn.relu(Z1):** 计算 Z1（可以是任意维度）中所有元素的 ReLU 激活值。函数的完整文档可以参考[这里(https://www.tensorflow.org/api_docs/python/tf/nn/relu)\n", "\n", "- **tf.contrib.layers.flatten(P)**: 给定输入 P，这个函数会将所有样本打散为1D，同时保留批次大小。它返回一个打散后的张量，维度为 [batch_size, k]。函数的完整文档可以参考[这里](https://www.tensorflow.org/api_docs/python/tf/contrib/layers/flatten)\n", "\n", "- **tf.contrib.layers.fully_connected(F, num_outputs):** 给定打散后的输入 F，它返回全连接层计算后的输出。函数的完整文档可以参考[这里](https://www.tensorflow.org/api_docs/python/tf/contrib/layers/fully_connected)\n", "\n", "上面的最后一个函数 (`tf.contrib.layers.fully_connected`)，这个全连接层会自动初始化权重，并随着模型训练，自动更新权重。因而我们不需要手动为全连接层初始化权重。\n", "\n", "**练习**: \n", "\n", "实现下面的 `forward_propagation` 函数，来构建: `CONV2D -> RELU -> MAXPOOL -> CONV2D -> RELU -> MAXPOOL -> FLATTEN -> FULLYCONNECTED` 的模型。我们需要使用上面说明的这些函数。\n", "\n", "具体而言，我们将会对各个步骤使用一下超参：\n", " - Conv2D: 步长 1，同一补齐\n", " - ReLU\n", " - Max pool: 8 × 8 过滤器，8 × 8 步长，同一补齐\n", " - Conv2D: 步长 1，同一补齐\n", " - ReLU\n", " - Max pool: 4 × 4 过滤器，4 × 4 stride，同一补齐\n", " - 打散之前的输出\n", " - FULLYCONNECTED (FC) layer: 使用全连接层，最后不需要再跟非线性的激活函数。请不要在这里调用softmax函数。这里会输入6个神经元，之后作为输入传递给softmax。在Tensorflow中，softmax函数和成本函数一起，形成了单一的函数，我们会在计算成本时调用这个函数。" ] }, { "cell_type": "code", "execution_count": 25, "metadata": { "collapsed": true }, "outputs": [], "source": [ "# GRADED FUNCTION: forward_propagation\n", "\n", "def forward_propagation(X, parameters):\n", " \"\"\"\n", " Implements the forward propagation for the model:\n", " CONV2D -> RELU -> MAXPOOL -> CONV2D -> RELU -> MAXPOOL -> FLATTEN -> FULLYCONNECTED\n", " \n", " Arguments:\n", " X -- input dataset placeholder, of shape (input size, number of examples)\n", " parameters -- python dictionary containing your parameters \"W1\", \"W2\"\n", " the shapes are given in initialize_parameters\n", "\n", " Returns:\n", " Z3 -- the output of the last LINEAR unit\n", " \"\"\"\n", " \n", " # Retrieve the parameters from the dictionary \"parameters\" \n", " W1 = parameters['W1']\n", " W2 = parameters['W2']\n", " \n", " ### START CODE HERE ###\n", " # CONV2D: stride of 1, padding 'SAME'\n", " Z1 = tf.nn.conv2d(X, W1, [1, 1, 1, 1],'SAME')\n", " # RELU\n", " A1 = tf.nn.relu(Z1)\n", " # MAXPOOL: window 8x8, sride 8, padding 'SAME'\n", " P1 = tf.nn.max_pool(A1, [1, 8, 8, 1], [1, 8, 8, 1], 'SAME')\n", " # CONV2D: filters W2, stride 1, padding 'SAME'\n", " Z2 = tf.nn.conv2d(P1, W2, [1, 1, 1, 1], 'SAME')\n", " # RELU\n", " A2 = tf.nn.relu(Z2)\n", " # MAXPOOL: window 4x4, stride 4, padding 'SAME'\n", " P2 = tf.nn.max_pool(A2, [1, 4, 4, 1], [1, 4, 4, 1], 'SAME')\n", " # FLATTEN\n", " P2 = tf.contrib.layers.flatten(P2)\n", " # FULLY-CONNECTED without non-linear activation function (not not call softmax).\n", " # 6 neurons in output layer. Hint: one of the arguments should be \"activation_fn=None\" \n", " Z3 = tf.contrib.layers.fully_connected(P2, 6, activation_fn=None)\n", " ### END CODE HERE ###\n", "\n", " return Z3" ] }, { "cell_type": "code", "execution_count": 26, "metadata": { "collapsed": false }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Z3 = [[-0.44670227 -1.57208765 -1.53049231 -2.31013036 -1.29104376 0.46852064]\n", " [-0.17601591 -1.57972014 -1.4737016 -2.61672091 -1.00810647 0.5747785 ]]\n" ] } ], "source": [ "tf.reset_default_graph()\n", "\n", "with tf.Session() as sess:\n", " np.random.seed(1)\n", " X, Y = create_placeholders(64, 64, 3, 6)\n", " parameters = initialize_parameters()\n", " Z3 = forward_propagation(X, parameters)\n", " init = tf.global_variables_initializer()\n", " sess.run(init) \n", " a = sess.run(Z3, {X: np.random.randn(2,64,64,3), Y: np.random.randn(2,6)})\n", " print(\"Z3 = \" + str(a))" ] }, { "cell_type": "markdown", "metadata": { "collapsed": true }, "source": [ "**预期输出**:\n", "\n", " \n", " \n", " \n", "
\n", " Z3 =\n", " \n", " [[-0.44670227 -1.57208765 -1.53049231 -2.31013036 -1.29104376 0.46852064]
\n", " [-0.17601591 -1.57972014 -1.4737016 -2.61672091 -1.00810647 0.5747785 ]]\n", "
" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### 13.4 计算成本\n", "\n", "实现下面的计算成本函数。下面两个函数可能会有帮助：\n", "\n", "- **tf.nn.softmax_cross_entropy_with_logits(logits = Z3, labels = Y):** 计算softmax熵损失。这个函数会同时计算softmax激活函数和损失。完整的文档可以查看[这里](https://www.tensorflow.org/api_docs/python/tf/nn/softmax_cross_entropy_with_logits)\n", "- **tf.reduce_mean:** 计算一个张量不同维度之间的平均值。使用这个函数来对所有的训练样本求损失之和。完整的文档可以查看[这里](https://www.tensorflow.org/api_docs/python/tf/reduce_mean)\n", "\n", "**练习**: 使用上面的两个函数，计算成本" ] }, { "cell_type": "code", "execution_count": 27, "metadata": { "collapsed": true }, "outputs": [], "source": [ "# GRADED FUNCTION: compute_cost \n", "\n", "def compute_cost(Z3, Y):\n", " \"\"\"\n", " Computes the cost\n", " \n", " Arguments:\n", " Z3 -- output of forward propagation (output of the last LINEAR unit), of shape (6, number of examples)\n", " Y -- \"true\" labels vector placeholder, same shape as Z3\n", " \n", " Returns:\n", " cost - Tensor of the cost function\n", " \"\"\"\n", " \n", " ### START CODE HERE ### (1 line of code)\n", " cost = tf.reduce_mean(tf.nn.softmax_cross_entropy_with_logits(logits = Z3, labels = Y))\n", " ### END CODE HERE ###\n", " \n", " return cost" ] }, { "cell_type": "code", "execution_count": 28, "metadata": { "collapsed": false }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "cost = 2.91034\n" ] } ], "source": [ "tf.reset_default_graph()\n", "\n", "with tf.Session() as sess:\n", " np.random.seed(1)\n", " X, Y = create_placeholders(64, 64, 3, 6)\n", " parameters = initialize_parameters()\n", " Z3 = forward_propagation(X, parameters)\n", " cost = compute_cost(Z3, Y)\n", " init = tf.global_variables_initializer()\n", " sess.run(init)\n", " a = sess.run(cost, {X: np.random.randn(4,64,64,3), Y: np.random.randn(4,6)})\n", " print(\"cost = \" + str(a))" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "**预期输出**: \n", "\n", "\n", " \n", " \n", " \n", "
\n", " cost =\n", " \n", " 2.91034\n", "
" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### 13.4 模型\n", "\n", "最后我们需要把这些辅助函数组合在一起构建一个模型，然后使用SIGNS数据集来训练。\n", "\n", "**练习**: 完成下面的函数\n", "\n", "模型需要包括以下的组成成分:\n", "\n", "- create placeholders\n", "- initialize parameters\n", "- forward propagate\n", "- compute the cost\n", "- create an optimizer\n", "\n", "最后我们需要创建一个session，然后在for循环中迭代num_epochs次，每次取一个微批，对其进行优化。[初始化变量的提示](https://www.tensorflow.org/api_docs/python/tf/global_variables_initializer)" ] }, { "cell_type": "code", "execution_count": 29, "metadata": { "collapsed": true }, "outputs": [], "source": [ "# GRADED FUNCTION: model\n", "\n", "def model(X_train, Y_train, X_test, Y_test, learning_rate = 0.009,\n", " num_epochs = 100, minibatch_size = 64, print_cost = True):\n", " \"\"\"\n", " Implements a three-layer ConvNet in Tensorflow:\n", " CONV2D -> RELU -> MAXPOOL -> CONV2D -> RELU -> MAXPOOL -> FLATTEN -> FULLYCONNECTED\n", " \n", " Arguments:\n", " X_train -- training set, of shape (None, 64, 64, 3)\n", " Y_train -- test set, of shape (None, n_y = 6)\n", " X_test -- training set, of shape (None, 64, 64, 3)\n", " Y_test -- test set, of shape (None, n_y = 6)\n", " learning_rate -- learning rate of the optimization\n", " num_epochs -- number of epochs of the optimization loop\n", " minibatch_size -- size of a minibatch\n", " print_cost -- True to print the cost every 100 epochs\n", " \n", " Returns:\n", " train_accuracy -- real number, accuracy on the train set (X_train)\n", " test_accuracy -- real number, testing accuracy on the test set (X_test)\n", " parameters -- parameters learnt by the model. They can then be used to predict.\n", " \"\"\"\n", " \n", " ops.reset_default_graph() # to be able to rerun the model without overwriting tf variables\n", " tf.set_random_seed(1) # to keep results consistent (tensorflow seed)\n", " seed = 3 # to keep results consistent (numpy seed)\n", " (m, n_H0, n_W0, n_C0) = X_train.shape \n", " n_y = Y_train.shape[1] \n", " costs = [] # To keep track of the cost\n", " \n", " # Create Placeholders of the correct shape\n", " ### START CODE HERE ### (1 line)\n", " X, Y = create_placeholders(n_H0, n_W0, n_C0, n_y)\n", " ### END CODE HERE ###\n", "\n", " # Initialize parameters\n", " ### START CODE HERE ### (1 line)\n", " parameters = initialize_parameters()\n", " ### END CODE HERE ###\n", " \n", " # Forward propagation: Build the forward propagation in the tensorflow graph\n", " ### START CODE HERE ### (1 line)\n", " Z3 = forward_propagation(X, parameters)\n", " ### END CODE HERE ###\n", " \n", " # Cost function: Add cost function to tensorflow graph\n", " ### START CODE HERE ### (1 line)\n", " cost = compute_cost(Z3, Y)\n", " ### END CODE HERE ###\n", " \n", " # Backpropagation: Define the tensorflow optimizer. Use an AdamOptimizer that minimizes the cost.\n", " ### START CODE HERE ### (1 line)\n", " optimizer = tf.train.AdamOptimizer(learning_rate).minimize(cost)\n", " ### END CODE HERE ###\n", " \n", " # Initialize all the variables globally\n", " init = tf.global_variables_initializer()\n", " \n", " # Start the session to compute the tensorflow graph\n", " with tf.Session() as sess:\n", " \n", " # Run the initialization\n", " sess.run(init)\n", " \n", " # Do the training loop\n", " for epoch in range(num_epochs):\n", "\n", " minibatch_cost = 0.\n", " num_minibatches = int(m / minibatch_size) # number of minibatches of size minibatch_size in the train set\n", " seed = seed + 1\n", " minibatches = random_mini_batches(X_train, Y_train, minibatch_size, seed)\n", "\n", " for minibatch in minibatches:\n", "\n", " # Select a minibatch\n", " (minibatch_X, minibatch_Y) = minibatch\n", " # IMPORTANT: The line that runs the graph on a minibatch.\n", " # Run the session to execute the optimizer and the cost, the feedict should contain a minibatch for (X,Y).\n", " ### START CODE HERE ### (1 line)\n", " _ , temp_cost = sess.run([optimizer, cost], feed_dict={X: minibatch_X, Y: minibatch_Y})\n", " ### END CODE HERE ###\n", " \n", " minibatch_cost += temp_cost / num_minibatches\n", " \n", "\n", " # Print the cost every epoch\n", " if print_cost == True and epoch % 5 == 0:\n", " print (\"Cost after epoch %i: %f\" % (epoch, minibatch_cost))\n", " if print_cost == True and epoch % 1 == 0:\n", " costs.append(minibatch_cost)\n", " \n", " \n", " # plot the cost\n", " plt.plot(np.squeeze(costs))\n", " plt.ylabel('cost')\n", " plt.xlabel('iterations (per tens)')\n", " plt.title(\"Learning rate =\" + str(learning_rate))\n", " plt.show()\n", "\n", " # Calculate the correct predictions\n", " predict_op = tf.argmax(Z3, 1)\n", " correct_prediction = tf.equal(predict_op, tf.argmax(Y, 1))\n", " \n", " # Calculate accuracy on the test set\n", " accuracy = tf.reduce_mean(tf.cast(correct_prediction, \"float\"))\n", " print(accuracy)\n", " train_accuracy = accuracy.eval({X: X_train, Y: Y_train})\n", " test_accuracy = accuracy.eval({X: X_test, Y: Y_test})\n", " print(\"Train Accuracy:\", train_accuracy)\n", " print(\"Test Accuracy:\", test_accuracy)\n", " \n", " return train_accuracy, test_accuracy, parameters" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "执行下面的代码块，对模型训练100个epochs。" ] }, { "cell_type": "code", "execution_count": 30, "metadata": { "collapsed": false }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Cost after epoch 0: 1.917929\n", "Cost after epoch 5: 1.506757\n", "Cost after epoch 10: 0.955359\n", "Cost after epoch 15: 0.845802\n", "Cost after epoch 20: 0.701174\n", "Cost after epoch 25: 0.571977\n", "Cost after epoch 30: 0.518435\n", "Cost after epoch 35: 0.495806\n", "Cost after epoch 40: 0.429827\n", "Cost after epoch 45: 0.407291\n", "Cost after epoch 50: 0.366394\n", "Cost after epoch 55: 0.376922\n", "Cost after epoch 60: 0.299491\n", "Cost after epoch 65: 0.338870\n", "Cost after epoch 70: 0.316400\n", "Cost after epoch 75: 0.310413\n", "Cost after epoch 80: 0.249549\n", "Cost after epoch 85: 0.243457\n", "Cost after epoch 90: 0.200031\n", "Cost after epoch 95: 0.175452\n" ] }, { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYkAAAEZCAYAAABiu9n+AAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3XecVPX1//HXQUARKWIXFEUxGoJiA5QoiAZFRRMj9hI1\nkcQaSwrGX0C/ydfu167RACopEgkqtohtwRKVUMQoKASkgwWQxQbI+f1x7oZh2dkdlpmd9n4+HvPY\nmTt37j33inPm083dERERqUmjfAcgIiKFS0lCRETSUpIQEZG0lCRERCQtJQkREUlLSUJERNJSkpCS\nYmbPmNmZ+Y5DpFQoSUhWmNksM+ud7zjc/Wh3H57vOADM7GUzO7cBztPUzIaa2WdmtsDMLqtj/9PM\n7EMzqzSzUWbWOtNjmVk/M3vHzJab2atmtleurksKg5KEFA0z2yTfMVQppFiAa4DdgJ2A3sAvzaxP\nTTuaWSfgPuB0YDvgS+DeTI5lZh2BPwHnA62Bp4DRZqbvkRKm/7iSc2Z2rJlNMrOlya/Pzinv/crM\nZiS/TP9tZt9Pee/sZP9bzewTYFCy7RUzu8nMlpjZf8zsqJTP/PfXewb77mJmY5NfzWPM7C4zq7EU\nYmY9zWyumf3SzBYCQ82stZk9aWYfmdmnyfMdk/1/BxwC3JVc2x3J9j2Tc31qZlPNrH8WbvFZwLXu\nvtzdpwH3Az9Ks+9pwGh3f83dvwD+H3CCmTXP4Fh9gFfc/Z/uvga4AWgL9MzCNUiBUpKQnDKzfYEh\nwE+ANsAfiF+fTZJdZgA93L0l8Sv2T2a2XcohuiX7bAv8PmXbVGAr4Kbk+Ol0rWXfvwBvJO9dA5wJ\n1DZPzfbEL+idiV/TjYChxK/unYEvgLsB3P1q4BXgIndv6e6XmNnmwBji1/jWwCnA3Wa2Z00nM7O7\nk8S6JOVv1fPJyT6tgR2AKSkffRvolOYaOiXvk8Q5E/ga2KMex2oEGPCdNO9LCVCSkFz7CXCfu//L\nw3DiS6k7gLv/3d0XJ88fBaYTX+xV5rv7Pe6+xt2/TrZ96O5DPSYeewjYwcy2TXP+2TXta2Y7AQcA\ng9x9tbu/Boyu41q+SfZf5e5fu/sSd38sef45cB1waC2fPxaY5e4PJ/fibWAUUGNpwt0vdPct3b1N\nyt+q512S3bYgEttnKR9dDrRIE8MW1fZN3b+uY70A9DSzQ5MkfxXQBNi8lmuWIqckIbnWHrgi9Vcw\n0A6oqpY5K6Uqainxq3XrlM/PreGYi6qeuPuXydMt0pw/3b47Akvc/as6zpXqY3dfVfXCzJqZ2R+S\nRuBlwFigtZlZms+3B7pXuxenESWU+lqR/G2Zsq0VUFnL/i2rbavav9Zjufv7wNlEaWkBUTJ8D5hX\nz9ilCChJSK7NBX5f7VfwFu4+wsx2Juq8L0i2bwm8S1RhVMnVNMULgTZmtlnKtp3q+Ez1WK4AOgIH\nuntr1pYiLM3+c4GKaveipbtfWNPJzOzepAfS8mqPSjN7B8DdlyXXsk/KR/ch7mNN3k3d18x2I0oD\nH2RyLHcf5e6d3X0bYDCwKzA+zbmkBChJSDY1NbNNUx6bAA8APzWzrgBm1tzMjk4aSpsDa4BPzKyR\nmZ1DA9Vvu/sc4F/AYDNrYmYHAf028DAtiN5By82sDfGlmWox0CHl9VNE3f8ZZtY4Oe8B6dok3P1n\n7t4iSSSpjxbu3jll1+HA1UlD+l5EFd+wNDH/GehnZj2S/wbXAn9PqsvqPJaZ7Zf8t9qGSPCPu/sH\ndd0oKV5KEpJNTxONt18mfwe5+wTii+YuM1sCfEBUWeDuU4FbiMbjRURV06v1OK+neV7XvqcDBwOf\nEF+WjxDtJZm6jaiP/wR4HXim2vu3A/2Tnky3ufsKoofQKUR1zQLgeqDpBpyzJoOAmcBs4CXgend/\nvurNpOTRA8Dd3wN+SjTaLwKaARdmeqzkmpYRnQE+JRrwpYRZLhcdMrN2wMNEf+w1wAPufkcN+90B\n9AU+B37k7pNzFpRIGmb2CDDV3a/JdywihSLXJYnVwOXu3gk4CLiwetHazPoCu7l7R2AAMdBHJOeS\nqp4OFo4CjgMez3dcIoWkcS4P7u6LSHqXuPsKM5tKDL6ZlrLb8URpA3d/08xamdl2Vd0iRXJoe6IL\nahuih85Pk26pIpLIaZJIZWa7AF2AN6u91ZZ1ux7OT7YpSUhOuftTRGOyiKTRIA3XZrYFMBK4NGm8\nExGRIpDzkoSZNSYSxHB3f6KGXeazbv/0dsm26sfJXQu7iEgJc/d0Azzr1BAliaHAe+5+e5r3RxOT\nimFm3YFl6doj3F0PdwYNGpT3GArloXuhe6F7UftjY+W0JJH0zT4deMfMJhF91K8ipidwd7/f3Z9J\nBlfNILrAnpPLmEREJHO57t30GlDnvPvuflEu4xARkfrRiOsi1KtXr3yHUDB0L9bSvVhL9yJ7cjri\nOpvMzIslVhGRQmFmeIE3XIuISJFSkhARkbSKKknMmJHvCEREyktRJYnu3WHAAJi/3lA7ERHJhaJK\nEu+/D61awXe+A337wt13w5w5+Y5KRKR0FWXvpuXL4bnn4Mkn4Zln4IorYODAPAcoIlKANrZ3U1Em\niVTz58N++8ETT0R1lIiIrFX2XWDbtoV77oEzzoDKynxHIyJSWoq+JFHlvPPAHYYObcCgREQKXNmX\nJKrcfju88gqMHJnvSERESkfJlCQAXngBrrwSJk9uoKBERApc2Tdcp1q5Etq0gQULoGXLBgpMRKSA\nqbopRdOm0dPpjTfyHYmISGkoqSQB0KMHvPZavqMQESkNShIiIpJWSbVJACxZArvsEn8b53TdPRGR\nwqc2iWratIF27eCdd/IdiYhI8Su5JAGqchIRyZaSTBIHH6wkISKSDSWZJFSSEBHJjpJMEh07wldf\nwdy5+Y5ERKS4lWSSMFOVk4hINpRkkoCocnr99XxHISJS3Eo6SagkISKycUpuMF2Vr7+OMROLF8MW\nW+QwMBGRAqbBdGlsuinsthtMn57vSEREilfJJgmA3XeHGTPyHYWISPFSkhARkbSUJEREJC0lCRER\nSUtJQkRE0irZLrAAa9ZA8+bwySfxV0Sk3KgLbC0aNYJdd4WZM/MdiYhIcSrpJAGqchIR2RhKEiIi\nkpaShIiIpKUkISIiaSlJiIhIWiXdBRZg9ero/rp8eUz6JyJSTtQFtg6NG0P79jBrVr4jEREpPiWf\nJEBVTiIi9aUkISIiaSlJiIhIWkoSIiKSlpKEiIikldMkYWZDzGyxmU1J835PM1tmZhOTx9W5iGOX\nXWDuXFi1KhdHFxEpXbkuSQwDjqxjn3Huvl/y+F0ugmjaFHbcEWbPzsXRRURKV06ThLu/CiytY7d6\nD/LYEKpyEhHZcIXQJnGQmU02s6fN7Nu5Osnuu8MHH+Tq6CIipalxns8/AdjZ3b8ws77A48Ae6XYe\nPHjwf5/36tWLXr16ZXyizp1h0qR6xykiUhQqKiqoqKjI2vFyPneTmbUHnnT3vTPYdxawv7svqeG9\nes3dVOX11+HSS2H8+HofQkSk6BTD3E1GmnYHM9su5XlXImmtlyCyoXNnePfdmPBPREQyk9PqJjP7\nC9AL2MrM5gCDgKaAu/v9wIlm9jNgFfAlcHKuYmnRAtq1g/ffh06dcnUWEZHSUvJThac66ST4/vfh\ntNOyFJSISIErhuqmgrHPPjB5cr6jEBEpHmWVJLp0UZIQEdkQZZkkiqSGTUQk78oqSey4YySIRYvy\nHYmISHEoqyRhpionEZENUVZJAtR4LSKyIcouSagkISKSOSUJERFJq6wG00EsPNSqFXz8MTRvnoXA\nREQKmAbTbaAmTWCvveCdd/IdiYhI4Su7JAGqchIRyZSShIiIpFW2SWLChHxHISJS+Mqu4Rrgq69g\n661hwQJo2TIrhxQRKUhquK6HzTaDbt3glVfyHYmISGEryyQBcNhh8NJL+Y5CRKSwlW2S6N1bSUJE\npC5l2SYBMahuq61g1qz4KyJSitQmUU9NmkCPHjB2bL4jEREpXGWbJEBVTiIidVGSUJIQEUmrrJNE\nly6xSt3ChfmORESkMJV1kthkEzj0UKioyHckIiKFqayTBKjKSUSkNkoSShIiImmVfZLo1AkqK+HD\nD/MdiYhI4Sn7JGEGRxwBL7yQ70hERApP2ScJgD59YMyYfEchIlJ4ynZajlTz58Pee8NHH0WPJxGR\nUqFpObKgbVvYfnstRCQiUp2SRKJPH3j++XxHISJSWJQkEmqXEBFZn9okEp9/HlVOCxZAixY5O42I\nSINSm0SWNG8OBx6oqcNFRFIpSaRQlZOIyLqUJFIoSYiIrEtJIkWXLvDppzBnTr4jEREpDEoSKRo1\ngu99D557Lt+RiIgUBiWJak48Ef70p3xHISJSGNQFtppVq2DnneHll2HPPXN+OhGRnFIX2Cxr0gTO\nOQfuvz/fkYiI5J9KEjWYORO6dYO5c2GzzRrklCIiOaGSRA506AD77gujRuU7EhGR/MooSZhZ/0y2\nlZLzz1eVk4hIRtVNZjbR3fera1suNWR1E8DKldGAPXYsfOtbDXZaEZGs2tjqplqThJn1BY4GTgJG\npLzVEvi2u3et74k3VEMnCYCBA6O30803N+hpRUSyJtdJYh+gC3At8NuUtyqBl919aX1PvKHykSRm\nzIAePWLlusaNG/TUIiJZkdOGa3d/290fAnZ394eS56OBGZkkCDMbYmaLzWxKLfvcYWbTzWyymXXZ\n4CvIod13jyqniop8RyIikh+Z9m563sxamlkbYCLwgJn9XwafGwYcme7NpDprN3fvCAwA7sswngZz\n8skwYkTd+4mIlKJMk0Qrd18OnAA87O7dgMPr+pC7vwrUVuI4Hng42fdNoJWZbZdhTA2if3947LFo\nmxARKTeZJonGZrYD0YD9VBbP3xaYm/J6frKtYLRvDx07wosv5jsSEZGGl2lz7LXAc8Br7j7ezDoA\n03MXVs0GDx783+e9evWiV69eDXLeqiqno45qkNOJiNRbRUUFFVlsSM35tBxm1h540t33ruG9+4he\nUiOS19OAnu6+uIZ9G7x3U5X586FzZ1i0CJo2zUsIIiL10iDTcphZOzN7zMw+Sh5/N7N2mcaYPGoy\nGjgrOUd3YFlNCSLf2raFTp20ap2IlJ9M2ySGEV/oOyaPJ5NttTKzvwCvA3uY2RwzO8fMBpjZ+QDu\n/gwwy8xmAH8ALqjHNTQI9XISkXKU6bQck929S13bcimf1U0QVU177QV33AErVsTjiCNiIkARkUK1\nsdVNmTZcf2pmZwB/TV6fCnxa35MWo+23h0suiaVNW7SItbBffhmeeSbfkYmI5E6mJYn2wJ3AQYAT\nVUgXu/vcWj+YRfkuSVRXWRltFXPmQOvW+Y5GRKRmDbWexLXA2e6+jbtvC5wLXFPfk5aCFi2gd294\n8sl8RyIikjuZJom9U+dqcvclQNnXxv/whzByZL6jEBHJnUyTRCMz27LqRTKHU9nPi9qvX0z+V1mZ\n70hERHIj0yRxC/BPM/sfM/sfok3ixtyFVRxat4bvfheefjrfkYiI5EZGScLdHyYm91ucPE5w9+G5\nDKxYqMpJREpZzqflyJZC691U5dNPoUMHWLAAmjfPdzQiIutqqN5NksZWW0G3bvCPf+Q7EhGR7FOS\nyAJVOYlIqVKSyIIf/hDGjoW77853JCIi2aUkkQVbbw2vvgp33glXXAFr1qy/zzffwPLl8MUXDR+f\niEh9qeE6i5YsgR/8INopDjgAJk+Ox+zZsHp1NGw3awYzZ6qRW0QahhquC0ibNrHmRMeOsHQpHH98\nrI+9dGkkieXLoWtXePTRfEcqIpIZlSQa2OOPw623wrhx+Y5ERMrBxpYklCQa2KpVsNNOkST22CPf\n0YhIqVN1U5Fp0gTOPBOG1bmun4hI/qkkkQdTp8Lhh8daFI3LfppEEckllSSK0F57wS67aJS2iBQ+\nJYk8OfdcGDIk31GIiNRO1U15UlkZDdjTpsX62SIiuaDqpiLVogWccw4MHJjvSERE0lNJIo9WrIDO\nneGee6Bv33xHIyKlSCWJIrbFFvDHP8KAATEaW0Sk0KgkUQDOPx8aNYL77st3JCJSajTiugR89llU\nOw0dCkccke9oRKSUqLqpBLRqFSOwTzsNnntu/fenT4dPPmn4uERElCQKxOGHx4yxZ5+9dvzEwoUx\nnqJHD9h9dzj1VHj5ZSjRApWIFCAliQLSowe88gpcdx2ccEJUQW2zTZQkZs2Cgw+GSy6Bnj3V0C0i\nDUNtEgXo44+jW+yZZ0KHDuu+t2YNXHwxTJgQ03q0bp2fGEWkOKjhugy5w89/Dq+/HoscbbllviMS\nkUKlhusyZAa33QaHHhptGQsX5jsiESlVShJFygxuvjnaLrp1g0mT8h2RiJQiJYkiZgZXXx3Lofbp\nE72j0lm2rOHiEpHSoSRRAk48EZ59Nno+XXttNG6nuvfe6CU1bVp+4hOR4qWG6xKycCH07w9t2sDw\n4THT7G9+A6NGwYEHwg47wE035TtKEWlI6t0k61i5Ei6/PHo97b03LFgAo0fDp59GQ/e8ebHOtoiU\nB/VuknU0bQp33RVtFVtuCS++CFtvDd/6FuyxBzz1VL4jFJFiopJEGXnoIXj0USUKkXKi6ibJ2Oef\nx5Kp77wDbdvmOxoRaQiqbpKMNW8eDdsPPrh22wcfRO+nCRNg9eq8hSYiBUpJosz8+MexbsVnn8Ev\nfhGTBr72Gpx1VvSK6tsX5szJd5QiUiiUJMrMAQdEiWKXXWDJEvj3v+FPf4J331070+z3vgeLF+c7\nUhEpBGqTKEPjx8dyqfvvX/P7gwfH6O2KCk0eKFLs1HAtWecOV14Z1VAPPgibbgqbbBLVUVtssWHH\nWr06xm5sssnah9X7n6uIbCg1XEvWVU0eePDBcOyx0Lt3LIjUsSNMnbruvu5wzTWxf3Uffgi77hrJ\npUWLGMNxySUNcgkikiUqSUjGhg+Hq66CcePiy98dLrssXq9YARddtDYJLF0aiWXAALj00ti2bFkM\n6nv++RgNLiK5t7ElicbZDKYmZnYUcBtRahni7jdUe78n8AQwM9k0yt1/l+u4ZMOdeSZUVsIRR8DY\nsfD738cU5S+9FL2lDjkEWrWCU06B738fjjpqbYKAWEXvt7+FK66IaUNU7SRS+HJakjCzRsAHwOHA\nAmA8cIq7T0vZpydwhbsfV8exVJIoEDfcAL/7HXTpAk8/DS1bxvZp0+Cww2LJ1R13hBEjooE81apV\nsXb3rbfC0Uc3fOwi5abQ2yS6AtPdfba7rwIeAY6vYT/9piwiv/oV/PnPscZ2VYIA2HPPSBp77hlV\nU9UTBMTkgjffHA3jGrwnUvhynSTaAnNTXs9LtlV3kJlNNrOnzezbOY5JsuC442K8RXX77QdDhsBm\nm6X/7DHHxLTlDzyQu/hEJDty3iaRgQnAzu7+hZn1BR4H9qhpx8GDB//3ea9evejVq1dDxCdZZga3\n3AJHHhm9p3baKd8RiZSOiooKKioqsna8XLdJdAcGu/tRyetfA1698braZ2YB+7v7kmrb1SZRYm68\nMQbtjR0b3WNFJPsKvU1iPLC7mbU3s6bAKcDo1B3MbLuU512JxLUEKXlXXgnbbgu//GXt+61Y0TDx\niMj6cpok3P0b4CJgDPAu8Ii7TzWzAWZ2frLbiWb2bzObRHSVPTmXMUnhaNQoRnSPHh3rXFS3alWs\nsrfVVlHiEJGGp8F0kncTJsSYiquvhhNOiDaKefPg5JNjtPall8IZZ0S32dNOy3e0IsWl0KubROq0\n//5RUpg8OcZedOsGBx4I/frBE0/E4L0XXoipzf/4x/zEOGYM/Oc/+Tm3SD6pJCEFZdWqmH22detI\nFKmmT49pzDt3hnPOiZ5RjRtHArnvPnj1VXj88ZhzKlvWrIFrr40BhMccAyNHZu/YNfnmG+jZEwYN\nimsV2ViaBVbKyooV8UU9bFhMNrjFFjEVyM9+Fm0XF1yQvbmhPv8czj4bFiyAhx+OEs6ECbEWR66M\nGhXzXzVpEsvMbuisuyLVKUlI2ZoxA5Yvh333XTsP1COPxNxQ48bBbrvV/9hLl8Lhh0ey+cMfYrr0\nK6+MSQ1vuSU78VfnDt27x4j20aMj+d1+e27OVZcVK2D+/JiQUYqbkoRINffdF2MwfvCDWHnv3Xfj\n1//TT8cXb6qvv441LhqnDCv98kvo0ydW8bv11rUJaPbsGFH+4Ycx9Xm2VVTErLnvvRcTJn7nO1Fq\nymb1WaZ++9tY5nb6dGjWrOHPL9mjhmuRan7602hH2HZbuPhieOWVKBEce2xUIVV59dVIHvvuC2+8\nEdu++SZ6UO20U5QYUmeqbd8+ShfDhuUm7htuiMb5qgWebr8dzjsPvvoqN+dLZ80aeOgh2HpruPfe\nhj23FB6VJKQsrFkTjd0LF0ZVztChsUzrQw/Fr/bLLoMTT4wv5A8/jFJHTaPA//nP6I77wQfxZZ4t\nU6ZEN+CZM9fOe+UeXYL33z+6BzeUqp5kw4dHUpw+fd2JHKW4qLpJJEOrV8Opp8K//hWTEz7+OOy+\ne7y3ZEl8Mb7/Pjz7bPrqpKp2g4EDY82M6tasgY8/jsbuBQuiLePww+teO+OMM6LX1q9+te72Dz6I\nxZtmzWq4RuzTT49G+ksugbPOiradQYMa5tySfUoSIhtg5coYa3HWWfX/0n300RgJ/vOfx3G22SYa\nuocOhbvuioWZ2raNNTWqlnC9++74W5OJE6O768yZ67eZAPTvH+0Sl11Wv3g3xGefRbXajBlR3TRz\nJnTtGmuFbL117s8v2ac2CZEN0LRpdJPdmF/l/fvDX/8aXVQ7doySQocOMRhwxAj45BN4++0okbz9\ndqzYd+CB0Zi+atW6x3rvvVh86YEHak4QAL/+dTSgr1xZ/5gzNWJEDF6sSggdOsBJJ8H11+f+3FKY\nVJIQ2QiffRajsQ85BLbfPv1+//kPXHhhTDdyzz1w6KGxrWfP+AI+44zaz9OnTywLe+652Y2/uoMO\ngt/8Jhr5qyxcGFVhY8dCp065Pb9kn6qbRIqEO/z971Ft1LMnvPZalBIGDKj7sy+9FCWgd9/NfoP5\nzjvHCPepU6F3b5g7d90uwRA9um6+Gd56q+bFpqRwKUmIFJkVK+B//xfatYsv/kykDrQ74YTsxDFk\nSEzTvnJlJIrNNouqsxtvrPn8Z50V1XVDhmTn/NIwlCREysRjj0WPo65d48u6WbNIGvUZFT1+fMxF\nNW5c9PCaMiVKCccfH0vL1mTFihhgePXVUT22enX0FFuyJKrDqpc+pDAoSYiUCfeodlq2LH79T58e\nPbXGjl2359T778M110R7yddfR2P5scdGm8jmm0cX3QMOgNtui1HpG2LKlChtHHIIvPxyDDrcfPNo\nazn/fPjJT9InmXTmzYseYXvuWXdXYdlwShIiZeyee6KtYNy4qL4aMQIuuiiqkfbaK8ZpuMP998Pr\nr8NVV0WJpHt3+P3v63fOl16KeZ2OOGJtQnj77Rid/be/xWSIqQ3ftfnzn6MrcbNmkSD69oXjjov1\nz7PZ9lLOlCREytzNN0cX2sMOi9HSjz4aU41UN3FizMnUuHE0oOfiS/jNN2MdkOHD44s+nRUrYsqU\n11+PxLbPPtEd+NlnI/5Fi6Jkct55tfcak7opSYgI110XVUH33hs9lfLptddiNPqIEdFbqrr586MN\no2tXuPPOmsesTJoU1zJyZCScY45Z9/3Vq2M0+re/nZtrKCVKEiJScMaOjbmw7r9/3XaPefOixHPe\nedH9ty5vvBHVT88/H6UNiDaWM86ItTeGDat7jEm504hrESk4PXvG0rMDB8aX/IcfxviLXr1iXEgm\nCQKi7eTOO6MKa8GCaIjv3z9m833rrejd9fDDubwSUac1EcmJgw+OBu1bboneVJttFgtCbegcVCef\nHHNJ9esX7RObbhqliKZN4cUXo7eVe6wiKNmn6iYRyblZs2IBqH796vd591gn5KuvottvkyZr33v/\n/ehp1a1b9Ozq2TN6So0fHxMrjhsXVVIXXFCejeBqkxCRsldZGdVOd90VvbeaNYOPPoq1zw87DB58\nMCZl7NcvGs07dYpxGZWVsb7IY4/FwMDevaM666ijYvxHKVCSEBFJVA04/Prr9cdaLFkSyeKtt2IO\nrBkzourqyCOjcb1r12ggHzkySiGXXx5tKqmllmKkJCEiUg+rVkVSqWkFwrlzo4F90aJYvbBz54aP\nL1vUu0lEpB6aNKk5QUBMN/L009HG0bt39MaaP3/dfdyjPeSLL3IfK8Cnn66/HklDUJIQEamBWazf\nMXFidLnt3DnW9Bg1Kpa67dgx1gXZd1+YMCH9cVavjoQzc2b9Y5k6Nebn2mqraC+54QaYPbv+x9sQ\nShIiIrXYaacYqzFrVizKdNdd0TA+cmRUR11zTcw5deONscY5RClj0aKYH2vXXWHw4Bjz8cgjG37+\nVatimvYbb4zxJgMGwJw5sN9+MRdXZWU2r3Z9apMQEdlIs2fDmWfGL/7Vq2Nuqk03hVNPjdl3u3SJ\nqUZOPjkGFN52W+a9p669Nua4evbZdWfJnTcvksSLL8b8XaeeWvPn1XAtIlIA1qyJdovmzWM+qpra\nOyorY7zHmDExwLBLl/h73HE196KaODGqlyZNgrZtaz7vm2/GOuSDB8M556z/vpKEiEiRmTsXJk+O\nR9XU69dfH11xq0oLc+dGNdbAgXD66bUfb9q0GA9yzz3rrxGiJCEiUuTGjInG8BYtYLfdYpT4ihUx\n1chNN2W2GNOECZFU/vrXmKqkipKEiEgJ+OabmF69sjJ6TdVnpb6xY2PE+Jtvrl2tUElCRET+a9q0\nWPe8KsEoSYiISFoacS0iIjmjJCEiImkpSYiISFpKEiIikpaShIiIpKUkISIiaSlJiIhIWkoSIiKS\nlpKEiIikpSQhIiJpKUmIiEhaOU8SZnaUmU0zsw/M7Fdp9rnDzKab2WQz65LrmEREJDM5TRJm1gi4\nCzgS6AScamZ7VtunL7Cbu3cEBgD35TKmUlBRUZHvEAqG7sVauhdr6V5kT65LEl2B6e4+291XAY8A\nx1fb53gnCID8AAAH9UlEQVTgYQB3fxNoZWbb5Tiuoqb/AdbSvVhL92It3YvsyXWSaAvMTXk9L9lW\n2z7za9hHRETyQA3XIiKSVk4XHTKz7sBgdz8qef1rwN39hpR97gNedvcRyetpQE93X1ztWFpxSESk\nHjZm0aHG2QykBuOB3c2sPbAQOAU4tdo+o4ELgRFJUllWPUHAxl2kiIjUT06ThLt/Y2YXAWOIqq0h\n7j7VzAbE236/uz9jZkeb2Qzgc+CcXMYkIiKZK5o1rkVEpOEVRcN1JgPySpWZtTOzl8zsXTN7x8wu\nSbZvaWZjzOx9M3vOzFrlO9aGYGaNzGyimY1OXpfrfWhlZo+a2dTk30a3Mr4Xl5nZv81sipn92cya\nltO9MLMhZrbYzKakbEt7/WY2MBm8PNXM+tR1/IJPEpkMyCtxq4HL3b0TcBBwYXL9vwZecPdvAS8B\nA/MYY0O6FHgv5XW53ofbgWfcfS9gH2AaZXgvzGxH4GJgP3ffm6hCP5XyuhfDiO/HVDVev5l9GzgJ\n2AvoC9xjZrW29xZ8kiCzAXkly90Xufvk5PkKYCrQjrgHDyW7PQR8Pz8RNhwzawccDfwxZXM53oeW\nwCHuPgzA3Ve7+2eU4b1IbAI0N7PGQDNirFXZ3At3fxVYWm1zuus/Dngk+TfzITCd+I5NqxiSRCYD\n8sqCme0CdAHeALar6gXm7ouAbfMXWYP5P+AXQGpDWjneh12BT8xsWFL1dr+ZbU4Z3gt3XwDcAswh\nksNn7v4CZXgvqtk2zfVv8ODlYkgSApjZFsBI4NKkRFG9x0FJ90Aws2OAxUmpqrbicUnfh0RjYD/g\nbnffj+gV+GvK7N8EgJm1Jn41twd2JEoUp1OG96IO9b7+YkgS84GdU163S7aVjaQYPRIY7u5PJJsX\nV81xZWbbAx/lK74G0gM4zsxmAn8FepvZcGBRmd0HiNL0XHf/V/L670TSKLd/EwBHADPdfYm7fwM8\nBhxMed6LVOmufz6wU8p+dX6fFkOS+O+APDNrSgzIG53nmBraUOA9d789Zdto4EfJ87OBJ6p/qJS4\n+1XuvrO7dyD+Dbzk7mcCT1JG9wEgqUaYa2Z7JJsOB96lzP5NJOYA3c1ss6QB9nCiY0O53Qtj3RJ2\nuusfDZyS9ADbFdgdeKvWAxfDOAkzO4rozVE1IO/6PIfUYMysBzAOeIcoMjpwFfEf9m/Er4LZwEnu\nvixfcTYkM+sJXOHux5lZG8rwPpjZPkQDfhNgJjEIdRPK814MIn44rAImAT8GWlAm98LM/gL0ArYC\nFgODgMeBR6nh+s1sIHAecb8udfcxtR6/GJKEiIjkRzFUN4mISJ4oSYiISFpKEiIikpaShIiIpKUk\nISIiaSlJiIhIWkoSkndm9mryt72ZVV+5cGOPPbDa61ezefwazne8mV2do2NnfSZTM/uOmQ3L9nGl\ndGichBQMM+tFDJLrtwGf2SSZjiHd+5Xu3iIb8WUYz2tAP3dfspHHWe+6cnUtZjYGONfd52X72FL8\nVJKQvDOzyuTpdcB3k5lNL00WGLrRzN40s8lm9pNk/55mNs7MniCmo8DMHjOz8cnCTD9Otl0HNEuO\nN7zauTCzm5L93zazk1KO/XLKgj7DU/a/PlncZrKZ3VjDdXQEvqpKEMksrfcmcU1LJimsWjgpo+tK\nOXZN13J6coyJyXms6hrN7HfJsV83s22S7f2T651kZhUph3+KGLEssj5310OPvD6A5cnfnsDolO0/\nAa5Knjcl5vFqn+xXCeycsm/r5O9mxBQmW6Yeu4Zz/RB4Lnm+LTF1wXbJsZcCOxBz4bxOTBjXBpiW\ncpyWNVzHj4CbUl4PIxYGgpgjZ25yHRlfV02xJ8/3JObh2SR5fTdwRvJ8DXB08vyGlHNNAXaoHn9y\nfU/k+9+BHoX5aLxBGUWkYfUBOptZ/+R1S6AjMefMW+4+J2Xfn5tZ1cIq7ZL9apu4rAcxmyzu/lHy\ny/pA4kv6LXdfCGBmk4FdgDeBL83sj8DTxK/v6nYAPq627W/JOWaY2X+IL/cNua50Didmfh2flCA2\nAxYl761092eS5xOImVIBXgUeMrO/AaNSjvURMc22yHqUJKSQGXCxuz+/zsaY4O/zaq97A93c/Wsz\ne5n40qw6RqbnqvJ1yvNvgMbu/o2ZdSW+nPsDFyXPU31JfOGnSm30s+R1RtdVR4wGPOTuv6lhv5XV\n4wdw9wvM7EDgWGCCme3n7kuJe/VlLeeVMqY2CSkEVV9+lcTsnVWeAy5I1tPAzDparMBWXStgaZIg\n9gS6p7y3surz1c71CnBy0j6wDXAItZQ8kvO2dvd/AJcDe9ew21SiRJCqv4XdiBXl3t+A66pupZlt\nkjx/ETgxpb1hSzOrWiegxsRoZh3cfby7DyJKD1X77wH8O4PzSxlSSUIKQdWv7SnAGjObBDzo7rdb\nLNk6MalS+Yia1yr+B/BTM3uX+BL+Z8p79wNTzGyCx/oTDuDuj5lZd+Btog7/F0m1015pYmsJPGFm\nVSWUy2qIYxxwc7Vtc4jk0wIY4O4rkyqrTK6ruvuBd6quxcz+HzDGzBoRpYcLiXaPdF0Wb0oa1wFe\ndPcpyfPDiCo0kfWoC6xIFpnZ/wFPuvtLyfiDJ919VF2fyxeLhbwqgO+6+5o8hyMFSNVNItn1v0BV\n1VEx/ALbGfi1EoSko5KEiIikpZKEiIikpSQhIiJpKUmIiEhaShIiIpKWkoSIiKSlJCEiImn9f+of\nHp5K/5gbAAAAAElFTkSuQmCC\n", "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" }, { "name": "stdout", "output_type": "stream", "text": [ "Tensor(\"Mean_1:0\", shape=(), dtype=float32)\n", "Train Accuracy: 0.940741\n", "Test Accuracy: 0.783333\n" ] } ], "source": [ "_, _, parameters = model(X_train, Y_train, X_test, Y_test)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "**预期输出**: \n", "\n", " \n", "\n", " \n", "\n", " \n", "\n", "\n", " \n", "\n", " \n", "\n", "\n", " \n", "\n", " \n", " \n", "\n", "\n", " \n", "\n", " \n", " \n", "
\n", " **Cost after epoch 0 =**\n", " \n", " 1.917929\n", "
\n", " **Cost after epoch 5 =**\n", " \n", " 1.506757\n", "
\n", " **Train Accuracy =**\n", " \n", " 0.940741\n", "
\n", " **Test Accuracy =**\n", " \n", " 0.783333\n", "
" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "恭喜！这样我们就构建了一个针对SIGN数据集的模型，在测试集上达到了接近80%的准确率。而通过调参或者使用正则化，这个模型的准确率还可以更高。" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": true }, "outputs": [], "source": [] } ], "metadata": { "anaconda-cloud": {}, "kernelspec": { "display_name": "Python [default]", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.5.2" } }, "nbformat": 4, "nbformat_minor": 1 }