{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "oHH2oeYeiF8z"
   },
   "source": [
    "# 使用重复元素的网络（VGG）\n",
    "\n",
    "AlexNet在LeNet的基础上增加了3个卷积层。但AlexNet作者对它们的卷积窗口、输出通道数和构造顺序均做了大量的调整。虽然AlexNet指明了深度卷积神经网络可以取得出色的结果，但并没有提供简单的规则以指导后来的研究者如何设计新的网络。我们将在本章的后续几节里介绍几种不同的深度网络设计思路。\n",
    "\n",
    "本节介绍VGG，它的名字来源于论文作者所在的实验室Visual Geometry Group [1]。VGG提出了可以通过重复使用简单的基础块来构建深度模型的思路。\n",
    "\n",
    "## VGG块\n",
    "\n",
    "VGG块的组成规律是：连续使用数个相同的填充为1、窗口形状为$3\\times 3$的卷积层后接上一个步幅为2、窗口形状为$2\\times 2$的最大池化层。卷积层保持输入的高和宽不变，而池化层则对其减半。使用`vgg_block`函数直接生成所有的vgg块。\n",
    "\n",
    "\n",
    "*注：原文使用 `vgg_block` 函数来实现这个基础的VGG块，它可以指定卷积层的数量 `num_convs` 和输出通道数 `num_channels` 。但是因为 PyTorch 与 MxNet 在实现模型上存在差异，这里采用 torchvision 中的构建方式。*"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "NC30-4yHiGoI"
   },
   "outputs": [],
   "source": [
    "import d2ltorch as d2lt\n",
    "import torch\n",
    "from torch import nn, optim\n",
    "from torchsummary import summary_depth\n",
    "\n",
    "def vgg_block(in_channels, cfg):\n",
    "    blk = []\n",
    "    for v in cfg:\n",
    "        if v == 'M':\n",
    "            blk += [nn.MaxPool2d(kernel_size=2, stride=2)]\n",
    "        else:\n",
    "            blk += [nn.Conv2d(in_channels, v, kernel_size=3, padding=1), nn.ReLU(inplace=True)]\n",
    "            in_channels = v\n",
    "\n",
    "    return nn.Sequential(*blk)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "bBckK3R7iKMo"
   },
   "source": [
    "## VGG网络\n",
    "\n",
    "与AlexNet和LeNet一样，VGG网络由卷积层模块后接全连接层模块构成。卷积层模块串联数个`vgg_block`，其超参数由变量`conv_arch`定义。该变量指定了每个VGG块里卷积层个数和输出通道数。全连接模块则跟AlexNet中的一样。\n",
    "\n",
    "现在我们构造一个VGG网络。它有5个卷积块，前2块使用单卷积层，而后3块使用双卷积层。第一块的输出通道是64，之后每次对输出通道数翻倍，直到变为512。因为这个网络使用了8个卷积层和3个全连接层，所以经常被称为VGG-11。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "EagepBQZiMzX"
   },
   "outputs": [],
   "source": [
    "cfg = [64, 'M', 128, 'M', 256, 256, 'M', 512, 512, 'M', 512, 512, 'M']"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "C4Q72yjtiPPx"
   },
   "source": [
    "下面我们实现VGG-11。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "bkYJVA3FiRNw"
   },
   "outputs": [],
   "source": [
    "class MyVGG(nn.Module):\n",
    "    def __init__(self, in_channels, out_channels, cfg, num_classes=10, **kwargs):\n",
    "        super(MyVGG, self).__init__(**kwargs)\n",
    "        \n",
    "        # 卷积层部分\n",
    "        self.conv = vgg_block(in_channels, cfg)\n",
    "        \n",
    "        # torchvision中vgg的实现在进入全连接层之前有一层nn.AdaptiveAvgPool2d((7, 7))保证输出大小为7*7\n",
    "        # 这里默认输出大小为7*7\n",
    "        \n",
    "        # 全连接层部分\n",
    "        self.linear = nn.Sequential(\n",
    "            nn.Linear(out_channels*7*7, 4096), nn.ReLU(inplace=True), nn.Dropout(0.5),\n",
    "            nn.Linear(4096, 4096), nn.ReLU(inplace=True), nn.Dropout(0.5),\n",
    "            nn.Linear(4096, 10)\n",
    "        )\n",
    "        \n",
    "    def forward(self, x):\n",
    "        x = self.conv(x)\n",
    "        x = x.reshape(x.shape[0], -1)\n",
    "        x = self.linear(x)\n",
    "        return x"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "UxDsJbvNiToB"
   },
   "source": [
    "下面构造一个高和宽均为224的单通道数据样本来观察每一层的输出形状。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 728
    },
    "colab_type": "code",
    "id": "45R_friAiWdY",
    "outputId": "37967a80-4826-482c-9714-36b81ad82aaf"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "------------------------------------------------------------------------------------------\n",
      "Layer (type:depth-idx)                   Output Shape              Param #        \n",
      "==========================================================================================\n",
      "├─Sequential: 1-1                        [-1, 512, 7, 7]           --             \n",
      "|    └─Conv2d: 2-1                       [-1, 64, 224, 224]        640            \n",
      "|    └─ReLU: 2-2                         [-1, 64, 224, 224]        --             \n",
      "|    └─MaxPool2d: 2-3                    [-1, 64, 112, 112]        --             \n",
      "|    └─Conv2d: 2-4                       [-1, 128, 112, 112]       73,856         \n",
      "|    └─ReLU: 2-5                         [-1, 128, 112, 112]       --             \n",
      "|    └─MaxPool2d: 2-6                    [-1, 128, 56, 56]         --             \n",
      "|    └─Conv2d: 2-7                       [-1, 256, 56, 56]         295,168        \n",
      "|    └─ReLU: 2-8                         [-1, 256, 56, 56]         --             \n",
      "|    └─Conv2d: 2-9                       [-1, 256, 56, 56]         590,080        \n",
      "|    └─ReLU: 2-10                        [-1, 256, 56, 56]         --             \n",
      "|    └─MaxPool2d: 2-11                   [-1, 256, 28, 28]         --             \n",
      "|    └─Conv2d: 2-12                      [-1, 512, 28, 28]         1,180,160      \n",
      "|    └─ReLU: 2-13                        [-1, 512, 28, 28]         --             \n",
      "|    └─Conv2d: 2-14                      [-1, 512, 28, 28]         2,359,808      \n",
      "|    └─ReLU: 2-15                        [-1, 512, 28, 28]         --             \n",
      "|    └─MaxPool2d: 2-16                   [-1, 512, 14, 14]         --             \n",
      "|    └─Conv2d: 2-17                      [-1, 512, 14, 14]         2,359,808      \n",
      "|    └─ReLU: 2-18                        [-1, 512, 14, 14]         --             \n",
      "|    └─Conv2d: 2-19                      [-1, 512, 14, 14]         2,359,808      \n",
      "|    └─ReLU: 2-20                        [-1, 512, 14, 14]         --             \n",
      "|    └─MaxPool2d: 2-21                   [-1, 512, 7, 7]           --             \n",
      "├─Sequential: 1-2                        [-1, 10]                  --             \n",
      "|    └─Linear: 2-22                      [-1, 4096]                102,764,544    \n",
      "|    └─ReLU: 2-23                        [-1, 4096]                --             \n",
      "|    └─Dropout: 2-24                     [-1, 4096]                --             \n",
      "|    └─Linear: 2-25                      [-1, 4096]                16,781,312     \n",
      "|    └─ReLU: 2-26                        [-1, 4096]                --             \n",
      "|    └─Dropout: 2-27                     [-1, 4096]                --             \n",
      "|    └─Linear: 2-28                      [-1, 10]                  40,970         \n",
      "==========================================================================================\n",
      "Total params: 128,806,154\n",
      "Trainable params: 128,806,154\n",
      "Non-trainable params: 0\n",
      "------------------------------------------------------------------------------------------\n",
      "Input size (MB): 0.19\n",
      "Params size (MB): 491.36\n",
      "------------------------------------------------------------------------------------------\n"
     ]
    }
   ],
   "source": [
    "net = MyVGG(in_channels=1, out_channels=512, cfg=cfg)\n",
    "summary_depth(net, (1, 224, 224), device='cpu')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "W9Ayk4Q-iZp-"
   },
   "source": [
    "**注：  \n",
    "第一个vgg块: Conv2d: 2-1   → MaxPool2d: 2-3  ，输出形状：[-1, 64, 112, 112]  \n",
    "第二个vgg块: Conv2d: 2-4   → MaxPool2d: 2-6  ，输出形状：[-1, 128, 56, 56]  \n",
    "第三个vgg块: Conv2d: 2-7   → MaxPool2d: 2-11，输出形状：[-1, 256, 28, 28]  \n",
    "第四个vgg块: Conv2d: 2-12 → MaxPool2d: 2-16，输出形状：[-1, 512, 14, 14]  \n",
    "第五个vgg块: Conv2d: 2-17 → MaxPool2d: 2-21，输出形状：[-1, 512, 7, 7]**"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "j6ICXRcAidCn"
   },
   "source": [
    "可以看到，每次我们将输入的高和宽减半，直到最终高和宽变成7后传入全连接层。与此同时，输出通道数每次翻倍，直到变成512。因为每个卷积层的窗口大小一样，所以每层的模型参数尺寸和计算复杂度与输入高、输入宽、输入通道数和输出通道数的乘积成正比。VGG这种高和宽减半以及通道翻倍的设计使得多数卷积层都有相同的模型参数尺寸和计算复杂度。\n",
    "\n",
    "## 获取数据和训练模型\n",
    "\n",
    "因为VGG-11计算上比AlexNet更加复杂，出于测试的目的我们构造一个通道数更小，或者说更窄的网络在Fashion-MNIST数据集上进行训练。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "PBK_BYDVihET"
   },
   "outputs": [],
   "source": [
    "small_cfg = [16, 'M', 32, 'M', 64, 64, 'M', 64, 64, 'M', 64, 64, 'M']\n",
    "net = MyVGG(in_channels=1, out_channels=64, cfg=small_cfg)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "tYn0gnPIikFq"
   },
   "source": [
    "除了使用了稍大些的学习率，模型训练过程与上一节的AlexNet中的类似。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 121
    },
    "colab_type": "code",
    "id": "RDQ9eh7pimbE",
    "outputId": "c8389cd8-d9ea-41b9-f4b0-3cea70e680f2"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "training on cuda:0\n",
      "epoch 1, loss 0.0113, train acc 0.470, test acc 0.819, time 48.7 sec\n",
      "epoch 2, loss 0.0035, train acc 0.835, test acc 0.857, time 49.1 sec\n",
      "epoch 3, loss 0.0028, train acc 0.870, test acc 0.878, time 49.2 sec\n",
      "epoch 4, loss 0.0024, train acc 0.888, test acc 0.889, time 49.2 sec\n",
      "epoch 5, loss 0.0022, train acc 0.898, test acc 0.900, time 49.3 sec\n"
     ]
    }
   ],
   "source": [
    "root = '~/dataset/'\n",
    "lr, num_epochs, batch_size, device = 0.05, 5, 128, d2lt.try_gpu()\n",
    "\n",
    "d2lt.params_init(net, init=nn.init.xavier_uniform_)\n",
    "optimizer = optim.SGD(net.parameters(), lr=lr)\n",
    "train_iter, test_iter = d2lt.load_data_fashion_mnist(root, batch_size, resize=224)\n",
    "d2lt.train_ch5(net, train_iter, test_iter, batch_size, optimizer, device, num_epochs)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "JCWJ1ksKixwg"
   },
   "source": [
    "## 小结\n",
    "\n",
    "* VGG-11通过5个可以重复使用的卷积块来构造网络。根据每块里卷积层个数和输出通道数的不同可以定义出不同的VGG模型。\n",
    "\n",
    "## 练习\n",
    "\n",
    "* 与AlexNet相比，VGG通常计算慢很多，也需要更多的内存或显存。试分析原因。\n",
    "* 尝试将Fashion-MNIST中图像的高和宽由224改为96。这在实验中有哪些影响？\n",
    "* 参考VGG论文里的表1来构造VGG其他常用模型，如VGG-16和VGG-19 [1]。\n",
    "\n",
    "\n",
    "\n",
    "## 参考文献\n",
    "\n",
    "[1] Simonyan, K., & Zisserman, A. (2014). Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556.\n",
    "\n",
    "## 扫码直达[讨论区](https://discuss.gluon.ai/t/topic/1277)\n",
    "\n",
    "![](../img/qr_vgg.svg)"
   ]
  }
 ],
 "metadata": {
  "accelerator": "GPU",
  "colab": {
   "collapsed_sections": [],
   "name": "vgg.ipynb",
   "provenance": [],
   "version": "0.3.2"
  },
  "kernelspec": {
   "display_name": "Python [conda env:pytorch]",
   "language": "python",
   "name": "conda-env-pytorch-py"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.9"
  },
  "toc": {
   "base_numbering": 1,
   "nav_menu": {},
   "number_sections": true,
   "sideBar": true,
   "skip_h1_title": false,
   "title_cell": "Table of Contents",
   "title_sidebar": "Contents",
   "toc_cell": false,
   "toc_position": {},
   "toc_section_display": true,
   "toc_window_display": false
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}