{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Shipping SqueezeNet from PyTorch to ONNX to Android App"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "_Notebook last updated on 2018-12-31 using PyTorch 1.0 nightly with Android fixes. Code changes:_\n",
    "\n",
    "- _Exported the two Protobuf files (`squeeze_init_net_v1.pb` and `squeeze_predict_net_v1.pb`) again for verification.\n",
    "- _Modified SqueezeNet network to use `nn.AdaptiveAvgPool2d((1, 1))`._"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In this notebook we will show you how to export SqueezeNet which is implemented and trained in fastai library (**TODO**) and PyTorch to run on mobile devices.\n",
    "\n",
    "Let's get started. First, you should have [PyTorch](https://pytorch.org/) and [ONNX](https://onnx.ai/) installed in your environment and git cloned [AICamera](https://github.com/bwasti/AICamera) repo.\n",
    "\n",
    "_NOTE: Caffe2 pre-built binaries were installed together when you install PyTorch as the [Caffe2 source code now lives in the PyTorch repository](https://github.com/caffe2/caffe2)._\n",
    "\n",
    "1. [Install PyTorch 1.0 preview locally](https://pytorch.org/get-started/locally/#start-locally). Run this command:\n",
    "```sh\n",
    "conda install pytorch-nightly cuda92 -c pytorch\n",
    "```\n",
    "\n",
    "2. Install ONNX. See the instructions in this [notebook](https://nbviewer.jupyter.org/github/cedrickchee/data-science-notebooks/blob/master/notebooks/deep_learning/fastai_mobile/onnx_from_pytorch_to_caffe2.ipynb#Install-ONNX)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Import some Python packages**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import io\n",
    "import numpy as np\n",
    "import torch.onnx"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "_**Note: we are using PyTorch 1.0 preview (nightly) released on 2018-12-31.**_"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "1.0.0.dev20181231\n"
     ]
    }
   ],
   "source": [
    "print(torch.__version__)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "_**NOTE: as the work to bridge ResNet-family models build using [fastai v1 library](https://docs.fast.ai/) to pure PyTorch land continues, for now, the steps below will use an example of mobile-first CNN, SqueezeNet available from torchvision. This model was developed in plain PyTorch (not in fastai v1).**_"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Network - SqueezeNet v1.1"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Efficient Convolutional Neural Networks (CNNs) for Mobile Vision\n",
    "\n",
    "SqueezeNet is a small CNN which achieves AlexNet level accuracy on ImageNet with 50x fewer parameters. [Paper](http://arxiv.org/abs/1602.07360).\n",
    "\n",
    "**Use cases**\n",
    "\n",
    "SqueezeNet models perform image classification—they take images as input and classify the major object in the image into a set of pre-defined classes. They are trained on ImageNet dataset which contains images from 1000 classes. SqueezeNet models are highly efficient in terms of size and speed while providing good accuracies. This makes them ideal for platforms with strict constraints on size.\n",
    "\n",
    "**SqueezeNet version 1.1**\n",
    "\n",
    "SqueezeNet 1.1 presented in the [official SqueezeNet repo](https://github.com/DeepScale/SqueezeNet/tree/master/SqueezeNet_v1.1) is an improved version of SqueezeNet 1.0 from the [paper](http://arxiv.org/abs/1602.07360). \n",
    "\n",
    "SqueezeNet version 1.1 requires 2.4x less computation than version 1.0, without sacrificing accuracy. [Jun 2016]\n",
    "\n",
    "[SqueezeNet 1.1 pre-trained model weights](https://github.com/DeepScale/SqueezeNet/tree/master/SqueezeNet_v1.1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The following [SqueezeNet implementation in PyTorch](https://github.com/pytorch/vision/blob/master/torchvision/models/squeezenet.py) by Marat Dukhan and it is part of `torchvision`:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [],
   "source": [
    "import torch\n",
    "import torch.nn as nn\n",
    "import torch.nn.init as init\n",
    "import torch.utils.model_zoo as model_zoo\n",
    "\n",
    "\n",
    "__all__ = ['SqueezeNet', 'squeezenet1_0', 'squeezenet1_1']\n",
    "\n",
    "\n",
    "model_urls = {\n",
    "    'squeezenet1_0': 'https://download.pytorch.org/models/squeezenet1_0-a815701f.pth',\n",
    "    'squeezenet1_1': 'https://download.pytorch.org/models/squeezenet1_1-f364aa15.pth',\n",
    "}\n",
    "\n",
    "\n",
    "class Fire(nn.Module):\n",
    "\n",
    "    def __init__(self, inplanes, squeeze_planes,\n",
    "                 expand1x1_planes, expand3x3_planes):\n",
    "        super(Fire, self).__init__()\n",
    "        self.inplanes = inplanes\n",
    "        self.squeeze = nn.Conv2d(inplanes, squeeze_planes, kernel_size=1)\n",
    "        self.squeeze_activation = nn.ReLU(inplace=True)\n",
    "        self.expand1x1 = nn.Conv2d(squeeze_planes, expand1x1_planes,\n",
    "                                   kernel_size=1)\n",
    "        self.expand1x1_activation = nn.ReLU(inplace=True)\n",
    "        self.expand3x3 = nn.Conv2d(squeeze_planes, expand3x3_planes,\n",
    "                                   kernel_size=3, padding=1)\n",
    "        self.expand3x3_activation = nn.ReLU(inplace=True)\n",
    "\n",
    "    def forward(self, x):\n",
    "        x = self.squeeze_activation(self.squeeze(x))\n",
    "        return torch.cat([\n",
    "            self.expand1x1_activation(self.expand1x1(x)),\n",
    "            self.expand3x3_activation(self.expand3x3(x))\n",
    "        ], 1)\n",
    "\n",
    "\n",
    "class SqueezeNet(nn.Module):\n",
    "\n",
    "    def __init__(self, version=1.0, num_classes=1000):\n",
    "        super(SqueezeNet, self).__init__()\n",
    "        if version not in [1.0, 1.1]:\n",
    "            raise ValueError(\"Unsupported SqueezeNet version {version}:\"\n",
    "                             \"1.0 or 1.1 expected\".format(version=version))\n",
    "        self.num_classes = num_classes\n",
    "        if version == 1.0:\n",
    "            self.features = nn.Sequential(\n",
    "                nn.Conv2d(3, 96, kernel_size=7, stride=2),\n",
    "                nn.ReLU(inplace=True),\n",
    "                nn.MaxPool2d(kernel_size=3, stride=2, ceil_mode=False),\n",
    "                Fire(96, 16, 64, 64),\n",
    "                Fire(128, 16, 64, 64),\n",
    "                Fire(128, 32, 128, 128),\n",
    "                nn.MaxPool2d(kernel_size=3, stride=2, ceil_mode=False),\n",
    "                Fire(256, 32, 128, 128),\n",
    "                Fire(256, 48, 192, 192),\n",
    "                Fire(384, 48, 192, 192),\n",
    "                Fire(384, 64, 256, 256),\n",
    "                nn.MaxPool2d(kernel_size=3, stride=2, ceil_mode=False),\n",
    "                Fire(512, 64, 256, 256),\n",
    "            )\n",
    "        else:\n",
    "            self.features = nn.Sequential(\n",
    "                nn.Conv2d(3, 64, kernel_size=3, stride=2),\n",
    "                nn.ReLU(inplace=True),\n",
    "                nn.MaxPool2d(kernel_size=3, stride=2, ceil_mode=False),\n",
    "                Fire(64, 16, 64, 64),\n",
    "                Fire(128, 16, 64, 64),\n",
    "                nn.MaxPool2d(kernel_size=3, stride=2, ceil_mode=False),\n",
    "                Fire(128, 32, 128, 128),\n",
    "                Fire(256, 32, 128, 128),\n",
    "                nn.MaxPool2d(kernel_size=3, stride=2, ceil_mode=False),\n",
    "                Fire(256, 48, 192, 192),\n",
    "                Fire(384, 48, 192, 192),\n",
    "                Fire(384, 64, 256, 256),\n",
    "                Fire(512, 64, 256, 256),\n",
    "            )\n",
    "        # Final convolution is initialized differently form the rest\n",
    "        final_conv = nn.Conv2d(512, self.num_classes, kernel_size=1)\n",
    "        self.classifier = nn.Sequential(\n",
    "            nn.Dropout(p=0.5),\n",
    "            final_conv,\n",
    "            nn.ReLU(inplace=True),\n",
    "            nn.AdaptiveAvgPool2d((1, 1))\n",
    "        )\n",
    "\n",
    "        for m in self.modules():\n",
    "            if isinstance(m, nn.Conv2d):\n",
    "                if m is final_conv:\n",
    "                    init.normal_(m.weight, mean=0.0, std=0.01)\n",
    "                else:\n",
    "                    init.kaiming_uniform_(m.weight)\n",
    "                if m.bias is not None:\n",
    "                    init.constant_(m.bias, 0)\n",
    "\n",
    "    def forward(self, x):\n",
    "        x = self.features(x)\n",
    "        x = self.classifier(x)\n",
    "        return x.view(x.size(0), self.num_classes)\n",
    "\n",
    "\n",
    "def squeezenet1_0(pretrained=False, **kwargs):\n",
    "    r\"\"\"SqueezeNet model architecture from the `\"SqueezeNet: AlexNet-level\n",
    "    accuracy with 50x fewer parameters and <0.5MB model size\"\n",
    "    <https://arxiv.org/abs/1602.07360>`_ paper.\n",
    "    Args:\n",
    "        pretrained (bool): If True, returns a model pre-trained on ImageNet\n",
    "    \"\"\"\n",
    "    model = SqueezeNet(version=1.0, **kwargs)\n",
    "    if pretrained:\n",
    "        model.load_state_dict(model_zoo.load_url(model_urls['squeezenet1_0']))\n",
    "    return model\n",
    "\n",
    "\n",
    "def squeezenet1_1(pretrained=False, **kwargs):\n",
    "    r\"\"\"SqueezeNet 1.1 model from the `official SqueezeNet repo\n",
    "    <https://github.com/DeepScale/SqueezeNet/tree/master/SqueezeNet_v1.1>`_.\n",
    "    SqueezeNet 1.1 has 2.4x less computation and slightly fewer parameters\n",
    "    than SqueezeNet 1.0, without sacrificing accuracy.\n",
    "    Args:\n",
    "        pretrained (bool): If True, returns a model pre-trained on ImageNet\n",
    "    \"\"\"\n",
    "    model = SqueezeNet(version=1.1, **kwargs)\n",
    "    if pretrained:\n",
    "        model.load_state_dict(model_zoo.load_url(model_urls['squeezenet1_1']))\n",
    "    return model"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Model"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can get the PyTorch model by calling the following function:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/home/ubuntu/anaconda3/envs/caffe2/lib/python3.6/site-packages/ipykernel_launcher.py:94: UserWarning: nn.init.kaiming_uniform is now deprecated in favor of nn.init.kaiming_uniform_.\n",
      "/home/ubuntu/anaconda3/envs/caffe2/lib/python3.6/site-packages/ipykernel_launcher.py:92: UserWarning: nn.init.normal is now deprecated in favor of nn.init.normal_.\n",
      "Downloading: \"https://download.pytorch.org/models/squeezenet1_1-f364aa15.pth\" to /home/ubuntu/.torch/models/squeezenet1_1-f364aa15.pth\n",
      "100%|██████████| 4966400/4966400 [00:01<00:00, 3173216.83it/s]\n"
     ]
    }
   ],
   "source": [
    "# Get pre-trained SqueezeNet model\n",
    "torch_model = squeezenet1_1(True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## ONNX"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "_Note: we are using ONNX 1.3.0 here._"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "1.3.0\r\n"
     ]
    }
   ],
   "source": [
    "!cat ~/development/resources/onnx/VERSION_NUMBER"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Export the PyTorch model as ONNX model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "from torch.autograd import Variable\n",
    "batch_size = 1 # just a random number"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Input to the model:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "x = Variable(torch.randn(batch_size, 3, 224, 224), requires_grad=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Export the model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "collapsed": true
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/home/ubuntu/anaconda3/envs/caffe2/lib/python3.6/site-packages/torch/onnx/symbolic.py:131: UserWarning: ONNX export failed on max_pool2d_with_indices because ceil_mode not supported\n",
      "  warnings.warn(\"ONNX export failed on \" + op + \" because \" + msg + \" not supported\")\n"
     ]
    },
    {
     "ename": "RuntimeError",
     "evalue": "ONNX export failed: Couldn't export operator aten::max_pool2d_with_indices\n\nDefined at:\n/home/ubuntu/anaconda3/envs/caffe2/lib/python3.6/site-packages/torch/nn/functional.py(416): max_pool2d_with_indices\n/home/ubuntu/anaconda3/envs/caffe2/lib/python3.6/site-packages/torch/nn/functional.py(424): _max_pool2d\n/home/ubuntu/anaconda3/envs/caffe2/lib/python3.6/site-packages/torch/_jit_internal.py(129): fn\n/home/ubuntu/anaconda3/envs/caffe2/lib/python3.6/site-packages/torch/nn/modules/pooling.py(148): forward\n/home/ubuntu/anaconda3/envs/caffe2/lib/python3.6/site-packages/torch/nn/modules/module.py(477): _slow_forward\n/home/ubuntu/anaconda3/envs/caffe2/lib/python3.6/site-packages/torch/nn/modules/module.py(487): __call__\n/home/ubuntu/anaconda3/envs/caffe2/lib/python3.6/site-packages/torch/nn/modules/container.py(92): forward\n/home/ubuntu/anaconda3/envs/caffe2/lib/python3.6/site-packages/torch/nn/modules/module.py(477): _slow_forward\n/home/ubuntu/anaconda3/envs/caffe2/lib/python3.6/site-packages/torch/nn/modules/module.py(487): __call__\n<ipython-input-3-f5e9e8cddb5f>(98): forward\n/home/ubuntu/anaconda3/envs/caffe2/lib/python3.6/site-packages/torch/nn/modules/module.py(477): _slow_forward\n/home/ubuntu/anaconda3/envs/caffe2/lib/python3.6/site-packages/torch/nn/modules/module.py(487): __call__\n/home/ubuntu/anaconda3/envs/caffe2/lib/python3.6/site-packages/torch/jit/__init__.py(253): forward\n/home/ubuntu/anaconda3/envs/caffe2/lib/python3.6/site-packages/torch/nn/modules/module.py(489): __call__\n/home/ubuntu/anaconda3/envs/caffe2/lib/python3.6/site-packages/torch/jit/__init__.py(198): get_trace_graph\n/home/ubuntu/anaconda3/envs/caffe2/lib/python3.6/site-packages/torch/onnx/utils.py(192): _trace_and_get_graph_from_model\n/home/ubuntu/anaconda3/envs/caffe2/lib/python3.6/site-packages/torch/onnx/utils.py(224): _model_to_graph\n/home/ubuntu/anaconda3/envs/caffe2/lib/python3.6/site-packages/torch/onnx/utils.py(281): _export\n/home/ubuntu/anaconda3/envs/caffe2/lib/python3.6/site-packages/torch/onnx/__init__.py(22): _export\n<ipython-input-7-8e5c012efbf1>(4): <module>\n/home/ubuntu/anaconda3/envs/caffe2/lib/python3.6/site-packages/IPython/core/interactiveshell.py(3265): run_code\n/home/ubuntu/anaconda3/envs/caffe2/lib/python3.6/site-packages/IPython/core/interactiveshell.py(3183): run_ast_nodes\n/home/ubuntu/anaconda3/envs/caffe2/lib/python3.6/site-packages/IPython/core/interactiveshell.py(3018): run_cell_async\n/home/ubuntu/anaconda3/envs/caffe2/lib/python3.6/site-packages/IPython/core/async_helpers.py(67): _pseudo_sync_runner\n/home/ubuntu/anaconda3/envs/caffe2/lib/python3.6/site-packages/IPython/core/interactiveshell.py(2843): _run_cell\n/home/ubuntu/anaconda3/envs/caffe2/lib/python3.6/site-packages/IPython/core/interactiveshell.py(2817): run_cell\n/home/ubuntu/anaconda3/envs/caffe2/lib/python3.6/site-packages/ipykernel/zmqshell.py(536): run_cell\n/home/ubuntu/anaconda3/envs/caffe2/lib/python3.6/site-packages/ipykernel/ipkernel.py(294): do_execute\n/home/ubuntu/anaconda3/envs/caffe2/lib/python3.6/site-packages/tornado/gen.py(326): wrapper\n/home/ubuntu/anaconda3/envs/caffe2/lib/python3.6/site-packages/ipykernel/kernelbase.py(534): execute_request\n/home/ubuntu/anaconda3/envs/caffe2/lib/python3.6/site-packages/tornado/gen.py(326): wrapper\n/home/ubuntu/anaconda3/envs/caffe2/lib/python3.6/site-packages/ipykernel/kernelbase.py(267): dispatch_shell\n/home/ubuntu/anaconda3/envs/caffe2/lib/python3.6/site-packages/tornado/gen.py(326): wrapper\n/home/ubuntu/anaconda3/envs/caffe2/lib/python3.6/site-packages/ipykernel/kernelbase.py(357): process_one\n/home/ubuntu/anaconda3/envs/caffe2/lib/python3.6/site-packages/tornado/gen.py(1147): run\n/home/ubuntu/anaconda3/envs/caffe2/lib/python3.6/site-packages/tornado/gen.py(1233): inner\n/home/ubuntu/anaconda3/envs/caffe2/lib/python3.6/site-packages/tornado/stack_context.py(300): null_wrapper\n/home/ubuntu/anaconda3/envs/caffe2/lib/python3.6/site-packages/tornado/ioloop.py(758): _run_callback\n/home/ubuntu/anaconda3/envs/caffe2/lib/python3.6/asyncio/events.py(145): _run\n/home/ubuntu/anaconda3/envs/caffe2/lib/python3.6/asyncio/base_events.py(1440): _run_once\n/home/ubuntu/anaconda3/envs/caffe2/lib/python3.6/asyncio/base_events.py(427): run_forever\n/home/ubuntu/anaconda3/envs/caffe2/lib/python3.6/site-packages/tornado/platform/asyncio.py(132): start\n/home/ubuntu/anaconda3/envs/caffe2/lib/python3.6/site-packages/ipykernel/kernelapp.py(505): start\n/home/ubuntu/anaconda3/envs/caffe2/lib/python3.6/site-packages/traitlets/config/application.py(658): launch_instance\n/home/ubuntu/anaconda3/envs/caffe2/lib/python3.6/site-packages/ipykernel_launcher.py(16): <module>\n/home/ubuntu/anaconda3/envs/caffe2/lib/python3.6/runpy.py(85): _run_code\n/home/ubuntu/anaconda3/envs/caffe2/lib/python3.6/runpy.py(193): _run_module_as_main\n\n\nGraph we tried to export:\ngraph(%0 : Float(1, 3, 224, 224)\n      %1 : Float(64, 3, 3, 3)\n      %2 : Float(64)\n      %3 : Float(16, 64, 1, 1)\n      %4 : Float(16)\n      %5 : Float(64, 16, 1, 1)\n      %6 : Float(64)\n      %7 : Float(64, 16, 3, 3)\n      %8 : Float(64)\n      %9 : Float(16, 128, 1, 1)\n      %10 : Float(16)\n      %11 : Float(64, 16, 1, 1)\n      %12 : Float(64)\n      %13 : Float(64, 16, 3, 3)\n      %14 : Float(64)\n      %15 : Float(32, 128, 1, 1)\n      %16 : Float(32)\n      %17 : Float(128, 32, 1, 1)\n      %18 : Float(128)\n      %19 : Float(128, 32, 3, 3)\n      %20 : Float(128)\n      %21 : Float(32, 256, 1, 1)\n      %22 : Float(32)\n      %23 : Float(128, 32, 1, 1)\n      %24 : Float(128)\n      %25 : Float(128, 32, 3, 3)\n      %26 : Float(128)\n      %27 : Float(48, 256, 1, 1)\n      %28 : Float(48)\n      %29 : Float(192, 48, 1, 1)\n      %30 : Float(192)\n      %31 : Float(192, 48, 3, 3)\n      %32 : Float(192)\n      %33 : Float(48, 384, 1, 1)\n      %34 : Float(48)\n      %35 : Float(192, 48, 1, 1)\n      %36 : Float(192)\n      %37 : Float(192, 48, 3, 3)\n      %38 : Float(192)\n      %39 : Float(64, 384, 1, 1)\n      %40 : Float(64)\n      %41 : Float(256, 64, 1, 1)\n      %42 : Float(256)\n      %43 : Float(256, 64, 3, 3)\n      %44 : Float(256)\n      %45 : Float(64, 512, 1, 1)\n      %46 : Float(64)\n      %47 : Float(256, 64, 1, 1)\n      %48 : Float(256)\n      %49 : Float(256, 64, 3, 3)\n      %50 : Float(256)\n      %51 : Float(1000, 512, 1, 1)\n      %52 : Float(1000)) {\n  %53 : Float(1, 64, 111, 111) = onnx::Conv[dilations=[1, 1], group=1, kernel_shape=[3, 3], pads=[0, 0, 0, 0], strides=[2, 2]](%0, %1, %2), scope: SqueezeNet/Sequential[features]/Conv2d[0]\n  %54 : Float(1, 64, 111, 111) = onnx::Relu(%53), scope: SqueezeNet/Sequential[features]/ReLU[1]\n  %55 : int[] = onnx::Constant[value= 3  3 [ CPULongType{2} ]]()\n  %56 : int[] = onnx::Constant[value= 2  2 [ CPULongType{2} ]]()\n  %57 : int[] = onnx::Constant[value= 0  0 [ CPULongType{2} ]]()\n  %58 : int[] = onnx::Constant[value= 1  1 [ CPULongType{2} ]]()\n  %59 : Long() = onnx::Constant[value={1}](), scope: SqueezeNet/Sequential[features]/MaxPool2d[2]\n  %60 : Float(1, 64, 55, 55), %61 : Long(1, 64, 55, 55) = aten::max_pool2d_with_indices(%54, %55, %56, %57, %58, %59), scope: SqueezeNet/Sequential[features]/MaxPool2d[2]\n  %62 : Float(1, 16, 55, 55) = onnx::Conv[dilations=[1, 1], group=1, kernel_shape=[1, 1], pads=[0, 0, 0, 0], strides=[1, 1]](%60, %3, %4), scope: SqueezeNet/Sequential[features]/Fire[3]/Conv2d[squeeze]\n  %63 : Float(1, 16, 55, 55) = onnx::Relu(%62), scope: SqueezeNet/Sequential[features]/Fire[3]/ReLU[squeeze_activation]\n  %64 : Float(1, 64, 55, 55) = onnx::Conv[dilations=[1, 1], group=1, kernel_shape=[1, 1], pads=[0, 0, 0, 0], strides=[1, 1]](%63, %5, %6), scope: SqueezeNet/Sequential[features]/Fire[3]/Conv2d[expand1x1]\n  %65 : Float(1, 64, 55, 55) = onnx::Relu(%64), scope: SqueezeNet/Sequential[features]/Fire[3]/ReLU[expand1x1_activation]\n  %66 : Float(1, 64, 55, 55) = onnx::Conv[dilations=[1, 1], group=1, kernel_shape=[3, 3], pads=[1, 1, 1, 1], strides=[1, 1]](%63, %7, %8), scope: SqueezeNet/Sequential[features]/Fire[3]/Conv2d[expand3x3]\n  %67 : Float(1, 64, 55, 55) = onnx::Relu(%66), scope: SqueezeNet/Sequential[features]/Fire[3]/ReLU[expand3x3_activation]\n  %68 : Float(1, 128, 55, 55) = onnx::Concat[axis=1](%65, %67), scope: SqueezeNet/Sequential[features]/Fire[3]\n  %69 : Float(1, 16, 55, 55) = onnx::Conv[dilations=[1, 1], group=1, kernel_shape=[1, 1], pads=[0, 0, 0, 0], strides=[1, 1]](%68, %9, %10), scope: SqueezeNet/Sequential[features]/Fire[4]/Conv2d[squeeze]\n  %70 : Float(1, 16, 55, 55) = onnx::Relu(%69), scope: SqueezeNet/Sequential[features]/Fire[4]/ReLU[squeeze_activation]\n  %71 : Float(1, 64, 55, 55) = onnx::Conv[dilations=[1, 1], group=1, kernel_shape=[1, 1], pads=[0, 0, 0, 0], strides=[1, 1]](%70, %11, %12), scope: SqueezeNet/Sequential[features]/Fire[4]/Conv2d[expand1x1]\n  %72 : Float(1, 64, 55, 55) = onnx::Relu(%71), scope: SqueezeNet/Sequential[features]/Fire[4]/ReLU[expand1x1_activation]\n  %73 : Float(1, 64, 55, 55) = onnx::Conv[dilations=[1, 1], group=1, kernel_shape=[3, 3], pads=[1, 1, 1, 1], strides=[1, 1]](%70, %13, %14), scope: SqueezeNet/Sequential[features]/Fire[4]/Conv2d[expand3x3]\n  %74 : Float(1, 64, 55, 55) = onnx::Relu(%73), scope: SqueezeNet/Sequential[features]/Fire[4]/ReLU[expand3x3_activation]\n  %75 : Float(1, 128, 55, 55) = onnx::Concat[axis=1](%72, %74), scope: SqueezeNet/Sequential[features]/Fire[4]\n  %76 : int[] = onnx::Constant[value= 3  3 [ CPULongType{2} ]]()\n  %77 : int[] = onnx::Constant[value= 2  2 [ CPULongType{2} ]]()\n  %78 : int[] = onnx::Constant[value= 0  0 [ CPULongType{2} ]]()\n  %79 : int[] = onnx::Constant[value= 1  1 [ CPULongType{2} ]]()\n  %80 : Long() = onnx::Constant[value={1}](), scope: SqueezeNet/Sequential[features]/MaxPool2d[5]\n  %81 : Float(1, 128, 27, 27), %82 : Long(1, 128, 27, 27) = aten::max_pool2d_with_indices(%75, %76, %77, %78, %79, %80), scope: SqueezeNet/Sequential[features]/MaxPool2d[5]\n  %83 : Float(1, 32, 27, 27) = onnx::Conv[dilations=[1, 1], group=1, kernel_shape=[1, 1], pads=[0, 0, 0, 0], strides=[1, 1]](%81, %15, %16), scope: SqueezeNet/Sequential[features]/Fire[6]/Conv2d[squeeze]\n  %84 : Float(1, 32, 27, 27) = onnx::Relu(%83), scope: SqueezeNet/Sequential[features]/Fire[6]/ReLU[squeeze_activation]\n  %85 : Float(1, 128, 27, 27) = onnx::Conv[dilations=[1, 1], group=1, kernel_shape=[1, 1], pads=[0, 0, 0, 0], strides=[1, 1]](%84, %17, %18), scope: SqueezeNet/Sequential[features]/Fire[6]/Conv2d[expand1x1]\n  %86 : Float(1, 128, 27, 27) = onnx::Relu(%85), scope: SqueezeNet/Sequential[features]/Fire[6]/ReLU[expand1x1_activation]\n  %87 : Float(1, 128, 27, 27) = onnx::Conv[dilations=[1, 1], group=1, kernel_shape=[3, 3], pads=[1, 1, 1, 1], strides=[1, 1]](%84, %19, %20), scope: SqueezeNet/Sequential[features]/Fire[6]/Conv2d[expand3x3]\n  %88 : Float(1, 128, 27, 27) = onnx::Relu(%87), scope: SqueezeNet/Sequential[features]/Fire[6]/ReLU[expand3x3_activation]\n  %89 : Float(1, 256, 27, 27) = onnx::Concat[axis=1](%86, %88), scope: SqueezeNet/Sequential[features]/Fire[6]\n  %90 : Float(1, 32, 27, 27) = onnx::Conv[dilations=[1, 1], group=1, kernel_shape=[1, 1], pads=[0, 0, 0, 0], strides=[1, 1]](%89, %21, %22), scope: SqueezeNet/Sequential[features]/Fire[7]/Conv2d[squeeze]\n  %91 : Float(1, 32, 27, 27) = onnx::Relu(%90), scope: SqueezeNet/Sequential[features]/Fire[7]/ReLU[squeeze_activation]\n  %92 : Float(1, 128, 27, 27) = onnx::Conv[dilations=[1, 1], group=1, kernel_shape=[1, 1], pads=[0, 0, 0, 0], strides=[1, 1]](%91, %23, %24), scope: SqueezeNet/Sequential[features]/Fire[7]/Conv2d[expand1x1]\n  %93 : Float(1, 128, 27, 27) = onnx::Relu(%92), scope: SqueezeNet/Sequential[features]/Fire[7]/ReLU[expand1x1_activation]\n  %94 : Float(1, 128, 27, 27) = onnx::Conv[dilations=[1, 1], group=1, kernel_shape=[3, 3], pads=[1, 1, 1, 1], strides=[1, 1]](%91, %25, %26), scope: SqueezeNet/Sequential[features]/Fire[7]/Conv2d[expand3x3]\n  %95 : Float(1, 128, 27, 27) = onnx::Relu(%94), scope: SqueezeNet/Sequential[features]/Fire[7]/ReLU[expand3x3_activation]\n  %96 : Float(1, 256, 27, 27) = onnx::Concat[axis=1](%93, %95), scope: SqueezeNet/Sequential[features]/Fire[7]\n  %97 : int[] = onnx::Constant[value= 3  3 [ CPULongType{2} ]]()\n  %98 : int[] = onnx::Constant[value= 2  2 [ CPULongType{2} ]]()\n  %99 : int[] = onnx::Constant[value= 0  0 [ CPULongType{2} ]]()\n  %100 : int[] = onnx::Constant[value= 1  1 [ CPULongType{2} ]]()\n  %101 : Long() = onnx::Constant[value={1}](), scope: SqueezeNet/Sequential[features]/MaxPool2d[8]\n  %102 : Float(1, 256, 13, 13), %103 : Long(1, 256, 13, 13) = aten::max_pool2d_with_indices(%96, %97, %98, %99, %100, %101), scope: SqueezeNet/Sequential[features]/MaxPool2d[8]\n  %104 : Float(1, 48, 13, 13) = onnx::Conv[dilations=[1, 1], group=1, kernel_shape=[1, 1], pads=[0, 0, 0, 0], strides=[1, 1]](%102, %27, %28), scope: SqueezeNet/Sequential[features]/Fire[9]/Conv2d[squeeze]\n  %105 : Float(1, 48, 13, 13) = onnx::Relu(%104), scope: SqueezeNet/Sequential[features]/Fire[9]/ReLU[squeeze_activation]\n  %106 : Float(1, 192, 13, 13) = onnx::Conv[dilations=[1, 1], group=1, kernel_shape=[1, 1], pads=[0, 0, 0, 0], strides=[1, 1]](%105, %29, %30), scope: SqueezeNet/Sequential[features]/Fire[9]/Conv2d[expand1x1]\n  %107 : Float(1, 192, 13, 13) = onnx::Relu(%106), scope: SqueezeNet/Sequential[features]/Fire[9]/ReLU[expand1x1_activation]\n  %108 : Float(1, 192, 13, 13) = onnx::Conv[dilations=[1, 1], group=1, kernel_shape=[3, 3], pads=[1, 1, 1, 1], strides=[1, 1]](%105, %31, %32), scope: SqueezeNet/Sequential[features]/Fire[9]/Conv2d[expand3x3]\n  %109 : Float(1, 192, 13, 13) = onnx::Relu(%108), scope: SqueezeNet/Sequential[features]/Fire[9]/ReLU[expand3x3_activation]\n  %110 : Float(1, 384, 13, 13) = onnx::Concat[axis=1](%107, %109), scope: SqueezeNet/Sequential[features]/Fire[9]\n  %111 : Float(1, 48, 13, 13) = onnx::Conv[dilations=[1, 1], group=1, kernel_shape=[1, 1], pads=[0, 0, 0, 0], strides=[1, 1]](%110, %33, %34), scope: SqueezeNet/Sequential[features]/Fire[10]/Conv2d[squeeze]\n  %112 : Float(1, 48, 13, 13) = onnx::Relu(%111), scope: SqueezeNet/Sequential[features]/Fire[10]/ReLU[squeeze_activation]\n  %113 : Float(1, 192, 13, 13) = onnx::Conv[dilations=[1, 1], group=1, kernel_shape=[1, 1], pads=[0, 0, 0, 0], strides=[1, 1]](%112, %35, %36), scope: SqueezeNet/Sequential[features]/Fire[10]/Conv2d[expand1x1]\n  %114 : Float(1, 192, 13, 13) = onnx::Relu(%113), scope: SqueezeNet/Sequential[features]/Fire[10]/ReLU[expand1x1_activation]\n  %115 : Float(1, 192, 13, 13) = onnx::Conv[dilations=[1, 1], group=1, kernel_shape=[3, 3], pads=[1, 1, 1, 1], strides=[1, 1]](%112, %37, %38), scope: SqueezeNet/Sequential[features]/Fire[10]/Conv2d[expand3x3]\n  %116 : Float(1, 192, 13, 13) = onnx::Relu(%115), scope: SqueezeNet/Sequential[features]/Fire[10]/ReLU[expand3x3_activation]\n  %117 : Float(1, 384, 13, 13) = onnx::Concat[axis=1](%114, %116), scope: SqueezeNet/Sequential[features]/Fire[10]\n  %118 : Float(1, 64, 13, 13) = onnx::Conv[dilations=[1, 1], group=1, kernel_shape=[1, 1], pads=[0, 0, 0, 0], strides=[1, 1]](%117, %39, %40), scope: SqueezeNet/Sequential[features]/Fire[11]/Conv2d[squeeze]\n  %119 : Float(1, 64, 13, 13) = onnx::Relu(%118), scope: SqueezeNet/Sequential[features]/Fire[11]/ReLU[squeeze_activation]\n  %120 : Float(1, 256, 13, 13) = onnx::Conv[dilations=[1, 1], group=1, kernel_shape=[1, 1], pads=[0, 0, 0, 0], strides=[1, 1]](%119, %41, %42), scope: SqueezeNet/Sequential[features]/Fire[11]/Conv2d[expand1x1]\n  %121 : Float(1, 256, 13, 13) = onnx::Relu(%120), scope: SqueezeNet/Sequential[features]/Fire[11]/ReLU[expand1x1_activation]\n  %122 : Float(1, 256, 13, 13) = onnx::Conv[dilations=[1, 1], group=1, kernel_shape=[3, 3], pads=[1, 1, 1, 1], strides=[1, 1]](%119, %43, %44), scope: SqueezeNet/Sequential[features]/Fire[11]/Conv2d[expand3x3]\n  %123 : Float(1, 256, 13, 13) = onnx::Relu(%122), scope: SqueezeNet/Sequential[features]/Fire[11]/ReLU[expand3x3_activation]\n  %124 : Float(1, 512, 13, 13) = onnx::Concat[axis=1](%121, %123), scope: SqueezeNet/Sequential[features]/Fire[11]\n  %125 : Float(1, 64, 13, 13) = onnx::Conv[dilations=[1, 1], group=1, kernel_shape=[1, 1], pads=[0, 0, 0, 0], strides=[1, 1]](%124, %45, %46), scope: SqueezeNet/Sequential[features]/Fire[12]/Conv2d[squeeze]\n  %126 : Float(1, 64, 13, 13) = onnx::Relu(%125), scope: SqueezeNet/Sequential[features]/Fire[12]/ReLU[squeeze_activation]\n  %127 : Float(1, 256, 13, 13) = onnx::Conv[dilations=[1, 1], group=1, kernel_shape=[1, 1], pads=[0, 0, 0, 0], strides=[1, 1]](%126, %47, %48), scope: SqueezeNet/Sequential[features]/Fire[12]/Conv2d[expand1x1]\n  %128 : Float(1, 256, 13, 13) = onnx::Relu(%127), scope: SqueezeNet/Sequential[features]/Fire[12]/ReLU[expand1x1_activation]\n  %129 : Float(1, 256, 13, 13) = onnx::Conv[dilations=[1, 1], group=1, kernel_shape=[3, 3], pads=[1, 1, 1, 1], strides=[1, 1]](%126, %49, %50), scope: SqueezeNet/Sequential[features]/Fire[12]/Conv2d[expand3x3]\n  %130 : Float(1, 256, 13, 13) = onnx::Relu(%129), scope: SqueezeNet/Sequential[features]/Fire[12]/ReLU[expand3x3_activation]\n  %131 : Float(1, 512, 13, 13) = onnx::Concat[axis=1](%128, %130), scope: SqueezeNet/Sequential[features]/Fire[12]\n  %132 : Float(1, 512, 13, 13), %133 : Tensor = onnx::Dropout[ratio=0.5](%131), scope: SqueezeNet/Sequential[classifier]/Dropout[0]\n  %134 : Float(1, 1000, 13, 13) = onnx::Conv[dilations=[1, 1], group=1, kernel_shape=[1, 1], pads=[0, 0, 0, 0], strides=[1, 1]](%132, %51, %52), scope: SqueezeNet/Sequential[classifier]/Conv2d[1]\n  %135 : Float(1, 1000, 13, 13) = onnx::Relu(%134), scope: SqueezeNet/Sequential[classifier]/ReLU[2]\n  %136 : Float(1, 1000, 1, 1) = onnx::GlobalAveragePool(%135), scope: SqueezeNet/Sequential[classifier]/AdaptiveAvgPool2d[3]\n  %137 : Long() = onnx::Constant[value={0}](), scope: SqueezeNet\n  %138 : Tensor = onnx::Shape(%136), scope: SqueezeNet\n  %139 : Long() = onnx::Gather[axis=0](%138, %137), scope: SqueezeNet\n  %140 : Long() = onnx::Constant[value={1000}](), scope: SqueezeNet\n  %141 : Tensor = onnx::Unsqueeze[axes=[0]](%139)\n  %142 : Tensor = onnx::Unsqueeze[axes=[0]](%140)\n  %143 : Tensor = onnx::Concat[axis=0](%141, %142)\n  %144 : Float(1, 1000) = onnx::Reshape(%136, %143), scope: SqueezeNet\n  return (%144);\n}\n",
     "output_type": "error",
     "traceback": [
      "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
      "\u001b[0;31mRuntimeError\u001b[0m                              Traceback (most recent call last)",
      "\u001b[0;32m<ipython-input-7-8e5c012efbf1>\u001b[0m in \u001b[0;36m<module>\u001b[0;34m\u001b[0m\n\u001b[1;32m      2\u001b[0m                                \u001b[0mx\u001b[0m\u001b[0;34m,\u001b[0m                       \u001b[0;31m# model input (or a tuple for multiple inputs)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m      3\u001b[0m                                \u001b[0;34m\"squeezenet-v1.onnx\"\u001b[0m\u001b[0;34m,\u001b[0m       \u001b[0;31m# where to save the model (can be a file or file-like object)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m----> 4\u001b[0;31m                                export_params=True)      # store the trained parameter weights inside the model file\n\u001b[0m",
      "\u001b[0;32m~/anaconda3/envs/caffe2/lib/python3.6/site-packages/torch/onnx/__init__.py\u001b[0m in \u001b[0;36m_export\u001b[0;34m(*args, **kwargs)\u001b[0m\n\u001b[1;32m     20\u001b[0m \u001b[0;32mdef\u001b[0m \u001b[0m_export\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m*\u001b[0m\u001b[0margs\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m**\u001b[0m\u001b[0mkwargs\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m     21\u001b[0m     \u001b[0;32mfrom\u001b[0m \u001b[0mtorch\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0monnx\u001b[0m \u001b[0;32mimport\u001b[0m \u001b[0mutils\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m---> 22\u001b[0;31m     \u001b[0;32mreturn\u001b[0m \u001b[0mutils\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_export\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m*\u001b[0m\u001b[0margs\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m**\u001b[0m\u001b[0mkwargs\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m     23\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m     24\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;32m~/anaconda3/envs/caffe2/lib/python3.6/site-packages/torch/onnx/utils.py\u001b[0m in \u001b[0;36m_export\u001b[0;34m(model, args, f, export_params, verbose, training, input_names, output_names, operator_export_type, export_type, example_outputs, propagate)\u001b[0m\n\u001b[1;32m    285\u001b[0m     \u001b[0mdefer_weight_export\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mexport_type\u001b[0m \u001b[0;32mis\u001b[0m \u001b[0;32mnot\u001b[0m \u001b[0mExportTypes\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mPROTOBUF_FILE\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    286\u001b[0m     \u001b[0;32mif\u001b[0m \u001b[0mexport_params\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 287\u001b[0;31m         \u001b[0mproto\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mexport_map\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mgraph\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_export_onnx\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mparams\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0m_onnx_opset_version\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mdefer_weight_export\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0moperator_export_type\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    288\u001b[0m     \u001b[0;32melse\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    289\u001b[0m         \u001b[0mproto\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mexport_map\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mgraph\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_export_onnx\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0m_onnx_opset_version\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;32mFalse\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0moperator_export_type\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;31mRuntimeError\u001b[0m: ONNX export failed: Couldn't export operator aten::max_pool2d_with_indices\n\nDefined at:\n/home/ubuntu/anaconda3/envs/caffe2/lib/python3.6/site-packages/torch/nn/functional.py(416): max_pool2d_with_indices\n/home/ubuntu/anaconda3/envs/caffe2/lib/python3.6/site-packages/torch/nn/functional.py(424): _max_pool2d\n/home/ubuntu/anaconda3/envs/caffe2/lib/python3.6/site-packages/torch/_jit_internal.py(129): fn\n/home/ubuntu/anaconda3/envs/caffe2/lib/python3.6/site-packages/torch/nn/modules/pooling.py(148): forward\n/home/ubuntu/anaconda3/envs/caffe2/lib/python3.6/site-packages/torch/nn/modules/module.py(477): _slow_forward\n/home/ubuntu/anaconda3/envs/caffe2/lib/python3.6/site-packages/torch/nn/modules/module.py(487): __call__\n/home/ubuntu/anaconda3/envs/caffe2/lib/python3.6/site-packages/torch/nn/modules/container.py(92): forward\n/home/ubuntu/anaconda3/envs/caffe2/lib/python3.6/site-packages/torch/nn/modules/module.py(477): _slow_forward\n/home/ubuntu/anaconda3/envs/caffe2/lib/python3.6/site-packages/torch/nn/modules/module.py(487): __call__\n<ipython-input-3-f5e9e8cddb5f>(98): forward\n/home/ubuntu/anaconda3/envs/caffe2/lib/python3.6/site-packages/torch/nn/modules/module.py(477): _slow_forward\n/home/ubuntu/anaconda3/envs/caffe2/lib/python3.6/site-packages/torch/nn/modules/module.py(487): __call__\n/home/ubuntu/anaconda3/envs/caffe2/lib/python3.6/site-packages/torch/jit/__init__.py(253): forward\n/home/ubuntu/anaconda3/envs/caffe2/lib/python3.6/site-packages/torch/nn/modules/module.py(489): __call__\n/home/ubuntu/anaconda3/envs/caffe2/lib/python3.6/site-packages/torch/jit/__init__.py(198): get_trace_graph\n/home/ubuntu/anaconda3/envs/caffe2/lib/python3.6/site-packages/torch/onnx/utils.py(192): _trace_and_get_graph_from_model\n/home/ubuntu/anaconda3/envs/caffe2/lib/python3.6/site-packages/torch/onnx/utils.py(224): _model_to_graph\n/home/ubuntu/anaconda3/envs/caffe2/lib/python3.6/site-packages/torch/onnx/utils.py(281): _export\n/home/ubuntu/anaconda3/envs/caffe2/lib/python3.6/site-packages/torch/onnx/__init__.py(22): _export\n<ipython-input-7-8e5c012efbf1>(4): <module>\n/home/ubuntu/anaconda3/envs/caffe2/lib/python3.6/site-packages/IPython/core/interactiveshell.py(3265): run_code\n/home/ubuntu/anaconda3/envs/caffe2/lib/python3.6/site-packages/IPython/core/interactiveshell.py(3183): run_ast_nodes\n/home/ubuntu/anaconda3/envs/caffe2/lib/python3.6/site-packages/IPython/core/interactiveshell.py(3018): run_cell_async\n/home/ubuntu/anaconda3/envs/caffe2/lib/python3.6/site-packages/IPython/core/async_helpers.py(67): _pseudo_sync_runner\n/home/ubuntu/anaconda3/envs/caffe2/lib/python3.6/site-packages/IPython/core/interactiveshell.py(2843): _run_cell\n/home/ubuntu/anaconda3/envs/caffe2/lib/python3.6/site-packages/IPython/core/interactiveshell.py(2817): run_cell\n/home/ubuntu/anaconda3/envs/caffe2/lib/python3.6/site-packages/ipykernel/zmqshell.py(536): run_cell\n/home/ubuntu/anaconda3/envs/caffe2/lib/python3.6/site-packages/ipykernel/ipkernel.py(294): do_execute\n/home/ubuntu/anaconda3/envs/caffe2/lib/python3.6/site-packages/tornado/gen.py(326): wrapper\n/home/ubuntu/anaconda3/envs/caffe2/lib/python3.6/site-packages/ipykernel/kernelbase.py(534): execute_request\n/home/ubuntu/anaconda3/envs/caffe2/lib/python3.6/site-packages/tornado/gen.py(326): wrapper\n/home/ubuntu/anaconda3/envs/caffe2/lib/python3.6/site-packages/ipykernel/kernelbase.py(267): dispatch_shell\n/home/ubuntu/anaconda3/envs/caffe2/lib/python3.6/site-packages/tornado/gen.py(326): wrapper\n/home/ubuntu/anaconda3/envs/caffe2/lib/python3.6/site-packages/ipykernel/kernelbase.py(357): process_one\n/home/ubuntu/anaconda3/envs/caffe2/lib/python3.6/site-packages/tornado/gen.py(1147): run\n/home/ubuntu/anaconda3/envs/caffe2/lib/python3.6/site-packages/tornado/gen.py(1233): inner\n/home/ubuntu/anaconda3/envs/caffe2/lib/python3.6/site-packages/tornado/stack_context.py(300): null_wrapper\n/home/ubuntu/anaconda3/envs/caffe2/lib/python3.6/site-packages/tornado/ioloop.py(758): _run_callback\n/home/ubuntu/anaconda3/envs/caffe2/lib/python3.6/asyncio/events.py(145): _run\n/home/ubuntu/anaconda3/envs/caffe2/lib/python3.6/asyncio/base_events.py(1440): _run_once\n/home/ubuntu/anaconda3/envs/caffe2/lib/python3.6/asyncio/base_events.py(427): run_forever\n/home/ubuntu/anaconda3/envs/caffe2/lib/python3.6/site-packages/tornado/platform/asyncio.py(132): start\n/home/ubuntu/anaconda3/envs/caffe2/lib/python3.6/site-packages/ipykernel/kernelapp.py(505): start\n/home/ubuntu/anaconda3/envs/caffe2/lib/python3.6/site-packages/traitlets/config/application.py(658): launch_instance\n/home/ubuntu/anaconda3/envs/caffe2/lib/python3.6/site-packages/ipykernel_launcher.py(16): <module>\n/home/ubuntu/anaconda3/envs/caffe2/lib/python3.6/runpy.py(85): _run_code\n/home/ubuntu/anaconda3/envs/caffe2/lib/python3.6/runpy.py(193): _run_module_as_main\n\n\nGraph we tried to export:\ngraph(%0 : Float(1, 3, 224, 224)\n      %1 : Float(64, 3, 3, 3)\n      %2 : Float(64)\n      %3 : Float(16, 64, 1, 1)\n      %4 : Float(16)\n      %5 : Float(64, 16, 1, 1)\n      %6 : Float(64)\n      %7 : Float(64, 16, 3, 3)\n      %8 : Float(64)\n      %9 : Float(16, 128, 1, 1)\n      %10 : Float(16)\n      %11 : Float(64, 16, 1, 1)\n      %12 : Float(64)\n      %13 : Float(64, 16, 3, 3)\n      %14 : Float(64)\n      %15 : Float(32, 128, 1, 1)\n      %16 : Float(32)\n      %17 : Float(128, 32, 1, 1)\n      %18 : Float(128)\n      %19 : Float(128, 32, 3, 3)\n      %20 : Float(128)\n      %21 : Float(32, 256, 1, 1)\n      %22 : Float(32)\n      %23 : Float(128, 32, 1, 1)\n      %24 : Float(128)\n      %25 : Float(128, 32, 3, 3)\n      %26 : Float(128)\n      %27 : Float(48, 256, 1, 1)\n      %28 : Float(48)\n      %29 : Float(192, 48, 1, 1)\n      %30 : Float(192)\n      %31 : Float(192, 48, 3, 3)\n      %32 : Float(192)\n      %33 : Float(48, 384, 1, 1)\n      %34 : Float(48)\n      %35 : Float(192, 48, 1, 1)\n      %36 : Float(192)\n      %37 : Float(192, 48, 3, 3)\n      %38 : Float(192)\n      %39 : Float(64, 384, 1, 1)\n      %40 : Float(64)\n      %41 : Float(256, 64, 1, 1)\n      %42 : Float(256)\n      %43 : Float(256, 64, 3, 3)\n      %44 : Float(256)\n      %45 : Float(64, 512, 1, 1)\n      %46 : Float(64)\n      %47 : Float(256, 64, 1, 1)\n      %48 : Float(256)\n      %49 : Float(256, 64, 3, 3)\n      %50 : Float(256)\n      %51 : Float(1000, 512, 1, 1)\n      %52 : Float(1000)) {\n  %53 : Float(1, 64, 111, 111) = onnx::Conv[dilations=[1, 1], group=1, kernel_shape=[3, 3], pads=[0, 0, 0, 0], strides=[2, 2]](%0, %1, %2), scope: SqueezeNet/Sequential[features]/Conv2d[0]\n  %54 : Float(1, 64, 111, 111) = onnx::Relu(%53), scope: SqueezeNet/Sequential[features]/ReLU[1]\n  %55 : int[] = onnx::Constant[value= 3  3 [ CPULongType{2} ]]()\n  %56 : int[] = onnx::Constant[value= 2  2 [ CPULongType{2} ]]()\n  %57 : int[] = onnx::Constant[value= 0  0 [ CPULongType{2} ]]()\n  %58 : int[] = onnx::Constant[value= 1  1 [ CPULongType{2} ]]()\n  %59 : Long() = onnx::Constant[value={1}](), scope: SqueezeNet/Sequential[features]/MaxPool2d[2]\n  %60 : Float(1, 64, 55, 55), %61 : Long(1, 64, 55, 55) = aten::max_pool2d_with_indices(%54, %55, %56, %57, %58, %59), scope: SqueezeNet/Sequential[features]/MaxPool2d[2]\n  %62 : Float(1, 16, 55, 55) = onnx::Conv[dilations=[1, 1], group=1, kernel_shape=[1, 1], pads=[0, 0, 0, 0], strides=[1, 1]](%60, %3, %4), scope: SqueezeNet/Sequential[features]/Fire[3]/Conv2d[squeeze]\n  %63 : Float(1, 16, 55, 55) = onnx::Relu(%62), scope: SqueezeNet/Sequential[features]/Fire[3]/ReLU[squeeze_activation]\n  %64 : Float(1, 64, 55, 55) = onnx::Conv[dilations=[1, 1], group=1, kernel_shape=[1, 1], pads=[0, 0, 0, 0], strides=[1, 1]](%63, %5, %6), scope: SqueezeNet/Sequential[features]/Fire[3]/Conv2d[expand1x1]\n  %65 : Float(1, 64, 55, 55) = onnx::Relu(%64), scope: SqueezeNet/Sequential[features]/Fire[3]/ReLU[expand1x1_activation]\n  %66 : Float(1, 64, 55, 55) = onnx::Conv[dilations=[1, 1], group=1, kernel_shape=[3, 3], pads=[1, 1, 1, 1], strides=[1, 1]](%63, %7, %8), scope: SqueezeNet/Sequential[features]/Fire[3]/Conv2d[expand3x3]\n  %67 : Float(1, 64, 55, 55) = onnx::Relu(%66), scope: SqueezeNet/Sequential[features]/Fire[3]/ReLU[expand3x3_activation]\n  %68 : Float(1, 128, 55, 55) = onnx::Concat[axis=1](%65, %67), scope: SqueezeNet/Sequential[features]/Fire[3]\n  %69 : Float(1, 16, 55, 55) = onnx::Conv[dilations=[1, 1], group=1, kernel_shape=[1, 1], pads=[0, 0, 0, 0], strides=[1, 1]](%68, %9, %10), scope: SqueezeNet/Sequential[features]/Fire[4]/Conv2d[squeeze]\n  %70 : Float(1, 16, 55, 55) = onnx::Relu(%69), scope: SqueezeNet/Sequential[features]/Fire[4]/ReLU[squeeze_activation]\n  %71 : Float(1, 64, 55, 55) = onnx::Conv[dilations=[1, 1], group=1, kernel_shape=[1, 1], pads=[0, 0, 0, 0], strides=[1, 1]](%70, %11, %12), scope: SqueezeNet/Sequential[features]/Fire[4]/Conv2d[expand1x1]\n  %72 : Float(1, 64, 55, 55) = onnx::Relu(%71), scope: SqueezeNet/Sequential[features]/Fire[4]/ReLU[expand1x1_activation]\n  %73 : Float(1, 64, 55, 55) = onnx::Conv[dilations=[1, 1], group=1, kernel_shape=[3, 3], pads=[1, 1, 1, 1], strides=[1, 1]](%70, %13, %14), scope: SqueezeNet/Sequential[features]/Fire[4]/Conv2d[expand3x3]\n  %74 : Float(1, 64, 55, 55) = onnx::Relu(%73), scope: SqueezeNet/Sequential[features]/Fire[4]/ReLU[expand3x3_activation]\n  %75 : Float(1, 128, 55, 55) = onnx::Concat[axis=1](%72, %74), scope: SqueezeNet/Sequential[features]/Fire[4]\n  %76 : int[] = onnx::Constant[value= 3  3 [ CPULongType{2} ]]()\n  %77 : int[] = onnx::Constant[value= 2  2 [ CPULongType{2} ]]()\n  %78 : int[] = onnx::Constant[value= 0  0 [ CPULongType{2} ]]()\n  %79 : int[] = onnx::Constant[value= 1  1 [ CPULongType{2} ]]()\n  %80 : Long() = onnx::Constant[value={1}](), scope: SqueezeNet/Sequential[features]/MaxPool2d[5]\n  %81 : Float(1, 128, 27, 27), %82 : Long(1, 128, 27, 27) = aten::max_pool2d_with_indices(%75, %76, %77, %78, %79, %80), scope: SqueezeNet/Sequential[features]/MaxPool2d[5]\n  %83 : Float(1, 32, 27, 27) = onnx::Conv[dilations=[1, 1], group=1, kernel_shape=[1, 1], pads=[0, 0, 0, 0], strides=[1, 1]](%81, %15, %16), scope: SqueezeNet/Sequential[features]/Fire[6]/Conv2d[squeeze]\n  %84 : Float(1, 32, 27, 27) = onnx::Relu(%83), scope: SqueezeNet/Sequential[features]/Fire[6]/ReLU[squeeze_activation]\n  %85 : Float(1, 128, 27, 27) = onnx::Conv[dilations=[1, 1], group=1, kernel_shape=[1, 1], pads=[0, 0, 0, 0], strides=[1, 1]](%84, %17, %18), scope: SqueezeNet/Sequential[features]/Fire[6]/Conv2d[expand1x1]\n  %86 : Float(1, 128, 27, 27) = onnx::Relu(%85), scope: SqueezeNet/Sequential[features]/Fire[6]/ReLU[expand1x1_activation]\n  %87 : Float(1, 128, 27, 27) = onnx::Conv[dilations=[1, 1], group=1, kernel_shape=[3, 3], pads=[1, 1, 1, 1], strides=[1, 1]](%84, %19, %20), scope: SqueezeNet/Sequential[features]/Fire[6]/Conv2d[expand3x3]\n  %88 : Float(1, 128, 27, 27) = onnx::Relu(%87), scope: SqueezeNet/Sequential[features]/Fire[6]/ReLU[expand3x3_activation]\n  %89 : Float(1, 256, 27, 27) = onnx::Concat[axis=1](%86, %88), scope: SqueezeNet/Sequential[features]/Fire[6]\n  %90 : Float(1, 32, 27, 27) = onnx::Conv[dilations=[1, 1], group=1, kernel_shape=[1, 1], pads=[0, 0, 0, 0], strides=[1, 1]](%89, %21, %22), scope: SqueezeNet/Sequential[features]/Fire[7]/Conv2d[squeeze]\n  %91 : Float(1, 32, 27, 27) = onnx::Relu(%90), scope: SqueezeNet/Sequential[features]/Fire[7]/ReLU[squeeze_activation]\n  %92 : Float(1, 128, 27, 27) = onnx::Conv[dilations=[1, 1], group=1, kernel_shape=[1, 1], pads=[0, 0, 0, 0], strides=[1, 1]](%91, %23, %24), scope: SqueezeNet/Sequential[features]/Fire[7]/Conv2d[expand1x1]\n  %93 : Float(1, 128, 27, 27) = onnx::Relu(%92), scope: SqueezeNet/Sequential[features]/Fire[7]/ReLU[expand1x1_activation]\n  %94 : Float(1, 128, 27, 27) = onnx::Conv[dilations=[1, 1], group=1, kernel_shape=[3, 3], pads=[1, 1, 1, 1], strides=[1, 1]](%91, %25, %26), scope: SqueezeNet/Sequential[features]/Fire[7]/Conv2d[expand3x3]\n  %95 : Float(1, 128, 27, 27) = onnx::Relu(%94), scope: SqueezeNet/Sequential[features]/Fire[7]/ReLU[expand3x3_activation]\n  %96 : Float(1, 256, 27, 27) = onnx::Concat[axis=1](%93, %95), scope: SqueezeNet/Sequential[features]/Fire[7]\n  %97 : int[] = onnx::Constant[value= 3  3 [ CPULongType{2} ]]()\n  %98 : int[] = onnx::Constant[value= 2  2 [ CPULongType{2} ]]()\n  %99 : int[] = onnx::Constant[value= 0  0 [ CPULongType{2} ]]()\n  %100 : int[] = onnx::Constant[value= 1  1 [ CPULongType{2} ]]()\n  %101 : Long() = onnx::Constant[value={1}](), scope: SqueezeNet/Sequential[features]/MaxPool2d[8]\n  %102 : Float(1, 256, 13, 13), %103 : Long(1, 256, 13, 13) = aten::max_pool2d_with_indices(%96, %97, %98, %99, %100, %101), scope: SqueezeNet/Sequential[features]/MaxPool2d[8]\n  %104 : Float(1, 48, 13, 13) = onnx::Conv[dilations=[1, 1], group=1, kernel_shape=[1, 1], pads=[0, 0, 0, 0], strides=[1, 1]](%102, %27, %28), scope: SqueezeNet/Sequential[features]/Fire[9]/Conv2d[squeeze]\n  %105 : Float(1, 48, 13, 13) = onnx::Relu(%104), scope: SqueezeNet/Sequential[features]/Fire[9]/ReLU[squeeze_activation]\n  %106 : Float(1, 192, 13, 13) = onnx::Conv[dilations=[1, 1], group=1, kernel_shape=[1, 1], pads=[0, 0, 0, 0], strides=[1, 1]](%105, %29, %30), scope: SqueezeNet/Sequential[features]/Fire[9]/Conv2d[expand1x1]\n  %107 : Float(1, 192, 13, 13) = onnx::Relu(%106), scope: SqueezeNet/Sequential[features]/Fire[9]/ReLU[expand1x1_activation]\n  %108 : Float(1, 192, 13, 13) = onnx::Conv[dilations=[1, 1], group=1, kernel_shape=[3, 3], pads=[1, 1, 1, 1], strides=[1, 1]](%105, %31, %32), scope: SqueezeNet/Sequential[features]/Fire[9]/Conv2d[expand3x3]\n  %109 : Float(1, 192, 13, 13) = onnx::Relu(%108), scope: SqueezeNet/Sequential[features]/Fire[9]/ReLU[expand3x3_activation]\n  %110 : Float(1, 384, 13, 13) = onnx::Concat[axis=1](%107, %109), scope: SqueezeNet/Sequential[features]/Fire[9]\n  %111 : Float(1, 48, 13, 13) = onnx::Conv[dilations=[1, 1], group=1, kernel_shape=[1, 1], pads=[0, 0, 0, 0], strides=[1, 1]](%110, %33, %34), scope: SqueezeNet/Sequential[features]/Fire[10]/Conv2d[squeeze]\n  %112 : Float(1, 48, 13, 13) = onnx::Relu(%111), scope: SqueezeNet/Sequential[features]/Fire[10]/ReLU[squeeze_activation]\n  %113 : Float(1, 192, 13, 13) = onnx::Conv[dilations=[1, 1], group=1, kernel_shape=[1, 1], pads=[0, 0, 0, 0], strides=[1, 1]](%112, %35, %36), scope: SqueezeNet/Sequential[features]/Fire[10]/Conv2d[expand1x1]\n  %114 : Float(1, 192, 13, 13) = onnx::Relu(%113), scope: SqueezeNet/Sequential[features]/Fire[10]/ReLU[expand1x1_activation]\n  %115 : Float(1, 192, 13, 13) = onnx::Conv[dilations=[1, 1], group=1, kernel_shape=[3, 3], pads=[1, 1, 1, 1], strides=[1, 1]](%112, %37, %38), scope: SqueezeNet/Sequential[features]/Fire[10]/Conv2d[expand3x3]\n  %116 : Float(1, 192, 13, 13) = onnx::Relu(%115), scope: SqueezeNet/Sequential[features]/Fire[10]/ReLU[expand3x3_activation]\n  %117 : Float(1, 384, 13, 13) = onnx::Concat[axis=1](%114, %116), scope: SqueezeNet/Sequential[features]/Fire[10]\n  %118 : Float(1, 64, 13, 13) = onnx::Conv[dilations=[1, 1], group=1, kernel_shape=[1, 1], pads=[0, 0, 0, 0], strides=[1, 1]](%117, %39, %40), scope: SqueezeNet/Sequential[features]/Fire[11]/Conv2d[squeeze]\n  %119 : Float(1, 64, 13, 13) = onnx::Relu(%118), scope: SqueezeNet/Sequential[features]/Fire[11]/ReLU[squeeze_activation]\n  %120 : Float(1, 256, 13, 13) = onnx::Conv[dilations=[1, 1], group=1, kernel_shape=[1, 1], pads=[0, 0, 0, 0], strides=[1, 1]](%119, %41, %42), scope: SqueezeNet/Sequential[features]/Fire[11]/Conv2d[expand1x1]\n  %121 : Float(1, 256, 13, 13) = onnx::Relu(%120), scope: SqueezeNet/Sequential[features]/Fire[11]/ReLU[expand1x1_activation]\n  %122 : Float(1, 256, 13, 13) = onnx::Conv[dilations=[1, 1], group=1, kernel_shape=[3, 3], pads=[1, 1, 1, 1], strides=[1, 1]](%119, %43, %44), scope: SqueezeNet/Sequential[features]/Fire[11]/Conv2d[expand3x3]\n  %123 : Float(1, 256, 13, 13) = onnx::Relu(%122), scope: SqueezeNet/Sequential[features]/Fire[11]/ReLU[expand3x3_activation]\n  %124 : Float(1, 512, 13, 13) = onnx::Concat[axis=1](%121, %123), scope: SqueezeNet/Sequential[features]/Fire[11]\n  %125 : Float(1, 64, 13, 13) = onnx::Conv[dilations=[1, 1], group=1, kernel_shape=[1, 1], pads=[0, 0, 0, 0], strides=[1, 1]](%124, %45, %46), scope: SqueezeNet/Sequential[features]/Fire[12]/Conv2d[squeeze]\n  %126 : Float(1, 64, 13, 13) = onnx::Relu(%125), scope: SqueezeNet/Sequential[features]/Fire[12]/ReLU[squeeze_activation]\n  %127 : Float(1, 256, 13, 13) = onnx::Conv[dilations=[1, 1], group=1, kernel_shape=[1, 1], pads=[0, 0, 0, 0], strides=[1, 1]](%126, %47, %48), scope: SqueezeNet/Sequential[features]/Fire[12]/Conv2d[expand1x1]\n  %128 : Float(1, 256, 13, 13) = onnx::Relu(%127), scope: SqueezeNet/Sequential[features]/Fire[12]/ReLU[expand1x1_activation]\n  %129 : Float(1, 256, 13, 13) = onnx::Conv[dilations=[1, 1], group=1, kernel_shape=[3, 3], pads=[1, 1, 1, 1], strides=[1, 1]](%126, %49, %50), scope: SqueezeNet/Sequential[features]/Fire[12]/Conv2d[expand3x3]\n  %130 : Float(1, 256, 13, 13) = onnx::Relu(%129), scope: SqueezeNet/Sequential[features]/Fire[12]/ReLU[expand3x3_activation]\n  %131 : Float(1, 512, 13, 13) = onnx::Concat[axis=1](%128, %130), scope: SqueezeNet/Sequential[features]/Fire[12]\n  %132 : Float(1, 512, 13, 13), %133 : Tensor = onnx::Dropout[ratio=0.5](%131), scope: SqueezeNet/Sequential[classifier]/Dropout[0]\n  %134 : Float(1, 1000, 13, 13) = onnx::Conv[dilations=[1, 1], group=1, kernel_shape=[1, 1], pads=[0, 0, 0, 0], strides=[1, 1]](%132, %51, %52), scope: SqueezeNet/Sequential[classifier]/Conv2d[1]\n  %135 : Float(1, 1000, 13, 13) = onnx::Relu(%134), scope: SqueezeNet/Sequential[classifier]/ReLU[2]\n  %136 : Float(1, 1000, 1, 1) = onnx::GlobalAveragePool(%135), scope: SqueezeNet/Sequential[classifier]/AdaptiveAvgPool2d[3]\n  %137 : Long() = onnx::Constant[value={0}](), scope: SqueezeNet\n  %138 : Tensor = onnx::Shape(%136), scope: SqueezeNet\n  %139 : Long() = onnx::Gather[axis=0](%138, %137), scope: SqueezeNet\n  %140 : Long() = onnx::Constant[value={1000}](), scope: SqueezeNet\n  %141 : Tensor = onnx::Unsqueeze[axes=[0]](%139)\n  %142 : Tensor = onnx::Unsqueeze[axes=[0]](%140)\n  %143 : Tensor = onnx::Concat[axis=0](%141, %142)\n  %144 : Float(1, 1000) = onnx::Reshape(%136, %143), scope: SqueezeNet\n  return (%144);\n}\n"
     ]
    }
   ],
   "source": [
    "torch_out = torch.onnx._export(torch_model,             # model being run\n",
    "                               x,                       # model input (or a tuple for multiple inputs)\n",
    "                               \"squeezenet-v1.onnx\",       # where to save the model (can be a file or file-like object)\n",
    "                               export_params=True)      # store the trained parameter weights inside the model file"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "_NOTE: Fix the previous error by reverting the changes to `class SqueezeNet`. Revert `ceil_mode=True` to `ceil_mode=False`._"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [],
   "source": [
    "torch_out = torch.onnx._export(torch_model,             # model being run\n",
    "                               x,                       # model input (or a tuple for multiple inputs)\n",
    "                               \"squeezenet-v1.onnx\",       # where to save the model (can be a file or file-like object)\n",
    "                               export_params=True)      # store the trained parameter weights inside the model file"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This step will output a `squeezenet-v1.onnx` file (around 5 MB) in your server/computer storage."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Caffe2"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "After that, we can prepare and run the model and **verify** that the result of the model running on PyTorch matches the result running on **ONNX (with Caffe2 backend)**."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [],
   "source": [
    "import onnx\n",
    "import caffe2.python.onnx.backend\n",
    "from onnx import helper"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Load the ONNX GraphProto object**. Graph is a standard Python protobuf object."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [],
   "source": [
    "model = onnx.load(\"squeezenet-v1.onnx\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Prepare the Caffe2 backend for executing the model. This **converts the ONNX graph into a Caffe2 NetDef** that can execute it."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [],
   "source": [
    "prepared_backend = caffe2.python.onnx.backend.prepare(model)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Run the model in Caffe2.**\n",
    "\n",
    "Construct a map from input names to Tensor data.\n",
    "\n",
    "The graph itself contains inputs for all weight parameters, followed by the input image.\n",
    "\n",
    "Since the weights are already embedded, we just need to pass the input image.\n",
    "\n",
    "Last parameter is the input to the graph."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [],
   "source": [
    "W = {model.graph.input[0].name: x.data.numpy()}"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Run the Caffe2 net:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [],
   "source": [
    "c2_out = prepared_backend.run(W)[0]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Verify the numerical correctness upto 3 decimal places."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [],
   "source": [
    "np.testing.assert_almost_equal(torch_out.data.cpu().numpy(), c2_out, decimal=3)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Export the model to run on mobile devices"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Leverage the cross-platform capability of Caffe2."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Export to mobile\n",
    "from caffe2.python.onnx.backend import Caffe2Backend as c2"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "`Caffe2Backend` is the backend for running ONNX on Caffe2."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Rewrite ONNX graph to Caffe2 NetDef:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [],
   "source": [
    "init_net, predict_net = c2.onnx_graph_to_caffe2_net(model)\n",
    "\n",
    "with open(\"squeeze_init_net_v1.pb\", \"wb\") as f:\n",
    "    f.write(init_net.SerializeToString())\n",
    "with open(\"squeeze_predict_net_v1.pb\", \"wb\") as f:\n",
    "    f.write(predict_net.SerializeToString())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You'll see 2 files, `squeeze_init_net.pb` and `squeeze_predict_net.pb` in the same directory of this notebook. Let's make sure it can run with `Predictor` since that's what we'll use in the mobile app."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Loading Pre-Trained Models\n",
    "\n",
    "Optional read or for reference:\n",
    "- [Tutorial](https://caffe2.ai/docs/tutorial-loading-pre-trained-models.html)\n",
    "  - In this tutorial, they will use the SqueezeNet model to identify objects in images.\n",
    "  - You'll learn how to read the protobuf files (i.e.: init_net.pb, predict_net.pb), use the Predictor function in your Caffe2 workspace to load the blobs from the protobufs, and run the net and get the results."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Verify it runs with `Predictor`**"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Read the protobuf (`*.pb`) files:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {
    "collapsed": true
   },
   "outputs": [
    {
     "ename": "UnicodeDecodeError",
     "evalue": "'utf-8' codec can't decode byte 0xf0 in position 24: invalid continuation byte",
     "output_type": "error",
     "traceback": [
      "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
      "\u001b[0;31mUnicodeDecodeError\u001b[0m                        Traceback (most recent call last)",
      "\u001b[0;32m<ipython-input-16-5b4eb965df20>\u001b[0m in \u001b[0;36m<module>\u001b[0;34m\u001b[0m\n\u001b[1;32m      1\u001b[0m \u001b[0;32mwith\u001b[0m \u001b[0mopen\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m\"squeeze_init_net.pb\"\u001b[0m\u001b[0;34m)\u001b[0m \u001b[0;32mas\u001b[0m \u001b[0mf\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m----> 2\u001b[0;31m     \u001b[0minit_net\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mf\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mread\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m      3\u001b[0m \u001b[0;32mwith\u001b[0m \u001b[0mopen\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m\"squeeze_predict_net.pb\"\u001b[0m\u001b[0;34m)\u001b[0m \u001b[0;32mas\u001b[0m \u001b[0mf\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m      4\u001b[0m     \u001b[0mpredict_net\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mf\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mread\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;32m~/anaconda3/envs/caffe2/lib/python3.6/codecs.py\u001b[0m in \u001b[0;36mdecode\u001b[0;34m(self, input, final)\u001b[0m\n\u001b[1;32m    319\u001b[0m         \u001b[0;31m# decode input (taking the buffer into account)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    320\u001b[0m         \u001b[0mdata\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mbuffer\u001b[0m \u001b[0;34m+\u001b[0m \u001b[0minput\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 321\u001b[0;31m         \u001b[0;34m(\u001b[0m\u001b[0mresult\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mconsumed\u001b[0m\u001b[0;34m)\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_buffer_decode\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mdata\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0merrors\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mfinal\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    322\u001b[0m         \u001b[0;31m# keep undecoded input until the next call\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    323\u001b[0m         \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mbuffer\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mdata\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0mconsumed\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;31mUnicodeDecodeError\u001b[0m: 'utf-8' codec can't decode byte 0xf0 in position 24: invalid continuation byte"
     ]
    }
   ],
   "source": [
    "# with open(\"squeeze_init_net_v1.pb\") as f:\n",
    "#     init_net = f.read()\n",
    "# with open(\"squeeze_predict_net_v1.pb\") as f:\n",
    "#     predict_net = f.read()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Fix** the previous `UnicodeDecodeError` error.\n",
    "\n",
    "Solution: [adding `rb` flag when opening the file](https://github.com/pytorch/pytorch/issues/10070#issuecomment-410979572)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [],
   "source": [
    "with open(\"squeeze_init_net_v1.pb\", \"rb\") as f:\n",
    "    init_net = f.read()\n",
    "with open(\"squeeze_predict_net_v1.pb\", \"rb\") as f:\n",
    "    predict_net = f.read()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "[Workspace](https://caffe2.ai/docs/workspace.html) is a key component of Caffe2.\n",
    "\n",
    "\n",
    "> Workspace is a class that holds all the related objects created during runtime:\n",
    ">\n",
    "> 1. all blobs, and\n",
    "> 2. all instantiated networks. It is the owner of all these objects and deals with the scaffolding logistics.\n",
    "\n",
    "I think this concept is somewhat similar to TensorFlow Session."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Use the `Predictor` function in your `Workspace` to load the blobs from the protobufs:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [],
   "source": [
    "from caffe2.python import workspace\n",
    "\n",
    "p = workspace.Predictor(init_net, predict_net) # create Predictor by using init NetDef and predict NetDef"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Finally, **run the net and get the results**!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(1, 1000)\n"
     ]
    }
   ],
   "source": [
    "img = np.random.rand(1, 3, 224, 224).astype(np.float32) # create a random image tensor\n",
    "\n",
    "result, = p.run([img])\n",
    "print(result.shape) # our model produces prediction for each of ImageNet 1000 classes"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Fast.ai Mobile Camera Project"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Integrating Caffe2 on Mobile\n",
    "\n",
    "Caffe2 is optimized for mobile integrations, both Android and iOS and running models on lower powered devices.\n",
    "\n",
    "In this notebook, we will go through what you need to know to implement Caffe2 in your mobile project."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Shipping the models into the Android app\n",
    "\n",
    "After we are sure that it runs with `Predictor`, we can copy `squeeze_init_net_v1.pb` and `squeeze_predict_net_v1.pb` to \n",
    "`AICamera/app/src/main/assets` directory.\n",
    "\n",
    "Now we can launch Android Studio and import the AICamera project. Next, run the app by pressing the `Shift + F10` shortcut keys.\n",
    "\n",
    "You can check [Caffe2 AI Camera tutorial](https://caffe2.ai/docs/AI-Camera-demo-android.html) for more details of how Caffe2 can be invoked in the Android mobile app."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Android App Development using Android Studio"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We are building our mobile app using Android Studio version 3.2.1 and **NDK version r18** and above."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Some of the problems we encountered (OUTDATED AS OF 2019-01-01):\n",
    "\n",
    "- [RESOLVED] Gradle Sync problems:\n",
    "  - Resolution: Install [Android Native Development Kit (NDK)](https://developer.android.com/ndk/) version r15.\n",
    "- [RESOLVED] Error: \"Unable to get the CMake version located at: /home/cedric/m/dev/android/sdk/cmake/bin\"\n",
    "  - Install CMake 3.6.xxxxxxx using SDK Manager:\n",
    "  ![](../../../images/fastai_mobile/Android_Studio_SDK_Manager_Install_CMake.png)\n",
    "- [RESOLVED] Error: \"Expected NDK STL shared object file at `/home/cedric/m/dev/android/sdk/ndk-bundle/sources/cxx-stl/gnu-libstdc++/4.9/libs/armeabi-v7a/libgnustl_shared.so`\"\n",
    "  - [GitHub Issue](https://github.com/caffe2/AICamera/issues/55)\n",
    "- [RESOLVED] Error: \"This Gradle plugin requires a newer IDE able to request IDE model level 3. For Android Studio this means version 3.0+\"\n",
    "  - [GitHub issue](https://github.com/caffe2/AICamera/issues/55)\n",
    "  - [StackOverflow question](https://stackoverflow.com/questions/45171647/this-gradle-plugin-requires-android-studio-3-0-minimum)\n",
    "- [RESOLVED] Error: \"(5, 0) Could not find method google() for arguments [] on repository container.\"\n",
    "  - [GitHub Issue](https://github.com/react-native-community/react-native-svg/issues/584)\n",
    "- [RESOLVED] Error: \"A problem occurred configuring project ':app'. > buildToolsVersion is not specified.\"\n",
    "  - [GitHub Issue](https://github.com/react-native-community/react-native-svg/issues/584)\n",
    "  - [StackOverflow question](https://stackoverflow.com/questions/32153544/errorcause-buildtoolsversion-is-not-specified)\n",
    "- [WIP] Error: \"android A/libc Fatal signal 6 (SIGABRT), code -6\"\n",
    "  - [GitHub Issue—AICamera demo with Other Networks](https://github.com/caffe2/AICamera/issues/37)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Android project, Java source code, and the Android Studio work space\n",
    "\n",
    "- `ClassifyCamera.java` code, initialize Caffe2 core C++ libraries, `squeeze_predict_net_v1.pb` protobuf file.\n",
    "\n",
    "![](../../../images/fastai_mobile/Android_Studio_ClassifyCamera_Java_predict_protobuf.png \"Android Studio - ClassifyCamera.java code, initialize Caffe2, squeeze_predict_net.pb protobuf file\")\n",
    "\n",
    "- NDK external native C++ build handled by CMake tooling and a CMakeLists file.\n",
    "\n",
    "![](../../../images/fastai_mobile/Android_NDK_external_build_cmake_c_plus_plus.png)\n",
    "\n",
    "- CMakeLists source code and JNI libs such as Caffe2 libraries for ARM architecture (i.e.`armeabi-v7a/libCaffe2.a`, etc)\n",
    "\n",
    "![](../../../images/fastai_mobile/Android_Studio_cmake_libCaffe2.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Demo"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Check out a working Caffe2 implementation on mobile:\n",
    "\n",
    "[Android camera app demo (video)](https://youtu.be/TYkoaVNCMos)\n",
    "\n",
    "#### Technical specifications\n",
    "\n",
    "- Network architecture: SqueezeNet 1.1\n",
    "- Real-time image classification from video stream\n",
    "- Performance: average 3 fps (frames per second)\n",
    "\n",
    "#### Steps\n",
    "\n",
    "- Deploy mobile config and the models to devices.\n",
    "- Instantiate a Caffe2 instance (Android) or caffe2::Predictor instance (iOS) to expose the model to your Java or iOS code.\n",
    "- Pass inputs to the model and get outputs back.\n",
    "\n",
    "#### Objects in graph\n",
    "\n",
    "- caffe2::NetDef - (binary-serialized) protocol buffer instance that encapsulates the computation graph and the pre-trained weights.\n",
    "- caffe2::Predictor - stateful class that is instantiated with an \"initialization\" NetDef and a \"predict\" NetDef, and executes the \"predict\" NetDef with the input and returns the output.\n",
    "\n",
    "#### Mobile app layout in pure C++\n",
    "\n",
    "- Caffe2 core library, composed of the Workspace, Blob, Net, and Operator classes.\n",
    "- Caffe2 operator library, a range of Operator implementations (such as convolution, etc)\n",
    "- Non-optional dependencies:\n",
    "  - Google Protobuf (the lite version, around 300kb)\n",
    "  - Eigen, a BLAS (on Android) is required for certain primitives, and a vectorized vector/matrix manipulation library, and Eigen is the fastest benchmarked on ARM.\n",
    "- [NEW 2019-01-01!] Quantized Neural Network PACKage ([QNNPACK](https://github.com/pytorch/QNNPACK))—mobile-optimized implementation of quantized neural network operators\n",
    "- NNPACK, which specifically optimizes convolutions on ARM\n",
    "\n",
    "#### Model\n",
    "\n",
    "A model consists of two parts—a set of weights that represent the learned parameters (updated during training), and a set of 'operations' that form a computation graph that represent how to combine the input data (that varies with each graph pass) with the learned parameters (constant with each graph pass). The parameters (and intermediate states in the computation graph live in a Caffe2 Workspace (like TensorFlow Session), where a Blob represents an arbitrary typed pointer, typically a TensorCPU, which is an *n-*dimensional array (like PyTorch’s Tensor).\n",
    "\n",
    "The core class is caffe2::Predictor, which exposes the constructor:\n",
    "\n",
    "```c++\n",
    "Predictor(const NetDef& init_net, const NetDef& predict_net)\n",
    "```\n",
    "where the two `NetDef` inputs are Google Protocol Buffer objects that represent the 2 computation graphs described above:\n",
    "- the `init_net` typically runs a set of operations that deserialize weights into the Workspace\n",
    "- the `predict_net` specifies how to execute the computation graph for each input\n",
    "\n",
    "The Predictor is a stateful class.\n",
    "\n",
    "#### Performance considerations\n",
    "\n",
    "Currently Caffe2 is optimized for ARM CPUs with NEON (basically any ARM CPU since 2012). There are other advantages to offloading compute onto the GPU/DSP, and it's an active work in progress to expose these in Caffe2.\n",
    "\n",
    "For a convolutional implementation, it is recommended to use NNPACK since that's substantially faster (around 2x-3x) than the standard `im2col/sgemm` implementation used in most frameworks.\n",
    "\n",
    "For non-convolutional (e.g. ranking) workloads, the key computational primitive are often fully-connected layers (e.g. FullyConnectedOp in Caffe2, InnerProductLayer in Caffe, nn.Linear in Torch). For these use cases, you can fall back to a BLAS library, specifically Accelerate on iOS and Eigen on Android.\n",
    "\n",
    "#### Memory considerations\n",
    "\n",
    "The model for memory usage of an instantiated and run Predictor is that it’s the sum of the size of the weights and the total size of the activations. There is no ‘static’ memory allocated, all allocations are tied to the Workspace instance owned by the Predictor, so there should be no memory impact after all Predictor instances are deleted."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.7"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}