{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": false,
    "editable": false,
    "nbgrader": {
     "checksum": "f2b4f7deb5cbe6a88fca4d3616c19c96",
     "grade": false,
     "grade_id": "cell-ff3f3e0205bff3e4",
     "locked": true,
     "schema_version": 1,
     "solution": false
    }
   },
   "source": [
    "# Part 2: Denoising Autoencoder"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "deletable": false,
    "editable": false,
    "nbgrader": {
     "checksum": "512b9d458893e474da69aa7e23a01e24",
     "grade": false,
     "grade_id": "cell-9e42398fb4955fba",
     "locked": true,
     "schema_version": 1,
     "solution": false
    }
   },
   "outputs": [],
   "source": [
    "# Execute this code block to install dependencies when running on colab\n",
    "try:\n",
    "    import torch\n",
    "except:\n",
    "    from os.path import exists\n",
    "    from wheel.pep425tags import get_abbr_impl, get_impl_ver, get_abi_tag\n",
    "    platform = '{}{}-{}'.format(get_abbr_impl(), get_impl_ver(), get_abi_tag())\n",
    "    cuda_output = !ldconfig -p|grep cudart.so|sed -e 's/.*\\.\\([0-9]*\\)\\.\\([0-9]*\\)$/cu\\1\\2/'\n",
    "    accelerator = cuda_output[0] if exists('/dev/nvidia0') else 'cpu'\n",
    "\n",
    "    !pip install -q http://download.pytorch.org/whl/{accelerator}/torch-1.0.0-{platform}-linux_x86_64.whl torchvision"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": false,
    "editable": false,
    "nbgrader": {
     "checksum": "f18d024a6245ca61bd7d423b6ead9ecc",
     "grade": false,
     "grade_id": "cell-2122f281579eb211",
     "locked": true,
     "schema_version": 1,
     "solution": false
    }
   },
   "source": [
    "### Extend the Autoencoder you implemented in Part 1 to a Denoising Autoencoder\n",
    "\n",
    "Recall from the lecture, a denoising autoencoder's architecture is very similar to a standard autoencoder. The difference is the input to the autoencoder has noise added to it. However, when computing the loss function, make sure the original (non-noisy) version is used for backpropagation.\n",
    "\n",
    "Again, let's start by loading the Fashion-MNIST dataset and transforming it to a flattened tensor."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "deletable": false,
    "editable": false,
    "nbgrader": {
     "checksum": "2a5ae3ba06d7ee0f67407e62f07576a0",
     "grade": false,
     "grade_id": "cell-93479840905ca9f8",
     "locked": true,
     "schema_version": 1,
     "solution": false
    }
   },
   "outputs": [],
   "source": [
    "%matplotlib inline\n",
    "\n",
    "import torchvision\n",
    "import torchvision.transforms as transforms\n",
    "\n",
    "import numpy as np\n",
    "\n",
    "batch_size = 256\n",
    "image_dim = 784  # [flattened]\n",
    "\n",
    "# dataset construction\n",
    "transform = transforms.Compose([\n",
    "    transforms.ToTensor(), # convert to tensor\n",
    "    transforms.Lambda(lambda x: x.view(image_dim)) # flatten into vector\n",
    "    ])\n",
    "\n",
    "train_set = torchvision.datasets.FashionMNIST(\n",
    "    root='./data/FashionMNIST'\n",
    "    ,train=True\n",
    "    ,download=True\n",
    "    ,transform=transform\n",
    ")\n",
    "\n",
    "train_loader = torch.utils.data.DataLoader(\n",
    "    train_set, batch_size=batch_size\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": false,
    "editable": false,
    "nbgrader": {
     "checksum": "c6c6c49a7df533da3fb6d0ba9a32e73f",
     "grade": false,
     "grade_id": "cell-c4a66cd9fec76585",
     "locked": true,
     "schema_version": 1,
     "solution": false
    }
   },
   "source": [
    "## Build a Denoising Autoencoder"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": false,
    "editable": false,
    "nbgrader": {
     "checksum": "895626fdc719173fe5603de420affa8e",
     "grade": false,
     "grade_id": "cell-5c3c6923a69817e7",
     "locked": true,
     "schema_version": 1,
     "solution": false
    }
   },
   "source": [
    "Now, define the Encoder and Decoder classes for your denoising autoencoder, called DN_Encoder, DN_Decoder, respectively. You can define these architectures how you like; some suggested architectures are given as comments in the classes below."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "deletable": false,
    "nbgrader": {
     "checksum": "53b00202e7794014773cd92c6e1e0f1e",
     "grade": true,
     "grade_id": "cell-cd2ff87b0e33565c",
     "locked": false,
     "points": 3,
     "schema_version": 1,
     "solution": true
    }
   },
   "outputs": [],
   "source": [
    "import torch\n",
    "import torch.nn as nn\n",
    "import torch.nn.functional as F\n",
    "import torch.optim as optim\n",
    "\n",
    "from tqdm.autonotebook import tqdm\n",
    "from itertools import chain\n",
    "\n",
    "\n",
    "class DN_Encoder(nn.Module):\n",
    "    '''\n",
    "    Denoising encoder with a single input, hidden and output layer\n",
    "    '''\n",
    "    def __init__(self, input_dim, hidden_dim, output_dim):\n",
    "        super(DN_Encoder, self).__init__()\n",
    "        # YOUR CODE HERE\n",
    "        raise NotImplementedError()\n",
    "    \n",
    "    def forward(self, x):\n",
    "        # YOUR CODE HERE\n",
    "        raise NotImplementedError()\n",
    "\n",
    "        \n",
    "class DN_Decoder(nn.Module):\n",
    "    '''\n",
    "    Denoising decoder: single dense hidden layer followed by \n",
    "    output layer with a sigmoid to squish values\n",
    "    '''\n",
    "    def __init__(self, input_dim, hidden_dim, output_dim):\n",
    "        super(DN_Decoder, self).__init__()\n",
    "        # YOUR CODE HERE\n",
    "        raise NotImplementedError()\n",
    "\n",
    "    def forward(self, x):\n",
    "        # YOUR CODE HERE\n",
    "        raise NotImplementedError()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": false,
    "editable": false,
    "nbgrader": {
     "checksum": "7b37e6a8a4c85000c31c7006fb152759",
     "grade": false,
     "grade_id": "cell-de510173c317d06d",
     "locked": true,
     "schema_version": 1,
     "solution": false
    }
   },
   "source": [
    "## Learning your Denoising Autoencoder \n",
    "\n",
    "Start from the training procedure used in Part 1 for the autoencoder and extend this to get your denoising autoencoder working. Again, include images of both the data with added noise as well as the reconstructed images in the submitted notebook. Regarding the noise to add to your images, add Gaussian noise with a mean of 0 and a standard deviation of 1."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "deletable": false,
    "nbgrader": {
     "checksum": "507c78c8590b753846a0bb5bf41ebb32",
     "grade": true,
     "grade_id": "cell-1083635b21dee1c5",
     "locked": false,
     "points": 7,
     "schema_version": 1,
     "solution": true
    }
   },
   "outputs": [],
   "source": [
    "# YOUR CODE HERE\n",
    "raise NotImplementedError()"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.7"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}