![](\"images/gan1.jpeg\")
"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Some other examples:"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"![](\"images/gan2.gif\")
"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Overview"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"![](\"images/gan3.jpeg\")
\n",
"\n",
"[source](http://www.kdnuggets.com/2017/01/generative-adversarial-networks-hot-topic-machine-learning.html)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Generative Adversarial Networks are composed of two models:\n",
"\n",
"* The first model is called a **Generator** $G(z,\\theta_1)$ and it aims to generate new data similar to the expected one, such that, the noice $z$ is mapped to the desired distribution $x$. The Generator could be asimilated to a human art forger, which creates fake works of art.\n",
"\n",
"* The second model is named the **Discriminator** $D(x,\\theta_2)$. This model’s goal is to recognize if an input data is ‘real’ — belongs to the original dataset — or if it is ‘fake’ — generated by a forger. In this scenario, a Discriminator is analogous to the police (or an art expert), which tries to detect artworks as truthful or fraud.\n",
"\n",
"How do these models interact? Paraphrasing the original paper which proposed this framework, it can be thought of the Generator as having an adversary, the Discriminator. The Generator (forger) needs to learn how to create data in such a way that the Discriminator isn’t able to distinguish it as fake anymore. The competition between these two teams is what improves their knowledge, until the Generator succeeds in creating realistic data.\n",
"\n",
"As a result, the Discriminator is trained to correctly classify the input data as either real or fake. This means it’s weights are updated as to maximize the probability that any real data input x is classified as belonging to the real dataset, while minimizing the probability that any fake image is classified as belonging to the real dataset. In more technical terms, the loss/error function used maximizes the function $D(x)$, and it also minimizes $D(G(z))$.\n",
"\n",
"Furthermore, the Generator is trained to fool the Discriminator by generating data as realistic as possible, which means that the Generator’s weight’s are optimized to maximize the probability that any fake image is classified as belonging to the real datase. Formally this means that the loss/error function used for this network maximizes $D(G(z))$."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Since during training both the Discriminator and Generator are trying to optimize opposite loss functions, they can be thought of two agents playing a minimax game with value function $V(G,D)$. In this minimax game, the generator is trying to maximize it’s probability of having it’s outputs recognized as real, while the discriminator is trying to minimize this same value."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"![](\"images/gan4.png\")
"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Training a GAN\n",
"\n",
"![](\"images/gan5.png\")
"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"The fundamental steps to train a GAN can be described as following:\n",
"\n",
"1. Sample a noise set and a real-data set, each with size m.\n",
"2. Train the Discriminator on this data.\n",
"3. Sample a different noise subset with size m.\n",
"4. Train the Generator on this data.\n",
"5. Repeat from Step 1."
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "slide"
}
},
"source": [
"# Example: MNIST Handwritten Digit Recognition"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"The MNIST problem is a dataset developed by Yann LeCun, Corinna Cortes and Christopher Burges for evaluating machine learning models on the handwritten digit classification problem.\n",
"\n",
"The dataset was constructed from a number of scanned document dataset available from the National Institute of Standards and Technology (NIST). This is where the name for the dataset comes from, as the Modified NIST or MNIST dataset.\n",
"\n",
"Images of digits were taken from a variety of scanned documents, normalized in size and centered. This makes it an excellent dataset for evaluating models, allowing the developer to focus on the machine learning with very little data cleaning or preparation required.\n",
"\n",
"Each image is a 28 by 28 pixel square (784 pixels total). A standard spit of the dataset is used to evaluate and compare models, where 60,000 images are used to train a model and a separate set of 10,000 images are used to test it.\n",
"\n",
"It is a digit recognition task. As such there are 10 digits (0 to 9) or 10 classes to predict. Results are reported using prediction error, which is nothing more than the inverted classification accuracy.\n",
"\n",
"Excellent results achieve a prediction error of less than 1%. State-of-the-art prediction error of approximately 0.2% can be achieved with large Convolutional Neural Networks. There is a listing of the state-of-the-art results and links to the relevant papers on the MNIST and other datasets on Rodrigo Benenson’s webpage."
]
},
{
"cell_type": "code",
"execution_count": 1,
"metadata": {},
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
"/home/al/anaconda3/lib/python3.6/site-packages/h5py/__init__.py:36: FutureWarning: Conversion of the second argument of issubdtype from `float` to `np.floating` is deprecated. In future, it will be treated as `np.float64 == np.dtype(float).type`.\n",
" from ._conv import register_converters as _register_converters\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"['/job:localhost/replica:0/task:0/device:GPU:0']\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"Using TensorFlow backend.\n"
]
}
],
"source": [
"import tensorflow as tf\n",
"sess = tf.Session(config=tf.ConfigProto(log_device_placement=True))\n",
"import keras\n",
"from keras import backend as K\n",
"print(K.tensorflow_backend._get_available_gpus())"
]
},
{
"cell_type": "code",
"execution_count": 2,
"metadata": {},
"outputs": [],
"source": [
"# Plot ad hoc mnist instances\n",
"from keras.datasets import mnist\n",
"import matplotlib.pyplot as plt\n",
"import matplotlib.animation as animation\n",
"import matplotlib.image as mgimg\n",
"from IPython.display import HTML\n",
"\n",
"import os\n",
"import numpy as np\n",
"from tqdm import tqdm\n",
"\n",
"%matplotlib notebook\n",
"%matplotlib notebook"
]
},
{
"cell_type": "code",
"execution_count": 3,
"metadata": {},
"outputs": [
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAT4AAAD8CAYAAADub8g7AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvNQv5yAAAF3BJREFUeJzt3XtsFdX2B/DvEsUXESgKVEDApKL4C4gPRC8iXsQgasC3RKVEYk0EgwYN6EUjUbE+Ex+goPJSAl6DCGqMklogRmwAH/cCFYokYLEBEREQlYuu3x8dt7PHnvY85szMOfv7SZqufXZ7Zl277mJmzp4ZUVUQEbnkiLgTICKKGhsfETmHjY+InMPGR0TOYeMjIuew8RGRc9j4iMg5OTU+ERkmIptEZIuITA4rKaK4sbaLm2S7gFlEWgHYDGAogHoAawCMUtWN4aVHFD3WdvE7Moff7Q9gi6puBQARWQRgBICUxSEivEwkOXar6klxJ5FQGdU26zpR0qrrXA51uwD41jeu916jwrAt7gQSjLVduNKq61z2+KSJ1/72L5+IVACoyGE7RFFrsbZZ14Utl8ZXD6Cbb9wVwHfBH1LVWQBmATwkoILRYm2zrgtbLoe6awCUiUhPEWkN4CYAy8JJiyhWrO0il/Uen6oeFpHxAD4E0ArAbFXdEFpmRDFhbRe/rJezZLUxHhIkyTpVPTfuJIoB6zpR0qprXrlBRM5h4yMi57DxEZFz2PiIyDlsfETkHDY+InIOGx8ROSeXS9aIqEidc8451nj8+PEmHj16tDU3f/58E7/wwgvW3Oeff56H7HLHPT4icg4bHxE5h42PiJzDa3Wb0KpVK2vctm3btH/Xfy7kuOOOs+Z69epl4nHjxllzTz/9tIlHjRplzf36668mrqystOamTp2adm4BvFY3JIVS180566yzrPHHH39sjU844YS03uenn36yxh06dMgtsczxWl0ioqaw8RGRc4p6Ocspp5xijVu3bm3iCy+80JobOHCgidu1a2fNXXvttaHkU19fb+Lnn3/emrv66qtNvH//fmvuq6++MvHKlStDyYWof//+Jl68eLE1Fzy94z8lFqzPQ4cOmTh4aDtgwAATB5e2+H8vatzjIyLnsPERkXPY+IjIOUW3nMX/sXzwI/lMlqWE4Y8//rDGt912m4kPHDiQ8vcaGhqs8Y8//mjiTZs2hZQdl7OEJcnLWfxLqs4++2xr7o033jBx165drTkR+wmb/j4RPFf35JNPmnjRokUp32fKlCnW3OOPP95s7lnichYioqaw8RGRc4puOcv27dtN/MMPP1hzYRzq1tTUWOO9e/da40suucTEwY/rX3/99Zy3T5SJmTNnmjh4RVC2gofMbdq0MXFwudXgwYNN3KdPn1C2Hwbu8RGRc9j4iMg5bHxE5JyiO8e3Z88eE993333W3JVXXmniL774wpoLXkLm9+WXX5p46NCh1tzPP/9sjc8880wTT5gwIY2MicITvHPyFVdcYeLgEhW/4Lm5d9991xr77x703XffWXP+/y/5l14BwD//+c+0th817vERkXNabHwiMltEdonIet9rJSKyXETqvO/t85smUfhY2+5q8coNERkE4ACA+ar6f95rTwLYo6qVIjIZQHtVndTixmJe4e6/mWLwDhP+j/3Hjh1rzd1yyy0mXrhwYZ6yi5zzV26EVdtx13VzVys1dwPRDz74wMTBpS4XX3yxNfYvRXn11Vetue+//z7lNn7//XcTHzx4MOU2QnwoUThXbqjqKgB7Ai+PADDPi+cBGJlxekQxY227K9sPNzqpagMAqGqDiHRM9YMiUgGgIsvtEEUtrdpmXRe2vH+qq6qzAMwC4j8kIAoL67qwZdv4dopIqfcvYimAXWEmlS/79u1LORd8SIrf7bffbuI333zTmgvegYUKXuJr+7TTTrPG/mVbwcsyd+/ebeLgXX/mzZtn4uDdgt5///1mx9k49thjrfHEiRNNfPPNN+f8/pnIdjnLMgDlXlwOYGk46RDFjrXtgHSWsywEsBpALxGpF5GxACoBDBWROgBDvTFRQWFtu6vobkSareOPP97EwVXr/o/dL7/8cmvuo48+ym9i+eP8cpawRFHXRx99tInfeusta2748OEmDh6y3njjjSZeu3atNec/9PQ/CCtM/uUswV6zevVqE1900UVhbZI3IiUiagobHxE5h42PiJxTdHdnyZb/Liv+5SuAfTnNK6+8Ys1VV1dbY/95lOnTp1tzUZ5PpeLSr18/E/vP6QWNGDHCGvMB9E3jHh8ROYeNj4icw0PdJnzzzTfWeMyYMSaeM2eONXfrrbemHPuXyADA/PnzTRxcRU/UnGeffdbEwRt6+g9nk3Zoe8QRf+1bJekqJ+7xEZFz2PiIyDlsfETkHJ7jS8OSJUtMXFdXZ835z70AwJAhQ0w8bdo0a6579+4mfuyxx6y5HTt25JwnFQ//g7EA+y7LwWVRy5YtiySnbPjP6wXz9j/EK2rc4yMi57DxEZFz2PiIyDk8x5eh9evXW+MbbrjBGl911VUmDq75u+OOO0xcVlZmzQUfVE5uC96tuHXr1ibetcu+KXTwruBR898y6+GHH075c8EnwN1///35SqlF3OMjIuew8RGRc3iom6O9e/da49dff93EwQcvH3nkX/+5Bw0aZM0NHjzYxCtWrAgvQSo6v/32mzWO+vJH/6EtAEyZMsXE/gcfAfadnZ955hlrLni36Chxj4+InMPGR0TOYeMjIufwHF+G+vTpY42vu+46a3zeeeeZ2H9OL2jjxo3WeNWqVSFkRy6I4xI1/yVzwfN4/ie5LV1qP4b42muvzW9iWeIeHxE5h42PiJzDQ90m9OrVyxqPHz/exNdcc40117lz57Tf1/9w5eAShCTdnZbiF7zLsn88cuRIa27ChAmhb/+ee+6xxg8++KCJ27Zta80tWLDAxKNHjw49l3zgHh8ROafFxici3USkWkRqRWSDiEzwXi8RkeUiUud9b5//dInCw9p2Vzp7fIcBTFTVMwAMADBORHoDmAygSlXLAFR5Y6JCwtp2VIvn+FS1AUCDF+8XkVoAXQCMADDY+7F5AFYAmJSXLPMgeG5u1KhRJvaf0wOAHj16ZLUN/8PFAfuuy0m+a64rklzbwbsV+8fB2n3++edNPHv2bGvuhx9+MPGAAQOsOf8TAfv27WvNde3a1Rpv377dxB9++KE1N2PGjL//D0i4jM7xiUgPAP0A1ADo5BXOnwXUMezkiKLC2nZL2p/qikgbAIsB3K2q+4KfOjXzexUAKrJLjyj/sqlt1nVhS6vxichRaCyMBar6tvfyThEpVdUGESkFsKup31XVWQBmee+jTf1MvnTq1Mka9+7d28QvvviiNXf66adntY2amhpr/NRTT5k4uIqdS1aSJ9vajrOuW7VqZY3vvPNOEwevlNi3b5+Jgze/bc6nn35qjaurq0380EMPpf0+SZXOp7oC4DUAtarqf6TYMgDlXlwOYGnwd4mSjLXtrnT2+P4B4FYA/xWRP58H9wCASgD/FpGxALYDuD4/KRLlDWvbUel8qvsJgFQnPYakeJ0o8Vjb7ir4S9ZKSkqs8cyZM03sv6MEAJx66qlZbcN/viN4F9ngR/u//PJLVtsg8lu9erU1XrNmjYn9dwAKCi51CZ7n9vMvdVm0aJE1l4/L4JKEl6wRkXPY+IjIORJcIZ7XjWX5sf/5559vjf03Quzfv78116VLl2w2gYMHD5rYvxIeAKZNm2bin3/+Oav3T6B1qnpu3EkUgyiWs5SWlprY/3xmwH7YT3ANov//388995w199JLL5l4y5YtoeSZAGnVNff4iMg5bHxE5Bw2PiJyTkGc46usrLTGwYedpBJ8oM97771n4sOHD1tz/mUqwYeEFyme4wtJ1JesUbN4jo+IqClsfETknII41KW84KFuSFjXicJDXSKiprDxEZFz2PiIyDlsfETkHDY+InIOGx8ROYeNj4icw8ZHRM5h4yMi57DxEZFzon7Y0G4A2wCc6MVJ4Gou3SPajguSWNdAsvKJKpe06jrSa3XNRkXWJuU6UeZCYUna3y9J+SQpF4CHukTkIDY+InJOXI1vVkzbbQpzobAk7e+XpHySlEs85/iIiOLEQ10icg4bHxE5J9LGJyLDRGSTiGwRkclRbtvb/mwR2SUi632vlYjIchGp8763jyiXbiJSLSK1IrJBRCbEmQ/lJs7aZl1nLrLGJyKtAEwHcDmA3gBGiUjvqLbvmQtgWOC1yQCqVLUMQJU3jsJhABNV9QwAAwCM8/57xJUPZSkBtT0XrOuMRLnH1x/AFlXdqqqHACwCMCLC7UNVVwHYE3h5BIB5XjwPwMiIcmlQ1c+9eD+AWgBd4sqHchJrbbOuMxdl4+sC4FvfuN57LW6dVLUBaPyjAegYdQIi0gNAPwA1SciHMpbE2o69jpJc11E2PmniNefX0ohIGwCLAdytqvvizoeywtoOSHpdR9n46gF08427Avguwu2nslNESgHA+74rqg2LyFFoLI4Fqvp23PlQ1pJY26zrZkTZ+NYAKBORniLSGsBNAJZFuP1UlgEo9+JyAEuj2KiICIDXANSq6rNx50M5SWJts66bo6qRfQEYDmAzgG8A/CvKbXvbXwigAcD/0Piv9FgAHdD4KVOd970kolwGovFw6D8AvvS+hseVD79y/nvGVtus68y/eMkaETmHV24QkXNyanxxX4lBlC+s7eKW9aGut1p9M4ChaDyvsAbAKFXdGF56RNFjbRe/XJ65YVarA4CI/LlaPWVxiAhPKCbHblU9Ke4kEiqj2mZdJ0padZ3LoW4SV6tT+rbFnUCCsbYLV1p1ncseX1qr1UWkAkBFDtshilqLtc26Lmy5NL60Vqur6ix4t53mIQEViBZrm3Vd2HI51E3ianWiMLC2i1zWe3yqelhExgP4EEArALNVdUNomRHFhLVd/CK9coOHBImyThP0gOdCxrpOlLTqmlduEJFz2PiIyDlsfETkHDY+InIOGx8ROYeNj4icw8ZHRM5h4yMi57DxEZFz2PiIyDlsfETknFxuS0UhGjJkiIkXLFhgzV188cUm3rRpU2Q5EaVjypQpJp46dao1d8QRf+1bDR482JpbuXJlXvNqDvf4iMg5bHxE5JyCONQdNGiQNe7QoYOJlyxZEnU6eXHeeeeZeM2aNTFmQtS8MWPGWONJkyaZ+I8//kj5e1HeAq8l3OMjIuew8RGRc9j4iMg5BXGOL/gxeFlZmYkL9Ryf/2N+AOjZs6eJu3fvbs2JNPW0Q6J4BOvzmGOOiSmT7HGPj4icw8ZHRM4piEPd0aNHW+PVq1fHlEl4SktLrfHtt99u4jfeeMOa+/rrryPJiSiVSy+91MR33XVXyp8L1uqVV15p4p07d4afWJa4x0dEzmHjIyLnsPERkXMK4hxfcOlHMXj11VdTztXV1UWYCdHfDRw40BrPmTPHxG3btk35e0899ZQ13rZtW7iJhaTFjiIis0Vkl4is971WIiLLRaTO+94+v2kShY+17a50dqXmAhgWeG0ygCpVLQNQ5Y2JCs1csLad1OKhrqquEpEegZdHABjsxfMArAAwCSHq06ePiTt16hTmWydCc4cLy5cvjzATd8VV24WgvLzcGp988skpf3bFihUmnj9/fr5SClW2J886qWoDAHjfO4aXElGsWNsOyPuHGyJSAaAi39shihLrurBlu8e3U0RKAcD7vivVD6rqLFU9V1XPzXJbRFFKq7ZZ14Ut2z2+ZQDKAVR635eGlpFn+PDhJj722GPDfvtY+M9V+u/GErRjx44o0qGm5b22k+jEE0+0xrfddps19t9Zee/evdbco48+mr/E8iSd5SwLAawG0EtE6kVkLBqLYqiI1AEY6o2JCgpr213pfKo7KsXUkBSvExUE1ra7EnvlRq9evVLObdiwIcJMwvP000+bOLhEZ/PmzSbev39/ZDmRu3r06GHixYsXp/17L7zwgjWurq4OK6XIFN+1YERELWDjIyLnsPERkXMSe46vOUl64PYJJ5xgjYcN++vSz1tuucWau+yyy1K+zyOPPGLi4HIBonzw16r/EtGmVFVVmfi5557LW05R4R4fETmHjY+InFOQh7olJSVZ/V7fvn1NHHxWrf9hKl27drXmWrdubeKbb77ZmgveJPWXX34xcU1NjTX322+/mfjII+3/9OvWrWs2d6JcjRw50hpXVqZem/3JJ59YY//dWn766adwE4sB9/iIyDlsfETkHDY+InJOYs/x+c+Vqao19/LLL5v4gQceSPs9/R/ZB8/xHT582MQHDx605jZu3Gji2bNnW3Nr1661xitXrjRx8AHK9fX1Jg7ecYYPDad8yPaytK1bt1rjJD0MPAzc4yMi57DxEZFz2PiIyDmJPcd35513mjj4UOILL7wwq/fcvn27id955x1rrra21sSfffZZVu8fVFFhP5LhpJNOMnHwHApRPkya9NcD4vx3UW5Jc2v8igH3+IjIOWx8ROScxB7q+j3xxBNxp5CVIUNS38E8k6UFROk666yzrHFzdwTyW7rUfqbSpk2bQsspibjHR0TOYeMjIuew8RGRcwriHF8xWrJkSdwpUBH66KOPrHH79u1T/qx/2daYMWPylVIicY+PiJzDxkdEzuGhLlER6dChgzVu7mqNGTNmmPjAgQN5yymJuMdHRM5psfGJSDcRqRaRWhHZICITvNdLRGS5iNR531OfRSVKINa2u9LZ4zsMYKKqngFgAIBxItIbwGQAVapaBqDKGxMVEta2o1o8x6eqDQAavHi/iNQC6AJgBIDB3o/NA7ACwKQm3oI8/rs+n3baadZcWHeEofQVS23PmTPHxMGn/jXn008/zUc6BSGjDzdEpAeAfgBqAHTyCgeq2iAiHVP8TgWAiqbmiJIi09pmXRe2tBufiLQBsBjA3aq6L/jMilRUdRaAWd57aAs/ThS5bGqbdV3Y0mp8InIUGgtjgaq+7b28U0RKvX8RSwHsyleSxcL/0KRMDkkofwqxtoN3YLn00ktNHFy+cujQIRNPnz7dmiu2BwhlIp1PdQXAawBqVfVZ39QyAH8+Xr0cwNLg7xIlGWvbXens8f0DwK0A/isiX3qvPQCgEsC/RWQsgO0Ars9PikR5w9p2VDqf6n4CINVJj9R32iRKONa2u3jJWkwuuOACazx37tx4EqGC065dO2vcuXPnlD+7Y8cOE9977715y6nQ8Aw7ETmHjY+InMND3Qilu/aRiPKLe3xE5Bw2PiJyDhsfETmH5/jy6IMPPrDG11/PdbCUu6+//toa+++yMnDgwKjTKUjc4yMi57DxEZFzxH/HkLxvjLfvSZJ1qnpu3EkUA9Z1oqRV19zjIyLnsPERkXPY+IjIOWx8ROQcNj4icg4bHxE5h42PiJzDxkdEzmHjIyLnsPERkXOivjvLbgDbAJzoxUngai7dI9qOC5JY10Cy8okql7TqOtJrdc1GRdYm5TpR5kJhSdrfL0n5JCkXgIe6ROQgNj4ick5cjW9WTNttCnOhsCTt75ekfJKUSzzn+IiI4sRDXSJyTqSNT0SGicgmEdkiIpOj3La3/dkisktE1vteKxGR5SJS531vH1Eu3USkWkRqRWSDiEyIMx/KTZy1zbrOXGSNT0RaAZgO4HIAvQGMEpHeUW3fMxfAsMBrkwFUqWoZgCpvHIXDACaq6hkABgAY5/33iCsfylICansuWNcZiXKPrz+ALaq6VVUPAVgEYESE24eqrgKwJ/DyCADzvHgegJER5dKgqp978X4AtQC6xJUP5STW2mZdZy7KxtcFwLe+cb33Wtw6qWoD0PhHA9Ax6gREpAeAfgBqkpAPZSyJtR17HSW5rqNsfNLEa85/pCwibQAsBnC3qu6LOx/KCms7IOl1HWXjqwfQzTfuCuC7CLefyk4RKQUA7/uuqDYsIkehsTgWqOrbcedDWUtibbOumxFl41sDoExEeopIawA3AVgW4fZTWQag3IvLASyNYqMiIgBeA1Crqs/GnQ/lJIm1zbpujqpG9gVgOIDNAL4B8K8ot+1tfyGABgD/Q+O/0mMBdEDjp0x13veSiHIZiMbDof8A+NL7Gh5XPvzK+e8ZW22zrjP/4pUbROQcXrlBRM5h4yMi57DxEZFz2PiIyDlsfETkHDY+InIOGx8ROYeNj4ic8//wLdlPC/zTWAAAAABJRU5ErkJggg==\n",
"text/plain": [
"![](\"images/gan7.png\")
"
]
},
{
"cell_type": "code",
"execution_count": 10,
"metadata": {},
"outputs": [],
"source": [
"depth = 64\n",
"input_shape = (28, 28, 1)\n",
"\n",
"optimizer = Adam(0.0002, 0.5)"
]
},
{
"cell_type": "code",
"execution_count": 11,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"_________________________________________________________________\n",
"Layer (type) Output Shape Param # \n",
"=================================================================\n",
"conv2d_1 (Conv2D) (None, 14, 14, 64) 1664 \n",
"_________________________________________________________________\n",
"leaky_re_lu_1 (LeakyReLU) (None, 14, 14, 64) 0 \n",
"_________________________________________________________________\n",
"dropout_1 (Dropout) (None, 14, 14, 64) 0 \n",
"_________________________________________________________________\n",
"conv2d_2 (Conv2D) (None, 7, 7, 128) 204928 \n",
"_________________________________________________________________\n",
"leaky_re_lu_2 (LeakyReLU) (None, 7, 7, 128) 0 \n",
"_________________________________________________________________\n",
"dropout_2 (Dropout) (None, 7, 7, 128) 0 \n",
"_________________________________________________________________\n",
"conv2d_3 (Conv2D) (None, 4, 4, 256) 819456 \n",
"_________________________________________________________________\n",
"leaky_re_lu_3 (LeakyReLU) (None, 4, 4, 256) 0 \n",
"_________________________________________________________________\n",
"dropout_3 (Dropout) (None, 4, 4, 256) 0 \n",
"_________________________________________________________________\n",
"conv2d_4 (Conv2D) (None, 4, 4, 512) 3277312 \n",
"_________________________________________________________________\n",
"leaky_re_lu_4 (LeakyReLU) (None, 4, 4, 512) 0 \n",
"_________________________________________________________________\n",
"dropout_4 (Dropout) (None, 4, 4, 512) 0 \n",
"_________________________________________________________________\n",
"flatten_1 (Flatten) (None, 8192) 0 \n",
"_________________________________________________________________\n",
"dense_1 (Dense) (None, 1) 8193 \n",
"_________________________________________________________________\n",
"activation_1 (Activation) (None, 1) 0 \n",
"=================================================================\n",
"Total params: 4,311,553\n",
"Trainable params: 4,311,553\n",
"Non-trainable params: 0\n",
"_________________________________________________________________\n"
]
}
],
"source": [
"Discriminator = Sequential()\n",
"\n",
"# First layer\n",
"Discriminator.add(Conv2D(depth*1, 5, strides=2, input_shape=input_shape, padding='same'))\n",
"Discriminator.add(LeakyReLU(0.2))\n",
"Discriminator.add(Dropout(0.4))\n",
"\n",
"# Second layer\n",
"Discriminator.add(Conv2D(depth*2, 5, strides=2, padding='same'))\n",
"Discriminator.add(LeakyReLU(0.2))\n",
"Discriminator.add(Dropout(0.4))\n",
"\n",
"#Third layer\n",
"Discriminator.add(Conv2D(depth*4, 5, strides=2, padding='same'))\n",
"Discriminator.add(LeakyReLU(0.2))\n",
"Discriminator.add(Dropout(0.4))\n",
"\n",
"#Fourth layer\n",
"Discriminator.add(Conv2D(depth*8, 5, strides=1, padding='same'))\n",
"Discriminator.add(LeakyReLU(0.2))\n",
"Discriminator.add(Dropout(0.4))\n",
"\n",
"# Output layer\n",
"Discriminator.add(Flatten())\n",
"Discriminator.add(Dense(1))\n",
"Discriminator.add(Activation('sigmoid'))\n",
"\n",
"Discriminator.summary()\n",
"\n",
"# Define the network\n",
"img = Input(shape=input_shape)\n",
"\n",
"# Output\n",
"validity = Discriminator(img)\n",
"\n",
"Discriminator = Model(img, validity)\n",
"\n",
"\n",
"Discriminator.compile(loss='binary_crossentropy', optimizer=optimizer, metrics=['accuracy'])\n"
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "subslide"
}
},
"source": [
"## Generator\n",
"\n",
"The generator synthesizes fake images. The fake image is generated from a 100-dimensional noise (uniform distribution between -1.0 to 1.0) using the inverse of convolution, called transposed convolution. Instead of fractionally-strided convolution as suggested in DCGAN, upsampling between the first three layers is used since it synthesizes more realistic handwriting images. In between layers, batch normalization stabilizes learning. The activation function after each layer is a ReLU. The output of the sigmoid at the last layer produces the fake image. Dropout of between 0.3 and 0.5 at the first layer prevents overfitting.\n",
"\n",
"![](\"images/gan6.png\")
"
]
},
{
"cell_type": "code",
"execution_count": 12,
"metadata": {},
"outputs": [],
"source": [
"dim = 7\n",
"depth = 256\n",
"randomDim = 100"
]
},
{
"cell_type": "code",
"execution_count": 13,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"_________________________________________________________________\n",
"Layer (type) Output Shape Param # \n",
"=================================================================\n",
"dense_2 (Dense) (None, 12544) 1266944 \n",
"_________________________________________________________________\n",
"batch_normalization_1 (Batch (None, 12544) 50176 \n",
"_________________________________________________________________\n",
"activation_2 (Activation) (None, 12544) 0 \n",
"_________________________________________________________________\n",
"reshape_1 (Reshape) (None, 7, 7, 256) 0 \n",
"_________________________________________________________________\n",
"dropout_5 (Dropout) (None, 7, 7, 256) 0 \n",
"_________________________________________________________________\n",
"up_sampling2d_1 (UpSampling2 (None, 14, 14, 256) 0 \n",
"_________________________________________________________________\n",
"conv2d_transpose_1 (Conv2DTr (None, 14, 14, 128) 819328 \n",
"_________________________________________________________________\n",
"batch_normalization_2 (Batch (None, 14, 14, 128) 512 \n",
"_________________________________________________________________\n",
"activation_3 (Activation) (None, 14, 14, 128) 0 \n",
"_________________________________________________________________\n",
"up_sampling2d_2 (UpSampling2 (None, 28, 28, 128) 0 \n",
"_________________________________________________________________\n",
"conv2d_transpose_2 (Conv2DTr (None, 28, 28, 64) 204864 \n",
"_________________________________________________________________\n",
"batch_normalization_3 (Batch (None, 28, 28, 64) 256 \n",
"_________________________________________________________________\n",
"activation_4 (Activation) (None, 28, 28, 64) 0 \n",
"_________________________________________________________________\n",
"conv2d_transpose_3 (Conv2DTr (None, 28, 28, 32) 51232 \n",
"_________________________________________________________________\n",
"batch_normalization_4 (Batch (None, 28, 28, 32) 128 \n",
"_________________________________________________________________\n",
"activation_5 (Activation) (None, 28, 28, 32) 0 \n",
"_________________________________________________________________\n",
"conv2d_transpose_4 (Conv2DTr (None, 28, 28, 1) 801 \n",
"_________________________________________________________________\n",
"activation_6 (Activation) (None, 28, 28, 1) 0 \n",
"=================================================================\n",
"Total params: 2,394,241\n",
"Trainable params: 2,368,705\n",
"Non-trainable params: 25,536\n",
"_________________________________________________________________\n"
]
}
],
"source": [
"Generator = Sequential()\n",
"\n",
"# First layer\n",
"Generator.add(Dense(dim*dim*depth, input_dim=randomDim))\n",
"Generator.add(BatchNormalization(momentum=0.9))\n",
"Generator.add(Activation('relu'))\n",
"Generator.add(Reshape((dim, dim, depth)))\n",
"Generator.add(Dropout(0.4))\n",
"\n",
"# Second layer\n",
"Generator.add(UpSampling2D())\n",
"Generator.add(Conv2DTranspose(int(depth/2), 5, padding='same'))\n",
"Generator.add(BatchNormalization(momentum=0.9))\n",
"Generator.add(Activation('relu'))\n",
"\n",
"# Third layer\n",
"Generator.add(UpSampling2D())\n",
"Generator.add(Conv2DTranspose(int(depth/4), 5, padding='same'))\n",
"Generator.add(BatchNormalization(momentum=0.9))\n",
"Generator.add(Activation('relu'))\n",
"\n",
"# Fourth layer\n",
"Generator.add(Conv2DTranspose(int(depth/8), 5, padding='same'))\n",
"Generator.add(BatchNormalization(momentum=0.9))\n",
"Generator.add(Activation('relu'))\n",
"\n",
"# Fifth layer\n",
"Generator.add(Conv2DTranspose(1, 5, padding='same'))\n",
"Generator.add(Activation('tanh'))\n",
"\n",
"Generator.summary()\n",
"\n",
"# Create network\n",
"noise = Input(shape=(randomDim,))\n",
"img = Generator(noise)\n",
"\n",
"Generator = Model(noise, img)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Adversarial model\n",
"\n",
"The adversarial model is just the generator-discriminator stacked together The Generator part is trying to fool the Discriminator and learning from its feedback at the same time. The training parameters are the same as in the Discriminator model except for a reduced learning rate and corresponding weight decay.\n",
"\n",
"![](\"images/gan8.png\")
"
]
},
{
"cell_type": "code",
"execution_count": 14,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"_________________________________________________________________\n",
"Layer (type) Output Shape Param # \n",
"=================================================================\n",
"input_3 (InputLayer) (None, 100) 0 \n",
"_________________________________________________________________\n",
"model_2 (Model) (None, 28, 28, 1) 2394241 \n",
"_________________________________________________________________\n",
"model_1 (Model) (None, 1) 4311553 \n",
"=================================================================\n",
"Total params: 6,705,794\n",
"Trainable params: 2,368,705\n",
"Non-trainable params: 4,337,089\n",
"_________________________________________________________________\n"
]
}
],
"source": [
"# The generator takes noise as input and generates imgs\n",
"noise = Input(shape=(randomDim,))\n",
"img = Generator(noise)\n",
"\n",
"# For the combined model we will only train the generator\n",
"Discriminator.trainable = False\n",
"\n",
"# The discriminator takes generated image as input and determines validity\n",
"valid = Discriminator(img)\n",
"\n",
"# The combined model (stacked generator and discriminator)\n",
"# Trains the generator to fool the discriminator\n",
"dcgan = Model(noise, valid)\n",
"dcgan.compile(loss='binary_crossentropy', optimizer=optimizer)\n",
"\n",
"dcgan.summary()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Training\n",
"\n",
"Training is the hardest part. We determine first if Discriminator model is correct by training it alone with real and fake images. Afterwards, the Discriminator and Adversarial models are trained one after the other."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"![](\"images/gan10.png\")
"
]
},
{
"cell_type": "code",
"execution_count": 15,
"metadata": {},
"outputs": [],
"source": [
"# Create a wall of generated MNIST images\n",
"def plotGeneratedImages(epoch, noise, examples=100, dim=(10, 10), figsize=(10, 10)):\n",
" \n",
" generatedImages = Generator.predict(noise_plots) * 0.5 + 0.5\n",
"\n",
" plt.figure(figsize=figsize)\n",
" for i in range(generatedImages.shape[0]):\n",
" plt.subplot(dim[0], dim[1], i+1)\n",
" plt.imshow(generatedImages[i,:,:,0], cmap='gray')\n",
" plt.axis('off')\n",
"# plt.tight_layout()\n",
" plt.savefig(\"images/gan_images/mnist_%d.png\" % epoch)\n",
" plt.close()"
]
},
{
"cell_type": "code",
"execution_count": 16,
"metadata": {},
"outputs": [],
"source": [
"def train_epoch():\n",
" \n",
" # Adversarial ground truths\n",
" valid = np.ones((batchSize, 1))\n",
" fake = np.zeros((batchSize, 1))\n",
" \n",
" # Get a random set of input noise and images\n",
" noise = np.random.normal(0, 1, size=[batchSize, randomDim])\n",
" idx = np.random.randint(0, X_train.shape[0], size=batchSize)\n",
" imageBatch = X_train[idx]\n",
"\n",
" # Generate fake MNIST images\n",
" generatedImages = Generator.predict(noise)\n",
"\n",
" # Train the discriminator\n",
" Discriminator.trainable = True\n",
" d_loss_real = Discriminator.train_on_batch(imageBatch, valid)\n",
" d_loss_fake = Discriminator.train_on_batch(generatedImages, fake)\n",
" d_loss = 0.5 * np.add(d_loss_real, d_loss_fake)\n",
" Discriminator.trainable = False\n",
"\n",
" # Train the generator\n",
" g_loss = dcgan.train_on_batch(noise, valid)\n",
"\n",
" return d_loss, g_loss"
]
},
{
"cell_type": "code",
"execution_count": 17,
"metadata": {},
"outputs": [],
"source": [
"np.random.seed(42)\n",
"batchSize = 32\n",
"epochs = 4000\n",
"save_interval = 50"
]
},
{
"cell_type": "code",
"execution_count": 18,
"metadata": {},
"outputs": [],
"source": [
"# For plots\n",
"noise_plots = np.random.normal(0, 1, size=[100, randomDim])"
]
},
{
"cell_type": "code",
"execution_count": 19,
"metadata": {},
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
"100%|██████████| 4000/4000 [25:31<00:00, 2.61it/s]\n"
]
}
],
"source": [
"Dloss, Gloss = [], []\n",
"for epoch in tqdm(range(epochs)):\n",
" d_loss, g_loss = train_epoch()\n",
" Dloss.append(d_loss)\n",
" Gloss.append(g_loss)\n",
"\n",
" # If at save interval => save generated image samples\n",
" if epoch % save_interval == 0:\n",
" plotGeneratedImages(epoch, noise_plots)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Plot results"
]
},
{
"cell_type": "code",
"execution_count": 20,
"metadata": {
"scrolled": false
},
"outputs": [
{
"data": {
"text/html": [
"\n",
"\n",
"\n",
"\n",
"\n",
" ![]()
\n",
"
\n", " \n", "
\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
\n",
"\n",
"\n",
"\n", " \n", "
\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "