{ "cells": [ { "cell_type": "code", "execution_count": 1, "metadata": {}, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "Using TensorFlow backend.\n" ] } ], "source": [ "from keras.applications.resnet50 import ResNet50\n", "from quiver_engine import server\n", "from keras.models import Model" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "# Visualizing CNN's intermediate layer output\n", "\n", "We'll use [quiver](https://github.com/keplr-io/quiver) which works for Keras models." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "![panda](../../assets/panda.png)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "
outputs of Resnet50's max_pooling2d_1 layer
" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Model setup\n", "\n", "We use InceptionV3 but any model from `keras.application` could be used." ] }, { "cell_type": "code", "execution_count": 2, "metadata": { "collapsed": true }, "outputs": [], "source": [ "input_shape = (224, 224, 3) # size of our image: width x height x number of channels\n", "\n", "model = ResNet50(\n", " include_top=False,\n", " weights='imagenet',\n", " input_shape=input_shape # quiver is going to need that\n", ")" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Select intermediate layer (so that the graph is not too big)" ] }, { "cell_type": "code", "execution_count": 3, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "166" ] }, "execution_count": 3, "metadata": {}, "output_type": "execute_result" } ], "source": [ "len(model.layers_by_depth)" ] }, { "cell_type": "code", "execution_count": 4, "metadata": {}, "outputs": [], "source": [ "intermediate_layer = model.get_layer(name=\"add_3\")\n", "\n", "truncated_model = Model(inputs=[model.input], outputs=[intermediate_layer.output])" ] }, { "cell_type": "code", "execution_count": 5, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Number of layers in truncated model: 36\n", "__________________________________________________________________________________________________\n", "Layer (type) Output Shape Param # Connected to \n", "==================================================================================================\n", "input_1 (InputLayer) (None, 224, 224, 3) 0 \n", "__________________________________________________________________________________________________\n", "conv1 (Conv2D) (None, 112, 112, 64) 9472 input_1[0][0] \n", "__________________________________________________________________________________________________\n", "bn_conv1 (BatchNormalization) (None, 112, 112, 64) 256 conv1[0][0] \n", "__________________________________________________________________________________________________\n", "activation_1 (Activation) (None, 112, 112, 64) 0 bn_conv1[0][0] \n", "__________________________________________________________________________________________________\n", "max_pooling2d_1 (MaxPooling2D) (None, 55, 55, 64) 0 activation_1[0][0] \n", "__________________________________________________________________________________________________\n", "res2a_branch2a (Conv2D) (None, 55, 55, 64) 4160 max_pooling2d_1[0][0] \n", "__________________________________________________________________________________________________\n", "bn2a_branch2a (BatchNormalizati (None, 55, 55, 64) 256 res2a_branch2a[0][0] \n", "__________________________________________________________________________________________________\n", "activation_2 (Activation) (None, 55, 55, 64) 0 bn2a_branch2a[0][0] \n", "__________________________________________________________________________________________________\n", "res2a_branch2b (Conv2D) (None, 55, 55, 64) 36928 activation_2[0][0] \n", "__________________________________________________________________________________________________\n", "bn2a_branch2b (BatchNormalizati (None, 55, 55, 64) 256 res2a_branch2b[0][0] \n", "__________________________________________________________________________________________________\n", "activation_3 (Activation) (None, 55, 55, 64) 0 bn2a_branch2b[0][0] \n", "__________________________________________________________________________________________________\n", "res2a_branch2c (Conv2D) (None, 55, 55, 256) 16640 activation_3[0][0] \n", "__________________________________________________________________________________________________\n", "res2a_branch1 (Conv2D) (None, 55, 55, 256) 16640 max_pooling2d_1[0][0] \n", "__________________________________________________________________________________________________\n", "bn2a_branch2c (BatchNormalizati (None, 55, 55, 256) 1024 res2a_branch2c[0][0] \n", "__________________________________________________________________________________________________\n", "bn2a_branch1 (BatchNormalizatio (None, 55, 55, 256) 1024 res2a_branch1[0][0] \n", "__________________________________________________________________________________________________\n", "add_1 (Add) (None, 55, 55, 256) 0 bn2a_branch2c[0][0] \n", " bn2a_branch1[0][0] \n", "__________________________________________________________________________________________________\n", "activation_4 (Activation) (None, 55, 55, 256) 0 add_1[0][0] \n", "__________________________________________________________________________________________________\n", "res2b_branch2a (Conv2D) (None, 55, 55, 64) 16448 activation_4[0][0] \n", "__________________________________________________________________________________________________\n", "bn2b_branch2a (BatchNormalizati (None, 55, 55, 64) 256 res2b_branch2a[0][0] \n", "__________________________________________________________________________________________________\n", "activation_5 (Activation) (None, 55, 55, 64) 0 bn2b_branch2a[0][0] \n", "__________________________________________________________________________________________________\n", "res2b_branch2b (Conv2D) (None, 55, 55, 64) 36928 activation_5[0][0] \n", "__________________________________________________________________________________________________\n", "bn2b_branch2b (BatchNormalizati (None, 55, 55, 64) 256 res2b_branch2b[0][0] \n", "__________________________________________________________________________________________________\n", "activation_6 (Activation) (None, 55, 55, 64) 0 bn2b_branch2b[0][0] \n", "__________________________________________________________________________________________________\n", "res2b_branch2c (Conv2D) (None, 55, 55, 256) 16640 activation_6[0][0] \n", "__________________________________________________________________________________________________\n", "bn2b_branch2c (BatchNormalizati (None, 55, 55, 256) 1024 res2b_branch2c[0][0] \n", "__________________________________________________________________________________________________\n", "add_2 (Add) (None, 55, 55, 256) 0 bn2b_branch2c[0][0] \n", " activation_4[0][0] \n", "__________________________________________________________________________________________________\n", "activation_7 (Activation) (None, 55, 55, 256) 0 add_2[0][0] \n", "__________________________________________________________________________________________________\n", "res2c_branch2a (Conv2D) (None, 55, 55, 64) 16448 activation_7[0][0] \n", "__________________________________________________________________________________________________\n", "bn2c_branch2a (BatchNormalizati (None, 55, 55, 64) 256 res2c_branch2a[0][0] \n", "__________________________________________________________________________________________________\n", "activation_8 (Activation) (None, 55, 55, 64) 0 bn2c_branch2a[0][0] \n", "__________________________________________________________________________________________________\n", "res2c_branch2b (Conv2D) (None, 55, 55, 64) 36928 activation_8[0][0] \n", "__________________________________________________________________________________________________\n", "bn2c_branch2b (BatchNormalizati (None, 55, 55, 64) 256 res2c_branch2b[0][0] \n", "__________________________________________________________________________________________________\n", "activation_9 (Activation) (None, 55, 55, 64) 0 bn2c_branch2b[0][0] \n", "__________________________________________________________________________________________________\n", "res2c_branch2c (Conv2D) (None, 55, 55, 256) 16640 activation_9[0][0] \n", "__________________________________________________________________________________________________\n", "bn2c_branch2c (BatchNormalizati (None, 55, 55, 256) 1024 res2c_branch2c[0][0] \n", "__________________________________________________________________________________________________\n", "add_3 (Add) (None, 55, 55, 256) 0 bn2c_branch2c[0][0] \n", " activation_7[0][0] \n", "==================================================================================================\n", "Total params: 229,760\n", "Trainable params: 226,816\n", "Non-trainable params: 2,944\n", "__________________________________________________________________________________________________\n" ] } ], "source": [ "print(\"Number of layers in truncated model:\", len(truncated_model.layers))\n", "truncated_model.summary()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Sanity check - this should be `(None, 224, 224, 3)` (`None` corresponds to batch size) " ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "(None, 224, 224, 3)" ] }, "execution_count": 6, "metadata": {}, "output_type": "execute_result" } ], "source": [ "truncated_model.get_input_shape_at(0)" ] }, { "cell_type": "markdown", "metadata": { "collapsed": true }, "source": [ "Be sure you ran \n", "\n", "```\n", "make load_101_categories\n", "```\n", "\n", "before you run that to setup the images.\n", "\n", "Also you could point *quiver* to other directory with images.\n", "\n", "Let's run the visualization engine!" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "scrolled": false }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Starting webserver from: /opt/anaconda3/envs/nnets/lib/python3.5/site-packages/quiver_engine\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "::ffff:127.0.0.1 - - [2018-02-04 12:11:52] \"GET /model HTTP/1.1\" 200 32476 0.015062\n", "::ffff:127.0.0.1 - - [2018-02-04 12:11:52] \"GET /inputs HTTP/1.1\" 200 975 0.001692\n" ] } ], "source": [ "imgs_path = \"../../data/101_ObjectCategories/panda\"\n", "\n", "server.launch(\n", " truncated_model,\n", " temp_folder=\"../../data/tmp\",\n", " input_folder=imgs_path,\n", " port=5000\n", ")" ] } ], "metadata": { "hide_input": false, "kernelspec": { "display_name": "Python 3", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.5.3" } }, "nbformat": 4, "nbformat_minor": 2 }