{
"nbformat": 4,
"nbformat_minor": 0,
"metadata": {
"colab": {
"name": "Semantic Dictionaries - Building Blocks of Interpretability",
"version": "0.3.2",
"views": {},
"default_view": {},
"provenance": [
{
"file_id": "1lPRYcRiLweDFSMwTl7CUwboCo0YHkAYB",
"timestamp": 1518586294543
}
],
"collapsed_sections": []
},
"kernelspec": {
"name": "python2",
"display_name": "Python 2"
},
"accelerator": "GPU"
},
"cells": [
{
"metadata": {
"id": "JndnmDMp66FL",
"colab_type": "text"
},
"cell_type": "markdown",
"source": [
"##### Copyright 2018 Google LLC.\n",
"\n",
"Licensed under the Apache License, Version 2.0 (the \"License\");"
]
},
{
"metadata": {
"id": "hMqWDc_m6rUC",
"colab_type": "code",
"colab": {
"autoexec": {
"startup": false,
"wait_interval": 0
}
},
"cellView": "both"
},
"cell_type": "code",
"source": [
"# Licensed under the Apache License, Version 2.0 (the \"License\");\n",
"# you may not use this file except in compliance with the License.\n",
"# You may obtain a copy of the License at\n",
"#\n",
"# https://www.apache.org/licenses/LICENSE-2.0\n",
"#\n",
"# Unless required by applicable law or agreed to in writing, software\n",
"# distributed under the License is distributed on an \"AS IS\" BASIS,\n",
"# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n",
"# See the License for the specific language governing permissions and\n",
"# limitations under the License."
],
"execution_count": 0,
"outputs": []
},
{
"metadata": {
"id": "pNqKk1MmrakH",
"colab_type": "text"
},
"cell_type": "markdown",
"source": [
"# Semantic Dictionaries -- Building Blocks of Interpretability\n",
"\n",
"This colab notebook is part of our **Building Blocks of Intepretability** series exploring how intepretability techniques combine together to explain neural networks. If you haven't already, make sure to look at the [**corresponding paper**](https://distill.pub/2018/building-blocks) as well!\n",
"\n",
"This notebook studies **semantic dictionaries**. The basic idea of semantic dictionaries is to marry neuron activations to visualizations of those neurons, transforming them from abstract vectors to something more meaningful to humans. Semantic dictionaries can also be applied to other bases, such as rotated versions of activations space that try to disentangle neurons.\n",
"\n",
" \n",
"\n",
" \n",
"\n",
"This tutorial is based on [**Lucid**](https://github.com/tensorflow/lucid), a network for visualizing neural networks. Lucid is a kind of spiritual successor to DeepDream, but provides flexible abstractions so that it can be used for a wide range of interpretability research.\n",
"\n",
"**Note**: The easiest way to use this tutorial is [as a colab notebook](), which allows you to dive in with no setup. We recommend you enable a free GPU by going:\n",
"\n",
"> **Runtime** → **Change runtime type** → **Hardware Accelerator: GPU**\n",
"\n",
"Thanks for trying Lucid!\n"
]
},
{
"metadata": {
"id": "hOBBuzMaxU37",
"colab_type": "text"
},
"cell_type": "markdown",
"source": [
"# Install / Import / Load"
]
},
{
"metadata": {
"id": "UL1yOZtjqkcj",
"colab_type": "text"
},
"cell_type": "markdown",
"source": [
"This code depends on [Lucid](https://github.com/tensorflow/lucid) (our visualization library), and [svelte](https://svelte.technology/) (a web framework). The following cell will install both of them, and dependancies such as TensorFlow. And then import them as appropriate."
]
},
{
"metadata": {
"id": "AA17rJBLuyYH",
"colab_type": "code",
"colab": {
"autoexec": {
"startup": false,
"wait_interval": 0
},
"output_extras": [
{}
],
"base_uri": "https://localhost:8080/",
"height": 102
},
"outputId": "3acd867e-4fc2-4369-8684-cbdcd3f70c7d",
"executionInfo": {
"status": "ok",
"timestamp": 1520312194763,
"user_tz": 480,
"elapsed": 15116,
"user": {
"displayName": "",
"photoUrl": "",
"userId": ""
}
}
},
"cell_type": "code",
"source": [
"!pip install --quiet lucid==0.0.5\n",
"!npm install -g svelte-cli@2.2.0\n",
"\n",
"import numpy as np\n",
"import tensorflow as tf\n",
"\n",
"import lucid.modelzoo.vision_models as models\n",
"import lucid.optvis.render as render\n",
"from lucid.misc.io import show, load\n",
"from lucid.misc.io.showing import _image_url\n",
"import lucid.scratch.web.svelte as lucid_svelte"
],
"execution_count": 0,
"outputs": [
{
"output_type": "stream",
"text": [
"\u001b[K\u001b[?25h/tools/node/bin/svelte -> /tools/node/lib/node_modules/svelte-cli/bin.js\n",
"/tools/node/lib\n",
"└─┬ \u001b[40m\u001b[33msvelte-cli@2.2.0\u001b[39m\u001b[49m \n",
" └── \u001b[40m\u001b[33msvelte@1.56.2\u001b[39m\u001b[49m \n",
"\n"
],
"name": "stdout"
}
]
},
{
"metadata": {
"id": "HfG1iu5KdmUV",
"colab_type": "text"
},
"cell_type": "markdown",
"source": [
"# Semantic Dictionary Code"
]
},
{
"metadata": {
"id": "GABn20jcn4MC",
"colab_type": "text"
},
"cell_type": "markdown",
"source": [
"## **Defining the interface**\n",
"\n",
"First, we define our \"semantic dictionary\" interface as a [svelte component](https://svelte.technology/). This makes it easy to manage state, like which position we're looking at."
]
},
{
"metadata": {
"id": "ZhUtaAmcOIrw",
"colab_type": "code",
"colab": {
"autoexec": {
"startup": false,
"wait_interval": 0
},
"output_extras": [
{}
],
"base_uri": "https://localhost:8080/",
"height": 102
},
"outputId": "20678755-ab0c-43fc-fa6c-24bdc361fc96",
"executionInfo": {
"status": "ok",
"timestamp": 1520312330316,
"user_tz": 480,
"elapsed": 542,
"user": {
"displayName": "",
"photoUrl": "",
"userId": ""
}
}
},
"cell_type": "code",
"source": [
"%%html_define_svelte SemanticDict\n",
"\n",
"
\n",
"
\n",
" \n",
" \n",
"
\n",
"
\n",
" {{#each present_acts as act, act_ind}}\n",
"
\n",
" \n",
" \n",
"
\n",
" {{/each}}\n",
"
\n",
"
\n",
"\n",
"\n",
"\n",
"\n",
""
],
"execution_count": 0,
"outputs": [
{
"output_type": "stream",
"text": [
"Trying to build svelte component from html...\n",
"svelte compile --format iife /tmp/svelte_plOnQK/SemanticDict_1ed5293.html > /tmp/svelte_plOnQK/SemanticDict_1ed5293.js\n",
"svelte version 1.56.2\n",
"compiling ../tmp/svelte_plOnQK/SemanticDict_1ed5293.html...\n",
"\n"
],
"name": "stdout"
}
]
},
{
"metadata": {
"id": "F4l9Ki-UoVko",
"colab_type": "text"
},
"cell_type": "markdown",
"source": [
"## **Spritemaps**\n",
"\n",
"In order to use the semantic dictionaries, we need \"spritemaps\" of channel visualizations.\n",
"These visualization spritemaps are large grids of images (such as [this one](https://storage.googleapis.com/lucid-static/building-blocks/googlenet_spritemaps/sprite_mixed4d_channel.jpeg)) that visualize every channel in a layer.\n",
"We provide spritemaps for GoogLeNet because making them takes a few hours of GPU time, but\n",
"you can make your own channel spritemaps to explore other models. [Check out other notebooks](https://github.com/tensorflow/lucid#notebooks) on how to\n",
"make your own neuron visualizations.\n",
"\n",
"It's also worth noting that GoogLeNet has unusually semantically meaningful neurons. We don't know why this is -- although it's an active area of research for us. More sophisticated interfaces, such as neuron groups, may work better for networks where meaningful ideas are more entangled or less aligned with the neuron directions."
]
},
{
"metadata": {
"id": "BpGLiyEEoPfB",
"colab_type": "code",
"colab": {
"autoexec": {
"startup": false,
"wait_interval": 0
}
}
},
"cell_type": "code",
"source": [
"layer_spritemap_sizes = {\n",
" 'mixed3a' : 16,\n",
" 'mixed3b' : 21,\n",
" 'mixed4a' : 22,\n",
" 'mixed4b' : 22,\n",
" 'mixed4c' : 22,\n",
" 'mixed4d' : 22,\n",
" 'mixed4e' : 28,\n",
" 'mixed5a' : 28,\n",
" }\n",
"\n",
"def googlenet_spritemap(layer):\n",
" assert layer in layer_spritemap_sizes\n",
" size = layer_spritemap_sizes[layer]\n",
" url = \"https://storage.googleapis.com/lucid-static/building-blocks/googlenet_spritemaps/sprite_%s_channel_alpha.jpeg\" % layer\n",
" return size, url"
],
"execution_count": 0,
"outputs": []
},
{
"metadata": {
"id": "eP9jcxiLowkZ",
"colab_type": "text"
},
"cell_type": "markdown",
"source": [
"## **User facing constructor**\n",
"\n",
"Now we'll create a convenient API for creating semantic dictionary visualizations. It will compute the network activations for an image, grab an appropriate spritemap, and render the interface."
]
},
{
"metadata": {
"id": "9czK9sf1d1bU",
"colab_type": "code",
"colab": {
"autoexec": {
"startup": false,
"wait_interval": 0
}
}
},
"cell_type": "code",
"source": [
"googlenet = models.InceptionV1()\n",
"googlenet.load_graphdef()\n",
"\n",
"\n",
"def googlenet_semantic_dict(layer, img_url):\n",
" img = load(img_url)\n",
" \n",
" # Compute the activations\n",
" with tf.Graph().as_default(), tf.Session():\n",
" t_input = tf.placeholder(tf.float32, [224, 224, 3])\n",
" T = render.import_model(googlenet, t_input, t_input)\n",
" acts = T(layer).eval({t_input: img})[0]\n",
" \n",
" # Find the most interesting position for our initial view\n",
" max_mag = acts.max(-1)\n",
" max_x = np.argmax(max_mag.max(-1))\n",
" max_y = np.argmax(max_mag[max_x])\n",
" \n",
" # Find appropriate spritemap\n",
" spritemap_n, spritemap_url = googlenet_spritemap(layer)\n",
" \n",
" # Actually construct the semantic dictionary interface\n",
" # using our *custom component*\n",
" lucid_svelte.SemanticDict({\n",
" \"spritemap_url\": spritemap_url,\n",
" \"sprite_size\": 110,\n",
" \"sprite_n_wrap\": spritemap_n,\n",
" \"image_url\": _image_url(img),\n",
" \"activations\": [[[{\"n\": n, \"v\": float(act_vec[n])} for n in np.argsort(-act_vec)[:4]] for act_vec in act_slice] for act_slice in acts],\n",
" \"pos\" : [max_y, max_x]\n",
" })"
],
"execution_count": 0,
"outputs": []
},
{
"metadata": {
"id": "k0-gZUpApZyz",
"colab_type": "text"
},
"cell_type": "markdown",
"source": [
"# Now let's make some semantic dictionaries!"
]
},
{
"metadata": {
"id": "MEWC-UKdqRGC",
"colab_type": "code",
"colab": {
"autoexec": {
"startup": false,
"wait_interval": 0
},
"output_extras": [
{}
],
"base_uri": "https://localhost:8080/",
"height": 265
},
"outputId": "096bc577-d5d9-4c48-a914-411b8e77b03a",
"executionInfo": {
"status": "ok",
"timestamp": 1520312336706,
"user_tz": 480,
"elapsed": 2455,
"user": {
"displayName": "",
"photoUrl": "",
"userId": ""
}
}
},
"cell_type": "code",
"source": [
"googlenet_semantic_dict(\"mixed4d\", \"https://storage.googleapis.com/lucid-static/building-blocks/examples/dog_cat.png\")"
],
"execution_count": 0,
"outputs": [
{
"output_type": "display_data",
"data": {
"text/plain": [
""
],
"text/html": [
"\n",
" \n",
" \n",
" \n",
" "
]
},
"metadata": {
"tags": []
}
}
]
},
{
"metadata": {
"id": "Izf_YqCRe6E7",
"colab_type": "code",
"colab": {
"autoexec": {
"startup": false,
"wait_interval": 0
},
"output_extras": [
{}
],
"base_uri": "https://localhost:8080/",
"height": 265
},
"outputId": "ed2b0826-52b4-4714-e019-773162c2b170",
"executionInfo": {
"status": "ok",
"timestamp": 1520312338047,
"user_tz": 480,
"elapsed": 1259,
"user": {
"displayName": "",
"photoUrl": "",
"userId": ""
}
}
},
"cell_type": "code",
"source": [
"googlenet_semantic_dict(\"mixed4d\", \"https://storage.googleapis.com/lucid-static/building-blocks/examples/flowers.png\")"
],
"execution_count": 0,
"outputs": [
{
"output_type": "display_data",
"data": {
"text/plain": [
""
],
"text/html": [
"\n",
" \n",
" \n",
" \n",
" "
]
},
"metadata": {
"tags": []
}
}
]
}
]
}
![](\"https://storage.googleapis.com/lucid-static/building-blocks/notebook_heroes/semantic-dictionary.jpeg\")
\n",
"