{
"nbformat": 4,
"nbformat_minor": 0,
"metadata": {
"colab": {
"name": "Neuron Groups - Building Blocks of Interpretability",
"version": "0.3.2",
"views": {},
"default_view": {},
"provenance": [
{
"file_id": "1QA7GEHXJYOWld7jitZei-Qy5US0iTeN_",
"timestamp": 1518982029535
},
{
"file_id": "1uRqpBNPg-aW3tRU-uo-mWg6cQxuAquHW",
"timestamp": 1518822563463
}
],
"collapsed_sections": []
},
"kernelspec": {
"name": "python2",
"display_name": "Python 2"
},
"accelerator": "GPU"
},
"cells": [
{
"metadata": {
"id": "JndnmDMp66FL",
"colab_type": "text"
},
"cell_type": "markdown",
"source": [
"##### Copyright 2018 Google LLC.\n",
"\n",
"Licensed under the Apache License, Version 2.0 (the \"License\");"
]
},
{
"metadata": {
"id": "hMqWDc_m6rUC",
"colab_type": "code",
"colab": {
"autoexec": {
"startup": false,
"wait_interval": 0
}
},
"cellView": "both"
},
"cell_type": "code",
"source": [
"# Licensed under the Apache License, Version 2.0 (the \"License\");\n",
"# you may not use this file except in compliance with the License.\n",
"# You may obtain a copy of the License at\n",
"#\n",
"# https://www.apache.org/licenses/LICENSE-2.0\n",
"#\n",
"# Unless required by applicable law or agreed to in writing, software\n",
"# distributed under the License is distributed on an \"AS IS\" BASIS,\n",
"# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n",
"# See the License for the specific language governing permissions and\n",
"# limitations under the License."
],
"execution_count": 0,
"outputs": []
},
{
"metadata": {
"id": "pNqKk1MmrakH",
"colab_type": "text"
},
"cell_type": "markdown",
"source": [
"# Neuron Groups -- Building Blocks of Interpretability\n",
"\n",
"This colab notebook is part of our **Building Blocks of Intepretability** series exploring how intepretability techniques combine together to explain neural networks. If you haven't already, make sure to look at the [**corresponding paper**](https://distill.pub/2018/building-blocks) as well!\n",
"\n",
"This notebook demonstrates **Neuron Groups**, a technique for exploring how detectors a different spatial positions in the network effected its output.\n",
"\n",
"
\n",
"\n",
"
\n",
"\n",
"
\n",
"\n",
"This tutorial is based on [**Lucid**](https://github.com/tensorflow/lucid), a network for visualizing neural networks. Lucid is a kind of spiritual successor to DeepDream, but provides flexible abstractions so that it can be used for a wide range of interpretability research.\n",
"\n",
"**Note**: The easiest way to use this tutorial is [as a colab notebook](), which allows you to dive in with no setup. We recommend you enable a free GPU by going:\n",
"\n",
"> **Runtime** → **Change runtime type** → **Hardware Accelerator: GPU**\n",
"\n",
"Thanks for trying Lucid!\n"
]
},
{
"metadata": {
"id": "hOBBuzMaxU37",
"colab_type": "text"
},
"cell_type": "markdown",
"source": [
"# Install / Import / Load"
]
},
{
"metadata": {
"id": "UL1yOZtjqkcj",
"colab_type": "text"
},
"cell_type": "markdown",
"source": [
"This code depends on [Lucid](https://github.com/tensorflow/lucid) (our visualization library), and [svelte](https://svelte.technology/) (a web framework). The following cell will install both of them, and dependancies such as TensorFlow. And then import them as appropriate."
]
},
{
"metadata": {
"id": "AA17rJBLuyYH",
"colab_type": "code",
"colab": {
"autoexec": {
"startup": false,
"wait_interval": 0
},
"output_extras": [
{}
],
"base_uri": "https://localhost:8080/",
"height": 99
},
"outputId": "5e6b9516-0e6c-4a41-c877-925c6ec16d51",
"executionInfo": {
"status": "ok",
"timestamp": 1520297343023,
"user_tz": 480,
"elapsed": 4989,
"user": {
"displayName": "Christopher Olah",
"photoUrl": "//lh5.googleusercontent.com/-GhJP0RTFLEs/AAAAAAAAAAI/AAAAAAAAEZ8/wDVK-lwJYfA/s50-c-k-no/photo.jpg",
"userId": "104171973056281402320"
}
}
},
"cell_type": "code",
"source": [
"!pip install --quiet lucid==0.0.5\n",
"!npm install -g svelte-cli@2.2.0\n",
"\n",
"import numpy as np\n",
"import tensorflow as tf\n",
"\n",
"import lucid.modelzoo.vision_models as models\n",
"from lucid.misc.io import show\n",
"import lucid.misc.io.showing as showing\n",
"from lucid.misc.channel_reducer import ChannelReducer\n",
"import lucid.optvis.param as param\n",
"import lucid.optvis.objectives as objectives\n",
"import lucid.optvis.render as render\n",
"from lucid.misc.io import show, load\n",
"from lucid.misc.io.reading import read\n",
"from lucid.misc.io.showing import _image_url\n",
"import lucid.scratch.web.svelte as lucid_svelte\n",
"from lucid.misc.gradient_override import gradient_override_map"
],
"execution_count": 0,
"outputs": [
{
"output_type": "stream",
"text": [
"\u001b[K\u001b[?25h/tools/node/bin/svelte -> /tools/node/lib/node_modules/svelte-cli/bin.js\n",
"/tools/node/lib\n",
"└─┬ \u001b[40m\u001b[33msvelte-cli@2.2.0\u001b[39m\u001b[49m \n",
" └── \u001b[40m\u001b[33msvelte@1.56.2\u001b[39m\u001b[49m \n",
"\n"
],
"name": "stdout"
}
]
},
{
"metadata": {
"id": "0cUPBCRyG9xE",
"colab_type": "text"
},
"cell_type": "markdown",
"source": [
"# Attribution & UI Code"
]
},
{
"metadata": {
"id": "FWHqimIqk2Bs",
"colab_type": "code",
"colab": {
"autoexec": {
"startup": false,
"wait_interval": 0
}
}
},
"cell_type": "code",
"source": [
"model = models.InceptionV1()\n",
"model.load_graphdef()"
],
"execution_count": 0,
"outputs": []
},
{
"metadata": {
"id": "xIDcG0vjaDtk",
"colab_type": "code",
"colab": {
"autoexec": {
"startup": false,
"wait_interval": 0
}
}
},
"cell_type": "code",
"source": [
"labels_str = read(\"https://gist.githubusercontent.com/aaronpolhamus/964a4411c0906315deb9f4a3723aac57/raw/aa66dd9dbf6b56649fa3fab83659b2acbf3cbfd1/map_clsloc.txt\")\n",
"labels = [line[line.find(\" \"):].strip() for line in labels_str.split(\"\\n\")]\n",
"labels = [label[label.find(\" \"):].strip().replace(\"_\", \" \") for label in labels]\n",
"labels = [\"dummy\"] + labels"
],
"execution_count": 0,
"outputs": []
},
{
"metadata": {
"id": "PNiXhK5AYR0d",
"colab_type": "code",
"colab": {
"autoexec": {
"startup": false,
"wait_interval": 0
},
"output_extras": [
{}
],
"base_uri": "https://localhost:8080/",
"height": 164
},
"outputId": "93326ac0-ebbf-4caf-c2fe-c39415edd2b6",
"executionInfo": {
"status": "ok",
"timestamp": 1520297344449,
"user_tz": 480,
"elapsed": 571,
"user": {
"displayName": "Christopher Olah",
"photoUrl": "//lh5.googleusercontent.com/-GhJP0RTFLEs/AAAAAAAAAAI/AAAAAAAAEZ8/wDVK-lwJYfA/s50-c-k-no/photo.jpg",
"userId": "104171973056281402320"
}
}
},
"cell_type": "code",
"source": [
"%%html_define_svelte GroupWidget\n",
"\n",
"