![\"drawing\"](\"https://artsandculture.withgoogle.com/naturalhistorymuseum/images/logo-black.svg\"){kind=logo}
\n",
"\n", "\n", "\n", "\n", "Google’s **Alexander Mordvintsev** is the creator of DeepDream, a computer vision program that uses a neural network to find and create patterns in images. The end result often leads to dream-like, hallucinogenic, hyper-processed images.\n", "\n", "## Available to everyone\n", "\n", "The idea for Infinite Patterns came from a trip to the [Google Arts & Culture Lab](https://experiments.withgoogle.com/collection/arts-culture) in Paris, where [Pinar&Viola](https://www.pinar-viola.com/) saw images created by [DeepDream](https://en.wikipedia.org/wiki/DeepDream) first hand – and noticed similarities between the neural network’s creations and their own body of work. Noticing an opportunity for collaboration, “it felt as if all we have ever done in our career was done to bring us here,” said the artists.\n", "\n", "In December 2018, Pinar&Viola joined the lab as artists-in-residence, where they were able to work with Alex in the creation of new work. **Alex created a tool for the artists**, allowing them to create infinite patterns with DeepDream. First, the artists curated a selection of pictures, which were fed into the neural nets tool to extract inspiring patterns. Next, they combined these ML patterns with the original images to create the final design.\n", "\n", "**Today, the tool made by Alexander is available for you to use in the creation of your own art works!**\n", "\n", "This tools can create a pattern from any picture. More info about how the algorithms works [here](https://distill.pub/2018/differentiable-parameterizations/). " ] }, { "cell_type": "markdown", "metadata": { "id": "BwMfcuH3aWS-", "colab_type": "text" }, "source": [ "# Instructions ([video](https://www.youtube.com/watch?v=pFXaE7MLiTU))" ] }, { "cell_type": "markdown", "metadata": { "id": "Rw_stzrEWhTR", "colab_type": "text" }, "source": [ "1. Click **CONNECT** in the top right corner\n", "\n", "\n", "\n", "2. Run the cells by clicking the play button on the top left corner\n", "\n", "\n", "\n", "3. Click \"download\" link in the upper-left corner of the resulting image to save it." ] }, { "cell_type": "markdown", "metadata": { "id": "gBDgjeQvaaL8", "colab_type": "text" }, "source": [ "# Upload an image" ] }, { "cell_type": "code", "metadata": { "id": "xy0Fs7UZ21ak", "colab_type": "code", "cellView": "form", "outputId": "b09e7e58-c135-4ec7-bb4f-d5842060e473", "colab": { "base_uri": "https://localhost:8080/", "height": 221 } }, "source": [ "#@title ← Click this button to upload a different image\n", "\n", "try:\n", " import lucid\n", "except ImportError:\n", " !pip install -q lucid\n", "\n", "from __future__ import print_function\n", "import os\n", "import io\n", "import string\n", "import numpy as np\n", "import PIL\n", "import base64\n", "from glob import glob\n", "import itertools\n", "\n", "import cv2\n", "\n", "import matplotlib.pylab as pl\n", "\n", "import tensorflow as tf\n", "from tensorflow.contrib import slim\n", "\n", "import IPython\n", "from IPython.display import clear_output, Image, display, HTML\n", "\n", "from google.colab import files\n", "from google.colab import output\n", "from google.colab import drive\n", "\n", "from lucid.modelzoo import vision_models\n", "import lucid.misc.io.showing as show\n", "from lucid.optvis import objectives\n", "from lucid.optvis import render\n", "from lucid.misc.tfutil import create_session\n", "from lucid.optvis import style\n", "from lucid.modelzoo.util import forget_xy\n", "\n", "\n", "def imwrite(fn, img):\n", " if len(img.shape) == 4:\n", " img = img[0]\n", " img = np.uint8(img.clip(0, 1)*255)\n", " im = PIL.Image.fromarray(img)\n", " im.save(fn, quality=95)\n", "\n", "def show_tiled_image(img):\n", " url = show._image_url(img, fmt='jpeg')\n", " h, w = img.shape[:2]\n", "\n", " display(HTML('''\n", " \n", " \n", " '''.format(url=url, h=h, w=w)))\n", "\n", "\n", "def anorm(a, axis=None, keepdims=False):\n", " return (a*a).sum(axis=axis, keepdims=keepdims)**0.5\n", "\n", "\n", "def composite_activation(x):\n", " x = tf.atan(x)\n", " # Coefficients computed by:\n", " # def rms(x):\n", " # return np.sqrt((x*x).mean())\n", " # a = np.arctan(np.random.normal(0.0, 1.0, 10**6))\n", " # print(rms(a), rms(a*a))\n", " return tf.concat([x/0.67, (x*x)/0.6], -1)\n", "\n", "\n", "def composite_activation_unbiased(x):\n", " x = tf.atan(x)\n", " # Coefficients computed by:\n", " # a = np.arctan(np.random.normal(0.0, 1.0, 10**6))\n", " # aa = a*a\n", " # print(a.std(), aa.mean(), aa.std())\n", " return tf.concat([x/0.67, (x*x-0.45)/0.396], -1)\n", "\n", "\n", "def relu_normalized(x):\n", " x = tf.nn.relu(x)\n", " # Coefficients computed by:\n", " # a = np.random.normal(0.0, 1.0, 10**6)\n", " # a = np.maximum(a, 0.0)\n", " # print(a.mean(), a.std())\n", " return (x-0.40)/0.58\n", "\n", "\n", "def image_cppn(\n", " size,\n", " offset=0.0,\n", " num_output_channels=3,\n", " num_hidden_channels=24,\n", " num_layers=8,\n", " activation_fn=composite_activation,\n", " normalize=False):\n", " coord_range = tf.to_float(tf.range(size))/tf.to_float(size)*2.0*np.pi\n", " #coord_range = tf.linspace(-np.pi, np.pi, size)\n", " y, x = tf.meshgrid(coord_range, coord_range, indexing='ij')\n", " net = tf.expand_dims(tf.stack([x, y], -1), 0) # add batch dimension\n", " net += offset\n", " net = tf.concat([tf.sin(net), tf.cos(net)], -1)\n", " \n", "\n", " with slim.arg_scope([slim.conv2d], kernel_size=1, activation_fn=None):\n", " for i in range(num_layers):\n", " in_n = int(net.shape[-1])\n", " net = slim.conv2d(\n", " net, num_hidden_channels,\n", " # this is untruncated version of tf.variance_scaling_initializer\n", " weights_initializer=tf.random_normal_initializer(0.0, np.sqrt(1.0/in_n)),\n", " )\n", " if normalize:\n", " net = slim.instance_norm(net)\n", " net = activation_fn(net)\n", " \n", " rgb = slim.conv2d(net, num_output_channels, activation_fn=tf.nn.sigmoid,\n", " weights_initializer=tf.zeros_initializer())\n", " return rgb\n", "\n", "\n", "\n", "def render_graph(fn, size=224, stripe_width=256):\n", " graph_def = tf.GraphDef.FromString(open(fn, 'rb').read())\n", " g = tf.Graph()\n", " with g.as_default():\n", " tf.import_graph_def(graph_def, name='')\n", " with tf.Session(graph=g) as sess:\n", " ty, tx = 'meshgrid/mul:0', 'meshgrid/mul_1:0'\n", " y, x = sess.run([ty, tx], {'size:0': size})\n", " stripes = []\n", " for s in range(0, len(x), stripe_width):\n", " stripe = sess.run('image:0', \n", " {tx: x[s:s+stripe_width], ty: y[s:s+stripe_width]})\n", " stripes.append(stripe[0])\n", " return np.vstack(stripes)\n", "\n", "\n", "model = vision_models.InceptionV1_caffe()\n", "model.load_graphdef()\n", "\n", "print('\\n↓ Now click the \"Choose Files\" button and select an image\\n')\n", "\n", "from google.colab import files\n", "uploaded = files.upload()\n", "image_name, _ = uploaded.popitem()\n", "\n", "clear_output()\n", "\n", "im = PIL.Image.open(image_name)\n", "g_image = np.float32(im)[...,:3]/255.0\n", "show.image(g_image)\n", "\n", "print('\\rimage uploaded, size:', im.size, end='')" ], "execution_count": 0, "outputs": [ { "output_type": "display_data", "data": { "text/html": [ "