{
"cells": [
{
"cell_type": "markdown",
"metadata": {
"collapsed": true,
"slideshow": {
"slide_type": "slide"
}
},
"source": [
"# Deep Learning in Action"
]
},
{
"cell_type": "markdown",
"metadata": {
"nbpresent": {
"id": "483671b9-13dd-4991-842c-609cbf51009b"
},
"slideshow": {
"slide_type": "slide"
}
},
"source": [
"## Current state of AI"
]
},
{
"cell_type": "markdown",
"metadata": {
"nbpresent": {
"id": "5bd46de7-2344-42a5-82ab-4e5ca519c7d6"
},
"slideshow": {
"slide_type": "fragment"
}
},
"source": [
"\n",
"\n",
"> In the early days of artificial intelligence, the field rapidly tackled and solved problems that are intellectually difficult for human beings but relatively straightforward for computers - problems that can be described by a list of formal, mathematical rules. The true challenge to artificial intelligence proved to be solving the tasks that are easy for people to perform but hard for people to describe formally—problems that we solve intuitively, that feel automatic, like recognizing spoken words or faces in images.\n",
"\n",
"Goodfellow et al. 2016, [Deep Learning](http://www.deeplearningbook.org/)"
]
},
{
"cell_type": "markdown",
"metadata": {
"nbpresent": {
"id": "84e21a78-4fd8-447b-a706-02f8581f31f8"
},
"slideshow": {
"slide_type": "slide"
}
},
"source": [
"## Easy for us. Difficult for computers"
]
},
{
"cell_type": "markdown",
"metadata": {
"nbpresent": {
"id": "f89dd283-4742-4c38-8e9a-9b60fee06679"
},
"slideshow": {
"slide_type": "fragment"
}
},
"source": [
"\n",
"- object recognition\n",
"- speech recognition\n",
"- speech _generation_\n",
"- labeling images"
]
},
{
"cell_type": "markdown",
"metadata": {
"nbpresent": {
"id": "d67fc7da-4808-436d-baa4-1531a2b4238e"
},
"slideshow": {
"slide_type": "slide"
}
},
"source": [
"## Representations matter\n"
]
},
{
"cell_type": "markdown",
"metadata": {
"nbpresent": {
"id": "91e11f87-8c8c-4f73-aff2-ebf6c28b8e00"
},
"slideshow": {
"slide_type": "fragment"
}
},
"source": [
"\n",
"\n",
" 
\n",
" Source: Goodfellow et al. 2016, Deep Learning\n",
"\n"
]
},
{
"cell_type": "markdown",
"metadata": {
"nbpresent": {
"id": "6652d9c6-f1aa-499f-8958-849e25f9d1a4"
},
"slideshow": {
"slide_type": "slide"
}
},
"source": [
"## Just feed the network the right features?"
]
},
{
"cell_type": "markdown",
"metadata": {
"nbpresent": {
"id": "6d84a892-2d33-43c2-b34b-c4d5b104ec45"
},
"slideshow": {
"slide_type": "fragment"
}
},
"source": [
"\n",
"\n",
"- What are the correct pixel values for a \"bike\" feature?\n",
" - race bike, mountain bike, e-bike?\n",
" - pixels in the shadow may be _much_ darker\n",
" - what if bike is mostly obscured by rider standing in front?"
]
},
{
"cell_type": "markdown",
"metadata": {
"nbpresent": {
"id": "6e44d7d3-586a-4727-a2e4-e947bb101311"
},
"slideshow": {
"slide_type": "slide"
}
},
"source": [
"## Let the network pick the features\n"
]
},
{
"cell_type": "markdown",
"metadata": {
"nbpresent": {
"id": "47b1580b-9347-4b5a-aa56-9770bffb3e3d"
},
"slideshow": {
"slide_type": "fragment"
}
},
"source": [
"\n",
"... a layer at a time\n",
"\n",
"\n",
" 
\n",
" Source: Goodfellow et al. 2016, Deep Learning\n",
"\n",
""
]
},
{
"cell_type": "markdown",
"metadata": {
"nbpresent": {
"id": "0280fc7d-0248-4c17-b161-5eadc14bbf0b"
},
"slideshow": {
"slide_type": "slide"
}
},
"source": [
"## Deep Learning, 2 ways to think about it"
]
},
{
"cell_type": "markdown",
"metadata": {
"nbpresent": {
"id": "48b0d007-c995-45ec-8d52-71cf599d8e4c"
},
"slideshow": {
"slide_type": "fragment"
}
},
"source": [
"\n",
"\n",
"- hierarchical feature extraction (start simple, end complex)\n",
"- function composition (see http://colah.github.io/posts/2015-09-NN-Types-FP/)"
]
},
{
"cell_type": "markdown",
"metadata": {
"nbpresent": {
"id": "12356ed9-0363-4c2c-8e00-704eae432a9f"
},
"slideshow": {
"slide_type": "slide"
}
},
"source": [
"# A Short History of (Deep) Learning "
]
},
{
"cell_type": "markdown",
"metadata": {
"nbpresent": {
"id": "1cf1a17e-669b-4f3c-9159-c5a54ca1baf6"
},
"slideshow": {
"slide_type": "slide"
}
},
"source": [
"## The first wave: cybernetics (1940s - 1960s)"
]
},
{
"cell_type": "markdown",
"metadata": {
"nbpresent": {
"id": "ed8bf68c-84cc-49fe-bf4d-c83ecea6d122"
},
"slideshow": {
"slide_type": "fragment"
}
},
"source": [
"\n",
"\n",
"- neuroscientific motivation\n",
"- linear models"
]
},
{
"cell_type": "markdown",
"metadata": {
"nbpresent": {
"id": "8354e8d4-c88e-45b6-a60a-7535c808296e"
},
"slideshow": {
"slide_type": "slide"
}
},
"source": [
"## McCulloch-Pitts Neuron (MCP, 1943, a.k.a. Logic Circuit)"
]
},
{
"cell_type": "markdown",
"metadata": {
"nbpresent": {
"id": "2579c7a9-db52-45c6-971a-cedbaa78fe37"
},
"slideshow": {
"slide_type": "fragment"
}
},
"source": [
"\n",
"\n",
"- binary output (0 or 1)\n",
"- neurons may have inhibiting (negative) and excitatory (positive) inputs\n",
"- each neuron has a threshold that has to be surpassed by the sum of activations for the neuron to get active (output 1)\n",
"- if just one input is inhibitory, the neuron will not activate\n",
"\n",
"
\n",
"\n",
"\n",
" 
\n",
" Source: https://uwaterloo.ca/data-science/sites/ca.data-science/files/uploads/files/lecture_1_0.pdf\n",
""
]
},
{
"cell_type": "markdown",
"metadata": {
"nbpresent": {
"id": "078a2142-e5be-4e23-b1c4-6cad277d648f"
},
"slideshow": {
"slide_type": "slide"
}
},
"source": [
"## Perceptron (Rosenblatt, 1958): Great expectations"
]
},
{
"cell_type": "markdown",
"metadata": {
"nbpresent": {
"id": "e3408549-4969-486a-a7e0-f6d55797eeaf"
},
"slideshow": {
"slide_type": "fragment"
}
},
"source": [
"\n",
"\n",
"- compute linear combination of inputs\n",
"- return +1 if result is positive, -1 if result is negative\n",
"\n",
"\n",
" 
\n",
" Source: https://uwaterloo.ca/data-science/sites/ca.data-science/files/uploads/files/lecture_1_0.pdf\n",
""
]
},
{
"cell_type": "markdown",
"metadata": {
"nbpresent": {
"id": "a5d088e6-c0c7-4b16-9228-017dce9102a4"
},
"slideshow": {
"slide_type": "slide"
}
},
"source": [
"## Minsky & Papert (1969), \"Perceptrons\": the great disappointment"
]
},
{
"cell_type": "markdown",
"metadata": {
"nbpresent": {
"id": "f965626d-4d0a-40eb-9d77-fd722555b540"
},
"slideshow": {
"slide_type": "fragment"
}
},
"source": [
"\n",
"\n",
"- Perceptrons can only solve linearly separable problems\n",
"- Big loss of interest in neural networks\n",
"\n"
]
},
{
"cell_type": "markdown",
"metadata": {
"nbpresent": {
"id": "ba593c28-3d21-4d20-b444-424be1386366"
},
"slideshow": {
"slide_type": "slide"
}
},
"source": [
"## The second wave: Connectionism (1980s, mid-1990s)"
]
},
{
"cell_type": "markdown",
"metadata": {
"nbpresent": {
"id": "624ae641-ea47-4831-9c18-07a12f31f1d7"
},
"slideshow": {
"slide_type": "fragment"
}
},
"source": [
"\n",
"\n",
"- distributed representations\n",
"- backpropagation gets popular \n"
]
},
{
"cell_type": "markdown",
"metadata": {
"nbpresent": {
"id": "1caf128f-a511-4955-95bb-fd557b9b7739"
},
"slideshow": {
"slide_type": "slide"
}
},
"source": [
"## The magic ingredient: backpropagation"
]
},
{
"cell_type": "markdown",
"metadata": {
"nbpresent": {
"id": "5b1ce3cd-b35d-49a7-8e3e-9e51f99152ca"
},
"slideshow": {
"slide_type": "fragment"
}
},
"source": [
"Several \"origins\" in different fields, see e.g.\n",
"\n",
"- Henry J. Kelley (1960). Gradient theory of optimal flight paths. Ars Journal, 30(10), 947-954.\n",
"\n",
"- Arthur E. Bryson (1961, April). A gradient method for optimizing multi-stage allocation processes. In Proceedings of the Harvard Univ. Symposium on digital computers and their applications.\n",
"\n",
"- Paul Werbos (1974). Beyond regression: New tools for prediction and analysis in the behavioral sciences. PhD thesis, Harvard University.\n",
"\n",
"- Rumelhart, David E.; Hinton, Geoffrey E.; Williams, Ronald J. (8 October 1986). \"Learning representations by back-propagating errors\". Nature. 323 (6088): 533–536. \n",
"\n"
]
},
{
"cell_type": "markdown",
"metadata": {
"nbpresent": {
"id": "1217bdf0-eb28-4ec8-9801-1abf13ef7217"
},
"slideshow": {
"slide_type": "slide"
}
},
"source": [
"## Backprop: How could the magic fail?"
]
},
{
"cell_type": "markdown",
"metadata": {
"nbpresent": {
"id": "8b3135bc-1f3c-47dc-9696-30692e2460d9"
},
"slideshow": {
"slide_type": "fragment"
}
},
"source": [
"\n",
"\n",
"- Only applicable in case of supervised learning\n",
"- Doesn't scale well to multiple layers (as they thought _at the time_)\n",
"- Can converge to poor local minima (as they thought _at the time_)"
]
},
{
"cell_type": "markdown",
"metadata": {
"nbpresent": {
"id": "cecdb782-f8dc-4fba-87b6-e5c1f772a364"
},
"slideshow": {
"slide_type": "slide"
}
},
"source": [
"## The third wave: Deep Learning"
]
},
{
"cell_type": "markdown",
"metadata": {
"nbpresent": {
"id": "a329ad3e-b11f-437b-a2ef-918915a637b0"
},
"slideshow": {
"slide_type": "fragment"
}
},
"source": [
"- everything starts with: Hinton, G. E., Osindero, S., & Teh, Y. W. (2006). A fast learning algorithm for deep belief nets. Neural computation, 18(7), 1527-1554.\n",
"- deep neural networks can be trained efficiently, if the weights are initialized intelligently\n",
"- return of backpropagation\n",
"\n"
]
},
{
"cell_type": "markdown",
"metadata": {
"nbpresent": {
"id": "6973c6c7-73a6-4ee6-a9e2-36a287f030d1"
},
"slideshow": {
"slide_type": "slide"
}
},
"source": [
"## The architectures en vogue now (CNN, RNN, LSTM...) have mostly been around since the 1980s/1990s.\n",
"\n"
]
},
{
"cell_type": "markdown",
"metadata": {
"nbpresent": {
"id": "f21bbcaa-796a-4253-9262-45545eefbd99"
},
"slideshow": {
"slide_type": "fragment"
}
},
"source": [
"So why the ~~hype~~ success _now_?\n"
]
},
{
"cell_type": "markdown",
"metadata": {
"nbpresent": {
"id": "8b75395c-ac9d-480d-a8b3-afa488018b3d"
},
"slideshow": {
"slide_type": "slide"
}
},
"source": [
"## Big data"
]
},
{
"cell_type": "markdown",
"metadata": {
"nbpresent": {
"id": "3e3eefa3-61ec-423e-9068-bd65d217cf3d"
},
"slideshow": {
"slide_type": "fragment"
}
},
"source": [
"\n",
"\n",
"> It is true\n",
"that some skill is required to get good performance from a deep learning algorithm.\n",
"Fortunately, the amount of skill required reduces as the amount of training data\n",
"increases. The learning algorithms reaching human performance on complex tasks\n",
"today are nearly identical to the learning algorithms that struggled to solve toy\n",
"problems in the 1980s [...].\n",
"\n",
"Goodfellow et al. 2016, [Deep Learning](http://www.deeplearningbook.org/)"
]
},
{
"cell_type": "markdown",
"metadata": {
"nbpresent": {
"id": "e2e0d246-7071-494e-9675-c7f1b8421376"
},
"slideshow": {
"slide_type": "slide"
}
},
"source": [
"## Dataset size - rule of thumb"
]
},
{
"cell_type": "markdown",
"metadata": {
"nbpresent": {
"id": "dc0d475f-fce7-47b6-9142-54a6aaa767cb"
},
"slideshow": {
"slide_type": "fragment"
}
},
"source": [
"\n",
"\n",
"> As of 2016, a rough rule of thumb\n",
"is that a supervised deep learning algorithm will generally achieve acceptable\n",
"performance with around 5,000 labeled examples per category, and will match or\n",
"exceed human performance when trained with a dataset containing at least 10\n",
"million labeled examples.\n",
"\n",
"Goodfellow et al. 2016, [Deep Learning](http://www.deeplearningbook.org/)"
]
},
{
"cell_type": "markdown",
"metadata": {
"nbpresent": {
"id": "e59b759d-ae9e-4ee3-b9f7-1291ebe4aec9"
},
"slideshow": {
"slide_type": "slide"
}
},
"source": [
"## Big models"
]
},
{
"cell_type": "markdown",
"metadata": {
"nbpresent": {
"id": "9bb62e93-4382-493b-b53b-4d7df63a9435"
},
"slideshow": {
"slide_type": "fragment"
}
},
"source": [
"\n",
"\n",
"thanks to faster/better\n",
"- hardware (CPUs, GPUs)\n",
"- network infrastructure\n",
"- software implementations\n",
"\n",
"> Since the introduction of hidden units, artificial neural networks have doubled in size roughly every 2.4 years.\n",
"\n",
"Goodfellow et al. 2016, [Deep Learning](http://www.deeplearningbook.org/)"
]
},
{
"cell_type": "markdown",
"metadata": {
"nbpresent": {
"id": "d98313bd-f40e-42c3-9506-6dfa5a1867db"
},
"slideshow": {
"slide_type": "slide"
}
},
"source": [
"## Big impact"
]
},
{
"cell_type": "markdown",
"metadata": {
"nbpresent": {
"id": "747cfe3c-cdcd-4a50-9538-067256fa8cbe"
},
"slideshow": {
"slide_type": "fragment"
}
},
"source": [
"\n",
"\n",
"- deep networks consistently win prestigious competitions (e.g., ImageNet)\n",
"- deep learning solves increasingly complex problems (e.g., sequence-to-sequence learning)\n",
"- deep learning has started to fuel _other research areas_ \n",
"\n",
"and most importantly: Deep learning is _highly profitable_\n",
"\n",
"> Deep learning is now used by many top technology companies including Google, Microsoft, Facebook, IBM, Baidu, Apple, Adobe, Netflix, NVIDIA and NEC.\n",
"\n",
"Goodfellow et al. 2016, [Deep Learning](http://www.deeplearningbook.org/)"
]
},
{
"cell_type": "markdown",
"metadata": {
"nbpresent": {
"id": "82276496-388e-46e7-a183-0b53bbfaa885"
},
"slideshow": {
"slide_type": "slide"
}
},
"source": [
"# Deep Learning Architectures"
]
},
{
"cell_type": "markdown",
"metadata": {
"nbpresent": {
"id": "8d29ce74-5e12-4ee4-ad42-efb90d3024e1"
},
"slideshow": {
"slide_type": "slide"
}
},
"source": [
"## Feedforward Deep Neural Network\n"
]
},
{
"cell_type": "markdown",
"metadata": {
"nbpresent": {
"id": "5bdc92b9-1c76-428a-a122-81905b12f335"
},
"slideshow": {
"slide_type": "fragment"
}
},
"source": [
"\n",
"\n",
"\n",
" 
\n",
" Source: https://uwaterloo.ca/data-science/sites/ca.data-science/files/uploads/files/lecture_1_0.pdf\n",
""
]
},
{
"cell_type": "markdown",
"metadata": {
"nbpresent": {
"id": "2e533bf3-7dd8-4432-9d33-6d69bcae0caa"
},
"slideshow": {
"slide_type": "slide"
}
},
"source": [
"# Multi-layer Perceptron (MLP)\n",
"\n"
]
},
{
"cell_type": "markdown",
"metadata": {
"nbpresent": {
"id": "24e64ebd-5016-4cec-a560-d5f7a8a0b2ce"
},
"slideshow": {
"slide_type": "slide"
}
},
"source": [
"## Caveat (terminology-related)"
]
},
{
"cell_type": "markdown",
"metadata": {
"nbpresent": {
"id": "dd17fbad-5b9c-4c96-801c-dc4decadc4dd"
},
"slideshow": {
"slide_type": "fragment"
}
},
"source": [
"\n",
"\n",
">So “multi-layer” neural networks do not use the perceptron learning\n",
"procedure.\n",
"\n",
">They should never have been called multi-layer perceptrons.\n",
"\n",
"Geoffrey Hinton, [Neural Networks for Machine Learning Lec. 3](http://www.cs.toronto.edu/~tijmen/csc321/slides/lecture_slides_lec3.pdf)\n",
"\n",
"##### What people mean by MLP is just a _deep_ _feedforward_ neural network.\n"
]
},
{
"cell_type": "markdown",
"metadata": {
"nbpresent": {
"id": "a85c9206-dcb8-4d3f-9cda-043d6b009f02"
},
"slideshow": {
"slide_type": "slide"
}
},
"source": [
"## Why hidden layers?"
]
},
{
"cell_type": "markdown",
"metadata": {
"nbpresent": {
"id": "e647d3ae-1e01-4227-955a-0af82ae6f20b"
},
"slideshow": {
"slide_type": "fragment"
}
},
"source": [
"### Learning XOR"
]
},
{
"cell_type": "markdown",
"metadata": {
"nbpresent": {
"id": "5ae6e833-7afc-467c-8ee8-afdc908218de"
},
"slideshow": {
"slide_type": "fragment"
}
},
"source": [
"\n",
"\n",
"We want to predict\n",
"- 0 from [0,0]\n",
"- 0 from [1,1]\n",
"- 1 from [0,1]\n",
"- 1 from [1,0]\n",
"\n",
"\n",
"\n",
" 
\n",
" \n",
""
]
},
{
"cell_type": "markdown",
"metadata": {
"nbpresent": {
"id": "58005fd2-d829-4e88-bda6-369db29d0f23"
},
"slideshow": {
"slide_type": "slide"
}
},
"source": [
"## Trying a linear model "
]
},
{
"cell_type": "markdown",
"metadata": {
"nbpresent": {
"id": "c35e7ccc-ce28-448b-96cd-99b8e7e13101"
},
"slideshow": {
"slide_type": "fragment"
}
},
"source": [
"\n",
"\n",
"$f(\\mathbf{x}; \\mathbf{w}, b) = \\mathbf{x}^T\\mathbf{w} + b$ \n",
"\n",
"\n",
"- with Mean Squared Error cost (MSE), this leads to: $\\mathbf{w}=0, b=0.5$\n",
"- mapping every point to 0.5!"
]
},
{
"cell_type": "markdown",
"metadata": {
"nbpresent": {
"id": "b6015267-c8bc-4658-bc34-30a58385560b"
},
"slideshow": {
"slide_type": "slide"
}
},
"source": [
"## Introduce hidden layer "
]
},
{
"cell_type": "markdown",
"metadata": {
"nbpresent": {
"id": "552b7d6d-222f-419a-892b-0f0dd611261f"
},
"slideshow": {
"slide_type": "fragment"
}
},
"source": [
"\n",
"\n",
"\n",
" 
\n",
" Source: Goodfellow et al. 2016, Deep Learning\n",
""
]
},
{
"cell_type": "markdown",
"metadata": {
"nbpresent": {
"id": "9e05ef09-a476-41d8-9f71-83cd95e8c009"
},
"slideshow": {
"slide_type": "slide"
}
},
"source": [
"## Calculation with hidden layer"
]
},
{
"cell_type": "markdown",
"metadata": {
"nbpresent": {
"id": "64a4d022-dc47-494d-abf5-bec4c5f914aa"
},
"slideshow": {
"slide_type": "fragment"
}
},
"source": [
"$f(\\mathbf{x}; \\mathbf{W}, \\mathbf{c}, \\mathbf{w}, b) = \\mathbf{w}^T (\\mathbf{W}^T\\mathbf{x} + \\mathbf{c}) + b$\n",
"\n",
"\n",
"- Design matrix: $\\mathbf{X} = \\begin{bmatrix}0 & 0 \\\\0 & 1\\\\1 & 0 \\\\1 & 1\\end{bmatrix}$\n",
"\n",
"- Parameters: $\\mathbf{W} = \\begin{bmatrix}1 & 1 \\\\1 & 1\\end{bmatrix}$, $\\mathbf{c} = \\begin{bmatrix}0 \\\\ -1 \\end{bmatrix}$, $\\mathbf{w} = \\begin{bmatrix}1 \\\\ -2 \\end{bmatrix}$\n",
"\n",
"- Input to hidden layer: $\\mathbf{X}\\mathbf{W} = \\begin{bmatrix}0 & 0 \\\\1 & 1\\\\1 & 1 \\\\2 & 2\\end{bmatrix}$, add $\\mathbf{c}$ to every row ==> $\\begin{bmatrix}0 & -1 \\\\1 & 0\\\\1 & 0 \\\\2 & 1\\end{bmatrix}$\n"
]
},
{
"cell_type": "markdown",
"metadata": {
"nbpresent": {
"id": "5ddc1f0e-0ede-4a0f-8153-d674d062e14b"
},
"slideshow": {
"slide_type": "slide"
}
},
"source": [
"## Which gives us...\n"
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "fragment"
}
},
"source": [
"\n",
"\n",
" 
\n",
" \n",
""
]
},
{
"cell_type": "markdown",
"metadata": {
"nbpresent": {
"id": "798103d5-eaaf-48f7-9f2d-c0e7ab72da03"
},
"slideshow": {
"slide_type": "slide"
}
},
"source": [
"## Introducing nonlinearity"
]
},
{
"cell_type": "markdown",
"metadata": {
"nbpresent": {
"id": "2c81b353-91b2-499b-a032-4cbd760e90d9"
},
"slideshow": {
"slide_type": "fragment"
}
},
"source": [
"\n",
"\n",
"$f(\\mathbf{x}; \\mathbf{W}, \\mathbf{c}, \\mathbf{w}, b) = \\mathbf{w}^T max(0, \\mathbf{W}^T\\mathbf{x} + \\mathbf{c}) + b$\n",
"\n",
"\n",
"Output of rectified linear transformation: $\\begin{bmatrix}0 & 0 \\\\1 & 0\\\\1 & 0 \\\\2 & 1\\end{bmatrix}$\n",
"\n",
" 
\n",
" \n",
"\n",
"\n",
"The remaining hidden-to-output transformation is linear, but the classes are already linearly separable."
]
},
{
"cell_type": "markdown",
"metadata": {
"nbpresent": {
"id": "5fb55ca8-260a-4873-8e7e-772f010fce8c"
},
"slideshow": {
"slide_type": "slide"
}
},
"source": [
"## How to train a deep network (1): Gradient Descent"
]
},
{
"cell_type": "markdown",
"metadata": {
"nbpresent": {
"id": "3a2e4af8-5c43-491d-80cf-87fb083ad6c7"
},
"slideshow": {
"slide_type": "slide"
}
},
"source": [
"## Optimization"
]
},
{
"cell_type": "markdown",
"metadata": {
"nbpresent": {
"id": "a085a712-b63a-4a9a-85eb-4e15518d0953"
}
},
"source": [
"\n",
"\n",
"- Like other machine learning algorithms, neural networks learn by _minimizing a cost function_.\n",
"- Cost functions in neural networks normally are not convex and so, cannot be optimized in closed form.\n",
"- The solution is to do gradient descent.\n",
"\n",
"\n",
" 
\n",
" Source: Goodfellow et al. 2016, Deep Learning\n",
""
]
},
{
"cell_type": "markdown",
"metadata": {
"nbpresent": {
"id": "76416d5c-74d7-4355-80da-fda820fd96da"
},
"slideshow": {
"slide_type": "slide"
}
},
"source": [
"## Local minima"
]
},
{
"cell_type": "markdown",
"metadata": {
"nbpresent": {
"id": "578b0eac-adbd-458f-8a55-688e4a5cc3ee"
},
"slideshow": {
"slide_type": "fragment"
}
},
"source": [
"\n",
"\n",
"\n",
" 
\n",
" Source: Goodfellow et al. 2016, Deep Learning\n",
""
]
},
{
"cell_type": "markdown",
"metadata": {
"nbpresent": {
"id": "b6b2ca1e-33a1-42d8-a087-ecb2f9cf982f"
},
"slideshow": {
"slide_type": "slide"
}
},
"source": [
"## Closed-form vs. gradient descent optimization by example: Least Squares"
]
},
{
"cell_type": "markdown",
"metadata": {
"nbpresent": {
"id": "47327248-30f0-4f29-ae6f-e9ebc57b1d71"
},
"slideshow": {
"slide_type": "fragment"
}
},
"source": [
"\n",
"\n",
"- Minimize squared error $f(\\mathbf{x}) = ||\\mathbf{X\\hat{\\beta} - y}||^2_2$\n",
"- Closed form: solve __normal equations__ $\\mathbf{\\hat{\\beta}} = (\\mathbf{X}^T\\mathbf{{X}})^{-1} \\mathbf{X}^T \\mathbf{y}$\n",
"\n",
"\n",
"- Alternatively, follow the gradient: $\\nabla_x f(\\mathbf{x})= \\mathbf{X}^T\\mathbf{X}\\mathbf{\\hat{\\beta}} - \\mathbf{X^Ty}$\n",
"\n",
" \n",
" \n",
"\n",
"\n",
" 
\n",
" Source: Goodfellow et al. 2016, Deep Learning\n",
""
]
},
{
"cell_type": "markdown",
"metadata": {
"nbpresent": {
"id": "822ea43b-4ca4-4a7e-b398-61c63ea42274"
},
"slideshow": {
"slide_type": "slide"
}
},
"source": [
"## This gives us a way to train one weight matrix.\n",
"\n",
"How about a net with several layers?"
]
},
{
"cell_type": "markdown",
"metadata": {
"nbpresent": {
"id": "c7a247da-28b0-4a68-8035-504fa4a342ee"
},
"slideshow": {
"slide_type": "slide"
}
},
"source": [
"## How to train a deep network (2): Backpropagation"
]
},
{
"cell_type": "markdown",
"metadata": {
"nbpresent": {
"id": "98dfb06b-88e1-46b2-8613-892755563625"
},
"slideshow": {
"slide_type": "slide"
}
},
"source": [
"## Who else to ask but Geoff Hinton..."
]
},
{
"cell_type": "markdown",
"metadata": {
"nbpresent": {
"id": "bf740d18-39b3-459a-a5f4-d53b4d0bbb8a"
},
"slideshow": {
"slide_type": "fragment"
}
},
"source": [
"\n",
" \n",
" \n",
"\n",
" 
\n",
" Source: Geoffrey Hinton, Neural Networks for Machine Learning Lec. 3\n",
""
]
},
{
"cell_type": "markdown",
"metadata": {
"nbpresent": {
"id": "48cc25f3-cb47-4db5-8618-4afbe26c2b7a"
},
"slideshow": {
"slide_type": "slide"
}
},
"source": [
"## The mechanics of backprop\n"
]
},
{
"cell_type": "markdown",
"metadata": {
"nbpresent": {
"id": "7bc25aab-7d00-425f-852e-b8ffa6b67ffd"
},
"slideshow": {
"slide_type": "fragment"
}
},
"source": [
"\n",
"- basically, just the chain rule: $\\frac{dz}{dx} = \\frac{dz}{dy} \\frac{dy}{dx}$\n",
"- chained over several layers:\n",
" \n",
"\n",
" 
\n",
" Source: https://colah.github.io/posts/2015-08-Backprop/\n",
"\n",
"\n",
"\n"
]
},
{
"cell_type": "markdown",
"metadata": {
"nbpresent": {
"id": "2599d4ec-ad01-48cf-8b71-a042868fee2e"
},
"slideshow": {
"slide_type": "slide"
}
},
"source": [
"## Backprop example: logistic neuron"
]
},
{
"cell_type": "markdown",
"metadata": {
"nbpresent": {
"id": "42bc0b4e-2d9d-4664-9284-b34ecf7b4980"
},
"slideshow": {
"slide_type": "fragment"
}
},
"source": [
"\n",
" \n",
" \n",
"\n",
" 
\n",
" Source: Geoffrey Hinton, Neural Networks for Machine Learning Lec. 3\n",
""
]
},
{
"cell_type": "markdown",
"metadata": {
"nbpresent": {
"id": "0bf4e6f4-1ad3-43be-8ad4-caa02d6804a0"
},
"slideshow": {
"slide_type": "slide"
}
},
"source": [
"## Decisions (1): Which loss function should I choose?\n"
]
},
{
"cell_type": "markdown",
"metadata": {
"collapsed": true,
"nbpresent": {
"id": "b7a21973-1821-4c79-a142-063e5ad29db8"
},
"slideshow": {
"slide_type": "fragment"
}
},
"source": [
"\n",
"- the _loss_ (or _cost_) function indicates the cost incurred from false prediction / misclassification\n",
"\n",
"- probably the best-known loss function in machine learning is __mean squared error__: \n",
"\n",
" $\\frac{1}{n} \\sum_n{(\\hat{y} - y)^2}$\n",
" \n",
"- most of the time, in deep learning we use __cross entropy__:\n",
"\n",
" $- \\sum_j{t_j log(y_j)}$\n",
" \n",
" This is the negative log probability of the right answer."
]
},
{
"cell_type": "markdown",
"metadata": {
"nbpresent": {
"id": "d3e2bf05-df2b-4af1-8c4d-295644e11a2f"
},
"slideshow": {
"slide_type": "slide"
}
},
"source": [
"## Decisions (2): Which activation function to choose?"
]
},
{
"cell_type": "markdown",
"metadata": {
"nbpresent": {
"id": "4c07e89a-43d9-497f-96e8-8a040339c155"
},
"slideshow": {
"slide_type": "fragment"
}
},
"source": [
"\n",
"\n",
"- purpose of activation function: introduce nonlinearity (see above)\n",
"- for a long time, the sigmoid (logistic) activation function was used a lot:\n",
"\n",
" $y = \\frac{1}{1 + e^{-z}}$\n",
" \n",
" \n",
" 
\n",
" \n",
" \n",
" \n",
"- now _rectified linear units_ (ReLUs) are preferred:\n",
"\n",
" $y = max(0, z)$\n",
" \n",
" \n",
" 
\n",
" "
]
},
{
"cell_type": "markdown",
"metadata": {
"nbpresent": {
"id": "c5161136-7f9f-4a01-b48d-eb18a86d3cb3"
},
"slideshow": {
"slide_type": "slide"
}
},
"source": [
"# Convolutional Neural Networks"
]
},
{
"cell_type": "markdown",
"metadata": {
"nbpresent": {
"id": "4fd82baf-07f9-4f85-b7f2-7b41357ed8d3"
},
"slideshow": {
"slide_type": "slide"
}
},
"source": [
"## Why Conv Nets?"
]
},
{
"cell_type": "markdown",
"metadata": {
"collapsed": true,
"nbpresent": {
"id": "a3f33fe9-3726-428e-ab5f-d1c7ca9c37fb"
},
"slideshow": {
"slide_type": "fragment"
}
},
"source": [
"\n",
"\n",
"- conventional feedforward networks need equally sized input (images for example normally aren't!)\n",
"- convolution operation extracts image features"
]
},
{
"cell_type": "markdown",
"metadata": {
"nbpresent": {
"id": "025d461b-ce92-448d-a6ce-bec06bbf35a6"
},
"slideshow": {
"slide_type": "slide"
}
},
"source": [
"\n",
" 
\n",
" Source: http://cs231n.github.io/convolutional-networks/\n",
""
]
},
{
"cell_type": "markdown",
"metadata": {
"nbpresent": {
"id": "c55ed8a3-4b84-4d63-b5fd-51a723ee0c0b"
},
"slideshow": {
"slide_type": "slide"
}
},
"source": [
"## The Convolution Operation"
]
},
{
"cell_type": "markdown",
"metadata": {
"nbpresent": {
"id": "a22179af-1c53-45a4-a358-c5d26dd968a3"
},
"slideshow": {
"slide_type": "fragment"
}
},
"source": [
"\n",
" 
\n",
" Source: http://cs231n.github.io/convolutional-networks/ (Live Demo on website!)\n",
""
]
},
{
"cell_type": "markdown",
"metadata": {
"nbpresent": {
"id": "1b854720-e1b0-49db-9332-4e7098f2e61a"
},
"slideshow": {
"slide_type": "slide"
}
},
"source": [
"## Convolution and cross-correlation"
]
},
{
"cell_type": "markdown",
"metadata": {
"nbpresent": {
"id": "abe24c13-7a29-470e-8733-abc0e025b419"
},
"slideshow": {
"slide_type": "fragment"
}
},
"source": [
"\n",
"\n",
"- Strictly, the operation shown above (and implemented in most DL libraries) is not convolution, but cross-correlation\n",
"- 1-dimensional discrete convolution: $s(t) = (x * w)(t) = \\sum_a{x(a) w(t-a)}$\n",
"- 2-dimensional convolution: $S(i,j) = I * K (i,j) = \\sum_m \\sum_n{I(m,n)K(i-m,j-n)}$\n",
"- 2-dimensional cross-correlation: $S(i,j) = I * K (i,j) = \\sum_m \\sum_n{I(i+m,j+n)K(m,n)}$\n",
"\n",
"\n"
]
},
{
"cell_type": "markdown",
"metadata": {
"nbpresent": {
"id": "2972f29b-6150-4059-addd-cf26842a30c3"
},
"slideshow": {
"slide_type": "slide"
}
},
"source": [
"## Octave demo"
]
},
{
"cell_type": "markdown",
"metadata": {
"nbpresent": {
"id": "d62f7d1a-2719-4460-aae4-403305a239de"
},
"slideshow": {
"slide_type": "fragment"
}
},
"source": [
"\n",
"\n",
"```\n",
"A = [1,2,3;4,5,6;7,8,9] # input \"image\"\n",
"# padded input matrix, for easier visualization\n",
"A_padded = [zeros(1,size(A,2)+2); [zeros(size(A,1),1), A, zeros(size(A,1),1)]; zeros(1,size(A,2)+2)]\n",
"B = [1,0;0,0] # kernel\n",
"\n",
"# real convolution\n",
"C_full = conv2(A,B,'full') # default\n",
"C_same = conv2(A,B,'same') \n",
"C_valid = conv2(A,B,'valid')\n",
"\n",
"# cross-correlation\n",
"XC = xcorr2(A,B) \n",
"```"
]
},
{
"cell_type": "markdown",
"metadata": {
"nbpresent": {
"id": "29fc973b-e69f-49e8-8f23-7e8d4aa70664"
},
"slideshow": {
"slide_type": "slide"
}
},
"source": [
"## Gimp demo\n"
]
},
{
"cell_type": "markdown",
"metadata": {
"nbpresent": {
"id": "840c726c-e3f8-40a2-ada8-7fb351a5684f"
}
},
"source": [
"\n",
" \n",
"\n",
"- Edge enhance: $\\begin{bmatrix}0 & 0 & 0\\\\-1 & 1 & 0\\\\0 & 0 & 0\\end{bmatrix}$, edge detect: $\\begin{bmatrix}0 & 1 & 0\\\\1 & -4 & 1\\\\0 & 1 & 0\\end{bmatrix}$\n",
"\n",
" \n",
"\n",
"- Blur: $\\begin{bmatrix}1 & 1 & 1\\\\1 & 1 & 1\\\\1 & 1 & 1\\end{bmatrix}$, sharpen: $\\begin{bmatrix}0 & -1 & 0\\\\-1 & 5 & -1\\\\0 & -1 & 0\\end{bmatrix}$\n",
"\n",
"\n",
"see: https://docs.gimp.org/en/plug-in-convmatrix.html"
]
},
{
"cell_type": "markdown",
"metadata": {
"nbpresent": {
"id": "fb4f44f5-38fe-4be5-bea8-2c43e8bb9324"
},
"slideshow": {
"slide_type": "slide"
}
},
"source": [
"# Language modeling and word vectors\n"
]
},
{
"cell_type": "markdown",
"metadata": {
"collapsed": true,
"nbpresent": {
"id": "210ea80f-c891-41cf-82b5-aba1ac61ab89"
},
"slideshow": {
"slide_type": "slide"
}
},
"source": [
"## Language modeling\n",
"\n",
"- predict next word given preceding ones\n",
"- based on statistical properties of the distribution of sequences of words "
]
},
{
"cell_type": "markdown",
"metadata": {
"nbpresent": {
"id": "b82b2474-bb2d-4b57-86d4-061ba495b8cd"
},
"slideshow": {
"slide_type": "slide"
}
},
"source": [
"## Distributional hypothesis: linguistic items with similar distributions have similar meanings\n"
]
},
{
"cell_type": "markdown",
"metadata": {
"nbpresent": {
"id": "ae5ee5ff-fdc5-4bd6-aa62-7963297ae98f"
},
"slideshow": {
"slide_type": "fragment"
}
},
"source": [
"\n",
"- n-gram/count-based (e.g., Latent Semantic Analysis)\n",
"- predictive (neural network language models, e.g., word2vec)"
]
},
{
"cell_type": "markdown",
"metadata": {
"nbpresent": {
"id": "6845b8a7-2f11-489b-aec6-88492ffd80bb"
},
"slideshow": {
"slide_type": "slide"
}
},
"source": [
"## ngram-based"
]
},
{
"cell_type": "markdown",
"metadata": {
"nbpresent": {
"id": "6cc4b015-3b21-4ffb-aece-bfec06b2b791"
},
"slideshow": {
"slide_type": "fragment"
}
},
"source": [
"\n",
"\n",
"- choose ngram-size n\n",
"- estimate the probability $P(w_{t+1}|w_1,...,w_{t−2},w_{t−1},w_t)$ by ignoring context beyond n−1 words and dividing by the count of all given words up till $w_t$\n",
"- e.g., with bigrams: $P(w_{t+1}|w_t = \\frac{count(w_{t+1},w_t)}{count(w_t)})$"
]
},
{
"cell_type": "markdown",
"metadata": {
"nbpresent": {
"id": "3a07640f-61ed-409e-b89f-c26485b8bea8"
},
"slideshow": {
"slide_type": "slide"
}
},
"source": [
"## neural network example (Bengio et al 2001, Bengio et al 2003)"
]
},
{
"cell_type": "markdown",
"metadata": {
"nbpresent": {
"id": "de61fb12-9bfb-4e13-9ebb-b33dae688737"
},
"slideshow": {
"slide_type": "fragment"
}
},
"source": [
"\n",
"\n",
"- choose a context size n, as in ngrams\n",
"- map each word $w_{t−i}$ in the $n-1$ - word context to an associated d-dimensional feature vector $C_{w_{t-i}}$\n",
"- predict next word using standard NN architecture with tanh (hidden layer) resp. softmax (output layer) activation functions \n",
"- train network to maximize log likelihood $L(θ)=\\sum_t{log P(w_t|w_{t−n+1}, ..., w_{t−1}})$ using stochastic gradient descent "
]
},
{
"cell_type": "markdown",
"metadata": {
"nbpresent": {
"id": "52c70350-0896-498f-b322-8d4880a15d0c"
},
"slideshow": {
"slide_type": "slide"
}
},
"source": [
"## Word embeddings: word2vec\n"
]
},
{
"cell_type": "markdown",
"metadata": {
"nbpresent": {
"id": "8908acc8-32b8-43a5-91c3-c90120ac2c00"
},
"slideshow": {
"slide_type": "fragment"
}
},
"source": [
"\n",
"Mikolov et al (2013a). Efficient estimation of word representations in vector space. arXiv:1301.3781.\n",
"\n",
"- Continuous Bag of Words (CBOW)\n",
"- Skip-Gram"
]
},
{
"cell_type": "markdown",
"metadata": {
"nbpresent": {
"id": "a00e78a9-dcac-492c-82f9-4512d9f3cc51"
},
"slideshow": {
"slide_type": "slide"
}
},
"source": [
"## Continuous Bag of Words "
]
},
{
"cell_type": "markdown",
"metadata": {
"collapsed": true,
"nbpresent": {
"id": "b2c44061-460d-4392-82cd-8984d3045761"
},
"slideshow": {
"slide_type": "fragment"
}
},
"source": [
"\n",
"\n",
" \n",
"\n",
"\n",
" 
\n",
" Source: Mikolov et al. 2013, Efficient estimation of word representations in vector space. arXiv:1301.3781.\n",
""
]
},
{
"cell_type": "markdown",
"metadata": {
"nbpresent": {
"id": "4674031e-ae69-451b-ba6a-22ae52d20d61"
},
"slideshow": {
"slide_type": "slide"
}
},
"source": [
"## Skip-gram"
]
},
{
"cell_type": "markdown",
"metadata": {
"collapsed": true,
"nbpresent": {
"id": "52479713-4d93-43f3-9c48-bbc0442c9ecd"
},
"slideshow": {
"slide_type": "fragment"
}
},
"source": [
"\n",
"\n",
" \n",
"\n",
"\n",
" 
\n",
" Source: Mikolov et al. 2013, Efficient estimation of word representations in vector space. arXiv:1301.3781.\n",
""
]
},
{
"cell_type": "markdown",
"metadata": {
"nbpresent": {
"id": "ab78016d-76f3-4e42-bd0a-ba648e59448e"
},
"slideshow": {
"slide_type": "slide"
}
},
"source": [
"## Semantic & syntactic relationships\n"
]
},
{
"cell_type": "markdown",
"metadata": {
"collapsed": true,
"nbpresent": {
"id": "fd14b49b-be1b-4bc9-b46f-42304cedfdf9"
},
"slideshow": {
"slide_type": "fragment"
}
},
"source": [
"\n",
" \n",
"\n",
"\n",
" 
\n",
" Source: Mikolov et al. 2013, Efficient estimation of word representations in vector space. arXiv:1301.3781.\n",
""
]
},
{
"cell_type": "markdown",
"metadata": {
"nbpresent": {
"id": "98d2a2b1-f4c5-4af1-b6bd-ec842a2a73d2"
},
"slideshow": {
"slide_type": "slide"
}
},
"source": [
"## word2vec visualizations"
]
},
{
"cell_type": "markdown",
"metadata": {
"nbpresent": {
"id": "891db879-3f19-4629-8e7b-a380029adc8c"
},
"slideshow": {
"slide_type": "fragment"
}
},
"source": [
"\n",
"\n",
" \n",
"\n",
"\n",
" 
\n",
" Source: https://www.tensorflow.org/versions/r0.11/tutorials/word2vec/index.html\n",
""
]
},
{
"cell_type": "markdown",
"metadata": {
"nbpresent": {
"id": "24950d94-37dc-4fc1-8677-b4452159ecdd"
},
"slideshow": {
"slide_type": "slide"
}
},
"source": [
"# Recurrent neural networks (RNNs)"
]
},
{
"cell_type": "markdown",
"metadata": {
"nbpresent": {
"id": "b88358a4-c469-4a37-9fc3-3f1b685fac3e"
},
"slideshow": {
"slide_type": "slide"
}
},
"source": [
"## Why have recursion ?"
]
},
{
"cell_type": "markdown",
"metadata": {
"nbpresent": {
"id": "e961b635-0d28-4ba7-a7a5-ded381a20eb8"
},
"slideshow": {
"slide_type": "fragment"
}
},
"source": [
"\n",
"\n",
"- cannot process sequential data with \"normal\" feedforward networks\n",
"- in NLP, the n-gram approach cannot handle long-term relationships\n",
"\n",
"> Jane walked into the room. John walked in too. It was late in the\n",
"day, and everyone was walking home after a long day at work. Jane said hi to ___ \n",
"\n",
"([Stanford CS 224D Deep Learning for NLP Lecture Notes](http://cs224d.stanford.edu/lecture_notes/LectureNotes4.pdf))"
]
},
{
"cell_type": "markdown",
"metadata": {
"collapsed": true,
"nbpresent": {
"id": "2b7349aa-dd61-4fac-a82d-f02021436693"
},
"slideshow": {
"slide_type": "slide"
}
},
"source": [
"## Two representations of RNNs"
]
},
{
"cell_type": "markdown",
"metadata": {
"collapsed": true,
"nbpresent": {
"id": "d65875d4-f608-4a3d-a033-6af98aa2e416"
},
"slideshow": {
"slide_type": "fragment"
}
},
"source": [
"\n",
"\n",
" \n",
"\n",
"\n",
" 
\n",
" Source: Goodfellow et al. 2016, Deep Learning\n",
""
]
},
{
"cell_type": "markdown",
"metadata": {
"nbpresent": {
"id": "05b5e5e1-489d-4f22-b4c2-dd372c9e6fdf"
},
"slideshow": {
"slide_type": "slide"
}
},
"source": [
"## The recursion: example code"
]
},
{
"cell_type": "markdown",
"metadata": {
"nbpresent": {
"id": "214e2d6f-e81e-4c4f-8a8b-2463798e55bc"
},
"slideshow": {
"slide_type": "fragment"
}
},
"source": [
"\n",
"\n",
"```\n",
"def rnn_cell(rnn_input, state):\n",
" with tf.variable_scope('rnn_cell', reuse=True):\n",
" W = tf.get_variable('W', [num_classes + state_size, state_size])\n",
" b = tf.get_variable('b', [state_size], initializer=tf.constant_initializer(0.0))\n",
" return tf.tanh(tf.matmul(tf.concat(1, [rnn_input, state]), W) + b)\n",
"\n",
"state = init_state\n",
"rnn_outputs = []\n",
"for rnn_input in rnn_inputs:\n",
" state = rnn_cell(rnn_input, state) \n",
" rnn_outputs.append(state)\n",
"final_state = rnn_outputs[-1]\n",
"```\n",
"from: http://r2rt.com/recurrent-neural-networks-in-tensorflow-i.html\n"
]
},
{
"cell_type": "markdown",
"metadata": {
"collapsed": true,
"nbpresent": {
"id": "eda9c15c-5d3a-4d10-85c1-4718d8bd0954"
},
"slideshow": {
"slide_type": "slide"
}
},
"source": [
"## RNNs in practice: The need to forget"
]
},
{
"cell_type": "markdown",
"metadata": {
"nbpresent": {
"id": "3791e96a-cbb6-40d6-ba12-ed820eea8c66"
},
"slideshow": {
"slide_type": "slide"
}
},
"source": [
"## Gated Recurrent Units (GRUs)"
]
},
{
"cell_type": "markdown",
"metadata": {
"nbpresent": {
"id": "c62afb25-272e-4263-bfd2-3cd27916e73f"
},
"slideshow": {
"slide_type": "fragment"
}
},
"source": [
"\n",
"\n",
" \n",
"\n",
"\n",
" 
\n",
" Source: Stanford CS 224D Deep Learning for NLP Lecture Notes\n",
""
]
},
{
"cell_type": "markdown",
"metadata": {
"nbpresent": {
"id": "698b56a8-e8b4-4e78-bdae-ebe1080aa9e8"
},
"slideshow": {
"slide_type": "slide"
}
},
"source": [
"## Long Short Term Memory (LSTM)"
]
},
{
"cell_type": "markdown",
"metadata": {
"nbpresent": {
"id": "06c7a259-df52-48dd-9f68-c00df0cec78f"
},
"slideshow": {
"slide_type": "fragment"
}
},
"source": [
"\n",
"\n",
" \n",
"\n",
"\n",
" 
\n",
" Source: Stanford CS 224D Deep Learning for NLP Lecture Notes\n",
""
]
},
{
"cell_type": "markdown",
"metadata": {
"nbpresent": {
"id": "a2ea6e73-816b-426d-8c40-4edc3c3ab894"
},
"slideshow": {
"slide_type": "slide"
}
},
"source": [
"## GRU vs. LSTM: code example (Tensorflow)\n",
"### Baseline: Simple RNN Cell"
]
},
{
"cell_type": "markdown",
"metadata": {
"nbpresent": {
"id": "4a5a9b7f-fbb0-4d7d-a0c8-94aaebab0116"
},
"slideshow": {
"slide_type": "fragment"
}
},
"source": [
"\n",
"\n",
" \n",
"\n",
"\n",
"```\n",
"class BasicRNNCell(RNNCell):\n",
" \"\"\"The most basic RNN cell.\"\"\"\n",
"\n",
" def __init__(self, num_units, input_size=None, activation=tanh):\n",
" self._num_units = num_units\n",
" self._activation = activation\n",
"\n",
" @property\n",
" def state_size(self):\n",
" return self._num_units\n",
"\n",
" def __call__(self, inputs, state, scope=None):\n",
" \"\"\"Most basic RNN: output = new_state = act(W * input + U * state + B).\"\"\"\n",
" with vs.variable_scope(scope or \"basic_rnn_cell\"):\n",
" output = self._activation(\n",
" _linear([inputs, state], self._num_units, True, scope=scope))\n",
"return output, output\n",
"```\n"
]
},
{
"cell_type": "markdown",
"metadata": {
"nbpresent": {
"id": "3079b635-a3ff-422d-888c-e648ebbf7e06"
},
"slideshow": {
"slide_type": "slide"
}
},
"source": [
"## GRU vs. LSTM: code example (Tensorflow)\n",
"### GRU Cell"
]
},
{
"cell_type": "markdown",
"metadata": {
"nbpresent": {
"id": "7061a7ed-534e-4fd7-9e20-53d3a7c8f508"
},
"slideshow": {
"slide_type": "fragment"
}
},
"source": [
"\n",
" \n",
"\n",
"\n",
"```\n",
"class GRUCell(RNNCell):\n",
" \"\"\"Gated Recurrent Unit cell (cf. http://arxiv.org/abs/1406.1078).\"\"\"\n",
"\n",
" def __call__(self, inputs, state, scope=None):\n",
" \"\"\"Gated recurrent unit (GRU) with nunits cells.\"\"\"\n",
" with vs.variable_scope(scope or \"gru_cell\"):\n",
" with vs.variable_scope(\"gates\"): # Reset gate and update gate.\n",
" # We start with bias of 1.0 to not reset and not update.\n",
" r, u = array_ops.split(value=_linear([inputs, state],\n",
" 2 * self._num_units,\n",
" True,\n",
" 1.0,\n",
" scope=scope),\n",
" num_or_size_splits=2,\n",
" axis=1)\n",
" r, u = sigmoid(r), sigmoid(u)\n",
" \n",
" with vs.variable_scope(\"candidate\"):\n",
" c = self._activation(_linear([inputs, r * state],\n",
" self._num_units,\n",
" True, scope=scope))\n",
" \n",
" new_h = u * state + (1 - u) * c\n",
" \n",
"return new_h, new_h\n",
"```\n"
]
},
{
"cell_type": "markdown",
"metadata": {
"nbpresent": {
"id": "7094e510-13ac-4e3d-8cbf-fefcc81e6d41"
},
"slideshow": {
"slide_type": "slide"
}
},
"source": [
"## GRU vs. LSTM: code example (Tensorflow)\n",
"### LSTM Cell"
]
},
{
"cell_type": "markdown",
"metadata": {
"nbpresent": {
"id": "e7289eea-badb-4549-97f5-36404ff9b061"
},
"slideshow": {
"slide_type": "fragment"
}
},
"source": [
"\n",
" \n",
"\n",
"```\n",
"class BasicLSTMCell(RNNCell):\n",
"\n",
" def __call__(self, inputs, state, scope=None):\n",
" with vs.variable_scope(scope or \"basic_lstm_cell\"):\n",
" \n",
" c, h = array_ops.split(1, 2, state)\n",
" concat = _linear([inputs, h], 4 * self._num_units, True, scope=scope)\n",
"\n",
" # i = input_gate, j = new_input, f = forget_gate, o = output_gate\n",
" i, j, f, o = array_ops.split(1, 4, concat)\n",
"\n",
" new_c = (c * sigmoid(f + self._forget_bias) + sigmoid(i) * self._activation(j))\n",
" new_h = self._activation(new_c) * sigmoid(o)\n",
" \n",
" new_state = array_ops.concat_v2([new_c, new_h], 1)\n",
" return new_h, new_state\n",
"\n",
"```\n"
]
},
{
"cell_type": "markdown",
"metadata": {
"nbpresent": {
"id": "4ca25d34-0eb3-4303-99c7-1d3d613fff15"
},
"slideshow": {
"slide_type": "slide"
}
},
"source": [
"## Mapping sequences to sequences: seq2seq"
]
},
{
"cell_type": "markdown",
"metadata": {
"nbpresent": {
"id": "fc2c2684-6874-4c04-9aa0-900b6790e453"
},
"slideshow": {
"slide_type": "fragment"
}
},
"source": [
"\n",
"\n",
"\n",
" \n",
"\n",
"\n",
" 
\n",
" Source: Tensorflow seq2seq tutorial\n",
"\n",
"\n",
"- first RNN encodes the input, second decodes the output\n",
"- applications: e.g., machine translation - though basically, all sequence-to-sequence translation!"
]
},
{
"cell_type": "markdown",
"metadata": {
"nbpresent": {
"id": "b392e02c-8b5d-458b-8102-5ddc2171f885"
},
"slideshow": {
"slide_type": "slide"
}
},
"source": [
"## Combining modes/models example: Images and Descriptions"
]
},
{
"cell_type": "markdown",
"metadata": {
"nbpresent": {
"id": "ccc7a759-5856-4315-8acf-b89a76b496ab"
},
"slideshow": {
"slide_type": "fragment"
}
},
"source": [
"\n",
"\n",
"- [Andrej Karpathy, Li Fei-Fei: Deep Visual-Semantic Alignments for Generating Image Descriptions](http://cs.stanford.edu/people/karpathy/cvpr2015.pdf)\n",
"- combining CNNs, bidirectional RNNs, and multimodal embeddings\n",
"- [Demo](http://cs.stanford.edu/people/karpathy/deepimagesent/rankingdemo/)\n",
"\n",
"\n",
" 
\n",
" Source: Deep Visual-Semantic Alignments for Generating Image Descriptions\n",
"\n"
]
},
{
"cell_type": "markdown",
"metadata": {
"nbpresent": {
"id": "5ed884bf-59a3-4b1b-864f-c1b238a54709"
},
"slideshow": {
"slide_type": "slide"
}
},
"source": [
"# Tensorflow Demo: Generating text"
]
},
{
"cell_type": "markdown",
"metadata": {
"nbpresent": {
"id": "cc51a728-16c4-4970-a503-3ee8647f9a4d"
},
"slideshow": {
"slide_type": "slide"
}
},
"source": [
"## What is TensorFlow?"
]
},
{
"cell_type": "markdown",
"metadata": {
"nbpresent": {
"id": "593daf72-4dbe-471b-89bb-2382c2da473d"
},
"slideshow": {
"slide_type": "fragment"
}
},
"source": [
">\"If you can express your computation as a data flow graph, you can use TensorFlow.\"\n",
"\n",
"\n",
" 
\n",
" Source: www.tensorflow.org\n",
"\n",
"\n",
"\n",
"\n",
"- represent computations as graphs\n",
"- nodes are operations\n",
"- edges are _Tensors_ (multidimensional matrices) input to/output from operations\n",
"- to make anything happen, execute the graph in a _Session_\n",
"- a _Session_ places and runs a graph on a _Device_ (GPU, CPU)"
]
},
{
"cell_type": "markdown",
"metadata": {
"nbpresent": {
"id": "bdf27d31-2e36-4bb3-987a-23eca68c6377"
},
"slideshow": {
"slide_type": "slide"
}
},
"source": [
"# Basic TensorFlow workflow\n",
"\n",
"[demo](tf_workflow.ipynb)\n"
]
},
{
"cell_type": "markdown",
"metadata": {
"collapsed": true,
"nbpresent": {
"id": "ce398af3-13a5-4d65-9bef-f14ab7acb3fa"
},
"slideshow": {
"slide_type": "slide"
}
},
"source": [
"# Let's generate some text! \n",
"\n",
"char-rnn demo\n",
"\n",
"(based on https://github.com/sherjilozair/char-rnn-tensorflow)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Questions? Thank you!"
]
}
],
"metadata": {
"anaconda-cloud": {},
"celltoolbar": "Slideshow",
"kernelspec": {
"display_name": "Python [conda env:tensorflow2]",
"language": "python",
"name": "conda-env-tensorflow2-py"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.5.2"
},
"nbpresent": {
"slides": {
"02eaa597-aed6-435d-a4b7-0668b299001b": {
"id": "02eaa597-aed6-435d-a4b7-0668b299001b",
"prev": "802302f3-5375-44de-b2a8-3ef8d6d037ca",
"regions": {
"3cd1909e-9073-4aa5-a562-a69c802bfdbf": {
"attrs": {
"height": 0.8,
"width": 0.8,
"x": 0.1,
"y": 0.1
},
"content": {
"cell": "d67fc7da-4808-436d-baa4-1531a2b4238e",
"part": "whole"
},
"id": "3cd1909e-9073-4aa5-a562-a69c802bfdbf"
},
"bdcb4ade-1094-4361-b925-92faab363805": {
"attrs": {
"height": 0.4,
"width": 0.8,
"x": 0.1,
"y": 0.5
},
"content": {
"cell": "91e11f87-8c8c-4f73-aff2-ebf6c28b8e00",
"part": "whole"
},
"id": "bdcb4ade-1094-4361-b925-92faab363805"
}
}
},
"05eedcfe-475b-4862-b777-4fa153d9849a": {
"id": "05eedcfe-475b-4862-b777-4fa153d9849a",
"prev": "3ea92b9f-4927-4923-823b-c10a06f4e803",
"regions": {
"592ec91b-a51e-485a-8dc0-25d8a061d871": {
"attrs": {
"height": 0.4,
"width": 0.8,
"x": 0.1,
"y": 0.5
},
"content": {
"cell": "e961b635-0d28-4ba7-a7a5-ded381a20eb8",
"part": "whole"
},
"id": "592ec91b-a51e-485a-8dc0-25d8a061d871"
},
"81df3e02-43b0-4cf1-8aa2-9e76979cb96a": {
"attrs": {
"height": 0.8,
"width": 0.8,
"x": 0.1,
"y": 0.1
},
"content": {
"cell": "b88358a4-c469-4a37-9fc3-3f1b685fac3e",
"part": "whole"
},
"id": "81df3e02-43b0-4cf1-8aa2-9e76979cb96a"
}
}
},
"08db0978-d376-4a23-91a7-f5718f0f1c51": {
"id": "08db0978-d376-4a23-91a7-f5718f0f1c51",
"prev": "5f515d2d-36d3-4b08-b7e9-293e1a1ff3c9",
"regions": {
"a10a19d3-94ec-4f25-95c5-55a8c3db33d0": {
"attrs": {
"height": 0.4,
"width": 0.8,
"x": 0.1,
"y": 0.5
},
"content": {
"cell": "ae5ee5ff-fdc5-4bd6-aa62-7963297ae98f",
"part": "whole"
},
"id": "a10a19d3-94ec-4f25-95c5-55a8c3db33d0"
},
"e5630af5-dee1-4733-8281-d9ae4ffde1d4": {
"attrs": {
"height": 0.8,
"width": 0.8,
"x": 0.1,
"y": 0.1
},
"content": {
"cell": "b82b2474-bb2d-4b57-86d4-061ba495b8cd",
"part": "whole"
},
"id": "e5630af5-dee1-4733-8281-d9ae4ffde1d4"
}
}
},
"0abb7a8d-409e-41ba-9bf4-7c9e012960c5": {
"id": "0abb7a8d-409e-41ba-9bf4-7c9e012960c5",
"prev": "46048207-14fe-4087-bbcb-520952af1585",
"regions": {
"1cb49bbf-69ce-41a9-a08a-9c44c368f872": {
"attrs": {
"height": 0.4,
"width": 0.8,
"x": 0.1,
"y": 0.5
},
"content": {
"cell": "5bd46de7-2344-42a5-82ab-4e5ca519c7d6",
"part": "whole"
},
"id": "1cb49bbf-69ce-41a9-a08a-9c44c368f872"
},
"8936ffc0-089c-41ad-b4f8-2cf94c69627c": {
"attrs": {
"height": 0.8,
"width": 0.8,
"x": 0.1,
"y": 0.1
},
"content": {
"cell": "483671b9-13dd-4991-842c-609cbf51009b",
"part": "whole"
},
"id": "8936ffc0-089c-41ad-b4f8-2cf94c69627c"
}
}
},
"0acf341c-d2cf-4590-b9cd-b44130bb5309": {
"id": "0acf341c-d2cf-4590-b9cd-b44130bb5309",
"prev": "62842951-dcae-4b6d-8ca0-e93057d27a1b",
"regions": {
"47d1fa00-caef-4899-b7ae-37f9550aedc3": {
"attrs": {
"height": 0.8,
"width": 0.8,
"x": 0.1,
"y": 0.1
},
"content": {
"cell": "c55ed8a3-4b84-4d63-b5fd-51a723ee0c0b",
"part": "whole"
},
"id": "47d1fa00-caef-4899-b7ae-37f9550aedc3"
},
"637b3bd8-f5ef-4f63-b878-b1b2920676aa": {
"attrs": {
"height": 0.4,
"width": 0.8,
"x": 0.1,
"y": 0.5
},
"content": {
"cell": "a22179af-1c53-45a4-a358-c5d26dd968a3",
"part": "whole"
},
"id": "637b3bd8-f5ef-4f63-b878-b1b2920676aa"
}
}
},
"0b177343-6e12-4a66-ad41-c77db7354d28": {
"id": "0b177343-6e12-4a66-ad41-c77db7354d28",
"prev": "ab3a3b2a-f646-4972-9385-8fa39ec860aa",
"regions": {
"aa9caf6b-439e-4124-ab5b-a49c9186f00f": {
"attrs": {
"height": 0.8,
"width": 0.8,
"x": 0.1,
"y": 0.1
},
"content": {
"cell": "6d84a892-2d33-43c2-b34b-c4d5b104ec45",
"part": "whole"
},
"id": "aa9caf6b-439e-4124-ab5b-a49c9186f00f"
}
}
},
"0eba2662-c9fd-406d-ae98-1477c861e9a4": {
"id": "0eba2662-c9fd-406d-ae98-1477c861e9a4",
"prev": "ba6a43ee-7ff8-4dff-a537-3a96eb12b83b",
"regions": {
"673bc708-349c-4b29-8c89-84c44f70280e": {
"attrs": {
"height": 0.4,
"width": 0.8,
"x": 0.1,
"y": 0.5
},
"content": {
"cell": "5bdc92b9-1c76-428a-a122-81905b12f335",
"part": "whole"
},
"id": "673bc708-349c-4b29-8c89-84c44f70280e"
},
"f440f92b-ecb8-464b-9bbe-c79a5273c4cf": {
"attrs": {
"height": 0.8,
"width": 0.8,
"x": 0.1,
"y": 0.1
},
"content": {
"cell": "8d29ce74-5e12-4ee4-ad42-efb90d3024e1",
"part": "whole"
},
"id": "f440f92b-ecb8-464b-9bbe-c79a5273c4cf"
}
}
},
"11aca522-914f-4376-a844-1fe562a18cc9": {
"id": "11aca522-914f-4376-a844-1fe562a18cc9",
"prev": "3058b105-1a3b-4ac8-a399-2387f417aa90",
"regions": {
"84ec6811-cbc6-4d0f-98fa-2940287b998c": {
"attrs": {
"height": 0.4,
"width": 0.8,
"x": 0.1,
"y": 0.5
},
"content": {
"cell": "d62f7d1a-2719-4460-aae4-403305a239de",
"part": "whole"
},
"id": "84ec6811-cbc6-4d0f-98fa-2940287b998c"
},
"8a142e7d-cd28-44e5-af7d-0c568a0e8ced": {
"attrs": {
"height": 0.8,
"width": 0.8,
"x": 0.1,
"y": 0.1
},
"content": {
"cell": "2972f29b-6150-4059-addd-cf26842a30c3",
"part": "whole"
},
"id": "8a142e7d-cd28-44e5-af7d-0c568a0e8ced"
}
}
},
"16bedbd6-e898-48a2-afa2-cf71d68b0e83": {
"id": "16bedbd6-e898-48a2-afa2-cf71d68b0e83",
"prev": "11aca522-914f-4376-a844-1fe562a18cc9",
"regions": {
"1a6fd1cd-9d01-43f2-a4ff-10ee58867d3d": {
"attrs": {
"height": 0.4,
"width": 0.8,
"x": 0.1,
"y": 0.5
},
"content": {
"cell": "840c726c-e3f8-40a2-ada8-7fb351a5684f",
"part": "whole"
},
"id": "1a6fd1cd-9d01-43f2-a4ff-10ee58867d3d"
},
"ecae84d4-9fa1-4341-9791-0e114550bff8": {
"attrs": {
"height": 0.8,
"width": 0.8,
"x": 0.1,
"y": 0.1
},
"content": {
"cell": "29fc973b-e69f-49e8-8f23-7e8d4aa70664",
"part": "whole"
},
"id": "ecae84d4-9fa1-4341-9791-0e114550bff8"
}
}
},
"1d353827-f707-4d86-b1db-740fdd771573": {
"id": "1d353827-f707-4d86-b1db-740fdd771573",
"prev": "8e6ddfe2-8fe0-48c1-8fa1-9614d34faf77",
"regions": {
"515be511-bbb5-493b-abe8-cad1f9381747": {
"attrs": {
"height": 0.8,
"width": 0.8,
"x": 0.1,
"y": 0.1
},
"content": {
"cell": "c7a247da-28b0-4a68-8035-504fa4a342ee",
"part": "whole"
},
"id": "515be511-bbb5-493b-abe8-cad1f9381747"
}
}
},
"1e52131c-cfee-47c4-9774-d5503ed0c561": {
"id": "1e52131c-cfee-47c4-9774-d5503ed0c561",
"prev": "2e7c9d1b-440b-4bce-8958-af9f27f397cb",
"regions": {
"b18b4579-38d5-47f7-b610-d74805730c34": {
"attrs": {
"height": 0.4,
"width": 0.8,
"x": 0.1,
"y": 0.5
},
"content": {
"cell": "de61fb12-9bfb-4e13-9ebb-b33dae688737",
"part": "whole"
},
"id": "b18b4579-38d5-47f7-b610-d74805730c34"
},
"e3bbf857-db2b-41cb-b311-071d5f71b83d": {
"attrs": {
"height": 0.8,
"width": 0.8,
"x": 0.1,
"y": 0.1
},
"content": {
"cell": "3a07640f-61ed-409e-b89f-c26485b8bea8",
"part": "whole"
},
"id": "e3bbf857-db2b-41cb-b311-071d5f71b83d"
}
}
},
"247887c7-574b-4fc3-bc25-e0be67e579b8": {
"id": "247887c7-574b-4fc3-bc25-e0be67e579b8",
"prev": "f89b6695-6ec6-4edb-b345-7fd691743aba",
"regions": {
"25e07397-3cb1-439d-a050-5852ba74efc6": {
"attrs": {
"height": 0.4,
"width": 0.8,
"x": 0.1,
"y": 0.5
},
"content": {
"cell": "c35e7ccc-ce28-448b-96cd-99b8e7e13101",
"part": "whole"
},
"id": "25e07397-3cb1-439d-a050-5852ba74efc6"
},
"9c0406df-921c-43ea-8622-888c1b94ec9c": {
"attrs": {
"height": 0.8,
"width": 0.8,
"x": 0.1,
"y": 0.1
},
"content": {
"cell": "58005fd2-d829-4e88-bda6-369db29d0f23",
"part": "whole"
},
"id": "9c0406df-921c-43ea-8622-888c1b94ec9c"
}
}
},
"2e7c9d1b-440b-4bce-8958-af9f27f397cb": {
"id": "2e7c9d1b-440b-4bce-8958-af9f27f397cb",
"prev": "08db0978-d376-4a23-91a7-f5718f0f1c51",
"regions": {
"4cea934e-cff6-4d82-82ec-cadade8e4e4d": {
"attrs": {
"height": 0.4,
"width": 0.8,
"x": 0.1,
"y": 0.5
},
"content": {
"cell": "6cc4b015-3b21-4ffb-aece-bfec06b2b791",
"part": "whole"
},
"id": "4cea934e-cff6-4d82-82ec-cadade8e4e4d"
},
"92e2c75d-973c-4b5e-9c2a-bdb576ab7379": {
"attrs": {
"height": 0.8,
"width": 0.8,
"x": 0.1,
"y": 0.1
},
"content": {
"cell": "6845b8a7-2f11-489b-aec6-88492ffd80bb",
"part": "whole"
},
"id": "92e2c75d-973c-4b5e-9c2a-bdb576ab7379"
}
}
},
"3058b105-1a3b-4ac8-a399-2387f417aa90": {
"id": "3058b105-1a3b-4ac8-a399-2387f417aa90",
"prev": "0acf341c-d2cf-4590-b9cd-b44130bb5309",
"regions": {
"c3660711-b9bf-49fb-905d-f1f91d5aaa0f": {
"attrs": {
"height": 0.8,
"width": 0.8,
"x": 0.1,
"y": 0.1
},
"content": {
"cell": "1b854720-e1b0-49db-9332-4e7098f2e61a",
"part": "whole"
},
"id": "c3660711-b9bf-49fb-905d-f1f91d5aaa0f"
},
"e3f642b6-5611-4a83-9a38-69b781bb210c": {
"attrs": {
"height": 0.4,
"width": 0.8,
"x": 0.1,
"y": 0.5
},
"content": {
"cell": "abe24c13-7a29-470e-8733-abc0e025b419",
"part": "whole"
},
"id": "e3f642b6-5611-4a83-9a38-69b781bb210c"
}
}
},
"32e95611-4585-44dc-a22d-934e38f9c9ae": {
"id": "32e95611-4585-44dc-a22d-934e38f9c9ae",
"prev": "8f0a1e4b-795e-47d2-a5b3-99c117906bdc",
"regions": {
"10432e54-05c0-4dc5-a286-a525be9ac37d": {
"attrs": {
"height": 0.4,
"width": 0.8,
"x": 0.1,
"y": 0.5
},
"content": {
"cell": "48b0d007-c995-45ec-8d52-71cf599d8e4c",
"part": "whole"
},
"id": "10432e54-05c0-4dc5-a286-a525be9ac37d"
},
"7575fe50-c7ce-41e4-b715-7f5134c44d2e": {
"attrs": {
"height": 0.8,
"width": 0.8,
"x": 0.1,
"y": 0.1
},
"content": {
"cell": "0280fc7d-0248-4c17-b161-5eadc14bbf0b",
"part": "whole"
},
"id": "7575fe50-c7ce-41e4-b715-7f5134c44d2e"
}
}
},
"3e2d4143-d5f8-4612-aada-8308e9bdf4fb": {
"id": "3e2d4143-d5f8-4612-aada-8308e9bdf4fb",
"prev": "a02815f7-8a80-434d-bab7-43146bc4ce66",
"regions": {
"97b5732c-e152-452c-bf0a-edc346576988": {
"attrs": {
"height": 0.8,
"width": 0.8,
"x": 0.1,
"y": 0.1
},
"content": {
"cell": "c5161136-7f9f-4a01-b48d-eb18a86d3cb3",
"part": "whole"
},
"id": "97b5732c-e152-452c-bf0a-edc346576988"
}
}
},
"3ea92b9f-4927-4923-823b-c10a06f4e803": {
"id": "3ea92b9f-4927-4923-823b-c10a06f4e803",
"prev": "7b920a90-bf3f-4a0c-b220-1e5d609f713c",
"regions": {
"4d897e2f-6a70-40d0-9028-e2a7c78828bc": {
"attrs": {
"height": 0.8,
"width": 0.8,
"x": 0.1,
"y": 0.1
},
"content": {
"cell": "24950d94-37dc-4fc1-8677-b4452159ecdd",
"part": "whole"
},
"id": "4d897e2f-6a70-40d0-9028-e2a7c78828bc"
}
}
},
"3f27bae0-13b3-4208-82b1-addf0aef954d": {
"id": "3f27bae0-13b3-4208-82b1-addf0aef954d",
"prev": "79a12673-d632-4173-a0d7-dea7a7c0049c",
"regions": {
"4e441375-d8ed-4873-b090-c73c91a4b1d4": {
"attrs": {
"height": 0.8,
"width": 0.8,
"x": 0.1,
"y": 0.1
},
"content": {
"cell": "bdf27d31-2e36-4bb3-987a-23eca68c6377",
"part": "whole"
},
"id": "4e441375-d8ed-4873-b090-c73c91a4b1d4"
}
}
},
"407a43f7-0f13-4928-88c8-5b19055dcfc3": {
"id": "407a43f7-0f13-4928-88c8-5b19055dcfc3",
"prev": "8414982b-4bcb-44e9-94d3-cffe41a748e0",
"regions": {
"0d8c1c20-3a9e-4c15-a9f2-757f3de4b76b": {
"attrs": {
"height": 0.4,
"width": 0.8,
"x": 0.1,
"y": 0.5
},
"content": {
"cell": "803546a8-dc4a-4150-ad12-cb597ba5adc0",
"part": "whole"
},
"id": "0d8c1c20-3a9e-4c15-a9f2-757f3de4b76b"
},
"2cbe2df8-dd50-4038-9261-c20a6d7335db": {
"attrs": {
"height": 0.8,
"width": 0.8,
"x": 0.1,
"y": 0.1
},
"content": {
"cell": "0a6faa87-c1d4-4272-8da1-0a68650c7dc3",
"part": "whole"
},
"id": "2cbe2df8-dd50-4038-9261-c20a6d7335db"
}
}
},
"42e3c596-98c1-4abe-941a-0f9ccf6ac609": {
"id": "42e3c596-98c1-4abe-941a-0f9ccf6ac609",
"prev": "9440249f-5c6f-48b5-b2e1-e75a42536ece",
"regions": {
"14d74d7b-4047-43f3-9800-29d6d71c7bb3": {
"attrs": {
"height": 0.4,
"width": 0.8,
"x": 0.1,
"y": 0.5
},
"content": {
"cell": "747cfe3c-cdcd-4a50-9538-067256fa8cbe",
"part": "whole"
},
"id": "14d74d7b-4047-43f3-9800-29d6d71c7bb3"
},
"81a5f4cc-e335-45f7-86ae-20e7ab5043dd": {
"attrs": {
"height": 0.8,
"width": 0.8,
"x": 0.1,
"y": 0.1
},
"content": {
"cell": "d98313bd-f40e-42c3-9506-6dfa5a1867db",
"part": "whole"
},
"id": "81a5f4cc-e335-45f7-86ae-20e7ab5043dd"
}
}
},
"443646d2-381d-4302-8c34-f336484d9b89": {
"id": "443646d2-381d-4302-8c34-f336484d9b89",
"prev": "3e2d4143-d5f8-4612-aada-8308e9bdf4fb",
"regions": {
"c4b79a0c-2d19-404b-b5ec-84ceab8be00e": {
"attrs": {
"height": 0.8,
"width": 0.8,
"x": 0.1,
"y": 0.1
},
"content": {
"cell": "4fd82baf-07f9-4f85-b7f2-7b41357ed8d3",
"part": "whole"
},
"id": "c4b79a0c-2d19-404b-b5ec-84ceab8be00e"
},
"e65facb6-5878-422c-b9b9-d7228095ccd0": {
"attrs": {
"height": 0.4,
"width": 0.8,
"x": 0.1,
"y": 0.5
},
"content": {
"cell": "a3f33fe9-3726-428e-ab5f-d1c7ca9c37fb",
"part": "whole"
},
"id": "e65facb6-5878-422c-b9b9-d7228095ccd0"
}
}
},
"452e80ee-c961-440f-bde6-e683956d69ef": {
"id": "452e80ee-c961-440f-bde6-e683956d69ef",
"prev": "8ae6c40a-d85a-437d-aee5-967da65cea00",
"regions": {
"5bbb8b79-e05d-4750-8ef8-c9430bcf2a70": {
"attrs": {
"height": 0.4,
"width": 0.8,
"x": 0.1,
"y": 0.5
},
"content": {
"cell": "3e3eefa3-61ec-423e-9068-bd65d217cf3d",
"part": "whole"
},
"id": "5bbb8b79-e05d-4750-8ef8-c9430bcf2a70"
},
"9ba51a2d-d8c5-4b01-82df-c1b6fb7fc14c": {
"attrs": {
"height": 0.8,
"width": 0.8,
"x": 0.1,
"y": 0.1
},
"content": {
"cell": "8b75395c-ac9d-480d-a8b3-afa488018b3d",
"part": "whole"
},
"id": "9ba51a2d-d8c5-4b01-82df-c1b6fb7fc14c"
}
}
},
"46048207-14fe-4087-bbcb-520952af1585": {
"id": "46048207-14fe-4087-bbcb-520952af1585",
"prev": null,
"regions": {
"6990da83-e3d9-46d6-9042-063e7a2fc164": {
"attrs": {
"height": 0.8,
"width": 0.8,
"x": 0.1,
"y": 0.1
},
"content": {
"cell": "2c5113e2-a6cb-4cbb-b28b-490e8818b113",
"part": "whole"
},
"id": "6990da83-e3d9-46d6-9042-063e7a2fc164"
}
}
},
"4ca922b9-2bd0-44d0-9078-78fa564fadda": {
"id": "4ca922b9-2bd0-44d0-9078-78fa564fadda",
"prev": "cf02493a-823d-4df3-9865-8f4c0955bae3",
"regions": {
"1b725896-1adb-444b-8a90-c2922b77da0d": {
"attrs": {
"height": 0.4,
"width": 0.8,
"x": 0.1,
"y": 0.5
},
"content": {
"cell": "fc2c2684-6874-4c04-9aa0-900b6790e453",
"part": "whole"
},
"id": "1b725896-1adb-444b-8a90-c2922b77da0d"
},
"a49077c1-bf27-410a-91a7-b88a72d1e405": {
"attrs": {
"height": 0.8,
"width": 0.8,
"x": 0.1,
"y": 0.1
},
"content": {
"cell": "4ca25d34-0eb3-4303-99c7-1d3d613fff15",
"part": "whole"
},
"id": "a49077c1-bf27-410a-91a7-b88a72d1e405"
}
}
},
"4d001e63-ce4e-46a2-a137-7ebd7b232f83": {
"id": "4d001e63-ce4e-46a2-a137-7ebd7b232f83",
"prev": "f9fa2328-e23d-4584-826b-e1bf0a935d67",
"regions": {
"1c105e05-394a-401e-8abd-f3e40e2fade9": {
"attrs": {
"height": 0.8,
"width": 0.8,
"x": 0.1,
"y": 0.1
},
"content": {
"cell": "2599d4ec-ad01-48cf-8b71-a042868fee2e",
"part": "whole"
},
"id": "1c105e05-394a-401e-8abd-f3e40e2fade9"
},
"a532ef65-67b3-4d6f-9be8-90a0702b6f57": {
"attrs": {
"height": 0.4,
"width": 0.8,
"x": 0.1,
"y": 0.5
},
"content": {
"cell": "42bc0b4e-2d9d-4664-9284-b34ecf7b4980",
"part": "whole"
},
"id": "a532ef65-67b3-4d6f-9be8-90a0702b6f57"
}
}
},
"58f3634a-3f7b-4ce7-b429-1645c7de66c3": {
"id": "58f3634a-3f7b-4ce7-b429-1645c7de66c3",
"prev": "60a36f9f-4cde-45dc-baf3-82e85979eb02",
"regions": {
"1c3de161-89ea-41f6-a613-89f0cefab75c": {
"attrs": {
"height": 0.8,
"width": 0.8,
"x": 0.1,
"y": 0.1
},
"content": {
"cell": "9e05ef09-a476-41d8-9f71-83cd95e8c009",
"part": "whole"
},
"id": "1c3de161-89ea-41f6-a613-89f0cefab75c"
},
"58ba51a6-e47e-4f4a-9f0f-9a3122cde868": {
"attrs": {
"height": 0.4,
"width": 0.8,
"x": 0.1,
"y": 0.5
},
"content": {
"cell": "5ddc1f0e-0ede-4a0f-8153-d674d062e14b",
"part": "whole"
},
"id": "58ba51a6-e47e-4f4a-9f0f-9a3122cde868"
},
"5da4d51d-bb46-4dfa-98c8-8fb57bcbaeb4": {
"attrs": {
"height": 0.4,
"width": 0.8,
"x": 0.1,
"y": 0.5
},
"content": {
"cell": "64a4d022-dc47-494d-abf5-bec4c5f914aa",
"part": "whole"
},
"id": "5da4d51d-bb46-4dfa-98c8-8fb57bcbaeb4"
}
}
},
"591d6bea-4d9d-43be-840a-75f934127fd0": {
"id": "591d6bea-4d9d-43be-840a-75f934127fd0",
"prev": "daa50dee-37ef-4997-8293-f1dbd7cf68c4",
"regions": {
"da6bc66e-14f9-4f2a-8bec-6d07b60d3129": {
"attrs": {
"height": 0.8,
"width": 0.8,
"x": 0.1,
"y": 0.1
},
"content": {
"cell": "17b86f90-099e-41de-889e-c76865f1bfef",
"part": "whole"
},
"id": "da6bc66e-14f9-4f2a-8bec-6d07b60d3129"
},
"f356373e-796c-4673-9e1d-cb3d239bb8ea": {
"attrs": {
"height": 0.4,
"width": 0.8,
"x": 0.1,
"y": 0.5
},
"content": {
"cell": "694380ef-ccda-4f28-b8a8-6cf0ad519b5b",
"part": "whole"
},
"id": "f356373e-796c-4673-9e1d-cb3d239bb8ea"
}
}
},
"5a9c3c6d-e78d-46a5-9d26-d76753a4839b": {
"id": "5a9c3c6d-e78d-46a5-9d26-d76753a4839b",
"prev": "f43c3aec-5d2a-4e5d-b976-eb68819cbd41",
"regions": {
"a0254fda-251b-4658-adf6-ed1d31d01380": {
"attrs": {
"height": 0.4,
"width": 0.8,
"x": 0.1,
"y": 0.5
},
"content": {
"cell": "214e2d6f-e81e-4c4f-8a8b-2463798e55bc",
"part": "whole"
},
"id": "a0254fda-251b-4658-adf6-ed1d31d01380"
},
"b2013091-f3c2-4766-9435-cade2d3c0afc": {
"attrs": {
"height": 0.8,
"width": 0.8,
"x": 0.1,
"y": 0.1
},
"content": {
"cell": "05b5e5e1-489d-4f22-b4c2-dd372c9e6fdf",
"part": "whole"
},
"id": "b2013091-f3c2-4766-9435-cade2d3c0afc"
}
}
},
"5f515d2d-36d3-4b08-b7e9-293e1a1ff3c9": {
"id": "5f515d2d-36d3-4b08-b7e9-293e1a1ff3c9",
"prev": "16bedbd6-e898-48a2-afa2-cf71d68b0e83",
"regions": {
"6b587411-6470-4b72-ac0e-7421c3a1d954": {
"attrs": {
"height": 0.8,
"width": 0.8,
"x": 0.1,
"y": 0.1
},
"content": {
"cell": "fb4f44f5-38fe-4be5-bea8-2c43e8bb9324",
"part": "whole"
},
"id": "6b587411-6470-4b72-ac0e-7421c3a1d954"
},
"961e8aea-e333-441a-9222-2e5b51908421": {
"attrs": {
"height": 0.4,
"width": 0.8,
"x": 0.1,
"y": 0.5
},
"content": {
"cell": "210ea80f-c891-41cf-82b5-aba1ac61ab89",
"part": "whole"
},
"id": "961e8aea-e333-441a-9222-2e5b51908421"
}
}
},
"60a36f9f-4cde-45dc-baf3-82e85979eb02": {
"id": "60a36f9f-4cde-45dc-baf3-82e85979eb02",
"prev": "247887c7-574b-4fc3-bc25-e0be67e579b8",
"regions": {
"2538f65c-276e-45cb-b464-070570fb5033": {
"attrs": {
"height": 0.8,
"width": 0.8,
"x": 0.1,
"y": 0.1
},
"content": {
"cell": "b6015267-c8bc-4658-bc34-30a58385560b",
"part": "whole"
},
"id": "2538f65c-276e-45cb-b464-070570fb5033"
},
"be441b6b-6c68-4b6a-a6e6-ff427c06925a": {
"attrs": {
"height": 0.4,
"width": 0.8,
"x": 0.1,
"y": 0.5
},
"content": {
"cell": "552b7d6d-222f-419a-892b-0f0dd611261f",
"part": "whole"
},
"id": "be441b6b-6c68-4b6a-a6e6-ff427c06925a"
}
}
},
"62842951-dcae-4b6d-8ca0-e93057d27a1b": {
"id": "62842951-dcae-4b6d-8ca0-e93057d27a1b",
"prev": "443646d2-381d-4302-8c34-f336484d9b89",
"regions": {
"7aeee93d-a4ca-4780-bd9b-87c2bbd3a3f8": {
"attrs": {
"height": 0.8,
"width": 0.8,
"x": 0.1,
"y": 0.1
},
"content": {
"cell": "025d461b-ce92-448d-a6ce-bec06bbf35a6",
"part": "whole"
},
"id": "7aeee93d-a4ca-4780-bd9b-87c2bbd3a3f8"
}
}
},
"65bf7605-eb1d-4c9e-8b34-0c9775cf695b": {
"id": "65bf7605-eb1d-4c9e-8b34-0c9775cf695b",
"prev": "ff5dec66-40f4-4ae4-9170-2df11f95c422",
"regions": {
"bd4a28b7-41d5-40f4-ba92-a1c8ff547d50": {
"attrs": {
"height": 0.8,
"width": 0.8,
"x": 0.1,
"y": 0.1
},
"content": {
"cell": "5ed884bf-59a3-4b1b-864f-c1b238a54709",
"part": "whole"
},
"id": "bd4a28b7-41d5-40f4-ba92-a1c8ff547d50"
}
}
},
"65ca0680-a098-40ab-8a41-e8a753d7228d": {
"id": "65ca0680-a098-40ab-8a41-e8a753d7228d",
"prev": "d67b2752-be57-416d-9d30-d49a530cbaee",
"regions": {
"6c924b28-7f68-4c75-b8ba-5cbf3254a809": {
"attrs": {
"height": 0.4,
"width": 0.8,
"x": 0.1,
"y": 0.5
},
"content": {
"cell": "7061a7ed-534e-4fd7-9e20-53d3a7c8f508",
"part": "whole"
},
"id": "6c924b28-7f68-4c75-b8ba-5cbf3254a809"
},
"b8873332-6340-41af-ba79-a000140594ec": {
"attrs": {
"height": 0.8,
"width": 0.8,
"x": 0.1,
"y": 0.1
},
"content": {
"cell": "3079b635-a3ff-422d-888c-e648ebbf7e06",
"part": "whole"
},
"id": "b8873332-6340-41af-ba79-a000140594ec"
}
}
},
"698f2be8-8608-4051-ae18-072812614952": {
"id": "698f2be8-8608-4051-ae18-072812614952",
"prev": "bf981077-1f22-4084-ae49-2706477a5da9",
"regions": {
"9d18ba7f-dade-49c3-afc6-421380799b42": {
"attrs": {
"height": 0.8,
"width": 0.8,
"x": 0.1,
"y": 0.1
},
"content": {
"cell": "1caf128f-a511-4955-95bb-fd557b9b7739",
"part": "whole"
},
"id": "9d18ba7f-dade-49c3-afc6-421380799b42"
},
"bcc58285-c566-44ed-ae32-a1f0c28756fb": {
"attrs": {
"height": 0.4,
"width": 0.8,
"x": 0.1,
"y": 0.5
},
"content": {
"cell": "5b1ce3cd-b35d-49a7-8e3e-9e51f99152ca",
"part": "whole"
},
"id": "bcc58285-c566-44ed-ae32-a1f0c28756fb"
}
}
},
"78d614df-b65e-40c7-b0c4-d0025cf42d3c": {
"id": "78d614df-b65e-40c7-b0c4-d0025cf42d3c",
"prev": "ad96e441-8744-4044-a385-7fae743e1c2b",
"regions": {
"21bb49a1-7c61-4047-9427-069822b859bc": {
"attrs": {
"height": 0.4,
"width": 0.8,
"x": 0.1,
"y": 0.5
},
"content": {
"cell": "52479713-4d93-43f3-9c48-bbc0442c9ecd",
"part": "whole"
},
"id": "21bb49a1-7c61-4047-9427-069822b859bc"
},
"6ca6db99-c0c3-49a1-8cc1-4b764256f78b": {
"attrs": {
"height": 0.8,
"width": 0.8,
"x": 0.1,
"y": 0.1
},
"content": {
"cell": "4674031e-ae69-451b-ba6a-22ae52d20d61",
"part": "whole"
},
"id": "6ca6db99-c0c3-49a1-8cc1-4b764256f78b"
},
"809efed1-a11d-42b3-b0df-d12bc6e424b9": {
"attrs": {
"height": 0.4,
"width": 0.8,
"x": 0.1,
"y": 0.5
},
"content": {
"cell": "ab78016d-76f3-4e42-bd0a-ba648e59448e",
"part": "whole"
},
"id": "809efed1-a11d-42b3-b0df-d12bc6e424b9"
},
"b9c4b4e1-2383-4c63-bdb8-5fcf4a40c682": {
"attrs": {
"height": 0.4,
"width": 0.8,
"x": 0.1,
"y": 0.5
},
"content": {
"cell": "fd14b49b-be1b-4bc9-b46f-42304cedfdf9",
"part": "whole"
},
"id": "b9c4b4e1-2383-4c63-bdb8-5fcf4a40c682"
}
}
},
"79a12673-d632-4173-a0d7-dea7a7c0049c": {
"id": "79a12673-d632-4173-a0d7-dea7a7c0049c",
"prev": "65bf7605-eb1d-4c9e-8b34-0c9775cf695b",
"regions": {
"cdd5b86b-ee6e-4199-86d2-fc6468e4c13b": {
"attrs": {
"height": 0.4,
"width": 0.8,
"x": 0.1,
"y": 0.5
},
"content": {
"cell": "593daf72-4dbe-471b-89bb-2382c2da473d",
"part": "whole"
},
"id": "cdd5b86b-ee6e-4199-86d2-fc6468e4c13b"
},
"df4928fc-75b1-45d9-b0af-cad0d7d1ef58": {
"attrs": {
"height": 0.8,
"width": 0.8,
"x": 0.1,
"y": 0.1
},
"content": {
"cell": "cc51a728-16c4-4970-a503-3ee8647f9a4d",
"part": "whole"
},
"id": "df4928fc-75b1-45d9-b0af-cad0d7d1ef58"
}
}
},
"7b920a90-bf3f-4a0c-b220-1e5d609f713c": {
"id": "7b920a90-bf3f-4a0c-b220-1e5d609f713c",
"prev": "78d614df-b65e-40c7-b0c4-d0025cf42d3c",
"regions": {
"0b65078c-bbf4-4fa8-b690-8a4ed677be15": {
"attrs": {
"height": 0.8,
"width": 0.8,
"x": 0.1,
"y": 0.1
},
"content": {
"cell": "98d2a2b1-f4c5-4af1-b6bd-ec842a2a73d2",
"part": "whole"
},
"id": "0b65078c-bbf4-4fa8-b690-8a4ed677be15"
},
"9ede8124-5211-451b-bfe1-f8ea53916b29": {
"attrs": {
"height": 0.4,
"width": 0.8,
"x": 0.1,
"y": 0.5
},
"content": {
"cell": "891db879-3f19-4629-8e7b-a380029adc8c",
"part": "whole"
},
"id": "9ede8124-5211-451b-bfe1-f8ea53916b29"
}
}
},
"802302f3-5375-44de-b2a8-3ef8d6d037ca": {
"id": "802302f3-5375-44de-b2a8-3ef8d6d037ca",
"prev": "0abb7a8d-409e-41ba-9bf4-7c9e012960c5",
"regions": {
"209643bd-687f-4169-9fbf-1ae613746f88": {
"attrs": {
"height": 0.8,
"width": 0.8,
"x": 0.1,
"y": 0.1
},
"content": {
"cell": "84e21a78-4fd8-447b-a706-02f8581f31f8",
"part": "whole"
},
"id": "209643bd-687f-4169-9fbf-1ae613746f88"
},
"ba4f2d4c-7757-40f5-854d-64dd44420957": {
"attrs": {
"height": 0.4,
"width": 0.8,
"x": 0.1,
"y": 0.5
},
"content": {
"cell": "f89dd283-4742-4c38-8e9a-9b60fee06679",
"part": "whole"
},
"id": "ba4f2d4c-7757-40f5-854d-64dd44420957"
}
}
},
"8414982b-4bcb-44e9-94d3-cffe41a748e0": {
"id": "8414982b-4bcb-44e9-94d3-cffe41a748e0",
"prev": "b3db1ddb-1797-4cdd-9d3e-86186f93f8e7",
"regions": {
"0c57910e-7aeb-45ea-9563-fbedf7ad0725": {
"attrs": {
"height": 0.8,
"width": 0.8,
"x": 0.1,
"y": 0.1
},
"content": {
"cell": "698b56a8-e8b4-4e78-bdae-ebe1080aa9e8",
"part": "whole"
},
"id": "0c57910e-7aeb-45ea-9563-fbedf7ad0725"
},
"ebf537b5-6785-43b8-ba30-fcbd3ffbb8dc": {
"attrs": {
"height": 0.4,
"width": 0.8,
"x": 0.1,
"y": 0.5
},
"content": {
"cell": "06c7a259-df52-48dd-9f68-c00df0cec78f",
"part": "whole"
},
"id": "ebf537b5-6785-43b8-ba30-fcbd3ffbb8dc"
}
}
},
"86e361b5-bea4-4688-bbb8-433dd9a52cb8": {
"id": "86e361b5-bea4-4688-bbb8-433dd9a52cb8",
"prev": "58f3634a-3f7b-4ce7-b429-1645c7de66c3",
"regions": {
"64053282-2416-493b-b8a9-646df8a8d7e6": {
"attrs": {
"height": 0.8,
"width": 0.8,
"x": 0.1,
"y": 0.1
},
"content": {
"cell": "798103d5-eaaf-48f7-9f2d-c0e7ab72da03",
"part": "whole"
},
"id": "64053282-2416-493b-b8a9-646df8a8d7e6"
},
"781f2b8a-01a1-4bab-b10e-d3307c89ecc0": {
"attrs": {
"height": 0.4,
"width": 0.8,
"x": 0.1,
"y": 0.5
},
"content": {
"cell": "2c81b353-91b2-499b-a032-4cbd760e90d9",
"part": "whole"
},
"id": "781f2b8a-01a1-4bab-b10e-d3307c89ecc0"
}
}
},
"8a576a2c-f03a-4666-ac1b-b4d22158e0ae": {
"id": "8a576a2c-f03a-4666-ac1b-b4d22158e0ae",
"prev": "f6971449-a9b8-4dda-898b-2bd363ef7a9e",
"regions": {
"12106903-6ec0-457c-a545-f4237ec431bb": {
"attrs": {
"height": 0.8,
"width": 0.8,
"x": 0.1,
"y": 0.1
},
"content": {
"cell": "b6b2ca1e-33a1-42d8-a087-ecb2f9cf982f",
"part": "whole"
},
"id": "12106903-6ec0-457c-a545-f4237ec431bb"
},
"826f9188-d953-456c-bccd-9a8887598d84": {
"attrs": {
"height": 0.4,
"width": 0.8,
"x": 0.1,
"y": 0.5
},
"content": {
"cell": "47327248-30f0-4f29-ae6f-e9ebc57b1d71",
"part": "whole"
},
"id": "826f9188-d953-456c-bccd-9a8887598d84"
}
}
},
"8ae6c40a-d85a-437d-aee5-967da65cea00": {
"id": "8ae6c40a-d85a-437d-aee5-967da65cea00",
"prev": "ee4477e9-b437-4bf9-a09e-4497009d4258",
"regions": {
"0dced8f7-e3c3-4360-8e14-abe931fee193": {
"attrs": {
"height": 0.4,
"width": 0.8,
"x": 0.1,
"y": 0.5
},
"content": {
"cell": "f21bbcaa-796a-4253-9262-45545eefbd99",
"part": "whole"
},
"id": "0dced8f7-e3c3-4360-8e14-abe931fee193"
},
"1fc547c4-80f6-4bcc-98fb-84b051065920": {
"attrs": {
"height": 0.8,
"width": 0.8,
"x": 0.1,
"y": 0.1
},
"content": {
"cell": "6973c6c7-73a6-4ee6-a9e2-36a287f030d1",
"part": "whole"
},
"id": "1fc547c4-80f6-4bcc-98fb-84b051065920"
}
}
},
"8e6ddfe2-8fe0-48c1-8fa1-9614d34faf77": {
"id": "8e6ddfe2-8fe0-48c1-8fa1-9614d34faf77",
"prev": "8a576a2c-f03a-4666-ac1b-b4d22158e0ae",
"regions": {
"68014c74-d03c-48e9-810f-f4e97ef00a84": {
"attrs": {
"height": 0.8,
"width": 0.8,
"x": 0.1,
"y": 0.1
},
"content": {
"cell": "822ea43b-4ca4-4a7e-b398-61c63ea42274",
"part": "whole"
},
"id": "68014c74-d03c-48e9-810f-f4e97ef00a84"
}
}
},
"8f0a1e4b-795e-47d2-a5b3-99c117906bdc": {
"id": "8f0a1e4b-795e-47d2-a5b3-99c117906bdc",
"prev": "0b177343-6e12-4a66-ad41-c77db7354d28",
"regions": {
"365c151f-a5fd-4b2b-97e2-b6c75ca05650": {
"attrs": {
"height": 0.8,
"width": 0.8,
"x": 0.1,
"y": 0.1
},
"content": {
"cell": "6e44d7d3-586a-4727-a2e4-e947bb101311",
"part": "whole"
},
"id": "365c151f-a5fd-4b2b-97e2-b6c75ca05650"
},
"3788ac16-36ec-4609-85a6-fbf436a2afb8": {
"attrs": {
"height": 0.4,
"width": 0.8,
"x": 0.1,
"y": 0.5
},
"content": {
"cell": "47b1580b-9347-4b5a-aa56-9770bffb3e3d",
"part": "whole"
},
"id": "3788ac16-36ec-4609-85a6-fbf436a2afb8"
}
}
},
"9257a9c6-3c21-46d3-b581-c792bc816481": {
"id": "9257a9c6-3c21-46d3-b581-c792bc816481",
"prev": "b0d2e189-e6a6-4aa4-b760-452525752b8b",
"regions": {
"30c16d7f-15f0-4328-a643-68e118006c79": {
"attrs": {
"height": 0.4,
"width": 0.8,
"x": 0.1,
"y": 0.5
},
"content": {
"cell": "f965626d-4d0a-40eb-9d77-fd722555b540",
"part": "whole"
},
"id": "30c16d7f-15f0-4328-a643-68e118006c79"
},
"7b4886bf-9155-4554-9227-b19f5397c22a": {
"attrs": {
"height": 0.8,
"width": 0.8,
"x": 0.1,
"y": 0.1
},
"content": {
"cell": "a5d088e6-c0c7-4b16-9228-017dce9102a4",
"part": "whole"
},
"id": "7b4886bf-9155-4554-9227-b19f5397c22a"
}
}
},
"92f47635-a687-48d5-9373-1a07ce14ae6c": {
"id": "92f47635-a687-48d5-9373-1a07ce14ae6c",
"prev": "1d353827-f707-4d86-b1db-740fdd771573",
"regions": {
"838c4519-b10c-477f-8598-62d25ac37d49": {
"attrs": {
"height": 0.8,
"width": 0.8,
"x": 0.1,
"y": 0.1
},
"content": {
"cell": "98dfb06b-88e1-46b2-8613-892755563625",
"part": "whole"
},
"id": "838c4519-b10c-477f-8598-62d25ac37d49"
},
"defcb8e4-4693-487d-94c5-5b6da3612053": {
"attrs": {
"height": 0.4,
"width": 0.8,
"x": 0.1,
"y": 0.5
},
"content": {
"cell": "bf740d18-39b3-459a-a5f4-d53b4d0bbb8a",
"part": "whole"
},
"id": "defcb8e4-4693-487d-94c5-5b6da3612053"
}
}
},
"9440249f-5c6f-48b5-b2e1-e75a42536ece": {
"id": "9440249f-5c6f-48b5-b2e1-e75a42536ece",
"prev": "bae1d274-f442-4d15-af4f-033aae4151fc",
"regions": {
"9a64accc-b8ee-4a08-9681-7affe80f9063": {
"attrs": {
"height": 0.4,
"width": 0.8,
"x": 0.1,
"y": 0.5
},
"content": {
"cell": "9bb62e93-4382-493b-b53b-4d7df63a9435",
"part": "whole"
},
"id": "9a64accc-b8ee-4a08-9681-7affe80f9063"
},
"b73ea9f3-90fe-40a4-ae34-abf7fa465bca": {
"attrs": {
"height": 0.8,
"width": 0.8,
"x": 0.1,
"y": 0.1
},
"content": {
"cell": "e59b759d-ae9e-4ee3-b9f7-1291ebe4aec9",
"part": "whole"
},
"id": "b73ea9f3-90fe-40a4-ae34-abf7fa465bca"
}
}
},
"9e426f7d-51c2-4154-815e-b6efdcf33584": {
"id": "9e426f7d-51c2-4154-815e-b6efdcf33584",
"layout": "grid",
"prev": "3f27bae0-13b3-4208-82b1-addf0aef954d",
"regions": {
"160f8e71-f66d-4c5d-8768-6b5e199c2910": {
"attrs": {
"height": 0.4166666666666667,
"pad": 0.01,
"width": 0.8333333333333334,
"x": 0.08333333333333333,
"y": 0.5
},
"content": {
"cell": "75e42a70-4a9d-4c27-bbf0-9b5305192f4f",
"part": "whole"
},
"id": "160f8e71-f66d-4c5d-8768-6b5e199c2910"
},
"185e440c-fabb-42b6-8626-2df3525cd700": {
"attrs": {
"height": 0.4166666666666667,
"pad": 0.01,
"width": 0.8333333333333334,
"x": 0.08333333333333333,
"y": 0.5
},
"content": {
"cell": "c4c00d91-15f7-43d7-bafe-e81298384e79",
"part": "whole"
},
"id": "185e440c-fabb-42b6-8626-2df3525cd700"
},
"c8523bb4-4beb-45df-bb10-31cc25f50b2d": {
"attrs": {
"height": 0.8333333333333334,
"pad": 0.01,
"width": 0.8333333333333334,
"x": 0.08333333333333333,
"y": 0.08333333333333333
},
"content": {
"cell": "ce398af3-13a5-4d65-9bef-f14ab7acb3fa",
"part": "whole"
},
"id": "c8523bb4-4beb-45df-bb10-31cc25f50b2d"
}
}
},
"a02815f7-8a80-434d-bab7-43146bc4ce66": {
"id": "a02815f7-8a80-434d-bab7-43146bc4ce66",
"prev": "f5c82eaf-37a4-4380-871c-8d78e28e4a52",
"regions": {
"117c6c02-efc0-4b76-b221-10fea1f39561": {
"attrs": {
"height": 0.8,
"width": 0.8,
"x": 0.1,
"y": 0.1
},
"content": {
"cell": "d3e2bf05-df2b-4af1-8c4d-295644e11a2f",
"part": "whole"
},
"id": "117c6c02-efc0-4b76-b221-10fea1f39561"
},
"9260cc56-3059-4a5d-9c5d-2257355c9929": {
"attrs": {
"height": 0.4,
"width": 0.8,
"x": 0.1,
"y": 0.5
},
"content": {
"cell": "4c07e89a-43d9-497f-96e8-8a040339c155",
"part": "whole"
},
"id": "9260cc56-3059-4a5d-9c5d-2257355c9929"
}
}
},
"ab3a3b2a-f646-4972-9385-8fa39ec860aa": {
"id": "ab3a3b2a-f646-4972-9385-8fa39ec860aa",
"prev": "02eaa597-aed6-435d-a4b7-0668b299001b",
"regions": {
"48384580-7d69-447c-bda7-1da8c3ef1e21": {
"attrs": {
"height": 0.8,
"width": 0.8,
"x": 0.1,
"y": 0.1
},
"content": {
"cell": "6652d9c6-f1aa-499f-8958-849e25f9d1a4",
"part": "whole"
},
"id": "48384580-7d69-447c-bda7-1da8c3ef1e21"
}
}
},
"aba0a392-86eb-4564-bd3e-3ddc24021e84": {
"id": "aba0a392-86eb-4564-bd3e-3ddc24021e84",
"prev": "32e95611-4585-44dc-a22d-934e38f9c9ae",
"regions": {
"6db5ad51-42cd-45c5-9f9a-50501dd341a9": {
"attrs": {
"height": 0.8,
"width": 0.8,
"x": 0.1,
"y": 0.1
},
"content": {
"cell": "12356ed9-0363-4c2c-8e00-704eae432a9f",
"part": "whole"
},
"id": "6db5ad51-42cd-45c5-9f9a-50501dd341a9"
}
}
},
"ad96e441-8744-4044-a385-7fae743e1c2b": {
"id": "ad96e441-8744-4044-a385-7fae743e1c2b",
"prev": "d9bd716a-a32f-4045-8625-8c4e1932f806",
"regions": {
"9752a39b-fa90-44f3-a39a-808940631ad2": {
"attrs": {
"height": 0.8,
"width": 0.8,
"x": 0.1,
"y": 0.1
},
"content": {
"cell": "a00e78a9-dcac-492c-82f9-4512d9f3cc51",
"part": "whole"
},
"id": "9752a39b-fa90-44f3-a39a-808940631ad2"
},
"ea6139c5-2b89-44e6-a1e5-59050c013a0f": {
"attrs": {
"height": 0.4,
"width": 0.8,
"x": 0.1,
"y": 0.5
},
"content": {
"cell": "b2c44061-460d-4392-82cd-8984d3045761",
"part": "whole"
},
"id": "ea6139c5-2b89-44e6-a1e5-59050c013a0f"
}
}
},
"b0bfdbe3-c4cb-435e-8452-ab71962a08f1": {
"id": "b0bfdbe3-c4cb-435e-8452-ab71962a08f1",
"prev": "aba0a392-86eb-4564-bd3e-3ddc24021e84",
"regions": {
"23e9d835-d998-447f-a2e6-76369566e4eb": {
"attrs": {
"height": 0.8,
"width": 0.8,
"x": 0.1,
"y": 0.1
},
"content": {
"cell": "1cf1a17e-669b-4f3c-9159-c5a54ca1baf6",
"part": "whole"
},
"id": "23e9d835-d998-447f-a2e6-76369566e4eb"
},
"ce6cdcd9-be21-4526-8d16-961a3b6778e0": {
"attrs": {
"height": 0.4,
"width": 0.8,
"x": 0.1,
"y": 0.5
},
"content": {
"cell": "ed8bf68c-84cc-49fe-bf4d-c83ecea6d122",
"part": "whole"
},
"id": "ce6cdcd9-be21-4526-8d16-961a3b6778e0"
}
}
},
"b0d2e189-e6a6-4aa4-b760-452525752b8b": {
"id": "b0d2e189-e6a6-4aa4-b760-452525752b8b",
"prev": "b0bfdbe3-c4cb-435e-8452-ab71962a08f1",
"regions": {
"0e779fca-fdb1-4df0-ae4c-cabd3daaa31d": {
"attrs": {
"height": 0.8,
"width": 0.8,
"x": 0.1,
"y": 0.1
},
"content": {
"cell": "8354e8d4-c88e-45b6-a60a-7535c808296e",
"part": "whole"
},
"id": "0e779fca-fdb1-4df0-ae4c-cabd3daaa31d"
},
"177c92f6-7712-4c01-af69-4898058503d5": {
"attrs": {
"height": 0.4,
"width": 0.8,
"x": 0.1,
"y": 0.5
},
"content": {
"cell": "e3408549-4969-486a-a7e0-f6d55797eeaf",
"part": "whole"
},
"id": "177c92f6-7712-4c01-af69-4898058503d5"
},
"56959001-53a9-4880-9c89-511d8472c1f9": {
"attrs": {
"height": 0.4,
"width": 0.8,
"x": 0.1,
"y": 0.5
},
"content": {
"cell": "2579c7a9-db52-45c6-971a-cedbaa78fe37",
"part": "whole"
},
"id": "56959001-53a9-4880-9c89-511d8472c1f9"
},
"fd8f749c-dcac-418d-9396-7d468e35826f": {
"attrs": {
"height": 0.4,
"width": 0.8,
"x": 0.1,
"y": 0.5
},
"content": {
"cell": "078a2142-e5be-4e23-b1c4-6cad277d648f",
"part": "whole"
},
"id": "fd8f749c-dcac-418d-9396-7d468e35826f"
}
}
},
"b39a769c-bfd8-4f92-ae58-b5f2da66e57b": {
"id": "b39a769c-bfd8-4f92-ae58-b5f2da66e57b",
"prev": "86e361b5-bea4-4688-bbb8-433dd9a52cb8",
"regions": {
"336d0d2b-e70a-40d4-81bb-64b42d4a0051": {
"attrs": {
"height": 0.4,
"width": 0.8,
"x": 0.1,
"y": 0.5
},
"content": {
"cell": "a085a712-b63a-4a9a-85eb-4e15518d0953",
"part": "whole"
},
"id": "336d0d2b-e70a-40d4-81bb-64b42d4a0051"
},
"433b5223-7354-4854-b32c-2e934ed82f83": {
"attrs": {
"height": 0.4,
"width": 0.8,
"x": 0.1,
"y": 0.5
},
"content": {
"cell": "3a2e4af8-5c43-491d-80cf-87fb083ad6c7",
"part": "whole"
},
"id": "433b5223-7354-4854-b32c-2e934ed82f83"
},
"8d91f5b1-26bd-4c77-ace1-b2d28b99214d": {
"attrs": {
"height": 0.8,
"width": 0.8,
"x": 0.1,
"y": 0.1
},
"content": {
"cell": "5fb55ca8-260a-4873-8e7e-772f010fce8c",
"part": "whole"
},
"id": "8d91f5b1-26bd-4c77-ace1-b2d28b99214d"
}
}
},
"b3db1ddb-1797-4cdd-9d3e-86186f93f8e7": {
"id": "b3db1ddb-1797-4cdd-9d3e-86186f93f8e7",
"prev": "f8092d43-e1ef-434b-98af-2f33f1d58c48",
"regions": {
"367435a5-805c-4107-a3a7-2a1a9eef5dbb": {
"attrs": {
"height": 0.8,
"width": 0.8,
"x": 0.1,
"y": 0.1
},
"content": {
"cell": "3791e96a-cbb6-40d6-ba12-ed820eea8c66",
"part": "whole"
},
"id": "367435a5-805c-4107-a3a7-2a1a9eef5dbb"
},
"9dc57157-f265-43a9-8943-4e60b0da832b": {
"attrs": {
"height": 0.4,
"width": 0.8,
"x": 0.1,
"y": 0.5
},
"content": {
"cell": "c62afb25-272e-4263-bfd2-3cd27916e73f",
"part": "whole"
},
"id": "9dc57157-f265-43a9-8943-4e60b0da832b"
}
}
},
"ba6a43ee-7ff8-4dff-a537-3a96eb12b83b": {
"id": "ba6a43ee-7ff8-4dff-a537-3a96eb12b83b",
"prev": "42e3c596-98c1-4abe-941a-0f9ccf6ac609",
"regions": {
"53fc5110-959e-4391-a069-b6b61a5d7e30": {
"attrs": {
"height": 0.8,
"width": 0.8,
"x": 0.1,
"y": 0.1
},
"content": {
"cell": "82276496-388e-46e7-a183-0b53bbfaa885",
"part": "whole"
},
"id": "53fc5110-959e-4391-a069-b6b61a5d7e30"
}
}
},
"bae1d274-f442-4d15-af4f-033aae4151fc": {
"id": "bae1d274-f442-4d15-af4f-033aae4151fc",
"prev": "452e80ee-c961-440f-bde6-e683956d69ef",
"regions": {
"93529ef8-62c7-411c-b19a-b0fcb76baf03": {
"attrs": {
"height": 0.4,
"width": 0.8,
"x": 0.1,
"y": 0.5
},
"content": {
"cell": "dc0d475f-fce7-47b6-9142-54a6aaa767cb",
"part": "whole"
},
"id": "93529ef8-62c7-411c-b19a-b0fcb76baf03"
},
"f11c5200-4879-40d9-9e55-73415abd2479": {
"attrs": {
"height": 0.8,
"width": 0.8,
"x": 0.1,
"y": 0.1
},
"content": {
"cell": "e2e0d246-7071-494e-9675-c7f1b8421376",
"part": "whole"
},
"id": "f11c5200-4879-40d9-9e55-73415abd2479"
}
}
},
"bf981077-1f22-4084-ae49-2706477a5da9": {
"id": "bf981077-1f22-4084-ae49-2706477a5da9",
"prev": "9257a9c6-3c21-46d3-b581-c792bc816481",
"regions": {
"0553b03e-ae75-45a2-9931-4a4844d211b5": {
"attrs": {
"height": 0.4,
"width": 0.8,
"x": 0.1,
"y": 0.5
},
"content": {
"cell": "624ae641-ea47-4831-9c18-07a12f31f1d7",
"part": "whole"
},
"id": "0553b03e-ae75-45a2-9931-4a4844d211b5"
},
"dd720800-a86c-4d6e-a601-e90351d6fcc9": {
"attrs": {
"height": 0.8,
"width": 0.8,
"x": 0.1,
"y": 0.1
},
"content": {
"cell": "ba593c28-3d21-4d20-b444-424be1386366",
"part": "whole"
},
"id": "dd720800-a86c-4d6e-a601-e90351d6fcc9"
}
}
},
"cdc520e2-3bde-49f5-bda0-3bd42c48640c": {
"id": "cdc520e2-3bde-49f5-bda0-3bd42c48640c",
"prev": "0eba2662-c9fd-406d-ae98-1477c861e9a4",
"regions": {
"63960e30-634f-4954-a810-54bbd63f2b05": {
"attrs": {
"height": 0.8,
"width": 0.8,
"x": 0.1,
"y": 0.1
},
"content": {
"cell": "2e533bf3-7dd8-4432-9d33-6d69bcae0caa",
"part": "whole"
},
"id": "63960e30-634f-4954-a810-54bbd63f2b05"
}
}
},
"cf02493a-823d-4df3-9865-8f4c0955bae3": {
"id": "cf02493a-823d-4df3-9865-8f4c0955bae3",
"prev": "65ca0680-a098-40ab-8a41-e8a753d7228d",
"regions": {
"0b945f38-7654-4fe0-afaf-640ca1994ec5": {
"attrs": {
"height": 0.4,
"width": 0.8,
"x": 0.1,
"y": 0.5
},
"content": {
"cell": "e7289eea-badb-4549-97f5-36404ff9b061",
"part": "whole"
},
"id": "0b945f38-7654-4fe0-afaf-640ca1994ec5"
},
"411017ab-1e39-439a-abd9-3dff831cf17b": {
"attrs": {
"height": 0.8,
"width": 0.8,
"x": 0.1,
"y": 0.1
},
"content": {
"cell": "7094e510-13ac-4e3d-8cbf-fefcc81e6d41",
"part": "whole"
},
"id": "411017ab-1e39-439a-abd9-3dff831cf17b"
}
}
},
"d67b2752-be57-416d-9d30-d49a530cbaee": {
"id": "d67b2752-be57-416d-9d30-d49a530cbaee",
"prev": "407a43f7-0f13-4928-88c8-5b19055dcfc3",
"regions": {
"55be1703-f583-4e6b-a121-543f8c088daa": {
"attrs": {
"height": 0.8,
"width": 0.8,
"x": 0.1,
"y": 0.1
},
"content": {
"cell": "a2ea6e73-816b-426d-8c40-4edc3c3ab894",
"part": "whole"
},
"id": "55be1703-f583-4e6b-a121-543f8c088daa"
},
"a287b67f-361b-49e4-a84d-529169012054": {
"attrs": {
"height": 0.4,
"width": 0.8,
"x": 0.1,
"y": 0.5
},
"content": {
"cell": "4a5a9b7f-fbb0-4d7d-a0c8-94aaebab0116",
"part": "whole"
},
"id": "a287b67f-361b-49e4-a84d-529169012054"
}
}
},
"d9bd716a-a32f-4045-8625-8c4e1932f806": {
"id": "d9bd716a-a32f-4045-8625-8c4e1932f806",
"prev": "1e52131c-cfee-47c4-9774-d5503ed0c561",
"regions": {
"c3c3257b-3e70-48fb-aed9-8f2cdb5b8904": {
"attrs": {
"height": 0.8,
"width": 0.8,
"x": 0.1,
"y": 0.1
},
"content": {
"cell": "52c70350-0896-498f-b322-8d4880a15d0c",
"part": "whole"
},
"id": "c3c3257b-3e70-48fb-aed9-8f2cdb5b8904"
},
"f284b611-d55a-4332-b97a-e5f1f00afc42": {
"attrs": {
"height": 0.4,
"width": 0.8,
"x": 0.1,
"y": 0.5
},
"content": {
"cell": "8908acc8-32b8-43a5-91c3-c90120ac2c00",
"part": "whole"
},
"id": "f284b611-d55a-4332-b97a-e5f1f00afc42"
}
}
},
"daa50dee-37ef-4997-8293-f1dbd7cf68c4": {
"id": "daa50dee-37ef-4997-8293-f1dbd7cf68c4",
"prev": "698f2be8-8608-4051-ae18-072812614952",
"regions": {
"29541d7f-c385-4434-bbdc-d70e29523a61": {
"attrs": {
"height": 0.8,
"width": 0.8,
"x": 0.1,
"y": 0.1
},
"content": {
"cell": "1217bdf0-eb28-4ec8-9801-1abf13ef7217",
"part": "whole"
},
"id": "29541d7f-c385-4434-bbdc-d70e29523a61"
},
"8fe0deef-f942-4851-bbee-1bdd0433275b": {
"attrs": {
"height": 0.4,
"width": 0.8,
"x": 0.1,
"y": 0.5
},
"content": {
"cell": "8b3135bc-1f3c-47dc-9696-30692e2460d9",
"part": "whole"
},
"id": "8fe0deef-f942-4851-bbee-1bdd0433275b"
}
}
},
"ee4477e9-b437-4bf9-a09e-4497009d4258": {
"id": "ee4477e9-b437-4bf9-a09e-4497009d4258",
"prev": "591d6bea-4d9d-43be-840a-75f934127fd0",
"regions": {
"0482cb87-2823-4254-b88c-ab7152faa50e": {
"attrs": {
"height": 0.8,
"width": 0.8,
"x": 0.1,
"y": 0.1
},
"content": {
"cell": "cecdb782-f8dc-4fba-87b6-e5c1f772a364",
"part": "whole"
},
"id": "0482cb87-2823-4254-b88c-ab7152faa50e"
},
"74629420-37c5-44f7-90ed-f61a10762d36": {
"attrs": {
"height": 0.4,
"width": 0.8,
"x": 0.1,
"y": 0.5
},
"content": {
"cell": "a329ad3e-b11f-437b-a2ef-918915a637b0",
"part": "whole"
},
"id": "74629420-37c5-44f7-90ed-f61a10762d36"
}
}
},
"f43c3aec-5d2a-4e5d-b976-eb68819cbd41": {
"id": "f43c3aec-5d2a-4e5d-b976-eb68819cbd41",
"prev": "05eedcfe-475b-4862-b777-4fa153d9849a",
"regions": {
"dbef7b36-bb1d-4e42-8efc-dad695038a58": {
"attrs": {
"height": 0.4,
"width": 0.8,
"x": 0.1,
"y": 0.5
},
"content": {
"cell": "d65875d4-f608-4a3d-a033-6af98aa2e416",
"part": "whole"
},
"id": "dbef7b36-bb1d-4e42-8efc-dad695038a58"
},
"e01bd7b3-2ad4-464a-b088-1bb4a80d1bae": {
"attrs": {
"height": 0.8,
"width": 0.8,
"x": 0.1,
"y": 0.1
},
"content": {
"cell": "2b7349aa-dd61-4fac-a82d-f02021436693",
"part": "whole"
},
"id": "e01bd7b3-2ad4-464a-b088-1bb4a80d1bae"
}
}
},
"f5c82eaf-37a4-4380-871c-8d78e28e4a52": {
"id": "f5c82eaf-37a4-4380-871c-8d78e28e4a52",
"prev": "4d001e63-ce4e-46a2-a137-7ebd7b232f83",
"regions": {
"001469fd-9c7d-4eac-a6b2-85abea27d3df": {
"attrs": {
"height": 0.4,
"width": 0.8,
"x": 0.1,
"y": 0.5
},
"content": {
"cell": "b7a21973-1821-4c79-a142-063e5ad29db8",
"part": "whole"
},
"id": "001469fd-9c7d-4eac-a6b2-85abea27d3df"
},
"da8adf4b-f7fe-42ac-bc12-87ba0a831896": {
"attrs": {
"height": 0.8,
"width": 0.8,
"x": 0.1,
"y": 0.1
},
"content": {
"cell": "0bf4e6f4-1ad3-43be-8ad4-caa02d6804a0",
"part": "whole"
},
"id": "da8adf4b-f7fe-42ac-bc12-87ba0a831896"
}
}
},
"f6971449-a9b8-4dda-898b-2bd363ef7a9e": {
"id": "f6971449-a9b8-4dda-898b-2bd363ef7a9e",
"prev": "b39a769c-bfd8-4f92-ae58-b5f2da66e57b",
"regions": {
"89d6b351-a3bb-4425-8371-b4a82ecf70d7": {
"attrs": {
"height": 0.8,
"width": 0.8,
"x": 0.1,
"y": 0.1
},
"content": {
"cell": "76416d5c-74d7-4355-80da-fda820fd96da",
"part": "whole"
},
"id": "89d6b351-a3bb-4425-8371-b4a82ecf70d7"
},
"faf33047-1c2a-4486-8411-15e00e808be3": {
"attrs": {
"height": 0.4,
"width": 0.8,
"x": 0.1,
"y": 0.5
},
"content": {
"cell": "578b0eac-adbd-458f-8a55-688e4a5cc3ee",
"part": "whole"
},
"id": "faf33047-1c2a-4486-8411-15e00e808be3"
}
}
},
"f8092d43-e1ef-434b-98af-2f33f1d58c48": {
"id": "f8092d43-e1ef-434b-98af-2f33f1d58c48",
"prev": "5a9c3c6d-e78d-46a5-9d26-d76753a4839b",
"regions": {
"cd0d22d5-0c9d-478b-9768-fd23a5746612": {
"attrs": {
"height": 0.8,
"width": 0.8,
"x": 0.1,
"y": 0.1
},
"content": {
"cell": "eda9c15c-5d3a-4d10-85c1-4718d8bd0954",
"part": "whole"
},
"id": "cd0d22d5-0c9d-478b-9768-fd23a5746612"
}
}
},
"f89b6695-6ec6-4edb-b345-7fd691743aba": {
"id": "f89b6695-6ec6-4edb-b345-7fd691743aba",
"prev": "fe2387a3-0a45-4561-af46-78d1fa7f7048",
"regions": {
"51662075-a88f-485b-aa7a-18470ced4815": {
"attrs": {
"height": 0.4,
"width": 0.8,
"x": 0.1,
"y": 0.5
},
"content": {
"cell": "5ae6e833-7afc-467c-8ee8-afdc908218de",
"part": "whole"
},
"id": "51662075-a88f-485b-aa7a-18470ced4815"
},
"a95e2658-44d3-44f5-9c99-42b2d9b64b4e": {
"attrs": {
"height": 0.8,
"width": 0.8,
"x": 0.1,
"y": 0.1
},
"content": {
"cell": "a85c9206-dcb8-4d3f-9cda-043d6b009f02",
"part": "whole"
},
"id": "a95e2658-44d3-44f5-9c99-42b2d9b64b4e"
},
"dc8d2b8b-6354-453c-a860-2d9c7aa93874": {
"attrs": {
"height": 0.4,
"width": 0.8,
"x": 0.1,
"y": 0.5
},
"content": {
"cell": "e647d3ae-1e01-4227-955a-0af82ae6f20b",
"part": "whole"
},
"id": "dc8d2b8b-6354-453c-a860-2d9c7aa93874"
}
}
},
"f9fa2328-e23d-4584-826b-e1bf0a935d67": {
"id": "f9fa2328-e23d-4584-826b-e1bf0a935d67",
"prev": "92f47635-a687-48d5-9373-1a07ce14ae6c",
"regions": {
"bab52c1e-5438-48fc-a21c-2078d20cd850": {
"attrs": {
"height": 0.8,
"width": 0.8,
"x": 0.1,
"y": 0.1
},
"content": {
"cell": "48cc25f3-cb47-4db5-8618-4afbe26c2b7a",
"part": "whole"
},
"id": "bab52c1e-5438-48fc-a21c-2078d20cd850"
},
"f2c91094-c949-4144-a7b9-55986a2ccd42": {
"attrs": {
"height": 0.4,
"width": 0.8,
"x": 0.1,
"y": 0.5
},
"content": {
"cell": "7bc25aab-7d00-425f-852e-b8ffa6b67ffd",
"part": "whole"
},
"id": "f2c91094-c949-4144-a7b9-55986a2ccd42"
}
}
},
"fe2387a3-0a45-4561-af46-78d1fa7f7048": {
"id": "fe2387a3-0a45-4561-af46-78d1fa7f7048",
"prev": "cdc520e2-3bde-49f5-bda0-3bd42c48640c",
"regions": {
"01bf0dd1-a945-465b-ad4e-e9aada2cdc78": {
"attrs": {
"height": 0.4,
"width": 0.8,
"x": 0.1,
"y": 0.5
},
"content": {
"cell": "dd17fbad-5b9c-4c96-801c-dc4decadc4dd",
"part": "whole"
},
"id": "01bf0dd1-a945-465b-ad4e-e9aada2cdc78"
},
"904bf395-01ca-41b3-a661-089f1537efd3": {
"attrs": {
"height": 0.8,
"width": 0.8,
"x": 0.1,
"y": 0.1
},
"content": {
"cell": "24e64ebd-5016-4cec-a560-d5f7a8a0b2ce",
"part": "whole"
},
"id": "904bf395-01ca-41b3-a661-089f1537efd3"
}
}
},
"ff5dec66-40f4-4ae4-9170-2df11f95c422": {
"id": "ff5dec66-40f4-4ae4-9170-2df11f95c422",
"prev": "4ca922b9-2bd0-44d0-9078-78fa564fadda",
"regions": {
"362fb761-461d-49c8-ad83-954926cf40cf": {
"attrs": {
"height": 0.8,
"width": 0.8,
"x": 0.1,
"y": 0.1
},
"content": {
"cell": "b392e02c-8b5d-458b-8102-5ddc2171f885",
"part": "whole"
},
"id": "362fb761-461d-49c8-ad83-954926cf40cf"
},
"3d677cf0-3ecd-4d4b-b363-a9a1252e217d": {
"attrs": {
"height": 0.4,
"width": 0.8,
"x": 0.1,
"y": 0.5
},
"content": {
"cell": "ccc7a759-5856-4315-8acf-b89a76b496ab",
"part": "whole"
},
"id": "3d677cf0-3ecd-4d4b-b363-a9a1252e217d"
}
}
}
},
"themes": {
"default": "931d8d81-9413-4296-8bf7-0c6fd361f32a",
"theme": {
"931d8d81-9413-4296-8bf7-0c6fd361f32a": {
"id": "931d8d81-9413-4296-8bf7-0c6fd361f32a",
"palette": {
"19cc588f-0593-49c9-9f4b-e4d7cc113b1c": {
"id": "19cc588f-0593-49c9-9f4b-e4d7cc113b1c",
"rgb": [
252,
252,
252
]
},
"31af15d2-7e15-44c5-ab5e-e04b16a89eff": {
"id": "31af15d2-7e15-44c5-ab5e-e04b16a89eff",
"rgb": [
68,
68,
68
]
},
"50f92c45-a630-455b-aec3-788680ec7410": {
"id": "50f92c45-a630-455b-aec3-788680ec7410",
"rgb": [
155,
177,
192
]
},
"c5cc3653-2ee1-402a-aba2-7caae1da4f6c": {
"id": "c5cc3653-2ee1-402a-aba2-7caae1da4f6c",
"rgb": [
43,
126,
184
]
},
"efa7f048-9acb-414c-8b04-a26811511a21": {
"id": "efa7f048-9acb-414c-8b04-a26811511a21",
"rgb": [
25.118061674008803,
73.60176211453744,
107.4819383259912
]
}
},
"rules": {
"blockquote": {
"color": "50f92c45-a630-455b-aec3-788680ec7410"
},
"code": {
"font-family": "Anonymous Pro"
},
"h1": {
"color": "c5cc3653-2ee1-402a-aba2-7caae1da4f6c",
"font-family": "Lato",
"font-size": 8
},
"h2": {
"color": "c5cc3653-2ee1-402a-aba2-7caae1da4f6c",
"font-family": "Lato",
"font-size": 6
},
"h3": {
"color": "50f92c45-a630-455b-aec3-788680ec7410",
"font-family": "Lato",
"font-size": 5.5
},
"h4": {
"color": "c5cc3653-2ee1-402a-aba2-7caae1da4f6c",
"font-family": "Lato",
"font-size": 5
},
"h5": {
"font-family": "Lato"
},
"h6": {
"font-family": "Lato"
},
"h7": {
"font-family": "Lato"
},
"pre": {
"font-family": "Anonymous Pro",
"font-size": 4
}
},
"text-base": {
"font-family": "Merriweather",
"font-size": 4
}
},
"af2c4a42-a557-4795-b977-37bfc8df6e30": {
"backgrounds": {
"dc7afa04-bf90-40b1-82a5-726e3cff5267": {
"background-color": "31af15d2-7e15-44c5-ab5e-e04b16a89eff",
"id": "dc7afa04-bf90-40b1-82a5-726e3cff5267"
}
},
"id": "af2c4a42-a557-4795-b977-37bfc8df6e30",
"palette": {
"19cc588f-0593-49c9-9f4b-e4d7cc113b1c": {
"id": "19cc588f-0593-49c9-9f4b-e4d7cc113b1c",
"rgb": [
252,
252,
252
]
},
"31af15d2-7e15-44c5-ab5e-e04b16a89eff": {
"id": "31af15d2-7e15-44c5-ab5e-e04b16a89eff",
"rgb": [
68,
68,
68
]
},
"50f92c45-a630-455b-aec3-788680ec7410": {
"id": "50f92c45-a630-455b-aec3-788680ec7410",
"rgb": [
197,
226,
245
]
},
"c5cc3653-2ee1-402a-aba2-7caae1da4f6c": {
"id": "c5cc3653-2ee1-402a-aba2-7caae1da4f6c",
"rgb": [
43,
126,
184
]
},
"efa7f048-9acb-414c-8b04-a26811511a21": {
"id": "efa7f048-9acb-414c-8b04-a26811511a21",
"rgb": [
25.118061674008803,
73.60176211453744,
107.4819383259912
]
}
},
"rules": {
"a": {
"color": "19cc588f-0593-49c9-9f4b-e4d7cc113b1c"
},
"blockquote": {
"color": "50f92c45-a630-455b-aec3-788680ec7410",
"font-size": 3
},
"code": {
"font-family": "Anonymous Pro"
},
"h1": {
"color": "19cc588f-0593-49c9-9f4b-e4d7cc113b1c",
"font-family": "Merriweather",
"font-size": 8
},
"h2": {
"color": "19cc588f-0593-49c9-9f4b-e4d7cc113b1c",
"font-family": "Merriweather",
"font-size": 6
},
"h3": {
"color": "50f92c45-a630-455b-aec3-788680ec7410",
"font-family": "Lato",
"font-size": 5.5
},
"h4": {
"color": "c5cc3653-2ee1-402a-aba2-7caae1da4f6c",
"font-family": "Lato",
"font-size": 5
},
"h5": {
"font-family": "Lato"
},
"h6": {
"font-family": "Lato"
},
"h7": {
"font-family": "Lato"
},
"li": {
"color": "50f92c45-a630-455b-aec3-788680ec7410",
"font-size": 3.25
},
"pre": {
"font-family": "Anonymous Pro",
"font-size": 4
}
},
"text-base": {
"color": "19cc588f-0593-49c9-9f4b-e4d7cc113b1c",
"font-family": "Lato",
"font-size": 4
}
}
}
}
}
},
"nbformat": 4,
"nbformat_minor": 1
}