{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "# Exercise 3.3 - Solution\n", "## Checkerboard\n", "\n", "Open the Tensorflow Playground (www.playground.tensorflow.org) and select on the left the checkerboard pattern as the data basis.\n", "\n", "The data is taken from a two-dimensional probability distribution and is represented by the value pairs $x_1$ and $x_2$. The regions $x1$, $x_2 > 0$ and $x_1$, $x_2 < 0$ are shown by one color. For value pairs with $x_1 > 0$, $x_2 < 0$ and $x_1 < 0$, $x_2 > 0$, the regions are indicated by a different color. \n", "\n", "In features, select the two independent variables $x_1$ and $x_2$ and start the network training. The network learns that $x_1$ and $x_2$ are for these data not independent variables, but are taken from the probability distribution of the checkerboard pattern.\n", "\n", "[![Checkerboard](./images/checkerboard_tf_playground.png)](https://playground.tensorflow.org/#activation=relu&batchSize=10&dataset=xor®Dataset=reg-plane&learningRate=0.03®ularizationRate=0&noise=0&networkShape=4,2&seed=0.20784&showTestData=false&discretize=false&percTrainData=50&x=true&y=true&xTimesY=false&xSquared=false&ySquared=false&cosX=false&sinX=false&cosY=false&sinY=false&collectStats=false&problem=classification&initZero=false&hideText=false)\n", "\n", "## Tasks\n", "1. Try various settings for the number of layers and neurons using ReLU as activation function. What is the smallest network that gives a good fit result?\n", "2. What do you observe when training networks with the same settings multiple times? Explain your observations.\n", "3. Try additional input features: Which one is most helpful?\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Solutions\n", "Hint: click on the images to open the correct playground settings needed to solve the task, respectively." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Task 1\n", "Try various settings for the number of layers and neurons using ReLU as activation function. What is the smallest network that gives a good fit result?\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "[![Checkerboard](./images/checkerboard_tf_playground.png)](https://playground.tensorflow.org/#activation=relu&batchSize=10&dataset=xor®Dataset=reg-plane&learningRate=0.03®ularizationRate=0&noise=0&networkShape=3&seed=0.10528&showTestData=false&discretize=false&percTrainData=50&x=true&y=true&xTimesY=false&xSquared=false&ySquared=false&cosX=false&sinX=false&cosY=false&sinY=false&collectStats=false&problem=classification&initZero=false&hideText=false)\n", "\n", " A network with a single layer holding 3 nodes. However, this configuration is not stable.\n", "\n", "A network with a single layer, holding 4 nodes, is way more stable. " ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Task 2\n", "What do you observe when training networks with the same settings multiple times? Explain your observations." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ " Due to the random initialization of weights, the network training always develops a little bit differently, leading to different results. " ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Task 3\n", "Try additional input features: Which one is most helpful?" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "[![Checkerboard](./images/checkerboard_3_2_task_3.png)](https://playground.tensorflow.org/#activation=relu&batchSize=10&dataset=xor®Dataset=reg-plane&learningRate=0.03®ularizationRate=0&noise=0&networkShape=&seed=0.10528&showTestData=false&discretize=false&percTrainData=50&x=false&y=false&xTimesY=true&xSquared=false&ySquared=false&cosX=false&sinX=false&cosY=false&sinY=false&collectStats=false&problem=classification&initZero=false&hideText=false)\n", "\n", " Obviously, the $x_1\\cdot x_2$ feature is most helpful. " ] } ], "metadata": { "kernelspec": { "display_name": "Python 3", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.6.9" } }, "nbformat": 4, "nbformat_minor": 4 }