{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Exercise 3.3 - Solution\n",
"## Checkerboard\n",
"\n",
"Open the Tensorflow Playground (www.playground.tensorflow.org) and select on the left the checkerboard pattern as the data basis.\n",
"\n",
"The data is taken from a two-dimensional probability distribution and is represented by the value pairs $x_1$ and $x_2$. The regions $x1$, $x_2 > 0$ and $x_1$, $x_2 < 0$ are shown by one color. For value pairs with $x_1 > 0$, $x_2 < 0$ and $x_1 < 0$, $x_2 > 0$, the regions are indicated by a different color. \n",
"\n",
"In features, select the two independent variables $x_1$ and $x_2$ and start the network training. The network learns that $x_1$ and $x_2$ are for these data not independent variables, but are taken from the probability distribution of the checkerboard pattern.\n",
"\n",
"[![Checkerboard](./images/checkerboard_tf_playground.png)](https://playground.tensorflow.org/#activation=relu&batchSize=10&dataset=xor®Dataset=reg-plane&learningRate=0.03®ularizationRate=0&noise=0&networkShape=4,2&seed=0.20784&showTestData=false&discretize=false&percTrainData=50&x=true&y=true&xTimesY=false&xSquared=false&ySquared=false&cosX=false&sinX=false&cosY=false&sinY=false&collectStats=false&problem=classification&initZero=false&hideText=false)\n",
"\n",
"## Tasks\n",
"1. Try various settings for the number of layers and neurons using `ReLU` as activation function. What is the smallest network that gives a good fit result?\n",
"2. What do you observe when training networks with the same settings multiple times? Explain your observations.\n",
"3. Try additional input features: Which one is most helpful?\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Solutions\n",
"Hint: click on the images to open the correct playground settings needed to solve the task, respectively."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Task 1\n",
"Try various settings for the number of layers and neurons using ReLU as activation function. What is the smallest network that gives a good fit result?\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"[![Checkerboard](./images/checkerboard_tf_playground.png)](https://playground.tensorflow.org/#activation=relu&batchSize=10&dataset=xor®Dataset=reg-plane&learningRate=0.03®ularizationRate=0&noise=0&networkShape=3&seed=0.10528&showTestData=false&discretize=false&percTrainData=50&x=true&y=true&xTimesY=false&xSquared=false&ySquared=false&cosX=false&sinX=false&cosY=false&sinY=false&collectStats=false&problem=classification&initZero=false&hideText=false)\n",
"\n",
" A network with a single layer holding 3 nodes. However, this configuration is not stable.\n",
"\n",
"A network with a single layer, holding 4 nodes, is way more stable. "
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Task 2\n",
"What do you observe when training networks with the same settings multiple times? Explain your observations."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
" Due to the random initialization of weights, the network training always develops a little bit differently, leading to different results. "
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Task 3\n",
"Try additional input features: Which one is most helpful?"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"[![Checkerboard](./images/checkerboard_3_2_task_3.png)](https://playground.tensorflow.org/#activation=relu&batchSize=10&dataset=xor®Dataset=reg-plane&learningRate=0.03®ularizationRate=0&noise=0&networkShape=&seed=0.10528&showTestData=false&discretize=false&percTrainData=50&x=false&y=false&xTimesY=true&xSquared=false&ySquared=false&cosX=false&sinX=false&cosY=false&sinY=false&collectStats=false&problem=classification&initZero=false&hideText=false)\n",
"\n",
" Obviously, the $x_1\\cdot x_2$ feature is most helpful. "
]
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.6.9"
}
},
"nbformat": 4,
"nbformat_minor": 4
}