{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Exercise 5.1\n",
    "## Overtraining and Regularization:\n",
    "Open the Tensorflow Playground\n",
    "(https://playground.tensorflow.org) and select on the left the checkerboard\n",
    "pattern as the data basis (see [Exercise 3.3](Exercise_3_3.ipynb)). As input features, select the two\n",
    "independent variables $x_1$ and $x_2$ and set the noise to $50\\%$.\n",
    "\n",
    "\n",
    "[![Checkerboard](./images/checkerboard_regularization.png)](https://playground.tensorflow.org/#activation=relu&batchSize=10&dataset=xor&regDataset=reg-plane&learningRate=0.03&regularizationRate=0&noise=50&networkShape=6,6,6,6,6&seed=0.82577&showTestData=false&discretize=false&percTrainData=50&x=true&y=true&xTimesY=false&xSquared=false&ySquared=false&cosX=false&sinX=false&cosY=false&sinY=false&collectStats=false&problem=classification&initZero=false&hideText=true)\n",
    "\n",
    "\n",
    "## Tasks\n",
    "1. Choose a deep (many layers) and wide (many nodes) network and train it for more than 1000 epochs. Comment on your observations.\n",
    "2. Apply L2 regularization to reduce overfitting. Try low and high regularization rates. What do you observe?\n",
    "3. Compare the effects of L1 and L2 regularization."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.9"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}