{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "# Fine-tuning keras models\n", "> Learn how to optimize your deep learning models in Keras. Start by learning how to validate your models, then understand the concept of model capacity, and finally, experiment with wider and deeper networks. This is the Summary of lecture \"Introduction to Deep Learning in Python\", via datacamp.\n", "\n", "- toc: true \n", "- badges: true\n", "- comments: true\n", "- author: Chanseok Kang\n", "- categories: [Python, Datacamp, Tensorflow-Keras, Deep_Learning]\n", "- image: images/of.png" ] }, { "cell_type": "code", "execution_count": 18, "metadata": {}, "outputs": [], "source": [ "import numpy as np\n", "import pandas as pd\n", "import tensorflow as tf\n", "import matplotlib.pyplot as plt\n", "\n", "plt.rcParams['figure.figsize'] = (8, 8)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Understanding model optimization\n", " - Why optimization is hard\n", " - Simultaneously optimizing 1000s of parameters with complex relationships\n", " - Updates may not improve model meaningfully\n", " - Updates too small (if learning rate is low) or too large (if learning rate is high)\n", " - Vanishing gradients\n", " - Occurs when many layers have very small slopes (e.g. due to being on flat part of tanh curve)\n", " - In deep networks, updates to backprop were close to 0" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Changing optimization parameters\n", "It's time to get your hands dirty with optimization. You'll now try optimizing a model at a very low learning rate, a very high learning rate, and a \"just right\" learning rate. You'll want to look at the results after running this exercise, remembering that a low value for the loss function is good.\n", "\n", "For these exercises, we've pre-loaded the predictors and target values from your previous classification models (predicting who would survive on the Titanic). You'll want the optimization to start from scratch every time you change the learning rate, to give a fair comparison of how each learning rate did in your results." ] }, { "cell_type": "code", "execution_count": 4, "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", " | survived | \n", "pclass | \n", "age | \n", "sibsp | \n", "parch | \n", "fare | \n", "male | \n", "age_was_missing | \n", "embarked_from_cherbourg | \n", "embarked_from_queenstown | \n", "embarked_from_southampton | \n", "
---|---|---|---|---|---|---|---|---|---|---|---|
0 | \n", "0 | \n", "3 | \n", "22.0 | \n", "1 | \n", "0 | \n", "7.2500 | \n", "1 | \n", "False | \n", "0 | \n", "0 | \n", "1 | \n", "
1 | \n", "1 | \n", "1 | \n", "38.0 | \n", "1 | \n", "0 | \n", "71.2833 | \n", "0 | \n", "False | \n", "1 | \n", "0 | \n", "0 | \n", "
2 | \n", "1 | \n", "3 | \n", "26.0 | \n", "0 | \n", "0 | \n", "7.9250 | \n", "0 | \n", "False | \n", "0 | \n", "0 | \n", "1 | \n", "
3 | \n", "1 | \n", "1 | \n", "35.0 | \n", "1 | \n", "0 | \n", "53.1000 | \n", "0 | \n", "False | \n", "0 | \n", "0 | \n", "1 | \n", "
4 | \n", "0 | \n", "3 | \n", "35.0 | \n", "0 | \n", "0 | \n", "8.0500 | \n", "1 | \n", "False | \n", "0 | \n", "0 | \n", "1 | \n", "
\n", " | 0 | \n", "1 | \n", "2 | \n", "3 | \n", "4 | \n", "5 | \n", "6 | \n", "7 | \n", "8 | \n", "9 | \n", "... | \n", "775 | \n", "776 | \n", "777 | \n", "778 | \n", "779 | \n", "780 | \n", "781 | \n", "782 | \n", "783 | \n", "784 | \n", "
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
0 | \n", "5 | \n", "0 | \n", "0.1 | \n", "0.2 | \n", "0.3 | \n", "0.4 | \n", "0.5 | \n", "0.6 | \n", "0.7 | \n", "0.8 | \n", "... | \n", "0.608 | \n", "0.609 | \n", "0.61 | \n", "0.611 | \n", "0.612 | \n", "0.613 | \n", "0.614 | \n", "0.615 | \n", "0.616 | \n", "0.617 | \n", "
1 | \n", "4 | \n", "0 | \n", "0.0 | \n", "0.0 | \n", "0.0 | \n", "0.0 | \n", "0.0 | \n", "0.0 | \n", "0.0 | \n", "0.0 | \n", "... | \n", "0.000 | \n", "0.000 | \n", "0.00 | \n", "0.000 | \n", "0.000 | \n", "0.000 | \n", "0.000 | \n", "0.000 | \n", "0.000 | \n", "0.000 | \n", "
2 | \n", "3 | \n", "0 | \n", "0.0 | \n", "0.0 | \n", "0.0 | \n", "0.0 | \n", "0.0 | \n", "0.0 | \n", "0.0 | \n", "0.0 | \n", "... | \n", "0.000 | \n", "0.000 | \n", "0.00 | \n", "0.000 | \n", "0.000 | \n", "0.000 | \n", "0.000 | \n", "0.000 | \n", "0.000 | \n", "0.000 | \n", "
3 | \n", "0 | \n", "0 | \n", "0.0 | \n", "0.0 | \n", "0.0 | \n", "0.0 | \n", "0.0 | \n", "0.0 | \n", "0.0 | \n", "0.0 | \n", "... | \n", "0.000 | \n", "0.000 | \n", "0.00 | \n", "0.000 | \n", "0.000 | \n", "0.000 | \n", "0.000 | \n", "0.000 | \n", "0.000 | \n", "0.000 | \n", "
4 | \n", "2 | \n", "0 | \n", "0.0 | \n", "0.0 | \n", "0.0 | \n", "0.0 | \n", "0.0 | \n", "0.0 | \n", "0.0 | \n", "0.0 | \n", "... | \n", "0.000 | \n", "0.000 | \n", "0.00 | \n", "0.000 | \n", "0.000 | \n", "0.000 | \n", "0.000 | \n", "0.000 | \n", "0.000 | \n", "0.000 | \n", "
5 rows × 785 columns
\n", "