{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "## Preface\n", "([article source](https://machinelearningmastery.com/evaluate-performance-deep-learning-models-keras/)) **Keras is an easy to use and powerful Python library for deep learning.**\n", "\n", "**There are a lot of decisions to make when designing and configuring your deep learning models.** Most of these decisions must be resolved empirically through trial and error and evaluating them on real data.\n", "\n", "**As such, it is critically important to have a robust way to evaluate the performance of your neural networks and deep learning models. In this post you will discover a few ways that you can use to evaluate model performance using Keras.**\n", "\n", "### Agenda\n", "* [**Empirically Evaluate Network Configurations**](#sect1)\n", "* [**Data Splitting**](#sect2)\n", "* [**Manual k-Fold Cross Validation**](#sect3)" ] }, { "cell_type": "code", "execution_count": 25, "metadata": {}, "outputs": [], "source": [ "import numpy as np\n", "import pandas as pd\n", "from keras.models import Sequential\n", "from keras.layers import Dense\n", "from sklearn.model_selection import train_test_split\n", "from sklearn.model_selection import StratifiedKFold\n", "from matplotlib import pyplot as plt\n", "\n", "# fix random seed for reproducibility\n", "seed = 7\n", "\n", "# fix random seed for reproducibility\n", "np.random.seed(seed)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Data Set\n", "All examples in this post use the [Pima Indians onset of diabetes dataset](https://archive.ics.uci.edu/ml/datasets/Pima+Indians+Diabetes). You can [download it](https://archive.ics.uci.edu/ml/machine-learning-databases/pima-indians-diabetes/pima-indians-diabetes.data) from the UCI Machine Learning Repository and save the data file in your current working directory with the filename pima-indians-diabetes.csv ([update: download from here](https://www.kaggle.com/uciml/pima-indians-diabetes-database))." ] }, { "cell_type": "code", "execution_count": 6, "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", " | Pregnancies | \n", "Glucose | \n", "BloodPressure | \n", "SkinThickness | \n", "Insulin | \n", "BMI | \n", "DiabetesPedigreeFunction | \n", "Age | \n", "Outcome | \n", "
---|---|---|---|---|---|---|---|---|---|
0 | \n", "6 | \n", "148 | \n", "72 | \n", "35 | \n", "0 | \n", "33.6 | \n", "0.627 | \n", "50 | \n", "1 | \n", "
1 | \n", "1 | \n", "85 | \n", "66 | \n", "29 | \n", "0 | \n", "26.6 | \n", "0.351 | \n", "31 | \n", "0 | \n", "
2 | \n", "8 | \n", "183 | \n", "64 | \n", "0 | \n", "0 | \n", "23.3 | \n", "0.672 | \n", "32 | \n", "1 | \n", "
3 | \n", "1 | \n", "89 | \n", "66 | \n", "23 | \n", "94 | \n", "28.1 | \n", "0.167 | \n", "21 | \n", "0 | \n", "
4 | \n", "0 | \n", "137 | \n", "40 | \n", "35 | \n", "168 | \n", "43.1 | \n", "2.288 | \n", "33 | \n", "1 | \n", "