{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "# Predicting Diabetes with Keras" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### A Very Short Introduction to Hyperparameter Optimization with Talos" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The goal of a Talos experiment, is to find a set of suitable hyperparameters for Keras model. In order to do this, you need to have three things: \n", "\n", "- Keras model \n", "- Talos hyperparameter dictionary\n", "- Talos experiment configuration\n", "\n", "Below we will briefly overview each." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### The Keras Model\n", "\n", "As a model, any Keras model will do. Let's consider as an example a very simple model that makes a prediction on the classic *Pima Indians Diabetes* dataset. A brief overview of the dataset can be found [here](https://www.kaggle.com/uciml/pima-indians-diabetes-database) and the dataset we will use can be found [here](https://raw.githubusercontent.com/jbrownlee/Datasets/master/pima-indians-diabetes.data.csv). The below model does not require you to separately download the file." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "from numpy import loadtxt\n", "\n", "dataset = loadtxt(\"https://raw.githubusercontent.com/jbrownlee/Datasets/master/pima-indians-diabetes.data.csv\", delimiter=\",\")\n", "x = dataset[:,0:8]\n", "y = dataset[:, 8]" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "from tensorflow.keras.models import Sequential\n", "from tensorflow.keras.layers import Dense\n", "\n", "def diabetes():\n", " \n", " model = Sequential()\n", " model.add(Dense(12, input_dim=8, activation='relu'))\n", " model.add(Dense(8, activation='relu'))\n", " model.add(Dense(1, activation='sigmoid'))\n", " model.compile(loss='binary_crossentropy', optimizer='adam', metrics=['accuracy'])\n", " model.fit(X, Y, epochs=100, batch_size=10, verbose=0)\n", " \n", " return model" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Talos Hyperparameter Dictionary" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Let's prepare for an experiment where we will optimize against three common attributes:\n", "\n", "- neurons on the first layer\n", "- activations\n", "- batch_sizes" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "from tensorflow.keras.activations import relu, elu\n", "\n", "p = {\n", " 'first_neuron': [12, 24, 48],\n", " 'activation': ['relu', 'elu'],\n", " 'batch_size': [10, 20, 30]\n", "}" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Configuring the Keras Model for Talos" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "In order to prepare a Keras model for a Talos experiment, we need to do four things:\n", "\n", "- add input parameters to the function\n", "- replace the hyperparameter inputs with references to params dictionary\n", "- make sure model.fit() stores the history object\n", "- modify the output of the model\n", "\n", "These steps are always the same." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "# add input parameters to the function\n", "def diabetes(x_train, y_train, x_val, y_val, params):\n", " \n", " # replace the hyperparameter inputs with references to params dictionary \n", " model = Sequential()\n", " model.add(Dense(params['first_neuron'], input_dim=8, activation=params['activation']))\n", " #model.add(Dense(8, activation=params['activation']))\n", " model.add(Dense(1, activation='sigmoid'))\n", " model.compile(loss='binary_crossentropy', optimizer='adam', metrics=['accuracy'])\n", " \n", " # make sure history object is returned by model.fit()\n", " out = model.fit(x=x, \n", " y=y,\n", " validation_data=[x_val, y_val],\n", " epochs=100,\n", " batch_size=params['batch_size'],\n", " verbose=0)\n", " \n", " # modify the output model\n", " return out, model" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "That's it, there is nothing more to it. A more complicated experiment would just entail more of the same in terms of the way the params dictionary references are made. Otherwise the changes would always be exactly the same." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Talos Experiment" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The Talos experiment is performed through the Scan() command. In case you don't have Talos installed already, you can do that now." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "import talos" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "While many configurations are possible, the only things that you absolutely must input to a Talos experiment are: \n", "\n", "- x\n", "- y\n", "- params (the dictionary 'p' we created above)\n", "- model (the 'diabetes' we created above)" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "t = talos.Scan(x=x, y=y, params=p, model=diabetes, experiment_name='diabetes')" ] } ], "metadata": { "kernelspec": { "display_name": "Python 3", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.6.8" } }, "nbformat": 4, "nbformat_minor": 2 }