{ "nbformat": 4, "nbformat_minor": 0, "metadata": { "colab": { "name": "logistic_regression.ipynb", "version": "0.3.2", "views": {}, "default_view": {}, "provenance": [], "collapsed_sections": [ "dPpJUV862FYI", "i2e3TlyL57Qs", "wCugvl0JdWYL", "copyright-notice" ] } }, "cells": [ { "metadata": { "colab_type": "text", "id": "copyright-notice" }, "source": [ "#### Copyright 2017 Google LLC." ], "cell_type": "markdown" }, { "outputs": [], "metadata": { "colab_type": "code", "id": "copyright-notice2", "colab": { "autoexec": { "startup": false, "wait_interval": 0 } }, "cellView": "both" }, "source": [ "# Licensed under the Apache License, Version 2.0 (the \"License\");\n", "# you may not use this file except in compliance with the License.\n", "# You may obtain a copy of the License at\n", "#\n", "# https://www.apache.org/licenses/LICENSE-2.0\n", "#\n", "# Unless required by applicable law or agreed to in writing, software\n", "# distributed under the License is distributed on an \"AS IS\" BASIS,\n", "# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n", "# See the License for the specific language governing permissions and\n", "# limitations under the License." ], "cell_type": "code", "execution_count": 0 }, { "metadata": { "id": "g4T-_IsVbweU", "colab_type": "text" }, "cell_type": "markdown", "source": [ " # R\u00e9gression logistique" ] }, { "metadata": { "id": "LEAHZv4rIYHX", "colab_type": "text" }, "cell_type": "markdown", "source": [ " **Objectifs d'apprentissage\u00a0:**\n", " * Reconstruire le pr\u00e9dicteur du prix m\u00e9dian des logements (sur la base des exercices pr\u00e9c\u00e9dents) sous la forme d'un mod\u00e8le de classification binaire\n", " * Comparer l'efficacit\u00e9 de la r\u00e9gression logistique par rapport \u00e0 la r\u00e9gression lin\u00e9aire pour un probl\u00e8me de classification binaire" ] }, { "metadata": { "id": "CnkCZqdIIYHY", "colab_type": "text" }, "cell_type": "markdown", "source": [ " Comme pour les exercices pr\u00e9c\u00e9dents, vous allez utiliser l'ensemble de donn\u00e9es sur l'immobilier en Californie. Cette fois, cependant, vous allez cr\u00e9er un probl\u00e8me de classification binaire en pr\u00e9disant la chert\u00e9 d'un \u00eelot urbain. Vous allez \u00e9galement r\u00e9tablir les caract\u00e9ristiques par d\u00e9faut (pour l'instant)." ] }, { "metadata": { "id": "9pltCyy2K3dd", "colab_type": "text" }, "cell_type": "markdown", "source": [ " ## Pr\u00e9senter le probl\u00e8me en tant que classification binaire\n", "\n", "La cible de cet ensemble de donn\u00e9es est le prix m\u00e9dian des logements (`median_house_value`), qui est une caract\u00e9ristique num\u00e9rique (\u00e0 valeur continue). Vous pouvez cr\u00e9er une \u00e9tiquette bool\u00e9enne en appliquant un seuil \u00e0 cette valeur continue.\n", "\n", "Compte tenu des caract\u00e9ristiques qui d\u00e9crivent un \u00eelot urbain, vous souhaitez en pr\u00e9dire la chert\u00e9. Afin de pr\u00e9parer les cibles pour les donn\u00e9es d'apprentissage et d'\u00e9valuation, vous allez d\u00e9finir un seuil de classification du 75e\u00a0percentile pour le prix m\u00e9dian des logements (une valeur d'environ 265\u00a0000). Toutes les valeurs sup\u00e9rieures \u00e0 ce seuil re\u00e7oivent l'\u00e9tiquette `1`, tandis que les autres se voient affecter l'\u00e9tiquette `0`." ] }, { "metadata": { "id": "67IJwZX1Vvjt", "colab_type": "text" }, "cell_type": "markdown", "source": [ " ## Configuration\n", "\n", "Ex\u00e9cutez les cellules ci-dessous pour charger les donn\u00e9es, et pr\u00e9parer les caract\u00e9ristiques d'entr\u00e9e et les cibles." ] }, { "metadata": { "id": "fOlbcJ4EIYHd", "colab_type": "code", "colab": { "autoexec": { "startup": false, "wait_interval": 0 } } }, "source": [ "from __future__ import print_function\n", "\n", "import math\n", "\n", "from IPython import display\n", "from matplotlib import cm\n", "from matplotlib import gridspec\n", "from matplotlib import pyplot as plt\n", "import numpy as np\n", "import pandas as pd\n", "from sklearn import metrics\n", "import tensorflow as tf\n", "from tensorflow.python.data import Dataset\n", "\n", "tf.logging.set_verbosity(tf.logging.ERROR)\n", "pd.options.display.max_rows = 10\n", "pd.options.display.float_format = '{:.1f}'.format\n", "\n", "california_housing_dataframe = pd.read_csv(\"https://storage.googleapis.com/mledu-datasets/california_housing_train.csv\", sep=\",\")\n", "\n", "california_housing_dataframe = california_housing_dataframe.reindex(\n", " np.random.permutation(california_housing_dataframe.index))" ], "cell_type": "code", "execution_count": 0, "outputs": [] }, { "metadata": { "id": "lTB73MNeIYHf", "colab_type": "text" }, "cell_type": "markdown", "source": [ " Vous remarquerez que le code ci-dessous diff\u00e8re l\u00e9g\u00e8rement de celui des exercices pr\u00e9c\u00e9dents. Au lieu d'utiliser `median_house_value` comme cible, vous allez cr\u00e9er une cible binaire\u00a0: `median_house_value_is_high`." ] }, { "metadata": { "id": "kPSqspaqIYHg", "colab_type": "code", "colab": { "autoexec": { "startup": false, "wait_interval": 0 } } }, "source": [ "def preprocess_features(california_housing_dataframe):\n", " \"\"\"Prepares input features from California housing data set.\n", "\n", " Args:\n", " california_housing_dataframe: A Pandas DataFrame expected to contain data\n", " from the California housing data set.\n", " Returns:\n", " A DataFrame that contains the features to be used for the model, including\n", " synthetic features.\n", " \"\"\"\n", " selected_features = california_housing_dataframe[\n", " [\"latitude\",\n", " \"longitude\",\n", " \"housing_median_age\",\n", " \"total_rooms\",\n", " \"total_bedrooms\",\n", " \"population\",\n", " \"households\",\n", " \"median_income\"]]\n", " processed_features = selected_features.copy()\n", " # Create a synthetic feature.\n", " processed_features[\"rooms_per_person\"] = (\n", " california_housing_dataframe[\"total_rooms\"] /\n", " california_housing_dataframe[\"population\"])\n", " return processed_features\n", "\n", "def preprocess_targets(california_housing_dataframe):\n", " \"\"\"Prepares target features (i.e., labels) from California housing data set.\n", "\n", " Args:\n", " california_housing_dataframe: A Pandas DataFrame expected to contain data\n", " from the California housing data set.\n", " Returns:\n", " A DataFrame that contains the target feature.\n", " \"\"\"\n", " output_targets = pd.DataFrame()\n", " # Create a boolean categorical feature representing whether the\n", " # medianHouseValue is above a set threshold.\n", " output_targets[\"median_house_value_is_high\"] = (\n", " california_housing_dataframe[\"median_house_value\"] > 265000).astype(float)\n", " return output_targets" ], "cell_type": "code", "execution_count": 0, "outputs": [] }, { "metadata": { "id": "FwOYWmXqWA6D", "colab_type": "code", "colab": { "autoexec": { "startup": false, "wait_interval": 0 } } }, "source": [ "# Choose the first 12000 (out of 17000) examples for training.\n", "training_examples = preprocess_features(california_housing_dataframe.head(12000))\n", "training_targets = preprocess_targets(california_housing_dataframe.head(12000))\n", "\n", "# Choose the last 5000 (out of 17000) examples for validation.\n", "validation_examples = preprocess_features(california_housing_dataframe.tail(5000))\n", "validation_targets = preprocess_targets(california_housing_dataframe.tail(5000))\n", "\n", "# Double-check that we've done the right thing.\n", "print(\"Training examples summary:\")\n", "display.display(training_examples.describe())\n", "print(\"Validation examples summary:\")\n", "display.display(validation_examples.describe())\n", "\n", "print(\"Training targets summary:\")\n", "display.display(training_targets.describe())\n", "print(\"Validation targets summary:\")\n", "display.display(validation_targets.describe())" ], "cell_type": "code", "execution_count": 0, "outputs": [] }, { "metadata": { "id": "uon1LB3A31VN", "colab_type": "text" }, "cell_type": "markdown", "source": [ " ## Comment fonctionne une r\u00e9gression lin\u00e9aire\u00a0?\n", "Pour comprendre l'efficacit\u00e9 de la r\u00e9gression logistique, commencez par entra\u00eener un mod\u00e8le na\u00eff qui utilise la r\u00e9gression lin\u00e9aire. Ce mod\u00e8le utilisera des \u00e9tiquettes avec des valeurs comprises dans l'ensemble `{0, 1}` et tentera de pr\u00e9dire une valeur continue aussi proche que possible de `0` ou `1`. Puisque vous souhaitez \u00e9galement interpr\u00e9ter la sortie comme une probabilit\u00e9, l'id\u00e9al serait que la sortie se situe dans l'intervalle `(0, 1)`. Ensuite, un seuil de `0.5` sera appliqu\u00e9 pour d\u00e9terminer l'\u00e9tiquette.\n", "\n", "Ex\u00e9cutez les cellules ci-dessous pour entra\u00eener le mod\u00e8le de r\u00e9gression lin\u00e9aire \u00e0 l'aide de la classe [LinearRegressor](https://www.tensorflow.org/api_docs/python/tf/contrib/learn/LinearRegressor)." ] }, { "metadata": { "id": "smmUYRDtWOV_", "colab_type": "code", "colab": { "autoexec": { "startup": false, "wait_interval": 0 } } }, "source": [ "def construct_feature_columns(input_features):\n", " \"\"\"Construct the TensorFlow Feature Columns.\n", "\n", " Args:\n", " input_features: The names of the numerical input features to use.\n", " Returns:\n", " A set of feature columns\n", " \"\"\"\n", " return set([tf.feature_column.numeric_column(my_feature)\n", " for my_feature in input_features])" ], "cell_type": "code", "execution_count": 0, "outputs": [] }, { "metadata": { "id": "B5OwSrr1yIKD", "colab_type": "code", "colab": { "autoexec": { "startup": false, "wait_interval": 0 } } }, "source": [ "def my_input_fn(features, targets, batch_size=1, shuffle=True, num_epochs=None):\n", " \"\"\"Trains a linear regression model of one feature.\n", " \n", " Args:\n", " features: pandas DataFrame of features\n", " targets: pandas DataFrame of targets\n", " batch_size: Size of batches to be passed to the model\n", " shuffle: True or False. Whether to shuffle the data.\n", " num_epochs: Number of epochs for which data should be repeated. None = repeat indefinitely\n", " Returns:\n", " Tuple of (features, labels) for next data batch\n", " \"\"\"\n", " \n", " # Convert pandas data into a dict of np arrays.\n", " features = {key:np.array(value) for key,value in dict(features).items()} \n", " \n", " # Construct a dataset, and configure batching/repeating\n", " ds = Dataset.from_tensor_slices((features,targets)) # warning: 2GB limit\n", " ds = ds.batch(batch_size).repeat(num_epochs)\n", " \n", " # Shuffle the data, if specified\n", " if shuffle:\n", " ds = ds.shuffle(10000)\n", " \n", " # Return the next batch of data\n", " features, labels = ds.make_one_shot_iterator().get_next()\n", " return features, labels" ], "cell_type": "code", "execution_count": 0, "outputs": [] }, { "metadata": { "id": "SE2-hq8PIYHz", "colab_type": "code", "colab": { "autoexec": { "startup": false, "wait_interval": 0 } } }, "source": [ "def train_linear_regressor_model(\n", " learning_rate,\n", " steps,\n", " batch_size,\n", " training_examples,\n", " training_targets,\n", " validation_examples,\n", " validation_targets):\n", " \"\"\"Trains a linear regression model.\n", " \n", " In addition to training, this function also prints training progress information,\n", " as well as a plot of the training and validation loss over time.\n", " \n", " Args:\n", " learning_rate: A `float`, the learning rate.\n", " steps: A non-zero `int`, the total number of training steps. A training step\n", " consists of a forward and backward pass using a single batch.\n", " batch_size: A non-zero `int`, the batch size.\n", " training_examples: A `DataFrame` containing one or more columns from\n", " `california_housing_dataframe` to use as input features for training.\n", " training_targets: A `DataFrame` containing exactly one column from\n", " `california_housing_dataframe` to use as target for training.\n", " validation_examples: A `DataFrame` containing one or more columns from\n", " `california_housing_dataframe` to use as input features for validation.\n", " validation_targets: A `DataFrame` containing exactly one column from\n", " `california_housing_dataframe` to use as target for validation.\n", " \n", " Returns:\n", " A `LinearRegressor` object trained on the training data.\n", " \"\"\"\n", "\n", " periods = 10\n", " steps_per_period = steps / periods\n", "\n", " # Create a linear regressor object.\n", " my_optimizer = tf.train.GradientDescentOptimizer(learning_rate=learning_rate)\n", " my_optimizer = tf.contrib.estimator.clip_gradients_by_norm(my_optimizer, 5.0)\n", " linear_regressor = tf.estimator.LinearRegressor(\n", " feature_columns=construct_feature_columns(training_examples),\n", " optimizer=my_optimizer\n", " )\n", " \n", " # Create input functions \n", " training_input_fn = lambda: my_input_fn(training_examples, \n", " training_targets[\"median_house_value_is_high\"], \n", " batch_size=batch_size)\n", " predict_training_input_fn = lambda: my_input_fn(training_examples, \n", " training_targets[\"median_house_value_is_high\"], \n", " num_epochs=1, \n", " shuffle=False)\n", " predict_validation_input_fn = lambda: my_input_fn(validation_examples, \n", " validation_targets[\"median_house_value_is_high\"], \n", " num_epochs=1, \n", " shuffle=False)\n", "\n", " # Train the model, but do so inside a loop so that we can periodically assess\n", " # loss metrics.\n", " print(\"Training model...\")\n", " print(\"RMSE (on training data):\")\n", " training_rmse = []\n", " validation_rmse = []\n", " for period in range (0, periods):\n", " # Train the model, starting from the prior state.\n", " linear_regressor.train(\n", " input_fn=training_input_fn,\n", " steps=steps_per_period\n", " )\n", " \n", " # Take a break and compute predictions.\n", " training_predictions = linear_regressor.predict(input_fn=predict_training_input_fn)\n", " training_predictions = np.array([item['predictions'][0] for item in training_predictions])\n", " \n", " validation_predictions = linear_regressor.predict(input_fn=predict_validation_input_fn)\n", " validation_predictions = np.array([item['predictions'][0] for item in validation_predictions])\n", " \n", " # Compute training and validation loss.\n", " training_root_mean_squared_error = math.sqrt(\n", " metrics.mean_squared_error(training_predictions, training_targets))\n", " validation_root_mean_squared_error = math.sqrt(\n", " metrics.mean_squared_error(validation_predictions, validation_targets))\n", " # Occasionally print the current loss.\n", " print(\" period %02d : %0.2f\" % (period, training_root_mean_squared_error))\n", " # Add the loss metrics from this period to our list.\n", " training_rmse.append(training_root_mean_squared_error)\n", " validation_rmse.append(validation_root_mean_squared_error)\n", " print(\"Model training finished.\")\n", " \n", " # Output a graph of loss metrics over periods.\n", " plt.ylabel(\"RMSE\")\n", " plt.xlabel(\"Periods\")\n", " plt.title(\"Root Mean Squared Error vs. Periods\")\n", " plt.tight_layout()\n", " plt.plot(training_rmse, label=\"training\")\n", " plt.plot(validation_rmse, label=\"validation\")\n", " plt.legend()\n", "\n", " return linear_regressor" ], "cell_type": "code", "execution_count": 0, "outputs": [] }, { "metadata": { "id": "TDBD8xeeIYH2", "colab_type": "code", "colab": { "autoexec": { "startup": false, "wait_interval": 0 } } }, "source": [ "linear_regressor = train_linear_regressor_model(\n", " learning_rate=0.000001,\n", " steps=200,\n", " batch_size=20,\n", " training_examples=training_examples,\n", " training_targets=training_targets,\n", " validation_examples=validation_examples,\n", " validation_targets=validation_targets)" ], "cell_type": "code", "execution_count": 0, "outputs": [] }, { "metadata": { "id": "JjBZ_q7aD9gh", "colab_type": "text" }, "cell_type": "markdown", "source": [ " ## T\u00e2che\u00a01\u00a0: Est-il possible de calculer la perte logistique pour ces pr\u00e9dictions\u00a0?\n", "\n", "**Examinez les pr\u00e9dictions et d\u00e9terminez si elles peuvent \u00eatre utilis\u00e9es pour le calcul de la perte logistique.**\n", "\n", "Le co\u00fbt\u00a0L2 utilis\u00e9 par `LinearRegressor` ne s'av\u00e8re pas tr\u00e8s efficace pour p\u00e9naliser les classifications erron\u00e9es lorsque la sortie est interpr\u00e9t\u00e9e comme une probabilit\u00e9. Ainsi, il devrait y avoir une \u00e9norme diff\u00e9rence selon qu'un exemple n\u00e9gatif est class\u00e9 comme positif avec une probabilit\u00e9 de 0,9 ou de 0,9999. Cependant, avec le co\u00fbt\u00a0L2, la diff\u00e9renciation n'est pas tr\u00e8s nette.\n", "\n", "En revanche, `LogLoss` p\u00e9nalise beaucoup plus fortement ces \"erreurs de confiance\". Pour rappel, `LogLoss` se d\u00e9finit comme suit\u00a0:\n", "\n", "$$Log Loss = \\sum_{(x,y)\\in D} -y \\cdot log(y_{pred}) - (1 - y) \\cdot log(1 - y_{pred})$$\n", "\n", "\n", "Avant toute chose, il est n\u00e9cessaire d'obtenir les valeurs de pr\u00e9diction. Pour ce faire, vous pouvez utiliser `LinearRegressor.predict`.\n", "\n", "Compte tenu des pr\u00e9dictions et des cibles, est-il possible de calculer `LogLoss`\u00a0?" ] }, { "metadata": { "id": "dPpJUV862FYI", "colab_type": "text" }, "cell_type": "markdown", "source": [ " ### Solution\n", "\n", "Cliquez ci-dessous pour afficher la solution." ] }, { "metadata": { "id": "kXFQ5uig2RoP", "colab_type": "code", "colab": { "autoexec": { "startup": false, "wait_interval": 0 } } }, "source": [ "predict_validation_input_fn = lambda: my_input_fn(validation_examples, \n", " validation_targets[\"median_house_value_is_high\"], \n", " num_epochs=1, \n", " shuffle=False)\n", "\n", "validation_predictions = linear_regressor.predict(input_fn=predict_validation_input_fn)\n", "validation_predictions = np.array([item['predictions'][0] for item in validation_predictions])\n", "\n", "_ = plt.hist(validation_predictions)" ], "cell_type": "code", "execution_count": 0, "outputs": [] }, { "metadata": { "id": "rYpy336F9wBg", "colab_type": "text" }, "cell_type": "markdown", "source": [ " ## T\u00e2che\u00a02\u00a0: Entra\u00eener un mod\u00e8le de r\u00e9gression logistique et calculer la perte logistique sur l'ensemble de validation\n", "\n", "Pour utiliser la r\u00e9gression logistique, remplacez simplement le mod\u00e8le `LinearRegressor` par [LinearClassifier](https://www.tensorflow.org/api_docs/python/tf/estimator/LinearClassifier). Ex\u00e9cutez le code ci-dessous.\n", "\n", "**REMARQUE**\u00a0: Lorsque vous ex\u00e9cutez `train()` et `predict()` sur un mod\u00e8le `LinearClassifier`, vous pouvez acc\u00e9der aux valeurs num\u00e9riques r\u00e9elles des probabilit\u00e9s pr\u00e9dites au moyen de la cl\u00e9 `\"probabilities\"` du dictionnaire renvoy\u00e9\u00a0: `predictions[\"probabilities\"]`, par exemple. La fonction [log_loss](http://scikit-learn.org/stable/modules/generated/sklearn.metrics.log_loss.html) de Sklearn est pratique pour calculer la perte logistique \u00e0 l'aide de ces probabilit\u00e9s.\n", "" ] }, { "metadata": { "id": "JElcb--E9wBm", "colab_type": "code", "colab": { "autoexec": { "startup": false, "wait_interval": 0 } } }, "source": [ "def train_linear_classifier_model(\n", " learning_rate,\n", " steps,\n", " batch_size,\n", " training_examples,\n", " training_targets,\n", " validation_examples,\n", " validation_targets):\n", " \"\"\"Trains a linear regression model of one feature.\n", " \n", " In addition to training, this function also prints training progress information,\n", " as well as a plot of the training and validation loss over time.\n", " \n", " Args:\n", " learning_rate: A `float`, the learning rate.\n", " steps: A non-zero `int`, the total number of training steps. A training step\n", " consists of a forward and backward pass using a single batch.\n", " batch_size: A non-zero `int`, the batch size.\n", " training_examples: A `DataFrame` containing one or more columns from\n", " `california_housing_dataframe` to use as input features for training.\n", " training_targets: A `DataFrame` containing exactly one column from\n", " `california_housing_dataframe` to use as target for training.\n", " validation_examples: A `DataFrame` containing one or more columns from\n", " `california_housing_dataframe` to use as input features for validation.\n", " validation_targets: A `DataFrame` containing exactly one column from\n", " `california_housing_dataframe` to use as target for validation.\n", " \n", " Returns:\n", " A `LinearClassifier` object trained on the training data.\n", " \"\"\"\n", "\n", " periods = 10\n", " steps_per_period = steps / periods\n", " \n", " # Create a linear classifier object.\n", " my_optimizer = tf.train.GradientDescentOptimizer(learning_rate=learning_rate)\n", " my_optimizer = tf.contrib.estimator.clip_gradients_by_norm(my_optimizer, 5.0)\n", " linear_classifier = # YOUR CODE HERE: Construct the linear classifier.\n", " \n", " # Create input functions\n", " training_input_fn = lambda: my_input_fn(training_examples, \n", " training_targets[\"median_house_value_is_high\"], \n", " batch_size=batch_size)\n", " predict_training_input_fn = lambda: my_input_fn(training_examples, \n", " training_targets[\"median_house_value_is_high\"], \n", " num_epochs=1, \n", " shuffle=False)\n", " predict_validation_input_fn = lambda: my_input_fn(validation_examples, \n", " validation_targets[\"median_house_value_is_high\"], \n", " num_epochs=1, \n", " shuffle=False)\n", " \n", " # Train the model, but do so inside a loop so that we can periodically assess\n", " # loss metrics.\n", " print(\"Training model...\")\n", " print(\"LogLoss (on training data):\")\n", " training_log_losses = []\n", " validation_log_losses = []\n", " for period in range (0, periods):\n", " # Train the model, starting from the prior state.\n", " linear_classifier.train(\n", " input_fn=training_input_fn,\n", " steps=steps_per_period\n", " )\n", " # Take a break and compute predictions. \n", " training_probabilities = linear_classifier.predict(input_fn=predict_training_input_fn)\n", " training_probabilities = np.array([item['probabilities'] for item in training_probabilities])\n", " \n", " validation_probabilities = linear_classifier.predict(input_fn=predict_validation_input_fn)\n", " validation_probabilities = np.array([item['probabilities'] for item in validation_probabilities])\n", " \n", " training_log_loss = metrics.log_loss(training_targets, training_probabilities)\n", " validation_log_loss = metrics.log_loss(validation_targets, validation_probabilities)\n", " # Occasionally print the current loss.\n", " print(\" period %02d : %0.2f\" % (period, training_log_loss))\n", " # Add the loss metrics from this period to our list.\n", " training_log_losses.append(training_log_loss)\n", " validation_log_losses.append(validation_log_loss)\n", " print(\"Model training finished.\")\n", " \n", " # Output a graph of loss metrics over periods.\n", " plt.ylabel(\"LogLoss\")\n", " plt.xlabel(\"Periods\")\n", " plt.title(\"LogLoss vs. Periods\")\n", " plt.tight_layout()\n", " plt.plot(training_log_losses, label=\"training\")\n", " plt.plot(validation_log_losses, label=\"validation\")\n", " plt.legend()\n", "\n", " return linear_classifier\n", "\n", "linear_classifier = train_linear_classifier_model(\n", " learning_rate=0.000005,\n", " steps=500,\n", " batch_size=20,\n", " training_examples=training_examples,\n", " training_targets=training_targets,\n", " validation_examples=validation_examples,\n", " validation_targets=validation_targets)" ], "cell_type": "code", "execution_count": 0, "outputs": [] }, { "metadata": { "id": "VM0wmnFUIYH9", "colab_type": "code", "colab": { "autoexec": { "startup": false, "wait_interval": 0 } } }, "source": [ "linear_classifier = train_linear_classifier_model(\n", " learning_rate=0.000005,\n", " steps=500,\n", " batch_size=20,\n", " training_examples=training_examples,\n", " training_targets=training_targets,\n", " validation_examples=validation_examples,\n", " validation_targets=validation_targets)" ], "cell_type": "code", "execution_count": 0, "outputs": [] }, { "metadata": { "id": "i2e3TlyL57Qs", "colab_type": "text" }, "cell_type": "markdown", "source": [ " ### Solution\n", "\n", "Cliquez ci-dessous pour afficher la solution.\n", "\n", "" ] }, { "metadata": { "id": "5YxXd2hn6MuF", "colab_type": "code", "colab": { "autoexec": { "startup": false, "wait_interval": 0 } } }, "source": [ "def train_linear_classifier_model(\n", " learning_rate,\n", " steps,\n", " batch_size,\n", " training_examples,\n", " training_targets,\n", " validation_examples,\n", " validation_targets):\n", " \"\"\"Trains a linear regression model of one feature.\n", " \n", " In addition to training, this function also prints training progress information,\n", " as well as a plot of the training and validation loss over time.\n", " \n", " Args:\n", " learning_rate: A `float`, the learning rate.\n", " steps: A non-zero `int`, the total number of training steps. A training step\n", " consists of a forward and backward pass using a single batch.\n", " batch_size: A non-zero `int`, the batch size.\n", " training_examples: A `DataFrame` containing one or more columns from\n", " `california_housing_dataframe` to use as input features for training.\n", " training_targets: A `DataFrame` containing exactly one column from\n", " `california_housing_dataframe` to use as target for training.\n", " validation_examples: A `DataFrame` containing one or more columns from\n", " `california_housing_dataframe` to use as input features for validation.\n", " validation_targets: A `DataFrame` containing exactly one column from\n", " `california_housing_dataframe` to use as target for validation.\n", " \n", " Returns:\n", " A `LinearClassifier` object trained on the training data.\n", " \"\"\"\n", "\n", " periods = 10\n", " steps_per_period = steps / periods\n", " \n", " # Create a linear classifier object.\n", " my_optimizer = tf.train.GradientDescentOptimizer(learning_rate=learning_rate)\n", " my_optimizer = tf.contrib.estimator.clip_gradients_by_norm(my_optimizer, 5.0) \n", " linear_classifier = tf.estimator.LinearClassifier(\n", " feature_columns=construct_feature_columns(training_examples),\n", " optimizer=my_optimizer\n", " )\n", " \n", " # Create input functions\n", " training_input_fn = lambda: my_input_fn(training_examples, \n", " training_targets[\"median_house_value_is_high\"], \n", " batch_size=batch_size)\n", " predict_training_input_fn = lambda: my_input_fn(training_examples, \n", " training_targets[\"median_house_value_is_high\"], \n", " num_epochs=1, \n", " shuffle=False)\n", " predict_validation_input_fn = lambda: my_input_fn(validation_examples, \n", " validation_targets[\"median_house_value_is_high\"], \n", " num_epochs=1, \n", " shuffle=False)\n", " \n", " # Train the model, but do so inside a loop so that we can periodically assess\n", " # loss metrics.\n", " print(\"Training model...\")\n", " print(\"LogLoss (on training data):\")\n", " training_log_losses = []\n", " validation_log_losses = []\n", " for period in range (0, periods):\n", " # Train the model, starting from the prior state.\n", " linear_classifier.train(\n", " input_fn=training_input_fn,\n", " steps=steps_per_period\n", " )\n", " # Take a break and compute predictions. \n", " training_probabilities = linear_classifier.predict(input_fn=predict_training_input_fn)\n", " training_probabilities = np.array([item['probabilities'] for item in training_probabilities])\n", " \n", " validation_probabilities = linear_classifier.predict(input_fn=predict_validation_input_fn)\n", " validation_probabilities = np.array([item['probabilities'] for item in validation_probabilities])\n", " \n", " training_log_loss = metrics.log_loss(training_targets, training_probabilities)\n", " validation_log_loss = metrics.log_loss(validation_targets, validation_probabilities)\n", " # Occasionally print the current loss.\n", " print(\" period %02d : %0.2f\" % (period, training_log_loss))\n", " # Add the loss metrics from this period to our list.\n", " training_log_losses.append(training_log_loss)\n", " validation_log_losses.append(validation_log_loss)\n", " print(\"Model training finished.\")\n", " \n", " # Output a graph of loss metrics over periods.\n", " plt.ylabel(\"LogLoss\")\n", " plt.xlabel(\"Periods\")\n", " plt.title(\"LogLoss vs. Periods\")\n", " plt.tight_layout()\n", " plt.plot(training_log_losses, label=\"training\")\n", " plt.plot(validation_log_losses, label=\"validation\")\n", " plt.legend()\n", "\n", " return linear_classifier" ], "cell_type": "code", "execution_count": 0, "outputs": [] }, { "metadata": { "id": "UPM_T1FXsTaL", "colab_type": "code", "colab": { "autoexec": { "startup": false, "wait_interval": 0 } } }, "source": [ "linear_classifier = train_linear_classifier_model(\n", " learning_rate=0.000005,\n", " steps=500,\n", " batch_size=20,\n", " training_examples=training_examples,\n", " training_targets=training_targets,\n", " validation_examples=validation_examples,\n", " validation_targets=validation_targets)" ], "cell_type": "code", "execution_count": 0, "outputs": [] }, { "metadata": { "id": "i-Xo83_aR6s_", "colab_type": "text" }, "cell_type": "markdown", "source": [ " ## T\u00e2che\u00a03\u00a0: Calculer la justesse et repr\u00e9senter graphiquement une courbe ROC pour l'ensemble de validation\n", "\n", "La [justesse](https://en.wikipedia.org/wiki/Accuracy_and_precision#In_binary_classification) du mod\u00e8le, la [courbe ROC] (https://fr.wikipedia.org/wiki/Courbe_ROC) et la zone sous la courbe ROC (AUC) figurent parmi les mesures utiles pour la classification. Nous allons les examiner.\n", "\n", "`LinearClassifier.evaluate` calcule des mesures utiles telles que la justesse et l'AUC." ] }, { "metadata": { "id": "DKSQ87VVIYIA", "colab_type": "code", "colab": { "autoexec": { "startup": false, "wait_interval": 0 } } }, "source": [ "evaluation_metrics = linear_classifier.evaluate(input_fn=predict_validation_input_fn)\n", "\n", "print(\"AUC on the validation set: %0.2f\" % evaluation_metrics['auc'])\n", "print(\"Accuracy on the validation set: %0.2f\" % evaluation_metrics['accuracy'])" ], "cell_type": "code", "execution_count": 0, "outputs": [] }, { "metadata": { "id": "47xGS2uNIYIE", "colab_type": "text" }, "cell_type": "markdown", "source": [ " Vous pouvez utiliser des probabilit\u00e9s de classe, telles que celles calcul\u00e9es par `LinearClassifier.predict`\n", "et la fonction [roc_curve](http://scikit-learn.org/stable/modules/model_evaluation.html#roc-metrics) de Sklearn, pour\n", "obtenir les taux de vrais positifs et de vrais n\u00e9gatifs n\u00e9cessaires pour la repr\u00e9sentation graphique d'une courbe\u00a0ROC." ] }, { "metadata": { "id": "xaU7ttj8IYIF", "colab_type": "code", "colab": { "autoexec": { "startup": false, "wait_interval": 0 } } }, "source": [ "validation_probabilities = linear_classifier.predict(input_fn=predict_validation_input_fn)\n", "# Get just the probabilities for the positive class\n", "validation_probabilities = np.array([item['probabilities'][1] for item in validation_probabilities])\n", "\n", "false_positive_rate, true_positive_rate, thresholds = metrics.roc_curve(\n", " validation_targets, validation_probabilities)\n", "plt.plot(false_positive_rate, true_positive_rate, label=\"our model\")\n", "plt.plot([0, 1], [0, 1], label=\"random classifier\")\n", "_ = plt.legend(loc=2)" ], "cell_type": "code", "execution_count": 0, "outputs": [] }, { "metadata": { "id": "PIdhwfgzIYII", "colab_type": "text" }, "cell_type": "markdown", "source": [ " **Voyons maintenant s'il est possible d'optimiser les param\u00e8tres d'apprentissage du mod\u00e8le entra\u00een\u00e9 \u00e0 la T\u00e2che\u00a02 en vue d'am\u00e9liorer l'AUC.**\n", "\n", "Bien souvent, certaines mesures sont am\u00e9lior\u00e9es au d\u00e9triment d'autres. Votre r\u00f4le consiste alors \u00e0 identifier les param\u00e8tres qui offrent un bon compromis.\n", "\n", "**V\u00e9rifiez si toutes les mesures s'am\u00e9liorent en m\u00eame temps.**" ] }, { "metadata": { "id": "XKIqjsqcCaxO", "colab_type": "code", "colab": { "autoexec": { "startup": false, "wait_interval": 0 } } }, "source": [ "# TUNE THE SETTINGS BELOW TO IMPROVE AUC\n", "linear_classifier = train_linear_classifier_model(\n", " learning_rate=0.000005,\n", " steps=500,\n", " batch_size=20,\n", " training_examples=training_examples,\n", " training_targets=training_targets,\n", " validation_examples=validation_examples,\n", " validation_targets=validation_targets)\n", "\n", "evaluation_metrics = linear_classifier.evaluate(input_fn=predict_validation_input_fn)\n", "\n", "print(\"AUC on the validation set: %0.2f\" % evaluation_metrics['auc'])\n", "print(\"Accuracy on the validation set: %0.2f\" % evaluation_metrics['accuracy'])" ], "cell_type": "code", "execution_count": 0, "outputs": [] }, { "metadata": { "id": "wCugvl0JdWYL", "colab_type": "text" }, "cell_type": "markdown", "source": [ " ### Solution\n", "\n", "Cliquez ci-dessous pour afficher une solution." ] }, { "metadata": { "id": "VHosS1g2aetf", "colab_type": "text" }, "cell_type": "markdown", "source": [ " Une solution efficace consiste simplement \u00e0 allonger la dur\u00e9e d'apprentissage, \u00e0 condition de ne pas verser dans le surapprentissage. \n", "\n", "Pour ce faire, vous pouvez augmenter le nombre de pas, la taille du lot ou les deux.\n", "\n", "Toutes les mesures s'am\u00e9liorent en m\u00eame temps. Notre mesure de perte\n", "appara\u00eet donc comme un bon interm\u00e9diaire pour l'AUC et la justesse.\n", "\n", "Comme vous pouvez le constater, il faut beaucoup plus d'it\u00e9rations\n", "pour obtenir quelques unit\u00e9s d'AUC de plus. C'est une situation courante.\n", "Cependant, m\u00eame ce l\u00e9ger gain en vaut g\u00e9n\u00e9ralement la peine." ] }, { "metadata": { "id": "dWgTEYMddaA-", "colab_type": "code", "colab": { "autoexec": { "startup": false, "wait_interval": 0 } } }, "source": [ "linear_classifier = train_linear_classifier_model(\n", " learning_rate=0.000003,\n", " steps=20000,\n", " batch_size=500,\n", " training_examples=training_examples,\n", " training_targets=training_targets,\n", " validation_examples=validation_examples,\n", " validation_targets=validation_targets)\n", "\n", "evaluation_metrics = linear_classifier.evaluate(input_fn=predict_validation_input_fn)\n", "\n", "print(\"AUC on the validation set: %0.2f\" % evaluation_metrics['auc'])\n", "print(\"Accuracy on the validation set: %0.2f\" % evaluation_metrics['accuracy'])" ], "cell_type": "code", "execution_count": 0, "outputs": [] } ] }