{ "nbformat": 4, "nbformat_minor": 0, "metadata": { "colab": { "name": "feature_sets.ipynb", "version": "0.3.2", "views": {}, "default_view": {}, "provenance": [], "collapsed_sections": [ "IGINhMIJ5Wyt", "pZa8miwu6_tQ", "copyright-notice" ] } }, "cells": [ { "source": [ "#### Copyright 2017 Google LLC." ], "metadata": { "id": "copyright-notice", "colab_type": "text" }, "cell_type": "markdown" }, { "outputs": [], "source": [ "# Licensed under the Apache License, Version 2.0 (the \"License\");\n", "# you may not use this file except in compliance with the License.\n", "# You may obtain a copy of the License at\n", "#\n", "# https://www.apache.org/licenses/LICENSE-2.0\n", "#\n", "# Unless required by applicable law or agreed to in writing, software\n", "# distributed under the License is distributed on an \"AS IS\" BASIS,\n", "# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n", "# See the License for the specific language governing permissions and\n", "# limitations under the License." ], "metadata": { "cellView": "both", "colab": { "autoexec": { "startup": false, "wait_interval": 0 } }, "id": "copyright-notice2", "colab_type": "code" }, "cell_type": "code", "execution_count": 0 }, { "cell_type": "markdown", "metadata": { "id": "zbIgBK-oXHO7", "colab_type": "text" }, "source": [ " # Ensembles de caract\u00e9ristiques" ] }, { "cell_type": "markdown", "metadata": { "id": "bL04rAQwH3pH", "colab_type": "text" }, "source": [ " **Objectif d'apprentissage\u00a0:** Cr\u00e9er un ensemble de caract\u00e9ristiques minimal offrant des performances identiques \u00e0 un ensemble plus complexe" ] }, { "cell_type": "markdown", "metadata": { "id": "F8Hci6tAH3pH", "colab_type": "text" }, "source": [ " Jusqu'\u00e0 pr\u00e9sent, toutes les caract\u00e9ristiques ont \u00e9t\u00e9 int\u00e9gr\u00e9es dans le mod\u00e8le. Cependant, un mod\u00e8le qui contient moins de caract\u00e9ristiques utilise moins de ressources et s'av\u00e8re plus facile \u00e0 g\u00e9rer. Voyons s'il est possible de construire un mod\u00e8le bas\u00e9 sur un ensemble minimal de caract\u00e9ristiques immobili\u00e8res, mais offrant les m\u00eames performances qu'un mod\u00e8le int\u00e9grant toutes les caract\u00e9ristiques de l'ensemble de donn\u00e9es." ] }, { "cell_type": "markdown", "metadata": { "id": "F5ZjVwK_qOyR", "colab_type": "text" }, "source": [ " ## Configuration\n", "\n", "Une fois encore, vous allez commencer par charger et pr\u00e9parer les donn\u00e9es sur l'immobilier en Californie." ] }, { "metadata": { "id": "SrOYRILAH3pJ", "colab_type": "code", "colab": { "autoexec": { "startup": false, "wait_interval": 0 } } }, "source": [ "from __future__ import print_function\n", "\n", "import math\n", "\n", "from IPython import display\n", "from matplotlib import cm\n", "from matplotlib import gridspec\n", "from matplotlib import pyplot as plt\n", "import numpy as np\n", "import pandas as pd\n", "from sklearn import metrics\n", "import tensorflow as tf\n", "from tensorflow.python.data import Dataset\n", "\n", "tf.logging.set_verbosity(tf.logging.ERROR)\n", "pd.options.display.max_rows = 10\n", "pd.options.display.float_format = '{:.1f}'.format\n", "\n", "california_housing_dataframe = pd.read_csv(\"https://storage.googleapis.com/mledu-datasets/california_housing_train.csv\", sep=\",\")\n", "\n", "california_housing_dataframe = california_housing_dataframe.reindex(\n", " np.random.permutation(california_housing_dataframe.index))" ], "cell_type": "code", "execution_count": 0, "outputs": [] }, { "metadata": { "id": "dGnXo7flH3pM", "colab_type": "code", "colab": { "autoexec": { "startup": false, "wait_interval": 0 } } }, "source": [ "def preprocess_features(california_housing_dataframe):\n", " \"\"\"Prepares input features from California housing data set.\n", "\n", " Args:\n", " california_housing_dataframe: A Pandas DataFrame expected to contain data\n", " from the California housing data set.\n", " Returns:\n", " A DataFrame that contains the features to be used for the model, including\n", " synthetic features.\n", " \"\"\"\n", " selected_features = california_housing_dataframe[\n", " [\"latitude\",\n", " \"longitude\",\n", " \"housing_median_age\",\n", " \"total_rooms\",\n", " \"total_bedrooms\",\n", " \"population\",\n", " \"households\",\n", " \"median_income\"]]\n", " processed_features = selected_features.copy()\n", " # Create a synthetic feature.\n", " processed_features[\"rooms_per_person\"] = (\n", " california_housing_dataframe[\"total_rooms\"] /\n", " california_housing_dataframe[\"population\"])\n", " return processed_features\n", "\n", "def preprocess_targets(california_housing_dataframe):\n", " \"\"\"Prepares target features (i.e., labels) from California housing data set.\n", "\n", " Args:\n", " california_housing_dataframe: A Pandas DataFrame expected to contain data\n", " from the California housing data set.\n", " Returns:\n", " A DataFrame that contains the target feature.\n", " \"\"\"\n", " output_targets = pd.DataFrame()\n", " # Scale the target to be in units of thousands of dollars.\n", " output_targets[\"median_house_value\"] = (\n", " california_housing_dataframe[\"median_house_value\"] / 1000.0)\n", " return output_targets" ], "cell_type": "code", "execution_count": 0, "outputs": [] }, { "metadata": { "id": "jLXC8y4AqsIy", "colab_type": "code", "colab": { "autoexec": { "startup": false, "wait_interval": 0 } } }, "source": [ "# Choose the first 12000 (out of 17000) examples for training.\n", "training_examples = preprocess_features(california_housing_dataframe.head(12000))\n", "training_targets = preprocess_targets(california_housing_dataframe.head(12000))\n", "\n", "# Choose the last 5000 (out of 17000) examples for validation.\n", "validation_examples = preprocess_features(california_housing_dataframe.tail(5000))\n", "validation_targets = preprocess_targets(california_housing_dataframe.tail(5000))\n", "\n", "# Double-check that we've done the right thing.\n", "print(\"Training examples summary:\")\n", "display.display(training_examples.describe())\n", "print(\"Validation examples summary:\")\n", "display.display(validation_examples.describe())\n", "\n", "print(\"Training targets summary:\")\n", "display.display(training_targets.describe())\n", "print(\"Validation targets summary:\")\n", "display.display(validation_targets.describe())" ], "cell_type": "code", "execution_count": 0, "outputs": [] }, { "cell_type": "markdown", "metadata": { "id": "hLvmkugKLany", "colab_type": "text" }, "source": [ " ## T\u00e2che\u00a01\u00a0: \u00c9laborer un ensemble de caract\u00e9ristiques de bonne qualit\u00e9\n", "\n", "**Quelles performances peut-on obtenir avec simplement 2 ou 3\u00a0caract\u00e9ristiques\u00a0?**\n", "\n", "Une **matrice de corr\u00e9lation** affiche des corr\u00e9lations par paire, dans lesquelles chaque caract\u00e9ristique est compar\u00e9e \u00e0 la cible, ainsi qu'aux autres caract\u00e9ristiques.\n", "\n", "Dans le cas pr\u00e9sent, la corr\u00e9lation est d\u00e9finie comme le [Coefficient de corr\u00e9lation de Pearson](https://fr.wikipedia.org/wiki/Corr\u00e9lation_(statistiques)#Coefficient_de_corr\u00e9lation_lin\u00e9aire_de_Bravais-Pearson). Vous ne devez pas n\u00e9cessairement comprendre tous les d\u00e9tails math\u00e9matiques pour effectuer cet exercice.\n", "\n", "La signification des valeurs de corr\u00e9lation est la suivante\u00a0:\n", "\n", " * `-1.0`\u00a0: corr\u00e9lation n\u00e9gative parfaite\n", " * `0.0`\u00a0: pas de corr\u00e9lation\n", " * `1.0`\u00a0: corr\u00e9lation positive parfaite" ] }, { "metadata": { "id": "UzoZUSdLIolF", "colab_type": "code", "colab": { "autoexec": { "startup": false, "wait_interval": 0 }, "test": { "output": "ignore", "timeout": 600 } }, "cellView": "both" }, "source": [ "correlation_dataframe = training_examples.copy()\n", "correlation_dataframe[\"target\"] = training_targets[\"median_house_value\"]\n", "\n", "correlation_dataframe.corr()" ], "cell_type": "code", "execution_count": 0, "outputs": [] }, { "cell_type": "markdown", "metadata": { "id": "RQpktkNpia2P", "colab_type": "text" }, "source": [ " Id\u00e9alement, les caract\u00e9ristiques devraient entretenir une forte corr\u00e9lation avec la cible.\n", "\n", "Il serait \u00e9galement souhaitable qu'elles ne soient pas aussi fortement corr\u00e9l\u00e9es les unes avec les autres, afin qu'elles ajoutent des informations ind\u00e9pendantes.\n", "\n", "Utilisez ces informations pour essayer de supprimer des caract\u00e9ristiques. Vous pouvez \u00e9galement essayer de cr\u00e9er d'autres caract\u00e9ristiques synth\u00e9tiques, telles que des ratios de deux caract\u00e9ristiques brutes.\n", "\n", "Pour plus de facilit\u00e9, nous avons inclus le code d'apprentissage de l'exercice pr\u00e9c\u00e9dent." ] }, { "metadata": { "id": "bjR5jWpFr2xs", "colab_type": "code", "colab": { "autoexec": { "startup": false, "wait_interval": 0 } } }, "source": [ "def construct_feature_columns(input_features):\n", " \"\"\"Construct the TensorFlow Feature Columns.\n", "\n", " Args:\n", " input_features: The names of the numerical input features to use.\n", " Returns:\n", " A set of feature columns\n", " \"\"\" \n", " return set([tf.feature_column.numeric_column(my_feature)\n", " for my_feature in input_features])" ], "cell_type": "code", "execution_count": 0, "outputs": [] }, { "metadata": { "id": "jsvKHzRciH9T", "colab_type": "code", "colab": { "autoexec": { "startup": false, "wait_interval": 0 } } }, "source": [ "def my_input_fn(features, targets, batch_size=1, shuffle=True, num_epochs=None):\n", " \"\"\"Trains a linear regression model of one feature.\n", " \n", " Args:\n", " features: pandas DataFrame of features\n", " targets: pandas DataFrame of targets\n", " batch_size: Size of batches to be passed to the model\n", " shuffle: True or False. Whether to shuffle the data.\n", " num_epochs: Number of epochs for which data should be repeated. None = repeat indefinitely\n", " Returns:\n", " Tuple of (features, labels) for next data batch\n", " \"\"\"\n", " \n", " # Convert pandas data into a dict of np arrays\n", " features = {key:np.array(value) for key,value in dict(features).items()} \n", " \n", " # Construct a dataset, and configure batching/repeating\n", " ds = Dataset.from_tensor_slices((features,targets)) # warning: 2GB limit\n", " ds = ds.batch(batch_size).repeat(num_epochs)\n", "\n", " # Shuffle the data, if specified\n", " if shuffle:\n", " ds = ds.shuffle(10000)\n", " \n", " # Return the next batch of data\n", " features, labels = ds.make_one_shot_iterator().get_next()\n", " return features, labels" ], "cell_type": "code", "execution_count": 0, "outputs": [] }, { "metadata": { "id": "g3kjQV9WH3pb", "colab_type": "code", "colab": { "autoexec": { "startup": false, "wait_interval": 0 } } }, "source": [ "def train_model(\n", " learning_rate,\n", " steps,\n", " batch_size,\n", " training_examples,\n", " training_targets,\n", " validation_examples,\n", " validation_targets):\n", " \"\"\"Trains a linear regression model.\n", " \n", " In addition to training, this function also prints training progress information,\n", " as well as a plot of the training and validation loss over time.\n", " \n", " Args:\n", " learning_rate: A `float`, the learning rate.\n", " steps: A non-zero `int`, the total number of training steps. A training step\n", " consists of a forward and backward pass using a single batch.\n", " batch_size: A non-zero `int`, the batch size.\n", " training_examples: A `DataFrame` containing one or more columns from\n", " `california_housing_dataframe` to use as input features for training.\n", " training_targets: A `DataFrame` containing exactly one column from\n", " `california_housing_dataframe` to use as target for training.\n", " validation_examples: A `DataFrame` containing one or more columns from\n", " `california_housing_dataframe` to use as input features for validation.\n", " validation_targets: A `DataFrame` containing exactly one column from\n", " `california_housing_dataframe` to use as target for validation.\n", " \n", " Returns:\n", " A `LinearRegressor` object trained on the training data.\n", " \"\"\"\n", "\n", " periods = 10\n", " steps_per_period = steps / periods\n", "\n", " # Create a linear regressor object.\n", " my_optimizer = tf.train.GradientDescentOptimizer(learning_rate=learning_rate)\n", " my_optimizer = tf.contrib.estimator.clip_gradients_by_norm(my_optimizer, 5.0)\n", " linear_regressor = tf.estimator.LinearRegressor(\n", " feature_columns=construct_feature_columns(training_examples),\n", " optimizer=my_optimizer\n", " )\n", " \n", " # Create input functions\n", " training_input_fn = lambda: my_input_fn(training_examples, \n", " training_targets[\"median_house_value\"], \n", " batch_size=batch_size)\n", " predict_training_input_fn = lambda: my_input_fn(training_examples, \n", " training_targets[\"median_house_value\"], \n", " num_epochs=1, \n", " shuffle=False)\n", " predict_validation_input_fn = lambda: my_input_fn(validation_examples, \n", " validation_targets[\"median_house_value\"], \n", " num_epochs=1, \n", " shuffle=False)\n", "\n", " # Train the model, but do so inside a loop so that we can periodically assess\n", " # loss metrics.\n", " print(\"Training model...\")\n", " print(\"RMSE (on training data):\")\n", " training_rmse = []\n", " validation_rmse = []\n", " for period in range (0, periods):\n", " # Train the model, starting from the prior state.\n", " linear_regressor.train(\n", " input_fn=training_input_fn,\n", " steps=steps_per_period,\n", " )\n", " # Take a break and compute predictions.\n", " training_predictions = linear_regressor.predict(input_fn=predict_training_input_fn)\n", " training_predictions = np.array([item['predictions'][0] for item in training_predictions])\n", " \n", " validation_predictions = linear_regressor.predict(input_fn=predict_validation_input_fn)\n", " validation_predictions = np.array([item['predictions'][0] for item in validation_predictions])\n", " \n", " # Compute training and validation loss.\n", " training_root_mean_squared_error = math.sqrt(\n", " metrics.mean_squared_error(training_predictions, training_targets))\n", " validation_root_mean_squared_error = math.sqrt(\n", " metrics.mean_squared_error(validation_predictions, validation_targets))\n", " # Occasionally print the current loss.\n", " print(\" period %02d : %0.2f\" % (period, training_root_mean_squared_error))\n", " # Add the loss metrics from this period to our list.\n", " training_rmse.append(training_root_mean_squared_error)\n", " validation_rmse.append(validation_root_mean_squared_error)\n", " print(\"Model training finished.\")\n", "\n", " \n", " # Output a graph of loss metrics over periods.\n", " plt.ylabel(\"RMSE\")\n", " plt.xlabel(\"Periods\")\n", " plt.title(\"Root Mean Squared Error vs. Periods\")\n", " plt.tight_layout()\n", " plt.plot(training_rmse, label=\"training\")\n", " plt.plot(validation_rmse, label=\"validation\")\n", " plt.legend()\n", "\n", " return linear_regressor" ], "cell_type": "code", "execution_count": 0, "outputs": [] }, { "cell_type": "markdown", "metadata": { "id": "varLu7RNH3pf", "colab_type": "text" }, "source": [ " Vous allez maintenant consacrer cinq\u00a0minutes \u00e0 la recherche d'un ensemble satisfaisant de caract\u00e9ristiques et de param\u00e8tres d'apprentissage. Pour rappel, les param\u00e8tres d'apprentissage peuvent \u00eatre diff\u00e9rents suivant les caract\u00e9ristiques." ] }, { "metadata": { "id": "DSgUxRIlH3pg", "colab_type": "code", "colab": { "autoexec": { "startup": false, "wait_interval": 0 } } }, "source": [ "#\n", "# Your code here: add your features of choice as a list of quoted strings.\n", "#\n", "minimal_features = [\n", "]\n", "\n", "assert minimal_features, \"You must select at least one feature!\"\n", "\n", "minimal_training_examples = training_examples[minimal_features]\n", "minimal_validation_examples = validation_examples[minimal_features]\n", "\n", "#\n", "# Don't forget to adjust these parameters.\n", "#\n", "train_model(\n", " learning_rate=0.001,\n", " steps=500,\n", " batch_size=5,\n", " training_examples=minimal_training_examples,\n", " training_targets=training_targets,\n", " validation_examples=minimal_validation_examples,\n", " validation_targets=validation_targets)" ], "cell_type": "code", "execution_count": 0, "outputs": [] }, { "cell_type": "markdown", "metadata": { "id": "IGINhMIJ5Wyt", "colab_type": "text" }, "source": [ " ### Solution\n", "\n", "Cliquez ci-dessous pour afficher une solution." ] }, { "metadata": { "id": "BAGoXFPZ5ZE3", "colab_type": "code", "colab": { "autoexec": { "startup": false, "wait_interval": 0 } } }, "source": [ "minimal_features = [\n", " \"median_income\",\n", " \"latitude\",\n", "]\n", "\n", "minimal_training_examples = training_examples[minimal_features]\n", "minimal_validation_examples = validation_examples[minimal_features]\n", "\n", "_ = train_model(\n", " learning_rate=0.01,\n", " steps=500,\n", " batch_size=5,\n", " training_examples=minimal_training_examples,\n", " training_targets=training_targets,\n", " validation_examples=minimal_validation_examples,\n", " validation_targets=validation_targets)" ], "cell_type": "code", "execution_count": 0, "outputs": [] }, { "cell_type": "markdown", "metadata": { "id": "RidI9YhKOiY2", "colab_type": "text" }, "source": [ " ## T\u00e2che\u00a02\u00a0: Mieux exploiter la caract\u00e9ristique Latitude\n", "\n", "La repr\u00e9sentation graphique des valeurs `latitude` par rapport \u00e0 `median_house_value` montre qu'il n'y a pas vraiment de lien entre les deux.\n", "\n", "On constate, en revanche, la pr\u00e9sence de quelques pics qui correspondent approximativement \u00e0 Los\u00a0Angeles et San\u00a0Francisco." ] }, { "metadata": { "id": "hfGUKj2IR_F1", "colab_type": "code", "colab": { "autoexec": { "startup": false, "wait_interval": 0 }, "test": { "output": "ignore", "timeout": 600 } }, "cellView": "both" }, "source": [ "plt.scatter(training_examples[\"latitude\"], training_targets[\"median_house_value\"])" ], "cell_type": "code", "execution_count": 0, "outputs": [] }, { "cell_type": "markdown", "metadata": { "id": "6N0p91k2iFCP", "colab_type": "text" }, "source": [ " **Essayez de cr\u00e9er quelques caract\u00e9ristiques synth\u00e9tiques qui offrent de meilleures performances avec la latitude.**\n", "\n", "Vous pouvez, par exemple, cr\u00e9er une caract\u00e9ristique qui associe `latitude` \u00e0 la valeur `|latitude - 38|` et lui attribuer le nom `distance_from_san_francisco`.\n", "\n", "Vous pouvez \u00e9galement diviser l'espace en 10\u00a0buckets (`latitude_32_to_33`, `latitude_33_to_34`, etc.) affichant chacun une valeur de `1.0` si `latitude` se situe dans cette plage de buckets et une valeur de `0.0` dans le cas contraire.\n", "\n", "Utilisez la matrice de corr\u00e9lation pour faciliter le d\u00e9veloppement et ajoutez ensuite les buckets \u00e0 votre mod\u00e8le si vous trouvez un \u00e9l\u00e9ment satisfaisant.\n", "\n", "Quelles sont les meilleures performances qu'il est possible d'obtenir en termes de validation\u00a0?" ] }, { "metadata": { "id": "wduJ2B28yMFl", "colab_type": "code", "colab": { "autoexec": { "startup": false, "wait_interval": 0 } }, "cellView": "form" }, "source": [ "#\n", "# YOUR CODE HERE: Train on a new data set that includes synthetic features based on latitude.\n", "#" ], "cell_type": "code", "execution_count": 0, "outputs": [] }, { "cell_type": "markdown", "metadata": { "id": "pZa8miwu6_tQ", "colab_type": "text" }, "source": [ " ### Solution\n", "\n", "Cliquez ci-dessous pour afficher une solution." ] }, { "cell_type": "markdown", "metadata": { "id": "PzABdyjq7IZU", "colab_type": "text" }, "source": [ " Outre `latitude`, vous allez conserver la valeur `median_income` pour effectuer des comparaisons avec les r\u00e9sultats pr\u00e9c\u00e9dents.\n", "\n", "Nous avons d\u00e9cid\u00e9 d'effectuer un binning sur la latitude. C'est une op\u00e9ration relativement simple dans Pandas gr\u00e2ce \u00e0 `Series.apply`." ] }, { "metadata": { "id": "xdVF8siZ7Lup", "colab_type": "code", "colab": { "autoexec": { "startup": false, "wait_interval": 0 } } }, "source": [ "LATITUDE_RANGES = zip(range(32, 44), range(33, 45))\n", "\n", "def select_and_transform_features(source_df):\n", " selected_examples = pd.DataFrame()\n", " selected_examples[\"median_income\"] = source_df[\"median_income\"]\n", " for r in LATITUDE_RANGES:\n", " selected_examples[\"latitude_%d_to_%d\" % r] = source_df[\"latitude\"].apply(\n", " lambda l: 1.0 if l >= r[0] and l < r[1] else 0.0)\n", " return selected_examples\n", "\n", "selected_training_examples = select_and_transform_features(training_examples)\n", "selected_validation_examples = select_and_transform_features(validation_examples)" ], "cell_type": "code", "execution_count": 0, "outputs": [] }, { "metadata": { "id": "U4iAdY6t7Pkh", "colab_type": "code", "colab": { "autoexec": { "startup": false, "wait_interval": 0 } } }, "source": [ "_ = train_model(\n", " learning_rate=0.01,\n", " steps=500,\n", " batch_size=5,\n", " training_examples=selected_training_examples,\n", " training_targets=training_targets,\n", " validation_examples=selected_validation_examples,\n", " validation_targets=validation_targets)" ], "cell_type": "code", "execution_count": 0, "outputs": [] } ] }