{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "[![Open in Binder](https://mybinder.org/badge_logo.svg)](https://mybinder.org/v2/gh/justmarkham/scikit-learn-tips/master?filepath=notebooks%2F01_column_transformer.ipynb)\n", "\n", "[![Open in Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/justmarkham/scikit-learn-tips/blob/master/notebooks/01_column_transformer.ipynb)\n", "\n", "# 🤖⚡ scikit-learn tip #1 ([video](https://www.youtube.com/watch?v=NGq8wnH5VSo&list=PL5-da3qGB5ID7YYAqireYEew2mWVvgmj6&index=1))\n", "\n", "Use ColumnTransformer to apply different preprocessing to different columns:\n", "\n", "- select from DataFrame columns by name\n", "- passthrough or drop unspecified columns\n", "\n", "Requires scikit-learn 0.20+\n", "\n", "See example 👇" ] }, { "cell_type": "code", "execution_count": 1, "metadata": {}, "outputs": [], "source": [ "import pandas as pd\n", "df = pd.read_csv('http://bit.ly/kaggletrain', nrows=6)" ] }, { "cell_type": "code", "execution_count": 2, "metadata": {}, "outputs": [], "source": [ "cols = ['Fare', 'Embarked', 'Sex', 'Age']\n", "X = df[cols]" ] }, { "cell_type": "code", "execution_count": 3, "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
FareEmbarkedSexAge
07.2500Smale22.0
171.2833Cfemale38.0
27.9250Sfemale26.0
353.1000Sfemale35.0
48.0500Smale35.0
58.4583QmaleNaN
\n", "
" ], "text/plain": [ " Fare Embarked Sex Age\n", "0 7.2500 S male 22.0\n", "1 71.2833 C female 38.0\n", "2 7.9250 S female 26.0\n", "3 53.1000 S female 35.0\n", "4 8.0500 S male 35.0\n", "5 8.4583 Q male NaN" ] }, "execution_count": 3, "metadata": {}, "output_type": "execute_result" } ], "source": [ "X" ] }, { "cell_type": "code", "execution_count": 4, "metadata": {}, "outputs": [], "source": [ "from sklearn.preprocessing import OneHotEncoder\n", "from sklearn.impute import SimpleImputer\n", "from sklearn.compose import make_column_transformer" ] }, { "cell_type": "code", "execution_count": 5, "metadata": {}, "outputs": [], "source": [ "ohe = OneHotEncoder()\n", "imp = SimpleImputer()" ] }, { "cell_type": "code", "execution_count": 6, "metadata": {}, "outputs": [], "source": [ "ct = make_column_transformer(\n", " (ohe, ['Embarked', 'Sex']), # apply OneHotEncoder to Embarked and Sex\n", " (imp, ['Age']), # apply SimpleImputer to Age\n", " remainder='passthrough') # include remaining column (Fare) in the output" ] }, { "cell_type": "code", "execution_count": 7, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "array([[ 0. , 0. , 1. , 0. , 1. , 22. , 7.25 ],\n", " [ 1. , 0. , 0. , 1. , 0. , 38. , 71.2833],\n", " [ 0. , 0. , 1. , 1. , 0. , 26. , 7.925 ],\n", " [ 0. , 0. , 1. , 1. , 0. , 35. , 53.1 ],\n", " [ 0. , 0. , 1. , 0. , 1. , 35. , 8.05 ],\n", " [ 0. , 1. , 0. , 0. , 1. , 31.2 , 8.4583]])" ] }, "execution_count": 7, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# column order: Embarked (3 columns), Sex (2 columns), Age (1 column), Fare (1 column)\n", "ct.fit_transform(X)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Want more tips? [View all tips on GitHub](https://github.com/justmarkham/scikit-learn-tips) or [Sign up to receive 2 tips by email every week](https://scikit-learn.tips) 💌\n", "\n", "© 2020 [Data School](https://www.dataschool.io). All rights reserved." ] } ], "metadata": { "kernelspec": { "display_name": "Python 3", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.8.2" } }, "nbformat": 4, "nbformat_minor": 4 }