{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "[![Open in Binder](https://mybinder.org/badge_logo.svg)](https://mybinder.org/v2/gh/justmarkham/scikit-learn-tips/master?filepath=notebooks%2F33_function_transformer.ipynb)\n", "\n", "[![Open in Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/justmarkham/scikit-learn-tips/blob/master/notebooks/33_function_transformer.ipynb)\n", "\n", "# 🤖⚡ scikit-learn tip #33 ([video](https://www.youtube.com/watch?v=s1gL82BxKos&list=PL5-da3qGB5ID7YYAqireYEew2mWVvgmj6&index=33))\n", "\n", "Want to do feature engineering within a ColumnTransformer or Pipeline?\n", "\n", "1. Select an existing function (or write your own)\n", "2. Convert it into a transformer using FunctionTransformer\n", "3. 🥳\n", "\n", "See example 👇" ] }, { "cell_type": "code", "execution_count": 1, "metadata": {}, "outputs": [], "source": [ "import pandas as pd\n", "import numpy as np\n", "from sklearn.compose import make_column_transformer" ] }, { "cell_type": "code", "execution_count": 2, "metadata": {}, "outputs": [], "source": [ "X = pd.DataFrame({'Fare':[200, 300, 50, 900],\n", " 'Code':['X12', 'Y20', 'Z7', np.nan],\n", " 'Deck':['A101', 'C102', 'A200', 'C300']})" ] }, { "cell_type": "code", "execution_count": 3, "metadata": {}, "outputs": [], "source": [ "from sklearn.preprocessing import FunctionTransformer" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Convert existing function into a transformer:" ] }, { "cell_type": "code", "execution_count": 4, "metadata": {}, "outputs": [], "source": [ "clip_values = FunctionTransformer(np.clip, kw_args={'a_min':100, 'a_max':600})" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Convert custom function into a transformer:" ] }, { "cell_type": "code", "execution_count": 5, "metadata": {}, "outputs": [], "source": [ "# extract the first letter from each string\n", "def first_letter(df):\n", " return df.apply(lambda x: x.str.slice(0, 1))" ] }, { "cell_type": "code", "execution_count": 6, "metadata": {}, "outputs": [], "source": [ "get_first_letter = FunctionTransformer(first_letter)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Include them in a ColumnTransformer:" ] }, { "cell_type": "code", "execution_count": 7, "metadata": {}, "outputs": [], "source": [ "ct = make_column_transformer(\n", " (clip_values, ['Fare']),\n", " (get_first_letter, ['Code', 'Deck']))" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Apply the transformations:" ] }, { "cell_type": "code", "execution_count": 8, "metadata": { "scrolled": true }, "outputs": [ { "data": { "text/html": [ "
\n", " | Fare | \n", "Code | \n", "Deck | \n", "
---|---|---|---|
0 | \n", "200 | \n", "X12 | \n", "A101 | \n", "
1 | \n", "300 | \n", "Y20 | \n", "C102 | \n", "
2 | \n", "50 | \n", "Z7 | \n", "A200 | \n", "
3 | \n", "900 | \n", "NaN | \n", "C300 | \n", "