{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# EqualFrequencyDiscretiser + WoEEncoder\n",
"\n",
"This is very useful for linear models, because by using discretisation + a monotonic encoding, we create monotonic variables with the target, from those that before were not originally. And this tends to help improve the performance of the linear model. "
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## EqualFrequencyDiscretiser\n",
"\n",
"The EqualFrequencyDiscretiser() divides continuous numerical variables\n",
"into contiguous equal frequency intervals, that is, intervals that contain\n",
"approximately the same proportion of observations.\n",
"\n",
"The interval limits are determined by the quantiles. The number of intervals,\n",
"i.e., the number of quantiles in which the variable should be divided is\n",
"determined by the user.\n",
"\n",
"Note : Check out the EqualFrequencyDiscretiser notebook to larn more about this transformer."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## WoEEncoder\n",
"\n",
"This encoder replaces the labels by the weight of evidence.\n",
"\n",
"**It only works for binary classification.**\n",
"\n",
"Note: Check out the WoEEncoder notebook to learn more about this transformer."
]
},
{
"cell_type": "code",
"execution_count": 1,
"metadata": {},
"outputs": [],
"source": [
"import pandas as pd\n",
"import numpy as np\n",
"import matplotlib.pyplot as plt\n",
"\n",
"from sklearn.model_selection import train_test_split\n",
"from sklearn.pipeline import Pipeline\n",
"\n",
"from feature_engine.discretisation import EqualFrequencyDiscretiser\n",
"from feature_engine.encoding import WoEEncoder\n",
"\n",
"plt.rcParams[\"figure.figsize\"] = [15,5]"
]
},
{
"cell_type": "code",
"execution_count": 2,
"metadata": {},
"outputs": [],
"source": [
"# Load titanic dataset from OpenML\n",
"\n",
"def load_titanic():\n",
" data = pd.read_csv('https://www.openml.org/data/get_csv/16826755/phpMYEkMl')\n",
" data = data.replace('?', np.nan)\n",
" data['cabin'] = data['cabin'].astype(str).str[0]\n",
" data['pclass'] = data['pclass'].astype('O')\n",
" data['age'] = data['age'].astype('float').fillna(data.age.median())\n",
" data['fare'] = data['fare'].astype('float').fillna(data.fare.median())\n",
" data['embarked'].fillna('C', inplace=True)\n",
" data.drop(labels=['boat', 'body', 'home.dest', 'name', 'ticket'], axis=1, inplace=True)\n",
" return data"
]
},
{
"cell_type": "code",
"execution_count": 3,
"metadata": {},
"outputs": [
{
"data": {
"text/html": [
"\n",
"\n",
"
\n",
" \n",
" \n",
" pclass \n",
" survived \n",
" sex \n",
" age \n",
" sibsp \n",
" parch \n",
" fare \n",
" cabin \n",
" embarked \n",
" \n",
" \n",
" \n",
" \n",
" 0 \n",
" 1 \n",
" 1 \n",
" female \n",
" 29.0000 \n",
" 0 \n",
" 0 \n",
" 211.3375 \n",
" B \n",
" S \n",
" \n",
" \n",
" 1 \n",
" 1 \n",
" 1 \n",
" male \n",
" 0.9167 \n",
" 1 \n",
" 2 \n",
" 151.5500 \n",
" C \n",
" S \n",
" \n",
" \n",
" 2 \n",
" 1 \n",
" 0 \n",
" female \n",
" 2.0000 \n",
" 1 \n",
" 2 \n",
" 151.5500 \n",
" C \n",
" S \n",
" \n",
" \n",
" 3 \n",
" 1 \n",
" 0 \n",
" male \n",
" 30.0000 \n",
" 1 \n",
" 2 \n",
" 151.5500 \n",
" C \n",
" S \n",
" \n",
" \n",
" 4 \n",
" 1 \n",
" 0 \n",
" female \n",
" 25.0000 \n",
" 1 \n",
" 2 \n",
" 151.5500 \n",
" C \n",
" S \n",
" \n",
" \n",
"
\n",
"
"
]
},
"metadata": {
"needs_background": "light"
},
"output_type": "display_data"
}
],
"source": [
"# we will use two continuous variables for the transformations\n",
"X_train[[\"age\", 'fare']].hist(bins=30)\n",
"plt.show()"
]
},
{
"cell_type": "code",
"execution_count": 8,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"Pipeline(memory=None,\n",
" steps=[('EqualFrequencyDiscretiser',\n",
" EqualFrequencyDiscretiser(q=4, return_boundaries=False,\n",
" return_object=True,\n",
" variables=['age', 'fare'])),\n",
" ('WoEEncoder', WoEEncoder(variables=['age', 'fare']))],\n",
" verbose=False)"
]
},
"execution_count": 8,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"# set up the discretiser\n",
"\n",
"efd = EqualFrequencyDiscretiser(\n",
" q=4,\n",
" variables=['age', 'fare'],\n",
" # important: return values as categorical\n",
" return_object=True)\n",
"\n",
"# set up the encoder\n",
"woe = WoEEncoder(variables=['age', 'fare'])\n",
"\n",
"# pipeline\n",
"transformer = Pipeline(steps=[('EqualFrequencyDiscretiser', efd),\n",
" ('WoEEncoder', woe),\n",
" ])\n",
"\n",
"transformer.fit(X_train, y_train)"
]
},
{
"cell_type": "code",
"execution_count": 9,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"{'age': [-inf, 23.0, 28.0, 35.0, inf],\n",
" 'fare': [-inf, 7.8958, 14.4542, 31.275, inf]}"
]
},
"execution_count": 9,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"transformer.named_steps['EqualFrequencyDiscretiser'].binner_dict_"
]
},
{
"cell_type": "code",
"execution_count": 10,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"{'age': {0: 0.07533270507296917,\n",
" 1: -0.260402163917158,\n",
" 2: 0.3237107275657203,\n",
" 3: 0.05769015189511875},\n",
" 'fare': {0: -0.5990108946387251,\n",
" 1: -0.41504696424627724,\n",
" 2: 0.142571903020815,\n",
" 3: 0.7852653023249282}}"
]
},
"execution_count": 10,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"transformer.named_steps['WoEEncoder'].encoder_dict_"
]
},
{
"cell_type": "code",
"execution_count": 11,
"metadata": {},
"outputs": [
{
"data": {
"text/html": [
"\n",
"\n",
"
\n",
" \n",
" \n",
" pclass \n",
" sex \n",
" age \n",
" sibsp \n",
" parch \n",
" fare \n",
" cabin \n",
" embarked \n",
" \n",
" \n",
" \n",
" \n",
" 1139 \n",
" 3 \n",
" male \n",
" 0.057690 \n",
" 0 \n",
" 0 \n",
" -0.599011 \n",
" n \n",
" S \n",
" \n",
" \n",
" 533 \n",
" 2 \n",
" female \n",
" 0.075333 \n",
" 0 \n",
" 1 \n",
" 0.142572 \n",
" n \n",
" S \n",
" \n",
" \n",
" 459 \n",
" 2 \n",
" male \n",
" 0.057690 \n",
" 1 \n",
" 0 \n",
" 0.142572 \n",
" n \n",
" S \n",
" \n",
" \n",
" 1150 \n",
" 3 \n",
" male \n",
" -0.260402 \n",
" 0 \n",
" 0 \n",
" 0.142572 \n",
" n \n",
" S \n",
" \n",
" \n",
" 393 \n",
" 2 \n",
" male \n",
" -0.260402 \n",
" 0 \n",
" 0 \n",
" 0.785265 \n",
" n \n",
" S \n",
" \n",
" \n",
"
\n",
"
"
]
},
"metadata": {
"needs_background": "light"
},
"output_type": "display_data"
}
],
"source": [
"# let's explore the monotonic relationship\n",
"plt.figure(figsize=(7,5))\n",
"pd.concat([test_t,y_test], axis=1).groupby(\"fare\")[\"survived\"].mean().plot()\n",
"plt.title(\"Relationship between fare and target\")\n",
"plt.xlabel(\"fare\")\n",
"plt.ylabel(\"Mean of target\")\n",
"plt.show()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Note how now the intervals are monotonically sorted respect to the target."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": []
}
],
"metadata": {
"kernelspec": {
"display_name": "fengine",
"language": "python",
"name": "fengine"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.8.2"
},
"toc": {
"base_numbering": 1,
"nav_menu": {},
"number_sections": true,
"sideBar": true,
"skip_h1_title": false,
"title_cell": "Table of Contents",
"title_sidebar": "Contents",
"toc_cell": false,
"toc_position": {},
"toc_section_display": true,
"toc_window_display": false
}
},
"nbformat": 4,
"nbformat_minor": 4
}