{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "# Tabular training\n", "\n", "> How to use the tabular application in fastai" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "To illustrate the tabular application, we will use the example of the [Adult dataset](https://archive.ics.uci.edu/ml/datasets/Adult) where we have to predict if a person is earning more or less than $50k per year using some general data." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "from fastai2.tabular.all import *" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "We can download a sample of this dataset with the usual command:" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "(#3) [Path('/home/sgugger/.fastai/data/adult_sample/models'),Path('/home/sgugger/.fastai/data/adult_sample/adult.csv'),Path('/home/sgugger/.fastai/data/adult_sample/export.pkl')]" ] }, "execution_count": null, "metadata": {}, "output_type": "execute_result" } ], "source": [ "path = untar_data(URLs.ADULT_SAMPLE)\n", "path.ls()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Then we can have a look at how the data is structured:" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
ageworkclassfnlwgteducationeducation-nummarital-statusoccupationrelationshipracesexcapital-gaincapital-losshours-per-weeknative-countrysalary
049Private101320Assoc-acdm12.0Married-civ-spouseNaNWifeWhiteFemale0190240United-States>=50k
144Private236746Masters14.0DivorcedExec-managerialNot-in-familyWhiteMale10520045United-States>=50k
238Private96185HS-gradNaNDivorcedNaNUnmarriedBlackFemale0032United-States<50k
338Self-emp-inc112847Prof-school15.0Married-civ-spouseProf-specialtyHusbandAsian-Pac-IslanderMale0040United-States>=50k
442Self-emp-not-inc822977th-8thNaNMarried-civ-spouseOther-serviceWifeBlackFemale0050United-States<50k
\n", "
" ], "text/plain": [ " age workclass fnlwgt education education-num \\\n", "0 49 Private 101320 Assoc-acdm 12.0 \n", "1 44 Private 236746 Masters 14.0 \n", "2 38 Private 96185 HS-grad NaN \n", "3 38 Self-emp-inc 112847 Prof-school 15.0 \n", "4 42 Self-emp-not-inc 82297 7th-8th NaN \n", "\n", " marital-status occupation relationship race \\\n", "0 Married-civ-spouse NaN Wife White \n", "1 Divorced Exec-managerial Not-in-family White \n", "2 Divorced NaN Unmarried Black \n", "3 Married-civ-spouse Prof-specialty Husband Asian-Pac-Islander \n", "4 Married-civ-spouse Other-service Wife Black \n", "\n", " sex capital-gain capital-loss hours-per-week native-country salary \n", "0 Female 0 1902 40 United-States >=50k \n", "1 Male 10520 0 45 United-States >=50k \n", "2 Female 0 0 32 United-States <50k \n", "3 Male 0 0 40 United-States >=50k \n", "4 Female 0 0 50 United-States <50k " ] }, "execution_count": null, "metadata": {}, "output_type": "execute_result" } ], "source": [ "df = pd.read_csv(path/'adult.csv')\n", "df.head()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Some of the coumns are continuous (like age) and we will treat them as float numbers we can feed our model directly. Others are categorical (like workclass or education) and we will convert them to a unique index that we will feed to embedding layers. We can specify our categorical and continuous column names, as well as the name of the dependent variable in `TabularDataLoaders` factory methods:" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "dls = TabularDataLoaders.from_csv(path/'adult.csv', path=path, y_names=\"salary\",\n", " cat_names = ['workclass', 'education', 'marital-status', 'occupation', 'relationship', 'race'],\n", " cont_names = ['age', 'fnlwgt', 'education-num'],\n", " procs = [Categorify, FillMissing, Normalize])" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The last part is the list of pre-processors we apply to our data:\n", "\n", "- `Categorify` is going to take every categorical variable and make a map from integer to unique categories, then replace the values by the corresponding index.\n", "- `FillMissing` will fille the missing values in the continuous variables by the median of existing values (you can choose a specific value if you prefer)\n", "- `Normalize` will normalize the continuous variables (substract the mean and divide by the std)\n", "\n", "The `show_batch` method works like for every other application:" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [ { "data": { "text/html": [ "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
workclasseducationmarital-statusoccupationrelationshipraceeducation-num_naagefnlwgteducation-numsalary
0?Some-collegeNever-married?Own-childWhiteFalse22.00000032731.99643610.0<50k
1Private7th-8thMarried-civ-spouseMachine-op-inspctHusbandWhiteFalse44.00000099202.9985784.0<50k
2PrivateHS-gradDivorcedFarming-fishingNot-in-familyWhiteFalse63.000001117680.9969979.0<50k
3PrivateHS-gradMarried-civ-spouseMachine-op-inspctHusbandWhiteFalse33.000000194141.0001709.0<50k
4PrivateAssoc-vocDivorcedTransport-movingNot-in-familyWhiteFalse35.000000172570.99973211.0<50k
5Local-govHS-gradDivorcedExec-managerialUnmarriedAmer-Indian-EskimoFalse43.000000196308.0000369.0<50k
6PrivateHS-gradNever-marriedExec-managerialNot-in-familyWhiteFalse43.000000336642.9962359.0<50k
7PrivateHS-gradNever-marriedOther-serviceNot-in-familyWhiteFalse27.000000158156.0010819.0<50k
8?BachelorsNever-married?UnmarriedWhiteFalse26.000000130832.00175613.0<50k
9PrivateAssoc-vocMarried-civ-spouseTech-supportHusbandWhiteFalse27.00000062737.00346111.0<50k
" ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "dls.show_batch()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "We can define a model using the `tabular_learner` method. When we define our model, `fastai` will try to infer the loss function based on our `y_names` earlier. \n", "\n", "**Note**: Sometimes with tabular data, your `y`'s may be encoded (such as 0 and 1). In such a case you should explicitly pass `y_block = CategoryBlock` in your constructor so `fastai` won't presume you are doing regression." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "learn = tabular_learner(dls, metrics=accuracy)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "And we can train that model with the `fit_one_cycle` method (the `fine_tune` method won't be useful here since we don't have a pretrained model)>" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [ { "data": { "text/html": [ "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
epochtrain_lossvalid_lossaccuracytime
00.3667270.3515240.83584200:05
" ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "learn.fit_one_cycle(1)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "We can then have a look at some predictions:" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [ { "data": { "text/html": [], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "text/html": [ "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
workclasseducationmarital-statusoccupationrelationshipraceeducation-num_naagefnlwgteducation-numsalarysalary_pred
05.012.03.015.01.05.01.0-0.333356-0.900977-0.4199341.00.0
17.012.05.06.05.05.01.00.916167-1.457755-0.4199340.00.0
25.010.03.02.01.05.01.0-0.774364-0.0309441.1507260.00.0
35.013.03.05.01.05.01.0-0.259855-0.6684911.5433900.01.0
45.013.01.013.02.05.01.00.6221610.4090601.5433901.00.0
53.016.03.04.01.05.01.00.254654-0.870132-0.0272691.00.0
65.012.05.013.02.05.01.0-0.259855-0.464552-0.4199340.00.0
75.09.03.04.01.05.01.00.989668-0.4305620.3653961.01.0
86.016.03.04.01.03.01.0-0.627362-0.110140-0.0272690.00.0
" ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "learn.show_results()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Or use the predict method on a row:" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [ { "data": { "text/html": [], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "text/plain": [ "( workclass education marital-status occupation relationship race \\\n", " 0 5.0 8.0 3.0 0.0 6.0 5.0 \n", " \n", " education-num_na age fnlwgt education-num salary \n", " 0 1.0 0.769164 -0.835926 0.758061 0.0 ,\n", " tensor(0),\n", " tensor([0.5200, 0.4800]))" ] }, "execution_count": null, "metadata": {}, "output_type": "execute_result" } ], "source": [ "learn.predict(df.iloc[0])" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "To get prediction on a new dataframe, you can use the `test_dl` method of the `DataLoaders`. That dataframe does not need to have the dependent variable in its column." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "test_df = df.copy()\n", "test_df.drop(['salary'], axis=1, inplace=True)\n", "dl = learn.dls.test_dl(test_df)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Then `Learner.get_preds` will give you the predictions:" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [ { "data": { "text/html": [], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "text/plain": [ "(tensor([[0.5200, 0.4800],\n", " [0.5536, 0.4464],\n", " [0.9767, 0.0233],\n", " ...,\n", " [0.6025, 0.3975],\n", " [0.7228, 0.2772],\n", " [0.5157, 0.4843]]), None)" ] }, "execution_count": null, "metadata": {}, "output_type": "execute_result" } ], "source": [ "learn.get_preds(dl=dl)" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [] } ], "metadata": { "jupytext": { "split_at_heading": true }, "kernelspec": { "display_name": "Python 3", "language": "python", "name": "python3" } }, "nbformat": 4, "nbformat_minor": 2 }