{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "# Baseline submission for the challenge DBSRA\n", "### Author - Pulkit Gera" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/ayushshivani/aicrowd_educational_baselines/blob/master/DBSRA_baseline.ipynb)\n" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": true }, "outputs": [], "source": [ "!pip install numpy\n", "!pip install pandas\n", "!pip install sklearn" ] }, { "cell_type": "code", "execution_count": 1, "metadata": { "collapsed": true }, "outputs": [], "source": [ "import numpy as np\n", "import pandas as pd\n", "from sklearn.model_selection import train_test_split \n", "from sklearn.linear_model import LogisticRegression\n", "from sklearn.preprocessing import LabelEncoder\n", "from sklearn import metrics" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Download data\n", "The first step is to download out train test data. We will be training a classifier on the train data and make predictions on test data. We submit our predictions" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": true }, "outputs": [], "source": [ "!wget https://s3.eu-central-1.wasabisys.com/aicrowd-public-datasets/aicrowd_educational_dbsra/data/public/test.csv\n", "!wget https://s3.eu-central-1.wasabisys.com/aicrowd-public-datasets/aicrowd_educational_dbsra/data/public/train.csv" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Load Data\n", "We use pandas library to load our data. Pandas loads them into dataframes which helps us analyze our data easily. Learn more about it [here](https://www.tutorialspoint.com/python_pandas/index.htm)" ] }, { "cell_type": "code", "execution_count": 2, "metadata": { "collapsed": true }, "outputs": [], "source": [ "train_data = pd.read_csv('train.csv')" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Clean and Analyse Data" ] }, { "cell_type": "code", "execution_count": 3, "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
racegenderageweightadmission_type_iddischarge_disposition_idadmission_source_idtime_in_hospitalpayer_codemedical_specialty...citogliptoninsulinglyburide-metforminglipizide-metforminglimepiride-pioglitazonemetformin-rosiglitazonemetformin-pioglitazonechangediabetesMedreadmitted
0AfricanAmericanFemale[70-80)?1172?InternalMedicine...NoSteadyNoNoNoNoNoNoYes1
1CaucasianFemale[90-100)?3118SPPulmonology...NoDownNoNoNoNoNoChYes1
2CaucasianFemale[80-90)?1271MCOsteopath...NoSteadyNoNoNoNoNoNoYes0
3CaucasianMale[60-70)?3166MCRadiologist...NoSteadyNoNoNoNoNoChYes0
4?Female[70-80)?1363UNInternalMedicine...NoNoNoNoNoNoNoNoNo0
\n", "

5 rows × 48 columns

\n", "
" ], "text/plain": [ " race gender age weight admission_type_id \\\n", "0 AfricanAmerican Female [70-80) ? 1 \n", "1 Caucasian Female [90-100) ? 3 \n", "2 Caucasian Female [80-90) ? 1 \n", "3 Caucasian Male [60-70) ? 3 \n", "4 ? Female [70-80) ? 1 \n", "\n", " discharge_disposition_id admission_source_id time_in_hospital payer_code \\\n", "0 1 7 2 ? \n", "1 1 1 8 SP \n", "2 2 7 1 MC \n", "3 1 6 6 MC \n", "4 3 6 3 UN \n", "\n", " medical_specialty ... citoglipton insulin glyburide-metformin \\\n", "0 InternalMedicine ... No Steady No \n", "1 Pulmonology ... No Down No \n", "2 Osteopath ... No Steady No \n", "3 Radiologist ... No Steady No \n", "4 InternalMedicine ... No No No \n", "\n", " glipizide-metformin glimepiride-pioglitazone metformin-rosiglitazone \\\n", "0 No No No \n", "1 No No No \n", "2 No No No \n", "3 No No No \n", "4 No No No \n", "\n", " metformin-pioglitazone change diabetesMed readmitted \n", "0 No No Yes 1 \n", "1 No Ch Yes 1 \n", "2 No No Yes 0 \n", "3 No Ch Yes 0 \n", "4 No No No 0 \n", "\n", "[5 rows x 48 columns]" ] }, "execution_count": 3, "metadata": {}, "output_type": "execute_result" } ], "source": [ "train_data = train_data.drop('encounter_id',1)\n", "train_data = train_data.drop('patient_nbr',1)\n", "train_data.head()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Since most of the columns have categorical columns we have to convert it into integers. The most basic way is to do an Ordinal Mapping. Note: Here we have not replaced question marks with some other data and they are also accounted into ordinal mapping." ] }, { "cell_type": "code", "execution_count": 5, "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
racegenderageweightadmission_type_iddischarge_disposition_idadmission_source_idtime_in_hospitalpayer_codemedical_specialty...citogliptoninsulinglyburide-metforminglipizide-metforminglimepiride-pioglitazonemetformin-rosiglitazonemetformin-pioglitazonechangediabetesMedreadmitted
010711172019...0210000111
1309131181551...0010000011
230811271830...0210000110
331613166852...0210000010
4007113631619...0110000100
\n", "

5 rows × 48 columns

\n", "
" ], "text/plain": [ " race gender age weight admission_type_id discharge_disposition_id \\\n", "0 1 0 7 1 1 1 \n", "1 3 0 9 1 3 1 \n", "2 3 0 8 1 1 2 \n", "3 3 1 6 1 3 1 \n", "4 0 0 7 1 1 3 \n", "\n", " admission_source_id time_in_hospital payer_code medical_specialty ... \\\n", "0 7 2 0 19 ... \n", "1 1 8 15 51 ... \n", "2 7 1 8 30 ... \n", "3 6 6 8 52 ... \n", "4 6 3 16 19 ... \n", "\n", " citoglipton insulin glyburide-metformin glipizide-metformin \\\n", "0 0 2 1 0 \n", "1 0 0 1 0 \n", "2 0 2 1 0 \n", "3 0 2 1 0 \n", "4 0 1 1 0 \n", "\n", " glimepiride-pioglitazone metformin-rosiglitazone metformin-pioglitazone \\\n", "0 0 0 0 \n", "1 0 0 0 \n", "2 0 0 0 \n", "3 0 0 0 \n", "4 0 0 0 \n", "\n", " change diabetesMed readmitted \n", "0 1 1 1 \n", "1 0 1 1 \n", "2 1 1 0 \n", "3 0 1 0 \n", "4 1 0 0 \n", "\n", "[5 rows x 48 columns]" ] }, "execution_count": 5, "metadata": {}, "output_type": "execute_result" } ], "source": [ "labelencoder = LabelEncoder()\n", "n_train_data = train_data\n", "for col in train_data.columns:\n", " s = train_data[col]\n", " if s.dtype == 'O':\n", " s = labelencoder.fit_transform(s)\n", " n_train_data[col] = s\n", "n_train_data.head()\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Split Data into Train and Validation\n", "Now we want to see how well our classifier is performing, but we dont have the test data labels with us to check. What do we do ? So we split our dataset into train and validation. The idea is that we test our classifier on validation set in order to get an idea of how well our classifier works. This way we can also ensure that we dont [overfit](https://machinelearningmastery.com/overfitting-and-underfitting-with-machine-learning-algorithms/) on the train dataset. There are many ways to do validation like [k-fold](https://machinelearningmastery.com/k-fold-cross-validation/),[leave one out](https://en.wikipedia.org/wiki/Cross-validation_(statistics), etc" ] }, { "cell_type": "code", "execution_count": 6, "metadata": { "collapsed": true }, "outputs": [], "source": [ "X = n_train_data.drop('readmitted',1)\n", "y = n_train_data['readmitted']\n", "# Validation testing\n", "X_train, X_val, y_train, y_val = train_test_split(X, y, test_size=0.2, random_state=42)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Here we have selected the size of the validation data to be 20% of the total data. You can change it and see what effect it has on the accuracies. To learn more about the train_test_split function [click here](https://scikit-learn.org/stable/modules/generated/sklearn.model_selection.train_test_split.html)." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Define the Classifier and Train\n", "Now we come to the juicy part. We have fixed our data and now we train a classifier. The classifier will learn the function by looking at the inputs and corresponding outputs. There are a ton of classifiers to choose from some being [Logistic Regression](https://towardsdatascience.com/logistic-regression-detailed-overview-46c4da4303bc), [SVM](https://towardsdatascience.com/support-vector-machine-introduction-to-machine-learning-algorithms-934a444fca47), [Random Forests](https://towardsdatascience.com/support-vector-machine-introduction-to-machine-learning-algorithms-934a444fca47), [Decision Trees](https://towardsdatascience.com/decision-trees-in-machine-learning-641b9c4e8052), etc. \n", "Tip: A good model doesnt depend solely on the classifier but on the features(columns) you choose. So make sure to play with your data and keep only whats important. " ] }, { "cell_type": "code", "execution_count": 7, "metadata": {}, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "/home/gera/anaconda3/lib/python3.6/site-packages/sklearn/linear_model/logistic.py:432: FutureWarning: Default solver will be changed to 'lbfgs' in 0.22. Specify a solver to silence this warning.\n", " FutureWarning)\n", "/home/gera/anaconda3/lib/python3.6/site-packages/sklearn/linear_model/logistic.py:469: FutureWarning: Default multi_class will be changed to 'auto' in 0.22. Specify the multi_class option to silence this warning.\n", " \"this warning.\", FutureWarning)\n" ] }, { "data": { "text/plain": [ "LogisticRegression(C=1.0, class_weight=None, dual=False, fit_intercept=True,\n", " intercept_scaling=1, l1_ratio=None, max_iter=100,\n", " multi_class='warn', n_jobs=None, penalty='l2',\n", " random_state=None, solver='warn', tol=0.0001, verbose=0,\n", " warm_start=False)" ] }, "execution_count": 7, "metadata": {}, "output_type": "execute_result" } ], "source": [ "classifier = LogisticRegression()\n", "classifier.fit(X_train,y_train)\n", "\n", "# from sklearn.svm import SVC\n", "# clf = SVC(gamma='auto')\n", "# clf.fit(X_train, y_train)\n", "\n", "# from sklearn import tree\n", "# clf = tree.DecisionTreeClassifier()\n", "# clf = clf.fit(X_train, y_train)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "We have used [Logistic Regression](https://en.wikipedia.org/wiki/Logistic_regression) as a classifier here and set few of the parameteres. But one can set more parameters and increase the performance. To see the list of parameters visit [here](https://scikit-learn.org/stable/modules/generated/sklearn.linear_model.LogisticRegression.html). \n", "Also given are SVM and Decision Tree examples. Check out SVM's parameters [here](https://scikit-learn.org/stable/modules/generated/sklearn.svm.SVC.html) and Decision Tree's [here](https://scikit-learn.org/stable/modules/tree.html)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "got a warning! Dont worry, its just beacuse the number of iteration is very less(defined in the classifier in the above cell).Increase the number of iterations and see if the warning vanishes.Do remember increasing iterations also increases the running time.( Hint: max_iter=500)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Predict on Validation\n", "Now we predict our trained classifier on the validation set and evaluate our model" ] }, { "cell_type": "code", "execution_count": 8, "metadata": { "collapsed": true }, "outputs": [], "source": [ "y_pred = classifier.predict(X_val)" ] }, { "cell_type": "code", "execution_count": 9, "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
ActualPredicted
2634210
5914210
5753710
5812800
2982110
6289700
4357200
6232920
4430900
2088200
4907500
2066800
7685610
3285811
7429210
8054910
858810
5776810
1065810
5156900
5991410
3287400
5465610
7745600
3530000
\n", "
" ], "text/plain": [ " Actual Predicted\n", "26342 1 0\n", "59142 1 0\n", "57537 1 0\n", "58128 0 0\n", "29821 1 0\n", "62897 0 0\n", "43572 0 0\n", "62329 2 0\n", "44309 0 0\n", "20882 0 0\n", "49075 0 0\n", "20668 0 0\n", "76856 1 0\n", "32858 1 1\n", "74292 1 0\n", "80549 1 0\n", "8588 1 0\n", "57768 1 0\n", "10658 1 0\n", "51569 0 0\n", "59914 1 0\n", "32874 0 0\n", "54656 1 0\n", "77456 0 0\n", "35300 0 0" ] }, "execution_count": 9, "metadata": {}, "output_type": "execute_result" } ], "source": [ "df = pd.DataFrame({'Actual': y_val, 'Predicted': y_pred})\n", "df1 = df.head(25)\n", "df1" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Evaluate the Performance\n", "We use the same metrics as that will be used for the test set. \n", "[F1 score](https://en.wikipedia.org/wiki/F1_score) and [Log Loss](https://scikit-learn.org/stable/modules/generated/sklearn.metrics.log_loss.html) are the metrics for this challenge" ] }, { "cell_type": "code", "execution_count": 10, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "F1 score Score: 0.5688110513843131\n" ] } ], "source": [ "print('F1 score Score:', metrics.f1_score(y_val, y_pred,average='micro')) " ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Load Test Set\n", "Load the test data now" ] }, { "cell_type": "code", "execution_count": 11, "metadata": { "collapsed": true }, "outputs": [], "source": [ "test_data = pd.read_csv('test.csv')" ] }, { "cell_type": "code", "execution_count": 12, "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
racegenderageweightadmission_type_iddischarge_disposition_idadmission_source_idtime_in_hospitalpayer_codemedical_specialty...examidecitogliptoninsulinglyburide-metforminglipizide-metforminglimepiride-pioglitazonemetformin-rosiglitazonemetformin-pioglitazonechangediabetesMed
03071116111516...0021000011
13151111160...0011000011
23061361460...0011000011
3313121112410...0011000011
41061127100...0011000011
\n", "

5 rows × 47 columns

\n", "
" ], "text/plain": [ " race gender age weight admission_type_id discharge_disposition_id \\\n", "0 3 0 7 1 1 1 \n", "1 3 1 5 1 1 1 \n", "2 3 0 6 1 3 6 \n", "3 3 1 3 1 2 1 \n", "4 1 0 6 1 1 2 \n", "\n", " admission_source_id time_in_hospital payer_code medical_specialty ... \\\n", "0 6 11 15 16 ... \n", "1 1 1 6 0 ... \n", "2 1 4 6 0 ... \n", "3 1 12 4 10 ... \n", "4 7 1 0 0 ... \n", "\n", " examide citoglipton insulin glyburide-metformin glipizide-metformin \\\n", "0 0 0 2 1 0 \n", "1 0 0 1 1 0 \n", "2 0 0 1 1 0 \n", "3 0 0 1 1 0 \n", "4 0 0 1 1 0 \n", "\n", " glimepiride-pioglitazone metformin-rosiglitazone metformin-pioglitazone \\\n", "0 0 0 0 \n", "1 0 0 0 \n", "2 0 0 0 \n", "3 0 0 0 \n", "4 0 0 0 \n", "\n", " change diabetesMed \n", "0 1 1 \n", "1 1 1 \n", "2 1 1 \n", "3 1 1 \n", "4 1 1 \n", "\n", "[5 rows x 47 columns]" ] }, "execution_count": 12, "metadata": {}, "output_type": "execute_result" } ], "source": [ "test_data = test_data.drop('encounter_id',1)\n", "test_data = test_data.drop('patient_nbr',1)\n", "n_test_data = test_data\n", "for col in test_data.columns:\n", " s = test_data[col]\n", " if s.dtype == 'O':\n", " s = labelencoder.fit_transform(s)\n", " n_test_data[col] = s\n", "n_test_data.head()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Predict Test Set\n", "Time for the moment of truth! Predict on test set and time to make the submission." ] }, { "cell_type": "code", "execution_count": 13, "metadata": { "collapsed": true }, "outputs": [], "source": [ "y_test = classifier.predict(test_data)" ] }, { "cell_type": "code", "execution_count": 14, "metadata": { "collapsed": true }, "outputs": [], "source": [ "df = pd.DataFrame(y_test,columns=['readmitted'])\n", "df.to_csv('submission.csv',index=False)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## To download the generated csv in collab run the below command" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": true }, "outputs": [], "source": [ "from google.colab import files\n", "files.download('submission.csv') " ] }, { "cell_type": "markdown", "metadata": { "collapsed": true }, "source": [ "To participate in the challenge click [here](https://www.aicrowd.com/challenges/dbsra-diabettes-readmission-prediction/)" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": true }, "outputs": [], "source": [] } ], "metadata": { "kernelspec": { "display_name": "Python 3", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.6.10" } }, "nbformat": 4, "nbformat_minor": 2 }