{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "# Exercise 06\n", "\n", "# Fraud Detection" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Introduction\n", "\n", "- Fraud Detection Dataset from Microsoft Azure: [data](http://gallery.cortanaintelligence.com/Experiment/8e9fe4e03b8b4c65b9ca947c72b8e463)\n", "\n", "Fraud detection is one of the earliest industrial applications of data mining and machine learning. Fraud detection is typically handled as a binary classification problem, but the class population is unbalanced because instances of fraud are usually very rare compared to the overall volume of transactions. Moreover, when fraudulent transactions are discovered, the business typically takes measures to block the accounts from transacting to prevent further losses. " ] }, { "cell_type": "code", "execution_count": 2, "metadata": {}, "outputs": [], "source": [ "%matplotlib inline\n", "import pandas as pd\n", "import numpy as np\n", "from sklearn.model_selection import cross_val_score\n", "from sklearn.linear_model import LogisticRegression\n", "from sklearn.tree import DecisionTreeClassifier\n", "from sklearn import metrics" ] }, { "cell_type": "code", "execution_count": 3, "metadata": {}, "outputs": [], "source": [ "import zipfile\n", "with zipfile.ZipFile('../datasets/fraud_detection.csv.zip', 'r') as z:\n", " f = z.open('15_fraud_detection.csv')\n", " data = pd.io.parsers.read_table(f, index_col=0, sep=',')" ] }, { "cell_type": "code", "execution_count": 4, "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
accountAgedigitalItemCountsumPurchaseCount1DaysumPurchaseAmount1DaysumPurchaseAmount30DaypaymentBillingPostalCode - LogOddsForClass_0accountPostalCode - LogOddsForClass_0paymentBillingState - LogOddsForClass_0accountState - LogOddsForClass_0paymentInstrumentAgeInAccountipState - LogOddsForClass_0transactionAmounttransactionAmountUSDipPostalCode - LogOddsForClass_0localHour - LogOddsForClass_0Label
02000000.00720.255.0645330.4212141.3121860.5663953279.5743061.218157599.00626.1646501.2595434.7454020
162111185.442530.370.5389960.4818384.4013704.50015761.9701394.0356011185.441185.4400003.9811184.9213490
22000000.000.005.0645335.0963963.0563573.1552260.0000003.31418632.0932.0900005.0084904.7423030
31100.000.005.0645335.0963963.3311543.3312390.0000003.529398133.28132.7295541.3249254.7454020
41100.00132.735.4128850.3429455.5636774.0869650.0013893.529398543.66543.6600002.6934514.8767710
\n", "
" ], "text/plain": [ " accountAge digitalItemCount sumPurchaseCount1Day sumPurchaseAmount1Day \\\n", "0 2000 0 0 0.00 \n", "1 62 1 1 1185.44 \n", "2 2000 0 0 0.00 \n", "3 1 1 0 0.00 \n", "4 1 1 0 0.00 \n", "\n", " sumPurchaseAmount30Day paymentBillingPostalCode - LogOddsForClass_0 \\\n", "0 720.25 5.064533 \n", "1 2530.37 0.538996 \n", "2 0.00 5.064533 \n", "3 0.00 5.064533 \n", "4 132.73 5.412885 \n", "\n", " accountPostalCode - LogOddsForClass_0 \\\n", "0 0.421214 \n", "1 0.481838 \n", "2 5.096396 \n", "3 5.096396 \n", "4 0.342945 \n", "\n", " paymentBillingState - LogOddsForClass_0 accountState - LogOddsForClass_0 \\\n", "0 1.312186 0.566395 \n", "1 4.401370 4.500157 \n", "2 3.056357 3.155226 \n", "3 3.331154 3.331239 \n", "4 5.563677 4.086965 \n", "\n", " paymentInstrumentAgeInAccount ipState - LogOddsForClass_0 \\\n", "0 3279.574306 1.218157 \n", "1 61.970139 4.035601 \n", "2 0.000000 3.314186 \n", "3 0.000000 3.529398 \n", "4 0.001389 3.529398 \n", "\n", " transactionAmount transactionAmountUSD ipPostalCode - LogOddsForClass_0 \\\n", "0 599.00 626.164650 1.259543 \n", "1 1185.44 1185.440000 3.981118 \n", "2 32.09 32.090000 5.008490 \n", "3 133.28 132.729554 1.324925 \n", "4 543.66 543.660000 2.693451 \n", "\n", " localHour - LogOddsForClass_0 Label \n", "0 4.745402 0 \n", "1 4.921349 0 \n", "2 4.742303 0 \n", "3 4.745402 0 \n", "4 4.876771 0 " ] }, "execution_count": 4, "metadata": {}, "output_type": "execute_result" } ], "source": [ "data.head()" ] }, { "cell_type": "code", "execution_count": 5, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "((138721, 16), 797, 0.0057453449730033666)" ] }, "execution_count": 5, "metadata": {}, "output_type": "execute_result" } ], "source": [ "data.shape, data.Label.sum(), data.Label.mean()" ] }, { "cell_type": "code", "execution_count": 6, "metadata": { "collapsed": true }, "outputs": [], "source": [ "X = data.drop(['Label'], axis=1)\n", "y = data['Label']" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "# Exercice 06.1\n", "\n", "Estimate a Logistic Regression\n", "\n", "Evaluate using the following metrics:\n", "* Accuracy\n", "* F1-Score\n", "* F_Beta-Score (Beta=10)\n", "\n", "Comment about the results" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": true }, "outputs": [], "source": [] }, { "cell_type": "markdown", "metadata": {}, "source": [ "# Exercice 06.2\n", "\n", "Under-sample the negative class using random-under-sampling\n", "\n", "Which is parameter for target_percentage did you choose?\n", "How the results change?\n", "\n", "**Only apply under-sampling to the training set, evaluate using the whole test set**" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": true }, "outputs": [], "source": [] }, { "cell_type": "markdown", "metadata": {}, "source": [ "# Exercice 06.3\n", "\n", "Now using random-over-sampling" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": true }, "outputs": [], "source": [] }, { "cell_type": "markdown", "metadata": {}, "source": [ "# Exercice 06.4\n", "Evaluate the results using SMOTE\n", "\n", "Which parameters did you choose?" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": true }, "outputs": [], "source": [] } ], "metadata": { "kernelspec": { "display_name": "Python 3", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.6.3" } }, "nbformat": 4, "nbformat_minor": 1 }