{ "cells": [ { "cell_type": "code", "execution_count": 1, "metadata": {}, "outputs": [], "source": [ "import seaborn as sns\n", "sns.set_context('talk')" ] }, { "cell_type": "code", "execution_count": 2, "metadata": {}, "outputs": [], "source": [ "from matplotlib.axes._axes import _log as matplotlib_axes_logger\n", "matplotlib_axes_logger.setLevel('ERROR')" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "# Interpret scikit-learn machine learning models" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "This notebook uses some features from scikit-learn which are under-development:\n", "\n", "* DataFrame handling with OpenML datasets: https://github.com/scikit-learn/scikit-learn/pull/13902\n", "* Fast partial dependence plot for Gradient Boosting Decsision Trees: https://github.com/scikit-learn/scikit-learn/pull/13769\n", "* Permutation feature importance: https://github.com/scikit-learn/scikit-learn/pull/13146\n", "\n", "These features have been combined into a scikit-learn branch in the following repository: https://github.com/glemaitre/scikit-learn/tree/workshop\n", "\n", "You can refer to the following documentation to install scikit-learn from such source: https://scikit-learn.org/stable/developers/advanced_installation.html#install-bleeding-edge" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Dataset: Current Population Survey (1985)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "We will use data from the \"Current Population Survey\" from 1985 and fetch it from [OpenML](http://openml.org/)." ] }, { "cell_type": "code", "execution_count": 3, "metadata": {}, "outputs": [], "source": [ "from sklearn.datasets import fetch_openml\n", "\n", "survey = fetch_openml(data_id=534, as_frame=True)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "We can get more information regarding by looking at the description of the dataset." ] }, { "cell_type": "code", "execution_count": 4, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "**Author**: \n", "**Source**: Unknown - Date unknown \n", "**Please cite**: \n", "\n", "Determinants of Wages from the 1985 Current Population Survey\n", "\n", "Summary:\n", "The Current Population Survey (CPS) is used to supplement census information between census years. These data consist of a random sample of 534 persons from the CPS, with information on wages and other characteristics of the workers, including sex, number of years of education, years of work experience, occupational status, region of residence and union membership. We wish to determine (i) whether wages are related to these characteristics and (ii) whether there is a gender gap in wages.\n", "Based on residual plots, wages were log-transformed to stabilize the variance. Age and work experience were almost perfectly correlated (r=.98). Multiple regression of log wages against sex, age, years of education, work experience, union membership, southern residence, and occupational status showed that these covariates were related to wages (pooled F test, p < .0001). The effect of age was not significant after controlling for experience. Standardized residual plots showed no patterns, except for one large outlier with lower wages than expected. This was a male, with 22 years of experience and 12 years of education, in a management position, who lived in the north and was not a union member. Removing this person from the analysis did not substantially change the results, so that the final model included the entire sample.\n", "Adjusting for all other variables in the model, females earned 81% (75%, 88%) the wages of males (p < .0001). Wages increased 41% (28%, 56%) for every 5 additional years of education (p < .0001). They increased by 11% (7%, 14%) for every additional 10 years of experience (p < .0001). Union members were paid 23% (12%, 36%) more than non-union members (p < .0001). Northerns were paid 11% (2%, 20%) more than southerns (p =.016). Management and professional positions were paid most, and service and clerical positions were paid least (pooled F-test, p < .0001). Overall variance explained was R2 = .35.\n", "In summary, many factors describe the variations in wages: occupational status, years of experience, years of education, sex, union membership and region of residence. However, despite adjustment for all factors that were available, there still appeared to be a gender gap in wages. There is no readily available explanation for this gender gap.\n", "\n", "Authorization: Public Domain\n", "\n", "Reference: Berndt, ER. The Practice of Econometrics. 1991. NY: Addison-Wesley.\n", "\n", "Description: The datafile contains 534 observations on 11 variables sampled from the Current Population Survey of 1985. This data set demonstrates multiple regression, confounding, transformations, multicollinearity, categorical variables, ANOVA, pooled tests of significance, interactions and model building strategies.\n", "\n", "Variable names in order from left to right:\n", "EDUCATION: Number of years of education.\n", "SOUTH: Indicator variable for Southern Region (1=Person lives in \t\tSouth, 0=Person lives elsewhere).\n", "SEX: Indicator variable for sex (1=Female, 0=Male).\n", "EXPERIENCE: Number of years of work experience.\n", "UNION: Indicator variable for union membership (1=Union member, \t\t0=Not union member).\n", "WAGE: Wage (dollars per hour).\n", "AGE: Age (years).\n", "RACE: Race (1=Other, 2=Hispanic, 3=White).\n", "OCCUPATION: Occupational category (1=Management, \t\t2=Sales, 3=Clerical, 4=Service, 5=Professional, 6=Other).\n", "SECTOR: Sector (0=Other, 1=Manufacturing, 2=Construction).\n", "MARR: Marital Status (0=Unmarried, 1=Married)\n", "\n", "\n", "Therese Stukel\n", "Dartmouth Hitchcock Medical Center\n", "One Medical Center Dr.\n", "Lebanon, NH 03756\n", "e-mail: stukel@dartmouth.edu\n", "\n", "\n", "Information about the dataset\n", "CLASSTYPE: numeric\n", "CLASSINDEX: none specific\n", "\n", "Downloaded from openml.org.\n" ] } ], "source": [ "print(survey.DESCR)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The data are stored in a pandas dataframe." ] }, { "cell_type": "code", "execution_count": 5, "metadata": {}, "outputs": [ { "data": { "text/html": "
\n | EDUCATION | \nSOUTH | \nSEX | \nEXPERIENCE | \nUNION | \nAGE | \nRACE | \nOCCUPATION | \nSECTOR | \nMARR | \n
---|---|---|---|---|---|---|---|---|---|---|
0 | \n8.0 | \nno | \nfemale | \n21.0 | \nnot_member | \n35.0 | \nHispanic | \nOther | \nManufacturing | \nMarried | \n
1 | \n9.0 | \nno | \nfemale | \n42.0 | \nnot_member | \n57.0 | \nWhite | \nOther | \nManufacturing | \nMarried | \n
2 | \n12.0 | \nno | \nmale | \n1.0 | \nnot_member | \n19.0 | \nWhite | \nOther | \nManufacturing | \nUnmarried | \n
3 | \n12.0 | \nno | \nmale | \n4.0 | \nnot_member | \n22.0 | \nWhite | \nOther | \nOther | \nUnmarried | \n
4 | \n12.0 | \nno | \nmale | \n17.0 | \nnot_member | \n35.0 | \nWhite | \nOther | \nOther | \nMarried | \n