{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "# Summary Statistics with Python\n", "> Summary statistics gives you the tools you need to boil down massive datasets to reveal the highlights. In this chapter, you'll explore summary statistics including mean, median, and standard deviation, and learn how to accurately interpret them. You'll also develop your critical thinking skills, allowing you to choose the best summary statistics for your data. This is the Summary of lecture \"Introduction to Statistics in Python\", via datacamp.\n", "\n", "- toc: true \n", "- badges: true\n", "- comments: true\n", "- author: Chanseok Kang\n", "- categories: [Python, Datacamp, Statistics]\n", "- image: images/iqr.png" ] }, { "cell_type": "code", "execution_count": 1, "metadata": {}, "outputs": [], "source": [ "import numpy as np\n", "import pandas as pd\n", "import matplotlib.pyplot as plt\n", "\n", "plt.rcParams['figure.figsize'] = (10, 8)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## What is statistics?\n", "- Statistics\n", " - the practice and study of collecting and analyzing data\n", " - Summary Statistic - A fact about or summary of some data\n", "- Example\n", " - How likely is someone to purchase a product? Are people more likely to purchase it if they can use a different payment system?\n", " - How many occupants will your hotel have? How can you optimize occupancy?\n", " - How many sizes of jeans need to be manufactured so they can fit 95% of the population? Should the same number of each size be produced?\n", " - A/B tests: Which ad is more effective in getting people to purchase product?\n", "- Type of statistics\n", " - Descriptive statistics\n", " - Describe and summarize data\n", " - Inferential statistics\n", " - Use a sample of data to make inferences about a larger population\n", "- Type of data\n", " - Numeric (Quantitative)\n", " - Continuous (Measured)\n", " - Discrete (Counted)\n", " - Categorical (Qualitative)\n", " - Nomial (Unordered)\n", " - Ordinal (Ordered)\n", " " ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Measures of center\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Mean and median\n", "In this chapter, you'll be working with the [2018 Food Carbon Footprint Index](https://www.nu3.de/blogs/nutrition/food-carbon-footprint-index-2018) from nu3. The `food_consumption` dataset contains information about the kilograms of food consumed per person per year in each country in each food category (`consumption`) as well as information about the carbon footprint of that food category (`co2_emissions`) measured in kilograms of carbon dioxide, or $CO_2$, per person per year in each country.\n", "\n", "In this exercise, you'll compute measures of center to compare food consumption in the US and Belgium." ] }, { "cell_type": "code", "execution_count": 2, "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
countryfood_categoryconsumptionco2_emission
1Argentinapork10.5137.20
2Argentinapoultry38.6641.53
3Argentinabeef55.481712.00
4Argentinalamb_goat1.5654.63
5Argentinafish4.366.96
\n", "
" ], "text/plain": [ " country food_category consumption co2_emission\n", "1 Argentina pork 10.51 37.20\n", "2 Argentina poultry 38.66 41.53\n", "3 Argentina beef 55.48 1712.00\n", "4 Argentina lamb_goat 1.56 54.63\n", "5 Argentina fish 4.36 6.96" ] }, "execution_count": 2, "metadata": {}, "output_type": "execute_result" } ], "source": [ "food_consumption = pd.read_csv('./dataset/food_consumption.csv', index_col=0)\n", "food_consumption.head()" ] }, { "cell_type": "code", "execution_count": 3, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "42.132727272727266\n", "12.59\n", "44.650000000000006\n", "14.58\n" ] } ], "source": [ "# Filter for Belgium\n", "be_consumption = food_consumption[food_consumption['country'] == 'Belgium']\n", "\n", "# Filter for USA\n", "usa_consumption = food_consumption[food_consumption['country'] == 'USA']\n", "\n", "# Calculate mean and median consumption in Belgium\n", "print(np.mean(be_consumption['consumption']))\n", "print(np.median(be_consumption['consumption']))\n", "\n", "# Calculate mean and median consumption of USA\n", "print(np.mean(usa_consumption['consumption']))\n", "print(np.median(usa_consumption['consumption']))" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "or " ] }, { "cell_type": "code", "execution_count": 4, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ " mean median\n", "country \n", "Belgium 42.132727 12.59\n", "USA 44.650000 14.58\n" ] } ], "source": [ "# Subset for Belgium and USA\n", "be_and_usa = food_consumption[(food_consumption['country'] == 'Belgium') | \n", " (food_consumption['country'] == 'USA')]\n", "\n", "# Group by country, select consumption column, and compute mean and median\n", "print(be_and_usa.groupby('country')['consumption'].agg([np.mean, np.median]))" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Mean vs. median\n", "In the video, you learned that the mean is the sum of all the data points divided by the total number of data points, and the median is the middle value of the dataset where 50% of the data is less than the median, and 50% of the data is greater than the median. In this exercise, you'll compare these two measures of center." ] }, { "cell_type": "code", "execution_count": 5, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "mean 37.591615\n", "median 15.200000\n", "Name: co2_emission, dtype: float64\n" ] }, { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAlAAAAHSCAYAAAAjcvULAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjMsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+AADFEAAAYJUlEQVR4nO3db4xld33f8c83LDTgTfwHwtQ1qEuLRSFYOPEI0aJGszikgKvalaACIbogt9sHhJAGqdnkCanUSIuqhKZqGskNlK1EGCgB2cKEglymqFIh7IIbAy4yJRuC7dr5YxsGolCTbx/MdTrYMzv3t3tn753L6yVZM/fcM2e+yY+7euucM/dWdwcAgOn9wLwHAAA4aAQUAMAgAQUAMEhAAQAMElAAAIMEFADAoEMX85c94xnP6CNHjsz8uN/61rdyySWXzPy4zIf1XC7Wc/lY0+ViPXd35syZP+7uH9npuakCqqr+eZJ/kqST3JXkTUmuTLKe5Iokn0vyhu7+zrmOc+TIkZw+fXpg9OlsbGxkbW1t5sdlPqzncrGey8eaLhfrubuq+oPdntvzEl5VXZXkZ5KsdvcLkzwpyWuTvCPJO7v76iQPJbl5NuMCACy2ae+BOpTkqVV1KMnTktyf5GVJPjh5/lSSm2Y/HgDA4qlpPsqlqt6a5JeT/FmSjyd5a5JPd/dzJ88/O8nvTM5QPf5njyc5niQrKyvXra+vz276ic3NzRw+fHjmx2U+rOdysZ7Lx5ouF+u5u6NHj57p7tWdntvzHqiqujzJjUmek+ThJP85ySt32HXHEuvuW5LckiSrq6u9H9dZXb9dLtZzuVjP5WNNl4v1PD/TXML7ySS/391/1N3/N8mHkvydJJdNLuklybOS3LdPMwIALJRpAuprSV5SVU+rqkpyfZIvJflkkldP9jmW5Nb9GREAYLHsGVDd/Zls3Sz+uWy9hcEPZOuS3M8n+bmq+kqSpyd51z7OCQCwMKZ6H6jufnuStz9u81eTvHjmEwEALDgf5QIAMEhAAQAMElAAAIMEFADAIAEFADBIQAEADBJQAACDBBQAwCABBQAwSEABAAwSUAAAg6b6LLyD5MiJ2+c9wsycPXnDvEcAAHbgDBQAwCABBQAwSEABAAwSUAAAgwQUAMAgAQUAMEhAAQAMElAAAIMEFADAIAEFADBIQAEADBJQAACDBBQAwCABBQAwSEABAAwSUAAAgwQUAMAgAQUAMEhAAQAMElAAAIMEFADAIAEFADBIQAEADBJQAACDBBQAwCABBQAwSEABAAwSUAAAgwQUAMAgAQUAMEhAAQAMElAAAIMEFADAoD0DqqqeV1V3bvvvG1X1s1V1RVV9oqrumXy9/GIMDAAwb3sGVHd/ubuv7e5rk1yX5NtJPpzkRJI7uvvqJHdMHgMALL3RS3jXJ/nf3f0HSW5Mcmqy/VSSm2Y5GADAoqrunn7nqncn+Vx3/7uqeri7L9v23EPd/YTLeFV1PMnxJFlZWblufX19BmN/r83NzRw+fDhJcte9j8z8+PNyzVWXznuEudi+nhx81nP5WNPlYj13d/To0TPdvbrTc1MHVFU9Jcl9SX60ux+YNqC2W11d7dOnTw+MPp2NjY2sra0lSY6cuH3mx5+XsydvmPcIc7F9PTn4rOfysabLxXrurqp2DaiRS3ivzNbZpwcmjx+oqisnv+DKJA9e2JgAAAfDSEC9Lsn7tj2+LcmxyffHktw6q6EAABbZVAFVVU9L8vIkH9q2+WSSl1fVPZPnTs5+PACAxXNomp26+9tJnv64bX+Srb/KAwD4vuKdyAEABgkoAIBBAgoAYJCAAgAYJKAAAAYJKACAQQIKAGCQgAIAGCSgAAAGCSgAgEECCgBgkIACABgkoAAABgkoAIBBAgoAYJCAAgAYJKAAAAYJKACAQQIKAGCQgAIAGCSgAAAGCSgAgEECCgBgkIACABgkoAAABgkoAIBBAgoAYJCAAgAYJKAAAAYJKACAQQIKAGCQgAIAGCSgAAAGCSgAgEECCgBgkIACABgkoAAABgkoAIBBAgoAYJCAAgAYJKAAAAYJKACAQQIKAGCQgAIAGDRVQFXVZVX1war6X1V1d1X97aq6oqo+UVX3TL5evt/DAgAsgmnPQP1ako91999K8qIkdyc5keSO7r46yR2TxwAAS2/PgKqqH07yE0nelSTd/Z3ufjjJjUlOTXY7leSm/RoSAGCRTHMG6m8k+aMk/7GqPl9Vv1lVlyRZ6e77k2Ty9Zn7OCcAwMKo7j73DlWrST6d5KXd/Zmq+rUk30jylu6+bNt+D3X3E+6DqqrjSY4nycrKynXr6+uznD9Jsrm5mcOHDydJ7rr3kZkff16uuerSeY8wF9vXk4PPei4fa7pcrOfujh49eqa7V3d6bpqA+qtJPt3dRyaP/2627nd6bpK17r6/qq5MstHdzzvXsVZXV/v06dPn8X/CuW1sbGRtbS1JcuTE7TM//rycPXnDvEeYi+3rycFnPZePNV0u1nN3VbVrQO15Ca+7/0+SP6yqx+Lo+iRfSnJbkmOTbceS3DqDWQEAFt6hKfd7S5L3VtVTknw1yZuyFV8fqKqbk3wtyWv2Z0QAgMUyVUB1951JdjqFdf1sxwEAWHzeiRwAYJCAAgAYJKAAAAYJKACAQQIKAGCQgAIAGCSgAAAGCSgAgEECCgBgkIACABgkoAAABgkoAIBBAgoAYJCAAgAYJKAAAAYJKACAQQIKAGCQgAIAGCSgAAAGCSgAgEECCgBgkIACABgkoAAABgkoAIBBAgoAYJCAAgAYJKAAAAYJKACAQQIKAGCQgAIAGCSgAAAGCSgAgEECCgBgkIACABgkoAAABgkoAIBBAgoAYJCAAgAYJKAAAAYJKACAQQIKAGCQgAIAGCSgAAAGCSgAgEECCgBg0KFpdqqqs0m+meS7SR7t7tWquiLJ+5McSXI2yT/q7of2Z0wAgMUxcgbqaHdf292rk8cnktzR3VcnuWPyGABg6V3IJbwbk5yafH8qyU0XPg4AwOKbNqA6ycer6kxVHZ9sW+nu+5Nk8vWZ+zEgAMCiqe7ee6eqv9bd91XVM5N8IslbktzW3Zdt2+eh7r58h589nuR4kqysrFy3vr4+s+Efs7m5mcOHDydJ7rr3kZkff16uuerSeY8wF9vXk4PPei4fa7pcrOfujh49embbrUvfY6qA+p4fqPqlJJtJ/mmSte6+v6quTLLR3c8718+urq726dOnh37fNDY2NrK2tpYkOXLi9pkff17Onrxh3iPMxfb15OCznsvHmi4X67m7qto1oPa8hFdVl1TVDz32fZKfSvKFJLclOTbZ7ViSW2czLgDAYpvmbQxWkny4qh7b/7e6+2NV9dkkH6iqm5N8Lclr9m9MAIDFsWdAdfdXk7xoh+1/kuT6/RgKAGCReSdyAIBBAgoAYJCAAgAYJKAAAAYJKACAQQIKAGCQgAIAGCSgAAAGCSgAgEECCgBgkIACABgkoAAABgkoAIBBAgoAYJCAAgAYJKAAAAYJKACAQQIKAGCQgAIAGCSgAAAGCSgAgEECCgBgkIACABgkoAAABgkoAIBBAgoAYJCAAgAYJKAAAAYJKACAQQIKAGCQgAIAGCSgAAAGCSgAgEECCgBgkIACABgkoAAABgkoAIBBAgoAYJCAAgAYJKAAAAYJKACAQQIKAGCQgAIAGCSgAAAGCSgAgEFTB1RVPamqPl9VH5k8fk5Vfaaq7qmq91fVU/ZvTACAxTFyBuqtSe7e9vgdSd7Z3VcneSjJzbMcDABgUU0VUFX1rCQ3JPnNyeNK8rIkH5zscirJTfsxIADAopn2DNS/SfIvkvzF5PHTkzzc3Y9OHn89yVUzng0AYCEd2muHqvr7SR7s7jNVtfbY5h127V1+/niS40mysrKSjY2N85v0HDY3N//yuG+75tFz73yA7Mf/rw6C7evJwWc9l481XS7W8/zsGVBJXprkH1TVq5L8YJIfztYZqcuq6tDkLNSzkty30w939y1JbkmS1dXVXltbm8Xc32NjYyOPHfeNJ26f+fHn5ezr1+Y9wlxsX08OPuu5fKzpcrGe52fPS3jd/Qvd/azuPpLktUn+a3e/Psknk7x6stuxJLfu25QAAAvkQt4H6ueT/FxVfSVb90S9azYjAQAstmku4f2l7t5IsjH5/qtJXjz7kQAAFpt3IgcAGCSgAAAGCSgAgEECCgBgkIACABgkoAAABgkoAIBBAgoAYJCAAgAYJKAAAAYJKACAQQIKAGCQgAIAGCSgAAAGCSgAgEECCgBgkIACABgkoAAABgkoAIBBAgoAYJCAAgAYJKAAAAYJKACAQQIKAGCQgAIAGCSgAAAGCSgAgEECCgBgkIACABgkoAAABgkoAIBBAgoAYJCAAgAYJKAAAAYJKACAQQIKAGCQgAIAGCSgAAAGCSgAgEECCgBgkIACABgkoAAABgkoAIBBAgoAYJCAAgAYtGdAVdUPVtXvVtX/rKovVtW/nGx/TlV9pqruqar3V9VT9n9cAID5m+YM1J8neVl3vyjJtUleUVUvSfKOJO/s7quTPJTk5v0bEwBgcewZUL1lc/LwyZP/OsnLknxwsv1Ukpv2ZUIAgAVT3b33TlVPSnImyXOT/HqSf53k09393Mnzz07yO939wh1+9niS40mysrJy3fr6+uymn9jc3Mzhw4eTJHfd+8jMjz8v11x16bxHmIvt68nBZz2XjzVdLtZzd0ePHj3T3as7PXdomgN093eTXFtVlyX5cJLn77TbLj97S5JbkmR1dbXX1tam+ZVDNjY28thx33ji9pkff17Ovn5t3iPMxfb15OCznsvHmi4X63l+hv4Kr7sfTrKR5CVJLquqxwLsWUnum+1oAACLaZq/wvuRyZmnVNVTk/xkkruTfDLJqye7HUty634NCQCwSKa5hHdlklOT+6B+IMkHuvsjVfWlJOtV9a+SfD7Ju/ZxTgCAhbFnQHX37yX5sR22fzXJi/djKACAReadyAEABgkoAIBBAgoAYJCAAgAYJKAAAAYJKACAQQIKAGCQgAIAGCSgAAAGCSgAgEECCgBgkIACABgkoAAABgkoAIBBAgoAYJCAAgAYJKAAAAYJKACAQQIKAGCQgAIAGCSgAAAGCSgAgEECCgBgkIACABgkoAAABgkoAIBBAgoAYJCAAgAYJKAAAAYJKACAQQIKAGCQgAIAGCSgAAAGCSgAgEECCgBgkIACABgkoAAABgkoAIBBAgoAYJCAAgAYJKAAAAYJKACAQQIKAGCQgAIAGCSgAAAG7RlQVfXsqvpkVd1dVV+sqrdOtl9RVZ+oqnsmXy/f/3EBAOZvmjNQjyZ5W3c/P8lLkry5ql6Q5ESSO7r76iR3TB4DACy9PQOqu+/v7s9Nvv9mkruTXJXkxiSnJrudSnLTfg0JALBIqrun37nqSJJPJXlhkq9192Xbnnuou59wGa+qjic5niQrKyvXra+vX+DIT7S5uZnDhw8nSe6695GZH39errnq0nmPMBfb15ODz3ouH2u6XKzn7o4ePXqmu1d3em7qgKqqw0n+W5Jf7u4PVdXD0wTUdqurq3369OmB0aezsbGRtbW1JMmRE7fP/PjzcvbkDfMeYS62rycHn/VcPtZ0uVjP3VXVrgE11V/hVdWTk/x2kvd294cmmx+oqisnz1+Z5MFZDAsAsOim+Su8SvKuJHd3969ue+q2JMcm3x9LcuvsxwMAWDyHptjnpUnekOSuqrpzsu0Xk5xM8oGqujnJ15K8Zn9GBABYLHsGVHf/9yS1y9PXz3YcAIDF553IAQAGCSgAgEECCgBgkIACABgkoAAABgkoAIBBAgoAYJCAAgAYJKAAAAYJKACAQQIKAGCQgAIAGCSgAAAGCSgAgEECCgBgkIACABh0aN4DsLsjJ26f9wgzc/bkDfMeAQBmxhkoAIBBAgoAYJCAAgAYJKAAAAYJKACAQQIKAGCQgAIAGCSgAAAGCSgAgEECCgBgkIACABgkoAAABgkoAIBBAgoAYJCAAgAYdGjeA/D94ciJ26fe923XPJo3Dux/MZ09ecO8RwBgATgDBQAwSEABAAwSUAAAgwQUAMAgAQUAMEhAAQAMElAAAIMEFADAIAEFADBIQAEADBJQAACD9gyoqnp3VT1YVV/Ytu2KqvpEVd0z+Xr5/o4JALA4pjkD9Z4kr3jcthNJ7ujuq5PcMXkMAPB9Yc+A6u5PJfnTx22+Mcmpyfenktw047kAABbW+d4DtdLd9yfJ5OszZzcSAMBiq+7ee6eqI0k+0t0vnDx+uLsv2/b8Q929431QVXU8yfEkWVlZuW59fX0GY3+vzc3NHD58OEly172PzPz4XFwrT00e+LN5T7Gza666dN4jHDjbX58sB2u6XKzn7o4ePXqmu1d3eu7QeR7zgaq6srvvr6orkzy4247dfUuSW5JkdXW119bWzvNX7m5jYyOPHfeNJ26f+fG5uN52zaP5lbvO93+a++vs69fmPcKBs/31yXKwpsvFep6f872Ed1uSY5PvjyW5dTbjAAAsvmnexuB9Sf5HkudV1der6uYkJ5O8vKruSfLyyWMAgO8Le14n6e7X7fLU9TOeBQDgQPBO5AAAgwQUAMAgAQUAMEhAAQAMElAAAIMEFADAIAEFADBIQAEADBJQAACDBBQAwCABBQAwSEABAAwSUAAAgwQUAMAgAQUAMEhAAQAMElAAAIMOzXsAOEiOnLh93iPMzNmTN8x7BIADyxkoAIBBAgoAYJCAAgAYJKAAAAa5iRy+T12sG+Lfds2jeeMS3Xy/39zcDweDM1AAAIMEFADAIAEFADBIQAEADBJQAACDBBQAwCABBQAwSEABAAwSUAAAgwQUAMAgH+UCsEAu1kfsXIhpPp7HR9Kw7JyBAgAYJKAAAAYJKACAQQIKAGCQm8gBmLmDcDP8tJbphvid1mWaPwpYNIuwJs5AAQAMElAAAIMEFADAIAEFADDITeQAcA7LdEM8s+MMFADAoAsKqKp6RVV9uaq+UlUnZjUUAMAiO++AqqonJfn1JK9M8oIkr6uqF8xqMACARXUhZ6BenOQr3f3V7v5OkvUkN85mLACAxXUhAXVVkj/c9vjrk20AAEvtQv4Kr3bY1k/Yqep4kuOTh5tV9eUL+J27eUaSP96H4zIHP2M9l4r1XD7WdLkcxPWsd1y0X/XXd3viQgLq60meve3xs5Lc9/iduvuWJLdcwO/ZU1Wd7u7V/fwdXDzWc7lYz+VjTZeL9Tw/F3IJ77NJrq6q51TVU5K8NsltsxkLAGBxnfcZqO5+tKp+Osl/SfKkJO/u7i/ObDIAgAV1Qe9E3t0fTfLRGc1yIfb1EiEXnfVcLtZz+VjT5WI9z0N1P+G+bwAAzsFHuQAADDrwAeXjZA6+qjpbVXdV1Z1VdXqy7Yqq+kRV3TP5evm852RnVfXuqnqwqr6wbduO61db/u3k9fp7VfXj85ucneyynr9UVfdOXqN3VtWrtj33C5P1/HJV/b35TM1uqurZVfXJqrq7qr5YVW+dbPcavUAHOqB8nMxSOdrd1277U9oTSe7o7quT3DF5zGJ6T5JXPG7bbuv3yiRXT/47nuQ3LtKMTO89eeJ6Jsk7J6/Rayf3v2by7+1rk/zo5Gf+/eTfZRbHo0ne1t3PT/KSJG+erJvX6AU60AEVHyezzG5Mcmry/akkN81xFs6huz+V5E8ft3m39bsxyX/qLZ9OcllVXXlxJmUau6znbm5Mst7df97dv5/kK9n6d5kF0d33d/fnJt9/M8nd2frUEK/RC3TQA8rHySyHTvLxqjozeef6JFnp7vuTrX8AkjxzbtNxPnZbP6/Zg+unJ5d03r3tkrr1PECq6kiSH0vymXiNXrCDHlBTfZwMC++l3f3j2Tp1/Oaq+ol5D8S+8Zo9mH4jyd9Mcm2S+5P8ymS79Twgqupwkt9O8rPd/Y1z7brDNmu6g4MeUFN9nAyLrbvvm3x9MMmHs3UJ4IHHThtPvj44vwk5D7utn9fsAdTdD3T3d7v7L5L8h/z/y3TW8wCoqidnK57e290fmmz2Gr1ABz2gfJzMAVdVl1TVDz32fZKfSvKFbK3jsclux5LcOp8JOU+7rd9tSf7x5C99XpLkkccuI7C4HncPzD/M1ms02VrP11bVX6mq52TrxuPfvdjzsbuqqiTvSnJ3d//qtqe8Ri/QBb0T+bz5OJmlsJLkw1uv8RxK8lvd/bGq+mySD1TVzUm+luQ1c5yRc6iq9yVZS/KMqvp6krcnOZmd1++jSV6VrZuNv53kTRd9YM5pl/Vcq6prs3Up52ySf5Yk3f3FqvpAki9l66+93tzd353H3OzqpUnekOSuqrpzsu0X4zV6wbwTOQDAoIN+CQ8A4KITUAAAgwQUAMAgAQUAMEhAAQAMElAAAIMEFADAIAEFADDo/wE+vdnyohNGQgAAAABJRU5ErkJggg==\n", "text/plain": [ "
" ] }, "metadata": { "needs_background": "light" }, "output_type": "display_data" } ], "source": [ "# Subset for food_category equals rice\n", "rice_consumption = food_consumption[food_consumption['food_category'] == 'rice']\n", "\n", "# Histogram of co2_emission for rice and show plot\n", "rice_consumption['co2_emission'].hist();\n", "\n", "# Calculate mean and median of co2_emission with .agg()\n", "print(rice_consumption['co2_emission'].agg([np.mean, np.median]))" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Measures of spread\n", "- Variance\n", " - Average distance from each data point to the data's mean\n", "- Standard deviation\n", "- Mean absolute deviation\n", "- Standard deviation vs. mean absolute deviation\n", " - Standard deviation squares distances, penalizing longer distances more than shorter ones\n", " - Mean absolute deviation penalizes each distance equally\n", "- Quantiles (Or percentiles)\n", "- Interquartile range (IQR)\n", " - Height of the box in a boxplot\n", "- Outliers\n", " - Data point that is substantially different from the others\n", " " ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Quartiles, quantiles, and quintiles\n", "Quantiles are a great way of summarizing numerical data since they can be used to measure center and spread, as well as to get a sense of where a data point stands in relation to the rest of the data set. For example, you might want to give a discount to the 10% most active users on a website.\n", "\n", "In this exercise, you'll calculate quartiles, quintiles, and deciles, which split up a dataset into 4, 5, and 10 pieces, respectively." ] }, { "cell_type": "code", "execution_count": 6, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "[ 0. 5.21 16.53 62.5975 1712. ]\n", "[ 0. 3.54 11.026 25.59 99.978 1712. ]\n", "[0.00000000e+00 9.05555556e-01 4.19111111e+00 8.05333333e+00\n", " 1.32000000e+01 2.10944444e+01 3.58666667e+01 7.90622222e+01\n", " 1.86115556e+02 1.71200000e+03]\n" ] } ], "source": [ "# Calculate the quartiles of co2_emission\n", "print(np.quantile(food_consumption['co2_emission'], np.linspace(0, 1, 5)))\n", "\n", "# Calculate the quintiles of co2_emission\n", "print(np.quantile(food_consumption['co2_emission'], np.linspace(0, 1, 6)))\n", "\n", "# Calculate the deciles of co2_emission\n", "print(np.quantile(food_consumption['co2_emission'], np.linspace(0, 1, 10)))" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Variance and standard deviation\n", "Variance and standard deviation are two of the most common ways to measure the spread of a variable, and you'll practice calculating these in this exercise. Spread is important since it can help inform expectations. For example, if a salesperson sells a mean of 20 products a day, but has a standard deviation of 10 products, there will probably be days where they sell 40 products, but also days where they only sell one or two. Information like this is important, especially when making predictions.\n", "\n" ] }, { "cell_type": "code", "execution_count": 7, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ " var std\n", "food_category \n", "beef 88748.408132 297.906710\n", "dairy 17671.891985 132.935669\n", "eggs 21.371819 4.622966\n", "fish 921.637349 30.358481\n", "lamb_goat 16475.518363 128.356996\n", "nuts 35.639652 5.969895\n", "pork 3094.963537 55.632396\n", "poultry 245.026801 15.653332\n", "rice 2281.376243 47.763754\n", "soybeans 0.879882 0.938020\n", "wheat 71.023937 8.427570\n" ] }, { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAlAAAAHSCAYAAAAjcvULAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjMsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+AADFEAAAW9ElEQVR4nO3df4xld3nf8c8TLySUBf8IMHINzZrgoiBWATxCrmiiWUj45RS7bYhAKFm3bleVkogojpRNkSoqtappRFJVRY3cgrKpEhZKgmzhULBcJlGl4GQXDAt1iI27STFbWwHjsBSlXfrtH3MWZjczO/eZHzuzO6+XNJp7zz1z9juPz/W+9947d2qMEQAAZvdd270AAIBLjYACAGgSUAAATQIKAKBJQAEANAkoAICmPRfzD3vOc54z9u3bt+HjfOMb38gzn/nMjS9oFzCr2ZnV7MxqdmbVY16zM6vZrXdWx48f//MxxnNXuu2iBtS+ffty7NixDR9ncXExCwsLG1/QLmBWszOr2ZnV7Myqx7xmZ1azW++squpPV7vNU3gAAE0CCgCgSUABADQJKACAJgEFANAkoAAAmgQUAECTgAIAaBJQAABNAgoAoElAAQA0CSgAgCYBBQDQJKAAAJoEFABA055Zdqqqk0m+nuRbSc6MMear6pokH0iyL8nJJD8xxnhya5YJALBzdB6BOjDGeNkYY366fjjJ/WOMG5LcP10HALjsbeQpvFuSHJkuH0ly68aXAwCw880aUCPJx6vqeFUdmrbNjTFOJcn0+XlbsUAAgJ2mxhhr71T118cYX66q5yW5L8nPJrlnjHHVsn2eHGNcvcLXHkpyKEnm5uZuPHr06IYXffr06ezdu3fF20489tSGj79T7L/uyg0f40Kz4lxmNTuzmp1Z9ZjX7Mxqduud1YEDB44ve+nSOWYKqHO+oOqdSU4n+cdJFsYYp6rq2iSLY4wXX+hr5+fnx7Fjx1p/3koWFxezsLCw4m37Dt+74ePvFCfvvHnDx7jQrDiXWc3OrGZnVj3mNTuzmt16Z1VVqwbUmk/hVdUzq+pZZy8neW2SzyW5J8nBabeDSe5urwwA4BI0y9sYzCX5cFWd3f+3xhj/par+KMkHq+r2JH+W5M1bt0wAgJ1jzYAaYzya5AdX2P6VJK/ZikUBAOxk3okcAKBJQAEANAkoAIAmAQUA0CSgAACaBBQAQJOAAgBoElAAAE0CCgCgSUABADQJKACAJgEFANAkoAAAmgQUAECTgAIAaBJQAABNAgoAoElAAQA0CSgAgCYBBQDQJKAAAJoEFABAk4ACAGgSUAAATQIKAKBJQAEANAkoAIAmAQUA0CSgAACaBBQAQJOAAgBoElAAAE0CCgCgSUABADQJKACAJgEFANAkoAAAmgQUAECTgAIAaBJQAABNAgoAoElAAQA0CSgAgCYBBQDQJKAAAJoEFABAk4ACAGgSUAAATQIKAKBJQAEANAkoAIAmAQUA0CSgAACaBBQAQJOAAgBoElAAAE0CCgCgSUABADQJKACAJgEFANAkoAAAmgQUAECTgAIAaBJQAABNAgoAoElAAQA0CSgAgCYBBQDQJKAAAJoEFABAk4ACAGgSUAAATQIKAKBJQAEANM0cUFV1RVV9uqo+Ml2/vqoeqKqHq+oDVfX0rVsmAMDO0XkE6u1JHlp2/V1JfnWMcUOSJ5PcvpkLAwDYqWYKqKp6fpKbk/zH6XoleXWSD027HEly61YsEABgp6kxxto7VX0oyb9K8qwkv5DktiSfHGO8aLr9BUk+OsZ46QpfeyjJoSSZm5u78ejRoxte9OnTp7N3794Vbzvx2FMbPv5Osf+6Kzd8jAvNinOZ1ezManZm1WNeszOr2a13VgcOHDg+xphf6bY9a31xVf1YkifGGMerauHs5hV2XbHExhh3JbkrSebn58fCwsJKu7UsLi5mtePcdvjeDR9/pzj5toUNH+NCs+JcZjU7s5qdWfWY1+zManZbMas1AyrJq5K8qaremOR7kjw7yb9JclVV7RljnEny/CRf3tSVAQDsUGu+BmqM8UtjjOePMfYleUuS/zrGeFuSTyT58Wm3g0nu3rJVAgDsIBt5H6hfTPLzVfVIku9N8t7NWRIAwM42y1N43zbGWEyyOF1+NMkrN39JAAA7m3ciBwBoElAAAE0CCgCgSUABADQJKACAJgEFANAkoAAAmgQUAECTgAIAaBJQAABNAgoAoElAAQA0CSgAgCYBBQDQJKAAAJoEFABAk4ACAGgSUAAATQIKAKBJQAEANAkoAIAmAQUA0CSgAACaBBQAQJOAAgBoElAAAE0CCgCgSUABADQJKACAJgEFANAkoAAAmgQUAECTgAIAaBJQAABNAgoAoElAAQA0CSgAgCYBBQDQJKAAAJoEFABAk4ACAGgSUAAATQIKAKBJQAEANAkoAIAmAQUA0CSgAACaBBQAQJOAAgBoElAAAE0CCgCgSUABADQJKACAJgEFANAkoAAAmgQUAECTgAIAaBJQAABNAgoAoElAAQA0CSgAgCYBBQDQJKAAAJoEFABAk4ACAGgSUAAATQIKAKBJQAEANAkoAIAmAQUA0CSgAACaBBQAQJOAAgBoElAAAE1rBlRVfU9V/WFVfaaqPl9V/3zafn1VPVBVD1fVB6rq6Vu/XACA7TfLI1B/meTVY4wfTPKyJK+vqpuSvCvJr44xbkjyZJLbt26ZAAA7x5oBNZacnq4+bfoYSV6d5EPT9iNJbt2SFQIA7DA1xlh7p6orkhxP8qIk70nyy0k+OcZ40XT7C5J8dIzx0hW+9lCSQ0kyNzd349GjRze86NOnT2fv3r0r3nbisac2fPydYv91V274GBeaFecyq9mZ1ezMqse8ZmdWs1vvrA4cOHB8jDG/0m17ZjnAGONbSV5WVVcl+XCSH1hpt1W+9q4kdyXJ/Pz8WFhYmOWPvKDFxcWsdpzbDt+74ePvFCfftrDhY1xoVpzLrGZnVrMzqx7zmp1ZzW4rZtX6KbwxxteSLCa5KclVVXU2wJ6f5MubujIAgB1qlp/Ce+70yFOq6hlJfiTJQ0k+keTHp90OJrl7qxYJALCTzPIU3rVJjkyvg/quJB8cY3ykqv57kqNV9S+SfDrJe7dwnQAAO8aaATXG+GySl6+w/dEkr9yKRQEA7GTeiRwAoElAAQA0CSgAgCYBBQDQJKAAAJoEFABAk4ACAGgSUAAATQIKAKBJQAEANAkoAIAmAQUA0CSgAACaBBQAQJOAAgBoElAAAE0CCgCgSUABADQJKACAJgEFANAkoAAAmgQUAECTgAIAaBJQAABNAgoAoElAAQA0CSgAgCYBBQDQJKAAAJoEFABAk4ACAGgSUAAATQIKAKBJQAEANAkoAIAmAQUA0LRnuxfA6vYdvnfDx7hj/5nctgnH2aiTd9683UsAgE3jESgAgCYBBQDQJKAAAJoEFABAk4ACAGgSUAAATQIKAKBJQAEANAkoAIAmAQUA0CSgAACaBBQAQJOAAgBoElAAAE0CCgCgSUABADQJKACAJgEFANAkoAAAmgQUAECTgAIAaBJQAABNAgoAoElAAQA0CSgAgCYBBQDQJKAAAJoEFABAk4ACAGgSUAAATQIKAKBJQAEANO3Z7gWwO+w7fO92L2FNd+w/k9vWWOfJO2++SKsBYCfzCBQAQJOAAgBoElAAAE1rBlRVvaCqPlFVD1XV56vq7dP2a6rqvqp6ePp89dYvFwBg+83yCNSZJHeMMX4gyU1JfrqqXpLkcJL7xxg3JLl/ug4AcNlbM6DGGKfGGJ+aLn89yUNJrktyS5Ij025Hkty6VYsEANhJWq+Bqqp9SV6e5IEkc2OMU8lSZCV53mYvDgBgJ6oxxmw7Vu1N8ntJ/uUY43eq6mtjjKuW3f7kGOOvvA6qqg4lOZQkc3NzNx49enTDiz59+nT27t274m0nHntqw8e/nMw9I3n8m9u9ikvDLLPaf92VF2cxO9yF7oOcy6x6zGt2ZjW79c7qwIEDx8cY8yvdNlNAVdXTknwkycfGGL8ybftCkoUxxqmqujbJ4hjjxRc6zvz8/Dh27Fj7Gzjf4uJiFhYWVrztUnjDxovpjv1n8u4T3i91FrPMyhtpLrnQfZBzmVWPec3OrGa33llV1aoBNctP4VWS9yZ56Gw8Te5JcnC6fDDJ3e2VAQBcgmZ5aOJVSX4yyYmqenDa9k+T3Jnkg1V1e5I/S/LmrVkiAMDOsmZAjTH+W5Ja5ebXbO5yAAB2Pu9EDgDQJKAAAJoEFABAk4ACAGgSUAAATQIKAKBJQAEANAkoAIAmAQUA0CSgAACaBBQAQJOAAgBoElAAAE0CCgCgSUABADQJKACAJgEFANAkoAAAmgQUAECTgAIAaNqz3QuAS8m+w/du9xI2zck7b97uJQBcsjwCBQDQJKAAAJoEFABAk4ACAGgSUAAATQIKAKBJQAEANAkoAIAmAQUA0CSgAACaBBQAQJOAAgBoElAAAE0CCgCgSUABADQJKACAJgEFANAkoAAAmgQUAECTgAIAaBJQAABNAgoAoElAAQA0CSgAgCYBBQDQJKAAAJoEFABAk4ACAGgSUAAATQIKAKBJQAEANAkoAIAmAQUA0CSgAACaBBQAQJOAAgBoElAAAE0CCgCgSUABADQJKACAJgEFANAkoAAAmgQUAECTgAIAaBJQAABNAgoAoElAAQA0CSgAgCYBBQDQJKAAAJoEFABAk4ACAGgSUAAATWsGVFW9r6qeqKrPLdt2TVXdV1UPT5+v3tplAgDsHLM8AvXrSV5/3rbDSe4fY9yQ5P7pOgDArrBmQI0xfj/JV8/bfEuSI9PlI0lu3eR1AQDsWOt9DdTcGONUkkyfn7d5SwIA2NlqjLH2TlX7knxkjPHS6frXxhhXLbv9yTHGiq+DqqpDSQ4lydzc3I1Hjx7d8KJPnz6dvXv3rnjbicee2vDxLydzz0ge/+Z2r+LSsNtmtf+6K9f9tRe6D3Ius+oxr9mZ1ezWO6sDBw4cH2PMr3TbnnWu5fGqunaMcaqqrk3yxGo7jjHuSnJXkszPz4+FhYV1/pHfsbi4mNWOc9vhezd8/MvJHfvP5N0n1vufeXfZbbM6+baFdX/the6DnMusesxrdmY1u62Y1XqfwrsnycHp8sEkd2/OcgAAdr5Z3sbg/Un+IMmLq+pLVXV7kjuT/GhVPZzkR6frAAC7wprPV4wx3rrKTa/Z5LUAAFwSvBM5AECTgAIAaBJQAABNAgoAoElAAQA0CSgAgCYBBQDQJKAAAJoEFABAk4ACAGgSUAAATQIKAKBJQAEANAkoAIAmAQUA0CSgAACaBBQAQJOAAgBoElAAAE0CCgCgac92LwBgo/Ydvne7l7CqO/afyW2N9Z288+YtXA2wWTwCBQDQJKAAAJoEFABAk4ACAGgSUAAATQIKAKBJQAEANHkfKNilNvLeSd33NgK43HgECgCgSUABADQJKACAJgEFANAkoAAAmgQUAECTgAIAaBJQAABNAgoAoElAAQA0CSgAgCYBBQDQJKAAAJoEFABAk4ACAGgSUAAATQIKAKBJQAEANAkoAIAmAQUA0CSgAACaBBQAQJOAAgBoElAAAE0CCgCgSUABADQJKACAJgEFANAkoAAAmgQUAECTgAIAaBJQAABNe7Z7AQB8x77D9273EjbFyTtv3u4lwJbyCBQAQJOAAgBoElAAAE0CCgCgSUABADQJKACAJgEFANAkoAAAmryRJgCbbr1vCHrH/jO5bYe9mejl9Kag3qh183gECgCgSUABADQJKACApg0FVFW9vqq+UFWPVNXhzVoUAMBOtu6AqqorkrwnyRuSvCTJW6vqJZu1MACAnWojj0C9MskjY4xHxxj/J8nRJLdszrIAAHaujQTUdUn+57LrX5q2AQBc1mqMsb4vrHpzkteNMf7RdP0nk7xyjPGz5+13KMmh6eqLk3xh/cv9tuck+fNNOM5uYFazM6vZmdXszKrHvGZnVrNb76y+b4zx3JVu2MgbaX4pyQuWXX9+ki+fv9MY464kd23gz/krqurYGGN+M495uTKr2ZnV7MxqdmbVY16zM6vZbcWsNvIU3h8luaGqrq+qpyd5S5J7NmdZAAA717ofgRpjnKmqn0nysSRXJHnfGOPzm7YyAIAdakO/C2+M8btJfneT1tKxqU8JXubManZmNTuzmp1Z9ZjX7Mxqdps+q3W/iBwAYLfyq1wAAJouqYDyq2POVVUvqKpPVNVDVfX5qnr7tP2dVfVYVT04fbxx2df80jS/L1TV67Zv9dujqk5W1YlpLsembddU1X1V9fD0+eppe1XVv53m9dmqesX2rv7iqaoXLzt/Hqyqv6iqn3NuLamq91XVE1X1uWXb2udRVR2c9n+4qg5ux/ey1VaZ1S9X1R9P8/hwVV01bd9XVd9cdn792rKvuXG67z4yzbO24/vZSqvMqn2f2w1/V64yqw8sm9PJqnpw2r4159UY45L4yNIL1b+Y5IVJnp7kM0lest3r2uaZXJvkFdPlZyX5kyz9Wp13JvmFFfZ/yTS3705y/TTPK7b7+7jIMzuZ5DnnbfvXSQ5Plw8nedd0+Y1JPpqkktyU5IHtXv82zeyKJP8ryfc5t779/f5wklck+dx6z6Mk1yR5dPp89XT56u3+3i7SrF6bZM90+V3LZrVv+X7nHecPk/ytaY4fTfKG7f7eLtKsWve53fJ35UqzOu/2dyf5Z1t5Xl1Kj0D51THnGWOcGmN8arr89SQP5cLvBn9LkqNjjL8cY/yPJI9kaa673S1JjkyXjyS5ddn23xhLPpnkqqq6djsWuM1ek+SLY4w/vcA+u+rcGmP8fpKvnre5ex69Lsl9Y4yvjjGeTHJfktdv/eovrpVmNcb4+BjjzHT1k1l6H8FVTfN69hjjD8bS33q/ke/M97Kxynm1mtXuc7vi78oLzWp6FOknkrz/QsfY6Hl1KQWUXx1zAVW1L8nLkzwwbfqZ6eHx9519KiFmmCQjycer6ngtvUt+ksyNMU4lS1Ga5HnTdvNa8pac+z8i59bKuueRmS35h1n6l/9Z11fVp6vq96rqh6Zt12VpPmfttll17nPOq+SHkjw+xnh42bZNP68upYBa6XlJP0KYpKr2JvntJD83xviLJP8+yfcneVmSU1l6KDMxwyR51RjjFUnekOSnq+qHL7Dvrp9XLb1J7puS/Odpk3Orb7XZ7PqZVdU7kpxJ8pvTplNJ/sYY4+VJfj7Jb1XVs7O7Z9W9z+3mWZ311pz7j74tOa8upYCa6VfH7DZV9bQsxdNvjjF+J0nGGI+PMb41xvh/Sf5DvvNUyq6f4Rjjy9PnJ5J8OEuzefzsU3PT5yem3Xf9vLIUmp8aYzyeOLfW0D2PdvXMphfN/1iSt01Pn2R6Ouor0+XjWXotz9/M0qyWP823a2a1jvvcbj+v9iT5e0k+cHbbVp1Xl1JA+dUx55me531vkofGGL+ybPvy1+n83SRnf0rhniRvqarvrqrrk9yQpRfQ7QpV9cyqetbZy1l6IevnsjSXsz8BdTDJ3dPle5L81PRTVDcleersUzS7yDn/knNuXVD3PPpYktdW1dXT0zKvnbZd9qrq9Ul+Mcmbxhj/e9n251bVFdPlF2bpPHp0mtfXq+qm6f97P5XvzPeyto773G7/u/JHkvzxGOPbT81t2Xm13a+k73xk6adZ/iRL9fiO7V7Pdn8k+dtZerjxs0kenD7emOQ/JTkxbb8nybXLvuYd0/y+kMvwp1jWmNcLs/QTKZ9J8vmz51CS701yf5KHp8/XTNsryXumeZ1IMr/d38NFntdfS/KVJFcu2+bcWvpe35+lpwX+b5b+FXv7es6jLL3+55Hp4x9s9/d1EWf1SJZep3P2/1u/Nu3796f75meSfCrJ31l2nPksxcMXk/y7TG8EfTl9rDKr9n1uN/xdudKspu2/nuSfnLfvlpxX3okcAKDpUnoKDwBgRxBQAABNAgoAoElAAQA0CSgAgCYBBQDQJKAAAJoEFABA0/8H82s3heHAJKUAAAAASUVORK5CYII=\n", "text/plain": [ "
" ] }, "metadata": { "needs_background": "light" }, "output_type": "display_data" } ], "source": [ "# Print variance and sd of co2_emission for each food_category\n", "print(food_consumption.groupby('food_category')['co2_emission'].agg([np.var, np.std]))\n", "\n", "# Create histogram of co2_emission for food_category 'beef'\n", "food_consumption[food_consumption['food_category'] == 'beef']['co2_emission'].hist();" ] }, { "cell_type": "code", "execution_count": 8, "metadata": {}, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAlAAAAHSCAYAAAAjcvULAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjMsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+AADFEAAAWxUlEQVR4nO3df4zle33X8ddbliqy5HKvt4zbK3HEECKyKYUJQdFmNlgKXFPAWANp8F7BbBtLQ5M1cdMm9camya0WmmgaDZUbbg2yqAUhd2+FG2RLmgi6Sy7sJbftpWRbuVz3BqEXFkl06cc/5uwyd5jZM++dM3POzDweyWTOj+9857Pv+Z6Z555zZk6NMQIAwPb9qXkvAABgvxFQAABNAgoAoElAAQA0CSgAgCYBBQDQdGQvP9ntt98+lpeXZ7rPb33rW3n2s589030eNGZ0Y+YznRlNZ0bTmdGNmc90ez2jCxcufHWM8f2bXbenAbW8vJzz58/PdJ/nzp3L6urqTPd50JjRjZnPdGY0nRlNZ0Y3Zj7T7fWMquoPt7rOQ3gAAE0CCgCgSUABADQJKACAJgEFANAkoAAAmgQUAECTgAIAaBJQAABNAgoAoElAAQA0CSgAgCYBBQDQJKAAAJoEFABA09SAqqrnV9Unq+rRqvpCVb1zcvk9VfV4VT08eXv97i8XAGD+jmxjm6tJTo0xPltVz0lyoaoemlz3q2OMX9m95QEALJ6pATXGeCLJE5PT36yqR5PcsdsLAwBYVK3nQFXVcpIfSvKZyUXvqKrPV9V9VXXrjNcGALCQaoyxvQ2rjib57SS/NMb4UFUtJflqkpHkF5McG2O8bZOPO5nkZJIsLS29/MyZM7Nae5LkypUrOXr06PXzFx9/aqb7n6fjd9wyk/1snBFPZz7TmdF0ZjSdGd2Y+Uy31zM6ceLEhTHGymbXbSugquqZSR5I8rExxrs3uX45yQNjjJfcaD8rKyvj/Pnz21nztp07dy6rq6vXzy+fPjvT/c/TpXvvnMl+Ns6IpzOf6cxoOjOazoxuzHym2+sZVdWWAbWd38KrJO9N8uj6eKqqY+s2e1OSR3a6UACA/WA7v4X3qiRvTXKxqh6eXPZzSd5SVS/N2kN4l5L85K6sEABgwWznt/B+J0ltctWDs18OAMDi85fIAQCaBBQAQJOAAgBoElAAAE0CCgCgSUABADQJKACAJgEFANAkoAAAmgQUAECTgAIAaBJQAABNAgoAoElAAQA0CSgAgCYBBQDQJKAAAJoEFABAk4ACAGgSUAAATQIKAKBJQAEANAkoAIAmAQUA0CSgAACaBBQAQJOAAgBoElAAAE0CCgCgSUABADQJKACAJgEFANAkoAAAmgQUAECTgAIAaBJQAABNAgoAoElAAQA0CSgAgCYBBQDQJKAAAJoEFABAk4ACAGgSUAAATUfmvQBgPpZPn932tqeOX83dje332qV775z3EoBDxj1QAABNAgoAoElAAQA0CSgAgCYBBQDQJKAAAJoEFABAk4ACAGgSUAAATQIKAKBJQAEANAkoAIAmAQUA0CSgAACaBBQAQJOAAgBoElAAAE0CCgCgSUABADQJKACAJgEFANAkoAAAmgQUAECTgAIAaBJQAABNAgoAoElAAQA0CSgAgCYBBQDQNDWgqur5VfXJqnq0qr5QVe+cXH5bVT1UVY9N3t+6+8sFAJi/7dwDdTXJqTHGX0nyyiQ/XVUvTnI6ySfGGC9M8onJeQCAA29qQI0xnhhjfHZy+ptJHk1yR5I3JLl/stn9Sd64W4sEAFgkNcbY/sZVy0k+leQlSf5ojPHcddd9fYzxPQ/jVdXJJCeTZGlp6eVnzpzZ4ZKf7sqVKzl69Oj18xcff2qm+5+n43fcMpP9bJwRT3dY59O5rSw9K7n87V1czA7N6rayE4f1OOowoxszn+n2ekYnTpy4MMZY2ey6bQdUVR1N8ttJfmmM8aGq+uPtBNR6Kysr4/z5842lT3fu3Lmsrq5eP798+uxM9z9Pl+69cyb72Tgjnu6wzqdzWzl1/GredfHILq5mZ2Z1W9mJw3ocdZjRjZnPdHs9o6raMqC29Vt4VfXMJL+Z5P1jjA9NLr5cVccm1x9L8uQsFgsAsOi281t4leS9SR4dY7x73VUfTXLX5PRdST4y++UBACye7dwn/6okb01ysaoenlz2c0nuTfIfqurtSf4oyY/vzhIBABbL1IAaY/xOktri6lfPdjkAAIvPXyIHAGgSUAAATQIKAKBJQAEANAkoAIAmAQUA0CSgAACaBBQAQJOAAgBoElAAAE0CCgCgaTsvJgzAHlk+fXbeS5iJS/feOe8lwK5yDxQAQJOAAgBoElAAAE0CCgCgSUABADQJKACAJgEFANAkoAAAmgQUAECTgAIAaBJQAABNAgoAoElAAQA0CSgAgCYBBQDQJKAAAJoEFABAk4ACAGgSUAAATQIKAKBJQAEANAkoAIAmAQUA0CSgAACaBBQAQJOAAgBoElAAAE0CCgCgSUABADQJKACAJgEFANAkoAAAmgQUAECTgAIAaBJQAABNAgoAoElAAQA0CSgAgCYBBQDQJKAAAJoEFABAk4ACAGgSUAAATQIKAKBJQAEANAkoAIAmAQUA0CSgAACaBBQAQJOAAgBoElAAAE0CCgCgSUABADQJKACAJgEFANAkoAAAmgQUAECTgAIAaBJQAABNAgoAoElAAQA0CSgAgKapAVVV91XVk1X1yLrL7qmqx6vq4cnb63d3mQAAi2M790C9L8lrN7n8V8cYL528PTjbZQEALK6pATXG+FSSr+3BWgAA9oUaY0zfqGo5yQNjjJdMzt+T5O4k30hyPsmpMcbXt/jYk0lOJsnS0tLLz5w5M4Nlf9eVK1dy9OjR6+cvPv7UTPc/T8fvuGUm+9k4I57usM6nc1tZelZy+du7uJgdmtVtZSdmdRwdlO9hm31NDuttbbvMZ7q9ntGJEycujDFWNrvuZgNqKclXk4wkv5jk2BjjbdP2s7KyMs6fP7/9lW/DuXPnsrq6ev388umzM93/PF26986Z7GfjjHi6wzqfzm3l1PGredfFI7u4mp2Z1W1lJ2Z1HB2U72GbfU0O621tu8xnur2eUVVtGVA39Vt4Y4zLY4zvjDH+JMmvJ3nFThYIALCf3FRAVdWxdWfflOSRrbYFADhopt4nX1UfSLKa5Paq+nKSf5pktapemrWH8C4l+cldXCMAwEKZGlBjjLdscvF7d2EtAAD7gr9EDgDQJKAAAJoEFABAk4ACAGgSUAAATQIKAKBJQAEANAkoAIAmAQUA0CSgAACaBBQAQNPU18IDWHTLp8/Oewk5dfxq7l6AdQB7wz1QAABNAgoAoElAAQA0CSgAgCYBBQDQJKAAAJoEFABAk4ACAGgSUAAATQIKAKBJQAEANAkoAIAmAQUA0CSgAACaBBQAQNOReS+Aw2H59Nl5L2FLp45fzd3bXN+le+/c5dUAsB+4BwoAoElAAQA0CSgAgCYBBQDQJKAAAJoEFABAk4ACAGgSUAAATQIKAKBJQAEANAkoAIAmAQUA0CSgAACaBBQAQJOAAgBoElAAAE0CCgCgSUABADQJKACAJgEFANAkoAAAmgQUAECTgAIAaBJQAABNAgoAoElAAQA0CSgAgCYBBQDQJKAAAJoEFABAk4ACAGgSUAAATQIKAKDpyLwXAPvJ8umz814CAAvAPVAAAE0CCgCgSUABADQJKACAJgEFANAkoAAAmgQUAECTgAIAaBJQAABNAgoAoElAAQA0TQ2oqrqvqp6sqkfWXXZbVT1UVY9N3t+6u8sEAFgc27kH6n1JXrvhstNJPjHGeGGST0zOAwAcClMDaozxqSRf23DxG5LcPzl9f5I3znhdAAAL62afA7U0xngiSSbvnze7JQEALLYaY0zfqGo5yQNjjJdMzv/xGOO5667/+hhj0+dBVdXJJCeTZGlp6eVnzpyZwbK/68qVKzl69Oj18xcff2qm+5+n43fcMpP9bJzRPCzy12XpWcnlb897FYvNjKYzo6fb7PvXInwvWmTmM91ez+jEiRMXxhgrm1135Cb3ebmqjo0xnqiqY0me3GrDMcZ7krwnSVZWVsbq6upNfsrNnTt3Luv3effpszPd/zxd+onVmexn44zmYZG/LqeOX827Lt7sTeFwMKPpzOjpNvv+tQjfixaZ+Uy3SDO62YfwPprkrsnpu5J8ZDbLAQBYfNv5MwYfSPLfkryoqr5cVW9Pcm+SH6mqx5L8yOQ8AMChMPX+5jHGW7a46tUzXgsAwL7gL5EDADQJKACAJgEFANAkoAAAmgQUAECTgAIAaBJQAABNAgoAoElAAQA0CSgAgCYBBQDQJKAAAJqmvpgwAHQtnz77PZedOn41d29y+aK7dO+d814CC8g9UAAATQIKAKBJQAEANAkoAIAmAQUA0CSgAACaBBQAQJOAAgBoElAAAE0CCgCgSUABADQJKACAJgEFANAkoAAAmgQUAEDTkXkvgK0tnz47k/2cOn41d89oXwCAe6AAANoEFABAk4ACAGgSUAAATQIKAKBJQAEANAkoAIAmAQUA0CSgAACaBBQAQJOAAgBoElAAAE0CCgCgSUABADQJKACAJgEFANAkoAAAmgQUAECTgAIAaBJQAABNAgoAoElAAQA0CSgAgCYBBQDQJKAAAJoEFABAk4ACAGgSUAAATQIKAKBJQAEANAkoAIAmAQUA0CSgAACaBBQAQJOAAgBoElAAAE0CCgCgSUABADQJKACAJgEFANAkoAAAmgQUAECTgAIAaBJQAABNAgoAoOnITj64qi4l+WaS7yS5OsZYmcWiAAAW2Y4CauLEGOOrM9gPAMC+4CE8AICmnQbUSPLxqrpQVSdnsSAAgEVXY4yb/+CqHxhjfKWqnpfkoSQ/M8b41IZtTiY5mSRLS0svP3PmzE7W+z2uXLmSo0ePXj9/8fGnZrr/g2DpWcnlb897FYvLfKYzo+nMaLr9OqPjd9yyJ59n48+z3bDff0auP4b24uty4sSJC1s9v3tHAfW0HVXdk+TKGONXttpmZWVlnD9/fiaf75pz585ldXX1+vnl02dnuv+D4NTxq3nXxVk83e1gMp/pzGg6M5puv87o0r137snn2fjzbDfs95+R64+hvfi6VNWWAXXTD+FV1bOr6jnXTid5TZJHbnZ/AAD7xU7+K7CU5MNVdW0//36M8V9msioAgAV20wE1xvhSkh+c4VoAAPYFf8YAAKBJQAEANAkoAIAmAQUA0CSgAACaBBQAQJOAAgBoElAAAE0CCgCgSUABADQJKACAJgEFANB00y8mDACHwfLps3vyeU4dv5q79+hzsXPugQIAaBJQAABNAgoAoElAAQA0CSgAgCYBBQDQJKAAAJoEFABAk4ACAGgSUAAATQIKAKBJQAEANAkoAIAmAQUA0CSgAACaBBQAQJOAAgBoElAAAE0CCgCgSUABADQJKACAJgEFANAkoAAAmgQUAECTgAIAaBJQAABNAgoAoElAAQA0CSgAgCYBBQDQJKAAAJoEFABAk4ACAGgSUAAATQIKAKBJQAEANAkoAIAmAQUA0CSgAACaBBQAQJOAAgBoElAAAE0CCgCgSUABADQJKACAJgEFANAkoAAAmgQUAECTgAIAaBJQAABNAgoAoElAAQA0CSgAgCYBBQDQJKAAAJoEFABAk4ACAGgSUAAATQIKAKBJQAEANAkoAICmHQVUVb22qn6vqr5YVadntSgAgEV20wFVVc9I8mtJXpfkxUneUlUvntXCAAAW1U7ugXpFki+OMb40xvi/Sc4kecNslgUAsLh2ElB3JPmf685/eXIZAMCBVmOMm/vAqh9P8qNjjH84Of/WJK8YY/zMhu1OJjk5OfuiJL9388vd1O1JvjrjfR40ZnRj5jOdGU1nRtOZ0Y2Zz3R7PaO/OMb4/s2uOLKDnX45yfPXnf8LSb6ycaMxxnuSvGcHn+eGqur8GGNlt/Z/EJjRjZnPdGY0nRlNZ0Y3Zj7TLdKMdvIQ3v9I8sKq+ktV9X1J3pzko7NZFgDA4rrpe6DGGFer6h1JPpbkGUnuG2N8YWYrAwBYUDt5CC9jjAeTPDijtdysXXt48AAxoxszn+nMaDozms6Mbsx8pluYGd30k8gBAA4rL+UCANC0bwJq2svGVNWfrqoPTq7/TFUt7/0q56Oqnl9Vn6yqR6vqC1X1zk22Wa2qp6rq4cnbL8xjrfNUVZeq6uLk339+k+urqv7l5Bj6fFW9bB7rnJeqetG64+PhqvpGVf3shm0O3XFUVfdV1ZNV9ci6y26rqoeq6rHJ+1u3+Ni7Jts8VlV37d2q984W8/kXVfW7k9vRh6vquVt87A1vkwfFFjO6p6oeX3dbev0WH3soXjJtixl9cN18LlXVw1t87HyOozHGwr9l7Unqf5DkBUm+L8nnkrx4wzb/KMm/mZx+c5IPznvdezifY0leNjn9nCS/v8l8VpM8MO+1znlOl5LcfoPrX5/kt5JUklcm+cy81zzHWT0jyf/K2t9AWX/5oTuOkvxwkpcleWTdZf88yenJ6dNJfnmTj7styZcm72+dnL513v+ePZrPa5IcmZz+5c3mM7nuhrfJg/K2xYzuSfKPp3zc1J99B+VtsxltuP5dSX5hkY6j/XIP1HZeNuYNSe6fnP5PSV5dVbWHa5ybMcYTY4zPTk5/M8mj8Vfhb8YbkvzGWPPpJM+tqmPzXtScvDrJH4wx/nDeC5m3Mcanknxtw8Xrv9/cn+SNm3zojyZ5aIzxtTHG15M8lOS1u7bQOdlsPmOMj48xrk7Ofjprfyfw0NriGNqOQ/OSaTea0eRn+d9L8oE9XdQU+yWgtvOyMde3mdxwn0ry5/ZkdQtk8tDlDyX5zCZX/7Wq+lxV/VZV/dU9XdhiGEk+XlUXJn8hfyMvT/Rdb87W36wO+3GUJEtjjCeStf/AJHneJts4nta8LWv37G5m2m3yoHvH5GHO+7Z4GNgxtOZvJrk8xnhsi+vnchztl4Da7J6kjb8+uJ1tDrSqOprkN5P87BjjGxuu/mzWHo75wST/Ksl/3uv1LYBXjTFeluR1SX66qn54w/WH/hhKkskfxv2xJP9xk6sdR9t36I+nqvr5JFeTvH+LTabdJg+yf53kLyd5aZInsvYQ1UaH/hiaeEtufO/TXI6j/RJQ23nZmOvbVNWRJLfk5u4y3Zeq6plZi6f3jzE+tPH6McY3xhhXJqcfTPLMqrp9j5c5V2OMr0zeP5nkw1m7e3y9bb080SHwuiSfHWNc3niF4+i6y9ce3p28f3KTbQ718TR50vzfTvITY/JElY22cZs8sMYYl8cY3xlj/EmSX8/m//ZDfQwl13+e/50kH9xqm3kdR/sloLbzsjEfTXLtt1z+bpL/utWN9qCZPD783iSPjjHevcU2f/7ac8Kq6hVZ+9r/771b5XxV1bOr6jnXTmftSa6PbNjso0n+/uS38V6Z5KlrD9McMlv+b++wH0frrP9+c1eSj2yyzceSvKaqbp08PPOayWUHXlW9Nsk/SfJjY4z/s8U227lNHlgbnl/5pmz+b/eSacnfSvK7Y4wvb3blXI+jeT/zfrtvWfsNqd/P2m8k/Pzksn+WtRtokvyZrD3k8MUk/z3JC+a95j2czd/I2t26n0/y8OTt9Ul+KslPTbZ5R5IvZO23OD6d5K/Pe917PKMXTP7tn5vM4doxtH5GleTXJsfYxSQr8173HOb0Z7MWRLesu+xQH0dZi8knkvy/rN0j8PasPb/yE0kem7y/bbLtSpJ/u+5j3zb5nvTFJP9g3v+WPZzPF7P23J1r34+u/Yb0DyR5cHJ609vkQXzbYkb/bvJ95vNZi6JjG2c0Of89P/sO4ttmM5pc/r5r33/WbbsQx5G/RA4A0LRfHsIDAFgYAgoAoElAAQA0CSgAgCYBBQDQJKAAAJoEFABAk4ACAGj6/wauE2yikZqJAAAAAElFTkSuQmCC\n", "text/plain": [ "
" ] }, "metadata": { "needs_background": "light" }, "output_type": "display_data" } ], "source": [ "# Create histogram of co2_emission for food_category 'eggs'\n", "food_consumption[food_consumption['food_category'] == 'eggs']['co2_emission'].hist();" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Finding outliers using IQR\n", "Outliers can have big effects on statistics like mean, as well as statistics that rely on the mean, such as variance and standard deviation. Interquartile range, or IQR, is another way of measuring spread that's less influenced by outliers. IQR is also often used to find outliers. If a value is less than $Q1−1.5×IQR$ or greater than $Q3+1.5×IQR$, it's considered an outlier. \n", "![iqr](image/iqr.png)\n", "In this exercise, you'll calculate IQR and use it to find some outliers." ] }, { "cell_type": "code", "execution_count": 9, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "country\n", "Albania 1777.85\n", "Algeria 707.88\n", "Angola 412.99\n", "Argentina 2172.40\n", "Armenia 1109.93\n", " ... \n", "Uruguay 1634.91\n", "Venezuela 1104.10\n", "Vietnam 641.51\n", "Zambia 225.30\n", "Zimbabwe 350.33\n", "Name: co2_emission, Length: 130, dtype: float64\n" ] } ], "source": [ "# Calculate total co2_emission per country: emissions_by_country\n", "emissions_by_country = food_consumption.groupby('country')['co2_emission'].sum()\n", "\n", "print(emissions_by_country)" ] }, { "cell_type": "code", "execution_count": 10, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "country\n", "Argentina 2172.4\n", "Name: co2_emission, dtype: float64\n" ] } ], "source": [ "# Compute the first and third quartiles and IQR of emissions_by_country\n", "q1 = np.quantile(emissions_by_country, 0.25)\n", "q3 = np.quantile(emissions_by_country, 0.75)\n", "iqr = q3 - q1\n", "\n", "# Calculate the lower and upper cutoffs for outliers\n", "lower = q1 - 1.5 * iqr\n", "upper = q3 + 1.5 * iqr\n", "\n", "# Subset emissions_by_country to find outliers\n", "outliers = emissions_by_country[(emissions_by_country > upper) | (emissions_by_country < lower)]\n", "print(outliers)" ] } ], "metadata": { "kernelspec": { "display_name": "Python 3", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.7.6" } }, "nbformat": 4, "nbformat_minor": 4 }