{ "cells": [ { "cell_type": "code", "execution_count": null, "metadata": { "deletable": false, "editable": false }, "outputs": [], "source": [ "# Initialize OK\n", "from client.api.notebook import Notebook\n", "ok = Notebook('lab05.ok')" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "# Lab 5: Simulations\n", "\n", "Welcome to Lab 5! \n", "\n", "We will go over [iteration](https://www.inferentialthinking.com/chapters/09/2/Iteration.html) and [simulations](https://www.inferentialthinking.com/chapters/09/3/Simulation.html), as well as introduce the concept of [randomness](https://www.inferentialthinking.com/chapters/09/Randomness.html) and [conditional probability](https://www.inferentialthinking.com/chapters/18/Updating_Predictions.html).\n", "\n", "The data used in this lab will contain salary data and other statistics for basketball players from the 2014-2015 NBA season. This data was collected from the following sports analytic sites: [Basketball Reference](http://www.basketball-reference.com) and [Spotrac](http://www.spotrac.com).\n", "\n", "**Lab Queue**: You can find the Lab Queue at [lab.data8.org](https://lab.data8.org/). Whenever you feel stuck or need some further clarification, add yourself to the queue to get help from a GSI or academic intern! Please list your name, breakout room number, and purpose on your ticket!\n", "\n", "**Deadline**: If you are not attending lab, you have to complete this lab and submit by Wednesday, 2/24 before 8:59 A.M. PST in order to receive lab credit. Otherwise, please attend the lab you are enrolled in, get checked off with your GSI or academic intern **AND** submit this assignment by the end of the lab section (with whatever progress you've made) to receive lab credit.\n", "\n", "**Submission**: Once you're finished, select \"Save and Checkpoint\" in the File menu and then execute the submit cell below (or at the end). The result will contain a link that you can use to check that your assignment has been submitted successfully. \n", "\n", "First, set up the notebook by running the cell below." ] }, { "cell_type": "code", "execution_count": 40, "metadata": {}, "outputs": [], "source": [ "# Run this cell, but please don't change it.\n", "\n", "# These lines import the Numpy and Datascience modules.\n", "import numpy as np\n", "from datascience import *\n", "\n", "# These lines do some fancy plotting magic\n", "import matplotlib\n", "%matplotlib inline\n", "import matplotlib.pyplot as plt\n", "plt.style.use('fivethirtyeight')\n", "\n", "from client.api.notebook import *\n", "def new_save_notebook(self):\n", " \"\"\" Saves the current notebook by\n", " injecting JavaScript to save to .ipynb file.\n", " \"\"\"\n", " try:\n", " from IPython.display import display, Javascript\n", " except ImportError:\n", " log.warning(\"Could not import IPython Display Function\")\n", " print(\"Make sure to save your notebook before sending it to OK!\")\n", " return\n", "\n", " if self.mode == \"jupyter\":\n", " display(Javascript('IPython.notebook.save_checkpoint();'))\n", " display(Javascript('IPython.notebook.save_notebook();'))\n", " elif self.mode == \"jupyterlab\":\n", " display(Javascript('document.querySelector(\\'[data-command=\"docmanager:save\"]\\').click();')) \n", "\n", " print('Saving notebook...', end=' ')\n", "\n", " ipynbs = [path for path in self.assignment.src\n", " if os.path.splitext(path)[1] == '.ipynb']\n", " # Wait for first .ipynb to save\n", " if ipynbs:\n", " if wait_for_save(ipynbs[0]):\n", " print(\"Saved '{}'.\".format(ipynbs[0]))\n", " else:\n", " log.warning(\"Timed out waiting for IPython save\")\n", " print(\"Could not automatically save \\'{}\\'\".format(ipynbs[0]))\n", " print(\"Make sure your notebook\"\n", " \" is correctly named and saved before submitting to OK!\".format(ipynbs[0]))\n", " return False \n", " else:\n", " print(\"No valid file sources found\")\n", " return True\n", "\n", "def wait_for_save(filename, timeout=600):\n", " \"\"\"Waits for FILENAME to update, waiting up to TIMEOUT seconds.\n", " Returns True if a save was detected, and False otherwise.\n", " \"\"\"\n", " modification_time = os.path.getmtime(filename)\n", " start_time = time.time()\n", " while time.time() < start_time + timeout:\n", " if (os.path.getmtime(filename) > modification_time and\n", " os.path.getsize(filename) > 0):\n", " return True\n", " time.sleep(0.2)\n", " print(\"\\nERROR!\\n YOUR SUBMISSION DID NOT GO THROUGH. PLEASE TRY AGAIN. IF THIS PROBLEM PERSISTS POST ON PIAZZA RIGHT AWAY.\\n ERROR!\" + \"\\n\"*20)\n", " return False\n", "\n", "Notebook.save_notebook = new_save_notebook\n", "\n", "# Don't change this cell; just run it. \n", "from client.api.notebook import Notebook\n", "ok = Notebook('lab05.ok')" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## 1. Nachos and Conditionals" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "In Python, the boolean data type contains only two unique values: `True` and `False`. Expressions containing comparison operators such as `<` (less than), `>` (greater than), and `==` (equal to) evaluate to Boolean values. A list of common comparison operators can be found below!\n", "\n", "" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Run the cell below to see an example of a comparison operator in action." ] }, { "cell_type": "code", "execution_count": 41, "metadata": {}, "outputs": [], "source": [ "3 > 1 + 1" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "We can even assign the result of a comparison operation to a variable." ] }, { "cell_type": "code", "execution_count": 42, "metadata": {}, "outputs": [], "source": [ "result = 10 / 2 == 5\n", "result" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Arrays are compatible with comparison operators. The output is an array of boolean values." ] }, { "cell_type": "code", "execution_count": 43, "metadata": {}, "outputs": [], "source": [ "make_array(1, 5, 7, 8, 3, -1) > 3" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "One day, when you come home after a long week, you see a hot bowl of nachos waiting on the dining table! Let's say that whenever you take a nacho from the bowl, it will either have only **cheese**, only **salsa**, **both** cheese and salsa, or **neither** cheese nor salsa (a sad tortilla chip indeed). \n", "\n", "Let's try and simulate taking nachos from the bowl at random using the function, `np.random.choice(...)`." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### `np.random.choice`\n", "\n", "`np.random.choice` picks one item at random from the given array. It is equally likely to pick any of the items. Run the cell below several times, and observe how the results change." ] }, { "cell_type": "code", "execution_count": 44, "metadata": {}, "outputs": [], "source": [ "nachos = make_array('cheese', 'salsa', 'both', 'neither')\n", "np.random.choice(nachos)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "To repeat this process multiple times, pass in an int `n` as the second argument to return `n` different random choices. By default, `np.random.choice` samples **with replacement** and returns an *array* of items. \n", "\n", "Run the next cell to see an example of sampling with replacement 10 times from the `nachos` array." ] }, { "cell_type": "code", "execution_count": 45, "metadata": {}, "outputs": [], "source": [ "np.random.choice(nachos, 10)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "To count the number of times a certain type of nacho is randomly chosen, we can use `np.count_nonzero`" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### `np.count_nonzero`\n", "\n", "`np.count_nonzero` counts the number of non-zero values that appear in an array. When an array of boolean values are passed through the function, it will count the number of `True` values (remember that in Python, `True` is coded as 1 and `False` is coded as 0.)\n", "\n", "Run the next cell to see an example that uses `np.count_nonzero`." ] }, { "cell_type": "code", "execution_count": 46, "metadata": {}, "outputs": [], "source": [ "np.count_nonzero(make_array(True, False, False, True, True))" ] }, { "cell_type": "markdown", "metadata": { "deletable": false, "editable": false }, "source": [ "**Question 1.1** Assume we took ten nachos at random, and stored the results in an array called `ten_nachos` as done below. Find the number of nachos with only cheese using code (do not hardcode the answer). \n", "\n", "*Hint:* Our solution involves a comparison operator (e.g. `=`, `<`, ...) and the `np.count_nonzero` method.\n", "\n", "" ] }, { "cell_type": "code", "execution_count": 47, "metadata": {}, "outputs": [], "source": [ "ten_nachos = make_array('neither', 'cheese', 'both', 'both', 'cheese', 'salsa', 'both', 'neither', 'cheese', 'both')\n", "number_cheese = ...\n", "number_cheese" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "deletable": false, "editable": false }, "outputs": [], "source": [ "ok.grade(\"q11\");" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "**Conditional Statements**\n", "\n", "A conditional statement is a multi-line statement that allows Python to choose among different alternatives based on the truth value of an expression.\n", "\n", "Here is a basic example.\n", "\n", "```\n", "def sign(x):\n", " if x > 0:\n", " return 'Positive'\n", " else:\n", " return 'Negative'\n", "```\n", "\n", "If the input `x` is greater than `0`, we return the string `'Positive'`. Otherwise, we return `'Negative'`.\n", "\n", "If we want to test multiple conditions at once, we use the following general format.\n", "\n", "```\n", "if :\n", " \n", "elif :\n", " \n", "elif :\n", " \n", "...\n", "else:\n", " \n", "```\n", "\n", "Only the body for the first conditional expression that is true will be evaluated. Each `if` and `elif` expression is evaluated and considered in order, starting at the top. `elif` can only be used if an `if` clause precedes it. As soon as a true value is found, the corresponding body is executed, and the rest of the conditional statement is skipped. If none of the `if` or `elif` expressions are true, then the `else body` is executed. \n", "\n", "For more examples and explanation, refer to the section on conditional statements [here](https://www.inferentialthinking.com/chapters/09/1/conditional-statements.html)." ] }, { "cell_type": "markdown", "metadata": { "deletable": false, "editable": false }, "source": [ "**Question 1.2** Complete the following conditional statement so that the string `'More please'` is assigned to the variable `say_please` if the number of nachos with cheese in `ten_nachos` is less than `5`. Use the if statement to do this (do not directly reassign the variable `say_please`). \n", "\n", "*Hint*: You should be using `number_cheese` from Question 1.\n", "\n", "" ] }, { "cell_type": "code", "execution_count": 49, "metadata": { "for_assignment_type": "student" }, "outputs": [], "source": [ "say_please = '?'\n", "\n", "if ...:\n", " say_please = 'More please'\n", "say_please" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "deletable": false, "editable": false }, "outputs": [], "source": [ "ok.grade(\"q12\");" ] }, { "cell_type": "markdown", "metadata": { "deletable": false, "editable": false }, "source": [ "**Question 1.3** Write a function called `nacho_reaction` that returns a reaction (as a string) based on the type of nacho passed in as an argument. Use the table below to match the nacho type to the appropriate reaction.\n", "\n", "\n", "\n", "*Hint:* If you're failing the test, double check the spelling of your reactions.\n", "\n", "" ] }, { "cell_type": "code", "execution_count": 51, "metadata": { "for_assignment_type": "student" }, "outputs": [], "source": [ "def nacho_reaction(nacho):\n", " if nacho == \"cheese\":\n", " return ...\n", " ... :\n", " ...\n", " ... :\n", " ...\n", " ... :\n", " ...\n", "\n", "spicy_nacho = nacho_reaction('salsa')\n", "spicy_nacho" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "deletable": false, "editable": false }, "outputs": [], "source": [ "ok.grade(\"q13\");" ] }, { "cell_type": "markdown", "metadata": { "deletable": false, "editable": false }, "source": [ "**Question 1.4** Create a table `ten_nachos_reactions` that consists of the nachos in `ten_nachos` as well as the reactions for each of those nachos. The columns should be called `Nachos` and `Reactions`.\n", "\n", "*Hint:* Use the `apply` method. \n", "\n", "" ] }, { "cell_type": "code", "execution_count": 56, "metadata": { "for_assignment_type": "student" }, "outputs": [], "source": [ "ten_nachos_tbl = Table().with_column('Nachos', ten_nachos)\n", "...\n", "ten_nachos_reactions" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "deletable": false, "editable": false }, "outputs": [], "source": [ "ok.grade(\"q14\");" ] }, { "cell_type": "markdown", "metadata": { "deletable": false, "editable": false }, "source": [ "**Question 1.5** Using code, find the number of 'Wow!' reactions for the nachos in `ten_nachos_reactions`.\n", "\n", "" ] }, { "cell_type": "code", "execution_count": 58, "metadata": {}, "outputs": [], "source": [ "number_wow_reactions = ...\n", "number_wow_reactions" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "deletable": false, "editable": false }, "outputs": [], "source": [ "ok.grade(\"q15\");" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## 2. Simulations and For Loops\n", "Using a `for` statement, we can perform a task multiple times. This is known as iteration. The general structure of a for loop is:\n", "\n", "`for in :` followed by indented lines of code that are repeated for each element of the `array` being iterated over. You can read more about for loops [here](https://www.inferentialthinking.com/chapters/09/2/Iteration.html). \n", "\n", "**NOTE:** We often use `i` as the `placeholder` in our class examples, but you could name it anything! Some examples can be found below." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "One use of iteration is to loop through a set of values. For instance, we can print out all of the colors of the rainbow." ] }, { "cell_type": "code", "execution_count": 61, "metadata": {}, "outputs": [], "source": [ "rainbow = make_array(\"red\", \"orange\", \"yellow\", \"green\", \"blue\", \"indigo\", \"violet\")\n", "\n", "for color in rainbow:\n", " print(color)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "We can see that the indented part of the `for` loop, known as the body, is executed once for each item in `rainbow`. The name `color` is assigned to the next value in `rainbow` at the start of each iteration. Note that the name `color` is arbitrary; we could easily have named it something else. The important thing is we stay consistent throughout the `for` loop. " ] }, { "cell_type": "code", "execution_count": 62, "metadata": {}, "outputs": [], "source": [ "for another_name in rainbow:\n", " print(another_name)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "In general, however, we would like the variable name to be somewhat informative. " ] }, { "cell_type": "markdown", "metadata": { "deletable": false, "editable": false }, "source": [ "**Question 2.1** In the following cell, we've loaded the text of _Pride and Prejudice_ by Jane Austen, split it into individual words, and stored these words in an array `p_and_p_words`. Using a `for` loop, assign `longer_than_five` to the number of words in the novel that are more than 5 letters long.\n", "\n", "*Hint*: You can find the number of letters in a word with the `len` function.\n", "\n", "" ] }, { "cell_type": "code", "execution_count": 63, "metadata": { "for_assignment_type": "student" }, "outputs": [], "source": [ "austen_string = open('Austen_PrideAndPrejudice.txt', encoding='utf-8').read()\n", "p_and_p_words = np.array(austen_string.split())\n", "\n", "longer_than_five = ...\n", "\n", "# a for loop would be useful here\n", "\n", "longer_than_five" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "deletable": false, "editable": false }, "outputs": [], "source": [ "ok.grade(\"q21\");" ] }, { "cell_type": "markdown", "metadata": { "deletable": false, "editable": false }, "source": [ "**Question 2.2** Using a simulation with 10,000 trials, assign `num_different` to the number of times, in 10,000 trials, that two words picked uniformly at random (with replacement) from Pride and Prejudice have different lengths. \n", "\n", "*Hint 1*: What function did we use in section 1 to sample at random with replacement from an array? \n", "\n", "*Hint 2*: Remember that `!=` checks for non-equality between two items.\n", "\n", "" ] }, { "cell_type": "code", "execution_count": 65, "metadata": { "for_assignment_type": "student" }, "outputs": [], "source": [ "trials = 10000\n", "num_different = ...\n", "\n", "for ... in ...:\n", " ...\n", "num_different" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "deletable": false, "editable": false }, "outputs": [], "source": [ "ok.grade(\"q22\");" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## 3. Probabilities in Probabilities\n", "\n", "Table manipulation lets us see quickly what groups each row in a table belong to. We would then like to use probability to ask questions about the data in our tables. Let's start with a 100 row table that tracks 100 students `Year` and `Major`.\n", "\n", "Run the cell below to load the data." ] }, { "cell_type": "code", "execution_count": 67, "metadata": {}, "outputs": [], "source": [ "# Just run this cell\n", "year = np.array(['Second']*60 + ['Third']*40)\n", "major = np.array(['Undeclared']*30+['Declared']*30+['Undeclared']*8+['Declared']*32)\n", "students = Table().with_columns(\n", " 'Year', year,\n", " 'Major', major\n", ")\n", "students.show(3)" ] }, { "cell_type": "markdown", "metadata": { "deletable": false, "editable": false }, "source": [ "**Question 3.1** On the line below, please restructure the `students` table so that counts for the following combinations are displayed: (Declared, Second Years), (Undeclared, Second Years), (Declared, Third Years), (Undeclared, Third Years). Create a new table `all_combos` such that the years of students are the rows, and the Declared/Undeclared statuses are the columns. \n", "\n", "*Hint:* What data should you start with? What methods reshape a 100 row table into a table with as many rows and columns as you have combos?\n", "\n", "" ] }, { "cell_type": "code", "execution_count": 68, "metadata": { "for_assignment_type": "student" }, "outputs": [], "source": [ "all_combos = ...\n", "all_combos" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "deletable": false, "editable": false }, "outputs": [], "source": [ "ok.grade(\"q31\");" ] }, { "cell_type": "markdown", "metadata": { "deletable": false, "editable": false }, "source": [ "Now that you can see the how the 100 students belong to four catagories.\n", "\n", "**Question 3.2** If you know that a student row had `Year` == `Second`, but you could not see the `Major` coulmn, what is the chance (proportion) that the student you have picked was Declared?\n", "\n", "*Hint:* No tricks here, just look at the table you've just made!\n", "\n", "" ] }, { "cell_type": "code", "execution_count": 71, "metadata": { "for_assignment_type": "student" }, "outputs": [], "source": [ "declared_given_second = ..." ] }, { "cell_type": "code", "execution_count": null, "metadata": { "deletable": false, "editable": false }, "outputs": [], "source": [ "ok.grade(\"q32\");" ] }, { "cell_type": "markdown", "metadata": { "deletable": false, "editable": false }, "source": [ "**Question 3.3** Final question (harder!). Imagine again that you are looking at the students table, but can only see the `Major` column. You see the first row says `Declared`, what is the chance this student is a `Third` year?\n", "\n", "To answer this question, presenting your data in a new visualization called a tree diagram is the best thing to do.\n", "\n", "![](https://www.inferentialthinking.com/images/tree_students.png)\n", "\n", "Today we've filled in this tree for you, but connect it back to the pivot table you made above.\n", "\n", "*Hint:* The variable names followed by `...` have hard statistical definitions. If you are wondering how these definitions help you calculate the answer to our question, the first cell in this notebook links you to the relevant textbook section.\n", "\n", "" ] }, { "cell_type": "code", "execution_count": 74, "metadata": { "for_assignment_type": "student" }, "outputs": [], "source": [ "prob_of_third_year = ...\n", "likelihood_declared_given_third = ...\n", "total_probability_declared = ...\n", "\n", "# Just run this line, it was defined for you. \n", "# If you're interested, we came up with this equation using Bayes' Rule.\n", "prob_third_given_declared = (prob_of_third_year * likelihood_declared_given_third)\\\n", " / total_probability_declared" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "deletable": false, "editable": false }, "outputs": [], "source": [ "ok.grade(\"q33\");" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## 4. Sampling Basketball Data\n", "\n", "We will now introduce the topic of sampling, which we’ll be discussing in more depth in this week’s lectures. We’ll guide you through this code, but if you wish to read more about different kinds of samples before attempting this question, you can check out [section 10 of the textbook](https://www.inferentialthinking.com/chapters/10/Sampling_and_Empirical_Distributions.html).\n", "\n", "Run the cell below to load player and salary data that we will use for our sampling. " ] }, { "cell_type": "code", "execution_count": 77, "metadata": {}, "outputs": [], "source": [ "player_data = Table().read_table(\"player_data.csv\")\n", "salary_data = Table().read_table(\"salary_data.csv\")\n", "full_data = salary_data.join(\"PlayerName\", player_data, \"Name\")\n", "\n", "# The show method immediately displays the contents of a table. \n", "# This way, we can display the top of two tables using a single cell.\n", "player_data.show(3)\n", "salary_data.show(3)\n", "full_data.show(3)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Rather than getting data on every player (as in the tables loaded above), imagine that we had gotten data on only a smaller subset of the players. For 492 players, it's not so unreasonable to expect to see all the data, but usually we aren't so lucky. \n", "\n", "If we want to make estimates about a certain numerical property of the population (known as a statistic, e.g. the mean or median), we may have to come up with these estimates based only on a smaller sample. Whether these estimates are useful or not often depends on how the sample was gathered. We have prepared some example sample datasets to see how they compare to the full NBA dataset. Later we'll ask you to create your own samples to see how they behave." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "To save typing and increase the clarity of your code, we will package the analysis code into a few functions. This will be useful in the rest of the lab as we will repeatedly need to create histograms and collect summary statistics from that data." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "We've defined the `histograms` function below, which takes a table with columns `Age` and `Salary` and draws a histogram for each one. It uses bin widths of 1 year for `Age` and $1,000,000 for `Salary`." ] }, { "cell_type": "code", "execution_count": 78, "metadata": { "scrolled": true }, "outputs": [], "source": [ "def histograms(t):\n", " ages = t.column('Age')\n", " salaries = t.column('Salary')/1000000\n", " t1 = t.drop('Salary').with_column('Salary', salaries)\n", " age_bins = np.arange(min(ages), max(ages) + 2, 1) \n", " salary_bins = np.arange(min(salaries), max(salaries) + 1, 1)\n", " t1.hist('Age', bins=age_bins, unit='year')\n", " plt.title('Age distribution')\n", " t1.hist('Salary', bins=salary_bins, unit='million dollars')\n", " plt.title('Salary distribution') \n", " \n", "histograms(full_data)\n", "print('Two histograms should be displayed below')" ] }, { "cell_type": "markdown", "metadata": { "deletable": false, "editable": false }, "source": [ "**Question 4.1**. Create a function called `compute_statistics` that takes a table containing ages and salaries and:\n", "- Draws a histogram of ages\n", "- Draws a histogram of salaries\n", "- Returns a two-element array containing the average age and average salary (in that order)\n", "\n", "You can call the `histograms` function to draw the histograms! \n", "\n", "*Note:* More charts will be displayed when running the test cell. Please feel free to ignore the charts.\n", "\n", "" ] }, { "cell_type": "code", "execution_count": 79, "metadata": {}, "outputs": [], "source": [ "def compute_statistics(age_and_salary_data):\n", " ...\n", " age = ...\n", " salary = ...\n", " ...\n", " \n", "\n", "full_stats = compute_statistics(full_data)\n", "full_stats" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "deletable": false, "editable": false }, "outputs": [], "source": [ "ok.grade(\"q41\");" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Simple random sampling\n", "A more justifiable approach is to sample uniformly at random from the players. In a **simple random sample (SRS) without replacement**, we ensure that each player is selected at most once. Imagine writing down each player's name on a card, putting the cards in an box, and shuffling the box. Then, pull out cards one by one and set them aside, stopping when the specified sample size is reached." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Producing simple random samples\n", "Sometimes, it’s useful to take random samples even when we have the data for the whole population. It helps us understand sampling accuracy.\n", "\n", "### `sample`\n", "\n", "The table method `sample` produces a random sample from the table. By default, it draws at random **with replacement** from the rows of a table. It takes in the sample size as its argument and returns a **table** with only the rows that were selected. \n", "\n", "Run the cell below to see an example call to `sample()` with a sample size of 5, with replacement." ] }, { "cell_type": "code", "execution_count": 82, "metadata": {}, "outputs": [], "source": [ "# Just run this cell\n", "\n", "salary_data.sample(5)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The optional argument `with_replacement=False` can be passed through `sample()` to specify that the sample should be drawn without replacement.\n", "\n", "Run the cell below to see an example call to `sample()` with a sample size of 5, without replacement." ] }, { "cell_type": "code", "execution_count": 83, "metadata": {}, "outputs": [], "source": [ "# Just run this cell\n", "\n", "salary_data.sample(5, with_replacement=False)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "**Question 4.2** Produce a simple random sample of size 44 from `full_data`. Run your analysis on it again. Run the cell a few times to see how the histograms and statistics change across different samples.\n", "\n", "- How much does the average age change across samples? \n", "- What about average salary?" ] }, { "cell_type": "code", "execution_count": 84, "metadata": { "scrolled": false }, "outputs": [], "source": [ "my_small_srswor_data = ...\n", "my_small_stats = ...\n", "my_small_stats" ] }, { "cell_type": "markdown", "metadata": { "deletable": false, "manual_problem_id": "q_3_7_samples" }, "source": [ "*Write your answer here, replacing this text.*" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## 5. Submission\n", "\n", "Congratulations, you're done with Lab 5! Be sure to \n", "- **Run all the tests** (the next cell has a shortcut for that). \n", "- **Save and Checkpoint** from the `File` menu.\n", "- **Run the cell at the bottom to submit your work**.\n", "- If you're in lab, ask one of the staff members to check you off.\n", "\n", "**Note:** If you did not complete the optional section of this lab (Question 6), when you run the next cell you will not be passing all of the autograder tests. As long as you are passing all tests for the required section of this lab (Questions 1-4), you will earn full credit." ] }, { "cell_type": "code", "execution_count": 55, "metadata": { "scrolled": false }, "outputs": [], "source": [ "# For your convenience, you can run this cell to run all the tests at once!\n", "import os\n", "_ = [ok.grade(q[:-3]) for q in os.listdir(\"tests\") if q.startswith('q')]" ] }, { "cell_type": "code", "execution_count": 56, "metadata": {}, "outputs": [], "source": [ "_ = ok.submit()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## 6. Optional Section\n", "\n", "Everything below this cell is optional and will not count towards your lab grade. It is recommended that you complete this section, but not required!" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Simulations and For Loops (cont.)" ] }, { "cell_type": "markdown", "metadata": { "deletable": false, "editable": false }, "source": [ "**Question 6.1** We can use `np.random.choice` to simulate multiple trials.\n", "\n", "Allie is playing darts. Her dartboard contains ten equal-sized zones with point values from 1 to 10. Write code that simulates her total score after 1000 dart tosses.\n", "\n", "*Hint:* First decide the possible values you can take in the experiment (point values in this case). Then use `np.random.choice` to simulate Allie's tosses. Finally, sum up the scores to get Allie's total score.\n", "\n", "" ] }, { "cell_type": "code", "execution_count": 85, "metadata": {}, "outputs": [], "source": [ "possible_point_values = ...\n", "num_tosses = 1000\n", "simulated_tosses = ...\n", "total_score = ...\n", "total_score" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "deletable": false, "editable": false }, "outputs": [], "source": [ "ok.grade(\"q61\");" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Simple random sampling (cont.)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "**Question 6.2** As in the previous question, analyze several simple random samples of size 100 from `full_data`. \n", "- Do the histogram shapes seem to change more or less across samples of 100 than across samples of size 44? \n", "- Are the sample averages and histograms closer to their true values/shape for age or for salary? What did you expect to see?" ] }, { "cell_type": "code", "execution_count": 87, "metadata": {}, "outputs": [], "source": [ "my_large_srswor_data = ...\n", "my_large_stats = ...\n", "my_large_stats" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "*Write your answer here, replacing this text.*" ] } ], "metadata": { "anaconda-cloud": {}, "kernelspec": { "display_name": "Python 3", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.7.6" } }, "nbformat": 4, "nbformat_minor": 1 }