{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Homework 5: Pivot Tables and Iteration"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Reading**: \n",
    "* [Cross-Classifying by more than one variable](https://www.inferentialthinking.com/chapters/08/3/cross-classifying-by-more-than-one-variable.html) \n",
    "* [Bike Sharing](https://www.inferentialthinking.com/chapters/08/5/bike-sharing-in-the-bay-area.html) \n",
    "* [Randomness](https://www.inferentialthinking.com/chapters/09/randomness.html)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Please complete this notebook by filling in the cells provided. Before you begin, execute the following cell. Each time you start your server, you will need to execute this cell again to load the tests.\n",
    "\n",
    "Homework 5 is due Thursday, 2/20, at 11:59pm.  Start early so that you can come to office hours if you're stuck. Check the website for the office hours schedule. Late work will not be accepted as per the policies of this course.\n",
    "\n",
    "Directly sharing answers is not okay, but discussing problems with the course staff or with other students is encouraged. Refer to the policies page to learn more about how to learn cooperatively."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Don't change this cell; just run it. \n",
    "\n",
    "import numpy as np\n",
    "from datascience import *\n",
    "\n",
    "# These lines do some fancy plotting magic.\n",
    "import matplotlib\n",
    "%matplotlib inline\n",
    "import matplotlib.pyplot as plt\n",
    "plt.style.use('fivethirtyeight')\n",
    "import warnings\n",
    "warnings.simplefilter('ignore', FutureWarning)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 1. Causes of Death by Year\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This exercise is designed to give you practice using the Table method `pivot`. [Here](http://data8.org/fa18/python-reference.html) is a link to the Python reference page in case you need a quick refresher.\n",
    "\n",
    "We'll be looking at a [dataset](http://www.healthdata.gov/dataset/leading-causes-death-zip-code-1999-2013) from the California Department of Public Health that records the cause of death, as recorded on a death certificate, for everyone who died in California from 1999 to 2013.  The data is in the file `causes_of_death.csv.zip`. Each row records the number of deaths by a specific cause in one year in one ZIP code.\n",
    "\n",
    "To make the file smaller, we've compressed it; run the next cell to unzip and load it."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!unzip -o causes_of_death.csv.zip\n",
    "causes = Table.read_table('causes_of_death.csv')\n",
    "causes"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The causes of death in the data are abbreviated.  We've provided a table called `abbreviations.csv` to translate the abbreviations."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "abbreviations = Table.read_table('abbreviations.csv')\n",
    "abbreviations.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The dataset is missing data on certain causes of death for certain years.  It looks like those causes of death are relatively rare, so for some purposes it makes sense to drop them from consideration.  Of course, we'll have to keep in mind that we're no longer looking at a comprehensive report on all deaths in California.\n",
    "\n",
    "**Question 1.1.** Let's clean up our data. First, filter out the HOM, HYP, and NEP rows from the table for the reasons described above. Next, join together the abbreviations table and our causes of death table so that we have a more detailed description of each disease in each row. Lastly, drop the column which contains the acronym of the disease, and rename the column with the full description 'Cause of Death'. Assign the variable `cleaned_causes` to the resulting table. \n",
    "\n",
    "*Hint:* You should expect this to take more than one line. Use many lines and many intermediate tables to complete this question. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "cleaned_causes = ...\n",
    "cleaned_causes"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We're going to examine the changes in causes of death over time.  To make a plot of those numbers, we need to have a table with one row per year, and the information about all the causes of death for each year.\n",
    "\n",
    "**Question 1.2.** Create a table with one row for each year and a column for each kind of death, where each cell contains the number of deaths by that cause in that year. Call the table `cleaned_causes_by_year`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "cleaned_causes_by_year = ...\n",
    "cleaned_causes_by_year.show(15)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Question 1.3.** Make a plot of all the causes of death by year, using your cleaned-up version of the dataset.  There should be a single plot with one line per cause of death.\n",
    "\n",
    "*Hint:* Use the Table method `plot`.  If you pass only a single argument, a line will be made for each of the other columns."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "..."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "After seeing the plot above, we would now like to examine the distributions of diseases over the years using percentages. Below, we have assigned `distributions` to a table with all of the same columns, but the raw counts in the cells are replaced by the percentage of the the total number of deaths by a particular disease that happened in that specific year. \n",
    "\n",
    "Try to understand the code below. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def percents(array_x):\n",
    "    return np.round( (array_x/sum(array_x))*100, 2)\n",
    "\n",
    "labels = cleaned_causes_by_year.labels\n",
    "distributions = Table().with_columns(labels[0], cleaned_causes_by_year.column(0),\n",
    "                                     labels[1], percents(cleaned_causes_by_year.column(1)),\n",
    "                                     labels[2], percents(cleaned_causes_by_year.column(2)),\n",
    "                                     labels[3], percents(cleaned_causes_by_year.column(3)),\n",
    "                                     labels[4], percents(cleaned_causes_by_year.column(4)),\n",
    "                                     labels[5], percents(cleaned_causes_by_year.column(5)),\n",
    "                                     labels[6], percents(cleaned_causes_by_year.column(6)),\n",
    "                                     labels[7], percents(cleaned_causes_by_year.column(7)),\n",
    "                                     labels[8], percents(cleaned_causes_by_year.column(8)),\n",
    "                                     labels[9], percents(cleaned_causes_by_year.column(9)),\n",
    "                                     labels[10], percents(cleaned_causes_by_year.column(10)),\n",
    "                                     labels[11], percents(cleaned_causes_by_year.column(11)))\n",
    "distributions.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Question 1.4.** What is the sum (roughly) of each of the columns (excluding the Year column) in the table above? Why does this make sense? "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": false,
    "manual_problem_id": "causes_of_death_4",
    "scrolled": true
   },
   "source": [
    "*Write your answer here, replacing this text.*"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Question 1.5:** We suspect that the larger percentage of stroke-related deaths over the years 1999-2013 happened in the earlier years, while the larger percentage of deaths related to Chronic Liver Disease over this time period occured in the most recent years. Draw a bar chart to display both of the distributions of these diseases over the time period. \n",
    "\n",
    "*Hint:* The relevant column labels are \"Cerebrovascular Disease (Stroke)\" and \"Chronic Liver Disease and Cirrhosis\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "...\n",
    "\n",
    "# Don't change the code below this comment.\n",
    "plt.title(\"% of total deaths / disease per year\")\n",
    "plt.xlabel(\"% of total deaths\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## (Optional) Unrolling Loops"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**The rest of this homework is optional. Do it for your own practice, but it will not be incorporated into the final grading!**\n",
    "\n",
    "\"Unrolling\" a `for` loop means to manually write out all the code that it executes.  The result is code that does the same thing as the loop, but without the structure of the loop.  For example, for the following loop:\n",
    "\n",
    "    for num in np.arange(3):\n",
    "        print(\"The number is\", num)\n",
    "\n",
    "The unrolled version would look like this:\n",
    "\n",
    "    print(\"The number is\", 0)\n",
    "    print(\"The number is\", 1)\n",
    "    print(\"The number is\", 2)\n",
    "\n",
    "\n",
    "Unrolling a `for` loop is a great way to understand what the loop is doing during each step. In this exercise, you'll practice unrolling `for` loops.\n",
    "\n",
    "\n",
    "In each question below, write code that does the same thing as the given code, but with any `for` loops unrolled.  It's a good idea to run both your answer and the original code to verify that they do the same thing.  (Of course, if the code does something random, you'll get a different random outcome than the original code!)\n",
    "\n",
    "First, run the cell below to load data that will be used in a few questions.  It's a table with 52 rows, one for each type of card in a deck of playing cards.  A playing card has a \"suit\" (\"♠︎\", \"♣︎\", \"♥︎\", or \"♦︎\") and a \"rank\" (2 through 10, J, Q, K, or A).  There are 4 suits and 13 ranks, so there are $4 \\times 13 = 52$ different cards."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "deck = Table.read_table(\"deck.csv\")\n",
    "deck"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": false
   },
   "source": [
    "**Optional Question 1.** Unroll the code below."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# This table will hold the cards in a randomly-drawn hand of\n",
    "# 5 cards.  We simulate cards being drawn as follows: We draw\n",
    "# a card at random from the deck, make a copy of it, put the\n",
    "# copy in our hand, and put the card back in the deck.  That\n",
    "# means we might draw the same card multiple times, which is\n",
    "# different from a normal draw in most card games.\n",
    "hand = Table().with_columns(\"Rank\", make_array(), \"Suit\", make_array())\n",
    "for suit in np.arange(5):\n",
    "    card = deck.row(np.random.randint(deck.num_rows))\n",
    "    hand = hand.with_row(card)\n",
    "hand"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "hand = Table().with_columns(\"Rank\", make_array(), \"Suit\", make_array())\n",
    "..."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": false,
    "editable": false
   },
   "source": [
    "**Optional Question 2.** Unroll the code below."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "for joke_iteration in np.arange(4):\n",
    "    print(\"Knock, knock.\")\n",
    "    print(\"Who's there?\")\n",
    "    print(\"Banana.\")\n",
    "    print(\"Banana who?\")\n",
    "print(\"Knock, knock.\")\n",
    "print(\"Who's there?\")\n",
    "print(\"Orange.\")\n",
    "print(\"Orange who?\")\n",
    "print(\"Orange you glad I didn't say banana?\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "..."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Optional Question 3.** Unroll the code below.\n",
    "\n",
    "*Hint:* `np.random.randint` returns a random integer between 0 (inclusive) and the value that's passed in (exclusive)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# This table will hold the cards in a randomly-drawn hand of\n",
    "# 4 cards.  The cards are drawn as follows: For each of the\n",
    "# 4 suits, we draw a random card of that suit and put it into\n",
    "# our hand.  The cards within a suit are drawn uniformly at\n",
    "# random, meaning each card of the suit has an equal chance of\n",
    "# being drawn.\n",
    "hand_of_4 = Table().with_columns(\"Rank\", make_array(), \"Suit\", make_array())\n",
    "for suit in make_array(\"♠︎\", \"♣︎\", \"♥︎\", \"♦︎\"):\n",
    "    cards_of_suit = deck.where(\"Suit\", are.equal_to(suit))\n",
    "    card = cards_of_suit.row(np.random.randint(cards_of_suit.num_rows))\n",
    "    hand_of_4 = hand_of_4.with_row(card)\n",
    "hand_of_4"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "..."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 3. Submission\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "Once you're finished, submit your assignment as a .ipynb (Jupyter Notebook) and .pdf (download as .html, then print to save as a .pdf) on the class Canvas site."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.6"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}