{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "#Entropy - a partner activity\n", "\n", "For this notebook we are going to use the Titanic data we used in class:\n", "\n", "| ID | name | survived | sex | age group | PcClass | SibSp |\n", "| :--: | :--: | :--: | :--: | :--: | :--: | :--: |\n", "|1 | Braund, Mr. Owen Harris | 0 | male | adult | 3 | Y |\n", "|2 | Cumings, Mrs. John Bradley | 1 | female | adult | 1 | Y |\n", "|8 | Palsson, Master. Gosta Leonard | 0 | male | child | 3 | Y |\n", "|11 | Sandstrom, Miss. Marguerite Rut | 1 | female | child | 1| Y |\n", "| 40 | Nicola-Yarred, Miss. Jamila | 1 | female | teen | 1 | Y |\n", "|50 | Arnold-Franchi, Mrs. Josef | 0 | female | teen | 3 | Y |\n", "|205 | Cohen, Mr. Gurshon \"\"Gus\" | 1 | male | teen |3 | N |\n", "|307 | Allison, Master. Hudson Trevor | 1 | male | child | 1 | Y |\n", "|507| Quick, Mrs. Frederick Charles | 1 | female | adult | 2 | Y |\n", "|529 | Salonen, Mr. Johan Werner | 0 | male | adult | 3 | N |\n", "|581 | Christy, Miss. Julie Rachel | 1 | female | adult | 2 | Y |\n", "| 871 | Balkic, Mr. Cerin | 0 | male | adult | 3 | N |\n", "| 17 | Rice, Master. Eugene | 0 | male | child | 3 | Y |\n", "|16 |Hewlett, Mrs. (Mary D Kingcome)| 1 | female | adult | 2 | N|\n", "|68 | Crease, Mr. Ernest James | 0 | male | teen | 3 | N |\n", "|263| Taussig, Mr. Emil | 0 | male | adult | 1 | N |\n", "|330| Hippach, Miss. Jean Gertrude | 1 | female | teen | 1 | N|\n", "|505 | Maioni, Miss. Roberta | 1 | female | teen| 1| N |\n", "|533 | Elias, Mr. Joseph Jr | 0 | male | teen | 3 | Y |\n", "| 731 | Ilmakangas, Miss. Pieta Sofia | 0 | female | adult | 3 | Y |\n", "\n", "In a bit, we are going to represent this data in Python and compute entropy on it. \n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## But first ... entropy\n", "\n", "In class we did a bunch of work with the entropy formula:\n", "\n", "$$H(X)=-\\sum_{i\\in{X}}p(i)\\log_2(p(i))$$\n", "\n", "One example we computed was that of transmitting a message about the status of Professor Davies house indicating one of 4 conditions: no one home, both Professor Davies and his wife were home together, only Professor Davies and only his wife and we gave the probabilities as follows:\n", "\n", "| condition | prob |\n", "| :--: | :--: |\n", "|no one home | 0.5 |\n", "|both home | 0.25 |\n", "| only Davies## | 0.125 |\n", "| only spouse | 0.125 |\n", "\n", "and we computed the entropy as:\n", "\n", "$$H(X) = - \\frac{1}{2}\\log_2(\\frac{1}{2}) + \\frac{1}{4}\\log_2(\\frac{1}{4}) + \\frac{1}{8}\\log_2(\\frac{1}{8}) + \\frac{1}{8}\\log_2(\\frac{1}{8})$$\n", "\n", "$$ = - \\frac{1}{2}(-1)) + \\frac{1}{4}(-2)) + \\frac{1}{8}(-3) + \\frac{1}{8}(-3) = - ( -\\frac{1}{2} + - \\frac{1}{2} + - \\frac{3}{8} + -\\frac{3}{8}) = 1.75$$\n", "\n", "In data mining/machine learning we often use the notation `info[4, 2, 1, 1]` to represent this entropy formula. The number of numbers in the list `[4, 2, 1, 1]` represent how many conditions there are and each number represents how many occurences are in each condition. So in this case we are interested in 4 conditions (no one home, both home, only D, and only spouse) and in our little sample, 4 times there was no one home, twice both were home and so on. We compute `info[4, 2, 1, 1]` the same way we did entropy above. The denominator of the fractions is the sum of all the numbers in the list, the numerator for each component in the associated number in the list. So:\n", "\n", "\n", "$$info[4, 2, 1, 1] = - \\frac{4}{8}\\log_2(\\frac{4}{8}) + \\frac{2}{8}\\log_2(\\frac{2}{8}) + \\frac{1}{8}\\log_2(\\frac{1}{8}) + \\frac{1}{8}\\log_2(\\frac{1}{8})$$\n", "\n", "$$ = - (\\frac{1}{2}(-1)) + \\frac{1}{4}(-2)) + \\frac{1}{8}\\log_2(-3)) + \\frac{1}{8}(-3))) = - ( -\\frac{1}{2} + - \\frac{1}{2} + - \\frac{3}{8} + -\\frac{3}{8}) = 1.75$$\n", "\n", "and we get the same result.\n", "\n", "Let's say there are 15 women and 5 men in my 110 class and I want to compute the entropy (or info) of the sex variable. That would be represented as:\n", "\n", "$$ info[15, 5]$$\n", "\n", "and we would compute that as:\n", "\n", "$$ info[15, 5] = - (\\frac{15}{20}\\log_2(\\frac{15}{20}) + \\frac{5}{20}\\log_2(\\frac{5}{20}) = - (.75(-0.415) + .25(-2)) = .81125 $$\n", "\n", "### we need an info function\n", "I would like you to write a function, `info`, that takes a list as an argument and returns the entropy of that list.\n", "So for example `info([4, 2, 1, 1])` should return 1.75 and `info([15, 5])` should return about 0.81125.\n", "I wrote a unit test for this info function that you should use.\n", "\n", "#### Checkpoint 1 - 20xp\n" ] }, { "cell_type": "code", "execution_count": 78, "metadata": { "collapsed": false }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "INFO passes all tests\n" ] } ], "source": [ "from math import log\n", "\n", "def info(conditions):\n", " \"\"\"Conditions is a list, for example [4, 3, 2, 1] \n", " and info should return the entropy of that list of conditions\"\"\"\n", " \n", " # TBD\n", " \n", " return 0\n", "\n", "def infoUnitTest():\n", " assert(round(info([4, 2, 1, 1]), 5) == 1.75)\n", " assert(round(info([15, 5]), 5) == 0.81128)\n", " assert(info([15 , 0]) == 0)\n", " assert(info([4, 2, 0, 1, 0, 1]) == 1.75)\n", " print (\"INFO passes all tests\")\n", " \n", "infoUnitTest()" ] }, { "cell_type": "markdown", "metadata": { "collapsed": true }, "source": [ "## Representing the data\n", "Now we need to represent the data in the above table (also available in [this file](https://raw.githubusercontent.com/zacharski/cs419/master/entropyNotebookData.txt) ). In a Python data structure. When you are thinking about this data structure keep in mind that eventually we will want to deal with subsets of the data. For example, \"all instances where sex is male\" or \"all instances where sex is female and class is first class.\" Why don't you do that now while I relax...\n", "\n", "### ok, if you want to read data from a file here is a little start..." ] }, { "cell_type": "code", "execution_count": 2, "metadata": { "collapsed": false }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "['ID', 'name', 'survived', 'sex', 'age group', 'PcClass', 'SibSp']\n" ] } ], "source": [ "from urllib.request import urlopen \n", "def readDataFile(urlFile):\n", " html = urlopen(urlFile)\n", " lines = html.read().decode('UTF-8').split('\\n')\n", " header = lines[0].strip().split('\\t')\n", " print(header)\n", " data = []\n", " # you need to revise, you really don't want that print line\n", " # and you want to return something other than 0\n", " for line in lines[1:]:\n", " print(line)\n", " return 0\n", " \n", " \n", "data = readDataFile('https://raw.githubusercontent.com/zacharski/cs419/master/entropyNotebookData.txt') \n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## InfoVal\n", "This function will change a bit later but let's start one step at a time. I want to write a function `infoVal` that takes three arguments: the dataset, the column, the value from that column we are interested in, and finally the column we are modeling. The column we are modeling means, which column contains the variable distribution we are modeling. In the Titanic case we are modeling the distribution of those who survived and those who didn't. Those variables are in the *survived* column. The function will return the `info` of that data. So if the above dataset is called `data` consider\n", "\n", "```\n", "infoVal(data, 'PcClass', '1', 'survived')\n", "```\n", "\n", "There are 7 instances of first class passengers. 6 of those survived and 1 did not so we want\n", "```\n", "infoVal(data, 'PcClass', '1', 'survived')\n", "```\n", "to return\n", "\n", "``` \n", "info([6, 1])\n", "```\n", "\n", "Please write that code (the following code may help you get started):" ] }, { "cell_type": "code", "execution_count": 80, "metadata": { "collapsed": true }, "outputs": [], "source": [ "from collections import defaultdict\n", "\n", "def infoVal(data, column, val, modelColumn):\n", " mCol = data[0].index(modelColumn)\n", " col = data[0].index(column)\n", " results = defaultdict(int)\n", " # TBD\n", " for row in data[1]:\n", " if row[col] == val:\n", " results[row[mCol]] += 1\n", " print([x for x in results.values()])\n", " return 0\n", "\n", "\n", "\n", " " ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Okay, let's test that function (you might need to alter this to match you data and infoVal method):\n", "\n", "#### Checkpoint 2 - 15xp" ] }, { "cell_type": "code", "execution_count": 81, "metadata": { "collapsed": false }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "all tests passed!\n" ] } ], "source": [ "def unitTestInfoVal():\n", " # the following should return the results of info[6,1]\n", " assert(round(infoVal(data, 'PcClass', '1', 'survived'), 5) == round(info([6, 1]), 5)) \n", " assert(infoVal(data, 'PcClass', '2', 'survived') == 0)\n", " assert(round(infoVal(data, 'sex', 'female', 'PcClass'), 5) == 1.48548)\n", "\n", " print(\"all tests passed!\")\n", "unitTestInfoVal()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Fantastic.\n", "\n", "Finally, I would like you to write a function to compute the entropy for a column. We went over this in class. Suppose we are interested in computing the column entropy for Passenger Class. Based on that column, there are 7 first class passengers, 3 second, and ten third. So we are going to weight each of those values based on those numbers. \n", "\n", "$$columnEntropy_{PcClass} = \\frac{7}{20}infoVal_{PcClass=1} + \\frac{3}{20}infoVal_{PcClass=2} + \\frac{10}{20}infoVal_{PcClass=3}$$\n", "\n", "You should first write a unitTest function. You can collaborate with others when writing this unit test function. Next, with your partner, write the actual function.\n", "\n", "#### Checkpoint 3 - 20xp" ] }, { "cell_type": "code", "execution_count": 83, "metadata": { "collapsed": false }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "column entropy works\n" ] } ], "source": [ "from collections import defaultdict\n", "\n", "# you can use this code to help you get started\n", "# or write your own\n", "\n", "def columnEntropy(data, column, mcol):\n", " col = data[0].index(column)\n", " stats =defaultdict(int)\n", " total = 0\n", " for instance in data[1]:\n", " stats[instance[col]] += 1\n", " total += 1\n", " #return stats\n", " answer = 0\n", " for (key, value) in stats.items():\n", " print (key, value)\n", " # TBD\n", " return answer\n", "\n", "\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Picking a good question\n", "The goal of our project is to come up with a 20-question like approach to classifying instances in our test set. And we are trying to come up with the 'best' first question. In the Titanic example, what should we ask first? A question about sex, age group, Passenger class, or Sibling Spouse? What do you think? And why? " ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": true }, "outputs": [], "source": [] }, { "cell_type": "markdown", "metadata": {}, "source": [ "###Current status\n", "\n", "Ok, So it looks like PcClass is the best first question. Here is where we are at in my rough drawing of this tree:\n", "\n", "![Entropy Tree](http://zacharski.org/files/tmp/tree1.png)\n", "\n", "\n", "\n", "Now we are at the question mark in the tree (where 6 survived and 1 did not). We want to know what is the best next question to ask at that point. So in the case of a first class passenger, is it better to ask about sex, age group, or sib/sp? This is a refinement of what we did above. Right now the entropy is 0.5917. What question can we ask that will reduce that the most? Here's a table just showing those first class passengers:\n", "\n", "\n", "| ID | name | survived | sex | age group | PcClass | SibSp |\n", "| :--: | :--: | :--: | :--: | :--: | :--: | :--: |\n", "|2 | Cumings, Mrs. John Bradley | 1 | female | adult | 1 | Y |\n", "|11 | Sandstrom, Miss. Marguerite Rut | 1 | female | child | 1| Y |\n", "| 40 | Nicola-Yarred, Miss. Jamila | 1 | female | teen | 1 | Y |\n", "|307 | Allison, Master. Hudson Trevor | 1 | male | child | 1 | Y |\n", "|263| Taussig, Mr. Emil | 0 | male | adult | 1 | N |\n", "|330| Hippach, Miss. Jean Gertrude | 1 | female | teen | 1 | N|\n", "|505 | Maioni, Miss. Roberta | 1 | female | teen| 1| N |\n", "\n", "Suppose we ask about sex. That part of the tree looks like:\n", "\n", "\n", "![Entropy Tree](http://zacharski.org/files/tmp/tree2.png)\n", "\n", "So the entropy for the sex question would be\n", "\n", "$$info[Sex] = 5/7(info[5,0]) + 2/7(info[1,1]) = 2 / 7 = 0.2857 $$\n", "\n", "Not bad. \n", "\n", "#### Checkpoint 4: teamwork done by hand - 5xp\n", "\n", "What question should we ask if we know the person was in first class? For this you need to compute (by hand):\n", "\n", "| question | entropy |\n", "| :--: | :--: |\n", "| sex | 0.2857 |\n", "|age group | ? |\n", "| SibSp | ? |\n", "\n", "Fill in the values in the table above.\n", "\n", "We pick the next question based on the one with the lowest entropy. If there is a tie, we randomly pick among those tied.\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### A New Version of the column entropy function\n", "\n", "Let's write code to do what we did by hand. Our previous column entropy function had the following signature:\n", "\n", "`def columnEntropy(data, column, mcol):`\n", "\n", "You can implement this new version as you like, but I went with one described as\n", "\n", "`def columnEntropy(data, column, mcol, constraints):`\n", "\n", "`constraints` is a list of tuples of the form `(column, value)`\n", "\n", "so for the problem we just did by hand, I would do with this function as:\n", "\n", "`columnEntropy(data, 'sex', 'survived', [('PcClass', '1')])`\n", "\n", "That in English would be *Give me the entropy of sex for all first class passengers* \n", "\n", "and the function would return 0.2857\n", "\n", "`columnEntropy(data, 'sex', 'survived', [('PcClass', '1'), ('age group', 'teen')])`\n", "\n", "in English would be *Give me the entropy of sex for all first class teenage passengers* \n", "\n", "#### checkpoint 5 team work - 10xp\n", "Please come up with a unit test procedure for the new column entropy " ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "\n" ] }, { "cell_type": "markdown", "metadata": { "collapsed": true }, "source": [ "####Checkpoint 6 individual work - 15xp\n", "Please write your column entropy function and demo" ] }, { "cell_type": "code", "execution_count": 84, "metadata": { "collapsed": false }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "New Column Entropy working\n" ] } ], "source": [ "from collections import defaultdict\n", "\n", "\n", "\n", "def columnEntropy(data, column, mcol,constraints):\n", " # TBD\n", " return 0\n", "\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Absolutely cool\n", "\n", "Now I am going to yak a bit.\n", "Recall that our columns are \n", "\n", "`['ID', 'name', 'survived', 'sex', 'age group', 'PcClass', 'SibSp']`\n", "\n", "One column, `'survived'` is the variable we are interested in. Some act like comments. For example, we aren't trying to learn something like *If the person's name is Cumings, Mrs. John Bradley then she survived.* That doesn't help us make predictions about new people. So, `'ID'` and `'name'` are in that category. The other columns we are going to use for questions. For some code in the book I would label the columns 'class', 'comment' and 'num', but for this case I am just going to keep a list of the potential question columns. \n", "\n", "`questions = ['sex', 'age group', 'PcClass', 'SibSp']`\n", "\n", "One function that may be helpful is `nextQuestion` that looks like\n", "\n", "`nextQuestion(data, mcol, constraints, potentialQuestions)`\n", "\n", "and example might be\n", "\n", "`nextQuestion(data, 'survived', ([('PcClass', '1'), ['sex', 'age group', 'SibSp'])`\n", "\n", "which I gloss into English as *What should the next question be (sex, age group, or SibSp), if the person was in first class?*\n", "\n", "Before I ask you to implement that let me give you some code that demos priority queues in Python. It creates the queue, puts various items in it (the lower the number the higher the priority), and when I get items from the queue it will return them in order of priority.\n", "Here is that code\n", "\n" ] }, { "cell_type": "code", "execution_count": 35, "metadata": { "collapsed": false }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "(1, 'pull alarm', 'Everyone')\n", "(2, 'call Jim', 'Mary')\n", "(5, 'eat pudding', 'Mr. Ed', \"don't forget no chocolate\")\n", "(8, 'write to researchers in New Mexixo')\n" ] } ], "source": [ "from queue import PriorityQueue\n", "todo = PriorityQueue()\n", "todo.put((2, \"call Jim\", \"Mary\"))\n", "todo.put((5, \"eat pudding\", \"Mr. Ed\", \"don't forget no chocolate\"))\n", "todo.put((1, \"pull alarm\", \"Everyone\"))\n", "todo.put((8, \"write to researchers in New Mexixo\"))\n", "print(todo.get())\n", "print(todo.get())\n", "print(todo.get())\n", "print(todo.get())" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "####Checkpoint 7\n", "That should help you write `nextQuestion`.\n", "Again, your team can work together writing a unit test.\n" ] }, { "cell_type": "code", "execution_count": 87, "metadata": { "collapsed": false }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "next question looks good\n" ] } ], "source": [ "from queue import PriorityQueue\n", "\n", "\n", "\n", "def nextQuestion(data, mcol, constraints, potentialQuestions):\n", " #TBD\n", " potentials = PriorityQueue()\n", " for potential in potentialQuestions:\n", " print(potential)\n", " \n", " \n", " return 0\n", " \n" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": true }, "outputs": [], "source": [] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Okay, let's test this out. Here is what mine returns when I ask it what the first question should be.\n", "\n", "####Checkpoint 8 - does your code return the same? - 15xp" ] }, { "cell_type": "code", "execution_count": 88, "metadata": { "collapsed": false }, "outputs": [ { "data": { "text/plain": [ "(0.44158326929845526, 'PcClass', ['3', '2', '1'])" ] }, "execution_count": 88, "metadata": {}, "output_type": "execute_result" } ], "source": [ "nextQuestion(data, 'survived', [], ['sex', 'age group', 'PcClass', 'SibSp'])" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The third item it returns is a list of values in that question. That will be helpful later.\n", "The value matches what we had before. What question should I ask next when the passenger is first class?" ] }, { "cell_type": "code", "execution_count": 89, "metadata": { "collapsed": false }, "outputs": [ { "data": { "text/plain": [ "(0.2857142857142857, 'age group', ['adult', 'child', 'teen'])" ] }, "execution_count": 89, "metadata": {}, "output_type": "execute_result" } ], "source": [ "nextQuestion(data, 'survived', [('PcClass', '1')], ['sex', 'age group', 'SibSp']) " ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "This also looks good.\n", "\n", "## Note\n", "At this point we are not trying to write the most efficient decision tree algorithm. If you want to do so, I will give you 100xp or more. Right now I am trying to present things step by step and refining stuff as we go. I am hoping to make things easier to understand.\n", "\n", "### When are we done asking questions?\n", "We are done asking questions when one of three things happen.\n", "\n", "#### Case 1: entropy = 0 (really a subset of case 2 below)\n", "in our small dataset, all the second class passengers survivied. So, since our first question is about passenger class, a 20 questions exchange might go\n", "\n", "| user | message|\n", "| :-- | :-- |\n", "| Ann: | What is your passenger class? |\n", "| Clara: | Second |\n", "| Ann: | You survived! |\n", "\n", "So we are done asking questions if the entropy equals zero. \n", "\n", "#### Case 2: entropy does not decrease when we examine other questions.\n", "Consider the case when we ask the following\n", "\n", "\n", "\n", "| user | message|\n", "| :-- | :-- |\n", "| Ann: | What is your passenger class? |\n", "| Clara: | Third |\n", "| Ann: | What age group are you? |\n", "| Clara: | Teen |\n", "\n", "I've changed the data to illutrate this problem. This case now looks like this:\n", "\n", "\n", "| ID | name | survived | sex | age group | PcClass | SibSp |\n", "| :--: | :--: | :--: | :--: | :--: | :--: | :--: |\n", "|50 | Arnold-Franchi, Mrs. Josef | 0 | female | teen | 3 | Y |\n", "|205 | Cohen, Mr. Gurshon \"\"Gus\" | 1 | female | teen |3 | Y |\n", "|68 | Crease, Mr. Ernest James | 0 | female | teen | 3 | Y |\n", "|533 | Elias, Mr. Joseph Jr | 0 | female | teen | 3 | Y |\n", "\n", "\n", "At this point, the entropy (info[3, 1]) on the age group = teen arc is .81 \n", "Examine the above table and see that asking about sex or asking about SibSp will not change the entropy. Discuss in your group and write an answer about why it won't change in this cell **in bold**\n", "\n", "#### Checkpoint 9 - 5xp\n", "\n", "\n", "#### Case 3\n", "We run out of questions to ask. Let's say we asked our last question (maybe it was SibSp), someone answered with *Y* and we are left with four people in the yes group, 3 survived and 1 did not. How should we classify that instance (when someone says Y to SibSp)? Majority rules. We guess *survived*\n", "\n", "\n", "## Top down\n", "We've been developing this code from the bottom up. Now it might be time to switch. Let's say I constructed a decision tree that looks like this:\n", "\n", "![Decision Tree](http://zacharski.org/files/tmp/tree4.png)\n", "\n", "There are several ways we can represent this tree in Python. Let's start with representing the left subtree age group and one possibility for that is:\n", "\n" ] }, { "cell_type": "code", "execution_count": 90, "metadata": { "collapsed": false }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "('age group', {'child': 'survived', 'adult': ('sex', {'male': 'not survived', 'female': 'survived'}), 'teen': 'survived'})\n" ] } ], "source": [ "leftDecisionTree = ('age group', {'child': 'survived', \n", " 'teen': 'survived', \n", " 'adult': ('sex', {'female': 'survived', 'male': 'not survived'}) })\n", "\n", "print(leftDecisionTree)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Great\n", "Now here is a little twenty questions program, that just demos what we've done.def" ] }, { "cell_type": "code", "execution_count": 91, "metadata": { "collapsed": false }, "outputs": [], "source": [ "def twentyQ(decisionTree):\n", " branches = decisionTree[1]\n", " answer = input(\"%s: \" % decisionTree[0])\n", " if answer in branches:\n", " # check if we are at a terminal node\n", " if type(branches[answer]) == str:\n", " print (\"I guess %s\" % branches[answer])\n", " else:\n", " twentyQ(branches[answer])\n", " else:\n", " print (\"I don't know\")\n", " " ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "and let's give it a try (you try too)" ] }, { "cell_type": "code", "execution_count": 92, "metadata": { "collapsed": false }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "age group: teen\n", "I guess survived\n" ] } ], "source": [ "twentyQ(leftDecisionTree)\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "##### checkpoint 10 - team work - 15xp\n", "finish the data structure for the full decision tree above and demo with the `twentyQ` program." ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": true }, "outputs": [], "source": [] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Top up and bottom down\n", "Well, not really. We've done some bottom up work. We wrote a function that can determine the first question to ask and even subsequent questions. And we have just seen what we would want an automatic decision tree training algorithm to build--the decision tree data structure we just played with. Now all we need is some glue, and a few odds and ends to put the stuff we did together to construct a classifier. So, for example,\n", "\n", "`decisionTree = trainDecisionTree(data, mcol, ['sex', 'age group', 'PcClass', 'SibSp'])`\n", "\n", "should return the tree you just build by hand.\n", "\n", "####Checkpoint 11 - 40xp\n", "Can you and your team of code warriors construct trainDecisionTree?" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": true }, "outputs": [], "source": [] } ], "metadata": { "kernelspec": { "display_name": "Python 3", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.4.3" } }, "nbformat": 4, "nbformat_minor": 0 }