{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# COMP 364: Advanced Python Structures\n",
    "\n",
    "Today we briefly will covere some intermediate-to-advanced Python concepts.\n",
    "\n",
    "Topics:\n",
    "\n",
    "* generators\n",
    "* decorators\n",
    "\n",
    "\n",
    "\n",
    "Knowledge of these concepts will take your Python to a more advanced level.\n",
    "\n",
    "\n",
    "## Lazy lists: generators\n",
    "\n",
    "Remember: Any data Python computes during runtime is stored as an object in **memory** (RAM).\n",
    "\n",
    "** Not to be confused with long term storage.** \n",
    "\n",
    "Objects take up **space**. Just like books on a bookshelf.\n",
    "\n",
    "In general, we want to keep the memory footprint of our programs to a minimum.\n",
    "\n",
    "Misuse of memory can cause:\n",
    "\n",
    "1) Program and system slowdowns\n",
    "2) Full program crashes (`MemoryError`)\n",
    "\n",
    "In today's world of \"Big Data\", memory usage is a growing concern.\n",
    "\n",
    "### Problem\n",
    "\n",
    "Typical computer memory size is a couple of GB ~4-16 GB of memory.\n",
    "\n",
    "What happens when you need to process a data file that is 50GB large?\n",
    "\n",
    "Generators to the rescue!\n",
    "\n",
    "### Pre-computing vs lazy computing\n",
    "\n",
    "Under the hood we've been dealing with generators all along but thinking of them as lists.\n",
    "\n",
    "A **list** is a pre-computed container object because all its values exist in memory all at once.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 84,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# this is a list of the first 10 numbers \n",
    "\n",
    "nums = [0, 1, 2, 3, 4, 5, 6, 7, 8, 9]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can access each element directly as they are all stored in memory."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 85,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "6"
      ]
     },
     "execution_count": 85,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "nums[6]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "I can also create this list with a loop."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 86,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "def firstn(n):\n",
    "    nums = []\n",
    "    count = 0\n",
    "    while count < n:\n",
    "        nums.append(count)\n",
    "        count += 1\n",
    "    return nums"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 87,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[0, 1, 2, 3, 4, 5, 6, 7, 8, 9]\n"
     ]
    }
   ],
   "source": [
    "nums = firstn(10)\n",
    "print(nums)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "But what if we don't need all the numbers stored all at once?\n",
    "\n",
    "If we know the rule for computing the next number in the sequence we can just not do anything and spit out the next number when asked for it.\n",
    "\n",
    "This is what **generator** functions do.\n",
    "\n",
    "**generator** funcions return **generator** objects and can **yield** values during their execution.\n",
    "\n",
    "In this case, the rule is \"the next number is the previous number + 1\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 88,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "def firstn_gen(n):\n",
    "    count = 0\n",
    "    while count < n:\n",
    "        yield count\n",
    "        count += 1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 89,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "nums = firstn_gen(10)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 90,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<generator object firstn_gen at 0x1798696d0>\n"
     ]
    }
   ],
   "source": [
    "print(nums)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Ok so we got a `generator` object instead of a list.\n",
    "\n",
    "`generator` objects have an attribute called `__next__()` which is a function that returns the next item in the sequence."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 91,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0"
      ]
     },
     "execution_count": 91,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "nums.__next__()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 94,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "3\n"
     ]
    }
   ],
   "source": [
    "i = nums.__next__()\n",
    "print(i)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "If we put a geneator in a for loop, python repeatedly calls `__next__()` for you until it runs out of items."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 95,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0\n",
      "1\n",
      "2\n",
      "3\n",
      "4\n",
      "5\n",
      "6\n",
      "7\n",
      "8\n",
      "9\n"
     ]
    }
   ],
   "source": [
    "nums = firstn_gen(10)\n",
    "for i in nums:\n",
    "    print(i)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Note: once we pulled out all the items in a geneator, we can't re-use it."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 98,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "for i in nums:\n",
    "    print(i)\n",
    "g = firstn_gen(100)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Okay back to the funny `yield` statement."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 99,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "def firstn_gen(n):\n",
    "    count = 0\n",
    "    print(\"before loop\")\n",
    "    while count < n:\n",
    "        print(\"yielding number\")\n",
    "        yield count\n",
    "        print(\"i'm back. incrementing count\")\n",
    "        count += 1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 100,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "nums = firstn_gen(10)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "`nums` is a generator and we can call its `__next__` method."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Generators behave like functions except their execution can be interrupted and re-started.\n",
    "\n",
    "When we reach a `yield` statement, the function exits, yields a number, and all varible assignments are maintained.\n",
    "\n",
    "With a normal function, once a function is exited, all local memory is lost."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "`__next__` executes the function code. \n",
    "\n",
    "If it's the first time `__next__` is called, execution begins at the top of the definition and stops when `yield` is reached.\n",
    "\n",
    "Subsequent calls resume after the `yield`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 101,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "before loop\n",
      "yielding number\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "0"
      ]
     },
     "execution_count": 101,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#first call\n",
    "nums.__next__()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 106,
   "metadata": {},
   "outputs": [
    {
     "ename": "StopIteration",
     "evalue": "",
     "output_type": "error",
     "traceback": [
      "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
      "\u001b[0;31mStopIteration\u001b[0m                             Traceback (most recent call last)",
      "\u001b[0;32m<ipython-input-106-65d4a4f802fb>\u001b[0m in \u001b[0;36m<module>\u001b[0;34m()\u001b[0m\n\u001b[1;32m      1\u001b[0m \u001b[0;31m#second call, resumes after 'yield'\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m----> 2\u001b[0;31m \u001b[0mnums\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m__next__\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m      3\u001b[0m \u001b[0mx\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0;36m5\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m      4\u001b[0m \u001b[0md\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0;34m{\u001b[0m\u001b[0;34m}\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;31mStopIteration\u001b[0m: "
     ]
    }
   ],
   "source": [
    "#second call, resumes after 'yield'\n",
    "nums.__next__()\n",
    "x = 5\n",
    "d = {}"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Because the generator remembers its current state, we can continue with the rest of the numbers with a for loop."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 103,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "i'm back. incrementing count\n",
      "yielding number\n",
      "2\n",
      "i'm back. incrementing count\n",
      "yielding number\n",
      "3\n",
      "i'm back. incrementing count\n",
      "yielding number\n",
      "4\n",
      "i'm back. incrementing count\n",
      "yielding number\n",
      "5\n",
      "i'm back. incrementing count\n",
      "yielding number\n",
      "6\n",
      "i'm back. incrementing count\n",
      "yielding number\n",
      "7\n",
      "i'm back. incrementing count\n",
      "yielding number\n",
      "8\n",
      "i'm back. incrementing count\n",
      "yielding number\n",
      "9\n",
      "i'm back. incrementing count\n"
     ]
    }
   ],
   "source": [
    "for n in nums:\n",
    "    print(n)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Handling large data files with generators\n",
    "\n",
    "Often we want to read from a very large file and do something with each line but we don't need the whole file loaded all at once.\n",
    "\n",
    "I [downloaded](https://www.kaggle.com/stackoverflow/pythonquestions/data) all the questions about Python on Stack Overflow, a website I'm sure you're familiar with by now."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 107,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "filepath = \"Questions.csv\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Here is the wrong way."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 108,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "def load_data_list(path, encoding=\"utf-8\"):\n",
    "    file_handle = open(path, \"r\", encoding=encoding)\n",
    "    file_lines = file_handle.readlines()\n",
    "    file_handle.close()\n",
    "    return file_lines"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 109,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "#this file has a non-standard encoding \"latin-1\" so I have to specify that. don't worry about file encodings.\n",
    "questions = load_data_list(filepath, encoding=\"latin-1\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can use `memory_profiler` (doesn't work in Notebooks) to see how much memory this uses."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 110,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "['Id,OwnerUserId,CreationDate,Score,Title,Body\\n', '469,147,2008-08-02T15:11:16Z,21,How can I find the full path to a font from its display name on a Mac?,\"<p>I am using the Photoshop\\'s javascript API to find the fonts in a given PSD.</p>\\n', '\\n', '<p>Given a font name returned by the API, I want to find the actual physical font file that that font name corresponds to on the disc.</p>\\n', '\\n', \"<p>This is all happening in a python program running on OSX so I guess I'm looking for one of:</p>\\n\", '\\n', '<ul>\\n', '<li>Some Photoshop javascript</li>\\n', '<li>A Python function</li>\\n']\n"
     ]
    }
   ],
   "source": [
    "print(questions[:10])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 111,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "list"
      ]
     },
     "execution_count": 111,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "type(questions)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now let's lazily read the lines from the file."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 112,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "def lazy_read(filepath, encoding=\"utf-8\"):\n",
    "    file_handle = open(filepath, \"r\", encoding=encoding)\n",
    "    while True:\n",
    "        yield file_handle.readline()\n",
    "    file_handle.close()\n",
    "    "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 113,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "g = lazy_read(filepath, encoding=\"latin-1\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 114,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'Id,OwnerUserId,CreationDate,Score,Title,Body\\n'"
      ]
     },
     "execution_count": 114,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "g.__next__()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 115,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'469,147,2008-08-02T15:11:16Z,21,How can I find the full path to a font from its display name on a Mac?,\"<p>I am using the Photoshop\\'s javascript API to find the fonts in a given PSD.</p>\\n'"
      ]
     },
     "execution_count": 115,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "g.__next__()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 116,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'\\n'"
      ]
     },
     "execution_count": 116,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "g.__next__()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 118,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "\n",
      "<p>This is all happening in a python program running on OSX so I guess I'm looking for one of:</p>\n",
      "\n",
      "\n",
      "\n",
      "<ul>\n",
      "\n",
      "<li>Some Photoshop javascript</li>\n",
      "\n",
      "<li>A Python function</li>\n",
      "\n",
      "<li>An OSX API that I can call from python</li>\n",
      "\n",
      "</ul>\n",
      "\n",
      "\"\n",
      "\n",
      "502,147,2008-08-02T17:01:58Z,27,Get a preview JPEG of a PDF on Windows?,\"<p>I have a cross-platform (Python) application which needs to generate a JPEG preview of the first page of a PDF.</p>\n",
      "\n",
      "\n",
      "\n",
      "<p>On the Mac I am spawning <a href=\"\"http://developer.apple.com/documentation/Darwin/Reference/ManPages/man1/sips.1.html\"\">sips</a>.  Is there something similarly simple I can do on Windows?</p>\n",
      "\n"
     ]
    },
    {
     "ename": "KeyboardInterrupt",
     "evalue": "",
     "output_type": "error",
     "traceback": [
      "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
      "\u001b[0;31mKeyboardInterrupt\u001b[0m                         Traceback (most recent call last)",
      "\u001b[0;32m<ipython-input-118-3631c47cbbba>\u001b[0m in \u001b[0;36m<module>\u001b[0;34m()\u001b[0m\n\u001b[1;32m      1\u001b[0m \u001b[0mcount\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0;36m12\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m      2\u001b[0m \u001b[0mc\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0;36m0\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m----> 3\u001b[0;31m \u001b[0;32mfor\u001b[0m \u001b[0mline\u001b[0m \u001b[0;32min\u001b[0m \u001b[0mg\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m      4\u001b[0m     \u001b[0;32mif\u001b[0m \u001b[0mc\u001b[0m \u001b[0;34m<\u001b[0m \u001b[0mcount\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m      5\u001b[0m         \u001b[0mprint\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mline\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;32m<ipython-input-112-158a12bc81d4>\u001b[0m in \u001b[0;36mlazy_read\u001b[0;34m(filepath, encoding)\u001b[0m\n\u001b[1;32m      2\u001b[0m     \u001b[0mfile_handle\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mopen\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mfilepath\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m\"r\"\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mencoding\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mencoding\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m      3\u001b[0m     \u001b[0;32mwhile\u001b[0m \u001b[0;32mTrue\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m----> 4\u001b[0;31m         \u001b[0;32myield\u001b[0m \u001b[0mfile_handle\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mreadline\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m      5\u001b[0m     \u001b[0mfile_handle\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mclose\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m      6\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;31mKeyboardInterrupt\u001b[0m: "
     ]
    }
   ],
   "source": [
    "count = 12\n",
    "c = 0\n",
    "for line in g:\n",
    "    if c < count:\n",
    "        print(line)\n",
    "    c += 1"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This is part of what Python does for you when you open a file using `with open()`."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Generator comprehensions\n",
    "\n",
    "Just like list comprehensions, we can create generator comprehensions.\n",
    "\n",
    "Generator comprehensions look exactly like list comprehensions except they use round brackets ()."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 121,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "evens = (x for x in range(100) if x % 2 == 0)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 122,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0"
      ]
     },
     "execution_count": 122,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "evens.__next__()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 123,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "2"
      ]
     },
     "execution_count": 123,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "next(evens)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 124,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "4"
      ]
     },
     "execution_count": 124,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "next(evens)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Is equivalent to:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 125,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "def evens():\n",
    "    x = 0\n",
    "    while x < 100:\n",
    "        if x % 2 == 0:\n",
    "            yield x\n",
    "        x += 1\n",
    "e = evens()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Generators can create an infinite amount of information while requiring almost zero memory!\n",
    "\n",
    "Example: [fibonacci](https://en.wikipedia.org/wiki/Fibonacci_number) numbers\n",
    "\n",
    "$F_n = F_{n-1} + F_{n-2}$ where $F_0 = 0$ and $F_1 = 1$\n",
    "\n",
    "e.g. $0, 1, 1, 2, 3, 5, 8, 13, 21, ...$\n",
    "\n",
    "![](http://seyferseed.ru/wp-content/uploads/2017/03/Fibonacci-Spiral.png)\n",
    "\n",
    "![](https://qph.ec.quoracdn.net/main-qimg-0281d782e4ec471ce2d5091d2c40f1b5-c)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 126,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "def fib():\n",
    "    a = 0\n",
    "    b = 1\n",
    "    while True:\n",
    "        yield a\n",
    "        a, b = b, a + b"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 127,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0\n",
      "1\n",
      "1\n",
      "2\n",
      "3\n",
      "5\n",
      "8\n",
      "13\n",
      "21\n",
      "34\n"
     ]
    }
   ],
   "source": [
    "f = fib()\n",
    "for i in range(10):\n",
    "    print(next(f))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "So now whenever we want to pull out a fibonacci number, we can just call our generator again."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Iterables as function arguments: the * operator\n",
    "\n",
    "Another cool thing we can do with iterables (such as generators and lists) is pass them as arguments to functions.\n",
    "\n",
    "When we pass an iterable to a function with a * before its name, Python unpacks the items as positional arguments."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 128,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "def polynomial(a, b, c):\n",
    "    return ((a*x**2) + (b * x) + c for x in range(50))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 129,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "coefficients = [2, 2, 3]\n",
    "poly_gen = polynomial(*coefficients)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This is the same as"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 132,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<generator object polynomial.<locals>.<genexpr> at 0x110850e60>"
      ]
     },
     "execution_count": 132,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "poly_gen = polynomial(1, 2, 3)\n",
    "arg_gen = (x for x in range(3))\n",
    "polynomial(*arg_gen)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 131,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "3\n",
      "6\n",
      "11\n",
      "18\n",
      "27\n",
      "38\n",
      "51\n",
      "66\n",
      "83\n",
      "102\n",
      "123\n",
      "146\n",
      "171\n",
      "198\n",
      "227\n",
      "258\n",
      "291\n",
      "326\n",
      "363\n",
      "402\n",
      "443\n",
      "486\n",
      "531\n",
      "578\n",
      "627\n",
      "678\n",
      "731\n",
      "786\n",
      "843\n",
      "902\n",
      "963\n",
      "1026\n",
      "1091\n",
      "1158\n",
      "1227\n",
      "1298\n",
      "1371\n",
      "1446\n",
      "1523\n",
      "1602\n",
      "1683\n",
      "1766\n",
      "1851\n",
      "1938\n",
      "2027\n",
      "2118\n",
      "2211\n",
      "2306\n",
      "2403\n",
      "2502\n"
     ]
    }
   ],
   "source": [
    "for p in poly_gen:\n",
    "    print(p)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Mapping types as keyword arguments, the ** operator\n",
    "\n",
    "If a function takes keyword arguments, we can also unpack them using the ** operator."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "def student(grade=4.0, name='bob'):\n",
    "    print(f\"{name}'s gpa is {grade}\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "s = {'name': 'carlos', 'grade':3.5}"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 133,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "3.5's gpa is carlos\n"
     ]
    }
   ],
   "source": [
    "student(s['name'], s['grade'])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 134,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "carlos's gpa is 3.5\n"
     ]
    }
   ],
   "source": [
    "student(**s)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "## Wrapping functions: Decorators\n",
    "\n",
    "\n",
    "In the spirit of Christmas, let's talk about **wrapping** functions and **decorators**.\n",
    "\n",
    "![](http://mybbaddict.com/wp-content/uploads/2017/10/delightful-decoration-snoopy-christmas-tree-274-best-peanuts-images-on-pinterest-charlie.jpg)\n",
    "\n",
    "First let's do a little recap on functions.\n",
    "\n",
    "**Functions are objects**\n",
    "\n",
    "Just like objects functions can be:\n",
    "\n",
    "* passed as arguments\n",
    "* bound to names\n",
    "* returned\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 136,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "10"
      ]
     },
     "execution_count": 136,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "def foo(x):\n",
    "    return x\n",
    "f = foo\n",
    "b = foo\n",
    "f(5)\n",
    "b(10)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 137,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "calling is_even with input 5 result is: False\n",
      "calling is_odd with input 5 result is: True\n"
     ]
    }
   ],
   "source": [
    "def caller(func, arg):\n",
    "    return f\"calling {func.__name__} with input {arg} result is: {func(arg)}\"\n",
    "\n",
    "def is_even(num):\n",
    "    return not num % 2\n",
    "\n",
    "def is_odd(num):\n",
    "    return bool(num % 2)\n",
    "\n",
    "print(caller(is_even, 5))\n",
    "print(caller(is_odd, 5))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can also define functions inside other functions and return them."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 138,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<class 'function'>\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "5"
      ]
     },
     "execution_count": 138,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "def foo():\n",
    "    x = 5\n",
    "    def boo():\n",
    "        return 5\n",
    "    return boo\n",
    "\n",
    "five = foo()\n",
    "print(type(five))\n",
    "five()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "What if I want to time how long a function takes to run?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 139,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "import time\n",
    "import random\n",
    "\n",
    "#this function runs for 0 to 5 seconds\n",
    "def foo():\n",
    "    print(\"doing some stuff..\")\n",
    "    time.sleep(random.randrange(5))\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 140,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "doing some stuff..\n",
      "time elapsed: 0.004309892654418945 seconds\n"
     ]
    }
   ],
   "source": [
    "start = time.time() #get current time\n",
    "foo()\n",
    "print(f\"time elapsed: {time.time() - start} seconds\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "That's nice but what happens next time I want to time a different function?\n",
    "\n",
    "I have to write the same code again... repetitive code is no good."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 141,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "doing some other stuff\n",
      "time elapsed: 2.004883050918579 seconds\n"
     ]
    }
   ],
   "source": [
    "def boo():\n",
    "    print(\"doing some other stuff\")\n",
    "    time.sleep(random.randrange(3))\n",
    "start = time.time()\n",
    "boo()\n",
    "print(f\"time elapsed: {time.time() - start} seconds\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "What if I want to make it so that I can automatically modify any function with timing functionality?\n",
    "\n",
    "This is where decorators come in.\n",
    "\n",
    "You can think of decorators as functions that create functions but with some useful \"decorations\"."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Since we know that we can return functions, why don't we write a function that takes a function as input and returns a decorated version of it?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 142,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "def timer(func):\n",
    "    def wrapped():\n",
    "        start = time.time()\n",
    "        func()\n",
    "        print(f\"{func.__name__} took: {time.time() - start} seconds\")\n",
    "    #return the decorated function\n",
    "    return wrapped"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 143,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "def stuff():\n",
    "    print(\"doing some stuff..\")\n",
    "    time.sleep(random.randrange(3))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's transform our boring `stuff` function into a decorated version of itself."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 144,
   "metadata": {},
   "outputs": [],
   "source": [
    "timed_stuff = timer(stuff)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 145,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "function"
      ]
     },
     "execution_count": 145,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "type(timed_stuff)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now we can use the improved version of our function."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 146,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "doing some stuff..\n",
      "stuff took: 2.0035247802734375 seconds\n"
     ]
    }
   ],
   "source": [
    "timed_stuff()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "And we can use the decorator to transform any function (**that takes no arguments**) without having to re-write any code."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 147,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "def other_stuff():\n",
    "    print(\"doing some other stuff..\")\n",
    "    time.sleep(random.randrange(3))\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 148,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "doing some other stuff..\n",
      "other_stuff took: 0.0008678436279296875 seconds\n"
     ]
    }
   ],
   "source": [
    "timed_other_stuff = timer(other_stuff)\n",
    "timed_other_stuff()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### The @ operator\n",
    "\n",
    "Python makes using decorators a little easier with the `@` operator.\n",
    "\n",
    "Instead of creating a new function explicitly, we can just put `@decoratorname` before any function we want to decorate."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 149,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "@timer\n",
    "def fibonaccis():\n",
    "    f = fib()\n",
    "    max_fib = 10\n",
    "    for i, num in enumerate(f):\n",
    "        print(num)\n",
    "        if i > max_fib:\n",
    "            break\n",
    "        i += 1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 150,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0\n",
      "1\n",
      "1\n",
      "2\n",
      "3\n",
      "5\n",
      "8\n",
      "13\n",
      "21\n",
      "34\n",
      "55\n",
      "89\n",
      "fibonaccis took: 0.0009970664978027344 seconds\n"
     ]
    }
   ],
   "source": [
    "fibonaccis()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Another good use of decorators is argument type checking. (source: https://www.python-course.eu/python3_decorators.php)\n",
    "\n",
    "Here is one for debugging."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 151,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "calling multiply on arguments (4, 5)\n",
      "calling add on arguments (1, 2, 3, 4)\n"
     ]
    }
   ],
   "source": [
    "def debug(func):\n",
    "    def helper(*args):\n",
    "        print(f\"calling {func.__name__} on arguments {args}\")\n",
    "        func(*args)\n",
    "    return helper\n",
    "\n",
    "@debug\n",
    "def multiply(a, b):\n",
    "    return a * b\n",
    "\n",
    "@debug\n",
    "def add(*args):\n",
    "    return sum(args)\n",
    "multiply(4, 5)\n",
    "add(1, 2, 3, 4)\n"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.2"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}