{
 "metadata": {
  "name": ""
 },
 "nbformat": 3,
 "nbformat_minor": 0,
 "worksheets": [
  {
   "cells": [
    {
     "cell_type": "heading",
     "level": 1,
     "metadata": {},
     "source": [
      "Understanding the Reaction to Amazon Prime Air (Or: Tapping Twitter's Firehose for Fun and Profit with *pandas*) "
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "On Cyber Monday Eve, Jeff Bezos revealed that Amazon may have intentions to one day deliver many of its goods by unmanned aerial vehicles through a service called *Amazon Prime Air* as part of an segment for the television show *60 Minutes*. This notebook explores ~125k tweets from Twitter's firehose that were captured shortly after the announcement and teaches you how you can be equipped to capture interesting data within moments of announcements for your own analysis.\n",
      "\n",
      "<br />\n",
      "<center><iframe width=\"560\" height=\"315\" src=\"//www.youtube.com/embed/98BIu9dpwHU\" frameborder=\"0\" allowfullscreen></iframe><br /><center>\n",
      "<br />"
     ]
    },
    {
     "cell_type": "heading",
     "level": 2,
     "metadata": {},
     "source": [
      "Aspire"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "Let's seek to better understand the \"Twitter reaction\" to Amazon's announcement that drones may one day be delivering packages right to our doorsteps."
     ]
    },
    {
     "cell_type": "heading",
     "level": 2,
     "metadata": {},
     "source": [
      "Acquire"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "Twitter is an ideal source of data that can help you to understand the reaction to newsworthy events, because it has more than 200M active monthly users who tend to use it to frequently share short informal thoughts about anything and everything. Although Twitter offers a Search API that can be used to query for \"historical data\", tapping into the firehose with the Streaming API is a preferred option because it provides you the ability to acquire much larger volumes of data with keyword filters in real-time.\n",
      "\n",
      "There are numerous options for storing the data that you acquire from the firehose. A document-oriented database such as <a href=\"http://www.mongodb.org/\">MongoDB</a> makes a fine choice and can provide useful APIs for filtering and analysis. However, we'll opt to simply store the tweets that we fetch from the firehose in a newline-delimited text file, because we'll use the *pandas* library to analyze it as opposed to relying on MongoDB or a comparable option. \n",
      "\n",
      "Note: Should you have preferred to instead sink the data to MongoDB, the *mongoexport* commandline tool could have exported it to a newline delimited format that is exactly the same as what we will be writing to a file. Either way, you're covered."
     ]
    },
    {
     "cell_type": "heading",
     "level": 3,
     "metadata": {},
     "source": [
      "Python Dependencies"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "There are only a few third-party packages that are required to use the code in this notebook: \n",
      "\n",
      "* The *twitter* package trivializes the process of tapping into Twitter's Streaming API for easily capturing tweets from the firehose\n",
      "* The *pandas* package provides a highly-performant \"spreadsheet-like interface\" into large collections of records such as tweets\n",
      "* The *nltk* packages provides some handy functions for processing natural language (the \"140 characters\" of content) in the tweets\n",
      "\n",
      "You can easily install these packages in a terminal with *pip install twitter pandas nltk*, or you can install them from within IPython Notebook by using \"Bash magic\". Bash magic is just a way of running Bash commands from within a notebook as shown below where the first line of a cell prefixed with *%%bash*."
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "%%bash\n",
      "\n",
      "pip install twitter pandas nltk"
     ],
     "language": "python",
     "metadata": {},
     "outputs": []
    },
    {
     "cell_type": "heading",
     "level": 3,
     "metadata": {},
     "source": [
      "Tapping Twitter's Firehose"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "It's a lot easier to tap into Twitter's firehose than you might imagine if you're using the right library. The code below show you how to create a connection to <a href=\"https://dev.twitter.com/docs/streaming-apis\">Twitter's Streaming API</a> and filter the firehose for tweets containing keywords. For simplicity, each tweet is saved in a newline-delimited file as a JSON document."
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "import io\n",
      "import json\n",
      "import twitter\n",
      "\n",
      "# XXX: Go to http://twitter.com/apps/new to create an app and get values\n",
      "# for these credentials that you'll need to provide in place of these\n",
      "# empty string values that are defined as placeholders.\n",
      "#\n",
      "# See https://vimeo.com/79220146 for a short video that steps you\n",
      "# through this process\n",
      "#\n",
      "# See https://dev.twitter.com/docs/auth/oauth for more information \n",
      "# on Twitter's OAuth implementation.\n",
      "\n",
      "CONSUMER_KEY = ''\n",
      "CONSUMER_SECRET = ''\n",
      "OAUTH_TOKEN = ''\n",
      "OAUTH_TOKEN_SECRET = ''\n",
      "\n",
      "# The keyword query\n",
      "\n",
      "QUERY = 'Amazon'\n",
      "\n",
      "# The file to write output as newline-delimited JSON documents\n",
      "OUT_FILE = QUERY + \".json\"\n",
      "\n",
      "\n",
      "# Authenticate to Twitter with OAuth\n",
      "\n",
      "auth = twitter.oauth.OAuth(OAUTH_TOKEN, OAUTH_TOKEN_SECRET,\n",
      "                           CONSUMER_KEY, CONSUMER_SECRET)\n",
      "\n",
      "# Create a connection to the Streaming API\n",
      "\n",
      "twitter_stream = twitter.TwitterStream(auth=auth)\n",
      "\n",
      "\n",
      "print 'Filtering the public timeline for \"{0}\"'.format(QUERY)\n",
      "\n",
      "# See https://dev.twitter.com/docs/streaming-apis on keyword parameters\n",
      "\n",
      "stream = twitter_stream.statuses.filter(track=QUERY)\n",
      "\n",
      "# Write one tweet per line as a JSON document. \n",
      "\n",
      "with io.open(OUT_FILE, 'w', encoding='utf-8', buffering=1) as f:\n",
      "    for tweet in stream:\n",
      "        f.write(unicode(u'{0}\\n'.format(json.dumps(tweet, ensure_ascii=False))))\n",
      "        print tweet['text']"
     ],
     "language": "python",
     "metadata": {},
     "outputs": []
    },
    {
     "cell_type": "heading",
     "level": 2,
     "metadata": {},
     "source": [
      "Analyze"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "Assuming that you've amassed a collection of tweets from the firehose in a line-delimited format, one of the easiest ways to load the data into *pandas* for analysis is to build a valid JSON array of the tweets."
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "Note: With *pandas*, you will need to have an amount of working memory proportional to the amount of data that you're analyzing. For reference, it takes on the order of ~8GB of memory to analyze ~125k tweets as shown in this notebook. (Bear in mind that each tweet is roughly 5KB of text when serialized out to a file.)"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "import pandas as pd\n",
      "\n",
      "# A text file with one tweet per line\n",
      "\n",
      "DATA_FILE = \"tmp/Amazon.json\"\n",
      "\n",
      "# Build a JSON array\n",
      "\n",
      "data = \"[{0}]\".format(\",\".join([l for l in open(DATA_FILE).readlines()]))\n",
      "\n",
      "# Create a pandas DataFrame (think: 2-dimensional table) to get a \n",
      "# spreadsheet-like interface into the data\n",
      "\n",
      "df = pd.read_json(data, orient='records')\n",
      "\n",
      "print \"Successfully imported\", len(df), \"tweets\""
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "output_type": "stream",
       "stream": "stdout",
       "text": [
        "Successfully imported 125697 tweets\n"
       ]
      }
     ],
     "prompt_number": 1
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "Whereas you may be used to thinking of data such as a list of dictionaries in a rows-oriented paradigm, pandas DataFrame exposes a convenient columnar view of the data that makes it easy to slice and dice by particular fields in each record. You can print the data frame to display the columnar structure and some stats about each column."
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "# Printing a DataFrame shows how pandas exposes a columnar view of the data\n",
      "\n",
      "print df"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "output_type": "stream",
       "stream": "stdout",
       "text": [
        "<class 'pandas.core.frame.DataFrame'>\n",
        "Int64Index: 125697 entries, 0 to 125696\n",
        "Data columns (total 27 columns):\n",
        "_id                          125697  non-null values\n",
        "contributors                 0  non-null values\n",
        "coordinates                  1102  non-null values\n",
        "created_at                   125681  non-null values\n",
        "entities                     125681  non-null values\n",
        "favorite_count               125681  non-null values\n",
        "favorited                    125681  non-null values\n",
        "filter_level                 125681  non-null values\n",
        "geo                          1102  non-null values\n",
        "id                           125681  non-null values\n",
        "id_str                       125681  non-null values\n",
        "in_reply_to_screen_name      10001  non-null values\n",
        "in_reply_to_status_id        5927  non-null values\n",
        "in_reply_to_status_id_str    5927  non-null values\n",
        "in_reply_to_user_id          10001  non-null values\n",
        "in_reply_to_user_id_str      10001  non-null values\n",
        "lang                         125681  non-null values\n",
        "limit                        16  non-null values\n",
        "place                        1442  non-null values\n",
        "possibly_sensitive           90143  non-null values\n",
        "retweet_count                125681  non-null values\n",
        "retweeted                    125681  non-null values\n",
        "retweeted_status             40297  non-null values\n",
        "source                       125681  non-null values\n",
        "text                         125681  non-null values\n",
        "truncated                    125681  non-null values\n",
        "user                         125681  non-null values\n",
        "dtypes: datetime64[ns](1), float64(13), object(13)\n"
       ]
      }
     ],
     "prompt_number": 2
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "Some of the items in a data frame may be null values, and these null values can wreak all kinds of havoc during analysis. Once you understand why they exist, it's wise to filter them out if possible. The null values in this collection of tweets are caused by \"limit notices\", which Twitter sends to tell you that you're being rate-limited. Notice in the columnar output above that the \"limit\" field (which is not typically part of a tweet) appears 16 times. This indicates that we received 16 limit notices and means that there are effectively 16 \"rows\" in our data frame that has null values for all of the fields we'd have expected to see.\n",
      "\n",
      "Per the Streaming API guidelines, Twitter will only provide up to 1% of the total volume of the firehose, and anything beyond that is filtered out with each \"limit notice\" telling you how many tweets were filtered out. This means that tweets containing \"Amazon\" accounted for at least 1% of the total tweet volume at the time this data was being collected. The next cell shows how to \"pop\" off the column containing the sixteen limit notices and sum up the totals across these limit notices so that we can learn exactly how many tweets were filtered out across the aggregate."
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "# Observe the \"limit\" field that reflects \"limit notices\" where the streaming API\n",
      "# couldn't return more than 1% of the firehose.\n",
      "# See https://dev.twitter.com/docs/streaming-apis/messages#Limit_notices_limit\n",
      "\n",
      "# Capture the limit notices by indexing into the data frame for non-null field\n",
      "# containing \"limit\"\n",
      "\n",
      "limit_notices = df[pd.notnull(df.limit)]\n",
      "\n",
      "# Remove the limit notice column from the DataFrame entirely\n",
      "\n",
      "df = df[pd.notnull(df['id'])]\n",
      "\n",
      "print \"Number of total tweets that were rate-limited\", sum([ln['track'] for ln in limit_notices.limit])\n",
      "print \"Total number of limit notices\", len(limit_notices)"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "output_type": "stream",
       "stream": "stdout",
       "text": [
        "Number of total tweets that were rate-limited 1062\n",
        "Total number of limit notices 16\n"
       ]
      }
     ],
     "prompt_number": 3
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "From this output, we can observe that ~1k tweets were not provided out of ~125k, more than 99% of the tweets about \"Amazon\" were received for the time period that they were being captured. In order to learn more about the bounds of that time period, let's create a time-based index on the *created_at* field of each tweet so that we can perform a time-based analysis."
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "# Create a time-based index on the tweets for time series analysis\n",
      "# on the created_at field of the existing DataFrame.\n",
      "\n",
      "df.set_index('created_at', drop=False, inplace=True)\n",
      "\n",
      "print \"Created date/time index on tweets\""
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "output_type": "stream",
       "stream": "stdout",
       "text": [
        "Created date/time index on tweets\n"
       ]
      }
     ],
     "prompt_number": 4
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "With a time-based index now in place, we can trivially do some useful things like calculate the boundaries, compute histograms, etc. Since tweets through to our filter in roughly the order in which they are created, no additional sorting should be necessary in order to compute the timeframe for this dataset; we can just slice the DataFrame like a list."
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "# Get a sense of the time range for the data\n",
      "\n",
      "print \"First tweet timestamp (UTC)\", df['created_at'][0]\n",
      "print \"Last tweet timestamp (UTC) \", df['created_at'][-1]"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "output_type": "stream",
       "stream": "stdout",
       "text": [
        "First tweet timestamp (UTC) 2013-12-02 01:41:45\n",
        "Last tweet timestamp (UTC)  2013-12-02 05:01:18\n"
       ]
      }
     ],
     "prompt_number": 5
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "Operations such as grouping by a time unit are also easy to accomplish and seem a logical next step. The following cell illustrates how to group by the \"hour\" of our data frame, which is exposed as a datetime.datetime timestamp since we now have a time-based index in place."
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "# Let's group the tweets by hour and look at the overall volumes with a simple\n",
      "# text-based histogram\n",
      "\n",
      "# First group by the hour\n",
      "\n",
      "grouped = df.groupby(lambda x: x.hour)\n",
      "\n",
      "print \"Number of relevant tweets by the hour (UTC)\"\n",
      "print\n",
      "\n",
      "# You can iterate over the groups and print \n",
      "# out the volume of tweets for each hour \n",
      "# along with a simple text-based histogram\n",
      "\n",
      "for hour, group in grouped:\n",
      "    print hour, len(group), '*'*(len(group) / 1000)"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "output_type": "stream",
       "stream": "stdout",
       "text": [
        "Number of relevant tweets by the hour (UTC)\n",
        "\n",
        "1"
       ]
      },
      {
       "output_type": "stream",
       "stream": "stdout",
       "text": [
        " 14788 **************\n",
        "2 43286 *******************************************\n",
        "3 36582 ************************************\n",
        "4 30008 ******************************\n",
        "5 1017 *\n"
       ]
      }
     ],
     "prompt_number": 6
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "Bearing in mind that we just previously learned that tweet acquisition began at 1:41 UTC and ended at 5:01 UTC, it could be helpful to further subdivide the time ranges into smaller intervals so as to increase the resolution of the extremes. Therefore, let's group into a custom interval by dividing the hour into 15-minute segments. The code is pretty much the same as before except that you call a custom function to perform the grouping; pandas takes care of the rest."
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "# Let's group the tweets by (hour, minute) and look at the overall volumes with a simple\n",
      "# text-based histogram\n",
      "\n",
      "def group_by_15_min_intervals(x):\n",
      "    if   0 <= x.minute <= 15: return (x.hour, \"0-15\")\n",
      "    elif 15 < x.minute <= 30: return (x.hour, \"16-30\")\n",
      "    elif 30 < x.minute <= 45: return (x.hour, \"31-45\")\n",
      "    else: return (x.hour, \"46-00\")\n",
      "\n",
      "\n",
      "grouped = df.groupby(lambda x: group_by_15_min_intervals(x))\n",
      "\n",
      "print \"Number of relevant tweets by intervals (UTC)\"\n",
      "print\n",
      "\n",
      "for interval, group in grouped:\n",
      "    print interval, len(group), \"\\t\", '*'*(len(group) / 200)\n",
      "\n",
      "# Since we didn't start or end precisely on an interval, let's\n",
      "# slice off the extremes. This has the added benefit of also\n",
      "# improving the resolution of the plot that shows the trend\n",
      "plt.plot([len(group) for hour, group in grouped][1:-1])\n",
      "plt.ylabel(\"Tweet Volume\")\n",
      "plt.xlabel(\"Time\")"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "output_type": "stream",
       "stream": "stdout",
       "text": [
        "Number of relevant tweets by intervals (UTC)\n",
        "\n",
        "(1, '31-45')"
       ]
      },
      {
       "output_type": "stream",
       "stream": "stdout",
       "text": [
        " 2875 \t**************\n",
        "(1, '46-00') 11913 \t***********************************************************\n",
        "(2, '0-15') 13611 \t********************************************************************\n",
        "(2, '16-30') 11265 \t********************************************************\n",
        "(2, '31-45') 10452 \t****************************************************\n",
        "(2, '46-00') 7958 \t***************************************\n",
        "(3, '0-15') 10386 \t***************************************************\n",
        "(3, '16-30') 9542 \t***********************************************\n",
        "(3, '31-45') 8727 \t*******************************************\n",
        "(3, '46-00') 7927 \t***************************************\n",
        "(4, '0-15') 9042 \t*********************************************\n",
        "(4, '16-30') 7543 \t*************************************\n",
        "(4, '31-45') 7074 \t***********************************\n",
        "(4, '46-00') 6349 \t*******************************\n",
        "(5, '0-15') 1017 \t*****\n"
       ]
      },
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 7,
       "text": [
        "<matplotlib.text.Text at 0x1e9a9d50>"
       ]
      },
      {
       "metadata": {},
       "output_type": "display_data",
       "png": "iVBORw0KGgoAAAANSUhEUgAAAZEAAAEKCAYAAADTgGjXAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3XtcVHX6wPEPyphtKl5SyBkLk0FEEU1FctOlCC+U6KZp\nmAHaxdXN1Wpba/dXye4qmLtt1opbu5hkJpqZWCmZbaiV0nrJTLtM7SgwIJmkUV4Q/P7++OKsN+Ti\nzJxheN6vl6+Fw5xznsPSPPO9Pn5KKYUQQgjRAM2MDkAIIUTjJUlECCFEg0kSEUII0WCSRIQQQjSY\nJBEhhBANJklECCFEg7k8iUyePJnAwEAiIiIu+Nlf//pXmjVrRllZmfNYWloaVquVsLAwNmzY4Dy+\nY8cOIiIisFqtzJgxw3n85MmTjB8/HqvVSnR0NAcOHHD1IwghhKgjlyeRSZMmkZube8HxwsJC3n33\nXa677jrnsX379rFixQr27dtHbm4u06ZN48yylalTp5KZmYnNZsNmszmvmZmZSYcOHbDZbDz00EPM\nmjXL1Y8ghBCijlyeRAYPHky7du0uOP7www/z9NNPn3MsJyeHxMRETCYTwcHBhISEkJ+fT0lJCeXl\n5URFRQGQlJTEmjVrAFi7di3JyckAjBkzhvfee8/VjyCEEKKOPDImkpOTg8VioXfv3uccLy4uxmKx\nOL+3WCw4HI4LjpvNZhwOBwAOh4MuXboA4O/vT0BAwDndY0IIITzH3903OHbsGHPnzuXdd991HvPE\nTit+fn5uv4cQQvii+rxHu70l8s0337B//34iIyPp2rUrRUVF9OvXj9LSUsxmM4WFhc7XFhUVYbFY\nMJvNFBUVXXAcdKukoKAAgMrKSo4ePUr79u0vem+llM/+e+qppwyPQZ5Pnq+pPVtTeL76cnsSiYiI\noLS0FLvdjt1ux2KxsHPnTgIDA0lISCA7O5uKigrsdjs2m42oqCiCgoJo06YN+fn5KKVYunQpo0aN\nAiAhIYGsrCwAVq1aRWxsrLsfQQghRA1c3p2VmJjIpk2bOHz4MF26dOGPf/wjkyZNcv787G6m8PBw\nxo0bR3h4OP7+/mRkZDh/npGRQUpKCsePHyc+Pp7hw4cDcO+993LPPfdgtVrp0KED2dnZrn4EIYQQ\ndeSnGtJ+aQT8/Pwa1DRrLPLy8oiJiTE6DLeR52u8fPnZwPefr77vnZJEhBBCONX3vVO2PRFCCNFg\nkkSEEEI0mCQRIYQQDSZJxEVk+EUI0RRJEnGBP/0JUlKMjkIIITxPZmddpv/+F6Ki4PRp+PRTOGvL\nLyGEaHRkim81TyWRUaMgOhqKiyEgAP78Z7ffUggh3EaSSDVPJJHcXJg+HT77DPbvhyFD4MABaNnS\nrbcVQgi3kXUiHlJRATNmwLPPwhVXQPfu0KcPrFxpdGRCCOE5kkQaaMECCAmB227737Hp0+H552Wm\nlhCi6ZDurAYoLobevWHrVrBa/3e8qgpCQ2HZMj1OIoQQjY10Z3nArFlw//3nJhCA5s3h17/WrREh\nhGgKpCVSTx9+COPHwxdfQKtWF/78yBG4/nrYtw+Cglx+eyGEcCtpibhRVZUe93j66YsnEIC2bXWS\neeEFz8YmhBBGkJZIPbzwgh7v2LQJLlXCfe9euPVWPd23RQuXhiCEEG4lLRE3KSuDJ5/U4x2XSiAA\nPXtCeDisWuWZ2IQQwiiSROroySdh7FiIjKzb689M9xVCCF8m3Vl1sHs3DB2qB8s7dKjbOVVV0K2b\nbo307++SMIQQwu0M786aPHkygYGBREREOI898cQTREZG0qdPH2JjYyksLHT+LC0tDavVSlhYGBs2\nbHAe37FjBxEREVitVmbMmOE8fvLkScaPH4/VaiU6OpoDBw64+hHOoZRuVaSm1j2BgJ7uO22atEaE\nED5OudjmzZvVzp07Va9evZzHfvjhB+fXzz33nLr33nuVUkrt3btXRUZGqoqKCmW321W3bt3U6dOn\nlVJKDRgwQOXn5yullBoxYoRav369UkqphQsXqqlTpyqllMrOzlbjx4+/aByuerRXX1WqTx+lKivr\nf+533ynVtq1SpaUuCUUIIdyuvu+dLm+JDB48mHbt2p1zrHXr1s6vf/zxR66++moAcnJySExMxGQy\nERwcTEhICPn5+ZSUlFBeXk5UVBQASUlJrFmzBoC1a9eSnJwMwJgxY3jvvfdc/QhnxQq/+51uTTRv\nXv/zO3SAMWPgn/90fWxCCOEN/D11oz/84Q8sXbqUK6+8ko8//hiA4uJios/aH8RiseBwODCZTFjO\nKsxhNptxOBwAOBwOunTpooP39ycgIICysjLat2/v8pjnzoVf/AJuuqnh15g+Xe+v9bvfgcnkutiE\nEMIbeCyJzJkzhzlz5pCens7MmTN56aWX3H7P2bNnO7+OiYkhJiamzud+/TW8+KIuNHU5IiP1APsb\nb8C4cZd3LSGEcLW8vDzy8vIafL7HksgZEyZMID4+HtAtjLMH2YuKirBYLJjNZoqKii44fuacgoIC\nOnfuTGVlJUePHq2xFXJ2EqmvmTN166Fz5wZfwmn6dL3rryQRIYS3Of8Ddmpqar3O98g6EZvN5vw6\nJyeHvn37ApCQkEB2djYVFRXY7XZsNhtRUVEEBQXRpk0b8vPzUUqxdOlSRo0a5TwnKysLgFWrVhEb\nG+vyeN9+G776StcLcYXRo3XRqk8+cc31hBDCW7i8JZKYmMimTZv47rvv6NKlC6mpqaxbt44vv/yS\n5s2b061bNxYtWgRAeHg448aNIzw8HH9/fzIyMvCrXg6ekZFBSkoKx48fJz4+nuHDhwNw7733cs89\n92C1WunQoQPZ2dkujf/kSd0Kee45XWzKFfz9YepUPUCfmemaawohhDeQxYbnSU+Hjz6CtWtdG8+h\nQ7rWyNdf12+9iRBCeJLUWK/WkCTicOhiUx9/rAfDXS0lBXr00PVIhBDCG0kSqdaQJDJhAnTtCnPm\nuCemHTvgjjvgm290F5cQQngbw7c9aay2bIEPPoDf/9599+jXDywWePNN991DCCE8SZII/ys2NX8+\nXHWVe+81fboetBdCCF8gSQRdbKptW8+s4xgzRk8f3rPH/fcSQgh3a/JjIt99pwtIbdyoB9U94Y9/\n1IP4UkJXCOFtZGC9Wl1/EVOn6kFuT27ZfvCgnqX13//CeXtVCiGEoSSJVKvLL2LXLhgxAj7/3PNv\n5hMnQt++8Mgjnr2vEEJciiSRarX9IpSCwYMhORnuv9+DgVXLz4fERLDZGrbNvBBCuINM8a2jZcvg\n+HGYPNmY+w8cCFdfDevWGXN/IYRwhSbZEikvh7AwXf/8xhs9HNhZXnkFXn4ZzqoKLIQQhpLurGqX\n+kXMmqUHt6s3AzbMyZNw3XXw/vt6oF0IIYwmSaRaTb+IL7+En/9cr9O45hoDAjvPk0/C4cOwcKHR\nkQghhCQRp4v9IpSC+HiIjYXf/tagwM5TXAy9eoHdDgEBRkcjhGjqZGD9Et58U79Z/+Y3RkfyP507\nw9ChsGSJ0ZEIIUT9NZmWyIkT0LMnLFqk37S9yYcf6m3iv/wSmjWptC6E8DbSEqnBX/+qtzXxtgQC\nMGgQtG4N77xjdCRCCFE/TaIlUlgIffrAf/4D119vcGA1eOkleO01WTcihDCWDKxXO/sXMX48dO+u\nNz70VidOwLXX6q4tq9XoaIQQTZXh3VmTJ08mMDCQiIgI57FHH32UHj16EBkZyR133MHRo0edP0tL\nS8NqtRIWFsaGs1bd7dixg4iICKxWKzNmzHAeP3nyJOPHj8dqtRIdHc2BAwcuGU9eHmzbBo895rpn\ndIeWLeG++2SqrxCicXF5Epk0aRK5ubnnHBs6dCh79+5l9+7dhIaGkpaWBsC+fftYsWIF+/btIzc3\nl2nTpjkz4NSpU8nMzMRms2Gz2ZzXzMzMpEOHDthsNh566CFmXaJgeWWlnon117/Cz37m6id1valT\n9Qr28nKjIxFCiLpxeRIZPHgw7c7bEjcuLo5m1dOOBg4cSFFREQA5OTkkJiZiMpkIDg4mJCSE/Px8\nSkpKKC8vJyoqCoCkpCTWrFkDwNq1a0lOTgZgzJgxvPfeezXGsmiR3p9qzBhXP6V7dOkCt9yiE4kQ\nQjQG/p6+4eLFi0lMTASguLiY6Oho588sFgsOhwOTyYTFYnEeN5vNOBwOABwOB126dAHA39+fgIAA\nysrKaN++/QX3euyx2aSkQGoqxMTEEBMT474Hc5Hp0+FXv4Jp08DPz+hohBC+Li8vj7y8vAaf79Ek\nMmfOHFq0aMGECRM8cr/775/Ns8965FYuM2QImEy60mJcnNHRCCF83fkfsFNTU+t1vsfWiSxZsoR1\n69axbNky5zGz2UxhYaHz+6KiIiwWC2az2dnldfbxM+cUFBQAUFlZydGjRy/aCgGYPdsND+Jmfn66\nNeLJSotCCNFQHkkiubm5zJ8/n5ycHFq2bOk8npCQQHZ2NhUVFdjtdmw2G1FRUQQFBdGmTRvy8/NR\nSrF06VJGjRrlPCerevvdVatWERsbW+N927Z173O5y913w0cf6fK5QgjhzVy+TiQxMZFNmzbx3Xff\nERgYSGpqKmlpaVRUVDhbDDfeeCMZGRkAzJ07l8WLF+Pv78+CBQsYNmwYoKf4pqSkcPz4ceLj43nu\nuecAPcX3nnvuYdeuXXTo0IHs7GyCg4MvfLB6znX2No8+qjeM/MtfjI5ECNGUyGLDao09iezfD/37\nw4EDcNVVRkcjhGgqDF9sKFwjOBhuuklXPxRCCG8lScSLnRlgb8QNKiGEj5Mk4sVuuUUnkMuYwi2E\nEG4lScSL+fnBgw/KdF8hhPeSgXUv9+OPcN11sHOn/l8hhHAnGVj3Ma1aQVKS3gdMCCG8jbREGoGv\nv9bVDw8cgCuvNDoaIYQvk5aIDwoJgagoWL7c6EiEEOJckkQaienT4bnnZLqvEMK7SBJpJOLi4Phx\n+OADoyMRQoj/kSTSSDRrJtN9hRDeRwbWG5EfftDboXz6KZxVs0sIIVxGBtZ9WJs2epv4f/zD6EiE\nEEKTlkgj8+WXuvrhgQNwVmkWIYRwCWmJ+Lju3aFPH3jhBaMjEUIID9dYF67xzDNw221w6BD88Y96\n0F0IIYwg3VmN1KFDMHYsBATomiNt2hgdkRDCF7ilO2v//v1s3LgRgGPHjvHDDz80LDrhMh07wsaN\n0KULREeDzWZ0REKIpqjWJPLiiy9y5513MmXKFACKior45S9/6fbARO1MJli4EGbO1FUQ33nH6IiE\nEE1NrUlk4cKFfPDBB7Sp7i8JDQ3l22+/rfH1kydPJjAwkIiICOex1157jZ49e9K8eXN27tx5zuvT\n0tKwWq2EhYWxYcMG5/EdO3YQERGB1WplxowZzuMnT55k/PjxWK1WoqOjOXDgQN2f1kc98AC8/jpM\nmgR/+YtsjSKE8Jxak8gVV1zBFVdc4fy+srISPz+/Gl8/adIkcnNzzzkWERHBG2+8wZAhQ845vm/f\nPlasWMG+ffvIzc1l2rRpzr64qVOnkpmZic1mw2azOa+ZmZlJhw4dsNlsPPTQQ8yaNavuT+vDbroJ\n8vP1Jo1JSXqLFCGEcLdak8gvfvEL5syZw7Fjx3j33Xe58847GTlyZI2vHzx4MO3atTvnWFhYGKGh\noRe8Nicnh8TEREwmE8HBwYSEhJCfn09JSQnl5eVERUUBkJSUxJo1awBYu3YtycnJAIwZM4b33nuv\n7k/r47p0gS1boKpKryUpKjI6IiGEr6t1im96ejqZmZlERETwwgsvEB8fz3333eeSmxcXFxMdHe38\n3mKx4HA4MJlMWM7a18NsNuNwOABwOBx06dJFB+/vT0BAAGVlZbRv3/6C68+ePdv5dUxMDDExMS6J\n25v97GewbBnMnw8DB8Jrr+laJEIIcTF5eXnk5eU1+Pxak0jz5s154IEHeOCBBxp8E6OcnUSaEj8/\n+N3voFcvGD0a0tLg3nuNjkoI4Y3O/4Cdmppar/Nr7c5688036du3L+3ataN169a0bt3aOch+ucxm\nM4WFhc7vi4qKsFgsmM1mis7qizlz/Mw5BQUFgB6fOXr06EVbIQLi43X31vz5uh7JqVNGRySE8DW1\nJpGZM2eSlZXF4cOHKS8vp7y8/LLWiZy9iCUhIYHs7GwqKiqw2+3YbDaioqIICgqiTZs25Ofno5Ri\n6dKljBo1ynlOVlYWAKtWrSI2NrbBsTQF3bvrAff//heGDoXvvjM6IiGET1G1GDJkiKqsrKztZU53\n3XWXuuaaa5TJZFIWi0VlZmaqN954Q1ksFtWyZUsVGBiohg8f7nz9nDlzVLdu3VT37t1Vbm6u8/j2\n7dtVr169VLdu3dT06dOdx0+cOKHuvPNOFRISogYOHKjsdvtF46jDozUplZVKPf64UsHBSn3yidHR\nCCG8VX3fO2vd9mTbtm08+eST3HzzzbRo0QLQy+IffvhhD6S4hvP1bU8aKjtbd20tWqS3TRFCiLPV\n972z1oH1J554gtatW3PixAkqKiouKzhhvLvugtBQ+OUvYfduSE2VDRyFEA1Xa0ukV69efPbZZ56K\nx2WkJXJp336rWyLt2sHSpbKBoxBCc/kGjPHx8bwjmzL5nE6d9AaO11wDN94IX39tdERCiMao1pZI\nq1atOHbsGC1atMBkMumT/Py8fidfaYnU3T/+AU89pVskQ4caHY0Qwkj1fe+UeiICgM2bYfx4vUhx\n5ky9YFEI0fS4PIls3rz5osfP30zR20gSqb8DB/QK9969dfldqeEuRNPj8iRy++23O3ftPXHiBB9/\n/DH9+vXj3//+9+VF6maSRBrm2DGYPFkvTnzjDTCbjY5ICOFJbu/OKiwsZMaMGaxevbrewXmSJJGG\nUwrmzYPnn4dVq/TAuxCiaXBLedyzWSwWPv/88/qeJhoRPz947DF48UUYNQpeesnoiIQQ3qrWxYbT\np093fn369Gk++eQT+vXr59aghHe47TY94J6QAM2b62JXQghxtlq7s5YsWeL82t/fn+DgYG666SZ3\nx3XZpDvLdd55B37/e9ixw+hI3KOqSidJIYRM8XWSJOI6p09DSAisXAn9+xsdjWudPg09e+pt859+\nWpKJEC7bOysiIuKSN/n000/rF5lotJo1g/vv19N+fS2JvPMOtGgBu3bp6c2vvgqtWxsdlRCNR40t\nkf3791/yxODgYDeE4zrSEnGtgwehRw+9lsSX9tlKSNCTB5KSYNo0+PhjePNNuPZaoyMTwhhu6c4q\nLS3l448/xs/Pj6ioKDp16nRZQXqCJBHXGzsWbr0VfvUroyNxjQMH4IYboKAArrpKT21+5hn97403\nICrK6AiF8DyXT/FduXIlUVFRvPbaa+d8LZqeKVN0l5av5OYXX4SJE3UCAT21+ZFHdK2V22/XY0BC\niEurtSXSu3dvNm7c6Gx9HDp0iNjYWK8fE5GWiOudPg1Wqy5sNWCA0dFcnooK3WWVlwdhYRf+fPdu\n3dV1333wf/8ne4mJpsPlLRGlFB07dnR+36FDB3lzbqLOHmBv7FavhvDwiycQgMhI2LZNj4/ccw+c\nOOHZ+IRoLGpNIsOHD2fYsGEsWbKEl156ifj4eEaMGOGJ2IQXSkmB11+Ho0eNjuTyLFqkB9Iv5Zpr\ndEulogJiY3UhLyHEuWpMIitXruTEiRPMnz+fKVOmsHv3bvbs2cOUKVN4+umna7zg5MmTCQwMPGeK\ncFlZGXFxcYSGhjJ06FCOHDni/FlaWhpWq5WwsDA2bNjgPL5jxw4iIiKwWq3MmDHDefzkyZOMHz8e\nq9VKdHQ0Bw4caPDDi/oLCtKD66++anQkDffZZ2Cz6VlZtfnZz3T33S23QHQ07N3r/viEaFRUDUaN\nGqWuvvpqNXHiRPX222+rU6dO1fTSc2zevFnt3LlT9erVy3ns0UcfVfPmzVNKKZWenq5mzZqllFJq\n7969KjIyUlVUVCi73a66deumTp8+rZRSasCAASo/P18ppdSIESPU+vXrlVJKLVy4UE2dOlUppVR2\ndrYaP378ReO4xKOJy/Tuu0pFRipV/X9Vo/PrXyv15JP1P2/pUqU6dlSq+k9RCJ9U3/fOS776yJEj\n6qWXXlLDhg1TgYGBasqUKSovL6/Wi9rt9nOSSPfu3dXBgweVUkqVlJSo7t27K6WUmjt3rkpPT3e+\nbtiwYWrr1q2quLhYhYWFOY8vX75cTZkyxfmabdu2KaWUOnXqlLr66qsv/mCSRNymqkqp669Xqvr/\nhkalvFypdu2UKixs2PkffKBUUJBSzz/v2riE8Bb1fe+85AaMAQEBpKSkkJKSwnfffcfrr7/O9OnT\nKSsro6ioqM6tndLSUgIDAwEIDAyktLQUgOLiYqKjo52vs1gsOBwOTCYTFovFedxsNuNwOABwOBx0\n6dIF0Ht5BQQEUFZWRvv27S+47+zZs51fx8TEEBMTU+eYRc2aNYMHHtBTZAcONDqa+lm2DGJi4Kw/\nr3r5+c/hww9h5Ej44gt49lnwr3UbUyG8V15eHnl5eQ0+v05//t9//z2rV69mxYoVlJWVceeddzb4\nhn5+fs4iV+52dhIRrjVpEnTvrhfmBQQYHU3dKAUZGfCXv1zeda6/Hj76SJcTvv12WLGi8fwOhDjf\n+R+wU1NT63V+jQPr5eXlvPzyy8THx9OjRw+2b9/OE088QWFhIX/729/qdZPAwEAOHjwIQElJiXPN\nidlsprCw0Pm6oqIiLBYLZrP5nJbOmeNnzikoKACgsrKSo0ePXrQVItyrUyeIi9Of7BuLrVt15cbY\n2Mu/VkAAvPWW3phy0CBdCVKIpqjGJNK1a1feeecdpk2bRkFBAS+++CI333xzg1oRCQkJZGVlAZCV\nlcXo0aOdx7Ozs6moqMBut2Oz2YiKiiIoKIg2bdqQn5+PUoqlS5cyqnoqzdnXWrVqFbGueEcQDdLY\nVrBnZMDUqbo7zhX8/eHvf9fXPNPNJUSTU9NgyU8//dSgQZm77rpLXXPNNcpkMimLxaIWL16sDh8+\nrGJjY5XValVxcXHq+++/d75+zpw5qlu3bqp79+4qNzfXeXz79u2qV69eqlu3bmr69OnO4ydOnFB3\n3nmnCgkJUQMHDlR2u/2icVzi0YSLVFUpFRKi1NatRkdSu2+/VSogQKnDh91z/fXr9cytpUvdc30h\nPKW+751ST0Rclqef1gPMixcbHcmlPf00fP65e0v97t2rx0juvhv++EfXtXiE8CQpSlVNkohnfPut\nHmC326FtW6OjubgzRbWys92/M++338IvfwlmMyxZohcrCtGYuHzvrIvt2Cu7+IozOnWCYcPglVeM\njqRm77wD7dt7ZtPITp3gvffgiiv0VOKSEvffUwgj1ZpE5s6dW6djouk6s2bEWxt+ZwbUPbUTb8uW\n8PLLehfg6Gj45BPP3FcII9S4TmT9+vWsW7cOh8PBb37zG2fzpry8HJPJ5LEAhfe7+WY4flzvenvj\njUZHc679+/WajhUrPHtfPz+9hXxoqJ4KnZmpk4oQvqbGJNK5c2f69etHTk4O/fr1QymFn58frVu3\nrvc6EeHb/Px0a+SFF7wvibz4ot7K3aixiXHjIDhYj5PYbPDww1KbRPiWWgfWT506xalTpygoKCCs\npuILXkgG1j3r0CFdsMpuh3btjI5GO3kSrruu5sJTnlRQoLdKGTgQFi4EacwLb+XygfX169fTt29f\nhg8fDsCuXbtIkHa5OE/HjjBihHcNsK9eDT17Gp9AQFdR/OADPdA+bBgcPmx0REK4Rq1JZPbs2eTn\n59Ou+uNl3759+a/s8SAu4kyXlrc0ABct0gPq3qJ1a1izBvr31y0SqU0ifEGtScRkMtH2vAUAzWQV\nlbiImBg4dUrvUWW0zz6Db76pW+EpT2reXC98nD1bT0jIyTE6IiEuT63ZoGfPnixbtozKykpsNhvT\np09n0KBBnohNNDJnD7AbbdEiuO8+7x17mDgR3n4bHnwQ/vQn72m9CVFftQ6s//TTT8yZM8dZunbY\nsGE88cQTtGzZ0iMBNpQMrBvju+/06nAjB9jLy/WA+qefNrxuiKeUlMAdd+g4X3oJWrUyOiLR1Llt\n25OffvqJq666qsGBeZokEeNMmKAX2f3mN8bc/x//gA0b9MB6Y3DypB672bFDd28FBxsdkWjKXD47\n66OPPiI8PNw5vXf37t1Mmzat4REKn2fkFvFKed+Aem2uuEIvRpw8WSffyygyJ4TH1ZpEZs6cSW5u\nLldffTUAkZGRbNq0ye2BicZryBCoqtIrxT3to4/06vnGVmbGzw9mzNBTpMeP11u1SENaNAZ1mmZ1\n7bXXnvO9vxSVFpdg5AD7okXwq1813m3Yb71VJ8KMDP0cFRVGRyTEpdX6n9q1117Lh9Ul2yoqKvjL\nX/5Cjx493B6YaNySk2HtWigr89w9Dx3SJWtTUjx3T3fo1k1Pky4t1S2qb781OiIhalZrElm0aBEL\nFy7E4XBgNpvZtWsXCxcu9ERsohHr0AFuuw2WLvXcPRcv1ntUtW/vuXu6S+vWemLALbfoLex37TI6\nIiEurtbZWSdOnPD66bwXI7OzjLd5s+6S2bvX/ZsOVlXpvbtWrPBM3RBPWrUKpk2D55/X4yVCuFN9\n3ztrHdzo2bMngYGBDBkyhMGDB3PTTTcREBBwWUGKpmHwYD04/MEH+mt38mThKU8bO1YnyNGjYfdu\n+POfG++Yj/A9tf4pfvPNNyxfvpyIiAjeeustevfuTZ8+fRp0swULFhAREUGvXr1YsGABAGVlZcTF\nxREaGsrQoUM5cuSI8/VpaWlYrVbCwsKcix0BduzYQUREBFarlRkzZjQoFuF+ZwbYX3zR/fdatEh/\nWvdVkZHw8cd60H3UKPjhB6MjEkKrNYkUFRXx4YcfsmXLFnbt2kXPnj0Z34A29Weffca//vUv/vOf\n/7B7927eeustvvnmG9LT04mLi+Orr74iNjaW9PR0APbt28eKFSvYt28fubm5TJs2zdnEmjp1KpmZ\nmdhsNmw2G7m5ufWOR3hGUhK8+aZ7d609U3jqrrvcdw9v0LEjvPuu3hE4OlrXJ2nqHA69xkYYp06z\nsxYsWMDWPjUeAAAZn0lEQVTw4cPZunUr69at4/HHH6/3jb744gsGDhxIy5Ytad68Ob/4xS94/fXX\nWbt2LcnJyQAkJyezZs0aAHJyckhMTMRkMhEcHExISAj5+fmUlJRQXl5OVFQUAElJSc5zhPfp0AFu\nv12Xi3UXowtPeZLJpOuRzJwJN92ku/GaqiNHYPhw/btYv97oaJquGsdEKisr8ff3Z9euXWzZsoXl\ny5czb948rFYrQ4YM4b777qvXjXr16sUf/vAHysrKaNmyJevWraN///6UlpYSGBgIQGBgIKWlpQAU\nFxcTHR3tPN9iseBwODCZTFjO2hDJbDbjcDgues/Zs2c7v46JiSEmJqZeMQvXmDJFd2vNnOn6AfaT\nJ/Un0c2bXXtdb/fAA9Cjhx5of+SRplcx8cQJ3a0XG6vLDqekwJ49IMO19ZeXl0feZWyTUGMSiYqK\nYufOnURGRnL99dcTEhLC5s2beeWVV8jLy6t3EgkLC2PWrFkMHTqUq666ij59+tC8efNzXuPn54ef\nC/9LODuJCOPcdJN+g9uyRa9md6XVq6FXL+je3bXXbQwGD9Z17c8MuL/4IjTCiZT1VlWld0EOCoJn\nntGTDG67TSdS6dqqv/M/YKemptbr/Bq7s86MP/Tv359BgwaxevVqwsPD2bJlCwUFBQ0KdvLkyWzf\nvp1NmzbRrl07QkNDCQwM5ODBgwCUlJTQqVMnQLcwCgsLnecWFRVhsVgwm80UFRWdc9xsNjcoHuEZ\n7hxg9/UB9dqcqZhYUaETdA2Ncp+hlN4epqxMd5GemaX29NPw739Lt5YRakwihw4d4plnnmH8+PGk\npKTQvXt3SktLWbVqFc8880yDbvZt9dLbgoICVq9ezYQJE0hISCArKwuArKwsRo8eDUBCQgLZ2dlU\nVFRgt9ux2WxERUURFBREmzZtyM/PRynF0qVLnecI75WUpOtnuHKAfc8eXXiqqVdr/tnPYPlyvdBy\n4EDdOvFV6em6RfvGG3rjyjNat9atkAce0GMlwnNq7M6qqqqivLzcpTcbO3Yshw8fxmQykZGRQUBA\nAI899hjjxo0jMzOT4OBgVq5cCUB4eDjjxo0jPDwcf39/MjIynF1dGRkZpKSkcPz4ceLj453134X3\nat8eRo6ErCzd7eAK//gH3H+/9xae8iQ/P3j8cYiI0El1/ny99YwvWbJEt2Y//PDiYx+33KL/xh5+\nWO9eIDyjxhXrffv2ZVcj3mtBVqx7nw8+0NUGP//88geBzxSe2rMHpDfzXJ9/rgedb7tNJxNf2C91\n/XqYNAk2bbr0+Fd5OfTurTewHDHCc/H5EpfXExHCVX7+c11j3BUzqZYt0zXKJYFcqEcPyM+Hffsg\nPt6zm2C6w8cf6+7QN96ofQKFdGt5Xo1JZOPGjZ6MQzQBrtoiXin9SbMxFZ7ytHbt9BhU794QFQWf\nfWZ0RA3z1Ve6VbV4Mdx4Y93OObtbS7hfncvjNjbSneWdvv8eunaFr7+G6jpn9fbhh7pr44svZA+p\nunjlFXjoIT2TbexYo6Opu4MHdev18cd1N2h9/PijTqB//7tujYm6k+4s4dXatdOfLKsn5DXImVaI\nJJC6mThRr2z/7W/h97/X6yy83Q8/6Df/lJT6JxCAVq3gX//SC12lW8u9pCUiPO7DD3U98S++qP8A\n+6FDekfb//7XN+qGeNKhQ3qF+xVXwKuv6oTujSoq9KSAbt106+lyJmFMm6bLJb/0kuvi83XSEhFe\nb9AgPS1306b6n7t4MdxxhySQhujYETZs0APvAwZ45zjJ6dO6q7JVK71H2OXO4nv6af13tm6da+IT\nF5IkIjzOz093M9R3gL2qSq8NkQH1hvP311uFzJ6tZ7etWmV0ROf63e/gwAHdUjpvV6QGadVKz9aS\nbi33ke4sYYgzA+w2m/6EXBfr1sFTT8F//uPe2JqKnTt1q27CBPjTn1zzpn05nnlGj2N88IHrW5q/\n/jUcOybdWnUh3VmiUWjXTm8cWJ8BdpnW61o33KAT8rZterv+7783Lpbly+Fvf4PcXPd0Vc6bp7u1\n3n7b9ddu6iSJCMNMmaK3sajLh579+2HrVt8vPOVp3jBO8t57ukzAunV6Q0l3OLtby8hk6YskiQjD\nREfrmUJ1KWXwwgt61XJTKDzlaUaOk+zaBYmJsHKl3vfLnW6+WU8vl0WIriVjIsJQf/+77gPPzq75\nNSdP6k+omzc3zbohnuTJcRK7Xdeaee45GDPGffc525lFiM8/r6cRiwvV971Tkogw1JEjEByst7eo\nLiVzgVdf1VN7ZScez/DEepJDh3QCmT4dHnzQ9de/lPff1+WU9+zx3rUyRpKBddGotG2r62BcaoB9\n0SIZUPekM+MkYWHuGSf56Sc9kD9mjOcTCPyvW+uhhzx/b18kLRFhuG3b9CfDL7+8cCuTPXtg+HA9\nsC51Qzxv6VI9huCqfbdOndKz8jp21NNtjaoLL91aNZOWiGh0Bg6EK6/U3QznW7RICk8Z6Z57XLfv\nllJ6dpRS8M9/GpdAQM/WWrxYZmu5grREhFdYuFAPnK9Y8b9jUnjKexw6BOPGQcuWDR8n+b//091k\n778PV13l+hgb4sEHdatkyRKjI/Ee0hIRjdLEifoN5ttv/3fslVek8JS36NgR3n234eMkGRl6Gu/b\nb3tPAgFds33zZlmEeDkkiQivEBCgB9jPbEuhlAyoext/f72q/Kmn6reeZPVqmDNHr0av6xY3niLd\nWpfPo0kkLS2Nnj17EhERwYQJEzh58iRlZWXExcURGhrK0KFDOXLWLmlpaWlYrVbCwsLYsGGD8/iO\nHTuIiIjAarUyY8YMTz6CcKMpU3Rf+enTerv4kyd1lTrhXeozTrJlC/zqV/Dmm3D99Z6LsT5iYvRg\n/8yZRkfSSCkPsdvtqmvXrurEiRNKKaXGjRunlixZoh599FE1b948pZRS6enpatasWUoppfbu3asi\nIyNVRUWFstvtqlu3bur06dNKKaUGDBig8vPzlVJKjRgxQq1fv/6C+3nw0YSLnD6tVGSkUu++q9SE\nCUo984zREYlL+fZbpWJilBo+XKmysgt/vmePUp06KbVhg+djq6/ycqWuv16pN980OhLj1fe902Mt\nkTZt2mAymTh27BiVlZUcO3aMzp07s3btWpKTkwFITk5mzZo1AOTk5JCYmIjJZCI4OJiQkBDy8/Mp\nKSmhvLycqKgoAJKSkpzniMbtTA32uXN1H3VKitERiUu51DhJYaGuTPi3v0FcnHEx1tWZvbV+9Svp\n1qovf0/dqH379jzyyCNce+21XHnllQwbNoy4uDhKS0sJDAwEIDAwkNLSUgCKi4uJjo52nm+xWHA4\nHJhMJiwWi/O42WzG4XBc9J6zZ892fh0TE0NMTIzrH0y41N13w6OP6hXTsprY+50ZJ7nhBj1OsmgR\nxMbqtT0zZujtUxqLmBg9Ljdz5uWVb25s8vLyyKvLBnY18FgS+eabb3j22WfZv38/AQEB3Hnnnbzy\nyivnvMbPzw8/F04ePzuJiMYhIEDvpTRkiNGRiPq45x7o2VPvuwX6fx95xNiYGiItDSIj9RjOyJFG\nR+MZ53/ATk1Nrdf5Hksi27dvZ9CgQXTo0AGAO+64g61btxIUFMTBgwcJCgqipKSETtUbKJnNZgoL\nC53nFxUVYbFYMJvNFBUVnXPcLHNAfcq99xodgWiIM/VJVq3SkyQaozPdWhMn6r29pDVcO4+NiYSF\nhbFt2zaOHz+OUoqNGzcSHh7OyJEjyapuO2ZlZTF69GgAEhISyM7OpqKiArvdjs1mIyoqiqCgINq0\naUN+fj5KKZYuXeo8RwhhrI4d9bTs87evaUzO7tYStfNYSyQyMpKkpCT69+9Ps2bNuOGGG3jggQco\nLy9n3LhxZGZmEhwczMqVKwEIDw9n3LhxhIeH4+/vT0ZGhrOrKyMjg5SUFI4fP058fDzDhw/31GMI\nIZqA9HS9t1ZT6tZqKNn2RAghLmLTJj3Ro6ltGS/1RKpJEhFCXK7p0+HoUXj5ZaMj8RxJItUkiQgh\nLtdPP+lurWefbTrdWpJEqkkSEUK4wqZNer3Lnj3Qvr3R0bifJJFqkkSEEK7ym9/oUs5NoVtLkkg1\nSSJCCFdpSt1akkSqSRIRQrhSU+nWkqJUQgjhBr/4BYwZo//t2GF0NN5DkogQQtTR/PmQkACjRsHQ\nofDvf+sCak2ZdGcJIUQ9VVTo8s1PPw1t2sBjj+nCVo15u5czZEykmiQRIYS7VVVBTo7eJuWHH+B3\nv9ObN7ZoYXRkDSdJpJokESGEpygF77+vk8nnn8PDD8P99+tdgRsbGVgXQggP8/ODW26BDRtgzRrY\nuhW6doWnnoLvvjM6OveSJCKEEC7Urx+sXAkffgglJRAaqqs8FhQYHZl7SBIRQgg3CA2FF1/Uteev\nuAL69oXkZNi71+jIXEuSiBBCuFHnznoW19dfQ/fuugb9qFG6y8sXyMC6EEJ40PHjsHixXnMSHKyn\nBw8bpsdVvIHMzqomSUQI4c1OndJjJ+np0Ly5TiZjx4K/x+rNXpwkkWqSRIQQjYFSsG6dTibFxfDo\no5CSAi1bGhOP107x/fLLL+nbt6/zX0BAAM899xxlZWXExcURGhrK0KFDOXLkiPOctLQ0rFYrYWFh\nbNiwwXl8x44dREREYLVamTFjhqceQQghXM7PD267DbZsgawsePttPT04PV1XVfR6ygBVVVUqKChI\nFRQUqEcffVTNmzdPKaVUenq6mjVrllJKqb1796rIyEhVUVGh7Ha76tatmzp9+rRSSqkBAwao/Px8\npZRSI0aMUOvXr7/gHgY9mhBCXLZPP1Xq7ruVat9eqT//WalTpzx37/q+dxoyO2vjxo2EhITQpUsX\n1q5dS3JyMgDJycmsWbMGgJycHBITEzGZTAQHBxMSEkJ+fj4lJSWUl5cTFRUFQFJSkvMcIYTwBRER\nem+u7dvhvff0QkZvXWdiSBLJzs4mMTERgNLSUgIDAwEIDAyktLQUgOLiYiwWi/Mci8WCw+G44LjZ\nbMbhcHgweiGE8IyuXeHdd2HECOjfH15/3eiILuTxeQAVFRW8+eabzJs374Kf+fn54efCeW6zZ892\nfh0TE0NMTIzLri2EEJ7QvDk8/rhujUyYoJPKM8/Az37mmuvn5eWRl5fX4PM9nkTWr19Pv3796Nix\nI6BbHwcPHiQoKIiSkhI6deoE6BZGYWGh87yioiIsFgtms5mioqJzjpvN5ove6+wkIoQQjdnAgbBz\nJ0ydCgMGwPLlumTv5Tr/A3Zqamq9zvd4d9by5cudXVkACQkJZGVlAZCVlcXo0aOdx7Ozs6moqMBu\nt2Oz2YiKiiIoKIg2bdqQn5+PUoqlS5c6zxFCCF8WEADLlukt52NjYeFC44tieXSdyE8//cR1112H\n3W6ndevWAJSVlTFu3DgKCgoIDg5m5cqVtG3bFoC5c+eyePFi/P39WbBgAcOGDQP0FN+UlBSOHz9O\nfHw8zz333IUPJutEhBA+7KuvIDERLBbIzISrr3bNdWWxYTVJIkIIX1dRAb//PaxYAS+/DDfffPnX\nlCRSTZKIEKKpeOcdmDRJ/5s9G0ymhl/La1esCyGEcI9hw2DXLj3wPmQI2O2eu7ckESGE8AGBgXrL\nlDvvhKgoPXvLE6Q7SwghfMzOnXDXXfDzn8Pzz9ev1rt0ZwkhRBN3ww06kfj5/e9rd5EkIoQQPqhV\nK138KjVVj5k88wycPu36+0h3lhBC+Di7XW+Z0rYtLFmix09qIt1ZQgghztG1K2zeDP36Qd++cFZ5\npssmLREhhGhC3n8fkpL0wPucOdCixbk/l5aIEEKIGt18s15T8tVXMGgQ2GyXdz1JIkII0cRcfTWs\nWaNXuA8apLdMaWjHjXRnCSFEE/bpp3ojxz59YNEiCAiQ7iwhhBB11Ls3/Oc/0Lq1HnSvL2mJCCGE\nAGD1ahgzRnbxBSSJCCFEQ8jsLCGEEB4jSUQIIUSDSRIRQgjRYJJEhBBCNJhHk8iRI0cYO3YsPXr0\nIDw8nPz8fMrKyoiLiyM0NJShQ4dy5MgR5+vT0tKwWq2EhYWx4azNXnbs2EFERARWq5UZM2Z48hG8\nRl5entEhuJU8X+Ply88Gvv989eXRJDJjxgzi4+P5/PPP+fTTTwkLCyM9PZ24uDi++uorYmNjSU9P\nB2Dfvn2sWLGCffv2kZuby7Rp05wzBqZOnUpmZiY2mw2bzUZubq4nH8Mr+Pofsjxf4+XLzwa+/3z1\n5bEkcvToUbZs2cLkyZMB8Pf3JyAggLVr15KcnAxAcnIya9asASAnJ4fExERMJhPBwcGEhISQn59P\nSUkJ5eXlREVFAZCUlOQ8RwghhGd5LInY7XY6duzIpEmTuOGGG7j//vv56aefKC0tJbB6c/vAwEBK\nS0sBKC4uxmKxOM+3WCw4HI4LjpvNZhwOh6ceQwghxFn8PXWjyspKdu7cyd///ncGDBjAzJkznV1X\nZ/j5+eHn5+eye7ryWt4oNTXV6BDcSp6v8fLlZwPff7768FgSsVgsWCwWBgwYAMDYsWNJS0sjKCiI\ngwcPEhQURElJCZ06dQJ0C6OwsNB5flFRERaLBbPZTFFR0TnHzWbzBfeT1epCCOF+HuvOCgoKokuX\nLnz11VcAbNy4kZ49ezJy5EiysrIAyMrKYvTo0QAkJCSQnZ1NRUUFdrsdm81GVFQUQUFBtGnThvz8\nfJRSLF261HmOEEIIz/JYSwTg+eef5+6776aiooJu3brx0ksvUVVVxbhx48jMzCQ4OJiVK1cCEB4e\nzrhx4wgPD8ff35+MjAxn91RGRgYpKSkcP36c+Ph4hg8f7snHEEIIcYbyQevXr1fdu3dXISEhKj09\n3ehwXKqgoEDFxMSo8PBw1bNnT7VgwQKjQ3K5yspK1adPH3X77bcbHYrLff/992rMmDEqLCxM9ejR\nQ23dutXokFxq7ty5Kjw8XPXq1UslJiaqEydOGB3SZZk0aZLq1KmT6tWrl/PY4cOH1a233qqsVquK\ni4tT33//vYERXp6LPd9vf/tbFRYWpnr37q1++ctfqiNHjlzyGj63Yr2qqooHH3yQ3Nxc9u3bx/Ll\ny/n888+NDstlTCYTf/vb39i7dy/btm1j4cKFPvV8AAsWLCA8PNwnJ0acv1aqR48eRofkMvv37+ef\n//wnO3fuZM+ePVRVVZGdnW10WJdl0qRJF6xDq2ltW2N0secbOnQoe/fuZffu3YSGhpKWlnbJa/hc\nEvn4448JCQkhODgYk8nEXXfdRU5OjtFhuUxQUBB9+vQBoFWrVvTo0YPi4mKDo3KdoqIi1q1bx333\n3edzkyNqWivlK9q0aYPJZOLYsWNUVlZy7Nixi056aUwGDx5Mu3btzjlW09q2xuhizxcXF0ezZjo1\nDBw48JyJTBfjc0nE4XDQpUsX5/dn1pf4ov3797Nr1y4GDhxodCgu89BDDzF//nznH7EvudhaqWPH\njhkdlsu0b9+eRx55hGuvvZbOnTvTtm1bbr31VqPDcrma1rb5osWLFxMfH3/J1/jcf6m+2AVyMT/+\n+CNjx45lwYIFtGrVyuhwXOKtt96iU6dO9O3b1+daIfC/tVLTpk1j586dXHXVVY26K+R833zzDc8+\n+yz79++nuLiYH3/8kWXLlhkdllu5em2bN5kzZw4tWrRgwoQJl3ydzyWR89eXFBYWnrPC3RecOnWK\nMWPGMHHiRJ+a3vzRRx+xdu1aunbtSmJiIv/+979JSkoyOiyXudhaqZ07dxoclets376dQYMG0aFD\nB/z9/bnjjjv46KOPjA7L5QIDAzl48CDAOWvbfMmSJUtYt25dnT4E+FwS6d+/Pzabjf3791NRUcGK\nFStISEgwOiyXUUpx7733Eh4ezsyZM40Ox6Xmzp1LYWEhdrud7OxsbrnlFl5++WWjw3KZmtZK+Yqw\nsDC2bdvG8ePHUUqxceNGwsPDjQ7L5RISEi66ts1X5ObmMn/+fHJycmjZsmXtJ7hz+phR1q1bp0JD\nQ1W3bt3U3LlzjQ7HpbZs2aL8/PxUZGSk6tOnj+rTp49av3690WG5XF5enho5cqTRYbjcJ598ovr3\n71/n6ZONzbx585xTfJOSklRFRYXRIV2Wu+66S11zzTXKZDIpi8WiFi9erA4fPqxiY2N9Yorv+c+X\nmZmpQkJC1LXXXut8f5k6deolr+GnlA92PgshhPAIn+vOEkII4TmSRIQQQjSYJBEhhBANJklECCFE\ng0kSEcIFDh8+TN++fenbty/XXHMNFouFvn370rp1ax588EGjwxPCbWR2lhAulpqaSuvWrXn44YeN\nDkUIt5OWiBBucOazWV5eHiNHjgRg9uzZJCcnM2TIEIKDg1m9ejW//e1v6d27NyNGjKCyshKAHTt2\nEBMTQ//+/Rk+fLhzdbQQ3kiSiBAeZLfbef/991m7di0TJ04kLi6OTz/9lCuvvJK3336bU6dOMX36\ndF5//XW2b9/OpEmT+MMf/mB02ELUyKOVDYVoyvz8/BgxYgTNmzenV69enD59mmHDhgEQERHB/v37\n+eqrr9i7d69z99uqqio6d+5sZNhCXJIkESE8qEWLFgA0a9YMk8nkPN6sWTMqKytRStGzZ0+f3LhQ\n+CbpzhLCQ+oyh6V79+4cOnSIbdu2AXrH5n379rk7NCEaTJKIEG5wpsbE2fUmzq89cX4dCj8/P0wm\nE6tWrWLWrFn06dOHvn37snXrVs8FLkQ9yRRfIYQQDSYtESGEEA0mSUQIIUSDSRIRQgjRYJJEhBBC\nNJgkESGEEA0mSUQIIUSD/T9LZJfF/OY7uQAAAABJRU5ErkJggg==\n",
       "text": [
        "<matplotlib.figure.Figure at 0x1e585210>"
       ]
      }
     ],
     "prompt_number": 7
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "In addition to time-based analysis, we can do other types of analysis as well. Generally speaking, one of the first things you'll want to do when exploring new data is count things, so let's compute the Twitter accounts that authored the most tweets and compare it to the total number of unique accounts that appeared."
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "from collections import Counter\n",
      "\n",
      "# The \"user\" field is a record (dictionary), and we can pop it off\n",
      "# and then use the Series constructor to make it easy to use with pandas.\n",
      "\n",
      "user_col = df.pop('user').apply(pd.Series)\n",
      "\n",
      "# Get the screen name column\n",
      "authors = user_col.screen_name\n",
      "\n",
      "# And count things\n",
      "authors_counter = Counter(authors.values)\n",
      "\n",
      "# And tally the totals\n",
      "\n",
      "print\n",
      "print \"Most frequent (top 25) authors of tweets\"\n",
      "print '\\n'.join([\"{0}\\t{1}\".format(a, f) for a, f in authors_counter.most_common(25)])\n",
      "print\n",
      "\n",
      "# Get only the unique authors\n",
      "\n",
      "num_unique_authors = len(set(authors.values))\n",
      "print \"There are {0} unique authors out of {1} tweets\".format(num_unique_authors, len(df))"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "output_type": "stream",
       "stream": "stdout",
       "text": [
        "\n",
        "Most frequent (top 25) authors of tweets\n",
        "_net_shop_\t165\n",
        "PC_shop_japan\t161\n",
        "work_black\t160\n",
        "house_book_jp\t160\n",
        "bousui_jp\t147\n",
        "Popular_goods\t147\n",
        "pachisuro_777\t147\n",
        "sweets_shop\t146\n",
        "bestshop_goods\t146\n",
        "__electronics__\t142\n",
        "realtime_trend\t141\n",
        "gardening_jp\t140\n",
        "shikaku_book\t139\n",
        "supplement_\t139\n",
        "__travel__\t138\n",
        "disc_jockey_jp\t138\n",
        "water_summer_go\t138\n",
        "Jungle_jp\t137\n",
        "necessaries_jp\t137\n",
        "marry_for_love\t137\n",
        "trend_realtime\t136\n",
        "sparkler_jp\t136\n",
        "PandoraQQ\t133\n",
        "flypfox\t133\n",
        "Promo_Culturel\t132"
       ]
      },
      {
       "output_type": "stream",
       "stream": "stdout",
       "text": [
        "\n",
        "\n",
        "There are 71794 unique authors out of 125681 tweets"
       ]
      },
      {
       "output_type": "stream",
       "stream": "stdout",
       "text": [
        "\n"
       ]
      }
     ],
     "prompt_number": 8
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "At first glance, it would appear that there are quite a few bots accounting for a non-trivial portion of the tweet volume, and many of them appear to be Japanese! As usual, we can plot these values to get better intution about the underlying distrubution, so let's take a quick look at a frequency plot and histogram. We'll use logarithmic adjustments in both cases, so pay close attention to axis values."
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "# Plot by rank (sorted value) to gain intution about the shape of the distrubtion\n",
      "\n",
      "author_freqs = sorted(authors_counter.values())\n",
      "\n",
      "plt.loglog(author_freqs)\n",
      "plt.ylabel(\"Num Tweets by Author\")\n",
      "plt.xlabel(\"Author Rank\")\n",
      "\n",
      "# Start  a new figure\n",
      "\n",
      "plt.figure()\n",
      "\n",
      "# Plot a histogram to \"zoom in\" and increase resolution.\n",
      "\n",
      "plt.hist(author_freqs, log=True)\n",
      "plt.ylabel(\"Num Authors\")\n",
      "plt.xlabel(\"Num Tweets\")"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 9,
       "text": [
        "<matplotlib.text.Text at 0x21c29fd0>"
       ]
      },
      {
       "metadata": {},
       "output_type": "display_data",
       "png": "iVBORw0KGgoAAAANSUhEUgAAAYwAAAEUCAYAAAA4DAFkAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAHfNJREFUeJzt3XtUVXXex/HPQcW7VpOSCfOAtwELJcgxTZSmmy3HSysv\nZI0J+Fg20GXZxbQnsRqVrMmymVyWZcsxNNNCyahQUbsoljpmZpnCivEyTjoGQylwzn7+YDyIAXvD\nOWefA7xfa5112Jezf19/S86X32X/tsMwDEMAAJgI8ncAAIDGgYQBALCEhAEAsISEAQCwhIQBALCE\nhAEAsISEAQCwhIQBALAkoBLGgQMHNG3aNI0fP15Lly71dzgAgPM4AvFOb5fLpcTERL311lv+DgUA\n8F8+b2EkJycrJCRE0dHR1fbn5OQoMjJSvXv3VkZGhnv/+vXrNWLECCUmJvo6NABAPfi8hbFt2zZ1\n6NBBkyZN0pdffilJcjqd+s1vfqPc3Fx1795dAwYMUGZmpqKiotyfGz16tLKysnwZGgCgHlr6uoD4\n+HgVFhZW25efn69evXopPDxckpSYmKisrCydOHFCa9eu1ZkzZ3Tdddf5OjQAQD34PGHU5MiRIwoL\nC3Nvh4aGaseOHRo2bJiGDRtm+nmHw+HL8ACgyfKkU8kvs6S88YVvGIbXXrNnz/bq+XUdr+mY2b4L\nj9d1rKnVRX22qQvqgrqoe9tTfkkY3bt3V1FRkXu7qKhIoaGh/ghFkpSQkODV8+s6XtMxs30XHq9v\nvPURaHVR321voi4afm3qwvr5jaouDBsUFBQYV155pXu7vLzc6NGjh1FQUGCcPXvW6N+/v7F//37L\n15NkzJ4929i8ebMPom1cZs+e7e8QAgZ1UYW6qEJdGMbmzZuN2bNnG55+5fs8YSQmJhrdunUzgoOD\njdDQUOO1114zDMMwNmzYYPTp08fo2bOnMXfu3Hpd06Y81yiQNKtQF1WoiyrURRVPvzsD8sY9Mw6H\nwyv9cQDQnHj63RlQS4MAAAJXo00Y6enpysvL83cYABDw8vLylJ6e7vF16JICgGaCLikAgC1IGAAA\nSxptwmAMAwCsYQyj8YUNAH7FGAYAwBYkDACAJSQMAIAljTZhMOgNANYw6N34wgYAv2LQGwBgCxIG\nAMASEgYAwBISBgDAkkabMJglBQDWMEuq8YUNAH7FLCkAgC1IGAAAS0gYAABLSBgAAEtIGAAASxpt\nwmBaLQBYw7Taxhc2APgV02oBALYgYQAALCFhAAAsIWEAACwhYQAALCFhAAAsIWEAACwhYQAALGm0\nCYM7vQHAGu70bnxhA4Bfcac3AMAWJAwAgCUkDACAJSQMAIAlJAwAgCUkDACAJSQMAIAlJAwAgCUk\nDACAJSQMAIAlJAwAgCWNNmGw+CAAWMPig40vbADwKxYfBADYgoQBALCkzoThdDr1/PPP2xULACCA\n1ZkwWrRooTfffNOuWAAA9fTZZ9K779pTVkuzE4YMGaLU1FRNmDBB7du3d++PjY31aWAAAHO7d0v7\n9kljxvi+LNOEsXv3bjkcDj3xxBPV9m/evNlnQQEArLFzwqhpwuBeBwAIbA6HPeWYzpI6ffq0Hnzw\nQcXFxSkuLk7Tp0/Xjz/+aEdsAAATdrYwTBNGcnKyOnXqpNWrV+utt95Sx44dlZSUZEdsAAAL7Gph\nmHZJHTp0SGvXrnVvp6enq3///j4NCgBgTUC1MNq2batt27a5tz/++GO1a9fOp0EBAKwLmBbG4sWL\nNWnSJPe4xcUXX6w33njD54EBAMwF1CypmJgY7d27V8XFxZKkTp06+TwoAIB1AdPCOHPmjNasWaPC\nwkI5nU4ZhlHjfRkAAPsFVAtj9OjRuuiiixQXF6c2bdrYERMAoB4CpoVx5MgRffDBB3bEIknKysrS\ne++9p+LiYqWkpOjGG2+0rWwAaGwCapbU4MGDtXfvXjtikVTZolmyZIkWL16sVatW2VYuADRWfr/T\nOzo6WtHR0fr4448VFxenPn36uPf169evXoUkJycrJCRE0dHR1fbn5OQoMjJSvXv3VkZGRrVjTz/9\ntFJTU+tVDgA0NwExhrF+/XpJNT/Sz1HPdJaUlKS0tDRNmjTJvc/pdCo1NVW5ubnq3r27BgwYoFGj\nRikyMlIzZszQLbfcopiYmHqVAwDNkd/HMMLDwyVJf/jDH7R8+fJqx2raV5f4+HgVFhZW25efn69e\nvXq5y0lMTFRWVpZyc3O1ceNGFRcX67vvvtPdd99tuRwAaG4CooVxzr59+6ptV1RU6IsvvvC44CNH\njigsLMy9HRoaqh07dmjRokVKS0sz/Xx6err754SEBCUkJHgcEwA0RrW1MPLy8ry64nitCWPu3Lma\nN2+efv75Z3Xs2NG9v1WrVpo6darHBde3W+tC5ycMAGiu6mphXPjH9Jw5czwqq9ZB75kzZ6qkpEQP\nPfSQSkpK3K9Tp05p/vz5HhUqSd27d1dRUZF7u6ioSKGhoR5fFwCaG7+PYZxzyy23aOvWrb/YP3To\nUI8Kvvrqq3Xw4EEVFhbq8ssv16pVq5SZmenRNQGguQmoMYwFCxa4u4/OnDmj/Px8xcXFadOmTZYL\nuf3227VlyxadPHlSYWFhevLJJ5WUlKSXXnpJN998s5xOp1JSUhQVFWX5munp6YxdAIDMWxjeGstw\nGBfOmTVRVFSk+++/v9ozMuxW01RfAGiOnn1WOn688t2Mp9+dpnd6Xyg0NFRff/11gwsEAHhPQHVJ\nnT/F1eVyac+ePYqLi/NpUFbQJQUAlQKmS2rZsmXuZkzLli0VERGha6+91uOCPUGXFABUeuYZ6V//\nkhYsMD/X0+9O0xbG5MmTq21///33WrBggR5++OEGFwoA8B6/Lz54vhMnTugvf/mLhgwZooSEBB0/\nftzXcQEALAiIMYzi4mKtXbtWmZmZ+u677zRmzBgVFBToyJEj9kUHADDl9xv3QkJCdOONN2rOnDm6\n5pprJMmvU2kvxKA3AFhrYfh80HvhwoXKzMxUeXm5xo8fr3HjxumGG25QQUGBx4V6ikFvAKg0f750\n+nTluxmf3YfxwAMPaMeOHVq9erWcTqfGjBmjY8eOKSMjQ99++22DCwQAeE9APaK1Z8+emjVrlr78\n8kvt3LlTP/74o2655RY7YgMAWGDXGEa9lwYJBHRJAUCluXOlkhJp3jzzc21fGiRQpKene/XBIADQ\nWFm509sbzxCihQEAjdif/iSVlla2NMz4vIWxbt06uVyuBhcAAPCtgLnTe9WqVerVq5ceeeQRHThw\nwI6YAAAWBdQsqRUrVmj37t3q0aOHJk+erEGDBmnJkiUqKSmxIz4AgImAaWFIUufOnTV27FhNmDBB\nR48e1TvvvKOrrrpKL774oq/jAwDUIaBaGFlZWbr11luVkJCg8vJy7dy5U++//7727t2rP//5z3bE\nWCNmSQFApYCZJXXXXXcpJSVFQ4cO/cWx3Nxc3XDDDR4HUV/MkgKASk8+KZWXS089ZX6up9+dlqbV\nHjt2TPn5+QoKCtKAAQN02WWXNbhAbyBhAEClJ5+UKioq3834fFrtq6++qoEDB2rt2rVavXq1Bg4c\nqKVLlza4QACA9wTE8zDOeeaZZ7R792796le/kiSdPHlSgwYNUkpKis+DAwCYC5hZUpdeeqk6dOjg\n3u7QoYMuvfRSnwYFALAmIFoYzz33nCSpV69eGjhwoMaMGSOpctZUv3797IkOAGDK70/cKykpkcPh\nUM+ePdWjRw85/hvR6NGj3T/7E0/cA4DKFoaVabU+feJeIGOWFABUmj1bCgqqfDfTbJc3BwAE2J3e\nAIDAFjCzpAAAgSugWhgPP/ywiouLVV5eruuvv16XXnqpli9fbkdsAAALAqaF8eGHH6pTp07Kzs5W\neHi4Dh06pAULFtgRGwDAREC1MCoqKiRJ2dnZGjt2rDp37hwQ02oBAJX8fh/GOSNHjlRkZKTatGmj\nl19+WSdOnFCbNm3siA0AYCKgWhjp6en65JNP9Pnnnys4OFjt27dXVlaWHbGZxsXzMAAggJ6HERsb\nq127dpnusxM37gFApZkzpfbtpVmzzM/19Luz1i6pY8eO6ejRo/rpp5+0a9cuGYYhh8Oh4uJi/fTT\nTw0uEADgXX4fw/jwww+1bNkyHTlyRNOnT3fv79ixo+bOnWtLcACAutnZ2WLaJfX2229r7NixdsVj\nCV1SAFDpscekTp0q3834fC2pIUOGKCUlRcOHD5ck7d+/nyfuAUCACKhZUpMnT9ZNN92ko0ePSpJ6\n9+6t559/3ueBAQCsCZg7vX/44QdNmDBBLVq0kCS1atVKLVua3r4BAGhiTBNGhw4ddPLkSff29u3b\n1blzZ58GBQCwJiAe0XrOc889p5EjR+rw4cMaPHiw/vWvf+ntt9+2IzYAgIniYumSS+wpyzRhxMXF\naevWrfrmm2/kcrkUGRmpVq1a2REbAMBEu3ZSaak9ZZl2SZWWlmrevHlauHChoqOjVVhYqOzsbDti\nAwCYMAzpoovsKcs0YSQlJSk4OFiffvqpJOnyyy/XLCv3oAMAfM7lCqBZUocOHdKjjz6q4OBgSVL7\n9u19HpQVLD4IAJUtjCCTb3JvLT5oOobRunVr/fzzz+7tQ4cOqXXr1h4X7Clv/OMBoLFzucwTRkJC\nghISEjRnzhyPyjJNGOnp6Ro+fLj+8Y9/aOLEifrkk0+0bNkyjwoFAHiHlYThLaYJ46abblJsbKy2\nb98uSXrhhRfUpUsXnwcGADBn5xiGacK48847NWzYMMXHxysyMtKOmAAAFlkZw/AW02KSk5N19OhR\npaWlKSIiQrfddpsWLlxoR2wAABN2dkmZLm8uSRUVFfr888+1adMmLV68WG3bttU333xjR3w1Ynlz\nAKg0ZYo0cKD0v/9rfq7Pnrh3zvXXX6/S0lINGjRIQ4YM0eeff66uXbs2uEAAgPcEVJdUv3791KpV\nK+3bt0979+7Vvn37qk2zBQD4T0DNkjr37IuSkhItW7ZMSUlJOn78uM6ePevz4AAAdbMzYdRazMyZ\nMyVJixYt0vjx4xUTE6OsrCwlJydrw4YN9kQHAKhTQEyrff/99zV37lydOXNG06dPV2xsLKvUAkCA\nsXMMo9aE4XQ6derUKSUnJ8vhcKikpKTa8UvsWoAdAFCrgBjDOHDggOLi4mo85nA4dPjwYZ8FBQCw\n5tSpAOiSuuKKK7R79257ogAANEjLllJ5uT1l2dSQAQD4gsMhXXyxPWXVmjDuu+8+eyIAADSY0xkA\n02qTkpLsiQAA0GAul9SihT1lBVSXVEFBgaZMmaJx48b5OxQAaBQCooXhDxEREXr11Vf9HQYANBoB\n1cI4fPiwHnzwQd16660aOXKkRo4cqVGjRlkuIDk5WSEhIYqOjq62PycnR5GRkerdu7cyMjLqHzkA\nIDDuwzhnzJgxmjJlikaOHKmg/0blqMek36SkJKWlpWnSpEnufU6nU6mpqcrNzVX37t01YMAAjRo1\nSlFRUQ34JwBA82Vnl5RpwmjTpo1HM6bi4+NVWFhYbV9+fr569eql8PBwSVJiYqKysrIUEhKimTNn\nas+ePcrIyNCjjz7a4HIBoDmws0vKNGGkpaUpPT1dN998s1q3bu3eHxsb2+BCjxw5orCwMPd2aGio\nduzYoUsuuUSLFy+2dI309HT3zwkJCUpISGhwPADQWNXVwsjLy1NeXp7XyjJNGF999ZWWL1+uzZs3\nu7ukJGnz5s0NLrQ+XVq1OT9hAEBzVVcL48I/pufMmeNRWaYJY/Xq1SooKFBwcLBHBZ2ve/fuKioq\ncm8XFRUpNDTUa9cHgObihx/sW0vKdKgkOjpa//73v71a6NVXX62DBw+qsLBQZWVlWrVqVb1mXkmV\nLQxvNrUAoDFyOiWzJ0/k5eV5pVfGYZg8EXzYsGHau3evBgwY4B7DcDgcWrdunaUCbr/9dm3ZskUn\nT55U165d9eSTTyopKUnvv/++HnjgATmdTqWkpOixxx6zHrSHDzIHgKaiTx8pO7vy3Yyn352mCaO2\nv+L9OchMwgCASj16SB99JPXsaX6up9+dpmMYzD4CgMDldAbQtNoOHTq4ZzWVlZWpvLxcHTp0UHFx\nsc+Dq0t6ejrTaQE0e05n5TMx6uKt6bWmXVLnc7lcWrdunbZv36758+d7XHhD0SUFAJUuu0zavVvq\n1s38XJ+PYdQkJiZGe/bsaXChniJhAEClLl2k/fsr3834fAxjzZo17p9dLpe++OILtW3btsEFAgC8\n54cfAmgMY/369e4xjJYtWyo8PFxZWVk+D8wMYxgAmrtzjYVOneo+zy9jGIGCLikAkMrLpXbtKt+t\n8FmXVG1rjpxrbTzxxBMNLhQA4LmyMvO7vL2p1oTRvn37XywSWFpaqqVLl+qHH34gYQCAn5WVSV5c\n5s+UpS6p4uJivfjii1q6dKnGjx+v6dOnq2vXrnbEVyO6pABAKiysXBKkrMza+T6dJXXy5Ek9//zz\nWrFihSZNmqRdu3bp4osvbnBh3sSgN4Dm7tgxa9NpfT7o/dBDD+mdd97R1KlTde+996pjx44eF+Yt\ntDAAQNq6VZo1S9q2zdr5PrtxLygoSMHBwWpVw4iKw+Hw69IgJAwAkDZulP70J2nTJmvn+6xLyuVy\nNfiiAADfKy+3d5aU6QOUAACBiYQBALDk4EH7Hs8qNeKEwSNaATR3xcXSr35lfp5tj2gNRAx6A4D0\n+ONS69bS//2ftfM9/e5stC0MAGjuKirMH57kTSQMAGikrDxtz5tIGADQSNHCAABYQsIAAFiyYwcJ\nwxKm1QJozv7zH2nnTmngQPNzmVbb+MIGAK85fVr6n/+RfvzR+meYVgsAzZDTKbVoYW+ZJAwAaIRI\nGAAAS0gYAABLSBgAAEtIGAAASyoqSBgAABOGIZWUkDAs48Y9AM3Vyy9LAwZU3odhBTfuNb6wAcAr\n5s2rfHjSvHn1+xw37gFAM+NySUF++PYmYQBAI0PCAABYQsIAAFhCwgAAWOKPm/YkEgYANDq0MAAA\nlpAwAACWkDAAAJa4XIxhAAAsoIUBALCEhAEAqNGMGVKnTlWvv/618t1uLe0v0jvS09OVkJCghIQE\nf4cCAD5VVCQ995w0YULVvo4drX8+Ly/PK6t7s1otAAS4iROlESOkO+7w7DqsVgsATZzLJTkc/o6C\nhAEAAc8w/DPIfaEACAEAUBfDoIUBALCAhAEAsISEAQCwxF836l0oAEIAANSFFgYAwBISBgDAEhIG\nAMASxjAAAJbQwgAAWELCAABYwtIgAABLWHwQAGBJoHRJBdQDlEpLS3XvvfeqdevWSkhI0MSJE/0d\nEgD4XaAkjIBqYaxdu1bjx4/XkiVLtG7dOn+H0yh44ylaTQV1UYW6qNIU6qLZjGEkJycrJCRE0dHR\n1fbn5OQoMjJSvXv3VkZGhiTpyJEjCgsLkyS1aNHC16E1CU3hl8FbqIsq1EWVplAXzaaFkZSUpJyc\nnGr7nE6nUlNTlZOTo/379yszM1Nff/21QkNDVVRUJElyuVy+Ds2tvv+hzM6v63hNx8z2XXjcl78A\ngVYX9d32Juqi4demLqyfb6Uuzh/09mdd+HwMIz4+XoWFhdX25efnq1evXgoPD5ckJSYmKisrS/fd\nd59SU1P13nvvadSoUXVed/p078X42Wd5GjQowWvn13W8pmNm+y48Xv2YVFJiOXRTgVYX9dmmLqiL\nmrabQl0cPFg9YSQkVJ1f322PGDYoKCgwrrzySvf26tWrjSlTpri3ly9fbqSmplq+niRevHjx4tWA\nlyf8MkvK4WFnXGXOAADYyS/j7t27d3ePVUhSUVGRQkND/REKAMAivySMq6++WgcPHlRhYaHKysq0\natUq0zELAIB/+Txh3H777Ro8eLC+/fZbhYWF6fXXX1fLli310ksv6eabb1bfvn01YcIERUVF+ToU\nAIAHHAYDAgAACwLg3kEAQGPQJBJGaWmp7rrrLk2dOlVvvvmmv8Pxq4KCAk2ZMkXjxo3zdyh+l5WV\npalTpyoxMVEfffSRv8PxmwMHDmjatGkaP368li5d6u9w/K60tFQDBgzQe++95+9Q/CovL0/x8fGa\nNm2atmzZYukzTSJhsAZVlYiICL366qv+DiMgjB49WkuWLNHixYu1atUqf4fjN5GRkXr55Ze1cuVK\nffDBB/4Ox++eeeYZTZgwwd9h+F1QUJA6duyos2fPWp6lGrAJgzWoqtSnLpq6htTF008/rdTUVDvD\n9Ln61sP69es1YsQIJSYm2h2qz9WnLj766CP17dtXXbp08UeoPlefuoiPj9eGDRs0f/58zZ4921oB\nHt3250Nbt241du3aVe0O8YqKCqNnz55GQUGBUVZWZvTv39/Yv3+/sXz5ciM7O9swDMNITEz0V8g+\nU5+6OGfs2LH+CNXn6lMXLpfLeOSRR4zc3Fw/RuwbDfk/YRiGMWrUKLtD9bn61MWsWbOMBx54wLjp\nppuM0aNHGy6Xy4+Re19D/l+cPXvW8vdFQD0P43y+WoOqMapPXYSEhGjmzJnas2ePMjIy9Oijj9of\nsA/Vpy5yc3O1ceNGFRcX67vvvtPdd99tf8A+Up96OHHihNauXaszZ87ouuuusz9YH6tPXTz99NOS\npDfeeENdunTxeNWJQFOfujhw4IA++OADnT59WmlpaZauH7AJoybndz1JUmhoqHbs2KF27drptdde\n82Nk9qutLi655BItXrzYj5HZr7a6WLRokeVfhKagtnoYNmyYhg0b5sfI7FdbXZxz1113+SMsv6it\nLmbMmKFbb721XtcK2DGMmjS1vwY8QV1UoS4qUQ9VqIsq3qyLRpUwWIOqCnVRhbqoRD1UoS6qeLMu\nGlXCYA2qKtRFFeqiEvVQhbqo4tW68PowvZckJiYa3bp1M4KDg43Q0FDjtddeMwzDMDZs2GD06dPH\n6NmzpzF37lw/R2kP6qIKdVGJeqhCXVTxdV2wlhQAwJJG1SUFAPAfEgYAwBISBgDAEhIGAMASEgYA\nwBISBgDAEhIGAMASEgaatHfffVdBQUH65ptvLJ2/cOFC/fzzz+7tDh06eC2WvLw8de7cWVdddZX6\n9u2rxx9/3KNrjRw50muxAVaQMNCkZWZm6ve//70yMzMtnf/CCy/op59+cm97snCb0+n8xb6hQ4dq\n9+7d2rVrl9asWaMvvviiwdcH7EbCQJP1n//8Rzt27NBLL71U7RGtF/51npqaqjfeeEOLFi3S0aNH\ndd111+n66693H3/88ccVExOjQYMG6cSJE5KkwsJC/e53v1P//v11ww03uBd3mzx5su655x5dc801\ndT6LpE2bNoqJidHhw4clSa+88op++9vfKiYmRmPHjnW3ciZPnqz7779f1157rXr27Kk1a9b84lo7\nd+5UbGysCgoKPKgtwBwJA01WVlaWhg8frl//+tfq0qWLdu3aVeN5DodDDodDaWlpuvzyy5WXl6eN\nGzdKkkpLSzVo0CDt2bNHQ4cO1SuvvCJJSktLU1JSkv7+97/rjjvu0H333ee+3tGjR/XZZ5/p2Wef\nrTW2U6dOKT8/X3379pUk3XbbbcrPz9eePXsUFRWlpUuXus89fvy4PvnkE2VnZ2vGjBnVrvPpp59q\n2rRpWrdunSIiIhpWUYBFJAw0WZmZmRo3bpwkady4cZa7pc4XHBysESNGSJLi4uLcTzPbvn27Jk6c\nKEm688479fHHH0uqTD7jxo2rtStr27ZtiomJUVhYmMaMGaMrrrhCkvTll18qPj5e/fr104oVK7R/\n/3739caMGSNJioqK0j//+U/3tb7++mvdfffdys7ObrZLd8NeJAw0SadOndLmzZuVkpKiiIgILViw\nQG+99ZYkqWXLlnK5XO5zzx/kvlCrVq3cPwcFBamiosK9Xdu6ne3atav1evHx8dqzZ4+++uorrV27\ntlpX1l//+lft3btXs2fPrhZTcHBwjWV269ZNbdu2rbXlBHgbCQNN0ttvv61JkyapsLBQBQUF+v77\n7xUREaFt27YpPDxc+/fvV1lZmU6fPq1Nmza5P9exY0cVFxebXn/w4MFauXKlJGnFihUaOnRoveIL\nDw/X/fffr6eeekpS5XjLZZddpvLycv3tb3+zNNh+0UUXKTs7W4899pi2bNlSr/KBhiBhoElauXLl\nL55XfNttt2nlypUKDQ3V+PHjdeWVV2rChAmKjY11nzN16lQNHz7cPeh9/hf3ubEOSVq0aJFef/11\n9e/fXytWrNALL7xQ7byanP95SbrnnnuUk5OjoqIiPfXUUxo4cKCGDBmiqKioX3yutp+7du2q7Oxs\n/fGPf9TOnTst1w/QEDwPAwBgCS0MAIAlJAwAgCUkDACAJSQMAIAlJAwAgCX/Dx4S5SE/PFdRAAAA\nAElFTkSuQmCC\n",
       "text": [
        "<matplotlib.figure.Figure at 0x1e6cfdd0>"
       ]
      },
      {
       "metadata": {},
       "output_type": "display_data",
       "png": "iVBORw0KGgoAAAANSUhEUgAAAYkAAAEOCAYAAAB8aOvdAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAH8RJREFUeJzt3X1UVHX+B/D3EKCWaFpCyRAQiCOKIIIVZ4nZVQ/LrpCV\nKfRggeaWC3vck67W7hbuKYTV7UE9Z9sHQ4+bI7tajZlO5sOomwECaq7YQjbzc8LURFiQXRsevr8/\nkEmK6wzD3HsHeL/O4RzuRe73PQz68X6frkYIIUBERNQDH7UDEBGR92KRICIiSSwSREQkiUWCiIgk\nsUgQEZEkFgkiIpLEIkFERJJYJIiISJJXFAmz2Yzk5GQ8++yzOHjwoNpxiIjoGq8oEj4+PggICMA3\n33wDrVardhwiIrpGtiKRk5ODoKAgxMTEdDtvMpmg0+kwbtw4FBUVAQCSk5Oxa9cuFBYW4qWXXpIr\nEhER9ZJsRSI7Oxsmk6nbufb2duTm5sJkMqG6uhoGgwGnT5+GRqMBANx666345ptv5IpERES95CvX\nhZOTk2G1WrudKy8vR2RkJMLCwgAAmZmZMBqN+Oyzz/Dhhx+isbEReXl5ckUiIqJekq1I9KSurg4h\nISGOY61Wi7KyMqxYsQIPPvigklGIiMgFihaJrm4ltb6fiGiwcvepEIrObgoODobNZnMc22y2Xs9m\nEkJ41cdLL72keob+kMlbczETMw2GXH2haJFISEhAbW0trFYr7HY7SkpKkJGR0atr5Ofnw2w2yxOQ\niGgAMZvNyM/P79M1ZCsSWVlZSEpKQk1NDUJCQlBcXAxfX1+sX78eqampiI6Oxrx58zBhwoReXTc/\nPx96vV6e0EREA4her+9zkZBtTMJgMPR4Pi0tDWlpaW5ft6tIeEuh8JYc1/PGTIB35mIm1zCT67wp\nl9ls7nPPi0b0tcNKQRqNBidOnMC//vUvVdqPiIjAPffco0rbRETu0mg0bo9N9Lsiodeno7S0EX5+\nym7f0d5+GcHBX6OmplLRdomI+qovRULRKbCe8MUX/8bVq0/i6tUXFG65Cu3tCxVuk4jIfZ7obvKK\nDf56IzQ0CkCM0z9HRDTYeWLgut8VCSIiUk6/6276v/+rAXASQLraUYiIvBq7m4iISBK7m4iISFbs\nbiIiGqDY3URERJLY3URERLJikSAiIkkckyAiGqA4JkFERJI4JkFERLJikSAiIkksEkREJIlFgoiI\nJHF2ExHRAMXZTUREJImzm4iISFYsEkREJIlFgoiIJLFIEBGRJBYJIiKSxCJBRESSuE6CiGiA4joJ\nIiKSxHUSREQkKxYJIiKSxCJBRESSWCSIiEgSiwQREUlikSAiIkleUyRaWlqQmJiIDz74QO0oRER0\njdcUid///veYN2+e2jGIiOg6shWJnJwcBAUFISam+8I3k8kEnU6HcePGoaioCADw0UcfITo6GmPG\njJErDhERuUG2IpGdnQ2TydTtXHt7O3Jzc2EymVBdXQ2DwYDTp0/j4MGDKC0txZYtW/CXv/wFQgi5\nYhERUS/ItndTcnIyrFZrt3Pl5eWIjIxEWFgYACAzMxNGoxEvv/wyAGDTpk0YM2YMNBqNXLGIiKgX\nFN3gr66uDiEhIY5jrVaLsrIyx/GTTz7p9BqdG/wZAFQC0F/7ICKiLp7Y2K+LokXCE3cIoaFROHs2\nC9wFloioZ3q9Hnq93nG8cuVKt6+laJEIDg6GzWZzHNtsNmi12l5dg1uFExG5pt9tFZ6QkIDa2lpY\nrVbY7XaUlJQgIyOjV9fgVuFERK7x6q3Cs7KykJSUhJqaGoSEhKC4uBi+vr5Yv349UlNTER0djXnz\n5mHChAlyRSAioj6SrbvJYDD0eD4tLQ1paWluX5fdTURErul33U2ewO4mIiLXeKK7ic+4JiIaoHgn\nQUREkrx64JqIiPo/djcREQ1Q7G4iIiJJ7G4iIiJZsUgQEZEkjkkQEQ1QHJMgIiJJHJMgIiJZsUgQ\nEZEkjkkQEQ1QHJMgIiJJHJMgIiJZsUgQEZEkFgkiIpLEIkFERJI4u4mIaIDi7CYiIpLE2U1ERCQr\nFgkiIpLEIkFERJJYJIiISBKLBBERSeIUWCKiAYpTYImISBKnwBIRkaxYJIiISBKLBBERSXJaJJYt\nW4ampia0trZi+vTpuP3227F582YlshERkcqcFok9e/ZgxIgR2LlzJ8LCwnDmzBmsXr1aiWxERKQy\np0Wira0NALBz507MmTMHI0eOhEajkT0YERGpz2mRSE9Ph06nQ2VlJaZPn46LFy9i6NChHg3x2Wef\n4dlnn8XcuXOxYcMGj16biIjcd8Mi0dHRgfT0dHz88ceoqKiAv78/brnlFhiNRo+G0Ol0+OMf/4it\nW7fiww8/9Oi1iYjIfTcsEj4+Pvj5z3+O2267Db6+nYuzb7nlFtxxxx1OL5yTk4OgoCDExHRf+GYy\nmaDT6TBu3DgUFRU5zr///vv46U9/iszMTHdeBxERycBpd9OMGTOwbds2CCF6deHs7GyYTKZu59rb\n25GbmwuTyYTq6moYDAacPn0aQGe31u7du7Fp06ZetUNERPJxunfTm2++iVdffRU33XSTYyxCo9Gg\nqanpht+XnJwMq9Xa7Vx5eTkiIyMRFhYGAMjMzITRaMTFixfxzjvv4OrVq/jhD3/o3ishIiKPc1ok\nrly54rHG6urqEBIS4jjWarUoKytDSkoKUlJSPNYOERF5hku7wBqNRhw6dAgajQYpKSlIT3dvB1ZP\nTJ3t3AXWAKASgP7aBxERdfHE7q9dnBaJFStW4OjRo3jssccghMDatWtx5MgRrFq1qteNBQcHw2az\nOY5tNhu0Wm2vrhEaGoWzZ7PArcKJiHqm1+uh1+sdxytXrnT7Wk6LxAcffIDjx4/jpptuAgA89dRT\niIuLc6tIJCQkoLa2FlarFWPHjkVJSQkMBkOvrsHnSRARuUaR50loNBo0NjY6jhsbG13qNsrKykJS\nUhJqamoQEhKC4uJi+Pr6Yv369UhNTUV0dDTmzZuHCRMm9CownydBROQaTzxPwumdxPPPP4/4+HjH\nrcvBgwdRWFjo9MJSdwhpaWlIS0vrXcrr8E6CiMg1nriT0AgXFkCcO3cOR48ehUajwbRp01xaTCcH\njUaD5ORZOHx4EZQvElW4++6FOHOmSuF2iYj6RqPR9HqtWxeXnichhMDtt9+OkSNHoqamBocOHXKr\nMSIi6l+cdjctX74cJSUliI6OdgxeA8D9998vazAp7G4iInKNIt1NUVFROHnyJIYMGdKnhjyB3U1E\nRL0na3dTREQE7Ha7WxcnIqL+TbK7KS8vDwBw8803Iy4uDtOnT3fcTWg0Gqxdu1aZhN/B7iYiItd4\nortJskhMnTrVsR4iPT3d8bkQQtUn03WuuOY6CSIiZ7pWXsuy4vqpp54CALz++utYsmRJt6+9/vrr\nbjdIRET9h9MxiZ6e77Bx40Y5srjk2+4mIiK6EbPZLN+Ka4PBgC1btsBisXTb9bW5uRm33XZbnxrt\nC3Y3ERG5RtbupqSkJNx55534+uuvsXTpUsf0qYCAAMTGxrrdIBER9R+SRSI0NBShoaEoLS1VMg8R\nEXkRpyuuAwICHJ/b7Xa0trZi+PDhTh9fSkRE/Z/TItHc3Oz4vKOjAzt27FD17oLrJIiIXKPYLrDf\nFRcXh+PHj/epYXdwWw4iot7ry7YcTu8ktm/f7vi8o6MDlZWVGDZsmFuNERFR/+K0SLz//vuOFda+\nvr4ICwuD0WiUPRgREanPaZHoaeHc0aNHERgYKEceIiLyIk6LRJdTp07BYDBg69atGDlyJCorK+XM\nJYkD10RErpF94NpisWDr1q0wGAzw9/eH1WpFRUUFwsLC+tSou9QeuNZokiDENwq32ykgYBSami6r\n0jYR9W+yDFzfd999sNvteOSRR/Dee+/h7rvvRnh4uGoFwht0Fgj3ftB91dys3s67RDR4SW7wFxQU\nhP/85z+4cOECLl68qGQmIiLyEpJF4r333kNFRQViYmLw4osv4u6770ZDQwPKysqUzEdERCpyeTHd\nhQsX8Pe//x0GgwE2mw02m03ubN+j9pgEMBVqdTcB7vcpEtHgJuszrrsEBQUhLy8PR44cweHDh91q\njIiI+heXi8T1BvPgNRHRYOLyOglvwXUSRESuUW2DP7VwTKLfvFVE5EVk3eDviy++wLp162C1WtHW\n1uZocMeOHW41SERE/YfTIjF79mwsXLgQ6enp8PHpHMLo2vCPiIgGNqdFYujQofjFL36hRBYiIvIy\nTotEXl4e8vPzkZqaiiFDhjjOx8fHyxqMiIjU57RInDp1Cps3b8aBAwcc3U0AcODAAVmDERGR+pwW\niX/84x+wWCzw9/dXIg8REXkRp4vpYmJi0NDQIHsQo9GIRYsWITMzEx999JHs7RERkXNO7yQaGhqg\n0+mQmJjoGJOQYwrsAw88gAceeACNjY1YunQpZs6c6dHrExFR7zktEitXrnT74jk5Ofjggw8QGBiI\nkydPOs6bTCYsWbIE7e3tWLhwIZYvX+742ssvv4zc3Fy32yQiIs9xWiT0er3bF8/OzkZeXh7mz5/v\nONfe3o7c3Fzs3bsXwcHBSExMREZGBnQ6HVasWIG0tDTExcW53SYREXmO0yIxfPhwx+I5u92O1tZW\nDB8+HE1NTU4vnpycDKvV2u1ceXk5IiMjHZsEZmZmwmg0Yu/evdi3bx+amprw+eef42c/+1nvXw0R\nEXmU0yJx5coVx+cdHR3YsWMHSktL3W6wrq4OISEhjmOtVouysjKsW7cOeXl5bl+XiIg8r1e7wPr4\n+GD27NnIz89HYWGhWw32dUuPzl1gDQAqAeivfRARURdP7P7axWmR2L59u+Pzjo4OVFZWYtiwYW43\nGBwc3O2pdjabDVqt1uXvDw2NwtmzWeBW4UREPdPr9d3Gk/syAclpkXj//fcd//v39fVFWFgYjEaj\n2w0mJCSgtrYWVqsVY8eORUlJCQwGg8vfz+dJEBG5xuufJ5GVlYWDBw+ivr4egYGB+N3vfofs7Gzs\n3r3bMQV2wYIFeP75510Ly+dJqNQ2EfVnfXmehGSRkLo96bqrePHFF91qsC80Gg3uuisKZ88+CeAF\nhVtnkSCi/qXrTmLlypWeLxJr1qz53iBzS0sLNmzYgEuXLqGlpcWtBvuCdxIsEkTUe7I8mW7p0qWO\nz5uamrB27VoUFxcjMzMTzz33nFuNeQLHJIiIXOOJMYkbbvBXX1+P3/zmN4iNjUVrayuqqqpQVFSE\nwMDAPjXaF6GhUQBiVGtfPb7QaDSKf4wYMVrtF05EbtLr9cjPz+/TNW54J/Huu+9i0aJF+PTTTxEQ\nENCnhqiv2qBGV1dzMx9VSzSYSY5J+Pj4wN/fH35+ft//Jo3GpW05PG2wD1yr0zbHQoj6K1kHrr3R\nYB+4ZpEgInf0ZeDa6UOHiIho8GKRICIiSb3a4M8bcAosEZFrvH5bDk/jmATHJIio9zgmQUREsmCR\nICIiSRyTICIaoDgmoSiOSRBR/8QxCSIikgWLBBERSWKRICIiSSwSREQkibObiIgGKM5uUtRgnd3k\nh85nWSgvIGAUmpouq9I20UAiy+NLiTqp87AjgA88IvIGLBLkxTof2ao03sEQfYtFgrwYH9lKpDbO\nbiIiIkksEkREJIlFgoiIJPW7MQmukyAaWEaMGI3m5gbF2x0MExS4TkJRg3WdhFrtqtk2d75VUucM\nNr7PcuIusEREJAsWCSIiksQiQUREklgkiIhIEosEERFJYpEgIiJJXlEkLBYLFi5ciEceeUTtKETo\n2lhQjY8RI0ar8opHjBit2msm7+YVRSI8PBx//etf1Y5BdE3XxoLKf6ixqAzAtXbVec3k3WQrEjk5\nOQgKCkJMTEy38yaTCTqdDuPGjUNRUZFczRMRkQfIViSys7NhMpm6nWtvb0dubi5MJhOqq6thMBhw\n+vRpuSIQEVEfyVYkkpOTMWrUqG7nysvLERkZibCwMPj5+SEzMxNGoxGXL1/GM888g+PHj/PugojI\niyi6wV9dXR1CQkIcx1qtFmVlZRg9ejTefPNNl67RucGfAUAlAP21DyIi6uKJjf26KFokPDGTITQ0\nCmfPZoG7wBIR9Uyv10Ov1zuOV65c6fa1FC0SwcHBsNlsjmObzQatVtura3CrcCIi13jijkLRKbAJ\nCQmora2F1WqF3W5HSUkJMjIyenWN0NAoADFO/xwR0WCn1+uRn5/fp2vIViSysrKQlJSEmpoahISE\noLi4GL6+vli/fj1SU1MRHR2NefPmYcKECb267rd3EkREdCNms7nPRYIPHXIZHzo0eNpW9zWr8VdS\nvQf/AHy4lPz40CEiIpIFn3FNRDRA8RnXimJ30+Bpm91NCreuUtvsbnIFu5uIiEgSu5uIiAYodjcp\nit1Ng6dtdjcp3LpKbbO7yRXsbiIiIkksEkREJIljEkRexZeP9CSP4ZiEojgmMXja5mseHG1zTMIV\n7G4iIiJJLBJERCSJYxJENEipN/4TEDAKTU2XZW+HYxKK4pjE4Gmbr3lwtD141sNwTIKIiGTBIkFE\nRJJYJIiISBKLBBERSeLsJiKiAYqzmxTF2U2Dp22+5sHRNmc3uYLdTUREJIlFgoiIJLFIEBGRJBYJ\nIiKSxCJBRESSOAWWiGiA4hRYRXEK7OBpm695cLTNKbCuYHcTERFJYpEgIiJJLBJERCSJRYKIiCSx\nSBARkSQWCSIikuQV6yRaWlqwePFiDBkyBHq9Ho8++qjakYiICF5yJ/HOO+9g7ty5+POf/4wdO3ao\nHYeIiK6RrUjk5OQgKCgIMTEx3c6bTCbodDqMGzcORUVFAIC6ujqEhIQAAG666Sa5IsnErHaAHpjV\nDiDBrHaAHpjVDtADs9oBemBWO0APzGoHkGBWO4BHyVYksrOzYTKZup1rb29Hbm4uTCYTqqurYTAY\ncPr0aWi1WthsNgBAR0eHXJFkYlY7QA/MageQYFY7QA/MagfogVntAD0wqx2gB2a1A0gwqx3Ao2Qr\nEsnJyRg1alS3c+Xl5YiMjERYWBj8/PyQmZkJo9GIhx56CNu3b8fixYuRkZEhVyQiIuolRQeur+9W\nAgCtVouysjLcfPPNeOutt5SMQkRELlC0SGg0mj5f4/DhnQB29j2M23p6DStVbFuKJzP1/X37Vm9z\nebJtKT1lUqJdKSuh3O/U9Zy9Zjkzufvz7msmud5n57k88e+hEhQtEsHBwY6xBwCw2WzQarUuf38/\n2rCWiGhAUHQKbEJCAmpra2G1WmG321FSUsIxCCIiLyZbkcjKykJSUhJqamoQEhKC4uJi+Pr6Yv36\n9UhNTUV0dDTmzZuHCRMmyBWBiIj6SvQDu3fvFuPHjxeRkZGisLBQtRxnz54Ver1eREdHi4kTJ4o3\n3nhDCCFEfX29mDFjhhg3bpyYOXOmaGhoUDxbW1ubiIuLE7NmzfKKTA0NDeLhhx8WOp1OTJgwQZSW\nlqqeqaCgQERHR4tJkyaJrKwscfXqVcUzZWdni8DAQDFp0iTHuRtlKCgoEJGRkWL8+PHiww8/VDTX\n0qVLhU6nE5MnTxYPPvigaGxsVDRXT5m6rFmzRmg0GlFfX+8VmdauXSt0Op2YOHGi+NWvfqV6prKy\nMpGYmCji4uJEQkKCKC8vdzuT1xeJtrY2ERERISwWi7Db7SI2NlZUV1erkuWrr74Sx44dE0II0dzc\nLKKiokR1dbVYtmyZKCoqEkIIUVhYKJYvX654tj/84Q/i0UcfFenp6UIIoXqm+fPniw0bNgghhGht\nbRWNjY2qZrJYLCI8PFxcvXpVCCHE3LlzxcaNGxXPdOjQIVFVVdXtL7RUhlOnTonY2Fhht9uFxWIR\nERERor29XbFce/bscbS3fPlyxXP1lEmIzv+spaamirCwMEeRUDPT/v37xYwZM4TdbhdCCHHx4kXV\nM6WkpAiTySSEEGLXrl1Cr9e7ncnri8SRI0dEamqq43jVqlVi1apVKib61gMPPCA++ugjMX78eHH+\n/HkhRGchGT9+vKI5bDabmD59uti/f7/jTkLNTI2NjSI8PPx759XMVF9fL6KiosTly5dFa2urmDVr\nltizZ48qmSwWS7e/0FIZCgoKut05p6amik8++USxXNd75513xGOPPaZ4rp4yzZkzR5w4caJbkVAz\n0yOPPCL27dv3vT+nZqbMzExRUlIihBBiy5YtfXrvvGLvphvpaW1FXV2diok6Wa1WHDt2DPfccw8u\nXLiAoKAgAEBQUBAuXLigaJZf/vKXWL16NXx8vn071cxksVgwZswYZGdnIz4+Hk8//TRaWlpUzTR6\n9Gg899xzuOuuuzB27FjceuutmDlzpurvHSD9Xp07d67b7D81f/ffeust/OQnP1E9l9FohFarxeTJ\nk7udVzNTbW0tDh06hHvvvRd6vR4VFRWqZyosLHT8vi9btgyrVq1yO5PXFwlvnEt85coVPPzww3jj\njTcQEBDQ7WsajUbRzDt37kRgYCCmTJkiOUVY6UxtbW2oqqrC4sWLUVVVhVtuuQWFhYWqZjpz5gxe\nf/11WK1WnDt3DleuXMHf/vY3VTP1xFkGNfK98sor8Pf3v+HuzErk+u9//4uCggKsXPntGgSp33ml\nMgGdv+8NDQ0oLS3F6tWrMXfuXNUzLViwAGvXrsXZs2fx2muvIScnx+1MXl8k+rq2wtNaW1vx8MMP\n44knnsDs2bMBdP7v7/z58wCAr776CoGBgYrlOXLkCHbs2IHw8HBkZWVh//79eOKJJ1TNpNVqodVq\nkZiYCACYM2cOqqqqcMcdd6iWqaKiAklJSbjtttvg6+uLhx56CJ988omqmbpIvVff/d3/8ssvERwc\nrGi2jRs3YteuXXj77bcd59TKdebMGVitVsTGxiI8PBxffvklpk6digsXLqj6s9JqtXjooYcAAImJ\nifDx8cGlS5dUzVReXo4HH3wQQOffv/LycgDuvXdeXyS8aW2FEAILFixAdHQ0lixZ4jifkZGBTZs2\nAQA2bdrkKB5KKCgogM1mg8ViwdatW/GjH/0ImzdvVjXTHXfcgZCQENTU1AAA9u7di4kTJyI9PV21\nTDqdDqWlpfjf//4HIQT27t2L6OhoVTN1kXqvMjIysHXrVtjtdlgsFtTW1mLatGmK5TKZTFi9ejWM\nRiOGDh3aLa8auWJiYnDhwgVYLBZYLBZotVpUVVUhKChI1Z/V7NmzsX//fgBATU0N7HY7br/9dlUz\nRUZG4uDBgwCA/fv3IyoqCoCb752Hx09ksWvXLhEVFSUiIiJEQUGBajkOHz4sNBqNiI2NFXFxcSIu\nLk7s3r1b1NfXi+nTp6s6BVYIIcxms2N2k9qZjh8/LhISErpNn1Q7U1FRkWMK7Pz584Xdblc8U2Zm\nprjzzjuFn5+f0Gq14q233rphhldeeUVERESI8ePHO2arKJFrw4YNIjIyUtx1112O3/Vnn31W0Vxd\nmfz9/R0/q+uFh4d3mwKrVia73S4ef/xxMWnSJBEfHy8OHDigSqbrf6eOHj0qpk2bJmJjY8W9994r\nqqqq3M6kEYJ7XRARUc+8vruJiIjUwyJBRESSWCSIiEgSiwQREUlikaABx8fHB0uXLnUcr1mzptsC\nLE8oLi7GlClTMGXKFPj7+2Py5MmYMmUKXnjhBY+2s2nTJnz11VcevSZRb7BI0IDj7++Pd999F/X1\n9QDkWeWanZ2NY8eO4dixYwgODobZbMaxY8dQUFDg0XY2btyIc+fOefSaRL3BIkEDjp+fHxYtWoTX\nXnvte1976qmnsH37dsfx8OHDAQBmsxkpKSmYPXs2IiIisGLFCmzevBnTpk3D5MmT8cUXX9ywTSEE\nVq9ejXXr1gHo3E9r+vTpADoXMz3++OMAgD179iApKQlTp07F3Llz0dLSAgCorKyEXq9HQkICfvzj\nH+P8+fPYtm0bKioq8NhjjyE+Ph5Xr17FihUrMHHiRMTGxmLZsmV9/2EROcEiQQPS4sWL8fbbb6Op\nqanb+e/eVVx//Omnn+JPf/oTTp8+jc2bN+PMmTMoLy/HwoULHf/4S9FoNLj//vtx+PBhAJ3bgLS0\ntKCtrQ2HDx9GSkoKLl26hFdeeQX79u1DZWUlpk6dildffRVtbW3Iy8vD9u3bUVFRgezsbPz617/G\nnDlzkJCQgC1btqCqqgotLS147733cOrUKZw4cQK//e1vPfTTIpKm6DOuiZQSEBCA+fPnY+3atRg2\nbJhL35OYmOjYjTUyMhKpqakAgEmTJuHAgQNOvz8+Ph6VlZVobm7G0KFDkZCQgIqKCvzzn//EunXr\nUFpaiurqaiQlJQEA7HY7kpKS8O9//xunTp3CjBkzAADt7e0YO3as47pd611HjhyJoUOHYsGCBZg1\naxZmzZrl+g+EyE0sEjRgLVmyBPHx8cjOznac8/X1RUdHBwCgo6MDdrvd8bUhQ4Y4Pvfx8XEc+/j4\noK2tzWl7fn5+CA8Px8aNG5GUlITJkydj//79+Pzzz6HT6fD5559j5syZ2LJlS7fvO3nyJCZOnIgj\nR470eN2uux1fX1+Ul5dj37592LZtG9avX499+/a5+NMgcg+7m2jAGjVqFObOnYsNGzY4/qENCwtD\nZWUlAGDHjh1obW31aJvJyclYs2YNUlJSkJycjDfffBPx8fEAgHvuuQcff/wxzpw5AwBoaWlBbW0t\ndDodvv76a5SWlgLo3Gm4uroaQOcdUVeXWUtLCxobG5GWloZXX30VJ06c8Gh2op6wSNCAc/04w3PP\nPYdLly45jp9++mkcPHgQcXFxKC0tdQxcf/f7vns9V5/x8IMf/ADnz5/Hfffdh8DAQAwbNgzJyckA\ngDFjxmDjxo3IyspCbGyso6vJz88P27Ztw/LlyxEXF4cpU6bgk08+AdA50P7MM88gPj4ezc3NSE9P\nR2xsLJKTk3scmCfyNG7wR0REkngnQUREklgkiIhIEosEERFJYpEgIiJJLBJERCSJRYKIiCT9P5mJ\n/sxT+Yz6AAAAAElFTkSuQmCC\n",
       "text": [
        "<matplotlib.figure.Figure at 0x21945f50>"
       ]
      }
     ],
     "prompt_number": 9
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "Although we could filter the DataFrame for coordinates (or locations in user profiles), an even simpler starting point to gain rudimentary insight about where users *might* be located is to inspect the language field of the tweets and compute the tallies for each language. With pandas, it's just a quick one-liner."
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "# What languages do authors of tweets speak? This might be a useful clue\n",
      "# as to who is tweeting. (Also bear in mind the general timeframe for the \n",
      "# data when interpreting these results.)\n",
      "\n",
      "df.lang.value_counts()"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 10,
       "text": [
        "en     79151\n",
        "ja     35940\n",
        "und     3197\n",
        "es      2713\n",
        "de      1142\n",
        "fr       717\n",
        "id       442\n",
        "pt       434\n",
        "ko       283\n",
        "vi       248\n",
        "nl       212\n",
        "th       209\n",
        "zh       135\n",
        "sk       114\n",
        "ru        84\n",
        "da        73\n",
        "it        65\n",
        "sl        65\n",
        "pl        64\n",
        "ht        63\n",
        "et        56\n",
        "tr        53\n",
        "tl        43\n",
        "ar        38\n",
        "lt        30\n",
        "no        17\n",
        "lv        16\n",
        "fi        15\n",
        "hu        13\n",
        "sv        12\n",
        "bg         8\n",
        "ne         7\n",
        "el         5\n",
        "he         5\n",
        "fa         4\n",
        "uk         3\n",
        "my         2\n",
        "is         2\n",
        "ta         1\n",
        "dtype: int64"
       ]
      }
     ],
     "prompt_number": 10
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "A staggering number of Japanese speakers were talking about \"Amazon\" at the time the data was collected. Bearing in mind that it was already mid-day on Monday in Japan when it the news of the Amazon drones started to surface in the United States on Sunday evening, is this really all that surprising given Twitter's popularity in Japan?\n",
      "\n",
      "Filtering on language also affords us to remove some noise from analysis since we can filter out only tweets in a specific language for inspection, which will be handy for some analysis on the content of the tweets themselves. Let's filter out only the 140 characters of text from tweets where the author speaks English and use some natural language processing techniques to learn more about the reaction."
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "# Let's just look at the content of the English tweets by extracting it\n",
      "# out as a list of text\n",
      "\n",
      "en_text = df[df['lang'] == 'en'].pop('text')"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [],
     "prompt_number": 11
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "Although NLTK provides some advanced tokenization functions, let's just split the English text on white space, normalize it to lowercase, and remove some common trailing punctuation and count things to get an initial glance in to what's being talked about."
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "from collections import Counter\n",
      "\n",
      "tokens = []\n",
      "for txt in en_text.values:\n",
      "    tokens.extend([t.lower().strip(\":,.\") for t in txt.split()])\n",
      "    \n",
      "# Use a Counter to construct frequency tuples\n",
      "tokens_counter = Counter(tokens)\n",
      "\n",
      "# Display some of the most commonly occurring tokens\n",
      "tokens_counter.most_common(50)"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 12,
       "text": [
        "[(u'amazon', 54778),\n",
        " (u'rt', 36409),\n",
        " (u'the', 25749),\n",
        " (u'drones', 24474),\n",
        " (u'to', 24040),\n",
        " (u'a', 21341),\n",
        " (u'delivery', 18557),\n",
        " (u'in', 17531),\n",
        " (u'of', 15741),\n",
        " (u'on', 14095),\n",
        " (u'drone', 13800),\n",
        " (u'by', 13422),\n",
        " (u'is', 12034),\n",
        " (u'for', 10988),\n",
        " (u'@amazon', 9318),\n",
        " (u'i', 9263),\n",
        " (u'and', 8793),\n",
        " (u'prime', 8783),\n",
        " (u'30', 8319),\n",
        " (u'air', 8026),\n",
        " (u'with', 7956),\n",
        " (u'future', 7911),\n",
        " (u'deliver', 7890),\n",
        " (u'get', 6790),\n",
        " (u'you', 6573),\n",
        " (u'your', 6543),\n",
        " (u'via', 6444),\n",
        " (u'deliveries', 6432),\n",
        " (u'this', 5899),\n",
        " (u'bezos', 5738),\n",
        " (u'will', 5703),\n",
        " (u'#primeair', 5680),\n",
        " (u'unmanned', 5442),\n",
        " (u'aerial', 5313),\n",
        " (u'under', 5308),\n",
        " (u'-', 5257),\n",
        " (u'mins', 5199),\n",
        " (u'that', 4890),\n",
        " (u'vehicles', 4835),\n",
        " (u'my', 4728),\n",
        " (u'from', 4720),\n",
        " (u'peek', 4699),\n",
        " (u'sneak', 4684),\n",
        " (u'unveils', 4555),\n",
        " (u'it', 4473),\n",
        " (u'minutes', 4459),\n",
        " (u'just', 4396),\n",
        " (u'at', 4394),\n",
        " (u'http://t.c\\u2026', 4391),\n",
        " (u'packages', 4302)]"
       ]
      }
     ],
     "prompt_number": 12
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "Not surprisingly, \"amazon\" is the most frequently occurring token, there are lots of retweets (actually, \"quoted retweets\") as evidenced by \"rt\", and lots of stopwords (commonly occurring words like \"the\", \"and\", etc.) at the top of the list. Let's further remove some of the noise by removing stopwords."
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "import nltk\n",
      "\n",
      "# Download the stopwords list into NLTK\n",
      "\n",
      "nltk.download('stopwords')\n",
      "\n",
      "# Remove stopwords to decrease noise\n",
      "for t in nltk.corpus.stopwords.words('english'):\n",
      "    tokens_counter.pop(t)\n",
      "    \n",
      "# Redisplay the data (and then some)\n",
      "tokens_counter.most_common(200)"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "output_type": "stream",
       "stream": "stdout",
       "text": [
        "[nltk_data] Downloading package 'stopwords' to /usr/share/nltk_data...\n",
        "[nltk_data]   Package stopwords is already up-to-date!\n"
       ]
      },
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 13,
       "text": [
        "[(u'amazon', 54778),\n",
        " (u'rt', 36409),\n",
        " (u'drones', 24474),\n",
        " (u'delivery', 18557),\n",
        " (u'drone', 13800),\n",
        " (u'@amazon', 9318),\n",
        " (u'prime', 8783),\n",
        " (u'30', 8319),\n",
        " (u'air', 8026),\n",
        " (u'future', 7911),\n",
        " (u'deliver', 7890),\n",
        " (u'get', 6790),\n",
        " (u'via', 6444),\n",
        " (u'deliveries', 6432),\n",
        " (u'bezos', 5738),\n",
        " (u'#primeair', 5680),\n",
        " (u'unmanned', 5442),\n",
        " (u'aerial', 5313),\n",
        " (u'-', 5257),\n",
        " (u'mins', 5199),\n",
        " (u'vehicles', 4835),\n",
        " (u'peek', 4699),\n",
        " (u'sneak', 4684),\n",
        " (u'unveils', 4555),\n",
        " (u'minutes', 4459),\n",
        " (u'http://t.c\\u2026', 4391),\n",
        " (u'packages', 4302),\n",
        " (u'jeff', 4040),\n",
        " (u'http://t.co/w6kugw4egt', 3922),\n",
        " (u\"amazon's\", 3669),\n",
        " (u'flying', 3599),\n",
        " (u'ceo', 3205),\n",
        " (u'#amazon', 3074),\n",
        " (u'new', 2870),\n",
        " (u'free', 2797),\n",
        " (u'testing', 2585),\n",
        " (u'could', 2568),\n",
        " (u'shipping', 2541),\n",
        " (u'', 2422),\n",
        " (u'says', 2343),\n",
        " (u\"'60\", 2324),\n",
        " (u'like', 2300),\n",
        " (u'stuff', 2263),\n",
        " (u'years', 2194),\n",
        " (u'60', 2157),\n",
        " (u'use', 2134),\n",
        " (u'using', 1939),\n",
        " (u'&amp;', 1901),\n",
        " (u\"minutes'\", 1868),\n",
        " (u'kindle', 1735),\n",
        " (u\"it's\", 1657),\n",
        " (u'plans', 1655),\n",
        " (u'cyber', 1622),\n",
        " (u'one', 1617),\n",
        " (u'gift', 1614),\n",
        " (u\"i'm\", 1604),\n",
        " (u'monday', 1568),\n",
        " (u'wants', 1538),\n",
        " (u'first', 1522),\n",
        " (u'order', 1519),\n",
        " (u'good', 1479),\n",
        " (u'going', 1459),\n",
        " (u'package', 1446),\n",
        " (u'fire', 1400),\n",
        " (u'look', 1386),\n",
        " (u'plan', 1378),\n",
        " (u'4', 1377),\n",
        " (u'delivering', 1376),\n",
        " (u'@60minutes', 1371),\n",
        " (u'make', 1369),\n",
        " (u'experimenting', 1341),\n",
        " (u'30-minute', 1336),\n",
        " (u'book', 1330),\n",
        " (u'primeair', 1310),\n",
        " (u'real', 1285),\n",
        " (u'online', 1274),\n",
        " (u'coming', 1261),\n",
        " (u'think', 1195),\n",
        " (u'see', 1152),\n",
        " (u'video', 1149),\n",
        " (u'next', 1149),\n",
        " (u'would', 1135),\n",
        " (u'system', 1131),\n",
        " (u'service', 1115),\n",
        " (u'thing', 1099),\n",
        " (u'something', 1069),\n",
        " (u'hour', 1052),\n",
        " (u'black', 1043),\n",
        " (u'card', 1040),\n",
        " (u'half', 1033),\n",
        " (u'want', 1018),\n",
        " (u'half-hour', 1016),\n",
        " (u'futuristic', 1016),\n",
        " (u\"you're\", 998),\n",
        " (u'know', 987),\n",
        " (u'love', 985),\n",
        " (u'people', 964),\n",
        " (u'aims', 964),\n",
        " (u'(video)', 958),\n",
        " (u'day', 954),\n",
        " (u'shot', 936),\n",
        " (u'deploy', 921),\n",
        " (u'delivered', 919),\n",
        " (u'amazon\\u2019s', 906),\n",
        " (u'basically', 902),\n",
        " (u'within', 888),\n",
        " (u'shop', 886),\n",
        " (u'really', 882),\n",
        " (u'buy', 876),\n",
        " (u'check', 859),\n",
        " (u'\\u2026', 855),\n",
        " (u'us', 844),\n",
        " (u'time', 829),\n",
        " (u'autonomous', 817),\n",
        " (u'wait', 815),\n",
        " (u'right', 801),\n",
        " (u'@mashable', 787),\n",
        " (u'finding', 786),\n",
        " (u'go', 780),\n",
        " (u'2015', 779),\n",
        " (u\"can't\", 774),\n",
        " (u'@buzzfeed', 774),\n",
        " (u'top', 774),\n",
        " (u'cool', 770),\n",
        " (u'rebel', 767),\n",
        " (u'@amazondrone', 766),\n",
        " (u'(also', 762),\n",
        " (u'helpful', 762),\n",
        " (u'#drones', 761),\n",
        " (u'rifle', 759),\n",
        " (u'reveals', 759),\n",
        " (u'door', 755),\n",
        " (u'bases', 752),\n",
        " (u'store', 751),\n",
        " (u'hoth.)', 748),\n",
        " (u'shit', 748),\n",
        " (u'@bradplumer', 743),\n",
        " (u'waiting', 743),\n",
        " (u'looks', 735),\n",
        " (u'@deathstarpr', 735),\n",
        " (u\"don't\", 732),\n",
        " (u'5', 731),\n",
        " (u'win', 731),\n",
        " (u'floats', 717),\n",
        " (u'friday', 717),\n",
        " (u'way', 717),\n",
        " (u'great', 713),\n",
        " (u'http://t.co/jlfdnihzks', 711),\n",
        " (u'company', 710),\n",
        " (u'need', 709),\n",
        " (u'read', 704),\n",
        " (u'home', 704),\n",
        " (u'watch', 697),\n",
        " (u'moment', 691),\n",
        " (u'http://t.co/bxsavvzxzf', 690),\n",
        " (u'best', 685),\n",
        " (u'notion', 680),\n",
        " (u'news', 669),\n",
        " (u'blog', 669),\n",
        " (u'announces', 667),\n",
        " (u'got', 658),\n",
        " (u'$25', 654),\n",
        " (u'products', 646),\n",
        " (u'big', 645),\n",
        " (u'still', 642),\n",
        " (u'2', 642),\n",
        " (u'gonna', 642),\n",
        " (u'tip', 636),\n",
        " (u'sales', 623),\n",
        " (u'awkward', 618),\n",
        " (u'\"amazon', 617),\n",
        " (u'idea', 609),\n",
        " (u'take', 604),\n",
        " (u'working', 600),\n",
        " (u'books', 597),\n",
        " (u\"won't\", 597),\n",
        " (u'hovers', 593),\n",
        " (u'wow', 589),\n",
        " (u'live', 587),\n",
        " (u'promises', 579),\n",
        " (u'back', 576),\n",
        " (u'package-delivery', 573),\n",
        " (u'@badbanana', 570),\n",
        " (u'soon', 563),\n",
        " (u'deals', 560),\n",
        " (u'+', 558),\n",
        " (u'work', 555),\n",
        " (u'ever', 552),\n",
        " (u\"'octocopter'\", 551),\n",
        " (u'$50', 549),\n",
        " (u'hit', 549),\n",
        " (u'holy', 546),\n",
        " (u'night', 537),\n",
        " (u'hdx', 535),\n",
        " (u'today', 526),\n",
        " (u'bits', 521),\n",
        " (u'many', 520),\n",
        " (u'awesome', 519),\n",
        " (u'amazing', 508),\n",
        " (u'window', 506)]"
       ]
      }
     ],
     "prompt_number": 13
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "What a difference removing a little bit of noise can make! We now see much more meaningful data appear at the top of the list: drones, signs that a phrase \"30 mins\" (which turned out to be a possible timeframe for a Prime Air delivery by a drone according to Bezos) might appear based the appearance of \"30\" and \"mins\"/\"minutes\" near the top of the list), signs of another phrase \"prime air\" (as evidenced by \"prime\", \"air\" and the hashtag \"#primeair\"), references to Jeff Bezos, URLs to investigate and more!\n",
      "\n",
      "Even though we've already learned a lot, one of the challenges with only employing crude tokenization techniques is that you aren't left with any phrases. One of the simplest ways of disocvering meaningful phrases in text is to treat the problem as one of discovering statistical collocations. NLTK provides some routines to find collocations and includes a \"demo\" function that's a quick one-liner."
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "nltk_text = nltk.Text(tokens)\n",
      "nltk_text.collocations()"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "output_type": "stream",
       "stream": "stdout",
       "text": [
        "Building collocations list\n",
        "prime air; sneak peek; unmanned aerial; aerial vehicles;\n",
        "http://t.co/w6kugw4egt http://t.c\u2026; vehicles http://t.co/w6kugw4egt;\n",
        "#primeair future; future deliveries; delivery drones; jeff bezos;\n",
        "@amazon get; amazon prime; '60 minutes'; amazon unveils; cyber monday;\n",
        "deliver packages; flying delivery; unveils flying; kindle fire; (also\n",
        "helpful"
       ]
      },
      {
       "output_type": "stream",
       "stream": "stdout",
       "text": [
        "\n"
       ]
      }
     ],
     "prompt_number": 14
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "Even without any prior analysis on tokenization, it's pretty clear what the topis is about as evidenced by this list of collocations. But what about the *context* in which these phrases appear? As it turns out, NLTK supplies another handy data structure that provides some insight as to how words appear in context called a concordance. Trying out the \"demo functionality\" for the concordance is as simple as just calling it as shown below. \n",
      "\n",
      "Toward the bottom of the list of commonly occurring tokens, the words \"amazing\" and \"holy\" appear. The word \"amazing\" is interesting, because it is usually the basis of an emotional reaction, and we're interested in examining the reaction. What about word \"holy\"? What might it mean? The concordance will help us to find out..."
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "nltk_text.concordance(\"amazing\")\n",
      "print\n",
      "nltk_text.concordance(\"holy\")"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "output_type": "stream",
       "stream": "stdout",
       "text": [
        "Building index...\n",
        "Displaying 25 of 508 matches:"
       ]
      },
      {
       "output_type": "stream",
       "stream": "stdout",
       "text": [
        "\n",
        "s - @variety http://t.c\u2026 this looks amazing how will it impact drone traffic? -\n",
        "it? amazon prime air delivery looks amazing http://t.co/icizw5vfrw rt @jeffreyg\n",
        "gift card? @budgetearth &amp; other amazing bloggers are giving one away! ends \n",
        "k? damn that amazon prime air looks amazing im sure it would freak out some peo\n",
        "egt http://t.c\u2026 @munilass amazon is amazing for what i use it for i'm okay with\n",
        "wwglyqrq just in bonnie sold again! amazing book http://t.co/jce902iros #best-s\n",
        "ase of 1000) http://t.co/d6l8p3jgbz amazing prospects! \u201c@brianstelter on heels \n",
        "riety http://t.c\u2026 rt @dangillmor by amazing coincidence amazon had a youtube dr\n",
        "rd_ferguson amazon prime air sounds amazing *hot* kindle fire hdx wi-fi 7' tabl\n",
        "t.co/hrgxtrlumx this is going to be amazing if the faa allows it welcome to the\n",
        "lying grill #primeair is absolutely amazing i'm excited to see the progress/dev\n",
        ".co/w6kugw4egt http://t.c\u2026 the most amazing thing to me about amazon - when bez\n",
        "//t.co/cad9zload3 rt @dangillmor by amazing coincidence amazon had a youtube dr\n",
        "that 60 minutes piece on amazon was amazing what an incredible company and deli\n",
        " jesus christ this is real and it\u2019s amazing erohmygerd http://t.co/m4salqm0lo r\n",
        "/t.co/0trwr9qsoc rt @semil the most amazing thing to me about amazon - when bez\n",
        "yeah no this @amazon drone stuff is amazing me but i have the same questions as\n",
        "hqfg\u2026 30 minutes delivery by amazon amazing http://t.co/ofu39suiov i really don\n",
        "eat show at #60minutes amazon is an amazing company! rt @zachpagano next year d\n",
        "on's future drone shipping tech was amazing to see amazon unveils futuristic pl\n",
        "the first review on this product is amazing http://t.co/20yn3jguxu rt @amazon g\n",
        "ttp://t.co/s2shyp48go this would be amazing  jeff bezos promises half-hour ship\n",
        "wugrxq2oju have you guys seen these amazing steals on amazon?? wow!! some of my\n",
        "ttp://t.co/mhqfg\u2026 rt @dangillmor by amazing coincidence amazon had a youtube dr\n",
        "bezo http://t.co/2jt6pgn8an this is amazing rt @rdizzle7 rt @marquisgod dog rt \n",
        "\n",
        "Displaying 25 of 546 matches:\n",
        " @brocanadian http://t.co/zxyct2renf holy shit rt @amazon get a sneak peek of \n",
        "our shipping with amazon prime air - holy cow this is the future - http://t.co\n",
        "eo) http://t.co/hi3gviwto7 #technews holy shit wtf http://t.co/p3h2wn5pba awes\n",
        "es'! (other 1 suggested was usa 1!!) holy shit jeff bezos promises half-hour s\n",
        "es http://t.co/k\u2026 rt @joshuatopolsky holy shit jeff bezos promises half-hour s\n",
        "//t.co/tjdtdpkaaf rt @joshuatopolsky holy shit jeff bezos promises half-hour s\n",
        "//t.co/0gpvgsyatm rt @joshuatopolsky holy shit jeff bezos promises half-hour s\n",
        " when amazon prime air is available? holy shit very funny! @amazon rt tim sied\n",
        "w4egt http://t.c\u2026 rt @joshuatopolsky holy shit jeff bezos promises half-hour s\n",
        "drones rt @alexpenn amazon prime air holy shit http://t.co/g2b7dumgbl amazon i\n",
        "ijk0 via @oliryan rt @joshuatopolsky holy shit jeff bezos promises half-hour s\n",
        "s http://t.co/w6kugw4egt http://t.c\u2026 holy shit amazon what? https://t.co/qrhkx\n",
        "w4egt http://t.c\u2026 rt @joshuatopolsky holy shit jeff bezos promises half-hour s\n",
        "//t.co/zggekdoyhv rt @joshuatopolsky holy shit jeff bezos promises half-hour s\n",
        " me for free? http://t.co/euutqyuoox holy crap @60minutes by using drones amaz\n",
        "//t.c\u2026 rt @alexpenn amazon prime air holy shit http://t.co/g2b7dumgbl amazon i\n",
        "one #primeair http://t.co/jgflwmcth5 holy fucking shit this is badass watch th\n",
        "d in a back yard? rt @joshuatopolsky holy shit jeff bezos promises half-hour s\n",
        " @brocanadian http://t.co/zxyct2renf holy shit of course! \u201c@verge delivery dro\n",
        "how many lawyers\u2026 rt @joshuatopolsky holy shit jeff bezos promises half-hour s\n",
        "#business #market rt @joshuatopolsky holy shit jeff bezos promises half-hour s\n",
        "ke from amazon ;d rt @joshuatopolsky holy shit jeff bezos promises half-hour s\n",
        " fan of commercials rt @maryforbes14 holy crap @60minutes by using drones amaz\n",
        "//t.co/lefbeec5ht rt @joshuatopolsky holy shit jeff bezos promises half-hour s\n",
        "g each other down rt @joshuatopolsky holy shit jeff bezos promises half-hour s\n"
       ]
      }
     ],
     "prompt_number": 15
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "It would appear that there is indeed a common thread of amazement in the data, and it's evident that @joshuatopolsky (who turns out to be Editor-in-chief of <a href=\"http://www.theverge.com/\">The Verge</a>) is a commonly occurring tweet entity that warrants further investigation. Speaking of tweet entities, let's take an initial look at usernames, hashtags, and URLs by employing a simple heuristic to look for words prefixed with @, RT, #, and http to see what some of the most commonly occurring tweet entiteis are in the data."
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "# An crude look at tweet entities\n",
      "\n",
      "entities = []\n",
      "for txt in en_text.values:\n",
      "    for t in txt.split():\n",
      "        if t.startswith(\"http\") or t.startswith(\"@\") or t.startswith(\"#\") or t.startswith(\"RT @\"):\n",
      "            if not t.startswith(\"http\"):\n",
      "                t = t.lower()\n",
      "            entities.append(t.strip(\" :,\"))\n",
      "\n",
      "entities_counter = Counter(entities)\n",
      "for entity, freq in entities_counter.most_common()[:100]:\n",
      "    print entity, freq"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "output_type": "stream",
       "stream": "stdout",
       "text": [
        "@amazon 8994\n",
        "#primeair. 4767\n",
        "http://t.c\u2026 4391\n",
        "http://t.co/w6kugw4EGt 3922\n",
        "#amazon 3032\n",
        "@60minutes 1325\n",
        "#primeair 911\n",
        "@mashable 787\n",
        "@buzzfeed 774\n",
        "@amazondrone 743\n",
        "@bradplumer 743\n",
        "@deathstarpr 735\n",
        "#drones 729\n",
        "http://t.co/JlFdNiHzks 711\n",
        "http://t.co/BxSAVVzXZf 690\n",
        "@badbanana 570\n",
        "#kindle 467\n",
        "@thenextweb 458\n",
        "#amexamazon 441\n",
        "http://t.co/MHqFG\u2026 434\n",
        "#giveaway 421\n",
        "http:/\u2026 417\n",
        "#win 409\n",
        "http:\u2026 406\n",
        "@techcrunch 391\n",
        "#drone 383\n",
        "#60minutes 380\n",
        "http://t\u2026 357\n",
        "#tech 342\n",
        "@levie 340\n",
        "@variety 337\n",
        "@breakingnews 331\n",
        "@youtube 326\n",
        "#cybermonday 325\n",
        "@huffposttech 322\n",
        "http://\u2026 320\n",
        "@jonrussell 304\n",
        "@realjohngreen 300\n",
        "#news 298\n",
        "http://t.co/FNndPuMouA 294\n",
        "@washingtonpost 284\n",
        "@kotaku 283\n",
        "@usatoday 283\n",
        "http://t.\u2026 280\n",
        "#amazondrones 278\n",
        "@nycjim 277\n",
        "http://t.co/NG8\u2026 270\n",
        "http://t.co/rUu14XpvGo 270\n",
        "@brianstelter 268\n",
        "@majornelson 260\n",
        "@benbadler 258\n",
        "http://t.co/M7kqd26jVR 255\n",
        "http\u2026 254\n",
        "@businessinsider 249\n",
        "@huffingtonpost 245\n",
        "http://t.co/DOEjXCC1vL 241\n",
        "@sai 241\n",
        "http://t.co/\u2026 240\n",
        "@verge 237\n",
        "http://t.co/tAVunIXxo3 230\n",
        "http://t.co/OqAZXaZdh9 223\n",
        "http://t.co/sMBvNrasSN 221\n",
        "#amazonprimeair 221\n",
        "@buzzfeednews 214\n",
        "@mattsinger 211\n",
        "@ericstangel 211\n",
        "#1 205\n",
        "@byjasonng 200\n",
        "#free 198\n",
        "http://t.co/GQCvN0xZ7n 193\n",
        "@americanexpress 190\n",
        "#csrracing 183\n",
        "@nickkristof 178\n",
        "@orgasmicgomez 170\n",
        "http://t.co/REJl0wLImt 168\n",
        "http://t.co/zUWaeRjFC8 167\n",
        "#ebook 165\n",
        "http://t.co/4zk01TUugX 165\n",
        "@joshuatopolsky 161\n",
        "@percival 161\n",
        "@lanceulanoff 160\n",
        "@time 158\n",
        "http://t.co/xQCjG6VQyb 157\n",
        "#romance 154\n",
        "#technology 154\n",
        "#rt 148\n",
        "@engadget 145\n",
        "@arstechnica 142\n",
        "@sapnam 142\n",
        "http://t.co/JxBSx8rLBZ 141\n",
        "http://t.co/IyV1qBhtJg 141\n",
        "@youranonnews 139\n",
        "@gizmodo 138\n",
        "@abc 135\n",
        "@mckaycoppins 133\n",
        "http://t.co/zGgEkdOyhv 133\n",
        "http://t.co/9ZBYtKpHce 132\n",
        "@newsbreaker 132\n",
        "http://t.co/z5JQkD4svO 132\n",
        "@ 131\n"
       ]
      }
     ],
     "prompt_number": 16
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "As you can see, there are lots of intersting tweet entities that give you helpful context for the announcement. One particularly notable observation is the appearance of \"comedic accounts\" such as @deathstarpr and @amazondrone near the top of the list, relaying a certain amount of humor. The <a href=\"https://twitter.com/DeathStarPR/status/407335634940878848\">tweet</a> embedded below that references *Star Wars* was eventually retweeted over 1k times in response to the announcement! It wouldn't be difficult to determine how many retweets occurred just within the ~3 hour timeframe corresponding to the dataset we're using here.\n",
      "\n",
      "<center>\n",
      "<blockquote class=\"twitter-tweet\" lang=\"en\"><p>First look at Amazon&#39;s new delivery drone. (Also helpful for finding Rebel bases on Hoth.) <a href=\"http://t.co/JlFdNiHzks\">pic.twitter.com/JlFdNiHzks</a></p>&mdash; Death Star PR (@DeathStarPR) <a href=\"https://twitter.com/DeathStarPR/statuses/407335634940878848\">December 2, 2013</a></blockquote>\n",
      "<script async src=\"//platform.twitter.com/widgets.js\" charset=\"utf-8\"></script>\n",
      "</center>\n",
      "\n",
      "When you take a closer look at some of the developed news stories, you also see sarcasm, unbelief, and even a bit of frustration about this being a \"publicity stunt\" for Cyber Monday.\n",
      "\n",
      "Note: There proper way of parsing out tweet entities from the *entities* field that you can see in the DataFrame. It's marginally more work but has the primary advantage that you can see the \"expanded URL\" which provides better insight into the nature of the URL since you'll know its domain name. See <a href=\"http://nbviewer.ipython.org/github/ptwobrussell/Mining-the-Social-Web-2nd-Edition/blob/master/ipynb/Chapter%209%20-%20Twitter%20Cookbook.ipynb#Example-10.-Extracting-tweet-entities\">Example 9-10, Extracting Tweet Entities</a> from *Mining the Social Web* for more on how to do that."
     ]
    },
    {
     "cell_type": "heading",
     "level": 2,
     "metadata": {},
     "source": [
      "Summarize"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "We *aspired* to learn more about the general reaction to Amazon's announcement about Prime Air by taking an initial look at the data from Amazon's firehose, and it's fair to say that we learned a few things about the data without too much effort. Lots more could be discovered, but a few of the themes that we were able to glean included:\n",
      "\n",
      "* Amazement\n",
      "* Humor\n",
      "* Disbelief\n",
      "\n",
      "Although these reactions aren't particularly surprising for such an outrageous announcement, you've hopefully learned enough that you could tap into Twitter's firehose to capture and analyze data that's of interest to you. There is no shortage of fun to be had, and as you've learned, it's easier than it might first appear.\n",
      "\n",
      "Enjoy!"
     ]
    },
    {
     "cell_type": "heading",
     "level": 2,
     "metadata": {},
     "source": [
      "Recommended Resources"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "If you enjoy analyzing data from social websites like Twitter, then you might enjoy the book *Mining the Social Web, 2nd Edition* (O'Reilly). You can learn more about it at <a href=\"http://MiningTheSocialWeb.com\">MiningTheSocialWeb.com</a>. All source code is available in IPython Notebook format at <a href=\"http://bit.ly/MiningTheSocialWeb2E\">GitHub</a> and can be previewed in the <a href=\"http://bit.ly/1cOlK6q\">IPython Notebook Viewer</a>.\n",
      "\n",
      "The book itself is a form of \"premium support\" for the source code and is available for purchase from <a href=\"http://amzn.to/GPd59m\">Amazon</a> or <a href=\"http://bit.ly/135dHfs\">O'Reilly Media</a>."
     ]
    }
   ],
   "metadata": {}
  }
 ]
}