{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "# Working with Data APIs\n", "\n", "**Sam Maurer // DCRP PhD Student // maurer@berkeley.edu // Oct. 7, 2015 // CP 255 demo**\n", "\n", "This notebook provides a demonstration of data-access APIs that operate over the web.\n", "\n", "In Part 1, we'll load and parse data from an automated USGS feed of earthquakes. In Part 2, we'll add query parameters to the workflow, using the Google Maps Geolocation API as an example. In Part 3, we'll use authenticated APIs to access (public) Twitter data. " ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Setup (not required until Part 3)\n", "\n", "1. **From the command line, install this Python package for connecting to Twitter APIs:** \n", " `pip install TwitterAPI` \n", " \n", " There are several different package managers for Python. We're using \"pip install\" instead of \"conda install\" because Anaconda doesn't include this package in its index. The lab computers should already have TwitterAPI installed. \n", "  \n", "\n", "2. **Make sure `keys.py` is in the same directory as this notebook** \n", " \n", " `keys.py` (provided separately) contains demo authentication keys for the Twitter APIs. For folks not taking the class, you can edit the `keys-example.py` file, following the instructions in the next step to obtain your own credentials. \n", "  \n", " \n", "3. **OPTIONAL: Sign up for your own Twitter API credentials (5 minutes)** \n", " \n", " Twitter limits the number of simultaneous connections from a single account, so if you're willing to sign up for your own credentials, it will help the in-class demo go smoother! \n", " \n", " * Log into Twitter or create an account: http://twitter.com \n", "  \n", " \n", " * Register a new app development project: https://dev.twitter.com/apps/new \n", "   \n", " (The form is geared toward people making smartphone or web apps, but you still have to fill it out... You can call the app an in-class demo and give the URL of your own Twitter page, for example) \n", "  \n", " \n", " * Submit the form, go to the \"Keys and Access Tokens\" tab, and click on \"Create my access token\" at the bottom of the page \n", "  \n", " \n", " * Copy these four codes into the `keys.py` file, replacing the demo credentials: \n", " (a) consumer key, (b) consumer secret, (c) access token, (d) access token secret \n", "  \n", " \n", " * All done! Try running the first few code blocks in Part 3 to see if it worked\n", " \n", "----- \n", "\n", "-----" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "# Part 1: Reading from an automated data feed\n", "\n", "### USGS real-time earthquake feeds\n", "\n", "This is an API for near-real-time data on earthquakes. Results are provided in JSON format over the web. No authentication is needed, and rather than accepting queries, the API has a separate endpoint for each permutation of the data that users might want.\n", "\n", "**API documentation:** \n", "http://earthquake.usgs.gov/earthquakes/feed/v1.0/geojson.php\n", "\n", "**Sample API endpoint, for magnitude 4.5+ earthquakes in past day:** \n", "http://earthquake.usgs.gov/earthquakes/feed/v1.0/summary/4.5_day.geojson \n" ] }, { "cell_type": "code", "execution_count": 1, "metadata": { "collapsed": true }, "outputs": [], "source": [ "%matplotlib inline\n", "\n", "import pandas as pd\n", "import urllib\n", "import json" ] }, { "cell_type": "code", "execution_count": 2, "metadata": { "collapsed": false }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "{\"type\":\"FeatureCollection\",\"metadata\":{\"generated\":1444158814000,\"url\":\"http://earthquake.usgs.gov/earthquakes/feed/v1.0/summary/2.5_week.geojson\",\"title\":\"USGS Magnitude 2.5+ Earthquakes, Past Week\",\"status\":200,\"api\":\"1.0.17\",\"count\":255},\"features\":[{\"type\":\"Feature\",\"properties\":{\"mag\":2.96,\"place\":\"1km N of The Geysers, California\",\"time\":1444157633260,\"updated\":1444158221836,\"tz\":-420,\"url\":\"http://earthquake.usgs.gov/earthquakes/eventpage/nc72536056\",\"detail\":\"http://earthquake.usgs.gov/\n", "\n" ] } ], "source": [ "# use endpoint for magnitude 2.5+ quakes in past week\n", "endpoint_url = \"http://earthquake.usgs.gov/earthquakes/feed/v1.0/summary/2.5_week.geojson\"\n", "\n", "# open a connection to the URL\n", "connection = urllib.urlopen(endpoint_url)\n", "\n", "# download the results\n", "results = connection.read()\n", "\n", "print results[:500] # first 500 characters\n", "print type(results)" ] }, { "cell_type": "code", "execution_count": 3, "metadata": { "collapsed": false }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "{u'geometry': {u'type': u'Point', u'coordinates': [-122.7580032, 38.7903328, 3.56]}, u'type': u'Feature', u'properties': {u'rms': 0.06, u'code': u'72536056', u'cdi': 2, u'sources': u',nc,', u'nst': 38, u'tz': -420, u'title': u'M 3.0 - 1km N of The Geysers, California', u'magType': u'md', u'detail': u'http://earthquake.usgs.gov/earthquakes/feed/v1.0/detail/nc72536056.geojson', u'sig': 135, u'net': u'nc', u'type': u'earthquake', u'status': u'automatic', u'updated': 1444158221836L, u'felt': 3, u'alert': None, u'dmin': 0.03405, u'mag': 2.96, u'gap': 63, u'types': u',dyfi,focal-mechanism,general-link,geoserve,nearby-cities,origin,phase-data,scitech-link,', u'url': u'http://earthquake.usgs.gov/earthquakes/eventpage/nc72536056', u'ids': u',nc72536056,', u'tsunami': 0, u'place': u'1km N of The Geysers, California', u'time': 1444157633260L, u'mmi': None}, u'id': u'nc72536056'}\n", "\n" ] } ], "source": [ "# the results are a string with JSON-formatted data inside\n", "\n", "# parse the string into a Python data structure\n", "data = json.loads(results)\n", "\n", "print data['features'][0] # first item from the array called 'features'\n", "print type(data)" ] }, { "cell_type": "code", "execution_count": 4, "metadata": { "collapsed": false }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "M 3.0 - 1km N of The Geysers, California\n", "M 4.6 - 43km SE of Sary-Tash, Kyrgyzstan\n", "M 2.8 - 14km S of Volcano, Hawaii\n", "M 2.6 - 10km SE of Chester, California\n", "M 4.2 - 75km SW of Coquimbo, Chile\n", "M 3.5 - 6km WSW of Perry, Oklahoma\n", "M 2.7 - 46km W of Hawi, Hawaii\n", "M 5.1 - 39km WNW of Illapel, Chile\n", "M 2.7 - 3km SSE of Pole Ojea, Puerto Rico\n", "M 4.8 - 58km SE of Ofunato, Japan\n", "M 2.7 - 34km SSE of Redoubt Volcano, Alaska\n", "M 5.1 - 154km SSW of Puerto El Triunfo, El Salvador\n", "M 4.9 - 65km N of Sikabaluan, Indonesia\n", "M 2.8 - 32km SE of Tok, Alaska\n", "M 3.2 - 26km E of Honaunau-Napoopoo, Hawaii\n", "M 4.5 - 122km ENE of Ndoi Island, Fiji\n", "M 4.5 - 18km N of Taltal, Chile\n", "M 2.9 - 66km NNW of Charlotte Amalie, U.S. Virgin Islands\n", "M 4.7 - 89km WSW of Coquimbo, Chile\n", "M 2.9 - 44km NE of Road Town, British Virgin Islands\n", "M 2.6 - 8km WNW of Anza, California\n", "M 2.7 - 9km NNW of Shelbyville, Kentucky\n", "M 4.6 - 84km W of Illapel, Chile\n", "M 4.2 - 70km NW of Rapar, India\n", "M 3.9 - 135km NE of Chignik Lake, Alaska\n", "M 2.7 - 54km ESE of Lovelock, Nevada\n", "M 4.1 - 69km WNW of La Ligua, Chile\n", "M 4.2 - 71km WNW of La Ligua, Chile\n", "M 2.5 - 68km ESE of Lakeview, Oregon\n", "M 2.7 - 68km ESE of Lakeview, Oregon\n", "M 2.6 - 8km SE of Pole Ojea, Puerto Rico\n", "M 2.6 - 23km N of Upper Lake, California\n", "M 2.8 - 23km W of Perry, Oklahoma\n", "M 5.7 - 74km N of Amatignak Island, Alaska\n", "M 4.7 - 158km WSW of Ovalle, Chile\n", "M 5.9 - 43km SSW of Coquimbo, Chile\n", "M 2.9 - 69km ESE of Lakeview, Oregon\n", "M 2.6 - 42km SSW of Larsen Bay, Alaska\n", "M 3.0 - 95km WSW of Healy, Alaska\n", "M 3.4 - 85km N of Road Town, British Virgin Islands\n", "M 2.5 - 54km E of Old Iliamna, Alaska\n", "M 3.0 - 94km WSW of Healy, Alaska\n", "M 2.9 - 83km W of Cantwell, Alaska\n", "M 4.5 - 32km N of Ramechhap, Nepal\n", "M 3.5 - 20km E of Cherokee, Oklahoma\n", "M 2.9 - 81km NNW of Talkeetna, Alaska\n", "M 4.7 - 200km SE of L'Esperance Rock, New Zealand\n", "M 4.1 - 57km WSW of Illapel, Chile\n", "M 4.2 - 47km WSW of Amatignak Island, Alaska\n", "M 3.0 - 48km WSW of Stella, Puerto Rico\n", "M 5.0 - Off the east coast of the North Island of New Zealand\n", "M 4.3 - 102km ESE of Iwaki, Japan\n", "M 4.6 - 43km N of Palue, Indonesia\n", "M 3.8 - 26km SSE of Boca de Yuma, Dominican Republic\n", "M 2.8 - 67km SW of Anchor Point, Alaska\n", "M 4.7 - 124km SSW of Kotaagung, Indonesia\n", "M 4.4 - 20km SE of Jarm, Afghanistan\n", "M 3.0 - 30km SE of Redoubt Volcano, Alaska\n", "M 2.6 - 9km SW of Honoka'a, Hawaii\n", "M 2.8 - 7km NNE of Helena, Oklahoma\n", "M 4.8 - South of the Fiji Islands\n", "M 4.9 - 81km NNE of Obelobel, Indonesia\n", "M 2.6 - 9km W of Templeton, California\n", "M 4.9 - 93km W of Ovalle, Chile\n", "M 4.6 - South of the Fiji Islands\n", "M 4.8 - 146km S of Bambanglipuro, Indonesia\n", "M 3.3 - 24km NE of Fritz Creek, Alaska\n", "M 2.9 - 12km SSW of Alberto Oviedo Mota, B.C., MX\n", "M 4.8 - 91km ENE of Hihifo, Tonga\n", "M 5.4 - 33km SSW of Lluta, Peru\n", "M 3.0 - 70km ESE of Lakeview, Oregon\n", "M 4.4 - 58km NW of San Antonio de los Cobres, Argentina\n", "M 3.3 - 71km ESE of Lakeview, Oregon\n", "M 4.7 - 100km NE of Chichi-shima, Japan\n", "M 3.1 - 90km NNW of Chirikof Island, Alaska\n", "M 5.2 - 63km W of Ovalle, Chile\n", "M 4.6 - 64km WSW of Ovalle, Chile\n", "M 4.7 - 205km ESE of Antonio Enes, Mozambique\n", "M 3.0 - 90km NNW of Road Town, British Virgin Islands\n", "M 2.8 - 100km W of Willow, Alaska\n", "M 4.6 - South of the Fiji Islands\n", "M 2.6 - 71km N of Tierras Nuevas Poniente, Puerto Rico\n", "M 2.6 - 6km N of Edmond, Oklahoma\n", "M 2.9 - 45km W of Rincon, Puerto Rico\n", "M 4.7 - 20km NNE of Taniwel, Indonesia\n", "M 2.8 - 73km N of Tierras Nuevas Poniente, Puerto Rico\n", "M 3.3 - 8km NNE of Helena Valley Northwest, Montana\n", "M 5.3 - 85km WSW of Coquimbo, Chile\n", "M 2.9 - 8km NNE of Helena Valley Northwest, Montana\n", "M 4.5 - 207km WSW of Abepura, Indonesia\n", "M 2.6 - 50km NE of Road Town, British Virgin Islands\n", "M 2.6 - 14km SSE of Yosemite Valley, California\n", "M 2.7 - 21km NW of Ludlow, California\n", "M 3.9 - 172km W of Illapel, Chile\n", "M 4.1 - 30km SE of Al Marj, Libya\n", "M 2.8 - 6km N of Waikoloa, Hawaii\n", "M 2.7 - 19km SSE of Pahala, Hawaii\n", "M 2.7 - 6km N of Waikoloa, Hawaii\n", "M 4.6 - 86km SW of Canete, Chile\n", "M 4.3 - 53km N of La Serena, Chile\n", "M 4.4 - 62km ESE of Ishinomaki, Japan\n", "M 4.2 - 28km SW of Ovalle, Chile\n", "M 4.0 - 49km W of Ovalle, Chile\n", "M 2.9 - 7km S of Anthony, Kansas\n", "M 2.6 - 27km ENE of Road Town, British Virgin Islands\n", "M 2.6 - 19km SSE of Ridgemark, California\n", "M 4.9 - 29km SE of Nago, Japan\n", "M 2.5 - 49km NE of Road Town, British Virgin Islands\n", "M 2.5 - 27km SSW of Healy, Alaska\n", "M 4.5 - 43km WNW of Coquimbo, Chile\n", "M 4.9 - 198km NW of Tanahmerah, Indonesia\n", "M 2.6 - 7km NNE of Helena Valley Northwest, Montana\n", "M 2.8 - 64km SSW of Redoubt Volcano, Alaska\n", "M 2.9 - 6km ESE of Perry, Oklahoma\n", "M 4.9 - 43km WNW of Coquimbo, Chile\n", "M 4.9 - Indian Ocean Triple Junction\n", "M 4.7 - 126km W of Illapel, Chile\n", "M 5.4 - Indian Ocean Triple Junction\n", "M 2.8 - 45km NNE of Road Town, British Virgin Islands\n", "M 4.9 - 99km SSW of Biha, Indonesia\n", "M 5.0 - Northern Mid-Atlantic Ridge\n", "M 4.6 - 76km SW of Puerto Madero, Mexico\n", "M 5.8 - 45km SSW of Coquimbo, Chile\n", "M 5.6 - 17km N of Lebu, Chile\n", "M 2.5 - 30km NW of Rincon, Puerto Rico\n", "M 2.6 - 10km NE of Helena, Oklahoma\n", "M 2.7 - 11km SE of Esperanza, Puerto Rico\n", "M 4.3 - 22km NE of Agar, China\n", "M 4.4 - 111km E of Gualaquiza, Ecuador\n", "M 3.0 - 21km NW of Hawthorne, Nevada\n", "M 3.3 - 10km NW of Caldwell, Kansas\n", "M 4.4 - 59km WNW of Valparaiso, Chile\n", "M 5.1 - 188km SSE of L'Esperance Rock, New Zealand\n", "M 4.6 - 165km ENE of Olonkinbyen, Svalbard and Jan Mayen\n", "M 2.6 - 35km W of Kenai, Alaska\n", "M 4.4 - 183km ENE of Olonkinbyen, Svalbard and Jan Mayen\n", "M 4.3 - 61km SSE of Vanimo, Papua New Guinea\n", "M 4.3 - 38km SW of San Antonio, Chile\n", "M 3.0 - 3km SSE of Perry, Oklahoma\n", "M 2.5 - 125km WNW of Talkeetna, Alaska\n", "M 4.0 - 55km SSW of Ovalle, Chile\n", "M 3.1 - 5km SSE of Circle Hot Springs Station, Alaska\n", "M 4.5 - 100km W of Ambunti, Papua New Guinea\n", "M 3.3 - 144km NNE of Road Town, British Virgin Islands\n", "M 4.2 - 4km SSW of Izumi, Japan\n", "M 3.4 - 102km N of San Juan, Puerto Rico\n", "M 3.3 - 92km N of San Juan, Puerto Rico\n", "M 2.9 - 13km E of Pahala, Hawaii\n", "M 2.5 - 14km E of Pahala, Hawaii\n", "M 2.7 - 124km NW of Talkeetna, Alaska\n", "M 5.2 - 162km WSW of Ovalle, Chile\n", "M 5.1 - 73km W of Ovalle, Chile\n", "M 4.5 - 37km WSW of Ashkasham, Afghanistan\n", "M 5.5 - 15km ESE of Tondano, Indonesia\n", "M 4.5 - 94km WNW of La Ligua, Chile\n", "M 3.4 - 1km WNW of Campanilla, Puerto Rico\n", "M 3.2 - 66km WSW of Anchor Point, Alaska\n", "M 3.2 - 6km NNW of Stroud, Oklahoma\n", "M 2.7 - 9km NW of Ferndale, California\n", "M 4.6 - 33km SW of Hakha, Burma\n", "M 2.5 - 43km E of Cape Yakataga, Alaska\n", "M 4.3 - 25km SSW of El Rosario, El Salvador\n", "M 3.1 - 11km SE of Esperanza, Puerto Rico\n", "M 2.8 - 62km N of San Juan, Puerto Rico\n", "M 2.8 - 11km SE of Esperanza, Puerto Rico\n", "M 4.6 - 14km E of Coihueco, Chile\n", "M 2.9 - 9km SE of Esperanza, Puerto Rico\n", "M 2.8 - 9km SE of Esperanza, Puerto Rico\n", "M 2.9 - 7km ESE of Esperanza, Puerto Rico\n", "M 2.5 - 47km NNE of Vieques, Puerto Rico\n", "M 2.9 - 0km SW of Pawnee, Oklahoma\n", "M 2.9 - 33km ESE of Vieques, Puerto Rico\n", "M 2.8 - 34km SSE of Culebra, Puerto Rico\n", "M 2.6 - 0km ESE of Pawnee, Oklahoma\n", "M 3.8 - 13km ENE of Helena, Oklahoma\n", "M 2.7 - 18km E of Anchorage, Alaska\n", "M 4.2 - 18km SE of Esperanza, Puerto Rico\n", "M 4.6 - 81km N of Dobo, Indonesia\n", "M 5.4 - 31km S of Hihifo, Tonga\n", "M 4.6 - 12km S of Antsahe, Comoros\n", "M 2.7 - 0km E of Irving, Texas\n", "M 3.0 - 71km ESE of Lakeview, Oregon\n", "M 4.9 - 40km NW of Lluta, Peru\n", "M 4.5 - 60km ENE of Petite Riviere Salee, Martinique\n", "M 4.2 - 9km ENE of Quepos, Costa Rica\n", "M 2.5 - 27km W of Medford, Oklahoma\n", "M 4.1 - 50km WNW of Valparaiso, Chile\n", "M 2.8 - 12km SE of Alva, Oklahoma\n", "M 4.4 - 4km NE of Ishikawa, Japan\n", "M 2.9 - 39km NE of Luquillo, Puerto Rico\n", "M 3.1 - 55km NNE of Road Town, British Virgin Islands\n", "M 2.6 - 46km S of Redoubt Volcano, Alaska\n", "M 2.5 - 89km NW of Talkeetna, Alaska\n", "M 3.2 - 5km SE of Perry, Oklahoma\n", "M 2.7 - 6km WSW of Alberto Oviedo Mota, B.C., MX\n", "M 3.1 - 54km ESE of Road Town, British Virgin Islands\n", "M 2.7 - 61km NNE of Coldfoot, Alaska\n", "M 3.7 - 6km SE of Perry, Oklahoma\n", "M 2.9 - 69km ESE of Lakeview, Oregon\n", "M 2.7 - 21km ENE of Goldfield, Nevada\n", "M 4.4 - 103km WSW of Illapel, Chile\n", "M 5.6 - 139km SSW of Biha, Indonesia\n", "M 4.6 - 71km WSW of Coquimbo, Chile\n", "M 2.8 - 1km N of Fuig, Puerto Rico\n", "M 4.7 - 142km NE of San Pedro de Atacama, Chile\n", "M 4.5 - 116km W of Hihifo, Tonga\n", "M 5.3 - 86km SSW of Biha, Indonesia\n", "M 4.2 - 87km WNW of Ovalle, Chile\n", "M 4.4 - 35km WSW of Ovalle, Chile\n", "M 4.2 - 49km SSE of Jahrom, Iran\n", "M 3.3 - 98km S of Atka, Alaska\n", "M 4.4 - 94km W of Illapel, Chile\n", "M 2.9 - 24km WNW of Fairview, Oklahoma\n", "M 2.5 - 5km SE of Perry, Oklahoma\n", "M 4.2 - 50km W of Ovalle, Chile\n", "M 5.5 - 36km ESE of Tres Palos, Mexico\n", "M 2.6 - 4km NE of Langston, Oklahoma\n", "M 5.7 - 60km NNW of Visokoi Island, South Georgia and the South Sandwich Islands\n", "M 4.5 - 260km SE of Lambasa, Fiji\n", "M 2.8 - 4km NE of Langston, Oklahoma\n", "M 2.9 - 20km W of Perry, Oklahoma\n", "M 3.8 - 54km ESE of Boca de Yuma, Dominican Republic\n", "M 4.5 - Northern East Pacific Rise\n", "M 2.5 - 74km E of Chernabura Island, Alaska\n", "M 3.5 - 77km E of Chernabura Island, Alaska\n", "M 2.6 - 11km S of Alva, Oklahoma\n", "M 2.5 - 91km W of Ferndale, California\n", "M 3.3 - 120km N of Brenas, Puerto Rico\n", "M 2.8 - 36km SSE of Amukta Island, Alaska\n", "M 4.7 - 275km SE of Hachijo-jima, Japan\n", "M 2.6 - 16km W of Wellington, Kansas\n", "M 3.0 - 6km SSE of Cherokee, Oklahoma\n", "M 2.6 - 9km SSW of Amukta Island, Alaska\n", "M 3.0 - 75km SSW of Unalaska, Alaska\n", "M 2.5 - 23km E of Cherokee, Oklahoma\n", "M 2.8 - 89km E of Akutan, Alaska\n", "M 2.7 - 25km E of Cherokee, Oklahoma\n", "M 5.0 - 150km SW of Panguna, Papua New Guinea\n", "M 4.8 - 49km W of Ovalle, Chile\n", "M 4.8 - 227km SE of L'Esperance Rock, New Zealand\n", "M 3.8 - 49km W of Ovalle, Chile\n", "M 4.1 - 135km S of Atka, Alaska\n", "M 2.5 - 54km N of Amatignak Island, Alaska\n", "M 2.7 - 10km SW of Charleston, Missouri\n", "M 4.7 - 58km WNW of Illapel, Chile\n", "M 3.1 - 7km ENE of Greater Sudbury, Canada\n", "M 4.4 - 85km W of Kashi, China\n", "M 2.6 - 6km SE of Perry, Oklahoma\n", "M 5.4 - 42km W of Abepura, Indonesia\n", "M 3.3 - 10km NNE of Helena, Oklahoma\n", "M 2.5 - 10km NNE of Helena, Oklahoma\n", "M 2.7 - 18km NE of Fairview, Oklahoma\n", "M 2.6 - 17km SW of Amukta Island, Alaska\n", "M 5.2 - 181km SSW of Lorengau, Papua New Guinea\n", "M 4.2 - 5km WNW of Hualian, Taiwan\n" ] } ], "source": [ "# pull out the event descriptions\n", "\n", "for quake in data['features']:\n", " print quake['properties']['title']" ] }, { "cell_type": "code", "execution_count": 5, "metadata": { "collapsed": false }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "255\n" ] } ], "source": [ "# pull out magnitudes and depths into a pandas dataframe\n", "\n", "# first, set up a dictionary of empty arrays\n", "d = {'magnitude': [], 'depth': []}\n", "\n", "# loop through the earthquakes and pull out datapoints\n", "for quake in data['features']:\n", " d['magnitude'].append(quake['properties']['mag'])\n", " d['depth'].append(quake['geometry']['coordinates'][2])\n", "\n", "# then load it all into a dataframe\n", "df = pd.DataFrame.from_dict(d)\n", "\n", "print len(df)" ] }, { "cell_type": "code", "execution_count": 6, "metadata": { "collapsed": false }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ " depth magnitude\n", "0 3.56 2.96\n", "1 25.46 4.60\n", "2 0.73 2.82\n", "3 2.85 2.62\n", "4 9.02 4.20\n" ] } ], "source": [ "print df.head()" ] }, { "cell_type": "code", "execution_count": 7, "metadata": { "collapsed": false }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ " depth magnitude\n", "count 255.000000 255.000000\n", "mean 45.110569 3.671843\n", "std 81.903339 0.973804\n", "min 0.000000 2.500000\n", "25% 9.010000 2.800000\n", "50% 19.000000 3.300000\n", "75% 46.015000 4.550000\n", "max 551.360000 5.900000\n" ] } ], "source": [ "print df.describe()" ] }, { "cell_type": "code", "execution_count": 8, "metadata": { "collapsed": false }, "outputs": [ { "data": { "text/plain": [ "" ] }, "execution_count": 8, "metadata": {}, "output_type": "execute_result" }, { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAZEAAAEPCAYAAACDTflkAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3X10XXWd7/H3Nw2hgRZKKJQC5cEKQmecMajIFb3N3GsS\n9boKbR3AGTGjXNFhrnglOG0RbutQHsSpot7FIKhDZJCZLrGuOGLSgI0MXgGVopVSAYcqdWwV8Ykx\nM6XwvX/s30l2Ts7jPtlnn5N8Xmud1X323mefX37dZ3/373GbuyMiIpJES9YJEBGR5qUgIiIiiSmI\niIhIYgoiIiKSmIKIiIgkpiAiIiKJZRpEzGyBmX3RzB4zs51m9hoz6zCzETN73My2mtmC2P7rzOwJ\nM9tlZj1Zpl1ERLIviXwCuNvdTwf+CNgFrAVG3P1U4N7wHjNbBpwPLAPeCNxkZlmnX0RkVsvsImxm\nhwOvd/fPAbj7AXf/DbACGAi7DQDnhuVzgDvd/Xl33w08CZxZ31SLiEhclnfyJwO/MLO/N7OHzexW\nMzsUWOTu+8I++4BFYflYYE/s83uA4+qXXBERyZdlEGkFzgBucvczgH8nVF3leDQnS6l5WTRni4hI\nhloz/O49wB53/3Z4/0VgHbDXzI5x971mthj4edj+U2BJ7PPHh3WTmJkCi4hIAu5u1X4ms5KIu+8F\nnjazU8OqNwCPAl8B+sK6PuDLYXkQuMDM2szsZOAU4KEix26o1/r16zNPg9I0s9KlNClN0/1KKsuS\nCMD7gDvMrA34EfBOYA6w2cwuAnYD5wG4+04z2wzsBA4Al3gtf7mIiNQs0yDi7t8DXl1g0xuK7H8t\ncG2qiRIRkYppnEUddHV1ZZ2EKZSmyjViupSmyihN6bOZViNkZqrlEhGpkpnhzdSwLiIizU9BRERE\nElMQERGRxBREREQkMQURERFJTEFEREQSUxARaSLDw8P09Kymp2c1w8PDWSdHRONERJrF8PAwK1f2\nMTb2EQDa29ewZcsAvb29GadMZoKk40QURESaRE/PakZGVjAxP+kA3d2DbN16V5bJkhlCgw1FRKTu\nsp7FV0Qq1N9/Mfff38fYWPS+vX0N/f0DpT8kkjJVZ4k0keHhYTZtugWIgoraQ2S6qE0kUBAREame\n2kRERKTuFEREZgiNIZEsqDpLZAbQGBKplaqzRGaxTZtuCQGkD4iCSa4BvhIqxUhS6uIrMsvll2Lu\nv79PpRipmIKIyAxQyxiSyaUYGBuL1imISCUURERmgN7eXrZsGYiNIVFJQupDDesis5wa5QWadLCh\nme0Gfgu8ADzv7meaWQfwT8CJwG7gPHf/ddh/HfCusP+l7r61wDEVRESqpJHw0qxB5Cngle7+bGzd\nDcAz7n6Dma0BjnD3tWa2DPgC8GrgOOAe4FR3fzHvmAoiIiJVauYuvvmJXgHkWgQHgHPD8jnAne7+\nvLvvBp4EzqxLCkUaRKmuuOqmK1nIumHdgXvM7AXg0+5+K7DI3feF7fuARWH5WOCB2Gf3EJVIRGaF\nUl1x1U1XspJ1EDnb3X9mZkcBI2a2K77R3d3MStVNFdy2YcOG8eWuri66urqmIaki2SrVFVfddKVa\no6OjjI6O1nycTIOIu/8s/PsLM9tCVD21z8yOcfe9ZrYY+HnY/afAktjHjw/rpogHERERmSr/BvvD\nH/5wouNk1iZiZoeY2fywfCjQA+wABpl4/mcf8OWwPAhcYGZtZnYycArwUH1TLZKd/v6LaW9fQ9RU\nOBAGFF5cdptImjLrnRUCwZbwthW4w92vC118NwMnMLWL7xVEXXwPAO939ymth+qdJTNZqa646qYr\ntWjKLr5pUBAREaleM3fxFRGRJqUgIiIiiSmIiIhIYgoiIiKSmIKIiIgkpiAiIiKJKYiIiEhiCiIi\nIpKYgoiIiCSmICIiIokpiIiISGIKIiIikpiCiIiIJKYgIiIiiSmIiIhIYgoiIiKSmIKIiIgkpiAi\nIiKJKYiIiEhiCiIiIpKYgoiIiCSmICIiIokpiIiISGKZBxEzm2Nm283sK+F9h5mNmNnjZrbVzBbE\n9l1nZk+Y2S4z68ku1SIiAg0QRID3AzsBD+/XAiPufipwb3iPmS0DzgeWAW8EbjKzRki/iMislelF\n2MyOB94MfAawsHoFMBCWB4Bzw/I5wJ3u/ry77waeBM6sX2pFRCRf1nfyHwc+CLwYW7fI3feF5X3A\norB8LLAntt8e4LjUUygiIkW1ZvXFZvYW4Ofuvt3Mugrt4+5uZl5oW26XQis3bNgwvtzV1UVXV8HD\ni4jMWqOjo4yOjtZ8HHMvdY1Oj5ldC1wIHADmAocBXwJeDXS5+14zWwxsc/fTzGwtgLtfHz4/BKx3\n9wfzjutZ/U0iIs3KzHB3K7/nZJlVZ7n7Fe6+xN1PBi4Avu7uFwKDQF/YrQ/4clgeBC4wszYzOxk4\nBXio3ukWEZEJmVVnFZArPlwPbDazi4DdwHkA7r7TzDYT9eQ6AFyiIoeISLYyq85Ki6qzRESq13TV\nWSIi0vwUREREJDEFERERSUxBREREElMQERGRxBREREQkMQURERFJTEFEREQSUxAREZHEFERERCQx\nBREREUlMQURERBJTEBERkcQUREREJDEFERERSUxBREREElMQERGRxBREREQkMQURERFJTEFEREQS\nUxARkbKGh4fp6VlNT89qhoeHs06ONBBz96zTMK3MzGfa3ySSpeHhYVau7GNs7CMAtLevYcuWAXp7\nezNOmUwnM8PdrerPzbQLroKIyPTq6VnNyMgKoC+sGaC7e5CtW+/KMlkyzZIGkcyqs8xsrpk9aGaP\nmNlOM7surO8wsxEze9zMtprZgthn1pnZE2a2y8x6skq7iIhEWrP6Ynf/DzP7E3f/vZm1Aveb2euA\nFcCIu99gZmuAtcBaM1sGnA8sA44D7jGzU939xaz+BpHZoL//Yu6/v4+xseh9e/sa+vsHsk2UNIxM\nG9bd/fdhsQ2YA/yKKIjkztAB4NywfA5wp7s/7+67gSeBM+uXWpHZqbe3ly1boiqs7u5BtYfIJJmV\nRADMrAV4GFgK/J27P2pmi9x9X9hlH7AoLB8LPBD7+B6iEomIpKy3t1eBQwrKNIiEqqhXmNnhwLCZ\n/UnedjezUq3kBbdt2LBhfLmrq4uurq7aEysiMoOMjo4yOjpa83EapneWmV0FjAH/E+hy971mthjY\n5u6nmdlaAHe/Puw/BKx39wfzjqPeWSIiVWrG3lkLcz2vzKwd6Aa2A4NM9CXsA74clgeBC8yszcxO\nBk4BHqpvqkVEJC7L6qzFwEBoF2kBbnf3e81sO7DZzC4CdgPnAbj7TjPbDOwEDgCXqMghIpKtiqqz\nzGwOUQP3eNBx95+kmK7EVJ0lIlK9pNVZZUsiZvY+YD3wc+CF2KaXV/tlIiIys5QtiZjZj4Az3f2X\n9UlSbVQSERGpXpoN6z8Bflt9kkREZKYrWhIxs/6wuAw4DfhnYH9Y5+7+sfSTVz2VRGQmGx4eZtOm\nW4BoOhINAJTpkkabyHyiwXw/AZ4mmpqkLVnyRKRW+VOy339/n6YgkcxV0iZynrtvLreuUagkIjOV\npmSXNKXZJrKuwnUiIjLLFK3OMrM3AW8GjjOzTwK5CDUfeL4OaRORGE3JLo2oVMP6HwOdwN8AV8U2\n/Y5oPqtfpZ+86qk6S2YyNaxLWlJ7PK6ZHURU7XUaUUP7LnffX/JDGVIQERGpXmoj1oEe4GbgX8P7\nl5jZe9z97mq/TEREZpZKSiI/BP6Huz8Z3i8F7nb3l9UhfVVTSUREpHpp9s76bS6ABP+KRrCLiAiV\nlURuBk4AcuNC/pRoAOIIgLt/Kc0EVkslERGR6qXZsH5bWMztaLFl3P2d1X5pmhRERESql1oQaTYK\nItLM1IVXspJmSeRlwE3AMe7+B2b2R8AKd9+YLKnpUhCRZpU/N1Z7+xrNjSV1k2bD+q3AFUzM4LsD\neFu1XyQipW3adEsIIH1AFExypZJKDA8P09Ozmp6e1QwPD6eWTpG4SoLIIe7+YO5NuM3XtCciGSgW\nKHKlmJGRFYyMrGDlyj4FEqmLSgYb/sLMXpp7Y2ZvBX6WXpJEZqdyc2OVmgp+cikGxsaiko2qwiRt\nlQSR/wV8GniZmf0b8BTw56mmSmQW6u3tZcuWgVjD+uT2EAUKaUSlZvHtj739GrCNqPrr98AqoCGf\nbCgSN1t6O2mGX8mMuxd8ARuA9cAXgCeATeH1OPAPxT6X9Sv6k0Tch4aGvL19kcNtDrd5e/siHxoa\nyjpZRZVLbyXbu7tXeXf3qob+O6UxhWtn9dfcsjvAvwDzY+/nA/+S5MvyjruEqHTzKPAD4NKwvoNo\nNPzjwFZgQewz60JA2wX0FDnu9OeuNKXu7lXhguvhdZt3d6/KOllFVZJeBQpJS9IgUkmbyNFM7o31\nfFhXq+eBD7j7I2Y2D/iumY0A7wRG3P0GM1sDrAXWmtky4HxgGXAccI+ZneruL05DWlIxW6pSpH56\ne3t1HklDqSSIfB54yMy+RDTlyblAzZWt7r4X2BuWnzOzx4iCwwpgedhtABglCiTnAHe6+/PAbjN7\nEjgTeKDWtKShVE8aqY9maydotvSKQIXTnpjZK4HXE82ZdZ+7b5/WRJidBHwD+EPgJ+5+RFhvwLPu\nfoSZfQp4wN3vCNs+A3zN3e/KO5ZX8jelradnNSMjK8j1pIEBursH2br1rlIfk2nWbKXBZkuvzBxp\nPpQKd/8u8N2qU1WBUJV1F/B+d/9dFDfGv9fNrFREKLhtw4YN48tdXV10dXVNS1ql+TRb9U+zpVea\n1+joKKOjozUfJ9MJGMOjd/+ZqERxY1i3C+hy971mtpjoee6nmdlaAHe/Puw3BKz32Gj6sL4hSiKa\nB0lEmknTzeIbqqoGgF+6+wdi628I6z4SAscCd881rH+BqB3kOOAe4KX5EaNRggioakJEmkczBpHX\nAfcB32eiWmod8BDRA7BOAHYD57n7r8NnrgDeBRwgqv6aMjlQIwUREZFm0XRBJC0KIiIi1UtzKngR\nEZGCFERERCQxBRGRWUIPrZI0qE1EZBZQl3MpRw3rgYKIyFSaQUHKUcO6iIjUXUXTnohIc9PkjpIW\nVWeJzBKaQUFKUZtIoCAiIlI9tYmIiEjdKYiIiEhiCiIyo2mAnUi61CYiM5YG2IlUTg3rgYKI5GiA\nnUjl1LAuIiJ1p8GGMmNpgJ1I+lSdJTOaBtiJVEZtIoGCiIhI9dQmIiIidacgInWh8RrTQ/lYGeVT\nHbn7jHpFf1JjGBoa8u7uVd7dvcqHhoYq3lbLcRvR0NCQt7cvcrjN4TZvb1/UFOluNMrHyiifkgnX\nzuqvuUk+1MivRgkipU7kWk7yZvyBdHevCun18LrNu7tXZZ2spqN8rIzyKZmkQUTVWSnZtOmWMFK6\nD4hGTed6CZXaVstxRaR2qgqrTqZBxMw+Z2b7zGxHbF2HmY2Y2eNmttXMFsS2rTOzJ8xsl5n1ZJPq\nRrADWB1eO6ZsLfUjqOUHkvSz/f0X096+BhgABsJ4jYur+m4pn4+6+EVqOd9yU+WMjKxgZGQFK1f2\nzeq8rEiS4st0vYDXA53Ajti6G4C/DstrgOvD8jLgEeAg4CTgSaClwDGntYiXVKlqp40bNzocNr4N\nDvONGzdWdNxyn23UarRma8dpVMXysRmrOdOU9HybzVVhNGubSAgI8SCyC1gUlo8BdoXldcCa2H5D\nwFkFjjd9uVqjYidydKL2O6wKr/6KT9RyJ3mp7bX8QGbzj6sZ6P9neszmfEwaRBpx2pNF7r4vLO8D\nFoXlY4EHYvvtAY6rZ8Kq1dvbW2KE9MuBvw3LA8BT9UmUyAyRxmwEmiqneo0YRMa5u5tZqeHnBbdt\n2LBhfLmrq4uurq7pTViFip3ktZyo5T5banua3yvZmm3/P8PDw6xYcSH7938UgG9840IGB2+vOZD0\n9vayZctA7Hc7cx8dMDo6yujoaO0HSlJ8mc4XhauzjgnLi5mozloLrI3tNwS8psDxpqlwV5tyddRp\njhPR+JSZK63/22bT2bl8SrVTZ+fyrJPV1JhBbSI3ENo+QuDIb1hvA04GfkSY+yvveNOZr4nN5rpV\nSYcazyd0dCyd8vvq6Fg6LceeTcE4LmkQybQ6y8zuBJYDC83saeD/ANcDm83sImA3cB6Au+80s83A\nTuAAcEn4w0VmhcljhGBsLFo3U6tbSjnxxGN49tnLY2su58QTX1bzcfOfhnn//X16GmYZmQYRd39b\nkU1vKLL/tcC16aVo+sy2OmqRerruuqtYseIC9u+/GYC2tgNcd91VNR9Xgbp6GrGeklwDXXf3IN3d\ng1PuZjQwTKqlQZsTent7GRz8R7q7j6W7+1gGB/9RF/qsJKkDa+QXDdImUkojD9xL69iztZ55uikf\n0zWb251o1ob16X41QxCppdG9kpM86YUmrR/QbP5hSvOZrYFaQWSGBZHSo92Lf7aWC3ZaPcrUU22y\n2XqRqiflcfWSBpGGHmw4U5VrdK+lh4gaBhtbrb1/9Mz48tTDqs6SRJ5GftEEJRH30ndKpe7cy5U0\n0q4qS/I3NWt1Vhp3s5WUJEsNJswqH5tpkKNKvsmg6qzmCiKl1HqhaWs7avxC09Z2VFUj5Tdu3Ogd\nHUu9o2NpxTML546b1gj9LKR1wc7qBqEWac0MnRYFkWQURGZQECn3wyx1oY+CyAKHsxzO8ra2BZN+\n8MW2VfK9pcy0H25af0+pqfxrmaE5TWnNDJ2WRgxszSBpENE4kQZUaozJNddcw5VX3sCzz17Fs89e\nxZVX3sA111wz/tlNm25h//4bgW8B32L//hvH69DXrbua/ftbgfcC72X//lbWrbt60meTPjXxmWd+\nWdG62aLYOKBvfONh4N3AYHi9O6wrr7//YtraPkhunEhb2werGicyW8YmlRujJdMsSeRp5BcNVBJJ\nox653JxBnZ1nh5LGKoehSXeG5T5by11l9L0dsTvsDu/sPLviv6vRpPUAr3LVWeWqIkuVJGv5e5K2\nxcy0aszZDFVnNVYQSaseef78E6ZchObPP2H8uPGLECycdKEpN/NpLemKgshh4xc4OKypg4h7Ok/H\nK1WdVS5IpPVQsUr+38tVoRbrUJE06En9KYg0WBBJqx556dJlUy5CS5cuK3rcpUtfMf7Zcne67skb\n1isJULPljrRUXkT/R4Wfallq28T2WoJIqe+d/rFH0Y3Fwkk3Nc1+Y+E+c8/lpEFE40RSklYbwWGH\nHUE0ufGVYc2BsK6wp556muHh4fGnLA4O3h4bZ3D7lPm8rrnmU+P966+5Zg2vetWrKqpPXrjwyKLr\nZl+//QPApcDN4f1OoidZ5hR+quUzz+wDRoGPhW2X8cwzp49/qpZJPZcvP4ORkRuAT4Y1l7J8+V9X\n9Nmorex1QNR+Njb2uorGHv34x3uJ/s6+2Lqri+7fDGbfuVyBJJGnkV80SEmk1F1YqSqNuEJ3PKXu\ncoeGhryl5YjYcRdNuZstJa0xJo3Yg6dWpe5Go//7ebGqvXnj//el8mnp0leULEmW+96kY4/KnY+l\nSr+lNPKDo9KoqqzluI0AVWc1VhCppdrCvfjFptxnSzWsV5bm5Bf7vr4+b2092ltbj/a+vr6Kj5vm\nDy+NY5er3lm69OVTbiCWLn35+PZiVYal2rsqSVNr65Hj39naemTF7SnlzqkoXZO3V5KuSqpPs5BW\nV/Zm71qsINJgQaTWO/Po7u94h6UOGz3Xkyq6QBW/a0yrV1E55RqMsxisVuuxiwWgcv9/pXrBlUrT\nvHmLpwSfefMWV5TWqBQz+UJfaXtYuSBSLl219ELM4s5dJe7CFEQaLIi4J58GpNAFGVb7RPVI8Tuh\n7u5V3tm53Ds7z070w0yr63HSC3ItsrpYJG1Yr6WHW0vL4VMu9C0th0/6e4r1lCpfnVW8mi2rm5Za\n1HrOZXEu14OCSAMGkVJK9YKaP39JOBmHwoXmLIdDY++nnqhZF6WTPvO6UYNI+fEcxbuulrool2or\nq6VLrFnHlPSadVT099QeFKc/j3P5kUYpJa3fSta/wVolDSIasZ6iYiOEc72gcqPOr7nmU5O2j439\nJ7CDqFfLCqIR5nPC1ouBS4AlwBJaW/+K/v6L80ab72Fs7CDe8pZ3TBrNnqbLLnsnUY+kgfC6NKwr\nLc2n9aX7JMCDyI38j5YnRCPQu4l6M10NdMdGpbcS/R/lRqz3kXtKdSVP60tv1PkOYHV47cjbdgC4\nnIn/28vDuvTkekGNjKxgZGQFK1f2Tfl7k+ZFb28vH/rQ++jouJqOjqv50IfeNy29q2btSPkkkaeR\nXzRISaR8HXTxO7DFi1/iE717JhrIo/erC97lThyzdNVEmnXQtUze2GgN67VVZxUvbZQbO5F0XrSo\n3WJyj7D8douk1VnR37vao/a5pQ6rp6UxuZY8bsZqtEaHqrMaK4gUqwKITuBjp2xrbz/Gu7tX+caN\nG33u3I68H3XUVbejY6m3th7t+XXqHR1LYz+M46f84OMj2tN6KmIzStroW0v1T6lt5S7mpQJQX1+f\nwyGxIHLIpB5ypXpvlft7yqWrlnOmWNAsl6Y0q9FmKwWRBgsik9sIhhzO8jlzjgp3g1NLExPvD3M4\nKu8k73c4wjs7l3tb29QG1Pb2o9w992OfO+XYZoe4e+k7Svfa79CSlkSyUEv7Qy0N0aUuYBNtYRPb\n5s9fMn7c6JyaegPhXr6EU0u7Ri1tF7XNOF28e7CCyPRTEGmwILJ48UkOBzsc49HEhKvDidvhsNjh\noLC8INxBdjj0hX2OjF3sl3hUTREPOGdPCgS5H1gUAI6eEiggamAtN2is1otF/p1wpYEki+qsWqbk\nKNedtrV13pR8bm2dN56eYoF6zpz8m4fbfM6co8aPG1VzTk7z4sUvcXcvUro9NpamqcdubY3ffJQq\nARUPQKX+nlpKVrV0XihH1VmFzZogArwR2AU8AawpsH068rNmLS1TSwS58R5RT6vDJv14ootSLkDM\nKbB9aPwHHAWeySWNiQBweIHvnRu2H17gInXEeJqr6ZFkFo3CrnWMQ5oD0kpdLJL2JnP3UKWYf0E+\nenx7dEMwOcjkAnkuXYUCW3v7UVPyMFfKdC9dUpkz58gp2+bMOTKWpkLn41x3Lz9OpNTFvtQ5Uy6P\nS22vrHRUesBuKbOp2rZSSYNIU82dZWZzgP8LvAH4KfBtMxt098eyTdlUL77YDnyc+LxBUU+dq4CD\nieZHim8bJJrX6APAYQU+ewuQ6+lx2qRt7h+Izcs1p8Cx/zcjIyuI5mWa7IUXbHy5v/9itm07nwMH\nojmfWlsfo7//n4D8Z5GA++Vs3/5LVq6M5g4aGztA/jxJY2MfLJAzk61bdx379390/HP790frpuOZ\n46WeN3/iicfz7LOTj3XiicdX9J1z587huecuj625nLlz4z20XmDq/FgvjG/NzWOW77TTTmX79h1M\nzLm1n9NOm5hz66CD2qZ8Jreuvb2N5567LLblMtrbD46nGriI6DyD6Jkmn41tLzyfF8DChYuAs2Kf\n7WPhwqeoVS3/B5HiaS6n2P+BJJAk8mT1Av4LMBR7vxZYm7fPNMTk2sERU+6kokbvRbGSRHxb7s6r\nwyeeyxHfnhtkeHi4A4tv6/C2tgXhjr7QZzvC8rwpd5QwbzzNpaoICt01RtVl0R1ie/viKdvb28uX\nRGopEZSrlihfskpWAipXdRdVO8XnMDtivNqp3N+TtAqnXJo6Ok4skM8nVpSPpbbXUp1V6v+gljRJ\nMsyG6izgrcCtsfdvBz6Vt8905GfNilUfRNVSnV6sOmvx4hN83rwFBT67MFwcDiqwLZpzqbPzbDc7\npMD2eeHCcdSUC01Hx0R1SakLeqF6cVg+fmFO+lCqWtomau0GmkavotxxW1rmj+dzS8v8qrqflkpT\nqe8tn6aJatCWlqmPRk7SU63ctnKdLZIet5LtUp3ZEkRWVxJE1q9fP/7atm3bNGRv9aIulwd7VPo4\n3uFgb2k5OHaxbAl3q7mG9YXe0nLw+A8DlnnUwH5kCBLHO5zlra2HutnB4cK7MHzHxESL0d3f5O9t\nbT3UJ0oz8eMum3TRLTfn0+QHXkX1/rkLc9LeTrX0kqqkl01WF5pGvMA1YpokO9u2bZt0rZwtQeSs\nvOqsdeQ1rjdKScR96qy2hX7ExdbF76Db2o6aNBfW0NCQd3Yu95aWI0MJZmo1QvzuL/cdnZ1nTwoE\nlczZVWg8QLG5uZJepGr5nKo0RKbHbAkircCPgJOANuAR4PS8faYnRzNWyYU1ycW3luqSRqS7a5Hp\nkTSIWPTZ5mFmbwJuJOqG9Fl3vy5vuzfb3yQikjUzw92t/J55n5tpF1wFERGR6iUNIprFV0REElMQ\nERGRxBREREQkMQURERFJTEFEREQSUxAREZHEFERERCQxBREREUlMQURERBJTEBERkcQUREREJDEF\nERERSUxBREREElMQERGRxBREREQkMQURERFJTEFEREQSUxAREZHEFERERCQxBREREUlMQURERBJT\nEBERkcQyCSJm9qdm9qiZvWBmZ+RtW2dmT5jZLjPria1/pZntCNs+Uf9Ui4hIvqxKIjuAlcB98ZVm\ntgw4H1gGvBG4ycwsbP474CJ3PwU4xczeWMf01mR0dDTrJEyhNFWuEdOlNFVGaUpfJkHE3Xe5++MF\nNp0D3Onuz7v7buBJ4DVmthiY7+4Phf0+D5xbn9TWrhFPGqWpco2YLqWpMkpT+hqtTeRYYE/s/R7g\nuALrfxrWi4hIhlrTOrCZjQDHFNh0hbt/Ja3vFRGR+jF3z+7LzbYB/e7+cHi/FsDdrw/vh4D1wI+B\nbe5+elj/NmC5u7+3wDGz+4NERJqYu1v5vSZLrSRShXiiB4EvmNnHiKqrTgEecnc3s9+a2WuAh4AL\ngU8WOliSTBARkWSy6uK70syeBs4CvmpmXwNw953AZmAn8DXgEp8oKl0CfAZ4AnjS3Yfqn3IREYnL\ntDpLRESaW6P1zqqImS0xs21hwOIPzOzSIvt9MgxO/J6ZdWadJjPrMrPfmNn28Loy5TTNNbMHzewR\nM9tpZtfOKvhwAAAGUklEQVQV2a+e+VQ2TfXOp9j3zgnfV7DjRz3zqZI0ZZhPu83s++E7HyqyT13z\nqlyassgrM1tgZl80s8fCuX5WgX3qnU8l05Qon9y96V5Evb5eEZbnAT8ETs/b583A3WH5NcADDZCm\nLmCwznl1SPi3FXgAeF2W+VRhmuqeT+F7LwPuKPTdWeRTBWnKKp+eAjpKbM/inCqXpix+ewPAu8Jy\nK3B4A+RTuTRVnU9NWRJx973u/khYfg54jGgsSdwKogzD3R8EFpjZoozTBJM7EqTO3X8fFtuAOcCz\nebvUNZ8qTBPUOZ/M7HiiH/Vninx33fOpgjRRYn3aSn1v3fOqgjRVsn3amNnhwOvd/XMA7n7A3X+T\nt1td86nCNEGV+dSUQSTOzE4COoEH8zYdBzwde78HOD7jNDnw2lB0vTtM85J2WlrM7BFgH1E36Z15\nu9Q9nypIU93zCfg48EHgxSLbszifyqUpi3zKfe89ZvYdM3t3ge1Z5FW5NNU7r04GfmFmf29mD5vZ\nrWZ2SN4+9c6nStJUdT41dRAxs3nAF4H3h7v/KbvkvU+9F0GZND0MLHH3PwY+BXw57fS4+4vu/gqi\nk/O/mllXgd3qmk8VpKmu+WRmbwF+7u7bKX0XVrd8qjBNdT+fgrPdvRN4E/BXZvb6AvvU+7dXLk31\nzqtW4AzgJnc/A/h3YG2B/eqZT5Wkqep8atogYmYHAXcB/+Duhf7QnwJLYu+PD+syS5O7/y5XlePu\nXwMOMrOONNMU++7fAF8FXpW3qe75VC5NGeTTa4EVZvYUcCfw38zs83n71DufyqYpq/PJ3X8W/v0F\nsAU4M2+Xup9T5dKUQV7tAfa4+7fD+y8SXcDj6p1PZdOUJJ+aMoiYmQGfBXa6+41FdhsE3hH2Pwv4\ntbvvyzJNZrYo7IeZnUnUxbpQe8B0pWmhmS0Iy+1AN7A9b7d651PZNNU7n9z9Cndf4u4nAxcAX3f3\nd+TtVtd8qiRN9c6n8D2HmNn8sHwo0EM0K3dcvc+psmnK4JzaCzxtZqeGVW8AHs3brd7nVNk0Jcmn\nRhixnsTZwNuB75tZ7gJ0BXACgLt/2t3vNrM3m9mTRMW2d2adJuCtwF+a2QHg90QXhzQtBgbMrIXo\nhuF2d7/XzN6TS1MG+VQ2TdQ/n/I5QMb5VDZNZJNPi4At4TrTCtzh7lszzquyaSKbvHofcIeZtQE/\nAt7VAOdUyTSRIJ802FBERBJryuosERFpDAoiIiKSmIKIiIgkpiAiIiKJKYiIiEhiCiIiIpKYgohI\nnZnZe8zswrD8F2a2OMExdtdrtgORUpp1sKFI0wqDunL6iEZX/6zaw0xfikSSU0lEZj0zO8nMdoXZ\nTX9oZneYWY+ZfdPMHjezV4fX/wuzn34zN3VEmHJjs0UPI/uSmT1gZmeEbc+Z2UaLHsD1LTM7Oqzf\nYGb9ZraaaN6wO8Jx58ZLGGb2KjPbFpaPNLOtFj3w7FZiE/eZ2dstetDXdjO7OcwGIFIXOtlEIkuB\nvwVOA14GnO/uZwOXE01f8xjRsxjOANYD14bPXQL80t3/ALgKeGXsmIcA3wozFt8H5KYod8Dd/S7g\nO8CfufsZ7v4fFC9hrAfuc/c/JJpg8AQAMzsdOA94bZjF9kXgz2vKCZEqqDpLJPKUuz8KYGaPAveE\n9T8ATgIWALeb2UuJLvS5387ZwI0A7v6omX0/dsz97v7VsPxdoskmC6nkIUCvB1aG77nbzH4VPvff\niQLXd8LcUe3A3gqOJzItFEREIv8ZW34R2B9bbgWuBu5195UWPXRsW2z/YkHg+bxjFvu9xUsfB5io\nIZibt1+x7xlw9yuKbBNJlaqzRMoz4DDg38L7v4ht+yZRdRIWPQXu5RUeLxcQfheOnbObieerrI6t\nvw/4s/A9bwKOIAo+9wJvNbOjwrYOMzuhgjSITAsFEZFIfltE/P2LwEeB68zsYaLnwue23wQcFarA\nriZ6PsNvChzDY+/jy7cBN+ca1oEPA58ws28TlUpy+32Y6CmQPyCq1voxgLs/BlwJbDWz7wFbgWOq\n/utFEtJU8CI1CD2hDnL3/zSzpcAIcKq7H8g4aSJ1oTYRkdocCnzdokcjG/CXCiAym6gkIiIiialN\nREREElMQERGRxBREREQkMQURERFJTEFEREQSUxAREZHE/j8lbflnHYNWQAAAAABJRU5ErkJggg==\n", "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "# plot the depth vs. magnitude\n", "\n", "df.plot(x='magnitude', y='depth', kind='scatter')" ] }, { "cell_type": "code", "execution_count": 9, "metadata": { "collapsed": false }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "file saved\n" ] } ], "source": [ "# save dataframe to disk\n", "\n", "df.to_csv('usgs_earthquake_data.csv')\n", "\n", "print 'file saved'" ] }, { "cell_type": "code", "execution_count": 10, "metadata": { "collapsed": false }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ " depth magnitude\n", "0 3.56 2.96\n", "1 25.46 4.60\n", "2 0.73 2.82\n", "3 2.85 2.62\n", "4 9.02 4.20\n" ] } ], "source": [ "# read it back later\n", "\n", "new_df = pd.DataFrame.from_csv('usgs_earthquake_data.csv')\n", "\n", "print new_df.head()" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": true }, "outputs": [], "source": [] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": true }, "outputs": [], "source": [] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": true }, "outputs": [], "source": [] }, { "cell_type": "markdown", "metadata": {}, "source": [ "# Part 2: Querying an API endpoint\n", "\n", "### Google Maps Geocoding API\n", "\n", "Google Maps has several APIs for getting search results programmatically. This one looks up latitude-longidtude coordinates (and other place information) for street addresses, which is called geocoding. \n", "\n", "It works similarly to the earthquakes example, with query parameters added to the URL endpoint.\n", "\n", "**API documentation:** \n", "https://developers.google.com/maps/documentation/geocoding/intro\n", "\n", "**API endpoint:** \n", "https://maps.googleapis.com/maps/api/geocode/json\n", "\n", "**API endpoint with query parameters:** \n", "https://maps.googleapis.com/maps/api/geocode/json?address=Wurster+Hall" ] }, { "cell_type": "code", "execution_count": 11, "metadata": { "collapsed": false }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "https://maps.googleapis.com/maps/api/geocode/json?address=Wurster+Hall%2C+Berkeley%2C+CA\n" ] } ], "source": [ "# we have to encode the search query so that it can be passed as a URL, \n", "# with spaces and other special characters removed\n", "\n", "endpoint = 'https://maps.googleapis.com/maps/api/geocode/json'\n", "\n", "params = {\n", " 'address': 'Wurster Hall, Berkeley, CA',\n", "}\n", "\n", "url = endpoint + '?' + urllib.urlencode(params)\n", "print url" ] }, { "cell_type": "code", "execution_count": 12, "metadata": { "collapsed": false }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "{u'status': u'OK', u'results': [{u'geometry': {u'location_type': u'ROOFTOP', u'bounds': {u'northeast': {u'lat': 37.871116, u'lng': -122.2541912}, u'southwest': {u'lat': 37.8700084, u'lng': -122.2552699}}, u'viewport': {u'northeast': {u'lat': 37.8719111802915, u'lng': -122.2533815697085}, u'southwest': {u'lat': 37.86921321970851, u'lng': -122.2560795302915}}, u'location': {u'lat': 37.8707352, u'lng': -122.2548935}}, u'address_components': [{u'long_name': u'Wurster Hall', u'types': [u'premise'], u'short_name': u'Wurster Hall'}, {u'long_name': u'Berkeley', u'types': [u'locality', u'political'], u'short_name': u'Berkeley'}, {u'long_name': u'Alameda County', u'types': [u'administrative_area_level_2', u'political'], u'short_name': u'Alameda County'}, {u'long_name': u'California', u'types': [u'administrative_area_level_1', u'political'], u'short_name': u'CA'}, {u'long_name': u'United States', u'types': [u'country', u'political'], u'short_name': u'US'}, {u'long_name': u'94720', u'types': [u'postal_code'], u'short_name': u'94720'}], u'place_id': u'ChIJnzKeUCV8hYARccYjWTGEJUg', u'formatted_address': u'Wurster Hall, Berkeley, CA 94720, USA', u'types': [u'premise']}]}\n" ] } ], "source": [ "# open a connection to the URL\n", "connection = urllib.urlopen(url)\n", "\n", "# download and parse the results\n", "results = json.loads(connection.read())\n", "\n", "print results" ] }, { "cell_type": "code", "execution_count": 13, "metadata": { "collapsed": false }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Wurster Hall, Berkeley, CA 94720, USA\n" ] } ], "source": [ "# pull out the formatted addresses\n", "\n", "for item in results['results']:\n", " print item['formatted_address']" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": true }, "outputs": [], "source": [] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": true }, "outputs": [], "source": [] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": true }, "outputs": [], "source": [] }, { "cell_type": "markdown", "metadata": {}, "source": [ "# Part 3: Querying an API with authentication\n", "\n", "### Twitter REST and Streaming APIs\n", "\n", "Twitter's APIs also operate over the web, but they require a back-and-forth authentication process at the beginning of a connection. It's easier to have a Python library handle this than to create the query URLs ourselves.\n", "\n", "The REST APIs perform stand-alone operations: we submit a query and receive results, like in earlier examples. The Streaming API continues sending results in real time until we disconnect.\n", "\n", "(REST is a set of principles describing how data transactions should work over the web, while the actual communication protocol is called HTTP. Web pages work through HTTP and REST too, but the browser steps in to interpret and display the content for you.)\n", "\n", "**API documentation:** \n", "https://dev.twitter.com/rest/public \n", "https://dev.twitter.com/streaming/overview\n", "\n", "**Documentation for third-party Python \"wrapper\"**: \n", "https://github.com/geduldig/TwitterAPI" ] }, { "cell_type": "code", "execution_count": 14, "metadata": { "collapsed": true }, "outputs": [], "source": [ "from TwitterAPI import TwitterAPI" ] }, { "cell_type": "code", "execution_count": 15, "metadata": { "collapsed": true }, "outputs": [], "source": [ "# import API credentials from keys.py file in the\n", "# same directory as this notebook\n", "\n", "from keys import *" ] }, { "cell_type": "code", "execution_count": 16, "metadata": { "collapsed": false }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Connection is set up but not tested\n" ] } ], "source": [ "# set up an API connection using credentials from the keys file\n", "\n", "api = TwitterAPI(consumer_key, consumer_secret, \n", " access_token, access_token_secret)\n", "\n", "print \"Connection is set up but not tested\"" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Making a simple data request" ] }, { "cell_type": "code", "execution_count": 17, "metadata": { "collapsed": false }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Can the Bay Area house everyone who wants to live there without altering its character? Yes. https://t.co/jalpboUz9y http://t.co/rtQz8leoEC\n" ] } ], "source": [ "# Most recent tweet from @GBoeing's timeline\n", "\n", "endpoint = 'statuses/user_timeline'\n", "params = {\n", " 'screen_name': 'gboeing', \n", " 'count': 1\n", "}\n", "r = api.request(endpoint, params)\n", "\n", "for tweet in r.get_iterator():\n", " print tweet['text']" ] }, { "cell_type": "code", "execution_count": 18, "metadata": { "collapsed": false }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "[u'contributors', u'truncated', u'text', u'is_quote_status', u'in_reply_to_status_id', u'id', u'favorite_count', u'source', u'retweeted', u'coordinates', u'entities', u'in_reply_to_screen_name', u'id_str', u'retweet_count', u'in_reply_to_user_id', u'favorited', u'user', u'geo', u'in_reply_to_user_id_str', u'possibly_sensitive', u'lang', u'created_at', u'in_reply_to_status_id_str', u'place', u'extended_entities']\n" ] } ], "source": [ "# What other data is there?\n", "\n", "print tweet.keys()" ] }, { "cell_type": "code", "execution_count": 19, "metadata": { "collapsed": false }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Tweet // Can the Bay Area house everyone who wants to live there without altering its character? Yes. https://t.co/jalpboUz9y http://t.co/rtQz8leoEC\n", "Timestamp // Tue Oct 06 19:13:33 +0000 2015\n", "Retweets // 0\n", "Favorites // 0\n", "Geotag // None\n", "Language // en\n", "User // gboeing\n", "Followers // 1676\n", "Profile // PhD candidate in urban planning at @UCBerkeley. Cities, complexity, livability, urban design, data.\n" ] } ], "source": [ "# Contents of some additional fields...\n", "# Here are the definitions: https://dev.twitter.com/overview/api/tweets\n", "\n", "for tweet in r.get_iterator():\n", " print \"Tweet // \", tweet['text']\n", " print \"Timestamp // \", tweet['created_at']\n", " print \"Retweets // \", tweet['retweet_count']\n", " print \"Favorites // \", tweet['favorite_count']\n", " print \"Geotag // \", tweet['coordinates']\n", " print \"Language // \", tweet['lang']\n", " print \"User // \", tweet['user']['screen_name']\n", " print \"Followers // \", tweet['user']['followers_count']\n", " print \"Profile // \", tweet['user']['description']" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Other API endpoints allow different types of searches" ] }, { "cell_type": "code", "execution_count": 20, "metadata": { "collapsed": false }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Dear #Muni: You always wreak of mothballs. I'm afraid to ask what the source may be...\n", "\n", "RT @sfexaminer: Governor signs bus-camera ticketing bill, #Muni to target double parkers\n", "http://t.co/fadMGLCos6 http://t.co/agz3xYFsAy\n", "\n", "Hey #Muni!! What did you do? Lol lol. The things us suburban gals get amused by.\n", "\n", "New Jim Colby video on #Fed’s September hike delay and Puerto Rico default: http://t.co/cyjhi9I8qX #muni #bonds http://t.co/SUrgQgR0hz\n", "\n", "RT @Scott_Wiener: Kicking off new campaign to put #art on #Muni buses. The 5 winning artists did a great job beautifying our #transit. http…\n", "\n" ] } ], "source": [ "# Search for public tweets about #muni\n", "\n", "endpoint = 'search/tweets'\n", "params = {\n", " 'q': '#muni', \n", " 'count': 5\n", "}\n", "r = api.request(endpoint, params)\n", "\n", "for tweet in r.get_iterator():\n", " print tweet['text'] + '\\n'" ] }, { "cell_type": "code", "execution_count": 21, "metadata": { "collapsed": false }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Sheesh\n", "\n", "RT @sardanarohit: जब घर में हैं आज़म खान, तो क्यों चाहिए पाकिस्तान? \n", "https://t.co/JoZrUYGBDs\n", "\n", "Makkah Mina vipathil, Ilankayar oruvar maranam.\n", "\n", "गंगा माता बिलख रही थी लाचारी के घाटों पर,\n", "क्रूर लाठियां बरस रही थीं चन्दन लगे ललाटों पर,\n", "\n", "RT @rverma1080: सत्य परेशान हो सकता है लेकिन पराजित नहीं हो सकता।। http://t.co/dXjMyyIIFq\n", "\n" ] } ], "source": [ "# Search for public tweets in Hindi\n", "\n", "endpoint = 'search/tweets'\n", "params = {\n", " 'q': '*', \n", " 'lang': 'hi', \n", " 'count': 5\n", "} \n", "r = api.request(endpoint, params)\n", "\n", "for tweet in r.get_iterator():\n", " print tweet['text'] + '\\n'" ] }, { "cell_type": "code", "execution_count": 22, "metadata": { "collapsed": false }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Classes! (@ South Hall - @cal in Berkeley, CA) https://t.co/1EMp7qMgU0\n", "\n", "The new Lower Sproul Plaza and Eshleman Hall, by Moore Ruble Yudell Architects--already very popular… https://t.co/hooRcLhRA5\n", "\n", "I'm at University of California, Berkeley - @cal in Berkeley, CA https://t.co/rQBd4E0woh\n", "\n", "Morning #meditation.\n", "And laundry.\n", "#dormlife #theearlybirdgetsthedryer #GradSchool #GottaGetItAllDone… https://t.co/w5q1Iyieh0\n", "\n", "hey is this avant-garde @ UC Berkeley https://t.co/xqGnhy58s4\n", "\n" ] } ], "source": [ "# Search for public tweets geotagged near the UC Berkeley campus\n", "\n", "endpoint = 'search/tweets'\n", "params = {\n", " 'q': '*', \n", " 'geocode': '37.873,-122.260,0.5km', \n", " 'count': 5\n", "} \n", "r = api.request(endpoint, params)\n", "\n", "for tweet in r.get_iterator():\n", " print tweet['text'] + '\\n'" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Exercise\n", "\n", "1. Try some different search queries!\n", "2. Display some more data fields in addition to the tweet text\n", "3. Advanced: can you figure out how to use the API to *post* a tweet?\n", "\n", "Here's the search documentation: https://dev.twitter.com/rest/reference/get/search/tweets\n", "\n", "\n" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": true }, "outputs": [], "source": [] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": true }, "outputs": [], "source": [] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": true }, "outputs": [], "source": [] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Streaming live tweets in real time " ] }, { "cell_type": "code", "execution_count": 24, "metadata": { "collapsed": false }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Tue Oct 06 19:16:53 +0000 2015\n", "Manaus, Amazonas, Brasil\n", "SHAJSJAKDKSKDK\n", "\n", "Tue Oct 06 19:16:53 +0000 2015\n", "Yemen, Yemen\n", "@M8pnE @motalka_m7roma انا زبي قام وصل\n", "\n", "Tue Oct 06 19:16:53 +0000 2015\n", "Quinta de Tilcoco, Chile, Chile\n", "@sacaloprendio he pecado, el ego es incontrolable: la estadística es sólo en Santiago..\n", "http://t.co/xGrRhmBBcs\n", "Me flagelo\n", "\n", "Tue Oct 06 19:16:54 +0000 2015\n", "Paris, Ile-de-France, France\n", "I can't tell you how beautiful this blue is. You just have to go see it. #YvesKlein @ Centre Pompidou https://t.co/imzJavALCg\n", "\n", "Tue Oct 06 19:16:54 +0000 2015\n", "Italia, Italia\n", "Una que me baile Twerk\n", "\n", "Tue Oct 06 19:16:54 +0000 2015\n", "Salwa, Kuwait, دولة الكويت\n", "اللهم اني اسالك ان اتخرج من الثانوي بكامل قواي العقليه .\n", "\n", "Tue Oct 06 19:16:53 +0000 2015\n", "Norway, Norge\n", "@PatrickJD84 @CaribbeanChilly @Grummz and my talents http://t.co/i5WKRSOziR\n", "\n", "Tue Oct 06 19:16:54 +0000 2015\n", "京都府 京都市 伏見区, 日本\n", "きつねかわいい！！！（2243回目）\n", "\n", "Tue Oct 06 19:16:53 +0000 2015\n", "Oaxaca de Juárez, Oaxaca, México\n", "En Semifinales el #TorneoIntramuros2015 de la @Admon_GobOax http://t.co/Hr93Ghls6x .@GabinoCue .@GobOax #Oaxaca http://t.co/IBdTaJA81j\n", "\n", "Tue Oct 06 19:16:54 +0000 2015\n", "Poitiers, Poitou-Charentes, France\n", "@kfdoh Radiohead tambien esta en la lista.\n", "\n", "Tue Oct 06 19:16:53 +0000 2015\n", "South East, England, United Kingdom\n", "@KTHopkins when didn't the police attend to a burglary? Ive had a couple and the police attended each time .. what did I miss?\n", "\n", "Tue Oct 06 19:16:54 +0000 2015\n", "La Línea de la Concepción, Andalucía, España\n", "Pos na de tranquis cn la bal jaja\n", "\n", "Tue Oct 06 19:16:54 +0000 2015\n", "Buenos Aires, Argentina, Argentina\n", "#EternamenteJuntosViciconte\n", "Gonza la 6G Te Espera | ochenta y uno\n", "\n", "Tue Oct 06 19:16:54 +0000 2015\n", "Santa Monica, CA, United States\n", "@DrueMaggetti in studio w me and hrosmakeup 💄in #SantaMonica #dwphotograph #davidwalden.com #fresnel… https://t.co/EsirRHn9Xe\n", "\n", "Tue Oct 06 19:16:53 +0000 2015\n", "Paisley, Scotland, United Kingdom\n", "I can't believe I see one direction live tomorrow\n", "\n", "Tue Oct 06 19:16:53 +0000 2015\n", "Saen Suk, Chon Buri, ประเทศไทย\n", "@nutmeelia http://t.co/IoG3cWmkyy\n", "\n", "Tue Oct 06 19:16:53 +0000 2015\n", "Viana de Cega, Castilla y León, España\n", "No podemos hacer nada por cambiar el rumbo.\n", "\n", "Tue Oct 06 19:16:54 +0000 2015\n", "New York, NY, United States\n", "@NaughtyMabel <3 the tutu, Mabel!\n", "\n", "Tue Oct 06 19:16:53 +0000 2015\n", "Villarrica, La Araucanía, Chile\n", "La Más Granosa xD Jaja Chicas Yo LAS BANCOO ♥ Seguro Que Al Choclo La Miras De Cerca Y Te Regala Granos xD 😂😂😝😝 https://t.co/TOjXqeHsKs\n", "\n", "Tue Oct 06 19:16:54 +0000 2015\n", "Samarahan, Sarawak, Malaysia\n", "Bad molah lawak😒 https://t.co/rpXbV2J7Y1\n", "\n", "Tue Oct 06 19:16:53 +0000 2015\n", "الجيزة, مصر, مصر\n", "#اللهم ارزقني سجدة في بيت #نبيك احكي فيها عن كل ما في قلبي وينتهي بها كربي وارتاح بها من همي \n", "الشيخ د. #ماهر_المعيقلى http://t.co/X1n5WFmbJT\n", "\n", "Tue Oct 06 19:16:54 +0000 2015\n", "Paddington, London, United Kingdom\n", "@Nhash40 @AlkwuaityQ هلا وغلا بنهاش فتى الصغل 😜\n", "\n" ] }, { "ename": "AttributeError", "evalue": "'TwitterResponse' object has no attribute 'close'", "output_type": "error", "traceback": [ "\u001b[1;31m---------------------------------------------------------------------------\u001b[0m", "\u001b[1;31mAttributeError\u001b[0m Traceback (most recent call last)", "\u001b[1;32m\u001b[0m in \u001b[0;36m\u001b[1;34m()\u001b[0m\n\u001b[0;32m 14\u001b[0m \u001b[1;32mif\u001b[0m \u001b[1;33m(\u001b[0m\u001b[0mi\u001b[0m \u001b[1;33m>\u001b[0m \u001b[1;36m20\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m:\u001b[0m \u001b[1;32mbreak\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m 15\u001b[0m \u001b[1;33m\u001b[0m\u001b[0m\n\u001b[1;32m---> 16\u001b[1;33m \u001b[0mr\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mclose\u001b[0m\u001b[1;33m(\u001b[0m\u001b[1;33m)\u001b[0m \u001b[1;31m# close streaming connection\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m", "\u001b[1;31mAttributeError\u001b[0m: 'TwitterResponse' object has no attribute 'close'" ] } ], "source": [ "# Twitter limits simultaneous connections to the streaming API,\n", "# so this part may not work using the demo API keys during class\n", "\n", "endpoint = 'statuses/filter'\n", "params = {'locations': '-180,-90,180,90'}\n", "r = api.request(endpoint, params)\n", "\n", "# 'enumerate' lets us count tweets as we receive them\n", "\n", "for i, tweet in enumerate(r.get_iterator()):\n", " print tweet['created_at']\n", " print tweet['place']['full_name'] + ', ' + tweet['place']['country']\n", " print tweet['text'] + '\\n'\n", " if (i > 20): break\n", "\n", "r.close() # close streaming connection" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Loading tweets into a dataframe" ] }, { "cell_type": "code", "execution_count": 26, "metadata": { "collapsed": false }, "outputs": [ { "ename": "AttributeError", "evalue": "'TwitterResponse' object has no attribute 'close'", "output_type": "error", "traceback": [ "\u001b[1;31m---------------------------------------------------------------------------\u001b[0m", "\u001b[1;31mAttributeError\u001b[0m Traceback (most recent call last)", "\u001b[1;32m\u001b[0m in \u001b[0;36m\u001b[1;34m()\u001b[0m\n\u001b[0;32m 8\u001b[0m \u001b[0mtweets\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mappend\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mtweet\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m 9\u001b[0m \u001b[1;33m\u001b[0m\u001b[0m\n\u001b[1;32m---> 10\u001b[1;33m \u001b[0mr\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mclose\u001b[0m\u001b[1;33m(\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m\u001b[0;32m 11\u001b[0m \u001b[1;32mprint\u001b[0m \u001b[0mlen\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mtweets\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n", "\u001b[1;31mAttributeError\u001b[0m: 'TwitterResponse' object has no attribute 'close'" ] } ], "source": [ "# first, save some tweets to an array instead of just printing them\n", "\n", "r = api.request(endpoint, params)\n", "tweets = []\n", "\n", "for i, tweet in enumerate(r.get_iterator()):\n", " if (i >= 500): break\n", " tweets.append(tweet)\n", "\n", "r.close()\n", "print len(tweets)" ] }, { "cell_type": "code", "execution_count": 27, "metadata": { "collapsed": false }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "[{u'contributors': None, u'truncated': False, u'text': u'@thomasbernardes \\xe9 sim', u'is_quote_status': False, u'in_reply_to_status_id': 651476149676212224L, u'id': 651476328290627584L, u'favorite_count': 0, u'source': u'Twitter Web Client', u'retweeted': False, u'coordinates': None, u'timestamp_ms': u'1444159035166', u'entities': {u'user_mentions': [{u'id': 39062231, u'indices': [0, 16], u'id_str': u'39062231', u'screen_name': u'thomasbernardes', u'name': u'schaeffer'}], u'symbols': [], u'hashtags': [], u'urls': []}, u'in_reply_to_screen_name': u'thomasbernardes', u'id_str': u'651476328290627584', u'retweet_count': 0, u'in_reply_to_user_id': 39062231, u'favorited': False, u'user': {u'follow_request_sent': None, u'profile_use_background_image': False, u'default_profile_image': False, u'id': 3027528856L, u'verified': False, u'profile_image_url_https': u'https://pbs.twimg.com/profile_images/628396788584710144/E9w5oCAa_normal.jpg', u'profile_sidebar_fill_color': u'000000', u'profile_text_color': u'000000', u'followers_count': 129, u'profile_sidebar_border_color': u'000000', u'id_str': u'3027528856', u'profile_background_color': u'000000', u'listed_count': 4, u'profile_background_image_url_https': u'https://abs.twimg.com/images/themes/theme1/bg.png', u'utc_offset': -25200, u'statuses_count': 11833, u'description': u'semelhantes.', u'friends_count': 164, u'location': None, u'profile_link_color': u'DD2E44', u'profile_image_url': u'http://pbs.twimg.com/profile_images/628396788584710144/E9w5oCAa_normal.jpg', u'following': None, u'geo_enabled': True, u'profile_banner_url': u'https://pbs.twimg.com/profile_banners/3027528856/1442813970', u'profile_background_image_url': u'http://abs.twimg.com/images/themes/theme1/bg.png', u'name': u'thalos', u'lang': u'pt', u'profile_background_tile': False, u'favourites_count': 515, u'screen_name': u'kpthales', u'notifications': None, u'url': None, u'created_at': u'Tue Feb 10 01:41:02 +0000 2015', u'contributors_enabled': False, u'time_zone': u'Pacific Time (US & Canada)', u'protected': False, u'default_profile': False, u'is_translator': False}, u'geo': None, u'in_reply_to_user_id_str': u'39062231', u'lang': u'pt', u'created_at': u'Tue Oct 06 19:17:15 +0000 2015', u'filter_level': u'low', u'in_reply_to_status_id_str': u'651476149676212224', u'place': {u'country_code': u'BR', u'url': u'https://api.twitter.com/1.1/geo/id/80e53f07bd506475.json', u'country': u'Brasil', u'place_type': u'city', u'bounding_box': {u'type': u'Polygon', u'coordinates': [[[-54.702115, -31.653423], [-54.702115, -30.838278], [-53.429682, -30.838278], [-53.429682, -31.653423]]]}, u'full_name': u'Bag\\xe9, Rio Grande do Sul', u'attributes': {}, u'id': u'80e53f07bd506475', u'name': u'Bag\\xe9'}}, {u'contributors': None, u'truncated': False, u'text': u'@soz71 initial thinking out loud operation #helpsarah', u'is_quote_status': False, u'in_reply_to_status_id': 651476083821428737L, u'id': 651476327837601792L, u'favorite_count': 0, u'source': u'Twitter for iPhone', u'retweeted': False, u'coordinates': None, u'timestamp_ms': u'1444159035058', u'entities': {u'user_mentions': [{u'id': 255517623, u'indices': [0, 6], u'id_str': u'255517623', u'screen_name': u'soz71', u'name': u'Sarah S'}], u'symbols': [], u'hashtags': [{u'indices': [43, 53], u'text': u'helpsarah'}], u'urls': []}, u'in_reply_to_screen_name': u'Shane2510', u'id_str': u'651476327837601792', u'retweet_count': 0, u'in_reply_to_user_id': 2334163613L, u'favorited': False, u'user': {u'follow_request_sent': None, u'profile_use_background_image': True, u'default_profile_image': False, u'id': 2334163613L, u'verified': False, u'profile_image_url_https': u'https://pbs.twimg.com/profile_images/647775215771123712/WXr58y5v_normal.jpg', u'profile_sidebar_fill_color': u'DDEEF6', u'profile_text_color': u'333333', u'followers_count': 11, u'profile_sidebar_border_color': u'C0DEED', u'id_str': u'2334163613', u'profile_background_color': u'C0DEED', u'listed_count': 4, u'profile_background_image_url_https': u'https://abs.twimg.com/images/themes/theme1/bg.png', u'utc_offset': None, u'statuses_count': 3114, u'description': None, u'friends_count': 50, u'location': u'Plymouth', u'profile_link_color': u'0084B4', u'profile_image_url': u'http://pbs.twimg.com/profile_images/647775215771123712/WXr58y5v_normal.jpg', u'following': None, u'geo_enabled': True, u'profile_banner_url': u'https://pbs.twimg.com/profile_banners/2334163613/1443276376', u'profile_background_image_url': u'http://abs.twimg.com/images/themes/theme1/bg.png', u'name': u'Shane Harris', u'lang': u'en', u'profile_background_tile': False, u'favourites_count': 2839, u'screen_name': u'Shane2510', u'notifications': None, u'url': None, u'created_at': u'Sun Feb 09 18:03:12 +0000 2014', u'contributors_enabled': False, u'time_zone': None, u'protected': False, u'default_profile': True, u'is_translator': False}, u'geo': None, u'in_reply_to_user_id_str': u'2334163613', u'lang': u'en', u'created_at': u'Tue Oct 06 19:17:15 +0000 2015', u'filter_level': u'low', u'in_reply_to_status_id_str': u'651476083821428737', u'place': {u'country_code': u'GB', u'url': u'https://api.twitter.com/1.1/geo/id/7aeb72d9e8a1c889.json', u'country': u'United Kingdom', u'place_type': u'city', u'bounding_box': {u'type': u'Polygon', u'coordinates': [[[-4.203701, 50.360251], [-4.203701, 50.444179], [-4.082159, 50.444179], [-4.082159, 50.360251]]]}, u'full_name': u'Plymouth, England', u'attributes': {}, u'id': u'7aeb72d9e8a1c889', u'name': u'Plymouth'}}, {u'contributors': None, u'truncated': False, u'text': u\"I'm at Michell Som in Recife, PE https://t.co/CZba2ya91M\", u'is_quote_status': False, u'in_reply_to_status_id': None, u'id': 651476327921524736L, u'favorite_count': 0, u'source': u'Foursquare', u'retweeted': False, u'coordinates': {u'type': u'Point', u'coordinates': [-34.94155121, -8.08021069]}, u'timestamp_ms': u'1444159035078', u'entities': {u'user_mentions': [], u'symbols': [], u'hashtags': [], u'urls': [{u'url': u'https://t.co/CZba2ya91M', u'indices': [33, 56], u'expanded_url': u'https://www.swarmapp.com/c/iaAzHNXlDc0', u'display_url': u'swarmapp.com/c/iaAzHNXlDc0'}]}, u'in_reply_to_screen_name': None, u'id_str': u'651476327921524736', u'retweet_count': 0, u'in_reply_to_user_id': None, u'favorited': False, u'user': {u'follow_request_sent': None, u'profile_use_background_image': True, u'default_profile_image': False, u'id': 1112894017, u'verified': False, u'profile_image_url_https': u'https://pbs.twimg.com/profile_images/551388293678387200/I8nLAs2l_normal.jpeg', u'profile_sidebar_fill_color': u'F6FFD1', u'profile_text_color': u'333333', u'followers_count': 1304, u'profile_sidebar_border_color': u'FFF8AD', u'id_str': u'1112894017', u'profile_background_color': u'FFF04D', u'listed_count': 1, u'profile_background_image_url_https': u'https://abs.twimg.com/images/themes/theme19/bg.gif', u'utc_offset': -10800, u'statuses_count': 9089, u'description': None, u'friends_count': 1307, u'location': u'recife', u'profile_link_color': u'F5ABB5', u'profile_image_url': u'http://pbs.twimg.com/profile_images/551388293678387200/I8nLAs2l_normal.jpeg', u'following': None, u'geo_enabled': True, u'profile_banner_url': u'https://pbs.twimg.com/profile_banners/1112894017/1419688227', u'profile_background_image_url': u'http://abs.twimg.com/images/themes/theme19/bg.gif', u'name': u'CineideBetalab2015', u'lang': u'pt', u'profile_background_tile': False, u'favourites_count': 10, u'screen_name': u'CineideBETA2015', u'notifications': None, u'url': None, u'created_at': u'Tue Jan 22 23:17:57 +0000 2013', u'contributors_enabled': False, u'time_zone': u'Brasilia', u'protected': False, u'default_profile': False, u'is_translator': False}, u'geo': {u'type': u'Point', u'coordinates': [-8.08021069, -34.94155121]}, u'in_reply_to_user_id_str': None, u'possibly_sensitive': True, u'lang': u'en', u'created_at': u'Tue Oct 06 19:17:15 +0000 2015', u'filter_level': u'low', u'in_reply_to_status_id_str': None, u'place': {u'country_code': u'BR', u'url': u'https://api.twitter.com/1.1/geo/id/f207b85be9f1513e.json', u'country': u'Brasil', u'place_type': u'city', u'bounding_box': {u'type': u'Polygon', u'coordinates': [[[-35.019805, -8.157554], [-35.019805, -7.929652], [-34.858893, -7.929652], [-34.858893, -8.157554]]]}, u'full_name': u'Recife, Pernambuco', u'attributes': {}, u'id': u'f207b85be9f1513e', u'name': u'Recife'}}, {u'contributors': None, u'truncated': False, u'text': u'Besyocunun hali bi ba\\u015fka oluyor ya :-D :-D @ Kastamonu \\xdcniversitesi Besyo https://t.co/9aiFGLyeBU', u'is_quote_status': False, u'in_reply_to_status_id': None, u'id': 651476328013799424L, u'favorite_count': 0, u'source': u'Instagram', u'retweeted': False, u'coordinates': {u'type': u'Point', u'coordinates': [33.77157419, 41.43254392]}, u'timestamp_ms': u'1444159035100', u'entities': {u'user_mentions': [], u'symbols': [], u'hashtags': [], u'urls': [{u'url': u'https://t.co/9aiFGLyeBU', u'indices': [74, 97], u'expanded_url': u'https://instagram.com/p/8ge_6TBRwf/', u'display_url': u'instagram.com/p/8ge_6TBRwf/'}]}, u'in_reply_to_screen_name': None, u'id_str': u'651476328013799424', u'retweet_count': 0, u'in_reply_to_user_id': None, u'favorited': False, u'user': {u'follow_request_sent': None, u'profile_use_background_image': False, u'default_profile_image': False, u'id': 2983706218L, u'verified': False, u'profile_image_url_https': u'https://pbs.twimg.com/profile_images/643064819168509952/2Gl-yitB_normal.jpg', u'profile_sidebar_fill_color': u'000000', u'profile_text_color': u'000000', u'followers_count': 211, u'profile_sidebar_border_color': u'000000', u'id_str': u'2983706218', u'profile_background_color': u'000000', u'listed_count': 1, u'profile_background_image_url_https': u'https://abs.twimg.com/images/themes/theme1/bg.png', u'utc_offset': None, u'statuses_count': 609, u'description': u'Sahte kahramanlarla i\\u015fim olmaz benim kahraman\\u0131m Atat\\xfcrk. Ya\\u015fas\\u0131n \\xfclk\\xfcc\\xfc T\\xfcrk gen\\xe7li\\u011fi #CcC#MHP', u'friends_count': 454, u'location': u'kayseri develi', u'profile_link_color': u'3B94D9', u'profile_image_url': u'http://pbs.twimg.com/profile_images/643064819168509952/2Gl-yitB_normal.jpg', u'following': None, u'geo_enabled': True, u'profile_banner_url': u'https://pbs.twimg.com/profile_banners/2983706218/1442057111', u'profile_background_image_url': u'http://abs.twimg.com/images/themes/theme1/bg.png', u'name': u'Erhan Mahmut', u'lang': u'tr', u'profile_background_tile': False, u'favourites_count': 584, u'screen_name': u'erhan_mahmut', u'notifications': None, u'url': None, u'created_at': u'Sun Jan 18 07:22:49 +0000 2015', u'contributors_enabled': False, u'time_zone': None, u'protected': False, u'default_profile': False, u'is_translator': False}, u'geo': {u'type': u'Point', u'coordinates': [41.43254392, 33.77157419]}, u'in_reply_to_user_id_str': None, u'possibly_sensitive': False, u'lang': u'tr', u'created_at': u'Tue Oct 06 19:17:15 +0000 2015', u'filter_level': u'low', u'in_reply_to_status_id_str': None, u'place': {u'country_code': u'TR', u'url': u'https://api.twitter.com/1.1/geo/id/6d7ab717d3a54d55.json', u'country': u'T\\xfcrkiye', u'place_type': u'city', u'bounding_box': {u'type': u'Polygon', u'coordinates': [[[33.750449, 41.332337], [33.750449, 41.435605], [33.814936, 41.435605], [33.814936, 41.332337]]]}, u'full_name': u'Kastamonu, T\\xfcrkiye', u'attributes': {}, u'id': u'6d7ab717d3a54d55', u'name': u'Kastamonu'}}, {u'contributors': None, u'truncated': False, u'text': u'Rabbim do\\u011frular\\u0131 g\\xf6ren ve do\\u011fru konu\\u015fanlar\\u0131n say\\u0131s\\u0131n\\u0131 artt\\u0131rs\\u0131n... http://t.co/XsISXBOMqf', u'is_quote_status': False, u'in_reply_to_status_id': None, u'id': 651476327007170560L, u'favorite_count': 0, u'source': u'Twitter for iPhone', u'retweeted': False, u'coordinates': None, u'timestamp_ms': u'1444159034860', u'entities': {u'user_mentions': [], u'symbols': [], u'hashtags': [], u'urls': [], u'media': [{u'expanded_url': u'http://twitter.com/Ayenurzderya/status/651476327007170560/photo/1', u'display_url': u'pic.twitter.com/XsISXBOMqf', u'url': u'http://t.co/XsISXBOMqf', u'media_url_https': u'https://pbs.twimg.com/media/CQqCQSJUsAAZPOF.jpg', u'id_str': u'651476312368918528', u'sizes': {u'large': {u'h': 606, u'resize': u'fit', u'w': 640}, u'small': {u'h': 321, u'resize': u'fit', u'w': 340}, u'medium': {u'h': 568, u'resize': u'fit', u'w': 600}, u'thumb': {u'h': 150, u'resize': u'crop', u'w': 150}}, u'indices': [67, 89], u'type': u'photo', u'id': 651476312368918528L, u'media_url': u'http://pbs.twimg.com/media/CQqCQSJUsAAZPOF.jpg'}]}, u'in_reply_to_screen_name': None, u'id_str': u'651476327007170560', u'retweet_count': 0, u'in_reply_to_user_id': None, u'favorited': False, u'user': {u'follow_request_sent': None, u'profile_use_background_image': True, u'default_profile_image': False, u'id': 408349043, u'verified': False, u'profile_image_url_https': u'https://pbs.twimg.com/profile_images/463768014542749696/vIA_8cnE_normal.jpeg', u'profile_sidebar_fill_color': u'DDEEF6', u'profile_text_color': u'333333', u'followers_count': 20, u'profile_sidebar_border_color': u'C0DEED', u'id_str': u'408349043', u'profile_background_color': u'C0DEED', u'listed_count': 0, u'profile_background_image_url_https': u'https://abs.twimg.com/images/themes/theme1/bg.png', u'utc_offset': None, u'statuses_count': 73, u'description': u'\\u0130stanbul', u'friends_count': 152, u'location': None, u'profile_link_color': u'0084B4', u'profile_image_url': u'http://pbs.twimg.com/profile_images/463768014542749696/vIA_8cnE_normal.jpeg', u'following': None, u'geo_enabled': True, u'profile_banner_url': u'https://pbs.twimg.com/profile_banners/408349043/1436256720', u'profile_background_image_url': u'http://abs.twimg.com/images/themes/theme1/bg.png', u'name': u'Ay\\u015fenur \\xd6zderya', u'lang': u'tr', u'profile_background_tile': False, u'favourites_count': 11, u'screen_name': u'Ayenurzderya', u'notifications': None, u'url': None, u'created_at': u'Wed Nov 09 09:29:37 +0000 2011', u'contributors_enabled': False, u'time_zone': None, u'protected': False, u'default_profile': True, u'is_translator': False}, u'geo': None, u'in_reply_to_user_id_str': None, u'possibly_sensitive': False, u'lang': u'tr', u'created_at': u'Tue Oct 06 19:17:14 +0000 2015', u'filter_level': u'low', u'in_reply_to_status_id_str': None, u'place': {u'country_code': u'TR', u'url': u'https://api.twitter.com/1.1/geo/id/5e02a0f0d91c76d2.json', u'country': u'T\\xfcrkiye', u'place_type': u'city', u'bounding_box': {u'type': u'Polygon', u'coordinates': [[[28.632104, 40.802734], [28.632104, 41.239907], [29.378341, 41.239907], [29.378341, 40.802734]]]}, u'full_name': u'\\u0130stanbul, T\\xfcrkiye', u'attributes': {}, u'id': u'5e02a0f0d91c76d2', u'name': u'\\u0130stanbul'}, u'extended_entities': {u'media': [{u'expanded_url': u'http://twitter.com/Ayenurzderya/status/651476327007170560/photo/1', u'display_url': u'pic.twitter.com/XsISXBOMqf', u'url': u'http://t.co/XsISXBOMqf', u'media_url_https': u'https://pbs.twimg.com/media/CQqCQSJUsAAZPOF.jpg', u'id_str': u'651476312368918528', u'sizes': {u'large': {u'h': 606, u'resize': u'fit', u'w': 640}, u'small': {u'h': 321, u'resize': u'fit', u'w': 340}, u'medium': {u'h': 568, u'resize': u'fit', u'w': 600}, u'thumb': {u'h': 150, u'resize': u'crop', u'w': 150}}, u'indices': [67, 89], u'type': u'photo', u'id': 651476312368918528L, u'media_url': u'http://pbs.twimg.com/media/CQqCQSJUsAAZPOF.jpg'}]}}]\n" ] } ], "source": [ "# the raw data is very messy though!\n", "\n", "print tweets[0:5]" ] }, { "cell_type": "code", "execution_count": 28, "metadata": { "collapsed": false }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "80\n" ] } ], "source": [ "# we'll pull out some pieces into a dataframe\n", "\n", "# first, set up a dictionary of empty arrays\n", "d = {'place': [], 'latitude': [], 'longitude': []}\n", "\n", "for t in tweets:\n", " try:\n", " # first check whether the fields we want exist\n", " _test = t['coordinates']['coordinates']\n", " \n", " # then pull out the data\n", " d['place'].append(t['place']['name'])\n", " d['latitude'].append(t['coordinates']['coordinates'][1])\n", " d['longitude'].append(t['coordinates']['coordinates'][0])\n", " \n", " except:\n", " # if the test failed, continue to next tweet\n", " continue\n", "\n", "# load it into a dataframe\n", "df = pd.DataFrame.from_dict(d)\n", "\n", "print len(df)" ] }, { "cell_type": "code", "execution_count": 29, "metadata": { "collapsed": false }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ " latitude longitude place\n", "0 -8.080211 -34.941551 Recife\n", "1 41.432544 33.771574 Kastamonu\n", "2 3.094285 103.085270 Bebar\n", "3 39.091116 -94.415507 Independence\n", "4 34.967096 135.772691 京都市 伏見区\n" ] } ], "source": [ "print df.head()" ] }, { "cell_type": "code", "execution_count": 30, "metadata": { "collapsed": false }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ " latitude longitude place\n", "21 41.007700 39.617398 Akçaabat\n", "64 32.930000 -111.580000 Arizona\n", "45 29.274240 47.932065 Ashbeliah\n", "17 39.330337 26.663693 Ayvalık\n", "2 3.094285 103.085270 Bebar\n", "34 42.347485 -71.076020 Boston\n", "69 -15.783500 -47.899164 Brasília\n", "26 51.454582 -2.598296 Bristol\n", "9 51.505778 -0.099747 Camberwell\n", "14 44.861972 -93.571570 Chanhassen\n", "36 3.736233 -75.423942 Chaparral\n", "16 36.810974 -119.713646 Clovis\n" ] } ], "source": [ "print df.sort('place').head(12)" ] }, { "cell_type": "code", "execution_count": 31, "metadata": { "collapsed": false }, "outputs": [ { "data": { "text/plain": [ "" ] }, "execution_count": 31, "metadata": {}, "output_type": "execute_result" }, { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYwAAAEPCAYAAABRHfM8AAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAHTpJREFUeJzt3X+U3HV97/HnexP3sPKjcUmbyI9CpPyQH6YJCrTgzZ5j\ndhe5NRBSpbfUs1IPOZXWcMiKIcZKrpIGwaWi91Ka2qtbFCgWw0nuoZksHDeY2oKSACEhBnqJ1whZ\nWLAVdLkB9n3/+H4nOzuZ2Xzn1/fHzOtxzpzM9zO/3jM7+b7n89vcHRERkcNpSzoAERHJBiUMERGJ\nRAlDREQiUcIQEZFIlDBERCQSJQwREYkk0YRhZjPM7J/M7Bkz22Vm55tZp5kNmdkeM9tsZjOSjFFE\nRAJJ1zBuBx509/cC7wN2AzcAQ+5+GvBweCwiIgmzpCbumdlvANvd/T1F5buBBe4+YmazgWF3PyOR\nIEVE5KAkaxhzgJfN7Jtmts3M/s7MjgRmuftIeJ8RYFZyIYqISF6SCWM6MB+4w93nA7+iqPnJg+qP\n1i4REUmB6Qm+9j5gn7v/KDz+J2AlsN/MZrv7fjN7N/BS8QPNTElERKQK7m7VPjaxGoa77wd+Zman\nhUULgZ3ARqAvLOsDHijz+MxebrzxxsRjUPzJx6H4s3fJcuzutf/OTrKGAfBp4Dtm1g78O3AVMA24\nz8w+CewFPpZceCIikpdownD3J4EPlLhpYdyxiIjI1JKeh9GSurq6kg6hJoo/WYo/OVmOvR4Sm4dR\nCzPzLMYtIpIkM8Oz2OktIiLZooQhIiKRKGGIiEgkShgiIhKJEoY0hVwuR0/PEnp6lpDL5RSLSANo\nlJRkXi6XY/HiPsbGvgxAR8cK1q8fpLe3t6VjESlW6ygpJQzJvJ6eJQwNLWJiRZlBurs3sHnz/S0d\ni0gxDasVEZFYJL2WlEjN+vuXsnVrH2NjwXFHxwr6+wdbPhaRelOTlDSFXC7HwMA6IDhpJ9lnkKZY\nRAqpD0NkCoUn7+OOO5qNG7cCsHz5VaxatSrJ0ERiV2vCUJOUNK3iEUuwDLgaOIfPf34ZgJKGSAVU\nw5CmVWrEEmwA7gcG6ez8Eq+88lxi8YnETaOkREQkFmqSkqZVPGJpoklqEFjG8uWfTSw2kSxSk5Q0\ntSid3hrVJK1Co6REKlSYIBYsmM+aNV/XUh7SEpQwRCpQPHKqra2f8fEBtJSHtAINqxWpwMDAujBZ\nBAlifPzOZAMSyRAlDGlxF9LWdh3j48GRlvIQKU8JQ1rKoWs9fZtVq/rZsmVDeLv6L0TKUR+GtByN\nipJWpU5vERGJRDO9RUQkFkoYKaA9oFuX/vaSJWqSSlhSe0CrHT952v9b4qZ5GBlXPC9gbIyDJ/JG\nndCLT1Rbt/bpRJWAcn97/R0krdQklUKjoyMsXtzH0NAihoYWsXhxX12bKyafqILEkU9OMkHNRSKT\nqYaRsFJ7QMMZ+uWZsDhqYdr/W7JGNYyE9fb2sn59sH5Rd/cG1q8fZObMYxv6mv39S8PENAgMhieq\npQ19zTjUs0YQRy2s1N9ePwok1dw9c5cg7Oa1adMm7+iY5fAth295R8cs37RpU1XP0919uXd3X37I\n46e6LYvq9ZnldXdfHj6Xh5dveXf35XWMWCR+4bmz+nNvLQ9O6tLsCcO99hP6pk2bvL19hsMFDhd4\ne/uMpkgM5dT7BF/vBCSSBrUmDDVJpVRvby+bN9/P5s33V9VMsXLllzhwYDrwZ8CfceDAdFau/FLd\n42xW1TYXTdUspk50ybxask1SF5q0hlHPZqLOzlMO+cXd2XlKnSJNnzTUCKaK4XC3NVPzoKQXWW+S\nAqYB24GN4XEnMATsATYDM0o8pq4fYhrU64TX19fn06f/lkPnIQlj3rwF9Q88RYpPvHGfiKdqFit3\nWxoSnbSOWhNGGobVXgvsAo4Oj28Ahtz9FjNbER7fkFRwcanHJK7u7m4eeugx4GvADmDZwdva269n\n7dq76hpz2vT29h78vLIyOVGT9yRLEu3DMLMTgEuAbwD56eqLCMZ7Ev57WQKhZU4ul+Ohhx4nSBZ9\nwFeAq4F+urs3sGHDXS11EkpicuJUw5WbdSiztJakaxh/DVwPHFNQNsvdR8LrI8Cs2KOKWS6XY3R0\npKad34KT4elFpecwffo07U8dk97eXlat+jS33RYMLli+/NMHk3S+E31gYB2joyPAGQwMrGPBgvls\n3bpCk/ckExJLGGb2B8BL7r7dzLpK3cfd3cxKrjK4evXqg9e7urro6ir5FDVZs2YNt932TQCWL7+K\nVatW1f01Jjed7KCtrZ+5c89m7dpqmk8uBFYUHC/jyisX1y/YFCteTDGJWdS5XI41a75+sBlszZoV\nvP/975+UNAAWLfo4Bw7cCsCWLdfzhS9cqx3/pCGGh4cZHh6u3xPW0gFSywX4K+BnwPPAi8CvgLuA\n3cDs8D7vBnaXeGz9eoHKuOmmmxyOOdgZCcf4TTfdVPfXObQztN87O0+puKN2ovO0P5x78S5fuHBh\n3eNNo3Idx8l1em9yuNzhAp8378JJ95k3b0HLDUaQ9CDro6SC98ACJkZJ3QKsCK/fANxc4v51/AhL\ni2tY6uSEsclhZtUjZlp1eGZaZmUHcfQ7TCSvtrZ3TfpbtNpwZ0mXWhNG0n0YhfJNTzcD95nZJ4G9\nwMcSiygGk5tO7iTorK5uxEzhKKHWtoPHH3+Snp4lse710d+/lIcfvpLx8QHyf8Px8cl/w5NOms2r\nr36m4FGf4aSTivueRNIpFTO93X2Luy8Kr7/q7gvd/TR373H3/0gipuXLryIYljoYXpbxkY9cVPfX\nKZxR3Nn5ctXP08qziCePQPoM8He8+upfNmRp+Kn09vYyd+7ZU95nyZIPY/Y28HngZtrb32Lt2r+M\nJT6RmtVSPUnqQkwT9/r6+sIJcCc4LGn4pKpqJ3Fp8tdEc1y5Jp+4Po9KZnu3tb2rIf1iIuXQDH0Y\nFQcdU8JIom28mn6ItLThp0GpzwIuiDWJlvsb6u8kSas1YaSpD0NQP0StiofTBsOMBxkb2x/bDGr9\nDbMlWCFhOwALF85jaGgo4YhSrJZsk9SFmGoYWWnqyUqccdm0aVPYNHVBOPIsHb/m9XdKhyOOOCJs\nau4M53lNHj7fzMPRqbGGYcFzZIuZeVxxF08IS+svx6zEGZfitaQ6OlakYi0p/Z2S1dHRwRtvtBMs\noQPBwJZzgK3h8SDQj/toEuE1nJnh7nb4e5Z5vBKGNCudnKWY2bHAbeSHPU+MrHu54FgJoxz1YWRE\n4clvwYL5bNmyDdCJcCrqS5Bo3mZivdNlLFx4XpLBpJpqGBlQ3LwSVKOvBs5JTVOLSBaUapIyew33\nTqD5O73VJJWwOJo9enqWMDS0iMnV6A3A/UAw6U8r0opEEySNdwJwxBG/ZmxiSF3TU5NUgrKySY+I\nTGilBFFvqmHUYOKX/2xgHfAC8+ZNY9u2rYd5ZGXUJCUi9aAmqQQFCWMO8G0gOJm3tV3Hgw/eU/cT\nuDq9RaRWShgJyuVyXHLJ5NVJ1acgImlVa8JIxWq1WRVldVIRkWahhFGjtWtXFiytPRhuBbo08uOr\nWZa8lZcyF5HkqEmqDqodWlvN8hVpXfJCRNJPfRgZVmp+xeH6P6p5jGSbljiRetE8DJEmprk+kiZK\nGAkq3rsh6P8YrPtjJLsGBtaFyaK6fd5F6kkJI0H5/bwnmhsO/8uxmseIiNSD+jBEUkyDHKSe1Okt\n0uTU6S31ooQhIiKRaKa3iIjEQgkjozTbW0TipiapDFJHqIhUQ30YLUizvUWkGurDEBGRWGjiXgZp\ntreIJEFNUhmlsfkiUin1YYiISCTqwxARkVgoYYiISCRKGCIiEokShoiIRJJYwjCzE83s+2a208ye\nNrNlYXmnmQ2Z2R4z22xmM5KKsR60hIeINIvERkmZ2Wxgtrs/YWZHAY8DlwFXAaPufouZrQDe5e43\nFD02E6OktISHiKRJ0wyrNbMHgP8RXha4+0iYVIbd/Yyi+2YiYWgJDxFJk6YYVmtmJwPzgEeBWe4+\nEt40AsxKKKyGGB19RU1UIpJJiS8NEjZH3Q9c6+6vmU0kP3d3M0t/VaKM4iU82tuvZ+fONzlw4KsA\nbN3apyYqEcmMRBOGmb2DIFnc5e4PhMUjZjbb3feb2buBl0o9dvXq1Qevd3V10dXV1eBoK9fb28v6\n9YMHl/AYHT2N7duvJt9ENTYGAwPrYk0YWlJEpHUMDw8zPDxct+dLstPbgEHgFXe/rqD8lrDsy2Z2\nAzAjq53exZLu01AnvEhry2ynt5ldBDwCPAXkg1gJPAbcB/w2sBf4mLv/R9FjM5kwkj5hJ52wRCRZ\ntSaMxJqk3H0r5TvdF8YZS1yKm6j6+/XrXkSyIzXDaiuR1RpG0pKu4YhIshreJGVmpwN3EEyyO8vM\n3gcscvebqn3RWilhVE+d3iKtK46E8QhwPXCnu88LO6ufdvezqn3RWilhiIhULo6Je+9090fzB+GZ\n+s1qX1BERLIpSsJ42cx+J39gZn8IvNi4kEREJI2iJIy/AP4WOMPMXgCuAz7V0KhERA5DK0HHL/Io\nKTM7Emhz99caG1KkWNSHIdLCNOKvOg3r9Daz/oLDQ+7k7rdV+6K1UsIQaW2ahFqdRk7cO5ogUZwO\nfADYABjwBwSzsUVEpIWUTRjuvhrAzH4AzM83RZnZjcCDsUQnIlJC8UrQHR0r6O8fTDaoFhBlHsZP\ngLnu/kZ4fATwpLufHkN85WJSk5RIi9Mk1MrFMXFvFXAF8D2CJqnLgH9097+q9kVrpYQhIlK5WFar\nNbNzgQ8S9Gk84u7bq33BelDCEBGpXBw1jN/OXw3/dQB3/7/VvmitlDBERCoXR8J4molhtUcAc4Cf\naC0pEZFsafh+GO5+dtELzgf+vNoXFBGRbIqyNMgk7r4NOL8BsYiISIodtoZRNOO7DZgP/LxhEYmI\nSCpF2aI1P+Mb4C3gfwOafy8i0mKiJIxd7n5fYYGZfRT4bmNCEhGRNIoySmq7u887XFmcNEpKRKRy\nDRslZWYfBi4BjjezrzExD+NotOOeiEjLmapJ6gXgceDS8N98wvglwSZKIiLSQqI0Sb3D3VNVo1CT\n1NS0KJuIlNLIDZS+6+4fNbMdJW52d39ftS9aKyWM8rQTmYiU08iEcZy7v2BmJzHRHJXn7v7Tal+0\nVkoY5QU7kc0Bng9L5tDd/bx2IhORmhNG2Zne7v5CePUad99beAGuqfYFpbFGR0eAQWBReBkMy0RE\nahNlaZCeEmWX1DsQqZfpwFcI9jruC69HmW4jIjK1qYbVfoqgJnFKUT/G0cC/NDowqc7MmcdGKhMR\nqdRUPz3vBv4ZuBlYwUQ/xmvu/kqjA5PK5XI5RkdHaGu7jvHxoEx7HYtIvUTacQ/AzH6LYD8MQBso\npc3k0VE7aGv7FnPnns3atSs1QkpEgAZ2ehe8wCIze5Zg2M0WYC9BzUNSZGBgXZgsgn6L8fEBZs48\nVslCUimXy9HTs4SeniXkcrmkw5GIonR63wT8HrDH3ecAHwIebWhUItK08rXhoaFFDA0tYvHiPiWN\njIgyfOZNdx81szYzm+bu3zez2xsemVSkv38pW7f2MTYWHKvvQtJqcm0YxsaCMtWG0y9KwviFmR0N\n/AD4jpm9BLze2LCkUr29vaxfP1iwJIhmd4tIfUVZS+ooYIyg+epK4BjgO0mOlFKnt0h2afma5DRs\naZAkmdnFwFeBacA33P3LRbcrYYhkmBbITEYj15J6nYmtWYu5ux9T7YtOGZDZNOAnwEKCvcN/BPw3\nd3+m4D5KGCIiFWrYBkruflS1T1qj84DnwjWrMLN7CfbkeGaqB4mISGNFGVYbt+OBnxUc7wvLREQk\nQWlMGGprEhFJoTQuY/pz4MSC4xMJahmTrF69+uD1rq4uurq6Gh2XiEimDA8PMzw8XLfnS90oKTOb\nTtDp/SGCfcUfQ53eIiI1a/haUnFz97eAvwBywC7gHwuThUxo1Ho8WudHREpJXQ0jCtUwGjf5SZOq\nRJpXU07cOxwljPze3YvIr8cDg3R3b6h57+5GPa+IJK/pmqRERCSd0jhKSiJo1Oq0WvVWRMpRk1SG\nNWo9Hq3zI9Kc1IchIiKRqA9DRERioYQhIiKRKGGIiEgkShgiIhKJEoaIiESihCEiIpEoYUgstKCh\nSPZpHoY0nBY0FEkHTdyT1NOChiLpoIl7knqjoyPAncASgm1ORCSLtPigNFQul2Pnzj3ArWHJn9De\n/hb9/fcmGZaIVEEJQxpqYGAdBw7cykRzFJx11jfVfyGSQWqSktjNnHls0iGISBVUw5CG0v4aIs1D\no6Sk4bS/hkg6aFit6IQsIpEoYbQ4TYoTkaiUMFqcJsWJSFSauCciIrHQKKmM0ygkEYmLmqSagDq9\nRSQK9WGIiEgk6sMQEZFYKGFIxbQZkkhrUpOUVETzPkSyS30YEivN+xDJLvVhiIhILDQPQyqieR8i\nrUtNUlIxzfsQySb1YYiISCTqwxARkVgkkjDM7FYze8bMnjSz75nZbxTcttLMnjWz3WbWk0R8IiJy\nqKRqGJuBs9x9LrAHWAlgZmcCVwBnAhcDd5iZakEiIimQyMnY3YfcfTw8fBQ4Ibx+KXCPu7/p7nuB\n54DzEghRRESKpOHX+58CD4bXjwP2Fdy2Dzg+9ohEROQQDZuHYWZDwOwSN33O3TeG91kFHHD3u6d4\nqpLDoVavXn3weldXF11dXVXHKiLSjIaHhxkeHq7b8yU2rNbMPgFcDXzI3d8Iy24AcPebw+NNwI3u\n/mjRYzWsVkSkQpkcVmtmFwPXA5fmk0VoA/BHZtZuZnOAU4HHkohRKqMVbEWaXyI1DDN7FmgHXg2L\n/tXdrwlv+xxBv8ZbwLXufsjZRzWMdFmzZg1f+MIA4+N/DWgFW5G00kxvSVQul+OSS65kfHwArWAr\nkm6ZbJKS5jEwsI7x8VOTDkNEYqDVaqVquVyOxx9/ErgMWHGwvK3tOvr770ksLhFpDCUMqcrEznt/\nAgwSNEfdSVvbs3zxi/3qvxBpQkoYUpWBgXXhNq19QDewms7Ol7n77u8oWYg0KSUMqYNeYD/nnrtB\nyUKkiSlhSFW0855I69GwWqmadt4TyRbNwxARkUg0D0NERGKhhCEiIpEoYYiISCRKGCIiEokShoiI\nRKKEISIikShhiIhIJEoYIiISiRKGiIhEooQhIiKRKGGIiEgkShgiIhKJEoaIiESihCEiIpEoYTSx\nXC5HT88SenqWkMvl6n5/EWkt2g+jSeVyORYv7gv33Q52xFu/frDsJkeV3l9EskcbKElJPT1LGBpa\nBPSFJYN0d29g8+b763J/EckebaAkDTM6+krSIYhIikxPOgBpjP7+pWzd2sfYWHDc0bGC/v7BKe+/\nZcvHOXAgX/IZdu58i1wup2YpEQHUJNXUcrkcAwPrgCAhHO7EP3/+RWzf/jZwHLAU2K9mKZEmUmuT\nlGoYTay3t7ei2sHMmbOAyf0YIiJ5ShhyUKXNWCLSWtQkJZNU2owlItmhYbUiIhKJhtWKiEgslDBE\nRCQSJQwREYkk0YRhZv1mNm5mnQVlK83sWTPbbWY9ScYnIiITEksYZnYi0A38tKDsTOAK4EzgYuAO\nM2u6WtDw8HDSIdRE8SdL8Scny7HXQ5In49uAzxaVXQrc4+5vuvte4DngvLgDa7Ssf+kUf7IUf3Ky\nHHs9JJIwzOxSYJ+7P1V003HAvoLjfcDxsQUmIiJlNWymt5kNAbNL3LQKWAkU9k9MNS5YEy5ERFIg\n9ol7ZnY28DDw67DoBODnwPnAVQDufnN4303Aje7+aNFzKImIiFQh0zO9zex54Fx3fzXs9L6boN/i\neOAh4Hc0rVtEJHlpWHzwYDJw911mdh+wC3gLuEbJQkQkHRKvYYiISDakfo6DmX3UzHaa2dtmNr+g\n/GQzGzOz7eHljoLbzjWzHeEEwNuTibx87OFtJScopiX2Yma22sz2FXzeHy64LROTLc3s4jDGZ81s\nRdLxRGFme83sqfAzfyws6zSzITPbY2abzWxG0nHmmdn/MrMRM9tRUFY23rR9d8rEn4nvvpmdaGbf\nD885T5vZsrC8fp+/u6f6ApwBnAZ8H5hfUH4ysKPMYx4DzguvPwhcnLLYzwSeAN4Rvo/nmKjtpSL2\nEu/lRmB5ifJS76Ut6XhLxDktjO3kMNYngPcmHVeEuJ8HOovKbgE+G15fAdycdJwFsX0QmFf4f7Nc\nvGn87pSJPxPffYJRqb8bXj8K+Anw3np+/qmvYbj7bnffE/X+ZvZu4Gh3fyws+gfgsoYEdxhTxF5q\nguL5aYq9jFKjK7Iy2fI84Dl33+vubwL3EsSeBcWf+yImtkMcJEXfEXf/AfCLouJy8abuu1MmfsjA\nd9/d97v7E+H114FnCAYP1e3zT33COIw5YRVx2MwuCsuOZ/Lkv5+Tvsl/5SYoFpenLfZPm9mTZvb3\nBdXarEy2PB74WcFxWuMs5sBDZvZjM7s6LJvl7iPh9RFgVjKhRVYu3qx8dyBj330zO5mgpvQodfz8\n0zBKaqpJfp9z941lHvYCcKK7/yLsH3jAzM5qWJBlVBl7Kh1msuXfAF8Mj78EDACfLPNUaRxJkcaY\norjQ3V80s98Ehsxsd+GN7u5ZmpcUId40vpdMfffN7CjgfuBad3/NbKJyVOvnn4qE4e7dVTzmAHAg\nvL7NzP4dOJXgV/kJBXfNTwxsiGpiJ4jnxILjEwiye6yxF4v6XszsG0A+GZZ6L7HFXIHiOE9k8q+r\nVHL3F8N/Xzaz9QRNBiNmNtvd94fNmC8lGuThlYs3E98ddz/4+ab9u29m7yBIFne5+wNhcd0+/6w1\nSR1MlWY208ymhdffQ5As/k/4H+yXZna+Ban148ADJZ8tXoVtoBuAPzKzdjObQxD7Y+6+n3TGnu8b\nylsM5EeRlHwvcccXwY+BUy0YXddOsCryhoRjmpKZvdPMjg6vH0mwnM4Ogrj7wrv1kZLvyBTKxZuJ\n705WvvvhOePvgV3u/tWCm+r3+SfVo19Bz/9igrbnMWA/8M9h+RLgaWA78DjwXwsecy7BH/U54Gtp\niz287XNhfLuB3rTFXuK9/APwFPBk+IWbdbj3krYL8GGCkSPPASuTjidCvHMIRrE8EX7XV4blnQSr\nIOwBNgMzko61IOZ7CJqLD4Tf/aumijdt350S8f9pVr77wEXAePh92R5eLq7n56+JeyIiEknWmqRE\nRCQhShgiIhKJEoaIiESihCEiIpEoYYiISCRKGCIiEokShrQkM3u9Ac/5kfyy6WZ2mZm9t4rnGDaz\nc+sdm0g9KGFIq6r7BCR33+juXw4PLyNYPrripyEF6xGJlKKEIS3NArdasGnVU2b2sbC8K/y1/10z\ne8bMvl3wmEvCsh+b2dfMbGNY/gkz+7qZ/R7wEeBWM9tmZu8prDmEy9o8H17vMLN7zWyXmX0P6Ch4\nnR4z+6GZPW5m94XLg4gkJhWLD4ok6HJgLvA+4DeBH5nZI+Ftv0tQS3gR+Bcz+31gG3An8EF3/6mZ\n3U1RjcDd/9XMNgAb3f17AOEKoaVqDp8CXnf3M83snPD5MbOZBKsEf8jdx8KmruUEq6WKJEIJQ1rd\nRcDdHqyR85KZbQE+APySYEHIFwDM7AmCtZ1+TbDI5U/Dx98DLC3z3KU23Sn2QeB2AHffYWZPheUX\nECSrH4bLU7cDP6zwvYnUlRKGtDrn0BN7vibw/wrK3ib4/1JcS5gqKRTe9y0mmoCPOMxz5I+H3P2P\np3h+kVipD0Na3Q+AK8ysLdyk6L8QLPFcKhE4wWq37zGzk8KyKyjd1PQacEzB8V7g/eH1PywofwT4\nYwAzO5ugacyBfwMuNLNTwtuONLNTK353InWkhCGtygHcfT0TS1c/DFzvwYY5Jfsc3P0N4Bpgk5n9\nmKDp6j8LnjP/mHuB68MO6znAV4BPmdk24NiC+/0NcJSZ7QL+O8G+Hbj7KPAJ4B4ze5KgOer0ur17\nkSpoeXORCpnZke7+q/D6/wT2uPvtCYcl0nCqYYhU7moz225mOwmanf426YBE4qAahoiIRKIahoiI\nRKKEISIikShhiIhIJEoYIiISiRKGiIhEooQhIiKR/H9FxtqktdrfXQAAAABJRU5ErkJggg==\n", "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "df.plot(x='longitude', y='latitude', kind='scatter')" ] }, { "cell_type": "code", "execution_count": 32, "metadata": { "collapsed": false }, "outputs": [], "source": [ "# Note that when working with text strings that include characters from \n", "# other alphabets, you need to keep track of the text encoding.\n", "\n", "# Some interesting related reading:\n", "# - http://www.joelonsoftware.com/articles/Unicode.html\n", "\n", "df.to_csv('saved_coords.csv', encoding='utf-8')" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": true }, "outputs": [], "source": [] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Exercise for the remainder of class\n", "\n", "Choose one:\n", "\n", "1. Using one of the APIs from this demo, save and graph a different aspect of the data. \n", "  \n", "\n", "2. Or, search the web for another API that provides data you're interested in. Can you figure out how to connect to it using Python code?\n", "\n", "Some common terms for describing these APIs that operate over the web are \"HTTP\" and \"REST\". The most frequent data format they provide is JSON, but with some code modifications you can parse other formats as well." ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": true }, "outputs": [], "source": [] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": true }, "outputs": [], "source": [] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": true }, "outputs": [], "source": [] } ], "metadata": { "kernelspec": { "display_name": "Python 2", "language": "python", "name": "python2" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 2 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython2", "version": "2.7.10" } }, "nbformat": 4, "nbformat_minor": 0 }