{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "[![Py4Life](https://raw.githubusercontent.com/Py4Life/TAU2015/gh-pages/img/Py4Life-logo-small.png)](http://py4life.github.io/TAU2015/)\n", "# Using web APIs with Python - 9.6.2015\n", "### Last update: 9.6.2015\n", "### Tel-Aviv University / 0411-3122 / Spring 2015" ] }, { "cell_type": "code", "execution_count": 1, "metadata": { "collapsed": true }, "outputs": [], "source": [ "%matplotlib inline\n", "from IPython.display import HTML\n", "import matplotlib.pyplot as plt\n", "import requests\n", "import pandas as pd\n", "import numpy as np\n", "import seaborn as sns\n", "sns.set_style('white')\n", "sns.set_context('talk', font_scale=1.2)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## The Very Hungry Caterpillar example" ] }, { "cell_type": "code", "execution_count": 2, "metadata": { "collapsed": false }, "outputs": [ { "data": { "text/html": [ "
Hmm, I don't know about this caterpillar rearing manual. I thought P.rapae had an obligate association w/ Brassica. pic.twitter.com/M10dqbOYlN
— Christie Bahlai (@cbahlai) מאי 10, 2015
"
],
"text/plain": [
"Hmm, I don't know about this caterpillar rearing manual. I thought P.rapae had an obligate association w/ Brassica. pic.twitter.com/M10dqbOYlN
— Christie Bahlai (@cbahlai) מאי 10, 2015
')"
]
},
{
"cell_type": "code",
"execution_count": 3,
"metadata": {
"collapsed": false
},
"outputs": [
{
"data": {
"text/html": [
"This is a terrible dataset about caterpillar diet. How did it got published? pic.twitter.com/XkAq51HxEP
— Timothée Poisot (@tpoi) אפריל 23, 2015
"
],
"text/plain": [
"This is a terrible dataset about caterpillar diet. How did it got published? pic.twitter.com/XkAq51HxEP
— Timothée Poisot (@tpoi) אפריל 23, 2015
')"
]
},
{
"cell_type": "code",
"execution_count": 4,
"metadata": {
"collapsed": false
},
"outputs": [
{
"data": {
"text/html": [
"— Yoav Ram (@yoavram) May 10, 2015" ], "text/plain": [ "
— Yoav Ram (@yoavram) May 10, 2015')" ] }, { "cell_type": "code", "execution_count": 5, "metadata": { "collapsed": false }, "outputs": [ { "data": { "text/html": [ "
" ], "text/plain": [ "[blog] How hungry are caterpillars anyway? http://t.co/SvImkHYHhR #opendata
— Timothée Poisot (@tpoi) May 10, 2015
')" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "We will learn how to use the Global Biotic Interactions (globi) **web API** with Python to check **[How hungry are caterpillars anyway?](http://timotheepoisot.fr/2015/05/10/hungry-caterpillars/)** (sort of).\n", "\n", "## Web API\n", "First, what is a web API?\n", "\n", "Modern web APIs use are RESTful Application Programming Interface that return responses in JSON format (or sometimes XML) over HTTP (or securely over HTTPS - guess what the S stands for).\n", "\n", "- HTTP: a protocol for transfering text files on the internet\n", "- JSON: a file format, very similar to Python's `dict`.\n", "- REST: a common convention for designing web applications that allow querying and retrieving (and sometimes creating, changing and deleting) data.\n", "\n", "A web API is a language that allows computers to talk to computers (so my computer, running Python, to talk to a computer abroad used as the server for globi). \n", "\n", "The nouns of the languages are URLs. These are Universal Resource Locations (also sometimes URI, with Identifier) and are used to identify resources on the World Wide Web. For example, our course website URL is at[blog] How hungry are caterpillars anyway? http://t.co/SvImkHYHhR #opendata
— Timothée Poisot (@tpoi) May 10, 2015
\n", " | source_taxon_external_id | \n", "source_taxon_name | \n", "source_taxon_path | \n", "source_specimen_life_stage | \n", "source_specimen_basis_of_record | \n", "interaction_type | \n", "target_taxon_external_id | \n", "target_taxon_name | \n", "target_taxon_path | \n", "target_specimen_life_stage | \n", "target_specimen_basis_of_record | \n", "latitude | \n", "longitude | \n", "study_title | \n", "
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
0 | \n", "EOL:174006 | \n", "Pieris marginalis | \n", "Animalia | Bilateria | Protostomia | Ecdysozoa... | \n", "None | \n", "None | \n", "eats | \n", "EOL:29914 | \n", "Rubus | \n", "Plantae | Tracheophyta | Magnoliopsida | Rosal... | \n", "None | \n", "None | \n", "None | \n", "None | \n", "None | \n", "
1 | \n", "EOL:174006 | \n", "Pieris marginalis | \n", "Animalia | Bilateria | Protostomia | Ecdysozoa... | \n", "None | \n", "None | \n", "eats | \n", "EOL:37457 | \n", "Arabis | \n", "Plantae | Tracheophyta | Magnoliopsida | Brass... | \n", "None | \n", "None | \n", "None | \n", "None | \n", "None | \n", "
2 | \n", "EOL:174006 | \n", "Pieris marginalis | \n", "Animalia | Bilateria | Protostomia | Ecdysozoa... | \n", "None | \n", "None | \n", "eats | \n", "EOL:37718 | \n", "Rorippa | \n", "Plantae | Tracheophyta | Magnoliopsida | Brass... | \n", "None | \n", "None | \n", "None | \n", "None | \n", "None | \n", "
3 | \n", "EOL:174006 | \n", "Pieris marginalis | \n", "Animalia | Bilateria | Protostomia | Ecdysozoa... | \n", "None | \n", "None | \n", "eats | \n", "EOL:37667 | \n", "Cardamine | \n", "Plantae | Tracheophyta | Magnoliopsida | Brass... | \n", "None | \n", "None | \n", "None | \n", "None | \n", "None | \n", "
4 | \n", "EOL:176683 | \n", "Pieris rapae | \n", "Animalia | Arthropoda | Insecta | Lepidoptera ... | \n", "None | \n", "None | \n", "eats | \n", "EOL:467679 | \n", "Centaurea melitensis | \n", "Plantae | Tracheophyta | Magnoliopsida | Aster... | \n", "None | \n", "None | \n", "None | \n", "None | \n", "None | \n", "
\n", " | source_taxon_name | \n", "target_taxon_name | \n", "
---|---|---|
0 | \n", "Pieris marginalis | \n", "Rubus | \n", "
1 | \n", "Pieris marginalis | \n", "Arabis | \n", "
2 | \n", "Pieris marginalis | \n", "Rorippa | \n", "
3 | \n", "Pieris marginalis | \n", "Cardamine | \n", "
4 | \n", "Pieris rapae | \n", "Centaurea melitensis | \n", "
\n", " | source_taxon_name | \n", "target_taxon_name | \n", "
---|---|---|
0 | \n", "Pieris brassicae | \n", "55 | \n", "
1 | \n", "Pieris brassicoides | \n", "3 | \n", "
2 | \n", "Pieris canidia | \n", "10 | \n", "
3 | \n", "Pieris cheiranthi | \n", "1 | \n", "
4 | \n", "Pieris deota | \n", "1 | \n", "
\n", " | Pieris species | \n", "Number of known items in diet | \n", "
---|---|---|
12 | \n", "Pieris rapae | \n", "91 | \n", "
0 | \n", "Pieris brassicae | \n", "55 | \n", "
11 | \n", "Pieris napi | \n", "51 | \n", "
2 | \n", "Pieris canidia | \n", "10 | \n", "
13 | \n", "Pieris virginiensis | \n", "6 | \n", "
8 | \n", "Pieris marginalis | \n", "4 | \n", "
1 | \n", "Pieris brassicoides | \n", "3 | \n", "
6 | \n", "Pieris krueperi | \n", "3 | \n", "
5 | \n", "Pieris ergane | \n", "2 | \n", "
7 | \n", "Pieris mannii | \n", "2 | \n", "
10 | \n", "Pieris naganum | \n", "2 | \n", "
3 | \n", "Pieris cheiranthi | \n", "1 | \n", "
4 | \n", "Pieris deota | \n", "1 | \n", "
9 | \n", "Pieris melete | \n", "1 | \n", "