{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "# Jupyter R Notebook Demo\n", "\n", "The notebook like an Rmd file, allows us to blend text, code and code outputs." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Using Markdown\n", "\n", "Just as in Rmd, we can write nicely styled text from vanilla markdown:\n", "\n", "- yet\n", "- another\n", "- *styled* __list__" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "We can write `inline code` or language sensitively code styled blocks:\n", "\n", "```R\n", "# ggplot2 examples\n", "library(ggplot2)\n", "\n", "# create factors with value labels \n", "mtcars$gear <- factor(mtcars$gear,levels=c(3,4,5),\n", " labels=c(\"3gears\",\"4gears\",\"5gears\")) \n", "```\n", "\n", "\n", "We can write inline equations using latex — $e=mc^2$ as well as across multiple lines:\n", "\n", "\\begin{align}\n", "\\dot{x} & = \\sigma(y-x) \\\\\n", "\\dot{y} & = \\rho x - y - xz \\\\\n", "\\dot{z} & = -\\beta z + xy\n", "\\end{align}" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Using Code\n", "\n", "Code is entered and executed via code cells. The execution environment is determined by the notebook *kernel* attached to the notebook.\n", "\n", "This notebook has been associated with an R kernel. Which means we can write R code in the cells:" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "# ggplot2 examples\n", "library(ggplot2)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "As in Rmd documents, the state set or packages loaded by executing code in one cell is available to later executed cells.\n", "\n", "So we can access one of the `ggplot2` loaded datasets:" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "# create factors with value labels \n", "mtcars$gear <- factor(mtcars$gear,levels=c(3,4,5),\n", " labels=c(\"3gears\",\"4gears\",\"5gears\")) \n", "mtcars$am <- factor(mtcars$am,levels=c(0,1),\n", " labels=c(\"Automatic\",\"Manual\")) \n", "mtcars$cyl <- factor(mtcars$cyl,levels=c(4,6,8),\n", " labels=c(\"4cyl\",\"6cyl\",\"8cyl\")) " ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "As with Rmd, if the last line of a code cell returns an object, we can display it. This includes things like charts:" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "# Kernel density plots for mpg\n", "# grouped by number of gears (indicated by color)\n", "g = qplot(mpg, data=mtcars, geom=\"density\", fill=gear, alpha=I(.5), \n", " main=\"Distribution of Gas Mileage\", xlab=\"Miles Per Gallon\", \n", " ylab=\"Density\")\n", "g" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "As in any other R environment, we can change the aesthetics through the application of a particular theme.\n", "\n", "For example, the `theme_commonslib()` theme is a theme used by the House of Commons Library; we can install it from it's Github repository:" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "install.packages(\"remotes\")\n", "remotes::install_github(\"olihawkins/clplot\")" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "And then simply add the theme to the chart:" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "library(ggplot2)\n", "library(clplot)\n", "\n", "g + theme_commonslib()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "It's some time since I looked properly at how we could embed interactive elements in a notebook.\n", "\n", "For example, it's easy enough to generate a widget:" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "#install.packages(\"leaflet\")\n", "library(leaflet)\n", "\n", "m <- leaflet() %>%\n", " addTiles() %>% # Add default OpenStreetMap map tiles\n", " addMarkers(lng=174.768, lat=-36.852, popup=\"The birthplace of R\")" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "But I'm not sure if we can directly embed the result yet?\n", "\n", "This is one of my old workarounds: save the widget HTML to a file and then load it back in via an IFrame..." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "library(htmlwidgets)\n", "library(IRdisplay)\n", "\n", "saveWidget(m, 'demo.html', selfcontained = TRUE)\n", "display_html('')" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "We can generate nicely styled tables with a bit of help..." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "install.packages(\"kableExtra\")" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Generate an HTML table and display it as such, via a slight hack..." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "library(knitr)\n", "library(kableExtra)\n", "\n", "kable(summary(cars)) %>%\n", " kable_styling(bootstrap_options = c(\"striped\", \"hover\")) %>%\n", " as.character() %>% display_html()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Loading Data From APIs\n", "\n", "This example is cribbed in part from https://medium.com/@traffordDataLab/querying-apis-in-r-39029b73d5f1 and uses the UK Police API, which returns JSON data of the form: https://data.police.uk/api/crimes-street/burglary?lat=52.0406&lng=-0.7594&date=2018-05" ] }, { "cell_type": "code", "execution_count": 29, "metadata": {}, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "── \u001b[1mAttaching packages\u001b[22m ─────────────────────────────────────── tidyverse 1.2.1 ──\n", "\n", "\u001b[32m✔\u001b[39m \u001b[34mtibble \u001b[39m 2.1.1 \u001b[32m✔\u001b[39m \u001b[34mpurrr \u001b[39m 0.3.2 \n", "\u001b[32m✔\u001b[39m \u001b[34mtidyr \u001b[39m 0.8.3 \u001b[32m✔\u001b[39m \u001b[34mdplyr \u001b[39m 0.8.0.\u001b[31m1\u001b[39m\n", "\u001b[32m✔\u001b[39m \u001b[34mreadr \u001b[39m 1.3.1 \u001b[32m✔\u001b[39m \u001b[34mstringr\u001b[39m 1.4.0 \n", "\u001b[32m✔\u001b[39m \u001b[34mtibble \u001b[39m 2.1.1 \u001b[32m✔\u001b[39m \u001b[34mforcats\u001b[39m 0.4.0 \n", "\n", "── \u001b[1mConflicts\u001b[22m ────────────────────────────────────────── tidyverse_conflicts() ──\n", "\u001b[31m✖\u001b[39m \u001b[34mdplyr\u001b[39m::\u001b[32mfilter()\u001b[39m masks \u001b[34mstats\u001b[39m::filter()\n", "\u001b[31m✖\u001b[39m \u001b[34mdplyr\u001b[39m::\u001b[32mgroup_rows()\u001b[39m masks \u001b[34mkableExtra\u001b[39m::group_rows()\n", "\u001b[31m✖\u001b[39m \u001b[34mdplyr\u001b[39m::\u001b[32mlag()\u001b[39m masks \u001b[34mstats\u001b[39m::lag()\n", "\n", "\n", "Attaching package: ‘jsonlite’\n", "\n", "\n", "The following object is masked from ‘package:purrr’:\n", "\n", " flatten\n", "\n", "\n" ] } ], "source": [ "library(tidyverse)\n", "library(httr)\n", "library(jsonlite)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "We can make a query onto the API and get a JSON file back:" ] }, { "cell_type": "code", "execution_count": 66, "metadata": {}, "outputs": [], "source": [ "path <- \"https://data.police.uk/api/crimes-street/burglary?\"\n", "\n", "request <- GET(url = path, \n", " query = list(\n", " lat = 52.0406,\n", " lng = -0.7594,\n", " date = \"2018-05\")\n", " )" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Decoding the JSON returns everything as chars...:" ] }, { "cell_type": "code", "execution_count": 67, "metadata": {}, "outputs": [ { "data": { "text/html": [ "\n", "\n", "\n", "\t\n", "\t\n", "\n", "\n", "\t\n", "\t\n", "\t\n", "\t\n", "\t\n", "\t\n", "\n", "
A data.frame: 6 × 13
categorylocation_typecontextpersistent_ididlocation_subtypemonthlocation.latitudelocation.longitudelocation.street.idlocation.street.nameoutcome_status.categoryoutcome_status.date
<chr><chr><chr><chr><int><chr><chr><chr><chr><int><chr><chr><chr>
1burglaryForcefa6b515b3d9658626797bb6db06caefcab8d05e4ac387914c12d3077368b402d653733602018-0552.052319-0.7565751213725On or near Bryony Place Investigation complete; no suspect identified2018-08
2burglaryForce7480cb4911243cc2f48669e71fa2fb7320f2197b6ccb6390be06281c898b5074653797072018-0552.044662-0.7672131213735On or near Wandsworth Place Investigation complete; no suspect identified2018-08
3burglaryForce6ffc8736dc157b33cf0d9f8d3a858ad1d4d41fc92331e34145be0ef465cf8eab653827102018-0552.033215-0.7466231213362On or near Trevone Court Unable to prosecute suspect 2018-08
4burglaryForce133441a73f4690a1d4e1873b7696fdd0240b1c330cc84337bf6243435d41159d653702812018-0552.027977-0.7606031212564On or near Rashleigh Place Investigation complete; no suspect identified2018-08
5burglaryForce1e38aec62bd62383218f27bbbe41567c28d39900a5bb965e00b0a4e5e6db4844653717362018-0552.044649-0.7583041213743On or near North Tenth StreetUnable to prosecute suspect 2018-08
6burglaryForce04c25368c11760f02c45b701d21e1583fba2e31bb53ad4e1aa4c9dd55ba5c412653777702018-0552.030466-0.7434791213353On or near Ashby Investigation complete; no suspect identified2018-05
\n" ], "text/latex": [ "A data.frame: 6 × 13\n", "\\begin{tabular}{r|lllllllllllll}\n", " & category & location\\_type & context & persistent\\_id & id & location\\_subtype & month & location.latitude & location.longitude & location.street.id & location.street.name & outcome\\_status.category & outcome\\_status.date\\\\\n", " & & & & & & & & & & & & & \\\\\n", "\\hline\n", "\t1 & burglary & Force & & fa6b515b3d9658626797bb6db06caefcab8d05e4ac387914c12d3077368b402d & 65373360 & & 2018-05 & 52.052319 & -0.756575 & 1213725 & On or near Bryony Place & Investigation complete; no suspect identified & 2018-08\\\\\n", "\t2 & burglary & Force & & 7480cb4911243cc2f48669e71fa2fb7320f2197b6ccb6390be06281c898b5074 & 65379707 & & 2018-05 & 52.044662 & -0.767213 & 1213735 & On or near Wandsworth Place & Investigation complete; no suspect identified & 2018-08\\\\\n", "\t3 & burglary & Force & & 6ffc8736dc157b33cf0d9f8d3a858ad1d4d41fc92331e34145be0ef465cf8eab & 65382710 & & 2018-05 & 52.033215 & -0.746623 & 1213362 & On or near Trevone Court & Unable to prosecute suspect & 2018-08\\\\\n", "\t4 & burglary & Force & & 133441a73f4690a1d4e1873b7696fdd0240b1c330cc84337bf6243435d41159d & 65370281 & & 2018-05 & 52.027977 & -0.760603 & 1212564 & On or near Rashleigh Place & Investigation complete; no suspect identified & 2018-08\\\\\n", "\t5 & burglary & Force & & 1e38aec62bd62383218f27bbbe41567c28d39900a5bb965e00b0a4e5e6db4844 & 65371736 & & 2018-05 & 52.044649 & -0.758304 & 1213743 & On or near North Tenth Street & Unable to prosecute suspect & 2018-08\\\\\n", "\t6 & burglary & Force & & 04c25368c11760f02c45b701d21e1583fba2e31bb53ad4e1aa4c9dd55ba5c412 & 65377770 & & 2018-05 & 52.030466 & -0.743479 & 1213353 & On or near Ashby & Investigation complete; no suspect identified & 2018-05\\\\\n", "\\end{tabular}\n" ], "text/markdown": [ "\n", "A data.frame: 6 × 13\n", "\n", "| | category <chr> | location_type <chr> | context <chr> | persistent_id <chr> | id <int> | location_subtype <chr> | month <chr> | location.latitude <chr> | location.longitude <chr> | location.street.id <int> | location.street.name <chr> | outcome_status.category <chr> | outcome_status.date <chr> |\n", "|---|---|---|---|---|---|---|---|---|---|---|---|---|---|\n", "| 1 | burglary | Force | | fa6b515b3d9658626797bb6db06caefcab8d05e4ac387914c12d3077368b402d | 65373360 | | 2018-05 | 52.052319 | -0.756575 | 1213725 | On or near Bryony Place | Investigation complete; no suspect identified | 2018-08 |\n", "| 2 | burglary | Force | | 7480cb4911243cc2f48669e71fa2fb7320f2197b6ccb6390be06281c898b5074 | 65379707 | | 2018-05 | 52.044662 | -0.767213 | 1213735 | On or near Wandsworth Place | Investigation complete; no suspect identified | 2018-08 |\n", "| 3 | burglary | Force | | 6ffc8736dc157b33cf0d9f8d3a858ad1d4d41fc92331e34145be0ef465cf8eab | 65382710 | | 2018-05 | 52.033215 | -0.746623 | 1213362 | On or near Trevone Court | Unable to prosecute suspect | 2018-08 |\n", "| 4 | burglary | Force | | 133441a73f4690a1d4e1873b7696fdd0240b1c330cc84337bf6243435d41159d | 65370281 | | 2018-05 | 52.027977 | -0.760603 | 1212564 | On or near Rashleigh Place | Investigation complete; no suspect identified | 2018-08 |\n", "| 5 | burglary | Force | | 1e38aec62bd62383218f27bbbe41567c28d39900a5bb965e00b0a4e5e6db4844 | 65371736 | | 2018-05 | 52.044649 | -0.758304 | 1213743 | On or near North Tenth Street | Unable to prosecute suspect | 2018-08 |\n", "| 6 | burglary | Force | | 04c25368c11760f02c45b701d21e1583fba2e31bb53ad4e1aa4c9dd55ba5c412 | 65377770 | | 2018-05 | 52.030466 | -0.743479 | 1213353 | On or near Ashby | Investigation complete; no suspect identified | 2018-05 |\n", "\n" ], "text/plain": [ " category location_type context\n", "1 burglary Force \n", "2 burglary Force \n", "3 burglary Force \n", "4 burglary Force \n", "5 burglary Force \n", "6 burglary Force \n", " persistent_id id \n", "1 fa6b515b3d9658626797bb6db06caefcab8d05e4ac387914c12d3077368b402d 65373360\n", "2 7480cb4911243cc2f48669e71fa2fb7320f2197b6ccb6390be06281c898b5074 65379707\n", "3 6ffc8736dc157b33cf0d9f8d3a858ad1d4d41fc92331e34145be0ef465cf8eab 65382710\n", "4 133441a73f4690a1d4e1873b7696fdd0240b1c330cc84337bf6243435d41159d 65370281\n", "5 1e38aec62bd62383218f27bbbe41567c28d39900a5bb965e00b0a4e5e6db4844 65371736\n", "6 04c25368c11760f02c45b701d21e1583fba2e31bb53ad4e1aa4c9dd55ba5c412 65377770\n", " location_subtype month location.latitude location.longitude\n", "1 2018-05 52.052319 -0.756575 \n", "2 2018-05 52.044662 -0.767213 \n", "3 2018-05 52.033215 -0.746623 \n", "4 2018-05 52.027977 -0.760603 \n", "5 2018-05 52.044649 -0.758304 \n", "6 2018-05 52.030466 -0.743479 \n", " location.street.id location.street.name \n", "1 1213725 On or near Bryony Place \n", "2 1213735 On or near Wandsworth Place \n", "3 1213362 On or near Trevone Court \n", "4 1212564 On or near Rashleigh Place \n", "5 1213743 On or near North Tenth Street\n", "6 1213353 On or near Ashby \n", " outcome_status.category outcome_status.date\n", "1 Investigation complete; no suspect identified 2018-08 \n", "2 Investigation complete; no suspect identified 2018-08 \n", "3 Unable to prosecute suspect 2018-08 \n", "4 Investigation complete; no suspect identified 2018-08 \n", "5 Unable to prosecute suspect 2018-08 \n", "6 Investigation complete; no suspect identified 2018-05 " ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "response <- content(request, as = \"text\", encoding = \"UTF-8\")\n", "df <- fromJSON(response, flatten = TRUE) %>% \n", " data.frame()\n", "\n", "head(df)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "So I'm going to be lazy in how I cast the lat/long to a numeric and use a package to help:" ] }, { "cell_type": "code", "execution_count": 68, "metadata": {}, "outputs": [], "source": [ "#install.packages(\"hablar\")\n", "library(hablar)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The cast is then just a `convert()` pipeline step:" ] }, { "cell_type": "code", "execution_count": 69, "metadata": {}, "outputs": [ { "data": { "text/html": [ "\n", "\n", "\n", "\t\n", "\t\n", "\n", "\n", "\t\n", "\t\n", "\t\n", "\t\n", "\t\n", "\t\n", "\n", "
A tibble: 6 × 5
monthcategorylocationlonglat
<chr><chr><chr><dbl><dbl>
2018-05burglaryOn or near Bryony Place -0.75657552.05232
2018-05burglaryOn or near Wandsworth Place -0.76721352.04466
2018-05burglaryOn or near Trevone Court -0.74662352.03321
2018-05burglaryOn or near Rashleigh Place -0.76060352.02798
2018-05burglaryOn or near North Tenth Street-0.75830452.04465
2018-05burglaryOn or near Ashby -0.74347952.03047
\n" ], "text/latex": [ "A tibble: 6 × 5\n", "\\begin{tabular}{lllll}\n", " month & category & location & long & lat\\\\\n", " & & & & \\\\\n", "\\hline\n", "\t 2018-05 & burglary & On or near Bryony Place & -0.756575 & 52.05232\\\\\n", "\t 2018-05 & burglary & On or near Wandsworth Place & -0.767213 & 52.04466\\\\\n", "\t 2018-05 & burglary & On or near Trevone Court & -0.746623 & 52.03321\\\\\n", "\t 2018-05 & burglary & On or near Rashleigh Place & -0.760603 & 52.02798\\\\\n", "\t 2018-05 & burglary & On or near North Tenth Street & -0.758304 & 52.04465\\\\\n", "\t 2018-05 & burglary & On or near Ashby & -0.743479 & 52.03047\\\\\n", "\\end{tabular}\n" ], "text/markdown": [ "\n", "A tibble: 6 × 5\n", "\n", "| month <chr> | category <chr> | location <chr> | long <dbl> | lat <dbl> |\n", "|---|---|---|---|---|\n", "| 2018-05 | burglary | On or near Bryony Place | -0.756575 | 52.05232 |\n", "| 2018-05 | burglary | On or near Wandsworth Place | -0.767213 | 52.04466 |\n", "| 2018-05 | burglary | On or near Trevone Court | -0.746623 | 52.03321 |\n", "| 2018-05 | burglary | On or near Rashleigh Place | -0.760603 | 52.02798 |\n", "| 2018-05 | burglary | On or near North Tenth Street | -0.758304 | 52.04465 |\n", "| 2018-05 | burglary | On or near Ashby | -0.743479 | 52.03047 |\n", "\n" ], "text/plain": [ " month category location long lat \n", "1 2018-05 burglary On or near Bryony Place -0.756575 52.05232\n", "2 2018-05 burglary On or near Wandsworth Place -0.767213 52.04466\n", "3 2018-05 burglary On or near Trevone Court -0.746623 52.03321\n", "4 2018-05 burglary On or near Rashleigh Place -0.760603 52.02798\n", "5 2018-05 burglary On or near North Tenth Street -0.758304 52.04465\n", "6 2018-05 burglary On or near Ashby -0.743479 52.03047" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "df <- select(df,\n", " month, category, \n", " location = location.street.name,\n", " long = location.longitude,\n", " lat = location.latitude) %>%\n", " convert(num(long, lat))\n", "\n", "head(df)" ] }, { "cell_type": "code", "execution_count": 70, "metadata": {}, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "Assuming \"long\" and \"lat\" are longitude and latitude, respectively\n", "\n" ] }, { "data": { "text/html": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "m = leaflet(df) %>% addTiles() %>% addMarkers(\n", " clusterOptions = markerClusterOptions(), popup = ~as.character(location)\n", ")\n", "\n", "saveWidget(m, 'demo.html', selfcontained = TRUE)\n", "display_html('')" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [] } ], "metadata": { "kernelspec": { "display_name": "R", "language": "R", "name": "ir" }, "language_info": { "codemirror_mode": "r", "file_extension": ".r", "mimetype": "text/x-r-source", "name": "R", "pygments_lexer": "r", "version": "3.6.3" } }, "nbformat": 4, "nbformat_minor": 2 }