{
"cells": [
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "slide"
}
},
"source": [
"# Literate / reproducible exploratory data analysis using Jupyter Notebooks\n",
"\n",
"Tony Hirst\n",
"\n",
"*Computing and Communications*\n",
"\n",
"https://blog.ouseful.info\n",
"https://github.com/ouseful-demos/"
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "skip"
}
},
"source": [
"Jupyter notebooks are a browser based, interactive environment widely used in computational research and commercial data science environments. Notebooks provide a blended research environment within which rich text (HTML), code and outputs generated from code, such as charts and tables, or even static or interactive maps, can be created and displayed.\n",
" \n",
"Using simple analysis scripts, typically written using Python or R, researchers can create narrated, readable, reproducible, shareable research workflows as well as all-in-one “text+source code” versions of research papers, where the code used to produce analyses reported in the paper is self-contained within the notebook itself.\n",
" \n",
"In this workshop you will have an opportunity to see how notebooks can be used and even try them out yourself. All you need is a desktop computer, laptop, tablet or even phone running a Chrome or Firefox browser."
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "slide"
}
},
"source": [
"# Play Along"
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "slide"
}
},
"source": [
"# Reproducibility...\n",
"\n",
"Can you:\n",
"\n",
"- find that data source again?\n",
"- clean the data the same way as last time?\n",
"- generate the same diagram?\n",
"- recreate the analysis?\n",
"- get the code to work the same way?"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Literate Computing\n",
"\n",
"Is there:\n",
" \n",
"- a clear linear narrative;\n",
"- comments, qualifications and context to help you make sense, a year down, the line of:\n",
" - your data?\n",
" - your code?\n",
" - your analyses?\n",
" - your results?"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Jupyter...\n",
"\n",
"- is an ecosystem of tools and protocols for helping you:\n",
" - access self-contained computational environments\n",
" - create rich, interactive documents that blend:\n",
" - text;\n",
" - code;\n",
" - code outputs."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# This is a presentation powered by a Jupyter Notebook...\n",
"\n",
"...one possible interface you can use to access Jupyter computing environments."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# "
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Using Code\n",
"\n",
"Code is entered and executed via code cells. The execution environment is determined by the notebook kernel attached to the notebook.\n",
"\n",
"This notebook has been associated with an Python kernel. Which means we can write — and execute — Python code in the cells:"
]
},
{
"cell_type": "code",
"execution_count": 1,
"metadata": {
"slideshow": {
"slide_type": "-"
}
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"hello world\n"
]
}
],
"source": [
"print(\"hello world\")"
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "slide"
}
},
"source": [
"# [`Jupyter_py_and_R.ipynb` Demo](./Jupyter_py_and_R.ipynb)"
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "slide"
}
},
"source": [
"# [`Food Standards Agency.ipynb` Demo](./Food%20Standards%20Agency.ipynb)"
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "slide"
}
},
"source": [
"## Text Analysis\n",
"\n",
"A wide range of packages support analysis of English, as well as classical, texts."
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "slide"
}
},
"source": [
"# Spacy — Visualise Sentence Morphology"
]
},
{
"cell_type": "code",
"execution_count": 4,
"metadata": {},
"outputs": [
{
"data": {
"text/html": [
""
],
"text/plain": [
""
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"import spacy\n",
"from spacy import displacy\n",
"\n",
"nlp = spacy.load(\"en_core_web_sm\")\n",
"doc = nlp(\"This is a sentence.\")\n",
"displacy.render(doc, style=\"dep\")"
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "slide"
}
},
"source": [
"# Spacy — Semantic Tagging"
]
},
{
"cell_type": "code",
"execution_count": 7,
"metadata": {
"scrolled": false
},
"outputs": [
{
"data": {
"text/html": [
"
The current Chancellor of \n",
"\n",
" the Open University\n",
" ORG\n",
"\n",
" is \n",
"\n",
" Martha Lane Fox\n",
" PERSON\n",
"\n",
".