{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "# The OzGLAM workbench" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "
\n", "**This is an experimental project, so expect changes!**\n", "
" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "This is a collection of [Jupyter](http://jupyter.org/) notebooks to help you explore and use data from Australian GLAM institutions. It's aimed at researchers in the humanities, but will include tutorials of more general interest – such as an introduction to the Trove API." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "----\n", "\n", "## Contents\n", "\n", "This list will grow as I add more institutions, datasets, and examples – check back often!\n", "\n", "* [Using these notebooks](2-Using-these-notebooks.ipynb) – a brief introduction to the wonders of Jupyter notebooks\n", "* [Trove](Trove/) – the National Library of Australia's essential, all purpose, research resource\n", "* [RecordSearch](RecordSearch/) – getting data out of the National Archives of Australia's online database\n", "* CSV Explorer – find, use, and analyse a variety of CSV-formatted datasets from GLAM institutions\n", "\n", "----" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## About this project\n", "\n", "Over the past decade I've created and shared a wide variety of digital tools, examples, tutorials, and datasets. Some like [QueryPic](http://dhistory.org/querypic/) and [TroveHarvester](http://timsherratt.org/digital-heritage-handbook/docs/trove-newspaper-harvester/) are fairly polished and well documented. Others are just fragments of code. All of them are intended to support research into our rich cultural collections.\n", "\n", "But even though something like the TroveHarvester is pretty easy to use, it does require a bit of set-up, and I've been very aware that this can be a barrier to people starting their explorations. I created the [dhistory](http://dhistory.org/) site many years ago to provide the foundation for a digital workbench, but I couldn't quite achieve what I wanted – tools that were easy to use and required minimal setup, but also tools that exposed their own workings, that inspired novice users to question and to tinker.\n", "\n", "So here we are. My plan is to use Jupyter, GitHub, [Binder](https://mybinder.org/), and [Docker](https://www.docker.com/) to bring together all those tools, examples, tutorials, and datasets in a way that supports people's explorations through digital GLAM collections. I'm really excited, for example, that I can create a notebook that provides a [deconstructed (or perhaps see-through) version of QueryPic](Trove/Cookbook/DIY-QueryPic.ipynb) – that enables you to build, step by step, the same sort of visualisations, while learning about how it works. And at the end you can download the results as a CSV for further analysis. I love the way that Jupyter notebooks combine learning with real, live, digital tools and methods. You don't have to read a tutorial then go away and try to follow the instructions on your own. It's all together. Live code. Real research. Active learning.\n", "\n", "Like most of my projects this is in itself an experiment. I'm still learning what's possible and what works. But I'm hopeful." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "----\n", "## Using the workbench\n", "\n", "### Where's the data?\n", "\n", "Any data that you harvest using this workbench is available for easy download from the [data directory](data).\n", "\n", "### View the notebooks\n", "\n", "If you just want to have a look around, you can browse the notebooks in the project's [GitHub repository](https://github.com/wragge/ozglam-workbench). Even better, you can [explore them using the Jupyter Project's Notebook Viewer](https://nbviewer.jupyter.org/github/wragge/ozglam-workbench/blob/master/1-Introduction-and-table-of-contents.ipynb). But these will be static, read-only, versions -- you won't be able to run any of the code.\n", "\n", "### Use the notebooks online with MyBinder\n", "\n", "The easiest way to use the notebooks is on MyBinder. Just click on the button!\n", "\n", "[![Binder](https://mybinder.org/badge.svg)](https://mybinder.org/v2/gh/wragge/ozglam-workbench/master)\n", "\n", "MyBinder launches the notebooks in a custom computing environment with all the software you'll need pre-installed. You can explore, harvest, and analyse cultural heritage data without ever leaving your web browser.\n", "\n", "I've harvested thousands of Trove newspaper articles using MyBinder, but there are some limitations. The main one is that the environments it creates are only temporary. There's no way of saving your session and going back to it later. So if you harvest data, you'll need to download it before the session ends. But don't worry, I've provided a few handy tools to make downloading easy.\n", "\n", "MyBinder sessions will also shut down after about 10 minutes of inactivity. So don't go and have lunch without making sure you've finished what you're doing.\n", "\n", "### Run locally with Jupyter\n", "\n", "If you have [Jupyter installed](http://jupyter.org/install) on your system, you can clone this repository and then run fire up the notebooks:\n", "\n", "\n", "```\n", "git clone https://github.com/wragge/ozglam-workbench.git\n", "cd ozglam-workbench\n", "jupyter notebook\n", "```\n", "\n", "The workbench will open in your browser.\n", "\n", "### Run locally with Docker\n", "\n", "I've created a Docker image that includes these notebooks and all the necessary software. Assuming you have [Docker installed](https://docs.docker.com/install/), just open up a terminal and run:\n", "\n", "``` shell\n", "docker run -it --name=Workbench -v MyData:/home/jovyan/data -v MyWorkbench:/home/jovyan/workbench -p 8888:8888 wragge/ozglam-workbench\n", "```\n", "\n", "This command downloads the image and builds a container to run the workbench. The `-v MyData:/home/jovyan/data` bit creates a persistent volume called `MyData` where any data you harvest will be stored. You can update the image, create new containers, and still access your data. Similarly, `-v MyWorkbench:/home/jovyan/workbench` creates a persistent volume to store the notebooks themselves. This means that any changes you make to the notebooks will also be stored independently of the container itself. This should give you a bit of flexibility in how you use the workbench, and allow you to customise it to your needs.\n", "\n", "Once Jupyter starts up it'll display a url in the terminal that looks something like:\n", "\n", "```\n", "http://localhost:8888/?token=262718512d11cc3efcb1b2878f4jja9uca071924e328d554\n", "```\n", "\n", "Just copy and paste this into your browser to open the notebooks.\n", "\n", "To restart the `WorkBench` container using the same data volumes:\n", "\n", "``` shell\n", "docker start -ai Workbench\n", "```\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "----\n", "\n", "Created by [Tim Sherratt](https://timsherratt.org) ([@wragge](https://twitter.com/wragge))\n", "\n", "Support this project on [Patreon](https://www.patreon.com/timsherratt). \n" ] }, { "cell_type": "code", "execution_count": 3, "metadata": {}, "outputs": [ { "data": { "text/html": [ "\n", " \n", " " ], "text/plain": [ "" ] }, "execution_count": 3, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# Here's a button to fork this complete repo\n", "from IPython.display import IFrame\n", "IFrame('https://ghbtns.com/github-btn.html?user=wragge&repo=ozglam-workbench&type=fork&count=true&size=large', width=158, height=30, frameborder=0, scrolling=0)" ] } ], "metadata": { "kernelspec": { "display_name": "Python 3", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.6.5" } }, "nbformat": 4, "nbformat_minor": 2 }