{ "cells": [ { "cell_type": "markdown", "id": "0fbbd111-6c72-4452-8747-f8a8e68155d9", "metadata": {}, "source": [ "# Create a network graph visualisation of Australian government departments\n", "\n", "This notebook visualises changes in Australian government departments over time, using data from Wikidata. It creates a hierarchically-ordered network graph where each agency is represented as a node whose position and colour is determined by the decade in which the agency was created. The size of the node indicates how long the agency was in existence, while edges between nodes connect agencies to their successors. Earliest agencies will be at the top of the graph.\n", "\n", "You can [view the query](https://w.wiki/5tVh) used to generate this graph using the Wikidata Query Service." ] }, { "cell_type": "code", "execution_count": 11, "id": "6dfdf0a6-eb86-47a9-b595-0640faa16fba", "metadata": {}, "outputs": [], "source": [ "import json\n", "\n", "import arrow\n", "import pandas as pd\n", "from IPython.display import IFrame, display\n", "from pyvis.network import Network\n", "from SPARQLWrapper import JSON, SPARQLWrapper" ] }, { "cell_type": "code", "execution_count": 12, "id": "14fe492f-5528-4c63-b9c5-61ae0eefa985", "metadata": { "tags": [ "hide_cell" ] }, "outputs": [], "source": [ "sparql = SPARQLWrapper(\"https://query.wikidata.org/sparql\")\n", "\n", "sparql.setQuery(\n", " \"\"\"\n", "SELECT\n", " ?item ?label\n", " ?id ?start_date ?end_date ?after_id\n", "WHERE\n", "{\n", " ?item wdt:P31 wd:Q57605562;\n", " wdt:P10856 ?id;\n", " wdt:P571 ?start_date;\n", " rdfs:label ?agency_label.\n", " OPTIONAL { ?item wdt:P576 ?end_date. }\n", " OPTIONAL { ?item wdt:P1366 ?after.\n", " ?after wdt:P10856 ?after_id. }\n", " FILTER (lang(?agency_label) = \"en\").\n", " # Combine start and end year into a single string, setting end date to \"\" if it doesn't exist\n", " BIND(concat(xsd:string(YEAR(?start_date)), \"-\", COALESCE(xsd:string(YEAR(?end_date)), \"\")) as ?date_range)\n", " # Combine dept name and date range into a single string\n", " BIND(concat(?agency_label, \" (\", ?date_range, \")\") as ?label)\n", " SERVICE wikibase:label { bd:serviceParam wikibase:language \"[AUTO_LANGUAGE],en\". }\n", "}\n", "\"\"\"\n", ")" ] }, { "cell_type": "code", "execution_count": 13, "id": "600960d8-d8e1-400b-8bc8-6c68ac8c5d3e", "metadata": {}, "outputs": [], "source": [ "sparql.setReturnFormat(JSON)\n", "results = sparql.query().convert()\n", "df = pd.json_normalize(results[\"results\"][\"bindings\"], sep=\"_\")" ] }, { "cell_type": "code", "execution_count": 14, "id": "4bfd91b9-c1a7-45e9-a356-7f57ffbf2eb6", "metadata": { "tags": [ "hide_cell" ] }, "outputs": [ { "data": { "text/html": [ "
\n", " | item_type | \n", "item_value | \n", "after_id_type | \n", "after_id_value | \n", "id_type | \n", "id_value | \n", "start_date_datatype | \n", "start_date_type | \n", "start_date_value | \n", "end_date_datatype | \n", "end_date_type | \n", "end_date_value | \n", "label_type | \n", "label_value | \n", "
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
0 | \n", "uri | \n", "http://www.wikidata.org/entity/Q16956105 | \n", "literal | \n", "CA 15 | \n", "literal | \n", "CA 8 | \n", "http://www.w3.org/2001/XMLSchema#dateTime | \n", "literal | \n", "1901-01-01T00:00:00Z | \n", "http://www.w3.org/2001/XMLSchema#dateTime | \n", "literal | \n", "1916-11-14T00:00:00Z | \n", "literal | \n", "Department of Home Affairs (1901-1916) | \n", "
1 | \n", "uri | \n", "http://www.wikidata.org/entity/Q16956105 | \n", "literal | \n", "CA 14 | \n", "literal | \n", "CA 8 | \n", "http://www.w3.org/2001/XMLSchema#dateTime | \n", "literal | \n", "1901-01-01T00:00:00Z | \n", "http://www.w3.org/2001/XMLSchema#dateTime | \n", "literal | \n", "1916-11-14T00:00:00Z | \n", "literal | \n", "Department of Home Affairs (1901-1916) | \n", "
2 | \n", "uri | \n", "http://www.wikidata.org/entity/Q16956110 | \n", "literal | \n", "CA 27 | \n", "literal | \n", "CA 24 | \n", "http://www.w3.org/2001/XMLSchema#dateTime | \n", "literal | \n", "1928-12-10T00:00:00Z | \n", "http://www.w3.org/2001/XMLSchema#dateTime | \n", "literal | \n", "1932-04-12T00:00:00Z | \n", "literal | \n", "Department of Home Affairs (1928-1932) | \n", "
3 | \n", "uri | \n", "http://www.wikidata.org/entity/Q16956114 | \n", "literal | \n", "CA 3068 | \n", "literal | \n", "CA 2474 | \n", "http://www.w3.org/2001/XMLSchema#dateTime | \n", "literal | \n", "1977-12-20T00:00:00Z | \n", "http://www.w3.org/2001/XMLSchema#dateTime | \n", "literal | \n", "1980-11-03T00:00:00Z | \n", "literal | \n", "Department of Home Affairs (1977-1980) | \n", "
4 | \n", "uri | \n", "http://www.wikidata.org/entity/Q16956119 | \n", "literal | \n", "CA 4131 | \n", "literal | \n", "CA 3068 | \n", "http://www.w3.org/2001/XMLSchema#dateTime | \n", "literal | \n", "1980-11-03T00:00:00Z | \n", "http://www.w3.org/2001/XMLSchema#dateTime | \n", "literal | \n", "1984-12-13T00:00:00Z | \n", "literal | \n", "Department of Home Affairs and Environment (19... | \n", "