{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# OSMnx overview: querying, simplifying, visualizing, saving\n",
    "\n",
    "Author: [Geoff Boeing](https://geoffboeing.com/)\n",
    "\n",
    "  - [Overview of OSMnx](http://geoffboeing.com/2016/11/osmnx-python-street-networks/)\n",
    "  - [GitHub repo](https://github.com/gboeing/osmnx)\n",
    "  - [Examples, demos, tutorials](https://github.com/gboeing/osmnx-examples)\n",
    "  - [Documentation](https://osmnx.readthedocs.io/en/stable/)\n",
    "  - [Journal article/citation](http://geoffboeing.com/publications/osmnx-complex-street-networks/)\n",
    "  \n",
    "Once you've perused the [features demo notebook](00-osmnx-features-demo.ipynb), this notebook demonstrates more details on querying for place boundaries and street networks, visualizing, and saving models to disk."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import geopandas as gpd\n",
    "import osmnx as ox\n",
    "\n",
    "%matplotlib inline\n",
    "ox.__version__"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You can configure OSMnx using the `settings` module. See the [documentation](https://osmnx.readthedocs.io/en/stable/osmnx.html#module-osmnx.settings) for the settings you can configure. For example, by default OSMnx caches all server responses to prevent repeatedly hitting the server for the same query every time you run it. This both makes our code faster on subsequent runs and helps us be a \"good neighbor\" to the server. But you can turn caching off (or back on again) with the `use_cache` setting."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# turn response caching off\n",
    "ox.settings.use_cache = False\n",
    "\n",
    "# turn it back on and turn on/off logging to your console\n",
    "ox.settings.use_cache = True\n",
    "ox.settings.log_console = False"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Part 1: get place boundaries from OpenStreetMap\n",
    "\n",
    "OSMnx lets you download place boundary geometries from OpenStreetMap, project them, and plot them. For a more in-depth demonstration of querying by place, see [this notebook](03-graph-place-queries.ipynb)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# get the boundary polygon for manhattan, project it, and plot it\n",
    "city = ox.geocode_to_gdf(\"Manhattan, New York, USA\")\n",
    "city_proj = ox.project_gdf(city)\n",
    "ax = city_proj.plot(fc=\"gray\", ec=\"none\")\n",
    "_ = ax.axis(\"off\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# get boundary polygons for several cities, save as GeoPackage, project to UTM, and plot\n",
    "place_names = [\n",
    "    \"Berkeley, California, USA\",\n",
    "    \"Oakland, California, USA\",\n",
    "    \"Piedmont, California, USA\",\n",
    "    \"Emeryville, California, USA\",\n",
    "    \"Alameda, Alameda County, CA, USA\",\n",
    "]\n",
    "east_bay = ox.geocode_to_gdf(place_names)\n",
    "east_bay.to_file(\"./data/east_bay.gpkg\", driver=\"GPKG\")\n",
    "east_bay = ox.project_gdf(east_bay)\n",
    "ax = east_bay.plot(fc=\"gray\", ec=\"none\")\n",
    "_ = ax.axis(\"off\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# if you know the OSM ID of the place(s) you want, you can query it directly\n",
    "ox.geocode_to_gdf([\"R357794\", \"N8170768521\", \"W427818536\"], by_osmid=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Part 2: download and model street networks\n",
    "\n",
    "OSMnx lets you download street network data and build topologically-corrected street networks, project and plot the networks, and save the street network as SVGs, GraphML files, GeoPackages, or shapefiles for later use. The street networks are directed and preserve one-way directionality. For a more in-depth demonstration of creating street networks, see [this notebook](03-graph-place-queries.ipynb).\n",
    "\n",
    "You can download a street network by providing OSMnx any of the following (demonstrated in the examples below):\n",
    "  - a bounding box\n",
    "  - a lat-long point plus a distance\n",
    "  - an address plus a distance\n",
    "  - a place name or list of place names (to automatically geocode and get the boundary of)\n",
    "  - a polygon of the desired street network's boundaries\n",
    "  - a .osm formatted xml file\n",
    "\n",
    "You can also specify several different network types:\n",
    "  - 'drive' - get drivable public streets (but not service roads)\n",
    "  - 'drive_service' - get drivable streets, including service roads\n",
    "  - 'walk' - get all streets and paths that pedestrians can use (this network type ignores one-way directionality)\n",
    "  - 'bike' - get all streets and paths that cyclists can use\n",
    "  - 'all' - download all non-private OSM streets and paths (this is the default network type unless you specify a different one)\n",
    "  - 'all_private' - download all OSM streets and paths, including private-access ones"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    " #### Method #1, pass a bounding box ####\n",
    " This constructs the network from all the OSM nodes and ways within the bounding box."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# define a bounding box in San Francisco\n",
    "north, south, east, west = 37.79, 37.78, -122.41, -122.43\n",
    "\n",
    "# create network from that bounding box\n",
    "G = ox.graph_from_bbox(north, south, east, west, network_type=\"drive_service\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Method #2, pass a lat-lng point and bounding box distance in meters\n",
    "This creates a bounding box *n* meters North, South, East, and West of the point, then constructs the network from all the OSM nodes and ways within the bounding box."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# define a point at the corner of California St and Mason St in SF\n",
    "location_point = (37.791427, -122.410018)\n",
    "\n",
    "# create network from point, inside bounding box of N, S, E, W each 750m from point\n",
    "G = ox.graph_from_point(location_point, dist=750, dist_type=\"bbox\", network_type=\"drive\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Method #3, pass a lat-lng point and *network* distance in meters ####\n",
    "\n",
    "This creates a bounding box *n* meters North, South, East, and West of the point, then constructs the network from all the OSM nodes and ways within the bounding box. Then it truncates the network by removing all nodes further than *n* meters from the point along the network."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# same point again, but create network only of nodes within 500m along the network from point\n",
    "G = ox.graph_from_point(location_point, dist=500, dist_type=\"network\")\n",
    "fig, ax = ox.plot_graph(G, node_color=\"r\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "*Note* the plot above shows the network within 500m (traveling distance along the network) from the `location_point`. By default, the `network_type` parameter value is 'all', meaning that we do not filter out paths that restrict certain types of traffic. This also means that one-way streets are honored as one-way and you cannot travel the wrong direction down them. Thus, the 500m takes into account only those nodes you can reach within 500m while only traveling in the allowed direction of the street. Instead (below), we can specify `network_type='walk'` to build a street network only of paths that walking is allowed on. This also makes every path bi-directional in the directed network, because you can walk in either direction on the sidewalk of a one-way street. Thus, the 500m now takes into account those nodes you can reach within 500m while traveling in either direction (even if it's a one-way street)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# create network only of nodes within 500m walking along the network from point\n",
    "G = ox.graph_from_point(location_point, dist=500, dist_type=\"network\", network_type=\"walk\")\n",
    "fig, ax = ox.plot_graph(G, node_color=\"r\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Method #4, pass an address and distance (*bounding box* or *network*) in meters ####\n",
    "This geocodes the address, creates a bounding box, downloads the network, then truncates it by network distance (if distance_type='network')."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# network from address, including only nodes within 1km along the network from the address\n",
    "G = ox.graph_from_address(\n",
    "    address=\"350 5th Ave, New York, NY\",\n",
    "    dist=1000,\n",
    "    dist_type=\"network\",\n",
    "    network_type=\"drive\",\n",
    ")\n",
    "\n",
    "# you can project the network to UTM (zone calculated automatically)\n",
    "G_projected = ox.project_graph(G)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Method #5, pass a place name ####\n",
    "\n",
    "This geocodes the place name, gets the place's boundary shape polygon and bounding box, downloads the network within the bounding box, then truncates it to the place's boundary polygon."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# create the street network within the city of Piedmont's borders\n",
    "G = ox.graph_from_place(\"Piedmont, California, USA\", network_type=\"drive\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# you can also pass multiple places as a mixed list of strings and/or dicts\n",
    "places = [\n",
    "    \"Los Altos, California, USA\",\n",
    "    {\"city\": \"Los Altos Hills\", \"state\": \"California\"},\n",
    "    \"Loyola, California\",\n",
    "]\n",
    "G = ox.graph_from_place(places, truncate_by_edge=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# save to disk as GeoPackage file then plot\n",
    "ox.save_graph_geopackage(G)\n",
    "fig, ax = ox.plot_graph(G, node_size=0, edge_color=\"w\", edge_linewidth=0.2)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Method #6, pass a polygon ####\n",
    "\n",
    "This example loads the [Mission District](http://www.zillow.com/howto/api/neighborhood-boundaries.htm)'s polygon from a shapefile, then downloads the network within its bounding box, then prunes all nodes that lie outside the place's boundary polygon."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "calif = gpd.read_file(\"input_data/ZillowNeighborhoods-CA\")\n",
    "mission_district = calif[(calif[\"CITY\"] == \"San Francisco\") & (calif[\"NAME\"] == \"Mission\")]\n",
    "polygon = mission_district[\"geometry\"].iloc[0]\n",
    "\n",
    "G2 = ox.graph_from_polygon(polygon, network_type=\"drive_service\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Method #7, load a .osm xml file"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# create graph from .osm extract file\n",
    "G = ox.graph_from_xml(\"./input_data/West-Oakland.osm.bz2\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Part 3: simplifying street network topology\n",
    "\n",
    "Simplification is normally done by OSMnx automatically under the hood, but we can break it out to see how it works. OpenStreetMap nodes are weird. They include intersections, but they also include all the points along a single block where the street curves. The latter are not nodes in the graph theory sense, so we remove them algorithmically and consolidate the set of edges between \"true\" network nodes into a single edge. There are two simplification modes, strict and non-strict. The main difference is that unlike strict mode, non-strict mode allows simplification to an \"expansion graph\" (ie, if the graph were undirected, nodes with degree 2 as long as the incident edges have different OSM IDs). For a more in-depth demonstration of topological simplification with OSMnx, see [this notebook](04-simplify-graph-consolidate-nodes.ipynb)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# create a network around some (lat, lng) point but do not simplify it yet\n",
    "location_point = (33.299896, -111.831638)\n",
    "G = ox.graph_from_point(location_point, network_type=\"drive_service\", dist=500, simplify=False)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# turn off strict mode and see what nodes we'd remove, in yellow\n",
    "nc = [\"r\" if ox.simplification._is_endpoint(G, node) else \"y\" for node in G.nodes()]\n",
    "fig, ax = ox.plot_graph(G, node_color=nc)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The dots above are OSM nodes. We'll remove the nodes in yellow as they're not real network nodes (intersections/dead-ends)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# simplify the network\n",
    "G = ox.simplify_graph(G)\n",
    "fig, ax = ox.plot_graph(G, node_color=\"r\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# show the simplified network with edges colored by length\n",
    "ec = ox.plot.get_edge_colors_by_attr(G, attr=\"length\", cmap=\"plasma_r\")\n",
    "fig, ax = ox.plot_graph(\n",
    "    G, node_color=\"w\", node_edgecolor=\"k\", node_size=50, edge_color=ec, edge_linewidth=3\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# highlight all parallel (multiple) edges\n",
    "ec = [\"gray\" if k == 0 or u == v else \"r\" for u, v, k in G.edges(keys=True)]\n",
    "fig, ax = ox.plot_graph(\n",
    "    G, node_color=\"w\", node_edgecolor=\"k\", node_size=50, edge_color=ec, edge_linewidth=3\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# highlight all one-way edges in the mission district network from earlier\n",
    "ec = [\"r\" if data[\"oneway\"] else \"w\" for u, v, key, data in G2.edges(keys=True, data=True)]\n",
    "fig, ax = ox.plot_graph(G2, node_size=0, edge_color=ec, edge_linewidth=1.5, edge_alpha=0.7)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Part 4: saving networks to disk\n",
    "\n",
    "For more examples of saving and loading networks to/from disk, see [this notebook](05-save-load-networks.ipynb)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# save street network as GeoPackage to work with in GIS\n",
    "ox.save_graph_geopackage(G, filepath=\"./data/network.gpkg\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# save street network as GraphML file to work with later in OSMnx or networkx or gephi\n",
    "ox.save_graphml(G, filepath=\"./data/network.graphml\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Part 5: calculate basic network indicators"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# calculate basic street network metrics and display average circuity\n",
    "stats = ox.basic_stats(G)\n",
    "stats[\"circuity_avg\"]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In this street network, the streets are ~16% more circuitous than the straight-lines paths would be.\n",
    "\n",
    "For examples of analyzing street networks, see [this example](06-stats-indicators-centrality.ipynb)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "anaconda-cloud": {},
  "kernelspec": {
   "display_name": "Python (ox)",
   "language": "python",
   "name": "ox"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.8"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}