{ "cells": [ { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "#### telluric\n", "\n", "## Interactive vector and raster geospatial data manipulation in Python\n", "\n", "#### Guy Doulberg - 2018-06-04 @ PyCon-IL\n", "\n", "https://github.com/satellogic/telluric-talks\n", "\n", "![telluric](img/bg.png)" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "notes" } }, "source": [ "(Introduction)\n", "\n", "1. Quick introduction to geospatial data\n", " - Vector data\n", " - Raster data\n", " - Formats\n", "2. Existing software\n", " - GDAL, GEOS\n", " - rasterio, Fiona, Shapely\n", " - Still, too low level\n", "3. telluric library\n", " - Easy manipulation of geometries\n", " - Features and collections\n", " - Input and output\n", " - Raster data\n", "4. Distributed computing\n", "5. Future work\n", "\n", "(Questions?)" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "# [**Satellogic**](https://www.satellogic.com/)\n", "\n", "* We are building a constellation of satellites that orbits earth.\n", "* We are building an infrastructure to collect and store data (mostly images) from this constellation.\n", "* We are producing insights out of this data and other sources of data.\n", "* We are hiring\n", "\n", "![satelloguc](img/concetalation.png)\n", "\n" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "subslide" } }, "source": [ "# Guy\n", "\n", "* I am part of the team which is responsilbe for collecting storing and accesing the Raster and Vector Data in *Satellogic*.\n", "* Software engineer with experience in big data projects." ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "# 1. Quick intro to geospatial data\n", "\n", "> **Geospatial data**, geographic data, georeferenced data, or just geodata \"are defined in the ISO/TC 211 series of standards as data and information having an implicit or explicit association with a location relative to the Earth\"\n", ">\n", "> -- https://en.wikipedia.org/wiki/Geographic_data_and_information\n", "\n", "> A **geographic information system (GIS)** is a system designed to capture, store, manipulate, analyze, manage, and present spatial or geographic data.\n", ">\n", "> -- https://en.wikipedia.org/wiki/Geographic_information_system" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "subslide" } }, "source": [ "## Two forms: vector and raster\n", "\n", "* **Vector data**: \n", " * Defining shapes\n", "* **Raster data**: \n", " * Assigning pixels values\n", "\n", "![Vector vs Raster](img/vector-raster2.png)\n", "\n", "
Image by Víctor Olaya \"Sistemas de Información Geográfica\", CC-BY
" ] }, { "attachments": {}, "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "skip" } }, "source": [ "Another example:\n", "![Vector vs Raster2](http://vector-conversions.com/images/vector-vs-raster-jawlensky.jpg)\n", "\n", "more info about rasters vs vectors:\n", "http://vector-conversions.com/vectorizing/raster_vs_vector.html" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "subslide" } }, "source": [ "## Vector data\n", "\n", "* **Entities**: Points, Lines and Polygons\n", "* **Common formats**: ESRI Shapefiles, GeoJSON, TopoJSON, ...\n", "\n", "![Vector data](img/simple-vector-map.png)\n", "\n", "Image by M. W. Toews https://en.wikipedia.org/wiki/File:Simple_vector_map.svg, CC-BY
" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "subslide" } }, "source": [ "## Raster data\n", "\n", "* **Common formats**: GeoTIFF, JPEG2000, SRTM3, ... \n", "\n", "![Raster data](img/simple-raster.png)" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "subslide" } }, "source": [ "# GEO referenced \n", "\n", "* Vectors and Rasters can exist in fields other than GIS.\n", "* The are part of a GIS when they are Geo-Referenced.\n", "* For example: \n", " * A *Polygon* is Geo-Referenced when you know how to draw it on earth\n", " * A *Raster* is Geo-Referenced when you know where is the raster located " ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "subslide" } }, "source": [ "## Projections\n", "\n", "A method to locate points on earth\n", "\n", "TL;DR: A mess.\n", "\n", "* The Earth is approximately a sphere, which is not a [developable surface](https://en.wikipedia.org/wiki/Developable_surface) (i.e. _it cannot be flattened onto a plane without distortion_)\n", "* Therefore, we need _projections_ that attempt to represent the Earth on a plane, with various tradeoffs and limitations\n", "\n", "![Projections](img/projections.png)" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "# 2. ~~Why yet another~~ Existing software\n", "\n", "The landscape is complicated and somewhat difficult to navigate at times:\n", "\n", "![Landscape](img/landscape.png)" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "subslide" } }, "source": [ "Summary of key libraries:\n", "\n", "| Python | | C/C++ |\n", "|----------|---|--------|\n", "| pyproj | ⇒ | PROJ.4 |\n", "| Fiona | ⇒ | OGR |\n", "| Shapely | ⇒ | GEOS |\n", "| rasterio | ⇒ | GDAL |" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "# 3. telluric library\n", "\n", "_telluric is an open source (MIT) Python library to manage vector and raster geospatial data in an interactive and easy way._\n", "\n", "* Projection-aware geometric operations\n", "* Simple I/O for raster and vector data\n", "* Interactive visualization\n", "* 🕮 Documentation http://telluric.readthedocs.io/\n", "* 🔧 Source https://github.com/satellogic/telluric" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "subslide" } }, "source": [ "Importing for interactive use is short:" ] }, { "cell_type": "code", "execution_count": 1, "metadata": { "slideshow": { "slide_type": "fragment" } }, "outputs": [], "source": [ "import telluric as tl\n", "from telluric.constants import WGS84_CRS, WEB_MERCATOR_CRS" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "subslide" } }, "source": [ "Basic geometry definition using `GeoVector`: in the simplest case, the bounds and the projection (CRS)" ] }, { "cell_type": "code", "execution_count": 2, "metadata": { "slideshow": { "slide_type": "fragment" } }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "GeoVector(shape=POLYGON ((34.6 32, 34.6 32.3, 35 32.3, 35 32, 34.6 32)), crs=CRS({'init': 'epsg:4326'}))\n" ] } ], "source": [ "gush_dan = tl.GeoVector.from_bounds(\n", " xmin=34.6, ymin=32, xmax=35.0, ymax=32.3, crs=WGS84_CRS\n", ")\n", "print(gush_dan)" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "subslide" } }, "source": [ "### We added extra attention to Jupyter noteobook visualization, checkout this plot " ] }, { "cell_type": "code", "execution_count": 3, "metadata": { "slideshow": { "slide_type": "fragment" } }, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "/home/guyd/pEnvs/telluric-dev/local/lib/python3.5/site-packages/telluric/plotting.py:141: UserWarning: Plotting a limited representation of the data, use the .plot() method for further customization\n", " \"Plotting a limited representation of the data, use the .plot() method for further customization\")\n" ] }, { "data": { "image/svg+xml": [ "" ], "text/html": [ "