{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "# Example of DOV search methods for lithologische beschrijvingen" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "[![Binder](https://mybinder.org/badge_logo.svg)](https://mybinder.org/v2/gh/DOV-Vlaanderen/pydov/master?filepath=docs%2Fnotebooks%2Fsearch_lithologische_beschrijvingen.ipynb)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Use cases:\n", "* Select records in a bbox\n", "* Select records in a bbox with selected properties\n", "* Select records in a municipality\n", "* Get records using info from wfs fields, not available in the standard output dataframe" ] }, { "cell_type": "code", "execution_count": 1, "metadata": {}, "outputs": [], "source": [ "%matplotlib inline\n", "import os, sys\n", "import inspect" ] }, { "cell_type": "code", "execution_count": 2, "metadata": {}, "outputs": [], "source": [ "import pydov" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Get information about code base" ] }, { "cell_type": "code", "execution_count": 3, "metadata": {}, "outputs": [ { "output_type": "execute_result", "data": { "text/plain": [ "'Wanneer een boring wordt uitgevoerd wordt er een beschrijving gemaakt van het materiaal dat men aangeboord heeft. Men beschrijft de textuur, het soort materiaal, de kleur, mineralen, fossielen, ... per diepte-interval. Dit kan zowel op het veld gebeuren als achteraf indien de stalen bewaard worden. Deze beschrijvingen kunnen gedaan worden door bijvoorbeeld de boormeester of door een geoloog. Meestal wordt de boring beschreven per diepte-interval waarover men hetzelfde materiaal herkent. De mate van detail van de beschrijving kan afhankelijk zijn van het doeleinde waarvoor men de boring uitvoert of wie de beschrijving doet: is het om een geologische kaart te maken, dienen er monsters genomen te worden voor laboratoriumanalyses, ... In DOV worden de beschrijvingen van ontsluitingen boven het aardoppervlak, aangeduid als boringen met diepte 0 m, beschreven van 0 m tot 0 m.'" ] }, "metadata": {}, "execution_count": 3 } ], "source": [ "from pydov.search.interpretaties import LithologischeBeschrijvingenSearch\n", "ip_litho = LithologischeBeschrijvingenSearch()\n", "\n", "# information about the HydrogeologischeStratigrafie type (In Dutch):\n", "ip_litho.get_description()" ] }, { "cell_type": "code", "execution_count": 4, "metadata": {}, "outputs": [ { "output_type": "stream", "name": "stdout", "text": [ "pkey_interpretatie\nType_proef\nProefnummer\npkey_boring\nx\ny\nstart_interpretatie_mtaw\ndiepte_tot_m\ngemeente\nAuteurs\nDatum\nOpdrachten\nbetrouwbaarheid_interpretatie\nGeldig_van\nGeldig_tot\ndiepte_laag_van\ndiepte_laag_tot\nbeschrijving\n" ] } ], "source": [ "# information about the available fields for a HydrogeologischeStratigrafie object\n", "fields = ip_litho.get_fields()\n", "# print available fields\n", "for f in fields.values():\n", " print(f['name'])" ] }, { "cell_type": "code", "execution_count": 5, "metadata": {}, "outputs": [ { "output_type": "execute_result", "data": { "text/plain": [ "{'name': 'beschrijving',\n", " 'type': 'string',\n", " 'definition': 'Lithologische beschrijving van de laag in vrije tekst (onbeperkt in lengte)',\n", " 'notnull': False,\n", " 'query': False,\n", " 'cost': 10}" ] }, "metadata": {}, "execution_count": 5 } ], "source": [ "# print information for a certain field\n", "fields['beschrijving']" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The cost is an arbitrary attribute to indicate if the information is retrieved from a wfs query (cost = 1), \n", "or from an xml (cost = 10)" ] }, { "cell_type": "code", "execution_count": 6, "metadata": {}, "outputs": [ { "output_type": "execute_result", "data": { "text/plain": [ "{'name': 'Type_proef',\n", " 'definition': \"Het type van de proef waarvan de lithologische beschrijving een interpretatie is. Bij een lithogische beschrijving zal dit altijd 'Boring' zijn.\",\n", " 'type': 'string',\n", " 'notnull': False,\n", " 'query': True,\n", " 'cost': 1,\n", " 'values': {'Boring': 'De lithologische beschrijving is een interpretatie van een boring.'}}" ] }, "metadata": {}, "execution_count": 6 } ], "source": [ "# if an attribute can have several values, these are listed under 'values', e.g. for 'Type_proef':\n", "fields['Type_proef']" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Try-out of use cases" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Select interpretations in a bbox" ] }, { "cell_type": "code", "execution_count": 7, "metadata": {}, "outputs": [ { "output_type": "stream", "name": "stdout", "text": [ "[000/020] ....................\n" ] }, { "output_type": "execute_result", "data": { "text/plain": [ " pkey_interpretatie \\\n", "0 https://www.dov.vlaanderen.be/data/interpretat... \n", "1 https://www.dov.vlaanderen.be/data/interpretat... \n", "2 https://www.dov.vlaanderen.be/data/interpretat... \n", "3 https://www.dov.vlaanderen.be/data/interpretat... \n", "4 https://www.dov.vlaanderen.be/data/interpretat... \n", "\n", " pkey_boring \\\n", "0 https://www.dov.vlaanderen.be/data/boring/1895... \n", "1 https://www.dov.vlaanderen.be/data/boring/1895... \n", "2 https://www.dov.vlaanderen.be/data/boring/1879... \n", "3 https://www.dov.vlaanderen.be/data/boring/1879... \n", "4 https://www.dov.vlaanderen.be/data/boring/1879... \n", "\n", " betrouwbaarheid_interpretatie x y start_interpretatie_mtaw \\\n", "0 goed 152195.0 205501.0 11.0 \n", "1 goed 152195.0 205501.0 11.0 \n", "2 goed 152558.0 205063.0 9.5 \n", "3 goed 152558.0 205063.0 9.5 \n", "4 goed 152558.0 205063.0 9.5 \n", "\n", " diepte_laag_van diepte_laag_tot \\\n", "0 0.00 2.00 \n", "1 2.00 2.70 \n", "2 0.00 0.30 \n", "3 0.30 1.15 \n", "4 1.15 1.20 \n", "\n", " beschrijving \n", "0 Sable limoneux \n", "1 Sable verdâtre,fossilifère \n", "2 Terre végétale argileuse \n", "3 Sable jaune de plus en plus argileux \n", "4 Sable jaune de plus en plus argileux, argileux... " ], "text/html": "
\n\n\n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n
pkey_interpretatiepkey_boringbetrouwbaarheid_interpretatiexystart_interpretatie_mtawdiepte_laag_vandiepte_laag_totbeschrijving
0https://www.dov.vlaanderen.be/data/interpretat...https://www.dov.vlaanderen.be/data/boring/1895...goed152195.0205501.011.00.002.00Sable limoneux
1https://www.dov.vlaanderen.be/data/interpretat...https://www.dov.vlaanderen.be/data/boring/1895...goed152195.0205501.011.02.002.70Sable verdâtre,fossilifère
2https://www.dov.vlaanderen.be/data/interpretat...https://www.dov.vlaanderen.be/data/boring/1879...goed152558.0205063.09.50.000.30Terre végétale argileuse
3https://www.dov.vlaanderen.be/data/interpretat...https://www.dov.vlaanderen.be/data/boring/1879...goed152558.0205063.09.50.301.15Sable jaune de plus en plus argileux
4https://www.dov.vlaanderen.be/data/interpretat...https://www.dov.vlaanderen.be/data/boring/1879...goed152558.0205063.09.51.151.20Sable jaune de plus en plus argileux, argileux...
\n
" }, "metadata": {}, "execution_count": 7 } ], "source": [ "from pydov.util.location import Within, Box\n", "\n", "# Get all lithological descriptions in a bounding box (llx, lly, ulx, uly)\n", "# the pkey_boring link is not available below, but is in the df\n", "df = ip_litho.search(location=Within(Box(152145, 204930, 153150, 206935)))\n", "df = df[df.beschrijving.notnull()]\n", "df.head()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Select interpretations in a bbox with selected properties" ] }, { "cell_type": "code", "execution_count": 8, "metadata": {}, "outputs": [ { "output_type": "execute_result", "data": { "text/plain": [ "['PropertyIsBetween',\n", " 'PropertyIsEqualTo',\n", " 'PropertyIsGreaterThan',\n", " 'PropertyIsGreaterThanOrEqualTo',\n", " 'PropertyIsLessThan',\n", " 'PropertyIsLessThanOrEqualTo',\n", " 'PropertyIsLike',\n", " 'PropertyIsNotEqualTo',\n", " 'PropertyIsNull',\n", " 'SortProperty']" ] }, "metadata": {}, "execution_count": 8 } ], "source": [ "# list available query methods\n", "methods = [i for i,j in inspect.getmembers(sys.modules['owslib.fes'], \n", " inspect.isclass) \n", " if 'Property' in i]\n", "methods" ] }, { "cell_type": "code", "execution_count": 9, "metadata": {}, "outputs": [], "source": [ "from owslib.fes import PropertyIsGreaterThanOrEqualTo" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The property feature methodes listed above are available from the owslib module. These were not adapted for use in pydov." ] }, { "cell_type": "code", "execution_count": 10, "metadata": {}, "outputs": [ { "output_type": "stream", "name": "stdout", "text": [ "[000/001] c\n" ] }, { "output_type": "execute_result", "data": { "text/plain": [ " pkey_interpretatie \\\n", "0 https://www.dov.vlaanderen.be/data/interpretat... \n", "1 https://www.dov.vlaanderen.be/data/interpretat... \n", "2 https://www.dov.vlaanderen.be/data/interpretat... \n", "3 https://www.dov.vlaanderen.be/data/interpretat... \n", "4 https://www.dov.vlaanderen.be/data/interpretat... \n", "\n", " pkey_boring \\\n", "0 https://www.dov.vlaanderen.be/data/boring/1974... \n", "1 https://www.dov.vlaanderen.be/data/boring/1974... \n", "2 https://www.dov.vlaanderen.be/data/boring/1974... \n", "3 https://www.dov.vlaanderen.be/data/boring/1974... \n", "4 https://www.dov.vlaanderen.be/data/boring/1974... \n", "\n", " betrouwbaarheid_interpretatie x y start_interpretatie_mtaw \\\n", "0 goed 153147.0 206931.0 14.12 \n", "1 goed 153147.0 206931.0 14.12 \n", "2 goed 153147.0 206931.0 14.12 \n", "3 goed 153147.0 206931.0 14.12 \n", "4 goed 153147.0 206931.0 14.12 \n", "\n", " diepte_laag_van diepte_laag_tot beschrijving \n", "0 0.0 0.5 bruinachtige zandhoudende leem, geen kalk \n", "1 0.5 1.0 bleekgrijze leem \n", "2 1.0 1.5 bruingroen leemhoudend fijn zand \n", "3 1.5 2.0 bleekgrijs leemhoudend fijn zand \n", "4 2.0 2.5 idem, met veenresten en kalkhoudend " ], "text/html": "
\n\n\n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n
pkey_interpretatiepkey_boringbetrouwbaarheid_interpretatiexystart_interpretatie_mtawdiepte_laag_vandiepte_laag_totbeschrijving
0https://www.dov.vlaanderen.be/data/interpretat...https://www.dov.vlaanderen.be/data/boring/1974...goed153147.0206931.014.120.00.5bruinachtige zandhoudende leem, geen kalk
1https://www.dov.vlaanderen.be/data/interpretat...https://www.dov.vlaanderen.be/data/boring/1974...goed153147.0206931.014.120.51.0bleekgrijze leem
2https://www.dov.vlaanderen.be/data/interpretat...https://www.dov.vlaanderen.be/data/boring/1974...goed153147.0206931.014.121.01.5bruingroen leemhoudend fijn zand
3https://www.dov.vlaanderen.be/data/interpretat...https://www.dov.vlaanderen.be/data/boring/1974...goed153147.0206931.014.121.52.0bleekgrijs leemhoudend fijn zand
4https://www.dov.vlaanderen.be/data/interpretat...https://www.dov.vlaanderen.be/data/boring/1974...goed153147.0206931.014.122.02.5idem, met veenresten en kalkhoudend
\n
" }, "metadata": {}, "execution_count": 10 } ], "source": [ "# Get deep boreholes in a bounding box \n", "from owslib.fes import PropertyIsEqualTo\n", "# the propertyname can be any of the fields of the hydrogeological interpretations object that belong to the wfs source\n", "# the literal is always a string, no matter what its definition is in the boring object (string, float...)\n", "query = PropertyIsGreaterThanOrEqualTo(\n", " propertyname='betrouwbaarheid_interpretatie', literal='goed')\n", "df = ip_litho.search(location=Within(Box(153145, 206930, 153150, 206935)),\n", " query=query\n", " )\n", "df.head()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Select interpretations in a municipality" ] }, { "cell_type": "code", "execution_count": 11, "metadata": {}, "outputs": [ { "output_type": "stream", "name": "stdout", "text": [ "[000/055] ..................................................\n", "[050/055] .....\n" ] }, { "output_type": "execute_result", "data": { "text/plain": [ " pkey_interpretatie \\\n", "0 https://www.dov.vlaanderen.be/data/interpretat... \n", "1 https://www.dov.vlaanderen.be/data/interpretat... \n", "2 https://www.dov.vlaanderen.be/data/interpretat... \n", "3 https://www.dov.vlaanderen.be/data/interpretat... \n", "4 https://www.dov.vlaanderen.be/data/interpretat... \n", "\n", " pkey_boring \\\n", "0 https://www.dov.vlaanderen.be/data/boring/1870... \n", "1 https://www.dov.vlaanderen.be/data/boring/1870... \n", "2 https://www.dov.vlaanderen.be/data/boring/1870... \n", "3 https://www.dov.vlaanderen.be/data/boring/1870... \n", "4 https://www.dov.vlaanderen.be/data/boring/1870... \n", "\n", " betrouwbaarheid_interpretatie x y start_interpretatie_mtaw \\\n", "0 goed 151724.0 203063.0 16.0 \n", "1 goed 151724.0 203063.0 16.0 \n", "2 goed 151724.0 203063.0 16.0 \n", "3 goed 151724.0 203063.0 16.0 \n", "4 goed 151724.0 203063.0 16.0 \n", "\n", " diepte_laag_van diepte_laag_tot \\\n", "0 0.0 0.0 \n", "1 0.0 1.0 \n", "2 1.0 5.0 \n", "3 5.0 10.2 \n", "4 10.2 21.0 \n", "\n", " beschrijving \n", "0 Terre végétale. \n", "1 Sable jaune, mouvant à la base \n", "2 Sable jaune, mouvant à la base glauconifére, p... \n", "3 Argile verte compacte, avec pyrites \n", "4 Un separia. " ], "text/html": "
\n\n\n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n
pkey_interpretatiepkey_boringbetrouwbaarheid_interpretatiexystart_interpretatie_mtawdiepte_laag_vandiepte_laag_totbeschrijving
0https://www.dov.vlaanderen.be/data/interpretat...https://www.dov.vlaanderen.be/data/boring/1870...goed151724.0203063.016.00.00.0Terre végétale.
1https://www.dov.vlaanderen.be/data/interpretat...https://www.dov.vlaanderen.be/data/boring/1870...goed151724.0203063.016.00.01.0Sable jaune, mouvant à la base
2https://www.dov.vlaanderen.be/data/interpretat...https://www.dov.vlaanderen.be/data/boring/1870...goed151724.0203063.016.01.05.0Sable jaune, mouvant à la base glauconifére, p...
3https://www.dov.vlaanderen.be/data/interpretat...https://www.dov.vlaanderen.be/data/boring/1870...goed151724.0203063.016.05.010.2Argile verte compacte, avec pyrites
4https://www.dov.vlaanderen.be/data/interpretat...https://www.dov.vlaanderen.be/data/boring/1870...goed151724.0203063.016.010.221.0Un separia.
\n
" }, "metadata": {}, "execution_count": 11 } ], "source": [ "query = PropertyIsEqualTo(propertyname='gemeente',\n", " literal='Aartselaar')\n", "df = ip_litho.search(query=query)\n", "df.head()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Visualize results" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Using Folium, we can display the results of our search on a map." ] }, { "cell_type": "code", "execution_count": 12, "metadata": {}, "outputs": [], "source": [ "# import the necessary modules (not included in the requirements of pydov!)\n", "import folium\n", "from folium.plugins import MarkerCluster\n", "from pyproj import Transformer" ] }, { "cell_type": "code", "execution_count": 13, "metadata": {}, "outputs": [], "source": [ "# convert the coordinates to lat/lon for folium\n", "def convert_latlon(x1, y1):\n", " transformer = Transformer.from_crs(\"epsg:31370\", \"epsg:4326\", always_xy=True)\n", " x2,y2 = transformer.transform(x1, y1)\n", " return x2, y2\n", "\n", "df['lon'], df['lat'] = zip(*map(convert_latlon, df['x'], df['y'])) \n", "# convert to list\n", "loclist = df[['lat', 'lon']].values.tolist()" ] }, { "cell_type": "code", "execution_count": 14, "metadata": {}, "outputs": [ { "output_type": "execute_result", "data": { "text/plain": [ "" ], "text/html": "
Make this Notebook Trusted to load map: File -> Trust Notebook
" }, "metadata": {}, "execution_count": 14 } ], "source": [ "# initialize the Folium map on the centre of the selected locations, play with the zoom until ok\n", "fmap = folium.Map(location=[df['lat'].mean(), df['lon'].mean()], zoom_start=12)\n", "marker_cluster = MarkerCluster().add_to(fmap)\n", "for loc in range(0, len(loclist)):\n", " # limit marker size for folium (:10)\n", " folium.Marker(loclist[loc], popup=df['beschrijving'][loc][:10]).add_to(marker_cluster)\n", "fmap" ] } ], "metadata": { "kernelspec": { "display_name": "Python 3.7.5 64-bit ('venv': venv)", "language": "python", "name": "python37564bitvenvvenv7153db2e85324c8ea2b6eb355f534b21" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.8.5-final" } }, "nbformat": 4, "nbformat_minor": 4 }