{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "b7f20a06-9169-463a-be95-d588df540d60",
   "metadata": {
    "editable": true,
    "slideshow": {
     "slide_type": ""
    },
    "tags": []
   },
   "source": [
    "---\n",
    "title: Get data from the Hamburg Urban Data Portal using WFS\n",
    "subtitle: Learn how to access vector data from the Hamburg Urban Data Portal with WFS\n",
    "authors:\n",
    "  - name: Anne Fouilloux\n",
    "    orcid: 0000-0002-1784-2920\n",
    "    github: annefou\n",
    "    affiliations:\n",
    "      - id: Simula Research Laboratory\n",
    "        institution: Simula Research Laboratory\n",
    "        ror: 00vn06n10\n",
    "date: 2025-05-14\n",
    "keywords : flooding\n",
    "---"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "fd45f284-40b9-4848-92d1-b7bb352bf950",
   "metadata": {},
   "source": [
    "![Hambburg Urban data platform logo](https://www.en.urbandataplatform.hamburg/resource/crblob/797502/9c2f2d2edc1149673134bf88ece1dda5/logo-udp-data.jpg)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "98abf159-1910-4022-aac3-1786b3e6f599",
   "metadata": {
    "editable": true,
    "slideshow": {
     "slide_type": ""
    },
    "tags": []
   },
   "source": [
    "(introduction)=\n",
    "## Introduction\n",
    "\n",
    "### The Hamburg Urban Data Platform\n",
    "The [Hamburg Urban Data Platform (UDP_HH)](https://www.urbandataplatform.hamburg/) is a central component of the city's digital transformation strategy, serving as a comprehensive hub for collecting, integrating, and disseminating urban data. Developed by the State Office for Geoinformation and Surveying (LGV) in collaboration with the CityScienceLab at HafenCity University, the platform offers standardized interfaces and real-time access to a wide array of datasets, including traffic counts, environmental measurements, and infrastructure details. These datasets are accessible to public authorities, businesses, researchers, and citizens, facilitating data-driven decision-making and fostering innovation. The platform's user-friendly interface, known as the UDP Cockpit, allows users to explore and visualize data interactively, supporting various applications from urban planning to environmental monitoring. By promoting transparency and interoperability, the Urban Data Platform plays a pivotal role in enhancing Hamburg's status as a smart city. \n",
    "\n",
    "### Purpose of this Jupyter Notebook\n",
    "This jupyter notebook shows how to access data from the Hamburg Urban Data portal using WFS. In this example, we get:\n",
    "\n",
    "- Data from the Hamburg Urban Data Portal [Hochwasserschutzlinie im Land Bremen](https://www.metaver.de:443/trefferanzeige?docuuid=0EAD7F6B-5A72-4906-B686-6B674BAB27E5&rstart=20&currentSelectorPage=1&f=type:opendata;&lang=de) into a geopandas data frame\n",
    "- Save the data into geosjon with **epsg:4326** projection for further usage for instance in JupyterGIS."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b6be7f45-833f-4548-8813-6e4cb40e6b65",
   "metadata": {
    "editable": true,
    "slideshow": {
     "slide_type": ""
    },
    "tags": []
   },
   "source": [
    "(overview)=\n",
    ":::{hint} Overview\n",
    "**Questions**\n",
    "- How can we access flood-related geospatial data from the Hamburg Urban Data Portal?\n",
    "- What is the role of WFS (Web Feature Service) in retrieving data for flood risk analysis?\n",
    "- How can we convert this data into a format suitable for further geospatial analysis (e.g., GeoJSON with EPSG:4326)?\n",
    "\n",
    "**Objectives**\n",
    "- Retrieve geospatial flood protection data (e.g., flood protection lines) from the Hamburg Urban Data Portal using WFS.\n",
    "- Convert and save the data in a standardized coordinate reference system (EPSG:4326) for use in tools like JupyterGIS or QGIS.\n",
    ":::"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "12ddd5e7-d1ac-41fa-8014-dcbbecb50bfc",
   "metadata": {
    "editable": true,
    "slideshow": {
     "slide_type": ""
    },
    "tags": []
   },
   "source": [
    "(requirements)=\n",
    "## Requirements\n",
    "Start installing and importing the necessary libraries\n",
    "### Install Python Packages"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "7704891b-0444-4bb5-8905-ecf675a7d8aa",
   "metadata": {
    "editable": true,
    "hide-output": true,
    "jupyter": {
     "outputs_hidden": true
    },
    "slideshow": {
     "slide_type": ""
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "pip install jupyterlab_myst geopandas geojson matplotlib"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5af740ea-7c43-4a8a-b65e-c1edcb85b694",
   "metadata": {
    "editable": true,
    "slideshow": {
     "slide_type": ""
    },
    "tags": []
   },
   "source": [
    "(setup)=\n",
    "### Import Python Packages"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "3e673948-b48a-4628-8a0b-0fe2c2594a12",
   "metadata": {},
   "outputs": [],
   "source": [
    "import os\n",
    "from io import BytesIO  # Import BytesIO\n",
    "\n",
    "import geopandas as gpd\n",
    "import requests"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e49f4f52-d7db-48d1-85d5-a8d345dd369b",
   "metadata": {
    "editable": true,
    "slideshow": {
     "slide_type": ""
    },
    "tags": []
   },
   "source": [
    "(parameters)=\n",
    "## Input data required\n",
    "### Set input parameters for WFS request"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "dd7b60a1-e239-4f3b-a71e-5630bc3508bf",
   "metadata": {
    "editable": true,
    "slideshow": {
     "slide_type": ""
    },
    "tags": [
     "parameters"
    ]
   },
   "outputs": [],
   "source": [
    "# WFS URL\n",
    "wfs_url: str = (\n",
    "    \"https://geoportal.saarland.de/arcgis/services/Internet/Hochwasser_WFS/MapServer/WFSServer?&request=GetCapabilities&VERSION=1.1.0&SERVICE=WFS\"\n",
    ")\n",
    "\n",
    "# Feature type\n",
    "feature_typename: str = \"Hochwasser_WFS:Flaeche100_generalisiert\"\n",
    "\n",
    "# Output file name (Geojson)\n",
    "filename: str = \"HQ100_Flaeche100_generalisiert_4326.geojson\""
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0896a02c-6da6-4072-a522-e715ff2e6b88",
   "metadata": {
    "editable": true,
    "slideshow": {
     "slide_type": ""
    },
    "tags": []
   },
   "source": [
    "(retrieve)=\n",
    "## Get data from the Hamburg Urban Data Portal using WFS"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "81a20f66-d553-4d36-ad33-72c382de329b",
   "metadata": {
    "editable": true,
    "slideshow": {
     "slide_type": ""
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "# Specify parameters\n",
    "params = {\n",
    "    \"service\": \"WFS\",\n",
    "    \"request\": \"GetFeature\",\n",
    "    \"typeName\": feature_typename,\n",
    "}\n",
    "\n",
    "try:\n",
    "    # Fetch data\n",
    "    r = requests.get(wfs_url, params=params)\n",
    "    r.raise_for_status()  # Check for HTTP errors\n",
    "\n",
    "    # Read GML directly into GeoDataFrame\n",
    "    gml_data = BytesIO(r.content)\n",
    "    data = gpd.read_file(gml_data, driver=\"GML\")\n",
    "\n",
    "except requests.exceptions.RequestException as e:\n",
    "    print(f\"Request failed: {e}\")\n",
    "except Exception as e:\n",
    "    print(f\"Failed to parse GML: {e}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c7425a97-ca53-4646-a858-c8f5ae6deadd",
   "metadata": {
    "editable": true,
    "slideshow": {
     "slide_type": ""
    },
    "tags": []
   },
   "source": [
    "(figure)=\n",
    "## Quick visualisation"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "677c96bf-f557-434d-9ef8-6cf77c34ccbe",
   "metadata": {
    "editable": true,
    "slideshow": {
     "slide_type": ""
    },
    "tags": [
     "figure"
    ]
   },
   "outputs": [],
   "source": [
    "data.plot()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "360f4dad-972e-4fa9-9034-4377e421b95b",
   "metadata": {
    "editable": true,
    "slideshow": {
     "slide_type": ""
    },
    "tags": []
   },
   "source": [
    "(outputs)=\n",
    "## Save to geojson with WSG84 (epsg:4326) projection"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "451aa919-77ee-4e2b-940f-40396c8759a2",
   "metadata": {
    "editable": true,
    "slideshow": {
     "slide_type": ""
    },
    "tags": [
     "outputs"
    ]
   },
   "outputs": [],
   "source": [
    "if os.path.exists(filename):\n",
    "    print(\"Skipped creation: file already exists.\")\n",
    "else:\n",
    "    data.to_crs(4326).to_file(filename, encoding=\"utf-8\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "9c2dc38b-2c33-4c29-9025-12eb2adaf71a",
   "metadata": {
    "editable": true,
    "slideshow": {
     "slide_type": ""
    },
    "tags": []
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "3c091c7c-221a-4f3c-85d3-34183e3abe11",
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.13.1"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}