{ "cells": [ { "cell_type": "markdown", "metadata": { "id": "Q0tHKpvv4wAT" }, "source": [ "# Converting Multi-layer KML to a GeoPackage\n", "\n", "We have a complex KML file with 2000+ individual layers. Each layer contains only a single feature but having large number of layers means it takes a long time to read and parse them. We can use GeoPandas to read all the layers and merge layers having the same geometry and write the cleaned layers to a GeoPacjage." ] }, { "cell_type": "markdown", "metadata": { "id": "5hm2Ypxk4wAW" }, "source": [ "## Setup and Import\n", "\n", "The following blocks of code will install and import the required packages in Colab environment." ] }, { "cell_type": "code", "execution_count": 3, "metadata": { "id": "zGzb4G0m4wAX" }, "outputs": [], "source": [ "try:\n", " import geopandas\n", "except ModuleNotFoundError:\n", " if 'google.colab' in str(get_ipython()):\n", " !apt install -qq libspatialindex-dev\n", " !pip install fiona shapely pyproj rtree --quiet\n", " !pip install geopandas --quiet\n", " else:\n", " print('geopandas not found, please install via conda in your environment')" ] }, { "cell_type": "code", "execution_count": 4, "metadata": { "id": "2DA0u8bm4wAY" }, "outputs": [], "source": [ "import os\n", "import pandas as pd\n", "import geopandas as gpd\n", "import fiona" ] }, { "cell_type": "markdown", "source": [ "## Methodology" ], "metadata": { "id": "w5u2Ux1HDay3" } }, { "cell_type": "markdown", "source": [ "Create a list of layers as a Pandas Series." ], "metadata": { "id": "IapiDXY__67n" } }, { "cell_type": "code", "source": [ "file_path = 'input.kml'\n", "layers = pd.Series(fiona.listlayers(file_path))\n", "layers" ], "metadata": { "id": "CzQMdXBk5nim", "outputId": "fe572ed1-b4c2-45a3-bd4b-a2a726723cbf", "colab": { "base_uri": "https://localhost:8080/" } }, "execution_count": 5, "outputs": [ { "output_type": "execute_result", "data": { "text/plain": [ "0 Text [7193A]\n", "1 Point [71939]\n", "2 Text [71937]\n", "3 Point [71936]\n", "4 Text [71934]\n", " ... \n", "2177 MText [70BD3]\n", "2178 MText [70BD2]\n", "2179 MText [70BD1]\n", "2180 Polyline [70BD0]\n", "2181 Untitled layer\n", "Length: 2182, dtype: object" ] }, "metadata": {}, "execution_count": 5 } ] }, { "cell_type": "markdown", "source": [ "Iterate through each layer and read it using GeoPandas. Create a list of GeoDataFrames for each layer.\n", "\n", "This step can take time.\n", "\n", " Use `tqdm` to display a progress bar." ], "metadata": { "id": "87OQHFTPAF4y" } }, { "cell_type": "code", "execution_count": 9, "metadata": { "id": "5ymb_dco4wAY", "outputId": "47e91ce4-a6eb-4769-917e-c40117c1701a", "colab": { "base_uri": "https://localhost:8080/", "height": 49, "referenced_widgets": [ "b343481e79fa47a894a1229cb903e653", "6acc74aa88394debbbfff1a759332e83", "c313fcd671cc4292821f9dba2d183141", "e74da2b122924def95b6514108f321a0", "4159e1d7eda74084af4656484f37d9f0", "595b954b5a334bc1a62cb09e64dcb243", "3e3d365c317d4551a45c9f87144ff85c", "51b04ec861ae493eae500a124ffa5d52", "34c771fd83b4424a918e28861431b359", "25f4bc144e6948e38b52d05317169a01", "2db6e63c7b5b405ea5ab8bb0c3f2b0fe" ] } }, "outputs": [ { "output_type": "display_data", "data": { "text/plain": [ " 0%| | 0/2182 [00:00 Generic Propert... \n", "0 B2-460 \n", "0 Point [71936]:0
Generic Propert... \n", "0 B2-459 \n", "\n", " geometry \n", "0 POINT Z (78.50425 17.68199 0.00000) \n", "0 MULTILINESTRING Z ((78.50424 17.68198 0.00000,... \n", "0 POINT Z (78.49496 17.68535 0.00000) \n", "0 MULTILINESTRING Z ((78.49496 17.68535 0.00000,... \n", "0 POINT Z (78.49141 17.69321 0.00000) " ], "text/html": [ "\n", "
\n", "
\n", "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
NameDescriptiongeometry
0B2-461POINT Z (78.50425 17.68199 0.00000)
0Point [71939]:0<table> <tr> <td align=\"right\">Generic Propert...MULTILINESTRING Z ((78.50424 17.68198 0.00000,...
0B2-460POINT Z (78.49496 17.68535 0.00000)
0Point [71936]:0<table> <tr> <td align=\"right\">Generic Propert...MULTILINESTRING Z ((78.49496 17.68535 0.00000,...
0B2-459POINT Z (78.49141 17.69321 0.00000)
\n", "
\n", " \n", " \n", " \n", "\n", " \n", "
\n", "
\n", " " ] }, "metadata": {}, "execution_count": 13 } ] }, { "cell_type": "markdown", "source": [ "The geometry column includes different geometries, each geometry type has to be saved into a separate layers. Iterate over goemetry types and write out a layer in the output geopackage." ], "metadata": { "id": "ld_6MtS3AXtO" } }, { "cell_type": "code", "source": [ "output_file = 'merged.gpkg'" ], "metadata": { "id": "jK_7eXbb6m6y" }, "execution_count": 11, "outputs": [] }, { "cell_type": "code", "source": [ "for geomtype in merged.geom_type.unique():\n", " merged[merged.geom_type == geomtype].to_file(output_file, driver='GPKG', layer=geomtype)" ], "metadata": { "id": "iGSu59tE6zNc" }, "execution_count": 12, "outputs": [] } ], "metadata": { "kernelspec": { "display_name": "Python 3 (ipykernel)", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.10.5" }, "colab": { "provenance": [] }, "widgets": { "application/vnd.jupyter.widget-state+json": { "b343481e79fa47a894a1229cb903e653": { "model_module": "@jupyter-widgets/controls", "model_name": "HBoxModel", "model_module_version": "1.5.0", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "HBoxModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "HBoxView", "box_style": "", "children": [ "IPY_MODEL_6acc74aa88394debbbfff1a759332e83", "IPY_MODEL_c313fcd671cc4292821f9dba2d183141", "IPY_MODEL_e74da2b122924def95b6514108f321a0" ], "layout": "IPY_MODEL_4159e1d7eda74084af4656484f37d9f0" } }, "6acc74aa88394debbbfff1a759332e83": { "model_module": "@jupyter-widgets/controls", "model_name": "HTMLModel", "model_module_version": "1.5.0", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "HTMLModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "HTMLView", "description": "", "description_tooltip": null, "layout": "IPY_MODEL_595b954b5a334bc1a62cb09e64dcb243", "placeholder": "​", "style": "IPY_MODEL_3e3d365c317d4551a45c9f87144ff85c", "value": "100%" } }, "c313fcd671cc4292821f9dba2d183141": { "model_module": "@jupyter-widgets/controls", "model_name": "FloatProgressModel", "model_module_version": "1.5.0", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "FloatProgressModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "ProgressView", "bar_style": "success", "description": "", "description_tooltip": null, "layout": "IPY_MODEL_51b04ec861ae493eae500a124ffa5d52", "max": 2182, "min": 0, "orientation": "horizontal", "style": "IPY_MODEL_34c771fd83b4424a918e28861431b359", "value": 2182 } }, "e74da2b122924def95b6514108f321a0": { "model_module": "@jupyter-widgets/controls", "model_name": "HTMLModel", "model_module_version": "1.5.0", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "HTMLModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "HTMLView", "description": "", "description_tooltip": null, "layout": "IPY_MODEL_25f4bc144e6948e38b52d05317169a01", "placeholder": "​", "style": "IPY_MODEL_2db6e63c7b5b405ea5ab8bb0c3f2b0fe", "value": " 2182/2182 [08:58<00:00, 3.86it/s]" } }, "4159e1d7eda74084af4656484f37d9f0": { "model_module": "@jupyter-widgets/base", "model_name": "LayoutModel", "model_module_version": "1.2.0", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "595b954b5a334bc1a62cb09e64dcb243": { "model_module": "@jupyter-widgets/base", "model_name": "LayoutModel", "model_module_version": "1.2.0", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "3e3d365c317d4551a45c9f87144ff85c": { "model_module": "@jupyter-widgets/controls", "model_name": "DescriptionStyleModel", "model_module_version": "1.5.0", "state": { "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "DescriptionStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "StyleView", "description_width": "" } }, "51b04ec861ae493eae500a124ffa5d52": { "model_module": "@jupyter-widgets/base", "model_name": "LayoutModel", "model_module_version": "1.2.0", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "34c771fd83b4424a918e28861431b359": { "model_module": "@jupyter-widgets/controls", "model_name": "ProgressStyleModel", "model_module_version": "1.5.0", "state": { "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "ProgressStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "StyleView", "bar_color": null, "description_width": "" } }, "25f4bc144e6948e38b52d05317169a01": { "model_module": "@jupyter-widgets/base", "model_name": "LayoutModel", "model_module_version": "1.2.0", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "2db6e63c7b5b405ea5ab8bb0c3f2b0fe": { "model_module": "@jupyter-widgets/controls", "model_name": "DescriptionStyleModel", "model_module_version": "1.5.0", "state": { "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "DescriptionStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "StyleView", "description_width": "" } } } } }, "nbformat": 4, "nbformat_minor": 0 }