{
"cells": [
{
"cell_type": "markdown",
"id": "0f09e1c5",
"metadata": {},
"source": [
"# Speedtest Data\n",
"\n",
"This example will use data collected from Ookla's Speed Test application and [shared publicly in the AWS Open Data Registry](https://registry.opendata.aws/speedtest-global-performance/). From the AWS page:\n",
"\n",
"> Global fixed broadband and mobile (cellular) network performance, allocated to zoom level 16 web mercator tiles (approximately 610.8 meters by 610.8 meters at the equator). Data is provided in both Shapefile format as well as Apache Parquet with geometries represented in Well Known Text (WKT) projected in EPSG:4326. Download speed, upload speed, and latency are collected via the Speedtest by Ookla applications for Android and iOS and averaged for each tile.\n",
"\n",
"You can view a [hosted version of this notebook on Notebook Sharing Space](https://notebooksharing.space/view/2c2fc0b1ef5f93c70a8c36de30b560d1316d16760714742dcf22a119f4991762#displayOptions=) (35MB download).\n"
]
},
{
"cell_type": "markdown",
"id": "fbfeace7-81ee-48d4-af01-5172618e83ea",
"metadata": {},
"source": [
"### Dependencies\n",
"\n",
"```\n",
"pip install lonboard \"geopandas>=1\" palettable sidecar\n",
"```\n"
]
},
{
"cell_type": "markdown",
"id": "7dd1dfef-3756-49d9-9480-9a4cdba22345",
"metadata": {},
"source": [
"## Imports\n"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "d1678764",
"metadata": {},
"outputs": [],
"source": [
"from pathlib import Path\n",
"\n",
"import geopandas as gpd\n",
"import pandas as pd\n",
"import shapely\n",
"from palettable.colorbrewer.diverging import BrBG_10\n",
"from sidecar import Sidecar\n",
"\n",
"from lonboard import Map, ScatterplotLayer\n",
"from lonboard.colormap import apply_continuous_cmap"
]
},
{
"cell_type": "markdown",
"id": "d51ca576",
"metadata": {},
"source": [
"## Fetch data\n"
]
},
{
"cell_type": "markdown",
"id": "c747d8b9-94b9-421a-967a-8350bf72de9a",
"metadata": {},
"source": [
"The URL for a single data file for mobile network speeds in the first quarter of 2019:\n"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "34ac8eae",
"metadata": {},
"outputs": [],
"source": [
"url = \"https://ookla-open-data.s3.us-west-2.amazonaws.com/parquet/performance/type=mobile/year=2019/quarter=1/2019-01-01_performance_mobile_tiles.parquet\""
]
},
{
"cell_type": "markdown",
"id": "5991ef2c-5db0-4110-b6a1-b33fcbddad0d",
"metadata": {},
"source": [
"The data used in this example is relatively large. In the cell below, we cache the downloading and preparation of the dataset so that it's faster to run this notebook the second time.\n",
"\n",
"We fetch two columns — `avg_d_kbps` and `tile` — from this data file directly from AWS. The `pd.read_parquet` command will perform a network request for these columns from the data file, so it may take a while on a slow network connection. `avg_d_kbps` is the average download speed for that data point in kilobits per second. `tile` is the WKT string representing a given zoom-16 Web Mercator tile.\n",
"\n",
"The `tile` column contains _strings_ representing WKT-formatted geometries. We need to parse those strings into geometries. Then for simplicity we'll convert into their centroids.\n"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "7c20cb4c-9746-486f-aef7-95dd2dedd6a5",
"metadata": {},
"outputs": [],
"source": [
"local_path = Path(\"internet-speeds.parquet\")\n",
"if local_path.exists():\n",
" gdf = gpd.read_parquet(local_path)\n",
"else:\n",
" columns = [\"avg_d_kbps\", \"tile\"]\n",
" df = pd.read_parquet(url, columns=columns)\n",
"\n",
" tile_geometries = shapely.from_wkt(df[\"tile\"])\n",
" tile_centroids = shapely.centroid(tile_geometries)\n",
" gdf = gpd.GeoDataFrame(df[[\"avg_d_kbps\"]], geometry=tile_centroids, crs=\"EPSG:4326\")\n",
" gdf.to_parquet(local_path)"
]
},
{
"cell_type": "markdown",
"id": "5852aa94-2d18-4a1b-b379-be19682d57eb",
"metadata": {},
"source": [
"We can take a quick look at this data:\n"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "4b27e9a4",
"metadata": {},
"outputs": [],
"source": [
"gdf.head()"
]
},
{
"cell_type": "markdown",
"id": "65436a4a-c498-4f40-ba79-1082062376bf",
"metadata": {},
"source": [
"To ensure that this demo is snappy on most computers, we'll filter to a bounding box over Europe.\n",
"\n",
"If you're on a recent computer, feel free to comment out the next line.\n"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "80326895-70ba-4f4b-a7b3-106b4bbd36d9",
"metadata": {},
"outputs": [],
"source": [
"gdf = gdf.cx[-11.83:25.5, 34.9:59]"
]
},
{
"cell_type": "markdown",
"id": "3cc2215e-7706-4ab3-b674-3de4ca41899c",
"metadata": {},
"source": [
"Even this filtered data frame still has 800,000 rows, so it's still a lot of data to explore:\n"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "a34a6a27-0259-4da9-94c4-923466da05fb",
"metadata": {},
"outputs": [],
"source": [
"gdf"
]
},
{
"cell_type": "markdown",
"id": "81a61ec4-2a09-40c0-aa92-7dca570bbd49",
"metadata": {},
"source": [
"To render point data, first create a `ScatterplotLayer` and then add it to a `Map` object:\n"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "733fe65a-599e-474a-9e31-0435193b1fd3",
"metadata": {},
"outputs": [],
"source": [
"sidecar = Sidecar()"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "570ab332-3767-4246-8d83-df4625b2ae48",
"metadata": {},
"outputs": [],
"source": [
"layer = ScatterplotLayer.from_geopandas(gdf)\n",
"m = Map(layer, _height=800)\n",
"with sidecar:\n",
" display(m)"
]
},
{
"cell_type": "markdown",
"id": "6f4d89c3-282a-4beb-9f35-68eb9645e8c0",
"metadata": {},
"source": [
"We can look at the [documentation for `ScatterplotLayer`](https://developmentseed.org/lonboard/latest/api/layers/scatterplot-layer/) to see what other rendering options it allows. Let's set the fill color to something other than black:\n"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "3912b241-577f-4ac3-b78f-2702e89d6010",
"metadata": {},
"outputs": [],
"source": [
"layer.get_fill_color = [0, 0, 200, 200]"
]
},
{
"cell_type": "markdown",
"id": "b66a73b6-69c1-4661-a7ad-2dd3941c8753",
"metadata": {},
"source": [
"Blue is pretty, but the map would be more informative if we colored each point by a relevant characteristic. In this case, we have the download speed associated with each location, so let's use that!\n"
]
},
{
"cell_type": "markdown",
"id": "ce630455-3e19-47f1-bb69-81fdcd99b126",
"metadata": {},
"source": [
"Here we compute a linear statistic for the download speed. Given a minimum bound of `5000` and a maximum bound of `50,000` the normalized speed is linearly scaled to between 0 and 1.\n"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "179071b3",
"metadata": {},
"outputs": [],
"source": [
"min_bound = 5000\n",
"max_bound = 50000\n",
"download_speed = gdf[\"avg_d_kbps\"]\n",
"normalized_download_speed = (download_speed - min_bound) / (max_bound - min_bound)"
]
},
{
"cell_type": "markdown",
"id": "7f678b8e-ad23-4ebd-842c-c5158e1ec741",
"metadata": {},
"source": [
"`normalized_download_speed` is now linearly scaled based on the bounds provided above. Keep in mind that the **input range of the colormap is 0-1**. So any values that are below 0 will receive the left-most color in the colormap, while any values above 1 will receive the right-most color in the colormap.\n"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "a8df3963-2bc2-4f89-8a38-20e232a13932",
"metadata": {},
"outputs": [],
"source": [
"normalized_download_speed"
]
},
{
"cell_type": "markdown",
"id": "dc0e388f-eeed-4339-bd54-0491f79f45aa",
"metadata": {},
"source": [
"We can use any colormap provided by the [`palettable`](https://github.com/jiffyclub/palettable) package. Let's inspect the `BrBG_10` diverging colormap below:\n"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "9d5347e2-84c7-40bc-af45-c8638188709e",
"metadata": {},
"outputs": [],
"source": [
"BrBG_10.mpl_colormap"
]
},
{
"cell_type": "markdown",
"id": "8eb59fd6-3b7d-4db9-ad9b-a66e4e58152f",
"metadata": {},
"source": [
"Now let's apply the colormap on `normalized_download_speed` using a helper provided by `lonboard`. We can set it on `layer.get_fill_color` to update the existing colors.\n"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "5a77f728-9cbe-4372-9bfd-d6dee4b93a01",
"metadata": {},
"outputs": [],
"source": [
"layer.get_fill_color = apply_continuous_cmap(\n",
" normalized_download_speed,\n",
" BrBG_10,\n",
" alpha=0.7,\n",
")"
]
},
{
"cell_type": "markdown",
"id": "44611c75-f9e6-4f53-af7d-c640d641dc15",
"metadata": {},
"source": [
"After running the above cell, you should see the map above update with a different color per point!\n"
]
},
{
"cell_type": "markdown",
"id": "0bd168fe-53a1-4313-9c11-720edbfa5c6c",
"metadata": {},
"source": [
"We can pass an array into any of the \"accessors\" supported by the layer (this is any attribute that starts with `get_*`).\n",
"\n",
"For demonstration purposes, let's also set `get_radius` to `normalized_download_speed`.\n"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "579233ef-e077-4c8f-a111-f33d44f30a0d",
"metadata": {},
"outputs": [],
"source": [
"layer.get_radius = normalized_download_speed * 200\n",
"layer.radius_units = \"meters\"\n",
"layer.radius_min_pixels = 0.5"
]
},
{
"cell_type": "markdown",
"id": "06e2338e-8461-49f7-9884-475fffc64789",
"metadata": {},
"source": [
"After running the above cell, you should see the map updated to have a different radius per point!\n"
]
}
],
"metadata": {
"kernelspec": {
"display_name": "lonboard",
"language": "python",
"name": "lonboard"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.11.8"
},
"widgets": {
"application/vnd.jupyter.widget-state+json": {
"state": {},
"version_major": 2,
"version_minor": 0
}
}
},
"nbformat": 4,
"nbformat_minor": 5
}