{ "cells": [ { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "

DotNetStac

\n", "

\n", ".Net library for working with Spatio Temporal Asset Catalogs (STAC)\n", "

\n", "\n", "### **DotNetStac** helps you to work with [STAC](https://stacspec.org) ([catalog](https://github.com/radiantearth/stac-spec/tree/master/catalog-spec), [collection](https://github.com/radiantearth/stac-spec/tree/master/collection-spec), [item](https://github.com/radiantearth/stac-spec/tree/master/catalog-spec))\n", "In a nutshell, the library allows import/export of STAC JSON documents (Serialization/Deserialization using [Newtonsoft.JSON](https://www.newtonsoft.com/json)) to typed object with properties represented in enhanced objects such as geometries, time stamp/period/span, numerical values and many more via STAC extension plugins engine." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Getting Started\n", "\n", "In this notebook, we are going to review the library's features through a series of code examples.\n", "\n", "#### Important Note about .Net programming and C# language usage in this library\n", "\n", "We chose to design and implement the objects of this library as plain as possible (e.g. POCO) in order to keep the structured and typed nature of the C# language. So, we will deal with normal classes, without more attributes describing infrastructure concerns or other responsibilities that your domain objects shouldn't have." ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "### 1 Install DotNetStac\n", "First, we either install lastest version of DotNetStac in the notebook or we use the locally built binaries" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "dotnet_interactive": { "language": "csharp" } }, "outputs": [], "source": [ "// Use built lib\n", "#i \"/home/emathot/Workspace/Terradue/Components/sugar/DotNetStac/src/DotNetStac/bin/Debug/netstandard2.0/publish/\"\n", "#r \"/home/emathot/Workspace/Terradue/Components/sugar/DotNetStac/src/DotNetStac/bin/Debug/netstandard2.0/publish/DotNetStac.dll\"" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "### 2 Deserialization of STAC documents\n", "\n", "The (de)serialiation methods are wrapped in methods in [class `StacConvert`](https://terradue.github.io/DotNetStac/api/Stac.StacConvert.html) that is the main entry point from/to JSON/.Net.\n", "\n", "Let's start reading a STAC catalog online. Please note that DotNetStac does not provide with data access middleware. You can integrate own data access or you can test the [`Stars` SDK](https://github.com/Terradue/Stars) that provides with integrated functions to manipulate STAC objects and their storage.\n", "\n", "The following code is a very simple function loading a catalog and printing it's `id`, `description` and `stac_version`." ] }, { "cell_type": "code", "execution_count": null, "metadata": { "dotnet_interactive": { "language": "csharp" } }, "outputs": [], "source": [ "using Stac;\n", "using Stac.Schemas;\n", "using System;\n", "using System.Net;\n", "using Newtonsoft.Json.Schema;\n", "\n", "var webc = new WebClient();\n", "Uri catalogUri = new Uri(\"https://cbers-stac-1-0-rc.s3.amazonaws.com/catalog.json\");\n", "StacValidator stacValidator = new StacValidator(new JSchemaUrlResolver());\n", "\n", "// StacConvert.Deserialize is the helper to start loading any STAC document\n", "var json = webc.DownloadString(catalogUri);\n", "bool valid = stacValidator.ValidateJson(json);\n", "StacCatalog catalog = StacConvert.Deserialize(json);\n", "\n", "Console.Out.WriteLine(catalog.Id + \": \" + catalog.Description + (valid ? \" [VALID]\" : \"[INVALID]\"));\n", "Console.Out.WriteLine(catalog.StacVersion);\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### 3 Navigation from a Catalogue\n", "\n", "Using the previously loaded catalog, the following code executes a recursive function navigating from a root catalog through all it's tree structure recursing in the `child` STAC links and listing the `item` links and their `assets`.\n", "\n", "Please note the following:\n", "\n", "- `GetChildrenLinks` and `GetItemLinks` are the recommanded ways to get the links for navigating through the tree.\n", "- The previous functions as the rest of the library does not alter the `Uri`s. It is then up to the developer to resolve the relative ones. As in the code, Uri class provides with all the necessary methods to easily join a base Url with a relative one.\n", "- The `StacConvert.Deserialize<>` methods allows to specify the interfaces to ease the deserialization when the STAC type is unknown: [`IStacObject`](https://terradue.github.io/DotNetStac/api/Stac.IStacObject.html) and [`IStacCatalog`](https://terradue.github.io/DotNetStac/api/Stac.IStacCatalog.html)." ] }, { "cell_type": "code", "execution_count": null, "metadata": { "dotnet_interactive": { "language": "csharp" } }, "outputs": [], "source": [ "using System.Linq;\n", "using Stac.Extensions.ItemCollections;\n", "\n", "public static void ListChildrensItemsAndAssets(IStacParent catalog, Uri baseUri, WebClient webc, StacValidator stacValidator, string prefix = \"\", int limit = 2)\n", "{\n", " // Get children and items (sub catalogs, collections and items)\n", " foreach (var childLink in catalog.GetChildrenLinks().Concat(catalog.GetItemLinks()).Take(limit))\n", " {\n", " // IMPORTANT: Relative Uri resolution\n", " Uri childUri = childLink.Uri;\n", " if (!childUri.IsAbsoluteUri)\n", " childUri = new Uri(baseUri, childUri.ToString());\n", " var childjson = webc.DownloadString(childUri);\n", " // STAC schema validation\n", " bool valid = stacValidator.ValidateJson(childjson);\n", " // STAC object loading (using the IStacObject interface)\n", " IStacObject child = null;\n", " try {\n", " child = StacConvert.Deserialize(childjson);\n", " } \n", " catch (Exception e) {\n", " Console.Error.WriteLine(string.Format(\"Error deserializing STAC object at '{0}' : {1}\", childLink.Uri, e.Message));\n", " Console.Error.WriteLine(e.InnerException.StackTrace);\n", " }\n", "\n", " if (child is StacCatalog || child is StacCollection)\n", " Console.Out.WriteLine(prefix + child.Id + \": \" + child.Title + (valid ? \" [VALID]\" : \" [INVALID]\"));\n", "\n", " List items = new List();\n", "\n", " // Item or ItemCollection\n", " if (child is StacItem)\n", " items.Add(child as StacItem);\n", " if (child is ItemCollection)\n", " items = (child as ItemCollection).Features;\n", " // List assets if item\n", " foreach (var item in items)\n", " {\n", " // Print the item\n", " Console.Out.WriteLine(prefix + \" \" + item.Id + \": \" + item.Title + (valid ? \" [VALID]\" : \" [INVALID]\"));\n", " foreach (var asset in item.Assets.Values) {\n", " Console.Out.WriteLine(prefix + \" *[\" + asset.MediaType + \"] \" + asset.Uri);\n", " }\n", " }\n", "\n", " // Go deeper if catalog or collection\n", " if (child is StacCatalog || child is StacCollection){\n", " ListChildrensItemsAndAssets(child as IStacParent, childUri, webc, stacValidator, prefix + \" \", limit);\n", " }\n", " \n", " }\n", "}\n", "\n", "// Start the navigation\n", "ListChildrensItemsAndAssets(catalog as IStacParent, catalogUri, webc, stacValidator);" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "### 4 Creation and serialization of STAC documents\n", "\n", "#### 4.1 Collection creation from scratch\n", "\n", "Let's now create a STAC collection programmatically using the plain Collection object. Here all collection specific fields are set manually." ] }, { "cell_type": "code", "execution_count": null, "metadata": { "dotnet_interactive": { "language": "csharp" } }, "outputs": [], "source": [ "using Newtonsoft.Json;\n", "using Newtonsoft.Json.Linq;\n", "using Stac;\n", "using Stac.Collection;\n", "using System;\n", "using System.Collections.Generic;\n", "\n", "\n", "// First the mandatory elements of a Collection\n", "// Spatial et Temporal Extent\n", "StacExtent extent = new StacExtent(\n", " new StacSpatialExtent(-180, -56, 180, 83),\n", " new StacTemporalExtent(DateTime.Parse(\"2015-06-23T00:00:00Z\").ToUniversalTime(), null)\n", " );\n", "\n", "// Create the Collection\n", "StacCollection collection = new StacCollection(\"COPERNICUS/S2\",\n", " \"Sentinel-2 is a wide-swath, high-resolution, multi-spectral\\nimaging mission supporting Copernicus Land Monitoring studies,\\nincluding the monitoring of vegetation, soil and water cover,\\nas well as observation of inland waterways and coastal areas.\\n\\nThe Sentinel-2 data contain 13 UINT16 spectral bands representing\\nTOA reflectance scaled by 10000. See the [Sentinel-2 User Handbook](https://sentinel.esa.int/documents/247904/685211/Sentinel-2_User_Handbook)\\nfor details. In addition, three QA bands are present where one\\n(QA60) is a bitmask band with cloud mask information. For more\\ndetails, [see the full explanation of how cloud masks are computed.](https://sentinel.esa.int/web/sentinel/technical-guides/sentinel-2-msi/level-1c/cloud-masks)\\n\\nEach Sentinel-2 product (zip archive) may contain multiple\\ngranules. Each granule becomes a separate Earth Engine asset.\\nEE asset ids for Sentinel-2 assets have the following format:\\nCOPERNICUS/S2/20151128T002653_20151128T102149_T56MNN. Here the\\nfirst numeric part represents the sensing date and time, the\\nsecond numeric part represents the product generation date and\\ntime, and the final 6-character string is a unique granule identifier\\nindicating its UTM grid reference (see [MGRS](https://en.wikipedia.org/wiki/Military_Grid_Reference_System)).\\n\\nFor more details on Sentinel-2 radiometric resoltuon, [see this page](https://earth.esa.int/web/sentinel/user-guides/sentinel-2-msi/resolutions/radiometric).\\n\",\n", " extent);\n", "\n", "collection.Title = \"Sentinel-2 MSI: MultiSpectral Instrument, Level-1C\";\n", "\n", "collection.Links.Add(StacLink.CreateSelfLink(new Uri(\"https://storage.cloud.google.com/earthengine-test/catalog/COPERNICUS_S2.json\")));\n", "collection.Links.Add(StacLink.CreateParentLink(new Uri(\"https://storage.cloud.google.com/earthengine-test/catalog/catalog.json\")));\n", "collection.Links.Add(StacLink.CreateRootLink(new Uri(\"https://storage.cloud.google.com/earthengine-test/catalog/catalog.json\")));\n", "collection.Links.Add(new StacLink(new Uri(\"https://scihub.copernicus.eu/twiki/pub/SciHubWebPortal/TermsConditions/Sentinel_Data_Terms_and_Conditions.pdf\"), \"license\", \"Legal notice on the use of Copernicus Sentinel Data and Service Information\", null));\n", "\n", "collection.Keywords.Add(\"copernicus\");\n", "collection.Keywords.Add(\"esa\");\n", "collection.Keywords.Add(\"eu\");\n", "collection.Keywords.Add(\"msi\");\n", "collection.Keywords.Add(\"radiance\");\n", "collection.Keywords.Add(\"sentinel\");\n", "\n", "collection.Providers.Add(new StacProvider(\"European Union/ESA/Copernicus\",\n", " new List() { StacProviderRole.producer, StacProviderRole.licensor })\n", "{\n", " Uri = new Uri(\"https://sentinel.esa.int/web/sentinel/user-guides/sentinel-2-msi\")\n", "});\n", "\n", "collection.Summaries.Add(\"datetime\",\n", " new StacSummaryRangeObject(\n", " DateTime.Parse(\"2015-06-23T00:00:00Z\").ToUniversalTime(),\n", " DateTime.Parse(\"2019-07-10T13:44:56Z\").ToUniversalTime()\n", " )\n", ");\n", "\n", "var platforms = new StacSummaryValueSet();\n", "platforms.Add(\"sentinel-2a\");\n", "platforms.Add(\"sentinel-2b\");\n", "\n", "collection.Summaries.Add(\"platform\", platforms);\n", "\n", "collection.Summaries.Add(\"constellation\",\n", " new StacSummaryValueSet(new string[] { \"sentinel-2\" })\n", ");\n", "\n", "collection.Summaries.Add(\"instruments\",\n", " new StacSummaryValueSet(new string[] { \"msi\" })\n", ");\n", "\n", "collection.Summaries.Add(\"view:off_nadir\",\n", " new StacSummaryRangeObject(\n", " 0.0,\n", " 100\n", " )\n", ");\n", "\n", "collection.Summaries.Add(\"view:sun_elevation\",\n", " new StacSummaryRangeObject(\n", " 6.78,\n", " 89.9\n", " )\n", ");\n", "\n", "collection.Summaries.Add(\"sci:citation\",\n", " new StacSummaryValueSet(new string[] { \"Copernicus Sentinel data [Year]\" })\n", ");\n", "\n", "collection.Summaries.Add(\"gsd\",\n", " new StacSummaryValueSet(new int[] {\n", " 10,\n", " 30,\n", " 60\n", " })\n", ");\n", "\n", "collection.Summaries.Add(\"proj:epsg\",\n", " new StacSummaryValueSet(new int[]\n", " { 32601,32602,32603,32604,32605,32606,32607,32608,32609,32610,32611,32612,32613,32614,32615,32616,32617,32618,32619,32620,32621,32622,32623,32624,32625,32626,32627,32628,32629,32630,32631,32632,32633,32634,32635,32636,32637,32638,32639,32640,32641,32642,32643,32644,32645,32646,32647,32648,32649,32650,32651,32652,32653,32654,32655,32656,32657,32658,32659,32660}\n", " )\n", ");\n", "\n", "collection.Summaries.Add(\"eo:bands\",\n", " new StacSummaryValueSet(new JObject[] {\n", " new JObject {\n", " { \"name\", \"B1\" },\n", " { \"common_name\", \"coastal\" },\n", " { \"center_wavelength\", 4.439 }\n", " },\n", " new JObject {\n", " { \"name\", \"B2\"},\n", " { \"common_name\", \"blue\"},\n", " { \"center_wavelength\", 4.966}\n", " },\n", " new JObject {\n", " { \"name\", \"B3\"},\n", " { \"common_name\", \"green\"},\n", " { \"center_wavelength\", 5.6}\n", " },\n", " new JObject {\n", " { \"name\", \"B4\"},\n", " { \"common_name\", \"red\"},\n", " { \"center_wavelength\", 6.645}\n", " },\n", " new JObject {\n", " { \"name\", \"B5\"},\n", " { \"center_wavelength\", 7.039}\n", " },\n", " new JObject {\n", " { \"name\", \"B6\"},\n", " { \"center_wavelength\", 7.402}\n", " },\n", " new JObject {\n", " { \"name\", \"B7\"},\n", " { \"center_wavelength\", 7.825}\n", " },\n", " new JObject {\n", " { \"name\", \"B8\"},\n", " { \"common_name\", \"nir\"},\n", " { \"center_wavelength\", 8.351}\n", " },\n", " new JObject {\n", " { \"name\", \"B8A\"},\n", " { \"center_wavelength\", 8.648}\n", " },\n", " new JObject {\n", " { \"name\", \"B9\"},\n", " { \"center_wavelength\", 9.45}\n", " },\n", " new JObject {\n", " { \"name\", \"B10\"},\n", " { \"center_wavelength\", 1.3735}\n", " },\n", " new JObject {\n", " { \"name\", \"B11\"},\n", " { \"common_name\", \"swir16\"},\n", " { \"center_wavelength\", 1.6137}\n", " },\n", " new JObject {\n", " { \"name\", \"B12\"},\n", " { \"common_name\", \"swir22\"},\n", " { \"center_wavelength\", 2.2024}\n", " }\n", " })\n", ");\n", "\n", "\n", "// Serialize\n", "JsonSerializerSettings serSettings = new JsonSerializerSettings() { Formatting = Formatting.Indented };\n", "var json = StacConvert.Serialize(collection, serSettings);\n", "// Print JSON!\n", "Console.WriteLine(json)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "#### 4.2 Collection generation from a set of Items\n", "\n", "[`StacCollection` class](https://terradue.github.io/DotNetStac/api/Stac.StacCollection.html) has static methods allowing the automatic generation of a collection from a set of `StacItem`. The following code loads the items of [the examples folder from STAC repository](https://github.com/radiantearth/stac-spec/tree/master/examples) and generates the corresponding collection with\n", "- **Spatial and temporal extent** from geometry and time merge of the items\n", "- **Fields summaries** with stats objects or value sets of the items' fields values\n", " \n", "Please note that the function takes also the eventual uri of the collection in input. If specified, the items Uri are made relative to that uri." ] }, { "cell_type": "code", "execution_count": null, "metadata": { "dotnet_interactive": { "language": "csharp" } }, "outputs": [], "source": [ "Uri simpleItemUri = new Uri(\"https://raw.githubusercontent.com/radiantearth/stac-spec/dev/examples/simple-item.json\");\n", "Uri coreItemUri = new Uri(\"https://raw.githubusercontent.com/radiantearth/stac-spec/dev/examples/core-item.json\");\n", "Uri extendedItemUri = new Uri(\"https://raw.githubusercontent.com/radiantearth/stac-spec/dev/examples/extended-item.json\");\n", "\n", "StacItem simpleItem = StacConvert.Deserialize(webc.DownloadString(simpleItemUri));\n", "StacItem coreItem = StacConvert.Deserialize(webc.DownloadString(coreItemUri));\n", "StacItem extendedItem = StacConvert.Deserialize(webc.DownloadString(extendedItemUri));\n", "\n", "Dictionary items = new Dictionary();\n", "items.Add(simpleItemUri, simpleItem);\n", "items.Add(coreItemUri,coreItem);\n", "items.Add(extendedItemUri, extendedItem);\n", "\n", "StacCollection stacCollection = StacCollection.Create(\"simple-collection\",\n", " \"A simple collection demonstrating core catalog fields with links to a couple of items\",\n", " items,\n", " \"CC-BY-4.0\", null, null);\n", "\n", "Console.Out.Write(StacConvert.Serialize(collection, serSettings));\n", "\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### 5 STAC Extensions support\n", "\n", "The library implements STAC extensions as extensions classes for enabling quick fields accessors. A extension class may also implement helpers related to the extensions such as \n", "- `projection`: WKT generation from EPSG identifier\n", "- `file`: file extension fields generation from `FileInfo` or `Stream` with Multihash checksum generation\n", "- `raster`: Optical calibration parameters" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "5.1 Projection extension\n", "\n", "In the following example, we create a simple STAC item and we use the projection extension helpers to set the coordinate system fields by:\n", "- using the [EPSG](https://epsg.org/home.html) system reference identifier\n", "- using a [Proj.Net CoordinateSystem](https://github.com/NetTopologySuite/ProjNet4GeoAPI) object" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "using GeoJSON.Net;\n", "using GeoJSON.Net.Geometry;\n", "using Itenso.TimePeriod;\n", "using ProjNet.CoordinateSystems;\n", "using Stac.Extensions.Projection;\n", "\n", "var coordinates = new[]\n", "{\n", " new List\n", " {\n", " new Position(37.488035566,-122.308150179),\n", " new Position(37.538869539,-122.597502109),\n", " new Position(37.613537207,-122.576687533),\n", " new Position(37.562818007,-122.288048600),\n", " new Position(37.488035566,-122.308150179)\n", " }\n", "};\n", "var geometry = new Polygon(new LineString[] { new LineString(coordinates[0]) });\n", "StacItem item = new StacItem(\"CS3-20160503_132130_04\", geometry);\n", "item.DateTime = new TimeInterval(DateTime.Parse(\"2016-05-03T13:21:30.040Z\"));\n", "// Set the UTM#33 north coordinate system\n", "item.ProjectionExtension().SetCoordinateSystem(32633);\n", "// or from ProjNet object\n", "item.ProjectionExtension().SetCoordinateSystem(ProjectedCoordinateSystem.WGS84_UTM(33, true));\n", "string json = StacConvert.Serialize(item, serSettings);\n", "stacValidator.ValidateJson(json);\n", "Console.Out.WriteLine(json);" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "5.1 File extension\n", "\n", "In the following example, we create a simple STAC item and we use the file extension helpers to set the `file` extension fields from a file on the local file system or from a stream." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "using Stac.Extensions.File;\n", "StacAsset stacAsset = StacAsset.CreateDataAsset(item,\n", " new Uri(\"file:///srid.csv\"),\n", " new System.Net.Mime.ContentType(\"text/csv\"),\n", " \"System reference Ids\");\n", "await stacAsset.FileExtension().SetFileExtensionProperties(new System.IO.FileInfo(\"SRID.csv\"));\n", "item.Assets.Add(\"srid\", stacAsset);\n", "string json = StacConvert.Serialize(item, serSettings);\n", "stacValidator.ValidateJson(json);\n", "Console.Out.WriteLine(json);" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "

Thank you!

\n", "\n", "\n", "www.terradue.com
\n", "Emmanuel Mathot
\n", "emmanuel.mathot@terradue.com
" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "dotnet_interactive": { "language": "csharp" } }, "outputs": [], "source": [] } ], "metadata": { "celltoolbar": "Format de la Cellule Texte Brut", "kernelspec": { "display_name": ".NET (C#)", "language": "C#", "name": ".net-csharp" }, "language_info": { "file_extension": ".cs", "mimetype": "text/x-csharp", "name": "C#", "pygments_lexer": "csharp", "version": "8.0" }, "metadata": { "interpreter": { "hash": "e7370f93d1d0cde622a1f8e1c04877d8463912d04d973331ad4851f04de6915a" } } }, "nbformat": 4, "nbformat_minor": 4 }