{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "# Use HTTP requests for WPS rook\n", "\n", "* rook wps: https://github.com/roocs/rook\n", "* wps documentation: http://geoprocessing.info/wpsdoc/\n" ] }, { "cell_type": "code", "execution_count": 1, "metadata": {}, "outputs": [], "source": [ "import requests\n", "\n", "url = 'http://rook.dkrz.de/wps'" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## GetCapabilities" ] }, { "cell_type": "code", "execution_count": 2, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "'http://rook.dkrz.de/wps?service=WPS&request=GetCapabilities'" ] }, "execution_count": 2, "metadata": {}, "output_type": "execute_result" } ], "source": [ "req_url = f\"{url}?service=WPS&request=GetCapabilities\"\n", "req_url" ] }, { "cell_type": "code", "execution_count": 3, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "True" ] }, "execution_count": 3, "metadata": {}, "output_type": "execute_result" } ], "source": [ "resp = requests.get(req_url)\n", "resp.ok" ] }, { "cell_type": "code", "execution_count": 4, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "\n", "\n", "\n", " \n", " rook\n", " A WPS service for roocs.\n", " \n", " PyWPS\n", " WPS\n", " OGC\n", " processing\n", " birdhouse\n", " roocs\n", " demo\n", " cp4cds\n", " copernicus\n", " ecmwf\n", " theme\n", " \n", " WPS\n", " 1.0.0\n", " 2.0.0\n", " \n", " \n", " open access\n", " \n", " \n", " \n", " rook4.cloud.dkrz.de\n", " \n", " \n", " DKRZ\n", " Position Title\n", " \n", " \n", " +xx-xxx-xxx-xxxx\n", " \n", " \n", " \n", " \n", " Hamburg\n", " \n", " Zip or Postal Code\n", " Germany\n", " Email Address\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " subset\n", " Subset\n", " Run subsetting on climate model data. Calls daops operators.\n", " \n", " \n", " \n", " average\n", " Average\n", " Run averaging on climate model data. Calls daops operators.\n", " \n", " \n", " \n", " orchestrate\n", " Orchestrate\n", " Run a workflow with combined operations. A workflow can be build using the rooki client.\n", " \n", " \n", " \n", " \n", " \n", " en-US\n", " \n", " \n", " en-US\n", " \n", " \n", "\n" ] } ], "source": [ "print(resp.text)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## DescribeProcess subset" ] }, { "cell_type": "code", "execution_count": 5, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "'http://rook.dkrz.de/wps?service=WPS&version=1.0.0&request=DescribeProcess&identifier=subset'" ] }, "execution_count": 5, "metadata": {}, "output_type": "execute_result" } ], "source": [ "req_url = f\"{url}?service=WPS&version=1.0.0&request=DescribeProcess&identifier=subset\"\n", "req_url" ] }, { "cell_type": "code", "execution_count": 6, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "True" ] }, "execution_count": 6, "metadata": {}, "output_type": "execute_result" } ], "source": [ "resp = requests.get(req_url)\n", "resp.ok" ] }, { "cell_type": "code", "execution_count": 7, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "\n", "\n", "\n", " \n", " subset\n", " Subset\n", " Run subsetting on climate model data. Calls daops operators.\n", " \n", " \n", " \n", " collection\n", " Collection\n", " A dataset identifier or list of comma separated identifiersExample: c3s-cmip5.output1.ICHEC.EC-EARTH.historical.day.atmos.day.r1i1p1.tas.latest\n", " \n", " string\n", " \n", " \n", " \n", " time\n", " Time Period\n", " The time period to subset over separated by /Example: 1860-01-01/1900-12-30\n", " \n", " string\n", " \n", " \n", " \n", " area\n", " Area\n", " The area to subset over as 4 comma separated values.Example: 0.,49.,10.,65\n", " \n", " string\n", " \n", " \n", " \n", " level\n", " Level\n", " The level range to subset over separated by a /Example: 0/1000\n", " \n", " string\n", " \n", " \n", " \n", " pre_checked\n", " Pre-Checked\n", " Use checked data only.\n", " \n", " boolean\n", " False\n", " \n", " \n", " \n", " apply_fixes\n", " Apply Fixes\n", " Apply fixes to datasets.\n", " \n", " boolean\n", " False\n", " \n", " \n", " \n", " original_files\n", " Original Files\n", " Return original files only.\n", " \n", " boolean\n", " False\n", " \n", " \n", " \n", " \n", " \n", " output\n", " METALINK v4 output\n", " Metalink v4 document with references to NetCDF files.\n", " \n", " \n", " \n", " application/metalink+xml; version=4.0\n", " metalink/4.0/metalink4.xsd\n", " \n", " \n", " \n", " \n", " application/metalink+xml; version=4.0\n", " metalink/4.0/metalink4.xsd\n", " \n", " \n", " \n", " \n", " \n", " prov\n", " Provenance\n", " Provenance document using W3C standard.\n", " \n", " \n", " \n", " application/json\n", " \n", " \n", " \n", " \n", " application/json\n", " \n", " \n", " \n", " \n", " \n", " prov_plot\n", " Provenance Diagram\n", " Provenance document as diagram.\n", " \n", " \n", " \n", " image/png\n", " base64\n", " \n", " \n", " \n", " \n", " image/png\n", " base64\n", " \n", " \n", " \n", " \n", " \n", " \n", "\n" ] } ], "source": [ "print(resp.text)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Execute subset (sync mode)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "**Edit data inputs**" ] }, { "cell_type": "code", "execution_count": 8, "metadata": {}, "outputs": [], "source": [ "collection = \"CMIP6.CMIP.IPSL.IPSL-CM6A-LR.historical.r1i1p1f1.Amon.rlds.gr.v20180803\"\n", "time = \"1985-01-01/2014-12-30\"\n" ] }, { "cell_type": "code", "execution_count": 9, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "'http://rook.dkrz.de/wps?service=WPS&version=1.0.0&request=Execute&identifier=subset&DataInputs=collection=CMIP6.CMIP.IPSL.IPSL-CM6A-LR.historical.r1i1p1f1.Amon.rlds.gr.v20180803;time=1985-01-01/2014-12-30'" ] }, "execution_count": 9, "metadata": {}, "output_type": "execute_result" } ], "source": [ "datainputs = f\"DataInputs=collection={collection};time={time}\"\n", "req_url = f\"{url}?service=WPS&version=1.0.0&request=Execute&identifier=subset&{datainputs}\"\n", "req_url" ] }, { "cell_type": "code", "execution_count": 10, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "True" ] }, "execution_count": 10, "metadata": {}, "output_type": "execute_result" } ], "source": [ "resp = requests.get(req_url)\n", "resp.ok" ] }, { "cell_type": "code", "execution_count": 11, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "\n", "\n", " \n", " subset\n", " Subset\n", " Run subsetting on climate model data. Calls daops operators.\n", "\t\n", " \n", " PyWPS Process Subset finished\n", "\t\n", "\t\n", "\t\t\n", " output\n", " METALINK v4 output\n", " Metalink v4 document with references to NetCDF files.\n", " \n", "\t\t\n", "\t\t\n", " prov\n", " Provenance\n", " Provenance document using W3C standard.\n", " \n", "\t\t\n", "\t\t\n", " prov_plot\n", " Provenance Diagram\n", " Provenance document as diagram.\n", " \n", "\t\t\n", "\t\n", "\n" ] } ], "source": [ "print(resp.text)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "**Load metalink result document**\n", "\n", "Replace the metalink output URL.\n", "```python\n", "metalink_url = ''\n", "```" ] }, { "cell_type": "code", "execution_count": 12, "metadata": {}, "outputs": [], "source": [ "metalink_url = 'http://rook4.cloud.dkrz.de/outputs/rook/8c23a070-87f2-11eb-bc89-fa163e1098db/provenance.json'\n" ] }, { "cell_type": "code", "execution_count": 13, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "{\"prefix\": {\"provone\": \"http://purl.dataone.org/provone/2015/01/15/ontology#\", \"dcterms\": \"http://purl.org/dc/terms/\", \"default\": \"http://purl.org/roocs/prov#\"}, \"agent\": {\"copernicus_CDS\": {\"prov:type\": \"prov:Organization\", \"dcterms:title\": \"Copernicus Climate Data Store\"}, \"rook\": {\"prov:type\": \"prov:SoftwareAgent\", \"dcterms:source\": \"https://github.com/roocs/rook/releases/tag/v0.4.0\"}, \"daops\": {\"prov:type\": \"prov:SoftwareAgent\", \"dcterms:source\": \"https://github.com/roocs/daops/releases/tag/v0.5.0\"}}, \"wasAttributedTo\": {\"_:id1\": {\"prov:entity\": \"rook\", \"prov:agent\": \"copernicus_CDS\"}}, \"activity\": {\"subset\": {\"time\": \"1985-01-01/2014-12-30\", \"apply_fixes\": false}}, \"entity\": {\"CMIP6.CMIP.IPSL.IPSL-CM6A-LR.historical.r1i1p1f1.Amon.rlds.gr.v20180803\": {}, \"rlds_Amon_IPSL-CM6A-LR_historical_r1i1p1f1_gr_19850116-20141216.nc\": {}}, \"wasStartedBy\": {\"_:id2\": {\"prov:activity\": \"subset\", \"prov:trigger\": \"rook\", \"prov:starter\": \"daops\"}}, \"wasDerivedFrom\": {\"_:id3\": {\"prov:generatedEntity\": \"rlds_Amon_IPSL-CM6A-LR_historical_r1i1p1f1_gr_19850116-20141216.nc\", \"prov:usedEntity\": \"CMIP6.CMIP.IPSL.IPSL-CM6A-LR.historical.r1i1p1f1.Amon.rlds.gr.v20180803\", \"prov:activity\": \"subset\"}}}\n" ] } ], "source": [ "print(requests.get(metalink_url).text)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Execute subset (async mode)" ] }, { "cell_type": "code", "execution_count": 14, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "'http://rook.dkrz.de/wps?service=WPS&version=1.0.0&request=Execute&identifier=subset&DataInputs=collection=CMIP6.CMIP.IPSL.IPSL-CM6A-LR.historical.r1i1p1f1.Amon.rlds.gr.v20180803;time=1985-01-01/2014-12-30&status=true&storeExecuteResponse=true'" ] }, "execution_count": 14, "metadata": {}, "output_type": "execute_result" } ], "source": [ "req_url = f\"{url}?service=WPS&version=1.0.0&request=Execute&identifier=subset&{datainputs}\"\n", "req_url += \"&status=true&storeExecuteResponse=true\"\n", "req_url" ] }, { "cell_type": "code", "execution_count": 15, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "True" ] }, "execution_count": 15, "metadata": {}, "output_type": "execute_result" } ], "source": [ "resp = requests.get(req_url)\n", "resp.ok" ] }, { "cell_type": "code", "execution_count": 16, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "\n", "\n", " \n", " subset\n", " Subset\n", " Run subsetting on climate model data. Calls daops operators.\n", "\t\n", " \n", " PyWPS Process subset accepted\n", "\t\n", "\n" ] } ], "source": [ "print(resp.text)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "**Poll status location**\n", "\n", "Replace the `statusLocation` URL.\n", "```python\n", "statusLocation = ''\n", "```" ] }, { "cell_type": "code", "execution_count": 17, "metadata": {}, "outputs": [], "source": [ "# statusLocation = ''\n", "statusLocation = 'http://rook4.cloud.dkrz.de/outputs/rook/bc97d460-87f2-11eb-b8ed-fa163e1098db.xml'\n" ] }, { "cell_type": "code", "execution_count": 18, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "\n", "\n", " \n", " subset\n", " Subset\n", " Run subsetting on climate model data. Calls daops operators.\n", "\t\n", " \n", " PyWPS Process Subset finished\n", "\t\n", "\t\n", "\t\t\n", " output\n", " METALINK v4 output\n", " Metalink v4 document with references to NetCDF files.\n", " \n", "\t\t\n", "\t\t\n", " prov\n", " Provenance\n", " Provenance document using W3C standard.\n", " \n", "\t\t\n", "\t\t\n", " prov_plot\n", " Provenance Diagram\n", " Provenance document as diagram.\n", " \n", "\t\t\n", "\t\n", "\n" ] } ], "source": [ "resp = requests.get(statusLocation)\n", "print(resp.text)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "**Load metalink document**\n", "\n", "Replace the metalink output URL.\n", "```python\n", "metalink_url = ''\n", "```" ] }, { "cell_type": "code", "execution_count": 19, "metadata": {}, "outputs": [], "source": [ "metalink_url = 'http://rook4.cloud.dkrz.de:80/outputs/rook/bc97d460-87f2-11eb-b8ed-fa163e1098db/input.meta4'\n" ] }, { "cell_type": "code", "execution_count": 20, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "\n", "\n", " 2021-03-18T15:03:43Z\n", " PyWPS/4.4.0\n", "\n", " \n", " NetCDF file\n", " 20313784\n", " http://rook4.cloud.dkrz.de:80/outputs/rook/bfe9ffee-87f2-11eb-a863-fa163e1098db/rlds_Amon_IPSL-CM6A-LR_historical_r1i1p1f1_gr_19850116-20141216.nc\n", " \n", " \n", "\n", "\n" ] } ], "source": [ "print(requests.get(metalink_url).text)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "**Download netCDF output**\n", "\n", "Replace the download URL.\n", "\n", "```python\n", "download_url = ''\n", "```" ] }, { "cell_type": "code", "execution_count": 21, "metadata": {}, "outputs": [], "source": [ "download_url = 'http://rook4.cloud.dkrz.de:80/outputs/rook/bfe9ffee-87f2-11eb-a863-fa163e1098db/rlds_Amon_IPSL-CM6A-LR_historical_r1i1p1f1_gr_19850116-20141216.nc\n", "\n", "\tsubset\n", "\t\n", "\t\t\n", "\t\t\tcollection\n", "\t\t\t\n", "\t\t\t\tc3s-cmip6.CMIP.IPSL.IPSL-CM6A-LR.historical.r1i1p1f1.Amon.rlds.gr.v20180803\n", "\t\t\t\n", " \n", " \n", "\t\t\ttime\n", "\t\t\t\n", "\t\t\t\t1860-01-01/1900-12-30\n", "\t\t\t\n", "\t\t\n", "\t\n", "\t\n", "\t\t\n", "\t\t\t\n", "\t\t\t\toutput\n", "\t\t\t\n", "\t\t\n", "\t\n", "\n", "\"\"\"" ] }, { "cell_type": "code", "execution_count": 24, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "True" ] }, "execution_count": 24, "metadata": {}, "output_type": "execute_result" } ], "source": [ "resp = requests.post(url, data=xml)\n", "resp.ok" ] }, { "cell_type": "code", "execution_count": 25, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "\n", "\n", " \n", " subset\n", " Subset\n", " Run subsetting on climate model data. Calls daops operators.\n", "\t\n", " \n", " \n", " \n", " \n", " Process error: Some or all of the requested collection are not in the list of available data.\n", " \n", " \n", " \n", "\t\n", "\n" ] } ], "source": [ "print(resp.text)" ] } ], "metadata": { "kernelspec": { "display_name": "Python 3", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.9.2" } }, "nbformat": 4, "nbformat_minor": 4 }