{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Use HTTP requests for WPS rook\n",
"\n",
"* rook wps: https://github.com/roocs/rook\n",
"* wps documentation: http://geoprocessing.info/wpsdoc/\n"
]
},
{
"cell_type": "code",
"execution_count": 1,
"metadata": {},
"outputs": [],
"source": [
"import requests\n",
"\n",
"url = 'http://rook.dkrz.de/wps'"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## GetCapabilities"
]
},
{
"cell_type": "code",
"execution_count": 2,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"'http://rook.dkrz.de/wps?service=WPS&request=GetCapabilities'"
]
},
"execution_count": 2,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"req_url = f\"{url}?service=WPS&request=GetCapabilities\"\n",
"req_url"
]
},
{
"cell_type": "code",
"execution_count": 3,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"True"
]
},
"execution_count": 3,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"resp = requests.get(req_url)\n",
"resp.ok"
]
},
{
"cell_type": "code",
"execution_count": 4,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"\n",
"\n",
"\n",
" \n",
" rook\n",
" A WPS service for roocs.\n",
" \n",
" PyWPS\n",
" WPS\n",
" OGC\n",
" processing\n",
" birdhouse\n",
" roocs\n",
" demo\n",
" cp4cds\n",
" copernicus\n",
" ecmwf\n",
" theme\n",
" \n",
" WPS\n",
" 1.0.0\n",
" 2.0.0\n",
" \n",
" \n",
" open access\n",
" \n",
" \n",
" \n",
" rook4.cloud.dkrz.de\n",
" \n",
" \n",
" DKRZ\n",
" Position Title\n",
" \n",
" \n",
" +xx-xxx-xxx-xxxx\n",
" \n",
" \n",
" \n",
" \n",
" Hamburg\n",
" \n",
" Zip or Postal Code\n",
" Germany\n",
" Email Address\n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" subset\n",
" Subset\n",
" Run subsetting on climate model data. Calls daops operators.\n",
" \n",
" \n",
" \n",
" average\n",
" Average\n",
" Run averaging on climate model data. Calls daops operators.\n",
" \n",
" \n",
" \n",
" orchestrate\n",
" Orchestrate\n",
" Run a workflow with combined operations. A workflow can be build using the rooki client.\n",
" \n",
" \n",
" \n",
" \n",
" \n",
" en-US\n",
" \n",
" \n",
" en-US\n",
" \n",
" \n",
"\n"
]
}
],
"source": [
"print(resp.text)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## DescribeProcess subset"
]
},
{
"cell_type": "code",
"execution_count": 5,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"'http://rook.dkrz.de/wps?service=WPS&version=1.0.0&request=DescribeProcess&identifier=subset'"
]
},
"execution_count": 5,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"req_url = f\"{url}?service=WPS&version=1.0.0&request=DescribeProcess&identifier=subset\"\n",
"req_url"
]
},
{
"cell_type": "code",
"execution_count": 6,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"True"
]
},
"execution_count": 6,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"resp = requests.get(req_url)\n",
"resp.ok"
]
},
{
"cell_type": "code",
"execution_count": 7,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"\n",
"\n",
"\n",
" \n",
" subset\n",
" Subset\n",
" Run subsetting on climate model data. Calls daops operators.\n",
" \n",
" \n",
" \n",
" collection\n",
" Collection\n",
" A dataset identifier or list of comma separated identifiersExample: c3s-cmip5.output1.ICHEC.EC-EARTH.historical.day.atmos.day.r1i1p1.tas.latest\n",
" \n",
" string\n",
" \n",
" \n",
" \n",
" time\n",
" Time Period\n",
" The time period to subset over separated by /Example: 1860-01-01/1900-12-30\n",
" \n",
" string\n",
" \n",
" \n",
" \n",
" area\n",
" Area\n",
" The area to subset over as 4 comma separated values.Example: 0.,49.,10.,65\n",
" \n",
" string\n",
" \n",
" \n",
" \n",
" level\n",
" Level\n",
" The level range to subset over separated by a /Example: 0/1000\n",
" \n",
" string\n",
" \n",
" \n",
" \n",
" pre_checked\n",
" Pre-Checked\n",
" Use checked data only.\n",
" \n",
" boolean\n",
" False\n",
" \n",
" \n",
" \n",
" apply_fixes\n",
" Apply Fixes\n",
" Apply fixes to datasets.\n",
" \n",
" boolean\n",
" False\n",
" \n",
" \n",
" \n",
" original_files\n",
" Original Files\n",
" Return original files only.\n",
" \n",
" boolean\n",
" False\n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
"\n"
]
}
],
"source": [
"print(resp.text)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Execute subset (sync mode)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"**Edit data inputs**"
]
},
{
"cell_type": "code",
"execution_count": 8,
"metadata": {},
"outputs": [],
"source": [
"collection = \"CMIP6.CMIP.IPSL.IPSL-CM6A-LR.historical.r1i1p1f1.Amon.rlds.gr.v20180803\"\n",
"time = \"1985-01-01/2014-12-30\"\n"
]
},
{
"cell_type": "code",
"execution_count": 9,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"'http://rook.dkrz.de/wps?service=WPS&version=1.0.0&request=Execute&identifier=subset&DataInputs=collection=CMIP6.CMIP.IPSL.IPSL-CM6A-LR.historical.r1i1p1f1.Amon.rlds.gr.v20180803;time=1985-01-01/2014-12-30'"
]
},
"execution_count": 9,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"datainputs = f\"DataInputs=collection={collection};time={time}\"\n",
"req_url = f\"{url}?service=WPS&version=1.0.0&request=Execute&identifier=subset&{datainputs}\"\n",
"req_url"
]
},
{
"cell_type": "code",
"execution_count": 10,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"True"
]
},
"execution_count": 10,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"resp = requests.get(req_url)\n",
"resp.ok"
]
},
{
"cell_type": "code",
"execution_count": 11,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"\n",
"\n",
" \n",
" subset\n",
" Subset\n",
" Run subsetting on climate model data. Calls daops operators.\n",
"\t\n",
" \n",
" PyWPS Process Subset finished\n",
"\t\n",
"\t\n",
"\t\t\n",
" output\n",
" METALINK v4 output\n",
" Metalink v4 document with references to NetCDF files.\n",
" \n",
"\t\t\n",
"\t\t\n",
" prov\n",
" Provenance\n",
" Provenance document using W3C standard.\n",
" \n",
"\t\t\n",
"\t\t\n",
" prov_plot\n",
" Provenance Diagram\n",
" Provenance document as diagram.\n",
" \n",
"\t\t\n",
"\t\n",
"\n"
]
}
],
"source": [
"print(resp.text)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"**Load metalink result document**\n",
"\n",
"Replace the metalink output URL.\n",
"```python\n",
"metalink_url = ''\n",
"```"
]
},
{
"cell_type": "code",
"execution_count": 12,
"metadata": {},
"outputs": [],
"source": [
"metalink_url = 'http://rook4.cloud.dkrz.de/outputs/rook/8c23a070-87f2-11eb-bc89-fa163e1098db/provenance.json'\n"
]
},
{
"cell_type": "code",
"execution_count": 13,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"{\"prefix\": {\"provone\": \"http://purl.dataone.org/provone/2015/01/15/ontology#\", \"dcterms\": \"http://purl.org/dc/terms/\", \"default\": \"http://purl.org/roocs/prov#\"}, \"agent\": {\"copernicus_CDS\": {\"prov:type\": \"prov:Organization\", \"dcterms:title\": \"Copernicus Climate Data Store\"}, \"rook\": {\"prov:type\": \"prov:SoftwareAgent\", \"dcterms:source\": \"https://github.com/roocs/rook/releases/tag/v0.4.0\"}, \"daops\": {\"prov:type\": \"prov:SoftwareAgent\", \"dcterms:source\": \"https://github.com/roocs/daops/releases/tag/v0.5.0\"}}, \"wasAttributedTo\": {\"_:id1\": {\"prov:entity\": \"rook\", \"prov:agent\": \"copernicus_CDS\"}}, \"activity\": {\"subset\": {\"time\": \"1985-01-01/2014-12-30\", \"apply_fixes\": false}}, \"entity\": {\"CMIP6.CMIP.IPSL.IPSL-CM6A-LR.historical.r1i1p1f1.Amon.rlds.gr.v20180803\": {}, \"rlds_Amon_IPSL-CM6A-LR_historical_r1i1p1f1_gr_19850116-20141216.nc\": {}}, \"wasStartedBy\": {\"_:id2\": {\"prov:activity\": \"subset\", \"prov:trigger\": \"rook\", \"prov:starter\": \"daops\"}}, \"wasDerivedFrom\": {\"_:id3\": {\"prov:generatedEntity\": \"rlds_Amon_IPSL-CM6A-LR_historical_r1i1p1f1_gr_19850116-20141216.nc\", \"prov:usedEntity\": \"CMIP6.CMIP.IPSL.IPSL-CM6A-LR.historical.r1i1p1f1.Amon.rlds.gr.v20180803\", \"prov:activity\": \"subset\"}}}\n"
]
}
],
"source": [
"print(requests.get(metalink_url).text)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Execute subset (async mode)"
]
},
{
"cell_type": "code",
"execution_count": 14,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"'http://rook.dkrz.de/wps?service=WPS&version=1.0.0&request=Execute&identifier=subset&DataInputs=collection=CMIP6.CMIP.IPSL.IPSL-CM6A-LR.historical.r1i1p1f1.Amon.rlds.gr.v20180803;time=1985-01-01/2014-12-30&status=true&storeExecuteResponse=true'"
]
},
"execution_count": 14,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"req_url = f\"{url}?service=WPS&version=1.0.0&request=Execute&identifier=subset&{datainputs}\"\n",
"req_url += \"&status=true&storeExecuteResponse=true\"\n",
"req_url"
]
},
{
"cell_type": "code",
"execution_count": 15,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"True"
]
},
"execution_count": 15,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"resp = requests.get(req_url)\n",
"resp.ok"
]
},
{
"cell_type": "code",
"execution_count": 16,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"\n",
"\n",
" \n",
" subset\n",
" Subset\n",
" Run subsetting on climate model data. Calls daops operators.\n",
"\t\n",
" \n",
" PyWPS Process subset accepted\n",
"\t\n",
"\n"
]
}
],
"source": [
"print(resp.text)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"**Poll status location**\n",
"\n",
"Replace the `statusLocation` URL.\n",
"```python\n",
"statusLocation = ''\n",
"```"
]
},
{
"cell_type": "code",
"execution_count": 17,
"metadata": {},
"outputs": [],
"source": [
"# statusLocation = ''\n",
"statusLocation = 'http://rook4.cloud.dkrz.de/outputs/rook/bc97d460-87f2-11eb-b8ed-fa163e1098db.xml'\n"
]
},
{
"cell_type": "code",
"execution_count": 18,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"\n",
"\n",
" \n",
" subset\n",
" Subset\n",
" Run subsetting on climate model data. Calls daops operators.\n",
"\t\n",
" \n",
" PyWPS Process Subset finished\n",
"\t\n",
"\t\n",
"\t\t\n",
" output\n",
" METALINK v4 output\n",
" Metalink v4 document with references to NetCDF files.\n",
" \n",
"\t\t\n",
"\t\t\n",
" prov\n",
" Provenance\n",
" Provenance document using W3C standard.\n",
" \n",
"\t\t\n",
"\t\t\n",
" prov_plot\n",
" Provenance Diagram\n",
" Provenance document as diagram.\n",
" \n",
"\t\t\n",
"\t\n",
"\n"
]
}
],
"source": [
"resp = requests.get(statusLocation)\n",
"print(resp.text)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"**Load metalink document**\n",
"\n",
"Replace the metalink output URL.\n",
"```python\n",
"metalink_url = ''\n",
"```"
]
},
{
"cell_type": "code",
"execution_count": 19,
"metadata": {},
"outputs": [],
"source": [
"metalink_url = 'http://rook4.cloud.dkrz.de:80/outputs/rook/bc97d460-87f2-11eb-b8ed-fa163e1098db/input.meta4'\n"
]
},
{
"cell_type": "code",
"execution_count": 20,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"\n",
"\n",
" 2021-03-18T15:03:43Z\n",
" PyWPS/4.4.0\n",
"\n",
" \n",
" NetCDF file\n",
" 20313784\n",
" http://rook4.cloud.dkrz.de:80/outputs/rook/bfe9ffee-87f2-11eb-a863-fa163e1098db/rlds_Amon_IPSL-CM6A-LR_historical_r1i1p1f1_gr_19850116-20141216.nc\n",
" \n",
" \n",
"\n",
"\n"
]
}
],
"source": [
"print(requests.get(metalink_url).text)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"**Download netCDF output**\n",
"\n",
"Replace the download URL.\n",
"\n",
"```python\n",
"download_url = ''\n",
"```"
]
},
{
"cell_type": "code",
"execution_count": 21,
"metadata": {},
"outputs": [],
"source": [
"download_url = 'http://rook4.cloud.dkrz.de:80/outputs/rook/bfe9ffee-87f2-11eb-a863-fa163e1098db/rlds_Amon_IPSL-CM6A-LR_historical_r1i1p1f1_gr_19850116-20141216.nc\n",
"\n",
"\tsubset\n",
"\t\n",
"\t\t\n",
"\t\t\tcollection\n",
"\t\t\t\n",
"\t\t\t\tc3s-cmip6.CMIP.IPSL.IPSL-CM6A-LR.historical.r1i1p1f1.Amon.rlds.gr.v20180803\n",
"\t\t\t\n",
" \n",
" \n",
"\t\t\ttime\n",
"\t\t\t\n",
"\t\t\t\t1860-01-01/1900-12-30\n",
"\t\t\t\n",
"\t\t\n",
"\t\n",
"\t\n",
"\t\t\n",
"\t\t\t\n",
"\t\t\t\toutput\n",
"\t\t\t\n",
"\t\t\n",
"\t\n",
"\n",
"\"\"\""
]
},
{
"cell_type": "code",
"execution_count": 24,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"True"
]
},
"execution_count": 24,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"resp = requests.post(url, data=xml)\n",
"resp.ok"
]
},
{
"cell_type": "code",
"execution_count": 25,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"\n",
"\n",
" \n",
" subset\n",
" Subset\n",
" Run subsetting on climate model data. Calls daops operators.\n",
"\t\n",
" \n",
" \n",
" \n",
" \n",
" Process error: Some or all of the requested collection are not in the list of available data.\n",
" \n",
" \n",
" \n",
"\t\n",
"\n"
]
}
],
"source": [
"print(resp.text)"
]
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.9.2"
}
},
"nbformat": 4,
"nbformat_minor": 4
}