{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Augmented _Gradients_ Transect Data\n",
"\n",
"Here we (1) retrieve [SCOPE-Gradients](http://scope.soest.hawaii.edu/data/gradients/data/) cruise data from the [Simons' CMAP](https://cmap.readthedocs.io/en/latest/) data base and (2) collocate with mean dymanic topography / SSH.\n",
"\n",
"![\"Drawing\"](\"cbiomes-01.png\")
"
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "skip"
}
},
"source": [
"### Set up tools\n",
"\n",
"[PyCmap](https://github.com/simonscmap/pycmap) is the Python API that we will use in Julia, via [PyCall.jl](https://github.com/JuliaPy/PyCall.jl), to query the CMAP data base. [Plots.jl](http://docs.juliaplots.org/latest/) is a common `Julia` plotting package and `helper_functions.jl` adds a few convenience functions."
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "skip"
}
},
"source": [
"_Pre-requisites:_\n",
"\n",
"- 1. _install the [PyCmap](https://github.com/simonscmap/pycmap) python package and its dependencies using `pip`_\n",
"- 2. _compile [PyCall.jl](https://github.com/simonscmap/pycmap) using external python distribution that installed `PyCmap`_\n",
"- 3. _obtain your own API key from [Simons' CMAP](https://simonscmap.com) (free; takes `<30s`)_\n",
"- 4. _import `pycmap` (via `pycall`), `Plots`, and `helper functions`_\n",
"\n",
"_You may need to replace `your-own-API-key` (as outline below) with your own API key from [Simons' CMAP](https://simonscmap.com) and uncomment the command below._"
]
},
{
"cell_type": "code",
"execution_count": 1,
"metadata": {
"cell_style": "split",
"slideshow": {
"slide_type": "skip"
}
},
"outputs": [],
"source": [
"if false\n",
" run(`pip install pycmap`) #pycmap is used via PyCall later\n",
" run(pipeline(`which python`,\"whichpython.txt\")) #external python path\n",
" ENV[\"PYTHON\"]=readline(\"whichpython.txt\")\n",
" import Pkg; Pkg.build(\"PyCall\")\n",
"end"
]
},
{
"cell_type": "code",
"execution_count": 2,
"metadata": {
"cell_style": "split",
"slideshow": {
"slide_type": "skip"
}
},
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
"┌ Info: Precompiling MeshArrays [cb8c808f-1acf-59a3-9d2b-6e38d009f683]\n",
"└ @ Base loading.jl:1273\n",
"┌ Info: Precompiling MITgcmTools [62725fbc-3a66-4df3-9000-e33e85b3a198]\n",
"└ @ Base loading.jl:1273\n"
]
},
{
"data": {
"text/plain": [
"Main.cbiomes_helpers"
]
},
"execution_count": 2,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"using PyCall\n",
"PyCmap = pyimport(\"pycmap\")\n",
"#cmap = PyCmap.API(token=\"your-own-API-key\")\n",
"cmap = PyCmap.API()\n",
"\n",
"using Plots\n",
"include(\"helper_functions.jl\")"
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "skip"
}
},
"source": [
"### Get Data Catalog\n",
"\n",
"_Downloading the catalog is a simple way to verify that cmap is all set-up. The commented `df.to_csv` command writes the content of `df` to a new `catalog.csv` file. Alternatively, `Pandas.jl` can be used as also shown._"
]
},
{
"cell_type": "code",
"execution_count": 3,
"metadata": {
"slideshow": {
"slide_type": "skip"
}
},
"outputs": [],
"source": [
"df = cmap.get_catalog();\n",
"#df.to_csv(\"catalog.csv\")\n",
"\n",
"#df=Pandas.DataFrame(cmap.get_catalog());\n",
"#to_csv(df,\"catalog.csv\")"
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "skip"
}
},
"source": [
"### Download Data (`.csv` Files)\n",
"\n",
"A simple method is to download data from `CMAP` and store it to a `CSV` file which any software can then reload. Below, `cmap_helpers.tables` provide CMAP `table` lists for [SCOPE-Gradients](http://scope.soest.hawaii.edu/data/gradients/data/) cruise data, which `cmap.get_dataset` downloads one at a time. _Alternatively one can use the computer's memory (later slides)._"
]
},
{
"cell_type": "code",
"execution_count": 4,
"metadata": {
"slideshow": {
"slide_type": "skip"
}
},
"outputs": [],
"source": [
"pth=\"../samples/gradients/\"\n",
"!isdir(\"$pth\") ? mkdir(\"$pth\") : nothing\n",
"\n",
"Γ=1:3\n",
"γ=filter(x -> occursin(\"tblKM1906\",x), readdir(pth))\n",
"!isempty(γ) ? Γ = [] : nothing\n",
"\n",
"for g in Γ\n",
" list0=cmap_helpers.tables(\"G$g\")\n",
" for i in list0\n",
" df=cmap.get_dataset(i)\n",
" df.to_csv(\"$pth$i.csv\")\n",
" end\n",
"end"
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "skip"
}
},
"source": [
"### Read Data (& Meta-Data)\n",
"\n",
"As an example below we read, and then plot, the `LISST` data collected during the `Gradients 3` cruise."
]
},
{
"cell_type": "code",
"execution_count": 5,
"metadata": {
"slideshow": {
"slide_type": "skip"
}
},
"outputs": [
{
"data": {
"text/plain": [
"Dict{String,Any} with 8 entries:\n",
" \"Long_Name\" => \"LISST C 20-100.0 micron\"\n",
" \"Unit\" => \"umol C/L\"\n",
" \"lat\" => [21.2464, 21.2464, 21.2464, 21.2464, 21.2465, 21.2465, 21.24…\n",
" \"time\" => [\"2019-04-10T00:39:37\", \"2019-04-10T00:42:07\", \"2019-04-10T0…\n",
" \"Variable\" => \"LISST_large\"\n",
" \"val\" => [NaN, NaN, NaN, NaN, NaN, NaN, NaN, NaN, NaN, NaN … NaN, N…\n",
" \"lon\" => [-158.034, -158.037, -158.041, -158.045, -158.048, -158.052,…\n",
" \"Data_Source\" => \"Prof. Dr. Angelicque A. White, University of Hawaii, aewhit…"
]
},
"execution_count": 5,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"s=cmap_helpers.get(\"tblKM1906_Gradients3_uway_optics\",\"LISST_small\")\n",
"m=cmap_helpers.get(\"tblKM1906_Gradients3_uway_optics\",\"LISST_medium\")\n",
"l=cmap_helpers.get(\"tblKM1906_Gradients3_uway_optics\",\"LISST_large\")"
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "skip"
}
},
"source": [
"### Collocate Ancillary Data\n",
"\n",
"Here we interpolate a `sea surface height` climatology estimate ([Forget et al 2015](http://doi.org/10.5194/gmd-8-3071-2015)) along the ship track. _Any number of commonly available methods can readily be used to interpolate gridded estimates to observed locations. For `MITgcm` output in our example, we use `MeshArrays.jl` method._"
]
},
{
"cell_type": "code",
"execution_count": 6,
"metadata": {
"slideshow": {
"slide_type": "skip"
}
},
"outputs": [
{
"data": {
"text/plain": [
"Dict{String,Any} with 7 entries:\n",
" \"lat\" => [21.2464, 21.2464, 21.2464, 21.2464, 21.2465, 21.2465, 21.24…\n",
" \"Long_Name\" => \"Sea Surface Height (Mean Dynamic Topography)\"\n",
" \"Unit\" => \"m\"\n",
" \"Variable\" => \"SSH\"\n",
" \"val\" => [0.701766, 0.701738, 0.70171, 0.70168, 0.70165, 0.701618, 0.…\n",
" \"lon\" => [-158.034, -158.037, -158.041, -158.045, -158.048, -158.052,…\n",
" \"Data_Source\" => \"ECCOv4r2 (Gael Forget)\""
]
},
"execution_count": 6,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"ssh=cbiomes_helpers.myinterp(pth,\"SSH\",s[\"lon\"],s[\"lat\"])\n",
"ssh=merge(ssh,Dict(\"Unit\" => \"m\", \"Variable\" => \"SSH\", \"Long_Name\" => \"Sea Surface Height (Mean Dynamic Topography)\"))"
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "skip"
}
},
"source": [
"### Plot v Latitude\n",
"\n",
" LISST data collected in Gradients 3 and the sea surface height climatology."
]
},
{
"cell_type": "code",
"execution_count": 7,
"metadata": {
"slideshow": {
"slide_type": "skip"
}
},
"outputs": [
{
"data": {
"image/svg+xml": [
"\n",
"