{ "cells": [ { "cell_type": "markdown", "id": "14239690-7838-46fd-9b7a-2cc42ca6835b", "metadata": {}, "source": [ "# Regridding to DGGS\n", "\n", "## Context\n", "\n", "### Purpose\n", "\n", "The goal is to regrid a sample of RiOMar data into DGGS (Healpix).\n", "\n", "### Description\n", "\n", "In this notebook, we will:\n", "- Open a RiOMar data file\n", "- Prepare the DGGS grid for regridding\n", "- Transform the RiOMar dataset into DGSS (Healpix) grid.\n", "- Save the transformed data in Zarr\n", "\n", "## Contributions\n", "\n", "### Notebook\n", "\n", "\n", "- Tina Odaka (author), IFREMER (France), @tinaok\n", "- Anne Fouilloux (reviewer), Simula Research Laboratory (Norway), @annefou\n", "\n", "## Bibliography and other interesting resources\n", "\n", "- [RiOMar](https://coast.ifremer.fr/Laboratoires-Environnement-Ressources/LER-Pertuis-Charentais-La-Tremblade/Projets/RIOMAR-2024-2030)\n" ] }, { "cell_type": "markdown", "id": "f43352db-5ee5-4d14-a168-3f50c88e0e40", "metadata": {}, "source": [ "## Install packages \n", "- Install `xarray-healpy` if you do not have it in your environment." ] }, { "cell_type": "code", "execution_count": 4, "id": "b771abea-b30f-4a47-a5e4-2dab36590393", "metadata": { "collapsed": true, "jupyter": { "outputs_hidden": true } }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Collecting git+https://github.com/IAOCEA/xarray-healpy.git\n", " Cloning https://github.com/IAOCEA/xarray-healpy.git to /tmp/pip-req-build-xeh8o3ta\n", " Running command git clone --filter=blob:none --quiet https://github.com/IAOCEA/xarray-healpy.git /tmp/pip-req-build-xeh8o3ta\n", " Resolved https://github.com/IAOCEA/xarray-healpy.git to commit 9e457091957aa0d7833280558e60e1addc500ec7\n", " Installing build dependencies ... \u001b[?25ldone\n", "\u001b[?25h Getting requirements to build wheel ... \u001b[?25ldone\n", "\u001b[?25h Preparing metadata (pyproject.toml) ... \u001b[?25ldone\n", "\u001b[?25hRequirement already satisfied: xarray>=2023.11.0 in /srv/conda/envs/notebook/lib/python3.12/site-packages (from xarray-healpy==0.1.dev26+g9e45709) (2024.7.0)\n", "Collecting healpy (from xarray-healpy==0.1.dev26+g9e45709)\n", " Using cached healpy-1.18.0-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (8.7 kB)\n", "Requirement already satisfied: sparse in /srv/conda/envs/notebook/lib/python3.12/site-packages (from xarray-healpy==0.1.dev26+g9e45709) (0.15.4)\n", "Collecting opt_einsum (from xarray-healpy==0.1.dev26+g9e45709)\n", " Using cached opt_einsum-3.4.0-py3-none-any.whl.metadata (6.3 kB)\n", "Requirement already satisfied: numba in /srv/conda/envs/notebook/lib/python3.12/site-packages (from xarray-healpy==0.1.dev26+g9e45709) (0.60.0)\n", "Requirement already satisfied: numpy in /srv/conda/envs/notebook/lib/python3.12/site-packages (from xarray-healpy==0.1.dev26+g9e45709) (1.26.4)\n", "Requirement already satisfied: scikit-learn in /srv/conda/envs/notebook/lib/python3.12/site-packages (from xarray-healpy==0.1.dev26+g9e45709) (1.5.1)\n", "Collecting cdshealpix (from xarray-healpy==0.1.dev26+g9e45709)\n", " Using cached cdshealpix-0.7.1-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (4.3 kB)\n", "Requirement already satisfied: astropy in /srv/conda/envs/notebook/lib/python3.12/site-packages (from xarray-healpy==0.1.dev26+g9e45709) (6.1.2)\n", "Requirement already satisfied: packaging>=23.1 in /srv/conda/envs/notebook/lib/python3.12/site-packages (from xarray>=2023.11.0->xarray-healpy==0.1.dev26+g9e45709) (24.1)\n", "Requirement already satisfied: pandas>=2.0 in /srv/conda/envs/notebook/lib/python3.12/site-packages (from xarray>=2023.11.0->xarray-healpy==0.1.dev26+g9e45709) (2.2.2)\n", "Requirement already satisfied: pyerfa>=2.0.1.1 in /srv/conda/envs/notebook/lib/python3.12/site-packages (from astropy->xarray-healpy==0.1.dev26+g9e45709) (2.0.1.4)\n", "Requirement already satisfied: astropy-iers-data>=0.2024.7.1.0.34.3 in /srv/conda/envs/notebook/lib/python3.12/site-packages (from astropy->xarray-healpy==0.1.dev26+g9e45709) (0.2024.8.5.0.32.23)\n", "Requirement already satisfied: PyYAML>=3.13 in /srv/conda/envs/notebook/lib/python3.12/site-packages (from astropy->xarray-healpy==0.1.dev26+g9e45709) (6.0.2)\n", "Requirement already satisfied: llvmlite<0.44,>=0.43.0dev0 in /srv/conda/envs/notebook/lib/python3.12/site-packages (from numba->xarray-healpy==0.1.dev26+g9e45709) (0.43.0)\n", "Requirement already satisfied: scipy>=1.6.0 in /srv/conda/envs/notebook/lib/python3.12/site-packages (from scikit-learn->xarray-healpy==0.1.dev26+g9e45709) (1.14.0)\n", "Requirement already satisfied: joblib>=1.2.0 in /srv/conda/envs/notebook/lib/python3.12/site-packages (from scikit-learn->xarray-healpy==0.1.dev26+g9e45709) (1.4.2)\n", "Requirement already satisfied: threadpoolctl>=3.1.0 in /srv/conda/envs/notebook/lib/python3.12/site-packages (from scikit-learn->xarray-healpy==0.1.dev26+g9e45709) (3.5.0)\n", "Requirement already satisfied: python-dateutil>=2.8.2 in /srv/conda/envs/notebook/lib/python3.12/site-packages (from pandas>=2.0->xarray>=2023.11.0->xarray-healpy==0.1.dev26+g9e45709) (2.8.2)\n", "Requirement already satisfied: pytz>=2020.1 in /srv/conda/envs/notebook/lib/python3.12/site-packages (from pandas>=2.0->xarray>=2023.11.0->xarray-healpy==0.1.dev26+g9e45709) (2024.1)\n", "Requirement already satisfied: tzdata>=2022.7 in /srv/conda/envs/notebook/lib/python3.12/site-packages (from pandas>=2.0->xarray>=2023.11.0->xarray-healpy==0.1.dev26+g9e45709) (2024.1)\n", "Requirement already satisfied: six>=1.5 in /srv/conda/envs/notebook/lib/python3.12/site-packages (from python-dateutil>=2.8.2->pandas>=2.0->xarray>=2023.11.0->xarray-healpy==0.1.dev26+g9e45709) (1.16.0)\n", "Using cached cdshealpix-0.7.1-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.3 MB)\n", "Using cached healpy-1.18.0-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (8.9 MB)\n", "Using cached opt_einsum-3.4.0-py3-none-any.whl (71 kB)\n", "Building wheels for collected packages: xarray-healpy\n", " Building wheel for xarray-healpy (pyproject.toml) ... \u001b[?25ldone\n", "\u001b[?25h Created wheel for xarray-healpy: filename=xarray_healpy-0.1.dev26+g9e45709-py3-none-any.whl size=12889 sha256=c2911e3d940b2f01e383675364c11446cdc24965bccc7a5554b2424ba445133b\n", " Stored in directory: /tmp/pip-ephem-wheel-cache-yc3jlh9e/wheels/13/12/8e/cc158525fce8db1608cf79d96cd2ec7abe59daf089b7e4f8d6\n", "Successfully built xarray-healpy\n", "Installing collected packages: opt_einsum, healpy, cdshealpix, xarray-healpy\n", "Successfully installed cdshealpix-0.7.1 healpy-1.18.0 opt_einsum-3.4.0 xarray-healpy-0.1.dev26+g9e45709\n", "Note: you may need to restart the kernel to use updated packages.\n" ] } ], "source": [ "pip install git+https://github.com/IAOCEA/xarray-healpy.git" ] }, { "cell_type": "markdown", "id": "ed10fb6d-e5a9-4347-9a98-8d70e072bbd2", "metadata": {}, "source": [ "## Import Libraries" ] }, { "cell_type": "code", "execution_count": 5, "id": "59338848-16fc-42dd-9638-9a2e304ce262", "metadata": {}, "outputs": [], "source": [ "import xarray as xr\n", "import fsspec\n", "from xarray_healpy import HealpyGridInfo, HealpyRegridder\n", "import numpy as np" ] }, { "cell_type": "markdown", "id": "a18500c4-d279-4beb-80d0-976c5d8824a7", "metadata": {}, "source": [ "## Open RiOMar sample dataset\n", "- This RiOMar data has been generated from the original RiOMar data (netCDF) by selected a few timesteps and saving the results into Zarr to facilitate the access from the cloud (S3 Bucket)." ] }, { "cell_type": "code", "execution_count": 6, "id": "2a2517a9-056c-4675-8d43-28463062046a", "metadata": {}, "outputs": [], "source": [ "url = \"https://data-fair2adapt.ifremer.fr/riomar/small.zarr\"" ] }, { "cell_type": "code", "execution_count": 14, "id": "207e2b12-c703-4e39-9551-626af89d7654", "metadata": {}, "outputs": [ { "data": { "text/html": [ "
<xarray.Dataset> Size: 498MB\n",
"Dimensions: (y_rho: 838, x_rho: 727, s_rho: 40, time_counter: 5)\n",
"Coordinates:\n",
" nav_lat_rho (y_rho, x_rho) float64 5MB dask.array<chunksize=(838, 727), meta=np.ndarray>\n",
" nav_lon_rho (y_rho, x_rho) float64 5MB dask.array<chunksize=(838, 727), meta=np.ndarray>\n",
" * s_rho (s_rho) float32 160B -0.9875 -0.9625 ... -0.0375 -0.0125\n",
" * time_counter (time_counter) datetime64[ns] 40B 2004-01-01T00:58:30 ... 2...\n",
" time_instant (time_counter) datetime64[ns] 40B dask.array<chunksize=(1,), meta=np.ndarray>\n",
"Dimensions without coordinates: y_rho, x_rho\n",
"Data variables:\n",
" ocean_mask (y_rho, x_rho) bool 609kB dask.array<chunksize=(838, 727), meta=np.ndarray>\n",
" temp (time_counter, s_rho, y_rho, x_rho) float32 487MB dask.array<chunksize=(1, 40, 838, 727), meta=np.ndarray>\n",
"Attributes: (12/45)\n",
" CPP-options: REGIONAL GAMAR MPI TIDES OBC_WEST OBC_NORTH XIOS USE_CALE...\n",
" Conventions: CF-1.6\n",
" Cs_r: have a look at variable Cs_r in this file\n",
" Cs_w: have a look at variable Cs_w in this file\n",
" SRCS: main.F step.F read_inp.F timers_roms.F init_scalars.F ini...\n",
" Tcline: 15.0\n",
" ... ...\n",
" title: GAMAR_GLORYS\n",
" tnu4_expl: biharmonic mixing coefficient for tracers\n",
" units: meter4 second-1\n",
" uuid: 06f6b784-fcc0-4422-aceb-17da2a5aa9fa\n",
" v_sponge: 0.0\n",
" x_sponge: 0.0<xarray.Dataset> Size: 457kB\n",
"Dimensions: (cells: 19047)\n",
"Coordinates:\n",
" cell_ids (cells) uint64 152kB 56417655 56417659 ... 56565776 56565792\n",
" latitude (cells) float64 152kB 47.03 47.03 47.03 ... 48.98 48.98 48.98\n",
" longitude (cells) float64 152kB -6.282 -6.331 -6.306 ... -6.44 -6.542\n",
" resolution float64 8B 0.0002498\n",
"Dimensions without coordinates: cells\n",
"Data variables:\n",
" *empty*\n",
"Attributes:\n",
" grid_type: healpix\n",
" level: 12\n",
" nside: 4096\n",
" lat: 0\n",
" lon: 0\n",
" rot_lat: 0\n",
" rot_lon: 0<xarray.Dataset> Size: 31MB\n",
"Dimensions: (cells: 19047, time_counter: 5, s_rho: 40)\n",
"Coordinates:\n",
" cell_ids (cells) uint64 152kB 56417655 56417659 ... 56565776 56565792\n",
" latitude (cells) float64 152kB 47.03 47.03 47.03 ... 48.98 48.98 48.98\n",
" longitude (cells) float64 152kB -6.282 -6.331 -6.306 ... -6.44 -6.542\n",
" resolution float64 8B 0.0002498\n",
" * s_rho (s_rho) float32 160B -0.9875 -0.9625 ... -0.0375 -0.0125\n",
" * time_counter (time_counter) datetime64[ns] 40B 2004-01-01T00:58:30 ... 2...\n",
" time_instant (time_counter) datetime64[ns] 40B dask.array<chunksize=(1,), meta=np.ndarray>\n",
"Dimensions without coordinates: cells\n",
"Data variables:\n",
" ocean_mask (cells) float64 152kB dask.array<chunksize=(19047,), meta=np.ndarray>\n",
" temp (time_counter, s_rho, cells) float64 30MB dask.array<chunksize=(1, 40, 19047), meta=np.ndarray>\n",
"Attributes:\n",
" grid_type: healpix\n",
" level: 12\n",
" nside: 4096\n",
" lat: 0\n",
" lon: 0\n",
" rot_lat: 0\n",
" rot_lon: 0<xarray.Dataset> Size: 57MB\n",
"Dimensions: (y: 130, x: 267, time_counter: 5, s_rho: 40)\n",
"Coordinates:\n",
" resolution float64 8B 0.0002498\n",
" * s_rho (s_rho) float32 160B -0.9875 -0.9625 ... -0.0375 -0.0125\n",
" * time_counter (time_counter) datetime64[ns] 40B 2004-01-01T00:58:30 ... 2...\n",
" time_instant (time_counter) datetime64[ns] 40B dask.array<chunksize=(1,), meta=np.ndarray>\n",
" cell_ids (y, x) int64 278kB 56389452 56389454 ... 56569807 56569829\n",
" latitude (y, x) float64 278kB 45.55 45.56 45.57 ... 50.36 50.37 50.38\n",
" longitude (y, x) float64 278kB -7.719 -7.721 -7.723 ... -5.2 -5.201\n",
"Dimensions without coordinates: y, x\n",
"Data variables:\n",
" ocean_mask (y, x) float64 278kB dask.array<chunksize=(130, 267), meta=np.ndarray>\n",
" temp (time_counter, s_rho, y, x) float64 56MB dask.array<chunksize=(1, 40, 130, 267), meta=np.ndarray>\n",
"Attributes:\n",
" grid_type: healpix\n",
" level: 12\n",
" nside: 4096\n",
" lat: 0\n",
" lon: 0\n",
" rot_lat: 0\n",
" rot_lon: 0<xarray.Dataset> Size: 57MB\n",
"Dimensions: (y: 130, x: 267, s_rho: 40, time_counter: 5)\n",
"Coordinates:\n",
" cell_ids (y, x) int64 278kB dask.array<chunksize=(65, 267), meta=np.ndarray>\n",
" latitude (y, x) float64 278kB dask.array<chunksize=(65, 267), meta=np.ndarray>\n",
" longitude (y, x) float64 278kB dask.array<chunksize=(65, 267), meta=np.ndarray>\n",
" resolution float64 8B ...\n",
" * s_rho (s_rho) float32 160B -0.9875 -0.9625 ... -0.0375 -0.0125\n",
" * time_counter (time_counter) datetime64[ns] 40B 2004-01-01T00:58:30 ... 2...\n",
" time_instant (time_counter) datetime64[ns] 40B dask.array<chunksize=(1,), meta=np.ndarray>\n",
"Dimensions without coordinates: y, x\n",
"Data variables:\n",
" ocean_mask (y, x) float64 278kB dask.array<chunksize=(130, 267), meta=np.ndarray>\n",
" temp (time_counter, s_rho, y, x) float64 56MB dask.array<chunksize=(1, 40, 130, 267), meta=np.ndarray>\n",
"Attributes:\n",
" grid_type: healpix\n",
" lat: 0\n",
" level: 12\n",
" lon: 0\n",
" nside: 4096\n",
" rot_lat: 0\n",
" rot_lon: 0<xarray.Dataset> Size: 56MB\n",
"Dimensions: (cell: 34710, s_rho: 40, time_counter: 5)\n",
"Coordinates:\n",
" cell_ids (cell) int64 278kB 56389452 56389453 ... 56569828 56569829\n",
" resolution float64 8B 0.0002498\n",
" * s_rho (s_rho) float32 160B -0.9875 -0.9625 ... -0.0375 -0.0125\n",
" * time_counter (time_counter) datetime64[ns] 40B 2004-01-01T00:58:30 ... 2...\n",
" time_instant (time_counter) datetime64[ns] 40B 2004-01-01T00:58:30 ... 2...\n",
"Dimensions without coordinates: cell\n",
"Data variables:\n",
" ocean_mask (cell) float64 278kB nan nan nan nan nan ... nan nan nan nan\n",
" temp (time_counter, s_rho, cell) float64 56MB nan nan ... nan nan\n",
"Attributes:\n",
" grid_type: healpix\n",
" lat: 0\n",
" level: 12\n",
" lon: 0\n",
" nside: 4096\n",
" rot_lat: 0\n",
" rot_lon: 0<xarray.Dataset> Size: 56MB\n",
"Dimensions: (cell: 34710, s_rho: 40, time_counter: 5)\n",
"Coordinates:\n",
" * cell_ids (cell) int64 278kB 56389452 56389453 ... 56569828 56569829\n",
" resolution float64 8B 0.0002498\n",
" * s_rho (s_rho) float32 160B -0.9875 -0.9625 ... -0.0375 -0.0125\n",
" * time_counter (time_counter) datetime64[ns] 40B 2004-01-01T00:58:30 ... 2...\n",
" time_instant (time_counter) datetime64[ns] 40B 2004-01-01T00:58:30 ... 2...\n",
"Dimensions without coordinates: cell\n",
"Data variables:\n",
" ocean_mask (cell) float64 278kB nan nan nan nan nan ... nan nan nan nan\n",
" temp (time_counter, s_rho, cell) float64 56MB nan nan ... nan nan\n",
"Indexes:\n",
" cell_ids HealpixIndex(nside=12, indexing_scheme=nested)\n",
"Attributes:\n",
" grid_type: healpix\n",
" lat: 0\n",
" level: 12\n",
" lon: 0\n",
" nside: 4096\n",
" rot_lat: 0\n",
" rot_lon: 0