{ "cells": [ { "cell_type": "markdown", "id": "52fe876a-7747-4ab9-8f62-d7a6bae1726d", "metadata": {}, "source": [ "## Access Radar Velocity hourly average delayed QC At Turquoise Coast data in Zarr\n", "A jupyter notebook to show how to access and plot the AODN radar_TurquoiseCoast_velocity_hourly_average_delayed_qc dataset available as a [Zarr](https://zarr.dev/) dataset on S3" ] }, { "cell_type": "code", "execution_count": 1, "id": "0c26364d-9a41-4cc8-bb11-eff892d3718b", "metadata": {}, "outputs": [], "source": [ "dataset_name=\"radar_TurquoiseCoast_velocity_hourly_average_delayed_qc\"" ] }, { "cell_type": "markdown", "id": "124dc987-1d2d-451e-9a66-bf314df60d06", "metadata": {}, "source": [ "## Install/Update packages and Load common functions" ] }, { "cell_type": "code", "execution_count": 2, "id": "8fd14800-1108-4b48-ab86-5940a3b0f2de", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Requirement already satisfied: uv in /home/lbesnard/miniforge3/envs/AodnCloudOptimised/lib/python3.12/site-packages (0.4.18)\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "Using CPython 3.12.6 interpreter at: \u001b[36m/home/lbesnard/miniforge3/envs/AodnCloudOptimised/bin/python\u001b[39m\n", "Creating virtual environment at: \u001b[36m.venv\u001b[39m\n", "Activate with: \u001b[32msource .venv/bin/activate\u001b[39m\n", "\u001b[2mAudited \u001b[1m230 packages\u001b[0m \u001b[2min 29ms\u001b[0m\u001b[0m\n" ] } ], "source": [ "# only run once, then restart session if needed\n", "!pip install uv\n", "\n", "import os\n", "import sys\n", "\n", "def is_colab():\n", " try:\n", " import google.colab\n", " return True\n", " except ImportError:\n", " return False\n", "\n", "# Get the current directory of the notebook\n", "current_dir = os.getcwd()\n", "\n", "# Check if requirements.txt exists in the current directory\n", "local_requirements = os.path.join(current_dir, 'requirements.txt')\n", "if os.path.exists(local_requirements):\n", " requirements_path = local_requirements\n", "else:\n", " # Fall back to the online requirements.txt file\n", " requirements_path = 'https://raw.githubusercontent.com/aodn/aodn_cloud_optimised/main/notebooks/requirements.txt'\n", "\n", "# Install packages using uv and the determined requirements file\n", "if is_colab():\n", " os.system(f'uv pip install --system -r {requirements_path}')\n", "else:\n", " os.system('uv venv')\n", " os.system(f'uv pip install -r {requirements_path}')" ] }, { "cell_type": "code", "execution_count": 3, "id": "11689024-47e1-4530-b799-09f2eee3f593", "metadata": {}, "outputs": [], "source": [ "import requests\n", "import os\n", "if not os.path.exists('parquet_queries.py'):\n", " print('Downloading parquet_queries.py')\n", " url = 'https://raw.githubusercontent.com/aodn/aodn_cloud_optimised/main/aodn_cloud_optimised/lib/ParquetDataQuery.py'\n", " response = requests.get(url)\n", " with open('parquet_queries.py', 'w') as f:\n", " f.write(response.text)" ] }, { "cell_type": "code", "execution_count": 4, "id": "6818ca6e-e3fe-43d7-8715-b2ce97370012", "metadata": {}, "outputs": [], "source": [ "from parquet_queries import plot_gridded_variable, create_timeseries, plot_time_coverage" ] }, { "cell_type": "code", "execution_count": 5, "id": "67dd6387-5e3e-4a9d-8249-5a21fe2111b5", "metadata": {}, "outputs": [], "source": [ "import xarray as xr\n", "import fsspec" ] }, { "cell_type": "code", "execution_count": 6, "id": "2a6964b2-b36a-4699-8822-f92ca90c554b", "metadata": {}, "outputs": [ { "data": { "text/html": [ "
<xarray.Dataset> Size: 807MB\n", "Dimensions: (I: 60, J: 59, TIME: 7127)\n", "Coordinates:\n", " * I (I) int32 240B 1 2 3 4 5 6 7 ... 54 55 56 57 58 59 60\n", " * J (J) int32 236B 1 2 3 4 5 6 7 ... 53 54 55 56 57 58 59\n", " LATITUDE (I, J) float64 28kB dask.array<chunksize=(59, 59), meta=np.ndarray>\n", " LONGITUDE (I, J) float64 28kB dask.array<chunksize=(59, 59), meta=np.ndarray>\n", " * TIME (TIME) datetime64[ns] 57kB 2020-01-01 ... 2020-12-3...\n", "Data variables:\n", " GDOP (I, J) float32 14kB dask.array<chunksize=(59, 59), meta=np.ndarray>\n", " NOBS1 (TIME, I, J) float32 101MB dask.array<chunksize=(1000, 59, 59), meta=np.ndarray>\n", " NOBS2 (TIME, I, J) float32 101MB dask.array<chunksize=(1000, 59, 59), meta=np.ndarray>\n", " UCUR (TIME, I, J) float32 101MB dask.array<chunksize=(1000, 59, 59), meta=np.ndarray>\n", " UCUR_quality_control (TIME, I, J) float32 101MB dask.array<chunksize=(1000, 59, 59), meta=np.ndarray>\n", " UCUR_sd (TIME, I, J) float32 101MB dask.array<chunksize=(1000, 59, 59), meta=np.ndarray>\n", " VCUR (TIME, I, J) float32 101MB dask.array<chunksize=(1000, 59, 59), meta=np.ndarray>\n", " VCUR_quality_control (TIME, I, J) float32 101MB dask.array<chunksize=(1000, 59, 59), meta=np.ndarray>\n", " VCUR_sd (TIME, I, J) float32 101MB dask.array<chunksize=(1000, 59, 59), meta=np.ndarray>\n", "Attributes: (12/40)\n", " Conventions: CF-1.6,IMOS-1.4\n", " abstract: The IMOS Ocean Radar Facility is producing...\n", " acknowledgement: Any users (including re-packagers) of IMOS...\n", " author: Cosoli, Simone; Hetzel, Yasha\n", " author_email: simone.cosoli@uwa.edu.au; yasha.hetzel@uwa...\n", " citation: The citation in a list of references is: I...\n", " ... ...\n", " source: Terrestrial HF radar\n", " ssr_Stations: Lancelin (LANC), Green Head (GHED)\n", " standard_name_vocabulary: NetCDF Climate and Forecast (CF) Metadata ...\n", " time_coverage_end: 2020-12-26T17:00:00Z\n", " time_coverage_start: 2020-12-26T17:00:00Z\n", " title: IMOS Ocean Radar Facility Turqoise Coast (...